| opendevreview | melanie witt proposed openstack/nova master: Call volume detach rollback API if detach fails https://review.opendev.org/c/openstack/nova/+/880399 | 00:02 |
|---|---|---|
| opendevreview | Merged openstack/nova stable/zed: Fix rescue volume-based instance https://review.opendev.org/c/openstack/nova/+/872116 | 09:34 |
| frickler | bauzas: I think this meeting is long obsolete and could be dropped from the calendar? https://meetings.opendev.org/#Placement_Team_Office_Hour | 11:41 |
| bauzas | frickler : sure | 11:41 |
| bauzas | Want me to janitor it? | 11:41 |
| frickler | that'd be nice, then I can approve it | 11:43 |
| frickler | likely https://meetings.opendev.org/#Nova_Monthly_Asia_Friendly_Team_Meeting too | 11:43 |
| bauzas | cool, lemme do this this afternoon | 11:51 |
| bauzas | frickler: https://meetings.opendev.org/meetings/placement/2019/ | 12:22 |
| bauzas | shit, wrong paste | 12:23 |
| bauzas | frickler: https://review.opendev.org/c/opendev/irc-meetings/+/880485 | 12:23 |
| *** tosky_ is now known as tosky | 12:53 | |
| bauzas | gibi: sean-k-mooney: I have this long open doc change, can you round it again ? https://review.opendev.org/c/openstack/nova/+/875730 | 14:13 |
| gibi | bauzas: +2 | 14:19 |
| bauzas | thanks for the swing | 14:21 |
| dansmith | bauzas: did you see my ping yesterday about that resize claim thing? | 14:23 |
| dansmith | the more I think about it, the more I think I should just bite the bullet and add a raise there as part of that series | 14:24 |
| dansmith | and call it a "tightening of the host->node relationship" if nothing else | 14:24 |
| bauzas | dansmith: yeah I could be okay with an exception | 14:25 |
| dansmith | alright | 14:26 |
| bauzas | that's what we discussed on Wed | 14:26 |
| bauzas | I was a bit afraid we could say 'hehe, sure' if we weren't providing now an exception, and without having a migration record | 14:26 |
| dansmith | I know, and I said I didn't want to | 14:26 |
| bauzas | ++ | 14:26 |
| dansmith | again, I don't think the exception isn't required to keep it consistent, | 14:27 |
| dansmith | it will just make any breakage obvious | 14:27 |
| sean-k-mooney | bauzas: gibi i proably wont have the weigher updated today before ye finsih but ill try and get it done latere today so it will be avaiable for ye on monday | 15:45 |
| gibi | sean-k-mooney: ack, no worriews | 15:45 |
| bauzas | sorry folks, I'm done for today | 15:45 |
| sean-k-mooney | bauzas: do you still need more eye on that ping | 15:55 |
| sean-k-mooney | ill review it in a bit just finsihsing something locally | 15:55 |
| bauzas | sean-k-mooney: sorry wdym? | 15:55 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/875730/4/doc/source/contributor/ptl-guide.rst | 15:55 |
| bauzas | Ack ok no worries | 15:56 |
| bauzas | and no rush ;-) | 15:56 |
| sean-k-mooney | assuming all is well ill hit that before i finish today | 15:56 |
| noonedeadpunk | folks, do you have any idea, why metadata on config-drive won't contain fixed ip address, but will have `"type": "ipv4_dhcp"`? | 16:30 |
| noonedeadpunk | probably worth postponing this question till Monday... | 16:34 |
| sean-k-mooney | noonedeadpunk: we changed that in antelope | 16:37 |
| sean-k-mooney | the logic being if hte network has dhcp enabeld it should not be set so that cloud init wont hardcode it | 16:37 |
| sean-k-mooney | actully | 16:38 |
| sean-k-mooney | maybe im mixing that up with something else | 16:38 |
| sean-k-mooney | let me see if i can fidn the patch im thinking of | 16:38 |
| sean-k-mooney | i dont recall if it was only the mtu that we stopped passing | 16:38 |
| noonedeadpunk | Well, we've started seeing that right in antelope:) | 16:38 |
| noonedeadpunk | I just had a chat with octavia folks, they still see ip for net with enabled dhcp... | 16:39 |
| noonedeadpunk | But we can have slightly different versions | 16:39 |
| sean-k-mooney | https://github.com/openstack/nova/commit/6bdc79af30151f683c0f462bc6c69da30ebcbcf9 | 16:40 |
| sean-k-mooney | ok so that was just metadta | 16:40 |
| sean-k-mooney | * mtu | 16:40 |
| sean-k-mooney | it should not affect ips | 16:41 |
| noonedeadpunk | well. test__get_link_mtu has none for IP ? | 16:41 |
| sean-k-mooney | https://github.com/openstack/nova/commit/6bdc79af30151f683c0f462bc6c69da30ebcbcf9#diff-f04bc33149c06fd545308c204ae23cb7c83abd01443bdf8f85acce861b0547f4R266 | 16:42 |
| sean-k-mooney | yes but this is all that functioanlly changed | 16:42 |
| sean-k-mooney | noonedeadpunk: this was not chagned by that but is proably more of interest to youhttps://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L279-L330 | 16:43 |
| sean-k-mooney | wait you said fixed_ips | 16:44 |
| noonedeadpunk | specifically https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L295 | 16:44 |
| noonedeadpunk | johnsom: ^ | 16:45 |
| noonedeadpunk | so, the thing is that johnsom have on their sandbox this in metadata https://www.irccloud.com/pastebin/fEVIIOCM/ | 16:45 |
| sean-k-mooney | the only fixed_ip info is the ec2 compat stuff https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L227-L240 | 16:45 |
| noonedeadpunk | and I have https://paste.openstack.org/show/bNGH0dceR991yU0OthqO/ | 16:45 |
| noonedeadpunk | nah, forget about `fixed_ip`... | 16:45 |
| noonedeadpunk | I was trying to express expectations rather then reffering any data | 16:46 |
| sean-k-mooney | ok | 16:46 |
| noonedeadpunk | At we both have dhcp enabled | 16:46 |
| sean-k-mooney | ya so the irccloud linke has ip_address in the data | 16:47 |
| sean-k-mooney | which is what i expect in the networks section | 16:47 |
| sean-k-mooney | that has "type": "ipv4" | 16:48 |
| sean-k-mooney | vs "type": "ipv4_dhcp" | 16:48 |
| noonedeadpunk | but it's based on `subnet.get_meta('dhcp_server')`? | 16:49 |
| sean-k-mooney | its been a while but if i recall correctly when its type ipv4_dhcp that is not ment to have teh ip info where as tyep ipv4 is for static netwrokgin without dhcpu | 16:49 |
| noonedeadpunk | specifically this block I assume https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L292-L297 | 16:49 |
| sean-k-mooney | if the subnet does not have dhcpu enabled we store the static ip info so cloud init can configure it | 16:49 |
| sean-k-mooney | but if it has dhcpu enabel we expect cloud init to use dhcpu | 16:49 |
| noonedeadpunk | ok, yes, I see. I wonder how we managed to get this working for years then... | 16:50 |
| noonedeadpunk | that is totally different question though | 16:50 |
| noonedeadpunk | thanks sean-k-mooney! | 16:50 |
| sean-k-mooney | noonedeadpunk: perhaps you modifed the template | 16:50 |
| noonedeadpunk | have a good weekend | 16:50 |
| johnsom | That is odd given I have DHCP on the subnet, but the metadata still has the fixed IP | 16:51 |
| sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.injected_network_template | 16:51 |
| * noonedeadpunk needs to run now but wil read all back | 16:51 | |
| noonedeadpunk | I for sure have not changed that | 16:51 |
| sean-k-mooney | thats good becasue that is not tested anywhwere | 16:52 |
| noonedeadpunk | I was debugging OSA CI job that start failing month ago | 16:52 |
| sean-k-mooney | and has not been for years | 16:52 |
| sean-k-mooney | so it proably does not work | 16:52 |
| noonedeadpunk | Or well. I t was passing last time month ago | 16:52 |
| noonedeadpunk | and in the meanwhile what we changed was nova/octavia/neutron versions we're isntalling and ansible collection that creates networks | 16:52 |
| sean-k-mooney | so thats this https://github.com/openstack/nova/blob/master/nova/virt/interfaces.template but thats not actuly for the data your looking at | 16:53 |
| noonedeadpunk | but it's interesting why me and johnsomhave different result given we both have ddhcp enabled | 16:53 |
| sean-k-mooney | johnsom: noonedeadpunk what ml2 drivers are you using | 16:53 |
| sean-k-mooney | ml2/ovs ml2/ovn | 16:54 |
| sean-k-mooney | this code should not care but just wondering | 16:54 |
| noonedeadpunk | ovn | 16:55 |
| johnsom | ovn | 16:55 |
| noonedeadpunk | btw, upgrade job that is passing is using lxb | 16:55 |
| noonedeadpunk | (upgrade from Y to 2023.1) | 16:55 |
| sean-k-mooney | ok so it started breaking in ovn but only somethimes | 16:56 |
| noonedeadpunk | I have a sandbox vm where it's reproducible | 16:56 |
| johnsom | We haven't seen any changes in the Octavia jobs, they are all still passing as expected. | 16:57 |
| sean-k-mooney | johnsom: noonedeadpunk are ye seeign this in yoru downstream installs or is this also seen in upstram devstack tempest jobs | 16:57 |
| * noonedeadpunk don't utilize devstack | 16:57 | |
| sean-k-mooney | noonedeadpunk: your using OSA right | 16:57 |
| noonedeadpunk | but it's in OSA jobs | 16:57 |
| noonedeadpunk | yup | 16:57 |
| noonedeadpunk | No idea about downstream yet | 16:57 |
| sean-k-mooney | ya im wondering if there is some neturon/nova config setting at play that changes this behvior | 16:58 |
| sean-k-mooney | or an enabled neutron api extention | 16:58 |
| noonedeadpunk | well, it could be... | 16:58 |
| sean-k-mooney | most of that code has not changed in 4-6 years | 16:58 |
| noonedeadpunk | we changed neither of that and yeah - code is quite old indeed | 16:58 |
| sean-k-mooney | but that does not mean other code around it had not changed the data its reciving | 16:58 |
| noonedeadpunk | I wonder if that could boil down to smth like sdk version :D | 16:59 |
| sean-k-mooney | not in this case since we are not using the sdk altough maybe neutron clinet | 16:59 |
| noonedeadpunk | that changes output of `subnet.get_meta('dhcp_server')` | 16:59 |
| noonedeadpunk | but that should be neutron client object? | 16:59 |
| noonedeadpunk | sorry really need to run as otherwise my wife will feed my cold corps to pigs | 17:00 |
| sean-k-mooney | not nessisarly | 17:00 |
| sean-k-mooney | cool | 17:00 |
| sean-k-mooney | ping us on monday | 17:00 |
| noonedeadpunk | sure, sorry for that :) | 17:01 |
| noonedeadpunk | have a good weekends again | 17:01 |
| sean-k-mooney | no its an iteresting issue | 17:01 |
| sean-k-mooney | i was jut checkign if it was perhaps oru Subnet model object we use form the network info case but no | 17:02 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/network/model.py#L299-L339 | 17:02 |
| sean-k-mooney | that does not have .get_meta | 17:02 |
| sean-k-mooney | https://github.com/openstack/nova/commit/c7f572d65b57e034c1391ed49d84f6e5f1d672ad seams related | 17:06 |
| sean-k-mooney | johnsom: noonedeadpunk when ye are aroudn on monday check if there is a dhcp port on the network in question | 17:07 |
| sean-k-mooney | this might be caused by useing ovn with or without the dhcp agent | 17:07 |
| sean-k-mooney | if deployed without it we might not have the info | 17:08 |
| sean-k-mooney | btu with it we would? | 17:08 |
| * johnsom is around all day if you want me to look at things | 17:08 | |
| sean-k-mooney | that is my best guess right now | 17:08 |
| dansmith | bauzas: you would not believe how many unit tests depend on this broken behavior (incorrectly) | 17:08 |
| sean-k-mooney | johnsom: looking at https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L292-L297 | 17:09 |
| sean-k-mooney | we dont have the same workaroudn for that edgecase | 17:09 |
| johnsom | I don't have the dhcp agent running, mine is pure ovn | 17:10 |
| sean-k-mooney | and you have the info | 17:10 |
| johnsom | correct | 17:11 |
| sean-k-mooney | ok lets see if the its the same for noonedeadpunk on monday | 17:11 |
| sean-k-mooney | the way we are doing the check might still be realted but im not sure | 17:11 |
| sean-k-mooney | johnsom: this si workinf form info in the network infocache by the way | 17:15 |
| sean-k-mooney | https://github.com/openstack/nova/blob/ae42400b7663bc58d5562de99e976c95131b77a9/nova/virt/netutils.py#L168-L224 | 17:16 |
| sean-k-mooney | ovn is racy with how its sends network_vif_plugged events | 17:17 |
| sean-k-mooney | so its posible there is a race here | 17:17 |
| dansmith | ugh, the whole point of network_vif_plugged was to close a race :/ | 17:19 |
| sean-k-mooney | ya i know but ovn did not have a way orgian to knwo if the flwo were programed | 17:20 |
| sean-k-mooney | so it said it was pluged when it was bound | 17:20 |
| dansmith | yeah, that's just unfortunate | 17:20 |
| sean-k-mooney | now it has a websocket callback meachnium | 17:20 |
| sean-k-mooney | but it still does not wait for the ovn metadta agaent to compltes | 17:21 |
| sean-k-mooney | which makes metadata and i think dncp racy on boot which as you know is why network vif plugged was added | 17:22 |
| sean-k-mooney | we dont have the dhcp race if using ovn native dhcp | 17:22 |
| sean-k-mooney | but if you use the dhcp agent for ironic support i think it has a race but not sure | 17:22 |
| noonedeadpunk | I'd say my attempts fail quite reliably many times in a row without passing any single time | 19:09 |
| noonedeadpunk | so doesn't look like race in my case at least | 19:10 |
| *** EugenMayer41 is now known as EugenMayer4 | 19:17 | |
| noonedeadpunk | thankfully it's super easy to fix now. But this doesn't really solve the mystery of why we're having different results with same input | 19:26 |
| *** EugenMayer48 is now known as EugenMayer4 | 19:32 | |
| opendevreview | melanie witt proposed openstack/nova master: Call volume detach rollback API if detach fails https://review.opendev.org/c/openstack/nova/+/880399 | 20:38 |
| opendevreview | Merged openstack/nova master: Update to the PTL guide https://review.opendev.org/c/openstack/nova/+/875730 | 21:30 |
| opendevreview | Robert Breker proposed openstack/nova master: Fix nova-manage image_property show unexpected keyword https://review.opendev.org/c/openstack/nova/+/880557 | 22:36 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!