| opendevreview | Merged openstack/neutron master: [eventlet-deprecation] Remove ``common.utils.spawn`` https://review.opendev.org/c/openstack/neutron/+/939090 | 01:05 |
|---|---|---|
| opendevreview | Elod Illes proposed openstack/os-vif unmaintained/victoria: [CI] Remove devstack-gate requirement https://review.opendev.org/c/openstack/os-vif/+/939160 | 08:39 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [eventlet-deprecation] Reimplement ``common.utils.spawn_n`` https://review.opendev.org/c/openstack/neutron/+/939095 | 09:42 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [eventlet-deprecation] Remove ``subprocess_popen`` https://review.opendev.org/c/openstack/neutron/+/939097 | 09:42 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [eventlet-deprecation] Remove ``subprocess_popen`` https://review.opendev.org/c/openstack/neutron/+/939097 | 09:43 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [eventlet-deprecation] Remove eventlet from ``TestNovaNotify`` https://review.opendev.org/c/openstack/neutron/+/939210 | 09:52 |
| bpetermann | ralonsoh: A question regarding https://review.opendev.org/c/openstack/neutron/+/938905 - should I change it to add the existing PeriodicWorker to should_post_fork_initialize? I need some kind of periodic worker with access to OVN IDL in vpnaas | 10:11 |
| ralonsoh | bpetermann, first, when using the neutron-periodic-workers.service (where the periodic workers are executed), there is no AFTER_INIT event and "post_fork_initialize" is never executed. So your implementation is actually not initializing the OVN IDLs | 10:36 |
| ralonsoh | second, the periodic service will be executed in all controllers (if you have HA). That means this method will be executed several times. Is that needed? | 10:37 |
| ralonsoh | third, why don't you use the maintenance worker? (now the question is how to add new tasks externally... because it's not prepared for that) | 10:37 |
| ralonsoh | ^^ the OVN maintenance, I mean | 10:38 |
| bpetermann | vpnaas for ovn somehow needs to find out when some vpn agent died, so the check is run via a PeriodWorker. In our setup we patched should_post_fork_initialize to include the PeriodicWorker and that made it work, as we don't use the neutron-periodic-workers.service. Running on all controllers is redundant, yes, and maybe not needed. And with the ovn maintenance worker there's the problem to add external tasks | 10:43 |
| vsaienko_ | ralonsoh: please review https://review.opendev.org/c/openstack/neutron/+/938657 I've addressed your comments, thanks in advance! | 10:46 |
| ralonsoh | bpetermann, I see https://review.opendev.org/c/openstack/neutron/+/938905/1/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py#312 and I'm using it too | 10:52 |
| ralonsoh | what kind of Neutron API are you using? eventlet? | 10:52 |
| ralonsoh | are you running all workers in the Neutron API? | 10:52 |
| ralonsoh | with wsgi that implementation won't work | 10:52 |
| ralonsoh | vsaienko_, let me check | 10:53 |
| bpetermann | ralonsoh, the ovn-vpnaas plugin is based on neutron_lib.services.ServicePluginBase and uses add_worker to run the PeriodicWorker. add_worker is called from __init__ of the plugin, i.e. when it's loaded in the neutron-server API | 11:07 |
| ralonsoh | bpetermann, so you are using the eventlet Neutron API. This is deprecated already so I recommend you to use the WSGI one | 11:08 |
| ralonsoh | in that case, you need to spawn a different process for the periodic workers | 11:08 |
| bpetermann | I see. So I have to find a different way to get the vpn agent list in a periodic check. | 11:11 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: Make API policies for tags to be working with resource attributes https://review.opendev.org/c/openstack/neutron/+/938135 | 11:12 |
| ralonsoh | bpetermann, the best way to do this is to use the OVN maintenance worker. Everything is ready for OVN: the IDL, the sync between processes (HA concurrency lock) | 11:19 |
| ralonsoh | so let's find a way to make this class configurable (or extendible) | 11:20 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [eventlet-deprecation] Replace ``eventlet.spawn_n`` usage https://review.opendev.org/c/openstack/neutron/+/938541 | 11:21 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [eventlet-deprecation] Replace ``eventlet.spawn_n`` usage https://review.opendev.org/c/openstack/neutron/+/938541 | 11:21 |
| opendevreview | Slawek Kaplonski proposed x/whitebox-neutron-tempest-plugin master: Wait a bit longer for tcpdump to store captures to the file https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/939229 | 11:22 |
| slaweq | ralonsoh ykarel hi, please check ^^ when You will have few minutes. It is trivial patch :) | 11:23 |
| ralonsoh | slaweq, sure | 11:27 |
| ralonsoh | slaweq, I wish we had a better method to do this... but I don't know how, to be honest | 11:28 |
| ralonsoh | slaweq, please check https://review.opendev.org/c/openstack/ovsdbapp/+/939122 | 11:29 |
| opendevreview | Merged openstack/os-vif unmaintained/victoria: [CI] Remove devstack-gate requirement https://review.opendev.org/c/openstack/os-vif/+/939160 | 11:46 |
| slaweq | ralonsoh I thik we could implement something smarter like actively checking file for some time (wait_until_true) but I'm not sure if it is worth such kind of effort here really | 11:52 |
| ralonsoh | right, you also need to ssh into the VM in this active wait | 11:52 |
| slaweq | yeap | 11:55 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: WIP == Retrieve the port with write context https://review.opendev.org/c/openstack/neutron/+/939232 | 12:29 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: DNM - Test "neutron-ovn-tempest-ipv6-only-ovs*" with WSGI https://review.opendev.org/c/openstack/neutron/+/932601 | 12:30 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: DNM - Test "neutron-ovn-tempest-ipv6-only-ovs*" with WSGI https://review.opendev.org/c/openstack/neutron/+/932601 | 12:33 |
| opendevreview | Merged openstack/ovsdbapp master: Revert "Add Northbound Logical_Router_Port name index" https://review.opendev.org/c/openstack/ovsdbapp/+/939122 | 13:08 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron stable/2024.2: ovn: Disable activation-strategy=rarp for DPDK ports https://review.opendev.org/c/openstack/neutron/+/939235 | 13:58 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron stable/2024.1: ovn: Disable activation-strategy=rarp for DPDK ports https://review.opendev.org/c/openstack/neutron/+/939236 | 13:58 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron stable/2023.2: ovn: Disable activation-strategy=rarp for DPDK ports https://review.opendev.org/c/openstack/neutron/+/939237 | 13:58 |
| haleyb | #startmeeting networking | 14:00 |
| opendevmeet | Meeting started Tue Jan 14 14:00:32 2025 UTC and is due to finish in 60 minutes. The chair is haleyb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'networking' | 14:00 |
| ihrachys | o/ | 14:00 |
| haleyb | Ping list: bcafarel, elvira, frickler, mlavalle, mtomaska, obondarev, slaweq, tobias-urdin, ykarel, lajoskatona, jlibosva, averdagu, amotoki, haleyb, ralonsoh | 14:00 |
| mlavalle | \o | 14:00 |
| s3rj1k | hi all | 14:00 |
| slaweq | o/ | 14:00 |
| rubasov | o/ | 14:01 |
| obondarev | o/ | 14:01 |
| bcafarel | o/ | 14:01 |
| haleyb | #announcements | 14:01 |
| haleyb | hi everyone! | 14:01 |
| lajoskatona | o/ | 14:02 |
| haleyb | first, thanks for everyone that helped run meetings while i was out, i think i now have the energy to get through the cycle :) | 14:02 |
| * mlavalle will have to leave at 30 minutes after the hour | 14:03 | |
| haleyb | ack | 14:03 |
| haleyb | We are currently in week R-11 | 14:03 |
| haleyb | #link https://releases.openstack.org/epoxy/schedule.html | 14:03 |
| ralonsoh | hello | 14:03 |
| haleyb | E-2 milestone was last week | 14:03 |
| haleyb | i don't remember seeing a neutron release though i could have missed it | 14:04 |
| haleyb | i did see neutron-lib | 14:04 |
| slaweq | I didn't propose it for sure | 14:05 |
| haleyb | slaweq: did i miss it? maybe you approved it | 14:05 |
| haleyb | i thought there was some automation there | 14:05 |
| slaweq | And I didn't saw it neither | 14:05 |
| haleyb | i'll look into it | 14:06 |
| frickler | eventlet is now bumped to the latest version https://review.opendev.org/c/openstack/requirements/+/933257 | 14:06 |
| haleyb | 'F' release naming (hopefully everyone voted) | 14:06 |
| haleyb | 'Flamingo' is the code name for 2025.2 | 14:06 |
| haleyb | Reminder: If you have a topic for the drivers meeting on Friday, please add it to the wiki @ https://wiki.openstack.org/wiki/Meetings/NeutronDrivers | 14:07 |
| mlavalle | nice | 14:07 |
| haleyb | And of course a reminder to use the priorities dashboard for patches in the "ready to merge" state | 14:08 |
| haleyb | I had sent a draft email to the other cores on Linuxbridge driver removal, thanks for the responses, i just haven't read them yet | 14:09 |
| haleyb | i will send that email to the ML after this meeting hopefully | 14:10 |
| haleyb | thanks to ihrachys for pushing the patch | 14:10 |
| ihrachys | fixed the last fullstack issue there yesterday, should be green now (minus random issues in gate) https://review.opendev.org/c/openstack/neutron/+/927216 | 14:10 |
| haleyb | slaweq did have one question on the timeline - were we going to do it in the E cycle | 14:11 |
| lajoskatona | thank you both for pushing this topic | 14:11 |
| slaweq | I asked about it mostly because of the distros who are packaging Neutron as they will need to remove neutron-linuxbridge-agent binary | 14:12 |
| ihrachys | afaiu the general rule is one cycle deprecated, then remove; in this case, it was not "deprecated" but it was "hidden behind a experimental flag with a huge warning that it sucks". whether these are the same is a subject for debate among friends. | 14:12 |
| haleyb | ihrachys: i think you looked into the question ^^ and we should be doing it in a slurp ? but i don't remember | 14:13 |
| slaweq | ihrachys that is correct, I am just not sure if we are not "too late" in the cycle as we already passed M-2 milestone | 14:13 |
| haleyb | slaweq: i'm not sure we even package it any more, need to double-check | 14:13 |
| ihrachys | if experimental >= deprecated in terms of abandonness / admin awareness of its badness, then we can do it any time I think | 14:13 |
| slaweq | I think that RDO for example packages it | 14:14 |
| ihrachys | are there rules that would not allow it in M2 vs M1? | 14:14 |
| slaweq | I don't think there are rules about that regarding parts of the project | 14:14 |
| slaweq | like in this case | 14:14 |
| slaweq | haleyb RDO is for sure packaging it still: https://review.rdoproject.org/r/plugins/gitiles/openstack/neutron-distgit/+/refs/heads/rpm-master | 14:15 |
| ihrachys | I can send a patch removing it in RDO, shouldn't be a problem, 10mins work | 14:16 |
| ihrachys | my take is the only relevant determination as to whether we CAN do it is whether we equate deprecation to experimental. | 14:16 |
| slaweq | regarding SLURP/non-SLURP release, I think we should be good to remove it in SLURP now as we already had it deprecated/experimental in the previous SLURP | 14:16 |
| ihrachys | apart from it, it's a question whether we PREFER doing it now or later | 14:16 |
| haleyb | ubuntu is packaging it as well, since it's automated and hasn't broken | 14:16 |
| haleyb | ihrachys: right, is deprecation == experimental is the real question | 14:17 |
| ihrachys | my answer to both is 1) we can since experimental >= deprecated; and 2) I prefer now because it's better to do it earlier and remove the burden from the code base (plus some personal reasons!) | 14:18 |
| lajoskatona | +1 for doing it in SLURP | 14:18 |
| haleyb | and i agree i think having it experimental means it can go away at any time | 14:18 |
| haleyb | maybe i need to clarify in my email our plan for this cycle | 14:19 |
| ihrachys | ok I guess some believe we should wait and some don't. we can vote or just let someone named Brian to make a call. I can do both really, not hard to rebase it for another two months. ultimately it's for the team to carry the maintenance of these code paths. | 14:20 |
| ralonsoh | LB has been laying in the repository for too long, without any support | 14:20 |
| haleyb | ihrachys: vote would be better, consensus is telling me people will vote removal | 14:21 |
| ralonsoh | we already marked it as experimental because calling it "deprecated" implies not testing | 14:21 |
| haleyb | and it is untested except for experimental | 14:21 |
| ralonsoh | so removing it now is a logical action | 14:21 |
| ralonsoh | (after 5 cycles as experimental) | 14:21 |
| slaweq | I'm fine with removing it now, just wanted to be on safe side and ask just to make sure there are no any obvious problems with it | 14:21 |
| ihrachys | slurp users will get it removed before next slurp anyway. they will skip Flamingo. | 14:22 |
| slaweq | I (not formally) vote to go with it now but lets haleyb decide | 14:22 |
| ihrachys | wait, I think I mixed release types. | 14:22 |
| ihrachys | so next is slurp? then we are in agreement? | 14:22 |
| ralonsoh | no, E is slurp (this one) | 14:22 |
| slaweq | ihrachys 2025.1 will be slurp | 14:23 |
| ihrachys | yeah ok, then I confused everyone and lajoskatona was also for removing it now. | 14:23 |
| ralonsoh | so, as slaweq, I vote for removing it now | 14:23 |
| ihrachys | LFG then! | 14:23 |
| lajoskatona | +1 | 14:24 |
| haleyb | ok, i'll see what we get for reponses, and if there is some technical reason on wait until F | 14:24 |
| mlavalle | +1 | 14:24 |
| haleyb | thanks for the discussion | 14:25 |
| haleyb | i had no other announcements | 14:25 |
| haleyb | #topic Bugs | 14:26 |
| haleyb | last week ralonsoh was the deputy, his report is at | 14:26 |
| haleyb | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/GH4YANL6MFGMD4YGV6QAT2GU7EHWSZP6/ | 14:26 |
| haleyb | there are two unassigned | 14:26 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bug/2093327 | 14:27 |
| ralonsoh | I'll take 2093327 | 14:27 |
| haleyb | sold! | 14:27 |
| haleyb | that might be a bug in OVN? | 14:27 |
| ralonsoh | I need to check with ovn/ovs folks what is happening there | 14:27 |
| ralonsoh | maybe, yes | 14:27 |
| haleyb | ack | 14:28 |
| haleyb | the other one was a doc bug | 14:28 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bug/2093317 | 14:28 |
| haleyb | vpnaas documention | 14:28 |
| haleyb | bpetermann: is that something you could look at? seems related to https://review.opendev.org/c/openstack/neutron-vpnaas/+/895651 | 14:30 |
| * mlavalle dropping off | 14:31 | |
| lajoskatona | yes the documentation request is related to the OVN driver as I remember | 14:31 |
| bpetermann | Yes, I can see what I can add to the documentation | 14:31 |
| haleyb | bpetermann: thanks | 14:31 |
| bpetermann | Regarding the auto-failover there's another bug ticket. It currently doesn't work because a missing OVN IDL access in the periodic worker | 14:32 |
| ihrachys | I think we generate some web pages from oslo.config e.g. for neutron https://docs.openstack.org/neutron/pike/configuration/ml2-conf.html | 14:32 |
| ihrachys | if not, vpnaas should do the same and then I guess there will be some "documentation" for the missing options. | 14:32 |
| ihrachys | there is already https://docs.openstack.org/neutron-vpnaas/latest/configuration/index.html is it not enough? | 14:33 |
| lajoskatona | bpetermann: this is the bug you refer I think: https://bugs.launchpad.net/neutron/+bug/2089250 | 14:33 |
| haleyb | ihrachys: yes, things are automated via some setup.cfg entries, assuming everything was added | 14:33 |
| bpetermann | lajoskatona: yes | 14:34 |
| lajoskatona | ihrachys: the cfg options in the bug report are not in the above docs, good question why | 14:35 |
| *** eagles is now known as beagles | 14:36 | |
| ihrachys | they are not listed in neutron_vpnaas/opts.py probably | 14:37 |
| haleyb | we can maybe figure it out in the context of the bug, could be something simple | 14:39 |
| ihrachys | I'll send a patch for this in 15m | 14:39 |
| haleyb | any other bugs to discuss? | 14:39 |
| haleyb | lajoskatona is the deputy list week, ykarel will be next week | 14:39 |
| lajoskatona | my eyes on launchpad | 14:40 |
| haleyb | +1 | 14:40 |
| haleyb | ok, moving on | 14:40 |
| haleyb | #topic community goals | 14:40 |
| haleyb | lajoskatona: good progress on neutronclient deprecation | 14:40 |
| haleyb | #link https://review.opendev.org/q/topic:%22bug/1999774%22 | 14:41 |
| lajoskatona | some progress at least :-) | 14:41 |
| haleyb | i had some comments on the FIP one | 14:42 |
| haleyb | they're in the review | 14:42 |
| lajoskatona | haleyb: I saw, but had no time to go back this week, thanks for checking | 14:42 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron-vpnaas master: Register VPN_AGENTS_SCHEDULER_OPTS in plugin config sample https://review.opendev.org/c/openstack/neutron-vpnaas/+/939246 | 14:42 |
| haleyb | lajoskatona: ack, keep up the good work :) | 14:43 |
| lajoskatona | :-) | 14:43 |
| haleyb | other goal is eventlet deprecation | 14:43 |
| ralonsoh | yes, I have two main topics | 14:44 |
| ralonsoh | https://review.opendev.org/q/topic:%22bug/2087943%22 | 14:44 |
| ralonsoh | L3 eventlet deprecation | 14:44 |
| ralonsoh | the reviews: reducing the pool will slow down the processing | 14:44 |
| ralonsoh | I'll provide a script, creating networks and routers connected to external network | 14:44 |
| ralonsoh | and how to test it with both codes | 14:44 |
| ralonsoh | there is no performance impact | 14:44 |
| ralonsoh | apart from this, using threads won't provide more speed in the python code, same as in the dhcp agent | 14:45 |
| ralonsoh | if we really want to improve the dhcp/l3 performance, we need to create some kind of multiprocessing agent | 14:45 |
| ralonsoh | please check these patches | 14:45 |
| ralonsoh | as I commented, I'll add this script | 14:45 |
| ralonsoh | the second topic: the metadata | 14:45 |
| ralonsoh | https://review.opendev.org/c/openstack/neutron/+/937545 | 14:45 |
| ralonsoh | https://review.opendev.org/c/openstack/neutron/+/938393 | 14:46 |
| ralonsoh | both the ovn metadata agent and the ovn agent | 14:46 |
| ralonsoh | both tested ok in the CI | 14:46 |
| ralonsoh | of course, as I documented: we lost the capability right now to spawn multiprocess metadata server | 14:46 |
| ralonsoh | this can be implemented in a closer future | 14:46 |
| ralonsoh | but right now, we need to remove eventlet | 14:47 |
| ralonsoh | that's my update | 14:47 |
| lajoskatona | thanks fo the update | 14:47 |
| haleyb | ralonsoh: i saw the comments on L3 and thanks for digging into that issue - i will let the data drive any decision as i'm remembering from years ago the thread pool helped, but i was proven wrong with the dhcp-agent already | 14:48 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add LB sync logic https://review.opendev.org/c/openstack/ovn-octavia-provider/+/925324 | 14:48 |
| ralonsoh | I'll provide a script to create several routers. Then, is just a matter of restarting the L3 agent and check the times | 14:48 |
| ralonsoh | I have no performance impact (in my dev env, of course) | 14:49 |
| haleyb | ralonsoh: and regarding the remaining pieces/bugs, i see 4 remaining, although the LB one should go away | 14:50 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bugs?field.tag=eventlet-deprecation | 14:50 |
| ralonsoh | OVS is being handled by sahid | 14:50 |
| ralonsoh | we can remove the LB one | 14:50 |
| ralonsoh | I'm testing right now the Neutron API | 14:51 |
| ralonsoh | and the sriov agent, once we have a oslo.services release without eventlet, should be trivial | 14:51 |
| haleyb | so that leaves dhcp-agent | 14:51 |
| ralonsoh | ah, same comment | 14:51 |
| ralonsoh | I already did some patches (maybe not related to this bug, because I didn't create it yet) | 14:52 |
| ralonsoh | but (1) with the metadata server, (2) the oslo.services and (3) some minor methods, it will be ready | 14:52 |
| ralonsoh | (I think so, I'm optimistic...) | 14:52 |
| ralonsoh | in any case, the DHCP agent is widely tested in the CI | 14:52 |
| ralonsoh | that's all | 14:53 |
| haleyb | ack, let's focus in the next week at reviewing/merging what is proposed | 14:53 |
| haleyb | thanks for the update | 14:54 |
| haleyb | #topic on-demand | 14:54 |
| haleyb | there was a topic from last week | 14:55 |
| haleyb | merge of https://review.opendev.org/c/openstack/neutron/+/936235 without two +2's | 14:56 |
| haleyb | i have left comments regarding this | 14:56 |
| lajoskatona | ralonsoh also mentioned that he has technical questions anyway for the commit | 14:57 |
| haleyb | ralonsoh: did you have a technical issue with the change? | 14:57 |
| haleyb | the open question was should we revert | 14:57 |
| ralonsoh | I'll comment today this patch, I need to make a reasonable statement | 14:57 |
| ralonsoh | maybe not, but it needs to be very clear for anyone in the community\ | 14:57 |
| ralonsoh | specially for cores | 14:58 |
| ralonsoh | that we cannot auto-merge our patches | 14:58 |
| ralonsoh | having said that | 14:58 |
| lajoskatona | +1 | 14:58 |
| ralonsoh | I did that 1 month ago, but with a trivial patch that was affecting the CI | 14:58 |
| ralonsoh | and I commented that in the patch | 14:58 |
| ihrachys | oh that's author merging their own patch without any +2 from anyone else? wow :) | 14:58 |
| ihrachys | (thought at first it's just +2 instead of two; nah that's zero) | 14:59 |
| ralonsoh | so we need to be honest having +2 powers | 14:59 |
| opendevreview | Merged openstack/os-vif unmaintained/victoria: Update .gitreview for unmaintained/victoria https://review.opendev.org/c/openstack/os-vif/+/911271 | 15:00 |
| haleyb | right. i can understand there being something lost in translation, culturally, where words imply things like being Ok with the patch | 15:00 |
| haleyb | but please always wait for +2's from others | 15:01 |
| haleyb | i can understand Liu is frustrated as it seems he is the only one doing ML2/OVS work | 15:02 |
| ralonsoh | I disagree with this statement | 15:02 |
| ralonsoh | we all do OVS support, he does features | 15:02 |
| lajoskatona | +1, others also work on it and use and give feedback with bugs and patches | 15:02 |
| ralonsoh | if he wants OVS to be alive, we could have assigned the eventlet depreaction | 15:02 |
| ralonsoh | for example | 15:02 |
| slaweq | TBH I don't really got his frustration - I didn't saw him on IRC asking for reviews of those patches, he is not active on the meetings, etc. | 15:02 |
| haleyb | ralonsoh: sorry, bad wording on my part | 15:03 |
| slaweq | and doing things like that is IMO violating our rules, nobody else of us is doing similar things really | 15:03 |
| haleyb | slaweq: understood, and i asked that he do that going forward, there is some overlap with EU at least to ping people | 15:03 |
| ihrachys | let's just keep an eye that it's not happening going forward. everyone makes mistakes sometimes. :) if there's technical argument beyond rules, this can be brought up of course too. | 15:05 |
| lajoskatona | +1, I think from all timezones there will be somebody who overlaps so can be asked on IRC or on mail list | 15:06 |
| haleyb | so i'm in a bind here as PTL obviously, not sure how far to push. if it happens again though i think we will need to re-visit things | 15:06 |
| ralonsoh | ^ agree | 15:07 |
| lajoskatona | haleyb: ack | 15:07 |
| slaweq | haleyb +1 to what You said | 15:07 |
| svinota | @lajoskatona, looks like I missed the meeting for one hour :) TZ issues. I'll try next week then | 15:07 |
| haleyb | i know lajoskatona and myself have been reviewing one of his series, but they are complicated things | 15:07 |
| ralonsoh | svinota, good to read you! | 15:07 |
| ralonsoh | join us next week, of course | 15:07 |
| lajoskatona | svinota: Hi :-) | 15:07 |
| svinota | ralonsoh, =^_^= | 15:07 |
| lajoskatona | Just one more thing for on-demand (overtime...): bpetermann has a patch long waiting in n-lib: https://review.opendev.org/c/openstack/neutron-lib/+/903971 | 15:08 |
| svinota | hallå dudes, long time no see | 15:08 |
| haleyb | ok, we are over time, thanks for the comments on ^^^ | 15:08 |
| lajoskatona | if you have time please check it | 15:08 |
| ralonsoh | lajoskatona, I still keep my comment in this patch | 15:08 |
| bpetermann | I'm not sure what I should change though | 15:09 |
| haleyb | ok, have a good week everyone, and ping people here for reviews and/or add them to the priorities board | 15:09 |
| lajoskatona | ralonsoh: ack, thanks, | 15:09 |
| haleyb | #endmeeting | 15:09 |
| opendevmeet | Meeting ended Tue Jan 14 15:09:47 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:09 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/networking/2025/networking.2025-01-14-14.00.html | 15:09 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/networking/2025/networking.2025-01-14-14.00.txt | 15:09 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/networking/2025/networking.2025-01-14-14.00.log.html | 15:09 |
| ralonsoh | bye | 15:09 |
| lajoskatona | bye | 15:10 |
| slaweq | o/ | 15:10 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: Add limit of tags for every resource https://review.opendev.org/c/openstack/neutron/+/937887 | 15:10 |
| ralonsoh | bpetermann, you are changing the API because you are modifying VPN_SUPPORTED_ENCRYPTION_ALGORITHMS | 15:10 |
| bpetermann | How can I add values to an existing enum? | 15:11 |
| ralonsoh | instead of that, you should create a new API that includes the old API and the new algorithms | 15:11 |
| slaweq | ralonsoh hey, I addressed your comment in ^^ but I'm not sure about unit test you are requesting there | 15:11 |
| ralonsoh | slaweq, let me check | 15:11 |
| slaweq | can you take a look at it again when you will have few minutes? | 15:11 |
| slaweq | thx | 15:11 |
| lajoskatona | bpetermann: can you try to add a new extension where the IPSEC_POLICIES extension is extended with the new algorithm values? | 15:11 |
| ihrachys | is it a change in api when you allow new options for the same field?.. | 15:11 |
| ralonsoh | ihrachys, yes, if the field had initially a defined set | 15:13 |
| ralonsoh | ihrachys, let me give you an example (DSCP, one sec) | 15:13 |
| ralonsoh | https://review.opendev.org/c/openstack/neutron-lib/+/854117 | 15:14 |
| ralonsoh | that patch, that is mine, is incorrect | 15:15 |
| ihrachys | fair enough; I haven't noticed the validator. does a plugin then report support for both extensions? no issues with loading both? | 15:15 |
| ralonsoh | you can make an extension dependant on another one | 15:15 |
| ralonsoh | actually the Neutron extension can accept both API definitions | 15:16 |
| ihrachys | yes; but then both extensions extend same field definition. which one will be activated: the old or new? do we have guarantees for it? | 15:16 |
| ralonsoh | and check if the second one is loaded or not | 15:16 |
| ralonsoh | the new one implies the old one | 15:16 |
| ihrachys | when a api user requests /extensions endpoint, will they see both? | 15:17 |
| ralonsoh | we have REQUIRED_EXTENSIONS in the n-lib API definitions | 15:17 |
| ralonsoh | yes, it should show both extensions | 15:17 |
| ralonsoh | I would prefer to have Nova API versioning, but we choose the use of extensions | 15:18 |
| bpetermann | that's what my patch is using: the existing 'vpn | 15:18 |
| bpetermann | ... | 15:18 |
| ihrachys | ok; as long as both are reported and as long as the "new" validator is in effect (and not the old one), then I agree that adding a new extension is preferred. | 15:18 |
| ralonsoh | bpetermann, that's the point, you should create a new one because you are extending an existing one | 15:18 |
| ralonsoh | let me find a good example (if we have in the code) | 15:19 |
| ralonsoh | ah yes, we have | 15:19 |
| bpetermann | the existing vpn extension uses the old list of choices, the new vpn-aes-ctr extension has REQUIRED_EXTENSIONS=[vpn.ALIAS] | 15:19 |
| bpetermann | and sets a different validation | 15:19 |
| ihrachys | bpetermann: imagine there's a new api client that knows that a new choice may be available for the field. (because that's what api extension from neutron-lib claims). then the new client attempts to call to an older neutron-api that did not support the new choices. the api server will - unexpectedly to api client - return validation error. it would be better if the api | 15:20 |
| ihrachys | client could, by fetching /extensions endpoint, determine whether new choices are available, or not | 15:20 |
| ralonsoh | bpetermann, let me check your last PS | 15:20 |
| ralonsoh | bpetermann, maybe I'm just stuck in PS4 (my review) | 15:20 |
| ralonsoh | bpetermann, ooooook, so yes, PS7 addressed my comments | 15:21 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: Fix neutron-status upgrade check tool https://review.opendev.org/c/openstack/neutron/+/939248 | 15:21 |
| ralonsoh | sorry because I was talking about the version I had in memory (that was an old one) | 15:22 |
| ihrachys | ralonsoh: what the latest PS changes is only the list const but not extension. so maybe no issue in the end? unless someone relies on the list itself. (they shouldn't) | 15:22 |
| ralonsoh | ihrachys, hmmmmm irght | 15:22 |
| ralonsoh | it is dangerous to modify a n-lib list | 15:22 |
| ralonsoh | bpetermann, you need to do the opposite | 15:22 |
| ralonsoh | https://review.opendev.org/c/openstack/neutron-lib/+/903971/7/neutron_lib/api/definitions/vpn.py#b63 | 15:23 |
| ralonsoh | this list must keep its name | 15:23 |
| ralonsoh | and you need to rename the new one | 15:23 |
| ihrachys | introduce a ALGORITHMS_NEW list | 15:23 |
| ralonsoh | because someone else outside neutron can use n-lib | 15:23 |
| ihrachys | then Rodolfo will be happy I guess :) | 15:23 |
| ralonsoh | exactly, just keep the previous name and rename the new list | 15:23 |
| ralonsoh | keep in mind that n-lib is a library that can be used anywhere | 15:24 |
| bpetermann | understood. Is _NEW a good choice. What will happen the next time _NEW_NEW? | 15:24 |
| ralonsoh | maybe not _NEW but _CRT | 15:24 |
| ralonsoh | VPN_SUPPORTED_ENCRYPTION_ALGORITHMS_WITH_CRT | 15:24 |
| bpetermann | ok. sounds good. I will change that | 15:24 |
| ihrachys | bpetermann: next time it will be _NEW_NEW_DRAFT3_FINAL1_NOW_FOR_REAL | 15:25 |
| slaweq | ihrachys and then next one will be _NEW_NEW_DRAFT3_FINAL1_NOW_FOR_REAL_BUT_REALLY :D | 15:28 |
| bpetermann | oh yes, fun naming things | 15:28 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: Add limit of tags for every resource https://review.opendev.org/c/openstack/neutron/+/937887 | 15:30 |
| opendevreview | Bodo Petermann proposed openstack/neutron-lib master: vpnaas: add support for AES CTR https://review.opendev.org/c/openstack/neutron-lib/+/903971 | 15:33 |
| ihrachys | seeing 'ERROR ovsdbapp.backend.ovs_idl.vlog [-] attempting to write bad value to column networks (ovsdb error: 0 values when type requires between 1 and 9223372036854775807): ovs.db.error.Error: ovsdb error: 0 values when type requires between 1 and 9223372036854775807' in gate, wonder if there is some regression | 15:34 |
| ihrachys | bpetermann: sorry for PITA; wondering if new values / list should go to the new module too. | 15:35 |
| ihrachys | the ERROR ^ is when AddLRouterPortCommand is called (with networks=[]) | 15:37 |
| lajoskatona | haleyb: thanks again for checking the taas patches, https://review.opendev.org/c/openstack/tap-as-a-service/+/885357 , guys if you have some free time please check the series :-) | 15:38 |
| lajoskatona | The topic for the taas feature is: https://review.opendev.org/q/topic:%22bug/2015471%22 | 15:39 |
| opendevreview | Bodo Petermann proposed openstack/neutron-lib master: vpnaas: add support for AES CTR https://review.opendev.org/c/openstack/neutron-lib/+/903971 | 15:44 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: WIP: ovn: don't attempt to create router port when no fixed-ips are set https://review.opendev.org/c/openstack/neutron/+/939253 | 15:54 |
| ihrachys | do we have any known issues where ovs monitor for sb would lose some port-binding updates when port becomes up when processing multiple events at the same time? I'm seeing something in gate that is weird... | 17:07 |
| ihrachys | basically, in ovn-controller logs, two different ports are being claimed / set to UP in sb at the same time; but in neutron-server log, ovsdb monitor logs claim that two events received for the same row. probably that's why one of the ports is not set to UP by neutron and eventually nova fails while waiting for vif | 17:09 |
| ihrachys | otherwiseguy: rings a bell by chance? ^ | 17:11 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add LB sync logic https://review.opendev.org/c/openstack/ovn-octavia-provider/+/925324 | 18:23 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add Listener sync logic https://review.opendev.org/c/openstack/ovn-octavia-provider/+/931250 | 18:23 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add Pool sync logic https://review.opendev.org/c/openstack/ovn-octavia-provider/+/931266 | 18:23 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add Member sync logic https://review.opendev.org/c/openstack/ovn-octavia-provider/+/931267 | 18:23 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add Health monitor sync logic https://review.opendev.org/c/openstack/ovn-octavia-provider/+/931288 | 18:23 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add sync floating IP support https://review.opendev.org/c/openstack/ovn-octavia-provider/+/929039 | 18:23 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Add sync floating IP support https://review.opendev.org/c/openstack/ovn-octavia-provider/+/929039 | 18:27 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: Remove shorturls from code https://review.opendev.org/c/openstack/neutron/+/939272 | 19:00 |
| opendevreview | Bodo Petermann proposed openstack/neutron-lib master: vpnaas: add support for AES CTR https://review.opendev.org/c/openstack/neutron-lib/+/903971 | 21:11 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM trying to reproduce bug 2094840 with more debug logs https://review.opendev.org/c/openstack/neutron/+/939285 | 22:12 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM trying to reproduce bug 2094840 with more debug logs https://review.opendev.org/c/openstack/neutron/+/939286 | 22:14 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM trying to reproduce bug 2094840 with more debug logs https://review.opendev.org/c/openstack/neutron/+/939287 | 22:14 |
| ihrachys | haleyb: I may be biased but I checked the recheck you posted in https://review.opendev.org/c/openstack/neutron/+/939272 and I wonder if the ovsdb-monitor failure there is somehow related to https://bugs.launchpad.net/neutron/+bug/2094840 that I'm trying to fish for right now... also ovsdb-monitor related, also claiming some events missing... | 22:17 |
| ihrachys | something fishy happening with the monitor I think. worth trying to figure out, seems to have different ways to fail. | 22:18 |
| haleyb | yeah, i started looking too and see that ovsdb-monitor failure in multiple places | 22:18 |
| haleyb | AssertionError: Interface monitor not filtered did not received an event for ports {'port2f3dc9', 'portcbe51a'} | 22:19 |
| ihrachys | in tempest, it seems like it shows as nova failing to build vm, there's a traceback that we could use to see if logstash query could pinpoint to a moment in time when it started. | 22:21 |
| ihrachys | of course assuming it's not 2weeks+ old :) | 22:21 |
| ihrachys | "nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed" | 22:21 |
| ihrachys | nah doesn't seem like there's a spike of THESE. goes back to start of the year where obv there are slightly fewer hits just because folks were on ptos. | 22:24 |
| haleyb | i can't seem to find logstash.openstack.org so i can't help :-/ | 22:28 |
| ihrachys | it's https://opensearch.logs.openstack.org now | 22:29 |
| haleyb | need a degree to figure out the syntax :( shows how much i use it | 22:41 |
| ihrachys | haleyb: also... ovn-rally job times out and when you look into logs... neutron-server logs are at 100mb+ | 22:42 |
| ihrachys | as example https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ad7/927216/10/check/neutron-ovn-rally-task/ad7f979/controller/logs/index.html | 22:43 |
| ihrachys | (looks like it's using q-svc not neutron-api prefixes for devstack services? not sure if this is important but I noticed) | 22:44 |
| haleyb | shouldn't matter afaik | 22:45 |
| ihrachys | it seems wrong that any job would produce 100mb logs but maybe I'm missing smth | 22:45 |
| ihrachys | I suspect infra folks may not be happy about crunching these :p | 22:46 |
| haleyb | neutron has been a pig for a long time | 22:48 |
| haleyb | anyways, there are some interesting errors in that log file | 22:48 |
| haleyb | ovsdbapp queue.Full amongst others | 22:49 |
| ihrachys | rally timeouts is something very new. first hit today. | 22:50 |
| ihrachys | haleyb: oh interesting re queue | 22:51 |
| ihrachys | otherwiseguy: ^ what would queue.full in ovsdbapp mean? | 22:51 |
| haleyb | also some eventlet errors like "RuntimeError: Second simultaneous write on fileno 9 detected" wth | 22:54 |
| haleyb | "Unless you really know what you're doing, make sure that only one greenthread can write any particular socket" - like from a movie | 22:54 |
| ihrachys | no I do not know what I'm doing yikes rude | 22:56 |
| ihrachys | time to revert eventlet stuff? :) | 22:56 |
| ihrachys | looking at latest merged https://review.opendev.org/q/status:merged+project:openstack/neutron | 22:57 |
| haleyb | and i need to take off for the day, sorry i can't help more, but yeah, maybe something like https://review.opendev.org/c/openstack/neutron/+/939078 is causing issues? | 22:57 |
| haleyb | or the spawn one, guess it wouldn't hurt to propose reverts and see if gate is happy | 22:58 |
| ihrachys | i dunno that one seems like something agent specific. you see these messages in neutron-server? | 22:58 |
| ihrachys | spawn one is just cleanup, shouldn't do a thing | 22:59 |
| ihrachys | well could it be eventlet bump from today? https://review.opendev.org/c/openstack/requirements/+/933257 | 23:00 |
| haleyb | oh, could be. could try a pin | 23:01 |
| ihrachys | will revert that and set depends-on to see if it does anything good | 23:01 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM seeing if reverting eventlet to old helps with rally timeouts https://review.opendev.org/c/openstack/neutron/+/939294 | 23:02 |
| haleyb | thanks, i really have to go now, will check later | 23:03 |
| ihrachys | timing is very suspicious. first hit of timeouts in rally 08:43, bump for eventlet merged at 07:42 | 23:04 |
| ihrachys | haleyb: ciao | 23:04 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM seeing if reverting eventlet to old helps with rally timeouts https://review.opendev.org/c/openstack/neutron/+/939295 | 23:05 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM seeing if reverting eventlet to old helps with rally timeouts https://review.opendev.org/c/openstack/neutron/+/939296 | 23:05 |
| ihrachys | interesting I don't see similar eventlet errors in a tempest job that ran to validate the eventlet bump in requirements repo: https://049f382ac557ff1e6df3-e238d8c9e51a1bd911917d5225bba33a.ssl.cf5.rackcdn.com/933257/10/gate/tempest-full-py3/7e848ed/controller/logs/index.html | 23:28 |
| ihrachys | maybe something special about rally job that it triggers this. maybe it's the scale of requests (rally bombs with lots more requests than tempest) | 23:29 |
| ihrachys | the size of neutron-server logs in the tempest job is also obnoxious, 70mb, so probably the mere size is not the issue. (I mean... I think it's obnoxious and makes neutron look bad and probably should be fixed... just maybe not directly related to the gate busted) | 23:30 |
| ihrachys | ok gotta drop now; hope folks to the right of greenwich will pick this up tomorrow :) | 23:31 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!