| opendevreview | Liushy proposed openstack/neutron master: [ovn]Add one periodic task of 'agent_health_check' in api worker https://review.opendev.org/c/openstack/neutron/+/916903 | 04:00 |
|---|---|---|
| opendevreview | Liushy proposed openstack/neutron master: [ovn]Add one periodic task of 'agent_health_check' in api worker https://review.opendev.org/c/openstack/neutron/+/916903 | 04:03 |
| opendevreview | Liushy proposed openstack/neutron master: [ovn]Add one periodic task of 'agent_health_check' in api worker https://review.opendev.org/c/openstack/neutron/+/916903 | 04:05 |
| sahid | o/ | 07:05 |
| opendevreview | ZhouHeng proposed openstack/neutron-fwaas master: _get_default_fwg_id replace use CONTEXT_READER https://review.opendev.org/c/openstack/neutron-fwaas/+/916968 | 07:35 |
| sahid | quick question guys | 07:39 |
| sahid | I don't get the relation between increasing rpc_workers and the issue vif creation fail, the timeout behind that is around 300 sec | 07:40 |
| sahid | what could block all the workers so the event gets not directed to nova | 07:40 |
| sahid | there is something that i'm missing... | 07:41 |
| sahid | lajoskatona: you may have some ideas? | 07:41 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: Skip abandoning of the old patches in unmaintained branches https://review.opendev.org/c/openstack/neutron/+/915053 | 07:51 |
| opendevreview | Sahid Orentino Ferdjaoui proposed openstack/neutron master: dhcp: fix dhcp cleaning stale devices process when enable action https://review.opendev.org/c/openstack/neutron/+/907250 | 07:57 |
| lajoskatona | sahid: Hi, could you give more details, or do you have a bug report for this issue? | 08:03 |
| sahid | well it's more about to understand the behavior, we are suffering number of vif creation fail, we have noticed (and is indicated in doc) that, increading rpc_workers can help | 08:05 |
| sahid | which is the case from our tests | 08:06 |
| sahid | but I d'ont really get how this helps | 08:06 |
| sahid | I want to know whether some of you could point me on the botleneck that increasing rpc_workers fix | 08:06 |
| opendevreview | Liushy proposed openstack/neutron master: [ovn]Add one periodic task of 'agent_health_check' in api worker https://review.opendev.org/c/openstack/neutron/+/916903 | 08:09 |
| sahid | question still not clear :-) ahaha | 08:19 |
| opendevreview | Sebastian Lohff proposed openstack/neutron master: Improve default gateway selection on DHCP agent https://review.opendev.org/c/openstack/neutron/+/917053 | 08:34 |
| opendevreview | Liushy proposed openstack/neutron master: [ovn]Add one periodic task for 'agent_health_check' https://review.opendev.org/c/openstack/neutron/+/916903 | 08:52 |
| opendevreview | ZhouHeng proposed openstack/neutron-fwaas master: _get_default_fwg_id replace use CONTEXT_READER https://review.opendev.org/c/openstack/neutron-fwaas/+/916968 | 08:54 |
| lajoskatona | sahid: I think the magic is that the notifier (in this case nova notifier) receives registry notifications, but the notification can be sent out when the server has the "feedback" from the agent, so i.e. the port really created on the host by the agent and that is the bottleneck | 09:08 |
| ykarel | slaweq, lajoskatona can you check backport https://review.opendev.org/c/openstack/neutron/+/911605 | 09:14 |
| ykarel | thx in advance | 09:14 |
| lajoskatona | ykarel: sure | 09:15 |
| opendevreview | Lajos Katona proposed openstack/neutron unmaintained/victoria: UM only: remove postgres and mariadb jobs https://review.opendev.org/c/openstack/neutron/+/916105 | 09:19 |
| opendevreview | Sebastian Lohff proposed openstack/neutron master: Improve default gateway selection on DHCP agent https://review.opendev.org/c/openstack/neutron/+/917053 | 10:58 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: Add pre-commit configuration https://review.opendev.org/c/openstack/neutron/+/917131 | 11:12 |
| mlavalle1 | haleyb: are we going to meet today? | 14:00 |
| haleyb | mlavalle1: yes, just running late in another meeting | 14:00 |
| mlavalle1 | cool | 14:00 |
| mlavalle1 | no rush | 14:00 |
| haleyb | just ended | 14:00 |
| haleyb | always interesting to have a meeting with our president | 14:01 |
| haleyb | #startmeeting neutron_drivers | 14:01 |
| opendevmeet | Meeting started Fri Apr 26 14:01:49 2024 UTC and is due to finish in 60 minutes. The chair is haleyb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| opendevmeet | The meeting name has been set to 'neutron_drivers' | 14:01 |
| haleyb | ping list: ykarel, mlavalle, mtomaska, slaweq, obondarev, tobias-urdin, lajoskatona, amotoki, haleyb, ralonsoh, JayF | 14:02 |
| TheJulia | o/ | 14:02 |
| obondarev | o/ | 14:02 |
| JayF | o/ | 14:02 |
| ykarel | o/ | 14:02 |
| slaweq | o/ | 14:02 |
| jcosmao | o/ | 14:02 |
| lajoskatona | o/ | 14:02 |
| mlavalle | \o | 14:02 |
| haleyb | ok, looks like we have quorom, two items in the agenda today | 14:03 |
| haleyb | first is regarding ironic | 14:04 |
| haleyb | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/5SX35AQZ65G6HWQ7WARLJTJK3ASAOWNU/ | 14:04 |
| haleyb | does someone want to give the quick overview? | 14:04 |
| lajoskatona | I suppose the best would be JayF or TheJulia :-) | 14:05 |
| TheJulia | The tl;dr is Ironic is looking for a way for standalone operators to toggle network device config remotely in a secure way which is compatible with separation of duties requirements which we can then extend to better support DPU/IPU network configuration at a later point in time | 14:05 |
| TheJulia | We have a generalized idea which involves leveraging the basic design of the ML2 interface for part of the mechanism, but disjointed from Neutron entirely. With that, we see a path to also be compatible and address some of the more classical integrated challenges as well, but one step at a time. | 14:06 |
| * mlavalle met with TheJulia on line last week so very well aware of the proposal | 14:07 | |
| TheJulia | We're definitely not trying to replace Neutron with this, just a focused effort on a use path which needs attention and capabilities since approximately half of Ironic's use base doesn't have other openstack services | 14:08 |
| TheJulia | Questions? Concerns? | 14:09 |
| haleyb | TheJulia: so in some cases there isn't really an openstack cloud per-se, just running services like neutron? | 14:10 |
| lajoskatona | if I understand well big chunk of these usecases are already from k8s | 14:10 |
| obondarev | so ideally you want smth like restricted minimalistic Neutron/ML2 version that you could use and that could also work without rest of OpenStack? | 14:10 |
| TheJulia | haleyb: ironic running as either a hidden detail or a user facing service in standalone mode. | 14:10 |
| mlavalle | I wasn'r aware of that statistic. very interesting and higher than I would have guessed | 14:11 |
| JayF | metal3.io, for instance, is powered by Ironic | 14:11 |
| TheJulia | haleyb: so no actual "cloud" | 14:11 |
| JayF | many of their users aren't aware we are involved there, either :D | 14:11 |
| TheJulia | obondarev: Well, I think we'll build it but sort of accessible as a single entity service which holds no state, so it can be owned by a "network" team | 14:12 |
| JayF | One thing that's useful context, I suspect many of you are aware, Ironic already holds the repos for generic physical switch config (networking-generic-switch) and the MVP is more or less connecting Ironic to this service for basic servcing | 14:13 |
| TheJulia | mlavalle: realistically, we see our standalone usage growing because people need the tooling to manage the lower levels of infrastucture. | 14:13 |
| mlavalle | yeap, it makes sense | 14:13 |
| haleyb | so looking at your spec Proposed Change section | 14:14 |
| haleyb | #link https://review.opendev.org/c/openstack/ironic-specs/+/916126/4/specs/approved/mercury.rst#91 | 14:14 |
| lajoskatona | this service should be part of ironic? or a plugin like thing? | 14:14 |
| haleyb | what things are going to be required of neutron? | 14:15 |
| TheJulia | lajoskatona: Many of them are from k8s, but at the same time the separation of duties/configuration issues we're sort of circling around seem like they could also help the overall model of use for Neutron as well. | 14:15 |
| TheJulia | haleyb: I think it might be as short as "please don't break the ml2 interface" :) | 14:15 |
| JayF | haleyb: I think a nonzero part of this consultation is also for "if we're doing network stuff, we want the blessing of the networking cloud folks" as well | 14:15 |
| TheJulia | haleyb: and no hate mail as well, please :) | 14:15 |
| obondarev | so maybe moving some parts of Neutron to a package/lib could be useful for you, right? | 14:15 |
| JayF | neutron-lib is already used by NGS | 14:16 |
| lajoskatona | +1 for neutron-lib | 14:16 |
| obondarev | neutron-lib is good but getting too large maybe.. | 14:16 |
| lajoskatona | that can be a good boilerplate lib | 14:16 |
| TheJulia | lajoskatona: I've kind of proposed this be modeled as an rpc style service, so a node may be configured for it, or realistically neutron directly. It is up to the infrastructure operator | 14:16 |
| lajoskatona | TheJulia: thanks | 14:17 |
| JayF | The exciting thing is also this provides some interesting bootstrap capabilitites | 14:17 |
| TheJulia | obondarev: I'm not sure, it looks like the ML2 interface is cleanly enough defined that I'm not sure we'll need other things from Neutron to directly pull this off | 14:17 |
| JayF | since some Ironic nodes could use mercury to get network control initially to bootstrap a neutron/integrated ironic installation | 14:17 |
| mlavalle | JayF: like I said to TheJulia during our conversation, if it makes sense for use cases, you have my humble blessing and no hate email from me | 14:18 |
| TheJulia | Yes, DPU/IPU devices are definitely "weird" to be mentally modeled since they are basically computer systems inside of computer systems which networking can pass through. | 14:18 |
| obondarev | separating ML2 interface + models might make sense, I'm thinking if anyone else could benefit for it.. | 14:19 |
| haleyb | on obondarev's point, we routinely move things to neutron-lib, are there other logical things we can move to facilitate things? just curious | 14:19 |
| TheJulia | obondarev: I think just the ml2 interface unless there are objects getting passed in that I'm not aware of | 14:19 |
| TheJulia | obondarev: this is sort of where my neutron context starts to disappear | 14:19 |
| obondarev | TheJulia: those are technical details :) | 14:20 |
| TheJulia | obondarev: indeed :) | 14:20 |
| lajoskatona | From my side the spec can be used to add such high level details if this can be done in n-lib and that can be used ironic | 14:22 |
| TheJulia | I *suspect* splitting ml2 stuffs out into a separate library would be super clean for ML2 drivers as well and reduce the neutron-lib dependency | 14:22 |
| lajoskatona | by ironic ----^ | 14:22 |
| mlavalle | beyond addressing the political side, are you recruting help today? | 14:22 |
| TheJulia | mlavalle: I don't know, if folks are interested in helping move along the idea that would be awesome, but we're still very early in ironic's discussions | 14:23 |
| lajoskatona | if we have to move code to n-lib that is help :-) | 14:23 |
| lajoskatona | or to some new ml2-lib | 14:23 |
| obondarev | not sure is ML2 interface is changed often, probably not. With that I believe making it a standalone lib/package that could be used by Neutron (of course) and something else is a sane idea | 14:23 |
| mlavalle | TheJulia: ok, I | 14:23 |
| mlavalle | I | 14:24 |
| haleyb | TheJulia: creating and maintaining a new ML2 library would fall on this team of course | 14:24 |
| mlavalle | i'll take a look and see if i can help. I'll start by reviewing the spec | 14:24 |
| TheJulia | I suspect that would be awesome, it might make sense to double-check networking-generic-switch's use of the neutron-lib to make sure nothing else gets used. I'm not sure about networking-baremetal though | 14:25 |
| TheJulia | networking-baremetal has a different reason for existing | 14:25 |
| TheJulia | mlavalle: much appreciated | 14:25 |
| TheJulia | haleyb: that would be awesome | 14:25 |
| lajoskatona | do we need to track the study of this cut to new lib in Neutron RFE for example? | 14:26 |
| haleyb | well, i didn't say "yes" :) i see obondarev mentioned it as well but don't know how much work that would be | 14:26 |
| obondarev | haleyb: fair point | 14:27 |
| TheJulia | haleyb: well, yeah. I think a starting point is to wrap a brain or two around it :) | 14:27 |
| haleyb | lajoskatona: if some end goal is to have another library, yes that would be its own RFE, etc | 14:27 |
| lajoskatona | perhaps we can create an RFE for this and track it and write the conclusion first to that as an input for ironic spec and for Neutron team also to see the amount of work predicted | 14:28 |
| mlavalle | +1 | 14:28 |
| TheJulia | lajoskatona: that sounds like an excellent path | 14:28 |
| haleyb | right. i'm guessing any short-term work in neutron is fixing of bugs as you find them, in those grey areas, as happens already today | 14:30 |
| TheJulia | Yeah, what lajoskatona suggests is sort of an ideal path at the moment | 14:31 |
| TheJulia | Anything else? | 14:32 |
| haleyb | ack. can someone be the point person for neutron for this work? i.e. read specs, attend discussions? | 14:32 |
| slaweq | do I understand correctly that the plan is to create new, kind of "neutron-lib" library under neutron team and move parts of the code from neutron there? | 14:32 |
| lajoskatona | slaweq: not a plan at themoment, just a possibility | 14:33 |
| slaweq | ok | 14:33 |
| obondarev | slaweq: I'd say not at this point | 14:33 |
| slaweq | but that is the possiblitity discussed here so far, right? | 14:33 |
| obondarev | yes | 14:34 |
| slaweq | ok | 14:34 |
| lajoskatona | the best would be to use n-lib for this also, as that is used by many projects already | 14:34 |
| slaweq | thx for clarification | 14:34 |
| mlavalle | haleyb: I can do it, bandwidth permitting of course | 14:34 |
| slaweq | lajoskatona ++ | 14:34 |
| haleyb | mlavalle: great, thanks, and feel free to pull me in when necessary | 14:35 |
| haleyb | ok, so the outcome is we won't break ML2, and won't send hate mail? :) | 14:36 |
| mlavalle | i'll start by filing the rfe lajoskatona suggested | 14:36 |
| lajoskatona | I also try to allocate some time for this to have an idea at least | 14:36 |
| TheJulia | :) | 14:36 |
| obondarev | haleyb: +1! | 14:36 |
| lajoskatona | mlavalle: thanks | 14:36 |
| lajoskatona | haleyb: :D | 14:37 |
| haleyb | let's just keep the discussion open here (that's the real goal) | 14:37 |
| TheJulia | ++ | 14:38 |
| haleyb | alright, thanks everyone for the great discussion, and good luck! | 14:39 |
| TheJulia | Thanks! | 14:39 |
| mlavalle | TheJulia JayF thanks for the consideration of bringing us to speed and requesting our opinion. we certainly don't need civil wars in this community | 14:39 |
| TheJulia | Yeah, we definitely want a model of mutually beneficial should this move forward. | 14:40 |
| lajoskatona | Thanks for the time and preparation of this topic | 14:41 |
| haleyb | we did have one other item, non-ironic related | 14:42 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bug/2007674 | 14:42 |
| haleyb | jcosmao: i don't think we ever triaged this when it initially was filed | 14:42 |
| haleyb | hopefully julien is still here | 14:43 |
| lajoskatona | yes as I remember it was opened as general bug | 14:43 |
| jcosmao | actually, i started working on this topic before finding this rfe | 14:44 |
| haleyb | #link https://etherpad.opendev.org/p/neutron-rfe-2007674 | 14:44 |
| jcosmao | yes, i added some details about changes i want to propose in this etherpad | 14:44 |
| jcosmao | for the full context, same topic than what was discussed at large scale meeting by amorin: https://meetings.opendev.org/meetings/large_scale_sig/2024/large_scale_sig.2024-04-24-09.01.log.html#l-58 | 14:45 |
| slaweq | so IIUC this really needs first changes in oslo.messaging,right? | 14:46 |
| obondarev | jcosmao: did you have a chance to try your changes at scale? It sound like right thing to do, just not sure what other limitations of rabbit it could face.. | 14:47 |
| jcosmao | yes exactly, most changes are in oslo.messaging | 14:47 |
| slaweq | and another question: should we maybe ask for opinion oslo cores? | 14:48 |
| mlavalle | obondarev: good point | 14:48 |
| jcosmao | obondarev yes, it's in production since few weeks | 14:48 |
| obondarev | jcosmao: oh that's very convincing point :) | 14:48 |
| mlavalle | jcosmao: and are you seeig measurable improvements? | 14:49 |
| jcosmao | we reduced a lot nb of queues + connection. and yes rolling update of neutron agent are smooth now | 14:49 |
| mlavalle | wow, so what is not to like? let us see the code, please | 14:50 |
| lajoskatona | +1 | 14:50 |
| jcosmao | we also worked previously to move on rabbitmq streams for 'fanout' queues. this one was also a great improvment (queues are shared instead of 1 queue per agent) | 14:50 |
| obondarev | to be on the safe side I think we need to know the "capacity" of a connection: how many load it could handle? When is the right time to open new connection | 14:50 |
| obondarev | *rather than continue loading existing one | 14:52 |
| jcosmao | i don't think tcp connection is a bottleneck, even with 1 connexion it should be ok | 14:52 |
| haleyb | jcosmao: have you approached the oslo team or have patches proposed there? | 14:53 |
| obondarev | if also rabbit/oslo folks say this as well I think we are safe :) | 14:53 |
| lajoskatona | +1 perhaps only neutron doc and cfg should be changed? | 14:54 |
| jcosmao | we already submitted few patch to move on quorum queues / stream queues, it's not yet default config. But we did not yet submitted for this specific rfe | 14:55 |
| lajoskatona | ack | 14:56 |
| jcosmao | so, i can work on that to submit change on solo, and maybe link to this rfe ? | 14:56 |
| obondarev | so +1 from me with confirmation from oslo or rabbit cores (nova is a good proof but still it uses queue not that much as neutron) | 14:56 |
| haleyb | jcosmao: do you just see the neutron changes being small to facilitate new arguments/flags to the messaging code? | 14:57 |
| haleyb | it might be good to have a spec on what this entails | 14:58 |
| mlavalle | and maybe in that spec share some of the measured benefirs already apparent in jcosmao's deployment | 15:00 |
| jcosmao | there is not much changes in neutron, most of them, is ensure to pass proper parameter to oslo.messaging Target. | 15:00 |
| jcosmao | ok, is there a process to follow to create a spec ? | 15:01 |
| slaweq | for me it sounds reasonable from neutron PoV as long as oslo.messaging cores will accept those changes and will not have anything against | 15:01 |
| slaweq | and I don't think that RFE is really needed in this case, at least not for neutron | 15:01 |
| obondarev | agree with slaweq | 15:03 |
| haleyb | jcosmao: we have a neutron-specs repo, but as slaweq mentioned, if the changes are very small to neutron we probably don't need one, they can be described in the bug and/or change | 15:03 |
| jcosmao | slaweq agree also | 15:04 |
| lajoskatona | +1, don't make to heavy in Neutron if most of the changes are in oslo | 15:05 |
| haleyb | we should vote as we're over time | 15:05 |
| jcosmao | so if it's ok for you, i can work on submitting patch to oslo team, with neutron context | 15:05 |
| lajoskatona | jcosmao: thanks | 15:05 |
| jcosmao | and if ok, i'll continue on neutron side | 15:05 |
| mlavalle | +1 from me | 15:05 |
| lajoskatona | +1 from me | 15:05 |
| haleyb | jcosmao: yes, that works | 15:05 |
| haleyb | +1 from me | 15:06 |
| obondarev | +1 | 15:06 |
| mlavalle | jcosmao: we like this minimal effort improvements, thanks! | 15:07 |
| haleyb | yes, less rabbit queues is good | 15:07 |
| jcosmao | :) thx, i'll keep you updated on that | 15:07 |
| obondarev | more rabbit holes! :D | 15:07 |
| haleyb | slaweq: were you a +1 ? | 15:07 |
| haleyb | rabbit stew? | 15:08 |
| obondarev | haleyb: eventually yes | 15:08 |
| haleyb | jcosmao: i will approve it, ping people here when you get to the neutron work, thanks for taking this on | 15:09 |
| jcosmao | haleyb ok great | 15:10 |
| haleyb | ok, thanks for the time everyone, enjoy your weekend | 15:10 |
| haleyb | #endmeeting | 15:10 |
| opendevmeet | Meeting ended Fri Apr 26 15:10:53 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:10 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/neutron_drivers/2024/neutron_drivers.2024-04-26-14.01.html | 15:10 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/neutron_drivers/2024/neutron_drivers.2024-04-26-14.01.txt | 15:10 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/neutron_drivers/2024/neutron_drivers.2024-04-26-14.01.log.html | 15:10 |
| lajoskatona | o/ | 15:10 |
| jcosmao | o/ | 15:11 |
| obondarev | o/ | 15:11 |
| mlavalle | \o | 15:12 |
| otherwiseguy | tkajinam: On the "predictable signal handling order" backports, I've been holding off on the neutron patches that use that because they'll need a requirements.txt bump for oslo.service which I can't do until releases are made. Do you think it will be possible to get the backports merged/releases made in the near future? | 16:53 |
| opendevreview | Brian Haley proposed openstack/neutron master: Simplify recursive handling of nested ovn networks for snat https://review.opendev.org/c/openstack/neutron/+/916939 | 17:03 |
| opendevreview | Sebastian Lohff proposed openstack/neutron master: Improve default gateway selection on DHCP agent https://review.opendev.org/c/openstack/neutron/+/917053 | 17:06 |
| opendevreview | Merged openstack/neutron stable/2023.1: Use the system-dependent string for IP protocol 4 https://review.opendev.org/c/openstack/neutron/+/911605 | 21:47 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!