| ttx | o/ | 15:00 |
|---|---|---|
| genekuo | o/ | 15:00 |
| mdelavergne | o/ | 15:00 |
| ttx | #startmeeting large_scale_sig | 15:00 |
| opendevmeet | Meeting started Wed Sep 15 15:00:29 2021 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'large_scale_sig' | 15:00 |
| oneswig_ | hi | 15:00 |
| ttx | #topic Rollcall | 15:00 |
| belmoreira | o/ | 15:00 |
| ttx | full house today! amorin ? | 15:00 |
| oneswig_ | o/ | 15:00 |
| amorin | hello! | 15:00 |
| ttx | alright, let's get this started | 15:01 |
| ttx | Our agenda for today is at: | 15:01 |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-meeting | 15:01 |
| ttx | #topic Next OpenInfraLive session | 15:01 |
| ttx | So we'll have another "Large Scale OpenStack" episode on OpenInfra.Live on Oct 24 | 15:01 |
| ttx | We had a TODO for belmoreira to draft the episode description | 15:01 |
| ttx | But that raised questions whether "Architecture choices" was the best topic | 15:02 |
| ttx | belmoreira was considering "ops pain points" instead, but I'm not sure the format would work well | 15:02 |
| ttx | Since we can have only a few selected guests, it will end up being their pain points | 15:02 |
| ttx | with no way to simply validate that those are shared more widely | 15:02 |
| ttx | After meeting a couple of users lately there was a recurrent theme I would like to propose instead | 15:03 |
| ttx | "Neutron scaling best practices" | 15:03 |
| belmoreira | yeah... sorry for bringing the live episode discussion again | 15:03 |
| ttx | Because I got LOTS of questions on the "proper" way to do Neutron at scale in 2021. | 15:03 |
| ttx | Like... am I the only one using RBAC and encountering scaling issues? | 15:03 |
| ttx | Is OVS or OVN the way of the future? Are provider networks a good idea? | 15:03 |
| ttx | We could invite the Neutron PTL and a few others to get their view on those best practices | 15:04 |
| amorin | if we do so, we should invite someone from neutron team to discuss | 15:04 |
| ttx | Maybe Kolla-Ansible people on their choices of defaults | 15:04 |
| ttx | amorin: you read my mindn | 15:04 |
| oneswig_ | We are about to start an investigation on routed networks and OVN. I think this is a good topic :-) | 15:04 |
| amorin | (you were quicker than me) | 15:04 |
| ttx | + a few new users at that moment in their scaling journey that they are wondering if they are betting on the right horse | 15:04 |
| ttx | What do you think? | 15:04 |
| amorin | great topic in my opinion | 15:05 |
| genekuo | Sounds good | 15:05 |
| ttx | happy to include one of you if you think you have a great model story on that front | 15:05 |
| belmoreira | sounds good | 15:05 |
| oneswig_ | I can ask the kolla-ansible ptl | 15:06 |
| ttx | but the more I look the more I find people who are not 100% sure of their setup | 15:06 |
| amorin | we are using custom model, not upstream, so I dont think we are good speakers for this | 15:06 |
| ttx | amorin: ack | 15:06 |
| ttx | genekuo: do y'all at LINE have a good setup to share? Or struggling too? | 15:06 |
| ttx | belmoreira: IIRC your setup is "keep it simple and avoid issues"? | 15:07 |
| genekuo | We also have written our own plug-ins, so probably not the best to be one of the speaker | 15:07 |
| ttx | heh maybe writing up your own thing is the best practice lol | 15:07 |
| amorin | :) | 15:07 |
| ttx | Good speakers are people with an horizontal view of what works and what doesn't | 15:08 |
| ttx | + a few users with candid questions | 15:08 |
| belmoreira | cern uses linux bridge (mostly to have an easier migration from nova-network and it's simple!). If we go for pain points it would be mostly rabbit. However we are investigating Tungsten fabric. | 15:08 |
| amorin | maybe zigo? | 15:08 |
| belmoreira | the best would be to check with my colleague that manages Neutron | 15:08 |
| ttx | One of the users I talked to was struggling with Neutron RBAC generating a lot of load | 15:08 |
| amorin | dont remember his company | 15:08 |
| ttx | Infomaniak | 15:09 |
| amorin | I know he was active on some neutron topics recently | 15:09 |
| amorin | but I dont know the scale | 15:09 |
| ttx | Like does anyone else use Neutron RBAC? Is that even a good idea? | 15:09 |
| ttx | imtiazc: we are discussing doing "Neutron scaling best practices" as the next OILive episode | 15:10 |
| ttx | belmoreira: ++ | 15:10 |
| imtiazc | Thanks! @ttx | 15:11 |
| belmoreira | at scale we need to be gentle with Neutron. Any new addition (RBAC) can disturb the force | 15:11 |
| ttx | imtiazc: I talked to a few users and they all cited Neutron architectural choices as being one of the most nebulous choice they need to make | 15:11 |
| ttx | belmoreira: you sound like Yoda | 15:11 |
| ttx | But yeah, Neutron is clearly the new RabbitMQ in terms of ops top-of-mind | 15:12 |
| ttx | you can get it wrong and be stuck with it | 15:12 |
| imtiazc | There are lots of provider options to choose from. I believe operators also have their different use cases. | 15:13 |
| ttx | but even answering " what would you recommend as a driver those days" is a very good question to candidly answer | 15:13 |
| oneswig_ | ttx: can you confirm the date? | 15:13 |
| ttx | Oct 24 14utc | 15:13 |
| ttx | week after PTG | 15:13 |
| oneswig_ | Could it be Oct 28th? 24th is a Sunday | 15:14 |
| ttx | haha. hmm | 15:14 |
| * ttx triplechecks | 15:14 | |
| ttx | probably 28 yes, checking | 15:15 |
| imtiazc | We can go over how we made our decision lately after considering all the viable options. | 15:16 |
| ttx | October 14! | 15:16 |
| ttx | https://meetings.opendev.org/meetings/large_scale_sig/2021/large_scale_sig.2021-09-01-15.01.log.html#l-30 | 15:17 |
| ttx | The logs do not lie | 15:17 |
| ttx | Sorry for the confusion | 15:17 |
| ttx | So... week BEFORE Ptg | 15:17 |
| oneswig_ | thanks ttx! | 15:17 |
| ttx | oneswig_: are you gong to approach the Kolla-Ansible PTL about this? If he can bring the StackHPC experience trying multiple solutions there, taht would be awesome | 15:18 |
| ttx | going* | 15:18 |
| oneswig_ | I can do that, yes. | 15:18 |
| oneswig_ | Coincidentally he's done quite a lot with OVN upstream as well. | 15:19 |
| ttx | #info Oct 14 tentative episode topic: "Neutron scaling best practices" | 15:19 |
| ttx | #action oneswig_ to contact Kolla-Ansible PTL about participating | 15:19 |
| ttx | #action ttx to contact Neutron PTL about participating | 15:19 |
| ttx | #action belmoreira to connect with internal CERN neutron people to see if they have a good angle to bring | 15:20 |
| ttx | could be interesting to have Rd Hat or Canonical people on their own choices of drivers / configs that work well in their experience | 15:20 |
| imtiazc | We worked closely with RH last year. Their recommendation is OVN. | 15:21 |
| ttx | Cool, great to know | 15:21 |
| ttx | If we manage to get good answers we could document them in our large Scale Journey wiki | 15:22 |
| ttx | but worst case scenario it's a great topic to openly discuss | 15:22 |
| ttx | any other idea of who could bring interesting insights to the table? | 15:22 |
| ttx | I don;t really want to turn it into a drivers fight with each brand fighting. I prefer to have a bunch of curators discuss what works well for them | 15:23 |
| genekuo | It will be great if we can invite someone using Calico | 15:23 |
| genekuo | IMO, it's a very different approach compare to other common plug-ins | 15:23 |
| amorin | AFAIK it has limitations | 15:24 |
| imtiazc | Bloomberg has been using Calico. We just started building with Calico. Salesforce has been using Calico in production for years. | 15:24 |
| ttx | hmm yes or at least make sure that angle is covered | 15:24 |
| oneswig_ | +1 | 15:24 |
| ttx | OK, other thoughts? | 15:24 |
| genekuo | Yeah, it have limitations, but it works very well in some cases | 15:24 |
| amorin | ack | 15:24 |
| ttx | if you accept limitations it works well" is good insight | 15:25 |
| ttx | alright if no other suggestions/actions, we'll move on to next topic | 15:26 |
| ttx | #topic Large Scale Journey updates | 15:26 |
| imtiazc | Every choice of driver has trade-offs :) Software architecture is all about making the decision that best suits one's needs. | 15:26 |
| ttx | imtiazc: no we want absolute answers! | 15:26 |
| amorin | lol | 15:26 |
| ttx | OVN good! OVS bad! (or is it the other way around??) | 15:27 |
| reedip | multi-driver war... | 15:27 |
| ttx | #link https://wiki.openstack.org/wiki/Large_Scale_SIG | 15:27 |
| ttx | Does anyone has updates/blockers on actions related to documenting the Journey? | 15:27 |
| genekuo | I have some updates on the docs | 15:27 |
| genekuo | thanks to reedip | 15:27 |
| genekuo | Still have to adjust the layout though | 15:28 |
| reedip | thanks for the reviews @genekuo... | 15:29 |
| ttx | any specific part updated? | 15:30 |
| mdelavergne | maybe after the next live ? :P | 15:30 |
| genekuo | Yeah, mostly on how to track error rates and saturation/performance issues | 15:31 |
| ttx | nice! | 15:31 |
| ttx | alright, anyone else? | 15:31 |
| ttx | if not... | 15:31 |
| ttx | #topic Next meeting | 15:31 |
| ttx | #info next IRC meeting on Sept 29 | 15:31 |
| ttx | My goal is to have most of the guests for the episode confirmed by then | 15:32 |
| ttx | as well as a clear episode description | 15:32 |
| ttx | so tat we can discuss last details | 15:33 |
| ttx | #topic Open discussion | 15:33 |
| ttx | Alright, does anyone have a question to discuss with the group? | 15:33 |
| genekuo | Nothing from me | 15:33 |
| ttx | FWIW I've been inviting to this group those users that are on the verge of scaling up dramatically | 15:34 |
| ttx | Because they have lots of questions, which could be a good way of rapidly improving our doc | 15:34 |
| ttx | we'll see if they take the bait | 15:35 |
| genekuo | It's always good to have more people joining the discussion :) | 15:35 |
| amorin | agree | 15:35 |
| ttx | yeah, and they usually have experience of newer releases so this can be a two-way street | 15:35 |
| ttx | Hopefully that open discussion section of the meeting will be more lively as a result! | 15:36 |
| ttx | But for today... if nobody has questions (yet)... | 15:36 |
| belmoreira | +1 | 15:36 |
| ttx | Let's close early again! Have a great day/week, talk to you all soon | 15:37 |
| ttx | #endmeeting | 15:37 |
| opendevmeet | Meeting ended Wed Sep 15 15:37:17 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:37 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/large_scale_sig/2021/large_scale_sig.2021-09-15-15.00.html | 15:37 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/large_scale_sig/2021/large_scale_sig.2021-09-15-15.00.txt | 15:37 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/large_scale_sig/2021/large_scale_sig.2021-09-15-15.00.log.html | 15:37 |
| genekuo | Thanks all! | 15:37 |
| amorin | bye | 15:37 |
| mdelavergne | thanks everyone, see you! | 15:37 |
| oneswig_ | see you | 15:38 |
| reedip | o/ | 15:38 |
| *** osmanlicilegi is now known as Guest0 | 21:20 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!