Wednesday, 2023-05-10

opendevreviewMichal Nasiadka proposed openstack/kolla stable/xena: fluentd: cap elasticsearch again  https://review.opendev.org/c/openstack/kolla/+/88277207:04
opendevreviewMichal Nasiadka proposed openstack/kolla stable/xena: fluentd: cap elasticsearch again  https://review.opendev.org/c/openstack/kolla/+/88277207:06
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Fix publish take 2 - add secret  https://review.opendev.org/c/openstack/kolla/+/88280507:22
opendevreviewMerged openstack/kolla master: CI: Fix publish take 2 - add secret  https://review.opendev.org/c/openstack/kolla/+/88280508:33
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: Revert "Revert "ansible: bump min to 2.13 and max to 2.14""  https://review.opendev.org/c/openstack/kolla-ansible/+/88101809:09
opendevreviewMerged openstack/kolla stable/xena: fluentd: cap elasticsearch again  https://review.opendev.org/c/openstack/kolla/+/88277210:39
opendevreviewMichal Nasiadka proposed openstack/kolla stable/wallaby: fluentd: cap elasticsearch again  https://review.opendev.org/c/openstack/kolla/+/88277610:42
opendevreviewMaksim Malchuk proposed openstack/kayobe master: Fix AttributeError: 'Runtime' object has no attribute 'exec'  https://review.opendev.org/c/openstack/kayobe/+/88282012:12
opendevreviewMaksim Malchuk proposed openstack/kayobe master: CI: fix molecule job  https://review.opendev.org/c/openstack/kayobe/+/88282012:43
mnasiadkamgoddard mnasiadka hrw  bbezak frickler kevko SvenKieske - meeting in 1012:50
SvenKieskemnasiadka: meeting? :)13:02
mnasiadkaah sorry13:02
mnasiadka#startmeeting kolla13:02
opendevmeetMeeting started Wed May 10 13:02:23 2023 UTC and is due to finish in 60 minutes.  The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot.13:02
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:02
opendevmeetThe meeting name has been set to 'kolla'13:02
mnasiadka#topic rollcall13:02
ramona-beermann[m]o/13:02
mnasiadkao/13:02
bbezako/13:02
mmalchuk\o13:03
SvenKieskeo/13:03
opendevreviewMaksim Malchuk proposed openstack/kayobe master: Add custom user_data content configuration for Bifrost  https://review.opendev.org/c/openstack/kayobe/+/87955413:03
SvenKieskesorry, my laptop just crashed..13:03
mnasiadka#topic agenda13:04
mnasiadka* CI status13:04
mnasiadka* Release tasks13:04
mnasiadka* Regular stable releases (first meeting in a month)13:04
mnasiadka* Current cycle planning13:04
mnasiadka* Additional agenda (from whiteboard)13:04
mnasiadka* Open discussion13:04
mnasiadka#topic CI Status13:04
mnasiadkaseems we're green now13:05
mnasiadkaafter all those weird breakages13:05
mnasiadka#topic Release tasks13:05
mnasiadkaBasically 1st of June is release deadline for cycle-trailing projects13:05
mmalchukKayobe CI red13:05
mnasiadkammalchuk: so please update the whiteboard13:05
mmalchukfix: https://review.opendev.org/c/openstack/kayobe/+/88282013:05
mmalchuklets merge it and it will be green13:06
mnasiadkaSo we need to start cutting corners - for the 1st June deadline13:06
mnasiadkaI had a look in the Ansible bump issue13:06
mnasiadkaFound the commit in ansible-core that broke us, will start thinking how to overcome it13:07
mmalchukit not ansible, it molecule13:07
mnasiadka#link https://github.com/ansible/ansible/issues/7977613:07
mmalchuksetup.cfg have ansible-compat >= 2.2.013:07
mnasiadkammalchuk: I'm not speaking about your patch13:07
mmalchukok. sorry13:07
SvenKieskethis is the patch in question mnasiadka talks about afaik: https://review.opendev.org/c/openstack/kolla-ansible/+/88101813:08
SvenKieskemnasiadka: if I could speed up something/help somewhere, just ask :)13:09
mnasiadkayup, updated it with a link to a bug that broke us ;-)13:09
SvenKieskenice13:09
mnasiadkaThere is one patch in Kolla that will need merging, but first Rocky needs to have centos-release-openstack-antelope - I raised a bug in Rocky bug tracker and it seems it's in progress13:10
SvenKieskethere's an upstream fix already merged it seems? https://github.com/ansible/ansible/pull/7980413:10
mnasiadkaSvenKieske: it's the fix that broke us :)13:10
SvenKieskeouch :D13:10
mnasiadkatesting now with force_handlers: true/false if that's an easy fix13:10
mnasiadkabut probably not13:10
SvenKieskeok13:11
mmalchukmnasiadka whiteboard updated13:13
mnasiadkaWould be nice to get some reviews on https://review.opendev.org/c/openstack/kolla/+/882343 - it seems the version check is broken now ;-)13:13
mnasiadkaand we need it for https://review.opendev.org/c/openstack/kolla/+/825786/2713:14
SvenKieskeI can review that. in general it would be nice if dependencies are tracked/mentioned somewhere in the commit message or the bugtracker13:15
mnasiadkawell, those patches are stacked one on another, you can see that in the gerrit ui13:16
mnasiadkarelation chain - top right 13:16
mnasiadka#topic Current cycle planning13:16
mnasiadkaI think we've done that in the release tasks (I mean the planning)13:16
mnasiadka#topic Additional agenda (from whiteboard)13:17
SvenKieskeyeah, in this case it's there, but not always ;) related to that: WIPs are often initially not marked as WIP, but maybe I'm just to fast to review.13:17
mnasiadkaSvenKieske: well, some people mark WIP in commit message, some people mark it as a flag in Gerrit, some people do both, some people do none :)13:17
mnasiadka(mattcrees): With this precheck now added, should we enable RabbitMQ HA queues by default in Antelope? Precheck: https://review.opendev.org/c/openstack/kolla-ansible/+/88027413:18
mnasiadkamattcrees: around?13:18
mattcreesYep13:18
mattcreesBasically as the topic says, if people are happy that the precheck is enough then IMO we should enable RabbitMQ HA queues by default13:18
mnasiadkaI think it would be good to change the default, I already squashed 5+ bugs from people running non-HA on multinode with known problems13:19
SvenKieskeyeah, that would be nice, was there already a change open for that?13:19
mattcreesThere's also backports of the precheck ongoing13:19
mattcreesSvenKieske: I don't think so, I'll propose one now13:20
mnasiadkaI just merged the backports13:20
SvenKieskenice, will review, you can ping me about it13:20
mattcreesWill do13:20
mnasiadka#agreed to change the RMQ HA queues by default in Antelope13:21
SvenKieskethe backports already all have one +2 and are just missing workflow/verified, so I guess I can't do much there13:21
mnasiadkaThere's a topic by ihalomi - but he's not around13:21
mnasiadkaso I'll leave it for next week13:21
mnasiadka#topic Open discussion13:21
mnasiadkaAnyone, anything?13:21
SvenKieskeyeah  regarding: https://review.opendev.org/c/openstack/kolla/+/825786 he asked for help but there are still unadressed comments, so.13:21
mmalchukKayobe reviews13:22
mmalchuk1. https://review.opendev.org/c/openstack/kayobe/+/88282013:22
mmalchuk2. https://review.opendev.org/c/openstack/kayobe/+/88167913:22
mmalchukboth CI13:22
mmalchukmgoddard ask to fix kayobe-seed-images-* jobs ^^^ it fixed13:23
mmalchukmolecule broken recently13:23
mmalchukalso fixed13:23
mnasiadkaI've sent that on a different communication channel to Kayobe cores inside StackHPC, let's see if that gets reviewed.13:24
mmalchukthanks13:24
mnasiadkaSvenKieske: I have a feeling we need to push it over so it gets merged13:24
SvenKieskeyeah, might be, the docs comments can be addressed by us, I think. the tests not passing is the bigger issue I guess.13:26
mnasiadkaI'll have a look, probably we need to mock the docker version in more places13:27
mnasiadkaor once for everything that is not pre3.0.013:27
SvenKieskeappreciated, as I'm still not very familiar with this part of the code.13:28
mnasiadkaI started triaging the bugs, kolla and kayobe look fine, kolla-ansible is another 180 bugs to triage13:28
mnasiadkabut I should get there before the summit, and we could start having weekly bug reports on the ML13:29
mnasiadkaor bugs managed in a proper way13:30
mnasiadkaOk, anything else? or should we call it a day?13:30
SvenKieskeyeah, bug reports on the ML might be nice, I like that when neutron et al do this13:30
SvenKieskeI have the next meeting already scheduled, so..13:30
mnasiadkaOk then, thanks for coming - see you next week :)13:31
mnasiadka#endmeeting13:31
opendevmeetMeeting ended Wed May 10 13:31:07 2023 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:31
opendevmeetMinutes:        https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-05-10-13.02.html13:31
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-05-10-13.02.txt13:31
opendevmeetLog:            https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-05-10-13.02.log.html13:31
mmalchukmnasiadka thanks13:31
SvenKieskeyeah, thanks!13:32
opendevreviewMatt Crees proposed openstack/kolla-ansible master: Enable RabbitMQ HA queues by default  https://review.opendev.org/c/openstack/kolla-ansible/+/88282513:33
opendevreviewMatt Crees proposed openstack/kolla-ansible master: Enable RabbitMQ HA queues by default  https://review.opendev.org/c/openstack/kolla-ansible/+/88282513:48
opendevreviewMerged openstack/kayobe master: CI: fix molecule job  https://review.opendev.org/c/openstack/kayobe/+/88282013:50
opendevreviewMatt Crees proposed openstack/kolla-ansible master: Enable RabbitMQ HA queues by default  https://review.opendev.org/c/openstack/kolla-ansible/+/88282513:50
opendevreviewMerged openstack/kolla stable/wallaby: fluentd: cap elasticsearch again  https://review.opendev.org/c/openstack/kolla/+/88277613:53
mattcreesOn the topic of RabbitMQ, there's also a backport chain still ongoing that would be nice to get merged: https://review.opendev.org/c/openstack/kolla-ansible/+/87683014:01
SvenKieskemattcrees: as already said, either mnasiadka or frickler (on vacation) need to +2 them and start gating, I don't hold such powers for now, sorry. I could ping my boss if it's really urgent.14:07
mattcreesNo worries, the urgent ones were for the precheck, which mnasiadka has set off to merge now. These others can wait for frickler to get back, they're only neccessary to faciliate disabling RMQ HA when it was previously enabled14:10
opendevreviewMaksim Malchuk proposed openstack/kayobe master: CI: Fix kayobe-seed-images-* jobs  https://review.opendev.org/c/openstack/kayobe/+/88167914:17
opendevreviewMaksim Malchuk proposed openstack/kayobe master: Add custom user_data content configuration for Bifrost  https://review.opendev.org/c/openstack/kayobe/+/87955414:18
opendevreviewMaksim Malchuk proposed openstack/kayobe master: Add custom user_data content configuration for Bifrost  https://review.opendev.org/c/openstack/kayobe/+/87955414:18
opendevreviewMerged openstack/kolla-ansible stable/yoga: Add precheck to fail if RabbitMQ HA needs configuring  https://review.opendev.org/c/openstack/kolla-ansible/+/88084114:55
opendevreviewMerged openstack/kolla-ansible stable/zed: Add precheck to fail if RabbitMQ HA needs configuring  https://review.opendev.org/c/openstack/kolla-ansible/+/88084014:55
opendevreviewMerged openstack/kolla-ansible stable/xena: Add precheck to fail if RabbitMQ HA needs configuring  https://review.opendev.org/c/openstack/kolla-ansible/+/88084215:00
opendevreviewMaksim Malchuk proposed openstack/kayobe master: Add custom user_data content configuration for Bifrost  https://review.opendev.org/c/openstack/kayobe/+/87955415:30
atmarkhow often the kolla images in dockerhub are built?16:34
mnasiadkain theory daily, unless a publish job fails16:47
mnasiadkaah, dockerhub16:47
mnasiadkanot really, weekly16:47
mnasiadkaand we're rather going to drop the dockerhub repo16:48
mnasiadkaquay.io is daily16:48
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: Revert "Revert "ansible: bump min to 2.13 and max to 2.14""  https://review.opendev.org/c/openstack/kolla-ansible/+/88101816:51
supamattmnasiadka: I'm hitting this bug, https://launchpad.net/bugs/187522318:20
supamattand I saw you had a change to fix this issue, https://review.opendev.org/c/openstack/kolla-ansible/+/86892918:20
supamattDo I need to define the leader in the ovn-nb-db_leader inventory group? 18:20
opendevreviewsean mooney proposed openstack/kolla-ansible master: always add service_user section to nova.conf  https://review.opendev.org/c/openstack/kolla-ansible/+/88289320:05
sean-k-mooneyhi folks o/ just in case you are not aware the embargo on https://bugs.launchpad.net/nova/+bug/2004555 lifted eairlier today. when the fix merges in nova it will break the ablity for kolla to deploy nova20:06
sean-k-mooneyas of the cve fix the [service_user] section will become requried for nova to work20:07
sean-k-mooneyit has been recommended but optional for years but now it will be a requriement20:07
sean-k-mooneyi put togheter a minimal patch above but i have not tested it20:07
sean-k-mooneythat shoudl be sufficent to deploy nova but it likely need a release note and it will need to be backported to all releases where the cve is resloved. it looks like the plan is to backport it to wallaby 20:09
mmalchuk_embargo? lol20:09
*** mmalchuk_ is now known as mmalchuk20:09
sean-k-mooneycan someone form the kolla team take over that patch. i might have time to update it but likely not20:09
sean-k-mooneymmalchuk: ya the CVE was embargoed until eiarler today. i was not read in on it so just leaned of it when it was made public20:10
mmalchuksean-k-mooney I'm on #openstack-security jfyi20:11
sean-k-mooneyoh ok cool20:11
sean-k-mooneyso all the installer tools that dont already use the [service_user] will need to be updated20:12
mmalchuklets wait for CI and wait for rewivers20:12
sean-k-mooneyoh ya no rush the nova patch is not meged yet20:12
sean-k-mooneythere isnt an issue in kolla20:12
sean-k-mooneyit just kolla will need to be adapted to ensure the reslvent config section is preent for nova to work20:13
sean-k-mooneyi use kolla at home so just decied to get a minimal patch up before i finish for the day20:13
mmalchukpass -1 until it merged and CI passed20:14
sean-k-mooneywell the code change does not depend on the nova fix20:14
sean-k-mooneythe servie user has existed and is used for other things for many years20:14
sean-k-mooneyi was suprised that kolla was not already using it20:15
sean-k-mooneybut sure no rush20:15
mmalchuksure, but openstack itself does20:15
sean-k-mooneyanyway food arrieved so chat to ye later o/20:15
mmalchukI'v added PTL in review20:15
mmalchuktoo20:15
mmalchukthanks a lot20:16
mmalchukfolks, please review: https://review.opendev.org/c/openstack/kolla-ansible/+/88146920:20
mmalchukthis need to backported asasp20:21
mmalchukasap*20:21

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!