Wednesday, 2021-09-29

opendevreviewjayonlau proposed openstack/kolla master: venus: add log management system  https://review.opendev.org/c/openstack/kolla/+/79379501:17
opendevreviewPierre Riteau proposed openstack/kayobe master: Bump up Ansible supported versions  https://review.opendev.org/c/openstack/kayobe/+/81141005:54
opendevreviewPierre Riteau proposed openstack/kayobe master: Bump up Ansible supported versions  https://review.opendev.org/c/openstack/kayobe/+/81141005:57
jinyuanliu__https://review.opendev.org/c/openstack/kolla/+/79379506:00
*** amoralej|off is now known as amoralej06:13
opendevreviewconna proposed openstack/kolla-ansible master: Add skyline role  https://review.opendev.org/c/openstack/kolla-ansible/+/81056607:05
opendevreviewPierre Riteau proposed openstack/kayobe master: CI: Disable libvirt debug logs  https://review.opendev.org/c/openstack/kayobe/+/81156207:54
kevkomorning \o/08:01
opendevreviewPierre Riteau proposed openstack/kayobe master: Ubuntu: add upgrade jobs in CI  https://review.opendev.org/c/openstack/kayobe/+/79762608:03
opendevreviewMichal Nasiadka proposed openstack/kayobe master: seed: Disable SNAT configuration by default  https://review.opendev.org/c/openstack/kayobe/+/80268408:32
opendevreviewMichal Nasiadka proposed openstack/kayobe master: seed: Disable SNAT configuration by default  https://review.opendev.org/c/openstack/kayobe/+/80268409:01
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add skyline role  https://review.opendev.org/c/openstack/kolla-ansible/+/81056609:24
opendevreviewMerged openstack/kayobe master: CI: Disable libvirt debug logs  https://review.opendev.org/c/openstack/kayobe/+/81156211:54
opendevreviewMark Goddard proposed openstack/kayobe stable/wallaby: CI: Disable libvirt debug logs  https://review.opendev.org/c/openstack/kayobe/+/81150011:55
opendevreviewMark Goddard proposed openstack/kayobe stable/victoria: CI: Disable libvirt debug logs  https://review.opendev.org/c/openstack/kayobe/+/81150111:55
opendevreviewMark Goddard proposed openstack/kayobe stable/ussuri: CI: Disable libvirt debug logs  https://review.opendev.org/c/openstack/kayobe/+/81150211:55
opendevreviewMark Goddard proposed openstack/kayobe stable/ussuri: CI: Disable libvirt debug logs  https://review.opendev.org/c/openstack/kayobe/+/81150211:56
opendevreviewMark Goddard proposed openstack/kayobe stable/ussuri: CI: Disable libvirt debug logs  https://review.opendev.org/c/openstack/kayobe/+/81150211:59
*** amoralej is now known as amoralej|lunch12:17
*** amoralej|lunch is now known as amoralej13:09
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: CI: Fail on fluentd log parsing errors  https://review.opendev.org/c/openstack/kolla-ansible/+/74239913:33
jingvarHi!14:22
jingvarI faced with undertandable merge. etc/kayobe/kolla/globals.yml:neutron_plugin_agent: "ovn" , but etc/kolla/globals.yml has 2 records  neutron_plugin_agent: "openvswitch" and neutron_plugin_agent: ovn14:24
jingvarIs in OK?14:25
priteauI think if the one you want is defined second, that's ok14:36
priteauBut you could cherry-pick this patch to avoid: https://review.opendev.org/c/openstack/kayobe/+/79707214:36
mnasiadkamgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak parallax Fl1nt - meeting in 514:54
yoctozeptooh noez14:56
mnasiadkaWe can skip if you prefer ;-)14:58
mnasiadka#startmeeting kolla15:00
opendevmeetMeeting started Wed Sep 29 15:00:48 2021 UTC and is due to finish in 60 minutes.  The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'kolla'15:00
mnasiadka#topic rollcall15:00
headphoneJameso/15:01
yoctozeptoo/15:01
hrw[°][o][o]15:01
mnasiadka\o15:01
yoctozeptomnasiadka: it was about the time running too fast15:02
mgoddard\o15:02
priteauo/15:02
mnasiadkaok then15:04
mnasiadka#topic agenda15:04
mnasiadka* Review action items from the last meeting15:04
mnasiadka* CI status15:04
mnasiadka* Discuss switching default image type to source15:04
mnasiadka* What do we do with Monasca?15:04
mnasiadka* Release tasks15:04
mnasiadka* Xena cycle planning15:04
mnasiadka* Yoga PTG planning15:04
mnasiadka* Open discussion15:04
mnasiadka#topic Review action items from the last meeting15:04
mnasiadkaI've done all that were assigned to me, yoctozepto posted his change about using upstream MariaDB repos in CentOS and syncing version to what Deb/Ubuntu uses.15:05
mnasiadka#topic CI status15:05
mnasiadkaHow's the CI? green?15:05
yoctozeptogreen15:06
yoctozeptokolla STRONK15:06
yoctozepto:D15:06
mnasiadkaWhiteboard says Kayobe is amber, priteau I guess after all those changes it should be green again?15:06
priteauI believe Kayobe is mostly green15:06
mnasiadkacan we update the whiteboard to green and revisit once we see failures again?15:07
priteauUpdating whiteboard15:07
priteaudone15:07
mnasiadkaDo we really have Kayobe Queens as EM?15:08
mnasiadkaKolla has been marked EOL for Queens15:08
mnasiadkaL139 in the whiteboard15:08
priteauNeeds update15:08
mgoddardkayobe queens was unofficial IIRC15:09
mnasiadkaI think the first official was Train?15:09
mnasiadkaAnyway, let's remove it from there and done.15:10
mnasiadkaor yes, mark as unmaintained.15:10
mnasiadkaLet's move on.15:10
mnasiadka#topic Discuss switching default image type to source15:10
yoctozeptodo we want to track EM for kolla and k-a?15:10
yoctozepto(sorry for late question)15:11
yoctozeptoI know it's broken btw15:11
mnasiadkaQuestion if we're planning to fix it, or not.15:11
yoctozeptome not15:11
yoctozeptohrw not15:11
yoctozeptoquestion whether mnasiadka and/or mgoddard15:11
yoctozepto;-)15:11
mnasiadkamgoddard: do we?15:11
mnasiadkaI guess if Kolla is broken, then it impacts Kayobe15:12
mgoddardwe're talking about queens still?15:12
priteauI think EM in general so Train?15:13
yoctozeptotrain15:13
yoctozeptotrain is b0rken15:13
mnasiadkano, queens is eol15:13
mnasiadkarocky/stein/train I guess15:13
priteauAnd older15:13
yoctozeptoand older is broken possibly too15:13
mgoddardkayobe should track kolla15:13
priteauWhat broke train?15:13
yoctozeptosetuptools hell15:13
yoctozeptoplus some other issues15:13
priteauAh yes15:13
yoctozeptohrw may know more15:14
hrwkolla/train is broken, EM, resting etc15:14
mgoddardwe'll probably end up fixing it at some point15:14
hrwneither centos7 nor centos8 are buildable and it is beyond us to fix it15:14
mgoddardthere's always a way :)15:14
hrwI refuse to work on train15:14
mnasiadkaUsing the opportunity - do we want to EOL Rocky/Stein? or leave them in EM for now?15:15
mgoddardpeople are still using train15:15
mgoddardnobody's asking you to work on train hrw 15:15
priteauI have an upgrade to train in 10 days :D15:15
mnasiadkapriteau: so you have an opportunity to fix it ;-)15:15
priteauyay15:15
hrwmgoddard: it was to say 'in my opinion train is broken too far to be fixable'15:16
yoctozeptoit's always fixable15:16
yoctozeptobut perhaps in pain15:16
mnasiadkaOk, seems we as a company will work on it in some time - let's leave it if it's fixable or not.15:17
mnasiadkaAnd we've diverted off topic already.15:17
hrwmnasiadka: then check my 'use pip 20' patch in gerrit. will help you get CI working at all15:17
mnasiadkahrw: thank you15:18
mnasiadkaok, let's go back to "Discuss switching default image type to source"15:18
hrwI abandoned it15:18
hrw+1 for source by default15:18
opendevreviewPierre Riteau proposed openstack/kayobe master: Bump up Ansible supported versions  https://review.opendev.org/c/openstack/kayobe/+/81141015:18
mnasiadkamgoddard: proposed that as a topic for PTG, but I feel we could try switching in this cycle (although it's late).15:18
mnasiadkaOpinions?15:19
mgoddardpro: pretty low risk15:19
mgoddardcon: pretty high impact15:19
opendevreviewRadosław Piliszek proposed openstack/kolla-ansible master: Do not set net.ipv4.ip_forward sysctl  https://review.opendev.org/c/openstack/kolla-ansible/+/80997715:20
mgoddardbut it's not the kind of change that needs time to settle down, so I guess we could make a late change15:20
yoctozepto++15:20
yoctozeptolet's switch it15:20
mnasiadkaI don't think if we would make this change early in the cycle, it would make people to read the renos better.15:20
yoctozeptothis will send the right message15:20
yoctozeptothe current default sends a wrong one ;d15:20
mnasiadkaFor visibility - we can send a mail to the openstack-discuss ML15:20
mnasiadkaif that helps in anything these days...15:20
mnasiadkaok then, anybody against switching?15:21
mnasiadkaseems not, then agreed15:22
mgoddardwho will do it?15:22
mnasiadkaIs there a volunteer to post patch(es)?15:22
mnasiadkaI guess we need one in Kolla-Ansible and a second in Kolla (and a third one in Kayobe?)15:22
mgoddardI can do it15:24
mnasiadkaOk then, if you'll lack time - just shout.15:24
mnasiadka#agreed to switch default image type to source, mgoddard to post patches to do it15:25
mnasiadkaNext topic15:25
mnasiadka#topic What do we do with Monasca?15:25
yoctozeptoI meant mostly CI job in here15:25
yoctozeptobut yeah, what do we do15:25
mnasiadkaCurrently I think the log api is broken, from what I heard probably since Wallaby - but I may be mistaken.15:25
mnasiadkaAnd I think users that decide to deploy Monasca - think of it as the obvious choice.15:26
yoctozeptook, I can hide it behind the extra flag we planned15:26
yoctozeptoand also, we drop the CI job?15:26
mnasiadkaAs already commented in mgoddard's patch - I think we need to at least make it visible for users, that it's broken now.15:27
mnasiadkaAnd if nobody fixes it in Yoga... then we need to analyze what to do.15:27
mnasiadkamgoddard: should we drop the CI job? I think monasca-thresh also plays a role in defining alerts for thresholds - so that makes Monasca a bit crippled?15:29
mgoddardmonasca thresh works15:29
mgoddardwell15:29
yoctozeptowhat does not work? only the log api?15:30
mgoddardcorrect15:30
mgoddardAFAIK15:30
yoctozeptook15:30
yoctozeptoack, fair15:30
mgoddardthresh is just a bit weird - it got fixed recently to submit jobs to storm rather than processing them locally15:30
yoctozeptook /me to propose the magic flag, we can update the docs accordingly15:30
yoctozeptocould we then allow to reroute logs to its central logging path?15:31
mgoddardthe implementation ended up with a container in the 'exited' state15:31
yoctozeptodespite monasca being enabled?15:31
mnasiadkaah, so we ,,broke'' thresh?15:31
mgoddardI didn't particularly like it, but it works15:31
mgoddardno, as I said - thresh works15:31
mgoddardit just now shows as 'exited'15:31
mnasiadkabut it's in exited state?15:31
mgoddardwhich failed the container check script15:32
mgoddardso I added a special case15:32
mgoddardit's now a 'one shot'15:32
yoctozeptowhat does it do in there?15:32
mnasiadkaone shot container?15:32
* yoctozepto knows nothing15:32
mnasiadkaI'm also curious to understand this case :)15:32
yoctozeptothat makes the two of us15:33
mgoddardit pushes a job to storm, then exits15:33
mgoddardthe container is kept around to allow the kolla container diff magic to work15:34
mgoddardso a new image updates the job15:34
mgoddardit's a bit unusual, but apparently works15:34
mnasiadkacan we just make it sleep or something, so it's not in exited state?15:35
mgoddardcan we just ignore it and move on? :)15:35
yoctozeptowell, oneshot makes sense15:35
yoctozeptoyeah, I think let's not split hair on this15:35
mnasiadkaOk, let's then ignore it.15:36
yoctozeptoso15:36
yoctozeptothe plan is15:36
yoctozeptodocument the issue15:36
yoctozeptoand hide monasca15:36
mgoddardI suggested a different approach,but the author had already agreed on this one and didn't want to go back15:36
yoctozeptowhat about the log rerouting?15:36
mgoddardcentral logging can still work with monasca IIRC15:36
yoctozeptocould you check it?15:37
mgoddardmonasca_ingest_control_plane_logs15:37
mnasiadkaIs that documented in Monasca guide?15:37
mgoddardyes15:37
mgoddardhttps://docs.openstack.org/kolla-ansible/latest/reference/logging-and-monitoring/monasca-guide.html15:37
yoctozeptoyeah, but we need to drop a big warning now ;d15:37
mnasiadkaso, we drop a note/warning before "you can optionally bypass Monasca for logs"15:38
mnasiadkadoes it make any sense?15:38
yoctozeptomhm15:40
yoctozeptoyou do the docs then15:40
mnasiadkaMe? I have no clue about Monasca.15:40
mnasiadkamgoddard: keen to update the docs?15:40
mgoddardok15:41
opendevreviewMark Goddard proposed openstack/kolla master: Build source type images by default  https://review.opendev.org/c/openstack/kolla/+/81174315:41
mnasiadka#action mgoddard to update Monasca docs around using central logging, due to Monasca log problems15:43
mnasiadkayoctozepto: what's your action? :)15:43
yoctozeptomnasiadka: the old one still current :D15:44
mnasiadkaOk :)15:44
mnasiadkaLet's go further... and forget about Monasca ;)15:44
yoctozeptoadd the flag to hide the "bad quality" services15:44
mnasiadka#topic Release tasks15:44
yoctozeptook15:44
yoctozeptolet's roll15:44
mnasiadkaHmm, we wrote in docs that R-2 is feature freeze week for Kolla15:45
mnasiadkaR-2 was last week15:45
yoctozeptooh-my-openstack15:46
mnasiadkaSo I assume all changes that are not marked as priorities on the Kolla Whiteboard - are frozen, if needed we need to set Review Priority on those15:46
yoctozeptoall feature changes ;-)15:47
mnasiadkaAnd let's focus on merging what's left this week15:47
headphoneJamesany chance we could get https://review.opendev.org/c/openstack/kolla-ansible/+/692179 in? Or does that now get pushed to Yoga?15:47
mnasiadkabecause we marked R-1 as prepare to RC1 and stable branch creation15:47
yoctozeptoyeah, I guess we could try to better stay on track this cycle15:48
yoctozeptoas even debian has already released lol15:48
mgoddarddo we need to update cycle highlights?15:48
mgoddardI think we just had two initially15:48
mnasiadkaYes, I guess so.15:48
yoctozeptotbh, I don't think many people read those (if you have better things to do ;-) )15:49
mnasiadkaSo let's mark it as a low priority item.15:49
mgoddardI think they feed into marketing15:49
mgoddardand foundation staff sometimes cherry pick them15:49
yoctozeptothey do15:50
mnasiadkaI understand that we have some changes left for review, that we'd like to merge this week - and we could cut stable branches and RC1 somewhere next week? Or what's the plan?15:50
mgoddardwhich is how prometheus v2 got to the top of some wallaby presentation15:50
yoctozeptoit did15:50
priteauAlso here: https://www.openstack.org/software/wallaby/15:50
yoctozeptoalso because of cncf cooperation15:50
mgoddardheadphoneJames: I think it's ready, I'll do a final pass through15:51
mnasiadkaOk, we have 7 minutes left.15:53
mnasiadkaDo we agree to work on reviewing existing patches from the priority list on the whiteboard until next meeting, and then agree if we're cutting stable branches?15:54
yoctozeptomakes sense to me15:55
yoctozeptoplus the fixes I'm pouring in15:55
yoctozeptopretty please :-)15:55
mnasiadkaobviously15:55
mgoddardwell15:55
mgoddardwhen is actual feature freeze?15:56
mgoddardnext week?15:56
mnasiadkathe one we defined in docs was last week15:56
mnasiadkaR-2 Sep 20 - Sep 2415:56
priteauMy bad, I thought it was freeze at OpenStack release date15:56
mnasiadkaAccording to Xena Release calendar - https://releases.openstack.org/xena/schedule.html15:57
mgoddardyeah, sounds correct15:57
mnasiadkaWe just missed that, I was sure it's this week.15:58
mgoddardwhich features are we hoping to merge before next week?15:58
opendevreviewMark Goddard proposed openstack/kolla-ansible master: Deploy source type images by default  https://review.opendev.org/c/openstack/kolla-ansible/+/81174415:58
mnasiadkaSo, keystone system scope is one15:59
mnasiadkaCeph RADOS GW looks like in good shape if yoctozepto agrees15:59
mgoddardceph RGW pls15:59
mnasiadkaand I guess we should also remove chrony role?15:59
yoctozeptodid not we merge ceph rgw?15:59
mgoddardnot yet16:00
mnasiadkano, you complained ;-)16:00
yoctozeptook, will look16:00
mgoddardremove chrony pls16:00
yoctozeptooh, sorry16:00
yoctozeptoyes, that one too16:00
yoctozeptocould someone write it down in one place on whiteboard16:00
yoctozeptosingle list16:00
mgoddardhttps://review.opendev.org/c/openstack/kolla-ansible/+/79174316:00
yoctozeptoone after the other16:00
mgoddardwe have a list already16:00
yoctozeptoah16:00
mgoddardcould just add DEFERRED to some16:00
yoctozeptook16:00
mgoddardswift won't make it16:01
mgoddardtoo late for a change that big IMO16:01
yoctozeptoyeah, I don't think it's worth it either16:01
yoctozeptoit's just pain to keep it in bad shape16:01
yoctozeptobut let's release without hassle16:01
mnasiadkaI just added DEFERRED to the rest16:01
mnasiadkaProxySQL probably also too big to get anywhere this week16:02
mgoddardkayobe has a few with 1x +216:02
mnasiadkaok, I guess I need to stop the meeting, and I'll go through the Kayobe list and review those that I haven't.16:03
mnasiadka#stopmeeting kolla16:04
mnasiadka#endmeeting kolla16:04
opendevmeetMeeting ended Wed Sep 29 16:04:11 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:04
opendevmeetMinutes:        https://meetings.opendev.org/meetings/kolla/2021/kolla.2021-09-29-15.00.html16:04
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/kolla/2021/kolla.2021-09-29-15.00.txt16:04
opendevmeetLog:            https://meetings.opendev.org/meetings/kolla/2021/kolla.2021-09-29-15.00.log.html16:04
mnasiadkaugh16:04
mnasiadkaThanks for attending.16:04
mnasiadkaLet's merge what's possible and cut RC's next week.16:04
yoctozeptomnasiadka: ceph rgw - gone over the comments; 2x needs reply; 1x unresolved16:05
mnasiadkayoctozepto: tell that to mgoddard :)16:05
yoctozeptomnasiadka: also, endmeeting does not need argument16:05
yoctozeptomgoddard: I'm telling you that16:05
mnasiadkayoctozepto: oh, good to know16:05
yoctozeptomnasiadka: and sorry for confusing names16:05
mgoddardyoctozepto: seems I had some draft comments16:07
yoctozeptomgoddard: ok; then only 1x unresolved16:09
yoctozeptoand we merge it16:09
yoctozeptoyou can just say "yes" there (in override) and it just works without touching16:10
yoctozeptomgoddard: ^16:10
mgoddardyoctozepto: more context?16:11
mgoddardyoctozepto: leave a comment, I'll pick it up tomorrow16:12
yoctozeptomgoddard: https://review.opendev.org/c/openstack/kolla-ansible/+/776662/16/tests/templates/globals-default.j2#13616:12
yoctozeptoack16:12
yoctozeptothe comment is there twice already16:12
opendevreviewMark Goddard proposed openstack/kolla-ansible master: Add support for Ceph RadosGW integration  https://review.opendev.org/c/openstack/kolla-ansible/+/77666216:13
mgoddardhmm, missed it16:14
mgoddardyoctozepto: fixed - should be wallaby16:14
yoctozeptomgoddard: yeah, but, as I said, this is mostly irrelevant16:16
yoctozepto:D16:16
opendevreviewPierre Riteau proposed openstack/kayobe master: Import merge_configs and merge_yaml from Kolla Ansible  https://review.opendev.org/c/openstack/kayobe/+/77899416:37
*** amoralej is now known as amoralej|off16:38
opendevreviewPierre Riteau proposed openstack/kayobe master: seed: Disable SNAT configuration by default  https://review.opendev.org/c/openstack/kayobe/+/80268416:46
jingvarpriteau: I used https://review.opendev.org/c/openstack/kayobe/+/797072 but , OVN docs says type_drivers = local,flat,vlan,geneve16:49
jingvarBus is starange that openstack network create my-local-net  -> Network type vxlan is not supported.16:51
jingvarWith your patch I have neutron-server/ml2_conf.ini:tenant_network_types = flat,vlan,vxlan , type_drivers = flat,vlan,vxlan16:53
jingvarvxlan is present 16:53
opendevreviewPierre Riteau proposed openstack/kayobe master: Bump up Ansible supported versions  https://review.opendev.org/c/openstack/kayobe/+/81141016:56
priteaujingvar: you need to override tenant_network_types with custom config, see the discussion on the patch16:57
jingvarpriteau: thanks16:59
opendevreviewPierre Riteau proposed openstack/kayobe master: Bump up Ansible supported versions  https://review.opendev.org/c/openstack/kayobe/+/81141017:07
opendevreviewMerged openstack/kayobe master: Make setup module arguments configurable  https://review.opendev.org/c/openstack/kayobe/+/79197617:56
opendevreviewMerged openstack/kayobe master: Support Ansible collections  https://review.opendev.org/c/openstack/kayobe/+/79749718:19
opendevreviewPierre Riteau proposed openstack/kayobe master: Bump up Ansible supported versions  https://review.opendev.org/c/openstack/kayobe/+/81141020:31
yuriysI have heard rumors that kolla may be migrating to Quay and was wondering if I could get some insight on that from here21:03
yuriysfrom Dockerhub to Quay that is.21:03

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!