Tuesday, 2024-12-10

opendevreviewGhanshyam proposed openstack/election master: Fix linters error on noble  https://review.opendev.org/c/openstack/election/+/93741900:53
opendevreviewGhanshyam proposed openstack/election master: Add TC liaison in election page  https://review.opendev.org/c/openstack/election/+/93741200:54
opendevreviewGhanshyam proposed openstack/election master: Add TC liaison in election page  https://review.opendev.org/c/openstack/election/+/93741200:54
*** diablo_rojo_phone is now known as Guest263908:38
*** ralonsoh_ is now known as ralonsoh09:21
fungifinal openinfra foundation board of directors meeting of 2024 starts in 19 minutes: https://board.openinfra.dev/meetings/2024-12-1014:41
gouthamrtc-members: gentle reminder that we will be meeting here for our weekly meeting in ~55 minutes 17:05
spotz[m]Will need post back to back meeting coffee run to the kitchen if late:)17:07
cardoeonly 1 back to back meeting for you? lucky! My days have become 8am to 2pm straight on Zoom. The TC meeting is my one respite from Zoom.17:09
fungisounds like a tuesday to me17:10
gouthamr#startmeeting tc18:01
opendevmeetMeeting started Tue Dec 10 18:01:26 2024 UTC and is due to finish in 60 minutes.  The chair is gouthamr. Information about MeetBot at http://wiki.debian.org/MeetBot.18:01
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.18:01
opendevmeetThe meeting name has been set to 'tc'18:01
gouthamrWelcome to the weekly meeting of the OpenStack Technical Committee. A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct.18:02
gouthamrToday's meeting agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee18:02
gouthamr#topic Roll Call18:02
cardoeo/18:02
slaweqo/18:02
frickler\o18:02
gtemao/18:02
noonedeadpunkO/18:02
gmanno/18:02
fricklerbauzas might still be doing kitchen design ;)18:03
gouthamrcourtesy-ping: bauzas, spotz[m] probably grabbing coffee18:03
spotz[m]Thanks!18:04
gouthamrthank you for joining, lets get started.. 18:04
opendevreviewMerged openstack/election master: Fix linters error on noble  https://review.opendev.org/c/openstack/election/+/93741918:04
gouthamr#topic Last Week's AIs18:04
gouthamrin the gate health topic, we brought up noble migration having affected horizon plugin CI jobs18:05
gouthamrwe didn't take an AI though.. but, was there anything note worthy that changed the situation here?18:06
gmannI am not aware of that bug but we have separate topic for noble things, maybe we can discuss there.18:06
gouthamrack18:06
gouthamrwe got a early-heads-up regarding CPU requirements for CentOS 10 that aren't available in many providers18:07
gouthamr#link https://review.opendev.org/c/openstack/diskimage-builder/+/93404518:07
frickleror rather not at all in any of our providers?18:07
gouthamroh, thought we had RAX-FLEX and OpenMetal 18:08
gouthamrbut i see Karolina's post on that review: Available infrastructure has disabled AVX and AVX2 CPU flags (even if it's Haswell), which effectively disabling building Centos 10 which is compiled for x86_64-v3.18:08
fungiwe think they might be on in rackspace flex, and if they're not in openmetal we're probably able to set that ourselves through kolla18:08
gmanntime to have new provider who need  CentOS18:08
fungibut neither of those providers currently provide many test nodes for us18:09
gmanndo we know how many project test CentOS as voting job? 18:09
slaweqin Neutron we have some in periodic queue only18:09
slaweqbut I think we had FIPs jobs using Centos, right?18:10
gmannlast time when we discussed about its stability most of project moved it to non voting or periodic 18:10
slaweqand there was some problems with running those on Ubuntu IIRC18:10
gmannslaweq: I think yes or they are running on ubuntu focla?18:10
gmannfocal18:10
fungihard to measure that empirically, but yes the fips compliance testing effort was driving some increase/resurgence at one point in recent history18:10
frickleralso, slightly related: https://www.phoronix.com/news/Torvalds-Mind-Fart-x86_64-Level18:10
gouthamrbreathe, repeat: "we suck at naming"18:12
clarkbmy perspective on it is that centos/rhel should be working with cloud providers to uplevel the available cpus18:13
clarkbit really isn't openstack's responsibility to be mediator here18:13
gouthamrokay; i don't think we had any AIs here.. and it looks like RH folks working on this are aware, but i don't see any call to attention from them.. we can revisit this issue if they need us to do anything18:13
gouthamrclarkb: ack18:13
gouthamrthe last AI i was tracking was asking the ML for objections for EOL'ing Victoria (V), Wallaby (W), and Xena (X) branches18:14
cardoeI agree with clarkb here. You might have to end up disabling old Rackspace.18:14
clarkbthen if they decide like openshift to make it impossible to test their platform we'll just stop testing their platform18:14
fungicardoe: old rackspace and ovh provide the bulk of our test resources today (and neither support the newer instructions)18:15
gouthamri own that last AI, and haven't finished my ML post.. will catch up on it after this meeting 18:15
fricklerjust fyi elodilles has claimed multiple times to want to keep those branches alive18:16
gouthamrthat's all the AIs i was tracking, was there anything else you were working on?18:16
gmannI think all unmaintained branches except latest one needs to be opt-in explicitly right? otherwise by default they move to EOL18:16
gouthamrfrickler: ack, and he does a bulk of the job reviewing/maintaining them.. but, like you noted a ton of projects have not merged even the .gitreview updates18:16
spotz[m]After speaking some with thejulia on this it seems that V3 has been around for a long time and folks just aren't implementing it18:17
gmannis there any change in unmaintained policy than what is written?18:17
fricklergmann: yes, but what happens like in this case when someone says "I opt in", but they don't fulfill the written requirements?18:17
gouthamrokay; lets switch to our real topics so we can have the discussion in the right place18:17
gouthamr#topic Maintaining Unmaintained branches ¯\_(ツ)_/¯18:18
* gouthamr continue please18:18
frickler:)18:18
gmannso if they do not fulfil the requirement that means we  should not approve the opt-in for them18:18
gouthamryes, we had a default/global opt-in at the beginning18:18
gmannwhen anyone want to opt-in, we need to judge their maintainability and whether they fill the required things to stay as unmaintianed18:19
fricklerso how do we do that? we don't have a process to handle this opt-in and alternative eoling18:19
fungia driver for the opn-in caretaking of unmaintained branches was to keep the list of them from growing unbounded, since it puts a strain on opendev to continue, e.g., running periodic jobs for an ever-increasing number of them, especially when most of those jobs have been perpetually failing for years and don't get looked at18:20
gmanngouthamr: not at beginning but after any branch is in unmaintained and that is not latest unmaintained then it goes to EOL by default and should be opt-in explicitly to stay in unmaintianed state 18:20
fungis/opn-in/opt-in/18:20
gmannfrickler: I think we documented somewhere the implementation 18:20
gmannunmaintained branches will be moved to EOL and anyone want to keep them in unmaintained needs to -1 there18:21
gouthamr#link https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branches.html#unmaintained-branches (unmaintained branches resolution)18:21
gmannand then we check the requirement to keep it in unmaitined or not18:21
fungigmann: there was a call for volunteers that went out to the openstack-discuss ml a while after the clarifications to unmaintained caretaker groups in gerrit were merged18:21
gouthamrand the stable branch policy here:18:21
gouthamr#link https://docs.openstack.org/project-team-guide/stable-branches.html#unmaintained (Stable Maintenance)18:21
fricklerso I'm not aware of any tooling that implements what is written in the policy there18:21
fungibut i don't think anyone ever got a clear process documented for how people express their desire to volunteer for those, and how the periodic re-opt-in is tracked for them18:22
fungiand i've definitely not been seeing calls anywhere for the caretakers to re-opt-in for existing unmaintained branches twice yearly18:23
frickleralso this will be a lot of review work for the release team if this is to happen individually for every repo18:23
gmannits there in p-t-g18:23
gmann"By default, only the latest eligible Unmaintained branch is kept open. To prevent an Unmaintained branch from automatically transitioning to End of Life once a newer eligible branch enters the status, the Unmaintained branch liaison must manually opt-in as described below for each branch."18:23
gmannhere18:23
gmann#link https://docs.openstack.org/project-team-guide/stable-branches.html#unmaintained18:23
fricklerthat's policy, not implementation18:23
gmann"To opt-in to keep the Unmaintained branch open, the PTL or Unmaintained liaison must -1 the appropriate patch in the openstack/releases repo to EOL the branch. "18:23
fricklerthe policy fine, but it needs to actually get applied18:23
gmannfrickler: yeah, I think we should implement that in this transition 18:24
fungiwho proposes the changes to close the branches?18:24
fricklercurrently nobody18:24
fungii suppose that's supposed to happen immediately after each new coordinated release18:25
gmannI think its release team to move all V-Z branches to EOL and let volunteer to -1 those and we check the requirement18:25
* gouthamr spies a change on release tooling18:25
gouthamr#link https://review.opendev.org/c/openstack/releases/+/936637 (Update tooling to delete unmaintained branches)18:25
fricklergouthamr: that's only to actually delete branches that were marked eol in the release data18:25
gmannbut I might be wrong if its not release team doing or want to do. that is not actually discussed previously or in policy 18:26
gmannthat is open thing I think who will do it18:26
cardoeIts seemed a bit inconsistently recently18:26
fricklerrelease team doesn't have the capacity I'd think18:26
gouthamrfrickler: ah, ack - although the script you're adding will do the bulk abandons if we could re-use it in this context18:26
gmannfrickler: yeah i can understand 18:27
fricklerI started to create some eol patches semi-manually for those repos that didn't merge their .gitreview updates18:27
fricklerthat lead to https://review.opendev.org/q/topic:%22eom-to-eol%22 were some repos were opted out of eol-ing, so quasi opted-in18:27
gmannI will say just force retire/EOL them as nobody wanted those as .gitreview change is not merged18:28
gouthamr#link https://review.opendev.org/c/openstack/releases/+/935373 (Transition victoria-eom branches to EOL)18:28
gouthamr#link https://review.opendev.org/c/openstack/releases/+/935374 (Transition wallaby-eom branches to EOL)18:28
gouthamr#link https://review.opendev.org/c/openstack/releases/+/935375  (Transition xena-eom branches to EOL)18:28
fricklerbut those still don't fulfill the "CI in good shape"18:28
gouthamr^ will stop there because, during the last meeting, we did have some buy in to keep yoga/zed "unmaintained"18:29
fricklerand then there are more with zuul config errors, which I would like to tackle next18:29
gmannwhole point of unmaintained was to shrink and filter out the *not maintianed branches* to go out but if we want to keep unlimited unmaintained branches then we are stuck in same situation what we were in EM model18:29
fricklerso how would we do this? have the TC do "executive overrides" against the -1 on the eol release patches?18:30
gmannthat is why we should default to EOL and let volutneer to come and tell us if they need it and want t maintian18:30
slaweqgmann++18:30
gmannI think yes, we ask volunteer who is -1 to tell us the plan 1. gate green or not 2. testing requirement 3. maintainers list etc18:31
fricklerwho wants to implement the "default to EOL" tooling? otherwise this isn't happening18:31
gmannyeah so issue here is we do not have bandwidth to implement the policy which set us back to original problem we faced during EM18:32
fungii guess one change to eol all of e.g. unmaintained/xena branches in every repository in the openstack namespace, and then if someone wants to volunteer to be the caretaker for unmaintained/xena on specific repositories those have to be revised out of the change? i don't see opening 500 separate changes as a reasonable approach anyway18:32
fricklerdoing 300 updates to that single change doesn't sound much more feasible18:33
fungibut also repeatedly revising the bulk eol change adds manual effort for someone18:33
fungiright, that18:33
gouthamri don't think it'll be 30018:33
fricklereven 10 is a lot of work, which is kind of what I already had with the subset above18:33
gouthamrit'll be more like 2-3.. but we can timebox the reviews18:33
gouthamrfrickler: ack; i think we could get some more attention on the ML and tack a deadline for the transition18:34
gmannwe have 1 month time as written in policy "The patch to EOL the Unmaintained branch will be merged no earlier than one month after its proposal. "18:34
gouthamrnice18:34
gmannso volunteer has 1 month to -1 and fulfil the requirement18:34
fricklero.k., so we update the patch only if CI is free of errors18:35
fungito be clear, while i appreciate the work frickler has put into identifying which branches have not merged automated changes and which ones have broken zuul configs, i think that's way more effort than should be going into it, and we should be starting with a blanket eol and, i dunno, maybe ask anyone who wants to volunteer to take care of some specific repos to revise that18:35
fungichange themselves?18:35
gmannmaybe we should discuss release team if they can implement it but as frickler mentioned team is already occupied so we should find someone else?18:35
fricklerI guess I can propose a patch like that for victoria first and see how it goes18:35
gmannfungi: ++18:35
fungiput as much work on the volunteer caretakers as possible to minimize work we're creating for the release managers and/or tc18:36
gmannyeah, doing a single/blanket EOL and see who all need what all repo to stay and we can filter out those based on requirement fulfil 18:36
fricklermaybe a volunteer from TC to review -1s on that eol patch and update if appropiate could be found?18:36
gmannand that needs to stay open for a month so volunteer have enough time to respond 18:36
gouthamri can be that guinea pig.. i don't think this would be done often after we're done pruning branches until Zed18:37
gmannfrickler: good point. I can volunteer and bring up/validate the requirement to stay as unmaintained 18:37
gouthamrcrap, missed the opportunity, until "Zed is ded"18:37
spotz[m]hehe18:37
frickleronce per year after that18:38
fungiif a volunteer caretaker doesn't have the time to push up a revision to the blanket eol change, then they certainly don't have the time to actually take care of those branches18:38
gmannyeah, its once a year18:38
gmannonce we get rid of <=zed it will be more easy and smooth as only SLURP goes to unmaintained18:38
fricklerok, sounds like we have a plan18:39
gmannfungi: exactly. if any volunteer to keep any repo in unmaintined then we just handover to them to take care of everything including filterout form EOL and maintianed CI etc18:39
gouthamrack; just to recap, frickler will attempt a bulk EOL change for V, and gmann and gouthamr will assist with the rest of the branches until Zed.. each change will be open for exactly 1 month providing liaisons time to review18:41
gmannI think I volunteer to help on validating the -1 from volunter on EOL proposal change :)18:42
gouthamrif done right, this should bring us to implement our policy correctly; i.e., keep only one unmaintained branch (Antelope) 18:43
gmann++ yeah that is end goal to have one unmaintained branches if no one volutneer to maintain more18:43
gouthamrso are we settling on liaisons showing up at the time of the EOL proposal? we don't maintain a list anywhere? 18:44
fungialso remember that older branches have to go eol if newer ones do, so that needs to be taken into account for future iterations18:45
gmannyeah18:45
fungi(or older slurp when newer slurp goes eol, eventually_18:45
fungii suppose future blanket eol changes could do multiple branches in one change to facilitate that18:46
gouthamryes18:46
gouthamri don't think that's a problem - but if peole will get confused by a patch that says: "EOL for Bobcat and Caracal, but you can only object to Caracal", don't mind there being two patches 18:48
funginot-slurp branches don't transition to unmaintained, by policy, so putting them in their own change is a good idea just for expediency, as there's no process for objecting to them18:48
fungibut eol for unmaintained/2024.1 and unmaintained/2025.1 in the same change could make sense18:49
gmann++18:49
gouthamranything else to be agreed upon/debated for $topic?18:50
gouthamri think this was super helpful, even though it feels like we took a bulk of the meeting for this.. 18:50
gouthamri appreciate frickler raising attention to this, and to all the thoughts expressed here, its grunt work this sort of  housekeeping..but it'll pay off to keep our contributors sane as we've seen over the years18:52
gouthamr#topic Status on migrate CI to Ubuntu Noble18:52
gouthamrgmann: how goes it?18:52
gmannI sent latest status on ML yesterday18:52
gmann#link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/U5HIACZZVSGUHCPR47YS7KQWBCA6OKC5/18:53
gmannwe have 3 projects still failing and doc job migration still pending18:53
gmannout of those 3 projects, we have two project team looking into the failure bt 3rd one skyline is no response 18:53
gmann#link https://review.opendev.org/c/openstack/skyline-apiserver/+/93560018:54
gmann#link https://review.opendev.org/c/openstack/skyline-apiserver/+/935604/218:54
gmannnot sure what to do on this, I sent it on ML also and added core member in review18:54
gmannother than that, it seems monasca gate is broken and no response from team18:55
gmann#link https://etherpad.opendev.org/p/migrate-to-noble#L16518:55
gmannmonsaca is in inactive projetc list with extensions to this cycle but I am not seeing any progress to make it active18:55
gmann#link https://governance.openstack.org/tc/reference/emerging-technology-and-inactive-projects.html#current-inactive-projects18:55
fricklertime for retirement, then? we're close to e-2, too18:56
gmanngood thing in this migration is we also checked all team whether their gate is broken and maintainer are responsive or not. and all good except these two project i mentioned above18:57
gouthamrwhat's worked with wu_wenxiang and hasan.acar in the past has been direct email... they don't pay attention to the ML, and neither do any of the contributors unfortunately 18:57
gmannfrickler: I think yes18:57
gouthamr+118:57
gmanngouthamr: on horizon plugin failure, can you point me to the failure. I tested some plugin and they worked fine but have note tested all.18:57
gmannhorizon also greenon noble18:58
gmann#link https://review.opendev.org/c/openstack/horizon/+/93541318:58
fricklerI think horizon merged some fix last week?18:58
gmanndjango jobs running on py312 also18:58
gmann#link https://review.opendev.org/c/openstack/horizon/+/936801/418:58
gmannafter these two it should be green but feel free to add bug/failure in etherpad18:59
gouthamr#link https://review.opendev.org/c/openstack/manila-ui/+/936972 (horizon-tox-python3-django42 failure on noble) 18:59
gouthamrthis may be one, i'll look and confirm on the etherpad18:59
noonedeadpunki THINK THIS ONE SHOULD BE FIXED NOW18:59
gmanngouthamr: recheck that, it is fixed now18:59
noonedeadpunksory18:59
gmannyeah18:59
gouthamrbeautiful, hey don't shout at me, noonedeadpunk 18:59
gmannheh18:59
noonedeadpunkI didn't mean to, sorry :)18:59
gouthamrokay, we're at the top of the hour18:59
gmannthanks noonedeadpunk for fixing and help there19:00
* noonedeadpunk multitasking19:00
gouthamrthanks for the updates gmann 19:00
slaweq:)19:00
gouthamranything else to be added in the minutes here?19:00
gouthamr#topic Open Discussion19:00
* gouthamr waits the full minute19:00
gouthamrone thing could be that two of our meetings fall on year-end-holidays19:01
slaweqI will not attend meeting on 24th and 31st of December for sure :)19:02
gouthamrand, due to a personal life-change, i might need help running the next meeting (Dec 17th)19:02
gouthamrwe can catch up on these here, async.. 19:02
gouthamrthank you all for attending today19:02
slaweqthx, bye19:02
gmanngouthamr: I can run on 17th if no other volunteer. let me know19:02
spotz[m]Thanks! I'll be out fo the rest of the year but will probably pop in19:03
gouthamrgmann++ that's awesome, thank you very much!! :) 19:03
gouthamrspotz[m]: ack19:03
gouthamr#endmeeting19:03
opendevmeetMeeting ended Tue Dec 10 19:03:15 2024 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)19:03
opendevmeetMinutes:        https://meetings.opendev.org/meetings/tc/2024/tc.2024-12-10-18.01.html19:03
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/tc/2024/tc.2024-12-10-18.01.txt19:03
opendevmeetLog:            https://meetings.opendev.org/meetings/tc/2024/tc.2024-12-10-18.01.log.html19:03
gouthamrgmann: noonedeadpunk mentioned he'd be unavailable too, but, would try to join if necessary19:04
gouthamrif anyone else feels motivated to try to run this meeting on 17th, please ping me :) else, i'll chat with gmann and cue the summary/AIs/agenda19:05
gmanngouthamr: ack19:05
gouthamri think its prudent that we cancel the meetings on 24th and 31st 19:05
gouthamrany objection to this? 19:06
gmannyeah, make sense. I might be available but agree to cancel19:06
spotz[m]Works for me:)19:07
gouthamrcardoe gtema bauzas noonedeadpunk frickler: please let me know if you're okay canceling the meeting on 24th and 31st (when you see this ping).. fungi clarkb: you two as well, please :) 19:08
cardoeyes make sense to me.19:08
bauzasGood to me19:08
fungiwfm. i'll also be driving a car at this time next week so will be missing that one as well19:08
gtema+119:08
clarkbI will not be here19:09
clarkbso good for me19:09
noonedeadpunk+1 to that19:09
frickler+119:09
bauzasSorry for not be here for the meeting, I was off of my home19:09
gouthamrthanks everyone19:23
gmannsent monasca situation on ML and asking if any volunteer to maintain it  https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/4CAP3U3IYSKUD2M7AHM4KOIUAX2RO3XX/19:37
*** atmark_ is now known as atmark19:37
gmannAs per timeline, this is cycle we either should retire it or give another extensions to stay as Inactive19:37
gmanngouthamr: Re manila-ui failure. all green there https://review.opendev.org/c/openstack/manila-ui/+/93697219:38
gouthamrnice; thanks for re-checking that gmann 19:38
gouthamrand for the ML post 19:39
gouthamrwill add this topic to next week's agenda19:39
gmann++19:39
opendevreviewIan Y. Choi proposed openstack/election master: Add configuration for 2025.2/"F" elections  https://review.opendev.org/c/openstack/election/+/93740821:18
opendevreviewMerged openstack/election master: Add TC liaison in election page  https://review.opendev.org/c/openstack/election/+/93741221:26

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!