| clarkb | Hello anyone here for the meeting? | 19:00 |
|---|---|---|
| frickler | o/ | 19:00 |
| clarkb | I suspect it might be another quiet one. But we do have some things to call out | 19:00 |
| fungi | helo/ehlo | 19:00 |
| clarkb | #startmeeting infra | 19:01 |
| opendevmeet | Meeting started Tue Jan 25 19:01:19 2022 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 19:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 19:01 |
| opendevmeet | The meeting name has been set to 'infra' | 19:01 |
| clarkb | #link https://lists.opendev.org/pipermail/service-discuss/2022-January/000315.html Our Agenda | 19:01 |
| clarkb | #topic Announcements | 19:01 |
| clarkb | OpenInfra Summit CFP and programming committee need your input: https://openinfra.dev/summit/ | 19:01 |
| clarkb | I'm going to propose something related to opendev once I can sort out what I want to talk about | 19:02 |
| clarkb | if you've got something interesting to share definitely go for it | 19:02 |
| clarkb | Service coordinator nominations run January 25, 2022 - February 8, 2022 That is now :) | 19:02 |
| clarkb | It has been 6 months since the last service coordinator election. I'm more than happy for someone else to take it one and run these meetings :) | 19:02 |
| clarkb | If you are interested but need more info feel free to reach out | 19:03 |
| clarkb | You have a couple of weeks to make up your mind | 19:03 |
| clarkb | #topic Actions from last meeting | 19:03 |
| clarkb | #link http://eavesdrop.openstack.org/meetings/infra/2022/infra.2022-01-18-19.01.txt minutes from last meeting | 19:03 |
| clarkb | None recorded. We can dive right into the agenda then | 19:03 |
| clarkb | #topic Topics | 19:03 |
| clarkb | #topic Gerrit 3.4 upgrade | 19:03 |
| clarkb | This is a last minute meeting chair addition to the agenda. | 19:04 |
| clarkb | Yesterday we renamed a couple of projects and upgraded Gerrit to 3.4 from 3.3 | 19:04 |
| clarkb | Overall Gerrit seems to be working, but we did discovered that acls for pushing tags don't work correctly. | 19:04 |
| clarkb | fungi has a workaround update in progress which should get project releases moving again. I diagnosed the issue and filed a bug upstream as well as pushed a fix to upstream gerrit | 19:04 |
| clarkb | #link https://bugs.chromium.org/p/gerrit/issues/detail?id=15616 Gerrit 3.4 tag pushing acl bug | 19:05 |
| clarkb | #link https://gerrit-review.googlesource.com/c/gerrit/+/328839 Gerrit 3.4 tag pushing acl fix | 19:05 |
| fungi | yeah, workaround is deploying now and i'll send an announcement to service-announce (already drafted) once that's done | 19:05 |
| clarkb | I expect we'll be able to move past this one relatively quickly once we get fungi's workaround in place. We can then remove those updates once upstream lands a/the fix | 19:06 |
| fungi | the removal is also already proposed, just wip | 19:06 |
| clarkb | Please let us know if you notice any other odd behavior. At least one user noticed that gerrit doesn't check mergeablity anymore (this was documented as a breaking change in the release notes) | 19:06 |
| fungi | it may need some rebasing/updating before it can merge, depending on how long we go without a proper fix applied | 19:06 |
| clarkb | good point | 19:06 |
| fungi | the litner script change included in it will tell us if that's the case | 19:07 |
| clarkb | Also a huge thank you to everyone who helped with the renames and upgrade and all the testing and prep and work that goes into making this happen relatively smoothly | 19:07 |
| clarkb | It is nice that we don't end up stuck on gerrit releases for half a decade because upgrading is almost impossibl3 | 19:07 |
| fungi | so it's safe to approve if in doubt, and we can update it if zuul kicks it back | 19:07 |
| clarkb | we can also audit the repo after it merges | 19:08 |
| clarkb | #topic Improving OpenDev's CD throughput | 19:09 |
| clarkb | ianw isn't able to make it today, but I think he largely needs reviews on this. unfortunately due to all the other happenings (like project renames and gerrit upgrades) I haven't had time to get to this yet. | 19:09 |
| clarkb | #link https://review.opendev.org/c/opendev/infra-specs/+/821645 -- spec outlining some of the issues with secrets | 19:09 |
| clarkb | #link https://review.opendev.org/c/opendev/system-config/+/821155 -- sample of secret writing; more info in changelog | 19:09 |
| clarkb | please take a look and leave your thoughts when you have time | 19:09 |
| clarkb | #topic Container maintenance | 19:10 |
| clarkb | Similarly I haven't had time to dive into the dedicated user chagne stuff. jentoio we should just set up an hour to go over this and then I'll commit to it :) | 19:10 |
| clarkb | #link https://etherpad.opendev.org/p/opendev-container-maintenance | 19:10 |
| clarkb | I did start to classify how difficult/painful I thought each of these might be | 19:10 |
| clarkb | that info is in the etherpad | 19:10 |
| jentoio | sounds good, will take a look at it | 19:11 |
| clarkb | #topic Nodepool Image Cleanup | 19:11 |
| clarkb | In the email announcements I sent about this I basically said end of January we'd remove centos 8 | 19:11 |
| clarkb | seems a number of projects are working through the switch to stream (or removing the jobs entirely) | 19:12 |
| clarkb | I think we are still on track to remove centos 8 from nodepool and our mirrors starting next week as a result | 19:12 |
| clarkb | One thing to keep in mind is that the whole ping doesn't work situation is still a thing aiui. So some people converting to stream may run into problems. But there isn't much we can do about that. CentOS 8 Stream is actually broken | 19:12 |
| fungi | broken for anyone who want to run ping as a normal user at least | 19:13 |
| clarkb | right that | 19:13 |
| clarkb | #topic Spring cleaning for old reviews | 19:14 |
| clarkb | frickler: brought this up and I asked people to think a bit about it last week | 19:14 |
| clarkb | For me personally I think we can go ahead and do a time based abandoning. I would start at 2 years rather than one though just looking at my changes some of them are older than a yaer and still potentially worth having in my list | 19:15 |
| fungi | a component of this may also be spring cleaning for git repos we could be retiring entirely | 19:15 |
| clarkb | I think a big part of that is our switch to ansible + docker really picked up steam in that time frame | 19:15 |
| clarkb | fungi: ya I noticed a number of our old puppet module repos are getting the tag workaround and many of them could just be retired I expect | 19:16 |
| fungi | i do have quite a few reviews waiting since 2014, but a bunch are obsolete because of exactly that | 19:17 |
| clarkb | maybe we take two axis then. First is determine which repos can have all changes abandoned and then retire the repos. Then for everything else do an auto abandon at ~2 years? | 19:17 |
| clarkb | frickler: does something like that seem reasonable? | 19:17 |
| fungi | looks like the oldest thing i have not obsoleted by puppet droppage is a stack to rework return codes in bindep from 2017 | 19:18 |
| fungi | but some stuff >2yo like https://review.opendev.org/489711 probably warrants a second look | 19:19 |
| fungi | probably got done or obsoleted more recently, but i'd want to be sure | 19:19 |
| frickler | sorry, was distracted, catching up | 19:19 |
| clarkb | fungi: ++ | 19:19 |
| frickler | so for comparison, devstack is in a similar situation. I started going through old reviews there manually, but it is very time consuming and I haven't gotten very far yet | 19:21 |
| frickler | otoh I did find a small number of patches that were worth reviving | 19:21 |
| fungi | yeah, i'm okay with forging ahead, i'll just make a very close inspection of the abandomnents gerrit notifies me about and hopefully find time to look into whether they need to be restored | 19:22 |
| clarkb | right we can always restore later. | 19:22 |
| clarkb | And I think abandoning things that should be retired next will prune quite a bit potentially | 19:23 |
| clarkb | If that sounds good I can probably help with the actual tasking end of next week? I've got a bunch of stuff I need to get done this week like CFP things and the gerrit stuff and prep for centos 8 removal and so on | 19:24 |
| clarkb | But once that tapers off I should have some time to help | 19:25 |
| clarkb | Anything else on this topic? | 19:26 |
| frickler | I don't think that I would have much time for anything beyond doing things scripted | 19:26 |
| frickler | but if we agree to not do this yet, I'm fine with it, too | 19:26 |
| clarkb | frickler: ok, maybe I can help produce a list of repos to retire and you can script up abandoning all changes for those repos | 19:26 |
| frickler | ack | 19:27 |
| clarkb | and we can give everyone a week or two to look over old changes if they like and make them not old then do a 2 year old abandonment | 19:27 |
| clarkb | I can help with the digging aspect if you do the scripting bits. I think that should work well | 19:27 |
| fungi | probably retiring things and then abandoning older changes in whatever's left is the order we want to go in | 19:27 |
| clarkb | fungi: ya I guess that would prevent new changes | 19:28 |
| clarkb | since the repos go RO | 19:28 |
| clarkb | but ya I can start on that aspect late next week I expect | 19:28 |
| clarkb | #topic Updating Grafana | 19:29 |
| clarkb | #link https://review.opendev.org/c/opendev/grafyaml/+/825990 needed to fix api use against newer grafana | 19:29 |
| fungi | well, also it makes it easier for those of us with abandoned changes to know which ones were abandoned because the repos were retired | 19:29 |
| clarkb | ianw pointed out this fix to grafyaml is necessary to accomodate api changes in grafana before we upgrade | 19:29 |
| clarkb | fungi: ++ | 19:29 |
| clarkb | If we can get the grafyaml change reviewed that would be good then we can look at updating grafana | 19:29 |
| clarkb | #link https://review.opendev.org/c/opendev/system-config/+/825410 update to latest grafana | 19:29 |
| clarkb | That was basically all I had on the agenda. | 19:31 |
| clarkb | #topic Open Discussion | 19:31 |
| clarkb | Is there anything else to call out? | 19:31 |
| fungi | the tagging workaround deployed successfully so i'm sending the announcement about it to the ml now | 19:32 |
| clarkb | I'm going to send mail to the gerrit mailing list after the meeting showing what we found in case anyone else is running into this. Then lunch and then testing my fix | 19:32 |
| fungi | thanks | 19:33 |
| clarkb | sounds like that may be everything | 19:35 |
| clarkb | Thank you everyone! | 19:35 |
| clarkb | We'll see you here next week | 19:35 |
| fungi | thanks clarkb! | 19:35 |
| clarkb | #endmeeting | 19:36 |
| opendevmeet | Meeting ended Tue Jan 25 19:36:10 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 19:36 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/infra/2022/infra.2022-01-25-19.01.html | 19:36 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/infra/2022/infra.2022-01-25-19.01.txt | 19:36 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/infra/2022/infra.2022-01-25-19.01.log.html | 19:36 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!