| opendevreview | Michal Nasiadka proposed opendev/glean master: Ensure files are created with 0600 perms https://review.opendev.org/c/opendev/glean/+/953276 | 04:04 |
|---|---|---|
| opendevreview | Michal Nasiadka proposed opendev/glean master: Stop adding uuid= in keyfiles https://review.opendev.org/c/opendev/glean/+/953320 | 05:56 |
| opendevreview | Michal Nasiadka proposed opendev/glean master: Ensure files are created with 0600 perms https://review.opendev.org/c/opendev/glean/+/953276 | 07:16 |
| opendevreview | Michal Nasiadka proposed opendev/glean master: Stop adding uuid= in keyfiles https://review.opendev.org/c/opendev/glean/+/953320 | 07:16 |
| opendevreview | Michal Nasiadka proposed opendev/glean master: Ensure files are created with 0600 perms https://review.opendev.org/c/opendev/glean/+/953276 | 08:01 |
| opendevreview | Michal Nasiadka proposed opendev/glean master: Stop adding uuid= in keyfiles https://review.opendev.org/c/opendev/glean/+/953320 | 08:01 |
| opendevreview | Antoine Musso proposed zuul/zuul-jobs master: upload-logs-s3: allow logging output on upload https://review.opendev.org/c/zuul/zuul-jobs/+/953441 | 12:53 |
| opendevreview | Antoine Musso proposed zuul/zuul-jobs master: upload-logs-swift: allow logging output on upload https://review.opendev.org/c/zuul/zuul-jobs/+/953442 | 12:53 |
| fungi | i'm rebooting wiki.openstack.org again to get openid logins back to a working state | 13:12 |
| fungi | #status log Rebooted wiki.openstack.org again to get openid logins back to a working state | 13:14 |
| opendevstatus | fungi: finished logging | 13:16 |
| fungi | yeesh, load average is already above 80 a mere 3 minutes after rebooting | 13:16 |
| mnasiadka | it seems glean network manager implementation needs some love - Rocky Linux 10 didn't like two aspects of current config - so would be grateful for some reviews on https://review.opendev.org/c/opendev/glean/+/953276 and https://review.opendev.org/c/opendev/glean/+/953320/8 | 14:24 |
| opendevreview | Michal Nasiadka proposed opendev/zuul-providers master: Add CentOS Stream 10 builds https://review.opendev.org/c/opendev/zuul-providers/+/953460 | 14:33 |
| fungi | mnasiadka: +2 from me on both glean changes. if we can get an extra set of eyes on that, i'm happy to tag a new release as soon as they merge | 14:34 |
| fungi | #status log Pruned /opt/backups-202010 on backup02.ca-ymq-1.vexxhost reducing volume usage from 91% to 59% | 14:52 |
| opendevstatus | fungi: finished logging | 14:54 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Replace zk06 with zk01 https://review.opendev.org/c/opendev/system-config/+/951164 | 14:56 |
| clarkb | I realized there was some additional zk06 cleanup missing from ^ | 14:57 |
| clarkb | infra-root ^ I don't want to rush into that so reviews are very much appreciated then I'll try and start that first thing on a quiet morning (guessing not today just because there haven't bee nreviews yet and I think I have a morning meeting tomorrow so maybe monday?) | 14:58 |
| clarkb | and thats fine. The etherpad upgrade is still failing so I need to debug that too | 14:59 |
| clarkb | mnasiadka: fungi: both glean changes lgtm I've approevd them | 15:01 |
| mnasiadka | Yay, thanks :) | 15:01 |
| clarkb | rubencabrera[m]: I'm looking at https://review.opendev.org/c/openstack/diskimage-builder/+/952875 and while the fundamental aspects of the change seem fine (need to sudo to write to that directory and that requires tee beacuse > doesn't get sudo privs) I'm confused as to why anyone would need that element anymore | 15:04 |
| clarkb | rubencabrera[m]: the default for debian now is systemd right? so that element seems like it shouldn't be necessary at all at this point. A leftover from the debian systemd transition period? | 15:05 |
| clarkb | fungi: before we tag any glean releases can we ensure that the rockylinux jobs pass on https://review.opendev.org/c/openstack/diskimage-builder/+/952548 ? | 15:06 |
| rubencabrera[m] | <clarkb> "rubencabrera: the default for..." <- I misread what the element meant and thought that for some reason the plain `debian` wasn't using systemd. But this being a leftover from the transition makes much more sense. I'm happy for the change to be rejected, then. | 15:11 |
| clarkb | rubencabrera[m]: ya debian-systemd is an easy way to override the default here: https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/debian/install.d/10-cloud-opinions#L24-L31 | 15:14 |
| clarkb | rubencabrera[m]: looks like wheezy was the last one defaulting to sysvinit and everything after that should be systemd-sysv | 15:14 |
| rubencabrera[m] | Is it OK if I send another change to update the debian-systemd README so I don't make the same mistake the next time I look at this? 😅 | 15:20 |
| clarkb | rubencabrera[m]: yup that would be great | 15:25 |
| clarkb | I believe https://github.com/panva/node-oidc-provider/commit/11e1b330c8df82c566c68897815f9af4a3b688b7 this change is what broke etherpad oidc token generation and then etherpad 2.3.1/2.3.2 upadted deps to pick that up | 15:29 |
| clarkb | I'm getting this error on the held node: 'resource indicator must be an absolute URI' | 15:29 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Update etherpad to v2.3.2 https://review.opendev.org/c/opendev/system-config/+/953328 | 15:50 |
| opendevreview | Clark Boylan proposed opendev/system-config master: DNM force etherpad failure to hold node https://review.opendev.org/c/opendev/system-config/+/840972 | 15:50 |
| clarkb | I guess the oidc library just needs a resource identify in the token generation request? We'll see if that makes testing happy | 15:55 |
| clarkb | I ended up hacking the js and restarting things on the held node to get more logging output. I'm glad nodejs doesn't need to run minified content :) | 15:55 |
| fungi | clarkb: are the dib jobs testing glean from source? | 15:59 |
| clarkb | fungi: yes | 16:02 |
| clarkb | that change depends on the two glean changes for that reason and won't pass with glean as is | 16:02 |
| clarkb | https://zuul.opendev.org/t/openstack/build/8a6131614d194b3b8b104c04f185c03c/log/diskimage-rocky-container.log#346-349 is the current issue which isn't realted to glean but unfortunately means we fail before we check glean | 16:04 |
| clarkb | I guess if the centos 10 job is passing then its good enough to know we didn't break things worse and a release is probably fine. But we may need more fixes for rock linux | 16:04 |
| clarkb | and centos 10 stream did pass so a release is probably fine | 16:05 |
| opendevreview | Merged opendev/glean master: Ensure files are created with 0600 perms https://review.opendev.org/c/opendev/glean/+/953276 | 16:15 |
| clarkb | etherpad update passes testing now and held node is at 66.70.103.71. I haven't tested on that yet but it should be ready for that when we get around to it | 16:38 |
| opendevreview | Merged opendev/glean master: Stop adding uuid= in keyfiles https://review.opendev.org/c/opendev/glean/+/953320 | 16:47 |
| clarkb | connected to the held etherpad from firefox and chrome and it seems to work for me. I used the pad clarkb-test if anyone else wants to look wiht 66.70.103.71 as etherpad.opendev.org in /etc/hosts | 17:15 |
| clarkb | I suspect we can upgrade etherpad whenever someone else reviews the change (and ideally tests it too via the held node)) | 17:17 |
| mnasiadka | clarkb: did another recheck, it was building fine yesterday, so it looks like some rocky mirror problem | 17:30 |
| clarkb | mnasiadka: ya or maybe the packages updated in a way that just doesn't work right now | 17:31 |
| clarkb | NeilHanlon may have insight (but it isn't urgent I think its safe to update glean and do a release based on the centos 10 stream testing) | 17:31 |
| NeilHanlon | o/ | 17:31 |
| * NeilHanlon reds | 17:31 | |
| NeilHanlon | Yeah our propagation looks bad the last few hours | 17:32 |
| NeilHanlon | https://drop1.neilhanlon.me/irc/uploads/26290448bf19d150/image.png | 17:32 |
| NeilHanlon | i will go slap filebeat in the face | 17:32 |
| clarkb | thansk! | 17:33 |
| NeilHanlon | of course of course | 17:33 |
| fungi | clarkb: test held etherpad lgtm, i made some edits to your clarkb-test pad there | 18:07 |
| clarkb | fungi: cool I should be able to monitor if we want to deploy that | 18:08 |
| fungi | yeah, lemme approve just a sec | 18:08 |
| fungi | and done | 18:09 |
| opendevreview | Merged opendev/system-config master: Update etherpad to v2.3.2 https://review.opendev.org/c/opendev/system-config/+/953328 | 18:51 |
| clarkb | still waiting for ^ to update the container on the server | 18:54 |
| clarkb | it just restarted and is coming up now | 18:55 |
| clarkb | Up 17 seconds (healthy) | 18:55 |
| clarkb | https://etherpad.opendev.org/p/opendev-zookeeper-upgrade-2025 still loads for me | 18:55 |
| clarkb | as does the old zk server update etherpad: https://etherpad.opendev.org/p/opendev-zookeeper-upgrade-2021 | 18:55 |
| clarkb | is there anything specific we want to test before I eat lunch? | 18:56 |
| clarkb | https://etherpad.opendev.org/p/gerrit-upgrade-3.10 makes use of more etherpad features and also seems to load properly | 18:59 |
| clarkb | I think I'm happy enough with this for now | 18:59 |
| mnasiadka | clarkb, fungi: rocky 10 passed in dib - it’s safe to release glean ;) | 19:16 |
| fungi | okay thanks for confirming! working on that now | 19:21 |
| fungi | clarkb: yeah, upgraded etherpad server lgtm | 19:21 |
| fungi | seeing as how those two bugfix changes from today are the only thing merged for glean since 1.25.0 i propose we release this as 1.25.1 | 19:30 |
| fungi | commit 547dfdcb04877e8bc211d02d2b9c16bbc9e0d13f (HEAD, tag: 1.25.1, origin/master, origin/HEAD, gerrit/master, gerrit/HEAD) | 19:32 |
| fungi | infra-root: ^ objections? | 19:32 |
| Clark[m] | Sounds good to me | 19:37 |
| fungi | okay, hearing no concerns, i'm pushing that now | 19:59 |
| fungi | https://pypi.org/project/glean/ has updated now | 20:04 |
| fungi | mnasiadka: should be able to move forward with new glean now | 20:05 |
| fungi | #status log Released glean 1.25.1 | 20:05 |
| opendevstatus | fungi: finished logging | 20:05 |
| clarkb | thanks | 20:05 |
| clarkb | fungi: https://review.opendev.org/c/opendev/glean/+/953163 is a quick review for glean as part of cleanup for the testing changes | 20:05 |
| clarkb | (it didn't need to go in before the release so thats all fine. Just thinking now that the release is done we should do the last bit of cleanup there) | 20:06 |
| clarkb | I've +2'd the rocky linux 10 dib change as well. I think that looks good and is ready to go in. We may be able to doa dib release soon with support for EL 10 and a few of the other things that have gone in recently | 20:07 |
| clarkb | while trying to sort out some pip dep solving problems with zuul's requirements.txt file I notice that tools that release super often make this problem worse | 20:26 |
| clarkb | that said there must be some transitive dep or something that casues pip to go crazy and scan the entire history of all these deps rather than just take the latest version | 20:27 |
| corvus | niz update: i think the launchers are slowly catching up with the image upload workload. it's a bit hard to tell what they're doing, so i proposed some logging changes to improve that. | 20:32 |
| corvus | i also found that we're setting timestamps where i don't think we should, which is causing some confusion. it's not critical, but it's causing some confusion about which uploads to delete. that's addressed in https://review.opendev.org/953469 | 20:32 |
| clarkb | for the timestasmp thing we're essentially having the creation timestamp occur at the very beginning then don't update it again so that we preserve order right? I +2'd the stack they all seemed reasonable to me | 20:39 |
| corvus | yep. so it'll be pretty close to the time at the point the artifact build completed. | 20:42 |
| corvus | i noticed it as an issue because due to the errors/crashes earlier in the week, we ended up uploading old artifacts much later than we would have normally | 20:43 |
| fungi | clarkb: for pip dep solver going crazy, usual suspects are any (direct or transitive) dependencies with skipped or capped versions forcing it to evaluate older versions | 20:48 |
| clarkb | zuul doesn't have a lot of those but I did notice that pyparsing and httplib2 were capped by google's cloud api lib which probably contributed to the last round I got past. Now its complaining about boto3 | 20:50 |
| clarkb | not sure if others saw but openmetal will be migrating the physical location of our openmetal cloud sometime between july 21 and august 22. They'll send us followup email with more accurate timing info as we get closer | 21:00 |
| clarkb | I don't anticipate that to be a big issue for us. We can set max-servers to 0 if we're wanting to be cautious | 21:01 |
| fungi | yeah, i skimmed the announcement, seems non-urgent for now but good to be aware of | 21:01 |
| clarkb | I'm going to pop out for a bike ride nowish. Today is nice and cool but summer is coming | 21:06 |
| fungi | sounds like a great idea | 21:17 |
| fungi | i'm staying indoors to hide from the heat and unusual lack of breeze | 21:17 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
