| Clark[m] | That seems reasonable | 00:08 |
|---|---|---|
| JayF | tonyb: I believe there's also actively code in the did Gentoo element to keep those files out of the way. If it's not working or it's missing something, we should consider enhancing it | 00:20 |
| tonyb | I didn't start from the DIB image, but that is probably a reasonable step the next time I need this image. | 02:32 |
| tonyb | For now, apart from the size it works reasonably well | 02:32 |
| JayF | the dib element builds from a stage3 tarball | 02:59 |
| JayF | I would suspect it's probably still a B- experience at this stage but the bones are good :D | 03:00 |
| tonyb | JayF: Oh. Interesting. That'd be somewhat compatible with what I have then. | 03:01 |
| JayF | https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/gentoo | 03:01 |
| opendevreview | Michal Nasiadka proposed opendev/zuul-providers master: Add osuosl provider https://review.opendev.org/c/opendev/zuul-providers/+/947922 | 11:32 |
| opendevreview | Michal Nasiadka proposed opendev/zuul-providers master: Add ubuntu noble arm64 image https://review.opendev.org/c/opendev/zuul-providers/+/947841 | 11:40 |
| opendevreview | Michal Nasiadka proposed opendev/zuul-providers master: Add ubuntu noble arm64 image https://review.opendev.org/c/opendev/zuul-providers/+/947841 | 11:41 |
| mnasiadka | corvus, clarkb : I updated the patch stack at https://review.opendev.org/c/opendev/zuul-providers/+/947841 - I don't know flavor names in OSUOSL so I did some guesswork - feel free to update them - once that's merged I can continue with Jammy and Debian (and then the CentOS/Rocky bits) | 11:43 |
| clarkb | mnasiadka: thanks, let me get a flavor listing from osuosul that I can post to the change | 14:46 |
| clarkb | mnasiadka: and posted | 14:51 |
| dpanech | Hi, we are getting mirror-related errors in Zuul (from pip install) similar to "404 Client Error: Not Found for url: https://mirror.gra1.ovh.opendev.org/wheel/debian-11-x86_64/pip/" . Any idea why? Here's the build log: https://zuul.opendev.org/t/openstack/build/52b4e6f0e39943cb9656e82244d68378 | 15:08 |
| clarkb | dpanech: https://mirror.gra1.ovh.opendev.org/wheel/debian-11-x86_64/pip/ is a 404, but I think this is expected. We only have wheels in the wheel mirror for packages that do not have wheels on pypi. Pip almost certainly publishes their own wheels on pypi | 15:11 |
| clarkb | dpanech: the way pip is configured the wheel mirrors are set up as fallbacks to the primary pypi proxy cache and pip is supposed to check there if the primary doesn't have what it is looking for. In this case the wheel mirror is a red herring | 15:12 |
| clarkb | dpanech: ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts | 15:12 |
| clarkb | this is the issue. pip could not resolve the set of dependencies you gave it. You need to work backward in the log and see why that may be | 15:12 |
| clarkb | dpanech: which will get you to https://zuul.opendev.org/t/openstack/build/52b4e6f0e39943cb9656e82244d68378/log/job-output.txt#72460-72468 | 15:13 |
| fungi | clarkb: slight correction... pip doesn't have "fallbacks" rather it checks all configured locations and then choses the "best" result | 15:15 |
| clarkb | oh right it will use the newest version possible and in the event of version collisions pick wheels by default | 15:15 |
| fungi | depending on other pip options you set, yes | 15:16 |
| clarkb | anyway not having a package in the wheel mirror is fine as long as pypi has the package. So ignore that 404 and look at the logged ERROR messages | 15:16 |
| clarkb | I'm like 98% certain the last link I shared is the actual cause of the failure | 15:16 |
| fungi | kubernetes 30.1.0 depends on oauthlib>=3.2.2, The user requested (constraint) oauthlib===3.1.0 | 15:16 |
| fungi | yeah, probably need to either constrain kubernetes to a compatible version or adjust the oauthlib constraint | 15:17 |
| fungi | or stop constraining oauthlib so pip can pick a compatible version | 15:17 |
| dpanech | fungi,clarkb: thanks for the pointers, I'll have a look | 15:24 |
| opendevreview | Merged opendev/zone-opendev.org master: Set review.opendev.org TTL back to default of 1 hour https://review.opendev.org/c/opendev/zone-opendev.org/+/947858 | 15:48 |
| clarkb | ok I think I'm about ready to finally shutdown the screen session on review03. Last call for any objections to me doing so | 16:50 |
| fungi | sounds good to me | 16:54 |
| fungi | i'm disappearing for a bit to run a couple of errands, maybe 45 minutes-ish | 16:54 |
| clarkb | and screen is gone now | 16:57 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Use podman to build non docker hub container images https://review.opendev.org/c/opendev/system-config/+/947872 | 17:07 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Migrate gerrit images to quay.io https://review.opendev.org/c/opendev/system-config/+/882900 | 17:07 |
| clarkb | that is just a rebase for the gerrit change. In the parent change I've updated things to force a hound rebuild to ensure we're checking this end to end upfront and don't discover problems with hound built by podman accidentally later | 17:07 |
| clarkb | I think we can proceed with 947872 if that comes back with successful test results too | 17:08 |
| clarkb | that will be our first podman built image if so. Alternatively or concurrently we can also land https://review.opendev.org/c/opendev/lodgeit/+/947873 to do the same thing with lodgeit | 17:08 |
| opendevreview | Michal Nasiadka proposed opendev/zuul-providers master: Add osuosl provider https://review.opendev.org/c/opendev/zuul-providers/+/947922 | 17:21 |
| opendevreview | Michal Nasiadka proposed opendev/zuul-providers master: Add osuosl provider https://review.opendev.org/c/opendev/zuul-providers/+/947922 | 17:24 |
| opendevreview | Michal Nasiadka proposed opendev/zuul-providers master: Add ubuntu noble arm64 image https://review.opendev.org/c/opendev/zuul-providers/+/947841 | 17:24 |
| opendevreview | Michal Nasiadka proposed opendev/zuul-providers master: Add ubuntu noble arm64 image https://review.opendev.org/c/opendev/zuul-providers/+/947841 | 17:26 |
| mnasiadka | clarkb: thanks, I think I worked it out, let me know if anything needs updating in these patches | 17:27 |
| fungi | seems things are still quiet, so i'll probably disappear again shortly to eat something while there's still a lull | 17:40 |
| clarkb | mnasiadka: thanks. First change lgtm and I left some thoughts on the second one | 17:50 |
| clarkb | fungi: when you get back I can be around to ensure both paste and codesearch restart on new images if we land https://review.opendev.org/c/opendev/system-config/+/947872 and or https://review.opendev.org/c/opendev/lodgeit/+/947873 | 17:51 |
| clarkb | also happy if we want ot wait for more feedback on having podman build our images too but in the meeting that seemed to be generally where consensus was headed | 17:52 |
| opendevreview | Michal Nasiadka proposed opendev/zuul-providers master: Add ubuntu noble arm64 image https://review.opendev.org/c/opendev/zuul-providers/+/947841 | 17:53 |
| mnasiadka | clarkb: thanks, makes sense - updated | 17:54 |
| fungi | oh, i haven't left yet | 17:56 |
| fungi | i'll approve 947872 and 947873 now | 17:56 |
| fungi | i went ahead and approved both | 17:58 |
| fungi | straightforward and meet the consensus we came to in yesterday's meeting | 17:59 |
| opendevreview | Merged opendev/lodgeit master: Update lodgeit to use podman to build its container image https://review.opendev.org/c/opendev/lodgeit/+/947873 | 18:05 |
| fungi | the way we reuse a paused opendev-buildset-registry job in multiple contexts makes zuul's time estimate for it extremely inaccurate | 18:13 |
| fungi | i guess it's only periodic jobs that will deploy the updated lodgeit container image? | 18:16 |
| clarkb | ya we can manually update it though which was my plan | 18:17 |
| opendevreview | Merged opendev/zuul-providers master: Add osuosl provider https://review.opendev.org/c/opendev/zuul-providers/+/947922 | 18:17 |
| fungi | cool, wfm | 18:17 |
| fungi | the promote succeeded anyway | 18:17 |
| clarkb | fungi: the hound change should auto deploy beacuse I updated teh image within system-config. Lodgeit being a separate repo doesn't trigger the opendev infra prod jobs | 18:18 |
| fungi | right | 18:18 |
| fungi | separately, the hound change has been waiting for a some node assignments longer than i expected | 18:18 |
| fungi | s/for a/for/ | 18:19 |
| fungi | grafana indicates we're not running at/near capacity | 18:19 |
| clarkb | paste is updated and my osuosl flavors paste from earlier loads: https://paste.opendev.org/show/b14FJrGjXpYZAIQflG0V/ I'll make a new test paste too | 18:20 |
| clarkb | fungi: sometimes that can happen if the node has to run in a specific region (due to the buildset registry) and taht region is full up or slow | 18:20 |
| fungi | lgtm https://paste.opendev.org/show/belQ8atKKM7EFa9N0PIQ/ | 18:21 |
| clarkb | https://paste.opendev.org/show/bVj6oZZG62xMLrDOqjG2/ this test seems to work too | 18:21 |
| fungi | i did mine with the pastebinit cli tool, which is still working fine as well | 18:21 |
| opendevreview | James E. Blair proposed opendev/zuul-providers master: Move labels-arm64 into labels.yaml https://review.opendev.org/c/opendev/zuul-providers/+/947984 | 18:22 |
| corvus | clarkb: ^ i did that in a followup change so we're not constantly asking mnasiadka to change things back and forth | 18:23 |
| fungi | registry job is running in rax-dfw, so i guess it's just struggling to boot new instances at the moment | 18:23 |
| clarkb | corvus: ack I'm reviewing both now. | 18:24 |
| clarkb | corvus: ok I approved mnasiadka's change but not yours as I'm theo nly reviewer on yours. Maybe we should approve it since its a small touch up? | 18:25 |
| corvus | done | 18:26 |
| clarkb | looking at gerrit 3.11 upgrade topics Java 21 is recommended but appears that 17 is still supported. I brought this up briefly in the past and i Think the idea was to upgrade to 3.11 on java 17 with bookworm. Then while on gerrit 3.11 switch to trixie with java 21 | 18:32 |
| clarkb | I guess I should double check trixie has java 21 and then maybe also ask upstream if there is any concern with running on 17 with 3.11 while we wait for debian to update | 18:32 |
| clarkb | https://packages.debian.org/search?suite=trixie&searchon=names&keywords=openjdk this does show openjdk 21 (as well as 17, 22, 24, and 25). fungi debian won't trim that list at this point right? | 18:34 |
| fungi | https://tracker.debian.org/pkg/openjdk-17 mentions it's marked for autoremoval in a couple weeks: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1102113 | 18:38 |
| fungi | though https://tracker.debian.org/pkg/openjdk-21 does not, so it'll probably be in the final release | 18:38 |
| clarkb | perfect | 18:38 |
| clarkb | also I didn't realize they would still remove package this late in the release process | 18:39 |
| fungi | 22 and 24 are also marked for autoremoval, 25 is not at present | 18:39 |
| fungi | well, it's a matter of which transitions are still in progress i guess | 18:40 |
| clarkb | fungi: that makes sense 21 and 25 are LTS releases but 22,23,24 are not | 18:40 |
| clarkb | oh and 25 doesn't properly release until september | 18:41 |
| clarkb | so 21 is the latest current released LTS until then and makes sens that is what trixie will ship with | 18:41 |
| fungi | per https://release.debian.org/testing/freeze_policy.html this is stil the soft freeze, though in theory the removals shouldn't have happened after the transition and toolchain freeze which ended a week ago | 18:41 |
| clarkb | upstream gerrit indicates running 3.11 on java 17 should be fine but we have to buidl our own wars (we do anyway) | 18:42 |
| fungi | adding and removing packages in trixie isn't supposed to be happening as of 2025-04-15 but i guess the release managers made an exception | 18:42 |
| fungi | maybe if they were already marked for autoremoval as of that date it was good enough | 18:43 |
| clarkb | also its java and I doubt many debian system tool rely on java so its not a huge deal to the distro? | 18:43 |
| fungi | right, it's there to support java-based applications that are packaged, primarily, as well as any downstream development i guess | 18:44 |
| fungi | the hound image upload is finally happening | 18:44 |
| clarkb | looking at the release notes a few other things stand out to me: the gerrit cookie will be set such that client side scripts can no longer access it. I think a good thing but may impact some users if they have client side scripts that rely on that. A per request cache that in some cases may corrupt repos is enabled by default. That seems like an easy disable on our side since | 18:52 |
| clarkb | performance seems good enough without it on 3.10. A new changes_by_project cache doesn't play nice with inconsistencies in repos like refs/meta with wrong gerrit serverids. The workaround is to disable this cache. I think we have a few changes that we know are inconsistent due to the sands of time so we may need to disable this cache post upgrade if we see problems. And finall | 18:52 |
| clarkb | change, revision, and ListFiles REST API objects no longer have etag values | 18:52 |
| clarkb | I am not aware of any rest api clients relying on etags but maybe gertty does? | 18:52 |
| clarkb | oh the per request ref cache is/was in 3.9 and 3.10 too | 18:56 |
| clarkb | so I guess we're alredy running that risk :/ | 18:56 |
| clarkb | and changes_by_project is there too and I don't think we'redisabling thiose so for now we can ignore those as upgrade todos I guess | 18:56 |
| clarkb | looking at https://issues.gerritcodereview.com/issues/309098227 it seems like maybe they think jgit largely fixed the issue with concurrency and the per request ref cache? | 18:58 |
| clarkb | fungi: looks like hound will go in after hourly runs at this point. I'm about to eat lunch but will check the service afterwards | 19:00 |
| fungi | sounds good | 19:00 |
| fungi | it takes a while to restart at deploy for reindexing too | 19:01 |
| opendevreview | Merged opendev/system-config master: Use podman to build non docker hub container images https://review.opendev.org/c/opendev/system-config/+/947872 | 19:06 |
| fungi | the image promote succeeded and bridge bootstrap is already in progress | 19:08 |
| fungi | and now codesearch is deploying | 19:09 |
| fungi | and it's restarting/reindexing | 19:11 |
| fungi | looks like it worked, will just take time to come back online | 19:11 |
| fungi | on that note, i'm gonna go grab something to eat for a bit | 19:11 |
| Clark[m] | It is up and returning search results again. I'll check the host directly in a bit | 19:32 |
| opendevreview | Merged opendev/zuul-providers master: Add ubuntu noble arm64 image https://review.opendev.org/c/opendev/zuul-providers/+/947841 | 20:04 |
| opendevreview | Merged opendev/zuul-providers master: Move labels-arm64 into labels.yaml https://review.opendev.org/c/opendev/zuul-providers/+/947984 | 20:04 |
| corvus | mnasiadka: huzzah! thanks! | 20:09 |
| fungi | Clark[m]: yeah, codesearch lgtm now | 20:11 |
| clarkb | yup looks good on the server itself too | 20:16 |
| clarkb | thinking maybe tomorrow we start rolling out changes that take review02 out of config management? | 20:18 |
| clarkb | that will have been ~3 days post switch. If we don't think that is long enough let me know and I can delay fruther | 20:19 |
| fungi | yeah, that'll be fine timing | 20:21 |
| fungi | if we haven't heard any complaints by now... | 20:22 |
| clarkb | things seem quiet today. I'm going to take the opportunity to pop out for a bike ride in a bit | 20:37 |
| opendevreview | Aurelio Jargas proposed zuul/zuul-jobs master: Remove Artifactory cleanup playbook https://review.opendev.org/c/zuul/zuul-jobs/+/947041 | 20:42 |
| opendevreview | Merged zuul/zuul-jobs master: Pin artifactory Docker image to 7.77.12 https://review.opendev.org/c/zuul/zuul-jobs/+/947685 | 20:51 |
| fungi | good idea. i took the opportunity to mow the lawn | 21:23 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!