Friday, 2023-04-21

-@gerrit:opendev.org- Tobias Urdin proposed: [zuul/zuul] 877587: web: add dark mode and theme selection https://review.opendev.org/c/zuul/zuul/+/87758711:24
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 881237: Release the thread lock when we delete a node https://review.opendev.org/c/zuul/nodepool/+/88123715:35
@jim:acmegating.comClark: fungi ^ this should fix the observed malbehavior in opendev15:35
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 881238: Reduce statemachine driver logging https://review.opendev.org/c/zuul/nodepool/+/88123815:37
@jim:acmegating.comi have finished copying all the images to quay.io, so i think we can make the switch in repo now15:46
@jim:acmegating.com * i have finished copying all the images to quay.io, so i think we can make the switch in the publishing jobs now15:47
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 881238: Reduce statemachine driver logging https://review.opendev.org/c/zuul/nodepool/+/88123815:52
@jim:acmegating.comClark: fungi ^ can you rereview that pls?15:52
@fungicide:matrix.orgi like the term "malbehavior," it seems perfectly cromulent15:53
@fungicide:matrix.org881238,2 lgtm. what was the problem with using f-strings?15:55
@jim:acmegating.comit's not ideal for logging -- if you screw up a log message with %s and arguments, then the logging handler is able to still produce a coherent error message.  but if you mess up f strings, the error isn't so great.  so trying to remember to keep using %s there15:57
@jim:acmegating.comnot important enough to go yank stuff out that's working, but whenever we touch it, probably a good idea15:57
@fungicide:matrix.orgmakes sense. so nothing inherently wrong with f-strings, they're just not robust enough for some logging edge cases16:12
@jim:acmegating.comyeah, i'll use 'em everywhere else16:13
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 881243: Publish container images to quay.io https://review.opendev.org/c/zuul/zuul/+/88124316:24
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 881243: Publish container images to quay.io https://review.opendev.org/c/zuul/zuul/+/88124316:29
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 881244: Publish container images to quay.io https://review.opendev.org/c/zuul/nodepool/+/88124416:31
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/nodepool] 881237: Release the thread lock when we delete a node https://review.opendev.org/c/zuul/nodepool/+/88123716:49
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-operator] 881245: Publish container images to quay.io https://review.opendev.org/c/zuul/zuul-operator/+/88124516:56
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-registry] 881247: Publish container images to quay.io https://review.opendev.org/c/zuul/zuul-registry/+/88124717:03
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-preview] 881249: Publish container images to quay.io https://review.opendev.org/c/zuul/zuul-preview/+/88124917:06
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-storage-proxy] 881250: Publish container images to quay.io https://review.opendev.org/c/zuul/zuul-storage-proxy/+/88125017:10
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/gcp-authdaemon] 881251: Publish container images to quay.io https://review.opendev.org/c/zuul/gcp-authdaemon/+/88125117:13
@jim:acmegating.comThat's the lot of them.17:14
@clarkb:matrix.orgcorvus: zuul-registry failed because it appears to still be trying to use docker jobs for some reason17:18
@clarkb:matrix.orgor docker playbooks/roles anyway17:18
@clarkb:matrix.orgoh it explicitly runs that playbook out of the local repo17:19
@clarkb:matrix.orgshould just be a matter of udpating the job I think17:19
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-jobs] 881252: Move containerfile setting in container build https://review.opendev.org/c/zuul/zuul-jobs/+/88125217:20
@jim:acmegating.comClark: thx, i'll look.  ^ that is why zuul-operator failed17:20
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-registry] 881247: Publish container images to quay.io https://review.opendev.org/c/zuul/zuul-registry/+/88124717:23
@clarkb:matrix.orgcorvus: storage proxy was similar to the registry17:24
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-storage-proxy] 881250: Publish container images to quay.io https://review.opendev.org/c/zuul/zuul-storage-proxy/+/88125017:24
@jim:acmegating.cominteresting.  i wonder if we'd write those jobs that way now...17:24
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-operator] 881245: Publish container images to quay.io https://review.opendev.org/c/zuul/zuul-operator/+/88124517:25
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/nodepool] 881238: Reduce statemachine driver logging https://review.opendev.org/c/zuul/nodepool/+/88123817:36
@jim:acmegating.comClark: ianw https://review.opendev.org/881244 is showing a possible issue related to the buildx/multi-arch path that i'm not in a position to track down, if you or anyone else wants to look into it19:28
@jim:acmegating.comi will recheck just to be sure, but i think that's likely a legit error19:28
@clarkb:matrix.orgcorvus: could that be related to the skopeo version thing? maybe skopeo isn't new enough there to understand the manifest?19:38
@jim:acmegating.comClark: no idea -- but i don't think we changed push-to-intermediate-registry19:39
@clarkb:matrix.orgcorvus: the skopeo that failed is on localhost which I believe is up to date19:44
@clarkb:matrix.orgoh it is failing when talking to the local registry19:45
@clarkb:matrix.orgcorvus: I think I see the issue (haven't look at code yet to know how/why though) compare the paths in https://zuul.opendev.org/t/zuul/build/8ce8d32ce1e54064ae6e5eaab13cf47f/log/job-output.txt#7996 to https://zuul.opendev.org/t/zuul/build/8ce8d32ce1e54064ae6e5eaab13cf47f/log/job-output.txt#10672 the buildset registry which we are trying to pull from has the image without the quay.io prefix but then when we try to send it to the intermediate registry it does20:01
@clarkb:matrix.orgPretty sure that is a valid 404 as a result20:01
@jim:acmegating.comyeah, compare to zuul pushing to the buildset registry; it has quay.io: https://zuul.opendev.org/t/zuul/build/20690c6ade924b05badee1f3987f867f/log/job-output.txt#1008020:06
@jim:acmegating.comso seems like we need to narrow it down to the buildx temporary registry->buildset registry area20:07
@clarkb:matrix.orgya I think I see it in the code.20:07
@clarkb:matrix.orgin build-container-image push.yaml we use zj_image.repository as a prefix but we don't in buildx.yaml20:08
@clarkb:matrix.orglet me work on a patch that adds that in20:08
@jim:acmegating.comkk, thx20:08
@clarkb:matrix.orgI think I introduced this issue since we didn't want the registry name in some places but do in others?20:09
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 881277: Use full image url in container buildx path https://review.opendev.org/c/zuul/zuul-jobs/+/88127720:16
@clarkb:matrix.orgcorvus: ^ I think that may fix it20:17
@jim:acmegating.comcool; i think the next challenge is the py27 job failing in zuul-jobs20:19
@jim:acmegating.comhttps://zuul.opendev.org/t/zuul/build/953e65f109664c5ea80ff6e3cd9943ac20:19
@clarkb:matrix.orgcorvus: the python27 issue is one that swift ran into as well. New(er) virtualenv execs a python script under what it thinks is a detected python interpreter for some reason and now that script includes f strings so python2.7 explodes20:31
@clarkb:matrix.orgswift fixed this by running their tox job on focal I think to get python3.8 which used older virtualenv? I can take a look at that for zuul jobs too to see if something makes sense20:31
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 881279: Pin virtualenv in tox environments https://review.opendev.org/c/zuul/zuul-jobs/+/88127920:40
@clarkb:matrix.orgI think that might work20:40
@jim:acmegating.comClark: can you change that comment to say "necessary for python 2.7"?20:42
@jim:acmegating.comso we know when to remove20:42
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 881279: Pin virtualenv in tox environments https://review.opendev.org/c/zuul/zuul-jobs/+/88127920:43
@jim:acmegating.comi approved that; the gate will tell us if it works20:45
@clarkb:matrix.orgthe py27 job did pass20:56
-@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [zuul/zuul-jobs] 881279: Pin virtualenv in tox environments https://review.opendev.org/c/zuul/zuul-jobs/+/88127920:57
@jim:acmegating.comi'm going to approve 881252 then21:23
@jim:acmegating.comrechecking the nodepool change21:23
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 877834: Add ensure-quay-repo role https://review.opendev.org/c/zuul/zuul-jobs/+/87783421:31
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul-jobs] 881252: Move containerfile setting in container build https://review.opendev.org/c/zuul/zuul-jobs/+/88125221:34
@clarkb:matrix.orgthe ensure-quay-repo role update is just a rebase. I'm trying to update it to mimic some of the other work we've done recently more closely21:38
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 877834: Add ensure-quay-repo role https://review.opendev.org/c/zuul/zuul-jobs/+/87783422:05
@clarkb:matrix.orgcorvus: ianw ^ thats a bit of a rewrite and I haven't manually tested it yet. BUt I realized as I was sketching out the system-config job updates that it would be nice to have the role work like this. I'll get the sketch of the system-config change up soon too22:05
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 877834: Add ensure-quay-repo role https://review.opendev.org/c/zuul/zuul-jobs/+/87783422:08
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 877834: Add ensure-quay-repo role https://review.opendev.org/c/zuul/zuul-jobs/+/87783422:10
@clarkb:matrix.orgapologies I'm failing at git22:10
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 881277: Use full image url in container buildx path https://review.opendev.org/c/zuul/zuul-jobs/+/88127722:26
@clarkb:matrix.orgthat fixed a merge conflict with the containerfile fix22:27
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 881287: Add Azure gallery image support https://review.opendev.org/c/zuul/nodepool/+/88128722:40
@clarkb:matrix.orgcorvus: I'm not sure how to deal with this failure https://zuul.opendev.org/t/zuul/build/310b4777d09549d580796a9ed4d0ae1e in the test suite for that change22:52
@clarkb:matrix.orgI guess maybe the port doesn't matter and I can just drop it?22:52
@clarkb:matrix.orgoh no except that is what defines where you push to in the end22:52
@clarkb:matrix.orgcan we/should we url encode there? I' mwondering why the non buildx promotion testing doesn't seem to have this problem22:56
@clarkb:matrix.orgoh it could be because we push everything into the multiarch registry to collate things together I think22:57
@clarkb:matrix.orghrm no this fails before that in the buildx command22:57
@clarkb:matrix.orgI think this works in the single arch case because we don't cover the buildset registry case there which would push to the buildset registry using an invalid tag specifier23:18
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 881277: Use full image url in container buildx path https://review.opendev.org/c/zuul/zuul-jobs/+/88127723:33
@clarkb:matrix.orgI don't expect ^ that to work, but if it does that may be a shorter term workaround. Basically we rely on localhost having the docker like registry rather than a hostname:port combo that can't be used in the urls23:33
@clarkb:matrix.orgBut we should probably come up with some scheme to support 127.0.0.1:5100 like hosts. Maybe replace : with _. I think the regular non multiarch side may need cleaning up too.23:34
@clarkb:matrix.orgWow that last patchset looks like it may actually work. I'm not sure I'm a fan but if it works and gets us moving forward we can figure out the existing issue with host:port in buildset registry paths later I guess23:40

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!