| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: | 16:08 | |
| - [zuul/zuul] 867900: Pin tox to 3 https://review.opendev.org/c/zuul/zuul/+/867900 | ||
| - [zuul/zuul] 867057: Add noxfile and switch to nox https://review.opendev.org/c/zuul/zuul/+/867057 | ||
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 868229: Switch to nox-docs https://review.opendev.org/c/zuul/zuul/+/868229 | 16:08 | |
| @clarkb:matrix.org | ianw: I updated the zuul annual report draft to try and capture the pf4 suggestion | 17:13 |
|---|---|---|
| @jim:acmegating.com | Clark: i believe it's "unpin tox in zuul-jobs" day | 20:45 |
| @jim:acmegating.com | did we have a change for that already? | 20:46 |
| @jim:acmegating.com | yes - https://review.opendev.org/866943 | 20:46 |
| @clarkb:matrix.org | yup looks like ou found it | 20:47 |
| @clarkb:matrix.org | and that should be self testing that things generally work if you've got a working config | 20:48 |
| @clarkb:matrix.org | corvus: I just removed my -W if you want to send it in | 20:48 |
| @jim:acmegating.com | done | 20:49 |
| @jim:acmegating.com | it looks like the nox docs promote job worked for zuul, so i think that's confirmation that all the nox stuff is working for the zuul repo | 20:50 |
| @clarkb:matrix.org | corvus: https://zuul.opendev.org/t/zuul/build/7a55918c47a145c582b9f22e8a4c80a6 this passed the other day and I thought I had corrected that | 21:16 |
| @clarkb:matrix.org | by removing tox-bindep | 21:17 |
| @jim:acmegating.com | Clark: https://review.opendev.org/866945 | 21:19 |
| @jim:acmegating.com | Clark: i can't quite put together why it passed before... | 21:20 |
| @clarkb:matrix.org | my guess is that they released newer tox v4 that doesn't bootstrap the specified tox version for the test env via requires properly | 21:20 |
| @clarkb:matrix.org | I think we can flip the order of the two changes and land it that way though | 21:21 |
| @jim:acmegating.com | yes; or squash | 21:21 |
| @jim:acmegating.com | (but flip might be better just to keep things clear?) | 21:21 |
| @clarkb:matrix.org | ++ to flip | 21:21 |
| @clarkb:matrix.org | do you want ot do that or should I? | 21:21 |
| @jim:acmegating.com | you -- unless you're away from a terminal then i can | 21:22 |
| @clarkb:matrix.org | I can, I just have to load my keys | 21:22 |
| @clarkb:matrix.org | haven't needed to do that yet today | 21:22 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 866943: Flip ensure-tox to tox v4 by default https://review.opendev.org/c/zuul/zuul-jobs/+/866943 | 21:24 | |
| -@gerrit:opendev.org- Clark Boylan proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul-jobs] 866945: Avoid tox-bindep in test-tox job https://review.opendev.org/c/zuul/zuul-jobs/+/866945 | 21:24 | |
| @clarkb:matrix.org | corvus: ^ if you want to reapprove | 21:24 |
| @clarkb:matrix.org | nevermind it preserved votes so I went ahead and +Ad | 21:25 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 868438: Switch to nox https://review.opendev.org/c/zuul/nodepool/+/868438 | 21:32 | |
| @clarkb:matrix.org | corvus: some of those nox sessions for nodepool run zuul manage-ansible is that intentional? | 21:35 |
| @clarkb:matrix.org | also those functional job playbooks might need to run nox? | 21:37 |
| @jim:acmegating.com | derp | 21:40 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 868438: Switch to nox https://review.opendev.org/c/zuul/nodepool/+/868438 | 21:41 | |
| @jim:acmegating.com | Clark: thx :) | 21:41 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-client] 868440: Switch to nox https://review.opendev.org/c/zuul/zuul-client/+/868440 | 22:04 | |
| @jim:acmegating.com | ianw: does this failure mean anything to you? https://zuul.opendev.org/t/zuul/build/3a2808d00b1f4cc49a5808c81d7e99cb | 22:17 |
| @iwienand:matrix.org | corvus: i guess yes in so far as it seems cri-dockerd isn't starting, which is the serivce that maps docker->cri-o api, but i'm not sure why that would be | 22:19 |
| @iwienand:matrix.org | unfortunately we don't collect the journal logs that would be required to see what that is i don't think | 22:20 |
| @iwienand:matrix.org | https://zuul.opendev.org/t/zuul/build/3a2808d00b1f4cc49a5808c81d7e99cb/console#1/0/125/ubuntu-jammy looks good | 22:21 |
| @iwienand:matrix.org | https://github.com/Mirantis/cri-dockerd/releases | 22:21 |
| @iwienand:matrix.org | 2 days ago | 22:21 |
| @iwienand:matrix.org | i would *strongly* suspect that ... | 22:22 |
| @jim:acmegating.com | ah | 22:22 |
| @iwienand:matrix.org | although really it doesn't seem like much in that release ... but ... i don't know sorry | 22:22 |
| @jim:acmegating.com | there's a very specific docker version we install: https://opendev.org/zuul/nodepool/src/branch/master/playbooks/nodepool-functional-k8s/pre.yaml#L8 | 22:22 |
| @iwienand:matrix.org | IMO, as expressed in https://lists.zuul-ci.org/archives/list/zuul-discuss@lists.zuul-ci.org/thread/GGBXGLA5GGF5QDPVL6X6J2TC3ZCYZ5PH/, debugging this is not particularly fruitful | 22:25 |
| @jim:acmegating.com | ianw: you reckon the microk8s switch is the way to proceed? | 22:25 |
| @iwienand:matrix.org | well, yes, my thoughts are above, although i admit that's very long :) | 22:26 |
| @iwienand:matrix.org | that came out of me having to debug more or less the same thing, with the cri-dockerd dameon not working with jammy, which in that case turned out to be paths of packages and config files moving. that may very well have regressed | 22:28 |
| @jim:acmegating.com | well, this problem is new, but perhaps similar/related to what you were addressing before. so i'm just verifying that i correctly understand your position is that the problems are similar enough that you believe the microk8s move is also the best way to proceed now. :) | 22:29 |
| @iwienand:matrix.org | yeah -- if we investigate this is likely, but not for sure, something like an incorrect path, or missing package or something like that. but we keep hitting these problems going down this fairly winding path, and I can't see we won't keep hitting them. I can't claim that microk8s will never break, but it does seem to be the most supported path for a simple k8s deploy on ubuntu. | 22:32 |
| @jim:acmegating.com | ianw: thanks. i'm going to rebase your nodepool changes -- they conflict with master | 22:34 |
| @iwienand:matrix.org | ++ | 22:35 |
| @iwienand:matrix.org | i have to run off for christmas travel soon, so feel free to just do anything to any of my changes :) | 22:35 |
| @jim:acmegating.com | ianw: oh sorry i was mistaken, they're fine | 22:37 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 868438: Switch to nox https://review.opendev.org/c/zuul/nodepool/+/868438 | 22:39 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Ian Wienand: | 22:39 | |
| - [zuul/nodepool] 866955: nodepool-functional-k8s: switch to microk8s https://review.opendev.org/c/zuul/nodepool/+/866955 | ||
| - [zuul/nodepool] 867745: Unpin microk8s https://review.opendev.org/c/zuul/nodepool/+/867745 | ||
| @jim:acmegating.com | well, in combination with my nox change, they conflict, so i did rebase | 22:39 |
| @iwienand:matrix.org | actually i just noticed a bug in https://review.opendev.org/c/zuul/zuul-jobs/+/867063/32/roles/use-buildset-registry/tasks/microk8s-mirror.yaml | 22:40 |
| @jim:acmegating.com | Clark: https://zuul.opendev.org/t/zuul/build/473dd7dcc5e1499f8dbeff926fb41e32 is unhappy -- the tox-bindep change... :/ | 22:40 |
| @iwienand:matrix.org | line 24 should't hard-code quay.io, but use the loop variable | 22:40 |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 867063: use-buildset-registry: support microk8s https://review.opendev.org/c/zuul/zuul-jobs/+/867063 | 22:41 | |
| @clarkb:matrix.org | corvus: I think you have to use -- to separate posargs now. I'll write a fi | 22:44 |
| @clarkb:matrix.org | corvus it almost looks like it is running under tox v4 though | 22:48 |
| @clarkb:matrix.org | maybe not the logging is wrong | 22:48 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 866943: Flip ensure-tox to tox v4 by default https://review.opendev.org/c/zuul/zuul-jobs/+/866943 | 22:49 | |
| -@gerrit:opendev.org- Clark Boylan proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul-jobs] 866945: Avoid tox-bindep in test-tox job https://review.opendev.org/c/zuul/zuul-jobs/+/866945 | 22:49 | |
| @clarkb:matrix.org | I think ^ that will address it | 22:49 |
| @clarkb:matrix.org | hrm https://zuul.opendev.org/t/zuul/build/6e166bf5750a44f3b9c9c2cd44345393/log/job-output.txt is failing in the child now the same way it did before restacking. But with the different deps | 23:00 |
| @clarkb:matrix.org | this almost looks like tox is broken | 23:01 |
| @jim:acmegating.com | Clark: yeah, or -- tox3 can't handle installing tox4 in the venv? | 23:03 |
| @clarkb:matrix.org | ya looks like ensure-tox is installing tox 3.28.0 as expected. But then later we're ending up with tox 4.0.16 somehow? | 23:09 |
| @clarkb:matrix.org | the file path the exception is occuring in is not found in 3.28.0 so it is definitely updating tox somewhere | 23:10 |
| @clarkb:matrix.org | ".tox installdeps: requestsexceptions, tox >= 3.2" | 23:12 |
| @clarkb:matrix.org | maybe that is it and we aren't getting better logging than that | 23:12 |
| @clarkb:matrix.org | I can reproduce this locally | 23:15 |
| @clarkb:matrix.org | it does indeed install a tox 4.0.16 | 23:15 |
| @jim:acmegating.com | Clark: yeah, if tox3 can't handle installing tox4 in the recursive venv, then i think this feature is just broken with tox3. maybe we should squash them and merge together? | 23:16 |
| @jim:acmegating.com | oh, but the job fails on the second change too? | 23:16 |
| @clarkb:matrix.org | well its failing on the parent too. I think tox v4 is just broken | 23:16 |
| @clarkb:matrix.org | er child not parent | 23:16 |
| @jim:acmegating.com | failing on both changes in the same way -- so tox3 installing tox4 is broken, and tox4 installing tox4 is broken | 23:17 |
| @jim:acmegating.com | Clark: i think the only reasonable thing is to comment out this test? | 23:18 |
| @clarkb:matrix.org | in the tox v4 case it doesn't seem to install a nested tox | 23:18 |
| @jim:acmegating.com | ah ok | 23:18 |
| @clarkb:matrix.org | I think maybe we can drop the minversion requirement | 23:19 |
| @clarkb:matrix.org | let me test that | 23:19 |
| @clarkb:matrix.org | no that doesn't work because anytime you use the requires directive it seems to install latest tox | 23:21 |
| @clarkb:matrix.org | this is independent of our minversion. And latest tox seem sto be broken on something | 23:21 |
| @clarkb:matrix.org | so ya I think we comment it out | 23:21 |
| @clarkb:matrix.org | oh wait | 23:22 |
| @clarkb:matrix.org | I'm doing some hacky print debugging and the %s might be to blame? | 23:23 |
| @clarkb:matrix.org | yes replacing the `testenv:%s` "fixes" it | 23:23 |
| @clarkb:matrix.org | I think tox v4 may have made that an invalid target | 23:23 |
| @clarkb:matrix.org | they match targets against this regex: `re.compile('!?[\\w._][\\w._-]*')` and the % is breaking that? | 23:24 |
| @jim:acmegating.com | why is there a literal "%S" ? | 23:24 |
| @clarkb:matrix.org | I don't know | 23:24 |
| @clarkb:matrix.org | `30bc5d67ff96d960114245cfa785097e937e7835` do not interpolate values from show config | 23:25 |
| @clarkb:matrix.org | its part of some siblings thing that tox seems to have made invalid | 23:25 |
| @jim:acmegating.com | Clark: but this is failing in "Run tox with empty envlist" not "Run tox with tox_envlist = %s" right? | 23:25 |
| @clarkb:matrix.org | correct, but it appears to be listing all the targets internally when you don't give it a target and then detecting this is broken due to the regex match | 23:25 |
| @clarkb:matrix.org | I think we can drop the %s since tox doesn't allow it to be valid under latest v4? | 23:26 |
| @jim:acmegating.com | sounds good | 23:26 |
| @clarkb:matrix.org | hrm | 23:26 |
| @clarkb:matrix.org | its fine when you target it directly to make this even more confusing. SO this is almost certainly a tox v4 bug | 23:26 |
| @clarkb:matrix.org | that they will say is not a bug beuase you should never use a % in your env name | 23:27 |
| @clarkb:matrix.org | we use RawConfigParser in siblings now though so this should never be reintroduced unless we stop doing that. I think the risk is low and we should just drop the %s target and test. I'll write that change | 23:28 |
| @jim:acmegating.com | Clark: sounds good to me. basically that's the important part of the change and the %s was just a way to test it via side-effects. i don't know if there's a better way, but clearly that one won't work anymore. | 23:30 |
| @clarkb:matrix.org | that target was also used to test specifying a specific file path so I'm working out how to preserve that then will push updates | 23:32 |
| @clarkb:matrix.org | Also I think tox 4.0.15 is what broke this | 23:32 |
| @clarkb:matrix.org | they are up to 4.0.16 now so relatively recent | 23:32 |
| @clarkb:matrix.org | (I'm a bit surprised that always install latest tox is what they want though basically means you can't pin tox if there is a problem) | 23:32 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 23:40 | |
| - [zuul/zuul-jobs] 866943: Flip ensure-tox to tox v4 by default https://review.opendev.org/c/zuul/zuul-jobs/+/866943 | ||
| - [zuul/zuul-jobs] 868451: Stop trying to use a %s tox target https://review.opendev.org/c/zuul/zuul-jobs/+/868451 | ||
| -@gerrit:opendev.org- Clark Boylan proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul-jobs] 866945: Avoid tox-bindep in test-tox job https://review.opendev.org/c/zuul/zuul-jobs/+/866945 | 23:40 | |
| @clarkb:matrix.org | I stuck the fix for this at the very bottom as I think it should work that way | 23:40 |
| @jim:acmegating.com | Clark: should we approve those and let the gate handle it? | 23:49 |
| @clarkb:matrix.org | the fix change is almost done running that one job maybe confirm it is green first? | 23:49 |
| @jim:acmegating.com | k | 23:49 |
| @clarkb:matrix.org | I think it will be | 23:49 |
| @clarkb:matrix.org | yup it just reported success. I think you can approve things | 23:49 |
| @jim:acmegating.com | done, thanks! | 23:50 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!