Wednesday, 2022-01-19

-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Ian Wienand: [zuul/zuul] 825190: autohold: use canonical project name in REST interface https://review.opendev.org/c/zuul/zuul/+/82519000:02
@jim:acmegating.comianw: mind if i add test coverage to your change? ^00:02
@jim:acmegating.comClark: ^00:02
@iwienand:matrix.orgcorvus: thanks :)00:05
@clarkb:matrix.orgapproved00:20
@fzzfh:matrix.org> <@clarkb:matrix.org> You are probably best off asking the OpenStack QA team about this as those jobs and their inputs are things they help out with01:49
ok, thks!
-@gerrit:opendev.org- Zuul merged on behalf of Ian Wienand: [zuul/zuul] 825190: autohold: use canonical project name in REST interface https://review.opendev.org/c/zuul/zuul/+/82519003:02
-@gerrit:opendev.org- Benjamin Schanzel proposed: [zuul/nodepool] 825129: Fix tenant-quota for providers without label quota infos https://review.opendev.org/c/zuul/nodepool/+/82512907:36
-@gerrit:opendev.org- Dominik Jaeger proposed: [zuul/zuul] 823349: Allow to specify MySQL port for test environment https://review.opendev.org/c/zuul/zuul/+/82334908:38
-@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 825270: Ignore non-serializable objects in JobData https://review.opendev.org/c/zuul/zuul/+/82527008:40
-@gerrit:opendev.org- chandan kumar proposed: [zuul/zuul-jobs] 822503: Introduce iptables_package var https://review.opendev.org/c/zuul/zuul-jobs/+/82250309:41
-@gerrit:opendev.org- Dominik Jaeger marked as active: [zuul/zuul] 823349: Allow to specify MySQL port for test environment https://review.opendev.org/c/zuul/zuul/+/82334912:07
-@gerrit:opendev.org- Dominik Jaeger marked as active: [zuul/zuul] 823317: Make ZooKeeper wait for certs in test environment https://review.opendev.org/c/zuul/zuul/+/82331712:08
@d-j-j:matrix.orgHello everyone, I'm Dominik and just joined. I'm working for Nokia and we are using zuul more and more as our recommended CI system for new projects. We are still using an internal fork of Zuul V2 and Jenkins and are currently evaluating an upgrade to latest Zuul. We are looking to contribute a number of features to upstream which we implemented in our V2 fork over the last years. For a start I pushed 2 minor changes (see above) for the test environment which I came across during initial setup for zuul development. Any reviews would be appreciated and looking forward to working together to improve zuul :)12:14
@jim:acmegating.comd-j-j: welcome!  :)14:46
@d-j-j:matrix.orgthanks for the quick reviews :)15:02
-@gerrit:opendev.org- Zuul merged on behalf of Szymon Datko: [zuul/zuul-jobs] 825118: Add support for RPM packages https://review.opendev.org/c/zuul/zuul-jobs/+/82511815:45
-@gerrit:opendev.org- Zuul merged on behalf of Dominik Jaeger: [zuul/zuul] 823349: Allow to specify MySQL port for test environment https://review.opendev.org/c/zuul/zuul/+/82334915:56
-@gerrit:opendev.org- Zuul merged on behalf of chandan kumar: [zuul/zuul-jobs] 822503: Introduce iptables_package var https://review.opendev.org/c/zuul/zuul-jobs/+/82250315:59
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 824043: Re-elect James Blair as project lead https://review.opendev.org/c/zuul/zuul/+/82404316:23
-@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/zuul] 825270: Ignore non-serializable objects in JobData https://review.opendev.org/c/zuul/zuul/+/82527016:25
@clarkb:matrix.orgThe zuul client --reason arg on holds doesn't seem to accept quoted multi word strings? I'll look into that after I'm done debugging the focal stream functional tests I guess16:36
@clarkb:matrix.orgOh nevermind the issue is that zuul-client on our system is a wrapper shell script so you hve to double quote it16:41
-@gerrit:opendev.org- Zuul merged on behalf of Benjamin Schanzel: [zuul/nodepool] 823389: Export current tenant limit stats https://review.opendev.org/c/zuul/nodepool/+/82338916:56
@clarkb:matrix.orgcorvus: looking at the zuul stream functional thing I think the issue is that zuul isn't installed to the ansible version specific virtualenv and system site packages are disabled on the venv. That means that from zuul.ansible import paths doesn't find the global zuul install made by the job. What isn't clear to me is why this would work on Bionic or on the debian bullseye based zuul executor image. I guess next step is to look at the executor image17:06
@clarkb:matrix.orgI see you ran into a zuul restart problem. Ignore me on ^ for now it isn't urgent and I'm half thinking out loud17:07
@jim:acmegating.comClark: ack17:08
@jim:acmegating.comi was just performing a rolling restart of zuul for opendev, and ran into an issue17:09
@jim:acmegating.com```2022-01-19 17:05:32,280 ERROR zuul.Scheduler:   AttributeError: 'FrozenJob' object has no attribute 'config_hash'```17:09
@jim:acmegating.comi suspect that's fallout from https://review.opendev.org/824864 where we can have FrozenJobs already written to zk and when we deserialize them we don't load the config_hash, so the newer scheduler doesn't have the attribute present in order to re-serialize it.17:10
@jim:acmegating.comi think that change probably either needed some backwards-compat handling, or a reno saying to delete-state17:10
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825349: Add backwards compat handling for FrozenJob https://review.opendev.org/c/zuul/zuul/+/82534917:15
@jim:acmegating.comfungi: Clark tobiash felixedel : ^ we can either quickly merge that and then resume the opendev rollout, or we can add a release note to delete-state.17:16
@clarkb:matrix.orgreviewing now17:16
@jim:acmegating.com(and convert the opendev rollout to a full stop/start)17:16
@clarkb:matrix.orgcorvus: does the config_hash lookup need to use the _ prefix?17:18
@jim:acmegating.comno, is only for the job_data_attributes17:18
@jim:acmegating.com(or another way of double checking: hasattr should match getattr)17:18
@clarkb:matrix.orgah yup on line 199617:19
@fungicide:matrix.orglgtm, though more generally i guess there's not a reliable way to test for non-backwards-compatible changes in our serialization?17:20
@jim:acmegating.comoh i just realized, i think the job_data_attributes check may not be required; see the comment i just left17:20
@fungicide:matrix.orgi have a feeling this sort of bug is going to continue to crop up now that we persist state across restarts17:21
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 825349: Add backwards compat handling for FrozenJob https://review.opendev.org/c/zuul/zuul/+/82534917:21
@jim:acmegating.comfungi: yes.  https://review.opendev.org/825022 is a start at trying to improve that, but that will require writing good tests.  however, at least it's a framework where we can actually write those tests.17:22
@fungicide:matrix.orgoh, awesome. thanks! i missed that one17:22
@jim:acmegating.comthis *is* an error that would have been caught by the kind of test i would expect to write in that frame work (https://review.opendev.org/825023 is an example change)17:22
@jim:acmegating.comfungi: Clark i pushed a new patchset on top of the approval; can you re-review https://review.opendev.org/825349 ?17:23
@clarkb:matrix.orgyup done17:23
@jim:acmegating.comcool, i will await the merge from that and then resume the rollout.17:25
@clarkb:matrix.orgI think zuul-executor sets PYTHONPATH to make this work17:30
@clarkb:matrix.orgThe executor jobs set PYTHONPATH to `var/lib/zuul/ansible/2.9` in opendev which is bind mounted and contains the zuul ansible bits. I'm not sure what populates that as it isn't in our container image and gets bind mounted in. BUT that seems to have the content necessary to make `from zuul.ansible import paths` work.17:33
@clarkb:matrix.orgyup ok I think I may have found this17:34
@clarkb:matrix.org * yup ok I think I may have found the fix17:34
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 825152: DNM Testing the functional zuul stream jobs https://review.opendev.org/c/zuul/zuul/+/82515217:37
@jim:acmegating.comClark: just to expand a bit: even though it's bind-mounted, i think it's populated by the executor on startup (so it's not a situation where opendev is doing something external). 17:37
@clarkb:matrix.orgcorvus: ah ok17:37
@clarkb:matrix.organyway the trick seems to have been related to python path and 825152 should confirm. If that goes green I'll update the actual change. Then we can rebase some of the python 3.9 changes on top of the focal switch change17:38
@jim:acmegating.comClark: you fix makes sense... maybe adding a comment for next time would be a good idea :)17:38
@clarkb:matrix.org++ I'll make the actual chagne that is stacked atop the gearman removal a bit more clear17:39
@clarkb:matrix.orgGrepping the PYTHONPATH value out of the ze01 executor debug logs in opendev is what made me realize what was going on there17:39
@clarkb:matrix.orgI'm really grateful to whoever decided to log that value :)17:39
@tobias.henkel:matrix.orgcorvus: I agree, the backwards compat handling is better than reno :)17:42
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 825140: Bump test nodes up to Focal https://review.opendev.org/c/zuul/zuul/+/82514017:50
@clarkb:matrix.orgI expect ^ to pass now and that can be landed with the gearman removal17:50
@jim:acmegating.comClark: +2 with note17:54
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 825140: Bump test nodes up to Focal https://review.opendev.org/c/zuul/zuul/+/82514017:58
@clarkb:matrix.org> <@jim:acmegating.com> Clark: +2 with note17:58
Updated
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-website] 825389: Fix typo in get started page https://review.opendev.org/c/zuul/zuul-website/+/82538919:42
@clarkb:matrix.orgA couple minor things I Noticed on the get started page ^19:42
-@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [zuul/zuul-website] 825389: Fix typo in get started page https://review.opendev.org/c/zuul/zuul-website/+/82538920:11
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 825349: Add backwards compat handling for FrozenJob https://review.opendev.org/c/zuul/zuul/+/82534920:13
@jim:acmegating.comopendev zuul is at current master: d304f4134f05fa08aab70e9add6ec490370dc6e221:48
@jim:acmegating.comif no issues crop up by tomorrow, i say we tag that as 4.12.021:49
@jim:acmegating.comand then start landing the 5.0 changes (gearmanectomy)21:49
@clarkb:matrix.org++ and exciting21:51
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 821711: Add IBM Cloud VPC driver https://review.opendev.org/c/zuul/nodepool/+/82171123:15

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!