| @iwienand:matrix.org | ok i figured out the nodepool error, but not sure if it's boto or moto that's at fault. moto has stopped raising an error for deleted images, which it says is inline with the aws API, but then boto has this image that fails when you try to look at "image.state" | 01:47 |
|---|---|---|
| @iwienand:matrix.org | I have filed https://github.com/spulec/moto/issues/5067 but this isn't a great bug report, I know. I'm hoping there's enough there to narrow it down | 04:11 |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/nodepool] 839431: AWS tests : catch AttributeError raised by latest moto release https://review.opendev.org/c/zuul/nodepool/+/839431 | 04:19 | |
| @iwienand:matrix.org | Clark: the other option seems to be to leave the src values in, but pointed at the upstream CDN. modulo flags etc. if we like as well | 04:59 |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 839370: WIP: Fix a bug that re-enqueue may skip jobs unexpectly https://review.opendev.org/c/zuul/zuul/+/839370 | 05:59 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 839370: WIP: Fix a bug that re-enqueue may skip jobs unexpectly https://review.opendev.org/c/zuul/zuul/+/839370 | 06:50 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 839370: WIP: Fix a bug that re-enqueue may skip jobs unexpectly https://review.opendev.org/c/zuul/zuul/+/839370 | 07:23 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 839370: WIP: Fix a bug that re-enqueue may skip jobs unexpectly https://review.opendev.org/c/zuul/zuul/+/839370 | 07:50 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 839370: WIP: Fix a bug that re-enqueue may skip jobs unexpectly https://review.opendev.org/c/zuul/zuul/+/839370 | 08:29 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 839370: WIP: Fix a bug that re-enqueue may skip jobs unexpectly https://review.opendev.org/c/zuul/zuul/+/839370 | 08:51 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 839370: WIP: Fix a bug that re-enqueue may skip jobs unexpectly https://review.opendev.org/c/zuul/zuul/+/839370 | 09:21 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 839370: Fix a bug that re-enqueue may skip soft dependent jobs https://review.opendev.org/c/zuul/zuul/+/839370 | 09:57 | |
| @iwienand:matrix.org | corvus: i've mentioned you in https://github.com/spulec/moto/issues/5067#issuecomment-1110844418 as there's a question over if we saw the AWS API returning a state of "deleted", which would help the moto author decide if there might be a missing corner case. this check seems to have come in via https://review.opendev.org/c/zuul/nodepool/+/830527/3/nodepool/tests/unit/test_driver_aws.py | 10:39 |
| -@gerrit:opendev.org- Benjamin Schanzel proposed: [zuul/nodepool] 839514: Default limits for k8s labels and quota support https://review.opendev.org/c/zuul/nodepool/+/839514 | 11:40 | |
| @jpew:matrix.org | It's not clear to me how to add a release note. I see the `releasenotes/notes` directory, but I don't know how to come up with the SHA-1 suffix for the file | 13:17 |
| @jim:acmegating.com | jpew: https://zuul-ci.org/docs/zuul/latest/developer/releasenotes.html | 13:19 |
| @jpew:matrix.org | got it, thanks | 13:21 |
| -@gerrit:opendev.org- Benjamin Schanzel proposed: [zuul/nodepool] 839514: Default limits for k8s labels and quota support https://review.opendev.org/c/zuul/nodepool/+/839514 | 13:26 | |
| -@gerrit:opendev.org- Dominik Jaeger proposed: [zuul/zuul] 839550: Add spec for fileset feature https://review.opendev.org/c/zuul/zuul/+/839550 | 13:38 | |
| @fungicide:matrix.org | > <@iwienand:matrix.org> Clark: the other option seems to be to leave the src values in, but pointed at the upstream CDN. modulo flags etc. if we like as well | 13:49 |
| That seems sub-optimal. At the moment, every apt update in a job is pulling the deb-src indices from a configured local mirror, so simply keeping the deb-src lines and switching them to point to an official distro mirror would mean that every apt update pulls indices from outside the environment increasing risk of Internet-inflicted breakage. At least in OpenDev, what we really want is for jobs not to pull deb-src indices from anywhere by default, and then jobs can add deb-src lines for official distro mirrors in cases where they need source packages we're not mirroring. | ||
| @fungicide:matrix.org | (as already pointed out, this is how the role works for ubuntu nodes today, so the goal is to change the role in such a way that debian nodes work similarly to ubuntu) | 13:54 |
| @clarkb:matrix.org | Ya I'll push up a change that wraps it in a flag defaulted to true, then a child change that defaults to false we can land in a couple of weeks and after that a change to add the functionality to ubuntu as well | 15:25 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 16:17 | |
| - [zuul/zuul-jobs] 839592: Enable deb-src on Debian with a flag in configure-mirrors https://review.opendev.org/c/zuul/zuul-jobs/+/839592 | ||
| - [zuul/zuul-jobs] 839593: Switch enable_src_repos to False in configure-mirrors https://review.opendev.org/c/zuul/zuul-jobs/+/839593 | ||
| - [zuul/zuul-jobs] 839594: Add enable_deb_src_repos support for Ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/839594 | ||
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 839594: Add enable_deb_src_repos support for Ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/839594 | 16:19 | |
| @clarkb:matrix.org | corvus: ^ is that similar to what you had in mind after disucssion yesterday? I did leave one question on the first change that your input on would be helpful for | 16:19 |
| @jim:acmegating.com | Clark: replied | 16:28 |
| @clarkb:matrix.org | corvus: cool that makes sense to me too | 16:31 |
| @jim:acmegating.com | pushed 5.2.5 | 16:36 |
| @clarkb:matrix.org | corvus: something like https://etherpad.opendev.org/p/pijjODYCvIZD0GYpzjTx as the announcement? | 16:36 |
| @clarkb:matrix.org | I can send that as soon as 839592 lands if it looks good | 16:37 |
| @clarkb:matrix.org | fungi: ^ you may be interested in https://review.opendev.org/c/zuul/zuul-jobs/+/839592/ | 16:38 |
| @avass:vassast.org | I was going to ask if I should update test-mirror-workspace-git to do a base-test loop for: https://review.opendev.org/c/zuul/zuul-jobs/+/839225 but I noticed that the test role is quite old | 16:40 |
| @avass:vassast.org | should I update it and create a test-prepare-workspace-git, or does anyone wanna verify that ^ and +2 ? | 16:41 |
| @clarkb:matrix.org | Albin Vass: if you think it is testable via the testing in zuul-jobs directly that is probably preferable | 16:42 |
| @clarkb:matrix.org | but changes to mirror-workspace-git should probably be tested as wellas we can manage even if that means a base-test loop | 16:42 |
| @avass:vassast.org | yeah | 16:43 |
| @avass:vassast.org | mirror-workspace-git-repos hardcodes delegate_to: localhost so that would require a nested ansible until those limitations are removed :) | 16:44 |
| @jim:acmegating.com | fwiw, i think those limitations are going to be removed today :) | 16:44 |
| @avass:vassast.org | oh! | 16:44 |
| @avass:vassast.org | then I can wait :) | 16:44 |
| @avass:vassast.org | we're not in any hurry in any case | 16:44 |
| @jim:acmegating.com | as soon as the 5.2.5 release announcement is out, i'll start approving the ansible changes | 16:44 |
| @avass:vassast.org | (and if we we're we'd vendor/fork the role instead) | 16:45 |
| @avass:vassast.org | I have an idea how extend the 'roles:' import to pin roles to specific revisions using git worktrees btw. | 16:49 |
| @avass:vassast.org | * I have an idea to extend the 'roles:' import to pin roles to specific revisions using git worktrees btw. | 16:50 |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [zuul/zuul-jobs] 839592: Enable deb-src on Debian with a flag in configure-mirrors https://review.opendev.org/c/zuul/zuul-jobs/+/839592 | 16:58 | |
| @clarkb:matrix.org | Albin Vass: reading that change it isn't super clear to me why urlencoding before giving it to git would fix the issue if git is doing the translation | 17:00 |
| @avass:vassast.org | Clark: practically it adds another level of urlencoding so git decodes it to the original urlencoded value... | 17:01 |
| @clarkb:matrix.org | is zj_project.value.src_dir being converted by something before it gets to git? If so should we address this further upstream so that other users of that value don't have to urlencode too? | 17:01 |
| @clarkb:matrix.org | hrm isn't urlencoding a urlencoded url a noop? | 17:01 |
| @avass:vassast.org | src_dir is using urlencoding if `workspace-scheme: unique` is set: https://zuul-ci.org/docs/zuul/latest/config/job.html#value-job.workspace-scheme.unique | 17:03 |
| @avass:vassast.org | > <@clarkb:matrix.org> hrm isn't urlencoding a urlencoded url a noop? | 17:04 |
| `org/project` for example would urlencode to `org%2Fproject` which would in turn encode the percentage sign into `org%252Fproject`. At least I tested this locally and it seemed to produce the correct result | ||
| @clarkb:matrix.org | oh do things like the % get nested encoding? | 17:04 |
| @clarkb:matrix.org | ya ok its the % that gets escaped essentially since everything else should be safe after the first pass | 17:04 |
| @avass:vassast.org | but I can wait until ansible limitations are removed to set up a test case :) | 17:05 |
| @clarkb:matrix.org | Albin Vass: left a note. tldr is I think a comment in the code explaining some of this will be helpful otherwise it is largely magic | 17:07 |
| @avass:vassast.org | sure! | 17:07 |
| @jim:acmegating.com | release announcement sent; bombs away on ansible-relax. | 17:08 |
| @clarkb:matrix.org | corvus: does https://etherpad.opendev.org/p/pijjODYCvIZD0GYpzjTx look good to you? I can send that now that the change has landed | 17:13 |
| @jim:acmegating.com | Clark: lgtm! | 17:17 |
| @clarkb:matrix.org | corvus: cool sent. I think it may need to be moderated through? | 17:19 |
| @jim:acmegating.com | done | 17:21 |
| @clarkb:matrix.org | https://review.opendev.org/c/zuul/nodepool/+/839431 appears to be the fix for nodepool testing which would allow us to land some of those other changes | 17:36 |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed wip: [zuul/project-config] 839614: Add a zuul-publish-unversioned-tox-docs job https://review.opendev.org/c/zuul/project-config/+/839614 | 17:40 | |
| @fungicide:matrix.org | the new release reminded me we did ^ that for some opendev projects recently, if it's of interest | 17:45 |
| @jim:acmegating.com | fungi: nice; i'd love something like that, but i fear that the rewind will be a frequent occurence for us, so we should probably address that | 18:13 |
| @jim:acmegating.com | ianw: i ran through a full aws image upload/create/delete cycle and can not confirm that 'deleted' is a returned value. i updated the github issue accordingly. thanks for running that down. | 18:14 |
| @clarkb:matrix.org | if the job always build the latest/ build against HEAD of master that problem would be addressed right? | 18:20 |
| @clarkb:matrix.org | that probably implies two builds of the docs in the jobs. One for the tag and one for HEAD | 18:20 |
| @fungicide:matrix.org | Clark: yes, i think it's doable by adding a pre-run playbook to reset the repository state. does that seem like a sane approach? | 18:34 |
| @clarkb:matrix.org | fungi: I'm not sure if pre-run is correct if the job has to flip it over halfway through? | 18:34 |
| @clarkb:matrix.org | Or is the job for the tag separate | 18:35 |
| @fungicide:matrix.org | well, it's a separate job | 18:35 |
| @clarkb:matrix.org | ah then ya simply updating the checkout should do it | 18:35 |
| @fungicide:matrix.org | the idea is that the version-specific build and branch tip build would run independently | 18:35 |
| @clarkb:matrix.org | ya in that case it should be fine since they also publish to different locations | 18:36 |
| @clarkb:matrix.org | one to latest/ and the other to $tag/ | 18:36 |
| -@gerrit:opendev.org- Zuul merged on behalf of Ian Wienand: [zuul/nodepool] 839431: AWS tests : catch AttributeError raised by latest moto release https://review.opendev.org/c/zuul/nodepool/+/839431 | 18:39 | |
| @jpew:matrix.org | r.e. https://review.opendev.org/c/zuul/nodepool/+/833099 and a nodepool release: something came up and there isn't any need to hold a nodepool release for it (just in case you were waiting on it) | 18:58 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 839626: Delete AWS image builds https://review.opendev.org/c/zuul/nodepool/+/839626 | 19:01 | |
| @jim:acmegating.com | ianw: ^ i noticed that during my real-world testing. | 19:02 |
| @jim:acmegating.com | jpew: thanks. i think we can add that change any time. | 19:02 |
| @clarkb:matrix.org | ianw left notes on the zuul json logging stack. | 19:26 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 837418: Remove restricted Ansible documentation https://review.opendev.org/c/zuul/zuul/+/837418 | 20:04 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/nodepool] 836013: Add support for label quota https://review.opendev.org/c/zuul/nodepool/+/836013 | 20:36 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 839639: Fix updating change when emulating gerrit topic support https://review.opendev.org/c/zuul/zuul/+/839639 | 20:49 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/zuul] 839321: Don't try to update deps of non-change refs https://review.opendev.org/c/zuul/zuul/+/839321 | 20:49 | |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 837419: Remove ansible plugin tests https://review.opendev.org/c/zuul/zuul/+/837419 | 21:24 | |
| @jim:acmegating.com | to save anyone else the trouble of looking up the paramiko/blowfish warnings currently spammed by the tests: https://github.com/paramiko/paramiko/pull/2039 | 21:34 |
| @jpew:matrix.org | OK: I finally sorted out whats going on here, but I can't figure out what the best path forward. It turns out that https://review.opendev.org/c/zuul/nodepool/+/833099 is actually not strictly necessary for us, because `ignore-project-quota` actually does the thing we want (keeps retrying the node allocation on quota error)..... except it doesn't because https://review.opendev.org/c/zuul/nodepool/+/839226 causes it to raise a _different_ exception when a quota error occurs so none of the retrying occurs and the node allocation just fails instead. I'm not sure what to do here, https://review.opendev.org/c/zuul/nodepool/+/839226 is sufficent for my use case, but also means a change in behavior to what happens on quota failures from openstack, which might be an unexpected (unwelcome?) change? basically right now I don't think any of the handling for nodes in the zk.ABORTED state ever activates | 21:36 |
| @jpew:matrix.org | * OK: I finally sorted out whats going on here, but I can't figure out what the best path forward. It turns out that https://review.opendev.org/c/zuul/nodepool/+/833099 is actually not strictly necessary for us, because `ignore-provider-quota` actually does the thing we want (keeps retrying the node allocation on quota error)..... except it doesn't because https://review.opendev.org/c/zuul/nodepool/+/839226 causes it to raise a _different_ exception when a quota error occurs so none of the retrying occurs and the node allocation just fails instead. I'm not sure what to do here, https://review.opendev.org/c/zuul/nodepool/+/839226 is sufficent for my use case, but also means a change in behavior to what happens on quota failures from openstack, which might be an unexpected (unwelcome?) change? basically right now I don't think any of the handling for nodes in the zk.ABORTED state ever activates | 21:37 |
| @jpew:matrix.org | Granted, it probably doesn't make sense for OpenStack to have different behavior on Quota errors than another provider (which just looks like OpenShift ATM) | 21:41 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: | 21:46 | |
| - [zuul/zuul] 837436: Remove unused remote module tests https://review.opendev.org/c/zuul/zuul/+/837436 | ||
| - [zuul/zuul] 837420: Remove unused ansible modules https://review.opendev.org/c/zuul/zuul/+/837420 | ||
| @iwienand:matrix.org | Clark: thanks, i've responded to hopefully clear up what the path is there | 22:34 |
| @clarkb:matrix.org | i responded hopefully that makes sense? Basically I don't think we need to check whether or not we enter the block, just that if a string is provided instead of a dict and no_log is set that we don't leak the strings out | 22:43 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/nodepool] 839626: Delete AWS image builds https://review.opendev.org/c/zuul/nodepool/+/839626 | 22:58 | |
| @iwienand:matrix.org | Clark: i can add a no_log test to the follow-on. thinking about it, we might have enough coverage here to convince ourselves that we can actually remove all the no_log handling from the json callback -- because at this point, the callback never actually gets info it shouldn't log | 23:19 |
| @clarkb:matrix.org | ya thats possible if the test shows we're not exposing stuff anyway. However, for that case I think we should ensure the test is running with each of the ansible versions we support since it may differ betwene them | 23:20 |
| @iwienand:matrix.org | yeah this has all been fixed i think since 2.5 era | 23:21 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 838933: Add a gearman-style backoff for executor job starts https://review.opendev.org/c/zuul/zuul/+/838933 | 23:22 | |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 837421: Remove localhost restriction from command.py https://review.opendev.org/c/zuul/zuul/+/837421 | 23:31 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul] 838971: Add unit test for zuul_json text result parsing https://review.opendev.org/c/zuul/zuul/+/838971 | 23:46 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!