Wednesday, 2023-05-17

@clarkb:matrix.orgWorks for me00:23
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 883318: Report early failure from Ansible task failures https://review.opendev.org/c/zuul/zuul/+/88331800:47
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 883319: Refresh builds even less https://review.opendev.org/c/zuul/zuul/+/88331901:07
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 874047: Move statemachine node init into TPE https://review.opendev.org/c/zuul/nodepool/+/87404703:43
@jjbeckman:matrix.org> <@fungicide:matrix.org> good point, https://zuul-ci.org/docs/zuul/latest/drivers/github.html#configure-github is missing an explanation for why you might choose one option over the other. since i don't really manage any projects on github i wouldn't be confident in my explanation of the effective differences/limitations for those options, so best if someone else who does use github comes up with a paragraph we can add there04:50
Thank you for your comment. Yeah, I think some folks will find that information in the official documentation useful.
@jjbeckman:matrix.org> <@jim:acmegating.com> jjbeckman: there are additional features available with the app method, such as support for the github checks api, and it can support higher traffic since it has increased rate limits.  i have a configurator at https://acmegating.com/acme-enterprise-zuul/#start you can use to get my recommended configuration (everything there is also in the zuul docs, but is less opinionated).04:55
Thank you for the information. I will read up on how Zuul uses the GitHub Checks API and try to understand what additional features it provides.
@rancher:matrix.org> <@clarkb:matrix.org> I don't use Zuul with gitlab so don't know off the top of my head. But it would probably be helpful if you can share how it is failing under 15.11.05:36
It loads the configuration from the config project just fine (no errors in the logs), but there's no pipeline on the Zuul web interface. When opening the config project AND the default zuul/zuul-jobs project from opendev.org, the following error shows up: https://privatebin.net/?020eaac57c8b86be#2mUGMWmvTa6xUGnh2rogcSQMwZ6yJrU8swtfra7eSmtM
Another untrusted project that I have on the same GitLab server works (the basic config from the "Test Zuul Pipelines" section of the Quickstart Guide).
-@gerrit:opendev.org- Tobias Henkel proposed: [zuul/nodepool] 883329: Add timing information to "Node is ready" message https://review.opendev.org/c/zuul/nodepool/+/88332906:39
-@gerrit:opendev.org- Benedikt Löffler proposed: [zuul/nodepool] 883335: Log time how long the import-snapshot takes https://review.opendev.org/c/zuul/nodepool/+/88333508:14
@tony.breeds:matrix.org> <@iwienand:matrix.org> basically update https://opendev.org/zuul/zuul-website/src/branch/master/www/.htaccess for zuul-jobs, and make all the zuul-jobs links be zuul-jobs/latest/11:24
Wow, Element only just chose to show me this now,
@tony.breeds:matrix.org@ianw: Looks like it's already been fixed.11:26
@rancher:matrix.org> <@rancher:matrix.org> It loads the configuration from the config project just fine (no errors in the logs), but there's no pipeline on the Zuul web interface. When opening the config project AND the default zuul/zuul-jobs project from opendev.org, the following error shows up: https://privatebin.net/?020eaac57c8b86be#2mUGMWmvTa6xUGnh2rogcSQMwZ6yJrU8swtfra7eSmtM12:32
>
> Another untrusted project that I have on the same GitLab server works (the basic config from the "Test Zuul Pipelines" section of the Quickstart Guide).
I tried the same config files on gitlab.com (v16.0), and the same error appears when I open the same projects.
@clarkb:matrix.org> <@rancher:matrix.org> I tried the same config files on gitlab.com (v16.0), and the same error appears when I open the same projects.15:27
the error with getProjectMetadata() implies the project has no config. I'm not sure this is a gitlab issue. It may just be an improper/incomplete configuration. I think you need to look in your logs for where that project is getting loaded to see the initial error
@jim:acmegating.comi pushed the zuul tag15:50
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 883419: Use bwrap --disable-userns if possible https://review.opendev.org/c/zuul/zuul/+/88341917:08
@fungicide:matrix.orgsorry for not weighing in on the release, the element browser client apparently just stopped updating with new messages around 24 hours ago and gave no indication it had done so. i thought it seemed eerily quiet in here so i reloaded that browser tab and yep, it was apparently my connection 17:46
@jim:acmegating.combeen a little choppy in matrix land.  but everything's eventually consistent i guess.  :)17:48
@fungicide:matrix.orgcan anybody clue me in to what it is i'm doing wrong in https://review.opendev.org/883431 ? trying to update a trigger configuration for the recent require-approval deprecation18:21
-@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [zuul/zuul] 883419: Use bwrap --disable-userns if possible https://review.opendev.org/c/zuul/zuul/+/88341918:43
@jim:acmegating.comfungi: try:18:54
```
require:
approval:
- Verified: [-1, -2]
username: zuul
approval:
- Workflow: 1
```
@jim:acmegating.comalso, i'd probably re-order it so it makes more sense if you read it in english:18:55
```
- event: comment-added
approval:
- Workflow: 1
require:
approval:
- Verified: [-1, -2]
username: zuul
```
@fungicide:matrix.orgaha, okay thanks. reading the docs i wasn't quite sure what "same syntax as requirements configuration" meant, and the old syntax looked superficially the same as requirements configuration19:03
@jim:acmegating.commaybe we should put in an example.  but please not that one, no one should use clean-check unless at the utmost end of need.19:06
@fungicide:matrix.orgyes, i was hesitant to suggest an example since that's the only (anti)pattern i'm familiar with relying on it19:11
@jim:acmegating.comcould probably do something "innocuous" like "Policy-Compliant: 1"  :)19:13
-@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed: [zuul/zuul] 883441: Include examples for trigger require and reject https://review.opendev.org/c/zuul/zuul/+/88344119:25
@fungicide:matrix.orgcorvus: like that ^ ?19:25
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 883443: Only check bwrap execution under the executor https://review.opendev.org/c/zuul/zuul/+/88344320:20
@clarkb:matrix.orgcorvus: ^ that is a bit hacky. The commit message suggests a less hacky workaround. I'm totally happy if you would prefer to go that route, but I expect it to take more time and i need to eat lunch. I don't mind if ou push it instead I can abandon my change20:20
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Clark Boylan: [zuul/zuul] 883443: Only check bwrap execution under the executor https://review.opendev.org/c/zuul/zuul/+/88344320:24
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Tristan Cacqueray: [zuul/zuul] 687135: Replace docker by podman for quick-start https://review.opendev.org/c/zuul/zuul/+/68713520:39
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 883445: DNM: break scheduler to test quickstart https://review.opendev.org/c/zuul/zuul/+/88344520:40
@clarkb:matrix.orgcorvus: do the nox-client and nox remote jobs also need similar treatment?20:42
@jim:acmegating.comhttps://f44bf35a9c7254f845ed-21b9d8b77413feaeb233a2c7e38edf25.ssl.cf2.rackcdn.com/883443/2/gate/zuul-nox-remote/fd4fa85/testr_results.html20:42
@clarkb:matrix.orgI guess as an alternative we can drop the check entirely. It was a nice to have but then add it back in once things are more stable?20:42
@jim:acmegating.comClark: my first suspicion is that's just missing a change to the initializer20:42
@clarkb:matrix.orgoh I see. Let me try to fix that20:43
@jim:acmegating.comgrep says 2:20:43
```
tests/base.py: self.registerDriver(BubblewrapDriver())
zuul/driver/bubblewrap/__init__.py: driver = BubblewrapDriver()
```
@jim:acmegating.comi'm sure the second isn't used but should probably add for completeness20:44
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 883443: Only check bwrap execution under the executor https://review.opendev.org/c/zuul/zuul/+/88344320:45
@clarkb:matrix.orglike that maybe (gerritbot should catch up momentarily)20:45
@jim:acmegating.comwe should have it in all our test contexts.. i think.20:46
@clarkb:matrix.orgre zuul and speculative container testing ++ to just swapping out to podman. I think its far more contained in zuul CI than opendev production so we can just swap it and if tests pass merge it20:47
@clarkb:matrix.orgon the image building side of things I think buildx does speculative lookups properly so an easy "fix" there is to just enable buildx. But buildah wouldn't be bad either20:47
@jim:acmegating.comyeah -- does buildah do the multi arch stuff we use in nodepool?20:48
@jim:acmegating.comif not, maybe let's just do buildx everywhere for consistency20:48
@clarkb:matrix.orgit can, but the roles don't support it yet I don't think20:48
@clarkb:matrix.orgsince the steps you take are different for buildah than buildx20:49
@jim:acmegating.comok.  then buildx first, then buildah when someone gets around to it :)20:49
@clarkb:matrix.org++20:49
@jim:acmegating.comhuh apparently the way we chose is by having an arch defined20:51
@jim:acmegating.comthere's no "use_buildx" flag20:51
@clarkb:matrix.orgcorvus: for multiarch. For non multiarch in the container (not docker) roles you set an env var20:52
@clarkb:matrix.orgI have an example of that somewhere one sec20:52
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 883446: Use buildx to build images https://review.opendev.org/c/zuul/zuul/+/88344620:53
@jim:acmegating.comoh? okay20:53
@clarkb:matrix.orgcorvus: https://review.opendev.org/c/opendev/system-config/+/882898/1/zuul.d/docker-images/gitea.yaml#4620:54
@clarkb:matrix.orgbuildkit is buildx's backend and should do all of the image pulling so I think that works20:54
@clarkb:matrix.org(but we should double check)20:54
@jim:acmegating.comokay cool i'll switch to that.  i just assumed that running the buildx.yaml task list was required, but no, this does make sense.20:54
@jim:acmegating.com(that file could probably use a better name)20:55
@clarkb:matrix.org++20:55
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 883446: Use buildx to build images https://review.opendev.org/c/zuul/zuul/+/88344620:56
@clarkb:matrix.orgif that doesn't work we'll want to make a more generic buildx path and possibly have multiarch run portions of that20:57
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 883445: DNM: break scheduler to test quickstart https://review.opendev.org/c/zuul/zuul/+/88344520:57
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Tristan Cacqueray: [zuul/zuul] 687135: Replace docker by podman for quick-start https://review.opendev.org/c/zuul/zuul/+/68713520:57
@clarkb:matrix.orgbut I think it will work because docker client tools are pretty dumb and the backends do all the work20:57
@jim:acmegating.comyeah this matches my understanding20:57
@clarkb:matrix.orgLooks like we are about an hour away from the fix change merging. I'm going to take a break21:02
@clarkb:matrix.orgthe jobs that were angry did pass though21:02
@fungicide:matrix.orgyeah, i'm starting to cook dinner but will be around again after21:03
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 883445: DNM: break scheduler to test quickstart https://review.opendev.org/c/zuul/zuul/+/88344522:02
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Tristan Cacqueray: [zuul/zuul] 687135: Replace docker by podman for quick-start https://review.opendev.org/c/zuul/zuul/+/68713522:02
-@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [zuul/zuul] 883443: Only check bwrap execution under the executor https://review.opendev.org/c/zuul/zuul/+/88344322:04
@clarkb:matrix.orgcorvus: ^ once that promotes you should be good to restart services ya?22:05
@clarkb:matrix.orgthe promote jobs report success22:08
@fungicide:matrix.org``Build succeeded (promote pipeline).``22:08
@jim:acmegating.comw00t22:08
@fungicide:matrix.orgdo we want another change to reinstate the quickstart job now, or wait for the buildx/podman stuff to land?22:10
@clarkb:matrix.orgI don't mind focusing on getting the buildx/podman stuff going22:10
@jim:acmegating.comyeah i think it's pointless to re-enable it in gate until it's truly fixed22:12
@fungicide:matrix.orgthat's what i suspected22:12
@jim:acmegating.com(it just tells you that the previous change that landed broke)22:12
@jim:acmegating.com(and it's running in check, so that's good enough for that purpose :)22:12
@fungicide:matrix.orgdelayed reaction gating ;)22:12
@jim:acmegating.comwe're going to CI like it's 199922:13
@jim:acmegating.comClark: https://zuul.opendev.org/t/zuul/build/bbd5099cd38a42cc8066177a2125d7b3/console22:35
@jim:acmegating.comi'm trying to figure out what's wrong there22:35
@jim:acmegating.comoh is it just that one of those pods isn't running and that stops the playbook?22:36
@jim:acmegating.com(including before it gets to the copy-back step?)22:36
@clarkb:matrix.orgit looks like it is reporting a 404 though22:37
@clarkb:matrix.orgwhich implies something is running?22:37
@jim:acmegating.comright, i'd like to know what's not running and why, which is why i'd like the logs22:38
@jim:acmegating.comso i think that some pod (other than gerrit) is not running and that's causing the podman logs command to exit and the playbook to terminate22:38
@jim:acmegating.comso maybe just a "set -e" at the start there?22:38
@clarkb:matrix.orgthere is the collect-container-logs role (I think that is the name) which might be useful too22:38
@clarkb:matrix.orgya set -ex would help it fail early22:39
@clarkb:matrix.organd point more clearly to hwat is broken22:39
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Tristan Cacqueray: [zuul/zuul] 687135: Replace docker by podman for quick-start https://review.opendev.org/c/zuul/zuul/+/68713522:39
@jim:acmegating.comer i don't want it to fail22:39
@jim:acmegating.comi need the exit code of that to always be 022:39
@jim:acmegating.commaybe i should just add ignore_errors22:39
@clarkb:matrix.orgwell I mean if there is an error it will stop early? so you'd modify it to not exit non zero ya that22:40
@jim:acmegating.comwell22:40
@clarkb:matrix.orgbaically you can start with set -e to see where things break as soon as they do then clean it up?22:40
@jim:acmegating.comi mean, i want it to run every one of those commands22:40
@jim:acmegating.comi want to get the logs of every container that is running22:40
@clarkb:matrix.orgoh I see22:41
@jim:acmegating.comok i want +e22:41
@jim:acmegating.comand then ignore_errors22:41
@jim:acmegating.com(or end it with exit 0)22:41
@clarkb:matrix.orgya +e will allow it to proceed and then the collect-container-logs role can be helpful for organizing them at the end if the containers are still running at the end of the job22:41
@jim:acmegating.comsure, i'd be happy to switch to that but i don't want to do it now22:42
@jim:acmegating.comif i can fix this with one line instead of opening a second can of worms, that's progress22:42
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Tristan Cacqueray: [zuul/zuul] 687135: Replace docker by podman for quick-start https://review.opendev.org/c/zuul/zuul/+/68713522:42
@jim:acmegating.comawesome, no containers with those names23:05
@jim:acmegating.comi ran it locally and double checked the container names looked right23:05
@jim:acmegating.comha23:06
@jim:acmegating.comit runs as root23:06
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed on behalf of Tristan Cacqueray: [zuul/zuul] 687135: Replace docker by podman for quick-start https://review.opendev.org/c/zuul/zuul/+/68713523:06
@jim:acmegating.comClark: ^ :)23:07
@jim:acmegating.comrequires a shift in thinking23:07
@clarkb:matrix.orgI worried about that a bit in opendev stuff too but so far it has all been root there so its consistent23:10
@clarkb:matrix.orgthe inverse of your problem :)23:10
@fungicide:matrix.orgare we ready for https://storyboard.openstack.org/#!/story/2010753 to be switched to public now that the patch from it (and subsequent fix) has merged?23:14
@jim:acmegating.comhttps://zuul.opendev.org/t/zuul/build/f302506542404b3b9089674fff83617e/log/container_logs/gerritconfig.log looks like the next breadcrumb23:30
@jim:acmegating.comClark: that's doing an ansible `wait_for` with a host and port, so i wonder if the networking is different enough for that not to work (eg dns?)23:36

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!