| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 940872: Implement keystore functions for OIDC RS256 https://review.opendev.org/c/zuul/zuul/+/940872 | 06:48 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 941629: Use ZuulTreeCache for OIDC signing keys https://review.opendev.org/c/zuul/zuul/+/941629 | 06:48 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 940971: Manage OIDC signing key rotation https://review.opendev.org/c/zuul/zuul/+/940971 | 07:23 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 942432: Implement zuul-web OIDC endpoints https://review.opendev.org/c/zuul/zuul/+/942432 | 07:25 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 942886: Prepare oidc token for playbook execution in executor. https://review.opendev.org/c/zuul/zuul/+/942886 | 07:38 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 941235: Implement command for deleting OIDC signing keys https://review.opendev.org/c/zuul/zuul/+/941235 | 07:39 | |
| -@gerrit:opendev.org- Dong Zhang proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 947293: Update documentation for Zuul as OIDC ID Provider https://review.opendev.org/c/zuul/zuul/+/947293 | 07:42 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 942886: Prepare oidc token for playbook execution in executor. https://review.opendev.org/c/zuul/zuul/+/942886 | 08:28 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 941235: Implement command for deleting OIDC signing keys https://review.opendev.org/c/zuul/zuul/+/941235 | 08:29 | |
| -@gerrit:opendev.org- Dong Zhang proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 947293: Update documentation for Zuul as OIDC ID Provider https://review.opendev.org/c/zuul/zuul/+/947293 | 08:30 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Benjamin Schanzel: [zuul/zuul] 940500: web: Use a select filter for pipelines and queues on status page https://review.opendev.org/c/zuul/zuul/+/940500 | 12:06 | |
| -@gerrit:opendev.org- Andrei Dmitriev proposed: [zuul/nodepool] 916729: Cmd: add metastatic backing nodes removal https://review.opendev.org/c/zuul/nodepool/+/916729 | 16:37 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 947526: Gerrit: retry fetching refs and HEAD https://review.opendev.org/c/zuul/zuul/+/947526 | 17:15 | |
| @jim:acmegating.com | Clark: that change is based on the discussion/debugging we just did in #opendev where we observed a connection problem to gerrit cause zuul not to reconfigure a tenant after a new branch was created | 17:17 |
|---|---|---|
| @clarkb:matrix.org | corvus: one small piece of feedback on that | 17:22 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 947526: Gerrit: retry fetching refs and HEAD https://review.opendev.org/c/zuul/zuul/+/947526 | 17:29 | |
| @jim:acmegating.com | Clark: done | 17:29 |
| @clarkb:matrix.org | thanks +2 from me | 17:33 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 947530: Fix exception handling in Gerrit event connector https://review.opendev.org/c/zuul/zuul/+/947530 | 18:15 | |
| @jim:acmegating.com | Clark: ^ that should address the other oddity we saw | 18:16 |
| @clarkb:matrix.org | corvus: +2 on that one as well | 20:23 |
| @winter:catgirl.cloud | hey folks! i'm trying the containerized example, and i'm running into this issue with the executor: | 23:07 |
| ``` | ||
| executor-1 | 2025-04-16 23:04:49,608 ERROR zuul.BubblewrapDriver: Non zero return code executing: setpriv --ambient-caps -all choom -n 200 -- bwrap --dir /tmp --tmpfs /tmp --dir /var --dir /var/tmp --dir /run/user/0 --ro-bind /usr /usr --ro-bind /lib /lib --ro-bind /bin /bin --ro-bind /sbin /sbin --ro-bind /etc/ld.so.cache /etc/ld.so.cache --ro-bind /etc/resolv.conf /etc/resolv.conf --ro-bind /etc/hosts /etc/hosts --ro-bind /etc/localtime /etc/localtime --ro-bind /dev/null /dev/null --bind /tmp /tmp --tmpfs /tmp/tmp --proc /proc --dev /dev --chdir /tmp --unshare-all --share-net --die-with-parent --uid 0 --gid 0 --file 3 /etc/passwd --file 4 /etc/group --unshare-user --disable-userns --ro-bind /lib64 /lib64 --ro-bind /etc/nsswitch.conf /etc/nsswitch.conf --ro-bind /etc/alternatives /etc/alternatives --ro-bind /etc/ssl/certs /etc/ssl/certs --ro-bind /etc/subuid /etc/subuid --ro-bind /etc/containers /etc/containers id | ||
| executor-1 | Traceback (most recent call last): | ||
| executor-1 | File "/usr/local/bin/zuul-executor", line 8, in <module> | ||
| executor-1 | sys.exit(main()) | ||
| executor-1 | ^^^^^^ | ||
| executor-1 | File "/usr/local/lib/python3.11/site-packages/zuul/cmd/executor.py", line 133, in main | ||
| executor-1 | Executor().main() | ||
| executor-1 | File "/usr/local/lib/python3.11/site-packages/zuul/cmd/__init__.py", line 267, in main | ||
| executor-1 | self.run() | ||
| executor-1 | File "/usr/local/lib/python3.11/site-packages/zuul/cmd/executor.py", line 88, in run | ||
| executor-1 | self.configure_connections(sources=True, check_bwrap=True) | ||
| executor-1 | File "/usr/local/lib/python3.11/site-packages/zuul/cmd/__init__.py", line 217, in configure_connections | ||
| executor-1 | self.connections = zuul.lib.connections.ConnectionRegistry( | ||
| executor-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| executor-1 | File "/usr/local/lib/python3.11/site-packages/zuul/lib/connections.py", line 66, in __init__ | ||
| executor-1 | zuul.driver.bubblewrap.BubblewrapDriver(check_bwrap)) | ||
| executor-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| executor-1 | File "/usr/local/lib/python3.11/site-packages/zuul/driver/bubblewrap/__init__.py", line 224, in __init__ | ||
| executor-1 | raise Exception('bwrap execution validation failed. You can ' | ||
| executor-1 | Exception: bwrap execution validation failed. You can use `zuul-bwrap /tmp id` to investigate manually. | ||
| ``` | ||
| it seems that `bwrap` needs more parameters than what's passed, as running the command within the container manually just shows bwrap's help | ||
| @winter:catgirl.cloud | ah, no, that was my fault | 23:09 |
| @winter:catgirl.cloud | real issue is this: | 23:09 |
| > bwrap: Creating new namespace failed, likely because the kernel does not support user namespaces. bwrap must be installed setuid on such systems. | ||
| @winter:catgirl.cloud | which i guess is the fault of how i'm running the container | 23:09 |
| @jim:acmegating.com | yeah, that suggestion at the end is designed to try to show problems like that | 23:10 |
| @winter:catgirl.cloud | `zuul-bwrap /tmp id`'s output isn't any more helpful, maybe it'd be best to show the stdout/stderr from failed bwrap invocations? | 23:11 |
| @winter:catgirl.cloud | during this check | 23:11 |
| @clarkb:matrix.org | was this from bwrap stderr/stdout? | 23:16 |
| @winter:catgirl.cloud | correct | 23:17 |
| @clarkb:matrix.org | but yes iirc the images we published are based on debian bookworm bwrap which expects user namespaces to be enabled | 23:17 |
| @winter:catgirl.cloud | happy to send a CL to include it in that error output if y'all want | 23:17 |
| @jim:acmegating.com | did you see it when you ran `zuul-bwrap /tmp id` ? | 23:17 |
| @winter:catgirl.cloud | yeah i'm trying a different linux-on-vm thing | 23:17 |
| @winter:catgirl.cloud | * yeah i'm trying a different linux-on-macos thing now | 23:17 |
| @winter:catgirl.cloud | nope | 23:17 |
| @jim:acmegating.com | then that's probably a good idea. :) not sure why we didn't include that originally | 23:18 |
| @winter:catgirl.cloud | * yeah i'm trying a different linux-on-macos thing now, hopefully their kconfig enables user ns | 23:19 |
| @winter:catgirl.cloud | * yeah i'm trying a different linux-on-macos thing now, hopefully their kconfig enables user ns (was trying colima before, now trying docker desktop, lol) | 23:19 |
| @winter:catgirl.cloud | > `executor-1 | Exception: bwrap execution validation failed. You can use `zuul-bwrap /tmp id` to investigate manually.` | 23:20 |
| really. | ||
| @winter:catgirl.cloud | docker desktop's kernel too? :-/ | 23:20 |
| @winter:catgirl.cloud | seems colima is using Ubuntu's cloud images as a base. surprised they don't have user namespaces enabled...? | 23:23 |
| @clarkb:matrix.org | Is it possible they are then running the containers within a less privileged context (the zuul bwrap will actually do that too. It uses user namespaces to bootstrap then disables them for the runtime env they create) | 23:25 |
| @winter:catgirl.cloud | perhaps? they're just using `dockerd` | 23:25 |
| @winter:catgirl.cloud | do these images only work with podman? 😅 | 23:25 |
| @clarkb:matrix.org | no we run them with docker in production for opendev | 23:26 |
| @jim:acmegating.com | no they should work with either dockerd or podman | 23:26 |
| @winter:catgirl.cloud | yeah this isn't using any custom configuration, lemme poke around in the vm itsel | 23:26 |
| @winter:catgirl.cloud | * yeah this isn't using any custom configuration, lemme poke around in the vm itself | 23:26 |
| @winter:catgirl.cloud | ``` | 23:33 |
| { | ||
| "exec-opts": [ | ||
| "native.cgroupdriver=cgroupfs" | ||
| ], | ||
| "features": { | ||
| "buildkit": true | ||
| } | ||
| } | ||
| ``` | ||
| dockerd config looks sane | ||
| @winter:catgirl.cloud | > `kernel.unprivileged_userns_clone = 1` | 23:33 |
| @winter:catgirl.cloud | maybe the `privileged` attribute isn't actually being... passed...? | 23:38 |
| @winter:catgirl.cloud | but it's weird that that persists across multiple vm impls | 23:38 |
| @winter:catgirl.cloud | aaaaaaaah, i think i found the issue | 23:43 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!