| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 831107: Add option to show overall duration in buildset table https://review.opendev.org/c/zuul/zuul/+/831107 | 01:41 | |
| @iwienand:matrix.org | 831108 has now failed tox 3.8, then the next run 3.9, then the next run a error: Status code: 503 from centos upstream | 02:14 |
|---|---|---|
| @iwienand:matrix.org | i guess just lucky | 02:15 |
| @iwienand:matrix.org | but i do think that maybe we've missed setting up local mirroring for 9-stream, after we moved the nodepool job from building centos-7 to stream | 02:16 |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [dnm] testing https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 02:22 | |
| -@gerrit:opendev.org- Ian Wienand proposed: | 02:48 | |
| - [zuul/zuul-registry] 831131: [dnm] testing https://review.opendev.org/c/zuul/zuul-registry/+/831131 | ||
| - [zuul/zuul-registry] 831133: Fix podman install source https://review.opendev.org/c/zuul/zuul-registry/+/831133 | ||
| -@gerrit:opendev.org- Ian Wienand proposed: | 03:11 | |
| - [zuul/zuul-registry] 831133: Fix podman install source https://review.opendev.org/c/zuul/zuul-registry/+/831133 | ||
| - [zuul/zuul-registry] 831131: [dnm] testing https://review.opendev.org/c/zuul/zuul-registry/+/831131 | ||
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [dnm] testing https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 03:40 | |
| -@gerrit:opendev.org- Ian Wienand proposed: | 03:51 | |
| - [zuul/zuul-registry] 831131: [dnm] testing https://review.opendev.org/c/zuul/zuul-registry/+/831131 | ||
| - [zuul/zuul-registry] 831135: Update testing to Ubuntu Focal https://review.opendev.org/c/zuul/zuul-registry/+/831135 | ||
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 831136: ensure-pip: fix typo in ensure_pip_virtualenv_command documentation https://review.opendev.org/c/zuul/zuul-jobs/+/831136 | 03:53 | |
| -@gerrit:opendev.org- Ian Wienand proposed: | 04:09 | |
| - [zuul/zuul-registry] 831135: Update testing to Ubuntu Focal https://review.opendev.org/c/zuul/zuul-registry/+/831135 | ||
| - [zuul/zuul-registry] 831131: [WIP] enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | ||
| -@gerrit:opendev.org- Ian Wienand proposed: | 04:28 | |
| - [zuul/zuul-registry] 831135: Update testing to Ubuntu Focal https://review.opendev.org/c/zuul/zuul-registry/+/831135 | ||
| - [zuul/zuul-registry] 831131: [WIP] enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | ||
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 04:57 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 05:25 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 05:32 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 08:50 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 830995: Correctly reset failing cycle behind failing item https://review.opendev.org/c/zuul/zuul/+/830995 | 09:02 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 09:38 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 09:40 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 10:13 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] Enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 10:25 | |
| @y2kenny:matrix.org | Hi, my Zuul install seems to be stuck at some weird state and in need some help. I think this started out with a developer pushing ~500 changes to Gerrit. It's not a branch that the CI needs to do anything about but since Zuul sees everything, it got affected by it. As far as I can tell, Zuul seems to be stuck in a state that continuously adding those changes to the ChangeQueue of a particular pipeline... possibly for at least 12 hours. Looking at the log, the scheduler seems to be adding one change to the queue per second. The WebUI status page is frozen and cause Chrome to return RESULT_CODE_HUNG. I then went and killed the scheduler and that has stopped the loop. But after I restarted the scheduler, the webUI status page continue to stay stuck. The scheduler initially took a very long time to initialize but it came back up on a second restart try but the webUI status continue to stay stuck. Now I am attempting to clear things out by delete-state. Is there any other way to recover from this? I just notice there's a delete-pipeline-state and I plan to try that next. | 13:53 |
| @tristanc_:matrix.org | Kenny Ho: are the changes depending on each other? I think the delete-pipeline-state should clear things up, if you don't mind loosing the states. | 14:18 |
| @y2kenny:matrix.org | they are all part of one single push yes | 14:19 |
| @y2kenny:matrix.org | I am running delete-pipeline-state now but it's been around 15 mintes and it's still running | 14:19 |
| @y2kenny:matrix.org | I hope it doesn't take as long to delete as the add... | 14:26 |
| @avass:vassast.org | Regarding: https://review.opendev.org/c/zuul/zuul/+/830840 | 14:49 |
| I'm wondering if it's better to piggy back on `zuul_return` so it's possible do something similar do: | ||
| ``` | ||
| - zuul_return: | ||
| data: | ||
| retry: false | ||
| - fail: | ||
| msg: ... | ||
| ``` | ||
| which could also make it possible to set `retry: true` to retry more dynamically in a `run` or `post` playbook. | ||
| Or if it's better to implement the "fail early" functionality with a dedicated `zuul_fail` module. | ||
| @y2kenny:matrix.org | so the delete command is still running... does anyone know if the delete is run incrementally? Assuming there are 43200 items queued in zookeeper, is the delete-pipeline-state done in one single transaction? I am looking at https://opendev.org/zuul/zuul/src/branch/master/zuul/cmd/client.py#L957 but I can't tell what it's doing since I don't know anything about zk | 14:56 |
| @y2kenny:matrix.org | ok... so the delete command has been running for more than an hour with no resolution... I am guessing I need to wipe zookeeper entirely... are there any other suggestions? | 15:16 |
| -@gerrit:opendev.org- Matthieu Huin https://matrix.to/#/@mhuin:matrix.org proposed: [zuul/zuul] 831222: GUI: Do not show sign-in button if no IdP is available https://review.opendev.org/c/zuul/zuul/+/831222 | 15:30 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 831226: Add docs about reconfiguring on a single scheduler https://review.opendev.org/c/zuul/zuul/+/831226 | 16:10 | |
| @clarkb:matrix.org | Kenny Ho: If this is before the queue states were moved to zookeeper (sounds like not) you could just restart zuul | 16:18 |
| @jim:acmegating.com | see the docs for the 2 commands. https://zuul-ci.org/docs/zuul/latest/client.html#delete-state -- shut everything down, run delete-state, then start again. | 16:22 |
| @y2kenny:matrix.org | Clark: yea, I tried restarting zuul and there's no improvement. | 16:23 |
| @jim:acmegating.com | never run delete-pipeline-state | 16:23 |
| @jim:acmegating.com | (even the docs say don't use it) | 16:23 |
| @y2kenny:matrix.org | corvus: ok I will do that | 16:24 |
| @y2kenny:matrix.org | corvus: I was trying delete-state initially but it never return so I was hoping delete-pipeline-state will complete quicker. But that also didn't complete after an hour. | 16:28 |
| @jim:acmegating.com | did you shut everything down before running delete-state? | 16:28 |
| @y2kenny:matrix.org | so I am now wondering if the delete state will take as long as the state accumulated (which is possibly 12 hrs...) | 16:28 |
| @y2kenny:matrix.org | yup | 16:29 |
| @y2kenny:matrix.org | everything is shutdown right now except zk | 16:29 |
| @y2kenny:matrix.org | everything is shutdown when I was trying delete-state and delete-pipeline-state | 16:30 |
| @jim:acmegating.com | then while it's running, i'd check zk monitoring to see if the size is decreasing | 16:34 |
| -@gerrit:opendev.org- Matthieu Huin https://matrix.to/#/@mhuin:matrix.org proposed: [zuul/zuul] 831231: Doc: stress the role of HS256 authenticator https://review.opendev.org/c/zuul/zuul/+/831231 | 16:39 | |
| @jpew:matrix.org | Just to verify: in the Gerrit driver, the `approve` section of a `comment-added` event only applies *to that comment* not to the gerrit as a whole? For example, if I require a `+1 Approve` and `+1 Code-Review` in the event, the same user must provided both because it's only look at the specific comment, not any of the current scoring on the Gerrit? | 16:58 |
| @jpew:matrix.org | And, if I want separate users to be able to provide those scores, I should make 2 event triggers (one for each) and then have a pipeline `require` section that enforces both are set? | 17:01 |
| @jim:acmegating.com | jpew: https://zuul-ci.org/docs/zuul/latest/drivers/gerrit.html#attr-pipeline.trigger.%3Cgerrit%20source%3E.require-approval | 17:02 |
| @jpew:matrix.org | Ah, OK.... why is there `require-approval` in the trigger and `approval` in the pipeline, and which should I use? | 17:03 |
| @jim:acmegating.com | i don't understand the question. those are both pipeline trigger options. | 17:04 |
| @jpew:matrix.org | Sorry, `pipeline.require.<gerrit source>.approval` vs `pipeline.trigger.<gerrit source>.require-approval` | 17:05 |
| @jim:acmegating.com | one's a pipeline requirement: https://zuul-ci.org/docs/zuul/latest/drivers/gerrit.html#requirements-configuration the other is a pipeline trigger: https://zuul-ci.org/docs/zuul/latest/drivers/gerrit.html#trigger-configuration | 17:06 |
| @jim:acmegating.com | a pipeline requirement must match for an item to be in a pipeline. a trigger must match for an event to enqueue an item in the pipeline. | 17:07 |
| @jim:acmegating.com | triggers are evaluated only against the triggering event; pipeline requirements are evaluated more often. | 17:08 |
| @jpew:matrix.org | So if the conditions on the pipeline change, the item will be dequeued? | 17:08 |
| @jim:acmegating.com | not currently, no. | 17:11 |
| @jim:acmegating.com | let me try again: the trigger event must match in order for the pipeline to process the event and attempt to enqueue the change; the requirements must match in order for the change to be enqueued. | 17:12 |
| @jim:acmegating.com | they both have to be true for a change to be added | 17:12 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-registry] 831235: Perform atomic upload updates v2 https://review.opendev.org/c/zuul/zuul-registry/+/831235 | 17:33 | |
| @clarkb:matrix.org | corvus: ^ I don't know if/how that impacts the swift implementation. | 17:33 |
| @clarkb:matrix.org | corvus: but I figured i could get the filesystem version up and we can work through whether or not this appraoch is sensible | 17:33 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-registry] 831235: Perform atomic upload updates v2 https://review.opendev.org/c/zuul/zuul-registry/+/831235 | 17:37 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 831240: Retain created_time in node requests https://review.opendev.org/c/zuul/nodepool/+/831240 | 18:04 | |
| @jim:acmegating.com | there is one panel missing on our new performance metrics dashboard https://grafana.opendev.org/d/2e89fb78e5/zuul-performance-metrics?orgId=1&var-tenant=openstack -- that ^ is an easy change to fix that. | 18:09 |
| @jim:acmegating.com | Clark: it seems like no changes should be required for swift, but i have not thought deeply about it. | 18:09 |
| @jim:acmegating.com | Clark: i think you can have a zuul-jobs change depends-on the registry... i'm not 100% sure why we don't run those jobs on zuul-registry... maybe because of catch-22s setting it up in the first place. | 18:11 |
| @clarkb:matrix.org | corvus: ya I think all of the swift requests will not interfere with each other too. Swift should do a good job there | 18:26 |
| @clarkb:matrix.org | I'll try the depends on | 18:26 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 831243: DNM testing depends-on https://review.opendev.org/c/zuul/zuul-jobs/+/831243 | 18:28 | |
| @clarkb:matrix.org | Something like ^ maybe? | 18:28 |
| @clarkb:matrix.org | corvus: if those jobs don't explicitly require the container built by zuul-registry will the system still sort it out via the depends-on? I guess I can check logs once some jobs have completed and check docker image versions | 18:34 |
| @clarkb:matrix.org | I'm pretty sure the testing done in zuul-registry isn't self testing since it shows the lock file messages in the log for that job still | 18:48 |
| @clarkb:matrix.org | oh wait we might bootstrap the registry build with the previous registry version, then we start a new one? I'm still looking through logs to be sure I undersatnd it | 18:52 |
| @clarkb:matrix.org | aha yup that is it. No locking logged in the second registry running on a different port | 18:52 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 831246: Add more pipeline processing stats https://review.opendev.org/c/zuul/zuul/+/831246 | 18:52 | |
| @jim:acmegating.com | Clark: yep, it's registry inception | 18:53 |
| @clarkb:matrix.org | this testing shows that the filesystem driver works in the simple case at least. That is promising | 18:54 |
| @clarkb:matrix.org | I'm still trying to understand this better to see if I can force it to do something a bit more meaningful (like concurrent uploads of the same image) | 18:54 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 831249: Add even more pipeline processing stats https://review.opendev.org/c/zuul/zuul/+/831249 | 19:07 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 831249: Add even more pipeline processing stats https://review.opendev.org/c/zuul/zuul/+/831249 | 19:10 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-registry] 831274: Add more robust testing of the registry https://review.opendev.org/c/zuul/zuul-registry/+/831274 | 19:28 | |
| @clarkb:matrix.org | I'm hoping that ^ will be able to shake out more of this stuff | 19:28 |
| @clarkb:matrix.org | https://zuul.opendev.org/t/zuul/build/8963e97878464cd58e482bf05e6bc8a3/log/docker/functionaltest_registry_1.txt I think that failed successfully (eg showing bugs in the implementation) | 19:45 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 20:15 | |
| - [zuul/zuul-registry] 831274: Add more robust testing of the registry https://review.opendev.org/c/zuul/zuul-registry/+/831274 | ||
| - [zuul/zuul-registry] 831312: Filesystem: Don't move chunks until upload is complete https://review.opendev.org/c/zuul/zuul-registry/+/831312 | ||
| @clarkb:matrix.org | corvus: ^ this is getting more complicated :( but the good news is my test updates definitely caught real problems | 20:15 |
| @clarkb:matrix.org | corvus: I'm worried that I'm hacking around the api you built a bit much, but I'm not sure how to resolve these issues otherwise. Feedback welcome or feel free to push up alternatives | 20:17 |
| @jim:acmegating.com | Clark: well, the storage system isn't supposed to know about namespaces or blobs or anything. it's just supposed to read and write files. | 20:21 |
| @jim:acmegating.com | that makes it a little easier to reason about whether all the backends work the same | 20:22 |
| @clarkb:matrix.org | ah so maybe this is more in line with that? | 20:24 |
| @clarkb:matrix.org | since I'm pushing some of that accounting down into the drivers | 20:24 |
| @jim:acmegating.com | Clark: maybe you can just generate the new chunk path? | 20:24 |
| @jim:acmegating.com | i was trying to say the opposite | 20:24 |
| @jim:acmegating.com | i was trying to say that the word "namespace" shouldn't appear in swift.py | 20:25 |
| @clarkb:matrix.org | ah | 20:25 |
| @jim:acmegating.com | because then swift and filesystem might end up doing something different | 20:25 |
| @clarkb:matrix.org | instead of pushing the parts down into the driver via chunks I could keep the old path determinations and push them both in chunks. Then let the driver decide if it needs to use one or the other or both? | 20:26 |
| @clarkb:matrix.org | Fundamentally they are operating differently though beacuse swift has different promises than posix filesystems. | 20:26 |
| @jim:acmegating.com | in the registry, the drivers make the same promises | 20:27 |
| @jim:acmegating.com | they are lowest common denominator | 20:27 |
| @clarkb:matrix.org | the thing that makes this weird is we can leak swift data if we make swift act like the filesystem. Making the fielsystem act like swift results in brokeness (the issue we currently observe) | 20:28 |
| @clarkb:matrix.org | I'm not sure how to resolve those two things | 20:28 |
| @clarkb:matrix.org | I guess we could go back to locking and do a block on acquiring the lock and force things to happen serially | 20:28 |
| @jim:acmegating.com | that might be more network-efficient | 20:29 |
| @clarkb:matrix.org | The reason I didn't do that is our locking is extremely fuzzy | 20:29 |
| @clarkb:matrix.org | it treats a lock that is less than 5 minutes old as held | 20:29 |
| @clarkb:matrix.org | https://review.opendev.org/c/zuul/zuul-registry/+/831274/ does pass this time around. At least that points to us identifying the issues even if we don't necessarily go with the current fixes | 20:30 |
| @jim:acmegating.com | here's what i'd ask: either keep the abstract driver api and find a way to do LCD. or get rid of the driver api entirely and do completely different things. i think the danger is in having an abstract driver api that behaves differently depending on the backend. | 20:30 |
| @clarkb:matrix.org | re network efficiency I also wasn't too worried about that as it seems like pushing the same exact object from multiple locations at the same time is a bit of a corner case since this hasn't broken us until recently. That said it is osmething to consider | 20:33 |
| @clarkb:matrix.org | The old code wasn't more network efficient though as it accepted the uploads and only noticed when it created the final object | 20:33 |
| @clarkb:matrix.org | or if it was it was minimally more efficient | 20:34 |
| @jim:acmegating.com | yeah, not a driver, but nice if it happens with a solution we like otherwise. | 20:34 |
| @jim:acmegating.com | why does swift leak data if we make it act like the filesystem? (and why doesn't the filesystem leak the same data?) | 20:35 |
| @jim:acmegating.com | (that's sort of what i mean about the driver api -- the drivers just read, write, move, cat... it should be the same for both) | 20:35 |
| @clarkb:matrix.org | corvus: because with the filesystem we create a new file that is the concetenated data from the upload dir. But with swift we create a meta object that just points to the chunks | 20:37 |
| @clarkb:matrix.org | corvus: in the filesystem case this means we can delete the upload data as soon as the actual file is concatenated. But with swift we can't do that as those upload chunks are what actually back the system | 20:37 |
| @clarkb:matrix.org | * corvus: in the filesystem case this means we can delete the upload data as soon as the actual file is concatenated. But with swift we can't do that as those upload chunks are what actually back the "file" | 20:38 |
| @clarkb:matrix.org | Which means if we allow multiple concurrent uploads in swift to use their upload/ chunks as the filesystem does in my change we can end up leaking them as we'll replace the metadata but not remove the old chunks | 20:38 |
| @clarkb:matrix.org | This means in the swift case it seems you do want to move the objects from the uploads/ dir to the blobs/ dir. But in the filesystem case this results in uploads overwritting each other. | 20:39 |
| @jim:acmegating.com | Clark: can we have a meetpad call about this? i feel like this might take us the rest of the day to type out.... | 20:40 |
| @clarkb:matrix.org | sure, is now good or is after lunch better? | 20:41 |
| @jim:acmegating.com | now is good, but can wait for you if you need food | 20:41 |
| @clarkb:matrix.org | no I'm good too | 20:42 |
| @jim:acmegating.com | best not think about this on an empty stomach :) | 20:42 |
| @clarkb:matrix.org | I had a very late breakfast :) | 20:42 |
| @jim:acmegating.com | https://meetpad.opendev.org/zuul-registry | 20:42 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 831226: Add docs about reconfiguring on a single scheduler https://review.opendev.org/c/zuul/zuul/+/831226 | 21:02 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] Enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 21:07 | |
| -@gerrit:opendev.org- Clark Boylan proposed: | 22:19 | |
| - [zuul/zuul-registry] 831235: Perform atomic upload updates v2 https://review.opendev.org/c/zuul/zuul-registry/+/831235 | ||
| - [zuul/zuul-registry] 831274: Add more robust testing of the registry https://review.opendev.org/c/zuul/zuul-registry/+/831274 | ||
| @clarkb:matrix.org | corvus: I think ^ that is what we discussed? | 22:19 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 831243: DNM testing depends-on https://review.opendev.org/c/zuul/zuul-jobs/+/831243 | 22:22 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-registry] 831131: [WIP] Enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | 22:23 | |
| @iwienand:matrix.org | Clark: corvus not sure if there's interest in ^ aside from me. one thing i've found is that i can't make docker go through the tracing proxy if it's using localhost:9000; it does though using the ip address of the host | 22:28 |
| @clarkb:matrix.org | weird. I think I managed to debug the current issue jsut from the service logs that we had from the ergistry though. | 22:29 |
| @clarkb:matrix.org | ianw: they are in the commit message of 831235 if you are interested | 22:29 |
| @iwienand:matrix.org | yep, fair enough. my main hope is that the next person who does need trace logs just gets it instantly, without having to worry about all the setup around getting it talking through the proxy, which is about 80% of the effort | 22:31 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 831316: Check if job request is still queued before yielding https://review.opendev.org/c/zuul/zuul/+/831316 | 22:31 | |
| @iwienand:matrix.org | the other two changes before that i think are important, updating testing to focal and fixing the install source for podman. because we don't want to be testing old stuff when it likes to change | 22:34 |
| @clarkb:matrix.org | ++ I +2'd both of them | 22:34 |
| @jim:acmegating.com | ianw: q on 135 | 22:36 |
| @jim:acmegating.com | Clark: lgtm | 22:38 |
| @iwienand:matrix.org | oh yeah, let me look back on that, i've got a bit tangled up in the stack. i thought i split that url change out into a separate change as wasn't sure it was my testing. | 22:38 |
| @iwienand:matrix.org | i'm not 100% sure if podman was wrong before and has fixed itself, or the test is wrong | 22:39 |
| @iwienand:matrix.org | https://827fc7bbd35003a9e6fe-a12b18ccd704b72d5d1cd1c70d4b1c51.ssl.cf2.rackcdn.com/831131/17/check/zuul-registry-build-image/e1ece4c/mitmdump.txt is more what i'd expect | 22:43 |
| -@gerrit:opendev.org- Zuul merged on behalf of Ian Wienand: [zuul/zuul-registry] 831133: Fix podman install source https://review.opendev.org/c/zuul/zuul-registry/+/831133 | 22:52 | |
| @iwienand:matrix.org | corvus: right, sorry, yeah i think the test is working around incorrect podman behaviour | 22:56 |
| @iwienand:matrix.org | failed_when: "'docker.io/test/image' not in image_list.stdout" | 22:56 |
| @iwienand:matrix.org | i mean, we didn't push the test/image to docker.io | 22:56 |
| @iwienand:matrix.org | i think that localhost:9000 -- the ephemeral zuul-registry we're testing against -- is right there; localhost:9000/test/image is what we pushed | 22:56 |
| @iwienand:matrix.org | the next question is why does this show up on the bionic->focal upgrade, and i don't quite know | 22:57 |
| @jim:acmegating.com | ianw: it's not what we pulled though. | 23:07 |
| -@gerrit:opendev.org- Ian Wienand proposed: | 23:08 | |
| - [zuul/zuul-registry] 831135: Update testing to Ubuntu Focal https://review.opendev.org/c/zuul/zuul-registry/+/831135 | ||
| - [zuul/zuul-registry] 831131: [WIP] Enable mitmproxy between docker/podman to dump test https://review.opendev.org/c/zuul/zuul-registry/+/831131 | ||
| - [zuul/zuul-registry] 831319: podman testing: dump image list https://review.opendev.org/c/zuul/zuul-registry/+/831319 | ||
| @iwienand:matrix.org | ^ will dump it so | 23:08 |
| @jim:acmegating.com | ianw: see additional comments on change. i'm pretty sure the test change is wrong. if the podman behavior has changed in the upgrade then that needs to be addressed by some way other than changing the test. | 23:09 |
| @y2kenny:matrix.org | are there any example of conf file of web client? The web UI login seems to provide conf file that is [<tenant>] url=<url> but the documentation seems to suggest [webclient] url=url but neither seems to work (zuul-client ask for --zuul-url or use a config file) | 23:17 |
| @y2kenny:matrix.org | when using with the conf file I tried with -c <path to config> | 23:18 |
| @iwienand:matrix.org | ok let's start by comparing the full output before and after, there might be other messages in there | 23:24 |
| @clarkb:matrix.org | Kenny Ho: https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/zuul-scheduler/templates/client.conf that is ours | 23:27 |
| @y2kenny:matrix.org | Clark: does the url have to be https or it shouldn't matter? | 23:28 |
| @clarkb:matrix.org | Kenny Ho: it carries sensitive info (the token) so probably should be https but I'm nto sure the client enforces that | 23:29 |
| @clarkb:matrix.org | looks like there is a verify option that can be set. Maybe if that is set to true it won't accept http | 23:31 |
| @y2kenny:matrix.org | ok. I am not sure why but zuul-client is not picking up the config. I am using the docker image docker.io/zuul/zuul-client. Do you have an example of an admin rule that allow enqueue dequeue? Looking at the doc, I am not sure if all functions need to be authorized individually or if I just need to set someone as admin | 23:32 |
| @y2kenny:matrix.org | I tried setting verify_ssl as false also but zuul-client seems to just ignore the config | 23:32 |
| @clarkb:matrix.org | Kenny Ho: https://opendev.org/openstack/project-config/src/branch/master/zuul/main.yaml that is our zuul config which has the auth stuff | 23:32 |
| @y2kenny:matrix.org | I ended up passing everything in via the flags like --auth-token and --url | 23:33 |
| @clarkb:matrix.org | are you bind mounting the config in? | 23:33 |
| @y2kenny:matrix.org | yes | 23:34 |
| @clarkb:matrix.org | I'm not sure then we definitely use it with a config file and seems to work | 23:34 |
| @y2kenny:matrix.org | is there a permission/user issue? | 23:34 |
| @y2kenny:matrix.org | um... ok | 23:34 |
| @y2kenny:matrix.org | it's weird... I tried overriding the entry point and I can definitely read the conf file | 23:35 |
| @clarkb:matrix.org | ianw: corvus is it possible one of the earlier test playbooks is leaking the localhost:9000 pushed file to the registry into when that playbook runs? Then when you check the listing it still shows as localhost:9000 because the sha matches what was in docker so the pull is a noop? | 23:50 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!