| @iwienand:matrix.org | what a mess ... The OCI distribution spec, which defines the api under /v2/ does say there is an endpint to delete tags | 00:07 |
|---|---|---|
| @iwienand:matrix.org | https://github.com/opencontainers/distribution-spec/blob/v1.0/spec.md#deleting-tags | 00:07 |
| @iwienand:matrix.org | doesn't work on docker hub registry though -> https://github.com/docker/hub-feedback/issues/1759#issuecomment-470317472 | 00:09 |
| @iwienand:matrix.org | it used to be that the test-suite enforced it failing, but with https://github.com/opencontainers/distribution-spec/pull/105/commits/bbe424c0f6328023d8221678e6881e0586fd2dff modified this to allow any of 202, 400 or 405 | 00:14 |
| @iwienand:matrix.org | which essentially took it from "no registry supports this" to "good luck writing any sort of generic workflow with this" :) | 00:14 |
| @iwienand:matrix.org | quay returns 405 | 00:15 |
| @iwienand:matrix.org | ``` | 00:15 |
| $ curl -X DELETE 'https://quay.io/v2/zuul-ci/zuul-client/manifests/testing' | ||
| <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> | ||
| <title>405 Method Not Allowed</title> | ||
| ``` | ||
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 878612: promote-image-container: do not delete tags https://review.opendev.org/c/zuul/zuul-jobs/+/878612 | 04:42 | |
| @iwienand:matrix.org | > <@jim:acmegating.com> (there's probably an option 2c as well, where we devise a tool of our own that pushes the underlying layers without pushing a tag; that would be fast and clean) | 04:45 |
| that feels like it might be unreliable wrt to the remote registry's garbage collection strategy? | ||
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 878614: [dnm] rough draft of deleting old quay tags https://review.opendev.org/c/zuul/zuul-jobs/+/878614 | 05:35 | |
| @iwienand:matrix.org | > <@jim:acmegating.com> here are some options: | 05:36 |
| > 1) don't have the promote role delete the temporary tags. leave them there forever. | ||
| > 1b) don't have the promote role delete the temporary tags, but have some new quay-api specific role come along and delete them later. | ||
| > 2) don't upload the image to quay in the upload role, instead, have the promote role copy the image from the intermediate registry to quay. this makes the promote role a little slower since it's actually moving data. | ||
| > 2b) have the upload role push to a single static tag on quay like "zuul_gate_pipeline_latest" just to get all the layers on quay, and then have the promote role copy from the intermediate registry. should still be relatively fast with all the layers already there. | ||
| i probably agree with 2. but i wonder if there is legitimate need for the promote model of making things as atomic as possible? i've tried to consolidate what i think the two models are doing a bit more in https://review.opendev.org/c/zuul/zuul-jobs/+/878612 | ||
| @iwienand:matrix.org | if we do think that maybe promoting and then you choose a cleanup role -- one for docker, one for quay, others to be written -- https://review.opendev.org/c/zuul/zuul-jobs/+/878614 is a rough draft of cleaning up old quay tags via the API. of course the timestamps, etc. are all subtly different to docker hub -- but that was matching things for me. i would image we'd have a partner "cleanup-docker-hub" role as well. and after promote you'd run one or the other depending | 05:38 |
| @iwienand:matrix.org | i do think that keeping the role pretty much as is, without the need for the intermediate registry, but something like "note you'll have the old tags there, but run the cleanup role suitable for you and they'll fall out eventually" might have something to it | 05:44 |
| @fungicide:matrix.org | trying to track down a hung node request in opendev, it looks like one of our launchers took the request and tried to satisfy it but ended up declining after a node failure, then another launcher took it and in the process of declining threw an "unable to modify missing request" error with ZKLockException "Request [...] does not hold a lock" https://meetings.opendev.org/irclogs/%23openstack-infra/%23openstack-infra.2023-03-27.log.html#t2023-03-27T13:28:30 | 14:03 |
| @fungicide:matrix.org | has anyone else seen anything like that? i guess i should start looking in the zk server logs next | 14:03 |
| @jim:acmegating.com | fungi: let's follow up in #_oftc_#opendev:matrix.org | 14:08 |
| @jim:acmegating.com | we tracked it down, and it looks like an old bug related to a race condition in the node request lock/unlock code | 14:20 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 878648: Mutex access to local lock attributes https://review.opendev.org/c/zuul/nodepool/+/878648 | 14:27 | |
| @jim:acmegating.com | fungi: ^ i think that's all we need | 14:27 |
| @clarkb:matrix.org | corvus: what prevents two threads in different processes from flowing through the same race? the mutex is process local right? | 14:39 |
| @jim:acmegating.com | Clark: the different processes won't share a NodeRequest object (the issue here is entirely related to local memory values; it's not a ZK problem; it's that one thread overwrites the local value in memory set by another thread) | 14:42 |
| @clarkb:matrix.org | aha | 14:42 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/nodepool] 877559: Run two statemachine driver threads https://review.opendev.org/c/zuul/nodepool/+/877559 | 16:41 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Tobias Urdin: [zuul/zuul] 878236: web: migrate pages to react-table https://review.opendev.org/c/zuul/zuul/+/878236 | 16:55 | |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: | 16:56 | |
| - [zuul/nodepool] 877560: Add API profiling logging of openstack list_volumes https://review.opendev.org/c/zuul/nodepool/+/877560 | ||
| - [zuul/nodepool] 877561: Add a LazyExecutorTTLCache to the OpenStack driver https://review.opendev.org/c/zuul/nodepool/+/877561 | ||
| - [zuul/nodepool] 878648: Mutex access to local lock attributes https://review.opendev.org/c/zuul/nodepool/+/878648 | ||
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 878725: Check Gerrit submit requirements https://review.opendev.org/c/zuul/zuul/+/878725 | 23:11 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 878725: Check Gerrit submit requirements https://review.opendev.org/c/zuul/zuul/+/878725 | 23:16 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!