| *** rfolco has quit IRC | 00:32 | |
| *** hamalq has joined #zuul | 01:16 | |
| *** hamalq has quit IRC | 01:18 | |
| *** lseki has quit IRC | 01:31 | |
| *** hamalq has joined #zuul | 01:33 | |
| *** sgw1 has quit IRC | 01:35 | |
| *** hamalq has quit IRC | 01:39 | |
| *** sgw1 has joined #zuul | 01:50 | |
| *** swest has quit IRC | 01:54 | |
| *** swest has joined #zuul | 02:09 | |
| *** hamalq has joined #zuul | 02:32 | |
| *** hamalq has quit IRC | 02:36 | |
| *** sgw1 has quit IRC | 02:47 | |
| *** hamalq has joined #zuul | 02:48 | |
| *** bhavikdbavishi has joined #zuul | 02:50 | |
| *** hamalq has quit IRC | 02:52 | |
| *** bhavikdbavishi1 has joined #zuul | 02:53 | |
| *** bhavikdbavishi has quit IRC | 02:54 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 02:54 | |
| *** rlandy|ruck|bbl is now known as rlandy | 03:10 | |
| *** Goneri has quit IRC | 03:11 | |
| *** sanjayu_ has joined #zuul | 03:14 | |
| *** vishalmanchanda has joined #zuul | 03:15 | |
| *** hamalq has joined #zuul | 03:25 | |
| *** sgw1 has joined #zuul | 03:28 | |
| *** hamalq has quit IRC | 03:30 | |
| *** hamalq has joined #zuul | 03:54 | |
| *** hamalq has quit IRC | 03:58 | |
| *** dmellado has quit IRC | 03:59 | |
| *** wuchunyang has joined #zuul | 04:02 | |
| *** sgw1 has quit IRC | 04:06 | |
| *** wuchunyang has quit IRC | 04:07 | |
| *** hamalq has joined #zuul | 04:10 | |
| *** hamalq has quit IRC | 04:15 | |
| *** sgw1 has joined #zuul | 04:18 | |
| *** wuchunyang has joined #zuul | 04:18 | |
| *** wuchunyang has quit IRC | 04:23 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #zuul | 04:33 | |
| *** bhavikdbavishi has quit IRC | 04:51 | |
| *** y2kenny has quit IRC | 04:59 | |
| *** wuchunyang has joined #zuul | 05:11 | |
| *** wuchunyang has quit IRC | 05:15 | |
| *** ysandeep|PTO is now known as ysandeep | 05:18 | |
| *** bhavikdbavishi has joined #zuul | 05:19 | |
| *** wuchunyang has joined #zuul | 05:21 | |
| *** bhavikdbavishi1 has joined #zuul | 05:22 | |
| *** bhavikdbavishi has quit IRC | 05:24 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 05:24 | |
| *** wuchunyang has quit IRC | 05:26 | |
| *** wuchunyang has joined #zuul | 05:41 | |
| *** felixedel has joined #zuul | 05:44 | |
| *** wuchunyang has quit IRC | 05:48 | |
| *** sgw has quit IRC | 05:48 | |
| openstackgerrit | Felix Edel proposed zuul/zuul-jobs master: [DNM] Test upload return values https://review.opendev.org/737441 | 05:51 |
|---|---|---|
| openstackgerrit | Felix Edel proposed zuul/zuul-jobs master: [DNM] Test upload return values https://review.opendev.org/737441 | 06:19 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Introduce Patternfly 4 https://review.opendev.org/736225 | 06:23 |
| *** rpittau|afk is now known as rpittau | 06:34 | |
| *** felixedel19 has joined #zuul | 06:45 | |
| *** felixedel has quit IRC | 06:45 | |
| *** hamalq has joined #zuul | 06:46 | |
| *** felixedel19 is now known as felixedel | 06:46 | |
| *** hamalq has quit IRC | 06:47 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 06:58 |
| openstackgerrit | Felix Edel proposed zuul/zuul-jobs master: Return upload_results in upload-logs-swift role https://review.opendev.org/733564 | 07:02 |
| *** hamalq has joined #zuul | 07:03 | |
| *** jcapitao has joined #zuul | 07:04 | |
| *** bhavikdbavishi has quit IRC | 07:06 | |
| *** hamalq has quit IRC | 07:08 | |
| *** bhavikdbavishi has joined #zuul | 07:21 | |
| *** bhavikdbavishi has quit IRC | 07:25 | |
| *** hashar has joined #zuul | 07:26 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Introduce Patternfly 4 https://review.opendev.org/736225 | 07:32 |
| *** tosky has joined #zuul | 07:35 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Introduce Patternfly 4 https://review.opendev.org/736225 | 07:36 |
| *** jpena|off is now known as jpena | 07:54 | |
| *** bhavikdbavishi has joined #zuul | 08:06 | |
| *** hamalq has joined #zuul | 08:08 | |
| *** nils has joined #zuul | 08:08 | |
| *** harrymichal has joined #zuul | 08:12 | |
| *** hamalq has quit IRC | 08:12 | |
| *** hashar has quit IRC | 08:16 | |
| *** hashar_ has joined #zuul | 08:16 | |
| *** hamalq has joined #zuul | 08:24 | |
| *** hashar_ is now known as hashar | 08:25 | |
| *** hamalq has quit IRC | 08:29 | |
| *** hamalq has joined #zuul | 08:30 | |
| *** hamalq has quit IRC | 08:30 | |
| *** ysandeep is now known as ysandeep|lunch | 08:37 | |
| *** hamalq has joined #zuul | 08:46 | |
| *** ysandeep|lunch is now known as ysandeep | 08:50 | |
| *** hamalq has quit IRC | 08:51 | |
| *** SotK has quit IRC | 09:08 | |
| *** tobiash has quit IRC | 09:08 | |
| *** SotK has joined #zuul | 09:09 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 09:11 | |
| *** tobiash has joined #zuul | 09:12 | |
| *** hamalq has joined #zuul | 09:21 | |
| *** hamalq has quit IRC | 09:26 | |
| *** hashar is now known as hasharAway | 09:31 | |
| *** felixedel has quit IRC | 09:33 | |
| mhu | Looks like there's a problem on Zuul-upload-image: FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/zuul/ansible/2.8/bin/pip': '/usr/local/lib/zuul/ansible/2.8/bin/pip' | 10:07 |
| *** rpittau is now known as rpittau|bbl | 10:12 | |
| *** hasharAway is now known as hasharLunch | 10:16 | |
| *** jcapitao is now known as jcapitao_lunch | 10:35 | |
| *** jhesketh has quit IRC | 10:36 | |
| *** jhesketh has joined #zuul | 10:37 | |
| *** holser_ has joined #zuul | 10:39 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: [WIP] Web UI: add i18n support, french translations https://review.opendev.org/737290 | 10:39 |
| *** holser has quit IRC | 10:41 | |
| avass | is there any way to configure the zuul log streaming port per node? | 10:48 |
| avass | like, we want to run multiple containers on a static node instead of just using multiple users. but that causes problems with the log streaming since they would all compete for the same port | 10:49 |
| avass | but being able to configure that port the same way you can configure the ssh port would solve that | 10:50 |
| *** wuchunyang has joined #zuul | 10:56 | |
| *** wuchunyang has quit IRC | 10:59 | |
| *** jpena is now known as jpena|lunch | 11:31 | |
| *** bhavikdbavishi has quit IRC | 11:46 | |
| openstackgerrit | Albin Vass proposed zuul/nodepool master: WIP: aws: add support for uploading diskimages https://review.opendev.org/735217 | 11:48 |
| *** bhavikdbavishi has joined #zuul | 11:48 | |
| openstackgerrit | Albin Vass proposed zuul/nodepool master: static driver: make log stream port configurable https://review.opendev.org/737501 | 11:49 |
| openstackgerrit | Albin Vass proposed zuul/nodepool master: static driver: make log stream port configurable https://review.opendev.org/737501 | 11:49 |
| *** hasharLunch is now known as hashar | 11:51 | |
| openstackgerrit | Albin Vass proposed zuul/zuul master: make log stream port configurable https://review.opendev.org/737502 | 11:54 |
| *** ysandeep is now known as ysandeep|afk | 12:00 | |
| *** rlandy has joined #zuul | 12:00 | |
| *** rlandy is now known as rlandy|ruck | 12:01 | |
| *** rfolco has joined #zuul | 12:01 | |
| guillaumec | avass, on top of the tutorials i'm working on, I have some pending changes about log stream and a more special one where I fix max-parallel-jobs on a static node allowing multiple jobs to run as the same user on a static node. | 12:07 |
| guillaumec | avass, i didn't see any issue, only 1 zuul_console can be up, but multiple zuul_stream are connecting to it and look at their own log file: /tmp/console-{log_uuid}.log | 12:08 |
| *** rpittau|bbl is now known as rpittau | 12:14 | |
| avass | guillaumec: yeah, but there would still be issues where you have multiple jobs locking dpkg at the same time, or installing conflicting versions of packages | 12:14 |
| *** jcapitao_lunch is now known as jcapitao | 12:15 | |
| avass | I guess you could bind mount /tmp and start the log streamer on the host. but since it's started by ansible it's easier to just forward a port and make the port configurable | 12:16 |
| *** ysandeep|afk is now known as ysandeep | 12:28 | |
| *** hamalq has joined #zuul | 12:30 | |
| guillaumec | avass, it did happened, but as ensure-* are executed in pre-run playbooks, first retry is launched, and as the package is now installed, it passes. | 12:32 |
| avass | yeah but that's still slower jobs compared to running them in containers where that's possible | 12:33 |
| guillaumec | I'm just pointing that I had no problem with multiple jobs using the same zuul_console | 12:33 |
| avass | yeah | 12:33 |
| avass | we already do that, but with multiple users. We just want to be able to contain the jobs | 12:33 |
| *** hamalq has quit IRC | 12:35 | |
| *** jpena|lunch is now known as jpena | 12:35 | |
| *** hamalq has joined #zuul | 12:46 | |
| *** sgw has joined #zuul | 12:49 | |
| *** hamalq has quit IRC | 12:51 | |
| *** felixedel has joined #zuul | 12:56 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: ensure-pip debian: update package lists https://review.opendev.org/737529 | 12:57 |
| felixedel | zuul-maint: I'd like to kindly ask for a review of https://review.opendev.org/#/c/633501/ and https://review.opendev.org/#/c/703983/ | 12:58 |
| openstackgerrit | Thierry Carrez proposed zuul/zuul-jobs master: upload-git-mirror: check after mirror operation https://review.opendev.org/737533 | 13:05 |
| ttx | fungi, clarkb, corvus: this should avoid false negatives on the upload-git-mirror runs. I could not find any occurrence where the mirror was out-of-sync recently, but that check should make it sure | 13:07 |
| ttx | I tested the commands locally but the env is probably different on the worker node, so... let me know when you merge it so that I can watch it run | 13:09 |
| corvus | avass: why does the streaming port need to be different? | 13:25 |
| avass | corvus: if you run multiple containers as 'static hosts' you need to forward different ports for each container | 13:26 |
| corvus | avass: i don't see why; it's built to support multiple uses | 13:27 |
| avass | I mean multiple containers on the same host | 13:27 |
| corvus | avass: oh, are you saying you want to run the log streamer inside a container on a static host? | 13:28 |
| avass | I believe ansible starts zuul_console inside the container and the callback tries to connect to a specific port | 13:28 |
| avass | yep | 13:28 |
| corvus | avass: can you use k8s instead of running in containers on the static node? | 13:29 |
| avass | we have fantastic software that wants to start virtual machines and has hard-coded paths :) | 13:29 |
| corvus | avass: are the containers pre-created? | 13:30 |
| avass | yeah | 13:30 |
| avass | or, yeah we create them ourselves | 13:30 |
| avass | I believe | 13:31 |
| corvus | but not in the job -- a container == a static node in nodepool? | 13:31 |
| avass | exactly | 13:31 |
| corvus | avass: can you run a global log streamer on the host and bind-mount in the tmp dir? | 13:31 |
| avass | I had that idea earlier, and yes :) | 13:32 |
| corvus | avass: another solution could be this: https://review.opendev.org/541434 | 13:32 |
| *** sgw1 has quit IRC | 13:34 | |
| avass | that connects from the remote to the executor instead of the executor connecting to the remote if I understand that correctly | 13:35 |
| *** sgw1 has joined #zuul | 13:35 | |
| avass | and yep that could work | 13:36 |
| corvus | avass: no, it's still executor to remote (that's important for firewalls). but it's just part of the ssh session | 13:36 |
| corvus | ssh connections are complicated; they can carry more than one connection. | 13:37 |
| corvus | but yes, then on the remote, it looks like it's making an outbound connection (though the next idea is to use unix domain sockets: https://review.opendev.org/542469 ) | 13:38 |
| avass | uh, yeah I got that backwards | 13:38 |
| avass | sure, I can take a look at that later | 13:40 |
| corvus | it's apparently been a hard problem, but it'll make a lot of things better, and would solve this without adding extra ties between zuul/nodepool | 13:40 |
| avass | sounds good, I've been wondering how the relationship between nodepool and zuul should be treated. it's a 'companion application' but how exclusive to zuul is it really? | 13:42 |
| avass | we might want to use it for other purposes to be able to share resources easier :) | 13:43 |
| corvus | i think we try to keep zuul as a 'consumer' of nodepool -- so a one-way dependency of zuul on nodepool. if we have to add something specific for zuul, we will, but it should at least be optional. the fewer things we add, the cleaner it is. | 13:44 |
| corvus | i'm not a hard -2 on the log port thing; if that's the only way to do it we can add it, but it's pretty zuul-specific, so it starts to add to the nodepool -> zuul linkage | 13:46 |
| avass | yeah I can see that | 13:47 |
| *** bhavikdbavishi has quit IRC | 13:49 | |
| mordred | tobiash: zuul-manage-ansible installs the virtualenvs into sys.exec_prefix by default. This means if you use a distro supplied python you'll wind up with them in /usr/lib/zuul rather than /usr/local/lib/zuul - even if zuul itself and its binaries are installed into /usr/local ... did we do that on purpose? | 13:50 |
| mordred | asking because my brain didn't have all of that paged in and I went looking for one of teh venv's to double-check what a version of a library was and then I was confused - it's not a big deal - just thought I'd check | 13:51 |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 13:51 | |
| tobiash | mordred: that's intended to use the same prefix as zuul's python interpreter | 13:51 |
| tobiash | no idea how that could fit into /usr/local/... | 13:52 |
| tobiash | the idea was that it lands within the venv if zuul is installed into a venv | 13:53 |
| mordred | ah - that makes sense | 13:53 |
| mordred | it's just feels weird to me to have things install into /usr/lib that weren't written there by distro package managers. but, I think the explanation of tying it to the python for virtualenvs makes a lot of sense | 13:53 |
| mordred | and I think that's more important | 13:54 |
| tobiash | if it feels too odd we could try to catch that use case | 13:55 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: ensure-pip debian: update package lists https://review.opendev.org/737529 | 13:58 |
| tristanC | avass: beware sharing tmp dir between containers may result in conflict when test code doesn't use uniq path for tmp files | 13:58 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Driver event ingestion https://review.opendev.org/717299 | 13:59 |
| avass | right, I hope that won't be a problem | 13:59 |
| mordred | tobiash, corvus: mind reviewing https://review.opendev.org/#/c/735739/ ? we need it in opendev before we can switch to docker containers for the executors | 14:01 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Add openafs-krb5 to bindep https://review.opendev.org/735739 | 14:02 |
| tobiash | mordred: we can add this but this is such a special dep so we might want to think if that would be better located in a downstream image that inherits from the official images. What do you think? | 14:04 |
| mordred | tobiash: I thought about that originally, but figured we had the afs jobs in zuul-jobs and it was a small package so maybe it was ok? I think we could also certainly do that if it feels like it's too much | 14:05 |
| tobiash | ah ok, if there is already stuff in zuul-jobs then it's probably the right location | 14:06 |
| corvus | tobiash, mordred: it's actually the sysctl support in zuul-bwrap that convinces me it's okay in the zuul image | 14:12 |
| corvus | i agree it's a really good question, and it's close to the line, and i don't want the images to get too big. but there's enough support required in zuul and zuul-jobs for this to make sense i think. | 14:13 |
| mordred | yeah - I think the sysctl support is actually the better argument | 14:13 |
| corvus | tobiash, mordred: there's also the idea that if these images get too big (because of stuff we add for things in zuul-jobs) we can make 2 versions of the zuul-executor image | 14:13 |
| corvus | tobiash, mordred: a lightweight and a heavyweight version | 14:13 |
| tobiash | yeah that makes sense if size becomes a problem | 14:15 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Driver event ingestion https://review.opendev.org/717299 | 14:24 |
| *** felixedel has quit IRC | 14:30 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 14:39 | |
| *** ysandeep is now known as ysandeep|afk | 14:39 | |
| corvus | fyi mnaser, mordred and i just had a conversation in #opendev about improving log upload robustness that's generally applicable here (basically, doing some kind of retry-to-other-log-region thing) | 14:45 |
| *** ysandeep|afk is now known as ysandeep | 14:46 | |
| *** Goneri has joined #zuul | 14:50 | |
| *** bhavikdbavishi has joined #zuul | 14:50 | |
| clarkb | guillaumec: re https://review.opendev.org/#/c/732066/22 it would be helpful to reviewers to not add changes until they are relevant to the change consuming them. | 14:53 |
| clarkb | guillaumec: adding duplicate files and unused variables is noise when attempting to review that particular change. Its ok to add the new file with delta and the new variable with use at the point they become useful | 14:54 |
| clarkb | guillaumec: all of the tutorials are added in series so we don't have to worry about merge conflicts either | 14:54 |
| *** bhavikdbavishi has quit IRC | 15:05 | |
| clarkb | mhu: tristanC: should we go ahead and land https://review.opendev.org/#/c/734134/7 ? | 15:08 |
| mhu | clarkb, yes please! unless you'd prefer me to remove the default list of verbs as you mentioned in your last comment | 15:09 |
| clarkb | mhu: no I think we can do that in a followon. I wanted to make sure it was seen before approving though in case it was more important to do that | 15:10 |
| mhu | clarkb, but there was a pb earlier with zuul-upload-image in the gate, is it fixed? | 15:10 |
| clarkb | pb? | 15:10 |
| *** bhavikdbavishi has joined #zuul | 15:10 | |
| mhu | Looks like there's a problem on Zuul-upload-image: FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/zuul/ansible/2.8/bin/pip': '/usr/local/lib/zuul/ansible/2.8/bin/pip' | 15:11 |
| clarkb | I'm just catching up on the reviews I did yseterday | 15:11 |
| mhu | clarkb, ^ | 15:11 |
| mhu | ok, I guess we can set workflow +1 but it might fail in the gate | 15:12 |
| clarkb | https://zuul.opendev.org/t/zuul/build/db967f3aaf0940c89d1b2c71e52261d2/log/job-output.txt#9566 | 15:12 |
| mhu | yes, similar one | 15:13 |
| mhu | this time it's ansible 2.9 though | 15:14 |
| clarkb | that is in the docker image's zuul ansible venvs | 15:15 |
| clarkb | I would expect that means this is not related to the pip and virtualenv changes on the VM images | 15:15 |
| clarkb | (since it is several layers deeper with docker and python3 venv | 15:15 |
| clarkb | ah we use virtualenv not venv. Maybe a bug in new virtualenv /me checks if they have done a recent release | 15:22 |
| clarkb | https://pypi.org/project/virtualenv/20.0.24/ is from yesterday | 15:22 |
| mhu | yay for breaking releases! \o/ :) | 15:23 |
| mordred | that is what was installed here | 15:23 |
| clarkb | for now that is my hunch that it isn't seeding pip properly but I don't have confirmation of that yet | 15:23 |
| clarkb | the changelog says they did change how seeding works | 15:23 |
| clarkb | https://virtualenv.pypa.io/en/latest/changelog.html | 15:23 |
| mordred | *AWESOME* | 15:23 |
| *** harrymichal has quit IRC | 15:24 | |
| mordred | clarkb: the behavior described seems fine | 15:26 |
| * mordred is going to try to replicate | 15:26 | |
| *** hamalq has joined #zuul | 15:28 | |
| mordred | clarkb, mhu: I cannot reproduce this behavior synthetically in a python-builder image pip installing virtualenv, creating a virtualenv and then calling the pip inside of it | 15:32 |
| clarkb | mordred: fwiw in that zuul job we do that like 4 times for all the ansibles and only one seems to fail | 15:32 |
| clarkb | (so it likely isn't a consistent failure whatever is causing it) | 15:33 |
| mordred | clarkb: I'm trying doing it multiple times | 15:34 |
| *** harrymichal has joined #zuul | 15:35 | |
| *** ysandeep is now known as ysandeep|away | 15:36 | |
| mordred | ok. I think I know what it is | 15:38 |
| mordred | sigh | 15:38 |
| mordred | SO ... in the new world order, there is a shared "/root/.local" dir which caches the seed packages on a per-python basis | 15:38 |
| clarkb | mordred: which was the problem with the first 20.0.0 release | 15:39 |
| mordred | we're installing our virtualenvs in a threadpool | 15:39 |
| clarkb | because that dir might go away or not be readable | 15:39 |
| mordred | so, while I haven't reproduced it - it would make perfect sense for a race condition between virtualenv commands to step on each other | 15:40 |
| mordred | what I think we shoudl do to be safer with how they're doing things ... | 15:40 |
| mordred | is to serially create the empty virtualenvs, then install the packages into those virtualenvs in parallel | 15:40 |
| clarkb | mordred: we can also set the seeder to pip | 15:40 |
| clarkb | whcih doesn't use the shared contents | 15:40 |
| clarkb | (I don't know if that has different potential races though) | 15:41 |
| clarkb | mordred: https://review.opendev.org/#/c/706871/2/zuul/lib/ansible.py go back to doing that basically | 15:44 |
| clarkb | I think we cleaned that up when 20.0.3 fixed it for us | 15:44 |
| clarkb | but we could also use seeder = pip | 15:44 |
| mordred | clarkb: I've got a patch coming | 15:44 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Create virtualenvs in series to avoid cache race https://review.opendev.org/737565 | 15:46 |
| mordred | clarkb, mhu ^^ | 15:46 |
| mhu | mordred, thanks! and good catch | 15:46 |
| mordred | also - I should update python-builder to do an rm of /root/.local/share/virtualenv when it's done | 15:47 |
| mordred | mnaser: ^^ | 15:47 |
| clarkb | mordred: should yuo remove the ensure_venv call in ensure_ansible? | 15:48 |
| clarkb | we're gonna get some logging noise at least otherwise | 15:48 |
| clarkb | also do we think using the pip seeder would address that? | 15:48 |
| clarkb | and can do everything in aprallel? | 15:48 |
| mordred | clarkb: we could remove the ensure_venv call, yeah. | 15:50 |
| mordred | I don't know about the pip seeder... not sure I have much opinion | 15:51 |
| corvus | if we remove that call, we should make sure it's not requierd (ie, by some other code path) | 15:51 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Create virtualenvs in series to avoid cache race https://review.opendev.org/737565 | 15:52 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Create virtualenvs in series to avoid cache race https://review.opendev.org/737565 | 15:53 |
| corvus | it looks like ensure_ansible only has the one invocation | 15:53 |
| mordred | corvus: yeah | 15:53 |
| corvus | (but there are other invocations of ensure_venv, which lead me to check this) | 15:53 |
| mordred | clarkb: I think I lean more towards serial-virtualenv - largely because once we get into alternate seeder land I worry that we'll be using a less-used codepath and our friendly upstreams might be more likely to break us | 15:53 |
| corvus | mordred: mhu also had a commitmsg nit if you're redoing stuff | 15:55 |
| *** hamalq has quit IRC | 15:55 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Create virtualenvs in series to avoid cache race https://review.opendev.org/737565 | 15:55 |
| mordred | corvus, mhu: ^^ | 15:55 |
| *** hamalq has joined #zuul | 15:55 | |
| bolg | yay | 15:55 |
| clarkb | mordred: not opposed just wondering if pip would be more resilient since it relies on the regular tool and not special cases | 15:56 |
| mordred | clarkb: yeah - it's a totally fair point. why don't I also push up a patch with that and we can recheck it a bunch and see how it does | 15:57 |
| *** hamalq_ has joined #zuul | 15:57 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Use seeder=pip in virtualenv creation https://review.opendev.org/737567 | 15:59 |
| mordred | clarkb: ^^ | 15:59 |
| *** rpittau is now known as rpittau|afk | 16:00 | |
| *** hamalq has quit IRC | 16:00 | |
| clarkb | mordred: oh I was suggesting that we do both things if we are extra worried. We can serialize virtualenvs but also seed with pip to avoid pesky cache dirs outside of normal expectations | 16:01 |
| clarkb | I guess we'll get infos either way | 16:01 |
| corvus | speaking of -- i just ran 'tox -epy36' in a project and i'm now sitting at /usr/bin/python -c from virtualenv.seed.wheels.periodic_update import do_update;do_update(u'setuptools', '3.6', '/usr/local/lib/python2.7/dist-packages/virtualenv/seed/wheels/embed/setuptools-47.3.1-py3-none-any.whl', '/home/corvus/.local/share/virtualenv', [], True) | 16:09 |
| corvus | so i guess time to get a coffee or something | 16:09 |
| mhu | Is there a list of a build's possible statuses somewhere? SUCCESS, FAILURE, SKIPPED, POST_FAILURE, anything else? | 16:13 |
| *** sgw1 has quit IRC | 16:13 | |
| corvus | mhu: i don't believe so | 16:13 |
| corvus | mhu: if you wanted to make a file of constants and replace all the string occurrences with that, that would be a fine idea i think :) | 16:14 |
| corvus | 'SUCCESS' -> build_results.SUCCESS | 16:14 |
| mhu | corvus, yeah I think that'd be handy, there's already a list of such values for node states | 16:14 |
| mhu | (and helpful for translations) | 16:14 |
| corvus | mhu: i think they are mostly in model.py and executor/server.py | 16:15 |
| *** mgoddard has quit IRC | 16:15 | |
| mhu | so there aren't any other states for builds right? | 16:16 |
| corvus | node_failure comes to mind | 16:16 |
| corvus | felix is planning on adding retry | 16:16 |
| mhu | right, the dreaded node_failure | 16:17 |
| AJaeger | NODE_FAILURE | 16:17 |
| AJaeger | RETRY_LIMIT | 16:17 |
| AJaeger | TIMED_OUT | 16:18 |
| AJaeger | CANCELLED | 16:18 |
| AJaeger | mhu: hope that's it ;) | 16:19 |
| *** sgw1 has joined #zuul | 16:19 | |
| mhu | AJaeger, thanks! that'll do it for now | 16:19 |
| AJaeger | SKIPPED as well, I think | 16:19 |
| AJaeger | ABORTED - not sure when that happens | 16:20 |
| mhu | what's the difference between ABORTED, CANCELLED, SKIPPED? | 16:21 |
| *** mgoddard has joined #zuul | 16:21 | |
| mhu | or at least between CANCELLED and ABORTED | 16:21 |
| corvus | mhu: canceled (note the spelling) is by request of the scheduler, aborted is unexpected error on executor | 16:24 |
| avass | there also 'ERROR' isn't there? | 16:24 |
| avass | but maybe that isn't a build result | 16:25 |
| corvus | i think a buildset can have an error state, but not a build | 16:25 |
| corvus | config_error at least | 16:25 |
| * mhu thinks it'd be indeed good to get all these in constants | 16:25 | |
| corvus | oh, yep there is an ERROR for builds | 16:26 |
| *** jcapitao has quit IRC | 16:26 | |
| clarkb | mhu: I'll approve the OPTIONS change now as mordred's virtualenv change is in the gate | 16:26 |
| mhu | clarkb, sweet thanks | 16:26 |
| avass | yeah I've seen ERROR once or twice, it's rare :) | 16:26 |
| corvus | i think having a role with a plugin causes error | 16:26 |
| corvus | basically, the executor can't run the job | 16:27 |
| clarkb | that will send everything through https://review.opendev.org/#/c/728098/8 into the gate | 16:27 |
| mhu | I would also requeue the followup patches in +3 but my girlfriend is banging at the door to get us to take a walk :) | 16:27 |
| corvus | avass: what's the status on the buildx test fix? | 16:27 |
| clarkb | mhu: it should happen automatically for that whole stack that is already approved | 16:28 |
| clarkb | mhu: if not I'll prod them. You go enjoy your walk | 16:28 |
| avass | corvus: it's failing on 'connection refused' but not sure why yet: https://review.opendev.org/#/c/737315/ | 16:28 |
| mhu | awesome, my gf will be happy to hear that! :) | 16:28 |
| avass | corvus: here: https://zuul.opendev.org/t/zuul/build/0f73b5c3f9ec4aa6b33e32f270f2e2b2/log/job-output.txt#848 | 16:28 |
| avass | unless I missed something I believe that's the only thing that causes the tests to fail | 16:30 |
| clarkb | mordred: hrm your change failed in the gate https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ba4/737565/4/gate/zuul-stream-functional-2.7/ba4ea67/job-output.txt | 16:34 |
| clarkb | mordred: that does look like an ensure-pip problem though. | 16:34 |
| corvus | avass: why did you start using zuul-jobs.buildset-registry in the cert? | 16:39 |
| avass | corvus: because I'm still not enitrely sure how certificates works and: 737315 | 16:43 |
| avass | oops | 16:43 |
| avass | https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/use-buildset-registry/tasks/main.yaml#L25 | 16:43 |
| corvus | avass: right, but we're not using the buildset-registry in this job, right? | 16:44 |
| avass | corvus: we have to since buildx requires it | 16:44 |
| corvus | oh interesting | 16:44 |
| avass | but it's the same registry as the one we push the release to | 16:44 |
| corvus | er, aren't you setting up a new registry in the pre playbook? | 16:45 |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Run ensure-pip in the console stream functional job https://review.opendev.org/737582 | 16:45 |
| corvus | "Start registry with basic auth" is starting a new docker library/registry:2 image | 16:45 |
| clarkb | mordred: ^ I can't see anything else installing pip so I think we need that | 16:46 |
| clarkb | oh we don't run the stream functional jobs most of the time | 16:46 |
| clarkb | that explains it | 16:46 |
| avass | corvus: I moved the registry to the pre playbook | 16:46 |
| clarkb | zuulians I think https://review.opendev.org/737582 is a bug fix necessary to land mordreds bug fix | 16:46 |
| corvus | avass: right, but you're starting a second registry, in addition to the buildset registry, right? | 16:46 |
| avass | corvus: nope, it's the same registry | 16:47 |
| corvus | avass: i think i'm going to need you to use a lot more words | 16:47 |
| avass | w:) | 16:47 |
| avass | corvus: the pre playbook sets up the registry and I configure buildx to use that with use-buildset-registry, but the same registry is used in upload-docker-image | 16:48 |
| avass | but the buildx dance pushes the images with specific tags, pulls them, re-tags them for release and pushes them | 16:49 |
| avass | or that's what supposed to happen | 16:49 |
| corvus | avass: let's introduce a new term: the "publication registry". in the plain docker version of this, that's the registry you set up to stand in for dockerhub | 16:50 |
| corvus | avass: i think we should keep the publication registry and the buildset registry separate | 16:50 |
| avass | sure | 16:50 |
| corvus | since the approach you're using to test this basically involves pushing an image to a different registry than dockerhub anyway, i think that may make sense | 16:51 |
| corvus | avass: oh, but i just realized something -- i'm not sure we should have a docker_registry variable on the upload role -- because if you want to publish to something other than dockerhub, you should just name your image tag appropriately | 16:51 |
| avass | yeah, but we could do that after we figured out why we're getting 'connection refused' since I believe it should look the same anyway, only with two registries running | 16:51 |
| avass | corvus: I added it for docker login | 16:52 |
| *** pabelanger has joined #zuul | 16:52 | |
| corvus | avass: we can probably just docker login before running the role | 16:52 |
| avass | but I guess that could be changed to parse the tag instead | 16:52 |
| corvus | avass: oh i see what you mean | 16:53 |
| corvus | so the docker login in the upload role knows which registry to log in to | 16:53 |
| avass | yeah | 16:53 |
| corvus | avass: okay, let's keep that as is for now | 16:53 |
| corvus | avass: i think it may be worth untangling the buildset/publication registry stuff now though; i think it will make things easier to debug | 16:53 |
| corvus | the buildset registry is really complicated, but the publication registry can be pretty simple | 16:54 |
| corvus | avass: do you want to try revising the patch to do that, and i can put in an autohold so we can try to fix it if we still have problems? | 16:54 |
| avass | sure | 16:55 |
| pabelanger | I wanted to ask about https://zuul-ci.org/docs/zuul/reference/jobs.html#pausing-the-job we. We are about to start work on buildset galaxy servers for testing ansible collections, but in some cases we have child jobs that run more then 2 hours. What I was hoping to figure out, in our case we only need the galaxy server online until a collection has been downloaded from it. Keepig it only after that, | 16:56 |
| pabelanger | isn't efficent and consumes limited resources. Basically, trying to ask, if we added zuul_return unpause support, if it would help in this case. | 16:56 |
| avass | it shouldn't be too hard, but it should be enough to run a normal docker registry for that right? | 16:56 |
| *** sgw1 has quit IRC | 16:56 | |
| pabelanger | basically, want to use zuul_return to unpause a paused job, not unpause when all child jobs finish | 16:56 |
| corvus | avass: yeah, just like you were doing before | 16:57 |
| corvus | pabelanger: hi! let me think about that a bit | 16:57 |
| pabelanger | thanks, no rush | 16:58 |
| corvus | pabelanger: the biggest challenge is going to be in returning data to the executor without ending the child job's playbook. currently the executor only reads that file when a job ends. | 16:59 |
| clarkb | mordred: if my role inclusion fails on the upload image thing we may need to sqash the changes. Any objection to doing that? (and ideas on alternatives?) | 16:59 |
| corvus | pabelanger: however, i think being able to return data to the executor during a job run is great and we should do it -- it lets us do things like indicate that a job is going to fail even before it finishes. will speed things up. | 16:59 |
| corvus | pabelanger: but we still have to figure out how to do that :) | 17:00 |
| corvus | pabelanger: the second thing to consider is a paused job with multiple children. does it unpause when all children tell it to unpause (or they all finish)? i think that makes sense, but we'd want to be explicit. | 17:00 |
| corvus | pabelanger: for the first thing, we could probably just have the executor periodically read the return json file? | 17:01 |
| corvus | if we have it check every 30s or something, that's probably good enough and easy enough? | 17:02 |
| clarkb | hrm I think the stream functional jobs may be failing on the same thing that the upload image job fails on now that pip is there. So we may need to squash | 17:03 |
| pabelanger | corvus: so to answer the multi child job, yah I was thinking only unpause after _all_ child jobs complete / return unpause. But realize that means needing to track child jobs too | 17:03 |
| pabelanger | periodic read might work | 17:04 |
| corvus | pabelanger: i think the scheduler knows the child jobs at that point, so that should be okay. | 17:05 |
| *** nils has quit IRC | 17:05 | |
| *** sgw1 has joined #zuul | 17:05 | |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Rework quick-start and prepare for other tutorials https://review.opendev.org/732066 | 17:05 |
| pabelanger | cool | 17:05 |
| corvus | pabelanger: cause i think the child jobs will tell the executor to unpause their parent, the executor will pass that information to the scheduler, the scheduler would decide to do that and tell the executor to actually perform the unpause | 17:05 |
| pabelanger | great, so sounds like something we could support, if I find the time to dig into it | 17:05 |
| corvus | pabelanger: ++ :) | 17:06 |
| corvus | avass: autohold in place | 17:07 |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Create virtualenvs in series to avoid cache race https://review.opendev.org/737565 | 17:09 |
| clarkb | corvus: mordred ^ squashed them as they won't merge without some intervention like that | 17:09 |
| *** jpena is now known as jpena|off | 17:11 | |
| corvus | clarkb: +w i think that's appropriate, urgent, and non-controversial | 17:11 |
| clarkb | thanks | 17:11 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 17:14 |
| avass | corvus: I believe doing something like that ^ should be enough | 17:14 |
| avass | actually no, both registries use the same port now | 17:15 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 17:15 |
| corvus | avass: yeah, and you should be able to drop the zuul-jobs.buildset-registry name from the cert; basically, i'm hoping we can get it just like the non-buildx case | 17:15 |
| avass | ah yep, I'll do that | 17:16 |
| *** pabelanger has left #zuul | 17:16 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 17:17 |
| avass | that should do it | 17:17 |
| avass | corvus: uh, is the buildset registry normally using a password? | 17:18 |
| avass | corvus: because this looks suspicious: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/use-buildset-registry/tasks/main.yaml#L25 | 17:19 |
| avass | looks like run-buildset-registry never configures any authentication | 17:20 |
| corvus | avass: it does here: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/run-buildset-registry/templates/registry.yaml.j2#L10 | 17:21 |
| avass | oh yep | 17:21 |
| corvus | generated here: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/run-buildset-registry/tasks/main.yaml#L30 | 17:21 |
| corvus | avass: here's the consuming side: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/use-buildset-registry/tasks/user-config.yaml#L25 | 17:22 |
| avass | oh sorry got the wrong link, that's what I was supposed to link :) | 17:22 |
| corvus | i have to run; back in ~30m | 17:23 |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 17:24 | |
| *** hashar has quit IRC | 17:25 | |
| *** hashar has joined #zuul | 17:25 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 17:33 |
| *** sgw1 has quit IRC | 17:34 | |
| *** harrymichal has quit IRC | 17:40 | |
| *** harrymichal has joined #zuul | 17:40 | |
| *** sgw1 has joined #zuul | 17:50 | |
| corvus | avass: do you have an ssh published somewhere i can copy? github launchpad etc? or if you just want to pastebin it, i can add your key to the held node. | 18:12 |
| avass | corvus: http://paste.openstack.org/show/795115/ | 18:15 |
| avass | corvus: it got one step further now though. it fails at being able to pull from the buildset registry instead :) | 18:15 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 18:17 |
| openstackgerrit | Merged zuul/zuul master: Create virtualenvs in series to avoid cache race https://review.opendev.org/737565 | 18:21 |
| openstackgerrit | Merged zuul/zuul master: Report retried builds in a build set via mqtt. https://review.opendev.org/632727 | 18:21 |
| corvus | avass: hrm, it grabbed a different build than that failed one; i wonder if the new patchsets caused that | 18:21 |
| corvus | avass: i'll clear it out and reset the hold | 18:22 |
| openstackgerrit | Merged zuul/zuul master: Report retried builds via sql reporter. https://review.opendev.org/633501 | 18:22 |
| clarkb | tobiash: did you see my note about respinning https://review.opendev.org/#/c/730624/ ? | 18:23 |
| *** hashar is now known as hasharAway | 18:23 | |
| clarkb | I think that would be a good improvement to land to the windowing if you've got time for it. Otherwise I can give ti a go if you'd prefer | 18:23 |
| tobiash | clarkb: yes I saw it. I'm out of office this week so feel free to respin this week if you like. Otherwise I'll respin it on monday | 18:24 |
| *** harrymichal has quit IRC | 18:26 | |
| *** bhavikdbavishi has quit IRC | 18:33 | |
| avass | corvus: I'm gonna guess that the buildset registry is not set up correctly and that docker tries to pull from dockerhub | 18:33 |
| openstackgerrit | Merged zuul/zuul master: zuul-web: support OPTIONS for protected endpoints https://review.opendev.org/734134 | 18:33 |
| *** harrymichal has joined #zuul | 18:35 | |
| corvus | avass: huh? | 18:35 |
| openstackgerrit | Merged zuul/zuul master: zuul-web: refactor auth token handling code https://review.opendev.org/734139 | 18:35 |
| openstackgerrit | Merged zuul/zuul master: CLI: Fix errors with the REST client https://review.opendev.org/728061 | 18:35 |
| corvus | avass: what are you looking at? | 18:36 |
| avass | corvus: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fc5/737315/23/check/zuul-jobs-test-build-docker-image-release-multiarch/fc5c501/job-output.txt | 18:38 |
| openstackgerrit | Merged zuul/zuul master: REST API: fix discrepancies between RPC and REST outputs for autohold https://review.opendev.org/728073 | 18:38 |
| avass | corvus: "Error response from daemon: pull access denied for testrepo, repository does not exist or may require 'docker login': denied: requested access to the resource is denied" | 18:38 |
| avass | at least that's my best guess at the moment | 18:39 |
| *** harrymichal has quit IRC | 18:42 | |
| *** harrymichal has joined #zuul | 18:43 | |
| corvus | avass: i still see you're doing stuff with 'buildest_registry' in your change | 18:44 |
| corvus | avass: you should not invoke the buildset-registry roles in this change; you need a completely new registry for the publication registry. and you definitely don't want to set it up with use-buildset-registry. | 18:45 |
| *** y2kenny has joined #zuul | 18:46 | |
| clarkb | tobiash: ah enjoy your time off. I'll see if I can get a respin at some point | 18:46 |
| tobiash | thanks! | 18:47 |
| *** armstrongs has joined #zuul | 18:47 | |
| corvus | avass: i think what you already had is pretty close: just start a registry, log in to it, and tell buildx to push to it | 18:47 |
| avass | corvus: build-docker-image requires a buildset registry for buildx | 18:48 |
| corvus | avass: yes, but you should be able to ignore that | 18:48 |
| avass | I'm not sure I follow. how? | 18:48 |
| openstackgerrit | Merged zuul/zuul master: Add simple testing for Zuul CLI & REST API https://review.opendev.org/728098 | 18:49 |
| corvus | avass: the theory behind your test is that we can't push to dockerhub because it's a production service, but we can push to a private registry. so your test creates a new private registry and pushes to it. it should have no relationship with either the intermediate or buildset registries (otherwise, it's not a valid test) | 18:50 |
| avass | corvus: yeah | 18:50 |
| corvus | avass: so just start a new registry, run it on port, i dunno, 5050, then build and push to localhost:5050/imagename and that should be it | 18:51 |
| avass | corvus: but we need to build an image with buildx using build-docker-image and that requires a buildset-registry because of the way that role works | 18:51 |
| corvus | avass: yes, but that's already taken care of, you should be able to ignore it | 18:51 |
| avass | it is? | 18:51 |
| *** armstrongs has quit IRC | 18:55 | |
| corvus | avass: are you saying the buildset-registry roles are not used in zuul-jobs-test-build-docker-image-release jobs? then it may be worth looking at how zuul-jobs-test-registry-docker-multiarch sets them up | 18:56 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Rework quick-start and prepare for other tutorials https://review.opendev.org/732066 | 18:56 |
| corvus | avass: regardless, i think the important point is that your publication registry should not be the buildset registry | 18:57 |
| avass | corvus: it's not anymore, run-buildset-registry sets it up doesn't it? | 18:57 |
| avass | so there's a buildset registry started from that role, and another one started like it was without buildx | 18:58 |
| corvus | avass: in the latest version of your change i see tasks like like "Set up up buildset registry" | 18:58 |
| corvus | it still looks like you're setting up your publication registry and passing its info to use-buildset-registry | 18:58 |
| corvus | that's going to mix the two up | 18:58 |
| corvus | if you need a buildset registry, then run run-buildset-registry, then use-buildset-registry, then don't do anything else with the buildset registry. :) then start your publication registry on a different port, and pass it to the build role for login and push | 19:00 |
| corvus | gotta run again | 19:00 |
| avass | I'm confused, that's what I'm doing | 19:00 |
| openstackgerrit | Merged zuul/zuul-jobs master: Simplify twine invocation for PyPI uploads https://review.opendev.org/735932 | 19:01 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 19:02 |
| corvus | avass: look at your change, and the release-pre.yaml playbook. line 65 you start the publication registry. line 80 you load the cert from the publication registry as the buildset_registry | 19:05 |
| *** lseki has joined #zuul | 19:07 | |
| avass | corvus: you the slurp task? that was just left by mistake and shouldn't have affected anything | 19:07 |
| avass | the 'set_fact' is there to cache the output from run-buildset-registry since the role itself doesn't do that | 19:08 |
| corvus | yes, the slurp is what caught my eye, if that's really a noop, then that may not be the issue. would probably be a good idea to remove it. :) | 19:10 |
| avass | already done :) | 19:11 |
| avass | maybe run-buildset-registry should cache the fact? it seems so when I look at the docs: https://zuul-ci.org/docs/zuul-jobs/docker-image.html#yoursite-build-docker-image | 19:15 |
| avass | is this why we load information from results.json? https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/build-docker-image/tasks/main.yaml#L10 | 19:15 |
| y2kenny | What is the expected behaviour with executor-zone for executor only jobs? (i.e. no nodeset) | 19:19 |
| avass | y2kenny: i guess the job would land on an executor in that zone | 19:19 |
| corvus | avass: yeah maybe it should; i don't think we had any fact caching going on at the time. | 19:20 |
| y2kenny | avass: but there's no zone definition since there is no nodeset. What I am see is that it won't schedule to any executor that has zone define. That's probably expected? | 19:20 |
| avass | y2kenny: oh, I guess it would land on any executor then. let me check | 19:21 |
| *** sgw1 has quit IRC | 19:21 | |
| *** sgw1 has joined #zuul | 19:22 | |
| avass | y2kenny: reading the docs make me believe it would land on any unzoned executor: https://zuul-ci.org/docs/zuul/discussion/components.html#attr-executor.zone | 19:23 |
| fungi | i think the expectation is that unless you need executor zoning, you should not set zones on your executors | 19:25 |
| y2kenny | fungi, avass: I see. Thanks for confirming. | 19:25 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 19:31 |
| tobiash | y2kenny, avass: without https://review.opendev.org/#/c/673840 you need at least one unzoned executor for this use case | 19:35 |
| fungi | we run without any executor zones, so all executors can use nodes from any provider, but if you have multiple providers and specific executors can only reach nodes in particular providers (perhaps the executors are in those providers and have dedicated private networks through which they communicate with nodes) then you'd set up zones for something like that | 19:36 |
| y2kenny | tobiash: yes, that's exactly what I am seeing. | 19:37 |
| fungi | i hadn't seen 673840 yet, neat | 19:38 |
| fungi | i agree that looks useful | 19:38 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "gate your first patch" https://review.opendev.org/732067 | 19:42 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "gate your first patch" https://review.opendev.org/732067 | 19:47 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 19:53 |
| openstackgerrit | Merged zuul/zuul master: CLI: add documentation on promote https://review.opendev.org/737027 | 20:14 |
| *** harrymichal has quit IRC | 20:23 | |
| *** harrymichal has joined #zuul | 20:24 | |
| *** sgw1 has quit IRC | 20:31 | |
| *** harrymichal has quit IRC | 20:33 | |
| *** harrymichal has joined #zuul | 20:34 | |
| mnaser | corvus: super unrelated but i've seen you do a lot with ruamel.yaml -- is there a way to create/ensure one single empty line between blocks? i'm dynamically generating jobs and it seems to put them directly with one after the other | 20:46 |
| *** sgw1 has joined #zuul | 20:47 | |
| fungi | i would use it to parse a yaml file like what you want, and then look at the data structure to see how it indicates the location of empty lines | 20:47 |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Don't decrease window size on merge failure https://review.opendev.org/730624 | 20:53 |
| clarkb | fungi: corvus ^ you previously reviewed that change if you want to look at it again | 20:54 |
| *** vishalmanchanda has quit IRC | 20:54 | |
| mordred | mnaser: I believe you can control that with formatting - or it's possible you have to write a custom formatter class | 20:55 |
| mordred | mnaser: you don't really nee ruamel.yaml to do it - I have nerdsniped myself to do that before, but have forgotten so I'd have to go learn again | 20:55 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "Use zuul jobs" https://review.opendev.org/732068 | 20:58 |
| fungi | i've overwritten pyyaml's IBSEmitter/IBSDumper to force indentation of block sequences | 20:58 |
| fungi | er, overridden | 20:58 |
| fungi | with subclass shadowing you can override a lot of the dumper behaviors fairly easily | 20:59 |
| fungi | er, i guess technically i subclassed yaml.emitter.Emitter and yaml.SafeDumper | 21:00 |
| fungi | but yaml.dump() allows you to pass a custom Dumper class, as well as having a number of options to influence flow style | 21:01 |
| corvus | mnaser: this does it for the auto-generated jobs in zuul-jobs: https://opendev.org/zuul/zuul-jobs/src/branch/master/tools/ruamellib.py#L44 | 21:02 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 21:04 |
| corvus | avass: i'm back around now; would the held node from build 'fc5c501d18944340a22b8b3b7b37122a' be useful? | 21:04 |
| avass | corvus: yes, but it's getting late and I'm planning on quitting for the night very soon | 21:05 |
| corvus | looks like that one is patchset 23: https://zuul.opendev.org/t/zuul/build/fc5c501d18944340a22b8b3b7b37122a | 21:05 |
| corvus | avass: okay, "ssh root@174.143.130.176" should work for you -- it'll be there whenever you want it :) | 21:06 |
| *** harrymichal has quit IRC | 21:08 | |
| avass | corvus: interesting, the registry lists the tags 3, 3.19, 3.19.0 but no manifests | 21:16 |
| mnaser | corvus: oh neat! | 21:18 |
| mordred | avass: that seems less than idea :) | 21:18 |
| mordred | ideal | 21:19 |
| avass | mordred: yeah and it looks like they should be there: https://zuul.opendev.org/t/zuul/build/fc5c501d18944340a22b8b3b7b37122a/log/job-output.txt#1010 | 21:19 |
| avass | but, it's getting way too late. I'll see if I can figure this out tomorrow | 21:19 |
| avass | mordred: oh whoops, one line above that :) | 21:20 |
| corvus | mhu: i'm looking at https://review.opendev.org/734082 and wondering about the callback setup -- you say you need to add an auth callback to google for every tenant, but that's not the case for keycloak. 1) why the difference? 2) should we think about making the auth_callback be a global-or-tenant endpoint like info? | 21:21 |
| avass | corvus, mordred: actually the manifests are there, but only reachable with the digest and not the tag | 21:22 |
| avass | I wonder if that could be a problem | 21:23 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 21:30 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "gate pipeline" https://review.opendev.org/732069 | 21:31 |
| avass | that last patch-set should work, but I don't understand why the mirror in registries.conf doesn't work | 21:31 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "job secrets" https://review.opendev.org/732070 | 21:39 |
| clarkb | mhu: can you see my comment on https://review.opendev.org/#/c/728118/ I'm trying to understand the change and some of the intent there may help | 21:40 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Test multiarch release builds https://review.opendev.org/737315 | 21:45 |
| *** Goneri has quit IRC | 21:46 | |
| *** hasharAway has quit IRC | 22:11 | |
| openstackgerrit | Merged zuul/zuul master: Add openafs-krb5 to bindep https://review.opendev.org/735739 | 22:15 |
| *** armstrongs has joined #zuul | 22:30 | |
| openstackgerrit | Merged zuul/zuul master: pagure: ensure files is list and not a dict_keys https://review.opendev.org/732171 | 22:32 |
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 22:32 | |
| *** armstrongs has quit IRC | 22:37 | |
| *** tosky has quit IRC | 22:42 | |
| *** clarkb has quit IRC | 22:44 | |
| mnaser | https://zuul.opendev.org/t/vexxhost/build/eeab2f7e09c745fea5ddfd4697fd2a56 -- hmm, we're using a network module _on the target system_ and getting "Use of network modules is prohibited" -- is that a bug or expected? (i would understand if we disallow it on 'localhost' aka executor but running it on the remote hosts seems.. reasonable?) | 22:57 |
| mnaser | it's not any different than running a curl or something | 22:57 |
| *** clarkb has joined #zuul | 22:58 | |
| mnaser | looks like this is implemented as an action plugin, so it might be hard to workaround :X | 22:58 |
| *** jamesmcarthur has joined #zuul | 23:04 | |
| *** jamesmcarthur has quit IRC | 23:07 | |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: tutorial: Add "job dependencies" https://review.opendev.org/732071 | 23:07 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Rename quick-start to zuul-tutorial-quick-start https://review.opendev.org/737656 | 23:07 |
| *** y2kenny has quit IRC | 23:20 | |
| fungi | mnaser: you'd need nested ansible on the remote node calling that module instead | 23:27 |
| mnaser | darn, i was hoping to just natively have zuul run ansible and get my testing that way :( | 23:28 |
| fungi | or we'd have to whitelist that plugin (after auditing it to make sure it can't be asked to make unsafe changes to the workspace on the executor) | 23:28 |
| *** kgz has quit IRC | 23:29 | |
| *** kgz has joined #zuul | 23:34 | |
| mordred | mnaser: we've had conversations before about not blocking plugins any more and relying on bubblewrap - I believe there are still a few unresolved issues to overcome, although tls zookeeper is an important one | 23:36 |
| mnaser | fungi: i guess my issue is why i can't run it even on the remote host | 23:36 |
| mnaser | i understand blocking it on localhost, but on the remote host seem a bit like just blocking a thing you can easily workaround | 23:37 |
| mordred | mnaser: yeah - it's because it's implemented as an action plugin, so the action plugin code runs on the executor even if the target of the action is a remote host | 23:37 |
| fungi | mnaser: ansible isn't running on the remote host, ansible is running on the executor, and that's python code basically being imported on the executor | 23:37 |
| fungi | even if it's acting on the remote node | 23:38 |
| mnaser | right, sorry, i mean the ansible module executes remotely | 23:38 |
| mnaser | if target_host == 'localhost' then stop else go | 23:38 |
| mordred | yeah - we just need code in its action plugin to do that | 23:39 |
| fungi | that's not always sufficient across the board, it's something which is going to be unique to each plugin | 23:39 |
| mordred | yeah - and I think so far nobody has asked for any of the network plugins, so nobody has looked to see if they need anything special or whatnot | 23:39 |
| fungi | it's certainly possible for a plugin which is targeting a remote node to still have local side effects, depending on how it's written | 23:39 |
| mordred | I'd happily +2 a patch to allow modules we're blocking if they're safe | 23:40 |
| mordred | mnaser: what's the module? | 23:41 |
| mnaser | mordred: net_ping | 23:41 |
| mnaser | or something along those lines | 23:41 |
| * mnaser finds change | 23:41 | |
| mnaser | mordred: https://review.opendev.org/#/c/737611/3/tests/test.yml | 23:42 |
| mordred | mnaser: jeez. I have almost no idea what ./lib/ansible/plugins/action/net_base.py does | 23:44 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!