| *** noonedeadpunk has quit IRC | 00:08 | |
| *** noonedeadpunk has joined #zuul | 00:10 | |
| *** hamalq_ has joined #zuul | 00:15 | |
| *** tosky has quit IRC | 00:18 | |
| *** hamalq has quit IRC | 00:19 | |
| *** hamalq_ has quit IRC | 00:21 | |
| *** hamalq has joined #zuul | 00:25 | |
| *** rfolco has joined #zuul | 00:43 | |
| *** rfolco has quit IRC | 00:47 | |
| *** sanjayu_ has joined #zuul | 02:09 | |
| *** hamalq has quit IRC | 02:32 | |
| *** sanjayu_ has quit IRC | 02:49 | |
| *** bhavikdbavishi has joined #zuul | 03:09 | |
| *** bhavikdbavishi1 has joined #zuul | 03:12 | |
| *** bhavikdbavishi has quit IRC | 03:13 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:13 | |
| *** manoj_kumar_kata has quit IRC | 04:23 | |
| *** manoj_kumar_kata has joined #zuul | 04:31 | |
| *** zenkuro has quit IRC | 04:38 | |
| *** rishabhhpe has joined #zuul | 04:45 | |
| *** rishabhhpe has quit IRC | 04:46 | |
| *** rishabhhpe has joined #zuul | 04:49 | |
| *** bhavikdbavishi has quit IRC | 05:02 | |
| *** ianw is now known as ianw_pto | 05:24 | |
| *** bhavikdbavishi has joined #zuul | 05:31 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #zuul | 05:33 | |
| *** zenkuro has joined #zuul | 05:34 | |
| *** bhavikdbavishi has quit IRC | 05:35 | |
| *** bhavikdbavishi has joined #zuul | 05:36 | |
| *** jfoufas1 has joined #zuul | 05:47 | |
| *** saneax has joined #zuul | 05:57 | |
| *** reiterative has quit IRC | 06:01 | |
| *** reiterative has joined #zuul | 06:01 | |
| *** ajitha has joined #zuul | 06:29 | |
| *** bhavikdbavishi has quit IRC | 06:41 | |
| *** bhavikdbavishi has joined #zuul | 06:42 | |
| *** bhavikdbavishi has quit IRC | 06:48 | |
| *** mach1na has joined #zuul | 07:09 | |
| *** bhavikdbavishi has joined #zuul | 07:21 | |
| *** bhavikdbavishi1 has joined #zuul | 07:26 | |
| *** rpittau|afk is now known as rpittau | 07:27 | |
| *** bhavikdbavishi has quit IRC | 07:28 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 07:28 | |
| *** mach1na has quit IRC | 07:28 | |
| *** mach1na has joined #zuul | 07:30 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Use logical timestamp to detect outdated changes https://review.opendev.org/c/zuul/zuul/+/763755 | 07:34 |
|---|---|---|
| *** jcapitao has joined #zuul | 07:48 | |
| *** jpena|off is now known as jpena | 07:52 | |
| *** sugaar has joined #zuul | 08:14 | |
| *** rishabhhpe has quit IRC | 08:31 | |
| *** tosky has joined #zuul | 08:34 | |
| *** rishabhhpe has joined #zuul | 08:43 | |
| *** vishalmanchanda has joined #zuul | 08:47 | |
| *** hashar has joined #zuul | 08:54 | |
| *** mach1na has quit IRC | 08:57 | |
| *** msuszko has joined #zuul | 09:07 | |
| *** piotrowskim has joined #zuul | 09:20 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 09:22 | |
| *** nils has joined #zuul | 09:30 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Required SQL reporters https://review.opendev.org/c/zuul/zuul/+/630472 | 09:34 |
| *** rishabhhpe has quit IRC | 09:34 | |
| *** manoj_kumar_kata has quit IRC | 09:41 | |
| *** holser_ has quit IRC | 09:45 | |
| *** holser has joined #zuul | 09:47 | |
| *** rishabhhpe has joined #zuul | 09:56 | |
| *** rishabhhpe has quit IRC | 09:59 | |
| msuszko | tobiash: Asynchronous reporting solved my trigger queuing issues. Thank you! | 10:00 |
| tobiash | msuszko: cool :) | 10:00 |
| tobiash | msuszko: did you also import the reporter metrics patch? | 10:01 |
| tobiash | in some cases this could indicate an actual problem with the services zuul is reporting to | 10:01 |
| *** mach1na has joined #zuul | 10:02 | |
| msuszko | there is merge conflict in zuul/manager/__init__.py, and I haven't spent time to find where to measure since reporter.report(item) is gone | 10:06 |
| tobiash | ah ok | 10:07 |
| msuszko | profilinh zuul-scheduler reveavle chardet using 38% time, so I monkey-patched requests to use cchardet instead | 10:08 |
| msuszko | s/profilinh/profiling/ | 10:08 |
| tobiash | did that help? | 10:08 |
| msuszko | it helped, but there is obviously something else. Now zuul-scheduler uses few % of CPU time tops, while before it was CPU bound | 10:11 |
| msuszko | chardet suggests extensive requests usage, is could be gerrit driver | 10:15 |
| *** mach1na has quit IRC | 10:25 | |
| *** rishabhhpe has joined #zuul | 10:27 | |
| *** mach1na has joined #zuul | 10:29 | |
| tobiash | msuszko: the git-upload-pack call done as part of reporting can get huge on large repos with gerrit which could explain both the high usage of chardet and slow reporting to gerrit | 10:46 |
| *** rfolco has joined #zuul | 10:59 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: encrypt: add --public-key argument https://review.opendev.org/c/zuul/zuul-client/+/765313 | 11:15 |
| avass | would it be possible to extend jobs.requires/jobs.provides to also work for the same change? | 11:33 |
| *** sanjayu_ has joined #zuul | 11:41 | |
| openstackgerrit | Merged zuul/zuul-client master: encrypt: fix bad indentation https://review.opendev.org/c/zuul/zuul-client/+/765112 | 11:43 |
| zbr | avass: what do you mean by that? | 11:43 |
| *** saneax has quit IRC | 11:43 | |
| *** rishabhhpe has quit IRC | 11:50 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: tools: Remove encrypt_secret.py, document zuul-client encrypt https://review.opendev.org/c/zuul/zuul/+/765316 | 11:53 |
| *** jcapitao has quit IRC | 11:54 | |
| mhu | tobiash, can we get W+1 on the zuul-client encrypt tests as well: https://review.opendev.org/c/zuul/zuul/+/754103 | 11:54 |
| *** mach1na has quit IRC | 11:58 | |
| avass | zbr: have jobs depend of eachother and pass artifacts between them without using job.depdencies and be able to run B without it being depdend on A running | 12:03 |
| avass | say it should wait for A if it runs but otherwise run without the dependency | 12:03 |
| avass | and pass zuul.artifacts between jobs in the same change, I believe it only works for cross change/project dependencies | 12:04 |
| zbr | like triggering the same job for two reasons, a change, or another job that finished, but not both. | 12:05 |
| avass | yeah I guess so but it would be the same job the only thing that differs if it should wait for a dependency or not | 12:06 |
| avass | and if the dependency isn't run for the current change it would still pick up artifacts for change ahead of queue | 12:06 |
| avass | but it might be better to just rebuild for the current change at that point | 12:07 |
| avass | I think I need to write this down so I don't get idea mixed up :) | 12:09 |
| avass | ideas* | 12:09 |
| avass | what I want however is that same change dependencies between jobs should be able to use the same logic as cross change dependencies | 12:10 |
| avass | something like this if that makes sense https://etherpad.opendev.org/p/GiqDGxULdN8G0FwCah6J | 12:20 |
| avass | if zuul-cache also reports artifacts to zuul and allows configurable metadata you can easily gate all depdencies without rebuilding everything all the time | 12:24 |
| tobiash | avass: you can use soft dependencies to wait if the parent runs | 12:28 |
| avass | tobiash: yeah but that's extra syntax :) | 12:28 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 12:29 |
| avass | tobiash: also the docs says queue items ahead of the current change for requires: https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.requires | 12:30 |
| *** bhavikdbavishi has quit IRC | 12:30 | |
| avass | so that's extra logic for handling same change dependencies | 12:30 |
| avass | I think the dependency passing between jobs could be simplified a lot that way | 12:32 |
| tobiash | provides/requires is not designed to model dependencies within a buildset | 12:32 |
| avass | I take it as it won't be trivial to allow for that behaviour then | 12:34 |
| *** jpena is now known as jpena|lunch | 12:35 | |
| tobiash | avass: is a missing ssh key a condition that mandates a warning? (re https://review.opendev.org/c/zuul/zuul/+/764584/3/zuul/executor/server.py) | 12:43 |
| zbr | i would have used .debug level instead | 12:44 |
| avass | tobiash: executor default ssh_key config always points to a path and I expect most zuuls to use ssh so that's why I put it there | 12:44 |
| tobiash | ah so that's for a k8s only executor | 12:45 |
| avass | yeah | 12:45 |
| tobiash | I wonder if it makes more sense to check for that in the initialization of the executor (you can emit a warning there I guess) and set self.private_key_file to None if the file doesn't exist and then don't add the key in AnsibleJob.execute? | 12:46 |
| avass | makes sense to me | 12:47 |
| avass | though technically that can change during runtime | 12:47 |
| tobiash | for that use case it feels a bit wrong to me to catch that in the ssh agent layer | 12:47 |
| tobiash | hm | 12:47 |
| avass | but I guess it's fine to do it during startup | 12:47 |
| tobiash | at least we should avoid printing warnings for every job in e legit use case | 12:48 |
| avass | yup | 12:48 |
| avass | and I don't expect people to move around ssh-keys in a deployed system | 12:48 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 12:52 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Throw a warning if the executors ssh-key can't be loaded https://review.opendev.org/c/zuul/zuul/+/764584 | 12:54 |
| avass | I'm gonna make sure that works as well | 12:54 |
| *** rlandy has joined #zuul | 12:59 | |
| *** mach1na has joined #zuul | 13:00 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 13:00 |
| *** wuchunyang has joined #zuul | 13:07 | |
| *** wuchunyang has quit IRC | 13:12 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 13:13 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 13:17 |
| *** msuszko has quit IRC | 13:23 | |
| *** mugsie has quit IRC | 13:32 | |
| *** jpena|lunch is now known as jpena | 13:32 | |
| openstackgerrit | Merged zuul/zuul master: Test zuul-client encrypt subcommand https://review.opendev.org/c/zuul/zuul/+/754103 | 13:37 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 13:44 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 13:49 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 13:59 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 14:14 |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 14:17 | |
| *** Goneri has joined #zuul | 14:20 | |
| pabelanger | man, that took way too long | 14:20 |
| avass | pabelanger: oh I've seen worse ;) | 14:25 |
| mordred | pabelanger: good call - I remember us updating that repo somewhere else - possibly in system-config | 14:28 |
| mordred | pabelanger: https://github.com/containers/podman/issues/5764 mwahahaha seems to have hit the error in your latest failure before | 14:31 |
| avass | mordred: I thought you were laughing at first hah | 14:32 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Throw a warning if the executors ssh-key can't be loaded https://review.opendev.org/c/zuul/zuul/+/764584 | 14:48 |
| avass | that should work | 14:48 |
| mordred | avass: it's a great name isn't it? | 14:58 |
| avass | mordred: haha yeah I like it | 15:02 |
| mordred | still doesn't beat mmmpork - which is my favorite nick | 15:03 |
| fungi | i miss mmmpork | 15:03 |
| *** jfoufas1 has quit IRC | 15:07 | |
| pabelanger | new issue, v1 / v2 syntax for podman in use-buildset-registry | 15:10 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 15:10 |
| *** sanjayu_ has quit IRC | 15:19 | |
| *** hashar is now known as hasharAway | 15:22 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 15:27 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 15:29 |
| corvus | tobiash, bolg: comments on https://review.opendev.org/630472 | 15:45 |
| tobiash | looking | 15:45 |
| corvus | ftr, i have deleted the stable/3.x branch (again) since the releasenotes changes merged | 15:46 |
| tobiash | corvus: ++ for the config change suggestion | 15:47 |
| tobiash | should we add that to this change or a follow-up? | 15:47 |
| corvus | tobiash: either way -- i do think we should do the addition (but not removal) as 4.0. i don't think it will be too big, probably just a few lines (and maybe one more config file syntax test case) | 15:48 |
| tobiash | I agree | 15:48 |
| tobiash | I'll take care of that tomorrow | 15:48 |
| corvus | tobiash: i don't really anticipate folks wanting >1 sql connection for a zuul instance, but if someone did, sql-connection-per-tenant would certainly be feasible; should we ask around and see if folks have that need? | 15:49 |
| corvus | mordred: you may have thoughts on that (eg, sharding per tenant?) first comment in https://review.opendev.org/630472 | 15:50 |
| corvus | tobiash: maybe i should send a -discuss email? | 15:50 |
| tobiash | corvus: good question, I guess a mail to the list won't harm | 15:50 |
| corvus | tobiash: okay, i'll do that now | 15:51 |
| tobiash | thanks! | 15:51 |
| avass | corvus, tobiash: I think having an sql-connection-per-tenant as an option would be a good idea | 15:51 |
| corvus | avass: great, why? | 15:51 |
| avass | not sure if it's entirely needed but as an extra secury measurement since zuul-web has access to sql right? | 15:52 |
| pabelanger | do we have docs on how to run zuul-jobs role libraries locally? | 15:52 |
| mordred | colooking | 15:52 |
| pabelanger | eg: test_golangci_lint_parse_output.py | 15:52 |
| corvus | pabelanger: "tox" | 15:53 |
| avass | so a white labeled tenant could use a separate database to make sure data doesn't get leaked | 15:53 |
| avass | I've been wondering if that's really needed though since I think we're gonna need to set up a white labeled tenant in the future | 15:53 |
| tristanC | pabelanger: there is also a spec to run zuul-jobs locally here: https://review.opendev.org/c/zuul/zuul/+/681277 | 15:54 |
| corvus | avass: hrm. i feel like if zuul supports tenant separation (it does!) then it shouldn't need to recommend that level of paranoia to users. however, i'll never argue against users who want that level of paranoia, but i might suggest that if that's the level of paranoia they're comfortable with, then perhaps a completely separate zuul is more appropriate? | 15:55 |
| avass | hmm that could be | 15:55 |
| pabelanger | corvus: tristanC: thanks, I should have figured tox was the answer | 15:55 |
| mordred | yeah. otoh - sharding per tenant for scale might be a thing? but also maybe that's a case for multi-zuul instead too | 15:55 |
| avass | that would be a very large zuul though? :) | 15:56 |
| tobiash | I think the paranoia use case should be separate deployments since there are other theoretical cross tenant attac vectors as well | 15:56 |
| corvus | tobiash: let's say, for now, "don't implement the [database]" change as a new patchset. if we do it, let's do it as a followup, but let's see what people think tomorrow. :) | 15:56 |
| tobiash | k | 15:56 |
| tobiash | I think the scalability argument is more compelling since we've already seen db bottlenecks | 15:57 |
| corvus | tobiash: oh do tell | 15:58 |
| tobiash | btw, we needed to add an additional index because our db broke down without it: https://review.opendev.org/c/zuul/zuul/+/758579 | 15:58 |
| corvus | (we have done very little db optimization, so i'm sure we can make improvements) | 15:58 |
| corvus | yep like that :) | 15:58 |
| tobiash | so atm it looks ok, but our db queries generally feel slower than needed so I can imagine that there is more optimization work to be done in the future | 15:59 |
| corvus | my gut says that if we actually start paying attention to db efficiency, we won't need to shard by tenant | 15:59 |
| tobiash | probably not | 15:59 |
| mordred | agree. but - sharding by tenant would be a clean place to do it if we did decide that was useful | 15:59 |
| tobiash | also I think we might need db retention mechanisms (e.g. as a ops tool) to keep the db smaller and faster | 16:01 |
| corvus | i will admit i'm very torn on this. i like the [database] idea for cleanliness, but i do worry just a little bit about backing ourselves into a corner in the future in case sharding becomes desirable | 16:01 |
| corvus | tobiash: well, i think a simple "max-age" for builds in the .conf file would let zuul do that automatically (ie, i don't think we need a new cli tool or command or anything) | 16:01 |
| corvus | i do agree we need to delete builds :) | 16:02 |
| tobiash | that's also an option | 16:02 |
| corvus | or at least have the option to delete builds | 16:02 |
| mordred | corvus: you don't want to roll old builds into a historical data warehouse? | 16:04 |
| *** rishabhhpe has joined #zuul | 16:05 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Required SQL reporters https://review.opendev.org/c/zuul/zuul/+/630472 | 16:06 |
| *** bhavikdbavishi has joined #zuul | 16:07 | |
| avass | tristanC, pabelanger: I've also had a crazy idea of a minizuul (like minikube) but I haven't made that a reality yet :) | 16:09 |
| avass | though if the zuul-runner works good enough it shouldn't be needed | 16:10 |
| *** hasharAway is now known as hashar | 16:11 | |
| *** bhavikdbavishi1 has joined #zuul | 16:12 | |
| avass | oooh or something like KIND but ZINC (zuul in containers) ! | 16:12 |
| *** bhavikdbavishi has quit IRC | 16:13 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 16:13 | |
| fungi | opendev already runs zuul in containers | 16:13 |
| fungi | as does our quickstart demo | 16:14 |
| fungi | unless i'm misunderstanding the deployment model you're attempting to describe | 16:14 |
| mordred | yeah - docker-compose up ftw | 16:15 |
| avass | I think just want a dumb acronym for it | 16:16 |
| corvus | never underestimate the power of a name | 16:16 |
| fungi | indeed! | 16:16 |
| avass | though the docker-compose up doesn give you a binary to easily start multiple zuul environments locally :) | 16:16 |
| *** piotrowskim has quit IRC | 16:19 | |
| avass | and the idea was more of 'please run this job for me. here is a config file with all the repos you need' | 16:23 |
| avass | in a real zuul environment | 16:23 |
| fungi | from what we've seen, working out which are "all the repos you need" is the hard part, without another running zuul to consult | 16:26 |
| avass | well there's already the tenant config file | 16:27 |
| avass | and 'hard' isn't the same as 'impossible' | 16:28 |
| fungi | sure, just noting that the tenant config can be huge and the subset of actual repos you need to run a given job might be very small by comparison | 16:28 |
| *** mach1na has quit IRC | 16:29 | |
| *** rpittau is now known as rpittau|afk | 16:49 | |
| *** jpena is now known as jpena|off | 16:54 | |
| *** zenkuro has quit IRC | 16:56 | |
| *** zenkuro has joined #zuul | 16:57 | |
| tristanC | avass: zuul-runner seems to address your use-case, the initial implementation already took care of cloning all the repos and setting up the ansible environment | 17:02 |
| corvus | tobiash: ml message sent; i also added another thought to 630472; i don't think it's actionable until we agree on [database] or not | 17:09 |
| avass | tristanC: yeah at this point I'm pretty much doing it for the challenge of it | 17:09 |
| tristanC | avass: another solution we are considering is converting the zuul-operator resources to a simpler ansible playbook to deploy the service on a podman pod locally | 17:10 |
| tristanC | by converting i mean a function that goes from zuul-crd -> ansible-tasks | 17:11 |
| tristanC | like that would get the zuul-registry or zuul-preview service if needed | 17:11 |
| erbarr | clarkb, hi I applied the change to openstack-ironic-ci and restarted zuul but now when I ran the zuul enqueue the jobs pop up on zuul web but then go away very quickly, any pointers as to how to find what's going on? | 17:12 |
| avass | that woiuld be nice | 17:12 |
| avass | though we already have kind + helm charts | 17:12 |
| corvus | tristanC: i'm about to send out a 'last call for review' for the zuul-runner spec, sound good? | 17:13 |
| tristanC | corvus: yes please, and i would like to know where to rebase the changes too | 17:13 |
| corvus | tristanC: maybe set dec 11 as a deadline? | 17:13 |
| tristanC | avass: but does kind works on podman? | 17:13 |
| clarkb | erbarr: can you be more specific about which change you applied and to whcih zuul version? | 17:13 |
| corvus | erbarr: in general you may be able to find more information in the scheduler log | 17:14 |
| avass | tristanC: I think it's supposed to | 17:14 |
| avass | tristanC: at least it had expiremental support in 0.7 or something | 17:15 |
| erbarr | http://paste.openstack.org/show/c1QUUjrT2AX90PiP33Rm/ to zuul 2.6.1.dev1 | 17:15 |
| erbarr | clarkb ^ | 17:15 |
| *** rishabhhpe has quit IRC | 17:16 | |
| *** rishabhhpe has joined #zuul | 17:17 | |
| avass | tristanC: according to the roadmap podman is supposed to be supported: https://kind.sigs.k8s.io/docs/contributing/1.0-roadmap | 17:17 |
| clarkb | erbarr: as mentioend in my email I have no way of testing it and that was a best effort fix for those running old zuul. The other Dell CI systems report it is working for them. I wonder if this is specific to enqueue | 17:17 |
| erbarr | clarkb, ok, let me do a recheck | 17:17 |
| tristanC | avass: i remember reading about cgroupv2 issues or something, i'm not familiar with the tool and it might be easier to simply run podman command | 17:18 |
| tristanC | avass: the trick with such solution is to convert the config-map to ansible copy task | 17:18 |
| tristanC | corvus: thanks! | 17:19 |
| erbarr | clarkb, same thing, let me dig around logs, thanks! | 17:20 |
| pabelanger | okay, struggling a little. Do we have an unit test example of comparing OrderedDict() ? assertEqual fails, due to sort order | 17:22 |
| mordred | pabelanger: assertDictEquals I think | 17:23 |
| pabelanger | Hmm | 17:29 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Add unit test for modify_registries_conf https://review.opendev.org/c/zuul/zuul-jobs/+/765352 | 17:29 |
| pabelanger | okay, will need to take a breat | 17:29 |
| pabelanger | break* | 17:29 |
| pabelanger | but ^ is racy | 17:29 |
| *** tosky has quit IRC | 17:29 | |
| avass | tristanC: i don't know about podman but with it's very easy to use as a locak k8s cluster with docker at least | 17:30 |
| tristanC | avass: yep, but if it's possible to translate the zuul-operator to ansible task, for example using podman generate systemd service and config-map in /etc, then it might be a better fit for non k8s user | 17:32 |
| avass | tristanC: yeah absolutely | 17:42 |
| *** rishabhhpe has quit IRC | 17:45 | |
| *** sean-k-mooney has joined #zuul | 17:57 | |
| sean-k-mooney | o/ just thinking out loud here how hard woudl it be to add a link to the build page for the current job in the inital comment set by zuul | 17:58 |
| sean-k-mooney | basically we can leave a comment sayign we are starting the job like this https://github.com/SeanMooney/ci-sean-mooney/blob/main/zuul.d/pipelines.yaml#L27-L29 | 17:59 |
| sean-k-mooney | there is proably a start-message i can override | 17:59 |
| sean-k-mooney | but im wondering if we coud dynamically generate the build url and provide that so that third party CIs coudl better integrate with gerrit | 18:00 |
| sean-k-mooney | or other providers | 18:00 |
| sean-k-mooney | yoctozepto: basically im thinink that woudl provide something for you sript to read | 18:01 |
| tristanC | sean-k-mooney: it may be possible to post the link, but the build is only reported once completed, thus the page will returns 404 | 18:01 |
| sean-k-mooney | hum ok | 18:02 |
| sean-k-mooney | you can see the jobs on the status page | 18:02 |
| corvus | tristanC, sean-k-mooney: in fact zuul doesn't know if the buildset at start will be the buildset at finish, so we can't report the buildset url | 18:02 |
| sean-k-mooney | how is that workign so | 18:02 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Add unit test for modify_registries_conf https://review.opendev.org/c/zuul/zuul-jobs/+/765352 | 18:03 |
| corvus | sean-k-mooney: what is possible is the change status link, eg: https://zuul.opendev.org/t/openstack/status/change/763718,1 | 18:03 |
| sean-k-mooney | oh yes that is what i was really looking for i guess | 18:03 |
| sean-k-mooney | or rather the api behind it | 18:03 |
| corvus | the last time someone tried that on opendev's zuul, it killed zuul, so i'd really appreciate it if people didn't put that in greasemonkey scripts. | 18:03 |
| corvus | let's please leave stuff like that to something that is controlled server-side, ie, a polygerrit plugin | 18:04 |
| sean-k-mooney | basically i was wondering if there was a way to tell gerrit a third part ci is running on this | 18:04 |
| sean-k-mooney | corvus: well ya i dont really want to do anything client side | 18:04 |
| sean-k-mooney | our side of the check api or continueing to parse the comment i was ondering if there was a simple midell ground | 18:05 |
| openstackgerrit | Merged zuul/zuul-operator master: Update to dhall lang v17 https://review.opendev.org/c/zuul/zuul-operator/+/739767 | 18:05 |
| corvus | i think there's a lot that can be done with a polygerrit plugin | 18:05 |
| sean-k-mooney | what i was really think was a pending or running badge/box with a link to the third party system that you could click on | 18:06 |
| sean-k-mooney | not something rendered on the page and updated | 18:06 |
| sean-k-mooney | perferably form either an api call by the ci to gerrrit or a coment which is more or less the same thing but less clean | 18:07 |
| fungi | that's also what the zuul-status plugin for gerrit theoretically does, right? | 18:07 |
| sean-k-mooney | in theoy i think so | 18:07 |
| sean-k-mooney | i saw that in teh plugin list and ment to ask | 18:07 |
| sean-k-mooney | are either of those plugins associated with upstream zuul | 18:08 |
| sean-k-mooney | or are they independt things people made | 18:08 |
| *** bhavikdbavishi has quit IRC | 18:08 | |
| fungi | the main problems we ran into were cumulative over time, seemed related to folks leaving lots of gerrit changes open in browser tabs, with every one of those periodically querying the api | 18:08 |
| fungi | if you have a few hundreds reviewers with 10 open tabs each, that's a lot of additional api traffic | 18:09 |
| sean-k-mooney | yep | 18:09 |
| sean-k-mooney | really you want this to be semi staic | 18:09 |
| sean-k-mooney | e.g. rednered once on page load and not upsated unless you refersh | 18:10 |
| sean-k-mooney | there are 2 zuul plugins for gerrit https://gerrit.googlesource.com/plugins/zuul and https://gerrit.googlesource.com/plugins/zuul-status/ | 18:10 |
| tristanC | fungi: i think zuul-web ui detects if the tab is visible before refreshing the status | 18:10 |
| *** bhavikdbavishi has joined #zuul | 18:10 | |
| fungi | granted, if memory serves at the time the rest api handler was not separated into its own daemon so was creating additional thread contention for the scheduler daemon. it may not be as bad as it was before, but we (opendev) sysadmins are going to want to perform some controlled testing to make sure it doesn't introduce problems | 18:11 |
| sean-k-mooney | tristanC: maybe but it did not always | 18:11 |
| clarkb | also it was hacky js on old gerrit | 18:11 |
| corvus | tristanC: right, i think fungi is recalling the time when we hade hideci querying the status api and putting that on the gerrit page | 18:11 |
| fungi | sean-k-mooney: we weren't using that plugin before | 18:11 |
| fungi | corvus: yes exactly | 18:11 |
| sean-k-mooney | fungi: we were not using either we had our own script right | 18:12 |
| sean-k-mooney | im just wondering if either of those actully work and or are usfeful | 18:12 |
| fungi | it was part of the javascript overlay we added to our gerrit webui before | 18:12 |
| sean-k-mooney | unfortuetly they dont seam to have docs or iamges of the plugsin in action | 18:12 |
| sean-k-mooney | apparently it looks like this https://gerrit.wikimedia.org/r/plugins/zuul/Documentation/index.html | 18:14 |
| sean-k-mooney | but i think they are both java based | 18:15 |
| sean-k-mooney | tristanC: ok ill go back to ingnoring this for now. long term the check plugin is proably what we shoudl just do | 18:16 |
| corvus | sean-k-mooney: i'm not sure the check plugin is a long-term answer | 18:16 |
| corvus | given that it's unclear if it will continue to be developed | 18:16 |
| corvus | er, 'checks' plugin | 18:16 |
| sean-k-mooney | oh i tought it was well maintianed | 18:16 |
| sean-k-mooney | experimental but maintianed | 18:17 |
| sean-k-mooney | ya that woudl be an issue | 18:17 |
| corvus | the proposal on the table is to replace it with a completely different system and google would not maintain checks (community might, but community contribs to checks plugin are thin) | 18:17 |
| *** bhavikdbavishi has quit IRC | 18:17 | |
| corvus | so it's possible, but i'm not comfortable pinning hopes on it | 18:18 |
| sean-k-mooney | ok so google are working on a replacment? | 18:18 |
| corvus | sean-k-mooney: yes, a replacement that would be completely unsuitable for the third-party ci use case | 18:18 |
| sean-k-mooney | even if the third party ci was to call the new api | 18:18 |
| corvus | sean-k-mooney: (the replacement proposal requires a publically accessible api server; zuul could implement it, but i wouldn't expect more than 10% of opendev's third-party ci systems to be able to meet that bar) | 18:19 |
| sean-k-mooney | for gerrit to call back too? | 18:19 |
| corvus | sean-k-mooney: yes or user's browser; it's unclear to me which | 18:20 |
| corvus | sean-k-mooney: but key thing is gerrit is not a data store. unless we want to implement another plugin to make gerrit a data store. | 18:20 |
| corvus | sean-k-mooney: but that would be up to us | 18:20 |
| sean-k-mooney | you might be able to run a shim application that parsed the comment and pressent the right api | 18:20 |
| sean-k-mooney | ah right | 18:20 |
| corvus | sean-k-mooney: better to just implement the polygerrit plugin we keep suggesting | 18:20 |
| sean-k-mooney | yep | 18:20 |
| sean-k-mooney | tristanC: i havent looked at your since you fix the darkmode and third party ci parsing | 18:21 |
| sean-k-mooney | are you still working on that | 18:21 |
| corvus | sean-k-mooney: it may well be that a long-term solution might be to use checks plugin for 3pci (if checks continues to be maintained), and the new thing for 1pci. but that would be far in the future; i think the best path now would be a polygerrit plugin that deals with the comments, makes a tab/table and go with that until we see what comes out of gerrit land. | 18:22 |
| tristanC | sean-k-mooney: you can see it in action here: https://104.130.172.52/c/openstack/diskimage-builder/+/554002 | 18:22 |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 18:22 | |
| fungi | sean-k-mooney: ianw has a slightly different approach in a plugin he started which it seemed like folks are converging on | 18:22 |
| sean-k-mooney | oh i see RDO ci :) | 18:22 |
| sean-k-mooney | fungi: the badges approch? i have not seen taht yet | 18:23 |
| tristanC | sean-k-mooney: you can create a tp account (ending in 'CI') and post a comment to see it updated | 18:23 |
| corvus | fungi, sean-k-mooney, tristanC: https://github.com/ianw/gerrit-zuul-summary-status i think? | 18:24 |
| sean-k-mooney | fungi: provide it has the smae basic info with the links, status, jobname and times group by pipline/ci that woudl be cool | 18:24 |
| corvus | that plugin uses the upstream gerrit tooling; it could be hosted in the upstream gerrit and potentially attract more contributors | 18:25 |
| fungi | i was mostly just trying to follow the discussion, but yeah the big plus is that it could be consistent with the existing gerrit plugin ecosystem | 18:26 |
| sean-k-mooney | ya is there a live instance of it. im about to grab dinner so dont really have time to deploy gerrit and figure out how to build it | 18:27 |
| corvus | sean-k-mooney: i don't know, but it's super easy to run gerrit in a throwaway docker container; should be able to bind-mount the result in | 18:28 |
| sean-k-mooney | ya my current side project is deploying k8s and getting ready to move my zuul install among other things so trying not to get too distracted with other thigns | 18:29 |
| sean-k-mooney | trying and failing :) | 18:29 |
| sean-k-mooney | but its time for dinner so i better go o/ | 18:30 |
| openstackgerrit | Merged zuul/zuul-operator master: Add reformat changes to the blame ignore list https://review.opendev.org/c/zuul/zuul-operator/+/739768 | 18:32 |
| *** hashar has quit IRC | 18:40 | |
| *** zenkuro has quit IRC | 18:57 | |
| *** hamalq has joined #zuul | 18:58 | |
| *** hashar has joined #zuul | 18:58 | |
| *** ashbullock has joined #zuul | 19:00 | |
| ashbullock | hey, I've managed to get an artifact set up using zuul return module which I can see in the logging server, but how do I pass this to child jobs, i've tried dependencies and provides/requires but I can't see the vars in the child job, is there something i could be missing? | 19:03 |
| *** nils has quit IRC | 19:05 | |
| corvus | ashbullock: artifacts aren't returned to dependent jobs; they're only shared between buildsets for different changes. you can return arbitrary data for use by child jobs like this: https://zuul-ci.org/docs/zuul/reference/jobs.html#return-values | 19:06 |
| ashbullock | corvus thanks, so the child job should have access to that dictionary under zuul vars? | 19:10 |
| corvus | ashbullock: not under 'zuul.' vars, but yes under vars supplied by zuul. if you say "zuul_return: {data: {my_artifact_url: http://...}}" in the first job, then the dependent job will have a variable ${my_artifact_url} available to it. | 19:13 |
| *** vishalmanchanda has quit IRC | 19:22 | |
| ashbullock | @corvus this is our zuul config http://paste.openstack.org/show/800718/ it's got dependencies on the base image, would you expect to see the vars in the child job with this config? | 19:26 |
| fungi | ashbullock: also not sure if it's apparent, but if you're collecting the inventory.yaml for each build you should see every zuul variable included in it, which is an easy way to double-check | 19:30 |
| ashbullock | also I wanted to catch up with you regarding testing for the bitbucketcloud driver, i'm struggling with setting up the tests, is it right to follow the base file here: https://opendev.org/zuul/zuul/src/branch/master/tests/base.py | 19:35 |
| ashbullock | also the unit tests, what's are mocks in the base.py the correct way to go | 19:38 |
| *** nils has joined #zuul | 19:45 | |
| clarkb | corvus: trying to catch up with zuulv4 things. Looks like the stack you pinged about earlier has landed. Are there other chagnes I should be reviewing? | 19:46 |
| corvus | clarkb: nope; just the message on the ml about sql; that change (and it's to-be-written followup) is next | 19:50 |
| *** sanjayu_ has joined #zuul | 19:51 | |
| corvus | ashbullock: yes, if the base-image-build job did a zuul_return as above, then terraform-image-build should see those vars | 19:53 |
| *** holser has quit IRC | 19:53 | |
| corvus | ashbullock: re bbc driver: yeah, following the pattern that the pagure driver does is the gold standard i think | 19:54 |
| *** sanjayu_ has quit IRC | 20:10 | |
| *** holser has joined #zuul | 20:28 | |
| *** nils has quit IRC | 20:28 | |
| *** sshnaidm|afk has quit IRC | 20:30 | |
| fungi | clarkb: oh, and the zuul-runner spec | 20:32 |
| clarkb | ah yup I see that email | 20:33 |
| fungi | (also mentioned on the ml) | 20:33 |
| *** nils has joined #zuul | 20:36 | |
| *** ashbullock has quit IRC | 20:44 | |
| *** tosky has joined #zuul | 20:55 | |
| *** openstackgerrit has quit IRC | 21:08 | |
| *** hashar has quit IRC | 21:16 | |
| *** rfolco has quit IRC | 21:34 | |
| *** ajitha has quit IRC | 21:46 | |
| *** rfolco has joined #zuul | 22:19 | |
| *** rfolco has quit IRC | 22:24 | |
| *** rfolco has joined #zuul | 22:42 | |
| *** rlandy has quit IRC | 23:04 | |
| *** nils has quit IRC | 23:13 | |
| *** rfolco has quit IRC | 23:17 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!