| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 813913: Serialize JobGraph objects to ZK https://review.opendev.org/c/zuul/zuul/+/813913 | 00:20 | |
| @goneri:matrix.org | Hi, I've messed up an upgrade and I end up with KeyError: 'build_set_uuid' https://paste.openstack.org/show/809993/ | 01:09 |
|---|---|---|
| @goneri:matrix.org | My understanding is that the scheduler cannot load the build requests from zk. Is there a way to flush them. | 01:10 |
| @clarkb:matrix.org | Gonéri https://zuul-ci.org/docs/zuul/reference/client.html#delete-state maybe? | 01:24 |
| @goneri:matrix.org | Thanks Clark. | 01:25 |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 813924: intermediate-registry: handle socat warning out https://review.opendev.org/c/zuul/zuul-jobs/+/813924 | 02:02 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 813924: intermediate-registry: handle socat warning out https://review.opendev.org/c/zuul/zuul-jobs/+/813924 | 04:27 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 813924: intermediate-registry: handle socat warning out https://review.opendev.org/c/zuul/zuul-jobs/+/813924 | 05:05 | |
| @ologinov:synapse.sardinasystems.com | > <@nborg:matrix.org> Oleg Loginov: I don't know the gitlab configuration. But I do know that we've had similar problems with reporting when using community edition gitlab while setting approval: true as you have. It is an premium or higher feature to be allowed to set the approval: https://docs.gitlab.com/ee/api/merge_request_approvals.html | 11:25 |
| > You could try setting approval: false and see if zuul is reporting back then. | ||
| I suppose that it doesn't matter at now. I noticed that Zuul can work with trigger correctly, but don't see that it tried try to add comment or approve MR(i mean GitLabReporter from Zuul).Although I sure that approve/comment feature work thorugh curl. | ||
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 813014: wip: only create build set when needed https://review.opendev.org/c/zuul/zuul/+/813014 | 12:52 | |
| -@gerrit:opendev.org- Simon Westphahl proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 813913: Serialize JobGraph objects to ZK https://review.opendev.org/c/zuul/zuul/+/813913 | 12:52 | |
| @westphahl:matrix.org | corvus: ^ fixed a small issue related to the change from dict to list for the jobs | 12:52 |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [zuul/zuul] 814002: Update dockerfile syntax line to recommended form https://review.opendev.org/c/zuul/zuul/+/814002 | 12:53 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 813014: wip: only create build set when needed https://review.opendev.org/c/zuul/zuul/+/813014 | 12:55 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 813014: wip: only create build set when needed https://review.opendev.org/c/zuul/zuul/+/813014 | 12:57 | |
| @westphahl:matrix.org | corv | 12:58 |
| @westphahl:matrix.org | * corvus: re 813014 I'd like to see if this makes a difference regarding test duration. seems we have to create a buildset for most of the items anyways at some point | 12:58 |
| @jim:acmegating.com | swesttobiash : opendev had a zk outage that caused zuul to get stuck and was restarted while i was asleep. i don't really have any debugging information to go on. but from the little i know, it's consistent with watches not being established. (but it's also consistent with threads getting stuck or killed). | 14:47 |
| @jim:acmegating.com | at the time of the outage, there were a lot of these https://paste.opendev.org/show/810000/ which is new to me. | 14:47 |
| @jim:acmegating.com | the only clue i have is that the run handler ran after the outage, but then never ran again (suggesting it wasn't getting woken up by events). | 14:48 |
| @westphahl:matrix.org | corvus: I can't remember seeing those exceptions on our side, but it definitely sounds like the watches were no longer setting the wake event. | 15:34 |
| -@gerrit:opendev.org- Zuul merged on behalf of Ian Wienand: [zuul/zuul-jobs] 813924: intermediate-registry: handle socat warning out https://review.opendev.org/c/zuul/zuul-jobs/+/813924 | 15:52 | |
| @ecsantos:matrix.org | hi guys | 17:30 |
| @ecsantos:matrix.org | I have a devstack job set up just as the upstream one [1] in my local Zuul instance, but I'm hitting that suds-jurko conflict issue [2]. why hasn't this issue affected the OpenDev devstack builds [3]? would like to know to be able to make my devstack go back to passing | 17:30 |
| [1] https://opendev.org/openstack/devstack/src/branch/master/.zuul.yaml#L438 | ||
| [2] https://bugs.launchpad.net/devstack/+bug/1946340 | ||
| [3] https://zuul.opendev.org/t/openstack/builds?job_name=devstack | ||
| @jim:acmegating.com | ecsantos: probably more of an #opendev or #openstack-infra question, but i think someone said opendev has old wheel builds in the caches that it's using, and the solution is probably going to be expiring those periodically. | 17:33 |
| @clarkb:matrix.org | yup and in the meantime openstack is dropping its use of suds-jurko to fix this | 17:35 |
| @jim:acmegating.com | yeah, sorry, solution to "why is opendev not matching reality" is clear the caches; solution to "suds-jurko is broken" is stop using suds-jurko. | 17:36 |
| @ecsantos:matrix.org | oh I see. thanks for the info! didn't know about the opendev channels, my bad | 17:41 |
| @clarkb:matrix.org | corvus: do you remember what that cert based artifact signing ssytem was that you had brought up in the past? I think it was you who mentioend it initially? | 17:41 |
| @clarkb:matrix.org | they integrated with github releases too I think | 17:41 |
| @jim:acmegating.com | Clark: hrm, that's not ringing a bell | 17:41 |
| @jim:acmegating.com | oooh | 17:42 |
| @jim:acmegating.com | i remember enough to try google, 1 sec | 17:42 |
| @clarkb:matrix.org | ok thanks, my googling has failed me | 17:42 |
| @jim:acmegating.com | Clark: https://github.com/merklecounty/rget | 17:51 |
| @clarkb:matrix.org | thanks also interesting they gave up on it | 17:51 |
| @jim:acmegating.com | see also https://github.com/transparencylog/tl and https://www.sigstore.dev/ | 17:52 |
| @clarkb:matrix.org | I was mostly curious in remembering how they were using public certs from LE but it seems that was the problem with the system unfortunately | 18:14 |
| @clarkb:matrix.org | A lot of the more modern stuff like sigstore and what docker do seem to rely on their own trust things and I was curious to see if they were still being usccessfully using the public trust | 18:15 |
| @jpew:matrix.org | We have a large set of things we are trying to build in different pipelines with slight variations; for example 1 job that we want to build with different arguments in 50 times in 3 different pipelines (so 150 total variations); I've been unable to do this in sane manner, other than having a script that expands all our job variations manually to create 150 jobs that each inherit from the same base job, just with different arguments.... this is a little unsatisfying, as it's hard to maintain a list of 50 jobs in 3 different pipelines.... is there a better way to do this? | 19:56 |
| @jpew:matrix.org | * We have a large set of things we are trying to build in different pipelines with slight variations; for example 1 job that we want to build with different arguments 50 times in 3 different pipelines (so 150 total variations); I've been unable to do this in sane manner, other than having a script that expands all our job variations manually to create 150 jobs that each inherit from the same base job, just with different arguments.... this is a little unsatisfying, as it's hard to maintain a list of 50 jobs in 3 different pipelines.... is there a better way to do this? | 19:57 |
| @clarkb:matrix.org | jpew: a project-template would help a bit. then you can manage the list in one place rather than 3 | 19:58 |
| @clarkb:matrix.org | but I think you still need to write out the combination at least once | 19:58 |
| @jim:acmegating.com | generally speaking, the tools that may be helpful that i can think of are: job inheritance, project-templates (as Clark said), yaml anchors, and a script. here's a script that's used in zuul-jobs to generate zuul.yaml files with jobs multiplied by operating system platform: https://opendev.org/zuul/zuul-jobs/src/branch/master/tools/update-test-platforms.py and one of the files it outputs: https://opendev.org/zuul/zuul-jobs/src/branch/master/zuul-tests.d/python-jobs.yaml | 20:01 |
| @jim:acmegating.com | (notably, it uses tags on the jobs to decide which ones it should expand; the first job there is hand-authored, the rest are auto-generated) | 20:02 |
| @jpew:matrix.org | Ah, I never thought to use job tags... that could work well | 20:03 |
| @jim:acmegating.com | also, there's a zuul job that runs that script to make sure the current repo content is what the script would generate | 20:04 |
| @jim:acmegating.com | (so it prevents manually mucking up the auto-generated output) | 20:04 |
| @jpew:matrix.org | Ya, I have a check job that makes sure the script output is up to date :) | 20:05 |
| @jim:acmegating.com | heh, i suspect anyone with a script gets about 3 changes in before they realize the necessity of that :) | 20:05 |
| @jim:acmegating.com | also some of the complexity of that script is that it rewrites the file, so the input and output files are the same. so it's doing some fancy yaml parsing with ruamel. | 20:06 |
| @clarkb:matrix.org | corvus: is the stack at https://review.opendev.org/c/zuul/zuul/+/809293/ still largely WIP? I'm trying to carve out some time for zuul sos reviews again and it seems that stack is the one next in line maybe? | 20:09 |
| @jim:acmegating.com | Clark: yes, i'm working on the tip of it today... i think we still have some major objects to do, and i don't think it's worth landing the early changes until the whole thing is ready. i've gone through and done really high level reviews but have not fussed too much on the details as things could still shift. | 20:11 |
| @clarkb:matrix.org | ok, I'll start looking at other zuul changes then. Looks like there are a few test improvements and the like | 20:11 |
| @clarkb:matrix.org | https://review.opendev.org/c/zuul/zuul/+/812410 and https://review.opendev.org/c/zuul/zuul/+/813255/ | 20:12 |
| @mhuin:matrix.org | @clark, if you got the bandwidth, the changes adding pagination to results in the GUI could use some love: https://review.opendev.org/c/zuul/zuul/+/793159 https://review.opendev.org/c/zuul/zuul/+/808041 | 20:15 |
| @clarkb:matrix.org | I'll see what I can do. The js stuff tends to require a lot more time from me because the js is so foreign and I feel apprehensive adding a +2 without a lot of double checking my understanding of stuff | 20:17 |
| @mhuin:matrix.org | oh I understand too well | 20:17 |
| @mhuin:matrix.org | the js stuff was absolutely alien to me until I put my nose into it | 20:18 |
| @dmsimard:matrix.org | I think the answer is no but making sure: is it possible to encrypt binary files or just strings/text files ? | 20:19 |
| @mhuin:matrix.org | there are 2 changes in the topic that are patching zuul-web only, if you prefer, but they're still going through the CI. They failed before on a timeout so I'm confident they're reviewable | 20:19 |
| @jim:acmegating.com | dmsimard: the answer is yes | 20:19 |
| @dmsimard:matrix.org | oh? color me pleasantly surprised | 20:20 |
| @jim:acmegating.com | dmsimard: but it's a bit hard to work with binary data in ansible so in practice i think we have b64 encoded binary stuff | 20:20 |
| @jim:acmegating.com | lemme dig a link | 20:20 |
| @dmsimard:matrix.org | a colleague is getting a stack trace when trying to encrypt a small .zip file | 20:21 |
| @mhuin:matrix.org | @Clark, actually, if you'd rather review python stuff, the first two changes in the admin-gui patch chain are on zuul-web: https://review.opendev.org/c/zuul/zuul/+/735586 https://review.opendev.org/c/zuul/zuul/+/736968 | 20:21 |
| @mhuin:matrix.org | if we get these merged it'll be easier to play with authentication using the docker compose in https://review.opendev.org/c/zuul/zuul/+/769943 - ie no need to build the zuul-web & zuul-scheduler containers | 20:23 |
| @jim:acmegating.com | dmsimard: the afs secrets are binary; but they might be b64 encoded; you can trace through and see how they're used: https://opendev.org/openstack/project-config/src/branch/master/zuul.d/secrets.yaml | 20:24 |
| @jim:acmegating.com | Clark: https://review.opendev.org/812237 is a debian image fix; zuul-preview is broken without it. | 20:26 |
| @jim:acmegating.com | https://review.opendev.org/812234 and child add important missing features for the operator | 20:27 |
| @jim:acmegating.com | https://review.opendev.org/807464 is an exciting new nodepool driver | 20:28 |
| @clarkb:matrix.org | corvus: got it. Also https://review.opendev.org/c/zuul/zuul/+/813255/2/tests/base.py does some really interesting magic with the git.util.IterableList. YOu basically tell it what attribute to check on in this case 'name' then you can check memership of complex entries using that key | 20:29 |
| @clarkb:matrix.org | corvus: left a note on https://review.opendev.org/c/zuul/zuul-preview/+/812237 was that missed or intentional? | 20:30 |
| @jim:acmegating.com | yeah, i think the iterablelist was from Pierre-Louis Bonicoli.... maybe they're pilou here? not sure. and yeah, pretty neat :) | 20:32 |
| @jim:acmegating.com | Clark: was missed; guess it works because libc is sufficiently compatible :) | 20:32 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul-preview] 812237: Switch base image to debian buster https://review.opendev.org/c/zuul/zuul-preview/+/812237 | 20:33 | |
| @dmsimard:matrix.org | > <@jim:acmegating.com> dmsimard: the afs secrets are binary; but they might be b64 encoded; you can trace through and see how they're used: https://opendev.org/openstack/project-config/src/branch/master/zuul.d/secrets.yaml | 20:37 |
| So this is still encrypting a string (b64) and not binary, but hey, it works :p | ||
| @dmsimard:matrix.org | Thanks for the tip. | 20:37 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 814065: Serialize ProjectMetadata on JobGraph https://review.opendev.org/c/zuul/zuul/+/814065 | 20:43 | |
| @clarkb:matrix.org | corvus: couple of questions on https://review.opendev.org/c/zuul/zuul/+/813255 | 20:45 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul-preview] 812237: Switch base image to debian buster https://review.opendev.org/c/zuul/zuul-preview/+/812237 | 20:49 | |
| @jim:acmegating.com | Clark: replied | 20:50 |
| @clarkb:matrix.org | thanks I responded and +2'd | 20:52 |
| @jim:acmegating.com | ack thx; i'll add the raise next time i have a clean working tree :) | 20:54 |
| @clarkb:matrix.org | super minor thing on https://review.opendev.org/c/zuul/zuul-operator/+/812234/ but +2'd because its minor | 21:00 |
| -@gerrit:opendev.org- Zuul merged on behalf of Ian Wienand: [zuul/zuul] 813733: gerrit: trim messages to "human length" https://review.opendev.org/c/zuul/zuul/+/813733 | 21:49 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 22:43 | |
| - [zuul/zuul] 813913: Serialize JobGraph objects to ZK https://review.opendev.org/c/zuul/zuul/+/813913 | ||
| - [zuul/zuul] 814065: Serialize ProjectMetadata on JobGraph https://review.opendev.org/c/zuul/zuul/+/814065 | ||
| - [zuul/zuul] 814069: Remove setBase from job freeze API https://review.opendev.org/c/zuul/zuul/+/814069 | ||
| - [zuul/zuul] 814070: Create Abstract and FrozenJob classes https://review.opendev.org/c/zuul/zuul/+/814070 | ||
| @jim:acmegating.com | tristanC: if you have a minute, can you take a quick look at https://review.opendev.org/814069 and let me know if you know of any reason that was important? i can't think of one. | 22:44 |
| @tristanc_:matrix.org | corvus: i can't remember why that was done... perhaps it was for the noop job? it can probably be removed | 22:48 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 22:57 | |
| - [zuul/zuul] 814069: Remove setBase from job freeze API https://review.opendev.org/c/zuul/zuul/+/814069 | ||
| - [zuul/zuul] 814070: Create Abstract and FrozenJob classes https://review.opendev.org/c/zuul/zuul/+/814070 | ||
| - [zuul/zuul] 814071: Add test_freeze_noop_job https://review.opendev.org/c/zuul/zuul/+/814071 | ||
| @jim:acmegating.com | tristanC: that's a good thought. i checked it by adding a test, and it looks like it's not necessary there either. https://review.opendev.org/814071 is the test i added, then i updated https://review.opendev.org/814069 to adjust the output accordingly. | 22:57 |
| @jim:acmegating.com | i think that'll be a good test to keep around since noop is special :) | 22:58 |
| @jim:acmegating.com | tristanC: thanks for looking | 22:58 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-preview] 814076: Use bullseye instead of buster https://review.opendev.org/c/zuul/zuul-preview/+/814076 | 23:23 | |
| @clarkb:matrix.org | corvus: ^ I don't know why that didn't occur to me until just now. But I ran a local build to confirm it seems to work with bullseye | 23:23 |
| @clarkb:matrix.org | i think it was the release of bullseye that allowed testing to move ahead and break? | 23:23 |
| @jim:acmegating.com | Clark: wfm, thx | 23:28 |
| @jim:acmegating.com | there's the notimplemented change | 23:30 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 23:30 | |
| - [zuul/zuul] 813255: Perform actual HTTP requests in gitlab tests https://review.opendev.org/c/zuul/zuul/+/813255 | ||
| - [zuul/zuul] 813256: Add gitlab disable_connection_pool option https://review.opendev.org/c/zuul/zuul/+/813256 | ||
| - [zuul/zuul] 813289: Stop GerritWebServer in tests https://review.opendev.org/c/zuul/zuul/+/813289 | ||
| @jim:acmegating.com | * there's the notimplemented change 👇️ | 23:30 |
| @clarkb:matrix.org | +2 thanks | 23:31 |
| @clarkb:matrix.org | I got distracted by pumpkin carving so didnt get nearly as much review done this afternoon as I thought I might | 23:31 |
| @clarkb:matrix.org | Will try to pick it upagain tomorrow between project renaming work and checkign meetpad before the PTG | 23:32 |
| @clarkb:matrix.org | The znode count looks really stable now | 23:32 |
| @jim:acmegating.com | well, with that in mind, how about this for a zuul 4.10.2 release tomorrow? commit 3066cbf9d60749ff74c1b1519e464f31f2132114 (HEAD -> master, tag: 4.10.2, origin/master) | 23:54 |
| @jim:acmegating.com | zuul-maint: ^ | 23:54 |
| @jim:acmegating.com | er, to be clear that's not actually master anymore... so more like: commit 3066cbf9d60749ff74c1b1519e464f31f2132114 (tag: 4.10.2) | 23:55 |
| @jim:acmegating.com | that is what opendev restarted on | 23:55 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!