Thursday, 2021-04-29

openstackgerritJames E. Blair proposed zuul/zuul master: WIP: Add ExecutorApi  https://review.opendev.org/c/zuul/zuul/+/78783200:18
openstackgerritJames E. Blair proposed zuul/zuul master: WIP change zone handling in ExecutorApi  https://review.opendev.org/c/zuul/zuul/+/78783300:18
openstackgerritJames E. Blair proposed zuul/zuul master: WIP ExecutorAPI test rude scheduler build request deletions  https://review.opendev.org/c/zuul/zuul/+/78862200:18
openstackgerritJames E. Blair proposed zuul/zuul master: WIP ExecutorApi add hold test  https://review.opendev.org/c/zuul/zuul/+/78862300:18
openstackgerritJames E. Blair proposed zuul/zuul master: WIP ExecutorApi handle locking nonexistent requests  https://review.opendev.org/c/zuul/zuul/+/78862400:18
*** hamalq has quit IRC01:34
openstackgerritIan Wienand proposed zuul/zuul master: getHoldRequest: pad zeros if not full length  https://review.opendev.org/c/zuul/zuul/+/78866801:50
*** rlandy|rover|bbl is now known as rlandy|rover02:00
*** rlandy|rover has quit IRC02:05
*** evrardjp has quit IRC02:33
*** evrardjp has joined #zuul02:33
*** sam_wan has joined #zuul02:34
*** bhavikdbavishi has joined #zuul03:11
*** _erlon_ has quit IRC03:21
*** bhavikdbavishi1 has joined #zuul03:22
*** bhavikdbavishi has quit IRC03:24
*** bhavikdbavishi1 is now known as bhavikdbavishi03:24
*** jamesmcarthur_ has quit IRC04:04
*** saneax has joined #zuul04:04
*** jamesmcarthur has joined #zuul04:04
*** jamesmcarthur has quit IRC04:09
*** saneax has quit IRC04:16
*** vishalmanchanda has joined #zuul04:24
*** jamesmcarthur has joined #zuul04:27
*** bhavikdbavishi has quit IRC04:39
*** bhavikdbavishi has joined #zuul04:40
*** jfoufas1 has joined #zuul04:44
*** openstackgerrit has quit IRC05:01
*** jamesmcarthur has quit IRC05:12
*** jamesmcarthur has joined #zuul05:13
*** bhavikdbavishi has quit IRC05:14
*** jamesmcarthur has quit IRC05:17
*** sam_wan has quit IRC05:42
*** jamesmcarthur has joined #zuul05:43
*** jamesmcarthur has quit IRC05:55
*** saneax has joined #zuul06:06
*** jamesmcarthur has joined #zuul06:10
*** avass has quit IRC06:13
*** ajitha has joined #zuul06:21
*** jamesmcarthur has quit IRC06:28
*** bhavikdbavishi has joined #zuul06:35
*** jamesmcarthur has joined #zuul06:41
*** openstackgerrit has joined #zuul06:43
openstackgerritBenjamin Schanzel proposed zuul/zuul master: Add tenant name on NodeRequests for Nodepool  https://review.opendev.org/c/zuul/zuul/+/78868006:43
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Optimize layout recalculations  https://review.opendev.org/c/zuul/zuul/+/78868106:45
*** jamesmcarthur has quit IRC06:46
*** reiterative has quit IRC06:49
*** reiterative has joined #zuul06:49
*** avass has joined #zuul06:58
*** jamesmcarthur has joined #zuul07:00
*** hashar has joined #zuul07:04
*** jamesmcarthur has quit IRC07:05
*** nils has joined #zuul07:19
*** jpena|off is now known as jpena07:21
*** rpittau|afk is now known as rpittau07:27
*** jamesmcarthur has joined #zuul07:30
openstackgerritIan Wienand proposed zuul/nodepool master: [wip] switch to using devstack's clouds.yaml  https://review.opendev.org/c/zuul/nodepool/+/78868407:37
*** tosky has joined #zuul07:44
*** bhavikdbavishi has quit IRC07:51
*** bhavikdbavishi has joined #zuul07:53
*** jcapitao has joined #zuul08:00
*** bhavikdbavishi has quit IRC08:11
*** bhavikdbavishi has joined #zuul08:23
*** jamesmcarthur has quit IRC08:23
*** jamesmcarthur has joined #zuul08:39
openstackgerritTobias Henkel proposed zuul/zuul master: Fix broken jobs overriding checkout to an unprotected branch  https://review.opendev.org/c/zuul/zuul/+/78869408:43
openstackgerritTobias Henkel proposed zuul/zuul master: Fix broken jobs overriding checkout to an unprotected branch  https://review.opendev.org/c/zuul/zuul/+/78869408:47
*** nhicher has quit IRC09:37
*** jpena is now known as jpena|off09:45
*** tobberydberg has quit IRC09:47
*** hashar has quit IRC09:48
*** tobberydberg has joined #zuul09:52
*** jpena|off has quit IRC09:54
*** fbo has joined #zuul10:09
*** nhicher has joined #zuul10:10
*** sam_wan has joined #zuul10:39
*** ajitha has quit IRC10:40
*** jpena has joined #zuul10:46
openstackgerritSimon Westphahl proposed zuul/zuul master: WIP: Optimize layout recalculations  https://review.opendev.org/c/zuul/zuul/+/78868110:46
*** iurygregory has quit IRC10:54
*** iurygregory has joined #zuul10:58
*** jcapitao is now known as jcapitao_lunch11:10
*** jpena is now known as jpena|lunch11:31
*** rlandy has joined #zuul11:35
*** Shrews has joined #zuul11:46
*** ajitha has joined #zuul11:54
*** hashar has joined #zuul12:09
*** jcapitao_lunch is now known as jcapitao12:13
*** bhavikdbavishi has quit IRC12:18
*** bhavikdbavishi has joined #zuul12:19
*** bhavikdbavishi has quit IRC12:25
*** jpena|lunch is now known as jpena12:27
*** jamesmcarthur has quit IRC12:38
*** jamesmcarthur has joined #zuul12:38
*** bhavikdbavishi has joined #zuul12:42
openstackgerritSimon Westphahl proposed zuul/zuul master: Optimize layout re-calculation after re-enqueue  https://review.opendev.org/c/zuul/zuul/+/78868112:45
*** bhavikdbavishi1 has joined #zuul12:47
*** bhavikdbavishi has quit IRC12:49
*** bhavikdbavishi1 is now known as bhavikdbavishi12:49
*** sam_wan has quit IRC13:16
*** nhicher has quit IRC13:27
*** fbo has quit IRC13:27
*** nhicher has joined #zuul13:28
*** ssmashnuk has joined #zuul13:30
*** vishalmanchanda has quit IRC13:30
*** bhavikdbavishi has quit IRC13:53
*** nhicher has quit IRC14:05
openstackgerritSimon Westphahl proposed zuul/zuul master: Optimize layout re-calculation after re-enqueue  https://review.opendev.org/c/zuul/zuul/+/78868114:08
*** nhicher has joined #zuul14:09
*** rlandy is now known as rlandy|mtg14:21
*** hashar has quit IRC14:26
openstackgerritAde Lee proposed zuul/zuul-jobs master: Add role to enable FIPS on a node  https://review.opendev.org/c/zuul/zuul-jobs/+/78877814:49
*** rlandy|mtg is now known as rlandy14:50
*** Shrews is now known as Shrews_14:51
*** Shrews_ is now known as Shrews14:51
openstackgerritAde Lee proposed zuul/zuul-jobs master: Add role to enable FIPS on a node  https://review.opendev.org/c/zuul/zuul-jobs/+/78877814:59
*** bhavikdbavishi has joined #zuul15:01
*** bhavikdbavishi1 has joined #zuul15:04
*** bhavikdbavishi has quit IRC15:05
*** bhavikdbavishi1 is now known as bhavikdbavishi15:05
*** jfoufas1 has quit IRC15:10
*** bhavikdbavishi has quit IRC15:36
*** jamesmcarthur has quit IRC16:07
*** jamesmcarthur has joined #zuul16:14
*** jpena|off has joined #zuul16:18
*** hamalq has joined #zuul16:22
*** hamalq has quit IRC16:23
*** hamalq has joined #zuul16:24
*** jpena has quit IRC16:25
*** jpena|off has quit IRC16:32
*** jamesmcarthur has quit IRC16:34
*** jamesmcarthur has joined #zuul16:36
*** bhavikdbavishi has joined #zuul16:39
*** jcapitao has quit IRC16:45
*** jamesmcarthur has quit IRC16:55
*** jamesmcarthur has joined #zuul16:56
*** iurygregory has quit IRC16:58
*** nils has quit IRC16:59
*** rpittau is now known as rpittau|afk16:59
*** rlandy is now known as rlandy|rover16:59
*** saneax has quit IRC17:03
*** iurygregory has joined #zuul17:31
*** iurygregory has quit IRC17:31
*** josefwells has joined #zuul17:37
*** iurygregory has joined #zuul17:38
*** bhavikdbavishi has quit IRC17:53
*** bhavikdbavishi has joined #zuul17:54
*** rlandy|rover is now known as rlandy|mtg18:00
*** bhavikdbavishi has quit IRC18:00
*** bhavikdbavishi has joined #zuul18:00
*** ssmashnuk has quit IRC18:01
openstackgerritMerged zuul/zuul master: Switch to ZooKeeper backed build result events  https://review.opendev.org/c/zuul/zuul/+/78293918:06
*** bhavikdbavishi has quit IRC18:32
*** nils has joined #zuul18:39
josefwellshello zuulers.  I'm a bit confused about the working directory for stuff.  I've scanned the old IRC logs but that has left me more confused.  I see "prepare_workspace" role dumps stuff in "zuul_workspace_root", but "prepare_workspace_git" and pretty much everything else, drops data in "ansible_user_dir".  The "ansible_user_dir" seems like a18:51
josefwellsbuilt in ansible thing (but I can't confirm that looking at ansible docs).18:51
clarkbjosefwells: yes ansible_user_dir is the homedir for the user that ansible used to ssh18:52
clarkbI think zuul_workspace_root may be executor side18:52
josefwellsoh, I didn't catch that distinction.  Is it possible to override "ansible_user_dir"?18:53
clarkbjosefwells: you could change the user's homedir or change the user. I believe it also arrives as a fact and you may be able to override facts (but I would need to read ansible var precedence docs to know)18:58
*** rlandy|mtg is now known as rlandy|rover19:01
mordredyah - but ... I wouldn't do so lightly - many of the jobs in zuul-jobs make some assumptions about src dir locations19:06
mordredlook for instances of zuul_work_dir and zuul.project.src_dir - as well as the src_dir attribute in each item of the zuul.projects[] list19:06
*** hashar has joined #zuul19:13
*** ssmashnuk has joined #zuul19:22
josefwellsclarkb: changing the homedir sounds like the best solution.. I can build that into the zuul container config so that would at least all be together (vs. symlinks in the users actual home-dir)19:25
*** jamesmcarthur has quit IRC19:26
josefwellsAlso need to figure out how to have a separate workdir-per-job, since I'm not able to containerize all the work and instead am using static nodes with a big ugly NFS disk19:26
ssmashnukFirst off, thanks for everyone's help getting WinRM working.  I took the advice and abandoned WinRM for SSH and I have connectivity to my windows nodes now.  Build key setup is working, but I'm now running into a issues with 'prepare-workspace'.  How far should I expect to be getting with the out of the box Zuul modules on Windows nodes?  Not19:30
ssmashnuksure if the current failures are my lack of experience or if there is extremely limited Windows node support?19:30
*** ikhan has quit IRC19:31
clarkbssmashnuk: prepare-workspace requires rsync (it uses the synchronize module). Not sure if that is expected to work with windows or not19:33
avassssmashnuk: I think there's a synchronize-repos for windows but it may not be merged. let me check19:35
ssmashnukclarkb: Thanks.  The other issue was zuul_console.  It appears to try and run the python code for this via powershell.  How important is the zuul_console step?19:37
avasssmyers: I think we're using this pretty much as is: https://review.opendev.org/c/zuul/zuul-jobs/+/74000519:37
smyersssmashnuk ^ :D19:37
avassssmashnuk: zuul_console is not yet supported on windows :)19:37
avasssamccann: oh we probably added this: https://review.opendev.org/c/zuul/zuul-jobs/+/747640/2 I can check tomorrow if there's anything else we've changed with that role19:39
ssmashnukThanks everyone.  Lots of good leads here.19:39
avassclarkb: I don't know if there's an rsync for windows. but git with ssh works :)19:40
*** ssmashnuk has quit IRC19:42
corvusjosefwells: if you have multiple jobs running on a static node, i would recommend setting up multiple users and registering each as a distinct static node rather than using the same user and trying to have different workspaces; as mordred mentioned, there's a lot that assumes ansible_user_dir in zuul-jobs.19:47
avassInteresting comment from daniel: https://review.opendev.org/c/zuul/zuul-jobs/+/780662/16/roles/add-build-sshkey/tasks/create-key-and-replace.yaml#3119:52
clarkbavass: that likely does affect linux nodes. I suspect the controlpersist setting is related to that19:53
avasssince linux jobs never reset the connection there is that ping still using the inital session?19:53
avassclarkb: so there should probably be a meta: reset_connection before the ping tasks in that case19:54
corvusavass: likely so and i think your solution is correct19:54
corvusit is entirely possible that was written before CP was working correctly :)19:55
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Reset connection before testing build ssh-keys  https://review.opendev.org/c/zuul/zuul-jobs/+/78882619:56
avasssamccann: you might also want this if you're using static nodes: https://review.opendev.org/c/zuul/zuul-jobs/+/78066220:02
corvusavass: i think you mean ssmashnuk, who left a bit ago20:03
avassoops20:03
*** samccann has left #zuul20:05
josefwellscorvus:I can't easily make new users on these static nodes.  But this is all within the zuul-jobs stuff right?  If I build my own jobs/playbooks/roles I should be ok?  Just manual and have to keep an eye on zuul-dev to make sure you don't change anything that requires a corresponding change to the jobs?20:24
avassjosefwells: if you write your own playbooks to set that up it should be fine. we have similar problems with static windows nodes20:27
corvusjosefwells: yep, as avass said.  hopefully the prepare-workspace roles can cope with an alternate work dir?20:28
corvusyeah, there's a zuul_workspace_root variable that prepare-workspace uses20:30
corvusi think the issue is that nothing else in zuul-jobs uses that20:30
avasslooks like preparere-workspace-git uses "{{ ansible_user_dir }}/{{ zuul.project.src_dir }}" everywhere20:30
avasscorvus: maybe they should, and default it to that ^ ?20:31
*** jamesmcarthur has joined #zuul20:31
corvusavass: probably so...  guillaumec: that might be a better approach to take in https://review.opendev.org/735980 ?20:32
avassor rather default zuul_workspace_root to ansible_user_dir20:32
corvusavass: right that.  definitely not the second thing (that's zuul_work_dir)20:32
avasscorvus: I don't like that it changes to zuul.build by default since the workdir can be used as a git cache20:33
corvusright.  default should be current behavior, but if zuul-jobs used zuul_workspace_root globally, then a user could set zuul_workspace_root to {{zuul.build}}20:34
corvusor {{ansible_user_dir}}/{{zuul.build}}.  or /tmp/{{zuul.build}} or whatever :)20:34
josefwellsComing from jenkins, so forgive my thinking, but you get a working directory per-build, so I'm trying to emulate that since I need a workspace per-project-per-prospective-build, but I don't grok the zuul variables to desribe that20:35
corvusjosefwells: we all came from jenkins, no apology necessary :)20:35
josefwellsjust most of our workloads are not yet container-able, and for now I have to use LSF to get to any real compute.  Enough of my sad story20:36
corvusjosefwells: to get highly technical here: zuul itself doesn't care; and it does in fact create a workspace per build on its own executors.  it's just that the "standard library" roles in zuul-jobs mostly assume that they own the remote node they will be run on.20:36
corvusjosefwells: so that's why you won't have any probs if you don't use stuff from zuul-jobs.  but we really want you to at least use the git prep roles (the ones that copy the git repos from the zuul executor to the remote node) because those are a pain to get right.20:37
josefwellsah, hence the prepare_workspace does an rsync of an already ready-to-go filesystem20:37
josefwellsI noticed that the git jobs were quite detailed, which is why I started looking again at some of my assumptions about variables20:38
corvusjosefwells: yep.  and it looks like you can use prepare-workspace today, but it's kind of the old-and-busted way of doing things; prepare-workspace-git is nicer and can take advantage of long-lived caches on the remote node to speed things up.  but it looks like it may not be ready for that use case.20:38
josefwellscorvus: yes, and I even have (readonly) git caches on my static nodes20:39
josefwellscorvus: but if I could just have zuul create its own internal-workspace in NFS space, I could skip the middle-man and get down to work20:39
avassjosefwells, corvus: it could also work if nodepool had a way to keep track of which "id" or something it gave zuul when using max-parallel-jobs20:42
josefwellscorvus: if I add a symlink in the ansible_user_dir I guess I'm still left with only one workspace20:42
avassso if you set max-parallel-job: 4 you also get a workspace from 1 to 4. that could be used to work the same way jenkins does20:43
corvusavass: i don't see how without another user on the host20:43
corvusavass: it's true that nodepool could supply, say, an integer, but that's no different than zuul.uuid; there's nothing magic about it like there is for a user's homedir20:44
corvussorry zuul.build20:44
avasscorvus: could nodepool be made to split a node into multiple nodes internally when given a max-parallel-jobs?20:44
corvusavass: i don't get it, why?20:45
avassso it can keep track of which does it has delegated to zuul or not and set a nodepool.<attribute> accordingly that can be consumed by the job20:45
corvusavass: but how is that different than just using zuul.build?20:46
avasscorvus: it wouldn't be unique20:46
corvusor any other unique thing?20:46
corvusyou don't want it to be unique?20:46
avassi mean unique as in unique per worker, not unique each time the node is delegated20:46
corvusso you want the exact behavior you would get by having multiple users but without actually issuing an "adduser" command?20:47
avassyeah20:47
avassif for some reason it's not possible to do that20:47
corvusi'm sure nodepool could be modified to accomplish that, but it's not set up for that today20:48
corvustoday nodepool identifies unique nodes via (hostname, username, port)20:49
corvusso if you can vary any one of those, you get a new node in nodepool20:49
josefwellscorvus: port?  like ssh to something besides 22, like unprivileged ports and it looks like a new node?  I can do that.20:50
corvusif it's in your power to set hostnames, you could make a bunch of hostnames like "worker1".."workerN" all with the same ip, register each as a node in nodepool but all with the same username and you'd get that effect.20:50
corvusjosefwells: or port, yes.20:50
avassit's something we've wanted to take a look at but haven't had time to do so we've sort of accepted slow checkouts to unique workspaces instead.20:51
josefwellscorvus: I can request aliases, that is crazy but doable.  Our repository is not small.. so we can't really live with slow there20:51
avassand that we will eventually get rid of the need to use max-parallel-jobs20:52
josefwellscorvus: even if we have separate nodes (via port/hostname magic) how does that give us separate work-dirs?20:53
avassjosefwells: you could map the host name to a specific workdir20:53
corvusjosefwells: it doesn't; it only solves avass's problem of wanting some kind of persistent id to use in a separet workdir20:53
corvusso you'd still have the "nothing uses zuul_workspace_root except prepare-workspace" problem, but for prepare-workspace and your own playbooks, you could tell it to use the nodepool hostname as a semi-permanent/semi-unique workspace root.20:54
*** jamesmcarthur has quit IRC21:14
*** Shrews has quit IRC21:20
*** jamesmcarthur has joined #zuul21:22
*** sduthil has quit IRC21:24
josefwellscorvus: seems like I can put a symlink on the zuul-executor for /var/lib/zuul/builds -> /NFS/wherever21:26
josefwells"additional level of indirection..."21:26
josefwellsthat will not work when we move this to k8s using your fancy new operator, since our k8s is not in NFS space.21:33
fungithe cluster host can't expose nfs mounts into the containers, i guess?21:33
josefwellsnot without lots of work, or using other protocols.  mostly I try to keep our container world from mounting anything and get data in more modern ways (artifactory, git, etc)21:37
*** ajitha has quit IRC21:40
fungi'heh, we even map afs mounts into containers21:44
*** nils has quit IRC21:48
openstackgerritMatthieu Huin proposed zuul/zuul-client master: Add output formatters when relevant  https://review.opendev.org/c/zuul/zuul-client/+/78883221:53
*** hashar has quit IRC22:06
josefwellsOk, unrelated to my previous queries about workdirs, I'm running a modified-for-github docker quickstart, and my web instance and on my Build pages, I get an error about not being able to load pages from the servier:8000 (log server), but I can load those separately.  Any idea what could be going wrong there?22:07
clarkbCORS maybe?22:09
fungithe web instance is serving javascript which asks your browser to load the manifest from the log archive location. could the domains be different and your browser is unhappy?22:09
clarkbI think those errors will show up in the dev console22:09
josefwellsyeah, I thought cross-site scripting but it is all the same host and the error is a zuul error, not a browser error22:10
josefwellsI don't see errors in the web or log docker logs22:11
clarkbis it a 500 or 404 or something else? I'd probably styart with identifying the type of error then use that to determine where to look next22:11
fungizuul will embed an error message in the page if your browser fails to load the manifest22:11
josefwellsok, slight lie, but I think unrelated : "GET /1/1/0ba6a23db9f688bb383259ba52a5232ccce5155c/check/q6job/11b5e1c/job-output.json.gz HTTP/1.1" 404 19622:11
josefwellsth rest of the logs show 200 or 206 response22:12
clarkbfor a 404 I would go check the docroot and see what is present22:12
clarkbsince it is basically saying that path isn't valid22:12
josefwellsyeah, it sure looks like cross-site: Access to XMLHttpRequest at 'http://docker-host:8000/1/1/0ba6a23db9f688bb383259ba52a5232ccce5155c/check/q6job/11b5e1c/zuul-manifest.json' from origin 'http://docker-host:9000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.22:14
josefwellsWhich is odd, because I see a lot of 'Access-Control-Allow-Origin' in the source-code of zuul/web22:16
clarkbThe log server isn't zuul-web though I don't think22:17
clarkbya the logs container is apache22:18
corvuswe might need to update the quickstart apache config?22:20
clarkbHeader set Access-Control-Allow-Origin "*" is set in the httpd.confg22:20
clarkbis it possible josefwells has an old image build without that?22:21
corvustrue, but that only merged in october... josefwells any chance your quickstart httpd config is from before then?22:21
corvusyeah that :)22:21
corvusjosefwells: https://review.opendev.org/755864 is the change if you want to check for that22:22
josefwellsOMG totally possible, sorry to be a pain, but moving gerrit->github in the quickstart was quite an operation22:23
josefwellsso I am not up on my pulls22:24
clarkbjosefwells: I think this is a lcoal build fwiw22:25
corvusyeah, so apply that change and 'docker-compose build' i think?22:27
josefwellsclarkb corvus: yep, all works now.  again, thanks and sorry for being behind22:33
josefwellsclarkb corvus: I'll see if I can clean up my changes so you can have a github-quickstart22:34
corvusjosefwells: thanks -- note though that we deliberately chose to focus on just one system for the quickstart so that people could most easily and reliably get a zuul up and running.  as you noted, it's quite complicated to use github, and it's not something we can automate.  i don't want to be discouraging, but i think it's not straightforward and i don't want you to do a bunch of work that we don't end up22:43
corvusmerging.  if we were to do something like that, i could maybe see doing it like this -- there are some patches that turn the quick-start into a series of tutorials.  i'd very much like to merge them, and then maybe we could do an add-on tutorial for adding a github connection?22:43
corvusforgot to link to tutorial patch series: https://review.opendev.org/73206722:50
josefwellsThat would make sense, then just an alternate docker-compose when you reach step X22:50
mhuIs setting the list of possible builds/buildsets results in constants still desirable?23:18
mhusee https://review.opendev.org/c/zuul/zuul/+/73863223:18
ianwtobiash: could i get you to loop back on https://review.opendev.org/c/zuul/nodepool/+/785821 which stops a pretty consistent error we're seeing on nl0323:35
ianwnote there's now a refactor change underneath it from your comments just to keep things simple23:36
*** tosky has quit IRC23:40
ianwzuul-maint: this is a small one that fixes not trying to copy logs if docker fails like it did the other day : https://review.opendev.org/c/zuul/zuul-jobs/+/78701923:42
openstackgerritMatthieu Huin proposed zuul/zuul-client master: Add search filters when relevant  https://review.opendev.org/c/zuul/zuul-client/+/78884723:52
mhuzuul-maint: we can search builds through zuul-client, let's add buildsets, and single build/buildset info to the mix! https://review.opendev.org/q/topic:%2522builds_query%2522+(status:open) :)23:57

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!