Tuesday, 2022-04-05

-@gerrit:opendev.org- Ian Wienand proposed: [zuul/nodepool] 836263: Docker image: use unstable debootstrap https://review.opendev.org/c/zuul/nodepool/+/83626303:14
-@gerrit:opendev.org- Simon Westphahl proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 835464: Add a blob store and store large secrets in it https://review.opendev.org/c/zuul/zuul/+/83546408:53
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 836616: WIP: Debug log for unexpected skip https://review.opendev.org/c/zuul/zuul/+/83661610:59
-@gerrit:opendev.org- Andy Ladjadj proposed: [zuul/zuul-jobs] 836640: Update upload-logs azure role to support endpoint override https://review.opendev.org/c/zuul/zuul-jobs/+/83664012:04
-@gerrit:opendev.org- Andy Ladjadj proposed: [zuul/zuul-jobs] 836640: Update upload-logs azure role to support endpoint override https://review.opendev.org/c/zuul/zuul-jobs/+/83664012:06
-@gerrit:opendev.org- Andy Ladjadj proposed: [zuul/zuul-jobs] 836640: Update upload-logs azure role to support endpoint override https://review.opendev.org/c/zuul/zuul-jobs/+/83664012:27
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 836616: WIP: Debug log for unexpected skip https://review.opendev.org/c/zuul/zuul/+/83661613:05
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 836616: WIP: Debug log for unexpected skip https://review.opendev.org/c/zuul/zuul/+/83661613:12
@clarkb:matrix.orgNot sure if this has been captured here yet but wanted to call it out in case others see this or if there is work in progress to address it. In the past if two changes in different repos with a depends-on relationship between them did not share a queue then zuul would effectively ignore any approvals in the child change and you would have to manually retrigger approvals to get the child to land after the parent has landed. Now it apperas that zuul may -2 the child change15:02
@clarkb:matrix.orghttps://review.opendev.org/c/openstack/openstack-ansible/+/836378 is a child where this appears to have happened?15:02
@clarkb:matrix.orgI think this is fallout from the better error rpeorting, but this may be a caes where we want to not report back an error and continue to noop?15:03
@clarkb:matrix.orgI remember revieiwing the code that I suspect changed this behavior, but  Idon't remember the specific change. Can try to dig that up after meetings today15:05
@clarkb:matrix.orghttps://review.opendev.org/c/zuul/zuul/+/832362 this is the chagne I think15:18
@clarkb:matrix.orgUnrelated: does the metastatic driver allow you to combine resources from two different providers into the same nodeset? This has come up in the context of OpenDev not having mixed x86 and arm64 providers. But if we could set up a metastatic provider that acted as proxy for x86 and arm64 instances could we get a single nodeset with nodes from different providers that way? 15:44
@fungicide:matrix.orgi have a vague recollection that one of the stated use cases for the metastatic driver was to be able to aggregate a nodeset built on top of multiple traditional node providers, though i can't find confirmation of that or any clear examples. did i dream it?15:45
@clarkb:matrix.orgI guess this might actually work when I thought it wouldn't since metastatic operates on backing-labels only and that doesn't imply providers?15:45
@fungicide:matrix.orghah, jinx15:45
@fungicide:matrix.orgclearly we're both in the same ptg session ;)15:45
@jim:acmegating.com> <@clarkb:matrix.org> https://review.opendev.org/c/zuul/zuul/+/832362 this is the chagne I think16:38
I think the only change there is that we now actually report that message to the user instead of silently -2ing it and only reporting that to the log.
@clarkb:matrix.orgcorvus: well we didn't report a -2 to gerrit previously. Instead zuul was silent.16:39
@clarkb:matrix.orgThe problem is with clean chekc the -2 is problematic16:39
@jim:acmegating.com> <@clarkb:matrix.org> I guess this might actually work when I thought it wouldn't since metastatic operates on backing-labels only and that doesn't imply providers?16:39
It's true that metastatic can get labels from multiple providers, but not simultaneously. it's kind of the opposite of what you are asking for. but the *structure* of the driver is an example of how to accomplish that.
@jim:acmegating.com> <@clarkb:matrix.org> corvus: well we didn't report a -2 to gerrit previously. Instead zuul was silent.16:40
I'm not sure that was universally true; or if it was, that it was intentional and not a bug. silently doing nothing when someone approves a change sounds like it could be problematic from a UX pov.
@clarkb:matrix.org> <@jim:acmegating.com> It's true that metastatic can get labels from multiple providers, but not simultaneously.  it's kind of the opposite of what you are asking for.  but the *structure* of the driver is an example of how to accomplish that.16:40
Ah ok so I wasn't misremembering. It will still preserve the all nodes in a nodeset are from the same provider behavior.
@clarkb:matrix.org> <@jim:acmegating.com> I'm not sure that was universally true; or if it was, that it was intentional and not a bug.  silently doing nothing when someone approves a change sounds like it could be problematic from a UX pov.16:41
I agree, I think we can report back a message but it does seem odd to -2 something that wasn't tested and doesn't merge conflict and is merely waiting for a parent change in another queue to be processed
@jim:acmegating.comi could be wrong -- that may have changed in that change16:42
@jim:acmegating.comand yes, clean check is terrible16:42
@clarkb:matrix.orgI think it has been common for reviewers to review changes and approve as approprtiate. Then change owners can recheck as parents merge16:43
@clarkb:matrix.organd since there wasn't a -2 or -1 before things would go straight to the gate but now they go into check first16:44
@jim:acmegating.comi think to improve that we would need a new kind of reporter; since right now the failure reporter says to "-2"16:44
@clarkb:matrix.orgsomething like a status reporter?16:44
@jim:acmegating.comif this is happening a lot, it may be an indication that the shared queues are not constructed correctly16:44
@clarkb:matrix.orgyes that was also mentioned16:45
@clarkb:matrix.orgI think the project (openstack ansible in this case) was going to look at it. I had to clarify that queues are a project level setting not pipeline level16:45
@jim:acmegating.comthat example was openstack-ansible depends-on openstack-ansible-plugins -- so... i mean... as a bystander if i had to guess whether those things should share a queue, i'd guess "yes? probably?"16:45
@clarkb:matrix.orgagreed. Possible the first step here is to set up queues appropriately then reevaluate if this is a persistent problem afterwards (lots of cross project testing where it doesn't make sense for shared queues? I don't know what situation that would be and don't expect it but theoretically possible)16:47
@nhicher:matrix.orghello, I'm in the middle of an upgrade from zuul 4.6 to 5.2 and I have to push a change to fix issue on the pipelines. How I can tell zuul to use the config from the git repo and not what is stored in zookeeper?17:58
@clarkb:matrix.orgnhicher: if you force merge the change while zuul is running i think it would see the event and update it (this is assuming you can't land it normally as zuul is running without a valid config)18:02
@clarkb:matrix.orgotherwise if that isn't viable I'm not sure what the best option is. You can always clear the entire zookeeper state using the command for that. The startup will be much slower than normal but I expect that would work?18:02
@nhicher:matrix.orgClark: ok, thanks I will try that18:03
@clarkb:matrix.orgnote you want to use the zuul command for clearing zk state as some content shouldn't be removed 18:03
@clarkb:matrix.orgthe command knows what to clear out18:04
@clarkb:matrix.orgrather than doing it manually I mean18:04
@nhicher:matrix.orgClark: yes I don't want to delete manually18:04
@tristanc_:matrix.orgClark: thanks, that seems to have done the trick. Though it's a bit brutal to stop everything to recover from such issues.19:15
@tristanc_:matrix.orgwhile debugging the issues, we also noticed that zuul-web was running many `git ls-remote` process, is that expected?19:16
@clarkb:matrix.orgzuul-web loads configs now but I thought it did all of that via zookeeper and the mergers. I'm not sure what remotes it would ls19:19
@nhicher:matrix.orgClark: thanks =)19:26
@clarkb:matrix.orgDo we render the zuul specs anywhere? I was going to link to the ansible restriction removal spec in the foundation newsletter19:45
@jim:acmegating.com> <@clarkb:matrix.org> Do we render the zuul specs anywhere? I was going to link to the ansible restriction removal spec in the foundation newsletter19:47
https://zuul-ci.org/docs/zuul/latest/developer/specs/unrestricted-ansible.html
@clarkb:matrix.orgthanks I just found them19:47
@jim:acmegating.comClark: use https://zuul-ci.org/docs/zuul/5.2.2/developer/specs/unrestricted-ansible.html it's more permanent19:47
@jim:acmegating.com(eventually -- like, hopefully within 1-2 months at most, we'll remove it from the latest builds)19:48
@clarkb:matrix.orgwill do19:48
@jim:acmegating.com(also, for avoidance of doubt, i am working on those changes)19:48
@tristanc_:matrix.orgClark: for what its worth, it seems like zuul-web is running git ls-remote for the projects that are using a git connection20:08
@clarkb:matrix.orgtristanC: ah it must be doing that to get the branch details for loading configs then20:11
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 836108: Handle greedy project name regexes https://review.opendev.org/c/zuul/zuul/+/83610823:36
@clarkb:matrix.orgcorvus: tobiash swest ^ ok I think that should do it23:36

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!