| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 828818: encrypt-file : role to encrypt a file https://review.opendev.org/c/zuul/zuul-jobs/+/828818 | 02:06 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 828818: encrypt-file : role to encrypt a file https://review.opendev.org/c/zuul/zuul-jobs/+/828818 | 02:53 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 828818: encrypt-file : role to encrypt a file https://review.opendev.org/c/zuul/zuul-jobs/+/828818 | 03:10 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 828818: encrypt-file : role to encrypt a file https://review.opendev.org/c/zuul/zuul-jobs/+/828818 | 03:19 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 828818: encrypt-file : role to encrypt a file https://review.opendev.org/c/zuul/zuul-jobs/+/828818 | 03:24 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 829491: Don't submit empty node requests to Zookeeper https://review.opendev.org/c/zuul/zuul/+/829491 | 08:43 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 829310: Release tenant read lock early on pending reconfig https://review.opendev.org/c/zuul/zuul/+/829310 | 09:00 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 829491: Don't submit empty node requests to Zookeeper https://review.opendev.org/c/zuul/zuul/+/829491 | 10:33 | |
| -@gerrit:opendev.org- Simon Westphahl proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 827935: Populate missing change cache entries https://review.opendev.org/c/zuul/zuul/+/827935 | 11:04 | |
| -@gerrit:opendev.org- Simon Westphahl proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 827935: Populate missing change cache entries https://review.opendev.org/c/zuul/zuul/+/827935 | 12:46 | |
| -@gerrit:opendev.org- Szymon Datko proposed: [zuul/zuul-jobs] 829533: [ensure-python] Fix for CentOS/RHEL 9 packages https://review.opendev.org/c/zuul/zuul-jobs/+/829533 | 12:48 | |
| -@gerrit:opendev.org- Szymon Datko proposed: [zuul/zuul-jobs] 829533: [ensure-python] Fix for CentOS/RHEL 9 packages https://review.opendev.org/c/zuul/zuul-jobs/+/829533 | 12:49 | |
| -@gerrit:opendev.org- Szymon Datko proposed: [zuul/zuul-jobs] 829533: [ensure-python] Fix for CentOS/RHEL 9 packages https://review.opendev.org/c/zuul/zuul-jobs/+/829533 | 13:27 | |
| -@gerrit:opendev.org- Szymon Datko proposed: [zuul/zuul-jobs] 829544: [ensure-python] Allow overriding package name https://review.opendev.org/c/zuul/zuul-jobs/+/829544 | 13:48 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org: [zuul/zuul] 828888: Add a reconfig reminder to tenant config doc https://review.opendev.org/c/zuul/zuul/+/828888 | 15:18 | |
| @y2kenny:matrix.org | Gerrit has some kind of automatic rebase (Rebase if necessary submit type) but for some reason, one of the change got stuck in 'merge conflict' state while Zuul seems to stuck in a loop repeatedly triggering jobs in the gate pipeline. A manual click on Gerrit's web UI rebase button resolved the issue (so the rebase is basically trivial.) Am I missing some configuration on Zuul side? | 15:31 |
|---|---|---|
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 829310: Release tenant read lock early on pending reconfig https://review.opendev.org/c/zuul/zuul/+/829310 | 15:37 | |
| @y2kenny:matrix.org | ok nevermind, looks like it may be a gerrit side issue with content merging. | 15:48 |
| @y2kenny:matrix.org | on a submit related issue, newer version of Gerrit has a way to "submit together" a relation chain, is that something that can be done with Zuul? | 15:54 |
| @clarkb:matrix.org | I don't know that there is submit together support explicitly in Zuul yet. But Zuul does allow cycles in its dependency trees to be submitted together and functionally they should be similar. May just be a matter of adding some equivalency to the cycle handling | 15:57 |
| @jpew:matrix.org | When I remove a required score (e.g. Workflow +1) on a Gerrit, Zuul doesn't deqeue it. Is this expected and can I (or should I) fix it? | 15:59 |
| @clarkb:matrix.org | The behavior is known. I think that Gerrit only evaluates submittablity when enqueing and maybe when it attempts to submit? The processing of a new vote doesn't retrigger that query which would dequeue. As a user I prefer this behavior because the approval may be removed while gating is proceeding. If reviewers decide to add it back again (maybe after additional review) before testing completes then you avoid wasting a gate reset cycle | 16:02 |
| @jpew:matrix.org | Ya, I think it's probably confusing from a reviewers perspective | 16:04 |
| @clarkb:matrix.org | * The behavior is known. I think that Zuul only evaluates submittablity when enqueing and maybe when it attempts to submit? The processing of a new vote doesn't retrigger that query which would dequeue. As a user I prefer this behavior because the approval may be removed while gating is proceeding. If reviewers decide to add it back again (maybe after additional review) before testing completes then you avoid wasting a gate reset cycle | 16:04 |
| @clarkb:matrix.org | A gate reset is implied by a dequeue and they are very expensive to perform if there is more than one change in the gate. | 16:05 |
| @clarkb:matrix.org | The change cannot merge without the necessary votes so that aspect is handled. | 16:06 |
| @jpew:matrix.org | Ah! got it. It revalidates the votes when zuul attempts to actually submit the change. I didn't realize that | 16:06 |
| @jpew:matrix.org | I realize this is probably a bad idea, but can I setup SSH port forwarding over ansible between the executor and the worker node? | 16:08 |
| @clarkb:matrix.org | > <@jpew:matrix.org> Ah! got it. It revalidates the votes when zuul attempts to actually submit the change. I didn't realize that | 16:10 |
| Yes, Gerrit enforces this. You do need to ensure that the label functions aren't overridden to work around this behavior but it is the default and unlikely to be overridden for something like workflow+1 | ||
| -@gerrit:opendev.org- Szymon Datko proposed: [zuul/zuul-jobs] 829544: [ensure-python] Allow overriding package name https://review.opendev.org/c/zuul/zuul-jobs/+/829544 | 16:33 | |
| @jim:acmegating.com | Clark: ianw i've confirmed an ansible upgrade is still on my todo list (but not immediate). i'll probably look at jumping versions to the greatest extent possible. | 17:29 |
| @jim:acmegating.com | Clarkianw and i'm going to start looking into the log streaming issue now | 17:30 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 829591: Protect stream handler thread from exceptions https://review.opendev.org/c/zuul/zuul/+/829591 | 18:11 | |
| @jim:acmegating.com | zuul-maint: ^ that would be good to merge soon; it's affecting opendev currently and seems like a high probability bug. | 18:13 |
| @clarkb:matrix.org | I'll take a look. Thank you for diving into that for us | 18:13 |
| @jim:acmegating.com | well, pretty sure i broke it. :) | 18:14 |
| @fungicide:matrix.org | you can't make omelette without breaking the oeufs | 18:15 |
| @fungicide:matrix.org | or maybe the reverse of that should be zuul's next slogan... /Zuul: now you *can* make omelette without breaking an oeuf!/ | 18:18 |
| @fungicide:matrix.org | if nothing else, it'd be a fun presentation title | 18:19 |
| @clarkb:matrix.org | corvus: question for you on that change | 18:20 |
| @jim:acmegating.com | hrm. i'm wondering if we should just go back to garbage collection. | 18:31 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 829591: Protect stream handler thread from exceptions https://review.opendev.org/c/zuul/zuul/+/829591 | 18:51 | |
| @jim:acmegating.com | tristanC: Clark ^ can you take another look? i think that may be a bit better. and thanks :) | 18:52 |
| @jim:acmegating.com | actually i think i spot one more thing we should check | 18:56 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 829591: Protect stream handler thread from exceptions https://review.opendev.org/c/zuul/zuul/+/829591 | 18:58 | |
| @jim:acmegating.com | tristanC: Clark ^ okay, once more. sorry and thanks for your patience :) | 18:58 |
| @clarkb:matrix.org | will do. Sorry juggling a couple things right now | 19:28 |
| @clarkb:matrix.org | corvus: I've approved it | 19:51 |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 828818: encrypt-file : role to encrypt a file https://review.opendev.org/c/zuul/zuul-jobs/+/828818 | 20:37 | |
| -@gerrit:opendev.org- Jonathan Rosser proposed: [zuul/zuul-jobs] 829028: Allow some configure-mirrors repositories to be disabled https://review.opendev.org/c/zuul/zuul-jobs/+/829028 | 20:38 | |
| -@gerrit:opendev.org- Jonathan Rosser proposed: [zuul/zuul-jobs] 829028: Allow some configure-mirrors repositories to be disabled https://review.opendev.org/c/zuul/zuul-jobs/+/829028 | 20:39 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Jonathan Rosser: [zuul/zuul-jobs] 829028: Allow some configure-mirrors repositories to be disabled https://review.opendev.org/c/zuul/zuul-jobs/+/829028 | 21:48 | |
| @clarkb:matrix.org | corvus: ianw dug into an interesting one in #opendev just now. I think what we have found is that when Zuul is talking to a code review system whose repos have two behaviors Zuul can fail to setRepoState and fail jobs with minimal reporting to the user. Specifically if the repo deletes branches during the time that zuul is trying to setRepoState and the ref the branch points to goes away (so isn't merged into another location) then zuul errors | 23:33 |
| @clarkb:matrix.org | This happened with the ansible-collections/community.general repo because they seem to do a rebase merge rather than a proper merge and they put branches into the repo itself that they PR against the repo (rather than in another repo). This means zuul sees their repo at sha foo, wants to setRepoState for that branch but if timing is correct that branch is deleted and so is the sha object because they did a rebase not a merge | 23:34 |
| @clarkb:matrix.org | I'm not sure how zuul might handle that better since the error git reports back is the same as the object actually not existing | 23:34 |
| @clarkb:matrix.org | we do want zuul to fail in the case the object doesn't exist | 23:34 |
| @clarkb:matrix.org | Maybe zuul needs to do two passes? (we could just race these conditions again with another branch though). Perhaps the best thing is to try and report back to the users a better picture that this happened | 23:35 |
| @jim:acmegating.com | Clark: normally you'd set this: https://zuul-ci.org/docs/zuul/latest/tenants.html#attr-tenant.untrusted-projects.%3Cproject%3E.exclude-unprotected-branches | 23:36 |
| @jim:acmegating.com | but that may not be useful in this case | 23:36 |
| @clarkb:matrix.org | is that beacuse all of their branches may be unprotected? | 23:37 |
| @jim:acmegating.com | Clark: but https://review.opendev.org/804177 may be useful | 23:37 |
| @jim:acmegating.com | Clark: yeah, or really that i just don't know their branch protection strategy and whether it maps to whatever you're interested in :) | 23:37 |
| @clarkb:matrix.org | got it and I agree that change looks like it would be useful | 23:38 |
| @clarkb:matrix.org | since we could make a proactive declaration about what we care and ignore their proection strategy | 23:38 |
| @jim:acmegating.com | yeah, it would let you say what you're interested in regardless of what's set remotely. yep exactly. | 23:38 |
| @jim:acmegating.com | Clark: i think it'd need the followup too, which appears broken | 23:39 |
| @jim:acmegating.com | https://review.opendev.org/804178 | 23:39 |
| @jim:acmegating.com | but you know, if there's some traction on the first one i can fix up the followup. | 23:39 |
| @clarkb:matrix.org | do you want me to review the first one before you rebase? Or just a promise that I'll do my best to review if rebased :) | 23:40 |
| @jim:acmegating.com | Clark: but be aware -- you're editing the repo in that case. i think the maintenance burden will be non-zero | 23:40 |
| @clarkb:matrix.org | corvus: right, you're effectively limitign the view on the repo. I think I'm ok with that and if they get failures because the branches they need aren't there that is on them? | 23:41 |
| @jim:acmegating.com | Clark: yeah, that's the contract :) | 23:41 |
| @clarkb:matrix.org | This really makes me think that non merge merge strategies should be avoided | 23:41 |
| @clarkb:matrix.org | this would've been fine if they just merged the commit properly | 23:42 |
| @clarkb:matrix.org | since the sha objects would all be present and we would set a branch to that sha for that job even though the branch has been deleted | 23:42 |
| @clarkb:matrix.org | then the next buildset would see the branch is gone and ignore it | 23:42 |
| @jim:acmegating.com | temporary branches can certainly cause problems for systems which guarantee reproducible builds :) | 23:42 |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul] 829617: merger: ignore a missing object when creating refs https://review.opendev.org/c/zuul/zuul/+/829617 | 23:53 | |
| @iwienand:matrix.org | even if that is totally wrong, it would be great to find some way to not swallow this error | 23:53 |
| @clarkb:matrix.org | ya I think reporting back that there was a setRepoState (more human readable string?) error would be good | 23:54 |
| @jim:acmegating.com | sure, reporting an error is good, but if i understand ianw's change correctly it just ignores the error? | 23:55 |
| @clarkb:matrix.org | ya I don't think we should do a blanket ignore | 23:55 |
| @jim:acmegating.com | so that's a -2 from me. | 23:55 |
| @clarkb:matrix.org | because if the repo is actually missing content we need that should be a failure | 23:56 |
| @iwienand:matrix.org | it logs and ignores there .. but my intiution is that if we need that ref later, we will get a better error message when we try to check it out? | 23:56 |
| @clarkb:matrix.org | either we need to determine if the branch is gone (and thus its content being gone is ok), or we need to report the error better | 23:56 |
| @jim:acmegating.com | we may never try to check it out | 23:56 |
| @jim:acmegating.com | but it can still produce incorrect job results | 23:56 |
| @jim:acmegating.com | (just the absence of a branch or tag can alter jobs) | 23:56 |
| @clarkb:matrix.org | ya we might be able to catch the error and then cehck the remote to see if the ref still exists. If it does then fail. If it doesn't then remove it from the state? | 23:57 |
| @jim:acmegating.com | Clark: we'd have to reset the item | 23:58 |
| @clarkb:matrix.org | but as mentioned before you can end up where the next deletion trips you over while you process the next pass | 23:58 |
| @jim:acmegating.com | it would be better to just report the error to the user | 23:58 |
| @jim:acmegating.com | then you+user can figure out how to resolve the branch selection situation | 23:58 |
| @iwienand:matrix.org | do you have any intuition on why this was not reported back as an error? | 23:59 |
| @jim:acmegating.com | did it retry_limit? | 23:59 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!