| @jjbeckman:matrix.org | @flaper87 I just tried out your solution and it solved my issue. Thanks a lot for your advice. I doubt I'd had been able to pinpoint this workaround alone. | 04:36 |
|---|---|---|
| @jjbeckman:matrix.org | > not kubernetes, but similar situation in opendev as well: | 04:39 |
| I see... interesting. | ||
| @jjbeckman:matrix.org | This workaround definitely solved my speed issues, but the Zuul Console still doesn't seem to function. | 05:16 |
| I guess I'll try to solve this issue next. Any advice would be appreciated. | ||
| ``` | ||
| root@zuul-executor-0:/# ps aux | ||
| USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND | ||
| root 1 0.0 0.0 2228 568 ? Ss 02:50 0:00 /usr/bin/dumb-init -- /usr/local/bin/zuul-executor -f -d --keep-jobdir | ||
| root 7 0.1 0.3 1463996 119372 ? Ssl 02:50 0:10 /usr/local/bin/python /usr/local/bin/zuul-executor -f -d --keep-jobdir | ||
| root 15 0.0 0.2 199876 88992 ? Sl 02:50 0:00 /usr/local/bin/python /usr/local/bin/zuul-executor -f -d --keep-jobdir | ||
| root 24 0.0 0.0 15192 12380 ? S 02:50 0:00 /usr/local/bin/python -c from multiprocessing.resource_tracker import main;main(12) | ||
| root 6146 0.0 0.3 124812 103968 ? S 04:30 0:01 /usr/local/bin/python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=23) --multiprocessing-fork | ||
| root 12123 0.0 0.0 9852 3596 ? S 05:03 0:00 git cat-file --batch-check | ||
| root 12344 0.0 0.0 9852 3584 ? S 05:04 0:00 git cat-file --batch-check | ||
| root 12972 0.0 0.0 6024 2704 ? Ss 05:07 0:00 ssh-agent | ||
| root 12983 0.0 0.0 6912 4016 ? S 05:07 0:00 git cat-file --batch-check | ||
| root 12986 0.0 0.0 6768 3944 ? S 05:07 0:00 git cat-file --batch-check | ||
| root 12995 0.0 0.0 6872 3792 ? S 05:07 0:00 git cat-file --batch-check | ||
| root 13005 0.0 0.0 9852 3704 ? S 05:07 0:00 git cat-file --batch-check | ||
| root 13028 0.0 0.0 6872 3804 ? S 05:07 0:00 git cat-file --batch-check | ||
| root 13038 0.0 0.0 9852 3584 ? S 05:07 0:00 git cat-file --batch-check | ||
| root 13047 0.0 0.2 1388428 71948 ? Sl 05:07 0:00 kubectl --kubeconfig=/var/lib/zuul/builds/2bda707d4aad46b3b36ccf8f86c7dd33/work/.kube/config --context=main-0000001375:zuul-worker/10-0-0-1:443 -n main-0000001375 port-forwa | ||
| root 13347 0.0 0.0 3428 708 ? Ss 05:07 0:00 bwrap --dir /tmp --tmpfs /tmp --dir /var --dir /var/tmp --dir /run/user/0 --ro-bind /usr /usr --ro-bind /lib /lib --ro-bind /bin /bin --ro-bind /sbin /sbin --ro-bind /etc/ld | ||
| root 13349 0.0 0.0 3560 1624 ? S 05:07 0:00 bwrap --dir /tmp --tmpfs /tmp --dir /var --dir /var/tmp --dir /run/user/0 --ro-bind /usr /usr --ro-bind /lib /lib --ro-bind /bin /bin --ro-bind /sbin /sbin --ro-bind /etc/ld | ||
| root 13350 1.1 0.1 217960 59432 ? Sl 05:07 0:04 /usr/local/lib/zuul/ansible/6/bin/python /usr/local/lib/zuul/ansible/6/bin/ansible-playbook -v /var/lib/zuul/builds/2bda707d4aad46b3b36ccf8f86c7dd33/untrusted/project_0/gith | ||
| root 13668 0.0 0.1 219748 51032 ? S 05:07 0:00 /usr/local/lib/zuul/ansible/6/bin/python /usr/local/lib/zuul/ansible/6/bin/ansible-playbook -v /var/lib/zuul/builds/2bda707d4aad46b3b36ccf8f86c7dd33/untrusted/project_0/gith | ||
| root 13700 0.0 0.2 1240964 69476 ? Sl 05:07 0:00 /usr/local/bin/kubectl --context main-0000001375:zuul-worker/10-0-0-1:443 exec -i debian-bullseye -- /bin/sh -c /bin/sh -c '/usr/bin/python3 /home/zuul/.ansible/tmp/ansible- | ||
| root 13830 0.0 0.0 6048 3824 pts/0 Ss 05:10 0:00 bash | ||
| root 14358 0.0 0.0 8644 3216 pts/0 R+ 05:13 0:00 ps aux | ||
| root@zuul-executor-0:/# | ||
| ``` | ||
| ``` | ||
| root@zuul-executor-0:/# ss -lp | grep 19885 | ||
| RTNETLINK answers: Invalid argument | ||
| root@zuul-executor-0:/# | ||
| ``` | ||
| @jjbeckman:matrix.org | Zuul Console doesn't appear to be working despite there being a console process and the console port being listened to. | 05:20 |
| ``` | ||
| root@debian-bullseye:/# ps aux|grep console | ||
| root 237 3.1 0.0 394460 16832 ? Sl 05:07 0:22 /usr/bin/python3 /home/zuul/.ansible/tmp/ansible-tmp-1686546450.6016178-54-222368108504795/AnsiballZ_zuul_console.py | ||
| root 1382 0.0 0.0 5124 708 pts/0 S+ 05:19 0:00 grep console | ||
| root@debian-bullseye:/# ss -lp | grep 19885 | ||
| RTNETLINK answers: Invalid argument | ||
| tcp LISTEN 0 1 *:19885 *:* users:(("python3",pid=237,fd=3)) | ||
| root@debian-bullseye:/# | ||
| ``` | ||
| @yoctozepto:matrix.org | Hi! Asking here since it's more lively here than on the mailing list. [The zuul-jobs-test-ensure-kubernetes-crio-ubuntu-bionic job is failing for almost half a year now](https://zuul.opendev.org/t/zuul/builds?job_name=zuul-jobs-test-ensure-kubernetes-crio-ubuntu-bionic&project=zuul/zuul-jobs) - does anybody mind if I propose to cease its testing? Is there any interest in keeping upstream bionic support? | 10:39 |
| @yoctozepto:matrix.org | (Or where to find a policy on this support/deprecations?) | 10:39 |
| @flaper87:matrix.org | Ok, so, the console is actually running, which is good. Can you confirm it is the very first thing that runs in your playbooks? | 11:54 |
| @flaper87:matrix.org | Now, I would recommend you to set the executor in debug mode and get the logs from there | 11:54 |
| @flaper87:matrix.org | put ansible in debug mode too | 11:55 |
| @flaper87:matrix.org | Do Zuul CI jobs produce temp container builds for a specific commit/review? I think I want to get a build from https://review.opendev.org/c/zuul/zuul/+/885455 and put it in our env to do further testing. If not, I can build it myself | 11:59 |
| @yoctozepto:matrix.org | flaper87: you mean like: https://zuul.opendev.org/t/zuul/build/fd673c04d523402cb7d3799538c20ad0/artifacts ? | 12:18 |
| @yoctozepto:matrix.org | not that I ever tried downloading these | 12:19 |
| @yoctozepto:matrix.org | they might be just internal references fwiw | 12:19 |
| @flaper87:matrix.org | those seem to point to an internal registry! I think I will create a custom build for now but, if someone knows a better way, I'm all ears | 13:06 |
| @fungicide:matrix.org | > <@flaper87:matrix.org> those seem to point to an internal registry! I think I will create a custom build for now but, if someone knows a better way, I'm all ears | 13:42 |
| insecure-ci-registry.opendev.org is publicly accessible, we regularly try those sorts of artifacts out with docker pull/run locally or whatever | ||
| @flaper87:matrix.org | > <@fungicide:matrix.org> insecure-ci-registry.opendev.org is publicly accessible, we regularly try those sorts of artifacts out with docker pull/run locally or whatever | 13:43 |
| oh, wow! thanks. I will give it a try :) | ||
| @fungicide:matrix.org | it's named that way because we don't want users to rely on those for production deployments, but pulling from it to try out the results of a proposed change is an intended use case | 13:43 |
| @flaper87:matrix.org | that worked, thanks fungi and yoctozepto :) | 14:46 |
| @yoctozepto:matrix.org | yw! | 14:47 |
| @yoctozepto:matrix.org | > <@yoctozepto:matrix.org> Hi! Asking here since it's more lively here than on the mailing list. [The zuul-jobs-test-ensure-kubernetes-crio-ubuntu-bionic job is failing for almost half a year now](https://zuul.opendev.org/t/zuul/builds?job_name=zuul-jobs-test-ensure-kubernetes-crio-ubuntu-bionic&project=zuul/zuul-jobs) - does anybody mind if I propose to cease its testing? Is there any interest in keeping upstream bionic support? | 14:47 |
| retrying in US hours ;-) | ||
| @yoctozepto:matrix.org | duh, just realised it's the summit week so folks are probably busy moving around | 14:52 |
| @yoctozepto:matrix.org | time flies | 14:52 |
| @tony.breeds:matrix.org | yoctozepto: Any idea why it's going to POST_FAILURE? | 16:48 |
| @yoctozepto:matrix.org | tony.breeds: it fails in run stage already; post just fails because it assumes the k8s at least got installed :D it seems minikube ceased to work properly on bionic or something like that, I did not debug further | 16:50 |
| @tony.breeds:matrix.org | yoctozepto: Okay, I guess we can look at swicthing it to jammy. and debugging that | 16:53 |
| @yoctozepto:matrix.org | tony.breeds: focal and jammy are running fine | 16:53 |
| @yoctozepto:matrix.org | they are sibling jobs | 16:53 |
| @tony.breeds:matrix.org | Ah okay. | 16:55 |
| @plscosta:matrix.org | Hello @zuul channel. | 21:09 |
| I recently started having contact with zuul in my workplace, currently I’m trying to create a pipeline with the goal of automatic review on some specific use cases. | ||
| I have read some of the documentation, and I was wondering if we could as a “require” validation specify a specific comment in the change so the pipeline would run. | ||
| it is not mentioned here: | ||
| https://zuul-ci.org/docs/zuul/latest/drivers/gerrit.html#requirements-configuration | ||
| but still, I would like to know if it’s possible, I know that comments can be used on the event triggers… it would be like that… but in the require section. | ||
| @clarkb:matrix.org | I think recent changes were made so that requires and triggers support the same set of rules so that should be possible | 21:46 |
| @plscosta:matrix.org | Hey Clark, | 21:58 |
| Thanks for the answer, do you know in which version this stands? | ||
| @fungicide:matrix.org | > <@plscosta:matrix.org> Hey Clark, | 22:36 |
| > Thanks for the answer, do you know in which version this stands? | ||
| "Gerrit pipeline triggers now support embedded require and reject filters in order to match. Any conditions set for the pipeline in require or reject filters may also be set for event trigger filters." https://zuul-ci.org/docs/zuul/latest/releasenotes.html#relnotes-8-3-0 | ||
| @fungicide:matrix.org | so as of 8.3.0 a few weeks ago | 22:37 |
| @plscosta:matrix.org | > <@fungicide:matrix.org> so as of 8.3.0 a few weeks ago | 22:47 |
| you don't have examples on this by any chance? | ||
| @fungicide:matrix.org | i'll check the docs | 22:47 |
| @fungicide:matrix.org | > <@plscosta:matrix.org> you don't have examples on this by any chance? | 22:54 |
| https://zuul-ci.org/docs/zuul/latest/drivers/gerrit.html#attr-pipeline.trigger.%3Cgerrit%20source%3E.require describes it and there are examples in zuul's own unit test fixtures: https://opendev.org/zuul/zuul/src/branch/master/tests/fixtures/layouts/gerrit-trigger-requirements.yaml | ||
| @plscosta:matrix.org | require: | 22:54 |
| gerrit: | ||
| open: True | ||
| current-patchset: True | ||
| - event: comment-added | ||
| comment: some-comment | ||
| fungi would the above be correct? | ||
| @fungicide:matrix.org | rereading your initial question, i'm not sure i complete grasp how what you're asking for is any different from just triggering on a specific comment pattern. what is the additional requirement you're trying to impose besides the comment match itself? | 23:00 |
| @fungicide:matrix.org | * rereading your initial question, i'm not sure i completely grasp how what you're asking for is any different from just triggering on a specific comment pattern. what is the additional requirement you're trying to impose besides the comment match itself? | 23:00 |
| @fungicide:matrix.org | like can you describe the expected workflow in more words? | 23:01 |
| @plscosta:matrix.org | so the first pipeline runs some code analyses: | 23:04 |
| - change requires human approval, and gives +1 | ||
| - change has something wrong and needs fixing, gives -1 | ||
| - change can be automatic reviewed, writes comment saying so and gives +1 | ||
| the second pipeline should require the +1 and the comment gives by the first pipeline... once these are validated it can give +2 | ||
| @plscosta:matrix.org | * so the first pipeline runs some code analyses: | 23:05 |
| - change requires human approval, and gives +1 | ||
| - change has something wrong and needs fixing, gives -1 | ||
| - change can be automatic reviewed, writes comment saying so and gives +1 | ||
| the second pipeline should require the +1 and the comment given by the first pipeline... once these are validated it can give +2 | ||
| @fungicide:matrix.org | can you explain what you mean when you say "change can be automatic reviewed"? what is the automation doing and how does that relate to a review? | 23:08 |
| @plscosta:matrix.org | the issue that I'm running into is that when the first pipeline writes the comment saying the in is approved it has not yet finished... and so the change does not have the +1 | 23:08 |
| @plscosta:matrix.org | * the issue that I'm running into is that when the first pipeline writes the comment saying that "it can be automatically be reviewed", the first pipeline is still running other jobs... and so the change does not have the +1 | 23:10 |
| and the second pipeline never gets triggered... | ||
| @plscosta:matrix.org | > <@fungicide:matrix.org> can you explain what you mean when you say "change can be automatic reviewed"? what is the automation doing and how does that relate to a review? | 23:11 |
| it's a simple job that looks for what was changed in that commit... if nothing critical was changed it can get a automatic +2 by the second pipeline... | ||
| @plscosta:matrix.org | My real problem is kind of a "racing condition " | 23:12 |
| @fungicide:matrix.org | oh, so a change/revision is uploaded, your auto-review pipeline is triggered by the upload, it checks whether the change is "safe" to not require human review, and automatically votes (in a code-review label?) if that's the case | 23:13 |
| @plscosta:matrix.org | that sums it up, yes | 23:14 |
| @fungicide:matrix.org | and you have two different vote requirements for the change to be enqueued into the gating pipeline, both of which must be satisfied. maybe what you need is two triggers, one that triggers when the first type of event happens as long as the second condition is met, another which triggers on the second type of event when the first condition is met? | 23:15 |
| @fungicide:matrix.org | so two triggers for your gating pipeline, either one of which would suffice because each one checks whether the other requirement has been satisfied | 23:16 |
| @fungicide:matrix.org | that way they can arrive in either order | 23:17 |
| @plscosta:matrix.org | yes you are correct, and that is were the "comment requirement" enters the picture... | 23:17 |
| @fungicide:matrix.org | and the second one will always be acted on | 23:17 |
| @fungicide:matrix.org | whichever one arrives second | 23:17 |
| @fungicide:matrix.org | does it have to be a string match on a comment? this is fairly easy to do with vote labels because they're basically aspects of the change. a comment string is isn't really part of the change, so what you're really saying is "someone has left a comment previously which matches this" and i don't think zuul could know that without scraping the comments since all prior comments aren't included in every new event emitted by gerrit | 23:20 |
| @fungicide:matrix.org | you can match on a comment string for a comment-added event because the comment content is included as part of that comment-added event gerrit sends | 23:21 |
| @fungicide:matrix.org | but i don't know of a way to retroactively match on the existence of an earlier comment's content | 23:21 |
| @fungicide:matrix.org | i've definitely seen plenty of what you're describing done by just adding additional vote labels for the project | 23:22 |
| @plscosta:matrix.org | my initial idea was to force the pipeline to "always run" but in the requirements have something like this: | 23:24 |
| - pipeline: | ||
| name: some-name | ||
| description: > | ||
| this will do review | ||
| require: | ||
| gerrit: | ||
| open: True | ||
| current-patchset: True | ||
| approval: | ||
| - Verified: 1 | ||
| - event: comment-added | ||
| comment: some-comment | ||
| @fungicide:matrix.org | as an example, this project uses a label called ptl-approved which gets set by a job run in another pipeline, and then that additional label is included in the merge requirements for the project's acl: https://opendev.org/openstack/project-config/src/commit/bf11008/zuul.d/pipelines.yaml#L260-L283 | 23:25 |
| @fungicide:matrix.org | the other trick there is that it avoids triggering itself when zuul's account votes, in order to avoid an infinite loop | 23:27 |
| @plscosta:matrix.org | what you are saying is that the ptl-approved would be set in the first pipeline? correct? | 23:28 |
| if that is the case, can the success output be defined conditionally? | ||
| @fungicide:matrix.org | using votes in multiple labels, you can query merge requirements for results from different pipelines | 23:29 |
| @fungicide:matrix.org | so each pipeline votes in a separate label | 23:29 |
| @plscosta:matrix.org | just a clarification, the first pipeline can outcome 3 results (as it is now): | 23:30 |
| - +1 but needs human review | ||
| - -1 there is something wrong with the change and needs fixing | ||
| - +1 and cab be automatically approved | ||
| @plscosta:matrix.org | * just a clarification, the first pipeline can outcome 3 results (as it is now): | 23:31 |
| - +1 but needs human review | ||
| - -1 there is something wrong with the change and needs fixing | ||
| - +1 and can be automatically approved | ||
| @fungicide:matrix.org | and you're using the +1 can be automatically approved vote to trigger automatic approval, which works when your test pipeline has already voted, but not as a trigger later because later won't be a comment-added event with your "can be automatically approved" string | 23:33 |
| @fungicide:matrix.org | i'm suggesting it's more robust and easier to reason about if the "something needs fixing/nothing needs fixing" and "needs human review/does not need human review" are independent labels, because then you can make each one a trigger with the other as a requirement and they'll be able to run in parallel and work no matter which order they report in | 23:35 |
| @fungicide:matrix.org | the trinary condition you're describing is not something i can see an easy way of making work, though maybe you could do it with a larger vote range for the label like -1..+3 so your needs human review is +1 and can be automatically approved uses +2 with +3 being the final gate success prior to merging | 23:37 |
| @fungicide:matrix.org | though that assumes you're only using one pipeline for the -1/+1/+2 results | 23:38 |
| @fungicide:matrix.org | multiple pipelines voting in the same label is something we've always tried to avoid in the projects i work on since you can't distinguish between which pipeline left a given vote otherwise | 23:39 |
| @fungicide:matrix.org | well, i should say multiple pipelines voting in the same label prior to final verified vote at merge time | 23:40 |
| @plscosta:matrix.org | fungi: big thanks for the help, i will have a look into this tomorrow with a fresh pair of eyes and with your input in mind! | 23:42 |
| @fungicide:matrix.org | others here may also be aware of alternative ways i'm not seeing to solve the problem | 23:48 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!