| @tony.breeds:matrix.org | fungi: In #openstack-elections you suggested it *might* be possible to extract text from the job-logs and have zuul leave that as a comment can you point me at some docs on that? | 00:27 |
|---|---|---|
| I don't think it's as simple as: https://zuul-ci.org/docs/zuul/latest/config/job.html#attr-job.success-message | ||
| @jim:acmegating.com | tony.breeds: maybe fungi was thinking of https://zuul-ci.org/docs/zuul/latest/job-content.html#leaving-warnings or https://zuul-ci.org/docs/zuul/latest/job-content.html#leaving-file-comments | 00:45 |
| @tony.breeds:matrix.org | corvus: Thanks. I don't know if that's what he was thinking but I'm pretty sure either would work. Something fun to play with next month (after the election closes) | 03:05 |
| @flaper87:matrix.org | 👋 hey hey! Long time no talk! | 08:13 |
| @flaper87:matrix.org | Like reaaaaaaaaaaaaaaaaaaaaaaaally long time :P | 08:13 |
| @flaper87:matrix.org | Probably a dumb question but, as I approach Zuul again, I noticed that there are multiple ways to install it on Kubernetes now. There's the Operator and the Helm Charts. Is there a general recommendation/preference between this two? (I know what an operator and charts do, I'm mostly asking from a stability, contribution, and easy of maintenance perspective) | 08:14 |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: | 09:05 | |
| - [zuul/zuul] 872799: Cleanup some Python ResourceWarnings in the test suite https://review.opendev.org/c/zuul/zuul/+/872799 | ||
| - [zuul/zuul] 873043: Fix ResourceWarnings in inventory testing https://review.opendev.org/c/zuul/zuul/+/873043 | ||
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 871259: Zuul-Web: Filter input validation for "Change" https://review.opendev.org/c/zuul/zuul/+/871259 | 10:59 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: | 11:26 | |
| - [zuul/zuul] 873044: Fix ResourceWarnings in fingergw https://review.opendev.org/c/zuul/zuul/+/873044 | ||
| - [zuul/zuul] 873045: Fix ResourceWarnings in the executor server https://review.opendev.org/c/zuul/zuul/+/873045 | ||
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [zuul/zuul] 873047: Fix DeprecationWarning: ssl.PROTOCOL_TLS is deprecated https://review.opendev.org/c/zuul/zuul/+/873047 | 11:32 | |
| -@gerrit:opendev.org- Benedikt Löffler proposed: [zuul/nodepool] 802255: Add option for an upload script https://review.opendev.org/c/zuul/nodepool/+/802255 | 13:12 | |
| @fungicide:matrix.org | > <@flaper87:matrix.org> Probably a dumb question but, as I approach Zuul again, I noticed that there are multiple ways to install it on Kubernetes now. There's the Operator and the Helm Charts. Is there a general recommendation/preference between this two? (I know what an operator and charts do, I'm mostly asking from a stability, contribution, and easy of maintenance perspective) | 14:04 |
| we don't deploy ours on kubernetes, so i'll let someone who does answer the question. i just wanted to say that it's great to see you again! hope we get to catch up at a conference some day | ||
| @newbie23:matrix.org | Hi guys, we are using Zuul+GitHub, CI running on protected branches only. | 14:17 |
| Then we have a periodic pipeline (defined to trigger jobs every night) and one job attached to it, like this: | ||
| periodic: | ||
| jobs: | ||
| - do-build-job: | ||
| branches: | ||
| - master | ||
| - protected-branch-A | ||
| - protected-branch-B | ||
| The intent would be to have the job to be executed every night, one time for each protected branch. | ||
| However the job is only executed on the master branch. | ||
| Note that the lines above are only present within the master branch (protected-branch-A and protected-branch-B have been created before those lines were introduced). | ||
| Why isn't do-build-job executed for all the protected branch? | ||
| I feel like I did not correctly the meaning of the "branches" option and/or how a periodic pipeline interacts with jobs | ||
| @fungicide:matrix.org | > <@tony.breeds:matrix.org> corvus: Thanks. I don't know if that's what he was thinking but I'm pretty sure either would work. Something fun to play with next month (after the election closes) | 14:32 |
| correct, i was thinking about file comments, but i forgot we also added warnings and those would probably be a better option | ||
| @jim:acmegating.com | newbie23: one way to describe it is that those are not imperative instructions, they are matchers. you can't instruct zuul to run a job one one branch from another one. the simplest and most intuitive way to run a periodic job on multiple branches using in-repo config is to put `periodic: {jobs: [do-build-job]}}` (with no branch matchers) in zuul.yaml in each of the branches you want to run. | 14:34 |
| @fungicide:matrix.org | an alternative is to put the pipeline addition for that project in a separate single-branch project (for openstack's periodic stable branch jobs we do that with a common project-template applied to the projects where we want those scheduled) | 14:38 |
| @jim:acmegating.com | yep and in that case, the config as originally written would be appropriate | 14:39 |
| @jim:acmegating.com | * yep, the same holds for the branch matchers. or you could put the config as originally written in a config-project. | 14:40 |
| @newbie23:matrix.org | like here? | 14:42 |
| https://github.com/openstack/openstack-zuul-jobs/blob/7d79c95fe5fd5fd5400d68de882dd108a3483b15/zuul.d/project-templates.yaml#L2559 | ||
| @jim:acmegating.com | yep | 14:43 |
| @jim:acmegating.com | but skip the branch matchers if you can | 14:43 |
| @newbie23:matrix.org | does it have to be a config project or doesn't it matter? | 14:43 |
| @jim:acmegating.com | the behavior difference comes about because first zuul decides what jobs to run using the project stanza on each branch. when it decides what to run on branch-A it just looks at the project stanza on branch-A, not master. (it also consults project stanzas in config projects). once it has collected the jobs to run, it filters them down by matching branches. | 14:45 |
| @jim:acmegating.com | so the easiest and most clear way to do this is just to put the jobs you want to run on the branches you want them to run. that means that they automatically satisfy both conditions: 1) they are in a project stanza on that branch. 2) they match the branch. | 14:46 |
| @jim:acmegating.com | if you do anything else, you have to make sure you meet both criteria on your own, and it's a lot of extra thinking and typing. :) | 14:46 |
| @jim:acmegating.com | for documentation reference, see paragraph 2 of https://zuul-ci.org/docs/zuul/latest/config/project.html | 14:48 |
| @jim:acmegating.com | (the approach in the openstack example will cause zuul to consider running all of those jobs on all of the branches where the template appears, and then exclude the ones where the branches don't match) | 14:50 |
| @newbie23:matrix.org | Thanks! I will look into it! | 14:53 |
| So if I got it right, ideally we should just do as you suggested i.e. no explicit branch matchers and job attached to the periodic pipeline for each protected branch we care about. | ||
| Our current configuration doesn't work because we actually *do have* a protected branch protected-branch-A with no such configurations. | ||
| A different approach would do as you do for openstack-zuul-jobs. | ||
| Out of curiosity, why did you use that approach for openstack-zuul-jobs instead? | ||
| Any advantages? | ||
| @flaper87:matrix.org | YAY! | 14:54 |
| @flaper87:matrix.org | Indeed, let's hope to meet at some conference in the near future 🙏 Maybe I get to go to some opendev conf soon | 14:54 |
| @jim:acmegating.com | newbie23: maybe fungi can elaborate/confirm, but i believe it was chosen because it allows a different team (the stable maintenance team) to more easily manage that template centrally | 14:54 |
| @newbie23:matrix.org | That makes sense: thanks! | 15:26 |
| @elpell:matrix.org | Hello! I have a question regarding the load-branch attribute of a config project. We have set up the a mirror of the zuul-jobs repo and use the load-branch attribute to load a stable version. However, jobs in our CI seem to fetch from the master branch of our mirror repository. One way to solve this is to add job.require-project and use override-checkout to point to the stable branch. I thought this should not be necessary when using load-branch or is this the intended way of working? | 15:30 |
| @clarkb:matrix.org | Per Wiklund: load-branch is a config project attribute to specify the loan valid branch on a config project. I wonder if this is a corner case in zuul's branch matching system finding the other branch because we don't expect additional branches on a config project? | 15:51 |
| @fungicide:matrix.org | > <@newbie23:matrix.org> That makes sense: thanks! | 16:01 |
| yes, exactly, the corpus of job configuration for openstack is so massive that we have different groups of reviewers responsible for reviewing the trusted config repo and try to limit the blast radius for mistakes there for safety. but also, the more things we can put outside the trusted config repo, the easier it is to test configuration changes pre-merge and make sure they're safe. to that end, we essentially have a three-tier configuration approach with the minimal necessary things in central trusted config repos, then things which benefit from centralized control but aren't security-sensitive in common single-branch untrusted repos, followed by project-specific configuration directly on the branches of individual projects | ||
| @fungicide:matrix.org | that way the infrastructure admins, the quality assurance group, and the subproject maintainers (and possibly even per-branch maintainers in some cases) all have control over their own configuration domains at the different levels of that hierarchy | 16:04 |
| @clarkb:matrix.org | corvus: Looking at my warning cleanup stack, it took a couple of rechecks to land. Before I attempt to look at those failures is the bug fix you referenced yesterday possibly related to flaky tests too? | 16:16 |
| @jim:acmegating.com | Clark: unlikely, that one is predictable | 16:19 |
| @elpell:matrix.org | > <@clarkb:matrix.org> Per Wiklund: load-branch is a config project attribute to specify the loan valid branch on a config project. I wonder if this is a corner case in zuul's branch matching system finding the other branch because we don't expect additional branches on a config project? | 16:19 |
| I was reading up on the idea behind it here: | ||
| https://review.opendev.org/c/zuul/zuul/+/705664 | ||
| And then it seemed to me that at least one branch could be expected in a config project, but maybe I am misinterpreting? We also instead of updating the stable branch created a new stable branch like "stable-1.1". Could this cause confusion during the branch matching? | ||
| @clarkb:matrix.org | Ya I'm not sure if tobiash is using it that way. We definitely only have a single branch (but also use the default of master and don't override the load branch) | 16:21 |
| @tobias.henkel:matrix.org | The idea behind this was that we can select a different branch for the config repo within a staging tenant. But it should behave just the same as the default branch with regards to branch matching. aka this means that the config repo still has only one branch that's considered in zuul | 16:24 |
| @jim:acmegating.com | keep in mand that what zuul checks out on disk for use in the job is different than what is used for job configuration (and can even be different from what zuul uses to run playbooks and roles) | 16:25 |
| @jim:acmegating.com | Per Wiklund: it checked out the master branch in the job because that was either the branch of the change or there was no matching branch for the change and that was the default branch. if you want to change the default branch for the project, use https://zuul-ci.org/docs/zuul/latest/config/project.html#attr-project.default-branch | 16:26 |
| @elpell:matrix.org | > <@tobias.henkel:matrix.org> The idea behind this was that we can select a different branch for the config repo within a staging tenant. But it should behave just the same as the default branch with regards to branch matching. aka this means that the config repo still has only one branch that's considered in zuul | 16:27 |
| Aha okey, then I did not understand this properly. And it does then make sense why it works like it does. | ||
| @jim:acmegating.com | * keep in mind that what zuul checks out on disk for use in the job is different than what is used for job configuration (and can even be different from what zuul uses to run playbooks and roles) | 16:28 |
| @jim:acmegating.com | Per Wiklund: the decisions on branch checkout are described in excruciating detail in the executor log | 16:29 |
| @elpell:matrix.org | > <@jim:acmegating.com> keep in mind that what zuul checks out on disk for use in the job is different than what is used for job configuration (and can even be different from what zuul uses to run playbooks and roles) | 16:29 |
| I did look at the default branch as well, thinking that could be my way out. Sadly no though. It seems like setting the default branch to my stable version also resulted in master being checked out and used. | ||
| @jim:acmegating.com | then i'd check the logs to see what the decision making is | 16:30 |
| @jim:acmegating.com | (but you didn't say what branch the change was on. if it's on master, that's your answer. and the right way to deal with that is override-checkout) | 16:34 |
| @elpell:matrix.org | > <@jim:acmegating.com> then i'd check the logs to see what the decision making is | 16:34 |
| Yes I was actually checking those logs to understand what was going on and that is how i landed in | ||
| https://zuul-ci.org/docs/zuul/latest/config/job.html#attr-job.required-projects.override-checkout | ||
| @elpell:matrix.org | > <@jim:acmegating.com> (but you didn't say what branch the change was on. if it's on master, that's your answer. and the right way to deal with that is override-checkout) | 16:37 |
| Yea sorry I forgot. And yes my change was on master. | ||
| @jim:acmegating.com | mystery solved. zuul checks out branches of the change under test unless you tell it otherwise. | 16:38 |
| @elpell:matrix.org | Yepp very nice! Thank you all for helping me with this! :) | 16:40 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 871863: Add an !unsafe change_message variable https://review.opendev.org/c/zuul/zuul/+/871863 | 16:45 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 871863: Add an !unsafe change_message variable https://review.opendev.org/c/zuul/zuul/+/871863 | 17:08 | |
| @clarkb:matrix.org | corvus: I think the last big python warnings are due to GitPython forking git processes that don't always cleanup quickly, and cherrypy not immediately shutting down its tcp socket when you ask it to exit (the exit call appears to be an event that is processed asynchronously). I'm not sure how much effort should be spent on fixing those. But if we did fix them we could theoretically make resource warnings errors. | 17:17 |
| @jim:acmegating.com | Clark: there's probably something equivalent to join for cherrypy. i wonder about gitpython though. is that something where we'd have to rely on gitpython fixing that and staying clean? or should we look to see if there's something there we can join as well? | 17:29 |
| @jim:acmegating.com | (also, is explicit object deletion an option for gitpython?) | 17:29 |
| @clarkb:matrix.org | re GitPython their readme says this is a known issue with the library and the maintainers aren't necessarily invested in trying to fix all of it. But it does sound like they would accept bug fixes from others. | 17:36 |
| @clarkb:matrix.org | Re cherrypy I'll have to look for tha | 17:37 |
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/zuul] 872519: Replace use of ProjectContext with SourceContext https://review.opendev.org/c/zuul/zuul/+/872519 | 18:05 | |
| -@gerrit:opendev.org- Ade Lee proposed: [zuul/zuul-jobs] 866881: Add ubuntu to enable-fips role https://review.opendev.org/c/zuul/zuul-jobs/+/866881 | 18:08 | |
| -@gerrit:opendev.org- Jeremy Stanley https://matrix.to/#/@fungicide:matrix.org proposed on behalf of Ade Lee: [zuul/zuul-jobs] 866881: Add ubuntu to enable-fips role https://review.opendev.org/c/zuul/zuul-jobs/+/866881 | 19:02 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Ade Lee: [zuul/zuul-jobs] 866881: Add ubuntu to enable-fips role https://review.opendev.org/c/zuul/zuul-jobs/+/866881 | 19:11 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 873037: Add OpenStack volume quota https://review.opendev.org/c/zuul/nodepool/+/873037 | 19:21 | |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 871671: Refresh dependencies for changes in pipelines https://review.opendev.org/c/zuul/zuul/+/871671 | 22:24 | |
| @clarkb:matrix.org | corvus: fwiw cheroot under cherrypy seems to be calling socket.close() from what I can see. But somehow python is losing track of the socket and thinks it has leaked? | 23:25 |
| @clarkb:matrix.org | maybe because one thread opens it and another closes it? I haven't verified this theory yet | 23:25 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!