| *** Goneri has quit IRC | 03:04 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #zuul | 04:33 | |
| *** bhavikdbavishi has joined #zuul | 05:54 | |
| *** bhavikdbavishi has quit IRC | 06:14 | |
| *** maxamillion has quit IRC | 07:24 | |
| *** evgenyl_ has quit IRC | 07:27 | |
| *** maxamillion has joined #zuul | 07:37 | |
| *** tosky has joined #zuul | 08:04 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Fix empty package, missing requirement https://review.opendev.org/751576 | 09:35 |
|---|---|---|
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Add zuul-client to requirements https://review.opendev.org/750196 | 09:36 |
| *** zenkuro has joined #zuul | 10:18 | |
| *** sassyn has joined #zuul | 11:59 | |
| sassyn | hi all | 11:59 |
| sassyn | can someone tell me how to I increase the worker in gearman? for the merger:cat ? | 12:00 |
| sassyn | I have added a gir repo - and it took almost 5 hours to scan it. | 12:00 |
| sassyn | what is merger:cat ? | 12:00 |
| sassyn | thanks | 12:00 |
| fungi | sassyn: "cat" is short for "concatenate" like the `cat` shell command... when zuul starts, it needs to find all of its job configuration, and since configs can be in any branch of any repository it attempts to speed up that operation by distributing the work across all the mergers. a cat job will be added to the gearman queue for each configured repository and the mergers will try to process those and return | 12:25 |
| fungi | any configuration they find back to the scheduler | 12:25 |
| fungi | note that when you initially add a new repository, the merger processing its cat job will need to clone it and checkout all branches from the origin remote, so if it's an especially large repository that could take some time | 12:27 |
| fungi | if you have lots of repositories you can speed that up by running additional mergers so that more cat jobs can be handled in parallel | 12:30 |
| fungi | but if it's just one huge repository which takes a long time to clone and scan, the only workaround is probably to pre-clone it manually on your executors/mergers and checkout all the branches so that when you restart the delta will be much smaller | 12:31 |
| fungi | there are also ways to mark certain repositories in the tenant configuration so that zuul won't try to find configs in them, if you know you won't be using in-repository configuration for those | 12:34 |
| fungi | and some connection drivers are able to skip specific branches, for example if you only want protected branches checked for configs in github | 12:36 |
| sassyn | Thank you fungi | 13:59 |
| sassyn | How all is well at your side. | 13:59 |
| sassyn | We start to communicate -5 months ago when all of this corona started | 14:00 |
| sassyn | and it still going on | 14:00 |
| sassyn | :-( | 14:00 |
| fungi | things are okay where i am. i just stay home. i live in a popular tourist area so it's better i just avoid everywhere | 14:01 |
| fungi | cooking a lot, doing more vegetable gardening | 14:01 |
| fungi | are things okay for you? | 14:01 |
| sassyn | I'm OK. working a lot also from home. | 14:02 |
| sassyn | Gain some weight | 14:02 |
| sassyn | which is not good | 14:02 |
| sassyn | cause there are days I almost not moving from house | 14:02 |
| sassyn | started now to swim | 14:03 |
| sassyn | I'm in TLV Israel | 14:03 |
| sassyn | is there any tourist going on? | 14:03 |
| fungi | i use a non-powered lawn mower, so that's mostly the extent of my outdoor exercise | 14:03 |
| sassyn | hmm | 14:03 |
| sassyn | my dream to have a yard | 14:04 |
| fungi | oh, yes, everyone here seems to want to be at the beach since schools and many jobs are remote right now | 14:04 |
| sassyn | that's cool | 14:04 |
| sassyn | where are u located if i May ask? | 14:04 |
| sassyn | I don't remember | 14:04 |
| fungi | a small island off the central atlantic coast of the usa | 14:04 |
| sassyn | sounds cool | 14:04 |
| sassyn | I have one more quick question. I have add an exciting gerrit repo into zuul. the repo have many branches and I got you answer about the merger:cat. However in this repo I have create my own branch let's call it zuul-branch and create there the .zuul.d directory. I want only this branch to be in zuul. I guess this is possible. | 14:07 |
| sassyn | But when I did that other branches also started to be trigger. so maybe you could give me a tip? | 14:08 |
| sassyn | existing* gerrit repo... | 14:08 |
| sassyn | this is some repo we starting to move off the jenkins.. | 14:09 |
| fungi | i think we've previously discussed branch filters for config finding, but i don't recall where that conversation ended up. i'll see if i can find whether we added any feature for it or if there are any proposed patches to implement that | 14:09 |
| sassyn | So basically what you are saying is that not supported out of the box? I was thinking only the branches that have the zuul.yaml will be included in the zuul. I can see in the log files that once zuul starting up he goes over the un-trusted project | 14:11 |
| sassyn | and looking for the .zuul.d zuu.yaml etc.... | 14:11 |
| sassyn | my understanding that only this branch will take part in zuul | 14:11 |
| fungi | branches with no configuration inherit their configuration from the default branch, normally | 14:12 |
| sassyn | the default branch I assume is master. But in my setup I have repo X, with clean zuul configuration on the master branch, and a secound branch name zull-branch, where only in this branch I have the zuul.yaml config files. | 14:14 |
| fungi | i believe zuul works out the default branch from whatever the remote sets as HEAD, which is often (but not always) named "master" yes | 14:17 |
| sassyn | but why zuul kick in when I push patches to the master branch | 14:18 |
| sassyn | is it suppose to work like this? | 14:18 |
| fungi | i don't entirely understand the question, but normally (unless you tell it otherwise) zuul watches for events for all your branches in every repository you tell it about | 14:19 |
| fungi | trusted config projects are handled a little differently, but for untrusted projects where you're allowed to do speculative testing of new job configuration, this is the applicable section of the docs: https://zuul-ci.org/docs/zuul/reference/tenants.html#attr-tenant.untrusted-projects | 14:20 |
| fungi | "...Zuul will read configuration files in all of their branches." | 14:21 |
| fungi | as mentioned earlier, there's the "exclude-unprotected-branches" option for projects reached through a github connection, that allows you to have zuul skip looking for configuration in (and events for, i suppose) branches which aren't marked as "protected" | 14:22 |
| fungi | but that's a github-only feature, meant to deal with some github-specific challenges, so not applicable to gerrit | 14:23 |
| sassyn | thank you for your patience and support here. I will try to rephrase the question: I have a gerrit repo called Simpsons. Repo Simpsons have 10 branches. The HEAD is the master branch. All these branches have no zuul configuration (meaning no yaml files in the folder). Repo Simpsons is configure as untrusted project. Now i have added a new branch. | 14:24 |
| sassyn | Will call it branch "zuul-branch" where I add the .zuul.yaml file on it. I was expecting that only this branch will start to work and would be trigger with zuul. | 14:24 |
| fungi | zuul will load the configuration you add in .zuul.yaml on zuul-branch and apply that configuration to any events specific to zuul-branch, but it will also check all the other branches to see if they contain zuul configuration, and if they don't then events for those branches will only trigger whatever behaviors you have configured globally (for example in your trusted config project) | 14:26 |
| sassyn | OK not I understand. | 14:28 |
| sassyn | Got u! | 14:28 |
| sassyn | Thanks you! | 14:28 |
| sassyn | may I ask what u where meaning here: "i think we've previously discussed branch filters for config finding, but i don't recall where that conversation ended up. i'll see if i can find whether we added any feature for it or if there are any proposed patches to implement that" | 14:29 |
| sassyn | NOW I UNDERSTAND :-) | 14:30 |
| fungi | yeah, at least once we've debated the possibility of an option to tell zuul to only look for configuration in specific branches of untrusted projects | 14:33 |
| fungi | but i don't remember whether we decided if that was a good idea. i'm not finding any current feature to allow it, nor do i see any pending changes in under review which might implement it | 14:34 |
| fungi | that's specific to the process of zuul looking for configuration to load, though. you can definitely already tell zuul to only trigger jobs on events for specific branches, you just can't currently tell it not to check specific branches for additional in-repository configuration in untrusted projects (or to only look in specific branches of untrusted projects for configuration) | 14:36 |
| fungi | you can also tell zuul not to load configuration from the entirety of an untrusted project, or what kinds of configuration to use from an untrusted project | 14:37 |
| fungi | just not on a branch-by-branch basis | 14:37 |
| sassyn | you can definitely already tell zuul to only trigger jobs on events for specific branches - how ? | 14:43 |
| sassyn | and also how to tell zuul not to load configuration from the entirety of an untrusted project, | 14:44 |
| openstackgerrit | Merged zuul/zuul master: Correct visibility check in web JS https://review.opendev.org/751425 | 14:48 |
| sassyn | you can definitely already tell zuul to only trigger jobs on events for specific branches - you mean set the repo in the untrusted project, and then do in the pipeline the magic? or u have better idea? | 14:53 |
| fungi | when you add projects to a tenant you can specify what kinds of configuration to load from them (jobs, roles, et cetera) and if you make that an empty list then you load no configuration at all from them | 15:19 |
| fungi | sassyn: here's an example https://opendev.org/openstack/project-config/src/branch/master/zuul/main.yaml#L19-L30 | 15:20 |
| fungi | sassyn: as for only triggering jobs on events for specific branches, that's what branch filters do https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.branches | 15:23 |
| *** tosky has quit IRC | 15:37 | |
| sassyn | so in my case if I put on the job the branch: "zuul-branch" | 15:41 |
| sassyn | then the job will only run of there is triggers in that branch | 15:42 |
| fungi | yes, from the docs there, "If the project has more than one branch, the branch containing the job definition is used as an implied branch specifier." | 15:43 |
| sassyn | other trigger might come to zuul, but since the job is limited to this branch only patch to this branch will run in the pipeline | 15:43 |
| *** Goneri has joined #zuul | 15:43 | |
| sassyn | OK understand | 15:43 |
| sassyn | again thanks u | 15:43 |
| fungi | so the jobs configured in you hypothetical zuul-branch branch should only be triggered for events involving that branch and not other branches | 15:44 |
| sassyn | perfect ! | 15:44 |
| fungi | you're welcome! | 15:44 |
| sassyn | I will remove the base job I have in the trusted-projec | 15:45 |
| fungi | well, you don't need to remove jobs from the trusted project as long as you don't configure anything to run them | 15:45 |
| fungi | also i think your base job has to be in a trusted config project | 15:46 |
| sassyn | in the https://opendev.org/zuul/project-config/src/branch/master/zuul.d/pipelines.yaml line | 15:46 |
| sassyn | 86 | 15:46 |
| fungi | you can just inherit from the base job in the jobs you define in your zuul-branch branch of the untrusted project, and not run the base job itself directly | 15:46 |
| fungi | you can also prevent anyone from attempting to run the base job directly by setting it as an abstract job | 15:46 |
| sassyn | WoW ! | 15:47 |
| sassyn | How can u explain - not fair u know everything | 15:47 |
| fungi | (also note that you don't need to explicitly inherit from base, if a job doesn't specify a parent it automatically inherits base) | 15:47 |
| sassyn | tenant.untrusted-projects.<project>.include if it's empty what does it mean. sorry I just can't understand it. it mean I don't load any config from the trust-project? | 16:01 |
| sassyn | I can still load it from the project itself only? | 16:02 |
| fungi | if tenant.untrusted-projects.<project>.include is empty, then zuul can still trigger jobs for events involving those projects but it won't read any configuration from them | 16:03 |
| sassyn | so only the config from the trusted project will be applied | 16:06 |
| sassyn | right? | 16:06 |
| sassyn | if tenant.untrusted-projects.<project>.include is empty | 16:07 |
| fungi | correct | 16:13 |
| sassyn | perfect ! | 16:14 |
| sassyn | thank u ! again | 16:14 |
| fungi | you're welcome | 16:14 |
| sassyn | when can I buy u a beer :-) | 16:14 |
| sassyn | I have one more question if I may | 16:14 |
| fungi | appreciated, but i have plenty of beer already ;) | 16:14 |
| sassyn | You said: "well, you don't need to remove jobs from the trusted project as long as you don't configure anything to run them | 16:14 |
| sassyn | https://pastebin.pl/view/845bd1a5 | 16:15 |
| sassyn | this is what I have | 16:15 |
| fungi | yes, not all jobs are run directly, some are only used to specify values which are inherited in other jobs (the base job is often treated this way, for example) | 16:15 |
| sassyn | than u said : you can just inherit from the base job in the jobs you define in your zuul-branch branch of the untrusted project, and not run the base job itself directly | 16:16 |
| sassyn | So I'm not sure I followed | 16:16 |
| sassyn | https://pastebin.pl/view/aef5ca9b this is the zuul-branch job | 16:17 |
| fungi | the base job in most deployments i'm familiar with isn't something you would ever actually run, it's typically treated as an abstract job (whether or not it is actually set to abstract), and serves as the parent job for any jobs which don't specify a parent | 16:17 |
| fungi | it serves as a place to put things you want included in every job in your system | 16:18 |
| fungi | maybe log collection and publishing, for example | 16:18 |
| fungi | or ssh key cleanup | 16:18 |
| sassyn | this is what I have | 16:18 |
| sassyn | https://pastebin.pl/view/845bd1a5 | 16:19 |
| fungi | your testjob there inherits implicitly from your base job | 16:19 |
| fungi | i need to step away for a few minutes, but i'll be back shortly | 16:19 |
| sassyn | Sure! | 16:20 |
| sassyn | I think I got it. To summaries: Since I have on the tenant configuration the Simpsons repo configued as an untrusted-project, when zuul comes up zuul will search (mergre:cat) for the .zuul configuration on all branches. | 16:44 |
| sassyn | the trusted-project, and also if .zuul configuration exists will run the explict job (even if the event is not to this specific branch. In other words: if the .zuul file is in the zuul-branch of simpsons, but the commited was to a branch name lisa, the explict jobs will still run). | 16:44 |
| sassyn | run if the event was on this branch. the base jobs will still run for other events but will not include this explict job. | 16:44 |
| sassyn | job.abstract | 16:46 |
| fungi | actually only the default branch of trusted projects is checked for configuration, not all branches of a trusted config project (but most of the time people just don't branch their trusted config projects anyway) | 16:56 |
| sassyn | not sure about "If however I tell in the job the branch name, the explict job will run if the event was on this branch. the base jobs will still run for other events but will not include this explict job." I think the explict job and the base jobs will run only when the event is for that branch since there is inherited | 16:56 |
| fungi | it's untrusted projects (where zuul will allow configuration to be applied speculatively) that get all their branches checked for configuration | 16:57 |
| fungi | normally the base job is not "run" for anything. you don't usually add your base job to any pipelines | 16:58 |
| fungi | it's just a convenient place for you to specify things you want all your other jobs to inherit | 16:58 |
| sassyn | i understand until now. | 16:59 |
| sassyn | but if I don't apply job branch, what will happend? Zuul will trigger the base job that match the pipeline correct? | 17:01 |
| sassyn | there will be for all branches in the repo. and for the branch zuul-branch zuull will run the base jobs + the explicit one | 17:02 |
| sassyn | sorry I'm driven you crazy. if it is too much I will try myself. | 17:03 |
| sassyn | I just want to have a untrusted-project where only event from it run the job define and not any other event on the repo. | 17:04 |
| sassyn | for a specific branch | 17:04 |
| sassyn | only. | 17:05 |
| fungi | zuul only runs jobs which you add to a pipeline | 17:10 |
| fungi | you don't normally add the base job to any pipelines | 17:10 |
| fungi | you can also prevent untrusted projects from adding the base job to their project-pipelines by setting it as an "abstract" job (one which is used exclusively for inheritance) | 17:10 |
| fungi | it might be useful to look at an example... here is the inheritance path from the build info for the "zuul-quick-start" job we run as a general integration exercise of changes for zuul: https://zuul.opendev.org/t/zuul/build/859d237e14414add967749c1f84e3a67/log/zuul-info/inventory.yaml#81-85 | 17:11 |
| sassyn | OK I will dig in and read more before asking stupid question | 17:15 |
| sassyn | just want to avoid the gate for my repo - only if it comes from the specific branch | 17:15 |
| fungi | reading it from bottom to top, you can see that the job we ran for that build was requested by the addition of the job in the project definition here: https://zuul.opendev.org/t/zuul/build/859d237e14414add967749c1f84e3a67/log/zuul-info/inventory.yaml#81-85 (the line number it references is actually the start of the project definition) | 17:15 |
| fungi | the next line up in the inheritance path is for the job definition to which it referred: https://opendev.org/zuul/zuul/src/branch/master/.zuul.yaml#L119 | 17:16 |
| sassyn | interesting | 17:17 |
| sassyn | very ! thank u again | 17:18 |
| sassyn | I need to send u bottle of wine | 17:18 |
| fungi | you can see that job declares opendev-buildset-registry-consumer as its parent, and indeed the next line up in the inheritance path points to https://opendev.org/opendev/base-jobs/src/branch/master/zuul.d/jobs.yaml#L223 | 17:18 |
| fungi | that job definition does not specify any parent | 17:18 |
| fungi | and as you'll notice the next line up in the inheritance path (the top line) is for opendev's base job definition: https://opendev.org/opendev/base-jobs/src/branch/master/zuul.d/jobs.yaml#L25 | 17:19 |
| sassyn | I see | 17:20 |
| fungi | so that zuul-quick-start build inherited job configuration from its opendev-buildset-registry-consumer (declared as a parent) which in turn inherited from the unspecified "base" job | 17:20 |
| fungi | zuul-quick-start is what we put in the gate pipeline though, we didn't directly run the base job | 17:20 |
| sassyn | OK, so i need to set the trust-project base jobs as job.abstract | 17:21 |
| sassyn | and then follow this | 17:21 |
| fungi | you don't *have* to set it as abstract, but you can as an extra safety measure so that nobody can explicitly add it to a pipeline | 17:22 |
| sassyn | i'm asking myself if this will give me what I need - cause as said I want to run the gate only if the event goes to a specific branch | 17:22 |
| fungi | however, zuul will only ever run jobs which are added to a pipeline (either directly included in the project-pipeline within that untrusted project's default branch or the branch for the event which was triggered, or in a trusted config project pipeline listing that untrusted project), or indirectly by including a project-template which contains that job | 17:23 |
| sassyn | thank u very very much again! really! I mean it! thank U. I will do my homework and probably will ping u again. not sure I got it all. but you help allot! | 17:25 |
| fungi | if you add the job to a project-pipeline within a non-default branch, then it should only ever run from events for that branch (we call that an "implied branch matcher"). if you add it in a project-pipeline definition on that untrusted project's default branch, you can specify that you only want it to run on a particular branch (we call that an "explicit branch matcher" or just a "branch matcher") | 17:25 |
| fungi | you can also add the job to the pipeline declaration for that untrusted project within a trusted config project and use an explicit branch matcher there too, just be aware that changes to things like that in your trusted config project are not run speculatively, for safety reasons, so will only take effect once they merge to the trusted config project | 17:26 |
| sassyn | too much information | 17:28 |
| fungi | and you're welcome! i'm happy to answer questions when i can | 17:28 |
| sassyn | if you add the job to a project-pipeline within a non-default branch, then it should only ever run from events for that branch (we call that an "implied branch matcher"). if you add it in a project-pipeline definition on that untrusted project's default branch, you can specify that you only want it to run on a particular branch (we call that an | 17:28 |
| sassyn | "explicit branch matcher" or just a "branch matcher") | 17:28 |
| sassyn | HOW? | 17:28 |
| fungi | https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.branches | 17:29 |
| sassyn | I mean the gate will run always, only the job will run for the specific branch | 17:29 |
| fungi | you can specify branch matchers in the job like that, *or*... | 17:29 |
| fungi | you can do it in a job variant within a pipeline or project-pipeline definition, i'm looking for the right place in the docs for that now | 17:31 |
| fungi | ahh, we don't call it out exactly but https://zuul-ci.org/docs/zuul/reference/project_def.html#attr-project.%3Cpipeline%3E.jobs does say "it may be a dictionary, in which case it is treated as a job variant local to this project and pipeline. In that case, the format of the dictionary is the same as the top level job definition. Any attributes set on the job here will override previous versions of the job." | 17:33 |
| fungi | or to restate, you can override parts of job definitions within a variant where you're adding it to the pipeline, and that includes branch matchers, for example: https://opendev.org/openstack/project-config/src/branch/master/zuul.d/projects.yaml#L41 | 17:35 |
| sassyn | I follow you, but still something is missing.... I have a repo and the repo get events. it is config as untrust project. the pipeline gets events and I understand that if I configure in the job the branch then this job will run only if the vent was on this branch. But still event not for this branch will get into zuul. and wil try to do the gate. | 17:42 |
| sassyn | and this is what I to avoid | 17:42 |
| clarkb | if all of the jobs have branch matchers they will only run against thosebranches | 17:44 |
| clarkb | this means the gate will not run for other branches | 17:44 |
| sassyn | now I understand! this was the missing thing | 17:44 |
| sassyn | so just add the branch to the job.. and then the job will run only when event come from this branch. simple as that | 17:45 |
| sassyn | if job run and pass - gate is done. | 17:46 |
| clarkb | yup | 17:48 |
| sassyn | fungi and clarkb - u are my angel for this week. thank thank thank. | 17:53 |
| sassyn | you!!!!! | 17:53 |
| fungi | my pleasure! | 17:53 |
| sassyn | have a nice weekend! | 17:53 |
| fungi | you too | 17:53 |
| fungi | now i need to press out these pitas and get them into the oven to bake | 17:54 |
| sassyn | I mean it when i say I will send u some wine! | 17:54 |
| sassyn | :-) | 17:54 |
| sassyn | I was in master chef few years ago | 17:54 |
| sassyn | I'm cooking very very good | 17:54 |
| sassyn | if u ever come to IL | 17:54 |
| sassyn | I will take u to nice places | 17:54 |
| fungi | it's a deal! there are (or at least were) some great tech conferences there i've never actually gotten to | 18:03 |
| fungi | hopefully travel will eventually become less of a chore again | 18:08 |
| *** sassyn has quit IRC | 18:10 | |
| *** rfolco has joined #zuul | 20:08 | |
| *** rfolco has quit IRC | 20:12 | |
| *** holser has quit IRC | 21:23 | |
| *** holser has joined #zuul | 21:25 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!