| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 00:42 | |
| - [zuul/zuul] 898267: Add option to use in-container pg_dump in tests https://review.opendev.org/c/zuul/zuul/+/898267 | ||
| - [zuul/zuul] 898758: Fix sql serialization in old migration https://review.opendev.org/c/zuul/zuul/+/898758 | ||
| - [zuul/zuul] 897874: Refactor bundle in sql connection https://review.opendev.org/c/zuul/zuul/+/897874 | ||
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 895988: Add spec for refactoring circular dependencies https://review.opendev.org/c/zuul/zuul/+/895988 | 01:12 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: | 10:16 | |
| - [zuul/zuul] 897664: Send job parent data + artifacts via build request https://review.opendev.org/c/zuul/zuul/+/897664 | ||
| - [zuul/zuul] 898921: dnm: Test model API <18 compatibility https://review.opendev.org/c/zuul/zuul/+/898921 | ||
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 897664: Send job parent data + artifacts via build request https://review.opendev.org/c/zuul/zuul/+/897664 | 11:40 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 898921: dnm: Test model API <18 compatibility https://review.opendev.org/c/zuul/zuul/+/898921 | 11:41 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 897664: Send job parent data + artifacts via build request https://review.opendev.org/c/zuul/zuul/+/897664 | 13:05 | |
| @vlotorev:matrix.org | Hi, I'd like to add second tenant to zuul. | 17:25 |
|---|---|---|
| The question is how to allocate and restrict nodepool resources for new tenant. | ||
| My idea was to have two static drivers (or two pools in one static driver): | ||
| ``` | ||
| providers: | ||
| - name: static-nodes-for-tenant1 | ||
| driver: static | ||
| - name: static-nodes-for-tenant2 | ||
| driver: static | ||
| ... | ||
| ``` | ||
| But looking at tenant configuration it seems that resource allocation per 'provider' or 'pool' is not possible https://zuul-ci.org/docs/zuul/latest/tenants.html. | ||
| The only way to filter nodes per tenant is possible per labels. | ||
| This is a way to go but would mean zuul admin should assert labels are not overlapping in two tenants. | ||
| Am I missing any tenant/nodepool configuration? | ||
| @vlotorev:matrix.org | * Hi, I'd like to add second tenant to zuul. | 17:26 |
| The question is how to allocate and restrict nodepool resources for new tenant. | ||
| My idea was to have two static drivers (or two pools in one static driver): | ||
| ``` | ||
| providers: | ||
| - name: static-nodes-for-tenant1 | ||
| driver: static | ||
| - name: static-nodes-for-tenant2 | ||
| driver: static | ||
| ... | ||
| ``` | ||
| But looking at tenant configuration it seems that resource allocation per 'provider' or 'pool' is not possible https://zuul-ci.org/docs/zuul/latest/tenants.html. | ||
| The only way to filter nodes per tenant is possible per labels. | ||
| We could go splitting the resources per tenant with labels, but it would mean zuul admin should assert labels are not overlapping in two tenants. | ||
| Am I missing any tenant/nodepool configuration? | ||
| @vlotorev:matrix.org | * Hi, I'd like to add second tenant to zuul. | 17:27 |
| The question is how to allocate and restrict nodepool resources for new tenant. | ||
| My idea was to have two static drivers (or two pools in one static driver): | ||
| ``` | ||
| providers: | ||
| - name: static-nodes-for-tenant1 | ||
| driver: static | ||
| - name: static-nodes-for-tenant2 | ||
| driver: static | ||
| ... | ||
| ``` | ||
| But looking at tenant configuration it seems that resource allocation per 'provider' or 'pool' is not possible https://zuul-ci.org/docs/zuul/latest/tenants.html. | ||
| The only way to filter nodes per tenant is possible per labels. | ||
| We can split resources per tenant with labels, but it would mean zuul admin should assert labels are not overlapping in two tenants. Also it would means jobs defined in in-house zuul-jobs can be used without redefiniting nodesets. | ||
| Am I missing any tenant/nodepool configuration? | ||
| @vlotorev:matrix.org | * Hi, I'd like to add second tenant to zuul. | 17:28 |
| The question is how to allocate and restrict nodepool resources for new tenant. | ||
| My idea was to have two static drivers (or two pools in one static driver): | ||
| ``` | ||
| providers: | ||
| - name: static-nodes-for-tenant1 | ||
| driver: static | ||
| - name: static-nodes-for-tenant2 | ||
| driver: static | ||
| ... | ||
| ``` | ||
| But looking at tenant configuration it seems that resource allocation per 'provider' or 'pool' is not possible https://zuul-ci.org/docs/zuul/latest/tenants.html. | ||
| The only way to filter nodes per tenant is possible per labels. | ||
| We can split resources per tenant with labels, but it would mean zuul admin should assert labels are not overlapping in two tenants. Also it would mean jobs defined in in-house zuul-jobs can't be used without redefiniting jobs' nodesets. | ||
| Am I missing any tenant/nodepool configuration? | ||
| @jamesharrisg:matrix.org | Has anyone found a way of handling zuul source integration with the AOSP repo tool? I'm having trouble conceptually bridging the gap between the repo tool's workspace and zuul's providing the ASOP source code as dependencies for jobs. Our current approach is to use repo to clone and populate the workspace and basically ignore or cp changes from the zuul provided directories into the relevant paths in the workspace. | 19:35 |
| This seems extremely wrong and I am not sure what kind of arrangement I would be looking at to work around it. | ||
| @jamesharrisg:matrix.org | Never mind I think I have arrived at a solution. | 20:56 |
| @clarkb:matrix.org | James Harris: I would be very very careful not using the git repos that zuul has prepared | 21:08 |
| @clarkb:matrix.org | having properly prepared repositories is very important for speculative gating and also provides additional info like where the non speculate head is currently at | 21:09 |
| @jamesharrisg:matrix.org | Here is my working idea: | 21:11 |
| the manifest repository is tracked in zuul and supplied as a dependency. I initalize the repo workspace by pointing the -u flag to the full /workspace/src/gerrit.server/manifest/.git path. This will give me the state of the branch and manifest I want. | ||
| @jamesharrisg:matrix.org | from there I will create a local_manifest/zuul.xml manifest that will contain overwritten references to the .git repositories of the other prepared repos. | 21:11 |
| @clarkb:matrix.org | we don't use the repo tool so I'm not familiar with the terms and flags. But I think what I would do is let zuul provide a workspace with git repos set up the way zuul wants them. Then have repo fetch from there. This should be quick as git will use hardlinks on a shared filesystem | 21:12 |
| @jamesharrisg:matrix.org | this is effectively what I am proposing. | 21:12 |
| @jamesharrisg:matrix.org | I assume the branch states from zuul's repos will include all of the relevant speculative patches. | 21:13 |
| @clarkb:matrix.org | > <@vlotorev:matrix.org> Hi, I'd like to add second tenant to zuul. | 21:13 |
| > The question is how to allocate and restrict nodepool resources for new tenant. | ||
| > My idea was to have two static drivers (or two pools in one static driver): | ||
| > | ||
| > ``` | ||
| > providers: | ||
| > - name: static-nodes-for-tenant1 | ||
| > driver: static | ||
| > | ||
| > - name: static-nodes-for-tenant2 | ||
| > driver: static | ||
| > ... | ||
| > ``` | ||
| > | ||
| > But looking at tenant configuration it seems that resource allocation per 'provider' or 'pool' is not possible https://zuul-ci.org/docs/zuul/latest/tenants.html. | ||
| > The only way to filter nodes per tenant is possible per labels. | ||
| > | ||
| > We can split resources per tenant with labels, but it would mean zuul admin should assert labels are not overlapping in two tenants. Also it would mean jobs defined in in-house zuul-jobs can't be used without redefiniting jobs' nodesets. | ||
| > | ||
| > Am I missing any tenant/nodepool configuration? | ||
| depending on the sort of restrictiosn you need tenant-resource-limits may be sufficient | ||
| @clarkb:matrix.org | > <@jamesharrisg:matrix.org> I assume the branch states from zuul's repos will include all of the relevant speculative patches. | 21:13 |
| correct | ||
| @jamesharrisg:matrix.org | i.e. if 3 patches are going in at once to the develoment/variant branch those three patches will be in development/variant on the git repo. | 21:13 |
| @jamesharrisg:matrix.org | the only solution that would be more ideal for me would be for zuul to more tightly integrate with repo (which is too big an ask in this case.) | 21:14 |
| @jamesharrisg:matrix.org | * the only solution that would be more ideal for me would be for zuul to more tightly integrate with repo (which is too big an ask to actually handle here.) | 21:15 |
| @clarkb:matrix.org | The workspace preparation roles are implemented in ansible and are replaceable | 21:17 |
| @jamesharrisg:matrix.org | really not sure how gating is supposed to work in this case when a bunch of random patches might come into many subprojects at once. the check pipeline uses build-on-topic to build the whole environment but the gate pipeline seems way more complex to get my mind around. | 21:17 |
| @clarkb:matrix.org | you could theoretically make a workspace preparation role that targets repo environments | 21:17 |
| @clarkb:matrix.org | I don't think anything in zuul itself will get in the way of that | 21:17 |
| @jamesharrisg:matrix.org | That is effectively what I am doing. | 21:18 |
| @jamesharrisg:matrix.org | I would probably want the executor to be able to directly track and setup the repo workspace the way it does multiple git projects currently. | 21:18 |
| @jamesharrisg:matrix.org | but that seems way more complicated. | 21:18 |
| @clarkb:matrix.org | > <@jamesharrisg:matrix.org> really not sure how gating is supposed to work in this case when a bunch of random patches might come into many subprojects at once. the check pipeline uses build-on-topic to build the whole environment but the gate pipeline seems way more complex to get my mind around. | 21:19 |
| zuul can support submit-whole-topic with Gerrit. There are plans to improve support for this here: https://zuul-ci.org/docs/zuul/latest/developer/specs/circular-dependencies-refactor.html | ||
| @clarkb:matrix.org | Logically it will treat multiple changes against a topic as a set of circular dependencies. It will figure out which jobs to dedup then run all the jobs. If all the jobs pass for all the changes it mergers all the changes. Note as with Gerrit there is the slight chance that something will fail to merge for $reason since the actual merge process across multiple git repos isn't atomic | 21:20 |
| @jamesharrisg:matrix.org | interesting. | 21:27 |
| @vlotorev:matrix.org | > <@clarkb:matrix.org> depending on the sort of restrictiosn you need tenant-resource-limits may be sufficient | 21:33 |
| tenant-resource-limits is only supported by AWS driver ( https://zuul-ci.org/docs/nodepool/latest/configuration.html#attr-tenant-resource-limits ) | ||
| @vlotorev:matrix.org | * In reply to @clarkb:matrix.org | 21:34 |
| depending on the sort of restrictiosn you need tenant-resource-limits may be sufficient | ||
| tenant-resource-limits is only supported by AWS driver ( https://zuul-ci.org/docs/nodepool/latest/configuration.html#attr-tenant-resource-limits ) | ||
| @clarkb:matrix.org | I think all of them support it. The AWS driver supports extra limitations beyond the core set | 21:34 |
| @jim:acmegating.com | James Harris: i spoke a bit about submit-whole-topic at the gerrit user summit; here's the talk: https://www.youtube.com/watch?v=-EsNWzajdNE | 21:38 |
| @jim:acmegating.com | James Harris: i think you've got the right general idea with repo. as Clark said, zuul wants to have complete control over the repo state, and doing something outside of that invalidates the guarantees it makes about what's being tested, so be careful :) | 21:39 |
| @vlotorev:matrix.org | > <@clarkb:matrix.org> I think all of them support it. The AWS driver supports extra limitations beyond the core set | 21:43 |
| Thanks, was reading the doc incorrectly. | ||
| @jim:acmegating.com | James Harris: another important concept to have front of mind is zuul's `required-projects`. and also semi-relevant is the python "siblings" roles in zuul-jobs. that's related in that it is performing setup of repos other than the one directly under test, so there are some concepts/ideas that may be useful there. | 21:44 |
| @jamesharrisg:matrix.org | I use required project for the manifest repo (where repo gets its manifest list.) I'll have to look at the "siblings" roles for python. | 21:46 |
| @jamesharrisg:matrix.org | > <@jim:acmegating.com> James Harris: i think you've got the right general idea with repo. as Clark said, zuul wants to have complete control over the repo state, and doing something outside of that invalidates the guarantees it makes about what's being tested, so be careful :) | 21:54 |
| yea. my best bet to make these play nice with each other is to get repo to use the zuul provided repositories as a source for those particular projects. Otherwise I get into fun triple maintenance problems where the manifest has to drive the zuul tennant configuration updates and the required-projects have to be automatically updated with manifest changes. that gets to be a bigger burden on my developers. | ||
| @jamesharrisg:matrix.org | tricking repo into pulling from the zuul provided repositories seems like the nicest way to get the speculative information into my workspace. | 21:56 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!