| @dpawlik:matrix.org | corvus: I will change the ensure-microshift role to use nested role as it was suggested by Clark and it was a main idea by Tengu | 07:20 |
|---|---|---|
| @dpawlik:matrix.org | thanks for help | 07:20 |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 871259: Zuul-Web: Filter input validation for "Change" https://review.opendev.org/c/zuul/zuul/+/871259 | 08:08 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 877454: wip: don't check branch cache when project not in min. ltimes https://review.opendev.org/c/zuul/zuul/+/877454 | 09:20 | |
| @g_gobi:matrix.org | Do we need "Auth Token" to run `zuul-admin dequeue` command? | 09:24 |
| @fungicide:matrix.org | tdlaw: i don't recall if `zuul-admin dequeue` uses the rest api (if it does you'll need a token), but be aware that subcommand is deprecated in favor of using zuul-client | 13:33 |
| @mhuin:matrix.org | yep you should use the zuul-client for this | 13:34 |
| @mhuin:matrix.org | generate a token with zuul-admin, then use it with zuul-client | 13:34 |
| @noonedeadpunk:matrix.org | Hey folks. I have a question on zuul usage. I'm trying to think on how to implement this workflow: | 13:35 |
| I have a projectA and projectB. projectB is a deliverable for projectA and specified in required-projects of projectA. projectA has a promote job and secrets for promoting. During promote I pack both projectA and projectB together and deploy it to production. | ||
| Question - is there any way to trigger this promote job on update of projectB without re-defining secrets? As eventually projectB is deliverable not only for projectA, but also for projectAB, projectAC, etc, each of them have different secrets ofc. | ||
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: | 13:37 | |
| - [zuul/nodepool] 872255: Output rate information in load-test.py https://review.opendev.org/c/zuul/nodepool/+/872255 | ||
| - [zuul/nodepool] 872714: Add debug log messages to handler assignment/removal https://review.opendev.org/c/zuul/nodepool/+/872714 | ||
| - [zuul/nodepool] 872256: Offload openstack delete api calls to an executor https://review.opendev.org/c/zuul/nodepool/+/872256 | ||
| @noonedeadpunk:matrix.org | As I got slightly lost on how to do that and if it's possible at all. I assume there should be some separate pipeline(s) and define some custom trigger driver or smth... | 13:37 |
| -@gerrit:opendev.org- Zuul merged on behalf of Tobias Henkel: [zuul/nodepool] 876881: Process paused requests first https://review.opendev.org/c/zuul/nodepool/+/876881 | 13:37 | |
| @fungicide:matrix.org | just to be clear, you have different secret data for each project, and you want different projects to be able to run jobs that use other projects' secret data? | 13:37 |
| @mhuin:matrix.org | And speaking of zuul-client and tokens, zuul-maint: please consider these improvements https://review.opendev.org/c/zuul/zuul-client/+/872967 (support for auth token as an env variable) and https://review.opendev.org/c/zuul/zuul-client/+/819118 (enable user + password login from the CLI if the identity provider allows it) | 13:37 |
| @fungicide:matrix.org | can you define all of that centrally in a trusted config project and add the job(s) to those projects' promote pipelines? | 13:38 |
| @noonedeadpunk:matrix.org | Well... Probably I can... Though I think I would prefer to have secrets encrypted by projects keys | 13:41 |
| @noonedeadpunk:matrix.org | So that leaking 1 key won't result in compromising all projects | 13:41 |
| @fungicide:matrix.org | secrets are encrypted for the key of whatever project they're defined in, which is the project the playbook/job directly using them is defined in | 13:42 |
| @fungicide:matrix.org | so if you put secrets in a trusted config project, you encrypt them for the key of the trusted config project | 13:42 |
| @noonedeadpunk:matrix.org | yeah, and now they're splitted in different repos, so they're encrypted with different keys, and they can't basically ask what they should not have access to | 13:43 |
| @fungicide:matrix.org | an exception is when using pass-to-parent | 13:43 |
| @fungicide:matrix.org | with pass-to-parent the secret is still encrypted for the key of the project it's defined in, but it can be used by a parent job's playbooks | 13:44 |
| @g_gobi:matrix.org | How to install zuul-client? @mhu fungi | 13:44 |
| @noonedeadpunk:matrix.org | well, pass-to-parent is an option | 13:44 |
| @noonedeadpunk:matrix.org | And I have a parent in trusted repo | 13:44 |
| @fungicide:matrix.org | tdlaw: you can pip install it into a venv, or i think there are container images on dockerhub, or possible your linux distro of choice has packaged it (though probably unlikely at this point) | 13:44 |
| @mhuin:matrix.org | @tdlaw: pip install zuul-client or you can pull the docker container, see the doc | 13:45 |
| @noonedeadpunk:matrix.org | But pass-to-parent is likely not an option here, as projectB can we child at best. | 13:46 |
| @noonedeadpunk:matrix.org | * But pass-to-parent is likely not an option here, as projectB can be child at best. | 13:46 |
| @fungicide:matrix.org | tdlaw: https://zuul-ci.org/docs/zuul-client/installation.html | 13:46 |
| @noonedeadpunk:matrix.org | I was likely thinking about some way to trigger job/pipeline from projectA when stuff lands to projectB, but not really defining promote job in projectB | 13:47 |
| @fungicide:matrix.org | Dmitriy Rabotyagov: usually you'd define the secret in an untrusted repo but then pass-to-parent in a job which has a parent in a trusted config repo that actually uses the secret (this is how git replication to github is implemented for some projects in the opendev collaboratory's zuul, for example) | 13:47 |
| @noonedeadpunk:matrix.org | That is exactly how I have things configured | 13:47 |
| @noonedeadpunk:matrix.org | I have common playbooks in trusted repo, and then secrets in untrusted and pass to parent secrets for actuall deployment not to duplicate code | 13:48 |
| @fungicide:matrix.org | Dmitriy Rabotyagov: we've talked about allowing events for a project to trigger jobs for another project, as an internal-to-zuul triggering event. opendev has some use cases for that with systems deployment for example, where we want a change merging in one repo to trigger jobs for another repo. i don't think that's been implemented yet though, probably still needs some design discussions | 13:49 |
| @noonedeadpunk:matrix.org | That is exactly what I was thinking about.... | 13:49 |
| @fungicide:matrix.org | there might be a lightweight workaround with a webhook-based connection driver and then have a post-merge job for one project call the webhook to trigger a pipeline for the other project, but that would probably still involve some development work to make the driver | 13:51 |
| @noonedeadpunk:matrix.org | I was thinking of leveraging mqtt or smtp but yeah, I was quite unsure how that's gonna work | 13:53 |
| @noonedeadpunk:matrix.org | And this can't be leveraged in any way for that? https://zuul-ci.org/docs/zuul/latest/drivers/zuul.html | 13:54 |
| @noonedeadpunk:matrix.org | Like - make pipeline that will send an event and another pipeline will listen for it..? | 14:02 |
| @noonedeadpunk:matrix.org | * Like - make job/pipeline that will send an event and another pipeline will listen for it in addition to gerrit? | 14:02 |
| @noonedeadpunk:matrix.org | But well, I guess for now then I will just issue `repromote` to the latest merged changed to projectA to get deliverables | 14:04 |
| @noonedeadpunk:matrix.org | And hope to get some time one day to contribute to this feature... | 14:04 |
| @fungicide:matrix.org | i think the mqtt and smtp drivers are only reporters, not source connection | 14:04 |
| @fungicide:matrix.org | oh, maybe the zuul-zuul driver is the answer, i probably missed when that got added | 14:05 |
| @noonedeadpunk:matrix.org | Yeah, I was just unsure here in zuul design in terms what it needs in drivers to do that | 14:05 |
| -@gerrit:opendev.org- Matthieu Huin https://matrix.to/#/@mhuin:matrix.org proposed: [zuul/zuul-client] 877481: Document the need for volumes when using client from a container https://review.opendev.org/c/zuul/zuul-client/+/877481 | 14:11 | |
| @mhuin:matrix.org | ^ That discussion above reminded me to document a little gotcha when using the containerized zuul-client | 14:12 |
| @fungicide:matrix.org | huh, git history says the zuul-zuul driver has existed at least since 2.5.0, so there must be some reason we weren't able to use it to do the things we were wanting to | 14:12 |
| -@gerrit:opendev.org- Matthieu Huin https://matrix.to/#/@mhuin:matrix.org proposed: [zuul/zuul-client] 872967: Support for ZUUL_TOKEN environment variable https://review.opendev.org/c/zuul/zuul-client/+/872967 | 14:25 | |
| @noonedeadpunk:matrix.org | > <@fungicide:matrix.org> huh, git history says the zuul-zuul driver has existed at least since 2.5.0, so there must be some reason we weren't able to use it to do the things we were wanting to | 14:26 |
| mhm, I see... | ||
| @noonedeadpunk:matrix.org | It actually looks like it's missing some part, but I wasn't digging down to the rabbit hole yet. As hope there's some ready recipe, as usecase sounds pretty much to what opendev indeed might want | 14:27 |
| @noonedeadpunk:matrix.org | * It actually looks like it's missing some part, but I wasn't digging down to the rabbit hole yet. As hoped there's some ready recipe, as usecase sounds pretty much to what opendev indeed might want | 14:28 |
| @jim:acmegating.com | opendev's issue isn't something that can be done with the zuul driver -- everything it needs could be done with a simple job defined in a 3rd repo and run by the 2 subject repos (with pass-to-parent as necessary). the issue blocking opendev is that some of the projects are in the wrong tenant because no one has spent the effort to move them. | 14:32 |
| @jim:acmegating.com | (and indeed for the limited set of projects that are in the same, correct tenant, there are some that are running that way) | 14:33 |
| @noonedeadpunk:matrix.org | ` a simple job defined in a 3rd repo and run by the 2 subject repos` -> not really | 14:36 |
| I have like 5 simmilar repos, with slightly different content though, each of them have their own secret for promoting (and destination for promoting). And I have a repo with deliverable, that should be packed with each of these 5 repos. So when these 5 do promote - they include this deliverable as have it mentioned in required-projects. But when deliverable is updated - there's kind of no way of triggering promote | ||
| @noonedeadpunk:matrix.org | Well, without placing secrets under trusted repo, but then it's pita and not very scalable | 14:36 |
| @noonedeadpunk:matrix.org | * Well, without placing secrets under trusted repo, but then it's a pita kindof | 14:36 |
| @jim:acmegating.com | > <@noonedeadpunk:matrix.org> ` a simple job defined in a 3rd repo and run by the 2 subject repos` -> not really | 14:37 |
| > | ||
| > I have like 5 simmilar repos, with slightly different content though, each of them have their own secret for promoting (and destination for promoting). And I have a repo with deliverable, that should be packed with each of these 5 repos. So when these 5 do promote - they include this deliverable as have it mentioned in required-projects. But when deliverable is updated - there's kind of no way of triggering promote | ||
| er, i was describing opendev | ||
| @noonedeadpunk:matrix.org | ah. sorry, yes | 14:37 |
| @noonedeadpunk:matrix.org | Would my issue be sorted out with zuul driver ? 🫠| 14:38 |
| @fungicide:matrix.org | corvus: oh! right that was the detail i was forgetting, we have things we wanted to trigger cross-tenant not just cross-project | 14:40 |
| @fungicide:matrix.org | or, as you say, find time to properly finish moving the projects into the new tenant | 14:40 |
| @jim:acmegating.com | right -- and, well, we don't really want them to be cross-tenant. we just need to not have the configuration for the production systems in the openstack tenant :) | 14:40 |
| @fungicide:matrix.org | exactly | 14:40 |
| @jim:acmegating.com | if you want zuul to run a job for a project, you use a project stanza to say that. so if you want zuul to run "promote-project-A" when changes merged to project B, you use a project stanza for project B that says "run promote-project-A". there are restrictions on this in the case of secrets, but that can be overridden if the project stanza is in a config project. | 14:46 |
| @jim:acmegating.com | you don't need an extra driver to pass the event around | 14:46 |
| @jim:acmegating.com | see https://zuul-ci.org/docs/zuul/latest/config/job.html#attr-job.allowed-projects paragraph 2 | 14:47 |
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/nodepool] 876943: Respect timeout when not accepting requests https://review.opendev.org/c/zuul/nodepool/+/876943 | 14:58 | |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: | 14:58 | |
| - [zuul/nodepool] 877044: Don't update node request in assign handlers loop https://review.opendev.org/c/zuul/nodepool/+/877044 | ||
| - [zuul/nodepool] 877045: Make assignHandlers a generator https://review.opendev.org/c/zuul/nodepool/+/877045 | ||
| @noonedeadpunk:matrix.org | `if the project stanza is in a config project` -> yeah, exactly, but that means all secrets for quite independant things will be encrypted with the same RSA, as they no longer will be split across projects but all be in a trusted one, which is quite meh | 15:13 |
| @noonedeadpunk:matrix.org | Oh, wait | 15:14 |
| @noonedeadpunk:matrix.org | So you're saying, that in trusted project I can define promote jobs for any project even given that secrets are stored not in trusted project& | 15:15 |
| @noonedeadpunk:matrix.org | Ok, that will work out then | 15:16 |
| @fungicide:matrix.org | yes, that's how the aforementioned github replication jobs are set up for some projects in opendev. the github credentials for the project are encoded in a secret for that project, but passed to a parent job definition in a trusted config repo | 15:17 |
| @noonedeadpunk:matrix.org | > <@fungicide:matrix.org> yes, that's how the aforementioned github replication jobs are set up for some projects in opendev. the github credentials for the project are encoded in a secret for that project, but passed to a parent job definition in a trusted config repo | 15:29 |
| Well, that's how basic pass-to-parent is assumed to work? But if I understood correctly, that in trusted repo I can define promote jobs for projectB, while secrets and job is defined in projectA. And it will run it and secrets are not a problem | ||
| @noonedeadpunk:matrix.org | Which I think is a bit different then github jobs? | 15:29 |
| @noonedeadpunk:matrix.org | I will try this out, thanks a ton for the help, I would never realize that paragraph | 15:32 |
| @noonedeadpunk:matrix.org | * I will try this out, thanks a ton for the help, I would never realize that paragraph on my own | 15:32 |
| @fungicide:matrix.org | Dmitriy Rabotyagov: if i understand what you're saying, a project-pipeline in projectB won't be able to add a job defined in projectA if that job defines a secret, since it would be possible then for the maintainers of projectB to leak/exfiltrate/capture projectA's secret once the build decrypted it | 15:41 |
| @noonedeadpunk:matrix.org | Yeah, unless it's done in project-config? | 15:42 |
| @noonedeadpunk:matrix.org | `If a job.secrets is used in a job definition in an untrusted-project, allowed-projects is automatically set to the current project only, and can not be overridden. However, a config-project may still add such a job to any project’s pipeline. ` | 15:42 |
| @noonedeadpunk:matrix.org | `add such a job to any project’s pipeline` specifically | 15:42 |
| @fungicide:matrix.org | right, you could use a project entry in a trusted config repo to add that job to a pipeline | 15:43 |
| @fungicide:matrix.org | pretty sure | 15:43 |
| @noonedeadpunk:matrix.org | But then I don't think there's much possibilities for the leak? | 15:43 |
| @fungicide:matrix.org | but not from a project-pipeline defined inside an untrusted repo | 15:43 |
| @noonedeadpunk:matrix.org | As they can't change the job | 15:43 |
| @noonedeadpunk:matrix.org | Yes, sure, from trusted. I didn't know it's possible:) | 15:44 |
| @fungicide:matrix.org | right, of course you still have to take care to design the job so that the secret is protected from being accessed by arbitrary code run by the job, but as long as you do it should be safe to run | 15:44 |
| @noonedeadpunk:matrix.org | Yeah, I know surely that I can't do that from untrusted repo. Though I didn't know I can do from trusted | 15:44 |
| @noonedeadpunk:matrix.org | SO thanks a ton again | 15:46 |
| @fungicide:matrix.org | hopefully that's sufficient to get it to do what you want. working with secrets is tricky because of the competing interests to secure them from access by people maintaining other projects while also allowing jobs to be reused between projects | 15:49 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 811017: Expose nodepool slot attribute https://review.opendev.org/c/zuul/zuul/+/811017 | 15:56 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Guillaume Chauvel: [zuul/zuul] 875056: tutorial: Update node to jammy https://review.opendev.org/c/zuul/zuul/+/875056 | 15:57 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Dong Zhang: [zuul/zuul] 871259: Zuul-Web: Filter input validation for "Change" https://review.opendev.org/c/zuul/zuul/+/871259 | 16:41 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 22:22 | |
| - [zuul/nodepool] 877559: Run two statemachine driver threads https://review.opendev.org/c/zuul/nodepool/+/877559 | ||
| - [zuul/nodepool] 877560: Add API profiling logging of openstack list_volumes https://review.opendev.org/c/zuul/nodepool/+/877560 | ||
| - [zuul/nodepool] 877561: Add a LazyExecutorTTLCache to the OpenStack driver https://review.opendev.org/c/zuul/nodepool/+/877561 | ||
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 877561: Add a LazyExecutorTTLCache to the OpenStack driver https://review.opendev.org/c/zuul/nodepool/+/877561 | 22:26 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 23:07 | |
| - [zuul/nodepool] 877431: Use image cache when launching nodes https://review.opendev.org/c/zuul/nodepool/+/877431 | ||
| - [zuul/nodepool] 877432: Use node cache in node deleter https://review.opendev.org/c/zuul/nodepool/+/877432 | ||
| -@gerrit:opendev.org- Zuul merged on behalf of Joshua Watt: [zuul/zuul] 873742: merger: Keep redundant cherry-pick commits https://review.opendev.org/c/zuul/zuul/+/873742 | 23:33 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 23:54 | |
| - [zuul/nodepool] 877431: Use image cache when launching nodes https://review.opendev.org/c/zuul/nodepool/+/877431 | ||
| - [zuul/nodepool] 877432: Use node cache in node deleter https://review.opendev.org/c/zuul/nodepool/+/877432 | ||
| - [zuul/nodepool] 877565: Log the reason we decline a request https://review.opendev.org/c/zuul/nodepool/+/877565 | ||
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!