| corvus | yeah, it'll be harder to do correctly since they're on different mm systems | 00:01 |
|---|---|---|
| corvus | however, for an announce-only list, should still be feasible. | 00:01 |
| fungi | yep, manageable | 00:06 |
| *** pwhalen has quit IRC | 00:06 | |
| *** sanjayu__ has quit IRC | 00:27 | |
| *** mattw4 has quit IRC | 00:44 | |
| *** spsurya has joined #zuul | 01:18 | |
| *** rlandy has quit IRC | 01:31 | |
| *** rfolco has quit IRC | 01:46 | |
| *** rfolco has joined #zuul | 01:47 | |
| *** threestrands has joined #zuul | 04:15 | |
| *** pcaruana has joined #zuul | 04:30 | |
| *** flepied has joined #zuul | 05:37 | |
| *** themroc has joined #zuul | 06:56 | |
| *** hashar has joined #zuul | 07:03 | |
| *** sshnaidm|off is now known as sshnaidm | 07:06 | |
| *** gtema has joined #zuul | 07:08 | |
| *** yolanda has joined #zuul | 07:15 | |
| *** gtema has quit IRC | 07:32 | |
| *** jpena|off is now known as jpena | 07:56 | |
| *** evrardjp has joined #zuul | 08:04 | |
| *** sshnaidm is now known as sshnaidm|afk | 08:07 | |
| *** panda|ruck has quit IRC | 08:14 | |
| *** panda has joined #zuul | 08:28 | |
| *** gtema has joined #zuul | 08:36 | |
| *** threestrands has quit IRC | 08:39 | |
| *** openstackgerrit has quit IRC | 08:47 | |
| *** panda is now known as panda|ruck | 08:52 | |
| tristanC | Hello, could this fix be reviewed please: https://review.opendev.org/659026 ? | 09:28 |
| *** hashar has quit IRC | 09:33 | |
| *** sanjayu__ has joined #zuul | 09:41 | |
| *** sanjayu__ has quit IRC | 09:52 | |
| tristanC | corvus: deployments using git.zuul-ci.org/zuul-jobs now have a config error "Unknown project zuul/zuul-jobs" for the zuul-jobs-test-registry job. | 09:56 |
| *** openstackgerrit has joined #zuul | 10:03 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: web: remove SafeLoader left-over from ZuulJSONEncoder https://review.opendev.org/659026 | 10:03 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: config: add tenant.toDict() method and REST endpoint https://review.opendev.org/621344 | 10:05 |
| *** sshnaidm|afk is now known as sshnaidm | 10:25 | |
| *** badboy has joined #zuul | 10:30 | |
| *** ParsectiX has joined #zuul | 10:33 | |
| *** gtema has quit IRC | 10:39 | |
| *** gtema has joined #zuul | 10:40 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: jsonutil: do not encode SourceContext or ZuulMark https://review.opendev.org/663298 | 10:50 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: config: add tenant.toDict() method and REST endpoint https://review.opendev.org/621344 | 10:55 |
| *** themroc has quit IRC | 11:04 | |
| *** jpena is now known as jpena|lunch | 11:07 | |
| *** themroc has joined #zuul | 11:07 | |
| *** panda|ruck is now known as panda|ruck|eat | 11:19 | |
| *** mgoddard has joined #zuul | 11:19 | |
| *** sanjayu__ has joined #zuul | 11:44 | |
| *** yolanda has quit IRC | 11:48 | |
| *** hashar has joined #zuul | 11:59 | |
| *** jpena|lunch is now known as jpena | 12:01 | |
| *** rlandy has joined #zuul | 12:15 | |
| *** badboy has quit IRC | 12:17 | |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Create a basic Bitbucket build status reporter https://review.opendev.org/658335 | 12:22 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Create a basic Bitbucket event source https://review.opendev.org/658835 | 12:22 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Upgrade formatting of the patch series. https://review.opendev.org/660683 | 12:22 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 12:22 |
| *** panda|ruck|eat is now known as panda|ruck | 12:30 | |
| *** ParsectiX has quit IRC | 12:33 | |
| *** migi has joined #zuul | 12:35 | |
| migi | hello, I have a question regarding job definition syntax. Is it possible to use for the *same* job two different nodesets based on different branch regex? | 12:36 |
| migi | migi: answered question https://zuul-ci.org/docs/zuul/user/config.html#attr-job.branches , sorry for noise | 12:39 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 13:08 |
| *** ParsectiX has joined #zuul | 13:08 | |
| fungi | migi: yes, the way we do that in our jobs is to have two variants of the job with different branch regexes and then associate different node types with them. an example: https://opendev.org/openstack/openstack-zuul-jobs/src/commit/a7aa530a6059b464b32df69509e3001dc97e2aed/zuul.d/jobs.yaml#L343-L378 | 13:10 |
| fungi | so projects can add that one job name, but which variant (and therefore which nodeset) is used will be determined by the branch associated with the triggering event | 13:11 |
| *** hashar has quit IRC | 13:11 | |
| *** gtema has quit IRC | 13:31 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: URLTrigger driver time based - artifact change jobs triggering driver https://review.opendev.org/635567 | 13:39 |
| jkt | hi there, I'm swicthing from upload-logs to upload-logs-swift, and I noticed that the resulting URLs do not end with "index.html" | 13:40 |
| jkt | I'm using our company-internal Swift installation, so I have no clue if this is a non-standard config or whatever | 13:40 |
| jkt | but should the URLs returned from Zuul include this /index.html as a suffix? | 13:40 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: URLTrigger driver time based https://review.opendev.org/635567 | 13:43 |
| *** jamesmcarthur has joined #zuul | 13:48 | |
| pabelanger | jkt: are you using htmlify-logs role too? | 13:49 |
| jkt | pabelanger: nope, but I see that this index.html gets generated and stored when I brwose the container's content | 13:50 |
| pabelanger | jkt: yah, you'll also need to include that role, as it will build out all the bits that swift needs | 13:50 |
| jkt | I tried playing with the X-Meta-Web-Index, now I see in `openstack container show ci-logs-public` that it's set, but I don't get that redirect | 13:51 |
| jkt | pabelanger: OK, let me try that one | 13:51 |
| pabelanger | jkt: https://github.com/ansible/project-config/blob/master/playbooks/base-minimal/post-logs.yaml is an example of our post-run playbook | 13:51 |
| jkt | pabelanger: but I'm confused -- I already see a pretty index with images and what not, the issue is that I *have* to adjust the URL so that it includes the trailing /index.html | 13:52 |
| pabelanger | jkt: yah, for vexxhost we need to ensure that url has trailing slash | 13:52 |
| pabelanger | then index.html is rendered | 13:52 |
| pabelanger | I don't believe we setup any meta data in swift manually | 13:53 |
| pabelanger | but been a while, would need to check properties of the container | 13:53 |
| jkt | pabelanger: I do not think that I need htmlify-logs, I can access these logs, it's just that I have to adjust my top-level URL | 13:54 |
| jkt | it looks that our Swift instance auto-redirects from a path/to/dir to a path/to/dir/ | 13:54 |
| *** spsurya has quit IRC | 13:55 | |
| ofosos | SpamapS: Can you have a look at https://review.opendev.org/#/c/658335/ (again) and at https://review.opendev.org/#/c/658835/ | 13:55 |
| ofosos | corvus: Can you have a look at https://review.opendev.org/#/c/658335/? | 13:58 |
| *** jamesmcarthur has quit IRC | 14:09 | |
| *** michael-beaver has joined #zuul | 14:11 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Don't use allowed-projects https://review.opendev.org/663339 | 14:12 |
| corvus | tristanC: thanks, i guess we need to start pushing folks to update connections if we're going to do that | 14:13 |
| *** jamesmcarthur has joined #zuul | 14:18 | |
| *** jamesmcarthur has quit IRC | 14:22 | |
| mordred | tristanC, corvus: yeah - I think we do need to push people to do that | 14:23 |
| *** jamesmcarthur has joined #zuul | 14:26 | |
| *** gtema has joined #zuul | 14:29 | |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 14:30 |
| jkt | pabelanger: so FYI, in our setup I had to run `openstack container set $CONTAINER --property web-listings=true` , and I have Swift's own listings | 14:32 |
| jkt | I don't know why it won't use the provided index.html | 14:32 |
| corvus | jkt: the intent of the roles is not to place any requirement on using the listings from swift -- the idea is to statically generate the listings so that isn't necessary | 14:34 |
| corvus | jkt: but i wonder if we have an implicit assumption that the swift index file is set to index.html | 14:35 |
| corvus | jkt: what if you turn off the web-listings and instead set "X-Container-Meta-Web-Index" to index.html? | 14:36 |
| jkt | corvus: I tried that before, but let me try once again (it looks that this Swift instance caches these settings somehow) | 14:37 |
| corvus | jkt: another option could be to make sure that we always include "index.html" in any links we generate | 14:37 |
| jkt | running curl to retrieve these objects *after* changing these properties, and *after* that object was already accessed, apparently ignores further updates to index properties of the container | 14:38 |
| corvus | that makes experimenting tricky :( | 14:38 |
| jkt | yes | 14:41 |
| jkt | also, `openstack container unset ci-logs-public --property Web-Listings` does not result in any change to the output of ... show | 14:41 |
| jkt | but if I set that property to false explicitly, then it disappears :) | 14:41 |
| jkt | I'm quite happy that this service is run by someone else to be honest :) | 14:42 |
| jkt | corvus: so, on an object created from scratch, I get a NoSuchKey: https://object-store.cloud.muni.cz/swift/v1/ci-logs-public/26/1726/1/check/dummy-build-A/90ddba9/ | 14:43 |
| jkt | http://paste.openstack.org/show/752535/ shows the container configuration | 14:44 |
| jkt | would you accept a patch with an option which turns off generating the index files? | 14:46 |
| *** themroc has quit IRC | 15:04 | |
| *** lennyb has quit IRC | 15:07 | |
| *** lennyb has joined #zuul | 15:07 | |
| openstackgerrit | Jan Kundrát proposed zuul/zuul-jobs master: upload-logs-swift: option for disabling the indexer https://review.opendev.org/663355 | 15:09 |
| jkt | here we go. | 15:09 |
| openstackgerrit | Merged zuul/zuul master: web: remove SafeLoader left-over from ZuulJSONEncoder https://review.opendev.org/659026 | 15:11 |
| *** felixgcb has joined #zuul | 15:14 | |
| felixgcb | Hey guys :) I'm again coming with a burning question. TL;DR: Why are variables only read once on branch creation? Once I create a branch in an untrusted repo, zuul will read this branches variables from the project configuration (.zuul.yaml). If I change those later during my normal debugging work, maybe I need to work on my project configuration as part of the branch, those changes are ignored. Is that a bug or intended behavior? | 15:16 |
| *** panda|ruck is now known as panda | 15:19 | |
| *** rlandy is now known as rlandy|ruck | 15:19 | |
| clarkb | by variables do you mean jobs? | 15:21 |
| clarkb | (examples of configuration that are not loaded may be helpful too) | 15:21 |
| fungi | felixgcb: are you asking why zuul doesn't notice changes you've made to branches out of band of changes it merges to them? | 15:22 |
| fungi | by what process do you "change those later during normal debugging work"? | 15:24 |
| *** yolanda has joined #zuul | 15:26 | |
| felixgcb | fungi: that's a great question, I do so by adding a new commit to the branch, and triggering zuul to run the check pipeline again. It will then work with the updated content of the branch, except it will ignore the updated variables | 15:26 |
| felixgcb | We're working with something similar to github where you have branches and zuul checks each new commit in this branch | 15:27 |
| fungi | also would be good to know if you actually mean vars, or you're speaking more generally about job configuration (per clarkb's question) | 15:29 |
| fungi | normally zuul should use the job configuration present in the branch, and expects the event stream to include messages indicating when those branches have updated so that the scheduler knows to reload them | 15:30 |
| fungi | this is generally independent of the branch state used by the mergers/executors/nodes | 15:31 |
| pabelanger | is this using the github driver? | 15:31 |
| felixgcb | I mean the project configuration for an untrusted project (.zuul.yaml or zuul.yaml). Here I can configure my check pipelines for my code repository. These would consist out of jobs which I want to parameterize using variables. The thing is they are only considered on branch-creation :/ | 15:31 |
| pabelanger | because, stacking commits in a PR, does work if a later commit updates zuul.yaml, IIRC | 15:32 |
| fungi | by "something similar to github" i take it to mean a custom connection driver | 15:32 |
| felixgcb | pabelanger: ah interesting, we (my colleague) is currently working on a bitbucket driver | 15:32 |
| fungi | which is why i'm suspecting some missing signalling | 15:32 |
| pabelanger | yah | 15:32 |
| pabelanger | or, the project isn't untrusted | 15:33 |
| pabelanger | actually | 15:33 |
| fungi | well, he's merging changes, so even if it's trusted | 15:33 |
| pabelanger | yah | 15:33 |
| pabelanger | sounds like what fungi said, maybe zuul hasn't reloaded the job changes yet | 15:33 |
| openstackgerrit | Merged zuul/zuul-jobs master: Don't use allowed-projects https://review.opendev.org/663339 | 15:34 |
| fungi | felixgcb: the short answer is that zuul is designed to update its configuration any time the branch state is updated. if it's not doing that, then i suspect either a problem with configuration or a problem with the connection driver (like it being incomplete) | 15:34 |
| fungi | at least for in-project pipeline additions and the like | 15:34 |
| fungi | it does still require manual signalling to update its main config, but it doesn't sound like that's what you're altering anyway | 15:35 |
| felixgcb | Yeah we don't want to update the main config, its just a dummy code repo | 15:36 |
| felixgcb | Okay I think you guys gave really good advice, we'll need to look at our driver again | 15:37 |
| felixgcb | fungi: "zuul is designed to update its configuration any time the branch state is updated", is there a "zuul-term" or "zuul-event" for this mechanism that you could recall from the top of your head? | 15:37 |
| felixgcb | Otherwise checking the github driver will be an option... | 15:38 |
| corvus | felixgcb: this is the code that handles it: https://opendev.org/zuul/zuul/src/branch/master/zuul/scheduler.py#L1103 | 15:40 |
| corvus | felixgcb: along with this: https://opendev.org/zuul/zuul/src/branch/master/zuul/scheduler.py#L1077 | 15:41 |
| felixgcb | Awesome, thank you guys so much :) <3 | 15:41 |
| felixgcb | Ah even more... | 15:41 |
| corvus | felixgcb: for that to happen, the driver needs to submit an event with a branch_updated and files attribute set -- that might be the missing functionality | 15:41 |
| felixgcb | I bet it is, our driver is just not emitting some needed event.. | 15:44 |
| openstackgerrit | Monty Taylor proposed zuul/nodepool master: Explicitly set use_direct_get to False https://review.opendev.org/663368 | 15:48 |
| *** AshBullock has joined #zuul | 15:49 | |
| AshBullock | Hey guys, trying to set up nodepool with k8s, how should I pass the k8's creds to nodepool? I can't see this being set in the provider config in the documentation | 15:51 |
| openstackgerrit | Paul Belanger proposed zuul/nodepool master: Clean up utils.nodescan function https://review.opendev.org/663372 | 15:55 |
| pabelanger | Shrews: clarkb: corvus: tobiash: any reason not to do ^? I can't figure out why we setup a socket, if we are not going to use it for nodescan | 15:56 |
| corvus | pabelanger: to make sure we can make a connection | 15:57 |
| corvus | pabelanger: ie, that the server is up | 15:57 |
| pabelanger | corvus: okay, that likely explains it. I still have host-key-scan true, but connection-type set to something other then ssh / network_cli | 15:58 |
| pabelanger | and nodepool isn't by passing that | 15:58 |
| pabelanger | since, the interface isn't public | 15:58 |
| clarkb | AshBullock: I believe you are expected to have a kubeconfig file that nodepool uses | 15:58 |
| pabelanger | I'll disable host-key-checking | 15:59 |
| pabelanger | darn, that looks to be only available at pool level | 16:01 |
| AshBullock | should this be added to the nodepool users home path? | 16:01 |
| *** panda is now known as panda|off | 16:02 | |
| AshBullock | not sure where the kube config should be placed | 16:03 |
| clarkb | AshBullock: I believe so, the docs dont say though :/ | 16:03 |
| *** mattw4 has joined #zuul | 16:13 | |
| *** ParsectiX has quit IRC | 16:22 | |
| *** felixgcb has quit IRC | 16:25 | |
| tobiash | pabelanger: for winrm e.g. we don't gather keys but still want to ensure that the port is reachable | 16:33 |
| *** panda|off has quit IRC | 16:33 | |
| pabelanger | tobiash: yah, I have a pretty odd use case, where this cisco appliance will not get a default gateway assigned via DHCP | 16:34 |
| *** panda has joined #zuul | 16:35 | |
| pabelanger | but, becaue I do multi-node, I can get into VM A, then connect via shared network into VM B, and configure it | 16:35 |
| openstackgerrit | Paul Belanger proposed zuul/nodepool master: Toggle host-key-checking for openstack provider.labels https://review.opendev.org/663378 | 16:35 |
| pabelanger | but I first need ^ | 16:35 |
| pabelanger | as, I also would like to keep the node, in the same pool as other nodes | 16:36 |
| jamesmcarthur | Hi Zuul team. Does anyone have a moment to take a look at this: https://review.opendev.org/#/c/660232/ It's just removing the Shanghai promo from the top of the site. | 16:36 |
| *** panda has quit IRC | 16:39 | |
| *** AshBullock has quit IRC | 16:44 | |
| *** AshBullock has joined #zuul | 16:45 | |
| *** mhu has quit IRC | 16:46 | |
| *** mhu has joined #zuul | 16:46 | |
| fungi | jamesmcarthur: thanks, what's the context for that? is there expected to be no zuul community presence in shanghai? | 16:47 |
| *** panda has joined #zuul | 16:48 | |
| pabelanger | tobiash: clarkb: Shrews: corvus: do you mind adding https://review.opendev.org/663378 to your review pipeline, this is to help with an odd use case where we have a network appliance that comes online without a default gateway. But want to label to be in the same pool, due to quota reasons | 16:48 |
| jamesmcarthur | fungi: I hope not! :) That promo is currently pointing to the schedule for Denver. We don't have a schedule launched yet for Shanghai. | 16:49 |
| fungi | oh, hah! | 16:49 |
| jamesmcarthur | And I'm not sure what the messaging is yet around promos for Shanghai. | 16:49 |
| jamesmcarthur | Although we're a bit further along now than when I first proposed the pathc. | 16:50 |
| jamesmcarthur | patch | 16:50 |
| mattw4 | Hi Zuul team. I'm trying to include (run) roles from devstack (and other projects) into Zuul, but I don't want to respond to events in the gerrit stream for those projects. Can someone point me to an example configuration for "including roles from other projects, but not testing those projects"? | 16:50 |
| fungi | jamesmcarthur: got it, i had incorrect context from your message in here saying it was removing the shanghai promo. i see in the diff it's removing the denver promo which makes much more sense | 16:50 |
| jamesmcarthur | oh, sorry - mistype :) | 16:51 |
| openstackgerrit | Merged zuul/zuul-website master: Revert "Revert promotional message banner and event list" https://review.opendev.org/660232 | 16:53 |
| mattw4 | I've gotten devstack roles to run by adding "roles: -zuul: opendev.org/openstack/devstack" to the project's .zuul.yaml, but I also had to add "- openstack/devstack: include: job" to the tenant configuration which didn't seem quite right. Should I be adding it as a project-template or something instead? | 16:53 |
| corvus | mattw4: as long as there is no "project" definition for a project, zuul won't run any jobs. what you did is exactly right. | 16:53 |
| mattw4 | corvus: thanks! So I can "include: job" for any project where I need their roles, just make sure not to include the project. Right? | 16:55 |
| corvus | mattw4: yep | 16:55 |
| corvus | mattw4: in fact, if you only need the *roles* and not the job definitions (eg the "devstack" job itself), you can "include: []" | 16:55 |
| mattw4 | Thank you so much! I accidentally pointed Zuul at some OpenStack projects a few months ago (you may remember that I caused some confusing comments) so I don't want to do that again! | 16:55 |
| mattw4 | good deal corvus, that's even cleaner | 16:56 |
| *** ParsectiX has joined #zuul | 16:57 | |
| flaper87 | is there documentation on how to upgrade zuul? one question I have is whether executors can be tainted so that they won't get new jobs and then updated? | 17:00 |
| flaper87 | unless there's a better way to do this | 17:00 |
| corvus | flaper87: no; we need to implement the "graceful" shutdown command for executors | 17:01 |
| corvus | shouldn't be hard, it's just that no one's done it yet | 17:02 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Return javascript content artifact records to Zuul https://review.opendev.org/663056 | 17:07 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Return python artifact records to Zuul https://review.opendev.org/663053 | 17:07 |
| pabelanger | yah, when we had it for zuul 2.5, it worked pretty well | 17:08 |
| pabelanger | (graceful) | 17:08 |
| *** AshBullock has quit IRC | 17:10 | |
| *** gtema has quit IRC | 17:13 | |
| *** jpena is now known as jpena|off | 17:13 | |
| openstackgerrit | Jeremy Stanley proposed zuul/zuul master: Safely add Ansible password lookup plugin https://review.opendev.org/662870 | 17:16 |
| fungi | though in our experience, stopping an executor with running jobs generally just gets those jobs rescheduled to another executor and restarted. so while it's a waste of test resources, it's not overly disruptive | 17:18 |
| fungi | our usual process is to install the new version, then stop the executor daemon, wait for everything to die, and start it again | 17:19 |
| tobiash | Flaper87: you at least can pause the executors and restart them manually when they're empty | 17:24 |
| *** ParsectiX has quit IRC | 17:24 | |
| tobiash | What we're missing though is that paused executors still serve merge calls | 17:25 |
| mattw4 | Hi again Zuul team, I'm trying to author a job that inherits from devstack-minimal, but seeing the error "Job devstack-minimal not defined". I *think* I've "included" devstack in the tenant configuration with "include: job", so I'm surprised at the error. What am I missing? | 17:28 |
| fungi | mattw4: checked for configuration errors? little alarm bell icon at the top-right of your zuul status page will show up if there are | 17:31 |
| corvus | mattw4: what fungi said -- and i may be able to guess what the problem is. devstack is tough case in that it requires a lot of openstack projects. so you're *probably* going to need to "include: []" for a whole bunch of openstack projects in order to get the devstack-minimal job to be able to resolve. | 17:33 |
| mattw4 | fungi: yeah, lots of "Unknown project opensev.org/openstack/..." and "Job devstack not defined (for various branches)" | 17:33 |
| mattw4 | not sure how to resolve those either :/ | 17:33 |
| corvus | mattw4: you can look at the list of required-projects there and work out which ones you add | 17:34 |
| fungi | i think you may also need to add the projects that depends on | 17:34 |
| corvus | er, which ones you need to add | 17:34 |
| fungi | yeah, those | 17:34 |
| fungi | devstack-minimal has opendev.org/openstack/requirements as a required-project | 17:34 |
| pabelanger | darn, labels are not shared across pools for nodepool providers. Which means, I do need: https://review.opendev.org/663378/ to get this cisco appliance to work | 17:34 |
| mattw4 | fungi, corvus: so I just need to add those as required-projects for the particular job? | 17:35 |
| fungi | corvus: and any listed in the roles list too, right? | 17:35 |
| corvus | mattw4: no you need to add them as projects in your zuul tenant, just like you did with devstack (but you can "include: []" for those) | 17:35 |
| corvus | fungi: yes, any time the job says it needs a project | 17:36 |
| fungi | mattw4: also you need to look at the parent jobs of that job and include any they also specify | 17:36 |
| corvus | mattw4: once you work out the necessary tenant config, i'm sure other folks would appreciate it if you sent a copy to the openstack-discuss mailing list; right now, every new third-party ci needs to repeat this process. to my knowledge, no one has posted the result yet. | 17:36 |
| fungi | mattw4: as for parent jobs, you have to keep iterating until you get to one with no parent parameter | 17:37 |
| mattw4 | gotcha, it's a bit of a dependency walk to gather the required projects. Thanks corvus & fungi, I will definitely be sharing once I work this out. | 17:37 |
| fungi | since required-projects and roles can be added anywhere in the ancestry | 17:37 |
| fungi | mattw4: and you'll discover that walk will take you into other projects' zuul configs besides just the devstack repository | 17:38 |
| corvus | fungi: it's just devstack and zuul-jobs | 17:39 |
| fungi | ahh, yep, that's convenient. at least zuul-jobs is likely already included | 17:39 |
| corvus | base > multinode > devstack-base > devstack-minimal | 17:39 |
| fungi | right, minimal comes from zuul-jobs | 17:40 |
| fungi | er, multinode comes from zuul-jobs i mean | 17:40 |
| fungi | m words | 17:40 |
| corvus | you can explore it here: https://zuul.openstack.org/jobs -- put 'devstack-minimal' into the box | 17:41 |
| corvus | it would be nice to be able to deep link to that. | 17:41 |
| mattw4 | fungi, corvus: is there any way to see a running example of a tenant config? I've found lots of examples of job/project config, but almost nothing for tenant & zuul config to enable those projects. | 17:42 |
| corvus | mattw4: here is a really big one: https://opendev.org/openstack/project-config/src/branch/master/zuul/main.yaml | 17:43 |
| mattw4 | corvus: that's perfect, thanks! | 17:45 |
| fungi | yeah, it's... *really* big | 17:46 |
| fungi | but it's the one which gates zuul/zuul and friends at the moment (soon to change!) | 17:46 |
| fungi | well, which tenant in that file gates zuul/zuul is soon to change. the location of that file is also going to change, but maybe less soon | 17:48 |
| pabelanger | okay, I've sent email to ML about current issue with host-key-checking, would love some feedback. | 18:03 |
| *** hashar has joined #zuul | 18:04 | |
| jkt | I don't think there's a standalone role for an ad-hoc upload to swift, or is there one? | 18:12 |
| jkt | (I'd like to get rid of rsyncing build artifacts now that logs are in Swift) | 18:12 |
| jkt | the os_object doesn't support setting auto-expiry, but maybe it could be a good start | 18:14 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Return javascript content artifact records to Zuul https://review.opendev.org/663056 | 18:17 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Return python artifact records to Zuul https://review.opendev.org/663053 | 18:17 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: DNM: test zuul-jobs artifact changes https://review.opendev.org/663081 | 18:17 |
| corvus | jkt: you mean uploading from the worker directly to swift rather than rsyncing to the executor first? | 18:19 |
| fungi | back in the zuul v2 logs-on-swift experiment in opendev we had logic which procured subtree-scoped temporary credentials and passed those to the job node, so the same could probably be implemented in v3 with some effort? one new challenge is that nodes in a multi-node set are now independent and we rely on the executor to aggregate artifacts from them, whereas if they were uploading on their own they'd need to | 18:24 |
| fungi | coordinate that or get individual publishing subtrees assigned per node | 18:24 |
| corvus | that's something we'd like to support, but it's going to take a while to get there, and even so, there may still be some caveats. first: a lot of roles in zuul-jobs assume that things will be rsynced back to the executor; mordred is going to write up a brief spec on what needs to change there. second: uploading from the worker means either trusting the worker with swift creds, or using the formpost | 18:25 |
| corvus | middleware which isn't guaranteed to be present (and even so, that case merely limits the amount of abuse that can happen); whether either of these options is okay will vary from site to site. | 18:25 |
| jkt | corvus: just from the executor to the "final place" | 18:26 |
| jkt | I am not even thinking about credential delegations | 18:26 |
| corvus | jkt: oh, so something like what the logs is doing but for a different swift destination for artifacts | 18:26 |
| *** migi has quit IRC | 18:26 | |
| *** pabelanger has quit IRC | 18:26 | |
| *** weshay has quit IRC | 18:26 | |
| *** mhu has quit IRC | 18:26 | |
| jkt | corvus: yes | 18:27 |
| fungi | ahh | 18:27 |
| *** mhu has joined #zuul | 18:27 | |
| *** weshay has joined #zuul | 18:27 | |
| jkt | this is probably trivial, something like http://paste.openstack.org/show/752546/ | 18:27 |
| fungi | i keep thinking of logs as a special class of artifacts, but i suppose zuul does treat logs and artifacts diffrently | 18:27 |
| corvus | jkt: gotcha; i don't think there's one, but i think it would be a fine addition, and i don't see as many problems with that :) | 18:27 |
| *** armstrongs has joined #zuul | 18:28 | |
| jkt | more context: this is about some C++ project which depens on external libraries that are not gated by zuul | 18:28 |
| jkt | so what works for me now is to build stuff in dependencies.git's "check", store the artifacts in the same manner as logs, and in the promote pipeline, copy them to something "more deterministic" | 18:29 |
| jkt | something like $swift_container/artifacts/dependencies.git/$commit.tar.xz | 18:29 |
| fungi | jkt: this sounds a lot like how we're doing container builds and then having child jobs consume them | 18:30 |
| jkt | yeah, I asked for advice here about an year ago | 18:31 |
| fungi | jkt: rather than stash our check pipeline builds in a persistent service we have an initial job which stands up a server to house the temporary artifacts, then pause that job and run dependent jobs which fetch the artifacts from it and use them | 18:31 |
| fungi | so first job could for example build those libraries and then start apache, that job is paused, child jobs get the url to that ephemeral service and retrieve the builds they want from it to use | 18:33 |
| jkt | I'm updating (and hence building) these libraries maybe once a week, maybe even less frequently | 18:35 |
| jkt | but the consumers are built much more often, and I do not want to mess with plenty of services | 18:35 |
| jkt | I really need just a method of passing a single tarball around | 18:35 |
| fungi | and not interested in being able to specify change dependencies on patches which haven't landed to the libraries you're talking about, i guess | 18:35 |
| jkt | fungi: my dependencies.git include all shared dependencies as git submodules, so I can simply point a submodule to a particular ref of my library with a patch applied as a git branch | 18:36 |
| fungi | (consuming project change depending on a not-yet-merged feature of the library, but still tested against it) | 18:36 |
| *** sshnaidm is now known as sshnaidm|afk | 18:36 | |
| jkt | yes, we use that | 18:36 |
| fungi | ahh, but how do you map that to the copy you're serving from swift? | 18:37 |
| jkt | fungi: name that artifact with a commit-sha1 of the dependencies.git | 18:37 |
| jkt | fungi: and also add dependencies.git as a submodule in my consumer | 18:38 |
| fungi | got it. and set an expiration on it in swift which is longer than you anticipate any consuming change needing it for (or just let swift content grow unbounded forever)? | 18:38 |
| jkt | https://gerrit.cesnet.cz/plugins/gitiles/CzechLight/netconf-cli/+/master/ci/build.sh#50 , this is how it currently works | 18:38 |
| jkt | with rsync and an apache for artifact serving | 18:39 |
| jkt | there are gotchas :) of course | 18:39 |
| jkt | a) zuul doesn't understand gerrit's topics, which is annoying | 18:39 |
| jkt | b) my two consumers live under different zuul tenants, and therefore I have to build dependencies.git twice | 18:40 |
| jkt | but I really wish Zuul understood gerrit topics (a topic can only be submitted "atomically") | 18:40 |
| jkt | because in the end, both leaf-A and leaf-B are going to end on the same embedded system, so they need all of their deps to be the same | 18:41 |
| fungi | oh, i see, you want to overload the gerrit topic metadata as an indicator of changes which don't have to be tested sequentially? | 18:41 |
| jkt | so my workflow when updating is to push three changes sharing the same topic: | 18:41 |
| jkt | 1) dependencies.git, to bump all the libraries, perhaps with non-upstream patches, and often with breaking changes | 18:42 |
| jkt | 2) leaf-A, Depends-on: change-1, perform any code changes to adapt to whatever upstream breakage happened | 18:42 |
| jkt | 3) leaf-B, Depends-on: change-1, similarly to change-2 | 18:42 |
| jkt | now, I'm only running "check", not "gate", and so it works reasonably well | 18:43 |
| fungi | i suppose that could work as long as the users of your gerrit don't also use topics for other purposes, though from the zuul side there are probably logistical challenges but i haven't thought about it long enough to know what the would be | 18:43 |
| jkt | if I was running gate, I don't know how to tell Zuul "hey, let's just tie these three changes together please" | 18:43 |
| *** armstrongs has quit IRC | 18:44 | |
| jkt | if this is to work, dependencies.git must be set (in Gerrit) to forward-only, no merges, because I need my resulting SHA1 to be the tested SHA1 | 18:44 |
| jkt | these topics, that's a new Gerrit thing, I think it came in 2.15 | 18:45 |
| fungi | probably it would need to 1. only consider the final change in the topic cluster for each project+branch involved, 2. defer merging any of them until those passed | 18:45 |
| jkt | yes, I think that's correct | 18:45 |
| fungi | jkt: do they call them something other than a "topic"? gerrit has had change topics for aeons | 18:45 |
| jkt | fungi: ah, it's a new option which is not active by default, https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#change.submitWholeTopic | 18:46 |
| fungi | oh, a configuration option | 18:47 |
| fungi | which is new | 18:47 |
| fungi | but leveraging the change topic info, which has been there pretty much forever | 18:47 |
| jkt | fungi: since they got hashtags (which depend on the git notedb backend, not the SQL reviewdb IIRC), there's IMHO little point in using topics just for hashtags | 18:47 |
| fungi | sure | 18:47 |
| jkt | but I have no idea why they did it this way, right | 18:47 |
| fungi | except when you have an established dataset where your users have been using topics that way forever, and so introducing hashtags doesn't mean they suddenly stop | 18:48 |
| jkt | yes :) | 18:48 |
| fungi | but for new deployments of gerrit it may make sense, or where you can convince your entire userbase to change their workflow | 18:48 |
| fungi | other corner cases an implementation in zuul would need to take into consideration are things like what happens of a topic is modified while changes are being tested (dequeue them?) | 18:49 |
| fungi | and what is the precedence of dependencies across topics? | 18:50 |
| fungi | if a change depends on another change with a different topic, does it actually depend on the entire topic cluster? what about when a change in a topic cluster depends on another change with a different topic? do they all depend on that change? | 18:51 |
| jkt | I tend to think of the Depends-On header as a band-aid that was needed when Gerrit did not support this feature | 18:51 |
| jkt | IOW, I wonder what they solve apart from what that change.submitWholeTopic already provides | 18:51 |
| jkt | I (hope) I understand the difference, Depends-on is just an unidirectional DAG, topic is a bidirectional notation, gluing changes together in both ways | 18:52 |
| fungi | not exactly, depends-on could have been designed differently to specify sets of changes which needed to merge at the same time. we discussed that and at the time concluded that for our (more tightly-scoped) userbase that soft of development workflow was a bad idea we should prevent them from engaging in | 18:53 |
| jkt | yeah, it's good that you can afford this compatibility | 18:53 |
| fungi | basically we intentionally designed depends-on in such a way as to prevent openstack developers from avoiding the always-deployable benefits of strictly sequenced changes across multiple repositories | 18:54 |
| fungi | in the years since zuul's design has grown to support other development workflows and systems, and i think we're probably not opposed to supporting something along those lines if somebody works out how to implement it cleanly | 18:55 |
| *** ParsectiX has joined #zuul | 18:58 | |
| fungi | but bi-directional change dependency between repositories generally implies a very tight coupling between those repositories, which at the time depends-on was added we considered an unhealthy design pattern (for openstack projects) | 18:58 |
| evgenyl | Hi everyone, is there a way to specify a tenant's `config-projects` which is hosted on a different git server? | 19:01 |
| corvus | evgenyl: sure, tenants can have projects from any connection | 19:05 |
| corvus | nothing special needs to be done | 19:05 |
| smcginnis | Howdy folks. I'm playing around with third party CI setup to try to document some things. Can someone give me a sanity check on this pipeline config: http://paste.openstack.org/show/752549/ | 19:11 |
| evgenyl | corvus: Sorry for not articulating the problem properly, or I misunderstand something. I have a tenant definition that goes under e.g. github connection, now I want to define a project config which is fetched from gerrit (for tenants hosted on github), when I try to do that, the scheduler fails to fetch project-config, because it tries to get it from github. | 19:11 |
| evgenyl | Actually in my case I have two different installation of gerrit, but the idea is the same. | 19:12 |
| smcginnis | I have a "gerrit" config for my local setup using the docker-compose example, then I have a "gerrit-upstream" defined for listening for upstream events. | 19:13 |
| corvus | evgenyl: here's an example of exactly that setup: https://opendev.org/openstack/project-config/src/branch/master/zuul/main.yaml#L17-L50 | 19:13 |
| corvus | evgenyl: note the different entries under "source" underneath the tenant | 19:13 |
| evgenyl | corvus: Oh! I see, awesome, thank you again :) | 19:13 |
| smcginnis | I saw my thirdpartyci noop job kick off after my local zuul left +1, but did not see anything from the "gerrit-upstream" zuul leaving +1 on a patch in opendev/ci-sandbox that I'm watching. | 19:14 |
| *** panda has quit IRC | 19:19 | |
| *** pabelanger has joined #zuul | 19:19 | |
| *** panda has joined #zuul | 19:20 | |
| smcginnis | Here's my main.yaml (http://paste.openstack.org/show/752550/) to go with my pipelines.yaml (http://paste.openstack.org/show/752549/) | 19:28 |
| *** ParsectiX has quit IRC | 19:28 | |
| jkt | can Zuul work with "no-node" jobs? Right now my promote pipeline really just calls wget and rsync, only to rsync stuff back to the executor which will then upload bits to their final destination | 19:35 |
| fungi | jkt: yep! | 19:36 |
| fungi | we have lots of those | 19:36 |
| jkt | (and the real reason for asking is of course because my build nodes are IPv6-only, while my wonderful Swift API GW is IPv4-only, aaaargh) | 19:36 |
| fungi | like, jobs which just post something to a rest api can totally happen straight from the executor | 19:36 |
| jkt | fungi: got an example? I feel a bit lost at times | 19:37 |
| fungi | it would be a playbook which just uses hosts: localhost | 19:37 |
| jkt | (and the "promote"-style example I looked at some time ago was most definitely using some host) | 19:38 |
| fungi | and no nodeset | 19:38 |
| jkt | fungi: excellent, thanks | 19:38 |
| fungi | in the job definition | 19:38 |
| fungi | i'll get you an example | 19:38 |
| jkt | fungi: ah, I know that -- that's delegation of tasks | 19:38 |
| jkt | fungi: my question is if I can avoid allocating a node from nodepool altogether | 19:38 |
| fungi | right, not having a nodeset | 19:39 |
| jkt | fungi: thanks! | 19:39 |
| fungi | you can explicitly add... | 19:40 |
| fungi | nodeset: | 19:40 |
| fungi | nodes: [] | 19:41 |
| fungi | so a nodeset with an empty nodes list will ensure that | 19:41 |
| fungi | that way you don't inherit any nodeset from your base job | 19:41 |
| fungi | like in this job: https://opendev.org/zuul/zuul-jobs/src/branch/master/zuul.yaml#L42-L51 | 19:42 |
| openstackgerrit | David Shrewsbury proposed zuul/zuul master: WIP: Add caching of autohold requests https://review.opendev.org/663412 | 19:42 |
| fungi | jkt: and here's the corresponding playbook with the localhost delegation: https://opendev.org/zuul/zuul-jobs/src/branch/master/playbooks/docker-image/promote.yaml | 19:43 |
| smcginnis | fungi: Would you have a minute to take a look at the pastes I left above to let me know if I'm missing something obvious? Not getting anything triggered in the pipeline I'm expecting to for opendev/ci-sandbox tests. | 19:44 |
| smcginnis | Here's my main.yaml (http://paste.openstack.org/show/752550/) to go with my pipelines.yaml (http://paste.openstack.org/show/752549/) | 19:45 |
| fungi | smcginnis: i started to take a look but am not confident in my ability to spot possible omissions | 19:45 |
| smcginnis | fungi: OK, at least nothing blaringly obvious then. That helps! | 19:45 |
| fungi | and also multi-tasking during the storyboard weekly meeting right now | 19:46 |
| smcginnis | No worries, thanks! | 19:46 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add spec for enhanced regional executor distribution https://review.opendev.org/663413 | 19:46 |
| smcginnis | In case anyone else has a chance to look, here's the projects.yaml for completeness: http://paste.openstack.org/show/752551/ | 19:46 |
| mordred | smcginnis: this is not your problem - but fwiw, you could totally just include zuul/zuul-jobs from the gerrit-upstream connection | 19:49 |
| mordred | smcginnis: that said - I also don't see any obvious ommissions | 19:50 |
| fungi | smcginnis: did you check the scheduler log to see if there were any exceptions when it tried to report? | 19:51 |
| smcginnis | mordred: If I wanted to define my own local jobs, I wouldn't want to include zuul/zuul-jobs, right? Or I suppose I could have my own in a separate local repo. | 19:52 |
| smcginnis | fungi: No expections that I saw, and I don't have it reporting right now, just saving to the DB. | 19:53 |
| mordred | smcginnis: yeah - I'd put your own jobs in a separate local repo - that way you can inherit from jobs in zuul-jobs in your local jobs | 19:53 |
| fungi | oh, got it. i think i may have misunderstood your problem statement | 19:53 |
| smcginnis | fungi: Is there a recommended way of viewing these running in docker containers? | 19:53 |
| smcginnis | fungi: Problem statement can be summarized that I am looking at the Builds screen and not seeing anything having run when I thought it should have. | 19:53 |
| fungi | what is the "these" you want to view? | 19:53 |
| smcginnis | mordred: I do have: | 19:54 |
| smcginnis | opendev.org: | 19:54 |
| smcginnis | untrusted-projects: | 19:54 |
| smcginnis | - zuul/zuul-jobs: | 19:54 |
| smcginnis | include: | 19:54 |
| smcginnis | - job | 19:54 |
| fungi | oh, okay, so you can access the status dashboard at least | 19:54 |
| smcginnis | fungi: Yeah, that part looks good, did a few local test patches. | 19:54 |
| fungi | and your problem is the builds list is empty? | 19:54 |
| fungi | i wonder if noop jobs don't actually get build entries in the db... did you check buildsets? | 19:55 |
| smcginnis | Then added the gerrit-upstream connection and defined the thirdpartyci pipeline to trigger on zuul leaving +1 on ci-sandbox patches. I see the stream activity in the logs, but nothing triggered. | 19:55 |
| mordred | smcginnis: yah - I was mostly just pointing out that since you have an upstream-gerrit connection/source, you could get zuul/zuul-jobs from it instead of from a git-driver opendev.org connection ... but it's totally not essential to the thing you're asking about ... | 19:55 |
| smcginnis | Ah, got it. Thanks mordred. | 19:55 |
| mordred | smcginnis: if you're using the docker-compose we've got, it's set up to have the services log to stdout - so you should just be able to run docker logs on the container in question | 19:56 |
| smcginnis | Learning and tinkering right now to try to wrap my head around things and eventually write up some more up to date third party CI instructions. | 19:56 |
| mordred | ++ | 19:56 |
| mordred | smcginnis: I'm in favor of such a thing! | 19:56 |
| smcginnis | Since none of the ones we've suggested that to with new cinder drivers over the last few releases have shared anything from how they've done it. | 19:56 |
| * mordred pondering | 19:57 | |
| smcginnis | fungi: The noop jobs from my local repos do show up in Builds. | 19:57 |
| mordred | oh good, that's good at least | 19:57 |
| fungi | okay | 19:57 |
| smcginnis | So the stream is coming in from opendev, just not triggering on what I had expected it to. | 19:57 |
| mordred | smcginnis: your issue is making me feel dumb :) | 19:58 |
| fungi | i see, the scheduler debug log should have some fairly verbose info on what it did around the event | 19:58 |
| smcginnis | Hah! | 19:58 |
| smcginnis | fungi: I just started with the instructions from https://zuul-ci.org/docs/zuul/admin/quick-start.html | 19:59 |
| smcginnis | Is there a good way to view those logs that have scrolled off in the window I ran docker-compose from? | 19:59 |
| corvus | smcginnis: okay, i have all 3 of your config files up: http://paste.openstack.org/show/752551/ and http://paste.openstack.org/show/752549/ and http://paste.openstack.org/show/752550/ -- what job should be running but is not? | 19:59 |
| fungi | i believe docker-compose has options to fetch log history | 19:59 |
| *** jamesmcarthur has quit IRC | 19:59 | |
| smcginnis | corvus: My expectation was that a patch to opendev/ci-sandbox that would receive a Zuul +1 would then trigger a noop job in my thirdpartyci pipeline. | 20:00 |
| mattw4 | Hi again Zuul team, more questions re: using roles & jobs from other repos: I iteratively added dependencies that popped up in Zuul error notifications, but now I'm seeing errors saying " Unable to freeze job graph: The nodeset "openstack-single-node-bionic" was not found. | 20:00 |
| mattw4 | " Can I exclude or override the nodeset? I have "nodeset: nodes: -name: ubuntu-bionic\nlabel: ubuntu-bionic" defined in my job definition, but still seeing the error. | 20:00 |
| fungi | mattw4: you can specify the nodeset you want in your job or in a variant of an included job | 20:01 |
| pabelanger | tobiash: more github issues :( http://paste.openstack.org/show/752552/ | 20:02 |
| pabelanger | connection timeouts | 20:02 |
| pabelanger | think there is infra issue going on righ tnow | 20:02 |
| mattw4 | fungi: I've added "nodeset: nodes: -name: ubuntu-bionic\nlabel: ubuntu-bionic" to my job definition. Isn't that the correct place? | 20:02 |
| pabelanger | zuul.o.o, also has them | 20:03 |
| corvus | smcginnis: this should not be the issue, but you probably want "approval" rather than "require-approval" there, otherwise it will trigger everytime someone adds a comment after zuul has left +1 (presumably you only want to trigger when the +1 is left). | 20:04 |
| corvus | smcginnis: i'm also stumped; i second the suggestion that you collect the scheduler logs so we can see what it's thinking | 20:04 |
| smcginnis | corvus: Oh, thanks, I'm sure I would have come back asking about that "approval" versus "require-approval" one. | 20:05 |
| smcginnis | Getting the logs now. | 20:05 |
| mattw4 | fungi: my job definition and zuul tenant configuration in a paste: http://paste.openstack.org/show/752553/ | 20:06 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Return javascript content artifact records to Zuul https://review.opendev.org/663056 | 20:06 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Return python artifact records to Zuul https://review.opendev.org/663053 | 20:07 |
| corvus | mattw4, smcginnis: there's a possibility the two of you are working on similar projects :) | 20:10 |
| smcginnis | corvus: Should a recheck-initiated +1 from zuul be treated the same as the initial +1? | 20:10 |
| mattw4 | corvus: I was thinking the same thing :) smcginnis: we should chat! | 20:11 |
| smcginnis | mattw4: You look like you may be a little further along than me. :) | 20:11 |
| smcginnis | corvus: Here's the logs: http://paste.openstack.org/show/752554/ | 20:13 |
| corvus | smcginnis: i think in opendev's gerrit, if there's already a zuul+1 and zuul leaves a second +1 after a recheck, the approval vote will not be included in the event. | 20:13 |
| corvus | smcginnis: (if it changed from a -1 to a +1, it would be, but if the approval vote is the same as the existing vote it's not included) | 20:14 |
| smcginnis | corvus: Ah, OK. | 20:14 |
| corvus | smcginnis: that is expected to change in a later version of gerrit | 20:14 |
| mattw4 | smcginnis: only a tiny bit ;) I'd be happy ti discuss our experience if you want. I intend to write something up, but I need to get it working first!! | 20:14 |
| corvus | smcginnis: given that, you may want to keep using "required-approval" for testing :) | 20:15 |
| smcginnis | mattw4: Definitely ping me if you would like someone to read through your docs. | 20:15 |
| smcginnis | corvus: I think ultimately I wouldn't want it to retrigger on that, just needed for this test. | 20:15 |
| smcginnis | I will abandon that patch, submit a new one, and grab the logging from that. | 20:16 |
| mattw4 | smcginnis: sounds good | 20:16 |
| corvus | smcginnis: i think if you "docker-compose pull" you will get some nice new logging with event ids that can be traced through multiple steps | 20:16 |
| tobiash | corvus, pabelanger, clarkb: yesterday I learned from a github engineer that getting a pr by sha is much cheaper when using graphql than using the search rest api. If we'd switch to that we might be able to remove the cache workaround in getpullbysha. | 20:17 |
| smcginnis | corvus: https://etherpad.openstack.org/p/zuul-thirdpartyci Zuul +1 around line 161, "DEBUG zuul.Pipeline.example-tenant.thirdpartyci: Finished queue processor: thirdpartyci (changed: False)" shortly after. | 20:18 |
| pabelanger | tobiash: yah, when I talked with github engineer, they also say graphql was the solution for something else | 20:19 |
| corvus | smcginnis: there should be more logs after that; the gerrit driver has a rolling 5 second delay | 20:19 |
| pabelanger | I really need to learn how the event system work at github, because I seem to get getting same events for this PR: https://github.com/ansible/ansible/pull/50201/commits | 20:20 |
| corvus | smcginnis: so it's not going to do anything with that event until 20:16:55 | 20:20 |
| pabelanger | but haven't received any others, so not sure if there is a serial issue or outage | 20:20 |
| pabelanger | I'm up to e6a905e | 20:20 |
| pabelanger | so only 2 more events, if I am right | 20:21 |
| pabelanger | then, to see what zuul does | 20:21 |
| tobiash | graphql is fully supported since github enterprise 2.15 which will be the oldest supported release in a few weeks so we may be able to require at least this version and start using graphql for some tasks | 20:21 |
| corvus | tobiash: sounds good | 20:21 |
| mordred | tobiash: cool | 20:22 |
| jkt | ...and executor runs its stuff within bubblewrap, which means that `curl` won't have access to its certificates :( | 20:22 |
| smcginnis | corvus: Well, I did a docker-compose pull, and now I'm just getting "Cannot resolve zk: [Errno -2] Name or service not known" errors. | 20:22 |
| fungi | i think we map those in on our executor | 20:22 |
| mordred | yah | 20:22 |
| fungi | jkt: ^ | 20:22 |
| smcginnis | I may need to start fresh and get back to this. | 20:23 |
| tobiash | btw, I have the github event parallelization in production for two days now and it works like a charm, no event queuing anymore | 20:23 |
| mordred | jkt: the bubblewrap bindmounts are configurable | 20:23 |
| mordred | tobiash: woot! | 20:23 |
| mordred | jkt: https://opendev.org/opendev/puppet-zuul/src/branch/master/templates/zuulv3.conf.erb#L68-L75 and https://opendev.org/opendev/system-config/src/branch/master/manifests/site.pp#L780-L782 are what we're doing in opendev | 20:25 |
| jkt | mordred: thanks! | 20:26 |
| corvus | mordred, fungi: topic:opendev-tarballs is ready for review | 20:27 |
| fungi | thanks!!! | 20:27 |
| mattw4 | Could somebody take a look at my job definitions and tenant config? I can't figure out how to override the nodeset for devstack: http://paste.openstack.org/show/752553/ | 20:27 |
| corvus | mattw4: that looks right to me. can you paste a larger log snippet from the error? | 20:28 |
| mattw4 | corvus, sure, just a sec | 20:28 |
| jkt | mordred: is there a reason for listing /var/lib/zuul/ssh in there? "My system works fine without it", but OTOH I don't see these explicitly getting whitelisted in the source | 20:35 |
| corvus | jkt: i think we have some old jobs which used a key that was in there. in general, it's probably a bad idea to include it. once we can drop it we should. | 20:36 |
| jkt | ok, I just wanted to ensure that this cannot be realted to the "regular" ansible-ified key provisioning | 20:36 |
| pabelanger | tobiash: Hmm, there might be something going on with github3.py, restarting zuul.a.c, seems to have improved my api.github.com timeout issues | 20:37 |
| pabelanger | will have to debug this more | 20:37 |
| pabelanger | with that, I am out for the day. It would be awesome if humans could check out ML about host-key-checking patch | 20:39 |
| mattw4 | corvus: here is the gerrit error and Zuul log from after a recheck on my patch trying to override the nodeset: http://paste.openstack.org/show/752555/ | 20:42 |
| corvus | mattw4: hrm, okay, i guess since the parent uses the nodeset, zuul still needs it to be defined. i guess you'll need to create a "dummy" nodeset for it. it doesn't have to be real -- it won't be used -- it just has to have the name it's looking for. | 20:45 |
| corvus | mattw4: sorry, i realize that's slightly counter-intuitive. hopefully we can fix that someday. | 20:46 |
| mattw4 | corvus: I'll give it a shot! | 20:46 |
| mattw4 | corvus: I added a nodeset to by zuul.yaml similar to that defined here: https://github.com/openstack/devstack/blob/master/.zuul.yaml#L1-L9 | 20:55 |
| mattw4 | but ubuntu-bionic instead of xenial | 20:55 |
| mattw4 | hrm...different errors now: setup-devstack setup-devstack : ERROR Unable to find role in /tmp/tmp82c4m80r/7ecfdcb50f7347e7a6b6da39f7e7e244/ansible/pre_playbook_3/role_1/requirements | 20:57 |
| *** pcaruana has quit IRC | 20:59 | |
| *** hashar has quit IRC | 21:09 | |
| *** dmsimard has quit IRC | 21:16 | |
| *** dmsimard8 has joined #zuul | 21:16 | |
| *** dmsimard8 is now known as dmsimard | 21:16 | |
| *** tjgresha has quit IRC | 21:18 | |
| mattw4 | Hi again Zuul-ers...I'm seeing a new error when running devstack roles in 3rd party CI: When executing the pre-task "Gather minimum local MTU", I see the error "fatal: [ubuntu-bionic]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: \'ansible_interfaces\' is undefined\\n\\nThe error appears to have been | 22:08 |
| mattw4 | in \'/tmp/tmpv4zde86w/e7ed5b01b1f24259a42c719afa19bb5e/untrusted/project_2/opendev.org/openstack/devstack/playbooks/pre.yaml\': line 3, column 7, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be:\\n\\n pre_tasks:\\n - name: G | 22:08 |
| mattw4 | ather minimum local MTU\\n ^ here\\n"}". I checked the ansible facts that are produced in the executor log and no ansible_interfaces is present. Is this an ansible version issue or can I ask Zuul to produce these details? | 22:08 |
| jkt | fungi: thanks a lot for your help, I think I'm mostly done now. Here's the playbook which drives this promote job: https://gerrit.cesnet.cz/plugins/gitiles/ci/project-config/+/master/playbooks/base/promote-artifacts.yaml | 22:08 |
| jkt | fungi: this role fetches stuff from previous "check" runs and prepares them at the executor, https://gerrit.cesnet.cz/plugins/gitiles/ci/zuul-jobs-cesnet/+/master/roles/download-artifacts-from-check/tasks/main.yaml | 22:09 |
| jkt | fungi: and finally, this one uploads the artifacts to swift, https://gerrit.cesnet.cz/plugins/gitiles/ci/zuul-jobs-cesnet/+/master/roles/upload-artifacts-swift/tasks/main.yaml | 22:10 |
| mattw4 | fungi, corvus: do you have ani ideas on the error condition I posted above ^^^^? | 22:10 |
| jkt | (I don't feel like changing these `jq` filters to Ansible filters, apparently the syntax is a bit different, so meh.) | 22:10 |
| corvus | mattw4: i don't know why ansible_interfaces is not defined, sorry | 22:13 |
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 22:14 | |
| fungi | tristanC: dmsimard: is one of you handling software factory ci? seeing "ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/tmp/fake-tox/.tox/dist/MagnificentElephantInKenya-0.1559765279.zip'" in the failing unit test on https://review.opendev.org/663056 | 22:19 |
| fungi | i'm assuming it's not caused by that change, as i don't see how it would be | 22:30 |
| mordred | fungi: I don't know what MagnificentElephantInKenya is - but I want to assume pip installing it causes a new Magnificent Elephant to spring to existence in Kenya | 22:54 |
| fungi | unfortunately it probably does not | 23:06 |
| fungi | i've been assuming it's a name someone made up on the spot to represent a test package | 23:06 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!