| * mnaser will bring it up tomorrow | 00:11 | |
| fungi | happy to discuss more tomorrow, for sure | 00:16 |
|---|---|---|
| fungi | it does seem like something we should find a way to make runnable from an executor | 00:16 |
| fungi | after all, executors can already push git refs to job nodes over ssh | 00:16 |
| fungi | fundamentally the same problem, i expect? | 00:17 |
| openstackgerrit | Tristan Cacqueray proposed zuul/nodepool master: config_validator: refactor the schema to a static method https://review.opendev.org/718582 | 00:26 |
| clarkb | fungi: ya it should be doable for that very reason | 00:28 |
| clarkb | I expect the process will be add ssh key into agent then push | 00:28 |
| *** sanjayu_ has quit IRC | 00:42 | |
| *** cdearborn has quit IRC | 00:50 | |
| *** swest has quit IRC | 01:24 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: Add role https://review.opendev.org/717639 | 01:32 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: export ensure_pip_virtualenv_command https://review.opendev.org/718224 | 01:32 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: fetch-zuul-cloner: use ensure-pip https://review.opendev.org/717882 | 01:32 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: fetch-subunit-output test: use ensure-pip https://review.opendev.org/718225 | 01:32 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: use ensure-pip role https://review.opendev.org/717663 | 01:32 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 01:32 |
| *** swest has joined #zuul | 01:41 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: export ensure_pip_virtualenv_command https://review.opendev.org/718224 | 01:45 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: fetch-zuul-cloner: use ensure-pip https://review.opendev.org/717882 | 01:45 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: fetch-subunit-output test: use ensure-pip https://review.opendev.org/718225 | 01:45 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: use ensure-pip role https://review.opendev.org/717663 | 01:45 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 01:45 |
| *** corvus has quit IRC | 01:48 | |
| *** corvus has joined #zuul | 01:49 | |
| *** ysandeep|away is now known as ysandeep|rover | 01:52 | |
| *** Goneri has quit IRC | 01:59 | |
| *** ysandeep|rover is now known as ysandeep|afk | 03:09 | |
| *** bhavikdbavishi has joined #zuul | 03:16 | |
| *** bhavikdbavishi has quit IRC | 03:24 | |
| *** bhavikdbavishi has joined #zuul | 03:32 | |
| *** bhavikdbavishi1 has joined #zuul | 03:37 | |
| *** bhavikdbavishi has quit IRC | 03:38 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:38 | |
| *** ysandeep|afk is now known as ysandeep | 04:00 | |
| *** evrardjp has quit IRC | 04:37 | |
| *** evrardjp has joined #zuul | 04:37 | |
| *** logan- has quit IRC | 04:45 | |
| *** logan_ has joined #zuul | 04:47 | |
| *** logan_ is now known as logan- | 04:48 | |
| *** ysandeep is now known as ysandeep|rover | 04:56 | |
| *** sgw has quit IRC | 05:01 | |
| tobiash | mnaser: you can totally run nodeless jobs. However what you can do on an executor is very limited (inside an untrusted playbook). E.g. the git push is an arbitrary command and cannot be run locally in an untrusted context. | 05:38 |
| openstackgerrit | Merged zuul/zuul-jobs master: tox: Don't inline python warnings https://review.opendev.org/718554 | 05:39 |
| *** reiterative has quit IRC | 05:42 | |
| *** reiterative has joined #zuul | 05:43 | |
| *** sanjayu_ has joined #zuul | 06:06 | |
| *** dpawlik has joined #zuul | 06:07 | |
| *** sanjayu__ has joined #zuul | 06:09 | |
| *** sanjayu_ has quit IRC | 06:12 | |
| *** bhavikdbavishi has quit IRC | 06:19 | |
| *** bhavikdbavishi has joined #zuul | 06:33 | |
| *** avass is now known as Guest67721 | 06:36 | |
| *** avass has joined #zuul | 06:37 | |
| *** dpawlik has quit IRC | 06:37 | |
| *** dpawlik has joined #zuul | 06:39 | |
| *** sshnaidm|afk is now known as sshnaidm | 06:55 | |
| *** jcapitao has joined #zuul | 06:57 | |
| *** gtema has joined #zuul | 06:57 | |
| *** zxiiro has quit IRC | 07:01 | |
| *** rfolco has quit IRC | 07:11 | |
| *** rpittau|afk is now known as rpittau | 07:18 | |
| *** tosky has joined #zuul | 07:42 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add --validate-tenants option to zuul scheduler https://review.opendev.org/542160 | 07:45 |
| *** zbr has quit IRC | 07:46 | |
| *** zbr has joined #zuul | 07:46 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add --validate-tenants option to zuul scheduler https://review.opendev.org/542160 | 07:48 |
| *** bhavikdbavishi has quit IRC | 07:52 | |
| *** threestrands has quit IRC | 08:00 | |
| *** ysandeep|rover is now known as ysandeep|lunch | 08:10 | |
| *** bhavikdbavishi has joined #zuul | 08:10 | |
| *** bhavikdbavishi has quit IRC | 08:24 | |
| *** bhavikdbavishi has joined #zuul | 08:25 | |
| *** newbie761 has joined #zuul | 08:38 | |
| newbie761 | Hi guys! I have a very newbie-sh question for you. | 08:39 |
| newbie761 | Is there a way for to tell the executor to properly initialize/update a repo submodules when it is preparing the workspace for the build nodes? | 08:40 |
| newbie761 | I can probably do that in the playbook, cloning the submodules from the remote repo, but it would make sense to use Zuul caches etc. like for any required project | 08:40 |
| avass | newbie761: I think submodules is something zuul does not handle properly | 08:59 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add --validate-tenants option to zuul scheduler https://review.opendev.org/542160 | 09:01 |
| avass | newbie761: found the log of the discussion we had when we planned to do something with submodules: http://eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2020-02-27.log.html | 09:02 |
| *** tobias-urdin has quit IRC | 09:02 | |
| *** dpawlik has quit IRC | 09:10 | |
| *** sanjayu__ is now known as saneax_lunch | 09:14 | |
| *** ysandeep|lunch is now known as ysandeep|rover | 09:18 | |
| *** hashar has joined #zuul | 09:20 | |
| jkt | newbie761: I have a playbook for this | 09:20 |
| jkt | newbie761: (a role, actually): https://gerrit.cesnet.cz/plugins/gitiles/ci/zuul-jobs-cesnet/+/refs/heads/master/roles/git-submodules/tasks/main.yaml | 09:20 |
| jkt | newbie761: it needs https://gerrit.cesnet.cz/plugins/gitiles/ci/zuul-jobs-cesnet/+/refs/heads/master/roles/zuul-var-to-json/tasks/main.yaml as well | 09:21 |
| newbie761 | thanks guys, I will look into the links you provided! | 09:26 |
| jkt | can I configure the gerrit reporter to link to zull-web's buildset results? | 09:27 |
| jkt | I'm reading zuul/reporter/, and there doesn't seem to be a thingy for that | 09:28 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Enables whitelisting and configuring callbacks https://review.opendev.org/717260 | 09:30 |
| openstackgerrit | Albin Vass proposed zuul/zuul master: Enables whitelisting and configuring callbacks https://review.opendev.org/717260 | 09:32 |
| avass | jky: I think it's in the tenant config, let me check | 09:39 |
| avass | jkt: I guess this is what you want: https://zuul-ci.org/docs/zuul/reference/tenants.html#attr-tenant.report-build-page | 09:39 |
| tobiash | zuul-maint: am I right that nodepool requests checksums from dib when building images but actually never uses them? If they have no use we might want a way to disable them as this then just adds up to the image build time (especially for larger images) | 09:40 |
| *** dpawlik has joined #zuul | 09:49 | |
| jkt | avass: that's something else if I'm reading the code directly. It's used only for per-job results, and I do not want to change these. I would like to keep them (I'm happy with my success-url and failure-url tweaks, for example) | 09:54 |
| jkt | avass: but I'd like my final build result, as reported to gerrit, to also include a link to the buildset landing page listing all jobs' results | 09:55 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Report retried builds in a build set via mqtt. https://review.opendev.org/632727 | 09:57 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Report retried builds via sql reporter. https://review.opendev.org/633501 | 09:57 |
| avass | jkt: ah, yeah not sure if that's possible at the moment then | 10:01 |
| avass | jkt: unless you do it from a job | 10:01 |
| *** rpittau is now known as rpittau|bbl | 10:13 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add optional support for circular dependencies https://review.opendev.org/685354 | 10:15 |
| *** saneax_lunch is now known as saneax_ | 10:34 | |
| *** gtema has quit IRC | 10:36 | |
| *** newbie761 has quit IRC | 10:42 | |
| jkt | avass: seems that I can only return stuff for reach job | 10:44 |
| jkt | avass: I guess I could inject an extra node-less job which just points to the executor :), but meh | 10:44 |
| jkt | I'll send a patch which adds formatting support for success-message and failure-message, that sounds better IMHO | 10:45 |
| *** hashar has quit IRC | 10:50 | |
| *** jcapitao is now known as jcapitao_lunch | 10:59 | |
| *** mgoddard has quit IRC | 11:02 | |
| *** hashar has joined #zuul | 11:08 | |
| *** hashar is now known as hasharLunch | 11:09 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Remove unused function prune in merger https://review.opendev.org/718670 | 11:24 |
| *** bhavikdbavishi has quit IRC | 11:27 | |
| *** rfolco has joined #zuul | 11:34 | |
| *** hasharLunch is now known as hashar | 11:39 | |
| *** ysandeep|rover is now known as ysandeep|coffee | 11:47 | |
| *** dpawlik has quit IRC | 11:56 | |
| *** rpittau|bbl is now known as rpittau | 11:57 | |
| *** ysandeep|coffee is now known as ysandeep|rover | 12:00 | |
| *** dpawlik has joined #zuul | 12:08 | |
| *** jcapitao_lunch is now known as jcapitao | 12:13 | |
| *** igordc has quit IRC | 12:18 | |
| *** bhavikdbavishi has joined #zuul | 12:43 | |
| *** cdearborn has joined #zuul | 12:59 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: dhall-diff: add new job https://review.opendev.org/718694 | 13:01 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: dhall-diff: add new job https://review.opendev.org/718694 | 13:11 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: dhall-diff: add new job https://review.opendev.org/718694 | 13:22 |
| *** gtema has joined #zuul | 13:23 | |
| *** vblando has joined #zuul | 13:24 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: dhall-diff: add new job https://review.opendev.org/718694 | 13:29 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: dhall-diff: add new job https://review.opendev.org/718694 | 13:34 |
| mnaser | https://review.opendev.org/#/c/717507/ -- can someone have a look at this? ianw mentioned that we should test the role but i'm not sure how we'd go about it | 13:36 |
| mnaser | i'll gladly do it as i'd like this to land | 13:36 |
| mnaser | but just need some help :X | 13:36 |
| tristanC | mnaser: i replied to ianw concerns. I'd also like to land this :) | 13:38 |
| *** gtema has quit IRC | 13:41 | |
| tristanC | tobiash: isn't dib checksum toggleable with the DIB_CHECKSUM env-vars? | 13:42 |
| *** sgw has joined #zuul | 13:44 | |
| *** bhavikdbavishi has quit IRC | 13:50 | |
| *** avass is now known as Guest78227 | 13:53 | |
| *** avass has joined #zuul | 13:53 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: dhall-diff: add new job https://review.opendev.org/718694 | 14:00 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Move zuul-quick-start requires to pipeline and reparent https://review.opendev.org/713545 | 14:02 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Use ZK TLS in quickstart https://review.opendev.org/712817 | 14:02 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Add requires to zuul-quick-start job https://review.opendev.org/718708 | 14:03 |
| AJaeger | https://review.opendev.org/#/c/718284/ is a zuul-jobs change to "ensure-tox: make idempotent and update testing | 14:04 |
| AJaeger | " - it looks good to me but since that is a central role, wanted to get an extra pair of eyes on it, please | 14:05 |
| AJaeger | (and on the complete stack) | 14:05 |
| AJaeger | zuul-maint, config-core ^ | 14:05 |
| tobiash | tristanC: nodepool hard codes it via commandline | 14:10 |
| *** gtema has joined #zuul | 14:12 | |
| tristanC | tobiash: i can't find the hardcoded value, where is it defined? | 14:13 |
| tobiash | tristanC: https://opendev.org/zuul/nodepool/src/branch/master/nodepool/builder.py#L801 | 14:14 |
| *** Goneri has joined #zuul | 14:14 | |
| clarkb | tobiash: they are supposed to be used by the upload process | 14:14 |
| tobiash | clarkb: it doesn't look that they are though | 14:15 |
| clarkb | however, I don't know that glance actually doesanything with thatinfo in the end so it may not be useful | 14:15 |
| tristanC | tobiash: oh i see, i got confused with the DIB_CHECKSUM reference in https://opendev.org/zuul/nodepool/src/branch/master/doc/source/configuration.rst | 14:16 |
| clarkb | tobiash: fwiw we also expose those checksums to our users so they can check the images when downlodaing them directly | 14:16 |
| clarkb | but we could toggle that on optionally | 14:16 |
| clarkb | https://nb01.openstack.org/images/ for example | 14:16 |
| tobiash | clarkb: so you host them as well | 14:17 |
| tobiash | in this case it makes sense | 14:17 |
| tristanC | AJaeger: left a comment, what is the purpose of doing `set_fact: {tox_executable: "{{ tox_executable }}"}` ? | 14:17 |
| Shrews | tobiash: that option should produce the .md5/.sha256 files that are then used on upload. check the DibImageFile (or whatever the name is) class | 14:18 |
| tobiash | Shrews: I found no uses though of that attributes of that class. Is that hidden in the openstacksdk? | 14:19 |
| *** gtema has quit IRC | 14:19 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Remove David Shrewsbury from Zuul Maintainers https://review.opendev.org/718712 | 14:19 |
| tristanC | AJaeger: nevermind, it seems like using cacheable makes it relevant | 14:19 |
| AJaeger | tristanC: thanks for reviewing | 14:20 |
| tobiash | I'm just wondering if something bad happens if I inhibit this in our dib wrapper (we have images where checksumming takes an hour) | 14:20 |
| Shrews | tobiash: it should be used here: https://opendev.org/zuul/nodepool/src/branch/master/nodepool/builder.py#L1079 | 14:20 |
| Shrews | not sure where that's set. you'll have to dig | 14:20 |
| tobiash | oh weird, no idea why I didn't find that... moar coffee... | 14:21 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Add Albin Vass to zuul-jobs maintainers https://review.opendev.org/718713 | 14:21 |
| clarkb | tobiash: Shrews I think you can disable it without any major issues though sdk might calculate a checksum as part of the upload if yo udo (so may not be faster) | 14:23 |
| corvus | i think that's the main motivation -- we do a lot of uploads, but only one image build | 14:24 |
| *** y2kenny has joined #zuul | 14:30 | |
| fungi | and so we only have to checksum the image once instead of for each provider where we upload it (though also i wonder what glance does if you provide an invalid checksum, i hate to say it but the checksum handling inside glance is not thoroughly baked because of needing to support use cases like images hosted at alternative urls not uploaded through its api) | 14:31 |
| fungi | oh actually if memory serves, what happens in that case is glance may not care, but then nova consults the checksum when pulling into its cache. though in boot from volume scenarios i have no idea what goes on with the checksums | 14:32 |
| fungi | however, also glance may be set up to autoconvert your images to a different format, in which case the checksum is bunk anyway | 14:33 |
| tobiash | in our case checksumming is disk bound (700gb), we'll try out how it performs if we omit the checksum parameter | 14:33 |
| clarkb | fungi: ya the value nova checks is calculated by glance | 14:33 |
| clarkb | I don't know that glance rejects images with a mismatched checksum | 14:33 |
| clarkb | iirc I discovered it didn't becaus ewe had na image fail due ot partial upload but glance used it anyway | 14:34 |
| *** gtema has joined #zuul | 14:34 | |
| clarkb | then I tried to work with sdk to get that addressed on the client side but we realized we can't trust the checksum glance gives to us becaues it oculd be for a differently formatted image :/ | 14:34 |
| clarkb | all that to say the hecksums are probably functionally useless | 14:35 |
| corvus | hecksums. hacksums. indeed. | 14:35 |
| clarkb | (except for when you serve the content somewhere elseand that may want to verify the sums like we do serving our images directly to users) | 14:35 |
| tobiash | btw, we're currently exporing a different way of building images. In this case the image is built within a ci job and pushed into swift. And we created a dib wrapper that pulls the latest image from swift when nodepoo 'builds' it | 14:37 |
| tobiash | with some images we have this is more efficient as we can re-use already existing caches from the earlier image in the build process and don't have to push/pull hundreds of gb to/from artifactory | 14:38 |
| clarkb | tobiash: fwiw dib enables you to cache artifacts like that on the dib builder | 14:38 |
| clarkb | the source-repository element (maybe poorly named) will cache any thing it manages | 14:39 |
| tobiash | clarkb: those caches are regenerated every night and need to be refetched completely in our current case | 14:39 |
| tobiash | while reusing the target image it can build and directly put the cache into the image without some hosting service in the middle | 14:40 |
| tobiash | but we're still in tryout phase of that | 14:40 |
| clarkb | ya you're using the entire image as an artifact there | 14:40 |
| fungi | interestingly, it sounds like a return to what we did with the old devstack-gate disk image jenkins jobs many years ago, which we eventually replaced with nodepool | 14:44 |
| tobiash | we're not throwing the old way away, just exploring several options | 14:45 |
| clarkb | also probabl worth mentioning that dib can build from existing images too | 14:45 |
| tobiash | and via a dib wrapper it fits pretty close and without changes to nodepool into the current system | 14:45 |
| tobiash | clarkb: however I don't want to run a software build in our builders ;) | 14:46 |
| clarkb | tobiash: thats fair :) a lot of people don't realize this can be done so I try to point it out. By default the base elements build from pstream images and that isn't really expressed so people odn't realize its how it works | 14:47 |
| clarkb | this came up a lot several years back when a group was arguing dib was bad because you had to rebuild from scratch every time :/ | 14:47 |
| clarkb | (which sin't true, dib has many problems but I don't think that is one) | 14:47 |
| tobiash | cool, I'll keep that in mind for when I have a usecase for this :) | 14:48 |
| clarkb | also probably doesn't help that we choose to use the -minimal elements which do build from scratch | 14:48 |
| *** bhavikdbavishi has joined #zuul | 14:50 | |
| *** ysandeep|rover is now known as ysandeep|afk | 14:51 | |
| y2kenny | Are there any documentation on the security restrictions for job playbook? I think I've read somewhere that zuul uses a custom ansible command module but I am wondering if there is something that document the limits more comprehensively. So far I have ran into two problems that, I think, may be security related but I am not 100%. | 14:54 |
| y2kenny | The first one is running remote command. I tried to run pwd or ls to try to figure out what is going on but the run didn't return anything. Then I try doing ls with a different path and looks like I can only do some of the command under project.src_root? Is there other path that I can have activity on the remote? I am wondering because I am | 14:55 |
| y2kenny | trying to organize the artifact output. The ensure-output-path role seems to default to <ansible-user>/zuul-output/ which doesn't work in my case because I am remoting into a container (as root user) and the default working dir for the container image is / (so the src_root is /src.) | 14:55 |
| openstackgerrit | Paul Albertella proposed zuul/zuul-jobs master: Add Bazel build and install roles https://review.opendev.org/693513 | 14:57 |
| clarkb | y2kenny: that is a good question. https://zuul-ci.org/docs/zuul/reference/glossary.html#term-untrusted-project mentions there are restrictions but not what they are | 14:57 |
| clarkb | corvus: ^ do you know if we document that more thoroughly somewhere else? | 14:58 |
| corvus | tobiash: we're also exploring using container building in dib -- https://review.opendev.org/700083 (basically use docker/podman for most of the building with dib handling making it a bootable image) | 15:04 |
| *** bhavikdbavishi has quit IRC | 15:04 | |
| *** gtema has quit IRC | 15:04 | |
| tobiash | corvus: cool | 15:05 |
| corvus | clarkb, y2kenny: there's some discussion here: https://zuul-ci.org/docs/zuul/reference/config.html | 15:05 |
| *** zxiiro has joined #zuul | 15:05 | |
| corvus | but that may still not be comprehensive enough | 15:06 |
| y2kenny | I see. | 15:06 |
| y2kenny | can you guys confirm that I am confined to within the src_root on the remote? | 15:06 |
| corvus | y2kenny: the only restrictions are for things that run on the executor; you should have completely unrestricted access to run anything on a remote node | 15:07 |
| y2kenny | corvus: then what I am seeing is very odd because when I run command: pwd, ansible return 'OK' but nothing else. | 15:08 |
| y2kenny | I thought that was because I was doing pwd on / | 15:09 |
| corvus | y2kenny: this is on a pod that you created and then added to the inventory? the zuul console daemon may not be running there | 15:09 |
| y2kenny | OH | 15:10 |
| y2kenny | ok. that may explains it. | 15:10 |
| corvus | y2kenny: so it'll be like before we added support to k8s for that. you can probably invoke that role on your new container after adding it to the inventory | 15:10 |
| y2kenny | but how come ls -la for src_root works? | 15:10 |
| corvus | y2kenny: you should still be able to see the output in the "console" tab on the build page (ie, it ends up in the json file, just not the txt file) | 15:10 |
| y2kenny | corvus: ok... let me check | 15:11 |
| y2kenny | ok yes | 15:12 |
| y2kenny | I see it | 15:12 |
| y2kenny | under stdout_lines | 15:12 |
| jkt | heh, I just spent half an hour figuring out how to reset my swift containers with Access-Control-Allow-Origin: *, just to find out that since c5e7174c2f1c359877bd917dc557c8e52df66bc5, Zuul does this by default when it creates these containers | 15:12 |
| jkt | looks like I've been using Zuul v3 for "too long" :) | 15:12 |
| corvus | jkt: hopefully soon you'll be able to put an end to that by upgrading to zuul v4 :) | 15:14 |
| jkt | corvus: but seriously, it would have helped me if there were release notes (or something!) for zuul-jobs | 15:14 |
| jkt | how does http://paste.openstack.org/show/791888/ look? | 15:15 |
| *** bhavikdbavishi has joined #zuul | 15:16 | |
| corvus | jkt: yes, that's a good point re release notes | 15:16 |
| corvus | jkt: i'd put the swift-specific info in the swift role, and then also indicate that it's set by default but manually created containers may need a command like that. (there's an entirely parallel GCE role now too (it also sets the cors header on creation)) | 15:17 |
| y2kenny | corvus: this is the observation I see: http://paste.openstack.org/show/791889/ Is that the result of the console daemon not running or the console daemon selectively filter out some stuff? | 15:18 |
| jkt | (half of my time was spent on "hey, what's the relation between 'properties' in `openstack container show` and these X-Container-Meta- prefixes) | 15:18 |
| corvus | jkt: (so maybe ~ the first sentence in the manifest role, then the specifics in the upload role) | 15:18 |
| y2kenny | or may be I am mixing up with all the moving parts :) | 15:18 |
| corvus | mordred: ^ do you know the answer to y2kenny's question? | 15:19 |
| jkt | corvus: I would not have seen that because I've been using my swift roles for more than a year, therefore it was there and public, just the CORS bit was missing | 15:19 |
| corvus | jkt: yeah, that's why i'm suggesting cross-referencing them | 15:19 |
| mordred | corvus: looking | 15:20 |
| jkt | okay, I'll write something and upload | 15:20 |
| corvus | jkt: we don't want the manifest role to accumulate a lot of information about individual object storage systems (we're going to have many of them). so introducing the cors concept in the manifest role, and referencing the swift/gce/(in the future aws, etc) docs for the specifics should help people find stuff and also keep it maintainable | 15:21 |
| corvus | jkt: thanks! :) and thanks for helping find the problems :) | 15:21 |
| jkt | well, happy to help, thanks for an awesome CI system :) | 15:22 |
| mordred | corvus, y2kenny: no - that's really weird | 15:22 |
| y2kenny | mordred: ok... so I wasn't going insane... it was driving me insane last night :) | 15:23 |
| y2kenny | I was trying to pump some output artifact from the buildpod but it keeps not showing up | 15:23 |
| y2kenny | I can't create directory with the default output path, etc. I ended up just putting it under /src and that was ok | 15:24 |
| jkt | y2kenny: just sharing my experience from an year ago -- I'm afraid you'll be struggling with this; a lot of roles were not so flexible when it came to directory naming | 15:25 |
| corvus | y2kenny: i'm curious why ensure-output-dirs doesn't work, do you know the specific error? | 15:25 |
| jkt | y2kenny: my use case was a runc-managed container with some (now long-obsolete) nodepool patch and a r/o rootfs | 15:25 |
| y2kenny | jkt: I see. I want to make sure I know where the walls are so that when I tell my developers to make their own jobs, they don't get frustrated | 15:27 |
| y2kenny | corvus: ensure-output-dirs seems to fail silently... it didn't have any error just that the path was not created. I have another example... let me paste it. | 15:28 |
| y2kenny | corvus: http://paste.openstack.org/show/791890/ | 15:29 |
| corvus | y2kenny: well, it's not failing, right? | 15:32 |
| y2kenny | not failing | 15:32 |
| corvus | y2kenny: if we assume there is a fault with the console output, then we can't trust the null output there | 15:32 |
| corvus | y2kenny: maybe check the console tab for that last command and see if there's a real directory listing there? | 15:32 |
| y2kenny | looking at the output.json, looks like the artifacts directory is not there | 15:33 |
| y2kenny | even though ensure-output-dirs supposed to create it (as far as I understand.) | 15:34 |
| y2kenny | oh wait | 15:35 |
| y2kenny | I misread what I did | 15:35 |
| y2kenny | artifacts directory is there | 15:35 |
| y2kenny | or at least the ls returned . and .. (empty dir) | 15:36 |
| tobiash | tristanC, corvus: In our deployment I'm preparing an improvement to the executor lifecycle on k8s. The plan is to call 'zuul-executor graceful' in a preStop hook. This will make it easy to do rolling upgrades on executors by just deleting the pods as pod deletions will be graceful then. | 15:36 |
| tristanC | tobiash: corvus: that's on my todo list for the operator, and i'm waiting for the current changes to be reviewed and/or merged before adding more changes... | 15:39 |
| corvus | tobiash: that sounds good -- does k8s handle that taking several hours? | 15:40 |
| tobiash | corvus: there is a grace period that can be configured on the container | 15:40 |
| tristanC | tobiash: i am also hoping we could add the prometheus server in zuul services and use it to indicate readyness | 15:41 |
| tobiash | tristanC: do you want a full featured prometheus as a stats driver or for limited use? | 15:45 |
| openstackgerrit | Jan Kundrát proposed zuul/zuul-jobs master: docs: Document CORS requirements of Zuul's web dashboard https://review.opendev.org/718740 | 15:46 |
| corvus | i think something to indicate readiness is a good idea (whether that's prometheus, or just a simple http endpoint) | 15:46 |
| tobiash | tristanC: cause the latest information I remember was that it has a performance impact on the scheduler | 15:46 |
| tobiash | nodepool already has a readiness endpoint btw | 15:46 |
| tristanC | tobiash: right, we realized that prometheus data model is not efficient to replace the existing statsd metric, however i think we can still use prometheus to indicate readyness of zuul service | 15:47 |
| corvus | yeah, i think that would be fine and easy | 15:47 |
| corvus | er | 15:47 |
| tobiash | ++ | 15:48 |
| corvus | i think the nodepool-style ready endpoint would be fine and easy :) | 15:48 |
| tristanC | tobiash: in particular, for the scheduler, it would be useful to be able to query if the tenant configuration has been loaded | 15:48 |
| tristanC | well i haven't looked at the nodepool readiness endpoint, but it sounds similar to what does the prometheus server (i mean the python module) | 15:50 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: WIP: Implement graceful termination for the executor https://review.opendev.org/718157 | 15:50 |
| tristanC | and if it's similar, i don't mind copying the nodepool code, or re-using the prometheus implementation, as you prefer | 15:51 |
| tobiash | in nodepool it was just an additional endpoint cause it already had a web server | 15:51 |
| corvus | it's very simple and adds no extra dependencies, i like it. | 15:52 |
| tobiash | https://review.opendev.org/695001 | 15:52 |
| tobiash | ftr, this was the implementation ^ | 15:52 |
| corvus | mordred: can you look at https://review.opendev.org/716298 ? | 15:54 |
| corvus | tristanC: didn't you have a change to make fetch-output work on kubectl? | 15:56 |
| corvus | tristanC: oh, it's fetch-output-openshift | 15:57 |
| corvus | tristanC: what do you think about merging that into 'fetch-output', and similarly merging 'prepare-workspace-openshift' into prepare-workspace (or is it prepare-workspace-git?). that way we can put all the conditional logic in the roles themselves instead of in the playbooks? | 15:57 |
| corvus | tristanC: (but maybe, for the moment at least, just add fetch-output-openshift to zuul-base-jobs as a conditional like the others?) | 15:58 |
| *** rpittau is now known as rpittau|afk | 15:59 | |
| tristanC | corvus: that would be another solution | 16:00 |
| tristanC | corvus: not sure how it would behave for user already doing the logic in the pre/post playbooks | 16:01 |
| tristanC | it seems like zuul-jobs roles are using more and more include_role. i don't mind moving the logic into the roles, but that makes the playbook less useful... | 16:02 |
| mordred | tristanC: fetch-output should be idemopotent | 16:02 |
| mordred | so if someone runs it elsewhere it should be at worst a re-copy | 16:02 |
| mordred | tristanC: yeah - but playbooks are hard to share - so the more logic we put in playbooks,the harder that logic is to reconsume | 16:03 |
| tristanC | corvus: actually if we make prepare-workspace and fetch-output works with kubectl, then they are no longer usable in untrusted playbook | 16:03 |
| tristanC | mordred: isn't fetch0output already idempotent? | 16:04 |
| corvus | tristanC: i think a simple base playbook would be good; ideally, most people should be able to use the same base playbooks with just a list of a few roles. so yeah, maybe include_role in roles is the way to go, for maximum ease of use, and maximum flexibility. make "prepare-workspace" include_role "prepare-workspace-git" or "prepare-workspace-openshift" depending on the connection. then have playbooks use | 16:04 |
| corvus | the "prepare-workspace" role. | 16:04 |
| tristanC | corvus: what about user performing prepare-workspace in untrusted job? | 16:05 |
| corvus | tristanC: re untrusted? why -- because of the "oc rsync" command? well, they aren't really intended to be used in an untrusted job, right? | 16:05 |
| tristanC | corvus: that's how paul has been doing the base/base-minimal job split in rdo and ansible | 16:06 |
| *** dpawlik has quit IRC | 16:06 | |
| tobiash | fwiw, mirror-workspace-git-repos (which should be used by prepare-workspace-git) should already work with k8s: https://review.opendev.org/711920 | 16:07 |
| corvus | i'm really confused by all the workspace setup roles and could use a good overview :) | 16:07 |
| tobiash | corvus: I guess mirror-workspace-git-repos should be merged into prepare-workspace-git | 16:08 |
| corvus | i just want people to be able to say "role: make-my-git-repos-be-on-the-nodes" and have that happen :) | 16:08 |
| tobiash | afaik mirror-workspace-git-repos was the initial role used by opendev? | 16:08 |
| tristanC | corvus: right now, one can use prepare-workspace and fetch-output in a untrusted base job, if we make them include the oc equivalent that will no longer be the case | 16:08 |
| corvus | tristanC: okay, what do we need to do to fix that? | 16:08 |
| tristanC | corvus: create a `make-my-git-repos-be-on-the-nodes` to rule them all? :-) | 16:09 |
| corvus | tristanC: because we need to be able to put repos on remote nodes, and it sounds like it's useful to have that happen in an untrusted job (that may be useful for cases where people add_host as well) | 16:09 |
| corvus | tristanC: sorry, i mean, how do we make the oc equivalent work on untrusted jobs? | 16:09 |
| tristanC | more seriously, i don't know what is the best strategy moving forward. I don't mind having that ansible_connection logic in the playbook, but i'm also happy with pushing the logic into the role to keep the playbook simpler | 16:10 |
| tobiash | corvus: I guess currently it's a localhost shell task. We might be able to wrap that into a zuul-provided module and allow that on localhost | 16:10 |
| y2kenny | corvus: so that one problem that I ran into using prepare-workspace-openshift is that it relies on zuul.resources entry | 16:11 |
| corvus | tristanC: yes, honestly, i think a top-level role that just includes the right role is the way to solve that. | 16:11 |
| tristanC | corvus: oc requires a command running on the executor. while we could whitelist the use-case, i don't think we should support that for untrusted jobs | 16:11 |
| y2kenny | doing add_host doesn't update it even though the pod is added to the inventory | 16:11 |
| corvus | tobiash: yeah, that could be an option, or maybe we can whitelist that specific command in the command module? | 16:12 |
| corvus | tristanC: why not? | 16:12 |
| corvus | tobiash, tristanC: maybe whitelist "oc rsync /workdir ..." ? | 16:12 |
| y2kenny | I try to use set_fact but set_fact doesn't update across play | 16:12 |
| corvus | y2kenny: ah good point, so as written, it won't work with new resources anyway | 16:13 |
| tobiash | corvus: having that as a module makes it more flexible (e.g. in fetch-output) and easier to validate paths | 16:13 |
| y2kenny | corvus: I pretty much copy and pasted the playbook into my own playbook for output. Or I just manually set_fact before I call the include_role | 16:13 |
| corvus | tobiash: yes, but i'm wary of zuul-specific modules/plugins for things that should be basic ansible tasks... tristanC has an outstanding patch to ansible to get rsync upstream | 16:14 |
| corvus | tobiash: if we make a zuul-specific module for this, we'll grow a small fork of ansible, and it will be hard to undo | 16:14 |
| corvus | (so far, our zuul-specific ansible plugins are not for things that ansible does itself) | 16:15 |
| tobiash | corvus: k, then the whitelist will work too, but we need more care to validate the commandline | 16:15 |
| tristanC | corvus: as tobiash said, if we care to validate the oc command line, then whitelist should work | 16:16 |
| corvus | i think there are helper functions available to us in our ansible modules to split a command line | 16:17 |
| corvus | and i think we could be really specific about this | 16:17 |
| corvus | (like, we don't have to support every possible set of options) | 16:17 |
| corvus | tristanC: how's the progress upstreaming the rsync command? | 16:18 |
| tristanC | corvus: the PR is https://github.com/ansible/ansible/pull/62107 , and i've been told it's not going to happen :( | 16:18 |
| mordred | corvus: I like the whitelist option - and I agree, we can be pretty specific about it | 16:20 |
| corvus | tristanC: oh? is there a record of that decision somewhere? | 16:20 |
| mordred | corvus: sivel's comment from sept 17 | 16:20 |
| corvus | mordred: i read that. i guess i don't understand subtext | 16:21 |
| tristanC | corvus: i think we can proceed with updating `prepare-workspace` to use oc for kubectl connection now, it won't break unstrusted prepare-workspace user because it is already broken for kubectl connection | 16:21 |
| fungi | did it ever get discussed in the core meeting? | 16:22 |
| corvus | mordred: i thought that said "bring it up at a meeting because this is hairy", but i guess it really means "we're not going to do this, but i don't want to say no"? | 16:22 |
| fungi | and yeah, i read those comments as "not now" not "never" | 16:22 |
| corvus | because, if i were at such a meeting, or part of such a conversation, i would say "making it easy to synchronize contents between the controller and the remote host while abstracting away platform differences is a big part of ansible's value proposition and would be hugely beneficial to users" | 16:23 |
| corvus | so i just want to understand why the ansible core team disagrees with that. so that, at the very least, i can understand how they see the project. | 16:24 |
| mordred | yeah - I read it more as the "we're not going to do it but if you want to come argue your case, you're welcome to" | 16:25 |
| y2kenny | Sorry to bud in again... so another weird thing I have been seeing is playbook showing up with no task even though there are definitely tasks in them (and previous pre playbook are successful as far as I can tell.) | 16:25 |
| mordred | my understanding of the core team's pov generally is that they actually are very conservative about abstracting away differences. the package: module was controversial - and you'll note there is not a vcsrepo: module | 16:26 |
| y2kenny | What I was able to do is launch a k8s pod from the executor, and the pod to the inventory with add_host and then run another play. | 16:26 |
| mordred | that said - the composition of the core team changes over time so it's possible that points of view can too | 16:26 |
| mordred | I mean - we have a new friend on that team now :) | 16:26 |
| y2kenny | This all worked in an untrusted project. So I figure I should refactor that into a base job in a trusted project. | 16:27 |
| mordred | corvus: (I obviously agree with your point of view on this - this is just me explaining why I read that comment that way) | 16:27 |
| corvus | mordred: thanks. | 16:29 |
| y2kenny | But when I did that I get many of the subsequent playbook returning with empty task | 16:29 |
| y2kenny | like this: http://paste.openstack.org/show/791893/ | 16:30 |
| jkt | in a scenario where I have just one repo under my control in a Zuul setup, the leaf project, can I put my roles somewhere else than into <root>/roles/? Perhaps something like .zuul.d/roles/ ? | 16:31 |
| corvus | tristanC, mordred, tobiash: okay, so i guess we either whitelist, or, honestly, at this point, any of you could probably convince me at this point to just make a new zuul-specific plugin for this. because if this is a "no" from ansible core, i feel less bad about diverging. | 16:31 |
| corvus | plugin/module/whichever one is best | 16:32 |
| tristanC | corvus: mordred: turns out synchronize is no longer hosted in ansible/ansible and it moved there: https://github.com/ansible-collections/ansible.posix/blob/master/plugins/modules/synchronize.py | 16:32 |
| mordred | corvus: pretty sure it's both | 16:33 |
| tobiash | corvus: i'd be fine with either, but see less risk in security issues with a module so if we need to maintain this for longer I'd be slightly in favor of a module | 16:33 |
| mordred | sync is a mix of action plugin and module | 16:33 |
| corvus | hey... | 16:33 |
| jkt | looking at https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.roles , it seems that I cannot really do much there | 16:33 |
| corvus | this is something we could make standalone and publish to galaxy, right? | 16:33 |
| mordred | because it has to do some dark-magic to make things like "pull" work | 16:33 |
| mordred | corvus: yes | 16:33 |
| mordred | corvus: well.... | 16:34 |
| corvus | collections whatever | 16:34 |
| mordred | corvus: we need to add collections support to zuul first | 16:34 |
| mordred | but then yes | 16:34 |
| corvus | so we could actually have a module which was "synchronize++" which did either 'rsync' or 'oc rsync' | 16:34 |
| mordred | ++ | 16:34 |
| corvus | and any ansible user could use that | 16:34 |
| mordred | yup | 16:34 |
| mordred | we could start just with a simple addition to manage-ansible-venv to add a known list of collections to the ansible install | 16:35 |
| corvus | yep | 16:35 |
| mordred | which wouldn't be as far as "allow jobs to specify this collection" - which is a whole other can of worms | 16:35 |
| mordred | corvus: we could actually publish our other ansible stuff that way too, just for completeness | 16:36 |
| corvus | y2kenny: that doesn't sound right. the only thing i can think of is to remember that trusted projects aren't speculative, so you have to land changes to them before they will take effect. | 16:36 |
| tristanC | aren't collection only supported by ansible-2.9? | 16:36 |
| mordred | yes | 16:36 |
| *** evrardjp has quit IRC | 16:37 | |
| mordred | corvus: I mean - it might be weird -but zuul_console + our callback plugin _could_ be something somebody else would want to use | 16:37 |
| *** evrardjp has joined #zuul | 16:37 | |
| *** jcapitao has quit IRC | 16:37 | |
| openstackgerrit | Merged zuul/zuul-operator master: Add job_volumes CR spec attribute https://review.opendev.org/706642 | 16:38 |
| corvus | tristanC, mordred: but maybe we can define our plugin/module now in such a way that it's forward compatible with collections; so we have it in zuul for now, but move it to collections once we're ready. keep the same name so playbooks still work? | 16:38 |
| corvus | jkt: correct; zuul expects that to be either a collection of roles under roles/ or else the whole repo is a role. you can symlink roles/ to somewhere else though. | 16:39 |
| mordred | corvus: it'll be slightly hard to do that - we'll need a transition period (it's not fully possible to do the thing you're talking about) | 16:39 |
| mordred | corvus: but - I think the basics "just do a normal plugin/module now, publish as collection later" | 16:39 |
| mordred | should cover it | 16:39 |
| corvus | mordred: what's involved in the transition? | 16:39 |
| mordred | namespacing | 16:39 |
| corvus | ack | 16:39 |
| y2kenny | corvus: I did that. so I refactored the pod launching tasks into a base job in the trusted project. Submitted it, and then 're-inherited' the job in the untrusted project to that newly created base job. When I test the job in the untrusted project, seems like all other playbooks were 'zeroed-out'. | 16:40 |
| mordred | there is a mechanism in ansible itself to route from old non-namepsaced names to new namespaced | 16:40 |
| mordred | corvus: but I'm not sure it's accessible to arbitrary external things | 16:40 |
| mordred | I'll go ask | 16:40 |
| * jkt goes to create that extra repo, then | 16:40 | |
| tristanC | fwiw i'm happy with all the above suggestion to make the prepare-workspace and fetch-output feature works on kubectl. please let me know the one you prefer and i can have a look at updating or writting the needed/missing bit to make it happen | 16:43 |
| mordred | corvus: I started asking - and then realzied - we can take care of that ourselves in manage-ansible-venv for a transition period | 16:44 |
| mordred | corvus: once we're installing it as a collection, we can make a symlink to the old non-namespaced name | 16:44 |
| corvus | tristanC, mordred, tobiash: how does this plan look? https://etherpad.openstack.org/p/vXl0TZu3I4 | 16:45 |
| tristanC | mordred: so is it possible to provide fake namespace module (e.g. a role named 'zuul.synchronize' (dot part of the filename)) in ansible<2.9 ? | 16:45 |
| *** ysandeep|afk is now known as ysandeep|out | 16:46 | |
| mordred | tristanC: that's a good question - I think we might be able to do that too | 16:47 |
| mordred | I'll investigate | 16:47 |
| y2kenny | corvus: this is what I mean by zero-ed out: http://paste.openstack.org/show/791895/ even the playbook before one of my pre play has no task in it. | 16:47 |
| y2kenny | corvus: is there a requirement on the pre playbook to have hosts: all (or other restrictions?) | 16:49 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add a zuul-ensure-passwords role https://review.opendev.org/717880 | 16:50 |
| corvus | y2kenny: are there any hosts in your inventory? | 16:50 |
| corvus | y2kenny: iirc, there aren't at the start of the job | 16:50 |
| corvus | y2kenny: your paste showed empty tasks in the pre-k8s playbook, but you didn't paste the pre-k8s playbook | 16:51 |
| y2kenny | corvus: the job has a nodeset definition that includes a namespace label type from k8s provider | 16:51 |
| corvus | y2kenny: right, so "hosts: all" won't run anything since there are no hosts | 16:51 |
| y2kenny | corvus: pre-k8s.yaml is just hosts:all with the prepare-workspace-openshift role in it | 16:52 |
| corvus | y2kenny: but you should be able to add_host and then delegate tasks to that host after adding it, or start a new play in the same playbook and set "hosts:" to whatever you added | 16:52 |
| mordred | corvus: fwiw - I've got collection work already for the openstack colllection we can copy/borrow | 16:52 |
| corvus | y2kenny: add_host doesn't persist across playbooks, but it does persist across plays. so if you add_host in pre-k8s.yaml, you can use "hosts: all" after doing that. but the add_host in pre-launch.yaml won't affect pre-k8s.yaml | 16:53 |
| y2kenny | OOOOOOOOH | 16:53 |
| corvus | y2kenny: my guess is the difference is maybe you are doing things in multiple playbooks that you were doing in the same playbook before | 16:53 |
| y2kenny | I think that's it | 16:53 |
| y2kenny | so I can't really refactor the pod launching from the actual run? | 16:54 |
| corvus | y2kenny: i don't think so; but i think you've got a good use case for maybe being able to update the inventory from a trusted playbook... | 16:56 |
| corvus | y2kenny, tristanC: this is relevant: https://review.opendev.org/590092 | 16:57 |
| corvus | tobiash: ^ maybe we should revisit that? | 16:58 |
| y2kenny | corvus: oh I think I like that. Me directly calling add_host felt hacky to me. | 16:59 |
| corvus | mordred: what would the future standalone synchronize++ collection module be called? | 17:00 |
| corvus | mordred: zuul.synchronize or something like that? | 17:00 |
| tobiash | corvus: wow, that is old, need to wrap my head around that | 17:00 |
| y2kenny | corvus: in addition to updating the inventory, I wonder if zuul.resources can be updated as well? (I will leave a comment in the review) | 17:00 |
| corvus | mordred: would you say "tasks: [zuul.synchronize: {src:..., dst:...}]" ? | 17:00 |
| corvus | y2kenny: yeah, that would be good to think about -- but also, maybe the future prepare-workspace situation won't actually need to use zuul.resources anymore | 17:02 |
| tobiash | corvus: I'm not sure if my comment there is still valid. At least this solves a new use case | 17:02 |
| y2kenny | corvus: ok understood. | 17:02 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Document the missing feature https://review.opendev.org/718755 | 17:02 |
| mordred | corvus: zuul.$something.synchronize | 17:03 |
| corvus | what sort of thing goes in $something? | 17:04 |
| mordred | some other word that we make up - there's just a 2-tiered thing | 17:04 |
| corvus | like 'ansible.posix.synchronize' i guess? | 17:04 |
| mordred | so top level namespace can have more than one collection | 17:04 |
| mordred | yeah | 17:04 |
| corvus | so 'zuul.posix.synchronize' or 'zuul.ansible.synchronize' ? | 17:04 |
| *** gtema has joined #zuul | 17:05 | |
| corvus | mordred: then is that what you put as the task? eg: "tasks: [zuul.posix.synchronize: {src:..., dst:...}]" ? | 17:05 |
| mordred | yes | 17:05 |
| corvus | mordred, tristanC, tobiash: i updated the etherpad https://etherpad.openstack.org/p/vXl0TZu3I4 can you take a look at that and sign off on the plan? | 17:08 |
| *** gtema has quit IRC | 17:09 | |
| tristanC | what would be the new project name, zuul/zuul-collections ? | 17:11 |
| *** sshnaidm is now known as sshnaidm|afk | 17:12 | |
| y2kenny | corvus: something I just noticed (base on what you say about playbook for hosts: all not running if group all is empty.) This would mean any role that has task delegate_to: localhost will also not run, is that right? | 17:13 |
| corvus | y2kenny: that should work fine -- localhost is always implicitly in the inventory, but it's not included in "all" | 17:14 |
| corvus | tristanC: good q -- mordred? (or maybe zuul/zuul.ansible) ? | 17:15 |
| tristanC | y2kenny: when a playbook is not running because the group all is empty, then i think any role, even the one using delegate_to: localhost attribute are also not running | 17:15 |
| y2kenny | tristanC: that aligns with what I have observed | 17:16 |
| mordred | I think zuul-collections - or there's a pattern elsewhere that would be ansible-collections-zuul | 17:16 |
| mordred | nevermind - ignore that last thing | 17:17 |
| corvus | y2kenny: oh i misunderstood, yes tristanC is right; all the tasks/roles still need to at least apply to something in the inventory, even if they are then delegated to something else. | 17:18 |
| mordred | the amazon.aws collection is: https://github.com/ansible-collections/amazon.aws | 17:18 |
| y2kenny | tristanC, corvus: semantically I think it make sense (all is empty so there's no one doing the delegating) but the intent within the context of zuul is not quite right I think (delegate_to localhost just mean I want something running on the executor.) | 17:18 |
| mordred | corvus: so maybe we should do zuul/zuul.posix | 17:18 |
| mordred | or maybe we don't follow that pattern and just do zuul-collection-$something | 17:19 |
| mordred | corvus: maybe instaed of zuul.ansible or zuul.posix we should go more generic and do like zuul.util | 17:19 |
| y2kenny | corvus: so perhaps I should just make some pre playbook with hosts: localhost actually... just to be explicit | 17:19 |
| corvus | mordred: i'm open to that, but i was thinking if we're shadowing a 'core' module, then just s/zuul/ansible/ might be nice? | 17:20 |
| corvus | like "this is the zuul fork of the ansible.posix.synchronize module" | 17:20 |
| corvus | y2kenny: that's probably the way to go | 17:20 |
| mordred | corvus: ah - so I was thinking we could also adopt collections for our other stuff too - so we've got one mechanism - but that's a good point, so maybe zuul.posix for synchronize, and zuul.util for things liek zuul_return and friends | 17:22 |
| *** bhavikdbavishi has quit IRC | 17:22 | |
| corvus | mordred: re the repo name, zuul.posix is nice because it's more "standard", but 'ansible-collection-posix' might better indicate to someone browsing the repo what it is | 17:22 |
| corvus | mordred: yeah, i'd be on board with zuul.util for the other stuff | 17:22 |
| *** bhavikdbavishi has joined #zuul | 17:22 | |
| mordred | ++ | 17:22 |
| mordred | I think I like ansible-collection-posix and ansible-collection-util | 17:22 |
| mordred | as repo names | 17:22 |
| corvus | yeah, it's growing on me | 17:22 |
| corvus | i updated the etherpad for that | 17:22 |
| corvus | this has been a great imromptu design session! i will summarize this in an email to the list soon | 17:23 |
| corvus | right after i get out of this chair because i'm getting cramps. :) | 17:23 |
| *** bhavikdbavishi has quit IRC | 17:26 | |
| *** panda is now known as panda|off | 17:27 | |
| *** sshnaidm|afk is now known as sshnaidm|off | 18:02 | |
| corvus | tristanC, mordred, tobiash: istr we had an idea about using git over a kubectl connection to set up workspaces too -- does anyone recall the result of that? | 18:17 |
| mordred | corvus: I don't know that anyone ever got it working | 18:18 |
| tobiash | corvus: yeah, that works and was the change I mentioned above | 18:18 |
| corvus | tobiash: oh i missed that; can you link again? | 18:18 |
| corvus | (there was *a lot* of text going by today :) | 18:18 |
| tobiash | so much backscroll | 18:18 |
| tobiash | searchung | 18:18 |
| corvus | (fwiw, even if we decide that's the best way to push the git repos to workers, a synchronize that works globally would still be really helpful for artifacts and other tasks) | 18:19 |
| fungi | and we still haven't gotten back around to mnaser's topic from yesterday, finding a way to do arbitrary git remote replication from the executor | 18:19 |
| mordred | corvus: ++ | 18:19 |
| tobiash | corvus: https://review.opendev.org/711920 | 18:19 |
| corvus | fungi: that's not ringing a bell; is there a summary of that? | 18:20 |
| tobiash | that's merged and tested in our deployment | 18:20 |
| mordred | and in ours too it seems | 18:20 |
| corvus | tobiash, tristanC: so.... "mirror-workspace-git-repos" works everywhere? | 18:20 |
| fungi | corvus: starting at 22:55 in scrollback, but basically upload-git-mirror needs a job node to be able to set up ssh keys, and there's probably a way around that limitation since we already push git refs to some places (job nodes) from the executor anyway | 18:21 |
| tobiash | so afaik prepare-workspace-git should just work as it doesn't require sychronize compared to plain old prepare-workspace | 18:21 |
| mordred | corvus: yes - but i think it won't work non-privileged | 18:21 |
| mordred | s/think// | 18:22 |
| tobiash | corvus: yes, and thus also prepare-workspace-git should work as it uses mirror-workspace-git-repos: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/prepare-workspace-git/tasks/main.yaml#L56 | 18:22 |
| tobiash | corvus: btw, I think we should ditch mirror-workspace-git-repos and move it into prepare-workspace-git | 18:22 |
| *** rfolco is now known as rfolco|bbl | 18:22 | |
| tobiash | I think mirror-workspace-git-repos was the very first version of a git based prepare-workspace and might still be in use by opendev directly? | 18:23 |
| corvus | mordred: hrm; maybe we need to rethink the non-privileged thing then | 18:24 |
| corvus | mordred: because, iirc, the git version of this is much more efficient than rsync | 18:24 |
| mordred | is it? why did I think rsync was more efficient | 18:24 |
| mordred | but also - while I totally appreciate teh base/base-minimal split - I think prepare-workspace is now pretty solid and one of those things that doesn't change very often, so having it in the base-minimal privileged base job shouldn't be too onerous | 18:25 |
| tobiash | mordred: rsync was the first simple thing but opendev (as we) did an early switch to the git based sync because it's much more efficient and needed to cache repos in the images | 18:25 |
| mordred | and it's really a role that _everyone_ should just have at the bsae of their stack | 18:25 |
| mordred | tobiash: ah yes - nod | 18:25 |
| corvus | mordred: yeah, and we also have really solid testing of those roles now | 18:26 |
| tristanC | corvus: there is also the issue of fetch-output | 18:26 |
| tobiash | even without caching the git based version saves roughly 50% bandwith | 18:26 |
| mordred | yeah - we need zuul.posix.synchronize for fetch-output | 18:26 |
| corvus | yes, i agree, i don't think this changes the need for zuul.posix.synchronize -- i just think it might mean we may not end up using it for git repo prep | 18:26 |
| mordred | ++ | 18:28 |
| tobiash | corvus: do we want to support rsync and git based workspace preparation in the long term? | 18:29 |
| tobiash | or does it make sense to default to the git based sync at some point in time? | 18:29 |
| corvus | tobiash: i would be comfortable calling the rsync repo setup roles deprecated and converging on a single git-based role that works everywhere. i think we should do that slowly and carefully though. we should probably start a new ml thread about it, to make sure we get the folks with the split base playbooks involved. | 18:30 |
| tobiash | our reference zuul-base-jobs still use the rsync based one: https://opendev.org/zuul/zuul-base-jobs/src/branch/master/playbooks/base/pre.yaml#L4 | 18:30 |
| corvus | tobiash: yeah, it was a change to that which sent us down this rabbit hole :) | 18:31 |
| corvus | tobiash: https://review.opendev.org/716298 | 18:31 |
| tobiash | wow, that is a complicated change :) | 18:31 |
| corvus | yeah, but in reality, it's necessary now. i'm okay landing something like that as an interim, and then our goal should be to get that back to more or less where it was, but have it work in containers and vms. | 18:32 |
| corvus | (if the base playbook is scary, it will turn people away from zuul) | 18:33 |
| corvus | (it should be a model of simplicity and community collaboration and reuse :) | 18:33 |
| *** mgoddard has joined #zuul | 18:46 | |
| corvus | i sent an email about the synchronize thing; i'll send another one about prepare-workspace after lunch | 18:48 |
| *** spsurya_ has quit IRC | 19:08 | |
| *** olaph has joined #zuul | 19:53 | |
| * olaph waves | 19:53 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: WIP: Implement graceful termination for the executor https://review.opendev.org/718157 | 19:57 |
| tobiash | tristanC: I have a working graceful pod delete using this and a preStop hook in a testenv ^ | 19:58 |
| tobiash | I'll finalize this next week probably | 19:58 |
| fungi | olaph!!! | 19:58 |
| *** hashar is now known as idoine | 19:58 | |
| fungi | we've missed you! | 19:59 |
| *** idoine is now known as quibble | 19:59 | |
| olaph | I heard that's what the cow said... :) | 19:59 |
| *** quibble is now known as hashar | 19:59 | |
| fungi | your animals talk to you now? that's awesome ;) | 19:59 |
| corvus | olaph: hay! | 19:59 |
| *** rfolco|bbl is now known as rfolco | 20:15 | |
| corvus | okay, i sent the other email too | 20:20 |
| *** hashar has quit IRC | 20:48 | |
| *** tjgresha has joined #zuul | 20:48 | |
| *** armstrongs has joined #zuul | 21:13 | |
| *** armstrongs has quit IRC | 21:22 | |
| y2kenny | Can the console tab on the web UI show more than one post playbook? | 21:35 |
| clarkb | y2kenny: all the playbooks should be expandable. | 21:35 |
| fungi | y2kenny: it should show all the playbooks which ran | 21:35 |
| y2kenny | um... | 21:35 |
| y2kenny | for some reason it's only showing 1 (I have 3 post playbooks) | 21:36 |
| y2kenny | even though job-output.txt has the log for 2 (which is odd in itself) | 21:36 |
| y2kenny | the stream showed all 3 | 21:37 |
| clarkb | y2kenny: the source of that is the json job output file | 21:37 |
| clarkb | maybe double check the contents of that file | 21:38 |
| mordred | remember that nothing that runs after log collection is going to show up in the logs | 21:38 |
| mordred | even though it will show up in the stream | 21:38 |
| y2kenny | mordred: oh yea... of course | 21:38 |
| y2kenny | sorry, my bad | 21:39 |
| mordred | (good reason to mkae log collection run as last as possible :) ) | 21:39 |
| * y2kenny thinking he should start the long weekend early... | 21:40 | |
| fungi | just realized i had originally planned to be on vacation all this week, before an epidemic broke out | 21:41 |
| clarkb | fungi: taking that friday off when I was supposed to be driving ended up being really good | 21:42 |
| clarkb | no road trip, but got to relax and do things without computers | 21:42 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add zuul-registry deployment https://review.opendev.org/710650 | 21:42 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add zuul-preview service https://review.opendev.org/718419 | 21:42 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Document the missing feature https://review.opendev.org/718755 | 21:42 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Document the missing feature https://review.opendev.org/718755 | 21:43 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add zuul-preview service https://review.opendev.org/718419 | 21:45 |
| *** tosky has quit IRC | 23:08 | |
| *** rfolco has quit IRC | 23:37 | |
| *** lseki has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!