| *** sdake has joined #zuul | 00:06 | |
| *** sdake has quit IRC | 00:29 | |
| tristanC | tobiash: go ahead, fine for me | 00:48 |
|---|---|---|
| *** sdake has joined #zuul | 00:49 | |
| *** sdake has quit IRC | 01:09 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add roles usage information to the build page https://review.openstack.org/633697 | 02:37 |
| *** bhavikdbavishi has joined #zuul | 02:50 | |
| *** sdake has joined #zuul | 03:00 | |
| *** saneax has joined #zuul | 03:19 | |
| tristanC | jhesketh: welcome back, re: freeze_job, since the patch stack is usable for the zuul-runner execute command, should we remove the WIP for the refact patches (607079 and 607078) ? | 03:41 |
| jhesketh | tristanC: Yep, I think it's probably ready to get some more eyes on it; however I'm still catching up on the changes myself :-) | 03:42 |
| tristanC | jhesketh: actually, I wanted to discuss with you about 79, i reworked the AnsibleJob base class to not diverge from the current implementation, 79 just move the re-usable procedure without modification | 03:42 |
| jhesketh | tristanC: I tried to move it as much as possible without modifications, but I can't remember what I had to do now. If you found a way to refactor it cleaner then that's great | 03:44 |
| tristanC | jhesketh: it's actually a verbatim copy so that rebasing is simple | 03:45 |
| jhesketh | awesome :-) | 03:46 |
| tristanC | jhesketh: the only difference between zuul-executor and zuul-runner is that the executor service has a self.job gearman object, and further patches just add if self.job to send gearman result when needed | 03:47 |
| *** rlandy|bbl is now known as rlandy | 03:49 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: config: add playbooks to job.toDict() https://review.openstack.org/621343 | 04:22 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Add API endpoint to get frozen jobs https://review.openstack.org/607077 | 04:22 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Get executor job params https://review.openstack.org/607078 | 04:22 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Separate out executor server from runner https://review.openstack.org/607079 | 04:22 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: implement prep-workspace https://review.openstack.org/607082 | 04:22 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: web: add /connections route https://review.openstack.org/631703 | 04:22 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: use connections endpoint for prepare-workspace https://review.openstack.org/631704 | 04:22 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add execute sub-command https://review.openstack.org/630944 | 04:22 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add support for depends-on https://review.openstack.org/632064 | 04:22 |
| jhesketh | tristanC: I was rebasing to solve merge conflicts, so took out the WIP's while I was there ^ | 04:22 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: config: add playbooks to job.toDict() https://review.openstack.org/621343 | 06:08 |
| *** chandankumar is now known as chkumar|ruck | 06:14 | |
| *** quiquell|off is now known as quiquell_rover | 06:20 | |
| *** swest has quit IRC | 07:06 | |
| *** quiquell_rover is now known as quiquell|rover | 07:07 | |
| *** quiquell|rover has quit IRC | 07:07 | |
| *** quiquell has joined #zuul | 07:07 | |
| *** quiquell is now known as quiquell|rover | 07:09 | |
| *** swest has joined #zuul | 07:09 | |
| *** quiquell|rover is now known as quique|rover|brb | 07:47 | |
| * quique|rover|brb is away: brb | 07:55 | |
| *** quique|rover|brb is now known as quiquell|rover | 07:55 | |
| *** quiquell|rover is now known as quique|rover|brb | 07:56 | |
| *** quique|rover|brb is now known as quique|rover|bbl | 07:56 | |
| * quique|rover|bbl is back (gone 00:04:03) | 07:59 | |
| * quique|rover|bbl is away: bbl | 07:59 | |
| *** themroc has joined #zuul | 08:11 | |
| *** avasss has joined #zuul | 08:12 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Add API endpoint to get frozen jobs https://review.openstack.org/607077 | 08:14 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Get executor job params https://review.openstack.org/607078 | 08:14 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Separate out executor server from runner https://review.openstack.org/607079 | 08:14 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: implement prep-workspace https://review.openstack.org/607082 | 08:14 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: web: add /connections route https://review.openstack.org/631703 | 08:14 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: use connections endpoint for prepare-workspace https://review.openstack.org/631704 | 08:14 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add execute sub-command https://review.openstack.org/630944 | 08:14 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add support for depends-on https://review.openstack.org/632064 | 08:14 |
| *** quique|rover|bbl is now known as quiquell|rover | 08:16 | |
| * quiquell|rover is back (gone 00:16:43) | 08:16 | |
| *** gtema has joined #zuul | 08:26 | |
| *** electrofelix has joined #zuul | 08:42 | |
| *** jpena|off is now known as jpena | 08:46 | |
| tobiash | adam_g: yeah, that probably needs caching of the users | 08:48 |
| *** mhu has quit IRC | 08:51 | |
| *** mhu has joined #zuul | 08:51 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Tolerate missing project https://review.openstack.org/579872 | 08:52 |
| *** sshnaidm|afk is now known as sshnaidm | 09:25 | |
| *** luizbag has joined #zuul | 09:42 | |
| *** badboy has joined #zuul | 09:44 | |
| *** panda|off is now known as panda | 09:46 | |
| badboy | shouldn't 'bubblewrap' be required while installing Zuul via pip? | 09:46 |
| badboy | it doesn't work without it an you have to dig through the logs to figure it out it's missing | 09:47 |
| *** bhavikdbavishi has quit IRC | 09:52 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Move isChangeReadyToBeEnqueued after pipeline requirements https://review.openstack.org/634191 | 09:59 |
| jkt | badboy: on the other hand, it's only needed for the executor which might be on a separate host than the scheduler | 10:09 |
| badboy | jkt: true but the docs aren't crystal clear on that, at least imo | 10:26 |
| tobiash | badboy: bubblewrap is not python so you cannot pull it in during pip install | 10:42 |
| tobiash | badboy: but you can use bindep to install binary deps | 10:43 |
| badboy | tobiash: ok, I've istalled it via apt | 10:53 |
| badboy | the point is that I am trying to setup the whole zuul system on one vm and the docs are helpful but many things are unclear | 10:54 |
| badboy | the quick start guide is simple but there's no real instructions on how to set up everything manually | 10:55 |
| badboy | atm I am trying to configure apache for the logs and it turns out that you have to have a mysql/othe db in order for the 'Builds' tab to be present | 10:56 |
| badboy | by present I mean with some real data | 10:56 |
| badboy | I'm not complaining, just giving you some feedback ;) | 10:56 |
| *** rfolco has joined #zuul | 11:05 | |
| tobiash | badboy: we plan to make sql mandatory in the future | 11:11 |
| tobiash | badboy: and we appreciate any feedback (also fixes, improvements to the docs are welcome ;) ) | 11:12 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Make git clone timeout configurable https://review.openstack.org/633936 | 11:25 |
| *** gtema has quit IRC | 11:37 | |
| avasss | is there a variable like zuul2 ZUUL_COMMIT in zuul3? | 11:38 |
| *** gtema has joined #zuul | 11:40 | |
| *** sdake has quit IRC | 11:41 | |
| tobiash | avasss: ZUUL_COMMIT was the commit to checkout after fetching the repo from the merger right? | 11:42 |
| avasss | tobiash: I think so | 11:42 |
| * tobiash didn't touch a zuulv2 since 1.5 years | 11:42 | |
| tobiash | avasss: you don't need that in v3 | 11:43 |
| avasss | tobiash: we hav esome functionality around it :) | 11:43 |
| tobiash | because zuulv3 is a push based system and the correct revision is already checked out | 11:43 |
| *** avasss is now known as avass | 11:43 | |
| tobiash | but you could use git to get that information if you need it: | 11:44 |
| tobiash | 'cd {{ zuul.project.src_dir }} && git rev-parse HEAD' should probably do it | 11:44 |
| avass | tobiash: we we're using it to cache some things in case the same job ran twice without any changes to master | 11:45 |
| tobiash | I thing that would be pretty much the same as ZUUL_COMMIT | 11:45 |
| tobiash | avass: but that revision will be different in this use case | 11:45 |
| tobiash | because of timestamps | 11:45 |
| tobiash | but I guess it should be different in v2 as well | 11:46 |
| avass | tobiash: we might need to rework that then, I don't think it's anything critical anyway | 11:46 |
| *** gtema has quit IRC | 11:46 | |
| *** sdake has joined #zuul | 11:47 | |
| *** gtema has joined #zuul | 11:47 | |
| *** gtema has quit IRC | 12:11 | |
| *** gtema has joined #zuul | 12:15 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /connections route https://review.openstack.org/631703 | 12:17 |
| *** jpena is now known as jpena|lunch | 12:27 | |
| *** gtema has quit IRC | 12:38 | |
| *** rlandy has joined #zuul | 12:54 | |
| *** gtema has joined #zuul | 13:02 | |
| *** ssbarnea has quit IRC | 13:17 | |
| *** zbr has joined #zuul | 13:17 | |
| *** zbr is now known as zbr|ssbarnea | 13:18 | |
| *** zbr|ssbarnea has quit IRC | 13:19 | |
| *** zbr|ssbarnea has joined #zuul | 13:19 | |
| *** bhavikdbavishi has joined #zuul | 13:20 | |
| *** zbr|ssbarnea has quit IRC | 13:21 | |
| *** bhavikdbavishi has quit IRC | 13:22 | |
| *** bhavikdbavishi has joined #zuul | 13:22 | |
| *** zbr|ssbarnea has joined #zuul | 13:26 | |
| *** jpena|lunch is now known as jpena | 13:31 | |
| *** bhavikdbavishi has quit IRC | 13:39 | |
| *** ParsectiX has joined #zuul | 13:48 | |
| sean-k-mooney | quick question is the zuul fingergw serice needed for zuul web or can zuul web pull the log stream directly form the executor? | 13:49 |
| *** pcaruana has quit IRC | 13:52 | |
| *** pcaruana has joined #zuul | 14:02 | |
| *** sdake has quit IRC | 14:08 | |
| *** chkumar|ruck is now known as chkumar|out | 14:15 | |
| *** gtema has quit IRC | 14:36 | |
| tobiash | sean-k-mooney: it's only needed if you want to use the finger command line client | 14:36 |
| sean-k-mooney | tobiash: cool thanks. i have never really got that to work properly since it is not obvios witout connecting to the executor what the uuid is for the build | 14:38 |
| sean-k-mooney | tobiash: i mean you can get it to work but the web ui seams simler | 14:38 |
| sean-k-mooney | its on my todo list to deploy in a week or two | 14:38 |
| sean-k-mooney | the web ui that is | 14:39 |
| *** sdake has joined #zuul | 14:39 | |
| *** themroc has quit IRC | 15:01 | |
| *** saneax has quit IRC | 15:12 | |
| *** bhavikdbavishi has joined #zuul | 15:16 | |
| * pabelanger still wants finger urls toggle on status page | 15:17 | |
| pabelanger | :) | 15:17 |
| *** sdake has quit IRC | 15:24 | |
| avass | is it possible to kill processes started before aborting jobs? | 15:27 |
| corvus | badboy: yes, i think bubblewrap should be mentioned here, but isn't: https://zuul-ci.org/docs/zuul/admin/installation.html | 15:29 |
| corvus | badboy: if you use bindep as mentioned here, it should pull in bubblewrap, i think: https://zuul-ci.org/docs/zuul/admin/zuul_install.html#installation | 15:30 |
| badboy | corvus: I have manually installed it but it took some time to figure it out | 15:31 |
| badboy | corvus: there's nothing in the docs on how to set up the log server and database | 15:32 |
| pabelanger | avass: you mean, kill zuul processes? | 15:32 |
| badboy | corvus: and it's not mentioned that nothing will appear in the 'Builds' tab unless those above are configured | 15:33 |
| avass | pabelanger: no processes started by ansible, when you dequeue a job it doesn't kill the processes on the node which is a bit of a problem on static nodes | 15:33 |
| corvus | badboy: if you could write all of these down in a bug report (or if you feel like writing patches to the docs) that would be great :) | 15:34 |
| corvus | avass: what do you mean by dequeue? | 15:35 |
| badboy | corvus: I will try to do that over the weekend | 15:35 |
| corvus | avass: you mean "zuul dequeue" ? | 15:35 |
| avass | coruvs: yeah | 15:35 |
| avass | corvus: or when submitting a new patch change while a job is running | 15:36 |
| badboy | pabelanger: https://github.com/openstack/ansible-role-zuul/blob/master/templates/etc/zuul/zuul.conf#L17 it should be 'user={{ zuul_user_name }}' | 15:36 |
| badboy | pabelanger: https://github.com/openstack/ansible-role-zuul/blob/master/templates/etc/systemd/system/zuul-scheduler.service#L7 also 'Group={{ zuul_group_name }} and so on | 15:37 |
| *** chkumar|out is now known as invincible | 15:37 | |
| avass | corvus: which aborts the job but doesn't kill any processes ansible started | 15:37 |
| pabelanger | badboy: the default zuul.conf is vary basic for the role, enough to bootstrap. It is better for you to pass in your own template, with site specific settings. for example: https://github.com/ansible-network/windmill-config/blob/master/zuul/zuul.conf.j2 with https://github.com/ansible-network/windmill-config/blob/master/ansible/group_vars/zuul.yaml#L17 | 15:38 |
| *** gtema has joined #zuul | 15:38 | |
| pabelanger | badboy: the you have the ability to override all those files manually | 15:39 |
| corvus | avass: it looks like 'zuul dequeue' will not abort jobs, which i think is a bug. but a new patchset should cause jobs to abort. | 15:39 |
| corvus | avass: oh, i just read your last message | 15:39 |
| corvus | avass: when you say 'processes ansible started', do you mean like daemons it launched? or do you just mean normal ansible processes? | 15:40 |
| *** quiquell|rover is now known as quique|rover|off | 15:40 | |
| avass | corvus: like any normal process on windows :) | 15:40 |
| badboy | pabelanger: you're right but there's no point in having a variable with zuul username and a hardcoded value | 15:42 |
| pabelanger | badboy: that is the default user | 15:42 |
| pabelanger | badboy: you can use systemd overrides to manage it | 15:43 |
| pabelanger | eg: http://git.openstack.org/cgit/openstack/windmill/tree/playbooks/templates/zuul/etc/systemd/system/zuul-scheduler.service.d/override.conf.j2 | 15:43 |
| corvus | avass: i don't know much about windows process handling. but in general, the intention is for zuul to clean up all of the ansible that it starts (with the exception of the zuul console streamer, which is okay to keep running), but if that ansible started daemons which used (under linux, i don't know the windows equivalent) double-fork method to detach from their parents, there is no way to identify those | 15:43 |
| corvus | processes and kill them (plus, it's not clear that they should be -- consider the deployment use case where zuul manages production servers) | 15:43 |
| pabelanger | badboy: then set zuul_file_zuul_scheduler_service_config_src | 15:43 |
| badboy | pabelanger: is there a reason for doing it that way? | 15:43 |
| badboy | just aking out of curiosity | 15:44 |
| pabelanger | badboy: because I don't want to template every possible setting for the role, it is more a package manage to lay down the bits, and ensure the services start. I've exposed almost every possible file for a deployer to module and change themside, with out having to patch the role for each new setting. It also allows for a user to easily extend the role, with their own site specific role | 15:45 |
| badboy | pabelanger: got it, thanks | 15:46 |
| avass | corvus: seems strange because it keeps running | 15:46 |
| pabelanger | badboy: you should look at windmill to see examples of how it all works, that is actually an all in one install for zuul. And right now, I have a POC running in vexxhost for a multi node deployment | 15:46 |
| pabelanger | But I also need to write way more documentation for it | 15:46 |
| tobiash | avass: zuul kills the process group of ansible when aborting jobs so that kills all local ansible processes. So I guess you mean that ansible doesn't kill remote processes on the node when it's killed? | 15:50 |
| avass | tobiash: yes | 15:50 |
| tobiash | avass: I guess under linux things are automatically killed by aborting the ssh connection, with windows it's not so easy because there is no persistent connection | 15:55 |
| avass | tobiash: that could be it | 15:56 |
| Shrews | our zuul-quick-start job seems very unhappy lately | 16:00 |
| Shrews | hrm, but only on the nodepool side it seems. curious | 16:02 |
| *** saneax has joined #zuul | 16:04 | |
| corvus | Shrews: that's weird, i'll take a look | 16:04 |
| avass | tobiash: I guess dequeueing isn't an option right now then | 16:06 |
| tobiash | avass: well, dequeueing happends during normal operations | 16:07 |
| avass | tobiash: what do you mean? | 16:07 |
| tobiash | avass: I have no idea about that session handling with winrm, maybe you can tweak the winrm settings with some session timeout that will ensure that the processes get killed | 16:07 |
| tobiash | avass: every time you update a change that currently runs jobs, that change is dequeued and the updated one enqueued | 16:08 |
| tobiash | that is normal operations | 16:08 |
| tobiash | so you need to be able to deal with this | 16:08 |
| avass | tobiash: ah, yeah but you can turn it off :) | 16:08 |
| tobiash | ? | 16:09 |
| avass | https://zuul-ci.org/docs/zuul/user/config.html#attr-pipeline.dequeue-on-new-patchset | 16:09 |
| tobiash | ah that one | 16:09 |
| tobiash | but still, a failing item in a gate pipeline will dequeue/reenqueue all following items too | 16:10 |
| tobiash | and that's normal operations too | 16:10 |
| corvus | or, rather, will cancel and restart jobs. but that's the same thing for our purposes. | 16:10 |
| corvus | could you start your job with a task that cleans up any old processes? | 16:11 |
| avass | corvus: I was thinking about that | 16:11 |
| avass | corvus: but we're probably going to running more than one job so we'll have to set up some way of identifying zombie processes | 16:12 |
| avass | one job per node | 16:12 |
| tobiash | avass: and you need static windows nodes? | 16:14 |
| tobiash | or could they be dynamic too? | 16:14 |
| avass | tobiash: it's what we've got to work on | 16:14 |
| tobiash | ah, ok | 16:14 |
| avass | tobiash: otherwise that would solve a lot more problems | 16:15 |
| corvus | Shrews: i see the problem | 16:16 |
| corvus | Shrews: the 'context' attributes here: http://git.zuul-ci.org/cgit/zuul/tree/.zuul.yaml#n68 | 16:17 |
| *** ParsectiX has quit IRC | 16:17 | |
| corvus | Shrews: i'll push up a patch | 16:18 |
| Shrews | corvus: should those be ../zuul instead of . ? | 16:20 |
| corvus | Shrews: yeah, but i'm gonna make it even fancier | 16:20 |
| Shrews | corvus: i wonder how this was working before now? | 16:20 |
| *** sdake has joined #zuul | 16:20 | |
| Shrews | corvus: also, what led you to that? i couldn't find anything very helpful in the logs | 16:21 |
| avass | does zuul close the connection to zuul_daemon when aborting jobs? | 16:21 |
| corvus | Shrews: yeah, took me a while: http://logs.openstack.org/92/633792/6/check/zuul-quick-start/b308be2/job-output.txt.gz#_2019-01-30_20_24_54_765271 | 16:22 |
| corvus | Shrews: or, specifically: http://logs.openstack.org/92/633792/6/check/zuul-quick-start/b308be2/job-output.txt.gz#_2019-01-30_20_24_54_766089 | 16:22 |
| Shrews | corvus: hrm, seems like we should be catching that earlier somewhere | 16:23 |
| avass | ah, wait i mean the zuul_console daemon | 16:23 |
| Shrews | i'll see if i can figure out enough to see if that's possible | 16:24 |
| *** quique|rover|off is now known as quiquell|rover | 16:24 | |
| corvus | Shrews: i think that's the first use of the file | 16:25 |
| corvus | if anyone wants to start planning now: https://www.openstack.org/summit/shanghai-2019/ | 16:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix context directories in image builds https://review.openstack.org/634266 | 16:30 |
| corvus | Shrews: ^ try a Depends-On: that | 16:30 |
| Shrews | corvus: k. and i think i might see a solution to catching that error earlier that i'm going to experiment with | 16:32 |
| Shrews | corvus: fancy | 16:34 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Revert "Revert "Add a timeout for the image build"" https://review.openstack.org/633792 | 16:37 |
| pabelanger | avass: no, you need to manually clean that up too | 16:37 |
| pabelanger | I have a post-run playbook to do that | 16:37 |
| corvus | you don't need to stop zuul_console if you don't want to; if it's already running on the next job it will still work. | 16:39 |
| avass | pabelanger: Was hoping that I could hack something together in python since zuul_console doesn't work for windows | 16:39 |
| pabelanger | avass: I know mordred has some logging changes up to remove the need for it, I haven't checked in a while the status of it. | 16:40 |
| avass | I have to get going, thanks for all your help | 16:48 |
| Shrews | corvus: oh, duh. nevermind, i see the post playbook starts immediately after that failure, so working as expected. | 16:52 |
| Shrews | i need more coffee i think | 16:52 |
| *** avass has quit IRC | 16:53 | |
| *** spsurya has quit IRC | 16:54 | |
| *** gtema has quit IRC | 17:02 | |
| Shrews | corvus: oops... OSError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack-infra/zuul/src/git.openstack.org/openstack-infra/zuul' | 17:10 |
| corvus | doh. | 17:10 |
| corvus | can fix | 17:10 |
| corvus | Shrews: well, we can fix 2 ways. we can set "zuul_work_dir: /" or we can do your idea. which do you think is better? | 17:11 |
| corvus | (or, i guess, option #3 would be to update the role to ignore zuul_work_dir if context starts with '/') | 17:12 |
| Shrews | corvus: meh? that's like asking if 1+1 is better than 2+0. They both equal 2 | 17:13 |
| corvus | Shrews: i know, why do you think i'm asking? :) | 17:13 |
| Shrews | lol | 17:13 |
| corvus | i'll do your thing | 17:13 |
| Shrews | corvus: yeah, that seems easiest/simplest to review | 17:13 |
| Shrews | btw, i did check my maths on the calculator | 17:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix context directories in image builds https://review.openstack.org/634266 | 17:14 |
| corvus | just don't ask me 2+2. | 17:14 |
| Shrews | pffft, like anyone could answer that | 17:15 |
| Shrews | (even with a rat trap attached to their head) | 17:16 |
| * Shrews watches classic reference float over heads | 17:16 | |
| corvus | neo's room in the matrix was a reference to that. | 17:18 |
| corvus | also, picard's "there are four lights!" | 17:19 |
| Shrews | corvus: heh, b/c of the room # (101)? | 17:20 |
| corvus | yep :) | 17:20 |
| Shrews | hah, neat. didn't know that | 17:20 |
| *** sdake has quit IRC | 17:28 | |
| pabelanger | so, we have some network images for ansible-network, that by default usually only support username / password via SSH. Most of the time we can add it SSH public keys but usually required a bit of effort to do so. Given ansible support ansible_ssh_username / ansible_ssh_password, I want to get the feel about maybe adding support for this into zuul-executor, but i believe that would mean some how skipping SSH | 17:47 |
| pabelanger | agent support and load them directly from inventory files | 17:47 |
| pabelanger | ansible_ssh_pass can be encrypted with vault I believe | 17:49 |
| *** bhavikdbavishi has quit IRC | 17:58 | |
| *** bhavikdbavishi has joined #zuul | 17:59 | |
| *** saneax has quit IRC | 18:05 | |
| *** sdake has joined #zuul | 18:07 | |
| *** jpena is now known as jpena|off | 18:08 | |
| tobiash | pabelanger: I don't think you need to change anything with the ssh agent | 18:09 |
| corvus | pabelanger: i suspect there will be some details to figure out, but i think it's worth looking into and coming up with a plan/proposal. | 18:10 |
| tobiash | Just set that ansible variable and it should take care of enabling pw auth | 18:11 |
| tobiash | I think the intetesting question is where to define the pw | 18:11 |
| corvus | i'd guess in nodepool, and ship it over zookeeper. best to have zk auth enabled. | 18:12 |
| corvus | and, perhaps eventually, we may want to use the idea of symmetric encryption for that too (like we're talking about for secrets in zuul v4) | 18:13 |
| tobiash | Probably, we also could enhancr this to support image specific private keys | 18:13 |
| corvus | ++ | 18:13 |
| corvus | maybe remove the key from zuul and handle everything this way | 18:14 |
| tobiash | Yes | 18:14 |
| pabelanger | okay, I'll test somethings locally, then make post to ML | 18:19 |
| pabelanger | I think this would actually work out of box if our secrets were not dicts. As we could just encrypt ansible_ssh_pass directly, but today it would be foo.ansible_ssh_pass | 18:20 |
| pabelanger | but first I am going to finish up 592160 and get it back up for review | 18:20 |
| *** bhavikdbavishi has quit IRC | 18:30 | |
| *** quiquell|rover is now known as quique|rover|off | 18:41 | |
| *** panda is now known as panda|off | 18:41 | |
| * mordred waves to zuul people - isn't really here - but has boarded his return flight and thus has started being bored | 18:47 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: stage-output: Add the ability to add .txt extensions to files https://review.openstack.org/634293 | 18:50 |
| corvus | mordred: http://zuul.opendev.org/tenants exists | 18:51 |
| corvus | mordred: as does https://review.openstack.org/633605 | 18:51 |
| corvus | guaranteed to stave off boredom | 18:52 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Allow run to be list of playbooks https://review.openstack.org/592160 | 18:53 |
| *** luizbag has quit IRC | 18:54 | |
| pabelanger | corvus: ^finally updated to address your comments | 18:56 |
| *** pvinci has joined #zuul | 19:09 | |
| *** electrofelix has quit IRC | 19:13 | |
| pvinci | when I have a gerrit connection defined that gives me an paramiko.ssh_exception.SSHException: Signature verification (ssh-ed25519) failed exception, it impacts/blocks other gerrit connections. | 19:13 |
| pvinci | When I comment out the bad connection, the other works without issue. | 19:14 |
| pvinci | Is that an expected behavior? | 19:15 |
| *** bhavikdbavishi has joined #zuul | 19:21 | |
| *** bhavikdbavishi has quit IRC | 19:23 | |
| *** pcaruana has quit IRC | 19:30 | |
| *** sshnaidm is now known as sshnaidm|off | 19:33 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix context directories in image builds https://review.openstack.org/634266 | 19:45 |
| corvus | Shrews: ^ okay that's really going to get it :) | 19:45 |
| Shrews | sure sure | 19:46 |
| corvus | pvinci: it's... disappointing behavior. i'm not sure there's a lot of experience with or testing of that case. i'm not sure what the implications are of not being able to contact a source; i think things would go downhill quickly. so even fixing the first symptom there might end up exposing more complicated problems. | 19:47 |
| corvus | pvinci: all of that to say, it sounds like a bug, and i'm sure a fix would be an improvement, but for anyone contemplating a fix, there may be dragons there. :) | 19:48 |
| mordred | corvus: neat! | 19:49 |
| pabelanger | pvinci: possible you have wrong SSH key for the connection, I've see paramiko raise the exection once all keys from server side as iterated through | 19:50 |
| pabelanger | there is also case where paramiko doesn't yet support RFC4716 keys | 19:52 |
| mordred | corvus: the requires/provides patch looks great - but I don't understand computers enough today to actually read it properly | 20:05 |
| corvus | pabelanger: can you elaborate on that last bit? i thought pvinci was running into https://bugs.chromium.org/p/gerrit/issues/detail?id=6504 i was unaware of a fault with paramiko. | 20:07 |
| pvinci | Corvus, yes. I believe it is that bug and am trying to get the 3rd party to upgrade. | 20:09 |
| corvus | pvinci: in the mean time, you should be able to work around it by manually fetching the rsa keys from that gerrit (assuming it also has rsa keys) and adding them to the known_hosts file on all the zuul systems | 20:09 |
| corvus | pvinci: that will cause paramiko to prefer the rsa key (which is what openssh does already) | 20:10 |
| pvinci | I did, and it seems paramiko still tries and fails the ed25519 key | 20:10 |
| corvus | pvinci: on the zuul scheduler too? | 20:12 |
| pvinci | This is on the scheduler, yes. | 20:12 |
| corvus | bummer. i'm sure that worked for us before when we connected to opendaylight: http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n845 | 20:13 |
| corvus | pvinci: make sure you add all the hostnames/ip addresses, and don't forget the port | 20:13 |
| pvinci | scheduler_1 | paramiko.ssh_exception.SSHException: Signature verification (ssh-ed25519) failed. | 20:13 |
| pvinci | I just wanted to see it you all felt it was worthwhile to open a defect | 20:15 |
| pabelanger | corvus: https://github.com/paramiko/paramiko/issues/602 I ran into that issue on fedora-28, since openssh key format because the default | 20:16 |
| pabelanger | now you need to force -m pem for ssh-keygen | 20:16 |
| pabelanger | s/because/became | 20:18 |
| pabelanger | Just checking my notes again, and cannot find the exception I was getting. But just remember 'BEGIN OPENSSH PRIVATE KEY' being the issue with the private key | 20:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Allow run to be list of playbooks https://review.openstack.org/592160 | 20:28 |
| pvinci | @corvus thanks! That's the same gerrit I'm working with, but I have a different canonical name. | 20:31 |
| corvus | pvinci: looks like you're using containers -- maybe the known_hosts file isn't propogating into the container? | 20:34 |
| pvinci | I've logged in. The keys are there. | 20:37 |
| pvinci | [gerrit.fd.io]:29418 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCjr1oez076EFMo5n25lAJ2zhftLAHIkTmwTdjwR82xA8sqQbN0FMz4znZyO7o2jlewlw/OqnVAwEIvEto6 [gerrit.fd.io]:29418 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEZKNxaLXuYrZvR0BTjrdTP7GVkeMpoa4sKSaOsMYMeIHFrq [gerrit.fd.io]:29418 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFcFjT0+oeAkqNFIj/71n/4WoRvC+HOIVv5nFdk8xDH5 | 20:38 |
| *** rfolco has quit IRC | 20:40 | |
| corvus | pvinci: *only* supply the rsa key | 20:45 |
| corvus | if paramiko sees the other keys, it will still prefer them. but if it only sees an rsa key, then it will use that, because that is better than trying to verify a new key. | 20:46 |
| pvinci | I am testing that | 20:46 |
| mrhillsman | question, hopefully it is clear, i do not want to create two periodic pipelines, i have one that uses a trigger for say 00|04|08|16 hours and another for the 22nd hour of the day, is there a way to have jobs only fire off at specific hours without having two periodic pipelines? | 21:25 |
| pvinci | corvus: putting in the reverse lookup seems to have helped. [gerrit.fd.io]:29418,[52.10.107.188]:29418 ssh-rsa | 21:39 |
| corvus | pvinci: yay! | 21:40 |
| corvus | mrhillsman: nope, you need 2 pipelines for that. | 21:40 |
| mrhillsman | thx | 21:40 |
| pvinci | Still seeing exceptions, but now pulling down refs. | 21:41 |
| pvinci | I'll let you know how it progresses. Thanks a million! | 21:41 |
| corvus | pvinci: you're welcome | 21:42 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add a role to run a buildset registry https://review.openstack.org/634319 | 21:44 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add a role to run a buildset registry https://review.openstack.org/634319 | 21:54 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: DNM: test buildset registry https://review.openstack.org/634323 | 22:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: DNM: test buildset registry https://review.openstack.org/634323 | 22:03 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add a role to run a buildset registry https://review.openstack.org/634319 | 22:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: DNM: test buildset registry https://review.openstack.org/634323 | 22:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add a role to run a buildset registry https://review.openstack.org/634319 | 22:21 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: DNM: test buildset registry https://review.openstack.org/634323 | 22:21 |
| *** sdake has quit IRC | 22:30 | |
| *** sdake has joined #zuul | 22:44 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Allow run to be list of playbooks https://review.openstack.org/592160 | 22:54 |
| pabelanger | tobiash: corvus: mordred: clarkb: ^ is green now, new feature to allow list of playbooks for job.run. If you'd like to add it into your review queue. | 22:55 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Proposed spec: tenant-scoped admin web API https://review.openstack.org/562321 | 22:58 |
| *** dkehn has joined #zuul | 23:04 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add a role to run a buildset registry https://review.openstack.org/634319 | 23:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: DNM: test buildset registry https://review.openstack.org/634323 | 23:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: WIP: Add a role to run a buildset registry https://review.openstack.org/634319 | 23:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: DNM: test buildset registry https://review.openstack.org/634323 | 23:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: WIP: add role to use buildset registry https://review.openstack.org/634346 | 23:28 |
| *** daniel2 has joined #zuul | 23:44 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!