| mordred | SpamapS: yah - they removed initramfs to make containers smaller | 00:00 |
|---|---|---|
| mordred | on xenial | 00:00 |
| mordred | an LTS | 00:00 |
| ianw | dib should be ready any minute with the merge of https://review.openstack.org/506398 | 00:11 |
| SpamapS | mordred: eh, I'm guessing that's not my problem. | 00:55 |
| SpamapS | given that I have a backtrace about a missing blockdev key in a dict | 00:55 |
| SpamapS | or not | 00:56 |
| SpamapS | hah | 00:56 |
| SpamapS | *weird* | 00:56 |
| SpamapS | hrm. My webstreaming done broke. | 03:39 |
| SpamapS | mordred: oh ow | 03:40 |
| jeblair | mordred, fungi: for tomorrow -- i rechecked https://review.openstack.org/506352 but also left some comments inline | 03:40 |
| jeblair | also something to look out for, it seems that my recheck with an inline comment didn't trigger zuul; maybe a new gerrit 2.13 behavior | 03:43 |
| SpamapS | ruhroh | 03:49 |
| SpamapS | mordred: annoying thing about finger: the standard client doesn't know how to use a non-standard port | 03:59 |
| SpamapS | There's likely a bug in zuul-web somewhere | 04:17 |
| SpamapS | mine was locked up | 04:17 |
| SpamapS | graceful kill did not end it | 04:18 |
| SpamapS | and it wasn't showing logs | 04:18 |
| SpamapS | probably has something to do with the "Log Stream did not terminate" problem too | 04:18 |
| SpamapS | leftover threads | 04:18 |
| SpamapS | ruh roh | 04:19 |
| SpamapS | recursive loop detected in template string: {{ zuul_site_image_manifest_files|default(validate_host_default_image_manifest_files) | 04:20 |
| jamielennox | SpamapS: also i don't thing the bonnyci code doesn't deploys the streamer correctly | 04:39 |
| SpamapS | jamielennox: I have it working | 04:39 |
| SpamapS | through nginx | 04:40 |
| jamielennox | nice, i think it was a known issue that you couldn't config the finger port on the streamer | 04:46 |
| jamielennox | first person to hit the problem has to fix it | 04:46 |
| *** bhavik1 has joined #zuul | 06:16 | |
| tobiash | jamielennox: configurable finger ports should have been done by 470243 and 473103 | 06:24 |
| *** yolanda_ has joined #zuul | 07:29 | |
| *** yolanda has quit IRC | 07:32 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Improve test case node_assignment_at_quota https://review.openstack.org/506134 | 07:33 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Honor cloud quotas before launching nodes https://review.openstack.org/503838 | 07:33 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Don't fail on quota exceeded https://review.openstack.org/503051 | 07:33 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Make max-servers optional https://review.openstack.org/504282 | 07:33 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support cores limit per pool https://review.openstack.org/504283 | 07:33 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support ram limit per pool https://review.openstack.org/504284 | 07:33 |
| *** ianw is now known as ianw|pto | 07:46 | |
| *** bhavik1 has quit IRC | 07:52 | |
| *** hashar has joined #zuul | 08:01 | |
| *** xinliang has quit IRC | 08:04 | |
| *** weshay_bbiab has quit IRC | 08:05 | |
| *** tobiash has quit IRC | 08:06 | |
| *** harlowja has quit IRC | 08:06 | |
| *** SpamapS has quit IRC | 08:06 | |
| *** tobiash has joined #zuul | 08:08 | |
| *** weshay has joined #zuul | 08:11 | |
| *** xinliang has joined #zuul | 08:18 | |
| *** xinliang has quit IRC | 08:18 | |
| *** xinliang has joined #zuul | 08:18 | |
| *** SpamapS has joined #zuul | 08:20 | |
| *** yolanda_ is now known as yolanda | 08:20 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Honor cloud quotas before launching nodes https://review.openstack.org/503838 | 08:31 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Don't fail on quota exceeded https://review.openstack.org/503051 | 08:31 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Make max-servers optional https://review.openstack.org/504282 | 08:31 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support cores limit per pool https://review.openstack.org/504283 | 08:31 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support ram limit per pool https://review.openstack.org/504284 | 08:31 |
| *** electrofelix has joined #zuul | 09:03 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Honor cloud quotas before launching nodes https://review.openstack.org/503838 | 09:06 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Don't fail on quota exceeded https://review.openstack.org/503051 | 09:06 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Make max-servers optional https://review.openstack.org/504282 | 09:06 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support cores limit per pool https://review.openstack.org/504283 | 09:06 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support ram limit per pool https://review.openstack.org/504284 | 09:06 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Migrate docs-draft jobs to emit to logs/html https://review.openstack.org/506316 | 09:32 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Extract required-projects from job content https://review.openstack.org/506335 | 09:33 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Allow dict-based new jobs to have templated names https://review.openstack.org/506344 | 09:37 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Extract required projects from embedded clonemaps https://review.openstack.org/506377 | 09:37 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Support zuul-cloner command line arguments https://review.openstack.org/506378 | 09:37 |
| tobiash | yay, just had the first working hello world windows job :) | 11:22 |
| tobiash | (with hard coded ansible stuff at the first try...) | 11:23 |
| tristanC | tobiash: nice! | 11:42 |
| tristanC | tobiash: so, is there a particular reason why the LogStreamingHandler is created at every console-stream request? | 11:47 |
| tristanC | it seems like we just have to create it at the ZuulWeb creation and use a single rpcclient instance | 11:51 |
| tobiash | tristanC: I think don't see any intended reason for that | 11:58 |
| tobiash | tristanC: I think this was just easier when adding these few calls to zuul-web in the beginning | 11:58 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: zuul-web: refactor LogStreamingHandler to keep a single rpcclient https://review.openstack.org/506602 | 12:19 |
| tobiash | jeblair, mordred: does anything speak agains enhancing nodepool builder such it can run an arbitrary command instead of diskimage builder? | 12:35 |
| tobiash | looks like it's fixed targeted at linux style file systems | 12:35 |
| tobiash | thought that we could hack windows generation via cloud, but it seems that it's much easier to optionally tell nodepool just a command for image building | 12:37 |
| tobiash | something like this: https://etherpad.openstack.org/p/7rx1HNhTl7 | 12:41 |
| *** dkranz has joined #zuul | 12:45 | |
| tobiash | ah, just found out that this is not needed | 13:20 |
| tobiash | one could just set the PATH env var and use that to inject a wrapper | 13:20 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: zuul-web: refactor LogStreamingHandler to keep a single rpcclient https://review.openstack.org/506602 | 13:33 |
| tristanC | tobiash: but then nodepool-builder won't be able to use the real disk-image-create right? | 13:35 |
| tobiash | tristanC: the envs you configure ther eare per image :) | 13:36 |
| tobiash | s/eare/are | 13:36 |
| tristanC | oh, then that's a nice work-around :-) | 13:37 |
| tobiash | tristanC: in the env-vars section of diskimages | 13:37 |
| dmsimard | tobiash: re: diskimage-builder .. I personally don't like DIB at all so, yes, a generic way of building and providing images would be nice :P | 13:42 |
| tristanC | dmsimard: we could try using libguestfs virt-dib in place using a fake disk-image-create wrapper | 13:43 |
| tobiash | dmsimard: so our way for windows will probably be an ansible script which 1. creates a windows vm on a cloud, 2. configures that, 3. downloads the image | 13:45 |
| tobiash | this all as a fake disk-image-create wrapper | 13:45 |
| dmsimard | ansible-container is in an interesting approach to building docker images but it's specific to docker :( | 13:52 |
| dmsimard | at least, last I checked | 13:52 |
| odyssey4me | dmsimard it looked pretty docker-specific last I heard - at least for the build itself... the product might be usable more generally though | 13:54 |
| odyssey4me | maybe I didn't look hard enough though | 13:54 |
| dmsimard | odyssey4me: yeah it used docker specific things, docker compose, etc. | 13:54 |
| dmsimard | odyssey4me: https://github.com/ansible/ansible-container/issues/228 | 13:56 |
| * odyssey4me subscribes | 13:56 | |
| dmsimard | just pinged a core about it, I'll let you know what I find out | 13:59 |
| *** hashar has quit IRC | 14:08 | |
| *** hashar has joined #zuul | 14:08 | |
| jeblair | tobiash: i think originally we were thinking there would be multiple kinds of builders, like there are multiple launchers. | 14:17 |
| tobiash | jeblair: you mean third party components implementing the zookeeper interface? | 14:18 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use username from node information if available https://review.openstack.org/453983 | 14:20 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Rename ssh_port to connection_port https://review.openstack.org/500799 | 14:20 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use connection type supplied from nodepool https://review.openstack.org/501976 | 14:20 |
| *** jkilpatr has quit IRC | 14:21 | |
| jeblair | tobiash: no i mean more nodepool components sharing the underlying framework. | 14:24 |
| tobiash | jeblair: ah, ok, so this would require a builder driver architecture like we have now for the launcher? | 14:27 |
| jeblair | tobiash: yes. i'm not saying that's needed for this (i haven't looked at your example yet, i just woke up). just saying that was the plan from earlier. :) | 14:28 |
| tristanC | jeblair: thinking about a container image ci workflow, do you think nodepool could be used to spawn a "speculative" image? e.g. zuul would request a image type 'project-name-container' and nodepool would somehow build the image using the zuul_change... | 14:43 |
| tristanC | or maybe this workflow is too different and it would be better implemented through a new service, that could be called containerpool or something | 14:44 |
| jeblair | tristanC: i really need to focus on the openstack transition today, sorry. we're one workday away, and we still haven't tested the output yet. i'd like us to talk about container workflow later when we're not under pressure. | 14:45 |
| tristanC | jeblair: sure, sorry for the distraction :) | 14:47 |
| jeblair | mordred: in http://logs.openstack.org/52/506352/4/check/devstack-legacy-tempest-dsvm-neutron-full/af062e6/job-output.txt.gz we're running up against a problem i ran into earlier. you can't use bash heredocs in ansible shell snippets with "|" line folding | 14:48 |
| jeblair | mordred: here's the bug: https://github.com/ansible/ansible/issues/12856 | 14:49 |
| mordred | jeblair: that claims to have been fixed - but yah - that's definitely an issue | 14:50 |
| jeblair | mordred: well, there's a workaround in the bug | 14:51 |
| jeblair | mordred: it got "fixed" by facilitating the workaround :| | 14:51 |
| jeblair | i'm testing it out locally | 14:51 |
| mordred | jeblair: oh- yah - so it would be shell: args: cmd: | blah right? | 14:51 |
| jeblair | mordred: either that, or _raw_command: -- i'm not sure? it's hard to follow | 14:52 |
| mordred | jeblair: yah - I think they mean putting a cmd: entry in the args: dict | 14:53 |
| mordred | where executable is - but I also agree, the thing there is tough to follow | 14:53 |
| jeblair | mordred: yep, cmd works. i'll update patch. | 14:54 |
| mordred | sweet. I'll update migration script | 14:55 |
| jeblair | mordred: done remote: https://review.openstack.org/506352 Remove devstack-legacy base job | 14:56 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Put sell snippets into task.args.cmd https://review.openstack.org/506683 | 14:58 |
| mordred | jeblair: maybe I should put *shell* snippets into task.args.cmd :) | 14:58 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Put shell snippets into task.args.cmd https://review.openstack.org/506683 | 15:00 |
| mordred | jeblair: fantastic: http://logs.openstack.org/52/506352/5/check/devstack-legacy-tempest-dsvm-neutron-full/9e804f1/job-output.txt.gz#_2017-09-22_14_59_17_466109 | 15:02 |
| mordred | jeblair: that makes no sense to me AT ALL | 15:03 |
| mordred | jeblair: oh! wait - when pip installs a console script in a venv it updates the shebang line to use the python from the venv | 15:04 |
| mordred | jeblair: I don't think we do that when we copy that file | 15:04 |
| tobiash | mordred: I have a comment/question at https://review.openstack.org/#/c/506415 | 15:05 |
| mordred | patch coming | 15:05 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Set shebang line of zuul-cloner script https://review.openstack.org/506685 | 15:08 |
| mordred | jeblair, tobiash: ^^ | 15:08 |
| mordred | tobiash: looking | 15:08 |
| mordred | tobiash: yah - that form is ok - it'll produce something like: http://paste.openstack.org/show/621724/ | 15:09 |
| mordred | if you search for devstack in http://logs.openstack.org/15/506415/2/check/zuul-migrate/e36c200/zuul.d/99converted-projects.yaml you can see some output examples from that bit | 15:10 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /tenants route https://review.openstack.org/503268 | 15:11 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/status route https://review.openstack.org/503269 | 15:11 |
| mordred | actually, openstack/puppet-aodh was theone that caused that patch - specifically the legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-puppet job | 15:11 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/jobs route https://review.openstack.org/503270 | 15:11 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/builds route https://review.openstack.org/466561 | 15:11 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: make console-stream tenant scoped https://review.openstack.org/505452 | 15:11 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{source}/{project}.pub route https://review.openstack.org/502530 | 15:11 |
| mordred | jeblair: updated the devstack-gate patch to add a depends-on to the zuul-jobs patch | 15:13 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Set shebang line of zuul-cloner script https://review.openstack.org/506685 | 15:19 |
| jeblair | back from breakfast and catching up | 15:22 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Rename tox_upper_constraints_file to tox_constraints_file https://review.openstack.org/502348 | 15:23 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Set shebang line of zuul-cloner script https://review.openstack.org/506685 | 15:25 |
| mordred | jeblair, tobiash: ^^ updated that to use the destination variable to set the shebang line instead of hard-coding the infra location | 15:25 |
| mordred | (since the role is parameterized after all) | 15:26 |
| jeblair | mordred: i was hoping we could stop installing zuul after the switch, but that still needs it :| | 15:26 |
| jeblair | mordred: i guess we can at least change it to install a virtualenv with pyyaml instead of all of zuul | 15:27 |
| mordred | yah | 15:27 |
| jeblair | mordred: also, that destination | dirname trick is neato | 15:28 |
| mordred | \o/ | 15:28 |
| mordred | jeblair: so - it seems that zuul aborts jobs if a depends-on gets rebased and does "this change depends on a change that failed to merge" | 15:28 |
| mordred | is that intentional? | 15:28 |
| mordred | jeblair: see message at bottom of https://review.openstack.org/#/c/506352/ - and the rebase of https://review.openstack.org/#/c/506685/ | 15:29 |
| jeblair | mordred: it could certainly be improved | 15:29 |
| jeblair | mordred: but yeah, is intentional inasmuch as the test is invalid at that point | 15:30 |
| mordred | cool | 15:30 |
| mordred | yah- might be nice to re-enqueue the dependent change with the new stack | 15:30 |
| jeblair | yep | 15:30 |
| mordred | I tihnk that finally explains various "failed to merge" messages I've seen before that have confused me | 15:30 |
| *** hashar is now known as hasharAway | 15:31 | |
| mordred | jeblair, tobiash: could I bother you to look at https://review.openstack.org/#/c/502348/ real quick? it's the second part of a change from denver and it's been languishing | 15:32 |
| mordred | as well as https://review.openstack.org/#/c/506401/ and https://review.openstack.org/#/c/506403/ | 15:33 |
| tobiash | mordred: could the 'failed to merge' thingy worked around with the parent-project-enqueued trigger? | 15:35 |
| jeblair | maybe; we should look into why we're not using that :) | 15:36 |
| mordred | ++ | 15:37 |
| tobiash | at least for the gate it should work | 15:37 |
| mordred | jeblair: hrm. that's troubling - the python template thing didn't seem to fix the devstack job | 15:46 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Set shebang line of zuul-cloner script https://review.openstack.org/506685 | 15:47 |
| mordred | jeblair: ^^ I have added a debug line to cat the file and make sure it's getting set like we think it is | 15:48 |
| mordred | jeblair: also - I wonder if the virtualenv python trick needs the shebang line to be #!{{ destination | dirname }}/env python | 15:49 |
| jeblair | #!/usr/zuul-env/bin/python | 15:54 |
| jeblair | mordred: it also worked? | 15:54 |
| mordred | jeblair: WHY DID THAT WORK???? | 15:55 |
| mordred | jeblair: didn't we make a role somewhere that outputs /etc/nodepool files? | 15:56 |
| mordred | AH - it's in the devstack-gate legacy run playbook ... patch coming | 15:57 |
| jeblair | should we add that to all legacy jobs? | 15:57 |
| mordred | yes | 15:58 |
| SpamapS | question about playbooks and jobs... | 15:58 |
| SpamapS | if I define a job named 'flame-yaml' | 15:58 |
| SpamapS | the repo that the job is defined in should have playbooks/flame-yaml/run.yaml yes? | 15:58 |
| SpamapS | and that playbooks existence and validity as a playbook, should be enough to get the job to run? | 15:58 |
| SpamapS | flamel-yaml flamel-yaml : ERROR No valid playbook found | 15:59 |
| jeblair | SpamapS: not quite; 1 sec. | 15:59 |
| tobiash | SpamapS: the default playbook would be playbooks/flame-yaml.yaml | 15:59 |
| mordred | SpamapS: if you want to not define the location of the playbook in the job definition, you want it to be playbooks/flame-yaml.yaml | 15:59 |
| SpamapS | (and yes I know it's flamel-yaml, I I misspelled some stuff in the initial question ;) | 15:59 |
| SpamapS | OH | 15:59 |
| SpamapS | got it | 15:59 |
| jeblair | SpamapS: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.run | 15:59 |
| mordred | SpamapS: since then we've started using a pattern of what you mentioned above in a lot of our jobs, but that is not a short-cut pattern zuul konws about | 15:59 |
| mordred | jeblair: ok - I'm a little bit torn here ... | 16:00 |
| mordred | jeblair: I don't want to have the migration script emit that into 10000 jobs - but I don't want to base everything on legacy-dsvm-base | 16:01 |
| jeblair | mordred: create legacy-base ? | 16:01 |
| mordred | jeblair: what if we make a legacy-base in o-z-j that has that nodepool thing in a pre-playbook and nothing else | 16:01 |
| mordred | yah | 16:01 |
| jeblair | mordred: wfm | 16:01 |
| mordred | and hav ethat be the base of legacy-dsvm-base and the default for things that don't have a base | 16:01 |
| mordred | patches coming | 16:01 |
| SpamapS | thanks, I cribbed the pattern without cribbing the job definition | 16:01 |
| jeblair | mordred: that works, or we can shorten the hierarchy by duplicating the nodepool thing in legacy-dsvm-base | 16:02 |
| mordred | jeblair: yah- good point | 16:02 |
| SpamapS | jeblair: thanks. I suppose I should actually read through that rather than trying to use whatever I've picked up through osmosis :) | 16:02 |
| jeblair | SpamapS: the playbooks/jobname/run pattern is nice for repos full of jobs. i rather like the playbooks/jobname pattern for repos with just a few jobs (ie, project repos). | 16:03 |
| jeblair | (it felt really good when i was putzing with the devstack-gate repo) | 16:03 |
| SpamapS | http://paste.openstack.org/show/621729/ | 16:04 |
| SpamapS | I still get no playbook found, but I wonder if ^ has something to do with that. | 16:04 |
| jeblair | SpamapS: that's harmless | 16:04 |
| Shrews | jeblair: mordred: so i think i've narrowed down the 2.4 fails to the Ansible yaml parser incorrectly comparing bytes and strings, so our inventory file never gets parsed | 16:04 |
| mordred | jeblair: remote: https://review.openstack.org/506703 Add legacy-base job to handle /etc/nodepool | 16:04 |
| SpamapS | actually yeah, that's after the ERROR anyway | 16:04 |
| * SpamapS digs more | 16:04 | |
| Shrews | not sure if this is a known issue yet | 16:04 |
| jeblair | Shrews: fascinating! | 16:05 |
| mordred | Shrews: oh - so it might bea python3 specific bug | 16:05 |
| mordred | Shrews: you're having all the fun | 16:05 |
| Shrews | mordred: possibly | 16:05 |
| Shrews | file_name=b'/tmp/tmpyo9gthmm/zuul-test/49186e9c06814162bc4463bae615e6af/ansible/inventory', ext=b'.yaml' | 16:05 |
| Shrews | valid ext: ['.yml', '.yaml', '.json'] | 16:06 |
| Shrews | it tries to compare b'.yaml' to that list, and fails | 16:06 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Use legacy-base job for base of all converted jobs https://review.openstack.org/506705 | 16:07 |
| mordred | Shrews: that certainly does seem like a py3 sadness | 16:08 |
| jeblair | mordred: do we also need to reparent legacy-puppet-openstack-integration? | 16:08 |
| SpamapS | so, next problem | 16:08 |
| SpamapS | http://paste.openstack.org/show/621730/ | 16:09 |
| Shrews | ok, seems the fix is backported to 2.4 | 16:09 |
| SpamapS | recursive loop in variables | 16:09 |
| mordred | Shrews: fwiw, I think making all of those strings in that valid ext: list be b'' will be valid for both 2 and 3 | 16:09 |
| jeblair | mordred: we can probably skip publish-openstack-artifacts -- unlikely to use the nodepool stuff. | 16:09 |
| SpamapS | I looked at the variables, they don't look recursive to me. :-P | 16:09 |
| mordred | jeblair: I just added the pre-playbook to it | 16:09 |
| SpamapS | (I'm using zuul-jobs as-is) | 16:09 |
| jeblair | mordred: ack | 16:09 |
| SpamapS | but my project-config is forked hard, and may be missing a necessary role. | 16:09 |
| mordred | jeblair: and yes - and I think if we find publication jobs that fail because of missing nodepool - we'll rewrite those jobs post-migration | 16:09 |
| mordred | SpamapS: awesome - thanks for hitting that. - we have zuul_site_image_manifest_files so don't hit that loop ourselves | 16:10 |
| mordred | oh. HAHAHAHA | 16:11 |
| mordred | wow what a nice typo | 16:11 |
| mordred | SpamapS: there's a leftover bit from a transition there ... | 16:12 |
| SpamapS | oh wait I DO see the loop | 16:13 |
| SpamapS | doh doh doh! :) | 16:13 |
| * Shrews attempst to figure out the proper requirements.txt magic for using git+https for ansible | 16:13 | |
| SpamapS | also, question about the current log copying base job.. why would it skip job-output.txt ? | 16:14 |
| mordred | Shrews: patch coming | 16:14 |
| SpamapS | that seems like the most important log to keep. | 16:14 |
| Shrews | mordred: patch is already backported | 16:14 |
| mordred | Shrews: woot! | 16:14 |
| Shrews | mordred: we just have to pull the latest devel branch | 16:14 |
| Shrews | not pypi | 16:14 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Finish transition to zuul_site_image_manifest_files https://review.openstack.org/506708 | 16:15 |
| mordred | SpamapS: ^^ that should fix you | 16:15 |
| SpamapS | mordred: indeed! | 16:16 |
| Shrews | mordred: if you're curious: https://github.com/ansible/ansible/commit/11cc64db2142277ecff272675f8384a0566da72d | 16:17 |
| mordred | Shrews: excellent. well - this is an area where a nice job testing ansible devel against zuul can provide value :) | 16:18 |
| mordred | jeblair: could you poke https://review.openstack.org/#/c/506683 ? | 16:20 |
| mordred | SpamapS, tobiash: https://review.openstack.org/#/c/506705 if you have a sec? | 16:20 |
| tobiash | mordred: that looks easy | 16:22 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Switch to build-openstack-sphinx-docs for doc build jobs https://review.openstack.org/501246 | 16:23 |
| mordred | tobiash: I like to keep it simple when I can :) | 16:24 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: DNM: Test Ansible 2.4 https://review.openstack.org/505354 | 16:26 |
| SpamapS | weird. ctrl-O doesn't work on my dumb mac terminal :-P | 16:28 |
| Shrews | SpamapS: save yourself some headaches and install iTerm2 | 16:29 |
| SpamapS | Shrews: doesn't work there either | 16:30 |
| SpamapS | I have both, and I have had zero relief of headaches by iterm | 16:30 |
| SpamapS | they both still run on a mac after all | 16:30 |
| mordred | jeblair: I'm going to update that devstack-gate job with an actual generated job | 16:30 |
| SpamapS | so I have to do everything with weird tools | 16:30 |
| Shrews | *shrug* | 16:31 |
| SpamapS | what's weird is _only_ ctrl-o doesn't work | 16:31 |
| SpamapS | ctrl-q, ctrl-r, meta-o, are fine | 16:32 |
| SpamapS | just ctrl-o | 16:32 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use legacy-dsvm-base as base job for dsvm jobs https://review.openstack.org/506379 | 16:33 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add support for puppet-openstack-integration based jobs https://review.openstack.org/506389 | 16:33 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Rename tox_upper_constraints_file to tox_constraints_file https://review.openstack.org/502348 | 16:33 |
| jeblair | SpamapS: http://git.openstack.org/cgit/openstack/gertty/tree/examples/reference-gertty.yaml#n110 | 16:39 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Finish transition to zuul_site_image_manifest_files https://review.openstack.org/506708 | 16:40 |
| SpamapS | https://apple.stackexchange.com/questions/3253/ctrl-o-behavior-in-terminal-app explains :-P | 16:40 |
| SpamapS | oh hah | 16:40 |
| SpamapS | so we've been here efore | 16:40 |
| SpamapS | before even | 16:40 |
| SpamapS | jeblair: my muscles want it to be ctrl-o, so I'll add the stty to my .bash_profile ;) | 16:41 |
| SpamapS | I'd guess this is some pedantic BSD thing :) | 16:41 |
| jlk | isn't all of BSD just a large pile of pedantry? | 16:42 |
| SpamapS | all would be inaccurate | 16:42 |
| *** electrofelix has quit IRC | 16:43 | |
| SpamapS | 97.3% is pedantry | 16:43 |
| SpamapS | the rest is hacks | 16:43 |
| jlk | are you nit picking my statement about pedantry? | 16:46 |
| jlk | :D | 16:47 |
| Shrews | ooh, my 2.4 change now passes all but one test | 16:48 |
| jeblair | mordred: interestingly, i did not do the same format for the shell/cmd thing. i believe your form and mine both work. can you take a look at my comment in 506683 before i approve? | 16:48 |
| mordred | jeblair: yes - also - would you look at http://logs.openstack.org/05/506405/1/gate/tox-py35/977ee71/testr_results.html.gz and see if that's a thing we should worry about? | 16:49 |
| mordred | jeblair: oh - I like your form better | 16:49 |
| jeblair | mordred: that's a disturbing error. i will try to think of the implications. | 16:50 |
| mordred | jeblair: I _think_ with that form we can put the executable thing in the same dict ... | 16:51 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Log exeptions for module failure cases https://review.openstack.org/500336 | 16:55 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Put shell snippets into task.shell.cmd https://review.openstack.org/506683 | 16:56 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Use legacy-base job for base of all converted jobs https://review.openstack.org/506705 | 16:56 |
| mordred | jeblair: updated ^^ | 16:56 |
| mordred | jeblair: it's ok to recheck the migrate patch with the disturbing error right? | 17:04 |
| mordred | jeblair: (I'm 100% certain the migrate patch didn't cause it ... but I wanted to make sure to raise it to you before forgetting about it) | 17:04 |
| jeblair | mordred: yes | 17:05 |
| mordred | jlk: mind re-+3-ing https://review.openstack.org/#/c/506683 ? | 17:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fix create_branch in tests https://review.openstack.org/506723 | 17:11 |
| jeblair | mordred: i kind of doubt that's the actual problem, but it's the first weird thing i spotted when looking into it ^. would be nice to try that first, then dig more. | 17:11 |
| mordred | ++ | 17:13 |
| jeblair | i've got it looping on my workstation | 17:13 |
| jeblair | biab. | 17:13 |
| Shrews | yay more bugs | 17:22 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Migrate node information to nodeset instead of nodes https://review.openstack.org/506405 | 17:27 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fixed a few earlier review nits https://review.openstack.org/506406 | 17:28 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Don't emit net-info macro https://review.openstack.org/506412 | 17:28 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Correctly apply irrelevant-files to simple job names https://review.openstack.org/506415 | 17:30 |
| Shrews | mordred: another ansible bug: https://github.com/ansible/ansible/issues/30771 based on what we have in zuul_return.py | 17:34 |
| Shrews | *hopefully* our last hurdle | 17:35 |
| jlk | mordred: done | 17:36 |
| mordred | jlk: thanks! | 17:37 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: DNM: Test Ansible 2.4 https://review.openstack.org/505354 | 17:39 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Use zuul | zuul_legacy_vars for legacy environment https://review.openstack.org/506732 | 17:39 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Switch to build-openstack-infra-sphinx-docs for doc build jobs https://review.openstack.org/501246 | 17:41 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Refactor provider config to driver module https://review.openstack.org/488384 | 17:44 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a static driver for Nodepool https://review.openstack.org/468624 | 17:44 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement an OpenContainer driver https://review.openstack.org/468753 | 17:44 |
| jeblair | mordred: i reproduced that error in my local loop even with that fix (we should still land the fix) | 17:48 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Put shell snippets into task.shell.cmd https://review.openstack.org/506683 | 17:48 |
| jeblair | that eliminates test isolation problems | 17:48 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Re-add the mapping for logs/$LOG_PATH https://review.openstack.org/506736 | 17:50 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use legacy-base job for base of all converted jobs https://review.openstack.org/506705 | 17:51 |
| mordred | jeblair: kk | 17:53 |
| mordred | jeblair: fwiw - I'm iterating right now with devstack-gate and the migration scrip | 17:53 |
| mordred | jeblair: 2017-09-22 18:04:44,583 INFO zuul.ExecutorDiskAccountant: /var/lib/zuul/builds/35998060bdb448bc89593acb34dda6fd is using 501MB (limit=500) | 18:06 |
| jeblair | mordred: yay it works! | 18:07 |
| mordred | jeblair: so - a) disk space killer wored! b) it does not communicate job aborted to due to disk-space issues | 18:07 |
| mordred | legacy-tempest-dsvm-neutron-full finger://ze02.openstack.org/35998060bdb448bc89593acb34dda6fd : POST_FAILURE in 10m 00s was what was reported | 18:07 |
| jeblair | mordred: yeah, looks like there is no facility for that, but could be added. | 18:08 |
| mordred | jeblair: yah | 18:09 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: DNM: Test Ansible 2.4 https://review.openstack.org/505354 | 18:09 |
| mordred | I have added an autohold on legacy-tempest-dsvm-neutron-full and have rechecked it | 18:09 |
| mordred | I do not konw why it was unhappy | 18:09 |
| mordred | jeblair: so - real quick... | 18:12 |
| mordred | jeblair: look at http://logs.openstack.org/57/490057/17/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/615f31a/_zuul_ansible/post_playbook | 18:12 |
| mordred | jeblair: and then at https://review.openstack.org/#/c/506352/12/playbooks/legacy/tempest-dsvm-neutron-full/post.yaml | 18:12 |
| mordred | jeblair: the v2.5 uses src of /home/jenkins/workspace/gate-tempest-dsvm-neutron-full-ubuntu-xenial/ (trailing slash) | 18:12 |
| mordred | jeblair: while the v3 one is using {{ ansible_user_dir }} | 18:12 |
| mordred | jeblair: do we need to append a / to get the rsync magic to work properly and match? | 18:13 |
| jeblair | mordred: ye | 18:13 |
| jeblair | s | 18:13 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Append / to end of ansible_user_dir for rsync src https://review.openstack.org/506740 | 18:13 |
| mordred | jeblair: k. thanks. done in both places | 18:14 |
| Shrews | hrm, we have sort of chicken and egg scenario with having zuul test itself with 2.4 | 18:31 |
| mordred | Shrews: whyso? | 18:32 |
| Shrews | mordred: well, maybe i'm wrong. i can't get the zuul-stream functional test to pass. not sure what "ansible" it's using to run that test | 18:33 |
| Shrews | mordred: http://logs.openstack.org/54/505354/6/check/zuul-stream-functional/ddcc366/job-output.txt.gz#_2017-09-22_18_17_02_854992 | 18:34 |
| Shrews | that *should* work if it were using the upstream devel branch (as i have in requirements.txt), but maybe that's not being used for that test? | 18:34 |
| Shrews | oh, i wonder if the installation of ara is downgrading it | 18:35 |
| jlk | it likely is | 18:36 |
| jlk | because ARA can't handle 2.4 yet | 18:36 |
| Shrews | so a different chicken with a different egg | 18:37 |
| Shrews | well, i think i have all of the zuul issues covered for when 2.4.1 drops, at least. 2.4 is a definite no-go | 18:41 |
| Shrews | jlk: how far did you get with the ansible modules? | 18:42 |
| Shrews | i saw mordred has some changes up | 18:43 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use zuul | zuul_legacy_vars for legacy environment https://review.openstack.org/506732 | 18:43 |
| mordred | Shrews: the way to find the info of a node that was autoheld is to ask nodepool yeah? | 18:44 |
| jeblair | mordred: yes, nodepool list --detail | 18:44 |
| Shrews | mordred: yeah. nodepool list --detail | 18:44 |
| jeblair | that worked out nicely :) | 18:45 |
| jlk | Shrews: I don't think anything was broken other than what mordred did | 18:46 |
| mordred | Shrews, jlk: KeyError: 'citycloud-sto2' | 18:46 |
| jlk | we COULD make improvements to a few, around boolean handling. | 18:46 |
| jlk | I.. what? | 18:46 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Re-add the mapping for logs/$LOG_PATH https://review.openstack.org/506736 | 18:46 |
| mordred | Shrews, jlk: I get that when I try to do nodepool delete 0000033403 on nl01.openstack.org | 18:46 |
| mordred | gah | 18:46 |
| mordred | sorry | 18:46 |
| mordred | jeblair: ^^ | 18:47 |
| mordred | (not jlk) | 18:47 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Append / to end of ansible_user_dir for rsync src https://review.openstack.org/506740 | 18:47 |
| *** dmsimard is now known as dmsimard|afk | 18:47 | |
| Shrews | mordred: neat. i wonder if citycloud-sto2 exists in nodepool.yaml | 18:48 |
| jeblair | mordred: hrm. i guess you have to do that on nl02. :| | 18:48 |
| jeblair | Shrews: not in the one on nl01 | 18:48 |
| mordred | ok. fixing that is definitely a thing to put onto the todo list | 18:48 |
| Shrews | jeblair: yeah. not sure why that's happening? it should just set state=DELETE | 18:48 |
| jeblair | Shrews: ++ hopefully we can avoid whatever lookup it's doing there | 18:49 |
| mordred | doing it on nl02 worked | 18:49 |
| jeblair | (if you did '--now' that would be different....) | 18:49 |
| Shrews | i'm done with the 2.4 stuff, so i'll look at fixing that next | 18:49 |
| jeblair | thx | 18:50 |
| mordred | I've put another autohold on for the d-g job | 18:50 |
| mordred | because I think my last one captured a node that I aborted the run on with a new patchset :) | 18:50 |
| jeblair | hah | 18:51 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix node delete command https://review.openstack.org/506750 | 18:56 |
| Shrews | mordred: i think ^^^ should fix that delete problem | 18:56 |
| jeblair | Shrews: those jobs went surprisingly poorly | 19:03 |
| jeblair | it's rare that everything *but* pep8 fails | 19:03 |
| jeblair | Shrews: spotted the error | 19:04 |
| jeblair | and in fact, pyflakes should have caught that | 19:04 |
| jeblair | i'm going to grab lunch | 19:04 |
| mordred | jeblair: remote: https://review.openstack.org/506753 Add zaqar to base legacy-dsvm-base required-projects | 19:05 |
| mordred | jeblair: that's why the dsvm job is failking | 19:05 |
| mordred | jeblair: I do not yet know why the log collection is failing | 19:05 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix node delete command https://review.openstack.org/506750 | 19:06 |
| *** weshay is now known as weshay_bbiab | 19:15 | |
| *** dmsimard|afk is now known as dmsimard | 19:33 | |
| dmsimard | mordred, jeblair: zuul-sphinx doesn't let us do basic RST like title/headers? :/ http://logs.openstack.org/54/504554/7/check/build-openstack-sphinx-docs/a711aec/ara/result/b41acceb-d0f2-45ad-ad96-92010b25f63b/ | 19:34 |
| jeblair | dmsimard: you're *inside* a section. | 19:37 |
| jeblair | dmsimard: or, rather, inside a content block | 19:38 |
| dmsimard | wait, the readme.rst are "fragments" ? | 19:38 |
| jeblair | dmsimard: yep | 19:38 |
| dmsimard | oh, uh | 19:38 |
| jeblair | dmsimard: it's the content of a ..zuul::role: | 19:39 |
| dmsimard | ok, I didn't realize that. I'll fix my stuff. | 19:39 |
| jeblair | dmsimard: so i think we've been making pseudo-sections like ** Role Variables ** | 19:39 |
| dmsimard | yeah that's what I was essentially doing actually | 19:39 |
| jeblair | right, but i mean with '**' to make it bold | 19:40 |
| dmsimard | jeblair: https://review.openstack.org/#/c/504554/7/roles/multi-node-bridge/README.rst | 19:40 |
| dmsimard | yeah ok I see what you mean. | 19:40 |
| jeblair | i wonder if file matchers should always implicitly include .zuul.yaml (and friends) | 19:57 |
| jeblair | for example, https://review.openstack.org/506739 currently has no direct testing because it's only changing a job definition of a job with file matchers. | 19:58 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Fix node delete command https://review.openstack.org/506750 | 19:59 |
| dmsimard | jeblair: zuul.yaml and friends should probably always trigger check/gate, yeah | 20:00 |
| dmsimard | otherwise we can't even merge the patch ? | 20:01 |
| jeblair | dmsimard: yep. (this is clarkb's objection to file matchers :) | 20:01 |
| jeblair | dmsimard: we can merge this patch right now, however, since it's still managed by zuulv2 | 20:01 |
| jeblair | but that's obviously a temporary situation | 20:01 |
| dmsimard | odyssey4me, tristanC, tobiash: got a pong from earlier this morning about building images with ansible-container, there's https://github.com/ansible/ansible-container/pull/683 in the pipe for buildah (whatever that is) | 20:03 |
| dmsimard | I've seen buildah mentioned before but have no clue what it is | 20:03 |
| dmsimard | oh, looks like it's an atomic wrapping for building OCI images, okay. | 20:03 |
| *** hasharAway is now known as hashar | 20:04 | |
| dmsimard | s/wrapping/wrapper/ | 20:04 |
| pabelanger | dmsimard: I actually prefer https://review.openstack.org/385608/ to ansible-container. DIB to bootstrap minimal image (debootstrap) then run ansible-playbook with connection=chroot | 20:13 |
| pabelanger | once zuulv3 is rolled out, I'm going to loop back to it | 20:14 |
| dmsimard | pabelanger: yeah we were just discussing different ways of building/distributing images out of nodepool builder | 20:14 |
| dmsimard | I'm personally not a fan of diskimage-builder so I'm biased against it from the beginning, the ansible element makes it a little bit better :p | 20:15 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Actually, you know, return the timeout https://review.openstack.org/506768 | 20:25 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Multi-node: Set up hosts file https://review.openstack.org/504552 | 20:26 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Multi-node: Set up firewalls https://review.openstack.org/504553 | 20:26 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Multi-node: Set up connectivity with default OVS bridge https://review.openstack.org/504554 | 20:26 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Multi-node: Streamline multi-node-known-hosts https://review.openstack.org/504629 | 20:26 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Append /sbin and /usr/sbin to $PATH for 'ip' commands https://review.openstack.org/504743 | 20:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Switch to build-openstack-infra-sphinx-docs for doc build jobs https://review.openstack.org/501246 | 20:32 |
| mordred | jeblair: I like the implicit .zuul.yaml file matcher | 20:34 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Switch to build-openstack-infra-sphinx-docs for doc build jobs https://review.openstack.org/501246 | 21:06 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Actually, you know, return the timeout https://review.openstack.org/506768 | 21:11 |
| *** harlowja has joined #zuul | 21:23 | |
| *** hashar has quit IRC | 21:33 | |
| SpamapS | man, github not letting one review their own PR is kind of lame. | 22:29 |
| SpamapS | makes testing zuul alone a bit complicated | 22:29 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Don't double quote rsync options https://review.openstack.org/506802 | 23:01 |
| jeblair | SpamapS: who doesn't review their own code? i find as many bugs in my code doing that as other people do. | 23:04 |
| jeblair | (if you're going to ask someone else to do it, the least you could do is do it yourself! it's just polite!) | 23:04 |
| dmsimard | I review my own code all the time :/ | 23:44 |
| dmsimard | I'm harder on myself than onto others lol | 23:44 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix create_branch in tests https://review.openstack.org/506723 | 23:44 |
| clarkb | I find it particularly useful especially if given a bit of time between first writing it then reviewing it | 23:46 |
| SpamapS | jeblair: yeah, and I've even done a few gerrit reviews where I submitted the first comments in-line, like "I'm not sure this is really a thing, please pay attention here" | 23:52 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!