| *** dkranz has quit IRC | 01:59 | |
| *** pbelamge has joined #zuul | 05:14 | |
| *** pbelamge has quit IRC | 05:18 | |
| *** hashar has joined #zuul | 05:56 | |
| *** dmsimard has quit IRC | 06:50 | |
| *** pbelamge has joined #zuul | 07:00 | |
| *** dmsimard has joined #zuul | 07:19 | |
| *** bhavik1 has joined #zuul | 09:08 | |
| *** bhavik1 has quit IRC | 09:40 | |
| *** SotK has quit IRC | 10:22 | |
| *** SotK has joined #zuul | 10:22 | |
| *** jkilpatr has quit IRC | 10:36 | |
| *** hashar is now known as hasharLunch | 10:38 | |
| *** jamielennox has quit IRC | 10:51 | |
| *** jamielennox has joined #zuul | 10:52 | |
| *** jkilpatr has joined #zuul | 10:58 | |
| *** hasharLunch is now known as hashar | 12:40 | |
| leifmadsen | jeblair: mordred: did a patch get pushed to gerrit yet? if so, I'd like to mention the link on the notes from last week | 13:03 |
|---|---|---|
| *** dkranz has joined #zuul | 13:27 | |
| mordred | leifmadsen: yah - https://review.openstack.org/#/c/498050/ | 13:44 |
| leifmadsen | hawt | 13:44 |
| leifmadsen | I'll note it! | 13:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add unittest for secrets data https://review.openstack.org/484911 | 13:56 |
| Shrews | hrm, we should get tristanC to look at 498050, too | 13:59 |
| leifmadsen | all teh lookz! | 14:02 |
| leifmadsen | Shrews: made really good progress on Friday with mordred and jeblair. We have a bunch of notes while stepping through if you're interested. Made it I think about half way. Got through to the point that nodepool can instantitate VMs in rdocloud | 14:02 |
| Shrews | leifmadsen: great. was out of town friday, so i'd be interested in seeing those notes | 14:03 |
| leifmadsen | linking | 14:04 |
| leifmadsen | https://etherpad.openstack.org/p/zuulv3-quickstart | 14:04 |
| leifmadsen | still more to do, but we started from a fresh CentOS 7.3 VM, and then started building things out | 14:04 |
| leifmadsen | the goal is to build a hello world, triggered from a GitHub PR and resulting in an Ansible run that does something dumb like echo "Hello world" :) | 14:05 |
| leifmadsen | per our conversation a couple weeks ago | 14:05 |
| *** _ari_ is now known as _ari_|conf | 14:05 | |
| leifmadsen | mordred: I had been thinking more over the weekend about why I like doing this, "start from scratch" type documentation, and realized that it's the same thing we did for the Asterisk book. Documenting things from pure scratch typically results in bugs found and fixed. We found (and fixed) a ton while writing the Asterisk book. | 14:06 |
| leifmadsen | Found at least one here too :) | 14:06 |
| mordred | leifmadsen: yah - I think it was really useful | 14:07 |
| leifmadsen | for sure, I learned a ton, and you learned a ton :D | 14:08 |
| leifmadsen | amazing what happens when you have to go back and redeploy your own software from scratch lol | 14:09 |
| leifmadsen | I'm really looking forward to getting this all done so that I can run with it and build out some larger scenarios | 14:09 |
| leifmadsen | get this out of the way, then start taking all the ideas jeblair had, and we could probably end up writing a small Zuul book | 14:10 |
| mordred | leifmadsen: we should write a large Zuul book ... because we're going to need chapters on the idea of gating in the first place, and how to philosophically appraoch development in a world where you have the ability to do multi-repo gating | 14:49 |
| mordred | leifmadsen: it'll be a fun book to write | 14:49 |
| mordred | jeblair: I've done the refactor on the nodepool patch | 14:59 |
| mordred | jeblair: one remaining thing is that I kind of think we should error in some way if you have a config that references a cloud-image that doesn't exist in your config - but we don't have much in the way of erroring on invalid config | 14:59 |
| mordred | jeblair: I'm just putting in a code TODO for now | 15:00 |
| mordred | or, actually, we do raise in one place - I'm going to try that and see how it feels | 15:00 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 15:03 |
| mordred | jeblair, leifmadsen, Shrews: ^^ I think that's nicer now - I moved the logic into the ProviderCloudImage class and had the config reading put the ProviderCloudImage into the ProviderLabel instead of just the cloud-image name | 15:05 |
| mordred | Shrews: ^^ that's the bug we found in nodepool working through setting up a minimal nodepool wth leifmadsen - it still needs a unit test which I haven't gotten to. makes me think we should also add a pre-existing image entry to the devstack job, maybe referencing that cirros image that devstack uploads or something | 15:06 |
| Shrews | mordred: i seem to recall tristanC intentionally used labelReady() instead of imageReady() for reasons. i'll let tristan comment on that | 15:06 |
| mordred | Shrews: yah - I re-updated the patch and it's back to labelReady | 15:06 |
| Shrews | oh | 15:07 |
| mordred | now that ProviderCloudImage is in place, the awkward dance of passing stuff around isn't needed anymore | 15:07 |
| Shrews | mordred: ??? https://review.openstack.org/#/c/498050/2/nodepool/driver/__init__.py | 15:07 |
| mordred | and I agree - the thing the launcher is asking about is "do we have images ready for this label" | 15:07 |
| mordred | gah | 15:07 |
| mordred | one sec | 15:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 15:11 |
| mordred | Shrews: thanks - I had updated it in the method and forgot to fix the call sites :) | 15:11 |
| Shrews | wow. that seems to break the world | 15:19 |
| mordred | yeah? | 15:20 |
| mordred | piddle | 15:20 |
| jeblair | the general approach makes sense to me though | 15:20 |
| Shrews | is "python -m venv env" the same as "virtualenv env" ? The first is new to me | 15:22 |
| mordred | Shrews: python -m venv doesn't seem to work for me -where's that? | 15:22 |
| Shrews | err, python3 -m venv | 15:23 |
| Shrews | it's in the etherpad leifmadsen linked | 15:23 |
| Shrews | line 227 | 15:24 |
| mordred | interesting - I think it may have something to do with ensurepip - it breaks in a different way on my laptop | 15:25 |
| Shrews | mordred: i guess it's new to you, too :) | 15:25 |
| clarkb | python3 has a built in virtualenv tool. I don't know that anyone actually uses it | 15:26 |
| mordred | Shrews, clarkb: yes - I have just verified that on centos7 if you install epel then instlal python34-pip and then run python3 -m venv env | 15:28 |
| mordred | that will create a virtualenv called env | 15:28 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 15:29 |
| leifmadsen | I'm the guy that throws a grenade in the room and lets all the engineers develop a system to put the pin back in | 15:35 |
| Shrews | jeblair: care to +A https://review.openstack.org/497480 so nodepool tests are stable again for us? | 15:36 |
| jeblair | Shrews: :( done | 15:37 |
| Shrews | jeblair: it's a head scratcher, for sure | 15:38 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Revert "Allow launcher to stop quicker when asked" https://review.openstack.org/497480 | 15:42 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Run post playbooks on job timeout https://review.openstack.org/498270 | 15:57 |
| jeblair | hah! i'm pretty sure the current job timeout is in seconds, but the docs say minutes. which should we use? | 16:02 |
| jeblair | (or should we stick with integer seconds but also have the configloader handle string values such as "90m" and "2h" ?) | 16:04 |
| pabelanger | 90m / 2h might be more user friendly | 16:06 |
| pabelanger | but, no preference here | 16:06 |
| jeblair | okay, i'm going to set another autohold so i can iterate on devstack-legacy in place | 16:08 |
| mordred | jeblair: cool | 16:14 |
| mordred | jeblair: I also agree - seconds - but if we can do friendly names with suffixes like 90m / 2h that would be nice | 16:15 |
| *** pbelamge has quit IRC | 16:16 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fix job timeout docs https://review.openstack.org/498513 | 16:20 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Fail early if people attempt to add zuul vars or secrets https://review.openstack.org/484000 | 16:22 |
| jlk | o/ | 16:41 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add role for adding ssh key to remote nodes https://review.openstack.org/498109 | 16:45 |
| jeblair | mordred: not sure if you saw jlk's post-approval comment on https://review.openstack.org/498109 | 16:46 |
| pabelanger | jeblair: just mocking up afs publisher here locally, if we kinit / aklog before our rsync, we should be able to pull directly from node to /afs/.openstack.org/docs. Or, would you like to first stage into jobsdir, then kinit / aklog, rsync to /afs/.openstack.org/docs | 16:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Run post playbooks on job timeout https://review.openstack.org/498270 | 16:47 |
| jeblair | pabelanger: if you can swing it in one pass, i think that's fine. i assume we had it in two because of some complication in matching the jenkins behavior. | 16:50 |
| pabelanger | k, I'll attempt that first | 16:51 |
| dmsimard | jlk: re: block dashes | 17:01 |
| dmsimard | jlk: I guess you're right, it's probably a syntax I picked up from openshift-ansible :) | 17:02 |
| jlk | it's somewhat sad it works both ways. | 17:02 |
| jlk | honestly I don't care much, so long as we all agree on a way of doing it | 17:02 |
| dmsimard | jlk: well, it's not so much ansible as it is yaml | 17:02 |
| dmsimard | yaml doesn't care about the order of the keys | 17:03 |
| dmsimard | but ++ on using the same syntax consistently everywhere | 17:03 |
| dmsimard | I don't like seeing the yaml syntax mixed with the ansiblyaml syntax (with ='s) | 17:04 |
| jlk | same | 17:04 |
| dmsimard | ansible-lint allows us to write custom rules iirc, maybe we can consider doing that later | 17:04 |
| jlk | mordred: jeblair: I'm poking at a follow up role to remove the added key bit that just got merged to zuul-jobs. | 17:04 |
| jeblair | jlk: thanks | 17:05 |
| jlk | jeblair: mordred: I wonder if add-build-sshkey and add-sshkey will trample on each other. | 17:07 |
| dmsimard | mordred: did you need to land anything in zuul *before* I ship the ara dot release ? | 17:07 |
| jeblair | dmsimard: nah we should be fine | 17:07 |
| jlk | it LOOKS like they might touch the same file (~/.ssh/id_rsa of whatever user ansible is connecting as) | 17:08 |
| dmsimard | ok, going through a last sanity check and more than likely tagging ara for release | 17:08 |
| jeblair | jlk: i think so. that's probably okay by virtue of their being used by different jobs, though really we should alter the proposal jobs to use a non-default key. but maybe we can do that post-cutover where there are fewer moving parts. | 17:09 |
| jlk | okay | 17:09 |
| jeblair | jlk: (by different jobs, i mean that the proposal jobs are not multinode, and so won't be performing any inter-node ssh ops) | 17:09 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul-jobs master: Add a role to remove an ssh private key https://review.openstack.org/498530 | 17:11 |
| *** bhavik1 has joined #zuul | 17:39 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Add create / destory roles for AFS tokens https://review.openstack.org/498547 | 17:53 |
| *** bhavik1 has quit IRC | 18:00 | |
| dmsimard | jeblair, mordred: btw, a question on openstack-infra http://lists.openstack.org/pipermail/openstack-infra/2017-August/005566.html | 18:47 |
| jeblair | dmsimard: well, the plan is that every project in openstack-infra will use zuulv3 if we can finish our outstanding tasks by the ptg | 18:52 |
| dmsimard | jeblair: technically ara is not openstack-infra :) | 18:52 |
| jeblair | dmsimard: every openstack project | 18:52 |
| jeblair | dmsimard: every project using the infrastructure run by openstack-infra | 18:53 |
| dmsimard | jeblair: oh, I see what you mean. I know the cutover is right before the PTG but I meant to ask if we could enable it sooner rather than later so I can start addressing potential changes required between zuul v3 and ara 1.0 progressively rather than in one big chunk. There's already a lot of work done in ara 1.0 and there will be a bunch more by the PTG. | 18:55 |
| jeblair | dmsimard: i think we need to focus exclusively on preparing for the ptg cutover or it won't happen | 18:55 |
| dmsimard | I don't necessarily disagree, but most of my work on ARA is on my personal time, not work time. Is enabling v3 for ara an arduous process ? | 18:57 |
| dmsimard | This won't prevent me from continuing to contribute to the migration efforts, but I can wait I guess | 18:59 |
| jeblair | dmsimard: thanks, i appreciate that | 18:59 |
| Shrews | mordred: curious about https://review.openstack.org/497968 ... zookeeper is required, but it's not required on the same host as nodepool. | 19:03 |
| jeblair | Shrews: that's a good point; i think we were very focused on the all-in-one scenario. maybe we can add another tag for that? | 19:06 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Set base environment as python3 https://review.openstack.org/496251 | 19:06 |
| Shrews | jeblair: tag? | 19:08 |
| Shrews | i assume that's a bindep thing. i haven't taken the time to learn that. perhaps i should :) | 19:09 |
| jeblair | Shrews: bindep [bracket thingies] are arbitrary | 19:09 |
| jeblair | Shrews: like "[test]" | 19:09 |
| jeblair | so maybe "zookeeperd [allinone]" or something? | 19:09 |
| Shrews | ah, i see. yes. i think that would be better | 19:10 |
| pabelanger | jeblair: I've been reading up on rsync and filters, by default ansible will pass the -F flag ( --filter='dir-merge /.rsync-filter') by default. This then allows us to potentially use the .rsync-filter file in our directories to exclude things. As an example, I am testing http://paste.openstack.org/show/619673/ locally and it does appear to be working how I _think_ it should be. When you did the original afs | 19:22 |
| pabelanger | docs, did you consider using -F for root-markers? | 19:22 |
| dmsimard | mordred, jeblair: ara 0.14.1 is tagged, it will be released when the gate is in a bit of a better shape | 19:28 |
| jeblair | pabelanger: i don't think i understand the connection you're making | 19:33 |
| jeblair | pabelanger: how is -F related to root-markers? | 19:33 |
| pabelanger | jeblair: sure, looking at the rsync command we are using today in zuulv2.5, I see we are setting up a filter, based on how the root-marker was found: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/ansible/library/zuul_afs.py#n72 We dynamically create the excludes file, then pass it to rsync. | 19:35 |
| pabelanger | I am curious if we still need to do that, and if so, could using -F (.rsync-filter) be an option moving forward in zuulv3 | 19:36 |
| jeblair | pabelanger: do you have a specific proposal in mind? | 19:36 |
| pabelanger | jeblair: not really, I'm just looking to see if we can leverage the existing functionality today with synchronize module otherwise, I am wonder if we might just import zuul_afs into project-config / zuul-jobs and continue using that for the sync logic. | 19:39 |
| pabelanger | I mean, the building of the root-marker logic, kinit / aklog is handled | 19:40 |
| openstackgerrit | Merged openstack-infra/nodepool master: Clarify diskimage names in docs https://review.openstack.org/490781 | 19:41 |
| jeblair | pabelanger: something has to build the root-marker list, right? after that, whether the exclude list is incorporated implicitly (-F) or explicitly (--filter=) doesn't seem consequential to me -- except one of them opens up an avenue for jobs to manipulate the result by laying down their own rsync filter files. | 19:42 |
| jeblair | pabelanger: at any rate, building the exclude list is the complicated part. you could probably do it with a find command. or you could use a module that uses the existing python code | 19:43 |
| jeblair | pabelanger: after that, i don't think using 'synchronize' vs 'command: rsync' buys us much, so i wouldn't expend too much energy trying to use it if it doesn't fit | 19:44 |
| mordred | yah- my thought was just putting zuul_afs module into zuul-jobs | 19:45 |
| pabelanger | okay, so are we open to the idea of having a job manipulate the result by adding their own rsync filter files? | 19:45 |
| jeblair | pabelanger: i think it's a bad idea | 19:46 |
| pabelanger | okay | 19:46 |
| jeblair | i think docs publishing is hard enough to understand as it is | 19:46 |
| dmsimard | jlk: this inventory and network overlay stuff is eating away at my patience :) | 19:49 |
| jlk | haha :( | 19:49 |
| pabelanger | okay, so sounds like we could move zuul_afs into zuul-jobs, extract out bits for kinit / aklog, and leave the rest of root-marker / rsync logic alone for now | 19:49 |
| dmsimard | jlk: found two new issues, fixed one.. working on the other | 19:50 |
| dmsimard | jlk: both fixed, https://review.openstack.org/#/c/498207/ had a stray host_counter left and https://review.openstack.org/#/c/435933/ I forgot to add [0] when retrieving the hostvars | 19:55 |
| dmsimard | hopefully this one is the good one | 19:55 |
| mordred | jlk, jeblair: from scrollback, I agree re: add-sskey and add-build-sshkey - add-sshkey will definitely step on the stuff multinode does - but as jeblair said I think that's fine - it's certainly worth clarifying things though | 19:57 |
| dmsimard | clarkb, jeblair: still looking for a last +2 on 496355, 496356 (I actually hit a 2.2 specific issue in a review) and 496935 | 19:57 |
| dmsimard | afk for a bit | 19:59 |
| *** jkilpatr has quit IRC | 20:00 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 20:00 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Create upload-afs role https://review.openstack.org/498588 | 20:19 |
| *** dkranz has quit IRC | 20:29 | |
| jeblair | reloading zuulv3 to pick up new tenant config | 21:03 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 21:04 |
| jeblair | jlk: can you help make sense of this error? http://paste.openstack.org/show/619694/ | 21:04 |
| jeblair | jlk, mordred: also, based on logs, it looks like we're getting github events for 3 ansible forks that aren't familiar to me | 21:05 |
| jeblair | they arrive together: http://paste.openstack.org/show/619696/ | 21:06 |
| *** jkilpatr has joined #zuul | 21:06 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Create upload-afs role https://review.openstack.org/498588 | 21:09 |
| mordred | jeblair: looking | 21:09 |
| jeblair | mordred: it doesn't seem to be urgent, so no worries if we just want to note that for later | 21:10 |
| mordred | jeblair: fascinating - yah, let's look at that later, I bet there is somethign in the fork data model we didn't know about | 21:11 |
| mordred | jeblair, pabelanger: ^^ oh neat! so with v3+bubblewrap we can just kinit and not have to wrap each call in k5start? | 21:30 |
| jeblair | mordred: yep (of course we can still use k5start if it's convenient) | 21:31 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add create / destory roles for AFS tokens https://review.openstack.org/498547 | 21:33 |
| mordred | jeblair: woot | 21:35 |
| mordred | pabelanger: +2 on the publish role - with comments. (as in, I'd rather forward progress continue, but the docs are confusing if anybody gets a moment) | 21:36 |
| pabelanger | sure, I can update it shortly | 21:37 |
| mordred | pabelanger: also, if you get a sec, 498111 and its two parents, 498559 and 497618 are all good to go | 21:43 |
| pabelanger | sure, looking now | 21:44 |
| pabelanger | 559 still in check, so only did +2 to wait un results. Everything +3 | 21:48 |
| mordred | \o/ | 21:49 |
| dmsimard | Zuul meeting in 10 minutes in #openstack-meeting-alt | 21:51 |
| dmsimard | mordred, jeblair, SpamapS, jlk, pabelanger, fungi, leifmadsen, clarkb ^ | 21:51 |
| fungi | yay! | 21:52 |
| dmsimard | Etherpads of interest: https://etherpad.openstack.org/p/zuulv3-pre-ptg && https://etherpad.openstack.org/p/AIFz4wRKQm | 21:54 |
| *** hashar has quit IRC | 22:10 | |
| clarkb | dmsimard: I actually see why it works now, cryptography publishes a generic linux wheel now | 22:18 |
| clarkb | dmsimard: dvr grenade multinode job doesn't look so happy | 22:40 |
| dmsimard | clarkb: yeah I commented on it | 22:40 |
| clarkb | I'm trying to find where (if anywhere) we've logged the steps taken to set up the ovs tunnel | 22:40 |
| dmsimard | clarkb: the initial tempest run came back green | 22:40 |
| clarkb | ara errors when I try to see the things | 22:41 |
| dmsimard | clarkb: actually it fails on the parent commit | 22:41 |
| dmsimard | clarkb: the inventory rework | 22:41 |
| dmsimard | clarkb: https://review.openstack.org/#/c/498207/ | 22:41 |
| dmsimard | tempest is green on what I assume is the original deployment http://logs.openstack.org/07/498207/6/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/7ad728b/logs/old/testr_results.html.gz | 22:41 |
| dmsimard | but then fails ... pinging cinder ? http://logs.openstack.org/07/498207/6/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/7ad728b/logs/grenade.sh.txt.gz#_2017-08-28_20_58_47_957 | 22:42 |
| * dmsimard not super familiar with grenade or devstack jobs | 22:42 | |
| mordred | dmsimard: I think mapping v3 git repo states into what's needed for grenade may still need some braining | 22:43 |
| mordred | oh - but that's a v2 run | 22:43 |
| clarkb | dmsimard: http://logs.openstack.org/33/435933/46/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/18ea9dc/logs/old/testr_results.html.gz its failing | 22:43 |
| clarkb | the link you have is an old run | 22:43 |
| clarkb | er its failing the first time around on failing to ssh so something is broken with the overlay I think | 22:43 |
| mordred | clarkb: those are two different job | 22:44 |
| dmsimard | clarkb: it's not worth looking at overlay patch until the parent inventory patch is passing | 22:44 |
| clarkb | oh got it | 22:44 |
| dmsimard | clarkb: which is the job results I was referring to came from | 22:44 |
| dmsimard | inventory patch: https://review.openstack.org/#/c/498207/ | 22:45 |
| clarkb | call me insane but I really don't like that patch... | 22:45 |
| clarkb | it is way more difficult to read | 22:46 |
| clarkb | also treating those bash files as ini is not really correct... though at this point unlikely to cause problems | 22:46 |
| dmsimard | clarkb: it's a way to get key=value vars from the file | 22:47 |
| clarkb | I think keeping that in bash will be significantly smaller since you can just source the files | 22:48 |
| clarkb | (and be much easier to read) | 22:48 |
| clarkb | and it will parse correctly | 22:48 |
| dmsimard | clarkb: there are things that ansible makes easier, like differentiating v4 vs v6 | 22:49 |
| dmsimard | clarkb: ultimately I use the "v3-style" hostvars in the network overlay patch to use the right ip addresses at the right place | 22:50 |
| dmsimard | writing it in bash kind of sucks because the var format sucks, it's like nodepool='{"key": "value", "anotherkey": "anothervalue"}' | 22:50 |
| dmsimard | doesn't mean I can't do it, but it's not going to be particularly clean | 22:51 |
| clarkb | ya but you can actually safely parse those fiels (which your ansible is not doing, its working by way of hacks) | 22:51 |
| dmsimard | It's not too hacky, the format is static and it's using a "properties" style of ini lookup | 22:52 |
| *** olaph1 is now known as olaph | 22:52 | |
| dmsimard | unless we start changing the /etc/nodepool/provider format it should not break | 22:52 |
| clarkb | so you'd have localhost connection=local nodepool='{ "az": "$AZ", "ip": "$IP" }' | 22:52 |
| clarkb | and you'd end up with like 10 lines of bash | 22:52 |
| SpamapS | damn.. meatspace got me twice in that hour :-P | 22:52 |
| pabelanger | SpamapS: meatspace, I need to use that more | 22:53 |
| fungi | that and "wetware" | 22:53 |
| dmsimard | clarkb: I don't have a strong opinion, I just write less and less bash now that Ansible is in my life.. I can be swayed one way or the other | 22:54 |
| fungi | clarkb: yeah, the manylinux1 wheel on pypi for cryptography is the one i was referring to anyway, not the ones from our wheel builders | 22:54 |
| fungi | speaking of which, i still wish i had time to figure out how to get our wheel builders to only build wheels when there aren't any already in the pypi mirror | 22:55 |
| pabelanger | mordred: re: wheel mirrors, are you thinking we'd still rsync data back via executor bwrap environment? | 22:55 |
| mordred | pabelanger: nope | 22:55 |
| mordred | pabelanger: for wheel mirrors I believe we need to keep the wheel mirror build hosts around for now - because one of the important things they do is keep a local cache of wheels built | 22:56 |
| clarkb | dmsimard: I'm writing up what it would like like keeping bash processing of bash files. I think it may help convince it is nice and simple :) | 22:56 |
| mordred | pabelanger: so I think for the wheel mirror jobs we do the add-to-inventory trick from tarballs.o.o until we can handle them with static nodes | 22:57 |
| pabelanger | mordred: Right, but we might be able to use afs for that part too? Or would the network preformance on that be too bad | 22:57 |
| mordred | pabelanger: the structure is a bit different unfortunately | 22:57 |
| dmsimard | clarkb: sure, one challenge I potentially see with bash is emulating the "with_together" from Ansible but ultimately we don't really care about the public ipv4's of the subnodes so it's probably not that big of a deal to just not set them | 22:57 |
| fungi | pabelanger: afs on the executor (so there's some cache maintained) or on the individual nodes? | 22:58 |
| mordred | pabelanger: so we'd have to engineer something clever - which we probably shoudl do - but for now I think keeping those as makeshift static hosts will get us over the hump | 22:58 |
| clarkb | dmsimard: yes I am going to completely ignore public and interface ip as they are unnecessary here | 22:58 |
| fungi | the latter seems like it would not work out so well (likely worse than rebuilding them there) | 22:58 |
| mordred | fungi: yah | 22:58 |
| pabelanger | mordred: okay, static node works | 22:58 |
| mordred | pabelanger: luckily we have these nice roles made already to do that :) | 22:58 |
| clarkb | also ipv6 | 22:59 |
| pabelanger | fungi: haven't tested it, but jeblair added a trusted bind mount to bubblewrap that should allow us to kinit / aklog to the /afs/.openstack.org volumes | 22:59 |
| dmsimard | clarkb: like, even stuff like az, cloud and stuff. We probably don't care about it. I just went with parity over what v3 hostvars provided | 22:59 |
| pabelanger | mordred: indeed | 22:59 |
| fungi | once we can grow an extensible artifact stashing and sharing solution between job runs, something like a reused wheel build cache is probably a viable solution | 22:59 |
| mordred | fungi, pabelanger: yah - that's what's gonna let us do docs publication and the like | 22:59 |
| dmsimard | clarkb: maybe I'm making this complicated on myself by overdoing it :) | 22:59 |
| clarkb | dmsimard: yes I think so :) | 22:59 |
| clarkb | dmsimard: if we trim it down to only the necessary info the change becomes tiny (and much easier to udnerstand/debug) | 23:00 |
| jeblair | dmsimard, clarkb: remember that this inventory shim is temporary; it won't be used in the forward-looking v3 jobs. | 23:00 |
| clarkb | jeblair: ya exactly | 23:00 |
| dmsimard | jeblair: aye | 23:00 |
| fungi | hrm... thinking about that... maybe afs thruogh the executors (with a cache) _is_ an eventual solution to sharing intermediate build artifacts between jobs | 23:00 |
| mordred | pabelanger: I'll push up some jobs real quick for the wheel mirror building | 23:00 |
| dmsimard | clarkb: with that in mind, let me put a patch up with bash instead | 23:00 |
| dmsimard | clarkb: unless you want to send a patch, that's cool too | 23:01 |
| pabelanger | mordred: wfm | 23:01 |
| clarkb | dmsimard: I've got it half written if you want to wait a moment | 23:02 |
| dmsimard | ack | 23:02 |
| mordred | pabelanger: I think once your "publsh XXX to AFS from artifacts" stuff is good to go, then I think that actually takes care of _all_ of the jobs that publish to afs and we cna take care of them in migration | 23:02 |
| mordred | pabelanger: so that one is one of the ones that is a big bang-for-the-buck :) | 23:03 |
| pabelanger | mordred: ya, I think we can start testing something tomorrow morning. The part that still needs work is some of the .root-marker setup. But for testing, I'm going to keep it super simple. | 23:03 |
| mordred | pabelanger: luckily zuul publishes its docs via afs - so we've got a good test case :) | 23:04 |
| pabelanger | exactly | 23:04 |
| clarkb | dmsimard: http://paste.openstack.org/show/619713/ how does that look? | 23:04 |
| clarkb | dmsimard: I am somewhat assuming the format there is json? | 23:05 |
| dmsimard | clarkb: yeah, it's a var (nodepool) that is a dict (json) | 23:05 |
| * dmsimard looks | 23:05 | |
| clarkb | dmsimard: I'll let you push it if you like it and feel free to make hints as I was somewhat inferring the json stuff | 23:05 |
| clarkb | dmsimard: oh except that isn't backward compatible | 23:06 |
| clarkb | you have to keep the counter stuff to be backward compat | 23:06 |
| * clarkb edits | 23:06 | |
| dmsimard | clarkb: yeah I had replaced the counter by the index of the host in the group | 23:06 |
| dmsimard | clarkb: I guess we can keep the counter to change even less things :) | 23:07 |
| clarkb | http://paste.openstack.org/show/619714/ | 23:07 |
| clarkb | ya I forgot we had to keep counter to be forward and backward compatible | 23:07 |
| clarkb | dmsimard: should I go ahead and push that up? | 23:14 |
| dmsimard | clarkb: I'm sending it now | 23:15 |
| clarkb | ok | 23:15 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Include the prepared projects list in zuul variables https://review.openstack.org/498618 | 23:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Include the prepared projects list in zuul variables https://review.openstack.org/498618 | 23:31 |
| dmsimard | clarkb: because I wanted to confirm that I wasn't crazy -- the wheel building stuff is relatively new indeed https://github.com/pyca/cryptography/commit/cc78c30fd95bb57fe75330ff7c702eb1f2ac4695 | 23:38 |
| dmsimard | some amount of follow up patches to that | 23:39 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Create upload-afs role https://review.openstack.org/498588 | 23:39 |
| mordred | jeblair: dependencies can be added to a job variant in a project-pipeline definition right? | 23:42 |
| jeblair | mordred: should be | 23:42 |
| mordred | jeblair: k, cool. I just wrote a job that wants to depend on a different job - but which job it depends on could vary by project, so putting the dependon the job itself seemed wrong | 23:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Create zuul_executor_dest variable for fetch-sphinx-output https://review.openstack.org/498624 | 23:44 |
| jeblair | mordred: ya, i think of that as the primary use case actually | 23:48 |
| mordred | jeblair: in your prepared projects list patch ... it says "the projects of any items this item depends on, as well as the projects that appead in job.required-projects" ... are those different sets? - like, does adding a depends-on automatically get that project added even if it's not in required-projects? | 23:49 |
| jeblair | mordred: yes | 23:51 |
| jeblair | maybe that's worth reconsidering -- after all, why include that project if the job doesn't use it? :) | 23:52 |
| jeblair | https://review.openstack.org/498270 and https://review.openstack.org/498513 are also ready to go | 23:53 |
| jeblair | might be nice to get the timeout fix in the same restart | 23:54 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add trigger-readthedocs job https://review.openstack.org/498626 | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!