| jeblair | mordred, pabelanger, clarkb, SpamapS: ^ can you take a look at that change? I'm thinking ahead to our afs publishing and that would sure be shiny. | 00:00 |
|---|---|---|
| jeblair | i have verified that we don't need all of proc -- we can actually rw-bind just the ioctl node; however bubblewrap has special support for proc which they deem safe and secure, so maybe that's okay? | 00:02 |
| * clarkb would have to think about that a bit | 00:08 | |
| *** xinliang has quit IRC | 02:43 | |
| *** xinliang has joined #zuul | 02:55 | |
| *** bhavik1 has joined #zuul | 06:51 | |
| *** bhavik1 has quit IRC | 09:37 | |
| *** openstackgerrit has quit IRC | 10:02 | |
| *** SpamapS has quit IRC | 11:41 | |
| *** amoralej has quit IRC | 11:41 | |
| *** mmedvede has quit IRC | 11:41 | |
| *** pbrobinson has quit IRC | 11:41 | |
| *** EmilienM has quit IRC | 11:41 | |
| *** jtanner has quit IRC | 11:41 | |
| *** TheJulia has quit IRC | 11:41 | |
| *** robcresswell has quit IRC | 11:41 | |
| *** patrickeast has quit IRC | 11:41 | |
| *** zaro has quit IRC | 11:41 | |
| *** mgagne has quit IRC | 11:41 | |
| *** fungi has quit IRC | 11:41 | |
| *** kklimonda has quit IRC | 11:41 | |
| *** smyers has quit IRC | 11:41 | |
| *** ChanServ has quit IRC | 11:41 | |
| *** lennyb has quit IRC | 11:41 | |
| *** yolanda has quit IRC | 11:41 | |
| *** dmsimard has quit IRC | 11:41 | |
| *** pabelanger has quit IRC | 11:41 | |
| *** leifmadsen has quit IRC | 11:41 | |
| *** rcarrillocruz has quit IRC | 11:41 | |
| *** kmalloc has quit IRC | 11:41 | |
| *** toabctl has quit IRC | 11:41 | |
| *** jamielennox has quit IRC | 11:41 | |
| *** robled has quit IRC | 11:41 | |
| *** maxamillion has quit IRC | 11:41 | |
| *** pleia2 has quit IRC | 11:41 | |
| *** Shrews has quit IRC | 11:41 | |
| *** adam_g has quit IRC | 11:41 | |
| *** clarkb has quit IRC | 11:41 | |
| *** xinliang has quit IRC | 11:41 | |
| *** eventingmonkey has quit IRC | 11:41 | |
| *** jeblair has quit IRC | 11:41 | |
| *** dkranz has quit IRC | 11:41 | |
| *** tobiash has quit IRC | 11:41 | |
| *** rfolco has quit IRC | 11:41 | |
| *** mnaser has quit IRC | 11:41 | |
| *** zigo has quit IRC | 11:41 | |
| *** _ari_ has quit IRC | 11:41 | |
| *** Diabelko has quit IRC | 11:41 | |
| *** ianw has quit IRC | 11:41 | |
| *** jlk has quit IRC | 11:41 | |
| *** timrc has quit IRC | 11:41 | |
| *** fbouliane has quit IRC | 11:41 | |
| *** jkilpatr has quit IRC | 11:41 | |
| *** mattclay has quit IRC | 11:41 | |
| *** mordred has quit IRC | 11:41 | |
| *** persia has quit IRC | 11:41 | |
| *** bstinson has quit IRC | 11:41 | |
| *** harlowja has quit IRC | 11:41 | |
| *** olaph has quit IRC | 11:41 | |
| *** SotK has quit IRC | 11:41 | |
| *** tflink has quit IRC | 11:41 | |
| *** tristanC has quit IRC | 11:41 | |
| *** cinerama has quit IRC | 11:41 | |
| *** jesusaur has quit IRC | 11:41 | |
| *** weshay has quit IRC | 11:41 | |
| *** jhesketh has quit IRC | 11:41 | |
| *** gothicmindfood has quit IRC | 11:41 | |
| *** openstack has joined #zuul | 12:59 | |
| pabelanger | morning | 13:01 |
| dmsimard | pabelanger: o/ | 13:31 |
| dmsimard | clarkb, jlk: probably have a solution for the offset in the network overlay config.. testing something. | 13:42 |
| dmsimard | clarkb, jlk: probably have a solution for the offset in the network overlay config.. testing something. | 13:47 |
| dmsimard | oops, wrong window | 13:47 |
| dmsimard | yup, works -- sending a patch | 13:50 |
| mordred | morning pabelanger ! | 14:17 |
| mordred | pabelanger: so - gpg key - I just pulled the pubring and secring values out of hiera | 14:18 |
| mordred | pabelanger: it's possible I did not pull them out properly | 14:18 |
| pabelanger | mordred: okay, that's what I thought. I am wondering if binary data in ansible vars is also an issue | 14:19 |
| pabelanger | mordred: let me reencrypt and see if they match | 14:19 |
| jeblair | pabelanger: the cyphertext will be different even if you re-encrypt the same values. | 14:28 |
| pabelanger | jeblair: okay, thanks | 14:30 |
| pabelanger | Do we have a proceedure to manually decrypt secrets? | 14:31 |
| jeblair | pabelanger: no. and we now can't use keep to get them anymore either. | 14:34 |
| jeblair | pabelanger: doing some searching, it looks like ansible doesn't really support binary data in variables | 14:35 |
| pabelanger | jeblair: Ya, that is what I am seeing also | 14:35 |
| jeblair | pabelanger: so we may need to go with clarkb's suggestion of exporting ascii-armored versions of the keys, and then import them. | 14:36 |
| mordred | nod. so we really might need to do export armor / import | 14:36 |
| mordred | yah | 14:36 |
| pabelanger | agree to export armor / import | 14:36 |
| jeblair | we can also look at implementing our own support for this with a custom module, but let's do that later. | 14:36 |
| mordred | ++ | 14:37 |
| pabelanger | okay, let me work on that now | 14:37 |
| leifmadsen | o/ | 14:54 |
| *** openstackgerrit has joined #zuul | 14:54 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Use gpg import for sign-artifacts tasks https://review.openstack.org/497953 | 14:54 |
| pabelanger | mordred: jeblair: do you mind a review of import commands for gpg^ I haven't tested just yet but think that is the proper process for importing public / private keys | 14:56 |
| mordred | pabelanger: that looks right to me | 14:57 |
| leifmadsen | mordred: jeblair: still on? | 15:03 |
| *** amoralej is now known as amoralej|off | 15:03 | |
| *** amoralej|off has quit IRC | 15:06 | |
| mordred | leifmadsen, jeblair: technology seems to be defeating me this morning | 15:11 |
| leifmadsen | sadness | 15:12 |
| dmsimard | pabelanger: wth.. can you see why https://review.openstack.org/#/c/496935/3..4/playbooks/roles/fix_disk_layout/tasks/main.yaml can yield a syntax error !? PS#3 worked, PS#4 fails with http://paste.openstack.org/raw/619438/ | 15:13 |
| dmsimard | pabelanger: there doesn't seem to be any garbage whitespaces (which tend to cause this) as far as I can tell.. | 15:14 |
| dmsimard | hrm, there's this http://logs.openstack.org/35/496935/4/check/gate-dg-hooks-dsvm/d93528a/logs/devstack-gate-setup-host.txt which goes "ERROR! failed at splitting arguments, either an unbalanced jinja2 block or quotes: set -ex" | 15:16 |
| dmsimard | weird. | 15:16 |
| dmsimard | ohhhhhhh, it's probably confusing jinja because set is a jinja declaration for setting a var o_O | 15:16 |
| pabelanger | dmsimard: most of the {} braces shouldn't be needed in that shell logic | 15:20 |
| pabelanger | maybe remove and see if jinja is happier | 15:20 |
| pabelanger | otherwise, you likely need to escape them | 15:20 |
| dmsimard | pabelanger: right, it's verbatim from functions.sh -- and it worked in PS3 too. I don't think jinja should be interpreting these unless it's double mustaches | 15:20 |
| dmsimard | I'm testing my "set" theory | 15:21 |
| dmsimard | I'm testing my "set" theory | 15:22 |
| dmsimard | wrong window again.. | 15:22 |
| pabelanger | we used set before in shell | 15:23 |
| mordred | pabelanger: zookeeper in centos ... where can leifmadsen learn about what you did? | 15:40 |
| dmsimard | mordred, leifmadsen: perhaps worth pinging software factory folks (like tristanC) too | 15:43 |
| dmsimard | We've got zk in SF2.6, it's already deployed for both review.rdoproject.org and sf.io | 15:44 |
| leifmadsen | cool, yea just trying to find a repository that would have it for CentOS 7.3 | 15:44 |
| pabelanger | mordred: there is a zookeeper-lite packages that SF forked. But we need to work on getting that into EPEL | 15:44 |
| pabelanger | otherwise, I just use fedora | 15:44 |
| pabelanger | that's what I've been running locally | 15:45 |
| dmsimard | leifmadsen: "yum install --nogpgcheck https://softwarefactory-project.io/repos/sf-release-2.6.rpm" will get you the repositories used in SF (there's a bunch in there), what you want is really sfrelease-2.6 afaik | 15:48 |
| dmsimard | the spec is here https://github.com/softwarefactory-project/zookeeper-lite-distgit | 15:48 |
| leifmadsen | cool thanks, I'll repackage it from the spec then | 15:49 |
| leifmadsen | don't want a messy repository with all the SF things | 15:49 |
| dmsimard | leifmadsen: and role here https://github.com/softwarefactory-project/sf-config/tree/master/ansible/roles/sf-zookeeper | 15:49 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Add zookeeper-server to bindep for Red Hat https://review.openstack.org/497962 | 15:49 |
| clarkb | dmsimard: left a question on the overlay change | 15:50 |
| pabelanger | https://bugzilla.redhat.com/show_bug.cgi?id=1280159 is the right fix for zookeeper in epel | 15:50 |
| openstack | bugzilla.redhat.com bug 1280159 in zookeeper "EPEL7 branch for zookeeper" [Unspecified,New] - Assigned to ctubbsii | 15:50 |
| clarkb | dmsimard: little fuzzy on how the offset stored in a file should be working | 15:50 |
| pabelanger | but there is a large dependency change of java | 15:50 |
| pabelanger | it should be in centos 8 however | 15:51 |
| dmsimard | clarkb: you probably missed the part where the increment is delegated to primary | 15:51 |
| clarkb | leifmadsen: pabelanger I just run it out of the upstream tarball on tumbleweed, works fine | 15:52 |
| clarkb | dmsimard: ah yup, the whole block is delegated to the primary | 15:52 |
| clarkb | dmsimard: maybe put the delegate to under the - block: so its clear when reading top to bottom? but that is a minor nit now that I see it | 15:53 |
| pabelanger | clarkb: leifmadsen: ya, that is an option too. I've been meaning to update zookeeper role to support that | 15:53 |
| dmsimard | clarkb: I usually do that, yeah, but if I put the delegate_to block under the when it yields a syntax error, not sure why | 15:53 |
| dmsimard | clarkb: I can try and figure out if I can put it where we want it, maybe it's just me /me looks again | 15:54 |
| dmsimard | clarkb: now that I'm trying it again, I guess the initial offset should be zero ? Because it increments *before* running so the first peer would be at offset 2 instead of 1 (not that it matters that much) | 15:57 |
| clarkb | dmsimard: ya it shouldn't matter too much, but we've done primary = 1, subnode_0 = 2, subnode_1 = 3 and so on | 15:58 |
| clarkb | dmsimard: so I think it is correct because earlier you wrote 1 to the file | 15:59 |
| dmsimard | clarkb: oh, we're fine then yeah | 15:59 |
| clarkb | actually | 15:59 |
| clarkb | offset is/was a parameter too | 15:59 |
| clarkb | so that will need to be fixed (because in some cases we do more than one set of addresses on different overlays | 16:00 |
| dmsimard | clarkb: found the issue with the block.. the "delegate_to" needs to be before the block | 16:01 |
| dmsimard | clarkb: ok, sure, so default offset to 1 but make it a parameter ? | 16:01 |
| clarkb | dmsimard: ya. Also do you need to set fact earlier so that the primary tasks get the offset of 1? looks like we just write it to a file but don't set fact at that point | 16:02 |
| clarkb | dmsimard: https://review.openstack.org/#/c/435933/33/playbooks/roles/ovs_vxlan_bridge_primary/tasks/main.yaml that tasks specifically sues it and not sure if it is set yet | 16:03 |
| dmsimard | clarkb: yeah, you might have figured out from my previous question about the initial offset that I was confused :) | 16:03 |
| * dmsimard didn't think there was an offset for the primary | 16:03 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Add zookeeper-server to bindep for Red Hat https://review.openstack.org/497962 | 16:04 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: zookeeper is always needed for nodepool now https://review.openstack.org/497968 | 16:04 |
| jeblair | SpamapS, jlk, clarkb, fungi: if you have a chance to think about https://review.openstack.org/497698 that would be great | 16:05 |
| SpamapS | jeblair: just sat down, will pop it into gertty now. | 16:09 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Use gpg import for sign-artifacts tasks https://review.openstack.org/497953 | 16:10 |
| dmsimard | pabelanger: omg I found the syntax error from the fix disk layout thing | 16:25 |
| dmsimard | pabelanger: it's an ansible parser issue -- definitely not a bash one | 16:25 |
| dmsimard | pabelanger: https://review.openstack.org/#/c/496935/3..4/playbooks/roles/fix_disk_layout/tasks/main.yaml notice this: [...] there's [...] | 16:26 |
| dmsimard | ansible is trying to match that single quote | 16:26 |
| dmsimard | even though it's in a comment block | 16:26 |
| pabelanger | ya | 16:28 |
| * dmsimard files upstream bug | 16:29 | |
| pabelanger | I mean, the right way to do this would be move it into a script and have ansible call that directly. But, we are just hacking ansible to speed up zuulv3 migrations | 16:30 |
| pabelanger | I'd never want to support a role that way long term | 16:30 |
| dmsimard | pabelanger: of course | 16:31 |
| clarkb | dmsimard: thinking about it more, is it sufficient to just use serial: 1 and not use a temp file at all? | 16:40 |
| clarkb | dmsimard: since we delegate the offset value computing to a single node it should be just as valid in memory or on disk? | 16:41 |
| dmsimard | clarkb: no, because of the problem jlk mentioned -- facts are actually per-host | 16:41 |
| dmsimard | ah, hmm | 16:41 |
| clarkb | ya I think what makes this work is the delegate not necessarily the file | 16:41 |
| dmsimard | let me try it locally | 16:41 |
| clarkb | well that and serial | 16:41 |
| clarkb | I guess it depends if that state crosses tasks | 16:42 |
| dmsimard | clarkb: it doesn't cross /plays/ | 16:43 |
| dmsimard | clarkb: but we only really need to increment during one play so it shouldn't be an issue | 16:44 |
| jlk | so delegate is weird | 16:44 |
| jlk | set fact with delegate may actually be setting the fact of the original host, not of the delegated host | 16:44 |
| SpamapS | gah.. finally alt-tabbing back to gertty 40 minutes later | 16:44 |
| dmsimard | jlk: yeah I thought that too :) | 16:44 |
| jlk | more likely what you should do is set fact on hostvars['host']['fact_name'] ? | 16:45 |
| dmsimard | jlk: but locally it behaved properly | 16:45 |
| jlk | dmsimard: ah okay | 16:45 |
| dmsimard | jlk: might be a bug though, don't get me wrong | 16:45 |
| clarkb | well that may be a reason for the file | 16:45 |
| jlk | it's a subtlety | 16:46 |
| clarkb | it reads the file on the delgated host then sets the fact on the original host | 16:46 |
| dmsimard | hmm, I guess it would be written better without the set_fact inside the delegation -- increment the value inside the copy content and then do a set_fact *outside* the delegation | 16:47 |
| * dmsimard tries that | 16:47 | |
| dmsimard | ohhhhhhh wait | 16:48 |
| jlk | you could also do something fun with an index of the hostname in the hosts dict | 16:48 |
| dmsimard | lookups always occur on the control node | 16:48 |
| jlk | index +1 or some such | 16:48 |
| SpamapS | clarkb: did you want to peek at 497698 before it gets +A'd? | 16:48 |
| SpamapS | I found the commit message fascinating. | 16:48 |
| clarkb | SpamapS: oh ya that was on tap yesterday /me looks | 16:48 |
| SpamapS | (AFS PAG's and /proc fun) | 16:49 |
| dmsimard | clarkb, jlk: Yeah, I'm not able to get it to work with using just set_fact .. only with the file, I'll send a new patchset though | 16:53 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Fix typo in public key import https://review.openstack.org/497983 | 16:53 |
| clarkb | with /proc mounted does that give an zuul process on an executor within bubblewrap access to all of the other different bubblewrapped processes? | 16:54 |
| pabelanger | jlk: SpamapS: jeblair: mordred: ^ can I get a +3 for typo fix | 16:54 |
| * clarkb tries to wrap brain around what /proc access means in a namespaces world | 16:54 | |
| clarkb | manpage isn't helping me much | 16:55 |
| clarkb | unshare-all implies unshare-pid which means the procfs exposed should not incldue info from other containers or the host | 16:58 |
| clarkb | SpamapS: ^ that a correct interpretation? | 16:58 |
| clarkb | the process won't have any special perms with regards to the rest of /proc (eg no root access to mangle the kernel) so that should be fine | 16:59 |
| dmsimard | jlk: hopefully less confusing: https://review.openstack.org/#/c/435933/35/playbooks/ovs_vxlan_bridge.yaml | 17:02 |
| clarkb | jeblair: SpamapS: my last concern is that we pass --uid which is given the value of the parent process uid. bwrap docs say that requires unshare-user implying even if it is the same value as the calling context it is truly in a proper namespace without overlap. Is that the case? | 17:05 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Fix typo in public key import https://review.openstack.org/497983 | 17:05 |
| SpamapS | clarkb: /proc does provide some attack vectors that weren't there before due to exposure of kernel structures exactly like the AFS ioctl target. The pids, however, are entirely in the namespace created by bubblewrap. | 17:11 |
| clarkb | SpamapS: even though every container is created with the same uid arg? | 17:12 |
| SpamapS | Let me re-read the --uid doc and code | 17:12 |
| SpamapS | clarkb: If I'm reading this right, I believe --uid means "the UID to use for the child inside the new user namespace" | 17:16 |
| SpamapS | clarkb: so while they're the same #, they are all in different namespaces. | 17:16 |
| clarkb | gotcha, so not a way to merge namespaces together across containers | 17:17 |
| SpamapS | correct | 17:17 |
| pabelanger | Yay | 17:18 |
| pabelanger | http://tarballs.openstack.org/sandbox/sandbox-0.0.19.tar.gz.asc | 17:18 |
| SpamapS | It's literally just used for uid_map which is documented in 'man 7 user_namespaces' well | 17:19 |
| pabelanger | jeblair: mordred: gpg key signing is done! | 17:20 |
| clarkb | I haev +2'd | 17:20 |
| pabelanger | http://logs.openstack.org/5d/d0ef1611e90eb5d3537d91371c004aa10a35295d/release/release-openstack-python/98ebd7f/job-output.txt.gz for the hotness | 17:20 |
| pabelanger | we actually could optimize sign-artifacts role, I don't think we need the public key, looks like when we import secret it creates the public too | 17:22 |
| clarkb | ya I think ssh keys work that way too | 17:23 |
| clarkb | private includes both, the separate public file is convenience of sharing to places that should never get the private key | 17:23 |
| pabelanger | ya | 17:25 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Use tempfile for ssh private key https://review.openstack.org/497988 | 17:26 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Remove GPG public key for sign-artifacts role https://review.openstack.org/497990 | 17:28 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Include tar.gz.asc / whl.asc if found https://review.openstack.org/497995 | 17:34 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Include tar.gz.asc / whl.asc for twine https://review.openstack.org/497995 | 17:35 |
| leifmadsen | mordred: now my VM is infinitely dirty | 17:38 |
| leifmadsen | re: emacs install :D | 17:38 |
| jlk | dmsimard: so this offset, its based on the order of the hosts in the group? Is there any reason why you couldn't just use the index of the place of the host in the groups['subnodes'] list? | 18:04 |
| dmsimard | jlk: there is a starting offset (ex: 1) on the primary node and then subnodes need a different offset | 18:06 |
| dmsimard | I guess doing 1 + index could work | 18:07 |
| jlk | sure, but if you have that starting offset.. | 18:07 |
| jlk | you just add the offset + 1 + the index location | 18:07 |
| jlk | Should remove the need to persist a file or try to iterate a single variable | 18:07 |
| dmsimard | Yeah, I totally understand what you mean. | 18:07 |
| dmsimard | Let's try that. | 18:08 |
| dmsimard | Hmm, is the order of the list guaranteed you think ? | 18:08 |
| jlk | I think so | 18:08 |
| jlk | inventory parsing is pretty reliable. | 18:09 |
| jlk | if you really wanted to, you could sort the list through a filter | 18:09 |
| dmsimard | jlk: works perfectly -- and no need for serial that way too | 18:20 |
| dmsimard | jlk++ | 18:21 |
| jlk | woo! | 18:22 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: WIP: fix cloud-image error https://review.openstack.org/498050 | 18:24 |
| clarkb | neat | 18:32 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add proc to bubblewrap https://review.openstack.org/497698 | 19:13 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Use tempfile for ssh private key https://review.openstack.org/497988 | 19:14 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Remove GPG public key for sign-artifacts role https://review.openstack.org/497990 | 19:15 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Include tar.gz.asc / whl.asc for twine https://review.openstack.org/497995 | 19:15 |
| dmsimard | mordred, jlk: that last patchset was broken | 19:21 |
| dmsimard | http://logs.openstack.org/33/435933/36/check/gate-grenade-dsvm-neutron-dvr-multinode-ubuntu-xenial-nv/8669bcb/console.html#_2017-08-25_18_41_48_172095 | 19:22 |
| jlk | d'oh | 19:22 |
| dmsimard | should be fixed | 19:22 |
| dmsimard | in the one I just sent | 19:22 |
| jlk | looking again | 19:23 |
| * dmsimard following dvr telnet://67.192.246.42:19885 | 19:25 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Return 404 on unknown tenants https://review.openstack.org/494642 | 19:52 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Remove conditional check for upload-pypi https://review.openstack.org/498075 | 19:53 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Remove conditional check for upload-pypi https://review.openstack.org/498075 | 20:05 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Revert "Include tar.gz.asc / whl.asc for twine" https://review.openstack.org/498085 | 20:33 |
| pabelanger | jeblair: mordred: what should I be focusing on now? Our publishing jobs are ready for PTG | 20:39 |
| jeblair | pabelanger: want to start working on afs publishing jobs? | 20:39 |
| jeblair | docs publishing that is | 20:39 |
| pabelanger | ya, I can do that | 20:39 |
| jeblair | pabelanger: with 497698 merged, after an executor restart, we should have everything we need to do that | 20:40 |
| pabelanger | okay | 20:40 |
| jeblair | pabelanger: you can add the keytab as a secret (that's another binary file; you may need to base64 encode it or something) | 20:40 |
| jeblair | pabelanger: there's a base64 command, so you can have ansible write the base64 data to disk, then "base64 -d infile > outfile.keytab" | 20:41 |
| jeblair | pabelanger: should be able to run 'kinit' and 'aklog' on the executor in a trusted playbook | 20:42 |
| clarkb | dmsimard: I think a couple of the variable names got mixed up, comments inline | 20:42 |
| jeblair | pabelanger: will also need to rw-mount /afs in our executor config file | 20:42 |
| clarkb | er on the overlay change, but otherwise lgtm | 20:42 |
| pabelanger | jeblair: and this is in bwrap right? | 20:43 |
| jeblair | pabelanger: and keep in mind that the afs tokens will only last for a single playbook invocation. | 20:43 |
| jeblair | pabelanger: yep | 20:43 |
| jeblair | pabelanger: so a single playbook will need to kinit, aklog, and copy the data into afs (that's probably what we'd do anyway) | 20:43 |
| clarkb | ansible jinja2 supprots base64 natively too | 20:43 |
| pabelanger | jeblair: understood | 20:43 |
| jeblair | pabelanger: you can look to the zuul v2 playbooks for inspiration since they're doing this already | 20:44 |
| jeblair | clarkb: oh? neato | 20:44 |
| clarkb | so you can do content: {{ secret | base64decode}} | 20:44 |
| jeblair | that's way better | 20:44 |
| clarkb | er b64decode | 20:44 |
| jeblair | assuming that actually ends up as correct binary on disk and not utf8? | 20:44 |
| jeblair | clarkb: cause i thought i read something about jinja treating everything as Strings | 20:45 |
| pabelanger | jeblair: is the reason we don't do this on a node from nodepool because we are missing kernel modules for openafs? | 20:45 |
| jeblair | pabelanger: well, we could install that on a nodepool node, though we'd have to trust it, which we don't after we've just run a docs build. so it's very similar to pypi publishing in that respect. | 20:46 |
| pabelanger | okay, cool. Was just curious | 20:47 |
| pabelanger | I'll start on the playbook here shortly | 20:47 |
| clarkb | jeblair: would be odd to have that if it didn't work but ya may need testing | 20:47 |
| jeblair | pabelanger: in addition to the kinit/aklog stuff, we also need to use the same rsync commands from zuul v2.5 | 20:48 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Revert "Include tar.gz.asc / whl.asc for twine" https://review.openstack.org/498085 | 20:53 |
| jeblair | clarkb, mordred, dmsimard, pabelanger, jlk: i just sent an email to openstack-infra with my current thoughts on devstack-gate | 21:09 |
| pabelanger | k, I'll look shortly | 21:12 |
| clarkb | jeblair: makes sense to me | 21:26 |
| jlk | jeblair: makes sense, but I'm pretty far from the problem to have much useful input :) | 21:30 |
| jeblair | mordred, Shrews: do we include the private ip in the info that nodepool provides? | 21:46 |
| jeblair | ok, yes, it looks like we have interface ip, public v4, public v6, private v4 | 21:48 |
| mordred | jeblair: yah. however, it's worth noting that in the /etc/nodepool data what we write as "private" is private or public | 21:52 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add IP info to nodepool hostvars https://review.openstack.org/498103 | 21:54 |
| jeblair | mordred: indeed -- to simplify shell logic; do you want to change that? | 21:54 |
| mordred | jeblair: I don't have a strong opinion - the "this is the ip to use for intra node communication" is an important thing to communicate | 21:55 |
| mordred | jeblair: so as long as we have something that people can rely on to be the 'best' way to talk between nodes I don't have much opinion on what we call it | 21:56 |
| clarkb | ya private got overloaded for that because it meant "no NAT" | 21:57 |
| mordred | yah | 21:57 |
| mordred | 'non-natted-address' | 21:57 |
| pabelanger | jeblair: ML post seems okay to me. I haven't had a chance to help much on devstack effort however. | 21:59 |
| mordred | jeblair, pabelanger: is there an intentional reason we don't have any roles in project-config? just because so far all of them have been widely applicable even if we need them to run in a trusted context? | 22:00 |
| jeblair | mordred: that's been my general feeling, but i haven't done a full audit | 22:01 |
| pabelanger | mordred: jeblair: ya, we likely need an audit, but so far zuul-jobs has been the default place for roles. I don't mind if we start adding trusted roles to project-config, then promote up into zuul-jobs if needed | 22:04 |
| jeblair | pabelanger: i think if the role is universally useful, it should go in zuul-roles; otherwise project-config if it's only locally useful for that job. | 22:05 |
| jeblair | clarkb, mordred, pabelanger: https://review.openstack.org/498103 is a blocker for devstack-gate work | 22:06 |
| pabelanger | jeblair: ya, I am a little concerned the first time we get a role that is not useful in openstack-infra but for some other zuul user. I could see us growing a large amount of things that might not be well tested. | 22:08 |
| clarkb | jeblair: how is interface IP special? is that what zuul ssh's to? | 22:08 |
| clarkb | jeblair: also I think the trailing comma in there ma be a syntax error | 22:09 |
| jeblair | clarkb: yes; it's what we use for ansible_host. we could drop it from that explicit list since we can access it that way; i just thought since we've got all the nodepool names/info in there | 22:09 |
| clarkb | ya lets include it | 22:09 |
| dmsimard | jeblair: this is going to be a stupid question but is there something ultimately preventing us from getting zuul v3 to run devstack-gate-vm-wrap ? | 22:10 |
| jeblair | clarkb: i think you're right; will fix | 22:10 |
| dmsimard | (just read your email and the challenges aren't immediately obvious to me) | 22:10 |
| pabelanger | jeblair: re: 498103, why couldn't we get the IP address from ansible setup_host task? | 22:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add IP info to nodepool hostvars https://review.openstack.org/498103 | 22:10 |
| clarkb | pabelanger: that only knows about the IPs on the host which isn't necessarily the same as what nodepool shares because NAT | 22:11 |
| jeblair | dmsimard: that's the approach i'm suggesting with devstack-legacy | 22:11 |
| pabelanger | clarkb: okay, that make sense | 22:11 |
| dmsimard | jeblair: ah, okay -- we're on the same wavelength then. I supposed that was always what we would end up going and iterate to further clean up things as we go along. | 22:11 |
| jeblair | dmsimard: however, zuulv3 was basically designed to eliminate almost everything in devstack-gate. as in, we shouldn't need *any* of that. that's what i want to do with the new devstack job. | 22:12 |
| jeblair | dmsimard: groovy | 22:12 |
| dmsimard | jeblair: so, continue in the same direction then ? Or are we further compromising splits in favor of something else ? | 22:13 |
| dmsimard | s/splits/splits and clean ansibilization/ | 22:13 |
| dmsimard | Or perhaps I should ask, what bits are the ones we know want to keep ? I guess the network overlay and fix disk layout are stuff we're keeping | 22:14 |
| dmsimard | All those setup workspace git shenanigans are indeed a bit messy | 22:14 |
| jeblair | dmsimard: right -- fix_disk and overlay are things we want roles for regardless | 22:15 |
| jeblair | dmsimard: so it's useful to keep working on those, and once they exist, we can add (probably a shallow copy of them) to the appropriate v3 places (one is a devstack role, the other should go in a v3 multinode base job) | 22:15 |
| jeblair | dmsimard: so basically i've got 2 irons in the fire: devstack-legacy is here: https://review.openstack.org/497699 and the v3 native devstack is here: https://review.openstack.org/496959 | 22:18 |
| clarkb | dmsimard: ok I think that looks good now. Lets see the test results | 22:23 |
| dmsimard | clarkb: thanks for your help, much appreciated | 22:23 |
| dmsimard | rcarrillocruz: network overlay is probably going to merge soon, thanks :D | 22:24 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add role for adding ssh key to remote nodes https://review.openstack.org/498109 | 22:26 |
| mordred | clarkb, jeblair: patch landed, but I'm +2 on it (the ip info one) - ftr, interface_ip is the shade provided "best" ip to use to connect to the vm - public_v6 if localhost supports v6, otherwise public_v4- or private_v4 if the cloud in question is configured with private: True | 22:29 |
| mordred | generally speaking, assuming connection info is configured properly, interface_ip is what one should use to connect to the server | 22:30 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add IP info to nodepool hostvars https://review.openstack.org/498103 | 22:48 |
| jeblair | i restarted ze01; pabelanger it should have the changes needed for afs | 22:58 |
| pabelanger | jeblair: great, thanks | 22:59 |
| mordred | jeblair: \o/ | 23:01 |
| mordred | dmsimard: remote: https://review.openstack.org/498125 Add environment variable to skip alembic logging | 23:47 |
| mordred | dmsimard: the fileConfig in ara/db/env.py is the thing that's causing us to lose logging of pre-tasks | 23:48 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!