| SpamapS | ok.. it's something with the way I'm loading yaml | 00:00 |
|---|---|---|
| SpamapS | probably metaclass problems | 00:01 |
| SpamapS | not the c loader itself | 00:01 |
| clarkb | as an aside reading this it looks like fromYaml is not actually fromYaml? | 00:01 |
| clarkb | its operating on things already deserialized so they are native python objects | 00:02 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Move assertReportedStat to test_node to avoid race https://review.openstack.org/451119 | 00:02 |
| clarkb | SpamapS: an ya guessing you just want to set the loader to C loader at line 870 in configloader.py | 00:05 |
| clarkb | and then against for safe_load_yaml | 00:05 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 00:06 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Force os-client-config to use ipv4 https://review.openstack.org/450983 | 00:06 |
| clarkb | SpamapS: I think its breaking in the super call at 96 | 00:08 |
| clarkb | since that expects SafeLoader to be around but its been monkey patched out? | 00:09 |
| clarkb | but now time for yard work | 00:09 |
| *** herlo has quit IRC | 00:41 | |
| *** jamielennox is now known as jamielennox|away | 01:02 | |
| *** jamielennox|away is now known as jamielennox | 01:16 | |
| *** nibalizer has quit IRC | 03:15 | |
| *** nibalizer has joined #zuul | 03:16 | |
| *** jeblair has quit IRC | 03:24 | |
| *** jeblair has joined #zuul | 03:25 | |
| *** gundalow_ has joined #zuul | 05:07 | |
| *** gundalow has quit IRC | 05:07 | |
| *** jamielennox is now known as jamielennox|away | 06:10 | |
| *** isaacb has joined #zuul | 06:24 | |
| *** jamielennox|away is now known as jamielennox | 06:25 | |
| *** gundalow_ is now known as gundalow | 06:30 | |
| *** jamielennox is now known as jamielennox|away | 06:50 | |
| *** jamielennox|away is now known as jamielennox | 06:57 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Fix test_leaked_node_not_deleted for v3 https://review.openstack.org/449375 | 07:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: I948f9f69ad911778fabb1c498aebd23acce8c89c https://review.openstack.org/451214 | 07:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Fix test_leaked_node_not_deleted for v3 https://review.openstack.org/449375 | 07:11 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Merge branch 'master' into feature/zuulv3 https://review.openstack.org/445325 | 07:12 |
| *** hashar has joined #zuul | 07:41 | |
| *** jamielennox is now known as jamielennox|away | 08:17 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Use libyaml if possible https://review.openstack.org/451113 | 08:25 |
| SpamapS | clarkb: FYI, the problem was actually that the C loader variants produce a different "Mark" class. | 08:26 |
| *** rcarrillocruz has joined #zuul | 08:52 | |
| tobiash_ | is there a reason behind that on zuul tests the expected and to check values are exchanged for self.assertEqual? | 11:28 |
| tobiash_ | signature of assertEqual is assertEqual(expected, observed,...) | 11:28 |
| tobiash_ | usage is e.g. assertEqual(A.reported, 0) | 11:29 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Fix test_leaked_node_not_deleted for v3 https://review.openstack.org/449375 | 11:33 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Merge branch 'master' into feature/zuulv3 https://review.openstack.org/445325 | 11:33 |
| pabelanger | Shrews: jeblair: mordred: https://review.openstack.org/#/q/topic:nodepool-dsvm gets our dsvm jobs voting again for nodepool. | 11:37 |
| Shrews | pabelanger: 451119 doesn't make sense to me. | 12:06 |
| Shrews | the nodes in test_node_assignment have to be online for the request to be fulfilled | 12:07 |
| pabelanger | Shrews: the difference I saw, is we didn't call self.waitForNodes() which seems to be the way we ensure nodes are online for tests | 12:09 |
| Shrews | waitForNodeRequest() and then validating it is FULFILLED does the same thing (in a round about way). | 12:10 |
| Shrews | because it can't be fulfilled until ready nodes have been assigned to the request | 12:11 |
| Shrews | the 'for node_id in req.nodes:' loop validates that | 12:11 |
| Shrews | pabelanger: do you have an example failure I could look at? | 12:12 |
| pabelanger | from what I seen, I don't think updateNodeStats() was getting called | 12:12 |
| pabelanger | sure | 12:12 |
| pabelanger | 1 sec | 12:12 |
| pabelanger | http://logs.openstack.org/83/450983/2/check/nodepool-coverage-ubuntu-xenial/2aceb27/console.html | 12:13 |
| pabelanger | maybe stopping to fast, before stats are added to statsd? | 12:18 |
| Shrews | hrm, gonna need more coffee for this one | 12:24 |
| Shrews | pabelanger: ok, there are a couple of bugs here i think. one is in the stats fixture, the other in the actual test | 12:42 |
| pabelanger | Shrews: oh, cool. Yay for bug finding | 12:47 |
| Shrews | i think i found one while sleeping last night. have yet to look at that one :) | 12:47 |
| Shrews | pabelanger: so... ok, no bug in the statsd fixture (just working in a weird way), but that assert causing the failure is not always true in that one test because 2 nodes are actually being created. | 12:55 |
| Shrews | pabelanger: if both nodes are ready when updateNodeStats is first called, it will never record a 1 value for nodepool.nodes.ready, but a 2. so the test will timeout and report that misleading error | 12:56 |
| Shrews | pabelanger: so i think your fix is actually ok, but the commit message is wrong | 12:57 |
| pabelanger | Shrews: okay, let me see if I can capture the issue. | 12:59 |
| Shrews | pabelanger: alternatively, you can leave the asserts where they're at and change the '1' to a '2' | 13:05 |
| Shrews | but i wonder if even that might be somewhat racey depending on when the delete thread runs | 13:06 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Fix test_leaked_node_not_deleted for v3 https://review.openstack.org/449375 | 13:08 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Merge branch 'master' into feature/zuulv3 https://review.openstack.org/445325 | 13:08 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Fix test_leaked_node_not_deleted for v3 https://review.openstack.org/449375 | 13:09 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Merge branch 'master' into feature/zuulv3 https://review.openstack.org/445325 | 13:09 |
| Shrews | pabelanger: oh! the _better_ fix for this is to put waitForNodes before the request creation. the only reason 2 node would be created if the 1st wasnt' ready when the request is handled | 13:11 |
| Shrews | (i think my coffee is starting to kick in \o/) | 13:13 |
| pabelanger | Shrews: okay, let me do that | 13:13 |
| pabelanger | and test | 13:13 |
| Shrews | i confirmed that eliminates the problem for me locally | 13:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Add waitForNodes() for test_node_assignment https://review.openstack.org/451119 | 13:21 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 13:21 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Force os-client-config to use ipv4 https://review.openstack.org/450983 | 13:21 |
| Shrews | pabelanger: yeah, that should do it. though pep8 may not like you using 'nodes' somewhere. maybe assertEqual(len(nodes), 1) after? | 13:22 |
| Shrews | not* using 'nodes' | 13:23 |
| pabelanger | Shrews: right | 13:26 |
| pabelanger | copy pasta | 13:26 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Add waitForNodes() for test_node_assignment https://review.openstack.org/451119 | 13:29 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 13:29 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Force os-client-config to use ipv4 https://review.openstack.org/450983 | 13:29 |
| *** dmsimard is now known as dmsimard|afk | 13:39 | |
| *** dkranz has joined #zuul | 14:13 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use unicode for change number extraction https://review.openstack.org/450704 | 14:15 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Move dependency cycle detection into pipelines https://review.openstack.org/451423 | 14:15 |
| pabelanger | Odd, we are doing ipv4 again | 14:21 |
| pabelanger | without force_ipv4 patch | 14:21 |
| pabelanger | must depend on the cloud we running the job on | 14:21 |
| jeblair | jlk: the semaphore isn't part of the secrets series, but conflicts with it, so it's stacked on top of it. | 14:44 |
| jeblair | tobiash_: the reason they are swapped is that i can't remember what order they are supposed to be in. also, in mathematics, "equals" means things on either side of the equality operator are the same -- it never made sense to me that one side would be more equal than the other. at any rate, feel free to use it correctly in new code, or fix existing usage if you are already modifying a test. but please don't write a patch that only fixes them; ... | 14:47 |
| jeblair | ... at this point, that's probably not worth your time to write or our time to review. :) | 14:47 |
| mordred | jeblair: I can never remember their order either ... | 14:48 |
| jeblair | mordred: also, i always make symlinks backwards, then rm them and make them the other way. | 14:50 |
| mordred | jeblair: me too | 14:50 |
| mordred | jeblair: no matter how hard I think about it | 14:50 |
| jeblair | mordred: the harder you think about it, the more backwards they end up? | 14:51 |
| mordred | yup | 14:55 |
| *** isaacb has quit IRC | 14:56 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Fetch list of AZs from nova if it's not configured https://review.openstack.org/450345 | 15:04 |
| mordred | Shrews: okie- I think that is good now ^^ | 15:04 |
| *** dmsimard|afk is now known as dmsimard | 15:11 | |
| *** isaacb has joined #zuul | 15:13 | |
| *** herlo has joined #zuul | 15:39 | |
| eggshell | anyone in here have opinions on https://storyboard.openstack.org/#!/story/2000878 ? | 15:49 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 15:50 |
| mordred | eggshell: I'm sure we all have opinions - but they may or may not be _good_ opinions | 15:52 |
| jeblair | mordred: left comments on 450345 | 15:52 |
| mordred | oh - that one | 15:52 |
| mordred | jeblair: I'm excited about that | 15:52 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Add ability to select flavor by name or id https://review.openstack.org/449784 | 15:53 |
| mordred | jeblair: btw - I tried that voluptuous thing you showed me - and it does not work for me | 15:53 |
| mordred | jeblair: the nice part is that the test totally blows up - so that's good | 15:53 |
| jeblair | mordred: bummer | 15:53 |
| mordred | jeblair: I mean - I was not 100% sure how to apply your example into the nodepool/config structure though | 15:53 |
| jeblair | mordred: we can always fall back to just doing an if statement after schema validation. | 15:54 |
| mordred | so it's 100% possible I just made a silly error | 15:54 |
| jeblair | eggshell: my first thoughts are "let's not go down the docker container path" | 15:54 |
| mordred | jeblair: (since your example was with nested schemas, but in config.py we have nested dicts that we pass to a single schema - so I took my best shot | 15:54 |
| mordred | ++ | 15:54 |
| jeblair | mordred: i can take a look at it in a minute and see if there's something that jumps out at me | 15:54 |
| * mordred likes the multi-node integration test option | 15:55 | |
| eggshell | jeblair: thought that might be the case, and agree. | 15:55 |
| mordred | since that gets us both testing of the action plugins _and_ testing of multi-node jobs in a real way | 15:55 |
| jeblair | eggshell, mordred: yeah, i think multi-node sounds good. we don't have a way of attaching a static node to zuul right now though... | 15:55 |
| mordred | jeblair: do we need a static node? | 15:55 |
| jeblair | well, we need a node | 15:55 |
| jeblair | and i'm not super keen on having this depend on devstack | 15:56 |
| mordred | golly no | 15:56 |
| mordred | oh | 15:57 |
| mordred | ok - I was asking the questions in more detail and as a result of typing I now understand the issue | 15:57 |
| mordred | jeblair: maybe we should just made test-action-plugins blocked on support-static-node ? | 15:58 |
| clarkb | could you preseed zookeeper with a record that says node (self on external IP) belongs to test box? | 15:59 |
| jeblair | part of me thinks "maybe we can do this as a functional test and put 127.0.0.1 (maybe aliased) in the inventory" but i think that's sort of "begging the question" as far as testing goes. since this is really designed to make sure we don't have any holes there, and that would greatly alter the very thing we're trying to test, yeah? | 15:59 |
| jeblair | which is why the "static node" idea is there, right? | 15:59 |
| mordred | yah | 15:59 |
| jeblair | clarkb: i think that could work. it'd be a small throwaway utility. i guess the question is: is it worth it to write that now, or defer till we have nodepool static nodes as mordred suggests? | 16:00 |
| eggshell | I was under the impression that testing the plugins locally was a no-go. | 16:00 |
| mordred | yah - testing them locally is a no-go | 16:00 |
| jeblair | eggshell: yeah, i think that's the case. i'm mostly just refreshing my memory as to why. :) | 16:01 |
| eggshell | jeblair: cool, just wanna make sure I'm on the same page. | 16:01 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Set socket timeout for SSH keyscan https://review.openstack.org/451470 | 16:02 |
| jeblair | clarkb: similarly -- we could run these in a zuul functional test environment with a modified version of zuul's fake nodepool which returns the static node. | 16:03 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Fetch list of AZs from nova if it's not configured https://review.openstack.org/450345 | 16:04 |
| jeblair | eggshell: so i think we've got 3 options for the multi-node job approach: 1) defer until zuul and/or nodepool support static nodes. 2) run with nodepool in fake mode but pre-seed the zookeeper data structure with the static node's info. 3) use zuul's fake nodepool and have it return the static node. | 16:08 |
| pabelanger | jeblair: clarkb: do you mind adding https://review.openstack.org/#/c/449938 to your review queue? I'd like to add debian images to our dsvm-nodepool jobs, currently experimental for glean / diskimage-buidler. To increase coverage. Also need to help debug glean on debian failures I am working on | 16:10 |
| *** isaacb has quit IRC | 16:12 | |
| *** dmsimard is now known as dmsimard|afk | 16:13 | |
| *** hashar has quit IRC | 16:18 | |
| eggshell | jeblair: 3) seems like the best approach today imo. Not sure what the time cost is on waiting for 1) | 16:20 |
| jeblair | pabelanger: done | 16:20 |
| pabelanger | jeblair: ty | 16:20 |
| jeblair | eggshell: i think static node support is not a very high priority task right now, so i agree, 3 is probably simplest. | 16:21 |
| SpamapS | tobiash_: unfortunately, the testtools authors messed that one up, and they're reversed from unittest2 | 16:23 |
| SpamapS | I wonder if we should see if we can create a testtools2 that fixes that. | 16:23 |
| jlk | good morning you grand Zuuleans! | 16:24 |
| jlk | Zuulmans? | 16:24 |
| SpamapS | Zuulites | 16:24 |
| jlk | Zuulanders? | 16:24 |
| SpamapS | Zuulanians! | 16:25 |
| jlk | Zuuliacs | 16:25 |
| SpamapS | clarkb: So I believe the yaml changes have an overall positive effect on test suite runtime. On my machine it went from 9 minutes to 8 minutes (sample size 2 without yaml change, 2 with) | 16:31 |
| *** rcarrill1 has joined #zuul | 16:35 | |
| *** rcarrillocruz has quit IRC | 16:36 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add canonical hostname to source object https://review.openstack.org/451110 | 16:39 |
| mordred | SpamapS: that's about a 12.5% impact! | 16:54 |
| SpamapS | mordred: we parse yaml a lot. | 16:56 |
| mordred | SpamapS: bah | 16:56 |
| SpamapS | It's pretty annoying that PyYAML doesn't a) just do this. and b) use the same yaml.error.Mark for marks in the C versions. | 16:56 |
| mordred | SpamapS: yah | 16:56 |
| SpamapS | I want to poke at rtyaml to see if maybe it does either of those things | 16:56 |
| SpamapS | if not.. I feel a pypi library coming on. | 16:56 |
| SpamapS | fastyaml? | 16:57 |
| mordred | I think it does - I like pretty much everything I read about rtyaml - except for his usage of ~ instead of null | 16:57 |
| SpamapS | mordred: yeah that is bonghits | 16:57 |
| mordred | like - it also uses OrderedDict by default | 16:57 |
| SpamapS | ~ is totally not more obvious as null than... null. | 16:57 |
| mordred | but - honestly, if it does all the other things well - I can learn to grok ~ as null | 16:57 |
| SpamapS | well also we don't dump yaml all that much | 16:58 |
| mordred | SpamapS: oh - although - it seems like rtyaml might have a hard-depend on libyaml - rather than trying and faling back gracefully | 16:59 |
| mordred | " | 16:59 |
| mordred | Writes unicode strings without any weird YAML tag. They just appear as strings. Output is UTF-8 encoded, and non-ASCII characters appear as Unicode without escaping." | 16:59 |
| mordred | is one of my favorite defaults he does - if we make a fastyaml - I think we should also do that :) | 16:59 |
| mordred | and "Writes multi-line strings in block mode rather than quoted with embedded “n“‘s, choosing between the literal or folded mode depending on what looks better for the length of the lines in the string." | 17:00 |
| SpamapS | Yeah | 17:00 |
| mordred | SpamapS: k - no. it does to graceful fallback - although it doesn't seem to do anything with Mark | 17:01 |
| SpamapS | another idea | 17:01 |
| SpamapS | can we talk to PyYAML? | 17:01 |
| SpamapS | mordred: yeah not doing things with Mark is pretty unsurprising.. that was a very complicated issue and it's really a shame PyYAML does it the way they've done it. | 17:01 |
| SpamapS | My guess is that any time a groundswell of activity happens around this, people realize they shouldn't be parsing yaml in any performance critical path and switch to msgpack or something | 17:02 |
| mordred | yah. this is also I think at the root of why the pip folks decided to use that thing that isn't yaml but that i do not mention | 17:02 |
| mordred | SpamapS: I imagine talking to pyyaml would be weird since our suggestion is "could you please change your default behavior" | 17:03 |
| jlk | "Have you tried just NOT being slow?" | 17:04 |
| mordred | SpamapS: we could probably send rtyaml a PR adding Mark support | 17:04 |
| SpamapS | mordred: the public domain grant is problematic | 17:05 |
| mordred | is it? whyso? | 17:05 |
| SpamapS | Oh, no it's not.. it's actually the proper CC0 | 17:06 |
| SpamapS | on the pypi page it just says public domain grant | 17:06 |
| SpamapS | but CC0 is actually something i can work with ;) | 17:06 |
| mordred | ++ | 17:06 |
| mordred | SpamapS: yah - I _think_ you'd just need to make a 3-line PR to rtyaml | 17:06 |
| * SpamapS forks | 17:07 | |
| mordred | SpamapS: oh - it's public domain because it's part of a project "A shared commons of data and tools for the United States. Made by the public, used by the public. | 17:07 |
| mordred | " | 17:07 |
| mordred | with a bunch of repos with info about congress and the states and whatnot | 17:07 |
| SpamapS | yeah that's pretty amazing | 17:07 |
| SpamapS | had no idea that existed | 17:08 |
| SpamapS | mordred: https://github.com/unitedstates/rtyaml/pull/1 <-- that 3 line patch? | 17:15 |
| jeblair | SpamapS: your pr, um, has a reply... | 17:22 |
| jeblair | lmgtfy | 17:24 |
| jlk | What's a Mark? | 17:24 |
| jlk | Can we get somebody named Mark to reply? | 17:24 |
| SpamapS | jeblair: RIGHT? | 17:24 |
| jeblair | can we get *lots* of people named Mark to reply? | 17:24 |
| SpamapS | Perhaps a significant percentage of the Mark's could reply | 17:24 |
| SpamapS | hrm | 17:25 |
| SpamapS | working on a patch to use rtyaml | 17:25 |
| SpamapS | something's not working right | 17:25 |
| * SpamapS notes there's not much hair left on this yak | 17:25 | |
| jlk | Mark's not pulling his weight. | 17:25 |
| jlk | GET WITH THE PROGRAM, MARK! | 17:25 |
| jeblair | now i know what "Great question!" means when SpamapS says it | 17:28 |
| pabelanger | we are adding a dependency on rtyaml? | 17:28 |
| SpamapS | jeblair: }:) | 17:28 |
| SpamapS | pabelanger: I'm just playing with it | 17:28 |
| pabelanger | k, that was mostly a packaging question | 17:29 |
| SpamapS | Its a 154 line python module with a CC0 license. Pretty sure we can get that packaged. ;) | 17:31 |
| SpamapS | heck, I'd be willing to vendor it | 17:32 |
| SpamapS | hm, OrderedDict seems to be confusing configloader | 17:32 |
| mordred | fascinating | 17:32 |
| mordred | jeblair: reviewed the secrets stack - it all looks great - there's one set of things on patch 2 - then a few different nits that folks have pointed out on up the stack on +2'd patches (so scanning for commnents even though the stack is mostly green is worth doing) | 17:34 |
| jeblair | mordred: cool thanks! i just replied to comments on the first patch and am revising it now. | 17:36 |
| mordred | jeblair: oh - actually - I made a comment on like, patch 6 or 7 that might be more relevant to patch 1 while you're on it | 17:39 |
| mordred | jeblair: do we need to add libssl or libssl-dev or anything like that to bindep? | 17:40 |
| clarkb | mordred: its probably already there for paramiko? | 17:40 |
| clarkb | which ansible pulls in | 17:40 |
| mordred | paramiko does not declare bindep depends | 17:41 |
| jeblair | mordred: https://cryptography.io/en/latest/installation/#building-cryptography-on-linux ? | 17:42 |
| clarkb | mordred: no but wouldn't zuul need to have a bindep entry already bceause ansible? | 17:42 |
| mordred | clarkb: we might just be getting lucky? also - we don't use the paramiko codepaths in ansible | 17:43 |
| mordred | jeblair: yah - so that says to me that we should put libssl-dev libffi-dev and python-dev in bindep.txt | 17:43 |
| clarkb | mordred: right but it will still install them iirc | 17:43 |
| clarkb | I ran into this with ansible in d-g | 17:43 |
| clarkb | we even had to pin paramiko even though we don't use it | 17:44 |
| mordred | sure. I'm just saying that with a depend on cryptography, zuul has a depend on some non-python packages being present, which is what bindep is there to communicate and to ensure are there before python installs happen | 17:44 |
| SpamapS | agggh.. no.. the thing that's getting me is rtyaml doesn't play nice with ZuulSafeLoader's construct_mapping | 17:45 |
| clarkb | mordred: yes, I am agreeing and saying I figured they would already be there because of ansible | 17:45 |
| mordred | clarkb: gotcha. nod | 17:45 |
| clarkb | mordred: because zuul -> ansible -> paramiko -> cryptography | 17:45 |
| SpamapS | YAK DEAD | 17:46 |
| mordred | SpamapS: you get all the day to the skin and decide to just keep the small file in zuul? | 17:46 |
| jeblair | so long Mark | 17:46 |
| SpamapS | Mark failed me | 17:46 |
| SpamapS | as usual | 17:46 |
| mordred | HI MARK | 17:46 |
| SpamapS | there's likely a way to make it work | 17:46 |
| SpamapS | but I don't care enough anymore | 17:47 |
| * SpamapS back to real tasks | 17:47 | |
| mordred | clarkb: do we have libssl-dev and libffi-dev on our wheel build host? so we're not seeing the issue in the gate because we're installing from pre-built wheels? | 17:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add per-repo public and private keys https://review.openstack.org/406382 | 17:47 |
| clarkb | mordred: oh that could very well be | 17:47 |
| jeblair | mordred, clarkb, SpamapS, jhesketh, jlk: ^ that addresses the comments and adds bindep stuffs. | 17:48 |
| jlk | coolios | 17:51 |
| mordred | jeblair: woot! | 17:51 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Mark bindep depends for dpkg and add rpm versions https://review.openstack.org/451499 | 17:52 |
| mordred | jeblair: that's a follow up to add the equiv lines for the RHs | 17:52 |
| jeblair | mordred: oh i thought there was automagic? | 17:52 |
| jeblair | maybe that needs more words... | 17:53 |
| jeblair | i thought bindep did some kind of translation for equivalent package names (like for the -dev / -devel thing)? | 17:54 |
| mordred | jeblair: nope, it does not - it allows you to specify both | 17:55 |
| mordred | jeblair: diskimage-builder has a tool that keeps package name mappings, and the experience of trying to deal with that I believe led to making bindep not try to do the same - instead one just annotates depends as needed | 17:56 |
| clarkb | fwiw I far prefer dibs thing to bindep | 17:56 |
| clarkb | but bindep is more straightforward for the simple case | 17:57 |
| mordred | I have actually grown to prefer bindep's approach - although I preferred the dib approach originally | 17:57 |
| clarkb | I like how the dib thing is a tree | 17:57 |
| mordred | I like that the file can be read by humans from any of the distros and they can understand what it means for them - withouth having to make rh people read ubuntu names or vice-versa | 17:57 |
| clarkb | its easier to understand how the names all fit into the linux galaxy | 17:57 |
| mordred | yah - that part is nice | 17:58 |
| jlk | I really really don't miss being a distro developer | 17:58 |
| clarkb | but ya teh tree is only necessary when you have complicated cases were fedora and centos and ubuntu and debian are all different | 17:59 |
| mordred | clarkb: yup | 18:00 |
| mordred | jlk: and nope | 18:00 |
| mordred | jlk: I love how much time and energy I put into trying to become a debian developer, except I did it at the wrong time or something, because after 2-3 years I still couldn't get signed off on, even though my packages were all _PERFECT_ | 18:01 |
| jlk | oh yeah, doing it as a hobby is the worst. | 18:02 |
| jlk | being paid to do it was also bad, because you can't just walk away when things get stupid shitty | 18:02 |
| clarkb | this is actually why I've stuck with tumbleweed. I file bugs, they get responded to quickly and then fixed | 18:03 |
| mordred | jeblair: don't know if you saw - but there's a unittest failure on patch one - likely want to get it before rebasing the rest of the stack | 18:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Fetch list of AZs from nova if it's not configured https://review.openstack.org/450345 | 18:09 |
| mordred | maybe it'll work this time. running the unittests locally before pushing gives me more confidence in that | 18:10 |
| jeblair | mordred: thanks. okay if i squash your bindep patch into it? | 18:15 |
| mordred | jeblair: please do! | 18:15 |
| mordred | jeblair: (although to be fair, my stackalytics numbers are much lower than usual, so maybe I shouldn't be so quick to give away patches :) ) | 18:15 |
| pabelanger | mordred: clarkb: do you mind +3 on https://review.openstack.org/#/c/449938/ | 18:19 |
| mordred | pabelanger: done | 18:20 |
| pabelanger | danke | 18:20 |
| openstackgerrit | Merged openstack-infra/nodepool master: Add debian-jessie DIB for dsvm testing https://review.openstack.org/449938 | 18:23 |
| pabelanger | mordred: clarkb: do you mind https://review.openstack.org/#/c/451383/ too. I thought it was merged this morning | 18:32 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add secret top-level config object https://review.openstack.org/446156 | 18:38 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add per-repo public and private keys https://review.openstack.org/406382 | 18:38 |
| clarkb | pabelanger: done | 18:44 |
| *** hashar has joined #zuul | 19:02 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Decrypt secrets and plumb to Ansible https://review.openstack.org/446688 | 19:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add 'allow-secrets' pipeline attribute https://review.openstack.org/447138 | 19:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Isolate encryption-related methods https://review.openstack.org/447087 | 19:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Augment references of pkcs1 with oaep https://review.openstack.org/447088 | 19:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Associate secrets with jobs https://review.openstack.org/446687 | 19:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add support for job allowed-projects https://review.openstack.org/447134 | 19:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Serve public keys through webapp https://review.openstack.org/446756 | 19:06 |
| mordred | jeblair: woot | 19:08 |
| mordred | jeblair: (although you have angered the pep8 gods) | 19:08 |
| jeblair | ugh | 19:11 |
| jeblair | i think i'll let those settle out over lunch then do another cleanup pass | 19:11 |
| mordred | jeblair: ++ | 19:14 |
| mordred | jeblair: anger of the gods notwithstanding, the stack seems like it's in great shape | 19:14 |
| SpamapS | jeblair: your favorite sandwich shop, Jersey Mike's, is donating 100% of its proceeds to Operation Gratitude today btw... FYI. | 19:25 |
| jlk | interestig | 19:25 |
| mordred | SpamapS: that makes me fell bad for not having eaten there today | 19:25 |
| mordred | SpamapS: but there is not one within walking distance of my house | 19:25 |
| SpamapS | Yeah I have one a block away. :) | 19:25 |
| jlk | I think we have some of those around here. | 19:25 |
| SpamapS | yeah I've seen one in Seattle | 19:26 |
| SpamapS | in fact I think that's where I learned it's jeblair's favorite sandwich shop ;) | 19:26 |
| mordred | Shrews: yay my nodepool patch actually passes tests!!! | 19:28 |
| Shrews | mordred: right? | 19:30 |
| *** openstackgerrit has quit IRC | 19:33 | |
| *** jamielennox|away is now known as jamielennox | 19:33 | |
| *** openstackgerrit has joined #zuul | 19:44 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Decrypt secrets and plumb to Ansible https://review.openstack.org/446688 | 19:44 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add 'allow-secrets' pipeline attribute https://review.openstack.org/447138 | 19:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Isolate encryption-related methods https://review.openstack.org/447087 | 19:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add secret top-level config object https://review.openstack.org/446156 | 19:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Augment references of pkcs1 with oaep https://review.openstack.org/447088 | 19:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Associate secrets with jobs https://review.openstack.org/446687 | 19:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add support for job allowed-projects https://review.openstack.org/447134 | 19:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add per-repo public and private keys https://review.openstack.org/406382 | 19:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Serve public keys through webapp https://review.openstack.org/446756 | 19:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add canonical hostname to source object https://review.openstack.org/451110 | 19:52 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add canonical hostname to source object https://review.openstack.org/451110 | 19:55 |
| *** dkranz has quit IRC | 19:56 | |
| mordred | jeblair: so - https://review.openstack.org/#/c/449784/ is up and only fails the new test it added - so that's good! | 19:57 |
| jeblair | mordred: yay! | 19:57 |
| jeblair | mordred: what do you think of the interaction with flavor-name and name-filter? | 19:58 |
| jeblair | mordred: they do different things, but it feels weird having two name related things... | 19:58 |
| mordred | jeblair: I dunno - that doesn't bother me too much - do you think we should do something different with them? | 20:01 |
| mordred | jeblair: like, I mean we could just skip the name-filter and have min-ram and name and if min-ram isn't specific enough for you, just use name? | 20:02 |
| mordred | jeblair: or maybe combining them isn't terrible - if only name: name must match. if min-ram and name, we'll first grab matching min-rams and then we'll filter those by if name in flavor.name | 20:04 |
| * mordred is clearly just thinking out loud | 20:04 | |
| jeblair | mordred: well, i think if we did that, we would have to switch to name (since rax has so many flavors). personally, i'd be okay with dropping min-ram and name-filter and only using flavor name (or id). but other folks on the team thought the min-ram approach was good. so i wonder about going fully in the direction of filtering: keep min-ram and name-filter, don't add flavor-name, but instead make min-ram optional. then use each of min-ram ... | 20:05 |
| jeblair | ... and name-filter to reduce the candidate flavor set and bail if it's > 1. | 20:05 |
| jeblair | mordred: sorry, it took me a long time to type that; it refers to your "skip name-filter" suggestion :) | 20:05 |
| jeblair | mordred: i think the thing i just said is a sort of general approach which lets us add filters in the future (disk? network?) without things being too confusing... | 20:06 |
| mordred | jeblair: I guess my concern is that for folks who just want to specify by name, doing that by putting the value ina field called "name-filter" when there are no other criteria feels weird | 20:06 |
| jeblair | mordred: yeah, could just rename it "name" | 20:06 |
| mordred | k. lemme noodle on those real quick ... | 20:07 |
| jeblair | mordred: so then it's "name: Rackspace 8GB Performance Flavor Esquire" if you want the full name. or "name: Performance; ram: 8096" if you want what we do now... | 20:07 |
| mordred | yah | 20:08 |
| jeblair | (i mean, that's hiding complexity, but it's hiding it in a way that is likely to make users accidentally do the "safe" thing and specify the full name) | 20:09 |
| jeblair | mordred: meanwhile, i spotted the error and left a comment | 20:11 |
| mordred | jeblair: wow. | 20:12 |
| jeblair | mordred: (also, your second suggestion is very similar in spirit to my suggestion, i think) | 20:13 |
| mordred | yes - I think we pretty much came to the same thing :) | 20:14 |
| jesusaur | /buffer 38 | 20:35 |
| SpamapS | only 38? amateur. | 20:46 |
| * rcarrill1 excited to see https://review.openstack.org/#/c/406382/ close to land | 20:47 | |
| rcarrill1 | sucks i got sucked on Ansible 2.3 and had to hand it off | 20:48 |
| rcarrill1 | thx a bunch jeblair | 20:48 |
| * rcarrill1 goes back to cisco and network gear madness PRs | 20:49 | |
| *** rcarrill1 is now known as rcarrillocruz | 20:49 | |
| *** hashar has quit IRC | 21:27 | |
| jesusaur | SpamapS: that could just as easily have been a /buffer 83; but anything over 100 triggers my channel garbage collector | 21:44 |
| jhesketh | Morning | 21:44 |
| *** jamielennox is now known as jamielennox|away | 22:09 | |
| *** jamielennox|away is now known as jamielennox | 22:12 | |
| eggshell | SpamapS: you around? | 22:27 |
| SpamapS | eggshell: back now. Wassup? | 23:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add source to projects https://review.openstack.org/451596 | 23:03 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add a project index to Tenant https://review.openstack.org/451597 | 23:03 |
| jeblair | jhesketh, mordred, SpamapS, jkl, fungi: i have either respended to or addressed all of the comments on the secrets stack (thanks!), and it is passing tests, so is ready for re-review. | 23:06 |
| jhesketh | jeblair: neat. Will do so today :-) | 23:06 |
| fungi | thanks, it's (finally!) on my agenda for this evening | 23:07 |
| eggshell | SpamapS: trying to get some info on how zuul creates a fake nodepool during testing. | 23:15 |
| SpamapS | jeblair: ty, will dive back in | 23:16 |
| SpamapS | eggshell: hm | 23:20 |
| SpamapS | eggshell: did you find tests/base.py:FakeNodepool yet? | 23:22 |
| * eggshell looks | 23:24 | |
| eggshell | facepalm. wasn't on feature/zuulv3 | 23:25 |
| eggshell | alright, I should be able to read and play around some. thanks | 23:25 |
| SpamapS | eggshell: I did that last week.. cost me a good 4 hours | 23:30 |
| SpamapS | master is just muscle-memoried in | 23:30 |
| SpamapS | have to start calling master branches random nouns to break that habit | 23:31 |
| clarkb | SpamapS: I found https://blog.jessfraz.com/post/getting-towards-real-sandbox-containers/ re sandbox containers | 23:34 |
| clarkb | and it points at bubblewrap | 23:34 |
| clarkb | so there is that | 23:34 |
| SpamapS | Indeed, I'm sold on bubblewrap | 23:35 |
| SpamapS | the USER_NS stuff... not as much.. but the approach I like. | 23:35 |
| SpamapS | just that USER_NS is really new | 23:35 |
| clarkb | but there is also subgraph | 23:35 |
| SpamapS | subgraph I had not looked at | 23:37 |
| SpamapS | But their front page graphic is _amazing_ | 23:37 |
| clarkb | but another point (right at the end) is that at least 11 months ago the author did not feel you could get a proper sandbox | 23:38 |
| SpamapS | I just realized the security spec I wrote doesn't even talk about Seccomp | 23:39 |
| SpamapS | but bubblewrap facilitates it, and we should probably use it. | 23:39 |
| SpamapS | clarkb: right, I'm still of the mind to use bubblewrap via setuid | 23:42 |
| SpamapS | otherwise we'll have to have a special kernel on kexecutors | 23:42 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!