| openstackgerrit | Matthew Treinish proposed openstack-infra/nodepool master: Enabled sending MQTT messages for node events https://review.openstack.org/453922 | 00:08 |
|---|---|---|
| openstackgerrit | Matthew Treinish proposed openstack-infra/nodepool master: Add mqtt module to enable pushing MQTT messages https://review.openstack.org/453915 | 00:08 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/nodepool master: Add support for sending MQTT messages on node events https://review.openstack.org/453922 | 00:18 |
| *** harlowja has quit IRC | 00:39 | |
| *** guoguo has joined #zuul | 00:45 | |
| *** harlowja has joined #zuul | 01:26 | |
| *** harlowja has quit IRC | 01:26 | |
| *** jamielennox is now known as jamielennox|away | 02:05 | |
| *** jamielennox|away is now known as jamielennox | 02:19 | |
| openstackgerrit | Jamie Lennox proposed openstack-infra/nodepool feature/zuulv3: Add username to build and upload information https://review.openstack.org/453968 | 03:43 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/nodepool feature/zuulv3: Show message if node hold not found https://review.openstack.org/453971 | 04:07 |
| jamielennox | jhesketh: do you know if nodepool v3 is supposed to have a wait period between job run and deletion as 2.5 did? | 04:11 |
| jamielennox | jhesketh: a job runs and i can't place a hold on the node because i can't get the lock, but the instant the lock is available it is being deleted | 04:11 |
| jamielennox | (or maybe the lock is never deleted - either way i can hold it) | 04:12 |
| jhesketh | jamielennox: don't know sorry | 04:24 |
| jhesketh | jamielennox: I suspect though that the actual fix would be to not require a lock to put it into hold mode | 04:24 |
| jhesketh | which may require some refactoring or a different zk model | 04:24 |
| jamielennox | yea, i'm not sure, i mean everything locks because it needs to load json, manipulate, store string so i don't think you can just not lock | 04:25 |
| jhesketh | perhaps something nodepool can check to see if anybody wanted to add a hold before deleting etc | 04:25 |
| jhesketh | jamielennox: yeah I don't mean not lock, but inform nodepool somehow else | 04:26 |
| jamielennox | jhesketh: that's ok i can check with others tomorrow - i wasn't sure if it was in your wheelhouse or not, but you're likely the only one here | 04:27 |
| jhesketh | yeah sorry I can't help more | 04:28 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Quote ansible command and include config file https://review.openstack.org/453974 | 04:58 |
| *** guoguo has quit IRC | 05:15 | |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Use username from node information if available https://review.openstack.org/453983 | 05:47 |
| jamielennox | gs | 05:48 |
| jamielennox | nope, not that window | 05:48 |
| *** bhavik1 has joined #zuul | 05:59 | |
| jeblair | jamielennox: https://storyboard.openstack.org/#!/story/2000905 | 06:01 |
| jamielennox | jeblair: an auto-hold would be awesome, but in this case all i need is like 10 seconds between USED and DELETED | 06:03 |
| jamielennox | jeblair: is in the auto-hold that was providing that in 2.5? | 06:03 |
| jamielennox | s/is in/was it | 06:03 |
| jeblair | jamielennox: no, that was just compensation for jenkins delays. | 06:04 |
| jeblair | jamielennox: is the reason that you need those 10 seconds because you want to hold a node after a job fails? | 06:04 |
| jamielennox | jeblair: right, i was running into what turned out to be https://review.openstack.org/#/c/453765/ | 06:04 |
| jamielennox | bad known hosts means ssh connection failures and couldn't connect | 06:05 |
| jamielennox | i was trying to run that job manually to figure out why the failure, but there is no time between the used lock finishing and the watcher deleting it | 06:05 |
| jamielennox | so yea, auto hold would be better, but i was prepped and was up+entering trying to put a hold on to stop the machine being deleted and there is never a point where the lock is free that i can set a hold marker | 06:07 |
| jeblair | jamielennox: yeah, i think the auto-hold feature will handle most of the use cases we have operationally. | 06:07 |
| jeblair | jamielennox: we may want to polish up the manual hold too, but we should probably do that properly in either nodepool or zuul. i think we'd get tired of trying to hit that 10 second window. :) | 06:08 |
| jamielennox | jeblair: oh, absolutely 10 seconds is not a goal :) | 06:09 |
| jamielennox | having given it only a few seconds thought i think you would have AUTO_HOLD as a distinct state from HOLD so that a user could transition a node from AUTO_HOLD->HOLD | 06:09 |
| jamielennox | and that would solve the problem | 06:10 |
| jeblair | jamielennox: (we could consider adding a zuul command to hold a node for a job; if we want to make it work in nodepool, we'll have to deal with the lock -- either get in line for the lock behind zuul and ahead of nodepool, or choose to ignore it, or add a hold flag for a node outside of the node record.) | 06:10 |
| jeblair | jamielennox: i don't think that state transition solves the problem, it's primarily one of locking. zuul owns the node record as long as it's locked, so it's the only thing that can, currently make the hold decision during use. | 06:12 |
| jamielennox | jeblair: well, if zuul set the AUTO_HOLD (or something better named) whilst it still had the lock and then released it would have the same HOLD effect on the nodepool deleter | 06:13 |
| jeblair | jamielennox: zuul should just set the state to hold in that case | 06:13 |
| jeblair | (that's the intent of https://storyboard.openstack.org/#!/story/2000905 ) | 06:13 |
| jamielennox | jeblair: the problem with that is if i have auto-hold = 3 nodepool-deleter can't distinguish between something i as a user want to hold for a long time vs the latest thing that failed in zuul | 06:14 |
| jamielennox | as an admin i need a way to take it out of the delete-soon queue into the hold indefinitely queue | 06:14 |
| jeblair | jamielennox: if auto-hold for job foo is 3 and there are 3 held nodes for job foo, then zuul should not hold any more | 06:15 |
| jeblair | (that's also described in the story) | 06:15 |
| jeblair | there's no delete-soon queue. held nodes are always held indefinitely and need an admin to delete them. | 06:15 |
| jeblair | jamielennox: generally the workflow is: job foo is failing, i need to catch a node it failed on. set auto-hold for foo. come back later and see if we caught anything. delete nodes + auto-hold configuration after problem is solved. | 06:17 |
| jeblair | jamielennox: (in zuulv3 of course "foo" could well be "base" and you'd get nodes for any failed job) | 06:18 |
| jamielennox | jeblair: ok, i read that differently, i would think that it's cheap to always hold the last 2 failing job nodes, and then cycle out the oldest when a new failing job is caught | 06:18 |
| jamielennox | so something that you can react to and grab, rather than something you need to set up auto-hold when you expect a problem | 06:19 |
| jeblair | jamielennox: at scale, you'd be racing the deletion, and the oldest 3 failures are usually as good as the newest 3. | 06:19 |
| jeblair | jamielennox: oh, you mean hold any job that fails always? | 06:19 |
| jamielennox | jeblair: right, and that was why i was thinking of the AUTO_HOLD state | 06:19 |
| jamielennox | jeblair: yes, up to X jobs | 06:20 |
| jeblair | jamielennox: jobs fail all the time, legitimately. i think no matter what scale you operated at, small or large, you would require a substantial portion of your quota in order to hold all of the failed jobs long enough for you to react to decide you wanted to hold a node for actual investigation. | 06:21 |
| jeblair | jamielennox: we've found it quite convenient to just say "hold the next 3 failures for this job" after we've encountered a problem which can't be diagnosed except by logging into a node (which is relatively rare in the scheme of things). | 06:21 |
| jamielennox | ok, both ways can work and that is certainly cheaper, it's generally not hard to retrigger the job and catch the next machine instead of the last | 06:23 |
| jeblair | every node counts :) | 06:24 |
| jamielennox | i guess i was looking at it from the point of view of racing the deleter | 06:24 |
| jeblair | i'm too lazy | 06:24 |
| jeblair | jamielennox: anyway, it's past my bedtime. good night! :) | 06:25 |
| jamielennox | jeblair: night - thanks for the explanation | 06:25 |
| *** isaacb has joined #zuul | 06:39 | |
| *** hashar has joined #zuul | 07:31 | |
| *** isaacb has quit IRC | 07:58 | |
| *** bhavik1 has quit IRC | 08:10 | |
| *** SpamapS has quit IRC | 09:04 | |
| *** SpamapS has joined #zuul | 09:42 | |
| *** isaacb has joined #zuul | 10:03 | |
| *** isaacb has quit IRC | 11:41 | |
| *** isaacb has joined #zuul | 11:41 | |
| Shrews | jamielennox: yep, the manual hold thing is definitely suboptimal. i think we should actually delete the 'hold' command because of that, IMO | 11:44 |
| Shrews | jeblair: how do you feel about deleting that command? the only other option i can think of is put a delay on USED nodes, that i don't particularly like | 11:46 |
| jamielennox | Shrews: i guess that's where i was going with my auto-hold idea, instead of a timer just hold the last X failed jobs | 11:59 |
| jamielennox | but i agree with jeblair that X is a hard thing to measure there because it will change drastically based on your scale | 11:59 |
| jamielennox | Shrews: i like the idea of a proper HOLD that an admin can request and actually play with something for a day or two that doesn't get counted in the auto-hold quota | 12:00 |
| jamielennox | but for simplicity now i don't care if we remove it and add it back when it makes sense | 12:00 |
| mordred | I could see manual hold maybe working if it's a zuul command - but it would have to be "hold the nodes for build {uuid}" - but then I'm not sure how that gets implemented locking wise or whatnot | 12:23 |
| mordred | 097831 | 12:23 |
| mordred | gah | 12:23 |
| mordred | nobody use that | 12:23 |
| mordred | :) | 12:23 |
| Zara | hunter2 | 12:30 |
| *** isaacb_ has joined #zuul | 12:32 | |
| *** isaacb has quit IRC | 12:35 | |
| *** bhavik1 has joined #zuul | 13:09 | |
| *** isaacb__ has joined #zuul | 13:56 | |
| *** isaacb_ has quit IRC | 13:58 | |
| *** patrickeast has quit IRC | 14:13 | |
| *** patrickeast has joined #zuul | 14:16 | |
| *** bhavik1 has quit IRC | 14:18 | |
| jeblair | jamielennox, Shrews, mordred: after further thought, i think having the hold command wait for the lock, while still not optimal, might be okay, and the best approach to take for now for manual hold, and worth a try. i think it's the technically correct thing to do in the current design and easy to do. any improvements would require more work. | 14:26 |
| jeblair | mordred: and yeah, having zuul do the hold is probably the next best/easiest thing to do. | 14:27 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Wait for lock when performing manual hold https://review.openstack.org/454217 | 14:33 |
| jeblair | jamielennox, Shrews, mordred: ^ that's what i'm thinking for manual hold | 14:33 |
| Shrews | jeblair: why not just block on the first call? | 14:36 |
| jeblair | Shrews: i wanted a nice print, but only if necessary. :) | 14:37 |
| jeblair | i don't feel strongly about that at all. :) | 14:37 |
| Shrews | i mean, it's going to print that 99.999999% of the time anyway | 14:38 |
| jeblair | that's true. maybe i should drop it. | 14:38 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Wait for lock when performing manual hold https://review.openstack.org/454217 | 14:38 |
| jeblair | Shrews: you convinced me | 14:38 |
| *** isaacb__ has quit IRC | 14:42 | |
| *** isaacb__ has joined #zuul | 14:48 | |
| jeblair | jamielennox: what are you going to do with https://review.openstack.org/445675 ? | 14:50 |
| jeblair | mordred: what's the eta on the lookup plugin? | 14:54 |
| mordred | jeblair: as soon as I'm done dealing with mirror breakages | 14:57 |
| jeblair | mordred: okay. i'm going to shut zuulv3 down until then. | 14:58 |
| mordred | jeblair: kk | 14:59 |
| jeblair | Shrews, pabelanger: +2 https://review.openstack.org/451470 with comments | 15:05 |
| pabelanger | o/ | 15:09 |
| pabelanger | just catching up | 15:09 |
| pabelanger | Is the security issue, a know issue or is this new? | 15:14 |
| pabelanger | known* | 15:14 |
| jeblair | pabelanger: for the lookup plugin issue. i mean, we've been running with it for a week, accidentally. next time we're going to be more careful about this. | 15:15 |
| pabelanger | k | 15:15 |
| jeblair | next vulnerability like that, we'll stop zuul immediately and make sure everyone knows it's a high priority issue and nominate someone to lead fixing it as their sole focus. | 15:16 |
| *** isaacb__ has quit IRC | 15:23 | |
| pabelanger | ++ | 15:24 |
| pabelanger | Shrews: mind a review on https://review.openstack.org/#/c/450045 | 15:25 |
| pabelanger | adds opensuse as an image so we can test it for glean and diskimage-builder jobs | 15:25 |
| Shrews | pabelanger: trade you for reviews on my docs changes, starting with https://review.openstack.org/453695 | 15:26 |
| pabelanger | Shrews: ah, yes. happy too | 15:28 |
| pabelanger | heh, found an issue in tox-docs jobs, we don't render html, like jenkins user does | 15:31 |
| pabelanger | fixing | 15:31 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Start blocking lookup plugins on insecure jobs https://review.openstack.org/454236 | 15:35 |
| mordred | jeblair, pabelanger, Shrews: ^^ there is the beginning - I'm working through the rest of the lookup plugins now but thought I'd go ahead and push that up | 15:36 |
| jeblair | mordred: cool, i'll start reviewing | 15:36 |
| openstackgerrit | Merged openstack-infra/nodepool master: Add opensuse 42.2 DIB testing https://review.openstack.org/450045 | 15:36 |
| pabelanger | ack | 15:36 |
| mordred | jeblair: OH - lookup plugins are also what implement with_foo (in additoin to the lookup function) | 15:37 |
| mordred | jeblair: so "with_items" is implemented by the items lookup plugin | 15:37 |
| mordred | (so just disabling all of them would be bad - thus why we have to look at each one of them) | 15:37 |
| jeblair | mordred: wow, so this is more important and expansive than i thought :) | 15:38 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add trigger developer doc https://review.openstack.org/453695 | 15:38 |
| mordred | jeblair: I think I'd like to have other folks take a pass through the list of lookup plugins too (github.com/ansible/ansible/lib/ansible/plugins/lookup) | 15:39 |
| jeblair | mordred: that 404s for me | 15:39 |
| mordred | some are - well, some make network calls, like the dig module - which aren't strictly a security risk, but also are executing code on the executor that's doing something, so may be a judgement call | 15:40 |
| mordred | oh - sorry -that's not a link | 15:40 |
| mordred | https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/lookup would be the link version | 15:40 |
| mordred | I'm currently just scanning for ones that do local file access | 15:41 |
| jeblair | mordred: gotcha. i can take a look. hopefully some other folks too | 15:42 |
| mordred | but like - dig seems fine-ish - consul_kv and etcd seem fine but also probably pointless since our users would not be able to set the host to use via config vars :) | 15:42 |
| jeblair | SpamapS, pabelanger, jlk: help requested ^ :) | 15:42 |
| mordred | actually ... | 15:42 |
| mordred | https://etherpad.openstack.org/p/zuul-ansible-lookup-plugins | 15:43 |
| mordred | maybe we should note which are good/bad/maybe | 15:43 |
| pabelanger | jeblair: mordred: we should at least add a test to validate this too. | 15:46 |
| mordred | pabelanger: oh - definitely | 15:47 |
| jeblair | pabelanger: doesn't this have the same issues as the rest of the plugin validation? | 15:47 |
| pabelanger | jeblair: good question | 15:48 |
| jeblair | pabelanger: i assume it would be the same as https://storyboard.openstack.org/#!/story/2000878 | 15:48 |
| pabelanger | jeblair: yes, we should start to make the a higher priority, if possible | 15:49 |
| jeblair | pabelanger, mordred: oh, well, actually, we can do the file-based tests now. i think we already have some. | 15:49 |
| Shrews | mordred: why does read_csv() call run() and not the super's read_csv()? | 15:50 |
| jeblair | we just can't test lookups which perform things on other hosts; i don't know if there is such a thing? | 15:50 |
| pabelanger | I think all looks are local IIRC | 15:50 |
| pabelanger | lookups* | 15:51 |
| pabelanger | Oh, I think I see what you mean. | 15:51 |
| Shrews | mordred: i think that's just a copy-pasta. if i'm wrong, i'll remove the -1 on that | 15:52 |
| jeblair | so yeah, i think we will probably be able to create tests now. | 15:53 |
| mordred | ok - first pass on the list is done | 15:57 |
| mordred | Shrews: nope - you're right - definite copy-pasta | 15:57 |
| jeblair | mordred: i've been thinking about env. maybe let's just reject for now and think about it later if it's an issue. | 15:57 |
| mordred | jeblair: kk | 15:58 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add upload-sphinx-draft role to tox-docs jobs https://review.openstack.org/454248 | 15:58 |
| jeblair | mordred: (we might decide to, instead, sanitize the zuul env, or add a whitelist or something. i agree it's grey). | 15:58 |
| mordred | yah. | 15:58 |
| * SpamapS reading | 16:00 | |
| jeblair | mordred: i don't understand indexed_items. | 16:12 |
| jeblair | mordred: i don't know what the basedir setting does there. | 16:12 |
| SpamapS | so, I think we should probably just whitelist the lookup plugins we think test users would need, and remove all others. | 16:13 |
| mordred | jeblair: basedir there seems to be copypasta | 16:14 |
| jeblair | mordred: that's what i was thinking, which would make that okay, yeah? | 16:15 |
| mordred | jeblair: the indexed_items just does that crazy list manipulation | 16:15 |
| mordred | yah | 16:15 |
| SpamapS | which I think is what is being suggested? | 16:15 |
| mordred | SpamapS: unfortunately, we can't _actually_ whitelist | 16:15 |
| mordred | SpamapS: we have to explicitly disable each module | 16:15 |
| mordred | but yes - the effective outcome of what you are saying is the plan | 16:15 |
| SpamapS | mordred: yeah.. we're back to that problem. :-P | 16:16 |
| jeblair | SpamapS: i think it's like the action plugins -- some of them are useful, say, looking up something from a file in the work area. so those we will filter input. the rest are either enable or disable. | 16:16 |
| SpamapS | moar indirection | 16:16 |
| mordred | jeblair: yes. that | 16:16 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Start blocking lookup plugins on insecure jobs https://review.openstack.org/454236 | 16:16 |
| mordred | this puts in a little bit more support (and fixes the copy-pasta Shrews found) | 16:16 |
| SpamapS | so when we say "BAD" on that etherpad.. that's "BAD, but might be filterable" ? | 16:17 |
| mordred | so hard-disabling a module should be a simple symlink | 16:17 |
| mordred | SpamapS: we should maybe differentiate | 16:17 |
| mordred | let em go do that real quick | 16:17 |
| SpamapS | because like, first_found might be really useful for stuff in the work dir. | 16:17 |
| mordred | SpamapS: yup. I just updated the list with "FILTER" for things I think we can filter | 16:19 |
| mordred | that does not include template, btw, because of how template is implemented - there's too much to easily override the parameter | 16:19 |
| jeblair | mordred: i don't see keyring.py. what branch are you on? | 16:20 |
| jeblair | hrm, i see it in github. i think my local git repo is fubar. | 16:20 |
| jeblair | mordred: okay, fixed. but keyring is only in devel, not stable-2.2 | 16:22 |
| mordred | jeblair: cool - we're ahead | 16:22 |
| mordred | jeblair: once we can do ansible/zuul gating, I'd love to put in a test that makes sure we notice when new plugins hit upstream devel | 16:23 |
| jeblair | ++ | 16:23 |
| SpamapS | I kind of wonder if there's a way to make a plugin plugin. | 16:26 |
| SpamapS | that just acts like a traffic cop for all plugins. | 16:26 |
| mordred | SpamapS: not currently | 16:27 |
| mordred | SpamapS: it has been discussed in the ansible core team | 16:27 |
| jeblair | SpamapS, mordred: the import stuff is weird enough that i'm not ready to say it would be impossible for us to do. but if other folks have considered that... | 16:28 |
| mordred | yah, bcoca does not want to do something like it until they're confident it'll actually work (he's concerned about adding an insecure security feature) | 16:29 |
| jeblair | oh hey i didn't know about 'pass'. that's neat. | 16:31 |
| mordred | pass is great | 16:32 |
| mordred | I mean, I haven't migrated to it from my gpg encrypted file of things personally yet | 16:32 |
| mordred | but I plan to | 16:32 |
| *** hashar has quit IRC | 16:34 | |
| SpamapS | yeah pass does look nice | 16:35 |
| SpamapS | so, shelvefile is out, IMO | 16:35 |
| mordred | ++ | 16:35 |
| jeblair | agree | 16:35 |
| SpamapS | Honestly I've never seen a use of shelve that turned out well. ;) | 16:37 |
| jeblair | dns, dnstxt, and url all seem of a class -- they could be useful, but would also allow internal network access of some kind. | 16:37 |
| SpamapS | I don't understand the NET classification. | 16:38 |
| SpamapS | url takes urls as arguments | 16:38 |
| SpamapS | so that seems like a BAD, not a NET | 16:38 |
| jeblair | SpamapS: it allows some form of network access | 16:38 |
| SpamapS | the description at the top suggests NET is "you can't configure the host so it's maybe ok?" | 16:39 |
| jeblair | SpamapS: (it's an undecided flag, at the moment) | 16:39 |
| SpamapS | or something like that | 16:39 |
| SpamapS | kk | 16:39 |
| jeblair | SpamapS: i think we've flippled all of them to bad except these 3. | 16:39 |
| jeblair | heh flippled. | 16:39 |
| jlk | Will dig through these lookups | 16:40 |
| SpamapS | I actually think any that do net access should be in the BAD category. It's likely the executor has access to stuff that nodes won't, and some of that stuff may be useful in breaking out of containment. | 16:40 |
| jeblair | i think the thing with url is maybe there's a use case for "download this image and copy it to all the nodes". | 16:40 |
| SpamapS | Though DNS seems like at least a very small hole to poke through. | 16:41 |
| SpamapS | jeblair: yeah, case could be made for thoughtful filtering | 16:41 |
| jlk | If you tell it to connect to a rogue DNS server, you could create a reverse tunnel | 16:41 |
| jlk | and by rogue, I mean not a real DNS server | 16:41 |
| SpamapS | jlk: indeed | 16:42 |
| SpamapS | have done that... :) | 16:42 |
| SpamapS | The 1s latency is really awful tho ;) | 16:42 |
| jlk | This effort is now whack-a-yak | 16:42 |
| jeblair | SpamapS: i think we can work around the url restriction with "have ansible download the url on one worker node, copy it back to executor workdir, then copy out to rest of the nodes". | 16:43 |
| SpamapS | jeblair: yeah I was just thinking that | 16:43 |
| jeblair | SpamapS: not ideal, but possible. that's enough to make me happy with "block this for now and reconsider later if needed" | 16:43 |
| SpamapS | Wait for user outrage to convince us to spend the time to carefully enable it. | 16:44 |
| mordred | ++ | 16:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Start blocking lookup plugins on insecure jobs https://review.openstack.org/454236 | 16:45 |
| mordred | k. that updates having linked all of our explicit BAD modules to _banned.py | 16:45 |
| jeblair | okay, i flipped the remaining nets to bad | 16:46 |
| mordred | cool | 16:46 |
| jeblair | i agree with all the annotations from spamaps; i'm not going to explicitly +1 them | 16:47 |
| SpamapS | Looks good. I'll feel even better with bwrap backstopping the effort. :) | 16:48 |
| jeblair | ayup | 16:48 |
| mordred | jlk: "Return a file (needle) in the task's expected search path." ... | 16:49 |
| mordred | jlk: what is "the task's expected search path" - does that mean anything do you? | 16:49 |
| jlk | huh, I recognize all those words, however in that arrangement, I have no idea what they mean. | 16:49 |
| jlk | which plugin? | 16:49 |
| mordred | I'm looking at filtering fileglob right now - but it calls a method called "find_file_in_search_path" in LookupBase | 16:50 |
| mordred | jlk: I mean, I figure I can just do the safe-file check on whatever is returned from that | 16:50 |
| mordred | but I kinda want to understand what the heck it even is | 16:51 |
| jlk | probably? | 16:51 |
| jlk | I wonder if it's looking locally, and doing role path shortcuts and such | 16:51 |
| * SpamapS is now curious | 16:51 | |
| SpamapS | ansible_search_path or role_path | 16:52 |
| mordred | yah. role_path | 16:52 |
| jeblair | https://github.com/ansible/ansible/issues/14341#issuecomment-234559431 | 16:52 |
| SpamapS | or '.' | 16:53 |
| mordred | ok. cool | 16:53 |
| SpamapS | so yeah, you would need to filter the output | 16:53 |
| jlk | but it looks like it could be overridden with ansible_search_path as a variable? | 16:53 |
| jlk | yeah, definitely filter. | 16:53 |
| mordred | yah - but that's still fine if we filter the results | 16:53 |
| jeblair | https://github.com/ansible/ansible/commit/3c39bb5633d0cbfa9cf22f9a4038296caef9c622 | 16:54 |
| mordred | and it _should_ alow things to work properly for tihngs in roles, since we copy those into the workspae so they're in the "ok" path area | 16:54 |
| SpamapS | since ansible_search_path=/ would let you search the whole FS | 16:54 |
| mordred | SpamapS: maybe we should also get crazy and filter ansible_search_path for acceptable paths? | 16:54 |
| jlk | what in the heck is a "shelvefile" ? | 16:54 |
| SpamapS | yes that seems like a valid thing to do | 16:54 |
| SpamapS | jlk: a database of pickled things | 16:55 |
| jlk | huh | 16:55 |
| SpamapS | or "the worst idea ever" | 16:55 |
| mordred | yah | 16:55 |
| SpamapS | take your pick | 16:55 |
| jlk | hahah, yeah, like, how would one use that effectively? | 16:55 |
| SpamapS | It's for writing python in 1998 | 16:55 |
| mordred | to be fair - jeblair and I used to hack on things that used the zodb - which was, in fact, an object database of pickled things | 16:55 |
| SpamapS | Zope? | 16:56 |
| mordred | yah baby | 16:56 |
| jlk | docs/docsite/rst/playbook_pahting.rst looks relevant | 16:56 |
| SpamapS | ZOMGpe no | 16:56 |
| jeblair | pretty nifty, the zodb. | 16:56 |
| mordred | and also non-zope things that used the zodb | 16:56 |
| SpamapS | Zope is one reason why launchpad is intractable. Though not zodb. | 16:56 |
| mordred | SpamapS: those who do not know Zope are condemned to reinvent it, poorly. | 16:57 |
| SpamapS | Zope + Storm + Postgres is like.. wow.. we found the two best ways to make a monolithic nightmare | 16:57 |
| mordred | yah | 16:57 |
| jlk | Now we have 6 problems | 16:57 |
| SpamapS | and then added the best database for that too | 16:57 |
| mordred | with custom index types | 16:57 |
| SpamapS | hey you gotta sort those dpkg versions | 16:57 |
| jeblair | yah, an app like launchpad is not, i think, leveraging zope's strengths. | 16:57 |
| mordred | so they actually used postgres' functional index feature | 16:57 |
| jlk | just stuff it all into a BerkeleyDB | 16:57 |
| mordred | which puts the computation right where it doesn't need to be - but I had MANY long arguments with lifeless about that back in the day | 16:58 |
| mordred | my argument was "sure, but just don't have a single table with all of the packages that you want to sort and filter by version in debian version sort order" | 16:58 |
| mordred | since you don't EVER need to sort multiple packages by single version - you should be able to just get the list of versions for a single package as opaque strings then sort _that_ app side | 16:59 |
| mordred | but silly me - I'm just a database scalability consultant | 16:59 |
| jlk | So I'm in a bit of a bind. My patch is failing upstream, generating an 87 meg log file. I can't get the tip of feature/zuulv3 to pass locally, so I have no good starting point from which to narrow down why things are failing in my patch tree :( | 17:08 |
| jlk | I guess I could wget the log file and trawl through it locally | 17:08 |
| jeblair | jlk: can you run one test locally? | 17:10 |
| jlk | Yeah I can get subsets of tests to pass locally | 17:10 |
| jlk | just not the entire suite | 17:10 |
| jlk | FAILED (id=6, failures=2 (-2), skips=28) | 17:11 |
| jlk | a ton of skips (I don't know why) and the two failures are kind of vague. | 17:11 |
| jlk | oh wait, this wasn't on tip. My mistake. let me re-run on tip | 17:11 |
| SpamapS | 28 is normal. | 17:13 |
| SpamapS | or around 28 I think | 17:14 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Start blocking lookup plugins on insecure jobs https://review.openstack.org/454236 | 17:14 |
| SpamapS | those are just the @skip's | 17:14 |
| mordred | jeblair: would you re-look at that before I go further? specifically the additoins to _paths and their use | 17:14 |
| mordred | jeblair: I believe we should have those 2 types of fails we want to produce | 17:14 |
| jeblair | mordred: ack | 17:16 |
| jeblair | mordred: i'll be surprised if we use _fail_lookup more than once, but other than that nit, lgtm. | 17:20 |
| mordred | jeblair: actually, good point. I'll go ahead and clean that up in the next pass | 17:22 |
| *** harlowja has joined #zuul | 17:23 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Simplify the log url https://review.openstack.org/438028 | 17:54 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Remove url_pattern config parameter https://review.openstack.org/447165 | 17:55 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Implement getSafeAttributes for other objects https://review.openstack.org/450459 | 17:56 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Make sql reporter score null when not set https://review.openstack.org/450460 | 17:56 |
| SpamapS | \o/ merging | 17:59 |
| SpamapS | jeblair: so, test_swift_instructions .. we can just remove that one, but we still need a "stdlib" post playbook that uses auth to load to swift, yes? | 18:05 |
| jeblair | SpamapS: yes, but i think we can defer that for a bit if desired. | 18:06 |
| jeblair | (defer creating the playbook) | 18:06 |
| SpamapS | right, sounds like it will happen naturally as we port infra jobs | 18:09 |
| jeblair | SpamapS: we aren't using that right now in infra, so from a bootstrap openstack-infra perspective, we can even defer it past that. jhesketh may be using it for a third-party ci setup though (as may be others), so it would be nice to get something in place shortly after. | 18:11 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Change mutex to counting semaphore https://review.openstack.org/442563 | 18:12 |
| SpamapS | jeblair: ACK | 18:14 |
| SpamapS | jeblair: I think it may be a good idea to add a column to the board that is "after 3.0" or "after infra" or something like that. | 18:15 |
| SpamapS | jeblair: feels like there's a need to focus down to the "infra runs it" milestone | 18:15 |
| jeblair | SpamapS: agreed, we're starting to collect those tasks. | 18:16 |
| SpamapS | another option is to make a new tag and move those things to that. | 18:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Rename tenant in updateConfigLayout https://review.openstack.org/454282 | 18:18 |
| jeblair | SpamapS: true. we could start tagging by milestone. | 18:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Rename tenant in updateConfigLayout https://review.openstack.org/454282 | 18:21 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Consume interface_ip from nodepool node https://review.openstack.org/449706 | 18:21 |
| SpamapS | jeblair: I think I'll just ponder on these through next week since it's vacation time for both of us. We can maybe make a push for reorganization of tasks after that. | 18:21 |
| jeblair | SpamapS: sounds good | 18:22 |
| SpamapS | now to see if I can get this test_timer_sshkey test to test something meaningful... :-P | 18:22 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Start blocking lookup plugins on insecure jobs https://review.openstack.org/454236 | 18:23 |
| jeblair | SpamapS: speaking of which .... i don't think we have a chair for the zuul meeting on monday. do folks want to have it? does someone want to chair? | 18:23 |
| mordred | okie. I think that'sall of them | 18:23 |
| jeblair | mordred: thx reviewing | 18:24 |
| SpamapS | The only chair I'm going to be concerned with on Monday is Mammoth Chair Lift #2. | 18:26 |
| jeblair | jesusaur, mordred, Shrews, clarkb, jlk, pabelanger, jhesketh: SpamapS, fungi, and I are all out monday. do any of you want a zuul meeting or should we skip? (sorry i forgot to ask at this week's meeting) | 18:28 |
| jesusaur | I'm fine skipping it | 18:28 |
| jesusaur | SpamapS: oh, you're going to mammoth? I've been meaning to go back there, I used to go yearly when my family lived in LA | 18:29 |
| Shrews | i'm out on monday? neat | 18:30 |
| SpamapS | jesusaur: Indeed... yearly pilgrimmage :) | 18:30 |
| jeblair | Shrews: you were before the colon; i was addressing you so you got a highlight. :) | 18:31 |
| Shrews | oh, there's a colon | 18:31 |
| mordred | Shrews: you miss parsed a : - check your lexer | 18:31 |
| mordred | jeblair: I'm fine skipping | 18:31 |
| Shrews | jeblair: mordred: i prefer to skip it, even still | 18:31 |
| jeblair | ...amongst the people out on monday are SpamapS, fungi, and I.... | 18:31 |
| Shrews | (the colon, but skipping the meeting is fine, too) | 18:31 |
| mordred | Shrews: I assume you'll be brunk by then anyway | 18:32 |
| mordred | also, I noticed the misspelling before hitting enter but decided to keep it | 18:32 |
| Shrews | mordred: brunk off my bass | 18:32 |
| jeblair | there's an urban dictionary entry for brunk. | 18:32 |
| * fungi would just as soon rather not know | 18:34 | |
| mordred | jeblair: I prefer definition 3 | 18:34 |
| mordred | "I got so Brunk last night I let Juan give me the ol Mexican Rice." | 18:35 |
| mordred | there's so much about that example sentence ... | 18:35 |
| jlk | Okay good news. Tip of zuulv3 passes tox locally | 18:36 |
| jlk | skips=31 | 18:36 |
| mordred | WOOT! | 18:36 |
| jlk | It's just my branch that won't pass :( | 18:40 |
| jeblair | mordred: missing 2 files by my count | 18:40 |
| jeblair | mordred: env and password | 18:40 |
| mordred | jeblair: oh - hey, lookie there | 18:41 |
| jeblair | (to be clear, my -1 was for that, not for the 'wishful thinking' comment i left inline) :) | 18:41 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Start blocking lookup plugins on insecure jobs https://review.openstack.org/454236 | 18:41 |
| mordred | jeblair: git add is my friend | 18:41 |
| jeblair | mordred: heh, it's never been mine :) | 18:42 |
| jlk | Is it your friend? Or is it just pretending to be your friend, while secretly sabotaging your work behind your back? | 18:42 |
| mordred | jeblair: yah. I agree with that wishful thinking comment | 18:42 |
| mordred | jlk: that. definitely that | 18:42 |
| mordred | I mean, I might could maybe make a PR to upstream to refactor the ones where we had to copy things because of sads | 18:43 |
| jeblair | mordred: env? | 18:43 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Start blocking lookup plugins on insecure jobs https://review.openstack.org/454236 | 18:43 |
| mordred | jeblair: yup. like jlk said. sabotaging my work | 18:44 |
| jeblair | +2 | 18:44 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Cleanup docs builds and error on warnings https://review.openstack.org/452964 | 18:50 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Cleanup docs builds and error on warnings https://review.openstack.org/452964 | 19:10 |
| jlk | FFFFffffff | 19:19 |
| mordred | jlk: I agree | 19:23 |
| jlk | Oof, when you're least expecting it: # (c) 2013, seth vidal <skvidal@fedoraproject.org> | 19:24 |
| *** openstack has joined #zuul | 19:48 | |
| mordred | jlk: yah | 19:56 |
| mordred | jlk: also, re: your comment in the review - I totally agree - I want to actually add a job once we have your gh patches merged that double checks that nothing adds new plugins to ansible without us knowing about it | 19:58 |
| *** harlowja has quit IRC | 20:03 | |
| jeblair | SpamapS, Shrews: can you prioritize https://review.openstack.org/454236 please? then we can merge it and restart zuul | 20:05 |
| mordred | jeblair, SpamapS, Shrews, jlk: fwiw, http://paste.openstack.org/show/605670/ is example output if that dir is enabled in lookup_plugins and I tried using with_env: | 20:14 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Clarify TriggerInterface documentation https://review.openstack.org/453713 | 20:22 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove source from pipelines (1/2) https://review.openstack.org/453362 | 20:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove source from pipelines (2/2) https://review.openstack.org/453821 | 20:26 |
| SpamapS | jeblair: certainly will look at it next | 20:31 |
| *** SpamapS has quit IRC | 20:44 | |
| *** SpamapS has joined #zuul | 20:45 | |
| *** SpamapS has quit IRC | 20:45 | |
| *** SpamapS has joined #zuul | 20:46 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove source from pipelines (1/2) https://review.openstack.org/453362 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Replace config/project repos with trusted/untrusted projects https://review.openstack.org/453347 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fix constructor arguments to source https://review.openstack.org/451102 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add hostname to TriggerEvent https://review.openstack.org/452348 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove source from pipelines (2/2) https://review.openstack.org/453821 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add canonical hostname to source object https://review.openstack.org/451110 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fully qualify project configuration names https://review.openstack.org/451970 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add source to project and remove unused tenant attrs https://review.openstack.org/451969 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Pass source to project instantiations https://review.openstack.org/451596 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add a project index to Tenant https://review.openstack.org/451597 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove unused Tenant.getRepo method https://review.openstack.org/451929 | 20:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Use new tenant project index for config references https://review.openstack.org/451928 | 20:56 |
| jeblair | pabelanger, mordred, jhesketh: anyone want to volunteer to push some simple test-enable patches in? | 20:59 |
| SpamapS | moar tests | 21:00 |
| jeblair | mordred: are you working on tests for your lookup blocks? | 21:01 |
| *** harlowja has joined #zuul | 21:03 | |
| *** openstackgerrit has quit IRC | 21:07 | |
| *** notmorgan is now known as morgan | 21:08 | |
| jeblair | harlowja: what do you think about the status of https://github.com/python-zk/kazoo/pull/419 ? | 21:15 |
| jeblair | i will single-core approve the tests with no logic changes | 21:20 |
| harlowja | i haven't been able to find the kazoo peoples : | 21:21 |
| harlowja | :( | 21:21 |
| jeblair | harlowja: it's looking increasing like it's you. :| | 21:22 |
| harlowja | :/ | 21:22 |
| jeblair | congratulations | 21:22 |
| harlowja | waaa waaa | 21:22 |
| harlowja | lol | 21:22 |
| SpamapS | They all work on etcd now | 21:23 |
| jlk | oh this isn't good. I left tests running when I went to lunch, like 2 hours ago. | 21:23 |
| jlk | and they're still going :( | 21:23 |
| SpamapS | jlk: Perhaps there's something getting stuck. ;) | 21:23 |
| pabelanger | jeblair: I am not sure what 'simple test-enable patches | 21:25 |
| pabelanger | are | 21:25 |
| jlk | take a test that is @skip, make it work. | 21:25 |
| pabelanger | k | 21:26 |
| jeblair | pabelanger: they are not very complex patches to zuul which enable tests. | 21:26 |
| pabelanger | I can focus on that for tomorrow | 21:26 |
| jeblair | i was going to make a list, but it didn't seem anyone was around so i've just single-core approved them. | 21:26 |
| jeblair | pabelanger: are you working on tests for mordred's lookup plugin stuff? | 21:27 |
| SpamapS | they're patches with Story: 2000773 | 21:27 |
| pabelanger | jeblair: not yet. I can see about adding something this evening | 21:28 |
| jeblair | pabelanger: was just a question. i'm trying to figure out if anyone is working on it now. | 21:28 |
| jeblair | i've got a 'no' from you, and am waiting on a response from mordred. | 21:29 |
| pabelanger | okay, I'm happy to write up something, if we need it | 21:29 |
| *** openstackgerrit has joined #zuul | 21:29 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Start blocking lookup plugins on insecure jobs https://review.openstack.org/454236 | 21:29 |
| jeblair | pabelanger: assuming mordred isn't working on it, i will probably do it now. | 21:30 |
| adam_g | is there a way to specify the ansible_ssh_user that gets used by an executor? i believe its configurable for the ansiblelaunchserver but dont see anything that allows it in executor/server.py | 21:30 |
| jeblair | adam_g: https://review.openstack.org/453983 | 21:30 |
| adam_g | jeblair: ha, thanks | 21:30 |
| jeblair | pabelanger, mordred, clarkb, jhesketh: if one of you could +3 https://review.openstack.org/453303 that would be great. it's almost trivial. | 21:32 |
| jeblair | pabelanger: i responded on https://review.openstack.org/453974 | 21:35 |
| jlk | Oh good, 3.8G of test output. | 21:36 |
| jeblair | harlowja: are you going to continue to wait for the other kazoo folks? | 21:37 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Re-enable test_queue_rate_limiting https://review.openstack.org/452835 | 21:37 |
| jeblair | harlowja: i'm about to take off for a week, but when i get back, maybe i should look at just copying that recipe into our local app for a while so we can start using it? | 21:39 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Re-enable test_queue_rate_limiting_dependent https://review.openstack.org/452864 | 21:39 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Re-enable remaining test_crd_check_* tests https://review.openstack.org/452996 | 21:39 |
| mordred | jeblair: I'm not this instant - but that's next on my list (I had to step afk for a little bit) | 21:46 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Quote ansible command and include config file https://review.openstack.org/453974 | 21:46 |
| jeblair | mordred: ok. i have a few minutes now; do you want me to handle it or do you want to keep it? | 21:47 |
| mordred | jeblair: either way - I do not feel emotionally attached to doing it - but I also don't want to take time you could be using to do something more jeblair-ish | 21:49 |
| jeblair | mordred: i'll trade you for as many reviews on 451113 453303 448555 446275 454217 453802 453797 as you can muster :) | 21:52 |
| harlowja | jeblair unsure, probably not | 21:52 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove source from pipelines (1/2) https://review.openstack.org/453362 | 21:53 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Replace config/project repos with trusted/untrusted projects https://review.openstack.org/453347 | 21:53 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove source from pipelines (2/2) https://review.openstack.org/453821 | 21:53 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fully qualify project configuration names https://review.openstack.org/451970 | 21:53 |
| *** openstack has joined #zuul | 22:04 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use libyaml if possible https://review.openstack.org/451113 | 22:04 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Re-enable test_live_reconfiguration_del_project https://review.openstack.org/453303 | 22:05 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Make sure services are running for test-setup.sh https://review.openstack.org/448555 | 22:06 |
| mordred | jeblair: I just landed thefirst patch in the canonical hostnames series - but patch 2 has a jenkins sad | 22:28 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Wait for lock when performing manual hold https://review.openstack.org/454217 | 22:29 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Fix internal doc refs to renamed section https://review.openstack.org/453802 | 22:29 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Start adding operational docs to zuulv3 https://review.openstack.org/453797 | 22:30 |
| jeblair | mordred: pretty sure that's a racy test. (but not very racy, it didn't hit any of the others). i rechecked. | 22:35 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 22:35 |
| mordred | jeblair: ok. cool. I'll keep going then | 22:37 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix constructor arguments to source https://review.openstack.org/451102 | 22:38 |
| jeblair | mordred: fyi, i got as far as verifying that a passwd lookup is blocked, but then the approach i chose to test more lookups has exposed an issue with dynamic reconfiguration, so i'm down the rabbit hole a bit. | 22:48 |
| mordred | jeblair: oh! well - maybe that wound up being a good thing | 22:50 |
| mordred | probably a good rabbit hole | 22:50 |
| mordred | mmm. rabbit is tasty | 22:50 |
| jeblair | ya | 22:50 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add canonical hostname to source object https://review.openstack.org/451110 | 22:55 |
| jhesketh | Morning | 22:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP fix dynamic reload https://review.openstack.org/454395 | 23:12 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP test lookups https://review.openstack.org/454396 | 23:12 |
| jeblair | mordred, SpamapS: WIP to illustrate approach | 23:13 |
| jeblair | mordred: i think probably we don't need to test 'lookup' separate from 'with'. would you agree? | 23:14 |
| jeblair | the thing i'm not sure how to do is verify that we hit that actual exception, vs some template formatting error. | 23:18 |
| jeblair | for the moment, i'm just reading the log output to make sure, but that's obviously something that would be good to automate so we don't regress. | 23:18 |
| mordred | jeblair: yah. I mean - it's in the text that shows up in the playbook output | 23:19 |
| mordred | so maybe when we get json output from playbooks, we can just parse that in the tests | 23:19 |
| jeblair | mordred: OH! i understand "list.py". that's so that "with_list" works! | 23:20 |
| jeblair | mordred: because "with_foo" means foo is dynamically mapped to "lookup plugin named foo" | 23:20 |
| mordred | yes | 23:21 |
| mordred | fun right? | 23:21 |
| Shrews | harlowja: so is kazoo abandoned then? | 23:26 |
| harlowja | idk | 23:26 |
| harlowja | i'm still around, lol | 23:26 |
| Shrews | should we recode everything in Rust? | 23:26 |
| harlowja | sure | 23:26 |
| harlowja | lol | 23:26 |
| Shrews | SpamapS will like that | 23:26 |
| SpamapS | Yeesss | 23:29 |
| harlowja | :-P | 23:30 |
| SpamapS | jeblair: mordred we have the same problem in detecting the action plugin fail. I broke it in funny ways with bwrap that would be detected if we verified the text of the fail log | 23:31 |
| mordred | SpamapS: k. I'll prioritize the json logging bit next | 23:34 |
| jlk | mordred: yeah, but what's the point of 'with_list' ? how is that different than with_items, handing it a list? | 23:34 |
| mordred | jlk: yes. it's literally the same thing | 23:35 |
| mordred | class LookupModule(LookupBase): | 23:36 |
| mordred | def run(self, terms, **kwargs): | 23:36 |
| jlk | ... but why? | 23:36 |
| mordred | return terms | 23:36 |
| mordred | that's the entire module | 23:36 |
| jlk | right I looked at the code and scratched my head | 23:36 |
| mordred | right? | 23:36 |
| mordred | I mena- I totally agree | 23:36 |
| mordred | maybe it's hysterical raisins | 23:36 |
| jlk | them cheeky buggers. | 23:37 |
| mordred | jlk: the commit message says "lookup plugin that does no flattening" | 23:37 |
| mordred | jlk: does with_items maybe flatten things? | 23:38 |
| jlk | yes, those are words | 23:38 |
| mordred | hahahaha | 23:38 |
| jlk | no, there is a with_flatten for that | 23:38 |
| mordred | I literally lol'd | 23:38 |
| jeblair | mordred, SpamapS, jlk: i think some of these filters are too aggressive -- the 'ansible_search_path' includes things like the playbook directory. so you can say "lookup this csvfile which i distribute alongside the playbook" (or presumably role as well?). those will be outside the working directory for some repos, but not others. | 23:46 |
| jeblair | weirdly, it's going to be the trusted-repos which won't be able to have playbooks which lookup data in files within their own repo | 23:46 |
| jeblair | untrusted playbooks will | 23:46 |
| jeblair | (because the untrusted playbooks are located within the working dir, trusted ones outside) | 23:46 |
| jlk | that's... weird? | 23:46 |
| jlk | Does the path filtering not allow it to look within? | 23:46 |
| jlk | OIC | 23:46 |
| jlk | so, trusted would have to include "." in the search path? | 23:47 |
| jlk | for whatever the workdir is? | 23:47 |
| jeblair | jlk: that (hopefully) won't work because it will still fail our "only access files within the work dir" check -- that's the filtering we are adding to these plugins. | 23:47 |
| jeblair | at any rate, i'm happy to continue with this for now and fix later if needed, since we're failing closed (the safer option), so to speak. | 23:48 |
| jlk | I think I get what you're saying | 23:48 |
| jlk | Trusted playbooks live outside the workdir, and may want to load something lazily from within their path. OUr filtering prevents loading anything outside the workdir | 23:49 |
| jlk | I approve of that stance, wait until something NEEDS that functionality to open it up carefully. | 23:49 |
| jeblair | exactly | 23:49 |
| jeblair | it's actually better than that. trusted playbooks don't run with the restricted configuration, so even that shouldn't be a problem | 23:50 |
| jeblair | however, if this extends to roles, we may have problems. | 23:51 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP test lookups https://review.openstack.org/454396 | 23:53 |
| jeblair | mordred: ^ it's EOD for me; if you want to continue that early tomorrow before i get up, be my guest. or anyone else who wants to continue it tonight. i was just planning on adding at least one test for each lookup plugin. | 23:54 |
| jlk | jeblair: I believe it goes into roles, searching for templates/files within the roles path | 23:54 |
| jlk | What is "Alarm clock" ? | 23:55 |
| jeblair | jlk: yeah, so then "trusted role run by untrusted playbook" would be the edge case i'm worried about. but our handling of that is currently wrong anyway; i need to fix it up. i think i'll be able to reason about it after i do that (tomorrow or when i return from vacation) | 23:55 |
| jeblair | jlk: it's the signal mechanism that the hard test timeout fixture uses to kill the running job | 23:55 |
| jeblair | er | 23:55 |
| jlk | alright. Honestly I haven't run across a lot of these lookups in the field though. | 23:55 |
| jeblair | 'running test' | 23:55 |
| jeblair | jlk: yeah, i'm not going to lose sleep over it :) | 23:56 |
| jlk | heh | 23:56 |
| jlk | I wonder if I need to tmpfs this. | 23:56 |
| jeblair | jlk: i do. | 23:57 |
| jlk | what was the commadn for that? | 23:57 |
| jeblair | jlk: i also tmpfs zookeeper | 23:57 |
| jeblair | jlk: set "ZUUL_TEST_ROOT=/path/to/tmpfs" env var | 23:57 |
| jeblair | jlk: or i think TMPDIR will just work too. | 23:58 |
| jeblair | (i haven't verified that) | 23:58 |
| jlk | tmpfs zookeeper? Is that fairly easy? | 23:58 |
| jeblair | jlk: yeah, i mount another tmpfs on /var/lib/zookeeper. just put it in fstab. | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!