| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add action plugins to restrict untrusted execution https://review.openstack.org/428798 | 00:12 |
|---|---|---|
| jeblair | mordred: i caused a merge conflict with that, so i fixed it ^ | 00:13 |
| jeblair | mordred: i added an argument (required) to findPlaybook.py | 00:13 |
| jeblair | mordred: if you want to take a look at that, i think we can go ahead and +3 it if it looks good | 00:13 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add __repr__ to jobtree https://review.openstack.org/430493 | 00:15 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Correct playbook requirement logic https://review.openstack.org/434564 | 00:17 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Move ProviderManagers into main NodePool thread https://review.openstack.org/434567 | 00:28 |
| Shrews | jeblair: ^^^ if that passes, I'll rebase the stack on it | 00:29 |
| Shrews | passed locally, at least | 00:29 |
| openstackgerrit | Merged openstack-infra/zuul master: Add support for sqlalchemy reporter https://review.openstack.org/223333 | 00:42 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Re-enable devstack test job https://review.openstack.org/431649 | 00:45 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add lock state to node listing https://review.openstack.org/434397 | 00:45 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Implement node cleanup https://review.openstack.org/433736 | 00:45 |
| jeblair | i guess so :) | 00:45 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Re-enable devstack test job https://review.openstack.org/431649 | 00:48 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add lock state to node listing https://review.openstack.org/434397 | 00:48 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Implement node cleanup https://review.openstack.org/433736 | 00:48 |
| Shrews | jeblair: both comments are now addressed | 00:48 |
| Shrews | and hopefully the restack passes | 00:50 |
| Shrews | pabelanger: fyi, looks like we now have a spurious fail for newly enabled test_image_upload_fail test: http://logs.openstack.org/97/434397/3/check/gate-nodepool-python27-ubuntu-xenial/2114564/testr_results.html.gz | 01:09 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix missing mutex release when aborting builds https://review.openstack.org/432211 | 01:13 |
| *** saneax-_-|AFK is now known as saneax | 03:29 | |
| *** saneax is now known as saneax-_-|AFK | 05:12 | |
| *** saneax-_-|AFK is now known as saneax | 05:22 | |
| *** Cibo_ has joined #zuul | 05:47 | |
| *** saneax is now known as saneax-_-|AFK | 06:11 | |
| *** saneax-_-|AFK is now known as saneax | 06:54 | |
| *** abregman has joined #zuul | 06:55 | |
| *** Cibo has quit IRC | 07:51 | |
| *** Cibo has joined #zuul | 08:04 | |
| *** bhavik1 has joined #zuul | 08:24 | |
| *** isaacb has joined #zuul | 08:51 | |
| *** hashar has joined #zuul | 09:10 | |
| *** abregman has quit IRC | 09:10 | |
| *** abregman has joined #zuul | 09:16 | |
| *** openstackgerrit has quit IRC | 09:32 | |
| *** isaacb has quit IRC | 10:05 | |
| *** Cibo has quit IRC | 10:08 | |
| *** isaacb has joined #zuul | 10:13 | |
| *** Cibo has joined #zuul | 10:22 | |
| *** bhavik1 has quit IRC | 10:48 | |
| *** isaacb has quit IRC | 10:48 | |
| *** isaacb has joined #zuul | 11:06 | |
| *** saneax is now known as saneax-_-|AFK | 11:22 | |
| *** saneax-_-|AFK is now known as saneax | 11:46 | |
| *** isaacb has quit IRC | 12:24 | |
| *** isaacb has joined #zuul | 12:26 | |
| *** openstackgerrit has joined #zuul | 12:55 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Use reno for release notes https://review.openstack.org/262597 | 12:55 |
| *** abregman is now known as abregman|mtg | 13:34 | |
| *** abregman|mtg has quit IRC | 13:34 | |
| pabelanger | Shrews: :( Looks like on the nodepool-builder side | 13:41 |
| pabelanger | I'll look into it this morning | 13:42 |
| Shrews | pabelanger: I think I see the problem. It's a race with the cleanup thread: [nodepool.builder.CleanupWorker.0] Removing failed upload record: | 13:48 |
| pabelanger | Shrews: Oh, neat. Have idea of a fix? | 13:48 |
| Shrews | pabelanger: We might need to come up with a way to disable that thread. Maybe if we set cleanup_interval to -1, not start it? | 13:49 |
| Shrews | will involve a very minor code change | 13:49 |
| *** bhavik1 has joined #zuul | 13:51 | |
| Shrews | pabelanger: or, alternatively, not delete records until they are X hours old or something | 13:52 |
| *** bhavik1 has quit IRC | 13:53 | |
| pabelanger | Right | 13:54 |
| *** abregman has joined #zuul | 14:09 | |
| Shrews | oh neat. looking at that has helped me find another race | 14:10 |
| *** _ari_ has quit IRC | 14:20 | |
| *** _ari_ has joined #zuul | 14:22 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Fix potential race in image upload cleanup https://review.openstack.org/434934 | 14:24 |
| Shrews | pabelanger: that ^^^ fixes the other thing I happened to see. It doesn't fix the original problem that caused the test to fail. | 14:25 |
| Shrews | pabelanger: I'm going to be afk for a while today. If you want to take a stab at the other issue, go for it. Otherwise I can later on. | 14:28 |
| pabelanger | ack | 14:28 |
| pabelanger | ya, I'll look shortly | 14:28 |
| *** isaacb has quit IRC | 14:28 | |
| Shrews | actually, doing the X hours old check would solve the thing 434934 is fixing, too. meh | 14:34 |
| *** saneax is now known as saneax-_-|AFK | 14:39 | |
| *** isaacb has joined #zuul | 14:39 | |
| *** hashar has quit IRC | 14:42 | |
| *** hashar has joined #zuul | 14:43 | |
| *** Cibo_ has quit IRC | 14:59 | |
| *** Cibo_ has joined #zuul | 14:59 | |
| jeblair | mordred: can you please +2 428798? | 15:12 |
| mordred | jeblair: I can! | 15:15 |
| pabelanger | jeblair: clarkb: if you have time, https://review.openstack.org/#/c/430418/ for standing up zuulv3-dev.o.o. Right now, it only installs merger bits. I'm working to abstract out the zuul-scheduler and zuul-launcher bits still | 15:17 |
| *** saneax-_-|AFK is now known as saneax | 15:18 | |
| jeblair | pabelanger: 2 small things i'd change | 15:21 |
| pabelanger | jeblair: ack | 15:22 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add action plugins to restrict untrusted execution https://review.openstack.org/428798 | 15:23 |
| pabelanger | remote: https://review.openstack.org/430418 Add zuulv3-dev.openstack.org server | 15:25 |
| pabelanger | updated | 15:25 |
| jeblair | pabelanger: thanks, lgtm | 15:28 |
| jeblair | mordred: \o/ | 15:28 |
| mordred | jeblair: woot. I'mma try to have the log callback plugin thing today | 15:29 |
| jeblair | sweet | 15:29 |
| *** saneax is now known as saneax-_-|AFK | 15:46 | |
| *** isaacb has quit IRC | 15:49 | |
| *** abregman has quit IRC | 16:13 | |
| *** Cibo_ has quit IRC | 16:21 | |
| SpamapS | jeblair: so, Adam Jacob, founder of Chef, is a huge fan of the speculative algorithm in Zuul | 16:29 |
| SpamapS | jeblair: he wants to try out Zuul for Habitat's CI (https://www.habitat.sh/) | 16:29 |
| SpamapS | jeblair: over beers, we decided that the algorithm needs a name. We have chosen "The Blair Algorithm" | 16:30 |
| pabelanger | TM pending | 16:31 |
| mordred | SpamapS: nice! | 16:37 |
| mordred | SpamapS: you know, if you wrote a wikipedia article on "The Blair Algorithm" I think you would effectively name it that | 16:37 |
| SpamapS | good idea | 16:40 |
| SpamapS | so apparently Adam actually lifted the algorithm and put it into one of Chef's proprietary products (Chef Automate I think?) | 16:40 |
| jeblair | it's described here, btw: http://amo-probos.org/post/16 | 16:41 |
| SpamapS | which is funny, because they can't use it for Habitat because Habitat is too open source. :) | 16:41 |
| jeblair | used to be called NNFI (nearest non-failing item) :) | 16:41 |
| SpamapS | jeblair: NNFI would be an interesting alternative if you object to your name being used. :) | 16:42 |
| jeblair | i don't think it's for me to say :) | 16:43 |
| jeblair | SpamapS: when would adam be looking to try zuul? | 16:44 |
| SpamapS | jeblair: He's not under pressure to change things. But they also don't gate on any significant tests and would like to. | 16:49 |
| jeblair | SpamapS: so we should check back with him in a bit when we think it might not be terrible for him to try? | 16:50 |
| SpamapS | jeblair: So I think they can wait for a functioning BonnyCI with in-repo config. | 16:50 |
| SpamapS | He's not interested in operating Zuul. :) | 16:50 |
| SpamapS | The toy thing we have now is too limited to be interesting to them. | 16:51 |
| jeblair | gotcha | 16:53 |
| SpamapS | jeblair: but he's also willing to deal with a terrible experience, so a good option for an early v3 beta user. :) | 16:58 |
| SpamapS | also Habitat is in Rust so........ | 16:58 |
| SpamapS | we have to | 16:58 |
| jeblair | SpamapS: the stars align | 16:58 |
| SpamapS | jeblair: on a bigger note.. I did not get to demo Zuul for CNCF, or even share slides, but I did get a better map of the territory and I think they're absolutely on a trajectory to need it. | 17:01 |
| SpamapS | anyway, time to board flight home | 17:02 |
| jeblair | SpamapS: cool, thanks! | 17:02 |
| rcarrillocruz | clarkb: WIP'd https://review.openstack.org/#/c/404243/16 , need to address the ini thing still | 17:22 |
| clarkb | rcarrillocruz: ok, its possible we just assert that its basically ini and not bash/shell | 17:24 |
| clarkb | rcarrillocruz: but we don't say that today | 17:24 |
| rcarrillocruz | well, i took it as a properties file, it's just the ini lookup accepts a type param, and using 'propeties' for it | 17:26 |
| rcarrillocruz | but yeah, isee what you mean about 'api' change here | 17:26 |
| rcarrillocruz | i can shellify it | 17:26 |
| *** hashar has quit IRC | 17:35 | |
| *** leifmadsen_ has joined #zuul | 18:08 | |
| *** leifmadsen has quit IRC | 18:11 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Reload tenant configuration on change merged https://review.openstack.org/426433 | 18:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Correct playbook requirement logic https://review.openstack.org/434564 | 18:18 |
| jeblair | okay, i think the auto-reconfigure logic is ready | 18:19 |
| jeblair | that should be handy during the PTG, as it means we can land changes to repos and they'll immediately take place | 18:19 |
| jeblair | also, it means that in zuulv3 we can stop doing the puppet reconfigure dance :) | 18:20 |
| mordred | jeblair: woot! | 18:26 |
| *** Cibo_ has joined #zuul | 18:27 | |
| *** Cibo_ has quit IRC | 18:31 | |
| *** hashar has joined #zuul | 19:27 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Honor NODE_CLEANUP time again https://review.openstack.org/435083 | 19:45 |
| *** Cibo_ has joined #zuul | 19:48 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Honor NODE_CLEANUP time again https://review.openstack.org/435083 | 19:54 |
| jeblair | Shrews: the thing we need to be careful with is that right now, the only way for a node to be cleaned up is via the NodeCleanupWorker. the intent is that nodes that we are finished with should be cleaned up immediately (eventually i think this should be a watch, but for now, a polling delay of a few tens of seconds is fine). but we don't want to put in an 8 hour delay between use and deletion. | 19:57 |
| Shrews | jeblair: there is no delay on those we have explicitly said to delete | 19:58 |
| Shrews | jeblair: this should be carrying over the identical behavior of the current nodepoold | 19:59 |
| Shrews | unless you want to change that | 19:59 |
| jeblair | Shrews: no, we're missing a piece -- the current nodepool knows when a node is finished because of zmq. that doesn't exist anymore. | 19:59 |
| jeblair | Shrews: zuul does not set a node to delete, it leaves it at 'used'. it's up to nodepool to take an unlocked used node and decide what to do with it. generally, that's delete. it won't always be the case though. | 20:00 |
| jeblair | that's why i think that we eventually need a watch so that nodepool can make that transition as quickly as possible. but we can add that later. | 20:01 |
| Shrews | jeblair: ah, i assumed zuul set it to DELETE | 20:01 |
| Shrews | jeblair: ok. i'll abandon that then | 20:01 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Wait for server deletion before deleting ZK node https://review.openstack.org/435091 | 20:05 |
| Shrews | jeblair: that one is something we missed ^^^^ | 20:05 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Partial dead code deletion https://review.openstack.org/435095 | 20:16 |
| openstackgerrit | Chris St. Pierre proposed openstack-infra/zuul master: Add optional basic auth for merge check https://review.openstack.org/435098 | 20:18 |
| jeblair | Shrews: ++ | 20:21 |
| jeblair | Shrews: your stack has +2s from me on all of them; i didn't +3 some things in case others want to take a look | 20:31 |
| Shrews | jeblair: your optimism on other reviewers amuses me :) | 20:33 |
| Shrews | my code scares folks away | 20:33 |
| Shrews | but thanks! | 20:33 |
| jeblair | pabelanger fears nothing | 20:33 |
| Shrews | true dat | 20:33 |
| Shrews | i think he has Canadian ice in his veins | 20:34 |
| Shrews | or maybe Canadian whiskey | 20:35 |
| pabelanger | Bacon | 20:38 |
| jeblair | canadian bacon? | 20:49 |
| pabelanger | Ham? | 21:14 |
| pabelanger | :p | 21:14 |
| pabelanger | okay, attempting to stand up zuulv3-dev.o.o again | 21:15 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Use thread name in launcher ID https://review.openstack.org/435125 | 21:28 |
| Shrews | jeblair: pabelanger: If you get a chance, ^^^ | 21:29 |
| Shrews | Also, I think we might have a race on the node request objects. Will have to track that down. | 21:30 |
| pabelanger | +2 | 21:31 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix possible race with node request handling https://review.openstack.org/435128 | 21:36 |
| Shrews | I think that may be the issue ^^^^ | 21:37 |
| pabelanger | http://zuulv3-dev.openstack.org/ | 21:37 |
| pabelanger | is a thing | 21:37 |
| Shrews | \o/ | 21:37 |
| pabelanger | just confirming puppet will run on the node | 21:39 |
| mordred | pabelanger: I just actually lol'd at "Bacon" from above | 21:54 |
| pabelanger | Feb 16 21:56:25 zuulv3-dev puppet-user[4332]: Finished catalog run in 6.09 seconds | 21:57 |
| pabelanger | yay | 21:57 |
| pabelanger | nice, so both servers are in place for monday | 21:57 |
| mordred | \o/ | 22:01 |
| jhesketh | Morning | 22:03 |
| *** leifmadsen_ is now known as leifmadsen | 22:04 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Move ProviderManagers into main NodePool thread https://review.openstack.org/434567 | 22:21 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Implement node cleanup https://review.openstack.org/433736 | 22:26 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Re-enable devstack test job https://review.openstack.org/431649 | 22:27 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Add lock state to node listing https://review.openstack.org/434397 | 22:27 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Wait for server deletion before deleting ZK node https://review.openstack.org/435091 | 22:28 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Partial dead code deletion https://review.openstack.org/435095 | 22:29 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Use thread name in launcher ID https://review.openstack.org/435125 | 22:29 |
| Shrews | w00t. thx mordred! | 22:33 |
| Shrews | pabelanger: jeblair: if we can get this last bug fix merged (https://review.openstack.org/435128) i think we're good-to-go for PTG | 22:34 |
| jeblair | whoops | 22:39 |
| jeblair | +3 | 22:39 |
| pabelanger | nice | 22:40 |
| pabelanger | going to be a fun week | 22:40 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix possible race with node request handling https://review.openstack.org/435128 | 22:44 |
| Shrews | gah. had to fix overwriting the req object which held the actual lock ^^^ | 22:44 |
| Shrews | *sigh* | 22:44 |
| Shrews | those two test failures does show we seem to hit that scenario quite often, though | 22:45 |
| Shrews | do show | 22:46 |
| *** saneax-_-|AFK is now known as saneax | 23:00 | |
| *** jamielennox is now known as jamielennox|away | 23:02 | |
| *** saneax is now known as saneax-_-|AFK | 23:02 | |
| jeblair | Shrews: we could add a 'refresh' method to the object to update fields and keep it around. it could have special handling for when the object disappears (which should never happen when we hold a lock anyway). | 23:05 |
| jeblair | Shrews: (+2 on current iteration) | 23:05 |
| *** jamielennox|away is now known as jamielennox | 23:14 | |
| *** hashar has quit IRC | 23:35 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!