| openstackgerrit | Merged openstack-infra/nodepool master: Add support for yappi and objgraph output https://review.openstack.org/653541 | 00:18 |
|---|---|---|
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Clarify .zuul.d directory https://review.openstack.org/653644 | 00:25 |
| *** jamesmcarthur has joined #zuul | 00:44 | |
| *** wxy-xiyuan_ has joined #zuul | 00:54 | |
| *** kmalloc_ has joined #zuul | 00:54 | |
| *** dmsimard5 has joined #zuul | 00:54 | |
| *** webknjaz_ has joined #zuul | 00:55 | |
| *** jamesmcarthur has quit IRC | 00:56 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: WIP: Support Ansible 2.8 https://review.openstack.org/631933 | 01:01 |
| *** webknjaz has quit IRC | 01:02 | |
| *** sshnaidm has quit IRC | 01:02 | |
| *** kmalloc has quit IRC | 01:02 | |
| *** adam_g has quit IRC | 01:02 | |
| *** dmsimard has quit IRC | 01:02 | |
| *** wxy-xiyuan has quit IRC | 01:02 | |
| pabelanger | tobiash: okay, figured out remote_zuul_stream issue, new stats are now reported: https://github.com/ansible/ansible/pull/48418 | 01:02 |
| *** kmalloc_ is now known as kmalloc | 01:02 | |
| *** webknjaz_ is now known as webknjaz | 01:02 | |
| *** wxy-xiyuan_ is now known as wxy-xiyuan | 01:02 | |
| *** dmsimard5 is now known as dmsimars | 01:03 | |
| *** dmsimars is now known as dmsimard | 01:03 | |
| *** irclogbot_1 has quit IRC | 01:05 | |
| *** irclogbot_0 has joined #zuul | 01:07 | |
| *** sshnaidm has joined #zuul | 01:08 | |
| *** jamesmcarthur has joined #zuul | 01:27 | |
| *** jamesmcarthur has quit IRC | 01:27 | |
| *** jamesmcarthur has joined #zuul | 01:28 | |
| *** rlandy|ruck has quit IRC | 01:38 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: WIP: Support Ansible 2.8 https://review.openstack.org/631933 | 01:41 |
| *** jamesmcarthur has quit IRC | 01:51 | |
| *** jamesmcarthur has joined #zuul | 01:54 | |
| *** jamesmcarthur has quit IRC | 02:08 | |
| *** jamesmcarthur has joined #zuul | 02:11 | |
| *** jamesmcarthur has quit IRC | 02:14 | |
| *** jamesmcarthur has joined #zuul | 02:14 | |
| *** jamesmcarthur has quit IRC | 02:17 | |
| *** jamesmcarthur has joined #zuul | 02:22 | |
| *** saneax has joined #zuul | 02:27 | |
| *** jamesmcarthur has quit IRC | 03:00 | |
| *** jamesmcarthur has joined #zuul | 03:01 | |
| *** bhavikdbavishi has joined #zuul | 03:10 | |
| *** bhavikdbavishi1 has joined #zuul | 03:13 | |
| *** bhavikdbavishi has quit IRC | 03:15 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:15 | |
| *** jamesmcarthur has quit IRC | 03:20 | |
| *** jamesmcarthur has joined #zuul | 03:21 | |
| *** jamesmcarthur has quit IRC | 04:07 | |
| *** saneax has quit IRC | 04:31 | |
| *** jamesmcarthur has joined #zuul | 04:38 | |
| *** swest has joined #zuul | 04:40 | |
| *** jamesmcarthur has quit IRC | 04:44 | |
| *** swest has quit IRC | 04:44 | |
| *** swest has joined #zuul | 04:44 | |
| tristanC | tobiash: iirc you are using multi-slot static host, how do you manage the prepare-workspace for those? | 04:55 |
| tobiash | We don't use this. But I think you'd need to prefix the workspace | 04:56 |
| tristanC | tobiash: thanks. so how are suppose to cleanup left-over workspaces if the job is cancelled? | 05:00 |
| tristanC | are we*, i guess we also need to setup a cron job to remove old workspace... | 05:01 |
| *** jamesmcarthur has joined #zuul | 05:07 | |
| *** jamesmcarthur has quit IRC | 05:17 | |
| *** jamesmcarthur has joined #zuul | 05:48 | |
| *** jamesmcarthur has quit IRC | 05:54 | |
| *** pwhalen has quit IRC | 06:06 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: docs: fix a typo in executor zone documentation https://review.openstack.org/653663 | 06:07 |
| *** pwhalen has joined #zuul | 06:12 | |
| *** pwhalen has quit IRC | 06:36 | |
| *** saneax has joined #zuul | 06:42 | |
| *** jamesmcarthur has joined #zuul | 06:50 | |
| *** pwhalen has joined #zuul | 06:54 | |
| *** jamesmcarthur has quit IRC | 06:55 | |
| *** pwhalen has quit IRC | 06:55 | |
| *** gtema has joined #zuul | 06:56 | |
| *** saneax has quit IRC | 06:58 | |
| *** ianw is now known as ianw_pto | 07:15 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool master: static: add host-checking toggle https://review.openstack.org/653679 | 07:29 |
| *** pcaruana has joined #zuul | 07:32 | |
| *** jamesmcarthur has joined #zuul | 07:51 | |
| *** jamesmcarthur has quit IRC | 07:56 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add web route and rpclistener https://review.openstack.org/554839 | 08:52 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build button to trigger job https://review.openstack.org/635716 | 08:52 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add support for parameterized trigger https://review.openstack.org/644484 | 08:52 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build modal with a parameter form https://review.openstack.org/644485 | 08:52 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add support for checkbox and list parameters https://review.openstack.org/648661 | 08:52 |
| *** jamesmcarthur has joined #zuul | 08:52 | |
| *** jamesmcarthur has quit IRC | 08:57 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add web route and rpclistener https://review.openstack.org/554839 | 09:06 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build button to trigger job https://review.openstack.org/635716 | 09:06 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add support for parameterized trigger https://review.openstack.org/644484 | 09:06 |
| *** gtema has quit IRC | 09:17 | |
| *** jamesmcarthur has joined #zuul | 09:26 | |
| *** jamesmcarthur has quit IRC | 09:30 | |
| *** electrofelix has joined #zuul | 09:31 | |
| *** gtema has joined #zuul | 10:15 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 10:26 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Allow operator to generate auth tokens through the CLI https://review.openstack.org/636197 | 10:26 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Zuul CLI: allow access via REST https://review.openstack.org/636315 | 10:26 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Add Authorization Rules configuration https://review.openstack.org/639855 | 10:26 |
| *** jamesmcarthur has joined #zuul | 10:26 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Web: plug the authorization engine https://review.openstack.org/640884 | 10:27 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Zuul Web: add /api/user/authorizations endpoint https://review.openstack.org/641099 | 10:27 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul-jobs master: Support ssh-enabled windows hosts in add-build-sshkey https://review.openstack.org/653712 | 10:27 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: authentication config: add optional token_expiry https://review.openstack.org/642408 | 10:27 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [DNM] admin REST API: docker-compose PoC, frontend https://review.openstack.org/643536 | 10:27 |
| *** jamesmcarthur has quit IRC | 10:31 | |
| *** ttx has quit IRC | 10:31 | |
| *** ttx has joined #zuul | 10:31 | |
| *** pcaruana has quit IRC | 10:36 | |
| *** pcaruana has joined #zuul | 10:58 | |
| *** panda is now known as panda|lunch | 11:26 | |
| *** jamesmcarthur has joined #zuul | 11:27 | |
| *** jamesmcarthur has quit IRC | 11:32 | |
| *** jamesmcarthur has joined #zuul | 12:15 | |
| *** bhavikdbavishi1 has joined #zuul | 12:25 | |
| *** bhavikdbavishi has quit IRC | 12:25 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 12:26 | |
| *** rlandy|ruck has joined #zuul | 12:34 | |
| *** hashar has joined #zuul | 12:53 | |
| *** snapiri has quit IRC | 13:08 | |
| *** pcaruana has quit IRC | 13:10 | |
| *** jamesmcarthur has quit IRC | 13:13 | |
| *** pcaruana has joined #zuul | 13:32 | |
| *** jamesmcarthur has joined #zuul | 13:59 | |
| *** rlandy|ruck is now known as rlandy|ruck|mtg | 14:02 | |
| *** gtema has quit IRC | 14:22 | |
| pabelanger | tobiash: what do you think the next steps are for 2.8 support in zuul? Are we just waiting for the release at this point? | 14:29 |
| tobiash | pabelanger: if tests are green, yes | 14:32 |
| pabelanger | tobiash: ack | 14:32 |
| pabelanger | tobiash: I'm going to to remove the WIP label and clean up commit message, testing should be good now | 14:33 |
| tobiash | :) | 14:35 |
| pabelanger | tobiash: there was a change to zuul_console that will need some reviews | 14:39 |
| *** yolanda_ has quit IRC | 14:40 | |
| *** rlandy|ruck|mtg is now known as rlandy|ruck | 15:18 | |
| *** hashar has quit IRC | 15:20 | |
| *** hashar has joined #zuul | 15:21 | |
| *** sshnaidm has quit IRC | 15:27 | |
| *** pcaruana has quit IRC | 15:31 | |
| *** sshnaidm has joined #zuul | 15:38 | |
| Shrews | zuul tox jobs seem to be failing in a spectacularly random fashion | 15:43 |
| clarkb | Shrews: yes I looked briefly a couple days ago before getting sucked into the opendev git inception and there are failures due to zk slowness but also races in the fs quota checker test and possibly others | 15:43 |
| *** mattw4 has joined #zuul | 15:52 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Support Ansible 2.8 https://review.openstack.org/631933 | 15:54 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Remove support for ansible 2.5 https://review.openstack.org/650431 | 15:54 |
| pabelanger | okay, I think ^ is ready for some reviews, even though 2.8 isn't released yet | 15:59 |
| pabelanger | I think we are in pretty good spot to maybe do a release just after 2.8.0 is released | 15:59 |
| pabelanger | so far, the only change I see was around task out, there is new stats which needed us to update zuul_console | 16:00 |
| pabelanger | I also think I might want to help take up the torch to push our log streaming stuff upstream into ansible-2.9, but likely need a refersher on we want it to work | 16:01 |
| *** mattw4 has quit IRC | 16:02 | |
| mordred | pabelanger: we should sync up on that | 16:02 |
| mordred | pabelanger: and also look in to what bcoca has or hasn't done upstream WRT empowering modules to stream results | 16:03 |
| mordred | I know he was working on adding scaffolding for that | 16:03 |
| mordred | I *know* our current approach is not viable to upstream. while I think we could sell the new approach that tobaish recent rebased in a vacuum, if there is a streaming direction in progress we'd probably need to figure out how to make use of it | 16:04 |
| mordred | pabelanger: in any case - I 100% think that's a great idea of a thing to work on | 16:04 |
| pabelanger | ++ | 16:04 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: WIP - Pagure driver - https://pagure.io/pagure/ https://review.openstack.org/604404 | 16:07 |
| *** panda|lunch is now known as panda | 16:11 | |
| SpamapS | clarkb:FYI, kubernetes released v9.0.0 on March 20. I'll be trying it out with latest nodepool from master some time next week. | 16:20 |
| *** hashar has quit IRC | 16:30 | |
| mordred | SpamapS: cool | 16:55 |
| *** pcaruana has joined #zuul | 17:02 | |
| *** jamesmcarthur_ has joined #zuul | 17:16 | |
| *** jamesmcarthur has quit IRC | 17:19 | |
| pabelanger | Hmm, zuul-build-image didn't link logs for some reason: https://review.openstack.org/650431/ | 17:39 |
| *** mattw4 has joined #zuul | 17:41 | |
| *** mattw4 has quit IRC | 18:03 | |
| *** electrofelix has quit IRC | 18:07 | |
| *** jamesmcarthur_ has quit IRC | 18:09 | |
| *** jamesmcarthur has joined #zuul | 18:28 | |
| *** sshnaidm is now known as sshnaidm|off | 18:32 | |
| clarkb | pabelanger: that is usually the ipv6 issue | 18:47 |
| clarkb | I don't know why the logs aren't linked though | 18:47 |
| *** jangutter has joined #zuul | 19:15 | |
| *** hashar has joined #zuul | 19:23 | |
| clarkb | SpamapS' email re blcak on the openstack -discuss list has me quickly check what the initial conversion cost for zuul would be: 197 files changed, 24047 insertions(+), 18233 deletions(-) that is with -l 79 and syntax compat with py35. So ya not small. But I do like the idea of easier to review diffs and fewer merge conflicts going forward | 19:29 |
| clarkb | and as expected pep8 does not pass so it is an either or | 19:30 |
| *** jangutter has quit IRC | 19:49 | |
| *** jamesmcarthur has quit IRC | 19:58 | |
| *** jangutter has joined #zuul | 20:04 | |
| *** jangutter has quit IRC | 20:09 | |
| SpamapS | clarkb: Yeah, it seems big, but it's a one time cost, and after that, it's even less intrusive than pep8/hacking/etc. | 20:14 |
| SpamapS | And the git hook thing is just one way. There are vim plugins, vscode, pycharm, etc. .. so generally the formatting "just works". | 20:16 |
| Shrews | In 24 hours, nodepool-launcher has steadily increased its memory usage by 4%. This makes me sad. | 20:17 |
| Shrews | used an additional 4% of total system memory, that is | 20:23 |
| *** jamesmcarthur has joined #zuul | 20:24 | |
| SpamapS | time to print out the objdump ;) | 20:27 |
| Shrews | yeah, i need to ask corvus how he usually does that with zuul. | 20:30 |
| corvus | you're not going to like it | 20:31 |
| corvus | Shrews: https://review.openstack.org/579962 | 20:31 |
| SpamapS | Shrews loves papering his floor with object graphs. | 20:31 |
| corvus | i added that to zuul manually, restarted, waited for things to get bad, then manually ran commands to write out data to a file | 20:32 |
| corvus | (i actually have the necessary license clearances to clean that up and merge it for real if we want it -- it's just been very low priority) | 20:32 |
| Shrews | i'm not quite sure that that does, but will be fun to learn, i'm sure | 20:33 |
| corvus | Shrews: adds a repl (read-eval-print-loop) to zuul. so i could enter python commands to inspect the memory state and use objgraph to dump it to a file | 20:34 |
| Shrews | corvus: you are correct. i don't like it | 20:35 |
| *** jamesmcarthur has quit IRC | 20:35 | |
| * Shrews cries a little | 20:35 | |
| corvus | yeah. it was a bit of an emergency situation (with zuul not able to run for even a day). | 20:36 |
| corvus | Shrews: you can start by doing the siguser objgraph dumps, which print out object counts and deltas | 20:36 |
| Shrews | corvus: if objgraph is installed on the system, does that slow things down? If so, did you manually install/remove that lib? | 20:36 |
| corvus | Shrews: so you can see if there's a class of objects that continues to increase | 20:37 |
| corvus | Shrews: nope, no slowdown | 20:37 |
| corvus | so it's safe to always have installed, and it's safe to print out the deltas as many times as you want | 20:37 |
| Shrews | that's not installed on our launchers, it seems. | 20:37 |
| corvus | yeah, we'll probably have to add it to puppet | 20:38 |
| Shrews | good to hear it doesn't affect performance tho | 20:38 |
| corvus | Shrews: so there's two things objgraph does that can help. the first is the object counts i mentioned | 20:39 |
| corvus | Shrews: the second is actually generating a diagram of object references to each other. if there's a loop, or otherwise references being retained that shouldn't, that's usually how you find it | 20:40 |
| corvus | Shrews: the sigusr2 stuff just does the object counts | 20:40 |
| corvus | i did the graph using the repl; and that's what SpamapS was referring to | 20:40 |
| corvus | xdot, however, is a great alternative to printing them out :) | 20:41 |
| Shrews | yeah, i'm not hopeful of the first thing finding the problem. If it is the cache, that's just a dict object. | 20:41 |
| corvus | Shrews: https://mg.pov.lt/objgraph/ is really good | 20:42 |
| corvus | i'm not opposed to adding something to spit out dot files on a signal or some other thing, it's just that usually i don't know exactly what to call objgraph.show_backrefs() with, so it takes a few tries. that's where the repl came in. | 20:43 |
| SpamapS | corvus:btw, they make super wide monitors now :) | 20:43 |
| Shrews | alright, got a plan of action. 1. puppet objgraph, 2. restart launchers, 3. run for a while, 4. get the obj count, 5. profit or come up with step 6 | 20:44 |
| SpamapS | I'm on a pretty nice 34" ultra wide.. definitely helps with those types of things | 20:44 |
| corvus | SpamapS: yeah, xdot can use every bit of that :) | 20:45 |
| corvus | Shrews: ++ | 20:45 |
| corvus | also, xdot makes me want to say "It's an object memory graph! I know this!" | 20:46 |
| *** pcaruana has quit IRC | 20:47 | |
| clarkb | I'm going to have to rewatch jurassic park now | 20:47 |
| *** jamesmcarthur has joined #zuul | 20:48 | |
| corvus | clarkb: it's been like 2 weeks right? way too long | 20:48 |
| Shrews | linux... letting your aberrations of science free since 1993 | 20:48 |
| clarkb | corvus: maybe 6 months and ya too long | 20:49 |
| clarkb | SpamapS: ya I'm actually pro black usage, just not to the point where I'd fight for it | 20:50 |
| clarkb | SpamapS: I really like the idea of avoiding merge conflicts | 20:50 |
| clarkb | I had a PR up to pip that languished forever in part because I had to rebase it every other day because people would do things like add or remove a comma in a list | 20:50 |
| *** jamesmcarthur has quit IRC | 20:59 | |
| *** hashar has quit IRC | 21:01 | |
| *** jamesmcarthur has joined #zuul | 21:02 | |
| *** jamesmcarthur has quit IRC | 21:29 | |
| *** jamesmcarthur has joined #zuul | 21:30 | |
| *** jamesmcarthur has quit IRC | 21:43 | |
| *** rlandy|ruck is now known as rlandy | 21:57 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add SANs to buildset registry cert https://review.openstack.org/653869 | 22:16 |
| SpamapS | clarkb:I also like the idea of 88 chars ;) | 22:59 |
| mordred | SpamapS: I violently do not | 23:09 |
| mordred | it's the biggest flaw with black, which is otherwise pretty much everything I've ever wanted for this topic - but that part is unworkably bad | 23:10 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/nodepool master: Fix AWS provider ignoring region in providers https://review.openstack.org/649423 | 23:17 |
| *** mattw4 has joined #zuul | 23:49 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!