corvus | (that change runs six jobs, and since it fails in pre, we have to burn through 18 failures till my job runs) | 00:00 |
---|---|---|
*** jamesmcarthur has quit IRC | 00:01 | |
*** jamesmcarthur has joined #zuul | 00:01 | |
*** jamesmcarthur has quit IRC | 00:06 | |
*** tosky has quit IRC | 00:10 | |
mordred | corvus: oh no - SORRY | 00:10 |
mordred | corvus: I'm too used to throwing stuff at opendev early | 00:11 |
*** zbr3 has joined #zuul | 00:13 | |
*** jamesmcarthur has joined #zuul | 00:13 | |
corvus | mordred: you seem to have stress-tested gcloud too -- it seems like it was having trouble cycling instances that fast. i think maybe we need to increase the boot timeout... | 00:15 |
*** zbr has quit IRC | 00:15 | |
*** zbr3 is now known as zbr | 00:15 | |
mordred | I continue to be the cloud wrecker | 00:15 |
fungi | i hope that's your pro wrestling moniker, and that you have a suitable luchador mask | 00:19 |
corvus | i wonder if public ip addrs have anything to do with it, and if setting up "google cloud nat" would be helpful... | 00:19 |
*** harrymichal has quit IRC | 00:22 | |
corvus | mordred: remote: https://gerrit-review.googlesource.com/c/zuul/ops/+/297462 Increase the nodepool boot timeout [NEW] | 00:23 |
clarkb | corvus: do we need to record which shas we restarted on? I suppose we can remember "noop dockerfile update" in zuul and "require tls" in nodepool | 00:24 |
corvus | good point i'll do that | 00:24 |
corvus | mordred: i made a job to grab a cache, but it was too big for the executor; i'm not actually sure how large it is, working on that now. | 00:27 |
corvus | 529MB (limit=500) :( | 00:28 |
corvus | i'm actually not sure how big the disks are on the k8s cluster | 00:28 |
mordred | corvus: eek. you might have to do direct push from build node? | 00:29 |
fungi | or batches, but that could get ugly too | 00:30 |
*** zbr6 has joined #zuul | 00:31 | |
*** zbr has quit IRC | 00:33 | |
*** zbr6 is now known as zbr | 00:33 | |
corvus | /dev/sda1 95G 6.6G 88G 7% /var/lib/zuul | 00:34 |
corvus | i think maybe we can bump up the limit a bit :) | 00:34 |
*** holser has quit IRC | 00:34 | |
corvus | that way we can avoid pushing from the build node (i think that would be less secure) | 00:35 |
corvus | mordred: remote: https://gerrit-review.googlesource.com/c/zuul/ops/+/297463 Increase disk limit per job [NEW] | 00:39 |
openstackgerrit | James E. Blair proposed zuul/nodepool master: Add ssh/winrm credentials to static nodes https://review.opendev.org/c/zuul/nodepool/+/774014 | 00:43 |
mordred | corvus: ok - I have the whole thing translated into python. I have not tested anything so I won't push it up for now | 00:44 |
mordred | corvus: http://paste.openstack.org/show/802851/ though - fwiw | 00:46 |
corvus | mordred: w00t! you could comment out all the jobs except test-gerrit-setup | 00:48 |
mordred | ooh - good point | 00:48 |
corvus | also, note that right now i have performed my own dos by doing a change which requires a full reconfiguration :) | 00:49 |
*** zbr3 has joined #zuul | 00:51 | |
mordred | \o/ | 00:53 |
*** zbr has quit IRC | 00:54 | |
*** zbr3 is now known as zbr | 00:54 | |
mordred | corvus: I pushed the latest up - in case anyone else wants to hack on it before I get back to poking at it | 00:54 |
mordred | and now I will dinner | 00:55 |
corvus | mordred: thanks! | 00:55 |
avass | mordred, corvus: the benefit of having a monorepo I guess is that we don't really experience the ansible overhead from looping over a ton of repos :) | 00:56 |
avass | instead our push/checkout is horrible | 00:56 |
corvus | avass: yeah, big data is big data either way | 00:57 |
*** zbr0 has joined #zuul | 01:00 | |
*** zbr has quit IRC | 01:02 | |
*** zbr0 is now known as zbr | 01:02 | |
mordred | Except when you misplace it | 01:07 |
*** zbr9 has joined #zuul | 01:25 | |
*** zbr has quit IRC | 01:27 | |
*** zbr9 is now known as zbr | 01:27 | |
*** sduthil has quit IRC | 01:33 | |
*** zbr7 has joined #zuul | 01:44 | |
*** saneax has quit IRC | 01:45 | |
*** zbr has quit IRC | 01:46 | |
*** zbr7 is now known as zbr | 01:46 | |
*** zbr5 has joined #zuul | 02:05 | |
*** piotrowskim has quit IRC | 02:05 | |
*** zbr has quit IRC | 02:06 | |
*** zbr5 is now known as zbr | 02:06 | |
*** zbr2 has joined #zuul | 02:27 | |
*** zbr has quit IRC | 02:28 | |
*** zbr2 is now known as zbr | 02:28 | |
*** zbr7 has joined #zuul | 02:50 | |
*** zbr has quit IRC | 02:52 | |
*** zbr7 is now known as zbr | 02:52 | |
*** EmilienM has quit IRC | 03:05 | |
*** zbr4 has joined #zuul | 03:10 | |
*** zbr has quit IRC | 03:13 | |
*** zbr4 is now known as zbr | 03:13 | |
*** iurygregory has quit IRC | 04:10 | |
*** zbr4 has joined #zuul | 05:04 | |
*** zbr has quit IRC | 05:06 | |
*** zbr4 is now known as zbr | 05:06 | |
*** wuchunyang has joined #zuul | 05:34 | |
*** evrardjp has quit IRC | 05:38 | |
*** evrardjp has joined #zuul | 05:39 | |
*** zbr3 has joined #zuul | 05:54 | |
*** zbr has quit IRC | 05:56 | |
*** zbr3 is now known as zbr | 05:56 | |
*** zbr3 has joined #zuul | 05:58 | |
*** zbr has quit IRC | 06:00 | |
*** zbr3 is now known as zbr | 06:00 | |
*** icey has quit IRC | 06:04 | |
*** icey has joined #zuul | 06:04 | |
*** zbr2 has joined #zuul | 06:34 | |
*** zbr has quit IRC | 06:36 | |
*** zbr2 is now known as zbr | 06:36 | |
*** jamesmcarthur has quit IRC | 06:51 | |
*** jamesmcarthur has joined #zuul | 06:51 | |
*** jamesmcarthur has quit IRC | 06:55 | |
*** zbr7 has joined #zuul | 06:57 | |
*** zbr has quit IRC | 06:59 | |
*** zbr7 is now known as zbr | 06:59 | |
*** zbr8 has joined #zuul | 07:22 | |
*** zbr has quit IRC | 07:24 | |
*** zbr8 is now known as zbr | 07:24 | |
*** zbr7 has joined #zuul | 07:31 | |
*** zbr has quit IRC | 07:33 | |
*** zbr7 is now known as zbr | 07:33 | |
*** zbr1 has joined #zuul | 07:57 | |
*** zbr has quit IRC | 07:59 | |
*** zbr1 is now known as zbr | 07:59 | |
*** harrymichal has joined #zuul | 08:15 | |
*** harrymichal has quit IRC | 08:20 | |
*** zbr5 has joined #zuul | 08:21 | |
*** zbr has quit IRC | 08:22 | |
*** zbr5 is now known as zbr | 08:22 | |
*** zbr4 has joined #zuul | 08:55 | |
*** zbr has quit IRC | 08:58 | |
*** zbr4 is now known as zbr | 08:58 | |
*** zbr has quit IRC | 09:05 | |
*** zbr has joined #zuul | 09:05 | |
*** zbr7 has joined #zuul | 09:08 | |
*** zbr has quit IRC | 09:10 | |
*** zbr7 is now known as zbr | 09:10 | |
*** nils has joined #zuul | 09:16 | |
*** harrymichal has joined #zuul | 09:30 | |
*** harrymichal has quit IRC | 09:30 | |
*** zbr4 has joined #zuul | 09:54 | |
*** zbr has quit IRC | 09:56 | |
*** zbr4 is now known as zbr | 09:56 | |
*** zbr6 has joined #zuul | 10:08 | |
*** zbr has quit IRC | 10:10 | |
*** zbr6 is now known as zbr | 10:10 | |
*** tosky has joined #zuul | 10:13 | |
avass | maybe it would be a good idea of showing play duration in the console tab (that's already oart of the job-output-json) and also log and show playbook duration | 10:25 |
*** wuchunyang has quit IRC | 10:41 | |
openstackgerrit | Albin Vass proposed zuul/zuul master: WIP: Display play duration in console tab https://review.opendev.org/c/zuul/zuul/+/776802 | 10:43 |
avass | something like that ^ someone else can probably make that look nicer :) | 10:44 |
avass | also I noticed that the play durations doesn't add up to the total duration. I guess that's because any cleanup isn't logged or because of time spent between playbooks | 10:52 |
*** noonedeadpunk has quit IRC | 11:16 | |
*** noonedeadpunk has joined #zuul | 11:17 | |
*** zbr6 has joined #zuul | 11:59 | |
*** zbr has quit IRC | 12:00 | |
*** zbr6 is now known as zbr | 12:00 | |
*** zbr4 has joined #zuul | 12:07 | |
*** zbr has quit IRC | 12:09 | |
*** zbr4 is now known as zbr | 12:09 | |
*** zbr9 has joined #zuul | 12:28 | |
*** zbr has quit IRC | 12:29 | |
*** zbr9 is now known as zbr | 12:29 | |
*** iurygregory has joined #zuul | 12:51 | |
*** zbr6 has joined #zuul | 13:21 | |
*** zbr has quit IRC | 13:22 | |
*** zbr6 is now known as zbr | 13:22 | |
*** zbr6 has joined #zuul | 13:46 | |
*** zbr has quit IRC | 13:48 | |
*** zbr6 is now known as zbr | 13:48 | |
*** zbr6 has joined #zuul | 13:55 | |
*** zbr has quit IRC | 13:56 | |
*** zbr6 is now known as zbr | 13:56 | |
*** zbr has quit IRC | 14:01 | |
*** zbr has joined #zuul | 14:01 | |
*** zbr3 has joined #zuul | 14:15 | |
*** zbr has quit IRC | 14:17 | |
*** zbr3 is now known as zbr | 14:17 | |
*** zbr5 has joined #zuul | 15:04 | |
*** zbr has quit IRC | 15:06 | |
*** zbr5 is now known as zbr | 15:06 | |
*** tosky has quit IRC | 15:35 | |
corvus | mordred: neat. the disk cache and http cache have a different layout. disk cache has sharded directories, web cache doesn't | 16:08 |
corvus | okay with the remote http cache, it had 408 out of about 900 tasks hit the remote cache; runtime was 160 seconds instead of 370. | 16:42 |
corvus | it doesn't sound like it's the lowest hanging fruit here. but at least we know now. | 16:44 |
*** zbr2 has joined #zuul | 17:20 | |
*** zbr has quit IRC | 17:22 | |
*** zbr has joined #zuul | 17:24 | |
*** zbr2 has quit IRC | 17:27 | |
mordred | corvus: yeah - good to know numbers | 17:50 |
mordred | corvus: also - of course the layouts are different :) | 17:50 |
*** nils has quit IRC | 18:55 | |
openstackgerrit | James E. Blair proposed zuul/nodepool master: Add ssh/winrm credentials to static nodes https://review.opendev.org/c/zuul/nodepool/+/774014 | 19:01 |
openstackgerrit | James E. Blair proposed zuul/zuul master: Support credentials supplied by nodepool https://review.opendev.org/c/zuul/zuul/+/774362 | 19:03 |
*** ikhan has joined #zuul | 19:51 | |
*** tosky has joined #zuul | 20:21 | |
*** jamesmcarthur has joined #zuul | 20:25 | |
*** ikhan has quit IRC | 21:50 | |
*** ikhan has joined #zuul | 22:17 | |
*** jamesmcarthur has quit IRC | 23:00 | |
*** jamesmcarthur_ has joined #zuul | 23:00 | |
*** EmilienM has joined #zuul | 23:22 | |
*** jamesmcarthur_ has quit IRC | 23:52 | |
*** jamesmcarthur has joined #zuul | 23:52 | |
*** jamesmcarthur has quit IRC | 23:57 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!