Saturday, 2021-02-20

corvus(that change runs six jobs, and since it fails in pre, we have to burn through 18 failures till my job runs)00:00
*** jamesmcarthur has quit IRC00:01
*** jamesmcarthur has joined #zuul00:01
*** jamesmcarthur has quit IRC00:06
*** tosky has quit IRC00:10
mordredcorvus: oh no - SORRY00:10
mordredcorvus: I'm too used to throwing stuff at opendev early00:11
*** zbr3 has joined #zuul00:13
*** jamesmcarthur has joined #zuul00:13
corvusmordred: you seem to have stress-tested gcloud too -- it seems like it was having trouble cycling instances that fast.  i think maybe we need to increase the boot timeout...00:15
*** zbr has quit IRC00:15
*** zbr3 is now known as zbr00:15
mordredI continue to be the cloud wrecker00:15
fungii hope that's your pro wrestling moniker, and that you have a suitable luchador mask00:19
corvusi wonder if public ip addrs have anything to do with it, and if setting up "google cloud nat" would be helpful...00:19
*** harrymichal has quit IRC00:22
corvusmordred: remote:   https://gerrit-review.googlesource.com/c/zuul/ops/+/297462 Increase the nodepool boot timeout [NEW]00:23
clarkbcorvus: do we need to record which shas we restarted on? I suppose we can remember "noop dockerfile update" in zuul and "require tls" in nodepool00:24
corvusgood point i'll do that00:24
corvusmordred: i made a job to grab a cache, but it was too big for the executor; i'm not actually sure how large it is, working on that now.00:27
corvus529MB (limit=500)   :(00:28
corvusi'm actually not sure how big the disks are on the k8s cluster00:28
mordredcorvus: eek. you might have to do direct push from build node?00:29
fungior batches, but that could get ugly too00:30
*** zbr6 has joined #zuul00:31
*** zbr has quit IRC00:33
*** zbr6 is now known as zbr00:33
corvus/dev/sda1        95G  6.6G   88G   7% /var/lib/zuul00:34
corvusi think maybe we can bump up the limit a bit :)00:34
*** holser has quit IRC00:34
corvusthat way we can avoid pushing from the build node (i think that would be less secure)00:35
corvusmordred: remote:   https://gerrit-review.googlesource.com/c/zuul/ops/+/297463 Increase disk limit per job [NEW]00:39
openstackgerritJames E. Blair proposed zuul/nodepool master: Add ssh/winrm credentials to static nodes  https://review.opendev.org/c/zuul/nodepool/+/77401400:43
mordredcorvus: ok - I have the whole thing translated into python. I have not tested anything so I won't push it up for now00:44
mordredcorvus: http://paste.openstack.org/show/802851/ though - fwiw00:46
corvusmordred: w00t!  you could comment out all the jobs except test-gerrit-setup00:48
mordredooh - good point00:48
corvusalso, note that right now i have performed my own dos by doing a change which requires a full reconfiguration :)00:49
*** zbr3 has joined #zuul00:51
mordred\o/00:53
*** zbr has quit IRC00:54
*** zbr3 is now known as zbr00:54
mordredcorvus: I pushed the latest up  - in case anyone else wants to hack on it before I get back to poking at it00:54
mordredand now I will dinner00:55
corvusmordred: thanks!00:55
avassmordred, corvus: the benefit of having a monorepo I guess is that we don't really experience the ansible overhead from looping over a ton of repos :)00:56
avassinstead our push/checkout is horrible00:56
corvusavass: yeah, big data is big data either way00:57
*** zbr0 has joined #zuul01:00
*** zbr has quit IRC01:02
*** zbr0 is now known as zbr01:02
mordredExcept when you misplace it01:07
*** zbr9 has joined #zuul01:25
*** zbr has quit IRC01:27
*** zbr9 is now known as zbr01:27
*** sduthil has quit IRC01:33
*** zbr7 has joined #zuul01:44
*** saneax has quit IRC01:45
*** zbr has quit IRC01:46
*** zbr7 is now known as zbr01:46
*** zbr5 has joined #zuul02:05
*** piotrowskim has quit IRC02:05
*** zbr has quit IRC02:06
*** zbr5 is now known as zbr02:06
*** zbr2 has joined #zuul02:27
*** zbr has quit IRC02:28
*** zbr2 is now known as zbr02:28
*** zbr7 has joined #zuul02:50
*** zbr has quit IRC02:52
*** zbr7 is now known as zbr02:52
*** EmilienM has quit IRC03:05
*** zbr4 has joined #zuul03:10
*** zbr has quit IRC03:13
*** zbr4 is now known as zbr03:13
*** iurygregory has quit IRC04:10
*** zbr4 has joined #zuul05:04
*** zbr has quit IRC05:06
*** zbr4 is now known as zbr05:06
*** wuchunyang has joined #zuul05:34
*** evrardjp has quit IRC05:38
*** evrardjp has joined #zuul05:39
*** zbr3 has joined #zuul05:54
*** zbr has quit IRC05:56
*** zbr3 is now known as zbr05:56
*** zbr3 has joined #zuul05:58
*** zbr has quit IRC06:00
*** zbr3 is now known as zbr06:00
*** icey has quit IRC06:04
*** icey has joined #zuul06:04
*** zbr2 has joined #zuul06:34
*** zbr has quit IRC06:36
*** zbr2 is now known as zbr06:36
*** jamesmcarthur has quit IRC06:51
*** jamesmcarthur has joined #zuul06:51
*** jamesmcarthur has quit IRC06:55
*** zbr7 has joined #zuul06:57
*** zbr has quit IRC06:59
*** zbr7 is now known as zbr06:59
*** zbr8 has joined #zuul07:22
*** zbr has quit IRC07:24
*** zbr8 is now known as zbr07:24
*** zbr7 has joined #zuul07:31
*** zbr has quit IRC07:33
*** zbr7 is now known as zbr07:33
*** zbr1 has joined #zuul07:57
*** zbr has quit IRC07:59
*** zbr1 is now known as zbr07:59
*** harrymichal has joined #zuul08:15
*** harrymichal has quit IRC08:20
*** zbr5 has joined #zuul08:21
*** zbr has quit IRC08:22
*** zbr5 is now known as zbr08:22
*** zbr4 has joined #zuul08:55
*** zbr has quit IRC08:58
*** zbr4 is now known as zbr08:58
*** zbr has quit IRC09:05
*** zbr has joined #zuul09:05
*** zbr7 has joined #zuul09:08
*** zbr has quit IRC09:10
*** zbr7 is now known as zbr09:10
*** nils has joined #zuul09:16
*** harrymichal has joined #zuul09:30
*** harrymichal has quit IRC09:30
*** zbr4 has joined #zuul09:54
*** zbr has quit IRC09:56
*** zbr4 is now known as zbr09:56
*** zbr6 has joined #zuul10:08
*** zbr has quit IRC10:10
*** zbr6 is now known as zbr10:10
*** tosky has joined #zuul10:13
avassmaybe it would be a good idea of showing play duration in the console tab (that's already oart of the job-output-json) and also log and show playbook duration10:25
*** wuchunyang has quit IRC10:41
openstackgerritAlbin Vass proposed zuul/zuul master: WIP: Display play duration in console tab  https://review.opendev.org/c/zuul/zuul/+/77680210:43
avasssomething like that ^ someone else can probably make that look nicer :)10:44
avassalso I noticed that the play durations doesn't add up to the total duration. I guess that's because any cleanup isn't logged or because of time spent between playbooks10:52
*** noonedeadpunk has quit IRC11:16
*** noonedeadpunk has joined #zuul11:17
*** zbr6 has joined #zuul11:59
*** zbr has quit IRC12:00
*** zbr6 is now known as zbr12:00
*** zbr4 has joined #zuul12:07
*** zbr has quit IRC12:09
*** zbr4 is now known as zbr12:09
*** zbr9 has joined #zuul12:28
*** zbr has quit IRC12:29
*** zbr9 is now known as zbr12:29
*** iurygregory has joined #zuul12:51
*** zbr6 has joined #zuul13:21
*** zbr has quit IRC13:22
*** zbr6 is now known as zbr13:22
*** zbr6 has joined #zuul13:46
*** zbr has quit IRC13:48
*** zbr6 is now known as zbr13:48
*** zbr6 has joined #zuul13:55
*** zbr has quit IRC13:56
*** zbr6 is now known as zbr13:56
*** zbr has quit IRC14:01
*** zbr has joined #zuul14:01
*** zbr3 has joined #zuul14:15
*** zbr has quit IRC14:17
*** zbr3 is now known as zbr14:17
*** zbr5 has joined #zuul15:04
*** zbr has quit IRC15:06
*** zbr5 is now known as zbr15:06
*** tosky has quit IRC15:35
corvusmordred: neat.  the disk cache and http cache have a different layout.  disk cache has sharded directories, web cache doesn't16:08
corvusokay with the remote http cache, it had 408 out of about 900 tasks hit the remote cache; runtime was 160 seconds instead of 370.16:42
corvusit doesn't sound like it's the lowest hanging fruit here.  but at least we know now.16:44
*** zbr2 has joined #zuul17:20
*** zbr has quit IRC17:22
*** zbr has joined #zuul17:24
*** zbr2 has quit IRC17:27
mordredcorvus: yeah - good to know numbers17:50
mordredcorvus: also - of course the layouts are different :)17:50
*** nils has quit IRC18:55
openstackgerritJames E. Blair proposed zuul/nodepool master: Add ssh/winrm credentials to static nodes  https://review.opendev.org/c/zuul/nodepool/+/77401419:01
openstackgerritJames E. Blair proposed zuul/zuul master: Support credentials supplied by nodepool  https://review.opendev.org/c/zuul/zuul/+/77436219:03
*** ikhan has joined #zuul19:51
*** tosky has joined #zuul20:21
*** jamesmcarthur has joined #zuul20:25
*** ikhan has quit IRC21:50
*** ikhan has joined #zuul22:17
*** jamesmcarthur has quit IRC23:00
*** jamesmcarthur_ has joined #zuul23:00
*** EmilienM has joined #zuul23:22
*** jamesmcarthur_ has quit IRC23:52
*** jamesmcarthur has joined #zuul23:52
*** jamesmcarthur has quit IRC23:57

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!