| *** rfolco has quit IRC | 00:37 | |
| *** igordc has quit IRC | 01:18 | |
| *** bhavikdbavishi has joined #zuul | 02:42 | |
| *** bhavikdbavishi1 has joined #zuul | 02:45 | |
| *** bhavikdbavishi has quit IRC | 02:46 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 02:46 | |
| *** jamesmcarthur has joined #zuul | 04:01 | |
| *** jamesmcarthur has quit IRC | 04:05 | |
| flaper87 | do any of you have links to Zuul presentations about the load it can manage, the architecture, etc? I think there was one from corvus in LCA | 04:55 |
|---|---|---|
| * flaper87 keeps googling | 04:55 | |
| flaper87 | Or any presentation that you find interesting and useful to share with people that are completely new to Zuul! | 04:56 |
| *** wxy-xiyuan has quit IRC | 05:02 | |
| *** bhavikdbavishi has quit IRC | 05:05 | |
| *** yolanda has quit IRC | 05:25 | |
| *** bhavikdbavishi has joined #zuul | 05:35 | |
| *** bhavikdbavishi has quit IRC | 06:05 | |
| *** yolanda has joined #zuul | 07:17 | |
| *** armstrongs has joined #zuul | 08:54 | |
| *** wxy-xiyuan has joined #zuul | 09:08 | |
| *** noorul has joined #zuul | 09:33 | |
| *** bhavikdbavishi has joined #zuul | 09:35 | |
| *** noorul has quit IRC | 09:56 | |
| *** noorul has joined #zuul | 10:12 | |
| *** tosky has joined #zuul | 10:14 | |
| *** noorul has quit IRC | 10:18 | |
| *** wxy-xiyuan has quit IRC | 11:38 | |
| *** bhavikdbavishi has quit IRC | 11:43 | |
| *** noorul has joined #zuul | 11:47 | |
| *** sshnaidm|off has quit IRC | 11:51 | |
| *** noorul has quit IRC | 11:53 | |
| *** noorul has joined #zuul | 12:59 | |
| *** noorul has quit IRC | 13:05 | |
| *** sshnaidm has joined #zuul | 14:16 | |
| *** mattw4 has joined #zuul | 14:50 | |
| *** noorul has joined #zuul | 15:38 | |
| noorul | Is there a nosetest result HTML publisher in zuul ? | 15:39 |
| *** tosky has quit IRC | 15:44 | |
| dmsimard | flaper87: there's bound to be a few recorded from previous summits and opendev. They're uploaded on YouTube. | 16:02 |
| *** pcaruana has joined #zuul | 16:13 | |
| *** noorul has quit IRC | 17:07 | |
| *** openstack has joined #zuul | 17:40 | |
| *** ChanServ sets mode: +o openstack | 17:40 | |
| *** pcaruana has joined #zuul | 17:52 | |
| *** saneax has joined #zuul | 17:55 | |
| *** jamesmcarthur has quit IRC | 18:12 | |
| *** pcaruana has quit IRC | 18:40 | |
| *** mattw4 has quit IRC | 18:52 | |
| *** mattw4 has joined #zuul | 18:55 | |
| *** pcaruana has joined #zuul | 18:57 | |
| mnaser | hmm | 18:59 |
| mnaser | i'm noticing a very odd slowdown when Zuul gets to "Run test-setup role" | 18:59 |
| mnaser | its exactly 1m30s | 19:00 |
| mnaser | https://www.irccloud.com/pastebin/5lojYxBp/ | 19:00 |
| clarkb | do you have an expensive tools/test-setup.sh script in the repo? | 19:05 |
| *** sanjayu_ has joined #zuul | 19:09 | |
| *** sanjayu__ has joined #zuul | 19:11 | |
| *** saneax has quit IRC | 19:11 | |
| *** sanjayu_ has quit IRC | 19:13 | |
| *** saneax has joined #zuul | 19:14 | |
| *** altlogbot_1 has quit IRC | 19:16 | |
| *** irclogbot_2 has quit IRC | 19:16 | |
| *** sanjayu__ has quit IRC | 19:16 | |
| *** sanjayu_ has joined #zuul | 19:16 | |
| *** saneax has quit IRC | 19:20 | |
| *** irclogbot_2 has joined #zuul | 19:20 | |
| *** sshnaidm has quit IRC | 19:33 | |
| *** tosky has joined #zuul | 19:41 | |
| *** sshnaidm has joined #zuul | 19:53 | |
| *** sshnaidm has quit IRC | 20:01 | |
| *** sshnaidm has joined #zuul | 20:02 | |
| mnaser | clarkb: nope it actually just skips because there's nothing that happens after | 20:22 |
| mnaser | the include takes a while, but the task itself afterward instantly skips cause there is no test script | 20:22 |
| mnaser | i've noticed ansible-playbook seems to be a defunct process "[ansible-playboo] <defunct>" | 20:22 |
| fungi | playBOO! | 20:23 |
| fungi | it's getting ready for hallowe'en | 20:23 |
| fungi | ghost in the playbook | 20:24 |
| mnaser | lol! | 20:24 |
| mnaser | the parent python process thats running ansible-playbook is at ~53% cpu usage | 20:24 |
| mnaser | eventually it moves on.. | 20:24 |
| fungi | yeah, that definitely sounds... wrong | 20:24 |
| mnaser | but its a whole 1m30s of sitting around doing nothing :( | 20:24 |
| fungi | if you can attach an strace to it (sudo strace -p $pid) it might help to know what it's spinning on | 20:25 |
| mnaser | hmm a lot of of connects | 20:26 |
| mnaser | i should ssh into any of the hosts | 20:26 |
| mnaser | interesting | 20:28 |
| mnaser | nothing is happening inside the hsot | 20:28 |
| mnaser | https://www.irccloud.com/pastebin/xS3Tdc7k/ | 20:28 |
| mnaser | [ 467.639030] TCP: request_sock_TCP: Possible SYN flooding on port 19885. Sending cookies. Check SNMP counters. | 20:28 |
| mnaser | is this tuned inside infra? | 20:30 |
| fungi | that's a kernel feature | 20:30 |
| mnaser | right, but i meant to say if you tune the kernel to avoid this from happening | 20:31 |
| fungi | happens when the kernel receives too many tcp/syn packets too quickly, i think? | 20:31 |
| fungi | probably tunable via sysctl | 20:31 |
| mnaser | (this is on the nodepool vms) | 20:31 |
| fungi | might be something to set in sysctl.conf in node images | 20:31 |
| fungi | not sure if dib does anything special with it, or if there's a targeted dib element for adjusting sysctl.conf | 20:32 |
| mnaser | net.core.message_cost and net.core.message_burst apparently | 20:32 |
| fungi | values might also differ by distro | 20:32 |
| mnaser | hmm | 20:37 |
| mnaser | i was running ansible with python3 | 20:37 |
| mnaser | maybe that had to do with it | 20:37 |
| mnaser | ill try again with py2 instead | 20:37 |
| fungi | if that makes a difference, then we *really* need to track down why | 20:39 |
| fungi | thanks for the info! | 20:39 |
| mnaser | well one of the ansible libs seemed to crash with one of those common py3 "expected bytes got str" thing | 20:39 |
| *** jamesmcarthur has joined #zuul | 20:40 | |
| fungi | ugh | 20:40 |
| mnaser | https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/tox/library/tox_install_sibling_packages.py#L109-L120 | 20:41 |
| fungi | which variable is it complaining about there? x in the returned list comprehension? | 20:43 |
| mnaser | indeed, subprocess.check_output seems to return a string that's prefixed with a b'..' | 20:44 |
| mnaser | fungi: subprocess.check_output(["ps"]).split("\n") will break | 20:44 |
| fungi | ahh | 20:44 |
| mnaser | which is a simplified way of "frozen_pkgs.split('\n')" | 20:44 |
| mnaser | frozen_pkgs.decode().split('\n') might work but im not sure how to make sure that works for b oth py2 and py3 | 20:46 |
| fungi | i think decode() might break if frozen_pkgs is already a str | 20:48 |
| fungi | we could use six's stringtype there, but making sure it's present might be a challenge. or we could guard it with something like if type(frozen_pkgs) is not str: ... | 20:49 |
| fungi | there are probably better ways, i'm far from an expert on this | 20:49 |
| fungi | try with an except on that exception object could work, assuming it's an exception which is already present in 2.7 (or maybe we can find a parent of that exception which is at least) | 20:50 |
| *** pcaruana has quit IRC | 20:51 | |
| *** jamesmcarthur has quit IRC | 20:55 | |
| *** jamesmcarthur has joined #zuul | 20:56 | |
| *** jamesmcarthur has quit IRC | 21:01 | |
| *** openstack has joined #zuul | 21:11 | |
| *** ChanServ sets mode: +o openstack | 21:11 | |
| *** dmellado has quit IRC | 21:18 | |
| *** dmellado has joined #zuul | 21:26 | |
| *** jamesmcarthur has joined #zuul | 21:27 | |
| *** bolg has joined #zuul | 21:31 | |
| *** jamesmcarthur has quit IRC | 21:34 | |
| mnaser | welp, anyone ever ran into something like this: rsync: chown "/var/lib/zuul/builds/c67586cdd4894cb19b773807298d3983/work/logs/tox/." failed: Invalid argument (22) | 22:03 |
| mnaser | the *only* odd thing is that this is running inside a container, but it's fully privileged | 22:03 |
| *** tosky has quit IRC | 22:03 | |
| *** jamesmcarthur has joined #zuul | 22:04 | |
| *** jamesmcarthur has quit IRC | 22:13 | |
| *** jamesmcarthur has joined #zuul | 22:16 | |
| fungi | is the path being passed as the first argument to the chown command instead of the second? | 22:23 |
| *** jamesmcarthur has quit IRC | 22:25 | |
| *** jamesmcarthur has joined #zuul | 22:27 | |
| *** mattw4 has quit IRC | 22:36 | |
| *** jamesmcarthur has quit IRC | 22:36 | |
| *** jamesmcarthur has joined #zuul | 22:37 | |
| *** bolg has quit IRC | 22:48 | |
| *** jamesmcarthur has quit IRC | 23:34 | |
| pabelanger | mnaser: yes, seen it a few times with users now. Either there is a file that is owned by root:root, and chown failed. Or maybe not using the same user / uuid as executor zuul users | 23:49 |
| pabelanger | mnaser: pretty sure flaper87 also had this issue | 23:49 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!