Thursday, 2019-05-09

*** jamesmcarthur has joined #zuul00:04
*** jamesmcarthur has quit IRC00:05
*** jamesmcarthur has joined #zuul00:05
*** jamesmcarthur has quit IRC01:12
SpamapSpabelanger:seems like jobs<->checks is a clear and obvious map so hopefully it's just a simple matter of code01:18
*** jamesmcarthur has joined #zuul01:43
*** jamesmcarthur has quit IRC01:44
*** jamesmcarthur_ has joined #zuul01:44
tristanCjhesketh: i think i found the issue with test_job_pause, i'll rebase the stack again (because of new conflict introduced yesterday) with a fix01:52
openstackgerritTristan Cacqueray proposed zuul/zuul master: Get executor job params  https://review.opendev.org/60707801:52
openstackgerritTristan Cacqueray proposed zuul/zuul master: Separate out executor server from runner  https://review.opendev.org/60707901:52
*** rlandy|bbl is now known as rlandy02:04
*** rlandy has quit IRC02:04
*** bhavikdbavishi has joined #zuul02:04
openstackgerritTristan Cacqueray proposed zuul/zuul master: Separate out executor server from runner  https://review.opendev.org/60707902:10
*** bhavikdbavishi has quit IRC02:15
openstackgerritTristan Cacqueray proposed zuul/zuul master: Move repository preparation into common class  https://review.opendev.org/64864202:16
openstackgerritTristan Cacqueray proposed zuul/zuul master: Separate out executor concerns from AnsibleJob  https://review.opendev.org/64864302:16
*** bhavikdbavishi has joined #zuul02:20
*** jamesmcarthur_ has quit IRC02:33
*** jamesmcarthur has joined #zuul02:34
*** jamesmcarthur has quit IRC03:08
*** bhavikdbavishi has quit IRC03:10
*** jamesmcarthur has joined #zuul03:11
tristanCnow it seems like zuul-tox-remote failre is not related, it randomly fails on TestZuulStream2[567].test_command03:15
openstackgerritTristan Cacqueray proposed zuul/zuul master: config: add tenant.toDict() method and REST endpoint  https://review.opendev.org/62134403:28
openstackgerritTristan Cacqueray proposed zuul/zuul master: gerrit: add support for report only connection  https://review.opendev.org/56821603:30
openstackgerritTristan Cacqueray proposed zuul/zuul master: [WIP] test_remote_zuul_stream: add debug  https://review.opendev.org/65791403:33
openstackgerritTristan Cacqueray proposed zuul/zuul master: runner: implement prep-workspace  https://review.opendev.org/60708203:34
*** jamesmcarthur has quit IRC03:34
*** jamesmcarthur has joined #zuul04:05
openstackgerritTristan Cacqueray proposed zuul/zuul master: [WIP] test_remote_zuul_stream: add debug  https://review.opendev.org/65791404:20
*** markwork has joined #zuul04:28
*** raukadah is now known as chandankumar04:30
*** swest has joined #zuul04:36
tobiashpabelanger: yes, there was already discussion about this and also an implementation by a collegue: https://review.opendev.org/#/q/status:open+project:zuul/zuul+branch:master+topic:bloeffler-report-reties04:39
tobiashwe may want to continue on that04:42
openstackgerritTristan Cacqueray proposed zuul/zuul master: runner: add configuration schema  https://review.opendev.org/64067204:52
openstackgerritTristan Cacqueray proposed zuul/zuul master: runner: add prepare-workspace command line interface  https://review.opendev.org/64477004:52
*** badboy has joined #zuul05:24
badboymordred: hi, does zuul support draft-created event?05:25
*** markwork has quit IRC05:32
*** bhavikdbavishi has joined #zuul05:44
*** markwork has joined #zuul06:05
*** markwork has quit IRC06:13
*** themroc has joined #zuul06:40
*** quiquell|off is now known as quiquell06:47
openstackgerritMark Meyer proposed zuul/zuul master: dd  https://review.opendev.org/65794506:53
AJaegerzuul team, the zuul-jobs fail on https://review.opendev.org/#/c/656908/ since Sphinx now needs python 3 and the tox-py27 is not using that. So, right now all of zuul-jobs gating looks broken ;(06:59
*** markwork has joined #zuul07:12
*** panda|off is now known as panda07:19
*** gtema has joined #zuul07:19
openstackgerritTristan Cacqueray proposed zuul/zuul master: runner: add execute sub-command  https://review.opendev.org/63094407:24
openstackgerritTristan Cacqueray proposed zuul/zuul master: runner: enable reuse of a job-dir  https://review.opendev.org/65795507:24
openstackgerritMark Meyer proposed zuul/zuul master: Add Bitbucket Server source functionality  https://review.opendev.org/65783707:24
*** jpena|off is now known as jpena07:45
*** jangutter has joined #zuul08:39
*** panda has quit IRC08:55
*** panda has joined #zuul08:55
*** wxy-xiyuan has quit IRC09:26
*** themroc has quit IRC09:59
*** themroc has joined #zuul09:59
*** mugsie has quit IRC10:25
*** mugsie has joined #zuul10:35
*** mugsie has quit IRC10:35
*** mugsie has joined #zuul10:36
*** mugsie has quit IRC10:38
*** mugsie has joined #zuul10:39
*** panda is now known as panda|lunch11:02
openstackgerritMark Meyer proposed zuul/zuul master: Add Bitbucket Server source functionality  https://review.opendev.org/65783711:03
*** themroc has quit IRC11:14
*** themroc has joined #zuul11:14
*** gtema has quit IRC11:18
*** gtema has joined #zuul11:18
openstackgerritMark Meyer proposed zuul/zuul master: Add Bitbucket Server source functionality  https://review.opendev.org/65783711:35
*** jpena is now known as jpena|lunch11:39
*** rf0lc0 is now known as rfolco|ruck11:49
*** panda|lunch is now known as panda12:06
*** abraverm has joined #zuul12:10
*** rlandy has joined #zuul12:20
*** jamesmcarthur has quit IRC12:21
*** jamesmcarthur has joined #zuul12:21
pabelangertobiash: great, thanks. I'll look and see what help it needs12:28
*** jamesmcarthur has quit IRC12:30
*** jpena|lunch is now known as jpena12:31
*** quiquell is now known as quiquell|lunch12:37
*** gtema has quit IRC12:37
openstackgerritMark Meyer proposed zuul/zuul master: Add Bitbucket Server source functionality  https://review.opendev.org/65783712:38
*** gtema has joined #zuul12:48
*** jamesmcarthur has joined #zuul12:49
pabelangertristanC: yah, that test needs to be debugged. It has been bouncing for a while13:02
pabelangerbadboy: you _should_ be able to create a trigger based on it. But know opendev doesn't use it today, feature is diabled.13:03
*** jamesmcarthur has quit IRC13:05
*** quiquell|lunch is now known as quiquell13:25
openstackgerritMark Meyer proposed zuul/zuul master: Add Bitbucket Server source functionality  https://review.opendev.org/65783713:30
*** gtema has quit IRC13:40
*** altlogbot_0 has quit IRC13:43
*** altlogbot_1 has joined #zuul13:46
*** quiquell is now known as quiquell|brb13:55
openstackgerritMonty Taylor proposed zuul/zuul-jobs master: tox siblings install: handle no metadata in setup.cfg  https://review.opendev.org/65690813:59
openstackgerritMonty Taylor proposed zuul/zuul-jobs master: Shift doc requirements to doc/requirements.txt  https://review.opendev.org/65808713:59
*** bhavikdbavishi has quit IRC14:02
openstackgerritMerged zuul/zuul-jobs master: Shift doc requirements to doc/requirements.txt  https://review.opendev.org/65808714:25
openstackgerritMerged zuul/zuul-jobs master: tox siblings install: handle no metadata in setup.cfg  https://review.opendev.org/65690814:25
*** gchenuet has joined #zuul14:25
gchenuetHi guys! Quick question about infra sizing. I saw that Executor nodes have also a merger role. Is it better to have separate executor and merger nodes or only executor ?14:28
gchenuetI also saw that OpenStack have 12 executors but can't find how merger nodes do you have. Can you share you some infos about your infra please ?14:29
Shrewsgchenuet: i think we use 8 mergers14:31
Shrewswhether or not you use separate mergers depends on how active your zuul is. not sure we've really quantified that somewhere, though14:33
pabelangergchenuet: for zuul.ansible.com, we run 2 executors and 2 mergers. This is mostly for HA reasons, right now. But if we disabled the 2 mergers, right now I think we'd be fine. Potentially slower job start times.14:33
ShrewsIt's talked about a bit here: https://zuul-ci.org/docs/zuul/admin/components.html#merger14:34
gchenuetthanks! for now, on Zuulv2 we have ~1.5K change merged by day and 3K builds on Jenkins.14:39
gchenuetI'll scale down our executors nodes and add merger nodes.14:39
*** gtema has joined #zuul14:47
markworkIs that just my impression or is the `zuul-tox-remote' a little bit flakey?14:49
markwork(The test job)14:49
pabelangermarkwork: yes, we need somebody to debug that failure14:51
pabelangerit looks to be around the zuul_stream plugin14:51
*** quiquell|brb is now known as quiquell|off15:01
*** markwork has quit IRC15:03
*** themroc has quit IRC15:06
*** themroc has joined #zuul15:07
*** jamesmcarthur has joined #zuul15:13
*** themroc has quit IRC15:14
Shrewsi *still* can't get the tox remote tests to run locally to try to help debug that. ssh during the test isn't working for some reason15:15
clarkb local firewall?15:20
Shrewsnope15:20
Shrewsmy suspicion is that the ssh key i'm telling it to use is not being used15:20
*** chandankumar is now known as raukadah15:28
corvusShrews: i have a vm on my local network with a zuul user with a dedicated ssh key, and i just verified this incantation works: ZUUL_SSH_KEY=~/.ssh/id_rsa_zuultest ZUUL_REMOTE_IPV4=192.168.1.219 ttrun tests.remote.test_remote_zuul_stream.TestZuulStream27.test_command15:39
Shrewscorvus: using pretty much the same command:  ZUUL_REMOTE_IPV4=192.168.1.26 ZUUL_SSH_KEY=/home/shrews/.ssh/test_id_rsa ttrun -e remote tests.remote.test_remote_zuul_stream.TestZuulStream27.test_command15:42
corvus(keep in mind it's going to want to log into the "remote" machine as the user "zuul" by default)15:42
Shrewsyep. i can: ssh -i /home/shrews/.ssh/test_id_rsa zuul@192.168.1.2615:43
Shrewsand that works15:43
Shrewsbut i keep getting: "msg": "SSH Error: data could not be sent to remote host \"192.168.1.26\". Make sure this host can be reached over ssh"15:43
Shrewsi cannot figure it out15:43
Shrewsthis *used* to work for me months ago. something is different (obviously)15:44
Shrewsi tried adding "-o IdentityFile" to the ansible ssh args out of desperation. that had no effect15:45
corvusi wonder if there could be any weird selinux/apparmor stuff going on15:46
Shrewscorvus: what OS version are you using?15:48
corvusi just restructured my command to be like yours (to make sure there wasn't an issue with the way i was doing tox envs) and it still works15:48
corvusubuntu 18.0415:48
Shrews18.04 here too15:49
Shrewsthis has been beyond frustrating over multiple days15:49
* Shrews should just admit defeat15:50
*** timburke has joined #zuul15:51
Shrewscorvus: it would be useful if we had a way to tell tests to keep the tmpdir test files around on test failures15:52
corvusShrews: KEEP_TEMPDIRS=115:53
Shrewsthat doesn't work15:53
Shrewscorvus: oh, sorry, it does! i tried that before and it didn't. maybe a typo15:54
corvusShrews: it just worked for me -- i ran with it and /tmpfs/tmp34tstkth/zuul-test/93b8fcd3bf9d46888cf2f7834d5aa5ea/ exists.  i found the path by looking for output about what ansible commands are running15:55
Shrewsprobably forgot the 'S'15:55
corvusah cool15:55
corvus(i have ZUUL_TEST_ROOT=/tmpfs also set in my env, so it doesn't thrash my ssd, thus the '/tmpfs')15:56
Shrewshrm15:56
Shrewsunix_listener: "/mnt/ramdisk/zuul/tmpf94rmq4u/zuul-test/0769046734924744a42208da6514073b/.ansible/cp/f544621055.8rOs7EfDhNObuQgx" too long for Unix domain socket15:56
Shrewswhen i try the logged SSH command manually15:56
corvusthat's new to me15:56
Shrewsmy test root is /mnt/ramdisk. maybe that's a few characters too much15:57
corvuswow that would be something15:57
corvusin production for us the "/mnt/ramdisk/zuul/tmpf94rmq4u/zuul-test" part is "/var/lib/zuul/builds" which is shorter as well.15:58
Shrews#define UNIX_PATH_MAX   10815:59
corvus113 > 10815:59
Shrewsfrom /usr/include/linux/un.h15:59
Shrews*sigh*15:59
corvusShrews: it's a pretty lame answer, but at least it's an answer.  :)16:00
corvusand we have all learned another piece of useless unix trivia today16:00
Shrewsi mean... just, wow16:00
corvus(i'll file that away with shebang path lengths max out at 128)16:01
mordred108 is the loneliest number16:01
* Shrews reconfigures his tmpfs to be on /t16:01
Shrewsit's a pretty random number16:02
corvusShrews: if you use single letter names for all files and dirs, you'll be safe16:02
Shrewscorvus++16:02
Shrewscorvus: seems like we could have captured that output somewhere16:03
corvusShrews: yeah, did you run ansible and it output that?16:03
*** bhavikdbavishi has joined #zuul16:04
Shrewscorvus: no, i ran the ssh command that ansible output during the test16:04
*** bhavikdbavishi has quit IRC16:07
*** bhavikdbavishi has joined #zuul16:07
ShrewsRan 1 test in 20.780s16:09
ShrewsOK16:09
Shrewsso very stupid16:09
Shrews*sigh*16:10
SpamapS"108 is the loneliest number", that 1973 one hit wonder by The Nerds.16:11
corvusand here i thought 0x01 was the loneliest number16:11
Shrewsi'm going to celebrate this victory with the eating of food. bbl16:17
SpamapScorvus:only on little endian16:24
mordredShrews: on osx, UNIX_PATH_MAX is 10416:25
corvusmust be to account for "C:\\"16:26
mordredhttps://serverfault.com/questions/641347/check-if-a-path-exceeds-maximum-for-unix-domain-socket in case anyone cares16:26
Shrewsmordred: anyone trying to run Zuul tests on osx deserves all the pain that would engender16:29
*** jamesmcarthur has quit IRC16:34
mordredShrews: I agree16:41
*** jpena is now known as jpena|off16:42
*** irclogbot_1 has quit IRC16:46
*** irclogbot_1 has joined #zuul16:48
SpamapShah, I did it once17:05
SpamapSstill have a scar17:05
*** vivobg has joined #zuul17:10
*** gtema has quit IRC17:14
*** bhavikdbavishi has quit IRC17:19
vivobgHello. I am trying to install Zuul after seeing it at the OpenInfra Summit in Denver. After following the zuul from scratch docs, I am getting the following error:17:37
vivobgZuul encountered an error while accessing the repo zuul/zuul-base- jobs.  The error was:    Cmd('git') failed due to: exit code(-13)     cmdline: git ls-remote https://opendev.org/zuul/zuul-base-jobs     stderr: 'Error reading command stream'17:37
vivobgDo you have any ideas on how to fix this? Thanks17:38
vivobgI see the above error in the scheduler logs, and in the Web UI notifications17:39
*** vivobg has quit IRC17:48
*** armstrongs has joined #zuul17:49
*** gchenuet has quit IRC18:08
*** armstrongs has quit IRC18:12
*** themroc has joined #zuul18:12
clarkbvivobg is gone but that ls remote command works for me. Given that I would suspect a firewall18:16
*** jamesmcarthur has joined #zuul18:34
*** jamesmcarthur has quit IRC18:38
*** markwork has joined #zuul18:42
*** jamesmcarthur has joined #zuul18:47
*** electrofelix has quit IRC18:56
*** themroc has quit IRC19:15
*** jamesmcarthur has quit IRC19:17
*** jamesmcarthur has joined #zuul19:31
*** jamesmcarthur has quit IRC19:39
*** jamesmcarthur has joined #zuul19:50
*** jamesmcarthur has quit IRC20:23
*** markwork has quit IRC20:35
*** VitaliyL has joined #zuul20:41
*** jamesmcarthur has joined #zuul20:54
*** armstrongs has joined #zuul20:56
*** logan- has quit IRC21:00
*** maxamillion has quit IRC21:02
*** maxamillion has joined #zuul21:03
*** logan- has joined #zuul21:03
*** jamesmcarthur has quit IRC21:05
*** armstrongs has quit IRC21:05
*** VitaliyL has quit IRC21:12
*** Armstrongs has joined #zuul21:16
*** jamesmcarthur has joined #zuul21:17
*** jamesmcarthur has quit IRC21:21
*** Armstrongs has quit IRC21:25
*** armstrongs has joined #zuul23:02
armstrongsregarding the issue above, we can run the git commands on the Zuul server from the command line, under the Zulu user. It just seems like the Zuul scheduler can't run the same commands successfully so we are a bit stuck23:05
armstrongsThis is our 1st attempt at running Zuul so we are thinking there may be a step we are missing in the docs?23:06
armstrongsThis is happening with the zuul-git connection listed here https://zuul-ci.org/docs/zuul/admin/zuul-from-scratch.html23:08
armstrongson centos 723:08
*** jamesmcarthur has joined #zuul23:36

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!