Wednesday, 2017-09-27

clarkb(something to wathc out for I guess but should be non issue for now)00:00
*** pabelanger has quit IRC00:01
*** mriedem1 has joined #openstack-infra00:05
*** mriedem has quit IRC00:05
*** mriedem1 is now known as mriedem00:08
*** thorst has joined #openstack-infra00:08
*** thorst has quit IRC00:09
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Fix race in test_in_repo_branch  https://review.openstack.org/50770800:14
jeblairmordred, clarkb, jlk: that change was running in a loop locally without error for a long enough time that i think that has fixed it.00:15
jlkk00:15
clarkbjeblair: the z-c change has the +2s, I didn't approve it as not entirely sure how to check it has fixed things, I guess just recheck project-config changes?00:21
jeblairclarkb: yeah.  let's go ahead and land it.  won't make things worse.  :)00:21
jeblairShrews: can retro-review it :)00:21
openstackgerritMerged openstack-infra/zuul-jobs master: Always create dest dir in zuul-cloner shim  https://review.openstack.org/50770400:30
clarkbthat update should apply immediately right?00:31
* clarkb looks for a change to recheck00:31
clarkb507222 has been rechecked00:32
*** hemna_ has joined #openstack-infra00:34
*** nicolasbock has quit IRC00:35
*** yamahata has quit IRC00:36
clarkblooks like post failures00:36
clarkbwhich is a different error I think00:36
clarkbprogress?00:36
* fungi had forgotten that insta-applying configuration when changes merge to the repo was a v3 feature00:37
clarkbhttp://logs.openstack.org/22/507222/3/check/legacy-bindep-fallback-centos-7/297d448/job-output.txt.gz#_2017-09-27_00_35_03_64250500:37
*** stakeda has joined #openstack-infra00:37
clarkband fedora is the permission denied thing00:39
clarkbhttp://logs.openstack.org/22/507222/3/check/legacy-bindep-fallback-debian-jessie/a25ef8b/job-output.txt.gz#_2017-09-27_00_37_37_648816 jessie is just broken maybe?00:39
*** Apoorva_ has joined #openstack-infra00:40
clarkbso that is an improvement over the old situation but still not fixed00:40
clarkbfungi: security.debian.org does not resolve earlier in that lo00:40
clarkbit does resolve locally though, so possible network problems?00:41
fungistrange00:41
fungiit certainly _should_ resolve00:41
fungior else, like, all debian users stop getting quick security updates00:41
clarkbya, I mean it totally resolves from here00:41
*** nicolasbock has joined #openstack-infra00:42
clarkbfungi: and is that libxml2-dev thing a broken package/repo? it deps on the old version but apt is going to install the new version00:42
ianwclarkb: isn't the fedora fail the same network issue there? http://logs.openstack.org/22/507222/3/check/legacy-bindep-fallback-fedora-26/fc748ef/job-output.txt.gz#_2017-09-27_00_34_48_85289000:42
*** Apoorva has quit IRC00:43
clarkbianw: oh it is and centos7 suffers the permission denied issue too00:43
clarkbso I think those are the same problem00:43
ianwoh i see, just above that00:43
clarkbso if we fix /home/zuul permissions that may make both of thoes pass00:44
clarkbthen we are just left with debian packaging catching up with reality00:45
clarkbjeblair: is there already a plan for fixing the homedir thing?00:46
ianw(sorry, catching up ...)00:48
ianwit's not really homedir permissions, but inability to log in?00:48
ianwPermission denied (publickey,gssapi-keyex,gssapi-with-mic,password).00:48
ianwor we've borked the authorized_keys or something?00:49
*** xarses has joined #openstack-infra00:49
clarkbianw: there was discussion of homedir perms at some point earlier today and I may have misparsed, could be auth failed because networking failed00:49
clarkbis it permission denied because of fs perms or ssh auth?00:50
ianwto me, http://logs.openstack.org/22/507222/3/check/legacy-bindep-fallback-fedora-26/fc748ef/job-output.txt.gz#_2017-09-27_00_34_46_189587 seems to say that rsync can not get in00:50
*** LindaWang has joined #openstack-infra00:51
jeblairclarkb: the homedir perms from earlier were that under fedora, /home/zuul was 700.  that's fine for most things, but caused us to upload the logs to logs.o.o with 700 (because of the rsync command).00:51
*** mugsie has quit IRC00:51
*** dingyichen has joined #openstack-infra00:51
clarkbgotcha so in this case likely actually what ianw says, auth00:52
jeblairclarkb, ianw: yeah, did we obliterate ~/.ssh ?00:52
ianwthat seems unlikely?  did something bindep install fiddle things somehow?00:53
jeblairmaybe the newly fixed zuul-cloner shim did it?00:54
clarkbjeblair: possibly, are we copying into the root of the homedir?00:54
*** Apoorva_ has quit IRC00:54
jeblairclarkb: yes00:55
clarkbbut that should add and not delete old things right? unless we have a .ssh/ in project-config00:55
clarkbalso it totally worked on ubuntu00:55
jeblairclarkb: agreed00:55
jeblair(i have to run now)00:56
clarkbprobably a good next step is putting an auto hold on that job, then looking at the node afterwards since root ssh shouldn't be touched00:57
*** xarses has quit IRC00:57
* clarkb goes to figure that out00:57
ianwclarkb: you read my mind :)00:57
clarkbarg my ssh key got unloaded /me first gets an ssh key loaded00:58
ianwthe suspect is https://review.openstack.org/#/c/507704/1/roles/fetch-zuul-cloner/templates/zuul-cloner-shim.py.j2 ?00:58
clarkbyes00:59
*** esberglu has joined #openstack-infra00:59
clarkbok autohold in place /me rechecks again01:00
clarkbdid the centos 7 job01:01
ianwhttp://logs.openstack.org/22/507222/3/check/legacy-bindep-fallback-fedora-26/fc748ef/job-output.txt.gz#_2017-09-27_00_33_42_73403601:02
ianwmkdir -p /home/zuul ... after zuul-cloner run, where's that come from?01:03
*** yamamoto has quit IRC01:03
*** yamamoto_ has joined #openstack-infra01:03
*** esberglu has quit IRC01:03
*** Sukhdev has quit IRC01:03
clarkblooks like the script that invokes the z-c shim01:03
clarkbbased on the level of +'s in that log01:03
clarkbthe jessie job looks happy now so going to blame that on repo syncage in mirrors01:04
*** thorst has joined #openstack-infra01:06
*** thorst has quit IRC01:06
ianwhttps://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/legacy/bindep-fallback-fedora-26/run.yaml#n2801:07
ianwstill not seeing where a mkdir comes into that01:07
clarkbthat job doesn't seem to use a debian mirror01:07
clarkbso that was upstream01:07
dimsclarkb : fungi : the mogan stuff can be done anytime they already stopped all work until the repo is switched - http://lists.openstack.org/pipermail/openstack-dev/2017-September/thread.html#12250901:07
clarkbianw: line 19 of that file maybe?01:07
clarkbianw: ya thats cleanup for the z-c invocation so it runs after z-c01:08
ianwclarkb: hmm, yep on the cleanup path, ok01:08
clarkbianw: ssh root@104.130.198.11601:09
clarkb/home/zuul/.ssh/authorized_keys lgtm on first glance01:10
clarkbmaybe selinux?01:11
*** cuongnv has joined #openstack-infra01:11
clarkbtype=USER_LOGIN msg=audit(1506474438.992:778): pid=3179 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login acct="zuul" exe="/usr/sbin/sshd" hostname=? addr=104.239.143.247 terminal=ssh res=failed'01:12
clarkbI'm going to add my public key to that authorized_keys file and see if I can reproduce01:13
clarkbya it asks for my password01:15
fungiso it's trashed .ssh (possibly entire homedir)?01:15
clarkbSep 27 01:14:56 centos-7-rax-ord-0000035830 sshd[3257]: Authentication refused: bad ownership or modes for directory /home/zuul01:15
fungiaha01:15
ianwls -ld /home/zuul01:15
ianwdrwxrwxr-x. 20 zuul zuul 4096 Sep 27 01:05 /home/zuul01:15
ianwya01:15
fungii bet it's world-readable .ssh01:15
clarkbshould it be 755?01:16
fungiwe inadvertently chmod it?01:16
clarkbdrwx------.  2 zuul zuul  4096 Sep 27 01:14 .ssh01:16
fungiaha, group-writeable is probably the issue, yeah01:17
clarkbfungi: the z-c shim copy may carry over its perms?01:17
clarkbi'm going to try chmod real quick01:17
clarkbyup that fixed ssh for me01:17
ianw"cp -al /home/zuul/src/git.openstack.org/openstack-infra/project-config/. /home/zuul"01:17
ianwdoes the -a fiddle it?01:17
clarkbianw: ya man page says that preserves mode01:18
clarkbI'm guessing centos and fedora have a different umask than debuntu which is why this breaks?01:19
clarkbwe could possibly fix this on an image level with a umask update01:19
clarkbor have a pre task ste it01:19
clarkbI too now need to look at dinner tasks01:21
clarkbI'll let others think about how best to address this problem01:21
clarkb(another option is move everything a dir level deeper01:21
ianwyep, "cp -al /some/dir /tmp/foo" changes /tmp/foo01:21
fungithat's mildly surprising behavior01:21
ianwor "cd /home/zuul ; cp -al $path ." ?01:21
fungibut yeah, i guess it's down to umask01:21
clarkbianw: oh that might be simplest01:22
*** jdandrea_ has quit IRC01:28
tonybAm I okay to +W stuff in openstack/releases?01:30
clarkbtonyb: I think so, all of that stuff should still be processed on zuulv201:30
clarkbwe're in a weird halfway spot for the moment while we get project-config on zuulv3 properly01:31
prometheanfire/tmp/in_target.d/install.d/60-zuul-worker: line 25: die: command not found01:32
prometheanfireboooo?01:32
tonybclarkb: Thanks.01:32
clarkbtonyb: but may be try a single thing and make sure I haven't mis interpreted the situation01:32
*** dimak has quit IRC01:32
clarkbprometheanfire: need more context01:32
fungiprometheanfire: is zuul telling you to fork off and die?01:32
*** lihi has quit IRC01:33
fungipretty rude, if so01:33
ianwprometheanfire: that's pretty closely related to your issue of finding somehwere to have a library function in the chroot i think01:33
tonybclarkb: Yeah I'll go slow and careful :)01:33
*** dimak has joined #openstack-infra01:33
prometheanfiretrying to build a nodepool image01:33
*** lihi has joined #openstack-infra01:33
*** ijw has quit IRC01:33
prometheanfirethat's not my code this time01:33
clarkbZUUL_USER_SSH_PUBLIC_KEY=${ZUUL_USER_SSH_PUBLIC_KEY:-$HOME/.ssh/id_rsa.pub}01:34
clarkbweird that it didn't fail in the extra data script since there is a die there too01:35
prometheanfire    die "Can not find Zuul public key!"01:35
clarkbya in the extra data script for that element its copying the public key into the chroot01:35
prometheanfireah, I'll just put a $HOME/.ssh/id_rsa.pub in there01:36
clarkband the line I pasted above is where it looks for the public key file01:36
prometheanfiremake it mine for the test01:36
prometheanfiretake another 3 hours to test :P01:36
prometheanfireturns out building a gentoo image takes a while01:36
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: zuul-cloner: enter directory before copying  https://review.openstack.org/50771301:37
*** caphrim007 has quit IRC01:37
clarkbianw: comment on ^ otherwise I think its worth using that route01:38
*** caphrim007 has joined #openstack-infra01:38
ianwclarkb: thanks, durr01:42
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: zuul-cloner: enter directory before copying  https://review.openstack.org/50771301:42
*** gildub has joined #openstack-infra01:48
clarkbdims: assuming things are moving with zuul tomorrow I can take a look at the mogan repo stuff, won't tonight because its late enough I don't want to be force pushing anything01:49
dimsclarkb yep of course, sounds good. thanks!01:50
*** Douhet has quit IRC01:51
*** Douhet has joined #openstack-infra01:52
mordredclarkb, ianw: looks good - +301:53
ianwdoes that clear known issues with project-config?01:54
*** gildub has quit IRC01:54
clarkbianw: ya I think once those two jobs work project-config is gateable on zuulv301:54
mordredyup01:54
*** Apoorva has joined #openstack-infra01:54
clarkbmordred: re tonyb's question about releases above, was I correct01:55
clarkbbasically zuulv2 is still managing all that and should be safe for release team to continue to make releases?01:55
tonybclarkb: well the 1 thing I tried worked ;P01:55
mordredyes - we haven't landed the switch to have v3 take over01:55
mordredonly project-config is in a weird state01:55
mordredand once that ianw patch lands and fixes fedora/centos and https://review.openstack.org/#/c/507222/ turns green - we'll be in solid shape01:56
mordredalso - yay for this turning up on that project-config patch and not, you know, on 10k jobs all at the same time01:56
*** hongbin has joined #openstack-infra01:58
openstackgerritMerged openstack-infra/zuul-jobs master: zuul-cloner: enter directory before copying  https://review.openstack.org/50771302:02
mordred\o/02:02
clarkbwant to recheck thtat change now?02:02
mordredI've rechecked https://review.openstack.org/#/c/50722202:02
mordredfingers crossed02:02
clarkbstill failed02:05
clarkbran with new code too according to log (I see the cd)02:06
clarkbI really need to afk now though02:06
*** thorst has joined #openstack-infra02:07
ianwhmm, post_failure is different though?  having a look02:08
ianwbummer02:13
ianwhttp://logs.openstack.org/22/507222/3/check/legacy-bindep-fallback-centos-7/74453e9/job-output.txt.gz#_2017-09-27_02_04_04_608530 ... the cd bit worked02:13
*** aeng has quit IRC02:16
*** SumitNaiksatam has joined #openstack-infra02:18
*** srobert has joined #openstack-infra02:20
*** gongysh has joined #openstack-infra02:38
*** esberglu has joined #openstack-infra02:42
*** esberglu has quit IRC02:47
*** erlon has quit IRC02:48
*** thorst has quit IRC02:48
*** kjackal_ has joined #openstack-infra02:55
*** liujiong has joined #openstack-infra03:00
*** ijw has joined #openstack-infra03:12
*** rakhmerov has quit IRC03:14
jeblairmordred, ianw, clarkb, Shrews: if we don't find a way to make this work, we can probably just decide to change $WORKSPACE from /home/zuul to /home/zuul/workspace, and add a step to legacy-base to ensure that dir is created.03:16
ianwjeblair: i've been testing and i think drop the -a03:16
jeblair(or even go so far as to make it /home/zuul/$JOB to be even more jenkins/zuulv2 like)03:16
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: Revert "zuul-cloner: enter directory before copying"  https://review.openstack.org/50772303:16
*** ijw has quit IRC03:17
ianwjeblair: ^03:17
jeblairianw: cool.  i'm not really here -- just wanted to throw that out there as an option -- there isn't anything terribly magical about the current value of WORKSPACE -- it just seemed convenient at the time.  less so in retrospect.03:18
ianwhttp://paste.openstack.org/show/621999/03:18
jeblairthere may be other reasons to move workspace down too -- like not surprising any jobs by having a .ssh dir inside the git working dir03:19
jeblairi may be talking myself into that :)03:20
jeblairanyway... g'night.03:20
ianwyeah, it's an option03:20
*** nicolasbock has quit IRC03:21
openstackgerritIan Wienand proposed openstack-infra/project-config master: Shift nodepool quota from v2 to v3  https://review.openstack.org/50722203:26
*** tojuvone has joined #openstack-infra03:27
*** tojuvone has left #openstack-infra03:27
*** rakhmerov has joined #openstack-infra03:28
*** vivsoni has joined #openstack-infra03:32
adriantI seem to be having some trouble getting a package published on pypi by the CI system :(03:37
adriantI pushed a signed tag, the tag does exist, a job did appear to run in zuul, openstackci is owner in pypi, but no pypi update :(03:38
adriantproject in question is python-adjutantclient03:40
adriantI mean, I can always manually push the new package to pypi but I thought I should do it properly03:40
clarkbadriant: http://logs.openstack.org/26/26f0b531fa5ce0bd0e413c0c9e3f9cb8ab95516b/release/python-adjutantclient-tarball/5eca9b6/console.html#_2017-09-27_03_20_57_640721 it failed to make the tarball sdist because no venv env in tox.ini03:43
adriantclarkb: bah! ty03:44
adriantclarkb: so after I add that how do I retrigger the release?03:44
adriantactually03:45
adriantnvm03:45
adriantnew commit, new tag03:45
*** thorst has joined #openstack-infra03:45
*** Sukhdev has joined #openstack-infra03:46
*** gongysh has quit IRC03:48
zhongjunSince the manila-image-elements  patch has been merged (https://review.openstack.org/#/c/507640/), but  the image at http://tarballs.openstack.org/manila-image-elements/images/ is still dated 2017-09-21.   Does anyone know how to update image in tarballs. thanks03:49
*** aeng has joined #openstack-infra03:53
ianwzhongjun: does it comes from a periodic job or what?03:54
ianwit's a post job03:58
ianwhttp://logs.openstack.org/ff/ffabcc1a591c55e60e93da9a1f2325a8ca46ed0d/post/gate-manila-publishimage-generic-ubuntu-xenial/29adc11/console.html03:58
*** krtaylor has quit IRC04:00
zhongjunianw: The image build from manila-image-elements code. I am not sure if it is a periodic job or anything else04:01
*** coolsvap has joined #openstack-infra04:01
*** krtaylor has joined #openstack-infra04:01
ianwzhongjun: i can try a rerun, per the above link the post job failed.  but that was during a time of fairly big instability so let's see if it works on rerun...04:06
*** krtaylor has quit IRC04:08
zhongjunianw: the post job failed because of we use the worry image from tarball04:09
ianwzhongjun: the job ^^ failed before it did anything cloning repos04:10
zhongjunhttp://logs.openstack.org/40/507640/1/check/gate-manila-tempest-dsvm-generic-scenario-custom-image-ubuntu-xenial-nv/9018a54/logs/devstacklog.txt.gz#_2017-09-26_18_58_48_63704:10
ianwi guess you're saying the current gate is broken because the post job failed to upload the right image?04:11
zhongjunYes04:13
zhongjunThe current image is not correct04:14
ianwzhongjun: ok, i reran the publish job for you, logs are at -> http://logs.openstack.org/ff/ffabcc1a591c55e60e93da9a1f2325a8ca46ed0d/post/gate-manila-publishimage-generic-ubuntu-xenial/rerun/console.html04:15
ianwmanila-service-image-None.qcow22017-09-27 04:12 323M04:16
ianwzhongjun: that seems up to date now?04:16
*** thorst has quit IRC04:16
*** ykarel|away has joined #openstack-infra04:17
zhongjunYes, thank you04:17
*** jaosorior has joined #openstack-infra04:19
*** ianychoi_ has joined #openstack-infra04:20
*** Apoorva_ has joined #openstack-infra04:21
*** hongbin has quit IRC04:21
*** peterlisak_ has joined #openstack-infra04:22
*** purp_too has joined #openstack-infra04:22
*** gongysh has joined #openstack-infra04:22
*** kashyap` has joined #openstack-infra04:22
*** zz_ja has joined #openstack-infra04:22
*** inc01 has joined #openstack-infra04:23
*** ykarel|away is now known as ykarel|training04:23
*** spotz_ has joined #openstack-infra04:24
*** admcleod_ has joined #openstack-infra04:24
*** thorre_se has joined #openstack-infra04:24
*** bradm_ has joined #openstack-infra04:24
*** rkukura_ has joined #openstack-infra04:24
*** colettecello has joined #openstack-infra04:25
*** matbu_ has joined #openstack-infra04:25
*** spiffxp_ has joined #openstack-infra04:25
*** migi_ has joined #openstack-infra04:26
*** Sukhdev has quit IRC04:29
*** Apoorva has quit IRC04:29
*** hemna_ has quit IRC04:29
*** mriedem has quit IRC04:29
*** xinliang has quit IRC04:29
*** rkukura has quit IRC04:29
*** danpawlik has quit IRC04:29
*** thorre has quit IRC04:29
*** cburgess has quit IRC04:29
*** spotz has quit IRC04:29
*** yolanda has quit IRC04:29
*** ianychoi has quit IRC04:29
*** inc0 has quit IRC04:29
*** jrist has quit IRC04:29
*** leifmadsen has quit IRC04:29
*** Diabelko has quit IRC04:29
*** flaper87 has quit IRC04:29
*** jab has quit IRC04:29
*** jcook has quit IRC04:29
*** chason has quit IRC04:29
*** abelur has quit IRC04:29
*** sambetts|afk has quit IRC04:29
*** purp has quit IRC04:29
*** peterlisak has quit IRC04:29
*** ericyoung has quit IRC04:29
*** migi has quit IRC04:29
*** stephenfin has quit IRC04:29
*** matbu has quit IRC04:29
*** admcleod has quit IRC04:29
*** honza has quit IRC04:29
*** dmsimard has quit IRC04:29
*** spiffxp has quit IRC04:29
*** kashyap has quit IRC04:29
*** bradm has quit IRC04:29
*** gothicmindfood has quit IRC04:29
*** rkukura_ is now known as rkukura04:29
*** peterlisak_ is now known as peterlisak04:29
*** honza has joined #openstack-infra04:29
*** stephenfin has joined #openstack-infra04:30
*** abelur has joined #openstack-infra04:30
*** honza is now known as Guest8588804:30
*** thorre_se is now known as thorre04:30
*** chason has joined #openstack-infra04:30
*** sambetts_ has joined #openstack-infra04:30
*** cburgess has joined #openstack-infra04:30
*** ericyoung has joined #openstack-infra04:30
*** claudiub has joined #openstack-infra04:30
*** leifmadsen has joined #openstack-infra04:32
*** jcook has joined #openstack-infra04:32
*** jklare has quit IRC04:32
*** edwarnicke has quit IRC04:32
*** nhicher has quit IRC04:33
*** edwarnicke has joined #openstack-infra04:33
*** slagle has quit IRC04:34
*** trown|outtypewww has quit IRC04:34
*** mdbooth has quit IRC04:34
*** jklare has joined #openstack-infra04:34
*** trown has joined #openstack-infra04:35
*** nhicher has joined #openstack-infra04:35
*** kjackal_ has quit IRC04:35
*** xinliang has joined #openstack-infra04:36
*** udesale has joined #openstack-infra04:36
*** jrist has joined #openstack-infra04:36
*** flaper87 has joined #openstack-infra04:36
*** yolanda has joined #openstack-infra04:37
*** dmsimard has joined #openstack-infra04:37
*** hemna_ has joined #openstack-infra04:37
*** Diabelko has joined #openstack-infra04:37
*** danpawlik has joined #openstack-infra04:37
*** Apoorva_ has quit IRC04:40
*** sree has joined #openstack-infra04:40
adriantclarkb: ty again! package now updated on pypi :)04:41
*** slagle has joined #openstack-infra04:41
*** ankkumar has joined #openstack-infra04:42
*** armax has joined #openstack-infra04:42
*** liujiong has quit IRC04:44
*** psachin has joined #openstack-infra04:47
*** mdbooth has joined #openstack-infra04:48
zhongjunianw: sorry, the image name is not correct, the correct image name is "manila-service-image-master.qcow2"04:58
ianwzhongjun: hmm, i may have missed an argument there05:01
ianwzhongjun: running it again with a refname of master05:01
zhongjunianw: thank you, how to rerun it?05:04
*** zhurong has joined #openstack-infra05:13
ianwzhongjun: it's a manual process, i guess you could commit some dummy change.  anyway, check again, i think it's right now05:13
*** thorst has joined #openstack-infra05:13
*** rlandy has quit IRC05:14
zhongjunianw: yeah. It's right now, thank you again05:17
*** dhajare has joined #openstack-infra05:18
*** esberglu has joined #openstack-infra05:25
*** esberglu has quit IRC05:29
tonybCan an infra core look at: https://review.openstack.org/#/c/505656/5  I want to make sure I'm processing the file correctly, and avoid +W'ing my own change05:32
ankkumarHi AJaeger_05:35
ankkumarCan you please review this patch https://review.openstack.org/#/c/462695/05:35
*** thorst has quit IRC05:47
*** ltomasbo has quit IRC05:49
*** jpena|off has quit IRC05:52
*** amoralej has quit IRC05:53
*** jamesmcarthur has joined #openstack-infra05:53
*** jamesmcarthur has quit IRC05:58
openstackgerritTin Lam proposed openstack-infra/project-config master: Add NFS test gate to OpenStack-Helm  https://review.openstack.org/50768205:58
ianwzhongjun: np, i'll delete the bad one06:01
tonybianw: Thanks!06:01
openstackgerritFlavio Percoco proposed openstack-infra/irc-meetings master: Reflect TC's office hours in the meetings calendar  https://review.openstack.org/50749006:02
*** pahuang has quit IRC06:04
*** aeng has quit IRC06:06
openstackgerritTony Breeds proposed openstack-infra/irc-meetings master: JJB meetings are being held in dedicated channel  https://review.openstack.org/49631406:07
openstackgerritMerged openstack-infra/irc-meetings master: Update channel list from project-config  https://review.openstack.org/50565606:09
openstackgerritMerged openstack-infra/irc-meetings master: Proposal for change of FWaaS meeting  https://review.openstack.org/50717206:10
openstackgerritMerged openstack-infra/irc-meetings master: Change heat meeting schedule to 1300 UTC Wed.  https://review.openstack.org/50690106:10
*** pgadiya has joined #openstack-infra06:16
*** jtomasek has quit IRC06:18
openstackgerritMerged openstack-infra/irc-meetings master: JJB meetings are being held in dedicated channel  https://review.openstack.org/49631406:19
*** armax has quit IRC06:19
*** fbouliane has quit IRC06:19
*** ltomasbo has joined #openstack-infra06:21
*** fbouliane has joined #openstack-infra06:22
*** hashar has joined #openstack-infra06:33
*** andreas_s has joined #openstack-infra06:36
*** rcernin has joined #openstack-infra06:39
*** rossella_s has joined #openstack-infra06:40
*** e0ne has joined #openstack-infra06:40
*** armax has joined #openstack-infra06:41
*** yamamoto_ has quit IRC06:41
*** thorst has joined #openstack-infra06:45
*** yamamoto has joined #openstack-infra06:45
*** armax has quit IRC06:50
*** pgadiya has quit IRC06:55
*** rossella_s has quit IRC06:56
*** Guest61942 has quit IRC07:03
*** yamamoto_ has joined #openstack-infra07:04
*** yamamoto has quit IRC07:04
*** florianf has joined #openstack-infra07:05
*** iyamahat has joined #openstack-infra07:05
*** iyamahat has quit IRC07:06
*** iyamahat has joined #openstack-infra07:06
*** sree has quit IRC07:06
*** sree has joined #openstack-infra07:07
*** e0ne has quit IRC07:08
*** yamamoto_ has quit IRC07:10
*** sree has quit IRC07:11
*** esberglu has joined #openstack-infra07:13
*** sree has joined #openstack-infra07:13
*** feisky has joined #openstack-infra07:15
*** lamt has quit IRC07:16
*** Dinesh_Bhor has quit IRC07:17
*** thorst has quit IRC07:17
*** esberglu has quit IRC07:17
*** tdasilva has quit IRC07:18
*** yamamoto has joined #openstack-infra07:18
openstackgerritMerged openstack/diskimage-builder master: Actually sort mount-point list  https://review.openstack.org/50481907:20
*** Dinesh_Bhor has joined #openstack-infra07:21
*** tesseract has joined #openstack-infra07:21
*** jpich has joined #openstack-infra07:29
*** jaosorior has quit IRC07:31
*** shardy has joined #openstack-infra07:35
*** threestrands has quit IRC07:38
*** mat128 has joined #openstack-infra07:38
*** tdasilva has joined #openstack-infra07:42
*** mat128 has quit IRC07:44
*** Hal has joined #openstack-infra07:45
*** Hal is now known as Guest1917307:45
*** iyamahat has quit IRC07:46
*** jaosorior has joined #openstack-infra07:46
*** amoralej has joined #openstack-infra07:47
*** jpena has joined #openstack-infra07:47
*** ijw has joined #openstack-infra07:47
*** ykarel|training is now known as ykarel07:53
*** alexchadin has joined #openstack-infra07:53
*** kashyap` is now known as kashyap07:57
*** ccamacho has joined #openstack-infra07:57
*** hashar has quit IRC07:58
*** ijw has quit IRC08:02
*** yamahata has joined #openstack-infra08:03
*** ralonsoh has joined #openstack-infra08:04
*** hashar has joined #openstack-infra08:04
*** efried has quit IRC08:08
*** zhurong has quit IRC08:11
*** egonzalez has joined #openstack-infra08:12
*** thorst has joined #openstack-infra08:14
*** Guest19173 has quit IRC08:15
*** makowals has quit IRC08:17
*** efried has joined #openstack-infra08:19
*** baoli has joined #openstack-infra08:28
*** yamahata has quit IRC08:29
*** zhurong has joined #openstack-infra08:29
*** dizquierdo has joined #openstack-infra08:30
*** makowals has joined #openstack-infra08:32
*** baoli has quit IRC08:32
*** dingyichen has quit IRC08:35
*** alexchadin has quit IRC08:41
*** iyamahat has joined #openstack-infra08:42
*** alexchadin has joined #openstack-infra08:42
*** iyamahat has quit IRC08:43
*** iyamahat has joined #openstack-infra08:43
*** derekh has joined #openstack-infra08:47
*** alexchadin has quit IRC08:48
*** chlong has quit IRC08:49
*** thorst has quit IRC08:50
*** iyamahat has quit IRC08:53
*** jamesmcarthur has joined #openstack-infra08:53
*** jascott1 has quit IRC08:54
*** jascott1 has joined #openstack-infra08:55
*** udesale has quit IRC08:57
*** udesale has joined #openstack-infra08:57
*** jamesmcarthur has quit IRC08:58
*** jascott1 has quit IRC08:59
*** jtomasek has joined #openstack-infra09:01
*** esberglu has joined #openstack-infra09:01
*** claudiub|2 has joined #openstack-infra09:05
*** esberglu has quit IRC09:06
*** claudiub|3 has joined #openstack-infra09:06
*** makowals has quit IRC09:07
*** alexchadin has joined #openstack-infra09:07
*** alexchadin has quit IRC09:08
*** claudiub has quit IRC09:08
*** alexchadin has joined #openstack-infra09:08
*** claudiub|2 has quit IRC09:10
*** alexchadin has quit IRC09:11
*** markmcd has quit IRC09:14
*** electrofelix has joined #openstack-infra09:15
*** makowals has joined #openstack-infra09:16
*** e0ne has joined #openstack-infra09:16
*** markmcd has joined #openstack-infra09:18
*** yamamoto has quit IRC09:19
*** Guest85888 is now known as honza09:22
*** pbourke has quit IRC09:23
*** pbourke has joined #openstack-infra09:24
*** tosky has joined #openstack-infra09:25
*** feisky has quit IRC09:25
*** numans has quit IRC09:25
*** iyamahat has joined #openstack-infra09:26
*** askb has quit IRC09:26
*** askb has joined #openstack-infra09:27
*** sambetts_ is now known as sambetts09:27
*** numans has joined #openstack-infra09:28
openstackgerritChandan Kumar proposed openstack-infra/project-config master: Added tempest dsvm job for neutron-tempest-plugin  https://review.openstack.org/50703809:30
*** iyamahat has quit IRC09:31
*** askb has quit IRC09:32
*** askb has joined #openstack-infra09:33
*** jtomasek has quit IRC09:38
*** electrofelix has quit IRC09:46
*** thorst has joined #openstack-infra09:47
*** afazekas is now known as afazekas|seek4fo09:48
*** hashar is now known as hasharAway09:50
*** sdague has joined #openstack-infra09:50
*** esberglu has joined #openstack-infra10:00
*** LindaWang has quit IRC10:01
*** stakeda has quit IRC10:02
*** d0ugal has joined #openstack-infra10:03
*** esberglu has quit IRC10:04
*** cuongnv has quit IRC10:05
openstackgerritAndrea Frittoli proposed openstack-infra/devstack-gate master: Add post step to Tempest base job  https://review.openstack.org/50683510:08
*** dizquierdo has quit IRC10:11
*** LindaWang has joined #openstack-infra10:16
*** thorst has quit IRC10:18
*** gongysh has quit IRC10:19
*** zhurong has quit IRC10:20
*** yamamoto has joined #openstack-infra10:20
*** yamahata has joined #openstack-infra10:27
*** egonzalez has quit IRC10:31
*** jkilpatr has quit IRC10:34
*** andreas_s_ has joined #openstack-infra10:37
*** andreas_s has quit IRC10:40
*** jascott1 has joined #openstack-infra10:42
*** apuimedo_ has joined #openstack-infra10:43
*** andreas_s_ has quit IRC10:45
*** apuimedo has quit IRC10:45
*** apuimedo_ is now known as apuimedo10:45
*** jascott1 has quit IRC10:46
*** ankkumar has quit IRC10:47
*** andreas_s has joined #openstack-infra10:47
*** d0ugal has quit IRC10:48
*** udesale has quit IRC10:52
*** vponomaryov has quit IRC10:54
*** hemna_ has quit IRC10:57
*** nicolasbock has joined #openstack-infra11:00
*** d0ugal has joined #openstack-infra11:02
*** jkilpatr has joined #openstack-infra11:02
*** nicolasbock has quit IRC11:04
*** jkilpatr has quit IRC11:07
*** kjackal_ has joined #openstack-infra11:10
*** pbourke has quit IRC11:11
*** timothyb89 has quit IRC11:11
*** jaosorior is now known as jaosorior_sick11:12
*** thorst has joined #openstack-infra11:15
*** vponomaryov has joined #openstack-infra11:16
*** jkilpatr has joined #openstack-infra11:21
*** kjackal_ has quit IRC11:21
*** strigazi has quit IRC11:23
*** strigazi has joined #openstack-infra11:24
*** egonzalez has joined #openstack-infra11:25
*** pbourke has joined #openstack-infra11:25
*** thorst has quit IRC11:27
*** pcaruana has joined #openstack-infra11:27
*** udesale has joined #openstack-infra11:28
*** kiennt26 has joined #openstack-infra11:29
*** jtomasek has joined #openstack-infra11:41
*** rossella_s has joined #openstack-infra11:41
*** dizquierdo has joined #openstack-infra11:42
*** rossella_s has quit IRC11:47
*** dave-mccowan has joined #openstack-infra11:49
*** bobh has joined #openstack-infra11:50
Shrewsjeblair: mordred: i'm inclined to go with changing $WORKSPACE11:54
*** jpena is now known as jpena|lunch11:54
*** thorst has joined #openstack-infra11:55
*** dprince has joined #openstack-infra11:56
mordredShrews, jeblair: yah - me too11:58
*** inc01 has quit IRC11:59
*** dave-mccowan has quit IRC12:01
*** kiennt26 has quit IRC12:02
*** mat128 has joined #openstack-infra12:03
*** wolverineav has joined #openstack-infra12:04
*** dave-mccowan has joined #openstack-infra12:04
*** trown is now known as trown|outtypewww12:05
*** trown|outtypewww is now known as trown|afk12:05
*** mat128 has quit IRC12:06
*** mat128 has joined #openstack-infra12:08
*** mat128 has quit IRC12:08
*** mat128 has joined #openstack-infra12:08
*** afazekas|seek4fo is now known as afazekas12:09
*** bobh has quit IRC12:10
Shrewsi wonder why there is a "mkdir -p $WORKSPACE" *after* we are already in said workspace doing zuul-cloner-shim things? http://logs.openstack.org/22/507222/4/check/legacy-bindep-fallback-centos-7/7b2fdba/job-output.txt.gz#_2017-09-27_03_31_45_05228812:10
Shrewsi can't find where that's coming from12:10
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Put legacy jobs down a directory level into workspace  https://review.openstack.org/50783412:14
mordredShrews: how's that look? ^^12:15
*** dtantsur|afk is now known as dtantsur12:16
*** alexchadin has joined #openstack-infra12:16
*** pblaho has quit IRC12:18
*** Hal has joined #openstack-infra12:18
*** threestrands has joined #openstack-infra12:18
*** threestrands has quit IRC12:18
*** threestrands has joined #openstack-infra12:18
*** Hal is now known as Guest6418012:18
*** pblaho has joined #openstack-infra12:19
Shrewsmordred: lgtm, but don't we need to change the workspace in more than just those locations? i would have thought that would be defined in a base job or such12:21
mordrednot really - the converted JJB jobs are all self-contained for the most part12:22
mordredthe whole idea of 'WORKSPACE' isn't really a real thing in v312:22
openstackgerritMonty Taylor proposed openstack-infra/openstack-zuul-jobs master: Add workspace directory for converted legacy jobs  https://review.openstack.org/50783512:23
openstackgerritMerged openstack-infra/zuul feature/zuulv3: We should await ws.send_str()  https://review.openstack.org/50767112:25
Shrewsoh, that mkdir is coming from https://review.openstack.org/#/c/507835/1/playbooks/legacy/zuul-nodepool/run.yaml12:29
mordredShrews: ah - so it is12:30
*** gouthamr has joined #openstack-infra12:30
openstackgerritAndrea Frittoli proposed openstack-infra/devstack-gate master: Add post step to Tempest base job  https://review.openstack.org/50683512:30
Shrewsmordred: i think that trap is still safe... yeah?12:31
Shrewsyeah, nothing is removed now, so it shouldn't do anything12:32
Shrewsso many pieces12:33
*** kjackal_ has joined #openstack-infra12:34
*** gouthamr_ has joined #openstack-infra12:37
*** gouthamr has quit IRC12:38
mordredShrews: yah12:40
mordredsrrsly12:40
*** rlandy has joined #openstack-infra12:44
*** gcb has quit IRC12:45
*** amoralej is now known as amoralej|lunch12:48
*** jcoufal has joined #openstack-infra12:51
*** mat128 has quit IRC12:53
*** Guest64180 has quit IRC12:53
Shrewsi think we still want the revert of ianw's change: https://review.openstack.org/50772312:54
mordredyah - we can at least land it and even test to see if it works while we're waiting on others to awaken12:55
*** alexchadin has quit IRC12:55
mordredI agree with jeblair from above (and pabelanger_ from yesterday) that adding a workspace dir to the mix may still be nicer for other reasons12:55
*** pabelanger_ is now known as pabelanger12:56
*** kiennt26 has joined #openstack-infra12:56
*** alexchadin has joined #openstack-infra12:56
*** jpena|lunch is now known as jpena12:57
Shrewsyeah12:57
*** esberglu has joined #openstack-infra12:58
pabelangermordred: just legacy jobs?13:00
*** jdandrea_ has joined #openstack-infra13:01
*** eharney has joined #openstack-infra13:01
mordredpabelanger: yah13:02
pabelanger++13:02
mordredwe don't need it for non-legacy jobs13:02
pabelangerYup, agree13:02
mordredpabelanger: check out https://review.openstack.org/507835 and https://review.openstack.org/50783413:02
pabelangerlooking13:03
mordredpabelanger: (need to land 834 and get the filter plugin change rolled out before 835 will work)13:03
*** tpsilva has joined #openstack-infra13:03
mordredjust rechecked https://review.openstack.org/#/c/507222 to see if ianw's change improved things too13:06
pabelangermordred: comment on 50783413:06
*** kgiusti has joined #openstack-infra13:07
openstackgerritOpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements  https://review.openstack.org/50364513:08
*** mriedem has joined #openstack-infra13:09
mordredpabelanger: I believe we don't want the slash - but one sec ...13:09
pabelangerI know the trailing / for copy task means everything inside the directory13:10
pabelangercannot remember for sync13:10
mordredpabelanger: nope. you're right, I'm wrong13:10
mordredhttp://logs.openstack.org/03/506803/1/check/gate-bindep-bindep-fallback-ubuntu-trusty/69c2538/_zuul_ansible/post_playbook13:10
mordredthat's the one that's been working- it has the slash13:10
pabelangerkk13:10
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Put legacy jobs down a directory level into workspace  https://review.openstack.org/50783413:11
*** jistr is now known as jistr|call13:12
mordredpabelanger, Shrews: there ya go ^^13:12
*** krtaylor has joined #openstack-infra13:13
mordredWOOT13:13
mordredianw's change fixed project-config jobs13:13
Shrewssweeeet13:13
pabelangerNice13:14
openstackgerritMonty Taylor proposed openstack-infra/openstack-zuul-jobs master: Add workspace directory for converted legacy jobs  https://review.openstack.org/50783513:14
pabelanger+3 on 83413:14
Shrewsthe "Conflicts With" section on gerrit UI is neat here: https://review.openstack.org/#/c/507835/13:15
pabelangeroh, guess we don't have mirrors working on debian-jessie13:15
Shrews"400 too many terms in query"13:15
pabelangerwe pulling from ftp.us.debian.org13:15
mordredhrm13:15
pabelangerI think that is okay for now, but going to make note of it13:16
mordred++13:17
*** kjackal_ has quit IRC13:17
pabelanger507222 is now green!13:18
pabelanger\o/13:18
*** trown|afk is now known as trown13:18
mordredwoot. so - we're actually set to flip the switch!!!13:18
* mordred hands pies to pabelanger and Shrews13:18
pabelangeractually, now that I think about it13:19
pabelangerwhy did legacy bindep-fallback run on that change?13:19
pabelangerin zuulv2 the jobs only run when we change bindep-fallback.txt13:19
pabelangermordred: Hmm, we don't seem to have files section on bindep-fallback template now13:21
pabelangerhttps://review.openstack.org/#/c/507219/10/zuul/layout.yaml are original13:21
pabelangerline 278713:21
Shrewsthe legacy jobs don't have any matchers13:22
pabelangerhttp://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/zuul-legacy-jobs.yaml#n968 is new jobs13:22
Shrewswhy, i do not know13:22
pabelangerkk, might be a regression13:23
*** amoralej|lunch is now known as amoralej13:23
mordredyah - maybe there is a bug in expansion of filters and project-templates in the migration script?13:24
*** kjackal_ has joined #openstack-infra13:25
mnaseri thought all the matchers went into the projects.yaml file?13:25
mordredShrews: I'm on a call - any chance you have a sec to look at that? (you'll probably have to pull an older version of project-config)13:25
*** baoli has joined #openstack-infra13:26
*** lbragstad has joined #openstack-infra13:27
Shrewsyep, just lemme finish eating bfast real quick13:27
*** sbezverk has joined #openstack-infra13:27
mordred\o/13:28
*** chlong has joined #openstack-infra13:29
*** chlong has quit IRC13:31
pabelangermordred: Shrews: I also think we need to add legacy-project-config-layout for project-config project. Possible we missed it by mistake?13:32
pabelangerin v2.5 that was the job we always ran, regardless of files settings13:32
mordredpabelanger: nope- omitted on purpose13:32
pabelangerHmm13:33
pabelangerokay, that could be an issue13:33
mordredit's a job to check the zuul v2 layout.yaml13:33
mordredwe don't have that in v3 anymore - and zuul does syntax checking on its config files directly13:33
pabelangermordred: right, but all the other jobs have potential file matchers. So, there will be a case now if we touch a file that doesn't match, no job will run13:34
pabelangerso, we should pick a new job and remove the file matches, to always run13:34
mordredah - great point13:34
pabelangermaybe docs job13:35
mordredyah13:35
mordredpabelanger: or we could add noop-jobs13:38
mordredpabelanger: if we don't have an actual test to run on a given piece of content, I'm not sure running the specs site index generation will tell us anything more than noop would13:39
pabelangermordred: k, lets see what others say. We also still need to add a linters job for project-config, so possible that might be the new job13:41
mordredpabelanger: ++13:42
mordredpabelanger: let's DEFINITELY at least at noop jobs while we figure it all out so that we don't wind up with no gate jobs :)13:42
pabelangeryah13:42
mordredremote:   https://review.openstack.org/507860 Add noop-jobs as a fallback for project-config13:45
* Shrews trying to create a minimal test case for debugging matcher bug13:46
*** lifeless has quit IRC13:46
mordredShrews: oh jeez. good luck with that :)13:46
Shrewsi don't know how else to do it13:47
openstackgerritOpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements  https://review.openstack.org/50364513:47
mordredoh! I ahve discovered another issue ...13:50
mordredin the 'add projects to gerrit main.yaml' patch, I didn't add projects that are listed as retired13:50
mordredbut we have some projects that are retired that have jobs in zuul.yaml13:51
*** alexchadin has quit IRC13:51
pabelangerk13:51
mordredlet's just add them for now13:52
mordredor - I take that back - leme remove them from projects.yaml13:53
*** jamesmcarthur has joined #openstack-infra13:54
*** lifeless has joined #openstack-infra13:54
*** dave-mccowan has quit IRC13:54
*** camunoz has joined #openstack-infra13:54
*** udesale__ has joined #openstack-infra13:55
fungiso it's safe to mix no-op jobs with real jobs in v3 now?13:56
fungior were you suggesting a custom no-op job?13:56
*** udesale has quit IRC13:56
*** hemna_ has joined #openstack-infra13:58
*** dangers_away has left #openstack-infra13:58
mordredfungi: I do not know the answer to that question13:58
Shrewscompare (^gate-bindep-bindep-fallback.*$, gate-{name}-bindep-fallback-centos-7)13:58
*** rtjure has quit IRC13:58
*** baoli has quit IRC13:58
Shrewsthat's what the migrate script is comparing for job names to apply matchers. we aren't doing the format() there13:59
*** baoli has joined #openstack-infra13:59
mordredShrews: ah. that's unfortunate14:00
Shrewsnot sure where to get {name} value from, atm14:01
openstackgerritMohammed Naser proposed openstack-infra/system-config master: Add Ceph Luminous Debian mirrors  https://review.openstack.org/50786414:01
mordredShrews: well- we do that substitution in Job14:02
mordredI believe14:02
mordredShrews: oh - aren't you in a project context there?14:02
Shrewsmordred: nope. processing templates14:02
mordredhrm14:03
Shrewsmordred: right below the "# Handle matchers" comment in getNewJobs14:03
*** rtjure has joined #openstack-infra14:03
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Migrate zuul v2 project pipelines to v3  https://review.openstack.org/50717914:04
*** nhicher has quit IRC14:05
Shrewsso this only affects templates where {name} is used. other templates should be fine14:05
*** pblaho has quit IRC14:05
*** nhicher has joined #openstack-infra14:05
mordredwhere name is used and where it doesn't have an explicit project: matcher defined - since those get handled in the project-template expansion step14:06
Shrewsoh, my layout file might be wrong too14:06
mordredShrews: oh  yah - you need to update project-config to be before we removed the project-config jobs from layout perhaps?14:07
Shrewscompare (^gate-project-config-bindep-fallback.*$, gate-{name}-bindep-fallback-centos-7)14:08
Shrewsthat's better. still need to get name="project-config" from somewhere14:08
Shrewsmordred: if we aren't in project context, we cant do this here, can we?14:09
Shrews*sigh*14:09
*** sree has quit IRC14:09
*** yamamoto has quit IRC14:10
*** felipemonteiro has joined #openstack-infra14:10
*** sree has joined #openstack-infra14:11
*** felipemonteiro_ has joined #openstack-infra14:12
*** mat128 has joined #openstack-infra14:12
mordredShrews: you might have to ... throw in a for loop and do a check for every project name?14:12
*** camunoz has quit IRC14:13
*** camunoz has joined #openstack-infra14:13
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci master: Remove promotion logic from the toci scripts  https://review.openstack.org/50707614:13
Shrewsi think we might need to duplicate that logic for cases where the job name wouldn't match in template context. trying that now14:15
*** yamamoto has joined #openstack-infra14:15
*** Goneri has joined #openstack-infra14:15
*** felipemonteiro has quit IRC14:16
*** sree has quit IRC14:16
jeblairmordred: is there a change to the migration script to add workspace/ ?14:18
*** alex_xu has quit IRC14:18
mordredjeblair: yes.14:19
*** srobert has quit IRC14:19
mordredjeblair: https://review.openstack.org/50783414:19
mordredjeblair: and also to the filter14:19
jeblairmordred: ah right, included in the filter change14:19
mordredjeblair: so - other than the thing Shrews is looking at, we're good to go (And even that could be fixed as a follow up)14:20
*** yamamoto has quit IRC14:20
mordredjeblair: also - see fungi's question about noop jobs - and the noop discussion with pabelanger about project-config jobs14:20
mordredjeblair: also - good morning!14:20
jeblairmordred: i'm a little worried that Shrews thing will affect a lot of projects...14:20
*** alex_xu has joined #openstack-infra14:21
mordredyah14:21
openstackgerritLuigi Toscano proposed openstack/os-testr master: Fix .testr.conf detection: test path follows discover  https://review.openstack.org/50387714:21
mordredjeblair: I think we should give him a sec before we flip the switch14:22
*** mat128 has quit IRC14:23
jeblairmordred, fungi: it's safe inasmuch as it's not going to pull us into any shared queues, but i think it's bad form, so i'd like to avoid it if possible.  i think temporarily adding it is fine.  but i'd like to remove it shortly.14:23
jeblairmordred: good morning!14:23
Shrewswell, i don't think is simple. the bindep template doesn't get expanded into project-config (no 'skip-if' to reference it). So we can't apply the matchers will processing the project. It has to be while processing the template, but that doesn't have a referencing project14:23
Shrewss/will/while/14:23
Shrewsso i'm not clear on how to handle this14:24
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci master: Remove promotion logic from the toci scripts  https://review.openstack.org/50707614:24
fungido we know if it affects any other jobs/templates besides that one?14:25
*** jamesmcarthur has quit IRC14:25
*** hemna_ has quit IRC14:25
Shrewsfungi: good question. i don't know offhand. if it's only a few, we could hand fix it14:26
* Shrews needs to make another cup o' joe. bias14:26
jeblairthe problem is basically: matchers that match jobs applied by a template,  right?14:26
jeblairrather14:26
jeblairthe problem is basically: matchers that match jobs with an embedded project name applied by a template,  right?14:27
*** dave-mccowan has joined #openstack-infra14:27
Shrewsjeblair: yes14:27
*** hongbin has joined #openstack-infra14:27
*** jistr|call is now known as jistr14:27
Shrewsand it will really only be 'files' matchers, i think14:29
SpamapSoh my, it's an early morning jeblair14:30
jeblairandreaf: i was thinking that perhaps we could use a job variable to list files that should be saved by a devstack/tempest job; then it would be really easy for jobs to specify additional files to save (since that's such a common thing)14:30
jeblairmordred: ^14:30
pabelangerfungi: check-tripleo-jobs template seems to be what I expect for files14:31
Shrewsi guess 'branch' and 'voting' might be affected there, too14:32
Shrewsmordred: jeblair: the simplest solution seems to be *always* expand templates14:34
*** hemna_ has joined #openstack-infra14:34
Shrewsthen apply matchers after expansion14:34
Shrewsor expand templates where {name} is referenced14:35
mordredShrews: I could see the second perhaps ... losing all the templates would kinda suck14:37
Shrewsyeah14:38
*** udesale__ has quit IRC14:38
*** dave-mccowan has quit IRC14:38
mordredShrews: oh - what if we expanded all the templates in the project template matcher thing - then applied the matchers- and if there were no changes (nothing matched) we go with the template rather than the expaned set of jobs14:38
SpamapS+114:38
SpamapSthat's what I was going to say14:38
SpamapSspeculative future calculation is a zuul tradition14:39
mordred:)14:39
*** threestrands has quit IRC14:40
*** makowals has quit IRC14:40
Shrewsmordred: i think use of "expand" in that sentence is confusing me for some reason. can you use different words?14:40
*** jascott1 has joined #openstack-infra14:41
mordredShrews: while processing a project in writeProject - do a pass of of expandTemplateIntoProject for ALL of the templates - but on a copy of the project14:41
mordredShrews: then, check the results and see whether exapnding a given template and applying matchers resulted in jobs with matchers/variants14:42
mordredif so - then it's a template that needs to be expanded in the real pass - if not, it's not14:42
Shrewsmordred: ah. yeah, i think that would work14:43
mordred(so it would have to be a loop of for template in project['template'] - expandTemplateIntoProject(template, project.copy) - if doIHaveMatchers(): templates_to_expand.append(template)14:43
mordred*waves hands*14:43
*** alex_xu has quit IRC14:44
*** dave-mccowan has joined #openstack-infra14:44
*** jascott1 has quit IRC14:45
*** jaypipes has joined #openstack-infra14:49
*** dave-mcc_ has joined #openstack-infra14:49
*** alex_xu has joined #openstack-infra14:49
*** rbrndt has joined #openstack-infra14:50
*** dave-mccowan has quit IRC14:51
*** xarses has joined #openstack-infra14:52
*** andreww has joined #openstack-infra14:54
mordredShrews: oh - although I guess it's not actualy applying applyProjectMatchers after expandTemplateIntoProject - it would be a copy of the Handle matchers logic against the jobs in the template expansion14:55
*** gongysh has joined #openstack-infra14:55
clarkbre workspace/ change I'm not going to check every single file changed, but looking at a sample few lgtm. Did not approve as unsure if above discussion involves waiting for some reason (still catching up)14:56
*** xarses has quit IRC14:56
*** mat128 has joined #openstack-infra14:56
*** baoli has quit IRC14:56
mordredclarkb: the workspace change to ozj is safe to land - the project-config change to add projects to pipelines in v3 and go live is not14:56
jeblairi think we could go ahead and merge workspace; the current discussion is likely to produce another patch, but it'll probably be easier to build on it?14:57
clarkbok I'll approve then14:57
mordredjeblair, clarkb: if we land the ozj change, we can re-check the project-config change to verify it's still working14:57
mordredjeblair: yah- the workspace change is also to playbooks, the upcoming template one will be a patch to the pipeline definitions in project-config14:57
jeblairmordred: ack14:58
*** baoli has joined #openstack-infra14:59
mordredjeblair, Shrews: I *think* from looking at it that if we wanted to it would not be super hard (if we get speculative expansion going - since that will ensure all things get touched at least once) ... to build a patch that keeps track of what instances of a given new job name and matchers applied to them - so that if every instance of a given job had the same matchers applied, we could apply it to the15:00
mordredjob definition and not expand the template15:00
mordredwhich means I *think* what I'm saying is slightly different than what I suggested above. If instead of doing a speculative stab at applyProjectMatchers in writeProject - we actually just did two entire loops of _calling_ writeProject ....15:01
mordredone that calls a version of writeProject that expands every template regardless of need and records informatoin about which expansions were useful ...15:02
clarkbdoes zuul need restarting to apply 834 properly before we can use 835?15:02
mordredand then a second one that uses that information to knmow what to expand15:02
mordredand update toJobDict to also read the recorded information15:02
*** jdandrea_ has quit IRC15:03
*** ihrachys has joined #openstack-infra15:03
mordredShrews, jeblair: a) does that make sense and b) is it worth trying to do?15:04
*** andreas_s_ has joined #openstack-infra15:04
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Add workspace directory for converted legacy jobs  https://review.openstack.org/50783515:04
mordredk. rechecking project-config patch15:05
*** ykarel has quit IRC15:05
*** jdandrea_ has joined #openstack-infra15:06
toskythe proposal bot started rebasing the same patch over and over (https://review.openstack.org/#/c/505885/), is it related to the zuul v3 changes15:06
tosky?15:06
pabelangertosky: no, a result of upgrading gerrit15:06
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Migrate zuul v2 project pipelines to v3  https://review.openstack.org/50717915:07
*** andreas_s has quit IRC15:07
toskyoh, the other even15:07
toskyevent*15:07
toskyok15:07
Shrewsmordred: maybe? i'm still trying to code the first thing15:07
fungitosky: known issue. it's on the list of things to fix soon after zuul v3 cutover is behind us15:07
pabelangertosky: I believe we have a task to make the proposal bot smarter15:07
toskybut will it stop at some point? I mean, is there a condition that triggers that over and over, out of the blue?15:08
mordredShrews: ok. cool. how about we just ignore me for right now and roll with it if you get the first thing going15:08
clarkbtosky: changes to openstack/requirements merging15:08
fungitosky: the short explanation is that gerrit used to reject new patchsets if the only appreciable change was the commit id but the diff itself was the same. now it only emits a warning in those cases and accepts the new patchset15:08
fungiand yes, it should only happen when the requirements-core reviewers approve a change to the global-requirements.txt file15:09
toskyoh, so now every change to openstack/requirements will retrigger this? I see15:09
toskythanks15:09
fungiwhich is why we didn't treat it as a higher priority15:09
funginot every change to openstack/requirements, just changes which alter the global-requirements.txt file15:09
fungiso, for example, daily updates to upper-constraints.txt shouldn't trigger that15:09
*** yamamoto has joined #openstack-infra15:10
*** makowals has joined #openstack-infra15:10
*** srobert has joined #openstack-infra15:12
*** makowals has quit IRC15:13
Shrewsmordred: i would not be opposed to us trying to code the two things in parallel. whoever gets something that seems to work first wins15:15
*** chlong has joined #openstack-infra15:17
mordredShrews: ok - I can take a stab at the thing I said15:17
openstackgerritLuigi Toscano proposed openstack/os-testr master: Fix .testr.conf detection: test path follows discover  https://review.openstack.org/50387715:18
clarkbfungi: tosky I think I see the problem in the script, the diff it does is against HEAD (eg tip of master) and not against the pervious patchset if the previous patchset exists15:20
dirkclarkb: fungi: that sounds fixable15:21
clarkbdirk: ya we'll just need to fetch any existing change and diff against that instead before pushing15:21
fungiclarkb: though while we're in there for an overhaul, i think that's probably also a good time to add the reqeusted logic to create a new commit instead of amending if the existing change is already approved15:22
*** gongysh has quit IRC15:23
dirkfungi: you mean w+1 with approved?15:23
dirkI think it just searches for an open review with a special topic15:23
dirkHmm15:24
* dirk is already out of office for today15:24
fungiyeah, we would likely check the workflow label for a +115:24
dirkI can take a look tomorrow15:24
dirkIn return, a few reviews on the stuff that I have pending wouldn't hurt ;-)15:25
*** gouthamr_ has quit IRC15:26
fungidirk: any particular changes of yours which are more urgent than others? is there open work on the suse images? i think a lot of stuff is just blocked on the zuul v3 cutover work so we don't create new churn in job configs we'd have to track in two places15:27
clarkbI think the 42.2 mirror cleanup and wheels for 42.3 are likely high on that list?15:28
jeblairi'm approving the master version of the case-sensitive labels change15:28
jeblaironly a day late15:28
fungijeblair: oh, thanks! i guess we said it was merging this week15:29
pabelangerclarkb: fungi: just looking into debian-jessie image for nodepool, right now we only have bindep-fallback and ansible-hardening using it. Maybe we should see what openstack-ansible is using it for15:29
*** Sukhdev has joined #openstack-infra15:30
pabelangerevrardjp: odyssey4me: mhayden:^ seems you're the only job using debian-jessie now. Is that something you plan on growing it the future? Would you be sadface if we removed the image from nodepool?15:32
*** inc0 has joined #openstack-infra15:32
odyssey4mepabelanger mhayden is OOO right now, but I'm pretty sure it'd be sadface15:32
odyssey4meif it's a burden to maintain then I'd suggest working out an exit plan with mhayden - maybe he can figure out some external CI for additional platforms15:33
openstackgerritMerged openstack-infra/zuul master: Case sensitive label matching  https://review.openstack.org/48285615:33
clarkbok 507222 is happy now, I guess its just a matter of getting the file matcher stuff in place then we move forward15:33
Shrewsmordred: so... ugh, template expansion expands a v3 template into a v3 project. our speculative expansion needs to compare v2 template job names, which presents a problem since we've lost those at that point15:33
dirkclarkb: the 42.2 clean should be mergable any time?15:34
*** dprince has quit IRC15:34
pabelangerodyssey4me: okay, not a burden currently. Something we talked about a while back since we removed deb-packaging repos. But if you are using it, we should be able to support it still.15:34
Shrewsmordred: i guess i could loop through the stored jobs looking for the old name?15:34
clarkbdirk: it will need an infra-root to clean up the afs side of things, I think the change itself is likely safe though (just don't want to forget to clean up afs as necessary)15:34
dirkMost open review i have are related to enabling tumbleweed, but that's not particularly urgent15:34
odyssey4methanks pabelanger15:35
*** dtantsur is now known as dtantsur|afk15:35
dirkclarkb: the rm -rf in there should do the cleaning?15:35
clarkbdirk: it won't remove the volume though15:35
*** dprince has joined #openstack-infra15:35
clarkbI'd have to doule check the volume setup there to know if that is necessary though15:35
dirkclarkb: I thought it's just one volume overall ?15:36
clarkbdirk: also the sync after the rm -rf might need to be run manually as it will be slow (due to the large delta)15:36
clarkbassuming it is one volume15:36
clarkbso either volume needs to be removed entirely, or volume sync needs to be watched closely15:36
fungiodyssey4me: pabelanger: i don't know that the debian-jessie images are especially burdensome on us are they? granted, it seems silly that we lack debian-stretch images a couple months after release... so i'd be inclined to suggest that we should either have a plan to get stretch working and move jobs to it, or to retire debian images instead15:36
openstackgerritsebastian marcet proposed openstack-infra/openstackid master: Improved Missing Scope error  https://review.openstack.org/50790015:37
pabelangerclarkb: dirk: ya, I would say we can just manually delete it and manually release volume, just to be safe. Rather then having puppet do it15:37
pabelangershouldn't need to delete any volumes at this time15:37
fungiodyssey4me: pabelanger: where said plan might entail keeping jessie around for a cycle or two after stretch is working, but at least we should have a plan15:37
jeblairfungi, clarkb, mordred, jlk, tobiash: i'm going to tag zuul 2.6.0 from master now.  sound ok?15:37
fungijeblair: sounds good, thanks15:38
odyssey4mefungi pabelanger sounds sensible, although I'm not privy to what the requirement is from an image point of view15:38
clarkbjeblair: yup15:38
*** andreas_s_ has quit IRC15:38
pabelangerfungi: odyssey4me: Ya, I don't actually know how long jessie gets updates for15:39
odyssey4mefungi perhaps pop an email to the ML with the [ansible-hardening] tag to solicit feedback on where to go next... mhayden should pick it up when he can, or perhaps some other consumers of the role will respond with their suggestions15:39
*** electrofelix has joined #openstack-infra15:39
fungiodyssey4me: pabelanger: likely the bulk of the work is just testing http://git.openstack.org/cgit/openstack/diskimage-builder/tree/diskimage_builder/elements/debian-minimal for stretch, and then fixing anything there which isn't quite working15:39
*** psachin has quit IRC15:40
*** vhosakot has joined #openstack-infra15:40
pabelangerfungi: odyssey4me: It shouldn't be too hard now to updated nodepool-dsvm job to start building stretch, then we just need to confirm our project-config elements work15:40
jeblairpushed zuul 2.6.015:40
pabelangerfungi: odyssey4me: once zuulv3 is migrated, we can loop back15:41
fungiyeah, i might even have time to poke at it myself at that point15:41
*** jascott1 has joined #openstack-infra15:42
*** rcernin has quit IRC15:43
mordredShrews: getOldJob can get you from new name to old name15:45
*** yamahata has quit IRC15:45
Shrewsmordred: yep, found that. though i'm trying a different approach15:46
*** jascott1 has quit IRC15:46
jeblairmordred: i think the nodesets you used in the migration collided with what we wanted to do in https://review.openstack.org/50425915:48
openstackgerritRaissa Sarmento proposed openstack-infra/project-config master: Add new repo for manila tempest plugin  https://review.openstack.org/50790515:48
jeblairmordred: i think we need to have the migration use "legacy-devstack-single-node"15:48
jeblairmordred: should we just fix that after the migration?15:48
jeblairandreaf: once we do that, we can land the nodesets in this change and then you'll have a way to select a node to run tempest on: https://review.openstack.org/50425915:49
mordredjeblair: for all jobs?15:50
mordredjeblair: we can just set the nodeset on legacy-base in OZJ15:50
jeblairmordred: no just devstack.15:50
fungipabelanger: as to when security support ends for jessie, it depends on how you define it. jessie (now as "oldstable") is supported by the debian security team until probably june (~a year from the stretch release date), at which point the volunteer lts team takes over providing crucial updates planned through april 2020: https://wiki.debian.org/LTS15:51
jeblairmordred: devstack-single-node is used by legacy-dsvm-base and legacy-puppet-openstack-integration.  so i think i'd just change those 2 places.15:51
openstackgerritRaissa Sarmento proposed openstack-infra/project-config master: Add new repo for manila tempest plugin  https://review.openstack.org/50790515:51
mordredjeblair: we can set that on legacy-dsvm-base and ... yah15:51
mordredjeblair: what you said15:51
jeblairmordred: assuming that doing so now would complicate the migration script/process, i'll just put than on the post-cutover list15:52
jeblairzuul 2.6.0 is on pypi; sending followup email15:53
*** trown is now known as trown|afk15:53
*** erlon has joined #openstack-infra15:55
*** xyang1 has joined #openstack-infra15:56
clarkbhow does one remove an autohold on a job? I want to cleanup after the debugging last night15:56
clarkbset count to 0 maybe?15:56
*** iyamahat has joined #openstack-infra15:56
*** egonzalez has quit IRC15:57
*** rossella_s has joined #openstack-infra15:57
pabelangerclarkb: I _think_ it gets auto removed once the jobs are held15:57
*** iyamahat has quit IRC15:58
clarkbI see, then just delete the nodes on the nodepool side?15:58
pabelangerya, that is what I have been doing. And seems to have work15:58
*** dizquierdo has quit IRC15:59
openstackgerritMerged openstack-infra/openstackid master: Improved Missing Scope error  https://review.openstack.org/50790016:00
*** bswartz has joined #openstack-infra16:01
openstackgerritMerged openstack-infra/release-tools master: Fix options to allow --warn-exit in eol-branch.sh  https://review.openstack.org/50559316:01
clarkbhrm I wonder if the node that got held was already deleted. I don't see it on nl01/02 or nodepool.o.o16:02
*** yamahata has joined #openstack-infra16:02
mordredjeblair: you can go ahead and change legacy-dsvm-base and legacy-puppet-openstack-integration - we only reference them by parent name16:03
*** timothyb89 has joined #openstack-infra16:03
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Print status of playbook run in console  https://review.openstack.org/50791216:03
*** Rockyg has joined #openstack-infra16:03
pabelangerlooks like issue with inap16:03
mgagnepabelanger: yes16:03
mordredjeblair: that said - it's possible we'll have places where a thing gets parent: legacy-dsvm-base but defines a different nodeset16:03
mgagnepabelanger: we are investigating right now16:03
*** sbezverk has quit IRC16:03
mordredjeblair: (like the multinode jobs)16:03
pabelangermgagne: Yay16:04
*** LindaWang has quit IRC16:04
pabelangermgagne: also, you must have irc highlight for inap :)16:04
mordredjeblair: so we may need to add a nodeset name mapping - which would complicate this patch, so let's do that next16:04
mgagnepabelanger: of course16:04
*** Apoorva has joined #openstack-infra16:04
mgagnepabelanger: but also, I was about to communicate the current issue16:05
mordredjeblair: although, if you felt like making an old-node-name to new-legacy-devstack-nodeset-name mapping and sticking it in zuul/mapping.yaml in project-config - I thnk that would likely be the easiest way to add that and to work on it in parallel16:05
openstackgerritRaissa Sarmento proposed openstack-infra/project-config master: Add new repo for manila tempest plugin  https://review.openstack.org/50790516:05
openstackgerritMerged openstack-infra/release-tools master: Set up gerrit remote manually in eol_branch.sh  https://review.openstack.org/50559416:05
openstackgerritMerged openstack-infra/release-tools master: Check the gerrit remote is as expected, eol_branch  https://review.openstack.org/50559516:05
mordredjeblair, pabelanger: https://review.openstack.org/507905 makes me realize we need to update the new project guide to mention adding the project to zuul/main.yaml16:05
mordredjeblair, Shrews: I think I have the second crazy version written - testing now16:06
pabelangermordred: good idea16:06
jeblairmordred: i'll just wait until we're finished :)16:06
Shrewsmordred: good. mine still is not working, but i did find we are calling writeProject() twice in writeJobs() for no apparent reason16:07
mordredShrews, jeblair: I predict you're REALLY going to like the globla variables and nested for loops in this patch16:09
fungimordred: what you really need is to liberally sprinkle it with goto16:09
*** r-daneel has joined #openstack-infra16:12
*** hemna_ has quit IRC16:15
*** iyamahat has joined #openstack-infra16:16
*** jpich has quit IRC16:16
mordredfungi: dont' think I haven't16:17
mordredI did just add "This may be the ugliest code I've ever written." to the commit message :)16:18
*** trown|afk has quit IRC16:18
Shrewsmordred: how can i get the short-name used for {name}?16:19
jeblairmordred: always pushing yourself to superlatives!  that's great!16:19
Shrewswithin writeProject()16:19
mordredShrews: project.split('/')[1]16:19
mordredShrews: project['name'].split('/')[1]16:20
Shrewsk. thought maybe that may already be done16:20
mordredit might be - but I've long since lost track of sanity in this codebase16:21
*** iyamahat has quit IRC16:23
openstackgerritRaissa Sarmento proposed openstack-infra/project-config master: Add new repo for manila tempest plugin  https://review.openstack.org/50790516:24
*** tesseract has quit IRC16:24
*** rossella_s has quit IRC16:24
*** yamahata has quit IRC16:30
*** dizquierdo has joined #openstack-infra16:30
mgagnepabelanger: service is now restored16:31
*** camunoz has quit IRC16:31
fungimgagne: are you still seeing issues with puppet jobs in ovh-gra1 today?16:32
*** Hal has joined #openstack-infra16:33
*** jascott1 has joined #openstack-infra16:33
mgagnefungi: wrong ping? =)16:33
*** Hal is now known as Guest7284916:33
pabelangermgagne: thanks, checking nodepool now16:34
*** florianf has quit IRC16:34
fungimgagne: oh, right, it was mnaser oops16:34
mnaserfungi i dont think so!16:35
*** jascott1 has quit IRC16:35
fungimnaser: thanks, it probably cleared up on its own in that case16:35
pabelangerOpenStackCloudHTTPError: (500) Server Error for url: https://compute.api.mtl01.cloud.iweb.com/v2/addf341f4fa44528b1ac73e2f5c68a21/servers The server has either erred or is incapable of performing the requested operation.16:35
pabelangermgagne: I'm seeing ^now16:36
mordredooh - that's a nice error :)16:36
*** jascott1 has joined #openstack-infra16:36
mordredI like the use of "erred"16:36
pabelangerokay, things happening16:36
*** jascott1 has quit IRC16:36
pabelangerdelete requests were successful16:36
pabelangerbut still issues launching16:37
*** jascott1 has joined #openstack-infra16:37
mgagnepabelanger: ok, will check16:37
pabelangerdanke!16:37
*** kjackal_ has quit IRC16:39
*** david-lyle has quit IRC16:40
*** SumitNaiksatam has quit IRC16:40
*** rossella_s has joined #openstack-infra16:40
*** gouthamr has joined #openstack-infra16:43
pabelangermgagne: there we go16:43
pabelangerbuilding now16:44
mgagnepabelanger: :D16:44
mgagnenow begins the root cause analysis :-/16:44
pabelangerYah16:44
*** hemna_ has joined #openstack-infra16:45
*** gouthamr_ has joined #openstack-infra16:45
mordredjeblair: you konw what's great? putting in a debug error message about a missing value - but putting it BEFORE the if checking for the missing valueso that it prints every iteration16:46
jeblair:)16:47
*** bobh has joined #openstack-infra16:47
*** gouthamr has quit IRC16:49
*** derekh has quit IRC16:49
*** d0ugal has quit IRC16:50
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Remove superfluous writeProject() call  https://review.openstack.org/50792816:50
clarkbmordred: at least you know it is working?16:50
mordredclarkb: :)16:50
*** e0ne has quit IRC16:52
*** xyang1 has quit IRC16:52
*** yamamoto has quit IRC16:55
*** yamamoto has joined #openstack-infra16:59
clarkbdims: your new mogan tree will require a force push over the top of the existing tree right? eg there is no common ancestor and it will not fast forward?17:00
dimsyes, it's a replace kind-of-thing17:00
*** sree has joined #openstack-infra17:00
mordredclarkb, dims: you could make a merge commit merging dims tree into the existing tree - and do the merge with --theirs17:00
mordredwhich should create a fast-forwardable commit with the same results17:01
dimsmordred : my tree has stuff from mogan, nova and cinder17:01
clarkbmordred: ya, but I think they are trying to make the history better reflect the nova heritage of the code, which won't be as obvious with that merge commit17:01
*** kiennt26 has quit IRC17:01
mordrednod.17:01
dimsyes, i did a lot of git filtering to make it work17:01
clarkbthe other question I had was, is there any reason to preserve the old tree? should be make a branch called old-master based off of current master before doing the push?17:01
dimsclarkb : sure for safety17:02
*** Apoorva_ has joined #openstack-infra17:03
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove superfluous writeProject() call  https://review.openstack.org/50792817:04
*** jpena is now known as jpena|off17:05
*** Apoorva has quit IRC17:06
*** sree has quit IRC17:06
clarkbold-master is now a branch on mogan (we can change the name if that one is ugly)17:06
*** dave-mcc_ has quit IRC17:06
*** hrubi has quit IRC17:07
*** tosky has quit IRC17:10
*** dave-mccowan has joined #openstack-infra17:11
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Run ansible setup at start of job to verify network  https://review.openstack.org/50793317:11
clarkbdims: actually looking at the two git trees I think it might be a fast forward17:13
jeblairclarkb, jlk, pabelanger, odyssey4me, mordred: ^ when you have a sec, can you take a look at https://review.openstack.org/507933 ?  it addresses the network error we saw yesterday.  it's not urgent though.17:13
dimsclarkb : your git-fu is way stronger than me. so sure :)17:13
*** david-lyle has joined #openstack-infra17:13
*** pcaruana has quit IRC17:14
fungiclarkb: if they started as a fork of nova/cinder/something which inherited the initial commit from nova, even if they later deleted/replaced a bunch of files, then that sounds plausible17:14
clarkbdims: there are new commits in your tree on top of 7744129c83839ab36801856f283fb165d71af32e, can you just propose those as normal changes and have them go through code review? You'll need the permission to push merge commits but we should be able to add it17:14
clarkbthis way normal mogan review process applies which is a nice papertrail for what is happening17:14
*** Guest72849 has quit IRC17:16
tobiashjeblair: is there a reason to put the setup playbook into zuul instead of the base job?17:17
pabelangerjeblair: left comment, but looks good in general17:18
openstackgerritOpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements  https://review.openstack.org/50364517:18
clarkbalso did a file not get added to the commit?17:18
mordredtobiash: it's a network validation check playbook with a different timeout so that it doesn't take ages to fail if there's an issue17:19
clarkbor is setup_playbook already a thing?17:19
mordredno - this is a whole new thing to address the issue we saw yesterday17:20
clarkbright I get that17:20
clarkbits juts we make a directory for the playbook but there is no content that I can see17:20
pabelangerI think we are creating it for ansible.cfg file17:20
pabelangerthen just use CLI17:21
*** trown has joined #openstack-infra17:21
clarkbpabelanger: we also pass the playbook to the cli command17:21
clarkbbut maybe thats just to honor an internal function api contract and is otherwise unused17:21
*** rossella_s has quit IRC17:22
*** rbrndt has quit IRC17:22
pabelangerclarkb: ya, playbook there is just the folder, which runAnsible uses to setup some things with secrets17:23
jeblairyeah, it's just there for the ansible.cfg file; every playbook gets one17:24
*** kjackal_ has joined #openstack-infra17:24
jeblairbut more to the point, in order to get one, we need a "playbook" directory, even though we don't have a playbook.  it's a slight api abuse but keeps things very simple17:24
tobiashmordred: crazy, I thought ansible would have a per task timeout, but there doesn't seem to exist one17:25
*** Hal has joined #openstack-infra17:25
mordredtobiash: does 'ansible hostname -m setup' work as expected for windows hosts?17:25
*** Hal is now known as Guest9059117:26
tobiashmordred: I have to check that tomorrow17:26
pabelangertobiash: in fact there is gather_timeout (default: 10) but doesn't seem to apply in this case17:26
tobiashHopefully yes17:26
mordredtobiash: I would certainly hope so -otherwise fact gathering on windows would be super broken17:28
mordredpabelanger, tobiash: I think those timeouts aren't applying becaue this is the ssh transport layer pausing - so I think the place that implements the timeout isn't being reached17:29
tobiashmordred: according to the docs, this should work17:29
*** shardy has quit IRC17:29
mordredpabelanger, tobiash: it might be worthwhile for us to file a bug upstream if we can produce a nice test case17:29
pabelangermordred: agree with transport layer17:30
pabelangeradding upstream bug is good too17:30
*** yamamoto has quit IRC17:31
*** yamamoto has joined #openstack-infra17:31
clarkbspeaking of iwndows did the idea of using certificate authentication work out?17:37
clarkbtobiash: ^17:37
tobiashclarkb: yes, it works and I had already a working hello world job with this17:38
mordred\o/17:38
clarkbnice17:38
mordredthat's exciting17:38
tobiash(using hard codes certs for the start)17:38
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Run ansible setup at start of job to verify network  https://review.openstack.org/50793317:38
tobiashConsole log streaming could get a challenge though17:39
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Run ansible setup at start of job to verify network  https://review.openstack.org/50793317:41
tobiashAs the zuul stream module won't work on windows17:41
dmsimardclarkb: hey, just wanted to mention I haven't forgotten I'm still on the hook for two specs 1) puppetboard to ara and 2) proactive monitoring17:42
dmsimardsoon (tm)17:42
clarkbdmsimard: no problem, I think everyone is swamped under zuulv3 anyways :)17:42
tobiashHow is the cutover going?17:42
SpamapSSeems like it's getting close17:43
* SpamapS observing from afar17:43
clarkbtobiash: mordred and shrews are sorting out a files matcher rule problem for project templates aiui, but once that is done we are ready to unleash it to everything17:43
mordredyah17:44
tobiash\o/17:44
Shrewsmordred and I are racing to see who can produce the worst spaghetti code EVA17:44
*** camunoz has joined #openstack-infra17:44
mordredShrews: hang on - you think you're a match for my abilities in producing TERRIBLE spaghetti code?17:45
*** trown has quit IRC17:46
mordredShrews: you write code that's way too nice and well thought out for that battle17:46
* tobiash just had spaghetti for dinner :)17:46
Shrewsi'm trying. this migration has brought out the worst in my abilities17:46
*** electrofelix has quit IRC17:47
mordredtobiash: mmm. probably tastier than what I'm producing17:47
mordredShrews: uhoh. I mean - you've worked on MySQL - this could get tricky17:47
*** rbrndt has joined #openstack-infra17:47
Shrewsnow where can i place this new THD object....17:48
*** Swami has joined #openstack-infra17:48
*** e0ne has joined #openstack-infra17:49
jeblairSpaghetti Battle: Whose routine will reign supreme?!17:49
* jeblair bites into bell pepper17:49
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Add an additional pass through project templates  https://review.openstack.org/50794117:50
clarkbnow I need to watch some alton brown17:50
mordredjeblair, Shrews, pabelanger, clarkb: ^^ ok - that is now at least running and initial spot-check is promising17:50
*** dizquierdo has quit IRC17:50
mordredI'm going to push up patches with the results of running it so that we can look at diffs17:50
*** amoralej is now known as amoralej|off17:51
*** sambetts is now known as sambetts|afk17:52
mordredok. I see one bug -fixing real quick17:52
*** jtomasek has quit IRC17:53
Shrewsmordred: do we need the 2 deepcopy's of expand_project?17:54
mordredShrews: no clue - just seemed safer17:54
Shrewslol, spoken like a true spaghetti artist!17:55
*** jascott1 has quit IRC17:56
clarkbwhat is with the change to the shell script? nrelated maybe?17:56
*** jascott1 has joined #openstack-infra17:57
*** jascott1 has quit IRC17:57
*** jascott1 has joined #openstack-infra17:57
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Add an additional pass through project templates  https://review.openstack.org/50794117:58
clarkbI definitely think a diff of the results would be helpful in understanding what is going on at least for me17:58
mordredyah- those are coming17:58
mordredclarkb: I just noticed it when running - it was doing the final logic when I was not running it that way17:58
mordredclarkb: so - yah - unrelated - could be split17:59
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Migrate zuul v2 project pipelines to v3  https://review.openstack.org/50717918:00
mordredok - there's that patch updated18:01
openstackgerritAndrea Frittoli proposed openstack-infra/devstack-gate master: Add post step to Tempest base job  https://review.openstack.org/50683518:01
*** raissa_ has quit IRC18:01
*** bobh has quit IRC18:01
*** raissa_ has joined #openstack-infra18:02
clarkbI just asked my browser to render a 26k line diff18:02
clarkbit got mad18:02
openstackgerritMonty Taylor proposed openstack-infra/openstack-zuul-jobs master: Update job definitions with per-job filters  https://review.openstack.org/50794318:03
mordredinfra-root: ^^ that's the other half of the output to diff (it's much smaller)18:03
mordredone of the pieces of new logic is one that determines if matchers/variant info apply to all instances of a job, and if so it puts it on the job and does not put it on the entry in teh project pipeline definition18:04
clarkbmordred: in 943 I don't see the bindep jobs with new matchers18:04
clarkbI guess that implies it is in 179 instead18:04
mordredyah - and I don't see it there either - so we may not be all the way home yet18:05
*** ralonsoh has quit IRC18:06
mordredoh - I thnk I see it18:06
*** raissa_ has quit IRC18:07
Shrewsmordred: you need the shortname for the project i think?18:07
Shrewsmordred: orig_name = orig_name.format(name=project['name'])18:07
mordredShrews: ++18:07
Shrewsi had the same issue using similar18:08
pabelangerI'm wondering if the usage of branches: master is actually needed in some cases18:08
mordredyah- it doens't put it all the places18:08
mordredso those _should_ be places where there is a matcher that's saying "only run this on the master branch"18:08
pabelangeradded comment to 507943 about branches18:08
mordredpabelanger: but would be great to verify that18:08
pabelangerkk18:09
pabelangerlet me look18:09
*** jkilpatr has quit IRC18:10
pabelangerAh, okay. So bindep-fallback is special it seems18:10
pabelangerwhich in zuulv3, isn't an issue18:11
pabelangerhttp://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml?id=d5e62922ebe92da7eae69792edc185d17e9549ae#n278318:12
pabelangerour branch filter could be removed now18:12
pabelangerbranches: ^stable/(newton|ocata).*$18:14
pabelangermordred: I think that is going to be an issue^18:14
pabelangerdoes branches support regex in zuulv3?18:14
Shrewspabelanger: yes  https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.branches18:15
jeblairpabelanger: yes it supports regexes18:15
*** pcaruana has joined #openstack-infra18:15
pabelangerShrews: thanks! Was just looking for docs18:15
* jlk looks around wonders how things go18:15
jeblairit's also a list, which is probably a friendlier way to write that in the future18:15
jeblairi'm going to go grab lunch; biab.18:16
*** Sukhdev has quit IRC18:17
mordredjeblair, pabelanger, Shrews: OH - so - there is a good reason why we're not seeing the legacy-bindep-fallback jobs be updated ...18:18
mordredand that's because my copy of project-config already has that removed from layout.yaml - lemme fix that real quick18:18
pabelangerah18:19
Shrewsmordred: i'm also fairly certain you need the project shortname fix, otherwise you'll end up with 'gate-openstack-infra/project-config-bindep...' job names18:19
mordredShrews: oh - yah - I already did that one locally18:20
Shrewscool cool cool18:20
Shrewsjust making sure18:20
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Add an additional pass through project templates  https://review.openstack.org/50794118:21
mordredShrews: ^^ there's the latest - also added sorted() calls to emitting irrelevant-files - as their order is changing from run to run making it hard to see changes in behavior18:21
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Run ansible setup at start of job to verify network  https://review.openstack.org/50793318:23
*** yamahata has joined #openstack-infra18:23
*** raissa_ has joined #openstack-infra18:23
*** jkilpatr has joined #openstack-infra18:23
clarkbmordred: don't you need to take [1] of the project name split not [0] ?18:24
pabelangercool, once we stop / start executors we could try bringing sto2 back online18:24
mordredclarkb: yes.thank you.18:24
openstackgerritMonty Taylor proposed openstack-infra/openstack-zuul-jobs master: Update job definitions with per-job filters  https://review.openstack.org/50794318:24
mordredthat ^^ is mostly just to baseline with sorted irrelevant-files18:26
Shrewssooo... just looking at openstack/governance project as an example, it has a 'files' job matcher but i do not see that applied, and i'm guessing because we automatically remove the 'ubuntu-xenial' portion from the job name18:30
Shrewsmatcher name: ^gate-governance-tox-validate-ubuntu-xenial$ to be specific18:31
Shrewsthis may actually be a different bug18:31
openstackgerritJesse Keating proposed openstack-infra/openstack-zuul-jobs master: Remove gratuitous use of xargs  https://review.openstack.org/50769118:32
*** pcrews has quit IRC18:32
Shrewsi actually do not see that project in the generated output  O.o18:32
*** pcrews has joined #openstack-infra18:33
Shrewswait... where ARE our legacy projects?18:34
pabelangerShrews: you mean 507179 ?18:35
Shrewspabelanger: ah, yes! thx.18:36
Shrewsok, ignore me then18:36
* Shrews might have been staring at this too long18:36
mordredShrews: that's ok - I just realized that my local fixup of layout.yaml so that I was migrating the right  thing still wasn't complete :)18:37
*** rwsu has quit IRC18:38
pabelangerHmm, I think I might be a little confused18:39
pabelangermordred: Shrews: looking at legacy-api-guide job in 507943, is it going to be an issue that we duplicate the branch and files in project-template? http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/zuul-legacy-project-templates.yaml#n10218:40
pabelangeror, can those be removed now?18:40
clarkbpabelanger: I think if I'm reading it correctly the project-template takes precendence18:41
clarkbpabelanger: so should be fine18:41
clarkb(just extra verbose)18:41
EmilienMhi, we're debugging a Depends-On issue, on https://review.openstack.org/#/c/487488/ - it seems like the patch in Depends-On wasn't seen when deploying the repo http://logs.openstack.org/88/487488/27/check/gate-python-tripleoclient-python27-ubuntu-xenial/35e09ce/console.html#_2017-09-27_10_33_11_179279 - any thoughts ?18:42
pabelangerclarkb: right, I'm thinking if these jobs are around for a bit, it will be confusing to update18:42
clarkbEmilienM: unittests typically don't have the machinery required to handle depends on like that18:43
mordredyes - there is a level of extra verbosity in some of these - but they're not optimized for long-term maint. the intent is that everything with legacy- prefix be deleted18:43
clarkbEmilienM: since they are not integration tests18:43
Shrewspabelanger: should be fine. we can't expect the migration script to clean up things like that. will have to be a post-migration thing18:43
mordredEmilienM, clarkb: that's fixed in v318:43
clarkbmordred: sort of, tox will still do the wrong thing18:44
mordrednope. tox will do the right thing in v3 :)18:44
clarkbwhich is fine imo18:44
clarkbwait what?18:44
mordredas long as you add the otherproject to required-projects18:44
clarkbhow, that will break (among other things) contraints18:44
EmilienMmatbu_: fyi ^18:44
EmilienMmordred: thx for the infos18:44
matbu_clarkb: hmm thx, so we need to merged the depends-on before ?18:44
matbu_mordred: ?18:45
clarkbmordred: and tox will rewrite both requirements.txt and constraints?18:45
mordredit won't break constraints - things are installed via constraints - then thesystem is checked for sibling repos that have been prepared as part of required-projects, and if they exist, they will be added to the tox venv18:45
clarkbmordred: gotcha, that could create unexpected behavior in the other direction (due to bypassing constraints) but at least that unexpected behavior is likely more expected18:45
mordredit doesn't rewrite reuqirements - it installs the sibling repo into the tox venv after tox has done normal processing18:45
smcginnisIs the zuul upgrade still going on?18:46
mordredclarkb: yah - you still have to opt in by having a job that lists the other project in required-projects ...18:46
mordredclarkb: but the only thing you need for tox-based cross-testing of multiple repos is to mention the other repo18:46
mordredwhich hopefully will make such things easier for people who want them18:46
clarkbmatbu_: yes, current tox jobs install what is in your requiremnts files, optionally with constraints applied. This isn't currently compatible with depends-on18:47
pabelangersmcginnis: yup, we still have zuulv2.5 gating18:47
matbu_clarkb: ack thx you for the infos :)18:47
pabelangerShrews: okay, wfm18:47
Shrewsmordred: is another update to 507943 forthcoming?18:48
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Add an additional pass through project templates  https://review.openstack.org/50794118:48
clarkband 719 and 941 I think18:48
clarkbthere is 94118:48
mordredclarkb, Shrews: ^^ ok - this is still not fully working - but it's now properly expanding the template - it's now just not applying the matchers and variants to it18:48
* mordred will push up the other repos to - so as that other people can look18:49
smcginnispabelanger: OK, thanks. Wasn't sure if I missed something.18:49
clarkbsmcginnis: we're sorting out the last problems in the migration output (that we are aware of) before making the big switch throw18:49
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Migrate zuul v2 project pipelines to v3  https://review.openstack.org/50717918:49
openstackgerritMonty Taylor proposed openstack-infra/project-config master: DNM Re-add v2 references for project-config  https://review.openstack.org/50794718:49
jeblairmordred: for tox siblings -- i'm not entirely sure listing in required-projects is necessary.  i think a depends-on (or just being ahead in the gate) will trigger it.18:50
smcginnisclarkb: ++18:50
openstackgerritMonty Taylor proposed openstack-infra/openstack-zuul-jobs master: Update job definitions with per-job filters  https://review.openstack.org/50794318:51
mordredjeblair: cool for depends-on - just being ahead in the gate might be a little weird and we might want to look at / think about that (once we're done with this)18:51
*** Rockyg has quit IRC18:53
*** sbezverk has joined #openstack-infra18:54
*** tosky has joined #openstack-infra18:55
pabelangerjeblair: want to restart zuul-executors to pickup 507933?18:55
*** dprince has quit IRC18:56
mordredpabelanger, jeblair: if restarting is happening- also restarting zuul-web on zuulv3.o.o to pick up Shrews two patches to it is not a terrible idea either18:56
jeblairpabelanger: sure, i can do that in about 15 mins, or feel free to if you want to do it now18:57
pabelangerdon't mind waiting, give me time to refresh coffee18:58
openstackgerritMerged openstack-infra/irc-meetings master: Reflect TC's office hours in the meetings calendar  https://review.openstack.org/50749018:59
*** slaweq_ has joined #openstack-infra19:00
*** pcaruana has quit IRC19:00
*** sree has joined #openstack-infra19:01
*** hemna_ has quit IRC19:02
*** r-daneel has quit IRC19:02
*** r-daneel has joined #openstack-infra19:03
*** mrunge_ has joined #openstack-infra19:04
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add projects.required to zuul vars  https://review.openstack.org/50795119:05
jeblairmordred, clarkb: ^ if we want to narrow the scope of tox siblings19:06
*** sree has quit IRC19:06
mordredjeblair: ++ I think we do19:06
*** mrunge has quit IRC19:06
jeblairmordred: i'm going to write the other half of the change real quicklike19:06
mordredjeblair: I think adding a job that does required-projects is great - and since it can be done speculatively it's still pretty flexible19:06
mordredjeblair: but auto-putting it there just because it shares a queue and is stacked seems ... less idea :)19:07
mordredideal19:07
dmsimardinfra-root: because I'm sure you have a bunch of free time between the zuul v3 shenanigans, I just wanted to point out that unless mistaken, the multinode stacks are ready to land: https://review.openstack.org/#/q/topic:zuulv3-multinode+(status:open+OR+status:merged)19:07
*** e0ne has quit IRC19:08
Shrewsmordred: ok, had to step away to rest my eyes a bit. do you need help debugging your current iteration of the migration script?19:08
*** hasharAway is now known as hashar19:09
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: sqlreporter: use newrev instead of refs/heads/  https://review.openstack.org/50795419:11
*** coolsvap has quit IRC19:11
mordredShrews: ok - I think I see the latest issue - one sec, I might need help fixing it19:11
Shrewsok19:11
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Only run tox-siblings on required projects  https://review.openstack.org/50795519:13
*** vivsoni has quit IRC19:14
*** vivsoni has joined #openstack-infra19:14
jeblairi'm restarting the executors now19:16
*** jcoufal has quit IRC19:17
*** dprince has joined #openstack-infra19:20
jeblairand restarted zuul-web19:21
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Delay executor noop packets  https://review.openstack.org/50796119:26
*** bobh has joined #openstack-infra19:26
jeblairbased on the number of times ze08 has appeared in our job logs since we started this, i think we're going to want that ^ in before too long.19:26
*** lukebrowning has joined #openstack-infra19:27
mordredjeblair: ++19:31
*** rwsu has joined #openstack-infra19:32
mordredthis migration script bug is not fun19:32
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: Add ref_url column to the buildset reporter table  https://review.openstack.org/50371419:38
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: Add oldrev/newrev column to the buildset reporter table  https://review.openstack.org/50795419:38
*** eharney has quit IRC19:44
Shrewsmordred: so... i run your version and i get matchers on the bindep jobs on my shortened example. gonna try on the entire project-config files19:46
mordredShrews: oh yay!19:47
*** colettecello has quit IRC19:47
mordredShrews: I do not get the matchers - I just get the bindep jobs expanded19:47
mordredShrews: oh - wait - you get them on the jobs?19:47
Shrewsmordred: yes19:47
*** gothicmindfood has joined #openstack-infra19:47
mordredShrews: I think the specific case now is the following19:47
mordredShrews: job in a project-template applied to two different projects. global matcher that applies to only one of them19:48
mordredShrews: so that the matchers need to be output into the project-pipeline definition rather than the job19:48
*** markvoelker has quit IRC19:48
Shrewsproject-pipeline ?19:49
mordredsorry - into the project - in writeProject19:49
*** markvoelker has joined #openstack-infra19:49
jeblairpabelanger, jlk, clarkb: can you look at https://review.openstack.org/507951 and https://review.openstack.org/507961 ?19:52
jlkopened19:53
mordredShrews: BOOM19:55
mordredShrews: got it19:55
Shrews\o/19:55
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Add an additional pass through project templates  https://review.openstack.org/50794119:56
openstackgerritMonty Taylor proposed openstack-infra/openstack-zuul-jobs master: Update job definitions with per-job filters  https://review.openstack.org/50794319:56
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Migrate zuul v2 project pipelines to v3  https://review.openstack.org/50717919:57
mordredinfra-root: please look at those ^^ as carefully as possible19:57
*** jascott1 has quit IRC19:57
*** jascott1 has joined #openstack-infra19:57
mordredwe'll want to remove the DNM patch in the project-config chain before we land it19:58
jeblairmordred: what dnm?19:59
Shrewsinteresting. i still see the matchers getting applied to jobs20:00
mordredjeblair: there is a patch I shoved in front of https://review.openstack.org/507179 to add entries back to layout.yaml so that the migration script would pick them up to generate the jobs20:00
mordredShrews: they'll get applied to jobs if all projects that use that job have the same matchers20:00
Shrewsoh, now i get it20:01
mordredbtw - legacy-tripleo-ci-centos-7-scenario004-multinode-oooq-container-upgrades is fun to look at20:01
mordredit has both files and irrelevant files :)20:01
*** jascott1 has quit IRC20:02
mordredalso - it apparently publishes artifacts as we have based it on the publish-openstack-artifacts base job20:02
jeblairi wonder if anyone knew what caused that job to run.  :)20:02
mordredjeblair: I think we might need a legacy-publish-openstack-artifacts to use for the things that publish artifacts - that does the nodepool dir writing20:03
jeblairmordred: wow, it uses that too?20:04
*** mat128 has quit IRC20:04
mordredyah20:04
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /tenants route  https://review.openstack.org/50326820:04
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/status route  https://review.openstack.org/50326920:04
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/jobs route  https://review.openstack.org/50327020:04
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/builds route  https://review.openstack.org/46656120:04
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add projects.required to zuul vars  https://review.openstack.org/50795120:05
*** jascott1 has joined #openstack-infra20:05
openstackgerritMonty Taylor proposed openstack-infra/openstack-zuul-jobs master: Add artifact publication job for legacy jobs  https://review.openstack.org/50797120:06
mordredjeblair: ^^20:06
*** yamamoto has quit IRC20:08
jeblairmordred: apparently puppet-dashboard needs to be added to main.yaml20:09
Shrewsmordred: i hope we never have to return to this code in the future20:09
mordredjeblair: oh - actually - it can just be removed - it's a retired project it just still has entries in layout.yaml - I cleaned that up earlire20:09
mordredjeblair: it's the only one20:09
*** jascott1 has quit IRC20:10
fungi507943 really shows how much we were missing because of that, i guess20:11
*** jkilpatr has quit IRC20:14
*** r-daneel has quit IRC20:14
*** r-daneel has joined #openstack-infra20:15
pabelangerback now, catching up20:16
jeblairmordred: are you working on a revision to 179 without dashboard?20:16
pabelanger507961 got -1 from zuul20:17
Shrewsmordred: overall concept seems sound to me, and the new code is doing the things i'd expect you'd have to do.20:17
*** dhajare has quit IRC20:17
Shrewsi say we run with it20:17
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Delay executor noop packets  https://review.openstack.org/50796120:18
mordredjeblair: yes20:18
Shrewsi did not +3 in case others want to examine the new migrate.py20:18
jeblairpabelanger: thx20:18
openstackgerritIhar Hrachyshka proposed openstack-infra/project-config master: Removed confusing comment  https://review.openstack.org/50797320:21
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate master: Remove other references to DEVSTACK_GATE_TEMPEST_STRESS  https://review.openstack.org/50797420:21
fungii clearly didn't read the entirety of the converted projects set, but it looks sane and nothing in the script jumped out at me as particularly buggish20:22
jeblairno "#BUG: ..." comments at least20:22
fungihah20:25
Shrewswell, working on a alternative solution in parallel, i saw a lot of similarities in things i needed to do (although my solution hit a rather rude dead end)20:26
mordredfungi, pabelanger: could I bother you for a +3 on https://review.openstack.org/#/c/507971/ real quick?20:26
clarkbI'm lunching and picking up csa box back to computer in half hour ornso20:26
mordredclarkb: by then we'll likely be ready to do things20:26
*** yamamoto has joined #openstack-infra20:27
openstackgerritMonty Taylor proposed openstack-infra/openstack-zuul-jobs master: Update parents of converted artifact jobs  https://review.openstack.org/50797520:27
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Migrate zuul v2 project pipelines to v3  https://review.openstack.org/50717920:27
mordredok. that discards the DNM patch and also deals with puppet-dashboard20:28
*** yamamoto has quit IRC20:31
*** gouthamr_ has quit IRC20:32
*** jdandrea_ has quit IRC20:32
*** jpena|off has quit IRC20:33
*** ltomasbo has quit IRC20:34
*** gouthamr has joined #openstack-infra20:35
*** amoralej|off has quit IRC20:35
jlksuppose I should do the food in the face thing.20:35
pabelangerYa, I'll be stepping away from computer in about 45mins to do the same20:36
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add support for result data in child jobs  https://review.openstack.org/50480820:36
*** ltomasbo has joined #openstack-infra20:37
*** jpena|off has joined #openstack-infra20:38
*** amoralej has joined #openstack-infra20:38
pabelanger+2 on 50717920:38
pabelangerbindep-fallback looks correct20:38
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Add artifact publication job for legacy jobs  https://review.openstack.org/50797120:39
*** gouthamr has quit IRC20:39
openstackgerritAndrea Frittoli proposed openstack-infra/devstack-gate master: Add post step to Tempest base job  https://review.openstack.org/50683520:40
openstackgerritAndrea Frittoli proposed openstack-infra/devstack-gate master: Basic processing of test results.  https://review.openstack.org/50798020:40
openstackgerritAndrea Frittoli proposed openstack-infra/devstack-gate master: Basic processing of test results  https://review.openstack.org/50798020:41
mordredpabelanger: \o/20:42
*** gouthamr has joined #openstack-infra20:42
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Delay executor noop packets  https://review.openstack.org/50796120:43
Shrewsanyone else want to look at https://review.openstack.org/507943 or shall i merge it?20:43
jeblairShrews: go for it20:44
*** gouthamr has quit IRC20:45
mordred++20:45
jeblairi think we're waiting on 943 and 975 to merge, then i believe we can throw the switch (179).  is that correct?20:46
mordredjeblair: yes20:46
mordredjeblair: shall we go ahead and merge https://review.openstack.org/#/c/507941/  too?20:46
Shrews+A'd 94120:46
mordredShrews: ++20:46
jeblairah missed that one, thx20:47
jeblairmaybe on purpose.  ;)20:47
mordredjeblair: don't EVER LOOK INSIDE THAT FILE20:47
Shrewssrsly20:47
*** esberglu has quit IRC20:47
mordredif there are any additional issues, it will in fact be easier to just deal with them as bug fixes to the new jobs :)20:48
*** esberglu has joined #openstack-infra20:48
jeblairi'm going to upgrade the executors and, once those changes land, restart them.  i'd like to do that before the big switch throwing.20:48
jeblairmostly for the load balancing change.20:48
mordredjeblair: ++20:48
mordredjeblair: yah - I'd lke to use more executors than just 01 and 0820:48
mordredjeblair: should we squash https://review.openstack.org/#/c/507275 onto the shift nodepool quota patch?20:49
jeblairmordred: well, what we should really do is add tripleo in one patch, then update the shift quota patch to include it20:50
jeblairotherwise, reverts get confusing20:50
mordrednod20:50
* mordred fixes20:50
*** rwsu has quit IRC20:52
*** esberglu has quit IRC20:52
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Update job definitions with per-job filters  https://review.openstack.org/50794320:53
clarkbok back to computer20:54
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Update parents of converted artifact jobs  https://review.openstack.org/50797520:54
*** wolverineav has quit IRC20:54
*** wolverineav has joined #openstack-infra20:55
jeblairoh we need a scheduler restart too to fix the 'dump all the file content to log' thing.20:55
*** kgiusti has left #openstack-infra20:55
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add an additional pass through project templates  https://review.openstack.org/50794120:58
jeblairandreaf: regarding 507980, we should have a subunit->html role in zuul-jobs20:58
jeblairrestarting executors now, then scheduler.20:59
*** wolverineav has quit IRC20:59
andreafjeblair: yeah we do but includes also the archive and copy part which I'd rather do with the save-file role20:59
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Add tripleo-test-cloud-rh1 into nl01  https://review.openstack.org/50727521:00
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Shift nodepool quota from v2 to v3  https://review.openstack.org/50722221:00
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Add OVH to nodepool v3  https://review.openstack.org/50798421:00
mordredjeblair, clarkb: ok - reworked that into a stack - the first 2 can land right now21:00
*** e0ne has joined #openstack-infra21:00
mordredandreaf: we're going to have eversomuch fun constructing job bits aren't we?21:01
*** iyamahat has joined #openstack-infra21:01
jeblairandreaf: what does save-file do differently?21:02
clarkbmordred: adding the two clouds (rh1 and ovh) can mrege now ?21:02
*** sree has joined #openstack-infra21:02
jeblairzuulv3 is restarting21:02
andreafjeblair: I'm tyring to make general purpose roles so that they can be re-used easily21:03
andreafjeblair: the zuul-jobs role assumes that the user has read/write access in the folder where the test results are stored21:04
*** akscram1 has quit IRC21:04
*** e0ne has quit IRC21:04
*** eharney has joined #openstack-infra21:04
andreafthe way I'm doing it is to use a stage folder where I generate archive files to be copied (it can be /tmp)21:05
*** e0ne has joined #openstack-infra21:05
jeblairandreaf: right, i'm supportive of that.  :)  the roles in zuul-jobs are intended to be reusable too, so i wondered if it's something we can improve there21:05
*** akscram1 has joined #openstack-infra21:05
* mordred is excited for andreaf to start hacking on things in zuul-jobs too :)21:06
andreafjeblair: sure that's the kind of feedback I was hoping to get :)21:06
jeblair\o/21:07
*** sree has quit IRC21:07
jeblairzuulv3 is up again21:08
*** esberglu has joined #openstack-infra21:08
jeblairmordred: you may want to recheck your changes21:08
mordredjeblair: https://review.openstack.org/#/c/507984 if you have a sec21:08
*** bobh has quit IRC21:08
mordredjeblair: kk21:08
*** thorst has quit IRC21:09
mordredjeblair: https://review.openstack.org/#/c/507275/ is +3d but not showing up in status  - am I missing something?21:09
jeblairmordred: yeah, no +1 vote; needs a recheck to go through check pipeline21:09
mordredah21:09
mordredcool21:10
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Shift nodepool quota from v2 to v3  https://review.openstack.org/50722221:10
*** thorst has joined #openstack-infra21:11
*** rwsu has joined #openstack-infra21:12
mordredinfra-root: I've updated the etherpad list of steps - and split the rollback section into hard and soft (soft meaning rollback to today's state - hard meaning remove all the projects from v3)21:12
*** LeoBud has joined #openstack-infra21:13
*** bnemec has quit IRC21:14
*** rossella_s has joined #openstack-infra21:14
*** thorst has quit IRC21:15
*** jpena|off has quit IRC21:16
*** ltomasbo has quit IRC21:16
*** amoralej has quit IRC21:16
mordredjeblair: so - the noop jobs patch: https://review.openstack.org/#/c/507860/ - I'm worried that once we flip the switch we'll stop being able to land patches to project-config because of the various job exclusions - how about I grab that patch and also add the legacy-bindep-fallback changes to it and we land that before we flip the switch so that switch-flipping doesn't impact project-config's21:16
mordredoperation21:17
jeblairmordred: why bring bindep into it?  why not just merge noop?21:17
*** ltomasbo has joined #openstack-infra21:18
*** jpena|off has joined #openstack-infra21:18
jeblairtrying to avoid churn on the big patches21:18
*** amoralej has joined #openstack-infra21:18
*** LeoBud has quit IRC21:20
mordredjeblair: mostly was thinking about rollback - but kk21:22
clarkbso I understand the noop jobs patch we need that so that we run something if nothing else were to run?21:22
jeblairmordred: i think we can leave noop in even after rollback21:23
clarkbdoes this still have the problem of putting everything with noop jobs into a common gate pipeline?21:23
openstackgerritMerged openstack-infra/project-config master: Add tripleo-test-cloud-rh1 into nl01  https://review.openstack.org/50727521:23
jeblairclarkb: no it doesn't have that problem, but we should still clean up our jobs and remove it because it's bad form and silly.21:23
mordredjeblair: yah - I meant getting bindep-fallback jobs out so that we're not running 6 un-needed jobs if we do rollback - but yes- let's not churn the big patch21:23
jeblairclarkb: we're mostly entertaining it here because so much stuff is in flux.21:24
mordred++21:24
clarkbgotcha21:24
*** e0ne has quit IRC21:24
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Add noop-jobs as a fallback for project-config  https://review.openstack.org/50786021:24
*** aeng has joined #openstack-infra21:24
jeblairpublish-openstack-specs-site failed: http://logs.openstack.org/7c/62c3f92e19e7282959de85e75bcb84166562597c/post/publish-openstack-specs-site/aa7b784/job-output.txt.gz21:28
*** yamamoto has joined #openstack-infra21:28
clarkbI think I saw a patch for that problem yesterday21:29
*** thorst has joined #openstack-infra21:29
clarkbwe don't seem to log the tox command though?21:29
openstackgerritMerged openstack-infra/project-config master: Add OVH to nodepool v3  https://review.openstack.org/50798421:30
jeblairclarkb: :(  it's in the .json file though -- tox -evenv -vv python setup.py build_sphinx21:30
*** srobert_ has joined #openstack-infra21:31
mordredjeblair: that should have been fixed by https://review.openstack.org/#/c/507678/21:31
clarkbhttp://git.openstack.org/cgit/openstack/openstack-specs/tree/setup.py the file is there though21:32
*** slaweq_ has quit IRC21:32
jeblairmordred: it's not a siblings file issue21:32
mordredclarkb: this isn't for openstack-specs21:32
mordredthe issue now will be that that job uses openstack-tox-docs as a base job - one sec21:32
jeblairya that sounds more like it; i imagine this job needs to be specialer21:33
mordredyu[21:33
*** ihrachys has quit IRC21:33
*** thorst has quit IRC21:34
jeblairactually, it's not parentage (its parent is base); it just re-uses playbooks from python-docs21:34
*** yamamoto has quit IRC21:34
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Don't use the docs cantrip for specs-site  https://review.openstack.org/50798921:35
mordredjeblair: yes. ^^21:35
*** srobert has quit IRC21:35
mordredjeblair: the main thign is that it certainly shouldn't be running python setup.py build_sphinx21:35
*** srobert_ has quit IRC21:36
*** yamahata has quit IRC21:36
*** yamahata has joined #openstack-infra21:37
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Emit a message about the final tox invocation  https://review.openstack.org/50799021:37
mordredjeblair, clarkb: that should help debugging issues with tox jobs without having to go to the json21:37
jeblairmordred: why don't we have zuul_stream emit the command?21:38
mordredjeblair: because it's VERY chatty if we do it for every command21:39
clarkbis it worse than bash set -x is/was21:39
*** andreww has quit IRC21:39
mordredjeblair: we could also remove the name: from that task though - which should cause it to emit the command since there is no name21:39
openstackgerritMerged openstack-infra/project-config master: Add noop-jobs as a fallback for project-config  https://review.openstack.org/50786021:40
clarkbmordred: I both like that idea adn not like it, the nice thing about named task is their grepability21:40
clarkbbut that is a simple fix for getting logging behavior we do want21:40
mordredclarkb: yah21:40
jeblairmordred: i'd like to spend some more time on this once you refactor zuul_stream and add testing21:41
mordredagre21:41
*** xarses has joined #openstack-infra21:42
*** dprince has quit IRC21:42
jeblairbecause i feel like "this is a command i ran" is something i almost always want to know, even more than many things which are currently in the streaming log.  so if it's too chatty, i'd like to be able to explore how to get what we need out of it.  as it is, i almost always have to go to json to debug, and i don't think adding ansible debug lines in strategic places is scalable.21:42
*** jascott1 has joined #openstack-infra21:43
*** dizquierdo has joined #openstack-infra21:43
jeblairmordred: i think we're no longer waiting on anything, is that right?21:45
clarkbI was just going to ask the same thing :)21:45
*** dave-mccowan has quit IRC21:45
mordredjeblair: correct21:45
mordredI believe we're ready to rock and roll21:45
jeblairmordred: and if my clock is right -- you're ready to eod, so should we pick this up tomorrow?  :)21:46
mordredjeblair: we keep saying that :)21:46
mordredjeblair: have I mentioned how much I did not enjoy today's migration script bug?21:46
* Shrews echos mordred's disdain for said bug21:47
mordredjeblair: but yes - I need to EOD in 45 minutes (we're taking scuba classes and they do not enjoy it when we're late)21:47
mordredjeblair: fwiw, I have no evening things tomorrow or friday - so as soon as we don't  need me to be able to squeeze an extra couple of hours in, I'll be free to do so21:47
*** wolverineav has joined #openstack-infra21:47
*** kjackal_ has quit IRC21:48
mordredthat said - we can probably judge immediate rollback need in 45 minutes21:48
clarkbI'm good with holding off too, I expect we'll be fielding lots of questions and working to iron out bugs after the switch and well having a good chunk of the day to that would be nice21:48
clarkbas well as extra eyeballs to untangle things21:49
mordredyah21:49
jeblairi agree with clarkb21:49
openstackgerritMerged openstack-infra/project-config master: Don't use the docs cantrip for specs-site  https://review.openstack.org/50798921:49
*** jascott1 has quit IRC21:51
*** andreww has joined #openstack-infra21:51
mordredkk. I shall write email update21:51
*** xarses has quit IRC21:52
*** jascott1 has joined #openstack-infra21:53
openstackgerritAndrea Frittoli proposed openstack-infra/devstack-gate master: Basic processing of test results  https://review.openstack.org/50798021:53
mordredinfra-root: https://etherpad.openstack.org/p/OmUqeXyu4R - that look ok for an update?21:54
*** baoli has quit IRC21:54
jeblairmordred: lgtm21:54
jeblairthe specs site job got farther -- http://logs.openstack.org/34/9480d876595193ee72ca3a1935213e261d42aa34/post/publish-openstack-specs-site/c1a5cbf/job-output.txt.gz#_2017-09-27_21_52_28_50495421:55
clarkbI'll even try to wake early so that west coast is ready early :)21:55
clarkbmordred: so message lgtm21:55
jeblairlooks like we need to collect the index from a different directory21:56
*** jkilpatr has joined #openstack-infra21:56
clarkboutdir = os.path.join(template_path, 'output')21:57
jeblairyah, and: 2017-09-27 21:52:21.307878 | ubuntu-xenial | Writing '/home/zuul/src/git.openstack.org/openstack-infra/project-config/specs/output/index.html'21:57
*** jascott1 has quit IRC21:57
*** yamamoto has joined #openstack-infra21:57
clarkbspecs/output?21:57
*** mriedem is now known as mriedem_away21:57
*** yamamoto has quit IRC21:57
*** dave-mccowan has joined #openstack-infra21:57
*** thorst has joined #openstack-infra21:58
*** kjackal_ has joined #openstack-infra21:58
mordredyah - need to tell it a diffrent src location ... patch coming21:58
*** dizquierdo has quit IRC21:59
*** hashar has quit IRC22:00
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Grab spec site content from where it actually comes  https://review.openstack.org/50799622:01
mordredclarkb, jeblair: ^^22:01
jeblairmordred: sure about no trailing / ?22:02
*** inc0 has quit IRC22:02
*** slaweq_ has joined #openstack-infra22:03
*** andreww has quit IRC22:03
*** dave-mccowan has quit IRC22:03
mordredjeblair: nope!22:03
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Grab spec site content from where it actually comes  https://review.openstack.org/50799622:03
jeblairoh weird, that's actually not the default for that role22:04
jeblairthe default has no trailing /22:04
*** xarses has joined #openstack-infra22:04
*** kjackal_ has quit IRC22:05
jeblairmaybe that's because by default, the role is being used for docs-draft, so ending up in a subdir is desired...22:05
jeblairso actually thinking this one through -- i'm guessing that we *do* want a trailing / and ps2 is good.22:06
jeblairthat's where i'm putting my nickel.22:06
mordredjeblair: \o/22:07
*** rwsu has quit IRC22:07
clarkbIm gonna have to read the manpage22:08
*** Goneri has quit IRC22:08
clarkbya trailing / is right22:08
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Expand docs for fetch-sphinx-output role  https://review.openstack.org/50799922:08
jeblairclarkb, mordred: ^ missing docs (ironic!)22:09
*** andreww has joined #openstack-infra22:09
*** xarses has quit IRC22:09
*** rossella_s has quit IRC22:10
openstackgerritMerged openstack-infra/project-config master: Grab spec site content from where it actually comes  https://review.openstack.org/50799622:17
pabelangerHmm, I think nl01 is not happy22:18
pabelangerKeyError: 'tripleo-centos-7'22:19
pabelangeron nodepool list22:19
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Add missing label for tripleo-centos-7  https://review.openstack.org/50800422:20
pabelangerand fix22:20
pabelangeros_client_config.exceptions.OpenStackConfigException: Region BHA1 is not a valid region name for cloud ovh. Valid choices are BHS1,GRA1. Please note that region names are case sensitive.22:21
pabelangergetting that on nl0222:21
fungiokay, finished with dinner and catching up22:22
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Fix typo with ovh-bhs1  https://review.openstack.org/50800522:23
pabelangerand nl02 fix22:23
*** rossella_s has joined #openstack-infra22:23
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Only run tox-siblings on required projects  https://review.openstack.org/50795522:23
*** rlandy is now known as rlandy|afk22:26
pabelangerI've added 508004 and 508005 to blockers section on zuulv3-cutover etherpad22:26
jeblairmordred, clarkb: specs site run looks good and specs.o.o has a recent update time now22:27
jeblairso!  next failure: http://logs.openstack.org/97/4102e7b6abcd6d5f5245e11597e0c5693111ab97/post/publish-openstack-python-docs-infra-index/b9db5a6/job-output.txt.gz22:27
jeblairpabelanger: +2+222:28
fungireviewing the nodepool config fixes since those seem urgent22:28
jeblairlooks like basically the same error22:29
fungialso, agreed on this being a good point to pause and then try to throw the freakishly big dr. frankenstein's lab like knife switch tomorrow and cackle insanely22:30
*** krtaylor has quit IRC22:30
*** inc0 has joined #openstack-infra22:30
pabelangertox -evenv python setup.py build_sphinx also doen't work for me locally on project-config22:31
pabelangerlooks like it should be: tox -einfra-docs22:32
jeblairpabelanger: yeah, i'm working on a patch22:33
pabelangerjeblair: kk22:34
*** slaweq_ has quit IRC22:36
*** iyamahat has quit IRC22:39
*** slaweq_ has joined #openstack-infra22:39
*** ari[m] has joined #openstack-infra22:40
openstackgerritMerged openstack-infra/project-config master: Add missing label for tripleo-centos-7  https://review.openstack.org/50800422:40
*** krtaylor has joined #openstack-infra22:43
*** jaypipes has quit IRC22:48
*** abelur_ has quit IRC22:49
*** abelur_ has joined #openstack-infra22:53
*** rossella_s has quit IRC22:56
*** bobh has joined #openstack-infra22:57
*** yamamoto has joined #openstack-infra22:58
openstackgerritJames E. Blair proposed openstack-infra/openstack-zuul-jobs master: Add a role to write a root marker  https://review.openstack.org/50801023:00
openstackgerritJames E. Blair proposed openstack-infra/project-config master: Fix publish infra docs index job  https://review.openstack.org/50801123:00
jeblairyowza that wasn't easy.23:00
jeblairpabelanger, clarkb: ^23:00
*** threestrands has joined #openstack-infra23:00
*** threestrands has quit IRC23:00
*** threestrands has joined #openstack-infra23:00
*** rtjure has quit IRC23:01
*** bobh has quit IRC23:02
*** claudiub|3 has quit IRC23:03
*** andreww has quit IRC23:04
*** rbrndt has quit IRC23:05
*** yamamoto has quit IRC23:05
*** lukebrowning has quit IRC23:08
*** slaweq_ has quit IRC23:13
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove gratuitous use of xargs  https://review.openstack.org/50769123:16
*** baoli has joined #openstack-infra23:18
*** slaweq_ has joined #openstack-infra23:18
*** baoli_ has joined #openstack-infra23:22
*** baoli has quit IRC23:22
*** lukebrowning has joined #openstack-infra23:23
*** hongbin has quit IRC23:25
*** lukebrowning has quit IRC23:28
*** felipemonteiro_ has quit IRC23:29
*** lukebrowning has joined #openstack-infra23:29
*** thorst has quit IRC23:30
*** mriedem_away is now known as mriedem23:31
*** Sukhdev has joined #openstack-infra23:32
*** lukebrowning has quit IRC23:34
*** stakeda has joined #openstack-infra23:34
*** krtaylor has quit IRC23:35
*** lukebrowning has joined #openstack-infra23:36
*** srobert has joined #openstack-infra23:39
*** lukebrowning has quit IRC23:40
*** dizquierdo has joined #openstack-infra23:41
*** lukebrowning has joined #openstack-infra23:42
*** ijw has joined #openstack-infra23:43
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Add a role to write a root marker  https://review.openstack.org/50801023:44
clarkbjeblair: ^23:45
*** ijw has quit IRC23:45
*** ijw has joined #openstack-infra23:45
*** tpsilva has quit IRC23:46
*** lukebrowning has quit IRC23:47
*** slaweq_ has quit IRC23:48
*** lukebrowning has joined #openstack-infra23:48
*** krtaylor has joined #openstack-infra23:51
*** krtaylor has quit IRC23:51
*** slaweq_ has joined #openstack-infra23:52
*** bnemec has joined #openstack-infra23:52
*** lukebrowning has quit IRC23:53
*** lukebrowning has joined #openstack-infra23:54
*** iyamahat has joined #openstack-infra23:56
*** xarses has joined #openstack-infra23:58
*** r-daneel has quit IRC23:58
*** yamahata has quit IRC23:58
*** lukebrowning has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!