Monday, 2019-09-30

*** jamesmcarthur has joined #zuul00:41
*** jamesmcarthur has quit IRC00:48
*** jamesmcarthur has joined #zuul00:49
*** jamesmcarthur has quit IRC00:54
*** jamesmcarthur has joined #zuul00:55
*** jamesmcarthur has quit IRC01:10
*** jamesmcarthur has joined #zuul01:11
*** jamesmcarthur has quit IRC01:16
*** jamesmcarthur has joined #zuul01:21
*** swest has quit IRC01:23
*** swest has joined #zuul01:38
*** jamesmcarthur has quit IRC02:02
*** bhavikdbavishi has joined #zuul02:46
*** bhavikdbavishi1 has joined #zuul02:53
*** bhavikdbavishi has quit IRC02:54
*** bhavikdbavishi1 is now known as bhavikdbavishi02:54
*** jamesmcarthur has joined #zuul03:45
*** toabctl has joined #zuul04:07
*** jamesmcarthur has quit IRC04:14
*** swest has quit IRC04:24
*** bolg has joined #zuul04:29
*** jlvillal has joined #zuul05:21
*** swest has joined #zuul05:21
*** pcaruana has joined #zuul05:24
*** pcaruana has quit IRC05:30
*** raukadah is now known as chandankumar05:33
*** jamesmcarthur has joined #zuul06:16
*** jamesmcarthur has quit IRC06:20
*** bhavikdbavishi has quit IRC06:22
*** bhavikdbavishi has joined #zuul06:29
*** pcaruana has joined #zuul06:41
openstackgerritSimon Westphahl proposed zuul/zuul master: Add optional support for circular dependencies  https://review.opendev.org/68535407:05
*** tosky has joined #zuul07:11
openstackgerritSimon Westphahl proposed zuul/zuul master: Add optional support for circular dependencies  https://review.opendev.org/68535407:19
*** arxcruz|zzz is now known as arxcruz|ruck07:25
*** tosky has quit IRC07:26
*** tosky has joined #zuul07:27
*** jpena|off is now known as jpena07:29
*** mhu has joined #zuul07:39
*** hashar has joined #zuul07:57
*** themroc has joined #zuul08:05
*** themroc has quit IRC08:17
*** themroc has joined #zuul08:24
*** hashar has quit IRC08:31
*** themr0c has joined #zuul08:43
*** themroc has quit IRC08:44
*** gtema_ has joined #zuul08:48
*** themr0c has quit IRC08:49
openstackgerritSimon Westphahl proposed zuul/zuul master: Add optional support for circular dependencies  https://review.opendev.org/68535408:53
*** jangutter has joined #zuul09:13
*** themroc has joined #zuul09:14
*** smcginnis has quit IRC09:14
openstackgerritMerged zuul/zuul-jobs master: Use iptables for openSUSE  https://review.opendev.org/68549109:21
*** zbr is now known as zbr|ruck09:24
*** tobiash has quit IRC09:27
*** tobiash has joined #zuul09:29
*** gtema_ has quit IRC10:14
*** tosky has quit IRC10:17
*** tosky has joined #zuul10:18
openstackgerritSimon Westphahl proposed zuul/zuul master: Add optional support for circular dependencies  https://review.opendev.org/68535410:32
*** bolg has quit IRC10:32
*** bolg has joined #zuul10:34
*** hashar has joined #zuul10:35
*** bolg has quit IRC10:38
*** bolg has joined #zuul10:38
*** bhavikdbavishi has quit IRC10:46
openstackgerritFabien Boucher proposed zuul/zuul master: WIP - gitlab - bootstrap the driver structure + Webhook support  https://review.opendev.org/68568211:10
*** chandankumar is now known as raukadah11:11
*** jpena is now known as jpena|lunch11:31
*** wxy-xiyuan has quit IRC11:37
*** ianychoi has quit IRC11:39
*** hashar has quit IRC11:39
*** ianychoi has joined #zuul11:40
*** bhavikdbavishi has joined #zuul11:53
*** hashar has joined #zuul11:55
*** bhavikdbavishi1 has joined #zuul11:56
*** bhavikdbavishi has quit IRC11:57
*** bhavikdbavishi1 is now known as bhavikdbavishi11:58
*** mariowood has joined #zuul12:07
*** jamesmcarthur has joined #zuul12:26
*** jpena|lunch is now known as jpena12:30
*** rfolco has joined #zuul12:33
*** jamesmcarthur has quit IRC12:35
*** bolg has quit IRC13:09
*** ironfoot has joined #zuul13:34
*** bolg has joined #zuul13:34
pabelangermordred: is your gerrit talk about zuul online some place?13:44
*** jangutter has quit IRC13:53
*** mariowood has quit IRC13:54
*** brendangalloway has joined #zuul13:56
Shrewspabelanger: i think he keeps all of his talks on inaugust.com13:57
Shrewszuul-maint: Umm, is anyone else seeing strange things on the zuul-ci.org website? Both FF and Safari are presenting the top header of the page oddly14:00
clarkbShrews: can confirm. I think that implies broken css14:01
Shrewswheee14:01
Shrewsguessing https://review.opendev.org/68046514:02
clarkbhttps://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_e02/680465/4/gate/zuul-website-build/e026ca8/html/ seems to do the same (the build for that change)14:03
clarkbif you push a revert we can check the build to see if it fixes14:03
Shrewsyeah, just looked at that too14:04
Shrewsrevert coming14:04
openstackgerritDavid Shrewsbury proposed zuul/zuul-website master: Revert "CSS fix for ul/li in FAQ"  https://review.opendev.org/68571714:04
*** jangutter has joined #zuul14:08
Shrewsclarkb: seems to fix it: https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/zuul_opendev_logs_875/685717/1/check/zuul-website-build/8754ced/html/14:11
clarkbShrews: fast approved14:13
openstackgerritMerged zuul/zuul-website master: Revert "CSS fix for ul/li in FAQ"  https://review.opendev.org/68571714:17
*** pcaruana has quit IRC14:18
Shrewszuul-ci.org seems back to normal now14:27
*** themroc has quit IRC14:37
*** hashar has quit IRC14:39
fungithanks for noticing/fixing Shrews!14:39
openstackgerritTristan Cacqueray proposed zuul/zuul master: Replace tox-py36 by tox-py37  https://review.opendev.org/68215815:03
clarkbtristanC: why replace?15:04
clarkbcentos and rhel 7/8 have 3.6. Seems like that is an important version to test?15:05
corvusclarkb: oh i was thinking we only needed to test oldest/newest15:06
clarkbmaybe? certainly oldest and newest should avoid adding in syntax that won't work in the middle15:07
tristanCclarkb: i think it's fine to not test 3.6, i've never seen something that works in 3.5 and 3.7 to not work in 3.615:07
corvustristanC said he hasn't seen anything caught by 3.6 that isn't caught by 3.7 (which would be my expectation too)15:07
clarkbfwiw I use 3.7 myself and haven't had any problems going that direction at least. So ya probably is fine15:11
*** armstrongs has joined #zuul15:12
*** armstrongs has quit IRC15:16
openstackgerritMerged zuul/zuul master: Support nodes setting 'auto' python-path  https://review.opendev.org/68227515:18
*** bolg has quit IRC15:23
openstackgerritMerged zuul/zuul master: Annotate cancelJobInQueue with build id  https://review.opendev.org/68437315:25
*** mattw4 has joined #zuul15:33
*** jamesmcarthur has joined #zuul15:41
*** pcaruana has joined #zuul15:46
*** brendangalloway has quit IRC16:30
SpamapSThe difference between 3.6 and 3.7 is all additional features. I'd definitely trust that 3.7 testing covered 3.6. The only corner cases would be things like that old bug we found forever ago in the Ubuntu python packages... it's nice to test on a real build of the interpreter.16:33
mordredour docker images are also built on top of latest python base image, which is 3.7.3 - and based on upstream python not distro python16:34
openstackgerritFabien Boucher proposed zuul/zuul master: Gitlab - bootstrap the driver structure + Webhook support  https://review.opendev.org/68568216:35
clarkbthat doesn't help everyone deploying on bionic, centos and rhel 7/8 though16:35
mordredso I'm also on board with doing 35 + 3716:35
clarkbI think it is enough of a corner case to not worry about it until it becomes a problem though16:35
mordredclarkb: yah16:35
mordredI think we'd want to replace 3.5 with 3.6 at whatever point we decide to stop supporting 3.516:36
mordredbut while we have 3.5 support - just testing 5 and 7 seems sufficient16:36
Shrewsjamesmcarthur: hi! not sure if you saw, but we had to revert your css change this morning: https://review.opendev.org/68571716:37
mordredtristanC: I think you need a different base image for that 3.7 patch to work16:37
clarkbmordred: bionic actually has 3.7 packages but you have to install them more explicitly16:41
openstackgerritTristan Cacqueray proposed zuul/zuul master: Replace tox-py36 by tox-py37  https://review.opendev.org/68215816:41
jamesmcarthurShrews: I did.  My apologies. There is a missing }16:43
jamesmcarthurI'll update the patch after I'm done with my current sprint of meetings16:43
*** jpena is now known as jpena|off16:45
fungiclarkb: bionic has packages for 3.7, so in theory users could deploy with it there fairly easily16:48
tristanCi picked fedora-latest as python3.7 is the default there16:49
clarkbfungi: that is true.16:49
clarkbmostly centos and rhel 7/8 then16:49
*** zbr|ruck is now known as zbr16:52
*** igordc has joined #zuul17:08
pabelangerwe are doing python36 for zuul.a.c, so would be nice to keep testing on that17:10
pabelangerbut so far, haven't really seen python36 specific issues17:11
ShrewsDoes anyone plan to do any sort of writeup about zuul related things at last week's AnsibleFest?17:27
clarkbin the last 7 days that ansible iptables var not being defined issue in persist-firewall role has hit us 208 times according to logstash. This is definitely not happening on every job which makes it extra weird. Also we retry these jobs because the failure happens in pre-run so not end of the world but it would be good to track that down17:28
clarkbpabelanger: ^ I think you maybe had ideas to try?17:28
clarkbpabelanger: it happens on different clouds and on different distros and in different jobs17:28
pabelangerclarkb: was that the -13 return code?17:30
clarkbpabelanger: ya rc -13 of the forked process not of ansible itself 2019-09-30 14:37:25,375 DEBUG zuul.AnsibleJob.output: [e: 53c73870b5b343b6a6c883a388b0070d] [build: bafbb86f7b924f35a595edd2d4aa7196] Ansible output: b'ok: [primary] => {"changed": false, "failed_when_result": false, "module_stderr": "", "module_stdout": "", "msg": "MODULE FAILURE\\nSee stdout/stderr for the exact error", "rc": -13}'17:32
clarkbansible exits 217:32
pabelangeryah, let me look at my notes again. I think I had an idea what it was, but post conference brain is hitting me17:32
clarkband it only happens on the second run on the persistent-firewals role17:32
pabelangerright, i think something to do with including same role more then once might be the key17:33
fungiShrews: i have some notes i still need to digest but could probably integrate into an ml post. it was an interesting experience, exciting to talk to so many unknown current and potential zuul users, and also interesting to contrast ansible community development against our own17:33
clarkbpabelanger: perhaps we can use import_role so that the role is less dynamic?17:35
pabelangeryah, I also think the usage of our variables might be related. eg: them crossing playbook bounds, over passing into role17:36
pabelangerI haven't testing import_role much, but worth a try17:36
clarkbother ideas: we can use shell instead of comamnd. maybe iptables-save needs to run in a more fully fledged environment?17:36
fungiShrews: i was particularly surprised that the community is told at the contributor summit what the secretly designed development roadmap holds for them in the future17:36
clarkbpabelanger: we don't pass anything into the role17:36
clarkbpabelanger: the role takes the current iptables ruleset and persists it to disk17:36
pabelangerclarkb: if we could figure out why SIGPIPE would be called, that would be helpful17:36
clarkbthere is no input17:36
pabelangerclarkb: need to look again, but I think we register some variables in the top level playbook, then the role references them17:37
Shrewsfungi: i'm sure a ML post about your thoughts/experiences would be well received  :)17:37
clarkbpabelanger: not that I see. The role appears to be fully self contained17:38
clarkbpabelanger: I'm going to push up a change to use import_role to see if it works at all17:38
pabelangerack17:39
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Use import_role on persistent-firewall  https://review.opendev.org/68576517:40
clarkbpabelanger: filename:"job-output.txt" AND message:"The task includes an option with an undefined variable." is my logstash query if you want to see more hits too17:45
clarkbimport_role seems to work in testing https://b3b0b103e21a776f5487-60046e015843f13b22f3609583ac17fc.ssl.cf1.rackcdn.com/685765/1/check/zuul-jobs-test-multinode-roles-centos-7/85017e7/job-output.txt 'List current ipv4 rules' is executed three times in a single play there17:55
clarkbat the very least it doesn't seem to regress17:55
clarkbmaybe we can merge that then check logstash to see if the behavior changes any, and if it doesn't then replace command with shell next?17:55
openstackgerritTristan Cacqueray proposed zuul/zuul master: WIP: Replace tox-py36 by tox-py37  https://review.opendev.org/68215818:19
*** bhavikdbavishi has quit IRC18:33
*** hashar has joined #zuul18:36
*** jamesmcarthur has quit IRC19:01
*** jamesmcarthur has joined #zuul19:06
openstackgerritTristan Cacqueray proposed zuul/zuul master: WIP: Replace tox-py36 by tox-py37  https://review.opendev.org/68215819:21
*** jamesmcarthur has quit IRC19:34
clarkbany other zuul jobs reviewers want to review https://review.opendev.org/#/c/685765/ we can track that with logstash to see if it helps once it lands19:35
* clarkb finds lunch19:35
*** jamesmcarthur has joined #zuul19:41
openstackgerritTristan Cacqueray proposed zuul/zuul master: WIP: Replace tox-py36 by tox-py37  https://review.opendev.org/68215819:51
openstackgerritTristan Cacqueray proposed zuul/zuul master: WIP: Replace tox-py36 by tox-py37  https://review.opendev.org/68215820:04
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Escape special filename characters in urls  https://review.opendev.org/68578520:27
openstackgerritTristan Cacqueray proposed zuul/zuul master: WIP: Replace tox-py36 by tox-py37  https://review.opendev.org/68215820:27
openstackgerritJames E. Blair proposed zuul/zuul-registry master: Initial implementation  https://review.opendev.org/68395820:36
corvusclarkb, tristanC: ^ i think that's the complete first stage -- it implements an authenticated+ssl registry with local and swift storage backends.  that's enough to run an intermediate registry.  next stage is building on that to add the pass-through support to make it a buildset registry.20:38
corvusbut before we do that, i think we can add tests, docker builds, and put it into production as the opendev intermediate registry.  hopefully that will fix our zero-byte upload issue (or at least make it possible to debug).20:39
corvusShrews: i just noticed your comments on the previous ps of that; i'll take a look now.20:39
corvusShrews: ayup.  i'll take care of adding docstrings in a followup, and squash it if i finish it before there are further reviews.20:43
pabelangercool, was just looking into docker registery stuff too! So may also be able to give some feedback of zuul-registry soon if needed too20:48
tristanCcorvus: that LGTM, the root dispatcher is very instructing20:48
*** pcaruana has quit IRC20:50
Shrewscorvus: the biggest issue for me was the lack of an abstract protocol class for the storage stuff20:51
*** irclogbot_2 has quit IRC20:54
*** irclogbot_0 has joined #zuul20:57
*** irclogbot_0 has quit IRC21:01
*** irclogbot_3 has joined #zuul21:03
clarkbI've pushed https://review.opendev.org/685792 to track the perstistent-firewall issue with e-r21:05
*** irclogbot_3 has quit IRC21:07
*** irclogbot_2 has joined #zuul21:09
*** jamesmcarthur has quit IRC21:11
openstackgerritClark Boylan proposed zuul/zuul-jobs master: Escape special filename characters in urls  https://review.opendev.org/68578521:12
*** irclogbot_2 has quit IRC21:13
*** jamesmcarthur has joined #zuul21:17
corvusi caught a couple cleanups i missed, so i'll definitely do this as a new revision21:21
*** irclogbot_0 has joined #zuul21:22
openstackgerritTristan Cacqueray proposed zuul/zuul master: Replace tox-py36 by tox-py37  https://review.opendev.org/68215821:22
*** irclogbot_0 has quit IRC21:25
openstackgerritJames E. Blair proposed zuul/zuul-registry master: Initial implementation  https://review.opendev.org/68395821:25
corvustristanC, clarkb, Shrews: ^ for realz this time :)21:26
*** irclogbot_0 has joined #zuul21:27
*** irclogbot_0 has quit IRC21:33
openstackgerritJeremy Stanley proposed zuul/zuul-website master: Link to instructions on vulnerability reporting  https://review.opendev.org/68579921:35
clarkbfungi: (and others) https://review.opendev.org/#/c/685785/ passes tests now and should fix the problem with :'s in filenames21:36
fungithanks!21:37
*** irclogbot_0 has joined #zuul21:37
clarkbsr21:40
clarkbderp21:40
*** irclogbot_0 has quit IRC21:41
corvusi'm working on the docker image for the registry now21:45
*** irclogbot_3 has joined #zuul21:50
openstackgerritMerged zuul/zuul-jobs master: Escape special filename characters in urls  https://review.opendev.org/68578521:52
*** irclogbot_3 has quit IRC21:53
*** irclogbot_3 has joined #zuul21:55
*** irclogbot_3 has quit IRC21:59
openstackgerritJames E. Blair proposed zuul/zuul-registry master: Fix container image build  https://review.opendev.org/68580821:59
*** irclogbot_0 has joined #zuul22:01
*** irclogbot_0 has quit IRC22:05
*** irclogbot_0 has joined #zuul22:07
*** irclogbot_0 has quit IRC22:15
*** irclogbot_0 has joined #zuul22:17
*** tosky has quit IRC22:24
clarkbhttp://status.openstack.org/elastic-recheck/index.html#1846093 is now tracking the thing that https://review.opendev.org/#/c/685765/ aims to influence22:33
clarkbI'm able to watch that go in if I can get reviews22:33
corvusclarkb: do you have a link to something where i can read about the reasoning there?  (it seems a little voodoo to me)22:34
corvusclarkb: (also, why do we think ansible telling us that "iptables returned -13" means "ansible broke" rather than "iptables broke"?)22:35
clarkbcorvus: fwiw its a lot of voodoo to me. Basically pabelanger and I noticed that it only happens on the second invocation of that role. Its always fine on the first run. We've noticed problems with recurring and nested includes in the past so this is a shot in the dark22:36
clarkbI'm trying to find the IRC log from before ansiblefest22:37
corvusi'm reading https://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse.html#dynamic-vs-static22:38
clarkbcorvus: http://eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2019-09-18.log.html#t2019-09-18T17:23:38 that was pabelangers analysis from a couple weeks ago22:39
corvusclarkb: ok.  i'm doubtful that this is the problem (based on the documentation, there should be no difference here), and all things being equal, i think we'd want to default to include_role as a habit, so we're not debugging statically assigned role arguments.  but i also value your hunch that this may be an ansible bug, and making this change would help confirm or exclude that.  so i'm +2 on this for that22:42
corvuspurpose, but can we switch this back if we are able to exclude this theory?22:42
clarkbcorvus: ya that is why I got the e-r bug up too. If this doesn't change the occurence of this bug we should revert22:43
corvusclarkb: sounds good, +322:43
clarkbcorvus: comment on https://review.opendev.org/#/c/685808/122:43
corvusclarkb: and thanks for the catch-up :)22:44
clarkbI'm working through the registry change now22:51
fungithe lambdas of lambdas are a bit mind-bending22:56
openstackgerritMerged zuul/zuul-jobs master: Use import_role on persistent-firewall  https://review.opendev.org/68576523:17
*** mattw4 has quit IRC23:26
clarkbcorvus: comments posted on the registry change23:39
openstackgerritJames E. Blair proposed zuul/zuul-registry master: Fix container image build  https://review.opendev.org/68580823:40
corvusokay, that should have the container build working, and it also adds a functional test23:40
*** hashar has quit IRC23:40
*** armstrongs has joined #zuul23:49
corvusclarkb: thx, responded -- some need to be fixed so i -1d till i get to that tomorrow23:55
*** jamesmcarthur has quit IRC23:57
*** jamesmcarthur has joined #zuul23:58
*** armstrongs has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!