Wednesday, 2018-05-02

*** acozine1 has quit IRC00:04
fungias has become typical of the summits, there's going to be an openstack foundation lounge where attendees who have random questions (really any kind you can imagine and a lot more you can't) and want to drop by can try to get answers... now that there's zuul and kata projects we're talking about trying to get some people from those communities to try and hang out in the lounge area when available... something to00:31
fungikeep in mind if any zuulies are interested in helping with that00:31
*** acozine1 has joined #zuul00:36
openstackgerritClark Boylan proposed openstack-infra/zuul master: Fix layout generation for trusted projects  https://review.openstack.org/56562400:54
*** acozine1 has quit IRC01:06
*** swest has quit IRC01:06
*** swest has joined #zuul01:22
*** rlandy has quit IRC01:31
*** spsurya has joined #zuul02:10
openstackgerritMerged openstack-infra/zuul master: Fix setting a change queue in a template  https://review.openstack.org/56558104:20
*** swest has quit IRC04:24
*** elyezer has quit IRC04:24
*** elyezer has joined #zuul04:25
tobiashclarkb, corvus: question on 56562404:33
clarkbtobiash: that is a good question I will sleep on it04:38
*** swest has joined #zuul05:02
*** bhavik1 has joined #zuul05:05
*** swest has quit IRC05:06
*** swest has joined #zuul05:21
*** bhavik1 has quit IRC05:25
SpamapSfungi: that's good to know, I'll try and hang in that lounge.05:29
openstackgerritFelix Schmidt proposed openstack-infra/zuul master: Add start and end timestamp to task and play result in zuul_json callback  https://review.openstack.org/56388805:38
openstackgerritMerged openstack-infra/zuul master: Fix regex project templates  https://review.openstack.org/56558405:44
*** elyezer has quit IRC06:05
*** elyezer has joined #zuul06:06
*** sshnaidm|afk is now known as sshnaidm06:41
*** bhavik1 has joined #zuul06:44
*** bhavik1 has quit IRC06:50
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: mqtt: add basic reporter  https://review.openstack.org/53554307:20
*** gtema has joined #zuul07:44
*** hashar has joined #zuul08:05
*** ssbarnea_ has joined #zuul08:08
*** xinliang has quit IRC08:42
*** xinliang has joined #zuul08:42
tobiashwohoo, just noticed that I can use upstream nodepool without any patches now :)10:44
*** xinliang has quit IRC11:08
tobiashcorvus, clarkb: any objection to +w 560700?11:20
*** ekan has joined #zuul12:05
ekanHi, I have a small question. Following install/config procedures on https://docs.openstack.org/infra/zuul/admin/zuul-from-scratch.html, I thought that zuul-web would provide a web-interface on 9000 (default). However, visiting <server>:9000 gives me a "Empty reply from server" error. Visiting ex. <server>:9000/api or similar gives back a 404. Just12:15
ekanwanted to jump in here to see if you guys have seen this before, or if it's something obvious that I'm missing...12:15
ekanLogs doesn't give me much, so if nothing else some kind of pointer to how to enable debug for zuul-web maybe?12:15
ekanThanks a ton in advance12:15
*** rlandy has joined #zuul12:32
rcarrillocruzekan: 9000 is the port to get webhooks from GH iirc12:49
rcarrillocruzalthough... i think that may no longer the case after all the web refactor12:51
rcarrillocruzmordred will know, he did all refactors of that code, there used to be to http endpoints12:52
rcarrillocruzs/to/two12:52
pabelangeryah, you'll need to POST for 9000, GET will fail13:08
pabelangeractually, 1 minute13:09
pabelangerI am wrong, only a sub path will require POST13:10
pabelangeryou should be able to do IP:9000/api/info13:10
*** dkranz has quit IRC13:11
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Update repo file for fedora-28  https://review.openstack.org/56575813:26
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Update repo file for fedora-28  https://review.openstack.org/56575813:39
*** hashar is now known as hasharAway13:53
clarkbtobiash: ya I think we can fallback to pipeline layoit there. Lets see what corvus says14:10
mordredekan, rcarrillocruz: there should just be a web dashboard at server:9000/ and it should contain the tenant list14:15
openstackgerritMerged openstack-infra/zuul master: reporter: do not expect branch attribute in Tag object  https://review.openstack.org/56070014:16
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Update repo file for fedora-28  https://review.openstack.org/56575814:16
mordredekan, rcarrillocruz: you know - we might be missing some instrucitons about installing the javascript build tools - lemme read14:16
mordredyup. that's missing - so the likely issue is that "sudo pip3 install ." worked, but since js build tools weren't there, it silently did not actually build the html/js14:17
corvusclarkb, tobiash: i agree.14:22
*** dkranz has joined #zuul14:23
clarkbcorvus: tobiash just rip out qll that logic and do else: layout = pipeline.layout?14:24
corvusclarkb: yeah, i think so.14:25
*** openstackgerrit has quit IRC14:34
*** openstackgerrit has joined #zuul14:35
openstackgerritClark Boylan proposed openstack-infra/zuul master: Fix layout generation for trusted projects  https://review.openstack.org/56562414:35
clarkbcorvus: tobiash ^ that should do it14:35
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Update repo file for fedora-28  https://review.openstack.org/56575814:39
dmsimardpabelanger: TIL about version_compare14:46
pabelangerall sorts of cool filters14:46
pabelangerpuppet has the same thing I believe14:47
mordredneat14:50
*** acozine1 has joined #zuul15:17
*** trishnag has joined #zuul15:20
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Update repo file for fedora-28  https://review.openstack.org/56575815:20
*** acozine1 has quit IRC15:29
*** acozine1 has joined #zuul15:29
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Update repo file for fedora-28  https://review.openstack.org/56575815:30
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Fix race in test_bubblewrap_leak  https://review.openstack.org/56581315:50
tobiashcorvus, clarkb: I think that should fix http://logs.openstack.org/24/565624/2/gate/tox-py35/a3016ff/testr_results.html.gz  ^15:50
corvustobiash: lgtm, thx15:50
clarkbtobiash: thanks. I'm at doctors office for the mroning so wasnt able to debug beyond not related to my change15:50
*** ssbarnea_ has quit IRC15:59
*** ssbarnea_ has joined #zuul16:00
*** gtema has quit IRC16:11
openstackgerritMerged openstack-infra/zuul master: Fix layout generation for trusted projects  https://review.openstack.org/56562416:15
openstackgerritMerged openstack-infra/zuul master: Fix race in test_bubblewrap_leak  https://review.openstack.org/56581316:24
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Update repo file for fedora-28  https://review.openstack.org/56575816:28
*** sshnaidm is now known as sshnaidm|rover16:57
*** myoung is now known as myoung|ruck16:59
*** weshay|ruck is now known as weshay17:00
*** openstackgerrit has quit IRC17:04
pabelangercould I get a review / +3 on ^ that fixes configure-mirror for fedora-28, while supporting fedora-27 still17:20
SpamapSSo..17:31
SpamapSwhen do we get Ansible 2.4+ in Zuul?17:31
SpamapSShould I start working on a multi-ansible-version spec?17:31
pabelangerSpamapS: Shrews suggested we go right to 2.5, I believe he has patches up17:32
corvusSpamapS: i think there's some WIP patches to upgrade to 2.4 and then to 2.5.  i'm not sure what the status or next steps are.  we should ask Shrews and tobiash.  i believe the thought is to go to 2.5 and then do multi-version after that -- just to keep things moving.17:33
corvusSpamapS: a multi-version spec would be great -- we should start talking about the mechanics of that.17:33
ShrewsThey're ready to go17:33
corvus(i feel like we can probably move to 2.5 while we work that out)17:33
pabelangeryan, so far is my testing 2.3 -> 2.5 ansible didn't break the world locally17:33
pabelangerbut we likey want all hands on deck to deal with edge cases fallout for jobs17:34
corvusShrews: so we're ready to send out a notice that version... 3.1.0? will use ansible 2.5, then merge them and release?17:34
SpamapSAre we sure that all 2.3 playbooks will work on 2.5?17:34
pabelangerno17:34
clarkbSpamapS: no, and previous experience says likely they wont17:34
pabelangerI'd expect some breakage17:34
pabelangerbut I haven't see anything major17:35
Shrewscorvus: yes, but I think the concern is the rollout17:35
corvusi think the main concern would be zuul-jobs -- are there things in zuul-jobs that won't work with 2.5?17:35
ShrewsI think that's part of the rollout concern17:38
pabelangerwe might also need to discuss how long we're going to support ansible 2.3 in zuul-jobs once we move to 2.5. Not that I know of anything that won't work17:39
corvuswell, if we can't predict in advance whether anything will be broken in zuul-jobs, then our two choices are to schedule a time to just do it and apologize that we're breaking everyone, or wait for multi-version support.17:40
corvusanyone have any other options?17:41
clarkbrun a single executor with newer ansible?17:41
clarkband then monitir it before full rollout maybe17:41
corvusclarkb: so, have openstack-infra perform the testing?17:41
pabelangermaybe setup 3pci with Shrews patches and run against devstack / tox-jobs to get coverage on zuul-jobs17:42
clarkbcorvus: ya17:42
clarkbor ^ should give reasonable coverage too17:43
corvuspabelanger: who's the third party in your suggestion?17:43
pabelangerI have access to some hardware locally I could use17:44
corvuspabelanger: how about instead, we make openstack-infra the (first) party, and run a zuul-dev with Shrews's patches?17:44
corvuswe can hook it up to openstack-infra's nodepool17:45
pabelangersure we can do that too17:45
*** electrofelix has quit IRC17:45
corvusthat's probably not too much harder than clarkb's idea, and maybe a little more managable?17:46
corvusso our options are now: (a) create a dev zuul in openstack-infra to help verify zuul-jobs under 2.5; then merge patches and release.  (b) declare a flag day and folks can make local forks if they don't want to participate.  (c) just stay on 2.3 until we have multi-version.17:47
corvusSpamapS, Shrews, pabelanger, clarkb, tobiash: ^ what sounds palatable?  :)17:47
pabelangerorder is okay for me17:50
Shrewsc seems fine as far as openstack is concerned. I know of no one pushing for a more recent version of ansible17:50
mordredcorvus: I like a and b the best17:50
corvusShrews: SpamapS and tobiash both want it I believe.17:50
pabelangerI'm not sure how much time I have in the next 2 weeks to work on zuul-dev.o.o, I'm still preparing for opendevconf17:51
mordredI agree with Shrews that c seems fine as far as openstack is concerned, but I worry that what we're shipping is still 2.317:51
mordredI'd rather flag-day up to 2.5 and then say that we'll wait for multi-version to upgrade past that17:52
corvusopenstack is just one downstream user of zuul :)17:52
pabelangerbut likey can allocate time after summit to work on it and / or happy to review code if somebody else wants to work on it17:52
mordredcorvus: totes. just saying - the one downstream user, openstack, should be fine on 2.3 - but there are other concerns and reasons why upgrading might/should still be desirable17:53
ShrewsI was just giving the openstack perspective. I know others want it17:53
pabelangerI want to say SF.io zuul is already using ansible 2.4, maybe we can see if there is any know issues on zuul-jobs too17:54
tobiashI'm also on 2.4 because of not working windows modules in 2.3 (with py3)17:54
corvusif someone can say "zuul-jobs generally works fine under 2.5" i think we can jump to the flag day.17:54
corvusit doesn't have to be openstack-infra that validates that17:54
tobiashbut I'm only using little of zuul-jobs currently17:55
corvusand it probably doesn't have to be perfect -- but knowing that, say, 'tox' works would be good to know.17:55
tobiashI think we use 'tox'17:56
corvusand maybe any fixes that we do have to make to zuul-jobs can have version checks, so we can continue to support 2.3 and allow folks to upgrade when ready?17:56
mordredsounds reasonable17:57
pabelanger+117:57
corvushow about i send a msg to the ml asking (a) have you tested zuul-jobs with 2.4 or 2.5?  (b) if folks would be okay with merging the 2.5 upgrade and releasing 3.1 on a specified date, soonish, with the idea that if we find that zuul-jobs needs any emergency fixes, we'll try our best to make them backwards compat.17:58
corvus?17:58
mordred++17:59
clarkbseems reasonable17:59
pabelanger3.1 ++17:59
tobiashsounds good17:59
pabelangercould I get a final +3 https://review.openstack.org/565758/18:06
pabelangerthanks18:06
*** acozine1 has quit IRC18:27
*** openstackgerrit has joined #zuul18:59
openstackgerritMerged openstack-infra/zuul-jobs master: Update repo file for fedora-28  https://review.openstack.org/56575818:59
*** spsurya has quit IRC19:09
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Add tox-py36 jobs  https://review.openstack.org/56588119:27
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Add config read and node deletion notifications  https://review.openstack.org/56588219:29
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: WIP: Pre-register static nodes  https://review.openstack.org/56588319:29
Shrewsdon't bother reviewing those ^^19:29
Shrewsunless you just _want_ head pain19:30
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Add tox-py36 job  https://review.openstack.org/56588419:33
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Add tox-py36 jobs  https://review.openstack.org/56588119:34
*** acozine1 has joined #zuul19:35
-openstackstatus- NOTICE: The Gerrit service at review.openstack.org will be offline starting at 20:00 (in roughly 25 minutes) for a server move and operating system upgrade: http://lists.openstack.org/pipermail/openstack-dev/2018-May/130118.html19:37
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Add tox-py36 jobs  https://review.openstack.org/56588119:48
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Add tox-py36 jobs  https://review.openstack.org/56588119:49
-openstackstatus- NOTICE: The Gerrit service at review.openstack.org will be offline over the next 1-2 hours for a server move and operating system upgrade: http://lists.openstack.org/pipermail/openstack-dev/2018-May/130118.html20:07
*** ChanServ changes topic to "The Gerrit service at review.openstack.org will be offline over the next 1-2 hours for a server move and operating system upgrade: http://lists.openstack.org/pipermail/openstack-dev/2018-May/130118.html"20:08
*** dkranz has quit IRC20:23
*** spsurya has joined #zuul20:27
*** openstackgerrit has quit IRC20:27
SpamapScorvus: I'd like to accelerate multi-version if possible. But, I should probably finish my re2 stuff first. ;)21:00
SpamapScorvus: so given that I'm not going to be able to *do* multi-version spec or implementation for many months, I like the flag day approach.21:00
SpamapSAlso worth noting that we're not debating raising the *minimum* ansible version21:00
SpamapSso if we make a zuul-jobs gate that runs everything in 2.3 and 2.5, that would be pretty awesome.21:01
corvusSpamapS: well, at the moment, Zuul works with exactly one ansible major version -- we don't have the facility for supporting the module overrides for more than one21:02
corvusSpamapS: but hopefully we can keep zuul-jobs supporting older versions of zuul, therefore also older versions of ansible21:03
*** acozine1 has quit IRC21:03
*** hasharAway has quit IRC21:09
*** ssbarnea_ has quit IRC21:53
clarkbProbably the biggest question for me is should zuul have an upfront whitelist of versions that it creates envs for on start or should it just install any version specified by user as long as it can be found on pypi?22:11
clarkband then just install them on the fly and fail jobs if they specify an invalid ansible version22:11
SpamapScorvus: right my thinking was that this gate would actually run ansible 2.3 and 2.5 as a simulation of what zuul really does.22:11
clarkb(and can set a default version in the base job)22:12
SpamapSJust basic stuff22:12
SpamapSmaybe even just syntax checks.22:12
SpamapSclarkb: yeah that's the longer term solution22:12
SpamapSI think if we can get somebody to work on that full time it won't take long to implement.22:12
SpamapSBut in the mean time, just trying to think of how we can aggressively move up to 2.5 without blowing up users of 2.3.22:13
*** ChanServ changes topic to "Discussion of the project gating system Zuul | Website: https://zuul-ci.org/ | Docs: https://zuul-ci.org/docs/ | Source: https://git.zuul-ci.org/ | Channel logs: http://eavesdrop.openstack.org/irclogs/%23zuul/ | Weekly updates: https://etherpad.openstack.org/p/zuul-update-email"22:13
-openstackstatus- NOTICE: Gerrit maintenance has concluded successfully22:13
clarkbya, I expect the most "fun" will be had dealing with the inevitable python env behavior we didn't expect upfront22:15
SpamapSAnother thought I've had is maybe something can be done to just have ansible versioned executors.22:17
clarkbfor example can venv create python2 venvs if we need python2 for ansible?22:17
clarkb(I don't know but something we'll have to sort out if we use that approach)22:17
SpamapSzuul:execute becomes "whatever you get" but then you can ask for a specific version and have a zuul:execute:ansible=2.3 and executors register the version they support.22:17
clarkbSpamapS: one downside to that approach is job distribution becomes potentially really funny22:18
corvusclarkb, SpamapS: i think we need to whitelist versions because each major version of ansible requires us to audit the modules we override22:18
clarkbif job writers don't all update you'll have a really busy 2.3 executor and bored 2.5 executor22:18
clarkbcorvus: good point22:18
SpamapSclarkb: that's assuming they all express 2.3.22:19
SpamapSIf the default updates from 2.3 to 2.5, most jobs just go forward with the default. The ones that fail, the authors need to pop down to lower version.22:19
SpamapSOr fix their job22:19
corvusi like the idea of executors supporting all the versions; i think if that's a feature of zuul, zuul should support it, not depend on the admin to implement it22:19
SpamapSI'd like for all the versions to be supported too.22:20
SpamapSAnd I do think venvs will be a relatively simple thing to have prepared at executor startup based on the whitelist.22:20
corvus++22:20
SpamapSSo if the version of Zuul says "I can do Ansible 2.3, 2.4, and 2.5, then executors just make virtualenvs in a workspace using bubblewrap and then add that workspace into jobs, with unexpressed versions being whatever one is default.22:21
SpamapSand I do think that won't be a mountain of work22:21
SpamapSJust... I'm having trouble with my molehill of work thus far. ;)22:22
clarkbI know the feeling22:22
clarkbin fact that reminds me I promised fdegir I'd write some things I don't think I ever did22:22
clarkbugh22:22
clarkbBUT tomorrow is a new day without doctor visits and gerrit upgrades :)22:22
*** rlandy has quit IRC22:24
* SpamapS is currently writing zuul jobs to test legacy puppet deploys22:25
SpamapSSO MUCH FUN22:25
*** elyezer has quit IRC22:35
*** elyezer has joined #zuul22:37
*** openstackgerrit has joined #zuul22:54
openstackgerritJames E. Blair proposed openstack-infra/zuul-website master: Replace background image  https://review.openstack.org/56590822:54
openstackgerritJames E. Blair proposed openstack-infra/zuul-website master: WIP: Add logo and move menu to top right  https://review.openstack.org/56590922:54
clarkbcorvus: ^ top right is weird with fullscreen browser and mega monitor23:00
clarkbcorvus: maybe align it right of the logo instead?23:00
clarkband then maybe the footer sections (get zuul, get involved) get added to the menu and docs gets added to footer too?23:01
corvusclarkb: i'll expand the menus later; i just wanted to re-establish the structure first23:03
corvusclarkb: can you screenshot what looks weird?23:03
clarkbcorvus: yup give me a few to finish this gerrit stuff first23:03
*** spsurya has quit IRC23:04
corvusclarkb: also, it doesn't quite look like the mockup yet -- it's too far to the right, and the underlines are too far away... maybe that's your objection?23:04
clarkbcorvus: ya its basically hidden in top right corner which is really far away on this montiro23:05
corvusif so, i agree -- i'll tweak it a bit more later23:05
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Update documentation URLs  https://review.openstack.org/56591023:06
*** andreaf has quit IRC23:09
*** mnaser_ has joined #zuul23:09
*** mhu` has joined #zuul23:15
*** jlvillal_ has joined #zuul23:15
*** robled- has joined #zuul23:16
*** mnaser has quit IRC23:16
*** mhu has quit IRC23:16
*** jlvillal has quit IRC23:16
*** robled has quit IRC23:16
*** clarkb has quit IRC23:16
*** robled- is now known as robled23:16
*** robled has quit IRC23:16
*** robled has joined #zuul23:16
*** mnaser_ is now known as mnaser23:16
*** jlvillal_ is now known as jlvillal23:19
*** jlvillal is now known as Guest1001423:19
*** clarkb has joined #zuul23:24
*** andreaf has joined #zuul23:25
corvusclarkb: we can give it padding on all the sizes, but also, because of the way it's set up to be responsive to screen sizes, we can also easily give it more padding on extra wide screens23:33
clarkbcorvus: ++23:33

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!