Monday, 2018-02-12

*** patriciadomin has quit IRC00:06
*** patriciadomin has joined #zuul00:09
*** JasonCL has quit IRC00:17
*** JasonCL has joined #zuul00:18
*** JasonCL has quit IRC00:33
*** JasonCL has joined #zuul00:34
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326800:34
*** JasonCL has quit IRC00:44
*** patriciadomin has quit IRC00:51
*** patriciadomin has joined #zuul00:52
*** JasonCL has joined #zuul00:57
*** JasonCL has quit IRC01:02
*** patriciadomin has quit IRC01:03
*** patriciadomin has joined #zuul01:04
*** patriciadomin has quit IRC01:09
*** patriciadomin has joined #zuul01:09
*** patriciadomin has quit IRC01:17
*** patriciadomin has joined #zuul01:18
*** patriciadomin has quit IRC01:56
*** patriciadomin has joined #zuul01:56
*** JasonCL has joined #zuul02:00
*** JasonCL has quit IRC02:05
*** JasonCL has joined #zuul02:45
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326802:49
*** JasonCL has quit IRC02:52
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326802:58
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Fix legacy dsvm jobs  https://review.openstack.org/54332702:58
*** patriciadomin has quit IRC03:11
*** patriciadomin has joined #zuul03:11
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326803:14
*** elyezer has quit IRC03:15
*** elyezer has joined #zuul03:16
ianwcorvus / Shrews : apropos the switch to native jobs for nodepool & trying to debug resulting log output ... https://review.openstack.org/543266 which means we can click on .log.gz files; reviews appreciated03:19
ianwthe builder .log files get caught by this03:19
*** patriciadomin has quit IRC03:24
ianw#status log received rax notification that host of nl02.o.o and zm06.o.o has some sort of issue; currently can't log into either.  updates from rax pending03:25
ianw(sorry wrong channel)03:25
*** patriciadomin has joined #zuul03:25
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Fix legacy dsvm jobs  https://review.openstack.org/54332703:47
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326803:47
*** patriciadomin has quit IRC03:48
*** patriciadomin has joined #zuul03:50
*** patriciadomin has quit IRC04:05
*** patriciadomin has joined #zuul04:05
*** JasonCL has joined #zuul04:26
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Fix legacy dsvm jobs  https://review.openstack.org/54332704:27
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326804:27
*** Wei_Liu has quit IRC04:30
*** JasonCL has quit IRC04:31
*** xinliang has joined #zuul04:36
*** patriciadomin has quit IRC04:42
*** patriciadomin has joined #zuul04:43
*** elyezer has quit IRC05:15
*** patriciadomin has quit IRC05:19
*** elyezer has joined #zuul05:20
*** patriciadomin has joined #zuul05:20
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326805:39
*** patriciadomin has quit IRC05:46
*** patriciadomin has joined #zuul05:47
*** patriciadomin has quit IRC05:59
*** patriciadomin has joined #zuul06:00
*** Wei_Liu has joined #zuul06:11
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326806:41
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Emit a warning when using unsafe public key url  https://review.openstack.org/54320606:47
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326806:48
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Revert fixes for legacy boot jobs  https://review.openstack.org/54335006:48
*** elyezer has quit IRC06:51
*** elyezer has joined #zuul06:52
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Remove unused method checkoutLocalBranch  https://review.openstack.org/54335907:09
*** swest has quit IRC07:10
*** swest has joined #zuul07:10
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326807:33
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Revert fixes for legacy boot jobs  https://review.openstack.org/54335007:33
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Don't decode str in LogStreamingHandler  https://review.openstack.org/54336507:42
*** hashar has joined #zuul07:44
*** elyezer has quit IRC07:47
*** elyezer has joined #zuul07:48
*** elyezer has quit IRC08:01
*** elyezer has joined #zuul08:03
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Fix broken fail_json in zuul_console  https://review.openstack.org/54337008:04
*** sc68cal has quit IRC08:34
*** jpena|off is now known as jpena08:45
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326808:53
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Revert fixes for legacy boot jobs  https://review.openstack.org/54335008:53
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326810:05
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Revert fixes for legacy boot jobs  https://review.openstack.org/54335010:05
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: zuul autohold: allow operator to specify nodes TTL  https://review.openstack.org/54340310:08
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Clean held nodes automatically after configurable timeout  https://review.openstack.org/53629510:17
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Add /label-list to the webapp  https://review.openstack.org/53556310:19
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Refactor status functions, add web endpoints, allow params  https://review.openstack.org/53630110:19
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Add separate modules for management commands  https://review.openstack.org/53630310:20
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: webapp: add optional admin endpoint  https://review.openstack.org/53631910:21
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: zuul autohold: allow operator to specify nodes TTL  https://review.openstack.org/54340310:23
*** elyezer has quit IRC10:25
*** elyezer has joined #zuul10:30
*** electrofelix has joined #zuul10:36
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: zuul autohold: allow operator to specify nodes TTL  https://review.openstack.org/54340310:40
*** mrhillsman has quit IRC10:57
*** mrhillsman has joined #zuul10:58
*** JasonCL has joined #zuul11:03
*** bhavik1 has joined #zuul11:13
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Clean held nodes automatically after configurable timeout  https://review.openstack.org/53629511:23
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: webapp: add optional admin endpoint  https://review.openstack.org/53631911:30
*** bhavik1 has quit IRC12:01
*** sshnaidm is now known as sshnaidm|ruck12:09
*** jpena is now known as jpena|lunch12:26
*** weshay|ruck is now known as weshay13:10
*** jpena|lunch is now known as jpena13:26
*** rlandy has joined #zuul13:32
*** myoung|off is now known as myoung13:53
*** dkranz has joined #zuul14:14
*** patriciadomin_ has joined #zuul14:16
*** patriciadomin has quit IRC14:19
pabelangerwith zuulv3, jobs that share the same name no longer imply a share queue for dependent pipeline, is that correct?15:25
AJaegerpabelanger: correct15:25
AJaegerpabelanger: that's why we have "queue: X" in the gate pipeline15:25
pabelangerYah, knew we had to add queue: foobar now to be more specific, was there a specific reason for no longer making that implied?15:27
corvuspabelanger: because jobs no longer have their project names in them.  almost every project in openstack runs a job called 'tox-py27', for example.15:30
corvuswe might be able to bring some of the automation back, however, by analyzing required-projects entries.  i haven't thought that through completely yet though.15:31
pabelangerkk, that make sense. Was mostly trying to see why it changed.15:32
*** gregdek has quit IRC15:43
*** gregdek has joined #zuul15:43
corvusSpamapS: any update on the pyre2 release?15:48
*** elyezer has quit IRC15:58
*** elyezer has joined #zuul16:02
*** mattclay has quit IRC16:03
*** mattclay has joined #zuul16:03
SpamapScorvus: the author said they'd try to cut a release, but gave no time frame. I took that to be "in the next few days"16:35
SpamapScorvus: and he did tag..16:37
SpamapShttps://github.com/facebook/pyre2/tree/v1.0.616:37
SpamapSso.. I assume he forgot his pypi password or something ;)16:37
SpamapSactually https://pypi.python.org/pypi/fb-re2 shows sid0 as the owner in pypi, so maybe sid0 was out last week or something.16:37
corvusit's so weird for tag and release to not be the same thing :)16:42
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: webapp: add optional admin endpoint  https://review.openstack.org/53631916:42
pabelangerhmm16:48
pabelangerthis doesn't look right16:48
pabelanger2018-02-12 16:40:10.718800 | ze01 | :stderr: aiohttp requires Python '>=3.5.3' but the running Python is 3.5.216:48
SpamapScorvus: spoken like a true robot wrangelr.16:48
pabelangergetting that from aiohttp now16:48
pabelangerseems to be a 3.0.0 release today16:49
corvusgreeeaaat16:49
SpamapSall hail the .0 release gods16:49
SpamapSfor they shall smite all thy automation and burn thy untested code to the ground.16:49
pabelangerMinimal supported Python version is 3.5.3.16:49
pabelangerouch16:49
corvusthere's also a 3.0.116:50
SpamapSWe might want to ping doko. He should be updating Xenial to latest patch release sometime soon.16:50
SpamapSOh wow16:50
SpamapS3.5.3 is like, < 30 days old16:50
corvuspabelanger: want to throw in a < pin?16:51
SpamapShttps://www.python.org/downloads/release/python-353/16:51
pabelangercorvus: sure16:51
SpamapSI love these commit messages. https://github.com/aio-libs/aiohttp/commit/8e9c6b5079f6acbbdfc352250e21ae5bf56c25e816:53
SpamapS"I did some stuff. Deal with it."16:53
SpamapS[paraphrased]16:53
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Cap aiohttp <3.0.0  https://review.openstack.org/54354616:53
corvusthe code says what you did.  the commit message doesn't need to repeat it.  neither one says why.  :(16:53
SpamapSno like, "because this bug was fixed" or "because we hate 3.5.2's face"16:54
corvuspabelanger: where does that error show up?16:55
SpamapSnot even a reference to a bug16:55
SpamapSnothing16:55
SpamapSwow16:55
SpamapSaiohttp devs.. you disappoint me. :(16:55
pabelangercorvus: I noticed it at: http://logs.openstack.org/84/543484/1/gate/windmill-ubuntu-xenial/0c8b364/job-output.txt.gz#_2018-02-12_16_40_10_71850016:55
corvusyeah... i don't think aiohttp has been an unqualified success.16:56
corvuspabelanger: ok.  zuul01.o.o hasn't upgraded it... yet...16:57
pabelangerkk16:57
SpamapSShame devs on Twitter [X]   https://twitter.com/spamaps/status/96309454973521100816:57
corvusSpamapS: you want to +3 https://review.openstack.org/543546 ?17:00
*** maxamillion has quit IRC17:04
*** maxamillion has joined #zuul17:04
*** maxamillion has quit IRC17:13
*** maxamillion has joined #zuul17:13
SpamapSlooks like tobiash beat me to it :)17:16
electrofelixDoes zuul have support for HA around the scheduling part? I recall that it can be gracefully restarted but can't seem to put my hands on details around upgrading/updating17:43
*** myoung is now known as myoung|food17:44
corvuselectrofelix: not yet.  that's what we call zuul v417:44
electrofelixcorvus: so all updates are currently lossy (requires retriggering)17:46
SpamapSZuul v4 - The Wrath of Zookeeper17:46
SpamapSelectrofelix: you can save the queue and re-load it. So scheduled restarts are not lossy.17:47
SpamapSBut if you lose the process unexpectedly, all must be retriggered.17:47
electrofelixSpamapS: good enough, I'm only concerned about the scheduled restart/update17:48
SpamapScorvus: part of me wonders if we can get a 3.x that more or less abuses zookeeper to store state in it without having the actual multi-scheduler HA model.17:48
corvusSpamapS: that's 99% of the work17:49
SpamapSlike just something coarse grained and dumb, like wrapping trigger handling in queue.save() calls17:49
SpamapSand making the queue save happen in zk, instead of on disk.17:49
SpamapScorvus: no, think dumber. :)17:50
corvusSpamapS: i am :)  you throw a lock around that and you've got HA17:50
SpamapSlike, the same thing we have now, but to zk, and more often.17:50
corvusSpamapS: we don't save the pipeline contents anywhere17:50
SpamapSooh17:50
SpamapSjust the queue then.17:51
SpamapSI see now, we are already as dumb as we wanna be. :)17:51
SpamapSbtw... latest updates to Zuul are nice. Little things getting better and better.17:51
corvusso the main task is creating the zk<->model mapping.  the rest falls out from there.17:51
SpamapSSorry I won't be in Dublin. Too insane at work to leave.17:52
corvusi was polishing last week17:52
corvusSpamapS: drat.  maybe we can irc/phone you in on some things if you have time17:52
SpamapSYeah I'm sure I can get in on some of those discussions.17:53
SpamapSJust can't be gone. one of our senior engs just left for Paternity leave.. so we're already skeleton crew. :-P17:53
SpamapScorvus: I'm especially interested in working out how we turn Zuul into a first class deployer.17:54
SpamapSWe have it CI'ing our stuff now, but we still hand-off to a deployment system for reasons.17:54
corvusSpamapS: great!  that's the dream :)17:55
SpamapSMainly that people want to pass --limit and --tags to the Ansible we run, and while we already let zuul run kolla-ansible on a runner VM... the whole thing of "now run it with these args. Now these args. Now these." .. not super smooth in zuul.17:55
corvusSpamapS: i responded on https://review.openstack.org/542518  -  i think most of the things are straightforward, but i think there was some secure.yaml/clouds.yaml confusion -- can you make sure i've sussed that out correctly and everything is straight?17:56
SpamapSHoping we can work out manual triggering / parameterization.17:56
corvusSpamapS: i don't see any reason we can't support tags, that seems straightforward.  the limit things sounds like a high-bandwidth conversation we should have to make sure we all get on the same page.17:57
SpamapSThere's this thing bubbling in my brain about having ansible-orchestration be a thing.17:58
SpamapSWhich sounds weird17:58
SpamapSbut it's like, sometimes I want to express a pattern for running a playbook with certain arguments.17:58
SpamapSThe specific example is doing AZ-aware canary-sensitive rolling updates on a schedule.17:58
SpamapSSo like, "You can update any of these, 10% at a time, start with 1, then do 2, then 5. Stop if any monitors goes off."17:59
SpamapSserial: sort of wants to be that, but it's not quite good enough18:00
*** bhavik1 has joined #zuul18:00
openstackgerritClark Boylan proposed openstack-infra/zuul master: Make timeout value apply to entire job  https://review.openstack.org/54148518:02
clarkbSpamapS: ^ the timeout is being passed to runAnsiblePlaybook so I called it ansible_timeout. Hopefully that is more clear18:02
*** bhavik1 has quit IRC18:04
*** harlowja has joined #zuul18:14
*** jpena is now known as jpena|off18:18
*** elyezer has quit IRC18:32
*** elyezer has joined #zuul18:33
*** electrofelix has quit IRC18:34
*** electrofelix has joined #zuul18:37
*** rlandy is now known as rlandy|brb18:46
corvusSpamapS: sudo adduser --system nodepool --home-dir /var/lib/nodepool --create-home18:47
corvusSpamapS: how's that look?18:47
*** elyezer has quit IRC18:49
*** elyezer has joined #zuul18:50
pabelangerIIRC using --system means you'd no longer be able to do sudo su nodepool18:50
*** electrofelix has quit IRC18:51
*** dkranz has quit IRC18:52
corvuspabelanger: seems to work18:54
pabelangercool!18:54
pabelangerwfm18:55
*** myoung|food is now known as myoung18:57
corvusSpamapS: your suggestion to use the openstackclient to create a keypair means we need to install 81 more dependencies19:01
corvusSpamapS: i'm not sure that's desirable...19:02
corvusmaybe that's why the todo note suggested shade19:02
corvusthen again, maybe we should go ahead and recommend the installation, that way it's there for nodepool debugging, etc... (eg openstack server list)19:03
clarkbdoesn't nova do keypair generation on the server side?19:04
corvusclarkb: i don't understand19:04
clarkboh you are talking to use teh create command nevermind I misunderstood (I thought you wanted a replacement for paramiko)19:05
*** rlandy|brb is now known as rlandy19:08
*** sshnaidm|ruck is now known as sshnaidm|off19:11
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Add zuul from scratch doc  https://review.openstack.org/54251819:34
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Reclassify some debug log lines as info  https://review.openstack.org/54304419:34
corvustobiash, SpamapS: ^ okay, I think that takes care of all your suggestions19:35
clarkbcorvus: fwiw some clouds publish an openrc and a clouds.yaml via horizon now. But I am not sure how common that is at this point (probably still more generic to make clouds.yaml from openrc)19:37
clarkbthere is also no mention of the firewall which surprises me. I thought fedora (and other red hat distros) had very restrictive firewall rules by default? maybe not on the cloud images?19:42
*** elyezer has quit IRC19:49
ianwcould i get a couple of reviews on https://review.openstack.org/#/c/543327 to fix the dib gate ... this is the minimal fix for the legacy nodepool boot jobs to keep working19:58
*** elyezer has joined #zuul19:59
corvusclarkb: the cloud i'm working with (rdo) doesn't have clouds.yaml.  and yeah, apparently firewall is not an issue using the fedora 27 cloud image.20:02
corvusclarkb: i'm certain it's stock, because i had to download and upload the image myself.20:02
ianwcorvus: re the copy of logs via zuul_copy_output @ https://git.openstack.org/cgit/openstack-infra/nodepool/tree/.zuul.yaml#n30 ... does that not copy in failure case?  see http://logs.openstack.org/28/543328/2/check/legacy-dsvm-nodepool-opensuse-src/6e9d662/logs/20:06
ianwoh, don't mind me, that's the legacy job20:15
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add native distro test jobs  https://review.openstack.org/54326820:22
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Revert fixes for legacy boot jobs  https://review.openstack.org/54335020:22
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: zuul web: add admin endpoint, enqueue commands  https://review.openstack.org/53900420:27
* rbergeron appears and babbles about yay fedora cloud images20:44
openstackgerritMerged openstack-infra/nodepool master: Fix legacy dsvm jobs  https://review.openstack.org/54332720:45
openstackgerritMerged openstack-infra/zuul master: Cap aiohttp <3.0.0  https://review.openstack.org/54354621:01
mnaserclarkb: late but the clouds.yaml change shipped thanks to mordred in pike21:01
*** ChanServ has quit IRC21:02
*** ChanServ has joined #zuul21:11
*** barjavel.freenode.net sets mode: +o ChanServ21:11
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: add abstract job attribute  https://review.openstack.org/54361921:15
dmsimardcorvus: I'll send a patch for it but do you have any suggestions on how to improve the headers/anchors in the docs such as these ? https://docs.openstack.org/infra/zuul/admin/drivers/gerrit.html21:46
dmsimardAn anchor can end up being displayed/linked as "https://docs.openstack.org/infra/zuul/admin/drivers/gerrit.html#attr-pipeline.trigger.%3Cgerrit%20source%3E.approval" which is not very parseable when not encoded21:47
dmsimardpabelanger linked me "https://docs.openstack.org/infra/zuul/admin/drivers/gerrit.html?highlight=verified#attr-pipeline.trigger.<gerrit source>.require-approval" earlier :)21:47
dmsimardI guess it depends on how each browser ends up displaying/copying the link21:48
corvusdmsimard: i don't have an immediate suggestion for that -- tbh, i was *thrilled* just to get that working at all :)21:50
corvusdmsimard: i agree about the desirability of it though :)21:51
jheskethMorning21:52
dmsimardFWIW Firefox properly encodes that for me21:52
pabelangerjhesketh: morning!21:52
dmsimardI mean, when I click on the header -- clicking on pabelanger's link is hopeless and my IRC client doesn't understand it :p21:52
corvusdmsimard: maybe we can translate <> to something else in add_target_and_index ?21:53
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Clean held nodes automatically after configurable timeout  https://review.openstack.org/53629521:53
corvusdmsimard: that's where i'd start poking21:53
dmsimardcorvus: worth a look21:53
corvusdmsimard: if we change it, we should do a matching change in zuul-sphinx.  and/or see if we can drop that from zuul and switch zuul itself over to depend on zuul-sphinx for doc builds.21:54
* jhesketh has to run to the airport so will only be intermittently at the zuul meeting21:54
corvusjhesketh: heading off to the ptg?  ;)21:55
jheskethcorvus: I will be :-), but not this early. Just dropping my wife off so not travelling anywhere myself this week21:55
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul master: Delete IncludeRole object from result object for include_role tasks  https://review.openstack.org/54363321:56
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul master: Delete IncludeRole object from result object for include_role tasks  https://review.openstack.org/54363321:58
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: zuul autohold: allow operator to specify nodes TTL  https://review.openstack.org/54340321:58
dmsimardZuul meeting imminent in #openstack-meeting-alt21:59
pabelangerwill only be around for first 1/2 today22:00
pabelangerthen off to local linux meetup22:00
pabelangeryear of the LUG22:00
pabelanger:)22:00
*** threestrands has joined #zuul22:12
openstackgerritMerged openstack-infra/zuul master: Emit a warning when using unsafe public key url  https://review.openstack.org/54320622:18
openstackgerritMerged openstack-infra/zuul master: Remove unused method checkoutLocalBranch  https://review.openstack.org/54335922:18
*** hashar has quit IRC22:28
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Add abstract job attribute  https://review.openstack.org/54361923:36
ianwhas there been any disucssion of the webapp endpoints of nodepool (dib-image-list) etc?23:54
ianwi'm just fixing up the puppet for the build logs and wondering if it's worth going to effort to expose them23:54
clarkbianw: I want to say it came up last last week?23:55
clarkbbut I have been a zombie the last 4 days or so so may misremember23:55
ianwok.  the complication comes if you want a builder on the same host as the launcher.  i don't want to over-engineer a solution that nobody ever uses23:58
dmsimardianw: there are already some patches in that direction I think23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!