Tuesday, 2017-05-16

*** harlowja has joined #zuul00:49
*** harlowja has quit IRC01:42
*** harlowja has joined #zuul01:56
*** harlowja has quit IRC02:20
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: launcher: store console log in workspace  https://review.openstack.org/46452104:27
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: launcher: add simple email publisher  https://review.openstack.org/46492304:32
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: launcher: add results_dir option  https://review.openstack.org/46452404:38
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: launcher: terminate console server after job ends  https://review.openstack.org/46452004:40
*** jamielennox is now known as jamielennox|away05:10
*** jamielennox|away is now known as jamielennox05:30
*** harlowja has joined #zuul05:47
*** jlk has quit IRC06:09
*** jlk has joined #zuul06:09
*** jlk has quit IRC06:09
*** jlk has joined #zuul06:09
*** isaacb has joined #zuul06:40
*** isaacb has quit IRC06:41
*** isaacb has joined #zuul06:41
*** harlowja has quit IRC06:50
*** DangerousDaren has joined #zuul06:54
*** isaacb_ has joined #zuul07:00
*** isaacb has quit IRC07:00
*** jamielennox is now known as jamielennox|away07:07
openstackgerritTobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add max-ready-age to label config  https://review.openstack.org/46333807:24
*** jamielennox|away is now known as jamielennox07:34
*** DangerousDaren has quit IRC08:19
*** DangerousDaren has joined #zuul08:19
*** isaacb_ has quit IRC08:59
*** isaacb has joined #zuul09:06
*** isaacb has quit IRC09:09
*** Cibo_ has joined #zuul09:25
*** Cibo_ has quit IRC09:57
*** jkilpatr has quit IRC10:41
*** jkilpatr has joined #zuul10:59
*** Cibo_ has joined #zuul11:06
*** Cibo_ has quit IRC11:10
*** yolanda has quit IRC11:27
*** jhesketh has quit IRC11:39
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Don't report start on unmanaged projects  https://review.openstack.org/45571111:41
*** jhesketh has joined #zuul11:41
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Whitelist pydevd debug threads  https://review.openstack.org/46504611:41
*** isaacb has joined #zuul11:43
*** yolanda has joined #zuul11:59
*** morgan_ is now known as morgan12:49
dmsimardpabelanger: did you get the chance to chat about nodepool boot from volume at summit ?13:15
*** dkranz_ has joined #zuul13:20
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix inventory vars containing spaces  https://review.openstack.org/46507213:55
tobiashhi14:08
tobiashIs something broken with the docs jobs? I'm not aware of changing something docs related in ^^14:08
pabelangerdmsimard: yes: https://review.openstack.org/46428314:09
pabelangerdmsimard: just need reviews / maybe gate job14:09
pabelangerdmsimard: and new shade release14:09
dmsimardpabelanger: nice, I'll have a look, thanks !14:10
pabelangertobiash: have a log?14:11
dmsimardpabelanger: lgtm at first glance14:11
pabelangertobiash: Hmm, looks like PBR. Wonder if there is a new release of something14:12
tobiashpabelanger: http://paste.openstack.org/show/609669/14:12
tobiashat least what I could find in the logs14:12
tobiashhttp://logs.openstack.org/72/465072/1/check/gate-zuul-docs-ubuntu-xenial/65c83d3/console.html14:12
tobiashpabelanger: will check tomorrow again, it's eod for me14:15
pabelangertobiash: thanks, I'll look into it now14:20
pabelangerya, sphinx 1.6.1 today14:22
mordredoh goodies15:37
*** isaacb has quit IRC15:57
openstackgerritLeif Madsen proposed openstack-infra/zuul master: Cleanup quickstart documentation  https://review.openstack.org/46510816:12
*** leifmadsen has joined #zuul16:12
*** dmsimard has quit IRC16:14
*** cmurphy has quit IRC16:14
*** dmsimard has joined #zuul16:17
*** isaacb has joined #zuul16:19
mordredleifmadsen: ^^ thanks - comments on the review16:19
leifmadsenrock on16:19
leifmadsenlooking16:19
leifmadsenahhh, didn't look at remote branches16:22
leifmadsenso I guess master is v2.5?16:22
SpamapSyep16:22
leifmadsennot sure how the branching is currently setup16:22
leifmadsenok thanks16:22
SpamapSfeature/zuulv3 is where most dev is happening16:22
jlko/16:22
leifmadsenSpamapS: cool thanks16:22
leifmadsenplaying / learning zuul today16:23
leifmadsenshould only take an hour or two to become and expert right?16:23
jlkprobably enough.16:24
mordredleifmadsen: you'll be in perfect shape in about 10 minutes ;)16:36
leifmadsenwhat will I do with the other 7 mins?16:36
leifmadsenmordred: so to your comments, I was wondering about the bindep.txt thing, but then realized the quickstart talks about using pip, which means there is no bindep.txt locally (that would imply you've cloned the repo)16:38
leifmadsenI could perhaps add a command to curl the bindep.txt file from the github repo I suppose16:38
ShrewsZuulers: FYI, some sort of summer ick has me totally out of commission today. Going back to bed. Hopefully more useful tomorrow.16:48
leifmadsenalso, I get the impression that bindep doesn't work on CentOS?16:56
leifmadsengetting issues with running it during the profiles list part16:56
pabelangerleifmadsen: centos works, we use it in the gate today. What is the error you are getting?17:05
leifmadsenhttps://paste.fedoraproject.org/paste/OkZylTQkX0oM4A2o6jVLfF5M1UNdIGYhyRLivL9gydE=17:05
jlkwonder if it's looking for something in redhat-lsb17:10
jlkI really hate that ERRno 2 never shows the file it was looking for17:10
jlklike, the MOST IMPORTANT PART OF THE ERROR...17:10
leifmadsenjlk: welp that's exactly it :)17:10
*** unrahul has joined #zuul17:11
leifmadsenrequires a `yum install redhat-lsb-core` before `bindep` works17:11
jlkCool, you've just found a doc contribution you can make :)17:11
*** isaacb has quit IRC17:11
leifmadsenthe worst^Wbest part of working on a new project is finding all the edge cases17:13
clarkbpretty sure that is dpcumebted somewhere, maybe just needs to be more prominent17:13
leifmadsenwell I'm fixing up the quickstart guide since it's missing stuff from quickstarting :)17:14
jlkMost of the v3 docs are going to be old/wrong17:14
leifmadsenI'm on 2.517:14
jlkall the v3 stuff was being done without updating the docs.17:14
jlkah okay17:14
leifmadsenwell, that's where I started17:14
leifmadsenjlk: just like all great open source projects17:14
jlkIf you're at all interested, we (BonnyCI) have a pile of Ansible for standing up a 2.5 (and a 3) environment. It uses ubuntu tho.17:15
leifmadsenmoving on :)17:17
jlkWasn't my choice, but I didn't want to disrupt inertia.17:18
leifmadsenall good, pabelanger has some playbooks as well17:18
leifmadsenI'm mostly just doing documentation review and writing17:18
leifmadsennot necessarily interested in spinning up infra yet17:18
leifmadsenhuh... sphinx-build --version returns 1.6.1 but still getting error: error in setup.cfg: command 'build_sphinx' has no such option 'warning_is_error'17:19
* mordred missed all sorts of good leifmadsen scrollback!17:22
leifmadsen"good"17:22
mordredleifmadsen: so - there's a bug that just hit today with openstack/pbr and sphinx 1.6.117:22
leifmadsenlol17:22
leifmadsenof course17:22
mordredwe're pinning back to 1.5 elsewhere in openstack while we figure it out17:22
mordredright?17:22
leifmadsenthat's pretty much how it works with me :)17:22
mordredyay for sphinx release today!17:22
leifmadsenhrm17:23
leifmadsenpinned to 1.5.1 locally, still same error17:23
leifmadsenI'm no longer convinced that the version in the venv is what is getting used17:24
jlkIs there an updated division of duties doc on what scheduler vs merger vs executor does in v3, and how they all talk to each other?17:28
* jlk is curious why on start up it seems both merger and executor are cloning down projects and scanning branches.17:31
*** unrahul has quit IRC17:38
*** Cibo_ has joined #zuul17:39
clarkbto load config I think17:40
jlkThey seem to be doing the exact same actions17:42
*** mattclay has quit IRC17:42
*** mattclay has joined #zuul17:45
jlkoh cool, caused a traceback by hitting the base /status url instead of a /<tenant>/status17:48
*** harlowja has joined #zuul17:50
SpamapSjlk: I believe there is a possible v3 future where merger doesn't do that, but I seem to recall we found a corner case where it was still needed.17:54
SpamapSor where merger isn't even a thing anymore.17:54
* SpamapS has been too deep in implementation to think theory lately17:55
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Add SSH Agent Primitives and usage  https://review.openstack.org/46271217:57
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Add support for bwrap  https://review.openstack.org/45385117:57
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Add bubblewrap to bindep / test-setup.sh  https://review.openstack.org/46184917:57
SpamapSpabelanger: ^ FYI, I rebased the bindep change17:57
SpamapSpabelanger: also, base job.. it's removed from zuul, so is openstack-zuul-jobs the only place I can find it?17:58
leifmadsensince I'm new to infra, where are the definitions for the jobs that test zuul itself?18:07
dmsimardWould it be a reasonable RFE for Zuul to provide the reason why a job is queued ? i.e, 'waiting for node: <nodepool-label>' or something like that. We just troubleshooted a broken job in #openstack-infra and had to crawl through "compiled" jjb to figure out the node allocation was wrong.18:15
dmsimardIn the patch json (i.e, http://zuul.openstack.org/status/change/465040,1 ) there could be an additional field18:16
*** jkilpatr_ has joined #zuul18:17
*** openstackgerrit has quit IRC18:17
*** jkilpatr has quit IRC18:20
jlkhrm, interesting. zuul-executor reads from the [merger] config block18:56
jlkSo in v3, executor pushes the merged repo content into the node, yes? there does not need to be anything connecting _to_ the executor? It'll get jobs from gearman, reach out to nodepool nodes, etc..?18:59
jlkoh what???19:02
jlkBoth the executor/server.py and merger/server.py register the "merger:merge" and "merger:cat" functions.19:02
jlkjeblair: what's the deal with ^^ ?19:02
*** DangerousDaren has quit IRC19:05
*** harlowja has quit IRC19:17
SpamapSjlk: correct, executor merges and pushes via ssh/ansible19:29
SpamapSjlk: I don't recall why we still need merger19:30
jlkit has a task that executor doesn't do, update19:30
SpamapSRight that's for zuul refs. If those are needed we might still need that. I think.19:32
jlkit's curious that the executor is registering the same functions as merger19:33
SpamapSAgreed19:33
*** jkilpatr_ has quit IRC19:40
*** Cibo_ has quit IRC19:42
jlkIs there a type of job I can have zuul do that won't involve nodepool? I tried a noop job, but that seems to be waiting on nodepool for some reason.19:45
*** pabelanger has quit IRC19:51
mordredleifmadsen: so - there are two answers to your question ... because we currently have a zuul 2.5 running tests on zuul AND a zuul v3 running tests on zuul19:54
mordredleifmadsen: the v3 config starts here: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/main.yaml19:55
mordredleifmadsen: which will get you to here: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.yaml19:55
*** pabelanger has joined #zuul19:55
leifmadsenmordred: cool thanks, I was just running into, what I figured were some pretty basic pip install issues, so wanted to see how that was being tested to see if 1) I was being dumb 2) I was testing a base scenario that wasn't actually built into a job yet19:55
mordredleifmadsen: and then this: http://git.openstack.org/cgit/openstack-infra/zuul/tree/.zuul.yaml?h=feature/zuulv319:56
leifmadsen+1 awesome will look through those19:56
mordredleifmadsen: ah - ok - so  in that case you might want to look at:19:56
mordredleifmadsen: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs and http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml - which are the v2 config- although that might be a bit hard to sift through19:57
mordredleifmadsen: http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/ and http://git.openstack.org/cgit/openstack-infra/openstack-zuul-roles/tree might also be interesting19:58
leifmadsenSpamapS: ping?20:02
*** dougbtv has joined #zuul20:03
jlkHe's at the Dr office I think.20:04
*** harlowja has joined #zuul20:04
SpamapSOh right it's time for the kubernetes testing sig meeting20:06
SpamapSleifmadsen: hey can you let them know I'm out sick at the moment? Thanks20:06
leifmadsenSpamapS: sure thing20:06
SpamapSDon't have that slack on my phone20:06
leifmadsennp20:06
leifmadsendone20:08
leifmadsenhopefully you're feeling better and we can hear about this next week!20:08
SpamapSIndeed20:15
jlkDoes zuulv3 require the zuulv3 branch of nodepool, or master?20:15
SpamapSzuulv3 I thought20:15
SpamapSUntil the shim is done20:16
SpamapSmordred: right?20:16
jlkdoesn't look like we differentiate in hoist20:16
SpamapSMaybe it merged.. I never followed it closely20:16
jlkoh wait20:17
jlkI'm wrong20:17
*** jkilpatr has joined #zuul20:53
*** eggshell has left #zuul21:03
*** openstackgerrit has joined #zuul21:16
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Block sphinx 1.6.1  https://review.openstack.org/46517621:16
mordredjlk, SpamapS, clarkb, pabelanger: ^^ that should generall workaround the sphinx issue leifmadsen ran in to earlier today21:16
*** jkilpatr has quit IRC21:19
jlkreviewed21:20
mordredjlk: also - I'm working through the GH stack - a bunch of it would be merged now - except for that sphinx silliness :)21:21
mordredjlk: did you ever make progress with our friends at gh on the whole review stickiness issue?21:22
jlkYES21:22
jlkthey implemented what we wanted mostly.21:22
mordredoh that's awesome news!!!21:22
* mordred hands jlk a gold cookie21:22
mordredwait. that's a terrible combinatoin. don't eat that21:23
jlkThere is now an option under branch protection where you can have positive reviews automatically dismissed when a PR code is updated.21:23
jlkbut first you have to turn branch protection on, _and_ require passing reviews to merge21:23
jlk"Dismiss stale pull request approvals when new commits are pushed21:25
jlkNew reviewable commits pushed to a branch will dismiss pull request review approvals."21:25
mordredI think that's a huge improvement21:28
pabelanger+221:30
mordredjlk: also - so far the gh patches have been very understandable -thank you for your work on them!21:32
jlkyaaaay. At this point I have no idea how far they've drifted from original author's versions.21:33
mordredwell- there's definitely jlk work in there I know - because some of it is based on discussions we've had in here21:34
openstackgerritMonty Taylor proposed openstack-infra/nodepool feature/zuulv3: Block sphinx 1.6  https://review.openstack.org/46517921:36
mordredpabelanger: next verse, same as the first ^^21:37
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Block sphinx 1.6.1  https://review.openstack.org/46517621:38
pabelangermordred: +2 also21:47
SpamapSsomething very wrong with my ssh agent stuff21:47
SpamapStest suite locks up :-P21:47
mordredSpamapS: that made me sing "ssh aaaaagent man" in my head21:49
jlkand now we're all singing that21:50
mordredyou're welcome21:51
* SpamapS salutes the song with a fit of hacking cough21:52
jlkAttributeError: 'list' object has no attribute 'hold_following_changes'21:57
jlkthat's... weird.21:57
jlkout of manager/__init__.py21:57
SpamapSjlk: likely a yaml indentation fail21:58
SpamapSSometimes I think yaml might have been a mistake.21:59
SpamapSThe structure seems too loose to validate fully.21:59
jlkI don't see anything odd with http://paste.openstack.org/show/609724/22:00
mordredjlk: in https://review.openstack.org/#/c/449365 I have left a -1 - but I also could be reading it wrong, so thought I'd ping you here22:00
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Support GitHub PR webhooks  https://review.openstack.org/43983422:00
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add basic Github Zuul Reporter.  https://review.openstack.org/44332322:00
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add 'push' and 'tag' github webhook events.  https://review.openstack.org/44394722:00
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add 'comment' github pull_request event action  https://review.openstack.org/44395922:00
SpamapSniiiiiiiiiiiiiiice22:01
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Configurable SSH access to GitHub  https://review.openstack.org/44403422:01
mordredSpamapS: it's happening22:01
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Support for github commit status  https://review.openstack.org/44406022:01
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Merge pull requests from github reporter  https://review.openstack.org/44446322:01
mordredjlk: also - no rush, because that's pretty much at the beginning of the part of the stack without a jeblair review22:02
openstackgerritMerged openstack-infra/zuul feature/zuulv3: support github pull request labels  https://review.openstack.org/44451122:02
SpamapSI can't believe I'm saying this.. but I wish ssh-agent implemented the systemd startup readiness thing.22:02
SpamapSletting it daemonize is making me twitch22:02
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Encapsulate determining the event purpose  https://review.openstack.org/44524222:02
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Support for dependent pipelines with github  https://review.openstack.org/44529222:02
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Allow github trigger to match on branches/refs  https://review.openstack.org/44562522:03
*** jkilpatr has joined #zuul22:03
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Block sphinx 1.6  https://review.openstack.org/46517922:08
jlkso... what are job variants?22:11
mordredjlk: I'm totally going to explain this wrong ...22:14
mordredjlk: but job variants are the thing that let you say stuff like "if the branch is stable/kilo, please run this on a trusty node instead of a xenial node"22:15
jlkhrm.22:16
mordredjlk: there's a section: http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#projects22:16
mordredthat talks about it a little22:17
mordredalthough it occurs to me that the zuul docs probably want a terminology/concept primer22:17
jlkyeah, okay. Wondering if because noop is a job built in, that something is missing and it's choking on the variant stuff22:19
mordredoh- interesting question22:21
jlkbut I would think it would choke in CI as a lot of the tests use noop22:22
pabelangeroh neat, github patches merge22:25
pabelangertime to setup a 3rd party CI test with github22:25
jlksome!22:26
jlknot all22:27
mordredpabelanger: yah - we have 2 really good candidates for that22:27
mordredpabelanger: we should lkely set up an "openstack-zuul" user - since the current user we use for things on gh is "openstack-gerrit" - which would be a weird user to test results to other communities with22:30
mordredactually - I say that ...22:30
mordredjlk: is that the "right" thing to do - or is there a different thing one wants to do for CI integrations?22:30
pabelangermordred: ya, openstack-zuul seems sane to me22:31
mordredpabelanger: it seems that the fine folks at bonny have set up an "integration": https://github.com/integration/bonnyci22:33
pabelangerOh, what is that?22:36
pabelangersomething needed for github access?22:36
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Add waitForImage to test_lost_request  https://review.openstack.org/46407522:37
mordredpabelanger: it's the way you register a service with github - and then projects that want to start interacting with your service just add it to their project22:38
jlkreading22:39
jlkwhat is it you want to do exactly?22:39
jlkThere's two ways Zuul can auth to interact with github API. One is with a user's API key, zuul then impersonates said user.22:40
jlkthe other way is as an integration bot, in which the integration service (zuul) will get short-lived access tokens that are scoped to specific projects (repositories) and are allowed to perform specific actions.22:40
pabelangerya, API is what I was thinking. Just want to read a PR, run local zuul, and see what happens22:41
pabelangerI doubt I'd even comment at this point22:41
jlkYou might be able to get away without even an API key22:42
jlkbut I haven't tested that22:42
pabelangerYa, I would assume that too22:42
jlkanonymously you can query the API and clone the repos22:42
mordredpabelanger: nod - for that absolutely, api seems the right way22:42
pabelangerbut ya, testing22:42
jlkSo what you need, is to stand up a zuul-scheduler that has a public exposed webapp (8001)22:43
mordredpabelanger: I think as we start poking at having Infra run tests triggered by PRs in associated projects - we'll want an actual integration bot22:43
mordredbut starting with API sounds great22:43
pabelangermordred: Ya, openstack-infra make sense22:43
jlkand configure a github repo's webhooks to send all events to that url, http://<whatever>:8001/connection/<name_of_github_connection>/payload22:43
jlk(send type json)22:43
jlkoooooh!22:57
jlkI think I just found a rather interesting bug!22:57
jlkcan somebody help me rubber ducky explain this?22:57
mordredjlk: I can try?22:59
jlkOkay, so I have a .zuul.yaml file in a untrusted repo23:00
jlkand in it there is a project definition, with a job23:00
jlkhttp://paste.openstack.org/show/609724/23:00
jlkthis is on master branch23:00
jlkTo test a pull request action, I opened a pull request, within the same repo. So then there was a second branch, "ferb" that had this exact same file.23:01
jlkZuul, when started, poked at things to find configs, found multiple branches of this repo, found zuul files in both branches, and eventually blew up23:02
jlkRemove the second branch, everything starts up fine23:02
jlksomething in the code that's trying to mash together config from multiple branches (whyyyy) is creating havoc23:02
jlkand I don't know that we have a test for this scenario anywhere in zuul23:04
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Update devstack plugin for security groups  https://review.openstack.org/46470923:07
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Add check for ZK node empty data  https://review.openstack.org/46466523:07
jlkmordred: does that sound sane?23:15
* mordred reading/grokking23:15
mordredjlk: it sounds sane - although not immediately why it broke. I believe the reason we look for config in branches is so that, for instance, nova's stable/newton branch can have one set of jobs and nova's master branch can have a different set of jobs - although typing that I feel stupid, so I might be wrong23:17
mordredI would have thought that things in the branch's .zuul.yaml would have an implied branch: ferb on them23:18
mordredbut at the very least I think this points to a place where there's either an edge case, a need for better docs - or both23:18
jlkyeah I don't know what zuul does internally when presented with multiple branches of zuul files23:39
mordredjlk: zuul/configloader.py _getImpliedBranches23:44
mordredthere's also a comment down in fromYaml23:45
jlkoh this is relatively new code I think23:48
jlkMay 3rd23:48
jlka5a12494dc34866388a772c75f79c40f4caa9edb23:48
mordredjlk: yah - so I think the intent is that what you did shoudl work and not confuse anything23:51
mordredjlk: so maybe something about the noop job is, in fact, the thing that's breaking here?23:52
jlkI think there may be a small bug in how the variant is initialized and something is getting a list object instead of a dict23:52
mordredoh - well that would also suck23:52
jlkI'm still reading the code23:52
jlkI need to figure out a better way to pdb around in this container, I'm not sure how to restart this execution23:52

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!