Wednesday, 2018-04-25

tristanCSpamapS: beside the image management part, it should work as expected00:19
tristanCthough i only tested it with the freenium plan, so it may needs some adjustment for more intensive usage00:20
SpamapSYeah just that I don't have to do all the baseline stuff of being the second driver. ;)00:23
*** yolanda_ has joined #zuul00:24
tristanCalso, we should probably have a global zuul auth section in nodepool.yaml to register the zuul ssh key00:24
tristanCit would be easier for driver to setup zuul access00:25
*** yolanda__ has quit IRC00:27
tristanCtobiash: note that there is a spec to implement cli action over zuul-web api: https://review.openstack.org/56232100:46
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an Amazon EC2 driver  https://review.openstack.org/53555801:17
*** bhavik1 has joined #zuul03:37
*** CrayZee has quit IRC03:38
openstackgerritMerged openstack-infra/zuul master: Don't try to delete non-existing local refs  https://review.openstack.org/56398203:47
*** swest has joined #zuul05:00
*** swest has quit IRC05:01
*** swest has joined #zuul05:02
*** bhavik1 has quit IRC05:26
*** AJaeger has quit IRC06:14
*** AJaeger has joined #zuul06:17
openstackgerritFelix Schmidt proposed openstack-infra/zuul master: Add start and end timestamp to task result in zuul_json callback  https://review.openstack.org/56388806:34
*** AJaeger has quit IRC06:35
*** AJaeger has joined #zuul06:38
openstackgerritFelix Schmidt proposed openstack-infra/zuul master: Add start and end timestamp to task result in zuul_json callback  https://review.openstack.org/56388807:24
*** jpena|off is now known as jpena07:42
*** electrofelix has joined #zuul07:58
*** ssbarnea_ has joined #zuul08:00
openstackgerritTobias Henkel proposed openstack-infra/zuul master: reporter: do not expect branch attribute in Tag object  https://review.openstack.org/56070008:10
tobiashtristanC: fixed your test case ^08:10
tobiashyou missed the sql reporting in the tag pipeline08:11
tristanCtobiash: thanks!08:14
*** xinliang has quit IRC08:49
*** xinliang has joined #zuul08:51
*** xinliang has quit IRC08:51
*** xinliang has joined #zuul08:51
mordredfungi: to be fair to npm devs (what the heck am I saying) the upath lib declares a max boundary which us installing node10 violates - so I'd say actually the system there is working and possibly prevented us from something being broken by the release09:58
mordred(like, prevented something from being broken at runtime by throwing a compile-time assert, which is way better than "we're goingto update a thing you can't reasonably have more than one version of without a way of you choosing behavior" :)09:59
*** gregdek has quit IRC11:19
*** gregdek has joined #zuul11:20
*** mattclay has quit IRC11:33
*** mattclay has joined #zuul11:33
*** jpena is now known as jpena|lunch11:35
*** mario-goulart has joined #zuul11:46
*** pwhalen_ has joined #zuul12:11
*** pwhalen has quit IRC12:11
*** yolanda_ has quit IRC12:23
*** gothicmindfood has quit IRC12:23
*** jpena|lunch is now known as jpena|off12:25
*** yolanda_ has joined #zuul12:26
*** gothicmindfood has joined #zuul12:26
*** eventingmonkey has quit IRC12:26
*** eventingmonkey has joined #zuul12:28
*** jpena|off is now known as jpena12:41
tobiashtristanC, corvus: I re-reviewed the mqtt driver and revoked my +2 because we should think if we want to change the message format slightly12:51
*** hashar has joined #zuul12:55
*** pwhalen_ is now known as pwhalen12:55
*** pwhalen has quit IRC12:56
*** pwhalen has joined #zuul12:56
tristanCtobiash: that sounds good, i'll have a look tomorrow13:08
fungimordred: yeah, seems sane enough. i just couldn't let the close timing of pip 10 and npm 10 pass unnoticed13:09
fungi(pip 10 is not especially broken either, they've just made some choices in the name of progress which require us to modernize a questionable practice)13:11
*** maxamillion has quit IRC13:12
*** maxamillion has joined #zuul13:13
*** hashar has quit IRC13:15
*** maxamillion has quit IRC13:24
*** maxamillion has joined #zuul13:24
mordredfungi: oh - TOTALLY :)14:09
*** dkranz has joined #zuul14:24
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: [WIP] Colors in job-output.txt  https://review.openstack.org/56406614:29
tobiashcorvus, clarkb: I think it would be good to have 560700 in the next release as it fixes sql reporting on tag triggered pipelines14:39
corvustobiash: well, we just restarted yesterday -- can we maybe get that one in the next release?14:43
tobiashcorvus: oh, didn't notice that you already restarted14:44
tobiashNext release is fine then14:44
corvustobiash: we also just missed your merger fix because of the node snafu :(14:45
tobiashcorvus: as I'm not tag driven I'm fine with having it in master :)14:46
Shrewsmordred: fyi, there is a sort of mini-discussion on https://review.openstack.org/536862 when you get a chance. tl;dr it would be nice if there was a way to make ttrun work the way it used to before stestr14:53
Shrewsmordred: but if there isn't an answer other than "set env vars when running", i guess i can live with it. just wondering if you had any other ideas14:55
mordredShrews: I should go read that ... in the zuul change I had to do something in the base test case - but the intent is that ttrun works the way it used to before stestr14:58
mordredShrews: so let's consider the fact that it doesn't right now to be a -1 worthy bug14:58
Shrewsokie dokie14:59
*** yolanda_ has quit IRC15:01
*** gothicmindfood has quit IRC15:01
*** jhesketh has quit IRC15:01
*** xinliang has quit IRC15:01
*** electrofelix has quit IRC15:01
*** sshnaidm|off has quit IRC15:01
*** pbrobinson has quit IRC15:01
*** toabctl has quit IRC15:01
*** nguyenhai has quit IRC15:01
*** rcarrillocruz has quit IRC15:01
*** zigo has quit IRC15:01
*** snapiri has quit IRC15:01
*** adam_g has quit IRC15:01
*** ChanServ has quit IRC15:01
*** nguyenhai has joined #zuul15:01
*** rcarrillocruz has joined #zuul15:01
*** zigo has joined #zuul15:01
*** snapiri has joined #zuul15:01
*** adam_g has joined #zuul15:01
*** ChanServ has joined #zuul15:01
*** barjavel.freenode.net sets mode: +o ChanServ15:01
*** nguyenhai has quit IRC15:02
clarkbShrews: mordred the idea I had was to check if invoked under testtools.run or subunit.run and change the behavior default in the tests based on that15:02
clarkbI think you can check the argv to get that info15:02
*** jhesketh has joined #zuul15:03
mordredclarkb: if it's like zuul I think there's actually likely a bug we can fix before needing to get that fancy - although that's a good idea if the other thing doesn't work15:03
*** yolanda_ has joined #zuul15:03
*** gothicmindfood has joined #zuul15:03
mordredlike - I have this working the way we want without needting to set env vars in several repos already15:03
Shrewsoh good15:04
*** xinliang has joined #zuul15:04
*** electrofelix has joined #zuul15:04
*** sshnaidm|off has joined #zuul15:04
*** pbrobinson has joined #zuul15:04
*** toabctl has joined #zuul15:04
*** nguyenhai_ has joined #zuul15:04
*** weshay has quit IRC15:04
mordred*but* - I like the way clarkb thinks :)15:05
*** nguyenhai_ has quit IRC15:05
*** nguyenhai_ has joined #zuul15:05
*** weshay has joined #zuul15:06
Shrewsmordred: this seems to be an error message from yarn in the nodepool-zuul-functional job: http://logs.openstack.org/45/564045/2/check/nodepool-zuul-functional/c4a75b8/job-output.txt.gz#_2018-04-24_20_51_45_13756515:24
Shrewsnot quite sure how to unravel that15:25
*** rlandy has joined #zuul15:25
clarkbShrews: I think that may have been the issue corvus fixed yesterday. Does it complain about upack and >4 < 9?15:25
clarkbif so a recheck should get it moving15:25
Shrewsclarkb: yeah15:25
Shrewscool. thx15:25
clarkbbasically nodejs 10.0.0 released. upack doesn't support >9 nodejs. Corvus updated our nodeenv install to install 8.x (latest stable)15:26
corvuswell, second latest stable :)15:30
clarkber right15:31
dimsanyone have an example of how i can specify a canonical_hostname in my .zuul.yaml?15:43
clarkbdims: I think that is determined by the zuul server configuration not the per project configuration15:44
clarkbdims: it has to do with where the repo is hosted canonically from zuul's view which is in the connection configuration15:45
*** andrea06590 has joined #zuul15:45
pabelangerzuul.project.canonical_hostname is the current value of it, from the inventory15:45
dimsclarkb : kubernetes uses vanity urls. github.com/kubernetes/cloud-provider-openstack uses k8s.io/kubernetes/cloud-provider-openstack as package name ... i wrongly assumed ?15:45
mordreddims: wow, that's awesome15:46
dimsguess i have to try some gymnastics ..15:46
clarkbin this case the canonical name would be github.com because that is wher the repo lives right?15:47
clarkbbut I guess that breaks the golang idea of git repos are the thing and they are all on github expcet for when they are not15:47
corvusclarkb: golang doesn't require github15:48
dimsthey tell folks to checkout code under $GOPATH/src/k8s.io ...15:48
corvus(and the golang format is exactly what zuul produces -- on purpose)15:49
clarkbcorvus: iirc there is a whitelist of urls that you have to get yourself added into to function with some tool15:49
corvusdims: maybe you could set up a special connection to github for k8s, and set the canonical hostname on that connection to k8s.io?15:49
clarkbso its not arbitrary urls but ya they are of that format15:49
clarkb`go get`?15:49
*** andrea06590 has quit IRC15:49
dimsin the openlab zuul run, /home/zuul/src/github.com/kubernetes/cloud-provider-openstack is where the code gets put down15:50
dimscorvus : y, i have to try some stuff15:50
corvusdims, mrhillsman: https://zuul-ci.org/docs/zuul/admin/drivers/github.html#attr-%3Cgithub%20connection%3E.canonical_hostname15:51
corvusthat's what we use in openstack to trick our zuul in to calling things "git.openstack.org/openstack/nova" instead of "review.openstack.org/openstack/nova"15:51
corvusi... never expected it to be used with github though.  but it should be supported.15:52
dimscorvus : fun right? :)15:52
corvusit will mean you'll have more than one connection to github.  one for just k8s.io projects, and another for github.com projects.15:52
dimscorvus : ack will try that15:53
dimsthanks15:53
mordreddims: does go get github.com/kubernetes/cloud-provider-openstack do the riht thing?15:53
* mordred is curious if there is a general declaration mechansim somewhere15:53
dimsmordred : best practice there is to use "go get k8s.io/kubernetes/cloud-provider-openstack"15:53
mordredGOTCHA15:54
mordredso yeah - what corvus said15:54
dimsas long i don't have to re-register the github app thingy15:54
dimsdo i have to?15:54
mrhillsmannope15:55
corvusdims: that's a really good question.  the app id is associated with the connection, but let me see what happens if you have 2 connections with the same app id15:56
dimsthanks corvus !15:56
dimsmrhillsman : you caught the full conversation right?15:57
mrhillsmanwell I thought no because. Note that Zuul will still only communicate with the GitHub server identified by server;15:57
mrhillsmanI'm reading back but I think I got it15:57
dimsthanks mrhillsman15:57
mrhillsmanAs I was reading, the src/github.com is problematic because best practice is k8s.io15:58
mrhillsmanbut now that I think about it, may have to register another app :(15:59
clarkbmrhillsman: sounds like corvus is going to test it15:59
corvusdims: the github app specifies the zuul connection name, so yeah, a second connection needs a second app16:00
mrhillsmanand it would make sense to have to16:00
corvusclarkb: well, i went to read code :)16:00
corvusmrhillsman: yep16:00
*** acozine1 has joined #zuul16:00
mordreda short term workaround would be to just put "ln -s src/github.com src/k8s.io" in before doing other things16:00
corvusit does seem like it would be nice to be able to make an alias like this in the tenant config.  but there's a lot of stuff in zuul that hangs off the canonical hostname, so that will take some careful thought to make sure it's possible.16:01
mordredthat way go will find the source code repos at $GOPATH/src/k8s.io when it expects to16:01
mordredcorvus: ++16:01
corvusmordred: yeah, that might not be a bad idea, if it's possible to incorporate those into the relevant jobs.16:02
dimsmordred : y that's what we did and "dep ensure" croaks16:04
mrhillsmanI need to catch up on this cause ^16:04
dimsmrhillsman : don't want to do a second app right away .... should be the last resort16:05
mrhillsmanI need to check the jobs as I am not sure how/when that symlink is created16:05
mrhillsmanGotcha dims16:05
dimscorvus : mordred : can we muck with zuul's data to hack this in?16:06
pabelangerdims: you can write a new role that updates prepare-workspaces to sync the repos into the right location, if symlinks fail16:06
clarkbfwiw k8s.io/kubernetes/cloud-provider-openstack 404s for me16:06
dimsto the existing connection16:06
clarkbif I http(s) it16:06
*** hashar has joined #zuul16:06
clarkbdims: I think you can probably apply that configuration corvus linked above but it will likely break anyone else using that connection expecting github.com16:07
dimsclarkb : y understood16:07
dimspabelanger : that sounds promising16:08
mrhillsmandims do you have a link to dep ensure output?16:08
dims"both /home/zuul/src/k8s.io/cloud-provider-openstack and /home/zuul/src/github.com/kubernetes/cloud-provider-openstack are in the same GOPATH /home/zuul"16:08
dimshttp://logs.openlabtesting.org/logs/39/139/3d22a1a45af104946b6cac6c03f89151fc214161/cloud-provider-openstack-unittest/cloud-provider-openstack-unittest/823ad14/job-output.txt.gz16:08
mordredmrhillsman, dims: you could also mv the dir if dep ensure croaks on symlinks16:08
mrhillsmanpabelanger: I think that is good16:08
dimsmordred : yep16:08
mrhillsmanYeah, we have a few options for sure16:08
mrhillsmanI need to look at the playbooks16:09
mrhillsmanjobs16:09
corvusdims: the quickest thing is going to be doing it in the jobs; so maybe mv as mordred suggests.  even the "easiest" changes to zuul's config are not going to be trivial.  this is fundamental stuff :)16:09
clarkbwait why have url scoped "packages" if you can't have two packages with the same name on different hosts16:09
*** andreaf_ has joined #zuul16:09
mordredI'd do mv as a short-term rather than a modified prepare-workspaces role - just so that it's a small surface area16:09
mrhillsmanWhatever lol16:09
mordredwhat corvus said16:09
mrhillsmanOk mordred16:09
dimsgotcha mordred corvus16:10
*** andreaf has quit IRC16:10
mordredlike, assume the base job is going to put things into zuul locations - then make a k8s-base job for things that are going to clone k8s repos that does mvs (you might want to only do mv src/github.com/kubernetes src/k8s.io/kubernetes)16:10
corvusclarkb: maybe it's because they both declare they are the same package?16:11
*** andreaf_ is now known as andreaf16:11
clarkbcorvus: maybe? I thought the url was part of the code declaration too (which is why dims needs to address this in this way I expect)16:11
dimsmrhillsman : will check back later ... need to get on a call16:11
mordredwell - I think the tooling assumption is that you would have used go get to clone the repos16:12
clarkbit seems like a weird design choice to scope thigns then ignore the scoping16:12
mordredand if you did that, having both would indicate an error16:12
mordredthat you should probably know about16:12
mrhillsmandims: ok16:12
corvusclarkb: right, but both github/foo and k8s/foo have a file in them that says "i am k8s/foo"...  that's what i'm *assuming* is the problem16:12
mrhillsmanwell that is true in the case of go proper16:13
mordredreconciling two systems that want to clone git for you turns out to be fun :)16:13
SpamapSI have a question about trusted/untrusted roles...16:13
mrhillsmanbut k8s practice is to use their namespace16:13
SpamapSYesterday I tried to make a generic "fetch all the journalctl outputs" role and put it in my shared jobs repo (godaddy-zuul-jobs)16:13
mrhillsmananywho, thanks for the insight gents16:14
SpamapSBut when I pulled that role in on my base job.. it failed because zuul.executor.log_root was readonly.16:14
mordredsounds like a nice role16:14
corvusSpamapS: log_root shouldn't be readonly for anyone16:14
SpamapSIs there something I missed? Is there an intermediary place to dump logs that will get pulled in later?16:14
corvusSpamapS: can you paste the error?16:15
SpamapSHm maybe I misinterpreted the error.16:15
mordredSpamapS: (did you accidentally run the role on the remote node expecting it to run on the executor, and thus zuul.executor.log_root didn't actually exist?)16:15
mordredor something like that - but yeah, if  you can share error that would be neat16:16
SpamapSah no16:16
SpamapSit was not log_root, good catch16:16
SpamapSit was zuul.project.src_dir16:16
mordredSpamapS: also ... I AM SORRY - I was supposed to have improved log collection months ago and still haven't16:16
corvuswrite your error message on an index card and put it in a white envelope and carnac the magnificent error message oracle will decipher it for you16:17
SpamapShttp://paste.openstack.org/show/719936/16:18
SpamapShere are the relevant sections16:18
SpamapSI think I'm using src_dir as scratch space16:18
SpamapSand that doesn't work :-P16:18
SpamapS2018-04-24 17:24:26.312446 | POST-RUN START: [trusted : github.secureserver.net/cloudplatform/project-config/playbooks/base/post.yaml@master]16:19
SpamapSrunning trusted.. so it's not that16:19
corvusSpamapS: zuul_work_dir is generally used in roles that expect to run on the remote node, so it has no prefix, it just starts with "src/", under the assumption that the task is running in the remote user's home directory16:20
corvusSpamapS: in this case, when running on the executor, it just ends up with a relative path, and pwd is usually the playbook16:21
corvusSpamapS: so that's actually trying to write to something outside the work dir.. like /trusted/project_0/project-config/src/github.... etc16:22
corvusSpamapS: since you're operating on the executor, you probably want to do what you said earlier: use log_root instead16:22
SpamapSI'm not sure why this was running on localhost actually.16:27
SpamapSoh derp16:29
SpamapSthis is in my base post playbook16:29
corvusSpamapS: oh you want to run it on the remote node to do a journalctl export first?16:29
SpamapSwhich only does stuff on localhost.16:29
SpamapSderp derp derp16:29
openstackgerritMonty Taylor proposed openstack-infra/nodepool master: Switch to stestr  https://review.openstack.org/53686216:29
SpamapScorvus: right16:29
SpamapSI think I have it now16:29
SpamapSI was lazy, not reading the playbook16:29
corvusSpamapS: you should be able to have that playbook run tasks on "all:" hosts16:29
SpamapSneed to have a 'hosts: all' play before the 'hosts: localhost'16:29
corvusya, cool.16:29
mordredShrews: ^^ I do not have internet bandwidth enough to validate that patch16:29
* SpamapS recedes into the shadows16:29
mordredShrews: but I believe it will provide you with more joy16:30
mordredSpamapS: come back! we like you!16:30
SpamapScorvus: you make an excellent rubber ducky16:30
corvusi make bathtime lots of fun!16:30
corvusi'm... gonna go work on memory caches and stuff.16:30
openstackgerritStephen Finucane proposed openstack-infra/zuul-jobs master: Default warning-is-error to True for non-legacy Sphinx projects  https://review.openstack.org/55934816:32
Shrewsmordred: i demand ponies, full of joy and goodness and joyness16:32
*** mrhillsman has quit IRC16:43
*** mrhillsman has joined #zuul16:43
*** ssbarnea_ has quit IRC17:08
*** jpena is now known as jpena|off17:12
*** hashar is now known as hasharAway17:13
clarkbcorvus: followup from yesterday's zuul restart, I've not noticed any issues17:14
*** acozine1 has quit IRC17:24
mordredShrews: I have boxes of rhinos17:36
corvusclarkb: cool, so maybe let's go ahead and tag it?17:46
clarkbcorvus: I think so17:46
corvuswow, there's 33 changes from 3.0.1 to 623533ab85ddf406bcac16452a638a706682775a17:49
corvusi'm going to tag http://git.zuul-ci.org/cgit/zuul/commit/?id=623533ab85ddf406bcac16452a638a706682775a as 3.0.2 -- it's what openstack-infra has been running since yesterday17:50
corvushttps://zuul-ci.org/docs/zuul/releasenotes.html#in-development doesn't show any major changes17:52
corvusclarkb, fungi, mordred, tobiash, Shrews: ^ sound good?17:53
tobiashcorvus: lgtm17:54
fungiyep, 3.0.2 seems fine17:54
clarkbya largely bug fixes I think17:54
fungihaven't seen any regressions in production today anyway17:54
corvusi also think nodepool has seen enough changes to warrant a release, but i'm not sure if we're running master.  maybe we should work on that next.  (i don't think it needs to be coordinated with zuul, just while we're thinking about it)17:54
fungiright, we agreed we weren't going to sync nodepool release times nor version numbers to zuul17:55
clarkbya likely needs a similar global restart and watch it for ~12 hours then call it good17:55
fungibut a nodepool release also seems warranted17:55
corvuszuul 3.0.2 pushed17:55
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add regex support to project stanzas  https://review.openstack.org/53571317:57
tobiashclarkb: that should address your comments ^17:58
*** ssbarnea_ has joined #zuul18:01
*** pwhalen has quit IRC18:04
*** pwhalen has joined #zuul18:08
*** ssbarnea_ has quit IRC18:12
*** ssbarnea_ has joined #zuul18:13
Shrewsi think there are some big enough nodepool changes that i'd like to see that run for a few days at least before another release18:22
*** ssbarnea_ has quit IRC18:32
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Fix race in test_hold_expiration_no_default  https://review.openstack.org/56431418:55
Shrewsmordred: ^^ is the reason your stestr change failed18:55
Shrewsrechecking it for you18:56
Shrewsoh shoot, that's wrong18:57
Shrewsgrrrr18:57
*** ssbarnea_ has joined #zuul19:03
*** myoung is now known as myoung|dr19:03
*** bhavik1 has joined #zuul19:09
Shrewshrm, i'm not certain how this test has been working at all, actually19:14
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Fix race in test_hold_expiration_no_default  https://review.openstack.org/56431419:25
Shrewsthat should do it19:25
*** lsell has joined #zuul19:33
*** bhavik1 has quit IRC20:10
*** myoung|dr is now known as myoung20:41
pabelangerOooh, a logo21:17
*** openstackgerrit has quit IRC21:20
*** ssbarnea_ has quit IRC21:55
*** hasharAway has quit IRC22:02
*** rlandy has quit IRC22:11
*** threestrands has joined #zuul22:17
*** openstackgerrit has joined #zuul22:45
openstackgerritJames E. Blair proposed openstack-infra/zuul master: [WIP] Cache configuration objects in addition to YAML dicts  https://review.openstack.org/56406122:45
corvusthat's really close to ready, but i'm going to split out some changes to make it more reviewable22:46
*** threestrands_ has joined #zuul23:35
*** threestrands has quit IRC23:38

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!