Thursday, 2017-12-14

openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add implied-branches pragma directive  https://review.openstack.org/52780500:07
*** harlowja has quit IRC00:45
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add debug project-pipeline option  https://review.openstack.org/52782801:19
*** JasonCL has quit IRC01:26
*** jkilpatr has quit IRC02:00
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add debug project-pipeline option  https://review.openstack.org/52782802:00
*** baiyi has joined #zuul02:05
*** baiyi has quit IRC02:09
*** threestrands_ has joined #zuul02:22
*** threestrands_ has quit IRC02:22
*** threestrands_ has joined #zuul02:22
*** threestrands has quit IRC02:22
*** threestrands_ has quit IRC02:22
*** threestrands_ has joined #zuul02:23
*** threestrands_ has quit IRC02:23
*** threestrands_ has joined #zuul02:23
*** threestrands_ has quit IRC02:32
*** dmellado has quit IRC02:32
*** ChanServ has quit IRC02:32
*** ChanServ has joined #zuul02:34
*** barjavel.freenode.net sets mode: +o ChanServ02:34
*** threestrands_ has joined #zuul02:36
*** dmellado has joined #zuul02:36
*** JasonCL has joined #zuul02:37
*** JasonCL has quit IRC03:24
*** JasonCL has joined #zuul03:25
*** shunde has joined #zuul05:10
shundeHi, can zuulv3's github driver work with a local bitbucket?05:11
jlkDoes bitbucket implement the GitHub REST API ?05:13
jlkZuul depends on that REST API for nearly everything it does, getting pull request details, setting pull request commit status updates, comments, merging, etc.. as well as depending on the format of the webhook payloads sent from github.05:14
jlkshunde: I'd assume that the answer is no.05:14
shundeit has rest api: https://developer.atlassian.com/static/rest/bitbucket-server/latest/bitbucket-rest.html05:16
shundenot sure if it is the same as github05:16
*** shunde has quit IRC05:58
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: fix key request route  https://review.openstack.org/52787006:43
*** flepied__ has quit IRC06:44
*** openstackgerrit has quit IRC06:47
*** openstackgerrit has joined #zuul06:50
openstackgerritMerged openstack-infra/zuul feature/zuulv3: doc: refine zuul_return instruction  https://review.openstack.org/52763306:50
*** threestrands_ has quit IRC06:53
SpamapS It would be cool to add a bitbucket driver. And a Stash driver. And a gitlab driver. 3.2? ;-)06:56
tobiashFirst needs, first implements ;)06:56
* SpamapS peeks at the zuulv3.0 tag to see if there's anything to be done now06:56
openstackgerritMerged openstack-infra/zuul feature/zuulv3: web: fix key request route  https://review.openstack.org/52787007:20
*** flepied__ has joined #zuul07:34
*** hashar has joined #zuul08:00
*** Wei_Liu has quit IRC08:03
*** flepied_ has joined #zuul08:27
*** flepied__ has quit IRC08:30
*** Wei_Liu has joined #zuul08:31
*** jpena|off is now known as jpena08:46
*** flepied_ is now known as flepied09:08
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: remove 'id' column from builds page  https://review.openstack.org/52791409:46
openstackgerritKrzysztof Klimonda proposed openstack-infra/zuul feature/zuulv3: Support autoholding nodes for specific changes/refs  https://review.openstack.org/51516909:54
openstackgerritKrzysztof Klimonda proposed openstack-infra/zuul feature/zuulv3: Support autoholding nodes for specific changes/refs  https://review.openstack.org/51516909:56
tobiashtristanC: +2 with comment on https://review.openstack.org/52791409:57
tobiashmordred: ^^ might interfere with your javascript stuff09:59
*** electrofelix has joined #zuul09:59
tristanCtobiash: mordred: sure, no worries this is just cosmetic10:28
*** jkilpatr has joined #zuul10:46
openstackgerritMerged openstack-infra/zuul feature/zuulv3: web: remove 'id' column from builds page  https://review.openstack.org/52791410:52
*** tumbarka has quit IRC11:08
*** umbarkar has joined #zuul11:08
*** jkilpatr has quit IRC11:08
Wei_Liuhello, how can I replace the finger url which posted in gerrit comment with http url?11:46
rcarrillocruzi think what you need to change is webapp.status_url11:47
rcarrillocruzhttps://docs.openstack.org/infra/zuul/feature/zuulv3/admin/components.html11:47
rcarrillocruzhmm, so11:47
rcarrillocruzit seems11:48
rcarrillocruzthe executor11:48
rcarrillocruzand merger11:48
rcarrillocruzneed [connection] section11:48
rcarrillocruzhowerver, i don't think it requires app_id app_name, does it?11:48
rcarrillocruzfrom code it seems the merger also uses the key, not sure for what11:48
Wei_Liuyes, I changed webapp.status_url, but it didn't work.11:50
Wei_Liuit is still the finger url in my gerrit comment.11:50
Wei_Liuhello, https://docs.openstack.org/infra/zuul/feature/zuulv3/admin/components.html, in this guide, there is a webapp in scheduler and one web server, what is different?11:55
*** jpena is now known as jpena|lunch12:01
rcarrillocruzWei_Liu: yeah, that's confusing12:05
rcarrillocruzso12:05
rcarrillocruzthe one in the scheduler12:05
rcarrillocruzjust gets the webhooks from github12:05
rcarrillocruzshould your zuul have github integration12:06
rcarrillocruzthe zuul-web provides the job logs streaming and the dashboard12:06
rcarrillocruzi.e. https://zuulv3.openstack.org/12:06
Wei_Liuno, we only use gerrit12:06
rcarrillocruzbut tristanC and jlk are going to move over the github webhook part to zuul-web12:06
Wei_Liuand zuul-web also provide secret public12:08
Wei_Liukeys12:08
rcarrillocruzyes, it has a route for that iirc12:09
tristanCrcarrillocruz: actually the change is up for review here https://review.openstack.org/#/c/504267/12:09
rcarrillocruzah neat12:10
rcarrillocruztristanC: do you know why merger would need connection key?12:10
tristanCrcarrillocruz: what do you mean by connection key?12:10
rcarrillocruzi mean, the github key is supposed to be for securing the webhooks messaging,12:10
tobiashrcarrillocruz: the merger needs to know the private ssh key for accessing the repos12:10
rcarrillocruzif the merger just clones and prepares repos not sure what is needed12:10
tristanCrcarrillocruz: sorry, i don't really know how github works12:11
rcarrillocruzi never realized this deps since i always ran zuul as AIO, but running all components on their own just saw this12:11
rcarrillocruzexecutor and merger need connection section12:12
rcarrillocruzanyway, i'll poke12:12
Wei_Liuin zuul project, what is the use of zuul/etc/status/public_html/* ?12:13
tristanCWei_Liu: it's deprecated, this is now served by the zuul-web,  this change actually remove this https://review.openstack.org/#/c/487538/12:14
Wei_Liuproviding job logs streaming like https://zuulv3.openstack.org/?12:14
tristanCWei_Liu: job logs streaming is also provided by the zuul-web, you can ignore the etc/status directory entirely12:17
Wei_LiutristanC: I found out that there are finger url in my gerrit comment, how can I change it to http?12:21
*** baiyi has joined #zuul12:22
Wei_LiuI changed web-app.statue_url, but it did not work.12:23
tobiashWei_Liu: you need a zuul_return task in your job12:24
tobiashthe job now tells zuul the correct log location12:24
tobiashWei_Liu: see last task: http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/upload-logs/tasks/main.yaml#n4712:24
tobiashWei_Liu: the finger urls are just a fallback12:25
Wei_Liutobiash: thanks, I just remove the post-logs.yaml from base since it cause some failure, I will fix it.12:32
*** jpena|lunch is now known as jpena13:00
*** flepied_ has joined #zuul13:01
*** flepied has quit IRC13:04
*** smyers has quit IRC13:13
*** flepied__ has joined #zuul14:01
*** flepied_ has quit IRC14:04
*** sshnaidm is now known as sshnaidm|mtg14:05
*** dkranz has joined #zuul14:20
Shrewsclarkb: after some thinking, left a suggestion on what i think we should do on 526234. interested to hear what you think.14:20
clarkbShrews: cool, I'll take a look shortly14:22
tobiashShrews: sounds good14:25
*** hashar has quit IRC14:37
*** hashar has joined #zuul14:37
clarkbShrews: tobiash comments seem reasonable to me, I'll get a new ps up14:52
*** JasonCL has quit IRC14:57
*** JasonCL has joined #zuul14:58
*** flepied_ has joined #zuul15:01
openstackgerritClark Boylan proposed openstack-infra/nodepool feature/zuulv3: Handle invalid request lock in request cleanup  https://review.openstack.org/52623415:03
clarkbShrews: tobiash ^ how does that look?15:03
*** flepied__ has quit IRC15:03
Shrewsclarkb: lgtm15:08
tobiash++15:08
Shrewsi'll start trying to hunt down the real bug, but only have today and tomorrow, then gone for a couple of weeks. hopefully someone can look next week if i don't find anything15:09
*** hashar is now known as hasharAway15:46
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Handle invalid request lock in request cleanup  https://review.openstack.org/52623415:50
jeblairclarkb, Shrews, tobiash: is it correct that we don't understand why the node request was unlocked in https://review.openstack.org/526234 ?15:56
clarkbjeblair: yes, however its less the that node request was unlocked and more that the lock itself became invalid (was set to None)15:57
clarkbafter the node request itself had been removed15:57
Shrewsjeblair: which *could* have happened if it were unlocked, but I do not see a code path that would unlock it15:57
Shrewswhich is why i suspect an overwrite15:57
jeblairclarkb: self.request.lock == None ?15:58
clarkbShrews: aiui that is what tobias and shrews concluded from their investigating yes15:58
jeblairand we saw the exception: "Request %s does not hold a lock" % request  ?15:58
clarkbjeblair: http://paste.openstack.org/show/628319/ is the traceback we saw (which is that string)15:59
Shrewsjeblair: launcher-debug.log.2017-12-05_09 on nl02 has the first occurrence of it (looking more at it now to see if i can determine what triggered it)16:00
Shrewsooh, there is this weird exception before it: Exception: Attempt to update non-existing request16:02
Shrewsbut that's for a different request16:02
jeblairthis change papers over a logic error.  that's okay as long as someone is tracking down the root cause, but now we need to stay on top of this and see it through.16:03
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Update javascript jobs to take npm_command variable  https://review.openstack.org/52777016:03
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Install yarn if needed in javascript jobs  https://review.openstack.org/52777116:03
Shrewsjeblair: totally agree16:03
Shrewsi pretty much suggested that in the review16:04
Shrewshopefully i can find it soon16:04
clarkbout of curiousity why is it that we can't lock the request znode itself?16:05
clarkb(seems like a lot of the complication here is in keeping two znodes roughly in sync for the purposes of locking)16:05
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Install yarn if needed in javascript jobs  https://review.openstack.org/52777116:08
*** flepied_ has quit IRC16:10
jeblairclarkb: i don't think ephemeral nodes (requests are ephemeral) can have children16:13
*** JasonCL has quit IRC16:16
Shrewsclarkb: jeblair: oh! i may have found the problem16:16
Shrewsand i think clarkb's change actually fixes it16:16
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Use yarn and webpack to manage status and streaming  https://review.openstack.org/48753816:17
clarkbjeblair: I see so you can't lock a terminal node and use it to store data as well?16:17
clarkbs/terminal/leaf/16:17
Shrewsclarkb: i *think* what happened is the request handler was removed for a completed request. but while processing other handlers, one raised an exception, causing us to leave the completed request still  on the list. when the list was processed again, our lock was None b/c we had unlocked it, expecting it to be removed from the list16:19
clarkbaha16:19
jeblairso the second part is the fix16:19
*** JasonCL has joined #zuul16:19
clarkbsounds like it16:19
Shrewsclarkb: so now that you catch Exception around that, we should properly remove it16:19
Shrewsand our live handler list gets updated properly16:20
Shrewsclarkb: i have NO idea on how to test that16:21
* Shrews brb16:22
*** sshnaidm|mtg is now known as sshnaidm16:44
*** jkilpatr has joined #zuul16:51
*** flepied_ has joined #zuul17:04
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Docs: group matchers together and explain them  https://review.openstack.org/52774817:07
jeblairfbo_: i left a comment on the git driver change 52561417:16
*** myoung is now known as myoung|lunch17:28
jeblairtobiash: what do you think of rcarrillocruz comment in https://review.openstack.org/501976 ?17:30
*** logan- has quit IRC17:44
*** logan- has joined #zuul17:45
*** flepied_ has quit IRC17:59
*** flepied_ has joined #zuul17:59
rcarrillocruzhrmpf18:12
rcarrillocruzAttributeError: 'module' object has no attribute 'inf'18:12
rcarrillocruzlooks like nodepool requires 3.5 now, after the cloud quota handling changes18:12
rcarrillocruzi'll need to build nodepool with software collections18:13
clarkbrcarrillocruz: I think we've been saying 3.5 since the type annotation stuff went in18:13
clarkb(though I guess taht was zuul not nodepoo)18:13
rcarrillocruzyeah, i had zuul months ago just for the asyncio thing18:13
pabelangerno, we've drop 2.7 support on nodepool a while back18:13
pabelangerbut we are still running python2.7 builders18:13
clarkbpabelanger: rcarrillocruz right but 3.x worked. but when we added type annotations you needed 3.5 or greater18:13
pabelangerclarkb: ah18:14
clarkb3.4 worked before type annotations, I tested it when testing the gerrit upgrade on review-dev18:14
rcarrillocruzyeah, was bitten yesterday by that18:14
rcarrillocruzas centos comes with 3.418:14
rcarrillocruzdoes not 3.518:14
rcarrillocruzbut, i just pulled SCL and made it work on the image entrypoint18:15
rcarrillocruzwill do the same for nodepool18:15
rcarrillocruzpabelanger , leifmadsen : btw, i glanced over your docs etherpad, see you use the cloudera zookeeper package18:16
rcarrillocruzi use bigtop repo, and works for me too18:16
rcarrillocruzin case you want to use something coming from apache itself18:16
*** jpena is now known as jpena|off18:16
clarkbrcarrillocruz: one of my suggestions in the meeting the other day was to use the official tarball instead as that will work on any distro18:16
clarkb(it is what I use on suse)18:17
rcarrillocruzyah, used that also in some other POC18:17
pabelangeryah, fedora does ship zookeeper today too18:17
pabelangerso, not sure why that is there still18:17
rcarrillocruzcos dunno, wasn't comfortable using cloudera18:17
rcarrillocruznot aware how well is that maintained18:17
pabelangerbut agree, maybe we just use tarball if no native disto package18:17
pabelangerrcarrillocruz: +118:17
*** sshnaidm is now known as sshnaidm|afk18:18
dmsimardtobiash: was it you that had particular network requirements in regards to zuul/nodepool deployments ? I forget18:29
*** harlowja has joined #zuul18:29
dmsimardlike, firewalled off zones or segregated networks18:29
dmsimardI've been thinking -- there's this private cloud that I could tap into through a jump box. I wonder to what extent we could do this since Ansible (or just plain SSH agent forwarding/ssh config) has native support for going through jump boxes.18:31
SpamapSMy centos7 zuul uses a locally compiled Python 3.518:31
SpamapSnot ideal.. but it works18:31
tobiashjeblair: I think that should be handled by zuuls ansible overrides18:34
dmsimardSpamapS: how does that work ? like all your libs need to point back to it no ?18:34
tobiashLocal connection also can be forced within the playbook18:34
tobiashdmsimard: yes, I have the nodepool nodes within a private network in a different tenant without using floating ips18:35
tobiashAnd it works18:35
dmsimardtobiash: you have a public nodepool spawning VMs in a private cloud ?18:35
dmsimardoh, you have a *launcher* in the private network ?18:36
dmsimardand an executor as well I guess ?18:36
tobiashdmsimard: and the nodes in a different private net18:36
dmsimardtobiash: so having a public geard is the only requirement for that to work I guess18:37
tobiashdmsimard: so in my case I have two private networks in different tenants on the same cloud. One contains the control plane (zuul, nodepool, etc) and the second contains the nodepool nodes18:40
tobiashAnd that without involving floating ips for the nodepool nodes18:40
dmsimardtobiash: I think we should totally document that use case18:40
rcarrillocruzwhat i'd like to tinker with and maybe blog/document if it works18:41
dmsimardlike document two or three reference architecture deployments18:41
rcarrillocruzis using something like ultrahook18:41
rcarrillocruzlots of people can't just allow github webhooks inbound18:41
rcarrillocruznetworks always getting in the way!18:42
tobiashThe solution to this use case is having routers to the same public net and adding a static route from the control plane router to the other18:42
*** hasharAway is now known as hashar18:43
tobiashrcarrillocruz: yeah, I have always problems with our mandatory athenticating proxy18:45
tobiashThat costs so much time at thousands of places18:46
rcarrillocruztobiash: i believe you use openshift https://blog.openshift.com/using-github-hooks-with-your-local-openshift-environment/18:46
rcarrillocruzjust came across that18:46
rcarrillocruzhoping to tinker with that in a few weeks18:46
tobiashYes18:46
tobiashI don't want to run an ansible based deployment anymore ;-)18:47
rcarrillocruzhah18:47
rcarrillocruzso funny enough18:47
rcarrillocruzhttps://github.com/rcarrillocruz/zuul-ci-container18:47
rcarrillocruzusing ansible-container to create images/deploy on openshift18:47
rcarrillocruznot bare roles on VMs/servers, but CONTAINERZ hype hype18:48
rcarrillocruzeugh, so seems net neutrality is officially killed ?18:51
* rcarrillocruz checks twitter and blogs18:52
*** myoung|lunch is now known as myoung18:57
*** hashar has quit IRC19:06
dmsimardrcarrillocruz: my hype for ansible-container is frozen by https://github.com/ansible/ansible-container/issues/39919:07
*** flepied_ has quit IRC19:08
*** flepied has joined #zuul19:13
*** hashar has joined #zuul19:15
*** openstack has joined #zuul20:33
*** ChanServ sets mode: +o openstack20:33
*** pabelanger has quit IRC20:39
*** pabelanger has joined #zuul20:39
*** JasonCL has joined #zuul21:00
*** electrofelix has quit IRC21:04
*** JasonCL has quit IRC21:05
jeblairrcarrillocruz: yeah, mordred's focus of work right now is js installation.  it won't be pip install, but it should be similarly automated.21:06
*** openstack has joined #zuul21:13
*** ChanServ sets mode: +o openstack21:13
*** JasonCL has joined #zuul21:22
*** JasonCL has quit IRC21:27
*** threestrands has joined #zuul21:31
mordredrcarrillocruz, jeblair: I had a moment thisafternoon of my brain working temporarily, so I've got an update to that stack coming (hopefully)21:33
rcarrillocruzsweet21:34
mordredjeblair, rcarrillocruz: the zuul-web on zuulv3.openstack.org could do with a restart to pick up the CORS header patch - which in turn lets local testing of js things for bulds/jobs pointing to live data work ... but I think me executing commands as root is a bad idea21:35
jeblairmordred: also it's going to break the keys endpoint and needs a puppet-zuul update -- i'm shepherding another set of possibly breaking changes so i don't want to do it right now; maybe i'll take a look later if no one else has21:39
jeblairit == zuul-web restart21:39
mordredjeblair: ah, nod - it's certainly not urgent -- mostly just want to  make sure that we can see a js build with the live preview of new code against real data before landing the first patch - since it's otherwise a really big blob of change that's hard ot validate21:42
jeblairmordred: ++21:44
*** dkranz has quit IRC21:45
*** JasonCL has joined #zuul21:46
jeblairtristanC: i'm seeing a lot of status_get lines in the log -- are we caching appropriately?21:47
jeblairlooks like about every second21:47
jeblairoh, that's the default cache time.21:48
*** JasonCL has quit IRC21:51
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Git driver  https://review.openstack.org/52561422:09
*** hashar has joined #zuul22:10
*** ChanServ has quit IRC22:17
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Git driver  https://review.openstack.org/52561422:18
*** hashar has quit IRC22:18
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Git driver  https://review.openstack.org/52561422:19
*** ChanServ has joined #zuul22:24
*** barjavel.freenode.net sets mode: +o ChanServ22:24
*** ChanServ has quit IRC22:28
*** ChanServ has joined #zuul22:31
*** barjavel.freenode.net sets mode: +o ChanServ22:31
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Remove unused function validate_conf  https://review.openstack.org/52810522:33
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Remove unused function validate_conf  https://review.openstack.org/52810522:39
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: encrypt_secret: remove the trailing '/' when building url  https://review.openstack.org/50627122:40
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: encrypt_secret: remove the trailing '/' when building url  https://review.openstack.org/50627122:42
*** JasonCL has joined #zuul22:45
*** threestrands_ has joined #zuul23:10
*** threestrands has quit IRC23:13
*** weshay is now known as weshay_pto23:17
*** harlowja has quit IRC23:32
*** harlowja has joined #zuul23:39
*** harlowja has quit IRC23:40
*** jkilpatr has quit IRC23:40
*** harlowja has joined #zuul23:42
*** harlowja has quit IRC23:50
*** lennyb has quit IRC23:51

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!