Thursday, 2018-03-08

*** pwhalen has quit IRC01:48
*** pwhalen has joined #zuul01:50
*** pwhalen has quit IRC01:50
*** pwhalen has joined #zuul01:50
pabelangerShrews: one for the morning, apparently fingergw.listen_address = 127.0.0.1 won't work for zuul.conf, get the following traceback: http://paste.openstack.org/show/694290/02:20
pabelangerfingergw.listen_address = localhost does work02:21
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Rename get_schema to getSchema  https://review.openstack.org/55067002:44
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Extend config API with getSupportedLabels()  https://review.openstack.org/55067102:44
Shrewspabelanger: maybe that has something to do with whether ipv6 is being used or not02:45
Shrewsif your host is ipv6 enabled, you should probably configure with ipv6 addresses02:45
Shrewsclarkb: 550671 implements that config API02:46
*** harlowja has quit IRC02:49
openstackgerritMerged openstack-infra/nodepool master: Store label info with launcher registration  https://review.openstack.org/54837603:01
*** rlandy|bbl is now known as rlandy04:03
*** harlowja has joined #zuul04:37
*** jappleii__ has joined #zuul04:42
*** jappleii__ has quit IRC04:43
*** threestrands_ has quit IRC04:44
*** _ari_ has quit IRC04:57
*** weshay|PTO has quit IRC04:58
*** pabelanger has quit IRC04:58
*** myoung|off has quit IRC04:59
*** threestrands has joined #zuul05:00
*** threestrands has quit IRC05:00
*** threestrands has joined #zuul05:00
*** threestrands has quit IRC05:25
*** swest has joined #zuul05:51
*** snapiri has joined #zuul05:51
*** swest has quit IRC06:05
*** harlowja has quit IRC06:05
*** swest has joined #zuul06:05
*** swest has quit IRC06:08
*** swest has joined #zuul06:08
*** jaianshu has joined #zuul06:13
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add queue size to tenant overview  https://review.openstack.org/54824806:27
tobiashmordred: that works ^06:28
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Fix console stream colors  https://review.openstack.org/55073007:26
tobiashmordred: that fixes the streaming colors ^07:26
*** hashar has joined #zuul07:47
*** jaianshu_ has joined #zuul09:16
*** jaianshu has quit IRC09:19
*** weshay has joined #zuul09:40
*** _ari_ has joined #zuul09:40
*** pabelanger has joined #zuul09:41
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: configloader: check for unknown shadow projects  https://review.openstack.org/55076309:56
openstackgerritMerged openstack-infra/zuul-jobs master: Add known hosts for all users  https://review.openstack.org/54864210:05
*** jpena|off is now known as jpena10:05
*** dmellado has quit IRC10:08
*** dmellado has joined #zuul10:13
*** jpena is now known as jpena|off10:19
*** electrofelix has joined #zuul10:28
*** jaianshu__ has joined #zuul10:30
*** jaianshu_ has quit IRC10:34
*** pabelanger_ has joined #zuul12:27
*** _ari__ has joined #zuul12:28
*** weshay_ has joined #zuul12:29
*** pabelanger has quit IRC12:29
*** _ari_ has quit IRC12:30
*** weshay has quit IRC12:31
*** jaianshu__ has quit IRC12:35
*** pabelanger_ is now known as pabelanger13:10
pabelangerShrews: not sure about ipv6, need to test again. But I would still think 127.0.0.1 works for a loopback device, if you only wanted ipv4.  I've used localhost for now in my testing, but I can dive more into it13:12
*** mickt has joined #zuul13:18
*** rlandy has joined #zuul13:34
pabelangerhttp://paste.openstack.org/show/695224/13:52
pabelangeranother bug in our callback module for logging13:52
mordredpabelanger: if you hav ea sec, mind +Aing https://review.openstack.org/#/c/550730/ ?14:12
pabelangersure +314:12
*** snapiri has quit IRC14:16
mordredthanks!14:17
mrhillsmanany thoughts on collapsible pipelines?14:17
mrhillsmanzuul-web that is14:18
mordredmrhillsman: on the status page?14:19
mrhillsmanyeah14:20
mrhillsmanok if there is nothing in the works or if it has even been considered14:20
mrhillsmanjust wondering14:20
mordredseems like a good idea to me - although I think step one is refactor the status page to use angular instead of jquery so it matches the rest of the things (which hopefully should make it easier to hide the queue labels when filtering to a project too)14:21
mrhillsmanwe have quite a few right now and i can foresee it getting out of hand pretty quickly14:21
mrhillsmanok cool14:21
mordredmrhillsman: basically, I think now that the javascript stack has landed and we have previews on the build job, we're ina good position to do a bunch of refactoring/improvements14:21
openstackgerritMerged openstack-infra/nodepool master: Rename get_schema to getSchema  https://review.openstack.org/55067014:22
mordredmrhillsman: speaking of - don't know if you've seen or not, but if you click on build-javascript-content, like on https://review.openstack.org/#/c/548248 - it shows a live preview of the zuul web content14:22
mrhillsmanclickety click14:22
openstackgerritMerged openstack-infra/nodepool master: Extend config API with getSupportedLabels()  https://review.openstack.org/55067114:24
mrhillsmanso you know i think for this one team we are talking i think queue is pipeline for them14:24
electrofelixAnyone working on the nodepool plugin for Jenkins?14:27
pabelangerelectrofelix: There has been discussion in channel, but unsure if anybody is actively working on it14:28
pabelangerelectrofelix: even a recently ML post14:29
electrofelixyeah, I thought someone mentioned someone being interested in picking it up, just can't recall who14:29
electrofelixpabelanger: that was me, I was looking at the zuul trigger plugin14:29
pabelangerAh!14:29
electrofelixbut worked out should really help with the nodepool one first for us to migrate14:30
pabelangerelectrofelix: I mean, I know some downstream people that are looking to use nodepool with jenkins still, but don't believe they are on IRC ATM14:30
* Shrews deletes his ansible slack server from weechat forever14:33
electrofelixwell hopefully they'll see the messages today or I'll mentioned it again tomorrow (or if I can find the original discussion in eavesdrop).14:33
pabelangerelectrofelix: agree, it would be great to get interested parties together and work on it. I'll keep an eye out14:33
electrofelixah, finally found the discussion (thanks for channel logs)14:34
electrofelixhttp://eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2018-02-02.log.html14:34
electrofelixhughsaunders: ping me when your around14:35
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: configloader: check for unknown shadow projects  https://review.openstack.org/55076314:42
*** mickt has quit IRC14:46
electrofelixWould it make sense to host both zuul-trigger and nodepool plugins for jenkins on review.openstack.org?14:49
electrofelixsimilar to the gearman plugin14:49
openstackgerritMerged openstack-infra/zuul master: Normalize hostname for statsd  https://review.openstack.org/54730914:50
mordredelectrofelix: I don't see any reason why not - seems like it would enable making some integration tests too14:51
clarkbelectrofelix: reading the email I'm a little confused why you would use grpc instead of gearman since gearman is the existing zuul run a job protocol?14:52
clarkband jenkins has a gearman plugin that you could probably modify relatively easily to support the current gearman run me a job behavior?14:53
mordredclarkb: probably based on line 77 of https://etherpad.openstack.org/p/zuulv3-jenkins-integration14:54
electrofelixclarkb: I think the assumption was that didn't necessarily work well for providing access to the console output14:54
electrofelixbut I'm not attached to doing it that way14:55
mordredyah - gRPC having support for streaming channels should allow the trigger plugin to provide log streaming14:55
clarkbbut jenkins already does log strearming14:55
clarkbso you just point at the jenkins log streaming?14:55
clarkb(its a bit convoluted in that its reimplementing http partial gets using a post protcol they invented themselves but it works14:56
mordredclarkb: right - but I think the idea here is to keep the interface as zuul and have jenkins be an impl detail in the ansible playbooks14:56
clarkbmordred: ya zuul would get the jenkins log streaming14:56
clarkbusing the existing built in log streaming in jenkins14:56
clarkbso rather than finger to the executors its http post to jenkins14:56
mordredclarkb: sure. I mean - whatever works for people14:56
mordredalso - gearman would require the existence of a gearman server running that job content to talk to14:58
clarkbwhich zuul starts for you14:58
mordredyou do not want playbooks in jobs to be able to submit jobs to the zuul gearman14:59
clarkbI'm not sure I understand why playbooks would be involved?14:59
mordredthe zuul-side impl of this is a base job, not a zuul plugin14:59
clarkboohhhhh14:59
mordredzuul itself won't know anyhting about jenkins14:59
tobiashthat's neat14:59
mordredsorry - maybe should have started with that :)14:59
clarkbin that case I think I would still use gearman simply beacuse its part of the existing technoligy being used14:59
clarkband jenkins has a plugin for it15:00
mordredyah - except we're also working towards making it go away15:00
clarkbmordred: but also not replacing it with grpc as far as I know?15:00
mordred(and I'd like to add a gRPC api to zuul as well)15:00
mordredclarkb: no - not replacing it with grpc - and a grpc api may never materialize15:00
electrofelixin any case I think basically my initial assessment came down to, work on the nodepool plugin first, then look closer what's needed for the zuul trigger plugin15:01
mordredbut once it's not there, I think adding the requirement that someone run a gearman server in order to hook up zuul playbooks to a jenkins server is a weird requirement15:01
clarkbfwiw my concern here is having jammed 0mq into places in the past being a terrible decision15:01
clarkband I think that using grpc to use grpc feels similar especially when existing tooling already does the job15:01
mordredclarkb: I guess it's just that I donmt think existing tooling does the job - someone has to spin up a gearman server and run it15:01
mordredwhereas the grpc listener is supported well in javaand can just be a listener thread in jenkis itself15:02
clarkbmordred: ya I had missed that you weren't replacing the executors but running below them15:02
mordredit's not a use-it-just-to-use-it sort of thing. it could also be 'make a rest listerner'15:02
clarkbhonestly in that case I still don't think I would grpc I would just jenkins rest api to trigger job and then use the console streaming api15:02
mordredexcept jenkins frequently breaks their rest api :)15:03
clarkbso your ansible job would be trigger job, get back console streaming url, then while job is running: http post to get log and echo back through executor15:03
mordredclarkb: but yah - that would potentially work too - I think the thing is the job trigger api isn't going to be sufficient, because you need a trigger, then a wait callback, then a continue15:03
mordredso _something_ in jenkins is going to have to be listening for a something in th emiddle of an execution flow for a pause/continue15:04
electrofelixyou know some of that may mean a plugin in zuul for jenkins to avoid streaming the console unless someone is looking at it might be better for network usage15:04
clarkbmordred: so that you can cancel jobs?15:06
mordredclarkb: no - so that zuul can push the git repos to the nodes before jenkins actually runs the content15:06
mordredclarkb: I've got a flow written up on that etherpad15:06
clarkbthis is quickly beginning to seem like far more work than its worth >_> but I'm also not sure what the motiviation is for using jenkins and zuul v3 together15:08
mordredclarkb: migratoin15:08
clarkbso you can mostly keep your old job configs?15:08
mordredclarkb: for people currently running a large jenkins install, doing a big-bang migration is unlikely15:08
clarkbI guess that does help migrating15:08
mordredalso - some people have a ton of jobs they'd totally migrate, but one or two super compex pipelines they need to keep for $reasons15:09
mordredbasically - taking a page from ansible/puppet - ansiblenot requiring us to rework everything just to start using ansible was super useful - even though dehann originally didn't want the puppet module in ansible because he didn't want people to think ansible couldn't handle config management15:10
clarkbya, I think one difference here though is for ebtter or worse zuul has sort of been designed to not work the way jenkins works (due to trouble we've had with jenkins) so some of these interactions get tricky15:11
mordredyup. totally agree15:11
mordredit's not going to be a one-day task15:11
openstackgerritMerged openstack-infra/zuul master: Fix console stream colors  https://review.openstack.org/55073015:11
clarkbother than zuul going back to running a git repo server I can't think of any good alternatives to a pause/resume job interrupt functionality15:13
clarkband I think we've largely said no git repo server15:13
clarkbsuper cheap method to prove out a system quickly may be to have a job step that just polls a disk location and zuul can touch that when its done15:14
clarkbdoubt this would be reliable enough for long term use15:14
clarkbShrews: I didn't get a chance to review the get supported labels change before it merged but looking at it now it looks great. Thanks15:21
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Update jquery to version 3  https://review.openstack.org/55087115:29
*** electrofelix has quit IRC15:52
openstackgerritMerged openstack-infra/zuul master: Update jquery to version 3  https://review.openstack.org/55087116:11
*** harlowja has joined #zuul16:54
*** hashar is now known as hasharDinner17:13
*** harlowja has quit IRC18:26
mordredtristanC: I just left a comment on https://review.openstack.org/#/c/535545 - I'll help with it soon if you don't get to it first though18:38
*** sshnaidm|ptg has joined #zuul18:53
*** JasonCL has joined #zuul19:03
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/pipelines route  https://review.openstack.org/54152119:03
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/jobs.html page to display jobs  https://review.openstack.org/53554519:03
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/projects web page  https://review.openstack.org/53787019:03
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Fix indentation and renable the eslint rule  https://review.openstack.org/54567119:03
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/jobs/{job_name} route  https://review.openstack.org/55097819:03
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/projects routes  https://review.openstack.org/55097919:03
mordredtristanC: ^^ there - did the split19:03
openstackgerritMerged openstack-infra/zuul-jobs master: Parameterize the test-setup role  https://review.openstack.org/51090719:08
*** harlowja has joined #zuul19:11
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: DNM: dib test ca-certificates-java  https://review.openstack.org/55098219:12
*** harlowja has quit IRC19:18
mordredtobiash, clarkb, pabelanger: I think we're good to go on topic:javascript up through and including https://review.openstack.org/#/c/545671  - I know corvus wanted to discuss the jobs graph and pipeline rending changes more19:29
mordredbut if we can land through https://review.openstack.org/#/c/541521/ and get a zuul-web restart in place, it should let us look at https://review.openstack.org/#/c/535545/ https://review.openstack.org/#/c/537870/14 and https://review.openstack.org/#/c/545671/ - as well as the two final rendering patches19:31
clarkbI feel like I'm the wrong person to review anything called topic:javascript :)19:31
*** harlowja has joined #zuul19:33
tobiashclarkb: you could at least review the routes19:35
tobiashthey're python ;)19:36
*** _ari__ is now known as _ari_19:36
clarkbya I'll try to fumble my way through it this afternoon19:37
clarkbjs is simple enough that I mostly grok it, but it is also foreign enough that I'm like wat am I doing :)19:37
tobiashmordred: you probably should have named that stack python-and-a-little-bit-of-javascript :-P19:37
mordredtobiash: :)19:38
mordredI mostly want to get landed up through the "fix indentation" patch - because I hate having anything like a mass-indentation patch lurking around for too long19:39
*** elyezer has joined #zuul19:40
tobiashmordred: comment on https://review.openstack.org/#/c/550979/119:42
mordredtobiash: golly. I have no idea!19:44
* mordred goes to look19:44
tobiashmordred: +2 with comment on the commit message: https://review.openstack.org/#/c/537870/1419:51
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/projects routes  https://review.openstack.org/55097919:55
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/pipelines route  https://review.openstack.org/54152119:55
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/jobs.html page to display jobs  https://review.openstack.org/53554519:55
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/projects web page  https://review.openstack.org/53787019:55
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Fix indentation and renable the eslint rule  https://review.openstack.org/54567119:55
mordredtobiash: good call on 550979 ... and darn, I missedyour comment on the other before I pushed ...19:56
openstackgerritMonty Taylor proposed openstack-infra/zuul master: web: add /{tenant}/projects web page  https://review.openstack.org/53787019:57
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Fix indentation and renable the eslint rule  https://review.openstack.org/54567119:57
pabelangerShrews: yah, so it seems using an IPv4 fingergw.listen_address on ipv6 hosts, are an issue: http://logs.openstack.org/34/550834/5/check/windmill-ubuntu-xenial/aeef056/logs/zf01/var/log/journal/zuul-fingergw.service.log as exception19:58
pabelangeris there some sort of hostname lookup happening there?19:58
pabelangerzuul.conf FYI: http://logs.openstack.org/34/550834/5/check/windmill-ubuntu-xenial/aeef056/logs/zf01/etc/zuul/zuul.conf19:59
tobiashmordred: +2 on 55097920:00
mordred\o/20:01
tobiashmordred: so the stack is +2 from me20:02
mordredtobiash: sweet. thanks for the reviews!20:02
tobiashnp20:02
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: DNM: dib test ca-certificates-with-fix  https://review.openstack.org/55099020:05
Shrewspabelanger: no. we do set the address family to AF_INET6 for the server. not sure if we need to do something different to support both at once20:07
*** sshnaidm|ptg is now known as sshnaidm|afk20:08
pabelangerShrews: yah, let me see if I can figure it out now that I can reproduce the failure20:09
*** JasonCL has quit IRC20:10
Shrewspabelanger: did you try 0.0.0.0?20:10
*** JasonCL has joined #zuul20:10
Shrewsor ''20:10
*** JasonCL has quit IRC20:11
Shrewsi think v6 and v4 should be supported, but 127.0.0.1 might be "special"20:11
pabelangerShrews: yah, I can test. I think that worked, but I was looking to bind to a specific interface for testing reasons. 0.0.0.0 will bind to all ipv4 interfances20:13
pabelangerinterfaces*20:13
pabelangerShrews: https://bugs.python.org/issue2021520:14
pabelangerthat implies ipv6 isn't supported20:15
Shrewspabelanger: that's 4 years old20:15
Shrewsv6 is definitely supported20:15
Shrewspabelanger: https://review.openstack.org/50213720:16
*** JasonCL has joined #zuul20:16
Shrewspabelanger: that change was necessary b/c the tests failed on ipv6 hosts20:17
pabelangerah, I remember now20:17
*** rlandy has quit IRC20:19
*** elyezer has quit IRC20:20
pabelangerso, is the issue when you use AF_INET6, the address must be ipv6?20:20
pabelangerhttps://docs.python.org/3.6/library/socketserver.html#socketserver.BaseServer.server_address20:21
*** elyezer has joined #zuul20:21
pabelangerreads that address depends on the protocol family from socket20:21
*** JasonCL has quit IRC20:21
Shrewspabelanger: i don't think so. fairly sure if you gave the ipv4 public address, it would work, but not accept ipv6 connections. where as if you gave it an ipv6 address to listen to, it would accept connections from either20:26
Shrewsbut i'm actually not sure b/c i keep forgetting that stuff20:28
tobiashShrews: normally that's right, I think I remember some situations where this didn't work though20:34
Shrewspabelanger: alternative, if you're determined to use the loopback, try '::1' which i think is the v6 equivalent20:38
pabelangerI seem to remember in nodepool, we checked the family of the ipaddress to setup the socket in paramiko to work around some ipv4 / ipv6 issues20:40
pabelangerlet me see if I can find that20:40
pabelangerhttp://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/nodeutils.py?id=d0c25fc333270054699bb22b4793f659b1b7db4a#n5620:46
pabelangeris what I was thinking of20:47
pabelangerlet me try a quick test20:47
*** JasonCL has joined #zuul20:53
*** JasonCL_ has joined #zuul20:54
*** JasonCL has quit IRC20:55
*** JasonCL_ has quit IRC20:56
*** JasonCL has joined #zuul21:00
*** JasonCL has quit IRC21:03
*** JasonCL has joined #zuul21:03
*** JasonCL has quit IRC21:05
*** elyezer has quit IRC21:19
*** elyezer has joined #zuul21:24
openstackgerritPaul Belanger proposed openstack-infra/zuul master: DNM  https://review.openstack.org/55101521:36
*** JasonCL has joined #zuul21:41
*** threestrands has joined #zuul21:47
*** JasonCL has quit IRC21:52
*** rlandy has joined #zuul21:56
*** JasonCL has joined #zuul22:01
*** JasonCL has quit IRC22:02
openstackgerritPaul Belanger proposed openstack-infra/zuul master: DNM  https://review.openstack.org/55101522:08
openstackgerritPaul Belanger proposed openstack-infra/zuul master: DNM  https://review.openstack.org/55101522:23
*** JasonCL has joined #zuul22:35
*** JasonCL has quit IRC22:39
*** JasonCL has joined #zuul22:40
*** hasharDinner has quit IRC22:46
*** JasonCL has quit IRC22:52
pabelangerShrews: tobiash: okay, something like ^ seems to fix the socket exception from above.  I'll clean up the code and see if I can expose the issue in a unit test too22:57
pabelangerhttp://logs.openstack.org/17/551017/3/check/windmill-ubuntu-bionic/4a9d62a/logs/zf01/var/log/zuul/fingergw-debug.log22:57
pabelangerthat is on ovh, where it was failing22:57
*** elyezer has quit IRC22:58
*** elyezer has joined #zuul22:59
*** JasonCL has joined #zuul23:09
*** JasonCL has quit IRC23:10
*** elyezer has quit IRC23:17
*** JasonCL has joined #zuul23:18
*** JasonCL has quit IRC23:19
*** JasonCL has joined #zuul23:19
*** JasonCL has quit IRC23:22
*** elyezer has joined #zuul23:28
*** threestrands_ has joined #zuul23:33
*** threestrands_ has quit IRC23:33
*** threestrands_ has joined #zuul23:33
*** threestrands has quit IRC23:36

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!