| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Make roles ordered https://review.openstack.org/483140 | 00:34 |
|---|---|---|
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Run playbooks with only those roles defined thus far https://review.openstack.org/483141 | 00:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add child roles before parent https://review.openstack.org/483142 | 00:34 |
| jeblair | mordred, pabelanger, jlk, SpamapS: ^ | 00:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Run playbooks with only those roles defined thus far https://review.openstack.org/483141 | 00:39 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add child roles before parent https://review.openstack.org/483142 | 00:39 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: Fix status_url section in zuul.conf-sample https://review.openstack.org/483166 | 03:11 |
| *** dpawar has joined #zuul | 03:50 | |
| *** dpawar has quit IRC | 04:07 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: Fix status_url section in zuul.conf-sample https://review.openstack.org/483166 | 05:09 |
| *** dpawar has joined #zuul | 05:16 | |
| *** bhavik1 has joined #zuul | 05:17 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add spacing lines and formatting to result dicts https://review.openstack.org/483042 | 05:24 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add play recap back in to the end of plays https://review.openstack.org/483043 | 05:24 |
| *** dpawar has quit IRC | 05:29 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Log execution phase and include information https://review.openstack.org/483044 | 05:30 |
| *** dpawar has joined #zuul | 06:06 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: Add /etc/hosts and /etc/nsswitch.conf to the bubblewrap https://review.openstack.org/483210 | 06:10 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: Add /etc/hosts and /etc/nsswitch.conf to the bubblewrap https://review.openstack.org/483210 | 06:30 |
| *** dpawar has quit IRC | 06:47 | |
| *** dpawar has joined #zuul | 06:49 | |
| *** dpawar has left #zuul | 06:50 | |
| *** isaacb has joined #zuul | 06:52 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix KeysView object does not support indexing https://review.openstack.org/483231 | 07:06 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix dictionary change during iteration https://review.openstack.org/483232 | 07:06 |
| tobiash | mordred, jeblair: the zuul_stream patches from yesterday broke logging. ^^ are fixes for that | 07:06 |
| tobiash | mordred, jeblair: was it intended to dump the whole result dict for every task? | 07:09 |
| tobiash | mordred, jeblair: in case yes we should think about also filtering stdout_lines (the synchronize task now produces a huge amount of logs) | 07:10 |
| *** isaacb_ has joined #zuul | 07:24 | |
| *** isaacb has quit IRC | 07:26 | |
| *** isaacb_ has quit IRC | 07:29 | |
| tobiash | cool, just noticed that executor restart doesn't break running jobs but just reexecutes them :) | 07:34 |
| *** isaacb_ has joined #zuul | 07:35 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Document start day of week https://review.openstack.org/483279 | 09:26 |
| *** bhavik1 has quit IRC | 10:39 | |
| *** jkilpatr has quit IRC | 11:01 | |
| *** jkilpatr has joined #zuul | 11:39 | |
| mordred | tobiash: thanks! I went ahead and just aproved those | 11:42 |
| mordred | tobiash: and no - the intent was just to better format the dicts that were being printed at the end of the line already - let me take another pass and see about filtering | 11:43 |
| mordred | oh - you know what - I should put dump-results back in ... one sec | 11:43 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix KeysView object does not support indexing https://review.openstack.org/483231 | 11:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Use _dump_results to filter the results https://review.openstack.org/483311 | 11:46 |
| mordred | tobiash: ^^ | 11:46 |
| mordred | tobiash: also - wow on the monday bit - I love it when people reimplement cron except different | 11:47 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix dictionary change during iteration https://review.openstack.org/483232 | 11:50 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Use package module to install bindep packages https://review.openstack.org/482584 | 11:58 |
| mordred | tobiash: although- you know - there's a part of me that's tempted to stop outputting the results dicts and let people add debug statements | 12:13 |
| mordred | oh - actually - I _think_ I may have just had an idea about how to expose a flag to allow debug to be toggled by the end user | 12:14 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Use _dump_results to filter the results https://review.openstack.org/483311 | 12:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Emit errors from ansible more cleanly https://review.openstack.org/483335 | 12:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Strip zuul workdir from the play banner line https://review.openstack.org/483336 | 12:44 |
| mordred | ok. there's some more cleanups to that stack | 12:44 |
| tobiash | mordred: I'll just try out that stuff... :) | 12:51 |
| mordred | tobiash: awesome - thanks! I'm trying to find the right balacing point between providing enough information and not spamming the logs too terribly | 12:55 |
| tobiash | mordred: dumping the results dict was that spammy that I patched it out temporarily ;) | 12:57 |
| mordred | tobiash: ++ | 12:57 |
| tobiash | mordred: is that the tip of your stack? https://review.openstack.org/#/c/483336/1 | 12:58 |
| mordred | tobiash: yes | 12:58 |
| tobiash | mordred: k, trying out in a few minutes | 12:58 |
| mordred | tobiash: wiht that you should get readable ansible errors, less spammy results dict (except for stat calls which are still ugly to me) and the PLAY lines should bea little shorter | 12:59 |
| tobiash | mordred: I also thought of adding new lines before tasks instead of after | 12:59 |
| mordred | tobiash: oh yeah? that was actually how the first versoin of the patch worked - could certainly make that change | 13:01 |
| tobiash | mordred: lets first check how the logs look like with your stack | 13:01 |
| *** deep-book-gk_ has joined #zuul | 13:01 | |
| mordred | yes! | 13:01 |
| tobiash | mordred: something is broken in your stack: http://paste.openstack.org/show/615261/ | 13:03 |
| * tobiash is checking | 13:03 | |
| *** deep-book-gk_ has left #zuul | 13:04 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Use package module to install bindep packages https://review.openstack.org/482584 | 13:07 |
| mordred | tobiash: oh! we don't set work_dir | 13:08 |
| mordred | tobiash: sorry - I had a bad local test | 13:08 |
| tobiash | mordred: the last patch of the stack was broken | 13:15 |
| tobiash | mordred: build log without the last patch: http://paste.openstack.org/show/615264/ | 13:15 |
| tobiash | mordred: (notice the too much data from the synchronize task) | 13:15 |
| tobiash | mordred: there it prints *every* file of the repos pushed to the node | 13:16 |
| *** dkranz has joined #zuul | 13:18 | |
| pabelanger | morning | 13:19 |
| tobiash | mordred: regarding the second patch (for e.g. ansible parse error handling): http://paste.openstack.org/show/615265/ | 13:24 |
| tobiash | mordred: could be decoded before so we don't have b'...' notation in the log | 13:24 |
| mordred | tobiash: cool - I'll work on fixing both of those things | 13:25 |
| tobiash | mordred: also attached example logs to these changes for reference | 13:28 |
| mordred | tobiash: aha! I think I see the issue | 13:41 |
| * mordred was wondering why we didn't have this issue before | 13:41 | |
| *** jkilpatr has quit IRC | 13:42 | |
| *** jkilpatr has joined #zuul | 13:47 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Emit errors from ansible more cleanly https://review.openstack.org/483335 | 14:15 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Strip zuul work dir from the play banner line https://review.openstack.org/483336 | 14:15 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Filter and print results more comprehensively https://review.openstack.org/483311 | 14:15 |
| mordred | tobiash: ok. that stack ^^ should be much friendlier (and thanks for the great example cases!) | 14:15 |
| mordred | tobiash: there is one thing though - the synchronize command returns a "msg" in its return value that shows the normal rsync output - I don't think systemically supressing that is the right thing -it would confuse folks - but I fixed the patches to obey no_log like they're supposed to | 14:16 |
| mordred | tobiash: in our base job we marked the synchronize invocations with no_log for that reason - if you do - it should hopefully be the amount of data you'd like :) | 14:16 |
| mordred | tobiash: http://paste.openstack.org/show/615274/ here's some sample output with both no_log and without | 14:21 |
| tobiash | mordred: trying... | 14:51 |
| * mordred crosses fingers | 14:52 | |
| *** amoralej|off is now known as amoralej | 14:55 | |
| tobiash | mordred: http://paste.openstack.org/show/615279/ | 14:59 |
| tobiash | mordred: example log including ansible parse error (still with b'...') | 14:59 |
| tobiash | mordred: but this time without errors in the debug log :) | 15:00 |
| tobiash | mordred: I personally would remove the empty line between task start and the log of the task | 15:01 |
| tobiash | mordred: and would add an empty line after the play recap | 15:01 |
| tobiash | mordred: or before PLAY, don't know what's better | 15:02 |
| jeblair | mordred: regarding stripping the path from the playbook -- i like that up to a point, but once we're inside the jobdir, there are multiple possible paths to the playbooks (they are not always run from playbook_0). would just stripping them down to the jobdir root be okay? | 15:02 |
| jeblair | mordred: also, i change those paths in https://review.openstack.org/483141 and it would be nice to not have to figure out all that manual path munging again. | 15:05 |
| mordred | jeblair: how about we just add a variable that the executor passes in like "canonical_playbook_name" and skip stripping at all? | 15:06 |
| *** isaacb_ has quit IRC | 15:06 | |
| jeblair | mordred: incidentally, after 141 lands, the paths, if stripped down to the jobdir, would look like "trusted/project_0/git.openstack.org/zuul-jobs/playbooks/foo" and "work/src/git.openstack.org/zuul/playbooks/bar" | 15:06 |
| jeblair | mordred: that works. we can even stick the branch on there too (@master). can we also add the 'trusted/untrusted' bit of information? i would find that useful in logs. | 15:07 |
| mordred | jeblair: yah - I think trusted/project_0/ and work/src are interesting to us but not to consumer users - I think git.openstack.org/zuul/playbooks/bar is the 'most' important part - although it's also important to indicate in some manner "playbook from git ref {ref}" if it was trusted and therefore a "real" git ref - or "playbook from speculative git ref" if it was not a playbook ina published state | 15:08 |
| mordred | jeblair: yes - I think trusted/untrusted is great info | 15:08 |
| mordred | jeblair: so maybe lets add a playbook_info section to the executor vars, and then we'll have it to be able to emit things into the logs | 15:09 |
| mordred | tobiash: agree re: blank line locations | 15:09 |
| jeblair | mordred: sounds good. let's build that on top of 483141 because lots of stuff changes in there. | 15:10 |
| mordred | jeblair: ++ | 15:21 |
| *** jkilpatr has quit IRC | 15:23 | |
| tobiash | just curious, do all of you use (only?) gertty or are you also using the gerrit web ui? | 15:26 |
| jeblair | i use only gertty | 15:39 |
| pabelanger | jeblair: do you have magic shell that will open review.o.o URLs into gerrty, or is that a manual process for you | 15:40 |
| jeblair | pabelanger: i use this: http://gertty.readthedocs.io/en/master/usage.html#terminal-integration | 15:41 |
| pabelanger | jeblair: Thanks, setting it up now | 15:42 |
| *** bhavik1 has joined #zuul | 15:45 | |
| *** bhavik1 has quit IRC | 16:18 | |
| *** jamielennox has quit IRC | 16:19 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Add html based websocket client for console stream https://review.openstack.org/481599 | 16:20 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Add html based websocket client for console stream https://review.openstack.org/481599 | 17:04 |
| Shrews | mordred: so, that ^^^ is what serving up tobiash's example html looks like from a zuul-web route. I'm not in love with it. | 17:05 |
| Shrews | oh wait. i just got another idea to try | 17:07 |
| jlk | o/ | 17:09 |
| *** jkilpatr has joined #zuul | 17:18 | |
| Shrews | nope, that idea didn't pan out | 17:22 |
| *** jkilpatr has quit IRC | 17:26 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Add html based websocket client for console stream https://review.openstack.org/481599 | 17:26 |
| *** harlowja has quit IRC | 17:27 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Add html based websocket client for console stream https://review.openstack.org/481599 | 17:27 |
| jeblair | Shrews: what don't you love? | 17:34 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Document start day of week https://review.openstack.org/483279 | 17:36 |
| Shrews | jeblair: i was trying to find a way to avoid inlining the html by redirecting, but i couldn't find any mechanism in place to do so while passing along the query parameters. Also, parameterizing the inline HTML (for uuid and logfile) meant doubling up the { and } characters | 17:36 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix status_url section in zuul.conf-sample https://review.openstack.org/483166 | 17:39 |
| mordred | Shrews: ah... I see - mind if I poke at it for a sec? I have a thought but I'm not sure how to effectively express it other than just writing a few lines in code | 17:39 |
| Shrews | mordred: plz do | 17:39 |
| Shrews | i'm actually not even convinced that works with the contained javascript | 17:41 |
| Shrews | i'm betting it doesn't, actually, now that i think about it | 17:41 |
| * Shrews *shrugs* ... need to find a way to actually test it with live jobs | 17:43 | |
| openstackgerrit | Ben Kero proposed openstack-infra/nodepool master: Correctly source JenkinsException https://review.openstack.org/483480 | 17:44 |
| jlk | getting to the point of live jobs is harder. | 17:45 |
| jlk | since you do actually need a nodepool with nodes on it | 17:45 |
| jlk | until we land the code that explicitly lets you run jobs entirely on the executor | 17:45 |
| Shrews | jlk: i was thinking of just asking tobiash to try it since he's on our bleeding edge anyway :) | 17:46 |
| jlk | there ya go | 17:46 |
| * jlk wonders how far Jamie got last night | 17:46 | |
| tobiash | Shrews: will try this out tomorrow | 17:49 |
| Shrews | mordred: assuming this doesn't work, i wonder if we should try ditching the example html and try aiohttp's streaming api? http://aiohttp.readthedocs.io/en/stable/streams.html | 17:49 |
| tobiash | Shrews: I probably have to patch this slightly as my zuul-web doesn't run in / | 17:50 |
| mordred | Shrews, tobiash: so - my thoughts are that we can serve the file without templating and just pass parameters in url ... | 17:50 |
| mordred | but I wanna poke at it for just a few seconds to make sure I'm not smoking something | 17:51 |
| tobiash | mordred, Shrews: that would mean we also need to pass eventually the root url | 17:52 |
| tobiash | getting uuid and filename worked pretty well via js | 17:52 |
| Shrews | tobiash: everything served by zuul-web should share the same root. not sure i follow? | 17:53 |
| mordred | Shrews: but that root can be rooted below / | 17:53 |
| mordred | Shrews: so it could be zuul.example.com/web/console-stream | 17:53 |
| mordred | and zuul.example.com/web/static/stream | 17:54 |
| openstackgerrit | Ben Kero proposed openstack-infra/nodepool master: Add SSH Host Key Verifier Strategy https://review.openstack.org/483485 | 17:54 |
| tobiash | Shrews: I saw an absolute path in the html for opening the socket: var ws = new WebSocket('/console-stream'); | 17:55 |
| tobiash | that wouldn't work well if zuul runs under /zuul | 17:55 |
| Shrews | tobiash: oh, well i don't know js very well. | 17:56 |
| tobiash | Shrews: in this variant (line 48 following) the parameters are gathered via js and not baked in statically | 17:58 |
| tobiash | Shrews: btw, I'm a js noob, so no guarantee for anything in that ;) | 17:59 |
| Shrews | well the problem with aiohttp handling the request is that i could not find a way to pass the query params through to your html (the thing i think mordred is trying), so it never got the params. that's why i baked them in and used format() to set them | 18:00 |
| Shrews | So I'm either thinking about this *totally* incorrectly (very possible since I'm not well versed in web programming), or this might best be left to a real web server | 18:01 |
| tobiash | Shrews: I think you don't have to hand out the query parameters to the browser as the browser already knows them | 18:01 |
| tobiash | Shrews: so when handling the uuid and filename in js you could completely ignore them in zuul-web | 18:03 |
| Shrews | the thing handling /static/stream knows them, yes. but simply returning your HTML as it was causes them to be lost. | 18:03 |
| Shrews | (i know i'm not explaining this clearly) | 18:04 |
| * Shrews attempts to think of better words | 18:04 | |
| tobiash | Shrews: hm, that's surprising | 18:05 |
| tobiash | Shrews: I thought that this should work as e.g. I thought apache also doesn't handle the query parameters | 18:06 |
| tobiash | Shrews: but I'm no web expert either | 18:06 |
| *** dmsimard is now known as dmsimard|afk | 18:08 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Run playbooks with only those roles defined thus far https://review.openstack.org/483141 | 18:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add child roles before parent https://review.openstack.org/483142 | 18:10 |
| Shrews | tobiash: mordred: oh! i just proved myself wrong. | 18:11 |
| Shrews | oh, no i didn't | 18:15 |
| Shrews | lol | 18:15 |
| Shrews | format() tricked me | 18:16 |
| *** jkilpatr has joined #zuul | 18:16 | |
| * Shrews steps away for some much needed tea | 18:16 | |
| jlk | jeblair: haha, i think we're both looking at the same story/task | 18:25 |
| jlk | jeblair: project pipeline addition via in-config speculative change | 18:26 |
| jeblair | jlk: i don't think i'm looking at that? | 18:26 |
| jlk | erm, I need to learn how to read timelines. | 18:27 |
| jlk | you created it, back in march. My mistake. | 18:27 |
| jeblair | whew :) | 18:27 |
| jlk | don't know why I'm so used to newest at the top oldest at the bottom. | 18:28 |
| jlk | but that's not how storyboard works | 18:28 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Serve html statically https://review.openstack.org/483503 | 18:31 |
| mordred | Shrews: so- the thing I meant is that the link we display in status.json would be something like /stream.html?uuid=uuid - so the user would click that link and it would take them to the page which would have uuid in its query params ... somthing like that ^^ | 18:31 |
| mordred | Shrews: but tea is much more important :) | 18:31 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Run playbooks with only those roles defined thus far https://review.openstack.org/483141 | 18:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add child roles before parent https://review.openstack.org/483142 | 18:33 |
| Shrews | mordred: yup, and that's how it works now | 18:33 |
| jeblair | SpamapS: leaving https://review.openstack.org/483210 for your +3 | 18:35 |
| Shrews | well, i mean, status.json doesn't do that yet, but the /static/stream handler is there | 18:35 |
| mordred | Shrews: right- but in the code you put up we're processing the url query params in python then doing a server-side template substitution on the html to generate a dynamic page. by pulling them out of the query string in the javascript we can avoid needing to process he file server side - which means we can add an apache rewrite rule for more complex installs that just serves the static files directly bypassing | 18:36 |
| mordred | zuul-web for the static routes | 18:36 |
| *** jamielennox has joined #zuul | 18:37 | |
| Shrews | mordred: so... how do i return the HTML so that things are handled client side? | 18:37 |
| mordred | Shrews: I has pushed up patch at https://review.openstack.org/483503 | 18:37 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Support relative urls in success-url https://review.openstack.org/481342 | 18:38 |
| mordred | Shrews: there's still a few gotchas we need to solve - namely -- oh, wait a sec - I may be overthinking part of something ... | 18:38 |
| mordred | tobiash: you run things on a suburl - do you serve status.json and console-stream from the same sub-url? if you don't, how difficult would it be for you to do so? | 18:39 |
| tobiash | mordred: nope | 18:39 |
| pabelanger | mordred: jeblair: are we okay to restart zuulv3 to pickup new zuul_stream changes? | 18:39 |
| tobiash | mordred: I unfortunately have a crazy setup in this regard | 18:39 |
| mordred | tobiash: I ask because _eventually_ we should have zuul-web serve status and console-stream (and all the things) so the intent isn't for them to be differenturls long-term | 18:39 |
| mordred | tobiash: in the future if zuul-web is the one serving all the http traffic, will that be a problem for you? (are they in different places right now because scheduler and zuul-web are different processes?) | 18:40 |
| tobiash | mordred: I have a main url where I server everything, but that runs via a proxy server breaking websockets (and I don't have crontrol of) | 18:41 |
| tobiash | mordred: thus I have the websocket running via a different domain | 18:41 |
| mordred | boo proxy servers that break things! | 18:41 |
| mordred | tobiash: gotcha. so for you at the very least having a place to be able to configure a specific location for the websocket stream is important | 18:41 |
| tobiash | mordred: yes | 18:42 |
| mordred | tobiash: all other non-websocket content are in the same place? so static/stream and status.json are adjacent? | 18:42 |
| tobiash | I have everyting running via e.g. ci.example.com (with this crazy proxy server and authentication stuff and reachable from outside) | 18:42 |
| tobiash | but need to server websockets via internalci.example.intra (which I have under control) | 18:43 |
| tobiash | mordred: I'm aware that this is a crazy setup and limiting streaming to internal users | 18:43 |
| mordred | ok. good to know - I think we've reached the point where I need to write up a quick spec and collection of inputs | 18:43 |
| tobiash | mordred: but it will take me a year or so to change that ;) | 18:44 |
| mordred | tobiash: well, I think you and your users are important andI think we can handle it :) | 18:44 |
| tobiash | :) | 18:44 |
| tobiash | mordred: I have no problem with zuul-web serving all as both endpoints proxy to the same zuul-web in the end, but just need to be able to configure the absolute streaming root url separately | 18:46 |
| mordred | tobiash: cool. I have a hunch you may not be the only one who winds up in that situation | 18:47 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Use package module to install bindep packages https://review.openstack.org/482584 | 18:50 |
| Shrews | mordred: yeah, so... that suffers from the same problem | 18:51 |
| pabelanger | mordred: jeblair: do you think we are ready to create a post pipeline for zuulv3.o.o and work on publishing jobs (eg: docs)? | 18:52 |
| Shrews | mordred: the js is not getting executed. I can modify it to set the pagecontent, but nothing happens | 18:52 |
| Shrews | mordred: i think add_static() is meant more for things like images?? | 18:52 |
| mordred | Shrews: kk. lemme poke at it a little more - it's triggered my ocd brainhole | 18:54 |
| *** harlowja has joined #zuul | 18:54 | |
| Shrews | mordred: hrm, might be a js error... lemme try something | 18:55 |
| Shrews | chrome dev tools are actually useful | 18:56 |
| jlk | jeblair: related to earlier conversation, I believe the task https://storyboard.openstack.org/?#!/story/2000898 is already complete, by reading an existing test. Can you validate my findings? If so I'll close the task/story and we'll have one less thing to do for v3 :D | 18:56 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Support relative urls in success-url https://review.openstack.org/481342 | 18:58 |
| pabelanger | mordred: jeblair: we'd create the afs publisher for that, which would need secrets enabled for said pipeline. So, maybe not post pipeline, but pipe-secrets pipeline? | 18:58 |
| tobiash | jlk: I had that use case lately, I was able to add a project to a check pipeline via specilative in-repo config, but not to a gate pipeline | 19:00 |
| jlk | so an independent, but not a dependent? | 19:00 |
| * jlk alters the test a bit to give that a shot | 19:00 | |
| tobiash | jlk: didn't dig deeper, maybe independent vs dependent or maybe submit or not submit | 19:01 |
| jlk | not sure what submit means here | 19:01 |
| tobiash | jlk: I meant including submit in the reporter, but I also guess it's more likely the dependent pipeline which is not triggered | 19:03 |
| tobiash | jlk: possibly to a not yet existing change queue | 19:03 |
| tobiash | (but just guessing) | 19:03 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Serve html statically https://review.openstack.org/483503 | 19:06 |
| Shrews | mordred: ok, so that ^^ actually works, short of the websocket url not being valid | 19:07 |
| jeblair | pabelanger: can we get the unit test jobs finished first? | 19:18 |
| jlk | tobiash: well in the test, I was able to get the gate to try to trigger the job, but then it didn't trigger because the change could not be merged, for whatever reason. Not sure yet why. | 19:19 |
| tobiash | jlk: interesting | 19:22 |
| jeblair | jlk, tobiash: yeah, i'm leaning toward thinking we have accidentally solved this for independent pipelines. | 19:25 |
| jeblair | and that test does cover that case | 19:25 |
| jlk | oooh, things don't work well when you vim a file in the fixtures git repo, and background the vim process. | 19:26 |
| jlk | gotta close it | 19:26 |
| jeblair | heh | 19:26 |
| tobiash | jlk: hm, sounds somewhat strange as independent pipelines also merge a change | 19:26 |
| jlk | we should probably avoid bringing in swap files | 19:26 |
| * tobiash has EOD and empty battery now | 19:28 | |
| jeblair | jlk, tobiash: i think the issue we're going to have with dependent pipelines is that they have static shared change queues. so because the project hasn't been added to the pipeline yet, there's no change queue for it, so there's nothing to add it to. | 19:28 |
| jeblair | (independent pipelines create a new change queue for every change) | 19:29 |
| jlk | maybe. First I have to track down why zuul thinks this change cannot be merged | 19:29 |
| tobiash | jeblair: that's also what I was guessing | 19:29 |
| jeblair | jlk: it'll need the appropriate votes | 19:30 |
| jlk | ahhhh right, derp. | 19:30 |
| jlk | because gerrit. | 19:30 |
| jeblair | A.addApproval('code-review', 2) | 19:31 |
| jeblair | self.fake_gerrit.addEvent(A.addApproval('approved', 1)) | 19:31 |
| *** amoralej is now known as amoralej|off | 19:31 | |
| pabelanger | jeblair: sure, feels like I have stale on some new jobs and favor of refactoring ansible playbooks (which I understand is important for other zuul systems) | 19:31 |
| jeblair | pabelanger: they are important for openstack too. we're not going into production with a bunch of shell scripts | 19:32 |
| jeblair | let's get our unit test jobs running from start to finish the way we like. then the publishing jobs will be easy to knock out. | 19:34 |
| pabelanger | k | 19:34 |
| jeblair | (and honestly, we're generating enough of a punchlist with just these at the moment that we don't need to go look for new problems yet :) | 19:34 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Remove run-cover role https://review.openstack.org/483527 | 19:37 |
| jlk | okay | 19:38 |
| pabelanger | mordred: ^ I don't think we use that any place any more | 19:38 |
| jlk | 2017-07-13 19:37:41,068 zuul.DependentPipelineManager DEBUG Considering adding change <Change 0x7f9177c60080 1,1> | 19:38 |
| jlk | 2017-07-13 19:37:41,068 zuul.DependentPipelineManager DEBUG Unable to find change queue for change <Change 0x7f9177c60080 1,1> in project org/project | 19:38 |
| jlk | You're right, this is where it's failing, there is no queue for this. | 19:39 |
| jeblair | jlk: i don't have an immediate answer for that. the words "special case" come to mind. | 19:42 |
| jeblair | jlk: and there are at least two variations of this | 19:42 |
| jeblair | 1) add a project to a pipeline in its own change queue | 19:42 |
| jlk | yeah, either we need to add a dynamic queue for this one shot, or we need to add the project to the pipeline or... | 19:42 |
| jeblair | 2) add a project to a pipeline directly into an already existing change queue with another project | 19:43 |
| jlk | nod | 19:43 |
| jeblair | 3) project is already in pipeline in its own change queue and the change merges it into another | 19:43 |
| jeblair | 4) project leaves a change queue it's a part of | 19:43 |
| jlk | that's an ugly set of possibilities. :( | 19:43 |
| jeblair | and yeah, it's *at least* two. :) | 19:44 |
| jlk | so.. do we want to support this on dependent pipelines, or not? | 19:44 |
| jeblair | 5) project leaves pipeline. this one seems easy. :) | 19:44 |
| jlk | (project leaves pipeline, should that dequeue any existing change that would be in that pipeline?) | 19:45 |
| jeblair | jlk: i think the biggest difficulty is that without some kind of support for this, a project could never add itself to a gating system. we'd have to have bootstrap entries in a config repo. | 19:46 |
| jeblair | jlk: it's worth noting that we do correctly handle all cases of updating the queues once a change lands. so if a change removes itself, once it lands, all the others will be removed too. | 19:46 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Serve html statically https://review.openstack.org/483503 | 19:47 |
| jlk | okay, so don't need to worry about that part. | 19:47 |
| jeblair | jlk: so one solution to this would be to try to run the re-enqueue process on changes like this. | 19:47 |
| jlk | a project couldn't speculatively add itself to the dependent pipeline. It'd have to get merged, then used upon next change (or any existing changes) | 19:48 |
| mordred | pabelanger: ah - my bad - actually I think we want to go the other way | 19:49 |
| jeblair | jlk: another solution might be to just handle the special case of change adding itself to a dependent pipeline. don't worry about shared change queues, etc. | 19:49 |
| jlk | could be a good first step at least. | 19:50 |
| jeblair | jlk: so for that one special case, dynamically create a non-shared change queue just for that change. | 19:50 |
| jlk | nod | 19:50 |
| mordred | pabelanger: so - the thing I did last week was to suck in the content of the various jenkins slave scripts into shell sections of roles - so that we could have self-contained patches where we refactor away from calling those scripts to just having it be in the roles | 19:50 |
| mordred | pabelanger: (so that we don't have any more of this 'script baked in to image' stuff) | 19:50 |
| mordred | pabelanger: I believe that what we want there is actually to update tox-cover to use run-cover instead of just setting tox_environmnet: cover ... and then we want to hack on run-cover until it's not just a script blob | 19:51 |
| jeblair | jlk: i think i'd start with that. i *think* it's tractable (though still not easy), and would file off this really rough edge. all the other edge cases (you won't share a queue until the change lands, etc) are all things we can live with as documentation footnotes. :) | 19:51 |
| mordred | pabelanger: same with run-tarball and run-wheel - although we probably want to think about those, becuase those are making tarballs and wheels in a very openstack specific way, rather than in a general "make a tarball for a python project" kind of way | 19:52 |
| jlk | yeah, I updated the story and will try throwing some code at the wall after lunch. | 19:52 |
| jeblair | jlk: cool. dig into the two different kinds of change queue context managers that the dependent and independent pipelines use | 19:52 |
| jlk | will do | 19:53 |
| mordred | pabelanger: it's worth us thinking about whether the thing we're doing there is actually needed - what a shared base job for python tarballs and python wheels would look like and whether we want/need an openstack specific version of those things too | 19:53 |
| mordred | pabelanger: (in short, we have much bikeshedding to do!) | 19:53 |
| pabelanger | mordred: right, the main difference with run-cover role is some NOSE and upper contraints today | 19:53 |
| mordred | Shrews, tobiash: https://review.openstack.org/483503 I think that should provide some options to deal with tobias' proxy/split-host situation, and also to plumb through info to the executor so that links have hte right info | 19:54 |
| pabelanger | then, pip freeze stuff | 19:54 |
| jeblair | jlk: there's a remote possibility you could actually just have the dependent pipeline manager generate a dynamic change queue in this special case and just have it work. maybe? <shrug> | 19:54 |
| mordred | pabelanger: yah. the upper-constraints part is obviously very openstack specific | 19:54 |
| jlk | it kind of looks that way, that was going to be my first attempt | 19:54 |
| jeblair | jlk: and i will lunch now too :) | 19:54 |
| pabelanger | mordred: let me quickly stab at upper-contraints.txt, I think we can do that pretty easy from openstack-zuul-jobs | 19:55 |
| pabelanger | mordred: just need a project that actually uses it :D | 19:55 |
| mordred | pabelanger: yah- maybe we can have an openstack-cover in openstack-zuul-jobs that has tox-cover as a parent perhaps? | 19:56 |
| mordred | pabelanger: haha | 19:56 |
| mordred | pabelanger: maybe add it to zuul for now so we can watch it | 19:56 |
| pabelanger | mordred: right, openstack-cover is what I was just about to create | 19:57 |
| mordred | pabelanger: ++ | 19:57 |
| Shrews | mordred: i think you overwrote my fixes | 19:57 |
| mordred | pabelanger: we're at a super fun and existential place with a few of these where thinking about some of them in an openstack vs. world sense is needed | 19:58 |
| mordred | Shrews: oh - sorry - I did not see that you'd made them - one sec, lemme fix | 19:58 |
| Shrews | mordred: i think you might just need to s/STATIC_PATH/STATIC_DIR/ | 19:59 |
| mordred | yah- that's what the diff looks like | 19:59 |
| pabelanger | mordred: ++ | 19:59 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Serve html statically https://review.openstack.org/483503 | 19:59 |
| *** dmsimard|afk is now known as dmsimard | 20:00 | |
| mordred | Shrews: I think I want to remove one of the config settings I added in that patch - I thnk it's actually a bad idea | 20:04 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Serve html statically https://review.openstack.org/483503 | 20:09 |
| mordred | Shrews: k- that cleans out the bits I added that were really overthinking things | 20:09 |
| mordred | tobiash: I'm assuming you are not having any issues with cross-domain things with your different websocket location? | 20:12 |
| mordred | tobiash: (wondering if we need to add CORS support) | 20:12 |
| mordred | tobiash: but if you can serve static.html from ci.example.com and have it connect to the websocket at internalci.example.intra - I think that's the only example we have so far of possibly needing CORS support (so far, I have one in mind but we can get to it later) - so if you don't have that problem we can defer actually implementing it | 20:14 |
| mordred | jlk: how strong is your callback-fu? | 20:18 |
| *** harlowja has quit IRC | 20:19 | |
| *** timolo has joined #zuul | 20:20 | |
| *** harlowja has joined #zuul | 20:22 | |
| *** timolo has quit IRC | 20:32 | |
| jeblair | pabelanger: did you restart zv3.o.o? | 20:38 |
| pabelanger | jeblair: I have not | 20:40 |
| jeblair | pabelanger: cool, i'll do it now | 20:40 |
| pabelanger | ++ | 20:40 |
| jeblair | done | 20:42 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: DNM - add upper-constraints to tox https://review.openstack.org/483548 | 20:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: DNM - add upper-constraints to tox https://review.openstack.org/483548 | 20:43 |
| jeblair | mordred, jlk, pabelanger: 483140, 483141, 483142 are all ready now whenever you have a moment. now that all the stuff blocking the base job move has merged, i'm going to start work on moving it. | 20:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add documentation for project shadowing https://review.openstack.org/483552 | 20:52 |
| pabelanger | mordred: thanks, job-output.txt looks much nicer now: http://logs.openstack.org/48/483548/2/check/e2d9f45/job-output.txt | 21:01 |
| mordred | pabelanger: WOOT! | 21:01 |
| mordred | pabelanger: indeed - I think we need to add no_log: true to our synchronize call | 21:01 |
| mordred | pabelanger: so that we don't get that giant amount of output | 21:02 |
| mordred | pabelanger: oh - actually- I have three more changes that fix all of that | 21:02 |
| pabelanger | mordred: however, I think I see an issue. no_log: true was outputted | 21:02 |
| pabelanger | ya | 21:02 |
| pabelanger | it is added, but no longer used | 21:02 |
| mordred | pabelanger: yup. we need to get the next three patches in | 21:04 |
| pabelanger | k, looking now | 21:04 |
| pabelanger | mordred: actually, it would be helpfully if no_log: true did output if the task failed | 21:04 |
| pabelanger | so we can see what the reason for failure was | 21:04 |
| mordred | pabelanger: https://review.openstack.org/#/c/483311/3 | 21:04 |
| mordred | pabelanger: we can't do that - it's a security feature | 21:05 |
| mordred | pabelanger: however - I have thoughts - one sec, phone call | 21:05 |
| pabelanger | mordred: Ya, I was thinking maybe submitting a patch to ansible for no_log: on_failure | 21:05 |
| pabelanger | or something like that | 21:05 |
| mordred | pabelanger: I ahve a whole thing planned I'll tell you about in just a sec | 21:05 |
| pabelanger | ack | 21:06 |
| jlk | mordred: not super strong, but I can hum a few bars. | 21:11 |
| jeblair | mordred: i'm looking at moving the base job over. what's your current thinking on validate-host? a) do you want to move it into zuul-jobs with the base job along with the openstack-specific stuff and have it broken until we implement site-local vars. b) put a base job in zuul-jobs without validate host, and override the job locally with validate host. c) put validate host in the zuul-jobs base vars, but override the base job locally and set ... | 21:24 |
| jeblair | ... those as job variables? | 21:24 |
| jeblair | mordred: http://git.openstack.org/cgit/openstack-infra/project-config/tree/playbooks/base/pre.yaml | 21:24 |
| jeblair | oh by "broken" in (a) i meant "broken for non-openstack" | 21:25 |
| jeblair | there's a lot of openstack specific stuff that needs either site-local variables or job variables, not just that. so i'm going to go with b for now. | 21:29 |
| jeblair | we'll have roles in zuul-jobs, and a skeleton base job, but the bulk of our base job will still be in project-config for a little longer. | 21:30 |
| jeblair | mordred: in the README for validate-host, it says: | 21:34 |
| jeblair | This is unsafe to run in Trusted jobs as it will write any secrets to the log. | 21:34 |
| jeblair | mordred: that kind of suggests that it shouldn't be in the base job. | 21:35 |
| mordred | jeblair: sorry - was on phone - reading | 21:44 |
| mordred | jeblair: yah - I think you're right wrt b | 21:55 |
| jeblair | mordred: cool. we should probably bump site local vars up on the list if we think it's a thing. it's probably the next big blocker for reusable zuul-jobs. | 21:55 |
| jeblair | mordred: what about the validate host trusted jobs thing? is that true, or is that a docs error? | 21:55 |
| mordred | jeblair: it's ... what it does is output all of the hostvars for each host | 21:58 |
| mordred | jeblair: variables that we set , like secrets, show up as hostvars | 21:59 |
| mordred | jeblair: and as of right now we don't have a mechanism to mask those | 21:59 |
| jeblair | okay, so if we added a secret, it's instant exposure. | 21:59 |
| jeblair | this is a good thing to know. | 21:59 |
| mordred | jeblair: we could maybe add a variable to what we pass in that lists the names of secrets | 21:59 |
| mordred | jeblair: and update that to filter out anything that matches the list of secrets - since we do know the names of all secrets we pass in | 22:00 |
| mordred | jeblair: we could also skip the hostvars output completely - we also copy the inventory into the logs dir now | 22:01 |
| jeblair | mordred: yep. no to be too contrary here, but given that we copy the.... yeah that. :) | 22:01 |
| mordred | jeblair: testing something real quick ... | 22:02 |
| jeblair | tbh, i prefer that unless there's a compelling reason to do the other. it's simpler, and i worry about accidentally breaking the masking. | 22:03 |
| mordred | jeblair: I have a better fix | 22:03 |
| mordred | which is not to output hostvars at all - but instead to run the setup module and write out its output | 22:04 |
| mordred | (the whole point of that thing is to record the information about the remote host that ansible knows) | 22:04 |
| mordred | setup does NOT display the hostvars of the host | 22:04 |
| jeblair | cool | 22:04 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Switch to using setup instead of hostvars https://review.openstack.org/483585 | 22:08 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Switch to using setup instead of hostvars https://review.openstack.org/483585 | 22:08 |
| mordred | sorry- it was on top of an unrelated patch | 22:08 |
| mordred | jeblair: oh - also - the other things where I have the note about site-local in validate-host should fail open | 22:12 |
| mordred | jeblair: which is to say - if those variables are not set, the role should still work | 22:12 |
| jeblair | mordred: ok | 22:12 |
| mordred | jeblair: same with bindep - although I need to finish the stack that removes support for fallback | 22:12 |
| *** mattclay has quit IRC | 22:13 | |
| *** mattclay has joined #zuul | 22:13 | |
| mordred | jeblair: I tried to make them so that they work fine, but if you want to set a few variables (which we could set as variables on our base job) it'll do stuff with them | 22:13 |
| *** Shrews has quit IRC | 22:13 | |
| *** Shrews has joined #zuul | 22:13 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Move some base job roles from openstack-zuul-roles https://review.openstack.org/483587 | 22:14 |
| jeblair | remote: https://review.openstack.org/483591 Update base-test to use new zuul-jobs roles | 22:17 |
| jeblair | mordred: cool, i'll look at doing that in a later stage. turns out stage one is big enough. :) | 22:17 |
| jeblair | fyi, pabelanger and i are mid-stream on some URL stuff, so zuulv3 log links are slightly broken right now. just delete the last directory component from the link and you'll be good. | 22:19 |
| jeblair | mordred, jlk: ^ | 22:19 |
| jlk | okay | 22:19 |
| * Shrews now sports the ansible/staff /whois Cloak of Invisibility and gains 10 experience points | 22:19 | |
| jlk | hah | 22:20 |
| Shrews | mordred: k on the config stuff. i need to thoroughly review that still | 22:20 |
| pabelanger | Shrews: nice | 22:23 |
| mordred | Shrews: also, feel free to either tell me to go away - or to tell me I touched it last and you're going to look at something else - both are understandable ;) | 22:23 |
| Shrews | mordred: that wouldn't be very sporting of me, either way. can i offer you a drink instead? | 22:24 |
| Shrews | Ketel and cran, iirc | 22:25 |
| mordred | Shrews: yes! | 22:26 |
| mordred | I accept | 22:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Filter and print results more comprehensively https://review.openstack.org/483311 | 22:27 |
| mordred | jeblair, Shrews, pabelanger, jlk: we actually need to get that landed - otherwise our logging leaks stuff marked no_log | 22:27 |
| mordred | pabelanger: (also, I haven't forgotten to follow up with you on next steps in my brainhole re: logging - but I have a spate of late-day calls today) | 22:28 |
| jlk | reviewing | 22:28 |
| jlk | was "impotant" on purpose in the commit message? | 22:28 |
| mordred | it wasn't - I can respin real quick ... | 22:29 |
| pabelanger | mordred: np, we can pick it up in the morning if you want | 22:29 |
| jlk | not super necessary | 22:29 |
| jeblair | mordred: you run validate-host in the unittest pre playbook as well as the base job | 22:30 |
| jeblair | mordred: this has the curious effect of actually making your host info change somewhat self-testing | 22:30 |
| mordred | jeblair: leftovers | 22:30 |
| mordred | jeblair: hah. nice | 22:30 |
| jeblair | mordred: http://logs.openstack.org/85/483585/2/check/tox-py35-on-zuul/5aa9bf2/zuul-info/ | 22:30 |
| pabelanger | mordred: +3 | 22:30 |
| jeblair | mordred: *both* versions are there | 22:30 |
| jeblair | mordred: ansible- from base, and host- from unittests | 22:31 |
| jeblair | mordred: it kind of freaked me out because i was not expecting that to be self testing | 22:31 |
| mordred | jeblair: and you can see that host- has interesting info without having variables that aren't set by ansible | 22:31 |
| jeblair | i had to go manually remove all the output that your log output patch removes to see the play structure. :) | 22:31 |
| mordred | :) | 22:32 |
| jeblair | mordred: it's a shocking amount of information. :) | 22:32 |
| mordred | pabelanger: the tl;dr is make our text log output file friendly and easy to read without too much extra crap in it - and then to ALSO emit a json log that has all of the things in it that we can have html/javascript present for the user with the ability to show/hide various kinds of info | 22:32 |
| mordred | pabelanger: so that for live-streaming we have a readable thing | 22:33 |
| mordred | pabelanger: but we collect all the info so that people can find all the data they need as they need to | 22:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: DNM: use base-test in unittest jobs https://review.openstack.org/483593 | 22:34 |
| mordred | pabelanger: both of those will still obey no_log though - since that one is an important way for tasks to indicate they contain sensitive information | 22:34 |
| mordred | pabelanger: BUT - it should allow us to mark many fewer things as "no_log" - since we won't have to use it to prevent things just from being chatty like we do today | 22:34 |
| pabelanger | ack, understood | 22:35 |
| mordred | pabelanger: I got the beginnings of the json thing today (unfortunately can't just use the json callback from ansible itself as is) | 22:35 |
| mordred | but should have that ready to go for tomorrow | 22:35 |
| pabelanger | cool | 22:37 |
| jeblair | pabelanger: does our puppet reconfigure zuul-scheduler on tenant config file (main.yaml) changes? | 22:39 |
| pabelanger | let me look | 22:40 |
| jeblair | pabelanger: i think based on the zuul logs the answer is no | 22:40 |
| pabelanger | Ya, I don't think we added the logic for that, mostly because I wasn't sure what was needed | 22:41 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Support dynamic dependent pipeline additions https://review.openstack.org/483597 | 22:41 |
| jeblair | pabelanger: just like the old layout.yaml change. SIGHUP | 22:41 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Filter and print results more comprehensively https://review.openstack.org/483311 | 22:41 |
| jeblair | i just sighupped zuulv3.o.o so it should have the shadow configuration now | 22:41 |
| jlk | jeblair: that does the simple dynamic change queue add, which seems to pass tests (including the test I added for the failing case). | 22:42 |
| jeblair | pabelanger, mordred: that means we're ready to land 483587 and 483591 | 22:42 |
| jeblair | jlk: wow! | 22:42 |
| pabelanger | Hmm | 22:42 |
| pabelanger | we should have called /etc/init.d/zuul-scheduler reload | 22:42 |
| pabelanger | when /etc/zuul/layout changes | 22:43 |
| jeblair | pabelanger: zuul-scheduler doesn't have socket command listener yet | 22:43 |
| jeblair | i would like to add one | 22:43 |
| pabelanger | k, so we have a bug | 22:43 |
| jeblair | but haven't had time. anyone else should feel free. :) | 22:43 |
| jeblair | pabelanger: yes, it's also why stop doesn't work. | 22:43 |
| jlk | jeblair: I want to play with that function in future changes a bit more, to see how it handles adding to an existing named queue, or adding a brand new named queue, and maybe adding a change that dependsOn a change that adds to a queue, and... | 22:44 |
| jeblair | jlk: ++ | 22:44 |
| pabelanger | jeblair: k, I'll get a patch up in the morning to SIGHUP for now | 22:44 |
| jeblair | pabelanger: yeah, the zuul-scheduler init script looks like the zuul-executor init script. which i would *love* to be the case, but that just hasn't been implemented yet. it should still work just like the v2 scheduler init script | 22:45 |
| pabelanger | +2 on patches from above | 22:46 |
| jeblair | jlk: just skimming that really quick, you may need to removeQueue in deQueueItem (like in independent pipeline manager) | 22:47 |
| jlk | oh I thought I grabbed that, oops! | 22:47 |
| jeblair | er "dequeueItem" | 22:48 |
| jlk | ohhhh need to make sure we only remove a dynamically added one though, right? since empty queues are a "normal" thing? | 22:49 |
| jeblair | jlk: yep | 22:49 |
| jeblair | jlk: and that's probably a good end-of-test assertion too | 22:49 |
| jlk | yup | 22:49 |
| jlk | cool, I'll add that bit | 22:49 |
| jeblair | peek at layout.pipeline[].queues or whatever | 22:49 |
| mordred | jeblair: landing | 22:51 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Move some base job roles from openstack-zuul-roles https://review.openstack.org/483587 | 22:52 |
| pabelanger | jeblair: mordred: maybe we should raise an exception some how in zuul, if we are not able to find the playbook for pre-run, run, post-run. Right now, we just get RETRY_LIMIT failures | 23:21 |
| pabelanger | 483548,2 for example | 23:21 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: DNM - add upper-constraints to tox https://review.openstack.org/483548 | 23:22 |
| jeblair | pabelanger: that is story 2001105 | 23:22 |
| pabelanger | had to log into ze01 to see what was going on | 23:22 |
| pabelanger | jeblair: Nice! | 23:22 |
| pabelanger | jeblair: how did we fix the 60s timeout issue with bubblewrap between playbooks? | 23:28 |
| mordred | pabelanger: we have not yet - that's also a story ... | 23:29 |
| mordred | pabelanger: https://storyboard.openstack.org/#!/story/2001072 | 23:30 |
| pabelanger | mordred: oh, odd. things seem faster now | 23:31 |
| pabelanger | or maybe I am just used to the delay now | 23:31 |
| mordred | pabelanger: I agree - they do | 23:31 |
| mordred | I wonder if the switch to the local key has somehow made it better | 23:31 |
| pabelanger | Oh, have we done something new with SSH keys? | 23:32 |
| mordred | pabelanger: first step in the base playbook now is to generate keys, add them to agent and delete the reference to the original key | 23:32 |
| pabelanger | Ah, nice. So we did implement that | 23:33 |
| mordred | pabelanger: http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/add-build-sshkey | 23:33 |
| pabelanger | mordred: do we keep the private key around inside bwrap for a reason? Doesn't look like we delete that | 23:35 |
| pabelanger | so, possible for somebody to output that during job run time to access node | 23:36 |
| mordred | pabelanger: don't know - although I've been thinking we should go ahead and distribute the private key to the remote hosts too so that it's easy for folks to make jobs that talk been nodes | 23:37 |
| mordred | pabelanger: seems like a good thing to talk through in both directions :) | 23:38 |
| mordred | pabelanger: actually - a person could just write a job to install their existing public key on the nodes without stealing the existing private key - so I don't think we can actually protect against that attack | 23:38 |
| pabelanger | mordred: ya, I'll have to think about it more | 23:40 |
| mordred | pabelanger: in fact, people could do that today in v2 by proposing a patch to devstack that wget's a key from a remote location | 23:41 |
| mordred | pabelanger: and in the 5 years of devstack-gate, I don't think anyone has ever tried ;) | 23:42 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: WIP: add environment for tox shell https://review.openstack.org/483608 | 23:43 |
| mordred | pabelanger: I'd be more worried about it if it was a more general key that could be used potentially to connect to servers we care about | 23:43 |
| mordred | pabelanger: ^^ ++ to that patch | 23:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: DNM - add upper-constraints to tox https://review.openstack.org/483548 | 23:43 |
| pabelanger | mordred: ya, I might do some renaming of tox role variable | 23:43 |
| pabelanger | mordred: want to see if it works first | 23:43 |
| mordred | ++ | 23:44 |
| pabelanger | mordred: so, I am no longer seeing ansible output in executor-debug.log, is that expected now? | 23:51 |
| pabelanger | Oh, hmm, I see some | 23:52 |
| pabelanger | [Zuul] Log Stream did not terminate | 23:53 |
| pabelanger | that is new | 23:53 |
| pabelanger | also notice: http://paste.openstack.org/show/615329/ | 23:53 |
| jeblair | pabelanger: mordred said that was showing up on jobs which don't output anything to stdout, so we may need to fix something in the log streamer | 23:54 |
| jeblair | or the stream callback | 23:54 |
| pabelanger | jeblair: okay, thanks | 23:54 |
| jeblair | pabelanger: that was re 'did not terminate' | 23:54 |
| pabelanger | I think that make sense, I don't think we stdout shell any more | 23:54 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: gzip console log before uploading it https://review.openstack.org/483611 | 23:56 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: WIP: add environment for tox shell https://review.openstack.org/483608 | 23:57 |
| mordred | pabelanger: we definitely don't put the stdout stream into the executor-debug log | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!