| *** threestrands has joined #zuul | 00:26 | |
| *** dkranz has quit IRC | 00:54 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Log provider names with quota https://review.openstack.org/529178 | 01:00 |
|---|---|---|
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: WIP: Fail on quota-exceeded (partial revert) https://review.openstack.org/529173 | 01:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Assume a quota limit of -1 means unlimited https://review.openstack.org/529180 | 01:00 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Use yarn and webpack to manage zuul-web javascript https://review.openstack.org/487538 | 01:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add babel transpiling enabling use of ES6 features https://review.openstack.org/528295 | 01:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add StandardJS linting and analysis https://review.openstack.org/528296 | 01:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Fix source_url handling for jobs view https://review.openstack.org/528373 | 01:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Fix StandardJS warnings and turn them to errors https://review.openstack.org/528297 | 01:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add bundle analysis to the lint target https://review.openstack.org/528298 | 01:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Make bundle of build web content https://review.openstack.org/528374 | 01:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Remove use strict https://review.openstack.org/528437 | 01:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Inject url endpoint information https://review.openstack.org/529193 | 01:07 |
| mordred | jeblair: ok. I got the unittest thing fixed more better - and also as of https://review.openstack.org/529193 the generated draft web code should totally work WRT internal links | 01:08 |
| mordred | tobiash: ^^ | 01:08 |
| mordred | I have a few more followup thoughts on top of https://review.openstack.org/529193 that I want to explore | 01:09 |
| tristanC | mordred: jeblair: so to get the api documentation in sphinx doc, it seems like we need to go the other way around: write a swagger file, then use sphinxcontrib-openapi for the doc and aiohttp_swagger for the '/api/doc' route | 01:14 |
| mordred | tristanC: wow. | 01:14 |
| tristanC | openapi sphinx extension looks pretty good to me, see: https://sphinxcontrib-openapi.readthedocs.io/ | 01:15 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Fail on quota-exceeded (partial revert) https://review.openstack.org/529173 | 01:17 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Assume a quota limit of -1 means unlimited https://review.openstack.org/529180 | 01:17 |
| mordred | tristanC: oh - you know - I guess that's also because the sphinx plugin would not have any way of connecting the docs for a call with the route - since we register routes and methods in code | 01:17 |
| mordred | so that makes sense | 01:17 |
| mordred | I think going that route would allow us to have the normal live swagger api rest api doc thing, as well as the ability to craft a narrative document about the calls | 01:18 |
| mordred | hah. I like that they link the status codes to the w3c definition of the code :) | 01:19 |
| tristanC | i'll have a look, in any case, it seems better to have a standalone swagger file instead of doing it in docstring | 01:19 |
| mordred | ++ | 01:20 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Inject url endpoint information https://review.openstack.org/529193 | 01:34 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Make bundle of build web content https://review.openstack.org/528374 | 01:34 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Remove use strict https://review.openstack.org/528437 | 01:34 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Log provider names with quota https://review.openstack.org/529178 | 01:41 |
| *** _ari_ is now known as ari|pto | 01:46 | |
| mordred | woot! | 01:56 |
| mordred | http://logs.openstack.org/93/529193/2/check/build-javascript-content/514252d/npm/html/status.html | 01:56 |
| mordred | tristanC: ^^ note that the jobs and builds links work properly - as do the websocket streaming links from the status page | 01:56 |
| tristanC | mordred: that's really neat, i'll rebase the new job page on top of it | 01:57 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Fail on quota-exceeded (partial revert) https://review.openstack.org/529173 | 02:04 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Fix build-javascript-content success-url https://review.openstack.org/529203 | 02:50 |
| *** rlandy|rover has quit IRC | 03:01 | |
| *** threestrands has quit IRC | 03:03 | |
| *** threestrands has joined #zuul | 03:04 | |
| *** threestrands has quit IRC | 03:04 | |
| *** threestrands has joined #zuul | 03:04 | |
| *** threestrands has quit IRC | 03:05 | |
| *** threestrands has joined #zuul | 03:06 | |
| *** threestrands has quit IRC | 03:06 | |
| *** threestrands has joined #zuul | 03:06 | |
| *** threestrands has quit IRC | 03:07 | |
| *** threestrands has joined #zuul | 03:07 | |
| *** threestrands_ has joined #zuul | 03:57 | |
| *** threestrands has quit IRC | 03:57 | |
| *** threestrands_ has quit IRC | 03:58 | |
| *** threestrands_ has joined #zuul | 03:59 | |
| *** ianw is now known as ianw_pto | 04:57 | |
| tobiash | jeblair: looking | 05:03 |
| tobiash | pabelanger: if you want old ready nodes being deleted you need to configure max-ready-age for the label | 05:04 |
| tobiash | The default is infinite | 05:04 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Assume a quota limit of -1 means unlimited https://review.openstack.org/529180 | 05:15 |
| *** bhavik has joined #zuul | 05:42 | |
| *** bhavik has quit IRC | 05:44 | |
| *** bhavik has joined #zuul | 05:45 | |
| *** bhavik has quit IRC | 05:54 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Assume a quota limit of -1 means unlimited https://review.openstack.org/529180 | 06:03 |
| tobiash | jeblair: that change just missed an 'import math' so I fixed it and approved based on the previous reviews ^^ | 06:08 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Test that -1 works for infinite quota https://review.openstack.org/529231 | 06:19 |
| tobiash | jeblair: and that is a followup that would catch the missing import not only in pep8 but also in the unit tests ^^ | 06:19 |
| tobiash | jlk, SpamapS: I think you had a discussion about github caching issues some weeks ago. Did you solve your problems? | 06:35 |
| tobiash | I have a gate requirement to a label 'merge' and that matches sometimes and sometimes not non-deterministically | 06:36 |
| *** threestrands_ has quit IRC | 06:57 | |
| *** flepied_ has quit IRC | 08:19 | |
| *** hashar has joined #zuul | 08:27 | |
| *** flepied has joined #zuul | 08:35 | |
| *** kmalloc has quit IRC | 08:41 | |
| *** jpena|off is now known as jpena | 08:44 | |
| *** flepied_ has joined #zuul | 09:51 | |
| *** flepied has quit IRC | 09:54 | |
| *** jeblair has quit IRC | 10:15 | |
| *** jeblair has joined #zuul | 10:16 | |
| *** flepied has joined #zuul | 10:17 | |
| *** flepied_ has quit IRC | 10:19 | |
| *** flepied has quit IRC | 10:24 | |
| *** flepied has joined #zuul | 10:25 | |
| *** jaianshu has joined #zuul | 10:37 | |
| jaianshu | Hi, does anyone know about this error: Ansible output: b"Can't find source path /opt/zuul-scripts: | 10:37 |
| jaianshu | No such file or directory" | 10:37 |
| jaianshu | i'm able to run ansible job but after that it fails - http://paste.openstack.org/show/629444/ | 10:37 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Set remote url on every getRepo in merger https://review.openstack.org/529293 | 10:48 |
| tobiash | jaianshu: you probably configured trusted_ro_paths in zuul conf? | 10:49 |
| tobiash | that is just an example how you can add files to the execution environment of ansible | 10:50 |
| tobiash | you should just remove this config | 10:50 |
| jaianshu | yes, i did....ok | 10:50 |
| *** hashar is now known as hasharAway | 10:54 | |
| *** yolanda__ has joined #zuul | 11:05 | |
| *** yolanda has quit IRC | 11:08 | |
| *** flepied_ has joined #zuul | 11:13 | |
| *** flepied has quit IRC | 11:14 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/zuul feature/zuulv3: Add run_ansible_setup_on_start flag for executor https://review.openstack.org/529300 | 11:24 |
| jaianshu | tobiash: now i'm getting Ansible exit code: 2 ... Ansible complete, result RESULT_NORMAL code 2 for the Ansible Job, any idea what does exit code: 2 means? | 11:24 |
| rcarrillocruz | mordred , jeblair : we talked about the runAnsibleSetup on start by default on all nodes not working for us (networking). I've been checking what we talked about maybe running that method to just nodes using ssh connection type (which relies upon the ansible_connection plumbing from nodepool up to zuul). However, that would be a bit complex, it would require I create a special purpose group on getHostList to | 11:27 |
| rcarrillocruz | put the groups to exclude on the inventory, so in runAnsibleSetup I can run the setup run against all excluding group not using ssh connection type | 11:27 |
| rcarrillocruz | long story short: i think is easier to have a general flag to disable it should it needed | 11:27 |
| rcarrillocruz | we would disable it, by default it will run setup on everything | 11:28 |
| rcarrillocruz | it's the patch above ^ | 11:28 |
| rcarrillocruz | let me know what you folks think when you get around | 11:28 |
| rcarrillocruz | other option would be changing runAnsibleSetup to do a 'ansible -m wait_for' , but in my experience is dodgy and it does not check end to end connectivity , which is a requirement for that function per jeblair | 11:30 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/zuul feature/zuulv3: Add run_ansible_setup_on_start flag for executor https://review.openstack.org/529300 | 11:35 |
| tobiash | rcarrillocruz: that is a all or nothing config, what do you think about that as a job attribute (which could be set by the base job)? | 11:38 |
| rcarrillocruz | imho, don't think that should by a by job attribute. Zuul/Nodepool is responsible for giving a sane underlying infra to run tests (obviously, network can be lost in between, but still). If anything, maybe a node attribute? | 11:41 |
| tobiash | yes, maybe an attribute coming from nodepool | 11:42 |
| tobiash | that also would allow mixed jobs with parts doing the setup and parts not | 11:42 |
| rcarrillocruz | i think all those options are not mutually exclusive tho. Thing is, v3 is supposed to be out of the door very soon, making schema changes to nodes and or jobs i don't think those will be in time | 11:44 |
| rcarrillocruz | thus my intent of having this all or nothing | 11:44 |
| rcarrillocruz | which is a valid use case too | 11:44 |
| rcarrillocruz | and getting it for 3.0 | 11:44 |
| rcarrillocruz | so it works for us | 11:44 |
| tobiash | rcarrillocruz: works for me | 11:44 |
| tobiash | (the all or nothing switch) | 11:45 |
| rcarrillocruz | i really think doing the per group inventory is not as elegant | 11:45 |
| rcarrillocruz | and moreover, we won't need that in the mid-term | 11:45 |
| rcarrillocruz | we have plans to plugify gather_facts, so in case of networking it calls the needed modles | 11:45 |
| rcarrillocruz | like | 11:46 |
| rcarrillocruz | 'if gather_facts: yes and i'm on networking connection type, let's guess underlying OS. Oh good, it's IOS, then let's call ios_facts' | 11:46 |
| rcarrillocruz | that kind of thing | 11:46 |
| tobiash | that sounds good | 11:48 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/zuul feature/zuulv3: Add run_ansible_setup_on_start flag for executor https://review.openstack.org/529300 | 12:06 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/zuul feature/zuulv3: Add run_ansible_setup_on_start flag for executor https://review.openstack.org/529300 | 12:16 |
| *** Wei_Liu has joined #zuul | 12:23 | |
| Wei_Liu | hello, I used role "prepare-workspace" to sync executor src to work node, but it hang sometime when executing synchronize and got message "Output suppressed because no_log was given". Is there anyone who get same issue? | 12:24 |
| Wei_Liu | hello? | 12:36 |
| *** yolanda__ is now known as yolanda | 12:53 | |
| *** jpena is now known as jpena|lunch | 12:58 | |
| Wei_Liu | hello, I used role "prepare-workspace" to sync executor src to work node, but it hang sometime when executing synchronize and got message "Output suppressed because no_log was given". Is there anyone who get same issue? | 12:59 |
| *** dmellado has quit IRC | 13:05 | |
| tobiash | Wei_Liu: you get the message because the task defines no_log to suppress log output | 13:07 |
| tobiash | Wei_Liu: you can remove that in your deployment for debugging | 13:07 |
| tobiash | mordred: any idea what could be the cause of http://logs.openstack.org/93/529293/1/check/tox-py35/45c66c4/testr_results.html.gz ? | 13:08 |
| tobiash | the log is not *that* helpful | 13:08 |
| tobiash | ran the tests locally several times without problems :( | 13:08 |
| *** openstackgerrit has quit IRC | 13:13 | |
| *** dmellado has joined #zuul | 13:15 | |
| *** dmellado has quit IRC | 13:19 | |
| *** dmellado has joined #zuul | 13:21 | |
| *** rlandy has joined #zuul | 13:23 | |
| *** rlandy is now known as rlandy|ruck | 13:24 | |
| *** jaianshu has quit IRC | 13:34 | |
| mordred | tobiash: looking | 13:34 |
| mordred | tobiash: wow. that's ... | 13:36 |
| tobiash | ... not helpful? | 13:36 |
| tobiash | ;) | 13:36 |
| tobiash | I cannot reproduce that locally | 13:36 |
| tobiash | I did a recheck, let's see that this does | 13:37 |
| mordred | tobiash: yah - not helpful at all - I don't see anything obvious in the code ... trying to think of other things to try :( | 13:40 |
| tobiash | mordred: that looks pretty similar like the process-returncode failure we all see occationally | 13:42 |
| mordred | yah | 13:43 |
| mordred | it's ... super unhelpful | 13:44 |
| tobiash | mordred: hrm, the recheck reproduced that | 13:46 |
| tobiash | mordred: there are two alarm clocks in the text log: http://logs.openstack.org/93/529293/1/check/tox-py35/8723364/job-output.txt.gz#_2017-12-20_13_25_07_773934 | 13:48 |
| tobiash | that might be somehow related | 13:48 |
| mordred | tobiash: oh - yah. that does seem like perhaps related | 13:49 |
| tobiash | mordred: a different change doesn't have the alarm clocks | 13:49 |
| mordred | :( | 13:49 |
| tobiash | and there's exaclty two like the two new tests | 13:49 |
| tobiash | mordred: one of these tests is relatively slow: http://paste.openstack.org/show/629457/ | 13:51 |
| *** rlandy|ruck is now known as rlandy|rover | 13:52 | |
| tobiash | maybe I disable that check see how that runs | 13:52 |
| *** openstackgerrit has joined #zuul | 13:53 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Set remote url on every getRepo in merger https://review.openstack.org/529293 | 13:53 |
| *** jpena|lunch is now known as jpena | 13:54 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Do not use --ignore-missing-args for rsync https://review.openstack.org/529320 | 13:56 |
| pabelanger | tobiash: Ah, thank you | 13:59 |
| tobiash | pabelanger: the nodepool patch? | 14:03 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Do not use --ignore-missing-args for rsync https://review.openstack.org/529320 | 14:05 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Set remote url on every getRepo in merger https://review.openstack.org/529293 | 14:05 |
| pabelanger | tobiash: re max-ready-age | 14:05 |
| tobiash | ah ;) | 14:06 |
| tobiash | lost the context | 14:06 |
| SpamapS | tobiash: I still have the caching issue. | 14:06 |
| *** logan- has quit IRC | 14:08 | |
| tobiash | SpamapS: :( | 14:10 |
| *** logan- has joined #zuul | 14:10 | |
| SpamapS | tobiash: in debugging I got lost in trying to see if github3.py was actually using etags | 14:15 |
| tobiash | SpamapS: I think I'll first watch a curl to check if that's a github problem or github3.py problem | 14:16 |
| SpamapS | tobiash: it _should_, but getting requests to log the reqs confused me. | 14:16 |
| SpamapS | tobiash: yeah maybe try a curl and twiddle the labels. | 14:17 |
| tobiash | SpamapS: so the curl label watch changes pretty instantaneously | 14:22 |
| tobiash | I cannot see there a noticable delay | 14:23 |
| SpamapS | tobiash: is your curl using an etag? | 14:23 |
| tobiash | does github3.py itself caching? | 14:23 |
| tobiash | no | 14:23 |
| SpamapS | tobiash: github3 does I think | 14:23 |
| SpamapS | just in-memory dict of resource<->etag | 14:24 |
| SpamapS | it uses requests' built-in caching | 14:24 |
| tobiash | SpamapS: looks like I'll first have to read what that does... | 14:24 |
| SpamapS | IIRC, you get an etag with every object. It stores that, and uses If-None-Match based on it. | 14:24 |
| SpamapS | I wonder if it's actually causing the problem. | 14:24 |
| SpamapS | Instead of using etag for invalidation, maybe github just keeps serving you the etag until it times out. | 14:25 |
| SpamapS | which would be... super annoying | 14:25 |
| SpamapS | but we could probably fix things if that's true by invalidating cache on receiving webhooks for things. | 14:25 |
| tobiash | SpamapS: ok, curl watched the etag header and that also changes without delay | 14:27 |
| tobiash | also the last modified header looks correct in the curl watch | 14:29 |
| SpamapS | so the next thing to wonder about is whether the internal caching turns around and just returns us the wrong thing without making a req at all. | 14:29 |
| SpamapS | assuming your curl is accurately reproducing what github3.py is requesitng | 14:29 |
| tobiash | I fear that means debugging github3.py | 14:30 |
| SpamapS | requesting | 14:30 |
| SpamapS | yeah debugging github3.py and requests was hard | 14:30 |
| SpamapS | I just wanted to turn on debug logging and it was like o_O | 14:30 |
| tobiash | so that's my watch: watch -n0.1 'curl -v -s -H "Authorization: token xxxx" https://example.com/api/v3/repos/codecraft/cc-zuul-conf/issues/3/labels 2>&1' | 14:30 |
| tobiash | so next step probably would be a simple python script doing that loop with github3.py | 14:31 |
| *** openstackgerrit has quit IRC | 14:33 | |
| *** openstackgerrit has joined #zuul | 14:34 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add consolidated role for processing subunit https://review.openstack.org/529339 | 14:34 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Remove testr and stestr specific roles https://review.openstack.org/529340 | 14:34 |
| *** sshnaidm has quit IRC | 14:50 | |
| *** sshnaidm has joined #zuul | 14:50 | |
| tobiash | SpamapS: I can reproduce it with a sample script: http://paste.openstack.org/show/629466/ | 14:53 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Do not use --ignore-missing-args for rsync https://review.openstack.org/529320 | 15:00 |
| *** dkranz has joined #zuul | 15:01 | |
| tobiash | SpamapS: removing the cache control adapter fixes the caching issue | 15:01 |
| SpamapS | tobiash: ugh.. well that's good and bad news | 15:05 |
| SpamapS | good that it's fixed | 15:05 |
| SpamapS | but bad that we have to figure out caching gaain | 15:05 |
| SpamapS | again | 15:05 |
| SpamapS | because rate limiting is a pain | 15:05 |
| tobiash | SpamapS: so etag matched requests don't count to the rate limit? | 15:07 |
| tobiash | ok, doc says that's true | 15:10 |
| SpamapS | Right that's why it was turned on I'm sure. | 15:11 |
| SpamapS | So we just have to invalidate etags on webhook I think. | 15:11 |
| tobiash | SpamapS: I just tried out etags in curl | 15:14 |
| jeblair | tobiash: re nodepool quota -- i was thinking maybe we could have nodepool put the request back into the 'requested' state (without declining the request) after an unexpected over-quota error. then the launcher can look at it again in the normal way and decide if it should take it based on its quota and other considerations. it's a bit more work, but it's probably the closest we can get to letting the algorithm work as designed. | 15:14 |
| tobiash | and they seem to work correctly | 15:14 |
| jeblair | rcarrillocruz: what is your connection type? | 15:15 |
| tobiash | whoa, context change | 15:15 |
| rcarrillocruz | network_cli | 15:15 |
| rcarrillocruz | can be netconf too | 15:15 |
| tobiash | jeblair: I think that sounds good | 15:15 |
| tobiash | SpamapS: so I think the etag handling with the cachecontroladapter is somehow broken on client side | 15:16 |
| *** sshnaidm is now known as sshnaidm|afk | 15:16 | |
| jeblair | rcarrillocruz: i think i'd prefer to do something automatic based on the connection type. we can re-write the inventory file (or make a second inventory file) so we don't have to have extra groups in it (which may affect how the job is run) | 15:17 |
| jeblair | rcarrillocruz: we eventually want things to work with mixed connection types -- a network appliance + 2 bare metal nodes, for instance -- so being able to have it adjust behavior to handle that would be good. | 15:19 |
| mordred | jeblair: I think making a second inventory file just for the setup filtered by connection type would be pretty straightforward | 15:19 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul feature/zuulv3: Attempt to improve tenant config loading in case of config issue https://review.openstack.org/529060 | 15:19 |
| rcarrillocruz | so | 15:19 |
| rcarrillocruz | leave one like | 15:19 |
| rcarrillocruz | 'all' | 15:19 |
| rcarrillocruz | with everything | 15:20 |
| rcarrillocruz | then | 15:20 |
| rcarrillocruz | 'ssh' | 15:20 |
| rcarrillocruz | 'network_cli' | 15:20 |
| rcarrillocruz | ... | 15:20 |
| *** myoung is now known as myoung|bbl | 15:20 | |
| rcarrillocruz | so at runAnsibleSetup we run it against 'ssh' inv file | 15:20 |
| rcarrillocruz | is that what you think of | 15:20 |
| rcarrillocruz | ? | 15:20 |
| mordred | rcarrillocruz: I think it can be even easier ... we can either have a blacklist or a whitelist of connection types that setup should be run against - and write a setup inventory and a run inventory | 15:21 |
| mordred | (thinking out loud) | 15:22 |
| jeblair | yeah, i think that will be safer because we allow user-defined groups. so if we kept one inv file, we would have to pick a name that would not collide ('_zuul_internal_ssh'), and then raise an error if the user named a group that way. | 15:23 |
| mordred | tobiash, SpamapS: I wrote a patch to os-service-types a few months ago that used that cachecontroladapter (because i'd seen it used in the gh driver) - and got feedback there that that cachecontrol adapter wasn't working fully correctly. I didn't complete the loop at the time because it seemed like it was working fine for our usecases ... but that could put some fuel on the 'the cachecontroladapter isn't | 15:23 |
| mordred | actually doing things right' fire | 15:23 |
| jeblair | 2 inv files, we don't have to worry about that. either way works though. | 15:23 |
| mordred | jeblair: yah - it also seems like polluting the test environment in a way we don't need to | 15:24 |
| jeblair | mordred: ++ | 15:24 |
| tobiash | mordred: thanks for the info, exactly this seems to break us currently | 15:24 |
| jeblair | mordred: right, even if they don't create a group with that name, someone could still write a play against it, which is awkward. | 15:24 |
| rcarrillocruz | tobiash: can you check my comment on https://review.openstack.org/#/c/501976/ when yo uget a change | 15:25 |
| jeblair | 2 files is best i think. :) | 15:25 |
| rcarrillocruz | i can't do the above without getting nodepool connection type up to zuul | 15:25 |
| tobiash | SpamapS, mordred: https://github.com/sigmavirus24/github3.py/issues/226 | 15:25 |
| tobiash | so according to the open issues in github3.py caching is not yet fully supported | 15:27 |
| tobiash | SpamapS: so I suggest to first remove it and when someone has time to fix that | 15:27 |
| jeblair | jlk: ^ fyi | 15:28 |
| rcarrillocruz | jeblair , mordred : whitelist/blacklist connection type, parameterizable via executor conf section or good to just hardcode now to whitelist = 'ssh'/'winrm' , blacklist=rest | 15:29 |
| rcarrillocruz | in other news , https://arstechnica.com/gadgets/2017/12/microsoft-quietly-snuck-an-ssh-client-and-server-into-the-latest-windows-10/ | 15:30 |
| tobiash | rcarrillocruz: thanks, read that, but hadn't time to answer | 15:31 |
| jeblair | rcarrillocruz: let's hardcode for now. we may need to make it configurable later, but let's see if we can keep up with the connection plugins for now, so it's easier for users (this is something where we know the "right" answer, so we should be able to do it without asking the user) | 15:32 |
| rcarrillocruz | k | 15:33 |
| mordred | jeblair, rcarrillocruz: ++ ... and I'd vote for blacklist fwiw | 15:33 |
| tobiash | rcarrillocruz: answered on 501976 | 15:33 |
| mordred | since running setup / gathering facts *probably* works on the majority of the connection types - and the ones where it doesn't are the exception | 15:34 |
| rcarrillocruz | tobiash: fair enough, wanna rebase? | 15:40 |
| rcarrillocruz | i can too | 15:40 |
| tobiash | does it need a rebase? | 15:40 |
| tobiash | feel free if you want | 15:40 |
| rcarrillocruz | yup, shows cannot merge | 15:40 |
| rcarrillocruz | k | 15:40 |
| tobiash | I'm just about pushing up a github change and have to run after that | 15:41 |
| rcarrillocruz | i can run it, np | 15:41 |
| tobiash | thanks | 15:41 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Remove github caching https://review.openstack.org/529361 | 15:42 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/zuul feature/zuulv3: Use connection type supplied from nodepool https://review.openstack.org/501976 | 15:42 |
| tobiash | SpamapS, jlk: ^^ | 15:42 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: DNM: Add github debugging script https://review.openstack.org/529362 | 15:44 |
| tobiash | SpamapS, jlk: this is a hacky debug script which helped me to find the issue ^^ | 15:45 |
| jeblair | i'd be happy to merge that, fwiw | 15:45 |
| tobiash | jeblair: the debug script? | 15:46 |
| jeblair | yep | 15:46 |
| tobiash | then I'll need to clean it up a bit to take parameters | 15:46 |
| jeblair | it seems like a nice bit of boilerplate for when we have a question about github | 15:46 |
| tobiash | k, I'll clean it up later and push without the DNM | 15:47 |
| jeblair | tobiash: either way -- i was just thinking it's something to start with any time someone needs to poke at the github api. no big deal. :) | 15:47 |
| tobiash | ok, thinking as a debugging template that make sense as is | 15:48 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add github debugging template https://review.openstack.org/529362 | 15:51 |
| tobiash | so that's a little nicer ^^ | 15:51 |
| rcarrillocruz | ah nice | 15:59 |
| rcarrillocruz | so the inventory you explicitly set it on the ansible.cfg the executor creates on the fly | 15:59 |
| rcarrillocruz | i.e. inventory.yaml | 15:59 |
| rcarrillocruz | so this will be shorter, i was worried the inventory was placed in a folder and runAnsible was just pointed to it, thus putting another file in there for the blacklist inventory may complicate my patch | 16:00 |
| rcarrillocruz | jeblair, mordred : if you're cool with comments on https://review.openstack.org/#/c/501976/ , can we merge ? | 16:01 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use python3 for docs publication https://review.openstack.org/525718 | 16:07 |
| *** flepied__ has joined #zuul | 16:12 | |
| *** flepied_ has quit IRC | 16:15 | |
| SpamapS | tobiash: thanks for debugging. It's been driving my team mates crazy to label things and have zuul do nothing. | 16:27 |
| tobiash | SpamapS: it drove me crazy too :) | 16:34 |
| tobiash | we'll go with v3 and github (with ghe apps) productive early january and I have to make this combination as useful as possible till then... | 16:36 |
| SpamapS | tobiash: People don't like gerrit? ;) | 16:37 |
| tobiash | SpamapS: we have gerrit (in a very old version) but it was a decision to take github as the de-facto standard in open source world | 16:38 |
| jlk | tobiash: oh man, that's really sad about caching. That's going to hurt our rate limits a lot :( | 16:38 |
| tobiash | it's part of a mind change we want to drive (more open and community driven) | 16:38 |
| jlk | I wonder if it's specifically a GHE problem, or something else? | 16:39 |
| tobiash | jlk: testing with curl, the ghe works correctly with etags | 16:39 |
| tobiash | so it must be something within the client | 16:39 |
| jlk | tobiash: any hints as to what's borken on the github3.py side? I have commit rights there now... | 16:39 |
| SpamapS | jlk: I have it too with our GHE | 16:40 |
| SpamapS | seems like it is a problem with how github3.py uses etags | 16:40 |
| tobiash | jlk: unfortunately I have no clue how github3.py works internally | 16:40 |
| tobiash | my current knowledge is that it doesn't work and I'm happy that I could fix that before going to production | 16:41 |
| tobiash | but the debug script from above could help in fixing the cache side | 16:41 |
| tobiash | just run it and toggle labels | 16:42 |
| tobiash | with cache it takes 30s to notice the change, without change it's without delay | 16:42 |
| jlk | maybe we just weren't dealing with labels much in public github | 16:43 |
| SpamapS | we did switch to reviews quickly | 16:43 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add role to fetch output from nodes https://review.openstack.org/511843 | 16:43 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Update upload-logs to process docs as well https://review.openstack.org/511853 | 16:44 |
| jlk | hrm, so I wonder if it's specifically an issue with labels | 16:44 |
| SpamapS | Well, I think it's an issue with fetching a single issue, labels or not. | 16:45 |
| tobiash | what I'm planning now is also to take over jamielennox's dynamic branch protection patch (471175) and make that work | 16:45 |
| mordred | SpamapS: is the cause that someone adds a label, which sends an event immediately which zuul notices, but then zuul has to fetch the issue in question to get the details and the fetched issue winds up not having the label? | 16:46 |
| tobiash | that would us the possibility to have a generic gate pipeline | 16:46 |
| SpamapS | mordred: exactly | 16:46 |
| SpamapS | but we think maybe it's getting a 304 and not actually fetching | 16:46 |
| SpamapS | instead of using the hook as a reason to drop the etag | 16:46 |
| mordred | nod. cool. I grok the issue then | 16:46 |
| SpamapS | it's kind of weird and hard to turn on requests debug logging | 16:47 |
| SpamapS | which it should not be | 16:47 |
| mordred | yah. it seems like if we get a hook event for an issue then we'd want to invalidate the cache for that issue | 16:47 |
| jlk | well | 16:47 |
| jlk | I think we put faith in the GitHub API that it would know the data behind the GET has changed | 16:47 |
| jlk | like, that's the entire point of the ETAG | 16:47 |
| mordred | yah. good point | 16:48 |
| jlk | and as tobiash said, using CURL directly works | 16:48 |
| tobiash | mordred: github works perfectly there if we send the correct etag (tested via curl) | 16:48 |
| SpamapS | Yeah I was speculating that it might be doing something different. | 16:48 |
| jlk | I had an open change somewhere I thought to add debug logging for github3.py | 16:48 |
| tobiash | I watched with curl in a loop getting 304, then toggled a label and got immediately the correct response after that | 16:48 |
| SpamapS | or maybe github3 is just not handling the cache miss properly. | 16:48 |
| tobiash | maybe it's on a different level | 16:50 |
| jlk | could be in requests somewhere | 16:50 |
| tobiash | github3.py makes 3 requests to get to the labels | 16:50 |
| jlk | since I think github3 just leans on that | 16:50 |
| tobiash | pull request -> issue -> label | 16:50 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul feature/zuulv3: Attempt to improve tenant config loading in case of config issue https://review.openstack.org/529060 | 16:50 |
| mordred | SpamapS: http://paste.openstack.org/show/629474/ | 16:51 |
| tobiash | if pull request IS unchanged (hypothetical, didn't validate that yet) then github3.py might not request updates to the issue and labels at all | 16:51 |
| mordred | SpamapS: there's the recipe for enabling request level debug logging | 16:51 |
| jlk | afk for a moment, making breakfast. | 16:51 |
| mordred | (although obviously where it does logging.basicConfig() you'd want to do whatever logging config you're aiming for) | 16:51 |
| jlk | tobiash: that could be it. Could be that because the data is indirect. | 16:52 |
| tobiash | maybe | 16:52 |
| jlk | I can dig into this, but we're in the middle of refactoring these objects | 16:52 |
| tobiash | jlk: maybe the refactoring will fix that by accident ;) | 16:52 |
| tobiash | so if that's the case then something would need to be changed in github3.py anyway to not do separate caching (don't know if that's the case) | 16:54 |
| jlk | I plan to look into this today | 16:56 |
| jlk | tobiash: thank you for uncovering this! | 16:58 |
| SpamapS | mordred: what i want is a recipe for logging.conf that does it :) | 16:58 |
| tobiash | jlk: thanks for looking into the caching :) | 16:59 |
| tobiash | jlk: maybe the problem is not github3.py but the cachecontrol adapter | 17:01 |
| *** hasharAway is now known as hashar | 17:01 | |
| tobiash | at least the log suggests that it does api requests regardless of the cachecontrol adapter | 17:02 |
| mordred | SpamapS: probably http://paste.openstack.org/show/629477/ ... but the "HTTPConnection.debuglevel = 1" is probably the missing piece | 17:02 |
| tobiash | http://paste.openstack.org/show/629478/ | 17:03 |
| mordred | SpamapS: (so you might have to hack that in to place somewhere, which is sad | 17:03 |
| tobiash | that's the same for each iteration | 17:03 |
| tobiash | so it looks like from point of github3.py it does requests | 17:03 |
| tobiash | mordred: that doesn't animate urllib to spit out logs :( | 17:05 |
| tobiash | mordred: the context is just urllib3 | 17:07 |
| tobiash | and now it's getting interesting: | 17:07 |
| tobiash | http://paste.openstack.org/show/629479/ | 17:07 |
| tobiash | that's with cache control adapter | 17:08 |
| tobiash | and that without the cachecontrol adapter: http://paste.openstack.org/show/629480/ | 17:09 |
| tobiash | so the cache control adapter catches the requests and doesn't make one based on etags | 17:09 |
| mordred | tobiash: maybe add 'cachecontrol.controller' at the debug level ... | 17:10 |
| tobiash | trying | 17:10 |
| mordred | tobiash: that seems to be where in the cachecontrol lib the logging is happening | 17:10 |
| tobiash | http://paste.openstack.org/show/629481/ | 17:11 |
| tobiash | so it has a max age which we probably shoud suppress | 17:12 |
| mordred | tobiash: well, we have cache_etag=True, which http://cachecontrol.readthedocs.io/en/latest/etags.html#turning-off-equal-priority-caching indicates will use the etags | 17:25 |
| tobiash | mordred: yes, that's what the docs say | 17:25 |
| tobiash | but not what it does... | 17:25 |
| mordred | tobiash: oh - actually - wait | 17:26 |
| tobiash | mordred: ah, I think github3.py would have to set the etag header | 17:26 |
| mordred | tobiash: there are two behaviors - Time Priority- ETag support only takes effect when the time has expired. and Equal Priority - decision to cache is either time base or due to the presense of an ETag | 17:27 |
| mordred | but what *we* want is to always ignore time when there is an etag header | 17:27 |
| mordred | so an ETag Priority | 17:27 |
| mordred | for that, I think we may need to do something like in http://cachecontrol.readthedocs.io/en/latest/custom_heuristics.html | 17:27 |
| tobiash | mordred: that's what I'm trying to do | 17:28 |
| tobiash | stripped out the cache-control header | 17:28 |
| tobiash | but that defeats caching completely | 17:28 |
| mordred | tobiash: what if you strip expires? what all headers do you get from gh? | 17:28 |
| tobiash | let me check | 17:29 |
| tobiash | ah no, it might work | 17:30 |
| tobiash | have to take a break | 17:30 |
| tobiash | brb | 17:30 |
| mordred | sweet | 17:33 |
| SpamapS | I've almost never seen Expires work well. | 17:34 |
| SpamapS | Even when it's useful, it hides real problems. | 17:34 |
| SpamapS | Part of me is like "can we just fork a squid off?" | 17:35 |
| *** apevec has joined #zuul | 17:36 | |
| pabelanger | So, looking to get some help with a deadlock issues we are seeing in nodepool for rdoproject, today jpena has been working on debuging the core we have of more recently deadlock in nodepool, and curious if people would mind helping. https://review.rdoproject.org/etherpad/p/nodepool-core has the most recent info. We initially thought it might be paramiko causing the issue, but upgraded to 2.1.1 yesterday and | 17:38 |
| pabelanger | still got a deadlock, right now I am not sure if this is a new deadlock or existing | 17:38 |
| pabelanger | so far, seem a deadlock happens once a week, but this core is from less then a day | 17:39 |
| pabelanger | jpena feel free to add anything I might have left out | 17:39 |
| jpena | so the symptoms are pretty similar to the paramiko issue pabelanger mentioned: nodepool hangs, stops sending any output to the logs, and kill -SIGUSR2 doesn't produce any output | 17:40 |
| jpena | we got a core from this morning's hang, so I'm happy to get whatever additional debugging info is needed | 17:40 |
| tobiash | hooray | 17:46 |
| tobiash | I think I have it: http://paste.openstack.org/show/629483/ | 17:46 |
| jeblair | jpena, pabelanger: so you think requests or urllib3 are stuck? | 17:46 |
| jpena | jeblair: that's my current suspect, it would explain why shade is stuck in self._finished.wait() | 17:47 |
| tobiash | this is the changed cachecontrol: http://paste.openstack.org/show/629484/ | 17:47 |
| jpena | jeblair: but I cannot confirm that | 17:48 |
| jeblair | jpena: what version of requests do you have? | 17:48 |
| jpena | jeblair: 2.11.1 | 17:49 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul-base-jobs master: Add generic base and base-test jobs/playbooks https://review.openstack.org/526140 | 17:49 |
| jeblair | jpena: fwiw, we're runinng 2.18.4. no idea if anything related has changed. | 17:50 |
| pabelanger | 2.11.0 seems to have bug with hanging | 17:55 |
| pabelanger | https://pypi.python.org/pypi/requests | 17:55 |
| pabelanger | I should say, was fixed in 2.11.0 | 17:56 |
| pabelanger | might be worth upgrading rdo, is an older release | 17:57 |
| apevec | python-requests-2.14.2 is in RDO >= Pike | 17:58 |
| pabelanger | requests>=2.14.2 # Apache-2.0 is what global requirements is currently using, going to see if that is for a specific reason | 18:03 |
| mordred | apevec, pabelanger: also - we're working on getting the version of shade cut thatincludes the connection timeout bug fix | 18:06 |
| pabelanger | mordred: great! | 18:07 |
| jeblair | jpena: what's thread 18 waiting on? | 18:07 |
| mordred | so it would be interesting to know if it's hanging in an initial network connection or not | 18:07 |
| jeblair | mordred: according to the etherpad, it's hanging during a socket close? i think? | 18:08 |
| mordred | oh - I guess the stack shows me the answer to that - that seems to be in ssl close - so it's probably not hanging on initial connection | 18:08 |
| mordred | yah | 18:08 |
| jeblair | i'm confused though because thread 18 doesn't have a * or x next to it, but does seem to be waiting on something? | 18:08 |
| jpena | jeblair: the backtrace for thread 18 is interesting (and huge), see https://review.rdoproject.org/paste/show/61/ | 18:09 |
| jeblair | PyThread_acquire_lock is the gil, right? | 18:09 |
| jpena | mordred: it's actually closing a connection that was reset. https://github.com/shazow/urllib3/blob/1.16/urllib3/connectionpool.py#L248-L250 | 18:10 |
| jeblair | oh, i guess PyThread_acquire_lock could be locks other than the gil | 18:11 |
| jeblair | it's waiting on futex 0x1ba53f0 -- is there a way to find out what other thread has that lock? | 18:14 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool feature/zuulv3: Log unknown providers during quota calculation https://review.openstack.org/529391 | 18:15 |
| jpena | I tried hard to get that, but I couldn't find it. That structure does not have an "owner" field | 18:15 |
| jpena | brb, have to pick up kid | 18:15 |
| jeblair | would a full bt of thread 31 confirm it's the other participant in the deadlock? | 18:17 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix github caching https://review.openstack.org/529392 | 18:17 |
| tobiash | jlk, SpamapS, mordred: this should fix (not remove) the github caching ^^ | 18:17 |
| mordred | tobiash: so it did wind up being cache-control you needed to remove. net | 18:22 |
| tobiash | mordred: that worked perfectly in my test script :) | 18:22 |
| jeblair | tobiash: does that warrant a # NOTE in the code? | 18:23 |
| tobiash | checked the headers and I get correct 304 responses which proves that caching works | 18:23 |
| clarkb | and possibly a bug to cachecontrol? | 18:23 |
| jeblair | (to answer the question "why are we deleting headers?") | 18:23 |
| tobiash | jeblair: I'm just about to write that note ;) | 18:23 |
| jeblair | cool | 18:24 |
| SpamapS | ossssuumm | 18:26 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix github caching https://review.openstack.org/529392 | 18:34 |
| tobiash | now with a note ^^ | 18:34 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use connection type supplied from nodepool https://review.openstack.org/501976 | 18:35 |
| jpena | jeblair: full bt of thread 31: https://review.rdoproject.org/paste/show/62/ | 18:43 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add github debugging template https://review.openstack.org/529362 | 18:49 |
| jeblair | jpena: okay, so i think the interesting question is still what is thread 18 waiting for | 18:52 |
| jeblair | jpena: is it possible to get a full bt from all threads and see if any of them mention futex=0x1ba53f0 ? | 18:52 |
| jeblair | (other than 18) | 18:52 |
| jpena | jeblair: I've tried that, it's not elsewhere (I can upload the full bt, but it's > 600K) | 18:55 |
| *** myoung|bbl is now known as myoung | 19:01 | |
| jlk | ugh, shakes fist at cachecontrol | 19:05 |
| tobiash | jlk: at least you can concentrate now at your refactoring ;) | 19:07 |
| *** dkranz has quit IRC | 19:25 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Set remote url on every getRepo in merger https://review.openstack.org/529293 | 19:34 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: DNM: shall fail https://review.openstack.org/529403 | 19:34 |
| SpamapS | tobiash: fantastic work chasing it down. | 19:35 |
| tobiash | SpamapS: thanks :) | 19:36 |
| SpamapS | I bounced off it twice. | 19:36 |
| SpamapS | So bravo. | 19:36 |
| *** jpena is now known as jpena|off | 19:37 | |
| *** rlandy|rover is now known as rlandy|rover|brb | 19:39 | |
| tobiash | yeah, so I'm currently working hard getting this stuff running right | 19:42 |
| tobiash | I refused to go prodictive without gh apps, so now I have to deliver ;) | 19:42 |
| tobiash | but now with that sorted out and the ghe apps patches and the merger url (529293) it looks pretty good now I think | 19:44 |
| SpamapS | Yeah that's nice that the apps are there. | 19:48 |
| SpamapS | Webhooks kinda suck. ;) | 19:48 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: DNM: shall fail https://review.openstack.org/529403 | 19:49 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Fix build-javascript-content success-url https://review.openstack.org/529203 | 19:50 |
| tobiash | mordred: re strange build errors a few hours ago: http://logs.openstack.org/03/529403/2/check/tox-py35/c1e632d/testr_results.html.gz | 20:03 |
| tobiash | that should fail one with an assertion and not fail the other test | 20:03 |
| tobiash | but again both with the same non-log | 20:04 |
| mordred | tobiash: :( | 20:05 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix github caching https://review.openstack.org/529392 | 20:05 |
| tobiash | mordred: but ps1 where both shall fail works as expected: http://logs.openstack.org/03/529403/1/check/tox-py35/ea62da4/testr_results.html.gz | 20:06 |
| tobiash | that is the test without simulating changing urls | 20:07 |
| tobiash | I wonder if that has something to do with this | 20:07 |
| tobiash | can it be that file://foo:bar@/some/path doesn't work on the xenial nodes? | 20:08 |
| tobiash | locally that works and gets just ignored (which I intended) | 20:08 |
| mordred | maybe? lemme pull and try on my laptop real quick | 20:10 |
| tobiash | you're on xenial? | 20:10 |
| mordred | yah | 20:10 |
| mordred | tobiash: well, for one thing I get this: | 20:12 |
| mordred | http://paste.openstack.org/show/629494/ | 20:12 |
| mordred | tobiash: and then it hangs | 20:12 |
| tobiash | mordred: ah, so when it hangs that triggers alarm clock and inhibits log output | 20:13 |
| tobiash | that makes sense | 20:13 |
| tobiash | ah shit, choices is py36 | 20:13 |
| *** rlandy|rover|brb is now known as rlandy|rover | 20:14 | |
| mordred | whoops | 20:14 |
| tobiash | thanks mordred, I wouldn't have been able to find this | 20:14 |
| mordred | well - at least that explains it! | 20:14 |
| tobiash | :) | 20:14 |
| tobiash | now fixing is easy | 20:14 |
| mordred | sure thing - also - I should have started with pulling and running locally as well ... oh well | 20:14 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Set remote url on every getRepo in merger https://review.openstack.org/529293 | 20:19 |
| tobiash | here we go, that should work ^^ | 20:19 |
| tobiash | mordred: I've a comment on 511843 | 20:28 |
| tobiash | mordred: and a question on 511853 | 20:32 |
| mordred | tobiash: yes - on 511853 I was just having that same thought ... and am trying to come up with a 'good' way to tell if the dir is empty or not | 20:34 |
| tobiash | ok | 20:35 |
| tobiash | mordred: yay, the remote url change is green now :) | 20:42 |
| tobiash | note to myself: rtfm more precisely | 20:42 |
| mordred | \o/ | 20:44 |
| openstackgerrit | Merged openstack-infra/zuul-base-jobs master: Initial boilerplate, packaging and testing https://review.openstack.org/526139 | 21:06 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Log unknown providers during quota calculation https://review.openstack.org/529391 | 21:10 |
| jeblair | mordred: i left a couple of +0 style questions on 529193 when you have a sec | 21:12 |
| *** threestrands_ has joined #zuul | 21:36 | |
| mordred | jeblair: I do not see any questions? | 21:46 |
| mordred | jeblair: btw - did you notice that as of 529193 the links on the draft status page are fully working- including jobs, builds and the log streaming? | 21:48 |
| jeblair | mordred: er, sorry, 487539 | 21:50 |
| jeblair | mordred: and yes! or, well, i noticed in review that had happened, i hadn't actually looked myself yet. but i believed you. :) | 21:50 |
| mordred | jeblair: 487538 I hope? yes - there they are | 21:51 |
| *** jappleii__ has joined #zuul | 22:35 | |
| *** threestrands_ has quit IRC | 22:36 | |
| *** apevec has quit IRC | 23:09 | |
| *** flepied__ has quit IRC | 23:30 | |
| *** hashar has quit IRC | 23:33 | |
| *** rlandy|rover is now known as rlandy|bbl | 23:47 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!