| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: On reconfiguration, re-enqueue items at the same position https://review.openstack.org/516799 | 00:07 |
|---|---|---|
| jeblair | that changes fixes a pretty significant performance degradation ^ i'd like to merge it asap | 00:13 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add support for override-checkout, deprecate override-branch https://review.openstack.org/516451 | 00:43 |
| *** hashar has quit IRC | 00:47 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: On reconfiguration, re-enqueue items at the same position https://review.openstack.org/516799 | 00:50 |
| *** xinliang has quit IRC | 01:37 | |
| *** xinliang has joined #zuul | 01:49 | |
| *** xinliang has quit IRC | 01:49 | |
| *** xinliang has joined #zuul | 01:49 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Authorize the multi-node-bridge network in iptables if there's one https://review.openstack.org/516757 | 02:04 |
| *** robled has quit IRC | 04:53 | |
| *** robled has joined #zuul | 07:21 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /tenants route https://review.openstack.org/503268 | 07:24 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/status route https://review.openstack.org/503269 | 07:25 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/jobs route https://review.openstack.org/503270 | 07:27 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/builds route https://review.openstack.org/466561 | 07:30 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: make console-stream tenant scoped https://review.openstack.org/505452 | 07:30 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{source}/{project}.pub route https://review.openstack.org/502530 | 07:32 |
| *** hashar has joined #zuul | 07:48 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/status route https://review.openstack.org/503269 | 09:28 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/jobs route https://review.openstack.org/503270 | 09:28 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/builds route https://review.openstack.org/466561 | 09:28 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: make console-stream tenant scoped https://review.openstack.org/505452 | 09:28 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{source}/{project}.pub route https://review.openstack.org/502530 | 09:28 |
| *** jkilpatr has joined #zuul | 09:56 | |
| *** electrofelix has joined #zuul | 09:57 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Refactor provider config to driver module https://review.openstack.org/488384 | 09:58 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a static driver for Nodepool https://review.openstack.org/468624 | 09:58 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add username to build and upload information https://review.openstack.org/453968 | 09:59 |
| *** jkilpatr has quit IRC | 10:11 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement an OpenContainer driver https://review.openstack.org/468753 | 10:12 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: builder: do not cleanup image for driver not managing image https://review.openstack.org/516920 | 10:14 |
| *** hashar is now known as hasharAway | 10:58 | |
| *** sambetts|afk is now known as sambetts | 10:59 | |
| *** weshay|PTO is now known as weshay | 12:09 | |
| *** bhavik1 has joined #zuul | 12:35 | |
| *** bhavik1 has quit IRC | 12:39 | |
| *** dkranz has joined #zuul | 12:51 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Use >= in instance count comparison to max-servers https://review.openstack.org/516988 | 13:05 |
| *** hasharAway is now known as hashar | 13:35 | |
| leifmadsen | pabelanger: jeblair: SpamapS: so I'm thinking that the "quickstart" playbooks I'm working on should be specifically biased and low configurability, and link back to the quickstart guide that basically runs the same commands that you would do in the guide. Then would it make sense to build out another more advanced version consuming the Windmill playbooks? I assume that's what Windmill is there for... ? | 13:58 |
| leifmadsen | (although I'm going to have to lock pabelanger in a room and make him write some docs on how to consume them) | 13:58 |
| *** rfolco has quit IRC | 14:27 | |
| leifmadsen | jeblair: ping | 15:01 |
| jeblair | leifmadsen: hi there! | 15:01 |
| leifmadsen | it's 11am :) | 15:01 |
| jeblair | leifmadsen: yes! may i have 5 more minutes to finish up my breakfast? sorry i got a slightly late start :/ | 15:02 |
| leifmadsen | absolutely not | 15:02 |
| leifmadsen | documentation waits for no one! | 15:02 |
| leifmadsen | (actually i could use a snack and coffee too :)) | 15:02 |
| leifmadsen | join when you can, I'm idling in the room now | 15:03 |
| pabelanger | leifmadsen: jeblair: I might join up in another few minutes or so, was running errends this morning before travels | 15:23 |
| leifmadsen | all good | 15:23 |
| leifmadsen | we're just debugging stuff ;) | 15:23 |
| SpamapS | leifmadsen: oh, you're working on playbooks? | 15:31 |
| SpamapS | leifmadsen: You could just pare down BonnyCI/hoist | 15:32 |
| leifmadsen | I don't want to follow another set of playbooks, but rather build against the docs | 15:32 |
| SpamapS | leifmadsen: hrm | 15:32 |
| SpamapS | playbooks are code | 15:32 |
| leifmadsen | neat | 15:33 |
| SpamapS | so I'm concerned that we'll have _three_ sets of playbooks to maintain | 15:33 |
| leifmadsen | we're not going to maintain this | 15:33 |
| SpamapS | (software factory, BonnyCI/hoist, and now these) | 15:33 |
| leifmadsen | it's going to be separate from what you would run to actually run | 15:33 |
| leifmadsen | don't worry, they are in my own personal repo | 15:33 |
| leifmadsen | we can discuss later when I'm not trying to do 3 things at once, but I'm following the framework that was discussed in our initial documentation overview for a quickstart | 15:35 |
| leifmadsen | which is the equiv of a "hello world", bare min | 15:35 |
| leifmadsen | make it "do something" | 15:35 |
| *** hashar has quit IRC | 15:39 | |
| leifmadsen | also technically there are already three if you count SF, BonnyCI, and Windmill | 15:42 |
| SpamapS | Yeah didn't even know about Windmill :-P | 15:45 |
| leifmadsen | blame pabelanger :) | 15:45 |
| SpamapS | oh that one, ok I kinda knew about it. | 15:45 |
| SpamapS | so anyway | 15:45 |
| *** hashar has joined #zuul | 15:45 | |
| SpamapS | On a related note.. I'm trying to fetch keys from my zuul, and finding that my zuul has no keys. | 15:45 |
| jeblair | SpamapS: it should generate them at startup | 15:46 |
| SpamapS | jeblair: yeah it has not. My /var/lib/zuul/secrets dir has one directory: 'openstack', which is odd since I have no openstack projects. | 15:47 |
| jeblair | SpamapS: should be /var/lib/zuul/keys | 15:48 |
| SpamapS | ah that dir does not exist at al | 15:49 |
| SpamapS | all | 15:49 |
| jeblair | /var/lib/zuul/keys/<connection name>/<project name> | 15:50 |
| jeblair | SpamapS: is /var/lib/zuul/ writable by zuul? if so it should create keys/ | 15:50 |
| jeblair | (and of course this is on the scheduler node) | 15:50 |
| SpamapS | drwx------ 8 zuul zuul 186 Oct 30 14:42 /var/lib/zuul | 15:52 |
| SpamapS | yeah | 15:52 |
| SpamapS | I see the os.mkdir.. not sure why that isn't happening | 15:53 |
| jeblair | SpamapS: hrm, there should be an exception if something went wrong, probably under "Exception in management event:" | 15:55 |
| SpamapS | jeblair: Yeah looking for something like that now and not finding it. It would be pretty early I'd think. | 15:56 |
| SpamapS | since we _get_project_key_dir pretty early on | 15:57 |
| SpamapS | http://paste.openstack.org/show/625226/ | 15:58 |
| SpamapS | so something is silently failing that shouldn't maybe | 15:58 |
| pabelanger | okay, ready to eavesdrop now | 15:59 |
| leifmadsen | pabelanger: in my bluejeans room | 16:00 |
| leifmadsen | fyi | 16:00 |
| leifmadsen | just about to start basic configuration of zuul | 16:00 |
| leifmadsen | so good timing | 16:00 |
| leifmadsen | pabelanger: reference etherpad we're working with: https://etherpad.openstack.org/p/zuulv3-quickstart | 16:00 |
| SpamapS | jeblair: ok, that's annoying. I dunno why, but in Bonny we actually separated /var/lib/zuul into role-specific state dirs | 16:05 |
| SpamapS | so the keys are in /var/lib/zuul/scheduler-state/keys | 16:05 |
| jeblair | leifmadsen: https://docs.openstack.org/infra/zuul/feature/zuulv3/admin/quick-start.html | 16:05 |
| jeblair | SpamapS: gotcha. yeah, it'll use state_dir +'/keys' | 16:06 |
| SpamapS | Yeah so they are in there | 16:07 |
| SpamapS | Now to figure out why I'm getting 404 on them | 16:07 |
| SpamapS | hm that regexp looks suspect | 16:10 |
| SpamapS | m = re.match('/keys/(.*?)/(.*?).pub', path) | 16:10 |
| SpamapS | my project name is 'something/something' | 16:10 |
| jeblair | SpamapS: that should be fine | 16:11 |
| * SpamapS never REALLY understands regexes until they work | 16:12 | |
| jeblair | SpamapS: you know about the encrypt_secret script? | 16:13 |
| jeblair | SpamapS: it should take arguments and construct the right url, etc | 16:13 |
| dmsimard | What happens if we reboot a node during a job ? Can we do that ? I guess the zuul console would need to be started... is there anything else ? | 16:19 |
| jeblair | dmsimard: should be possible. yeah, restarting zuul_console would be good. | 16:20 |
| SpamapS | jeblair: yes, it's not working | 16:21 |
| dmsimard | jeblair: I wonder if we should handle that through a systemd unit | 16:21 |
| jeblair | (though, strictly speaking, things should still work even without zuul_console running) | 16:21 |
| SpamapS | jeblair: which is why I'm looking at why my port 8001 is 404'ing on /keys | 16:21 |
| SpamapS | (8001 being where webapp is) | 16:21 |
| jeblair | dmsimard: not a bad idea, then there's less impact on playbooks | 16:21 |
| jeblair | SpamapS: keys are served through the old webapp | 16:22 |
| jeblair | which is probably 8001 | 16:22 |
| SpamapS | yeah same as status.json and such yes? | 16:22 |
| jeblair | just wanted to be clear, it's served from the scheduler-internal webapp, not the new zuul-web. | 16:22 |
| jeblair | SpamapS: yep | 16:22 |
| SpamapS | Yeah I'm adding comment='s to the HTTPNotFound's in webapp | 16:23 |
| jeblair | SpamapS: do you need to add or remove a tenant name? | 16:23 |
| SpamapS | so I know which 404 is firing | 16:23 |
| jeblair | like, in case you're proxying a level down or something | 16:23 |
| SpamapS | jeblair: it's not clear yet | 16:23 |
| SpamapS | and I'm not proxying /keys yet | 16:23 |
| SpamapS | wanted to see if it had a tenant component or not :) | 16:23 |
| SpamapS | so just trying to hit 127.0.0.1 from the scheduler | 16:24 |
| SpamapS | jeblair: ok, so it does seem to want ot have a tenant component... and when it does, I get a 500 | 16:28 |
| SpamapS | http://paste.openstack.org/show/625234/ | 16:29 |
| SpamapS | AttributeError: 'Project' object has no attribute 'public_key' | 16:29 |
| SpamapS | Oh I did find a doc bug, or a url bug, not sure which | 16:33 |
| SpamapS | http://127.0.0.1:8001/GoDaddy/keys/gd-github/cloudplatform/k8s-ansible.pem.pub | 16:33 |
| SpamapS | that's the URL needed | 16:33 |
| SpamapS | notice, not <project>.pub, but <project>.pem.pub | 16:33 |
| SpamapS | or that's a bug in the tool and the webapp that went hand in hand | 16:33 |
| jeblair | oh weird. we should fix that when we move this to zuul-web | 16:35 |
| SpamapS | So I am adding debug logging around, and the keys are loaded by the parser.. not sure why webapp later reports they don't have that attribute | 16:40 |
| SpamapS | hm that looks like a driver project model object, not a config object. | 16:42 |
| SpamapS | oh no it's even weirder | 16:47 |
| SpamapS | so, that .pem.pub is just a bug in the tools script I think | 16:47 |
| SpamapS | http://127.0.0.1:8001/GoDaddy/keys/gd-github/cloudplatform/project-config.pub works | 16:48 |
| SpamapS | with GoDaddy being the tenant, and gd-github being the source | 16:48 |
| leifmadsen | pabelanger: jeblair: back from lunch | 16:48 |
| SpamapS | oh no actually I'm just dumb, ok, the .pem thing was a red herring from a copy-paste-type-o | 16:49 |
| SpamapS | jeblair: all is well. Sorry for the confusion. We do have a 500 that should be a 404, so I'll submit a small patch for that. | 16:50 |
| SpamapS | Ugh, and I think I found a DoS bug in the /keys target too | 17:02 |
| *** openstackgerrit has quit IRC | 17:03 | |
| SpamapS | jeblair: https://github.com/openstack-infra/zuul/blame/458ab7bc90ecd3a0173831f81b351426c6fe302a/zuul/driver/gerrit/gerritsource.py#L49-L52 <--this lets anybody who can hit /keys fill up your memory with project objects. | 17:04 |
| SpamapS | we need some kind of "get project but only if it actually exists" ;) | 17:04 |
| *** openstackgerrit has joined #zuul | 17:06 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Improve error handling in webapp /keys https://review.openstack.org/517053 | 17:06 |
| * SpamapS just tries removing the 'add if not found' to see what breaks | 17:09 | |
| SpamapS | oh doh, like, everything | 17:10 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add additional launcher logging https://review.openstack.org/517057 | 17:12 |
| Shrews | I'd like to get this in ASAP, please ^^^ | 17:12 |
| pabelanger | looking | 17:13 |
| pabelanger | +2 | 17:13 |
| * SpamapS almost done with patch to shore up getProject problems | 17:17 | |
| *** hashar is now known as hasharAway | 17:35 | |
| *** hasharAway is now known as hasharDinner | 17:48 | |
| jeblair | Shrews: both lgtm | 18:00 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Add BaseSource.getProjectReadonly and refactor https://review.openstack.org/517067 | 18:06 |
| jeblair | 2017-11-01 18:08:20,993 ERROR zuul.GithubConnection: No installation ID available for project leifmadsen/dummy-commits | 18:10 |
| jeblair | SpamapS: we added an app to the leifmadsen org/account/whateveryoucall it to apply to all projects | 18:10 |
| jeblair | but still got that error | 18:10 |
| jeblair | SpamapS: any idea what could cause that? | 18:10 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Use >= in instance count comparison to max-servers https://review.openstack.org/516988 | 18:10 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Add additional launcher logging https://review.openstack.org/517057 | 18:10 |
| SpamapS | jeblair: did you wait 5 minutes? Because I don't trust github's API anymore. ;) | 18:11 |
| leifmadsen | ok :) | 18:11 |
| leifmadsen | no... we didn't wait maybe long enough... | 18:12 |
| SpamapS | It's like eating before swimming. | 18:12 |
| SpamapS | It may be a myth | 18:12 |
| leifmadsen | I love doing that! | 18:12 |
| SpamapS | but .. might as well wait and see if it helps. ;) | 18:12 |
| SpamapS | in all seriousness, I'm not sure why that doesn't work. | 18:12 |
| SpamapS | I have been relegated to github enterprise, so I don't get to use apps. | 18:12 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Unpause a declined request https://review.openstack.org/517068 | 18:13 |
| Shrews | jeblair: pabelanger: another issue i noticed ^^^ | 18:13 |
| jeblair | SpamapS, jlk, tobiash, leifmadsen: i'm looking at the code: https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/driver/github/githubconnection.py?id=feature/zuulv3#n516 | 18:17 |
| jeblair | that's where the error originates | 18:17 |
| jeblair | but i can't see how we're expected to have an installation key at all | 18:17 |
| jeblair | it kind of looks like that's only going to work *after* zuul has gotten an event from the project | 18:19 |
| SpamapS | jeblair: looks like it builds installation_map on events | 18:19 |
| jeblair | which is going to be hard at startup | 18:19 |
| SpamapS | agreed | 18:19 |
| SpamapS | jeblair: we may need to keep that state on disk. | 18:19 |
| pabelanger | jeblair: oh | 18:20 |
| pabelanger | what version of gitpython are you using? | 18:20 |
| leifmadsen | 0.9.3 | 18:20 |
| leifmadsen | I think | 18:20 |
| SpamapS | jeblair: so as a test of that theory, try reinstalling the app :) | 18:20 |
| leifmadsen | wait, that was before, let me check | 18:20 |
| pabelanger | rr | 18:20 |
| pabelanger | 1 sec | 18:20 |
| jeblair | SpamapS: while scheduler is running, presumably | 18:20 |
| * jlk tries to catch up | 18:20 | |
| SpamapS | jeblair: yeah | 18:20 |
| leifmadsen | 1.0.0a4 | 18:20 |
| leifmadsen | pabelanger: ^^ | 18:20 |
| pabelanger | leifmadsen: you need github3.py from master | 18:21 |
| leifmadsen | of course I do | 18:21 |
| jlk | I'm really sad they haven't released a new version yet | 18:21 |
| jlk | also tempted to stop waiting for graphql to be available to apps, and just rip out github3.py all together | 18:21 |
| pabelanger | leifmadsen: I had issues installing it with pip for some reason, and it fell back to download from pypi | 18:21 |
| jeblair | pabelanger: yeah, i think the same thing happened to us, but we've re-installed from master now | 18:24 |
| pabelanger | cool | 18:24 |
| jeblair | 1.0.0a4 is what zuulv3.o.o reports too, so i think that's good | 18:24 |
| jeblair | SpamapS: hrm.... the trick about your suggestion is that we can't get past the initial startup, so zuul is wedged | 18:25 |
| jeblair | okay, we are seeing this in openstack: | 18:26 |
| jeblair | 2017-11-01 14:24:38,161 ERROR zuul.GithubConnection: No installation ID available for project gtest-org/ansible | 18:26 |
| jeblair | so, somehow it's possible to get past this :) | 18:27 |
| jeblair | hrm, i wonder if this is a non-fatal error | 18:29 |
| jlk | Yeah I think you can get past it, if you don't get an installation ID I think it falls back to API key auth, or it drops the event. I'm not sure which | 18:29 |
| jeblair | we're getting this on 'getProjectBranches' | 18:30 |
| jlk | hrm, I don't like calling functions in a function argument. | 18:31 |
| jlk | (looking through this code) | 18:31 |
| jlk | so I guess it's a question of what github.login() does if it gets a token='' value | 18:31 |
| jeblair | yeah, it actually looks like we get past this just fine | 18:32 |
| jeblair | so maybe the bug here is in log severity :) | 18:32 |
| jlk | as others stated, the event itself is supposed to have an installation_id key within it, and when we handle said event, we put it into the installation map | 18:32 |
| jeblair | jlk: right. so this error should go away once events start happening. but on startup, we create a github object to get the project branches | 18:33 |
| jlk | Ooooh okay. yeah that's interesting. Probably okay, it just means we'll hit a different API limit | 18:34 |
| jeblair | but that will mean we have a lower rate limit? | 18:34 |
| jeblair | ya :) | 18:34 |
| jeblair | so we probably do want to do what SpamapS suggested and save these on disk to minimize having to rely on this | 18:34 |
| jlk | Are we hitting the limit at start up? | 18:34 |
| jeblair | 2017-11-01 18:31:55,464 DEBUG zuul.GithubConnection: GitHub API rate limit remaining: 54 reset: 1509563301 | 18:34 |
| jlk | ah okay | 18:34 |
| jeblair | that's what i just saw | 18:34 |
| jlk | I guess that would drain faster as we add more github based projects | 18:35 |
| jeblair | so probably okay in rare cases, but it won't scale too big | 18:35 |
| jlk | right, I don't think we thought about the start up case, where we iterate to see branches | 18:35 |
| jeblair | okay, 52 more chances to get this right before lunch :) | 18:36 |
| SpamapS | Pretty easy cache to maintain. | 18:36 |
| jeblair | SpamapS: ++ | 18:36 |
| jlk | it is, opens the door to other caches, like the cace of repo data | 18:36 |
| jlk | for both github and gerrit | 18:37 |
| jlk | where's the call to get the branches? | 18:37 |
| jeblair | jlk: getProjectBranches is the method... happens from configloader | 18:38 |
| jlk | ah blah | 18:39 |
| jlk | so that's within an iteration of 'for project in untrusted_projects' | 18:39 |
| jlk | With GraphQL we could do a neat thing where we do one query for all the branches of all our repos we want to know about (maybe some iterations if the list is larger than allowed in one query), but we'd have to know all the projects to query for. | 18:40 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Do not add invalid projets via the /keys API https://review.openstack.org/517078 | 18:41 |
| SpamapS | jeblair: ^^ DoS bug fix | 18:41 |
| SpamapS | jlk: that sounds pretty nice, and we should know all the projects to query for at startup and reconfig. | 18:42 |
| jlk | SpamapS: we do, it'd just require some code shuffling, do to one thing for github driver and another for all others | 18:42 |
| jlk | so, there is an API call we can hit to get a list of all our app installations. | 18:44 |
| jlk | We could maybe use that to prime the map at startup | 18:44 |
| * jlk ponders how that would work | 18:45 | |
| jlk | jeblair: SpamapS: if, at onLoad, we auth as the app, which will allow us to get a listing of our installations, we then use that listing to prime the installation_map, so that when we go to get branches for the repos we've likely already got the key in the map, thus we'll auth as the installation. Reasonable? | 18:50 |
| jeblair | jlk: ooh yeah, that sounds ideal | 18:54 |
| jlk | I'll whip up a patch | 18:55 |
| *** sambetts is now known as sambetts|afk | 19:02 | |
| *** openstackgerrit has quit IRC | 19:03 | |
| * jlk just spotted some stuff that will need to change once apps are supported in GHE | 19:07 | |
| jlk | our current apps code is hard coded for github.com | 19:07 |
| *** openstackgerrit has joined #zuul | 19:11 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Persist iptables rules https://review.openstack.org/513943 | 19:11 |
| *** hasharDinner has quit IRC | 19:31 | |
| leifmadsen | jlk: oh really? in theory there <github_connection>.server value should provide you a place to override "github.com" | 19:35 |
| jlk | yeah, except there is a hardcoded URL for some things that github3.py doesn't expose | 19:35 |
| leifmadsen | gotcha | 19:35 |
| *** hashar has joined #zuul | 19:36 | |
| *** electrofelix has quit IRC | 19:43 | |
| jlk | well this is interesting. Somehow I fucked up my deploy in that zuul can't contact the gearman it launched | 19:57 |
| leifmadsen | that sounds pretty neat... | 20:20 |
| jlk | oh got it | 20:21 |
| jlk | stupid. Forgot there's a bug in minikube in that a container cannot reach a service that's hosted on itself. | 20:21 |
| jeblair | jlk, SpamapS, tobiash: we still have the issue where if you turn on branch protection, you can't approve your own change, right? | 20:36 |
| jlk | If you turn on BP and require a review yes | 20:37 |
| jlk | as GH will not (yet) allow one to review ones own PR | 20:37 |
| jeblair | oh, heh, so you can turn on branch protection, but not enable any protections? | 20:37 |
| jlk | there are multiple protections | 20:38 |
| jeblair | yeah, i see 4: review, status, push, and "include admin" | 20:38 |
| jlk | you can require status, you can limit who can commit (regardless of reviews) | 20:38 |
| jlk | huh, bonnyci.org is failing to load. that's a github pages thing I thought | 20:39 |
| clarkb | it lacks dns records | 20:40 |
| jlk | SpamapS: hey there, you're hte owner for bonnyci.org, did something happen to the DNS for it? | 20:40 |
| jlk | jeblair: https://github.com/BonnyCI/bonnyci.org/blob/master/lore/end_users/setup/README.md#merge-options the last two sections here are of interest | 20:41 |
| jlk | oh except that doesn't cover PR reviews. Derp | 20:42 |
| jlk | blaahhhh. An installation may be a specific user, not necessarily a specific repository. | 20:45 |
| * jlk thinks more | 20:45 | |
| jeblair | jlk: well, my immediate concern was for leifmadsen's quickstart -- where we wanted to be able to submit a pr to a repo and have zuul run a check. | 20:46 |
| leifmadsen | I think we've worked around the issue though by making things less restrictive :) | 20:46 |
| jeblair | yeah, i think having branch protection on, but none of the extra restrictions will work for this | 20:46 |
| leifmadsen | I'll test real quick to confirm | 20:46 |
| jlk | what's the point of having BP on but no extra? what is the protection at that point? | 20:47 |
| jeblair | that lets us set "exclude-unprotected-branches" in the zuul config, which means we can push up a commit to a 'working' branch which is unprotected, and zuul will ignore that. | 20:47 |
| jeblair | then we can make a PR from 'working' to master | 20:47 |
| jeblair | jlk: apparently with branch protection alone, it blocks force-push and delete | 20:47 |
| jlk | ah, right. | 20:48 |
| jeblair | that seems like a pretty sensible base level of protection | 20:48 |
| jlk | Forgot about the exclude thing | 20:48 |
| jlk | I never make dev branches on a repo that would be getting CI done | 20:48 |
| jeblair | yeah, i wouldn't in prod, but it's really handy for single-user bootstrapping | 20:49 |
| leifmadsen | ++ | 20:49 |
| leifmadsen | jeblair: confirmed, all is well in the world now! | 20:49 |
| jeblair | yay | 20:49 |
| leifmadsen | I even got it to write a fail msg :D | 20:50 |
| jeblair | https://github.com/leifmadsen/dummy-commits/pull/2 | 20:50 |
| jeblair | nice :) | 20:50 |
| leifmadsen | (I forgot to rebase before commit) | 20:50 |
| leifmadsen | this is pretty sweet | 20:50 |
| jeblair | that's a PR from an unprotected branch (so zuul is ignoring it), to master (protected, and zuul is watching it) | 20:50 |
| leifmadsen | yea, that's pretty awesome | 20:51 |
| leifmadsen | I can see some use-cases for that already | 20:51 |
| pabelanger | cool | 20:51 |
| jeblair | jlk, tobiash, SpamapS: i have noticed that we perform reconfigurations when something is pushed to an unprotected branch. zuul still doesn't load configs from the branch, but it does the reconfig anyway. we can optimize that case and have zuul ignore a push event from an unprotected branch (if it's ignoring unprotected branches) | 20:51 |
| leifmadsen | so next up is some sort of "cowsay" ansible play that mostly does nothing, but at least runs on a remote node I guess | 20:51 |
| leifmadsen | this is why writing documentation is awesome :) look at all these bugs you can to fix! | 20:52 |
| leifmadsen | s/can to/get to | 20:52 |
| jeblair | ++ | 20:52 |
| leifmadsen | "awesome" | 20:53 |
| jlk | jeblair: sounds reasonable. Can you toss that on storyboard, and I'll pick it up when ready? | 20:53 |
| jeblair | jlk: yep | 20:53 |
| leifmadsen | ok, end of work day here basically, and kids are home, so I'm out for a bit | 20:53 |
| leifmadsen | jeblair: thanks again for all the time | 20:53 |
| jeblair | leifmadsen: thanks for all the documenting :) | 20:53 |
| leifmadsen | I haven't even started the documenting part! lol | 20:54 |
| leifmadsen | I'll see if I can at least get some of the quickstart stuff updated to look a bit more sane | 20:54 |
| leifmadsen | existing quickstart stuff* | 20:54 |
| *** dkranz has quit IRC | 20:56 | |
| jeblair | jlk: https://storyboard.openstack.org/#!/story/2001270 | 20:56 |
| jlk | thanks! | 20:57 |
| SpamapS | jlk: Name Server: NS1.SOFTLAYER.COM | 20:57 |
| SpamapS | Name Server: NS2.SOFTLAYER.COM | 20:57 |
| SpamapS | jlk: guessing they finally cleaned that one up :) | 20:57 |
| jlk | hah | 20:58 |
| SpamapS | I'll fix the GH pages A record | 20:58 |
| jlk | jeblair: okay, it appears we'll have to first get a list of installations, then for each installation, auth and get a list of repositories (projects) said installation covers. | 21:09 |
| jeblair | okay, finally sent out the roadmap email i've been trying to send since monday :) | 21:47 |
| jeblair | it is looking like 80 simultaneous builds for one of our executors is the sweet spot. so that's one vCPU for every 10 simultaneous builds. | 22:00 |
| jeblair | if folks haven't seen it recently, http://grafana.openstack.org/dashboard/db/zuul-status has useful and accurate information once again | 22:04 |
| tobiash | jeblair: I think I can look into the unprotected branch reconfig next week | 22:05 |
| jeblair | tobiash: okay, there's a story above ^ i'll let you and jlk negotiate for it :) | 22:06 |
| jeblair | thanks :) | 22:06 |
| SpamapS | oh neat | 22:11 |
| * SpamapS likes t3h graphs | 22:11 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Prime github app install map on connection load https://review.openstack.org/517121 | 22:29 |
| jlk | jeblair: SpamapS: https://review.openstack.org/517121 is the code to prime the mappings onLoad. Still needs tests, but I wanted y'all to see the code first. | 22:31 |
| *** Guest7 has joined #zuul | 22:33 | |
| *** Guest7 has quit IRC | 22:37 | |
| jeblair | jlk: ++ from a high level, that looks reasonable | 22:56 |
| jlk | okay, where the heck do we trigger a reconfigure... | 22:56 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Add BaseSource.getProjectReadonly and refactor https://review.openstack.org/517067 | 22:56 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Do not add invalid projets via the /keys API https://review.openstack.org/517078 | 22:56 |
| jeblair | jlk: it's also now the tiniest little jump to supporting "automatically add all installed repos as untrusted-repos" which would be a pretty good github UX | 22:56 |
| jeblair | jlk: in a test? you can call 'self.sched.reconfigure(self.config)' | 22:57 |
| jlk | no, I meant where in the zuul code does it decide that an event should cause a reconfigure | 22:57 |
| jeblair | jlk: that will do a 'full' reconfiguration (no cached data) | 22:57 |
| jeblair | jlk: oh heh | 22:57 |
| jeblair | jlk: https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/scheduler.py?id=feature/zuulv3#n846 | 22:58 |
| jlk | jeblair: re tiny jump, yeah that would be pretty neat, but it would only work on start up? or on driver reload events. maybe? | 22:58 |
| jeblair | jlk: i think there's a registration hook we're ignoring right now? | 22:59 |
| jlk | ahhhh, 'branch_update' is the thing. | 22:59 |
| jlk | jeblair: oh true, we do ignore a new install happening event, because we thought the app owner (eg bonnyci) should handle that instead of zuul, but maybe not! | 22:59 |
| jlk | (branch_updated that is) | 23:00 |
| jeblair | yeah, i think either is a reasonable choice depending on circumstances | 23:00 |
| jeblair | and, i mean, we can also do that for gerrit, but haven't yet. :) | 23:00 |
| jlk | I'll cook on that idea for a bit | 23:00 |
| jlk | re branch_updated, we get a 'push' event from github, and that event has all the details we need (thus far) to generate the event. We don't query the API for any more details, such as the repo branch protection status. Would have to add another query here, more API use. | 23:02 |
| SpamapS | I would love it if install of app meant that the repo was added. | 23:08 |
| SpamapS | (once I have apps) | 23:08 |
| SpamapS | And maybe some rules for sorting them into tenants or white/blacklisting repos that misbehave. | 23:08 |
| jeblair | jlk: we do have a cached list of branches in Project.unparsed_branch_config. we should be able to consult that to know to ignore a push to any change not in there, as long as we detect creating a new branch as something separate. | 23:11 |
| jeblair | er, "ignore a push to any *branch* not in there" rather | 23:12 |
| jeblair | i'm just doing word association at this point in the day | 23:12 |
| jlk | heh | 23:12 |
| jlk | yeah, I'm typing some notes into the story, rather than coding it. | 23:13 |
| jlk | Story updated with some notes | 23:25 |
| *** hashar has quit IRC | 23:27 | |
| jamielennox | anyone here coming to sydney? | 23:31 |
| SpamapS | wow.. good job Mac OS being ancient | 23:35 |
| SpamapS | $ openssl version | 23:35 |
| SpamapS | OpenSSL 0.9.8zh 14 Jan 2016 | 23:35 |
| SpamapS | jamielennox: sadly I'm out for this one. :( | 23:35 |
| SpamapS | was looking forward to seeing .au but I couldn't justify it given new position. | 23:35 |
| jeblair | jamielennox: i am, mordred, clarkb, fungi, jhesketh at least. ianw is organizing an infra evening thing if you want to join: http://lists.openstack.org/pipermail/openstack-infra/2017-October/005653.html | 23:35 |
| jamielennox | unfortunately i don't expect many people | 23:35 |
| jeblair | oh and pabelanger | 23:36 |
| jamielennox | that's useful, i should be able to get along to that | 23:37 |
| jeblair | yay! | 23:37 |
| clarkb | I'm waiting for my plane to show up so I can board | 23:37 |
| SpamapS | jeblair: so, FYI, encrypt_secret doesn't work on OS X | 23:39 |
| SpamapS | because the OpenSSL version is so old 'n busted | 23:39 |
| * SpamapS has fix | 23:41 | |
| jeblair | SpamapS: wow. thanks. yeah, my intent is for that to work anywhere (that's why it's super simple py27 builtins, and what i thought was pretty unexciting openssl :|) | 23:46 |
| jlk | at some point I think I brewed myself a newer openssl | 23:49 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Make encrypt_secret.py work with OpenSSL 0.x https://review.openstack.org/517133 | 23:50 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Prime github app install map on connection load https://review.openstack.org/517121 | 23:50 |
| SpamapS | jeblair: yeah, I like the work-anywhere of it. :) | 23:50 |
| jlk | oh turns out that the installation ID stuff and login via app isn't covered by any tests right now at all, so not oging to block on that to get this in | 23:52 |
| jlk | although now I"d love some ideas on how to actually test it :/ | 23:52 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!