| mordred | gundalow: zuul.branch is branch, zuul.project contains information about the project that triggered the job, but of course remember that a job can have multiple repos associated with it, so there's also zuul.projects which contains info about all projects involved in the job | 00:00 |
|---|---|---|
| mordred | gundalow: there can also be more than one commit involved in a collective git repo state, so you'll find that info in zuul.items | 00:01 |
| gundalow | mordred: Ace, thanks | 00:08 |
| tristanC | corvus: aren't project-templates configuration already expanded in the project_configs pipeline list? | 00:20 |
| *** eumel8 has quit IRC | 00:54 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 03:10 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 03:53 |
| *** goern has joined #zuul | 04:21 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 04:27 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 04:39 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 04:55 |
| *** goern has quit IRC | 04:55 | |
| *** goern has joined #zuul | 04:56 | |
| *** eumel8 has joined #zuul | 05:59 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 06:01 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 06:20 |
| *** pcaruana has joined #zuul | 06:27 | |
| *** jpena|off is now known as jpena | 07:46 | |
| *** AJaeger is now known as AJaeger_ | 07:50 | |
| *** darkwisebear has joined #zuul | 08:23 | |
| *** electrofelix has joined #zuul | 08:38 | |
| *** darkwisebear has quit IRC | 08:58 | |
| *** ssbarnea has quit IRC | 08:59 | |
| openstackgerrit | Markus Hosch proposed openstack-infra/zuul master: WIP: Unexpected behavior of a reject clause https://review.openstack.org/589762 | 08:59 |
| *** darkwisebear has joined #zuul | 09:05 | |
| *** gtema has joined #zuul | 09:05 | |
| openstackgerrit | Markus Hosch proposed openstack-infra/zuul master: WIP: Unexpected behavior of a reject clause https://review.openstack.org/589762 | 09:08 |
| *** ssbarnea has joined #zuul | 09:13 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Ensure build.start_time is defined https://review.openstack.org/535549 | 09:20 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: timer: do not skip projects using pipeline from template https://review.openstack.org/589836 | 10:50 |
| tristanC | corvus: AJaeger_: 589836 seems to fix the issue, the new test fail without the timer fix | 10:50 |
| AJaeger_ | tristanC: thanks! | 10:52 |
| AJaeger_ | tristanC: small fix - and large test case ;) Good to have that! | 10:54 |
| *** darkwisebear has quit IRC | 11:16 | |
| *** darkwisebear has joined #zuul | 11:23 | |
| *** jpena is now known as jpena|lunch | 11:35 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool master: Do not abort node launch if failed node cannot be deleted https://review.openstack.org/589854 | 11:43 |
| *** gtema has quit IRC | 11:46 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Add request reference when hitting a node failure https://review.openstack.org/589856 | 11:59 |
| *** panda|rover is now known as panda|ruck | 12:25 | |
| *** jpena|lunch is now known as jpena | 12:35 | |
| *** gtema has joined #zuul | 13:13 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: web: add tenant-scoped, JWT-protected actions https://review.openstack.org/576907 | 13:36 |
| *** gtema has quit IRC | 13:36 | |
| *** Shrews has quit IRC | 14:09 | |
| *** Shrews has joined #zuul | 14:10 | |
| *** nchakrab has joined #zuul | 14:18 | |
| *** AJaeger_ is now known as AJaeger | 14:20 | |
| corvus | tristanC: thanks! | 14:26 |
| *** gtema has joined #zuul | 14:26 | |
| *** nchakrab has quit IRC | 14:28 | |
| AJaeger | any other zuul cores to review tristanC's change https://review.openstack.org/589836 , please? We need this for all periodic jobs, especially for translation import | 14:34 |
| tobiash | lgtm | 14:42 |
| AJaeger | thanks! | 14:43 |
| corvus | gundalow: mordred's irc documentation of the zuul vars was great, of course, but in case you're interested, we also have a version with more words: https://zuul-ci.org/docs/zuul/user/jobs.html#zuul-variables :) | 14:48 |
| gundalow | corvus: Ace, thanks | 14:49 |
| darkwisebear | corvus: I've seen some surprising behavior if one uses reject clauses with Gerrit: If a reject clause is set, the change is ignored as long as no approval at all is there. If you then set some unrealted approval, the clause works as expected. Is there a reason for this behavior? See https://review.openstack.org/589762 for an example. | 14:50 |
| corvus | gundalow: since you asked about the commit -- you may also be interested in this bit: https://zuul-ci.org/docs/zuul/user/jobs.html#git-repositories | 14:54 |
| corvus | gundalow: aspecially the bit about the "origin" remote | 14:54 |
| gundalow | corvus: Oh, yup, thanks :) | 14:57 |
| corvus | darkwisebear: good question. i don't think we currently have any reject clauses in openstack, so that may explain why i haven't noticed the edge case. i'll think about that. | 14:58 |
| corvus | darkwisebear: my first reaction is to agree with you that that is surprising and probably a bug. but i'll think about it some more. :) | 14:59 |
| darkwisebear | corvus: Alright, just comment on the change if you're positive about changing the behavior to the suggested one and I'll add the impl change | 15:00 |
| openstackgerrit | Merged openstack-infra/zuul master: timer: do not skip projects using pipeline from template https://review.openstack.org/589836 | 15:07 |
| *** darkwisebear has quit IRC | 15:11 | |
| *** darkwisebear has joined #zuul | 15:15 | |
| *** pcaruana has quit IRC | 15:34 | |
| *** darkwisebear has quit IRC | 15:36 | |
| *** jlvillal is now known as jlv-sick | 15:40 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: WIP: cache branches in connections/sources https://review.openstack.org/589975 | 17:26 |
| corvus | tobiash: can you take a look at that ^ and let me know if you see any problems with the approach? | 17:27 |
| corvus | tobiash: in short, we shouldn't cache branches in the tenant, since branches are global (so a system with the same project in a lot of tenants shouldn't need to query the branch list for every tenant). but we can't store the cache in the abide, because exclude_unprotected_branches is a tenant configuration, so the abide can't know what to do with that. the only place i can find that has all the right | 17:30 |
| corvus | information is the source object (and its friend, the connection). so that puts the cache at that level. | 17:30 |
| corvus | clarkb: ^ you may also be interested. or you may be fishing. i guess those aren't exclusive though. | 17:32 |
| *** electrofelix has quit IRC | 17:44 | |
| tobiash | corvus: cool, just took a short look and the approach looks like the same approach I had in mind | 17:45 |
| tobiash | corvus: just curious, why did you choose two caches over one cache with attached protection info? | 17:46 |
| tobiash | corvus: as an improvement I think we should check and update the branch protection if we get a pull request event for a branch that is unprotected in the cache. That would work around the missing branch protection event from github. | 17:48 |
| tobiash | and if that changes the protection we would need to emit a branch created event to trigger a tenant reconfig | 17:48 |
| tobiash | but this is something for a followup | 17:48 |
| tobiash | corvus: I will be out of office for three weeks starting on monday | 17:49 |
| corvus | tobiash: different tenants might have different values for exclude_unprotected. if we have a single cache with protection info, and it contains a project which only has protected branches, and we get a request to *include* unprotected branches, how do we know if the cache has the values, or whether we only previously received requests for *exclude* unprotected? | 17:50 |
| corvus | tobiash: if we always queries for all branches, then we would know we had all the data and it would be okay | 17:50 |
| corvus | tobiash: but i was assuming that we also wanted to optimize for the case where all of the tenants with a project only have exclude_unprotected set (and therefore don't need to query, say, 4000 open pull request branches or something) | 17:51 |
| tobiash | corvus: I meant the other option is to filter within getprojectbranches. I think both options are fine and the two caches method avoids some list copying in getprojectbranches | 17:52 |
| corvus | tobiash: right, but filtering within getprojectbranches only works if the query to github is to include all branches | 17:53 |
| corvus | tobiash: so if we want to keep the optimization that if we only ever ask github for protected branches if that's what we want, then i think we need something like this | 17:53 |
| tobiash | corvus: ah right, I missed that | 17:53 |
| corvus | tobiash: if that's not important, and it would be okay to ask github for all branches, i agree, that would be the better approach | 17:54 |
| tobiash | so with a single cache we would need to query all branches and then query the branch protection for each branch | 17:54 |
| tobiash | so indeed the two cache method has a huge advantage there | 17:55 |
| tobiash | corvus: what do you think about the branch checking on pull request for unprotected branch? | 17:55 |
| tobiash | corvus: that could give us the possibility to automatically pick up new protected branches in I think at least most relevant use cases | 17:56 |
| corvus | tobiash: yeah, sounds promising | 17:56 |
| tobiash | right now we need to do a full reconfiguration to pick up new protected branches | 17:56 |
| *** jpena is now known as jpena|off | 18:07 | |
| tobiash | corvus: I noticed that nodepool gets slower if there are many nodes and I think I found one use of zk.nodeIterator which is on the critical path for every node: http://git.zuul-ci.org/cgit/nodepool/tree/nodepool/driver/openstack/provider.py#n150 | 18:19 |
| tobiash | that iterates over all nodes and I think basically has a cost of one network rtt per registered node | 18:19 |
| tobiash | corvus: I think with some care we could replace that with an atomic counter that gets increases/decreased on node creation deletion | 18:21 |
| tobiash | https://kazoo.readthedocs.io/en/latest/api/recipe/counter.html | 18:21 |
| corvus | tobiash: or we could use zk callbacks to mirror znodes to in-memory objects | 18:21 |
| tobiash | or that | 18:22 |
| corvus | tobiash: you're thinking a counter for each resource? | 18:22 |
| tobiash | for each node type | 18:23 |
| tobiash | but I think the callback method is better | 18:23 |
| tobiash | with the callback method we could eliminate the second full iteration on the critical path too: http://git.zuul-ci.org/cgit/nodepool/tree/nodepool/driver/__init__.py#n416 | 18:24 |
| corvus | tobiash: yeah, the callback has other advantages (it makes improvements like this more generally available) | 18:24 |
| tobiash | corvus: I think we need that as this limits scalability of nodepool. Maybe some topic for after my vacation. | 18:25 |
| Shrews | but makes handling session disconnects more complex, or more important rather | 18:25 |
| tobiash | Shrews: yes so we would need to handle the session events and refresh everything on reconnect | 18:26 |
| tobiash | or maybe just invalidate the in memory cache | 18:27 |
| tobiash | on disconnects | 18:27 |
| Shrews | you'd also have to re-register the callbacks | 18:28 |
| tobiash | do we also loose callbacks if reconnect was successful without loosing the session? | 18:28 |
| Shrews | i don't think so | 18:28 |
| Shrews | a lot like locks, iirc | 18:29 |
| corvus | yeah, it's definitely a different approach, but i think we grok zk enough to look into it now :) | 18:30 |
| tobiash | I think we need connection awareness anyway: https://kazoo.readthedocs.io/en/latest/basic_usage.html#understanding-kazoo-states | 18:30 |
| tobiash | > When a connection transitions to SUSPENDED, if the client is performing an action that requires agreement with other systems (using the Lock recipe for example), it should pause what it’s doing. | 18:30 |
| Shrews | tobiash: that's why i have all those "sleep-while-suspended" loops | 18:32 |
| Shrews | other places, we just handle any resulting errors (like lost locks) as gracefully as possible | 18:32 |
| tobiash | ah ok | 18:35 |
| tobiash | looks like kazoo already has a ready recipe for caching a tree: https://kazoo.readthedocs.io/en/latest/api/recipe/cache.html | 18:41 |
| tobiash | so we probably should not re-implement that outselfs :) | 18:41 |
| corvus | i love the recipes :) | 18:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: WIP: cache branches in connections/sources https://review.openstack.org/589975 | 20:14 |
| *** pcaruana has joined #zuul | 21:07 | |
| *** pcaruana has quit IRC | 22:02 | |
| *** adam_g has quit IRC | 23:02 | |
| *** mhu has quit IRC | 23:02 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 23:31 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 23:36 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 23:42 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: [WIP] kerberos & afs client roles https://review.openstack.org/589334 | 23:47 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!