| *** nilashishc_ has joined #zuul | 04:55 | |
| *** nilashishc_ is now known as nilashishc | 04:57 | |
| *** bhavikdbavishi has joined #zuul | 05:29 | |
| *** pcaruana has joined #zuul | 06:21 | |
| *** chkumar|off is now known as chandankumar | 06:45 | |
| *** themroc has joined #zuul | 06:56 | |
| *** pcaruana has quit IRC | 06:58 | |
| *** pcaruana has joined #zuul | 07:13 | |
| *** jpena|off is now known as jpena | 07:32 | |
| *** panda has joined #zuul | 08:13 | |
| *** sshnaidm|off is now known as sshnaidm|ruck | 08:22 | |
| *** cbx33 has joined #zuul | 08:40 | |
| cbx33 | Hi all, been reading the docs a little today, still a little confused, can it only be used for merging code if you use Gerrit or GitHub? | 08:40 |
|---|---|---|
| ssbarnea | can zuul work with pure py3 nodes?(one that do not have py2 and python command on them). | 09:03 |
| *** electrofelix has joined #zuul | 09:27 | |
| tobiash | ssbarnea: it currently hard codes ansible_python_interpreter to /usr/bin/python2 (not sure if this could be changed now with recent ansible) | 09:52 |
| cbx33 | tobiash, Do you know anything about non github/gerrit repos? | 10:03 |
| tobiash | cbx33: currently only gerrit/github is supported | 10:04 |
| cbx33 | tobiash, what does that mean for other git repos? | 10:06 |
| cbx33 | Can Zuul do anything? | 10:06 |
| cbx33 | tobiash, The documentation doesn't seem to say that you can only merge code if you are using GitHub or Gerrit, but that is what I am guessing is correct, am I right? | 10:08 |
| cbx33 | if so, what is the git driver for? | 10:08 |
| tobiash | cbx33: the git driver is only for fetching code | 10:09 |
| cbx33 | OK - Thank you for clearing it up | 10:09 |
| tobiash | cbx33: what service are you using? | 10:09 |
| cbx33 | I'm using GitLab | 10:09 |
| tobiash | I think I've heard of poeple wanting to add support for gitlab | 10:10 |
| goern | that would be a very interesting addition :) | 10:10 |
| cbx33 | It would be a very awesome addition | 10:10 |
| cbx33 | I currently have about 8-9 interdependant projects which need to remain internal currently, and only having access to GL gives me a bit of a problem | 10:11 |
| cbx33 | Hmmm, OK gives me something to think about - thank you | 10:13 |
| *** rfolco|rover has joined #zuul | 10:14 | |
| cbx33 | tobiash, Is there documentation on what methods a driver needs to provide? | 10:19 |
| *** jpena is now known as jpena|lunch | 11:23 | |
| *** pcaruana has quit IRC | 11:26 | |
| *** pcaruana has joined #zuul | 11:48 | |
| *** panda is now known as panda|lunch | 11:55 | |
| *** bhavikdbavishi has quit IRC | 12:06 | |
| *** pcaruana has quit IRC | 12:12 | |
| *** pcaruana has joined #zuul | 12:12 | |
| *** jpena|lunch is now known as jpena | 12:23 | |
| mordred | tobiash, ianw, Shrews: https://review.openstack.org/#/c/612168/ looks good now - the src job is failing as expected and the non-src are passing - and then https://review.openstack.org/#/c/612186/ which depends-on it is the fix in sdk and it makes the src job green again | 12:27 |
| mordred | cbx33: caphrim007 is the other person looking at gitlab driver, but he doesn't seem to be in channel at the moment | 12:27 |
| mordred | cbx33: http://git.zuul-ci.org/cgit/zuul/tree/zuul/driver/__init__.py has the interface definitions, but I don't think that's the whole story. if I were starting in on the gitlab driver I'd likely start by looking at http://git.zuul-ci.org/cgit/zuul/tree/zuul/driver/github - since gitlab and github are conceptually similar and I believe the gitlab webhooks and api are similar to the github webhooks / api | 12:29 |
| tobiash | clarkb: will there be further zuul stickers in berlin? | 12:30 |
| mordred | tobiash: I believe so - we had a pile in austin and I'm taking a pile to edinburgh this week | 12:34 |
| tobiash | mordred: cool :) | 12:35 |
| *** rlandy has joined #zuul | 12:35 | |
| Shrews | mordred: maybe it's lack of caffeine, but isn't that depends on backwards? | 12:44 |
| mordred | Shrews: I don't think so? I did the 'fix-test' patch first, since it's non-voting, so we can see that the sdk patch fixes the failure | 12:45 |
| *** jesusaur has quit IRC | 12:46 | |
| mordred | Shrews: sorry, since the nodepool-src job is nonvoting for nodepool - so it's ok that it's red for that patch (that's how we verify that the test job itself is doing the right thing) | 12:47 |
| Shrews | mordred: oh, yeah, nm | 12:47 |
| *** panda|lunch is now known as panda | 12:56 | |
| mordred | Shrews: lemme tell you how much I've enjoyed digging in to that particular issue | 13:00 |
| Shrews | mordred: it didn't add enjoyment to your weekend?? odd | 13:00 |
| openstackgerrit | Merged openstack-infra/zuul master: Handle missing node during hold check https://review.openstack.org/610969 | 13:08 |
| *** jesusaur has joined #zuul | 13:16 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Make functional src jobs actually install from source https://review.openstack.org/612168 | 13:58 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 14:04 |
| *** themroc has quit IRC | 14:14 | |
| *** themroc has joined #zuul | 14:17 | |
| *** mestery has left #zuul | 14:34 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Cleanup down ports https://review.openstack.org/609829 | 14:44 |
| *** pcaruana has quit IRC | 15:12 | |
| *** themroc has quit IRC | 15:24 | |
| *** bjackman has joined #zuul | 15:35 | |
| bjackman | Hi folks. I've been looking into setting Zuul up for the org I work at, it seems pretty well-suited except for one thing: we sometimes have cross-repository dependencies that are two-way, i.e. the changes need to be applied in lockstep. Do I understand correctly that Zuul does not handle that case? | 15:38 |
| openstackgerrit | Merged openstack-infra/nodepool master: Remove sqlalchemy from requirements https://review.openstack.org/611821 | 15:50 |
| clarkb | bjackman: currently that is correct. I think there has been talk about supporting "loops" and making that a configurable otpion in zuul as to whether or not they are acceptable but that work hasn't happened | 15:51 |
| bjackman | clarkb: OK, thanks. I wonder if there are any common workarounds? | 15:52 |
| bjackman | I suspect that a common response would be "if you often have two-way dependencies between two repos then you should probably combine those repos into one" | 15:52 |
| bjackman | Which isn't unreasonable | 15:52 |
| clarkb | bjackman: we (openstack using zuul) highly encourage people to write changes that are forward/backward compatible. In this particular case it aids with people that may want to continuously deploy the software as they have a migration path as they step through | 15:52 |
| clarkb | bjackman: that would be another possible workaround as it does indicate strong coupling | 15:53 |
| bjackman | Oh and another thing I wasn't immediately clear about - does Zuul always test every commit, or can it be persuaded to consider e.g. a Gerrit changeset as a single unit? | 15:54 |
| clarkb | That depends on the review system. With Gerrit each change is considered. With github then entire PR is considered | 15:55 |
| bjackman | Ah OK thanks | 15:56 |
| clarkb | it is possible that making the gerrit driver configurable to handle a change series vs change as the unit would be worthwhile | 15:56 |
| clarkb | (I'm not sure haven't used the newer change series model with gerrit) | 15:57 |
| bjackman | It would be a good way to reduce pressure on test resources | 15:57 |
| bjackman | In my case the tests need special HW which is quite scarce | 15:57 |
| *** bhavikdbavishi has joined #zuul | 15:59 | |
| tobiash | mordred: does that mean that now there is a sdk release that contains the parallel execution and nat source? | 16:09 |
| corvus | howdy! i should be around for a while now | 16:20 |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 16:27 | |
| *** bjackman has quit IRC | 16:38 | |
| *** bjackman has joined #zuul | 16:38 | |
| corvus | tristanC: not sure if you saw, but i think https://review.openstack.org/535557 is waiting for you to take a look at the functional tests now | 16:50 |
| corvus | tobiash, Shrews: i'll try to take a look at the zk caching stuff this week | 16:50 |
| tobiash | corvus: cool, thanks :) | 16:51 |
| *** jpena is now known as jpena|off | 16:55 | |
| Shrews | SpamapS: LinkedIn tells me interesting things about you | 17:20 |
| SpamapS | Shrews: lies | 17:24 |
| SpamapS | Shrews: if you're at all worried that I may be disappearing.. we already have a Zuul that runs on AWS .... ;-) | 17:27 |
| *** pcaruana has joined #zuul | 17:27 | |
| SpamapS | I've got a few AWS-centric zuul-jobs patches in the works | 17:28 |
| Shrews | SpamapS: i never worry about you... though i often wonder about you. and neat! | 17:28 |
| SpamapS | For things like pushing docker images to ECR and logging to S3. | 17:28 |
| * SpamapS sees bjackman and reads it as bojack horseman | 17:29 | |
| *** jlk has quit IRC | 17:29 | |
| Shrews | is it weird that i read it the same way? | 17:29 |
| *** jlk has joined #zuul | 17:30 | |
| SpamapS | clarkb: bjackman FYI, we used gerrit with merge commits at IBM and Zuul tested the "series" that way. | 17:30 |
| SpamapS | OpenStack's gerrit just has merge commits turned off, IIRC. | 17:31 |
| clarkb | SpamapS: we only enable it for the use case of maintaining feature branches and merging them back to master in openstack land, But that should work just fine if you use it as your normal workflow | 17:32 |
| SpamapS | right | 17:32 |
| corvus | oh neat, we should add that to the faq. when we write the faq. | 17:36 |
| bjackman | SpamapS: Oh that's interesting, cheers | 17:37 |
| *** electrofelix has quit IRC | 17:44 | |
| *** nilashishc has quit IRC | 17:55 | |
| *** pcaruana has quit IRC | 17:57 | |
| *** bhavikdbavishi has quit IRC | 18:07 | |
| *** bhavikdbavishi has joined #zuul | 18:08 | |
| dmsimard | SpamapS: don't wait for me if you want to work on the ec2 nodepool patch(es) | 18:08 |
| mordred | tobiash: yes, I think so. I'm going to try to get the rest of the task manager related nodepool patches and sdk patches finished up this week | 18:09 |
| mordred | tobiash: oh - well - there will be as soon as 0.18.1 finishes releasing | 18:17 |
| mordred | tobiash: 0.18.0 is broken with nodepool | 18:18 |
| tobiash | mordred: so I should wait until 0.18.1 before rebuilding nodepool? | 18:18 |
| mordred | tobiash: yes. 0.18.0 will break spectacularly | 18:18 |
| tobiash | Cool | 18:18 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Support node caching in the nodeIterator https://review.openstack.org/604648 | 18:20 |
| * tobiash tries to imagine how breaking nodepool spectacularly could look like | 18:20 | |
| Shrews | tobiash: ^^^ we needed a test to validate caching was working | 18:20 |
| tobiash | Shrews: thanks, I didn't add a test for the caching as I had no idea how to test this and there is no functional change | 18:22 |
| Shrews | the test was actually pretty simple | 18:23 |
| Shrews | (just testing the zk portion of it and not nodepool operations) | 18:23 |
| tobiash | Shrews: I think zhat test will race because the cache is updated asynchrounously using watches | 18:25 |
| Shrews | tobiash: oh. well that might be problematic in places | 18:26 |
| tobiash | So we might need some synchonization there or just sleep a second | 18:26 |
| tobiash | Shrews: that's why I enabled the cache not everywhere | 18:27 |
| Shrews | hrm, i'll have to look at your changes with that perspective | 18:27 |
| tobiash | It guarantees that the nodes are there (getting the list of nodes is uncached) | 18:28 |
| tobiash | But the content of a node might be slightly outdated because the asynchrounous update | 18:28 |
| Shrews | tobiash: does having a lock on a node affect updating the cached version at all? | 18:30 |
| Shrews | like, maybe the cache will not update while the lock is held? | 18:30 |
| mordred | tobiash: 'spectacularly' means tracebacks about not being able to start threads twice and nodepool being unable to do anything at all | 18:30 |
| mordred | :) | 18:30 |
| tobiash | Shrews: everywhere where we lock a node we must/do get an updated node anyway | 18:31 |
| tobiash | And where we do that are the only places where I enabled it (mostly on the critical path of request handling) | 18:32 |
| Shrews | tobiash: those words confuse me | 18:33 |
| tobiash | Let me switch to laptop, that might make my thoughts easier to express | 18:34 |
| Shrews | now i'm curious how node locking works since that is stored as part of the node | 18:36 |
| tobiash | Shrews: our node locking is kind of strange atm, it creates an ephemeral node to express the lock but the lock itself is only locally so if you do a getNode you don't know if we have the lock | 18:37 |
| tobiash | but that's independent of the caching | 18:38 |
| Shrews | tobiash: i don't think that's correct. node locks are part of the actual znode. | 18:39 |
| tobiash | so our typical workflow in many places is to try locking the node, if success get the node again (without cache) and work with that | 18:39 |
| Shrews | node requests work differently | 18:39 |
| tobiash | at least getNode doesn't initialize the local lock object of the node | 18:42 |
| SpamapS | dmsimard: ACK, I won't wait. :) | 18:45 |
| SpamapS | Though now that I have like, full time and not outside-biz-hours time to spend on it, I may not even end up on AWS. ;) | 18:46 |
| tobiash | Shrews: this is what I mean: https://git.zuul-ci.org/cgit/nodepool/tree/nodepool/launcher.py#n515 | 18:46 |
| tobiash | iterate over nodes, lock, re-get the node to check if the data is still as expected, do something with it | 18:47 |
| tobiash | Shrews: this pattern should be safe to operate on cached data | 18:47 |
| Shrews | tobiash: yes, but that was not the question i'm trying to answer. Our node locks do not use ephemeral nodes. They are part of the actual znode we are locking. If *that* information is also given to the cache asynchronously, what happens if two launchers try to lock the same node at the same time? Does the cache play any role in that? | 18:55 |
| Shrews | I'm assuming it works correctly, but I'd like to confirm to make sure | 18:56 |
| Shrews | i can probably hack a test script together | 18:56 |
| tobiash | Shrews: that information is not initialized in the getNode method (it deserializes from json) and that behavior is unchanged | 18:56 |
| tobiash | Shrews: this is the relevant part: https://git.zuul-ci.org/cgit/nodepool/tree/nodepool/zk.py#n601 | 18:57 |
| tobiash | so we don't get the lock if we do getNode. The cache is similar like the view of a different launcher | 18:58 |
| tobiash | Shrews: and lockNode works path based: https://git.zuul-ci.org/cgit/nodepool/tree/nodepool/zk.py#n1653 | 18:59 |
| tobiash | so independent of any cache | 19:00 |
| tobiash | but yes, a script might prove it :) | 19:01 |
| Shrews | i don't think i'm expressing my concern well. let me think about this a bit | 19:04 |
| tobiash | it's probably just too late here ;) | 19:05 |
| tobiash | Shrews: so if I understand correctly you mean that the lock is part of the znode and thus part of the cached data right? | 19:05 |
| *** bhavikdbavishi has quit IRC | 19:06 | |
| tobiash | I just double checked, the lock is its own child znode: https://git.zuul-ci.org/cgit/nodepool/tree/nodepool/zk.py#n725 | 19:07 |
| *** cbx33 has quit IRC | 19:34 | |
| *** rlandy has quit IRC | 19:35 | |
| AJaeger | corvus: could you check the comments by ianw on https://review.openstack.org/#/c/592850/1 again, please? is you -1 still valid? | 19:38 |
| AJaeger | dmsimard: want to remove your WIP from https://review.openstack.org/#/c/610381/ ? | 19:39 |
| AJaeger | corvus, some reviewers liked you to review https://review.openstack.org/#/c/607691/ - could you check that one as well, please? | 19:40 |
| *** rlandy has joined #zuul | 19:43 | |
| dmsimard | Removed | 19:44 |
| AJaeger | thanks | 19:44 |
| corvus | AJaeger, ianw: so, i (re)wrote the current version of the swift upload logs role. ianw is significantly rewriting it in that change. i can't really follow the code anymore with that change; i think ianw and i think about that code differently. i don't think i will be able to help maintain that role if we merge the change. so if you are sufficiently enthusiastic about taking over responsibility for that | 19:48 |
| corvus | role to proceed, i'm happy to get out of the way. | 19:48 |
| AJaeger | corvus: I let ianw followup - wanted to unblock the discussion only. | 19:49 |
| corvus | AJaeger, ianw, SpamapS: also responded on 607691. | 19:52 |
| AJaeger | thanks, corvus | 19:52 |
| ianw | corvus: 4/6th of that stack is in aid of not leaving temporary directories, which i'd count as a bug, 1/6th is making sure unicode paths are tested, and 1/6th is the download-all-logs feature, which i wrote because to keep parity with a similar feature i've written for the extant role, but was not accepted because it was felt we were switching to swift logs "soon" and didn't want feature disparity | 20:27 |
| ianw | so i dunno. if you'd like to implement your approach to fix up the bugs, i'm sure i can work around it the feature i'd like | 20:28 |
| corvus | ianw: it's probably the tempdirs thing that's causing the disconnect. i don't feel like it's an important enough bug to warrant the additional complexity and am okay with that for now. if you want to drop the first 2/3 and just do the last 1/3 i think i can engage with that. if you think the first 2/3 is very important, then i don't think i'm going to have time to come up with an independent solution to | 20:54 |
| corvus | that in the next... 2 months? so we could leave the series in stasis until then, or i could get out of the way. :) | 20:54 |
| ianw | the tempdirs are quite annoying when you're running the unit tests | 21:08 |
| ianw | the "upload-logs" role, however, really relies on the prior changes. it lives "inside" the FileList context, same as the index files. that's essentially why i took the approach i think you have the issue with -- because it makes a clear context you can "do things" to the file list | 21:12 |
| SpamapS | corvus: ty, pretty easy to convert markdownlint into a role | 21:20 |
| corvus | SpamapS: w00t | 21:20 |
| SpamapS | 90% git mv. ;) | 21:21 |
| corvus | ianw: ok, i don't think i can engage on alternate suggestions, but i don't think it's fair for me to stand in your way, so as long as you and other folks are happy to go forward in that way, i'm happy to pass the baton to you on that :) [to be clear, i'm happy to continue to make/review minor bugfixes to the new code, just that i can't really "own" it for whatever that's worth anymore] | 21:24 |
| corvus | ianw: i'll switch to a +0 vote and encourage other folks who agree with the approach to +2 and ignore my previous comments | 21:24 |
| *** dmellado has quit IRC | 21:32 | |
| *** gouthamr has quit IRC | 21:32 | |
| *** spsurya has quit IRC | 21:38 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul-jobs master: Add a markdownlint job and role https://review.openstack.org/607691 | 21:44 |
| SpamapS | ^ converted to a role | 21:46 |
| SpamapS | now with revoke-sudo! | 21:46 |
| corvus | SpamapS: cool, can you add a README.rst (irony!) to the role? | 21:55 |
| SpamapS | git add strikes again | 21:55 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul-jobs master: Add a markdownlint job and role https://review.openstack.org/607691 | 21:59 |
| *** ssbarnea has quit IRC | 23:40 | |
| *** jimi|ansible has quit IRC | 23:48 | |
| *** eumel8 has quit IRC | 23:54 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul-jobs master: Add a markdownlint job and role https://review.openstack.org/607691 | 23:55 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!