| *** saneax has quit IRC | 00:07 | |
| *** mattw4 has quit IRC | 00:12 | |
| *** sgw has quit IRC | 00:13 | |
| *** igordc has quit IRC | 00:13 | |
| *** rfolco has joined #zuul | 00:19 | |
| *** wxy-xiyuan has joined #zuul | 01:08 | |
| *** rfolco has quit IRC | 01:33 | |
| *** bhavikdbavishi has joined #zuul | 01:47 | |
| *** bhavikdbavishi1 has joined #zuul | 01:50 | |
| *** bhavikdbavishi has quit IRC | 01:51 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 01:51 | |
| *** sshnaidm has quit IRC | 02:05 | |
| *** bhavikdbavishi has quit IRC | 02:08 | |
| *** threestrands has joined #zuul | 02:54 | |
| *** bhavikdbavishi has joined #zuul | 03:07 | |
| *** mnaser has quit IRC | 03:19 | |
| *** mnaser has joined #zuul | 03:20 | |
| *** saneax has joined #zuul | 04:07 | |
| *** saneax has quit IRC | 04:25 | |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul-jobs master: Expand documentation of test-setup role https://review.opendev.org/667244 | 04:27 |
|---|---|---|
| *** sgw has joined #zuul | 04:34 | |
| *** sgw has quit IRC | 04:41 | |
| *** swest has joined #zuul | 04:46 | |
| *** pcaruana has joined #zuul | 05:56 | |
| *** pcaruana has quit IRC | 05:57 | |
| *** pcaruana has joined #zuul | 05:57 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Add install-devstack role https://review.opendev.org/667157 | 05:58 |
| openstackgerrit | Merged zuul/zuul-jobs master: Expand documentation of test-setup role https://review.opendev.org/667244 | 05:58 |
| *** saneax has joined #zuul | 06:17 | |
| *** gtema has joined #zuul | 06:30 | |
| *** altlogbot_0 has quit IRC | 06:46 | |
| *** altlogbot_1 has joined #zuul | 06:56 | |
| *** threestrands has quit IRC | 07:00 | |
| *** threestrands has joined #zuul | 07:00 | |
| *** themroc has joined #zuul | 07:14 | |
| *** saneax has quit IRC | 07:31 | |
| *** saneax has joined #zuul | 07:32 | |
| *** jangutter has joined #zuul | 07:41 | |
| *** jangutter has quit IRC | 07:42 | |
| *** jangutter_ has joined #zuul | 07:42 | |
| *** sshnaidm has joined #zuul | 07:42 | |
| *** jpena|off is now known as jpena | 07:48 | |
| *** jpena is now known as jpena|mtg | 07:48 | |
| *** threestrands has quit IRC | 07:57 | |
| *** hashar has joined #zuul | 08:00 | |
| *** themroc has quit IRC | 08:02 | |
| *** themroc has joined #zuul | 08:06 | |
| *** yolanda has joined #zuul | 09:07 | |
| zbr|ruck | does anyone know if the zuul example is currently broken? I got into https://etherpad.openstack.org/p/zuul-standalone | 09:33 |
| *** bhavikdbavishi has quit IRC | 09:47 | |
| *** panda has quit IRC | 09:55 | |
| *** panda has joined #zuul | 10:00 | |
| *** gtema_ has joined #zuul | 10:05 | |
| *** gtema has quit IRC | 10:05 | |
| zbr|ruck | can anyone give me some hints on how to create a dependency chain for zuul jobs? mainly I want to stack jobs into 3 classes or "waves" based on their resource costs. A) linters ... B) medium C) fat ones. Jobs in B would start only if all ones from A would pass. | 10:34 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 10:36 |
| *** tobiash has joined #zuul | 11:25 | |
| badboy | fungi: thank you! | 11:51 |
| fungi | badboy: did upgrading to a newer pyca/cryptography fix it? | 11:53 |
| badboy | fungi: pip3 install cryptography --upgrade | 11:54 |
| fungi | i expect paramiko failed to increase the minimum version of it in their install_requires | 11:54 |
| badboy | fungi: I had cryptography-2.1.4 but the latest 2.7 works fine | 11:55 |
| *** bhavikdbavishi has joined #zuul | 11:56 | |
| fungi | the function it was looking for was introduced in 2.5 according to their changelog: | 11:56 |
| fungi | "Added :meth:`~cryptography.hazmat.primitives.asymmetric.ec.EllipticCurvePublicKey.from_encoded_point`, which immediately checks if the point is on the curve and supports compressed points. Deprecated the previous method :meth:`~cryptography.hazmat.primitives.asymmetric.ec.EllipticCurvePublicNumbers.from_encoded_point`." | 11:56 |
| fungi | https://github.com/pyca/cryptography/blob/master/CHANGELOG.rst#25---2019-01-22 | 11:56 |
| badboy | fungi: good catch, I'm not up to date with that | 11:57 |
| badboy | fungi: so downgrading paramiko worked as well | 11:57 |
| fungi | huh, seems they did update their install_requires in https://github.com/paramiko/paramiko/commit/36fbe57629cbbb7bf0f4a1e98c43352b82fe181d which was released as far back as paramiko 2.5.0 | 11:59 |
| *** bhavikdbavishi1 has joined #zuul | 11:59 | |
| fungi | so i have to wonder why that didn't upgrade cryptography | 11:59 |
| badboy | I guess I have to update cryptography on my templates to avoid that issue in the future | 12:00 |
| *** bhavikdbavishi has quit IRC | 12:00 | |
| openstackgerrit | Andy Ladjadj proposed zuul/zuul master: [doc][monitoring] Fix the wait_time parent attribute https://review.opendev.org/667342 | 12:00 |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 12:00 | |
| openstackgerrit | Andy Ladjadj proposed zuul/zuul master: [doc][monitoring] Fix the wait_time parent attribute https://review.opendev.org/667342 | 12:01 |
| flaper87 | anyone that has windows workers? I'm setting up a couple of windows workers and I'm wondering how the login is done. Does zuul just use a certificate? I don't see in the docs how the password can be passed to nodepool | 12:02 |
| * flaper87 is certainly missing something | 12:02 | |
| badboy | flaper87: WinRM? | 12:02 |
| * badboy AFK | 12:03 | |
| flaper87 | badboy: yeah, it's WinRM. I have the node ready, tested connection, ran some ansible playbooks against it and I'm now configuring nodepool | 12:03 |
| flaper87 | nodepool allows me to set the username, connection type, connection port but I don't see how to pass the password/cert, etc | 12:04 |
| openstackgerrit | Andy Ladjadj proposed zuul/zuul master: [doc][monitoring] Fix the wait_time parent attribute https://review.opendev.org/667342 | 12:04 |
| flaper87 | unless it expects it to be just open | 12:04 |
| *** rfolco has joined #zuul | 12:12 | |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 12:15 |
| *** rlandy has joined #zuul | 12:29 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Remove non working tests/base.py ZuulTestCase.getPipeline method https://review.opendev.org/667351 | 12:40 |
| flaper87 | oh, by reading zuul's code it looks like it uses keys | 12:44 |
| zbr|ruck | I added a feature request to allow use of tags on dependencies, some feedback would be appreciated. https://storyboard.openstack.org/#!/story/2005951 | 12:46 |
| *** jangutter_ is now known as jangutter | 12:51 | |
| zbr|ruck | https://storyboard.openstack.org/#!/story/2005952 -- about allowing regex on dependencies | 12:57 |
| zbr|ruck | fungi: do ^ make sense? | 12:58 |
| fungi | that's an interesting idea... i guess it's scoped to only the set of jobs which would have been triggered for a particular event? | 13:09 |
| *** bhavikdbavishi has quit IRC | 13:39 | |
| tobiash | corvus: I think I found a problem with nodeless jobs when having all executors zoned. In this case there will be no executor that will accept that job. I'm not sure how we should handle that case. Should we pick a main zone in this case? | 13:47 |
| *** openstackgerrit has quit IRC | 13:48 | |
| *** tosky has joined #zuul | 13:52 | |
| *** openstackgerrit has joined #zuul | 13:53 | |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: wip: Allow proceeding with requests on quota exceeded https://review.opendev.org/667371 | 13:53 |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: wip: Allow proceeding with requests on quota exceeded https://review.opendev.org/667371 | 13:55 |
| *** portdirect has joined #zuul | 13:57 | |
| *** saneax has quit IRC | 13:58 | |
| *** michael-beaver has joined #zuul | 14:10 | |
| corvus | tobiash: or run at least one unzoned executor? or add an extra option to the executor to specify that it can also handle unzoned jobs? | 14:17 |
| tobiash | I think the last option would be great | 14:18 |
| tobiash | that could make it possible to define one set of executors as fallback if the node has no zone | 14:18 |
| *** sgw has joined #zuul | 14:25 | |
| *** hashar has quit IRC | 14:27 | |
| mnaser | clarkb, pabelanger: yeah, it was a ceph issue at the time and we were retuning to fix it and improve that issue | 14:29 |
| mnaser | it shouldn't happen again. | 14:29 |
| *** panda has quit IRC | 14:41 | |
| tobias-urdin | just upgraded zuul and nodepool to latest version in just a couple of minutes, release notes was on point, high five o/ | 14:42 |
| *** panda has joined #zuul | 14:43 | |
| *** igordc has joined #zuul | 14:49 | |
| corvus | tobias-urdin: \o/ | 14:59 |
| *** themroc has quit IRC | 15:02 | |
| *** jpena|mtg is now known as jpena|off | 15:11 | |
| *** themroc has joined #zuul | 15:12 | |
| *** flepied has quit IRC | 15:18 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: URLTrigger driver time based https://review.opendev.org/635567 | 15:26 |
| *** igordc has quit IRC | 15:32 | |
| *** mattw4 has joined #zuul | 15:52 | |
| *** themroc has quit IRC | 16:01 | |
| *** hashar has joined #zuul | 16:03 | |
| *** igordc has joined #zuul | 16:04 | |
| *** chandankumar is now known as raukadah | 16:13 | |
| *** hashar has quit IRC | 16:17 | |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Switch functional testing to a devstack consumer job https://review.opendev.org/665023 | 16:35 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs https://review.opendev.org/667156 | 16:35 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Switch functional testing to a devstack consumer job https://review.opendev.org/665023 | 16:37 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs https://review.opendev.org/667156 | 16:37 |
| *** hashar has joined #zuul | 16:42 | |
| *** pwhalen has quit IRC | 16:42 | |
| *** pwhalen has joined #zuul | 16:45 | |
| *** panda has quit IRC | 16:46 | |
| *** panda has joined #zuul | 16:48 | |
| *** igordc has quit IRC | 16:56 | |
| *** igordc has joined #zuul | 16:59 | |
| *** hashar has quit IRC | 17:03 | |
| openstackgerrit | Alex Schultz proposed zuul/zuul master: Additional note about branches for implied-branches https://review.opendev.org/667415 | 17:07 |
| openstackgerrit | Merged zuul/zuul master: Add command processor to zuul-web https://review.opendev.org/666307 | 17:09 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 17:10 |
| *** raukadah is now known as chandankumar | 17:11 | |
| *** chandankumar is now known as raukadah | 17:13 | |
| *** gtema_ has quit IRC | 17:16 | |
| *** gtema_ has joined #zuul | 17:16 | |
| *** gtema_ has quit IRC | 17:20 | |
| *** jeliu_ has joined #zuul | 17:21 | |
| openstackgerrit | Merged zuul/zuul master: Add repl server for debug purposes https://review.opendev.org/579962 | 17:22 |
| *** mattw4 has quit IRC | 17:38 | |
| *** mattw4 has joined #zuul | 17:45 | |
| *** sgw has left #zuul | 18:13 | |
| *** sgw has joined #zuul | 18:13 | |
| *** gtema has joined #zuul | 18:29 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Filter out unprotected branches from builds if excluded https://review.opendev.org/666664 | 18:37 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Filter out unprotected branches from builds if excluded https://review.opendev.org/666664 | 18:39 |
| *** ianychoi_ has quit IRC | 18:49 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Filter out unprotected branches from builds if excluded https://review.opendev.org/666664 | 18:50 |
| *** ianychoi_ has joined #zuul | 18:50 | |
| *** gtema has quit IRC | 19:11 | |
| *** hashar has joined #zuul | 19:13 | |
| *** gtema has joined #zuul | 19:13 | |
| *** mattw4 has quit IRC | 19:18 | |
| *** jeliu_ has quit IRC | 19:27 | |
| hwangbo | Is there a gerrit event I can specify to reject draft reviews from entering pipelines? | 19:31 |
| *** pcaruana has quit IRC | 19:38 | |
| *** tosky has quit IRC | 19:40 | |
| corvus | hwangbo: you might be able to set a 'status' requirement for the change, but i don't know what that would be off the top of my head | 19:43 |
| corvus | hwangbo: maybe something like "reject: status: draft" | 19:43 |
| clarkb | opendev's Gerrit avoids this by disabling pushes to the drafts magic ref in gerrit | 19:43 |
| corvus | hwangbo: yeah, if you can avoid using draft status at all, i would recommend that | 19:44 |
| corvus | hwangbo: but if you have to use it, look at https://zuul-ci.org/docs/zuul/admin/drivers/gerrit.html#requirements-configuration and see if that helps | 19:44 |
| evgenyl | Hi everyone, I'm trying to use standard zuul-jobs, and I have a failure for a log fetching job, so I was wondering if anybody experienced anything similar http://paste.openstack.org/raw/753396/ `rsync: chown "dst-path" failed: Invalid argument (22)` for the context, I run executor in a privileged container. | 19:46 |
| hwangbo | corvus: from the gerrit docs, I don't see a status for draft https://gerrit-review.googlesource.com/Documentation/user-search.html#status | 19:48 |
| corvus | hwangbo: hrm, then i'm not sure... if there's some other attribute for it, we can add it | 19:49 |
| fungi | hwangbo: one of the reasons we disabled drafts in opendev's gerrit was that they would trigger jobs which zuul for refs zuul could not retrieve | 19:50 |
| fungi | but that was also a very, very long time ago | 19:50 |
| corvus | evgenyl: i'm not sure what would cause that, but you can run "zuul-executor keep" in your executor container and zuul will keep the build directory around after the job finishes, so you can cd into it and inspect it | 19:51 |
| pabelanger | evgenyl: make sure your executor has permissions to chown the files you are rsyncing. Possible one file is own by root, lets say, and executor doesn't have privileges to --chown that | 19:51 |
| corvus | evgenyl: (just run that before starting the job) | 19:51 |
| hwangbo | fungi: Unfortunately, drafts are a pretty integral part of our existing CI workflow to avoid automatic testing | 19:51 |
| hwangbo | I'll keep poking around the API, and test different things. Thanks for the advice! | 19:54 |
| evgenyl | corvus: `zuul-executor keep` - this is really helpful, thank you. | 19:54 |
| evgenyl | pabelanger: Yeah, I will try to poke around with manual chowning, thanks! | 19:54 |
| corvus | evgenyl: 'nokeep' is the opposite to turn that off when you're done | 19:56 |
| *** pwhalen has quit IRC | 20:04 | |
| *** mattw4 has joined #zuul | 20:14 | |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Switch functional testing to a devstack consumer job https://review.opendev.org/665023 | 20:23 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs https://review.opendev.org/667156 | 20:23 |
| *** ianychoi_ has quit IRC | 20:59 | |
| *** ianychoi_ has joined #zuul | 21:00 | |
| hwangbo | It looks like you can use 'status: draft' as a require condition but not a reject condition | 21:05 |
| *** ianychoi_ has quit IRC | 21:05 | |
| *** ianychoi_ has joined #zuul | 21:09 | |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs https://review.opendev.org/667156 | 21:10 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Add Zypper to openstack func job https://review.opendev.org/667466 | 21:10 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Switch functional testing to a devstack consumer job https://review.opendev.org/665023 | 21:12 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs https://review.opendev.org/667156 | 21:12 |
| fungi | hwangbo: that seems like it could be a bug if we can require conditions we can't reject | 21:15 |
| hwangbo | fungi: from the documentation, it seems like we can only reject "approval" conditions, but not any of the others (open, current-patchset, status) | 21:16 |
| hwangbo | https://zuul-ci.org/docs/zuul/admin/drivers/gerrit.html#attr-pipeline.reject.%3Cgerrit%20source%3E | 21:16 |
| *** openstackgerrit has quit IRC | 21:18 | |
| *** ianychoi_ is now known as ianychoi | 21:18 | |
| fungi | hwangbo: in that case, it seems like it may be a well-documented bug ;) | 21:20 |
| fungi | the documentation calls reject a "mirror" of require, so that tells me that anything which require supports is at least fair game to add support for in reject | 21:22 |
| fungi | even if nobody's wanted it enough to spend time implementing that yet | 21:22 |
| *** gtema has quit IRC | 21:27 | |
| *** jeliu_ has joined #zuul | 21:30 | |
| *** hashar has quit IRC | 21:41 | |
| *** tobberydberg has quit IRC | 21:49 | |
| *** fdegir has quit IRC | 21:50 | |
| *** tobberydberg has joined #zuul | 21:51 | |
| *** fdegir has joined #zuul | 21:51 | |
| *** openstackgerrit has joined #zuul | 21:54 | |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Switch functional testing to a devstack consumer job https://review.opendev.org/665023 | 21:54 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs https://review.opendev.org/667156 | 21:54 |
| fungi | paladox mentioned this to me earlier... https://lists.wikimedia.org/pipermail/wikitech-l/2019-June/092227.html | 22:12 |
| fungi | zuul v3 in the running for new wikimedia ci system (along with gitlab ci and argo) | 22:12 |
| * paladox has a change that imports zuul v3 into our zuul repo. | 22:13 | |
| paladox | though that change won't be merged, as it was more of an experiment for me :) (testing a migration to scap deployments) | 22:14 |
| fungi | paladox: if it helps the opendev crowd, as well as at least a few other folks in here, have extensive experience porting zuul v2 jobs to v3 and are usually happy to provide guidance and recommendations | 22:14 |
| paladox | fungi oh, i'll pass that onto hashar. | 22:15 |
| fungi | yeah, i would have highlighted him too but he's not in channel at the moment | 22:15 |
| paladox | he tryed to migrate existing jobs (https://gerrit.wikimedia.org/r/#/c/integration/config/+/517545/, https://gerrit.wikimedia.org/r/#/c/integration/config/+/517546/ and https://gerrit.wikimedia.org/r/#/c/integration/config/+/517547/) | 22:16 |
| fungi | ahh, yeah, that's the script we originally used, though we had the luxury of not relying on many jenkins plugins and eventually (with zuul 2.5's experimental ansible launcher) using no jenkins plugins for a while before zuul v3 existed | 22:18 |
| paladox | ah | 22:19 |
| fungi | obviously the more your jobs rely on shell script literals or external executables, the easier they are to migrate | 22:19 |
| paladox | fungi also hashar left for the evening (he was on only ~30 mins ago - is on UTC +2) | 22:21 |
| fungi | the other worry i'd have now is that the ansible that generated was contemporary with the ansible versions available at the time, and ansible syntax has evolved in the couple intervening years, so some adjustments may need to be made to create valid modern ansible (though maybe not, it's fairly simple stuff) | 22:21 |
| paladox | yeh | 22:22 |
| fungi | figured. it's well late in his part of the globe | 22:22 |
| fungi | i'm only just coming up for air unfortunately | 22:22 |
| paladox | yup, he's only an hour ahead of me. | 22:22 |
| paladox | also i think some of the releng department experimented with different CI tools | 22:22 |
| paladox | hence why zuul v3 is in the running :) | 22:22 |
| fungi | nice! | 22:23 |
| paladox | fungi https://phabricator.wikimedia.org/T218138 | 22:24 |
| pabelanger | https://phabricator.wikimedia.org/T218138#5039263 is interesting. For the most part I would generally agree, but since the ansible 2.4 release, I've personally had minimal issues upgrading playbooks / roles used to deploy zuul / nodepool. In fact, zuul.ansible.com zuul jobs were just upgraded from ansible 2.7 to 2.8 without any changes required. | 22:27 |
| pabelanger | Testing in ansible/ansible has come a long way in last 2 years | 22:27 |
| fungi | pabelanger: thanks! that's got some great feedback in it | 22:28 |
| fungi | er, paladox i mean | 22:28 |
| paladox | your welcome :) | 22:28 |
| pabelanger | Yah, looks well done | 22:29 |
| fungi | and yeah, historical impressions of software having gaps or lack of stability can sometimes just be an indication that the individual last used it while it was still somewhat immature and evolving rapidly | 22:29 |
| clarkb | pabelanger: we've had to make changes to get 2.8 to work iirc | 22:30 |
| fungi | similarly, the impression of the webui for zuul being read-only is accurate, but likely to change *very* soon ;) | 22:31 |
| clarkb | ianw fixed them and we could test it all with zuul, but still not seamless | 22:31 |
| clarkb | things around handlers | 22:31 |
| clarkb | and include tasks | 22:31 |
| fungi | zuul giving you the option of trying different ansible versions concurrently also helps ease the ansible upgrade pain, even if there is still somewhat of a treadmill | 22:31 |
| pabelanger | clarkb: yah, I should have used a (*) there, our jobs are pretty minimal. No handlers, mostly shell / tox based jobs | 22:32 |
| pabelanger | clarkb: was the issue in zuul-jobs or some other place? | 22:33 |
| * paladox was the one that filed the task to upgrade to v3 https://phabricator.wikimedia.org/T186426 | 22:33 | |
| clarkb | pabelanger: it was in a role somewhere | 22:34 |
| clarkb | pabelanger: the issue was with handlers including tasks and that has to be done in some very specific way now | 22:35 |
| paladox | fungi oh yes, now i remember why i experimented with zuul v3, i did it to see if jenkins still worked with zuul v3. | 22:36 |
| pabelanger | clarkb: yah, I tend to avoid handlers as much as possible | 22:37 |
| fungi | even with ansible's warts, i'm still glad that zuul v3 went the way of not creating its own domain-specific language for remote execution, since there are plenty of projects out there already who specialize in doing that and we get network effects by siding with one rather than competing with all | 22:38 |
| pabelanger | I did notice the following when upgrading to 3.9.0: https://review.opendev.org/666972/ | 22:39 |
| pabelanger | seems to be something we missed when we merged multi-ansible bits | 22:39 |
| fungi | and we did pick one which is written in python and uses yaml for its data, so was not too dissimilar with what we would have ended up implementing independently | 22:39 |
| clarkb | that does have me thinking a bit about what sorts of config might be removable if we shipped with sane defaults (that could be overridden) | 22:50 |
| clarkb | In the gitlab example it was silly simple because it wasn't talking to gerrit (yet) and just runs single node jobs? | 22:51 |
| clarkb | makes me wonder if we could have a default zuul config that given a connection parameter did a sanish check style green red reporting and you'd only need in repo .zuul.yaml | 22:52 |
| clarkb | (I think the nodepool config becomes the complicated bit there) | 22:52 |
| clarkb | and maybe for nodepool using nodepool localhost as a static node resource would work? (I'm just throwing ideas out there thinking about what simplifying setup might look like) | 22:54 |
| pabelanger | what was the gitlab example? | 22:55 |
| clarkb | pabelanger: basically stick a repo in gitlab.com and go https://phabricator.wikimedia.org/T217594 | 22:55 |
| clarkb | so some of the simplicity there is that its a hosted service too. I'm sure that as soon as you run your own gitlab you've got to tell ti a lot more about where jobs can run and how many jobs can run and so on | 22:56 |
| pabelanger | yah | 22:57 |
| clarkb | https://phabricator.wikimedia.org/T217325 is the top level issue and has links and stuff | 22:57 |
| clarkb | ya you have to setup https://docs.gitlab.com/runner/configuration/advanced-configuration.html | 23:00 |
| clarkb | (and k8s and prometheus for additional features) | 23:00 |
| paladox | https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Futures_WG/Report#Zuul | 23:02 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Switch functional testing to a devstack consumer job https://review.opendev.org/665023 | 23:03 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs https://review.opendev.org/667156 | 23:03 |
| pabelanger | there seems to be some discrepency about in-tree configuration, but that aside, with zuul that could be done in a single if wanted | 23:07 |
| pabelanger | discrepancy* | 23:07 |
| *** jeliu_ has quit IRC | 23:08 | |
| clarkb | pabelanger: I think the criticism there is fair. Basically you have to configure what resources are available to tests via a separate nodepool config (and service) which you don't have to do if you put something on gitlab.com | 23:08 |
| clarkb | you do have to do similar when you setup up a gitlab runnre though | 23:09 |
| pabelanger | Agree, the report is well done. It is easier to get your own CI image in some cases with others | 23:10 |
| clarkb | and then there is the base job setup that is centrally managed. Also seems that one of the wants is to have project auto enroll with the ci system | 23:11 |
| clarkb | zuul doesn't directly support auto enroll but it owuldn't be too hard to put something between gerrit and zuul to do that (or possibly even add that as an option to the gerrit driver?) | 23:11 |
| SpamapS | paladox: Nodepool also supports OpenShift and AWS for nodes. | 23:12 |
| clarkb | basically process all gerrit events regardless of whether or not the project is in the project list | 23:12 |
| pabelanger | yah, auto-enroll is something I've seen asked for a few times for zuul.a.c. eg: click this button on github and poof, done | 23:12 |
| SpamapS | (the report only mentions OpenStack and Kubernetes) | 23:12 |
| clarkb | pabelanger: its harder with public github because you'd need a filter of somesort | 23:12 |
| clarkb | pabelanger: though maybe that is an acl on the app you install in github? | 23:13 |
| paladox | SpamapS i think we have our own nodes (wikimedia cloud) :) | 23:13 |
| clarkb | "these groups can use the app" then zuul processes all events it gets? | 23:13 |
| SpamapS | paladox:oh cool! | 23:13 |
| paladox | which runs openstack | 23:13 |
| SpamapS | I wasn't sure if that was still a thing. Good. | 23:13 |
| clarkb | paladox: fwiw the report notes that nodepool+ openstack would be an issue | 23:13 |
| pabelanger | clarkb: yah, not sure. I haven't looked to hard at it. | 23:13 |
| paladox | Yeh we had issues with nodepool. | 23:13 |
| paladox | ie was very slow for us | 23:13 |
| paladox | we only within the last year got rid of nodepool and replaced it with docker. But looks like we may be going back to nodepool :) | 23:14 |
| * SpamapS wonders if paladox is aware of just how big the "it's slow" iceberg is... | 23:14 | |
| clarkb | SpamapS: well that and I think they were running 4 year old nodepool | 23:14 |
| paladox | heh | 23:14 |
| clarkb | the software has gotten better over time :) hopefully the update paladox is pushing improves perceptions | 23:14 |
| SpamapS | I mean.. if you tell an engineer something's slow.. they're going to prove you wrong in 0.48s | 23:14 |
| paladox | yeh, was a pretty old version. | 23:15 |
| paladox | It was slow, but im not sure if that was the decision behind going to docker though. | 23:15 |
| SpamapS | docker.. like.. docker swarm? | 23:15 |
| SpamapS | or like, static nodes with docker to run concurrent isolated things on them? | 23:16 |
| * SpamapS is just curious. :) | 23:16 | |
| clarkb | thinking about the auto enrollment a bit more it probably does make sense to be able to have just trusted projects listed in the main config then any other project it sees events for get processed as untrusted projects | 23:17 |
| paladox | there static as far as i know. | 23:17 |
| SpamapS | clarkb:yes! | 23:17 |
| clarkb | that would greatly simplify configuration churn, I'll have to think about why that may be problematic more | 23:17 |
| paladox | https://integration.wikimedia.org/ci/ (integration-slave-docker-*) | 23:17 |
| clarkb | one potential problem with that is the initial config load, zuul does preload all configs for things it knows about | 23:19 |
| clarkb | but maybe it can ask gerrit for that list (or github) | 23:19 |
| clarkb | that is probably the biggest hurdle to get over to make that possible | 23:20 |
| clarkb | building the initial global config sanely with minimal info | 23:20 |
| paladox | i do quite like the idea of a .zuul* file (we have a task somewhere to create something similar) | 23:20 |
| SpamapS | clarkb: back in the Bonny days jamie lennox built a little python app that would generate the tenant.yaml for anybody that installed the BonnyCI GitHub app automatically. | 23:22 |
| SpamapS | IIRC it would add a whole new tenant for the org. | 23:22 |
| SpamapS | (or user) | 23:22 |
| clarkb | SpamapS: I think the concern from wmf is that when a new repo is created you hvae to create a new entry in the tenant config | 23:23 |
| clarkb | what is automateable for sure | 23:23 |
| clarkb | just wondering if zuul could do the right thing sanely without explicit listing in its config | 23:23 |
| SpamapS | GitHub apps can definitely query the API for installed orgs/repos. | 23:23 |
| SpamapS | Gerrit could probably just scan the list for config. | 23:24 |
| SpamapS | Or keep a cache. | 23:24 |
| clarkb | pabelanger: that bind mount chnage has me wondering where the venv install gets mounted in | 23:25 |
| clarkb | pabelanger: once I sort that out I'll leave a vote | 23:25 |
| pabelanger | my idea, much like SpamapS said, some script that will fire when github app installs, generate tenant configs, have human merge, then post jobs to open new PR on new repo to confirm a noop job ran, and point to some docs | 23:25 |
| paladox | Or you could create a new gerrit plugin that zuul listens on so that project owners can add there project to zuul using a rest api (created by the plugin) | 23:25 |
| clarkb | paladox: for us (opendev) that doesn't really chnage the amount of management work (project owners == people approving changes to zuul main config) | 23:29 |
| pabelanger | clarkb: yah, took me a bit to figure it out too: https://opendev.org/zuul/zuul/src/branch/master/zuul/executor/server.py#L1910 | 23:29 |
| paladox | oh | 23:29 |
| clarkb | pabelanger: thanks | 23:29 |
| clarkb | paladox: we don't give out project owner in gerrit because it allows a bit too much control :/ | 23:30 |
| paladox | ah yeh, as we recently found it also exposes something else :( | 23:31 |
| paladox | https://phabricator.wikimedia.org/T222653 (hidden) | 23:31 |
| clarkb | ya it implies a lot of permissions | 23:31 |
| paladox | Though im a gerrit manager for the wmf (in the group "Gerrit Manager" which grants me rights to create projects/groups). I also have +2 in refs/meta/* for All-Projects (but carn't merge as im not an admin) | 23:33 |
| *** mattw4 has quit IRC | 23:37 | |
| pabelanger | yah, that is a hard one for github right now. People want to tryout zuul, but not give admin access to their own repo. So, there still ways humans can force merge behind zuul and break things | 23:39 |
| fungi | this was an easier sell for openstack since we'd well before taken away anyone's ability to directly merge changes and already placed that under automated process | 23:41 |
| fungi | possibly the biggest hurdle for many organizations to jump | 23:41 |
| pabelanger | yah | 23:44 |
| pabelanger | there are also thing that github makes hard too, like doing git tags, you need push for that | 23:45 |
| pabelanger | and no ACL to just allow tag | 23:45 |
| pabelanger | all or nothing | 23:45 |
| fungi | i do really appreciate the granularity gerrit's acls offer | 23:46 |
| fungi | like, looking forward to being able to set acls explicitly allowing branch deletion in newer gerrit | 23:47 |
| pabelanger | I do think there is good use case to make some of the openstack/releases code more generic to use that workflow with a zuul-job. I looked at it briefly a few months ago, but just haven't found the time | 23:50 |
| pabelanger | gitops releases | 23:50 |
| fungi | push tag, enjoy release | 23:51 |
| pabelanger | I was more thinking, commit sha1 into yaml, then zuul does the tagging, etc | 23:54 |
| fungi | oh, you mean the release request reviewing and tag generation stuff | 23:54 |
| fungi | yeah | 23:54 |
| pabelanger | yah | 23:54 |
| fungi | a lot of that is simply working around one of the glaring gerrit feature omissions going on many years now: reviewing git tags | 23:54 |
| paladox | The ui for tags is nice :) | 23:57 |
| *** rlandy has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!