| *** jamesmcarthur has quit IRC | 00:12 | |
| *** jamesmcarthur has joined #zuul | 00:12 | |
| *** zxiiro has quit IRC | 00:13 | |
| *** jamesmcarthur has quit IRC | 00:18 | |
| *** jamesmcarthur has joined #zuul | 00:24 | |
| *** igordc has quit IRC | 00:25 | |
| *** igordc has joined #zuul | 00:25 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: docs: remove generated toc from the main index https://review.opendev.org/703468 | 00:28 |
|---|---|---|
| *** igordc has quit IRC | 00:49 | |
| *** rlandy is now known as rlandy|bbl | 00:55 | |
| *** jamesmcarthur has quit IRC | 01:04 | |
| *** jamesmcarthur has joined #zuul | 01:04 | |
| *** jamesmcarthur has quit IRC | 01:06 | |
| *** jamesmcarthur has joined #zuul | 01:06 | |
| *** rfolco has quit IRC | 01:41 | |
| *** jamesmcarthur has quit IRC | 01:56 | |
| *** jamesmcarthur has joined #zuul | 01:57 | |
| *** jamesmcarthur has quit IRC | 02:02 | |
| *** jamesmcarthur has joined #zuul | 02:24 | |
| *** bhavikdbavishi has joined #zuul | 02:56 | |
| *** jamesmcarthur has quit IRC | 03:04 | |
| *** jamesmcarthur has joined #zuul | 03:05 | |
| *** bhavikdbavishi1 has joined #zuul | 03:08 | |
| *** bhavikdbavishi has quit IRC | 03:10 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:10 | |
| *** jamesmcarthur has quit IRC | 03:11 | |
| *** jamesmcarthur has joined #zuul | 03:30 | |
| *** jamesmcarthur has quit IRC | 03:33 | |
| *** rlandy|bbl has quit IRC | 03:39 | |
| *** jamesmcarthur has joined #zuul | 04:42 | |
| *** saneax has quit IRC | 04:55 | |
| *** evrardjp has quit IRC | 05:34 | |
| *** evrardjp has joined #zuul | 05:34 | |
| *** jamesmcarthur has quit IRC | 05:35 | |
| *** jamesmcarthur has joined #zuul | 05:37 | |
| *** jamesmcarthur has quit IRC | 05:43 | |
| *** raukadah is now known as chandankumar | 05:45 | |
| *** saneax has joined #zuul | 06:01 | |
| *** yolanda has quit IRC | 06:04 | |
| *** saneax has quit IRC | 06:06 | |
| *** jamesmcarthur has joined #zuul | 06:06 | |
| *** jamesmcarthur has quit IRC | 06:13 | |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Handle event id in node requests https://review.opendev.org/703406 | 06:23 |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Centralize logging adapters https://review.opendev.org/703407 | 06:23 |
| *** saneax has joined #zuul | 06:53 | |
| *** themroc has joined #zuul | 07:09 | |
| *** jamesmcarthur has joined #zuul | 07:09 | |
| *** jamesmcarthur has quit IRC | 07:14 | |
| *** yolanda has joined #zuul | 07:25 | |
| *** yolanda has quit IRC | 07:33 | |
| *** yolanda has joined #zuul | 07:34 | |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul-jobs master: fetch-sphinx: Exclude doctrees directory https://review.opendev.org/703547 | 07:39 |
| *** bhavikdbavishi has quit IRC | 07:41 | |
| *** yolanda has quit IRC | 07:48 | |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Pass node request handler to launcher base class https://review.opendev.org/703549 | 07:50 |
| *** jamesmcarthur has joined #zuul | 08:10 | |
| *** jamesmcarthur has quit IRC | 08:14 | |
| *** bhavikdbavishi has joined #zuul | 08:15 | |
| *** bhavikdbavishi1 has joined #zuul | 08:18 | |
| *** bhavikdbavishi has quit IRC | 08:20 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 08:20 | |
| *** yolanda has joined #zuul | 08:21 | |
| *** armstrongs has joined #zuul | 08:29 | |
| *** armstrongs has quit IRC | 08:39 | |
| *** hashar has joined #zuul | 08:40 | |
| *** jpena|off is now known as jpena | 08:52 | |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Annotate logs in launcher https://review.opendev.org/703558 | 08:57 |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Annotate logs in node request handler https://review.opendev.org/703559 | 08:57 |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Include event id in node request listings https://review.opendev.org/703560 | 08:57 |
| *** tosky has joined #zuul | 09:07 | |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Annotate logs in zk module https://review.opendev.org/703561 | 09:10 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Add spec for scale out scheduler https://review.opendev.org/621479 | 09:11 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Add spec for scale out scheduler https://review.opendev.org/621479 | 09:15 |
| reiterative | clarkb: Thanks! That helped me to find the problem: my base job was including roles from zuul-jobs, but that should be zuul/zuul-jobs! | 09:21 |
| *** themr0c has joined #zuul | 09:25 | |
| *** yolanda has quit IRC | 09:27 | |
| *** themroc has quit IRC | 09:28 | |
| *** yolanda has joined #zuul | 09:33 | |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: Docs: fix stestr run example https://review.opendev.org/703566 | 09:40 |
| *** bhavikdbavishi has quit IRC | 09:41 | |
| *** yolanda has quit IRC | 09:49 | |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: tox: pass --slowest to stestr https://review.opendev.org/703571 | 09:58 |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: Divide concurrent tests by classes https://review.opendev.org/703575 | 10:08 |
| *** openstackgerrit has quit IRC | 10:12 | |
| *** bhavikdbavishi has joined #zuul | 10:39 | |
| *** bhavikdbavishi1 has joined #zuul | 10:42 | |
| *** bhavikdbavishi has quit IRC | 10:43 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 10:43 | |
| hashar | fun, running tests.unit.test_v3.TestAnsible28.test_plugins_1 suite, it seems Ansible detects python and ends up using /usr/bin/python | 11:08 |
| hashar | which turns out to be python2.7 on my machine ;) | 11:08 |
| hashar | "discovered_interpreter_python": "/usr/bin/python",' | 11:08 |
| reiterative | Is it possible to use the 'Depends-On' clause to specify a dependency on a change in zuul/zuul-jobs, which adds a role that the Zuul test job for your change uses? | 11:24 |
| reiterative | e.g. I have a proposed change (https://review.opendev.org/r/693513/) in zuul-jobs, and I want a test job (which uses Bazel) on my Zuul setup to use the install-bazel role that this change adds | 11:25 |
| *** ironfoot has joined #zuul | 11:40 | |
| *** rfolco has joined #zuul | 12:00 | |
| AJaeger | reiterative: yes, that should work. | 12:04 |
| *** bhavikdbavishi has quit IRC | 12:09 | |
| *** yolanda has joined #zuul | 12:10 | |
| *** yolanda has quit IRC | 12:10 | |
| sugaar | HI the line "[gearman] server=127.0.0.1" in zuul.conf specifies where gearman is going to launch?, so if I specify server=scheduller it will be launched "inisde" the scheduller? | 12:11 |
| *** yolanda has joined #zuul | 12:11 | |
| *** jpena is now known as jpena|lunch | 12:21 | |
| ironfoot | sugaar: looks like zuul-scheduler provides the gearman daemon. | 12:21 |
| ironfoot | "You may supply your own gearman server, but the Zuul scheduler includes a built-in server which is recommended. " in https://zuul-ci.org/docs/zuul/howtos/installation.html?highlight=gearman#zuul-components | 12:22 |
| ironfoot | so, to clarify, you don't decide where gearman is going to be launched. | 12:25 |
| sugaar | I understand that, however for some reason gearman does not get launched. I am trying to discover why. I have defined the a gearman port (2181) but it seems like it can not be reached | 12:41 |
| sugaar | https://paste.gnome.org/p5yjt1rvp | 12:42 |
| ironfoot | have you set the "[gearman_server] start=true" option? | 12:42 |
| sugaar | yes, ther is included in the conf file: [gearman] --> server=scheduler ; [gearman_server] --> start=true | 12:46 |
| pabelanger | heads up, pip 20.0 broken bindep jobs for us | 12:47 |
| AJaeger | there was just a report in #openstack-infra as well... | 12:48 |
| pabelanger | https://github.com/pypa/pip/issues/7217 | 12:49 |
| pabelanger | pip 20.0.1 released | 13:00 |
| *** rlandy has joined #zuul | 13:07 | |
| pabelanger | confirmed to fix our bindep errors | 13:07 |
| *** jamesmcarthur has joined #zuul | 13:18 | |
| *** zbr|drover has quit IRC | 13:18 | |
| *** zbr has joined #zuul | 13:19 | |
| *** jpena|lunch is now known as jpena | 13:23 | |
| tobiash | confirmed, 20.0 also broke us completely, 20.0.1 fixed it again | 13:24 |
| *** AJaeger has quit IRC | 13:33 | |
| *** jamesmcarthur has quit IRC | 13:33 | |
| *** AJaeger has joined #zuul | 13:34 | |
| *** avass has joined #zuul | 13:36 | |
| avass | does a nodeset request need to be fulfilled by a single pool in nodepool? | 13:43 |
| avass | I'm getting 'NODE_FAILURE' when trying to request a windows node and a linux node in the same job and we've separated those into two different pools. | 13:44 |
| Shrews | avass: yes | 13:44 |
| *** saneax has quit IRC | 13:44 | |
| *** jamesmcarthur has joined #zuul | 13:45 | |
| avass | Shrews: alright, we'll have to fix that then | 13:45 |
| avass | So that actually means there's no way to mix static nodes and dynamic nodes from a cloud provider? | 13:48 |
| Shrews | avass: that's correct. that is sort of implied in that you may specify only a single driver per provider (https://zuul-ci.org/docs/nodepool/configuration.html#attr-providers.driver) | 14:04 |
| fungi | that's mostly to work around locality challenges in multi-node jobs where connectivity between nodes in different providers isn't guaranteed | 14:07 |
| fungi | but the one-driver-per-provider situation does combine with that to make things somewhat less flexible | 14:07 |
| *** openstackgerrit has joined #zuul | 14:25 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Handle service restart when connections are changed https://review.opendev.org/703624 | 14:25 |
| reiterative | AJaeger Adding the Depends-On param to my change in Gerrit doesn't seem to have any effect when I trigger a recheck. Perhaps my Zuul instance is not correctly configured? I specified the dependency like this in the commit message (before the Change-Id): Depends-On: https://review.opendev.org/693513. Looking at the debug output for my zuul-executor, there's no sign of this dependent change being used when preparing the environment - it just uses the | 14:32 |
| reiterative | master branch to checkout opendev.org/zuul/zuul-jobs. | 14:32 |
| Shrews | swest: I left you some comments on your nodepool logging changes. I'm not a fan of passing entire objects around just to modify logging data. Let's try to keep these things loosely coupled. | 14:34 |
| fungi | reiterative: oh, you were asking about a depends-on between different code review systems? | 14:34 |
| fungi | reiterative: what connector are you using for opendev.org in your configuration? | 14:35 |
| fungi | reiterative: it needs to be the gerrit connector rather than the git connector for cross-platform dependencies to work | 14:35 |
| *** dtroyer has joined #zuul | 14:38 | |
| swest | swest: It just looks like the NodeLauncher (at least all of the concrete impls.) is anyways tightly coupled to the the NodeLauncher so I figured it might just pass it up to the base class. I'm all for decoupling, but this just looks like some artificial split since all the concrete implementations are already tightly coupled to the handler. | 14:39 |
| swest | Shrews: ^ | 14:39 |
| Shrews | swest: subclassing will be tightly coupled to the base class, yes. but we don't need to tie ourselves needlessly to other objects | 14:41 |
| Shrews | swest: can you show me where the NodeLauncher is already coupled to the handler? | 14:43 |
| swest | Shrews: what I meant to say was: all subclasses are tightly coupled to the handler already. So it seems like there is no use for a NodeLauncher that doesn't have a ref to a handler | 14:43 |
| swest | Shrews: e.g. here https://opendev.org/zuul/nodepool/src/branch/master/nodepool/driver/openstack/handler.py#L49 and in all other subclasses of the NodeLauncher | 14:44 |
| swest | basically all the instances that I modified in https://review.opendev.org/#/c/703549/ | 14:44 |
| Shrews | swest: ah, in the actual implementation. thx, been a while since i've looked at this code. let me look again.... | 14:45 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add tenant reconfiguration when main.yaml changed https://review.opendev.org/703631 | 14:46 |
| Shrews | swest: Ok, you changed my mind on the handler bit now that I've refreshed myself on the code. I still believe we shouldn't pass the NodeRequest object to the get_annotated_logger though. | 14:51 |
| swest | Shrews: k, you convinced me on that part :) There is no need to couple the event id to the node request, since that could also be used as a generic event id e.g. for related log messages from image builds | 14:53 |
| swest | I'll change that | 14:53 |
| Shrews | \ | 14:54 |
| Shrews | \o/ | 14:54 |
| *** armstrongs has joined #zuul | 14:59 | |
| *** avass has quit IRC | 14:59 | |
| *** avass has joined #zuul | 15:04 | |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: tox: reduce deps used for pep8 env https://review.opendev.org/703634 | 15:05 |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: tox: rename pep8 to linters https://review.opendev.org/703635 | 15:13 |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: tox: do not install bindep for linters https://review.opendev.org/703636 | 15:13 |
| *** electrofelix has joined #zuul | 15:16 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Handle service restart when connections are changed https://review.opendev.org/703624 | 15:17 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add networking.k8s.io apiGroups rbac for service account https://review.opendev.org/703637 | 15:17 |
| *** jamesmcarthur has quit IRC | 15:25 | |
| *** themr0c has quit IRC | 15:28 | |
| *** electrofelix has quit IRC | 15:31 | |
| *** jamesmcarthur has joined #zuul | 15:35 | |
| *** electrofelix has joined #zuul | 15:36 | |
| *** electrofelix has quit IRC | 15:36 | |
| *** electrofelix has joined #zuul | 15:36 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add spec for enhanced regional executor distribution https://review.opendev.org/663413 | 15:38 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Optionally allow zoned executors to process unzoned jobs https://review.opendev.org/673840 | 15:39 |
| corvus | Shrews: can you take a look at this tiny docs change? https://review.opendev.org/703471 | 15:48 |
| *** chandankumar is now known as raukadah | 15:48 | |
| Shrews | corvus: +A'd | 15:49 |
| tristanC | corvus: mnaser: https://review.opendev.org/703624 should implements the scheduler queues reload on restart | 15:50 |
| tristanC | though i think we need a ready/stop probe for the scheduler as the task enqueues happen too fast and some of the changes are enqueued on the dying scheduler :) | 15:53 |
| *** zxiiro has joined #zuul | 15:57 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add OpenShift SCC and functional test https://review.opendev.org/702758 | 15:58 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Handle service restart when connections are changed https://review.opendev.org/703624 | 15:58 |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Enable E741 flake8 check https://review.opendev.org/703650 | 16:00 |
| *** hashar has quit IRC | 16:02 | |
| sugaar | Hi, I am trying to understand better how gearman is launched. Reading https://opendev.org/zuul/zuul/src/branch/master/zuul/cmd/scheduler.py makes me think that the scheduler will automatically launch it, but I don' get when is it going to happen. I have a pod running a zuul-scheduler container and for some reason, the bl**dy gearman won't start. | 16:03 |
| sugaar | Obviously the container image must be bug free since it is used in the docker-compose example and in the helm charts. So there is something in my configuration/architecture that is stopping gearman from being launched or from zuul-scheduler detecting that gearman has been launched. | 16:03 |
| tobiash | sugaar: do you have configured start=true in the gearman_server section of zuul.conf? | 16:04 |
| sugaar | yes | 16:04 |
| sugaar | I am using the same zuul.conf than in the docker-compose example | 16:05 |
| tobiash | sugaar: it's started here: even before logging is setup: https://opendev.org/zuul/zuul/src/branch/master/zuul/cmd/scheduler.py#L136 | 16:05 |
| ironfoot | does 127.0.0.1 work inside a k8s pod? | 16:05 |
| tobiash | sugaar: it's started as a separate process so you should see two python processes in the scheduler container | 16:05 |
| tobiash | ironfoot: should work, but I'd rather suggest to add a service pointing to k8s and use this throughout the deployment | 16:06 |
| tobiash | (to have a common zuul.conf for scheduler, executor, web,..) | 16:06 |
| ironfoot | yeah | 16:06 |
| tobiash | *service pointing to gearman I meant ;) | 16:07 |
| ironfoot | like in the helm chart I've seen around | 16:07 |
| sugaar | tobiash but how can I check which processes are running? the container doesn't have ps or top on it | 16:08 |
| tobiash | sugaar: I thought ps is a builtin shell command? | 16:09 |
| clarkb | you can ps from the host side | 16:09 |
| tobiash | if you have access to the k8s host that's also an option | 16:10 |
| fungi | or check that the listening port is bound? | 16:11 |
| openstackgerrit | Merged zuul/zuul-jobs master: fetch-sphinx: Exclude doctrees directory https://review.opendev.org/703547 | 16:12 |
| sugaar | clark how do you do that? sending the command via ssh? | 16:13 |
| *** openstackgerrit has quit IRC | 16:13 | |
| sugaar | clarkb ^^ | 16:13 |
| clarkb | sugaar: log into the k8s host and ps there (ssh is one way) | 16:13 |
| sugaar | I will try | 16:14 |
| sugaar | thanks | 16:14 |
| *** openstackgerrit has joined #zuul | 16:14 | |
| openstackgerrit | Clément Mondion proposed zuul/nodepool master: add tags support for aws provider https://review.opendev.org/703651 | 16:14 |
| *** tosky has quit IRC | 16:33 | |
| openstackgerrit | Clément Mondion proposed zuul/nodepool master: add tags support for aws provider https://review.opendev.org/703651 | 16:35 |
| *** mattw4 has joined #zuul | 16:41 | |
| *** jpena is now known as jpena|brb | 16:46 | |
| *** armstrongs has quit IRC | 16:48 | |
| pabelanger | any objections on updating zuul UI to sort job names per-buildset? | 16:50 |
| clarkb | pabelanger: by what criteria? | 16:52 |
| pabelanger | alphabetical order mostly | 16:53 |
| corvus | pabelanger: the job list is supposed to be ordered. | 16:53 |
| pabelanger | https://dashboard.zuul.ansible.com/t/ansible/status | 16:53 |
| pabelanger | ansible-network/ansible_collections.ansible.netcommon jobs | 16:53 |
| pabelanger | for example | 16:53 |
| pabelanger | I think project-templates are what they might be ordered by? | 16:54 |
| corvus | the templates are in order, then the jobs | 16:54 |
| corvus | so you can change the order to whatever you want in the project-pipeline config (as long as it isn't inserting a new job between 2 templates) | 16:55 |
| pabelanger | k, that explains is. Was looking to order via just job name | 16:55 |
| pabelanger | I can also just live with this | 16:56 |
| corvus | pabelanger: are you unable to put them in the order you want? | 16:56 |
| pabelanger | in this case, no. github-workflow is top level project-template for all projects. So, I'm not able to move between other 2 tempaltes at project level, if that makes sense | 16:57 |
| pabelanger | but again, I can accept this layout too | 16:58 |
| corvus | pabelanger: gotcha | 16:58 |
| pabelanger | knowing project-template, then jobs is order is helpful | 16:59 |
| *** rfolco is now known as rfolco|brb | 17:16 | |
| openstackgerrit | Clément Mondion proposed zuul/nodepool master: add tags support for aws provider https://review.opendev.org/703651 | 17:23 |
| *** jpena|brb is now known as jpena | 17:26 | |
| *** evrardjp has quit IRC | 17:34 | |
| *** evrardjp has joined #zuul | 17:34 | |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: tox: do not install bindep for linters https://review.opendev.org/703636 | 17:36 |
| openstackgerrit | Merged zuul/zuul master: Docs: fix stestr run example https://review.opendev.org/703566 | 17:41 |
| *** yolanda has quit IRC | 17:52 | |
| *** yolanda has joined #zuul | 17:54 | |
| *** jamesmcarthur has quit IRC | 18:00 | |
| *** electrofelix has quit IRC | 18:34 | |
| *** jpena is now known as jpena|off | 18:34 | |
| Shrews | I think my wifi router is on its last leg. Getting random disconnects and strange log entries. :( | 18:37 |
| fungi | sounds a lot like my mind | 18:52 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Revert "Extract an abstract base Parser class" https://review.opendev.org/703669 | 18:52 |
| corvus | we should make sure to merge that before the next release | 18:52 |
| pabelanger | +2 | 18:53 |
| corvus | my power seems to be very unstable... | 18:55 |
| *** dustinc|PTO is now known as dustinc | 19:13 | |
| *** jamesmcarthur has joined #zuul | 19:20 | |
| *** tosky has joined #zuul | 19:25 | |
| zbr | i observed that zuul unittests fail often due to timeout, beinf very close to 1h, i suggest +20% bump of timeout for zuul tox jobs. example: https://review.opendev.org/#/c/703372/ | 19:29 |
| zbr | if it sounds ok, i will propose it. | 19:29 |
| AJaeger | zbr: there's a fix up for it already... | 19:30 |
| AJaeger | zbr: https://review.opendev.org/#/c/702473/ | 19:31 |
| fungi | which is even better than increasing the timeout | 19:32 |
| fungi | since we'll usually get results ~twice as fast after splitting | 19:32 |
| zbr | i personally do not like sharding, i still have PTSD from molecule where they are still used. | 19:32 |
| AJaeger | fungi: still takes a long time to prepare tests | 19:32 |
| zbr | pytest-xdist is to be preffered, when possible. | 19:33 |
| AJaeger | zbr: check the implementation, for local testing it's not sharded, only in CI | 19:33 |
| clarkb | fwiw I think there is a bit of room to optimize the zuul tests since they spend a lot of time waiting iirc | 19:33 |
| clarkb | as another approach once the bleeding is stopped | 19:34 |
| zbr | as i do not have a knowledge on how the test run I suppose there where reasons that prevented parallelization inside the same job. | 19:35 |
| clarkb | zbr: it is already using every available cpu to run a separate test runner within the job | 19:35 |
| zbr | bigger box? | 19:36 |
| tobiash | clarkb: actually the zuul tests are quite cpu bound | 19:36 |
| *** jamesmcarthur has quit IRC | 19:46 | |
| *** jamesmcarthur has joined #zuul | 19:50 | |
| openstackgerrit | David Shrewsbury proposed zuul/zuul-jobs master: ensure-tox: Output tox version https://review.opendev.org/701236 | 20:00 |
| *** jamesmcarthur has quit IRC | 20:01 | |
| *** jamesmcarthur has joined #zuul | 20:01 | |
| *** hashar has joined #zuul | 20:07 | |
| *** jamesmcarthur has quit IRC | 20:08 | |
| clarkb | oh we halved the cpus so we use available_cpus/2 | 20:11 |
| openstackgerrit | Merged zuul/zuul-website master: Remove some redirects https://review.opendev.org/703457 | 20:11 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Fix releasenotes redirects https://review.opendev.org/703687 | 20:14 |
| *** jamesmcarthur has joined #zuul | 20:14 | |
| clarkb | user reported ^ was broken so I've fixed that specific redirect. Not sure if there is a class of redirects needed there though | 20:15 |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Speed up ansible plugin tests https://review.opendev.org/703688 | 20:19 |
| clarkb | tobiash: ^ thats a first pass at speeding tests up based on the slow test list | 20:19 |
| clarkb | seems to have a measurable impact on my laptop | 20:19 |
| clarkb | always with the .keep file | 20:24 |
| hashar | I found a few oddities today when looking at those | 20:24 |
| clarkb | hashar: the redirects, tests, or .keep files :) | 20:25 |
| hashar | for a tests/unit/test_v3.py I noticed that ansible has python set to 'auto' which triggers some heuristic to find the python command | 20:25 |
| clarkb | yes it is supposed to use the platform default then fallback to any python aiui | 20:25 |
| hashar | potentially slows it down, then the ansible < 2.8 tests ended up failling for me locally when pointing python_path to my local python3.7 .. ;D | 20:26 |
| hashar | I tried in the FakeNodepool to use python_path = sys.executable (or something like that) | 20:26 |
| hashar | maybe the fake nodepool could be changed from 'auto' to just '/usr/bin/python', not sure whether that would speed it up anyway | 20:27 |
| clarkb | I think we actually want to test the auto code path though | 20:27 |
| clarkb | since users are relying on it | 20:27 |
| hashar | so that the ansible tests can act as some kind of integration tests against the different OS ? | 20:27 |
| tobiash | clarkb: keep file hit again ;) | 20:28 |
| clarkb | tobiash: ya once we have data on the current run I'll update the change and add the .keep file back | 20:29 |
| clarkb | unless I need that for the tests anyway? | 20:29 |
| clarkb | I probably do now that I think of it | 20:29 |
| *** yolanda has quit IRC | 20:29 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Speed up ansible plugin tests https://review.opendev.org/703688 | 20:29 |
| tobiash | clarkb: with this we don't have to mess again with the keep file: https://review.opendev.org/663108 | 20:29 |
| clarkb | thanks I'll review that after lunch | 20:30 |
| *** yolanda has joined #zuul | 20:30 | |
| tobiash | mordred, corvus: you also reviewed (and reverted) the first version of this ^ this version should work now | 20:33 |
| openstackgerrit | Merged zuul/zuul master: tox: pass --slowest to stestr https://review.opendev.org/703571 | 20:33 |
| hashar | \O/ | 20:34 |
| corvus | clarkb: maybe recheck 688 a few times? in the past we've worried about cpu contention killing zk, etc.... | 20:36 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: fix pipe race https://review.opendev.org/703689 | 20:36 |
| clarkb | corvus: ya, I was worried about that too, but locally (and this is human observation) once the test starts my cpu contention falls way off | 20:37 |
| clarkb | its the software install the makes my laptop slow | 20:37 |
| clarkb | we'll have to see what the ci data says and rechecking will get us more | 20:38 |
| openstackgerrit | Merged zuul/zuul master: doc: add links to components documentation https://review.opendev.org/703105 | 20:38 |
| tobiash | fyi, the timer jitter change also brought a test race with it. Benjamin is working on a fix (replace sleep by iterate_timeout). He'll provide it tomorrow. | 20:41 |
| *** yolanda has quit IRC | 20:46 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: use pip3 in preference to pip https://review.opendev.org/703694 | 20:46 |
| *** yolanda has joined #zuul | 20:50 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: fix pipe race https://review.opendev.org/703689 | 20:50 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: use pip3 in preference to pip https://review.opendev.org/703694 | 20:50 |
| clarkb | corvus: I see https://review.opendev.org/#/c/703456/2 is the proper fix for the redirects. I'll abandon mine but yours needs a small fix | 20:55 |
| openstackgerrit | James E. Blair proposed zuul/zuul-website master: Update redirects https://review.opendev.org/703456 | 20:56 |
| corvus | clarkb, fungi: ^ | 20:56 |
| clarkb | +2 thanks | 20:57 |
| *** jamesmcarthur has quit IRC | 20:58 | |
| openstackgerrit | Merged zuul/zuul master: Limit parallelity when installing ansible https://review.opendev.org/703126 | 21:04 |
| openstackgerrit | Merged zuul/zuul-website master: Update redirects https://review.opendev.org/703456 | 21:07 |
| openstackgerrit | Merged zuul/zuul-jobs master: ensure-tox: fix pipe race https://review.opendev.org/703689 | 21:20 |
| hashar | is anyone aware of a way to skip setUp() entirely for a specific test method? :) | 21:23 |
| hashar | or I guess I can just move that method to a standalone class | 21:23 |
| clarkb | ya I think ^ is the way you are supposed to do it | 21:24 |
| tristanC | hashar: perhaps adding the `@unittest.skip('noop')` would work? | 21:24 |
| hashar | I found a zuul test that loads the whole stack (due to ZuulTestCase) when it really just: assert('foo', repr(someobject)) :D | 21:25 |
| hashar | tristanC: that would skip it entirely ;] | 21:25 |
| hashar | I guess I will just create a new class that does not inherit from ZuulTestCase | 21:26 |
| corvus | hashar: switch the parent to BaseTestCase ? | 21:26 |
| corvus | yeah | 21:26 |
| hashar | there is probably a lot of those tests in test/unit/ files that could benefit from that | 21:26 |
| hashar | corvus: BaseTestCase looks like a good fit. Thx ! | 21:27 |
| clarkb | tobiash: my change hit the jitter race | 21:28 |
| openstackgerrit | Merged zuul/zuul master: Docs: change "config" title https://review.opendev.org/703471 | 21:29 |
| openstackgerrit | Merged zuul/zuul-jobs master: ensure-tox: Output tox version https://review.opendev.org/701236 | 21:32 |
| clarkb | tobiash: can you check the comment on https://review.opendev.org/#/c/663108/2 | 21:32 |
| *** jamesmcarthur has joined #zuul | 21:35 | |
| tobiash | clarkb: if the symlink exists then normally the indec.html is there as well and we don't thnter this branch | 21:36 |
| clarkb | tobiash: I think we should still guard against it | 21:36 |
| tobiash | but we could make it more safe and check both | 21:36 |
| clarkb | something like try: os.readlink() except: os.symlink() | 21:37 |
| tobiash | clarkb: regarding the jitter race, I'll help Benjamin tomorrow with the fix if that's soon enough | 21:37 |
| clarkb | ya no rush | 21:38 |
| clarkb | I think the timing data for the test_plugins_* is still mostly valid | 21:38 |
| clarkb | corvus: initial data at https://review.opendev.org/#/c/703688/2 | 21:46 |
| openstackgerrit | Merged zuul/zuul master: docs: improve job.role documentation https://review.opendev.org/703372 | 21:51 |
| corvus | clarkb: cool. i think the main thing to watch out for is zk death. did the overall runtime decrease? | 21:59 |
| clarkb | the job still timed out, but it also hit the jitter job fail (which is a sit and wait for timeout I think) | 22:00 |
| corvus | oh, so it's unclear. | 22:00 |
| clarkb | will need more data from successful jobs to see if it affects total runtime much (in theory it should since we are running 8 jobs and only 4 runners) | 22:00 |
| clarkb | I would expec to save about a minute based on those numbers and assuming a decent test distribution across runners | 22:01 |
| clarkb | fwiw test_playbook runs in this same manner, it doesn't serially run jobs | 22:01 |
| clarkb | test_playbook becomes the slowest test with my change in place though | 22:01 |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: test_repo_repr does not need to clone https://review.opendev.org/703698 | 22:02 |
| hashar | finally, I just went with some mocking instead of adding yet another class :] | 22:02 |
| clarkb | looking at test playbook I'm not sure there is a good way to speed it up easily. We could split it into multiple tests but then you incur more startup time cost | 22:02 |
| clarkb | then test_job_pause* | 22:02 |
| clarkb | I think test_job_pause might be slightly quicker if we reduce the total number of jobs being tested (not sure why test-good and test-fail are there, but also recognize they are likely there to catch na interaction and I don't want to mess with that | 22:03 |
| *** mattw4 has quit IRC | 22:06 | |
| *** mattw4 has joined #zuul | 22:06 | |
| corvus | hashar: perhaps we should remove that test rather than mock that | 22:07 |
| corvus | I6675f4f3a65ba975456687de91827694273862e1 was the change that introduced it | 22:08 |
| corvus | looks like it is really a test of the string format. | 22:09 |
| hashar | yup | 22:09 |
| hashar | though it explicitly invokes __repr__ | 22:10 |
| hashar | that does not have that much value probably | 22:10 |
| corvus | normally i would not advocate adding a test like that. i guess since it's already there, we could add the mock. or remove it. either way. :) | 22:10 |
| hashar | I can amend my change and just drop it | 22:10 |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: tests: remove test_repo_repr https://review.opendev.org/703698 | 22:14 |
| hashar | test removed :] | 22:14 |
| *** mattw4 has quit IRC | 22:14 | |
| *** mattw4 has joined #zuul | 22:15 | |
| clarkb | yappi says that git operations (like repo update, change merges) are a significant chunk of per test time | 22:16 |
| clarkb | I'm not sure there are good ways to make that run faster, but if we could that would likely have a large global impact on the test suite runtime | 22:17 |
| clarkb | git repos on tmpfs maybe? | 22:18 |
| *** avass has quit IRC | 22:41 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add OpenShift SCC and functional test https://review.opendev.org/702758 | 22:41 |
| *** jamesmcarthur has quit IRC | 22:42 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Handle service restart when connections are changed https://review.opendev.org/703624 | 22:46 |
| hashar | maybe the slow tests from tests.unit.test_v3 can be moved to their own jobs | 22:46 |
| hashar | or at least profile those ;) | 22:47 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add tenant reconfiguration when main.yaml changed https://review.opendev.org/703631 | 22:48 |
| tristanC | corvus: with 703624 and 703631, the zuul-operator is now properly managing zuul.conf and main.yaml modification, using pure ansible tasks | 22:51 |
| openstackgerrit | Merged zuul/zuul master: Revert "Extract an abstract base Parser class" https://review.opendev.org/703669 | 23:01 |
| *** hashar has quit IRC | 23:16 | |
| clarkb | thats weird tox-py37 ran for 703571 twice in the gate, but only one is recorded in logstash and the other is recorded in zuul's db? | 23:22 |
| clarkb | https://04a9f9fdd9afdf12de4e-f889a65b4dfb1f628c8309e9eb44b225.ssl.cf2.rackcdn.com/703571/1/gate/tox-py37/bb9aedd/job-output.txt not in db and https://de3c109f548194542df7-51174786df661fdc707186ccf04b5df9.ssl.cf1.rackcdn.com/703571/1/gate/tox-py37/9c9a684/job-output.txt is in the db | 23:22 |
| clarkb | attempts shows 1 for both | 23:23 |
| clarkb | maybe the result of a gate reset? | 23:23 |
| clarkb | in any case one of those has a runtime about the same as 703688 and the other is about 9 minutes faster | 23:23 |
| clarkb | all three ran in rax-dfw | 23:23 |
| clarkb | the slower ones ran on the same cpu model according to the host info and the faster one was a different model | 23:25 |
| fungi | that sounds a likely culprit then (maybe not the cpu specifically, but vintage of the host) | 23:26 |
| clarkb | the faster job ran on a slightly older model with more clock spins | 23:27 |
| fungi | neat | 23:27 |
| fungi | so maybe entirely the cpu then | 23:27 |
| *** rfolco|brb is now known as rfolco | 23:41 | |
| *** tosky has quit IRC | 23:47 | |
| clarkb | comparing just the tox runtimes old: Ran: 881 tests in 2353.5932 sec. new: Ran: 881 tests in 2280.6632 sec. | 23:48 |
| clarkb | so my change is worth just over a minute in tox runtime I think | 23:48 |
| clarkb | though that may depend on test sharding | 23:48 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!