| openstackgerrit | Merged zuul/nodepool master: Use rax.dfw mirror for arm64 wheels https://review.opendev.org/752553 | 00:10 |
|---|---|---|
| *** hamalq has quit IRC | 00:30 | |
| *** rfolco|ruck has joined #zuul | 00:38 | |
| *** rfolco|ruck has quit IRC | 00:42 | |
| *** zbr7 has joined #zuul | 01:16 | |
| *** Diabelko has joined #zuul | 01:19 | |
| *** zbr has quit IRC | 01:19 | |
| *** systemd has quit IRC | 01:19 | |
| *** zbr7 is now known as zbr | 01:19 | |
| *** saneax has quit IRC | 01:19 | |
| *** zenkuro has quit IRC | 01:24 | |
| *** rfolco|ruck has joined #zuul | 01:36 | |
| *** hamalq has joined #zuul | 01:37 | |
| *** rfolco|ruck has quit IRC | 01:41 | |
| *** hamalq_ has joined #zuul | 01:41 | |
| *** hamalq has quit IRC | 01:44 | |
| *** hamalq_ has quit IRC | 03:18 | |
| *** ianychoi has quit IRC | 04:29 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #zuul | 04:33 | |
| *** saneax has joined #zuul | 05:19 | |
| *** wuchunyang has joined #zuul | 05:29 | |
| *** wuchunyang has quit IRC | 05:33 | |
| *** vishalmanchanda has joined #zuul | 05:34 | |
| *** wuchunyang has joined #zuul | 05:41 | |
| *** hamalq has joined #zuul | 05:44 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Add support to use stow for ensure-python https://review.opendev.org/751611 | 05:56 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Change Pipeline.getAllItems to generator https://review.opendev.org/752493 | 06:08 |
| *** hamalq has quit IRC | 06:29 | |
| *** hamalq has joined #zuul | 06:32 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Change Pipeline.getAllItems to generator https://review.opendev.org/752493 | 06:40 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Remove an unneeded api call when creating check_runs https://review.opendev.org/752056 | 06:41 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Save superfluous api requests in check run reporting https://review.opendev.org/752079 | 06:41 |
| *** hamalq has quit IRC | 06:44 | |
| *** zbr5 has joined #zuul | 06:45 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Add logging and monitoring of reporting timings https://review.opendev.org/752443 | 06:47 |
| *** zbr5 has quit IRC | 06:49 | |
| *** zbr has quit IRC | 06:49 | |
| *** hamalq has joined #zuul | 06:52 | |
| *** gundalow has quit IRC | 06:52 | |
| *** guilhermesp has quit IRC | 06:53 | |
| *** mwhahaha has quit IRC | 06:53 | |
| *** johnsom has quit IRC | 06:53 | |
| *** Open10K8S has quit IRC | 06:53 | |
| *** ChrisShort has quit IRC | 06:53 | |
| *** rpittau|afk has quit IRC | 06:53 | |
| *** stevthedev has quit IRC | 06:53 | |
| *** donnyd has quit IRC | 06:53 | |
| *** aprice has quit IRC | 06:53 | |
| *** iamweswilson has quit IRC | 06:53 | |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: WIP: Update ensure-docker for new release https://review.opendev.org/752630 | 06:54 |
| *** Open10K8S has joined #zuul | 06:55 | |
| *** zbr has joined #zuul | 06:55 | |
| *** mwhahaha has joined #zuul | 06:56 | |
| *** aprice has joined #zuul | 06:56 | |
| *** ChrisShort has joined #zuul | 06:56 | |
| *** guilhermesp has joined #zuul | 06:56 | |
| *** stevthedev has joined #zuul | 06:56 | |
| *** iamweswilson has joined #zuul | 06:56 | |
| *** rpittau|afk has joined #zuul | 06:56 | |
| *** hashar has joined #zuul | 06:56 | |
| *** donnyd has joined #zuul | 06:57 | |
| *** johnsom has joined #zuul | 06:57 | |
| *** armstrongs has joined #zuul | 07:03 | |
| *** jfoufas1 has joined #zuul | 07:05 | |
| *** jfoufas1 has quit IRC | 07:05 | |
| *** hamalq has quit IRC | 07:05 | |
| *** jfoufas1 has joined #zuul | 07:06 | |
| *** armstrongs has quit IRC | 07:07 | |
| *** gundalow has joined #zuul | 07:10 | |
| *** jcapitao has joined #zuul | 07:10 | |
| *** tosky has joined #zuul | 07:25 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Make reporting asynchronous https://review.opendev.org/691253 | 07:38 |
| *** jpena|off is now known as jpena | 07:49 | |
| *** holser has joined #zuul | 08:09 | |
| *** nils has joined #zuul | 08:09 | |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: Update ensure-docker for new releases https://review.opendev.org/752630 | 08:43 |
| *** mordred has quit IRC | 08:52 | |
| *** masterpe has quit IRC | 08:52 | |
| *** Eighth_Doctor has quit IRC | 08:53 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Fix integration with Zuul before release https://review.opendev.org/751291 | 09:13 |
| *** zenkuro has joined #zuul | 09:15 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-website master: Add link to zuul-client documentation https://review.opendev.org/751312 | 09:16 |
| *** Eighth_Doctor has joined #zuul | 09:16 | |
| *** Eighth_Doctor has quit IRC | 09:19 | |
| *** masterpe has joined #zuul | 09:25 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Add zuul-client testing https://review.opendev.org/752039 | 09:35 |
| *** vishalmanchanda has quit IRC | 09:43 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Add zuul-client testing https://review.opendev.org/752039 | 09:44 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: [DNM] Add zuul-client to requirements https://review.opendev.org/752385 | 09:45 |
| *** Eighth_Doctor has joined #zuul | 09:49 | |
| *** mordred has joined #zuul | 09:49 | |
| *** Diabelko is now known as systemd | 09:53 | |
| *** vishalmanchanda has joined #zuul | 09:57 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Add cross testing with Zuul https://review.opendev.org/751264 | 09:59 |
| *** wuchunyang has quit IRC | 10:06 | |
| *** jcapitao is now known as jcapitao_lunch | 10:16 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Add cross testing with Zuul https://review.opendev.org/751264 | 10:23 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Remove superfluous flushes and queries from SQL reporter https://review.opendev.org/752664 | 10:33 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Add logging and monitoring of reporting timings https://review.opendev.org/752443 | 10:47 |
| *** hashar is now known as hasharAway | 10:47 | |
| *** saneax has quit IRC | 11:28 | |
| *** jpena is now known as jpena|lunch | 11:39 | |
| *** holser has quit IRC | 11:46 | |
| *** holser has joined #zuul | 11:51 | |
| *** hasharAway is now known as hashar | 11:51 | |
| *** bhavikdbavishi has joined #zuul | 11:53 | |
| *** jcapitao_lunch is now known as jcapitao | 11:59 | |
| *** mattd01 has joined #zuul | 12:00 | |
| *** rlandy has joined #zuul | 12:07 | |
| zbr | clarkb: apparently the bionic-arm is bit fractured (alongside focal one), https://review.opendev.org/#/c/752630/ | 12:09 |
| zbr | that is new, yesterday they were working | 12:09 |
| mhu | corvus, hello, I've answered your comments re: https://review.opendev.org/#/q/status:open+topic:zuul-client_0.0.2 - the order of review should be https://review.opendev.org/#/c/751291/ (fix zuul integration bugs), then https://review.opendev.org/#/c/752039/ (add test_zuulcient in zuul unit tests), then https://review.opendev.org/#/c/751264/ (add zuul functional testing in zuul-client) | 12:20 |
| mhu | then we can tag zuul-client 0.0.2 to fix the PyPI package, and proceed with https://review.opendev.org/#/c/752385/ (use zuul-client lib in zuul) | 12:21 |
| *** saneax has joined #zuul | 12:22 | |
| *** jpena|lunch is now known as jpena | 12:37 | |
| *** Goneri has joined #zuul | 12:42 | |
| *** saneax has quit IRC | 12:45 | |
| *** bhavikdbavishi has quit IRC | 12:57 | |
| *** jfoufas1 has quit IRC | 13:03 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Add build-info subcommand https://review.opendev.org/751070 | 13:04 |
| *** saneax has joined #zuul | 13:10 | |
| *** saneax has quit IRC | 13:17 | |
| *** saneax has joined #zuul | 13:18 | |
| *** tosky_ has joined #zuul | 13:18 | |
| *** tosky is now known as Guest83047 | 13:18 | |
| *** tosky_ is now known as tosky | 13:18 | |
| zenkuro | hi, Im curious if people here know is there dedicated guide on running devstack on centos8? | 13:18 |
| fungi | zenkuro: that sounds like a question for the #openstack-qa channel. it's not really related to zuul | 13:19 |
| zenkuro | fungi: yes, but CI people sometimes run devstack on centos... sorry for disturbance, I will ask at QA | 13:20 |
| *** saneax has quit IRC | 13:38 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Make reporting asynchronous https://review.opendev.org/691253 | 13:40 |
| *** hashar has quit IRC | 14:12 | |
| *** hashar has joined #zuul | 14:12 | |
| clarkb | zbr: the mirror had trouble in the linaro cloud. I would recheck for now | 14:49 |
| *** ianychoi has joined #zuul | 14:59 | |
| zbr | clarkb: ok, rechecked again. | 14:59 |
| zbr | clarkb: what do you think about https://review.opendev.org/#/c/748682/ ? -- i think we reached a scalability issue with how we run/trigger test jobs on zuul-jobs. | 15:02 |
| zbr | i have the feeling that will never merge because is almost impossible to get 100 jobs to be green at the same time. | 15:03 |
| zbr | one of the failure is known, we talked about it yesterday, but the issue is still here. | 15:04 |
| zbr | we cannot improve the code/testing because other broken jobs are preventing us from merging the changes. | 15:04 |
| zbr | it creates a vicious cycle | 15:05 |
| clarkb | zuul os running all those tests pre merge which means we can improve them? | 15:05 |
| fungi | most changes to zuul-jobs don't trigger every role job | 15:05 |
| zbr | if you are lucky is only one or two simple to fix one, but in some cases you uncover long chain | 15:06 |
| zbr | lets open pandora box: https://review.opendev.org/#/c/748606/ :D | 15:07 |
| zbr | we do know that "zuul-jobs-test-fetch-sphinx-tarball-with-zuul-output" is broken and i do not know any way to fix it. | 15:08 |
| zbr | i do not blame ianw for adding that comment, still... what can we do? | 15:08 |
| clarkb | to be clear that test was broken and no one knew about it because we didnt have need to run it. When it was last run it worked | 15:08 |
| clarkb | so the ladt time we made changes to the role it was properly tested | 15:09 |
| zbr | that being one of the reasons i started working on enabling periodic testing | 15:09 |
| zbr | yep, code rots, roles too. | 15:09 |
| clarkb | it is an unfortunate situation that our testing relied on unwanted behavior from zuul | 15:10 |
| zbr | my view here is that using xfail would be ok for the moment, until we have a better solution. | 15:10 |
| clarkb | another option is to not modify the role until we cant test it | 15:11 |
| clarkb | or modify the role using the existing base-test dance | 15:12 |
| clarkb | I'm not sure what the best option is but there are options | 15:12 |
| zbr | due to the nature of this chance it would be easier to re-enable it in a follow-up. | 15:13 |
| zbr | zuul-maint: is anyone against enabling periodic-weekly for testing zuul-jobs? https://review.opendev.org/#/c/748706/ | 15:30 |
| zbr | we already have the mailing list and I said i will keep an eye on it. | 15:31 |
| clarkb | I guess even if people dont subsvribe to emails they can check the zuul builds dashboard | 15:33 |
| clarkb | so is accessible on demand that way | 15:33 |
| tristanC | zbr: looking at https://review.opendev.org/748682 , i worry this will cause unreasonable load. Is it necessary to test all distro every week? | 15:34 |
| clarkb | I think that is why it is weekly not daily in a new pipeline | 15:35 |
| clarkb | it should also trogger during a quiet period | 15:35 |
| zbr | yep, that is why is weekly, we need schedule to discover what gets rotten. but we could later tune its recurrence if we want/need. | 15:36 |
| zbr | tristanC: if we test only some distros we provide not much protection, one not-tested distro may get rotten and when someone would have to touch a role that builds it, he will find the issue, blocking his patch. | 15:37 |
| zbr | but i guess that if we find some distros to be unreliable, we could endup removing support for them (and their jobs). | 15:38 |
| zbr | tristanC: while the number of jobs seems quite big, all are <10min, i bet the cumulative load cost is less than a single tripleo-ci buildset :D | 15:41 |
| zbr | in the future I could try to implement a feature to skip running a scheduled job if it did already run recently, like travis does. | 15:43 |
| zbr | which could address concerns of wasting compute resources | 15:43 |
| clarkb | hrm failures with the nodepool azure driver? ImportError: No module named 'msrestazure' | 16:01 |
| clarkb | azure-mgmt-compute and azure-mgmt-network were released a few hours ago | 16:06 |
| clarkb | I'm looking into what the fix is | 16:06 |
| *** jcapitao has quit IRC | 16:08 | |
| fungi | those jobs are also very quick. it's not like the hundreds of periodic jobs openstack schedules daily many of which run 2+ hours | 16:18 |
| clarkb | looks like azure's sdk libs are cool with non backward compat changes | 16:18 |
| clarkb | I've got a patch I'm trying to test with at least our azure tests to ensure things work | 16:19 |
| clarkb | wow it looks like they've even broken themselves internally. The azure.common module isn't compatible with the changes to the sub modules | 16:20 |
| fungi | mugsie may find that interesting | 16:21 |
| *** hashar has quit IRC | 16:21 | |
| mugsie | they did it again? | 16:21 |
| mugsie | -_- | 16:21 |
| fungi | if only there were a ci system capable of testing changes across multiple repos ;) | 16:22 |
| *** saneax has joined #zuul | 16:22 | |
| mugsie | :D | 16:22 |
| openstackgerrit | Clark Boylan proposed zuul/nodepool master: Cap azure dep versions due to breaking changes https://review.opendev.org/752724 | 16:23 |
| clarkb | I had started with a change to update the excepton to the new exception type. I also do a s/credentials/credential/ but then the use of azure.common functions was still broken so now we get ^ | 16:23 |
| *** vishalmanchanda has quit IRC | 16:23 | |
| clarkb | mugsie: looks like azure.common may not explicitly dep on msrestazure which causes an import error now that the other bits don't pull that in. Possible the azure.common fix is as simple as including that as a dep and making a release but I'm not sure | 16:24 |
| mugsie | clarkb: cool, I will ping some people here after my flood of US/EU crossover meetings :D | 16:26 |
| *** sanjayu_ has joined #zuul | 16:31 | |
| clarkb | it looks like azure tries to have stable releases and unstable releases except the unstable releases get proper versions in pypi so they are pip installed | 16:32 |
| clarkb | I think thatmeans our cap is actually the correct fix here, but I'm not sure that is a very user friendly release mechanism | 16:32 |
| *** saneax has quit IRC | 16:34 | |
| fungi | that's what pip/pypi added prerelease versioning to handle. if a version is a prerelease it won't be installed unless requested | 16:36 |
| *** jpena is now known as jpena|off | 16:56 | |
| *** hamalq has joined #zuul | 17:01 | |
| *** iurygregory has quit IRC | 17:10 | |
| *** mattd01 has quit IRC | 17:12 | |
| clarkb | https://review.opendev.org/752724 passes unittests now. Getting that approved to fix nodepool testing would be great | 17:13 |
| clarkb | (its still running check though) | 17:13 |
| corvus | clarkb: version differences seem weird; out of curiosity, why aren't the caps <15.0.0 for all? | 17:18 |
| corvus | (but also +2) | 17:19 |
| clarkb | corvus: they don't release them in sync with each other. https://azure.github.io/azure-sdk/releases/latest/all/python.html is a giant page of libs and their disparate versioning | 17:19 |
| clarkb | (also its not up to date today due to the new releases) | 17:20 |
| clarkb | I capped at less than today's release versions | 17:20 |
| *** sanjayu_ has quit IRC | 17:29 | |
| *** mattd01 has joined #zuul | 18:12 | |
| *** irclogbot_3 has quit IRC | 18:19 | |
| *** irclogbot_0 has joined #zuul | 18:25 | |
| *** irclogbot_0 has quit IRC | 18:31 | |
| *** irclogbot_0 has joined #zuul | 18:37 | |
| openstackgerrit | Clark Boylan proposed zuul/nodepool master: Stop using fedora-30 https://review.opendev.org/752743 | 18:37 |
| *** mattd01 has left #zuul | 18:52 | |
| clarkb | looks like fedora has dropped zookeper | 19:10 |
| clarkb | tristanC: ^ any thoughts on the best way to run it on fedora-32 for the openshift job? | 19:11 |
| clarkb | also if we're supplying it ourselves maybe we should switch that to centos8? | 19:11 |
| clarkb | doesn't look like epel has it so we either install from the upstream tarball (what I do on suse) or run the container? | 19:20 |
| tristanC | clarkb: perhaps we could use a ensure-zookeeper to do that, or switch to using a container | 19:32 |
| clarkb | looking more closely at the job we run oc on centos and nodepool on fedora | 19:50 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: wip: ensure-zookeeper: add role to setup zookeeper https://review.opendev.org/752757 | 19:52 |
| clarkb | tristanC: should I update my change to use ^ or do you plan to do that? | 19:55 |
| openstackgerrit | Tristan Cacqueray proposed zuul/nodepool master: wip: replace zookeeper package by role for openshift test https://review.opendev.org/752758 | 19:56 |
| tristanC | clarkb: well let's see how that goes ^ | 19:56 |
| openstackgerrit | Merged zuul/nodepool master: Cap azure dep versions due to breaking changes https://review.opendev.org/752724 | 19:58 |
| clarkb | after looking at the job more closely I think other options available to us are running the launcher on ubuntu and installing oc similar to how we install zk in your change. Or to use the built docker images in that job which includes oc | 19:58 |
| tristanC | clarkb: we might want a common ensure-zookeeper role in zuul-jobs to re-use the setup of zk-ca | 20:00 |
| clarkb | ya I think the role is useful either way | 20:00 |
| clarkb | just brainstorming options there | 20:06 |
| clarkb | zbr: I've approved the periodic-weekly change. FOr https://review.opendev.org/#/c/748682/4 I wonder if we need to split that out so that we don't include the jobs that currently fail | 20:12 |
| clarkb | When we address those jobs they can be added to periodic-weekly too. Its a weird chicken and egg because we're modifying the file that define the jobs | 20:13 |
| clarkb | which forces them to run | 20:13 |
| openstackgerrit | Merged zuul/project-config master: Assure periodic-weekly emails to zuul-jobs-failures https://review.opendev.org/748706 | 20:14 |
| *** veecue has joined #zuul | 20:33 | |
| *** veecue has quit IRC | 20:36 | |
| zbr | too late now but imho we should make them nv, that is the scope of nv, to still run a give time to fix them without blocking other changes. | 21:00 |
| zbr | i prefer to see few them as nv until they are either fixed or we decide to remove them. i see this as a natural process. | 21:02 |
| tristanC | zbr: I guess it makes sense to run the existing test periodically, though i now wonder if they are appropriate in zuul-jobs after all. What are zuul-jobs-failures@lists.zuul-ci.org readers expected to do when a job fails because upstream-x doesn't work on distro-y ? | 21:08 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: wip: ensure-zookeeper: add role to setup zookeeper https://review.opendev.org/752757 | 21:15 |
| *** rlandy has quit IRC | 21:18 | |
| tristanC | zbr: perhaps before using a periodic pipeline to run every test weekly, we could recheck https://review.opendev.org/748682 manually and see if that is useful | 21:26 |
| *** Goneri has quit IRC | 21:39 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: wip: ensure-zookeeper: add role to setup zookeeper https://review.opendev.org/752757 | 21:42 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: ensure-zookeeper: add role to setup zookeeper https://review.opendev.org/752757 | 22:11 |
| tristanC | clarkb: topic:ensure-zk seems to work, feel free to squash the nodepool change, otherwise i'll follow-up on monday | 22:18 |
| clarkb | tristanC: I think to start the idea is just to have the info | 22:22 |
| clarkb | tristanC: we'll get that both in the zuul db and email | 22:22 |
| clarkb | and from there we can see what needs to be done | 22:22 |
| clarkb | re ensure-zk ya I'll take a look at using it momentarily | 22:22 |
| openstackgerrit | Clark Boylan proposed zuul/nodepool master: Stop using fedora-30 https://review.opendev.org/752743 | 22:31 |
| clarkb | tristanC: ^ squashed | 22:31 |
| openstackgerrit | Clark Boylan proposed zuul/nodepool master: Use centos-8 to run openshift cluster in testing https://review.opendev.org/752769 | 22:32 |
| clarkb | I'll review ensure-zookeeper shortly too | 22:32 |
| *** hamalq has quit IRC | 23:02 | |
| *** EmilienM has quit IRC | 23:06 | |
| *** EmilienM has joined #zuul | 23:06 | |
| *** sanjayu_ has joined #zuul | 23:10 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: ensure-zookeeper: add role to setup zookeeper https://review.opendev.org/752757 | 23:27 |
| *** tosky has quit IRC | 23:31 | |
| *** sanjayu_ has quit IRC | 23:44 | |
| *** Goneri has joined #zuul | 23:44 | |
| *** sanjayu_ has joined #zuul | 23:44 | |
| *** hamalq has joined #zuul | 23:48 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!