| *** swest has quit IRC | 01:38 | |
| *** swest has joined #zuul | 01:54 | |
| *** irclogbot_3 has joined #zuul | 02:07 | |
| *** irclogbot_3 has quit IRC | 02:13 | |
| *** irclogbot_1 has joined #zuul | 02:17 | |
| *** irclogbot_1 has quit IRC | 02:19 | |
| *** altlogbot_0 has joined #zuul | 02:19 | |
| *** altlogbot_0 has quit IRC | 02:34 | |
| *** irclogbot_3 has joined #zuul | 03:21 | |
| *** irclogbot_3 has quit IRC | 03:25 | |
| *** rlandy|bbl is now known as rlandy | 03:36 | |
| *** hwangbo has quit IRC | 03:49 | |
| *** bhavikdbavishi has joined #zuul | 03:51 | |
| *** bhavikdbavishi1 has joined #zuul | 03:53 | |
| *** altlogbot_2 has joined #zuul | 03:55 | |
| *** bhavikdbavishi has quit IRC | 03:55 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:55 | |
| *** altlogbot_2 has quit IRC | 03:59 | |
| *** jamesmcarthur has joined #zuul | 04:21 | |
| *** jamesmcarthur has quit IRC | 04:40 | |
| *** jamesmcarthur has joined #zuul | 04:41 | |
| *** jamesmcarthur has quit IRC | 04:46 | |
| *** jamesmcarthur has joined #zuul | 04:51 | |
| *** pcaruana has joined #zuul | 04:55 | |
| *** jamesmcarthur has quit IRC | 04:56 | |
| *** jamesmcarthur has joined #zuul | 05:02 | |
| *** jamesmcarthur has quit IRC | 05:07 | |
| *** saneax has joined #zuul | 06:04 | |
| *** altlogbot_0 has joined #zuul | 06:19 | |
| *** altlogbot_0 has quit IRC | 06:23 | |
| *** altlogbot_3 has joined #zuul | 06:25 | |
| *** altlogbot_3 has quit IRC | 06:29 | |
| *** saneax has quit IRC | 06:31 | |
| *** saneax has joined #zuul | 06:33 | |
| *** gtema_ has joined #zuul | 06:38 | |
| *** gtema_ has quit IRC | 06:47 | |
| *** altlogbot_1 has joined #zuul | 06:57 | |
| *** altlogbot_1 has quit IRC | 07:01 | |
| *** ianychoi_ is now known as ianychoi | 07:01 | |
| *** themroc has joined #zuul | 07:06 | |
| *** irclogbot_0 has joined #zuul | 07:17 | |
| *** tosky has joined #zuul | 07:20 | |
| *** irclogbot_0 has quit IRC | 07:21 | |
| AJaeger | corvus: isn't https://review.opendev.org/#/c/670206/ running a few tests too many? I don't understand why zuul-jobs-test-add-authorized-keys is running... | 07:25 |
|---|---|---|
| *** irclogbot_3 has joined #zuul | 07:26 | |
| *** irclogbot_3 has quit IRC | 07:27 | |
| *** hashar has joined #zuul | 07:28 | |
| *** irclogbot_2 has joined #zuul | 07:54 | |
| *** irclogbot_2 has quit IRC | 07:55 | |
| *** irclogbot_0 has joined #zuul | 08:09 | |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul master: Fix typo in debug message https://review.opendev.org/670245 | 08:12 |
| *** irclogbot_0 has quit IRC | 08:13 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 08:32 |
| *** bolg has joined #zuul | 08:41 | |
| *** bkorren has joined #zuul | 08:51 | |
| *** ianychoi has quit IRC | 08:54 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Optionally support mitogen for job execution https://review.opendev.org/657024 | 09:04 |
| *** gtema_ has joined #zuul | 09:07 | |
| *** ianychoi has joined #zuul | 09:38 | |
| *** gtema has joined #zuul | 09:42 | |
| *** gtema_ has quit IRC | 09:42 | |
| *** gtema has quit IRC | 09:43 | |
| *** irclogbot_2 has joined #zuul | 09:50 | |
| *** dmsimard4 has joined #zuul | 09:50 | |
| *** dmsimard has quit IRC | 09:51 | |
| *** irclogbot_2 has quit IRC | 09:51 | |
| *** gtema_ has joined #zuul | 09:53 | |
| *** tosky has quit IRC | 10:02 | |
| *** tosky has joined #zuul | 10:05 | |
| *** altlogbot_3 has joined #zuul | 10:28 | |
| *** altlogbot_3 has quit IRC | 10:29 | |
| *** gtema_ has quit IRC | 10:29 | |
| *** altlogbot_3 has joined #zuul | 10:35 | |
| *** hashar has quit IRC | 10:38 | |
| *** altlogbot_3 has quit IRC | 10:39 | |
| tobiash | corvus: looks like we have a test race in the cleanup playbook tests: http://logs.openstack.org/45/670245/1/gate/tox-py35/512740a/testr_results.html.gz | 10:40 |
| *** altlogbot_0 has joined #zuul | 10:42 | |
| *** irclogbot_3 has joined #zuul | 10:42 | |
| *** altlogbot_0 has quit IRC | 10:45 | |
| *** irclogbot_3 has quit IRC | 10:45 | |
| *** altlogbot_2 has joined #zuul | 10:57 | |
| *** altlogbot_2 has quit IRC | 11:01 | |
| *** badboy has joined #zuul | 11:15 | |
| badboy | hi | 11:15 |
| badboy | why do I keep having this: tmp/f9998be797c44af2a6fdde48798e8c37/ansible/inventory.yaml did not meet host_list requirements, check plugin documentation if this is unexpected | 11:16 |
| badboy | and no logger console is shown | 11:16 |
| *** hashar has joined #zuul | 11:37 | |
| *** altlogbot_1 has joined #zuul | 11:57 | |
| *** altlogbot_1 has quit IRC | 12:01 | |
| *** hashar has quit IRC | 12:10 | |
| *** hashar has joined #zuul | 12:12 | |
| openstackgerrit | Merged zuul/zuul master: Fix typo in debug message https://review.opendev.org/670245 | 12:15 |
| *** rlandy has joined #zuul | 12:30 | |
| *** irclogbot_1 has joined #zuul | 12:34 | |
| *** irclogbot_1 has quit IRC | 12:35 | |
| *** badboy has quit IRC | 12:45 | |
| *** altlogbot_0 has joined #zuul | 12:53 | |
| *** altlogbot_0 has quit IRC | 12:53 | |
| *** dmsimard4 is now known as dmsimard | 12:58 | |
| zbr | is it possible for a single repo/project to have different zuul queues? for example having a linting job running in a queue that is a different queue than the others, so it will not be slowed down by a potential long queue. | 13:06 |
| *** sshnaidm|ruck is now known as sshnaidm|rover | 13:20 | |
| *** arxcruz|rover is now known as arxcruz|ruck | 13:20 | |
| *** hashar_ has joined #zuul | 13:30 | |
| *** hashar has quit IRC | 13:32 | |
| *** bolg has quit IRC | 13:36 | |
| mordred | zbr: no, because queues are queues of changes, not of jobs - so a change to a project has to go into a queue to be processed | 13:37 |
| zbr | mordred: by looking at configs I kinda guessed but I had to ask. | 13:39 |
| zbr | mainly i was wondering if there is a workaround for priorietization of quick tests over the heavy ones. | 13:39 |
| pabelanger | you could have it in or out of a queue, depending on the jobs however. So, if you only ran lint jobs, and it didn't setup the queue, it should be independant | 13:40 |
| pabelanger | but, can't mix jobs | 13:40 |
| pabelanger | but, the more I think about it, I think I am wrong | 13:40 |
| pabelanger | because I was thinking filematcher and queue logic would come into dynamic play | 13:41 |
| zbr | my impression is that I cannot really have both queues and jobs that run asap, maybe only if I would have a different zuul server doing it. | 13:42 |
| pabelanger | right, even if you could, that could get pretty confusing fast I think | 13:43 |
| pabelanger | trying to mental map which jobs were in queue A, then the other | 13:43 |
| mordred | zbr: the other thing you can do is make your expensive jobs run after your quick jobs - but we've found in the past that this doesn't wind up being as pleasing as one might think, because in the success case it actually increases total test time | 13:45 |
| fungi | zbr: those jobs would run asap if your zuul has available nodes. if not they are prioritized according to various rules (pipeline priority, queue activity, et cetera) | 13:48 |
| *** michael-beaver has joined #zuul | 13:49 | |
| pabelanger | Yah, I've usually found, faster jobs is asking mgmt for more funding and access to more testing resources | 13:56 |
| *** bkorren has quit IRC | 14:09 | |
| corvus | AJaeger: yes, i think it's picking up the changes in the dependency list; i'll see if i can tweak that | 14:14 |
| zbr | bit offtopic but I am wondering which is real avg success rate for a build (not individual job), my guesstimate is far less than 50% which means that optimizing for minimal time on best-scenario may have some unexpected side effects. | 14:18 |
| corvus | zbr: a "build" is one run of an individual job. | 14:21 |
| zbr | s/build/chage | 14:25 |
| AJaeger | corvus: yeah, looks like it. Doubt we have it that often that it's a problem but nice optimization. | 14:30 |
| fungi | zbr: you're probably thinking of a "buildset" then? | 14:32 |
| fungi | (a gruop of individual builds which all ran at the same time in the same pipeline for the same change/commit) | 14:33 |
| corvus | those data are not hard to get from the api or the sql database | 14:33 |
| corvus | (or even the code review system) | 14:33 |
| corvus | no need to guess | 14:34 |
| corvus | AJaeger: i think this is a behavior quirk of the check pipeline -- we don't generate the layout for the item ahead in check because we don't actually run jobs on it, so the "diff" it performs is against the currently running tenant layout, so it sees all of the changes in the series. but in gate, it would have the layout for the item ahead generated, and it should only run the one job. | 14:43 |
| corvus | to fix this, we'll have to waste a little bit more work generating layouts for non-live items in the check pipeline. but we can probably just do it for the first non-live item. | 14:44 |
| AJaeger | corvus: I'm fine considering this an optimization - and so, if that is too expensive, we can also keep status quo. | 14:48 |
| corvus | AJaeger: i'm looking at the code now | 14:51 |
| *** saneax has quit IRC | 14:54 | |
| *** bolg has joined #zuul | 15:04 | |
| *** hashar_ has quit IRC | 15:04 | |
| *** bolg has quit IRC | 15:08 | |
| *** pcaruana has quit IRC | 15:11 | |
| tobias-urdin | having some issues with depends-on, seems like the url is the problem, i'm using a newer gerrit version and zuul 3.9.0 | 15:18 |
| tobias-urdin | here is a reproduce http://paste.openstack.org/show/754303/ | 15:18 |
| tobias-urdin | maybe it's my proxy messing the url up, not sure, it works when removing the "/#" probably because it can't match both "/#" and "/+" | 15:19 |
| clarkb | project names in the url may not be handled either? | 15:20 |
| tobias-urdin | oh yeah prob, gerrit version is 2.15.7 with the old theme not the new polygerrit stuff | 15:21 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator https://review.opendev.org/668029 | 15:26 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Build layout of most recent non-live item https://review.opendev.org/670335 | 15:27 |
| fungi | tobias-urdin: you can try dropping the project name and as long as there is only one change 4255 in that gerrit it ought to still work | 15:27 |
| corvus | AJaeger: ^ there's a fix (hopefully) | 15:27 |
| fungi | tobias-urdin: but with notedb, there is no longer a guarantee that the change numbers are unique | 15:27 |
| corvus | well, it'd be great to support the url as used :) | 15:28 |
| fungi | i concur, sounds like it may be a zuul bug | 15:28 |
| corvus | if the change numbers are not unique, we'll have to start including the project name in them | 15:28 |
| tobias-urdin | fungi: ack, kind of sucks if they moved away from the change number not being unique :( | 15:28 |
| corvus | the increasingly inaccurately named change "number" | 15:29 |
| fungi | when zuul queries review.domain.tld for change 4255 without specifying puppet-ceph it should get back a list of changes, usually a list of length 1 | 15:29 |
| AJaeger | thanks, corvus | 15:29 |
| fungi | but it could be longer than 1 | 15:29 |
| corvus | (they were only ever a number in github in the sense that every sha1 is a number) | 15:29 |
| corvus | well, patchset at any rate | 15:30 |
| fungi | i mean, if you take the ascii byte values of puppet-ceph they're a number too ;) | 15:30 |
| fungi | (but yeah, you have type conversions to add the change integer) | 15:30 |
| *** rfolco is now known as rfolco_brb | 15:43 | |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Create tasks that spin up kubernetes and run the operator https://review.opendev.org/668029 | 15:49 |
| *** tosky has quit IRC | 15:53 | |
| *** pcaruana has joined #zuul | 15:57 | |
| *** hwangbo has joined #zuul | 16:05 | |
| corvus | fyi: https://review.opendev.org/670348 Move zuul/ repos to Zuul tenant | 16:27 |
| *** mattw4 has joined #zuul | 16:32 | |
| tobiash | \o/ | 16:41 |
| pabelanger | +2! | 16:41 |
| pabelanger | but not in a place to monitor | 16:41 |
| *** altlogbot_1 has joined #zuul | 16:46 | |
| *** altlogbot_1 has quit IRC | 16:49 | |
| corvus | +w. i'll monitor. i know some things are going to break (just because we need to switch to the new jobs). i'll plan on doing that as well as fixing anything unexpected. | 16:53 |
| *** hashar has joined #zuul | 16:59 | |
| *** themroc has quit IRC | 17:02 | |
| *** weshay is now known as weshay|rover | 17:12 | |
| *** sshnaidm|rover is now known as sshnaidm|off | 17:12 | |
| *** hashar has quit IRC | 17:18 | |
| *** altlogbot_2 has joined #zuul | 17:25 | |
| *** altlogbot_2 has quit IRC | 17:25 | |
| *** rfolco_brb is now known as rfolco | 17:27 | |
| *** altlogbot_2 has joined #zuul | 17:32 | |
| *** altlogbot_2 has quit IRC | 17:35 | |
| *** altlogbot_0 has joined #zuul | 17:37 | |
| *** altlogbot_0 has quit IRC | 17:41 | |
| AJaeger | corvus: the change for zuul tenant merged 40 mins ago, how long does it take until it gets applied? Last reconfigure on zuul.opendev.org was yesterday... | 17:46 |
| *** electrofelix has quit IRC | 17:50 | |
| fungi | tenant configs get explicitly loaded by ansible/puppet pulses | 17:50 |
| * fungi checks the current frequency | 17:50 | |
| fungi | last run completed at 17:41:29 | 17:51 |
| AJaeger | mmh, ten minutes ago - but http://zuul.opendev.org/t/zuul/status still has "Last reconfigured: Wed Jul 10 2019 21:32:19 GMT+0200 (Central European Summer Time)" | 17:52 |
| fungi | previous run completed at 17:11:34 | 17:52 |
| AJaeger | thanks for checking, fungi | 17:52 |
| fungi | i expect it merged after the last pulse began | 17:52 |
| fungi | and they're occurring ~30 minutes apart | 17:52 |
| fungi | so expect it to happen again at ~18:11z | 17:53 |
| AJaeger | it merged 17:04, so should have been in the 17:41 pulse, shouldn't it? | 17:53 |
| fungi | hrm, yeah probabl | 17:53 |
| fungi | y | 17:53 |
| AJaeger | We can also wait another pulse... | 17:53 |
| fungi | i'll check the config on the scheduler | 17:53 |
| AJaeger | but it might have failed to load due to bugs and therefore no reconfig happened | 17:54 |
| AJaeger | thanks, fungi | 17:54 |
| corvus | do we need to restart it to get it to load with errors? | 17:54 |
| *** bjackman_ has quit IRC | 17:55 | |
| fungi | looks like /etc/zuul/layout/main.yaml was last modified 17:11z and has zuul/zuul in the zuul tenant's untrusted-projects list | 17:55 |
| fungi | Jul 11 17:11:33 zuul01 puppet-user[30424]: (/Stage[main]/Zuul::Scheduler/Exec[zuul-reload]) Triggered 'refresh' from 1 events | 17:56 |
| corvus | remote: https://review.opendev.org/670359 Fix typo in Zuul tenant config | 17:56 |
| *** rlandy_ has joined #zuul | 17:56 | |
| corvus | fungi, AJaeger: ^ seems our validation of the tenant config is less than perfect | 17:57 |
| AJaeger | oops | 17:57 |
| AJaeger | +2 | 17:57 |
| corvus | voluptuous.error.MultipleInvalid: not a valid value @ data['source']['untrusted-projects'][1449]['zuul/nodepool']['include'][0] | 17:57 |
| corvus | that's in the scheduler log ^ | 17:57 |
| fungi | it's approved | 17:57 |
| AJaeger | fungi: but not single-core ;) | 17:58 |
| fungi | AJaeger: it was when i started typing! ;) | 17:58 |
| AJaeger | fungi: I was a few seconds faster :) | 17:58 |
| *** rlandy_ has quit IRC | 17:58 | |
| * AJaeger didn't type :) | 17:58 | |
| *** rlandy_ has joined #zuul | 17:58 | |
| *** rlandy has quit IRC | 17:59 | |
| corvus | AJaeger: the fix from earlier is going to take some refactoring of The Algorithm, so will take a bit more time. but it's going to be fun. :) | 17:59 |
| AJaeger | ;) | 18:00 |
| *** jeliu_ has joined #zuul | 18:02 | |
| mordred | corvus: ooh - touching The Algorithm | 18:03 |
| *** rlandy_ is now known as rlandy | 18:05 | |
| fungi | very touching | 18:06 |
| SpamapS | so algorithm | 18:12 |
| AJaeger | SpamapS: Algorithm with Captial A ;) | 18:13 |
| SpamapS | No that was an intentional adjectivication. ;) | 18:14 |
| AJaeger | ah | 18:15 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Add Kubernetes Operator Functional Test Job https://review.opendev.org/668029 | 18:48 |
| AJaeger | corvus, fungi, reconfiguration happened - and we have errors... | 18:50 |
| corvus | \o/ | 18:51 |
| fungi | woo! errors are better than radio silence ;) | 18:51 |
| AJaeger | yep - now we know what to fix... | 18:51 |
| AJaeger | just 10 errors ;) | 18:51 |
| fungi | that's within an order of magnitude of 1 error, which is almost no errors. success! | 18:52 |
| corvus | jeliu_: ^ just so you know, the Zuul error on your change is my fault -- i'm in the middle of moving all the zuul projects to their own tenant and it happened to catch your change. i'll let you know how to proceed in a few minutes. | 18:52 |
| corvus | where *do* we define our nodesets anyway? :) | 18:52 |
| fungi | we probably *should* define our nodesets for opendev in opendev/project-config, right? | 18:53 |
| fungi | whether or not we do yet is i guess the real question | 18:53 |
| corvus | yeah, looks like openstack-zuul-jobs defines them in the openstack tenant | 18:54 |
| *** bhavikdbavishi has quit IRC | 19:04 | |
| *** bhavikdbavishi has joined #zuul | 19:06 | |
| jeliu_ | corvus: sounds good | 19:08 |
| *** rlandy is now known as rlandy|brb | 19:10 | |
| *** tosky has joined #zuul | 19:26 | |
| *** jeliu_ has quit IRC | 19:30 | |
| *** altlogbot_3 has joined #zuul | 19:40 | |
| *** altlogbot_3 has quit IRC | 19:41 | |
| *** bhavikdbavishi has quit IRC | 19:47 | |
| *** rlandy|brb is now known as rlandy | 20:02 | |
| *** irclogbot_3 has joined #zuul | 20:15 | |
| *** irclogbot_3 has quit IRC | 20:16 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Switch to opendev release jobs https://review.opendev.org/670388 | 20:16 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Switch to opendev release/docs jobs https://review.opendev.org/670388 | 20:25 |
| *** mattw4 has quit IRC | 20:31 | |
| *** pcaruana has quit IRC | 20:31 | |
| *** mattw4 has joined #zuul | 20:31 | |
| *** igordc has joined #zuul | 20:32 | |
| openstackgerrit | James E. Blair proposed zuul/project-config master: Add publish-zuul-website job https://review.opendev.org/670391 | 20:34 |
| daniel2 | so the docker compose file included with the example for zuul has a hook that waits for gearman, but gearman isn't actually in the docker-compose yaml file. | 20:43 |
| fungi | daniel2: the zuul scheduler should fork off a gearman process of its own | 20:45 |
| fungi | unless configured not to | 20:46 |
| daniel2 | fungi: https://shafer.cc/paste/view/f3c39f02 This is all I keep getting | 20:53 |
| *** altlogbot_2 has joined #zuul | 21:02 | |
| *** altlogbot_2 has quit IRC | 21:05 | |
| *** hwangbo has quit IRC | 21:07 | |
| *** hwangbo has joined #zuul | 21:08 | |
| daniel2 | Looking at that script wait-to-start-gearman.sh, I'm not even sure how this is verifying gearman is up: cat < /dev/null > /dev/tcp/scheduler/4730 && return | 21:11 |
| clarkb | daniel2: its waiting for the write to the gearman socket to complete | 21:12 |
| clarkb | gearman listens on tcp port 4730 | 21:12 |
| clarkb | daniel2: did you make changes to the quickstart compose file? | 21:13 |
| clarkb | I wonder if either the gearman server has been disabled in config or if firewall rules or similar prevent the connection | 21:14 |
| daniel2 | clarkb: I updated the volume section and changed the mysql passwords. | 21:14 |
| clarkb | hrm neither of those should affect my two guesses | 21:14 |
| clarkb | can you docker exec into the zuul scheduler container and then check what zuul processes are running and what ports are being listened on? | 21:15 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: [WIP] Verify Operator Pod Running https://review.opendev.org/670395 | 21:29 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: Add dstat graph roles https://review.opendev.org/670402 | 22:02 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: Add dstat graph roles https://review.opendev.org/670402 | 22:03 |
| daniel2 | clarkb: tcp 0 0 127.0.0.11:39989 0.0.0.0:* LISTEN - | 22:05 |
| daniel2 | Thats all I got. | 22:05 |
| clarkb | daniel2: what about processes? | 22:05 |
| clarkb | there should be two zuul-scheduler processes | 22:05 |
| daniel2 | these images dont even have psutils installed :D | 22:05 |
| daniel2 | I'm just waiting for it to finish installing | 22:06 |
| fungi | ought to be able to see them from the host os | 22:06 |
| fungi | they're just namespaced processes anyway | 22:06 |
| clarkb | ya it just becomes trickier to filter out everything else | 22:06 |
| fungi | or you mean the host images lack psutils? | 22:06 |
| daniel2 | yeah, how do you see the processes for a single container? | 22:06 |
| daniel2 | no the docker container | 22:06 |
| clarkb | I think you can pstree under the docker container runc process (or whatever that process is called these days | 22:07 |
| daniel2 | https://shafer.cc/paste/view/1e49ce00 | 22:07 |
| daniel2 | Thats the processes running | 22:07 |
| daniel2 | I think it's expecting gearman to be on a remote host? | 22:08 |
| fungi | that doesn't look like the scheduler started either | 22:09 |
| fungi | if that's the scheduler container | 22:09 |
| daniel2 | It wont start because its trying to connect to gearman, however, there is no gearman. | 22:09 |
| clarkb | `/bin/bash /var/playbooks/` doesn't look right either | 22:09 |
| fungi | yeah, the gearman service is in the scheduler container | 22:09 |
| clarkb | ya there is no german because there is no scheduler | 22:09 |
| daniel2 | clarkb: - "./playbooks/:/var/playbooks/:z" | 22:10 |
| daniel2 | It creates a volume with that. | 22:10 |
| clarkb | ah well it seems that isn't completing and continuing to the other side of hte && whcih starts the zuul scheduler | 22:10 |
| clarkb | so I guess it could be related to your volume changes afterall | 22:10 |
| daniel2 | I didnt change that one. | 22:11 |
| daniel2 | Just the sshkeys and logs one. | 22:11 |
| clarkb | you'll need to figure out why that is hanging | 22:11 |
| clarkb | could try stracing it | 22:11 |
| daniel2 | Its hanging because it can't connect to gearman | 22:11 |
| fungi | it can't connect to gearman because the scheduler process isn't starting (or is insta-crashing) | 22:12 |
| clarkb | daniel2: `/usr/bin/dumb-init -- sh -c /var/playbooks/ && zuul-scheduler -d` is the command that will start the scheduler process | 22:12 |
| clarkb | that command seems to be hanging | 22:12 |
| clarkb | that is not the command that checks if gearman is started | 22:12 |
| clarkb | /bin/bash /var/playbooks/ <- seems to be the problem I think | 22:12 |
| fungi | the scheduler daemon starts and shortly thereafter forks a new process which listens on the gearmanport | 22:13 |
| fungi | under normal circumstances | 22:13 |
| clarkb | that should exit (still not quite sure what it is doing) then it will start the zuul-scheduler | 22:13 |
| clarkb | "sh -c '/var/playbooks/wait-to-start.sh && zuul-scheduler -d'" is what the command should be according to my zuul checkout | 22:14 |
| clarkb | and that is waiting for gerrit to start iirc | 22:15 |
| *** bolg has joined #zuul | 22:16 | |
| clarkb | daniel2: can you double check that ^ the comamnd I pasted just above is what you still hve in your docker compose file? then the other thing I would do is strace that process | 22:18 |
| daniel2 | I'm trying the default docker-compose file. | 22:18 |
| daniel2 | Removed all my changes. | 22:18 |
| *** bolg has quit IRC | 22:19 | |
| *** altlogbot_2 has joined #zuul | 22:20 | |
| *** irclogbot_0 has joined #zuul | 22:20 | |
| daniel2 | So it looks like my changes did break it. | 22:20 |
| daniel2 | the sshkey volumes should be blank as it generates them on compose up. | 22:21 |
| daniel2 | Now it's having issues with zookeeper . | 22:21 |
| fungi | progress! | 22:21 |
| daniel2 | scheduler_1 | 2019-07-11 22:21:03,125 WARNING zuul.zk.ZooKeeper: Retrying zookeeper connection | 22:22 |
| daniel2 | Thats all I got. | 22:22 |
| fungi | i guess we need to explicitly say in the quickstart that the sshkey volumes need to be empty | 22:22 |
| daniel2 | yeah, I thought it was a fill in the blank :D | 22:22 |
| clarkb | daniel2: for that you should check if zookeeper/java process is running and if so that it is listening on port 2182 (I think that is the right port) | 22:22 |
| fungi | just need somewhere durable to write the keys | 22:22 |
| *** altlogbot_2 has quit IRC | 22:23 | |
| *** irclogbot_0 has quit IRC | 22:24 | |
| *** rfolco has quit IRC | 22:24 | |
| *** rfolco has joined #zuul | 22:24 | |
| daniel2 | zookeeper container is running, netstat shows the ports are listening. Technically they are only listening on dockers local net | 22:26 |
| clarkb | is that warning you got persistent or did it go away? | 22:27 |
| daniel2 | oh good lord. It was because my configs are wrong. | 22:27 |
| clarkb | could just be a race between zuul starting and zookeeper starting | 22:27 |
| daniel2 | I tried to just copy and paste my old configs to the docker setup, which made the hosts wrong. | 22:28 |
| fungi | that could certainly do it | 22:31 |
| daniel2 | Is it Friday yet? | 22:34 |
| fungi | utc friday in t minus 85 minutes | 22:35 |
| daniel2 | Only 4:36pm here in Idaho. | 22:36 |
| daniel2 | I'm still at work, and it's too late to try and beat traffic so I'll probably stay till 6 since I live 26 miles away. | 22:36 |
| fungi | i'm in the atlantic ocean, but i live on utc out of convenience | 22:37 |
| daniel2 | Hows the fish? | 22:37 |
| fungi | good right now. lots of tilefish and wahoo | 22:37 |
| daniel2 | fungi: when you say you live in the atlantic this is all I could think about: https://www.youtube.com/watch?v=j1kftCx5-tA | 22:38 |
| fungi | i would also have accepted a clip from sealab 2021 | 22:40 |
| openstackgerrit | Merged zuul/project-config master: Add publish-zuul-website job https://review.opendev.org/670391 | 22:40 |
| fungi | but no, i'm technically on a sandbar ~10mi offshore | 22:40 |
| daniel2 | I remember that show. The old school adult swim block. | 22:40 |
| fungi | trying to compute during breaks from battling with my lawn | 22:40 |
| fungi | (grass *does* grow in sand, too well unfortunately) | 22:41 |
| daniel2 | Neighbors called the city on me for my lawn. I told the city to shove it since they have no reign over my lawn. There is no homeowners association where I live. | 22:41 |
| fungi | my town council will fine me us$50/day if my lawn gets too tall. they've already threatened after the jerk non-resident homeowner neighbors complained | 22:43 |
| fungi | i suppose i could challenge them... they could also turn off my fresh water supply and stop picking up my trash in retaliation | 22:44 |
| daniel2 | my neighbor is some old "get off my lawn" retired guy. He hated that a young couple bought a house on their street. | 22:45 |
| clarkb | daniel2: idaho has traffic? (also yay PNW) | 22:45 |
| clarkb | though now I wonder if idaho counts as PNW? I don't actually know | 22:45 |
| daniel2 | It does have traffic on the freeway. Boise currently is the fastest growing city in the US. | 22:45 |
| daniel2 | So our infrastructure can't handle the influx of residents. | 22:46 |
| clarkb | wikipedia says yes | 22:46 |
| daniel2 | Not only that, I work in Downtown Boise, and I live in a city called Caldwell 30 miles away | 22:46 |
| daniel2 | clarkb: PNW? | 22:46 |
| clarkb | pacific northwest | 22:46 |
| clarkb | I'm in portland | 22:46 |
| daniel2 | Ah, oregon is like 30 miles from me. | 22:46 |
| daniel2 | I do a trip to oregen every other month... for medical reasons. | 22:47 |
| fungi | hah. not going to dig into that one | 22:48 |
| *** mattw4 has quit IRC | 22:54 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add dstat graph roles https://review.opendev.org/670402 | 22:55 |
| *** mattw4 has joined #zuul | 22:56 | |
| *** altlogbot_1 has joined #zuul | 22:58 | |
| *** altlogbot_1 has quit IRC | 22:59 | |
| *** irclogbot_3 has joined #zuul | 23:02 | |
| *** mattw4 has quit IRC | 23:03 | |
| *** mattw4 has joined #zuul | 23:03 | |
| *** irclogbot_3 has quit IRC | 23:06 | |
| *** tosky has quit IRC | 23:07 | |
| *** irclogbot_1 has joined #zuul | 23:10 | |
| *** irclogbot_1 has quit IRC | 23:16 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Switch to opendev release/docs jobs https://review.opendev.org/670388 | 23:20 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Switch to zuul docs jobs https://review.opendev.org/670415 | 23:43 |
| *** armstrongs has joined #zuul | 23:48 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Switch to zuul docs jobs https://review.opendev.org/670415 | 23:54 |
| *** mattw4 has quit IRC | 23:56 | |
| *** dmsimard4 has joined #zuul | 23:58 | |
| *** dmsimard has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!