| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Build a devstack-gate image with diskimage-builder https://review.openstack.org/38871 | 00:03 |
|---|---|---|
| mordred | there's some improvements after kexecing and playing with the first image | 00:03 |
| clarkb | I think grenade is failing 100% of the time now | 00:15 |
| clarkb | something snuck past the gate. Grenade is failing in the euca.sh exercises because a volume is not coming active within 180 seconds | 00:16 |
| clarkb | maybe that should be advertised so that people stop +Aing stuff until it is fixed? | 00:16 |
| clarkb | ok, back to my weekend | 00:17 |
| jeblair | I'm going to restart jenkins now to get the scp plugin update | 00:22 |
| *** hartsocks1 has joined #openstack-infra | 00:25 | |
| *** hartsocks has quit IRC | 00:25 | |
| jeblair | #status log restarted jenkins to update scp plugin | 00:25 |
| clarkb | jeblair: you reverified those nova and glance changes :( | 00:30 |
| jeblair | clarkb: i did; their report was inaccurate. | 00:31 |
| jeblair | (lost instead of failed) | 00:31 |
| *** UtahDave has joined #openstack-infra | 00:31 | |
| clarkb | ya, but none of them will merge | 00:31 |
| jeblair | yes, but when someone looks at them, they will see that they failed because something is broken, not because i aborted the tests. | 00:32 |
| openstackgerrit | A change was merged to openstack-infra/config: Add progress and estimated completion times to zuul status https://review.openstack.org/38598 | 00:32 |
| jeblair | clarkb: is there a change that will fix things? i'd be happy to run that instead, but lacking that, i don't see how this is hurting anything. | 00:33 |
| openstackgerrit | A change was merged to openstack-infra/config: Periodically test devstack-node availability https://review.openstack.org/38186 | 00:33 |
| clarkb | jeblair: I am not aware of a change that fixes grenade | 00:33 |
| clarkb | jeblair: not necessarily hurting anything just of minimal value imo | 00:33 |
| openstackgerrit | A change was merged to openstack-infra/config: Add hieraedit.py https://review.openstack.org/38003 | 00:34 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Report an unknown time if a change ahead has failed https://review.openstack.org/38715 | 00:35 |
| jeblair | mordred: we do report success/failure of every job to statsd. | 00:36 |
| jeblair | have for 6 months now | 00:36 |
| clarkb | jeblair: do we distinguish pipelines in graphite? | 00:36 |
| clarkb | I should dig more in graphite because it really is better for the coutning of stuff | 00:36 |
| *** rainya has quit IRC | 00:37 | |
| jeblair | grr, i'm wrong, we don't do success/fail, only run count | 00:37 |
| *** rainya has joined #openstack-infra | 00:41 | |
| *** blamar has quit IRC | 00:46 | |
| *** blamar has joined #openstack-infra | 00:47 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Stats: break out job runtimes and counts by result https://review.openstack.org/38942 | 00:48 |
| jeblair | clarkb, mordred: ^ let's fix that. :) | 00:48 |
| mordred | woot | 00:48 |
| jeblair | (that's going to break the center graph on the status page, but should be easy to fix) | 00:49 |
| jeblair | so... i wonder if the thing that broke grenade landed on the grizzly branch? | 00:50 |
| clarkb | jeblair: can we also break out per pipeline? | 00:50 |
| jeblair | clarkb: good point | 00:50 |
| clarkb | since check trends are probably less interesting than gate trends | 00:51 |
| jeblair | clarkb, mordred: do you know if anyone looked into the possibility grizzly broke the grenade gate? | 00:53 |
| fungi | it's also possible grizzly insta-broke due to some sort of bitrot... | 00:53 |
| mordred | jeblair: I do not know - I haven't looked at grenade at all | 00:53 |
| jeblair | it is an asymmetrical gate ATM. | 00:53 |
| * fungi sees if this airplane will let him look at the periodic jobs for stable | 00:53 | |
| mordred | awesome | 00:53 |
| *** UtahDave has quit IRC | 00:53 | |
| jeblair | (i warned about it, but people really wanted it gating) | 00:54 |
| fungi | periodic-tempest-devstack-vm-stable-grizzly seems fine | 00:55 |
| jeblair | fungi: that would be -- it is gated | 00:55 |
| jeblair | fungi: but grenade upgrades from grizzly to master, but only changes to master run the grenade test | 00:56 |
| jeblair | clarkb suggested 'something snuck past the gate'; that's the most obvious place to check since we know that that path is not tested. | 00:56 |
| jeblair | just throwing it out there. | 00:57 |
| fungi | jeblair: i was checking for externally influenced bitrot on grizzly's devstack jobs in general, which wouldn't necessarily be protected by gating, but yeah | 00:57 |
| jeblair | fungi: ah, yeah | 00:57 |
| clarkb | jeblair: ooh good point, I think that is super probable | 00:58 |
| fungi | also, it would have to be some pretty bad bitrot since it would be something impacting just setting up a grizzly devstack in that case. was a long shot | 00:58 |
| * clarkb looks at logs more | 00:58 | |
| jeblair | https://review.openstack.org/#/c/28082/ | 00:59 |
| jeblair | https://review.openstack.org/#/q/status:merged+branch:stable/grizzly,n,z | 00:59 |
| jeblair | clarkb: that gives something to check log timestamps against. however, that first change (the one i linked to) is raising a flag because it has to do with volume attachments. | 01:00 |
| clarkb | jeblair: the first pass of exercises which I assume run against the first grizzly install pass | 01:00 |
| clarkb | then the second run of exercises is where euca.sh fails. | 01:00 |
| clarkb | so it can't be something purely in grizzly, and would have to be something in grizzly that affects the ability to upgrade and have working volumes | 01:01 |
| *** sarob has joined #openstack-infra | 01:01 | |
| clarkb | jeblair: according to logstash grenade was regurly passing in the gate up to 0800UTC 2013-07-27 | 01:02 |
| clarkb | 28082 was merged after that so is still a suspect | 01:02 |
| clarkb | however I haven't indexed that far yet | 01:03 |
| clarkb | (I am hoping the logstash system catches up over the weekend then will be in better shape next week with the extra worker nodes) | 01:03 |
| *** yamahata has quit IRC | 01:04 | |
| *** sarob has quit IRC | 01:06 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Stats: break out job runtimes and counts by result https://review.openstack.org/38942 | 01:07 |
| jeblair | clarkb: that wasn't easy! | 01:07 |
| clarkb | jeblair: looking at irc logs it looks like at least 2 changes merged after 28082 | 01:10 |
| clarkb | and https://jenkins.openstack.org/job/gate-grenade-devstack-vm/22127/console got past that timeout | 01:12 |
| clarkb | I wonder if that timeout is simply too conservative? | 01:12 |
| clarkb | jeblair: looks like you will need to update the stats test checks to use the new path | 01:13 |
| jeblair | yeah, time to resume wknd now though. | 01:16 |
| *** dkliban has quit IRC | 01:17 | |
| clarkb | me too | 01:20 |
| clarkb | cooking pierogies for dinner | 01:20 |
| *** yamahata has joined #openstack-infra | 01:21 | |
| *** maurosr has quit IRC | 01:31 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Build a devstack-gate image with diskimage-builder https://review.openstack.org/38871 | 01:34 |
| mordred | lifeless: there it is with extra-data and whatnot | 01:34 |
| *** rainya has quit IRC | 01:37 | |
| *** rainya has joined #openstack-infra | 01:37 | |
| *** maurosr has joined #openstack-infra | 01:38 | |
| *** rainya has quit IRC | 01:45 | |
| *** maurosr has quit IRC | 01:45 | |
| *** rainya has joined #openstack-infra | 01:46 | |
| *** rainya has quit IRC | 01:52 | |
| *** westmaas has quit IRC | 01:54 | |
| *** rainya has joined #openstack-infra | 01:54 | |
| *** maurosr has joined #openstack-infra | 01:55 | |
| *** westmaas has joined #openstack-infra | 01:55 | |
| mordred | lifeless, clarkb: 10 minutes to make the image with caches and stuff | 01:58 |
| *** rainya has quit IRC | 01:59 | |
| *** mrodden1 has quit IRC | 02:00 | |
| *** mrodden has joined #openstack-infra | 02:01 | |
| *** maurosr has quit IRC | 02:01 | |
| *** rainya has joined #openstack-infra | 02:04 | |
| *** maurosr has joined #openstack-infra | 02:05 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Proof of concept of kexec/takeovernode https://review.openstack.org/38887 | 02:10 |
| mordred | ok. that script probably is terrible - but I have tested the sequence of command that it runs and I wound up with a working machine | 02:10 |
| *** sarob has joined #openstack-infra | 02:21 | |
| *** sarob has quit IRC | 02:26 | |
| *** westmaas has quit IRC | 02:51 | |
| *** sarob has joined #openstack-infra | 02:52 | |
| *** westmaas has joined #openstack-infra | 02:53 | |
| *** pcrews has quit IRC | 02:56 | |
| *** sarob has quit IRC | 02:57 | |
| *** yaguang has joined #openstack-infra | 03:03 | |
| *** vipul-away is now known as vipul | 03:13 | |
| *** mriedem1 has quit IRC | 03:13 | |
| *** sarob has joined #openstack-infra | 03:22 | |
| *** beagles has quit IRC | 03:23 | |
| *** sarob has quit IRC | 03:27 | |
| *** yaguang has quit IRC | 03:39 | |
| *** SergeyLukjanov has joined #openstack-infra | 03:52 | |
| *** mikal has quit IRC | 04:13 | |
| *** sarob has joined #openstack-infra | 04:23 | |
| *** sarob has quit IRC | 04:28 | |
| *** SergeyLukjanov has quit IRC | 04:30 | |
| *** vogxn has joined #openstack-infra | 04:31 | |
| *** sarob has joined #openstack-infra | 04:34 | |
| *** dkliban has joined #openstack-infra | 04:46 | |
| *** reed has quit IRC | 04:53 | |
| *** UtahDave has joined #openstack-infra | 05:12 | |
| *** dims has quit IRC | 05:30 | |
| *** dkehn has quit IRC | 05:38 | |
| *** dkehn has joined #openstack-infra | 05:39 | |
| *** nati_ueno has joined #openstack-infra | 05:44 | |
| Alex_Gaynor | does grenade actually pass? | 06:15 |
| *** nati_ueno has quit IRC | 06:20 | |
| clarkb | Alex_Gaynor: sometimes | 06:27 |
| Alex_Gaynor | clarkb: I guess I should just keep rechecking then? | 06:28 |
| clarkb | Alex_Gaynor: the failure it is hitting is a timeout creating a volume in the euca exercises | 06:28 |
| Alex_Gaynor | yup | 06:28 |
| clarkb | Alex_Gaynor: imo no, I would prefer that someone fix the volume creation | 06:28 |
| Alex_Gaynor | clarkb: me too, that's same underlying issue as the other tempest issue yesterday wasn't it? | 06:28 |
| clarkb | Alex_Gaynor: maybe? I am not quite sure what was making /dev/vdb problematic for tempest | 06:29 |
| *** sdake has quit IRC | 06:31 | |
| *** vogxn has quit IRC | 06:44 | |
| *** wu_wenxiang has joined #openstack-infra | 06:50 | |
| wu_wenxiang | Hello, I want to recheck https://review.openstack.org/#/c/38950/, however it seems do not work | 06:51 |
| wu_wenxiang | It works. | 06:53 |
| *** vogxn has joined #openstack-infra | 07:05 | |
| *** nati_ueno has joined #openstack-infra | 07:10 | |
| *** nati_ueno has quit IRC | 07:13 | |
| *** sarob has quit IRC | 07:16 | |
| *** sarob has joined #openstack-infra | 07:16 | |
| lifeless | mordred: *with* cache? thats slow.. | 07:17 |
| *** UtahDave has quit IRC | 07:31 | |
| *** sarob has joined #openstack-infra | 07:47 | |
| *** sarob has quit IRC | 07:52 | |
| *** vogxn has quit IRC | 08:37 | |
| *** vogxn has joined #openstack-infra | 08:43 | |
| *** sarob has joined #openstack-infra | 08:47 | |
| *** sarob has quit IRC | 08:52 | |
| *** sileht has quit IRC | 08:54 | |
| *** sileht has joined #openstack-infra | 08:58 | |
| *** sarob has joined #openstack-infra | 09:48 | |
| *** sarob has quit IRC | 09:57 | |
| *** vogxn has quit IRC | 10:11 | |
| *** UtahDave has joined #openstack-infra | 10:14 | |
| *** sarob has joined #openstack-infra | 10:48 | |
| *** mikal has joined #openstack-infra | 10:48 | |
| *** UtahDave has quit IRC | 10:51 | |
| *** sarob has quit IRC | 10:53 | |
| *** shhu is now known as help | 11:00 | |
| *** help is now known as Guest45235 | 11:00 | |
| *** Guest45235 is now known as shhu | 11:01 | |
| *** SergeyLukjanov has joined #openstack-infra | 11:05 | |
| *** SergeyLukjanov has quit IRC | 11:15 | |
| *** dina_belova has joined #openstack-infra | 11:45 | |
| *** sarob has joined #openstack-infra | 11:49 | |
| *** sarob has quit IRC | 11:54 | |
| *** CaptTofu has joined #openstack-infra | 11:58 | |
| *** hashar has joined #openstack-infra | 12:04 | |
| *** hashar has quit IRC | 12:08 | |
| *** koobs has quit IRC | 12:09 | |
| *** koobs has joined #openstack-infra | 12:09 | |
| *** prad has joined #openstack-infra | 12:41 | |
| *** CaptTofu has quit IRC | 12:49 | |
| *** CaptTofu has joined #openstack-infra | 12:50 | |
| *** prad_ has joined #openstack-infra | 12:50 | |
| *** michchap has quit IRC | 12:50 | |
| *** michchap has joined #openstack-infra | 12:51 | |
| *** prad has quit IRC | 12:53 | |
| *** prad_ has quit IRC | 12:55 | |
| *** prad has joined #openstack-infra | 12:56 | |
| *** prad_ has joined #openstack-infra | 12:59 | |
| *** prad has quit IRC | 13:00 | |
| *** prad_ is now known as prad | 13:00 | |
| *** prad has quit IRC | 13:05 | |
| *** prad has joined #openstack-infra | 13:09 | |
| *** yaguang has joined #openstack-infra | 13:10 | |
| *** michchap has quit IRC | 13:13 | |
| *** isviridov has joined #openstack-infra | 13:18 | |
| *** sarob has joined #openstack-infra | 13:20 | |
| *** isviridov has quit IRC | 13:22 | |
| *** isviridov has joined #openstack-infra | 13:22 | |
| *** sarob has quit IRC | 13:24 | |
| *** CaptTofu has quit IRC | 13:29 | |
| *** CaptTofu has joined #openstack-infra | 13:29 | |
| *** SergeyLukjanov has joined #openstack-infra | 13:31 | |
| *** thomasbiege has joined #openstack-infra | 13:52 | |
| thomasbiege | hi | 13:53 |
| isviridov | Hi | 13:54 |
| openstackgerrit | Paul Belanger proposed a change to openstack-infra/jenkins-job-builder: Switch to flake8+hacking https://review.openstack.org/38013 | 13:55 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/jeepyb: Allow hacking 0.6.0 and fix versions https://review.openstack.org/38119 | 13:59 |
| *** prad has quit IRC | 14:00 | |
| *** yaguang has quit IRC | 14:01 | |
| *** dina_belova has quit IRC | 14:03 | |
| *** dina_belova has joined #openstack-infra | 14:07 | |
| *** thomasbiege has quit IRC | 14:07 | |
| *** isviridov_ has joined #openstack-infra | 14:13 | |
| *** dina_belova has quit IRC | 14:14 | |
| *** isviridov has quit IRC | 14:16 | |
| *** SergeyLukjanov has quit IRC | 14:17 | |
| *** dina_belova has joined #openstack-infra | 14:19 | |
| *** dina_belova has quit IRC | 14:20 | |
| *** jjmb has joined #openstack-infra | 14:25 | |
| *** mriedem has joined #openstack-infra | 15:02 | |
| *** isviridov_ has quit IRC | 15:03 | |
| *** isviridov has joined #openstack-infra | 15:05 | |
| *** sarob has joined #openstack-infra | 15:09 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Delay initialization of local repos https://review.openstack.org/38507 | 15:12 |
| *** sarob has quit IRC | 15:14 | |
| *** prad has joined #openstack-infra | 15:14 | |
| *** prad has left #openstack-infra | 15:15 | |
| pabelanger | GheRivero, any chance for a review and sponsor? https://mentors.debian.net/package/jenkins-job-builder | 15:15 |
| *** CaptTofu has quit IRC | 15:20 | |
| *** dina_belova has joined #openstack-infra | 15:21 | |
| *** CaptTofu has joined #openstack-infra | 15:21 | |
| *** dina_belova has quit IRC | 15:25 | |
| *** SergeyLukjanov has joined #openstack-infra | 15:26 | |
| *** dina_belova has joined #openstack-infra | 15:29 | |
| jeblair | #status log restarted zuul to upgrade | 15:30 |
| *** wu_wenxiang has quit IRC | 15:33 | |
| *** SergeyLukjanov has quit IRC | 15:43 | |
| *** dina_belova has quit IRC | 15:54 | |
| *** isviridov has quit IRC | 15:55 | |
| *** isviridov has joined #openstack-infra | 15:56 | |
| *** dina_belova has joined #openstack-infra | 15:57 | |
| *** dina_belova has quit IRC | 15:58 | |
| *** sarob has joined #openstack-infra | 16:10 | |
| *** jjmb has quit IRC | 16:10 | |
| *** sarob has quit IRC | 16:15 | |
| *** bnemec has joined #openstack-infra | 16:18 | |
| *** dina_belova has joined #openstack-infra | 16:28 | |
| *** SergeyLukjanov has joined #openstack-infra | 16:28 | |
| clarkb | jeblair: the estimated test times are shiny | 16:29 |
| *** jaypipes has joined #openstack-infra | 16:32 | |
| *** isviridov has quit IRC | 16:46 | |
| *** isviridov has joined #openstack-infra | 16:46 | |
| *** SergeyLukjanov has quit IRC | 17:03 | |
| *** sarob has joined #openstack-infra | 17:11 | |
| *** sarob has quit IRC | 17:15 | |
| *** isviridov has quit IRC | 17:36 | |
| *** isviridov has joined #openstack-infra | 17:36 | |
| *** CaptTofu has quit IRC | 17:37 | |
| *** CaptTofu has joined #openstack-infra | 17:37 | |
| *** SergeyLukjanov has joined #openstack-infra | 17:38 | |
| *** prad has joined #openstack-infra | 17:38 | |
| *** prad has quit IRC | 17:39 | |
| *** vijendar has joined #openstack-infra | 17:50 | |
| *** vijendar has joined #openstack-infra | 17:50 | |
| *** bnemec has quit IRC | 17:58 | |
| *** cyeoh has quit IRC | 17:58 | |
| *** bnemec has joined #openstack-infra | 17:58 | |
| *** cyeoh has joined #openstack-infra | 17:59 | |
| *** isviridov has quit IRC | 18:01 | |
| *** dina_belova has quit IRC | 18:04 | |
| *** CaptTofu has quit IRC | 18:05 | |
| *** CaptTofu has joined #openstack-infra | 18:05 | |
| *** sarob has joined #openstack-infra | 18:11 | |
| *** isviridov has joined #openstack-infra | 18:15 | |
| *** sarob has quit IRC | 18:16 | |
| *** vijendar has quit IRC | 18:19 | |
| *** dina_belova has joined #openstack-infra | 18:20 | |
| *** CaptTofu has quit IRC | 18:22 | |
| *** CaptTofu has joined #openstack-infra | 18:22 | |
| *** cody-somerville has joined #openstack-infra | 18:26 | |
| *** afazekas has joined #openstack-infra | 18:30 | |
| *** dina_belova has quit IRC | 18:31 | |
| clarkb | the grenade failures seem to get very consistent in the gate at 2013-0-27 1552UTC | 18:32 |
| clarkb | before that the success rate is much higher | 18:32 |
| *** isviridov has quit IRC | 18:33 | |
| clarkb | *2013-07-27 | 18:34 |
| clarkb | https://review.openstack.org/#/c/38810/ was the last change to pass grenade when it was semi consistently passing | 18:40 |
| clarkb | 38587 and 28082 seem like strong candidates for the breakage | 18:41 |
| *** ^demon has joined #openstack-infra | 18:46 | |
| *** ^demon has joined #openstack-infra | 18:46 | |
| *** ^demon has quit IRC | 18:51 | |
| *** erfanian has joined #openstack-infra | 18:54 | |
| *** nati_ueno has joined #openstack-infra | 18:55 | |
| *** UtahDave has joined #openstack-infra | 18:57 | |
| *** CaptTofu has quit IRC | 18:59 | |
| *** CaptTofu has joined #openstack-infra | 18:59 | |
| mordred | lifeless: well, not with squid cache - but with everything else cached | 19:00 |
| *** sarob has joined #openstack-infra | 19:12 | |
| *** isviridov has joined #openstack-infra | 19:15 | |
| *** sarob has quit IRC | 19:17 | |
| *** reed has joined #openstack-infra | 19:18 | |
| *** bnemec has quit IRC | 19:30 | |
| *** bnemec has joined #openstack-infra | 19:30 | |
| *** UtahDave has quit IRC | 19:39 | |
| *** dina_belova has joined #openstack-infra | 19:42 | |
| *** dina_belova has quit IRC | 19:47 | |
| *** dkliban has quit IRC | 19:48 | |
| *** blamar has quit IRC | 19:52 | |
| *** Ryan_Lane has quit IRC | 19:53 | |
| *** bnemec has quit IRC | 19:59 | |
| *** bnemec has joined #openstack-infra | 19:59 | |
| *** dina_belova has joined #openstack-infra | 20:04 | |
| *** sarob has joined #openstack-infra | 20:04 | |
| *** sarob has quit IRC | 20:05 | |
| *** openstackstatus has quit IRC | 20:08 | |
| *** dina_belova has quit IRC | 20:08 | |
| *** sarob has joined #openstack-infra | 20:16 | |
| *** SergeyLukjanov has quit IRC | 20:17 | |
| *** Alex_Gaynor has quit IRC | 20:20 | |
| *** Alex_Gaynor has joined #openstack-infra | 20:20 | |
| mordred | lifeless, clarkb: with a local squid cache - image build time is 3:47 | 20:45 |
| mordred | so I'm thinking a couple of things: | 20:45 |
| mordred | wait - it might be just one thing | 20:46 |
| mordred | and that's run a squid proxy on the node, and then in the step where I rsync the /var/cache/apt/archives from the image building node into the target image | 20:46 |
| mordred | I should rsync the squid cache too | 20:46 |
| mordred | so that building an image on a machine that was built by this, will be starting with fresh caches | 20:47 |
| *** annegentle has quit IRC | 20:50 | |
| *** annegentle has joined #openstack-infra | 20:51 | |
| *** sarob has quit IRC | 20:55 | |
| *** nati_ueno has quit IRC | 20:55 | |
| *** sarob has joined #openstack-infra | 20:55 | |
| *** CaptTofu has quit IRC | 21:00 | |
| *** CaptTofu has joined #openstack-infra | 21:00 | |
| *** sarob has quit IRC | 21:00 | |
| *** CaptTofu has quit IRC | 21:02 | |
| *** CaptTofu has joined #openstack-infra | 21:02 | |
| *** nati_ueno has joined #openstack-infra | 21:34 | |
| *** Ryan_Lane has joined #openstack-infra | 21:35 | |
| *** nati_ueno has quit IRC | 21:43 | |
| *** dkliban has joined #openstack-infra | 22:00 | |
| *** CaptTofu has quit IRC | 22:20 | |
| *** jhesketh has joined #openstack-infra | 22:24 | |
| *** dkliban has quit IRC | 22:26 | |
| *** erfanian has quit IRC | 22:31 | |
| *** nicedice has joined #openstack-infra | 22:46 | |
| *** dkliban has joined #openstack-infra | 22:56 | |
| *** nati_ueno has joined #openstack-infra | 22:58 | |
| *** woodspa has quit IRC | 23:10 | |
| *** jjmb has joined #openstack-infra | 23:27 | |
| *** wu_wenxiang has joined #openstack-infra | 23:29 | |
| wu_wenxiang | recheck always failed in yesterday and today, I upload 3 bugs, all they failed twice or more. | 23:30 |
| wu_wenxiang | http://status.openstack.org/zuul/ | 23:31 |
| *** michchap has joined #openstack-infra | 23:31 | |
| wu_wenxiang | Now they all failed again, totally confused. | 23:31 |
| wu_wenxiang | Why so many times checking failed happened? | 23:32 |
| clarkb | wu_wenxiang: are specific tests failing? | 23:33 |
| clarkb | the grenade test seems to have gone "bad" sometime yesterday resulting in a very high failure rate | 23:34 |
| *** CaptTofu has joined #openstack-infra | 23:40 | |
| wu_wenxiang | clarkb: Error Msg: [ERROR] /opt/stack/new/devstack/exercises/volumes.sh:128 server didn't become active! | 23:44 |
| wu_wenxiang | clarkb: all the fail testing report this error msg | 23:45 |
| clarkb | wu_wenxiang: yup thats the grenade failure | 23:47 |
| clarkb | it started sometime around 2013-07-27 1552UTC | 23:47 |
| clarkb | but I do not know why it is failing | 23:48 |
| wu_wenxiang | clarkb: Thanks, so I concern how could we pass the check processing now? just waiting? | 23:49 |
| clarkb | wu_wenxiang: I think at this point the grenade test will need fixing before tests regularly pass | 23:50 |
| wu_wenxiang | clarkb: Thanks, who will handle the fixing grenade test job? | 23:51 |
| clarkb | it will probably end up being sorted out by sdague and dtroyer, but anyone is welcome to help | 23:52 |
| wu_wenxiang | clarkb: Thanks all of you for your fixing work :) | 23:53 |
| * clarkb is going to disconnect for a bit to move irc connections | 23:58 | |
| *** clarkb has quit IRC | 23:58 | |
| * mordred is freaked out by clarkb not being connected | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!