*** suneelb has quit IRC | 00:01 | |
*** cody-somerville has joined #openstack-qa | 00:03 | |
*** bknudson has joined #openstack-qa | 00:05 | |
*** shashankhegde has joined #openstack-qa | 00:07 | |
*** mahito has joined #openstack-qa | 00:07 | |
*** yamamoto has quit IRC | 00:08 | |
openstackrecheck | openstack/nova change: https://review.openstack.org/165189 failed because of: gate-grenade-dsvm-ironic-sideways: https://bugs.launchpad.net/bugs/1323658 | 00:10 |
---|---|---|
openstack | Launchpad bug 1323658 in OpenStack Compute (nova) "Nova resize/restart results in guest ending up in inconsistent state with Neutron" [Medium,Confirmed] | 00:10 |
*** mahito has quit IRC | 00:10 | |
*** melwitt has joined #openstack-qa | 00:10 | |
*** mahito has joined #openstack-qa | 00:10 | |
*** patrickeast has quit IRC | 00:14 | |
*** achanda has joined #openstack-qa | 00:16 | |
*** sputnik13 has quit IRC | 00:19 | |
gmann | mtreinish: https://review.openstack.org/#/c/173136/ | 00:19 |
*** achanda_ has quit IRC | 00:19 | |
gmann | mtreinish: LGTM just 1 comment to use input cred directly while releasing the cred. | 00:20 |
*** achanda has quit IRC | 00:21 | |
*** tsekiyam_ has joined #openstack-qa | 00:22 | |
*** openstackgerrit has quit IRC | 00:22 | |
*** openstackgerrit has joined #openstack-qa | 00:22 | |
*** gamado has joined #openstack-qa | 00:24 | |
*** tsekiya__ has joined #openstack-qa | 00:24 | |
*** tsekiyam_ has quit IRC | 00:24 | |
*** tsekiya__ has quit IRC | 00:24 | |
*** tsekiyama has quit IRC | 00:26 | |
*** madhu_ak has quit IRC | 00:28 | |
*** gamado has quit IRC | 00:28 | |
*** mlavalle has quit IRC | 00:31 | |
*** openstack has joined #openstack-qa | 00:36 | |
*** jhenner has quit IRC | 00:39 | |
*** gamado has joined #openstack-qa | 00:48 | |
*** Poornima has joined #openstack-qa | 00:50 | |
*** luqas has joined #openstack-qa | 00:50 | |
*** gamado has quit IRC | 00:53 | |
*** luqas has quit IRC | 00:54 | |
*** jhenner has joined #openstack-qa | 00:56 | |
*** shashankhegde has quit IRC | 00:57 | |
*** Poornima has quit IRC | 01:00 | |
*** Swami has quit IRC | 01:00 | |
*** yamamoto has joined #openstack-qa | 01:00 | |
*** dboik has joined #openstack-qa | 01:05 | |
*** dboik_ has joined #openstack-qa | 01:06 | |
*** dboik has quit IRC | 01:10 | |
*** Nic1 has quit IRC | 01:12 | |
*** madhu_ak has joined #openstack-qa | 01:15 | |
*** zz_dimtruck is now known as dimtruck | 01:15 | |
*** madhu_ak_ has joined #openstack-qa | 01:15 | |
*** davideagnello has quit IRC | 01:18 | |
*** madhu_ak has quit IRC | 01:19 | |
*** dpaterson has joined #openstack-qa | 01:36 | |
*** mwagner_lap has joined #openstack-qa | 01:37 | |
*** mmedvede has quit IRC | 01:44 | |
*** madhu_ak_ has quit IRC | 01:44 | |
*** madhu_ak_ has joined #openstack-qa | 01:45 | |
*** jecarey has joined #openstack-qa | 01:45 | |
*** madhu_ak_ has quit IRC | 01:45 | |
*** madhu_ak_ has joined #openstack-qa | 01:45 | |
*** mmedvede has joined #openstack-qa | 01:48 | |
*** melwitt has quit IRC | 01:51 | |
*** bkopilov has quit IRC | 01:53 | |
*** dpaterson has quit IRC | 02:03 | |
*** liusheng has quit IRC | 02:16 | |
*** Poornima has joined #openstack-qa | 02:16 | |
*** sks has joined #openstack-qa | 02:27 | |
*** sputnik13 has joined #openstack-qa | 02:29 | |
*** madhu_ak__ has joined #openstack-qa | 02:31 | |
*** rfolco has quit IRC | 02:31 | |
*** madhu_ak__ has quit IRC | 02:33 | |
*** madhu_ak__ has joined #openstack-qa | 02:33 | |
*** madhu_ak__ is now known as madhu_ak | 02:34 | |
*** madhu_ak_ has quit IRC | 02:35 | |
*** achanda has joined #openstack-qa | 02:35 | |
*** harlowja is now known as harlowja_away | 02:35 | |
*** harlowja_away is now known as harlowja | 02:37 | |
openstackrecheck | openstack/nova change: https://review.openstack.org/172559 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1313168, gate-tempest-dsvm-neutron-full: unrecognized error | 02:40 |
openstack | Launchpad bug 1313168 in OpenStack-Gate "paramiko ValueError: Unable to compute factors p and q from exponent d" [Medium,Triaged] | 02:40 |
openstackgerrit | Eli Qiao proposed openstack/tempest: fixed_ip unreserve should use an ip liked address https://review.openstack.org/170027 | 02:41 |
*** sputnik1_ has joined #openstack-qa | 02:46 | |
*** sputnik13 has quit IRC | 02:48 | |
*** mmedvede has quit IRC | 02:56 | |
*** mmedvede has joined #openstack-qa | 02:59 | |
openstackrecheck | openstack/nova change: https://review.openstack.org/173151 failed because of: gate-tempest-dsvm-nova-v21-full: https://bugs.launchpad.net/bugs/1286635 | 03:10 |
openstack | Launchpad bug 1286635 in OpenStack-Gate ""Packages cannot be authenticated" when doing an apt-get in the gate" [Undecided,Won't fix] | 03:10 |
*** sks has quit IRC | 03:13 | |
*** artom has quit IRC | 03:19 | |
*** bkopilov has joined #openstack-qa | 03:19 | |
*** sabeen1 has joined #openstack-qa | 03:21 | |
*** jogo has quit IRC | 03:22 | |
*** jyuso1 has quit IRC | 03:23 | |
*** sabeen3 has quit IRC | 03:24 | |
*** woodster_ has quit IRC | 03:30 | |
*** achanda has quit IRC | 03:41 | |
*** annashen has joined #openstack-qa | 03:42 | |
*** jecarey has quit IRC | 03:45 | |
*** catohornet has left #openstack-qa | 03:46 | |
*** sabeen1 has quit IRC | 03:47 | |
*** luqas has joined #openstack-qa | 03:52 | |
*** mmedvede has quit IRC | 03:54 | |
openstackgerrit | Tiantian Gao proposed openstack-dev/devstack: Support config neutron api_workers and rpc_workers https://review.openstack.org/175704 | 03:55 |
*** harlowja is now known as harlowja_away | 03:56 | |
*** luqas has quit IRC | 03:56 | |
openstackgerrit | Merged openstack-infra/devstack-gate: redo devstack-gate grenade selection as a case statement https://review.openstack.org/175390 | 03:56 |
openstackrecheck | openstack-infra/devstack-gate change: https://review.openstack.org/175391 failed gate-grenade-dsvm, gate-grenade-dsvm-partial-ncpu in the gate queue with an unrecognized error | 03:57 |
*** gamado has joined #openstack-qa | 04:01 | |
*** mmedvede has joined #openstack-qa | 04:02 | |
*** BharatK has joined #openstack-qa | 04:04 | |
*** gamado has quit IRC | 04:05 | |
*** annashen has quit IRC | 04:05 | |
*** Swami has joined #openstack-qa | 04:06 | |
*** annashen has joined #openstack-qa | 04:06 | |
*** annashen has quit IRC | 04:11 | |
*** jogo has joined #openstack-qa | 04:15 | |
*** jogo has quit IRC | 04:17 | |
*** achanda has joined #openstack-qa | 04:17 | |
*** Swami has quit IRC | 04:17 | |
*** gamado has joined #openstack-qa | 04:17 | |
*** yfried|afk is now known as yfried__ | 04:18 | |
*** davideagnello has joined #openstack-qa | 04:19 | |
*** jogo has joined #openstack-qa | 04:20 | |
*** sputnik1_ has quit IRC | 04:23 | |
*** davideagnello has quit IRC | 04:24 | |
*** amotoki has joined #openstack-qa | 04:24 | |
*** woodster_ has joined #openstack-qa | 04:25 | |
*** davideagnello has joined #openstack-qa | 04:33 | |
*** Madan has joined #openstack-qa | 04:35 | |
*** yfried__ is now known as yfried|afk | 04:36 | |
*** yfried|afk is now known as yfried__ | 04:38 | |
*** Sukhdev has joined #openstack-qa | 04:44 | |
*** annashen has joined #openstack-qa | 04:45 | |
*** yfried__ is now known as yfried|afk | 04:52 | |
*** Longgeek has joined #openstack-qa | 04:52 | |
openstackgerrit | Ramakrishnan G proposed openstack-dev/devstack: Fix issue with ml2 plugin on using provider network https://review.openstack.org/174731 | 04:57 |
*** mmedvede has quit IRC | 04:59 | |
*** yfried|afk is now known as yfried__ | 05:01 | |
*** flaper87 has quit IRC | 05:01 | |
*** flaper87 has joined #openstack-qa | 05:04 | |
*** itzikb has joined #openstack-qa | 05:11 | |
*** mmedvede has joined #openstack-qa | 05:13 | |
*** dimtruck is now known as zz_dimtruck | 05:15 | |
*** davideagnello has quit IRC | 05:19 | |
*** shashankhegde has joined #openstack-qa | 05:23 | |
*** afazekas has quit IRC | 05:31 | |
*** neeti has joined #openstack-qa | 05:42 | |
*** anshul has joined #openstack-qa | 05:43 | |
*** yfried__ has quit IRC | 05:44 | |
*** pal has joined #openstack-qa | 05:52 | |
*** achanda has quit IRC | 05:52 | |
*** SridharG has joined #openstack-qa | 05:56 | |
*** k4n0 has joined #openstack-qa | 05:56 | |
*** pal_ has joined #openstack-qa | 05:56 | |
*** pal has quit IRC | 05:56 | |
*** achanda has joined #openstack-qa | 05:57 | |
*** achanda has quit IRC | 06:03 | |
*** armax_ has quit IRC | 06:06 | |
*** shashankhegde has quit IRC | 06:07 | |
*** armax has joined #openstack-qa | 06:07 | |
*** armax has quit IRC | 06:07 | |
*** armax has joined #openstack-qa | 06:08 | |
*** armax has left #openstack-qa | 06:08 | |
*** armax has joined #openstack-qa | 06:08 | |
*** armax has quit IRC | 06:10 | |
*** armax has joined #openstack-qa | 06:10 | |
*** armax has quit IRC | 06:10 | |
*** cody-somerville has quit IRC | 06:14 | |
*** afazekas_ has joined #openstack-qa | 06:22 | |
*** annashen has quit IRC | 06:23 | |
*** pal has joined #openstack-qa | 06:24 | |
*** pal_ has quit IRC | 06:25 | |
*** salv-orlando has quit IRC | 06:29 | |
*** yfried__ has joined #openstack-qa | 06:33 | |
*** jamielennox is now known as jamielennox|away | 06:34 | |
*** luqas has joined #openstack-qa | 06:34 | |
*** nmagnezi_ has joined #openstack-qa | 06:38 | |
*** Sukhdev has quit IRC | 06:44 | |
*** yfried__ is now known as yfried|afk | 06:47 | |
*** pal has quit IRC | 06:51 | |
*** pal has joined #openstack-qa | 06:56 | |
*** yfried|afk is now known as yfried__ | 06:59 | |
*** mestery has quit IRC | 07:03 | |
*** itzikb has quit IRC | 07:05 | |
*** SridharGaddam has joined #openstack-qa | 07:08 | |
*** k4n0 has quit IRC | 07:11 | |
*** pcaruana has quit IRC | 07:11 | |
*** itzikb has joined #openstack-qa | 07:12 | |
*** k4n0 has joined #openstack-qa | 07:12 | |
*** jyuso1 has joined #openstack-qa | 07:18 | |
*** woodster_ has quit IRC | 07:20 | |
*** flaper87 has quit IRC | 07:23 | |
*** flaper87 has joined #openstack-qa | 07:23 | |
*** sergsh has joined #openstack-qa | 07:24 | |
*** gfidente has joined #openstack-qa | 07:26 | |
*** pal has quit IRC | 07:28 | |
*** afazekas_ has quit IRC | 07:32 | |
*** mpaolino has joined #openstack-qa | 07:33 | |
*** shardy_z is now known as shardy | 07:36 | |
*** afazekas_ has joined #openstack-qa | 07:36 | |
*** pal has joined #openstack-qa | 07:37 | |
*** jamielennox|away is now known as jamielennox | 07:43 | |
*** mestery has joined #openstack-qa | 07:46 | |
*** itzikb has quit IRC | 07:51 | |
*** weshay has joined #openstack-qa | 07:54 | |
*** safchain has joined #openstack-qa | 07:58 | |
*** tkammer has joined #openstack-qa | 08:07 | |
*** tochi has quit IRC | 08:08 | |
*** zz_johnthetubagu is now known as johnthetubaguy | 08:08 | |
*** tochi has joined #openstack-qa | 08:08 | |
*** yamamoto has quit IRC | 08:15 | |
*** johnthetubaguy is now known as zz_johnthetubagu | 08:15 | |
*** jordanP has joined #openstack-qa | 08:16 | |
*** itzikb has joined #openstack-qa | 08:16 | |
*** zz_johnthetubagu is now known as johnthetubaguy | 08:20 | |
*** mmedvede has quit IRC | 08:23 | |
*** yog__ has joined #openstack-qa | 08:24 | |
*** apevec has joined #openstack-qa | 08:26 | |
*** weshay has quit IRC | 08:26 | |
*** weshay has joined #openstack-qa | 08:27 | |
*** ihrachyshka has joined #openstack-qa | 08:31 | |
*** e0ne has joined #openstack-qa | 08:34 | |
*** mmedvede has joined #openstack-qa | 08:35 | |
*** Longgeek has quit IRC | 08:42 | |
*** mmedvede has quit IRC | 08:42 | |
*** coolsvap|afk is now known as coolsvap | 08:45 | |
*** coolsvap is now known as coolsvap|afk | 08:46 | |
openstackgerrit | Flavio Percoco proposed openstack-dev/devstack: Enable pooling by default https://review.openstack.org/140269 | 08:58 |
*** mmedvede has joined #openstack-qa | 09:03 | |
*** yamamoto has joined #openstack-qa | 09:04 | |
*** sergsh has quit IRC | 09:06 | |
*** tochi has quit IRC | 09:08 | |
*** Longgeek has joined #openstack-qa | 09:14 | |
*** sergsh has joined #openstack-qa | 09:18 | |
*** yfried__ is now known as yfried|afk | 09:21 | |
*** tosky has joined #openstack-qa | 09:34 | |
*** yfried|afk is now known as yfried__ | 09:34 | |
*** yamamoto has quit IRC | 09:34 | |
*** yamamoto has joined #openstack-qa | 09:36 | |
*** Poornima has quit IRC | 09:40 | |
*** salv-orlando has joined #openstack-qa | 09:42 | |
*** pal has quit IRC | 09:42 | |
openstackgerrit | Merged openstack-dev/devstack: Add response time to keystone access log https://review.openstack.org/164042 | 09:43 |
*** luqas has quit IRC | 09:45 | |
*** yfried__ is now known as yfried|afk | 09:48 | |
*** yamamoto has quit IRC | 09:50 | |
*** yfried|afk is now known as yfried__ | 09:50 | |
*** luqas has joined #openstack-qa | 09:53 | |
*** cdent has joined #openstack-qa | 09:59 | |
*** yfried__ is now known as yfried|afk | 10:01 | |
*** e0ne is now known as e0ne_ | 10:02 | |
*** e0ne_ is now known as e0ne | 10:04 | |
*** boris-42 has quit IRC | 10:05 | |
*** boris-42 has joined #openstack-qa | 10:08 | |
*** yamahata has quit IRC | 10:12 | |
*** yfried|afk is now known as yfried__ | 10:13 | |
*** neeti has quit IRC | 10:18 | |
*** sslypushenko has quit IRC | 10:26 | |
*** mpaolino has quit IRC | 10:29 | |
*** e0ne is now known as e0ne_ | 10:33 | |
*** nmagnezi_ has quit IRC | 10:34 | |
*** e0ne_ is now known as e0ne | 10:34 | |
*** pc_m has joined #openstack-qa | 10:35 | |
openstackgerrit | Tiantian Gao proposed openstack-dev/devstack: Support config neutron api_workers and rpc_workers https://review.openstack.org/175704 | 10:52 |
*** rbak has joined #openstack-qa | 10:55 | |
*** rbak has quit IRC | 10:57 | |
*** rbak has joined #openstack-qa | 10:57 | |
openstackgerrit | Joe Hakim Rahme proposed openstack/tempest: Adds unit test for Javelin https://review.openstack.org/165775 | 10:59 |
openstackrecheck | openstack/nova change: https://review.openstack.org/174064 failed because of: gate-tempest-dsvm-large-ops: https://bugs.launchpad.net/bugs/1400905, gate-tempest-dsvm-neutron-large-ops: https://bugs.launchpad.net/bugs/1400905 | 11:01 |
openstack | Launchpad bug 1400905 in OpenStack-Gate "Hash Sum mismatch on apt mirrors" [Undecided,Confirmed] | 11:01 |
*** BharatK has quit IRC | 11:02 | |
*** tosky_ has joined #openstack-qa | 11:02 | |
*** tosky_ has quit IRC | 11:02 | |
*** tosky has quit IRC | 11:03 | |
*** tosky has joined #openstack-qa | 11:07 | |
*** BharatK has joined #openstack-qa | 11:15 | |
*** mahito has quit IRC | 11:34 | |
*** Madan has quit IRC | 11:37 | |
*** mwagner_lap has quit IRC | 11:38 | |
*** BharatK has quit IRC | 11:38 | |
*** yamamoto has joined #openstack-qa | 11:53 | |
*** yog__ has quit IRC | 12:04 | |
openstackgerrit | Sean Dague proposed openstack-dev/grenade: add lib/apache to keystone shutdown https://review.openstack.org/175831 | 12:09 |
openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: only force off KEYSTONE_USE_WSGI up until kilo https://review.openstack.org/175531 | 12:10 |
*** bkopilov has quit IRC | 12:12 | |
*** yamamoto has quit IRC | 12:12 | |
*** jyuso1 has quit IRC | 12:16 | |
*** salv-orlando has quit IRC | 12:18 | |
*** mwagner_lap has joined #openstack-qa | 12:19 | |
*** woodster_ has joined #openstack-qa | 12:20 | |
*** salv-orlando has joined #openstack-qa | 12:21 | |
*** aix has joined #openstack-qa | 12:21 | |
openstackrecheck | openstack/nova change: https://review.openstack.org/170089 failed because of: gate-grenade-dsvm-ironic-sideways: https://bugs.launchpad.net/bugs/1323658 | 12:23 |
openstack | Launchpad bug 1323658 in OpenStack Compute (nova) "Nova resize/restart results in guest ending up in inconsistent state with Neutron" [Medium,Confirmed] | 12:23 |
*** yamamoto has joined #openstack-qa | 12:23 | |
*** yamamoto has quit IRC | 12:30 | |
*** sergsh has quit IRC | 12:40 | |
*** mpaolino has joined #openstack-qa | 12:42 | |
*** yamamoto has joined #openstack-qa | 12:44 | |
*** amotoki has quit IRC | 12:48 | |
*** rbak has quit IRC | 12:51 | |
*** gamado has quit IRC | 12:52 | |
*** yamamoto has quit IRC | 12:52 | |
*** bknudson has quit IRC | 12:53 | |
*** dkranz has quit IRC | 12:58 | |
*** yamamoto has joined #openstack-qa | 13:01 | |
openstackgerrit | Sergey Slipushenko proposed openstack/tempest: Tool for generation user accounts from spec https://review.openstack.org/172605 | 13:03 |
*** e0ne is now known as e0ne_ | 13:04 | |
*** yamamoto has quit IRC | 13:05 | |
*** Longgeek has quit IRC | 13:05 | |
*** e0ne_ is now known as e0ne | 13:06 | |
*** salv-orlando has quit IRC | 13:06 | |
*** rbak has joined #openstack-qa | 13:11 | |
*** itzikb has quit IRC | 13:14 | |
*** e0ne is now known as e0ne_ | 13:16 | |
*** bknudson has joined #openstack-qa | 13:16 | |
*** yamamoto has joined #openstack-qa | 13:17 | |
*** yamamoto has quit IRC | 13:17 | |
*** yamamoto has joined #openstack-qa | 13:17 | |
*** itzikb has joined #openstack-qa | 13:18 | |
*** yamamoto has quit IRC | 13:19 | |
*** anshul has quit IRC | 13:20 | |
*** e0ne_ is now known as e0ne | 13:21 | |
*** Longgeek has joined #openstack-qa | 13:24 | |
*** Longgeek has quit IRC | 13:24 | |
*** Longgeek has joined #openstack-qa | 13:25 | |
*** e0ne is now known as e0ne_ | 13:26 | |
*** e0ne_ is now known as e0ne | 13:26 | |
*** dustins has joined #openstack-qa | 13:31 | |
*** mriedem_away is now known as mriedem | 13:36 | |
*** EmilienM has quit IRC | 13:40 | |
*** EmilienM has joined #openstack-qa | 13:41 | |
*** dboik_ has quit IRC | 13:43 | |
*** cbits has joined #openstack-qa | 13:45 | |
*** mlavalle has joined #openstack-qa | 13:49 | |
*** zz_jgrimm is now known as jgrimm | 13:49 | |
*** armax_ has joined #openstack-qa | 13:51 | |
*** yamahata has joined #openstack-qa | 13:52 | |
*** salv-orlando has joined #openstack-qa | 13:52 | |
*** armax_ has quit IRC | 13:52 | |
*** armax_ has joined #openstack-qa | 13:52 | |
*** amotoki has joined #openstack-qa | 13:54 | |
*** salv-orlando has quit IRC | 13:54 | |
openstackgerrit | Sean Dague proposed openstack-dev/grenade: support apache based upgrades for keystone https://review.openstack.org/175831 | 13:55 |
*** yamamoto has joined #openstack-qa | 13:55 | |
*** Sukhdev has joined #openstack-qa | 13:57 | |
*** rbak has quit IRC | 13:58 | |
*** salv-orlando has joined #openstack-qa | 14:00 | |
*** jecarey has joined #openstack-qa | 14:00 | |
*** dboik has joined #openstack-qa | 14:01 | |
*** rbak has joined #openstack-qa | 14:02 | |
*** ajmiller has joined #openstack-qa | 14:03 | |
*** mtanino has joined #openstack-qa | 14:06 | |
*** afazekas_ has quit IRC | 14:08 | |
*** armax_ has quit IRC | 14:08 | |
*** paul_glass has joined #openstack-qa | 14:09 | |
*** armax has joined #openstack-qa | 14:09 | |
*** BharatK has joined #openstack-qa | 14:11 | |
*** bkopilov has joined #openstack-qa | 14:12 | |
*** dkranz has joined #openstack-qa | 14:15 | |
*** nikil22 has joined #openstack-qa | 14:18 | |
itzikb | mtreinish: hi, when you have the time , can you please look at https://review.openstack.org/#/c/169659/? | 14:19 |
*** sputnik13 has joined #openstack-qa | 14:21 | |
*** sputnik13 has quit IRC | 14:28 | |
*** annashen has joined #openstack-qa | 14:29 | |
*** dboik has quit IRC | 14:30 | |
*** dboik has joined #openstack-qa | 14:31 | |
mtreinish | itzikb: sure, I'll put it on my list | 14:31 |
*** annashen has quit IRC | 14:34 | |
*** achanda has joined #openstack-qa | 14:35 | |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Add config guide section on service feature enabled sections https://review.openstack.org/174086 | 14:36 |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Add section to config guide for setting up services https://review.openstack.org/174149 | 14:36 |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Add a paragraph on the use of tempest_roles option https://review.openstack.org/174675 | 14:36 |
*** dpaterson has joined #openstack-qa | 14:37 | |
*** weshay has quit IRC | 14:37 | |
itzikb | mtreinish: thanks! :-) | 14:39 |
*** cbits has left #openstack-qa | 14:40 | |
*** itzikb has quit IRC | 14:40 | |
*** achanda has quit IRC | 14:43 | |
*** artom has joined #openstack-qa | 14:44 | |
*** jecarey has quit IRC | 14:44 | |
*** tkammer has quit IRC | 14:46 | |
*** zz_dimtruck is now known as dimtruck | 14:51 | |
*** marun_ has joined #openstack-qa | 14:53 | |
*** shashankhegde has joined #openstack-qa | 14:54 | |
*** marun_ is now known as marun | 14:54 | |
*** sabeen1 has joined #openstack-qa | 14:55 | |
*** k4n0 has quit IRC | 14:57 | |
*** sileht has quit IRC | 14:57 | |
*** hemnafk is now known as hemna | 15:04 | |
*** jecarey has joined #openstack-qa | 15:05 | |
*** cbader has joined #openstack-qa | 15:05 | |
*** jaypipes has quit IRC | 15:11 | |
*** mestery has quit IRC | 15:11 | |
*** mestery has joined #openstack-qa | 15:12 | |
clarkb | sdague: https://review.openstack.org/#/c/175391/ did you see that failed in the gate? | 15:15 |
*** masayukig has quit IRC | 15:15 | |
sdague | yep | 15:15 |
sdague | so... this will be the 4th irc channel I'll talk about this bug today :) | 15:16 |
*** yamahata has quit IRC | 15:16 | |
clarkb | heh | 15:16 |
sdague | https://bugs.launchpad.net/oslo-incubator/+bug/1446583 | 15:16 |
openstack | Launchpad bug 1446583 in Keystone "services no longer reliably stop in stable/kilo" [Critical,In progress] - Assigned to Julien Danjou (jdanjou) | 15:16 |
clarkb | nice | 15:16 |
*** apevec has quit IRC | 15:19 | |
mriedem | mtreinish: jogo: is this new? http://logs.openstack.org/59/172559/1/gate/gate-tempest-dsvm-neutron-full/e82860f/console.html#_2015-04-21_02_38_16_850 | 15:20 |
*** masayukig has joined #openstack-qa | 15:21 | |
mtreinish | mriedem: I've not seen that before | 15:22 |
mtreinish | a 404 on a create is kinda weird | 15:22 |
mriedem | yeah there are some errors in the swift logs, trying to find a fingerprint | 15:23 |
openstackgerrit | Sean Dague proposed openstack-dev/grenade: DNM: add process debugging when things fail https://review.openstack.org/175935 | 15:23 |
openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: add support for stable/kilo https://review.openstack.org/175391 | 15:24 |
sdague | clarkb: so I'm going to see if I can dump more there | 15:24 |
clarkb | ok | 15:25 |
*** sabeen3 has joined #openstack-qa | 15:27 | |
mtreinish | mriedem: http://paste.openstack.org/show/204996/ looks like the error | 15:27 |
jogo | his may not be related but error with PUT /sdb1/117/AUTH_3f7d425c19754ab0b4a0c263b3770c0d : #012Traceback | 15:27 |
jogo | http://logs.openstack.org/59/172559/1/gate/gate-tempest-dsvm-neutron-full/e82860f/logs/screen-s-account.txt.gz | 15:27 |
mriedem | mtreinish: yeah, it only shows up on my one change in the gate for nova | 15:27 |
mriedem | which is completely unrelated | 15:28 |
*** sabeen1 has quit IRC | 15:28 | |
mriedem | mtreinish: so i just rechecked | 15:28 |
jogo | mtreinish: you should show that to the swift folks | 15:29 |
mtreinish | notmyname: ^^^ | 15:29 |
*** Sukhdev has quit IRC | 15:31 | |
*** annashen has joined #openstack-qa | 15:31 | |
*** Swami has joined #openstack-qa | 15:32 | |
jogo | mtreinish mriedem: with only a single hit in 7 days maybe this was an issue with the VM itself | 15:33 |
mtreinish | jogo: that's what I was thinking | 15:34 |
mtreinish | although the traceback was on a mkdir | 15:34 |
mriedem | looking at the test_volume_boot_pattern ssh fail now | 15:35 |
*** annashen has quit IRC | 15:36 | |
*** david-lyle has quit IRC | 15:36 | |
*** sslypushenko has joined #openstack-qa | 15:37 | |
*** nikil22 has quit IRC | 15:39 | |
paul_glass | are there any projects using tempest-lib in any capacity yet (other than designate)? just interested in keeping track of how other projects are doing things. | 15:40 |
*** amotoki has quit IRC | 15:41 | |
*** andreykurilin__ has joined #openstack-qa | 15:41 | |
*** amotoki has joined #openstack-qa | 15:43 | |
*** amotoki has quit IRC | 15:43 | |
*** yfried__ is now known as yfried|afk | 15:44 | |
*** sputnik13 has joined #openstack-qa | 15:45 | |
*** yfried|afk is now known as yfried__ | 15:50 | |
openstackgerrit | Roman Podoliaka proposed openstack/tempest: Add a test case for rebuild of instances with volumes https://review.openstack.org/175949 | 15:52 |
*** yfried__ has quit IRC | 15:55 | |
*** dboik has quit IRC | 15:56 | |
*** dboik has joined #openstack-qa | 15:57 | |
*** sileht has joined #openstack-qa | 15:57 | |
notmyname | mtreinish: jogo: was on the phone. looks like ... I'm not sure. a missing directory, but I don't know why. some directory in "/opt/stack/data/swift/1/sdb1/accounts/117/00a" wasn't found | 15:57 |
*** dboik has quit IRC | 15:58 | |
*** dboik has joined #openstack-qa | 15:58 | |
*** armax has left #openstack-qa | 15:58 | |
*** Sukhdev has joined #openstack-qa | 15:58 | |
*** dboik has quit IRC | 15:59 | |
*** dboik has joined #openstack-qa | 15:59 | |
*** armax has joined #openstack-qa | 16:00 | |
*** tsekiyama has joined #openstack-qa | 16:01 | |
*** armax has left #openstack-qa | 16:01 | |
*** SridharG has left #openstack-qa | 16:01 | |
*** armax has joined #openstack-qa | 16:02 | |
*** david-lyle has joined #openstack-qa | 16:02 | |
*** luqas has quit IRC | 16:02 | |
mtreinish | notmyname: yeah that's what it looked like from the logs, but I don't know why either. | 16:13 |
mtreinish | it was doing a mkdir so I assumed it was trying to make that dir and something in the middle wasn't found | 16:13 |
mtreinish | paul_glass: a bunch of projects are using it, but mostly the clients so far | 16:14 |
*** mlavalle has quit IRC | 16:16 | |
*** safchain has quit IRC | 16:18 | |
*** davideagnello has joined #openstack-qa | 16:20 | |
*** Sukhdev has quit IRC | 16:22 | |
*** yamamoto has quit IRC | 16:22 | |
mtreinish | sdague, mkoderer: if you guys get a sec can you take a look at: https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:tempest-config-guide,n,z | 16:25 |
mtreinish | should be easy to review just more docs | 16:25 |
*** jordanP has quit IRC | 16:29 | |
*** jaypipes has joined #openstack-qa | 16:30 | |
*** ChanServ sets mode: +o jaypipes | 16:30 | |
*** annashen has joined #openstack-qa | 16:32 | |
*** annashen has quit IRC | 16:37 | |
*** mpaolino has quit IRC | 16:38 | |
sdague | mtreinish: have you seen this kind of issue | 16:39 |
sdague | running testr | 16:40 |
sdague | No module named gdbm | 16:40 |
sdague | error: testr failed (3) | 16:40 |
sdague | ? | 16:40 |
clarkb | sdague: that happens if your python is compiled without gdbm iirc. Are you running this with a funny python? | 16:42 |
sdague | not as far as I know | 16:42 |
sdague | os1:~/python-novaclient(ips)> ./.tox/functional/bin/python --version | 16:42 |
sdague | Python 2.7.6 | 16:42 |
*** mlavalle has joined #openstack-qa | 16:43 | |
*** e0ne has quit IRC | 16:45 | |
*** gfidente has quit IRC | 16:48 | |
*** pal has joined #openstack-qa | 16:49 | |
*** mpaolino has joined #openstack-qa | 16:50 | |
*** harlowja_away is now known as harlowja | 16:57 | |
*** shashankhegde has quit IRC | 16:59 | |
*** annashen has joined #openstack-qa | 16:59 | |
*** pal has quit IRC | 17:00 | |
*** luqas has joined #openstack-qa | 17:03 | |
*** Sukhdev has joined #openstack-qa | 17:05 | |
*** luqas has quit IRC | 17:08 | |
*** david-lyle has quit IRC | 17:08 | |
davideagnello | hello, I have been trying out Tempest and Rally for integration and scenario testing for stackforge project Cue. Nova, Neutron, ceilometer, sahara.. have tests in both frameworks, I would like to get some feedback tempest vs. rally, using one or both. | 17:09 |
davideagnello | it looks like we are going to use tempest for integration tests and rally for e2e scenario testing | 17:09 |
*** Sukhdev has quit IRC | 17:12 | |
mriedem | mtreinish: boris-42: jogo: ^ | 17:12 |
* mriedem ducks | 17:12 | |
boris-42 | mriedem: lol | 17:15 |
mtreinish | sdague: yeah what clarkb said, I've rarely seen that | 17:18 |
sdague | clarkb: python-gdbm is a separate package in ubuntu | 17:20 |
sdague | which is currently not installed for me | 17:20 |
clarkb | huh | 17:21 |
mtreinish | yeah I thought it was baked into the stdlib | 17:21 |
sdague | p python-gdbm | 17:22 |
*** patrickeast has joined #openstack-qa | 17:22 | |
*** edleafe has joined #openstack-qa | 17:23 | |
clarkb | I just install python-all everywhere and that must pull it in | 17:23 |
sdague | ok, that's not installed by devstack | 17:23 |
mtreinish | sdague: https://docs.python.org/2/library/gdbm.html yeah it's in the stdlib | 17:23 |
* mtreinish goes and digs up what ubuntu is doing | 17:23 | |
*** asselin has quit IRC | 17:24 | |
mtreinish | heh, so the ubuntu packages decides that python-tk and python-gdbm are stdlib extensions and has a separate metapackage to install them | 17:26 |
*** achanda has joined #openstack-qa | 17:26 | |
mtreinish | oh, but it's something they took from debian | 17:27 |
*** tosky has quit IRC | 17:29 | |
*** asselin has joined #openstack-qa | 17:29 | |
sdague | clarkb: nope, python-all doesn't pull it in | 17:31 |
clarkb | sdague: huh, I have that package installed I wonder what pulled it in for me | 17:31 |
clarkb | there is a way to look at reverse depends but I can never remember how to do it | 17:31 |
sdague | yeh, I can't either | 17:31 |
openstackgerrit | Sean Dague proposed openstack-dev/devstack: testr requires python's gdbm https://review.openstack.org/176012 | 17:33 |
mtreinish | clarkb, sdague: isn't it: apt-cache rdepends | 17:33 |
*** e0ne has joined #openstack-qa | 17:33 | |
*** e0ne is now known as e0ne_ | 17:33 | |
sdague | oh, the fact that I removed avahi probably pulled it out | 17:34 |
*** e0ne_ is now known as e0ne | 17:34 | |
clarkb | its a bit annoying that python-all does not pull in the split out stdlib libs | 17:35 |
*** asselin has quit IRC | 17:36 | |
sdague | I would agree | 17:37 |
sdague | but at least putting it into the general list will heal situations like this | 17:37 |
*** aix has quit IRC | 17:37 | |
*** asselin has joined #openstack-qa | 17:37 | |
clarkb | ya | 17:37 |
*** apevec has joined #openstack-qa | 17:43 | |
*** apevec has joined #openstack-qa | 17:43 | |
*** e0ne is now known as e0ne_ | 17:44 | |
*** shashankhegde has joined #openstack-qa | 17:45 | |
*** e0ne_ is now known as e0ne | 17:45 | |
*** mpaolino has quit IRC | 17:48 | |
*** ihrachyshka has quit IRC | 17:50 | |
*** e0ne is now known as e0ne_ | 17:51 | |
*** samueldmq has quit IRC | 17:53 | |
openstackgerrit | Merged openstack-infra/subunit2sql: Add db api methods to delete old runs and test_runs https://review.openstack.org/172190 | 17:54 |
openstackgerrit | Merged openstack-infra/subunit2sql: Add a cli subcommand for deleting old rows https://review.openstack.org/172200 | 17:54 |
mtreinish | pcrews: how goes the attachment table stuff? I'm wondering if I should wait for that for the next subunit2sql release or just push one soonish with ^^^ | 17:56 |
*** e0ne_ has quit IRC | 17:56 | |
*** yfried__ has joined #openstack-qa | 17:56 | |
pcrews | mtreinish: you can go ahead and push, am still working and getting my decrepit brain to grok things. | 17:57 |
mtreinish | pcrews: ok, well there is no rush one way or the other. Is there anything I can help with? | 17:58 |
pcrews | mtreinish: not really, just being my usual overthinking, overworrying, overcritical self ;) | 17:59 |
mtreinish | heh, ok :) | 17:59 |
*** melwitt has joined #openstack-qa | 18:00 | |
*** jhenner has quit IRC | 18:02 | |
*** cdent has quit IRC | 18:03 | |
*** shardy has quit IRC | 18:04 | |
*** shardy has joined #openstack-qa | 18:04 | |
*** jamesllondon has joined #openstack-qa | 18:05 | |
*** e0ne has joined #openstack-qa | 18:10 | |
*** yamamoto has joined #openstack-qa | 18:10 | |
openstackgerrit | Sean Dague proposed openstack-dev/grenade: DNM: add process debugging when things fail https://review.openstack.org/175935 | 18:12 |
*** shardy has quit IRC | 18:12 | |
*** shardy has joined #openstack-qa | 18:14 | |
*** Nic has joined #openstack-qa | 18:14 | |
*** achanda has quit IRC | 18:15 | |
*** yamamoto has quit IRC | 18:15 | |
*** notmyname has quit IRC | 18:16 | |
*** johnthetubaguy is now known as zz_johnthetubagu | 18:16 | |
*** notmyname has joined #openstack-qa | 18:17 | |
mriedem | sdague: https://review.openstack.org/#/c/172326/ has been through the check queue a few times but doesn't get to the gate, i've been watching it in zuul - does it need a 2nd +W to push it through? | 18:20 |
*** achanda has joined #openstack-qa | 18:20 | |
sdague | it has a Depends-On | 18:20 |
mriedem | oh balls | 18:20 |
sdague | on a thing you WIP | 18:21 |
mriedem | b/c of the devstack change | 18:21 |
*** ajmiller_ has joined #openstack-qa | 18:21 | |
mriedem | derp | 18:21 |
sdague | the devstack change Depends-On a tempest change which is WIP | 18:21 |
mriedem | yeah | 18:21 |
mriedem | sorted now | 18:22 |
*** shashankhegde has quit IRC | 18:22 | |
mriedem | gotta check experimental the devstack change again for sanity | 18:22 |
*** ajmiller has quit IRC | 18:24 | |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Fix non-locking test accounts doc section https://review.openstack.org/176026 | 18:25 |
*** mriedem has quit IRC | 18:29 | |
*** shashankhegde has joined #openstack-qa | 18:30 | |
*** shardy_ has joined #openstack-qa | 18:33 | |
*** mriedem has joined #openstack-qa | 18:35 | |
mtreinish | mriedem: so live migrating a paused instance is expected to still be paused post migration? | 18:36 |
mriedem | mtreinish: yeah | 18:36 |
mriedem | according to gilliard | 18:36 |
*** shardy has quit IRC | 18:36 | |
mtreinish | ok, I guess it makes sense | 18:36 |
mtreinish | and if it wasn't the case that devstack patch would fail | 18:37 |
mriedem | just rechecked the devstack change to run the multinode job again | 18:37 |
mriedem | it was failing before | 18:37 |
mriedem | yeah | 18:37 |
mriedem | last time it ran it didn't have some of the live migration fixes in nova | 18:37 |
*** openstackgerrit has quit IRC | 18:37 | |
*** openstackgerrit has joined #openstack-qa | 18:37 | |
*** david-lyle has joined #openstack-qa | 18:39 | |
*** davideagnello has quit IRC | 18:41 | |
*** davideagnello has joined #openstack-qa | 18:44 | |
*** david-ly_ has joined #openstack-qa | 18:46 | |
*** david-lyle has quit IRC | 18:46 | |
*** dkranz has quit IRC | 18:48 | |
*** wuhg has quit IRC | 18:59 | |
*** suneelb has joined #openstack-qa | 19:00 | |
openstackrecheck | openstack/horizon change: https://review.openstack.org/173120 failed gate-tempest-dsvm-full, gate-grenade-dsvm in the gate queue with an unrecognized error | 19:02 |
*** dkranz has joined #openstack-qa | 19:03 | |
*** melwitt has quit IRC | 19:03 | |
openstackgerrit | Shilla Saebi proposed openstack-dev/devstack: made several changes to guides to comply to doc conventions https://review.openstack.org/176029 | 19:03 |
*** melwitt has joined #openstack-qa | 19:04 | |
*** luqas has joined #openstack-qa | 19:05 | |
*** paul_glass has quit IRC | 19:05 | |
*** paul_glass has joined #openstack-qa | 19:08 | |
*** luqas has quit IRC | 19:09 | |
*** rbak has quit IRC | 19:11 | |
*** dkranz has quit IRC | 19:13 | |
openstackrecheck | openstack/horizon change: https://review.openstack.org/166374 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 19:13 |
*** Rockyg has joined #openstack-qa | 19:15 | |
*** shashankhegde has quit IRC | 19:15 | |
*** david-ly_ has quit IRC | 19:18 | |
*** shardy_ has quit IRC | 19:19 | |
*** shardy has joined #openstack-qa | 19:20 | |
*** dkranz has joined #openstack-qa | 19:25 | |
*** jamesllondon has quit IRC | 19:26 | |
openstackgerrit | Sean M. Collins proposed openstack-dev/devstack: [WIP] Switch the default to Linuxbridge https://review.openstack.org/168423 | 19:39 |
*** achanda has quit IRC | 19:41 | |
*** shashankhegde has joined #openstack-qa | 19:44 | |
openstackgerrit | Andrey Pavlov proposed openstack/tempest: Remove common/ssh.py due to migration to tempest_lib https://review.openstack.org/176039 | 19:46 |
openstackgerrit | Shilla Saebi proposed openstack-dev/devstack: made several changes to guides to comply to doc conventions https://review.openstack.org/176029 | 19:50 |
openstackgerrit | Baodong (Robert) Li proposed openstack-dev/devstack: Add /usr/local/bin to exec_dirs in rootwrap.conf https://review.openstack.org/176041 | 19:52 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db api method to get status time series https://review.openstack.org/176042 | 19:54 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add command to calculate failure count and graph it https://review.openstack.org/176043 | 19:54 |
mriedem | mtreinish: https://review.openstack.org/#/c/157054/ passed on the devstack change in the exp queue | 19:55 |
mriedem | http://logs.openstack.org/26/172326/2/experimental/check-tempest-dsvm-multinode-full/6c35688/console.html#_2015-04-21_19_06_13_742 | 19:55 |
mtreinish | mriedem: is that a hint to go +2 the tempest change | 19:56 |
mtreinish | mriedem: looks like it skipped on neutron though: http://logs.openstack.org/54/157054/10/experimental/check-tempest-dsvm-neutron-multinode-full/2a29de2/console.html#_2015-04-21_10_41_48_038 | 19:57 |
mriedem | hmmm | 19:58 |
mriedem | probably something in d-g or devstack | 19:58 |
sdague | mtreinish: can you move multinode in tempest up to check (non voting) | 20:00 |
sdague | it would be nice to have on all the tempest runs | 20:01 |
openstackrecheck | openstack/keystone change: https://review.openstack.org/159944 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 20:01 |
mtreinish | sdague: yeah that project-config change was on my list | 20:01 |
mtreinish | I was planning on dropping the postgres jobs from the tempest queue at the same time | 20:01 |
mtreinish | once we add the stable/kilo jobs tempest will eat a lot of nodes... | 20:02 |
mtreinish | mriedem: that new option isn't being set in the tempest.conf file by devstack on the neutron job | 20:03 |
mtreinish | but I was pretty sure the devstack change did it unconditionally on master | 20:03 |
mriedem | mtreinish: yeah i thought it was unconditional | 20:04 |
mriedem | looking | 20:04 |
mriedem | i'm multi-tasking poorly right now | 20:04 |
*** paul_glass1 has joined #openstack-qa | 20:05 | |
mtreinish | mriedem: oh you know I might have clicked the link from the tempest review instead of devstack | 20:05 |
mtreinish | yep that's exactly what I did | 20:06 |
mtreinish | it worked fine on neutron: http://logs.openstack.org/26/172326/2/experimental/check-tempest-dsvm-neutron-multinode-full/699a0cf/console.html#_2015-04-21_19_07_33_097 | 20:06 |
mtreinish | +2s all around | 20:06 |
mriedem | oh f yeah | 20:08 |
*** annashen has quit IRC | 20:08 | |
mriedem | oh snap even sdague got in on that action | 20:08 |
*** paul_glass has quit IRC | 20:08 | |
mtreinish | mriedem: I threw another +A on the devstack change to push it into the gate too | 20:09 |
mtreinish | so it should just be a matter of time... | 20:09 |
mriedem | cool | 20:09 |
*** andreykurilin__ has quit IRC | 20:09 | |
mtreinish | jogo, mriedem: btw you might want to play with: https://review.openstack.org/#/c/176043/ | 20:10 |
mriedem | so the devstack change is on master, the code is in kilo, i could backport the devstack change to stable/kilo | 20:10 |
mtreinish | mriedem: yep | 20:10 |
mriedem | mtreinish: can that be used on test_volume_boot_pattern then? | 20:10 |
*** dustins_ has joined #openstack-qa | 20:11 | |
mtreinish | mriedem: yep, it should work on that | 20:11 |
mriedem | is infra storing this info somewhere? or what is subunit2sql using for data? | 20:11 |
mtreinish | mriedem: use --database-connection mysql://query:query@logstash.openstack.org/subunit2sql on the command | 20:12 |
*** andreykurilin__ has joined #openstack-qa | 20:12 | |
openstackrecheck | openstack/horizon change: https://review.openstack.org/172805 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 20:12 |
mtreinish | mriedem: or whatever mysql connector you want to use instead | 20:13 |
*** achanda has joined #openstack-qa | 20:14 | |
*** dustins has quit IRC | 20:14 | |
mtreinish | mriedem: you'll want to use b1374ea0-4011-45eb-9bc9-d49c79837fac and 4fb3c7b9-f8db-4006-ab9a-030f9223d4c1 for the ids | 20:14 |
mtreinish | I need to add using test_id instead of the primary key to the tool still | 20:15 |
*** jamesllondon has joined #openstack-qa | 20:16 | |
*** jamesllondon has quit IRC | 20:16 | |
*** e0ne is now known as e0ne_ | 20:17 | |
*** dboik has quit IRC | 20:20 | |
*** david-lyle has joined #openstack-qa | 20:21 | |
*** dboik has joined #openstack-qa | 20:21 | |
*** jamesllondon has joined #openstack-qa | 20:23 | |
*** jamesllondon has quit IRC | 20:23 | |
mtreinish | mriedem: v1: http://imgur.com/7ESjmFI and v2: http://imgur.com/eTXI1Xx | 20:23 |
mtreinish | clearly I need to add date filtering | 20:23 |
mtreinish | and maybe change the plotting algorithm so it treats missing entries as 0 instead of skipping them | 20:24 |
mtreinish | that's at a daily resolution fwiw | 20:24 |
mtreinish | mriedem: I'll play with it some more after I head home | 20:25 |
openstackrecheck | openstack/nova change: https://review.openstack.org/165189 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full, gate-tempest-dsvm-neutron-full, gate-grenade-dsvm, gate-tempest-dsvm-large-ops, gate-tempest-dsvm-neutron-large-ops, gate-devstack-dsvm-cells, gate-grenade-dsvm-partial-ncpu, gate-grenade-dsvm-ironic-sideways in the gate queue with an unrecognized error | 20:29 |
mriedem | wow | 20:29 |
mriedem | suwweetttt | 20:29 |
mriedem | new gate bug http://logs.openstack.org/89/165189/1/gate/gate-tempest-dsvm-full/c896f3b/logs/devstacklog.txt.gz#_2015-04-21_20_20_26_378 | 20:30 |
mriedem | on stable | 20:30 |
mriedem | dhellmann: ^ | 20:30 |
mriedem | stevedore and cliff releaeses blowing up stable/juno | 20:30 |
openstackrecheck | openstack/horizon change: https://review.openstack.org/155511 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 20:30 |
*** Sukhdev has joined #openstack-qa | 20:31 | |
sdague | mriedem: conflict with openstackclient it looks like | 20:31 |
sdague | dtroyer: ? | 20:32 |
mriedem | oh master is also busted it looks like | 20:34 |
sdague | well, I sort of expected the giant library flush would end up like this | 20:34 |
mriedem | get a plunger | 20:34 |
sdague | given that we released everything all at once | 20:34 |
openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for stevedore/cliff release 4/21 bug 1446847 https://review.openstack.org/176065 | 20:35 |
openstack | bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] https://launchpad.net/bugs/1446847 | 20:35 |
mriedem | e-r query ^ | 20:35 |
mriedem | so osc 1.1.0 was released today | 20:38 |
mriedem | on stable/juno that's capped at python-openstackclient>=0.3.0,<=1.0.1 | 20:38 |
dtroyer | it probably should be >=0.3.0,<1.1.0 but that should work | 20:39 |
sdague | dtroyer: it installs a cliff that requires stevedore >=1.3.0 | 20:39 |
sdague | however all the stable/juno servers require stevedore <=1.2 | 20:39 |
dtroyer | so did cliff not get capped somehwere? | 20:40 |
mriedem | cliff>=1.7.0,<=1.9.0 # Apache-2.0 on stable/juno | 20:40 |
mriedem | let me see which versions we're actualling installing here | 20:41 |
sdague | but openstackclient installs sooner than that | 20:41 |
sdague | and pip has no memory | 20:41 |
mriedem | python-openstackclient==1.0.1 | 20:42 |
sdague | yes | 20:42 |
sdague | because that's the version in g-r | 20:42 |
dtroyer | the first thing I see in some log from e-r is barbicanclient with what looks like cliff>=1.7.0 | 20:42 |
sdague | for stable/juno | 20:42 |
dtroyer | no max value | 20:42 |
sdague | dtroyer: http://logs.openstack.org/12/176012/1/check/check-grenade-dsvm/ab2f02e/logs/grenade.sh.txt.gz#_2015-04-21_17_51_27_814 | 20:42 |
sdague | also openstackclient | 20:42 |
mriedem | stevedore==1.2.0 | 20:42 |
dtroyer | I swear, pure clients just need to vendor all the things…I'll rewrite OSC in Go before I die. Same problem but it'll be a build-time problem not a runtime problem | 20:44 |
*** cdent has joined #openstack-qa | 20:45 | |
dhellmann | dtroyer: we should add a require=False argument to the entry point .load() call in cliff like we have in stevedore | 20:46 |
dtroyer | is there stable/juno for any of the clients? | 20:47 |
dtroyer | is this uncapped requirements in a stable/juno job catching up to us? | 20:47 |
*** krotscheck has joined #openstack-qa | 20:47 | |
*** shardy_ has joined #openstack-qa | 20:48 | |
mriedem | stable/juno was good yesterday once we got the ironic sideways job passing | 20:48 |
mriedem | then all clients/libs were released today and breaky breaky | 20:48 |
dhellmann | dtroyer: I thought we had requirements capped in stable/juno now | 20:48 |
mriedem | not everytihng is capped | 20:49 |
dtroyer | I don't see stable/juno branches in python-neutronclient | 20:49 |
mriedem | not sure if marconiclient and zaqarclient are worrisome | 20:49 |
dtroyer | nor novaclient | 20:49 |
dtroyer | keystoneclient has one | 20:49 |
*** jamesllondon has joined #openstack-qa | 20:51 | |
dtroyer | so no stable/juno in neutronclient uses master and an uncapped cliff req | 20:51 |
*** shardy has quit IRC | 20:51 | |
dtroyer | osc and barbicanclient too | 20:51 |
*** openstackgerrit has quit IRC | 20:52 | |
*** openstackgerrit has joined #openstack-qa | 20:52 | |
openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for stevedore/cliff release 4/21 bug 1446847 https://review.openstack.org/176065 | 20:53 |
openstack | bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] https://launchpad.net/bugs/1446847 | 20:53 |
mtreinish | mriedem: ^^^ man you don't even wait for me to +A it | 20:54 |
mriedem | mtreinish: you said yo uwere going home | 20:54 |
mtreinish | I am home :) | 20:54 |
mriedem | well, i didn't know when judge judy was over in your TZ | 20:55 |
mriedem | so i moved forward | 20:55 |
* mtreinish wonders if stackalytics counts reviews on merged changes | 20:55 | |
sdague | dtroyer: so barbican client is getting dragged in because of test-requirements on cinder | 20:55 |
*** openstackrecheck has quit IRC | 20:58 | |
*** openstackrecheck has joined #openstack-qa | 20:58 | |
sdague | dtroyer: so we could revert the test requirements installation | 21:00 |
sdague | as one way to dig out of this | 21:00 |
mtreinish | sdague: did we backport that change, or are we hitting this on master too? | 21:01 |
*** annashen has joined #openstack-qa | 21:01 | |
sdague | we're definitely installing the test-requirements | 21:02 |
sdague | it might not be backported | 21:02 |
*** dustins_ has quit IRC | 21:03 | |
*** zz_johnthetubagu is now known as johnthetubaguy | 21:04 | |
*** suneelb has quit IRC | 21:04 | |
openstackgerrit | Sean M. Collins proposed openstack-infra/elastic-recheck: Add fingerprint for other kinds of mirror failures https://review.openstack.org/176076 | 21:04 |
*** suneelb has joined #openstack-qa | 21:04 | |
*** tkammer has joined #openstack-qa | 21:04 | |
*** annashen has quit IRC | 21:06 | |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db api method to get status time series https://review.openstack.org/176042 | 21:06 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add command to calculate failure count and graph it https://review.openstack.org/176043 | 21:06 |
*** jecarey has quit IRC | 21:07 | |
*** luqas has joined #openstack-qa | 21:07 | |
openstackrecheck | openstack/horizon change: https://review.openstack.org/173496 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847 | 21:08 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 21:08 |
*** openstackrecheck has quit IRC | 21:10 | |
*** openstackrecheck has joined #openstack-qa | 21:11 | |
*** luqas has quit IRC | 21:12 | |
*** marun has quit IRC | 21:14 | |
morganfainberg | mtreinish, what is verify-tempest-config? | 21:14 |
morganfainberg | mtreinish, as in... what would make it silently fail. | 21:14 |
mtreinish | morganfainberg: there is a tempest.log file, the traceback is in there | 21:15 |
morganfainberg | hm ok | 21:15 |
dhellmann | sdague, dtroyer : trying to cap cliff in cinder: https://review.openstack.org/176082 | 21:15 |
mtreinish | I need to add an override to force it to log to stdout if the tempest.conf is different | 21:15 |
morganfainberg | ugh | 21:16 |
mtreinish | morganfainberg: but that script basically does some basic api poking to check your config file against what the apis say | 21:16 |
morganfainberg | this thing assumes v2 keystone | 21:16 |
mtreinish | I thought there was a fix for that | 21:16 |
mtreinish | dkranz: ^^^ | 21:16 |
morganfainberg | request (main): 404 POST http://172.16.30.15:35357/v3/tenants 0.011s | 21:16 |
morganfainberg | it assumes v2 semantics | 21:17 |
morganfainberg | not v2 specific. | 21:17 |
mtreinish | morganfainberg: change the tempest conf auth_version option in the identity section to 'v3' | 21:17 |
*** dboik_ has joined #openstack-qa | 21:18 | |
* morganfainberg is trying to default everything to v3 for keystone in devstack | 21:18 | |
mtreinish | I thought we made verify-tempest-config run after that is set in devstack though | 21:18 |
mtreinish | morganfainberg: there is a devstack flag we use to tell tempest use v3 | 21:18 |
mtreinish | andreaf: added it in preparation for that | 21:18 |
mtreinish | let me dig it up | 21:18 |
mtreinish | (fwiw there is a tempest experimental job that runs with v3) | 21:19 |
dtroyer | dhellmann: I'm not certain that is enough, neutronclient also does that, so does OSC, none of these three have stable/juno branches | 21:19 |
morganfainberg | sigh. lots of places to change this | 21:19 |
morganfainberg | and hard coded v2.0 endpoints | 21:19 |
mtreinish | morganfainberg: TEMPEST_AUTH_VERSION=v3 in your localrc should do the right thing | 21:19 |
mtreinish | for tempest at leastr | 21:20 |
dtroyer | dhellmann: nova does the same thing with neutronclient | 21:20 |
* morganfainberg starts unwinding this. | 21:20 | |
openstackrecheck | openstack/horizon change: https://review.openstack.org/173562 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847 | 21:20 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 21:20 |
dhellmann | dtroyer: by same thing you mean it caps neutronclient? | 21:20 |
*** jgrimm is now known as zz_jgrimm | 21:20 | |
morganfainberg | ok so i need to go through and re-work any v2 -> v3 things | 21:20 |
*** shardy_ has quit IRC | 21:21 | |
dtroyer | nova installs master neutronclient with uncapped cliff req | 21:21 |
dhellmann | dtroyer: got it | 21:21 |
dtroyer | and horizon | 21:21 |
*** dboik has quit IRC | 21:21 | |
dhellmann | dtroyer: wait, why? why is it installing anything not from source? | 21:21 |
dhellmann | I mean not from pypi? | 21:21 |
dtroyer | I'm just walking through one of the logs looking for everything that wants cliff, only barbicanclient and neutronclient so far, I'm sure I'll get to osc before I'm done | 21:22 |
dtroyer | this is from pypi | 21:22 |
openstackrecheck | openstack/cinder change: https://review.openstack.org/175909 failed because of: gate-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm: unrecognized error, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1446847, gate-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/1446847, gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1446847 | 21:23 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 21:23 |
david-lyle | morganfainberg, dtroyer, Horizon is hard coded to use v2.0 in the settings file for Horizon | 21:23 |
dtroyer | http://paste.openstack.org/show/205028/ is the chunk of log for pip install ceilometer | 21:24 |
david-lyle | there is an additional complication to using v3 for horizon | 21:24 |
david-lyle | we need to use another session store in devstack or Horizon won't really work | 21:24 |
dhellmann | dtroyer: ok, so we might need to create some juno branches and release fixes with caps for those libs. I'll wait to see what the full list is when you're done | 21:24 |
dtroyer | at 2015-04-21 20:01:22.967 is where it checks for cliff and reports the neutronclient req that was being solved. Am I mis-reading that? | 21:25 |
dhellmann | dtroyer: looking... | 21:25 |
dhellmann | dtroyer: that's saying that cliff>=1.7.0 is needed by neutronclient and is already installed | 21:26 |
dtroyer | in this logfile it's only Neutron, barbican clients and OSC that appear to use cliff | 21:26 |
dhellmann | that sounds right | 21:27 |
dtroyer | ok, right, that means that neutron isn't capped and the already installed 1.11.0 is good | 21:27 |
dtroyer | but its not | 21:27 |
*** pc_m has quit IRC | 21:28 | |
dhellmann | dtroyer: osc should probably have a cap, but the clients shouldn't because they are libraries themselves | 21:29 |
dhellmann | although as a pragmatic thing we could cap them now, we want to try to avoid that in the future if we can | 21:29 |
dtroyer | so in this case it would "work" because OSC is installed last. IIRC that's probably why OSC was moved to be installed last. | 21:30 |
dhellmann | that makes sense | 21:31 |
*** e0ne_ has quit IRC | 21:31 | |
dtroyer | this makes my head hurt | 21:31 |
*** dkranz has quit IRC | 21:31 | |
dhellmann | yep | 21:32 |
dhellmann | I'm beginning to think that devstack should install all of global-requirements.txt and then later it should install the other projects without their dependencies | 21:32 |
dhellmann | although I'm sure there are issues with that | 21:32 |
dhellmann | dtroyer: can you reproduce this in a way that gives a traceback explaining what osc is doing that causes the error? is it from a plugin? | 21:34 |
dtroyer | I'll need to spin up a new devstack vm…a bit short on resources ATM | 21:37 |
*** zz_jgrimm is now known as jgrimm | 21:40 | |
fungi | yeah, i had sort of assumed when we were making stable branches of clients/libraries that we were branching for icehouse and juno on all of them to accommodate sync'ing from the associated requirements repo branches | 21:43 |
dhellmann | fungi: I think we were trying to avoid going back that far if we could | 21:43 |
fungi | i see | 21:43 |
dhellmann | apparently we can't | 21:43 |
fungi | i wouldn't have expected that we could, but i guess i was coming at it from a different perspective | 21:44 |
*** jecarey has joined #openstack-qa | 21:44 | |
*** mriedem is now known as mriedem_away | 21:44 | |
dhellmann | fungi: although we do now have related issues with capping the requirements of libs and making the release | 21:44 |
fungi | having already tried once in the past couple years (and failed) to cap requirements for stable branches | 21:45 |
dhellmann | if we cap requirements of libraries, we have the issue we had this week of juggling the caps and unfreezing master | 21:45 |
dhellmann | if we do not cap the requirements, we have the issue we're having now with juno | 21:45 |
fungi | so, as an example, consider how you would perform unit tests of a stable branch for a library which depends on another library | 21:46 |
fungi | if you don't cap those requirements, you'll end up with a much, much newer version that you may no longer work with | 21:46 |
*** dboik_ has quit IRC | 21:46 | |
dhellmann | that's true | 21:46 |
*** artom has quit IRC | 21:46 | |
dhellmann | I suppose what we need to do is create the stable branch and then introduce the caps | 21:47 |
dhellmann | instead of the other way around | 21:47 |
fungi | i started running into some of these complications when i tried the first time, and gave up when i realized it's pin everything or nothing | 21:47 |
*** dboik has joined #openstack-qa | 21:47 | |
*** mwagner_lap has quit IRC | 21:47 | |
openstackrecheck | openstack/nova change: https://review.openstack.org/149371 failed because of: gate-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm-ironic-sideways: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm-partial-ncpu: unrecognized error, gate-grenade-dsvm: unrecognized error, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1446847, gate-tempest-dsvm- | 21:47 |
openstackrecheck | large-ops: unrecognized error, gate-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/1446847, gate-tempest-dsvm-neutron-large-ops: https://bugs.launchpad.net/bugs/1446847, gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1446847 | 21:47 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 21:47 |
fungi | or else you need to maintain a double-standard for caps you use when installing the library directly vs not having caps when you install that same library as a dependency of something else which has capped all teh transitive deps | 21:47 |
fungi | in some ways we've gotten ourselves into this situation via pbr, and the expectation that you can safely have very specific version constraints in dependencies recursed through multiple transitive relationships | 21:49 |
fungi | where pip seems to have hand-waved that away by assuming your requirements are site-specific not package-specific | 21:50 |
*** ttx has joined #openstack-qa | 21:50 | |
dhellmann | yeah | 21:50 |
*** patrickeast has quit IRC | 21:51 | |
dhellmann | I guess that's the requirements.txt vs. install_requires discussion | 21:51 |
fungi | in a nutshell | 21:52 |
*** mtanino has quit IRC | 21:52 | |
dhellmann | so we need a list of requirements for unit tests and a list for integration tests and mostly those will be the same. Do we have examples of where they would be different? | 21:53 |
fungi | i think we would only have directly tracked requirements per tox env, and not install requirements via a transitive relationship at all | 21:55 |
*** mfalatic has joined #openstack-qa | 21:55 | |
clarkb | they would be different wherever a project gets out of sync with global reqs | 21:55 |
*** mtanino has joined #openstack-qa | 21:55 | |
clarkb | that could be due to transitive deps on the unittest side not explicitly installed or deps that haven't had their pin synced in from global reqs | 21:56 |
fungi | basically making pbr stop installing things listed in the requirements.txt would be one possibility. but that's a huge shift in behavior which would take a larger conversation | 21:56 |
fungi | preinstalling everything pbr might want to install from transitive relationships is a potential workaround | 21:57 |
openstackrecheck | openstack/nova change: https://review.openstack.org/168317 failed because of: gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847 | 21:57 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 21:58 |
*** johnthetubaguy is now known as zz_johnthetubagu | 21:58 | |
*** yamahata has joined #openstack-qa | 21:58 | |
dhellmann | fungi: we could do that in devstack, without changing pbr | 21:58 |
fungi | richard jones had started a dependency tracker check tool which identified whether you were using any modules directly which were only supplied via transitive dependency relationships. we might be able to modify that to catch when projects started using transitive dependencies not declared directly in their requirements lists | 21:58 |
dhellmann | fungi: jogo had something similar going where he was trying to ensure we capped all of our dependencies | 21:59 |
fungi | so basically if `pip install .` caused pip freeze to list any module which was not also in your requirements.txt, fail the job | 21:59 |
dhellmann | clarkb: we wouldn't want any drift, right? so we would want to enforce that | 21:59 |
dhellmann | fungi: yeah, that's a pretty easily described test | 21:59 |
jogo | dhellmann: Rirchard's is better | 22:00 |
clarkb | dhellmann: ya I think we would want to avoid drift in that scenario | 22:00 |
*** harlowja is now known as harlowja_away | 22:00 | |
dhellmann | fungi: and we could have devstack cat all of the requirements.txt files together, eliminating duplicate entries, and the install them all in one go | 22:00 |
dtroyer | of course, venv's for everyone was another solution, but itsn't even close to being backported to juno | 22:00 |
fungi | however, with loose (or no) requirements caps, that is still fragile since a new release of one of your declared dependencies could add a dependency you don't (yet) have listed | 22:00 |
ttx | About to go to bed. If you have to pick between fast and good for the fix, please do fast, we can iterate on good once we are past this nightmare | 22:00 |
dhellmann | dtroyer: venvs solve the problem for us, but not for diistros | 22:00 |
dhellmann | ttx: yep | 22:01 |
ttx | We kind of need to push stuff to RC2s tomorrow, and ideally do those stable/kilo lib releases | 22:01 |
dhellmann | ttx: still thinking about how to actually solve it -- I have a few patches in play, but I have another meeting starting | 22:01 |
* jogo wishes we could vendor and use venvs and let the distros just do there own thing | 22:01 | |
fungi | so if we're _only_ trying to solve this for devstack right now, yes the problem space is a bit reduced | 22:01 |
dhellmann | fungi: yeah, I think that's good enough for now | 22:01 |
ttx | gdnite! | 22:01 |
fungi | night ttx! | 22:02 |
fungi | i agree, the de-dupe requirements idea has a good chance of working | 22:02 |
fungi | running pip install more than once in an environment should just be considered undefined behavior | 22:02 |
fungi | we should try to avoid doing that | 22:02 |
*** suneelb has quit IRC | 22:03 | |
fungi | i haven't looked at devstack lately, but if its pip install into the system env is once per service rather than once total, we're asking for trouble on dependency resolution | 22:04 |
dtroyer | its once or more per service depending on optional features | 22:05 |
dtroyer | doing one giant pip install isn't going to be fun at all, totally breaks down with the plugins | 22:05 |
*** harlowja_away is now known as harlowja | 22:05 | |
fungi | argh, urgh | 22:06 |
jogo | fungi: isn't this fun :) | 22:06 |
openstackrecheck | openstack/nova change: https://review.openstack.org/160089 failed because of: gate-devstack-dsvm-cells: unrecognized error, gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847, gate-nova-docs: unrecognized error, gate-nova-pep8: unrecognized error, gate-nova-python27: unrecognized error, gate-nova-tox-functional: | 22:07 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 22:07 |
openstackrecheck | unrecognized error, gate-tempest-dsvm-full-ceph: unrecognized error, gate-tempest-dsvm-full: unrecognized error, gate-tempest-dsvm-large-ops: unrecognized error, gate-tempest-dsvm-neutron-full: unrecognized error, gate-tempest-dsvm-neutron-large-ops: unrecognized error, gate-tempest-dsvm-nova-v21-full: unrecognized error, gate-tempest-dsvm-postgres-full: unrecognized error | 22:07 |
*** Swami has quit IRC | 22:07 | |
fungi | jogo: i know, that's why i ran screaming when i looked at capping requirements on stable branches last time | 22:07 |
jogo | wow docs is down | 22:07 |
*** suneelb has joined #openstack-qa | 22:07 | |
fungi | jogo: good news! it's not hosted on openstack services at all | 22:07 |
fungi | rackspace cloudsites | 22:07 |
*** suneelb has quit IRC | 22:08 | |
fungi | though it's working for me | 22:08 |
*** suneelb has joined #openstack-qa | 22:08 | |
fungi | temporary? | 22:08 |
jogo | so this is fun http://logs.openstack.org/89/160089/10/gate/gate-nova-pep8/8b84408/console.html#_2015-04-21_21_53_42_338 | 22:08 |
jogo | fungi: no not docs the site, the job | 22:08 |
jogo | and pep8 | 22:08 |
jogo | TypeError: cannot create weak reference to 'Struct' object | 22:08 |
sdague | so, could we also just release a version of cliff that doesn't require stevedore >=1.3 | 22:09 |
fungi | dtroyer: how much fun would it be for devstack to collect requested pip installs (from service lists, plugins, et cetera) and batch them into one actual call? | 22:09 |
*** suneelb has quit IRC | 22:10 | |
* fungi has found a new and unexpected use for the word "fun" | 22:10 | |
sdague | fungi: that was the suggestion from lifeless in the thread about this | 22:10 |
*** suneelb has joined #openstack-qa | 22:10 | |
fungi | i thought i'd read the whole thread, but must have missed that one | 22:10 |
openstackrecheck | openstack/horizon change: https://review.openstack.org/142950 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847, gate-horizon-docs: unrecognized error, gate-horizon-pep8: unrecognized error, gate-horizon-python27-django14: unrecognized error, gate-horizon-python27: unrecognized error, gate-horizon-selenium: unrecognized error, gate-tempest-dsvm-full: unrecognized error | 22:12 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 22:12 |
jogo | looks like we have a new issue | 22:13 |
dtroyer | fungi: I'm looking to see if the ordering of things makes that hard or really hard | 22:13 |
*** suneelb has quit IRC | 22:13 | |
jogo | that is breaking everything even worse http://logs.openstack.org/35/171435/2/gate/gate-tempest-docs/49e026b/console.html#_2015-04-21_22_10_21_938 | 22:13 |
*** yamahata has quit IRC | 22:13 | |
*** suneelb has joined #openstack-qa | 22:14 | |
jogo | sdague fungi dtroyer | 22:14 |
*** andreykurilin__ has quit IRC | 22:14 | |
*** suneelb1 has joined #openstack-qa | 22:14 | |
dtroyer | also, devstack is one thing, grenade totally another on the target side | 22:14 |
fungi | my wife is going to gnaw my arm if we don't head out to eat, but i'll catch back up after and see where i can help on this front | 22:14 |
notmyname | to facilitate getting RC candidates out, does it make sense to disable grenade to make it non-voting until this is sorted? | 22:15 |
sdague | dtroyer: yeh, it feels fragile to me | 22:15 |
dtroyer | so, venvs for everyone and post-install work out the differences to produce a single list? | 22:15 |
lifeless | so, one call makes things easier given our current structure | 22:15 |
lifeless | fixing pip 2687 would eliminate the need for a single call | 22:16 |
sdague | lifeless: not when we move across 2 trees, in the upgrade case | 22:16 |
sdague | anyway, tactical fixes | 22:16 |
sdague | for installing cliff at a version in devstack early might help | 22:17 |
lifeless | sdague: it would even for the upgrade case, but yes | 22:17 |
lifeless | I guess I'm saying | 22:17 |
lifeless | if its a three week endeavour | 22:17 |
lifeless | fixing pip will be faster | 22:17 |
lifeless | (not all of pip, 2687 specifically) | 22:17 |
sdague | lifeless: yeh, don't know. right now we should be looking at tactical fix here to unblock folks. | 22:18 |
*** patrickeast has joined #openstack-qa | 22:18 | |
jogo | sdague: so fixing cliff isn't even the top issue anymore it looks like | 22:18 |
jogo | sdague: yes its blocking but cryptography is failing to even install | 22:18 |
*** suneelb has quit IRC | 22:18 | |
dhellmann | jogo: is that a bad release of cryptography? | 22:19 |
jogo | dhellmann: not sure, didn't see a recent one | 22:19 |
dhellmann | hmm | 22:19 |
sdague | the release was on the 11th, so I doubt it | 22:20 |
jogo | http://logs.openstack.org/50/142950/3/gate/gate-horizon-pep8/b3c9895/console.html#_2015-04-21_21_54_50_733 is all I know right now | 22:20 |
fungi | right, we need a one-evening endeavor, so if we can get things flowing again with a few tags and patches then that's fine for the moment | 22:20 |
jogo | and taht it started in the past few hours | 22:20 |
fungi | anyway, i'll be back as quickly as i can | 22:20 |
sdague | jogo: something else might have flushed it | 22:20 |
sdague | one of the other libraries | 22:20 |
jogo | sdague: very possible yeah | 22:21 |
*** patrickeast has quit IRC | 22:21 | |
jogo | https://bugs.launchpad.net/openstack-gate/+bug/1446882 | 22:22 |
openstack | Launchpad bug 1446882 in OpenStack-Gate "cryptography is failing to install" [Undecided,New] | 22:22 |
lifeless | barbicanclient is another proximate cause here | 22:22 |
sdague | jogo: http://logs.openstack.org/54/157054/10/check/gate-tempest-docs/d7107ee/console.html | 22:23 |
dtroyer | neutronclient too, the three things that use cliff are the first-order problem. there may be other problems… | 22:23 |
sdague | is a working one, it has all these versions | 22:23 |
*** paul_glass1 has quit IRC | 22:23 | |
lifeless | its whats http://logs.openstack.org/89/165189/1/gate/gate-tempest-dsvm-full/c896f3b/logs/devstacklog.txt.gz#_2015-04-21_20_16_53_293 | 22:23 |
sdague | lifeless: right, that's the conduit for installing a too new cliff | 22:23 |
jogo | dhellmann: I don't think its related | 22:23 |
jogo | http://logs.openstack.org/61/176061/2/check/gate-horizon-pep8/d51a2d4/console.html | 22:23 |
jogo | failed | 22:23 |
*** patrickeast has joined #openstack-qa | 22:24 | |
jogo | and didn't insall barbican | 22:24 |
dtroyer | horizon pulls in neutronclient | 22:24 |
dhellmann | sdague: what would we have to do outside of cliff to let us release a version of cliff that has a different stevedore requirement? | 22:24 |
sdague | dhellmann: nothing, right? | 22:24 |
lifeless | can't we just up the stevedore minimum in juno ? | 22:24 |
dhellmann | sdague: well, we would have to land a requirements change | 22:25 |
dhellmann | I guess those jobs are probably still turned off | 22:25 |
sdague | dhellmann: no, requirements enforcement is off | 22:25 |
sdague | lifeless: we do that, that's the issue | 22:25 |
lifeless | sdague: huh? | 22:25 |
dtroyer | to get around this tonight, can we pre-install the previous cliff and stevedore before everything else? | 22:25 |
openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Add fingerprint fur bug 1446882 https://review.openstack.org/176107 | 22:25 |
openstack | bug 1446882 in OpenStack-Gate "cryptography is failing to install" [Undecided,New] https://launchpad.net/bugs/1446882 | 22:25 |
dhellmann | dtroyer: that seems like a good short-term approach | 22:25 |
dtroyer | unless minimums went up, that should still be good? | 22:25 |
jogo | sdague dtroyer have to go AFK for a bit but first step for this bug would be to try reproducing it locally | 22:26 |
lifeless | dtroyer: no, the problem is that the install of cinder will force upgrade stevedore | 22:26 |
sdague | lifeless: all the servers require stevedore <= 1.2 | 22:26 |
lifeless | sorry | 22:26 |
*** Rockyg has quit IRC | 22:26 | |
lifeless | I meant force upgrade cliff | 22:26 |
sdague | right, that's actually where we are | 22:26 |
sdague | cliff is too new | 22:26 |
lifeless | sdague: if we put requirements enforcement back on and upgraded the requirements in global/juno | 22:26 |
*** marun has joined #openstack-qa | 22:26 | |
lifeless | sdague: wouldn't that work, tactically? | 22:26 |
sdague | lifeless: no, that won't help | 22:27 |
lifeless | requirement of stevedore | 22:27 |
sdague | we're in pypi | 22:27 |
clarkb | requirement enforcement should be independent of this | 22:27 |
clarkb | thats a separate job | 22:27 |
sdague | and pip isn't tracking what's installed | 22:27 |
lifeless | sdague: huh | 22:27 |
sdague | or more importantly what things need | 22:27 |
sdague | so it installed a new cliff | 22:27 |
lifeless | sdague: yes I got that. | 22:27 |
openstackrecheck | openstack/nova change: https://review.openstack.org/140669 failed because of: gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847, gate-tempest-dsvm-postgres-full: unrecognized error | 22:28 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 22:28 |
sdague | which violates the stevedore requirement that other software that was previously installed needs | 22:28 |
dtroyer | so I just ran a stable/juno devstack on a trusty that had pre-today packages and it worked | 22:28 |
lifeless | sdague: yes, if 2687 was fixed it would error isntead | 22:28 |
lifeless | sdague: which would be no better | 22:28 |
sdague | lifeless: correct | 22:29 |
lifeless | sdague: let me try asking my question a different way | 22:29 |
lifeless | sdague: what is it about stevedore > 1.2 that breaks servers? | 22:29 |
morganfainberg | mtreinish, ok i think i need to go unwind a bunch of v2/v3 keystone isms in devstack that seem to be arbitrarily different. | 22:29 |
morganfainberg | mtreinish, and that we hardcode v2.0 in all over... fantastic. | 22:29 |
sdague | lifeless: I don't know | 22:29 |
dtroyer | it's above the cap? | 22:29 |
lifeless | dtroyer: that was my assumption | 22:29 |
lifeless | so my proposal is just to lift the cap | 22:30 |
clarkb | lifeless: this is why I am a fan of not doing this | 22:30 |
dtroyer | morganfainberg: good luck my friend, that's a dark and perious journey you are about to undertake | 22:30 |
clarkb | because by not capping/pinning we actually know what works and what doesn't work | 22:30 |
morganfainberg | dtroyer, well we need to be able to deploy in devstack w/o v2.0 enabled | 22:30 |
clarkb | in our curret world we have none of that information so when it becomes important to change a thing we are doing it blind | 22:30 |
morganfainberg | dtroyer, the first step: default to v3, optionally allow v2.0 | 22:30 |
sdague | clarkb: that's fine, then it's just fire drill all the time on stable | 22:30 |
lifeless | clarkb: I agree | 22:30 |
lifeless | clarkb: ut | 22:30 |
lifeless | clarkb: but | 22:30 |
morganfainberg | dtroyer, i'm already sorry i started looking at this. | 22:30 |
clarkb | sdague: honestly I am not sure it is | 22:30 |
sdague | clarkb: it really was | 22:30 |
clarkb | sdague: we seem to be doing just as much firedrill with the stuff in place as we did then | 22:30 |
lifeless | clarkb: there are tradeoffs. lets not get distracted | 22:31 |
lifeless | right now we need tactical fix. | 22:31 |
sdague | yes, agreed, tactical | 22:31 |
lifeless | I was not happy before the super tight pinning | 22:31 |
lifeless | and I'm not happy now either :) | 22:31 |
*** Sukhdev has quit IRC | 22:31 | |
lifeless | so I, along with everyone that cares is going to put effort into figuring out a long solution: just not today :) | 22:31 |
mtreinish | morganfainberg: well to be fair a lot of that probably pre-dates v3 | 22:32 |
*** mwagner_lap has joined #openstack-qa | 22:32 | |
mtreinish | or at least I'm assuming | 22:32 |
sdague | so, agree, if we raise the cap, that might get us tactical fix | 22:32 |
sdague | lifeless: you want to propose that into stable/juno ? | 22:32 |
morganfainberg | mtreinish, sure. but it's like "oh we stick an option for v2 / v3 here" and here and here and over there | 22:32 |
lifeless | sdague: no, you can | 22:32 |
lifeless | sdague: I'm mainly sheparding the next release of pip right now - this gets a priority interrupt to help | 22:33 |
lifeless | sdague: but if there is someone else driving thats great | 22:33 |
morganfainberg | mtreinish, i think i'm going to rewire all this to use the IDENTITY_API_VERSION | 22:33 |
sdague | lifeless: well it's better if someone other than doug or I do it, as we're the only +2ers here. | 22:33 |
lifeless | that sounds like beauracracy gone wild | 22:34 |
morganfainberg | mtreinish, and i apologize in advance this wont be pretty... but at least it'll get us down to "use vX of keystone" in one location | 22:34 |
dhellmann | sdague: I'm willing to W+1 a patch you make if it fixes this | 22:35 |
*** cdent has quit IRC | 22:35 | |
dtroyer | so, in parallel i'm trying this: https://review.openstack.org/176111 | 22:35 |
sdague | dtroyer: you'll need to make something else that's on master Depends-On that for testing | 22:36 |
dtroyer | err, I did juno, should have done kilo? | 22:38 |
*** BadCub is now known as BadCub_Away | 22:38 | |
sdague | dtroyer: no, in juno | 22:38 |
mtreinish | morganfainberg: yeah I was going to recommend a global option for doing that. Anything else would be kinda crazy | 22:38 |
sdague | kilo upgrade hasn't landed yet, because of a keystone / oslo issue | 22:38 |
sdague | dhellmann: https://review.openstack.org/#/c/176113/ | 22:38 |
dhellmann | sdague: ok, I'll keep an eye on the tests for that | 22:39 |
sdague | I kind of assume this is going to explode in an odd way | 22:39 |
dhellmann | yeah, let's see what happens | 22:39 |
sdague | because what stevedore's requirements are | 22:39 |
*** jecarey has quit IRC | 22:39 | |
dhellmann | stevedore doesn't depend on much, just pbr and six | 22:40 |
openstackrecheck | openstack/tempest change: https://review.openstack.org/157054 failed because of: gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847, gate-tempest-docs: unrecognized error, gate-tempest-dsvm-full: unrecognized error, gate-tempest-dsvm-heat: unrecognized error, gate-tempest-dsvm-large-ops: unrecognized error, gate- | 22:41 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 22:41 |
openstackrecheck | tempest-dsvm-neutron-full: unrecognized error, gate-tempest-dsvm-neutron-large-ops: unrecognized error, gate-tempest-dsvm-nova-v21-full: unrecognized error, gate-tempest-dsvm-postgres-full: unrecognized error, gate-tempest-dsvm-trove: unrecognized error, gate-tempest-pep8: unrecognized error, gate-tempest-python27: unrecognized error | 22:41 |
*** bknudson has quit IRC | 22:41 | |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add command to calculate failure count and graph it https://review.openstack.org/176043 | 22:43 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db api method to get status time series https://review.openstack.org/176042 | 22:43 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add start and stop date options to subunit2sql-graph https://review.openstack.org/176114 | 22:43 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add modular subunit2sql-graph entry point https://review.openstack.org/176115 | 22:43 |
*** dboik_ has joined #openstack-qa | 22:50 | |
*** dboik has quit IRC | 22:53 | |
*** Swami has joined #openstack-qa | 22:53 | |
openstackrecheck | openstack/tempest change: https://review.openstack.org/175447 failed because of: gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847, gate-tempest-docs: unrecognized error, gate-tempest-dsvm-full: unrecognized error, gate-tempest-dsvm-heat: unrecognized error, gate-tempest-dsvm-large-ops: unrecognized error, gate- | 22:54 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 22:54 |
openstackrecheck | tempest-dsvm-neutron-full: unrecognized error, gate-tempest-dsvm-neutron-large-ops: unrecognized error, gate-tempest-dsvm-nova-v21-full: unrecognized error, gate-tempest-dsvm-postgres-full: unrecognized error, gate-tempest-dsvm-trove: unrecognized error, gate-tempest-pep8: unrecognized error, gate-tempest-python27: unrecognized error | 22:54 |
*** Longgeek has quit IRC | 22:54 | |
morganfainberg | mtreinish: so i need to do some magic replacement stuff | 22:56 |
morganfainberg | mtreinish: get_keystone_version_for_endpoint_url | 22:56 |
morganfainberg | mtreinish: this needs to be v2.0 or v3 | 22:56 |
openstackrecheck | openstack/tempest change: https://review.openstack.org/173136 failed because of: gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847, gate-tempest-docs: unrecognized error, gate-tempest-dsvm-full: unrecognized error, gate-tempest-dsvm-heat: unrecognized error, gate-tempest-dsvm-large-ops: unrecognized error, gate- | 22:56 |
openstack | Launchpad bug 1446847 in OpenStack-Gate "stable/juno gate is busted on stevedore/cliff releases on 4/21" [Undecided,New] | 22:56 |
openstackrecheck | tempest-dsvm-neutron-full: unrecognized error, gate-tempest-dsvm-neutron-large-ops: unrecognized error, gate-tempest-dsvm-nova-v21-full: unrecognized error, gate-tempest-dsvm-postgres-full: unrecognized error, gate-tempest-dsvm-trove: unrecognized error, gate-tempest-pep8: unrecognized error, gate-tempest-python27: unrecognized error | 22:56 |
morganfainberg | mtreinish: should i put this in functions or somewhere like in the lib/keystone | 22:56 |
openstackrecheck | openstack/tempest change: https://review.openstack.org/171435 failed because of: gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1446847, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1446847, gate-tempest-docs: unrecognized error, gate-tempest-dsvm-full: unrecognized error, gate-tempest-dsvm-heat: unrecognized error, gate-tempest-dsvm-large-ops: unrecognized error, gate- | 22:56 |
openstackrecheck | tempest-dsvm-neutron-full: unrecognized error, gate-tempest-dsvm-neutron-large-ops: unrecognized error, gate-tempest-dsvm-nova-v21-full: unrecognized error, gate-tempest-dsvm-postgres-full: unrecognized error, gate-tempest-dsvm-trove: unrecognized error, gate-tempest-pep8: unrecognized error, gate-tempest-python27: unrecognized error | 22:57 |
morganfainberg | mtreinish: i *think* lib/keystone makes sense... | 22:57 |
mtreinish | morganfainberg: sure, you just gotta make sure it's loaded everywhere you'll need the endpoint | 22:59 |
morganfainberg | mtreinish, it looks like it is loaded early in stach.sh | 22:59 |
morganfainberg | stack.sh | 22:59 |
morganfainberg | early enough | 22:59 |
morganfainberg | that is | 22:59 |
*** ibiris is now known as ibiris_away | 22:59 | |
*** dboik_ has quit IRC | 22:59 | |
openstackgerrit | Sean Dague proposed openstack-dev/devstack: testr requires python's gdbm https://review.openstack.org/176012 | 22:59 |
*** mlavalle has quit IRC | 23:03 | |
*** annashen has joined #openstack-qa | 23:03 | |
openstackgerrit | Merged openstack-infra/elastic-recheck: Add fingerprint fur bug 1446882 https://review.openstack.org/176107 | 23:05 |
openstack | bug 1446882 in OpenStack-Gate "cryptography is failing to install" [Undecided,New] https://launchpad.net/bugs/1446882 | 23:05 |
*** ajmiller_ is now known as ajmiller | 23:05 | |
*** annashen has quit IRC | 23:08 | |
sdague | dhellmann: ok, so I'm dropping off, but this is where I left things | 23:09 |
sdague | https://jenkins06.openstack.org/job/check-grenade-dsvm/25944/console is the grenade job for | 23:09 |
sdague | https://review.openstack.org/#/c/176012/ | 23:09 |
sdague | which is a noop in the gate, but it depends on the requirements change in juno | 23:09 |
sdague | so if that grenade job passes, the juno requirements thing worked | 23:10 |
sdague | I'll be back on in probably 11 hrs or so, and will see what pieces I can pick up if everything is still bonkers | 23:10 |
dhellmann | sdague: ok, I'll keep an eye on that job for a bit and if it passes approve the requirements thing | 23:11 |
*** yamamoto has joined #openstack-qa | 23:11 | |
*** hemna is now known as hemnafk | 23:12 | |
sdague | oh, never mind, everything is fubar by the cffi / pyparsing thing | 23:12 |
*** suneelb1 has quit IRC | 23:12 | |
*** suneelb has joined #openstack-qa | 23:13 | |
sdague | hey, that's need we've got 2 simultaneous gate wedges | 23:13 |
sdague | one because of capped requirements | 23:13 |
sdague | and one because of uncapped requirements | 23:13 |
sdague | ... giving up for the day, and getting a drink | 23:14 |
jogo | sdague: cffi is is https://bugs.launchpad.net/openstack-gate/+bug/1446882 ? | 23:16 |
openstack | Launchpad bug 1446882 in OpenStack-Gate "cryptography is failing to install" [Undecided,New] | 23:16 |
clarkb | jogo: yes | 23:16 |
clarkb | jogo: pyopenssl depends on cryptography for its ssl bindings, cryptography depends on cffi to do its binding which depends on pycparser to do its C parsing and pycparser released today and is broken | 23:17 |
jogo | doh so it did | 23:17 |
jogo | because that is a transitive dependency what do we do now? | 23:19 |
jogo | get cffi to cut a new lib? | 23:19 |
*** yamamoto has quit IRC | 23:21 | |
clarkb | we can pin pycparser everywhere | 23:21 |
clarkb | we can pin to a pre python cryptography release of pyopensslthough I think some projects use cryptography directly | 23:21 |
jogo | clarkb: we have to roll out patches to each repo for that though | 23:21 |
clarkb | jogo: yes likely | 23:22 |
jogo | wow https://github.com/pyca/cryptography/issues/1858 | 23:25 |
jogo | that travels fast | 23:25 |
clarkb | well its likely a bug in pycparser or cffi assuming that cryptography's C is valid C | 23:27 |
clarkb | even if it isn't valid C cffi/pycparser should error more gracefully so you can fix your C | 23:27 |
jogo | clarkb: very likely | 23:28 |
*** dimtruck is now known as zz_dimtruck | 23:34 | |
openstackgerrit | Dimitri John Ledkov proposed openstack-dev/devstack: Support os-release files. https://review.openstack.org/176129 | 23:37 |
fungi | so... scrollback is a doozy | 23:39 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add command to calculate failure count and graph it https://review.openstack.org/176043 | 23:39 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db api method to get status time series https://review.openstack.org/176042 | 23:39 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add start and stop date options to subunit2sql-graph https://review.openstack.org/176114 | 23:39 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add modular subunit2sql-graph entry point https://review.openstack.org/176115 | 23:39 |
fungi | actually only ~150 lines... not so bad | 23:39 |
*** sabeen3 has quit IRC | 23:40 | |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Add section to config guide for setting up services https://review.openstack.org/174149 | 23:41 |
*** openstackrecheck has quit IRC | 23:41 | |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Add a paragraph on the use of tempest_roles option https://review.openstack.org/174675 | 23:41 |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Fix non-locking test accounts doc section https://review.openstack.org/176026 | 23:41 |
*** openstackrecheck has joined #openstack-qa | 23:41 | |
*** ajmiller has quit IRC | 23:41 | |
*** tochi has joined #openstack-qa | 23:42 | |
fungi | i wonder if the pycparser issues are related to the memory usage reduction work they did after ianw and lifeless stumbled over how much additional memory newer pyopenssl was consuming because of it | 23:44 |
lifeless | fungi: I just kibbitzed, it was all ianw | 23:46 |
lifeless | I commented because it seemed to me that we were (and still are) in finger pointing mode in the bug | 23:46 |
fungi | yeah, it's not really gone anywhere other than pycparser squeezing out a mild (not order of magnitude) reduction in memory waste | 23:47 |
ianw | lifeless / fungi : yeah, for reference work was done on https://github.com/eliben/pycparser/issues/72 | 23:47 |
fungi | certainly not enough for us to start using it in devstack successfully anyway | 23:47 |
fungi | also, in response to sdague's comments about requirements checks being suspended, i only stopped them on master branches not stable branches | 23:48 |
fungi | and https://review.openstack.org/176036 can undo that as soon as we're ready | 23:49 |
fungi | i'm just hesitant to approve it until we're closer to having problems ironed out in stable | 23:49 |
*** bknudson has joined #openstack-qa | 23:54 | |
*** rwsu has quit IRC | 23:56 | |
*** melwitt has quit IRC | 23:56 | |
*** yamamoto has joined #openstack-qa | 23:57 | |
*** ibiris_away is now known as ibiris | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!