| *** fbo is now known as fbo_away | 00:00 | |
| *** che-arne has quit IRC | 00:00 | |
| *** rongze has quit IRC | 00:00 | |
| *** jcooley_ has joined #openstack-infra | 00:01 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add graphite graph of gate failure rates to e-r page https://review.openstack.org/58019 | 00:03 |
|---|---|---|
| *** jcooley_ has quit IRC | 00:06 | |
| *** w_ has joined #openstack-infra | 00:07 | |
| *** nono1 has quit IRC | 00:09 | |
| *** nono1_ has quit IRC | 00:10 | |
| *** olaph has quit IRC | 00:10 | |
| *** sarob_ has joined #openstack-infra | 00:10 | |
| *** matsuhashi has joined #openstack-infra | 00:12 | |
| *** weshay has joined #openstack-infra | 00:12 | |
| *** sarob has quit IRC | 00:13 | |
| *** ryanpetrello has joined #openstack-infra | 00:13 | |
| *** ryanpetrello has quit IRC | 00:13 | |
| *** sarob has joined #openstack-infra | 00:14 | |
| *** sarob_ has quit IRC | 00:15 | |
| *** dkranz has joined #openstack-infra | 00:15 | |
| *** yamahata_ has joined #openstack-infra | 00:16 | |
| *** gyee has quit IRC | 00:18 | |
| jog0 | the new graph looks good http://status.openstack.org/elastic-recheck/ | 00:21 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Install graphite-web and carbon with pip. https://review.openstack.org/58653 | 00:22 |
| jeblair | neato! | 00:22 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Fix the logprint in task_manager.py https://review.openstack.org/58294 | 00:23 |
| *** dolphm_a_ has joined #openstack-infra | 00:24 | |
| *** luisg has quit IRC | 00:25 | |
| openstackgerrit | A change was merged to openstack-infra/config: Use pip install instead of setup.py install https://review.openstack.org/51004 | 00:25 |
| clarkb | oh hmm I just realized my d-g refactor will change the names of all of those jobs >_> | 00:25 |
| jeblair | clarkb: https://jenkins01.openstack.org/job/gate-horizon-python27-django14/2199/console | 00:26 |
| jeblair | i just logged into precise5 and verified that git clean -xfdq on .tox in that workspace does work | 00:26 |
| *** dolphm_afk has quit IRC | 00:26 | |
| jeblair | i do not know why the job failed at the time :( | 00:26 |
| jeblair | fungi, mordred: ^ fyi | 00:27 |
| clarkb | maybe something had the file open? a stuck process | 00:28 |
| clarkb | s/the file/a file in .tox/ | 00:28 |
| *** dolphm_a_ has quit IRC | 00:28 | |
| * fungi is back, caught up on scrollback and following along | 00:28 | |
| jeblair | clarkb: yeah; can't think of what though. | 00:28 |
| jeblair | lsof showed nothing | 00:29 |
| jeblair | (as expected, since it worked, so i expect it was transient) | 00:29 |
| *** yamahata_ has quit IRC | 00:29 | |
| lifeless | jeblair: how's your queue looking ? ;) | 00:30 |
| fungi | we're going on the assumption that "warning: failed to remove .tox/" accompanied a nonzero return code from git-clean, failing out the job? bizarre circumstance to be sure | 00:30 |
| clarkb | fungi: that was my assumption | 00:30 |
| *** amotoki has joined #openstack-infra | 00:30 | |
| jeblair | lifeless: 8 reviews, but i'm caught up to "today" so i can reasonably task-switch to the gearman thing | 00:31 |
| fungi | so, yeah, with lsof turning up nothing post-mortem, i'd still go with "open file handle" but maybe only very, very briefly | 00:31 |
| fungi | like, i wonder if the git-reset had really finished when it returned control to the calling process | 00:32 |
| lifeless | jeblair: cool; thanks! I added a comment which I think will do the right thing, but I'm no expert | 00:33 |
| lifeless | jeblair: on https://review.openstack.org/#/c/58305/1/elements/testenv-worker/testenv-worker | 00:33 |
| fungi | clarkb: should we consider beefing up the git-clean calls in the run scripts with a || { sleep 1; git clean -blah } maybe? | 00:34 |
| jeblair | lifeless: is 'tripleo-gate' the job that is meant to run a test (eg, report failure to gerrit or similar)? | 00:36 |
| jeblair | fungi: couldn't hurt? | 00:36 |
| clarkb | fungi: maybe? would git reset open files in .tox? I suppose it probably does | 00:36 |
| fungi | clarkb: i suspect it could open files just about, you know, anywhere within the git root | 00:37 |
| lifeless | jeblair: yes, its devstack-gate but doing tripleo not devstack | 00:37 |
| fungi | clarkb: jeblair: i'll submit a "couldn't hurt" patch for those. belt and braces and what have you | 00:37 |
| *** paul-- has joined #openstack-infra | 00:39 | |
| *** sarob has quit IRC | 00:40 | |
| *** sarob has joined #openstack-infra | 00:40 | |
| fungi | clarkb: jeblair: was there a bug which prompted discussion of that gate-horizon-python27-django14 fail? | 00:41 |
| clarkb | ArxCruz: you about? looking at https://review.openstack.org/#/c/53432/6 and wondering about dprince's comment | 00:41 |
| clarkb | ArxCruz: devstack-cache.py may not work with utf8 (working to check with a quick test) and devstack-cache.py is very debian specific | 00:42 |
| *** sarob has quit IRC | 00:42 | |
| ArxCruz | clarkb: I did the changes | 00:42 |
| clarkb | ArxCruz: did you plan to update devstack-cache.py in a subsequent change? | 00:42 |
| *** sarob has joined #openstack-infra | 00:42 | |
| clarkb | nevermind i think it will handle utf8 | 00:43 |
| ArxCruz | clarkb: I'm not understanding | 00:43 |
| *** sarob has quit IRC | 00:43 | |
| ArxCruz | clarkb: you mean the Schrodinger's cat ? | 00:43 |
| *** sarob has joined #openstack-infra | 00:44 | |
| clarkb | ArxCruz: yes, but I think I have convinced myself the cat isn't a problem | 00:44 |
| clarkb | ArxCruz: but the deb stuff in devstack-cache.py will need to also do yum | 00:44 |
| ArxCruz | clarkb: okay, I can work on devstack-cache.py :) | 00:44 |
| clarkb | cool, I will go ahead and approve this first bit of work, but the whole stack may be unhappy if you throw it at fedora now :) | 00:45 |
| ArxCruz | :) | 00:45 |
| mgagne | looks like it's a bad idea to rename a jenkins slave while there is a build running... | 00:48 |
| jeblair | lifeless: responded; i think the thing you identified as a possible issue is an issue | 00:49 |
| lifeless | jeblair: nuts; going to read | 00:51 |
| *** pete5 has quit IRC | 00:52 | |
| *** markwash has quit IRC | 00:52 | |
| *** dolphm_afk has joined #openstack-infra | 00:53 | |
| openstackgerrit | A change was merged to openstack-infra/config: Adding nodepool compatibility with Fedora. https://review.openstack.org/53432 | 00:54 |
| *** dolphm_afk has quit IRC | 00:58 | |
| lifeless | jeblair: ok follow up q | 01:03 |
| *** CaptTofu has quit IRC | 01:03 | |
| *** ^demon|away has quit IRC | 01:04 | |
| *** CaptTofu has joined #openstack-infra | 01:07 | |
| *** reed has quit IRC | 01:07 | |
| openstackgerrit | Clay Gerrard proposed a change to openstack-dev/hacking: Add noqa support for H201 (bare except) https://review.openstack.org/57334 | 01:11 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: If running git clean after reset then retry https://review.openstack.org/58654 | 01:11 |
| *** markmcclain has joined #openstack-infra | 01:14 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/devstack-gate: If running git clean after reset then retry https://review.openstack.org/58655 | 01:18 |
| fungi | addressed for jenkins jobs, ggp and d-g ^ | 01:18 |
| clarkb | jeblair: can you read my comments at https://review.openstack.org/#/c/53917/ and see if that is something we should be considering as more compute drivers are required to have testing | 01:19 |
| clarkb | fungi: ^ your opinion appreciated as well. looking at your change now | 01:19 |
| *** nosnos has joined #openstack-infra | 01:20 | |
| fungi | clarkb: thanks | 01:21 |
| jog0 | http://status.openstack.org/zuul/ | 01:22 |
| jog0 | what does the gate graph mean? | 01:22 |
| jog0 | 5 lines coming off the top | 01:22 |
| jog0 | oh never mind | 01:23 |
| jog0 | it means lots of gate fails | 01:23 |
| clarkb | jog0: I think it is in the process of rerendering as it processes the results queue | 01:23 |
| fungi | jog0: it means a lot of recalculation and merges considered | 01:23 |
| clarkb | jog0: and yes lots of gate fails forcing it to do that | 01:23 |
| fungi | jog0: note the the "Queue lengths: 4 events, 39 results." | 01:23 |
| fungi | once those settle to 0/0 the subway map will probably make more sense too | 01:24 |
| jog0 | fungi: what does that mean | 01:24 |
| jog0 | 4 events | 01:24 |
| fungi | events received by zuul, results determined form the events | 01:24 |
| fungi | er, from | 01:24 |
| fungi | results determined from the jobs, rather. events are going to be gerrit stream events in this case? | 01:25 |
| jog0 | so we have kept the gate delay under 3 hours all day from what I can tell | 01:25 |
| jog0 | not too bad | 01:25 |
| fungi | anyway, yes, it's recalculating the subway map now. looks like a slew of conflicting and/or failing horizon changes | 01:25 |
| *** rongze has joined #openstack-infra | 01:26 | |
| fungi | oh, nice, a couple pbr changes further down which needed rebasing | 01:27 |
| fungi | as well as a tempest one in the same boat | 01:27 |
| jog0 | fungi: ahh I see now that makes sense | 01:27 |
| *** rongze has quit IRC | 01:31 | |
| *** sjing has joined #openstack-infra | 01:31 | |
| clarkb | jeblair: ArxCruz: for https://review.openstack.org/#/c/54196/18, is that new option required? | 01:32 |
| clarkb | jeblair: ArxCruz: I think we should make it optional so that we are backward compatible with old configs | 01:33 |
| ArxCruz | clarkb: yes, I will create a new patch for change in o-i/config | 01:33 |
| clarkb | ArxCruz: that is the other option, but making it optional makes other users of zuul happy I think | 01:33 |
| clarkb | ArxCruz: let us see what jeblair has to say | 01:33 |
| clarkb | and I will leave my thoughts on the review | 01:33 |
| ArxCruz | clarkb: okay up to you guys | 01:33 |
| *** mattoliverau has quit IRC | 01:34 | |
| *** matsuhashi has quit IRC | 01:34 | |
| lifeless | jeblair: do you think it's possible to timeout in the client code for that race condition on unregistered functions? | 01:35 |
| fungi | clarkb: the multi-cell tempest idea isn't without merit. i think maybe you should float it in front of the #-qa crowd when they're around too | 01:35 |
| clarkb | fungi: good idea, I will do that | 01:36 |
| fungi | clarkb: my only concern (and this might just be showing my ignorance of the implementation) is that introducing various cell types in a single tempest run could make failures harder to pin down and classify, but maybe that's just a question of having the right logging detail | 01:37 |
| clarkb | fungi: it definitely does add some complexity | 01:37 |
| clarkb | fungi: unfortunately I think some of the problems with my ideas are that nova isn't capable of doing what I expect it to do | 01:38 |
| fungi | i know people you can complain to about that, though i suspect the answer will be "patches welcome" | 01:38 |
| clarkb | fungi: I think this has been a long desired feature of nova | 01:39 |
| clarkb | :) | 01:39 |
| *** herndon has quit IRC | 01:39 | |
| *** yamahata_ has joined #openstack-infra | 01:42 | |
| openstackgerrit | A change was merged to openstack-infra/git-review: Preserve merges when doing a rebase https://review.openstack.org/50400 | 01:43 |
| *** sarob_ has joined #openstack-infra | 01:46 | |
| *** sarob has quit IRC | 01:49 | |
| *** mgagne has quit IRC | 01:50 | |
| *** matsuhashi has joined #openstack-infra | 01:51 | |
| *** sarob_ has quit IRC | 01:51 | |
| *** UtahDave has quit IRC | 01:52 | |
| *** dolphm_afk has joined #openstack-infra | 01:54 | |
| *** markmcclain has quit IRC | 01:55 | |
| *** senk has joined #openstack-infra | 01:56 | |
| clarkb | zaro: https://review.openstack.org/#/c/54639/ I have comments there. lgtm but want to double check the diff list as it looks small | 01:57 |
| *** dolphm_afk has quit IRC | 01:59 | |
| *** yongli has joined #openstack-infra | 02:02 | |
| *** bingbu has joined #openstack-infra | 02:05 | |
| *** yongli has quit IRC | 02:07 | |
| *** yongli has joined #openstack-infra | 02:08 | |
| *** bknudson has quit IRC | 02:08 | |
| *** rongze has joined #openstack-infra | 02:09 | |
| *** yongli has quit IRC | 02:10 | |
| *** guohliu has joined #openstack-infra | 02:11 | |
| *** yaguang has joined #openstack-infra | 02:12 | |
| *** melwitt has quit IRC | 02:12 | |
| *** yongli has joined #openstack-infra | 02:13 | |
| *** yongli has quit IRC | 02:16 | |
| *** yongli has joined #openstack-infra | 02:16 | |
| *** michchap has quit IRC | 02:18 | |
| *** senk has quit IRC | 02:19 | |
| *** weshay has quit IRC | 02:21 | |
| *** yongli has quit IRC | 02:24 | |
| *** yongli has joined #openstack-infra | 02:24 | |
| *** dims has quit IRC | 02:25 | |
| jog0 | http://status.openstack.org/elastic-recheck/ that looks a little short | 02:29 |
| *** mattoliverau has joined #openstack-infra | 02:33 | |
| *** jerryz has quit IRC | 02:33 | |
| jog0 | did something change how we handle bugs with no hits? | 02:35 |
| jog0 | because the last entry bug 1225664 doesn't appear to have any hits | 02:35 |
| *** jhesketh has quit IRC | 02:35 | |
| uvirtbot | Launchpad bug 1225664 in devstack "Swift: No space left on device" [Undecided,Fix committed] https://launchpad.net/bugs/1225664 | 02:35 |
| westmaas | mordred: around? | 02:37 |
| westmaas | just curious who owns the rs account you use for testing | 02:38 |
| westmaas | jeblair: ^^ | 02:38 |
| westmaas | mostly I'm curious what the tenant ID is and what size image you use, if you don't use the base image | 02:41 |
| clarkb | we use 8gb nodes | 02:43 |
| clarkb | with 4vcpu | 02:43 |
| clarkb | then start with the base precise and centos6 imagines and puppet them | 02:44 |
| clarkb | *images | 02:44 |
| westmaas | cool | 02:45 |
| westmaas | think I found the tenant, thanks! | 02:49 |
| *** Shrews has quit IRC | 02:49 | |
| *** fifieldt has joined #openstack-infra | 02:53 | |
| *** matsuhashi has quit IRC | 03:00 | |
| *** wenlock has joined #openstack-infra | 03:01 | |
| *** markwash has joined #openstack-infra | 03:01 | |
| openstackgerrit | lifeless proposed a change to openstack-infra/reviewstats: Fix importing of utils in two scripts. https://review.openstack.org/58662 | 03:11 |
| openstackgerrit | lifeless proposed a change to openstack-infra/reviewstats: Remove unused import in reviews_for_bugs. https://review.openstack.org/58663 | 03:11 |
| openstackgerrit | lifeless proposed a change to openstack-infra/reviewstats: Document the JSON schema a little. https://review.openstack.org/58664 | 03:15 |
| *** paul-- has quit IRC | 03:42 | |
| *** reaper has joined #openstack-infra | 03:44 | |
| mikal | jog0: you still around? | 03:55 |
| *** jhesketh has joined #openstack-infra | 03:55 | |
| *** pcrews has quit IRC | 03:58 | |
| *** mihgen has joined #openstack-infra | 04:06 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Fix utils import https://review.openstack.org/58666 | 04:18 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Remove received reviews from output. https://review.openstack.org/58667 | 04:18 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Fix utils import https://review.openstack.org/58666 | 04:20 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Remove received reviews from output. https://review.openstack.org/58667 | 04:20 |
| *** matsuhashi has joined #openstack-infra | 04:23 | |
| *** dolphm_afk has joined #openstack-infra | 04:25 | |
| *** chandankumar has quit IRC | 04:25 | |
| *** chandankumar has joined #openstack-infra | 04:25 | |
| *** dolphm_afk has quit IRC | 04:28 | |
| *** jcooley_ has joined #openstack-infra | 04:30 | |
| *** paul-- has joined #openstack-infra | 04:44 | |
| *** dolphm_afk has joined #openstack-infra | 04:44 | |
| *** rongze has quit IRC | 04:48 | |
| *** mriedem has quit IRC | 04:49 | |
| *** sandywalsh has quit IRC | 04:56 | |
| *** nicedice has quit IRC | 05:07 | |
| *** hartsocks has quit IRC | 05:07 | |
| *** yamahata_ has quit IRC | 05:08 | |
| *** hartsocks has joined #openstack-infra | 05:08 | |
| *** sarob has joined #openstack-infra | 05:15 | |
| *** rongze has joined #openstack-infra | 05:18 | |
| *** oubiwann has quit IRC | 05:19 | |
| lifeless | Alex_Gaynor: the pbr patches need rebasing :) | 05:22 |
| *** slong has joined #openstack-infra | 05:23 | |
| Alex_Gaynor | lifeless: I didn't notice it was based on the other one, it's a different error message then when the patch itself has a merge conflict :/ | 05:23 |
| lifeless | yah | 05:23 |
| *** slong has left #openstack-infra | 05:24 | |
| *** colinmcnamara has joined #openstack-infra | 05:25 | |
| *** rongze has quit IRC | 05:27 | |
| *** colinmcnamara has quit IRC | 05:28 | |
| *** sandywalsh has joined #openstack-infra | 05:31 | |
| *** michchap has joined #openstack-infra | 05:34 | |
| openstackgerrit | Michael Davies proposed a change to openstack-infra/config: Adding in Matt Oliver to rcbau team https://review.openstack.org/58674 | 05:36 |
| *** sjing has quit IRC | 05:40 | |
| openstackgerrit | Michael Davies proposed a change to openstack-infra/config: Adding in Matt Oliver to rcbau team https://review.openstack.org/58674 | 05:41 |
| *** sjing has joined #openstack-infra | 05:42 | |
| *** sarob has quit IRC | 05:44 | |
| *** sarob has joined #openstack-infra | 05:45 | |
| *** sarob has quit IRC | 05:49 | |
| *** vipul is now known as vipul-away | 05:52 | |
| *** jorisroovers has joined #openstack-infra | 05:53 | |
| *** afazekas has joined #openstack-infra | 05:53 | |
| openstackgerrit | Michael Davies proposed a change to openstack-infra/config: Adding in Matt Oliver to rcbau team https://review.openstack.org/58674 | 05:57 |
| *** SergeyLukjanov has joined #openstack-infra | 05:57 | |
| *** w_ has quit IRC | 05:57 | |
| *** senk has joined #openstack-infra | 05:58 | |
| *** jcooley_ has quit IRC | 05:59 | |
| *** jcooley_ has joined #openstack-infra | 05:59 | |
| *** vipul-away is now known as vipul | 06:01 | |
| *** jcooley_ has quit IRC | 06:05 | |
| *** dolphm_afk has quit IRC | 06:06 | |
| *** dolphm_a_ has joined #openstack-infra | 06:08 | |
| *** w_ has joined #openstack-infra | 06:09 | |
| yaguang | stable/grizzly jenkins fails duo to error: Installed distribution iso8601 0.1.4 conflicts with requirement iso8601>=0.1.8 | 06:10 |
| yaguang | full log is here http://logs.openstack.org/38/58538/1/check/check-tempest-devstack-vm-neutron/114ef62/console.html | 06:10 |
| *** dolphm_a_ has quit IRC | 06:10 | |
| *** sarob has joined #openstack-infra | 06:11 | |
| *** CaptTofu has quit IRC | 06:11 | |
| *** CaptTofu has joined #openstack-infra | 06:12 | |
| *** rongze has joined #openstack-infra | 06:13 | |
| *** mrda has quit IRC | 06:23 | |
| *** mattoliverau has quit IRC | 06:29 | |
| *** boris-42 has joined #openstack-infra | 06:30 | |
| *** nosnos_ has joined #openstack-infra | 06:30 | |
| *** nosnos has quit IRC | 06:34 | |
| *** ArxCruz has quit IRC | 06:36 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 06:37 | |
| *** _SergeyLukjanov has quit IRC | 06:38 | |
| *** guohliu has quit IRC | 06:42 | |
| *** yongli has quit IRC | 06:44 | |
| *** yongli has joined #openstack-infra | 06:44 | |
| *** w_ has quit IRC | 06:45 | |
| *** yongli has quit IRC | 06:47 | |
| *** yongli has joined #openstack-infra | 06:47 | |
| *** SergeyLukjanov has joined #openstack-infra | 06:52 | |
| *** SergeyLukjanov has quit IRC | 06:52 | |
| *** yongli has quit IRC | 06:57 | |
| *** yongli has joined #openstack-infra | 06:58 | |
| *** olaph has joined #openstack-infra | 07:00 | |
| *** matsuhashi has quit IRC | 07:05 | |
| *** matsuhashi has joined #openstack-infra | 07:05 | |
| *** nati_uen_ has quit IRC | 07:09 | |
| *** yonglihe_ has joined #openstack-infra | 07:13 | |
| *** yolanda has joined #openstack-infra | 07:13 | |
| *** yonglihe_ has quit IRC | 07:15 | |
| *** senk has quit IRC | 07:15 | |
| *** yonglihe_ has joined #openstack-infra | 07:15 | |
| *** yongli has quit IRC | 07:16 | |
| *** yonglihe_ has quit IRC | 07:16 | |
| *** yongli has joined #openstack-infra | 07:17 | |
| *** markwash has quit IRC | 07:17 | |
| *** yonglihe_ has joined #openstack-infra | 07:19 | |
| *** yonglihe_ has quit IRC | 07:19 | |
| *** slong has joined #openstack-infra | 07:19 | |
| *** paul-- has quit IRC | 07:19 | |
| *** yongli has quit IRC | 07:20 | |
| *** nosnos_ has quit IRC | 07:20 | |
| *** yongli has joined #openstack-infra | 07:21 | |
| *** nosnos has joined #openstack-infra | 07:21 | |
| *** dolphm_afk has joined #openstack-infra | 07:21 | |
| *** yonglihe_ has joined #openstack-infra | 07:22 | |
| *** yonglihe_ has quit IRC | 07:22 | |
| *** pblaho has joined #openstack-infra | 07:24 | |
| *** slong has quit IRC | 07:25 | |
| *** dolphm_afk has quit IRC | 07:25 | |
| *** denis_makogon has joined #openstack-infra | 07:26 | |
| *** yongli has quit IRC | 07:28 | |
| *** yongli has joined #openstack-infra | 07:28 | |
| *** yongli has quit IRC | 07:29 | |
| *** yongli has joined #openstack-infra | 07:30 | |
| openstackgerrit | Swann Croiset proposed a change to openstack-infra/config: Climate project: add gerrit tags ACL https://review.openstack.org/58688 | 07:30 |
| *** markwash has joined #openstack-infra | 07:31 | |
| *** yongli has quit IRC | 07:32 | |
| *** yongli has joined #openstack-infra | 07:32 | |
| *** nati_ueno has joined #openstack-infra | 07:34 | |
| *** yongli has quit IRC | 07:38 | |
| *** yongli has joined #openstack-infra | 07:38 | |
| *** sarob has quit IRC | 07:46 | |
| *** sarob has joined #openstack-infra | 07:47 | |
| *** flaper87|afk is now known as flaper87 | 07:49 | |
| *** yongli has quit IRC | 07:49 | |
| *** yongli has joined #openstack-infra | 07:50 | |
| *** yongli has quit IRC | 07:51 | |
| *** sarob has quit IRC | 07:51 | |
| *** yongli has joined #openstack-infra | 07:51 | |
| *** vipul is now known as vipul-away | 07:51 | |
| *** reaper has quit IRC | 07:53 | |
| *** yongli has quit IRC | 07:58 | |
| *** yongli has joined #openstack-infra | 07:59 | |
| *** boris-42 has quit IRC | 07:59 | |
| *** yonglihe_ has joined #openstack-infra | 07:59 | |
| *** ruhe has joined #openstack-infra | 08:01 | |
| *** matsuhashi has quit IRC | 08:01 | |
| *** matsuhashi has joined #openstack-infra | 08:03 | |
| *** yongli has quit IRC | 08:03 | |
| *** fifieldt has quit IRC | 08:03 | |
| *** yongli has joined #openstack-infra | 08:03 | |
| *** yongli has quit IRC | 08:04 | |
| *** fbo_away is now known as fbo | 08:05 | |
| *** yonglihe_ has quit IRC | 08:06 | |
| *** yongli has joined #openstack-infra | 08:06 | |
| *** yongli has quit IRC | 08:06 | |
| *** vipul-away is now known as vipul | 08:06 | |
| *** yongli has joined #openstack-infra | 08:07 | |
| *** yonglihe_ has joined #openstack-infra | 08:08 | |
| *** yonglihe_ has quit IRC | 08:10 | |
| *** yonglihe_ has joined #openstack-infra | 08:11 | |
| *** yonglihe_ has quit IRC | 08:11 | |
| *** yonglihe_ has joined #openstack-infra | 08:13 | |
| *** mihgen has quit IRC | 08:13 | |
| *** yonglihe_ has quit IRC | 08:13 | |
| *** yonglihe_ has joined #openstack-infra | 08:15 | |
| *** yonglihe_ has quit IRC | 08:15 | |
| *** yongli has quit IRC | 08:16 | |
| *** yongli has joined #openstack-infra | 08:17 | |
| *** yonglihe_ has joined #openstack-infra | 08:18 | |
| *** marun has quit IRC | 08:18 | |
| *** yonglihe_ has quit IRC | 08:18 | |
| *** paul-- has joined #openstack-infra | 08:27 | |
| *** yongli has quit IRC | 08:28 | |
| *** yongli has joined #openstack-infra | 08:29 | |
| *** yongli has quit IRC | 08:30 | |
| *** yongli has joined #openstack-infra | 08:30 | |
| *** yongli has quit IRC | 08:31 | |
| *** yongli has joined #openstack-infra | 08:31 | |
| *** yonglihe_ has joined #openstack-infra | 08:32 | |
| *** yonglihe_ has quit IRC | 08:33 | |
| *** che-arne has joined #openstack-infra | 08:34 | |
| *** sjing has quit IRC | 08:41 | |
| *** yongli has quit IRC | 08:42 | |
| *** yongli has joined #openstack-infra | 08:42 | |
| *** hashar has joined #openstack-infra | 08:42 | |
| *** denis_makogon has quit IRC | 08:44 | |
| *** SergeyLukjanov has joined #openstack-infra | 08:48 | |
| *** yonglihe_ has joined #openstack-infra | 08:52 | |
| *** yonglihe_ has quit IRC | 08:52 | |
| *** yongli has quit IRC | 08:53 | |
| *** yongli has joined #openstack-infra | 08:53 | |
| *** ruhe has quit IRC | 08:54 | |
| *** yonglihe_ has joined #openstack-infra | 08:54 | |
| *** yonglihe_ has quit IRC | 08:54 | |
| *** paul-- has quit IRC | 08:55 | |
| *** mihgen has joined #openstack-infra | 08:59 | |
| *** jpich has joined #openstack-infra | 09:01 | |
| *** nati_ueno has quit IRC | 09:01 | |
| *** markmc has joined #openstack-infra | 09:02 | |
| *** DinaBelova has joined #openstack-infra | 09:04 | |
| *** jcoufal has joined #openstack-infra | 09:05 | |
| *** derekh has joined #openstack-infra | 09:05 | |
| *** yongli has quit IRC | 09:09 | |
| *** yongli has joined #openstack-infra | 09:11 | |
| *** yassine has joined #openstack-infra | 09:12 | |
| *** yonglihe_ has joined #openstack-infra | 09:16 | |
| *** yongli has quit IRC | 09:16 | |
| *** yongli has joined #openstack-infra | 09:17 | |
| *** yongli has quit IRC | 09:17 | |
| *** dstanek has quit IRC | 09:21 | |
| *** yonglihe_ has quit IRC | 09:26 | |
| *** yongli has joined #openstack-infra | 09:26 | |
| *** yonglihe_ has joined #openstack-infra | 09:27 | |
| *** yonglihe_ has quit IRC | 09:27 | |
| openstackgerrit | Valeriy Ponomaryov proposed a change to openstack-infra/config: Add gate-manila-devstack job https://review.openstack.org/57987 | 09:28 |
| *** pblaho has quit IRC | 09:28 | |
| *** yongli has quit IRC | 09:29 | |
| *** yongli has joined #openstack-infra | 09:30 | |
| *** paul-- has joined #openstack-infra | 09:30 | |
| *** talluri has joined #openstack-infra | 09:30 | |
| *** yonglihe_ has joined #openstack-infra | 09:31 | |
| *** yonglihe_ has quit IRC | 09:32 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add gate-climate-devstack https://review.openstack.org/58702 | 09:38 |
| *** yongli has quit IRC | 09:38 | |
| *** yongli has joined #openstack-infra | 09:39 | |
| *** yonglihe_ has joined #openstack-infra | 09:40 | |
| *** yonglihe_ has quit IRC | 09:42 | |
| *** yongli has quit IRC | 09:42 | |
| *** yongli has joined #openstack-infra | 09:42 | |
| *** yonglihe_ has joined #openstack-infra | 09:44 | |
| *** yonglihe_ has quit IRC | 09:46 | |
| *** yongli has quit IRC | 09:46 | |
| *** yongli has joined #openstack-infra | 09:47 | |
| *** yongli has quit IRC | 09:49 | |
| *** yongli has joined #openstack-infra | 09:49 | |
| *** yamahata_ has joined #openstack-infra | 09:50 | |
| *** matsuhashi has quit IRC | 09:51 | |
| *** ben_duyujie has joined #openstack-infra | 09:52 | |
| *** yonglihe_ has joined #openstack-infra | 09:52 | |
| *** yonglihe_ has quit IRC | 09:52 | |
| *** pblaho has joined #openstack-infra | 09:52 | |
| *** yonglihe_ has joined #openstack-infra | 09:54 | |
| *** yonglihe_ has quit IRC | 09:54 | |
| *** matsuhas_ has joined #openstack-infra | 09:55 | |
| *** yonglihe_ has joined #openstack-infra | 09:56 | |
| *** yonglihe_ has quit IRC | 09:57 | |
| openstackgerrit | Swann Croiset proposed a change to openstack-infra/config: Climate project: add gerrit tags ACL https://review.openstack.org/58688 | 10:00 |
| *** ruhe has joined #openstack-infra | 10:05 | |
| *** osanchez has joined #openstack-infra | 10:07 | |
| *** ryanpetrello has joined #openstack-infra | 10:10 | |
| *** matsuhas_ has quit IRC | 10:11 | |
| *** matsuhashi has joined #openstack-infra | 10:11 | |
| *** ryanpetrello has quit IRC | 10:12 | |
| *** masayukig has joined #openstack-infra | 10:17 | |
| *** rongze has quit IRC | 10:21 | |
| *** nati_ueno has joined #openstack-infra | 10:21 | |
| *** jhesketh has quit IRC | 10:22 | |
| *** paul-- has quit IRC | 10:25 | |
| *** ruhe has quit IRC | 10:27 | |
| *** paul-- has joined #openstack-infra | 10:28 | |
| *** ruhe has joined #openstack-infra | 10:28 | |
| *** rongze_ has joined #openstack-infra | 10:31 | |
| *** ruhe has quit IRC | 10:33 | |
| *** boris-42 has joined #openstack-infra | 10:35 | |
| *** paul-- has quit IRC | 10:37 | |
| *** mrda has joined #openstack-infra | 10:38 | |
| *** DinaBelova has quit IRC | 10:42 | |
| *** ryanpetrello has joined #openstack-infra | 10:44 | |
| *** yamahata_ has quit IRC | 10:44 | |
| *** ryanpetrello has quit IRC | 10:44 | |
| *** paul-- has joined #openstack-infra | 10:45 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 10:46 | |
| *** _SergeyLukjanov has quit IRC | 10:47 | |
| *** avishayb has joined #openstack-infra | 10:48 | |
| chandankumar | Hello | 10:50 |
| chandankumar | Anybody can help me here? | 10:50 |
| chandankumar | I am not able to openstack Wiki through launchpad credentials | 10:51 |
| chandankumar | it is always saying There is no user by the name "chandankumar.093047". Usernames are case sensitive. Check your spelling, | 10:51 |
| chandankumar | i have tried with changing username in different case, but not able to login | 10:52 |
| chandankumar | Please help me? | 10:52 |
| *** mrda has quit IRC | 10:57 | |
| *** bingbu has quit IRC | 10:59 | |
| soren | chandankumar: https://launchpad.net/~chandankumar.093047 agrees that you don't exist. | 11:02 |
| chandankumar | soren, https://launchpad.net/~chandankumar-093047 | 11:02 |
| chandankumar | soren, i am viewing my profile | 11:03 |
| chandankumar | soren, here is my http://postimg.org/image/tr6u9eisz/ | 11:04 |
| *** SergeyLukjanov has joined #openstack-infra | 11:04 | |
| *** ArxCruz has joined #openstack-infra | 11:04 | |
| chandankumar | soren, what is happening with it? | 11:05 |
| soren | chandankumar: Well, a minus and a dot aren't the same thing. | 11:05 |
| *** rongze_ has quit IRC | 11:06 | |
| *** DinaBelova has joined #openstack-infra | 11:06 | |
| *** jorisroovers has quit IRC | 11:06 | |
| *** jorisroovers has joined #openstack-infra | 11:06 | |
| chandankumar | soren, There is no user by the name "chandankumar-093047". Usernames are case sensitive. Check your spelling, or create a new account. | 11:06 |
| chandankumar | soren, i have tried all of them, but donot why it is not working | 11:07 |
| soren | So you go to wiki.openstack.org, click the login link and you end up on login.launchpad.net, right? | 11:07 |
| *** pcm_ has joined #openstack-infra | 11:08 | |
| *** fbo is now known as fbo_away | 11:08 | |
| *** pcm_ has joined #openstack-infra | 11:08 | |
| chandankumar | soren, on clicking login i get this link https://wiki.openstack.org/w/index.php?title=Special:UserLogin&returntoquery=welcome%3Dyes | 11:08 |
| chandankumar | soren, there it doesnot come login.lauchpad.net | 11:09 |
| *** paul-- has quit IRC | 11:09 | |
| chandankumar | soren, any ide | 11:10 |
| chandankumar | *idea | 11:10 |
| soren | chandankumar: Try a different browser. You should definitely be taken to login.launchpad.net at some point in the process. | 11:10 |
| *** matsuhashi has quit IRC | 11:10 | |
| soren | Or, use your current browser's inkognito mode, if it has such a mode. | 11:11 |
| chandankumar | soren, i am suing firefox | 11:11 |
| chandankumar | *using | 11:12 |
| chandankumar | how to go in that mode? | 11:12 |
| chandankumar | soren, in different browser i get loggined | 11:13 |
| *** wenlock has quit IRC | 11:13 | |
| chandankumar | soren, Thank you | 11:15 |
| soren | Clear your cookies for the wiki.openstack.org site then and you should be fine. | 11:15 |
| *** rongze has joined #openstack-infra | 11:15 | |
| chandankumar | soren, yes | 11:15 |
| soren | chandankumar: You're welcome. | 11:16 |
| *** amotoki has quit IRC | 11:21 | |
| *** SergeyLukjanov has quit IRC | 11:34 | |
| *** SergeyLukjanov has joined #openstack-infra | 11:36 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 11:37 | |
| *** ryanpetrello has joined #openstack-infra | 11:38 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 11:38 | |
| *** smulcahy has joined #openstack-infra | 11:39 | |
| smulcahy | I've had a gate failure but can't see the cause other than that keystone did not start which seems unrelated to my change - https://jenkins01.openstack.org/job/check-tempest-devstack-vm-full/8457/ | 11:39 |
| smulcahy | am I missing something or are there any other known issues with keystone failing to start? | 11:39 |
| smulcahy | I couldn't see anything obvious on http://status.openstack.org/rechecks/ | 11:40 |
| *** pcm_ has quit IRC | 11:42 | |
| *** ruhe has joined #openstack-infra | 11:42 | |
| *** ryanpetrello has quit IRC | 11:43 | |
| *** ruhe has quit IRC | 11:48 | |
| *** ruhe has joined #openstack-infra | 11:49 | |
| *** nati_ueno has quit IRC | 11:49 | |
| ttx | hrm.. Who is this jishaomei who spams reviews with "starting gate jobs" comments ? | 11:53 |
| *** jroovers has joined #openstack-infra | 11:53 | |
| ttx | (https://review.openstack.org/#/c/58635/) | 11:53 |
| *** nosnos has quit IRC | 11:53 | |
| jd__ | ttx: haha, now we have Jenkins spammer ;-) | 11:54 |
| jd__ | (someone installed zuul with a default config or something?) | 11:54 |
| *** jorisroovers has quit IRC | 11:55 | |
| *** ruhe has quit IRC | 12:00 | |
| openstackgerrit | Arx Cruz proposed a change to openstack-infra/config: Zuul puppet support to zuul_url option https://review.openstack.org/58739 | 12:01 |
| *** rfolco has joined #openstack-infra | 12:01 | |
| *** ArxCruz has quit IRC | 12:03 | |
| *** sandywalsh has quit IRC | 12:04 | |
| *** lcestari has joined #openstack-infra | 12:05 | |
| *** pcm_ has joined #openstack-infra | 12:06 | |
| *** masayukig has quit IRC | 12:09 | |
| *** pcm_ has quit IRC | 12:12 | |
| *** pcm_ has joined #openstack-infra | 12:13 | |
| *** yaguang has quit IRC | 12:14 | |
| *** Loquacity has quit IRC | 12:15 | |
| *** weshay has joined #openstack-infra | 12:17 | |
| *** jroovers has quit IRC | 12:18 | |
| *** CaptTofu has quit IRC | 12:28 | |
| *** CaptTofu has joined #openstack-infra | 12:28 | |
| *** yamahata_ has joined #openstack-infra | 12:32 | |
| *** ruhe has joined #openstack-infra | 12:35 | |
| *** mattymo has joined #openstack-infra | 12:37 | |
| mattymo | this is just a general question regarding launchpad (no response in their channel) - anyone know how to set a restricted list of approvers for blueprints in a particular LP project? | 12:38 |
| mattymo | it's for a stackforge project, so it's still on topic | 12:38 |
| *** avishayb has quit IRC | 12:41 | |
| *** avishayb has joined #openstack-infra | 12:42 | |
| *** [1]avishayb has joined #openstack-infra | 12:43 | |
| soren | Hi. How does the elastic recheck thing guess which bugs are being hit? | 12:45 |
| soren | ...and how can I provide hints for it? | 12:45 |
| *** jorisroovers has joined #openstack-infra | 12:46 | |
| *** avishayb has quit IRC | 12:47 | |
| *** [1]avishayb is now known as avishayb | 12:47 | |
| *** jroovers has joined #openstack-infra | 12:47 | |
| ttx | soren: I think it relies on search patterns being posted as comments on bugs | 12:48 |
| soren | I think I just found it, actually. | 12:49 |
| *** fbo_away is now known as fbo | 12:49 | |
| soren | The elastic-recheck project has a queries.yaml file that I think is what I want. | 12:49 |
| ttx | sdague: what does "UNSTABLE" mean in test results ? Looked into the logs but could not find a clear failure (https://review.openstack.org/#/c/58634/) | 12:50 |
| *** rfolco has quit IRC | 12:50 | |
| ttx | mattymo: the project "drivers" have power to set blueprints fields | 12:50 |
| *** jorisroovers has quit IRC | 12:51 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 12:51 | |
| ttx | sdague: I guess it's all truncated | 12:52 |
| *** _SergeyLukjanov has quit IRC | 12:52 | |
| ttx | (no console.html) | 12:53 |
| *** sandywalsh has joined #openstack-infra | 12:54 | |
| openstackgerrit | Soren Hansen proposed a change to openstack-infra/elastic-recheck: Add pattern for bug 1236524 https://review.openstack.org/58749 | 12:54 |
| uvirtbot | Launchpad bug 1236524 in tempest "tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern fails sporadically" [Undecided,New] https://launchpad.net/bugs/1236524 | 12:54 |
| sdague | ttx: UNSTABLE means something went wrong with artifact collection | 12:56 |
| sdague | so it's unknown | 12:56 |
| ttx | sdague: is it fair game to reverify no bug UNSTABLE stuff ? | 12:57 |
| sdague | ttx: yes | 12:57 |
| ttx | I guess I'll do that then | 12:57 |
| sdague | soren: ?? curious about your elastic-recheck diving :) | 12:57 |
| BobBall | If we're removing reverify no bug , should we auto re-test unstable results once? | 12:58 |
| ttx | BobBall: or we could have a specific syntax. "reverify because UNSTABLE" | 12:58 |
| BobBall | yeah, that'd make sense | 12:59 |
| soren | ttx: https://jenkins01.openstack.org/job/gate-tempest-devstack-vm-full/23285/ <-- Says canceled. | 12:59 |
| ttx | soren: yeah, no reverifying it is | 12:59 |
| soren | sdague: I think I worked it out (see the elastic-recheck change proposed a few lines further up). | 12:59 |
| ttx | wow. When wa job gets cancelled due to something higher up failing, it gets automatically reverified when the root one gets reverified. Shiny. | 13:00 |
| soren | We've all been replaced by a small shell script. Darn. | 13:01 |
| ttx | soren: it's just a matter of time | 13:01 |
| ttx | small shell scripts already fight our wars for us. | 13:01 |
| *** jroovers has quit IRC | 13:02 | |
| BobBall | who would have thought that skynet would be written in bash | 13:03 |
| sdague | soren: do you have a review where it failed? we typically try to find an issue in one of the service logs which seems to be triggering it if we can as first stage triage, not just the test that failed. | 13:08 |
| *** CaptTofu has quit IRC | 13:08 | |
| *** CaptTofu has joined #openstack-infra | 13:09 | |
| sdague | and that match hits a lot of logstash hits, so I'm wondering if we already have other fingerprints for most of that bug | 13:09 |
| soren | sdague: Hm. No. Let me see if I can find anything useful. | 13:09 |
| *** sandywalsh_ has joined #openstack-infra | 13:11 | |
| *** dolphm_afk has joined #openstack-infra | 13:11 | |
| *** sandywalsh has quit IRC | 13:12 | |
| hashar | hello | 13:13 |
| *** mkoderer has quit IRC | 13:13 | |
| *** Alex_Gaynor has quit IRC | 13:14 | |
| smulcahy | I did a recheck on my review and it passed check-tempest-devstack-vm-full this but now fails on check-grenade-devstack-vm - whats the recommended way to deal with this? https://review.openstack.org/#/c/58328/ | 13:14 |
| smulcahy | 2013-11-27 12:07:12.829 | [ERROR] ./grenade.sh:268 Failure in upgrade-nova | 13:15 |
| *** marun has joined #openstack-infra | 13:15 | |
| smulcahy | sounds like https://bugs.launchpad.net/grenade/+bug/1249218 | 13:15 |
| uvirtbot | Launchpad bug 1249218 in grenade "Grenade - "Failure in upgrade-nova" due to apt-get failure" [Undecided,New] | 13:15 |
| *** ben_duyujie has quit IRC | 13:19 | |
| openstackgerrit | sebastian marcet proposed a change to openstack-infra/config: Review and create Puppet configuration for dev server https://review.openstack.org/53644 | 13:21 |
| *** sandywalsh has joined #openstack-infra | 13:22 | |
| *** sandywalsh_ has quit IRC | 13:22 | |
| *** Alex_Gaynor has joined #openstack-infra | 13:23 | |
| *** talluri has quit IRC | 13:23 | |
| sdague | smulcahy: hmm we should never be making apt calls | 13:25 |
| *** alexpilotti has joined #openstack-infra | 13:25 | |
| *** dolphm_afk is now known as dolphm | 13:27 | |
| soren | sdague: I see nothing interesting in any of the relevant logs :( | 13:32 |
| *** SergeyLukjanov has joined #openstack-infra | 13:32 | |
| sdague | ok. Can you put a known failing review into a comment on your patch? maybe I can swing around and see if I can help narrow it later | 13:35 |
| *** thomasem has joined #openstack-infra | 13:35 | |
| *** Shrews has joined #openstack-infra | 13:35 | |
| smulcahy | sdague: thats an existing bug but looks like it bit my review - definitely trying to apt-get | 13:36 |
| smulcahy | sdague: running a recheck now to see what happens this time :) | 13:36 |
| soren | sdague: I can... but I think I've found the cause of the bug, so I'll just try fixing that instead :) | 13:37 |
| sdague | soren: that works for me :) | 13:37 |
| *** eharney has joined #openstack-infra | 13:37 | |
| sdague | smulcahy: yeh, that one we should circle with infra on | 13:38 |
| *** mkoderer has joined #openstack-infra | 13:38 | |
| *** DinaBelova has quit IRC | 13:40 | |
| *** DinaBelova has joined #openstack-infra | 13:40 | |
| *** dstanek has joined #openstack-infra | 13:42 | |
| *** dprince has joined #openstack-infra | 13:43 | |
| *** julim has joined #openstack-infra | 13:44 | |
| openstackgerrit | Chuck Short proposed a change to openstack/requirements: Add HACKING.rst https://review.openstack.org/55909 | 13:46 |
| *** CaptTofu has quit IRC | 13:53 | |
| *** CaptTofu has joined #openstack-infra | 13:53 | |
| *** mfer has joined #openstack-infra | 13:54 | |
| * fungi checks to see what's broken while he slept | 13:59 | |
| *** dolphm has quit IRC | 14:04 | |
| *** rnirmal has joined #openstack-infra | 14:06 | |
| zul | sdague: hey does the sqlite database needs to exist in the nova testsuite? http://pastebin.ubuntu.com/6484453/ while running nova in our buildds | 14:06 |
| fungi | ttx: jishaomei has been requesting a service account for ibm china to do third-party testing... i have to assume for a nova or neutron driver, but not certain | 14:06 |
| * fungi circles back around to the ml to see if fellow ibmers finally steered them to provide the correct info. language barrier problems there, i've suspected | 14:07 | |
| mattymo | fungi, I'm having some issues with LP and maybe you have a fast answer. If not, it's fine. | 14:10 |
| mattymo | fungi, in our fuel group anyone can approve their own blueprint, but I couldn't do that if I was to make a nova blueprint | 14:10 |
| mattymo | why is that? | 14:10 |
| fungi | mattymo: you're trying to find the members list for a drivers group on an lp project, right? | 14:10 |
| mattymo | I have someone who approved his own BP that isn't a driver | 14:11 |
| mattymo | and I'm trying to figure out how it's possible | 14:11 |
| fungi | mattymo: it's almost certainly a configuration option for the project in lp | 14:11 |
| * fungi looks to see if he can tell | 14:11 | |
| mattymo | yeah but I couldn't find such an option | 14:11 |
| *** dolphm has joined #openstack-infra | 14:12 | |
| fungi | mattymo: so the person who self-approved their bp was not in this list? https://launchpad.net/~fuel-drivers/+members | 14:13 |
| mattymo | indeed | 14:13 |
| *** ruhe has quit IRC | 14:13 | |
| openstackgerrit | Julien Danjou proposed a change to openstack/requirements: Add support for Python 3 requirements https://review.openstack.org/58770 | 14:13 |
| openstackgerrit | Julien Danjou proposed a change to openstack/requirements: Add support for Python 3 requirements https://review.openstack.org/58770 | 14:14 |
| fungi | mmm, it's not in the obvious place (configure blueprints) | 14:14 |
| *** michchap has quit IRC | 14:15 | |
| fungi | mattymo: i definitely can't find any relevant option on a project i control. next place i'd probably ask would be #launchpad (they seem to be fairly attentive when i've had to ask them things) | 14:17 |
| *** ruhe has joined #openstack-infra | 14:18 | |
| mattymo | I got some sort of contradicting answer - that anyone can change if they are a member of the base group | 14:18 |
| mattymo | and approve even | 14:18 |
| mattymo | because it allows devels to move ahead without needing admins | 14:18 |
| *** marun has quit IRC | 14:19 | |
| fungi | base group meaning the project driver group, or some other group? | 14:19 |
| *** senk has joined #openstack-infra | 14:20 | |
| fungi | i will admit, i haven't done anything with blueprints before since the infra team projects tend not to use them | 14:21 |
| *** jergerber has joined #openstack-infra | 14:31 | |
| openstackgerrit | Julien Danjou proposed a change to openstack/requirements: Add oslo-incubator to automatic update https://review.openstack.org/58779 | 14:32 |
| *** yamahata_ has quit IRC | 14:32 | |
| *** yaguang has joined #openstack-infra | 14:36 | |
| *** sgordon has joined #openstack-infra | 14:36 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/reviewstats: Update tempest core group https://review.openstack.org/58780 | 14:38 |
| *** sandywalsh_ has joined #openstack-infra | 14:39 | |
| *** marun has joined #openstack-infra | 14:39 | |
| *** sandywalsh has quit IRC | 14:39 | |
| openstackgerrit | Chuck Short proposed a change to openstack/requirements: Add HACKING.rst https://review.openstack.org/55909 | 14:40 |
| *** dims has joined #openstack-infra | 14:45 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: ModuleRegistry now caches entry points lookup https://review.openstack.org/58783 | 14:49 |
| *** dolphm has quit IRC | 14:54 | |
| *** mriedem has joined #openstack-infra | 14:54 | |
| *** boris-42 has quit IRC | 14:54 | |
| *** dolphm has joined #openstack-infra | 14:54 | |
| *** oubiwann has joined #openstack-infra | 14:59 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: ModuleRegistry now caches entry points lookup https://review.openstack.org/58783 | 15:00 |
| *** smulcahy has left #openstack-infra | 15:00 | |
| *** esker has joined #openstack-infra | 15:00 | |
| *** rcleere has joined #openstack-infra | 15:04 | |
| *** dprince has quit IRC | 15:06 | |
| sdague | mattymo: there is a seperate nova-drivers group that is the admins for the launchpad project | 15:08 |
| sdague | only they can change blueprint status | 15:08 |
| *** talluri has joined #openstack-infra | 15:11 | |
| fungi | sdague: but what's odd is that the fuel project has a fuel-drivers group as its administrator and driver, yet someone apparently not a member of that group approved their own bp | 15:14 |
| *** ryanpetrello has joined #openstack-infra | 15:16 | |
| *** pcm__ has joined #openstack-infra | 15:17 | |
| *** pcm_ has quit IRC | 15:19 | |
| *** ruhe has quit IRC | 15:20 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Add new doctools project https://review.openstack.org/58623 | 15:22 |
| *** ruhe has joined #openstack-infra | 15:22 | |
| *** ryanpetrello has quit IRC | 15:22 | |
| openstackgerrit | Liang Chen proposed a change to openstack-dev/pbr: Make sphinx builders configurable in LocalBuildDoc https://review.openstack.org/57892 | 15:23 |
| *** avishayb has quit IRC | 15:23 | |
| *** sandywalsh_ has quit IRC | 15:23 | |
| alaski | A couple of us were trying to figure out lp fields for blueprints, and it seems that blueprint owners can set the definition to approved, but not the direction | 15:31 |
| *** DinaBelova has quit IRC | 15:33 | |
| *** mriedem has quit IRC | 15:35 | |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Update tempest core group https://review.openstack.org/58780 | 15:38 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Fix importing of utils in two scripts. https://review.openstack.org/58662 | 15:38 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Remove unused import in reviews_for_bugs. https://review.openstack.org/58663 | 15:39 |
| *** reaper has joined #openstack-infra | 15:39 | |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Document the JSON schema a little. https://review.openstack.org/58664 | 15:39 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Remove received reviews from output. https://review.openstack.org/58667 | 15:40 |
| jeblair | clarkb: I asked ArxCruz to make it a required parameter; optional was complicated, required seems like a Good Idea, and others in previous reviews agreed. | 15:42 |
| *** mrodden has joined #openstack-infra | 15:43 | |
| clarkb | jeblair: wfm, I will wait for the openstack-infra/config change before approving (though you can approve if you think it is a ppropriate) | 15:44 |
| clarkb | also I am not actually awake yet :) | 15:44 |
| *** senk has quit IRC | 15:44 | |
| *** senk has joined #openstack-infra | 15:45 | |
| SergeyLukjanov | I'm working on moving ttx/openstack-releasing to openstack org | 15:45 |
| SergeyLukjanov | and it have launchpadlib in requirements | 15:46 |
| SergeyLukjanov | is it ok to propose it to the global requirements? | 15:46 |
| *** pete5 has joined #openstack-infra | 15:46 | |
| jeblair | SergeyLukjanov: it might be better to move it to openstack-infra or openstack-dev. | 15:46 |
| SergeyLukjanov | ttx said that it'll be a part of release management program | 15:47 |
| openstackgerrit | A change was merged to openstack-infra/config: If running git clean after reset then retry https://review.openstack.org/58654 | 15:47 |
| clarkb | jeblair: so I can approve the zuul config change then the zuul code change now? | 15:47 |
| jeblair | clarkb: i think so | 15:47 |
| clarkb | ok | 15:48 |
| jeblair | SergeyLukjanov: sure -- but it's not part of "OpenStack" -- the piece of software we are producing, so the openstack org may not be the right place | 15:48 |
| jeblair | SergeyLukjanov: and in fact, the requirements question you have indicates that exact problem -- we don't want to add launchpadlib to openstack/requirements because launchpadlib is not required for openstack | 15:49 |
| SergeyLukjanov | jeblair, sounds reasonable, looks like it should be discussed too | 15:49 |
| clarkb | jeblair: done | 15:49 |
| SergeyLukjanov | jeblair, it was about the right place | 15:49 |
| openstackgerrit | A change was merged to openstack-infra/config: Zuul puppet support to zuul_url option https://review.openstack.org/58739 | 15:50 |
| *** ryanpetrello has joined #openstack-infra | 15:52 | |
| jeblair | SergeyLukjanov, ttx: I would chose between -infra and -dev based on whether it's something openstack developers are expected to use, or whether someone running a project like openstack would use. | 15:52 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Add the possibility to pass the zuul url https://review.openstack.org/54196 | 15:52 |
| jeblair | SergeyLukjanov, ttx: that leads me toward thinking -infra is the right place, but i'm not very familiar with the repo. | 15:52 |
| SergeyLukjanov | jeblair, that's a bunch of sh and py scripts for releasing OpenStack | 15:53 |
| jeblair | SergeyLukjanov, ttx: don't worry about the fact that there's an infra program; other git orgs have repos from different programs too. :) | 15:53 |
| *** alexpilotti has quit IRC | 15:53 | |
| *** jcooley_ has joined #openstack-infra | 15:53 | |
| ttx | jeblair: that would be a release management program repo though | 15:54 |
| ttx | jeblair: hmm kay | 15:55 |
| jeblair | ttx: i know. i'm not claiming it. i'm saying i think it's okay to have non-infra repos in the infra org. | 15:55 |
| ttx | jeblair: that could arguably be considered confusing though. | 15:55 |
| jeblair | ttx: anyway, if you think it would be better in -dev, that's fine. | 15:55 |
| jeblair | ttx: i thing it should not be in openstack/ though. | 15:55 |
| *** UtahDave has joined #openstack-infra | 15:55 | |
| ttx | jeblair: it's somethign that PTLs and release managers are expected to use. So it's a bit in between | 15:55 |
| jeblair | think | 15:55 |
| jeblair | ttx: ptl use changes things a bit, and maybe moves it closer to -dev in my mind. | 15:56 |
| jd__ | does this type of failure ring a bell http://logs.openstack.org/79/58779/1/check/check-swift-devstack-vm-functional/19eeb35/console.html ? | 15:56 |
| openstackgerrit | A change was merged to openstack/requirements: Set boto minimum version https://review.openstack.org/51131 | 15:56 |
| ttx | jeblair: so you think of github orgs as differentiating based on who the target user is ? | 15:56 |
| jeblair | ttx: yes -- that's their primary purpose afaict -- so that when you go to list projects, you get projects of interest to you; either on github or git.o.o | 15:57 |
| ttx | jeblair: (FTR I think it's fine, it's just that the naming collides with program namespaces in a slightly confusing way in the -infra case) | 15:57 |
| *** senk has quit IRC | 15:58 | |
| * ttx looks at the list to see if all projects abide to this rule | 15:58 | |
| *** wenlock has joined #openstack-infra | 15:58 | |
| jeblair | ttx: yes. perhaps we want to preserve the org==program for infra. but it's definitely not true for openstack-dev and openstack/ | 15:59 |
| openstackgerrit | A change was merged to openstack/requirements: Add oslo-incubator to automatic update https://review.openstack.org/58779 | 15:59 |
| ttx | tempest/grenade/devstack belonging to 2 separate orgs would be the major offender | 15:59 |
| *** ruhe has quit IRC | 16:00 | |
| *** rnirmal has quit IRC | 16:00 | |
| jeblair | ttx: oh, are you looking at it in the other direction? whether a program has repos in different orgs? | 16:00 |
| jeblair | ttx: (though tempest==qa, devstack/grenade==devstack programs) | 16:01 |
| ttx | jeblair: no, looing at it from a user perspective. To me users of grenade and tempest are in the same category | 16:01 |
| *** alexpilotti has joined #openstack-infra | 16:01 | |
| ttx | Ah, isn't grenade QA ? | 16:01 |
| ttx | we really need tat mapping in programs.yaml | 16:01 |
| jeblair | ttx: heh, i don't know if you don't. :) | 16:02 |
| ttx | jeblair: ok. then grenade is QA :P | 16:02 |
| *** jcoufal has quit IRC | 16:02 | |
| jeblair | ttx: anyway, tempest is an end-user product for openstack -- we expect it to have the same audience, as, say, nova. | 16:03 |
| ttx | jeblair: so yeah, it could belong to openstack-dev or openstack-infra depending on how you look at it. | 16:03 |
| ttx | though I suspect the high adherence on our infrastructure makes it a -infra thing | 16:03 |
| ttx | as long as I keep control over those scripts i don't care that much | 16:05 |
| jeblair | sorting hat says... gryffindor! | 16:05 |
| SergeyLukjanov | ttx, from my pov release-tools looks like to be placed to -infra org if compare with -dev one | 16:05 |
| ttx | SergeyLukjanov says... -infra! | 16:06 |
| ttx | SergeyLukjanov: you're now our new sorting hat. | 16:06 |
| SergeyLukjanov | jeblair, openstack-gryffindor? :) | 16:06 |
| *** pblaho has quit IRC | 16:06 | |
| ogelbukh | lol | 16:06 |
| jeblair | ha! | 16:06 |
| *** ruhe has joined #openstack-infra | 16:07 | |
| ttx | jeblair: may I still retain +2 if it lands under infra org ? | 16:07 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Reformat nova.json https://review.openstack.org/58805 | 16:07 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Add mriedem and dan-prince to nova-core https://review.openstack.org/58806 | 16:07 |
| jeblair | ttx: it'll be all yours | 16:07 |
| ttx | MINE | 16:07 |
| ttx | MY PRECIOUS launchpadlib scripts | 16:07 |
| *** michchap has joined #openstack-infra | 16:08 | |
| ttx | nasty infra hobbit cant have it | 16:08 |
| SergeyLukjanov | :) | 16:08 |
| ttx | jeblair: another topic. Any idea how "Force+Push" can be specified in the gerrit-dev UI ? | 16:09 |
| ttx | jeblair: it's documented but there doesn't seem to be UI support for it | 16:10 |
| *** yolanda has quit IRC | 16:10 | |
| ttx | jeblair: just to make sure Force+Push on refs/for/* wouldn't magically do the trick | 16:10 |
| ttx | (for branch deletion) | 16:10 |
| jeblair | ttx: do you have a project i should look at? | 16:10 |
| ttx | https://review-dev.openstack.org/#/admin/projects/gtest-org/gtest,access | 16:11 |
| ttx | jeblair: was playing with that one ^ | 16:11 |
| ttx | reference: https://review-dev.openstack.org/Documentation/access-control.html#category_push | 16:11 |
| ttx | "Force option" | 16:12 |
| jeblair | ttx: if you add 'Push' perms for a ref, and then add a group to it, a 'force push' checkbox will show up. | 16:12 |
| jeblair | ttx: so in other words, it's an extra option for the 'push' right. | 16:12 |
| ttx | hhhrm. I tried that and failed, let me retry | 16:12 |
| ttx | ah yes. Weird | 16:13 |
| ttx | ok thx | 16:13 |
| jeblair | jd__: http://logs.openstack.org/79/58779/1/check/check-swift-devstack-vm-functional/19eeb35/logs/devstack-gate-setup-workspace-new.txt | 16:15 |
| jeblair | jd__: the error is error: Couldn't resolve host 'zuul.openstack.org' while accessing http://zuul.openstack.org/p/openstack/requirements/info/refs | 16:16 |
| ttx | jeblair: so yeah, force+push on refs/for/* doesn't allow branch deletion, while force+push on refs/heads/proposed/* allows proposed/* branch deletion | 16:16 |
| ttx | but i suspect that would also let me have ultimate power. | 16:16 |
| *** hartsocks has left #openstack-infra | 16:16 | |
| jeblair | ttx: that makes sense :( i think the safe thing is to wait for 2.8 for that | 16:16 |
| ttx | so maybe better to wait for REST | 16:16 |
| devananda | clarkb: remember that db locking issue from last week? oslo guys suggested this fix: https://review.openstack.org/#/c/56516/12/tox.ini | 16:16 |
| ttx | jeblair: ok, will proceed according to plan I just posted then | 16:17 |
| clarkb | devananda: how does running testr with oslo lockutils help? | 16:17 |
| devananda | clarkb: because https://github.com/openstack/oslo-incubator/blob/master/openstack/common/lockutils.py#L284 | 16:17 |
| clarkb | devananda: so I think we may be talking past each other | 16:17 |
| clarkb | devananda: the problem is that they are locking | 16:17 |
| clarkb | not that they need to fix the locking | 16:18 |
| devananda | hah | 16:18 |
| jeblair | ttx: does your plan say you want to 'git push' to 'refs/heads/proposed/*' ? | 16:18 |
| clarkb | devananda: each test process can have its own DB schema | 16:18 |
| clarkb | devananda: then you don't worry about locks | 16:18 |
| *** pcrews has joined #openstack-infra | 16:18 | |
| *** afazekas has quit IRC | 16:19 | |
| ttx | jeblair: "git push gerrit milestone-proposed" to create MP branches (using Create reference perms) | 16:19 |
| jeblair | jd__: seems like rax's dns servers may have had a (hopefully temporary) problem. | 16:19 |
| clarkb | devananda: I mean locking is one way to solve the problem... but imo is worse than the alternative of using different buckets | 16:20 |
| jeblair | ttx: okay, that doesn't require that we add the 'push' perm (which would allow you to push an arbitrary commit to the branch (bypassing review))? | 16:20 |
| jd__ | jeblair: what's weird is that zuul reverified on its own, is it new? https://review.openstack.org/#/c/58779/ | 16:20 |
| ttx | jeblair: no it's actually leveraging existing permissions | 16:20 |
| jd__ | or you did that behind the curtain :) | 16:20 |
| clarkb | jd__: check tests failed, gate tests passed | 16:21 |
| jeblair | jd__: no, first one was the check job, second gate. someone approved the change before the check jobs reported back. | 16:21 |
| jd__ | ah the approving was too fast, that's why I mixed, sorry guys :) | 16:21 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: If running git clean after reset then retry https://review.openstack.org/58655 | 16:21 |
| jd__ | and thanks | 16:21 |
| devananda | clarkb: sure. so the locking is only done on pgsql right now -- do i recall you saying there was an issue creating schemata on-demand for pg? | 16:21 |
| ttx | jeblair: (Release managers group already has "Create Reference" on refs/* for All-Projects) | 16:21 |
| *** mgagne has joined #openstack-infra | 16:22 | |
| clarkb | devananda: there is no issue with pg (or shouldn't be) | 16:22 |
| clarkb | devananda: mysql is the problem, we need to grant create and drop to the test user for scehams | 16:22 |
| devananda | hmm. well, https://github.com/openstack/oslo-incubator/blob/master/openstack/common/db/sqlalchemy/test_migrations.py#L167 | 16:22 |
| jeblair | markmc: http://xkcd.com/1296/ maybe time for a wiki page update? :) | 16:23 |
| clarkb | devananda: I don't think those tests are running on our slaves are they? | 16:23 |
| *** nati_ueno has joined #openstack-infra | 16:24 | |
| clarkb | devananda: I don't see anything setting that env var that says postgres is available on our slaves | 16:24 |
| clarkb | this almost feels like someone has reengineered a second way to do db testing... | 16:24 |
| devananda | clarkb: i'm trying to sync them from oslo. and it failed repeatedly on taht | 16:24 |
| markmc | jeblair, heh :) | 16:24 |
| devananda | maybe the problem is that I shouldn't be syncing this from oslo :p | 16:24 |
| markmc | jeblair, at least he's not terminating his first lines with a period | 16:24 |
| clarkb | devananda: I think the problem is that oslo hasn't communicated what their needs are to us and have just done their own thing | 16:25 |
| markmc | jeblair, perhaps adopting all caps for commit messages would be cool, though | 16:25 |
| clarkb | devananda: those tests will fail without locking if the same schema is used | 16:25 |
| jeblair | markmc: that seems canonical and authoritative enough for me! | 16:25 |
| clarkb | devananda: the tests need to know to use different schemas and manage them properly | 16:25 |
| devananda | clarkb: that's splendid. hi markmc :) | 16:26 |
| markmc | devananda, hey | 16:26 |
| devananda | er, i mean, hi dhellmann | 16:27 |
| devananda | (sorry markmc - so used to pinging you with oslo questions) | 16:27 |
| markmc | what's the story? | 16:27 |
| markmc | sounds like the infrastructure for running unit tests against real DBs ? | 16:27 |
| markmc | the patches just landed? | 16:27 |
| devananda | markmc: db migration code in oslo | 16:27 |
| devananda | fairly recently, yes. i've tried to sync them to ironic and $fail | 16:28 |
| markmc | $fail == ? | 16:28 |
| devananda | https://review.openstack.org/#/c/56516/12 | 16:28 |
| devananda | and https://review.openstack.org/#/c/58044/ | 16:28 |
| devananda | was my attempt to create a fix and port it back to oslo | 16:29 |
| markmc | ok, hadn't seen the pgadmin-tests lock before | 16:29 |
| devananda | markmc: seems to have been this patch https://github.com/openstack/oslo-incubator/commit/274c7e2b19c2f573bdecce026e779228e6eda0de | 16:31 |
| markmc | sounds like you've got it under control? | 16:31 |
| *** mihgen has quit IRC | 16:32 | |
| devananda | i can work around it, but i think clarkb doesnt like the lock-based solution in the first place | 16:32 |
| sdague | markmc: this might have gotten missed because oslo doesn't run testr parallel | 16:32 |
| clarkb | right, locking is one way to solve the problem if we are limited to a single bucket, but we can make arbitrary buckets | 16:32 |
| *** Hefeweizen has quit IRC | 16:33 | |
| clarkb | and as long as each hose only ever fills one bucket everything should just work (tm) | 16:33 |
| *** cyeoh has quit IRC | 16:33 | |
| sdague | clarkb: sure | 16:33 |
| *** thomasem has quit IRC | 16:33 | |
| *** rcleere has quit IRC | 16:34 | |
| sdague | I'm more generally concerned that oslo isn't running in parallel though, as it means the first time those components get run in parallel is after import into a project, which seems like a bad way to import broken | 16:34 |
| markmc | sdague, only background I can find on not running parallel: https://review.openstack.org/#/c/33419/1/tox.ini | 16:34 |
| markmc | sdague, I've a vague recollection of filing a bug with more details | 16:34 |
| clarkb | sdague: I think that is a valid concern as well | 16:34 |
| sdague | markmc: yeh, I should hound dhellmann about this. This seems like something that should be a pretty high priority to get addressed. just turning off parallel is punting pretty hard :) | 16:35 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: defensive coding for docbook-properties.sh https://review.openstack.org/54642 | 16:36 |
| sdague | and kicking the can down the road to the next poor schmo | 16:36 |
| clarkb | jeblair: fungi: 54642 is an older change that was approved but failed due to a merge conflict. I went ahead and rebased it because I think I can just approve it, any opposition to me doing that? it was a very simple rebase | 16:36 |
| markmc | sdague, https://bugs.launchpad.net/oslo/+bug/1192207 | 16:37 |
| uvirtbot | Launchpad bug 1192207 in oslo "Dependencies between tests in tests.unit.rpc.test_kombu:RpcKombuTestCase" [High,Triaged] | 16:37 |
| markmc | sdague, could be resolved when we get rid of rpc from oslo-incubator | 16:37 |
| clarkb | jeblair: fungi: https://review.openstack.org/#/c/54642/3..4/modules/jenkins/files/slave_scripts/docbook-properties.sh is the diff between previously approved and rebased on that file | 16:37 |
| jeblair | clarkb: go for it | 16:37 |
| markmc | sdague, to be clear, it wasn't "switch of parallel" it was that we never enabled it | 16:38 |
| markmc | sdague, i.e. welcomed the port to nose even though we couldn't enable parallel at the time | 16:39 |
| markmc | from nose, rather | 16:39 |
| sdague | markmc so ... I threw a comment on that bug. That seems completely fixable by making queue and topic names random strings | 16:40 |
| openstackgerrit | A change was merged to openstack-infra/config: defensive coding for docbook-properties.sh https://review.openstack.org/54642 | 16:40 |
| sdague | with no other effort | 16:40 |
| SergeyLukjanov | I'm installing devstack and it fails while uploading cirros images to glance with "500 Internal Server Error: Failed to upload image" (http://paste.openstack.org/show/54084/) | 16:40 |
| sdague | but that's a cursory look at it, and I'm 20 minutes away from jumping in a car and driving to thanksgiving | 16:40 |
| clarkb | oh cool found the devstack-cache.py change danprince made, reviewing that now and will approve assuming it looks good | 16:40 |
| clarkb | that should make nodepool fedorable | 16:41 |
| *** ryanpetrello has quit IRC | 16:42 | |
| clarkb | just approved that change, will kick off some nodepool image builds in a bit | 16:43 |
| openstackgerrit | A change was merged to openstack-infra/config: Add merge-release-tags job to Savanna https://review.openstack.org/57667 | 16:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Update devstack-cache.py to install RPMs. https://review.openstack.org/54777 | 16:44 |
| *** dolphm is now known as dolphm_afk | 16:44 | |
| *** cyeoh has joined #openstack-infra | 16:45 | |
| SergeyLukjanov | jeblair, thanks for aprv merge-release-tags, could you please take a look at https://review.openstack.org/#/c/58616/ too? it's about missed milestone proposed branch section in ACL config for savanna | 16:46 |
| SergeyLukjanov | it adds savanna-milestone group usage | 16:46 |
| zaro | clarkb: opinion? https://review.openstack.org/#/c/46099 | 16:47 |
| clarkb | zaro: so, we slightly document tagging a release on the wiki. My concern is that python developers don't have as many suepr crazy insane notionsabout how a release tagging should work | 16:48 |
| clarkb | java devs think automagic that creates a bunch of commits that add a remove versions to a file then push some resulting thing to a central repository is ok | 16:49 |
| clarkb | I am mostly concerned that the shift for them will be dramatic enough that it needs some explanation | 16:49 |
| clarkb | zaro: maybe, a new wiki page that is linked to under the simpler taggign a release section of the grrit page on the wiki | 16:51 |
| clarkb | that way current documentation is nice and simple and leads folks to the in depth info | 16:51 |
| *** yassine has quit IRC | 16:53 | |
| zaro | yeah ok. so do you do not think that documenting the python release is necessary? | 16:54 |
| clarkb | zaro: I think we can probably descrieb it in terms that apply to python and java (rather than being super specific) | 16:54 |
| *** thomasem has joined #openstack-infra | 16:54 | |
| *** reed has joined #openstack-infra | 16:55 | |
| *** thomasem has left #openstack-infra | 16:55 | |
| * zaro agrees | 16:55 | |
| *** herndon has joined #openstack-infra | 16:55 | |
| *** yaguang has quit IRC | 16:56 | |
| *** jcoufal has joined #openstack-infra | 16:58 | |
| zaro | clarkb: what project are the wiki files in? | 17:03 |
| clarkb | zaro they are on the wiki :) just edit directly | 17:04 |
| zaro | duh! | 17:04 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: new maven plugin jobs to build and deploy maven plugins to maven nexus https://review.openstack.org/46099 | 17:05 |
| *** zoresvit has quit IRC | 17:12 | |
| *** SergeyLukjanov has quit IRC | 17:12 | |
| fungi | clarkb: no objections on rebasing/approving 54642 but i guess you already did, so cool | 17:12 |
| *** esker has quit IRC | 17:12 | |
| *** ruhe has quit IRC | 17:13 | |
| zaro | do you guys we should send gerrit events for stackforge/clouddocs project to this channel? | 17:17 |
| saper | 6 | 17:17 |
| *** markmc has quit IRC | 17:19 | |
| zaro | fung, clarkb : opinion? | 17:21 |
| jeblair | zaro: i would not find them useful | 17:22 |
| zaro | also what integrates gerrit and LP because clouddocs patches are not getting referenced in LP | 17:23 |
| jeblair | zaro: what's the lp project name? | 17:23 |
| zaro | jeblair: ohh yeah. i forgot not project for this in LP. that's why. | 17:23 |
| zaro | the bug is in openstackci project | 17:24 |
| *** reed has quit IRC | 17:24 | |
| jeblair | zaro: yeah, you should just update that bug manually | 17:25 |
| *** zoresvit has joined #openstack-infra | 17:25 | |
| *** bauzas has left #openstack-infra | 17:27 | |
| *** nati_ueno has quit IRC | 17:29 | |
| fungi | zaro: i tend to agree with jeblair about not having them update in this channel... maybe the docs channel would be interested in seeing updated on patches being merged to that though? you could ask them | 17:31 |
| *** fbo is now known as fbo_away | 17:31 | |
| zaro | fungi: good idea. i'll ask. | 17:32 |
| openstackgerrit | Khai Do proposed a change to openstack-dev/pbr: show transitive dependencies https://review.openstack.org/54639 | 17:36 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/gear: geard: wrong reference to setup_logging() https://review.openstack.org/58824 | 17:37 |
| *** ^d has joined #openstack-infra | 17:38 | |
| *** dizquierdo has joined #openstack-infra | 17:40 | |
| zaro | fungi: this is annoying. LP won't pickup new gerrit patches when it's already referenced a patch. take a look at this https://bugs.launchpad.net/openstack-ci/+bug/1172417 | 17:41 |
| uvirtbot | Launchpad bug 1172417 in openstack-ci "on check test for os/requiremests show transitive dependencies" [Low,In progress] | 17:41 |
| zaro | the ones taht are referenced have been abandoned. i uploaded 2 new patches that do not appear in the bug. | 17:43 |
| *** adalbas has quit IRC | 17:43 | |
| *** mfer has quit IRC | 17:44 | |
| *** mfer has joined #openstack-infra | 17:44 | |
| * fungi looks | 17:46 | |
| clarkb | building a new nodepool image for hpcloud az1 now | 17:46 |
| fungi | zaro: new patchsets to existing reviews? | 17:46 |
| *** chandankumar has quit IRC | 17:46 | |
| *** hashar has quit IRC | 17:47 | |
| fungi | zaro: remember, the update_byg.py script avoids adding comments to a bug unless it's mentioned in the commit message for patchset 1 | 17:47 |
| fungi | zaro: that script would need some additional state management or an ability to search the bug comments if we wanted to make it figure out whether it's commented before | 17:47 |
| clarkb | jeblair: fungi: if the az1 image goes through I am going to approve https://review.openstack.org/#/c/54778/ | 17:48 |
| fungi | clarkb: okay | 17:48 |
| *** senk has joined #openstack-infra | 17:48 | |
| fungi | dprince isn't here or i'd ask him whether in-place yum upgrades of a fedora 18 image wouldn't get us fedora 19 anyway (similar to when we were using centos 6.3 images to effectively build centos 6.4 servers) | 17:49 |
| *** jcooley_ has quit IRC | 17:51 | |
| *** jerryz has joined #openstack-infra | 17:51 | |
| *** dizquierdo has quit IRC | 17:52 | |
| *** derekh has quit IRC | 17:53 | |
| zaro | fungi: only if it's mentioned in ps 1? what if the first time mentioned was somewhere > ps 1? | 17:53 |
| fungi | zaro: then it won't automatically add a comment on the bug until it merges | 17:54 |
| zaro | fungi: ok. that's better than not at all. | 17:54 |
| fungi | right now that script maintains no state, and merely guesses based on the patchset number whether it should mention itself on the bug or not | 17:55 |
| *** sandywalsh has joined #openstack-infra | 17:55 | |
| *** adalbas has joined #openstack-infra | 17:56 | |
| *** dolphm_afk is now known as dolphm | 17:56 | |
| bnemec | dhellmann: sdague clarkb devananda: So the reason this got missed in Oslo is that it was only locking for postgres runs, which we don't do in Oslo. | 17:56 |
| bnemec | Which is a bad thing for it to be doing anyway, obviously. | 17:56 |
| bnemec | Also, I poked at the parallel testing thing the other day and ran into other issues besides the kombu tests. | 17:56 |
| bnemec | Haven't had a chance to look into it further. | 17:57 |
| *** jerryz_ has joined #openstack-infra | 17:57 | |
| devananda | bnemec: so, your fix, https://review.openstack.org/#/c/58830/, seems to be what clarkb would like to avoid. that is, there's no need for a global lock at all around db tests | 17:58 |
| *** nati_ueno has joined #openstack-infra | 17:58 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Add new doctools project https://review.openstack.org/58623 | 17:58 |
| bnemec | devananda: Yeah, that was a "better than what we had" change, not a final fix. | 17:58 |
| devananda | bnemec: ah, ok | 17:58 |
| bnemec | I'm very interested in the separate schemas because I'm pretty sure this locking is rather error prone right now. | 17:59 |
| *** jerryz has quit IRC | 18:00 | |
| *** SergeyLukjanov has joined #openstack-infra | 18:00 | |
| bnemec | (for example, if someone writes a test and forgets to add the lock decorator, or adds the wrong one, we might delete the db out from under them) | 18:00 |
| *** SergeyLukjanov has quit IRC | 18:01 | |
| *** SergeyLukjanov has joined #openstack-infra | 18:02 | |
| *** morganfainberg is now known as needscoffee | 18:02 | |
| *** rnirmal has joined #openstack-infra | 18:03 | |
| clarkb | devstack-cache.py failed, fix incoming | 18:05 |
| *** needscoffee is now known as morganfainberg | 18:06 | |
| *** senk has quit IRC | 18:09 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Handle lack of rpms or debs in devstack-cache. https://review.openstack.org/58832 | 18:09 |
| *** SergeyLukjanov_ has joined #openstack-infra | 18:09 | |
| clarkb | jeblair: fungi ^ quick review on that would be good (to keep image builds working, as an alternative we can revert the devstack-cache.py change) | 18:10 |
| fungi | k | 18:10 |
| clarkb | let me paste the traceback in the gerrit coments too | 18:10 |
| *** SergeyLukjanov has quit IRC | 18:12 | |
| clarkb | fungi: thinking about your comment, that would be better, but do I want to fix that as part of this change? | 18:13 |
| fungi | up to you. i can +2 and add a follow-on patch with my suggestion | 18:14 |
| jeblair | clarkb: i like fungi's suggestion. your call as to whether you want to fix now. consider you have my +2 either way. | 18:14 |
| devananda | clarkb: do you know if any othe rprojects use a periodic jenkisn job to keep their etc/$project/$project.conf.sample up to date? | 18:14 |
| clarkb | ok let me just do it properly as fungi suggests | 18:14 |
| clarkb | devananda: I don't | 18:14 |
| jeblair | clarkb: 1 sec | 18:14 |
| jeblair | clarkb: actually... i think we should gracefully handle not having either | 18:14 |
| *** nicedice has joined #openstack-infra | 18:14 | |
| fungi | oh! interesting. what's the use case? | 18:15 |
| jeblair | the use case is not needing any additional packages... | 18:15 |
| fungi | base image with no added packages at all maybe? | 18:15 |
| jeblair | that seems unlikely, but possible. | 18:15 |
| jeblair | hrm | 18:15 |
| fungi | right | 18:15 |
| jeblair | okay, i retract that, and agree with fungi's suggestion.... | 18:15 |
| jeblair | because it's so unlikely at this point it's probably an error | 18:16 |
| clarkb | jeblair: ok | 18:16 |
| jeblair | and, hey, if we ever hit it, it'll be easy to fix. :) | 18:16 |
| fungi | maybe a safer way to address the bare install use case would be to have an optional "none" key for those rarities | 18:16 |
| jeblair | fungi: right, but now we're getting into devstack territory; complicated. | 18:17 |
| fungi | to be able to say "yes, i really don't want to add anything, it's not a mistake" | 18:17 |
| jeblair | fungi: these items are filled by examining devstack. | 18:17 |
| fungi | ahh, very true | 18:17 |
| fungi | i keep forgetting that the script is extremely devstack-implementation-detail-specific anyway | 18:18 |
| jeblair | fungi: so i think we're far more likely to hit the proposed exception line because something in this complex interaction broke, than we are because suddenly devstack doesn't actually require additional packages. | 18:18 |
| fungi | completely agree there, and it's the former which worries me | 18:19 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Handle lack of rpms or debs in devstack-cache. https://review.openstack.org/58832 | 18:19 |
| clarkb | how does that look? | 18:19 |
| fungi | wfm | 18:20 |
| *** osanchez has quit IRC | 18:23 | |
| jeblair | clarkb: aprvd | 18:24 |
| clarkb | ty | 18:24 |
| fungi | clarkb: the only reason the original patchset worried me at all was that it was taking us from a "everything breaks when no debs are specified" to "silently succeed when no debs are specified" with no fallback in case that was actually a mistake | 18:25 |
| clarkb | ya | 18:26 |
| *** branen has quit IRC | 18:27 | |
| openstackgerrit | A change was merged to openstack-infra/config: Handle lack of rpms or debs in devstack-cache. https://review.openstack.org/58832 | 18:27 |
| *** markmcclain has joined #openstack-infra | 18:27 | |
| *** markmcclain has quit IRC | 18:29 | |
| mfer | folks, I'm starting to poke around http://git.openstack.org/cgit/stackforge/golang-client/. I was wondering if there was a way I could automate the testing | 18:30 |
| mfer | maybe even against more than one version of Go | 18:30 |
| mfer | anyone have any pointers where I could get started? | 18:30 |
| *** jpich has quit IRC | 18:30 | |
| clarkb | mfer: the openstack-infra/config project has modules/jenkins/manifests/params.pp and slave.pp. In there you can list the packages you need and get them to be installed | 18:31 |
| clarkb | our test slaves are precise and centos currently which may not give you up to date Go out of the box. I think go gcc is 1.0 on precise maybe | 18:31 |
| mfer | can you install from an outside PPA? | 18:32 |
| *** melwitt has joined #openstack-infra | 18:32 | |
| clarkb | but basically you would use those puppet manifests to install Go, then you can add jobs in modules/openstack_project/files/jenkins_job_builder/config and have zuul run them with modules/openstack_project/files/zuul/layout.yaml | 18:32 |
| fungi | right. for anything which is not provided officially in ubuntu 12.04 (or in ubuntu cloud archive for precise once we get it added), you'd need to provide your own local install which ran under the same unprivileged user as the tests themselves | 18:32 |
| *** ryanpetrello has joined #openstack-infra | 18:33 | |
| mfer | so, i'd need to create a jenkins job that installed go as the unpriviledged user and then run the test suite | 18:34 |
| clarkb | there is a golang package too, not sure what version of go that provides on precise | 18:34 |
| mfer | i'd want to test against 1.1 for now and 1.2 in a few weeks when it comes out | 18:34 |
| *** ^d has quit IRC | 18:34 | |
| mfer | i might want to have it run against 2 different versions | 18:35 |
| fungi | we have made a couple of ppa exceptions, but those are under some very specific circumstances... they're backports of packages available in trusty (14.04 lts), the ppas are maintained by ubuntu devs, and their use is purely to support officially-sanctioned testing initiatives (py3k and pypy conformance specifically) | 18:35 |
| * clarkb installs golang on precise to see what that gets him | 18:35 | |
| fungi | http://packages.ubuntu.com/search?keywords=golang | 18:35 |
| clarkb | right what is 2:1-5? | 18:36 |
| fungi | that seems to me to be pre-1.0.x | 18:36 |
| *** dolphm_ has joined #openstack-infra | 18:37 | |
| clarkb | go version go1 | 18:37 |
| mfer | https://wiki.ubuntu.com/Go | 18:38 |
| sdague | wait, are we really putting go in the gate? does that mean we have to stop with the go jokes? :) | 18:38 |
| fungi | trusty has 1.1.2 which seems to be flowing in from debian, so if the ubuntu golang maintainers import 1.2 after release and before the freeze for trusty, we'll likely be able to provide access to it on our slaves around the middle of next year | 18:39 |
| jog0 | sdague: ping | 18:39 |
| *** reed has joined #openstack-infra | 18:39 | |
| *** dolphm has quit IRC | 18:39 | |
| jog0 | got a hard bug fingerprint, wanted to get a second pair of eyes on it | 18:39 |
| jog0 | http://logstash.openstack.org/#eyJzZWFyY2giOiJcImxpYnZpcnRFcnJvclxcOiBVbmFibGUgdG8gcmVhZCBmcm9tIG1vbml0b3I6IENvbm5lY3Rpb24gcmVzZXQgYnkgcGVlclwiIEFORCBcIltyZXEtXCIgQU5EIE5PVCBcIiBsaW5lIDY5NiwgaW4gbWFuYWdlZFNhdmVcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiYWxsIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM4NTU3NzM4NDMzNH0= | 18:39 |
| mfer | so, ideally this would only be a gate for golang-client and nothing else | 18:39 |
| clarkb | mfer: http://blog.labix.org/2013/06/15/in-flight-deb-packages-of-go bothers me | 18:40 |
| sdague | jog0: ok, realize I'm tethered on 1xRTT right now (in a car) | 18:40 |
| sdague | so ... slowness | 18:40 |
| mfer | and the install should really only happen for golang-client if nothing else needs it. is that possible? | 18:40 |
| jog0 | sdague: never mind | 18:40 |
| jog0 | clarkb: ^ looking for a second pair of eyes on an elasticsearch query | 18:40 |
| *** mrmartin has joined #openstack-infra | 18:40 | |
| clarkb | mfer: the gophers PPA is apparently deprecated in favor of a much less automagic friendly process | 18:40 |
| mfer | well, isn't that fun | 18:41 |
| sdague | jog0: if you have time, there is also an image race that's showing up a lot, that's not yet fingerprinted | 18:41 |
| clarkb | that said writing a puppet module to deal with the less automagic friendly process may be the way to go | 18:41 |
| jog0 | will keep my eyes out | 18:41 |
| sdague | I think that's the only one I regularlly see pop in the tempest channel as unknown bug | 18:41 |
| mfer | clarkb can it only be installed when the golang-client needs it and not for everyone else? | 18:41 |
| jog0 | image race as in which project | 18:41 |
| jog0 | I found a libvirt issue | 18:41 |
| jog0 | libvirt is stacktracing :/ | 18:41 |
| clarkb | mfer: you could encode that into your build stuff yes | 18:42 |
| sdague | it's in setupClass of a compute test on ImageFilters | 18:42 |
| fungi | jog0: this is with 0.9.8 right? | 18:42 |
| sdague | setupClass is building 3 images | 18:42 |
| clarkb | mfer: have a little wrapper that grabs godeb, installs go, if go is not already present in the workspace | 18:42 |
| * jog0 checks | 18:42 | |
| sdague | and it seems like sometimes they don't complete | 18:42 |
| clarkb | mfer: that should all work as an unpriviledged test user, it will make the initial test run longer but once go is available shouldn't be much of a time sink | 18:42 |
| jog0 | fungi: yes 0.9.8 | 18:42 |
| jog0 | sdague: will keep my eye out for that one | 18:43 |
| clarkb | though godeb may want to do a global install | 18:43 |
| mfer | clarkb thanks. i'll investigate this further. this is my first time doing any of this in the openstack infrastructure. | 18:43 |
| sdague | jog0: you should take that one to the ML and get danpb looped in. It's helpful to have the libvirt maintainer on the project :) | 18:43 |
| clarkb | mfer: cool, let me know how that goes, I am curious now | 18:44 |
| fungi | jog0: perhaps that will increase developer interest in bug 1228977 in that case (i haven't had time to grab more info other than simply to confirm that 1.1.1 is probably still broken, but perhaps in different ways than 1.0.6 was) | 18:44 |
| uvirtbot | Launchpad bug 1228977 in nova "n-cpu seems to crash when running with libvirt 1.0.6 from ubuntu cloud archive" [High,Confirmed] https://launchpad.net/bugs/1228977 | 18:44 |
| clarkb | mfer: think of it this way, if there is a non crazy way to build golang in your homedir as an unprivileged user, jenkins should be able to deal with that | 18:44 |
| jog0 | fungi: good idea, | 18:44 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add a zuul client https://review.openstack.org/58841 | 18:45 |
| jeblair | that was fun! ^ | 18:45 |
| fungi | jeblair: !!! | 18:45 |
| jeblair | "zuul enqueue --trigger gerrit --pipeline gate --project gtest-org/test --change 2372 --patchset 1" | 18:46 |
| clarkb | jeblair: the discussion mfer started may interest you (how to get newish Go on our test slaves) | 18:46 |
| jog0 | btw http://status.openstack.org/elastic-recheck/ | 18:46 |
| jog0 | something is wrong with that page | 18:46 |
| clarkb | did the json jobs stop working? | 18:47 |
| clarkb | jog0: any new changes to e-r? | 18:47 |
| fungi | jeblair: would this be a viable replacement for the trigger-job script? | 18:47 |
| clarkb | I am going to start a new hpcloud az1 image build | 18:47 |
| jeblair | jog0: you might try running the json output from e-r locally | 18:47 |
| jeblair | fungi: trigger-job script? | 18:47 |
| *** ryanpetrello has quit IRC | 18:48 | |
| jeblair | fungi, clarkb, jog0: "zuul enqueue" is to replace our use of "reverify no bug" (also, 'recheck no bug') when we go mucking with the zuul queue. is pre-req for disabling 'reverify no bug'. | 18:48 |
| fungi | jeblair: i get that it gives us a backdoor other than recheck/reverify comments for infra requeuing of changes, but wondering if we can also work zuul/tools/trigger-job.py into the client mostly as-is? | 18:48 |
| clarkb | jeblair: it is a prereq so that infra can redo those jobs without other core approval? | 18:49 |
| jeblair | fungi: ah, yes, is intended for that too, but needs a little expansion to handle extra fields that post jobs have. | 18:49 |
| fungi | keen | 18:49 |
| jeblair | clarkb: yes, or using admin powers. | 18:49 |
| mfer | clarkb pm? | 18:50 |
| clarkb | mfer: sure. though feel free to keep discussing here (jeblair and fungi and others are full of good ideas too) | 18:50 |
| *** ^d has joined #openstack-infra | 18:50 | |
| *** ^d has joined #openstack-infra | 18:50 | |
| jog0 | clarkb: no new changes that I know of | 18:51 |
| clarkb | jog0: I would try what jeblair suggests, run the json output job locally | 18:51 |
| jog0 | running '/graph.py queries.yaml' | 18:51 |
| *** ryanpetrello has joined #openstack-infra | 18:51 | |
| *** boris-42 has joined #openstack-infra | 18:54 | |
| *** branen has joined #openstack-infra | 18:57 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add a zuul client https://review.openstack.org/58841 | 18:59 |
| dkranz | clarkb: Is there an example of doing queries against logstash from python rather than the web ui? | 18:59 |
| clarkb | dkranz: in elastic-recheck | 18:59 |
| clarkb | dkranz: but not as documentation. pyelasticsearch is the lib elastic-recheck is using | 19:00 |
| dkranz | clarkb: ok, great. | 19:00 |
| clarkb | which does have documentation. There is also an elasticsearch client. Which we should look at at some point probably | 19:00 |
| dkranz | clarkb: Yeah, it is pretty useful stuff | 19:00 |
| jeblair | jog0: able to reproduce the e-r page bug, or do i need to look for a failure on the server? | 19:01 |
| dkranz | clarkb: I was going to use it to report on the frequency of the log error bugs in the whitelist | 19:01 |
| clarkb | dkranz: you can probably overload e-r to do that, maybe have a class of "bug" that doesn't report as a reason for failure, but can still be tracked | 19:01 |
| clarkb | would require updating e-r but I think that use case is a good one | 19:02 |
| *** talluri has quit IRC | 19:02 | |
| *** talluri has joined #openstack-infra | 19:02 | |
| dkranz | clarkb: That's possible. I'll take a look. | 19:03 |
| mfer | it appears godeb will ask for a password if now executed under sudo. so, that's out for a newer version of go | 19:03 |
| clarkb | mfer: if we bootstrap godeb with puppet it will run as root. The puppet to do that isn't terrible, but can be clunky as you have to wget the file, check sha1/md5, then run the commands to do an actual install | 19:04 |
| clarkb | waiting on image builds is slow going. /me looks at zuul client | 19:05 |
| *** mihgen has joined #openstack-infra | 19:05 | |
| jeblair | clarkb: i don't expect we would do that for a non-openstack project | 19:05 |
| clarkb | jeblair: the build via puppet dance? | 19:06 |
| jeblair | clarkb: yep. | 19:06 |
| mfer | jeblair it's a stackforge project | 19:07 |
| mfer | an existing one. is that going to be fairly different | 19:07 |
| mfer | ?? | 19:07 |
| jeblair | the build environment is primarily for openstack projects; if stackforge projects find it useful, they're welcome to use it, but that's not the main objective | 19:07 |
| mfer | is there a way to do something different for stackforge projects? | 19:08 |
| *** luisg has joined #openstack-infra | 19:08 | |
| jeblair | mfer: different than what? | 19:08 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Add new openstack-doc-tools project https://review.openstack.org/58623 | 19:08 |
| mfer | well, this is a language bindings. it would be idea to test it against the api. | 19:08 |
| mfer | and it's in a language other than python to make the openstack api accessible in other languages | 19:09 |
| mfer | how can we test that well? | 19:09 |
| *** ruhe has joined #openstack-infra | 19:10 | |
| jeblair | mfer: the best bet would be to become an openstack project. | 19:11 |
| jeblair | mfer: some background: | 19:11 |
| jeblair | mfer: we have limited resources (especially people) and very complex systems, so we focus primarily on openstack projects | 19:12 |
| jeblair | mfer: in fact, that's what the TC asked us to do when they 'blessed' stackforge | 19:12 |
| jeblair | mfer: so we focus on configuring the test systems to be able to test where openstack is focused, we aren't able to accomodate test scenarios for stackforge projects that require significant work outside of that environment. | 19:13 |
| pleia2 | jeblair: to make sure I tested the right thing, your comment re: https://review.openstack.org/#/c/54386/2 is that you don't think we need that install headers line at all in devstack-vm-gate-wrap.sh anymore | 19:14 |
| mfer | jeblair so, for example, we should try to get the golang-client to become an openstack project so we can update the testing environment for that. if it's going to remain on stackforge it may need outside testing. do I have that right? | 19:14 |
| *** yolanda has joined #openstack-infra | 19:15 | |
| jeblair | mfer: that's about the gist of it, at the moment at least. unless you can get everything you need installed as an unprivileged user (i don't know how hard that is but could imagine it's not easy), or we change something about our test infrastructure (there are some plans in this area that may help but not immediately) | 19:15 |
| jog0 | jeblair: e-r graph seems to work | 19:16 |
| jeblair | mfer: however, i do think it's worth exploring getting official recognition for your project... | 19:16 |
| mfer | jeblair ok, i've got it | 19:16 |
| *** fbo_away is now known as fbo | 19:16 | |
| jeblair | mfer: as a user, i personally think that good client libs are something worth supporting. | 19:16 |
| mfer | jeblair i agree entirely. i'd just like to have a better way to test them. | 19:17 |
| clarkb | jeblair: fungi: woot image build is working now. I will approve dprince's nodepool fedora job after lunch when I have time to babysit that | 19:17 |
| mfer | i'll keep poking around and see what I can come up with | 19:17 |
| jeblair | mfer: i'm not sure how familiar you are with our community processes, governance, etc... | 19:18 |
| jeblair | mfer: but if you'd like to learn more about becoming an official program, you might start here: https://wiki.openstack.org/wiki/Programs | 19:19 |
| jeblair | mfer: or just start a thread on the openstack-dev mailing list about how the project can support client library efforts | 19:19 |
| mfer | jeblair before I do that I think I'd like to see the codebase get into a relatively usable shape and have more than me looking at it. I just got the keys from the original developer who hasn't touched it in months | 19:21 |
| mfer | i'll be thinking of it but i'd like to have the project be worthy of that and if something happened to me it wouldn't stop | 19:21 |
| fungi | if it's the project i'm thinking of, it never got touched after the initial import into our code review system | 19:21 |
| *** jcooley_ has joined #openstack-infra | 19:22 | |
| mfer | yeah, pretty much | 19:22 |
| *** pblaho has joined #openstack-infra | 19:22 | |
| mfer | i spoke with the dev. it seems they pivoted away from openstack so the work stopped | 19:22 |
| jog0 | jeblair: so server logs may be in order | 19:24 |
| jeblair | mfer: i should add one more thing... | 19:26 |
| jeblair | mfer: if you want to run tests against a running openstack (as set up by devstack) -- those happen on single-use slaves | 19:26 |
| jeblair | mfer: and the test process starts with full root access. so it actually should be pretty easy to do whatever you want in that case | 19:27 |
| jeblair | mfer: but i don't think we want to run unit tests on servers like that yet; so please don't abuse those | 19:27 |
| jeblair | (that's one of the projects that may make this easier in the long run, but we're not there yet) | 19:28 |
| mfer | ok | 19:28 |
| mfer | is there a plan to get there so I can abuse them? | 19:28 |
| clarkb | I am going to AFK here for a bit, running low on laptop battery and need to move from suburbs to city also lunch | 19:29 |
| *** wenlock has quit IRC | 19:30 | |
| jeblair | mfer: soon, i hope. | 19:30 |
| *** rongze has quit IRC | 19:30 | |
| *** rongze has joined #openstack-infra | 19:31 | |
| fungi | in about 45 minutes i have to disappear for an appointment and then to grab dinner, but hope to be back to working on things by 2300 utc | 19:32 |
| *** nati_uen_ has joined #openstack-infra | 19:32 | |
| jeblair | jog0: http://status.openstack.org/elastic-recheck/graph.json looks sane | 19:32 |
| jeblair | jog0: js console log makes it look like a flot related error | 19:33 |
| *** CaptTofu has quit IRC | 19:34 | |
| jog0 | jeblair: ahh so graph.py needs an update to make things ints? | 19:34 |
| *** CaptTofu has joined #openstack-infra | 19:34 | |
| *** ryanpetrello has quit IRC | 19:35 | |
| jeblair | jog0: flot is the graphing library | 19:35 |
| *** nati_ueno has quit IRC | 19:35 | |
| *** wenlock has joined #openstack-infra | 19:35 | |
| *** rongze has quit IRC | 19:36 | |
| devananda | has jenkins stopped submitting changes to global-requiremetns to individual projects? | 19:37 |
| clarkb | yes, it didn't work so we stopped it | 19:40 |
| lifeless | https://review.openstack.org/#/c/57892/ is ready for a second +2 | 19:40 |
| *** pblaho has quit IRC | 19:44 | |
| zul | how can you find out a specific dependency version installed by tox | 19:46 |
| lifeless | pip freeze in the vm | 19:47 |
| lifeless | sorry, virtualenv | 19:47 |
| jog0 | jeblair: oh | 19:47 |
| jeblair | jog0: bug 1225664 has no data in the json | 19:47 |
| uvirtbot | Launchpad bug 1225664 in devstack "Swift: No space left on device" [Undecided,Fix committed] https://launchpad.net/bugs/1225664 | 19:47 |
| *** CaptTofu has quit IRC | 19:47 | |
| *** CaptTofu has joined #openstack-infra | 19:48 | |
| jog0 | jeblair: in the past that ment the graph was just empty | 19:48 |
| jog0 | meant* | 19:48 |
| jog0 | at least thats what I thought | 19:48 |
| *** jcooley_ has quit IRC | 19:49 | |
| jeblair | jog0: yep. i think flot updated. it's CD. | 19:51 |
| jeblair | jog0: i have an old version locally, and it produces an empty graph. | 19:51 |
| fungi | jgriffith: not sure whether you're around, but are you okay with https://review.openstack.org/57953 (gating on cinderclient unit tests under pypy)? | 19:51 |
| jog0 | jeblair: ahh | 19:52 |
| jeblair | jog0: i'll fix it | 19:53 |
| jog0 | jeblair: thanks | 19:53 |
| *** sgordon has left #openstack-infra | 19:54 | |
| *** jerryz_ has quit IRC | 19:57 | |
| *** jcooley_ has joined #openstack-infra | 19:58 | |
| *** rongze has joined #openstack-infra | 20:01 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Don't draw an e-r graph if no data https://review.openstack.org/58850 | 20:01 |
| jeblair | jog0: ^ | 20:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Don't draw an e-r graph if no data https://review.openstack.org/58850 | 20:02 |
| mfer | clarkb jeblair thanks for the help and pointers today. | 20:02 |
| * mfer goes to relax now | 20:02 | |
| jeblair | mfer: you're welcome. sorry it isn't easy. have fun! | 20:03 |
| *** mfer has quit IRC | 20:03 | |
| jeblair | what's a one-word imperative for "move something to the head of the queue"? | 20:04 |
| lifeless | Promote? | 20:05 |
| lifeless | Advance? | 20:06 |
| *** vipul is now known as vipul-away | 20:06 | |
| *** vipul-away is now known as vipul | 20:06 | |
| reed | I hear people yell "coming through" when they need to get off the bus | 20:06 |
| *** rongze has quit IRC | 20:06 | |
| *** jcooley_ has quit IRC | 20:07 | |
| reed | bump | 20:07 |
| *** mihgen has quit IRC | 20:08 | |
| *** ruhe has quit IRC | 20:10 | |
| *** nati_uen_ has quit IRC | 20:14 | |
| lifeless | I like bump | 20:15 |
| jeblair | jump | 20:15 |
| pleia2 | I'd say jump | 20:16 |
| lifeless | oh man, ear worm | 20:17 |
| lifeless | jump, jump, jump, jump | 20:17 |
| *** reed has quit IRC | 20:17 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-dev/pbr: Make ChangeLog more condensed https://review.openstack.org/52367 | 20:19 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-dev/pbr: Let git do mailmap mapping for us https://review.openstack.org/54050 | 20:19 |
| fungi | lifeless: ^ tried addressing your suggestions on mordred's 52367 change there | 20:19 |
| * fungi disappears for the next few hours | 20:19 | |
| *** nati_ueno has joined #openstack-infra | 20:20 | |
| *** vipul is now known as vipul-away | 20:21 | |
| *** jcooley_ has joined #openstack-infra | 20:23 | |
| *** mihgen has joined #openstack-infra | 20:25 | |
| *** adalbas has quit IRC | 20:28 | |
| lifeless | fungi: thanks; reviewed. | 20:29 |
| *** rongze has joined #openstack-infra | 20:31 | |
| *** fbo is now known as fbo_away | 20:31 | |
| *** mrmartin has quit IRC | 20:31 | |
| jeremyb | oh, is this where i knew pleia2's nick from? :) | 20:33 |
| pleia2 | jeremyb :) | 20:33 |
| ^d | jeremyb: We need to stop hanging out in the same channels ;-) | 20:34 |
| *** vipul-away is now known as vipul | 20:35 | |
| jeremyb | hah, but i knew chad was here | 20:36 |
| jeremyb | ^d: was poking in #ubuntu and then later #ubuntu-website yesterday and pleia2 is the one that answered my question. i knew the nick was familiar but didn't know why | 20:37 |
| fungi | ha, the optometrist office has an unsecured access somewhere nearby. hacking continues | 20:38 |
| *** rongze has quit IRC | 20:40 | |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Reformat nova.json https://review.openstack.org/58805 | 20:41 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Add mriedem and dan-prince to nova-core https://review.openstack.org/58806 | 20:41 |
| jgriffith | fungi: delayed response but yes :) | 20:41 |
| jgriffith | fungi: added my +1 | 20:41 |
| fungi | jgriffith: awesome. approving in that case. thanks!!! | 20:41 |
| jgriffith | fungi: welcome | 20:41 |
| *** hashar has joined #openstack-infra | 20:43 | |
| *** weshay has quit IRC | 20:44 | |
| fungi | jeblair: also, my vote is for "jump the queue" since it's an established phrase with very similar implications | 20:44 |
| *** yolanda has quit IRC | 20:44 | |
| openstackgerrit | A change was merged to openstack-infra/config: Run the cinderclient tests under PyPy in gate and check https://review.openstack.org/57953 | 20:45 |
| fungi | Alex_Gaynor: ^ | 20:45 |
| Alex_Gaynor | fungi: thakns | 20:45 |
| *** dolphm_ has quit IRC | 20:50 | |
| *** lcestari has quit IRC | 20:51 | |
| *** talluri has quit IRC | 20:52 | |
| fungi | wow, that was a quick appointment. going to grab a bite and then back for more asap | 20:57 |
| *** rwsu has quit IRC | 20:57 | |
| jeblair | fungi: last appt before the optometrist goes home for the holiday? :) | 20:58 |
| *** senk has joined #openstack-infra | 21:03 | |
| jesusaurus | clarkb: did you ever resolve your elasticsearch garbage-collection woes? | 21:03 |
| *** denis_makogon_ has joined #openstack-infra | 21:03 | |
| *** senk has quit IRC | 21:04 | |
| clarkb | ya I made the object cache GCable | 21:04 |
| clarkb | look in my json files one of them sets the flag | 21:05 |
| jesusaurus | cool | 21:05 |
| *** markwash has quit IRC | 21:06 | |
| *** sarob has joined #openstack-infra | 21:08 | |
| *** fbo_away is now known as fbo | 21:09 | |
| Alex_Gaynor | Is pip broken or something? http://logs.openstack.org/66/58166/3/check/check-grenade-devstack-vm/624274d/console.html | 21:13 |
| *** wenlock_ has joined #openstack-infra | 21:15 | |
| jesusaurus | clarkb: you may be interested in https://github.com/elasticsearch/elasticsearch/issues/2838#issuecomment-15741816 | 21:16 |
| jesusaurus | are you sure you solved your problem? | 21:16 |
| clarkb | arg I guess not | 21:18 |
| clarkb | I havent had memory trouble since upgrading though | 21:18 |
| *** wenlock_ has quit IRC | 21:20 | |
| jesusaurus | I guess I should make upgrading more of a priority | 21:21 |
| *** senk has joined #openstack-infra | 21:25 | |
| clarkb | I am on 0.90.3 | 21:25 |
| *** dolphm has joined #openstack-infra | 21:27 | |
| jesusaurus | theyve released a 1.0.0-beta, so I was hoping to hold out for a 1.0.1 | 21:28 |
| jesusaurus | but if I can online-upgrade to 0.90.5 I might try to do that | 21:28 |
| clarkb | Alex_Gaynor will look as soon as I figure out wifi here | 21:29 |
| Alex_Gaynor | clarkb: hehe, thanks | 21:29 |
| *** ^d is now known as ^demon|meeting | 21:30 | |
| *** markwash has joined #openstack-infra | 21:30 | |
| *** rongze has joined #openstack-infra | 21:34 | |
| *** Loquacity has joined #openstack-infra | 21:36 | |
| *** senk has quit IRC | 21:36 | |
| *** mihgen has quit IRC | 21:36 | |
| *** jhesketh has joined #openstack-infra | 21:37 | |
| *** mrmartin has joined #openstack-infra | 21:38 | |
| *** rongze has quit IRC | 21:39 | |
| * clarkb is still waiting for the text with the confirmation code... go PSU | 21:40 | |
| *** mrmartin has quit IRC | 21:42 | |
| *** mattoliverau has joined #openstack-infra | 21:45 | |
| clarkb | woo I think I have teh internets now | 21:50 |
| *** jcoufal has quit IRC | 21:52 | |
| clarkb | approving the fedora nodepool change now then looking at Alex_Gaynor's test failure | 21:52 |
| *** talluri has joined #openstack-infra | 21:53 | |
| clarkb | Alex_Gaynor: pbr require pip>=1.4, I think it found pip 1.0 which is the system installable version on precise | 21:54 |
| Alex_Gaynor | clarkb: so, what's up with the failing test, I didn't touch any of that :-) | 21:54 |
| clarkb | I have no idea, maybe grenade isn't reliably upgrading pip for us? | 21:54 |
| clarkb | mordred: if you are around ^ would be somethign you should look at | 21:54 |
| *** nati_ueno has quit IRC | 21:55 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add Fedora 19 to the nodepool (devstack-f19-check) https://review.openstack.org/54778 | 21:55 |
| *** nati_ueno has joined #openstack-infra | 21:55 | |
| *** thomasem has joined #openstack-infra | 21:56 | |
| *** talluri has quit IRC | 21:57 | |
| jgriffith | mikal: updated https://bugs.launchpad.net/cinder/+bug/1254932 | 21:57 |
| uvirtbot | Launchpad bug 1254932 in cinder "Grenade failures due to failing to attach a volume" [High,Confirmed] | 21:57 |
| jgriffith | mikal: sadly I'm not sure what the root is yet, may need help from somebody more familiar w/ oslo's rpc and lock code | 21:58 |
| *** SergeyLukjanov_ has quit IRC | 21:58 | |
| *** julim has quit IRC | 21:59 | |
| *** pcm__ has quit IRC | 21:59 | |
| clarkb | I went ahaed and approved https://review.openstack.org/#/c/58360/ it got a lot of testing according to the comments and looks simple enough | 22:02 |
| jeblair | clarkb: do you have a 'test that a certain exception is raised' pattern handy? | 22:02 |
| *** slong has joined #openstack-infra | 22:02 | |
| clarkb | jeblair: for python? | 22:02 |
| clarkb | testtools has an assert that checks a function results in a particular exception being thrown | 22:03 |
| jeblair | clarkb: that's what i'm after | 22:03 |
| *** talluri has joined #openstack-infra | 22:03 | |
| clarkb | I will grab a link to the docs | 22:03 |
| jeblair | clarkb: istr discussion about the right way to do this recently | 22:03 |
| clarkb | jeblair: http://testtools.readthedocs.org/en/latest/for-test-authors.html#improved-assertraises assertRaises and expectedException | 22:04 |
| jeblair | clarkb: thx | 22:04 |
| openstackgerrit | A change was merged to openstack-infra/config: create a jobwrapper span to address long job names spillover https://review.openstack.org/58360 | 22:04 |
| clarkb | pleia2: re https://review.openstack.org/#/c/58653/ did you test just the commands or the entire python module? (I think testing just the commands is reasonable, btu curious to know what was done) | 22:05 |
| *** jcooley_ has quit IRC | 22:06 | |
| clarkb | zaro: did you see my comment on https://review.openstack.org/#/c/54639/ ? | 22:07 |
| *** talluri has quit IRC | 22:08 | |
| pleia2 | clarkb: I created a local.pp with the vcsrepos and exec commands for both | 22:09 |
| clarkb | pleia2: woot thanks | 22:10 |
| clarkb | I am going to remove the WIP tag | 22:10 |
| clarkb | someone with better bash foo than me. https://review.openstack.org/#/c/55827/4/devstack-vm-gate.sh does putting the -d and -f in those variables do the right thing in test? | 22:11 |
| clarkb | local testing for me seems to indicate we will just get true returned, but then how did the gate pass? | 22:13 |
| lifeless | clarkb: should | 22:16 |
| *** boris-42 has quit IRC | 22:16 | |
| lifeless | clarkb: | 22:17 |
| lifeless | TE="-f /etc/hostname" | 22:17 |
| lifeless | $ [ $TE ] && echo yes | 22:17 |
| lifeless | yes | 22:17 |
| lifeless | [ ! $TE ] && echo yes | 22:17 |
| lifeless | $ | 22:17 |
| *** thomasem has quit IRC | 22:18 | |
| clarkb | ya, I got it to work, apparently bash doesn't like it when give the var a value on the same line as test | 22:18 |
| *** mrda has joined #openstack-infra | 22:18 | |
| clarkb | is that because I was using a built in? | 22:23 |
| *** alexpilotti has quit IRC | 22:24 | |
| *** alexpilotti_ has joined #openstack-infra | 22:24 | |
| clarkb | no that doesn't seem to be it, I give up. More things to review | 22:24 |
| fungi | jeblair: yes, i think she got the last eye exam before the optometrist headed out for a chilly five-day golf weekend or something | 22:25 |
| *** jhesketh__ has joined #openstack-infra | 22:25 | |
| *** mattoliverau has quit IRC | 22:26 | |
| *** hashar has quit IRC | 22:30 | |
| *** michchap has quit IRC | 22:30 | |
| *** dolphm has quit IRC | 22:31 | |
| fungi | clarkb: cool on 58360. i know it's annoyed at least a few people, including me, but was cosmetic so i had a hard time prioritizing the review ahead of my stale backlog | 22:31 |
| clarkb | I have approved https://review.openstack.org/#/c/55844/, it won't take effect until nodepool restarts which I probably won't get to today (iirc jeblair has said approving nodepool changes in this manner is ok) | 22:31 |
| clarkb | fungi: yeah, I just got irritated enough to go review the fix | 22:31 |
| clarkb | looks so much better now | 22:32 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Add ability to filter on flavor name https://review.openstack.org/55844 | 22:32 |
| clarkb | though uhm, nodepool seems unhappy currently? maybe related to the f19 change? /me looks | 22:33 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Fix old pre-nodepoold reference in README https://review.openstack.org/58355 | 22:33 |
| *** michchap has joined #openstack-infra | 22:33 | |
| clarkb | fungi: when you are back from dinner a second set of eyes may be handy | 22:33 |
| mgagne | clarkb: remember puppet-lint and literal $ symbols? xarses is about to fix that one: https://github.com/rodjek/puppet-lint/pull/234 | 22:33 |
| clarkb | gah, I think I see what the problem is | 22:33 |
| clarkb | mgagne: \o/ | 22:33 |
| zaro | clarkb: yes. saw your comments. not sure how to verify diff list. can mordred do that? | 22:34 |
| clarkb | fungi: jeblair: I am stopping puppet on nodepool.o.o to test really quick without puppet interferring | 22:34 |
| clarkb | zaro: you should be able to create those lists locally and make sure they are sane | 22:34 |
| fungi | clarkb: on 55827 the test builtin (and test itself) may be reporting on the *string* being nonzero length, which is an unfortunate default behavior | 22:34 |
| *** rongze has joined #openstack-infra | 22:35 | |
| clarkb | fungi: no once I put the variable setting on its own line it got happy | 22:35 |
| clarkb | fungi: not sure what was happening there | 22:35 |
| fungi | ahh, yeah i think if you'd wrapped the invocation in "" that would have been the case, but what you have there is (probably?) okay | 22:37 |
| clarkb | fungi: yeah and it passes devstack-gate which it shouldn't if my question was an actual issue | 22:38 |
| clarkb | I have fixed nodepool locally, pushing a patch momentarily | 22:38 |
| zaro | clarkb: hmm. doesn't the check-requirements job run on mutliple projects? | 22:38 |
| zaro | clarkb: is checking one project enough to verify? | 22:39 |
| clarkb | zaro: yes, but you can run that script on a VM of yours | 22:39 |
| clarkb | no checking one projects isn't enough | 22:39 |
| *** rongze has quit IRC | 22:39 | |
| zaro | clarkb: ok. so to narrow the scope, is there a list of projects we should verrify agains? | 22:40 |
| clarkb | zaro: the list of projects in the devstack project list | 22:41 |
| clarkb | zaro: there is a variable in devstack or devstack gate that lists them all | 22:41 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Properly escape Schrodinger's Cat in yaml. https://review.openstack.org/58868 | 22:43 |
| fungi | clarkb: zaro: not http://git.openstack.org/cgit/openstack/requirements/tree/projects.txt ? | 22:43 |
| fungi | yay quote escaping | 22:44 |
| *** pete5 has quit IRC | 22:44 | |
| clarkb | fungi: jeblair ^ I am watching nodepool with that change in place locally, it seems much happier | 22:44 |
| * fungi will name his next release ☃ | 22:44 | |
| fungi | EWHYNOT | 22:45 |
| clarkb | according to graphs we are building a lot of servers | 22:45 |
| fungi | that is indeed a jarring change in the graph | 22:46 |
| clarkb | NotFound: No Image matching {'name': "Fedora 19 (Schrodinger's Cat)\n"}. (HTTP 404) apparently that made nodepool happy but not completely happy | 22:47 |
| * clarkb does a nova flavor-list | 22:47 | |
| clarkb | er image-list | 22:48 |
| fungi | yeah, ram and disk sizes won't help you here | 22:48 |
| openstackgerrit | A change was merged to openstack-infra/config: Properly escape Schrodinger's Cat in yaml. https://review.openstack.org/58868 | 22:48 |
| clarkb | oh the yaml string literal added a newline | 22:49 |
| * clarkb reads more about yaml strings | 22:49 | |
| mgagne | clarkb: was about to comment: In JJB, double-quotes works for me. Can't say for nodepool. | 22:49 |
| *** flaper87 is now known as flaper87|afk | 22:49 | |
| fungi | mgagne: so "Fedora 19 (Schrodinger\'s Cat)" i guess? | 22:49 |
| mgagne | no escape | 22:49 |
| zaro | fungi: clarkb : ok, will do a few, but not all of those projects | 22:50 |
| clarkb | mgagne: it is the same yaml parser, I will give that a try | 22:50 |
| fungi | er, right, base-image: "Fedora 19 (Schrodinger's Cat)" | 22:50 |
| mgagne | clarkb: http://paste.openstack.org/show/54105/ | 22:51 |
| *** denis_makogon_ has quit IRC | 22:51 | |
| *** paul-- has joined #openstack-infra | 22:51 | |
| *** senk has joined #openstack-infra | 22:53 | |
| clarkb | gah, git log is going to be ugly | 22:54 |
| fungi | clarkb: http://xkcd.com/1296/ ? | 22:55 |
| mgagne | :D | 22:55 |
| *** dkliban has quit IRC | 22:56 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Properly escape Schrodinger's Cat in yaml. https://review.openstack.org/58871 | 22:57 |
| clarkb | lets try that | 22:57 |
| *** jhesketh__ has quit IRC | 22:58 | |
| *** jhesketh__ has joined #openstack-infra | 22:58 | |
| * fungi will clearly approve *anything* the night before a major holiday... click! | 22:59 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: setup gerritbot messages to openstack-doc channel https://review.openstack.org/58872 | 22:59 |
| clarkb | fungi: 58871 is happier, but now we have quota limits on key errors for rax nodes | 23:00 |
| clarkb | and it can't copy things to /opt apparently | 23:00 |
| clarkb | jeblair: fungi: either of those errors have an apparent cause? | 23:01 |
| *** loq_mac has joined #openstack-infra | 23:02 | |
| *** paul-- has quit IRC | 23:02 | |
| *** mattoliverau has joined #openstack-infra | 23:04 | |
| *** talluri has joined #openstack-infra | 23:04 | |
| jeblair | clarkb: there's weird sudo stuff going on in nodepool scripts, you have to be careful. | 23:04 |
| jeblair | clarkb: i don't understand 'quota limits on key errors for rax nodes' | 23:05 |
| clarkb | jeblair: let me get the exact error message | 23:05 |
| *** paul-- has joined #openstack-infra | 23:05 | |
| clarkb | OverLimit: Quota exceeded, too many key pairs. (HTTP 413) | 23:05 |
| mgagne | jeblair: isn't it related to ssh keys quota? | 23:05 |
| jeblair | mgagne: yes, sounds right | 23:05 |
| clarkb | mgagne: I think it is, but why would nodepool be trying to use different keys? | 23:05 |
| clarkb | shouldn't it use the existing key? | 23:05 |
| jeblair | clarkb: it creates a key for each node it spins up | 23:05 |
| mgagne | jeblair: AFAIK, a temp key is created before spawning a slave. in case of failure, I suspect keys aren't cleared properly OR too many slaves are spawn at the same time. | 23:06 |
| clarkb | jeblair: interesting, maybe it will be happy once the number of concurrent nodes it is spinning up falls? | 23:06 |
| clarkb | jeblair: I will poke at key listing to see if there is any obvious stale stuff | 23:06 |
| jeblair | clarkb: i would not be surprised if there has been significant leakage | 23:06 |
| jeblair | hopefully should be getting better now that the cleanup thread is working | 23:07 |
| clarkb | there are definitely a bunch of keys there | 23:07 |
| clarkb | trying to correlate them (or fail to) to existing servers | 23:08 |
| *** talluri has quit IRC | 23:08 | |
| mgagne | jeblair: looks like there's no metadata linked to keypair, there is no way to tell how old the key is. | 23:08 |
| clarkb | there doesn't seem to be a 1:1 map between keypair and slave either | 23:09 |
| fungi | that's... well, gross is the only word which comes to mind | 23:10 |
| *** eharney has quit IRC | 23:10 | |
| fungi | they give you a quota on something autogenerated, but no correlation to what generated it | 23:11 |
| *** alexpilotti_ has quit IRC | 23:11 | |
| clarkb | the keypair names are devstack-precise-check-####### vs the rax-ord-##### stuff for servers | 23:11 |
| clarkb | or maybe that means those are keys for image builds instead of server launche? | 23:11 |
| fungi | too many simultaneous builds? | 23:12 |
| clarkb | or too many builds that didn't clean up after themselves | 23:12 |
| *** ryanpetrello has joined #openstack-infra | 23:12 | |
| fungi | also, kudos on the fauxfrench launche | 23:12 |
| fungi | i will have to use that | 23:12 |
| clarkb | ha, that was a tyop :) | 23:13 |
| clarkb | so https://review.openstack.org/#/c/58871/1 seems to be working, the problems are external to having the image name | 23:13 |
| lifeless | clarkb: and that wasn't ? | 23:13 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add a zuul client https://review.openstack.org/58841 | 23:13 |
| clarkb | tyop was an intentional typo | 23:13 |
| fungi | toooooop | 23:13 |
| *** ^demon|meeting has quit IRC | 23:13 | |
| *** jcooley_ has joined #openstack-infra | 23:14 | |
| jeblair | clarkb: yes, keypairs should only be created for image builds | 23:15 |
| *** senk has quit IRC | 23:15 | |
| jeblair | clarkb: so i was wrong earlier. sorry. am actually reading code now. | 23:16 |
| *** ryanpetrello has quit IRC | 23:17 | |
| jeblair | clarkb: a server should have a key_name attribute linking it to a key | 23:17 |
| jeblair | clarkb: (that is, servers that are being used to build an image) | 23:18 |
| clarkb | jeblair: gotcha, will try going that direction | 23:20 |
| jeblair | nova show d732995d-0784-4de5-8550-bffb0b2469fe|grep key | 23:20 |
| jeblair | | key_name | devstack-f19-check-1385594034 | | 23:20 |
| *** fbo is now known as fbo_away | 23:20 | |
| jeblair | clarkb: so, er, every other key in the list should be able to be deleted. | 23:20 |
| jeblair | clarkb: and since there are quite a number of 'devstack-f19-check' keys, it seems like there's a bug. | 23:21 |
| *** jcooley_ has quit IRC | 23:21 | |
| *** rnirmal has quit IRC | 23:22 | |
| clarkb | jeblair: I think the bug must be when you bomb out trying to copy /opt the keypair isn't being removed | 23:22 |
| sarob | we have a problem with redirects on http://docs.openstack.org/trunk/training-guide | 23:22 |
| clarkb | jeblair: I have the normal jenkins account creds sourced, nova list only shows static slaves. Do I have the wrong account? too much stuff going on for my head to keep it all straight... | 23:23 |
| jeblair | clarkb: ORD region | 23:23 |
| jeblair | clarkb: is where i was looking | 23:23 |
| clarkb | oh I bet the region is different | 23:24 |
| sarob | as listed in repo openstack-manuals/www/.htaccess, should be redirecting to docs.openstack.org/trunk/training-guides | 23:24 |
| jeblair | sarob: that's exactly what it does for me, though training-guides is a 404 | 23:25 |
| sarob | yeah, that problem too | 23:25 |
| sarob | it builds in the correct dir locally | 23:25 |
| clarkb | jeblair: though, is it worth cleaning those keys now if they are just going to leak all over again? probably better to fix the nodepool bug first | 23:25 |
| jeblair | clarkb: depends on if you want image builds to work or be broken | 23:26 |
| sarob | jeblair: i just updated the .htaccess file | 23:26 |
| sarob | jeblair: and renamed training-guide to training-guides and updated the pom.xml | 23:26 |
| sarob | jeblair: https://review.openstack.org/#/c/58869/ | 23:27 |
| *** fifieldt has joined #openstack-infra | 23:27 | |
| sarob | fifieldt jeblair idea why this went sideways? | 23:28 |
| jeblair | sarob: sorry, i'm digging into another problem now | 23:28 |
| sarob | np, let me know if there is something i should be changing | 23:29 |
| jeblair | sarob: here are logs for your change http://logs.openstack.org/7e/7ec78aef2d2fdb7e0513922921737055b3fa5136/post/ | 23:30 |
| jeblair | sarob: you might look for errors there | 23:30 |
| fungi | sarob: training-guides vs training-guide (i see the latter on the ftp site). computers are fairly unfamiliar with the singular vs parallel dilemma in english? | 23:30 |
| *** alexpilotti has joined #openstack-infra | 23:30 | |
| clarkb | jeblair: are you digging into the nodepool thing? | 23:31 |
| jeblair | clarkb: yeah | 23:31 |
| jeblair | clarkb: i'm not trying to fix anything, that's all you | 23:31 |
| jeblair | clarkb: i'm trying to understand if the keys were attempted to be deteleted | 23:31 |
| jeblair | blah | 23:32 |
| clarkb | jeblair: ok, I don't think deleting keys will help anything until nodepool is happy, as the image builds will continue to leak | 23:32 |
| clarkb | * will continue to leak keys | 23:32 |
| jeblair | clarkb: very slowly, it'll work for a couple months | 23:32 |
| *** heyongli has joined #openstack-infra | 23:33 | |
| fungi | sarob: it may be that training-guide is a greedy match which also matches a substring of training-guides | 23:33 |
| clarkb | jeblair: I will delete keys to see in that case | 23:33 |
| fungi | sarob: but anyway, it looks like things are only being published into the "trunk/training-guide" (singular) directory. i don't see the plural version existing on the ftp server | 23:34 |
| clarkb | jeblair: there are a bunch of f19 keys from failed builds | 23:34 |
| clarkb | jeblair: I think it will leak quite quickly until f19 builds work | 23:34 |
| jeblair | clarkb: definitely. i thought you fixed them. | 23:35 |
| clarkb | no, I was looking at the key thing :) | 23:35 |
| clarkb | I will look at the /opt stuff now | 23:35 |
| jeblair | clarkb: personally, i think you should revert the changes, delete the keys, and go have thanksgiving | 23:35 |
| jeblair | clarkb: but your call. :) | 23:35 |
| *** rongze has joined #openstack-infra | 23:36 | |
| clarkb | jeblair: wfm (I do need to head out shortly), I will propose the revert now, modify the file by hand, delete keys, then start puppet when revert is in | 23:36 |
| jeblair | also, we should not accept untested nodepool script changes. i kind of thought these were tested. i guess i'm going to have to be really really clear about asking that for each change. | 23:36 |
| *** mattoliverau has quit IRC | 23:36 | |
| sarob | fungi i think i figured it out the redirect is trunk/training-guide/ so it matches with the trailing / | 23:37 |
| *** ryanpetrello has joined #openstack-infra | 23:37 | |
| sarob | fungi: when adding back in the trailing string content the / has been matched and is gone | 23:37 |
| clarkb | jeblair: ++ I thought they were being tested offsite too | 23:38 |
| clarkb | nodepool config updated locally | 23:38 |
| fifieldt | sarob, you probably will need to pull in : https://github.com/openstack-infra/config | 23:38 |
| fifieldt | and make changes there | 23:38 |
| fungi | sarob: right, you may be hitting apache's implied rewrites matching on directory name | 23:38 |
| fifieldt | to get it to build | 23:38 |
| *** dkliban has joined #openstack-infra | 23:38 | |
| *** rongze has quit IRC | 23:40 | |
| jeblair | clarkb: ugh, nodepool is missing some logging info for me to be able to forensically determine what's going on. | 23:42 |
| clarkb | jeblair: :( | 23:42 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Revert "Add Fedora 19 to the nodepool (devstack-f19-check)" https://review.openstack.org/58887 | 23:43 |
| clarkb | jeblair: ^ | 23:43 |
| jeblair | clarkb: aprvd | 23:43 |
| *** paul-- has quit IRC | 23:44 | |
| fungi | sarob: fifieldt: so here... http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/projects.yaml#n1597 | 23:44 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Improve image logging https://review.openstack.org/58888 | 23:45 |
| fungi | you're passing "training guide" as a "manual" parameter to http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/manuals-jobs.yaml#n12 | 23:45 |
| fungi | the openstack-{manual} job template | 23:45 |
| fungi | that wants to publish from doc/training-manual/target/docbkx/webhelp/** | 23:46 |
| openstackgerrit | A change was merged to openstack-infra/config: Revert "Add Fedora 19 to the nodepool (devstack-f19-check)" https://review.openstack.org/58887 | 23:46 |
| clarkb | I am deleting f19 keypairs in ORD now | 23:46 |
| fungi | (er, passing "training-guide" with a hyphen i meant) | 23:46 |
| fungi | so probably the job name in openstack-infra/config needs to be pluralized in modules/openstack_project/files/jenkins_job_builder/config/projects.yaml and modules/openstack_project/files/zuul/layout.yaml | 23:48 |
| fungi | sarob: fifieldt: ^ | 23:49 |
| fifieldt | yes, I told sarob this over in docs :) | 23:49 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add a zuul client https://review.openstack.org/58841 | 23:49 |
| fungi | fifieldt: great! i'll shut up then ;) | 23:50 |
| sarob | fungi fifieldt: i working on it. | 23:50 |
| fifieldt | he's been very naughty :) | 23:50 |
| sarob | fungi fifieldt: thx for the reminder, im just scattered right now | 23:50 |
| sarob | fungi fifieldt: stupid thanksgiving makes everyone crazy | 23:50 |
| fungi | sarob: i'm always scattered. no worries | 23:51 |
| fifieldt | OpenStack does that to us all, I'm beginning to believe | 23:51 |
| fifieldt | sarob, about 50% of our weekly staff meeting this week was regarding turkey | 23:51 |
| fifieldt | I can get the sense of it :D | 23:51 |
| sarob | the smell is everywhere, not that im complaining about that part | 23:51 |
| fifieldt | :D | 23:51 |
| jeblair | this channel always smells like turkey | 23:52 |
| sarob | fifield fungi should i create it as a bug then patch? | 23:52 |
| jeblair | sarob: bugs aren't required for infra patches | 23:54 |
| clarkb | IAD is clean | 23:55 |
| clarkb | double checking ORD again | 23:55 |
| fungi | sarob: what fifieldtb said. if your commit message explains what you're fixing, a bug on top of that is really just busywork i think | 23:55 |
| fungi | er, jeblair sid that i guess. stupid irc client colors | 23:56 |
| sarob | fungi roger | 23:56 |
| clarkb | ORD is clear | 23:56 |
| *** dstanek has quit IRC | 23:57 | |
| *** ryanpetrello has quit IRC | 23:59 | |
| *** paul-- has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!