Thursday, 2014-06-05

*** eghobo has quit IRC00:04
*** eghobo has joined #tripleo00:05
SpamapSdo we know what this is: 2014-06-04 18:18:18.087 | Node d76976a6-c09e-4281-a898-37aed0beed7e is locked by host ubuntu, please retry after the current operation is completed. (HTTP 409)00:07
SpamapSalso, are we not going to have an ubuntu (and thus, debuggable) overcloud job back until R1 returns?00:07
lifelessSpamapS: we could switch out ubuntu for fedora00:10
SpamapSstevebaker: https://review.openstack.org/#/c/97703/ <-- quite a bit more efficient overcloud template. (100 nodes == 27000 lines instead of 300,000)00:10
SpamapSlifeless: I think that new overcloud template works00:10
lifelessSpamapS: locked by host the workaround for https://bugs.launchpad.net/ironic/+bug/131522400:11
uvirtbotLaunchpad bug 1315224 in ironic "Ironic should set the node power-state to off when registering a node" [Wishlist,In progress]00:11
lifelessSpamapS: mrda was looking at that (see the ironic channel)00:11
lifelessSpamapS: 390K lines for 2000 nodes00:11
lifelessSpamapS: register-nodes retries for 10 seconds, so that error means that the power thread took 10 seconds to assert power state off with the virt driver, which is odd00:12
SpamapSlifeless: ah00:13
openstackgerritYongli He proposed a change to openstack/tuskar: Rewrite policy to use the new changes of common policy  https://review.openstack.org/9772900:25
*** yamahata has joined #tripleo00:30
*** eghobo has quit IRC00:31
morganfainbergSpamapS, lifeless, I've been looking over the ID vs Name (domain, user, etc) issue with Heat, and I just want to confirm my understanding (before I sound dumb writing up an email) - the issue is00:40
morganfainbergHeat has been deployed and the configuration/template has no knowledge of what is in keystone (and can't ask) for the new IDs?00:40
morganfainbergTherefore, it would be easier to just be able to use a name vs the ID?00:40
morganfainbergis there a reason (besides it isn't implemented) that it is impossible to keep a stash of results when creating things like Domain/Project/etc and refer to them in a later-run template? [I'm catching up on Heat at the moment, so i'm a bit behind the curve]00:41
*** rwsu has quit IRC00:51
SpamapSmorganfainberg: we create those things _long_ after Heat is deployed.01:22
SpamapSmorganfainberg: we initialize the new cloud not from heat, because that would be a complicated circular dependency.01:22
SpamapSmorganfainberg: we could certainly ask for the IDs constantly until we find the name we're looking for...01:24
*** morazi has quit IRC01:25
SpamapSmorganfainberg: but we're suggesting that the lookup of name->ID be built into keystoneclient, and then we can let Heat try forever to initialize itself with the given name.01:25
*** yamahata has quit IRC01:26
*** yamahata has joined #tripleo01:26
*** xuhaiwei has joined #tripleo01:36
*** victor_lowther has quit IRC01:42
*** TravT has quit IRC01:42
*** jcooley has quit IRC01:44
*** victor_lowther has joined #tripleo01:45
*** nosnos has joined #tripleo01:46
morganfainbergSpamapS,ah ok cool. and it is built into keystoneclient at this point (or should be)- i'll 2x check01:47
morganfainbergtonight01:47
* morganfainberg needs to go get dinner01:47
*** jcooley has joined #tripleo01:49
SpamapSmorganfainberg: cool01:57
stevebakerSpamapS: allHostsConfig looks good02:14
stevebakerSpamapS: As an aside, why build the hosts as an epic Fn::Join rather than building a list of dicts which is iterated by the oac template?02:14
greghaynesLooking for cores on https://review.openstack.org/#/c/93104/ and the review it depends on (both have passed tests)02:21
openstackgerritA change was merged to openstack/tripleo-image-elements: Allow multiple binds per service in haproxy  https://review.openstack.org/8992502:27
openstackgerritA change was merged to openstack/tripleo-image-elements: Prevent duplication of net_binds per service  https://review.openstack.org/9509702:27
*** pcrews has quit IRC02:32
*** ramishra has joined #tripleo02:53
*** akuznetsov has quit IRC02:54
*** untriaged-bot has joined #tripleo03:00
untriaged-botUntriaged bugs so far:03:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/132631903:00
uvirtbotLaunchpad bug 1326319 in diskimage-builder "Add support for deploying for ironic from baremetal node" [Undecided,New]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132601603:00
uvirtbotLaunchpad bug 1326016 in tripleo "Different configuration files for different nova services are merged together not allowing to specify specific value for a specific service" [Undecided,In progress]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132576203:00
uvirtbotLaunchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New]03:00
*** untriaged-bot has quit IRC03:00
*** sballe has joined #tripleo03:11
*** sballe has quit IRC03:26
*** sballe has joined #tripleo03:27
*** cody-somerville has quit IRC03:42
*** nosnos has quit IRC03:43
*** sballe has quit IRC03:43
*** eghobo has joined #tripleo03:45
SpamapSstevebaker: "tradition"03:47
* tchaypo learns about awk -F03:55
tchaypowhy did i never know about that before?03:55
greghaynesF yea03:56
lifelessmerica03:57
tchaypoi just wanted to use a word to mean "like extrapolate, but backwards"03:58
tchaypoand almost said "backstrapolate", but then i started thinking about steak and got distracted03:59
lifelessinterpolate?04:00
tchaypoyeah, clealy interpolate is the word I wanted.04:00
lifelessclealy?04:00
tchaypoin other news04:00
tchaypoif we stop picking on my speling04:01
tchaypoactually i answered my own question, never mind.04:01
lifeless:)04:01
*** rpodolyaka1 has joined #tripleo04:04
* tchaypo is resigned to one messy rebase when the readme.md -> readme.rst change lands04:05
tchayposo many other changes that are in flight have updates to readme.md that I'm going to have to copy across.04:06
tchaypole sigh04:06
greghaynestchaypo: you could ask them to rebase off you04:08
greghayness/ask/beg04:08
greghaynesespecially if yours looks like its giong to merge04:08
tchaypowell actually04:10
tchaypohttps://review.openstack.org/#/c/97129/ and https://review.openstack.org/#/c/95308/ both have one +2 already, https://review.openstack.org/#/c/92749/ only has +1s04:11
tchaypobut since lifeless and greghaynes are around, that could all change very quickly...04:11
tchaypoand then I'd be happy to offer them new patches that do the rebase for them04:12
tchaypostevebaker: were you doing some work on setup_pki recently? Is https://review.openstack.org/#/c/95101/ relevant to you?04:13
tchaypoand once again with https://review.openstack.org/#/c/96383/ I'm being all persnickety about commit messages. I bet I'm going to develop a reputation.04:22
*** tzumainn has quit IRC04:22
*** sballe has joined #tripleo04:25
*** eghobo has quit IRC04:30
*** eghobo has joined #tripleo04:31
*** sballe has quit IRC04:32
tchaypolifeless: a question about the 3 reviews/day stat thing04:34
tchaypoif I'm coming back each day and commenting on the latest patchset on each of the 6 changes I'm following - does that count as meeting the 3 reviews/day quota?04:34
tchaypoor will each of those reviews only count once, regardless of how many patchsets on that change I review/comment on?04:34
*** cody-somerville has joined #tripleo04:35
*** jcoufal has joined #tripleo04:36
*** lazy_prince has joined #tripleo04:38
*** matsuhashi has joined #tripleo04:39
lifelesstchaypo: its a balance04:40
lifelesstchaypo: the point is to be reading a bunch of code04:40
lifelesstchaypo: if you're spending 2hrs a day doing code review, thats probably a decent balance04:40
*** nosnos has joined #tripleo04:41
SpamapStchaypo: in the stats, if you make a change to the total score (-1, +1, -2, +2) then it counts.04:43
* mordred is now striving for inbox zero04:43
lifelesswow, 50% deploy failure rate04:44
mordredwith code reviews04:44
mordredwhich is  made easier by gertty04:44
lifelesstchaypo: actually, let me re-answer differently04:44
lifelesstchaypo: the core commitment04:44
lifelesstchaypo: is to do three reviews; we haven't specified it more precisely, but the stats we have follow the rule SpamapS indicated (including repeated -1's etc)04:45
tchaypoSpamapS: ah, interseting. So if a review is already -1 and I add a -1 for another reason, it doesn't count; but if I change a review from +1 to -1 it does?04:46
clarkbmordred: I am so happy you find gertty useful because we get a ton of reviews out of you now :)04:46
lifelesstchaypo: no, it counts04:47
lifelesstchaypo: from a stats perspective, but the commitment isn't about stats04:47
tchayposo adding another reason to -1 a review is a useful contribution and happily counts towards the stats; just jumping on and giving an "I don't like it either" -1 isn't very useful but unhappily counts towards the stats?04:49
lifelessyes04:49
lifelesswhich is why meta-reviews of candidate core reviewers take time04:49
lifelessbecause I read through rather than relying on stats04:49
tchaypo+104:50
lifelessthe stats are a first pass filter04:50
tchaypolooking forward to doing a run through soon?04:50
lifelessno :)04:50
SpamapSI wonder how hard a meta-review dashboard would be04:51
SpamapSIMO a better use of peoples time than a different frontend04:51
clarkbSpamapS: that exists04:52
clarkbthough as a gertty user I find it to be so much better than alternative04:53
clarkbI get a lsit of things to review, I review them. I can kill threads that don't matter. And check out stuff locally for testing and review04:53
clarkbalso I get about 150 more reviews a week from mordred with gertty so it is a success04:53
tchaypotell me more about this gertty thing04:54
lifelessclarkb: gertty isn't a metareview04:54
SpamapSmetareview as in, review the reviews04:54
clarkblifeless: nope it is a different frontend04:54
SpamapSso look at somebody's -1 and their comments and judge that04:54
tchaypogoogle helpfully fixes my spelling "getty" and points me at getty.com.au04:55
clarkbSpamapS: right the reviewstats things from russellb does that apparently04:55
clarkbjeblair uses it04:55
*** akuznetsov has joined #tripleo04:55
tchaypoah, http://git.openstack.org/cgit/stackforge/gertty/04:55
lifelessclarkb: no, it doesn't04:55
clarkbtchaypo: it is a curses gerrit front end04:55
SpamapSclarkb: as in, shows you a person's reviews from the last 30 days?04:55
clarkblifeless: according to jeblair it does04:55
*** eguz has joined #tripleo04:55
lifelessclarkb: it just get aggregate stats04:55
lifelessclarkb: reviewstats most definitely doesn't - I help maintain it04:55
clarkblifeless: it can apparently generate in depth reports on the disagreements04:55
clarkbtchaypo: its a bit rough around the edges but a lot of the bugs have been killed and you can do offline reviews04:56
lifelessclarkb: news to me and I can't see anything about that in it04:56
clarkblifeless: maybe jeblair has hacked it on top. you would have to ask him04:56
lifelessclarkb: so cool, but I don't know how to access it04:56
tchaypooffline reviews sound like they'd be useful on a plane or something04:57
tchaypoexcept that I can never do anything useful on a plane04:57
clarkbtchaypo: yup. mordred did 89 reviews on monday apparently04:57
clarkband I am 99% sure that was all plane04:57
tchaypo!04:57
tchaypothat's impressive.04:57
tchaypoi just typed "cd srcf" and bash said "oh you meant src" and took me into the right folder04:58
*** eghobo has quit IRC04:58
tchaypospooky.04:58
SpamapSI can't even look at my laptop if I'm not in an exit row04:59
SpamapSlike literally, can't look at it.. the seat in front smashes it04:59
clarkbSpamapS: ya I tend to spend my time trying to sleep or reading a book or playing 3DS05:01
StevenKtchaypo: zsh will spelling correct, too05:01
tchaypoyeah, I don't think I'll be using it on a plane, but I suspect I might use it in other contexts.05:01
*** eghobo has joined #tripleo05:04
tchaypoI just realied that the "os" at the start of "oslo" probably comes from the same place as at the start of "os-*-config"05:04
*** eguz has quit IRC05:04
*** eghobo has quit IRC05:04
*** eghobo has joined #tripleo05:05
StevenKtchaypo: I'm not actually trolling (for once), but is your laptop actually small enough to open when sitting in a plane seat?05:05
lifelessmine isn't05:05
tchaypoI don't know, I haven't tried. I suspect I could manage it if I put it on my lap, but the screen would be hidden if the person in front reclines05:06
tchaypobut I'd have no space for the keyboard, mouse etc05:06
StevenK... tripod05:06
lifelesssuperglue05:06
StevenKI've used my 9470m on my lap on a plane for about 5 minutes, it was quite uncomfortable05:06
* tchaypo waits for gertty to sync05:06
*** dkehn has quit IRC05:06
tchayposo I'm finally forcing myself to start writing a spec for multiple env handling, if only to serve as a consolidation/documentation of current practices05:07
tchaypoI've searched but can't find any previous discussion, it seems like it Just Happened05:07
tchaypobut if anyone can point me at something I mised...05:07
*** dkehn has joined #tripleo05:08
*** eguz has joined #tripleo05:10
*** eguz has quit IRC05:10
*** eguz has joined #tripleo05:11
*** eghobo has quit IRC05:11
clarkbtchaypo: also if you run into crashes try pulling jeblairs tip of proposed changes05:12
clarkbtchaypo: a bunch of bugs are fixed and features added in that stack and I have been using it without major problems05:12
lifelesstchaypo: 'multiple env' ?05:13
*** rakesh_hs has joined #tripleo05:14
mordredclarkb: most of those have landed now05:19
mordredbut seriously - gertty has changed my life in terms of reviews05:19
lifelessso tchaypo05:23
*** jtomasek has joined #tripleo05:23
lifelesstchaypo: is there a way to create a URL for Jira that will get you to a partially filed out new issue ?05:23
greghayneshahah05:24
greghaynespoor tchaypo is go-to on jira help05:24
lifelesstchaypo: specifically, we have a number of fields to fill out when filing DC ops tickets, and manual data entry is not anyones friend05:24
lifelessgreghaynes: same way I am on LP :P05:24
greghaynesgood point05:24
StevenKOr me05:24
*** marun_afk is now known as marun05:29
*** rpodolyaka1 has quit IRC05:33
morganfainbergmordred, dude gerrtty is awesome, just getting used to it... but... OMG next time I need to travel, so awesome05:39
mordredmorganfainberg: it's the best thing evar - and it keeps getting better05:40
mordredI want "u" to get me from diff to review screen - I may add a patch05:40
morganfainbergmordred, unfortunately, i current have it crashing on a specific review, trying to debug why that review keeps crashing it out, but... other than that - excellent05:40
lifelessnuts, 5 hung deploys05:41
clarkbmorganfainberg: is there binary data? I think gertty may crash on diffs of binary files05:41
morganfainbergclarkb, hm. don't think so, it's a spec review05:41
morganfainbergclarkb, so RST only05:41
morganfainbergclarkb, and it's a new file.05:41
mordredis there a file-level comment?05:42
morganfainbergmordred, oooh that might be it. i'll check on that.05:42
mordredmorganfainberg: the fix for handling file level comments, and also reviews with missing git changes both just landed recently05:42
mordredmorganfainberg: try pulling again and re-installing05:42
morganfainbergmordred, well i pulled and installed this afternoon05:42
morganfainbergmordred, crashed with a different error, but same review05:42
mordrednice05:43
morganfainbergmordred, i'm sure i'll figure it out when i spend some time on it - then i'll post a fix!05:43
morganfainberg:)05:43
morganfainbergmordred, oooh have a question for you hold on a sec.05:45
*** mrunge has joined #tripleo05:46
morganfainberganyway catch you all tomrrow.05:47
*** mrunge has quit IRC05:47
lifelessmorganfainberg: night05:47
morganfainberglifeless, i think i've got my head around the user_domain stuff, gonna chat w/ SpamapS more tomorrow about it, but might have some info we can turn into something useful.05:50
lifelesscool05:51
lifelessmorganfainberg: a specific success story would be enabling the neutron callback stuff in tripleo deploys05:52
*** rpodolyaka1 has joined #tripleo05:52
morganfainberglifeless, well. lets see if we can make it really good news. i've been wrapping my head around triple-o here and... heat as well, it's very different than looking at identity code ;)05:53
openstackgerritA change was merged to openstack/tripleo-incubator: Add Swift roles required to poll for Swift usage  https://review.openstack.org/9473005:53
morganfainbergannnyway. i must sleep05:54
lifelessnight05:54
*** jcoufal has quit IRC05:56
*** akuznetsov has quit IRC05:58
SpamapSyeah.. this f20 overcloud job is useless06:02
SpamapShttp://logs.openstack.org/03/97703/3/check-tripleo/check-tripleo-overcloud-f20/0ff7e52/console.html06:02
lifelessits not useless - it tells us when its ok06:02
SpamapSfails..06:02
SpamapS"check the logs"06:02
lifelessjust not when its bad :)06:02
SpamapSexcept.. there are no logs06:02
SpamapSbecause lennart06:02
lifelessSpamapS: so - propose a switch to the ubuntu job06:02
lifelessSpamapS: (now) and ping mordred to get it live asap06:02
*** rpodolyaka1 has quit IRC06:02
SpamapSlifeless: I believe derekh has a "ship the journal" patch in progress. But my question is, why did we drop the ubuntu one?06:03
lifelessSpamapS: capacity06:03
SpamapSlifeless: how're we on R1 revivavl?06:03
SpamapSspelled correctly06:03
SpamapSTHIS I COMMAND06:03
lifelessdebugging the failing nodes06:03
* SpamapS has decided to channel Serpentor06:03
lifelessfound one with wedged IPMI, fixed06:03
SpamapSlifeless: I got your email about one still w/ no link06:04
lifelessone with no net link - stale Jira ticket, have pinged in hipchat about how to resurrect06:04
SpamapSlifeless: thought we fixed all those06:04
*** mrunge has joined #tripleo06:06
*** rpodolyaka1 has joined #tripleo06:08
SpamapSok started my own local devtest overcloud run06:09
SpamapSmay the mirrors be ever in my favor06:09
* SpamapS goes to fold laundry while that runs06:09
lifelessSpamapS: DC folk checked cabling, handed off to NIC folk, and it was closed in the middle.06:12
lifelessSpamapS: stayed closed.06:12
clarkbSpamapS: there is a huge heat stack in the gate that can't merge due to a merge failure06:14
*** rpodolyaka1 has quit IRC06:14
*** marun is now known as marun_afk06:14
clarkbnot sure if you care about pushing a rebase to clean that up06:14
clarkb95996 is where the stack starts06:14
*** akuznetsov has joined #tripleo06:20
lifelessSpamapS: so my goal is having all nodes in ironic either in maintenance (and up to date jira ticket) or working06:25
lifelessSpamapS: so nova boot --num-instances=available -> success06:25
lifelessSpamapS: since I know heat won't cope otherwise06:25
openstackgerritlokesh s proposed a change to openstack/tripleo-incubator: When following the install procedure in devtest.html, any proxy information in the environment (eg, $http_proxy) will not be seen by apt/yum/zypper being run inside install-dependencies, as those commands are run inside sudo.  https://review.openstack.org/9638306:26
lifelessgit first line fail06:26
*** loki184 has joined #tripleo06:27
*** jprovazn has joined #tripleo06:29
lifeless.16 is off in lala land06:29
lifelesspoor ilo06:30
openstackgerritlokesh s proposed a change to openstack/tripleo-incubator: Make install-dependencies use sudo -E to enable use of env proxy.  https://review.openstack.org/9638306:30
adam_ghey! does tripleo-incubator do anything to adjust neutron port quotas for larger scale clouds?06:31
lifelessadam_g: disables it06:31
adam_glifeless, hmph06:31
lifelessadam_g: for the deployment layer that is06:32
adam_glifeless, im hitting it on the 36th node of an overcloud06:32
lifelessadam_g: do you have enough ip addresses ?06:32
adam_glifeless, ah!06:34
adam_ghmm. neutron should be logging something else there.06:34
lifelessadam_g: neutron or nova? there's a bug open on nova06:34
lifelessI am fairly sure there is anyhow06:34
adam_glifeless, good catch, thanks.06:34
lifelessits a catch-by-http-code and then assume it means one thing, situation06:35
adam_glifeless, er, neutron should be raising something else or nova logging something else.06:35
adam_gyeah06:35
adam_ganyway, thanks again. i will sleep better now06:35
adam_gcya06:35
lifelesssadface06:37
lifelessMonitor is in graphics mode or an unsupported text mode.06:37
*** yfaykya has quit IRC06:37
*** rdopiera has joined #tripleo06:38
*** lsmola_ has joined #tripleo06:38
*** yfaykya has joined #tripleo06:39
*** lsmola has quit IRC06:39
*** yfaykya has quit IRC06:43
*** yfaykya has joined #tripleo06:45
*** yfaykya has quit IRC06:49
*** yfaykya has joined #tripleo06:51
GheRiveromorning all06:51
*** Guest_ has joined #tripleo06:55
*** yfaykya has quit IRC06:56
*** yfaykya has joined #tripleo06:57
*** xuhaiwei has quit IRC06:57
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Explicitly specify mysql node address  https://review.openstack.org/9777506:58
*** Guest_ has quit IRC06:58
*** loki184 has quit IRC06:59
*** yfaykya has quit IRC07:02
*** yfaykya has joined #tripleo07:03
*** AaronGr has quit IRC07:06
*** yfaykya has quit IRC07:07
*** yfaykya has joined #tripleo07:09
*** akuznetsov has quit IRC07:10
*** AaronGr has joined #tripleo07:13
*** yfaykya has quit IRC07:14
*** yfaykya has joined #tripleo07:15
*** yfaykya has quit IRC07:20
*** yfaykya has joined #tripleo07:21
*** jp_at_hp has joined #tripleo07:25
*** yfaykya has quit IRC07:26
tchaypolifeless: Yeah it's possible. Messy but possible. I'll ping you in the morning to get details07:26
lifelesstchaypo: see the hp internal etherpad linked on the hp r1 rebuild etherpad07:27
*** yfaykya has joined #tripleo07:28
tchaypoI think I have a bookmark for the r1 ether pad so I should be able to get there07:28
openstackgerritGhe Rivero proposed a change to openstack/tripleo-image-elements: Don't install openvswitch dkms modules in kernel >= 3.12  https://review.openstack.org/9515107:30
*** yfaykya has quit IRC07:32
*** yfaykya has joined #tripleo07:34
*** ci-overcloud has joined #tripleo07:36
ci-overcloud************** ci-overcloud complete status=1 ************07:36
*** ci-overcloud has quit IRC07:36
lifelessNg: we need a new ssl cert for ci-overcloud.hp1.tripleo.org too07:36
*** yfaykya has quit IRC07:38
lifelessok, doing a 40 node test build (current goal, flushing out second-layer deploy fails)07:38
*** yfaykya has joined #tripleo07:40
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Initial cirros element  https://review.openstack.org/9466407:41
*** yfaykya has quit IRC07:45
*** yfaykya has joined #tripleo07:46
*** andreaf has joined #tripleo07:50
*** yfaykya has quit IRC07:50
SpamapS| fault                                | {"message": "Timed out waiting for a reply to message ID c12b7798054b4b9e912ef26034a53d92", "code": 500, "details": "  File \"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/conductor/manager.py\", line 802, in build_instances |07:52
*** yfaykya has joined #tripleo07:52
SpamapShrm07:52
*** jistr has joined #tripleo07:53
SpamapSlifeless: lifeless we need to add the word 'text' to kernel cmdline to avoid graphics mode on trusty btw07:54
lifelessSpamapS: patch up for ironic ?07:55
SpamapSlifeless: I don't know what you are ?'ing07:56
*** yfaykya has quit IRC07:57
lifelessSpamapS: oh, adding text to the command line07:58
*** yfaykya has joined #tripleo07:58
SpamapSNo07:59
*** pelix has joined #tripleo08:01
lifelessSpamapS: https://review.openstack.org/9805008:02
*** yfaykya has quit IRC08:02
*** yfaykya has joined #tripleo08:04
*** e0ne has joined #tripleo08:06
SpamapSlifeless: thanks, I had no idea where to even start08:06
SpamapSlifeless: did you see my "Timed out waiting for a reply" error?08:09
SpamapSlifeless: ring any bells?08:09
*** yfaykya has quit IRC08:09
lifelessrpc worker exhaustion in the conductor08:09
SpamapSwhile booting.. 1???08:09
*** yfaykya has joined #tripleo08:10
SpamapS2014-06-05 06:09:57.520 5196 CRITICAL nova [-] ImportError: No module named ironic.nova.scheduler.ironic_host_manager08:12
SpamapSthats the problem08:12
lifelessyow :)08:13
lifelessthere's a patch up for that in ironic08:13
SpamapSShouldn't I expect to see ironic importable in the nova venv for that?08:13
SpamapS(I don't)08:13
*** dtantsur|afk is now known as dtantsur08:14
*** yfaykya has quit IRC08:15
SpamapSmy seed has nova-baremetal in the elements list, but is configured for ironic..08:16
giulivobnemec, regarding this https://review.openstack.org/#/c/93954/ , I replied to your comment , what you think would be best to do then?08:16
SpamapSmost odd08:17
*** yfaykya has joined #tripleo08:17
*** lucasagomes has joined #tripleo08:17
*** martyntaylor has joined #tripleo08:18
* SpamapS rips it all down and tries again08:18
SpamapSwhat a lovely opportunity to sleep08:18
*** e0ne has quit IRC08:19
*** yfaykya has quit IRC08:21
*** dtantsur is now known as dtantsur|afk08:23
*** yfaykya has joined #tripleo08:23
*** ramishra has quit IRC08:26
*** jcoufal has joined #tripleo08:27
*** yfaykya has quit IRC08:27
*** jcoufal has quit IRC08:27
*** jcoufal has joined #tripleo08:28
*** ramishra has joined #tripleo08:28
*** yfaykya has joined #tripleo08:29
*** athomas has joined #tripleo08:30
openstackgerritGiulio Fidente proposed a change to openstack/tripleo-incubator: Make good use of TRIPLEO_OS_{FAMILY,DISTRO} where useful.  https://review.openstack.org/9374508:30
*** yfaykya has quit IRC08:33
*** yfaykya has joined #tripleo08:35
*** yfaykya has quit IRC08:39
*** yfaykya has joined #tripleo08:41
*** yfaykya has quit IRC08:45
*** yfaykya has joined #tripleo08:47
*** yfaykya has quit IRC08:51
*** yfaykya has joined #tripleo08:53
*** yfaykya has quit IRC08:57
*** yfaykya has joined #tripleo08:59
*** untriaged-bot has joined #tripleo09:00
untriaged-botUntriaged bugs so far:09:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/132631909:00
uvirtbotLaunchpad bug 1326319 in diskimage-builder "Add support for deploying for ironic from baremetal node" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132601609:00
uvirtbotLaunchpad bug 1326016 in tripleo "Different configuration files for different nova services are merged together not allowing to specify specific value for a specific service" [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132576209:00
uvirtbotLaunchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New]09:00
*** untriaged-bot has quit IRC09:00
*** yfaykya has quit IRC09:04
openstackgerritTomas Sedovic proposed a change to openstack/tripleo-specs: Moving TripleO Heat Templates towards native Heat  https://review.openstack.org/9793909:05
*** yfaykya has joined #tripleo09:05
*** ci-overcloud has joined #tripleo09:09
ci-overcloud************** ci-overcloud complete status=1 ************09:09
*** ci-overcloud has quit IRC09:09
*** yfaykya has quit IRC09:10
*** yfaykya has joined #tripleo09:12
openstackgerritTomas Sedovic proposed a change to openstack/tripleo-specs: Moving TripleO Heat Templates towards native Heat  https://review.openstack.org/9793909:14
d0ugalshadower: sure, sorry :)09:15
shadowernp09:15
shadowerd0ugal: I haven't read  it in detail yet but they shouldn't09:15
shadowerd0ugal: my spec documents how to move away from merge.py (and that's it)09:16
d0ugalshadower: they both refer to removing merge.py AFAICT09:16
*** yfaykya has quit IRC09:16
shadowerd0ugal: so iiuic, https://review.openstack.org/#/c/97335/ should be about the future handling of templates by tuskar09:17
d0ugalshadower: I've only scanned jdob's spec, I'll read them both in detail.09:17
d0ugalright09:17
shadoweri.e. how to structure them, how to put them together, what about custom roles, etc.09:17
d0ugalshadower: I just seen under "Depenencies" it refers to your spec09:17
shadowerand all of that work kind of depends on merge.py being gone09:17
d0ugalshadower: so you can probably ignore me :)09:18
shadowerah cool09:18
*** yfaykya has joined #tripleo09:18
d0ugalMakes sense, thanks.09:18
shadowerd0ugal: no worries, we should probably be clearer in the specs09:18
shadowerd0ugal: was just about to link to my spec from there, you beat me to it :-)09:20
*** e0ne has joined #tripleo09:21
*** yfaykya has quit IRC09:22
*** ramishra has quit IRC09:23
*** yfaykya has joined #tripleo09:24
d0ugalshadower: heh, figured it might help somebody else confused like me :)09:25
shadowerwas a good call09:26
*** yfaykya has quit IRC09:28
*** yfaykya has joined #tripleo09:30
andrearosashadower: hi, I've just read your specs about the merge.py, good work...and I have a question :) is there anything special to change in the merge.py to be able to use the `environment.yaml` as template for mapping a role to a type?09:31
shadowerandrearosa: I'm not sure how would that work. Merge.py takes a bunch of templates and produces a single file. The environment mapping means you don't join them together but pass them all to heat during stack-create/stack-update09:33
shadowerin theory you could do both: use merge.py for things like scaling, Merge::Map, FileInclude, etc.09:33
shadowerand then pass the generated template along with environment.yaml and any other templates to heat09:34
andrearosaI see, I hope to have time to play with it, my need is to create a specific role with a specific configuration and that feature seems to fit perfectly for my case09:34
andrearosamy use case is quite simple09:34
shadowerright09:35
*** yfaykya has quit IRC09:35
shadoweryeah that should work09:35
shadoweralthough right now you might just be better off adding a new template and passing it to merge.py09:35
andrearosaI want to define a Nova-api role with a given configuration and a Nova-Admin_api role witch is the same service but using a different conf09:35
shadowerright09:36
shadowerandrearosa: although maybe you could just pass the different conf as a Heat Parameter?09:36
shadowerif everything else is the same09:36
*** yfaykya has joined #tripleo09:36
*** derekh_ has joined #tripleo09:37
andrearosaI have to dig into it, my problem is that the same service is the same image element so having a different heat resource is a way to "specialize" a specific service09:39
andrearosabutas I said I am quite new for tripleo and heat so I have to study more ;)09:39
shadowerandrearosa: so you want to deploy two services on two differen machines that have the same image element09:41
*** yfaykya has quit IRC09:41
shadowerbut you want that image element to have different configurations?09:41
andrearosayes09:41
shadowerandrearosa: right, so you should be able to do that with a single heat template. You can pass two sets of configuration -- one for each server09:42
*** yfaykya has joined #tripleo09:42
andrearosathe two sets of configuration are passed via Heat template or via t-i-e?09:43
*** jtomasek has quit IRC09:43
shadoweryou pass them to heat which will send them as metadata to each server where the elements pick them up using os-collect-config and os-apply-config09:44
andrearosathanks for the suggestions and sorry if my questions are dumb09:46
andrearosamuch appreciated09:46
shadowerandrearosa: not at all! Heat is a complex thing, it takes a while to understand it09:46
shadowerI wouldn't say I understand it all myself09:46
*** yfaykya has quit IRC09:47
shadowerfeel free to ask more questions whenever there's anything unclear09:47
andrearosasure thing, thanks again09:48
*** yfaykya has joined #tripleo09:48
*** dtantsur|afk is now known as dtantsur09:50
*** yfaykya has quit IRC09:53
*** yfaykya has joined #tripleo09:54
*** akrivoka has joined #tripleo09:56
*** yfaykya has quit IRC09:59
*** dkehn has quit IRC10:00
*** yfaykya has joined #tripleo10:01
*** dkehn has joined #tripleo10:02
*** rlandy has joined #tripleo10:04
*** yfaykya has quit IRC10:05
*** ramishra has joined #tripleo10:06
*** yfaykya has joined #tripleo10:07
*** yamahata has quit IRC10:10
*** yfaykya has quit IRC10:11
*** yfaykya has joined #tripleo10:13
*** yfaykya has quit IRC10:17
*** yfaykya has joined #tripleo10:19
openstackgerrittherese-mchale proposed a change to openstack/tripleo-image-elements: Use Percona Packages for Ubuntu  https://review.openstack.org/9013410:20
*** yfaykya has quit IRC10:23
openstackgerrittherese-mchale proposed a change to openstack/tripleo-image-elements: Use Percona Packages for Ubuntu  https://review.openstack.org/9013410:25
*** yfaykya has joined #tripleo10:25
*** yfaykya has quit IRC10:29
*** yfaykya has joined #tripleo10:31
*** matsuhashi has quit IRC10:31
*** matsuhashi has joined #tripleo10:34
*** yfaykya has quit IRC10:35
*** yfaykya has joined #tripleo10:37
*** yfaykya has quit IRC10:42
openstackgerrittherese-mchale proposed a change to openstack/tripleo-image-elements: Use Percona Packages for Ubuntu  https://review.openstack.org/9013410:43
*** yfaykya has joined #tripleo10:43
*** yfaykya has quit IRC10:48
*** yfaykya has joined #tripleo10:50
*** yfaykya has quit IRC10:54
*** e0ne_ has joined #tripleo10:54
*** yfaykya has joined #tripleo10:56
*** e0ne has quit IRC10:58
*** yfaykya has quit IRC11:00
*** yfaykya has joined #tripleo11:02
*** jtomasek has joined #tripleo11:02
*** dprince has joined #tripleo11:04
SpamapSwtf.. I'm getting baremetal _and_ ironic configuration in my seed when I set USE_IRONIC=111:06
SpamapSgah, sleep beckons11:06
*** matsuhashi has quit IRC11:06
*** yfaykya has quit IRC11:06
*** matsuhashi has joined #tripleo11:06
*** lazy_prince has quit IRC11:07
*** yfaykya has joined #tripleo11:08
*** matsuhashi has quit IRC11:10
*** yfaykya has quit IRC11:13
*** yfaykya has joined #tripleo11:14
*** yamahata has joined #tripleo11:15
*** yfaykya has quit IRC11:19
*** nosnos has quit IRC11:20
*** yfaykya has joined #tripleo11:20
openstackgerritRobert Parker proposed a change to openstack/tripleo-incubator: Setting up ceilometer SSL  https://review.openstack.org/9625911:22
*** lucasagomes is now known as lucas-hungry11:23
*** matsuhashi has joined #tripleo11:24
*** matsuhashi has quit IRC11:24
*** matsuhashi has joined #tripleo11:24
*** yfaykya has quit IRC11:25
openstackgerritA change was merged to openstack/tripleo-incubator: Setting the metering secrets  https://review.openstack.org/9420111:26
*** yfaykya has joined #tripleo11:26
*** matsuhashi has quit IRC11:29
*** yfaykya has quit IRC11:31
*** yfaykya has joined #tripleo11:32
*** yfaykya has quit IRC11:37
*** yfaykya has joined #tripleo11:39
*** dprince has quit IRC11:39
*** yfaykya has quit IRC11:43
*** yfaykya has joined #tripleo11:45
*** yfaykya has quit IRC11:50
*** yfaykya has joined #tripleo11:51
*** ramishra has quit IRC11:51
*** julim has joined #tripleo11:52
*** ramishra_ has joined #tripleo11:53
*** yfaykya has quit IRC11:55
*** yfaykya has joined #tripleo11:57
*** morazi has joined #tripleo11:58
*** akuznetsov has joined #tripleo12:00
*** dprince has joined #tripleo12:01
*** yfaykya has quit IRC12:01
*** yfaykya has joined #tripleo12:03
*** yfaykya has quit IRC12:07
*** yfaykya has joined #tripleo12:09
*** yfaykya has quit IRC12:14
*** yfaykya has joined #tripleo12:15
*** yfaykya has quit IRC12:20
*** yfaykya has joined #tripleo12:21
openstackgerritAlexis Lee proposed a change to openstack-infra/tripleo-ci: Split very long line in toci_devtest.sh  https://review.openstack.org/9742912:22
*** lucas-hungry is now known as lucasagomes12:24
*** yfaykya has quit IRC12:25
*** yfaykya has joined #tripleo12:28
*** yfaykya has quit IRC12:32
*** yfaykya has joined #tripleo12:34
*** weshay has joined #tripleo12:37
*** yfaykya has quit IRC12:38
*** yfaykya has joined #tripleo12:40
*** yfaykya has quit IRC12:44
*** yfaykya has joined #tripleo12:46
*** tzumainn has joined #tripleo12:46
*** yfaykya has quit IRC12:51
*** yfaykya has joined #tripleo12:52
openstackgerritGerry Drudy proposed a change to openstack/tripleo-image-elements: Enable rsync daemon on swift-storage  https://review.openstack.org/9517312:56
*** yfaykya has quit IRC12:57
*** yfaykya has joined #tripleo12:58
*** ramishra_ has quit IRC12:59
*** ramishra has joined #tripleo13:01
*** yfaykya has quit IRC13:02
openstackgerritDan Prince proposed a change to openstack-infra/tripleo-ci: Log a message when we skip the temprevert  https://review.openstack.org/9810313:03
*** yfaykya has joined #tripleo13:04
*** noslzzp has joined #tripleo13:06
*** yfaykya has quit IRC13:09
*** yfaykya has joined #tripleo13:10
*** ramishra has quit IRC13:14
*** ramishra has joined #tripleo13:15
*** yfaykya has quit IRC13:15
*** yfaykya has joined #tripleo13:17
*** matty_dubs|gone is now known as matty_dubs13:19
*** IvanBerezovskiy has joined #tripleo13:19
*** yfaykya has quit IRC13:21
*** yfaykya has joined #tripleo13:23
*** yfaykya has quit IRC13:27
*** yfaykya has joined #tripleo13:29
*** rakesh_hs has quit IRC13:41
*** TravT has joined #tripleo13:49
rbradydrpince: I've already had one set of eyes on https://review.openstack.org/#/c/96770/ .  It would be great if you could look at it when you get the chance.13:52
*** akrivoka has quit IRC13:53
dprincerbrady: ack13:53
*** akrivoka has joined #tripleo13:54
*** pcrews has joined #tripleo13:54
dprincerbrady: have you updated any elements to use it?13:54
dprincerbrady: would be nice to see those (with passing tests) in a patch series...13:55
rbradydprince: not yet.  I'll work on it13:55
dprincerbrady: don't need to see all of them mind you, just a single service would be enough proof that it works...13:55
*** eghobo has joined #tripleo13:55
*** sballe has joined #tripleo13:55
rbradydrpince: sounds reasonable.  I'll add a patch for that today13:56
*** eguz has quit IRC13:56
*** eghobo has quit IRC13:56
* dprince is apparently a Dr. pince now?13:56
lxsliWhy not Zoidberg?13:58
dprincelxsli: not sure, ask rbrady, he is the one who is changing me nic14:00
*** spzala has joined #tripleo14:00
lxslidprince: casual nick Friday tomorrow!14:01
TheJuliaevery irc channel should have a Zoidberg :)14:02
rbrady*dprince*, lxsli: need #irclint to catch my inaccurate typing14:02
rbradyZoidberg++14:02
dprincelxsli: Yeah! So I usually don't play that game though. Already too many Dan's running around in OpenStack, I find it just further confuses people (which is I guess the point but whatever)14:03
* dprince isn't up on this Zoidberg business14:04
dprincehe looks sort of like a red squid goblin with a nice stash14:05
lxslihttp://knowyourmeme.com/memes/futurama-zoidberg-why-not-zoidberg14:05
*** dtantsur is now known as dtantsur|afk14:05
*** pblaho has joined #tripleo14:07
*** ramishra has quit IRC14:09
*** e0ne_ has quit IRC14:14
*** e0ne has joined #tripleo14:14
*** andreaf has quit IRC14:16
*** athomas has quit IRC14:17
*** athomas has joined #tripleo14:21
*** jtomasek has quit IRC14:30
*** akrivoka has quit IRC14:33
lsmola_shadower: hello, quick question about L173 e.g. https://review.openstack.org/#/c/97939/3/specs/juno/tht_refactor.rst14:35
shadowerlsmola_: shoot14:36
lsmola_shadower: if i understand it correctly, the type: OS::TripleO::Compute, has to be unique in all templates?14:36
*** Lingo_ has joined #tripleo14:36
lsmola_shadower: thinking if there will be one config I want to reuse in more templates, i would have to have it as n different resources, right?14:36
*** Lingo_ is now known as TravelingBear14:37
shadowernot sure I understand14:37
*** akrivoka_ has joined #tripleo14:37
shadoweryou can't have two different templates map to the same resource type14:37
lsmola_shadower: right14:37
shadower(within a single stack-create request)14:37
shadowerbut you could swap them out by changing the url in enviroment.yaml's resource_registry14:38
openstackgerritRobert Parker proposed a change to openstack/tripleo-incubator: Setting up ceilometer SSL  https://review.openstack.org/9625914:38
lsmola_shadower: hm i guess it might not be needed, or we will build the config14:39
lsmola_shadower: e.g. there are some configs that will repeat in all templates14:39
*** TravelingBear is now known as BadCub0114:39
shadoweryeah14:39
shadowerthat shouldn't be a problem14:39
*** ramishra has joined #tripleo14:39
shadowerthis just allows you to use template as a resource. You can have multiple resources of the same type inside your master template. Same with this14:40
shadower(if you assign each config to a standalone template/resource)14:40
*** pblaho has quit IRC14:40
shadowerlsmola_: is that what you're getting at?14:40
lsmola_shadower: right14:42
lsmola_shadower: what i had in mind was e.g. same configs in the same deployment14:43
lsmola_shadower: if we will have fine grained configs, there will be e.g. Rabbit config, that will be on each node14:44
shadoweryeah14:44
lsmola_shadower: i was thinking about a way how to reuse that in all places14:44
shadowerbtw we're doing that today already, only with merge.py14:44
shadowerthat sholudn't be a problem14:44
lsmola_shadower: yeah14:44
shadowerso yeah, you can put each config to a separate template and assign a new type to it in the resource registry14:44
lsmola_shadower: but with heat it is not possible, because the resourne name must be unique, right?14:44
*** ramishra has quit IRC14:45
shadowerlsmola_: but you won't be using the same resource name. You'll use the custom type14:45
shadowere.g.14:45
shadowerrabbit-config.yaml has a RabbitConfig resource that's OS::Heat::StructuredConfig14:45
shadowerenvironment.yaml's resource_registry has OS::TripleO::RabbitConfig: file:///path/to/t-h-t/rabbit-config.yaml14:46
shadowerand then overcloud.yaml can have a NovaRabbitDeployment that has config: {ref: OS::TripleO::RabbitConfig} and ControllerRabbitDeployment that also has config: {ref: OS::TripleO::RabbitConfig}14:47
shadowerand since OS::TripleO::RabbitConfig isn't a resource name but a type, you don't worry about any uniqueness constraints14:47
lsmola_shadower: right, that sounds good, we just need to make resource name unique14:47
lsmola_shadower: cool14:47
shadowerlsmola_: you don't need to make the resource name unique14:48
shadowerthe types in the registry mapping have to be unique14:48
lsmola_shadower: i mean NovaRabbitDeployment has to be unie, right?14:48
shadoweryea14:48
*** jprovazn has quit IRC14:48
shadowerbut that's a deployment not a config14:48
lsmola_shadower: but the type can be used in multiple places14:48
shadowerup14:48
shadowerjust like OS::Nova::Server14:48
shadowerlsmola_: I'm about to go, you fine with this or want to drill down further?14:49
lsmola_shadower: hmm, it tangled my mind :-)14:49
*** rdopiera has quit IRC14:49
lsmola_shadower: let's talk about it next week, I will untagle it :-D14:50
shadowerok14:50
lsmola_shadower: thank you :-)14:50
shadowerlsmola_: I think there's a wiki by stevebaker that shows off different examples of splitting configs and deployments into files14:50
shadowerlemmy dig it up14:50
lsmola_shadower: cool14:50
shadowerlsmola_: this is it I think: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec14:51
shadowerlook at the examples at the bottom14:51
shadowerit starts from an all-in-one example and then splits it up14:52
lsmola_shadower: great, thank you, will check it out14:52
shadowerokay, gotta go, see ya tomorrow14:52
lsmola_shadower: see ya14:53
bnemecgiulivo: Responded to https://review.openstack.org/#/c/93954/  I just wanted to try it locally before +2'ing because that check has been subtly broken so many times before. :-)14:55
*** jistr has quit IRC14:58
*** ramishra has joined #tripleo14:59
*** jistr has joined #tripleo15:00
*** untriaged-bot has joined #tripleo15:00
untriaged-botUntriaged bugs so far:15:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/132631915:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132601615:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132576215:00
uvirtbotLaunchpad bug 1326319 in diskimage-builder "Add support for deploying for ironic from baremetal node" [Undecided,New]15:00
uvirtbotLaunchpad bug 1326016 in tripleo "Different configuration files for different nova services are merged together not allowing to specify specific value for a specific service" [Undecided,In progress]15:00
*** untriaged-bot has quit IRC15:00
uvirtbotLaunchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New]15:00
*** akrivoka_ is now known as akrivoka15:04
*** dprince has quit IRC15:13
giulivobnemec, yet I think it is not entirely correct to use sudo virsh in cleanup just to make sure we connect to qemu:///system15:19
jp_at_hpbnemec: see https://review.openstack.org/#/c/88084/ for https://bugs.launchpad.net/tripleo/+bug/130578515:20
uvirtbotLaunchpad bug 1305785 in tripleo "ubuntu cloud image: sha256sum: WARNING: 1 computed checksum did NOT match" [High,Triaged]15:20
giulivobnemec, and as you pointed, should that ever change, the usergroup script should be exectued before cleanup15:20
bnemecjp_at_hp: Thanks, I'll take a look.15:25
*** eghobo has joined #tripleo15:38
*** weshay has quit IRC15:39
*** athomas has quit IRC15:42
*** eghobo has quit IRC15:44
*** eguz has joined #tripleo15:44
*** eguz has quit IRC15:44
*** eghobo has joined #tripleo15:44
*** christop1eraedo has quit IRC15:45
*** robsparker has quit IRC15:45
*** jistr has quit IRC15:46
*** jistr has joined #tripleo15:47
*** jcoufal has quit IRC15:49
*** ramishra has quit IRC15:50
*** athomas has joined #tripleo15:52
*** rcarrillocruz has quit IRC15:53
*** rcarrillocruz has joined #tripleo15:58
*** ramishra has joined #tripleo16:00
*** robsparker has joined #tripleo16:00
*** marun_afk is now known as marun16:02
*** IvanBerezovskiy has left #tripleo16:02
*** ramishra has quit IRC16:05
*** weshay has joined #tripleo16:06
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Write log file for each systemd unit  https://review.openstack.org/9794716:11
*** e0ne has quit IRC16:13
*** openstackgerrit has quit IRC16:20
*** openstackgerrit has joined #tripleo16:21
*** weshay has quit IRC16:22
*** sballe has quit IRC16:26
*** jistr has quit IRC16:31
*** rcarrillocruz has quit IRC16:37
*** martyntaylor has left #tripleo16:39
*** weshay has joined #tripleo16:41
*** rcarrillocruz has joined #tripleo16:42
*** andreaf has joined #tripleo16:44
*** sballe has joined #tripleo16:46
*** mrunge has quit IRC16:51
*** akuznetsov has quit IRC16:52
*** akuznetsov has joined #tripleo16:53
openstackgerritA change was merged to openstack-infra/tripleo-ci: Extract upstart logs so they can be loaded into logstash  https://review.openstack.org/9630816:53
*** dprince has joined #tripleo16:54
* dprince just broke the TripleO CI gate16:55
derekh_dprince: ?16:55
dprincederekh_: maybe not :) I just sent Joe's elastic search log branch into tripleo-ci16:56
dprincederekh_: meant to +2 but I +1 approved as well16:56
derekh_dprince: it should be ok, there will just be log files in there top level log directory until this lands https://review.openstack.org/#/c/98043/16:57
derekh_dprince: then they should go to the subdirectory16:57
dprincederekh_: yeah, yours passed (which depended on it) so I think we are fine16:57
derekh_dprince: yup, think so too16:58
derekh_btw, if anybody needs to test something that ubuntu specific they can do a check-experimental16:59
derekh_please don't all rush out and do it at the same time16:59
dprincederekh_: yeah I noticed that yesterday when I ran it on a Nova patch17:01
*** matty_dubs is now known as matty_dubs|lunch17:02
*** eghobo has quit IRC17:02
*** derekh_ has quit IRC17:06
*** ramishra has joined #tripleo17:06
*** lucasagomes is now known as lucas-dinner17:07
*** jang has quit IRC17:08
*** akuznetsov has quit IRC17:10
*** ramishra has quit IRC17:10
*** akrivoka has quit IRC17:12
*** akuznetsov has joined #tripleo17:12
*** rakesh_hs has joined #tripleo17:20
*** sballe has quit IRC17:25
*** BadCub01 has quit IRC17:26
*** noslzzp has quit IRC17:31
*** athomas has quit IRC17:33
*** eghobo has joined #tripleo17:33
*** eghobo has quit IRC17:34
*** eghobo has joined #tripleo17:37
*** e0ne has joined #tripleo17:39
*** ramishra has joined #tripleo17:40
*** vinsh has quit IRC17:43
*** ramishra has quit IRC17:44
*** SlickNik has quit IRC17:45
*** spiffxp_ has quit IRC17:45
*** SlickNik has joined #tripleo17:46
*** pelix has quit IRC17:47
*** athomas has joined #tripleo17:47
*** spiffxp has joined #tripleo17:48
*** eguz has joined #tripleo17:48
*** akuznetsov has quit IRC17:49
*** matty_dubs|lunch is now known as matty_dubs17:49
*** eghobo has quit IRC17:51
*** jtomasek has joined #tripleo17:53
*** marun has quit IRC17:56
*** marun has joined #tripleo17:56
*** akuznetsov has joined #tripleo18:03
openstackgerritDerek Higgins proposed a change to openstack/tripleo-image-elements: Fall back to keystone-manage if pt-archiver isn't available  https://review.openstack.org/9819618:08
*** nati_ueno has joined #tripleo18:08
*** rpodolyaka1 has joined #tripleo18:10
*** eguz has quit IRC18:14
*** eghobo has joined #tripleo18:14
*** nealph has quit IRC18:24
*** jtomasek has quit IRC18:25
*** jang has joined #tripleo18:26
*** andreaf has quit IRC18:30
*** jtomasek has joined #tripleo18:34
*** jtomasek has quit IRC18:40
*** nati_ueno has quit IRC18:40
*** ramishra has joined #tripleo18:41
*** ramishra has quit IRC18:45
lifelessmorning y;all18:45
greghaynesO/18:45
*** vinsh has joined #tripleo18:46
*** jtomasek has joined #tripleo18:46
*** jtomasek has quit IRC18:46
TheJuliaEven though it is the afternoon here, coffee does sound good about now18:48
*** nati_ueno has joined #tripleo18:50
robsparkerneed a core reviewer to review and hopefully merge https://review.openstack.org/#/c/96259/ (tests have passed, at +6, no rebase required)18:52
*** e0ne has quit IRC18:53
*** e0ne has joined #tripleo18:53
*** vinsh_ has joined #tripleo18:53
*** akuznetsov has quit IRC18:56
lifelessTheJulia: how are you settling in?18:56
*** e0ne has quit IRC18:58
TheJuliaokay I guess, It is quite a bit to wrap one's head around, but things are starting to click so its a good sign.  I'm hoping to hit the ground running once I get my laptop.19:01
lifelessthere's a bit to it yeah :)19:02
TheJuliaI've also gotten a few crazy ideas, but I'm going to keep a lid them for now or I'll catch a lot of flack from a former manager and a bunch of friends/former coworkers who have ended up at Amazon RDS.19:06
*** nati_ueno has quit IRC19:07
*** funzo has quit IRC19:09
*** funzo has joined #tripleo19:09
openstackgerritA change was merged to openstack/tripleo-image-elements: Use the more common os_ names for novaclient  https://review.openstack.org/9045319:10
openstackgerritA change was merged to openstack/tripleo-image-elements: Refactor nagios3 element to refresh more often  https://review.openstack.org/9045419:10
*** stevebaker has quit IRC19:15
*** stevebaker has joined #tripleo19:15
*** rpodolyaka1 has quit IRC19:15
*** vinsh_ has quit IRC19:17
lifelessTheJulia: :)19:22
*** vinsh has quit IRC19:22
*** vinsh has joined #tripleo19:24
openstackgerritA change was merged to openstack/tripleo-incubator: Setting up ceilometer SSL  https://review.openstack.org/9625919:24
*** noslzzp has joined #tripleo19:25
*** e0ne has joined #tripleo19:29
lifelessNg: ping19:29
lifelessNg: (ssl certs)19:30
bnemecrobsparker: No problem.  Lucky for you incubator doesn't gate on anything right now. :-)19:31
*** rakesh_hs has quit IRC19:40
*** ramishra has joined #tripleo19:42
*** ramishra has quit IRC19:46
*** noslzzp has quit IRC19:47
adam_glifeless, what size overclouds have you been bringing up in your recent testing?19:48
lifelessadam_g: 3919:49
lifeless40 machines, 1 controller19:50
adam_glifeless, have you hit any scalability issues with heat? im looking at a debian deployment of 30 nodes that seems to cripple heat when all nodes are polling with os-collect-config19:51
*** nati_ueno has joined #tripleo19:51
lifelessadam_g: lots but it was only using 5 or so cores19:53
lifelessadam_g: during the deploy there is a very slow stage; pull in SpamapS's template fix for that19:53
adam_give even seen traces of https://bugs.launchpad.net/tripleo/+bug/118448419:53
uvirtbotLaunchpad bug 1184484 in tripleo "Quantum default settings will cause deadlocks due to overflow of sqlalchemy_pool" [Critical,Fix released]19:53
lifelessNg: specifically c{d,i}-overcloud.hp1.tripleo.org19:54
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Increase the pip default timeout to 60  https://review.openstack.org/9823219:54
adam_glifeless, this one? https://review.openstack.org/#/c/93817/19:55
lifelessadam_g: https://review.openstack.org/#/c/97703/19:57
*** weshay has quit IRC20:04
*** nati_ueno has quit IRC20:08
Nglifeless: both have been ordered and approved, so now we wait for them to be issued20:10
lifelessNg: wicked thanks20:11
*** weshay has joined #tripleo20:12
*** rlandy has quit IRC20:17
*** devananda has quit IRC20:19
*** andreaf has joined #tripleo20:27
*** julim has quit IRC20:34
*** ramishra has joined #tripleo20:42
*** ramishra has quit IRC20:47
*** weshay has quit IRC20:51
clarkbtripleo folks. I am running https://review.openstack.org/#/c/88479/14/tools/build-image.sh which executes dib to build nodepool useable images and it fails with./tmp/image.duevtB5e/hooks/root.d/../environment.d/10-dib-init-system.bash: line 1: /tmp/image.duevtB5e/hooks/root.d/../environment.d/../dib-init-system: Permission denied20:54
clarkbwhere might I start to debug this?20:54
clarkbI am running this as root20:55
*** morazi has quit IRC20:57
*** untriaged-bot has joined #tripleo21:00
untriaged-botUntriaged bugs so far:21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132631221:00
uvirtbotLaunchpad bug 1326312 in ironic "Broken HostManager: TypeError: __init__() takes at most 4 arguments (5 given)" [Critical,In progress]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132576221:00
uvirtbotLaunchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New]21:00
*** untriaged-bot has quit IRC21:00
*** stevebaker has quit IRC21:00
*** stevebaker has joined #tripleo21:00
clarkbI have discovered -x21:02
clarkblooks like dib-init-system is not executable...21:09
clarkbis that a bug?21:09
lifeless$ ls -l  ./elements/base/dib-init-system21:11
lifeless-rwxrwxr-x 1 robertc robertc 314 May 11 08:53 ./elements/base/dib-init-system21:11
lifelessclarkb: pbr breakage posible?21:11
clarkbI pip installed so could be21:11
lifelessso, bug, file on diskimage-builder21:11
*** e0ne has quit IRC21:12
clarkbwill do21:12
clarkbhttps://bugs.launchpad.net/diskimage-builder/+bug/132697321:14
uvirtbotLaunchpad bug 1326973 in diskimage-builder "dib-init-system not executable when dib is installed via pip on ubuntu precise." [Undecided,New]21:14
clarkblifeless: now I have Please include at least one distribution root element. so off to figure that out21:15
clarkbmy command has ubuntu as the first element which I would expect is a distribution root element?21:16
*** andreaf has quit IRC21:17
*** devananda has joined #tripleo21:17
*** cwolferh_ has joined #tripleo21:19
*** nati_ueno has joined #tripleo21:22
lifelessclarkb: you haven't specified an os :)21:22
*** cwolferh has quit IRC21:22
lifelessclarkb: e.g. ubuntu or fedora21:22
*** jang has quit IRC21:27
clarkblifeless: how do I do this?21:28
clarkbdisk-image-create -x -a amd64 --no-tmpfs -o devstack-gate-precise ubuntu vm openstack-repos puppet node-devstack fix-bad-clouds is my command21:28
clarkbwith DIB_RELEASE set to precise and DISTRO set to ubuntu21:28
openstackgerritChris Jones proposed a change to openstack/tripleo-incubator: Allow user-specified timeout for stack creation  https://review.openstack.org/9596821:30
lifelessclarkb: the 'ubuntu' element there does it21:31
lifelessclarkb: so something is going wrong, and should be visible in the output preceeding that error21:32
lifelessclarkb: alternatively, it may be a packaging issue with new metadata files not included/installed correctly, I guess.21:32
clarkbhttp://paste.openstack.org/show/83004/21:33
lifelessI rather think the distribution root element stuff is an antifeature, but I'm very minimal on my stuff21:33
lifelessah nope, this isn't the metadata stuff21:34
lifelessthis is     if [ -z "$(ls $TMP_MOUNT_PATH | grep -v '^lost+found\|tmp$')" ] ; then21:34
lifelessclarkb: whats in the ubuntu element tree on disk ?21:34
lifelessclarkb: oh, I knw21:34
lifelessclarkb: how did you fix the dib-init-system thing ?21:35
lifelessI'm going to bet you chmodded that one file21:35
clarkbhttp://paste.openstack.org/show/83006/21:35
clarkblifeless: yes I chmodded the one file21:35
lifelessthe problem is systemic, and that won't work21:36
clarkbI probably need to recursive chmod everything in /usr/local/share/disk* ?21:36
lifelessyou need to fix the problem, because there are hundreds of files some which need to be executable and others that shouldn't be21:36
lifelessclarkb: its pbr-ing time :)21:36
clarkbheh21:37
*** nati_uen_ has joined #tripleo21:39
*** nati_ueno has quit IRC21:39
*** ccrouch has quit IRC21:41
*** ramishra has joined #tripleo21:43
lifelessnuts21:43
lifelessI've lost an etherpad21:43
lifelessI had one with all sorts of deployment hints for the ci overcloud21:44
lifelessand now I can't find it21:44
*** ramishra has quit IRC21:44
*** ramishra has joined #tripleo21:45
clarkbwhoops21:49
clarkbso I can confirm the file in the dib sdist is +x21:49
clarkbso that half of the process lgtm21:49
*** ramishra has quit IRC21:50
clarkbbut pip installing that sdist removes the +x21:50
lifeless'yay'21:51
clarkbhuh upgrading to latest pip fixes it21:51
clarkbI blame pip21:51
clarkband now dib does stuff21:51
clarkblifeless: so I think we are no longer compatible with pip 1.021:51
lifelessphew, found https://etherpad.openstack.org/p/tripleo-test-cluster21:51
*** ccrouch has joined #tripleo21:52
*** jerryz has quit IRC21:57
lifelessSpamapS: we've got three machines that have their link on eth322:01
lifelessfiling tickets now22:02
*** stevebaker has quit IRC22:03
*** stevebaker has joined #tripleo22:03
clarkblifeless: is there a trick with dib in mapping the files in the tmpdir to the actual files?22:03
clarkbI have been using find on /usr/local/share/disk*22:04
clarkbwhich seems clunky22:04
lifelessclarkb: the tmpdir is compiled out of the elements included in the build22:04
clarkbright but the prefix is removed so you end up with a flat hooks for example22:05
clarkband I have no idea which element applied hook 98-foo22:05
lifelessright22:05
lifelessyes find22:05
*** devananda has quit IRC22:05
*** devananda has joined #tripleo22:06
*** devananda has quit IRC22:07
lifelesspoor p.o.o, thou hast too many 500s22:09
lifeless http://paste.openstack.org/show/62631/ specifically22:09
* clarkb puts money on trove connection dying22:10
clarkbworks now22:10
openstackgerritA change was merged to openstack/tripleo-image-elements: Make the image name configurable  https://review.openstack.org/9579122:11
openstackgerritBen Nemec proposed a change to openstack/tripleo-specs: TripleO on OpenStack  https://review.openstack.org/9264222:21
clarkblifeless: quick opinion time. dib needs root. we want to use dib with nodepool. Should nodepool run dib on the dib server thus requiring the nodepool service have enough access to run dib as root?22:24
clarkbI almost want to run it on a remote server22:24
lifelessdib on the dib server?22:24
lifelessclarkb: ^ dib on the dib server?22:25
clarkbsorry dib on the nodepool server22:25
lifelessdepends on your threat model22:25
lifelessfor dib, building images to run the cloud, we take the view that if you can't trust the os vendor & openstack code, you're already dead.22:26
clarkbits not the os vendor and openstack code and more not trusting nodepool22:26
lifelessyou could rootwrap it22:26
lifelessallow nodepool to run dib and only dib via sudo root wrap ...22:27
lifelesshaving a dib service machine is fine too22:27
clarkbya that would work22:27
lifelessmessage bus between for the win22:27
lifelessor transient image-builder machines22:27
clarkbI am mostly wrapping my head around what all of this entails as I dig into it properly22:27
lifelessack22:27
lifelesskeep asking ;)22:28
*** stevebaker has quit IRC22:28
*** stevebaker has joined #tripleo22:28
jogois Derek Higgins around?22:28
lifelessnot atm22:28
lifeless1130pm for him I think22:29
jogoI am gonna fix his e-r patch https://review.openstack.org/#/c/98085/22:29
jogoit should be the first tripleo patch to land :)22:29
lifelesscool22:29
lifelesswhhheeee ironic can't show ports for one node atm22:33
lifelessouch22:33
*** matty_dubs is now known as matty_dubs|gone22:36
jogolifeless: https://review.openstack.org/#/c/98085/ is going in22:37
lifelesscool22:39
lifelessthats the second tripleo query btw :)22:39
*** devananda has joined #tripleo22:45
*** devananda has quit IRC22:45
*** ramishra has joined #tripleo22:45
*** athomas has quit IRC22:47
*** devananda has joined #tripleo22:48
*** ramishra has quit IRC22:50
*** vinsh has quit IRC22:59
clarkblifeless: are dib scripts implicitly set -e? or do we have control over that? I have a puppet run that failed but dib kept going23:13
clarkband wondering if I need to make it fail fast for my sanity23:14
SpamapSclarkb: you have to set it yourself23:16
SpamapSclarkb: dib-lint will try to remind you23:16
clarkbSpamapS: thanks23:16
lifelessclarkb: As SpamapS says23:20
lifelessSpamapS: three of our machines have twox10G plugged in23:20
lifelessSpamapS: we either had ethN shufflin around or just the wrong mac23:20
lifelesstwo sorted, testing the third23:20
*** noslzzp has joined #tripleo23:21
*** spiffxp has quit IRC23:28
*** SlickNik has quit IRC23:28
*** SlickNik has joined #tripleo23:29
*** SlickNik has quit IRC23:30
*** giulivo has quit IRC23:34
lifelessrighty ho23:35
lifelesstime to try a full scale cloud deploy again23:35
*** ccrouch has quit IRC23:36
*** ramishra has joined #tripleo23:46
*** ccrouch has joined #tripleo23:48
clarkbalright puppet in dib fun. Our puppet manifests restart iptables and rsyslog on slaves to pick up new configs. This is fine on real servers it just works. But in dib it fails because you can't restart services. mordred seems to have worked around this by ignoring errors from puppet. This scares me because any other puppet failures should result in a discarded image build. Anything I can do here in dib land23:48
clarkbto possibly make those service restarts "happen"23:48
lifelessso the problem is that you're not in a real server23:49
lifelessyou're not even deployed23:49
*** dprince has quit IRC23:49
lifelessso things like - to pick an example - moving jenkins config files around on /mnt (ephemeral partition) are simply not applicable23:50
lifelessI agree with your fear; puppet must not do these things when preparing an image.23:50
*** ramishra has quit IRC23:50
clarkbhrm23:52
clarkbso part of me thinks dib should run in a vm now :P23:52
lifelessit wouldn't affect this aspect23:52
lifelessyou still wouldn't be in your final resting place23:52
clarkbwhy not?23:52
clarkball I need is for puppet to restart a service23:52
lifelesstake my ephemeral example again23:52
clarkband get a 0 back23:52
clarkbwe don't do anything with ephemeral partitions here23:52
lifelessthe ephemeral you'll have in  temporary vm isn't the one you get when deployed23:53
clarkbwhich is fine because we never do that23:53
lifelessclarkb: sure, but your fear is that mordreds workaround will mask other failures from puppet23:53
clarkbyes23:53
lifelesswhich is a case of a specific problem in a generic context23:53
lifelessI thought we nobbled the standard interface for starting services during the install23:54
lifelessper debian policy,a nd similarly on fedora23:54
lifelessso puppet is not obeying that, FAICT.23:54
clarkbyou probably did so when puppet says start it fails23:54
*** SlickNik has joined #tripleo23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!