Tuesday, 2016-01-26

« Monday, 2016-01-25 Index Wednesday, 2016-01-27 »
*** jtomasek has quit IRC00:14
*** chlong has quit IRC00:20
*** chlong has joined #tripleo00:20
*** jtomasek has joined #tripleo00:27
*** chlong has quit IRC00:41
*** chlong has joined #tripleo00:41
*** david-lyle has joined #tripleo00:42
*** chlong has quit IRC00:48
*** chlong has joined #tripleo00:48
*** chlong has quit IRC00:53
*** chlong has joined #tripleo00:53
*** mkovacik has quit IRC01:09
*** yuanying has joined #tripleo01:10
*** yuanying_ has quit IRC01:11
*** chlong has quit IRC01:55
*** chlong has joined #tripleo01:55
*** yuanying has quit IRC02:00
*** tiswanso has joined #tripleo02:02
*** sthillma has quit IRC02:03
*** yuanying has joined #tripleo02:03
*** david-lyle has quit IRC02:05
*** tiswanso has quit IRC02:06
*** tiswanso has joined #tripleo02:07
*** trozet has joined #tripleo02:12
*** davidlenwell has quit IRC02:14
*** davidlenwell has joined #tripleo02:16
*** ChanServ sets mode: +v davidlenwell02:16
*** shivrao has quit IRC02:19
*** yuanying has quit IRC02:25
*** yuanying has joined #tripleo02:27
*** chlong has quit IRC02:49
*** chlong has joined #tripleo02:49
*** superflyy has joined #tripleo02:50
*** pradk has quit IRC03:02
*** sbalukoff has quit IRC03:02
*** pradk has joined #tripleo03:16
*** chlong has quit IRC03:17
*** chlong has joined #tripleo03:18
*** chlong has quit IRC03:19
*** yuanying_ has joined #tripleo03:19
*** chlong has joined #tripleo03:19
*** yuanyin__ has joined #tripleo03:20
*** yuanying has quit IRC03:22
*** yuanying_ has quit IRC03:23
*** david-lyle has joined #tripleo03:25
*** yuanyin__ has quit IRC03:29
*** lazy_prince has joined #tripleo03:29
*** yuanying has joined #tripleo03:29
*** bnemec has quit IRC03:34
*** bnemec has joined #tripleo03:38
*** thrash is now known as thrash|g0ne03:39
openstackgerritSteve Baker proposed openstack/python-tripleoclient: openstack overcloud software deployment show  https://review.openstack.org/27111303:42
openstackgerritSteve Baker proposed openstack/python-tripleoclient: openstack overcloud failures  https://review.openstack.org/27111403:42
openstackgerritSteve Baker proposed openstack/python-tripleoclient: openstack overcloud software deployment list  https://review.openstack.org/25267003:42
*** yamahata has quit IRC03:43
prometheanfirewhat do you think about an element to specificly install cloud-init?03:46
prometheanfireI'd rather not have to do so in the gentoo element because it'll be inherited even if not wanted03:46
*** tzumainn has quit IRC03:47
*** yuanying has quit IRC03:57
*** yuanying has joined #tripleo03:58
*** superflyy has quit IRC04:01
*** yuanying has quit IRC04:02
*** yuanying has joined #tripleo04:08
prometheanfirewell, made one, it only installs / sets it up for gentoo atm, but the format is simple, check if DISTRO_NAME (or another identifier) is something, then set up init or whatever04:08
prometheanfireand it installs cloud-init via package-installs04:08
*** trozet has quit IRC04:21
*** sbalukoff has joined #tripleo04:22
*** lazy_prince has quit IRC04:29
*** rlandy has quit IRC04:29
*** tiswanso has quit IRC04:44
*** pradk has quit IRC04:56
*** yamahata has joined #tripleo04:56
*** cody-somerville has quit IRC05:00
*** ayoung is now known as ayoung_ZZZzzzz05:01
*** pradk has joined #tripleo05:08
prometheanfirebtw, here's my current diff...05:10
prometheanfirehttp://paste.openstack.org/show/484952/05:10
prometheanfiregood think for you is I think that's all the stuff needed in dib, the other changes to the elements are in nodepool05:11
*** masco has joined #tripleo05:18
*** shivrao has joined #tripleo05:26
*** masco has quit IRC05:27
*** cmyster has quit IRC05:32
*** cmyster has joined #tripleo05:33
*** cmyster has quit IRC05:33
*** cmyster has joined #tripleo05:33
*** masco has joined #tripleo05:38
*** dshulyak has joined #tripleo05:54
*** masco has quit IRC06:02
prometheanfire.06:14
prometheanfireis there a way for a user to pass in a package-installs.yml or the like?06:14
*** panda_ has quit IRC06:25
*** panda_ has joined #tripleo06:26
*** absubram has joined #tripleo06:40
*** absubram_ has joined #tripleo06:41
*** rbrady has quit IRC06:43
*** ukalifon1 has joined #tripleo06:44
*** absubram has quit IRC06:44
*** absubram_ is now known as absubram06:44
*** rbrady has joined #tripleo06:49
*** rcernin has joined #tripleo06:53
cmysterprometheanfire: like in deploy --templates -e package-installs.yml ?06:58
prometheanfireoh, I mean for dib06:59
*** absubram has quit IRC07:01
*** oshvartz has joined #tripleo07:11
*** cody-somerville has joined #tripleo07:19
*** cody-somerville has joined #tripleo07:19
*** nijaba has quit IRC07:27
*** nijaba has joined #tripleo07:27
*** jprovazn has joined #tripleo07:47
*** liverpooler has joined #tripleo07:54
*** rwsu has quit IRC07:58
*** jhenner has quit IRC08:02
*** jhenner has joined #tripleo08:02
*** hjensas has quit IRC08:03
*** fgimenez has joined #tripleo08:06
*** shivrao has quit IRC08:09
*** jhenner has quit IRC08:12
*** gfidente has joined #tripleo08:14
*** cody-somerville has quit IRC08:14
*** sthillma has joined #tripleo08:14
openstackgerritxin wu proposed openstack/python-tripleoclient: Install bigswitch networking agent by default  https://review.openstack.org/27199008:16
*** sthillma_ has joined #tripleo08:17
*** devvesa has joined #tripleo08:18
*** sthillma has quit IRC08:19
*** sthillma_ is now known as sthillma08:19
*** shardy has joined #tripleo08:24
*** jcoufal has joined #tripleo08:26
*** ifarkas has joined #tripleo08:30
*** jhenner has joined #tripleo08:33
*** jaosorior has joined #tripleo08:40
*** mcornea has joined #tripleo08:50
*** sthillma has quit IRC08:54
*** aufi has joined #tripleo08:55
*** derekh has joined #tripleo08:56
openstackgerritMarios Andreou proposed openstack/tripleo-common: Adds a simple overcloud tenant vm ping test to tripleo.sh  https://review.openstack.org/27219108:57
jaosoriormarios: Was the only thing added the cherry-picked-from ^^ ?08:58
mariosjaosorior: yeah commented now. didn't think/expect it to kill the +2s for that tho :/08:59
*** bvandenh has joined #tripleo08:59
jaosoriormarios: Didn't expect that either, bummer08:59
jaosoriormarios: Though... it already had two +2, you might as well merge it09:00
mariosjaosorior: ci09:00
mariosjaosorior: plus, i woulnd't feel comfortable. i mean, i just uploaded the patch. it is one thing to +2 it (really the unwritten rule is you don't +2 you own reviews, or it doesn't count anyway) but wouldn't want to be the one to merge it, unless emergency09:01
mariosjaosorior: in this case, bnemec made the cherrypick, so I think is ok to +2 it ('yeah this looks like a good cherrypick from the master')09:02
mariosjaosorior: ("uploaded the patch" i mean revision 2)09:02
jaosoriormarios: You uploaded a patch that changes the commit message, so I didn't think it would count09:02
mariosjaosorior: yeah you're right, i mean there were no code changes. but still I wouldn't want to merge unless it was really necessary/no-one else about09:04
derekhmarios: +2 from me also09:04
*** mkovacik has joined #tripleo09:05
mariosthx derekh o/ morning09:05
*** hjensas has joined #tripleo09:06
derekhmarios: morning, I think we're good to merge this, the ping test worked for the nonha test https://jenkins07.openstack.org/job/gate-tripleo-ci-f22-nonha/435/consoleText09:07
derekhmarios: going to see why the other 2 failed and merge if its unrelated09:07
mariosderekh: thanks, yeah lgtm too /me looks at that console09:07
mariosderekh: is a nice thing to see in the output :)09:08
marioshttps://jenkins07.openstack.org/job/gate-tripleo-ci-f22-ha/252/console still running09:10
derekhmarios: yup, it is, ok the ping test failed for the ceph test, but thats a problem with the cloud so I think we can still mere09:11
derekhmarios: thats for PS2 I'm looking at the results of PS109:12
*** lucas-dinner is now known as lucasagomes09:12
mariosderekh: what is the 'ceph test' is there a bug #? or you're referring generally to current ha ci being broken?09:12
mariosderekh: yeah realised since was still running (we may as well wait for it no? is there an emergency?)09:13
mariosjaosorior: ^^09:13
derekhmarios: need to look at the reason it failed, no bug yey that I know of09:13
*** fgimenez has quit IRC09:13
derekhmarios: ok, lets wait, no emergency09:14
derekhschool run, bbiab09:14
jaosoriormarios: Wasn't it the case that stable/liberty CI is broken? And that CR was an attempt to fix it?09:14
mariosderekh: o/ in a bit09:14
mariosjaosorior: i don't know or missed that conversation. i know ci was broken for a couple of reasons last few days but they were fixed. I didn't realise the ping test fixed something there09:15
jaosoriormarios: IIRC stable/ci tries to run the ping test, and the test hadn't been packported yet09:15
*** fgimenez has joined #tripleo09:16
*** fgimenez has joined #tripleo09:16
*** dtantsur|afk is now known as dtantsur09:16
mariosjaosorior: i see that makes sense and thanks for the info ... hmm i guess since landed in toci_instack now (for ci)09:17
mariosjaosorior: but this test adds it, so ci run for it should be ok if that was the problem09:17
mariosjaosorior: s/test/review09:17
marios(pingtest on the brain)09:18
mariosjaosorior: I mean v1 passed the non-ha like at https://jenkins07.openstack.org/job/gate-tripleo-ci-f22-nonha/435/consoleText09:18
mariosjaosorior: let's see what happens with v2 and we merge? is that reasonable?09:18
*** jistr has joined #tripleo09:19
jaosoriormarios: Sure man, no worries09:19
jistrmorning09:21
jaosoriorjistr: Sup dude09:21
*** mcornea has quit IRC09:24
*** nico_auv has joined #tripleo09:26
*** electrofelix has joined #tripleo09:28
*** olap has joined #tripleo09:28
*** akrivoka has joined #tripleo09:38
*** dtantsur is now known as dtantsur|brb09:48
*** olap has quit IRC09:49
*** olap has joined #tripleo09:51
shardyhttps://review.openstack.org/#/c/272194/ combined with https://review.openstack.org/#/c/272191/ was supposed to fix stable CI09:52
shardyI don't think the latter with pass without the former09:52
shardyalthough 272191 still isn't passing the HA job, not yet looked into why09:53
shardymarios: ^^ FYI09:53
jistrtried to look a while ago but CI logs are unavailable09:55
jistrsome infra problem, they're talking about it on #openstack-infra09:55
shardyYeah I saw the -dev mail about unstable jobs09:56
shardywow we've had a rough time with CI lately :(09:56
jaosoriorbummer09:57
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: [NO MERGY] Test a update in trunk repository version  https://review.openstack.org/22978909:58
derekhjistr: is your looking for ci logs you can get the console log directly from jenkins , find the jobs you looking for here http://tripleo.org/cistatus.html10:00
derekhjistr: then clock on the date10:00
derekh*click10:00
jistrderekh: cool, thanks!10:01
*** dshulyak has quit IRC10:01
derekhof course that doesn't help you find the other logs but at least its something10:02
*** dshulyak has joined #tripleo10:02
*** mkovacik has quit IRC10:04
jistrhttp://fpaste.org/314802/53802605/10:05
jistrpingtest failing -- when checking for the tenant stack creation, it got 503 from heat?10:05
mariosshardy: jistr thanks looking again now (v1 was passing, was waiting for v2 at https://jenkins07.openstack.org/job/gate-tripleo-ci-f22-ha/252/10:14
marioswell passing the non-ha afaik/saw10:16
jistrmarios: we probably need to recheck this one though, as it should fix HA and depends on the pingtest change: https://review.openstack.org/#/c/272194/ but for now openstack-infra advises not to recheck anything as they expect failures (see e-mail [openstack-dev] [all] UNSTABLE jobs - Jenkins failure - do not recheck/approve)10:18
mariosjistr: ack o/ morning10:19
jistrmorning :)10:19
*** rebrego has joined #tripleo10:22
*** regebro is now known as Guest8092710:22
*** rebrego is now known as regebro10:22
*** jaosorior has quit IRC10:23
mariosjistr: sorry just re-read. this one is from the pingtest i think (i.e. https://jenkins07.openstack.org/job/gate-tripleo-ci-f22-ha/252/console)10:23
*** jaosorior has joined #tripleo10:24
jistrmarios: yea but IIUC it cannot pass, as there's another bug which causes the HA job to fail (see the fix i linked above and its depends-on)10:25
*** athomas has quit IRC10:25
*** panda_ has quit IRC10:25
mariosjistr: ah ok thanks (yeah i am looking through the console output now). i was confused by the fpaste showing the pingtest fail (or does the pingtest run regardless of overcloud being deployed? wont we exit by then?)10:26
*** panda_ has joined #tripleo10:26
marios*try* to run and fail like you showed i mean10:26
*** paramite has joined #tripleo10:27
jistrmarios: it tried to run the pingtest and failed on the HA change. That job had (at least i hope so :) ) both fixes present (the mysql HA fix and backport of pingtest to tripleo.sh). Not sure why it got 503...10:28
jistrmarios: the fpaste is from the latest run on https://review.openstack.org/#/c/272194/10:28
mariosjistr: thx looking at http://logs.openstack.org/94/272194/2/check-tripleo/gate-tripleo-ci-f22-ha/21858d0/console.html10:29
marios(and see it now)10:29
mariosjistr: well, it makes sense if 'there is no overcloud heat'10:30
mariosjistr: i mean ERROR: <html><body><h1>503 Service Unavailable</h1>10:30
*** athomas has joined #tripleo10:30
jistrjust saw an e-mail to dev list re infra -- CI should be back to normal10:31
shardymust pass controller_virtual_ip to Class[Tripleo::Loadbalancer]10:32
shardyanyone seeing that error recently?10:33
mariosshardy: that rings a bell... thinking10:33
* shardy is failing to figure out what he's failed to update10:33
shardyI pulled latest t-h-t yesterday and rebuilt my images, but getting that since10:33
mariosshardy: ah gfidente from yesterday ^^^10:33
mariosshardy: is this like 7.1/7.2 update10:33
marios?10:33
shardymarios: no, it's upstream tripleo, I just pulled the latest templates etc10:34
shardywas working before that so I assume something changed, I'm just not yet sure what :)10:34
*** fgimenez has quit IRC10:34
gfidenteshardy, I noticed the ControlVirtualIP resource doesn't emit any IP if the underlying neutron port is deleted10:35
mariosshardy: ah ok  we saw it yesterday cos the ControlFixedIPs: was passed in param_defaults vs params10:35
*** paramite is now known as paramite|afk10:35
gfidenteshardy, in which case I got that very same error10:35
mariosgfidente: there was a bz i believe?10:35
gfidenteshardy, and I worked around it using replacement_policy: REPLACE_ALWAYS10:35
gfidenteso that it would recreate the neutron port10:35
gfidentebut I am not sure if/how the neutron port could get deleted in your case10:35
shardyYeah it's definitely not set in vip_data.yaml on the node10:36
shardygfidente: this is on a fresh stack create10:36
gfidentemarios, in the BZ they deleted the neutron port manually10:36
*** fgimenez has joined #tripleo10:36
*** fgimenez has joined #tripleo10:36
*** mcornea has joined #tripleo10:37
gfidenteshardy, oh I thought it was on upgrade ... so you see the actual ip_address with resource-show on ControlVirtualIP ?10:37
shardyYeah control_virtual_ip has no IPs, it's CREATE_COMPLETE, but the status is DOWN10:42
shardyI guess that explains it10:42
derekhLooking at PS2 ci results for the pingtest (in liberty), NONHA the ping didn't work (but it did try to ping the instance), in the Ha test the overcloud failed to deploy. In PS1 we have a successful ping, I think we have enough to show that the actaul ping test works. The problems are with the actaully deployment being tested. I'm thinking we just merge it to make deciphering these failures easier.10:43
derekhanybody disagree ? ^^10:43
*** tosky has joined #tripleo10:43
gfidenteshardy, not sure I have it in DOWN state too but works10:43
shardyderekh: +110:43
shardygfidente: Hmm, so it's DOWN, but fixed_ips isn't empty?10:43
mariosderekh: sounds good to me10:44
*** yamahata has quit IRC10:44
gfidenteshardy, not empty no10:44
shardyfixed_ip's for both control_virtual_ip and redis_virtual_ip are empty for me10:44
shardyhrm10:44
gfidentethat's exactly what I saw10:44
gfidentewhen the port is empty, the resource is empy heat can't get the actual ip_address10:44
derekhshardy: marios done10:44
gfidentebut in the BZ marios mentioned, people deleted their neutron port10:44
gfidenteand we recreated it using the replacement_policy10:45
gfidentewhy it is empty in your case, I dunno10:45
shardyI wonder if it's failing to delete the portws10:45
* shardy tries stack-delete again10:45
gfidenteshardy, I have also another strange thing here10:45
gfidentebasically I seem to have a scenario which consistely reproduced the Timed out waiting for message problem10:45
gfidenteregardless of the number of cores and memory, I have enough10:46
shardygfidente: that's strange, I've only ever seen it with insufficient engines, or when one gets killed10:46
openstackgerritMerged openstack/tripleo-common: Adds a simple overcloud tenant vm ping test to tripleo.sh  https://review.openstack.org/27219110:46
gfidenteyeah I am trying to understand what is triggering it10:46
cmystercan it be a missing signaling on failure to do something ?10:47
gfidenteI always get the timeout on the same resource10:47
gfidenteor better, when trying to update a specific resource10:47
* cmyster hmmms10:47
shardygfidente: to clarify, you mean an RPC error in the logs right?10:47
shardytimed out waiting for message... etc10:47
cmystersounds too familiar10:47
cmystergfidente: is there an open bug here with steps ?10:48
shardycmyster: it's different from the signalling I think10:48
gfidenteshardy, yeah rpc/client.py10:48
cmystershardy: I thought so, but this error message is familiar10:48
gfidentebut it says 'NotFound' on delete10:48
cmysteryup10:48
cmysterseen it on working on upgrading tests between different kilo versions10:49
cmysterbut I don't remember what I did in that case and it was sporadic and rare in my case10:49
shardycmyster: IIRC the downstream bugs with this error were fixed by increasing the undercloud resources10:50
shardyfolks were DoSing themselves by running a single heat-engine on a single-core VM, and/or running without enough RAM and no swap10:50
gfidenteyeah the interesting thing for me here is that I can reproduce it consistently10:50
shardygfidente: can you raise a bug with the steps?10:50
shardythe NotFound thing sounds like a good clue to investigate10:51
cmystergfidente: and please add how many cpus/ram10:51
gfidenteshardy, cmyster here is a trace10:51
gfidentehttp://paste.openstack.org/show/484977/10:51
gfidente# free -m10:51
gfidente              total        used        free      shared  buff/cache   available10:51
gfidenteMem:           7823        4507        1920         390        1395        266010:51
gfidenteSwap:             0           0           010:51
gfidente^^ that's the undercloud free10:51
gfidenteand it has 2 cores10:52
gfidenteI will try increasing the workers10:52
gfidentejust to see if that makes any difference10:52
cmystergfidente: its not enough...10:54
*** dtantsur|brb is now known as dtantsur11:00
gfidentecmyster, what is not enough? 2 cores?11:03
*** chlong has quit IRC11:05
cmysterback11:09
shardygfidente: It should be enough, but if we're still pinned to an old heat, you may want to bump the workers to 411:09
shardygfidente: and add some swap, just in case:11:09
*** chlong has joined #tripleo11:10
cmystergfidente: 8gb RAM is a limit11:10
shardyhttps://github.com/openstack-infra/tripleo-ci/blob/master/toci_instack.sh#L19811:10
gfidenteshardy, yeah I am trying with 6 workers now11:10
shardyI'm running a 6G undercloud with some swap, and it mostly doesn't need it11:10
shardyalthough I mostly do 2 node nonha deployments11:10
cmystergfidente: FYI every time I cook images nowadays I use a guest image and add 2gb swap as a file in /11:10
gfidenteshardy, so any chance this might have more to do with parallelism rather than memory?11:11
shardygfidente: Yes, that was the main issue11:11
shardygfidente: 1 worker is definitely not enough, 4 or more should be11:11
shardyotherwise you throw like 100 stacks at a single engine and it just falls over11:11
gfidenteshardy, ok so let me see how it goes with 6 workers, cause cpu average remained pretty low too ~0.611:11
gfidentebut I had 3 workers11:11
shardygfidente: If you had 3 heat-engine processes e.g via ps ax | grep heat-engine, that is only two workers11:12
shardythe parent process doesn't actually do anything11:12
gfidenteack, I had 2, one per core11:12
gfidente3 processes11:12
cmysterlegit11:12
*** mbound has joined #tripleo11:13
gfidenteso I set it to 6 in the config, I want to see if it passes and if maybe we 'consume' the cpu better with such a configuration11:13
shardyYeah, I've not tested that, it may not be enough - the heat default is now a minimum of 411:13
shardygfidente: it's more that the Linux scheduler does much better multi-tasking than the one inside heat11:13
gfidenteshardy, yeah so for the undercloud we might want to use a higher default regardless of the cores count11:14
gfidentelet's see if it passes first though11:14
shardygfidente: The heat default is now a minimum of 4, the problem is we're still pinned to a really old heat11:15
shardyI'm running heat-master locally FWIW and it works fine11:15
gfidenteshardy, I was scaling a 1ctrl+1cmp+1ceph to +1ceph11:15
gfidentebut interestingly scaling to +1cmp worked11:16
gfidenteas if the type of work needed to scale the ceph node was heavier11:17
gfidentefwiw scaling ceph requires an update of a resource which is shared with the controllers too11:17
gfidentewhile scaling computes doesn't afaik11:17
*** regebro has quit IRC11:26
openstackgerritDerek Higgins proposed openstack/tripleo-common: split overcloud setup out of pingtest  https://review.openstack.org/27063111:26
*** regebro has joined #tripleo11:31
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: [NO MERGY] Test a update in trunk repository version  https://review.openstack.org/22978911:34
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Switch the overcloud pingtest to use the new heat client  https://review.openstack.org/27247911:34
*** mgould has joined #tripleo11:34
*** fgimenez has quit IRC11:44
*** paramite|afk is now known as paramite11:55
*** fgimenez has joined #tripleo11:56
*** fgimenez has quit IRC11:56
*** fgimenez has joined #tripleo11:56
*** paramite is now known as paramite|afk12:00
*** paramite|afk is now known as paramite12:02
*** Marga_ has quit IRC12:03
*** Marga_ has joined #tripleo12:03
*** jaosorior has quit IRC12:12
*** jaosorior has joined #tripleo12:13
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Nothing to see here  https://review.openstack.org/11101112:35
*** weshay has quit IRC12:35
*** regebro has quit IRC12:39
*** regebro has joined #tripleo12:39
*** trellooobot has joined #tripleo12:41
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION12:41
trellooobot+------------------------------+-------------------------------+-------------------+-------------+12:41
trellooobot| Title                        |              URL              | Members           | Last Active |12:41
trellooobot+------------------------------+-------------------------------+-------------------+-------------+12:41
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |    0 min    |12:41
trellooobot+------------------------------+-------------------------------+-------------------+-------------+12:41
*** trellooobot has quit IRC12:41
*** jhenner1 has joined #tripleo12:47
*** jhenner has quit IRC12:50
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Support the deployment of Ceph over IPv6  https://review.openstack.org/27208912:51
*** rhallisey has joined #tripleo12:51
*** pradk has quit IRC12:52
*** jhenner1 has quit IRC12:53
mgouldhi everyone12:57
*** akrivoka has quit IRC12:58
mgouldhttps://review.openstack.org/#/c/270869/ includes a change to a Jenkins script and was merged at 1900UTC last night, but a build run at 1141UTC this morning was still using the old code: http://logs.openstack.org/36/265336/4/check/check-osc-plugins/a7b06f6/12:59
mgouldwill that sort itself out eventually, or is there some process to update the code run by the Jenkins workers?12:59
*** dprince has joined #tripleo12:59
*** trown|outttypeww is now known as trown13:04
*** lucasagomes is now known as lucas-hungry13:07
*** trellooobot has joined #tripleo13:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION13:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+13:10
trellooobot| Title                        |              URL              | Members           | Last Active |13:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+13:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |    29 min   |13:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+13:10
*** trellooobot has quit IRC13:10
*** jayg|g0n3 is now known as jayg13:14
*** weshay_xchat has joined #tripleo13:19
d0ugalmgould: I think you linked to the wrong patch13:21
*** jhenner has joined #tripleo13:22
*** aufi has quit IRC13:22
*** weshay_xchat is now known as weshay13:27
*** larsks has quit IRC13:27
*** akrivoka has joined #tripleo13:32
gfidenteshardy, fwiw, 6 workers didn't make the trick here, timed out at the time of updating the very same resources13:33
gfidenteshardy, but with a different trace, no kidding13:35
gfidentehttp://paste.openstack.org/show/484999/13:35
gfidentebut hey cpu average is much higher, like 2.1613:38
*** fgimenez has quit IRC13:39
*** Goneri has joined #tripleo13:40
*** fgimenez has joined #tripleo13:41
*** fgimenez has quit IRC13:41
*** fgimenez has joined #tripleo13:41
shardygfidente: check the logs and ensure a heat-engine worker didn't get killed due to OOM13:42
shardythat looks like it timed out trying to update a nested stack13:42
shardyand/or add some swap and see if it still happens :)13:43
gfidenteno still have 7 processes13:44
gfidenteand plenty of ram13:44
gfidentehttp://paste.openstack.org/show/485000/13:44
shardyhmm, strange13:45
shardyIs there any engine backtrace just after the update of the nested stack happens?13:46
shardysomething must be going wrong handling the stack_resource update (which happens via RPC)13:46
mariosjistr /me palmface with the brno nordic themed meeting room names (who was it that thought this would be memorable/useful/fun?13:48
mariosjistr: heh, though i guess it *is* memorable if you know what they mean :)13:49
*** julim has joined #tripleo13:50
mariosjistr: ddg tells me iceland13:50
jistrmarios: it was selection between proposals by a democratic vote :D (The participation wasn't too high IIRC, and people from all buildings voted, not just the ones who would be actually based in FBC2, which explains the result a bit :D)13:53
mariosjistr: i can only imagine the memolist flamefest fallout from an event like that13:54
jistryea :D13:55
*** tzumainn has joined #tripleo14:00
dprincemeeting time14:01
*** eggmaster has joined #tripleo14:01
derekhtrown: we got half a meatloaf song https://review.openstack.org/#/c/229789/14:02
*** mkovacik has joined #tripleo14:02
derekhtrown: one out of three aint bad14:02
trownderekh: lol14:02
*** lucas-hungry is now known as lucasagomes14:06
*** trellooobot has joined #tripleo14:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION14:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+14:10
trellooobot| Title                        |              URL              | Members           | Last Active |14:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+14:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |    89 min   |14:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+14:10
*** trellooobot has quit IRC14:10
*** dmacpher has quit IRC14:11
*** tiswanso has joined #tripleo14:13
*** rlandy has joined #tripleo14:14
gfidenteshardy, so yes there is an error14:16
*** pblaho has quit IRC14:17
gfidentebut is logged right after stack is created14:17
gfidentenot during update14:17
gfidentehttp://paste.openstack.org/show/485008/14:17
gfidenteso in despite that error, stack creation is reported as COMPLETE14:18
*** pblaho has joined #tripleo14:18
*** lblanchard has joined #tripleo14:19
shardygfidente: aha!14:20
shardyI bet that truncation is happening on the resource that gets stuck on update ;)14:21
gfidenteshardy, I am trying to understand that14:21
shardygfidente: can you work out what resource type it is from the lines before the error?14:21
mgouldd0ugal, I don't think so14:21
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Introduce update/upgrade workflow  https://review.openstack.org/27135814:21
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Split pacemaker common check_service function out of _restart.sh  https://review.openstack.org/26044314:21
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Add resources for major upgrade in Pacemaker scenario  https://review.openstack.org/25327614:21
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Use timeout to check for services status  https://review.openstack.org/25957314:21
* shardy wonders about a huge manifest in a SoftwareConfig14:21
jistr^ just rebases14:22
gfidenteshardy, that is it indeed14:22
mgouldd0ugal, I think those links are correct14:22
shardygfidente: Ah, I suspect that is the issue14:22
mgouldI just clicked on them both and they took me where I expected :-)14:22
shardyit's a heat bug, we should either handle it or fail on create14:22
*** absubram has joined #tripleo14:23
*** trozet has joined #tripleo14:24
mgouldone goes to the build log rather than the patch; the patch is https://review.openstack.org/#/c/265336/14:24
*** absubram_ has joined #tripleo14:24
*** hjensas has quit IRC14:25
*** panda_ has quit IRC14:25
*** thrash|g0ne is now known as thrash14:26
*** panda_ has joined #tripleo14:26
*** absubram has quit IRC14:28
*** absubram_ is now known as absubram14:28
openstackgerritMiles Gould proposed openstack/python-tripleoclient: Use Ironic API v1.11 to support ENROLL state  https://review.openstack.org/27220614:29
openstackgerritMiles Gould proposed openstack/python-tripleoclient: Remove tripleoclient.baremetal wrapper  https://review.openstack.org/26533614:29
*** larsks has joined #tripleo14:30
*** tiswanso has quit IRC14:31
*** tiswanso has joined #tripleo14:32
* mgould submits a new version in response to review comments, so we'll see what happens to the build this time14:33
d0ugalmgould: oh, sorry - I didn't understand the question. I do now. I don't know how often Jenkins is updated.14:33
mgouldhow many Jenkins workers are there?14:33
*** pradk has joined #tripleo14:34
*** pblaho has quit IRC14:35
*** davidlenwell has quit IRC14:35
d0ugalmgould: hundreds I think :)14:36
*** dmsimard is now known as dmsimard|afk14:39
*** pblaho has joined #tripleo14:39
d0ugalAlthough, I don't know how many are dedicated to TripleO14:40
*** jhenner has quit IRC14:42
*** jhenner has joined #tripleo14:42
*** mcornea has quit IRC14:42
*** davidlenwell has joined #tripleo14:49
*** ChanServ sets mode: +v davidlenwell14:49
*** fgimenez has quit IRC14:50
*** fgimenez has joined #tripleo14:53
*** fgimenez has quit IRC14:53
*** fgimenez has joined #tripleo14:53
openstackgerritBen Swartzlander proposed openstack/tripleo-heat-templates: Enable Manila integration  https://review.openstack.org/18813714:56
*** tosky has quit IRC14:59
*** tosky has joined #tripleo15:01
*** hjensas has joined #tripleo15:03
*** pblaho has quit IRC15:03
openstackgerritBen Swartzlander proposed openstack/tripleo-heat-templates: Add NetApp integration to Manila  https://review.openstack.org/18813815:07
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: [NO MERGY] Test a update in trunk repository version  https://review.openstack.org/22978915:07
trownderekh: I put the latest hash to pass RDOCI in there15:08
*** trellooobot has joined #tripleo15:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION15:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+15:10
trellooobot| Title                        |              URL              | Members           | Last Active |15:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+15:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |   149 min   |15:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+15:10
*** trellooobot has quit IRC15:10
derekhtrown: ack15:11
*** tiswanso has quit IRC15:15
*** rpothier has joined #tripleo15:15
*** tiswanso has joined #tripleo15:15
*** tiswanso has quit IRC15:15
*** mcornea has joined #tripleo15:16
*** ukalifon1 has quit IRC15:22
*** pblaho has joined #tripleo15:24
*** dshulyak has quit IRC15:25
*** dshulyak has joined #tripleo15:26
*** aufi has joined #tripleo15:26
*** yamahata has joined #tripleo15:29
*** tiswanso has joined #tripleo15:30
*** regebro has quit IRC15:31
*** egafford has joined #tripleo15:36
egafforddprince, bnemec, shardy: Any chance I could get a quick review on https://review.openstack.org/#/c/271042/ ? (One-line change to remove the Sahara default password in heat-templates.)15:37
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Nothing to see here  https://review.openstack.org/11101115:40
*** regebro has joined #tripleo15:40
*** rasca has quit IRC15:41
egaffordThanks shardy!15:42
*** rasca has joined #tripleo15:43
openstackgerritMerged openstack/tripleo-heat-templates: Removing Sahara password default  https://review.openstack.org/27104215:45
*** shardy_ has joined #tripleo15:53
*** shardy has quit IRC15:54
*** kukacz has joined #tripleo15:56
*** Goneri has quit IRC16:04
*** mbound has quit IRC16:09
*** trellooobot has joined #tripleo16:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION16:10
*** devvesa has quit IRC16:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+16:10
trellooobot| Title                        |              URL              | Members           | Last Active |16:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+16:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |   209 min   |16:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+16:10
*** trellooobot has quit IRC16:10
*** jcoufal_ has joined #tripleo16:11
*** mcornea has quit IRC16:12
*** jcoufal has quit IRC16:12
mgouldis anyone else still having problems with check-osc-plugins gate failures?16:12
mgouldand is there a way I can see all the recent runs of a given check script?16:14
jaosoriormgould: Isn't that because of the missing python-tuskarclient?16:15
jaosoriorIIRC it was recently deprecated, yet it's still trying to find it'16:15
mgouldjaosorior, I think so, but the patch to remove that was merged at 1900 UTC last night16:15
jaosoriorit's setup.py, and fails there16:15
mgouldyep16:15
jaosoriormgould: funky16:15
mgouldbecause python-tuskarclient is now only a README16:15
* mgould goes to hassle openstack-infra again16:16
*** oshvartz has quit IRC16:18
gfidenteshardy_, so there is no bug in LP either for the truncate bug?16:19
*** rbrady has quit IRC16:21
*** rbrady has joined #tripleo16:21
shardy_gfidente: No, AFAIK you're the first to break Heat in this way, congratulations! ;)16:24
gfidente:(16:24
shardy_Feel free to raise one with a reproducer16:24
gfidenteI was looking forward to do some copy/pate16:25
gfidentenot to be the first!16:25
*** devvesa has joined #tripleo16:26
shardy_gfidente: It's probably an easy fix, if you have a reproducer I'm fairly sure we can fix it quickly16:26
openstackgerritEthan Gafford proposed openstack/python-tripleoclient: Sahara integration  https://review.openstack.org/27262516:26
shardy_gfidente: the main issue is we'll ideally need to avoid a DB migration16:26
gfidentewhich makes it less of an easy fix16:27
openstackgerritEthan Gafford proposed openstack/tripleo-heat-templates: Removing Sahara password default  https://review.openstack.org/27262716:27
*** chlong is now known as chlong_zzz16:28
shardy_gfidente: If you've identified the failing config, you obviously have the option of splitting it so it's not so big as a workaround16:30
*** Goneri has joined #tripleo16:30
*** devvesa has quit IRC16:31
*** paramite has quit IRC16:31
*** aufi has quit IRC16:35
*** dtantsur is now known as dtantsur|afk16:39
*** fgimenez has quit IRC16:43
*** fgimenez has joined #tripleo16:44
*** fgimenez has joined #tripleo16:44
*** mbound has joined #tripleo16:44
* derekh still hasn't gotten any logs from the timed out overcloud, 16:44
*** mgould has quit IRC16:48
*** alop has joined #tripleo16:51
*** pblaho has quit IRC16:53
*** mcornea has joined #tripleo16:57
*** a2hill has left #tripleo17:01
*** julim_ has joined #tripleo17:02
*** mgould has joined #tripleo17:03
*** trozet_ has joined #tripleo17:05
*** fgimenez has quit IRC17:06
*** akrivoka has quit IRC17:07
*** trozet has quit IRC17:07
*** julim has quit IRC17:07
*** nijaba has quit IRC17:07
*** jtomasek has quit IRC17:07
*** akrivoka has joined #tripleo17:07
*** nijaba has joined #tripleo17:08
*** nijaba has quit IRC17:08
*** nijaba has joined #tripleo17:08
*** pblaho has joined #tripleo17:08
*** jtomasek has joined #tripleo17:08
*** trellooobot has joined #tripleo17:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION17:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+17:10
trellooobot| Title                        |              URL              | Members           | Last Active |17:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+17:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |   269 min   |17:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+17:10
*** trellooobot has quit IRC17:10
*** openstackgerrit has quit IRC17:17
*** openstackgerrit has joined #tripleo17:17
openstackgerritBen Swartzlander proposed openstack/tripleo-heat-templates: Enable Manila integration  https://review.openstack.org/18813717:22
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Increase galera sync timeout in yum_update.sh  https://review.openstack.org/27266417:26
*** sthillma has joined #tripleo17:29
*** sthillma_ has joined #tripleo17:30
*** dmsimard|afk has quit IRC17:30
*** EmilienM has quit IRC17:30
*** sthillma has quit IRC17:33
*** sthillma_ is now known as sthillma17:33
*** dmsimard has joined #tripleo17:34
*** EmilienM has joined #tripleo17:35
*** ayoung_ZZZzzzz is now known as ayoung17:35
*** ayoung has quit IRC17:36
openstackgerritMiles Gould proposed openstack/python-tripleoclient: Use Ironic API v1.11 to support ENROLL state  https://review.openstack.org/27220617:38
openstackgerritMiles Gould proposed openstack/python-tripleoclient: Remove tripleoclient.baremetal wrapper  https://review.openstack.org/26533617:38
shardy_derekh: FYI I just did a clean VM environment install w/tripleo.sh17:38
*** olap has quit IRC17:38
shardy_there's something wrong with the RPC, heat-api can't connect to rabbit17:38
shardy_I'm not sure why yet, could possibly be related to the timeout issue, all CLI calls to heat hang then time out17:39
slaglehey, i just saw that too17:39
derekhshardy_: yup, same here, heat  commands arn't responding , I'm thinking its this puppet-heat commit https://review.openstack.org/#/c/249711/17:39
slaglerpc_backend is set to qpid in /usr/share/heat!!17:39
derekhgonna try a puppet-heat pin in a minute17:39
shardy_In the heat logs there's ECONNREFUSED, can't connect to rabbit at all17:39
slagleshardy_: is it trying to use qpid or rabbit?17:39
slaglemine was trying to use qpid17:39
shardy_woah - it's qpid!17:40
slaglei was just looking into this17:40
slaglethe packaging has been fixed, https://github.com/openstack-packages/heat/commit/a2ed21a64ca39596bb94d9af063f02c7dba1cd0f17:40
slaglebut i guess our delorean is too old17:40
slaglebut soemthing else must have changed, maybe in puppet-heat17:40
*** ayoung has joined #tripleo17:41
shardy_Ah, that probably explains why I didn't hit this earlier when I was testing a locally build master heat package17:41
derekhslagle: https://review.openstack.org/#/c/249711/17:42
shardy_Ok, well thanks both for the clarification17:42
slagledoh, failed tripleo ci on that one :(17:42
slagleon a side node, config under /usr/share needs to die for all of openstack17:44
*** cody-somerville has joined #tripleo17:45
*** cody-somerville has joined #tripleo17:45
openstackgerritDerek Higgins proposed openstack/tripleo-common: Pin puppet heat  https://review.openstack.org/27267117:47
derekh<slagle> the packaging has been fixed, https://github.com/openstack-packages/heat/commit/a2ed21a64ca39596bb94d9af063f02c7dba1cd0f17:47
derekhthat explains why the test of the newer repository passed earlier17:47
*** regebro has quit IRC17:50
*** rwsu has joined #tripleo17:50
*** electrofelix has quit IRC17:53
*** cody-somerville has quit IRC17:53
*** kukacz has quit IRC17:53
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: [NO MERGY] Test a update in trunk repository version  https://review.openstack.org/22978917:54
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Switch the overcloud pingtest to use the new heat client  https://review.openstack.org/27247917:54
*** ifarkas has quit IRC17:54
derekhtrown ^^ I had a patch to use the new heat client patch you have proposed, when you updated the hash we wanted to test you dropped the patch17:54
*** jaosorior has quit IRC17:55
derekhSo, I gotta run, we got 2 options to unblock CI,17:55
derekh1. merge this if CI passes https://review.openstack.org/#/c/272671/17:55
derekh2. merge the 4 patches referenced in this patch https://review.openstack.org/#/c/229789/  and also update this link on the delorean server http://trunk.rdoproject.org/centos7/current-tripleo/17:56
*** regebro has joined #tripleo17:57
trownderekh: ah whoops, sorry about that17:59
derekhslagle: bnemec dprince ^^17:59
derekhtrown: no prob ;-)18:00
dprincederekh: +2 on your 272671 patch18:00
*** julim has joined #tripleo18:00
derekhdprince: ack, will check back in a bit18:00
*** derekh has quit IRC18:01
*** julim_ has quit IRC18:03
shardy_Interestingly I also get the "inc: command not found" No bootable device error on all nodes with the default undercloud deployed via tripleo.sh18:04
shardy_Upgrading ironic seems to fix it18:04
* shardy_ wonders why it works in CI18:04
*** lucasagomes is now known as lucas-dinner18:04
bnemecshardy_: CI is running on Fedora virt hosts, I believe.  They have a newer ipxe rom for the vms.18:08
shardy_bnemec: aha18:08
bnemecThis has been broken for a long time. :-(18:08
bnemecIt's one of the biggest reasons we need to update the CI pinned repo, because Ironic fixed it a long time ago too.18:09
shardy_Yeah I recall upgrading ironic last time I rebuilt my undercloud, but that was weeks ago :(18:09
shardy_tripleo.sh --delorean-build ftw, I now have upgraded all-the-things :)18:10
*** trellooobot has joined #tripleo18:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION18:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+18:10
trellooobot| Title                        |              URL              | Members           | Last Active |18:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+18:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |   329 min   |18:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+18:10
*** trellooobot has quit IRC18:10
*** jistr has quit IRC18:11
*** olap has joined #tripleo18:17
*** mbound has quit IRC18:19
*** tosky has quit IRC18:19
*** trozet_ is now known as trozet18:20
*** panda_ has quit IRC18:25
*** mgould has quit IRC18:25
*** eil397 has joined #tripleo18:26
*** panda_ has joined #tripleo18:26
*** penick has joined #tripleo18:28
openstackgerritBen Swartzlander proposed openstack/tripleo-heat-templates: Enable Manila integration  https://review.openstack.org/18813718:29
*** olap has quit IRC18:32
*** Marga_ has quit IRC18:33
*** gchamoul is now known as gchamoul|relocat18:39
shardy_slagle: if you get a moment I'd appreciate feedback on https://bugs.launchpad.net/tripleo/+bug/153808218:45
openstackLaunchpad bug 1538082 in tripleo "Re running undercloud install fails" [Undecided,New]18:45
shardy_The root cause appears to be missing selinux-policy-devel18:45
shardy_I tried removing selinux from the blacklist in centos-7-undercloud-packages.json and installing a new undercloud with an updated instack-undercloud18:46
shardy_I thought including that element would install the package, but it appears not18:46
slagleshardy_: can you try deleting /usr/libexec/os-refresh-config and running again18:46
slaglewe probably need to clean that dir on each run18:46
*** bvandenh has quit IRC18:46
slaglei thought there was a patch for this up already18:46
shardy_slagle: ah, I'll give that a try, thanks!18:47
slagleyes, here, https://review.openstack.org/#/c/270809/18:47
*** akuznetsov has joined #tripleo18:48
slaglehmm, it's merged on master already18:48
slagleshardy_: anyway, check if you have that patch18:48
slagleand if so, then i guess that's not the issue :)18:48
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Deploy Aodh services, replacing Ceilometer Alarm  https://review.openstack.org/24140818:49
EmilienMpradk: ^ let's see how CI works18:49
pradkcool18:49
shardy_slagle: ah, yeah I spotted that patch earlier and I do have it18:50
shardy_nvm, I'll dig a bit further, thanks!18:51
*** trown is now known as trown|lunch18:51
*** bvandenh has joined #tripleo18:51
pradkEmilienM, so for ha the tricky part with pacemaker is orchestrating the services? trying to understand whats left for ha18:52
EmilienMpradk: yeah18:55
EmilienMwith apache18:55
pradkEmilienM, ah running the api with wsgi?18:56
EmilienMyep18:56
*** shivrao has joined #tripleo18:57
*** jaosorior_ has joined #tripleo18:57
slagleshardy_: sorry, i was going to try reinstalling my uc in a minute18:58
slaglei'm currently stuck in some weird haproxy/httpd/keystone competing for the same port hell18:59
*** bswartz has joined #tripleo19:00
slagleEmilienM: oh hi!19:00
EmilienMslagle: oh hey james, how is snow?19:00
*** nico_auv has quit IRC19:00
EmilienMit's snowing as hell here19:00
slagleEmilienM: now that keystone is in wsgi on the undercloud, haproxy and httpd are both trying to bind to 3535719:00
openstackgerritDan Prince proposed openstack/tripleo-image-elements: Work around leak in dhcp-all-interfaces udev rule  https://review.openstack.org/27269719:01
EmilienMslagle: wait, on which port was keystone binded before?19:01
dprinceathomas ^^19:01
dprinceathomas: https://review.openstack.org/27269719:01
slagleEmilienM: the same one19:02
slagleEmilienM: ok, haproxy is only binding the vips on that port19:02
slaglebut httpd has <VirtualHost *:35357>19:02
EmilienMwe can change it !19:02
EmilienMlet me send a patch right now19:03
slaglein 10-keystone_wsgi_admin.conf19:03
slagledoesnt that mean httpd will also try and bind on the vips?19:03
EmilienMit will bind on 0.0.0.019:03
EmilienMwhich is iiuc something you don't want19:03
slaglei think we probably want the local_ip value there19:04
EmilienMwait19:04
slagleinstead of *19:04
*** shardy_ has quit IRC19:04
EmilienMslagle: https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L138-L13919:04
EmilienMhow do you explain that? It was binded on 0.0.0.0 before anyway19:05
EmilienManyway, I'm updating it so we can bind on local_ip19:05
openstackgerritBen Swartzlander proposed openstack/tripleo-heat-templates: Add NetApp integration to Manila  https://review.openstack.org/18813819:06
athomasdprince, Cool. Thanks.19:06
slagleEmilienM: i'm not sure19:06
EmilienMslagle: right we override it in the manifest19:07
EmilienMI'm cleaning it19:07
*** gfidente has quit IRC19:07
slaglei updated an existing undercloud to this config19:07
slaglemaybe some old config is leftover?19:07
openstackgerritDan Prince proposed openstack/tripleo-image-elements: Work around leak in dhcp-all-interfaces udev rule  https://review.openstack.org/27269719:07
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: keystone/wsgi: bind on local IP  https://review.openstack.org/27269919:08
EmilienMslagle: ^19:08
EmilienMoops ^19:08
EmilienMit should do the job19:08
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: keystone/wsgi: bind on local IP  https://review.openstack.org/27269919:08
*** sthillma has quit IRC19:08
EmilienMcleaning also hiera ^19:08
*** athomas has quit IRC19:09
*** trellooobot has joined #tripleo19:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION19:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+19:10
trellooobot| Title                        |              URL              | Members           | Last Active |19:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+19:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |   389 min   |19:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+19:10
*** trellooobot has quit IRC19:10
*** akuznetsov has quit IRC19:10
*** akrivoka has quit IRC19:11
*** Marga_ has joined #tripleo19:11
*** mcornea has quit IRC19:12
slagleEmilienM: ok, will try it. just fixing it locally, i had to also add the IP to the Listen directive in /etc/httpd/conf/ports.conf19:18
*** dprince has quit IRC19:21
*** athomas has joined #tripleo19:22
*** mkovacik has quit IRC19:25
*** gchamoul|relocat is now known as gchamoul19:28
*** sthillma has joined #tripleo19:31
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: keystone/wsgi: bind on local IP  https://review.openstack.org/27269919:35
*** rlandy has quit IRC19:35
EmilienMslagle: yeah, my patch should take care of this, we'll check though19:35
*** dshulyak has quit IRC19:37
*** mkovacik has joined #tripleo19:39
*** rlandy has joined #tripleo19:41
*** dprince has joined #tripleo19:42
*** eil397 has quit IRC19:43
*** eil397 has joined #tripleo19:46
*** trown|lunch is now known as trown19:56
*** rcernin has quit IRC19:58
*** lblanchard1 has joined #tripleo20:02
*** lblanchard has quit IRC20:03
*** rpothier has quit IRC20:09
*** sthillma has quit IRC20:10
*** trellooobot has joined #tripleo20:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION20:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+20:10
trellooobot| Title                        |              URL              | Members           | Last Active |20:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+20:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |   449 min   |20:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+20:10
*** trellooobot has quit IRC20:10
*** pradk has quit IRC20:14
*** pradk_ has joined #tripleo20:14
*** egafford has quit IRC20:14
*** pradk_ is now known as pradk20:14
*** mcornea has joined #tripleo20:18
*** bswartz has left #tripleo20:30
openstackgerritDerek Higgins proposed openstack/tripleo-common: Pin puppet heat  https://review.openstack.org/27267120:31
*** derekh has joined #tripleo20:31
* derekh screwed up the puppet-heat pin 20:31
*** rcernin has joined #tripleo20:32
derekhhad to start the puppet-heat pin CI again, screwed up the REPOREF variable name20:32
*** yamahata has quit IRC20:37
openstackgerritSteve Baker proposed openstack/python-tripleoclient: openstack overcloud software deployment show  https://review.openstack.org/27111320:47
openstackgerritSteve Baker proposed openstack/python-tripleoclient: openstack overcloud failures  https://review.openstack.org/27111420:47
*** bvandenh has quit IRC20:52
*** pradk has quit IRC20:53
*** derekh has quit IRC20:53
*** pradk has joined #tripleo20:54
alopRandom question... in DIB, why is finalize stage misspelled as 'finalise'?20:56
slaglewe aint all from 'murica20:57
slaglethere are some british spellings mixed in throughout tripleo20:58
alopTook me a while to figure out why my 'finalize.d' was getting ignored21:01
*** derekh has joined #tripleo21:06
*** trellooobot has joined #tripleo21:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION21:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+21:10
trellooobot| Title                        |              URL              | Members           | Last Active |21:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+21:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |   509 min   |21:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+21:10
*** trellooobot has quit IRC21:10
*** ccrouch has joined #tripleo21:12
openstackgerritBen Swartzlander proposed openstack/tripleo-heat-templates: Add integration with NetApp Manila driver  https://review.openstack.org/18813821:19
*** rcernin has quit IRC21:28
*** julim has quit IRC21:30
*** tiswanso has quit IRC21:32
*** jcoufal_ has quit IRC21:32
*** tiswanso has joined #tripleo21:33
*** sthillma has joined #tripleo21:34
*** trown is now known as trown|outttypeww21:37
*** jprovazn has quit IRC21:38
*** jaosorior_ has quit IRC21:39
*** regebro has quit IRC21:48
*** ayoung has quit IRC21:51
*** lblanchard1 has quit IRC21:53
*** lblanchard has joined #tripleo21:54
*** ayoung has joined #tripleo21:58
*** yamahata has joined #tripleo21:58
*** lblanchard has quit IRC22:00
*** yamahata has quit IRC22:01
*** yamahata has joined #tripleo22:01
*** ccrouch has quit IRC22:02
*** ccrouch has joined #tripleo22:04
*** dprince has quit IRC22:09
*** trellooobot has joined #tripleo22:10
trellooobotURGENT TRIPLEO TASKS NEED ATTENTION22:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+22:10
trellooobot| Title                        |              URL              | Members           | Last Active |22:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+22:10
trellooobot| overcloud deploys timing out | https://trello.com/c/xm08OSBy | **NEEDS MEMBERS** |   570 min   |22:10
trellooobot+------------------------------+-------------------------------+-------------------+-------------+22:10
*** trellooobot has quit IRC22:10
*** penick has quit IRC22:14
*** penick has joined #tripleo22:18
openstackgerritMerged openstack/tripleo-common: Pin puppet heat  https://review.openstack.org/27267122:20
*** davidlenwell has quit IRC22:24
*** penick has quit IRC22:24
*** davidlenwell has joined #tripleo22:25
*** ChanServ sets mode: +v davidlenwell22:25
*** rlandy has quit IRC22:26
*** derekh has quit IRC22:27
*** rlandy has joined #tripleo22:27
*** jayg is now known as jayg|g0n322:32
*** tiswanso has quit IRC22:47
*** ccrouch has quit IRC22:49
*** weshay has quit IRC22:52
greghaynesNot sure if you all have seen, but https://bugs.launchpad.net/diskimage-builder/+bug/153813522:54
openstackLaunchpad bug 1538135 in diskimage-builder "disk-image-create deleting system /dev during image create " [Critical,New]22:54
greghaynesbeware22:54
*** ccrouch has joined #tripleo22:57
*** sthillma_ has joined #tripleo23:05
*** sthillma has quit IRC23:05
*** sthillma_ is now known as sthillma23:05
*** shivrao has quit IRC23:06
*** shivrao has joined #tripleo23:07
*** mcornea has quit IRC23:07
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: keystone/wsgi: bind on local IP  https://review.openstack.org/27269923:10
*** sthillma has quit IRC23:11
*** sthillma has joined #tripleo23:12
*** derekh has joined #tripleo23:29
*** jcoufal has joined #tripleo23:33
*** ooolpbot has joined #tripleo23:49
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:49
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/153812723:49
openstackLaunchpad bug 1538127 in tripleo "overcloud deploys timing out" [Critical,Triaged]23:49
*** ooolpbot has quit IRC23:49
« Monday, 2016-01-25 Index Wednesday, 2016-01-27 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!