Wednesday, 2017-03-08

openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK  https://review.openstack.org/43027700:00
*** owalsh_ has joined #tripleo00:02
*** owalsh_ has quit IRC00:04
*** owalsh-afk has quit IRC00:04
*** owalsh-afk has joined #tripleo00:05
*** rlandy has quit IRC00:05
*** owalsh-afk has quit IRC00:05
*** rhallisey has quit IRC00:06
*** rhallisey has joined #tripleo00:07
*** ckyriakidou has quit IRC00:11
*** rhallisey has quit IRC00:12
*** rdopiera has quit IRC00:16
*** rdopiera has joined #tripleo00:18
*** owalsh has quit IRC00:21
*** owalsh has joined #tripleo00:22
*** limao has joined #tripleo00:23
*** limao has quit IRC00:24
*** limao has joined #tripleo00:25
*** lucas-afk has quit IRC00:34
*** smalleni has quit IRC00:35
*** rook has quit IRC00:36
*** lucasagomes has joined #tripleo00:37
*** sai has joined #tripleo00:37
*** saneax is now known as saneax-_-|AFK00:40
*** jaganathan has quit IRC00:40
*** rook has joined #tripleo00:42
*** rook is now known as Guest4961900:42
*** kjw3 has joined #tripleo00:48
*** lucasagomes has quit IRC00:48
*** Guest49619 has quit IRC00:49
*** sai has quit IRC00:49
*** rook_ has joined #tripleo00:50
*** lucasagomes has joined #tripleo00:50
*** sai has joined #tripleo00:50
*** jaganathan has joined #tripleo00:52
bkeroEmilienM: Do you know if scenario003's overcloud pingtest is supposed to fail like this? http://logs.openstack.org/66/442066/6/check/gate-tripleo-ci-centos-7-scenario003-multinode/717d795/console.html#_2017-03-07_13_56_14_17580900:57
bkeroactually that seems to be the cleanup failing, but that part should be skipped. Nevermind00:57
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869500:59
openstackgerritBen Kero proposed openstack/tripleo-quickstart-extras master: WIP: Test scenario jobs by manually setting deploy_args  https://review.openstack.org/44259501:00
*** liverpooler has quit IRC01:04
*** kjw3 has quit IRC01:05
*** jaganathan has quit IRC01:06
*** portdirect is now known as sdake______01:09
*** sdake______ is now known as portdirect01:09
*** flepied has joined #tripleo01:16
*** rhallisey has joined #tripleo01:16
*** jaganathan has joined #tripleo01:18
*** dougbtv has quit IRC01:21
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869501:32
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869501:33
*** Goneri has joined #tripleo01:43
*** owalsh has quit IRC01:50
*** remix_tj has quit IRC01:54
*** remix_tj has joined #tripleo01:55
*** owalsh has joined #tripleo01:57
*** remix_tj has quit IRC01:59
*** owalsh has quit IRC02:00
*** owalsh has joined #tripleo02:00
*** remix_tj has joined #tripleo02:01
*** cdearborn has quit IRC02:09
*** rhallisey has quit IRC02:12
*** pliu has joined #tripleo02:15
*** liupe__ has joined #tripleo02:17
*** pliu has quit IRC02:20
*** remix_tj has quit IRC02:20
*** remix_tj has joined #tripleo02:23
*** fzdarsky_ has joined #tripleo02:31
*** remix_tj has quit IRC02:32
*** remix_tj has joined #tripleo02:32
*** fzdarsky|afk has quit IRC02:35
*** lihi has quit IRC02:36
*** remix_tj has quit IRC02:36
*** remix_tj has joined #tripleo02:38
*** lihi has joined #tripleo02:39
*** remix_tj has quit IRC02:42
*** remix_tj has joined #tripleo02:43
*** owalsh has quit IRC02:45
*** owalsh has joined #tripleo02:45
*** liupe__ has quit IRC02:49
*** remix_tj has quit IRC02:50
*** pliu has joined #tripleo02:50
*** remix_tj has joined #tripleo02:51
*** liupe__ has joined #tripleo02:52
openstackgerritFeng Pan proposed openstack/puppet-tripleo master: Move etcd to step 2  https://review.openstack.org/44260802:52
openstackgerritFeng Pan proposed openstack/puppet-tripleo master: Add networking-vpp ML2 mechanism driver support  https://review.openstack.org/44252902:54
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/44012402:55
*** owalsh has quit IRC02:55
*** remix_tj has quit IRC02:56
*** remix_tj has joined #tripleo02:58
*** owalsh has joined #tripleo02:59
*** owalsh_ has joined #tripleo03:06
*** owalsh_ has quit IRC03:07
*** remix_tj has quit IRC03:08
*** remix_tj has joined #tripleo03:09
*** chlong_ has joined #tripleo03:15
*** chlong has quit IRC03:15
*** remix_tj has quit IRC03:18
*** remix_tj has joined #tripleo03:22
*** Goneri has quit IRC03:26
*** remix_tj has quit IRC03:27
*** remix_tj has joined #tripleo03:28
*** remix_tj has quit IRC03:31
*** gbarros has joined #tripleo03:37
*** remix_tj has joined #tripleo03:37
*** remix_tj has quit IRC03:49
*** remix_tj has joined #tripleo03:51
*** dmacpher has joined #tripleo03:54
*** gbarros has quit IRC03:56
*** fragatin_ has joined #tripleo04:01
*** yamahata has quit IRC04:02
*** remix_tj has quit IRC04:02
*** psahoo has joined #tripleo04:02
*** fragatina has quit IRC04:04
*** fragatina has joined #tripleo04:05
*** remix_tj has joined #tripleo04:05
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: WIP, enable quickstart to work with clean base OS images.  https://review.openstack.org/44276404:05
*** fragatin_ has quit IRC04:05
*** fragatina has quit IRC04:09
*** jaganathan has quit IRC04:10
*** limao has quit IRC04:11
*** limao has joined #tripleo04:12
*** limao has quit IRC04:17
*** remix_tj has quit IRC04:20
*** remix_tj has joined #tripleo04:22
*** remix_tj has quit IRC04:27
*** remix_tj has joined #tripleo04:28
*** masterjcool has quit IRC04:30
*** udesale has joined #tripleo04:31
*** remix_tj has quit IRC04:32
*** remix_tj has joined #tripleo04:33
*** jaganathan has joined #tripleo04:38
*** remix_tj has quit IRC04:38
*** remix_tj has joined #tripleo04:39
*** links has joined #tripleo04:41
*** masterjcool has joined #tripleo04:42
EmilienMbkero: lookin04:44
*** tzumainn has quit IRC04:45
EmilienMbkero: http://logs.openstack.org/66/442066/6/check/gate-tripleo-ci-centos-7-scenario003-multinode/717d795/console.html#_2017-03-07_13_57_22_88803304:45
EmilienMthe pingtest worked fine, isn't?04:45
*** dsariel has quit IRC04:46
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI test - never merge  https://review.openstack.org/44206604:50
openstackgerritFeng Pan proposed openstack/puppet-tripleo master: Add networking-vpp ML2 mechanism driver support  https://review.openstack.org/44252904:51
*** skramaja has joined #tripleo04:54
*** remix_tj has quit IRC04:56
*** remix_tj has joined #tripleo04:57
*** limao has joined #tripleo04:57
*** prateek has joined #tripleo04:59
*** remix_tj has quit IRC05:06
*** jaganathan is now known as jaganathan_afk05:06
*** remix_tj has joined #tripleo05:08
*** remix_tj has quit IRC05:12
*** remix_tj has joined #tripleo05:13
*** remix_tj has quit IRC05:22
*** remix_tj has joined #tripleo05:24
*** remix_tj has quit IRC05:29
*** yamahata has joined #tripleo05:29
*** remix_tj has joined #tripleo05:31
*** remix_tj has quit IRC05:36
*** jaganathan_afk is now known as jaganathan05:39
*** remix_tj has joined #tripleo05:40
skramajamarios: fyi - in newton (10), after removing the ovs special case, minor update failed because of ovs restarting and and failing.05:40
*** remix_tj has quit IRC05:45
*** fragatina has joined #tripleo05:46
*** remix_tj has joined #tripleo05:46
*** fragatin_ has joined #tripleo05:48
*** fragatina has quit IRC05:51
*** fragatin_ has quit IRC05:52
*** remix_tj has quit IRC05:55
*** yprokule has joined #tripleo06:00
*** dmacpher has quit IRC06:01
*** remix_tj has joined #tripleo06:01
openstackgerritPeng Liu proposed openstack/puppet-tripleo master: Adds ability to disable kdump service  https://review.openstack.org/44295606:05
*** pkovar has joined #tripleo06:07
*** limao has quit IRC06:13
*** limao_ has joined #tripleo06:13
*** abehl has quit IRC06:15
openstackgerritPeng Liu proposed openstack/puppet-tripleo master: Adds ability to disable kdump service  https://review.openstack.org/44295606:16
*** remix_tj has quit IRC06:16
*** remix_tj has joined #tripleo06:17
*** Vijayendra has quit IRC06:28
*** remix_tj has quit IRC06:30
*** remix_tj has joined #tripleo06:31
*** abregman_ has joined #tripleo06:33
bkeroEmilienM: not in my case06:36
bkeroEmilienM: http://logs.openstack.org/95/442595/4/experimental/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-nv/9559d24/logs/oooq/multinode-overcloud-pingtest.txt.gz06:37
*** fultonj has joined #tripleo06:37
bkeroERROR: HEAT-E99001 Service cinder is not available for resource type OS::Cinder::Volume, reason: cinder volumev3 endpoint is not in service catalog.06:38
*** jaganathan has quit IRC06:38
bkeroI'm wondering if Cinder would be expected to fail since it isn't in scenario003-multinode.yml06:41
*** skramaja_ has joined #tripleo06:42
*** remix_tj has quit IRC06:42
*** skramaja has quit IRC06:43
*** pcaruana has joined #tripleo06:43
*** remix_tj has joined #tripleo06:46
*** remix_tj has quit IRC06:49
*** abehl has joined #tripleo06:49
*** fultonj has quit IRC06:50
bkeroAh, I'm guessing I'm not getting TENANT_PINGTEST_TEMPLATE set correctly06:50
bkeroSince MULTINODE_ENV_NAME might be wrong06:50
*** remix_tj has joined #tripleo06:51
*** jaganathan has joined #tripleo06:51
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869506:53
openstackgerritJohn Fulton proposed openstack/puppet-tripleo stable/newton: Include nova::compute::libvirt::qemu from the libvirt profile  https://review.openstack.org/44297006:54
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869506:55
*** dparkes has joined #tripleo07:00
*** ealcaniz has joined #tripleo07:00
*** remix_tj has quit IRC07:00
*** remix_tj has joined #tripleo07:01
*** saneax-_-|AFK is now known as saneax07:05
*** remix_tj has quit IRC07:05
*** remix_tj has joined #tripleo07:06
*** aufi has joined #tripleo07:06
*** skramaja_ is now known as skramaja07:07
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Stop the chronyd service  https://review.openstack.org/44251407:08
bkeroYep, that seems to be the case. That last patchset fixes it ^07:09
*** remix_tj has quit IRC07:10
*** remix_tj has joined #tripleo07:11
*** iranzo has joined #tripleo07:14
*** dsneddon has quit IRC07:14
*** jtomasek has joined #tripleo07:16
*** hjensas has quit IRC07:17
*** remix_tj has quit IRC07:19
*** remix_tj has joined #tripleo07:21
*** dsneddon has joined #tripleo07:22
*** jprovazn has joined #tripleo07:27
*** skramaja_ has joined #tripleo07:34
*** skramaja has quit IRC07:35
*** leanderthal|afk is now known as leanderthal07:35
*** saibarspeis has joined #tripleo07:35
*** masco has joined #tripleo07:37
*** skramaja_ is now known as skramaja07:38
*** pgadiya has joined #tripleo07:38
*** rcernin has joined #tripleo07:40
*** remix_tj has quit IRC07:40
*** remix_tj has joined #tripleo07:40
*** tesseract has joined #tripleo07:43
*** remix_tj has quit IRC07:45
*** remix_tj has joined #tripleo07:46
*** jaosorior has joined #tripleo07:50
*** george_goh|afk has quit IRC07:51
*** b00tcat has joined #tripleo07:51
*** athomas has joined #tripleo07:54
*** remix_tj has quit IRC07:57
*** pkovar has quit IRC07:58
*** pkovar has joined #tripleo07:59
*** b00tcat is now known as b00tcat|away07:59
*** hewbrocca_afk is now known as hewbrocca08:00
*** remix_tj has joined #tripleo08:00
*** b00tcat|away is now known as b00tcat08:01
*** suuuper has joined #tripleo08:01
*** florianf has quit IRC08:03
*** tremble has joined #tripleo08:04
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job with MySQL  https://review.openstack.org/42529508:05
*** george_goh has joined #tripleo08:05
*** jaganathan has quit IRC08:05
*** gfidente has joined #tripleo08:06
*** gfidente has quit IRC08:06
*** gfidente has joined #tripleo08:06
*** remix_tj has quit IRC08:06
*** b00tcat has quit IRC08:07
*** b00tcat has joined #tripleo08:07
*** milan has joined #tripleo08:08
*** remix_tj has joined #tripleo08:08
*** florianf has joined #tripleo08:08
*** agurenko has joined #tripleo08:09
*** jlinkes has joined #tripleo08:10
*** hjensas has joined #tripleo08:10
*** hjensas has quit IRC08:10
*** hjensas has joined #tripleo08:10
*** fultonj has joined #tripleo08:11
*** fzdarsky_ is now known as fzdarsky08:11
*** remix_tj has quit IRC08:13
*** remix_tj has joined #tripleo08:13
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Respond to multiple messages from workflow executions  https://review.openstack.org/40617808:15
d0ugalCan somebody land this requirements fix? https://review.openstack.org/#/c/44033708:17
d0ugalUnit tests are failing without it for me08:17
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: barbican: Use versionless keystone endpoints  https://review.openstack.org/44298708:18
*** chlong_ has quit IRC08:18
d0ugaljaosorior: thanks!08:18
d0ugalmatbu: you too :)08:18
d0ugalmarios: *08:18
* d0ugal fails08:18
marioso/08:18
mariosd0ugal: its ok apparently bandini now has to change his nick to banidini08:19
d0ugallol08:19
d0ugalwhy?08:19
hewbroccamarios: or he could just change it to MR-PSE08:19
mariosd0ugal: i accidentally wrote that in an email this morning08:19
*** zoli|gone is now known as zoli08:19
d0ugalaha08:19
*** zoli is now known as zoli|wfh08:20
bandiniahahah I did not even notice08:20
matbud0ugal: no pb, thank you too :)08:21
*** jaganathan has joined #tripleo08:22
d0ugallol08:22
jaosoriorccamacho, marios: I'm in a review round. Got anything you would like checked?08:25
*** fultonj has quit IRC08:25
mariosjaosorior: thanks jaosorior will let you know.. most things have landed for the upgrades stuff... i have https://review.openstack.org/#/c/436990/ if you want to check it ... once it lands we have to get it to ocata ... and probably newton but that is tbd (actually assaf said this morning we would get it into osp10, i mean ovs 2.6 but i need to check if rdo repos already have it/not)08:27
*** paramite has joined #tripleo08:27
*** mcornea has joined #tripleo08:28
matbumarios: are you fine to land this : https://review.openstack.org/#/c/440337 ?08:30
matbumarios: im about to submit it08:30
*** amoralej|off is now known as amoralej08:31
mariosmatbu: yeah i already voted there i mean +2A did you mean a differentone?08:31
openstackgerritSagi Shnaidman proposed openstack/tripleo-heat-templates master: DON'T MERGE: testin oooq scenario  https://review.openstack.org/43885808:34
*** bogdando has joined #tripleo08:34
matbumarios: nop that fine08:36
matbumarios: erf sorry, /me needs coffee08:36
matbumarios: i was not talking about this review :)08:37
matbulet me find the right one08:37
matbumarios: https://review.gerrithub.io/#/c/351387/08:37
matbuthis ^ :)08:37
mariosmatbu: ah yeah sure, i mean i am really not an expert on ansible modules so i can't even commment on the convention (e.g. things like check_and_fail ;) ) i will be including latest in my runs today but if you already ran it then go ahead. i did include it yesterday but was v1208:39
matbumarios: ack, ok08:40
*** pkovar has quit IRC08:40
*** ealcaniz has quit IRC08:40
*** jpena|away is now known as jpena08:41
*** shardy has joined #tripleo08:43
*** sshnaidm|off is now known as sshnaidm08:45
*** karimb has joined #tripleo08:46
*** gchamoul|afk is now known as gchamoul08:46
*** jpich has joined #tripleo08:48
*** liupe__ has joined #tripleo08:49
*** sudipto has joined #tripleo08:50
*** sudipto_ has joined #tripleo08:50
*** pmannidi has quit IRC08:51
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Merge "Run indent checks in diskimage_builder/lib" into feature/v2  https://review.openstack.org/44172308:52
*** abregman_ is now known as abregman08:53
ccamachohey jaosorior not for today :) but thanks!08:55
ccamachojaosorior, maybe this one https://review.openstack.org/#/c/438959/08:56
*** pgadiya has quit IRC08:59
*** zzzeek has quit IRC09:00
*** ealcaniz has joined #tripleo09:00
*** zzzeek has joined #tripleo09:00
*** openstackgerrit has quit IRC09:03
chemmatbu: hi, commented on your review.  This is what is returned http://ix.io/oOp09:04
*** karimb has quit IRC09:05
matbuchem: ha yep nice09:07
matbuchem: i thought it was a string message09:07
matbuchem: the module was modified on my test env09:07
*** limao_ has quit IRC09:07
matbuchem: i landed it and pushed a new release, the new module should available soon via rpm09:07
chemmatbu: cool, you're the rpm master09:08
matbuchem: lol yep, i click on a button and some magics happen then, but idk how :D09:09
*** openstackgerrit has joined #tripleo09:09
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: glance: deploy services with Keystone v3 endpoints  https://review.openstack.org/44279809:09
*** fultonj has joined #tripleo09:09
*** dtantsur|afk is now known as dtantsur09:12
*** pgadiya has joined #tripleo09:12
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/44301309:13
ccamachohey bandini! morning, are you around?09:14
ccamachomorning all **** (~˘▾˘)~09:14
bandiniccamacho: hola carlos. sortof (in a call atm)09:14
ccamachoWont bug you then, I'll do it in 30 mins :P09:15
ccamachobandini^09:15
*** karimb has joined #tripleo09:16
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-ui stable/ocata: Imported Translations from Zanata  https://review.openstack.org/44301509:16
bandiniccamacho: feel free to pm stuff and then we'll chat ;)09:16
*** snecklifter has joined #tripleo09:18
*** jaosorior is now known as jaosorior_brb09:19
*** ukalifon has joined #tripleo09:24
*** ukalifon has quit IRC09:25
*** ealcaniz has quit IRC09:29
*** fzdarsky is now known as fzdarsky|afk09:30
*** ealcaniz has joined #tripleo09:30
*** derekh has joined #tripleo09:32
*** dtantsur is now known as dtantsur|bbl09:34
*** bogdando has quit IRC09:37
*** bogdando has joined #tripleo09:38
*** lmasiero has joined #tripleo09:40
*** ckyriaki has joined #tripleo09:47
*** tosky has joined #tripleo09:47
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates stable/ocata: Remove ha-by-default release note in Ocata  https://review.openstack.org/44304109:50
openstackgerritMerged openstack/python-tripleoclient master: Update to a newer hacking library  https://review.openstack.org/44033709:51
openstackgerritMerged openstack/tripleo-heat-templates master: Remove the openvswitch special case upgrade code  https://review.openstack.org/43699009:51
*** karimb has quit IRC09:52
openstackgerritMerged openstack/tripleo-common stable/newton: Update to a newer hacking library  https://review.openstack.org/43980409:52
*** salmankhan has joined #tripleo09:52
*** karimb has joined #tripleo09:56
openstackgerritMerged openstack/python-tripleoclient master: Show stack failures on error  https://review.openstack.org/43549110:01
openstackgerritMerged openstack/python-tripleoclient stable/ocata: Fixed overcloud stack update workflow name error  https://review.openstack.org/43653510:01
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Enable the HA deployment by default  https://review.openstack.org/35906010:01
*** yamahata has quit IRC10:02
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart-extras master: [LINTERS] Fix ansible-lint.sh and lint issues  https://review.openstack.org/44304410:02
*** remix_tj has quit IRC10:05
*** remix_tj has joined #tripleo10:06
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart master: Fix ansible-lint.sh script for roles and playbooks  https://review.openstack.org/43145610:07
*** iranzo has quit IRC10:07
*** iranzo has joined #tripleo10:07
*** iranzo has joined #tripleo10:07
skramajashardy: can you approve this backports for ocata - https://review.openstack.org/#/c/439486/, https://review.openstack.org/#/c/439706/10:07
*** fzdarsky|afk is now known as fzdarsky10:08
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Adding env file for a non-ha deployment  https://review.openstack.org/44305010:08
openstackgerritLukas Bezdicka proposed openstack/tripleo-common stable/ocata: Fix typo in package_update.yaml  https://review.openstack.org/44067510:09
*** iranzo has quit IRC10:11
*** iranzo has joined #tripleo10:12
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Respond to multiple messages from workflow executions  https://review.openstack.org/40617810:12
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Update the baremetal workflows to use the multiple message interface  https://review.openstack.org/44305310:12
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci master: Enable the HA deployment by default  https://review.openstack.org/41517910:16
*** bogdando has quit IRC10:17
*** bogdando has joined #tripleo10:17
*** bogdando has quit IRC10:20
*** bogdando has joined #tripleo10:23
*** karthiks has quit IRC10:25
*** liupe__ has quit IRC10:27
*** karimb has quit IRC10:29
*** akrivoka has joined #tripleo10:32
openstackgerritMerged openstack/tripleo-ui stable/ocata: Imported Translations from Zanata  https://review.openstack.org/44301510:33
shardyskramaja: sure, looking10:33
rook_shardy gfidente heya - not sure if either of you saw my message, however I am pretty much stuck hard on a deployment... Seems to be something with Ceph, but I am not sure how to pinpoint what.10:34
*** udesale has quit IRC10:34
*** rook_ is now known as rook10:42
rooki did just look at the heat-engine logs on the undercloud and caught this "10:43
rook2017-03-07 20:46:14.951 65502 ERROR oslo_db.sqlalchemy.exc_filters DataError: (1406, u"Data too long for column 'resource_properties' at row 1")10:43
rookshardy ^ and prior to the traceback, it is a huge dumping ground of text.10:43
shardyrook: hey, yeah sorry saw your message but not got to it yet10:45
rookshardy https://gist.github.com/jtaleric/6dec8d19a0eb591cd3e9dee1262b0b08 -- there are about 4 screens worth of dumping going on here.10:45
shardyrook: that heat error sounds wrong, lets see what caused it10:45
rookshardy: sure -- hoping the additional research this morning helpes pinpoint things.10:46
shardyrook: so, yeah, in the past we had some non-critical errors where event data got truncated10:47
shardyrook: but here, it's truncating resource_properties10:47
shardythat is probably very bad10:47
bogdandohi folks. I'd like to make a scratch deploy with oooq AND containers, and gonna ask stupid questions now, so please be kind. Here is what I have on my mind, please correct the list to guide me in the best direction:10:47
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: undercloud-deploy: export undercloud_network_cidr  https://review.openstack.org/44286210:47
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102910:49
shardyrook: actually wait - this is an event - what version of heat is this?10:49
rookshardy: whatever we are currently shipping10:49
bogdandoSo a scratch deploy:10:49
bogdando- with openstack containers enabled, only a controller node with only a keystone component10:49
bogdando- with oooq & ansible runner at my ubuntu xenial host OR https://github.com/dprince/undercloud_containers/blob/master/doit.sh in centos7 priv/nethost container or in a VM?10:49
bogdando- with current promoted repo https://trunk.rdoproject.org/centos7-master/current-tripleo-rdo/ OR consistent?10:49
bogdando- with undercloud vm running which image undercloud.qcow2 ?10:49
bogdando- with overcloud vm's running which images overcloud-full.qcow2 ? (don't want to wait for building to complete)10:49
bogdandolater, I'd want to deploy only a component under dev, like etcd in containers10:49
bogdandonote, for now I have only my laptop which runs xenial. I will have another one with RHEL, or what else options I have to deploy, like OVB?10:49
rookopenstack-heat-engine-7.0.1-2.el7ost.noarch shardy10:49
shardyrook: can you share the log lines before/after that error pls?10:49
shardytherve: ^^ Hey are you aware of any new issues when storing events?10:50
shardyI thought we fixed that a while ago10:50
bogdandowhat is the closest development approach you use normally? I'd like to not reinvent wheels for oooq etc.10:50
therveshardy, We changed the model, maybe it broke it again...10:50
rookshardy therve this only happened after a ton of scaling up ,and some scaling down.10:50
rooknot it seems wedged10:51
shardyhttps://github.com/openstack/heat/commit/57c5aae88b908002d84a933738293c591792617410:51
shardytherve: ah, so yeah I see we moved/removed the truncation workaround10:52
shardyremoved by the look of it, I guess that is the problem - AFAICS we also removed the test for storing oversized properties :(10:53
therveshardy, :/10:53
shardyrook, therve: I raised https://bugs.launchpad.net/heat/+bug/167104310:58
openstackLaunchpad bug 1671043 in heat "Storing events can fail (again) due to oversized properties" [Undecided,New]10:58
therveshardy, Thanks, I'll have a look10:58
shardytherve: thanks, much appreciated!10:58
rookshardy: giving that this is a live env -- what is the best path forward to keep it going10:59
rookthis is in the CNCF lab and we were hoping to take things to 900 baremetal nodes.10:59
openstackgerritMerged openstack/diskimage-builder master: Merge "Run indent checks in diskimage_builder/lib" into feature/v2  https://review.openstack.org/44172311:01
*** fragatina has joined #tripleo11:01
rooktherve ^^11:03
rookso as soon as something hits, I should be able to see if it helps this deployment11:03
*** karthiks has joined #tripleo11:04
shardyrook: I could suggest hacky workarounds, but can we give therve a while to propose a proper fix?11:05
*** fragatina has quit IRC11:06
shardyrook: this is a regression from newton, so it'd be best to fix it in heat properly I think11:06
* shardy goes to grab a coffee11:06
*** akrivoka has quit IRC11:07
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Send custom parameters in bootloader to GRUB_CMDLINE  https://review.openstack.org/44108611:07
*** akrivoka has joined #tripleo11:10
*** fultonj has quit IRC11:12
rookack shardy - it is just that we don't have this lab full time, and we are going to get the rest of the nodes soon... I am not trying to push the proper fix to not be good, but I would like to continue working on this lab if possible.11:13
rookThe current bug has me at a hard stop.11:14
rookbut I understand where you are coming from.11:14
*** hjensas has quit IRC11:15
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Send custom parameters in bootloader to GRUB_CMDLINE  https://review.openstack.org/44108611:16
*** jprovazn has quit IRC11:17
*** athomas has quit IRC11:18
shardyrook: ack, I get it's time sensitive, I think there are some possible workaround, e.g in the DB the heat.event.resource_properties column in blob, but the resource.properties_data column is longtext11:23
shardyI suspect the DB schema for event could be modified to longblob and it'd work11:24
shardybut really we want a way to not store giant properties for every event as it's probably not super useful11:24
*** athomas has joined #tripleo11:25
rookshardy: sure, however I worry that the fix will require me to nuke things and start over11:25
rookwhich is a noop...11:26
*** karthiks has quit IRC11:26
rookshardy: maybe I am wrong about the fix though?11:26
shardyrook: a heat fix wont - the previous fix that got removed in the commit I mentioned just truncated the data11:26
shardybasically some refactoring removed the limit we put in place some time ago11:26
shardywe need to put it back again11:27
rookah ha ok, good11:27
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Detect CREATE_COMPLETE instead of CREATE_FAILED  https://review.openstack.org/44307611:27
shardyrook: that's why I'd rather go for the heat fix first vs messing with the DB11:27
rookthanks -- ill capture the heat-engine log and share it - maybe there is other things we can identify11:27
rookshardy: ack.11:28
shardyrook: thanks, you can link any further info to https://bugs.launchpad.net/heat/+bug/167104311:28
openstackLaunchpad bug 1671043 in heat "Storing events can fail (again) due to oversized properties" [Critical,Triaged] - Assigned to Thomas Herve (therve)11:28
shardyrook: thanks for finding this, and sorry for the inconvenience11:29
*** chem has quit IRC11:29
*** ansmith has quit IRC11:29
*** dtantsur|bbl is now known as dtantsur11:37
*** karthiks has joined #tripleo11:37
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: undercloud-deploy: export undercloud_network_cidr  https://review.openstack.org/44286211:38
*** hjensas has joined #tripleo11:40
openstackgerritMerged openstack/os-net-config stable/ocata: Restart Openvswitch after adding DPDK ports  https://review.openstack.org/43948611:40
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates master: [WIP] Migrate Glance DB on upgrades  https://review.openstack.org/44308011:42
*** karthiks has quit IRC11:43
b00tcatHi, an example SERVICE_node_names variable of mine is ["overcloud-controller-0.internalapi.localdomain"] , however `hostname -f` returns "overcloud-controller-0.localdomain"11:47
b00tcatis it because in node_names the network segment is added in the middle in those variables?11:47
*** user` has joined #tripleo11:48
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart master: break out general_config into config and topology  https://review.openstack.org/42545111:48
*** user` is now known as chem11:48
*** chem has quit IRC11:48
*** chem has joined #tripleo11:48
*** zoli|wfh is now known as zoli|lunch11:49
lmasierohello guys, i'm facing some issues with ooo/newton when deploying in HA (3controllers). The nova-conductor on the compute nodes is timing out11:51
lmasieroany similiar issues anyone?11:51
lmasieroi'm about to collect data and submit as a bug request, bugs.launchpad.net/tripleo?11:52
*** pkovar has joined #tripleo11:53
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: pass the topology of the overcloud w/ topology vs extra-args  https://review.openstack.org/42545311:53
*** jprovazn has joined #tripleo11:55
*** karthiks has joined #tripleo11:55
*** dprince has joined #tripleo11:55
*** adarazs is now known as adarazs_lunch11:59
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add option not to skip building images  https://review.openstack.org/44308511:59
*** dparkes has quit IRC12:00
EmilienMhello12:02
*** jaosorior_brb is now known as jaosorior12:03
openstackgerritMartin André proposed openstack/tripleo-common master: Add heat-api-cloudwatch container  https://review.openstack.org/44309312:03
jaosoriorEmilienM: updated your commit here https://review.openstack.org/#/c/442798/ it needs a fix in glance_store12:04
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Move nova-scheduler data into puppet_config  https://review.openstack.org/44266412:04
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Enable Docker service for Compute role  https://review.openstack.org/44254912:04
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Docker service for heat-api-cloudwatch  https://review.openstack.org/44309512:04
EmilienMjaosorior: you rocks, you know?12:05
EmilienMjaosorior: I mean, thx for your help12:05
jaosoriorEmilienM: we shouldn't be using Admin anymore12:05
EmilienMjaosorior: why?12:05
jaosoriorit's not necessary at all for keystone12:05
EmilienMahhh12:05
EmilienMjaosorior: ok12:05
jaosoriorfor v312:05
jaosoriorfor v2 it is though12:05
EmilienMjaosorior: thx for the info12:06
jaosoriorEmilienM: I can explain when you're back12:06
jaosorioranyway12:06
jaosoriorenjoy Boston!12:06
*** dparkes has joined #tripleo12:06
EmilienMjaosorior: thanks12:07
*** dprince has quit IRC12:07
EmilienMjaosorior: can you update https://review.openstack.org/#/c/442620/ ?12:07
EmilienMjaosorior: and https://review.openstack.org/#/c/442793/12:08
jaosoriorEmilienM: will do12:08
*** nyechiel has joined #tripleo12:08
EmilienMjaosorior: not urgent otherwise I'll do it tonight12:08
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: sahara: configure keystone_authtoken parameters  https://review.openstack.org/44262012:09
EmilienMjaosorior: if you have time to review https://review.openstack.org/#/c/442802/ also12:09
jaosoriorack12:09
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: gnocchi: deploy services with Keystone v3 endpoints  https://review.openstack.org/44279312:11
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job with RabbitMQ  https://review.openstack.org/42630712:13
jaosoriormarios, shardy: hey, when do you think it's appropriate to add configuration options to undercloud.conf?12:15
*** flepied has quit IRC12:17
mariosnot something i've spend any time thinking about (maybe it came up at PTG?) jaosorior ... i guess you have a specific use case in mind12:17
*** lmasiero has quit IRC12:18
jaosoriormarios: https://review.openstack.org/#/c/442610/12:20
mariosthanks jaosorior will check it out12:21
openstackgerritOpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements  https://review.openstack.org/44003812:22
*** gkadam has quit IRC12:25
*** dprince has joined #tripleo12:25
*** ckyriaki has quit IRC12:25
*** lucasagomes is now known as lucas-hungry12:25
*** dparkes has quit IRC12:26
*** david-lyle has quit IRC12:30
*** dprince has quit IRC12:31
*** flepied has joined #tripleo12:32
*** david-lyle has joined #tripleo12:33
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/44012412:33
*** shardy has quit IRC12:34
*** egafford has quit IRC12:35
*** gkadam has joined #tripleo12:36
*** jkilpatr has joined #tripleo12:36
openstackgerritJiri Tomasek proposed openstack/tripleo-specs master: Deployment plan management changes  https://review.openstack.org/43891812:37
openstackgerritAna Krivokapic proposed openstack/tripleo-common master: Enhance plan creation and update with plan-environment  https://review.openstack.org/41416912:37
openstackgerritAna Krivokapic proposed openstack/tripleo-common master: Add plan export action and workflow  https://review.openstack.org/42278912:37
therverook, Hum that's strange, not storing events shouldn't block anything AFAIR12:37
*** dparkes has joined #tripleo12:38
*** dmacpher has joined #tripleo12:40
*** karimb has joined #tripleo12:41
*** psahoo has quit IRC12:41
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: test multinode, pingtest  https://review.openstack.org/44311112:43
*** gkadam has quit IRC12:43
akrivokajtomasek: spec looks good to me, I'll wait for some more review before going ahead with creating blueprints in case somebody comes up with changes to work items12:43
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: glance: deploy services with Keystone v3 endpoints  https://review.openstack.org/44279812:44
*** snecklifter has quit IRC12:45
*** snecklifter has joined #tripleo12:46
*** rhallisey has joined #tripleo12:46
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Keep existing data for containerized Swift  https://review.openstack.org/44311212:46
openstackgerritLuke Hinds proposed openstack/puppet-tripleo master: SSHD Service extensions  https://review.openstack.org/44311312:49
*** lmasiero has joined #tripleo12:52
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102912:52
*** bfournie has quit IRC12:54
*** dprince has joined #tripleo12:54
*** rhallisey has quit IRC12:54
jtomasekakrivoka, thanks!12:55
jtomasekakrivoka, will Depends-On prevent this one to merge before dependent patch is merged? https://review.openstack.org/#/c/437676/12:55
therverook, Ping me when you get back12:56
*** lmasiero has quit IRC12:57
hjensasshardy: Is it possible to read in introspection data from a heat template? I can see OS::Swift::Container, but it looks like it is there to create a swift container. not read from one?12:57
*** abregman has quit IRC12:58
akrivokajtomasek: I think so12:58
*** adarazs_lunch is now known as adarazs12:59
*** abregman has joined #tripleo12:59
thervehjensas, He's not around. And for sure the container resource is only to create one13:00
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart-extras master: Add ability to set novajoin_read|connect timeout  https://review.openstack.org/44262613:00
hjensastherve: ok, thanks.13:01
*** ansmith has joined #tripleo13:01
*** abregman has quit IRC13:02
*** ccamacho is now known as ccamacho|lunch13:02
jaosoriord0ugal: does mistral run multinode jobs from tripleo?13:03
*** abregman has joined #tripleo13:03
*** jayg|g0n3 is now known as jayg13:04
d0ugaljaosorior: yeah, well, it runs gate-tripleo-ci-centos-7-scenario003-multinode-puppet13:05
*** noslzzp has quit IRC13:05
jaosoriorgreat13:05
*** zoli|lunch is now known as zoli|wfh13:05
*** noslzzp has joined #tripleo13:05
*** morazi has joined #tripleo13:07
*** rhallisey has joined #tripleo13:07
*** Baggypants12000 has quit IRC13:08
rooktherve: pong13:12
therverook, So to confirm, this is newton right?13:12
rookyes therve, what we ship today13:12
akrivokad0ugal: I was wondering why these operations are not documented with their CLI commands but rather with calling actions/workflows directly? https://docs.openstack.org/developer/tripleo-docs/mistral-api/mistral-api.html13:13
therverook, OK so Steven diagnosis is probably wrong13:13
*** morazi has quit IRC13:13
akrivokad0ugal: AFAICT, we have corresponding CLI commands available for all of them13:13
therverook, This shouldn't be a fatal error. What's the problem that you have precisely?13:13
rooktherve so the deployment seems stuck on some Ceph action - if You have a moment, I can explain via bluejeans?13:13
akrivokad0ugal: or is it just that these docs were added before the commands, and now need to be updated?13:13
rooktherve: there is quite a bit of information13:13
*** pgadiya has quit IRC13:14
therverook, In a meeting right now for 45 mins13:14
*** dougbtv has joined #tripleo13:16
*** jkilpatr has quit IRC13:17
openstackgerritMerged openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/44301313:17
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart-extras master: Add ability to set novajoin_read|connect timeout  https://review.openstack.org/44262613:18
jpichakrivoka: I think the goal was to document how to drive deployments, etc only using Mistral and illustrate how that API works13:20
bandinimatbu: are you going to cut a new release of ansible-pacemaker and check with the rdo folks to get a new package in ocata or do other things need to happen?13:23
bandinichem: ^13:23
*** liupe__ has joined #tripleo13:24
chembandini: matbu told me that he pushed a button that should trigger  a rebuild of the rpm13:24
*** amoralej is now known as amoralej|lunch13:24
bandinichem: ah nice. will the rpm appear in the ocata repos as well?13:24
matbubandini: it's done13:24
matbubandini: yep13:24
matbubandini: i think it's a  matter of hours13:25
bandinimatbu: awesome, thanks! :)13:25
*** links has quit IRC13:25
*** morazi has joined #tripleo13:25
*** lucas-hungry is now known as lucasagomes13:27
d0ugalakrivoka: Yeah, what jpich said. That documentation is intended for people writing something like a GUI or integrating externally13:27
akrivokaah I see, thanks jpich d0ugal13:27
d0ugalI guess I failed to make that clear :)13:27
*** nyechiel has quit IRC13:28
d0ugalakrivoka: I do think the documentation is lacking coverage for many of the CLI commands - maybe worth a bug report13:29
*** bfournie has joined #tripleo13:30
akrivokad0ugal: nah, the first sentence clearly states the intent so that's clear, I was just think in a little bit different direction (why do we mention in the docs how to e.g. create a plan calling a mistral wofklow, but no mention of how to do it via a CLI command)13:30
akrivokad0ugal: yes, exactly, I think many of the CLI commands are still undocumented13:31
d0ugal+113:32
d0ugalakrivoka: I think that is partly because the plan management story from the CLI isn't that complete.13:32
akrivokad0ugal: true13:32
akrivokad0ugal: so a section for plan management in the docs could make sense (once all the commands are there)?13:33
d0ugalakrivoka: yup - starting the docs might even be a good way to figure out what is missing. I'm not sure.13:33
akrivokacool13:33
openstackgerritMerged openstack/tripleo-quickstart master: Add centos-opstools repo to default config for master  https://review.openstack.org/43118913:34
*** prateek has quit IRC13:34
akrivokad0ugal: https://bugs.launchpad.net/tripleo/+bug/167109413:35
openstackLaunchpad bug 1671094 in tripleo "Document plan management using the CLI" [Medium,Triaged]13:35
d0ugalakrivoka: thanks!13:35
*** rbrady-afk is now known as rbrady13:39
*** suuuper has quit IRC13:39
*** pblaho has joined #tripleo13:40
*** eck` is now known as eck`gone13:40
jpichyay more docs13:41
openstackgerritJames Slagle proposed openstack/python-tripleoclient master: Use tmp env path when rewriting absolute resource paths  https://review.openstack.org/44313113:41
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Send custom parameters in bootloader to GRUB_CMDLINE  https://review.openstack.org/44313213:41
openstackgerritMerged openstack/tripleo-ui master: Update version for Pike  https://review.openstack.org/44194313:42
*** egafford has joined #tripleo13:43
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869513:43
*** karimb has quit IRC13:43
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869513:43
*** liverpooler has joined #tripleo13:44
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Nic config mappings for deployed-server  https://review.openstack.org/44217013:44
*** rhallisey has quit IRC13:46
*** karimb has joined #tripleo13:46
*** jcoufal has joined #tripleo13:46
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: extras-common: add undercloud_network_cidr as common var  https://review.openstack.org/44286213:47
*** jkilpatr has joined #tripleo13:48
*** suuuper has joined #tripleo13:51
*** gbarros has joined #tripleo13:55
*** tzumainn has joined #tripleo13:55
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: New ovb playbook and settings  https://review.openstack.org/43156713:57
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: [WIP] Upgrade to webpack 2  https://review.openstack.org/44313714:00
*** jaosorior has quit IRC14:01
*** rhallisey has joined #tripleo14:01
*** jjung has joined #tripleo14:02
*** gbarros has quit IRC14:02
*** rbowen has joined #tripleo14:04
openstackgerritHonza Pokorny proposed openstack/puppet-tripleo master: Refactor enabled languages from an array to an object  https://review.openstack.org/44268314:04
*** gbarros has joined #tripleo14:05
ansiwendprince: https://bpaste.net/show/59f2d3d2482714:05
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Simplify addition of new languages  https://review.openstack.org/44078814:05
ansiwendprince: it's docker/post.yaml14:06
ansiwendprince: ha! there is a "<<<<<<< HEAD" inside... obviously a patch didn't work14:07
dprinceansiwen: weird, I've not seen that locally.14:07
dprinceansiwen: oh, right. That'll do it :)14:07
ansiwendprince: actually the post.j2.yaml already has it...14:09
ansiwendprince: I will just rerun your doit.sh and hope it's gone by now :-)14:09
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui master: Fix nodes actions toolbar  https://review.openstack.org/44313914:09
*** jmelvin has joined #tripleo14:10
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: extras-common: add undercloud_network_cidr as common var  https://review.openstack.org/44286214:13
*** ccamacho|lunch is now known as ccamacho14:13
*** shardy has joined #tripleo14:13
*** karimb_ has joined #tripleo14:13
trozetshardy: hi14:15
*** dparkes has quit IRC14:15
*** karimb has quit IRC14:15
*** dparkes has joined #tripleo14:16
*** jkilpatr has quit IRC14:17
*** pradk has quit IRC14:17
*** karimb_ has quit IRC14:19
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Send mail tool  https://review.openstack.org/42334014:20
shardytrozet: Hi!14:22
*** Goneri has joined #tripleo14:22
trozetshardy: hey man, can you help me out with some reviews?14:23
shardytrozet: sure14:24
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869514:24
trozetshardy: this one passed everything except containers job, I thought that was non-voting, maybe it got moved recently https://review.openstack.org/#/c/435081/14:26
trozetshardy: can you review, and do I need to recheck it?14:26
*** amoralej|lunch is now known as amoralej14:26
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo master: Correct haproxy's stat unix socket path  https://review.openstack.org/44314814:27
*** karimb has joined #tripleo14:29
ansiwendprince: it's because this has been merged already: https://review.openstack.org/#/c/43174614:34
dprinceansiwen: ack, we should remove those14:34
*** saibarspeis has quit IRC14:34
chemmatbu: scrum ?14:35
dprinceansiwen: https://github.com/dprince/undercloud_containers/commit/18a23e5e192c0c5c2a33b8172f55052c15b5f41414:37
matbuchem: yep im in14:37
ansiwendprince: cool14:38
shardytrozet: I think the containers job is broken ref https://review.openstack.org/#/c/442549/ and it's probably unrelated to your patch14:39
trozetshardy: ok cool14:39
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Add a generic interface to create nested stacks before the steps  https://review.openstack.org/40449914:40
*** cdearborn has joined #tripleo14:40
shardytrozet: it looks OK to me although the interface is kinda weird, e.g why an "IncreaseFileLimit" boolean vs a number like MysqlFileLimit ?14:40
*** lblanchard has joined #tripleo14:40
shardyI guess it's related to the underlying puppet interfaces?14:40
trozetshardy: yeah I had the same comment14:40
trozetshardy: look at my comment here https://review.openstack.org/#/c/408250/14:40
trozetshardy: apparently there is a different way it gets set to that value in pacemaker version14:41
openstackgerritToure Dunnon proposed openstack/tripleo-specs master: [WIP] Plan Upgrade and Update Workflow  https://review.openstack.org/44315214:41
*** dsariel has joined #tripleo14:41
trozetshardy: so Damien said we should just do this as a fix then change how it is set for both in another patch + support providing an int14:41
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Remove karma, jasmine, and phantomjs dependencies  https://review.openstack.org/44315314:41
*** ckyriakidou has joined #tripleo14:42
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Update the baremetal workflows to use the multiple message interface  https://review.openstack.org/44305314:42
shardytrozet: Ok thanks for clarifying, sounds like it's OK as a first step then14:43
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Send custom parameters in bootloader to GRUB_CMDLINE  https://review.openstack.org/44313214:44
*** eck`gone is now known as eck`14:44
trozetshardy: yeah I agree it is ugly though14:44
trozetshardy: I'll try to find some time to dig into how it gets set in HA and then provide a master only patch to change it to be int and set it in both types of deployments14:44
trozetshardy: then if you could also review this one please: https://review.openstack.org/#/c/419648/14:45
shardytrozet: small comment but looks good14:47
trozetshardy: ah yes missed that14:48
*** chlong_ has joined #tripleo14:48
*** markmc` is now known as markmc14:48
trozetshardy: ok will fix, thanks14:48
shardytrozet: or is the intention that you'd override the default ovs resource?14:48
shardytrozet: ack, thanks14:48
trozetshardy: yeah wait14:49
trozetshardy: yeah override the regular OVS14:49
shardytrozet: Ok so maybe adding an environment file is enough?14:50
trozetshardy: yeah I can add one14:51
shardytrozet: Ok thanks, at least then it's a bit clearer how it should be enabled14:51
trozetshardy: np thanks for reviews14:51
openstackgerritMerged openstack/tripleo-quickstart-extras master: Fix failure when id_rsa_undercloud doesn't exist  https://review.openstack.org/44249214:52
*** gbarros has quit IRC14:53
*** dmacpher is now known as dmacpher-afk14:55
honzajpich: did you have any ideas on what the visual highlight would be?  bold?  a different color?  https://bugs.launchpad.net/tripleo/+bug/166948514:57
openstackLaunchpad bug 1669485 in tripleo "Highlight the language currently in use" [Medium,Triaged] - Assigned to Honza Pokorny (hpokorny)14:57
jpichhonza: I was thinking the background of the cell would become black and the text white for the one currently selected, but your "bold" suggestion sounds just as good to me14:59
honzalooks like upgrade jobs are now succeeding on ui jobs?  nice14:59
honzajpich: k, i'll play around a bit (there is something in patternfly for all i know)14:59
*** yamahata has joined #tripleo15:00
jpichhonza: If you ask lblanchard for her input she'll probably have super awesome ideas too :)15:01
*** jaosorior has joined #tripleo15:01
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: [WIP] collect-logs: add rename task for uncompressed files too  https://review.openstack.org/44316315:02
* lblanchard looking15:03
*** saneax is now known as saneax-_-|AFK15:04
*** jcoufal has quit IRC15:04
lblanchardhonza, jpich: what do you guys think about using the treatment that's used in the Multiple Select widget in PatternFly? http://www.patternfly.org/pattern-library/widgets/#bootstrap-select15:05
*** jcoufal has joined #tripleo15:06
lblanchardblue highlight with a check next to the language that's selected.15:06
*** abregman is now known as abregman|mtg15:06
*** prateek has joined #tripleo15:07
honzalblanchard: that definitely looks nice15:07
honzalblanchard: right now, that part of the ui is handled by a dropdown widget though http://www.patternfly.org/pattern-library/widgets/#dropdowns15:08
lblanchardhonza: think you could replicate the check and blue if I get you an icon and hex value?15:08
honzalblanchard: i'm not sure if we could replace it with a select easily (might look out of place)15:08
*** tdasilva- has joined #tripleo15:08
lblanchardhonza: yeah I don't think we'd want a multi-select widget…just the same style for selection :)15:09
honzalblanchard: I think you're overestimating my css skills :)15:09
lblanchardhonza: hahahah…I think you are just being modest!15:09
lblanchardhonza: maybe start with the blue highlight…stretch goal is adding the check? :)15:09
honzalblanchard: good idea!  i'm looking at the code and we actually have a generic dropdown item which supports an "active" state15:10
lblanchardhonza: oh perfect15:11
*** Vijayendra has joined #tripleo15:13
*** pradk has joined #tripleo15:13
*** tdasilva- has quit IRC15:13
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Send mail tool  https://review.openstack.org/42334015:13
*** gbarros has joined #tripleo15:14
honzaflorianf: apparently, this needs your +1 https://review.openstack.org/#/c/440546/ :)15:18
*** karimb has quit IRC15:19
florianfhonza: huh. interesting15:19
honzayeah15:20
florianfhonza: done15:20
honzaflorianf: thanks15:20
*** karimb has joined #tripleo15:21
openstackgerritHolger Smolinski proposed openstack/diskimage-builder master: Add zipl support for s390x architecture  https://review.openstack.org/44317115:21
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Keep existing data for containerized Libvirt  https://review.openstack.org/44317215:22
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Keep existing data for containerized ironic-conductor  https://review.openstack.org/44317315:22
*** dprince has quit IRC15:24
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869515:26
*** tdasilva- has joined #tripleo15:28
*** ealcaniz has quit IRC15:32
beaglesdsneddon: did you see my reply to your comments on https://review.openstack.org/#/c/434472/15:34
beaglesdsneddon: basically it was a backport and just kept it "intact"... I can remove the sample files from the backport if you prefer however15:34
*** links has joined #tripleo15:35
*** fultonj has joined #tripleo15:36
rookgfidente: you open to a ceph question?15:37
gfidenterook, pong15:39
*** fultonj has quit IRC15:39
rookhttps://gist.github.com/jtaleric/e6dcbd77a22a93ac12747e1f88e235b0 gfidente15:39
rookgfidente: how can i repair this node, then hand it back to OOO ?15:40
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Add support for deploying Ceph using ceph-ansible via external workflow  https://review.openstack.org/44113715:41
gfidenterook, so it looks like nothing generated ceph.conf on the node at all?15:41
rookgfidente: yup, nothing there15:42
gfidenterook, so what is launching ceph-disk ? which role is this?15:42
*** yprokule has quit IRC15:42
rookthis is the ceph-storage role gfidente15:43
rookroot@overcloud-cephstorage-015:43
gfidenterook, ok it's strange I am not sure why puppet should be trying to run ceph-disk but not creating the config15:44
gfidenterook, any chance I can login on the node and we use tmux/tmate to check things together?15:45
jtomasekflorianf, which nodes action do you see that bug happening on?15:45
rookyes gfidente15:45
*** dmarlin has joined #tripleo15:46
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: Deploy multinode scenarios with deploy args  https://review.openstack.org/44318815:46
honzalblanchard: so, it works great, thanks for leading me down the right path :)15:46
jpichlblanchard rocks \o/15:47
lblanchardhonza: woo hoo!! Happy I could be helpful :)15:47
openstackgerritLuke Hinds proposed openstack/puppet-tripleo master: SSHD Service extensions  https://review.openstack.org/44311315:48
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Highlight current i18n language in navbar  https://review.openstack.org/44319115:49
*** sudipto has quit IRC15:52
*** sudipto_ has quit IRC15:52
openstackgerritHolger Smolinski proposed openstack/diskimage-builder master: Oops, left the (C) statement erronoeously in the patch.  https://review.openstack.org/44319715:53
*** mcornea has quit IRC15:54
jaosoriorshardy: I think the bug with the priorities is also affecting python packages https://review.openstack.org/#/c/443104/ this failure's stack trace matches the line numbers of the package before my change, so it seems it didn't take into account the depends-on15:54
florianfjtomasek: introspection, provide15:54
openstackgerritHolger Smolinski proposed openstack/diskimage-builder master: Add zipl support for s390x architecture  https://review.openstack.org/44317115:54
*** jaganathan has quit IRC15:55
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102915:57
*** rcernin has quit IRC15:58
shardyjaosorior: yeah ref my mail to openstack-dev I think everything is affected15:59
shardyjaosorior: I think priorities don't help on upgrade because we've already installed the newer version15:59
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK  https://review.openstack.org/43027716:00
jaosoriorshardy: oh, hadn't checked your mail16:00
*** jjung has quit IRC16:00
openstackgerritMonty Taylor proposed openstack/diskimage-builder master: Include systemd-sysv for debian stretch  https://review.openstack.org/44320116:01
*** nyechiel has joined #tripleo16:01
*** fragatina has joined #tripleo16:02
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Set multinode scenarios with quickstart  https://review.openstack.org/44320416:02
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: Deploy multinode scenarios with deploy args  https://review.openstack.org/44318816:03
*** cwolferh has quit IRC16:05
*** fragatina has quit IRC16:06
*** ckyriakidou has quit IRC16:07
openstackgerritElise Gafford proposed openstack/tripleo-heat-templates master: Default to empty list for CinderNfsServers  https://review.openstack.org/44203816:08
*** ckyriakidou has joined #tripleo16:08
openstackgerritMonty Taylor proposed openstack/diskimage-builder master: Include systemd-sysv for debian stretch  https://review.openstack.org/44320116:09
*** adarazs is now known as adarazs_afk16:10
*** dsariel has quit IRC16:11
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci master: Fix REVIEW_RELEASE: unbound variable error  https://review.openstack.org/44321616:12
openstackgerritMonty Taylor proposed openstack/diskimage-builder master: Include systemd-sysv for debian stretch  https://review.openstack.org/44320116:14
*** ealcaniz has joined #tripleo16:17
*** dsneddon has quit IRC16:20
*** jkilpatr has joined #tripleo16:21
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: test multinode, pingtest  https://review.openstack.org/44311116:24
*** dsneddon has joined #tripleo16:24
*** jaosorior has quit IRC16:24
openstackgerritToure Dunnon proposed openstack/tripleo-specs master: [WIP] Plan Upgrade and Update Workflow  https://review.openstack.org/44315216:27
*** dsneddon has quit IRC16:28
dtantsurEmilienM, hey! mind removing -2 from https://review.openstack.org/#/c/408581/ please?16:30
*** jkilpatr has quit IRC16:30
sshnaidmslagle, hi, isn't it enough? https://review.openstack.org/#/c/441706/216:30
*** lucasagomes is now known as lucas-afk16:30
*** karimb has quit IRC16:31
openstackgerritDmitry Tantsur proposed openstack/tripleo-common master: Validate nodes_json before trying to register or updates any nodes  https://review.openstack.org/40858116:31
slaglesshnaidm: you said "it wasn't ideal" :)16:31
sshnaidmslagle, yeah, that's why I use $HOME if no root :)16:32
slaglesshnaidm: what you've done is enough and an improvement, but the whole point of this issue is that $HOME is not guaranteed to be correct16:33
*** ckyriakidou has quit IRC16:35
slaglewas just suggesting a different approach, it's fine what you have16:35
sshnaidmslagle, I think it may have problem only if you use "sudo -u non_root_user instack ..."16:35
sshnaidmslagle, the improvement will be using arguments for all these options and not env variables16:36
slaglesure, that would work16:37
*** apevec has joined #tripleo16:37
trozetslagle: can you please review https://review.openstack.org/#/c/435081/ ?16:40
*** nyechiel has quit IRC16:40
mwhahahaanyone know where the various steps (1 through 5) are documented?16:41
shardymwhahaha: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/README.rst#deployment-steps16:41
*** jcoufal_ has joined #tripleo16:42
mwhahahaah ha16:42
shardyit's pretty sparse, we could perhaps use more detailed docs in tripleo-docs but I don't think there are any atm16:42
mwhahahawe should document some of these core concepts a bit better in tripleo-docs16:42
d0ugalIs there an easy way to add --debug to the CLI commands in CI?16:43
shardymwhahaha: also a FAQ has been how can we map the steps to what is included in puppet-tripleo, e.g at stepN [$services] are enabled16:43
shardymwhahaha: any ideas on how we could do that would be good, so far I've not found a good way to do it on the puppet side, e.g a dry-run to catch all the includes for each step16:44
slagletrozet: +2, but i dont think you answered bnemec's question16:44
mwhahahashardy: yea that'd be trickey since we do some splitting of service configuration between the steps as well16:44
shardymwhahaha: if anyone can solve that, I know how we can preview the generation of the composable manifest via t-h-t16:44
slagletrozet: he was asking if now that fd's are raised, is max connections going to be an issue?16:44
slagletrozet: e.g, is there some 1:1 fd per connection?16:44
trozetslagle: oh crap16:45
trozetslagle: hes asking about max connections?16:45
bnemecYes :-)16:45
slagletrozet: his comment was under that parameter :)16:45
*** jcoufal has quit IRC16:45
trozetbnemec: sorry16:45
bnemecI don't think it's 1:1, but I seem to recall needing to raise max_connections in big deployments.16:45
openstackgerritDmitry Tantsur proposed openstack/instack-undercloud master: Revert "Revert "Deploy and Upgrade Ironic to run in mod_wsgi""  https://review.openstack.org/35180316:46
trozetbnemec: i think it does have an effect, but dciabrin would know better than I16:46
* dciabrin is reading...16:46
bnemectrozet: np.  I think we're still better off after the change because by default we're limited to 1024.16:47
*** dparkes has quit IRC16:47
trozetdciabrin: bnemec is asking about the max connections, it is listed here https://review.openstack.org/#/c/435081/1/puppet/services/database/mysql.yaml as 409616:47
bnemecI just suspect we'll never be able to use the full open-file-limit with 4096 max_connections.16:47
trozetbnemec: well even though this is set to 4096 max connections, is that actually what gets configure in HA and non ha deployments16:47
bnemecBut that is configurable and I think big deployments will already be setting it.16:47
*** tremble has quit IRC16:48
trozetbnemec: i think there are system limits for that as well16:48
dciabrintrozet, bnemec so currently in HA we have max_connections set to 4096 and open-file-limit to 1638416:49
dciabrinthere's no 1:1, one connection can consumes many fd16:50
trozetbnemec: in my noha deployment it is 4096 connections, so I guess that is working OK16:50
trozetdciabrin:^16:50
*** myoung is now known as myoung|lunch16:51
*** myoung|lunch is now known as myoung16:51
dciabrintrozet yes I think 4096 is what we expect. I have the same in my fresh nonha deploy16:52
bnemecYeah, I was just wondering if we had removed one bottleneck while leaving another.16:53
*** jlinkes has quit IRC16:54
dciabrinbnemec, i think in the long run you're right, we should have it configurable as well, but for both nanha and ha. however the vast majority if not all deploy that i've seen are ok with that number16:54
*** rhallisey has quit IRC16:54
trozetdciabrin, bnemec: Keep in mind that too many connections can cause high RAM usage and lock up your MySQL server. Usually small websites will require between 100-200 connections while larger may require 500-800 or even more. The value you apply here strongly depends on your particular MySQL/MariaDB usage.16:54
*** links has quit IRC16:55
trozetdciabrin, bnemec: from that statement, it seems pretty high already16:55
*** milan has quit IRC16:55
dciabrintrozet, yeah true, but in openstack we eat connections like crazy, due to having many unix processes per service, i.e. we can't do connection pooling at that level16:56
bnemecdciabrin: It should already be configurable, so if people need to change it they can.16:56
bnemecThis was an honest question, not a passive-aggressive "I think something is wrong" question. :-)16:56
dciabrinbnemec, oh no offense taken :)16:56
bnemecI know it can be hard to tell since I do the latter so often. ;-)16:57
dciabrinhaha!16:57
trozetdciabrin: after backporting this fix...we need to submit another patch to master to make it into an integer.  Can you tell me where the HA value is configured/limit raised?16:57
bogdandoPTAL https://review.openstack.org/#/c/442608/2 I hope my point is valid16:57
bogdandoa related question for heat/tht/ooo steps and matching of those in puppet https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/README.rst#deployment-steps . Are there docs for pre/post steps as well?16:57
bogdandoalso related, how could we check a similar change request for a consistency across related repos and places? Backwards compatibility should be ok, thanks to mwhahaha for clarification on that: "there really isn't backwards compatibility as it is assumed that for each release you get matching puppet-tripleo and THT"16:57
dciabrintrozet, so that's a question i haven't asked yet on https://review.openstack.org/#/c/435081/. will changing the type of this parameter have any adverse effect on minor updates?16:58
*** athomas has quit IRC16:58
*** bfournie has quit IRC16:58
trozetdciabrin: minor updates to what?16:59
shardyHrm, anyone know what creates /tmp/profiledir.* on the undercloud?16:59
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui master: Fix nodes actions toolbar  https://review.openstack.org/44313916:59
shardyI've got 729689 of them, and growing fast16:59
dciabrintrozet, sorry.. I mean if you "overcloud deploy" with that patch, and then you "overcloud deploy" on the same stack a second time when we use an integer instead of a bool in  MysqlIncreaseFileLimit. will heat validation pass if type changed?17:01
* shardy adds audit rule17:02
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: [WIP] collect-logs: add rename task for uncompressed files too  https://review.openstack.org/44316317:02
*** dprince has joined #tripleo17:02
*** ansmith has quit IRC17:02
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102917:02
mwhahahashardy: my goolging shows it might happen during the install (as part of dib processes)17:03
mwhahahashardy: are you currently installing?17:03
trozetdciabrin: that heat parameter will change names to like MysqlMaxOpenFilesLimit17:03
mandreshardy: I believe it's dib-run-parts17:03
trozetdciabrin: but it will require changes to puppet-tripleo as well17:03
trozetdciabrin: it will only change in master, the current fix is good for backporting to stable branches so at least they dont fail after a couple days of running in noha17:04
shardymwhahaha: No my undercloud has been running for about a week17:04
mwhahahashardy: weird mine doesn't have any of those files17:04
trozetdciabrin: i'm just not sure what is setting the value for HA deployments, I thought you may how that is being done17:04
mandreshardy: I *only* have 6000 or so, my undercloud is fresher :)17:05
dciabrintrozet, yeah we pass that value to a pacemaker resource attribute, and changing it will most probably make pacemaker restart the galera resource on all node. i'm not sure yet how to update that without a small service outage :(17:05
*** prateek_ has joined #tripleo17:05
trozetdciabrin: ok cool.  I'll look into that17:07
shardyHmm, something is running mktemp every second and not cleaning up17:07
shardyaccording to auditd17:07
*** cwolferh has joined #tripleo17:08
*** leanderthal is now known as leanderthal|afk17:08
*** prateek has quit IRC17:08
openstackgerritSaravanan KR proposed openstack/os-net-config stable/ocata: Network service is failing with DPDK  https://review.openstack.org/44323717:10
mandreshardy: my undercloud has about 1000 defunct instack-haproxy processes too :-/17:11
shardyhttp://paste.openstack.org/show/601969/17:14
shardydtantsur: ^^ is ironic-inspector likely to be doing mktemp calls related to that perioidic task?17:15
*** skramaja has quit IRC17:15
shardythe batches of mktemps seem correlated when you combine the auditd and messages logs17:15
trozetshardy: with upgrades, what step does puppet actually run again for the service?17:15
shardytrozet: after all the upgrade_tasks are finished17:16
shardywe then run the normal puppet steps17:16
trozetshardy: ok so after step 517:16
shardyyup17:16
*** jpich has quit IRC17:16
dtantsurshardy, not unless it's hidden somewhere in processutils17:16
trozetshardy: is there any special handling for database stuff like mysql: https://github.com/openstack/tripleo-heat-templates/blob/stable/ocata/puppet/services/database/mysql.yaml#L10317:16
trozetshardy: where puppet doesnt get run for mysql ?17:16
shardytrozet: No puppet is run for all services, exactly the same as on create17:17
trozetshardy: ok thanks17:17
*** ealcaniz has quit IRC17:18
*** bfournie has joined #tripleo17:18
*** tesseract has quit IRC17:18
*** jaganathan has joined #tripleo17:19
*** dbecker has quit IRC17:20
shardydtantsur: Ok thanks, will keep digging17:20
*** masco has quit IRC17:20
openstackgerritMerged openstack/tripleo-quickstart-extras master: Updating skip file mechanism  https://review.openstack.org/43846517:20
*** aufi has quit IRC17:21
*** adarazs_afk is now known as adarazs17:23
*** fragatina has joined #tripleo17:24
*** fragatina has quit IRC17:24
*** abregman|mtg has quit IRC17:24
*** fragatina has joined #tripleo17:25
*** gfidente has quit IRC17:25
*** ansmith has joined #tripleo17:27
dciabrinshardy so quick question: I'd like to ensure that the fix in  https://review.openstack.org/#/c/435081 will work on an existing nonha overcloud. To simulate that, I'm assuming I just need to update t-h-t on the undercloud, and retrigger "openstack overcloud deploy". am I correct?17:28
*** nyechiel has joined #tripleo17:29
shardydciabrin: yes, assuming you have the related puppet-tripleo change in your image or have otherwise deployed it to the nodes (e.g via deploy artifacts)17:29
*** yamahata has quit IRC17:30
dciabrinshardy, right, all my controllers are deployed with the latest ocata image and they have the puppet-tripleo change in already. ok so i obviously did something wrong, I don't see the new systemd dropin file being generated after I reran "overcloud deploy". investigating :)17:32
*** ansmith has quit IRC17:33
*** ansmith has joined #tripleo17:34
*** kjw3 has joined #tripleo17:36
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui master: Fix nodes actions toolbar  https://review.openstack.org/44313917:36
*** florianf has quit IRC17:37
*** prateek_ has quit IRC17:37
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: WIP, enable quickstart to work with clean base OS images.  https://review.openstack.org/44276417:37
*** bogdando has quit IRC17:39
*** cwolferh has quit IRC17:45
*** shardy has quit IRC17:45
*** dtantsur is now known as dtantsur|afk17:45
*** zoli|wfh is now known as zoli|gone17:46
*** nyechiel has quit IRC17:47
*** derekh has quit IRC17:52
*** milan has joined #tripleo17:58
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: update master skip list for introspection test  https://review.openstack.org/44325517:58
*** yamahata has joined #tripleo17:59
*** hewbrocca is now known as hewbrocca_afk18:03
*** jaganathan has quit IRC18:04
*** milan has quit IRC18:05
*** b00tcat has quit IRC18:06
*** jpena is now known as jpena|off18:07
*** karimb has joined #tripleo18:07
openstackgerritMerged openstack/tripleo-ui master: Add jest dependencies  https://review.openstack.org/44054618:12
*** salmankhan has quit IRC18:15
openstackgerritTim Rozet proposed openstack/tripleo-puppet-elements stable/newton: Adds puppet-systemd  https://review.openstack.org/44326118:15
*** fzdarsky is now known as fzdarsky|afk18:17
*** rbrady is now known as rbrady-food18:18
*** dtrainor has quit IRC18:25
*** fragatina has quit IRC18:25
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: [WIP] collect-logs: add rename task for uncompressed files too  https://review.openstack.org/44316318:26
bnemecI see a bunch of multinode jobs failing on /tmp/tmpwolX3I/extra-data.d/10-create-pkg-map-dir: line 11: get_image_element_array: command not found18:27
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869518:29
*** pkovar has quit IRC18:32
*** chlong_ has quit IRC18:37
*** dsneddon_ has joined #tripleo18:38
*** rcernin has joined #tripleo18:42
bkerobnemec: I see those too. Did dib get a push that removed that command?18:47
bkeroLooks like it was added in this commit back in September: https://review.openstack.org/gitweb?p=openstack/diskimage-builder.git;a=commit;h=37a53354ec51a1d20c3ac7bfa70744fa858dcb8818:52
openstackgerritLuke Hinds proposed openstack/tripleo-heat-templates master: SSHD Service extensions  https://review.openstack.org/44328918:53
*** chlong_ has joined #tripleo18:53
openstackgerritLuke Hinds proposed openstack/tripleo-heat-templates master: SSHD Service extensions  https://review.openstack.org/44328918:54
openstackgerritLuke Hinds proposed openstack/tripleo-heat-templates master: SSHD Service extensions  https://review.openstack.org/44328918:55
*** tosky has quit IRC18:58
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: [WIP] collect-logs: add rename task for uncompressed files too  https://review.openstack.org/44316319:06
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: extras-common: add undercloud_network_cidr as common var  https://review.openstack.org/44286219:06
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart master: break out general_config into config and topology  https://review.openstack.org/42545119:07
*** karimb has quit IRC19:08
openstackgerritMerged openstack/os-cloud-config stable/ocata: Updated from global requirements  https://review.openstack.org/43168119:08
openstackgerritMerged openstack/os-collect-config stable/ocata: Updated from global requirements  https://review.openstack.org/43168219:08
*** rwsu has quit IRC19:09
*** rlandy has joined #tripleo19:10
*** pcaruana has quit IRC19:13
*** bkopilov_ has joined #tripleo19:14
*** bkopilov has quit IRC19:14
openstackgerritMerged openstack/tripleo-quickstart-extras master: update master skip list for introspection test  https://review.openstack.org/44325519:16
*** fragatina has joined #tripleo19:17
openstackgerritLeehom Li proposed openstack/diskimage-builder master: Fix problem unable disable iptables service  https://review.openstack.org/41970319:18
*** tosky has joined #tripleo19:20
honzajtomasek: I don't really think the new error handling buys you that much.  For some reason, I was expecting a bigger difference?19:21
honzajtomasek: What's the best part of it from your POV?19:21
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: tripleo docs have overcloud image build instructions, add that option  https://review.openstack.org/44329419:21
*** abehl has quit IRC19:22
openstackgerritPaul Belanger proposed openstack/diskimage-builder master: DNM: testing nodepool-dsvm job  https://review.openstack.org/44329619:26
*** Goneri has quit IRC19:27
*** agurenko has quit IRC19:28
*** suuuper has quit IRC19:34
*** karthiks has quit IRC19:35
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: WIP: Switch quickstart config from scenario to multinode-settings  https://review.openstack.org/43869519:37
*** amoralej is now known as amoralej|off19:39
*** dsneddon_ has quit IRC19:41
*** percevalbot` has quit IRC19:47
*** karthiks has joined #tripleo19:47
*** dsneddon has joined #tripleo19:50
*** alop has joined #tripleo19:53
*** dparkes has joined #tripleo19:56
*** noslzzp has quit IRC20:05
*** noslzzp has joined #tripleo20:05
*** noslzzp has quit IRC20:05
*** noslzzp has joined #tripleo20:06
*** dtrainor has joined #tripleo20:06
*** karimb has joined #tripleo20:06
*** gbarros_ has joined #tripleo20:12
*** gchamoul is now known as gchamoul|afk20:12
*** gbarros has quit IRC20:13
*** chlong_ has quit IRC20:16
*** noslzzp has quit IRC20:17
*** noslzzp has joined #tripleo20:17
*** rbrady-food is now known as rbrady20:18
*** iranzo has quit IRC20:26
trozetmwhahaha: hi20:27
*** chlong_ has joined #tripleo20:28
mwhahahatrozet: hey20:28
trozetmwhahaha: need some input from a puppet master :)20:28
mwhahahatrozet: sure what are you running into?20:29
trozetmwhahaha: https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/database/mysql.pp#L153 in newer versions of puppet I think this works, but in older versions there is an error with the hash key and i have to put it in single quotes to work around it20:29
mwhahahatrozet: ah yea it should probably be quoted20:30
saidprince: hey, during stack update one of the novacomputes goes into error..but in order to track the node i cant seem to find the associated isntance uuid in ironic node-list as in https://gist.github.com/smalleni/f26c9760cd56e2fed5f59e670ace80ea20:30
saiany idea whats going on?20:30
trozetmwhahaha: for only older versions?20:30
mwhahahatrozet: for consistency all versions20:31
openstackgerritPaul Belanger proposed openstack/diskimage-builder master: DNM: testing nodepool-dsvm job  https://review.openstack.org/44329620:31
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Move ironic into services-docker  https://review.openstack.org/44330820:31
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Move mistral into services-docker  https://review.openstack.org/44330920:31
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Move zaqar into services-docker  https://review.openstack.org/44331020:31
trozetmwhahaha: crap20:31
trozetmwhahaha: so can i just submit another puppet-tripleo patch to fix it for master and backport to ocata, then the original cherry pick for newton ( still havent posted that I can just do it now in the cherry pick) ?20:31
mwhahahatrozet: you could probably just fix it in the cherry picks for puppet 320:32
trozetmwhahaha: ok20:32
mwhahahatrozet: for puppet 4 it's more of a best practice20:33
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates master: Enables increasing mariadb open files for noha deployments  https://review.openstack.org/43508120:33
mwhahahatrozet: I'll see if I can find some doc's around it20:33
trozetmwhahaha: best practice to use quotes?20:34
openstackgerritDan Prince proposed openstack-infra/tripleo-ci master: Update to use new service-docker environments  https://review.openstack.org/44331120:34
trozetmwhahaha: the reason I didnt was cause in the puppet guide it doesnt use them20:34
mwhahahaYea and the puppet 3 vs 420:34
trozethttps://docs.puppet.com/puppet/4.9/lang_data_hash.html20:34
mwhahahaUpstream has moved on from 3 so probably why the docs don't have it20:35
mwhahahaBut for our case we still need to support 320:35
mwhahaha3 eoled in December20:35
*** rcernin has quit IRC20:36
dprincesai: ironic node-list should show an 'Instance UUID' column that has what you are looking for I think right?20:36
openstackgerritTim Rozet proposed openstack/puppet-tripleo stable/newton: mariadb: Move generation of systemd drop-in to puppet-tripleo  https://review.openstack.org/44331220:37
bkeroslagle: do you remember anything about tripleo-build-images? I think that dib had a release recently that broke multinode jobs20:37
saidprince: grep returns nothing, hence the confusion20:37
* bkero has been git blaming trying to figure out if anybody actually knows anything about this system anymore20:37
thrashbkero: what about it?20:39
thrashbkero: have a link to a job?20:39
bkerothrash: the dib calls it makes seem to be failing http://logs.openstack.org/95/442595/4/experimental/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-nv/e375def/console.html#_2017-03-08_18_43_55_41205220:39
openstackgerritTim Rozet proposed openstack/puppet-tripleo stable/newton: mariadb: Move generation of systemd drop-in to puppet-tripleo  https://review.openstack.org/44331220:39
bnemecbkero: The thing I don't understand is that a bunch of jobs have passed since I started seeing this.20:39
bkerobnemec: was it transient?20:40
thrashbkero: that doesn't look like a problem with tripleo-build-images tbh20:40
thrashunless I'm missing something20:40
bnemecbkero: I don't think so, because 3 of the last 4 multinode jobs on the cistatus page failed on it.20:40
bkerothrash: tripleo.sh's --bootstrap-subnodes calls calls tripleo-ci/scripts/bootstrap-overcloud-full.sh which calls tripleo-build-images20:40
bnemecAlthough it's possible there's some weird ordering on the status page.20:41
thrashbkero: sure, but there is nothing in tripleo-build-images that calls that 'get_image_element_array' directly.20:41
thrashbkero: looks like a problem with a specific element perhaps20:41
bkerothrash: and during that tripleo-build-images it's sourcing in element-info, which should create the bash function get_image_element_array, but doesn't20:41
bnemecIt's a very strange function.  Apparently it's generated by a different dib command: https://github.com/openstack/diskimage-builder/blob/master/diskimage_builder/lib/common-functions#L27620:42
bkeroYes, it's generated here: http://git.openstack.org/cgit/openstack/diskimage-builder/tree/diskimage_builder/lib/common-functions#n25320:42
bkeroThat was rewritten in V220:42
bkeroSorry, generated here: http://git.openstack.org/cgit/openstack/diskimage-builder/tree/diskimage_builder/element_dependencies.py#n29320:43
*** rbrady is now known as rbrady-afk20:43
thrashbkero: the deepest tripleo-build-images (and the code in tripleo-common) goes is calling disk-image-create20:43
openstackgerritDan Sneddon proposed openstack/tripleo-specs master: blueprint tripleo-routed-networks-deployment  https://review.openstack.org/42100920:44
bkerothrash: do other tripleo jobs still use tripleo-build-images?20:44
thrashbkero: tbh, I'm not sure which ones do.20:44
thrashwas gonna ask trown|outtypewww but...20:45
trozetmwhahaha: so i just added you to the cherry pick for newton20:45
trozetmwhahaha: and quoted it there20:45
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Containerize Redis service  https://review.openstack.org/44215120:45
trozetmwhahaha: however, stable/ocata was already merged.  So should I submit a patch to master to quote it and then backport that to stable ocata, can i just propose a change directly to ocata to quote it?20:45
*** paramite has quit IRC20:46
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: [WIP] collect-logs: add rename task for uncompressed files too  https://review.openstack.org/44316320:51
bkerobnemec: Since it's intermittent like it is, I wonder if it's a race condition.20:51
*** ealcaniz has joined #tripleo20:55
*** karimb has quit IRC20:55
*** hjensas has quit IRC20:55
thrashbkero: did the cli to disk-image-create change at all?20:59
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: Create repos configs for stable branches tripleoci  https://review.openstack.org/43950321:01
*** ealcaniz has quit IRC21:01
*** b00tcat has joined #tripleo21:02
mwhahahatrozet: ocata should be puppet 421:02
trozetmwhahaha: really? dciabrin just checked his ocata deployment and said it was 3.821:02
*** eck` is now known as eck`gone21:03
mwhahahatrozet: pretty sure it's supposed to be puppet 421:03
trozetmwhahaha: oh it is it looks like..im on his setup21:03
*** jprovazn has quit IRC21:03
bkerothrash: I don't know21:03
dciabrintrozet, mwhahaha I checked the available packages in http://mirror.centos.org/centos/7/cloud/x86_64/openstack-ocata/common/21:03
*** Goneri has joined #tripleo21:04
apevecdciabrin, it's not in common21:04
dciabrinbut yeah, trozet's right, my env is ocata and puppet-4 is installed21:04
dciabrinapevec, oh!21:04
apevecsince older rdo releases are not puppet4 ready21:05
dciabrinapevec, makes sense indeed, thx21:05
apevecit's one level up, in release specific tag21:05
trozetmwhahaha: well then something is broken with puppet 4?21:05
thrashbkero: tripleo-build-image's interfacing with diskimage-builder stops at 'disk-image-create'. It appears to me that this issue lies within dib itself.21:05
mwhahahatrozet: link to error message?21:05
apeveccurrently 4.6.2 http://cbs.centos.org/koji/buildinfo?buildID=1272121:06
trozetmwhahaha: https://paste.fedoraproject.org/paste/OuBKl-iS0w7A1VI13KYSZV5M1UNdIGYhyRLivL9gydE=/raw21:06
apevecbut we have 4.8 candidate, not sure what was blocking it http://cbs.centos.org/koji/buildinfo?buildID=1530521:06
dciabrinapevec, got it on http://mirror.centos.org/centos/7/cloud/x86_64/openstack-ocata/ thx21:06
thrashbkero: perhaps in the pkg-map element?21:06
bkerothrash: That's where I was looking before, looking back at it again21:07
*** thrash is now known as thrash|biab21:07
*** salmankhan has joined #tripleo21:08
mwhahahatrozet: might be a puppet issue it's ok on 4.921:09
dciabrinapevec, and it's puppet-3.8.7-2.el7 on newton and older, correct?21:09
trozetmwhahaha: is master OOO on puppet 4.9?21:09
mwhahahatrozet: no but my local env is21:09
trozetmwhahaha: crap21:09
mwhahahatrozet: so propose a fix with quotes for ocata/master21:09
trozetmwhahaha: ok will do21:10
apevecdciabrin, yes, from common http://cbs.centos.org/koji/buildinfo?buildID=1142521:10
apevecmwhahaha, so we need to update puppet to 4.9  ?21:10
*** dprince has quit IRC21:10
mwhahahaapevec: eventually, but in this case we should just quote the data21:11
apevecok21:11
mwhahahatrozet: https://docs.puppet.com/puppet/4.9/lang_data_string.html#bare-words21:11
trozetwell thanks dciabrin: for testing this out and finding these issues21:11
apevecI've tagged 4.8.2 build from Jan in common-pending, please test it21:11
mwhahahatrozet: i think it's because it starts with a capital L21:11
mwhahahayea that's the problem21:12
mwhahahaTIL21:12
trozetmwhahaha: ah!21:12
mwhahahaUnquoted words that begin with upper case letters are interpreted as data types or resource references, not strings.21:12
trozetmwhahaha: damn..21:12
dciabrinoh wow21:12
mwhahahatrozet: in the future cover with spec test21:12
mwhahahait would have uncovered this21:12
* mwhahaha shakes fist21:12
trozetmwhahaha: but in 4.9 it doesn't happen?21:12
mwhahahatrozet: no it still happens21:12
trozetmwhahaha: when you just tested it21:12
mwhahahai wasn't testing that combination21:13
*** salmankhan has quit IRC21:13
mwhahahahttp://paste.openstack.org/show/601989/21:13
trozetmwhahaha: do you want me to add a test when i submit this patch to master to fix the quotes?21:13
apevecso moral: quoting is good ?21:13
mwhahahayes quoting is good21:13
mwhahahatrozet: yes it would be a good idea21:13
*** liupe__ has quit IRC21:13
trozetmwhahaha: ok will do, thanks21:14
* mwhahaha will start -1 anything without tests that can be tested easily21:14
bnemecbkero: Hmm, this has been happening a lot today: http://logstash.openstack.org/#dashboard/file/logstash.json?query=build_name%3A%20*tripleo-ci*%20AND%20build_status%3A%20FAILURE%20AND%20message%3A%20%5C%22get_image_element_array%3A%20command%20not%20found%5C%2221:15
bkerobnemec: So it does21:17
bnemecHmm, I wonder if that's a red herring.  Maybe we get that particular message in every multinode job.21:17
*** jayg is now known as jayg|g0n321:17
bkerooh?21:18
bnemecThe thing that actually fails is the next script that gets run, on the AttributeError.21:19
bkerobnemec: I don't see it here: http://logs.openstack.org/43/409143/17/check/gate-tripleo-ci-centos-7-nonha-multinode/ccad6ce/logs/var/log/bootstrap-subnodes.txt.gz21:19
bnemecAlthough I'm getting the same number of results for that query, so they're probably related.21:19
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Set multinode scenarios with quickstart  https://review.openstack.org/44320421:20
bkerobnemec: this sounds like what happens when one of the element arrays would be blank21:20
*** lblanchard has quit IRC21:20
mwhahahait's something new cause my undercloud change is now throwing this error and it wasn't previously21:21
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: Deploy multinode scenarios with deploy args  https://review.openstack.org/44318821:21
mwhahahahttp://logs.openstack.org/43/442143/1/check/gate-tripleo-ci-centos-7-nonha-multinode/6b95dae/console.html#_2017-03-08_18_43_07_09107721:22
bkerobnemec: args_to_elements is only called after disk-image-create checks that at least 1 element has been specified21:24
bkeroAlthough it mixes $* and $@21:25
bkerobnemec: if this gets bad enough I can add a DIB_DEBUG_TRACE and see if that makes a diff, but the turnaround time for that is long21:27
bkeroDo you suppose it's the IMAGE_ELEMENT_YAML that's being referenced in the traceback?21:29
*** salmankhan has joined #tripleo21:29
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: test pingtest with scenarios  https://review.openstack.org/44332721:29
bnemecbkero: Oh, this is interesting.  The passing job has COMMON_ELEMENTS_PATH=/usr/share/diskimage-builder/elements21:30
bnemecThe failing one is COMMON_ELEMENTS_PATH=/usr/lib/python2.7/site-packages/diskimage_builder/elements21:31
bkeroI just checked overcloud-images.yaml and overcloud-images-centos7.yaml, but as expected both apssed a linter21:31
bkerohmm21:31
bkeroWe don't back up /usr from jobs, so can't really inspect that :<21:31
bnemecbkero: I don't think this is actually happening in tripleo-build-images.  I think that's doing its job.  It's the instack call that immediately follows that is failing.21:32
bnemecI wonder if there's an assumption in instack somewhere that the elements live at /usr/share/diskimage-builder21:32
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DON'T MERGE: testing pingtest scenario  https://review.openstack.org/44332821:33
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: test pingtest with scenarios  https://review.openstack.org/44332721:34
bkerobnemec: I think that's it21:34
bkerobnemec: look at this21:34
bkerohttp://git.openstack.org/cgit/openstack/instack/tree/instack/main.py#n10021:34
bkeroHmm, that's for lib though, not elements\21:35
bnemecbkero: Yep, that looks promising.21:35
bkeroThere's also http://git.openstack.org/cgit/openstack/instack/tree/instack/main.py#n4421:36
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates stable/ocata: Adds upgrade tasks for OpenDaylight services  https://review.openstack.org/44332921:37
openstackgerritJoe Talerico proposed openstack/puppet-tripleo master: Tuned should be configured properly  https://review.openstack.org/43769821:37
bkerobnemec: I don't see how COMMON_ELEMENTS_PATH is sourced in21:37
bnemecbkero: It's just added to ELEMENTS_PATH directly: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/bootstrap-overcloud-full.sh#n4621:38
bkeroAh, yes it's there21:38
*** egafford has quit IRC21:41
*** leanderthal|afk has quit IRC21:45
*** ansmith has quit IRC21:47
*** flepied has quit IRC21:50
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: tripleo docs have overcloud image build instructions, add that option  https://review.openstack.org/44329421:54
*** ccamacho has quit IRC21:55
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: Add possibility to clean up image cache directory  https://review.openstack.org/43919822:02
apevecany containers folks around? How do I use/apply contrib/tripleo_kolla_template_overrides.j2 ?22:04
apevecI've have not found an example usage in tripleo-ci22:05
*** eck`gone is now known as eck`22:06
stevebakerapevec: we use this wrapper script https://github.com/dprince/undercloud_containers/blob/master/build_kolla.sh22:08
*** thrash|biab is now known as thrash22:08
openstackgerritAlex Schultz proposed openstack/puppet-tripleo stable/newton: Stop the chronyd service  https://review.openstack.org/44333322:08
apevecstevebaker, thanks! Could that be moved/documented in tripleo-common ?22:08
stevebakerapevec: I don't think the script is needed, but we should document a kolla-build example in tripleo-common22:09
apevecthat works22:09
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/167126222:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1671262 in tripleo "Multinode jobs fail with "get_image_element_array: command not found"" [Critical,Triaged]22:10
apevecstevebaker, this is for experiment mentioned last day of ptg, to run kolla-build during tripleo-common rpm build22:10
*** ccamacho has joined #tripleo22:11
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates stable/newton: Deploy NTP with puppet-tripleo profile  https://review.openstack.org/44333422:11
stevebakerapevec: ah. can you run kolla-build with --template-only? No need to build the images I think. Also I really think we shouldn't package the resulting templates22:12
apevecyeah, just templates22:12
apevecwhy not?22:12
apevecanyway, it's an experiment, we'll see if it makes sense or not22:14
apevecstevebaker, review will tell, and you'll be in CC :)22:14
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo master: Restrict mongodb memory usage  https://review.openstack.org/41909022:14
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: [WIP] collect-logs: add rename task for uncompressed files too  https://review.openstack.org/44316322:16
*** akrivoka has quit IRC22:17
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: high level design doc for tripleo quickstart  https://review.openstack.org/44148022:17
stevebakerapevec: I don't think there will be a general set of templates useful to all users. Whatever we produce will be quite opinionated. The changes done by kolla_template_overrides.j2 are a good example of that22:17
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102922:18
apevecstevebaker, it would be templates for tripleo, in openstack-tripleo-common-containers subpkg22:18
apevecnot in openstack-kolla22:18
apevecit will be easier to see once I have patch ready for review tomorrow22:19
*** kjw3 has quit IRC22:20
*** salmankhan has quit IRC22:21
*** b00tcat has quit IRC22:21
*** artom has quit IRC22:21
openstackgerritMichael Johnson proposed openstack/diskimage-builder master: squash-package-install to use the correct python  https://review.openstack.org/44333622:22
stevebakerapevec: ah, that would be alright. The next challange would be to build images from those templates - I can't see a way to do that with kolla-build currently22:22
slaglebnemec: bkero : pretty sure it has something to do with new diskimage-builder22:23
*** jcoufal_ has quit IRC22:23
bnemecslagle: Yeah, it looks like dib hasn't been building properly in delorean for a while, so it's anybody's guess when the breakage actually happened. :-/22:24
slaglebnemec: yep, exactly22:24
*** jcoufal has joined #tripleo22:24
*** chlong_ has quit IRC22:24
bnemecThe logs don't even go back far enough to find the last successful build.22:24
*** jcoufal has quit IRC22:24
slagleso i suspect this: https://review.openstack.org/#/c/431237/22:24
slaglewhich is a bunch of stuff22:24
slaglebnemec: actually if you look at the build's repo right before the diskimage-builder pass, it has this version: diskimage-builder-1.27.1-0.20170302015627.65c6da5.el7.centos.noarch.rpm22:25
slagleso i guess march 2nd22:25
slaglemaybe not that big commit then22:25
bnemecAh, good point.22:25
openstackgerritMerged openstack/instack master: Remove link to modindex  https://review.openstack.org/42606022:26
*** artom has joined #tripleo22:27
*** salmankhan has joined #tripleo22:27
apevecDIB has merged v2 feature branch I think today?22:27
apevecwe had to pin it to 1.27 in stable branches22:28
apevechttps://review.rdoproject.org/r/565322:29
apeveccorrection 1.2822:29
*** flepied has joined #tripleo22:29
slagleyes, something with the huge merge22:30
bnemecExcept we're still getting 1.28 in ci.22:30
bnemecAnd I don't see the big scary merge v2 commit yet.22:30
apevecyou'll get it in next current-tripleo promotion I guess22:31
bnemecI don't think we promote dib.22:31
*** rwsu has joined #tripleo22:31
bnemecIt's supposed to be gated on tripleo, although I don't think we run the multinode jobs on it.  Which is something we should fix after this.22:31
apevecah you take it from "current"22:32
bnemecSo the last successful build before today was https://github.com/openstack/diskimage-builder/commit/65c6da58c3498c92633860dccdc506bd15b5912c22:32
bnemecThe next commit after that was https://github.com/openstack/diskimage-builder/commit/f1776d72aad14140d4bbf2be94729923eacc9fcd22:33
bnemecThe only really suspicious thing I see there is the rather broken-looking change to 04-install-pip, which affects centos7 for reasons I can't fathom at the moment.22:34
bnemec(opensuse}fedora|centos|centos7|rhel|rhel7)22:34
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921522:34
bnemecPretty sure that should be s/}/|/22:34
bnemec\o/ regexes22:34
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102922:35
mwhahahaso where is the undercloud stackrc generated?22:35
* mwhahaha is lost22:35
slaglebnemec: have you heard of my new distro?22:35
slaglebnemec: i'm calling it opensuse}fedora22:35
* bnemec thinks about taking up a new career in trademark law22:36
*** apevec has quit IRC22:37
*** dougbtv has quit IRC22:37
bnemecmwhahaha: It's an os-apply-config template22:37
bnemechttps://github.com/openstack/instack-undercloud/blob/master/elements/undercloud-install/os-apply-config/root/stackrc22:37
ianwbnemec: so i think what has to happen is that instack needs to merge it's feature/v2 branch to master to work with dib master22:39
ianwis that something you can do?22:39
*** apevec has joined #tripleo22:39
mwhahahabnemec: thanks. I guess I should have used find instead of grep :D22:39
ianwthe tripelo tests are passing on dib's feature/v2 branch22:39
apevecbnemec, so DLRN generated diskimage-builder-1.28.1-0.20170308163502.958ea8a.el722:40
apevecb/c 1.28.0 is what PBR sees as latest tag from current master head22:40
apevec python ./setup.py --version22:40
apevec1.28.1.dev522:40
apevecbut22:40
apevec git describe22:41
apevec2.0.0rc1-13-g958ea8a22:41
apevecPBR is super confused with those merges22:41
*** karthiks has quit IRC22:42
apevecianw, ^22:42
ianwapevec: hmm, 2.0.0rc1 was tagged on the feature/v2 branch22:42
* mwhahaha has version flashbacks and RU a away22:42
apevecianw, yes, and that branch was merged to master, right?22:42
ianwyes22:42
bnemecianw: I don't think that's it.  There's only one commit on instack's v2 branch, and it's already in master too.22:42
apevecanyway, commit is right, 958ea8a is current master head22:43
ianwapevec: if it helps, I can tag 2.0.022:43
apevecbnemec, so ignoring version part, you have latest DIB in rdo trunk master22:43
apevecianw, it would, just to clear confusion22:43
ianwok, i was going to wait on that incase we had to slip anything else in before a 2.0.0 release, but it seems to make more problems than it helps22:44
apevecianw, you could push rc222:44
*** jmelvin has quit IRC22:44
ianwwell since it's on master it's not really rc any more :)22:45
ianwi think at this point we just move to point releases22:45
openstackgerritOpenStack Proposal Bot proposed openstack/instack master: Updated from global requirements  https://review.openstack.org/43195122:45
apevecwhatever you prefer, in projects following cycle, final rc is retagged as final22:46
apevecso you could have rc2 if you want to wait some more22:46
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Containerize Redis service  https://review.openstack.org/44215122:46
*** ccamacho has quit IRC22:46
apevecthen just retag as 2.0.022:46
ianwapevec: alright, sounds reasonable to do rc222:47
ianwok, i have pushed a 2.0.0rc2 tag, hopefully that helps pbr22:48
ianwnow, back to the instack issue ... if we don't think it's a feature/v2 merge22:49
slaglei think it is22:50
slagleit's got to do with the changes in element_dependencies.py22:51
*** apevec has quit IRC22:53
*** karthiks has joined #tripleo22:54
slaglei think we probably just ought to pin to 1.28.0 in dlrn trunk for now22:54
ianwi'm sorry, i did think that the dib jobs were using from master, but others used requirements22:55
ianwthat was why i was not surprised the tripleo dib jobs failed on master after the merge, and i thought we had a window to fix that22:55
bnemecianw: We were missing some test coverage on dib.  We run an image building job, but not a multinode job where instack gets used in a unique way.22:56
ianwok, this might explain why i'm not able to correlate the error between the multinode job and the failing dib jobs22:57
slaglebnemec: even on the ovb jobs we run instack to install the undercloud, where i think we would hit this22:58
slaglebut maybe not22:58
bnemecslagle: I'm only seeing it in multinode jobs: http://logstash.openstack.org/#/dashboard/file/logstash.json?query=build_name:%20*tripleo-ci*%20AND%20build_status:%20FAILURE%20AND%20message:%20%5C%22Cannot%20allocate%20memory%5C%2222:58
slagleif the undercloud install is passing, we could merge this: https://review.openstack.org/#/c/425888/22:58
bnemecWhoops, that's the wrong url22:59
bnemechttp://logstash.openstack.org/#dashboard/file/logstash.json?query=build_name%3A%20*tripleo-ci*%20AND%20build_status%3A%20FAILURE%20AND%20message%3A%20%5C%22get_image_element_array%3A%20command%20not%20found%5C%2222:59
slaglewhich eliminates instack from bootstrapping subnodes22:59
slaglelet me recheck it22:59
slagleif it passes, we could merge it to "fix"22:59
ianwfor this error -> http://logs.openstack.org/95/442595/4/experimental/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-nv/e375def/console.html#_2017-03-08_18_43_55_41201622:59
ianwthe "arg_to_elements" function needs to be called to provide that variable in the environment -> https://git.openstack.org/cgit/openstack/diskimage-builder/tree/diskimage_builder/lib/common-functions#n25323:00
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102923:00
slagleianw: ok, i'm not sure how to do that23:00
slagleianw: instack calls dib-run-parts to run the hooks23:00
slagleso...maybe we could string a call together to call arg_to_elements first23:01
ianwwell presumable instack should call it before running elements ... this is part of the dib standard environment23:01
ianwinstack must be setting up a bunch of other stuff disk-image-create does too, looking...23:01
slagleyes, it sets up an environment23:02
slagleputting stuff into os.environ before calling dib-run-parts23:02
slaglenot sure you can wedge a bash fuction in there though :)23:02
slaglebnemec: maybe the undercloud install is passing b/c pkg-map/svc-map aren't used23:03
slaglebnemec: anyway, keep an eye on https://review.openstack.org/#/c/425888/23:04
slagleif that passes, we should just merge that23:04
slaglegotta go, but will check back later23:04
bnemecslagle: Yeah, although that will still leave multinode-upgrades broken.23:04
bnemecBecause of https://review.openstack.org/#/c/425888/13/toci_gate_test.sh23:04
ianwthere is a function to create these env vars -> https://git.openstack.org/cgit/openstack/diskimage-builder/tree/diskimage_builder/element_dependencies.py#n29323:04
ianwin theory we could make a version to populate os.environ with them i guess23:05
ianwcurrently they are printed, assuming bash will eval them because disk-image-create is doing the driving23:05
slaglei guess we should just capture stdout of _output_env_vars to a tmp file, and source it in the call to dib-run-parts23:05
ianwslagle: well the interface to get them would be to call "element-info --env $ELEMENTS"23:08
ianwthat could be done before dib-run-parts23:08
ianwyou basically need to eval() the output of that command and it sets up the environment vars that describe [[element,path],...]23:09
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/167126223:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1671262 in tripleo "Multinode jobs fail with "get_image_element_array: command not found"" [Critical,Triaged]23:10
ianwwe did this because we had multiple elements that would walk the element list and none of them supported overriding elements in the same way23:10
*** salmankhan has quit IRC23:10
ianwnow we have a canonical list of elements and path to the element, and we leave all processing up to element-info23:10
*** dmarlin has quit IRC23:10
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102923:10
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart-extras master: [WIP] collect-logs: add rename task for uncompressed files too  https://review.openstack.org/44316323:11
ianwok, instack is using expand_dependencies()23:13
bkeroslagle: so are you going forth with the removing-instack-from-multinode approach?23:13
ianwahh, that's right, we fiddled with that23:13
bkeroThat would fix it for me, although might not fix the more general case23:13
ianwwhat should really happen here is that dib should have a python-based element runner that instack could call instead of doing it itself23:15
ianwthat way we could func test that in the dib gate and be sure we're not breaking it, and have an api23:15
bnemecbkero: Yeah, we're rechecking https://review.openstack.org/#/c/425888/ to see if it fixes this.23:15
bnemecianw: I like the sound of that. :-)23:16
ianwbnemec: i think i can roughly see what needs doing in instack/runner.py .  i'm not clear though; does it need fixing or is it worked around?23:18
bnemecianw: Actually, thinking some more about it https://review.openstack.org/#/c/425888/ might fix all the jobs.  In the multinode-upgrades job I was worried about we'll deploy the previous stable release first.23:20
*** cdearborn has quit IRC23:20
bnemecWhich won't have dib 2.0, so the bootstrap should work.23:20
*** saneax-_-|AFK is now known as saneax23:21
openstackgerritDan Sneddon proposed openstack/tripleo-specs master: blueprint tripleo-routed-networks-deployment  https://review.openstack.org/42100923:22
*** dparkes has quit IRC23:22
bnemecianw: So I think at this point we just need to wait for results on that patch.  If it passes everything then the problem is worked around.23:22
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102923:22
bkerobnemec: That looks like it should work for me23:23
ianwbnemec: alright, so that gives us time to fix instack/dib223:24
*** morazi has quit IRC23:25
* bnemec is going to wait for ci results elsewhere23:25
bnemecWill check back later tonight23:25
bkerobnemec: let me know since I have some work that's dependent on this23:27
*** pmannidi has joined #tripleo23:27
bkeroI'm eager in seeing this merged asap, so need someone around for +W whenever jobs return23:27
*** rlandy is now known as rlandy|bbl23:31
*** bfournie has quit IRC23:35
*** Goneri has quit IRC23:38
openstackgerritDan Sneddon proposed openstack/tripleo-specs master: blueprint tripleo-routed-networks-ironic-inspector  https://review.openstack.org/42101123:39
openstackgerritDan Sneddon proposed openstack/tripleo-specs master: Spec: tripleo-routed-networks-deployment - Routed Spine and Leaf  https://review.openstack.org/42100923:47
openstackgerritDan Sneddon proposed openstack/tripleo-specs master: blueprint tripleo-routed-networks-ironic-inspector  https://review.openstack.org/42101123:47
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: DO NOT MERGE -1 WIP tripleo docs have overcloud image build instructions, add that option  https://review.openstack.org/44329423:49
*** pradk has quit IRC23:55
*** david-lyle has quit IRC23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!