Monday, 2016-09-19

*** flepied has joined #tripleo00:19
*** limao has joined #tripleo00:48
*** akshai has joined #tripleo01:14
*** rbrady_ has joined #tripleo01:50
*** rbrady has quit IRC01:52
*** akshai_ has joined #tripleo01:56
*** akshai has quit IRC01:59
*** michchap has joined #tripleo01:59
*** coolsvap has joined #tripleo02:01
*** akshai_ has quit IRC02:03
*** akshai has joined #tripleo02:03
*** akshai has quit IRC02:09
*** akshai has joined #tripleo02:16
*** akshai has quit IRC02:17
*** akshai has joined #tripleo02:17
*** akshai has quit IRC02:26
*** bana_k has joined #tripleo03:03
*** dmacpher has joined #tripleo03:15
*** rbrady_ has quit IRC03:52
*** dmacpher has quit IRC04:04
*** rbrady_ has joined #tripleo04:06
*** dmacpher has joined #tripleo04:17
*** limao has quit IRC04:29
*** oshvartz has quit IRC04:33
*** davidlenwell has quit IRC04:37
*** phpcodemonkey has joined #tripleo04:40
*** davidlenwell has joined #tripleo04:49
*** ChanServ sets mode: +v davidlenwell04:49
*** dmacpher has quit IRC04:57
*** dmacpher has joined #tripleo04:57
*** limao has joined #tripleo04:59
*** dmacpher has quit IRC05:01
*** pcaruana has quit IRC05:05
*** s_kramaja_ has joined #tripleo05:09
*** s_kramaja_ is now known as skramaja05:09
*** jaosorior has joined #tripleo05:11
*** dmacpher has joined #tripleo05:18
*** flepied has quit IRC05:19
*** flepied has joined #tripleo05:19
*** florianf|afk has quit IRC05:31
bandinimorning05:32
jaosoriorhey dude05:33
jaosoriorhow's it going?05:33
*** florianf has joined #tripleo05:39
*** mcornea has joined #tripleo05:40
jaosoriormcornea: hey dude, do you use quickstart to setup your deployment?05:42
mcorneajaosorior: hey, morning. no, I use infrared: https://github.com/rhosqeauto/InfraRed05:44
jaosoriorwow05:45
jaosoriorhadn't heard of that one05:45
mcorneawe use it in the qe group05:45
jaosoriormcornea: is it based on instack?05:46
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375005:46
mcorneajaosorior: no, it's based on ansible, wrapping libvirt05:47
jaosoriorIs ee05:48
jaosorior* I see05:48
*** apetrich has joined #tripleo05:54
*** oshvartz has joined #tripleo05:57
*** ccamacho has joined #tripleo05:58
*** flepied has quit IRC05:59
*** rcernin has joined #tripleo06:03
*** pgadiya has joined #tripleo06:05
ccamachomorning folks!!06:09
*** rasca has joined #tripleo06:11
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Add a function to upgrade from full HA to NG HA  https://review.openstack.org/35862606:12
*** pcaruana has joined #tripleo06:17
*** aufi has joined #tripleo06:26
*** bana_k has quit IRC06:28
*** dsariel has joined #tripleo06:29
tobias-fiberdatajaosorior, could you help out? getting this error right after the are being uploaded. http://paste.openstack.org/show/580908/06:30
tobias-fiberdataright after the plan*06:30
*** aufi has quit IRC06:30
*** jprovazn has joined #tripleo06:32
jaosoriortobias-fiberdata: haven't seen that one, but surely seems like there's a tab in one of the files you're using06:32
tobias-fiberdataso yaml is very sensitive in that way?06:33
*** fzdarsky has joined #tripleo06:33
tobias-fiberdatajust like python?06:33
tobias-fiberdatai'm a noob of those errormessages, i'm just bad at google aswell it seems06:34
tobias-fiberdatamy collegue is fixing it up06:34
openstackgerritAlexandre Maumené proposed openstack/tripleo-docs: Add how to fix firewall not adding the correct rules with node's MAC address  https://review.openstack.org/37224806:34
*** flepied has joined #tripleo06:36
*** matbu|brb is now known as matbu06:39
tobias-fiberdatajaosorior, why havent i/we used --debug before. can easily get more info that way06:45
*** dmacpher has quit IRC06:49
jaosoriorccamacho, marios hey guys, can you check this commit out? https://review.openstack.org/#/c/371903/06:51
ccamachojaosorior, sure man06:51
jaosoriorccamacho: congrats on the core nomination dude! well deserved06:52
ccamachojaosorior thanks man xD06:52
mariosjaosorior: will do06:54
mariosmorning  *06:54
marios:/ monday06:54
openstackgerritAlexandre Maumené proposed openstack/tripleo-docs: add deployment diagram  https://review.openstack.org/37225707:04
*** jbadiapa has joined #tripleo07:07
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375007:08
*** liverpooler has joined #tripleo07:10
*** coolsvap is now known as _coolsvap_07:13
*** masco has joined #tripleo07:14
*** phpcodemonkey has quit IRC07:24
*** ebarrera has joined #tripleo07:24
*** b00tcat` has joined #tripleo07:25
b00tcat`hi, when building overcloud-full from scratch how does os-apply-config get installed into the chrooted environment? or where should it be installed?07:26
b00tcat`basic building of overcloud-full can't find the command07:26
b00tcat`https://paste.fedoraproject.org/430884/74269899/07:26
*** ohamada has joined #tripleo07:27
*** hjensas has joined #tripleo07:27
*** jlinkes has joined #tripleo07:27
*** jtomasek has joined #tripleo07:28
*** ccamacho has quit IRC07:28
*** akuznetsov has joined #tripleo07:30
*** jpich has joined #tripleo07:33
*** ccamacho has joined #tripleo07:34
*** anshul has joined #tripleo07:38
openstackgerritmathieu bultel proposed openstack/tripleo-quickstart: WIP gate upgrade  https://review.openstack.org/34216107:40
shadowermarios: it's monday here too :-(07:40
*** zoli_gone-proxy is now known as zoliXXL07:44
*** zoliXXL is now known as zoli|trng07:44
mariosshadower: it has a nasty tendency of spreading like that07:46
openstackgerritBabu Shanmugam proposed openstack/puppet-tripleo: WIP - Depends on the OVS patches [1] to be merged.  https://review.openstack.org/37227407:46
shadoweryea, like an epidemic :/07:46
d0ugallol07:46
* d0ugal doesn't mind Monday07:47
d0ugalSend all your Mondays here07:47
mariosclearly this poor soul has lost the plot completely some kind of stockhold syndrome going on here clearly ^^^ shadower i think an intervention is in order07:47
mariosstockholm even07:48
shadoweryea07:48
*** radeks has joined #tripleo07:52
*** zoli|trng is now known as zoli_gone-proxy07:54
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Add an optional timeout when waiting for websocket messages  https://review.openstack.org/36425207:56
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Pass the timeout to the deploy workflow  https://review.openstack.org/37018607:56
openstackgerritDougal Matthews proposed openstack/tripleo-common: Remove the old, deprecated Mistral action names.  https://review.openstack.org/36652907:57
*** jpena|off is now known as jpena07:59
*** pblaho has joined #tripleo07:59
openstackgerritDougal Matthews proposed openstack/instack-undercloud: Verify that the Deployment Plan creation was successful  https://review.openstack.org/36924708:00
*** zoli_gone-proxy is now known as zoliXXL08:01
*** dmacpher has joined #tripleo08:02
*** shardy has joined #tripleo08:04
*** paramite has joined #tripleo08:04
openstackgerritDougal Matthews proposed openstack/tripleo-common: Fix the default plan creation  https://review.openstack.org/37134708:05
openstackgerritDougal Matthews proposed openstack/tripleo-common: Return the result of create_plan in create_deployment_plan workflow  https://review.openstack.org/37134808:05
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add template processing to the update plan workflow.  https://review.openstack.org/37102708:05
therved0ugal, I suspec that  https://review.openstack.org/369247 would work around the timeout bug too08:06
therve(Although it'd be really nice if it used the tripleoclient call, but I don't know if it can)08:06
d0ugaltherve: Yeah, I think so - but there is a good chance we will hit is somewhere else.08:06
therveSure08:06
d0ugaltherve: I wanted to switch to tripleoclient too, but I wasn't really sure how :)08:06
*** gfidente has joined #tripleo08:07
skramajashardy: i have a bug on os-net-config. i am confused on how to assign the milestone for tripleo rc2 - https://bugs.launchpad.net/os-net-config/+bug/161933008:08
openstackLaunchpad bug 1619330 in os-net-config "os-net-config fails on 2nd run with DPDK net config" [High,In progress] - Assigned to Saravanan KR (skramaja)08:08
skramajashardy: should it be on tripleo project or os-net-config project?08:09
shardyskramaja: marking it as affects both (as you have done) is fine08:09
shardyskramaja: we're trying to consolidate most tracking on the main tripleo launchpad project, so for release tracking we use that08:10
*** beagles has quit IRC08:13
*** b3nt_pin has joined #tripleo08:13
*** b3nt_pin is now known as Guest6457408:13
*** pblaho has quit IRC08:14
*** pblaho has joined #tripleo08:15
shardyskramaja: thanks for the fix, approved08:15
shardyskramaja: I made one comment that we should proably document the dpdk mapping format somewhere, if it's not already08:15
shardyI couldn't see any sample in the os-net-config repo, and I guess not everyone will see that comment08:16
jaosoriorshardy: hey dude, could you check this one out? https://review.openstack.org/#/c/371903/08:19
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Remove the previous plan when deploying  https://review.openstack.org/37146808:19
shardyjaosorior: Looks good, just one question, if someone deployed without OS::TripleO::Services::NovaVncProxy, would those settings break, or just be ignored?08:25
skramajathanks shardy. i will work on the follow-up patch for sample and the docs..08:25
shardyI'm wondering if at some point we might want those decoupled from the main compute template, but landing it as is seems OK for now08:25
skramajashardy: there are 2 patches for this bug - https://review.openstack.org/#/c/364354/ is yet to be reviewed. can you check this also?08:26
jaosoriorshardy: so initially I had that decoupled to another service profile08:26
*** athomas has joined #tripleo08:26
jaosoriorbut I was -1ed because of it and told to add it to the compute profile08:26
shardyskramaja: ack, will do08:27
shardyjaosorior: Ok, I don't see that in the review comments on that patch08:27
jaosoriorshardy: that's because it was a different patch, let me fetch that08:27
jaosoriorshardy: https://review.openstack.org/#/c/370331/08:28
shardyjaosorior: ack, I linked it and approved08:30
jaosoriorshardy: thanks08:30
jaosoriorshardy: is there any work left for the composable roles?08:31
shardyjaosorior: No, all the patches landed, so the remaining work is adding CI coverage and docs (along with fixing any bugs exposed by those)08:32
jaosoriorshardy: awesome08:33
jaosoriorCan someone review this docs patch? https://review.openstack.org/#/c/368601/08:33
shardyjaosorior: FYI we should release rc1 today: https://review.openstack.org/#/c/371897/08:34
shardy(including the composable roles stuff)08:34
jaosoriorshardy: does that include branching newton?08:35
shardyjaosorior: yes08:35
*** radeks has quit IRC08:35
jaosoriorshardy: so, do I have to cherry-pick the patch you just approved?08:35
shardyjaosorior: Yes you probably will, after the stable/newton branches are created08:36
shardyunless we end up rebasing the release patch I just linked for any reason08:36
shardyWe'll prune the backlog here to decide what needs to get backported before we declare newton final:08:37
shardyhttps://launchpad.net/tripleo/+milestone/newton-rc208:37
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Pass the timeout to the deploy workflow  https://review.openstack.org/37018608:38
*** derekh has joined #tripleo08:39
*** osp has joined #tripleo08:41
*** saneax-_-|AFK is now known as saneax08:42
openstackgerritDougal Matthews proposed openstack/tripleo-common: Fix the default plan creation  https://review.openstack.org/37134708:47
openstackgerritDougal Matthews proposed openstack/tripleo-common: Return the result of create_plan in create_deployment_plan workflow  https://review.openstack.org/37134808:47
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add template processing to the update plan workflow.  https://review.openstack.org/37102708:47
*** pmannidi has quit IRC08:47
openstackgerritDougal Matthews proposed openstack/instack-undercloud: Verify that the Deployment Plan creation was successful  https://review.openstack.org/36924708:48
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Print the execution ID when starting Mistral workflows  https://review.openstack.org/34071108:49
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Remove the environments from Mistral when removing from Swift  https://review.openstack.org/36948608:49
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Use a released version of tripleo-common  https://review.openstack.org/36442508:49
openstackgerritDougal Matthews proposed openstack/tripleo-common: Remove the unused service_host arg from node registration  https://review.openstack.org/32603608:49
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Update the Mistral action names  https://review.openstack.org/36651908:50
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Remove openstackclient imports in the new parameters command  https://review.openstack.org/37159408:50
*** yolanda has joined #tripleo08:53
*** radeks has joined #tripleo08:58
jaosoriorshardy: by the way, shouldn't there be a file for t-h-t here too? https://review.openstack.org/#/c/371897/309:03
*** hewbrocca-afk is now known as hewbrocca09:03
tobias-fiberdataCould someone help me interpret these errors? http://paste.openstack.org/show/580936/09:04
shardyjaosorior: gah, yeah -enocoffee09:04
shardythanks for spotting that09:04
jaosoriorshardy: haha in that case hopefully I don't need to cherry-pick that :P09:05
shardyjaosorior: Yeah, possibly, depends which CI run EmilienM is using to base the release from09:06
jaosoriorah, true that09:06
jaosorioraaanyway09:06
jaosoriorno biggie09:06
shardyYeah, bugs aren't such a big deal, as we can continue backporting them after the final release09:06
shardyat least all the features now landed :)09:07
jaosoriortrue that09:07
*** tosky has joined #tripleo09:09
jaosoriorshardy: hey dude, by the way, can you check this out https://review.openstack.org/#/c/370623/ ? Would hopefully enable us to have SSL for zaqar websockets, since that seems to be having proxy problems09:09
openstackgerritMerged openstack/tripleo-common: Handle missing mistral environment  https://review.openstack.org/36590709:09
openstackgerritBabu Shanmugam proposed openstack/puppet-tripleo: WIP - Depends on the OVS patches [1] to be merged.  https://review.openstack.org/37227409:10
*** mcornea has quit IRC09:10
*** mcornea has joined #tripleo09:10
*** stendulker has joined #tripleo09:13
*** masco has quit IRC09:13
hewbroccaGood morning everyone09:14
jaosoriorhewbrocca: hey dude, how's it going?09:15
jaosoriortobias-fiberdata: have you checked mistral execution-list?09:15
hewbroccajaosorior: it's great. Happy to be back after the vacation09:15
tobias-fiberdatajaosorior, i'll take a look09:15
shardyHey hewbrocca, welcome back09:16
tobias-fiberdatajaosorior, how do i take a closer look into the workflow name that fails?09:18
hewbroccaHow's everyone feeling about Newton this fine Monday?09:18
d0ugalScared :)09:19
hewbroccaGood, if you weren't scared I would be really worried :D09:19
tobias-fiberdatajaosorior, think i found the errors09:19
shardyhewbrocca: pretty good - we had to slip the RC1 release due to CI issues (surprise!), but we should cut it today09:19
tobias-fiberdatajaosorior, think i found the cause even09:19
hewbroccaWell, that is excellent news09:19
shardyhewbrocca: there's going to be a ton of bugfixes for an RC2, but all the features with FFEs landed09:19
hewbroccafolks, ^^^ is amazine09:20
hewbroccaamazing09:20
hewbroccavery very cool09:20
jaosoriortobias-fiberdata: what happened?09:22
*** phpcodemonkey has joined #tripleo09:22
tobias-fiberdatajaosorior, it errors on something with ceph09:22
d0ugaltobias-fiberdata: file:// ?09:23
tobias-fiberdatad0ugal, we have edit the file, that's probably why. i need to check09:23
d0ugaltobias-fiberdata: ah, okay. I thought you might be seeing the "Could not fetch contents for file" error.09:24
d0ugallike https://bugs.launchpad.net/tripleo/+bug/162472709:24
openstackLaunchpad bug 1624727 in tripleo "Could not fetch contents for file:///home/stack/tripleo-heat-templates/puppet/post.yaml" [Critical,New]09:24
openstackgerritFrederic Lepied proposed openstack/tripleo-specs: Step by step validation Spec  https://review.openstack.org/37233609:25
shardyd0ugal, gfidente: FYI I'm about to brew myself an extra-strength coffee and try to fix that, unless you guys already have it in progress09:25
d0ugalshardy: not yet, but I am happy to try and pair on it if that would help09:25
d0ugalshardy: I have just been pondering how to tackle it too :)09:25
d0ugalshardy: I think there are a few bugs that are all related.09:26
gfidenteso we have two different LP bugs for the same issue?09:26
d0ugalshardy: but we have other mistral problems that are blocking general progress a bit :/09:26
shardyd0ugal: ack, thanks, let me dig into it and I'll get back to you09:27
d0ugalshardy: k, thanks.09:27
d0ugalshardy: did you get the breakdown on the CI mistral issue last week?09:27
*** phpcodemonkey has quit IRC09:27
shardyd0ugal: Ok, would be good to get a summary of that in the context of the release09:28
shardyd0ugal: Yeah I saw something but it'd be good to understand the latest status, e.g what exactly we need to land/fix for Newton09:28
*** akuznetsov has quit IRC09:28
d0ugalshardy: k, I'll try and explain now quickly...09:28
d0ugalshardy: so, long story short. Mistral changed their oslo-messaging executor to blocking.09:29
d0ugalWe are trying to get that changed back09:29
d0ugalbut for now, if you have a workflow running and you run an action, it will error.09:29
d0ugalhttps://review.openstack.org/#/c/371435/1/mistral/engine/rpc_backend/oslo/oslo_server.py09:29
shardyOk, that sounds very bad :(09:29
d0ugal^ That works around the issue at the moment09:29
shardywhy would they change something like that so close to the release?09:29
d0ugalshardy: Yeah, it's terrible. It is an rc2 blocker for mistral09:29
d0ugalshardy: it was changed for "performance"09:30
shardyalright, is there a mistral bug I can follow?09:30
d0ugalbut there is a good chance people will hit this - CI hit it because it started the workflow to create the default plan, but didn't wait for it to finish. It then tried to list the plans09:30
*** pkovar has joined #tripleo09:30
d0ugalshardy: https://bugs.launchpad.net/mistral/+bug/162428409:30
openstackLaunchpad bug 1624284 in Mistral "MessagingTimeout when executing mistral actions" [Critical,Confirmed] - Assigned to Dawid Deja (dawid-deja-0)09:30
d0ugalshardy: Now, get this - the only reason that CI is passing now is because the default plan creation is broken.09:31
d0ugalshardy: all the mistral stuff is in a rather fragile state, because I can't fix things without CI starting to hit the error again09:31
shardyd0ugal: Ok, thanks for the summary09:32
shardyit seems the bug has been ack'd as critical by the mistral folks, so we can work with them to test any fix09:32
d0ugalshardy: yup, it is on the agenda for todays meeting (5pm our time)09:32
shardyif that looks like it will take some time, we should probably look at ways to tmprevert of pin mistral in our CI09:32
shardys/of/or09:33
*** electrofelix has joined #tripleo09:33
d0ugalshardy: The revert is tricky, the change was large and doesn't revert well now.09:33
d0ugalshardy: but that one line change I made does work - so maybe we can patch CI?09:33
d0ugalpatch in CI*09:34
shardyYeah if we have to we can hack the undercloud in CI as a temporary workaround09:34
*** akrivoka has joined #tripleo09:34
*** tosky has quit IRC09:34
openstackgerritDougal Matthews proposed openstack/tripleo-common: Provide: return task result in case of failure  https://review.openstack.org/36755209:36
openstackgerritJulie Pichon proposed openstack/python-tripleoclient: Provide more information when 'node provide' fails  https://review.openstack.org/36755309:37
*** pkovar has quit IRC09:37
openstackgerritDougal Matthews proposed openstack/tripleo-common: Remove logic for Ironic nodes in ENROLL state  https://review.openstack.org/34736209:38
*** tosky has joined #tripleo09:41
Jokke_shardy: morning09:44
Jokke_shardy: so do we have a chance to get these https://review.openstack.org/#/c/358525/ in or not?09:44
*** anshul has quit IRC09:45
tobias-fiberdatashardy, does redhat consult with upstream-versions or is that only the "Redhat versions?" i think we need help solving all these things.09:46
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Use osd_pool_default_* puppet parameters when creating the pools  https://review.openstack.org/37027009:52
bandinid0ugal: is there a definite list of reviews that fixes https://bugs.launchpad.net/tripleo/+bug/1622683 or any workarounds at all ? (I tried these https://bugs.launchpad.net/tripleo/+bug/1624727/comments/3 but no luck so far)09:54
openstackLaunchpad bug 1622683 in tripleo "Updating plans breaks deployment" [Critical,In progress] - Assigned to Dougal Matthews (d0ugal)09:54
openstackLaunchpad bug 1624727 in tripleo "Could not fetch contents for file:///home/stack/tripleo-heat-templates/puppet/post.yaml" [Critical,New]09:54
Jokke_marios: you want to push that quick edit or want me to do it? We got comment from tbarron that it should be false09:55
*** radez has quit IRC09:56
d0ugalbandini: no, not yet. I need to try and come up with something today.09:56
*** radez has joined #tripleo09:56
mariosJokke_: incoming09:56
mariosJokke_: just pulling the branch09:56
*** leifmadsen has quit IRC09:56
Jokke_cool09:56
bandinid0ugal: ack, thanks09:56
*** cmyster has quit IRC09:57
*** leifmadsen has joined #tripleo09:57
*** cmyster has joined #tripleo09:58
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375009:58
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Add all the gating functionality to full-deploy.sh  https://review.openstack.org/37019409:58
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Add missing provison tag to quickstart.yml  https://review.openstack.org/37236109:58
openstackgerritMarios Andreou proposed openstack/puppet-tripleo: Fixup manila-cephfs native backend defaults  https://review.openstack.org/36676009:59
mariosgfidente: please re-add your +2 when you get a chance, we just changed the default for share_servers for cephfs ^^^09:59
gfidentemarios, ack thanks10:01
*** abehl has joined #tripleo10:01
shardyJokke_: https://blueprints.launchpad.net/tripleo/+spec/manila-cephfs-integration is marked as implemented, is that wrong?10:02
shardythe time for landing features is pretty much over now, and I did try to chase this last week10:02
Jokke_shardy: any idea who marked it as complete when the tht change has been sitting in the review?10:03
shardyJokke_: it looks like EmilienM did, but AFAICS it's a mistake10:04
shardyJokke_: I'll move it to RC2, but we do have to land it very soon, or it will be deferred to Ocata10:05
ccamachohey gfidente morning!10:05
Jokke_shardy: by the looks of it all the concerns are addressed and the last minute change needs are there10:05
ccamachogfidente re https://review.openstack.org/#/c/368953/  do you know if we are deploying liberty with ext4 on CI?10:05
Jokke_shardy: it should be ready to go in10:05
shardyJokke_: ack, Ok lets try to get it landed then :)10:05
Jokke_shardy: I was trying to get it tested over last week (locally) with very little success for non-related reasons10:06
ccamachobecause a few moths ago the VMs were not booting up if using xfs on LIberty10:06
Jokke_but it _should_ be fine10:06
gfidenteccamacho, overcloud guests or overcloud nodes ?10:07
gfidentewe should look at how that is though as changing the default fs will diverge from wanted default10:07
gfidenteso it was fine as quick fix to move CI but if it's still failing with xfs we need to figure why that is10:08
*** akuznetsov has joined #tripleo10:08
ccamachogfidente, overcloud nodes10:08
gfidenteccamacho, so from ci results looks like it managed to boot them10:09
gfidentebut failed during puppet apply10:09
gfidenteso the filesystem shouldn't be an issue anymore10:09
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Remove roles-deploy.sh ci-script  https://review.openstack.org/37236710:10
ccamachoack, I ll redeploy liberty with xfs to double check it.10:10
gfidenteccamacho, well ci is doing it10:11
*** limao has quit IRC10:11
gfidenteLABEL=img-rootfs /                       xfs     defaults        0 010:11
gfidentewe need to make it pass though, I am worried there could be something unrelated causing liberty to fail10:12
ccamachogfidente ack ill check then CI logs10:13
*** fultonj has joined #tripleo10:15
skramajashardy: i have been following your patches on custom roles, our end goal is to deploy custom roles like compute, compute-dpdk and compute-sriov in the same cluster. if i understand correctly, it is the role_data.yaml which has to be modified accordingly.10:23
skramajashardy: like http://paste.openstack.org/show/580958/10:23
skramajashardy: where i have added 2 new roles - is this in line with custom role definition or am i missing something?10:24
*** pkovar has joined #tripleo10:25
*** fzdarsky has quit IRC10:25
*** openstackstatus has quit IRC10:28
*** abehl has quit IRC10:31
shardyskramaja: Yes, that's correct, but right now there's a few additional steps required - I'm planning a blog post & docs patch this week10:33
skramajashardy:  thanks.. there are couple of things i want to confirm in relation with DPDK..10:34
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart: Fix devmode script  https://review.openstack.org/37238510:35
*** akuznetsov has quit IRC10:35
skramajashardy:  for dpdk i have done in a way that ComputeNeutronOvsAgent can be override to priovide dpdk support - https://review.openstack.org/#/c/344084/, which will not work for compute and compute-dpdk roles..10:36
skramajashardy: so i have to make another serv ComputeNeutronOvsDpdkAgent, which will be applicable only to compute-dpdk role..10:37
skramajai will do this change..10:37
skramajasecondly, there are some pending things like flavor mapping for custom roles, which can be done easily. but i am worried how do we configure the NetworkConfig for mutliple compute roles..10:38
skramajahas anything been decided on those lines?10:38
*** dprince has joined #tripleo10:39
shardyskramaja: yeah these are the details we need to work out & document10:39
skramajaok shardy i will wait for the docs then..10:39
shardyskramaja: for the flavors, you just copy e.g puppet/cinder-storage.yaml to mycustom-role.yaml10:39
shardythen you sed the file to match your desired naming10:39
shardyand you can then just pass MyCustomRoleFlavor: foo in the environment10:39
shardyin future I'd like to avoid copying files for custom roles, but for now I'm planning a small helper script that makes it easier10:40
shardyit's pretty much the same story for networking - you can definitely wire in special configuration for your CustomRole, but it'll need some manual additions to the resource_registry10:40
shardyI expect we can refine these interfaces for Ocata when folks have provided feedback re their requirements10:41
*** thrash|g0ne is now known as thrash10:42
skramajathanks shardy.. i will be working on to demonstrate dpdk and sriov in same cluster.. so i will be bugging you now and then :)10:43
*** fzdarsky has joined #tripleo10:43
*** abehl has joined #tripleo10:43
mariosshardy: fyi/wdyt? when you get a chance please https://review.openstack.org/#/c/357192/4 - there are 3 related changes in total is a clean revert... coments/question about whether we can now land this10:45
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart: Fix devmode-tempest script  https://review.openstack.org/37238510:46
thrashshardy: marios yes please. :)10:46
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Add a function to upgrade from full HA to NG HA  https://review.openstack.org/35862610:48
thrashmarios: shardy jistr there is this bug that we can use to attach the revert https://bugs.launchpad.net/tripleo/+bug/156580710:50
openstackLaunchpad bug 1565807 in tripleo "[python-tripleoclient] [tripleo-common] upgrade commands incomplete" [Critical,Invalid] - Assigned to Brad P. Crochet (brad-9)10:50
thrashwith a bit of modification, but...10:51
thrashor just additional info. We originally wanted to revert these in liberty.10:51
thrashI hate that we've even carried them this long.10:51
jistrthrash: hehe yea. +1 on attaching on this one10:52
gfidenteshardy, can you check my comment in https://review.openstack.org/#/c/370270/5/puppet/services/ceph-mon.yaml10:53
gfidenteI am assuming ExtraConfig remains10:53
gfidenteand will continue to be useful for overrides10:53
gfidenteis that okay?10:53
shardymarios, thrash: I'm fine with the revert, but we'll need to use the bug reference for the backport to stable/newton10:54
b00tcat`can someone confirm that the tripleo `overcloud-full` image can be built without any problems? and if so, how do you build it? I can't get this to work..10:54
shardygfidente: Yeah - I think I replied to your ML message?  We probably need to move away from the ExtraConfig resources, but the parameters can probably stay10:55
gfidenteah I might have missed the reply to list10:56
gfidentebut ack , thanks10:56
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Deployment Plan page wizard styling  https://review.openstack.org/36997410:56
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Add Role Detail  https://review.openstack.org/36799310:56
*** pkovar has quit IRC10:57
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Deployment Plan page wizard styling  https://review.openstack.org/36997410:59
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Add Role Detail  https://review.openstack.org/36799310:59
openstackgerritJulie Pichon proposed openstack/tripleo-ui: Remove obsolete project references from the README  https://review.openstack.org/37240211:00
*** dprince has quit IRC11:01
*** yolanda has quit IRC11:02
shardygfidente: actually, sorry, looks like it didn't send, will send my draft now11:03
*** lucas-dinner is now known as lucasagomes11:03
*** zoliXXL is now known as zoli|trng11:07
openstackgerritBrad P. Crochet proposed openstack/python-tripleoclient: Revert "Upgrades: Add initialization stage to 'overcloud upgrade'"  https://review.openstack.org/35719211:13
openstackgerritBrad P. Crochet proposed openstack/python-tripleoclient: Revert "Upgrades: Add finish stage to 'overcloud upgrade'"  https://review.openstack.org/35719411:13
openstackgerritBrad P. Crochet proposed openstack/python-tripleoclient: Revert "Upgrades: Add 'stack upgrade' command"  https://review.openstack.org/35719511:13
thrashmarios: jistr shardy rebased and added the bug reference.11:14
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Revert "Upgrades: Add initialization to StackUpgradeManager"  https://review.openstack.org/35721911:18
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Revert "Upgrades: Add post-upgrade stack update"  https://review.openstack.org/35722011:18
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Revert "Upgrades: Add StackUpgradeManager"  https://review.openstack.org/35722111:18
jistrthrash: thanks, +2'd and i also lowered the bug priority from critical to high, as this doesn't cause any blocking breakage11:19
*** milan has joined #tripleo11:20
thrashjistr: ack11:22
jistri'll +2 the removal in tripleo-common after we land the tripleoclient part (we might get import errors if we land out of order?)11:25
*** stendulker has quit IRC11:25
tobias-fiberdatahmm11:25
tobias-fiberdatahttp://paste.openstack.org/show/580966/11:25
tobias-fiberdatababysteps forward baby, babysteps. This is like a relationship in dating-phase. TripleO is very complex.11:27
jaosoriortobias-fiberdata: that is actually a zaqar bug: https://review.openstack.org/#/c/369285/11:29
tobias-fiberdatajaosorior, oh11:29
jaosoriortobias-fiberdata: yeah, that needs to merge in order to have it fixed. You can, however, use it manually11:30
tobias-fiberdatajaosorior, will get my collegue on it11:30
tobias-fiberdatathanks alot11:30
tobias-fiberdatahe knows what to do to use it11:30
*** ccamacho is now known as ccamacho|lunch11:31
openstackgerritBabu Shanmugam proposed openstack/puppet-tripleo: WIP - Depends on the OVS patches [1] to be merged.  https://review.openstack.org/37227411:35
*** jpena is now known as jpena|lunch11:38
EmilienMhello11:41
*** gchamoul is now known as gchamoul|afk11:42
EmilienMshardy, jaosorior: sorry, indeed I missed tht11:42
*** gchamoul|afk is now known as gchamoul11:43
shardyEmilienM: np, I pushed it to RC2 as it was a FFE and it's only one patch11:43
EmilienMshardy: what did you push?11:44
shardyEmilienM: https://blueprints.launchpad.net/tripleo/+spec/manila-cephfs-integration11:45
shardyI meant I moved the BP to RC2 so we can track landing the final patch11:45
EmilienMok great11:45
EmilienMshardy: great, CI is still all green today :)11:45
shardy\o/11:45
EmilienMjistr: big kidos on your mongo timeout patch, I had to rework it a little but the idea was excellent11:46
shardyhopefully we can tag RC1 today, and start iterating on landing all-the-bugfixes11:46
EmilienMshardy: we don't have to wait, tags will be on the commit ID11:46
EmilienMas a reminder, when RC1 is tagged, stable/newton will be created, so we'll have to backport bug fixes11:46
jistrEmilienM: hi, thanks for fixing it up. I'm still a bit worried that the default timeout might be too low and we'd hit it as "false alarm" at some point. That's why i wanted to pass 600 there. Unfortunately i looked at mongodb_conn_validator directly and not on the class we have in puppet-tripleo, so the param was undefined :) If we hit problems with too low timeout, i will fix it up.11:48
EmilienMshardy: I'll catch-up rc1 with doug when he arrives11:48
EmilienMjistr: cool11:48
EmilienMshardy: also releasing puppet modules https://review.openstack.org/#/c/371965/11:48
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci: Test do not merge  https://review.openstack.org/37243111:52
tobias-fiberdatajaosorior, we cant even find the files they are pointing on, in the undercloud11:52
jaosoriortobias-fiberdata: so there is nothing under /usr/lib/python2.7/site-packages/zaqar ?11:54
tobias-fiberdatajaosorior, yea there is. so that's the file?11:54
jaosoriortobias-fiberdata: well, that's where the zaqar package is11:55
jaosoriortobias-fiberdata: so that's where you need to patch. Or, you could rebuild the zaqar package to include that change.11:55
*** dprince has joined #tripleo11:57
openstackgerritBrad P. Crochet proposed openstack-infra/tripleo-ci: debug: Run 'openstack stack failures list' upon overcloud failures  https://review.openstack.org/36943411:57
EmilienMdciabrin, shardy: should we care about ovb ha job for liberty? It's still red and I thought https://review.openstack.org/#/c/371031/ would fix it but it's not. Should I continue to work on it?11:57
tobias-fiberdatajaosorior, i make myself a fool, asking so much questions11:57
jaosoriortobias-fiberdata: don't worry about it11:57
jaosoriortobias-fiberdata: it's normal11:58
jaosoriortobias-fiberdata: but yeah, just know that most openstack services installed as regular python packages, and will be found around /usr/lib/python2.7/site-packages/ ; the same is true for the client packages11:59
jistrwow so green http://tripleo.org/cistatus.html12:00
dciabrinEmilienM, checking the log on the failing job12:00
jaosoriortobias-fiberdata: tripleo-specific stuff is a bit different and has other paths though. But yeah, if it's a regular openstack service, should be there12:00
*** Guest64574 is now known as beagles12:00
*** shardy is now known as shardy_lunch12:01
tobias-fiberdatajaosorior, like i said earlier TripleO and openstack in general are very complex. not easy to know everything about it :p12:01
jaosoriortobias-fiberdata: that's why it's good to ask ;) so don't worry too much. Hopefully someone in the channel will be able to help12:01
openstackgerrityuyafei proposed openstack/tripleo-common: Replace raw_input with input to make PY3 compatible  https://review.openstack.org/33712912:01
*** yolanda has joined #tripleo12:02
*** trown|outtypewww is now known as trown12:03
jaosoriord0ugal: what happened to the commit that fixed the user-environment.yaml problem?12:04
d0ugaljaosorior: Which one? :)12:04
d0ugaljaosorior: There are various fixes floating around for favour different problems12:04
jaosorioraaah crap12:05
jaosoriorwell, d0ugal what are the current patches needed to be able to do overcloud updates again?12:05
*** liverpooler has quit IRC12:05
d0ugaljaosorior: Not all the problems are resolved yet - so I'm not totally sure12:05
d0ugaljaosorior: I am trying to figure that out now.12:06
d0ugaljaosorior: I've not had a deploy work yet12:06
*** rodrigods has quit IRC12:06
*** rodrigods has joined #tripleo12:06
d0ugaljaosorior: after all the CI problems I am feeling quite behind.12:06
jaosoriord0ugal: well, I've been able to do a deploy. But no update :/12:07
d0ugaljaosorior: I would delete the plan manually as a work around.12:07
d0ugaljaosorior: swift delete overcloud && mistral environment-delete overcloud12:07
jaosoriorI'll try that12:07
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Deployment Plan page wizard styling  https://review.openstack.org/36997412:09
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Add Role Detail  https://review.openstack.org/36799312:09
d0ugaljaosorior: Thanks, it would be good to know if that works. I'm thinking the cleanest solution for Newton might just be to delete the plan before deploying12:12
d0ugaljaosorior: We didn't think through plan updating etc. and I think it'll just cause problems now12:12
dciabrinEmilienM, strange, the galera bootstrap is failing earlier than what the patch aims at fixing. I'm trying to understand why it would fail on liberty and not on mitaka12:13
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Fixes for devmode convert_image  https://review.openstack.org/37244112:15
tobias-fiberdatajaosorior, that patch made our zaqar to fail to start.12:15
jaosoriorwha12:15
jaosoriortobias-fiberdata: what do the logs say?12:16
tobias-fiberdatajaosorior, ah well.. it doesnt matter, we will clean install our undercloud-node and run stable mitaka12:17
tobias-fiberdatajaosorior, and hope it works12:17
*** rhallisey has joined #tripleo12:18
jaosoriortobias-fiberdata: that's quite strange. I am actually using that patch myself. But alright. Hope that works12:18
skramajad0ugal: jaosorior fyi - i had also hit with problem of update not starting.. above the suggestion, it worked - http://paste.openstack.org/show/580973/12:19
d0ugalskramaja: cool, thanks12:19
jaosoriorskramaja: awesome12:20
jaosoriormy initial deployment is still on-going. So I'll try that when that's done12:20
*** karthiks has joined #tripleo12:22
*** rhallisey has quit IRC12:22
openstackgerritRyan Hallisey proposed openstack/tripleo-docs: Docs for containerized compute node  https://review.openstack.org/25474312:26
*** rhallisey has joined #tripleo12:26
EmilienMd0ugal: just a reminder about all tripleoclient work, don't forget to backport it to stable/newton if required (bugfix)12:27
*** ayoung has joined #tripleo12:28
*** jayg|g0n3 is now known as jayg12:28
*** chem` has joined #tripleo12:30
*** jcoufal has joined #tripleo12:32
*** radeks has quit IRC12:33
*** jprovazn has quit IRC12:33
*** rlandy has joined #tripleo12:35
dciabrinEmilien: so not sure why, but we miss two options in the galera config in liberty. i think this is because https://review.openstack.org/#/c/300706/ was never backported to liberty12:35
EmilienMdciabrin: woops12:35
EmilienMdciabrin: nice shot12:35
EmilienMdciabrin: I could try to squash the 2 commits and see how it works12:36
d0ugalEmilienM: k, thanks. I'm still trying to figure out how to make master work :)12:36
* d0ugal -> lunch12:36
*** ccamacho|lunch is now known as ccamacho12:38
*** david-lyle has joined #tripleo12:39
dciabrinEmilienM, yep definitely the simplest. without those two settings, mariadb10 won't activate galera, hence the failure12:39
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: MySQL Galera backports  https://review.openstack.org/37103112:39
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375012:40
*** rbrady_ has quit IRC12:40
florianfjtomasek: is that one ready for review? https://review.openstack.org/#/c/367993/12:41
*** weshay_pto is now known as weshay12:41
jtomasekflorianf: yes12:41
*** david-lyle has quit IRC12:43
*** jpena|lunch is now known as jpena12:45
florianfjtomasek: excellent, thanks.12:46
jaosoriorHey guys, could someone check out this documentation CR? https://review.openstack.org/#/c/368601/12:46
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart: Fix small typo in README.rst file.  https://review.openstack.org/37246112:49
*** shardy_lunch is now known as shardy12:49
openstackgerritMerged openstack-infra/tripleo-ci: Stop removing the puppet-ceph git repository  https://review.openstack.org/36750512:50
openstackgerritMerged openstack-infra/tripleo-ci: Add quotes around systemctl command in test  https://review.openstack.org/36351812:50
openstackgerritMerged openstack-infra/tripleo-ci: Also set Haproxy addresses in no_proxy  https://review.openstack.org/37062312:50
*** hjensas has quit IRC12:51
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable TLS for Zaqar's websocket endpoint  https://review.openstack.org/36035012:53
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Make HAProxy terminate OpenStack internal/admin endpoints  https://review.openstack.org/36036612:53
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Always deploy HAProxy  https://review.openstack.org/36036812:53
openstackgerritMerged openstack/tripleo-ui: Remove obsolete project references from the README  https://review.openstack.org/37240212:56
openstackgerritMerged openstack/tripleo-ui: Update tripleo-ui-deps RPM  https://review.openstack.org/37077512:56
*** tzumainn has joined #tripleo12:56
ccamachohey  d0ugal after executing swift delete overcloud && mistral environment-delete overcloud  are you hitting this? https://bugs.launchpad.net/tripleo/+bug/162472712:57
openstackLaunchpad bug 1624727 in tripleo "Could not fetch contents for file:///home/stack/tripleo-heat-templates/puppet/post.yaml" [Critical,New]12:57
ccamachod0ugal just curious about it.. as im hitting it.12:57
openstackgerritMerged openstack/tripleo-quickstart: Fix small typo in README.rst file.  https://review.openstack.org/37246112:58
*** radeks has joined #tripleo12:58
*** weshay has quit IRC13:03
*** abehl has quit IRC13:04
*** noslzzp has quit IRC13:07
*** noslzzp has joined #tripleo13:07
*** jaosorior has quit IRC13:09
openstackgerritJulie Pichon proposed openstack/python-tripleoclient: Remove excessive output when configuring nodes  https://review.openstack.org/37247713:10
*** jaosorior has joined #tripleo13:10
gfidentejistr, regarding https://review.openstack.org/#/c/368953/3//COMMIT_MSG13:10
gfidenteI saw that but CI seems to be passing now on that?13:11
jistri think CI always passed on that IIRC, for some reason it only broke dev envs13:11
jistrgfidente: though if we aren't able to find a solution that would fix both, then we probably should prefer whatever makes CI green. Does the Ceph/ext4 issue break CI?13:13
gfidentejistr, it doesn't break the hammer version no13:14
gfidentethough it's kinda not testing what we want to test13:14
*** limao has joined #tripleo13:14
gfidentebut you think the job failed on an actual issue with xfs there?13:14
* jistr looks13:15
*** oshvartz has quit IRC13:15
jistrgfidente: no i think it probably failed on something else (in puppet). If it failed on the issue with XFS, it would fail earlier. But we never hit that issue in CI anyway AFAIK.13:17
gfidenteyeah13:17
gfidenteso we should make it pass13:17
gfidenteit's not urgent, but I would try to do that rather than staying on ext413:17
openstackgerritMerged openstack-infra/tripleo-ci: Add Emilien Macchi ssh key to TripleO admins  https://review.openstack.org/37148413:17
*** limao_ has joined #tripleo13:18
EmilienMgfidente: is there any upgrade constraint wrt ext4 vs xfs?13:18
gfidenteEmilienM, no I don't think so13:19
*** bnemec has joined #tripleo13:19
gfidentethe filesystem will be what was used when building the image13:19
jistrgfidente: ok... we could work around for dev setups with exporting the variable manually then... but anyway i don't consider this a big deal if it's not a problem for Ceph Hammer, and Liberty goes EOL in less than 2 months so maybe we don't need newer Ceph versions with liberty?13:19
EmilienMgfidente: ok13:19
gfidentejistr, we don't want newer ceph with liberty no13:20
*** cdearborn has joined #tripleo13:20
*** limao has quit IRC13:21
*** abehl has joined #tripleo13:21
*** rain has joined #tripleo13:22
jistrhmm then i'd be tempted to file it under "not worth the time+disruption" (dev setups may go broken if we merge it)13:23
openstackgerritAna Krivokapic proposed openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/36756213:23
*** rain is now known as leanderthal13:23
openstackgerritRyan Hallisey proposed openstack/tripleo-docs: Docs for containerized compute node  https://review.openstack.org/25474313:23
ccamachogfidente I remember that the puppet manifest were running in the OC nodes, then they reboot and got stuck (and the network deployment hangs forever) and what I cant remember exactly was the reason why it was not hitting CI13:23
EmilienMbandini: do you have some wip around https://bugs.launchpad.net/tripleo/+bug/1615983 ? or an idea how to fix this?13:24
openstackLaunchpad bug 1615983 in tripleo "if $::hostname == boostrap_node checks are fragile" [High,Confirmed]13:24
ccamachogfidente I think it was because the quickstart OC images were using ext413:24
*** morazi has joined #tripleo13:24
jistrccamacho, gfidente: i hit that issue too though IIRC, and i wasn't using quickstart13:25
jtomasekjrist: should I move this to 'Fix Released'? I thought that gerrit/launchpad will take care of it once the patch which fixes it gets merged/released13:25
*** lblanchard has joined #tripleo13:25
*** morazi has quit IRC13:25
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Remove support for legacy rh1  https://review.openstack.org/34791813:25
jtomasekjrist: https://bugs.launchpad.net/tripleo/+bug/161640313:25
openstackLaunchpad bug 1616403 in tripleo "Mistral actions error handling" [Medium,In progress] - Assigned to Jiri Tomasek (jtomasek)13:25
ccamachojistr yeah, me too and it was becasue we were using xfs13:26
jristjtomasek: I'm not sure13:26
ccamachoin the oc images (by default)13:26
*** morazi has joined #tripleo13:26
jristjtomasek: did you say Fixes-bug: ?13:26
jristin the commit13:26
EmilienMccamacho: did you have a chance to reproduce https://bugs.launchpad.net/tripleo/+bug/1619205 ?13:26
openstackLaunchpad bug 1619205 in tripleo "Overcloud API services go down after some time due to keystonemiddleware failure" [High,Confirmed]13:26
bandiniEmilienM: nopers I did not have time to take a look at it, I made a note when I noticed it the first time13:27
*** akshai has joined #tripleo13:27
EmilienMbandini: how would you make it more robust?13:27
ccamachowhat is weird is that why its hitting mitaka as it wasn't before.13:27
jtomasekjrist: https://review.openstack.org/#/c/359764/13:27
EmilienMpanda|Zz: I'm looking at ipv6 now. any progress so far?13:28
ccamachohey EmilienM yeahp, reproduced https://bugs.launchpad.net/tripleo/+bug/1619205 and I got the same error re the number of httpd procesess goes up to 263 and apache hangs13:28
openstackLaunchpad bug 1619205 in tripleo "Overcloud API services go down after some time due to keystonemiddleware failure" [High,Confirmed]13:28
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Add IPv6 network configuration for ipv6 job types  https://review.openstack.org/36367413:29
d0ugalshardy: Hey - did you have any luck looking into the issue?13:30
EmilienMccamacho: please update the bug with your results, it would help us to fix the bug13:30
jpichjtomasek, jrist: This merged before we connected tripleo-ui's Gerrit to the tripleo Launchpad, should be ok in the future but we might have to update Launchpad manually for the patches that merged/were submitted before that13:30
ccamachoEmilienM In in my case the RAM went off (because all those http processes)... Im just waiting for a mem upgrade to see again the behavior13:30
jtomasekjpich: ack13:30
ccamachoIt can be probably a memory leak issue13:31
ccamachoEmilienM updating it13:31
EmilienMccamacho: thanks13:31
ccamachonp man13:31
jtomasekjpich, jrist: so since that patch is already part of rc1, It should be ok to mark it  as 'fix released', right?13:31
jristjpich, jtomasek thanks13:31
jristI'll mark it - just didn't know :)13:32
jtomasekok13:32
jaosoriorbnemec: hey dude, could you check this out? https://review.openstack.org/#/c/368601/13:32
jpichjtomasek: Yep, and moved back to the right milestone probably13:32
jpichjrist: Thanks!13:32
jristcan you tell I'm triaging?13:32
jristhaha13:32
openstackgerritMerged openstack-infra/tripleo-ci: Enable shared networks for te-broker  https://review.openstack.org/35730813:33
openstackgerritMerged openstack/python-tripleoclient: Print the execution ID when starting Mistral workflows  https://review.openstack.org/34071113:33
*** limao_ has quit IRC13:33
*** limao has joined #tripleo13:33
skramajaEmilienM: https://review.openstack.org/#/c/364354/ is the parent of https://review.openstack.org/#/c/370012/ (which is +2ed and workflowed), i know it is a busy day.. when you have time take a look.13:34
EmilienMpanda|Zz: I'm rebasing the ipv6 patch, we fixed some mongo errors during the week-end, let's see how it goes now13:34
jpichjrist: Yeah, thanks for that too :)13:34
EmilienMskramaja: I'll look asap13:34
skramajaEmilienM: thanks13:35
EmilienMskramaja: doing recheck on https://review.openstack.org/#/c/364354/ is the best I can do now13:35
EmilienMwe don't merge patches that don't pass all ovb jobs13:35
skramajaEmilienM: ok. i will ping when it pass. just for the argument - ovb on https://review.openstack.org/#/c/370012/ is passed, which should take both. if this is not good enough, i will wait for it to pass individually..13:37
*** jprovazn has joined #tripleo13:38
shardyError: /Stage[main]/Main/Augeas[lvm.conf]: Could not evaluate: Saving failed, see debug13:39
shardyAnyone seen that from the undercloud install?13:39
*** limao has quit IRC13:39
jaosoriorshardy: yeah13:39
openstackgerritHonza Pokorny proposed openstack/tripleo-ui: Retrieve zaqar websocket url from keystone  https://review.openstack.org/37249913:39
shardyd0ugal: Not yet, sorry, been $interruptions all day, getting into it now13:39
jaosoriorshardy: as a workaround you gotta delete global_filter from lvm.conf13:39
jaosoriorand re-run undercloud install13:40
*** limao has joined #tripleo13:40
d0ugalshardy: No worries, I'm much the same and also trying to focus on that stuff now.13:40
shardyjaosorior: ack, thanks will give that a go13:40
jaosoriorshardy: I tried bringing it up several times, but since nobody had seen it/or answered when I asked, I guessed there was something strange in my deployment13:40
openstackgerritMerged openstack-infra/tripleo-ci: Remove galera hack  https://review.openstack.org/37218213:41
openstackgerritMerged openstack/python-tripleoclient: Remove the environments from Mistral when removing from Swift  https://review.openstack.org/36948613:41
openstackgerritMerged openstack/python-tripleoclient: Remove openstackclient imports in the new parameters command  https://review.openstack.org/37159413:41
shardyjaosorior: Yeah, well it seems like a real bug to me - did you raise one already?13:41
jaosoriorshardy: I didn't13:42
EmilienMshardy: I think I know what introduced it, give me a second13:42
*** rbrady has joined #tripleo13:42
EmilienMshardy, jaosorior: https://review.openstack.org/#/c/360790/13:43
bnemecjaosorior: Looks fine, but please see the comment I left on it.13:44
jaosoriorbnemec: ok, follow-up patch coming then. Thanks for checking it out dude13:45
bnemecjaosorior: Thanks13:45
*** osp has quit IRC13:45
shardyEmilienM: ack, thanks - I'll raise a bug and we can work out how to fix it13:45
bnemecIt would be so nice if the lvm people just fixed that so it behaves sanely by default.13:46
jaosoriorHey guys, could someone check out this SSL-related documentation patch? https://review.openstack.org/#/c/348038/13:46
shardyhttps://bugs.launchpad.net/tripleo/+bug/162519613:48
openstackLaunchpad bug 1625196 in tripleo "Undercloud install fails with Error: /Stage[main]/Main/Augeas[lvm.conf]: Could not evaluate" [High,Triaged]13:48
jaosoriorshardy: did removing that field work for you?13:48
openstackgerritMerged openstack/tripleo-docs: Document generate_service_certificate option for undercloud  https://review.openstack.org/36860113:49
*** cdearborn has quit IRC13:49
jaosoriorEmilienM, shardy: seems to me that the issue is that, the syntax didn't consider the list before. So when we do an update, augeas fails to parse the value as it was written before.13:50
openstackgerritMerged openstack/tripleo-quickstart: Update README about running a full end to end deployment  https://review.openstack.org/36965913:50
ccamachojaosorior, man quick question when having some time mind to check https://review.openstack.org/#/q/status:open++topic:puppet-cleanup to fix some puppet-lint warnings13:52
*** cdearborn has joined #tripleo13:52
jaosoriorccamacho: sure, will do13:52
shardyjaosorior: it did, thanks! :)13:52
jistrmarios: hi, have you seen bandini's latest https://review.openstack.org/#/c/358626/ yet?13:52
jistrcomments addressed, ci green13:52
jistrjust sayin13:52
*** Goneri has joined #tripleo13:53
*** myoung|gone is now known as myoung13:54
openstackgerritRyan Brady proposed openstack/tripleo-common: Port password generation from tripleoclient to tripleo-common  https://review.openstack.org/36815013:55
*** osp has joined #tripleo13:57
EmilienMtrozet: are you working on https://bugs.launchpad.net/tripleo/+bug/1585275 currently?13:58
openstackLaunchpad bug 1585275 in tripleo "SQL errors on undercloud/overcloud cause HA deployments to fail" [High,Triaged]13:58
openstackgerritRyan Brady proposed openstack/python-tripleoclient: Remove identifier and stack action params  https://review.openstack.org/37038313:58
*** skramaja has quit IRC13:59
*** weshay has joined #tripleo13:59
*** limao_ has joined #tripleo14:01
openstackgerritMerged openstack/tripleo-quickstart: Fix devmode-tempest script  https://review.openstack.org/37238514:01
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-docs: Move undercloud SSL admonition to SSL file  https://review.openstack.org/37250814:01
jaosoriorbnemec: ^^14:01
ospHi, in my overcloud deployment I am using OS::TripleO::ControllerExtraConfigPre to apply puppet config before the overcloud deployment. However, this will only run the .pp file is there a way to be able to include templates for puppet to use?14:03
*** radeks has quit IRC14:03
*** limao has quit IRC14:04
EmilienMderekh: hey, are you still working on https://bugs.launchpad.net/tripleo/+bug/1587656 ?14:08
openstackLaunchpad bug 1587656 in tripleo "undercloud install fails if devices exist with no dhcp" [High,Triaged]14:08
*** chem` is now known as chem14:09
derekhEmilienM: no, I'll close it, I wont be able to reproduce it at this stage, I'll reopen if it comes up again14:09
EmilienMderekh: ok thanks14:09
trozetEmilienM: hi.  No, since the entire code was refactored from mitaka->newton, we need to go test on newton and see if we still hit these issues, then I will update the bug.  A couple weeks out14:09
EmilienMtrozet: I moved the bug to ocata.14:10
EmilienMtrozet: why are you so late in your testing and still working on mitaka?14:11
trozetEmilienM: OPNFV is always one release behind in OpenStack :/14:11
EmilienMbnemec: do you know if any work is ongoing for https://bugs.launchpad.net/tripleo/+bug/1596373 ?14:11
openstackLaunchpad bug 1596373 in tripleo "40-hiera-datafiles takes over 20 seconds to run" [High,Triaged]14:11
EmilienMtrozet: that's a problem for you, as you need to fork tripleo ,that's a terrible model, you aware?14:11
bnemecEmilienM: I saw dprince pushed some patches that sounded related.14:12
EmilienMtrozet: i'm not sure how tripleo team can really help you, as you're not following any of our rules14:12
bnemecEmilienM: I haven't personally looked into it recently.14:12
EmilienMbnemec: can you maybe elaborate in the launchpad? /me trying to make rc2 triage today14:12
dprinceEmilienM: I'm working on a hiera hook FWIW14:12
EmilienMdprince: ok, please put it in the launchpad and retarget it for ocata if needed14:12
dprinceEmilienM: which will fix this bug, I will take it14:13
dprinceEmilienM: yes, ocata this is14:13
trozetEmilienM: well yeah.  If we could just work directly upstream we would, but given that we can't - this is what we have to do14:13
EmilienMdprince: awesome14:13
EmilienMdprince: triaged14:13
EmilienMtrozet: the reason of "we can't" is still very unclear to me, maybe can you ellaborate here?14:14
EmilienMshardy: is there any work ongoing for https://bugs.launchpad.net/tripleo/+bug/1611704 ?14:14
openstackLaunchpad bug 1611704 in tripleo "User defined passwords should be easier to define" [High,Triaged]14:14
trozetEmilienM: so OPNFV is a community project, and there are release requirements for installers, such as mitaka, and opendaylight beryllium.  We have to conform to those requirements per release14:15
trozetEmilienM: the next release is this week, which will use stable/mitaka.  Since we coudln't get 100% pass rate in deployments with TripleO in our CI, we had to fork to fix it and also add new features that we cant get into a stable branch upstream14:15
EmilienMtrozet: so OPNFV is not following upstream releases?14:16
trozetEmilienM: right14:16
EmilienMthat's a terrible development model14:16
trozetEmilienM: the reason is, because OPNFV pulls so many upstream projects into its CI, there is too much breakage pulling everything on master14:16
EmilienMtrozet: well, tripleo pulls a lot of upstream projects14:16
mariosjistr: hey i saw this morning but not looked since14:17
EmilienMtrozet: I don't think this is an excuse14:17
*** liverpooler has joined #tripleo14:17
EmilienMtrozet: but again, this is you being affected by this thing. not us14:17
EmilienMjust saying that it would be great to start collaboration14:17
EmilienMwe haven't seen much contributions from your group14:17
shardyEmilienM: No, I'd like to see it fixed but it'll have to wait until Ocata14:18
EmilienMand your bug reports are not really valid since you use old versions14:18
trozetEmilienM: my group is mostly focused on opendaylight14:18
* shardy defers to ocata-114:18
EmilienMshardy: thx14:18
trozetEmilienM: I'm the only one that really works on OpenStack in my group14:18
EmilienMtrozet: and michael?14:19
mariosjistr: bandini thanks voted14:19
trozetEmilienM: he is on OPNFV as well, but he's not part of my group at RH.  I'm not sure what his other assignments are14:19
marioshttps://review.openstack.org/#/c/358626/16 jist bandini here i mean14:19
mariosjistr:  even ^^^ sorry14:19
bandinijistr, marios: thanks guys! hopefully we can soon unblock the bugs that stop us from testing atm14:20
EmilienMtrozet: so iiuc, there is nothing you can do, and there is nothing we can do, right?14:20
trozetEmilienM: here is a list of upstream contributions for our installer project https://wiki.opnfv.org/display/apex/Upstream+Tracking14:21
*** pkovar has joined #tripleo14:21
socialmarios: why reinstalling OPM?14:22
EmilienMtrozet: openstack/puppet-congress is used in your installer now? or do you still your the initial module not upstream?14:22
trozetEmilienM: i think it is radez initial one still, the one upstream I checked has like nothing in it14:22
trozetEmilienM: we need to update those14:22
trozetEmilienM: (those meaning, tacker one as well)14:22
EmilienMtrozet: and puppet-tacker14:22
EmilienMyep14:22
trozetEmilienM: and get tacker into TripleO14:22
mariossocial: scrum in a minute? not clear which one you mean?14:23
EmilienMtrozet: using openstack/puppet-14:23
EmilienMtrozet: not the personal module on github, right?14:23
trozetEmilienM: right14:23
EmilienMtrozet: good.14:23
trozetEmilienM: you will probably see some new blueprints coming from us to add these things upstream, now that we are about to release and the pressure will be off - we can get it upstream14:25
jistrEmilienM: we can keep landing patches normally, it won't affect RC1 tagging, is that right?14:25
trozetEmilienM: also getting VPP integrated into TripleO and replacing OVS14:25
jistrEmilienM: i mean patches target for RC214:26
socialmarios: nvm it's review from augus14:26
jistr*targeted14:26
EmilienMtrozet: good, please try to get the blueprints before Barcelona14:27
trozetEmilienM: kk14:27
EmilienMjistr: from now, we only merge rc2 bugfixes14:27
EmilienMtracked in https://launchpad.net/tripleo/+milestone/newton-rc214:27
EmilienMjistr: exactly14:27
EmilienMjistr: the only feature we can merge is the last manila/ceph work14:27
jistrEmilienM: ok thanks. Yea i was asking because of https://review.openstack.org/#/c/358626/16 which is essentially a release blocker (upgrades prerequisite).14:28
EmilienMchem: are you working on https://bugs.launchpad.net/tripleo/+bug/1617413 ? Is there a patch to link?14:28
openstackLaunchpad bug 1617413 in tripleo "M/N upgrade upgrade-non-controller.sh don't detect ACTIVE node." [High,Triaged] - Assigned to Sofer Athlan-Guyot (sofer-athlan-guyot)14:29
*** pgadiya has quit IRC14:29
EmilienMjistr: everything related to upgrades NEEDS to be merged asap :)14:29
jistryea :)14:29
jaosoriorHas anybody seen something like this?14:29
jaosoriorep 19 14:25:13 overcloud-controller-0 os-collect-config[2119]: No local metadata found (['/var/lib/os-collect-config/local-data'])14:29
jaosoriorSep 19 14:25:53 overcloud-controller-0 os-collect-config[2119]: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/meta-data/ (Caused by ConnectTimeoutError(<requests.pac14:29
jaosoriorSep 19 14:25:53 overcloud-controller-0 os-collect-config[2119]: Source [ec2] Unavailable.14:29
jaosoriorSep 19 14:25:54 overcloud-controller-0 os-collect-config[2119]: /var/lib/os-collect-config/local-data not found. Skipping14:29
jaosoriorTrying to do a deployment, and the Controller/ComputeDeployment resources just stall14:29
chemEmilienM: yep, it was fixed and merged, but somehow launchpad didn't get the review linked back.14:30
EmilienMjaosorior: what is your version of os-cloud-config?14:30
chemEmilienM: I'm retrieving the review ...14:30
EmilienMchem: please set it to Fix Released14:30
EmilienMand update the patch with the link14:30
socialwhat all I have to clean up if I want to rerun overcloud deployment?14:30
chemEmilienM:ack14:30
jaosoriorEmilienM: in the nodes?14:30
EmilienMjaosorior: not os-cloud-config, a sec please14:30
jaosoriorEmilienM: os-cloud-config-5.0.0-0.20160914152513.e1dca00.el7.centos.noarch14:31
EmilienMjaosorior: openstack/os-collect-config14:31
jaosoriorEmilienM: os-collect-config-5.0.0-0.20160916004922.9ee07c9.el7.centos.noarch14:32
EmilienMjaosorior: it should be good, nevermind my comment then14:32
jistrEmilienM, jaosorior: hmm seems like it can't fetch the nova metadata? Perhaps checking of the network settings would be good, or i'd try restarting nova-api on undercloud...14:32
EmilienMjistr: i'll let you choose to approve or not https://review.openstack.org/#/c/358626/14:32
*** abehl has quit IRC14:32
jistrmarios: ^ wdyt? personally i'd lean towards +A14:33
*** abehl has joined #tripleo14:34
EmilienMshardy: same question for https://bugs.launchpad.net/tripleo/+bug/1620384 - is there any WIP you're aware?14:34
openstackLaunchpad bug 1620384 in tripleo "overcloud plan container collides with current upload artifacts location" [High,Triaged] - Assigned to Steven Hardy (shardy)14:34
EmilienMdprince: I saw some work from you around composable services bugs lately, have you seen https://bugs.launchpad.net/tripleo/+bug/1620595 ? do you know about some WIP?14:36
openstackLaunchpad bug 1620595 in tripleo "When Heat API services run on a different node than controller the heat-dbsync fails due to incorrect mysql permissions" [High,Triaged]14:36
dprincehttps://review.openstack.org/#/c/370573/14:37
*** openstackgerrit has quit IRC14:37
dprinceEmilienM: ^^^14:37
*** openstackgerrit has joined #tripleo14:37
EmilienMdprince: thanks14:37
shardyEmilienM: That one is fixed, I'll update it14:37
EmilienMshardy: thanks14:38
dprinceshardy: did you post a patch for this on Friday?14:38
shardyhttps://review.openstack.org/#/c/365783/14:38
shardydprince: No, it was a couple of weeks ago ^^14:38
shardyNot sure why the Closes-Bug didn't update the bug14:39
dprinceoh, I see. I misread the bug.14:39
dprinceEmilienM: this is the one I thought you linked https://bugs.launchpad.net/tripleo/+bug/162082914:39
openstackLaunchpad bug 1620829 in tripleo "Deployment fails when Keystone and AodhApi services are deployed on different nodes" [High,In progress] - Assigned to Dan Prince (dan-prince)14:39
EmilienMdprince: ok good14:39
shardydprince: Oh, no - I've not yet had time to hack on that one, sorry14:39
EmilienMjaosorior: are you working on https://bugs.launchpad.net/tripleo/+bug/1623805 ?14:40
openstackLaunchpad bug 1623805 in tripleo "SSL enabled undercloud installation fails due to haproxy user and group not existing" [High,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)14:40
dprinceshardy: its cool. I will try to get it today if I can14:40
jaosoriorEmilienM: that has merged already14:40
EmilienMjrist: could you please assign someone to https://bugs.launchpad.net/tripleo/+bug/1624390 ?14:40
openstackLaunchpad bug 1624390 in tripleo "[ui] Assigned Node Misrepresented in Nodes Table" [High,Triaged]14:40
*** david-lyle has joined #tripleo14:40
EmilienMjaosorior: please update with the patch and set it to "Fix Released"14:40
jaosoriorEmilienM: this is the fix https://review.openstack.org/#/c/370577/14:40
EmilienMjaosorior: please update launchpad ;-)14:40
EmilienMand thanks for the link, glad we fixed it :)14:40
jristEmilienM: looking14:41
jaosoriorEmilienM: I'm not sure if that fix is included in the hash you did for the release though14:41
*** mgarciam has joined #tripleo14:41
EmilienMjaosorior: it will14:41
EmilienMjaosorior: but nevermind14:41
jaosoriorEmilienM: ok, updated the status14:42
EmilienMshardy: https://bugs.launchpad.net/tripleo/+bug/1619074 - should we move it to o1?14:43
openstackLaunchpad bug 1619074 in tripleo "os-collect-config ec2 collector should attempt config-drive before nova metadata server" [Medium,Triaged] - Assigned to Steve Baker (steve-stevebaker)14:43
EmilienMI haven't seen progress on last 3 weeks14:44
shardyEmilienM: Yeah, +1 on moving it14:44
jaosoriorbnemec: hey man, so what do we do with the decoupled SSL documentation? Should it be left as it is? Or do you have time to update it?14:44
shardyalthough let me check, stevebaker did have some patches, sec14:44
EmilienMshardy: ack, i'll let you move it or not.14:44
jaosoriorjistr: restarting nova helped. Thanks man14:44
EmilienMdprince: anything for https://bugs.launchpad.net/tripleo/+bug/1618488 ?14:44
openstackLaunchpad bug 1618488 in tripleo "upload-swift-artifacts fails with: More than one endpoint exists with the name 'swift'." [Medium,Triaged] - Assigned to Dan Prince (dan-prince)14:44
shardyhttps://review.openstack.org/#/c/369731/ is WIP, so I'll move it14:45
*** Goneri has quit IRC14:45
*** david-lyle has quit IRC14:45
jistrjaosorior: hmm... there must be something broken... this is the 3rd time i hear about this issue, yet it seems to happen extremely infrequently14:45
EmilienMshardy: thx14:45
jaosoriorjistr: to be fair, I had messed around my deployment A LOT14:46
jaosoriorso I dunno really what happened14:46
*** TSCHAK has joined #tripleo14:48
shardyrbrady: Hey, shall we land https://review.openstack.org/#/c/371027 as a first step to resolving the update plan related issues, or did you want to break out the j2 stuff from the process_templates first?14:49
shardyI'm thinking we could land this then split up process_templates after?14:50
shardyd0ugal: ^^14:50
rbradyshardy: I agree.14:50
shardyrbrady: ack, thanks!14:50
rbradyshardy: splitting up is a lower priority than successful deployment :)14:50
shardyhehe, +3 :)14:50
d0ugalshardy: Sure, that's fine with me.14:51
tobias-fiberdatashardy, do you recommend that we use tripleo.sh for installing our undercloudnode for production?14:51
d0ugalshardy: just as an FYI, the update path isn't tested in CI at all14:51
tobias-fiberdatashardy, or is it exclusive for devel14:51
shardyd0ugal: Yeah, I'm trying to reproduce the file:/// thing locally, and it seems like at least part of those issues are update related14:51
shardybecause everything works OK for me with a clean initial plan create14:52
socialshardy: how doeas one now use swap.j2.yaml? how do I add it to extraconfig? I guess not by OS::TripleO::AllNodesExtraConfig: /usr/share/openstack-tripleo-heat-templates/extraconfig/all_nodes/swap.j2.yaml14:52
shardytobias-fiberdata: it just automates some steps from the docs, so yes I'd say using it is fine14:52
shardyhowever it does use bleeding edge master stuff by default14:53
tobias-fiberdatathanks shardy14:53
shardysocial: No, you'd reference swap.yaml in the same directory14:53
d0ugalshardy: I think I can tell you how to reproduce it... give me a few mins to find it again.14:53
shardyas when we create the plan, we j2 render swap.j2.yaml to create swap.yaml14:54
socialshardy: so the template used by user should not change? :/14:55
*** Goneri has joined #tripleo14:55
shardysocial: I'm not sure what you mean - any existing user environments referrencing swap.yaml should still work14:55
openstackgerritMerged openstack/tripleo-heat-templates: Use osd_pool_default_* puppet parameters when creating the pools  https://review.openstack.org/37027014:56
bnemecjaosorior: I dunno.  I'll try to take a look today, but if I don't get to it I'd be fine with it merging as-is.14:56
*** jbadiapa has quit IRC14:57
EmilienMd0ugal, sshnaidm: can you look again https://bugs.launchpad.net/tripleo/+bug/1611101 and say if the bug has been fixed or is WIP?14:57
openstackLaunchpad bug 1611101 in tripleo "Improve mistral errors logging in tripleo" [Low,Triaged]14:57
*** pkovar has quit IRC14:58
jaosoriorbnemec: alright, sounds like a plan14:59
d0ugalEmilienM: I don't understand this one, shouldn't it be a bug in Mistral?14:59
EmilienMd0ugal: most probably14:59
sshnaidmEmilienM, I think mistral logs are awful, this complicated json is not for people eyes at all. However I didn't see too much of them in last time..15:00
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Add devmode.sh and document Third Party CI  https://review.openstack.org/36000715:00
*** jbadiapa has joined #tripleo15:04
*** bnemec has quit IRC15:04
*** rajinir has joined #tripleo15:06
*** bnemec has joined #tripleo15:07
EmilienMsshnaidm: I would suggest to either defer https://bugs.launchpad.net/tripleo/+bug/1611101 to ocata-1 or even move it to mistral somewhere15:07
openstackLaunchpad bug 1611101 in tripleo "Improve mistral errors logging in tripleo" [Low,Triaged]15:07
*** pcaruana has quit IRC15:07
d0ugalEmilienM: +115:08
ccamachoMonday's are hard.. http://cdn77.sadanduseless.com/wp-content/uploads/2014/05/bird-with-arms16.jpg15:10
sshnaidmEmilienM, ok15:10
mariosjistr: sorry, was on scrum, will look in sec15:11
EmilienMshardy: wdyt of having a launchpad tag for backport candidates?15:12
shardyEmilienM: yup, we can use the normal newton-backport-potential tag?15:13
shardye.g like normal stable branch process?15:13
tobias-fiberdatashardy, think i've asked you earlier about this, but how do i specify a stable branch when using your script? :)15:14
EmilienMshardy: yes15:14
*** kjw3 has joined #tripleo15:15
EmilienMjpich, d0ugal: I created newton-backport-potential tag, please use it in your bugs for tripleoclient, it will help to make sure we backported everything into newton at the endof the cycle15:16
jpichEmilienM: Roger!15:16
shardyhttps://github.com/openstack-infra/tripleo-ci/blob/master/scripts/tripleo.sh#L9815:17
d0ugalEmilienM: good idea, thanks.15:17
shardytobias-fiberdata: export STABLE_RELEASE=mitaka15:17
* shardy disclaims sole responsibility for said script tho ;)15:17
tobias-fiberdatathanks alot shardy15:17
tobias-fiberdatai guess i can just remove the delorean files to redo the procedure15:18
*** mcornea has quit IRC15:20
tobias-fiberdatashardy, haha, i guess i should give some other guys credit for the script :p15:21
EmilienMgfidente: are you still working on https://review.openstack.org/#/c/338213/ ?15:22
*** saneax is now known as saneax-_-|AFK15:22
gfidenteEmilienM, it needs an update/rebase15:23
gfidenteit'd be a nice to have but it's not on my priorities right now15:23
EmilienMgfidente: I gave up the rebase, all files are modified lol15:23
EmilienMgfidente: ok should we defer it to ocata?15:23
gfidenteEmilienM, yeah15:23
gfidentesounds like we need to15:23
EmilienMok let me see if I can rebase it15:24
EmilienMotherwise i'll defer15:24
jpichd0ugal: About https://review.openstack.org/#/c/364425/ , I'm surprised the gate is passing tbh (maybe that means it's somehow all fine?). What would be the steps to do it the safe way?15:25
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Add devmode.sh and document Third Party CI  https://review.openstack.org/36000715:27
d0ugaljpich: CI tests against the latest packages that track master15:27
d0ugaljpich: only the unit tests will use the pip requirements.15:27
jpichd0ugal: Oh, ok15:27
*** spredzy has quit IRC15:28
d0ugaljpich: so I think it should be safe?15:28
*** leanderthal is now known as leanderthal|afk15:29
*** spredzy has joined #tripleo15:29
*** rcernin has quit IRC15:30
jpichd0ugal: I don't sense an overflowing sense of confidence :-)15:30
jpichd0ugal: Could we create a new tag just-in-case? Will the upper-constraint be ignored? (aka we can have a newer version than it?)15:31
d0ugaljpich: a new tag? do you mean a new release?15:31
*** ebarrera has quit IRC15:31
*** ramishra has quit IRC15:31
*** david-lyle has joined #tripleo15:32
*** yolanda has quit IRC15:32
jpichd0ugal: Yes, 5.0.1 or something?15:33
*** ramishra has joined #tripleo15:33
jpichd0ugal: Or if you're confident just remove the Workflow flag and go for it :) I don't understand how everything links together in terms of release / CI / tests / requirements + upper-constraints yet15:33
*** zoli|trng is now known as zoli|gone15:33
jaosoriorrcrit: hey dude, did deleting the swift container and the mistral environment work for you?15:34
*** zoli|gone is now known as zoli_gone-proxy15:34
jaosoriord0ugal: ping15:34
rcritjaosorior, it got past the error, still waiting for deploy to finish15:34
rcritlooking good so far15:34
rcritnovajoin worked anyway15:35
jaosoriornice!15:35
d0ugaljaosorior: pong15:35
d0ugaljpich: I think it will be fine!15:35
jpichd0ugal: Ok, cool! Thanks15:35
EmilienMshardy: something needs to be done in tripleoclient for https://bugs.launchpad.net/tripleo/+bug/1619415 ?15:36
openstackLaunchpad bug 1619415 in tripleo "tripleoclient events aren't always informative" [High,In progress] - Assigned to Steve Baker (steve-stevebaker)15:36
jaosoriord0ugal: hey dude, I deleted the container and the environment and keep getting this error  u'message': u"Failed to run action [action_ex_id=13a0c3a6-d285-4b31-9eb6-5dc037d10839, action_cls='<class 'mistral.actions.action_factory.DeployStackAction'>', attributes='{}', params='{u'container': u'overcloud', u'timeout': 240}']\n 'Result' object has no attribute 'copy'",15:37
d0ugaljaosorior: That's the obscured error one.15:38
jaosoriord0ugal: mistral execution-list shows both create_deployment_plan and deploy_plan as success15:38
d0ugaljaosorior: https://review.openstack.org/#/c/368081/15:38
EmilienMbandini: what is the patch related to https://bugs.launchpad.net/tripleo/+bug/1617520 ? can you update launchpad maybe?15:38
openstackLaunchpad bug 1617520 in tripleo "M/N upgrades - Need to migrate existing full HA to the new HA as deployed in Newton" [High,In progress] - Assigned to Michele Baldessari (michele)15:38
jaosoriorah, right, the logs show this InvalidSchema: No connection adapters were found for 'file:///home/stack/tripleo-heat-templates/puppet/manifests/overcloud_controller_pacemaker.pp'15:38
d0ugaljaosorior: try mistral action-execution-list?15:38
d0ugaljaosorior: aha, "file://..." it seems you are hitting the error me and shardy have been looking at15:39
EmilienMbnemec: can you see my comment on https://review.openstack.org/#/c/370438/ please? is it true?15:40
mariosjistr: bandini wfm i think we can land no objection here... https://review.openstack.org/#/c/358626/1615:41
*** myoung is now known as myoung|bbk15:41
*** myoung|bbk is now known as myoung|bbl15:41
bnemecEmilienM: Yeah, I think so.  I'm setting up a test env to verify.15:41
bnemecWell, I thought I was, but I forgot about the stupid password prompt. >:-(15:42
EmilienMbnemec: can you see if https://review.openstack.org/#/c/363674/ is the correct way of testing?15:42
EmilienMbnemec: I would rather try to make work in CI first15:42
*** b00tcat` has quit IRC15:42
bandinimarios: ack. shall we +A then?15:42
* hewbrocca +A15:43
bnemecEmilienM: I would echo sshnaidm's comment on that review.15:43
hewbroccaoh wait...15:43
bandinihewbrocca: eheheh15:43
*** dsariel has quit IRC15:44
EmilienMbnemec: right, i'll push over it15:44
bandiniEmilienM: done (not sure why launchpad did not pick it up right away)15:45
EmilienMcool thx15:45
jistr+A'd15:45
jistrthe NG HA migration15:45
openstackgerritBrent Eagles proposed openstack/tripleo-docs: Remove references to balance-tcp  https://review.openstack.org/37259015:45
openstackgerritMerged openstack/tripleo-quickstart: Add all the gating functionality to full-deploy.sh  https://review.openstack.org/37019415:45
openstackgerritMerged openstack/tripleo-quickstart: Add missing provison tag to quickstart.yml  https://review.openstack.org/37236115:46
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Add IPv6 network configuration for ipv6 job types  https://review.openstack.org/36367415:47
bandinithanks jistr, marios15:48
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Use a released version of tripleo-common  https://review.openstack.org/36442515:48
*** flepied has quit IRC15:49
jaosoriorthrash: duuuude15:51
jaosoriorthrash: ah, wait, nevermind15:51
*** pkovar has joined #tripleo15:51
EmilienMdciabrin: https://review.openstack.org/#/c/371031/15:56
EmilienM\o/15:56
EmilienMliberty jobs are green again15:56
*** hrybacki is now known as hrybacki|mtg15:56
openstackgerritMerged openstack/tripleo-heat-templates: Set VNC URL parameters for nova-compute  https://review.openstack.org/37190315:57
*** athomas has quit IRC15:59
jaosoriord0ugal: so... is there any workaround that you know of?15:59
*** pkovar has quit IRC16:00
jaosoriorEmilienM: now that this merged https://review.openstack.org/#/c/371903/ will this be included in newton? or do I have to cherry pick once there's a stable/newton branch?16:00
openstackgerritAlex Schultz proposed openstack/instack-undercloud: Add sysctl preflight install check  https://review.openstack.org/37260516:01
d0ugaljaosorior: I think you need to cherry-pick it.16:01
EmilienMno16:01
d0ugaljaosorior: I don't know of a workaround :/16:01
EmilienMwe don't have stable/newton for THT16:01
EmilienMand we won't have until RC2 is out16:01
d0ugaloh16:01
EmilienMbackport only tripleoclient.16:01
* d0ugal should be ignored16:01
openstackgerritMerged openstack/tripleo-quickstart: Fixes for devmode convert_image  https://review.openstack.org/37244116:02
*** dprince has quit IRC16:03
jaosoriorthrash: ok, duuuude finally got zaqar websockets working on CI https://review.openstack.org/#/c/360350/16:03
*** athomas has joined #tripleo16:03
*** fzdarsky has quit IRC16:04
d0ugalIf anyone is interested in the Mistral issues we have had lately in tripleo, the mistral meeting is starting in #openstack-meeting16:07
EmilienMd0ugal: thanks!16:08
dciabrinEmilienM, woot16:08
*** lucasagomes is now known as lucas-hungry16:08
*** stendulker has joined #tripleo16:12
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Fix and improve the ansible-lint gate job  https://review.openstack.org/36675816:14
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Fix ansible-lint errors in all playbooks and roles  https://review.openstack.org/36683716:14
*** rwsu has joined #tripleo16:14
*** ohamada has quit IRC16:16
*** jlinkes has quit IRC16:24
*** abehl has quit IRC16:24
*** dciabrin is now known as dciabrin|away16:24
*** dprince has joined #tripleo16:28
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Fix and improve the ansible-lint gate job  https://review.openstack.org/36675816:28
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Fix ansible-lint errors in all playbooks and roles  https://review.openstack.org/36683716:28
*** ebarrera has joined #tripleo16:30
*** gfidente is now known as gfidente|afk16:32
thrashjaosorior: excellent16:34
openstackgerritBrad P. Crochet proposed openstack-infra/tripleo-ci: debug: Run 'openstack stack failures list' upon overcloud failures  https://review.openstack.org/36943416:37
*** jbadiapa has quit IRC16:37
openstackgerritJulie Pichon proposed openstack/python-tripleoclient: Remove openstackclient imports in the new parameters command  https://review.openstack.org/37262116:39
*** jbadiapa has joined #tripleo16:41
jaosoriorEmilienM, shardy currently zaqar websockets are still plaintext even if we enable SSL in the undercloud... I have a fix for that. But not sure if we should consider it a bug and get it into newton, or if we'll defer that to Ocata16:43
*** jbadiapa has quit IRC16:44
bnemecd0ugal: Did something else merge that makes https://review.openstack.org/#/c/371027/ work?  It doesn't look like the instack-undercloud fix has yet.16:44
d0ugalbnemec: I split the change up - note it's a chain of three now.16:46
d0ugalbnemec: and it depends on the mistral workaround I am using16:46
*** jpich has quit IRC16:47
d0ugalbnemec: we might need to find a way to hack this in until Mistral figures out a proper solution: https://review.openstack.org/#/c/371435/16:47
bnemecd0ugal: Ah crap, the gerrit ui was cutting off in just the right spot so I didn't see there were related changes. :-)16:47
d0ugalbnemec: heh, gerrit UI is the best16:48
openstackgerritAna Krivokapic proposed openstack/tripleo-common: Support node untagging  https://review.openstack.org/37262816:48
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Add devmode.sh and document Third Party CI  https://review.openstack.org/36000716:50
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Fix 3rd part gate repo injection  https://review.openstack.org/37263016:50
*** hrybacki|mtg is now known as hrybacki16:51
*** rcernin has joined #tripleo16:51
ccamachohey jistr gfidente, there is a dependency issue for liberty when creating the overcloud images it's reported so, ill give you some feedback after the logs are here https://paste.fedoraproject.org/431066/47430360/16:56
*** lucas-hungry is now known as lucasagomes16:57
*** bana_k has joined #tripleo17:01
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375017:01
*** derekh has quit IRC17:02
*** panda|Zz is now known as panda17:07
openstackgerritAna Krivokapic proposed openstack/tripleo-ui: Integrate node tagging workflow  https://review.openstack.org/36756217:08
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates: Use correct password for keystone bootstrap  https://review.openstack.org/37263517:10
slaglebnemec: was there a launchpad bug for the undercloud install prompting for a password?17:12
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates: Use correct password for keystone bootstrap  https://review.openstack.org/37263517:15
bnemecslagle: There was, but I'm having trouble finding it.  It may have gotten duped, which hides it from search results.  Let me see if I can find it in my email history.17:19
slaglebnemec: i found https://bugs.launchpad.net/python-openstackclient/+bug/161927417:19
openstackLaunchpad bug 1619274 in python-openstackclient "openstack complete triggers password prompt" [Critical,Fix released] - Assigned to Dean Troyer (dtroyer)17:19
slaglei think that was it17:20
slaglecouldnt find anything in tripleo17:20
EmilienMmwhahaha: nice patch for keystone, you can also do instack-undercloud probably17:20
mwhahahak i'll find it17:20
*** trown is now known as trown|lunch17:21
bnemecslagle: Yep, that was it: https://bugs.launchpad.net/python-openstackclient/+bug/162152417:21
openstackLaunchpad bug 1619274 in python-openstackclient "duplicate for #1621524 openstack complete triggers password prompt" [Critical,Fix released] - Assigned to Dean Troyer (dtroyer)17:21
openstackgerritMerged openstack/tripleo-common: Check the result of the parent action when subclassing  https://review.openstack.org/36808117:23
slaglebnemec: hmm. i don't see the fix on the stable/newton branch of osc17:23
openstackgerritMerged openstack/tripleo-heat-templates: Add a function to upgrade from full HA to NG HA  https://review.openstack.org/35862617:23
openstackgerritMerged openstack/tripleo-heat-templates: MySQL Galera backports  https://review.openstack.org/37103117:23
slaglebnemec: i wonder if it needs backporting17:23
openstackgerritAlex Schultz proposed openstack/instack-undercloud: Use correct password for keystone bootstrap  https://review.openstack.org/37264017:23
bnemecslagle: Ah, that might explain why I'm still seeing it happen.17:23
bnemecI thought maybe we weren't using latest osc due to u-c or something.17:24
*** hewbrocca is now known as hewbrocca-afk17:24
*** ebarrera has quit IRC17:29
*** ebarrera has joined #tripleo17:31
*** akrivoka has quit IRC17:33
*** jaosorior has quit IRC17:36
*** jpena is now known as jpena|off17:37
*** rasca has quit IRC17:37
*** abehl has joined #tripleo17:38
pandawhy experimental job did not start on this :( https://review.openstack.org/36367417:38
pandaoh17:39
pandanevermind17:39
pandaI had to refresh the cache17:39
EmilienMshardy: https://review.openstack.org/#/c/369643/2/specs/branch-automation.rst17:45
EmilienMshardy: next cycle we'll use that to say when we want branches17:45
EmilienMpanda: job are in queue17:46
pandaEmilienM: do you have any suggestion to speed up progress on IPv6 ? I think my last run got the galera/IPv6 error from friday. Should I test more locally ?17:49
EmilienMpanda: speedup what?17:49
pandaAnd I think that one happens before setting up swift17:49
EmilienMwe merged a few patches this week end, let's see how it goes now17:50
pandaEmilienM: speed up testing and fixing error for Ipv617:50
EmilienMpanda: the mongodb failure is something we had in all CI jobs, we fixed it17:50
EmilienMpanda: maybe local testing17:50
pandaEmilienM: ok, how do you test locally ? external box wit manually created vms, then tripleo.sh ?17:51
*** myoung|bbl is now known as myoung17:51
shardyEmilienM: ack, sounds good17:51
EmilienMpanda: I'm sharing a box with mwhahaha17:52
EmilienMand I also test locally on my laptop, for undercloud17:52
*** shardy has quit IRC17:52
pandaEmilienM: the shared box is set up with manually created vm and tripleo.sh ?17:53
EmilienMpanda: I think mwhahaha used oooq and me, I'm using both tools17:55
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375017:55
pandaEmilienM: ok17:55
EmilienMpanda: it depends of what i want to achieve17:55
pandaEmilienM: uh ?17:56
EmilienMpanda: I'm mixing tools17:57
*** yamahata has joined #tripleo18:02
dprinceEmilienM: are you asking me to create the ocata directory here? https://review.openstack.org/#/c/352113/18:03
dprinceEmilienM: I don't like -2's on these things as it means people won't look at them (some people that is)18:04
EmilienMdprince: yes, please18:04
dprinceEmilienM: a -1 would do fine, FWIW18:04
EmilienMdprince: I -2 because newton is finished18:04
EmilienMdprince: done18:04
dprinceEmilienM: regardless, -2 is sticky and won't go away when I update it18:05
openstackgerritimain proposed openstack/python-tripleoclient: Add container support to undercloud  https://review.openstack.org/37267118:05
EmilienMdprince: I updated18:05
dprinceEmilienM: the directory change is just a minor thing, arguably a bug that we even have directories for specs I think18:05
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Enable L3 HA when multiple controllers and no DVR  https://review.openstack.org/37192618:06
dprincebeagles: hey, you pinged me about ^^ on Friday I think18:07
dprincebeagles: things going okay now?18:07
*** electrofelix has quit IRC18:07
*** tosky has quit IRC18:07
beaglesdprince: yup, yes, thanks! shardy pointed me in the right direction18:07
*** stendulker has quit IRC18:07
*** paramite has quit IRC18:08
*** andrey-mp has joined #tripleo18:10
*** trown|lunch is now known as trown18:11
andrey-mphi! how to install mitaka version with tripleo? I faced with problem with building image. tripleo-image-elements sets delorean-deps.repo to newton repository in any case - https://github.com/openstack/tripleo-image-elements/blob/master/elements/delorean-repo/pre-install.d/01-delorean-repo18:13
andrey-mpis it a bug or just a strange feature?18:13
*** athomas has quit IRC18:18
*** milan has quit IRC18:23
*** flepied has joined #tripleo18:25
*** ebarrera has quit IRC18:34
openstackgerritBen Nemec proposed openstack/tripleo-docs: Document decoupled SSL config  https://review.openstack.org/34803818:34
*** flepied1 has joined #tripleo18:35
*** jaosorior has joined #tripleo18:36
jaosoriorEmilienM: hey dude, soo... why the -1 here? https://review.openstack.org/#/c/282307/18:37
jaosoriorslagle: now that rc1 has been released can you remove the -2 from this? https://review.openstack.org/#/c/368643/18:37
EmilienMjaosorior: just to move it into ocata18:37
EmilienMjaosorior: update the folder etc18:37
EmilienMjaosorior: I don't think he can remove -2, ocata is not open for development yet18:38
jaosoriorEmilienM: will do18:38
jaosoriorEmilienM: aah ok18:38
jaosoriorthought it was18:38
jaosoriorfair enough then18:38
jaosoriorEmilienM: did you see my question about zaqar's websocket SSL18:38
slaglejaosorior: we're not branched. master is not open for new features18:38
slaglei'm glad the -2 is there :)18:38
EmilienMjaosorior: no, we released RC1 but not final18:38
*** flepied has quit IRC18:39
jaosoriorslagle: not sure why you would be glad, but sure, good that you are I guess18:39
slaglejaosorior: so that it is not merged18:39
jaosoriorslagle: do you think zaqar listening on plaintext is considered a bug? Got a CR for that. But not sure if it's a feature for Ocata or is it considered a bug18:40
EmilienMjaosorior: is there a FFE for certmonger?18:40
jaosoriorEmilienM: the question was not that much about the certmonger blueprint, but about zaqar's websocket endpoint listening on plaintext18:41
slaglejaosorior: which bug is it?18:41
EmilienMif not, the feature can't land in Newton cycle, and we have to block it until ocata is open, that's why James -2'ed, to respect our release policy18:41
jaosoriorslagle: there is no bug filed for it; because I don't know if I can set it as a bug or not, which is why I ask18:41
jaosoriorEmilienM: ok, sorry about the lack of context18:41
jaosoriorEmilienM: before I logged out, I asked you and shardy about zaqar's websocket SSL (which is a different patch)18:42
EmilienMjaosorior: no worries18:42
jaosoriorright now, it's plaintext, even if we enable SSL for the undercloud18:42
slaglejaosorior: if you want it in newton, and it's not a FFE, we need a bug18:42
EmilienM:(18:42
jaosoriorin my opinion, that's a bug.18:42
EmilienMyes, right18:42
EmilienMif it's a bug, we can land the code in this cycle or backport later18:42
slaglejaosorior: it sounds reasonable to me18:43
jaosoriorjust wanted to know your opinion about that, should I file a bug for it?18:43
slaglereasonable for newton i mean18:43
jaosoriorslagle: will file a bug18:43
jaosoriorthanks guys18:43
jaosorioralright, I'm off18:44
jaosoriorhave a good day folks!18:44
jaosoriorpretty late here18:44
*** jaosorior has quit IRC18:44
openstackgerritMerged openstack/tripleo-quickstart: Fix 3rd part gate repo injection  https://review.openstack.org/37263018:47
openstackgerritAlex Schultz proposed openstack/instack-undercloud: Add sysctl preflight install check  https://review.openstack.org/37260518:48
openstackgerritMerged openstack/tripleo-quickstart: Fix and improve the ansible-lint gate job  https://review.openstack.org/36675818:48
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Enable L3 HA when multiple controllers and no DVR  https://review.openstack.org/37192618:49
openstackgerritMerged openstack/tripleo-quickstart: Fix ansible-lint errors in all playbooks and roles  https://review.openstack.org/36683718:49
*** panda is now known as panda|dinner18:55
*** spredzy has quit IRC18:55
*** spredzy has joined #tripleo18:56
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375018:58
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Ceilometer Wsgi Mitaka->Newton upgrades  https://review.openstack.org/36000419:03
*** yamahata has quit IRC19:04
*** yamahata has joined #tripleo19:05
openstackgerritAlex Schultz proposed openstack/instack-undercloud: Add sysctl preflight install check  https://review.openstack.org/37260519:13
*** panda|dinner is now known as panda19:14
slaglebnemec: i proposed a backport of the osc fix19:15
*** dtrainor has joined #tripleo19:15
bnemecslagle: Nice, thanks19:15
*** paramite has joined #tripleo19:16
*** r-mibu has quit IRC19:17
*** davidlenwell has quit IRC19:17
*** r-mibu has joined #tripleo19:17
larsksIs there a fix for the "No connection adapters were found for 'file:..." problem?19:26
larsksI am having a hard time finding the relevant bug in lp.19:27
EmilienMpanda: something is wrong with https://review.openstack.org/#/c/363674/19:27
EmilienMall ovb jobs are failing19:28
trownlarsks: I havent seen that one yet19:28
EmilienMi'm looking into it19:28
larskstrown, try a basic oooq "overcloud-deploy"  with https://trunk.rdoproject.org/centos7-master/current/delorean.repo19:28
larskstrown, after a 'yum upgrade "*tripleo*"'19:29
larskson the undercloud19:29
*** davidlenwell has joined #tripleo19:29
*** ChanServ sets mode: +v davidlenwell19:29
pandaEmilienM: network problems19:29
pandaEmilienM: http://buildlogs.centos.org/centos/7/cloud/x86_64/openstack-newton/common/python-simplejson-3.5.3-5.el7.x86_64.rpm: [Errno 14] HTTP Error 302 - Found19:30
EmilienMpanda: ok19:30
panda[Errno 12] Timeout on http://buildlogs.cdn.centos.org/centos/7/cloud/x86_64/openstack-newton/common/python-simplejson-3.5.3-5.el7.x86_64.rpm: (28, 'Connection timed out after 30962 milliseconds')19:30
* larsks checks if a heat upgrade helps things.19:30
EmilienMpanda: i'll double check if params are good enough to trigger ipv619:31
panda/o\ other 6 hours to wait for this job to get in queue :(19:31
trownlarsks: k, will give it a go after my tripleo.sh deploy using https://review.openstack.org/#/c/371587/ passes19:31
trownor fails :P19:31
pandaEmilienM: I'm setting up local tests19:31
trownlarsks: the "--release master-tripleo-ci" might get you going though, that will use the overcloud-full.qcow2 from tripleo-ci and update the repos to match a tripleo-ci run19:32
EmilienMpanda: it's not good19:33
EmilienMit miss /usr/share/openstack-tripleo-heat-templates/environments/net-multiple-nics-v6.yaml19:33
EmilienMi'm adding it19:33
bnemecpanda: I can save you some time.  ipv6 is still broken: https://bugs.launchpad.net/tripleo/+bug/162533519:33
openstackLaunchpad bug 1625335 in tripleo "swift-proxy fails to start with ipv6" [Critical,Triaged]19:33
EmilienMpanda: I think?19:33
bnemecEmilienM: No, you don't want that environment.19:33
larskstrown, I will give that a shot.  I am currently using @config/release/master.yml19:33
EmilienMbnemec: mhh ok so why are we using it in update job?19:33
bnemecIt doesn't work with an OVB env.19:33
EmilienMwe use it in ovb-updates19:33
bnemecEmilienM: The update job has never actually been updated for OVB to my knowledge.19:33
EmilienMsee toci_gate_test.sh19:33
EmilienMok good19:34
EmilienMbnemec: ok thanks for the pointer19:34
EmilienMbnemec: for swift, your bug is different from what fixed panda ?19:34
bnemecEmilienM: Yeah, in this case the deployment succeeds, but when you try to ping test it you find out swift isn't actually running.19:34
EmilienMbnemec: so we are close :-)19:34
bnemecIt's at least different behavior from what I was seeing before. :-)19:35
bnemecI mean, I saw "Overcloud deployed"19:35
EmilienMbnemec: do you have it running in CI ? or your local env? It would be nice to pastebin the swift config19:35
pandabnemec: the deployment succeeds with my bugfix too ?19:35
* bnemec ships it19:35
EmilienMpanda: yes :) thanks again19:35
bnemecpanda: Yes, I just deployed what is in master right now, no extra patches.19:35
EmilienMnice19:35
trownlarsks: maybe `yum upgrade "*tripleo*"` is not upgrading something tripleo depends on... would be a packaging bug I suppose19:35
pandaok, you want me to fix the swift proxy too ?19:35
EmilienMbnemec: can you pastebin your swift config please?19:36
EmilienMI'm investigating why it broke in newton19:36
EmilienMit sounds like the memcache option19:36
pandaanyway. yay \o/ I fixed something!19:37
EmilienMok I got it19:37
EmilienMpanda: I got the memcache thing19:38
EmilienMbnemec: do you have your env working? i have a patch I would like you to test19:38
*** andrey-mp has left #tripleo19:38
bnemecEmilienM: http://paste.openstack.org/show/581195/19:39
bnemecBoth bind_ip and the memcache addresses are unbracketed.19:39
EmilienMyes I found the problem, I'm preparing a patch19:39
EmilienMbnemec: nice catch btw19:39
EmilienMthe bind_ip should not be a problem but I'm not sure19:39
bnemecEmilienM: Yeah, I see the bit about memcache in the traceback now.19:40
bnemecEmilienM: Define "working" :-)19:40
EmilienMbnemec: can you try to manually set brackets only for memcache option?19:40
pandaEmilienM: https://bugs.launchpad.net/tripleo/+bug/1625335 this one is memcached problem and you're working on it ?19:40
openstackLaunchpad bug 1625335 in tripleo "swift-proxy fails to start with ipv6" [Critical,Triaged] - Assigned to Emilien Macchi (emilienm)19:40
EmilienMpanda: yes19:40
pandaEmilienM:  ok, bnemec: there's still one more thing to check, my previous patch broke Ipv4 deployment, I have to check if the latest patch works for Ipv419:41
*** myoung is now known as myoung|bbl19:41
EmilienMpanda: which patch broke ipv4?19:41
*** _coolsvap_ has quit IRC19:42
bnemecEmilienM: Bracketing the memcache ips got swift to start.19:42
EmilienMbnemec: can you try to set memcache_servers = [fd00:fd00:fd00:2000::1e]:11211,[fd00:fd00:fd00:2000::14]:11211,[fd00:fd00:fd00:2000::17]:1121119:42
EmilienMbnemec: good19:42
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: swift: normalize memcache servers IP addresses  https://review.openstack.org/37273419:44
*** jprovazn has quit IRC19:44
EmilienMbnemec, panda: it should fix the bug ^19:44
mwhahahafun with ipv619:45
EmilienMpanda: can you depends-on my patch in your tripleo-ci and run experimental job again please?19:45
pandamwhahaha: fun with string handling19:45
pandaEmilienM: sure19:45
EmilienMbnemec: my patch should make the output like I pasted with the "can you try to set..."19:46
larskstrown, finally, found the bug: https://bugs.launchpad.net/tripleo/+bug/162355219:46
openstackLaunchpad bug 1623552 in tripleo "heatclient resolves paths for types and get_file calls that then don't make sense in swift" [Critical,Confirmed]19:46
bnemecWhoever designed ipv6 should be shot.19:46
bnemec"Hey guys, you know how colons are used to delimite ports and mac addresses?  Well let's use them for ipv6 too!"19:46
bnemecWCPGW?19:47
bnemec*delimit19:47
bnemecEmilienM: Is your patch posted?19:48
EmilienMbnemec: yes, https://review.openstack.org/37273419:48
openstackgerritJames Slagle proposed openstack/tripleo-docs: Update docs for undercloud upgrade  https://review.openstack.org/36813819:49
EmilienMchem: the normalize function is the most useful thing I've ever seen in ipv6 work :-P19:49
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Add IPv6 network configuration for ipv6 job types  https://review.openstack.org/36367419:49
larsksd0ugal, I've been reviewing irc logs re: https://bugs.launchpad.net/tripleo/+bug/1623552 and it looks like the current state is "worksforme"? Except maybe for updating a plan?19:50
openstackLaunchpad bug 1623552 in tripleo "heatclient resolves paths for types and get_file calls that then don't make sense in swift" [Critical,Confirmed]19:50
bnemecEmilienM: panda: With the manual fix, the ping test succeeds.  So this should be the last thing.19:54
bnemecI'm testing the patch locally.19:54
* panda dances19:54
EmilienMbnemec: ok thanks19:54
bnemecBleh, I can't test this locally.19:58
bnemecAt least not as easily as I thought.19:58
bnemecSecond deployment with the same plan blows up.19:58
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Include ceilometer in swift proxy pipeline  https://review.openstack.org/37195019:59
*** lucasagomes is now known as lucas-afk20:03
*** jayg is now known as jayg|g0n320:12
pandaI think I can go to sleep happy today.20:13
eggmasterpanda: does that mean you are sleeping these days?? I hope so :)20:16
trownsleep is overrated20:17
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: add tempest back into the quickstart-extras workflow  https://review.openstack.org/37274820:21
pandaeggmaster: I am, but the amount of hours per night in the last months created a permanent loss in long term memory. Who are you btw ?20:21
eggmasterpanda: oh, nobody :)20:22
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: add tempest back into the quickstart-extras workflow  https://review.openstack.org/37274820:22
*** panda is now known as panda|Zz20:28
openstackgerritLokesh Jain proposed openstack/tripleo-heat-templates: Updates for configuring Nuage in Mitaka release  https://review.openstack.org/37274920:31
*** radeks has joined #tripleo20:34
*** rwsu has quit IRC20:36
*** radeks has quit IRC20:37
*** Goneri has quit IRC20:39
*** yolanda has joined #tripleo20:50
EmilienMTripleO RC1 is out today, congrats everyone!20:53
trownnice one20:55
*** mgarciam has quit IRC20:56
*** mgarciam has joined #tripleo20:56
openstackgerritMerged openstack/python-tripleoclient: Use a released version of tripleo-common  https://review.openstack.org/36442520:57
*** lblanchard has quit IRC20:58
*** mgarciam has quit IRC20:59
*** mgarciam has joined #tripleo21:01
*** mgarciam has quit IRC21:01
*** trown is now known as trown|outtypewww21:04
thrashEmilienM: this look sane? https://review.openstack.org/#/c/372698/21:12
EmilienMthrash: looks like it does21:13
thrashsweet... I can be puppet challenged. :)21:13
*** rhallisey has quit IRC21:21
EmilienMthrash: i'll challenge you dude, I have more bugs21:26
bkeroWho has the most bugs?21:27
thrashEmilienM: I meant that more in the 'I don't know what I'm doing' sense. :)21:30
EmilienMthrash: my1.fr/files/emilien-right-now.jpg ?21:30
thrashEmilienM: but puppet challenges would be good...21:31
thrashEmilienM: YAASSSS21:31
bkerohttps://pbs.twimg.com/media/CSVzYL-XAAAcZhX.jpg21:32
*** bnemec has quit IRC21:33
*** fultonj has quit IRC21:35
openstackgerritLokesh Jain proposed openstack/tripleo-heat-templates: Updates for configuring Nuage in Mitaka release  https://review.openstack.org/37274921:37
*** akshai has quit IRC21:39
*** abehl has quit IRC21:44
*** yolanda has quit IRC21:51
*** spredzy has quit IRC21:51
*** gfidente|afk has quit IRC21:51
*** jtomasek has quit IRC21:51
*** tdasilva has quit IRC21:51
*** gchamoul has quit IRC21:51
*** spredzy has joined #tripleo21:51
*** gchamoul has joined #tripleo21:51
*** jtomasek has joined #tripleo21:51
*** yolanda has joined #tripleo21:51
*** gfidente|afk has joined #tripleo21:51
*** rwsu has joined #tripleo21:53
*** dprince has quit IRC21:54
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375021:59
*** yolanda has quit IRC22:02
*** dtrainor has quit IRC22:03
*** tdasilva has joined #tripleo22:06
*** cdearborn has quit IRC22:07
openstackgerritimain proposed openstack/python-tripleoclient: Add container support to undercloud  https://review.openstack.org/37267122:08
openstackgerritimain proposed openstack/python-tripleoclient: Deploy the undercloud with Heat  https://review.openstack.org/35135122:08
*** pmannidi has joined #tripleo22:10
*** social has quit IRC22:16
*** rcernin has quit IRC22:27
*** gfidente|afk has quit IRC22:31
*** mhenkel has joined #tripleo22:39
mhenkelHi all22:39
mhenkeltrying to run instack-virt-setup to bring up an undercloud22:40
mhenkelscript stops and tells me:22:40
mhenkelPool is restricted to virtual guests: ‘***[POOLID]***22:40
*** mhenkel has quit IRC22:52
*** yamahata has quit IRC22:52
*** chem` has joined #tripleo22:53
*** chem has quit IRC22:55
*** chem`` has joined #tripleo22:58
*** chem` has quit IRC23:00
*** limao_ has quit IRC23:06
*** chem``` has joined #tripleo23:11
*** chem`` has quit IRC23:13
*** mhenkel has joined #tripleo23:15
*** mhenkel has quit IRC23:20
*** chem``` has quit IRC23:21
*** rajinir has quit IRC23:25
openstackgerritIan Wienand proposed openstack/diskimage-builder: yum-minimal: Disable excludes when installing pkg manager  https://review.openstack.org/37280823:32
*** saneax-_-|AFK is now known as saneax23:37
*** openstackgerrit has quit IRC23:42
*** bkreitch has quit IRC23:42
*** mburned` has joined #tripleo23:43
*** mburned has quit IRC23:43
*** bkreitch has joined #tripleo23:46
*** openstackgerrit has joined #tripleo23:46
*** openstackgerrit has quit IRC23:51
*** hogepodge has quit IRC23:53
*** myoung|bbl is now known as myoung23:53
*** openstackgerrit has joined #tripleo23:54
*** hogepodge has joined #tripleo23:56
openstackgerritMerged openstack/instack-undercloud: Catch runtime exceptions during validation  https://review.openstack.org/37180223:57
*** dtrainor has joined #tripleo23:59
*** dtrainor has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!