*** yamahata has quit IRC | 00:04 | |
*** yamahata has joined #tripleo | 00:06 | |
*** chlong has joined #tripleo | 00:12 | |
*** shadower has quit IRC | 00:23 | |
*** shadower has joined #tripleo | 00:23 | |
*** shivrao has joined #tripleo | 00:32 | |
openstackgerrit | Zane Bitter proposed openstack/tripleo-heat-templates: Prototype SW deployments for Controller upgrades https://review.openstack.org/234485 | 00:36 |
---|---|---|
*** sdake has joined #tripleo | 00:43 | |
*** rbrady has quit IRC | 00:47 | |
*** mudassirlatif has joined #tripleo | 00:49 | |
*** tiswanso_ has quit IRC | 00:50 | |
openstackgerrit | Colleen Murphy proposed openstack/diskimage-builder: Fix devuser pubkey defaults https://review.openstack.org/234490 | 00:53 |
*** links has joined #tripleo | 00:53 | |
*** mudassirlatif has quit IRC | 00:54 | |
*** lazy_prince has joined #tripleo | 00:58 | |
*** sdake has quit IRC | 00:58 | |
*** jcoufal_ has quit IRC | 01:01 | |
*** spzala has quit IRC | 01:02 | |
*** sdake has joined #tripleo | 01:03 | |
*** Goneri has quit IRC | 01:05 | |
*** shivrao has quit IRC | 01:09 | |
*** aukhan has joined #tripleo | 01:11 | |
*** AzherKhan has joined #tripleo | 01:12 | |
*** julim has joined #tripleo | 01:19 | |
*** tiswanso has joined #tripleo | 01:27 | |
*** lazy_prince has quit IRC | 01:31 | |
*** lazy_prince has joined #tripleo | 01:32 | |
*** stendulker has joined #tripleo | 01:32 | |
*** killer_prince has joined #tripleo | 01:33 | |
*** AzherKhan has quit IRC | 01:33 | |
*** aukhan has quit IRC | 01:33 | |
*** lazy_prince has quit IRC | 01:37 | |
*** rhallisey has quit IRC | 01:42 | |
*** fvollero has quit IRC | 01:58 | |
*** shardy has quit IRC | 02:01 | |
*** jaosorior has joined #tripleo | 02:05 | |
*** sdake has quit IRC | 02:06 | |
*** chlong has quit IRC | 02:06 | |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Make root directory configurable https://review.openstack.org/230275 | 02:08 |
*** stendulker has quit IRC | 02:09 | |
*** weshay_xchat has quit IRC | 02:13 | |
*** rlandy has quit IRC | 02:14 | |
*** chlong has joined #tripleo | 02:23 | |
*** jaosorior has quit IRC | 02:29 | |
*** aukhan has joined #tripleo | 02:39 | |
*** killer_prince has quit IRC | 02:40 | |
*** killer_prince has joined #tripleo | 02:40 | |
*** yamahata has quit IRC | 02:40 | |
*** killer_prince has quit IRC | 02:48 | |
*** lazy_prince has joined #tripleo | 02:48 | |
*** penick has joined #tripleo | 02:50 | |
*** lazy_prince has quit IRC | 02:53 | |
*** lazy_prince has joined #tripleo | 02:54 | |
*** untriaged-bot has joined #tripleo | 03:00 | |
untriaged-bot | Untriaged bugs so far: | 03:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1505815 | 03:00 |
openstack | Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,New] | 03:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1504363 | 03:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1504263 | 03:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1505430 | 03:00 |
*** untriaged-bot has quit IRC | 03:00 | |
openstack | Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] | 03:00 |
openstack | Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] | 03:00 |
openstack | Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) | 03:00 |
*** lazy_prince has quit IRC | 03:05 | |
*** tiswanso has quit IRC | 03:08 | |
*** penick has quit IRC | 03:19 | |
*** chlong has quit IRC | 03:29 | |
*** penick has joined #tripleo | 03:35 | |
*** chlong has joined #tripleo | 03:45 | |
*** mudassirlatif has joined #tripleo | 03:45 | |
*** mudassirlatif has quit IRC | 03:50 | |
*** mudassirlatif has joined #tripleo | 03:52 | |
*** stendulker has joined #tripleo | 03:54 | |
*** sthillma has joined #tripleo | 03:59 | |
*** david-lyle has quit IRC | 04:01 | |
*** sthillma_ has joined #tripleo | 04:02 | |
*** stendulker_ has joined #tripleo | 04:02 | |
*** stendulker has quit IRC | 04:02 | |
*** sthillma has quit IRC | 04:03 | |
*** sthillma_ is now known as sthillma | 04:04 | |
*** stendulker has joined #tripleo | 04:13 | |
*** stendulker_ has quit IRC | 04:13 | |
*** AzherKhan has joined #tripleo | 04:37 | |
*** chlong has quit IRC | 04:49 | |
*** tzumainn has quit IRC | 04:51 | |
*** penick has quit IRC | 04:57 | |
*** lazy_prince has joined #tripleo | 05:22 | |
*** BharatK has joined #tripleo | 05:34 | |
*** masco has joined #tripleo | 05:44 | |
*** leanderthal has joined #tripleo | 05:46 | |
*** AzherKhan has quit IRC | 05:52 | |
*** sthillma has quit IRC | 05:58 | |
*** chlong has joined #tripleo | 06:19 | |
*** ukalifon1 has joined #tripleo | 06:27 | |
*** chlong has quit IRC | 06:27 | |
*** Marga_ has quit IRC | 06:42 | |
*** aufi has joined #tripleo | 06:43 | |
*** jprovazn has joined #tripleo | 06:45 | |
*** yog_ has joined #tripleo | 06:48 | |
*** shardy has joined #tripleo | 06:57 | |
*** yog_ has quit IRC | 07:00 | |
*** openstackgerrit has quit IRC | 07:01 | |
*** openstackgerrit has joined #tripleo | 07:01 | |
*** matbu has quit IRC | 07:04 | |
*** matbu has joined #tripleo | 07:06 | |
*** Marga_ has joined #tripleo | 07:11 | |
*** yog_ has joined #tripleo | 07:13 | |
*** Marga_ has quit IRC | 07:14 | |
*** Marga_ has joined #tripleo | 07:14 | |
*** dtantsur|afk is now known as dtantsur | 07:20 | |
*** ifarkas has joined #tripleo | 07:32 | |
*** mkovacik has joined #tripleo | 07:33 | |
*** gfidente has joined #tripleo | 07:38 | |
*** mkovacik has quit IRC | 07:40 | |
*** matbu has quit IRC | 07:44 | |
*** matbu_ has joined #tripleo | 07:44 | |
d0ugal | What is our deprecation policy for the CLI interface. Do we have an official one? | 07:51 |
d0ugal | Looking at this change - https://review.openstack.org/#/c/232105/ | 07:51 |
d0ugal | shardy: You mentioned deprecation the other day, what do you think? | 07:52 |
shardy | d0ugal: Most projects would not allow that sort of change, you'd have to support the old option with a warning for a while | 07:54 |
shardy | d0ugal: In heat we normally observe at least 2 cycles (e.g a year) | 07:54 |
shardy | I think that is typical | 07:54 |
shardy | Just ripping out options is not cool at all IMHO | 07:54 |
shardy | d0ugal: has that option been shipped in either a tagged pypi release, or downstream packages? | 07:56 |
d0ugal | shardy: Yeah, that makes sense - I guess we just need to figure out our poilicy and go with it | 07:57 |
d0ugal | I am not sure if it is released anywhere :) | 07:57 |
d0ugal | Not knowing what is and isn't released is part of the issue | 07:57 |
*** david-lyle has joined #tripleo | 07:57 | |
shardy | d0ugal: that is a good reason to start releasing regularly via pypi and rebasing downstream on those releases | 07:57 |
shardy | that's what most other clients do AFAIK | 07:58 |
*** hewbrocca has joined #tripleo | 07:58 | |
d0ugal | shardy: Makes sense, I guess I might need to take this to openstack-dev :) | 07:59 |
*** jcoufal has joined #tripleo | 07:59 | |
shardy | d0ugal: sounds good, I'll reply there | 07:59 |
d0ugal | shardy: Do you happen to know if there is an online reference I can use for other projects? like Heat? | 08:00 |
*** Marga_ has quit IRC | 08:01 | |
shardy | d0ugal: http://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html is one reference | 08:05 |
shardy | I think that's derived from some previous docs but I'm failing to find them atm | 08:05 |
* shardy notes Heat needs to be added there | 08:05 | |
d0ugal | shardy: Thanks | 08:06 |
shardy | d0ugal: for the most part it's common sense tho, e.g don't break your users without fair warning ;) | 08:06 |
d0ugal | Sure, that is nice in theory :) | 08:07 |
*** athomas has joined #tripleo | 08:08 | |
*** mbound has joined #tripleo | 08:15 | |
*** jistr has joined #tripleo | 08:18 | |
openstackgerrit | Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to locally build delorean package https://review.openstack.org/228157 | 08:20 |
openstackgerrit | Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to create a local Delorean environment https://review.openstack.org/228156 | 08:20 |
*** masco has quit IRC | 08:20 | |
openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Use tripleo.sh to build packages https://review.openstack.org/229425 | 08:21 |
openstackgerrit | Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 | 08:25 |
*** devvesa has joined #tripleo | 08:28 | |
*** masco has joined #tripleo | 08:33 | |
*** olap has quit IRC | 08:33 | |
*** matbu_ is now known as matbu | 08:37 | |
*** lucasagomes has joined #tripleo | 08:41 | |
*** mkovacik has joined #tripleo | 08:41 | |
spredzy | Has any of you already encountered "ERROR: InterpreterNotFound: python3.4" in python-tripleoclient py34 tox job ? | 08:42 |
*** lucasagomes_ has joined #tripleo | 08:42 | |
spredzy | (link. http://logs.openstack.org/95/231395/4/check/gate-python-tripleoclient-python34/a71fadb/console.html) | 08:42 |
gchamoul | spredzy: yes but with tripleo-common! | 08:44 |
gchamoul | spredzy: http://logs.openstack.org/08/234108/1/check/gate-tripleo-common-python34/d05120f/console.html | 08:44 |
spredzy | gchamoul, any workaround ? | 08:45 |
*** lucasagomes has quit IRC | 08:46 | |
gchamoul | spredzy: on the gate, nope ... but is py34 tox job working locally? | 08:47 |
spredzy | gchamoul, locally I have gcc issues (cffi related), so I was expecting the gate to let me know :) | 08:49 |
*** derekh has joined #tripleo | 08:50 | |
*** paramite has joined #tripleo | 08:54 | |
*** lucasagomes has joined #tripleo | 08:58 | |
*** untriaged-bot has joined #tripleo | 09:00 | |
untriaged-bot | Untriaged bugs so far: | 09:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1505815 | 09:00 |
openstack | Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,New] - Assigned to Ruchi Parmar (ruchiparmar143) | 09:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1504363 | 09:00 |
openstack | Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] | 09:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1505900 | 09:00 |
openstack | Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] | 09:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1504263 | 09:00 |
openstack | Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] | 09:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1505430 | 09:00 |
openstack | Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) | 09:00 |
*** untriaged-bot has quit IRC | 09:00 | |
devvesa | Hi. What should I do when I find a DELETE_FAILED status when removing the overcloud stack? | 09:03 |
openstackgerrit | Imre Farkas proposed openstack/instack-undercloud: Mostly revert "Disable tempest" https://review.openstack.org/234216 | 09:03 |
devvesa | shardy: ^ Can you help me? | 09:05 |
*** athomas has quit IRC | 09:05 | |
*** lucasagomes has quit IRC | 09:07 | |
*** lucasagomes_ is now known as lucasagomes | 09:08 | |
shardy | devvesa: Hi, there is occasionally a race between nova and ironic which can cause this - the best thing to do is to try to delete the stack again | 09:10 |
shardy | devvesa: does nova list show instances in ERROR state? | 09:10 |
devvesa | nova instances are in shutoff state | 09:10 |
devvesa | I'll try to start again and run stack-delete again | 09:11 |
devvesa | fails again: DELETE_FAILED status on stack-list | 09:12 |
devvesa | and nova instances are ACTIVE | 09:12 |
*** mcornea has joined #tripleo | 09:14 | |
*** athomas has joined #tripleo | 09:14 | |
*** pelix has joined #tripleo | 09:14 | |
*** paramite is now known as paramite|afk | 09:20 | |
*** links has quit IRC | 09:22 | |
*** jaosorior has joined #tripleo | 09:23 | |
shardy | devvesa: what reason is given in heat stack-show overcloud? | 09:24 |
*** dtantsur is now known as dtantsur|brb | 09:24 | |
shardy | there is a stack status_reason field which may tell us why the delete failed | 09:24 |
devvesa | http://paste.openstack.org/show/476229/ | 09:26 |
devvesa | It complains about the user: Could not find user: 080ff33bd1dc455c86c4aae723d180d1 | 09:27 |
devvesa | But this user is not in the `keystone user-list` response | 09:27 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Set shared secrets, keys and passwords as hidden https://review.openstack.org/233578 | 09:27 |
*** paramite|afk is now known as paramite | 09:29 | |
shardy | devvesa: are you running the latest upstream TripleO? | 09:31 |
devvesa | shardy: you mean the tripleo-heat-templates project? | 09:32 |
shardy | devvesa: also can you check the heat-engine log on the undercloud and see if there's an associated backtrace with that failed delete | 09:32 |
shardy | devvesa: well everything, but particularly is the heat on the undercloud up to date? | 09:33 |
devvesa | I created the instack virtualmachine where I am working on last Friday | 09:33 |
shardy | devvesa: Ok, so it should be close to lastest master, thanks | 09:33 |
devvesa | shardy: I find the same stack trace on the heat engine log about invalid user | 09:35 |
devvesa | I'm wondering from where it gets this user | 09:36 |
*** tosky has joined #tripleo | 09:37 | |
devvesa | ok. I'll create the instack machine again | 09:37 |
shardy | devvesa: sec, I'm trying to reproduce now | 09:38 |
shardy | devvesa: http://hardysteven.blogspot.co.uk/2014/04/heat-auth-model-updates-part-2-stack.html | 09:38 |
shardy | the users are created internally by heat, in the heat_stack domain | 09:38 |
shardy | so you won't see them via keystone user-list | 09:38 |
*** olap has joined #tripleo | 09:38 | |
shardy | http://fpaste.org/278997/44815459/ | 09:38 |
shardy | that shows how to list them via openstackclient | 09:39 |
shardy | if deleting one of those users causes the stack delete to fail, we have a heat regression | 09:39 |
jaosorior | shardy: I don't know if I already gave you feedback. But your blog posts have been extremely useful! | 09:41 |
shardy | jaosorior: thanks! I really need to find time to re-start them | 09:42 |
jaosorior | specially the one about debugging heat templates helped me a lot | 09:42 |
* hewbrocca notes that we're all so busy fighting off alligators, we've forgotten the mission was to drain the swamp | 09:42 | |
shardy | great, that is good to hear :) | 09:42 |
devvesa | shardy: http://fpaste.org/278999/14448157/ | 09:42 |
shardy | I think much of that info found it's way into the tripleo-docs now too | 09:43 |
shardy | hewbrocca: heh, very true ;) | 09:43 |
devvesa | shardy: Still don't see the user that causes the error here | 09:43 |
devvesa | shardy: reading you blog post, btw :) | 09:43 |
shardy | devvesa: ouch. I've just reproduced, it's a heat bug | 09:47 |
shardy | devvesa: looking into a fix | 09:47 |
devvesa | shardy: ouch | 09:47 |
devvesa | shardy: can I help you in any way? | 09:48 |
shardy | devvesa: not atm, sorry for the inconvenience, you can help test the fix when I work it out :) | 09:48 |
*** paramite is now known as paramite|afk | 09:48 | |
devvesa | shardy: sure | 09:48 |
devvesa | shardy: where this user comes from, then? | 09:49 |
shardy | devvesa: it does raise the question of why the user was deleted - can you see anything earlier in the heat logs referencing that user ID? | 09:49 |
devvesa | shardy: which log? the heat-engine one? | 09:49 |
shardy | devvesa: it's all in that blog post - each SoftwareDeployment resource creates a special user, inside the heat_stack domain | 09:49 |
devvesa | shardy: Ok. I'll keep reading :) | 09:49 |
shardy | that user is the identity used for signalling back to heat after the deployment create/update completes (or fails) | 09:50 |
shardy | devvesa: yup, the heat-engine one | 09:50 |
shardy | I recreated by manually deleting the user, but it'd be good to understand why it happened in your case, because really that user should still exist | 09:50 |
devvesa | shardy: http://fpaste.org/279014/44816375/ | 09:53 |
devvesa | In the beggining of the log, it returns data about the user, but suddenly (line 14), it seems it has been deleted | 09:53 |
jaosorior | I'm sorry if this has been asked many times before; But is there somewhere I could find documentation for the HA implementation? Tried searching in the docs but wasn't very successful | 09:53 |
shardy | devvesa: did the DELETE line happen during a previous attempt at a stack-delete? | 09:55 |
* devvesa is diving in logs | 09:56 | |
shardy | devvesa: e.g line 15 is definitely a bug, but there may be another one, e.g why did we delete the user, and not fully persist the delete of the associated resource | 09:56 |
gfidente | jaosorior, like informations on the general architecture? | 09:56 |
gfidente | jaosorior, or on the templates/manifests bits? | 09:56 |
jaosorior | gfidente: Well, I could do with the general architecture, but if you have links to the specific templates/manifests bits that would be very helpful | 09:59 |
gfidente | jaosorior, so regarding the architecture I think best source is https://github.com/beekhof/osp-ha-deploy/blob/master/ha-openstack.md ; this pretty much describes what the templates/manifests should be implementing | 09:59 |
shardy | devvesa: https://bugs.launchpad.net/heat/+bug/1505981 | 10:00 |
openstack | Launchpad bug 1505981 in heat "SoftwareDeployment delete fails if stack domain user is missing" [High,Triaged] - Assigned to Steven Hardy (shardy) | 10:00 |
shardy | you can add any further clues there :) | 10:00 |
devvesa | shardy: http://fpaste.org/279033/44481681/ | 10:00 |
devvesa | This is the first attempt to delete the overcloud. A minute before the deletion of the user | 10:01 |
devvesa | (heat-api.log) | 10:01 |
shardy | devvesa: Ok, thanks, and nothing unexpected happened during that delete, for example restarting heat-engine? | 10:02 |
devvesa | No. I was tracing constantly executing 'heat stack-list' to check out when it was deleted | 10:02 |
gfidente | but regarding the actual implementation I don't think we have much, the only hint I can give is that in addition to the regular puppet dependencies, it gets deployed in steps to have some hard-stops (which we should at some point validate) | 10:03 |
jaosorior | gfidente: So, only thing I knew is that deploying 3 controllers was the requirement. And I see there is pacemaker and HAProxy already running in the controllers. However, how the load balancing is actually working there is still pretty vague for me :/ | 10:04 |
d0ugal | instack-virt-setup is stuck in a loop for me displaying "instack never got an IP address from the libvirt default network." | 10:05 |
*** athomas has quit IRC | 10:05 | |
d0ugal | Is this a known issue? | 10:05 |
shardy | d0ugal: did the instack VM boot OK? | 10:06 |
*** akrivoka has joined #tripleo | 10:06 | |
shardy | I hit a similar issue yesterday and it failed to boot, re-running instack-virt-setup after destroying/underfining fixed it | 10:06 |
d0ugal | shardy: Thanks, I'll try that | 10:07 |
gfidente | jaosorior, the thing is that we have two archs, one uses pacemaker and the other doesnt, but only the one which uses pacemaker is currently capable of deploying multiple controllers | 10:10 |
cmyster | 10.35.7.32/w 10 | 10:10 |
cmyster | oops | 10:10 |
gfidente | the haproxy part is in common though, so it's in a shared module | 10:10 |
gfidente | jaosorior, https://github.com/openstack/puppet-tripleo/blob/master/manifests/loadbalancer.pp | 10:10 |
gfidente | the upstream nonha job is actually deploying the non-pacemaker arch | 10:11 |
jaosorior | gfidente: Thanks for the link. that's the manifest I was checking out when I did the TLS stuff. there's where I add the path so HAProxy gets the service certificate. But I don't have it very clear how the controllers and their endpoints behave with the pacemaker arch | 10:12 |
gfidente | jaosorior, oh you mean how we configure the endpoints in keystone? | 10:13 |
*** chlong has joined #tripleo | 10:13 | |
*** paramite|afk is now known as paramite | 10:13 | |
jaosorior | gfidente: Well, I would like to understand how it works in general; But ultimately what I need is to know how the endpoints will be configured and how they will behave in case of failure of a node | 10:14 |
openstackgerrit | Yanis Guenane proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet https://review.openstack.org/234704 | 10:15 |
gfidente | they point to the haproxy vips, we used to do this as a post-deployment step from a tool called os-cloud-config but this is moving into puppet now, spredzy working on it, check this series : https://review.openstack.org/#/c/230375/ | 10:16 |
gfidente | jaosorior, ^^ | 10:16 |
spredzy | jaosorior, as for now everything happens in here https://github.com/openstack/os-cloud-config/blob/master/os_cloud_config/keystone.py | 10:19 |
shardy | devvesa: http://paste.openstack.org/show/476232/ | 10:20 |
shardy | that's the fix - you can manually add those two lines to work around the issue | 10:20 |
shardy | proper heat patch in-progress | 10:20 |
devvesa | thanks! | 10:20 |
shardy | devvesa: it's a regression caused by some internal heat refactoring during liberty, unfortunately | 10:21 |
devvesa | shardy: Good to find it before the release then :) | 10:21 |
shardy | heh, yeah, although a few hours ago we confidently declared there was no need for an RC3 in the heat meeting | 10:21 |
* shardy sighs | 10:21 | |
jaosorior | gfidente, spredzy: Alright, I'll take a look. Thanks for the info :D | 10:22 |
openstackgerrit | Merged openstack/python-tripleoclient: Move common functions out into utils.py https://review.openstack.org/225244 | 10:27 |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Make root directory configurable https://review.openstack.org/230275 | 10:32 |
*** dtantsur|brb is now known as dtantsur | 10:34 | |
jistr | shardy: hi, i thought about https://review.openstack.org/#/c/224815/ -- if we didn't want another resource registry entry, i think we could generalize it from controller_pacemaker.yaml to controller_variant.yaml and have something like ControllerVariantHieraConfig emitted from controller-config-pacemaker.yaml via an output, and then have ControllerVariantHieraDeployment be in controller-post.yaml. This would keep the abstraction of the | 10:37 |
jistr | controller-config*.yaml files just providing configs without having to accept the `servers` param. However, i'm not sure how would we no-op that. Even controller-config.yaml (the non-pacemaker one), which doesn't need special hiera, would probably have to contain ControllerVariantHieraConfig resource, which would contain no hiera keys | 10:37 |
jistr | another registry entry is a possibility too i think, but the drawback there is that we'd be switching two registry entries always together in sync, which gives a hint that it should really be a single resource switch perhaps | 10:38 |
jistr | ... given that both of those resources would essentially specify controller configuration, just a different part of it. One would configure hiera, the other one would switch the puppet manifest file -- sounds like they should be a single resource. | 10:41 |
jprovazn | shardy: hi | 10:45 |
shardy | jistr: Yeah, I've been thinking about it too, and I think there may be a simpler way - since liberty heat can join multiple lists, so I think we can conditionally join the additional hieradata inside controller.yaml | 10:46 |
shardy | jistr: I'm about to go catch a train, but I'll pull your patch and work on an example which I'll share later | 10:46 |
shardy | jprovazn: hi! | 10:46 |
jprovazn | shardy: tricky heat question... | 10:47 |
jprovazn | here I define resource dependency: https://review.openstack.org/#/c/233573/1/puppet/compute.yaml | 10:47 |
jistr | shardy: ok, sounds good, thanks :) | 10:47 |
jprovazn | where RebootDeployment depends on UpdateDeployment (intention is to run reboot after update) | 10:47 |
jprovazn | shardy: but in real life, this dependency is not respected on when heat-config runs deployment scripts - it evaluates JSON (deployments key in JSON) in order it's passed to it, and this order looks like this: | 10:48 |
jprovazn | http://paste.openstack.org/show/476234/ | 10:48 |
jprovazn | reboot script is in this JSON before update - line ~634 | 10:49 |
jprovazn | shardy: am I missing some important bit about defining dependencies? | 10:49 |
gfidente | marios, it was unfair to ask about os-apply-config though! | 10:50 |
shardy | jprovazn: that doesn't sound right - the RebootDeployment should not exist in the metadata at all until UpdateDeployment is CREATE_COMPLETE | 10:50 |
gfidente | marios, what if I do extract it with list_join like you did for vni ranges but don't test it? | 10:50 |
jprovazn | shardy: this is UPDATE action | 10:50 |
shardy | jprovazn: we use that pattern everywhere, e.g all the steps of puppet deployments | 10:50 |
shardy | jprovazn: Hmm, it should work the same, if it's not then please raise a heat bug | 10:51 |
shardy | jprovazn: the tricky thing will be if UpdateDeployment doesn't need an update, then you will go directly to RebootDeployment | 10:51 |
marios | gfidente: reading | 10:51 |
marios | gfidente: well gfidente, you're welcome. seriously though, the issue has come up recently on other reviews. do we even bother trying to maintain parity with the os-apply-config templates? our ci doesn't even use them anymore | 10:52 |
*** Marga_ has joined #tripleo | 10:52 | |
shardy | jprovazn: have to go afk now, I'd suggest creating a minimal template example, then using heat resource-signal to step through each deployment | 10:54 |
shardy | then you should be able to prove/disprove if it's working as expected or if there's a bug | 10:55 |
marios | gfidente: you know, i also only just notice dyour comment at jun 16 there. so, you could also drop that part altogether i think? (i.e. the os-apply-config/)? maybe this is something to bring up at weekly too | 10:55 |
*** morazi has quit IRC | 10:55 | |
*** shardy is now known as shardy_afk | 10:55 | |
openstackgerrit | Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 | 10:55 |
gfidente | marios, indeed we know it doesn't work with oac as-is because we had CI back in june, but we don't anymore so breakage would go unnoticed and we have to decide what to do with those I think at this point | 10:56 |
hewbrocca | kill-with-fire | 10:56 |
jprovazn | shardy_afk: ok, thanks | 10:57 |
gfidente | marios, /me -1 the review and adding as topic for the weekly | 10:58 |
marios | gfidente: ok sounds good thanks | 10:59 |
gfidente | to be honest I am not sure if instack can get us an overcloud with OAC even if we wanted to | 11:00 |
devvesa | shardy: i've created a new environment using instack-virt-setup and then running the tripleo.sh scripts | 11:00 |
devvesa | it fails the deployment of the overcloud now | 11:00 |
devvesa | Looking at the heat-engine I see A LOT of ImportErrors | 11:01 |
marios | gfidente: i *think* we still use a couple of tripleo image elements if not mistaken. other than that, i have no idea. plus now with no more devtest.sh not easy to setup test env there | 11:01 |
devvesa | shardy: http://fpaste.org/279045/44482056/ <- is this normal? | 11:03 |
devvesa | But actually, I think the trace is this one: http://fpaste.org/279046/82067514/ | 11:05 |
gfidente | marios, actually I was thinking, maybe this is time to *not* paste template updates for the puppet implementation into the oac implementation anymore? | 11:05 |
gfidente | marios, so we don't break it either? | 11:05 |
gfidente | marios, yet we don't need to deprecate it as in theory it continues to work as-is ? | 11:05 |
gfidente | jistr, ^^ ? | 11:06 |
marios | gfidente: yeah exactly, this is what i meant before. just drop it. we did same for a review spredzy had recently. | 11:06 |
gfidente | oh okay so I was just late to the party, I'll update the review and mention this in the weekly anyway then | 11:07 |
gfidente | tks :) | 11:07 |
jistr | +1 on dropping it | 11:08 |
marios | devvesa: how did you deploy the overcloud? (cli command? 1/3 control? x compute. any params/env files? is this vm setup i think?) | 11:08 |
devvesa | marios: ./tripleo-common/scripts/tripleo.sh --overcloud-deploy | 11:10 |
devvesa | from the instack/undercloud machine | 11:11 |
*** aukhan has quit IRC | 11:11 | |
marios | devvesa: hm ok. those import errors are they from the controller? | 11:11 |
devvesa | I found them in the undercloud's heat-engine.log file | 11:11 |
marios | devvesa: ok, you could try logging onto the compute/controller and seeing if there is osmething interesting in there (like journalctl -fn 10000 | grep -ni error for ex) | 11:12 |
derekh | devvesa: import errors in the heat-engine log are normal | 11:12 |
marios | devvesa: ssh heat-admin@node_ip then sudo su | 11:12 |
marios | (nova list) | 11:12 |
devvesa | dereckh: Ok. Then I'll look for another cause | 11:13 |
derekh | devvesa: they just mean certain extensions can't be used, (if they are at the top of the log during startup | 11:13 |
marios | devvesa: also, fyi, heat resource-list overcloud | grep -ni fail (for ex) will show you which heat resource failed | 11:13 |
devvesa | marios: handy! I've deleted the stack (to see if it happenend again the previous error) and I am deploying again the overcloud | 11:13 |
devvesa | now with the `openstack overcloud deploy --templates` command | 11:14 |
devvesa | failed again: | 11:14 |
devvesa | [stack@instack ~]$ heat resource-list overcloud | grep -ni fail | 11:14 |
devvesa | 20:| ComputeNodesPostDeployment | 1fc35e96-7562-40a2-9f4d-1bf2661999d6 | OS::TripleO::ComputePostDeployment | CREATE_FAILED | 2015-10-14T11:09:33 | | 11:14 |
marios | devvesa: ok at that point i'd look on the compute for more info | 11:15 |
* devvesa in the compute right now | 11:15 | |
marios | just the compute postdeploy failed? | 11:15 |
devvesa | yes | 11:15 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Wire Neutron ML2 plugin and OVS agent settings as arrays https://review.openstack.org/187506 | 11:16 |
devvesa | And that's becoming interesting :) | 11:16 |
devvesa | http://fpaste.org/279048/44821403/ | 11:16 |
gfidente | marios, ^^ we need to hiera neutron::plugins::ml2::mechanism_drivers for the cisco ifs though; would need testing as well | 11:17 |
marios | devvesa: it is failing for package_manifest https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/manifests/overcloud_compute.pp#L105 | 11:20 |
marios | devvesa: sounds like the images you have don't have https://review.openstack.org/#/c/203803/ | 11:20 |
devvesa | yes | 11:20 |
devvesa | I built the image based on the overcloud full plus my midonet elements: https://review.openstack.org/#/c/232551/ | 11:22 |
devvesa | something like: | 11:22 |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 | 11:22 |
devvesa | openstack overcloud image build --type overcloud-full --builder-extra-args overcloud-network-midonet | 11:23 |
devvesa | using Pino's patch: https://review.openstack.org/#/c/229778/ | 11:23 |
marios | devvesa: sec, trying to confirm something | 11:23 |
devvesa | puppet-tripleo in my compute host does not have this patch | 11:25 |
marios | devvesa: so, i think using export DIB_INSTALLTYPE_puppet_modules=source for image build like http://docs.openstack.org/developer/tripleo-docs/basic_deployment/basic_deployment_cli.html#get-images would probably fix that (so you'll get latest vrsion of say puppet-tripleo which is where the issue in this case) | 11:25 |
devvesa | marios: that could be the reason. I understand that package_manifest is not released yet in last puppet-tripleo puppetforge module | 11:26 |
marios | devvesa: i also think you are in rights to file abug "it don't work!" since tripleo.sh doesn't do that. perhaps we can document it there... though in time, this particular issue won't exist (i.e. the deployed puppet-tripleo will eventually have that) | 11:26 |
*** stendulker has quit IRC | 11:27 | |
marios | devvesa: so not sure if bug worthy at this point | 11:27 |
*** devvesa has left #tripleo | 11:27 | |
*** devvesa has joined #tripleo | 11:27 | |
devvesa | knowing the issue, i'll build the image again | 11:27 |
marios | devvesa: but simple matter remains ' i tried triplso.sh and it didn't work because of missing dependency' | 11:27 |
devvesa | maybe tripleo.sh works | 11:30 |
devvesa | because I built the image using `openstack overcloud image build` command | 11:30 |
devvesa | to specify my element | 11:30 |
*** BharatK has quit IRC | 11:35 | |
openstackgerrit | Yanis Guenane proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet https://review.openstack.org/234704 | 11:37 |
openstackgerrit | Dmitry Tantsur proposed openstack/diskimage-builder: Fix tests/test_functions.bash https://review.openstack.org/229334 | 11:41 |
openstackgerrit | Dmitry Tantsur proposed openstack/diskimage-builder: Add a tox target to run functional tests locally https://review.openstack.org/229335 | 11:41 |
*** BharatK has joined #tripleo | 11:48 | |
*** weshay_xchat has joined #tripleo | 11:52 | |
*** weshay_xchat is now known as weshay | 11:52 | |
*** persia has quit IRC | 12:02 | |
*** persia has joined #tripleo | 12:03 | |
*** lucasagomes is now known as lucas-hungry | 12:03 | |
*** morazi has joined #tripleo | 12:05 | |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Collect the common logic in tripleo-common https://review.openstack.org/228991 | 12:05 |
openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates: Remove deleted Cinder rows https://review.openstack.org/209019 | 12:10 |
openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates: Remove deleted Nova rows https://review.openstack.org/207909 | 12:12 |
*** Prophidys has joined #tripleo | 12:13 | |
openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates: Keystone domain for Heat https://review.openstack.org/180566 | 12:13 |
*** thrash|g0ne is now known as thrash | 12:13 | |
*** Prophidys has quit IRC | 12:15 | |
*** Prophidys has joined #tripleo | 12:15 | |
*** rhallisey has joined #tripleo | 12:18 | |
*** jayg|g0n3 is now known as jayg | 12:19 | |
*** noslzzp has joined #tripleo | 12:20 | |
*** rbrady has joined #tripleo | 12:25 | |
*** spzala has joined #tripleo | 12:26 | |
openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates: Ceph cluster validation https://review.openstack.org/225248 | 12:27 |
*** dprince has joined #tripleo | 12:34 | |
*** Prophidys has quit IRC | 12:34 | |
*** athomas has joined #tripleo | 12:34 | |
*** noslzzp has quit IRC | 12:34 | |
*** noslzzp has joined #tripleo | 12:34 | |
*** trown|outttypeww is now known as trown | 12:35 | |
openstackgerrit | imain proposed openstack/os-net-config: Add ip addr flush on ifdown to ensure IP addresses are removed. https://review.openstack.org/234176 | 12:37 |
openstackgerrit | Jan Provaznik proposed openstack/tripleo-heat-templates: Add a resource for node reboot https://review.openstack.org/233573 | 12:39 |
*** shardy_a1k has joined #tripleo | 12:39 | |
*** paramite is now known as paramite|afk | 12:39 | |
dprince | gfidente: hi, so you added an agenda item for this to next weeks meeting "OAC templates breakage, better to just leave those as they are now that we don't have CI?". What breakage are you referring too? | 12:40 |
gfidente | dprince, yeah maybe it's not necessary to discuss it at the meeting and it's just me behind | 12:40 |
*** shardy_afk has quit IRC | 12:41 | |
dprince | gfidente: I'm fine to discuss it, just wanted to know more what we are discussing? | 12:41 |
gfidente | but the idea is, shall we continue to update os-apply-config when puppet files gets new parameters or a switched for example from string to another type? | 12:41 |
gfidente | *or are switched | 12:42 |
dprince | gfidente: oh, right. So I think we'll be moving os-apply-config to deprecated soon | 12:42 |
marios | gfidente: i think is worth having the discussion. for example, tht change that adds foo config item to puppet/controller and sthing in in controller/pacemaker. same change also updates the os-apply-config/controller | 12:42 |
marios | dprince: yeah that | 12:42 |
dprince | gfidente: but sure, lets bring these things up next week | 12:43 |
gfidente | one more thing is, do we know if instack can deploy with oac? | 12:43 |
dprince | gfidente: from an overcloud standpoint I don't see why not | 12:44 |
dprince | gfidente: the requirement there is unchanged (Heat w/ Ironic) | 12:44 |
gfidente | yeah should be just the images which need appropriate build scriptds | 12:44 |
Slower | dprince: did you comment on that patch? I don't see it.. | 12:45 |
*** shardy_a1k has quit IRC | 12:45 | |
dprince | Slower: the os-net-config patch? not yet... haven't had a chance to kick the tires yet | 12:45 |
Slower | dprince: I cant beleive all that f**kadoo with dhclient/nm etc resulted in such a simple patch.. | 12:45 |
*** shardy_afk has joined #tripleo | 12:46 | |
Slower | dprince: well voice your concerns damnit! :) | 12:46 |
Slower | dprince: I hear your the new PTL? | 12:47 |
dprince | Slower: sssh, tis a secret | 12:47 |
Slower | uh oh | 12:47 |
Slower | sorry! also sorry I'm canadian! | 12:48 |
hewbrocca | Slower: wait, you're Canadian? | 12:48 |
Slower | hewbrocca: yes, sorry! | 12:48 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Collect the common logic in tripleo-common https://review.openstack.org/228991 | 12:50 |
openstackgerrit | Daisuke Fujita proposed openstack/os-cloud-config: Fix typos in usage.rst https://review.openstack.org/234771 | 12:52 |
*** BharatK has quit IRC | 12:54 | |
*** bvandenh has joined #tripleo | 12:56 | |
dprince | Slower: commented on https://review.openstack.org/#/c/234176/2 | 12:58 |
jprovazn | d0ugal: thrash: hi, I'd like to get this merged https://review.openstack.org/#/c/230964/ + dependency: https://review.openstack.org/#/c/230969/, could you please take a look? | 12:59 |
Slower | dprince: cool thx | 12:59 |
thrash | jprovazn: looking | 12:59 |
jprovazn | thanks | 12:59 |
thrash | jprovazn: can you add a Depends-On to the tripleoclient patch? | 13:00 |
thrash | jprovazn: nm | 13:01 |
thrash | I missed it. :P | 13:01 |
jprovazn | thrash: d0ugal: and this one too: https://review.openstack.org/#/c/226682/ | 13:03 |
d0ugal | jprovazn: Looks like valid tests failures on https://review.openstack.org/#/c/230969/? | 13:04 |
jprovazn | d0ugal: there is that cross repo dependency | 13:04 |
*** paramite|afk is now known as paramite | 13:04 | |
jprovazn | once the first one is in, I can re-run CI | 13:04 |
d0ugal | oh, I see | 13:05 |
*** rlandy has joined #tripleo | 13:05 | |
d0ugal | I thought CI was more intelligent :) | 13:05 |
jprovazn | :) | 13:05 |
thrash | d0ugal: same | 13:05 |
*** masco has quit IRC | 13:07 | |
jprovazn | thrash: d0ugal: thanks | 13:09 |
openstackgerrit | Merged openstack/tripleo-common: Explicitly unset package update hooks when deleting a node https://review.openstack.org/230964 | 13:10 |
openstackgerrit | Merged openstack/tripleo-common: Fix wrong node count when deleting specific nodes https://review.openstack.org/226682 | 13:11 |
*** tzumainn has joined #tripleo | 13:11 | |
*** lucas-hungry is now known as lucasagomes | 13:12 | |
*** BharatK has joined #tripleo | 13:13 | |
*** Goneri has joined #tripleo | 13:14 | |
*** tiswanso has joined #tripleo | 13:25 | |
*** lblanchard has joined #tripleo | 13:26 | |
hewbrocca | shardy_afk, dprince question re heat templates | 13:26 |
hewbrocca | suppose I want to make significant changes to a role | 13:28 |
hewbrocca | add/remove a service, etc. via editing the templates themselves | 13:28 |
hewbrocca | can I do that via a stack update? | 13:28 |
hewbrocca | without redeploying the world | 13:29 |
dprince | hewbrocca: should be able to yes, puppet apply would apply those changes just fine I think | 13:29 |
hewbrocca | Right | 13:29 |
hewbrocca | that's what I was hoping | 13:29 |
hewbrocca | thanks dprince | 13:29 |
*** Prophidys has joined #tripleo | 13:31 | |
openstackgerrit | Jan Provaznik proposed openstack/python-tripleoclient: Explicitly unset package update hooks when re-deploying https://review.openstack.org/230969 | 13:46 |
*** shardy_afk is now known as shardy | 13:47 | |
shardy | hewbrocca: yup that's correct, but you'd need to be careful not to change anything like firstboot scripts, as when certain properties of a node/server change heat will replace it | 13:48 |
shardy | if it's only puppet manifests/hieradata tho it should be fine | 13:48 |
*** pradk has joined #tripleo | 13:52 | |
*** jistr is now known as jistr|mtg | 13:56 | |
openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: neutron: enable nova-event-callback by default https://review.openstack.org/234325 | 13:57 |
*** jprovazn has quit IRC | 13:58 | |
*** rasca has quit IRC | 14:00 | |
openstackgerrit | Yanis Guenane proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet https://review.openstack.org/234704 | 14:02 |
*** rasca has joined #tripleo | 14:07 | |
*** rgowrishankar has joined #tripleo | 14:18 | |
*** alop has joined #tripleo | 14:19 | |
*** marcusvrn_ has quit IRC | 14:25 | |
*** spzala has quit IRC | 14:28 | |
openstackgerrit | Merged openstack/instack-undercloud: Don't use dnsmasq lease file to retrieve address https://review.openstack.org/232584 | 14:46 |
*** thrash is now known as thrash|biab | 14:51 | |
*** shardy_ has joined #tripleo | 14:56 | |
*** shardy has quit IRC | 14:57 | |
*** untriaged-bot has joined #tripleo | 15:00 | |
untriaged-bot | Untriaged bugs so far: | 15:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1505815 | 15:00 |
openstack | Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) | 15:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1504363 | 15:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1505900 | 15:00 |
openstack | Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] | 15:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1504263 | 15:00 |
openstack | Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] | 15:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1505430 | 15:00 |
openstack | Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] | 15:00 |
*** untriaged-bot has quit IRC | 15:00 | |
openstack | Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) | 15:00 |
*** tiswanso has quit IRC | 15:01 | |
*** shardy_ has quit IRC | 15:01 | |
*** jistr|mtg is now known as jistr | 15:02 | |
*** tiswanso has joined #tripleo | 15:02 | |
*** shardy has joined #tripleo | 15:02 | |
*** jprovazn has joined #tripleo | 15:03 | |
*** jcoufal has quit IRC | 15:19 | |
openstackgerrit | Vinay Rao proposed openstack/tripleo-heat-templates: This change makes the following parameters user configurable: https://review.openstack.org/230764 | 15:20 |
*** david-lyle has quit IRC | 15:25 | |
*** david-lyle has joined #tripleo | 15:27 | |
*** mbound has quit IRC | 15:27 | |
*** yamahata has joined #tripleo | 15:30 | |
*** tiswanso has quit IRC | 15:39 | |
*** tiswanso has joined #tripleo | 15:41 | |
*** BharatK has quit IRC | 15:42 | |
*** paramite has quit IRC | 15:44 | |
*** mudassirlatif has quit IRC | 15:45 | |
*** david-ly_ has joined #tripleo | 15:46 | |
*** david-lyle has quit IRC | 15:47 | |
*** tosky has quit IRC | 15:53 | |
openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: neutron: enable nova-event-callback by default https://review.openstack.org/234325 | 15:54 |
*** david-ly_ is now known as david-lyle | 15:54 | |
*** tosky has joined #tripleo | 15:55 | |
*** jcoufal has joined #tripleo | 16:02 | |
openstackgerrit | Ben Nemec proposed openstack/python-tripleoclient: Set --min-tmpfs for overcloud-full build https://review.openstack.org/234860 | 16:03 |
devvesa | marios: shardy: available? | 16:04 |
*** jistr has quit IRC | 16:06 | |
*** dtantsur is now known as dtantsur|afk | 16:06 | |
*** thrash|biab is now known as thrash | 16:07 | |
*** yog_ has quit IRC | 16:09 | |
*** testing has joined #tripleo | 16:11 | |
*** testing is now known as Guest81146 | 16:11 | |
marios | devvesa: barely wasup | 16:11 |
devvesa | I still have problems deploying the overcloud... and these ones even weirder | 16:12 |
devvesa | The ./tripleo.sh scripts create the two 'barmetal vms' and I can see them on `nova list` | 16:13 |
devvesa | the controller has the ip : 192.0.2.20 | 16:13 |
devvesa | When I log into it and I try to follow the puppet executions | 16:13 |
devvesa | I see the following when execution db_syncs: | 16:14 |
devvesa | 2015-10-14 16:03:38.528 4384 TRACE cinder DBConnectionError: (OperationalError) (2003, "Can't connect to MySQL server on '192.0.2.18' (113)") None None | 16:14 |
devvesa | No node belongs the address 192.0.2.18 | 16:14 |
marios | devvesa: this should be ok is the vip... eventually the dbconnection should succees | 16:15 |
marios | devvesa: do you see any other errors like journalctl -fn 10000 | grep -ni error | 16:16 |
openstackgerrit | Ben Nemec proposed openstack/python-tripleoclient: Set --min-tmpfs for overcloud-full build https://review.openstack.org/234860 | 16:16 |
*** penick has joined #tripleo | 16:16 | |
devvesa | marios: only the dbconnections one | 16:16 |
marios | devvesa: what about the compute? | 16:17 |
devvesa | I have to say that the cinder database (for instance) has been created but the tables are not there... | 16:17 |
marios | devvesa: yeah for example at this point probably 'mysql' works fine | 16:18 |
devvesa | lots of errors on compute | 16:19 |
devvesa | ERROR oslo_messaging._drivers.impl_rabbit [req-736d10dc-2861-436d-a35d-b97a308eb12a - - - - -] AMQP server on 192.0.2.20:5672 is unreachable | 16:19 |
marios | devvesa: id say grab those errors to fpaste for now in case you need to file a bug, i have to run momentarily | 16:19 |
marios | devvesa: what about heat, which resources fail? | 16:19 |
devvesa | they are stuck | 16:20 |
marios | devvesa: yeah grab like heat resource-list overcloud at the very least | 16:20 |
*** kukacz has joined #tripleo | 16:20 | |
devvesa | http://fpaste.org/279161/48396491/ | 16:20 |
openstackgerrit | Ben Nemec proposed openstack/python-tripleoclient: Don't download fedora-user as part of --all build https://review.openstack.org/234870 | 16:20 |
marios | devvesa: oh so it didn't fail yet? | 16:21 |
devvesa | I have to leave as well. Keep talking tomorrow if you don't mind | 16:21 |
marios | devvesa: sure thing | 16:21 |
*** shardy has quit IRC | 16:25 | |
*** mudassirlatif has joined #tripleo | 16:27 | |
*** devvesa has quit IRC | 16:32 | |
*** Guest81146 has quit IRC | 16:37 | |
*** testing_ has joined #tripleo | 16:37 | |
trown | has anyone ever seen "/usr/bin/instack-virt-setup: line 175: ip: command not found" when running instack-virt-setup even though iproute is definitely installed and the ip command works? | 16:38 |
slagle | trown: i haven't, is it $PATH related? | 16:39 |
openstackgerrit | Merged openstack/diskimage-builder: Add Fedora 22 support to yum-minimal https://review.openstack.org/227127 | 16:40 |
trown | slagle: that is my only thought as well, this just started happening this afternoon | 16:40 |
slagle | does sudo ip work? | 16:40 |
trown | slagle: yep | 16:41 |
trown | slagle: it must be khaleesi related, I will take it up with those folks | 16:42 |
slagle | oh you didnt say khaleesi | 16:43 |
slagle | probably an ansible thing munging $PATH | 16:43 |
openstackgerrit | Ben Nemec proposed openstack/python-tripleoclient: Don't parse redhat-release for distro info https://review.openstack.org/234885 | 16:43 |
trown | does anyone know what could be messing with $PATH in the rdo-manager playbooks? I just started getting "/usr/bin/instack-virt-setup: line 175: ip: command not found" when doing instack-virt-setup | 16:47 |
trown | but if I log on to the virthost and run it, it works fine? | 16:47 |
trown | that last bit is not actually a question...it does work fine | 16:47 |
bnemec | trown: ip is in /usr/sbin. It might not be in the ansible user's path? | 16:47 |
trown | arg, wrong channel ... so much fail | 16:48 |
trown | thanks bnemec | 16:48 |
bnemec | trown: In general I try to stay out of khaleesi discussions though. That's a whole can of worms, and my plate is already full of worms. /mixed metaphors ;-) | 16:49 |
trown | bnemec: fair, luckily worms are full of protein | 16:50 |
*** sbadia has quit IRC | 16:50 | |
*** pino|work has quit IRC | 16:50 | |
*** stevebaker has quit IRC | 16:50 | |
*** SlickNik has quit IRC | 16:50 | |
*** edmondk has quit IRC | 16:50 | |
*** tiswanso has quit IRC | 16:50 | |
*** cmyster has quit IRC | 16:50 | |
*** lifeless has quit IRC | 16:50 | |
*** adrianopetrich has quit IRC | 16:50 | |
*** isq_ has quit IRC | 16:50 | |
*** shardy has joined #tripleo | 16:50 | |
trown | but you can only eat so many never the less | 16:50 |
bnemec | trown: I almost typed mixed meatphors. Yummy, yummy worm meat. :-D | 16:50 |
*** tiswanso has joined #tripleo | 17:00 | |
*** cmyster has joined #tripleo | 17:00 | |
*** lifeless has joined #tripleo | 17:00 | |
*** adrianopetrich has joined #tripleo | 17:00 | |
*** isq_ has joined #tripleo | 17:00 | |
*** aufi has quit IRC | 17:00 | |
*** tiswanso has quit IRC | 17:07 | |
*** cmyster has quit IRC | 17:07 | |
*** lifeless has quit IRC | 17:07 | |
*** adrianopetrich has quit IRC | 17:07 | |
*** isq_ has quit IRC | 17:07 | |
openstackgerrit | Ben Nemec proposed openstack/python-tripleoclient: Don't parse redhat-release for distro info https://review.openstack.org/234885 | 17:07 |
*** stevebaker has joined #tripleo | 17:07 | |
*** SlickNik has joined #tripleo | 17:07 | |
*** pino|work has joined #tripleo | 17:07 | |
*** edmondk has joined #tripleo | 17:08 | |
*** derekh has quit IRC | 17:08 | |
*** jaosorior has quit IRC | 17:09 | |
*** sbadia has joined #tripleo | 17:10 | |
*** tiswanso has joined #tripleo | 17:10 | |
*** cmyster has joined #tripleo | 17:10 | |
*** lifeless has joined #tripleo | 17:10 | |
*** adrianopetrich has joined #tripleo | 17:10 | |
*** isq_ has joined #tripleo | 17:10 | |
*** yamahata has quit IRC | 17:14 | |
*** shardy has quit IRC | 17:14 | |
*** lucasagomes is now known as lucas-dinner | 17:14 | |
*** jcoufal has quit IRC | 17:15 | |
*** athomas has quit IRC | 17:16 | |
openstackgerrit | James Slagle proposed openstack/python-tripleoclient: Install device-mapper-multipath on overcloud-full https://review.openstack.org/231690 | 17:29 |
*** lazy_prince has quit IRC | 17:30 | |
openstackgerrit | Merged openstack/os-cloud-config: Add SSL support for Ironic registration https://review.openstack.org/195333 | 17:32 |
*** trown is now known as trown|lunch | 17:32 | |
bnemec | Does anybody know if the missing python3.4 interpreter problem has been reported to infra? | 17:33 |
bnemec | I keep seeing failures like this: http://logs.openstack.org/85/234885/2/check/gate-python-tripleoclient-python34/b24c4af/console.html | 17:33 |
*** Marga_ has quit IRC | 17:38 | |
*** Marga_ has joined #tripleo | 17:38 | |
*** yamahata has joined #tripleo | 17:38 | |
*** testing_ has quit IRC | 17:39 | |
*** testing has joined #tripleo | 17:39 | |
*** testing is now known as Guest39846 | 17:39 | |
*** Guest39846 has quit IRC | 17:40 | |
*** rpothier has joined #tripleo | 17:41 | |
*** mkovacik has quit IRC | 17:43 | |
*** Marga_ has quit IRC | 17:44 | |
*** pelix has quit IRC | 17:45 | |
*** bvandenh has quit IRC | 17:46 | |
*** akrivoka has quit IRC | 17:52 | |
*** Marga_ has joined #tripleo | 18:00 | |
*** tosky has quit IRC | 18:01 | |
*** sthillma has joined #tripleo | 18:02 | |
openstackgerrit | Ben Nemec proposed openstack/instack-undercloud: Add check for sufficient memory to undercloud install https://review.openstack.org/224852 | 18:02 |
openstackgerrit | Ryan Hallisey proposed openstack/tripleo-heat-templates: Update docker compute environment to use json config https://review.openstack.org/234313 | 18:03 |
*** sthillma_ has joined #tripleo | 18:03 | |
*** sthillma has quit IRC | 18:06 | |
*** sthillma_ is now known as sthillma | 18:06 | |
*** dsneddon has quit IRC | 18:08 | |
*** dsneddon has joined #tripleo | 18:08 | |
*** sthillma has quit IRC | 18:16 | |
*** ukalifon1 has quit IRC | 18:17 | |
*** ifarkas has quit IRC | 18:18 | |
*** ukalifon has joined #tripleo | 18:19 | |
openstackgerrit | Merged openstack/tripleo-docs: Add note that qemu libvirt-type is needed for virt environment https://review.openstack.org/233535 | 18:22 |
*** ukalifon has quit IRC | 18:27 | |
*** trown|lunch is now known as trown | 18:28 | |
*** mudassirlatif_ has joined #tripleo | 18:30 | |
*** mudassirlatif has quit IRC | 18:31 | |
*** mudassirlatif_ is now known as mudassirlatif | 18:31 | |
*** mudassirlatif has quit IRC | 18:32 | |
*** olap has quit IRC | 18:32 | |
*** sthillma has joined #tripleo | 18:33 | |
kukacz | hi, I'm fighting with error "DRAC driver failed to create config job for node..." on one of our Dell servers during overcloud deploy | 18:34 |
*** mudassirlatif has joined #tripleo | 18:34 | |
kukacz | I've tried to restart conductor service, which didn't help | 18:34 |
kukacz | neither deleting the stack and attempting to recreate found the error on same server | 18:35 |
*** david-lyle has quit IRC | 18:35 | |
kukacz | any hints where to look for the cause? | 18:35 |
trown | kukacz: sounds like an issue on the DRAC itself, I have seen that where a config job gets stuck there and has to be manually cleared using iDRAC | 18:37 |
kukacz | trown: I did an iDRAC reset which did not help | 18:38 |
kukacz | is there anything else to make the job cleared? | 18:38 |
kukacz | trown: or what was the procedure you've applied? | 18:39 |
*** david-lyle has joined #tripleo | 18:39 | |
trown | kukacz: I have not actually done the clearing myself, so I am not sure exactly what that looks like, but the colleague I was working with went into the iDRAC console and cleared the jobs | 18:41 |
*** david-ly_ has joined #tripleo | 18:41 | |
kukacz | trown: thanks a lot! I'll look for Dell's docs related | 18:41 |
trown | np. kukacz what gen are the servers do you know? | 18:42 |
kukacz | they are R630s, not sure what gen it is | 18:42 |
kukacz | iDRAC version is 8 | 18:43 |
*** david-ly_ has quit IRC | 18:43 | |
*** david-ly_ has joined #tripleo | 18:44 | |
*** david-lyle has quit IRC | 18:44 | |
*** lucas-dinner has quit IRC | 18:44 | |
*** david-ly_ is now known as david-lyle | 18:45 | |
trown | kukacz: hmm, I think those are gen13, which I dont think we have seen that issue on | 18:45 |
*** david-lyle has quit IRC | 18:45 | |
kukacz | I've just cleared the job queue and am updating the stack again | 18:46 |
*** david-lyle has joined #tripleo | 18:47 | |
*** trown is now known as trown|mtg | 18:48 | |
*** bvandenh has joined #tripleo | 18:49 | |
*** jprovazn_ has joined #tripleo | 18:49 | |
*** penick has quit IRC | 18:58 | |
*** spzala has joined #tripleo | 18:59 | |
*** david-lyle has quit IRC | 19:00 | |
*** spzala has left #tripleo | 19:03 | |
openstackgerrit | Merged openstack/diskimage-builder: Update rhel7 element readme https://review.openstack.org/222351 | 19:03 |
*** david-lyle has joined #tripleo | 19:08 | |
slagle | EmilienM: can I use $::fqdn in a hiera file? | 19:10 |
EmilienM | I'm not sure, last time I tried it did not work. Let me check again | 19:11 |
slagle | i'm trying to specify a value for horizon::allowed_hosts | 19:11 |
slagle | horizon::allowed_hosts: - $::fqdn | 19:11 |
slagle | err | 19:11 |
slagle | horizon::allowed_hosts: | 19:11 |
EmilienM | yeah, I tried that last time, I had an issue | 19:11 |
EmilienM | let me check | 19:11 |
slagle | - $::fqdn | 19:11 |
slagle | - '192.1.1.1' | 19:11 |
slagle | something like that | 19:11 |
EmilienM | it should be: horizon::allowed_hots: "%{::fqdn}" | 19:12 |
EmilienM | for example | 19:12 |
slagle | ok, and i can make that value a list? | 19:12 |
EmilienM | yes | 19:12 |
slagle | thx | 19:12 |
bnemec | Where do I go to add a new dependency to the instack-undercloud package now? | 19:13 |
slagle | delorean | 19:14 |
slagle | or... | 19:14 |
slagle | https://review.gerrithub.io/#/q/project:openstack-packages/instack-undercloud | 19:15 |
slagle | is that what you meant? | 19:15 |
bnemec | Ah, openstack-packages. I kept looking for the wrong name. :-) | 19:15 |
bnemec | slagle: Thanks, I think that's what I'm looking for. | 19:16 |
kukacz | trown|mtg: thanks, your hint helped! clearing the jobqueue resolved my issue | 19:18 |
openstackgerrit | James Slagle proposed openstack/instack-undercloud: Add ip addresses to horizon allowed_hosts https://review.openstack.org/234939 | 19:21 |
*** bvandenh has quit IRC | 19:22 | |
*** penick has joined #tripleo | 19:25 | |
*** david-lyle has quit IRC | 19:25 | |
*** david-ly_ has joined #tripleo | 19:25 | |
*** Marga_ has quit IRC | 19:33 | |
*** david-ly_ has quit IRC | 19:38 | |
*** david-lyle has joined #tripleo | 19:38 | |
*** david-lyle has quit IRC | 19:39 | |
*** mcornea has quit IRC | 19:40 | |
*** david-lyle has joined #tripleo | 19:41 | |
*** trown|mtg is now known as trown | 19:43 | |
*** dprince has quit IRC | 19:51 | |
*** david-lyle has quit IRC | 19:56 | |
*** david-lyle has joined #tripleo | 19:58 | |
*** david-ly_ has joined #tripleo | 20:01 | |
*** bvandenh has joined #tripleo | 20:03 | |
*** david-lyle has quit IRC | 20:03 | |
*** david-ly_ is now known as david-lyle__ | 20:03 | |
*** david-lyle__ is now known as david-lyle___ | 20:03 | |
*** Marga_ has joined #tripleo | 20:03 | |
openstackgerrit | Jeff Peeler proposed openstack/tripleo-heat-templates: Update docker compute environment to use json config https://review.openstack.org/234313 | 20:05 |
openstackgerrit | Ben Nemec proposed openstack/instack-undercloud: Add check for sufficient memory to undercloud install https://review.openstack.org/224852 | 20:05 |
*** david-lyle has joined #tripleo | 20:06 | |
*** david-lyle___ has quit IRC | 20:06 | |
*** Marga_ has quit IRC | 20:09 | |
*** marcusvrn_ has joined #tripleo | 20:10 | |
*** gfidente has quit IRC | 20:17 | |
*** jayg is now known as jayg|g0n3 | 20:21 | |
openstackgerrit | Rohit Pagedar proposed openstack/tripleo-heat-templates: Added nova parameteres to controller: instance_name_template, use_forwarded_for After this change, the above parameters will be configurable via heat templates. https://review.openstack.org/234965 | 20:21 |
openstackgerrit | Rohit Pagedar proposed openstack/tripleo-heat-templates: Changes for configuring Nuage Added ExtraConfig templates and environment files for Nuage specific parameters. Modified overcloud_compute.pp to conditionally include nuage-metadata-agent. https://review.openstack.org/234970 | 20:27 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Remove unused RELEASE_RPMS variable https://review.openstack.org/234971 | 20:29 |
*** lblanchard has quit IRC | 20:42 | |
*** jprovazn_ has quit IRC | 20:50 | |
*** Marga_ has joined #tripleo | 20:51 | |
*** Marga_ has quit IRC | 20:55 | |
*** rhallisey has quit IRC | 20:59 | |
*** untriaged-bot has joined #tripleo | 21:00 | |
untriaged-bot | Untriaged bugs so far: | 21:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1505815 | 21:00 |
openstack | Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) | 21:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1504363 | 21:00 |
openstack | Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] | 21:00 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1505900 | 21:00 |
openstack | Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] | 21:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1504263 | 21:00 |
openstack | Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] | 21:00 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1505430 | 21:00 |
openstack | Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) | 21:00 |
*** untriaged-bot has quit IRC | 21:00 | |
*** trown is now known as trown|outttypeww | 21:08 | |
*** kukacz has quit IRC | 21:09 | |
*** mudassirlatif has quit IRC | 21:15 | |
*** mudassirlatif has joined #tripleo | 21:16 | |
*** Goneri has quit IRC | 21:21 | |
*** rgowrishankar has quit IRC | 21:23 | |
*** tiswanso has quit IRC | 21:24 | |
*** mudassirlatif has quit IRC | 21:26 | |
*** penick has quit IRC | 21:31 | |
*** penick has joined #tripleo | 21:31 | |
*** rpothier has quit IRC | 21:31 | |
*** penick has quit IRC | 21:32 | |
*** jtomasek has quit IRC | 21:33 | |
*** penick has joined #tripleo | 21:34 | |
*** Marga_ has joined #tripleo | 21:36 | |
*** rasca has quit IRC | 21:38 | |
*** mudassirlatif has joined #tripleo | 21:40 | |
*** penick has quit IRC | 21:43 | |
*** mudassirlatif has quit IRC | 21:45 | |
*** mudassirlatif has joined #tripleo | 21:46 | |
*** thrash is now known as thrash|g0ne | 21:48 | |
openstackgerrit | Rohit Pagedar proposed openstack/tripleo-heat-templates: Changes for configuring Nuage Added ExtraConfig templates and environment files for Nuage Networks specific parameters. Modified overcloud_compute.pp to conditionally include nuage-metadata-agent. https://review.openstack.org/234970 | 21:53 |
openstackgerrit | Rohit Pagedar proposed openstack/tripleo-heat-templates: Adding nova parameteres to controller: instance_name_template use_forwarded_for https://review.openstack.org/234965 | 21:57 |
*** mudassirlatif has quit IRC | 22:08 | |
*** pradk has quit IRC | 22:41 | |
*** bvandenh has quit IRC | 22:44 | |
*** shivrao has joined #tripleo | 22:45 | |
*** openstackgerrit has quit IRC | 22:46 | |
*** openstackgerrit has joined #tripleo | 22:46 | |
*** mudassirlatif has joined #tripleo | 22:52 | |
*** jobewan has joined #tripleo | 23:00 | |
*** rlandy has quit IRC | 23:04 | |
*** morazi has quit IRC | 23:05 | |
*** rlandy has joined #tripleo | 23:07 | |
*** weshay has quit IRC | 23:09 | |
*** rhallisey has joined #tripleo | 23:53 | |
*** penick has joined #tripleo | 23:56 | |
*** jobewan has quit IRC | 23:58 | |
*** alop has quit IRC | 23:59 | |
*** rhallisey has quit IRC | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!