*** r-mibu has quit IRC | 00:10 | |
*** rwsu has quit IRC | 00:14 | |
EmilienM | sshnaidm: I think we can land https://review.openstack.org/#/c/349995/ | 00:15 |
---|---|---|
*** fultonj has quit IRC | 00:28 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Don't set selinux in 'permissive' mode on CephStorage nodes https://review.openstack.org/347735 | 00:34 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Fixes type and description for NeutronL3HA https://review.openstack.org/349493 | 00:34 |
*** r-mibu has joined #tripleo | 00:35 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Fixes nova host configuration on control nodes https://review.openstack.org/344969 | 00:37 |
*** limao has joined #tripleo | 00:41 | |
*** limao has quit IRC | 00:41 | |
openstackgerrit | Giulio Fidente proposed openstack/puppet-tripleo: Allow use of raw disks for Swift storage role https://review.openstack.org/350840 | 00:42 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Permit usage of raw disks as backends for Swift storage https://review.openstack.org/350841 | 00:43 |
*** bana_k has quit IRC | 01:28 | |
*** dmacpher has joined #tripleo | 01:40 | |
*** weshay has quit IRC | 01:40 | |
*** jeckersb_gone is now known as jeckersb | 01:42 | |
*** hewbrocca-afk has quit IRC | 01:56 | |
*** hewbrocca-afk has joined #tripleo | 01:56 | |
*** jmiu has quit IRC | 01:57 | |
*** jmiu_ is now known as jmiu | 01:57 | |
*** jmiu_ has joined #tripleo | 01:58 | |
*** bfournie has joined #tripleo | 02:16 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/350676 | 02:40 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/os-collect-config: Updated from global requirements https://review.openstack.org/350905 | 02:40 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/348695 | 02:43 |
*** ebalduf has joined #tripleo | 02:51 | |
*** yamahata has quit IRC | 03:03 | |
*** tzumainn has quit IRC | 03:22 | |
*** coolsvap has joined #tripleo | 03:22 | |
*** jrist has joined #tripleo | 03:34 | |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot https://review.openstack.org/350489 | 04:03 |
*** ayoung has quit IRC | 04:08 | |
*** julim has quit IRC | 04:16 | |
*** links has joined #tripleo | 04:28 | |
*** bana_k has joined #tripleo | 04:36 | |
*** oshvartz has quit IRC | 04:40 | |
*** cmorandin_ has joined #tripleo | 04:53 | |
*** leanderthal|afk is now known as leanderthal | 04:54 | |
*** cmorandin_ has quit IRC | 04:58 | |
*** jraju has joined #tripleo | 05:04 | |
*** links has quit IRC | 05:06 | |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot https://review.openstack.org/350489 | 05:07 |
*** rcernin has joined #tripleo | 05:16 | |
openstackgerrit | Merged openstack/diskimage-builder: Optionally remove portage files https://review.openstack.org/348536 | 05:20 |
*** TSCHAK has joined #tripleo | 05:41 | |
*** bana_k has quit IRC | 05:49 | |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable SSL for ironic inspector https://review.openstack.org/332641 | 05:49 |
*** dmacpher has quit IRC | 05:49 | |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Refactor endpoint env variable population https://review.openstack.org/350396 | 05:49 |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/diskimage-builder: Nothing to see here https://review.openstack.org/345539 | 05:50 |
*** apetrich has quit IRC | 05:55 | |
*** apetrich has joined #tripleo | 06:00 | |
*** jtomasek has quit IRC | 06:09 | |
*** pcaruana has joined #tripleo | 06:10 | |
*** devvesa has joined #tripleo | 06:12 | |
bandini | morning | 06:13 |
*** tremble has joined #tripleo | 06:15 | |
*** tremble has joined #tripleo | 06:15 | |
openstackgerrit | Merged openstack/tripleo-docs: Document additional NODE vars https://review.openstack.org/350708 | 06:16 |
*** jaosorior has joined #tripleo | 06:20 | |
*** jbadiapa has joined #tripleo | 06:21 | |
*** oshvartz has joined #tripleo | 06:22 | |
*** pgadiya has joined #tripleo | 06:32 | |
*** limao has joined #tripleo | 06:41 | |
*** mcornea has joined #tripleo | 06:43 | |
*** radek_ has joined #tripleo | 06:46 | |
openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Next generation HA architecture work https://review.openstack.org/314208 | 06:46 |
*** bootsha has joined #tripleo | 06:52 | |
*** tesseract- has joined #tripleo | 06:53 | |
jaosorior | flaper87: Are you around? | 06:58 |
*** ebalduf has quit IRC | 07:06 | |
*** ohamada has joined #tripleo | 07:09 | |
gfidente | bandini, :) | 07:10 |
*** florianf has joined #tripleo | 07:11 | |
*** liverpooler has joined #tripleo | 07:11 | |
*** dculik has joined #tripleo | 07:12 | |
*** jpena|off is now known as jpena | 07:13 | |
*** ccamacho has quit IRC | 07:16 | |
*** yamahata has joined #tripleo | 07:18 | |
*** ccamacho has joined #tripleo | 07:18 | |
bandini | gfidente: wasn't me! | 07:22 |
* bandini claims complete innocence | 07:22 | |
*** abehl has joined #tripleo | 07:30 | |
*** openstackgerrit has quit IRC | 07:33 | |
*** openstackgerrit has joined #tripleo | 07:33 | |
ccamacho | morning! | 07:42 |
*** ebarrera has joined #tripleo | 07:44 | |
*** zoli_gone-proxy is now known as zoliXXL | 07:44 | |
*** radek_ has quit IRC | 07:45 | |
*** zoliXXL is now known as zoli|wfh | 07:47 | |
*** shardy has joined #tripleo | 07:48 | |
jaosorior | ccamacho: Morning, dude | 07:51 |
*** oshvartz has quit IRC | 07:51 | |
*** fzdarsky has joined #tripleo | 07:52 | |
*** milan has joined #tripleo | 07:57 | |
*** jpich has joined #tripleo | 08:00 | |
*** hewbrocca-afk is now known as hewbrocca | 08:05 | |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Add zaqar endpoint in HAProxy https://review.openstack.org/351052 | 08:09 |
jaosorior | shardy: Have you tried to build puppet-tripleo with tripleo.sh lately? | 08:12 |
*** jaosorior has quit IRC | 08:13 | |
shardy | jaosorior: No, I just use upload-puppet-modules | 08:13 |
*** jaosorior has joined #tripleo | 08:13 | |
jaosorior | ccamacho: Which map? | 08:14 |
jaosorior | ccamacho: regarding https://review.openstack.org/323436 | 08:14 |
ccamacho | jaosorior the endpoint_map **me messing it in the rebase, Im finishing to fix it | 08:14 |
*** Goneri has joined #tripleo | 08:15 | |
*** dtantsur|afk is now known as dtantsur | 08:16 | |
shardy | jaosorior: Yeah it doesn't work | 08:16 |
jaosorior | :( | 08:16 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable Mistral services https://review.openstack.org/323436 | 08:17 |
jaosorior | shardy: How are you testing puppet-* changes for the undercloud? | 08:17 |
shardy | jaosorior: just do a git clone | 08:18 |
d0ugal | shardy: btw, with temporary hard-coded keystone certs I get a CREATE_COMPLETE! | 08:18 |
shardy | then run puppet manually | 08:18 |
* d0ugal hasn't seen one of those for a while | 08:18 | |
shardy | obviously we do need to fix the packaging too tho | 08:18 |
shardy | d0ugal: woohoo! | 08:18 |
sshnaidm | does anybody deal with new introspection failures? | 08:18 |
shardy | next step, environments, then endpoints :) | 08:18 |
sshnaidm | I mean in CI jobs | 08:18 |
shadower | obviously we need to introduce new bugs :-) | 08:18 |
d0ugal | shadower: I think that is all I do | 08:19 |
*** bootsha has quit IRC | 08:19 | |
jaosorior | shardy: So the plan now is to do keystone endpoints via mistral? | 08:19 |
jaosorior | sshnaidm: Which? | 08:19 |
shardy | jaosorior: the plan is just to maintain the current os-cloud-config code and move it from tripleoclient to mistral | 08:19 |
shardy | jaosorior: unless someone tells me we can actually land the puppet endpoint stuff | 08:19 |
sshnaidm | jaosorior, http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/7566671/console.html#_2016-08-04_07_23_40_211292 | 08:19 |
openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 08:20 |
jaosorior | shardy: alright | 08:20 |
jaosorior | sshnaidm: Which patch? | 08:20 |
shardy | jaosorior: that seems like the most low risk increment, everything the same as now, but via mistral | 08:20 |
sshnaidm | jaosorior, it's in periodic job, but also in patches | 08:20 |
shardy | we can work out further changes after that (the mistral endpoint stuff will be trivial to disable) | 08:20 |
d0ugal | shardy: I am not sure how we are going to do it in Mistral :) | 08:21 |
d0ugal | shardy: The deploy workflow ends when the Heat stack create/update starts | 08:21 |
shardy | d0ugal: well we'll have to change the workflow ;) | 08:21 |
jaosorior | sshnaidm: that's funky... Have you been able to reproduce that? | 08:21 |
jaosorior | It seems to be a mistral issue | 08:21 |
shardy | so it polls for stack CREATE_COMPLETE then runs another action | 08:21 |
*** yamahata has quit IRC | 08:22 | |
d0ugal | shardy: we will hit a mistral bug with token timeouts - they don't renew them like Heat does. So deploys over 4 hours (default) will fail | 08:22 |
jaosorior | I ran into the same thing when trying to enable TLS for ironic-inspector. Since introspection is now driven by mistral | 08:22 |
shardy | e.g the same workflow we have hardcoded in tripleoclient? | 08:22 |
sshnaidm | jaosorior, not yet, just wanted to know if anybody look at it | 08:22 |
d0ugal | shardy: https://bugs.launchpad.net/mistral/+bug/1595084 | 08:22 |
openstack | Launchpad bug 1595084 in Mistral "Workflow execution lifespan is limited by auth token expire time" [High,Confirmed] | 08:22 |
d0ugal | shardy: Do we might need a post-deploy workflow. | 08:22 |
d0ugal | s/Do/So/ | 08:22 |
shardy | d0ugal: Ok, well we'll have to fix it (I fixed it in heat, so maybe I can help) | 08:22 |
shardy | but yeah, a workaround will be enable running just the post-deploy steps | 08:23 |
d0ugal | shardy: I thought you might have, I was going to ask you about it when things were quieter :) | 08:23 |
*** dtantsur is now known as dtantsur|bbl | 08:23 | |
shardy | so if the deploy command times out, you can still run the post-deploy | 08:23 |
d0ugal | shardy: I think the big issue is that Mistral is stuck of keystone v2 for now. | 08:23 |
jaosorior | sshnaidm: Do you have access to any of the CI environments? We could debug it | 08:23 |
shardy | d0ugal: Yeah, I'll admit it was me ;) It involves some exciting abuse of trusts | 08:23 |
d0ugal | lol | 08:23 |
sshnaidm | jaosorior, yeah, I think it would be reproducible in any nonha job, will run one now | 08:24 |
*** lucas-dinner is now known as lucasagomes | 08:24 | |
jaosorior | sshnaidm: Sure. only thing we need to check is the output of mistral-db-sync. That usually gives the culprit | 08:24 |
d0ugal | shardy: a post-deploy workflow sounds like a reasonable approach for now. if we can resolve the auth issue the deploy workflow could trigger it. | 08:24 |
sshnaidm | these error reports of mistral are completely unreadable.. | 08:24 |
shardy | d0ugal: Ok, I thought they were already using trusts for some reason | 08:24 |
d0ugal | sshnaidm: The bugs? | 08:24 |
jaosorior | sshnaidm: For some reason it's not in the logs that CI persists though | 08:24 |
shardy | d0ugal: ack | 08:24 |
d0ugal | oh, the actual errors. nvm | 08:25 |
sshnaidm | jaosorior, really? which log is this? | 08:25 |
shardy | d0ugal: also, we can revive the discussion about having puppet do the post-deploy stuff | 08:25 |
jaosorior | I have no clue where mistral would put that. I just know that it's not in any log I've found. So I ended up running that command manually | 08:25 |
shardy | that's only been going on for a year... ;) | 08:25 |
d0ugal | shardy: +1, that would be even better | 08:25 |
hewbrocca | Morning folks | 08:25 |
d0ugal | Morning | 08:25 |
ccamacho | o/ | 08:25 |
shardy | 'morning hewbrocca | 08:25 |
jaosorior | shardy, d0ugal: Any plans on moving mistral not to be extremely hardcoded for keystone v2? | 08:25 |
hewbrocca | dear god could we for the love of all that's good and holy, please, make puppet do the post-deploy stuff | 08:26 |
hewbrocca | I hate os-cloud-config so much | 08:26 |
d0ugal | jaosorior: There is some discussion in https://bugs.launchpad.net/mistral/+bug/1595084 | 08:26 |
openstack | Launchpad bug 1595084 in Mistral "Workflow execution lifespan is limited by auth token expire time" [High,Confirmed] | 08:26 |
shardy | lol | 08:26 |
jaosorior | d0ugal: It's gonna be a LOT of work X_x. I checked that they even do manual keystone catalog checking of stuff (cause why use the keystone libraries?) | 08:26 |
jaosorior | and a bunch of other details that just make life harder | 08:26 |
jaosorior | :( | 08:26 |
d0ugal | jaosorior: haha, oh dear. Maybe it is something I can look into once we have the deploy stuff working. | 08:27 |
d0ugal | Does Keystone v2 have an EOL? | 08:27 |
shardy | Using trusts really doesn't require mistral to be completely purged of v2.0 stuff | 08:27 |
jaosorior | d0ugal: As far as I know it has passed already, that EOL | 08:27 |
shardy | it just requires the v3 API to be actually accessible | 08:27 |
d0ugal | jaosorior: o.O | 08:28 |
shardy | for years (like, since 2013) Heat used mostly v2.0 with v3 internally for trusts | 08:28 |
jaosorior | d0ugal: it has been a LONG time the keystone team has been trying to get people to stop using v2 | 08:28 |
d0ugal | Okay, cool. I think I shall come back to this one. | 08:28 |
* d0ugal makes a note. | 08:28 | |
shardy | http://developer.openstack.org/api-ref/identity/v2/ | 08:28 |
shardy | SUPPORTED | 08:28 |
shardy | I don't think it's even officially deprecated yet | 08:29 |
jaosorior | shardy haha yeah, cause when they tried to deprecated, people complaigned that they hadn't been able to move out of it yet | 08:29 |
shardy | but I am aware of all the efforts trying to move away from it | 08:29 |
jaosorior | *deprecate it | 08:29 |
jaosorior | there was a pretty big backlash as far as i remember | 08:29 |
d0ugal | Maybe I can make this a goal for O in Mistral. | 08:29 |
shardy | d0ugal: sounds good - I'll also try to take a look, perhaps we can reuse some of the heat code | 08:30 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-docs: Add TripleO UI to the components docs https://review.openstack.org/340703 | 08:34 |
*** chem has joined #tripleo | 08:35 | |
*** cwolferh has quit IRC | 08:37 | |
shardy | d0ugal: I just commented, I may take a look later and see how hard it would be to fix | 08:42 |
shardy | life would certainly be easier if the timeout behavior of heat and mistral was the same | 08:43 |
d0ugal | shardy: It would indeed, thanks :) | 08:44 |
d0ugal | unrelated: I think we are getting into a bit of a mess with our action names. | 08:45 |
*** bootsha has joined #tripleo | 08:46 | |
shardy | the class names or entry points? | 08:46 |
d0ugal | shardy: entry points | 08:46 |
d0ugal | The class names are generally much more specific | 08:47 |
openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Don't run undercloud idempotent on tempest jobs https://review.openstack.org/351066 | 08:47 |
shardy | d0ugal: they seem OK to me, but you'd like a more crud for each item kind of interface? | 08:47 |
d0ugal | shardy: for example: https://github.com/openstack/tripleo-common/blob/master/setup.cfg#L62 | 08:48 |
d0ugal | The right side, which we will never see in a workflow, gives me much more info | 08:48 |
d0ugal | (you just need to know it is a heat related thing) | 08:48 |
d0ugal | I don't know what tripleo.deploy_config does. | 08:49 |
shardy | I'm not sure I get it, both say update capabilities | 08:49 |
* shardy probably needs more coffee | 08:49 | |
d0ugal | Maybe I need less coffee | 08:49 |
d0ugal | shardy: we have a heat namespace on the right, on the left we assume people will know it is heat related | 08:50 |
shardy | d0ugal: Yeah, maybe that can be refined before we release, sure | 08:50 |
shardy | let's focus on getting feature parity first tho :) | 08:50 |
shardy | d0ugal: has any thought been given to how this switch to mistral will work on upgrade? | 08:51 |
d0ugal | heh, sure | 08:51 |
*** apetrich has quit IRC | 08:51 | |
shardy | like, if an overcloud stack exists, and there is no stored plan? | 08:51 |
d0ugal | shardy: lol, nope. | 08:51 |
* shardy puts head in hands | 08:51 | |
shardy | d0ugal: Ok, I added a new API to heat which means we can probably create the plan from heat | 08:52 |
d0ugal | shardy: I mean, they would just deploy as normal from the CLI - we just need to allow a plan to work with an existing stack. | 08:52 |
d0ugal | ... or we can do that. | 08:52 |
shardy | d0ugal: the update of any existing stack is going to expect a plan though right? | 08:52 |
shardy | and we'd end up with a partial plan because we do PATCH updates | 08:53 |
d0ugal | shardy: Yeah, but I assume users have the templates they deployed. | 08:53 |
*** jtomasek has joined #tripleo | 08:53 | |
d0ugal | shardy: ah, forgot about PATCH - dang | 08:53 |
shardy | Ok, well something to thing about, but I'll stop distracting you from fixing the basic deploy stuff :) | 08:53 |
shardy | s/thing/think | 08:53 |
d0ugal | jtomasek: Do you know if upgrades have ever been discussed? | 08:57 |
d0ugal | jtomasek: (Deploying, and then upgrading and having a deployment but no plan) | 08:58 |
jtomasek | d0ugal, afaik, a deployment should be always tied to a plan | 08:59 |
d0ugal | jtomasek: but what if they have deployed before plans existed? | 09:00 |
shardy | jtomasek: Existing deployments have no plan | 09:00 |
jtomasek | shardy: they should be able to import the template set that was used to deploy | 09:00 |
shardy | jtomasek: our update workflow doesn't require you to have all of the original templates, so we'll have to reconstruct the plan from heat | 09:00 |
shardy | We'll need a new action for this, I'll take a look after we get the deploy fully working | 09:01 |
jtomasek | shardy: hmm, I thought that customers were supposed to keep the templateset used for deployment | 09:01 |
shardy | jtomasek: we use PATCH updates, so strictly speaking it is not required | 09:02 |
shardy | probably a good idea tho, I agree | 09:02 |
jtomasek | shardy, how does the update currently work if user does not have the original templates? | 09:02 |
shardy | jtomasek: for upgrade we probably do expect the original templates, but the upgrade deploy commmand isn't the same as the original deployment | 09:02 |
shardy | the environment files used are all different | 09:03 |
shardy | so there's some complexity here we'll have to work out | 09:03 |
shardy | we can't just create a plan based on the upgrade deploy command AFAICS | 09:03 |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Add zaqar API endpoint in HAProxy https://review.openstack.org/351052 | 09:04 |
jtomasek | d0ugal, I totally agree, that update parameters action should take just a subset of all parameters (just those that are actually set by the user in that action call) | 09:07 |
jtomasek | d0ugal, I am going to look at that patch today and test it with GUI | 09:07 |
shardy | jtomasek: what's the status of integrating the UI with the undercloud? Are we still blocked on packaging? | 09:08 |
jtomasek | shardy, yeah, honza was not in yesterday, but packaging is progressing. Although I think that actual undercloud install integration work can be done in paralel. Here is the blueprint that lists things that GUI setup requires to be done | 09:10 |
jtomasek | shardy, https://blueprints.launchpad.net/tripleo-ui/+spec/instack-undercloud-ui-config | 09:10 |
*** numans has joined #tripleo | 09:11 | |
jtomasek | shardy: I know that mandre and florianf have been looking into it, mandre is there an actual patch that starts that implementation? you mentioned some review comments on separate puppet repositories yesterday | 09:12 |
shardy | jtomasek: ack, thanks - seems there are some questions there, let me know if you need help with those | 09:12 |
jaosorior | sshnaidm: By the way, If you referred to this CR https://review.openstack.org/#/c/345539/2 that error you talked about is to be expected | 09:12 |
jaosorior | it was an issue in mistral that I've fixed not too long ago. So that needs to be packaged still | 09:13 |
florianf | shardy, jtomasek: That's the patch in question: https://review.openstack.org/#/c/344140/ | 09:13 |
jtomasek | florianf, thx | 09:13 |
jaosorior | damn | 09:14 |
jaosorior | nevermind, just saw it was in the periodic one | 09:14 |
*** jbadiapa has quit IRC | 09:15 | |
jtomasek | shardy: at some patch review, you mentioned generating password parameter values, can you please remind me which one was it? | 09:15 |
jaosorior | sshnaidm: have you had luck reproducing it? | 09:15 |
sshnaidm | jaosorior, waiting.. | 09:16 |
sshnaidm | jobs take a long time now | 09:16 |
d0ugal | jtomasek: cool, I'll change it to do an update. Maybe we don't need an action to set them all. | 09:16 |
*** dsneddon has quit IRC | 09:19 | |
shardy | jtomasek: Hmm, not sure which review tbh - there was https://review.openstack.org/#/c/298682 where I see you commented | 09:21 |
shardy | I need to try the capabilities stuff, I'm not sure it works for user provided environments | 09:21 |
shardy | the passwords are currently generated in python-tripleoclient, so that needs to move into an action | 09:21 |
florianf | jtomasek: This one: https://review.openstack.org/#/c/300221/ | 09:22 |
florianf | jtomasek: Is that it? | 09:22 |
* jtomasek looks | 09:23 | |
hewbrocca | and an hour later /me finishes reading the backchat from last night | 09:23 |
hewbrocca | you guys talk too much | 09:23 |
jtomasek | shardy, the capabilities stuff depends on capabilities-map, so user-provided environment needs to be added to capabilities map to get listed there | 09:24 |
jtomasek | shardy, but if user wants to just enable the environment using update_capabilities, he can just send {<path_of_that_environment>.yaml: true} to it | 09:26 |
jtomasek | shardy, (obviously that environment needs to be present in the plan) | 09:26 |
shardy | jtomasek: Ok, well we have to hide all that complexity from the user, because that's not how user provided environments currently work | 09:28 |
shardy | they don't exist in the capabilities map, we just add them to the list and pass them to heat | 09:28 |
*** stendulker has joined #tripleo | 09:28 | |
shardy | jtomasek: probably we'll have to push user environments to the plan e.g user_environments/foo.yaml | 09:28 |
shardy | then add user_environments/foo.yaml to the capabilities map, and enable it | 09:29 |
lucasagomes | hi all, if you have time mind taking a look at this instack-undercloud patch https://review.openstack.org/#/c/350627/ ? It's automating the configuration of UEFI for iPXE and is a small patch too | 09:29 |
shardy | jtomasek: I was expecting the capabilities selection to output a list of environments, which we could just append to | 09:29 |
shardy | but I guess we can do everything via the capabilities map if needed | 09:30 |
shardy | lucasagomes: Nice, I'll try to check it out - I have a uefi box locally which I have never got to boot via ironic/ipxe | 09:30 |
lucasagomes | shardy, cool yeah that would be great | 09:30 |
jtomasek | shardy, so capabilities map is currently used only for listing the environments (all of them that are part of plan) - considering that they are in sync | 09:30 |
jtomasek | shardy: we could enhance the get_capabilities action to list environments from capabilities-map (as it does now) + one more special group that would include all remaining environments that are not in capabilities-map | 09:32 |
*** dsneddon has joined #tripleo | 09:32 | |
jtomasek | shardy: adding environment to existing plan is basically a plan update, we should be able to do it (not entirely sure it is implemented but there should not be any blocker for it) | 09:32 |
openstackgerrit | Martin André proposed openstack/instack-undercloud: Add tripleo deployment config file paths to hiera https://review.openstack.org/329016 | 09:33 |
openstackgerrit | Martin André proposed openstack/instack-undercloud: Introduce 'enable_validations' option https://review.openstack.org/322893 | 09:33 |
*** zoli|wfh is now known as zoli|brb | 09:33 | |
jtomasek | shardy: also, get_capabilities lists all environments that are in plan and for each environment it specifies whether the environment is enabled or not | 09:33 |
shardy | jtomasek: Ok, well I guess we can work through this as d0ugal gets the CLI patch worked up, essentially we must fix this to enable feature parity with the CLI interfaces | 09:33 |
jtomasek | shardy, ack | 09:34 |
*** electrofelix has joined #tripleo | 09:35 | |
ccamacho | bandini I had the same issue than you with the aodh submission, so, I tried to isolate the problem, I did this, deploy as it is (same error than you), remove the wsgi step and got http://paste.openstack.org/show/547913/ | 09:36 |
ccamacho | now at least is not failing but still managing the service with systemd.. | 09:37 |
shardy | Can I get a review on https://review.openstack.org/#/c/350599/ please? | 09:37 |
shardy | passing CI and already +2'd | 09:37 |
ccamacho | cant give +2 :P | 09:38 |
*** apetrich has joined #tripleo | 09:39 | |
*** jbadiapa has joined #tripleo | 09:40 | |
shadower | shardy: would I be overstepping my boundaries if I +2/+A'd it? It looks good to me | 09:41 |
bandini | ccamacho: so the reference to pacemaker is totally okay when a service is managed by pacemaker | 09:41 |
mandre | I lost +A when rebasing https://review.openstack.org/#/c/329016/, can I get a quick review? | 09:41 |
*** mgould|afk is now known as mgould | 09:41 | |
bandini | ccamacho: so it is wsgi screwing up things | 09:41 |
ccamacho | so now the service is actually managed by pcm? | 09:41 |
shardy | shadower: sure, thanks! | 09:42 |
bandini | ccamacho: it always isin the HA deployment (until NG HA lands) | 09:42 |
ccamacho | I think that was the reason why I remove it in first place.. Was breaking the deployment.. | 09:42 |
ccamacho | ack | 09:42 |
ccamacho | Ill start checking this wsgi manifest.. | 09:43 |
shadower | shardy: yeah I wasn't sure because we should be using +A on the validations repo only, but this is a tiny patch | 09:43 |
bandini | ccamacho: so we need to figure out why wsgi is deciding to manage httpd via systemd anyway | 09:43 |
bandini | ccamacho: awesome, thanks! | 09:43 |
shardy | shadower: Yup, I think it's fine for you to use your judgement and as you say it's a pretty simple patch | 09:44 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Replace hard-coded regionOne with parameter references https://review.openstack.org/350599 | 09:46 |
*** dbecker has joined #tripleo | 09:46 | |
*** dbecker has quit IRC | 09:50 | |
*** dbecker has joined #tripleo | 09:50 | |
social | EmilienM: I'm thinking about changing all package{'package':} in undercloud to ensure_resource | 09:51 |
hewbrocca | pimping https://review.openstack.org/#/c/347399/ for workflow +1 | 09:53 |
social | lucasagomes: ping - https://review.openstack.org/#/c/334881 | 09:53 |
social | lucasagomes: I'm working on dropping whole ipxe element from undercloud | 09:53 |
lucasagomes | social, hi there | 09:54 |
hewbrocca | paramite spec, already received many +2 before last change | 09:54 |
lucasagomes | social, oh cool, so that patch will do part of the job that my patch is doing (copying the ipxe.efi image to /tftpboot) ? | 09:54 |
hewbrocca | for ops-tools... | 09:55 |
hewbrocca | anybody? | 09:55 |
*** limao has quit IRC | 09:55 | |
social | lucasagomes: file { "${tftp_root}/ipxe.efi": | 09:55 |
lucasagomes | nice | 09:55 |
hewbrocca | die elements die | 09:56 |
marios | hewbrocca: added to my review queue for tomorrow morning, fwiw, assuming it is still around then. i haven't seen it before and its a long read | 09:56 |
lucasagomes | social, what's the story with the -2 there? Are you working on breaking it up in smaller patches? | 09:56 |
social | hewbrocca: yeah it would be nice to remove elements and use puppet but we use puppet as shell script | 09:56 |
social | lucasagomes: yes, WIP | 09:56 |
hewbrocca | marios: thank you | 09:57 |
hewbrocca | ... we use puppet as shell script... | 09:57 |
shardy | hewbrocca: I had previously reviewed and now approved | 09:57 |
hewbrocca | shardy: thank you! | 09:57 |
lucasagomes | hewbrocca, +1 | 09:57 |
shardy | marios: feel free to review anyway and we can discuss any questions with paramite | 09:58 |
hewbrocca | oof | 09:58 |
marios | thanks shardy sure will do | 09:58 |
jtomasek | d0ugal, (shardy): So I've applied the templates processing and parameters actions patches and now when I run the action I get http://paste.openstack.org/show/548090/ | 09:59 |
marios | pimping reviews aint for the faint hearted it can get rough hewbrocca stay strong | 09:59 |
lucasagomes | social, so just to understand, I don't think we should block that patch because it's adding one line to the ipxe element copying the ipxe.efi binary to the /tftpboot right? | 10:00 |
hewbrocca | marios: LOL I know | 10:00 |
lucasagomes | social, once that patch to puppet-ironic merges we can remove the ipxe element all together | 10:00 |
jtomasek | d0ugal, can you please try to reproduce by calling mistral run-action tripleo.get_parameters '{"container": "<your_plan_name>"}' | 10:00 |
openstackgerrit | Merged openstack/tripleo-specs: Availability monitoring support https://review.openstack.org/347399 | 10:02 |
d0ugal | jtomasek: Yeah, I get the same error. | 10:04 |
d0ugal | jtomasek: The CLI doesn't use that one at the moment. | 10:05 |
jaosorior | sshnaidm: I see the issue now. It's a mistral error. Will try to fix ASAP | 10:05 |
sshnaidm | jaosorior, thanks, the reproducing stuck, trying again.. | 10:06 |
openstackgerrit | Merged openstack/tripleo-specs: Update rendered specs URL https://review.openstack.org/350618 | 10:06 |
*** jbadiapa has quit IRC | 10:12 | |
*** dtantsur|bbl is now known as dtantsur | 10:12 | |
*** rasca has quit IRC | 10:16 | |
*** rasca has joined #tripleo | 10:18 | |
shardy | jtomasek: I have reproduced, it's a heat issue | 10:19 |
shardy | jtomasek: for now we'll have to put some dummy values into the action | 10:20 |
shardy | heat template-validate --show-nested --template-file overcloud.yaml -e overcloud-resource-registry-puppet.yaml | 10:20 |
shardy | that will reproduce it | 10:20 |
*** jbadiapa has joined #tripleo | 10:27 | |
*** chlong has quit IRC | 10:28 | |
d0ugal | jtomasek: http://paste.openstack.org/show/548614/ | 10:29 |
d0ugal | jtomasek: do you have any idea why the environments are a dictionary with only the path key? | 10:29 |
d0ugal | jtomasek: why isn't it just a list of strings. | 10:29 |
*** tosky has joined #tripleo | 10:31 | |
*** zoli|brb is now known as zoli|wfh | 10:33 | |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds parameters actions https://review.openstack.org/298682 | 10:33 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow https://review.openstack.org/298732 | 10:33 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds action for template processing https://review.openstack.org/337615 | 10:33 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans https://review.openstack.org/351111 | 10:33 |
d0ugal | The deploy command is a mess of weird things that we need to figure out how to support with Mistral :( | 10:38 |
jpich | d0ugal: What kind of information are we using the Mistral environments for at the moment? | 10:39 |
d0ugal | jpich: http://paste.openstack.org/show/548614/ | 10:39 |
d0ugal | jpich: parameters, root template and a list of enable environments | 10:40 |
d0ugal | I think that is all | 10:40 |
* shardy wonders if mistral encrypts the environments | 10:40 | |
d0ugal | nope | 10:40 |
d0ugal | It isn't intended to be a secure store. | 10:40 |
shardy | Ok, then we'll have to move the password stuff into the deployment workflow | 10:40 |
shardy | I commented on that yesterday for other reasons e.g plan reuse | 10:41 |
d0ugal | but swift doesn't encrypt them either? | 10:41 |
shardy | d0ugal: heat does | 10:41 |
*** chlong has joined #tripleo | 10:41 | |
d0ugal | shardy: don't we need to store them somewhere else? | 10:41 |
jpich | d0ugal: "enable environments"? | 10:41 |
d0ugal | will heat give me my paswords back? | 10:41 |
shardy | d0ugal: If we can't do it semi securely, I'm not sure if we should | 10:42 |
d0ugal | shardy: I completely agree. | 10:42 |
* jpich reading up on http://docs.openstack.org/developer/heat/template_guide/environment.html#environments and trying to link all the uses of the term 'environment' | 10:42 | |
d0ugal | shardy: I have complained about this in the past :( | 10:42 |
shardy | d0ugal: Ok, well lets get it working, then we can revisit the discussion | 10:43 |
d0ugal | jpich: Yeah, sorry - I should say Heat environment. | 10:43 |
d0ugal | jpich: They are used to enable and disable certain features. | 10:43 |
shardy | jpich: they are kinda similar, except the syntax is different | 10:43 |
d0ugal | jpich: For example: http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/network_isolation_virt.html#modify-your-overcloud-deploy-to-enable-network-isolation - this adds the network isolation envs via the CLI | 10:43 |
jpich | d0ugal: Sorry, I'm starting from really the bottom. Thank you for the link and explanations | 10:45 |
d0ugal | jpich: no problem - I hope I am answering the correct question! | 10:45 |
jpich | shardy: Thank you. So we use them to do the same kind of thing, then? | 10:45 |
jpich | d0ugal: I'll tell you when I have that figured out, too ;) | 10:45 |
*** apetrich has quit IRC | 10:45 | |
shardy | jpich: Yes, in both cases they provide data that influences the thing that is run | 10:45 |
shardy | jpich: in mistral it's a workflow/action and in heat it's an orchstration template | 10:45 |
openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart: Fix wrong naming of image's name in release. https://review.openstack.org/350712 | 10:46 |
jpich | shardy: Cool! There's not much mistral documentation about environments so if I can adapt the heat docs to understand them I'm happy | 10:46 |
shardy | jpich: it's a little confusing because in this case we're basically storing a definition on the heat environment inside the mistral environment | 10:46 |
jpich | shardy: Is a Mistral environment associated with a workflow (one workflow = one env) or with an execution (different runs of a workflow can use a different environments)? | 10:47 |
*** r2bit has quit IRC | 10:47 | |
d0ugal | jpich: no, you refer to a environment by it's name. | 10:48 |
d0ugal | jpich: so it can be shared across workflows. We use one at the moment that has the same name as the Heat stack (and swift container) | 10:49 |
sshnaidm | shardy, can you please vote for https://review.openstack.org/#/c/351120/ ? | 10:50 |
sshnaidm | pabelanger, ^^ | 10:50 |
sshnaidm | seems like we lost tempest job in all these movements | 10:50 |
shardy | sshnaidm: +1 - do you know if it will run within the infra timeout? | 10:51 |
jtomasek | d0ugal: see discussion here: https://review.openstack.org/#/c/349732/1/tripleo_common/actions/heat_capabilities.py | 10:51 |
sshnaidm | shardy, to be sure I have another patch for it :) https://review.openstack.org/#/c/351066/ | 10:52 |
jpich | d0ugal: Ok, thank you! I can see in the Mistral docs how to call on environment variables from a workflow/task | 10:52 |
jpich | It's just not totally clear to me when/how a workflow gets linked with a particular environment but I'll figure that one out later | 10:52 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans https://review.openstack.org/351111 | 10:52 |
sshnaidm | shardy, in last month jobs took more and more time again, I need another 15 minutes for tempest, so removing the second undercloud install there | 10:53 |
*** ohamada has quit IRC | 10:53 | |
*** ohamada has joined #tripleo | 10:53 | |
d0ugal | jpich: I am not sure if they are ever accessed from the workflow. We access it from the actions: https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/heat_capabilities.py#L61 | 10:53 |
jpich | d0ugal: Oh, thank you for the link! That helps a lot | 10:54 |
jtomasek | jpich, mistral environment name = plan name | 10:54 |
d0ugal | but yeah, that is another reason to not store secure stuff in Mistral. Any action (or workflow maybe) can access it. I don't think it is tied to a user. | 10:54 |
d0ugal | shardy: ^ | 10:54 |
shardy | d0ugal: ouch :( | 10:55 |
openstackgerrit | Merged openstack/instack-undercloud: Add tripleo deployment config file paths to hiera https://review.openstack.org/329016 | 10:55 |
jpich | d0ugal: Scope seems to be same project_id or public - http://docs.openstack.org/developer/mistral/developer/webapi/v2.html#get--v2-environments | 10:56 |
d0ugal | jpich: oh, good. better than I thought | 10:56 |
jtomasek | d0ugal, shardy: regarding password generating, how do we find out there is a password to generate? We should not hardcode parameter names in workflows | 10:56 |
d0ugal | jtomasek: There is no way :/ | 10:57 |
d0ugal | ... that I know of. | 10:57 |
jtomasek | d0ugal, shardy: ideal would be parameter type password, but there isnt such, so how about a constraint (custom constraint) | 10:57 |
shardy | Well there are two ways I can think of - one is to introspect the templates via heat nested validation (which we do anyway, ref the bug jtomasek mentioned), then identify all *Passsword parameters that have no default (which should be nearly all of them) | 10:58 |
shardy | the other (long term better) way would be to have a way to get heat to generate a random string as a default value | 10:58 |
d0ugal | jtomasek, shardy: They are not all called "*Password" - i.e. NeutronMetadataProxySharedSecret | 10:59 |
shardy | jtomasek: We could just use either a naming convention, or parameter_groups | 10:59 |
shardy | jtomasek: as you say, a new parameter type would be a way to enable the auto-generated random password | 10:59 |
shardy | s/NeutronMetadataProxySharedSecret/NeutronMetadataProxySharedPassword | 11:00 |
shardy | ;) | 11:00 |
d0ugal | AdminToken :P | 11:00 |
d0ugal | (There is already an AdminPassword) | 11:00 |
* shardy throws something at d0ugal | 11:00 | |
d0ugal | lol | 11:00 |
*** bootsha has quit IRC | 11:01 | |
*** stendulker has quit IRC | 11:02 | |
*** r2bit has joined #tripleo | 11:02 | |
*** r2bit has quit IRC | 11:03 | |
*** r2bit has joined #tripleo | 11:06 | |
openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 11:06 |
jtomasek | shardy: yeah, if the parameter type password value is not provided, then autogenerate it... | 11:07 |
jtomasek | shardy: do you know how I could workaround that 'ERROR: The Parameter (DeployIdentifier) was not provided.' bug? | 11:08 |
openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Ignore the .eggs directory https://review.openstack.org/351126 | 11:08 |
*** akrivoka has joined #tripleo | 11:09 | |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow https://review.openstack.org/298732 | 11:11 |
sshnaidm | jaosorior, it's weird, I can't reproduce mistral issue, introspection was passed in my test job, can you send me your solution please when it's ready? i'll check it on my env | 11:14 |
*** bootsha has joined #tripleo | 11:14 | |
jaosorior | sshnaidm: Will do | 11:15 |
jtomasek | d0ugal: how is this command going to work? openstack overcloud deploy --templates /tmp/templates -e /tmp/templates/environments/enable_ceph.yaml -e ~/local_parameters.yaml | 11:17 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow https://review.openstack.org/298732 | 11:17 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans https://review.openstack.org/351111 | 11:17 |
jtomasek | d0ugal, is it going to be creating plan from provided templates and setting those environments 'enabled'? | 11:17 |
jtomasek | d0ugal, and then depoying? | 11:17 |
d0ugal | jtomasek: that is what we are trying to figure out | 11:17 |
d0ugal | jtomasek: but yeah, something like that. | 11:18 |
jtomasek | d0ugal: ok, so the command is either going to use --templates or --plan | 11:19 |
d0ugal | jtomasek: no, I don't think that will happen. | 11:19 |
d0ugal | well, it might | 11:19 |
d0ugal | I don't know. This hasn't been discussed. | 11:19 |
d0ugal | (other than in the last few days) | 11:19 |
jtomasek | d0ugal, ok, yeah. I haven't gotten to deployment yet, I need to get parameters actions ready to be able to progress (that bug is blocking right now) | 11:20 |
*** lucasagomes is now known as lucas-hungry | 11:21 | |
*** chlong has quit IRC | 11:27 | |
*** chlong has joined #tripleo | 11:27 | |
*** dsariel has joined #tripleo | 11:29 | |
*** oshvartz has joined #tripleo | 11:30 | |
*** sshnaidm is now known as sshnaidm|lnch | 11:31 | |
*** rhallisey has joined #tripleo | 11:34 | |
jaosorior | sshnaidm: This should work for mistral https://review.openstack.org/#/c/351134/1 | 11:40 |
jaosorior | sshnaidm: Are you able to test with that? | 11:43 |
jaosorior | sshnaidm: It seems to fix the issue in my environment. But of course, more reviewers/people testing should happen | 11:44 |
*** jpena is now known as jpena|lunch | 11:51 | |
*** weshay has joined #tripleo | 11:56 | |
*** bfournie has quit IRC | 11:56 | |
*** lucas-hungry is now known as lucasagomes | 11:58 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 12:05 |
*** egafford has quit IRC | 12:07 | |
jaosorior | EmilienM: quick CR removing code https://review.openstack.org/#/c/349812/ :D | 12:07 |
*** sshnaidm|lnch is now known as sshnaidm | 12:08 | |
EmilienM | I love when we remove code | 12:08 |
sshnaidm | jaosorior, yep, I'll give a try | 12:08 |
adarazs | folks, has anybody seen this recently? InputException: Invalid input [name=baremetal_introspection.wait_for_finish, class=NoneType, unexpected=[u'uuids']] | 12:10 |
adarazs | this started happening during introspection on the master branch, blocks promotion currently. | 12:11 |
jaosorior | adarazs: Yep, trying to fix it here | 12:11 |
jaosorior | adarazs: https://review.openstack.org/#/c/351134/ | 12:11 |
adarazs | jaosorior++ | 12:11 |
adarazs | thanks | 12:11 |
adarazs | weshay: ^ | 12:11 |
jaosorior | It's a mistral issue | 12:11 |
EmilienM | ccamacho: hey, can I help on aodh composable role? | 12:11 |
weshay | adarazs, k.. please add that to the etherpad | 12:12 |
ccamacho | was going to ask you for help | 12:12 |
ccamacho | EmilienM | 12:12 |
jaosorior | adarazs, weshay: If you guys can reproduce the error and help test the fix. It would be great. Seems to work in my environment, but more eyes and testers are appreciated | 12:12 |
ccamacho | :) | 12:12 |
weshay | adarazs, probably also want to run rdo-ci-check | 12:12 |
sshnaidm | jaosorior, can you please mention in your commit? https://bugs.launchpad.net/tripleo/+bug/1609688 | 12:12 |
openstack | Launchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Undecided,New] | 12:12 |
sshnaidm | adarazs, ^^ | 12:12 |
adarazs | jaosorior: yep, I can test your fix if you think that works | 12:12 |
EmilienM | ccamacho: good, do you need me to test? review? tell me | 12:13 |
adarazs | weshay: we could, but the error is not present in master, so I don't know if it will fix it :) | 12:13 |
ccamacho | EmilienM We are having an issue when using apache wgsi https://review.openstack.org/#/c/332854/15/manifests/profile/pacemaker/aodh/api.pp L41 | 12:13 |
*** apetrich has joined #tripleo | 12:13 | |
adarazs | weshay: only in the unpromoted master. | 12:13 |
*** jayg|g0n3 is now known as jayg | 12:13 | |
dtantsur | EmilienM, hi! when you have a second, could you please take a look at https://review.openstack.org/#/c/344702 ? seems to work in my testing.. | 12:14 |
*** dculik has quit IRC | 12:14 | |
EmilienM | dtantsur: sure | 12:14 |
EmilienM | ccamacho: ok | 12:14 |
ccamacho | EmilienM If not used then httpd seems to start http://paste.openstack.org/show/547913/ | 12:14 |
jaosorior | adarazs: Are you gonna test it in CI or will you try to reproduce it locally? | 12:14 |
sshnaidm | EmilienM, I think we can proceed with this: https://review.openstack.org/#/c/349995/ the error in job is timeout of postci, so everything succeeded | 12:15 |
weshay | adarazs, oh.. you know what.. there's a thought.. I wonder if the 3rd party gate job should pull the testing image | 12:15 |
weshay | adarazs, it probably should | 12:15 |
adarazs | jaosorior: so, I can do both. but the last promoted master image is a bit old, so I hope it will work. :) | 12:15 |
weshay | adarazs, know what I mean? | 12:15 |
EmilienM | sshnaidm: ok, wait a sec, looking other things now | 12:15 |
adarazs | weshay: we could maybe have a keyword to invoke it like that rdo-ci-check-test or something, but I would think it's better to use a promoted image by default that works. | 12:16 |
ccamacho | EmilienM im testing the wsgi manifest but not finding any useful clue yet. | 12:16 |
sshnaidm | EmilienM, sure, no hurry | 12:16 |
weshay | adarazs, sure.. as long as we have the option.. ++ | 12:16 |
adarazs | weshay: should I make a card for it? | 12:16 |
weshay | adarazs, aye | 12:16 |
weshay | sshnaidm, morning :) | 12:16 |
EmilienM | dtantsur: +A | 12:16 |
dtantsur | \o/ | 12:16 |
sshnaidm | weshay, hey :) | 12:17 |
EmilienM | ccamacho: ok looking logs now | 12:17 |
ccamacho | thanks! | 12:17 |
*** Goneri has quit IRC | 12:17 | |
EmilienM | ccamacho: why are we spending time on pacemaker/aodh | 12:18 |
EmilienM | I thought we were removing it | 12:18 |
adarazs | weshay: https://trello.com/c/pff4maAX/308-make-it-possible-to-run-3rd-party-ci-with-the-unpromoted-testing-images | 12:18 |
EmilienM | ccamacho: I'm sure we can sync with bandini and check if we really need Aodh with pacemaker | 12:18 |
openstackgerrit | Merged openstack/puppet-tripleo: Remove unnecessary certmonger manifest https://review.openstack.org/349812 | 12:18 |
ccamacho | would be great if we just can remove it, but not sure if we need it at least for now | 12:19 |
EmilienM | bandini: hey! in the new HA arch, do we still manage apache with pacemaker? | 12:20 |
EmilienM | I think yes, look https://review.openstack.org/#/c/314208/29/environments/puppet-pacemaker.yaml | 12:20 |
EmilienM | ccamacho: ^ | 12:20 |
EmilienM | ccamacho, bandini: I think we have a patch order problem on https://review.openstack.org/#/c/314208/ | 12:21 |
EmilienM | imho, we should rather push 314208 first and on top the aodh thing | 12:22 |
*** tbonds has joined #tripleo | 12:24 | |
openstackgerrit | Lukas Bezdicka proposed openstack/instack-undercloud: Use ensure_resource instead of package https://review.openstack.org/351148 | 12:25 |
*** Goneri has joined #tripleo | 12:25 | |
*** jeckersb is now known as jeckersb_gone | 12:25 | |
*** tzumainn has joined #tripleo | 12:25 | |
EmilienM | social: you could even declare them in an array :) | 12:26 |
*** zoli|wfh is now known as zoli|lunch | 12:27 | |
*** dprince has joined #tripleo | 12:28 | |
social | EmilienM: I'd like to drop them | 12:28 |
openstackgerrit | Merged openstack/puppet-tripleo: Fix Ironic dbsync ordering https://review.openstack.org/344702 | 12:29 |
shardy | Error: Could not find a suitable provider for mysql_datadir | 12:30 |
shardy | http://logs.openstack.org/41/348441/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/480fe5d/console.html | 12:30 |
shardy | hrm | 12:30 |
openstackgerrit | Merged openstack/puppet-tripleo: Add base nova-ironic profile https://review.openstack.org/341420 | 12:30 |
hewbrocca | Why the hell would we manage apache with pacemaker? | 12:30 |
*** bfournie has joined #tripleo | 12:31 | |
hewbrocca | it's stateless... | 12:31 |
hewbrocca | unless haproxy? | 12:31 |
jaosorior | shardy: if you go up there seem to be other errors | 12:31 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-heat-templates: DO NOT MERGE: testing ironic https://review.openstack.org/329872 | 12:31 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-heat-templates: Add composable service for configuring Nova compute with Ironic https://review.openstack.org/341433 | 12:31 |
*** ramishra has quit IRC | 12:31 | |
jaosorior | shardy: For instance, this http://paste.openstack.org/show/549218/ | 12:31 |
jaosorior | No such file or directory: '/sys/class/net/eth1/address' | 12:31 |
jaosorior | I remember there being a patch by slagle to check for eth1 in tripleo-ci. Might be related? | 12:32 |
sshnaidm | I think we have a problem in CI with ha jobs, pingtest is always failing for various patches | 12:32 |
*** ramishra has joined #tripleo | 12:33 | |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-heat-templates: DO NOT MERGE: testing ironic https://review.openstack.org/329872 | 12:34 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-heat-templates: Add composable service for configuring Nova compute with Ironic https://review.openstack.org/341433 | 12:34 |
jaosorior | hewbrocca: No why we would be managing apache with pacemaker... not sure how haproxy would be involved either | 12:34 |
hewbrocca | bandini: ^^^ ? | 12:35 |
*** morazi has joined #tripleo | 12:35 | |
shardy | http://logs.openstack.org/41/348441/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/480fe5d/logs/undercloud/var/log/undercloud_install.txt.gz#_2016-08-04_10_50_54_000 | 12:35 |
*** pradk_ has quit IRC | 12:35 | |
shardy | jaosorior: Yeah, you're right, thanks | 12:35 |
*** dculik has joined #tripleo | 12:35 | |
slagle | jaosorior: i dont think that error is related to what i was fixing | 12:37 |
shardy | https://github.com/openstack-infra/tripleo-ci/commit/d8b33a7f4253ea122902ff60269642b62e90eebb | 12:37 |
slagle | that error implies that there is no eth1 | 12:37 |
shardy | Yeah it seems to be a different issue | 12:37 |
jaosorior | I see | 12:38 |
slagle | looks like something wrong with how ovb setup the env | 12:38 |
slagle | b/c the instance for the uc only has an eth0 | 12:38 |
EmilienM | jaosorior, hewbrocca: I don't think we will manage apache with pacemaker, see https://review.openstack.org/#/c/314208/29/environments/puppet-pacemaker.yaml | 12:40 |
hewbrocca | It wouldn't make any sense if we did | 12:40 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow https://review.openstack.org/298732 | 12:40 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans https://review.openstack.org/351111 | 12:40 |
jaosorior | dprince: By the way, the zaqar issue I was telling you about was an actual bug | 12:41 |
jaosorior | dprince: can you check out this CR attempting to fix it? https://review.openstack.org/#/c/351039/ | 12:42 |
openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Add environment file to enable DVR https://review.openstack.org/332147 | 12:42 |
*** fultonj has joined #tripleo | 12:43 | |
shardy | gfidente: Hey, you wrote http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/node_specific_hieradata.html#collecting-the-node-uuid | 12:44 |
shardy | gfidente: and cschwede has referenced it on the ML | 12:44 |
shardy | gfidente: however, I don't think there is any way for a node to determine it's ironic UUID at runtime, is there? | 12:45 |
shardy | last time I tried to find it I gave up and used the DMI system ID intead | 12:45 |
shardy | instead | 12:45 |
cschwede | shardy: seems it is - it’s working in my tests | 12:45 |
shardy | cschwede: That's good news, but where do you get the ironic ID on the node during deployment? | 12:46 |
dprince | jaosorior: ack, glad you found it | 12:46 |
shardy | something may well have changed since I last investigated (or, I missed it at the time which is entirely possible :) | 12:46 |
cschwede | shardy: it’s not the ironic uuid, but the machine uuid | 12:46 |
shardy | cschwede: Ok, so the docs are wrong, cool | 12:47 |
* bandini back | 12:48 | |
gfidente | shardy, cschwede right we don't want the ironic uuid | 12:48 |
gfidente | we want the dmi id there | 12:48 |
cschwede | shardy: i think the doc is right? i was irritate at first and used the ironic uuid, but then re-read the docs again and recognized that it is the machine uuid | 12:48 |
shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/extraconfig/pre_deploy/per_node.yaml | 12:48 |
gfidente | because that's how the per-node hieradata is distributed | 12:48 |
shardy | Yeah, it references the template I wrote that uses the DMI id | 12:48 |
shardy | I'll post a patch to the docs | 12:48 |
shardy | thanks! | 12:48 |
*** jdob has quit IRC | 12:48 | |
shardy | "NODE-ID is the target node Ironic UUID" | 12:49 |
shardy | cschwede: ^^ | 12:49 |
*** jdob has joined #tripleo | 12:49 | |
shardy | I think that's wrong | 12:49 |
cschwede | shardy: that references the command above | 12:49 |
cschwede | „ The value returned by the above command will be a unique and immutable machine UUID which isn’t related to the Ironic node UUID. „ | 12:49 |
gfidente | shardy, exactly | 12:49 |
panda | pabelanger: you have time to chant in bj ? | 12:49 |
gfidente | it's the template which is describing it badly | 12:49 |
gfidente | what we want is the machine uuid | 12:49 |
gfidente | which is what the command gives | 12:49 |
bandini | EmilienM, ccamacho: (sorry just got back now) so no in the HA NG we do not run aodh nor apache under pacemaker. | 12:49 |
shardy | cschwede: gah, yeah, you're right, it's just confusingly worded | 12:49 |
shardy | Ok, carry on :) | 12:50 |
EmilienM | bandini: it simplifies things for us | 12:50 |
EmilienM | ccamacho: we can remove all pacemaker things for aodh and rebase on top of bandini | 12:50 |
bandini | EmilienM, ccamacho: I can certainly push NG HA before aodh too, I was mainly doing it because "we did it for all the other composable services" | 12:50 |
bandini | that will mean that the aodh pacemaker profile will be broken, but hey not really a big deal | 12:50 |
bandini | for ocata I was going to propose to remove all the ones we do not use | 12:51 |
EmilienM | bandini: yeah, we need both asap anyway | 12:51 |
ccamacho | EmiliemM ack, then removing all the pacemaker logic from THT and PT | 12:51 |
bandini | EmilienM: ack. let me rebase without aodh, run a local test. and then I will submit and ping you and ccamacho | 12:51 |
bandini | sounds good? | 12:51 |
EmilienM | bandini: I think we can even remove them all in newton | 12:51 |
EmilienM | bandini++ | 12:51 |
openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Improve readability of the overcloud failure output https://review.openstack.org/351155 | 12:51 |
bandini | EmilienM: ack, my only concern with removing them from newton is that if it turns out that a service does not cope well without pacemaker (because it needs some ordering constraints), we can still easily switch it to pacemaker | 12:52 |
bandini | that's why I'd let one release go with having them around | 12:53 |
bandini | maybe it is me just being extra cautious ;) | 12:53 |
openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Implement aodh profiles https://review.openstack.org/332854 | 12:54 |
*** dtantsur is now known as dtantsur|mtg | 12:54 | |
*** dciabrin has quit IRC | 12:55 | |
*** rlandy has joined #tripleo | 12:56 | |
chem | hi, is there a password defined by default for the heat-admin user, to log in using the console ? | 12:56 |
pabelanger | panda: not at the moment. But I can here in IRC | 12:56 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Add Aodh composable roles https://review.openstack.org/333556 | 12:57 |
*** jpena|lunch is now known as jpena | 12:58 | |
*** mgould is now known as mgould|mtg | 12:58 | |
ccamacho | EmilienM, bandini deploying aodh locally now | 12:59 |
*** lblanchard has joined #tripleo | 13:00 | |
bandini | ccamacho: what changes did you make in the last iteration? | 13:01 |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-docs: Upgrade documentation https://review.openstack.org/308985 | 13:01 |
ccamacho | remove the pacemaker profiles "EmilienM: ccamacho: we can remove all pacemaker things for aodh and rebase on top of bandini" | 13:02 |
*** jpich has quit IRC | 13:03 | |
ccamacho | bandini ^ | 13:03 |
bandini | ccamacho: ah ok. got you | 13:03 |
ccamacho | just testing all now | 13:03 |
*** pradk has quit IRC | 13:04 | |
*** numans has quit IRC | 13:05 | |
*** fultonj has quit IRC | 13:08 | |
openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Next generation HA architecture work https://review.openstack.org/314208 | 13:08 |
bandini | ccamacho: I am testing without the aodh stuff as well ^ | 13:08 |
*** fultonj has joined #tripleo | 13:09 | |
gfidente | shardy, cschwede sorry I was on a call but I think both the docs and the command are good, isn't it the case? | 13:09 |
shardy | gfidente: Yeah, I just misread it, sorry for the noise | 13:09 |
jaosorior | shardy, EmilienM, d0ugal: Now that we're using mistral more extensively, would it make sense to propose that tripleo's nonha job run on mistral commits? | 13:09 |
gfidente | shardy, cschwede I read the reply on the list | 13:10 |
EmilienM | jaosorior: yes, the multinode non ha | 13:10 |
gfidente | I'm fine building on the undercloud | 13:10 |
EmilienM | jaosorior: go for it | 13:10 |
jaosorior | EmilienM: Does the multinode nonha run introspection? | 13:10 |
shardy | jaosorior: Maybe, but fwiw we don't do that for any other repos, even heat | 13:10 |
shardy | so we mostly rely on the periodic promotion to fail and catch issues | 13:11 |
gfidente | it's the static mapping I was trying to avoid as well | 13:11 |
shardy | we could definitely add an experimental job | 13:11 |
jaosorior | shardy: I see | 13:11 |
*** liverpooler has quit IRC | 13:14 | |
EmilienM | jaosorior: no, then you need ovb afik | 13:18 |
openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Validation to verify the undercloud RAM reqs https://review.openstack.org/351179 | 13:18 |
d0ugal | shardy: okay! I think I have something working | 13:19 |
*** egafford has joined #tripleo | 13:19 | |
*** jeckersb_gone is now known as jeckersb | 13:20 | |
shardy | d0ugal: great! :) | 13:20 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Quick fixes https://review.openstack.org/351181 | 13:21 |
openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 13:22 |
d0ugal | shardy: With those two patches ^ | 13:22 |
d0ugal | shardy: The first one I need to move into the parent commits, but wanted to push something up now | 13:22 |
shardy | lol @ "Quick fixes" | 13:22 |
d0ugal | shardy: I'll sort that out when I get back from walking the dog. she is getting angry at being neglected :) | 13:22 |
shardy | def deploy_hacks(self) | 13:23 |
shardy | d0ugal: sounds good, I'll check it out after the deep-dive call | 13:23 |
d0ugal | k | 13:23 |
openstackgerrit | Lukas Bezdicka proposed openstack/instack-undercloud: Use ensure_resource instead of package https://review.openstack.org/351148 | 13:24 |
*** karthiks has quit IRC | 13:25 | |
EmilienM | the undercloud upgrade job I've been working on is now failing because of Mistral db sync | 13:27 |
EmilienM | http://logs.openstack.org/95/346995/30/experimental/gate-tripleo-ci-centos-7-undercloud-upgrades-nv/6aa9788/console.html#_2016-08-04_12_45_35_853741 | 13:27 |
EmilienM | anyone saw it before? | 13:27 |
EmilienM | anyone able to upgrade undercloud to trunk now? | 13:28 |
hewbrocca | I would be amazed if anyone has tried EmilienM | 13:28 |
panda | pabelanger: ok, so I remember you saying that you could not upload tripleo-ci base image to some place public because it was too big. Do you think it's possible to build an alternate image that doesn't include project clones and put that somewhere public ? I think 5 of those 6 Gigabytes are project clones. | 13:29 |
*** gfidente has quit IRC | 13:29 | |
openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Add environment file to enable DVR https://review.openstack.org/332147 | 13:30 |
*** julim has joined #tripleo | 13:31 | |
*** myoung|remote has joined #tripleo | 13:32 | |
jaosorior | EmilienM: yeah, it hit RDO too. Trying to fix it here https://review.openstack.org/#/c/351134/ | 13:34 |
pabelanger | panda: Ya, so after we talked the other day, I started looking into python-tripleoclient and seeing if we can create gate jobs that do just that. I'm testing with the overcloud-full at the moment, but if it works, we could do the same for the undercloud too | 13:34 |
EmilienM | oh so the CI job I'm doing now is actually useful? | 13:34 |
jaosorior | ah | 13:35 |
jaosorior | wait | 13:35 |
jaosorior | that might be another issue | 13:35 |
jaosorior | not sure | 13:35 |
*** gfidente has joined #tripleo | 13:36 | |
*** gfidente has quit IRC | 13:36 | |
*** gfidente has joined #tripleo | 13:36 | |
EmilienM | jaosorior: mistral dbsync fails ! | 13:36 |
EmilienM | I see it can be related to your patch | 13:36 |
jaosorior | EmilienM: where can I find the logs? | 13:36 |
jaosorior | for the undercloud | 13:36 |
EmilienM | I'm afraid there is no dbsync log but I can fix it | 13:36 |
EmilienM | let me a sec | 13:36 |
*** numans has joined #tripleo | 13:36 | |
jaosorior | EmilienM: that would be great! | 13:37 |
panda | pabelanger: great, thank! ping me if you need any help | 13:37 |
sshnaidm | pabelanger, hi, maybe you know why does it happen? "No JSON object could be decoded" in http://logs.openstack.org/15/337615/14/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/854e8a0/console.html#_2016-08-04_10_40_55_249963 | 13:38 |
*** karthiks has joined #tripleo | 13:39 | |
*** dciabrin has joined #tripleo | 13:40 | |
*** pkovar has joined #tripleo | 13:40 | |
EmilienM | jaosorior: found it | 13:42 |
EmilienM | in undercloud_install.txt | 13:42 |
openstackgerrit | Attila Darazs proposed openstack/tripleo-heat-templates: GATE TEST, please ignore https://review.openstack.org/346698 | 13:42 |
EmilienM | http://paste.openstack.org/show/Af8BXZOQopqDAOi69k6M/ | 13:42 |
EmilienM | jaosorior: ^ | 13:42 |
*** dculik has quit IRC | 13:43 | |
jaosorior | EmilienM: That actually doesn't seem related | 13:44 |
jaosorior | but it seems that a dependency is missing | 13:44 |
openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Validation to verify the undercloud CPU reqs https://review.openstack.org/351196 | 13:45 |
*** jpich has joined #tripleo | 13:45 | |
EmilienM | looks like cachetools is missing | 13:45 |
jaosorior | yep | 13:45 |
jaosorior | strangely | 13:45 |
EmilienM | I think I found why | 13:46 |
*** jraju has quit IRC | 13:46 | |
EmilienM | in the logs, cachetools is updated at 12:40:16 but dbsync executed at 12:39:38.000 | 13:46 |
EmilienM | so I guess we have an orchestration issue here | 13:46 |
openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Validation to verify the undercloud RAM reqs https://review.openstack.org/351179 | 13:46 |
EmilienM | ok it's a bug in https://github.com/rdo-packages/mistral-distgit/blob/rpm-master/openstack-mistral.spec | 13:47 |
*** zoli|lunch is now known as zoli|wfh | 13:50 | |
*** pgadiya has quit IRC | 13:51 | |
EmilienM | ok the mistral upgrade problem is in packaging, I'll solve it here http://review.rdoproject.org/r/1793 | 13:53 |
*** dculik has joined #tripleo | 13:58 | |
shardy | Tripleo Deep dive starting now | 14:00 |
shardy | https://etherpad.openstack.org/p/tripleo-deep-dive-topics | 14:00 |
*** milan has quit IRC | 14:01 | |
*** milan has joined #tripleo | 14:01 | |
openstackgerrit | Brad P. Crochet proposed openstack/tripleo-heat-templates: Composable Zaqar services https://review.openstack.org/331682 | 14:01 |
openstackgerrit | Brad P. Crochet proposed openstack-infra/tripleo-ci: Use tripleo-build-images for CI https://review.openstack.org/336312 | 14:06 |
*** pradk has joined #tripleo | 14:06 | |
openstackgerrit | Brad P. Crochet proposed openstack/tripleo-docs: Switch docs to use tripleo-build-images https://review.openstack.org/337669 | 14:06 |
*** coolsvap has quit IRC | 14:11 | |
*** apetrich has quit IRC | 14:12 | |
*** ebalduf has joined #tripleo | 14:13 | |
*** karthiks has quit IRC | 14:13 | |
*** dtantsur|mtg is now known as dtantsur | 14:14 | |
*** mgould|mtg is now known as mgould | 14:15 | |
*** jaosorior has quit IRC | 14:17 | |
*** apetrich has joined #tripleo | 14:20 | |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds parameters actions https://review.openstack.org/298682 | 14:21 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow https://review.openstack.org/298732 | 14:21 |
openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans https://review.openstack.org/351111 | 14:21 |
d0ugal | shardy: ^ tidied up a bit | 14:22 |
ccamacho | michele the deployment doesn't fail, but after the deployment http://paste.openstack.org/show/549239/ | 14:26 |
ccamacho | s/michele/bandini/ | 14:27 |
ccamacho | bandini^ | 14:27 |
*** karthiks has joined #tripleo | 14:30 | |
*** Goneri has quit IRC | 14:35 | |
*** dbecker has quit IRC | 14:38 | |
openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Validation to verify undercloud disk size reqs https://review.openstack.org/351224 | 14:38 |
hewbrocca | EmilienM: very nice... | 14:39 |
*** jcoufal has joined #tripleo | 14:45 | |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-common: Prevent replacement of Port, Net and Subnet https://review.openstack.org/308381 | 14:45 |
jistr | after a few months i finally had some bandwidth to revisit and test this ^^ :D | 14:47 |
*** dculik has quit IRC | 14:47 | |
hewbrocca | woowoo! | 14:47 |
hewbrocca | jistr: back on trunk | 14:47 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 14:48 |
pradk | ccamacho, the endpoint is really http://192.0.2.16:8041/v2/alarms ? | 14:49 |
pradk | ccamacho, i wonder if the bind_host is not set correctly | 14:49 |
bnemec | Sigh. The random bluejeans mute thing has been biting me a bunch lately. | 14:49 |
*** oshvartz has quit IRC | 14:49 | |
pradk | ccamacho, whats the host in aodh.conf? is it still 0.0.0.0 ? | 14:50 |
ccamacho | pradk :facepalm: for me.. Im running another test and I was testing against the undercloud.... So just a sec | 14:50 |
pradk | not sure if bandini fixed the host issue | 14:50 |
pradk | ccamacho, so i see you removed all the pacemaker files from https://review.openstack.org/#/c/332854/ .. we dont need them anymore? | 14:52 |
pradk | i thought we havent fully made the switch to lightweight arch | 14:52 |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-common: Prevent replacement of Port, Net and Subnet https://review.openstack.org/308381 | 14:53 |
ansiwen | beaker tests can't be run with libvirt provider? seems there are only vagrant boxes for virtualbox and vmware | 14:53 |
*** dtrainor has quit IRC | 14:54 | |
*** dtrainor has joined #tripleo | 14:55 | |
*** beagles is now known as beagles_brb | 14:56 | |
*** fpan has quit IRC | 14:59 | |
bnemec | So rh1 was hosed this morning. | 14:59 |
bnemec | It looks like the db ran out of file handles, which caused basically everything to go down. | 15:00 |
*** dculik has joined #tripleo | 15:00 | |
bnemec | I've cleaned up the leftover heat stacks so nodepool actually has quota again. | 15:00 |
bnemec | It looks like we have some configuration tuning on the controller to do though. | 15:01 |
bnemec | I'm going to keep an eye on it today. | 15:01 |
pradk | bandini, EmilienM, , regarding apache issue wonder if it is because we're explicitly including aodh::wsgi::apache which is automatically loading apache via pacemaker? | 15:01 |
bandini | ccamacho, pradk: nope I had not submitted the host fix yet, will do so shortly. | 15:02 |
d0ugal | rbrady, shardy: http://paste.openstack.org/show/549245/ - any idea how that could be possible? | 15:02 |
pradk | that would explain the 404 | 15:02 |
bandini | pradk: the reason for removing the pacemaker stuff, was because we weere thinking it is more efficient to first land HA next generation and then aodh and just skip the issue | 15:02 |
d0ugal | rbrady, shardy: the line is different because i'm in a dev version - but it is here: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L436 | 15:03 |
pradk | bandini, ah ok.. so do we have to do anything explicit for aodh services to work on their own? | 15:03 |
pradk | bandini, i.e via systemd | 15:04 |
d0ugal | rbrady, shardy: and here: https://github.com/openstack/os-cloud-config/blob/master/os_cloud_config/keystone.py#L157 | 15:04 |
bandini | pradk: nope, I don't think so. but am testing aodh specifically in my next ha next generation test run (am redeploying now) | 15:04 |
pradk | cool | 15:05 |
ccamacho | pradk bandini Im running the deployment also again but taking ages.. | 15:05 |
pradk | :) | 15:05 |
bandini | ccamacho: I have added a comment what needs to be fixed for the aodh host issue. best if you just address it in your next iteration? | 15:06 |
*** numans has quit IRC | 15:06 | |
pradk | bandini, actually i already fixed the host issue | 15:06 |
pradk | just remembered .. see the right of the patch | 15:06 |
pradk | https://review.openstack.org/#/c/333556/43/puppet/controller.yaml | 15:06 |
pradk | bandini, thats what we want correct? | 15:07 |
*** akshai has joined #tripleo | 15:07 | |
shardy | d0ugal: is there any chance you're running an old os_cloud_config? | 15:07 |
shardy | https://github.com/openstack/os-cloud-config/commit/6cf5d63527b8dd628b9634df2769daca82d35e19 | 15:07 |
bandini | pradk: ah ops, right. I am blind. ccamacho ignore my comment | 15:08 |
bandini | then we need to check what else is wonky still | 15:08 |
*** apetrich has quit IRC | 15:09 | |
*** rcernin has quit IRC | 15:09 | |
pradk | bandini, i'll do a new deploy now on top of your LW patches | 15:09 |
d0ugal | shardy: damn, I am. Good spot | 15:10 |
d0ugal | shardy: Looks like that version isn't released yet. | 15:10 |
bandini | pradk: ack. the LW stuff is currently deploying correctly. I need to just smoketest vm creation | 15:10 |
shardy | d0ugal: it's been released in both newton milestones | 15:11 |
d0ugal | shardy: but not to PyPI I guess :) | 15:11 |
d0ugal | oh, it had | 15:12 |
d0ugal | ah, I see - pip wont install a beta release | 15:12 |
*** akshai has quit IRC | 15:12 | |
d0ugal | (by default) | 15:12 |
pradk | bandini, ccamacho, wonder if this is the issue .. https://review.openstack.org/#/c/333556/43/puppet/services/aodh-api.yaml see line 31 | 15:14 |
pradk | its set to {get_param: AodhApiVirtualIP} but AodhApiVirtualIP defaults to ' ' few lines above | 15:14 |
pradk | thats explains my bind_host not configured correctly theory from above? | 15:15 |
dtrainor | can anyone tell me a little more about what happens behind the scenes with 'openstack baremetal configure boot'? The docs say "Assign the kernel and ramdisk images to all nodes". I'd like to know where that linkage is set etc. | 15:15 |
ccamacho | pradk If not defined yes will be '', can be, but remember that before we were running the pcm templates and that param were not defined, taking notes for when my deployment finish | 15:16 |
*** ayoung has joined #tripleo | 15:17 | |
pradk | ccamacho, shouldnt the pcmk inherit from base? | 15:17 |
*** leanderthal is now known as leanderthal|afk | 15:17 | |
d0ugal | dtrainor: it is actually quite simple, it just sets a few properties on the nodes: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/baremetal.py#L484 | 15:17 |
pradk | ccamacho, if not set in pcmk though that would actually be an issue to fix | 15:17 |
pradk | ccamacho, we need to tell apache what to bind on | 15:17 |
pradk | aodh::wsgi::apache::bind_host: would be required | 15:18 |
*** devvesa has quit IRC | 15:18 | |
dtrainor | oh, capabilities. | 15:18 |
dtrainor | excellent. thank you. | 15:18 |
d0ugal | dtrainor: np, hopefully that helps. I've not actually looked at the code in some time | 15:18 |
d0ugal | dtrainor: You can blame me for the original version :) | 15:19 |
openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui: Add packaging spec and config files https://review.openstack.org/344932 | 15:19 |
dtrainor | haha. it makes sense, yes. | 15:19 |
pradk | ccamacho, interestingly we dont set that for other services | 15:21 |
*** ebarrera has quit IRC | 15:21 | |
pradk | nor its set in mitaka.. i'll remove that and redeploy | 15:22 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Add Aodh composable roles https://review.openstack.org/333556 | 15:22 |
openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Validation to check the undercloud processes https://review.openstack.org/351248 | 15:25 |
*** rwsu has joined #tripleo | 15:26 | |
*** mcornea has quit IRC | 15:26 | |
Slower | anyone else try building undercloud recently? | 15:29 |
EmilienM | Slower: yes, a few min ago | 15:29 |
Slower | I'm getting mysql puppet errors when using pkgs and some repo error when using src | 15:29 |
EmilienM | yes | 15:29 |
EmilienM | there is a problem with mistral | 15:29 |
Slower | ah ok | 15:29 |
EmilienM | what is your trace? | 15:29 |
EmilienM | is it mistral db sync? | 15:29 |
d0ugal | seems like Mistral breaks us all the time :/ | 15:30 |
d0ugal | Is it our fault, or theirs? | 15:30 |
Slower | EmilienM: https://paste.fedoraproject.org/401279/14703246/ | 15:30 |
EmilienM | we need to improve their CI | 15:30 |
d0ugal | ok | 15:30 |
d0ugal | EmilienM: What is missing? | 15:30 |
Slower | no different sounds like | 15:30 |
EmilienM | d0ugal: ok not the same thing | 15:30 |
d0ugal | (or what needs to be improved) | 15:30 |
EmilienM | err, Slower ^ | 15:30 |
EmilienM | d0ugal: tripleo jobs ;-) | 15:30 |
EmilienM | Slower: sorry something I didn't hit | 15:31 |
Slower | EmilienM: or I can do puppet from source and I get something else | 15:31 |
openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart: Run dstat on the virthost and the undercloud https://review.openstack.org/349591 | 15:31 |
Slower | ah there's a cache.. lemme remoev that | 15:31 |
*** bana_k has joined #tripleo | 15:32 | |
Slower | EmilienM: from puppet src I get: https://paste.fedoraproject.org/401283/03247661/ | 15:33 |
Slower | EmilienM: any thoughts on how to get around one of those? :) | 15:33 |
Slower | I'd really like to be able to do work again | 15:33 |
EmilienM | Slower: sounds like a git issue | 15:33 |
Slower | yeah | 15:33 |
Slower | which is odd | 15:33 |
EmilienM | failed to get sha1 of fetch_master | 15:33 |
EmilienM | I was able to deploy an undercloud like 1h ago | 15:34 |
Slower | hrrm | 15:34 |
Slower | I was seeing this last night too | 15:34 |
Slower | EmilienM: you use puppet src I presume? | 15:34 |
EmilienM | Slower: now, I use packaging | 15:36 |
EmilienM | and I do my modifications manually | 15:36 |
EmilienM | weird | 15:36 |
EmilienM | can you check if mysql is running? | 15:36 |
Slower | EmilienM: it's not even installed.. | 15:37 |
openstackgerrit | Steven Hardy proposed openstack/instack-undercloud: Work around deletion of _member_ role assignments on upgrade https://review.openstack.org/307352 | 15:38 |
Slower | EmilienM: didn't know that was a requirement actually | 15:38 |
EmilienM | Slower: mysql is deployed by puppet | 15:40 |
EmilienM | you don't have anything to deploy yourself | 15:40 |
EmilienM | can someone review https://review.openstack.org/#/c/350763/ to Add undercloud job in cistatus ? | 15:42 |
Slower | EmilienM: I'm just trying to build the undercloud image | 15:42 |
Slower | eg openstack undercloud install | 15:43 |
d0ugal | shardy: Okay, with a updated os-cloud-config, a deploy with no errors! | 15:44 |
shardy | d0ugal: great, nice work! :) | 15:45 |
*** beagles_brb is now known as beagles | 15:45 | |
d0ugal | shardy: Now I think I need to integrate jpich's patch: https://review.openstack.org/#/c/347830/ | 15:45 |
d0ugal | shardy: So that users can specify their own templates, and don't need to pre-create a plan. | 15:45 |
*** dculik has quit IRC | 15:46 | |
d0ugal | rbrady: ^ update | 15:46 |
shardy | d0ugal: sounds good | 15:46 |
shardy | d0ugal: if you go ahead and do that, I'll pull everything tomorrow morning and test it | 15:46 |
shardy | hopefully we can start landing things | 15:46 |
d0ugal | shardy: Great, I'm not convinced I'll have that added by the end of today, but early tomorrow should be easy. | 15:47 |
d0ugal | shardy: +1, it would be good to start landing the workflows so we can then have CI test the CLI and iron out all the remaining bits | 15:47 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: remove overcloud roles from oooq https://review.openstack.org/351259 | 15:48 |
jpich | d0ugal: You could upload stuff to Swift directly and call the create_plan action, if creating the JSON is too much? I didn't add a client action for that because I imagine folks will upload templates as files directly with jsonifying, but maybe there's a need for another workflow for that too...? | 15:48 |
d0ugal | jpich: oh, is that how the create_plan workflow works. | 15:48 |
d0ugal | jpich: Maybe I should do that? What is the use-case for your patch if we are just uploading to swift directly? | 15:49 |
jpich | d0ugal: IIRC it takes a container and plan-ifyise it yeah :-) | 15:49 |
jpich | d0ugal: Have a proper workflow to do this, I think uploading files as JSON was more relevant to the GUI directly | 15:49 |
jpich | but from a CLI perspective I can't see it making as much sense | 15:50 |
*** yamahata has joined #tripleo | 15:50 | |
d0ugal | jpich: hmm, okay. I'll try uploading directly. In some ways I'd rather us all using the same workflows... but I'm happy to not depend on another patch | 15:50 |
jpich | d0ugal: which itself depends on another patch... ;) | 15:50 |
bandini | EmilienM, ccamacho, gfidente: I smoketested the last iteration of review 314208 and it looks okay. So I am okay with landing this now (we can land aodh on top afterwards) | 15:50 |
bandini | let's wait for CI in any case | 15:51 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Set Coordination url for central agent https://review.openstack.org/350702 | 15:52 |
gfidente | bandini, yep now it looks good :) | 15:52 |
*** pkovar has quit IRC | 15:52 | |
gfidente | I'd say the small reorg worked out | 15:52 |
gfidente | we'll see if it'll also pay or not | 15:52 |
bandini | gfidente: yes I think it was worth it. thanks again for pushing me in that direction ;) | 15:53 |
gfidente | (we had the full ha tested and are saying goodbye in style) :) | 15:53 |
rbrady | jtomasek, d0ugal, jpich: "but from a CLI perspective I can't see it making as much sense". I remember a time not too long ago we worried about cli divergence. now it seems the GUI is diverging. | 15:53 |
gfidente | bandini, honestly with marios talking bad about me all the times | 15:53 |
jpich | rbrady: The workflow is there in tripleo-common, for the client to use if we want it? | 15:54 |
bandini | gfidente: well marios is always right, so surely you did something evil | 15:54 |
gfidente | yeah | 15:54 |
gfidente | but it's not in tripleo | 15:55 |
bandini | ahahah right | 15:55 |
bandini | gfidente, marios: I'd like that we agree on "What happens in Barcelona, stays in Barcelona", just in case | 15:55 |
gfidente | I don't know, slagle is taking picutes | 15:56 |
slagle | only b/c i thought a felony was about to occur | 15:57 |
d0ugal | rbrady: Indeed, but we are getting closer. Had you imagined the GUI would upload to swift directly? | 15:57 |
gfidente | slagle, I know I can count on you | 15:57 |
dprince | d0ugal: I had, yes | 15:57 |
d0ugal | dprince: :) | 15:57 |
* bandini chuckles | 15:58 | |
rbrady | d0ugal: yeah..as previously discussed | 15:58 |
rbrady | jpich: I'm not sure I understand what you're asking | 15:58 |
d0ugal | dprince, rbrady: I don't know if there is a reason they can't do that? jtomasek jpich} | 15:58 |
d0ugal | s/}/?/ | 15:58 |
* jtomasek reads back | 15:59 | |
marios | bandini: honestly, i don't know what to do i think i have to report gfidente to hr for the greater good | 15:59 |
marios | bandini: i think it is the right thing to do for all concerned | 15:59 |
marios | bandini: is anj going? | 15:59 |
jtomasek | rbrady: where is that quote from? | 16:00 |
rbrady | jtomasek: scroll up a bit. it's from jpich | 16:00 |
d0ugal | jtomasek: a bit further up - jpich said it :) | 16:00 |
bandini | marios: ANJ says hello, as usual ;) | 16:00 |
jpich | rbrady: I actually wasn't around for the conversation around concerns related to CLI divergence - I assume it means code that lived in the CLI only? This one is in tripleo-common so it can be used by both CLI and GUI if wanted, but maybe your concern is different? | 16:00 |
*** liverpooler has joined #tripleo | 16:00 | |
marios | bandini: y u so mean | 16:01 |
dprince | jpich: simply putting code into tripleo-common isn't enough I think | 16:01 |
bandini | lol | 16:01 |
gfidente | I think I'm joining this marios is always right bus | 16:01 |
rbrady | jpich: I think it's important we attempt to have both UI's utilize the same underlying code | 16:01 |
rbrady | jpich: for CI and for consistency | 16:02 |
dprince | jpich: using in both places would be important. | 16:02 |
gfidente | we need jistr to say that as well though | 16:02 |
dprince | jpich: and if we can't get the CLI to use it in Newton, then I would argue the UI needs to try to adapt to do it the same way | 16:02 |
*** zoli|wfh is now known as zoli_gone-proxy | 16:02 | |
dprince | jpich: parity | 16:02 |
jtomasek | yeah, I agree, that both CLI and GUI should use most of the same workflows. In cases where the 2 diverge, we could simply have an overlapping workflows | 16:03 |
*** cwolferh has joined #tripleo | 16:03 | |
d0ugal | Alright, so I should use the JSON plan uploader? | 16:04 |
jpich | rbrady, dprince: I understand the idea behind it, and I agree with the principle. In this particular case it seems to me there is a way to do things (uploading files) that generally works differently depending on whether you're using a CLI to upload files or a web interface, and it's the more natural way to do it | 16:04 |
openstackgerrit | Merged openstack/tripleo-docs: Group advanced topics by type https://review.openstack.org/335232 | 16:05 |
EmilienM | bandini: +1 | 16:05 |
jpich | but I don't have extremely strong feelings toward file uploading, whatever people would prefer we use for creating a new plan from templates I'm happy to help implement | 16:05 |
jtomasek | jpich: so the problem is about CLI should not have to parse the files and send contents to a workflow as json? | 16:06 |
EmilienM | bnemec, slagle: you aware about this CI issue? http://logs.openstack.org/95/346995/31/experimental/gate-tripleo-ci-centos-7-nonha-multinode-upgrades-nv/6c6ff06/console.html#_2016-08-04_15_46_21_379809 | 16:06 |
*** apetrich has joined #tripleo | 16:06 | |
EmilienM | it looks like all CI is red | 16:06 |
bnemec | EmilienM: Yeah, I'm looking into it. There was a problem with the rh1 cloud this morning that broke everything, and it still looks like jobs aren't getting testenvs properly. | 16:07 |
jpich | jtomasek: I suppose we could do it that way! Is that how the UI does it too? Means we have to add additional parsing code in Python only to the CLI command, but maybe it makes sense for that one | 16:07 |
bnemec | Although the current failures may be teething problems from bringing the cloud back from the dead because I see jobs running right now. | 16:08 |
jtomasek | jpich: yeah, we'll have to parse the files on the client side (GUI) - only way to get them to mistral | 16:09 |
jtomasek | jpich, d0ugal: regarding adding files to plan, you use just the action you (d0ugal) created patch for recently | 16:10 |
d0ugal | jtomasek: What parsing do you do? | 16:10 |
jpich | jtomasek: Alright, I'll add a task to do it that way for the client as well. Thanks for the extra-background :-) | 16:10 |
d0ugal | jtomasek: There is no chance I am going to add a full plan one file at a time :) | 16:11 |
marios | gfidente: ymmv | 16:11 |
d0ugal | oh, actually, it accepts multiple files | 16:11 |
jpich | d0ugal: File objects to json strings, I think | 16:11 |
jtomasek | d0ugal: why one? | 16:11 |
jtomasek | yeah | 16:11 |
d0ugal | jpich: we almost wrote the same thing, lol | 16:11 |
* d0ugal face palms | 16:11 | |
jpich | d0ugal: Wait, what? Did I duplicate a patch? | 16:11 |
jtomasek | create plan workflow is a workflow because it does 2 things - puts files into swift and if it succeeds, creates plan | 16:12 |
d0ugal | jpich: Nah, it was me | 16:12 |
mandre | shardy: what was name of the python module you used to debug the endpoints in the tripleo deep dive earlier again? | 16:12 |
*** ohamada has quit IRC | 16:12 | |
jtomasek | to add files to plan you only need to do single action -> add files to swift | 16:12 |
d0ugal | jpich: I just wrote the action, it is very similar to yours. | 16:13 |
dprince | jpich: in each case I'd like to consider using just a tarball | 16:13 |
d0ugal | jpich: https://review.openstack.org/#/c/351111/ | 16:13 |
shardy | mandre: entry_point_inspector | 16:13 |
d0ugal | dprince: Can the GUI create tarballs? | 16:13 |
mandre | shardy: thx :) | 16:13 |
jtomasek | d0ugal: user provides a tarbal - that is how GUI currently gets the files | 16:13 |
d0ugal | jtomasek: oh, cool. | 16:14 |
jtomasek | d0ugal: but mistral is not able to accept file | 16:14 |
d0ugal | right | 16:14 |
dprince | d0ugal: I don't see why not | 16:14 |
d0ugal | it is all coming back to me now | 16:14 |
dprince | http://git.openstack.org/cgit/openstack/tripleo-common/tree/scripts/upload-swift-artifacts | 16:14 |
dprince | http://git.openstack.org/cgit/openstack/tripleo-common/tree/scripts/upload-puppet-modules | 16:14 |
dprince | jtomasek: mistral doesn't need a file | 16:14 |
shardy | dprince: I think the debate is does it go via swift or direct to mistral | 16:14 |
dprince | jtomasek: it needs to location of the file you uploaded elsewhere | 16:14 |
shardy | personally I'd be fine if we just always put the data directly into swift | 16:15 |
dprince | jtomasek: whoever said Mistral needs to do this? | 16:15 |
jtomasek | dprince: yes and that is exactly the problem | 16:15 |
d0ugal | shardy: +1, I think that was the original plan - it has just gotten a bit confused. | 16:15 |
jtomasek | dprince: the operation of creating a plan is 2 steps -> workflow | 16:15 |
dprince | jtomasek: upload the templates | 16:15 |
dprince | jtomasek: then create the plan | 16:15 |
dprince | I have no issue with that | 16:15 |
d0ugal | Seems fine to me. | 16:16 |
jtomasek | dprince: in this case we make client (GUI) handle the workflow -> errors such as a case when swift upload fails | 16:16 |
dprince | jtomasek: if it fails you'll get back an HTTP error. What could possibly be better than that? | 16:16 |
shardy | dprince: https://review.openstack.org/#/c/347830/ is what sparked the debate I think | 16:16 |
*** dtantsur is now known as dtantsur|afk | 16:16 | |
shardy | given the issues with messing with file limits there, I'd much prefer to just push the data to swift | 16:17 |
mandre | shardy: thanks, this came in really handy | 16:17 |
shardy | it can already handle it nicely | 16:17 |
jtomasek | this problem was actually the initial thing that lead to jpich and florianf looking into refactoring the plan creation workflow | 16:17 |
*** tremble has quit IRC | 16:18 | |
d0ugal | I think we have bigger issues to resolve. I'd vote for uploading to swift and then triggering a workflow. | 16:18 |
jtomasek | shardy, dprince: so you say that client needs to know that files are stored in swift and handle the 'workflow' of creating the plan by itself | 16:18 |
jtomasek | 1. upload tarball to swift and 2. run create_plan action | 16:18 |
openstackgerrit | Merged openstack/instack-undercloud: Use ensure_resource instead of package https://review.openstack.org/351148 | 16:19 |
shardy | jtomasek: yes | 16:19 |
shardy | I think that's easier than having to parse a bunch of files and format a json blob | 16:19 |
shardy | do you disagree? | 16:19 |
openstackgerrit | Merged openstack/instack-undercloud: Remove discover from test-requirements https://review.openstack.org/345832 | 16:19 |
d0ugal | Having the client know is kinda nice, maybe one day the workflow could support pulling from git | 16:19 |
d0ugal | (from git or various other places that might be useful) | 16:19 |
shardy | d0ugal: Yeah, that would be cool, and it could support an existing (non swift) http url | 16:20 |
jtomasek | shardy: optimal solution would be to let workflow tool do the workflow stuff (mistral) and at the same time make the mistral accept files | 16:20 |
shardy | but we're preoptimizing, lets get the deployment to work then refactor for adding interfaces :) | 16:20 |
d0ugal | +1 | 16:20 |
d0ugal | jtomasek: Okay, get Mistral to accept files and we can move to that ;) | 16:20 |
jtomasek | d0ugal: hm | 16:20 |
florianf | shardy, dprince, jtomasek: I should mention that it's not really just uploading to swift. There is some metadata that needs to be added to the container so tripleo-common can recognize it as a plan container. | 16:20 |
dprince | jtomasek: yes that is fine I think. Never parse the files. The whole point of using the Swift bulk middleware was to get us out of the mess of per-file management | 16:20 |
*** weshay is now known as weshay_food | 16:21 | |
dprince | florianf: get rid of the metadata | 16:21 |
shardy | florianf: there is an action that can set that metadata AIUI | 16:21 |
dprince | florianf: I didn't need it to deploy with the CLI https://github.com/dprince/python-tripleoclient/commit/77ffd2fa7b1642b9f05713ca30b8a27ec4b322b7 | 16:21 |
dprince | florianf: which is arguable a simpler case, but store the data in the Mistral environment would be better | 16:22 |
dprince | storing | 16:22 |
ccamacho | hey, today is 2²/2³/2⁴ | 16:22 |
jtomasek | d0ugal: "Having the client know is kinda nice, maybe one day the workflow could support pulling from git" I don't follow | 16:22 |
jtomasek | jtomasek: if the backend workflow was the one who interacted with git, that would be better, wouldn't it? | 16:23 |
gfidente | ccamacho, 0.03125 | 16:23 |
d0ugal | jtomasek: Just a future thought, so don't worry about it. However, at the moment the workflow knows how to get it from swift, we could add more options later (like git) | 16:23 |
dprince | d0ugal: git isn't openstack | 16:24 |
dprince | d0ugal: git -> tarball -> swift is How I'd recommend you do that | 16:24 |
jtomasek | d0ugal: yeah, optimal would be if workflow knew how to get files to swift and from it. but it doesnt | 16:24 |
d0ugal | jtomasek: So having the client and the user know where the files live isn't a bad thing particularly | 16:24 |
d0ugal | dprince: I didn't say it was, git was just an example of another place the files could live. | 16:24 |
d0ugal | but we are getting way off point | 16:24 |
jtomasek | ok | 16:24 |
dprince | d0ugal: the sources can be anywhere I think | 16:25 |
d0ugal | dprince: That's my point. | 16:25 |
dprince | d0ugal: the format we accept is tarball | 16:25 |
*** maeca1 has joined #tripleo | 16:25 | |
d0ugal | Okay | 16:25 |
jtomasek | dprince: ok, makes sense | 16:25 |
d0ugal | So I am going to have the CLI upload a tarball directly. | 16:25 |
dprince | d0ugal: think of it like DIB creating a qcow. DIB sources are stored in Git but they generate a qcow | 16:25 |
d0ugal | Thanks all | 16:25 |
d0ugal | dprince: Sure, I get it. | 16:25 |
ccamacho | gfidente, indeed if you assume left to right associativity :P | 16:26 |
dprince | d0ugal: like 234 here works great FWIW: https://github.com/dprince/python-tripleoclient/commit/77ffd2fa7b1642b9f05713ca30b8a27ec4b322b7 | 16:26 |
dprince | line, not like | 16:26 |
jpich | jtomasek, d0ugal, dprince: So, I should abandon the patches related to JSON file uploads then, is that correct? | 16:29 |
jtomasek | jpich: seems so, basically we agreed to keep what we currently do in GUI | 16:29 |
jtomasek | so the necessary workflows are in place | 16:29 |
jtomasek | d0ugal: so how about that addition of environments? how are you going to handle it? | 16:30 |
jtomasek | d0ugal: upload new files to plan in swift and then trigger an update_capabilities action? | 16:30 |
jtomasek | (to enable them) | 16:30 |
jpich | All right then | 16:30 |
jpich | florianf ^ - I'll update the blueprint | 16:30 |
*** kro_focused is now known as krotscheck | 16:32 | |
rbrady | jtomasek: will additional environment data only be file-based or do you envision text as well? | 16:33 |
d0ugal | jtomasek: Not sure yet. I have that patch that works, just need to figure out the best way now. | 16:33 |
florianf | jtomasek, jpich: We can't keep what we currently do in the GUI if the tarball is the only accepted format | 16:33 |
d0ugal | jtomasek: I don't think there was a plan for extra environments files like the CLI requires. | 16:34 |
jtomasek | rbrady: file-based only | 16:34 |
d0ugal | florianf: What does the GUI do now? | 16:34 |
ccamacho | hey pradk quick question | 16:34 |
jpich | florianf: I think maybe the idea becomes "you do whatever you want to get the data to the Swift container" | 16:34 |
florianf | jtomasek, jpich, d0ugal : We currently allow both tarballs and a JSON structure with filepath => content | 16:34 |
jtomasek | florianf: we can do both afaik | 16:35 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Add Aodh composable roles https://review.openstack.org/333556 | 16:35 |
florianf | jpich: yeah, we're going to have to rewrite that so all files are added to swift | 16:35 |
florianf | jtomasek: yes, we can. but we're going to have to rewrite that. and it's going to be quite a lot of swift calls :-) | 16:36 |
pradk | ccamacho, bandini, i think this should fix the apache host issue https://review.openstack.org/#/c/333556/44..45/puppet/controller.yaml | 16:36 |
*** oshvartz has joined #tripleo | 16:36 | |
pradk | we keep that for other services | 16:36 |
d0ugal | jtomasek, florianf: please don't upload to swift one file at a time. | 16:36 |
pradk | in controller.yaml | 16:36 |
d0ugal | the reason we used a tarball originally was to avoid that. | 16:36 |
ccamacho | the retry params for gnocchi they dont exist in puppet-gnocchi openstack/gnocchi or in the config files once the service its deployed. I did something to add the params in glance but they where in different manifest https://review.openstack.org/#/c/349463/ | 16:36 |
jtomasek | florianf: we can't bundle the files into single json to send it to swift? | 16:36 |
ccamacho | pradk ^ | 16:36 |
jtomasek | d0ugal: haha | 16:37 |
pradk | ccamacho, oh so we dont use oslo::db i guess in this case | 16:37 |
*** Ryjedo has joined #tripleo | 16:37 | |
florianf | jtomasek: Not sure if swift can create multiple objects from a single json structure. I'd have to check... | 16:38 |
ccamacho | pradk, I think | 16:39 |
pradk | ccamacho, we actaully do use oslo db.. i guess puppet module needs to be updated | 16:39 |
*** lucasagomes is now known as lucas|dinner | 16:39 | |
pradk | i'll submit a patch today | 16:39 |
*** myoung|remote has quit IRC | 16:40 | |
ccamacho | You can use this as reference https://review.openstack.org/#/c/349463/ | 16:40 |
pradk | we support both max retries and interval | 16:40 |
jpich | florianf: I'd be surprised but! | 16:40 |
d0ugal | jpich: I would too, but I was surprised at the tarball extracting feature... | 16:40 |
jpich | :-) | 16:41 |
*** weshay_food is now known as weshay | 16:43 | |
*** florianf is now known as florianf|biab | 16:44 | |
pabelanger | slagle: EmilienM: panda: bnemec: here's the first pass at adding an experimental jobs to test overcloud-full.qcow2 builds. https://review.openstack.org/#/c/351278/ | 16:46 |
pabelanger | slagle: EmilienM: panda: bnemec: Tested it locally, diskimage-builder will build the image, but eventually fails becasue it is actually broken atm | 16:47 |
*** david-lyle has quit IRC | 16:53 | |
*** david-lyle has joined #tripleo | 16:53 | |
*** tosky has quit IRC | 16:55 | |
*** jtomasek has quit IRC | 16:55 | |
*** jpich has quit IRC | 16:57 | |
*** mgould is now known as mgould|afk | 16:57 | |
*** tzumainn has quit IRC | 16:59 | |
slagle | bandini: hey, i'm getting this error trying to start corosync: http://paste.openstack.org/show/549272/ | 17:02 |
slagle | bandini: from what i can tell it's b/c of the hostname in corosync.conf: | 17:03 |
*** flaper87 has quit IRC | 17:03 | |
openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 17:03 |
slagle | bandini: http://paste.openstack.org/show/549273/ | 17:03 |
slagle | if i add .localdomain to those hostnames, it starts fine | 17:03 |
slagle | how does corosync attempt to resolve the hostname? | 17:03 |
slagle | b/c both ways are already in /etc/hosts | 17:04 |
slagle | i see what the problem is, something else has added 127.0.1.1 slagle-multinode-ha-4 to /etc/hosts | 17:07 |
slagle | what would be doing that | 17:07 |
slagle | devstack-gate of course | 17:10 |
*** tzumainn has joined #tripleo | 17:10 | |
openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 17:10 |
pabelanger | slagle: I've replied to your comments | 17:12 |
slagle | pabelanger: that was bnemec | 17:14 |
pabelanger | slagle: Ah, right. must get more coffee | 17:15 |
slagle | but i kinda agree with his point | 17:15 |
slagle | we document building images a different way | 17:16 |
pabelanger | how are they built? | 17:16 |
slagle | and I guess this job is just testing that an image *can* be built? | 17:16 |
pabelanger | excaltly | 17:16 |
pabelanger | exactly* | 17:16 |
pabelanger | because, today they can't be built | 17:16 |
slagle | probably b/c that's not the way we document to do it | 17:17 |
pabelanger | where are the docs so I can see how it should be done | 17:17 |
slagle | we delorean rpms | 17:17 |
slagle | *use | 17:17 |
slagle | pabelanger: http://docs.openstack.org/developer/tripleo-docs/basic_deployment/basic_deployment_cli.html#get-images | 17:18 |
slagle | that assumes you're on an undercloud with the repos setup and the packages installed | 17:18 |
slagle | which is doc'd earlier | 17:18 |
slagle | pabelanger: how is it failing for you? | 17:19 |
pabelanger | looks like systemd | 17:19 |
pabelanger | is failing: http://paste.openstack.org/show/549275/ | 17:19 |
slagle | weird, os-collect-config must have not gotten fully installed | 17:20 |
pabelanger | openstack overcloud image build appears to be the command? | 17:20 |
slagle | yes, it's the same command you have in the job | 17:21 |
pabelanger | http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/build_single_image.html#building-a-single-image is what I am trying to do | 17:21 |
pabelanger | from centos-7 | 17:21 |
pabelanger | which is basically, what the job does | 17:21 |
*** bfournie has quit IRC | 17:21 | |
slagle | should that be ELEMENT_PATH? or ELEMENTS? | 17:22 |
*** dbecker has joined #tripleo | 17:22 | |
slagle | oh i c | 17:22 |
slagle | i'd need to go look at this tox env | 17:22 |
slagle | maybe the virtualenv is confusing things? | 17:23 |
pabelanger | I don't think so, it fails outside virtualenv too | 17:23 |
pabelanger | I suspect something in systemd is broken | 17:23 |
bandini | slagle: am back. reading now | 17:23 |
bandini | slagle: ah you solved it already? | 17:24 |
*** jcoufal_ has joined #tripleo | 17:24 | |
slagle | bandini: yea, nm. it was a bogus IP in /etc/hosts | 17:24 |
bandini | slagle: ah ack | 17:24 |
openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs https://review.openstack.org/343017 | 17:24 |
*** dprince has quit IRC | 17:24 | |
bandini | slagle: for the record corosync uses getaddrinfo() to resolve names | 17:25 |
slagle | pabelanger: if you export break=after-error before you run the command, dib should drop you to a shell after an error | 17:26 |
*** jcoufal has quit IRC | 17:26 | |
slagle | i've never used that though when calling dib via tripleoclient, so not 100% sure it will work | 17:26 |
*** abehl has quit IRC | 17:27 | |
pabelanger | slagle: right, I haven't started debugging it yet. I wanted to land the experimental job first, then fix it and upload to gerrit to run the experimental job | 17:27 |
pabelanger | slagle: ya, I've had to hack in DIB_DEBUG_TRACE into python-tripleo-client to expose move debug into | 17:28 |
pabelanger | info* | 17:28 |
*** myoung has quit IRC | 17:28 | |
EmilienM | pabelanger: awesome | 17:34 |
*** Tristitia has quit IRC | 17:34 | |
slagle | pabelanger: ok, i like where you're going with the job. but i'm not sure about building images this way | 17:34 |
slagle | i guess there might be some utility in fixing that | 17:35 |
pabelanger | sure, how would you like the images to be build? | 17:35 |
slagle | how we do in tripleo-ci | 17:35 |
slagle | i think if you enabled the delorean repos, could use tripleo.sh --repo-setup for this | 17:35 |
slagle | installed the needed packages | 17:36 |
*** myoung|remote has joined #tripleo | 17:36 | |
slagle | then ran the command | 17:36 |
slagle | that would be equivalent to what we document/ci | 17:36 |
pabelanger | why do we need dlrn to test if building an overcloud image works? | 17:36 |
*** jpena is now known as jpena|away | 17:37 | |
slagle | we don't | 17:37 |
slagle | i just dont know that testing it can be build a different way is all that meaningful | 17:37 |
slagle | when we dont tell people to build that way, or know that those images work | 17:37 |
*** shardy has quit IRC | 17:39 | |
pabelanger | okay, so you want to make sure DIB is using packages from dlrn | 17:39 |
slagle | yea, basically | 17:39 |
slagle | i mean i'm not -1 on your patch | 17:39 |
slagle | i just don't want you spend a lot of time getting it working | 17:40 |
*** myoung has joined #tripleo | 17:40 | |
slagle | and then we wouldnt use this image in tripleo-ci, b/c it's now how we say to build images | 17:40 |
slagle | *not how | 17:40 |
*** Tristitia has joined #tripleo | 17:41 | |
*** dprince has joined #tripleo | 17:41 | |
pabelanger | Right, I'm not saying use this image for tripleo-ci right now. This is a POC to see if we can build an image in the gate. Which right now, appears we cannot. | 17:41 |
slagle | ok | 17:41 |
pabelanger | now, that could be because we are not using DRLN repos for DIB to install packages from, which I can make a change for | 17:41 |
slagle | is the os-collect-config rpm installed in the image? | 17:44 |
*** apetrich has quit IRC | 17:45 | |
*** apetrich has joined #tripleo | 17:45 | |
pabelanger | looks like it: installing os-collect-config from os-collect-config | 17:45 |
pabelanger | let me try again exporting the delorean vars | 17:46 |
*** tesseract- has quit IRC | 17:53 | |
*** bfournie has joined #tripleo | 17:56 | |
*** ccamacho has quit IRC | 18:01 | |
*** julim has quit IRC | 18:02 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: [WIP] remove overcloud roles from oooq https://review.openstack.org/351259 | 18:03 |
*** julim has joined #tripleo | 18:05 | |
*** sarath has joined #tripleo | 18:05 | |
gfidente | leaving guys | 18:07 |
gfidente | slagle, <3 | 18:07 |
slagle | come back soon now! | 18:07 |
gfidente | the day you'll try to kill me too I quit okay? | 18:07 |
gfidente | weshay, curiosity, did we get ooo fixed with the centos repos? | 18:08 |
slagle | i know better than to pull on a knife on someone named "giulio" | 18:08 |
weshay | gfidente, I'm getting there :) | 18:09 |
gfidente | weshay, ack | 18:09 |
*** abehl has joined #tripleo | 18:09 | |
beagles | slagle: are we experiencing time skewing occasionally in CI on the controller nodes? | 18:10 |
*** gfidente has quit IRC | 18:11 | |
slagle | i dunno :) | 18:11 |
beagles | I vaguely recall seeing something about it lately but didn't commit to memory where I saw it | 18:11 |
slagle | but the fact that you're asking makes me want to say yes | 18:11 |
beagles | lol | 18:11 |
beagles | yeah, just looking at some logs and they are a little suspicious... just doublechecking | 18:11 |
beagles | doublechecking the logs that is. | 18:12 |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: [WIP] remove ci-scripts from the oooq repo https://review.openstack.org/351324 | 18:12 |
sarath | dprince, bnemec: please help review https://review.openstack.org/#/c/345599/ | 18:13 |
*** jdob has quit IRC | 18:22 | |
*** myoung|remote has quit IRC | 18:22 | |
*** jdob has joined #tripleo | 18:22 | |
*** oshvartz has quit IRC | 18:23 | |
*** jbadiapa has quit IRC | 18:23 | |
beagles | are things like this: "error: Internal error: No action found for demote in redis:1" cause for concern or is this something we are living with? | 18:24 |
EmilienM | beagles: where? | 18:26 |
beagles | EmilienM, 1s lost tab | 18:28 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 18:28 |
beagles | EmilienM, file is http://logs.openstack.org/47/332147/19/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/464cd86/logs/overcloud-controller-2/var/log/messages | 18:33 |
*** florianf|biab is now known as florianf | 18:34 | |
*** jpena|away is now known as jpena|off | 18:34 | |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement non-ovb overcloud update job - Newton -> Newton https://review.openstack.org/351330 | 18:36 |
pabelanger | slagle: adding the DLRN repo was the trick, image built as expected | 18:36 |
EmilienM | beagles: looking | 18:37 |
EmilienM | beagles: it looks like a pacemaker thing | 18:37 |
EmilienM | and since we're removing it for redis, I would say, ignore it now | 18:37 |
beagles | EmilienM, ack. | 18:37 |
*** ayoung has quit IRC | 18:38 | |
*** florianf has quit IRC | 18:44 | |
openstackgerrit | Merged openstack/tripleo-quickstart: Fix wrong naming of image's name in release. https://review.openstack.org/350712 | 18:44 |
*** milan has quit IRC | 18:46 | |
*** Goneri has joined #tripleo | 18:47 | |
*** milan has joined #tripleo | 18:47 | |
*** mgarciam has joined #tripleo | 18:51 | |
*** bootsha has quit IRC | 18:51 | |
*** alhernan has joined #tripleo | 18:53 | |
*** fultonj has quit IRC | 18:59 | |
*** ccamacho has joined #tripleo | 19:05 | |
*** Goneri has quit IRC | 19:08 | |
*** fultonj has joined #tripleo | 19:09 | |
openstackgerrit | Frank Zdarsky proposed openstack/tripleo-validations: Add jsonschema validation for os-net-config data https://review.openstack.org/351345 | 19:10 |
ccamacho | pradk bandini EmilienM my env went to... Had to reinstall it with less nodes with more RAM for each node.. Anyway after that, aodh works as expected :) finally http://paste.openstack.org/show/549289/ | 19:10 |
EmilienM | ccamacho: woot | 19:11 |
pradk | ccamacho, this with latest ps? mine is still in progress with my last fix .. i guess the bind_host helped? | 19:11 |
rhallisey | anyone else been seeing mistral errors thrown around during baremetal setup steps in the undercloud? | 19:12 |
ccamacho | pradk indeed :) | 19:12 |
pradk | great, thx for confirming.. i'll let mine finish | 19:12 |
ccamacho | rhallisey yeahp, I saw a few some minutes ago when deploying my undercloud but didn't interfere with the deployment | 19:13 |
*** radek_ has joined #tripleo | 19:13 | |
ccamacho | pradk lets wait for CI | 19:13 |
rhallisey | yesterday it was failing with introspection. Today it fails when I import instackenv | 19:13 |
rhallisey | to the logs | 19:13 |
*** chem has quit IRC | 19:14 | |
rhallisey | wonder if it's mistral or ironic | 19:15 |
pradk | ccamacho, did a recheck lets see | 19:15 |
*** bana_k has quit IRC | 19:16 | |
*** dsneddon_ has joined #tripleo | 19:17 | |
*** oshvartz has joined #tripleo | 19:20 | |
rhallisey | mistral db error | 19:24 |
*** akrivoka has quit IRC | 19:31 | |
openstackgerrit | Dan Prince proposed openstack/python-tripleoclient: Deploy the undercloud with Heat https://review.openstack.org/351351 | 19:32 |
openstackgerrit | Dan Prince proposed openstack/python-tripleoclient: Deploy the undercloud with Heat https://review.openstack.org/351351 | 19:33 |
*** bana_k has joined #tripleo | 19:34 | |
openstackgerrit | Merged openstack-infra/tripleo-ci: Remove pinning of delorean https://review.openstack.org/349995 | 19:35 |
openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart: [WIP] Prototype tandem documentation generation https://review.openstack.org/347592 | 19:35 |
openstackgerrit | Merged openstack-infra/tripleo-ci: Add undercloud job in cistatus https://review.openstack.org/350763 | 19:35 |
*** dsneddon is now known as dsneddon_lunch | 19:37 | |
*** dsneddon_ has quit IRC | 19:38 | |
openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files https://review.openstack.org/350271 | 19:38 |
openstackgerrit | Ratnakar Kolli proposed openstack/tripleo-heat-templates: Adding optional aci-opflex hiera data https://review.openstack.org/351358 | 19:38 |
*** radek_ has quit IRC | 19:40 | |
*** jrist has quit IRC | 19:43 | |
*** lblanchard has quit IRC | 19:47 | |
*** fultonj has quit IRC | 19:48 | |
*** ayoung has joined #tripleo | 19:56 | |
*** jrist has joined #tripleo | 19:56 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Add Aodh composable roles https://review.openstack.org/333556 | 20:01 |
*** fultonj has joined #tripleo | 20:03 | |
*** rasca has quit IRC | 20:04 | |
dtrainor | anything else i can do to monitor the progress of introspection or see the data coming in, short of 'ironic node-list' and 'ironic node-show ...' ? | 20:04 |
*** oshvartz has quit IRC | 20:05 | |
*** mgarciam has quit IRC | 20:15 | |
*** oshvartz has joined #tripleo | 20:20 | |
*** dsneddon_lunch is now known as dsneddon | 20:21 | |
*** bana_k has quit IRC | 20:24 | |
*** jayg is now known as jayg|g0n3 | 20:25 | |
*** bana_k has joined #tripleo | 20:31 | |
*** jdob has quit IRC | 20:34 | |
*** jdob has joined #tripleo | 20:35 | |
EmilienM | I got an overcloud update job working in CI using multinode nodes https://review.openstack.org/#/c/351330/ | 20:35 |
EmilienM | my question is now: should I rename the CI job to be -updates instead of -upgrades ? (I'm doing Newton -> Newton now) | 20:36 |
EmilienM | and keep -upgades for Mitaka to Newton | 20:36 |
EmilienM | wdyt? | 20:36 |
*** jrist has quit IRC | 20:38 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: tripleo update job testing - do not merge https://review.openstack.org/351389 | 20:39 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: test if overcloud update can actually fail https://review.openstack.org/351391 | 20:41 |
*** abehl has quit IRC | 20:44 | |
*** alhernan has quit IRC | 20:46 | |
*** jrist has joined #tripleo | 20:49 | |
*** dsariel has quit IRC | 20:50 | |
*** rhallisey has quit IRC | 21:01 | |
*** tobias_fiberdata has joined #tripleo | 21:05 | |
*** julim has quit IRC | 21:18 | |
*** julim has joined #tripleo | 21:18 | |
*** egafford has quit IRC | 21:19 | |
*** julim has quit IRC | 21:23 | |
*** jeckersb is now known as jeckersb_gone | 21:28 | |
*** yamahata has quit IRC | 21:30 | |
*** morazi has quit IRC | 21:33 | |
*** fultonj has quit IRC | 21:40 | |
*** fpan has joined #tripleo | 21:41 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Remove some properties from overcloud.yaml Controller group https://review.openstack.org/347050 | 21:41 |
*** ayoung has quit IRC | 21:43 | |
*** yamahata has joined #tripleo | 21:49 | |
*** dprince has quit IRC | 21:59 | |
*** tbonds has quit IRC | 22:01 | |
*** rasca has joined #tripleo | 22:02 | |
openstackgerrit | Merged openstack/puppet-tripleo: Next generation HA architecture work https://review.openstack.org/342650 | 22:11 |
*** jcoufal_ has quit IRC | 22:19 | |
*** ayoung has joined #tripleo | 22:26 | |
*** pradk has quit IRC | 22:34 | |
*** tobias_fiberdata has quit IRC | 22:43 | |
openstackgerrit | Dan Sneddon proposed openstack/os-net-config: Enable setting of IPV6_AUTOCONF separately from DHCPV6C https://review.openstack.org/350794 | 22:46 |
*** dsneddon_ has joined #tripleo | 22:59 | |
*** panda is now known as panda|zz | 23:01 | |
*** ayoung has quit IRC | 23:06 | |
*** bana_k has quit IRC | 23:09 | |
*** dsneddon_ has quit IRC | 23:11 | |
*** tzumainn has quit IRC | 23:32 | |
*** bana_k has joined #tripleo | 23:39 | |
*** weshay has quit IRC | 23:56 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Next generation HA architecture work https://review.openstack.org/314208 | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!