Monday, 2016-08-22

*** limao has joined #tripleo00:42
*** limao has quit IRC00:43
*** limao has joined #tripleo00:43
*** fragatina has joined #tripleo00:43
*** ebalduf has quit IRC00:55
*** fragatin_ has joined #tripleo01:43
*** fragatina has quit IRC01:47
*** ramishra has joined #tripleo01:48
*** trozet has joined #tripleo02:03
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates: Adds OpenDaylight composable service  https://review.openstack.org/35514702:09
*** dmacpher has joined #tripleo02:10
*** ebalduf has joined #tripleo02:19
*** trozet has quit IRC02:31
*** jcoufal has joined #tripleo02:51
*** oshvartz has quit IRC02:56
*** xuao has joined #tripleo03:08
*** jcoufal has quit IRC03:16
*** jcoufal has joined #tripleo03:18
*** jcoufal has quit IRC03:20
*** julim has quit IRC03:27
*** ayoung has quit IRC03:43
*** bana_k has joined #tripleo03:45
*** yamahata has quit IRC04:05
*** limao has quit IRC04:21
*** nyechiel has joined #tripleo04:24
*** nyechiel has quit IRC04:33
*** david-lyle has quit IRC04:37
*** pcaruana has quit IRC04:55
*** pgadiya has joined #tripleo04:59
*** limao has joined #tripleo05:03
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Use separated SSL endpoint environment file  https://review.openstack.org/35648805:05
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Inject CA certificate via service profile  https://review.openstack.org/35779105:05
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Pass TRIPLEO_ROOT directory to heat_deploy_times.sh  https://review.openstack.org/35694605:07
*** yamahata has joined #tripleo05:18
*** fragatina has joined #tripleo05:20
*** fragatin_ has quit IRC05:20
*** jaosorior has joined #tripleo05:23
*** jtomasek has joined #tripleo05:28
*** bana_k has quit IRC05:35
*** links has joined #tripleo05:38
*** r-mibu has quit IRC05:47
*** r-mibu has joined #tripleo05:52
*** leanderthal|afk is now known as leanderthal05:57
*** saneax-_-|AFK is now known as saneax05:58
*** florianf has joined #tripleo06:00
*** skramaja has joined #tripleo06:07
openstackgerritMerged openstack/tripleo-ui: Add packaging spec and config files  https://review.openstack.org/34493206:11
*** ebalduf has quit IRC06:15
*** coolsvap has joined #tripleo06:15
*** coolsvap has quit IRC06:16
*** coolsvap has joined #tripleo06:16
*** adarazs has quit IRC06:18
*** adarazs has joined #tripleo06:18
*** bana_k has joined #tripleo06:19
openstackgerritSaravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bond  https://review.openstack.org/34918306:21
*** skramaja has quit IRC06:34
*** rasca has joined #tripleo06:40
*** tremble has joined #tripleo06:41
openstackgerritKarthik S proposed openstack/tripleo-heat-templates: SR-IOV nic agent changes in line with composable roles.  https://review.openstack.org/33219006:45
*** ohamada has joined #tripleo06:45
openstackgerritKarthik S proposed openstack/tripleo-heat-templates: Add sriov_numvfs in THT  https://review.openstack.org/34535206:47
*** jbadiapa has joined #tripleo06:49
openstackgerritRobin Cernin proposed openstack/tripleo-validations: Validate rabbitmq policies  https://review.openstack.org/35843006:49
*** rcernin has joined #tripleo06:51
*** pcaruana has joined #tripleo06:52
*** matbu|afk is now known as matbu06:54
*** rwsu has joined #tripleo06:58
*** jprovazn has joined #tripleo07:05
*** dsariel has joined #tripleo07:06
*** jbadiapa has quit IRC07:08
*** tesseract- has joined #tripleo07:09
*** skramaja has joined #tripleo07:12
*** milan has joined #tripleo07:18
*** bootsha has joined #tripleo07:19
*** bana_k has quit IRC07:20
*** bvandenh has joined #tripleo07:21
*** numans has joined #tripleo07:25
*** rcernin has quit IRC07:25
*** zoli_gone-proxy is now known as zoliXXL07:26
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE TESTING  https://review.openstack.org/31643607:28
bandinimorning07:29
*** jpich has joined #tripleo07:30
*** bootsha has quit IRC07:36
*** bana_k has joined #tripleo07:39
*** bana_k has quit IRC07:56
*** ifarkas_afk is now known as ifarkas07:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable SSL for ironic inspector  https://review.openstack.org/33264107:57
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Refactor endpoint env variable population  https://review.openstack.org/35039607:57
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable TLS for Zaqar API  https://review.openstack.org/35157607:57
*** ccamacho has joined #tripleo07:59
*** hewbrocca-afk is now known as hewbrocca08:01
hewbroccaMorning everyone08:02
*** mhenkel has joined #tripleo08:05
*** dmacpher has quit IRC08:06
*** gfidente has joined #tripleo08:08
*** zoliXXL is now known as zoli|trng08:08
ccamachomorning!08:08
gfidentemorning :)08:09
*** athomas has joined #tripleo08:10
jaosoriordtantsur|afk: are you around?08:10
jaosoriordo'h I see the afk now08:11
*** derekh has joined #tripleo08:16
jaosoriorderekh: hey dude, can you check this out? https://review.openstack.org/#/c/356946/08:17
derekhjaosorior: looking08:17
openstackgerritJulie Pichon proposed openstack/tripleo-specs: Add 'bug tagging' policy  https://review.openstack.org/35285208:17
*** coolsvap is now known as _coolsvap_08:18
*** jbadiapa has joined #tripleo08:18
openstackgerritMichael  Henkel proposed openstack/tripleo-puppet-elements: add puppet source repository for Contrail puppet modules  https://review.openstack.org/35846708:19
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts  https://review.openstack.org/35776508:21
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable usage of FQDNs for the endpoints  https://review.openstack.org/35641608:21
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add deployment of CAs via hieradata  https://review.openstack.org/35303708:22
*** abregman_ has joined #tripleo08:26
jaosoriorderekh: thanks for checking it out08:26
derekhjaosorior: np08:26
*** bootsha has joined #tripleo08:26
jaosoriormarios: when you get time, can you check this out? https://review.openstack.org/#/c/356416/08:27
abregman_hey. my overcloud deployment fails for the resource 'ControllerNodesPostDeployment' with the following message: resources.ControllerNodesPostDeployment.resources.ControllerPostPuppet.resources.ControllerPostPuppetRestartDeployment.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 108:27
abregman_where can I check what exactly went wrong?08:28
*** radeks has joined #tripleo08:28
jaosoriorabregman_: are you using master?08:28
abregman_jaosorior: no. kilo.08:28
jaosoriorabregman_: alright. I recommend following this http://hardysteven.blogspot.fi/2015/04/debugging-tripleo-heat-templates.html08:29
abregman_jaosorior: thanks. /me reading.08:29
*** jaosorior has quit IRC08:30
*** jaosorior has joined #tripleo08:31
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Remove support for legacy rh1  https://review.openstack.org/34791808:33
*** lucas-afk is now known as lucasagomes08:34
gfidenteping bandini08:34
derekhjaosorior: btw, I've been trying to get rid of a lot of those ssh's altogether https://review.openstack.org/#/c/347918/3 (just rebased now)08:35
bandinigfidente: pong08:36
jaosoriorderekh: I see. Will check that out08:37
jaosoriorjistr: hey dude, when you get some time can you check this out? https://review.openstack.org/#/c/356416/08:38
jistrok :)08:40
jistrgood morning all08:40
bandinimorning jistr jaosorior ccamacho08:44
jaosoriorhey people, what's up?08:44
ccamachobandini happy Monday man!08:44
ccamachostarting the week :)08:45
bandiniMonday's are Satan's invention!08:45
ccamachojaajaaj no way, I hate Wednesday's08:46
*** Goneri has joined #tripleo08:51
abregman_jaosorior: thanks for posting the link to the blog..found the issue: "pcs resource restart rabbitmq-clone\nError: Could not complete shutdown of rabbitmq-clone"08:51
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Add support for deploying Ceph RGW role  https://review.openstack.org/28902708:52
*** pgadiya has quit IRC08:52
jaosoriorbandini: have you seen something like what abregman_ is seeing?08:52
*** abehl has joined #tripleo08:53
bandinijaosorior: nope I haven't seen that specific issue. from a quick hunch I'd say a bit of VM starvation? Is this repeatable?09:00
*** chem``` is now known as chem09:01
*** chem has quit IRC09:02
*** chem has joined #tripleo09:02
jaosoriorgfidente: abregman_ ^^09:04
abregman_bandini, jaosorior: yeah..100% repeatable09:05
*** pgadiya has joined #tripleo09:06
*** ohamada has quit IRC09:06
*** ohamada has joined #tripleo09:06
*** ohamada has quit IRC09:06
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE TESTING  https://review.openstack.org/31643609:06
*** ohamada_ has joined #tripleo09:06
ccamachohey, some time ago I had the same issue but the error was related to OOM mind to find memory in the logs to see if you are getting OOM's?09:06
*** ohamada_ has quit IRC09:07
*** ohamada_ has joined #tripleo09:07
*** yamahata has quit IRC09:07
*** ohamada_ has quit IRC09:07
*** ebarrera has joined #tripleo09:07
*** ohamada_ has joined #tripleo09:07
openstackgerritRadoslaw Smigielski proposed openstack/tripleo-heat-templates: Add NeutronEnableMetadataNetwork to composable neutron dhcp service  https://review.openstack.org/35849309:08
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Add support for deploying Ceph RGW role  https://review.openstack.org/28902709:09
bandiniabregman_: any logs around?09:10
abregman_bandini: http://paste.openstack.org/raw/562173/ , what other logs should I paste?09:13
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates: Added a new composable service for enabling DPDK on compute nodes.  https://review.openstack.org/34408409:13
openstackgerritKarthik S proposed openstack/tripleo-heat-templates: SR-IOV nic agent changes in line with composable roles.  https://review.openstack.org/33219009:18
*** pgadiya has quit IRC09:18
*** pgadiya has joined #tripleo09:18
openstackgerritKarthik S proposed openstack/tripleo-heat-templates: Add sriov_numvfs in THT  https://review.openstack.org/34535209:19
*** bootsha has quit IRC09:20
*** akrivoka has joined #tripleo09:21
abregman_bandini, jaosorior: I checked and it also fails when trying manually: http://paste.openstack.org/raw/562174/09:22
ccamachohey mhenkel, re your Contrail submissions, Im testing them from puppet to tht Ill let you know when having some feedback09:23
ccamachothanks for https://review.openstack.org/#/c/358467/ ^09:23
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts  https://review.openstack.org/35776509:23
mhenkelthanks ccamacho!09:23
*** bootsha has joined #tripleo09:23
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates: Added a new composable service for enabling DPDK on compute nodes.  https://review.openstack.org/34408409:24
abregman_bandini, jaosorior: this solves the issue: "pcs resource update rabbitmq op stop timeout=300", but is there a way to preconfigure this before deploying the overcloud?09:24
ccamachomhenkel I wasnt sure which puppet repo was the correct to test the puppet-tripleo manifests, now testing them.09:25
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Add support for deploying Ceph RGW role  https://review.openstack.org/28902709:25
*** sshnaidm has quit IRC09:26
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates: Added a new composable service for enabling DPDK on compute nodes.  https://review.openstack.org/34408409:28
bandiniabregman_: can you send me the file generated by "pcs cluster cib /tmp/cib.xml"?09:29
*** athomas has quit IRC09:29
bandiniabregman_: we'd have to change the value in the tht templates. what version are you using?09:29
abregman_bandini: kilo. I don't have this issue in Mitaka or Liberty09:31
abregman_bandini: http://paste.openstack.org/raw/562177/09:31
bandiniabregman_: ah right, we did increase the timeouts starting from liberty09:32
*** athomas has joined #tripleo09:34
abregman_bandini: this line?: https://github.com/openstack/tripleo-heat-templates/blob/stable/liberty/puppet/manifests/overcloud_controller_pacemaker.pp#L11409:34
bandiniabregman_: we probably want something like this https://review.openstack.org/#/c/272026/ in kilo09:34
openstackgerritJiri Stransky proposed openstack/puppet-tripleo: Write restart flags to restart services only when necessary  https://review.openstack.org/35851009:34
bandiniabregman_: yep09:34
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Restart only services that need it  https://review.openstack.org/35851109:35
abregman_bandini: ack. thanks for the help!09:35
bandiniabregman_: np ;)09:36
*** dtantsur|afk is now known as dtantsur09:37
dtantsurjaosorior, hey, around now09:38
dtantsuralso folks, I need someone's help with tripleo. I can't get TFTP to work in overcloud, and I'm out of ideas why...09:38
jaosoriordtantsur: Hey, I'm trying to test a change I did for ironic-python-agent; And I'm not sure how to build that image with DIB that includes the change I did09:39
openstackgerritDmitry Tantsur proposed openstack/tripleo-docs: [WIP] Documentation for installing and using Ironic in overcloud  https://review.openstack.org/35401609:39
jaosoriordtantsur: in case you're wondernig, it's this https://review.openstack.org/#/c/358457/09:39
dtantsurjaosorior, do you need it RHEL-based or just any IPA image?09:40
jaosoriordtantsur: Trying to address the issue I brought up the other day regarding the need to rebuild images if we have TLS in the undercloud09:40
jaosoriordtantsur: well... whatever works with TripleO upstream09:40
dtantsurjaosorior, anything will work if you disable advanced introspection :) anyway, lemme remember it real quick09:41
mhenkelccamacho: should I just abandon the commit?09:41
ccamachoyeahp already landed long time ago, I'm trying now to install all dependencies and test the puppet-tripleo part, but BTW thanks for pointing out that submission09:42
*** tosky has joined #tripleo09:43
openstackgerritJiri Stransky proposed openstack/puppet-tripleo: Write restart flags to restart services only when necessary  https://review.openstack.org/35851009:43
openstackgerritDmitry Tantsur proposed openstack/tripleo-docs: Document using in-progress review of IPA  https://review.openstack.org/35851809:48
dtantsurjaosorior, please try ^^^ used to work for me some time ago09:48
jaosoriordtantsur: will do; thanks!09:49
ccamachoGuys not sure if anyone else is getting issues to upload images to swift, fixed here (+2ed and passing CI)  https://review.openstack.org/#/c/357147/ The issue was that the swift URL its empty depending on the openstackclient version, the fix adds two ways of getting it.09:49
derekhdtantsur: whats the problem with tftp on the overcloud, if you send me some details what you've tried etc... I'll see what I can figure out09:52
dtantsurderekh, hey there! I've documented everything I did in https://review.openstack.org/354016. The problem is that I can access TFTP from within the controller, but not from undercloud or the bare metal nodes.09:54
dtantsurderekh, in tcpdump is looks like request arrive, but no response is sent. no errors in logs, selinux permissive, flushing iptables tried.09:54
dtantsurif it matters, I'm using ooo-quickstart09:55
derekhdtantsur: ok, looking09:55
dtantsurthanks!09:56
dtantsurI have an environment, though I'm not sure how to share an ooo-quickstart created environment with someone09:56
dtantsurbut everything in this document seems to be reproducible09:56
jaosoriorjistr: I could do that, but I just think it's a usability issue :/09:56
shadowerd0ugal: do you know how the whole t-h-t in Swift thing works?09:57
shadoweror anyone? ^09:57
jistrjaosorior: yea i'm not convinced which one is better, just not sure enough to give a +2 ATM09:57
jaosoriorgfidente: what do you think? regarding jistr's comment in https://review.openstack.org/#/c/356416/3/overcloud.yaml09:58
openstackgerritErno Kuvaja proposed openstack/tripleo-heat-templates: Add integration with Manila CephFS Native driver  https://review.openstack.org/35852509:58
jistrjaosorior: e.g. i wonder if someone someday might come and want the e.g. InternalApi endpoints to not end with the same higher-level domain than the Public endpoints10:00
gfidentejistr, jaosorior right I think we need to figure if we want the per-network names or if don't10:00
jistri don't know why would someone do that but people sometimes want funny things10:01
jaosoriorgfidente: well, they can always configure it to use IP_ADDRESS and not use those names10:01
jaosoriorjistr: alright, so I could leave the CloudName functionality as is, and that parameter would disregard what CloudDomain contains. So if you actually want the domain to be used to have to specify it also in CloudName10:02
gfidentejaosorior, that is not what it used to do though right?10:02
jaosoriorgfidente: it really didn't do anything before, but show up in the endpoint map and in the heat stack name10:03
jistrgfidente: afaict that is what it used to do. That's actually why i started thinking about this -- the current state of tha patch changes a semantic of the CloudName param...10:04
*** milan has quit IRC10:04
* gfidente confused10:04
jaosoriorjistr: fair enough, I'll leave the functionality of CloudName as-is, yet add the other names10:05
jistrgfidente: lines 409-410 https://review.openstack.org/#/c/356416/3/overcloud.yaml10:05
jaosorioreither way, in net-iso, having CloudName for the internal endpoints would have never worked10:05
jistrjaosorior: yea i see. It would point everything to public, right?10:05
jaosoriorjistr: exactly10:06
jaosoriorand that's messed up10:06
jistrgfidente: vs. line 377 on the left side10:06
jistrjaosorior: yea10:06
gfidentejistr, ah thanks!10:06
*** Goneri has quit IRC10:07
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts  https://review.openstack.org/35776510:07
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable usage of FQDNs for the endpoints  https://review.openstack.org/35641610:07
jaosoriorjistr: what about that? ^^10:08
* gfidente comparing the actual endpoint_map10:08
d0ugalshadower: Sure, I know how it works. What do you need to know?10:08
d0ugalshadower: it is a little strange :)10:09
jistrjaosorior: yea but we should probably do the same for the rest of the CloudNames (the per-network ones), right?10:09
shadowerd0ugal: yea I figured :-). 1) how do the templates get to Swift and 2) how are they passed to Heat10:09
jaosoriorjistr: So, we will completely disregard the Clouddomain?10:09
*** limao has quit IRC10:09
jaosoriorjistr: that's gonna be pretty painful to support dude10:09
shadowerd0ugal: looks like we'll end up doing something similar for validations later on so I'd like to understand it better10:10
d0ugalshadower: They are manually uploaded to swift10:11
d0ugalshadower: then the deploy workflow sends them to heat10:11
shadowerd0ugal: how is Heat consuming them? As URLs or are they temporarily written to disk?10:11
d0ugalshadower: I think they are temporarily written to disk10:12
jistrjaosorior: well as i wrote on the patch, i'm a bit fuzzy on the concepts there. What's the exact difference between those params. One is for endpoints, the other one seems to be used in connection to undercloud Neutron.10:12
shadowerd0ugal: okay, thanks. Could you point me an the code?10:12
gfidentejaosorior, jistr I think the purpose there was to make it possible to stick to CLOUDNAME for all endpoints right?10:12
d0ugalshadower: Yup, getting links - it requires a few steps10:12
jistrperhaps i need to read up a bit more about this in the templates and look at deploymets10:13
jistrgfidente: nah that was already possible in previous state i think10:13
gfidentejistr, but in previous stat that would always point to the 'external' name10:13
gfidente*state10:13
gfidenteso the per-network names still sound nice to me, in which case it'd be how to deal with the external name in a backward compat way the issue, did I get it right?10:14
jistryea but you could still fill in the CloudName/CloudDomain parts to your liking10:14
jaosoriorjistr: and you still can10:15
jistryea10:15
gfidentejistr, you could but you'd get that same name for all endpoints type10:15
jaosoriorjistr: CLOUDNAME only worked for public endpoints. that's it10:15
jaosoriorjistr: and CloudDomain was disregarded in the endpoint generation10:16
gfidentejaosorior, for which I think the per-network names are fine10:16
gfidenteit's the public name which is backward incompat, what jistr pointed10:16
d0ugalshadower: sorry, just had a delivery. Okay, properly here now.10:16
jaosoriorgfidente: which I fixed in the last commit10:16
* gfidente refresh10:17
jaosoriorgfidente: https://review.openstack.org/#/c/356416/4/overcloud.yaml10:17
d0ugalshadower: First you need to call a mistral action to create the container. Then you upload the templates, then you call create_plan which parses them.10:17
jaosorioruhm... jistr: I think I see your point. So you're saying that we should only use CloudDomain for the management network?10:18
*** ebarrera has quit IRC10:18
d0ugalshadower: some people are against it, but if we add this, it will be much easier: https://review.openstack.org/#/c/347830/10:18
*** Goneri has joined #tripleo10:18
jistrjaosorior, gfidente: the thing i wasn't sure about was -- should we *always* make the CloudDomain part of the "final CloudName" in enpdoints. If yes, then maybe we should do it for all endpoints and doc the parameter change for upgrades, or if no, then we should probably have all the endpoint name params independent of it.10:18
* jistr lunch bbl10:19
gfidenteyeah as it looks like we misused clouddomain before to set the hostnames10:19
jaosoriorgfidente: I guess we can fix that... but then how do we set the domain for the nodes?10:19
d0ugalshadower: Writing you an example here: https://etherpad.openstack.org/p/tripleo-mistral-example10:20
gfidentejaosorior, though for the internal names I think current approach is valid10:20
jaosoriorgfidente: I think so too, to be honest10:21
gfidentewe dump hostnames as nodename.$network.$clouddomain10:21
gfidenteso $endpoint.$network.$clouddomain10:21
gfidenteat least matches the pattern10:21
jpichd0ugal: Do you think that patch would help the CLI as well? That gigantic JSON file still needs to be created... In your mind would it be the CLI command that starts the plan creation by doing that in the background?10:22
*** bootsha has quit IRC10:22
d0ugaljpich: I think it would help everyone. That patch calls create_container and create_plan, right?10:23
*** bootsha has joined #tripleo10:23
jpichd0ugal: It does10:24
shadowerd0ugal: thanks a lot!10:25
d0ugaljpich: Manually triggering two actions and uploading the files to swift seems much harder than just sending JSON to a workflow.10:25
jpichd0ugal: You still need to have that JSON file coming from somewhere - I had some issues with that but I think it was because of https://bugs.launchpad.net/tripleo/+bug/1613286 so maybe it's all fine nowadays10:25
openstackLaunchpad bug 1613286 in tripleo "[tripleo-common] Exclude more unneeded files from default plan" [High,In progress] - Assigned to Florian Fuchs (flo-fuchs)10:25
jpichd0ugal: Switching between Mistral / direct call / Mistral does feel a bit awkward10:26
jpichWell I'm keeping that patch nice and warm if it picks up interest again :)10:26
d0ugaljpich: Right, at the moment the CLI has to create a tarball of that directory to upload10:26
d0ugaljpich: I don't see making JSON as any harder :)10:26
jpichd0ugal: Fair enough! :)10:27
d0ugaljpich: huh, there are python files in t-h-t?10:27
jpichd0ugal: All the things :)10:27
jpichI suppose it's just something else to keep in mind10:28
d0ugaljpich: So there is. Interesting.10:28
d0ugaljpich: The CLI uses the code that is used to create the tarball for the actions, so we will have the same problem of picking up extra files.10:28
jpich(florianf ^ just a general FYI, chatting a bit about that tripleo-common bug you filed and how nice it might be to do the JSON upload at some point)10:29
jpichd0ugal: Yeah, something else to keep in mind10:30
*** sshnaidm has joined #tripleo10:32
*** oshvartz has joined #tripleo10:33
sshnaidmderekh, hi10:33
derekhsshnaidm: hi ya10:34
sshnaidmderekh, is 3.30 ok for you?10:34
derekhsshnaidm: utc? yes that would be fine10:35
sshnaidmderekh, yep, cool10:35
*** ebarrera has joined #tripleo10:38
jaosoriorgfidente: well, the last submission respects CloudName as it used to work before. and the domain is used like we did before too. So I think it makes sense10:40
gfidentejaosorior, hey I was just posting a comment10:41
jaosoriorgfidente: cause I don't think deployers will want to go through the hassle of having different domains per network (internal_api, storage, management) and so on. Having the ability to change the domain for the external makes sense though10:41
gfidenteso basing on the fact that we build ip>name maps hardcoding the network subdomain10:42
gfidenteAND the domain from clouddomain10:42
gfidentemaybe we should try to be consistent with that and build the per-network endpoints the same way?10:42
gfidenteif people is to manage a dns for the internal names, they will own the network-named subdomain so we can just add their internal endpoint name in there10:43
gfidenteon the other hand, cloudname is public and remains completely customizable10:43
gfidenteand backward compat10:43
jaosoriorgfidente: isn't that the current behavior?10:43
jaosoriorah wait10:44
gfidenteit is, and the behavior is fine to me10:44
jaosoriorso you mean that we should hardcode those VIP names too?10:44
gfidentethe network subdomain yes10:44
gfidentenot changing the behavior from patchset 410:44
gfidentejust there is a single param for the internal vip name, not dependent on the network10:45
gfidenteas network subdomain is appended to it as we do for hosts10:45
jaosorioraaah10:46
jaosoriorI see what you mean10:46
jaosoriorso with the defaults10:47
jaosoriorughm... wait up, let me code up what I understood10:47
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: [NO MERGE] Test Ceph RadosGW as replacement for Swift  https://review.openstack.org/35718210:47
jaosoriorgfidente: wait up; But maybe we don't want to do that10:49
gfidentejaosorior, right people might want to resolve those against a different domain purposely10:50
jaosoriorgfidente: well, for the external it makes sense to have a different domain10:51
florianfjpich, d0ugal, shadower : Yeah, it would sure be great to upload the whole JSON structure via a workflow (or maybe a custom swift middleware, like we discussed...). I'm working on the file-by-file upload from the UI right now, which comes down to roughly 300 HTTP requests. It works surprisingly well so far, but it feels soooo bad. :-)10:52
d0ugalflorianf: we really shouldn't do that :(10:53
d0ugalflorianf: is it possible for you to create a tarball?10:54
shadoweroof10:54
jpichflorianf: Note that the custom swift middleware may well be useful even if we use the upload to JSON workflow, since that workflow uploads files to Swift individually at the moment10:54
sshnaidmderekh, one more question, you created a dev env in prev environment, but now it's a little outdated, can we please update it? or we can discuss it in the meeting as well10:54
jpich...Or maybe that workflow should create temporary files based on the JSON, tarball that, then upload to Swift ?_?10:55
d0ugaljpich: oh, it does? It should maybe use the tarball too!10:55
d0ugaljpich: I think that might be better tbh10:55
d0ugaljpich: What happens if one swift request fails for some reason?10:55
florianfd0ugal: Uploading a tarball is one (already existing) option in the UI. The option I'm talking about is selecting the templates folder from the browser dialog and then reading the files. So, no, creating a tarball isn't an option from here.10:55
florianfjpich: true!10:56
jpichd0ugal: I think I had a problem around creating temporary files that can be used by different Mistral actions and then cleaning up properly at the end10:56
d0ugalflorianf: Sure, okay. I was just wondering if a tarball could be made by the javascript in the browser10:56
d0ugalI am -1 to uploading files one by one FWIW10:56
d0ugaljpich: I think it would be better to try and resolve that10:57
d0ugalThe create_default_plan and CLI upload as a tarball to try and ensure some sort of consistency - all the files, or none.10:58
florianfd0ugal: Not natively. Not sure if we want to find/evaluate/add a third-party library for that.10:58
d0ugalflorianf: I think it would be worth it, but *shrug*10:59
*** milan has joined #tripleo10:59
d0ugalI was the only one that seemed interested in atomic actions when this originally came up, so maybe it is still just me :)11:00
florianfd0ugal: I think the one-by-one upload is an acceptable intermediate version until we either use a workflow or a custom swift middleware11:00
*** akrivoka has quit IRC11:01
*** akrivoka has joined #tripleo11:03
*** openstackgerrit has quit IRC11:03
*** openstackgerrit has joined #tripleo11:03
jpichd0ugal: Looking into it a bit closer, it shouldn't be a major issue to get the whole temporary files creation + tarball + upload working if we do it all in the same action like it is now so, I'll add a note to the patch and try to put something together later in the week. Thanks for the food for thought!11:07
jpichI think I'd been trying to generalise and reuse UploadTemplatesAction at first but I couldn't figure out how/where to clean up the temporary files when doing it that way11:08
jaosoriorgfidente: soo... now it's not entirely clear to me. Should I leave that submission as-is>?11:08
*** ramishra has quit IRC11:08
*** bootsha has quit IRC11:10
*** zoli|trng is now known as zoli_gone-proxy11:10
*** rodrigods has quit IRC11:11
*** rodrigods has joined #tripleo11:11
d0ugaljpich: Thanks, I can try and help if needed!11:14
*** ramishra has joined #tripleo11:14
*** dtantsur is now known as dtantsur|brb11:14
*** ebarrera has quit IRC11:15
*** rasca has quit IRC11:15
gfidentejaosorior, it looks to me that if build them up from a single name it's more consistent with what we do for the hostnames11:15
*** bootsha has joined #tripleo11:15
jpichd0ugal: Thank you! I'll let you know11:15
*** fragatina has quit IRC11:15
*** jprovazn has quit IRC11:16
jaosoriorgfidente: what do you mean with a single name?11:16
*** rasca has joined #tripleo11:16
*** fragatina has joined #tripleo11:16
*** limao has joined #tripleo11:16
gfidentejaosorior, instead of having multiple parameters for the internal names, one per network11:16
gfidentewe get a single parameter to which we append the network subdomain11:16
*** ramishra has quit IRC11:17
*** weshay has quit IRC11:17
jaosoriorgfidente: I see... something like... InternalVipName? or something of the sort?11:17
gfidenteyes11:17
jaosoriorgfidente: OK, let me try that11:17
*** trown has quit IRC11:17
*** weshay has joined #tripleo11:18
*** trown has joined #tripleo11:18
*** ramishra has joined #tripleo11:19
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Move RabbitMQ settings out of controller.yaml  https://review.openstack.org/35664011:20
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Move MySQL settings out of puppet/controller.yaml  https://review.openstack.org/35650511:20
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Move Keepalived/HAproxy settings out of controller  https://review.openstack.org/35734411:20
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts  https://review.openstack.org/35776511:23
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable usage of FQDNs for the endpoints  https://review.openstack.org/35641611:23
jaosoriorgfidente: what about that? ^^11:23
*** thrash|g0ne is now known as thrash11:31
gfidentejaosorior, +211:32
gfidentenot sure about the actual param name but let's wait for jistr as well to see if he's happy with building it from the shortname, like we do with hostnames11:33
*** ohamada_ has quit IRC11:33
gfidenteand keep only CloudName as a customizable FQDN11:33
*** limao has quit IRC11:33
* jistr looks11:33
*** limao has joined #tripleo11:33
*** openstackstatus has quit IRC11:36
*** pkovar has joined #tripleo11:37
jistrcould we maybe try to come up with a practical example? so e.g. i own example.org and i'm deploying with tripleo11:38
jistrI set CloudDomain: cloud.example.org (i don't want individual machines to be 3rd level domain, along other things like www.example.org. I want them all scoped under cloud.example.org)11:39
*** adarazs is now known as adarazs_lunc11:39
*** adarazs_lunc is now known as adarazs_lunch11:39
*** bootsha has quit IRC11:39
*** openstackstatus has joined #tripleo11:39
*** ChanServ sets mode: +v openstackstatus11:39
jistrmaybe i want the Api endpoint to be on 3rd level though, so let's set CloudName: api.example.org11:39
jistr^^ does that sound as a valid use case? So maybe not only backwards compat, but actually we don't want the CloudDomain to be a part of the endpoints always?11:40
jaosoriorjistr: so, what I had proposed initially was that you could be able to do that per-network11:40
jistrregardless of what it does to backwards compat11:40
jaosoriorso the default for the internal_api network endpoints would be11:40
jaosoriorovercloud.internalapi.example.org11:40
jaosoriorgfidente suggested we do the same thing as we do for the node names. So now the network part (in that example being internalapi) is hardcoded11:41
jaosoriorjistr: one thing is that, what you mentioned as "Api endpoint" is the external endpoint, or public in keystone terms. And that works if you set CloudName11:42
jaosoriorbut, what about internal and admin (in keystone terms)11:42
jaosoriorthose are the ones I'm trying to address here11:42
*** jprovazn has joined #tripleo11:42
jaosoriorso if you set CloudName as api.example.org; That's exactly what you'll see in keystone. Nothing has changed there11:42
jaosoriorwhat now changes is that you can now tell the endpoint map to produce names for the internal endpoints11:43
gfidentejistr, we have per-network subdomain to resolve the hostnames against the different networks11:43
*** dprince has joined #tripleo11:43
gfidentejistr, and we have a parameter to decide against which network the hostname without subdomain should be resolved to11:43
*** bootsha has joined #tripleo11:43
jaosoriorjistr: and you can still get the public endpoint to have a different domain than what CloudDomain specifies11:44
*** maeca1 has joined #tripleo11:44
gfidentejistr, https://github.com/openstack/tripleo-heat-templates/blob/master/network/service_net_map.yaml#L44-L4811:45
jistrjaosorior: yea that's what i've been trying to outline above. Perhaps we want that as a feature, that public endpoint does not actually have to always end with CloudDomain. So regardless of backwards compatibilty, we might not want to mix those together. The question is, do we want to mix those together for the *other* (non-public) endpoints?11:46
jaosoriorjistr: so, that's the behavior. CloudName doesn't get mixed with CloudDomain (in the current submission)11:46
jistrjaosorior: it does for internal though, no?11:47
openstackgerritAna Krivokapic proposed openstack/python-tripleoclient: Fix doc page for overcloud deploy  https://review.openstack.org/35857711:47
jaosoriorjistr: yes. For internal endpoints we do use the CloudDomain; Same as is the case for the nodes11:47
jaosoriorjistr: I think supporting having different domains for different internal networks is gonna be pretty hard support-wise. And I don't think it's a very common case.11:49
jaosoriortremble: are you around?11:49
*** jaosorior has quit IRC11:50
*** nyechiel has joined #tripleo11:50
*** jaosorior has joined #tripleo11:50
* gfidente going for lunch, back soon11:51
*** jpena is now known as jpena|lunch11:51
jistrso in the example deployment i outlined above, the internal endpoint would be overcloud.internalapi.cloud.example.org11:52
*** ebarrera has joined #tripleo11:52
jistri could change the bottom-most domain part of it to something else to get e.g. endpoint.internalapi.cloud.example.org11:53
jistrbut i cannot get internal.cloud.example.org for example11:53
jaosoriorjistr: no;  you could do it with the first submission though.11:54
jaosoriorjistr: actually, even with patchset #411:54
tremblejaosorior: yes11:54
*** limao_ has joined #tripleo11:54
jistryea, but i still couldn't get e.g. internal-cloud.example.org11:55
jistri wonder if that's an issue or not11:55
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Print the execution ID when starting workflows  https://review.openstack.org/34071111:55
openstackgerritDougal Matthews proposed openstack/tripleo-common: Remove the underscore from the base action client methods  https://review.openstack.org/33592011:55
jistrjaosorior: we don't do any DNS setup within tripleo itself, both the public and internal VIP DNS recors have to be set up manually by the operator, right?11:56
jistrif so then i wonder what's the benefit of restricting those names to a certain pattern11:56
jaosoriorjistr: actually, with patchset #4; yes you could have gotten internal-cloud.example.org11:56
* jistr looks at ps #411:57
jaosoriorjistr: https://review.openstack.org/#/c/357765/11:57
*** limao has quit IRC11:57
*** dsariel has quit IRC11:57
jaosoriorjistr: that CR creates a composable service that will write those entries to /etc/hosts11:57
jaosoriorjistr: and it includes an environment to disable that in case the operator has a working DNS11:57
openstackgerritmathieu bultel proposed openstack/tripleo-common: Do only a strick grep on nova show status  https://review.openstack.org/35399611:58
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service  https://review.openstack.org/35643012:04
*** nyechiel has quit IRC12:04
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service  https://review.openstack.org/35643012:06
jaosoriordprince: Hey man, can you check this out? ^^12:06
jaosoriorI'm trying to set up certmonger for TLS (the main usage is internal TLS, but external TLS will be possible to if needed) And I'm doing it as a composable service12:07
jistrjaosorior: i think even ps #4 assumes that CloudDomain will be part of the internal endpoint always, if i read correctly. E.g. i could get internal.cloud.example.org but not internal-cloud.example.org... I'm not sure this is a big deal, but still wondering why are we putting these restrictions in, and why we want CloudInternalEndpointName behave inconsistently wrt CloudName (which would ideally be named12:07
jistrCloudPublicEndpointName). One doesn't include the CloudDomain, one does -- i still don't fully follow why would we do it like this?12:07
jaosoriorjistr: So you want to get rid of the usage of CloudDomain?12:07
jaosoriorjistr: CloudName doesn't include it because there is a possibility that the deployer will configure a different domain for the external endpoints12:08
jaosoriorjistr: however, the internal domain might be managed by another entity (IT maybe?) and that would fall somewhere else. Which is why there's the difference12:09
jistrjaosorior: yea, for the endpoints i'm thinking why not. Is there a reason we should bundle together how we name the individual nodes, vs. how are the VIP endpoints in Keystone named? I bet usually people will want to have it like this, but wondering what's the gain in excluding the other use cases when the operator wants to set up their DNS differently.12:09
jistr(the "yea, for the endpoints i'm thinking why not" was a response to "So you want to get rid of the usage of CloudDomain?")12:10
jaosoriorjistr: I'm not sure I follow12:10
jistri think we'd still want to keep using the CloudDomain to name individual nodes12:10
jaosoriorjistr: basically I want to give SOME support to having FQDNs12:10
jaosoriorcurrently there is none12:11
jaosoriorand this is a problem if we want TLS in the internal endpoints (which some customers expect)12:11
jaosoriorjistr: and CloudDomain is still being used for that. Not sure what the issue is.12:11
openstackgerritMatthew Thode proposed openstack/diskimage-builder: Update GRUB_MKCONFIG for detecting what's installed  https://review.openstack.org/35858812:12
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: remove devmode  https://review.openstack.org/35741812:12
*** maeca1 has quit IRC12:12
*** maeca1 has joined #tripleo12:13
jaosoriorjistr: nothing changes except for the possibility of adding FQDNs which can differ per-network in the keystone endpoints12:13
jistrjaosorior: yea i see. Basically what i'm saying is -- for public endpoint we specify the whole FQDN, why not do the same for internal endpoint FQDN too? Why have those two params behave differnetly? Why magically prepend CloudDomain for one but not the other?12:13
shadowermandre: this review has just a -1 without any comments: https://review.openstack.org/#/c/354775/12:13
jaosoriorjistr: because it makes no sense to have a clouddomain parameter that we don't use. And have the deployer type everything (which is very prone to errors)12:14
shadowermandre: any chance you forgot to add it?12:14
*** jeckersb_gone is now known as jeckersb12:14
*** jdob has joined #tripleo12:14
jaosoriorjistr: in the first patchsets I actually wanted to use CloudDomain for the external endpoint too. But yeah, I do agree that that specific one might be different.12:15
*** limao has joined #tripleo12:15
openstackgerritMatthew Thode proposed openstack/diskimage-builder: Update portage only if needed  https://review.openstack.org/35859212:16
*** pradk has joined #tripleo12:16
skramajafor DPDK feature, i have a requirement to set kernel args to for compute nodes, via first-boot scripts "OS::TripleO::NodeUserData".. but AFAIU user data is applied to all nodes including controller.. is there any way to apply user data to only compute node?12:18
*** limao_ has quit IRC12:19
jaosoriorjistr: I could set all those parameters to be FQDNs, that's actually not hard. I just think that it will be a pain for the deployer to have to repeat stuff that often12:19
*** lucasagomes is now known as lucas-hungry12:20
jistrjaosorior: yea i see. just feels a bit strange that we're passing full FQDN for public enpoint and just a part of FQDN for the internal one, and have the rest autocompleted (and maybe in some cases the autocomplete might not do what the operator would prefer wrt naming?). I'm not going to -1 but perhaps this is an issue that might warrant a look from others too. There are other folks on our team who have12:21
jistrmore experience with ops than i do :)12:21
jistrpersonally i like repeating myself a bit more than dealing with interface that feels internally inconsistent12:22
jistrthoug i agree that objectively it can be a tough decision when picking between those12:23
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts  https://review.openstack.org/35776512:25
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable usage of FQDNs for the endpoints  https://review.openstack.org/35641612:25
jaosorioralright, so here's the version that has all the parameters consistent ^^12:25
jaosoriorjistr: ^^12:26
jaosoriorjistr: one has to write the whole thing now, but alright, at least there's consistency now.12:26
*** jaosorior is now known as jaosorior_brb12:26
*** asbishop has joined #tripleo12:27
openstackgerritAna Krivokapic proposed openstack/instack-undercloud: Add default plan creation  https://review.openstack.org/34953212:28
*** adarazs_lunch is now known as adarazs12:30
*** rbowen has joined #tripleo12:31
jistrjaosorior_brb: thanks. Yea personally i'd prefer that one, just +1'd for now to wait for other's feedback too, in case someone disagrees on that :)12:31
*** vijaykc4 has joined #tripleo12:33
hewbroccaThe pursuit of a foolish consistency is the false hobgoblin of little minds12:33
hewbrocca:D12:33
openstackgerritSagi Shnaidman proposed openstack/tripleo-heat-templates: Fix path to sahara templates  https://review.openstack.org/35860712:34
openstackgerritErno Kuvaja proposed openstack/tripleo-heat-templates: Don't spread admin keys to ceph-clients  https://review.openstack.org/35860912:34
mandreshadower: dunno what happened to my comments on https://review.openstack.org/#/c/354775/, re-posted... thanks12:35
shadowernp12:35
*** lblanchard has joined #tripleo12:36
jistrhewbrocca: you've been learning Czech a lot recently, i see :D12:36
*** pgadiya has quit IRC12:36
*** limao has quit IRC12:36
*** limao has joined #tripleo12:37
*** rhallisey has joined #tripleo12:37
hewbroccaheh heh12:37
*** bfournie has quit IRC12:38
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: Composable Zaqar services  https://review.openstack.org/33168212:39
*** xuao has quit IRC12:39
*** zaneb has joined #tripleo12:40
*** rlandy has joined #tripleo12:40
*** jcoufal has joined #tripleo12:41
d0ugalAnyone want to review the Mistral deploy patch? It's all green and has had extensive testing. https://review.openstack.org/#/c/298732/12:41
d0ugal(It isn't actually used until a CLI patch picks it up, and that can't pass CI until this lands)12:42
thrashd0ugal: I'll take a look.12:46
d0ugalthrash: Thanks sir!12:47
thrashd0ugal: can it be run as a standalone workflow?12:47
d0ugalthrash: To test it?12:48
thrashd0ugal: si12:50
d0ugalthrash: Yeah, it can, but it is much easier to use https://review.openstack.org/#/c/342261/12:50
d0ugalthrash: Otherwise you need to manually set parameters etc. and call a few different actions and workflows12:51
*** vijaykc4 has quit IRC12:51
larskshey folksL what is the 'nova boot' command line that will select a node based on the properties in ironic? I would like to manually build an overcloud node on a system tagged with a specific property...12:52
Jokke_I see lots of these [Zuul] standard output/error still open after child exited Anyone familiar what's going on? Not a single job, but multiple jobs on multiple PSs12:52
larsksOh, that happens via flavor extra_specs...12:54
*** ccamacho is now known as ccamacho|lunch12:56
*** rwsu has quit IRC12:57
*** tzumainn has joined #tripleo12:57
*** limao has quit IRC12:57
thrashd0ugal: ack12:57
*** limao has joined #tripleo12:58
d0ugalthrash: It's unfortunately not that easy to test, but I can walk you through it if you want12:58
openstackgerritMerged openstack/tripleo-common: Add backwards compatibility for openstackclient  https://review.openstack.org/35714712:58
*** mgagne_ has quit IRC13:00
*** mgagne_ has joined #tripleo13:00
*** egafford has quit IRC13:00
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Rework the pacemaker_common_functions for M..N upgrades  https://review.openstack.org/32102713:03
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: WIP Add a function to upgrade from full HA to NG HA  https://review.openstack.org/35862613:03
openstackgerritRyan Brady proposed openstack/tripleo-common: Add Plan Deletion  https://review.openstack.org/29993713:04
*** jpena|lunch is now known as jpena13:06
jpichd0ugal: Would it be worth considering writing up some rough testing instructions into the blueprint or an etherpad, so people don't need to poke you each time?13:07
*** morazi has joined #tripleo13:08
d0ugaljpich: Yeah, probably a good idea.13:09
d0ugaljpich: I started writing actual documentation for the Mistral API earlier13:09
d0ugalbut some rough notes might be worth doing first.13:09
jpichd0ugal: Oooh nice :)13:09
jpichd0ugal: Testing in-progress patches can be a bit different yeah13:10
*** abregman_ is now known as abregman|mtg13:10
jpichI tried to write up some instructions on testing CLI patches with in-progress tripleo-common reviews last week and it makes it all look more complex than it probably deserves to be :)13:10
sshnaidmslagle, hi, are you around?13:12
slaglesshnaidm: yes, hi13:12
sshnaidmslagle, hi, I'd like to know how do you indicate the problem with ironic ssl on undercloud you resolved before. I suspect it's back on last nonha jobs, but can't find something in logs, just wanted to be sure it's same or different issue13:14
slaglesshnaidm: the issue was during instance deployment13:15
slaglesshnaidm: unfortunately i had to setup a local environment in order to see the error on the console from ironic-python-agent13:16
slaglesince that log is not available anywhere13:16
*** bfournie has joined #tripleo13:16
*** dtantsur|brb is now known as dtantsur13:16
slagledtantsur|brb: lucas-hungry : any thoughts about that ^? how can we save the output from ipa if it encounters an error (like the ssl one from last week)13:16
hewbroccaIt would sure be nice to have something like rsyslog for i-p-a13:17
dtantsurslagle, it's hard to save anything, if IPA can't access Ironic13:17
dtantsurhewbrocca, there is a spec up for exactly this, but it should be easy to set up even now13:17
dtantsurwe need to modify IPA defaults to use syslog though13:17
*** lucas-hungry is now known as lucasagomes13:18
hewbroccaSeems like a great idea13:18
*** julim has joined #tripleo13:18
dtantsurslagle, what we do in our upstream gate is saving output of `virsh console`13:19
dtantsurcould help tripleo too. lemme see how we do that13:19
slagledtantsur: yea, was just thinking about that13:19
slagleit would have be handled externally from our CI job13:19
slaglesince our actual jobs have no knowledge of anything "virsh" related13:20
dtantsurwe seem to set up a qemu hook like https://github.com/openstack/ironic/blob/master/devstack/files/hooks/qemu13:20
dtantsuroh true...13:20
dtantsurwell, it's OVB, right? so we have nova serial console13:20
slagleyea, we'd just have to call it after the job somehow13:20
*** trozet has joined #tripleo13:21
slaglesshnaidm: i think we could add this to destroy-env ^^13:21
*** limao has quit IRC13:21
slaglethat is the script that cleans up the ovb created testenv13:21
thrashd0ugal: ok.. I give. what's the secret sauce?13:21
d0ugalthrash: lol :) one sec13:22
slaglesshnaidm: we could add something there to capture the serial console and upload it somewheres13:22
sshnaidmslagle, I think such things are in postci13:22
d0ugalthrash: writing it out here: https://etherpad.openstack.org/p/tripleo-mistral-example13:22
sshnaidmslagle, ok, I'll look at it13:22
lucasagomesslagle, dtantsur a possible solution for getting the logs at that time would be to setup a syslog server to listen and collect the logs from the machines booting13:22
sshnaidmslagle, I see there "Deployment to node a008cf04-fbd8-4f97-9309-a2a8d53576d9 done", but it wasn't successful in your problem, right?13:22
*** radeks has quit IRC13:23
lucasagomesslagle, dtantsur you can configure systemd to redirect the logs to syslog13:23
*** limao has joined #tripleo13:23
slaglesshnaidm: i'm not sure we can do it from postci, but if you're able to, that would be fine too13:23
sshnaidmslagle, ok, I'll investigate how to do it better13:23
slaglesshnaidm: right, the ssl failures werent getting that far. can you link me a failed job?13:23
sshnaidmslagle, every nonha now, i.e.: http://logs.openstack.org/88/254788/39/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/4cb01f8/console.html#_2016-08-21_16_21_41_75027513:24
dtantsuris it still ironic problem? can we help somehow?13:24
dtantsurthe patch fixing our endpoint detection was merged a few days ago13:25
d0ugalthrash, jpich: The top of this should be enough: https://etherpad.openstack.org/p/tripleo-mistral-example13:25
d0ugalI'll expand on it if you hit issues.13:25
slagledtantsur: not sure yet13:25
dtantsurack13:25
slaglei think the ssl issue is fixed. we got a successful promote last night13:25
slaglew00t, btw13:25
slaglefresh promote13:26
jpichd0ugal: Sweet, thank you!13:26
dtantsurpromote \o/13:26
d0ugalshadower: I highjacked that etherpad to add other mistral testing notes btw13:26
dtantsurwe no longer get fresh clients from git master, right? only upper-constraints?13:26
shadowerno problem :-)13:26
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates: Add system-uuid based hostname entries  https://review.openstack.org/35864313:26
lucasagomesslagle, sshnaidm is it OVB ? If so you may want to take a look at https://review.openstack.org/#/c/257987/13:27
lucasagomesslagle, sshnaidm we use that in our gate to save the console logs from the testing VMs13:28
sshnaidmlucasagomes, thanks, will look13:29
*** adarazs is now known as adarazs_brb13:29
lucasagomessshnaidm, np13:29
*** sshnaidm has quit IRC13:36
*** cdearborn has joined #tripleo13:38
*** limao_ has joined #tripleo13:40
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Availability monitoring agents support  https://review.openstack.org/25478813:41
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Add support for deploying Ceph RGW role  https://review.openstack.org/28902713:43
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: [NO MERGE] Test Ceph RadosGW as replacement for Swift  https://review.openstack.org/35718213:43
*** limao has quit IRC13:43
*** abregman|mtg is now known as abregman13:44
*** dtrainor has joined #tripleo13:45
*** ebarrera has quit IRC13:46
*** dtrainor has quit IRC13:47
*** salmank has joined #tripleo13:47
*** dtrainor has joined #tripleo13:47
ccamacho|lunchHey guys FYI my undercloud only have 8GB and now getting OOMs also there :P which leads to the "no hosts available error" Trying now with 9GB13:49
*** ccamacho|lunch is now known as ccamacho13:50
socialccamacho: add some swap too, 2GB at least13:50
ccamachohey! social thanks for the tip13:50
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Migrate to using osc-lib  https://review.openstack.org/33546013:50
*** sshnaidm has joined #tripleo13:51
slagleccamacho: what's using all the memory?13:52
*** adarazs_brb is now known as adarazs13:52
*** rbowen has quit IRC13:52
pradkcan i get some reviews on https://review.openstack.org/#/c/352476/ and https://review.openstack.org/#/c/293642/ would like to get these merged13:53
*** dmacpher has joined #tripleo13:55
ccamachoslagle in the undercloud when running the deployment the mysql and amqp are taking lot of RAM, what I'm doing now it's adding more RAM and now monitoring all the processes until it finishes, the issue it's that with the OOM error its hard to debug for all the other non related errors. Ill let you know when it finishes completely which processes and in which part of the deployment are taking the RAM away.13:58
*** egafford has joined #tripleo14:01
*** limao has joined #tripleo14:02
dtantsurderekh, any ideas about this TFTP problem?14:03
derekhdtantsur: not yet, still looking14:04
*** limao__ has joined #tripleo14:04
dtantsurokie, thanks14:04
*** limao_ has quit IRC14:05
*** limao has quit IRC14:07
gfidenteakrivoka, nice pointing that out14:07
gfidenteI think I tested it locally with a string having both []14:07
gfidenteand given I had the result I expected I went ahead, but in devil is in the details as usual :)14:08
*** pkovar has quit IRC14:09
*** jcoufal has quit IRC14:10
*** pkovar has joined #tripleo14:10
akrivokagfidente: yeah :)14:10
*** cinerama has quit IRC14:11
*** jcoufal has joined #tripleo14:11
derekhdtantsur: I have reproduced the problem btw , my first few attempts failed because of memory issues14:12
dtantsurso it's not only my environment, good (or not very good :)14:12
*** rbowen has joined #tripleo14:18
*** radeks has joined #tripleo14:18
*** pkovar has quit IRC14:19
*** pkovar has joined #tripleo14:20
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart: Adds option to ci-script to mange image in OVB  https://review.openstack.org/35547614:20
*** bootsha has quit IRC14:20
*** sshnaidm has quit IRC14:21
thrashd0ugal: ERROR: Failed to validate: :14:23
thrash resources.ControllerServiceChain: : Failed to validate nested template: The template is not a JSON object or YAML mapping."14:23
shadowerjtomasek: does the GUI generate the overcloudrc file at the end of the deployment?14:24
jtomasekshadower: it does not14:24
jtomasekshadower: I am not sure how was it done before14:25
shadowerjtomasek: oh awesome.14:25
jtomasekshadower: We're able to trigger some Mistral action/workflow that does that after deployment finishes14:26
shadowerjtomasek: so as far as I can tell, it's currently generated by python-tripleoclient: https://github.com/openstack/python-tripleoclient/blob/f666339199165d447259ace45aea277d24f877ac/tripleoclient/v1/overcloud_deploy.py#L100614:26
*** abregman_ has joined #tripleo14:26
shadowerjtomasek: and unless I missed something, the switch to mistral isn't changing that: https://review.openstack.org/#/c/342261/14:27
*** limao__ has quit IRC14:27
*** abregman has quit IRC14:27
*** paramite has joined #tripleo14:27
*** limao_ has joined #tripleo14:28
jtomasekshadower: hm, if that file is required, we need to move its creation into a mistral action14:28
shadowerd0ugal:  ^ afaict the overcloudrc is only generated in python-tripleoclient even after the move to mistral14:29
jtomasekshadower: in addition the file itself is useless for GUI and GUI will need to display some relevant data too14:29
shadowerdid I miss something?14:29
shadowerjtomasek: yea that's where I was heading: we need to show that info and I wondered how does the gui does it?14:30
jtomasekshadower: what data does it contain? and what does it use to get them?14:30
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart: Adds option to ci-script to mange image in OVB  https://review.openstack.org/35547614:32
shadowerjtomasek: most importantly the admin password and keystone endpoint to talk to the overcloud14:32
d0ugalthrash: huh, weird - do you have the full traceback?14:33
thrashd0ugal: yes. one sec.14:33
*** bootsha has joined #tripleo14:33
d0ugalshadower: That is correct, we really need to move that to Mistral14:33
d0ugalshadower: but I really want to get some sort of Mistral deploy in CI first14:34
thrashd0ugal: http://paste.openstack.org/show/562228/14:34
shadowerd0ugal: yeah, I understand. Not being alarmist, but I'm finally trying to understand the whole thing and I'm asking questions along the way :-)14:34
*** cinerama has joined #tripleo14:34
d0ugalthrash: and that was just with a standard --templates deploy? no other envs etc?14:34
*** rbowen has quit IRC14:35
d0ugalshadower: Gotcha :)14:35
shadowerespecially since some of tihs stuff affects validations, too14:35
shadowerd0ugal: are you aware of a BZ for this? I'll create it otherwise14:35
d0ugalshadower: No, we should create one. Thanks14:35
shadowerand when I say BZ I mean a LP  of course :-)14:35
thrashd0ugal: no.. I have some other envs... but nothing crazy.14:35
thrashd0ugal: and tht is master14:36
d0ugalthrash: dang, I'll try deploying master. If that works I might ask for your envs to test with.14:37
*** sshnaidm has joined #tripleo14:37
*** social has quit IRC14:38
*** ayoung has joined #tripleo14:41
openstackgerritDougal Matthews proposed openstack/tripleo-docs: [WIP] Mistral API Documentation  https://review.openstack.org/35868514:41
d0ugalthrash: btw, does it mean anything to you? any idea what could be wrong?14:43
shadowerjtomasek: another question: does the UI ever talk to Heat directly (say for endpoints, etc.) or always through Mistral?14:45
jtomasekshadower: yes, also directly14:45
shadowerjtomasek: thanks14:45
thrashd0ugal: not yet.14:45
*** limao_ has quit IRC14:45
thrashd0ugal: have you done any deployments with the composable services?14:46
*** limao has joined #tripleo14:46
d0ugalthrash: no, I don't know much about that tbh14:46
thrashd0ugal: any deployments from N-cycle tht?14:49
d0ugalthrash: I've mostly been deploying the latest promoted t-h-t packages14:50
d0ugalthrash: so they should be close to master, right?14:50
thrashd0ugal: yes14:50
d0ugalthrash: Master is deploying now, it started fine.14:50
d0ugalthrash: so I guess I need your envs to see what is tripping it up14:50
thrashclose enough... so, yeah, something on my end. Let me take out envs first.14:50
*** bootsha has quit IRC14:53
*** Ryjedo has joined #tripleo14:56
d0ugalthrash: gah, I am now hitting an issue with the CLI patch and the new osc_lib.14:56
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: Ironic: add missing haproxy and firewall configuration  https://review.openstack.org/35653914:57
thrashd0ugal: removed all envs and it worked.14:59
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: Ironic: add missing haproxy and firewall configuration  https://review.openstack.org/35653914:59
thrashso let me package them up for you14:59
d0ugalthrash: Thanks14:59
thrashd0ugal: actually easy enough.14:59
*** ayoung has quit IRC14:59
thrashI was using $THT/environments/puppet-pacemaker.yaml14:59
thrashtripleo-ci/test-environments/worker-config.yaml and the puppet-modules.yaml to pull puppet from swift15:00
*** jaosorior_brb is now known as jaosorior15:00
*** numans has quit IRC15:02
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart: Adds option to ci-script to manage image in OVB  https://review.openstack.org/35547615:05
*** vijaykc4 has joined #tripleo15:05
*** limao has quit IRC15:06
*** limao has joined #tripleo15:07
*** yamahata has joined #tripleo15:08
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Add manifests to inject and trust CA certificates  https://review.openstack.org/35302115:09
*** radeks has quit IRC15:09
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add deployment of CAs via hieradata  https://review.openstack.org/35303715:10
jaosoriorjistr, gfidente can you guys check this out? https://review.openstack.org/#/c/356488/415:11
jaosoriorwould reduce the places to maintain endpoint related stuff for newton and beyond15:11
sshnaidmslagle, did you have a chance to look at job I pasted?15:12
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates: Fix check of rpm-python.  https://review.openstack.org/35702215:12
slaglesshnaidm: not in depth15:13
slaglewas updating my local env15:13
slaglewill see if it repros15:13
sshnaidmslagle, ok15:13
sshnaidmslagle, do you use tripleo-quickstart for it?15:13
slaglesshnaidm: that is the heat api that it can't talk to fwiw15:13
slaglesshnaidm: i do not use quickstart15:14
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts  https://review.openstack.org/35776515:17
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable usage of FQDNs for the endpoints  https://review.openstack.org/35641615:17
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart: Add option to ci-script to manage image in OVB  https://review.openstack.org/35547615:18
sshnaidmderekh, we need to take some time to update the dev env on CI, or explaining me how to create the new one. It is soooooo extremely helpful in investigations15:18
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: set up quickstart to replace instack-virt-setup  https://review.openstack.org/35808915:18
jaosoriorgfidente: for the answer to your comment, here's the commit https://review.openstack.org/#/c/357765/715:18
gfidentejaosorior, right that's why you wanted netipmap as output of endpoint maap15:19
jaosoriorgfidente: yep15:19
derekhsshnaidm: sure, are you using rh2?15:19
sshnaidmderekh, currently it's on rh2, but I don't know where is better to have it now15:20
sshnaidmderekh, does it make a difference?15:20
*** saneax is now known as saneax-_-|AFK15:21
derekhsshnaidm: we should have one setup on rh1, presumably some day weshay will want rh2 back15:21
hewbroccaderekh: yes, but to be clear, I think we'll keep rh2 available for TripleO CI until RDO Cloud is up and running15:22
derekhdtantsur: your problem is iptables on the client hosts15:22
sshnaidmderekh, right15:22
jaosoriorslagle: are we still using this element? https://github.com/openstack/instack-undercloud/blob/master/elements/undercloud-install/os-apply-config/var/opt/undercloud-stack/masquerade15:22
hewbroccathat'll be a few months yet15:22
derekhhewbrocca: ok15:22
dtantsurderekh, you mean undercloud?15:23
sshnaidmdoes anybody know about ETA for RDO cloud?15:23
hewbroccasshnaidm: too early to say15:23
hewbroccaGraeme is the one putting it together15:23
hewbroccaI would hope, not too long15:24
derekhdtantsur: after the client talks to the tftp server the server sends a udp packet back to whatever port the client is listening on, this is whats being blocked15:24
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Move out of roles any Ceph setting  https://review.openstack.org/35665715:24
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Set cinder::api::bind_host within service template  https://review.openstack.org/31834415:24
*** _coolsvap_ is now known as coolsvap15:24
*** vijaykc4 has quit IRC15:24
dtantsurderekh, I didn't see any responses when listening on overcloud-controller-0 br-ex... have you seen them?15:25
dtantsuralso this does not explain while nodes have the similar problem when booting15:25
sshnaidmhewbrocca, just curious if it's question of months or year15:25
derekhdtantsur: what filter were you listening for15:25
hewbroccasshnaidm: it had better not be more than two months or so15:25
*** nyechiel has joined #tripleo15:26
hewbroccafor the initial rollout, at least15:26
sshnaidmhewbrocca, cool, good to have it soon..15:26
derekhdtantsur: adding this to my undercloud iptables chain allowed things to work "-A INPUT -j ACCEPT"15:26
dtantsurderekh, tcpdump -i br-ex port 6915:27
dtantsurderekh, hmm, interesting. thanks, I'll retry once my environment finishes rebuilding.15:27
dtantsurmaybe the nodes have a different problem...15:27
derekhdtantsur: port 69 only gets the setup udp packet, try "tcpdump -i any -nn udp"15:27
*** leanderthal is now known as leanderthal|afk15:27
*** limao has quit IRC15:28
dtantsuraha! makes sense, thanks15:28
*** limao has joined #tripleo15:28
slaglejaosorior: yes15:29
jaosoriorslagle: thanks15:29
jaosoriorgfidente: hey dude, so how was your idea of using for_each in https://review.openstack.org/#/c/357765/7 ??15:29
slaglejaosorior: that specific iptables rule is only used if routing traffic through the undercloud15:29
slaglejaosorior: as is done in virt setups15:29
*** limao_ has joined #tripleo15:30
*** rbowen has joined #tripleo15:30
gfidentejaosorior, basically to set a repeat as value of tripleo::vip_hosts::hosts_spec15:30
*** salmank has quit IRC15:31
jaosoriorgfidente: but doesn't that take lists? And in our case we have a dict?15:33
*** limao has quit IRC15:33
*** social has joined #tripleo15:33
gfidentejaosorior, it takes a list of keys to be replaced15:34
gfidentebut it also emits a list and not a dict15:35
gfidenteI'm using it here https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/ceph-mon.yaml#L7915:36
gfidentebut if you can't find an easy fit it's not urgent, I'd just remove the hiera() calls and use get_param15:37
*** ifarkas is now known as ifarkas_afk15:37
*** nyechiel has quit IRC15:40
*** tremble has quit IRC15:40
*** zoli_gone-proxy is now known as zoliXXL15:41
*** tbonds_ has quit IRC15:41
jaosoriorgfidente: thanks for the example dude; I think I got it15:42
bandinichem: is there a way for me to trigger the upgrade job on a specific review? I would like to test an upgrade-only change (namely this one: https://review.openstack.org/358626)15:42
abregman_hey. my undercloud installation fails with the following error: http://paste.openstack.org/show/562241/ -> any idea why..? specifically for mitaka..liberty works fine15:42
gfidentejaosorior, it's a sort of str_replace in a loop15:43
gfidentejaosorior, I think it could help write a little less duplicated code around the networks15:44
gfidentebut it's not so human friendly to parse :P15:44
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts  https://review.openstack.org/35776515:45
jaosoriorgfidente: how about that?15:46
*** rbowen has quit IRC15:46
chembandini: well it really depends on your setup, AFIK, oooq has an $OPT_GERRIT environment option and an update.sh script (that I didn't use yet) for testing update.15:46
chembandini: sorry the OPT_GERRIT is for oooq not for the code inside15:46
bandinichem: oh I thought we had an upgrade job somewhere15:46
chembandini: that oooq using the update.sh script15:47
gfidentejaosorior, exactly looks nice no/15:47
gfidente?15:47
jaosoriorgfidente: yeah dude, looks better. Just didn't properly know how to use that function. Thanks15:47
bandiniabregman_: you need a later kernel :/15:47
bandiniabregman_: the ipv6.ip_nonlocal_bind was a backport that happened after rhel7 ga15:48
*** limao_ has quit IRC15:48
abregman_bandini: hmm I'm using rhel 7.215:48
gfidentejaosorior, so I think the resulting format is a list though so is that okay with tripleo::vip_hosts::hosts_spec or should that be string?15:48
bandiniabregman_: is the ipv6 module loaded?15:48
bandiniabregman_: what's your kernel version?15:48
chembandini: one such command : $WORKSPACE/tripleo-quickstart/ci-scripts/upgrade.sh mitaka delorean minimal periodic current-passed-ci True True master15:49
abregman_bandini: hmm I need to provision new setup..just a moment15:49
jaosoriorgfidente: we don't need those entries anymore. But I did think it would be useful. I was thinking of using those for getting the service certificate15:51
chembandini: but I don't know how you specify the GERRIT you need.15:51
jaosoriorbut I guess I can add them when I'm actually using them15:51
*** milan has quit IRC15:51
*** zoliXXL is now known as zoli|gone15:51
jaosoriorgfidente: I'll delete them, and re-add them when I actually take them into use15:51
openstackgerritAna Krivokapic proposed openstack/python-tripleoclient: Fix unbracket_ipv6()  https://review.openstack.org/35872915:52
akrivokagfidente: ^15:52
bandinichem: ack, letme take a look there. thanks!15:53
*** zoli|gone is now known as zoli_gone-proxy15:53
openstackgerritJeff Peeler proposed openstack/tripleo-common: Add List Roles Action  https://review.openstack.org/33028315:53
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts  https://review.openstack.org/35776515:54
*** hewbrocca is now known as hewbrocca-afk15:58
*** weshay is now known as weshay_afk15:58
jaosoriorgfidente: hey dude, got time to check a commit?15:58
jaosoriorgfidente: doesn't need to be in-depth15:58
*** gchamoul is now known as gchamoul|afk15:58
*** gchamoul|afk is now known as gchamoul15:58
*** gchamoul is now known as gchamoul|afk16:00
jaosoriorgfidente: So this is the stuff that I'm doing for getting the HAProxy certificates via certmonger in the overcloud https://review.openstack.org/#/c/356430/16:00
jaosoriorgfidente: it's a POC and still needs a lot of cleaning16:00
jaosoriorbut overall, what I'm thinking is having it as a composable service16:00
jaosoriorgfidente: what I'm not sure about16:00
jaosorioris having it that way16:01
jaosorioror have a sub-resource that's included inside the already existing haproxy profiles16:01
jaosoriorgfidente: so something like the base profile. But dedicated for TLS16:01
sshnaidmjaosorior, can you please take a look in your time? https://review.openstack.org/#/c/358607/16:02
jaosoriorgfidente: so the main use is to have TLS for the internal endpoints in haproxy be managed by certmonger (with a real CA), but the public endpoints could be done too16:02
*** sshnaidm has quit IRC16:04
*** lazy_prince has quit IRC16:05
*** lazy_prince has joined #tripleo16:07
*** 7GHABUWA1 has joined #tripleo16:09
gfidentejaosorior, this is complicated16:11
jaosoriorgfidente: haha sorry about that16:12
gfidentejaosorior, I need you to speed me up so I can be useful16:12
gfidenteotherwise I would just comment random bits withouth picture :)16:12
ccamachoslagle heat-engine FTW, spawning 4 processes taking about the 25% of the memory (Node with 9GB) do you think its a good idea maybe to reduce it to 1 worker in the UC? But not sure about the performance impact.16:15
shadowerd0ugal: launchpad for the overcloudrc thing: https://bugs.launchpad.net/tripleo/+bug/161572016:15
openstackLaunchpad bug 1615720 in tripleo "overcloudrc should not be managed by the CLI" [Medium,Triaged]16:15
openstackgerritCarlos Camacho proposed openstack/instack-undercloud: Reduce the heat-engine workers to 1  https://review.openstack.org/35874316:15
jaosoriorgfidente: lets have a call tomorrow, and if someone else wants to join that would be great. Thanks for taking a look at it dude16:16
slagleccamacho: i dont think so16:16
gfidentejaosorior, yeah that's be nice indeed :)16:16
slagleccamacho: we've been advised in the past to not run with 1 heat-engine worker16:16
openstackgerritgreghaynes proposed openstack/diskimage-builder: Spec for changing the block device handling  https://review.openstack.org/33694616:16
ccamachoyeah?16:16
ccamachoack then16:16
d0ugalshadower: Thanks!16:16
slagleyes, due to rpc timeouts16:16
*** jpich has quit IRC16:16
ccamachommmm I imagine that something similar could happen16:16
*** yamahata has quit IRC16:16
*** alhernan has joined #tripleo16:17
*** skramaja has quit IRC16:17
*** skramaja has joined #tripleo16:18
*** karthiks has quit IRC16:19
*** fragatina has quit IRC16:19
*** fragatina has joined #tripleo16:19
jaosoriorccamacho: if you want to change that number of workers, I suggest you use hieradata_override and post a commit to tripleo-ci. Unless we agree that this is something we should make configurable through a config option in the undercloud.16:22
jaosoriorccamacho: I think it would be a good change to reduce the number of workers. maybe to 216:23
*** abregman_ has quit IRC16:23
*** weshay_afk is now known as weshay16:24
ccamachojaosorior, yeahp reducing the amount of workers it will reduce the mem usage for sure, the thing its tha we might move the problem to other place like "rpc timeouts" performance issues.16:24
ccamachoin my case I reduce it  and manage to deploy without issues, but...16:24
jaosoriorccamacho: that is the case, but having it slightly lower (if 4 is the default... maybe 3?) wouldn't be too bad I guess16:24
ccamachonot ajajja that might work16:24
jaosoriorccamacho: anyway, I suggest you do a change in tripleo-ci, and try to use hieradata_overrides16:25
slaglethe current default is the # of cores, which i think is reasonable16:26
ccamachoI didn't pushed into tripleo-ci because this is currently not affecting CI its affecting my UC deployments16:26
ccamachoyeahp16:26
jaosoriorslagle: is that the default? Maybe I got confused but somewhere in the heat documentation it explicitly mentioned 416:26
slagleright, so i suspect it is due to heat engine not freeing memory16:26
ccamachoslagle the errors you were getting in CI re non host available16:26
ccamachoIn my case were re OOM in UC, can you post the submissions with those errors?16:27
jaosoriorccamacho: I tend to change the number of workers manually :/16:28
jaosoriorccamacho: what do you use to deploy? oooq?16:28
*** tesseract- has quit IRC16:28
slagleccamacho: it was from sagi earlier16:28
slaglecheck if the recent jobs are failing that way16:28
ccamachojaosorior, nope all deployed manually following the docs16:28
jaosoriorccamacho: alright16:28
ccamachoack Ill check it :)16:28
*** pcaruana has quit IRC16:29
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Don't trigger mariadb upgrade dump/restore when not needed  https://review.openstack.org/35875516:30
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates: Refactor upgrade checks.  https://review.openstack.org/35775016:30
*** karthiks has joined #tripleo16:31
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Permit usage of raw disks as backends for Swift storage  https://review.openstack.org/35084116:31
*** lucasagomes is now known as lucas-afk16:32
*** athomas has quit IRC16:36
slagleccamacho: i'm seeing failures like this one now: http://logs.openstack.org/82/357182/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/61dd73a/console.html16:37
slagleas opposed to No Valid Host16:37
jaosoriorslagle: Is that still the issue where ironic-python-agent gets SSL errors?16:39
ccamachohitting also this one https://review.openstack.org/#/c/355394/16:40
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Convert the full documentation to rST  https://review.openstack.org/35724816:41
jaosoriorslagle: by the way, if you have time for a review. A collegue came out with an idea to be able to use TLS without baking the cert into the image https://review.openstack.org/#/c/358457/ I still need to properly test it and fix the py3 issues. But some feedback on the approach would be appreciated :D16:41
gfidentetty tomorrow guys16:43
*** jbadiapa has quit IRC16:43
jaosoriorgfidente: have a good one16:44
*** Goneri has quit IRC16:44
ccamachosy16:45
*** rwsu has joined #tripleo16:46
*** gfidente has quit IRC16:46
*** fragatina has quit IRC16:47
*** bana_k has joined #tripleo16:49
*** Goneri has joined #tripleo16:50
*** sshnaidm has joined #tripleo16:51
*** yamahata has joined #tripleo16:52
openstackgerritgreghaynes proposed openstack/diskimage-builder: Add tests for building *-minimal images  https://review.openstack.org/18116216:53
*** jaosorior has quit IRC16:55
*** derekh has quit IRC16:55
*** lucas-afk is now known as lucasagomes16:58
*** bana_k has quit IRC16:58
openstackgerritMerged openstack/diskimage-builder: Add "audit"package to yum-minimal  https://review.openstack.org/35226317:00
*** bana_k has joined #tripleo17:04
*** fragatina has joined #tripleo17:09
*** jbadiapa has joined #tripleo17:10
*** abehl has quit IRC17:16
ccamachoslagle, jaosorior, re  I found these errors re  http://logs.openstack.org/82/357182/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/61dd73a/console.html      http://paste.openstack.org/show/562254/17:16
ccamachoA connection refused in several services17:17
dtantsurderek<TAB> :(17:20
dtantsurwho else might have an idea why TFTP won't work in overcloud, except for selinux and iptables?17:20
*** pkovar has quit IRC17:26
*** rajinir has joined #tripleo17:28
*** fragatina has quit IRC17:33
*** fragatina has joined #tripleo17:33
*** jbadiapa has quit IRC17:37
*** lucasagomes is now known as lucas-dinner17:43
*** akrivoka has quit IRC17:45
*** bana_k has quit IRC17:49
*** jdob has quit IRC17:50
*** Goneri has quit IRC17:50
*** jdob has joined #tripleo17:52
dtantsurfolks, when setting up network like in http://tripleo.org/basic_deployment/basic_deployment_cli.html#setup-the-overcloud-network, should the subnet match the subnet used in undercloud?17:55
*** ramishra has quit IRC17:55
*** ramishra has joined #tripleo17:55
*** tosky has quit IRC17:56
bnemec-ptodtantsur: If you're not using net-iso, then yes.17:56
*** bnemec-pto is now known as bnemec17:56
bnemecIf you are using net-iso, then it should be the external network.17:57
dtantsurwithout net-iso17:57
dtantsurso by default it must be in 192.0.2.0/24 subnet, right?17:57
dtantsurbnemec, ^^17:57
bnemecRight17:57
bnemecAnd make sure the allocation pools don't overlap any of the undercloud ranges.17:57
dtantsursure. thanks a lot bnemec, this is probably what I've been doing wrong all this time17:58
*** bana_k has joined #tripleo18:01
*** Goneri has joined #tripleo18:01
dtantsurJesus, it works! I owe you and Derek some beers one day :)18:09
*** dtantsur is now known as dtantsur|afk18:11
*** mcornea has joined #tripleo18:11
openstackgerritDmitry Tantsur proposed openstack/tripleo-docs: [WIP] Documentation for installing and using Ironic in overcloud  https://review.openstack.org/35401618:14
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Add constraint to prohibit balance-tcp from BondInterfaceOvsOptions  https://review.openstack.org/35507318:15
*** 7GHABUWA1 has quit IRC18:16
*** rbowen has joined #tripleo18:25
dprincerbrady, jpeeler: hi, re this https://bugs.launchpad.net/tripleo-common/+bug/160853918:29
openstackLaunchpad bug 1608539 in tripleo-common "fix the update_capabilities action input" [Undecided,In progress] - Assigned to Jeff Peeler (jpeeler-z)18:29
*** mgarciam has joined #tripleo18:29
dprincewhy do we now need to include 'True' 'False' in all these formats18:30
dprincewhy not simply have environments be an array (matching how heat handles them). Anything in the array is implied enabled.18:30
dprinceanything absent is implied disable.18:30
dprincejtomasek: ^^^18:31
* jtomasek looks18:35
jtomasekdprince: it is like that because that way it is simpler for GUI, and it was like that in TripleO API. I am ok with doing it your way if it is better for CLI (although it adds a bit of additional business logic to GUI)18:40
jtomasekdprince: It definitely makes sense18:41
dprincejtomasek: If the UI can handle either case I've got a slight preference for the less verbose data structure18:42
dprincejtomasek: I think I see your point about the extra business logic, I guess the UI perfers a slightly denormalized version of the data...18:43
*** florianf has quit IRC18:43
jtomasekdprince: the input used now is direct output of form data. to get an array of enabled environments, I'll need to process those and create an array. But it is fine18:44
*** maeca1 has quit IRC18:44
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Add constraint to prohibit balance-tcp from BondInterfaceOvsOptions  https://review.openstack.org/35507318:45
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Parameters listing and updating  https://review.openstack.org/35155418:48
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Refactor DeploymentPlan Component  https://review.openstack.org/35881418:48
jpeelerdprince: which way have you decided is best?18:49
dprincejpeeler: well, I certainly don't percieve it as being my decision18:50
dprincejpeeler: just asking the question as to why this format evolved the way it did18:50
jpeelerfair enough - just wondering if i'll need to redo anything18:50
*** mcornea has quit IRC18:51
dprincejpeeler: it is a JSON format that optimizes (de-normalizes) the data a bit to suite the UI. Which could be a fine thing, it just strikes me as an odd deviation from the way Heat proper manages environments18:51
dprincejpeeler: I like just treating them as an array, but I'd also say it could be managed either way. The more verbose way is just that18:53
dprincejpeeler: I dunno, ask around quickly perhaps. See what d0ugal and rbrady say.18:53
dprincejpeeler: if everyone is okay with it the way it is then perhaps we roll w/ it18:53
jpeelerthey already approved it18:53
dprincejpeeler: I will ask them18:54
* jpeeler checks to make sure he's looking at the correct review18:54
dprincejpeeler: https://review.openstack.org/#/c/349732/318:55
jpeelerryan should be around now18:55
jpeelerrbrady: see ~20 lines above18:55
rbradydprince: I have no strong opinion wrt to format.  I just want to land something already.18:55
dprincerbrady: yeah, I don't think it is a dealbreaker. And I'm keen to make progress on this18:56
rbradydprince: let me look at the CLI patch to see if it matters there (I am assuming it doesn't)18:56
dprincerbrady: my initial (a long time ago) CLI patch used an array18:56
dprincerbrady: heat uses an array. So I know it was possible... but I get that the UI handles things slightly differently18:57
*** jdob has quit IRC18:59
*** jdob has joined #tripleo19:01
*** Goneri has quit IRC19:01
rbradydprince: AFAIK, the CLI doesn't list/select envs at all19:03
rbradydprince: see https://review.openstack.org/#/c/34226119:04
*** alhernan has quit IRC19:04
dprincerbrady: yeah, at this point it doesn't19:07
dprincerbrady: perhaps I'm just wondering if we will at some point, to follow a similar UI'ish workflow19:08
rbradydprince: if that was the case, we would need commands in the cli to support displaying the list to users and then a command to select one or more environments19:09
*** mhenkel has quit IRC19:09
dprincerbrady: yep, all of this may come in time perhaps19:10
dprincejpeeler, rbrady: anyways I'm +2 here https://review.openstack.org/#/c/349732/319:10
dprincejpeeler: +A even19:10
jpeelerthanks!19:10
dprincejpeeler, rbrady: I'm waiting for a recheck on this. If it turns green ping me and we can send it19:11
rbradydprince: for now we might be able to use the UpdateCapabilities action from the CLI to ensure that function is covered in CI19:12
rbradydprince: we'd just need to hard code known environments19:12
jpeelerpending recheck for https://review.openstack.org/#/c/348537/ you mean?19:13
*** Goneri has joined #tripleo19:16
*** mgarciam has quit IRC19:17
thrashd0ugal: I've added back all of my envs with the packaged tht, and it worked.19:18
thrashd0ugal: so now to figure out what in master tht and my envs that are not working.19:18
dprincerbrady: commented here https://review.openstack.org/#/c/299937/1519:25
jpeelerit looks like a bulk delete still requires to specify all the files though. i said the same thing on patchset 7 and i thought that the API wouldn't allow simply deleting the container19:31
rbradyjpeeler, dprince: I don't think that option is supported in the swift client: http://paste.openstack.org/show/562275/19:32
rbradyjpeeler, dprince: no headers arg in delete_container: https://github.com/openstack/python-swiftclient/blob/master/swiftclient/client.py19:33
vkmc71595119:38
vkmcops sorry19:38
*** maeca1 has joined #tripleo19:41
jpeelerrbrady: does swift client have a bulk_delete command?19:41
*** rcernin has joined #tripleo19:41
*** ebarrera has joined #tripleo19:42
rbradyjpeeler: nope.19:42
rbradyjpeeler: I'm testing locally what it would take to expose the headers arg on delete (it's exposed in create) and see if the header dprince posted works.  if it does I'll push a patch to swiftclient19:44
dprincerbrady: yeah, the swift bulk middleware is there so we should be able to consume it I think19:45
dprincerbrady: perhaps similar to the create code?19:45
openstackgerritgreghaynes proposed openstack/diskimage-builder: Add element for setting sysctl values  https://review.openstack.org/35658219:45
dprincerbrady: just a thought, I think it'd be much faster. I hate to see us iterate over swift objects if we can avoid it19:46
rbradydprince: the swiftclient::put_container method exposes the headers arg, delete_container doesn't.  There's just no proper way to inject the header with swiftclient current state19:47
dprincerbrady: okay, so I think the opportunity here is lets add that to swiftclient?19:48
dprincerbrady: but not block on it19:48
rbradydprince: ack.  working on patch now19:48
dprincerbrady: comment back on the review that you investigated it and I can follow up and file a bug19:48
dprincerbrady: or you can :)19:48
rbradydprince: ack19:49
tdasilvadprince, rbrady: just fyi, i believe the bulk delete is a post request, not a delete request19:49
rbradytdasilva: the link pasted by dprince refers to "DELETE" request.  http://docs.openstack.org/juno/config-reference/content/object-storage-bulk-delete.html19:51
tdasilvarbrady: mm...i was going off of this doc: http://docs.openstack.org/developer/swift/middleware.html#bulk-delete19:52
*** beagles is now known as beagles_biab19:53
*** bfournie has quit IRC19:53
tdasilvarbrady: i've got a saio vm setup and will test19:53
tdasilvarbrady: also, i thought the swift client had a option to delete all objects in a container before deleting the container, but i could be wrong, let me double-check that too19:54
rbradytdasilva: weird.  it's like swift wasn't made by pythonistas - there's more than one way to do the same thing19:54
tdasilvarbrady: it's possible the first doc is also wrong....FWIW the juno/config-reference doc is not maintned by swift devs while the doc I referenced is...19:55
*** bnemec has quit IRC19:57
*** maeca2 has joined #tripleo20:02
*** ayoung has joined #tripleo20:03
*** maeca1 has quit IRC20:06
*** dbecker has quit IRC20:07
*** rbowen has quit IRC20:11
*** rbrady is now known as rbrady_20:15
*** rbrady_ is now known as rbrady20:16
*** asbishop has quit IRC20:16
*** dprince has quit IRC20:17
jpeeleris anybody else experiencing problems with the gate-tripleo-ci-centos-7-ovb-nonha job?20:17
*** jcoufal_ has joined #tripleo20:27
*** maeca2 has quit IRC20:28
*** jcoufal has quit IRC20:31
*** asbishop has joined #tripleo20:31
*** bfournie has joined #tripleo20:32
slaglewhat problems are you seeing?20:35
jpeelerslagle: the problems vary. this this case, it looks like a mirror issue:20:39
jpeelerhttp://logs.openstack.org/37/348537/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/0c7ed81/console.html20:39
*** rwsu has quit IRC20:39
jpeelerthis is for https://review.openstack.org/#/c/348537/20:40
jpeelerit passed on aug 9 and i don't think i've changed the code since then20:40
*** jprovazn has quit IRC20:44
colonwqjpeeler, slagle, mine just failed the same way. http://logs.openstack.org/56/347956/27/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/e31c0b8/console.html#_2016-08-22_20_35_08_17542120:47
*** maeca1 has joined #tripleo20:49
jpeelermine didn't get to registering the nodes, but yours got a bit farther20:51
slaglepabelanger: seeing a 404 trying to download metadata for the epel repo from the mirror server for rh1 ^20:52
slaglewonder if something is cached somewhere20:53
pabelangerslagle: that usually means the upstream server we mirror from is stale20:54
pabelangerslagle: I'll check the logs20:54
*** mhenkel has joined #tripleo20:55
*** yamahata has quit IRC20:56
*** yamahata has joined #tripleo20:56
pabelangerslagle: actually, looks like it was deleted just after you ran yum20:57
pabelangerso, recheck will fix it20:58
pabelangerbut there is a gap in our mirror that needs to be fixed20:58
pabelangerI'll work on a patch tomorrow20:58
rbradydprince: comment updated, bugs created in tripleo and python-swiftclient20:59
*** mhenkel has quit IRC21:00
slaglejpeeler: colonwq : should be fixed now ^^^21:01
*** lblanchard has quit IRC21:02
colonwqslagle, My liver will ask for mercy if too many of my patches start passing.21:02
colonwqThanks21:02
*** julim has quit IRC21:05
openstackgerritgreghaynes proposed openstack/diskimage-builder: Allow disabling manifests  https://review.openstack.org/35886521:20
*** ebarrera has quit IRC21:24
*** asbishop has quit IRC21:25
*** yamahata has quit IRC21:40
*** mburned_out is now known as mburned21:41
*** rhallisey has quit IRC21:44
openstackgerritMerged openstack/tripleo-common: Change input for UpdateCapabilitiesAction  https://review.openstack.org/34973221:44
sshnaidmslagle, can we please merge those: https://review.openstack.org/#/c/357532/  https://review.openstack.org/#/c/357217/21:44
*** yamahata has joined #tripleo21:46
*** egafford has quit IRC21:48
*** mgarciam has joined #tripleo21:50
openstackgerritRobin Cernin proposed openstack/tripleo-validations: Validate rabbitmq policies  https://review.openstack.org/35843021:54
*** morazi has quit IRC22:09
*** mburned is now known as mburned_out22:09
*** cdearborn has quit IRC22:12
*** jobewan has joined #tripleo22:15
*** rajinir has quit IRC22:25
*** sshnaidm is now known as sshnaidm|afk22:28
*** bnemec has joined #tripleo22:37
*** mpsairam has joined #tripleo22:37
*** beagles_biab is now known as beagles22:45
*** jcoufal_ has quit IRC22:51
*** rlandy has quit IRC23:02
*** tbarron is now known as tbarron|gone23:02
*** jobewan has quit IRC23:08
*** cmyster has quit IRC23:17
openstackgerritGraeme Gillies proposed openstack/tripleo-docs: Updated basic installation page to include links to prebuilt images  https://review.openstack.org/35682323:19
*** cmyster has joined #tripleo23:25
*** rwsu has joined #tripleo23:28
*** rwsu has quit IRC23:32
*** rwsu has joined #tripleo23:39
*** saneax-_-|AFK is now known as saneax23:44

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!