| *** limao has joined #tripleo | 00:28 | |
| *** yamahata has quit IRC | 00:53 | |
| *** hanchao has joined #tripleo | 00:55 | |
| *** apetrich has quit IRC | 00:58 | |
| *** apetrich has joined #tripleo | 00:58 | |
| *** yamahata has joined #tripleo | 01:07 | |
| *** saneax is now known as saneax_AFK | 01:12 | |
| *** stendulker has joined #tripleo | 01:15 | |
| *** beagles_biab is now known as beagles | 01:19 | |
| *** [1]cdearborn has joined #tripleo | 01:29 | |
| *** dmacpher has joined #tripleo | 01:35 | |
| *** limao has quit IRC | 01:41 | |
| *** limao_ has joined #tripleo | 01:41 | |
| *** cdearborn has quit IRC | 01:44 | |
| *** rajinir has quit IRC | 01:45 | |
| openstackgerrit | Jon Schlueter proposed openstack/python-tripleoclient: Use osc-lib instead of openstackclient https://review.openstack.org/343883 | 01:45 |
|---|---|---|
| *** shivrao has quit IRC | 01:52 | |
| openstackgerrit | Merged openstack/instack-undercloud: Modify LVM filter fix to blacklist iSCSI https://review.openstack.org/343100 | 01:56 |
| *** yamahata has quit IRC | 02:09 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add MysqlNoBracketsInternal to enable-tls https://review.openstack.org/343611 | 02:19 |
| *** yamahata has joined #tripleo | 02:21 | |
| *** stendulker has quit IRC | 02:25 | |
| *** yamahata has quit IRC | 02:34 | |
| *** ayoung has quit IRC | 03:01 | |
| *** limao_ has quit IRC | 03:04 | |
| *** limao has joined #tripleo | 03:09 | |
| *** akshai has joined #tripleo | 03:10 | |
| *** limao has quit IRC | 03:12 | |
| *** [1]cdearborn has quit IRC | 03:14 | |
| *** akshai_ has joined #tripleo | 03:26 | |
| *** akshai has quit IRC | 03:29 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/diskimage-builder: Remove the escape in the centos7 README file https://review.openstack.org/343997 | 03:32 |
| *** saneax_AFK is now known as saneax | 03:33 | |
| *** yamahata has joined #tripleo | 03:35 | |
| *** saneax is now known as saneax_AFK | 03:45 | |
| *** akshai_ has quit IRC | 03:50 | |
| *** shivrao has joined #tripleo | 04:03 | |
| *** limao has joined #tripleo | 04:13 | |
| *** limao has quit IRC | 04:18 | |
| *** jerrygb has joined #tripleo | 04:22 | |
| *** saneax_AFK is now known as saneax | 04:29 | |
| *** jrist has quit IRC | 04:38 | |
| *** shardy has joined #tripleo | 04:39 | |
| saneax | GM folks | 04:50 |
| *** limao has joined #tripleo | 04:50 | |
| *** jrist has joined #tripleo | 04:51 | |
| shardy | morning all | 04:52 |
| *** radek2 has joined #tripleo | 04:56 | |
| *** links has joined #tripleo | 05:01 | |
| *** jaosorior has joined #tripleo | 05:04 | |
| *** masco has joined #tripleo | 05:07 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: Fix packaging problems for Debian https://review.openstack.org/343342 | 05:12 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: [WIP] Move elements into dib package https://review.openstack.org/344017 | 05:13 |
| *** numans has joined #tripleo | 05:22 | |
| *** jerrygb has quit IRC | 05:30 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: [WIP] Move elements into dib package https://review.openstack.org/344017 | 05:31 |
| *** links has quit IRC | 05:36 | |
| *** apetrich has quit IRC | 05:50 | |
| *** skramaja has joined #tripleo | 05:54 | |
| *** apetrich has joined #tripleo | 05:54 | |
| *** milan has quit IRC | 05:59 | |
| *** milan has joined #tripleo | 05:59 | |
| *** abehl has joined #tripleo | 06:01 | |
| *** pkovar has joined #tripleo | 06:03 | |
| *** _milan_ has joined #tripleo | 06:04 | |
| *** jprovazn has joined #tripleo | 06:04 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: [WIP] Move elements into dib package https://review.openstack.org/344017 | 06:04 |
| *** milan has quit IRC | 06:05 | |
| *** anshul_ has joined #tripleo | 06:09 | |
| *** rcernin has joined #tripleo | 06:10 | |
| *** pgadiya has joined #tripleo | 06:10 | |
| *** abehl has quit IRC | 06:10 | |
| *** leanderthal|afk is now known as leanderthal | 06:11 | |
| *** oshvartz has joined #tripleo | 06:15 | |
| *** _milan_ has quit IRC | 06:16 | |
| *** milan has joined #tripleo | 06:16 | |
| *** derekjhyang has joined #tripleo | 06:17 | |
| *** florianf has joined #tripleo | 06:17 | |
| *** radek2 has quit IRC | 06:22 | |
| *** rasca_ has joined #tripleo | 06:26 | |
| *** ccamacho has joined #tripleo | 06:27 | |
| ccamacho | Good morning guys! | 06:31 |
| jaosorior | ccamacho hey dude | 06:32 |
| jaosorior | dtantsur: Are you around? | 06:32 |
| ccamacho | jaosorior, still fighting with the mistral stuff, kinda painful jajaaj... | 06:32 |
| *** shardy has quit IRC | 06:33 | |
| *** pcaruana has joined #tripleo | 06:36 | |
| openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Add pacemaker mistral profile https://review.openstack.org/343346 | 06:41 |
| *** tremble has joined #tripleo | 06:44 | |
| *** ramishra has quit IRC | 06:45 | |
| *** ramishra has joined #tripleo | 06:47 | |
| jaosorior | ccamacho how well can you debug mistral? | 06:49 |
| *** anshul_ has quit IRC | 06:49 | |
| *** cwolferh has quit IRC | 06:51 | |
| *** hparekh has quit IRC | 06:54 | |
| *** shardy has joined #tripleo | 06:56 | |
| *** tesseract- has joined #tripleo | 06:58 | |
| *** cwolferh has joined #tripleo | 06:59 | |
| *** devvesa has joined #tripleo | 07:00 | |
| *** ohamada has joined #tripleo | 07:02 | |
| jaosorior | Hey guys, can we merge this? https://review.openstack.org/#/c/343607/ | 07:02 |
| *** anshul_ has joined #tripleo | 07:02 | |
| *** liverpooler has joined #tripleo | 07:03 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: openstack-core resource does not have interleave=true https://review.openstack.org/342725 | 07:04 |
| jaosorior | shardy: how acquainted are you with mistral? | 07:05 |
| shardy | jaosorior: somewhat, I've been testing the new mistral actions/workflows and written some local workflows attempting to deploy direct to ironic | 07:05 |
| bandini | I see folks are putting "recheck dlrn fail" or comments like that when rechecking. Are there any guidelines about this? | 07:06 |
| jaosorior | shardy: For some reason it seems that when we re-enabled introspection... it's now broken | 07:06 |
| jaosorior | and it's weird | 07:06 |
| shardy | bandini: Yes, if possible it's best to do recheck bug #### | 07:06 |
| shardy | where the bug is an existing known issue | 07:06 |
| jaosorior | cause it's breaking in an action that comes with mistral | 07:06 |
| bandini | shardy: ack, got it | 07:06 |
| ccamacho | jaosorior, the issue is with the deployment, I believe is almost done, but after is installed and working I need to run a set of local tests to check that is actually working properly | 07:06 |
| shardy | bandini: for some things like spurious failiures e.g of delorean mirrors or something, we might do just a description | 07:06 |
| bandini | shardy: ok, I see. I thought there was like a common set of keywords that would allow us to gather some numbers about failures | 07:07 |
| jaosorior | shardy, ccamacho http://logs.openstack.org/69/342669/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/78969cb/console.html | 07:07 |
| bandini | shardy: thanks :) | 07:07 |
| shardy | bandini: the main thing to avoid doing repeatedly is "blind rechecks", as sometimes it's pretty clear folks don't even look at the failure reason and just do recheck recheck recheck | 07:07 |
| *** apetrich has quit IRC | 07:07 | |
| *** hparekh has joined #tripleo | 07:08 | |
| *** bootsha has joined #tripleo | 07:08 | |
| shardy | bandini: there is a tool called elastic recheck, http://status.openstack.org/elastic-recheck/ | 07:08 |
| bandini | shardy: yeah that makes sense, I know I have not been completely innocent on that front ;) | 07:08 |
| shardy | bandini: the idea of tagging a bug reference is you can see from those graphs what bugs are causing most failures | 07:08 |
| bandini | ah, this is nice | 07:09 |
| *** apetrich has joined #tripleo | 07:09 | |
| shardy | jaosorior: introspection is now driven via mistral action/workflow defined in tripleo-common | 07:10 |
| shardy | jaosorior: did we land anything related to those while the CI was disabled for introspection? | 07:10 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add MysqlNoBracketsInternal to enable-tls.yaml https://review.openstack.org/343607 | 07:12 |
| jaosorior | shardy: seems that's the case | 07:16 |
| jaosorior | so now either we disable introspection again or figure out how to fix it | 07:16 |
| jaosorior | shardy: Yep, here it is https://review.openstack.org/#/c/327780/ | 07:17 |
| jaosorior | ah wait | 07:17 |
| shardy | jaosorior: I tested that one locally, I suspect it's a later commit that broke things | 07:18 |
| jaosorior | no, actually I don't see anything that would immediately break those | 07:18 |
| *** ifarkas_afk is now known as ifarkas | 07:18 | |
| *** zoli_gone-proxy is now known as zoliXXL | 07:19 | |
| bandini | so I have two changes A) puppet-tripleo and B) tht that only make sense together. For now on change B) I put a depends-on A), but A) will never pass CI because it needs B) as well. Can I create a circular depends-on loop? | 07:20 |
| shardy | bandini: I'd put the Depends-On in the tht patch, then we'll merge them together | 07:20 |
| shardy | if it passes CI we can derive the puppet-tripleo patch is OK | 07:20 |
| bandini | shardy: ack, got it. thanks | 07:20 |
| shardy | I'm not sure if a circular dependency will work (suspect probably not) | 07:21 |
| bandini | yeah that is what I assumed | 07:21 |
| shardy | it'll definitely make them both impossible to merge tho :) | 07:21 |
| bandini | eheh right | 07:21 |
| *** fzdarsky has joined #tripleo | 07:23 | |
| *** jubapa has joined #tripleo | 07:23 | |
| *** fzdarsky has quit IRC | 07:24 | |
| *** paramite has joined #tripleo | 07:24 | |
| hewbrocca | RDO promotion is still 12 days behind | 07:25 |
| hewbrocca | If anyone here can fix any of the issues at the bottom of this etherpad: | 07:25 |
| hewbrocca | https://etherpad.openstack.org/p/delorean_master_current_issues | 07:25 |
| hewbrocca | Please do so ASAP | 07:25 |
| hewbrocca | https://dashboards.rdoproject.org/rdo-dev is useful for monitoring this | 07:26 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: [WIP] Move elements into dib package https://review.openstack.org/344017 | 07:27 |
| *** dsariel has joined #tripleo | 07:29 | |
| *** anshul_ has quit IRC | 07:29 | |
| *** ebarrera has joined #tripleo | 07:30 | |
| *** akuznetsov has joined #tripleo | 07:30 | |
| *** jerrygb has joined #tripleo | 07:32 | |
| *** paramite has quit IRC | 07:36 | |
| *** paramite has joined #tripleo | 07:37 | |
| *** jerrygb has quit IRC | 07:37 | |
| *** jpich has joined #tripleo | 07:41 | |
| *** abehl has joined #tripleo | 07:43 | |
| *** bootsha has quit IRC | 07:48 | |
| ccamacho | jmiu when available can you check this in your environment? https://review.openstack.org/#/c/343902 | 07:52 |
| *** bootsha has joined #tripleo | 07:53 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Remove two races during the L->M Keystone under httpd migration https://review.openstack.org/338879 | 07:53 |
| bandini | matbu: can you throw one of your upgrade jobs at the review above ^? It should fix the problem marios saw as well | 07:54 |
| marios | bandini: o/ i c there is an update with the "no outstanding transactions" bit | 07:56 |
| *** akuznetsov has quit IRC | 07:56 | |
| bandini | marios: there is more than that. I implemented andrew's suggestion from the RH BZ | 07:56 |
| bandini | I create openstack-core and wait for it to be started. that way we avoid a bunch of spurious restarts when we change the constraints | 07:57 |
| matbu | bandini: yes the ci jobs used this review | 07:58 |
| marios | bandini: ack thanks I will include it on next pass | 07:58 |
| bandini | matbu: can you run another one with today's change? | 07:58 |
| matbu | bandini: yes sure | 07:58 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration - as a composable controller service https://review.openstack.org/188137 | 07:59 |
| bandini | matbu: thank you sir! ;) | 07:59 |
| marios | jaosorior: merge conflict but it wiped your +2 appreciate a look when you have a chance ^^ | 08:00 |
| matbu | bandini: de nada :) , the keystone migration needs another fix ? | 08:00 |
| openstackgerrit | Merged openstack/tripleo-ui: Root components reorganization https://review.openstack.org/333869 | 08:01 |
| bandini | matbu: potentially yes, marios hit some snags during that upgrade steps | 08:01 |
| *** athomas has joined #tripleo | 08:03 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates: Added a new composable service for enabling DPDK on compute nodes. https://review.openstack.org/344084 | 08:03 |
| matbu | bandini: k, i hit weird issue in the controller upgrade step2 | 08:05 |
| matbu | marios: ^ | 08:05 |
| marios | o/ morning shardy.the manila review had a green run of ci at https://review.openstack.org/#/c/342961/ (see jenkins set 1) - i think we should consider landing https://review.openstack.org/188137 (manila) and https://review.openstack.org/#/c/339437/ (manila password tripleoclient) | 08:05 |
| matbu | only on a specific hardware | 08:05 |
| marios | matbu: which step do you mean controller upgrade? | 08:05 |
| marios | oh sry, i read that too fast | 08:05 |
| matbu | marios: major controller upgrade step 2 | 08:05 |
| marios | you mean controller_pacemaker_2 | 08:05 |
| marios | yah | 08:05 |
| matbu | marios: no pb, all the resource are unmanage | 08:05 |
| *** tremble has quit IRC | 08:08 | |
| marios | matbu: hmm it is strange they should be unmanaged... i mean we don't do that for pacemaker1/2 i mean it happens on converge when we do puppet/post-puppet | 08:09 |
| marios | matbu: so it would be from error/timeout i mean not expected at this step | 08:09 |
| marios | matbu: do you have logs at this point.would be good to capture info into a bug, i assume 8..9 related and specific bm setup? so probably bz at this point? | 08:10 |
| matbu | marios: nop i just have an env where i reproduce it | 08:11 |
| *** akrivoka has joined #tripleo | 08:11 | |
| matbu | marios: i'm looking at the issue for the moment | 08:11 |
| matbu | marios: i'm wondering if it not due to the size of the hardware | 08:12 |
| marios | matbu: yeah ok man, i mean capture whatever you can find but you're doing that | 08:12 |
| matbu | marios: (it's a virt env btw) | 08:12 |
| *** tremble has joined #tripleo | 08:12 | |
| marios | matbu: so are there very many cores/processors? or you mean not enough ram? | 08:12 |
| matbu | marios: yeah, not enough ram or core | 08:12 |
| matbu | marios: cause it looks like the cluster take too long time to bring resources up | 08:13 |
| matbu | and manage | 08:13 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui: Remove app.conf to use a JS-based config file https://review.openstack.org/343658 | 08:13 |
| marios | matbu: you should see some kind of error like "ERROR: cluster remained unstable for more than $timeout seconds, exiting." that we have in the /pacemaker_common_functions.sh or ./major_upgrade_controller_pacemaker_2.sh etc | 08:14 |
| marios | matbu: might be worth grepping on some of those strings to see in var/log/messages | 08:15 |
| *** milan has quit IRC | 08:16 | |
| marios | matbu: you can also try to manually increase those timeouts ./yum_update.sh:27:cluster_settle_timeout=1800 to test the theory | 08:16 |
| *** derekh has joined #tripleo | 08:17 | |
| matbu | marios: yes this is the error shown by heat | 08:18 |
| *** Guest53586 has quit IRC | 08:22 | |
| *** flaper87 has joined #tripleo | 08:22 | |
| *** flaper87 has joined #tripleo | 08:22 | |
| marios | matbu: ok well that is a good thing for the most part i think... i mean it sounds like h/w not some issue in the upgrade, though still might be... | 08:23 |
| marios | matbu: how much ram are we talking? | 08:23 |
| marios | matbu: and is this 3/1 net-iso or bigger? | 08:23 |
| matbu | marios: yep this is a 3 controller 1 compute, 1 UC and net iso | 08:24 |
| *** lucas-afk is now known as lucasagomes | 08:25 | |
| gchamoul | marios: hi, thx for your patch https://review.openstack.org/#/c/342032/1 but could you please revive it? | 08:25 |
| matbu | bandini: do you want me to re-kick the L to M job ? | 08:27 |
| bandini | matbu: yes please it should fetch the very latest version of the https://review.openstack.org/#/c/338879/ review | 08:28 |
| marios | gchamoul: done thanks :) didn't want to be 'stealing peoples reviews' so I thought you may have wanted to do it yrself | 08:30 |
| marios | gchamoul: but because of the 9-related bugzillas i filed as you were away end of the week. | 08:31 |
| gchamoul | marios: I was away! and you were right to do it! no worries ;-D | 08:32 |
| matbu | bandini: k | 08:32 |
| gchamoul | marios: thx! | 08:32 |
| *** radek2 has joined #tripleo | 08:32 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/puppet-tripleo: Ensure the Heat Domain parameters before starting heat-engine pcmk service https://review.openstack.org/342032 | 08:33 |
| marios | gchamoul: np & ty | 08:33 |
| matbu | bandini: done here: https://ci.centos.org/view/rdo/view/tripleo-periodic/job/tripleo-quickstart-upgrade-major-liberty-to-mitaka/34/console | 08:36 |
| bandini | matbu: perfect, thanks. will keep an eye on it | 08:38 |
| bandini | so all my experimental jobs seem to fail with http://logs.openstack.org/87/338387/13/experimental/gate-tripleo-ci-centos-7-nonha-multinode/0386df0/console.html and all my normal jobs seem to fail due to delorean issues http://logs.openstack.org/04/337304/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/c42139c/console.html | 08:41 |
| bandini | erm, how can I help to fix this? Start with a bug or ping delorean folks? | 08:41 |
| *** aufi has joined #tripleo | 08:41 | |
| jaosorior | marios: One small detail about that submissino | 08:45 |
| *** shivrao has quit IRC | 08:46 | |
| *** saneax is now known as saneax_AFK | 08:46 | |
| marios | jaosorior: ok thanks for that... gfidente|afk can you confirm later https://review.openstack.org/#/c/188137/83/puppet/services/manila-base.yaml - though jaosorior as noted see the review that adds this in ci (and you can see manila logs, afaics there aren't errors there) | 08:48 |
| dtantsur | morning folks, an easy patch to review please (1x +2, gate passed long ago): https://review.openstack.org/#/c/331584/ | 08:50 |
| openstackgerrit | Martin Mágr proposed openstack-infra/tripleo-ci: Add opstools candidate repo https://review.openstack.org/325095 | 08:51 |
| *** milan has joined #tripleo | 08:53 | |
| jaosorior | marios; YEah,w e had a lot of those issues in another submission. So thankfully gfidente reviewed it and pointed out that if we don't use the NoBrackets version of that endpoint it will break in ipv6 | 08:55 |
| *** gfidente|afk is now known as gfidente | 08:55 | |
| jaosorior | dtantsur: Hey dude, have you been testing introspection in master? | 08:55 |
| jaosorior | yesterday we re-enabled it and it seems to be broken | 08:55 |
| dtantsur | jaosorior, that's why you don't disable things in gate :/ link to the error handy? | 08:56 |
| dtantsur | I surely remember it worked at some moment yesterday | 08:56 |
| jaosorior | dtantsur: http://logs.openstack.org/69/342669/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/78969cb/console.html#_2016-07-19_06_15_59_176102 | 08:56 |
| *** jaosorior has quit IRC | 08:57 | |
| *** jaosorior has joined #tripleo | 08:58 | |
| *** tremble has quit IRC | 08:58 | |
| dtantsur | wow, that's amazingly non-readable | 08:58 |
| dtantsur | however, it seems like the workflow itself is broken, which again brings us to "please don't disable things in gate" point :/ | 08:58 |
| jaosorior | dtantsur: It's not like I went ahead and disabled it. | 08:59 |
| shardy | To be fair, the nonha job had been broken for over two days when we finally gave in and disabled it IIRC | 08:59 |
| shardy | I agree it should be a last resort | 08:59 |
| shardy | but some coverage is better than everyone ignoring the job because it's red for days on end | 09:00 |
| dtantsur | and this last resort can probably only happen on a release day | 09:00 |
| dtantsur | or not even then | 09:00 |
| dtantsur | I mean, if neutron was broken, would you remove neutron from tripleo "temporary" or try to fix it? | 09:00 |
| shardy | a failing job carries the same risk, because changes to all the puppet modules were getting landed with failing CI | 09:00 |
| shardy | because the jobs stopped at introspection | 09:00 |
| dtantsur | well, here we are: something breaks introspection and we have no idea what | 09:01 |
| *** tremble has joined #tripleo | 09:01 | |
| jaosorior | dtantsur, shardy: FWIW it seems to me that the mistral action for introspection is broken for some reason | 09:01 |
| dtantsur | InputException: Invalid input [name=baremetal_introspection.wait_for_finish, class=NoneType, unexpected=[u\'uuids\']] | 09:01 |
| dtantsur | exactly | 09:01 |
| jaosorior | that's as far as I could debug with my minimal knowledge of mistral | 09:01 |
| jaosorior | which makes no sense cause the wait_for_finish function in python-ironic-inspector-client does take a list of uuids | 09:02 |
| * shardy looks around for d0ugal | 09:02 | |
| jpich | I think he's at EuroPython all week | 09:03 |
| jaosorior | dtantsur: Anyway, I poked you cause you might have an idea of what's up. But it seems that you're pretty much in the same spot as me | 09:03 |
| dtantsur | also, can we please kindly move nonha from experimental gate? | 09:03 |
| dtantsur | everything in the experimental pipeline is broken by definition | 09:03 |
| shardy | dtantsur: It runs on all puppet module changes | 09:04 |
| shardy | we don't have the capacity to run it on all tripleo repos until rh1 comes back online | 09:04 |
| *** tzumainn has joined #tripleo | 09:04 | |
| *** electrofelix has joined #tripleo | 09:04 | |
| jaosorior | shardy: What about adding it to the puppet-tripleo repo? | 09:04 |
| dtantsur | shardy, well, then lets fix introspection after it's in the check pipeline | 09:05 |
| dtantsur | I don't see a reason bothering, if it can get broken the next patch | 09:05 |
| dtantsur | I know I'm a bit harsh, but we've learned all this the hard way in Ironic :) | 09:05 |
| shardy | dtantsur: We've got a temporary lack of capacity, is it really too much to ask for folks to test things locally once per day until we get that capacity back? | 09:06 |
| shardy | We found out when it broke because it is in the check pipeline for every single puppet module anyway | 09:06 |
| dtantsur | yes, it is too much. people don't do that, trust me | 09:07 |
| shardy | awesome attitude :\ | 09:07 |
| dtantsur | I've seen a lot of patches that are apparently and terribly broken | 09:07 |
| * shardy sighs, makes a coffee and pulls all-the-things so he can fix this himself | 09:07 | |
| dtantsur | it was completely apparent that people never dared doing even simplest checks | 09:07 |
| dtantsur | also the split between puppet-tripleo and THT makes testing a lot of things a black magic | 09:08 |
| dtantsur | I gave up on trying to test locally any changes that are split between these 2 repos | 09:08 |
| dtantsur | (it's not related to introspection, just a side note) | 09:08 |
| *** tobbe_fiberdata has joined #tripleo | 09:09 | |
| jaosorior | shardy: the undercloud is running mistral by default now, right? | 09:11 |
| jpich | shardy: Maybe I can try to help? I have a tiny teeny bit of experience poking at Mistral now. Are there instructions somewhere on how to reproduce that kind of CI run? | 09:11 |
| dtantsur | jpich, do you have a link where mistral actions are defined? | 09:11 |
| dtantsur | I can't find the baremetal introspection wait_for_finish action | 09:12 |
| shardy | dtantsur: it really isn't that hard, I'll write a blog post later and/or push a docs patch | 09:12 |
| shardy | tl;dr upload-puppet-modules -d puppet-modules | 09:12 |
| shardy | it was discussed on last week's deep-dive call | 09:12 |
| dtantsur | wow, I've never heard about upload-puppet-modules thing. a blog post would be awesome | 09:12 |
| * dtantsur has too many meetings already :( | 09:12 | |
| jaosorior | dtantsur: This is the closest thing I've found to a definition in the mistral code-base https://github.com/openstack/mistral/blob/master/mistral/actions/openstack/mapping.json#L1034 | 09:13 |
| jpich | dtantsur: I think that'd be there for the general workflow -> https://github.com/openstack/tripleo-common/blob/master/workbooks/baremetal.yaml#L173 and around here https://github.com/openstack/tripleo-common/tree/master/tripleo_common/actions for actions | 09:13 |
| jaosorior | dprince is more acquainted with that part, but he's some hours from waking up still | 09:13 |
| * jpich looks around too | 09:13 | |
| shardy | dtantsur: https://etherpad.openstack.org/p/tripleo-deep-dive-topics lines 23-31 is what I do | 09:13 |
| jaosorior | jpich: well, the action that seems to be failing is in the mistral stock actions | 09:13 |
| shardy | hack on puppet-modules/tripleo and just re-run the upload script after making any changes | 09:14 |
| dtantsur | shardy, cool. still my biggest problem was ensuring that my local THT checkout is in sync with whatever ends up being on the image | 09:14 |
| shardy | dtantsur: Yeah, this largely solves that | 09:14 |
| dtantsur | interesting, I'll try the next time I debug my ironic thingy | 09:14 |
| shardy | the same method can be used to update individual RPMs on the node, but I'm not sure you can update both puppet modules *and* RPMs, that may be an RFE | 09:14 |
| *** jaosorior is now known as jaosorior_lunch | 09:15 | |
| shardy | jpich: Sure, if you can help reproduce that would be great | 09:16 |
| shardy | jpich: All I'm about to do is yum -y update my undercloud, then re-run openstack undercloud install | 09:16 |
| dtantsur | I really wonder why class is None in this case | 09:16 |
| shardy | (actually I think just running undercloud install is enough, but I like seeing what has changed) | 09:16 |
| gfidente | marios, morning | 09:17 |
| gfidente | would be nice to restore the ipv6 job indeed | 09:17 |
| tobbe_fiberdata | does anyone have a good tip of font to use? | 09:18 |
| tobbe_fiberdata | best offtopic ever. | 09:18 |
| *** akuznetsov has joined #tripleo | 09:19 | |
| shardy | wingdings ;) | 09:19 |
| tobbe_fiberdata | shardy, hahaha, funny man :p | 09:19 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration - as a composable controller service https://review.openstack.org/188137 | 09:20 |
| marios | gfidente: jaosorior_lunch thanks and done ^^^ | 09:20 |
| dtantsur | a side note: I really hope we're not planning to show things like http://logs.openstack.org/69/342669/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/78969cb/console.html#_2016-07-19_06_15_59_176102 to users | 09:21 |
| dtantsur | it will scare the hell out of them :) | 09:21 |
| tobbe_fiberdata | I'm gonna have fun setting up a undercloud and overcloud for production. we'll see how that goes. putting up a lab on my local computer first lol. Wish me luck! | 09:21 |
| jpich | shardy: Ok, thank you, I'll go and do that too - don't let me slow you down if that wasn't actually all that time-consuming though! | 09:21 |
| marios | gfidente: but gfidente grep tells me we have like ./puppet/services/neutron-server.yaml:69: neutron::db::mysql::host: {get_param: [EndpointMap, MysqlInternal, host]} | 09:22 |
| marios | gfidente: on master | 09:22 |
| shardy | good luck tobbe_fiberdata | 09:22 |
| tobbe_fiberdata | shardy, thanks | 09:22 |
| gfidente | marios, yeah that's wrong | 09:22 |
| shardy | jpich: ack, thanks, I'll pulling the updates now, will see if I can reproduce | 09:22 |
| jpich | dtantsur: ...From what I've seen there's some space for improvement in terms of error reporting when using the workflows from the client | 09:23 |
| *** bootsha has quit IRC | 09:23 | |
| dtantsur | yeah | 09:23 |
| shardy | Yeah I hit some non-obvious error paths when testing too, we'll have to improve them for sure | 09:25 |
| *** mgould|afk is now known as mgould | 09:26 | |
| *** fzdarsky has joined #tripleo | 09:27 | |
| gfidente | marios, I have seen a submission fixing it though | 09:31 |
| *** radek2 has quit IRC | 09:31 | |
| marios | gfidente: ok thx just checking | 09:31 |
| *** saneax_AFK is now known as saneax | 09:35 | |
| ccamacho | derekh marios gfidente morning! Quick question, I'm hit now by timeout_mins = 80 when checking a submission, is it possible to increase this time in a "test" submission and link the THT one with a depends-on? If so, can you point me where is defined this timeout? | 09:35 |
| *** abehl has quit IRC | 09:36 | |
| shardy | ccamacho: it's calculated and based on a hard-limit set by openstack infra https://github.com/openstack-infra/tripleo-ci/blob/master/toci_gate_test.sh#L73 | 09:37 |
| shardy | ccamacho: you may find that increasing it simply means we hit the infra timeout and the job gets killed | 09:38 |
| shardy | so it'd be better to figure out where the time is going and optimize the walltime really | 09:38 |
| shardy | maybe that 90 can be reduced though, for jobs where we use cached images | 09:39 |
| ccamacho | shardy thanks, Im getting this executing a clustercheck, im my local env also takes time, Ill try to see then why is taking that long.. Thanks! | 09:40 |
| *** sshnaidm|afk has quit IRC | 09:46 | |
| *** abehl has joined #tripleo | 09:47 | |
| *** yamahata has quit IRC | 09:48 | |
| *** akuznetsov has quit IRC | 09:48 | |
| openstackgerrit | Martin André proposed openstack/instack-undercloud: Introduce 'enable_ui' option https://review.openstack.org/344140 | 09:49 |
| *** mcornea has joined #tripleo | 09:50 | |
| *** dsariel has quit IRC | 09:52 | |
| *** stendulker has joined #tripleo | 09:53 | |
| tobbe_fiberdata | how is the upgrade-path look like from mitaka to newton when there's a bit of differences between the versions? | 09:54 |
| tobbe_fiberdata | what i mean is for example the templates for network and such things? | 09:54 |
| tobbe_fiberdata | or perhaps it's to soon to say? | 09:54 |
| shardy | tobbe_fiberdata: it's a bit too soon to say, but our aim is to allow existing templates for network configs etc to continue working | 09:55 |
| tobbe_fiberdata | ah okey | 09:55 |
| shardy | that said, in https://review.openstack.org/#/c/271450/ I'm aiming to enable an alternative new format for os-net-config data | 09:56 |
| shardy | it's not landed yet tho, and the old method should continue working even if we deprecate it | 09:56 |
| *** bootsha has joined #tripleo | 09:57 | |
| tobbe_fiberdata | sounds great shardy! | 09:59 |
| tobbe_fiberdata | that has always been the biggest problem from what i see. you deploy an environment for production then you never touch it as it works, when the time comes to move up a version or two it's hard to do it just like that. requires alot of planning | 10:00 |
| tobbe_fiberdata | i'm just talking for myself atleast :P | 10:00 |
| tobbe_fiberdata | and our environment running RHEL-OSP7. we are though not planning on running RHEL OSP in the future though. i hope that will not bring me problem. | 10:02 |
| tobbe_fiberdata | but i bet it will | 10:02 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Stop Validation https://review.openstack.org/336561 | 10:05 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Triggering validations and messaging https://review.openstack.org/336560 | 10:05 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: List Validations and Validation results https://review.openstack.org/334770 | 10:05 |
| *** sshnaidm has joined #tripleo | 10:09 | |
| *** shardy has quit IRC | 10:16 | |
| *** tosky has joined #tripleo | 10:19 | |
| *** shardy has joined #tripleo | 10:20 | |
| *** limao has quit IRC | 10:24 | |
| *** abehl has quit IRC | 10:25 | |
| *** dsariel has joined #tripleo | 10:26 | |
| *** abehl has joined #tripleo | 10:26 | |
| akrivoka | is there any drawback to changing the parameter type of ControllerCount from number to string? then we could use the allowed_pattern constraint to limit it to odd numbers (allowed_pattern only works with strings and not with numbers). | 10:29 |
| akrivoka | https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.yaml#L150 | 10:29 |
| shardy | akrivoka: It'll probably work if you ensure valid input via a contraint, but personally I'd rather add a number compatible constraint to heat | 10:31 |
| *** devvesa has quit IRC | 10:32 | |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-quickstart: WIP gate upgrade https://review.openstack.org/342161 | 10:32 |
| akrivoka | shardy: I tried adding a custom constrain, but heat people don't seem keen https://review.openstack.org/#/c/342265/ | 10:32 |
| *** pkovar has quit IRC | 10:34 | |
| shardy | akrivoka: ack, thanks - I disagree and have +2'd the patch | 10:35 |
| shardy | akrivoka: I'll see if I can solicit some other opinions | 10:35 |
| akrivoka | heh, thanks shardy ! | 10:35 |
| shardy | in the meantime the string method is probably OK if you're blocked on this | 10:35 |
| tobbe_fiberdata | i'm trying to get this virtual lab up but it says that there's no network devices and there's no bootable device. it does PXEboot. | 10:35 |
| tobbe_fiberdata | when i run introspection | 10:36 |
| *** zoliXXL is now known as zoli|lunch | 10:40 | |
| *** jubapa has quit IRC | 10:43 | |
| *** rodrigods has quit IRC | 10:44 | |
| *** rhallisey has joined #tripleo | 10:44 | |
| *** rodrigods has joined #tripleo | 10:44 | |
| *** Goneri has joined #tripleo | 10:44 | |
| openstackgerrit | Giulio Fidente proposed openstack/instack-undercloud: Allow customization of undercloud disk size https://review.openstack.org/344161 | 10:45 |
| dtantsur | marios, gate passed on https://review.openstack.org/#/c/331584/ again, mind +A please? | 10:45 |
| *** devvesa has joined #tripleo | 10:46 | |
| *** jubapa has joined #tripleo | 10:46 | |
| *** jerrygb has joined #tripleo | 10:51 | |
| *** ccamacho is now known as ccamacho|lunch | 10:52 | |
| *** jerrygb has quit IRC | 10:56 | |
| *** osp has quit IRC | 11:00 | |
| *** jaosorior_lunch is now known as jaosorior | 11:03 | |
| hewbrocca | Ack early, Ack often! | 11:04 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-puppet-elements: Install osops-tools-monitoring-oschecks package https://review.openstack.org/324075 | 11:06 |
| matbu | bandini: it passed CI | 11:07 |
| bandini | matbu: w00t thanks! marios ^ | 11:08 |
| *** pkovar has joined #tripleo | 11:08 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: gnocchi: configure db_sync command in the profile https://review.openstack.org/343902 | 11:10 |
| *** stendulker has quit IRC | 11:12 | |
| openstackgerrit | Martin Mágr proposed openstack/puppet-tripleo: New composable service: Sensu https://review.openstack.org/323303 | 11:13 |
| openstackgerrit | Martin Mágr proposed openstack/puppet-tripleo: New composable service: Sensu https://review.openstack.org/323303 | 11:17 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable keystone to use the SSL middleware https://review.openstack.org/341354 | 11:18 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable glance to use the SSL middleware https://review.openstack.org/341367 | 11:18 |
| *** bootsha has quit IRC | 11:20 | |
| *** devvesa has quit IRC | 11:21 | |
| *** athomas has quit IRC | 11:23 | |
| *** weshay has joined #tripleo | 11:25 | |
| *** athomas has joined #tripleo | 11:28 | |
| *** radek2 has joined #tripleo | 11:32 | |
| *** bootsha has joined #tripleo | 11:35 | |
| marios | bandini: matbu: ack thanks | 11:39 |
| marios | dtantsur: looking | 11:39 |
| marios | dtantsur: done | 11:40 |
| dtantsur | thnx! | 11:40 |
| *** devvesa has joined #tripleo | 11:40 | |
| *** devvesa has quit IRC | 11:40 | |
| *** openstackgerrit has quit IRC | 11:41 | |
| *** openstackgerrit has joined #tripleo | 11:41 | |
| *** ccamacho|lunch has quit IRC | 11:44 | |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Install Ceph Jewel from Ceph repos https://review.openstack.org/340504 | 11:44 |
| openstackgerrit | Merged openstack/tripleo-common: Allow setting capabilities as a dict in addition to a string https://review.openstack.org/331584 | 11:46 |
| *** lucasagomes is now known as lucas-hungry | 11:49 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Handle empty list of introspected nodes https://review.openstack.org/344188 | 11:55 |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-quickstart: WIP gate upgrade https://review.openstack.org/342161 | 11:56 |
| *** jerrygb has joined #tripleo | 11:57 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Use certmonger for automatic cert generation https://review.openstack.org/332774 | 11:57 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Use certmonger for automatic cert generation https://review.openstack.org/332774 | 11:58 |
| *** bfournie has joined #tripleo | 12:01 | |
| *** pkovar has quit IRC | 12:01 | |
| *** fzdarsky is now known as fzdarsky|afk | 12:01 | |
| *** jeckersb is now known as jeckersb_gone | 12:03 | |
| *** ccamacho has joined #tripleo | 12:05 | |
| sshnaidm | fyi, https://bugs.launchpad.net/tripleo/+bug/1604380 | 12:06 |
| openstack | Launchpad bug 1604380 in tripleo "CI: nodes registrtion in periodic jobs fail because of ironic-inspector error" [High,New] | 12:06 |
| sshnaidm | fyi2, https://bugs.launchpad.net/tripleo/+bug/1604039 | 12:07 |
| openstack | Launchpad bug 1604039 in tripleo "CI: delorean build of tripleo-heat-templates fails because wrong spec" [Undecided,New] | 12:07 |
| *** sshnaidm is now known as sshnaidm|afk | 12:07 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment. https://review.openstack.org/341506 | 12:08 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment. https://review.openstack.org/341506 | 12:10 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable Mistral services https://review.openstack.org/323436 | 12:10 |
| openstackgerrit | Merged openstack/puppet-tripleo: Add principal to certmonger's haproxy helper https://review.openstack.org/343710 | 12:12 |
| *** zoli|lunch is now known as zoli | 12:17 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment. https://review.openstack.org/341506 | 12:19 |
| *** pkovar has joined #tripleo | 12:20 | |
| *** masco has quit IRC | 12:20 | |
| *** tobbe_fiberdata has quit IRC | 12:21 | |
| *** Goneri has quit IRC | 12:22 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/python-tripleoclient: Support node capabilities as dictionaries https://review.openstack.org/344206 | 12:22 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Handle empty list of introspected nodes https://review.openstack.org/344188 | 12:24 |
| EmilienM | hello | 12:25 |
| *** jayg|g0n3 is now known as jayg | 12:25 | |
| EmilienM | dtantsur: hey | 12:29 |
| dtantsur | hi EmilienM | 12:29 |
| *** trown|outtypewww is now known as trown | 12:29 | |
| EmilienM | dtantsur: I'm trying to promote Puppet CI to latest trunk and a commit since last 24h in Ironic is breaking tempest tests or making it unstable | 12:30 |
| EmilienM | http://logs.openstack.org/40/344040/1/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/93fc562/console.html#_2016-07-19_10_38_10_867525 | 12:30 |
| EmilienM | our ironic logs are here: http://logs.openstack.org/40/344040/1/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/93fc562/logs/ironic/ | 12:30 |
| dtantsur | yeah, I'm looking at them, but I don't see the API service doing anything | 12:32 |
| *** radek2 has quit IRC | 12:33 | |
| EmilienM | dtantsur: looks like 500 error in ironic API, see http://logs.openstack.org/40/344040/1/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/93fc562/console.html#_2016-07-19_10_50_21_043366 | 12:34 |
| dtantsur | yeah, but again, there is no signs of it in ironic API logs, unless I misinterpret something | 12:34 |
| dtantsur | the only candidate I can think of is https://github.com/openstack/ironic/commit/15c366580a18c77963d75cbc37e0805c96bc2a50 | 12:35 |
| EmilienM | dtantsur: http://logs.openstack.org/40/344040/1/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/93fc562/logs/apache/ironic_wsgi_error_ssl.txt.gz | 12:35 |
| honza | jtomasek: how frequently should we publish tripleo-ui rpms? on every commit, on release only? | 12:36 |
| dtantsur | EmilienM, yeah, it's probably my suspect. now I wonder why it does not fail in our gate | 12:37 |
| *** Goneri has joined #tripleo | 12:37 | |
| hewbrocca | honza: if you want the UI to eventually be tested in CI, then you probably want them published on every commit? | 12:37 |
| openstackgerrit | Miles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package https://review.openstack.org/344214 | 12:38 |
| shardy | honza: Yeah as hewbrocca says you neet to get a delorean build every commit e.g so it lands in https://trunk.rdoproject.org/centos7/current/ | 12:38 |
| shardy | that is what we'll use in upstream CI | 12:38 |
| *** jdob has joined #tripleo | 12:39 | |
| honza | hewbrocca: shardy: ok, makes sense, thanks // jtomasek | 12:39 |
| *** rlandy has joined #tripleo | 12:40 | |
| *** sshnaidm|afk is now known as sshnaidm | 12:40 | |
| *** dprince has joined #tripleo | 12:41 | |
| jaosorior | dprince: Hey dude, mistral question | 12:41 |
| dprince | jaosorior: hi, sup? | 12:42 |
| jaosorior | dprince: Where are the introspection actions defined in mistral? Seems that the wait_for_finish one is broken. As seen in this logs http://logs.openstack.org/69/342669/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/78969cb/console.html#_2016-07-19_06_15_59_176102 | 12:42 |
| dprince | jaosorior: 'mistral-db-manage populate' is the command that loads them | 12:44 |
| *** jeckersb_gone is now known as jeckersb | 12:44 | |
| *** pradk has quit IRC | 12:44 | |
| dprince | jaosorior: it uses the mapping.json file here... | 12:44 |
| dprince | jaosorior: http://git.openstack.org/cgit/openstack/mistral/tree/mistral/actions/openstack/mapping.json | 12:45 |
| dprince | jaosorior: see the baremetal_introspection actions in that file... | 12:45 |
| jaosorior | well, that seems straight forward | 12:45 |
| jaosorior | dprince: Any idea why it would be failing the way it does? | 12:45 |
| dprince | jaosorior: Invalid input [name=baremetal_introspection.wait_for_finish, class=NoneType, unexpected=[u\'uuids\']]\n'} | 12:46 |
| dprince | jaosorior: the last bit of the error shows the meaningful error | 12:46 |
| openstackgerrit | Miles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package https://review.openstack.org/344214 | 12:47 |
| jaosorior | dprince: right, but what puzzles me is that wait_for_finish does take an input like [u\'uuids\']] | 12:47 |
| jaosorior | which is why I don't understand why it's failing | 12:47 |
| dprince | jaosorior: looks like we changed the interface or something. I had it working but we did jiggle the interface names a bit during review | 12:47 |
| jaosorior | and what is that class=NoneType? | 12:47 |
| *** [1]cdearborn has joined #tripleo | 12:48 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment. https://review.openstack.org/341506 | 12:50 |
| EmilienM | jaosorior: hey dude, quick question: https://review.openstack.org/#/c/343661/ fails - do you think it's related again to quotes? ie; https://review.openstack.org/#/c/343661/3/puppet/services/nova-api.yaml | 12:50 |
| dprince | jaosorior: do you see the baremetal actions getting deployed? 'mistral action-list | grep baremetal' | 12:51 |
| jaosorior | EmilienM: That is ineed the case | 12:53 |
| *** myoung has quit IRC | 12:53 | |
| jaosorior | EmilienM: The resulting hieradata is | 12:53 |
| jaosorior | nova::cron::archive_deleted_rows::destination: /dev/null | 12:53 |
| jaosorior | nova::cron::archive_deleted_rows::hour: */12 | 12:53 |
| EmilienM | jaosorior: :( | 12:53 |
| EmilienM | jaosorior: so I should put "'blabla'" right? | 12:53 |
| dprince | jaosorior: is your patch like passing an empty array or something? | 12:54 |
| *** egafford has joined #tripleo | 12:54 | |
| jaosorior | dprince: Actually the patch sets inspector to a TLS endpoint, and does nothing to the nodes | 12:54 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment. https://review.openstack.org/341506 | 12:54 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Remove ::nova::cron::archive_deleted_rows https://review.openstack.org/343661 | 12:55 |
| *** myoung has joined #tripleo | 12:56 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Remove ::nova::cron::archive_deleted_rows https://review.openstack.org/343661 | 12:56 |
| *** weshay is now known as weshay_mtg | 12:57 | |
| flaper87 | Am I right to assume that there's currently no way to call this without creating a small script? https://github.com/openstack/tripleo-common/blob/master/tripleo_common/image/image_uploader.py#L26 | 12:59 |
| dprince | EmilienM: shouldn't we be filing bugs for all the remaining composable work? | 12:59 |
| EmilienM | dprince: what roles remain? | 13:00 |
| flaper87 | It looks like that code is new(ish) | 13:00 |
| EmilienM | dprince: it's all done, except aodh but it's tracked | 13:00 |
| dprince | EmilienM: all of this hiera code is a remnant of composability | 13:00 |
| EmilienM | yeah I can file a but for that | 13:00 |
| dprince | EmilienM: i.e. we havent' fully decomposed things until it is all moved out I think | 13:00 |
| dprince | EmilienM: I was going to file bugs for my incoming stuff | 13:00 |
| EmilienM | dprince: https://review.openstack.org/#/c/341616/ looks ready | 13:01 |
| dprince | flaper87: yes, we had "old" version of image build and image upload tools a long time ago in tripleo_incubator | 13:01 |
| EmilienM | jaosorior: ^ | 13:02 |
| EmilienM | jaosorior: thx for doing recheck, it's now passing CI | 13:02 |
| flaper87 | dprince: ok, so this is the "new" version that we'll use. I'm asking so I can create a script for it | 13:02 |
| flaper87 | dprince: kinda need it | 13:02 |
| flaper87 | :D | 13:02 |
| EmilienM | dprince: /me filing bugs now | 13:02 |
| flaper87 | Based on the contents of tripleo-common/scripts I'd assume the right thing to do is to add a script there | 13:02 |
| flaper87 | Like, `tripleo-upload-images` | 13:03 |
| flaper87 | Or should this be a mistral action? | 13:03 |
| *** rook-rocketship has quit IRC | 13:03 | |
| dprince | flaper87: is this something like what you are looking for? https://review.openstack.org/#/c/255535/ | 13:04 |
| flaper87 | dprince: yup | 13:05 |
| flaper87 | dprince: exactly that, I guess | 13:05 |
| flaper87 | will review | 13:06 |
| flaper87 | dprince: and thanks | 13:06 |
| *** lucas-hungry is now known as lucasagomes | 13:06 | |
| flaper87 | (review == I'll cry like a baby to get that - or something like that - in asap) | 13:06 |
| flaper87 | mmh, 9 weeks ago | 13:07 |
| flaper87 | jpeeler: dprince could you share why tripleoclient was chosen over a script in tripleo_common? https://review.openstack.org/#/c/255535/ | 13:08 |
| flaper87 | not saying it's wrong, I'm just curious to know if we're moving things into tripleo client now | 13:08 |
| flaper87 | or what the logic is there | 13:08 |
| * flaper87 is still tying pieces together in his head | 13:08 | |
| dprince | flaper87: I like scripts too FWIW | 13:09 |
| *** rook-rocketship has joined #tripleo | 13:09 | |
| *** pkovar has quit IRC | 13:09 | |
| *** lblanchard has joined #tripleo | 13:09 | |
| dprince | flaper87: there are some in TripleO who argue that scripts can't easily be unit tested though. SO I think python was prefered by some for this reason | 13:10 |
| jpeeler | i just built off of what was already done | 13:10 |
| jpeeler | the docker image uploader is heavily based on the glance image uploader | 13:11 |
| EmilienM | dprince: do you want me to create a single bug to track Hiera move? | 13:11 |
| flaper87 | dprince: sure but the script could've been a python script too. That said, I get the point | 13:11 |
| flaper87 | jpeeler: fair enough, thanks for sharing :) | 13:11 |
| flaper87 | jpeeler: are you planning to rebase that patch? | 13:11 |
| flaper87 | jpeeler: mind if I take over ? Is there anything blocking it ? | 13:11 |
| *** akuznetsov has joined #tripleo | 13:11 | |
| jpeeler | flaper87: that's a good question, do people still want to have the docker functionality? | 13:11 |
| flaper87 | jpeeler: I believe the answer is yes but I'll defer to dprince or folks that have been around longer | 13:12 |
| jpeeler | flaper87: i don't mind at all. note that the dependent tripleo-common review is still pending as well | 13:12 |
| flaper87 | jpeeler: yes sir, I'll take those over too | 13:13 |
| jpeeler | oops i didn't mean to say glance image uploader. i meant image builder | 13:13 |
| jpeeler | i was told to create that too a long time ago | 13:13 |
| jpeeler | (the glance image uploader) not being real clear here | 13:14 |
| flaper87 | gotcha, thanks! | 13:14 |
| jaosorior | EmilienM: so, regarding the issue with default values for resources. I could fix that with hiera, right? So here https://review.openstack.org/#/c/332774/24/elements/puppet-stack-config/puppet-stack-config.yaml.template If I want to configure a CA for every certmonger cert. would setting certmonger_certificate::ca: {{CERTIFICATE_GENERATION_CA}} | 13:15 |
| EmilienM | jaosorior: yes I think so | 13:15 |
| jaosorior | so setting that ^^ would use the same ca parameter for all the certmonger_certificate instances, right? | 13:15 |
| jaosorior | ok, gotta try then | 13:15 |
| dprince | EmilienM: how about something like this? https://bugs.launchpad.net/tripleo/+bug/1604412 | 13:16 |
| openstack | Launchpad bug 1604412 in tripleo "Composable services: move hiera settings out of compute role" [Undecided,New] - Assigned to Dan Prince (dan-prince) | 13:16 |
| EmilienM | dprince: well, you want to create multiple bugs then | 13:17 |
| EmilienM | for controller, etc | 13:17 |
| dprince | EmilienM: sure, why not | 13:17 |
| EmilienM | mhh ok | 13:17 |
| dprince | https://bugs.launchpad.net/tripleo/+bug/1604414 | 13:18 |
| openstack | Launchpad bug 1604414 in tripleo "Composable services: move hiera data out of controller role" [Medium,New] | 13:18 |
| dprince | EmilienM: ^^ | 13:18 |
| *** d0ugal has joined #tripleo | 13:19 | |
| EmilienM | cool | 13:20 |
| dprince | jistr: hi, I would like to get this in https://review.openstack.org/#/c/313544/ | 13:20 |
| dprince | jistr: one question, instead of a manifest. Could we live with just 'service_names'? | 13:21 |
| jistr | dprince: yea i guess. Don't we want to capture some service properties too perhaps? | 13:21 |
| dprince | jistr: specifically we could just call the services 'glance-api' and 'glance-api-pacemaker' | 13:21 |
| dprince | jistr: and I think that would give you everything you need | 13:21 |
| dprince | jistr: the service properties to me are similar to 'config' I guess | 13:22 |
| jistr | ok so we could capture things as hiera keys maybe if needed? | 13:22 |
| dprince | jistr: I could live with the manifest blob. but it seems a bit overkill when I think all we need right now is just a list of services | 13:22 |
| jistr | dprince: ok i'm fine with just a list | 13:23 |
| dprince | shardy: is there a way get_attr a nested stack that will give you the "name" of the file or something? | 13:23 |
| jistr | dprince: the service names will probably still need to be a keys in a map though, right? so that we can do merge on them | 13:24 |
| jistr | or maybe YAQL could help? | 13:24 |
| *** pkovar has joined #tripleo | 13:24 | |
| dprince | jistr: okay, I will try it. I may leave your patch there for comparison for now... but I will co-author you for the good idea. Sound good? | 13:24 |
| dprince | jistr: I was going for an array. Is an array not enough? | 13:25 |
| jistr | dprince: sounds good :) | 13:25 |
| *** akshai has joined #tripleo | 13:25 | |
| dprince | jistr: re. the merging. When would an array not be enough? I guess we can debate that in the implementation though | 13:26 |
| jistr | dprince: yea is fine. just wondering now how does heat fetch array items and merge them into an array | 13:26 |
| dprince | jistr: I would just list_join all the service names and pass them wherever we want them (probably into hiera) | 13:26 |
| jistr | dprince: list_join produces a string though, no? | 13:26 |
| jistr | dprince: i guess we could utilize the Heat->Hiera bug to make an array out of a string :D | 13:27 |
| dprince | jistr: yeah, list_join does give us a string. which like you point out may be fine for now | 13:28 |
| jistr | ok | 13:28 |
| dprince | jistr: perhaps just the raw output (array) from the resource_chain for this new attribute set would be preferable then | 13:28 |
| dprince | jistr: we can play with both implementations and see what works... | 13:29 |
| jistr | dprince: yea right, that's what i was missing. If it's a feature of a resource chain that if you define a string on each resource in the chain, and then query the chain as a whole and get array of strings, then we could perhaps do just that instead of feeding it through list_join. | 13:30 |
| dprince | jistr: I think that is how it works. So we might be good | 13:30 |
| *** dtantsur is now known as dtantsur|mtg | 13:31 | |
| jistr | dprince: ack, thanks. Was thinking it would be nice to start converging to using correct data types where possible, so that one day we could hopefully fix the Heat->Hiera bug. | 13:31 |
| jistr | dprince: now that we have YAQL we shouldn't need to pass 'raw' things into Hiera i hope | 13:31 |
| dprince | EmilienM: ^^^ I will use this mechanism for the firewall rules mechanism | 13:31 |
| flaper87 | dprince: if you have 5mins, mind taking a look at https://review.openstack.org/#/c/309623/ ? | 13:31 |
| dprince | jistr: agree | 13:31 |
| *** abehl has quit IRC | 13:33 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move Keystone and Neutron Server firewall rules into the service https://review.openstack.org/338527 | 13:35 |
| EmilienM | dprince: shardy is producing a PoC of it I guess | 13:36 |
| shardy | dprince, jistr, EmilienM: that's one way we could handle it, e.g move the map_merge so we pass both a list of config_settings and the merged map? | 13:36 |
| shardy | EmilienM: I just hacked in my idea for discussion, hope you don't mind me pushing it :) | 13:36 |
| EmilienM | shardy: I love it | 13:36 |
| dprince | EmilienM: is he? shardy do you have a patch for this? | 13:36 |
| shardy | I think we can use the same method to generate the services that are enabled | 13:37 |
| shardy | e.g just add service_name to each config_settings block | 13:37 |
| shardy | dprince: I just pushed it to https://review.openstack.org/#/c/338527/7/puppet/controller.yaml | 13:37 |
| *** bfournie has quit IRC | 13:37 | |
| shardy | (untested) | 13:37 |
| *** fultonj has joined #tripleo | 13:37 | |
| *** fultonj has quit IRC | 13:37 | |
| *** ayoung has joined #tripleo | 13:38 | |
| shardy | Ah, although the hiera keys will collide won't they | 13:38 |
| *** fultonj has joined #tripleo | 13:38 | |
| shardy | I'll have to use repeat to add a prefix | 13:38 |
| dprince | shardy: I would like to not change the services interface at this point I think | 13:39 |
| shardy | dprince: why? | 13:39 |
| *** d0ugal has quit IRC | 13:39 | |
| shardy | dprince: this actually doesn't change the services interface | 13:39 |
| shardy | only what we pass to the node in hiera | 13:39 |
| dprince | shardy: well, I guess so long as the end result is the same. Internal refactor is fine | 13:39 |
| dprince | shardy: if it works I could go for it I guess | 13:40 |
| shardy | dprince: open to other ideas, but FWIW I've been trying some stuff with yaql to enable composable bind_ip's and it gets pretty scary | 13:40 |
| dprince | shardy: I was going to take a slightly different approach and "prefix" the firewall rules with the service name | 13:40 |
| EmilienM | let's see if it works, I'll let you know | 13:40 |
| shardy | I actually implemented a new map_replace function for heat this morning so I could avoid using it ;) | 13:40 |
| EmilienM | wow :) | 13:40 |
| shardy | dprince: Cool, happy to consider either approach, this is just a 5min idea I had :) | 13:41 |
| dprince | shardy: I wasn't targetting this mechanism as something we would use for bind/local IPs. | 13:41 |
| dprince | shardy: I don't see a good way of moving those out just yet, but they aren't causing any harm to maintain in a single blob I think either | 13:41 |
| shardy | dprince: No, I was thinking we could either do a hiera lookup for the contents of the NetIpMap, or a map replacement on the merged config_settings | 13:41 |
| dprince | shardy: we could template them I guess | 13:42 |
| *** zoli is now known as zoli|brb | 13:42 | |
| dprince | shardy: yep, that could work | 13:42 |
| dprince | shardy: anyways EmilienM wanted help with firewall rules. Which is different | 13:42 |
| shardy | Yeah, I think we just need to dump the content of NetIpMap onto each node and somehow make the services/* config_settings reference it | 13:42 |
| dprince | shardy: and I think so long as we use map_merge the output would be the same | 13:42 |
| dprince | shardy: so I was going to create a "prefix" mechanism and just go with that. | 13:43 |
| shardy | dprince: Cool, go for it, I'll give this a try anyway & we can compare | 13:43 |
| shardy | EmilienM: I'll push any further prototyping under a different ChangeId so I don't stop on your patch again | 13:44 |
| shardy | stomp I mean | 13:44 |
| EmilienM | ok | 13:44 |
| dprince | shardy: correct me if I'm wrong. But all you are doing is moving the map_merge location right? and regardless of where we do that it is still going to merge up duplicate keys right? | 13:44 |
| dprince | shardy: I mean it will... | 13:44 |
| EmilienM | I'm a bit confusing by all your ideas right now | 13:44 |
| EmilienM | I just want to use hiera_merge which is an hiera feature we can't use right now | 13:45 |
| dprince | shardy: I don't think this works with what EmilienM intends... where as the service_names prefix idea will I think | 13:45 |
| shardy | dprince: No, I'm passing the unmerged and merged data to the node via hieradata | 13:45 |
| shardy | dprince: because EmilienM said he wanted to use hiera to merge it | 13:45 |
| shardy | dprince: Ok, no worries, just trying to kick-off the discussion | 13:46 |
| *** abehl has joined #tripleo | 13:46 | |
| ccamacho | guys did anyone installed tripleo-ui in the undercloud? | 13:46 |
| dprince | shardy: I see. So the service_configs_list is what you were going for | 13:46 |
| dprince | shardy: it could work, but the duplicate data structures with the same data are annoying I think | 13:47 |
| shardy | dprince: Yeah, then you could look up a list for all services in hiera | 13:47 |
| dprince | shardy: I gotta say. What I really want here is a proper hiera hook | 13:47 |
| shardy | dprince: Initially I added config_settings_list to the service templates, but then I thought this was a lower impact way (despite the data duplication) | 13:47 |
| dprince | shardy: and then we have a separate hiera file for each service I think. ANd those merge in hiera very nicely | 13:47 |
| shardy | dprince: Yeah, did you see stevebaker raised a bug about that? | 13:48 |
| shardy | apparently the o-a-c based hiera script is pretty slow | 13:48 |
| dprince | shardy: I hadn't seen his bug | 13:48 |
| shardy | so we may want to rewrite it for performance reasons anyway | 13:48 |
| dprince | shardy: but the script way isn't great. So many o-a-c calls | 13:48 |
| shardy | https://bugs.launchpad.net/tripleo/+bug/1596373 | 13:49 |
| openstack | Launchpad bug 1596373 in tripleo "40-hiera-datafiles takes over 20 seconds to run" [High,Triaged] | 13:49 |
| dprince | shardy: throwing twice as much hiera data at the hiera element is only going to make it slower FWIW | 13:49 |
| EmilienM | weekly meeting in 10 min, please add off items if needed: https://etherpad.openstack.org/p/tripleo-meeting-items | 13:50 |
| shardy | dprince: Yeah I guess so, but we can fix that - do you have any examples of your proposed approach for comparison? | 13:50 |
| dprince | shardy: I explained to Emilien that I thought the service names approach was a reasonable compremise now. ANd that long term a proper hiera hook that lives here http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements | 13:51 |
| dprince | shardy: no code yet. I was about to do it today based on my earlier conversation w/ jistr. EmilienM said you were working on it as well so that is why I asked | 13:51 |
| dprince | shardy: my approach would require a 'service_name' in each nested stack | 13:52 |
| EmilienM | no, just the PoC for composable variables | 13:52 |
| dprince | EmilienM: these things are related because they are going after similar mechanisms I think | 13:52 |
| EmilienM | oh nevermind, yes indeed :) | 13:52 |
| dprince | shardy: anyways, we probably want the service names in the nested stacks regardless because it helps with other things like upgrades. I think it would be useful to go on and add that now | 13:53 |
| shardy | dprince: +1 that sounds like a good first step, agreed | 13:53 |
| ayoung | shardy, what are you using for a development setup? Are you working single-system virt, or multiple physical? HA? | 13:54 |
| ayoung | dprince, same quesion for you. We need to get people up to speed on OOO dev and I want to make sure we are doing roughly the same thing as the core devs are doing | 13:56 |
| ayoung | I was under the impression that Quickstart had become the developers tool of choice, but I am starting to question that assumption | 13:58 |
| *** yamahata has joined #tripleo | 13:58 | |
| EmilienM | meeting in 2 min folks | 13:58 |
| ccamacho | ayoung /me using tripleo.sh | 13:59 |
| ayoung | EmilienM, in #openstack-meeting? | 13:59 |
| EmilienM | ayoung: -alt | 13:59 |
| dprince | ayoung: I still use a lot of tripleo-incubator and tripleo.sh too | 13:59 |
| ayoung | ccamacho, thanks | 13:59 |
| shardy | ayoung: I'm using instack-virt-setup and tripleo.sh https://paste.fedoraproject.org/392510/46893675 | 13:59 |
| dprince | ayoung: I think we have a mix of old and new tooling at this point | 13:59 |
| shardy | because it's clsoe to what we currently CI | 13:59 |
| jaosorior | ayoung: I'm using quickstart and tripleo.sj | 14:00 |
| jaosorior | * tripleo.sh | 14:00 |
| ayoung | dprince, shardy ccamacho and focused on HA dev, not .... whateve we call without-ha | 14:00 |
| jaosorior | ayoung: http://jaormx.github.io/2016/deploying-master-tripleo-quickstart/ | 14:00 |
| *** pradk has joined #tripleo | 14:00 | |
| ayoung | jaosorior, rock on! | 14:00 |
| shardy | ayoung: No, I don't test much with HA due to lack of ram for multiple controllers | 14:01 |
| ayoung | shardy, OK...so we are all in the same boat :) | 14:01 |
| ccamacho | ayoung in my case both ha and non-ha usually following http://www.anstack.com/blog/2016/07/04/manually-installing-tripleo-recipe.html or http://www.anstack.com/blog/2016/07/07/tripleo-deployment-with-inluch.html and then playing with the openstack overcloud environments options | 14:02 |
| jaosorior | ayoung: I usually only do ha to be honest | 14:02 |
| ayoung | ccamacho, added to my feedly! Thanks | 14:02 |
| ayoung | jaosorior, multiple physical machines? | 14:03 |
| ccamacho | ayoung nice! | 14:03 |
| jaosorior | ayoung: virtual | 14:03 |
| ayoung | jaosorior, how many VMs of what size do you need in order to do that? I think we only have one machine on the team capable of running sufficient | 14:05 |
| gfidente | ayoung, though you can deploy with HA on single node | 14:05 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Use certmonger for automatic cert generation https://review.openstack.org/332774 | 14:05 |
| ayoung | gfidente, a quorum of 1? | 14:05 |
| gfidente | that reads as, with pacemaker even though running on a single controller which is closer to what you want I think | 14:05 |
| gfidente | ayoung, yeah it'll go through the pacemaker codepath | 14:06 |
| gfidente | even though there is a single controller | 14:06 |
| ayoung | gfidente, yeah, I want to make sure that I am coding in the right set of files to make things work in an HA environement, but only have a single controller's worth of VMs available | 14:06 |
| jaosorior | EmilienM: hey dude, I need some help with https://review.openstack.org/332774 it's still not taking the CA I'm trying to set as default, and I tried to set it both via hiera and via the resource declaration | 14:06 |
| jaosorior | can you take a look? | 14:06 |
| ayoung | gfidente, I'm headed to the Keystone midcycle later on today, and we are going to be heavily discussing 0 downtime upgrades. I want to be able to apply that to Tripleo as soon as we have it working in Keystone | 14:07 |
| ayoung | and I think HA is going to have to play nice with that | 14:07 |
| ayoung | I suspect that what I will really need to test it is a 2 controller node set up | 14:07 |
| *** akuznetsov has quit IRC | 14:07 | |
| ayoung | Which I think I can squeeze out in my current setup.... | 14:08 |
| jaosorior | ayoung: this is pretty much my quickstart config http://paste.openstack.org/show/537433/ | 14:08 |
| *** liverpooler has quit IRC | 14:08 | |
| *** jcoufal has joined #tripleo | 14:08 | |
| ayoung | jaosorior, how much RAM do you need to make that happen? | 14:09 |
| jaosorior | ayoung: My machine has around 32G in total | 14:09 |
| ayoung | jaosorior, ok, that is what I have...I'll try your setup\ | 14:09 |
| *** rook-rocketship has quit IRC | 14:10 | |
| *** zoli|brb is now known as zoli | 14:10 | |
| *** zoli is now known as zoliXXL | 14:10 | |
| ayoung | jaosorior, BTW, do you symlink ~/.quickstart/playbooks etc to some other repo? I' | 14:10 |
| ayoung | ve had to do that for a few directories in order to have my git repo survive a clean wipe | 14:10 |
| jaosorior | I don't | 14:11 |
| EmilienM | jaosorior: will look after meeting | 14:11 |
| ayoung | jaosorior, I think it was because I was writing my own playbooks, not just roles | 14:12 |
| *** bfournie has joined #tripleo | 14:12 | |
| *** rook has joined #tripleo | 14:13 | |
| *** rajinir has joined #tripleo | 14:17 | |
| *** d0ugal has joined #tripleo | 14:18 | |
| weshay_mtg | matbu, apetrich ping re: https://review.openstack.org/#/c/336442/ | 14:20 |
| apetrich | weshay_mtg, looking | 14:20 |
| weshay_mtg | apetrich, matbu wondering since we're hitting issues w/ mitaka.. if we could fire up the third party ci for upgrade starting w/ liberty -> mitaka | 14:21 |
| *** abehl has quit IRC | 14:21 | |
| weshay_mtg | apetrich, take a look at EmilienM's comment on that review | 14:22 |
| apetrich | yeah I just did. | 14:22 |
| apetrich | weshay_mtg, I was thinking the same lines. I'll run one | 14:22 |
| weshay_mtg | EmilienM, ^ k.. thank you apetrich | 14:22 |
| *** d0ugal has quit IRC | 14:23 | |
| apetrich | weshay_mtg, EmilienM but right now we just upgrade the UC that has little effect on it. I'm changing the patch for a full upgrade in a few minutes | 14:23 |
| EmilienM | ooo meeting in progress FYI | 14:24 |
| weshay_mtg | apetrich, for liberty -> mitaka you have the full upgrade.. | 14:24 |
| weshay_mtg | ya.. sry | 14:24 |
| weshay_mtg | will wait | 14:24 |
| *** paramite is now known as paramite|afk | 14:24 | |
| *** d0ugal has joined #tripleo | 14:24 | |
| weshay_mtg | apetrich, we need to figure out what version he needs there.. but I suspect liberty -> mitaka | 14:24 |
| weshay_mtg | which is green | 14:24 |
| apetrich | that patch is for master | 14:24 |
| apetrich | I'm affraid it might bring more stuff than we are gating | 14:25 |
| matbu | EmilienM: there is no working upgrade job atm for master | 14:27 |
| EmilienM | matbu: one sec please, we're in meeting | 14:27 |
| *** radek2 has joined #tripleo | 14:30 | |
| *** skramaja has quit IRC | 14:30 | |
| *** ohamada_ has joined #tripleo | 14:32 | |
| *** ohamada has quit IRC | 14:32 | |
| *** jdob has quit IRC | 14:34 | |
| *** abehl has joined #tripleo | 14:34 | |
| *** jdob has joined #tripleo | 14:34 | |
| *** pgadiya has quit IRC | 14:36 | |
| *** apetrich has quit IRC | 14:38 | |
| *** skramaja has joined #tripleo | 14:39 | |
| *** apetrich has joined #tripleo | 14:42 | |
| *** jdob has quit IRC | 14:43 | |
| *** jdob has joined #tripleo | 14:43 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud: Fix ceilometer auth url on undercloud https://review.openstack.org/344279 | 14:48 |
| *** julim has joined #tripleo | 14:49 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud: Fix ceilometer auth url on undercloud https://review.openstack.org/344284 | 14:54 |
| *** devvesa has joined #tripleo | 14:58 | |
| *** jaosorior has quit IRC | 14:58 | |
| *** d0ugal has quit IRC | 14:59 | |
| *** fzdarsky|afk is now known as fzdarsky | 15:00 | |
| *** beagles is now known as beagles_brb | 15:00 | |
| *** paramite|afk is now known as paramite | 15:01 | |
| *** dtantsur|mtg is now known as dtantsur | 15:02 | |
| *** pkovar has quit IRC | 15:03 | |
| *** mgarciam has joined #tripleo | 15:04 | |
| *** tobias_fiberdata has joined #tripleo | 15:04 | |
| jpich | florianf: I took the liberty of subscribing you to https://blueprints.launchpad.net/tripleo-ui/+spec/tripleo-ui-mistral-refactoring - if you want to flesh out a bit or correct my mistakes in the whiteboard re:plan creation, that'd be awesome :-) | 15:08 |
| *** kairat has joined #tripleo | 15:11 | |
| *** jcoufal_ has joined #tripleo | 15:12 | |
| *** weshay_mtg is now known as weshay | 15:13 | |
| *** bootsha has quit IRC | 15:13 | |
| *** jcoufal has quit IRC | 15:15 | |
| ayoung | trown, as my team has been trying to (and failing to) affect change in the deployment, I wonder where the line is drawn between what is on the image and what is done by tripleo. Am I correct in understanding that puppet modules are in the image and hiera data is in the Metadata server, and this is the contract between heat and the controller? | 15:16 |
| *** milan has quit IRC | 15:16 | |
| *** pkovar has joined #tripleo | 15:19 | |
| *** tobias_fiberdata has quit IRC | 15:22 | |
| shardy | ayoung: Yes the puppet modules are in the image, and the hieradata is provided via heat metadata | 15:24 |
| shardy | ayoung: however for development the puppet modules can be uploaded at deployment time | 15:24 |
| shardy | e.g to enable a quicker hack/test loop | 15:25 |
| shardy | https://etherpad.openstack.org/p/tripleo-deep-dive-topics | 15:25 |
| shardy | see lines 23-31 there | 15:25 |
| ayoung | shardy, ah, thanks | 15:25 |
| shardy | basically we have a script that can download a tarball and unpack it on the node before we run puppet | 15:25 |
| shardy | that script automates pushing it into the undercloud swift and generating a tempurl which is read by the script on deployment | 15:25 |
| ayoung | shardy, so, for Federation, I have a similar issue. We have a tool that is used to register the Service provider (Keystone in this case) with the SAML2 IdP (Keycloak) | 15:26 |
| ayoung | the tool is designed to run on the host | 15:26 |
| ayoung | but, in an HA world, that sounds like it is violating the separat ion of concernts | 15:26 |
| *** athomas has quit IRC | 15:26 | |
| ayoung | as the data would have to be synchronized across multiple HA-controlled Keystone servers | 15:26 |
| shardy | ayoung: Not necessarily. we deploy a "bootstrap_nodeid" which allows you to run things only on one node | 15:27 |
| shardy | e.g we only run DB sync for services once, on the bootstrap node | 15:27 |
| shardy | sounds like this is similar | 15:27 |
| ayoung | shardy, yes | 15:27 |
| ayoung | shardy, so the workflow is something like this: | 15:28 |
| *** marios has quit IRC | 15:28 | |
| *** ebalduf_ has joined #tripleo | 15:28 | |
| shardy | https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/pacemaker/keystone.pp#L47 | 15:28 |
| shardy | ayoung: that shows how the data is consumed | 15:28 |
| *** marios has joined #tripleo | 15:28 | |
| shardy | in various places there are conditionals in the puppet profiles | 15:28 |
| *** ohamada_ has quit IRC | 15:29 | |
| ayoung | 1: call three keystone APIs to build Idp, mapping, protocol. either before or after, run the python based tool to register. So all that sould be done once, and then the generated files would get synced | 15:29 |
| *** athomas has joined #tripleo | 15:31 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Repurpose KeystoneWorkers add keystone::wsgi::apache::threads https://review.openstack.org/336472 | 15:31 |
| ayoung | shardy, in the ansible based prototype we had, we made keystone query calls to see if the various objects had been made to avoid re-creating the objects, and used the file objects to avoid re-calling the registration tool. I'm guessing...all of that should be done by puppet modules? And then...the hiera data would have a subsection that would reflect that structure? | 15:31 |
| shardy | ayoung: Yeah the first step is to wire in the logic needed to the puppet-keystone module | 15:32 |
| shardy | https://github.com/openstack/puppet-keystone/tree/master/manifests/federation | 15:32 |
| ayoung | shardy, the problem is that the federation puppet module was written for a very different use case | 15:32 |
| shardy | ayoung: to clarify, you run something on one node, then we have to automate copuing things to other nodes? | 15:32 |
| ayoung | shardy, someone squatted on the good name. That is really for Keystone2Keystone federation | 15:33 |
| greghaynes | ianw: cinerama I have an old patch https://review.openstack.org/#/c/181162/ that might be off your review radars - I'm hoping to get that merged before another regression slips in | 15:33 |
| ayoung | shardy, yes | 15:33 |
| shardy | ayoung: Ok, I guess you can take that up with EmilienM and the other puppet-keystone maintainers | 15:33 |
| shardy | from the tripleo perspective, it can be any module, that just seems like a logical place to wire it in | 15:34 |
| *** beagles_brb is now known as beagles | 15:34 | |
| ayoung | shardy, what is the relationship between puppet and hiera data? What says that puppet should find the set of Idps (or anything else) to manage from this subsection of hiera? | 15:34 |
| EmilienM | shardy: what? can I have tl;dr | 15:34 |
| ayoung | EmilienM, Federation | 15:34 |
| ayoung | EmilienM, Keystone puppet module stuff...I bugged you about it a few months back | 15:35 |
| shardy | ayoung: on each node we create a bunch of yaml files under /etc/puppet/hieradata | 15:35 |
| *** numans has quit IRC | 15:35 | |
| *** d0ugal has joined #tripleo | 15:35 | |
| shardy | ayoung: basically they are key/value pairs that are passed into the puppet modules that get applied | 15:35 |
| ayoung | shardy, OK, my HA deploy via quickstart seems to have just completed. So let me look at the hiera data for Keystone and see if I can make sense of it | 15:36 |
| shardy | ayoung: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/keystone.yaml#L105 | 15:36 |
| EmilienM | I'm not sure how can I help here, but please let me know | 15:36 |
| ayoung | shardy, I should look at the cached data in /var, right? | 15:36 |
| shardy | that's where it is defined in the heat templates, if you look inside /etc/puppet/hieradata/service_configs.yaml you should see the data (assuming you're on trunk not mitaka) | 15:37 |
| EmilienM | trown: we can wait before landing https://review.openstack.org/#/c/341616/ | 15:38 |
| ayoung | shardy, my understanding of hiera is that there is pre-packaced stuff in /etc/puppet/ but that gets overlayed by os-collect-config | 15:38 |
| EmilienM | trown: when ready, feel free to +a | 15:38 |
| ayoung | in /var/lib/os-collect-config/ I see a bunch of json files | 15:38 |
| shardy | ayoung: No, we just write some files to /etc/puppet/hieradata | 15:38 |
| ayoung | shardy, we update /etc after the node boots? | 15:38 |
| shardy | ayoung: well, yeah, but that's kinda confusing the issue, we use os-collect-config and some other agents to write the hieradata yaml files, and then later to run puppet | 15:39 |
| ayoung | ok...the json files look similar | 15:39 |
| trown | EmilienM: cool, I will test it right now | 15:39 |
| shardy | ayoung: Yes, we write those files after the node boots, here: | 15:39 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/controller.yaml#L685 | 15:39 |
| EmilienM | trown: thx | 15:39 |
| *** Goneri has quit IRC | 15:40 | |
| ayoung | shardy, I'm a little confused at how orchestration happens. There are a bunch of steps, like heat_config_ControllerOvercloudServicesDeployment_Step4.json | 15:42 |
| ayoung | I assume there is a callback from the controller to heat saying "ok this is done, give me the next" | 15:42 |
| shardy | https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/pacemaker/keystone.pp#L74 | 15:42 |
| ayoung | and then the next chunk of metadata is posted? More hiera data? | 15:43 |
| shardy | ayoung: Yes, we apply puppet six times, and have conditionals in the puppet code like if $step >= 3 | 15:43 |
| shardy | ayoung: The hieradata is all written out at the start, we just re-run puppet with the same manifest and a different "step" variable | 15:43 |
| ayoung | shardy, does heat drive that, or is it all done on the node, and heat just collects the status? | 15:44 |
| shardy | this allows us to re-assert all previous steps each time, so e.g step 5 doesn't undo something step3 did by mistake | 15:44 |
| shardy | ayoung: http://hardysteven.blogspot.co.uk/2015/05/heat-softwareconfig-resources.html | 15:44 |
| shardy | tha may help, as will the deep-dive recording from last week that I did | 15:45 |
| ayoung | shardy, OK, so the Federation code needs to be run after everything else is set up for keystone. It will, as a last step, need to restart the HTTPD instance | 15:45 |
| ayoung | and that kind of restart needs to play nice with the HA/pacemaker, right? | 15:45 |
| shardy | ayoung: tl;dr heat just updates the metadata with a new step value, which makes os-collect-config re-run the agents to run puppet | 15:45 |
| ayoung | shardy, Oh, so Heat *does* control the step | 15:46 |
| * ayoung lightbulb | 15:46 | |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/controller-post.yaml#L55 | 15:46 |
| shardy | ayoung: yes ^^ | 15:46 |
| shardy | ayoung: I'd say focus first on getting puppet to run things at the time you want, then we can talk later about the copying between nodes requirement | 15:48 |
| ayoung | shardy, OK...so that is how Heat orchestrates across multipe servers, too. So in the case of the Federation generated files, Heat would wait until the generation was done on one server, and then sync it to each in turn, triggering the HTTPD restart | 15:48 |
| ayoung | shardy, I want to spec the process out, soup to nuts | 15:49 |
| shardy | ayoung: Ok, well FYI there's a missing interface for the sync/restart part that will require some thought | 15:52 |
| ayoung | shardy, for HTTPD? | 15:53 |
| shardy | one way we might do that is just generate a writeable tempurl on the undercloud swift, then pass it in so the master node writes some data, and the non-master nodes poll the bucket waiting for it to be there | 15:53 |
| shardy | ayoung: to have one node pass data to the other nodes | 15:53 |
| *** akuznetsov has joined #tripleo | 15:54 | |
| *** pkovar has quit IRC | 15:54 | |
| ayoung | shardy, I'm a little worried about that. The SAML stuff, I think, uses a private key. Ugh, this is a mess | 15:54 |
| ccamacho | thrash around?! | 15:55 |
| *** tremble has quit IRC | 15:55 | |
| ayoung | we want the IdP to think of it as a single Service provider, but then we have 3+ instances actually saying "here is the rone and true identity" | 15:55 |
| ayoung | shardy, let me confer with the SAML2 experts on this and see if we can come to some sane approach | 15:55 |
| *** pkovar has joined #tripleo | 15:56 | |
| shardy | ayoung: Ok there are other ways but they all involve bouncing the data via the undercloud (either via heat or swift) | 15:56 |
| shardy | there's no peer-to-peer channel available | 15:56 |
| *** oshvartz has quit IRC | 15:56 | |
| gfidente | shardy, ayoung, I think the wanted behaviour though is similar to what we need to fix the swift rings building | 15:56 |
| *** devvesa has quit IRC | 15:56 | |
| shardy | gfidente: Yeah, I was just thinking that | 15:56 |
| *** tremble has joined #tripleo | 15:56 | |
| *** tremble has joined #tripleo | 15:56 | |
| gfidente | as we'll want one node to run the build and the resulting file to be copied on the others | 15:56 |
| shardy | perhaps we can solve both | 15:56 |
| hewbrocca | I was just gonna say, don't we already do this with swift? | 15:56 |
| hewbrocca | oh, we don't | 15:57 |
| shadower | shardy: sorry, I'm on vacation and didn't attend the meeting | 15:57 |
| shadower | shardy: could you summarise it, please? | 15:57 |
| openstackgerrit | Dan Prince proposed openstack/os-collect-config: Only re-exec on config change or HUP https://review.openstack.org/344338 | 15:58 |
| hewbrocca | It's simple, we just need to finish off that reimplementation of puppetdb | 15:58 |
| hewbrocca | :) | 15:58 |
| shardy | shadower: sure, do you want to maintain tripleo-validations in launchpad, or just use a tag in the main tripleo launchpad? | 15:58 |
| shardy | shadower: basically there's some fragmentation wrt tracking releases and we're trying to have a discussion around cleaning things up and consolidating | 15:59 |
| hewbrocca | hey dprince that review above ^^^ | 15:59 |
| shadower | shardy: I'm easy. We've put a few issues on the -validations one but whatever you prefer would be fine | 15:59 |
| shadower | using tripleo is perfectly fine by me | 15:59 |
| gfidente | shardy, ayoung so one simple way to solve this outside of heat and within puppet maybe could be an rsync, triggered one step after the files are built | 15:59 |
| hewbrocca | dprince: oh, never mind, I misunderstood what it'đ afford | 15:59 |
| shardy | shadower: really it depends on whether you want tripleo-validations to be tagged and released as part of the coordinated TripleO release? | 15:59 |
| shardy | shadower: if we want it to be considered part of the milestone deliverables, I'd rather it was in the main TripleO LP | 15:59 |
| hewbrocca | I misunderstood what it's for | 16:00 |
| shardy | as that's the only one I regularly track in the run up to a milestone | 16:00 |
| hewbrocca | sheesh | 16:00 |
| dprince | hewbrocca: yep, it isn't directly related to this if that is what you are wondering https://review.rdoproject.org/r/#/c/1678/ | 16:00 |
| shadower | shardy: I think we want to do that. Ideally, I'd like them to be closely integrated within the main workflow | 16:00 |
| dprince | hewbrocca: it is more related to optimizing for zaqar websockets... | 16:00 |
| hewbrocca | exactly | 16:00 |
| shadower | which means they sohuld be released at the same time, too | 16:00 |
| hewbrocca | ignore me, sorry | 16:00 |
| *** penick has joined #tripleo | 16:00 | |
| dprince | hewbrocca: its good man. Keep us in check | 16:00 |
| shardy | shadower: ack, thanks - we can look at creating a validations tag then and retiring the validations (and tripleo-common) LP, thanks! | 16:00 |
| gfidente | shardy, ayoung not sure there if it should be push or pull, probably push can be done by the producer within single step using puppet dependencies | 16:01 |
| shadower | shardy: great, thanks! mandre ^ FYI | 16:01 |
| shardy | jpich: ^^ FYI | 16:01 |
| ayoung | gfidente, I just had a side convo with a coworker...the more I think about it, the more complex it becomes. It might be a Barbican use case. Sharing private keys... | 16:02 |
| hewbrocca | Barbican is actually a good idea | 16:03 |
| dprince | shardy: thoughts on this https://review.openstack.org/#/c/344338/ | 16:03 |
| shardy | ayoung: ack - what credentials are required to access barbican, e.g if we were to run it on the undercloud? | 16:03 |
| *** leanderthal is now known as leanderthal|afk | 16:03 | |
| hewbrocca | it's like puppetdb for keys | 16:03 |
| hewbrocca | :) | 16:03 |
| dprince | shardy: a cleanup and a step towards not re-execing as much | 16:03 |
| jpich | Cool! Sounds like One Tracker To Rule Them All is the preferred approach in general :) | 16:03 |
| shardy | dprince: +1, lgtm | 16:04 |
| *** d0ugal has quit IRC | 16:04 | |
| ayoung | shardy, hmmmm, I think these would be considered resources managed by heat-admin? Or maybe the Keystone user? I'm not sure, I need to think it through. I'll discuss with the other Keystone core tomorrowish | 16:05 |
| shardy | ayoung: That's the reason for my question, we only ever deploy credentials in the special heat_stack domain, so we don't leak tenant credentials onto the nodes | 16:06 |
| gfidente | also, I think barbican can work as storage but it won't really fix the issue with orchestrating the data exchange | 16:07 |
| openstackgerrit | Merged openstack/instack-undercloud: ironic: turn on ipxe timeout https://review.openstack.org/294787 | 16:07 |
| *** zoliXXL is now known as zoli|gone | 16:07 | |
| gfidente | sounds like we'd have to do it with steps in puppet anyway | 16:07 |
| hewbrocca | gfidente: couldn't you just let it race? | 16:07 |
| hewbrocca | IOW, have all three nodes generate keys and try to write them to barbican | 16:08 |
| ayoung | shardy, it might not be approprate for barbican. There is a file format (PKCS11) that is used for sharing secrets betwen machines. It might be that we can use that to save the files we need to swift, and then unpack on the remote machines, but It still needs a secret of some sort. heat could manage that secret, but I don't like that.... | 16:08 |
| hewbrocca | first one wins, the other two give up and then retrieve the one that's already there | 16:08 |
| *** zoli|gone is now known as zoli_gone-proxy | 16:08 | |
| *** ifarkas is now known as ifarkas_afk | 16:08 | |
| shardy | ayoung: sure, open to suggestions on that. FWIW we already store all the paswords for the overcloud services in the heat DB | 16:09 |
| shardy | (encrypted obviously, but the key exists on the undercloud node) | 16:09 |
| ayoung | shardy, ok, we have options...this will be a lovely problem to mull over tonight on a cross country flight | 16:09 |
| shardy | If someone gets root access to the undercloud it's basically game over anyway tho tbh | 16:10 |
| gfidente | hewbrocca, probably, but for swift it's more than a single ring so we'd have to make it atomic on the entire set of files | 16:10 |
| gfidente | though we have this bootstrap_nodeid concept in place already so I was thinking to use just that for the time being | 16:10 |
| shardy | gfidente: Yeah, that's why I was thinking have the bootstrap node push the data to swift, then in a subsequent step have the other nodes read it | 16:11 |
| gfidente | but I am not good at big steps, I can only take small steps so maybe there is an enirely different approach which I don't even see | 16:11 |
| *** akuznetsov has quit IRC | 16:11 | |
| ayoung | shardy, let me confirm first that we need to have a single private key. It gets much simpler if that is not the case | 16:12 |
| gfidente | shardy, push to undercloud swift? | 16:12 |
| shardy | gfidente: Yes, we generate a tempurl and pass it to the overcloud nodes as a parameter | 16:12 |
| gfidente | ok I see | 16:13 |
| *** beagles is now known as beagles_brb | 16:13 | |
| *** fultonj has quit IRC | 16:15 | |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-quickstart: WIP gate upgrade https://review.openstack.org/342161 | 16:19 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-image-elements: Add zaqar.use_websockets option to o-c-c https://review.openstack.org/344352 | 16:20 |
| *** fultonj has joined #tripleo | 16:21 | |
| apetrich | EmilienM, is this https://review.openstack.org/#/c/336442 a liberty to mitaka issue or a mitaka to master? | 16:25 |
| ayoung | shardy, to be clear: all of the *_Step[1-6].json are posted to the metadata at the same time, and all are fetched together via os-collect-config, which updates its local cache. it also gets the step value which it stores somewhere in /var/lib/os-collect-config/ . Then os-refresh-config picks that up as does os-apply-config. Is it os-apply-config that decides what to run, based on the step value, or is it os-refresh-config, and why are | 16:27 |
| ayoung | they two separate things? | 16:27 |
| *** penick_ has joined #tripleo | 16:28 | |
| *** bvandenh has quit IRC | 16:29 | |
| *** jpich has quit IRC | 16:29 | |
| *** mgarciam has quit IRC | 16:29 | |
| *** Goneri has joined #tripleo | 16:30 | |
| *** penick has quit IRC | 16:31 | |
| *** penick_ is now known as penick | 16:31 | |
| ayoung | also...what is the mechanism that notifies Heat that the current step is completed? | 16:32 |
| shardy | ayoung: pls check the blog post I linked earlier, it does explain all of that IIRC | 16:36 |
| shardy | happy to answer any remaining questions if clarification is needed | 16:37 |
| ayoung | shardy, OK...thanks | 16:37 |
| *** abehl has quit IRC | 16:37 | |
| *** dtantsur is now known as dtantsur|afk | 16:38 | |
| shardy | https://etherpad.openstack.org/p/tripleo-deep-dive-topics also has a link to the software config deep dive I did last week, which contains a bit more detail about what happens on the nodes, and how to debug including running the puppet manifest on the node outside of heat/os-collect-config | 16:38 |
| *** d0ugal has joined #tripleo | 16:38 | |
| shardy | ayoung: ^^ | 16:38 |
| ayoung | shardy, excellent | 16:39 |
| ayoung | shardy, I'm still going through the one from the week before. I'll try to catch up | 16:39 |
| *** trown is now known as trown|lunch | 16:44 | |
| *** tesseract- has quit IRC | 16:47 | |
| *** tremble has quit IRC | 16:48 | |
| mandre | shardy: got it, I'm also for retiring the tripleo-validations LP | 16:48 |
| *** tremble has joined #tripleo | 16:49 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: WIP: Test SoftwareConfigTransport: ZAQAR_MESSAGE https://review.openstack.org/344365 | 16:49 |
| *** rcernin has quit IRC | 16:49 | |
| *** abehl has joined #tripleo | 16:50 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-image-elements: Add zaqar.use_websockets option to o-c-c https://review.openstack.org/344352 | 16:50 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci https://review.openstack.org/324777 | 16:54 |
| *** ebarrera has quit IRC | 16:55 | |
| slagle | review plz :) ^ or i will wear out the "r-e-b-a-s-e" keys on my kb | 16:56 |
| *** derekh has quit IRC | 17:00 | |
| *** tremble has quit IRC | 17:09 | |
| *** aufi has quit IRC | 17:10 | |
| *** fultonj has quit IRC | 17:11 | |
| *** dsariel has quit IRC | 17:12 | |
| *** fultonj has joined #tripleo | 17:12 | |
| openstackgerrit | Merged openstack/instack-undercloud: Allow customization of undercloud disk size https://review.openstack.org/344161 | 17:19 |
| *** fzdarsky is now known as fzdarsky|afk | 17:21 | |
| *** fzdarsky|afk has quit IRC | 17:23 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: WIP: Test SoftwareConfigTransport: ZAQAR_MESSAGE https://review.openstack.org/344365 | 17:27 |
| *** egafford has quit IRC | 17:27 | |
| *** pkovar has quit IRC | 17:30 | |
| *** paramite is now known as paramite|afk | 17:31 | |
| sshnaidm | shardy, did you have a chance to talk to apevec about delorean failures? | 17:35 |
| *** beagles_brb is now known as beagles | 17:35 | |
| openstackgerrit | Merged openstack/os-net-config: Make os-net-config idempotent with Linux bonds and MAC mapping https://review.openstack.org/336313 | 17:36 |
| EmilienM | apetrich: I don't know if it's an issue, but I want this patch tested from Mitaka to Newton | 17:36 |
| shardy | sshnaidm: No, IIRC I suggested you jump into #rdo and ask apevec or trown about it | 17:37 |
| *** shivrao has joined #tripleo | 17:37 | |
| shardy | I won't be able to look into it until tomorrow now unfortunately | 17:37 |
| openstackgerrit | Dan Prince proposed openstack/python-tripleoclient: Use the messaging-websocket endpoint https://review.openstack.org/344384 | 17:37 |
| shardy | so if someone else can work through fixing that it'd be much appreciated | 17:37 |
| *** paramite|afk is now known as paramite | 17:37 | |
| sshnaidm | shardy, hmm, so I misunderstood you | 17:39 |
| hrybacki | weshay: I'm going to break https://trello.com/c/whHpxvkO/269-investigate-ci-automated-documentation down into sub-cards aligned with the plan of action in ^^ | 17:39 |
| weshay | +1 | 17:39 |
| sshnaidm | shardy, I was trying to find anybody who will handle this the whole day, it didn't work out | 17:39 |
| *** shivrao_ has joined #tripleo | 17:40 | |
| sshnaidm | shardy, weshay, trown|lunch do you know how can I find somebody to solve this issue with delorean? Running over IRC channels with searching who can take it has appeared as not working solution | 17:40 |
| *** shivrao has quit IRC | 17:41 | |
| *** shivrao_ is now known as shivrao | 17:41 | |
| sshnaidm | is there any interface in RDO that I could come with this issue? bz, launchpad, any other rdo specific issues tracker? | 17:42 |
| *** egafford has joined #tripleo | 17:42 | |
| *** penick is now known as jpenick | 17:44 | |
| *** yamahata has quit IRC | 17:45 | |
| openstackgerrit | Dan Prince proposed openstack/python-tripleoclient: Use the messaging-websocket endpoint https://review.openstack.org/344384 | 17:46 |
| openstackgerrit | Marius Cornea proposed openstack/tripleo-heat-templates: Fix gnocchi::storage::ceph::ceph_keyring https://review.openstack.org/344387 | 17:47 |
| *** shardy has quit IRC | 17:50 | |
| *** electrofelix has quit IRC | 17:50 | |
| *** mcornea has quit IRC | 17:51 | |
| EmilienM | trown|lunch: weird issue :( | 17:53 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment. https://review.openstack.org/341506 | 17:55 |
| *** tosky has quit IRC | 17:56 | |
| weshay | sshnaidm, my suggestion is to ping apevec in #rdo | 17:56 |
| sshnaidm | weshay, I do it from morning | 17:56 |
| weshay | sshnaidm, k.. you can also email him + ping in the morning.. thanks for going after it!! | 17:57 |
| *** abehl has quit IRC | 17:57 | |
| *** gfidente is now known as gfidente|afk | 17:57 | |
| *** dsneddon has joined #tripleo | 17:59 | |
| sshnaidm | weshay, priv | 17:59 |
| *** paramite is now known as paramite|afk | 17:59 | |
| *** trown|lunch is now known as trown | 18:01 | |
| trown | EmilienM: indeed, I just tested it with the tripleo-ci produced images and I get the same thing | 18:01 |
| *** ebalduf_ has quit IRC | 18:01 | |
| EmilienM | trown: same error? | 18:01 |
| trown | EmilienM: I am pretty sure that patch will break RDO, but I am not even confident it will work for tripleo | 18:02 |
| EmilienM | weird | 18:02 |
| trown | ya same error | 18:02 |
| trown | same behavior where there is a bunch of missing hiera | 18:03 |
| EmilienM | it's weird it works in ooo ci | 18:05 |
| trown | ya, that is a bit concerning in itself as tripleo-quickstart devmode is supposed to be equivalent | 18:06 |
| trown | I wonder if there is something changed in tripleo-ci recently that is missing from tripleo-quickstart | 18:07 |
| trown | the other possibility is that tripleo-ci is not actually testing that patch | 18:08 |
| ayoung | trown, EmilienM, for Federation, I need to pass in a mapping file that is a JSON blob. It seems wrong to embed this in the middle of a heat template. Is there a way that I can specify a separate Metadata URL for the mapping file? A simplistic one looks like this: https://github.com/admiyo/rippowam/blob/master/roles/packstack/templates/mapping_sssd.json.j2 | 18:09 |
| *** abehl has joined #tripleo | 18:09 | |
| trown | EmilienM: uh oh... I think it is that tripleo-ci is not actually testing THT | 18:11 |
| trown | EmilienM: check yum log http://logs.openstack.org/16/341616/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/994d0f9/logs/undercloud/var/log/yum.txt.gz | 18:12 |
| trown | the version of THT installed there is the one from 20160614213831 | 18:12 |
| trown | ie 5 days ago | 18:12 |
| EmilienM | trown: WHAT | 18:13 |
| EmilienM | again this bug? | 18:13 |
| EmilienM | ayoung: it could be a template file in puppet-tripleo | 18:15 |
| EmilienM | trown: we have a serious problem if you're right | 18:15 |
| trown | indeed | 18:15 |
| ayoung | EmilienM, ok, when this fire is out, I might bug you more about the path to get there...I'll keep writing with the assumption that will work | 18:16 |
| EmilienM | I don't find 333fd28 | 18:16 |
| EmilienM | ayoung: ok np, not now, later plz | 18:16 |
| EmilienM | Jul 19 11:26:19 Installed: openstack-tripleo-heat-templates-5.0.0-0.20160719112011.333fd28.el7.centos.noarch | 18:16 |
| ayoung | ++ | 18:16 |
| EmilienM | I don't find the commit | 18:16 |
| EmilienM | let's look another job | 18:17 |
| EmilienM | no it pick master | 18:18 |
| trown | hmm I must have been looking at the wrong line, because that is a different timestamp... still not the right commit | 18:18 |
| EmilienM | see a recent job : http://logs.openstack.org/88/344188/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/459ab86/logs/undercloud/var/log/host_info.txt.gz | 18:18 |
| EmilienM | openstack-tripleo-heat-templates-5.0.0-0.20160719163217.970fcfc.el7.centos.noarch | 18:18 |
| trown | oh I got tripped up by openstack-heat-templates... | 18:19 |
| EmilienM | and 970fcfcb58b2d0d26a39c2d9deab3a0301eb65ca is master | 18:19 |
| EmilienM | yes :) | 18:19 |
| EmilienM | dude you almost killed me | 18:19 |
| EmilienM | ayoung: ok so back to you | 18:19 |
| trown | still though where is 333fd28 coming from | 18:19 |
| EmilienM | is your file static / dynamic? | 18:19 |
| trown | we cherry-pick instead of checkout? | 18:19 |
| EmilienM | ayoung: is your file always the same during all deployments? Or not? Can it change after a deployment? | 18:20 |
| EmilienM | ayoung: I need to know those 2 things | 18:20 |
| ayoung | EmilienM, no | 18:20 |
| ayoung | EmilienM, the file will be very likely to change | 18:20 |
| EmilienM | after a deployment, right? | 18:21 |
| ayoung | EmilienM, yes | 18:21 |
| ayoung | EmilienM, like this | 18:21 |
| ayoung | start by creaing IdP | 18:21 |
| ayoung | this is a keystone call, will not change after the fact | 18:21 |
| ayoung | second create a mapping ,useing that file | 18:21 |
| ayoung | 3 create a protocol, using IdP and mapping | 18:21 |
| ayoung | now, as things come up, user will want to update the mapping file, and upload to Keystone | 18:22 |
| ayoung | Its a Keystone API call, and, since we have a shared database, does not necessarily need to be done in Heat | 18:22 |
| EmilienM | trown: can you try again by cherry-picking? | 18:22 |
| EmilienM | trown: because my patch is supposed to be rebased | 18:22 |
| ayoung | it might make sense to have all the rest of this done in Heat, and use a placeholder mapping file | 18:22 |
| EmilienM | trown: the git cherry-pick doesn't rebase for you | 18:23 |
| dprince | ayoung: heat supports get_file. The keystone services (in t-h-t) could have get_file suck in an example federation mapping.json file I think | 18:23 |
| ayoung | then let the sys admin modify it after the face | 18:23 |
| EmilienM | trown: so you need to 1) rebase your tht on master and then cherry-pick my patch OR checkout my patch | 18:23 |
| dprince | ayoung: yes, I think this would work quite nicely | 18:23 |
| ayoung | dprince, so get_file would fetch the placeholder from metadata? | 18:23 |
| dprince | ayoung: yes, the file that the get_file references needs to live in tree though | 18:24 |
| trown | EmilienM: right, I am looking at what we do in tripleo.sh and I am not understanding how we get a different git hash | 18:25 |
| ayoung | And then we would say that the expected end state of the heat Federation deplopy would be a URL that would always return a 401 due to the mapping. In order to get an unscoped token would require an updated mapping file via the Keystone API | 18:25 |
| trown | in my env I have http://chunk.io/f/23742e8cca104436be58e427a2d09b0d which matches the git hash on the patch | 18:26 |
| trown | but in CI we are getting 333fd28 | 18:26 |
| EmilienM | mhh | 18:28 |
| trown | in any case testing that patch rebased... didn't know CI did a rebase before building | 18:31 |
| apetrich | EmilienM, ok. will do. but I'm not sure that mitaka -> Newton Overcloud upgrade is working | 18:37 |
| *** ayoung has quit IRC | 18:39 | |
| *** yamahata has joined #tripleo | 18:39 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Remove ::nova::cron::archive_deleted_rows https://review.openstack.org/343661 | 18:45 |
| EmilienM | trown: ok let me know! | 18:45 |
| EmilienM | trown: I'm glad there is no dlrn bug again :) | 18:46 |
| trown | EmilienM: hmm, maybe it is only broken with master heat | 18:50 |
| trown | EmilienM: I have newer heat in my env than in CI | 18:50 |
| trown | which is a bug in quickstart dev mode... it should have the same packages as CI | 18:51 |
| *** ebalduf_ has joined #tripleo | 18:57 | |
| *** egafford has quit IRC | 18:59 | |
| *** egafford has joined #tripleo | 19:00 | |
| *** abehl has quit IRC | 19:01 | |
| *** julim has quit IRC | 19:02 | |
| slagle | did something recently change in how we configure galera.cnf? i'm seeing it has bind-address=0.0.0.0 | 19:07 |
| slagle | is that new? | 19:07 |
| slagle | nvm, red herring | 19:08 |
| *** r-mibu has quit IRC | 19:13 | |
| *** r-mibu has joined #tripleo | 19:13 | |
| *** abehl has joined #tripleo | 19:14 | |
| *** jcoufal_ has quit IRC | 19:22 | |
| EmilienM | slagle: yes, I moved it to a composable role | 19:22 |
| EmilienM | but nothing should have changed in the config except if I did something wrong | 19:22 |
| *** florianf has quit IRC | 19:25 | |
| slagle | EmilienM: yea i think something broke my multinode CI jobs | 19:25 |
| slagle | puppet used /etc/my.cnf.d/server.cnf as the config file, where it has bind-address = 127.0.0.1 | 19:26 |
| EmilienM | it's a non ha job, right? | 19:26 |
| slagle | yes | 19:26 |
| EmilienM | config should be https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/database/mysql.pp#L67-L70 | 19:27 |
| slagle | but even on nonha jobs i thought we always configured via galera.cnf | 19:27 |
| EmilienM | is enable_galera = true? | 19:27 |
| slagle | it should be, i don't change it | 19:28 |
| *** ebalduf_ has quit IRC | 19:29 | |
| *** jubapa has quit IRC | 19:30 | |
| *** akrivoka has quit IRC | 19:36 | |
| EmilienM | slagle: I'm looking logs | 19:39 |
| EmilienM | slagle: ah, we're missing lot of logs http://logs.openstack.org/77/324777/84/experimental/gate-tripleo-ci-centos-7-nonha-multinode/fcc869d/logs/etc/ | 19:40 |
| EmilienM | should I dl primary_node.tar.xz ? | 19:40 |
| EmilienM | ok I got it | 19:41 |
| slagle | yes, all the normal tripleo logs should be there | 19:41 |
| slagle | EmilienM: i just kicked off a local run of this as well | 19:41 |
| slagle | EmilienM: don't let this distract you if you've got soemthign else to do :) | 19:42 |
| EmilienM | slagle: I can't continue by knowing I might have broken something | 19:42 |
| EmilienM | let me dig a bit | 19:42 |
| EmilienM | damn you're right | 19:44 |
| EmilienM | I checked a nonha job and we have /etc/my.cnf.d/galera.cnf | 19:44 |
| EmilienM | but not in your multinode job | 19:44 |
| slagle | the overcloud node logs are in http://logs.openstack.org/77/324777/84/experimental/gate-tripleo-ci-centos-7-nonha-multinode/fcc869d/logs/subnode-2/subnode-2.tar.xz | 19:44 |
| EmilienM | slagle: not subnode1? | 19:45 |
| slagle | there is a /etc/my.cnf.d/galera.cnf there, it's just not managed by puppet | 19:45 |
| slagle | no, they start numbering at 1 | 19:45 |
| slagle | err, 2 | 19:45 |
| slagle | i dunno why | 19:45 |
| EmilienM | k, a sec | 19:45 |
| EmilienM | slagle: subnode1 and subnode2 are 2 controllers? they both have mysql configured | 19:46 |
| EmilienM | its hyperconverged thing? | 19:47 |
| slagle | there's only primary_node and subnode-2 | 19:47 |
| slagle | primary_node is undercloud | 19:47 |
| EmilienM | ah ok | 19:48 |
| slagle | subnode-2 is aio overcloud | 19:48 |
| EmilienM | kk | 19:48 |
| slagle | it does use pacemaker | 19:48 |
| slagle | even though it is nonha | 19:48 |
| EmilienM | I see enable_galera to true | 19:48 |
| openstackgerrit | Merged openstack/instack-undercloud: Fix ceilometer auth url on undercloud https://review.openstack.org/344279 | 19:48 |
| EmilienM | let me check puppet logs | 19:48 |
| EmilienM | oh it use pacemaker | 19:48 |
| slagle | yea | 19:50 |
| EmilienM | I don't see tripleo::profile::base::database::mysql resource in the puppet catalog | 19:51 |
| EmilienM | slagle: it fails early right? I don't see any pacemaker logs in /var/log/message | 19:53 |
| slagle | i believe it failed during ControllerOvercloudServicesDeployment_Step3 | 19:54 |
| slagle | yea | 19:55 |
| slagle | it failed there | 19:55 |
| EmilienM | it's like puppet code to deploy mysql on ha is not deployed in your env | 19:55 |
| openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates: Add environment to preselect only VIP IP addresses https://review.openstack.org/336749 | 19:56 |
| slagle | EmilienM: should there be an include in puppet-tripleo/manifests/profile/pacemaker/database/mysql.pp to include the base profile? | 19:57 |
| EmilienM | no | 19:57 |
| EmilienM | it's in the tht service | 19:57 |
| slagle | ok | 19:57 |
| EmilienM | slagle: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/pacemaker/database/mysql.yaml#L20 | 19:58 |
| EmilienM | I'm investigating your packages | 19:58 |
| EmilienM | you're running on top of https://github.com/openstack/puppet-tripleo/commit/b613e77 | 19:58 |
| EmilienM | which is good | 19:58 |
| EmilienM | let's check tht | 19:58 |
| EmilienM | current master , mhh | 19:59 |
| *** akshai has quit IRC | 20:01 | |
| EmilienM | slagle: found it | 20:02 |
| *** julim has joined #tripleo | 20:02 | |
| EmilienM | slagle: https://review.openstack.org/#/c/324777/84/test-environments/multinode.yaml | 20:02 |
| EmilienM | we need to add MySQL | 20:02 |
| slagle | oh snap | 20:03 |
| EmilienM | dude you scared me | 20:03 |
| slagle | sorry man | 20:03 |
| slagle | i bet that tatoo jumped right off your skin! | 20:03 |
| EmilienM | I don't have it yet | 20:03 |
| slagle | everytime we add a new service, i will have to remember to update this list | 20:03 |
| EmilienM | to be honest I didn't go yet, I was out all recent week ends | 20:04 |
| slagle | i know, i saw on twitter | 20:04 |
| *** radek2 has quit IRC | 20:04 | |
| EmilienM | slagle: we need to merge your patch next time it's passing | 20:04 |
| EmilienM | slagle: FYI the next services are Aodh; when they'll merge I'll let you know. Nothing else AFIK. Everything else is done | 20:05 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci https://review.openstack.org/324777 | 20:05 |
| slagle | cool, thx | 20:05 |
| *** rook has quit IRC | 20:07 | |
| EmilienM | trown: do you have more success? | 20:10 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Add 'service_name' to composable services https://review.openstack.org/344440 | 20:10 |
| trown | EmilienM: nope, I even made a local fix for quickstart dev mode so that I have the same packages as in CI | 20:11 |
| trown | and I still get the missing hiera issue with that patch | 20:11 |
| *** shivrao has quit IRC | 20:11 | |
| dprince | EmilienM: ^^ that is the 'service_name' patch that will enable the firewall rule composability | 20:11 |
| dprince | EmilienM: a few more patches and I will have the example in place | 20:12 |
| EmilienM | cool | 20:15 |
| *** dprince has quit IRC | 20:16 | |
| *** ebalduf has quit IRC | 20:16 | |
| trown | EmilienM: I am sure that patch will break RDO at the very least, so if you don't mind it would be nice to get a promote there before merging if possible | 20:18 |
| EmilienM | sure thing | 20:18 |
| trown | EmilienM: we are 12 days behind as is, so merging that will be a major set back | 20:18 |
| EmilienM | ok | 20:19 |
| trown | it is pretty confusing why tripleo-ci does not have an issue though | 20:19 |
| *** paramite|afk is now known as paramite | 20:24 | |
| *** rbrady has quit IRC | 20:28 | |
| slagle | i just saw a gnocchi db-sync failure in tripleo-ci | 20:29 |
| *** shivrao has joined #tripleo | 20:29 | |
| slagle | http://logs.openstack.org/77/324777/84/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/35445f9/ | 20:30 |
| *** abehl has quit IRC | 20:34 | |
| openstackgerrit | Sumit Kumar proposed openstack/tripleo-heat-templates: Update NetApp configuration parameters. https://review.openstack.org/341778 | 20:34 |
| *** athomas has quit IRC | 20:35 | |
| *** jprovazn has quit IRC | 20:36 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs https://review.openstack.org/343017 | 20:37 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Pull all puppet modules from local sources https://review.openstack.org/344454 | 20:37 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Add ovb to mirrored git repo list https://review.openstack.org/344455 | 20:37 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs https://review.openstack.org/343017 | 20:39 |
| *** zeroshft has joined #tripleo | 20:42 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs https://review.openstack.org/343017 | 20:42 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Pull all puppet modules from local sources https://review.openstack.org/344454 | 20:42 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Add ovb to mirrored git repo list https://review.openstack.org/344455 | 20:42 |
| *** dtrainor has quit IRC | 20:46 | |
| *** abehl has joined #tripleo | 20:47 | |
| *** egafford has quit IRC | 20:50 | |
| *** rcernin has joined #tripleo | 20:55 | |
| *** jayg is now known as jayg|g0n3 | 20:55 | |
| *** fultonj has quit IRC | 20:59 | |
| *** dtrainor has joined #tripleo | 21:00 | |
| *** trown is now known as trown|outtypewww | 21:01 | |
| *** paramite has quit IRC | 21:02 | |
| *** isq_ has quit IRC | 21:04 | |
| EmilienM | slagle: oops | 21:10 |
| EmilienM | https://review.openstack.org/#/c/324777/85/test-environments/multinode.yaml | 21:10 |
| EmilienM | we're missing Pacemaker I think | 21:10 |
| EmilienM | let me check | 21:10 |
| EmilienM | yeah we need OS::TripleO::Services::Pacemaker | 21:11 |
| slagle | ok i'll add it | 21:11 |
| slagle | i'll just resync the full list | 21:12 |
| EmilienM | slagle: what about OVERCLOUD_DEPLOY_ARGS ? | 21:12 |
| EmilienM | don't you load puppet-pacemaker.yaml env? | 21:12 |
| EmilienM | slagle: you're missing Kernel role also | 21:13 |
| EmilienM | Kernel is required for Swift | 21:14 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-docs: Update overcloud SSL docs https://review.openstack.org/344475 | 21:14 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci https://review.openstack.org/324777 | 21:15 |
| *** isq has joined #tripleo | 21:15 | |
| slagle | EmilienM: it should have everything now, i pulled the full list over | 21:16 |
| EmilienM | err, one more, you'll kill me | 21:16 |
| *** jeckersb is now known as jeckersb_gone | 21:17 | |
| EmilienM | ceilocompute agent | 21:17 |
| EmilienM | a sec, what is name again | 21:17 |
| slagle | i'll get it | 21:17 |
| *** lblanchard has quit IRC | 21:17 | |
| EmilienM | ComputeCeilometerAgent | 21:17 |
| EmilienM | mhh and I wonder if we need or not ComputeNeutronOvsAgent | 21:18 |
| EmilienM | dprince created it but I don't like the name | 21:18 |
| slagle | EmilienM: I guess I need the full list from compute too | 21:18 |
| EmilienM | it's very tight to compute role | 21:18 |
| slagle | i need the union of both lists | 21:18 |
| EmilienM | I'm not sure it will work | 21:18 |
| EmilienM | ComputeNeutronCorePlugin and NeutronCorePlugin | 21:18 |
| EmilienM | dprince did that ^ | 21:18 |
| EmilienM | let me check if it will collide | 21:19 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci https://review.openstack.org/324777 | 21:19 |
| slagle | ok, just let me know | 21:19 |
| EmilienM | ok you don't need ComputeNeutronOvsAgent and ComputeNeutronCorePlugin | 21:20 |
| EmilienM | they are specific for vendors like plumgrid, etc | 21:20 |
| EmilienM | so we're good! | 21:20 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci https://review.openstack.org/324777 | 21:20 |
| EmilienM | slagle: /me afk running now, let me know if any blocker | 21:21 |
| *** isq has quit IRC | 21:32 | |
| *** isq has joined #tripleo | 21:36 | |
| *** abehl has quit IRC | 21:42 | |
| *** dtrainor has quit IRC | 21:43 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: Do not exit prior to the inventory of the deployment. https://review.openstack.org/341506 | 21:45 |
| *** weshay has quit IRC | 21:47 | |
| openstackgerrit | Merged openstack/diskimage-builder: Remove the escape in the centos7 README file https://review.openstack.org/343997 | 21:47 |
| *** ayoung has joined #tripleo | 21:52 | |
| *** zeroshft has quit IRC | 22:04 | |
| *** [1]cdearborn has quit IRC | 22:14 | |
| *** pradk has quit IRC | 22:16 | |
| *** shivrao has quit IRC | 22:23 | |
| *** bfournie has quit IRC | 22:30 | |
| *** jcoufal has joined #tripleo | 22:32 | |
| *** rlandy has quit IRC | 23:08 | |
| *** ayoung has quit IRC | 23:11 | |
| *** yamahata has quit IRC | 23:14 | |
| *** yamahata has joined #tripleo | 23:24 | |
| *** ramishra has quit IRC | 23:26 | |
| *** ramishra has joined #tripleo | 23:27 | |
| *** shivrao has joined #tripleo | 23:32 | |
| *** rhallisey has quit IRC | 23:43 | |
| slagle | EmilienM: seeing a lot of CI jobs failing with: Error: /Stage[main]/Gnocchi::Db::Sync/Exec[gnocchi-db-sync]: gnocchi-upgrade --config-file /etc/gnocchi/gnocchi.conf --skip-storage --create-legacy-resource-types returned 1 instead of one of [0]\u001b[0m\n", | 23:44 |
| slagle | related to https://review.openstack.org/#/c/343902/ i guess? | 23:44 |
| EmilienM | excactly | 23:45 |
| EmilienM | but why do you have this bug? | 23:45 |
| EmilienM | the patch is merged and should be in delorean | 23:45 |
| slagle | yea the patch is definnitely there, b/c it's running the command with --create-legacy-resource-types | 23:45 |
| EmilienM | honestly I'm not expert enough in Gnocchi but there is definitly something wrong here | 23:46 |
| slagle | i'm getting a bug filed | 23:46 |
| EmilienM | if the error still occurs, I would suggest to disable gnocchi in the meantime it's get stable | 23:46 |
| *** lblanchard has joined #tripleo | 23:46 | |
| slagle | https://bugs.launchpad.net/tripleo/+bug/1604624 | 23:48 |
| openstack | Launchpad bug 1604624 in tripleo "CI: Error: /Stage[main]/Gnocchi::Db::Sync/Exec[gnocchi-db-sync]: gnocchi-upgrade --config-file /etc/gnocchi/gnocchi.conf --skip-storage --create-legacy-resource-types returned 1 instead of one of [0]\u001b[0m\n", " [Critical,Triaged] | 23:48 |
| slagle | i'll email prad, maybe he can take a look when gets in | 23:48 |
| EmilienM | ok | 23:49 |
| slagle | there's nothing in the gnocchi log | 23:50 |
| *** penick has joined #tripleo | 23:50 | |
| *** jcoufal has quit IRC | 23:51 | |
| *** akshai has joined #tripleo | 23:51 | |
| *** penick has quit IRC | 23:52 | |
| *** jpenick has quit IRC | 23:52 | |
| EmilienM | slagle: in gnocchi-manage logs either? | 23:52 |
| slagle | nothing | 23:52 |
| *** dmacpher has quit IRC | 23:53 | |
| EmilienM | something I noticed is that tripleo does not show logoutput on failure for exec | 23:53 |
| EmilienM | if you see what I mean | 23:53 |
| EmilienM | I'll investigate that tomorrow, it would be so helpful to have Exec output in the logs | 23:53 |
| EmilienM | we could see TRACE etc | 23:53 |
| slagle | that would be helpful | 23:53 |
| *** gfidente|afk has quit IRC | 23:54 | |
| EmilienM | slagle: ok i'm out a bit, ttyl | 23:55 |
| slagle | bye | 23:55 |
| *** akshai has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!