| *** kiennt has joined #heat | 00:53 | |
| *** dimtruck is now known as zz_dimtruck | 00:54 | |
| *** EricGonczer_ has joined #heat | 01:11 | |
| *** masber has joined #heat | 01:14 | |
| *** zz_dimtruck is now known as dimtruck | 01:28 | |
| *** Shunli has joined #heat | 01:42 | |
| *** zsli_ has joined #heat | 01:44 | |
| *** zsli__ has joined #heat | 01:46 | |
| *** Shunli has quit IRC | 01:47 | |
| *** zsli_ has quit IRC | 01:49 | |
| *** ricolin has joined #heat | 01:52 | |
| *** EricGonc_ has joined #heat | 02:29 | |
| *** EricGonczer_ has quit IRC | 02:30 | |
| *** EricGonc_ has quit IRC | 02:30 | |
| *** namnh has joined #heat | 02:37 | |
| *** dixiaoli has joined #heat | 02:46 | |
| *** dixiaoli has quit IRC | 02:57 | |
| openstackgerrit | Merged openstack/heat master: Add an example on how to use 'contains' as condition https://review.openstack.org/476798 | 03:59 |
|---|---|---|
| *** skramaja has joined #heat | 04:23 | |
| *** yuanying has quit IRC | 04:24 | |
| *** yuanying has joined #heat | 04:24 | |
| ramishra | seems gate broken with devstack change, https://review.openstack.org/#/c/479108/ already in the queue to fix it. | 04:50 |
| *** prateek has joined #heat | 05:13 | |
| *** dimtruck is now known as zz_dimtruck | 05:23 | |
| *** zz_dimtruck is now known as dimtruck | 05:23 | |
| *** jnettlet has quit IRC | 05:27 | |
| *** jnettlet has joined #heat | 05:30 | |
| *** yuanying_ has joined #heat | 05:32 | |
| *** yuanying has quit IRC | 05:34 | |
| *** kiennt has quit IRC | 05:41 | |
| *** rcernin has joined #heat | 05:45 | |
| *** kiennt has joined #heat | 05:48 | |
| *** armaan has joined #heat | 05:59 | |
| *** armaan_ has joined #heat | 06:02 | |
| *** kiennt has quit IRC | 06:02 | |
| *** armaan has quit IRC | 06:06 | |
| *** links has quit IRC | 06:07 | |
| *** kiennt has joined #heat | 06:14 | |
| *** jprovazn has joined #heat | 06:21 | |
| *** links has joined #heat | 06:22 | |
| *** ccamacho has quit IRC | 06:38 | |
| *** ccamacho has joined #heat | 06:38 | |
| *** flg has joined #heat | 07:04 | |
| *** shardy has joined #heat | 07:05 | |
| *** ioggstream has joined #heat | 07:16 | |
| *** f13o has joined #heat | 07:20 | |
| *** shardy is now known as shardy_afk | 07:24 | |
| *** ccard has quit IRC | 07:38 | |
| openstackgerrit | Rabi Mishra proposed openstack/heat master: DNM Test cloud-init issue https://review.openstack.org/479199 | 07:44 |
| *** f13o has quit IRC | 07:49 | |
| *** boris-42__ has joined #heat | 07:55 | |
| *** armaan_ has quit IRC | 07:55 | |
| *** armaan has joined #heat | 07:56 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #heat | 08:02 | |
| strigazi | ramishra ricolin Have a look again at https://review.openstack.org/#/c/476415 Do you want me to change anything? | 08:04 |
| *** f13o has joined #heat | 08:11 | |
| *** shardy_afk is now known as shardy | 08:12 | |
| *** ccamacho has quit IRC | 08:15 | |
| *** ccamacho has joined #heat | 08:15 | |
| *** gfidente has joined #heat | 08:22 | |
| *** armaan_ has joined #heat | 08:28 | |
| *** armaan has quit IRC | 08:28 | |
| therve | ramishra, What's the status? | 08:49 |
| therve | https://review.openstack.org/479209 should be the one? | 08:50 |
| ramishra | therve: yep, I think so | 08:50 |
| *** toMeloos has joined #heat | 08:52 | |
| *** eliqiao has quit IRC | 09:06 | |
| toMeloos | Hi guys | 09:12 |
| toMeloos | We upgraded Neutron LBaaS v1 to v2 but now we are stuck with the heat stacks that use the lbaasv1 resources. Both delete and abandon result in errors like "ERROR: HEAT-E99001 Service neutron is not available for resource type OS::Neutron::PoolMember, reason: Service endpoint not in service catalog.". Any suggestions on how to force remove them? | 09:12 |
| andreykurilin | hi folks | 09:14 |
| therve | toMeloos, Delete from heat database | 09:25 |
| toMeloos | Aha. I already removed all the neutron/nova/etc. stuff manually but diving into the database is really the only option? | 09:26 |
| toMeloos | Feels like this calls for a --force flag for "openstack stack abandon" then | 09:27 |
| therve | Yeah that might work | 09:28 |
| *** zsli__ has quit IRC | 09:30 | |
| toMeloos | Upgrades seems like a major issue here. LBaaS v1->v2 removed resource types, so will FWaaS v1->v2 probably as well as other projects. Any way this can be supported in a way that upgrading those services don't break running heat stacks that used these services? | 09:32 |
| therve | You may be able to have both versions | 09:34 |
| therve | It's something to manage as an operator. Neutron left people behind a bit on that front IMO | 09:34 |
| toMeloos | You can dispense with the "a bit" part ;) | 09:35 |
| therve | As much as I am sympathetic with that situation, we (Heat) don't have the manpower to handle that issue | 09:37 |
| toMeloos | I would expect (unlike with LBaaS) a decent upgrade path that migrates existing resources to the new version. Calling on all your users to perform the upgrade by tearing down and rebuilding their services is not very customer-friendly. | 09:39 |
| therve | toMeloos, You would expect that from Heat? | 09:41 |
| toMeloos | Defintely not | 09:41 |
| therve | Ah ok :) | 09:41 |
| toMeloos | But the heat problem actually manifests if there is an automated migration script, because the resources referred to in running stacks no longer exist | 09:41 |
| therve | True. Then maybe using some kind of abandon/adopt you may recover | 09:42 |
| therve | Or if you keep both versions running, a stack update | 09:42 |
| toMeloos | For example, if there would have been an LBaaS v1->v2 upgrade path with migration script for operators to make this a smooth upgrade, users having active stacks with heat lbaasv1 resource types would still find themselves unable to remove their stacks.... | 09:43 |
| therve | Well there is https://github.com/openstack/neutron-lbaas/blob/master/tools/database-migration-from-v1-to-v2.py | 09:43 |
| toMeloos | Tried it, didn't work ;) | 09:44 |
| *** links has quit IRC | 09:44 | |
| toMeloos | Yes well option 1 won't work because abandon also doesn't work if resource types have been removed :( | 09:44 |
| toMeloos | Which leaves option 2 which requires A) old and new version of services being able to run in parallel and B) customers to run "stack update" for all their stacks | 09:46 |
| toMeloos | So in order to not break running heat stacks we need all openstack services to provide support for parallel versions and customers to handle the upgrade (no automated upgrading by the operator). Not ideal.... | 09:49 |
| *** askb has quit IRC | 09:51 | |
| therve | Well yeah | 09:51 |
| therve | We could support deleting resources when the service is not available | 09:51 |
| *** armaan_ has quit IRC | 09:52 | |
| therve | But that's as far as we could got I think | 09:52 |
| toMeloos | That would definitely be a big step. Either delete, or at least abandon when the service is not available | 09:55 |
| *** yassine has joined #heat | 09:57 | |
| toMeloos | Ideally there would be some translation system, so the migration script could log that old-resource-type:old-uuid has been replaced by new-resource-type:new-uuid. So that (in this example) when you delete the old stack with an lbaasv1 resource, you actually remove the lbaasv2 resource it was migrated to. | 09:59 |
| *** links has joined #heat | 10:02 | |
| *** jprovazn has quit IRC | 10:08 | |
| *** sai has quit IRC | 10:13 | |
| *** sai has joined #heat | 10:13 | |
| *** boris-42__ has quit IRC | 10:13 | |
| *** jaosorior has quit IRC | 10:28 | |
| *** armaan has joined #heat | 10:31 | |
| *** namnh has quit IRC | 10:32 | |
| *** kiennt has quit IRC | 10:33 | |
| *** armaan has quit IRC | 10:39 | |
| *** armaan has joined #heat | 10:44 | |
| *** gkadam has joined #heat | 10:50 | |
| *** armaan has quit IRC | 10:51 | |
| openstackgerrit | Van Hung Pham proposed openstack/python-heatclient master: Switch from oslosphinx to openstackdocstheme https://review.openstack.org/478410 | 10:56 |
| openstackgerrit | kavitha h r proposed openstack/python-heatclient master: Rename exc.py to exception.py https://review.openstack.org/479261 | 11:15 |
| *** armaan has joined #heat | 11:16 | |
| *** ioggstream has quit IRC | 11:18 | |
| *** gmoro has joined #heat | 11:19 | |
| *** armaan has quit IRC | 11:33 | |
| openstackgerrit | kavitha h r proposed openstack/python-heatclient master: Rename exc.py to exception.py https://review.openstack.org/479261 | 11:42 |
| *** ccamacho is now known as ccamacho|lunch | 11:46 | |
| *** gkadam has quit IRC | 11:49 | |
| *** askb has joined #heat | 11:55 | |
| *** askb has quit IRC | 11:56 | |
| andreykurilin | hi folks! we have too much `HTTPInternalServerError: ERROR:` from heat for the last several days (we cannot merge anything today). Can anyone help me with investigation? | 11:57 |
| *** ioggstream has joined #heat | 11:58 | |
| therve | andreykurilin, Sure | 11:59 |
| andreykurilin | therve: thanks. so we have a rally job which makes some actions with heat. the error message doesn't include at all) http://logs.openstack.org/04/479204/2/gate/gate-rally-dsvm-rally-heat/24de955/console.html#_2017-06-30_11_20_45_845403 | 12:00 |
| andreykurilin | it is an example of failure while obtaining heat stask | 12:00 |
| andreykurilin | and I see nothing useful here - http://logs.openstack.org/04/479204/2/gate/gate-rally-dsvm-rally-heat/24de955/logs/screen-h-eng.txt.gz? | 12:02 |
| andreykurilin | oh | 12:03 |
| andreykurilin | therve: http://logs.openstack.org/04/479204/2/gate/gate-rally-dsvm-rally-heat/24de955/logs/apache/heat_api.txt.gz#_2017-06-30_11_20_30_289655 | 12:03 |
| therve | andreykurilin, https://review.openstack.org/#/c/422983/ probably then. | 12:05 |
| andreykurilin | I can try make a revert and launch depends-on change | 12:05 |
| andreykurilin | should I? | 12:05 |
| therve | Yeah that's worth trying, though I'm not sure it will work | 12:08 |
| therve | andreykurilin, Do you know what those tests are doing? | 12:09 |
| andreykurilin | therve: you need the full actions or just failed actions? | 12:10 |
| therve | andreykurilin, Whatever that would help reproducing :) | 12:10 |
| andreykurilin | give me a moment | 12:10 |
| *** EricGonczer_ has joined #heat | 12:12 | |
| andreykurilin | therve: tha last several traces are about https://github.com/openstack/rally/blob/master/rally/plugins/openstack/scenarios/heat/utils.py#L43-L67 where template is https://github.com/openstack/rally/blob/master/rally/plugins/openstack/context/heat/stacks.py#L64-L73 | 12:15 |
| andreykurilin | it fails while obtaining created stack | 12:15 |
| andreykurilin | therve: from the logs I can see that previously 3 stacks were created successfuly | 12:16 |
| therve | andreykurilin, How often does it happen? | 12:16 |
| andreykurilin | in a loop | 12:17 |
| therve | I mean the failure | 12:17 |
| therve | We have the same thing in our tests, but we don't resolve outputs, so we don't get there | 12:18 |
| *** skramaja has quit IRC | 12:18 | |
| openstackgerrit | Thomas Herve proposed openstack/heat master: DNM resolve outputs to check test failure https://review.openstack.org/479279 | 12:19 |
| therve | andreykurilin, You can add resolve_outputs=False to work around it for now | 12:19 |
| andreykurilin | the failures are quite often. as I said, we cannot merge anything today | 12:20 |
| openstackgerrit | kavitha h r proposed openstack/python-heatclient master: Rename exc.py to exception.py https://review.openstack.org/479261 | 12:21 |
| *** f13o has quit IRC | 12:26 | |
| andreykurilin | therve: should I add it to get method? | 12:32 |
| *** ltomasbo has joined #heat | 12:33 | |
| therve | andreykurilin, Yep | 12:37 |
| *** p01nt3r75 has joined #heat | 12:40 | |
| *** dprince has joined #heat | 13:02 | |
| *** dimtruck is now known as zz_dimtruck | 13:09 | |
| *** links has quit IRC | 13:20 | |
| *** ccamacho|lunch is now known as ccamacho | 13:20 | |
| *** zaneb has joined #heat | 13:22 | |
| *** gkadam has joined #heat | 13:31 | |
| *** jcoufal has joined #heat | 13:33 | |
| *** felipemonteiro has joined #heat | 13:37 | |
| *** felipemonteiro_ has joined #heat | 13:41 | |
| *** jmlowe has quit IRC | 13:41 | |
| *** felipemonteiro has quit IRC | 13:44 | |
| *** annegentle has joined #heat | 13:58 | |
| openstackgerrit | Thomas Herve proposed openstack/heat master: DNM resolve outputs to check test failure https://review.openstack.org/479279 | 13:59 |
| therve | zaneb, ^^ looks like there is a bug in with attribute storing patch | 13:59 |
| zaneb | therve: I noticed that yesterday :) https://review.openstack.org/#/c/479074/ | 14:01 |
| therve | Oh sweet | 14:01 |
| zaneb | in related news, is the gate broken? | 14:01 |
| therve | Yeah | 14:01 |
| zaneb | because there's no way that can have broken the legacy job | 14:01 |
| *** jmlowe has joined #heat | 14:02 | |
| zaneb | therve: we should probably merge your patch to fix it now. mine is at the end of a long series | 14:02 |
| therve | zaneb, https://review.openstack.org/#/c/479209 | 14:03 |
| therve | Yeah I was going to say | 14:04 |
| therve | Though it will make your series conflict again I suspect | 14:04 |
| zaneb | it'll mean yet another rebase :/ | 14:04 |
| therve | I'll start by filling a bug | 14:04 |
| *** annegentle has quit IRC | 14:07 | |
| therve | zaneb, tripleo is getting another failure with conditions during upgrades, I wonder if you have an idea | 14:08 |
| therve | zaneb, It happens when depending a on resource conditionally created, presumably after a first failure | 14:08 |
| therve | I'm thinking it's another backup stack related issue, but I don't manage to reproduce it | 14:08 |
| zaneb | did the patch for the other thing merge yet? | 14:09 |
| zaneb | ah, no it didn't https://review.openstack.org/#/c/476622/ | 14:11 |
| *** annegentle has joined #heat | 14:11 | |
| * zaneb has too many patches | 14:12 | |
| zaneb | there's a good chance that might resolve this one too | 14:13 |
| therve | Hum it didn't merge indeed | 14:14 |
| therve | But I'm not sure it's the same | 14:14 |
| therve | It's not about conditions not being found, it's the resource itself | 14:14 |
| zaneb | good point, I guess that one only affects the 'if' function | 14:16 |
| *** zz_dimtruck is now known as dimtruck | 14:20 | |
| zaneb | once we create a resource, we should copy it into the current template http://git.openstack.org/cgit/openstack/heat/tree/heat/engine/update.py#n124 | 14:21 |
| zaneb | but maybe if we don't have the new conditions because of a failure it will appear not to exist? | 14:22 |
| zaneb | therve: do you have a link to the actual failure? | 14:23 |
| therve | zaneb, No unfortunately | 14:29 |
| therve | zaneb, https://bugs.launchpad.net/tripleo/+bug/1689553 is an example I think | 14:29 |
| openstack | Launchpad bug 1689553 in tripleo "The specified reference "ControllerUpgradeBatch_Step2" (in ControllerUpgradeConfig_Step0) is incorrect." [High,Fix released] - Assigned to Jiří Stránský (jistr) | 14:29 |
| therve | https://review.openstack.org/#/c/479016/ is the other one I got | 14:30 |
| *** jtomasek has quit IRC | 14:37 | |
| zaneb | why does step 0 depend on step 2??? | 14:41 |
| zaneb | oh, they're steps of different things | 14:41 |
| zaneb | therve: ah, I think I might know what's happening | 14:59 |
| zaneb | therve: so your fix https://review.openstack.org/#/c/419588/ works when all the resources are in the template but some are disabled | 14:59 |
| zaneb | therve: but when we copy resources over as they are created, we only copy the ones that are enabled | 15:00 |
| zaneb | so you can end up with a new resource in the template that depends on a non-existent resource that isn't even in the template | 15:00 |
| *** EricGonc_ has joined #heat | 15:09 | |
| *** EricGonczer_ has quit IRC | 15:11 | |
| *** rcernin has quit IRC | 15:11 | |
| *** prateek has quit IRC | 15:18 | |
| *** FL1SK has quit IRC | 15:19 | |
| *** ramishra has quit IRC | 15:25 | |
| *** dimtruck is now known as zz_dimtruck | 15:35 | |
| *** toMeloos has left #heat | 15:37 | |
| *** spotz_ is now known as spotz | 15:42 | |
| *** mdavidson has quit IRC | 16:03 | |
| *** zz_dimtruck is now known as dimtruck | 16:05 | |
| *** dimtruck is now known as zz_dimtruck | 16:08 | |
| *** p01nt3r75 has quit IRC | 16:09 | |
| *** jistr is now known as jistr|off | 16:21 | |
| openstackgerrit | Zane Bitter proposed openstack/heat master: Strip disabled resources from depends_on in add_resource() https://review.openstack.org/479367 | 16:27 |
| zaneb | therve: ^ I raised https://bugs.launchpad.net/heat/+bug/1701677 | 16:28 |
| openstack | Launchpad bug 1701677 in heat "Dependencies on conditionally disabled resources cause problems in legacy updates" [High,In progress] - Assigned to Zane Bitter (zaneb) | 16:28 |
| *** gfidente has quit IRC | 16:30 | |
| *** zz_dimtruck is now known as dimtruck | 16:33 | |
| *** shardy has quit IRC | 16:37 | |
| *** dimtruck is now known as zz_dimtruck | 16:38 | |
| *** ioggstream has quit IRC | 16:49 | |
| *** gkadam has quit IRC | 16:57 | |
| *** zaneb has quit IRC | 16:58 | |
| *** zaneb has joined #heat | 17:09 | |
| anupn | Hi All, I am trying to create a stack with simple template as a start, here is my template http://paste.openstack.org/show/614209/. But i am seeing in h-eng logs, "baremetal" flavor could not be found http://paste.openstack.org/show/614208/. Any idea why is this? I am seeing baremetal flavor on running flavor list | 17:22 |
| anupn | and after unable to find this flavor it is trying to create my devstack baremetal node which is failing due to some other reason | 17:23 |
| *** armaan has joined #heat | 17:33 | |
| *** zaneb has quit IRC | 17:35 | |
| *** armaan_ has joined #heat | 17:36 | |
| *** armaan has quit IRC | 17:38 | |
| *** zaneb has joined #heat | 17:48 | |
| *** hongbin has joined #heat | 18:05 | |
| *** zaneb has quit IRC | 18:20 | |
| *** EricGonc_ has quit IRC | 18:28 | |
| *** EricGonczer_ has joined #heat | 18:33 | |
| *** zaneb has joined #heat | 18:38 | |
| therve | zaneb, Thanks for looking! I think your diagnosis is good, though I don't really understand the patch yet | 19:18 |
| *** armaan has joined #heat | 19:19 | |
| zaneb | therve: we use add_resource() to add the definitions to the existing template. so the patch is just to skip any depends_on that depend on resources not actually in the template | 19:20 |
| therve | zaneb, Does it catch when you just pass random dependency? | 19:20 |
| zaneb | it strips those too. but by the time you get to the point of actually updating resources, you'll already have validated that there is nothing completely random in there | 19:21 |
| therve | OK | 19:21 |
| therve | zaneb, shardy suggested something in the other bug | 19:22 |
| *** Tyrantelf has joined #heat | 19:22 | |
| *** Tyrantelf_ has quit IRC | 19:22 | |
| therve | zaneb, Which is aliasing the resource to OS::Heat::None when disabled | 19:22 |
| therve | Instead of removing it | 19:22 |
| zaneb | I'm not convinced this is the whole story, because the tripleO gate shouldn't be failing and then retrying the upgrade | 19:22 |
| therve | That would solve the dependency chaining issue that tripleo has | 19:22 |
| * therve nods | 19:22 | |
| *** armaan_ has quit IRC | 19:23 | |
| therve | The last occurrence is on a failure though | 19:23 |
| therve | From what I've been told. CI was passing. | 19:23 |
| zaneb | ok. one of those bugs had a link to some gate logs, but they'd already been deleted so I don't know what actually happened | 19:24 |
| *** hoolio has quit IRC | 19:24 | |
| zaneb | the OS::Heat::None idea is probably a good one, but implementing that is its own whole can of worms | 19:24 |
| therve | Yeah | 19:24 |
| therve | Yeah we need to make sure we properly switch the type | 19:25 |
| therve | Anyway, throwing that out there, considering we have at least your pending fix | 19:26 |
| therve | That tactic would presumably make it obsolete | 19:26 |
| *** hoolio has joined #heat | 19:27 | |
| *** jmlowe has quit IRC | 19:46 | |
| openstackgerrit | Thomas Herve proposed openstack/heat master: Don't store attributes if resource not stored https://review.openstack.org/479279 | 20:16 |
| *** ricolin_ has joined #heat | 20:17 | |
| *** ricolin has quit IRC | 20:20 | |
| therve | zaneb, Responded to https://review.openstack.org/#/c/425057/ btw | 20:32 |
| zaneb | therve: are you saying that json.dump() would have converted to unicode already anyway? | 20:34 |
| therve | zaneb, json doesn't support bytes | 20:35 |
| *** armaan has quit IRC | 20:36 | |
| zaneb | clearly it did in python2 | 20:36 |
| zaneb | (not disagreeing, I just tried it and it indeed doesn't) | 20:36 |
| *** armaan has joined #heat | 20:36 | |
| therve | Yeah py3 is stricter about that :) | 20:36 |
| *** annegentle has quit IRC | 20:38 | |
| zaneb | my concern is that we can expect a random, user-defined command to necessarily print a valid utf-8 bytestrem to stdout | 20:42 |
| zaneb | *NOT expect | 20:42 |
| zaneb | maybe we just need to pass errors='replace' instead of (the default) errors='strict' | 20:44 |
| therve | yeah that sounds fine to me | 20:46 |
| therve | I said ignore but replace works too | 20:46 |
| *** dprince has quit IRC | 21:19 | |
| *** felipemonteiro_ has quit IRC | 21:48 | |
| openstackgerrit | Zane Bitter proposed openstack/heat master: Fix a DB error when updating the resource attributes https://review.openstack.org/479075 | 22:05 |
| *** askb has joined #heat | 22:14 | |
| *** hongbin has quit IRC | 23:23 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!