Friday, 2017-04-07

*** jistr has quit IRC00:00
*** jistr has joined #openstack-mistral00:01
*** bobh has quit IRC00:32
*** bobh has joined #openstack-mistral00:33
*** zhurong has joined #openstack-mistral00:43
kongok, the series changes: https://review.openstack.org/#/q/topic:mistral-lib-0.1.001:14
openstackgerritWinson Chan proposed openstack/mistral master: Add option to run actions locally on the engine  https://review.openstack.org/45441101:25
*** bobh has quit IRC01:56
*** bobh has joined #openstack-mistral02:08
*** vishwanathj has joined #openstack-mistral02:17
*** jrist has quit IRC02:19
*** vishwanathj has quit IRC02:19
*** vishwanathj has joined #openstack-mistral02:20
*** jrist has joined #openstack-mistral02:20
*** thrash is now known as thrash|g0ne02:33
*** bobh has quit IRC03:15
*** FL1SK has quit IRC03:26
*** zhurong has quit IRC03:51
*** zhurong has joined #openstack-mistral04:04
*** jamielennox is now known as jamielennox|away04:18
*** sharatss has joined #openstack-mistral05:08
rakhmerovkong: awesome, thanks05:32
rakhmerovd0ugal, thrash|g0ne: just following up on the thing you discussed yesterday05:47
rakhmerovso, I looked at the issues reported by thrash|g0ne, it's really hard to tell what the reason is w/o deep investigation. It might be related with concurrency as you assumed, with Apache itself (because it controls threads itself AFAIK) or e.g. with DB objects ordering issue (we used to observe this a lot)05:49
rakhmerovthrash|g0ne, d0ugal: let me know if there's any updates on that05:50
rakhmerovd0ugal: on CLI guide, I tend to agree with your change although some part of my consciousness says "hey, what if someone wants to see Mistral commands w/o installing Mistral yet?"05:52
rakhmerovd0ugal: and yes, initially it was auto generated, Nikolay Makhotkin wrote a script for that05:53
*** zhurong has quit IRC05:55
rakhmerovideally, I'd say it might be useful to have the full list of commands but only if:05:56
rakhmerov1) their docs are nicely and comprehensively designed05:56
rakhmerov2) their docs are properly auto-published from the source code05:56
rakhmerov3) they take a place in docs so that they are not annoying, meaning that if someone wants to see it they will see it but it doesn't jump right into user's eyes once they open the doc06:01
rakhmerovthrash|g0ne, d0ugal: on the security spec, let's continue discussion in the spec itself. I think what Moshe is suggesting is good but 1) it may be a good suggestion for the next DSL version 2) I always have the goal in mind to keep balance between simplicity and power when it comes to language so I think there should be no rush at least when discussing it06:08
rakhmerov3) and yes, I think it can be mentioned in "Alternative" for now and addressed in details in a separate spec06:08
rakhmerovso anyway, let's not hurry with this, I always ask about it when we discuss language changes06:09
*** zhurong has joined #openstack-mistral06:16
openstackgerritMerged openstack/mistral-lib master: Updated from global requirements  https://review.openstack.org/45449306:19
*** warface has joined #openstack-mistral06:36
d0ugalkong: thanks!06:57
*** jaosorior_away is now known as jaosorior07:09
*** jtomasek has quit IRC07:19
*** jtomasek has joined #openstack-mistral07:19
d0ugalrakhmerov: can you +1 this? https://review.openstack.org/#/c/454409/07:32
d0ugalrakhmerov: see the comment from Steve Martinelli07:32
*** shardy has joined #openstack-mistral07:34
*** jpich has joined #openstack-mistral07:37
apetrichd0ugal, about mistral-lib and context. should the tripleoAction use the context that comes from the run method for stuff like get_orchestration_client or do we leave as it is and it will still depend on mistral.context?07:51
apetrichd0ugal, I'm tempted to on the run method save the context to self.context and use that throughout07:51
d0ugalapetrich: yeah, saving it on self like that seems reasonable.07:52
d0ugalapetrich: or add context to each of the get_*_client methods07:52
apetrichbut it feels flimsy in the sense that it is required and if some Action doesn't do that it is an error07:52
d0ugalapetrich: yeah, maybe passing it on is the clearer approach?07:52
d0ugalI don't really mind.07:52
apetrichagreed.07:53
apetrichA lot more places to change but it is cleaner07:53
d0ugalapetrich: most actions only use one or two clients, so it shouldn't be that difficult to use in the end07:53
d0ugalcool07:53
*** warface has quit IRC07:55
*** warface has joined #openstack-mistral07:56
*** warface has quit IRC08:02
*** warface has joined #openstack-mistral08:04
jenner_guys, can someone help me understand the "join" semantics? consider this WF https://dpaste.de/7Prw (it's obviously broken) - what I'm trying to achieve is something like http://a.rulim.de/workflow.png (last condition should read status != 200). thing is, "ze_end" is waiting for *all* tasks to complete, even if "print_status" wasn't started since "get_url" finished successfully08:25
openstackgerritMerged openstack/mistral master: Updated from global requirements  https://review.openstack.org/45449208:29
*** zhurong has quit IRC08:41
d0ugaljenner_: I think that workflow will do what you want without the join.09:25
d0ugalso you don't have any parallel branches to join09:26
*** gongysh has quit IRC09:28
openstackgerritDougal Matthews proposed openstack/mistral master: Add support for mistral-lib to Mistral  https://review.openstack.org/45351709:29
*** gongysh has joined #openstack-mistral09:30
d0ugalrakhmerov: Maybe we could just reference this? https://docs.openstack.org/cli-reference/mistral.html09:34
d0ugalrakhmerov: or find out how that is generated and borrow it...09:34
apetrichd0ugal, I'm not completely sure about the context in the mistral default executor. How is it going to be able to access the correct thread_locals.09:36
d0ugalapetrich: hmm, I'm not sure09:37
d0ugalapetrich: honestly I am a bit confused with how the thread locals work - I'll try and understand more09:38
d0ugalapetrich: but I thought the same one was shared?09:38
apetrichd0ugal, yeah. I'm trying to test it right now. probably after lunch I will be sure that it is not an issue and I'm just misunderstanding09:39
d0ugalapetrich: cool, let me know either way because I know I don't fully understand :-D09:40
apetrichd0ugal, so I've moved all the context to come from the run method https://review.openstack.org/45463209:40
d0ugalapetrich: I commented on it already :)09:41
apetrichI just saw it09:42
apetrichcheers09:42
d0ugalapetrich: it doesn't need to be in the same patch, but we also need to change the imports of mistral.workflow09:42
d0ugaland replace it with an import of https://github.com/openstack/mistral-lib/blob/master/mistral_lib/actions/types.py#L2109:43
d0ugalResult should also be here to make the API easier to use. https://github.com/openstack/mistral-lib/blob/master/mistral_lib/actions/__init__.py#L1709:44
d0ugalI'm going to  put a patch up for that now09:44
apetrichd0ugal, fixed the actions import09:44
apetrichd0ugal, cool09:45
d0ugaland actually, there might be a chance needed in Mistral to make the Result class work - I'll look into that.09:46
openstackgerritDougal Matthews proposed openstack/mistral-lib master: Make the Result class more convienient to use  https://review.openstack.org/45464309:47
d0ugalapetrich: ^ you may want to depend on that.09:47
apetrichd0ugal, aye09:47
d0ugalapetrich: this is becoming quite the dep tree.09:48
d0ugalas you also indirectly depend on these: https://review.openstack.org/#/q/topic:mistral-lib-0.1.009:48
d0ugal:)09:48
*** zhurong has joined #openstack-mistral09:54
*** zhurong has quit IRC09:55
openstackgerritDougal Matthews proposed openstack/mistral-lib master: Make the Result class more convienient to use  https://review.openstack.org/45464310:07
*** jaosorior has quit IRC10:11
*** jaosorior has joined #openstack-mistral10:15
*** tung_doan_ has joined #openstack-mistral10:23
*** tung_doan has quit IRC10:24
rakhmerovd0ugal: yeah, let's just reference that guide for now10:25
*** tung_doan_ is now known as tung_doan10:25
rakhmerovand then we can borrow )10:25
rakhmerovjenner_: hi, still have that question about "join"?10:27
rakhmerovI can help10:27
warfacehey guys, i was wondering if there is any standalone GUI for managing mistral workflows ?10:39
warfaceAFAIK there is a horizon plugin, but we are using standalone mistral image, so curious if there's any GUI available as well ?10:41
*** jkilpatr has quit IRC10:42
rakhmerovwarface: it's being created now but not yet open sourced10:47
rakhmerovd0ugal: btw, my management supported the idea to demo that UI tool10:48
rakhmerovwe'll do it in a couple of weeks after releasing Pike-110:48
*** sharatss has quit IRC10:48
d0ugalnice!10:49
rakhmerovand we agreed that we're targeting Sydney summit to announce it officially10:49
*** sharatss has joined #openstack-mistral10:49
d0ugalheh, cool10:49
d0ugalshame I wont be there :)10:49
openstackgerritKevin Pouget proposed openstack/python-mistralclient master: Add CLI for event trigger operations  https://review.openstack.org/45466910:49
rakhmerovd0ugal: yeah, may be we'll demo CloudFlow (that's the current name of the tool) on a team weekly meeting via webex10:52
*** thrash|g0ne is now known as thrash10:52
*** jaosorior has quit IRC10:53
*** jaosorior has joined #openstack-mistral10:53
*** jaosorior has quit IRC10:53
thrashrakhmerov: thanks for looking into the concurrency problem. Don't bother spending much more time on it. I'm already neck-deep in a deep-dive on it. I just wanted to run it by you in case you had an instant insight on it.10:54
rakhmerovthrash: ok :)10:55
*** jaosorior has joined #openstack-mistral10:56
thrashrakhmerov: I think one thing I may do is to get the devstack install to support wsgi from apache...10:58
*** jkilpatr has joined #openstack-mistral10:59
*** jaosorior_ has joined #openstack-mistral11:00
*** jaosorior has quit IRC11:01
*** openstackgerrit has quit IRC11:18
rakhmerovthrash: you mean a new CI gate?11:19
thrashrakhmerov: that's what it could turn into, yes.11:19
rakhmerovooh, that would be useful11:19
rakhmerovok11:19
*** shardy is now known as shardy_lunch11:21
*** jaosorior_ is now known as jaosorior11:37
*** bobh has joined #openstack-mistral11:42
*** bobh has quit IRC11:43
*** bobh has joined #openstack-mistral11:44
*** bobh has quit IRC11:49
*** thrash is now known as thrash|brb11:52
*** openstackgerrit has joined #openstack-mistral11:55
openstackgerritKevin Pouget proposed openstack/python-mistralclient master: Add CLI for event trigger operations  https://review.openstack.org/45466911:55
*** sharatss has quit IRC12:04
*** sharatss has joined #openstack-mistral12:04
apetrichd0ugal, so I was wrong about the context. it worked12:06
d0ugalapetrich: phew :)12:06
apetrichd0ugal, I'm so scared that it worked fine12:06
d0ugalrakhmerov: hey12:07
d0ugalapetrich: lol, too easy?12:07
* apetrich is suspicious12:08
apetrichafter a week failing it in different ways. yeah a bit12:08
*** jaosorior has quit IRC12:09
*** jaosorior has joined #openstack-mistral12:09
*** jtomasek_ has joined #openstack-mistral12:12
apetrichd0ugal, yeah validate passes12:14
*** jtomasek_ has quit IRC12:18
d0ugalapetrich: neat12:19
d0ugalrakhmerov: can you also +1 https://review.openstack.org/#/c/454391/12:25
*** gongysh has quit IRC12:29
*** thrash|brb is now known as thrash12:30
*** shardy_lunch is now known as shardy12:32
jenner_rakhmerov: hi, yeah, kind of -- it works with join: 2, but it's kinda unexpected since the on-error task wasn't even started12:33
jenner_d0ugal:I wanted to simulate async/parallel execution12:34
*** dprince has joined #openstack-mistral12:35
*** warface has quit IRC12:44
d0ugaljenner_: I see, I can help you come up with an example if you want?12:46
jenner_d0ugal: this would be awesome! :)12:46
d0ugalokay, give me a few mins.12:46
*** sharatss has quit IRC12:48
*** jtomasek has quit IRC12:49
jenner_I'm also interested in best practices for custom action maintenance - if I'm reading the docs correctly, I need to inherit mistral.actions.base and implement the run() method (update DB etc.), however the docs also say I need to reinstall mistral - why is that?12:49
*** jtomasek has joined #openstack-mistral12:49
openstackgerritBrad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack  https://review.openstack.org/45471012:49
*** warface has joined #openstack-mistral12:49
*** jtomasek has quit IRC12:50
*** jtomasek has joined #openstack-mistral12:50
*** toure|biab is now known as toure12:58
openstackgerritBrad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack  https://review.openstack.org/45471013:00
*** catintheroof has joined #openstack-mistral13:03
d0ugaljenner_: sorry, I got distracted.13:13
d0ugaljenner_: this is actually a pretty good example of join: https://github.com/openstack/mistral-extra/blob/5eebb0d5dcaf8e1973a91e3fa6f72eb61cacab0c/examples/v2/capabilities/join/create_vm_with_volume.yaml13:13
d0ugalmistral-extra has a few examples in there, worth taking a look at13:13
d0ugaljenner_: regarding custom actions - you don't need to reinstall mistral, that sounds odd - can you link to where in the docs it says that?13:14
d0ugaljenner_: you do need to restart mistral tho'13:14
d0ugaljenner_: depending when you plan to use Mistral and what version - there is a new API for writing custom actions coming. It will be easier and clearer to use. http://specs.openstack.org/openstack/mistral-specs/specs/pike/approved/mistral-custom-actions-api.html13:15
warfacerakhmerov: ok, thanks13:22
jenner_d0ugal: https://docs.openstack.org/developer/mistral/developer/creating_custom_action.html13:33
*** chlong has joined #openstack-mistral13:33
jenner_d0ugal: thanks, I've already had a look at mistral-extra13:33
d0ugaljenner_: weird, we never reinstall Mistral anyway :)13:35
d0ugalrestarting is enough13:35
jenner_d0ugal: restarting bad though :( I'd need to find a way to mark an engine-node (only engine or does this apply to executors too?) as "in maintenance" so it doesn't pick new WFs, wait until it finishes all executions and then restart it... rinse and repeat for the rest of the cluster13:37
d0ugaljenner_: agreed, restarting is bad - this is mostly due to a limitation of Python AFAIK. However, we need to do something to work around it at some point13:37
jenner_d0ugal: does the engine really need to be restarted? I'd have thought I just need to update the DB so it knows about possible new actions and the restart only the executors13:39
jenner_*and then13:40
d0ugaljenner_: If you are changing an existing action - yes, because it will already be imported and in memory13:40
d0ugalif it is an existing action... maybe you don't need to restart everything. This would require some testing.13:40
jenner_d0ugal: but what if I run the engine on another host or in an extra process? the engine itself doesn't import the action, right? the executor process does though13:41
d0ugalWe very rarely add new actions in production - because I use Mistral in another OpenStack project (so release every 6 months) - so I don't hit this issue often :)13:41
d0ugaljenner_: yeah, that might work.13:42
d0ugalI guess the answer is; it might work, but the safe option is to restart if you can.13:43
jenner_k13:43
d0ugalSomebody that releases more often will have better experience of this :)13:43
d0ugalI would ask rakhmerov next week.13:43
jenner_well I guess it won't happen that often once we hit production, I just need to make sure we have a robust operational concept13:44
d0ugalsure13:45
jenner_what about orchestration? any way to say "executor foo.bar is going down in a minute, disregard it for new tasks"?13:46
d0ugalNo, I don't think so13:47
openstackgerritBrad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack  https://review.openstack.org/45471013:47
d0ugalbut I would need to check13:47
d0ugaljenner_: you could use the Task affinity to run workflows on a specific executor13:50
*** gongysh has joined #openstack-mistral13:50
d0ugalsee the "target" attribute in a workflow: https://docs.openstack.org/developer/mistral/dsl/dsl_v2.html#common-task-attributes13:50
d0ugalbut that would require updating each workflow every time you want to remove a executor13:50
jenner_exactly13:51
jenner_I need to do the opposite :)13:51
*** zhurong has joined #openstack-mistral13:52
*** zhurong has quit IRC13:53
*** bobh has joined #openstack-mistral13:56
*** warface has quit IRC13:59
jenner_d0ugal: there's an error in description btw: https://github.com/openstack/mistral-extra/blob/master/examples/v2/calculator/calculator.yaml#L93 :)14:04
d0ugaljenner_: thanks, I'll fix that14:05
openstackgerritDougal Matthews proposed openstack/mistral-extra master: Correct the description on a calculator task  https://review.openstack.org/45473114:07
d0ugaljenner_: ^14:07
*** amoralej is now known as amoralej|lunch14:20
*** jkilpatr has quit IRC14:24
openstackgerritBrad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack  https://review.openstack.org/45471014:34
*** thrash is now known as thrash|biab14:34
*** FL1SK has joined #openstack-mistral14:42
*** amoralej|lunch is now known as amoralej14:50
openstackgerritKevin Pouget proposed openstack/mistral master: Accept worfklow_name ou worfkflow_is as identifier for event trigger creation  https://review.openstack.org/45475014:56
*** thrash|biab is now known as thrash14:58
*** gongysh has quit IRC15:04
*** catintheroof has quit IRC15:06
*** catintheroof has joined #openstack-mistral15:06
*** catintheroof has quit IRC15:07
*** catintheroof has joined #openstack-mistral15:15
*** jtomasek_ has joined #openstack-mistral15:16
*** catintheroof has quit IRC15:16
*** catintheroof has joined #openstack-mistral15:26
*** catintheroof has quit IRC15:27
*** jkilpatr has joined #openstack-mistral15:29
*** jaosorior has quit IRC15:44
*** jtomasek_ has quit IRC15:51
*** jpich has quit IRC16:03
*** thrash is now known as thrash|f00dz16:22
*** jkilpatr has quit IRC16:34
*** catintheroof has joined #openstack-mistral16:51
openstackgerritDirk Mueller proposed openstack/mistral master: WIP: use pycodestyle instead of pep8  https://review.openstack.org/45479916:52
*** catintheroof has quit IRC16:57
*** dprince has quit IRC17:06
*** dprince has joined #openstack-mistral17:08
*** amoralej is now known as amoralej|off17:21
*** shardy has quit IRC17:35
*** jtomasek_ has joined #openstack-mistral17:35
*** thrash|f00dz is now known as thrash17:36
*** jtomasek_ has quit IRC17:40
*** chlong_ has joined #openstack-mistral17:46
*** jkilpatr has joined #openstack-mistral18:09
*** fultonj has quit IRC18:12
*** fultonj has joined #openstack-mistral18:12
jenner_d0ugal: +1 :)18:33
*** jtomasek_ has joined #openstack-mistral18:53
*** jtomasek_ has quit IRC19:12
openstackgerritBrad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack  https://review.openstack.org/45471019:19
*** openstackgerrit has quit IRC19:19
*** jkilpatr has quit IRC19:33
*** chlong_ has quit IRC19:37
*** chlong has quit IRC19:37
*** fultonj has quit IRC19:39
*** fultonj has joined #openstack-mistral19:40
*** jtomasek_ has joined #openstack-mistral19:42
*** chlong has joined #openstack-mistral19:42
*** jtomasek_ has quit IRC19:46
*** jtomasek_ has joined #openstack-mistral20:23
*** fultonj has quit IRC20:37
*** fultonj has joined #openstack-mistral20:37
*** jkilpatr has joined #openstack-mistral20:41
*** bobh_ has joined #openstack-mistral20:51
*** bobh has quit IRC20:55
*** bobh has joined #openstack-mistral20:56
*** bobh_ has quit IRC20:59
*** bobh has quit IRC21:04
*** toure is now known as toure|gone21:05
*** chlong has quit IRC21:08
*** fultonj has quit IRC21:09
*** catintheroof has joined #openstack-mistral21:12
*** catintheroof has quit IRC21:16
*** dprince has quit IRC21:24
*** jkilpatr has quit IRC21:26
*** aspiers has quit IRC21:30
*** bobh has joined #openstack-mistral21:30
*** bobh has quit IRC21:33
*** bobh has joined #openstack-mistral21:34
*** bobh has quit IRC21:36
*** jkilpatr has joined #openstack-mistral21:36
*** bobh has joined #openstack-mistral21:37
*** bobh has quit IRC21:41
*** amoralej|off has quit IRC21:55
*** jtomasek_ has quit IRC22:24
*** aspiers has joined #openstack-mistral22:32

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!