*** jistr has quit IRC | 00:00 | |
*** jistr has joined #openstack-mistral | 00:01 | |
*** bobh has quit IRC | 00:32 | |
*** bobh has joined #openstack-mistral | 00:33 | |
*** zhurong has joined #openstack-mistral | 00:43 | |
kong | ok, the series changes: https://review.openstack.org/#/q/topic:mistral-lib-0.1.0 | 01:14 |
---|---|---|
openstackgerrit | Winson Chan proposed openstack/mistral master: Add option to run actions locally on the engine https://review.openstack.org/454411 | 01:25 |
*** bobh has quit IRC | 01:56 | |
*** bobh has joined #openstack-mistral | 02:08 | |
*** vishwanathj has joined #openstack-mistral | 02:17 | |
*** jrist has quit IRC | 02:19 | |
*** vishwanathj has quit IRC | 02:19 | |
*** vishwanathj has joined #openstack-mistral | 02:20 | |
*** jrist has joined #openstack-mistral | 02:20 | |
*** thrash is now known as thrash|g0ne | 02:33 | |
*** bobh has quit IRC | 03:15 | |
*** FL1SK has quit IRC | 03:26 | |
*** zhurong has quit IRC | 03:51 | |
*** zhurong has joined #openstack-mistral | 04:04 | |
*** jamielennox is now known as jamielennox|away | 04:18 | |
*** sharatss has joined #openstack-mistral | 05:08 | |
rakhmerov | kong: awesome, thanks | 05:32 |
rakhmerov | d0ugal, thrash|g0ne: just following up on the thing you discussed yesterday | 05:47 |
rakhmerov | so, I looked at the issues reported by thrash|g0ne, it's really hard to tell what the reason is w/o deep investigation. It might be related with concurrency as you assumed, with Apache itself (because it controls threads itself AFAIK) or e.g. with DB objects ordering issue (we used to observe this a lot) | 05:49 |
rakhmerov | thrash|g0ne, d0ugal: let me know if there's any updates on that | 05:50 |
rakhmerov | d0ugal: on CLI guide, I tend to agree with your change although some part of my consciousness says "hey, what if someone wants to see Mistral commands w/o installing Mistral yet?" | 05:52 |
rakhmerov | d0ugal: and yes, initially it was auto generated, Nikolay Makhotkin wrote a script for that | 05:53 |
*** zhurong has quit IRC | 05:55 | |
rakhmerov | ideally, I'd say it might be useful to have the full list of commands but only if: | 05:56 |
rakhmerov | 1) their docs are nicely and comprehensively designed | 05:56 |
rakhmerov | 2) their docs are properly auto-published from the source code | 05:56 |
rakhmerov | 3) they take a place in docs so that they are not annoying, meaning that if someone wants to see it they will see it but it doesn't jump right into user's eyes once they open the doc | 06:01 |
rakhmerov | thrash|g0ne, d0ugal: on the security spec, let's continue discussion in the spec itself. I think what Moshe is suggesting is good but 1) it may be a good suggestion for the next DSL version 2) I always have the goal in mind to keep balance between simplicity and power when it comes to language so I think there should be no rush at least when discussing it | 06:08 |
rakhmerov | 3) and yes, I think it can be mentioned in "Alternative" for now and addressed in details in a separate spec | 06:08 |
rakhmerov | so anyway, let's not hurry with this, I always ask about it when we discuss language changes | 06:09 |
*** zhurong has joined #openstack-mistral | 06:16 | |
openstackgerrit | Merged openstack/mistral-lib master: Updated from global requirements https://review.openstack.org/454493 | 06:19 |
*** warface has joined #openstack-mistral | 06:36 | |
d0ugal | kong: thanks! | 06:57 |
*** jaosorior_away is now known as jaosorior | 07:09 | |
*** jtomasek has quit IRC | 07:19 | |
*** jtomasek has joined #openstack-mistral | 07:19 | |
d0ugal | rakhmerov: can you +1 this? https://review.openstack.org/#/c/454409/ | 07:32 |
d0ugal | rakhmerov: see the comment from Steve Martinelli | 07:32 |
*** shardy has joined #openstack-mistral | 07:34 | |
*** jpich has joined #openstack-mistral | 07:37 | |
apetrich | d0ugal, about mistral-lib and context. should the tripleoAction use the context that comes from the run method for stuff like get_orchestration_client or do we leave as it is and it will still depend on mistral.context? | 07:51 |
apetrich | d0ugal, I'm tempted to on the run method save the context to self.context and use that throughout | 07:51 |
d0ugal | apetrich: yeah, saving it on self like that seems reasonable. | 07:52 |
d0ugal | apetrich: or add context to each of the get_*_client methods | 07:52 |
apetrich | but it feels flimsy in the sense that it is required and if some Action doesn't do that it is an error | 07:52 |
d0ugal | apetrich: yeah, maybe passing it on is the clearer approach? | 07:52 |
d0ugal | I don't really mind. | 07:52 |
apetrich | agreed. | 07:53 |
apetrich | A lot more places to change but it is cleaner | 07:53 |
d0ugal | apetrich: most actions only use one or two clients, so it shouldn't be that difficult to use in the end | 07:53 |
d0ugal | cool | 07:53 |
*** warface has quit IRC | 07:55 | |
*** warface has joined #openstack-mistral | 07:56 | |
*** warface has quit IRC | 08:02 | |
*** warface has joined #openstack-mistral | 08:04 | |
jenner_ | guys, can someone help me understand the "join" semantics? consider this WF https://dpaste.de/7Prw (it's obviously broken) - what I'm trying to achieve is something like http://a.rulim.de/workflow.png (last condition should read status != 200). thing is, "ze_end" is waiting for *all* tasks to complete, even if "print_status" wasn't started since "get_url" finished successfully | 08:25 |
openstackgerrit | Merged openstack/mistral master: Updated from global requirements https://review.openstack.org/454492 | 08:29 |
*** zhurong has quit IRC | 08:41 | |
d0ugal | jenner_: I think that workflow will do what you want without the join. | 09:25 |
d0ugal | so you don't have any parallel branches to join | 09:26 |
*** gongysh has quit IRC | 09:28 | |
openstackgerrit | Dougal Matthews proposed openstack/mistral master: Add support for mistral-lib to Mistral https://review.openstack.org/453517 | 09:29 |
*** gongysh has joined #openstack-mistral | 09:30 | |
d0ugal | rakhmerov: Maybe we could just reference this? https://docs.openstack.org/cli-reference/mistral.html | 09:34 |
d0ugal | rakhmerov: or find out how that is generated and borrow it... | 09:34 |
apetrich | d0ugal, I'm not completely sure about the context in the mistral default executor. How is it going to be able to access the correct thread_locals. | 09:36 |
d0ugal | apetrich: hmm, I'm not sure | 09:37 |
d0ugal | apetrich: honestly I am a bit confused with how the thread locals work - I'll try and understand more | 09:38 |
d0ugal | apetrich: but I thought the same one was shared? | 09:38 |
apetrich | d0ugal, yeah. I'm trying to test it right now. probably after lunch I will be sure that it is not an issue and I'm just misunderstanding | 09:39 |
d0ugal | apetrich: cool, let me know either way because I know I don't fully understand :-D | 09:40 |
apetrich | d0ugal, so I've moved all the context to come from the run method https://review.openstack.org/454632 | 09:40 |
d0ugal | apetrich: I commented on it already :) | 09:41 |
apetrich | I just saw it | 09:42 |
apetrich | cheers | 09:42 |
d0ugal | apetrich: it doesn't need to be in the same patch, but we also need to change the imports of mistral.workflow | 09:42 |
d0ugal | and replace it with an import of https://github.com/openstack/mistral-lib/blob/master/mistral_lib/actions/types.py#L21 | 09:43 |
d0ugal | Result should also be here to make the API easier to use. https://github.com/openstack/mistral-lib/blob/master/mistral_lib/actions/__init__.py#L17 | 09:44 |
d0ugal | I'm going to put a patch up for that now | 09:44 |
apetrich | d0ugal, fixed the actions import | 09:44 |
apetrich | d0ugal, cool | 09:45 |
d0ugal | and actually, there might be a chance needed in Mistral to make the Result class work - I'll look into that. | 09:46 |
openstackgerrit | Dougal Matthews proposed openstack/mistral-lib master: Make the Result class more convienient to use https://review.openstack.org/454643 | 09:47 |
d0ugal | apetrich: ^ you may want to depend on that. | 09:47 |
apetrich | d0ugal, aye | 09:47 |
d0ugal | apetrich: this is becoming quite the dep tree. | 09:48 |
d0ugal | as you also indirectly depend on these: https://review.openstack.org/#/q/topic:mistral-lib-0.1.0 | 09:48 |
d0ugal | :) | 09:48 |
*** zhurong has joined #openstack-mistral | 09:54 | |
*** zhurong has quit IRC | 09:55 | |
openstackgerrit | Dougal Matthews proposed openstack/mistral-lib master: Make the Result class more convienient to use https://review.openstack.org/454643 | 10:07 |
*** jaosorior has quit IRC | 10:11 | |
*** jaosorior has joined #openstack-mistral | 10:15 | |
*** tung_doan_ has joined #openstack-mistral | 10:23 | |
*** tung_doan has quit IRC | 10:24 | |
rakhmerov | d0ugal: yeah, let's just reference that guide for now | 10:25 |
*** tung_doan_ is now known as tung_doan | 10:25 | |
rakhmerov | and then we can borrow ) | 10:25 |
rakhmerov | jenner_: hi, still have that question about "join"? | 10:27 |
rakhmerov | I can help | 10:27 |
warface | hey guys, i was wondering if there is any standalone GUI for managing mistral workflows ? | 10:39 |
warface | AFAIK there is a horizon plugin, but we are using standalone mistral image, so curious if there's any GUI available as well ? | 10:41 |
*** jkilpatr has quit IRC | 10:42 | |
rakhmerov | warface: it's being created now but not yet open sourced | 10:47 |
rakhmerov | d0ugal: btw, my management supported the idea to demo that UI tool | 10:48 |
rakhmerov | we'll do it in a couple of weeks after releasing Pike-1 | 10:48 |
*** sharatss has quit IRC | 10:48 | |
d0ugal | nice! | 10:49 |
rakhmerov | and we agreed that we're targeting Sydney summit to announce it officially | 10:49 |
*** sharatss has joined #openstack-mistral | 10:49 | |
d0ugal | heh, cool | 10:49 |
d0ugal | shame I wont be there :) | 10:49 |
openstackgerrit | Kevin Pouget proposed openstack/python-mistralclient master: Add CLI for event trigger operations https://review.openstack.org/454669 | 10:49 |
rakhmerov | d0ugal: yeah, may be we'll demo CloudFlow (that's the current name of the tool) on a team weekly meeting via webex | 10:52 |
*** thrash|g0ne is now known as thrash | 10:52 | |
*** jaosorior has quit IRC | 10:53 | |
*** jaosorior has joined #openstack-mistral | 10:53 | |
*** jaosorior has quit IRC | 10:53 | |
thrash | rakhmerov: thanks for looking into the concurrency problem. Don't bother spending much more time on it. I'm already neck-deep in a deep-dive on it. I just wanted to run it by you in case you had an instant insight on it. | 10:54 |
rakhmerov | thrash: ok :) | 10:55 |
*** jaosorior has joined #openstack-mistral | 10:56 | |
thrash | rakhmerov: I think one thing I may do is to get the devstack install to support wsgi from apache... | 10:58 |
*** jkilpatr has joined #openstack-mistral | 10:59 | |
*** jaosorior_ has joined #openstack-mistral | 11:00 | |
*** jaosorior has quit IRC | 11:01 | |
*** openstackgerrit has quit IRC | 11:18 | |
rakhmerov | thrash: you mean a new CI gate? | 11:19 |
thrash | rakhmerov: that's what it could turn into, yes. | 11:19 |
rakhmerov | ooh, that would be useful | 11:19 |
rakhmerov | ok | 11:19 |
*** shardy is now known as shardy_lunch | 11:21 | |
*** jaosorior_ is now known as jaosorior | 11:37 | |
*** bobh has joined #openstack-mistral | 11:42 | |
*** bobh has quit IRC | 11:43 | |
*** bobh has joined #openstack-mistral | 11:44 | |
*** bobh has quit IRC | 11:49 | |
*** thrash is now known as thrash|brb | 11:52 | |
*** openstackgerrit has joined #openstack-mistral | 11:55 | |
openstackgerrit | Kevin Pouget proposed openstack/python-mistralclient master: Add CLI for event trigger operations https://review.openstack.org/454669 | 11:55 |
*** sharatss has quit IRC | 12:04 | |
*** sharatss has joined #openstack-mistral | 12:04 | |
apetrich | d0ugal, so I was wrong about the context. it worked | 12:06 |
d0ugal | apetrich: phew :) | 12:06 |
apetrich | d0ugal, I'm so scared that it worked fine | 12:06 |
d0ugal | rakhmerov: hey | 12:07 |
d0ugal | apetrich: lol, too easy? | 12:07 |
* apetrich is suspicious | 12:08 | |
apetrich | after a week failing it in different ways. yeah a bit | 12:08 |
*** jaosorior has quit IRC | 12:09 | |
*** jaosorior has joined #openstack-mistral | 12:09 | |
*** jtomasek_ has joined #openstack-mistral | 12:12 | |
apetrich | d0ugal, yeah validate passes | 12:14 |
*** jtomasek_ has quit IRC | 12:18 | |
d0ugal | apetrich: neat | 12:19 |
d0ugal | rakhmerov: can you also +1 https://review.openstack.org/#/c/454391/ | 12:25 |
*** gongysh has quit IRC | 12:29 | |
*** thrash|brb is now known as thrash | 12:30 | |
*** shardy_lunch is now known as shardy | 12:32 | |
jenner_ | rakhmerov: hi, yeah, kind of -- it works with join: 2, but it's kinda unexpected since the on-error task wasn't even started | 12:33 |
jenner_ | d0ugal:I wanted to simulate async/parallel execution | 12:34 |
*** dprince has joined #openstack-mistral | 12:35 | |
*** warface has quit IRC | 12:44 | |
d0ugal | jenner_: I see, I can help you come up with an example if you want? | 12:46 |
jenner_ | d0ugal: this would be awesome! :) | 12:46 |
d0ugal | okay, give me a few mins. | 12:46 |
*** sharatss has quit IRC | 12:48 | |
*** jtomasek has quit IRC | 12:49 | |
jenner_ | I'm also interested in best practices for custom action maintenance - if I'm reading the docs correctly, I need to inherit mistral.actions.base and implement the run() method (update DB etc.), however the docs also say I need to reinstall mistral - why is that? | 12:49 |
*** jtomasek has joined #openstack-mistral | 12:49 | |
openstackgerrit | Brad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack https://review.openstack.org/454710 | 12:49 |
*** warface has joined #openstack-mistral | 12:49 | |
*** jtomasek has quit IRC | 12:50 | |
*** jtomasek has joined #openstack-mistral | 12:50 | |
*** toure|biab is now known as toure | 12:58 | |
openstackgerrit | Brad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack https://review.openstack.org/454710 | 13:00 |
*** catintheroof has joined #openstack-mistral | 13:03 | |
d0ugal | jenner_: sorry, I got distracted. | 13:13 |
d0ugal | jenner_: this is actually a pretty good example of join: https://github.com/openstack/mistral-extra/blob/5eebb0d5dcaf8e1973a91e3fa6f72eb61cacab0c/examples/v2/capabilities/join/create_vm_with_volume.yaml | 13:13 |
d0ugal | mistral-extra has a few examples in there, worth taking a look at | 13:13 |
d0ugal | jenner_: regarding custom actions - you don't need to reinstall mistral, that sounds odd - can you link to where in the docs it says that? | 13:14 |
d0ugal | jenner_: you do need to restart mistral tho' | 13:14 |
d0ugal | jenner_: depending when you plan to use Mistral and what version - there is a new API for writing custom actions coming. It will be easier and clearer to use. http://specs.openstack.org/openstack/mistral-specs/specs/pike/approved/mistral-custom-actions-api.html | 13:15 |
warface | rakhmerov: ok, thanks | 13:22 |
jenner_ | d0ugal: https://docs.openstack.org/developer/mistral/developer/creating_custom_action.html | 13:33 |
*** chlong has joined #openstack-mistral | 13:33 | |
jenner_ | d0ugal: thanks, I've already had a look at mistral-extra | 13:33 |
d0ugal | jenner_: weird, we never reinstall Mistral anyway :) | 13:35 |
d0ugal | restarting is enough | 13:35 |
jenner_ | d0ugal: restarting bad though :( I'd need to find a way to mark an engine-node (only engine or does this apply to executors too?) as "in maintenance" so it doesn't pick new WFs, wait until it finishes all executions and then restart it... rinse and repeat for the rest of the cluster | 13:37 |
d0ugal | jenner_: agreed, restarting is bad - this is mostly due to a limitation of Python AFAIK. However, we need to do something to work around it at some point | 13:37 |
jenner_ | d0ugal: does the engine really need to be restarted? I'd have thought I just need to update the DB so it knows about possible new actions and the restart only the executors | 13:39 |
jenner_ | *and then | 13:40 |
d0ugal | jenner_: If you are changing an existing action - yes, because it will already be imported and in memory | 13:40 |
d0ugal | if it is an existing action... maybe you don't need to restart everything. This would require some testing. | 13:40 |
jenner_ | d0ugal: but what if I run the engine on another host or in an extra process? the engine itself doesn't import the action, right? the executor process does though | 13:41 |
d0ugal | We very rarely add new actions in production - because I use Mistral in another OpenStack project (so release every 6 months) - so I don't hit this issue often :) | 13:41 |
d0ugal | jenner_: yeah, that might work. | 13:42 |
d0ugal | I guess the answer is; it might work, but the safe option is to restart if you can. | 13:43 |
jenner_ | k | 13:43 |
d0ugal | Somebody that releases more often will have better experience of this :) | 13:43 |
d0ugal | I would ask rakhmerov next week. | 13:43 |
jenner_ | well I guess it won't happen that often once we hit production, I just need to make sure we have a robust operational concept | 13:44 |
d0ugal | sure | 13:45 |
jenner_ | what about orchestration? any way to say "executor foo.bar is going down in a minute, disregard it for new tasks"? | 13:46 |
d0ugal | No, I don't think so | 13:47 |
openstackgerrit | Brad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack https://review.openstack.org/454710 | 13:47 |
d0ugal | but I would need to check | 13:47 |
d0ugal | jenner_: you could use the Task affinity to run workflows on a specific executor | 13:50 |
*** gongysh has joined #openstack-mistral | 13:50 | |
d0ugal | see the "target" attribute in a workflow: https://docs.openstack.org/developer/mistral/dsl/dsl_v2.html#common-task-attributes | 13:50 |
d0ugal | but that would require updating each workflow every time you want to remove a executor | 13:50 |
jenner_ | exactly | 13:51 |
jenner_ | I need to do the opposite :) | 13:51 |
*** zhurong has joined #openstack-mistral | 13:52 | |
*** zhurong has quit IRC | 13:53 | |
*** bobh has joined #openstack-mistral | 13:56 | |
*** warface has quit IRC | 13:59 | |
jenner_ | d0ugal: there's an error in description btw: https://github.com/openstack/mistral-extra/blob/master/examples/v2/calculator/calculator.yaml#L93 :) | 14:04 |
d0ugal | jenner_: thanks, I'll fix that | 14:05 |
openstackgerrit | Dougal Matthews proposed openstack/mistral-extra master: Correct the description on a calculator task https://review.openstack.org/454731 | 14:07 |
d0ugal | jenner_: ^ | 14:07 |
*** amoralej is now known as amoralej|lunch | 14:20 | |
*** jkilpatr has quit IRC | 14:24 | |
openstackgerrit | Brad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack https://review.openstack.org/454710 | 14:34 |
*** thrash is now known as thrash|biab | 14:34 | |
*** FL1SK has joined #openstack-mistral | 14:42 | |
*** amoralej|lunch is now known as amoralej | 14:50 | |
openstackgerrit | Kevin Pouget proposed openstack/mistral master: Accept worfklow_name ou worfkflow_is as identifier for event trigger creation https://review.openstack.org/454750 | 14:56 |
*** thrash|biab is now known as thrash | 14:58 | |
*** gongysh has quit IRC | 15:04 | |
*** catintheroof has quit IRC | 15:06 | |
*** catintheroof has joined #openstack-mistral | 15:06 | |
*** catintheroof has quit IRC | 15:07 | |
*** catintheroof has joined #openstack-mistral | 15:15 | |
*** jtomasek_ has joined #openstack-mistral | 15:16 | |
*** catintheroof has quit IRC | 15:16 | |
*** catintheroof has joined #openstack-mistral | 15:26 | |
*** catintheroof has quit IRC | 15:27 | |
*** jkilpatr has joined #openstack-mistral | 15:29 | |
*** jaosorior has quit IRC | 15:44 | |
*** jtomasek_ has quit IRC | 15:51 | |
*** jpich has quit IRC | 16:03 | |
*** thrash is now known as thrash|f00dz | 16:22 | |
*** jkilpatr has quit IRC | 16:34 | |
*** catintheroof has joined #openstack-mistral | 16:51 | |
openstackgerrit | Dirk Mueller proposed openstack/mistral master: WIP: use pycodestyle instead of pep8 https://review.openstack.org/454799 | 16:52 |
*** catintheroof has quit IRC | 16:57 | |
*** dprince has quit IRC | 17:06 | |
*** dprince has joined #openstack-mistral | 17:08 | |
*** amoralej is now known as amoralej|off | 17:21 | |
*** shardy has quit IRC | 17:35 | |
*** jtomasek_ has joined #openstack-mistral | 17:35 | |
*** thrash|f00dz is now known as thrash | 17:36 | |
*** jtomasek_ has quit IRC | 17:40 | |
*** chlong_ has joined #openstack-mistral | 17:46 | |
*** jkilpatr has joined #openstack-mistral | 18:09 | |
*** fultonj has quit IRC | 18:12 | |
*** fultonj has joined #openstack-mistral | 18:12 | |
jenner_ | d0ugal: +1 :) | 18:33 |
*** jtomasek_ has joined #openstack-mistral | 18:53 | |
*** jtomasek_ has quit IRC | 19:12 | |
openstackgerrit | Brad P. Crochet proposed openstack/mistral master: Enable WSGI under Apache in devstack https://review.openstack.org/454710 | 19:19 |
*** openstackgerrit has quit IRC | 19:19 | |
*** jkilpatr has quit IRC | 19:33 | |
*** chlong_ has quit IRC | 19:37 | |
*** chlong has quit IRC | 19:37 | |
*** fultonj has quit IRC | 19:39 | |
*** fultonj has joined #openstack-mistral | 19:40 | |
*** jtomasek_ has joined #openstack-mistral | 19:42 | |
*** chlong has joined #openstack-mistral | 19:42 | |
*** jtomasek_ has quit IRC | 19:46 | |
*** jtomasek_ has joined #openstack-mistral | 20:23 | |
*** fultonj has quit IRC | 20:37 | |
*** fultonj has joined #openstack-mistral | 20:37 | |
*** jkilpatr has joined #openstack-mistral | 20:41 | |
*** bobh_ has joined #openstack-mistral | 20:51 | |
*** bobh has quit IRC | 20:55 | |
*** bobh has joined #openstack-mistral | 20:56 | |
*** bobh_ has quit IRC | 20:59 | |
*** bobh has quit IRC | 21:04 | |
*** toure is now known as toure|gone | 21:05 | |
*** chlong has quit IRC | 21:08 | |
*** fultonj has quit IRC | 21:09 | |
*** catintheroof has joined #openstack-mistral | 21:12 | |
*** catintheroof has quit IRC | 21:16 | |
*** dprince has quit IRC | 21:24 | |
*** jkilpatr has quit IRC | 21:26 | |
*** aspiers has quit IRC | 21:30 | |
*** bobh has joined #openstack-mistral | 21:30 | |
*** bobh has quit IRC | 21:33 | |
*** bobh has joined #openstack-mistral | 21:34 | |
*** bobh has quit IRC | 21:36 | |
*** jkilpatr has joined #openstack-mistral | 21:36 | |
*** bobh has joined #openstack-mistral | 21:37 | |
*** bobh has quit IRC | 21:41 | |
*** amoralej|off has quit IRC | 21:55 | |
*** jtomasek_ has quit IRC | 22:24 | |
*** aspiers has joined #openstack-mistral | 22:32 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!