| *** harlowja has quit IRC | 01:05 | |
| *** bobh has joined #openstack-mistral | 02:26 | |
| *** bobh has quit IRC | 03:06 | |
| *** apetrich has quit IRC | 03:23 | |
| *** apetrich has joined #openstack-mistral | 03:24 | |
| rakhmerov | d0ugal, pgaxatte: we can't just change the merging strategy in a hard coded manner | 03:50 |
|---|---|---|
| rakhmerov | we've discussed it a lot with many people and yes, d0ugal is right, that many find this merging strategy confusing | 03:51 |
| rakhmerov | that's why we ended up having a "replace" vs "merge" | 03:51 |
| rakhmerov | but I think a config option (to make changes to all workflows) + a variable in workflow environments (to change it per workflow) would be ok | 03:52 |
| rakhmerov | but leaving the current one by default | 03:52 |
| *** harlowja has joined #openstack-mistral | 04:25 | |
| *** harlowja has quit IRC | 04:44 | |
| *** bobh has joined #openstack-mistral | 04:55 | |
| *** hardikjasani has joined #openstack-mistral | 05:28 | |
| *** bobh has quit IRC | 06:00 | |
| *** bobh has joined #openstack-mistral | 06:03 | |
| pgaxatte | rakhmerov: I agree the current behavior should stay the default one | 06:51 |
| *** gkadam has joined #openstack-mistral | 07:01 | |
| *** bobh has quit IRC | 07:15 | |
| *** bobh has joined #openstack-mistral | 07:48 | |
| *** kimamisa has joined #openstack-mistral | 07:49 | |
| d0ugal | rakhmerov: I don't like the idea of a config for this. | 07:59 |
| d0ugal | rakhmerov: it reduces workflow portability. | 08:00 |
| d0ugal | #startmeeting mistral | 08:00 |
| openstack | Meeting started Fri Jul 13 08:00:16 2018 UTC and is due to finish in 60 minutes. The chair is d0ugal. Information about MeetBot at http://wiki.debian.org/MeetBot. | 08:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 08:00 |
| *** openstack changes topic to " (Meeting topic: mistral)" | 08:00 | |
| openstack | The meeting name has been set to 'mistral' | 08:00 |
| rakhmerov | d0ugal: that's why the current behaviour should be the default one | 08:00 |
| d0ugal | Morning everyone! It is the Friday office hour slot! | 08:00 |
| rakhmerov | hi! | 08:00 |
| d0ugal | Etherpad: https://etherpad.openstack.org/p/mistral-office-hours | 08:00 |
| d0ugal | ^ apetrich, bobh, mcdoker181818 | 08:00 |
| d0ugal | rakhmerov: indeed, changing the default is bad. | 08:01 |
| d0ugal | rakhmerov: but couldn't the workflow DSL have something to control if it is merging or replacing? | 08:01 |
| rakhmerov | d0ugal: yeah, but maybe just having a reserved env variable is enough | 08:01 |
| rakhmerov | d0ugal: aaah, changing DSL again :) | 08:01 |
| rakhmerov | my guts are against it.. | 08:02 |
| d0ugal | lol | 08:02 |
| rakhmerov | :) | 08:02 |
| d0ugal | I guess we need somebody to come up with a proposal and then we can discuss it there | 08:02 |
| rakhmerov | yep | 08:02 |
| rakhmerov | Vitalii I guess ) | 08:02 |
| d0ugal | Indeed. | 08:02 |
| rakhmerov | btw, I have something to discuss | 08:02 |
| rakhmerov | https://blueprints.launchpad.net/mistral/+spec/mistral-http-action-binary-data | 08:02 |
| rakhmerov | just created | 08:02 |
| rakhmerov | my colleagues would like to have this ability in some form | 08:03 |
| d0ugal | Interesting | 08:03 |
| rakhmerov | but I have doubts about how to implement it properly | 08:03 |
| rakhmerov | yeah | 08:03 |
| rakhmerov | working with a file system doesn't look right to me | 08:04 |
| rakhmerov | so base64 encoding maybe | 08:04 |
| d0ugal | yeah, I would prefer base64 | 08:04 |
| d0ugal | but I guess binary data can sometimes be large? that might be bad for the database? | 08:04 |
| rakhmerov | d0ugal: of course.. | 08:05 |
| rakhmerov | yes | 08:05 |
| d0ugal | Using Mistral with another service in the case might be best? Something like Swift could work | 08:05 |
| rakhmerov | may be setting some limit would be reasonable | 08:05 |
| rakhmerov | or something like this, yes.. | 08:06 |
| rakhmerov | something that's specifically designed for that | 08:06 |
| rakhmerov | for storing large data | 08:06 |
| d0ugal | Is this something you hope to do for rocky? | 08:06 |
| rakhmerov | no | 08:06 |
| rakhmerov | I don't think it's feasible | 08:06 |
| rakhmerov | given all other things that i'm hoping to achieve | 08:06 |
| rakhmerov | next cycle | 08:07 |
| d0ugal | :) | 08:07 |
| d0ugal | but that is good, then we have time to think about it more | 08:08 |
| rakhmerov | yeah | 08:08 |
| rakhmerov | using an external storage seems the best solution for me BUT | 08:08 |
| rakhmerov | we need to have that external storage :) | 08:08 |
| rakhmerov | lots of people (including us) don't care about OpenStack that much | 08:09 |
| d0ugal | Indeed | 08:09 |
| rakhmerov | :) | 08:09 |
| d0ugal | I guess it could be pluggable | 08:09 |
| rakhmerov | yes | 08:09 |
| rakhmerov | through some interface | 08:09 |
| d0ugal | Yeah, and I guess the interface could be very simple | 08:09 |
| rakhmerov | otherwise, our DB would be abused | 08:09 |
| d0ugal | (so easy to implement on other backends) | 08:09 |
| rakhmerov | yep | 08:09 |
| d0ugal | The Mistral DB could even be a backend, but with with warnings and limitations | 08:10 |
| rakhmerov | so it could look like just an additional option in std.http to store content into that storage | 08:10 |
| rakhmerov | and then if it's needed we could have a YAQL/JINJA function to extract it | 08:10 |
| rakhmerov | yeah, agree with your latest thought | 08:10 |
| rakhmerov | yes | 08:10 |
| d0ugal | Sounds good | 08:10 |
| rakhmerov | architecturally it would be more proper solution | 08:11 |
| rakhmerov | I think I'll copy our conversation to the BP description | 08:11 |
| rakhmerov | do you mind? | 08:11 |
| d0ugal | rakhmerov: sure, go for it | 08:12 |
| rakhmerov | ok | 08:12 |
| d0ugal | it is in the meeting log anyway :) | 08:12 |
| rakhmerov | other than that I don't seem to have anything else important | 08:14 |
| pgaxatte | hello | 08:14 |
| d0ugal | I remember a bug/blueprint (but can't find it now) about passing files into workflows. I wonder if we could use the same storage mechanism to solve that | 08:14 |
| d0ugal | pgaxatte: hey | 08:14 |
| d0ugal | Found it. https://bugs.launchpad.net/mistral/+bug/1602599 | 08:14 |
| openstack | Launchpad bug 1602599 in Mistral "Accept multipart file upload as input to Mistral workflows" [Low,Triaged] | 08:14 |
| rakhmerov | d0ugal: yeah-yeah, I think it's all different sides of the same problem | 08:15 |
| rakhmerov | the bottom line is that we need some kind of storage that better works with large data and binary files | 08:16 |
| d0ugal | Yup | 08:16 |
| d0ugal | and then we could reduce some of our db col sizes :) | 08:16 |
| rakhmerov | yes | 08:17 |
| rakhmerov | we've also considered using Glare for some related things too | 08:17 |
| d0ugal | oh yes, I remember that. | 08:17 |
| d0ugal | I have not heard about Glare in some time... | 08:17 |
| rakhmerov | but, as it often happens, we didn't move too far with that | 08:17 |
| d0ugal | :) | 08:17 |
| pgaxatte | for the storage part, could it be done with swift if you have it available on your cloud | 08:17 |
| pgaxatte | ? | 08:18 |
| rakhmerov | d0ugal: yeah, because Mike is now your colleague :) | 08:18 |
| rakhmerov | pgaxatte: Swift, yes, don't see why not | 08:18 |
| *** shardy has joined #openstack-mistral | 08:18 | |
| pgaxatte | rakhmerov: like a swift container as destination of a std.http action | 08:19 |
| d0ugal | oh, I didn't realise he was the main contributor | 08:19 |
| pgaxatte | as destination or source | 08:19 |
| rakhmerov | pgaxatte: we already have swift integration now (swift actions) but the tricky part is that this integration needs to be deeper | 08:19 |
| rakhmerov | imagine std.http action | 08:19 |
| rakhmerov | it needs to be smart enough to understand to store its result into Swift | 08:19 |
| rakhmerov | w/o even using a swift action implicitly | 08:19 |
| rakhmerov | otherwise, we again face the problem of keeping the result temporarily in the context | 08:20 |
| pgaxatte | yes that's tricky :) | 08:20 |
| rakhmerov | which is relational DB | 08:20 |
| rakhmerov | yep | 08:20 |
| d0ugal | A very simple option would be to add specialised options std.http_swift for example | 08:20 |
| rakhmerov | so, seems like just having swift actions doesn't help much | 08:20 |
| pgaxatte | yes d0ugal that's what I was thinking | 08:20 |
| d0ugal | but I would prefer a more general solution I think | 08:21 |
| pgaxatte | d0ugal, rakhmerov: I have a question on another subject: is it possible to use two different executors that would have different network access and different security levels and specify in a single workflow that an action should be run on the more secure executor | 08:26 |
| pgaxatte | like a tag system | 08:26 |
| pgaxatte | no tag = this action can run anywhere | 08:26 |
| pgaxatte | specific tag = run on specific executor | 08:26 |
| d0ugal | pgaxatte: have you seen the target task attribute? | 08:27 |
| d0ugal | I have never actually used this, but it might do some of what you want | 08:27 |
| d0ugal | It isn't exactly the same... | 08:27 |
| d0ugal | https://docs.openstack.org/mistral/latest/user/wf_lang_v2.html#common-task-attributes | 08:27 |
| d0ugal | "target - String parameter. It defines an executor to which task action should be sent to. Target here physically means a name of executors group but task will be run only on one of them. Optional." | 08:28 |
| d0ugal | I can't find any other docs on it | 08:28 |
| pgaxatte | d0ugal: yeah that could be what we're looking for | 08:29 |
| d0ugal | pgaxatte: https://github.com/openstack/mistral/blob/1e577ae73925f7703a5d4e4452855208d0788679/mistral/tests/unit/engine/test_environment.py#L53 | 08:30 |
| d0ugal | It is used in the tests here :) | 08:30 |
| pgaxatte | but how do you group the executors? I don't see the option in conf | 08:30 |
| d0ugal | I was wondering that too! | 08:30 |
| d0ugal | I'm sure rakhmerov knows, but lets see if I can find out before he responds :) | 08:31 |
| pgaxatte | :D | 08:31 |
| rakhmerov | sorry | 08:34 |
| rakhmerov | I was out for a few mins.. | 08:34 |
| rakhmerov | reading.. | 08:34 |
| rakhmerov | pgaxatte: I think it's a problem of our doc again | 08:36 |
| pgaxatte | rakhmerov: yeah this info is hard to find | 08:36 |
| rakhmerov | there needs to be the "host" property in the executor config | 08:36 |
| rakhmerov | if I'm not mistaken.. | 08:36 |
| rakhmerov | but looks like it's not documented, gosh | 08:36 |
| *** bobh has quit IRC | 08:37 | |
| d0ugal | https://github.com/openstack/mistral/blob/master/mistral/config.py#L147-L153 | 08:37 |
| pgaxatte | it's in the config template at least :D | 08:37 |
| pgaxatte | # Name of the executor node. This can be an opaque identifier. It is | 08:37 |
| pgaxatte | # not necessarily a hostname, FQDN, or IP address. (unknown value) | 08:37 |
| pgaxatte | #host = 0.0.0.0 | 08:37 |
| d0ugal | :) | 08:37 |
| rakhmerov | yes, this is confusing | 08:37 |
| rakhmerov | the thing is that several executors can have the same value here | 08:38 |
| rakhmerov | like "executor_group_1" | 08:38 |
| rakhmerov | and then if the "target" is specified, the action will be sent to one of such executors | 08:38 |
| pgaxatte | maybe mentionning that this is related to the target option is a good start ^^ | 08:38 |
| rakhmerov | it's just a matter of building a topic for rabbit | 08:38 |
| d0ugal | and if they have the same value, how are executions chosen? round robin? | 08:38 |
| rakhmerov | pgaxatte: yep | 08:39 |
| rakhmerov | d0ugal: yeah, pretty much | 08:39 |
| rakhmerov | round robin by default, yes, it actually fully depends on the message broker | 08:39 |
| rakhmerov | because on the client side (mistral engine), it's simply a non-empty topic for the messaging system | 08:39 |
| rakhmerov | that's it | 08:40 |
| rakhmerov | then MQ decides what receiver is chosen | 08:40 |
| pgaxatte | rakhmerov: that's awesome because it is as simple as I was hoping :D | 08:40 |
| rakhmerov | yeah :) | 08:41 |
| rakhmerov | so, I have to admin that the doc is fairly bad for this | 08:41 |
| rakhmerov | including the comment in the config | 08:41 |
| rakhmerov | worth filing a bug | 08:41 |
| rakhmerov | .. to admit.. | 08:41 |
| rakhmerov | let me do it | 08:42 |
| d0ugal | Thanks :) | 08:43 |
| pgaxatte | rakhmerov: are there any blueprints to reorganize the whole documentation? | 08:44 |
| d0ugal | pgaxatte: there is at least one :) | 08:45 |
| rakhmerov | https://bugs.launchpad.net/mistral/+bug/1781556 | 08:45 |
| openstack | Launchpad bug 1781556 in Mistral "Improve the doc for "target" task parameter" [Undecided,New] | 08:45 |
| rakhmerov | pgaxatte: yeah, we keep improving the doc step by step but we've also discussed that we need to reorganize it | 08:46 |
| rakhmerov | d0ugal: not sure where this BP is.. I remember someone filed it recently (a month ago may be) | 08:46 |
| pgaxatte | I like how the heat documentation is structured: operating, using, developing | 08:47 |
| d0ugal | pgaxatte: yeah, we have discussed something like that several times. | 08:47 |
| d0ugal | The problem is finding somebody that knows Mistral well enough and has the time to write lots of documentation | 08:47 |
| d0ugal | so we end up stuck every time | 08:47 |
| pgaxatte | d0ugal: yeah that's hard to find | 08:48 |
| rakhmerov | there it is: https://blueprints.launchpad.net/mistral/+spec/mistral-new-docs | 08:48 |
| rakhmerov | there's even a spec for this | 08:49 |
| rakhmerov | see a link in the BP | 08:49 |
| d0ugal | Yup, the spec needs lots of discussion however | 08:49 |
| rakhmerov | yes | 08:49 |
| rakhmerov | I know I could do that well but I have no idea when I'll be able to have enough focus on that | 08:50 |
| rakhmerov | this work requires a lot of focus during a month-two | 08:50 |
| d0ugal | #endmeeting | 09:02 |
| *** openstack changes topic to "Mistral the Workflow Service for OpenStack. https://docs.openstack.org/mistral/latest/" | 09:02 | |
| openstack | Meeting ended Fri Jul 13 09:02:04 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 09:02 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/mistral/2018/mistral.2018-07-13-08.00.html | 09:02 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/mistral/2018/mistral.2018-07-13-08.00.txt | 09:02 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/mistral/2018/mistral.2018-07-13-08.00.log.html | 09:02 |
| rakhmerov | d0ugal: as far as https://review.openstack.org/#/c/581059/2/mistral/actions/std_actions.py | 09:27 |
| rakhmerov | not sure I understand actually.. | 09:27 |
| rakhmerov | the idea is that this is an async action and the return value of its "run" method is not important | 09:27 |
| rakhmerov | d0ugal: can you elaborate? | 09:27 |
| d0ugal | rakhmerov: hmm, yeah | 09:28 |
| d0ugal | I think I was confused | 09:28 |
| rakhmerov | maybe it's just not enough.. | 09:28 |
| openstackgerrit | Hardik Jasani proposed openstack/mistral master: [WIP] Add namespace support for workbooks and actions https://review.openstack.org/582482 | 09:29 |
| d0ugal | I seen the PR on github and thought it made sense | 09:29 |
| rakhmerov | just thinking about the case when if we didn't successfully reach the host | 09:29 |
| rakhmerov | in the first place | 09:29 |
| d0ugal | right | 09:29 |
| d0ugal | that is it | 09:29 |
| rakhmerov | so that it doesn't even make sense to wait for a result | 09:29 |
| rakhmerov | ok, that's right but | 09:29 |
| d0ugal | but then don't we have an issue because it will return a result and also has is_sync = False | 09:29 |
| rakhmerov | but I'm still not sure if this return value will be processed by Mistral somehow | 09:29 |
| d0ugal | will the result be used? | 09:29 |
| d0ugal | RIght | 09:30 |
| d0ugal | hmm | 09:30 |
| rakhmerov | I'm not sure | 09:30 |
| rakhmerov | most likely not | 09:30 |
| rakhmerov | let me look at the code.. | 09:30 |
| rakhmerov | executor | 09:30 |
| d0ugal | I'll mark it as -1 Workflow for now, sounds like I need to do some testing. | 09:30 |
| rakhmerov | 100% | 09:31 |
| rakhmerov | yes, look at this: https://github.com/openstack/mistral/blob/master/mistral/executors/default_executor.py#L142 | 09:33 |
| rakhmerov | d0ugal: ^ | 09:33 |
| rakhmerov | the result is just ignored if it's an async action | 09:33 |
| rakhmerov | Ooh, no! | 09:34 |
| rakhmerov | action.is_sync() or result.is_error() | 09:34 |
| rakhmerov | so if this is error then it's not ignored.. | 09:35 |
| rakhmerov | seems like | 09:35 |
| rakhmerov | yeah, d0ugal, it all should work | 09:37 |
| rakhmerov | the HTTP action (base class) has: | 09:38 |
| rakhmerov | if resp.status_code not in range(200, 307): | 09:38 |
| rakhmerov | return actions.Result(error=_result) | 09:38 |
| rakhmerov | so it returns an error result in this case | 09:38 |
| rakhmerov | false alarm | 09:38 |
| rakhmerov | but more testing would still be good | 09:38 |
| d0ugal | Agreed, | 09:42 |
| d0ugal | I'll do some manual testing and add a test case | 09:42 |
| rakhmerov | 👌 | 10:13 |
| hardikjasani | So I faced this issue this morning https://bugs.launchpad.net/mistral/+bug/1781548 | 10:15 |
| openstack | Launchpad bug 1781548 in Mistral "std.ssh doesn't work with python3" [Undecided,New] | 10:15 |
| hardikjasani | Probably missed in the office-hours | 10:16 |
| rakhmerov | ooh | 10:16 |
| rakhmerov | interesting | 10:16 |
| d0ugal | hardikjasani: we didn't get to bugs this time :) | 10:16 |
| hardikjasani | d0ugal: not blocking but important :) | 10:19 |
| d0ugal | hardikjasani: agreed. Hopefully it should be fairly easy to fix too | 10:28 |
| d0ugal | rakhmerov: I have a simple Mistral setup on my laptop with docker, and it is mostly working. workflow-list works fine but execution-create and run-action etc. just hang and I don't see anything in the logs | 10:39 |
| d0ugal | rakhmerov: does that mean it is failing to connect to rabbit? not sure how to debug it. | 10:39 |
| d0ugal | This is the only line I get in the logs: http://paste.openstack.org/show/725809/ | 10:40 |
| *** shardy is now known as shardy_mtg | 12:59 | |
| *** shardy_mtg is now known as shardy | 13:26 | |
| *** rbrady has quit IRC | 14:05 | |
| *** shardy has quit IRC | 14:46 | |
| *** EmilienM is now known as EvilienM | 15:18 | |
| *** hardikjasani has quit IRC | 15:26 | |
| *** bobh has joined #openstack-mistral | 15:47 | |
| *** harlowja has joined #openstack-mistral | 16:08 | |
| *** harlowja has quit IRC | 16:10 | |
| *** kimamisa has quit IRC | 16:30 | |
| *** gkadam has quit IRC | 16:55 | |
| *** bobh has quit IRC | 16:58 | |
| *** bobh has joined #openstack-mistral | 17:32 | |
| *** bobh has quit IRC | 17:42 | |
| *** jistr has quit IRC | 18:11 | |
| *** jistr has joined #openstack-mistral | 18:11 | |
| *** bobh has joined #openstack-mistral | 18:12 | |
| *** bobh has quit IRC | 18:16 | |
| *** kimamisa has joined #openstack-mistral | 18:17 | |
| *** bobh has joined #openstack-mistral | 18:30 | |
| *** kimamisa has quit IRC | 18:41 | |
| *** kimamisa has joined #openstack-mistral | 18:44 | |
| *** gkadam has joined #openstack-mistral | 18:50 | |
| *** gkadam has quit IRC | 19:17 | |
| *** kimamisa has quit IRC | 19:34 | |
| *** mmethot_ has quit IRC | 19:50 | |
| *** kimamisa has joined #openstack-mistral | 19:50 | |
| *** mmethot has joined #openstack-mistral | 19:51 | |
| *** kimamisa_ has joined #openstack-mistral | 19:56 | |
| *** kimamisa has quit IRC | 19:56 | |
| *** kimamisa_ has quit IRC | 20:04 | |
| *** kimamisa has joined #openstack-mistral | 20:05 | |
| *** kimamisa has quit IRC | 20:07 | |
| *** kimamisa has joined #openstack-mistral | 20:11 | |
| *** kimamisa has quit IRC | 20:18 | |
| *** kimamisa has joined #openstack-mistral | 20:40 | |
| *** kimamisa has quit IRC | 21:15 | |
| *** nguyenhai93 has joined #openstack-mistral | 21:26 | |
| *** nguyenhai_ has quit IRC | 21:29 | |
| *** bobh has quit IRC | 22:14 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!