| *** jamielennox|away is now known as jamielennox | 00:05 | |
| *** bobh has joined #openstack-mistral | 00:41 | |
| *** bobh has quit IRC | 00:46 | |
| *** gongysh has joined #openstack-mistral | 03:05 | |
| *** bobh has joined #openstack-mistral | 03:44 | |
| *** bobh has quit IRC | 03:49 | |
| rakhmerov | thrash|g0ne: yes, you could also use your previous form of expression but with "let" | 04:05 |
|---|---|---|
| rakhmerov | let(stack => $.stack) -> <your expression> | 04:07 |
| rakhmerov | it's because $ doesn't refer to a root context any more inside "where" | 04:08 |
| rakhmerov | you can also do a similar thing with the entire root context: | 04:08 |
| rakhmerov | let(root => $) -> <expression> | 04:08 |
| *** sharatss has joined #openstack-mistral | 04:21 | |
| *** gongysh has quit IRC | 04:36 | |
| *** bobh has joined #openstack-mistral | 04:45 | |
| *** bobh has quit IRC | 04:49 | |
| rakhmerov | d0ugal: hi Dougal, I left a comment in https://review.openstack.org/#/c/423299 | 04:58 |
| rakhmerov | pls take a look | 04:58 |
| rakhmerov | can be addressed separately but I'd like to ask you to do it one way or another | 04:58 |
| *** gongysh has joined #openstack-mistral | 05:16 | |
| *** gongysh has quit IRC | 05:32 | |
| *** sharatss has quit IRC | 06:00 | |
| *** bobh has joined #openstack-mistral | 06:01 | |
| *** bobh has quit IRC | 06:15 | |
| *** sharatss has joined #openstack-mistral | 06:20 | |
| *** sharatss has quit IRC | 07:33 | |
| *** sharatss has joined #openstack-mistral | 07:46 | |
| openstackgerrit | Merged openstack/mistral: Allow hyphens in Workflow and ad-hoc action names https://review.openstack.org/423299 | 07:53 |
| d0ugal | rakhmerov: underscores are already included :) | 07:55 |
| rakhmerov | how come? | 07:56 |
| rakhmerov | it's not a part of that regex, right? | 07:56 |
| d0ugal | It is actually | 07:56 |
| d0ugal | \w includes _ | 07:56 |
| d0ugal | Let me find the relevant bit in the docs... | 07:56 |
| d0ugal | "this is equivalent to the set [a-zA-Z0-9_]" | 07:56 |
| d0ugal | https://docs.python.org/2/library/re.html | 07:56 |
| d0ugal | rakhmerov: there is even one in the tests that I added (when I added it originally) | 07:58 |
| rakhmerov | ooh | 07:58 |
| rakhmerov | yeah | 07:58 |
| rakhmerov | I forgot | 07:58 |
| d0ugal | https://github.com/openstack/mistral/blob/master/mistral/tests/resources/workbook/v2/workbook_schema_test.yaml#L42 | 07:58 |
| rakhmerov | ok then | 07:58 |
| d0ugal | but the comment about allowing "-" is valid, maybe I should make it a bit better. | 07:59 |
| d0ugal | we don't want to just allow - or _, but we currently do | 07:59 |
| d0ugal | so a workflow could be named "_-_" lol | 07:59 |
| rakhmerov | I knew about digits and letters but forgot about underscore | 07:59 |
| rakhmerov | :))))) | 08:00 |
| rakhmerov | it's actually pretty cool | 08:00 |
| d0ugal | lol | 08:00 |
| d0ugal | I guess if people are doing that, then we should let them - it would be hard to do it by accident. | 08:00 |
| *** jtomasek has quit IRC | 08:15 | |
| *** jtomasek has joined #openstack-mistral | 08:17 | |
| *** jpich has joined #openstack-mistral | 08:52 | |
| *** shardy has joined #openstack-mistral | 09:02 | |
| openstackgerrit | Sharat Sharma proposed openstack/mistral: Add script for unit test coverage job https://review.openstack.org/417881 | 09:09 |
| openstackgerrit | Sharat Sharma proposed openstack/python-mistralclient: Add script for unit test coverage job https://review.openstack.org/417902 | 09:12 |
| *** sharat has joined #openstack-mistral | 09:57 | |
| rakhmerov | sharatss: hi | 10:20 |
| rakhmerov | can you please test Congress actions? | 10:20 |
| rakhmerov | asap | 10:20 |
| rakhmerov | I'd like to merge your patch | 10:20 |
| sharatss | rakhmerov: yes. i am on a holiday | 10:20 |
| sharatss | rakhmerov: will test it asap and let you know | 10:21 |
| d0ugal | so, I am looking at the Glare email thread... | 10:34 |
| d0ugal | "Mistral will store its workflows in Glare" - does that mean Mistral will depend on Glare? | 10:34 |
| rakhmerov | d0ugal: yes, I guess so | 10:35 |
| rakhmerov | on glare client | 10:35 |
| d0ugal | rakhmerov: right, but it will be optional? | 10:35 |
| rakhmerov | sure | 10:36 |
| d0ugal | cool | 10:36 |
| d0ugal | Just wondering about non-OpenStack users. | 10:36 |
| rakhmerov | IMO it won't be necessary to store WFs in Glare | 10:36 |
| rakhmerov | just one of the options | 10:36 |
| rakhmerov | yes | 10:36 |
| d0ugal | makes sense | 10:36 |
| d0ugal | thanks | 10:36 |
| d0ugal | it would also make things more complicated for us too :) | 10:36 |
| rakhmerov | the bigger thing IMO is that we should teach Mistral to have references to binaries in workflows | 10:37 |
| rakhmerov | somehow | 10:37 |
| d0ugal | references to binaries? | 10:37 |
| rakhmerov | yes | 10:37 |
| d0ugal | I don't understand what that is. | 10:37 |
| rakhmerov | sec | 10:37 |
| rakhmerov | think of the following (typical) example: I want to build a workflow in which I'm planning to use a number of shell scripts (or other types of scripts), I want to use some public keys, I want to use something else that is not really good to have hardcoded in my workflow | 10:40 |
| rakhmerov | even when we use std.ssh now we have to write SH scripts right in workflows | 10:40 |
| rakhmerov | so it would be cool to have some sort of an external storage where we could store all necessary resources we're going to use | 10:41 |
| rakhmerov | In fact, this is a replacement of a regular FS which we can't use normally | 10:42 |
| d0ugal | right | 10:42 |
| rakhmerov | since Mistral is a RESTful service | 10:42 |
| rakhmerov | yes, so that's the idea | 10:42 |
| d0ugal | so, you could maybe have an action that uses a shell script stored in Glare or Swift or .. | 10:42 |
| d0ugal | or any other executable | 10:43 |
| rakhmerov | you got it | 10:43 |
| rakhmerov | yes | 10:43 |
| d0ugal | yeah, I could see that being useful. | 10:43 |
| rakhmerov | it's basically a better separation of concerns | 10:43 |
| rakhmerov | workflow and resources stored separately | 10:43 |
| d0ugal | We have this problem in a way with TripleO, the solution is kind of ugly :) | 10:43 |
| rakhmerov | yes, a lot of middlemen | 10:44 |
| rakhmerov | :) | 10:44 |
| rakhmerov | with bent sticks :) | 10:44 |
| d0ugal | haha | 10:44 |
| d0ugal | we give the Mistral user extra permissions :-/ https://github.com/openstack/tripleo-common/blob/master/sudoers | 10:44 |
| d0ugal | so that it can do more | 10:44 |
| rakhmerov | yeah, I see | 10:45 |
| d0ugal | anyway, maybe this is another good PTG topic | 10:45 |
| rakhmerov | let me include it | 10:45 |
| rakhmerov | btw, Mike Fedosin will join us there | 10:45 |
| rakhmerov | we are going to discuss it anyway | 10:45 |
| d0ugal | great | 10:46 |
| rakhmerov | done, included | 10:48 |
| rakhmerov | d0ugal: I don't remember a real status of https://bugs.launchpad.net/mistral/+bug/1645295 | 10:48 |
| openstack | Launchpad bug 1645295 in Mistral "Custom actions can cause database errors if they pass the result class objects" [Critical,In progress] - Assigned to Dougal Matthews (d0ugal) | 10:48 |
| rakhmerov | can you please remind me? | 10:48 |
| rakhmerov | I'm going over our tickets for Ocata-3 | 10:49 |
| d0ugal | rakhmerov: I think you fixed it, in another related bug but we wanted to add a test for this particular case. | 10:49 |
| d0ugal | I keep forgetting about it, sorry. | 10:49 |
| rakhmerov | can you do it before Thu? | 10:50 |
| d0ugal | yes, I think so | 10:50 |
| rakhmerov | and can we close this exact ticket? | 10:50 |
| d0ugal | Yup, sounds good. | 10:51 |
| d0ugal | I dropped the priority as I don't think it should still be "critical" | 10:52 |
| * d0ugal will bbiab | 10:53 | |
| openstackgerrit | Sharat Sharma proposed openstack/mistral: Improve the CONTRIBUTING.rst https://review.openstack.org/424058 | 11:05 |
| *** sharatss_ has joined #openstack-mistral | 11:45 | |
| *** sharat has quit IRC | 11:45 | |
| *** sharatss has quit IRC | 11:45 | |
| *** sharatss has joined #openstack-mistral | 11:46 | |
| *** sharatss_ has quit IRC | 11:47 | |
| *** sharatss has quit IRC | 11:47 | |
| *** sharatss_ has joined #openstack-mistral | 11:47 | |
| *** sharatss has joined #openstack-mistral | 11:47 | |
| *** shardy is now known as shardy_lunch | 12:22 | |
| *** thrash|g0ne is now known as thrash | 12:37 | |
| *** catintheroof has joined #openstack-mistral | 12:45 | |
| *** catintheroof has quit IRC | 12:50 | |
| *** catintheroof has joined #openstack-mistral | 12:58 | |
| *** dprince has joined #openstack-mistral | 13:00 | |
| *** bobh has joined #openstack-mistral | 13:00 | |
| *** hrybacki|sick is now known as hrybacki | 13:04 | |
| *** sharatss_ has quit IRC | 13:20 | |
| *** bobh has quit IRC | 13:27 | |
| openstackgerrit | Dougal Matthews proposed openstack/mistral: Don't create actions with empty arg_lists https://review.openstack.org/412433 | 13:28 |
| *** shardy_lunch is now known as shardy | 13:28 | |
| openstackgerrit | Dougal Matthews proposed openstack/mistral: Don't create actions with empty arg_lists https://review.openstack.org/412433 | 13:41 |
| *** catinthe_ has joined #openstack-mistral | 14:29 | |
| *** rbrady is now known as rbrady-mtg | 14:30 | |
| *** catintheroof has quit IRC | 14:32 | |
| *** rbrady-mtg is now known as rbrady | 14:45 | |
| *** tuan_ has joined #openstack-mistral | 14:45 | |
| tuan_ | rakhmerov: Hi renat | 14:46 |
| tuan_ | i have attached the summary for the test of YAQL in the mailing loop | 14:46 |
| tuan_ | please take a look to that | 14:46 |
| openstackgerrit | Dawid Deja proposed openstack/mistral: Refactor rpc configuration loading https://review.openstack.org/424171 | 14:49 |
| *** jaosorior has joined #openstack-mistral | 14:56 | |
| *** bobh has joined #openstack-mistral | 15:07 | |
| *** bobh has quit IRC | 15:12 | |
| tuan_ | hey guys | 15:20 |
| tuan_ | someone online? | 15:20 |
| d0ugal | tuan_: Hi | 15:28 |
| tuan_ | Hi | 15:30 |
| tuan_ | thanks for being online | 15:30 |
| tuan_ | :D | 15:30 |
| tuan_ | may i have you some minutes | 15:30 |
| d0ugal | sure, what's up? | 15:30 |
| tuan_ | yeap | 15:32 |
| tuan_ | it seems like with the big data of executions | 15:32 |
| tuan_ | more tan 10GB | 15:32 |
| tuan_ | when we try do delete the execution based on expiration_time | 15:33 |
| tuan_ | as i saw in the code that we open only one transaction | 15:33 |
| tuan_ | and then when we delete the big data of execution based on expiration_time | 15:33 |
| tuan_ | we have the error of DB | 15:33 |
| tuan_ | it seems that only one sqlalchemy transaction opened | 15:34 |
| tuan_ | it is not enough for the big data | 15:34 |
| tuan_ | may be the timeout of sqlalchemy | 15:34 |
| tuan_ | then we have to list out all the execution | 15:35 |
| tuan_ | and then for each of them we open one transaction to delete | 15:35 |
| tuan_ | but you know that it does not seem good with sqlalchemy | 15:35 |
| tuan_ | do you guys have this use case before? | 15:36 |
| d0ugal | I've never seen anything like that | 15:38 |
| d0ugal | but we don't keep the executions for long | 15:38 |
| tuan_ | yeap | 15:38 |
| tuan_ | however in our usecase | 15:38 |
| tuan_ | the list of executions is thousands | 15:38 |
| tuan_ | and each of them contains quite a lot of task | 15:39 |
| tuan_ | therefore it can issue the problem of performance of sql | 15:39 |
| tuan_ | therefore i would like to ask you guys this problem | 15:39 |
| tuan_ | and if it really affects to the db query with large size of data | 15:40 |
| tuan_ | may be we have to limit the number of query of sqlalchemy | 15:40 |
| tuan_ | in order to ensure the normal behaviour with the large size of db | 15:41 |
| tuan_ | ?? | 15:41 |
| tuan_ | how do you think about this | 15:41 |
| *** shardy has quit IRC | 15:41 | |
| *** shardy has joined #openstack-mistral | 15:42 | |
| rakhmerov | tuan_: can you just tune expiration policy so that it triggers more frequently? | 16:01 |
| rakhmerov | that way you will keep these transactions smaller | 16:01 |
| rakhmerov | we're starting a meeting at #openstack-meeting | 16:01 |
| tuan_ | oh i am sorry | 16:02 |
| tuan_ | i will switch to that channel | 16:02 |
| rakhmerov | tuan_: I guess the problem really exist, you're right | 16:03 |
| rakhmerov | let's discuss it tomorrow | 16:03 |
| rakhmerov | I think that nobody has hit this issue with large amount of data yet | 16:03 |
| tuan_ | yeap | 16:04 |
| tuan_ | if there is no one in openstack meeting channel | 16:04 |
| tuan_ | we can delay it until tmorrow | 16:04 |
| tuan_ | so let's delay it | 16:04 |
| tuan_ | :d | 16:04 |
| tuan_ | meanwhile i am trying to find more information about performance of sqlalchemy | 16:05 |
| *** bobh has joined #openstack-mistral | 16:08 | |
| *** bobh has quit IRC | 16:13 | |
| d0ugal | sqlalchemy performs pretty well, the problem is usually how you use it :) | 16:13 |
| *** bobh has joined #openstack-mistral | 16:14 | |
| *** bobh has quit IRC | 16:21 | |
| *** mgershen1 is now known as mgershen | 16:22 | |
| rakhmerov | d0ugal: right | 16:22 |
| tuan_ | d0ugal: I agree, i have not had this experience before | 16:51 |
| tuan_ | but we still have to find out the solution | 16:51 |
| tuan_ | however, as i worked in Nova and Neutron before | 16:52 |
| tuan_ | in Nova they use tpool from evenlet to avoid the hanging db connection | 16:52 |
| *** bobh has joined #openstack-mistral | 16:53 | |
| tuan_ | i do not know that if we have a large size of db | 16:53 |
| tuan_ | time to query is big, therefore another connection has to wait for previous one | 16:53 |
| tuan_ | it may hang till it reaches the timeout | 16:53 |
| tuan_ | hmm, just only my guest | 16:54 |
| *** rbrady is now known as rbrady-food | 17:10 | |
| *** tuan_ has quit IRC | 17:23 | |
| *** jpich has quit IRC | 17:38 | |
| *** bobh has quit IRC | 17:43 | |
| *** bobh has joined #openstack-mistral | 17:55 | |
| *** mgershen has quit IRC | 17:56 | |
| *** bobh has quit IRC | 17:59 | |
| *** rbrady-food is now known as rbrady | 18:29 | |
| *** catintheroof has joined #openstack-mistral | 18:45 | |
| *** catinthe_ has quit IRC | 18:47 | |
| *** shardy is now known as shardy_afk | 18:55 | |
| *** bobh has joined #openstack-mistral | 18:57 | |
| *** catinthe_ has joined #openstack-mistral | 19:03 | |
| *** catintheroof has quit IRC | 19:06 | |
| *** jaosorior has quit IRC | 19:50 | |
| *** dprince has quit IRC | 20:10 | |
| kong | rakhmerov: re your example 'I want to build a workflow in which I'm planning to use a number of shell scripts ', that was what I tried to implement in https://review.openstack.org/#/c/373606/ | 21:20 |
| kong | rakhmerov: even we could store scripts or code in a third party place, we still need to find an elegant way to run scripts safely | 21:21 |
| kong | that's why i think FaaS can bring much additional benefit to Mistral | 21:21 |
| *** shardy_afk has quit IRC | 21:27 | |
| *** toure is now known as toure|biab | 21:32 | |
| *** harlowja has quit IRC | 22:03 | |
| *** dprince has joined #openstack-mistral | 22:07 | |
| *** dprince has quit IRC | 22:15 | |
| *** bobh has quit IRC | 23:01 | |
| *** catinthe_ has quit IRC | 23:23 | |
| *** harlowja has joined #openstack-mistral | 23:34 | |
| *** harlowja has quit IRC | 23:35 | |
| *** harlowja has joined #openstack-mistral | 23:48 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!