| *** tonytan4ever has quit IRC | 00:41 | |
| *** toddjohn has joined #openstack-mistral | 00:42 | |
| *** enykeev has quit IRC | 00:49 | |
| *** enykeev has joined #openstack-mistral | 00:50 | |
| *** Ephur has quit IRC | 00:55 | |
| *** cheneydc has joined #openstack-mistral | 01:06 | |
| *** bobh has joined #openstack-mistral | 01:14 | |
| *** chlong has joined #openstack-mistral | 01:27 | |
| *** toddjohn has quit IRC | 01:29 | |
| *** tonytan4ever has joined #openstack-mistral | 01:42 | |
| *** tonytan4ever has quit IRC | 01:47 | |
| *** bobh has quit IRC | 02:43 | |
| *** Ephur has joined #openstack-mistral | 02:55 | |
| *** cheneydc has quit IRC | 03:16 | |
| *** bobh has joined #openstack-mistral | 03:47 | |
| *** Ephur has quit IRC | 03:59 | |
| *** Ephur has joined #openstack-mistral | 03:59 | |
| *** Ephur has quit IRC | 04:00 | |
| *** bobh has quit IRC | 04:03 | |
| openstackgerrit | Renat Akhmerov proposed openstack/mistral: Towards non-locking model: make 'with-items' work w/o locks https://review.openstack.org/352386 | 04:20 |
|---|---|---|
| *** jaosorior has joined #openstack-mistral | 05:21 | |
| *** jtomasek_ has joined #openstack-mistral | 05:26 | |
| *** jtomasek_ is now known as jtomasek | 05:37 | |
| *** janki has joined #openstack-mistral | 05:42 | |
| *** brian_price has quit IRC | 05:51 | |
| *** Ravikiran_K has joined #openstack-mistral | 06:05 | |
| *** rbrady has quit IRC | 06:07 | |
| *** chlong is now known as chlong|mtg | 06:32 | |
| *** mgershen has joined #openstack-mistral | 07:06 | |
| *** jpich has joined #openstack-mistral | 07:29 | |
| *** sshnaidm|afk has quit IRC | 07:32 | |
| *** rbrady has joined #openstack-mistral | 07:34 | |
| *** shardy has joined #openstack-mistral | 07:54 | |
| *** sshnaidm|afk has joined #openstack-mistral | 08:11 | |
| *** sshnaidm|afk is now known as sshnaidm | 08:11 | |
| *** jaosorior is now known as jaosorior_brb | 08:49 | |
| *** brunograz has joined #openstack-mistral | 08:51 | |
| rakhmerov | hparekh: hi, here? | 09:08 |
| hparekh | rakhmerov, yeah hi | 09:09 |
| rakhmerov | I have a quick question about WSGI server that you recently added | 09:09 |
| rakhmerov | for API | 09:09 |
| hparekh | yes | 09:09 |
| rakhmerov | so, do you know exactly how it works? It creates multiple independent Python processes, right? | 09:10 |
| *** chlong|mtg has quit IRC | 09:10 | |
| rakhmerov | as far as I can tell by seeing "ps ax | grep mistral | 09:10 |
| rakhmerov | result | 09:10 |
| hparekh | yes | 09:10 |
| hparekh | depends on cpu cores | 09:10 |
| rakhmerov | yeah | 09:10 |
| rakhmerov | I'm observing a strange thing actually | 09:11 |
| hparekh | what is it ? | 09:11 |
| rakhmerov | I see that some objects are shared between those processes | 09:11 |
| rakhmerov | at least they have same ids | 09:11 |
| rakhmerov | I use id() function to see their identity | 09:11 |
| rakhmerov | but their state is different in every process, of course | 09:12 |
| rakhmerov | how is that possible? | 09:12 |
| rakhmerov | is it a normal Python behavior or these API processes are related somehow? | 09:12 |
| rakhmerov | just trying to understand why this is happenning... | 09:12 |
| hparekh | rakhmerov, not sure what is happening here | 09:19 |
| rakhmerov | ok | 09:19 |
| rakhmerov | ddeja: hi | 09:25 |
| rakhmerov | you here? | 09:25 |
| ddeja | rakhmerov: hi | 09:25 |
| rakhmerov | is there a way in o.m to send a message in broadcast mode? | 09:25 |
| ddeja | like, to everyone | 09:26 |
| ddeja | ? | 09:26 |
| rakhmerov | so that it is received by all listeners | 09:26 |
| rakhmerov | yes | 09:26 |
| rakhmerov | I need it badly :) | 09:26 |
| ddeja | not 100% sure, but it should be possible | 09:26 |
| rakhmerov | ok | 09:26 |
| rakhmerov | do you know where to look? | 09:27 |
| ddeja | let me look into code | 09:27 |
| openstackgerrit | Merged openstack/mistral: Towards non-locking model: adapt 'join' tasks to work w/o locks https://review.openstack.org/351727 | 09:27 |
| rakhmerov | ok, thanks | 09:28 |
| openstackgerrit | Merged openstack/python-mistralclient: Add error message when OS_USERNAME or OS_PASSWORD not provided https://review.openstack.org/348142 | 09:28 |
| rakhmerov | I realized that I made a serious mistake with spec caching | 09:28 |
| rakhmerov | what to fix it asap | 09:28 |
| ddeja | rakhmerov: look at this http://docs.openstack.org/developer/oslo.messaging/target.html | 09:33 |
| ddeja | this http://docs.openstack.org/developer/oslo.messaging/server.html | 09:33 |
| ddeja | and this http://docs.openstack.org/developer/oslo.messaging/rpcclient.html | 09:33 |
| ddeja | basicalyy, we want servers to listne to targets that only have topic specified | 09:33 |
| rakhmerov | yes | 09:34 |
| ddeja | also, target have to set 'fanaout' flag to true | 09:34 |
| rakhmerov | ooh, ok | 09:34 |
| rakhmerov | thanks a lot | 09:34 |
| rakhmerov | will be thinking further... :) | 09:34 |
| ddeja | no problem, just need a minute to find out how oslo hanlde it, I'm more pure rabbit guy ;) | 09:35 |
| rakhmerov | yeah | 09:35 |
| rakhmerov | I'm also wondering if it'll be possible to implement a similar thing in pure rabbit | 09:35 |
| rakhmerov | I guess it's even simpler | 09:35 |
| ddeja | sure | 09:35 |
| rakhmerov | btw, this is another task for you :) | 09:35 |
| ddeja | it is the same pattern | 09:35 |
| rakhmerov | to have such method on RPCClient :) | 09:36 |
| ddeja | just set fanaout==true nd it is all working | 09:36 |
| rakhmerov | it can be called just broadcast() | 09:36 |
| rakhmerov | I guess I'll fix my problem the other way because it's urgent but moving forward we can add this method on RPCClient | 09:37 |
| rakhmerov | what do you think? | 09:37 |
| ddeja | give me a minute, I'm looking into code base... | 09:37 |
| rakhmerov | sure | 09:38 |
| rakhmerov | can you create a BP pls? | 09:38 |
| ddeja | rakhmerov: I'm not sure if we want to procede this way | 09:39 |
| ddeja | basically, it is possible right now to do broadcast messages using komub driver | 09:40 |
| rakhmerov | great ) | 09:40 |
| rakhmerov | you're not sure about what exactly? | 09:40 |
| ddeja | well, OK, it is possible in the code base, it is not exposed to user | 09:41 |
| ddeja | I'm not sure if we want to add new method 'broadcast' | 09:41 |
| ddeja | becouse if the messages goes directly to one consumer or to all consumers depends on pre-configuration of the connection objects | 09:42 |
| *** jaosorior_brb is now known as jaosorior | 09:42 | |
| ddeja | we would rather have to connection objects stored in the memory and use the one that fit the case (direct message vs fanout message) | 09:43 |
| ddeja | I can start a blueprint for this | 09:44 |
| rakhmerov | oh, I see | 09:45 |
| rakhmerov | ok | 09:45 |
| *** chlong|mtg has joined #openstack-mistral | 09:47 | |
| *** sshnaidm is now known as sshnaidm|lnch | 10:39 | |
| *** Ravikiran_K has quit IRC | 10:59 | |
| *** sshnaidm|lnch is now known as sshnaidm | 11:25 | |
| *** jaosorior has quit IRC | 11:27 | |
| *** jaosorior has joined #openstack-mistral | 11:28 | |
| ddeja | rakhmerov: Hi, I was reviewing your latest change that makes 'with-items' working without locks | 11:32 |
| rakhmerov | ok | 11:32 |
| ddeja | unfortunately, it looks like it breaks things | 11:32 |
| ddeja | I've pasted my tested workflow - it goes into success, but task2 and task3 never run | 11:33 |
| rakhmerov | hm.. | 11:34 |
| rakhmerov | does it work with the previous version? | 11:34 |
| ddeja | yes | 11:34 |
| ddeja | well, let me double check | 11:34 |
| rakhmerov | yes, please | 11:35 |
| ddeja | becouse it might be previous patch that broke things... | 11:35 |
| rakhmerov | I wonder if a problem was introduced by the patch | 11:35 |
| rakhmerov | it might have been 'join' patch | 11:35 |
| rakhmerov | that is already merged btw | 11:35 |
| ddeja | Oh, it started working now... | 11:37 |
| ddeja | don't know what happend | 11:38 |
| ddeja | nevermind | 11:38 |
| rakhmerov | please try several times | 11:38 |
| rakhmerov | it might some concurrent problem that still exists | 11:38 |
| rakhmerov | it's possible | 11:38 |
| rakhmerov | try 10 times :) | 11:38 |
| ddeja | it looks like it working now, I might have my master branch in some older state | 11:39 |
| ddeja | my bad, sorry | 11:39 |
| ddeja | +2, +W | 11:41 |
| rakhmerov | :) | 11:42 |
| rakhmerov | thanks a lot!! | 11:42 |
| *** dprince has joined #openstack-mistral | 11:43 | |
| rakhmerov | ddeja: I really wanna ask you review something that I'm about to merge, it will fix a SUPER nasty bug | 11:43 |
| rakhmerov | which was introduced by me | 11:43 |
| ddeja | ok | 11:43 |
| rakhmerov | (I hope I won't get too many rotten tomatos.. ) | 11:43 |
| ddeja | rakhmerov: is this already in review? | 11:45 |
| rakhmerov | no, soon | 11:45 |
| rakhmerov | finishing it up.. | 11:45 |
| ddeja | ok | 11:45 |
| ddeja | rakhmerov: the blueprint you asked https://blueprints.launchpad.net/mistral/+spec/mistral-broadcast-messages | 11:48 |
| rakhmerov | ok, great! | 11:48 |
| *** dprince has quit IRC | 11:56 | |
| *** dprince has joined #openstack-mistral | 11:56 | |
| *** sshnaidm is now known as sshnaidm|afk | 11:56 | |
| openstackgerrit | Renat Akhmerov proposed openstack/mistral: Fix specification caching mechanism https://review.openstack.org/352869 | 12:02 |
| rakhmerov | ddeja: https://review.openstack.org/#/c/352869/ | 12:02 |
| rakhmerov | need to wait if the gates pass | 12:03 |
| rakhmerov | but you can take a look at the problem and see if the solution is valid conceptually | 12:03 |
| * ddeja is looking | 12:03 | |
| *** sshnaidm|afk is now known as sshnaidm | 12:33 | |
| *** sshnaidm has quit IRC | 12:35 | |
| openstackgerrit | Renat Akhmerov proposed openstack/mistral: Fix specification caching mechanism https://review.openstack.org/352869 | 12:37 |
| *** bobh has joined #openstack-mistral | 12:39 | |
| *** toddjohn has joined #openstack-mistral | 12:40 | |
| *** toddjohn has quit IRC | 12:45 | |
| *** catintheroof has joined #openstack-mistral | 12:46 | |
| ddeja | rakhmerov: Renat, I'm not sure what is the problem that you are solving... basically, you have added cache based on wf_ex insted of workflow, but how it helps? | 12:49 |
| *** toddjohn has joined #openstack-mistral | 12:49 | |
| rakhmerov | ddeja: ok, let me explain | 12:49 |
| rakhmerov | it's a little bit mindtwisting.. | 12:49 |
| rakhmerov | so, the problem was: cache instantiated on every Mistral instance, right? | 12:50 |
| rakhmerov | well, it's not a problem yet by itself.. | 12:50 |
| *** bobh has quit IRC | 12:50 | |
| ddeja | you mean, we have 3 engines and every have own cache? | 12:50 |
| rakhmerov | yes | 12:50 |
| rakhmerov | the problem was in workflow update operation | 12:50 |
| rakhmerov | imagine that we already have some cached values on those instances | 12:51 |
| rakhmerov | now we're making WF update | 12:51 |
| ddeja | oh, that is why you wanted boradcast message | 12:51 |
| rakhmerov | the WF definition is now different | 12:51 |
| rakhmerov | ddeja: yes, exactly, but I thought more and found a simpler solution | 12:51 |
| ddeja | ok | 12:52 |
| rakhmerov | so two options: 1) use RPC to invalidate cache on all instances | 12:52 |
| rakhmerov | 2) use smarter keys for getting cached values so that on cache key misses cache would be updated with a fresh value | 12:52 |
| ddeja | ok, I see - we make a spec for each execution | 12:52 |
| rakhmerov | yeah, this is even slightly a different problem | 12:53 |
| ddeja | which is better, because imagine what may happen if cached spec change during wf execution | 12:53 |
| rakhmerov | give me a sec pls... | 12:53 |
| ddeja | sure | 12:53 |
| rakhmerov | so, caching based on execution id is a must have anyway | 12:55 |
| rakhmerov | because, during the workflow execution lifecycle the spec should remain the same | 12:56 |
| rakhmerov | otherwise, if we update WF definition in parallel we'll break currently running executions | 12:56 |
| rakhmerov | this the 1st level where we need to cache | 12:56 |
| rakhmerov | the 2nd: caching based on WF definition id | 12:57 |
| rakhmerov | it's needed when we start a new WF | 12:57 |
| rakhmerov | instead of building a spec again we can use a different cache to get a value from it | 12:58 |
| rakhmerov | but | 12:58 |
| *** sshnaidm has joined #openstack-mistral | 12:58 | |
| rakhmerov | for this type of caching only WF definition id is not enough | 12:58 |
| rakhmerov | for the problem I mentioned above: distributed invalidation is required | 12:59 |
| ddeja | yup | 12:59 |
| rakhmerov | solution: use WF definition id and 'updated_at' field | 12:59 |
| ddeja | smart | 12:59 |
| rakhmerov | so that if 'updated_at' is changed it will be a cache miss | 12:59 |
| rakhmerov | and it will load a fresh value from DB | 13:00 |
| rakhmerov | cool, hah? :) | 13:00 |
| rakhmerov | no RPC is required | 13:00 |
| ddeja | it is good, since it should not be used for synchronisation | 13:00 |
| ddeja | only the DB | 13:00 |
| rakhmerov | yes | 13:00 |
| ddeja | yeah, I like it - elegant idea | 13:01 |
| rakhmerov | so I fixed two problems: 1) caching during execution lifecycle 2) Automatic invalidation | 13:01 |
| rakhmerov | yeah, so any old cache values are evicted by LRU algo | 13:02 |
| ddeja | rakhmerov: OK, waiting for Jenkins but for me it looks OK | 13:02 |
| rakhmerov | ddeja: one 'partial_join' test fails once in a while, I did investigation ant it seems like the reason is SQlight | 13:03 |
| rakhmerov | because it's really bad at transactional stuff | 13:03 |
| ddeja | yup | 13:03 |
| rakhmerov | I'll do more investigation though anyway.. | 13:04 |
| *** toddjohn has quit IRC | 13:11 | |
| openstackgerrit | Renat Akhmerov proposed openstack/mistral: Fix specification caching mechanism https://review.openstack.org/352869 | 13:17 |
| openstackgerrit | Renat Akhmerov proposed openstack/mistral: Towards non-locking model: remove pessimistic locks https://review.openstack.org/352895 | 13:17 |
| *** jpeeler has joined #openstack-mistral | 13:19 | |
| *** toddjohn has joined #openstack-mistral | 13:50 | |
| *** rbrady has quit IRC | 13:58 | |
| *** rbrady has joined #openstack-mistral | 13:58 | |
| *** jistr is now known as jistr|debug | 14:00 | |
| *** tonytan4ever has joined #openstack-mistral | 14:05 | |
| *** rrecio has joined #openstack-mistral | 14:06 | |
| *** rrecio_ has joined #openstack-mistral | 14:07 | |
| *** bobh has joined #openstack-mistral | 14:09 | |
| *** bobh has quit IRC | 14:10 | |
| *** bobh has joined #openstack-mistral | 14:10 | |
| *** rrecio has quit IRC | 14:11 | |
| *** vishwanathj has joined #openstack-mistral | 14:13 | |
| rakhmerov | ddeja: I found a bug with 'join' | 14:21 |
| rakhmerov | that's why 'partial_join' keeps failing | 14:21 |
| rakhmerov | till then, my patches won't be passing stabily | 14:21 |
| rakhmerov | I'll send a separate fix since it's not directly related to them | 14:22 |
| *** janki has quit IRC | 14:33 | |
| *** catintheroof has quit IRC | 14:47 | |
| *** catintheroof has joined #openstack-mistral | 14:48 | |
| *** brian_price has joined #openstack-mistral | 14:52 | |
| *** catintheroof has quit IRC | 14:53 | |
| *** tonytan4ever has quit IRC | 14:56 | |
| *** tonytan4ever has joined #openstack-mistral | 14:57 | |
| *** jistr|debug is now known as jistr | 15:23 | |
| *** catintheroof has joined #openstack-mistral | 15:25 | |
| *** Ephur has joined #openstack-mistral | 15:37 | |
| *** gokrokve has joined #openstack-mistral | 15:58 | |
| *** dprince has quit IRC | 16:04 | |
| *** dprince has joined #openstack-mistral | 16:05 | |
| *** toddjohn has quit IRC | 16:12 | |
| *** toddjohn has joined #openstack-mistral | 16:13 | |
| *** jpich has quit IRC | 16:15 | |
| *** toddjohn has quit IRC | 16:17 | |
| *** toddjohn has joined #openstack-mistral | 16:34 | |
| *** dprince has quit IRC | 16:42 | |
| *** dprince has joined #openstack-mistral | 16:43 | |
| *** dprince has quit IRC | 16:54 | |
| *** vishwanathj has quit IRC | 17:00 | |
| *** gokrokve has quit IRC | 17:02 | |
| *** vishwanathj has joined #openstack-mistral | 17:21 | |
| *** tonytan4ever has quit IRC | 17:28 | |
| *** dprince has joined #openstack-mistral | 17:32 | |
| *** jaosorior has quit IRC | 17:50 | |
| *** tonytan4ever has joined #openstack-mistral | 18:05 | |
| *** toddjohn has quit IRC | 18:12 | |
| *** toddjohn has joined #openstack-mistral | 18:13 | |
| *** toddjohn has quit IRC | 18:13 | |
| *** toddjohn has joined #openstack-mistral | 18:14 | |
| *** toddjohn has quit IRC | 18:36 | |
| *** toddjohn has joined #openstack-mistral | 18:37 | |
| *** toddjohn has quit IRC | 18:38 | |
| *** toddjohn has joined #openstack-mistral | 18:39 | |
| openstackgerrit | Renat Akhmerov proposed openstack/mistral: Fix workflow and join completion logic https://review.openstack.org/353063 | 18:45 |
| *** shardy is now known as shardy_afk | 18:58 | |
| *** catintheroof has quit IRC | 19:12 | |
| *** bobh has quit IRC | 19:35 | |
| *** bobh has joined #openstack-mistral | 19:36 | |
| *** bobh__ has joined #openstack-mistral | 19:37 | |
| *** bobh has quit IRC | 19:41 | |
| *** gyee has joined #openstack-mistral | 20:37 | |
| *** dprince has quit IRC | 20:37 | |
| *** shardy_afk is now known as shardy | 20:55 | |
| *** Ephur_ has joined #openstack-mistral | 20:58 | |
| *** Ephur has quit IRC | 21:02 | |
| *** Ephur_ has quit IRC | 21:06 | |
| *** shardy has quit IRC | 21:38 | |
| *** toddjohn has quit IRC | 21:54 | |
| *** rbrady has quit IRC | 21:57 | |
| *** rbrady has joined #openstack-mistral | 21:58 | |
| *** bobh__ has quit IRC | 22:00 | |
| *** rbrady has quit IRC | 22:01 | |
| *** tonytan4ever has quit IRC | 22:26 | |
| *** tonytan4ever has joined #openstack-mistral | 22:28 | |
| *** toddjohn has joined #openstack-mistral | 22:35 | |
| *** toddjohn has quit IRC | 22:40 | |
| *** toddjohn has joined #openstack-mistral | 22:41 | |
| *** rrecio_ has quit IRC | 23:18 | |
| *** bobh has joined #openstack-mistral | 23:27 | |
| *** bobh has quit IRC | 23:50 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!