| *** amitgandhinz has joined #openstack-zaqar | 00:37 | |
| *** amitgandhinz has quit IRC | 01:31 | |
| *** amalagon has quit IRC | 02:12 | |
| *** amitgandhinz has joined #openstack-zaqar | 02:34 | |
| *** amitgandhinz has quit IRC | 02:52 | |
| *** jeffrey4l has joined #openstack-zaqar | 03:16 | |
| *** kgriffs|afk is now known as kgriffs | 03:21 | |
| *** jeffrey4l has quit IRC | 04:32 | |
| *** achanda has joined #openstack-zaqar | 04:35 | |
| *** flwang1 has joined #openstack-zaqar | 04:36 | |
| *** jeffrey4l has joined #openstack-zaqar | 04:44 | |
| *** kgriffs is now known as kgriffs|afk | 04:52 | |
| *** jeffrey4l has quit IRC | 04:56 | |
| *** flwang1 has quit IRC | 04:57 | |
| *** achanda has quit IRC | 04:59 | |
| *** achanda has joined #openstack-zaqar | 05:01 | |
| *** achanda has quit IRC | 05:05 | |
| *** vkmc has quit IRC | 05:13 | |
| *** flwang1 has joined #openstack-zaqar | 05:14 | |
| *** jeffrey4l has joined #openstack-zaqar | 05:18 | |
| *** achanda has joined #openstack-zaqar | 05:19 | |
| *** flwang1 has quit IRC | 05:21 | |
| *** flwang1 has joined #openstack-zaqar | 05:25 | |
| *** achanda has quit IRC | 05:26 | |
| *** sgotliv_ has joined #openstack-zaqar | 05:26 | |
| *** sgotliv_ has quit IRC | 05:33 | |
| *** achanda has joined #openstack-zaqar | 05:52 | |
| *** achanda has quit IRC | 05:55 | |
| *** achanda has joined #openstack-zaqar | 05:56 | |
| *** flwang1 has quit IRC | 05:56 | |
| *** flwang1 has joined #openstack-zaqar | 05:59 | |
| *** sgotliv_ has joined #openstack-zaqar | 05:59 | |
| *** achanda has quit IRC | 06:00 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/zaqar: Imported Translations from Transifex https://review.openstack.org/140240 | 06:06 |
|---|---|---|
| *** reed has quit IRC | 06:41 | |
| *** flwang1 has quit IRC | 06:59 | |
| *** sgotliv_ has quit IRC | 07:14 | |
| *** X019 has joined #openstack-zaqar | 07:58 | |
| *** sgotliv has joined #openstack-zaqar | 08:32 | |
| *** sgotliv has quit IRC | 09:02 | |
| *** flwang1 has joined #openstack-zaqar | 09:20 | |
| flwang1 | flaper87: ping | 09:21 |
| flaper87 | flwang1: pong | 09:22 |
| flwang1 | flaper87: how you doing? | 09:22 |
| flaper87 | flwang1: doing great, you? | 09:22 |
| flwang1 | flaper87: do you have a few minutes to have a quick look for https://review.openstack.org/#/c/139792/ ? | 09:22 |
| flaper87 | sure, lemme check that | 09:23 |
| flwang1 | not bad, working on the mongodb driver for notifications | 09:23 |
| *** sgotliv has joined #openstack-zaqar | 09:23 | |
| flaper87 | flwang1: that looks good, it'd be good if you could put the base class and mongodb's one on the same patch | 09:24 |
| flaper87 | it'd make it easier to review and find corner cases | 09:24 |
| flaper87 | although, it looks good so far | 09:24 |
| flwang1 | flaper87: ok, I just want to raise the code asap to get some feedback | 09:25 |
| flwang1 | but i'm ok to submit them in a same patch | 09:26 |
| flaper87 | flwang1: absolutely, thanks a lot for that | 09:26 |
| flwang1 | flaper87: sure, thanks for review | 09:26 |
| *** sgotliv has quit IRC | 09:53 | |
| *** sgotliv has joined #openstack-zaqar | 09:59 | |
| *** sgotliv has quit IRC | 10:16 | |
| *** sgotliv has joined #openstack-zaqar | 10:28 | |
| *** sgotliv has quit IRC | 10:31 | |
| *** sgotliv has joined #openstack-zaqar | 10:32 | |
| *** amalagon has joined #openstack-zaqar | 10:43 | |
| *** jeffrey4l has quit IRC | 11:27 | |
| *** jeffrey4l has joined #openstack-zaqar | 11:28 | |
| *** achanda has joined #openstack-zaqar | 11:59 | |
| *** malini has joined #openstack-zaqar | 12:02 | |
| *** malini has quit IRC | 12:03 | |
| *** bradjones has joined #openstack-zaqar | 12:03 | |
| *** malini has joined #openstack-zaqar | 12:05 | |
| openstackgerrit | Merged openstack/zaqar-specs: Workflow documentation is now in infra-manual https://review.openstack.org/139405 | 12:18 |
| *** dynarro has joined #openstack-zaqar | 12:25 | |
| *** dynarro_ has joined #openstack-zaqar | 12:25 | |
| *** vkmc has joined #openstack-zaqar | 12:32 | |
| *** dynarro_ has left #openstack-zaqar | 12:32 | |
| vkmc | good morning :) | 12:33 |
| flaper87 | vkmc: goooooooooooood morning | 12:33 |
| vkmc | flaper87, hiiiiiii o/ | 12:37 |
| dynarro | good morning ;) | 12:37 |
| vkmc | dynarro, hey there o/ | 12:39 |
| vkmc | flaper87, if you have a moment later, wanted to discuss something about the protocol for websockets | 12:40 |
| vkmc | yes... I just joined and I'm already bothering you :D | 12:41 |
| flaper87 | vkmc: yo yo yo yo yo! | 12:41 |
| flaper87 | shoot | 12:41 |
| * flaper87 can't believe how annoying this girl is | 12:41 | |
| flaper87 | :P | 12:41 |
| vkmc | well, question is, do you think we have to add further control to the flow control? | 12:43 |
| vkmc | that is | 12:43 |
| * flaper87 sits tight | 12:43 | |
| vkmc | implement a buffer, have messages to control when a connection opens, control the time packages arrive the server, etc? | 12:43 |
| vkmc | we obviously need some time to process the request | 12:44 |
| vkmc | in the poc jim came with a quite complex algorithm to do this | 12:44 |
| vkmc | and I was thinking... this should be part of the wire protocol spec | 12:45 |
| flaper87 | what do you mean with "we obviously need some time to process the request" ? | 12:46 |
| flaper87 | w.r.t the buffer thing, I think we'll have to implement something that'll allow us to "stream" message to the client but that could probably come later | 12:47 |
| vkmc | e.g. when a message is received... inspect the headers to know what the client wants (send a message to the client), create the request instance, send it to the API, validate, send it to the storage backend, create a response, send the response | 12:47 |
| vkmc | that requires time | 12:47 |
| vkmc | oh and I didn't add serialize and deserialize | 12:48 |
| *** malini has quit IRC | 12:48 | |
| vkmc | with buffer I meant, incoming buffer | 12:50 |
| vkmc | ʕ •ᴥ•ʔ | 12:58 |
| vkmc | hmm the emoji didn't bring back flaper87 attention | 13:00 |
| * flaper87 back | 13:00 | |
| flaper87 | ok | 13:01 |
| flaper87 | there are definitely some things that must happen when the server gets a message | 13:01 |
| * flaper87 thinking | 13:02 | |
| flaper87 | I'm not sure how that is relevant to what you're doing now | 13:02 |
| flaper87 | I mean, the message headers are *in* the message dict | 13:02 |
| vkmc | that's right | 13:03 |
| flaper87 | (which is different from http) | 13:03 |
| vkmc | but shouldn't we add control frames? | 13:04 |
| vkmc | with low loads it doesn't matter | 13:04 |
| flaper87 | I don't think so since messages are stateless anyway | 13:04 |
| * flaper87 goes and reads jim's POC | 13:05 | |
| flaper87 | are messages stateful there? | 13:05 |
| vkmc | but with high loads... there is a risk that one message takes a little longer or for some reason one message cannot be parsed and you may end up losing messages | 13:05 |
| vkmc | the server has states in his poc, not the messages | 13:06 |
| vkmc | but yeah, he carry those in the messages | 13:07 |
| flaper87 | server side, it may not be necessary but probably client side, it is | 13:07 |
| flaper87 | but again, we can identify message by having a msg-id or something | 13:07 |
| * flaper87 digs into that | 13:07 | |
| vkmc | k, I'll try to do it without all that and run a rally benchmark on it | 13:08 |
| vkmc | see what happens | 13:08 |
| vkmc | just opening the connection and letting the client send messages | 13:09 |
| flaper87 | ok | 13:10 |
| * flaper87 reads the POC | 13:12 | |
| vkmc | the docs are a pseudocode, they are useful as well | 13:13 |
| vkmc | (btw, K1 is in one week) | 13:13 |
| flaper87 | yeah | 13:13 |
| flaper87 | :/ | 13:13 |
| vkmc | :') | 13:13 |
| *** achanda has quit IRC | 13:17 | |
| flaper87 | considering K-1 is next week, this looks really bad: https://launchpad.net/zaqar/+milestone/kilo-1 | 13:29 |
| flaper87 | we seriously need to do more reviews | 13:30 |
| vkmc | agree | 13:30 |
| vkmc | but we need to move forward with the implementations as well | 13:31 |
| flaper87 | right but if we don't review, those implementations won't land :P | 13:32 |
| flaper87 | https://review.openstack.org/#/q/status:open+project:openstack/zaqar,n,z | 13:32 |
| vkmc | yeah | 13:32 |
| * flaper87 starts reviewing | 13:32 | |
| * vkmc does the same | 13:32 | |
| flaper87 | vkmc: reviewed your ws patch | 13:47 |
| flaper87 | great start | 13:47 |
| * flaper87 is so jealous, vkmc is having fun | 13:47 | |
| vkmc | lool | 13:47 |
| vkmc | thx flaper87 :) | 13:48 |
| *** kgriffs|afk is now known as kgriffs | 14:00 | |
| *** jchai has joined #openstack-zaqar | 14:00 | |
| openstackgerrit | Merged openstack/zaqar: Imported Translations from Transifex https://review.openstack.org/140240 | 14:04 |
| openstackgerrit | Merged openstack/zaqar: Removes ZMQ directory https://review.openstack.org/140359 | 14:04 |
| vkmc | flaper87, what's the difference of having a decorator to declare the abstract method and having abc with a dummy body? https://review.openstack.org/#/c/139073/4/zaqar/storage/base.py | 14:15 |
| flaper87 | vkmc: abc is.... stupid | 14:19 |
| flaper87 | Basically, if you decorate the method, it'll set a __abstractmethod__ attribute (or something like that) on the method | 14:20 |
| flaper87 | which means doing something like: _list = my_decorated_method will make _list an abstractmethod as well | 14:20 |
| flaper87 | :( | 14:20 |
| flaper87 | I tried several different combinations and that one seemed to be the most readable and simple one | 14:20 |
| vkmc | I still don't understand :D hahaha | 14:22 |
| vkmc | what do you achieve by changing it that way? | 14:22 |
| flaper87 | ah, you're asking why I went with duplicating those methods? | 14:24 |
| vkmc | yes | 14:24 |
| vkmc | the outcome is the same AFAIU | 14:24 |
| flaper87 | In that patch, it is | 14:25 |
| flaper87 | The reason I did that is because for some methods we need to do some validations in the storage layer *before* we call the actual storage method | 14:25 |
| flaper87 | for example | 14:25 |
| vkmc | ohh... so the purpose will be seen in following patches | 14:25 |
| flaper87 | yup | 14:25 |
| flaper87 | :P | 14:25 |
| flaper87 | "This is the first patch of a series of patches that will update all the storage controllers." <- I didn't write this for fun | 14:25 |
| flaper87 | :P | 14:25 |
| vkmc | keep going though, I don't mind having some spoilers | 14:26 |
| * flaper87 gives vkmc a cookie | 14:26 | |
| vkmc | I know, I read it | 14:26 |
| * vkmc takes the cookie and gives back a coffee | 14:26 | |
| flaper87 | I was going to say, one use case for it is having better support for pools/flavors and be able to switch storage's depending on their configs | 14:27 |
| * flaper87 swears he had another use case in mind | 14:27 | |
| flaper87 | I just lost it | 14:27 |
| vkmc | so... for doing validations in the storage layer before calling the actual method | 14:29 |
| vkmc | you override _method? | 14:29 |
| vkmc | (that's the bit I'm missing) | 14:29 |
| *** amitgandhinz has joined #openstack-zaqar | 14:33 | |
| *** sriram has joined #openstack-zaqar | 14:36 | |
| vkmc | flaper87, ^ | 14:39 |
| * flaper87 reads | 14:49 | |
| flaper87 | nope, storages now override `_method` instead of `method` | 14:49 |
| flaper87 | and `method` will remain always the same, documented and used as reference. | 14:50 |
| flaper87 | the use of `method` will be internal only | 14:50 |
| vkmc | method calls _method | 14:50 |
| *** sgotliv has quit IRC | 14:53 | |
| *** malini has joined #openstack-zaqar | 15:06 | |
| *** achanda has joined #openstack-zaqar | 15:07 | |
| *** sgotliv has joined #openstack-zaqar | 15:10 | |
| *** kgriffs is now known as kgriffs|afk | 15:20 | |
| *** ametts has joined #openstack-zaqar | 15:32 | |
| *** achanda has quit IRC | 15:44 | |
| *** kgriffs|afk is now known as kgriffs | 15:51 | |
| *** achanda has joined #openstack-zaqar | 15:55 | |
| vkmc | flaper87, so in which part of the flow you do the controls before calling the _method? | 15:55 |
| vkmc | I'm still with that | 15:56 |
| *** kgriffs is now known as kgriffs|afk | 15:56 | |
| flaper87 | vkmc: in method | 15:57 |
| vkmc | ok... so you end up overriding the method as well | 15:58 |
| *** kgriffs|afk is now known as kgriffs | 15:59 | |
| *** achanda has quit IRC | 16:02 | |
| vkmc | I assume that is how are tackling it | 16:09 |
| *** achanda_ has joined #openstack-zaqar | 16:13 | |
| * flaper87 back | 16:17 | |
| flaper87 | vkmc: so, lemme start over with that | 16:17 |
| flaper87 | drivers implementation will override the private methods | 16:18 |
| flaper87 | driver implementations shouldn't override the public methods by any means | 16:18 |
| flaper87 | the public methos belong to the base - or internal, if you will - implementation | 16:19 |
| flaper87 | https://speakerdeck.com/flaper87/kilo-plans-for-openstack-messaging | 16:21 |
| *** X019 has quit IRC | 16:24 | |
| vkmc | that is clear | 16:26 |
| *** amalagon has quit IRC | 16:27 | |
| vkmc | my question was... how adding that wrapper changes the way you perform controls before calling the method | 16:27 |
| flaper87 | it's not that patch that changes it, but it allows for changing it | 16:27 |
| flaper87 | 2s | 16:27 |
| vkmc | sure | 16:28 |
| vkmc | I didn't want to review without understanding why that change was needed, but I trust your judgement | 16:28 |
| vkmc | great slides btw :) where are you presenting? | 16:29 |
| flaper87 | already did, it's a on-line preso | 16:29 |
| flaper87 | it'll be up in ~1h | 16:29 |
| vkmc | ooooh :) | 16:30 |
| vkmc | cool | 16:30 |
| *** kgriffs is now known as kgriffs|afk | 16:31 | |
| flaper87 | http://paste.openstack.org/show/148785/ | 16:31 |
| flaper87 | vkmc: ^ | 16:31 |
| flaper87 | does that clear this out ? | 16:31 |
| *** sgotliv has quit IRC | 16:32 | |
| vkmc | thanks, it does | 16:32 |
| vkmc | my concern is now that.. by doing that... you cannot perform controls for each driver differently | 16:33 |
| vkmc | but probably that is what you are looking for | 16:33 |
| flaper87 | what do you mean? | 16:33 |
| flaper87 | you can do further checks in the `_list` method if you need | 16:34 |
| *** kgriffs|afk is now known as kgriffs | 16:34 | |
| vkmc | that's ok | 16:34 |
| vkmc | no more concerns | 16:35 |
| vkmc | thanks | 16:35 |
| *** X019 has joined #openstack-zaqar | 16:37 | |
| flaper87 | :P | 16:37 |
| * vkmc STFU | 16:37 | |
| vkmc | its hard to follow what is in your mind | 16:38 |
| *** jchai is now known as jchai_afk | 16:39 | |
| *** jchai_afk is now known as jchai | 16:40 | |
| *** mpanetta has joined #openstack-zaqar | 16:41 | |
| *** mpanetta has quit IRC | 16:41 | |
| *** dynarro has quit IRC | 16:44 | |
| *** sgotliv has joined #openstack-zaqar | 16:44 | |
| *** zhiyan has quit IRC | 16:46 | |
| *** kgriffs is now known as kgriffs|afk | 16:48 | |
| *** jchai is now known as jchai_afk | 16:50 | |
| *** amalagon has joined #openstack-zaqar | 16:53 | |
| *** mpanetta has joined #openstack-zaqar | 16:57 | |
| *** achanda_ has quit IRC | 16:58 | |
| *** achanda has joined #openstack-zaqar | 17:02 | |
| *** dynarro has joined #openstack-zaqar | 17:30 | |
| *** dynarro_ has joined #openstack-zaqar | 17:30 | |
| *** dynarro_ has quit IRC | 17:36 | |
| *** dynarro has quit IRC | 17:36 | |
| *** jchai_afk is now known as jchai | 17:50 | |
| flaper87 | https://twitter.com/flaper87/status/542738822933807104 | 17:54 |
| *** JAHoagie has joined #openstack-zaqar | 17:55 | |
| *** kgriffs|afk is now known as kgriffs | 18:02 | |
| *** zhiyan has joined #openstack-zaqar | 18:09 | |
| *** amitgandhinz has quit IRC | 18:15 | |
| *** flwang has quit IRC | 18:40 | |
| *** flwang has joined #openstack-zaqar | 18:52 | |
| *** amitgandhinz has joined #openstack-zaqar | 19:02 | |
| *** reed has joined #openstack-zaqar | 19:03 | |
| *** reed has quit IRC | 19:05 | |
| *** flwang1 has quit IRC | 19:09 | |
| * vkmc hears Zaqar's PTL talking about mythologies | 19:12 | |
| flaper87 | :P | 19:18 |
| vkmc | that dud is crazy | 19:19 |
| *** reed has joined #openstack-zaqar | 19:22 | |
| *** X019 has quit IRC | 19:25 | |
| flaper87 | LOL, isn't he ? | 19:27 |
| vkmc | just finished listening to it, it was great | 19:28 |
| vkmc | I'm now scared about the [...] | 19:28 |
| flaper87 | LOL | 19:28 |
| flaper87 | Back to April 2015 | 19:28 |
| vkmc | ENOUGH | 19:28 |
| flaper87 | lmao | 19:28 |
| vkmc | hahaha | 19:28 |
| vkmc | it would be really nice if we could make the integration with Horizon as well | 19:29 |
| vkmc | but first things first... let's reach k-1 safe and sound | 19:29 |
| *** kgriffs is now known as kgriffs|afk | 19:56 | |
| *** sgotliv has quit IRC | 20:00 | |
| *** flwang1 has joined #openstack-zaqar | 20:00 | |
| *** malini has quit IRC | 20:06 | |
| *** jchai is now known as jchai_afk | 20:08 | |
| *** echevemaster has joined #openstack-zaqar | 20:12 | |
| vkmc | flaper87, I'm moving this to k-2 | 20:12 |
| vkmc | https://blueprints.launchpad.net/zaqar/+spec/api-v1.1-user-guide | 20:12 |
| vkmc | when we start working on the apiv2, I'll wrap up docs for apiv1.1 | 20:13 |
| flaper87 | vkmc: I moved it earlier today | 20:14 |
| flaper87 | didn't I ? | 20:14 |
| vkmc | nope | 20:15 |
| vkmc | just did | 20:20 |
| vkmc | http://logs.openstack.org/22/136922/8/experimental/gate-rally-dsvm-zaqar-rally/d9fda1a/rally-plot/results.html.gz#/ZaqarBasic.producer_consumer/details | 20:22 |
| flaper87 | omg, plots plots | 20:23 |
| vkmc | queue creation produces the highest overhead | 20:23 |
| vkmc | we have to get rid of queues dud | 20:23 |
| * flaper87 should update that spec | 20:24 | |
| *** malini has joined #openstack-zaqar | 20:24 | |
| vkmc | also | 20:25 |
| vkmc | something interesting to know | 20:25 |
| vkmc | with 1k ++ iterations | 20:25 |
| vkmc | keystone dies | 20:25 |
| flwang | vkmc: what did you mean 'queue creation produces the highest overhead'? | 20:28 |
| flwang | vkmc: I guess it's related to the record creating in database | 20:29 |
| flwang | is the rally test based on mongodb? | 20:29 |
| vkmc | flwang, the average time of queue creation is 0.45sec | 20:30 |
| vkmc | flwang, with peaks of 0.58sec | 20:30 |
| vkmc | flwang, while posting messages is 0.22sec | 20:30 |
| flwang | vkmc: yep, I saw it | 20:31 |
| flwang | it's interesting | 20:31 |
| vkmc | indeed | 20:31 |
| vkmc | its against mongo yeah :) | 20:31 |
| vkmc | it would be cool to try it with redis as well | 20:32 |
| vkmc | I'll do that in my local env and compare | 20:32 |
| flwang | vkmc: is it enabling pool? | 20:32 |
| vkmc | flwang, no | 20:32 |
| flwang | vkmc: ohhhhhhhhhhhhh | 20:32 |
| flwang | I was thinking the pool 'seeking' consumed some times | 20:33 |
| flwang | if we drop queues, then we may need rethink pool and a lot of stuff | 20:33 |
| *** kgriffs|afk is now known as kgriffs | 20:34 | |
| vkmc | yeah | 20:34 |
| *** riveter has left #openstack-zaqar | 20:45 | |
| *** kgriffs is now known as kgriffs|afk | 20:48 | |
| *** kgriffs|afk is now known as kgriffs | 20:52 | |
| vkmc | at first sight, this PS looks good https://review.openstack.org/#/c/132907/1 | 20:54 |
| vkmc | riveter has been waiting for a reply for too long :/ we should close that | 20:55 |
| *** JAHoagie has quit IRC | 21:03 | |
| *** jchai_afk is now known as jchai | 21:04 | |
| *** malini has quit IRC | 21:08 | |
| *** achanda has quit IRC | 21:15 | |
| *** X019 has joined #openstack-zaqar | 21:23 | |
| *** kgriffs is now known as kgriffs|afk | 21:24 | |
| *** JAHoagie has joined #openstack-zaqar | 21:27 | |
| flaper87 | vkmc: why did she create a new review ? | 21:36 |
| flaper87 | ah wait, this is something different | 21:36 |
| flaper87 | nvm | 21:36 |
| flaper87 | actually, it's not something different | 21:37 |
| * flaper87 confused | 21:37 | |
| * flaper87 will review tomorrow | 21:37 | |
| vkmc | flaper87, because kgriffs|afk asked to do the same with a different approach | 21:53 |
| flaper87 | vkmc: yup, I remember asking the same | 21:57 |
| flaper87 | but there was no need for a new review | 21:57 |
| flaper87 | not a big deal | 21:57 |
| flaper87 | I was just wondering if she knew it's possible to use the same change-id | 21:57 |
| vkmc | flaper87, probably she knew, but she wanted to create a different patch set just in case | 21:59 |
| vkmc | anyhow :) | 22:02 |
| vkmc | thanks for checking it out flaper87 | 22:02 |
| *** jchai has quit IRC | 22:05 | |
| *** JAHoagie has quit IRC | 22:31 | |
| *** sriram has quit IRC | 22:45 | |
| openstackgerrit | Victoria Martínez de la Cruz proposed openstack/zaqar: Adds websockets driver to Zaqar server https://review.openstack.org/140155 | 22:51 |
| vkmc | bbl | 22:53 |
| vkmc | \o) | 22:53 |
| vkmc | (o/ | 22:54 |
| *** JAHoagie has joined #openstack-zaqar | 23:13 | |
| *** amitgandhinz has quit IRC | 23:18 | |
| *** mpanetta has quit IRC | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!