*** bradjones has quit IRC | 00:18 | |
*** exploreshaifali has quit IRC | 00:21 | |
openstackgerrit | Victoria Martínez de la Cruz proposed openstack/zaqar: Adds websockets driver to Zaqar server https://review.openstack.org/140155 | 01:07 |
---|---|---|
*** amalagon has quit IRC | 01:16 | |
*** boris-42 has quit IRC | 02:03 | |
*** X019 has quit IRC | 02:26 | |
*** amalagon has joined #openstack-zaqar | 02:30 | |
openstackgerrit | Victoria Martínez de la Cruz proposed openstack/zaqar: Adds websockets driver to Zaqar server https://review.openstack.org/140155 | 02:58 |
vkmc | flwang1, around? | 03:39 |
flwang1 | vkmc: yep, i'm reviewing your patch now :) | 03:54 |
flwang1 | 's up? | 03:54 |
vkmc | ohh | 03:54 |
vkmc | I'm trying to come for a decent solution for a change | 03:54 |
vkmc | >.< | 03:54 |
flwang1 | anything i can help? | 03:54 |
vkmc | maybe you can give me your opinion | 03:54 |
vkmc | we are working on separating the API layer from the transport | 03:55 |
vkmc | that is... the transport receives the message and sends it to the API... and the API validates and send the message to the storage | 03:55 |
flwang1 | so another layer above current API layer? | 03:55 |
vkmc | well.. not really... its the same layer but it has more responsabilities | 03:56 |
flwang1 | what's the API layer mean 'wsgi/websocket'? | 03:56 |
vkmc | this new architecture will be used for the persistent transports only... we are not changing wsgi | 03:56 |
flwang1 | vkmc: i think i need a clear definition about the 'new' transport layer and the api layer | 03:57 |
vkmc | sure | 03:57 |
vkmc | both changes are related | 03:57 |
vkmc | this new API, besides validating the schemas as we were doing, will be in charge processing the requests | 03:58 |
flwang1 | which will be like an abstract layer above wsgi and websocket, is it? | 03:59 |
vkmc | nope, in the middle | 03:59 |
vkmc | e.g. websocket -> api -> mongodb | 03:59 |
flwang1 | and wsgi->api->mongodb? | 04:00 |
vkmc | wsgi won't be changed for now | 04:00 |
vkmc | its too complicated | 04:00 |
vkmc | so this will be for persistent transports only | 04:00 |
flwang1 | so that means zaqar client need to handle the difference? | 04:00 |
vkmc | right now the wsgi transport is in charge of validating and processing the messages received... that is something we discussed it was better to remove from the transport side | 04:01 |
vkmc | and put it in the api side | 04:01 |
vkmc | the transport will be in charge of transport only | 04:01 |
flwang1 | so what's the benefit to have an new, even thin, layer between transport and storage? | 04:01 |
flwang1 | ah, ok | 04:02 |
flwang1 | but seems flaper87 is also trying to move the validation(or part of them) to storage layer, right? | 04:02 |
vkmc | the benefit... less code I guess | 04:02 |
vkmc | we won't have to perform controls for every transport we might want to add | 04:02 |
vkmc | yeah, flaper87 is working on moving the data controls to the storage layer | 04:03 |
flwang1 | so any overlap/conflict with yours idea? | 04:03 |
vkmc | this is the description of the blueprint of the separated api thing | 04:04 |
vkmc | Define an API spec that supports extensions, versioning and that is also cross-transport. This will allow Zaqar to define its API in a single module and allow transport for translating the API to something supported by the protocol they implement. | 04:04 |
vkmc | its fancier than my 'less code' response haha | 04:04 |
vkmc | no, flaper's change is not overlapping my change because I'm making controls on the request received | 04:05 |
flwang1 | so from the api consumer view, they may be a little bit confused | 04:06 |
flwang1 | like me :) | 04:06 |
vkmc | haha I totally understand that because it took some time to me to process the whole idea | 04:06 |
flwang1 | unless we can make sure we will cover the magic in client side | 04:07 |
flwang1 | and provide the consistent 'api' (if i still can use this term) for our consumer/developers | 04:07 |
vkmc | the client has a similar structure as the one we are trying to achieve in the server | 04:07 |
flwang1 | vkmc: i see. | 04:08 |
flwang1 | my concern is I don't want to see any big change for our api | 04:08 |
flwang1 | given we want to graduate, and a stable api is really important | 04:08 |
vkmc | yeah... I feel the same way | 04:08 |
vkmc | also... the idea of having a way of handling websockets and another way to handle wsgi | 04:09 |
flwang1 | for websocket, IMHO, that's the way we are looking for improve zaqar's performance | 04:10 |
vkmc | yeah, but in the case of websocket | 04:10 |
vkmc | we can do something similar to what we have for wsgi | 04:10 |
flwang1 | you mean wsgi->^api^->storage? | 04:11 |
vkmc | and I think its a good adition that doesn't affect what we have already | 04:11 |
vkmc | right now its wsgi->storage | 04:11 |
flwang1 | yep, i see | 04:11 |
vkmc | we can do websockets->storage | 04:11 |
vkmc | its not mandatory for websockets to do this change | 04:11 |
flwang1 | i would like to see the new arch won't impact what we have currently | 04:11 |
vkmc | it won't I think | 04:12 |
vkmc | because we are not changing wsgi | 04:12 |
flwang1 | okay | 04:12 |
vkmc | but it may look odd :) | 04:12 |
flwang1 | right | 04:12 |
flwang1 | that's what my concern either ;) | 04:12 |
vkmc | haha | 04:13 |
vkmc | I mean | 04:13 |
flwang1 | the api/arch of zaqar will be weird | 04:13 |
vkmc | I have been thinking all night different architectures | 04:13 |
vkmc | and honestly I haven't come to a decent solution | 04:13 |
vkmc | I don't want to do a handler with a ton of if clauses | 04:13 |
flwang1 | vkmc: i have to go, my son got fervor | 04:14 |
flwang1 | vkmc: i will ping you later, sorry for that | 04:14 |
vkmc | flwang1, please, don't be sorry... poor lil hacker | 04:14 |
vkmc | flwang1, ttyl | 04:14 |
*** flwang1 has quit IRC | 04:18 | |
*** amalagon has quit IRC | 04:44 | |
*** amitgandhinz has quit IRC | 06:05 | |
*** sgotliv has joined #openstack-zaqar | 06:08 | |
openstackgerrit | Victoria Martínez de la Cruz proposed openstack/zaqar: API handler and API v1.1 endpoints https://review.openstack.org/141280 | 06:12 |
*** sgotliv has quit IRC | 06:13 | |
*** reed has quit IRC | 06:28 | |
*** vkmc has quit IRC | 06:31 | |
*** achanda has joined #openstack-zaqar | 06:31 | |
*** sgotliv has joined #openstack-zaqar | 06:57 | |
*** achanda has quit IRC | 07:04 | |
*** sgotliv has quit IRC | 07:04 | |
*** achanda has joined #openstack-zaqar | 07:05 | |
*** amalagon has joined #openstack-zaqar | 07:05 | |
*** achanda_ has joined #openstack-zaqar | 07:07 | |
*** achanda has quit IRC | 07:09 | |
*** amalagon has quit IRC | 07:10 | |
*** achanda_ has quit IRC | 07:17 | |
flaper87 | flwang: hey there | 07:24 |
*** achanda has joined #openstack-zaqar | 07:46 | |
flaper87 | achanda: hey hey :) | 07:47 |
achanda | flapper87: Hi there | 07:47 |
*** amitgandhinz has joined #openstack-zaqar | 07:49 | |
*** achanda has quit IRC | 08:11 | |
*** achanda has joined #openstack-zaqar | 08:12 | |
*** achanda has quit IRC | 08:16 | |
*** achanda has joined #openstack-zaqar | 08:18 | |
*** sgotliv has joined #openstack-zaqar | 10:08 | |
*** achanda has quit IRC | 10:28 | |
*** sgotliv has quit IRC | 10:40 | |
*** malini has joined #openstack-zaqar | 11:34 | |
*** jasondotstar has joined #openstack-zaqar | 11:43 | |
*** boris-42 has joined #openstack-zaqar | 11:55 | |
*** sgotliv has joined #openstack-zaqar | 12:18 | |
*** achanda has joined #openstack-zaqar | 12:39 | |
*** jasondotstar has quit IRC | 12:48 | |
openstackgerrit | Zhi Yan Liu proposed openstack/zaqar: Integrate OSprofiler with Zaqar https://review.openstack.org/141356 | 12:58 |
*** vkmc has joined #openstack-zaqar | 13:09 | |
*** vkmc has quit IRC | 13:09 | |
*** vkmc has joined #openstack-zaqar | 13:09 | |
vkmc | o/ | 13:10 |
flaper87 | vkmc: hey hey | 13:10 |
vkmc | flaper87, hey dud | 13:10 |
vkmc | flaper87, https://review.openstack.org/#/c/141280/ | 13:13 |
vkmc | ^ :D | 13:13 |
* flaper87 clicks | 13:15 | |
vkmc | need your feedback to continue | 13:15 |
kragniz | morning, zaqhaar | 13:23 |
vkmc | zakhaaaar | 13:24 |
flaper87 | vkmc: commented | 13:24 |
flaper87 | vkmc: it's looking great | 13:24 |
flaper87 | vkmc: any reason you made those methods private? | 13:25 |
vkmc | flaper87, thanks for reviewing :) | 13:25 |
vkmc | well, given that those are for internal use | 13:26 |
vkmc | but there is no need not that you mention | 13:27 |
vkmc | I'll fix that | 13:28 |
vkmc | I have one corcern, as always | 13:29 |
*** amitgandhinz has quit IRC | 13:29 | |
vkmc | well, tbh two | 13:29 |
vkmc | 1. yesterday we were talking with flwang that it might look weird to have one architecture for wsgi and another for websockets | 13:30 |
vkmc | and without a logic explanation | 13:31 |
vkmc | we should go all the way or nothing | 13:31 |
flaper87 | as in, use the API layer for ws and not wsgi ? | 13:31 |
vkmc | yea | 13:31 |
flaper87 | FWIW, we talked about this at the summit and when we discussed the spec | 13:32 |
flaper87 | We tried using this work for wsgi when the idea first came out | 13:32 |
vkmc | did we discuss this spec? I wasn't there | 13:32 |
vkmc | probably is one of the times I got lost | 13:32 |
flaper87 | and we didn't move forward because it'd have ended in code duplication | 13:32 |
flaper87 | (when we discussed the persistent transport one) | 13:32 |
flaper87 | the reason is that, in order to be RESTFul compliant, we need to raise proper exceptions and whatnot | 13:33 |
flaper87 | this means me need to register endpoints, catch exceptions and re-raise them | 13:33 |
vkmc | yes I'm aware about that, we discussed it | 13:33 |
flaper87 | it's different for wire protocols because we just translate the exception and send it back | 13:33 |
flaper87 | We can treat the protocol pretty much like rpc | 13:34 |
flaper87 | In addition to that, the wsgi transport is our *reference* transport | 13:34 |
vkmc | indeed | 13:34 |
flaper87 | what's coded there, must be supported by all other transports | 13:34 |
flaper87 | we don't want to mess with it | 13:34 |
flaper87 | if it makes it more clear, we can just rename the `api` package into `protocol/wire` ? | 13:35 |
vkmc | well, I'm raising this topic because its something that I noticed flwang didn't know about and had some doubts about it | 13:35 |
flaper87 | dunno, something that does not give the impression it's the API | 13:35 |
flaper87 | sure, re-visiting arguments is good | 13:35 |
vkmc | and it seems that this requires a clear up in the next meeting | 13:35 |
flaper87 | the next meeting is at 15 UTC | 13:35 |
flaper87 | I think he won't be around | 13:35 |
flaper87 | :( | 13:35 |
flaper87 | flwang: you, boy, read this ^ | 13:36 |
vkmc | ok :/ | 13:36 |
vkmc | let's pull all this comments in the blueprint board | 13:36 |
vkmc | I'll do so | 13:36 |
flaper87 | vkmc: could you also update the cross-api spec ? | 13:36 |
flaper87 | as in add these comments to the `Proposed Change` section | 13:37 |
flaper87 | I think it's worth it | 13:37 |
vkmc | that is what I meant | 13:37 |
flaper87 | ah sorry | 13:37 |
flaper87 | the blueprint term made me think of LP | 13:37 |
vkmc | no problem :D | 13:37 |
flaper87 | my bad | 13:37 |
flaper87 | :D | 13:37 |
* vkmc is a little worried that she is thinking like flaper87 | 13:37 | |
*** jasondotstar has joined #openstack-zaqar | 13:37 | |
vkmc | ok, will update both | 13:38 |
vkmc | your comments are welcome as well | 13:38 |
vkmc | hehe | 13:38 |
vkmc | aaaaaaaaaaaand, second concern | 13:39 |
vkmc | I noticed you or Cindy coded a validation method in the api | 13:39 |
vkmc | which is JSON only | 13:39 |
vkmc | da real thing is the validation in transport | 13:39 |
flaper87 | LP ruined that word for me | 13:39 |
vkmc | this validation https://github.com/openstack/zaqar/blob/master/zaqar/transport/validation.py | 13:40 |
vkmc | I know we chatted that the API should be in charge of doing the validation | 13:40 |
vkmc | but I think we should keep that to the client | 13:40 |
vkmc | sorry | 13:40 |
vkmc | transport | 13:40 |
vkmc | otherwise we are overcharging the api | 13:41 |
vkmc | and there are some attributes that are part of the transport | 13:41 |
flaper87 | so, that validation thing in the API validates the request form: It's based on jsonschema | 13:41 |
vkmc | yeah | 13:41 |
flaper87 | the remaining validations should go through transport.validation | 13:41 |
vkmc | that seems right | 13:41 |
flaper87 | we can't move the request validation to the client because one may use raw websockets | 13:42 |
flaper87 | I mean, one could send random requests to the server without using any client | 13:42 |
vkmc | o_o | 13:42 |
vkmc | so... we validate in both layers | 13:43 |
flaper87 | yup | 13:43 |
vkmc | ok | 13:43 |
flaper87 | I mean, we can allow clients to skip client validation | 13:44 |
flaper87 | but the server-side validation must exist | 13:44 |
vkmc | what do you mean by client here? | 13:44 |
vkmc | both validations are server side | 13:44 |
flaper87 | I don't know | 13:44 |
flaper87 | ah wait | 13:44 |
flaper87 | I thought by layers you meant client library and server | 13:45 |
flaper87 | you actually meant request schema and request data | 13:45 |
flaper87 | the answer is yes anyway | 13:45 |
flaper87 | :D | 13:45 |
vkmc | https://github.com/openstack/zaqar/blob/master/zaqar/common/api.py#L52 | 13:45 |
vkmc | that is the validation I'm confused about | 13:46 |
vkmc | but what I just understood is | 13:46 |
vkmc | transport validates incoming data, creates the request | 13:46 |
vkmc | api validates that the request is well formed? | 13:46 |
vkmc | seems unnecesary because we are building the request | 13:46 |
vkmc | we should be smart enough to build the request | 13:46 |
vkmc | in a sane way | 13:47 |
flaper87 | wait | 13:47 |
vkmc | I have to rush for a bit :( | 13:47 |
flaper87 | This evaluates the request form: https://github.com/openstack/zaqar/blob/master/zaqar/common/api.py#L52 | 13:47 |
flaper87 | this evaluates the request data, server limits, etc: https://github.com/openstack/zaqar/blob/master/zaqar/transport/validation.py | 13:48 |
flaper87 | those are different validations | 13:48 |
flaper87 | one validates required fields are in the request (former) the other validates the values (the later) | 13:48 |
flaper87 | We don't need the former for the wsgi transport | 13:48 |
flaper87 | which is why they are separated | 13:48 |
vkmc | cool | 13:53 |
vkmc | bbl | 13:53 |
vkmc | thanks for clear up things for meeee >.> | 13:54 |
*** malini has quit IRC | 13:55 | |
*** jchai has joined #openstack-zaqar | 13:57 | |
openstackgerrit | Flavio Percoco proposed openstack/zaqar: Wrap abstract method with base methods https://review.openstack.org/139073 | 14:06 |
openstackgerrit | Flavio Percoco proposed openstack/zaqar: Add capabilities property to the DataDriver https://review.openstack.org/135637 | 14:06 |
*** sriram has joined #openstack-zaqar | 14:07 | |
*** achanda has quit IRC | 14:11 | |
*** achanda has joined #openstack-zaqar | 14:12 | |
*** dynarro has joined #openstack-zaqar | 14:23 | |
*** jasondotstar has quit IRC | 14:23 | |
*** jchai has quit IRC | 14:24 | |
*** jchai has joined #openstack-zaqar | 14:32 | |
*** ametts has joined #openstack-zaqar | 14:46 | |
*** mpanetta has joined #openstack-zaqar | 14:49 | |
*** malini has joined #openstack-zaqar | 14:59 | |
*** jchai has quit IRC | 15:00 | |
openstackgerrit | Zhi Yan Liu proposed openstack/zaqar: Integrate OSprofiler with Zaqar https://review.openstack.org/141356 | 15:03 |
*** jchai has joined #openstack-zaqar | 15:06 | |
*** X019 has joined #openstack-zaqar | 15:08 | |
*** amitgandhinz has joined #openstack-zaqar | 15:09 | |
*** malini1 has joined #openstack-zaqar | 15:13 | |
*** malini has quit IRC | 15:15 | |
openstackgerrit | Zhi Yan Liu proposed openstack/zaqar-specs: Add OSProfiler to Zaqar https://review.openstack.org/135612 | 15:34 |
zhiyan | vkmc: hi, around? | 15:34 |
*** malini1 has left #openstack-zaqar | 15:35 | |
zhiyan | vkmc, kgriffs|afk flaper87 : when/if you ok, could you pls review osprofiler change and its spec (I'd like it can be flag in k1), any input are welcome, thanks. | 15:41 |
vkmc | zhiyan, heeeeeeey | 15:42 |
vkmc | zhiyan, sure thing | 15:42 |
vkmc | zhiyan, thanks for working on that | 15:42 |
zhiyan | vkmc: btw, could you educate me in the sepc change? i don't know which path should new proposed rst file save.. | 15:43 |
zhiyan | vkmc: glade to do this stuff in zaqar, useful for diagnose perf issue for dev and operator. | 15:45 |
vkmc | zhiyan, totally :) | 15:45 |
vkmc | zhiyan, so... the specs are in zaqar-specs repository | 15:45 |
vkmc | zhiyan, https://github.com/openstack/zaqar-specs | 15:45 |
vkmc | zhiyan, you can clone that repo, grab my change and amend it with your comments | 15:46 |
zhiyan | vkmc: oh, yes, i mean this http://logs.openstack.org/12/135612/5/check/gate-zaqar-specs-docs/cacf6f5/console.html#_2014-12-12_15_40_54_101 | 15:46 |
vkmc | ohhh I see | 15:47 |
zhiyan | seems "zaqar-specs/specs/kilo" path is wrong , at least fro the doc test | 15:48 |
openstackgerrit | Zhi Yan Liu proposed openstack/zaqar-specs: Add OSProfiler to Zaqar https://review.openstack.org/135612 | 16:00 |
zhiyan | vkmc: fixed ^^ | 16:00 |
vkmc | zhiyan, thanks, I got distracted, sorry | 16:01 |
*** cpallares has joined #openstack-zaqar | 16:11 | |
*** achanda has quit IRC | 16:11 | |
*** achanda has joined #openstack-zaqar | 16:11 | |
*** achanda has quit IRC | 16:16 | |
*** achanda has joined #openstack-zaqar | 16:18 | |
*** jasondotstar has joined #openstack-zaqar | 16:19 | |
*** reed has joined #openstack-zaqar | 16:19 | |
*** X019 has quit IRC | 16:21 | |
*** dynarro has quit IRC | 16:21 | |
*** boris-42 has quit IRC | 16:32 | |
*** achanda has quit IRC | 16:50 | |
*** achanda has joined #openstack-zaqar | 16:51 | |
*** achanda has quit IRC | 16:56 | |
jasondotstar | hi all | 16:57 |
jasondotstar | new here. just saying hi. | 16:58 |
*** amalagon has joined #openstack-zaqar | 17:00 | |
* kragniz huggles jasondotstar | 17:02 | |
*** achanda has joined #openstack-zaqar | 17:03 | |
*** miqui_ has quit IRC | 17:04 | |
*** miqui_ has joined #openstack-zaqar | 17:04 | |
* jasondotstar feels warm and fuzzy | 17:05 | |
jasondotstar | honestly, I'm looking for a project to get involved in. make my first contribs to. | 17:05 |
jasondotstar | saw the PTL update on Zaqar | 17:06 |
jasondotstar | interesting project you guys have here. | 17:06 |
kragniz | jasondotstar: you should totally work on zaqar | 17:09 |
kragniz | I personally don't, but they have the best irc channel around | 17:09 |
jasondotstar | is that rt? nice! | 17:09 |
vkmc | hey jasondotstar! | 17:25 |
vkmc | welcome :) | 17:25 |
jasondotstar | thx | 17:25 |
jasondotstar | feeling welcome already | 17:25 |
vkmc | that sounds great | 17:25 |
vkmc | I read you are interested in contributing to Zaqar | 17:25 |
vkmc | so please let me know if there is something I can do to make that easier for you to get involved | 17:26 |
jasondotstar | sure. | 17:26 |
jasondotstar | I'm starting to poke around the low-hanging fruit in the bug triage | 17:26 |
vkmc | that's cool | 17:27 |
jasondotstar | is there a roadmap for the project stashed somewhere? | 17:28 |
vkmc | let me see.. I don't know how updated is that | 17:29 |
vkmc | yeah, its outdated | 17:30 |
vkmc | but well, I can tell you right now that our main work for K-1 will be to implement a persistent transport (websockets), notifications, capabilities | 17:32 |
vkmc | and possibly... getting rid of the queue concept | 17:32 |
vkmc | we are not longer a queing service, but a messaging service | 17:32 |
vkmc | a more detailed information about those changes can be checked here https://github.com/openstack/zaqar-specs/tree/master/specs/kilo/approved | 17:33 |
*** openstackgerrit has quit IRC | 17:34 | |
*** openstackgerrit has joined #openstack-zaqar | 17:34 | |
vkmc | we also have some pending tasks that might be interesting to tackle on kilo | 17:35 |
vkmc | as the integration with the OpenStack Dashboard (Horizon) | 17:35 |
jasondotstar | cool. | 17:37 |
*** openstackgerrit has quit IRC | 17:49 | |
*** openstackgerrit has joined #openstack-zaqar | 17:49 | |
*** amitgandhinz has quit IRC | 18:22 | |
*** JAHoagie has joined #openstack-zaqar | 18:27 | |
*** sgotliv has quit IRC | 18:57 | |
*** boris-42 has joined #openstack-zaqar | 19:03 | |
*** amitgandhinz has joined #openstack-zaqar | 19:16 | |
* vkmc lurks | 19:43 | |
*** ametts has quit IRC | 20:27 | |
*** achanda has quit IRC | 20:37 | |
*** ametts has joined #openstack-zaqar | 20:41 | |
*** jchai is now known as jchai_afk | 20:45 | |
*** jasondotstar has quit IRC | 20:46 | |
*** jchai_afk is now known as jchai | 20:52 | |
*** amalagon is now known as alagon | 20:56 | |
*** X019 has joined #openstack-zaqar | 20:57 | |
*** jchai has quit IRC | 21:13 | |
*** cpallares has quit IRC | 21:14 | |
*** boris-42 has quit IRC | 21:33 | |
*** sriram has quit IRC | 21:40 | |
*** boris-42 has joined #openstack-zaqar | 21:55 | |
*** JAHoagie has quit IRC | 22:00 | |
*** mpanetta has quit IRC | 22:13 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/zaqar: Updated from global requirements https://review.openstack.org/140963 | 22:22 |
*** flwang1 has joined #openstack-zaqar | 22:31 | |
openstackgerrit | Merged openstack/zaqar: Add capabilities property to the DataDriver https://review.openstack.org/135637 | 22:59 |
openstackgerrit | Merged openstack/zaqar: Split Control and Data planes of Storage layer https://review.openstack.org/134910 | 22:59 |
*** amitgandhinz has quit IRC | 23:00 | |
*** flwang1 has quit IRC | 23:21 | |
openstackgerrit | Merged openstack/zaqar: Wrap abstract method with base methods https://review.openstack.org/139073 | 23:30 |
*** ametts has quit IRC | 23:30 | |
*** flwang1 has joined #openstack-zaqar | 23:48 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!