| kota_ | notmyname: hmm...eventlet==0.16.1 doesn't work in my envrionment but i got another issue test_get_conns stucked long time. | 01:48 |
|---|---|---|
| kota_ | notmyname: anyways, nosetests test/unit/common/test_memcached.py with eventlet>=0.17.1 worked for me. | 01:50 |
| kota_ | oh shit, wrong channel | 01:53 |
| kota_ | please ignore them | 01:54 |
| *** takashi_ has joined #openstack-storlets | 06:42 | |
| *** eranrom has joined #openstack-storlets | 08:19 | |
| eranrom | We are thinking of dropping query string for storlets parameters in favor of headers. The key reason being that with headers it is more natural to have 'X-Storlet-Parameter' prefix then on the quesry string. | 08:23 |
| eranrom | Any thoughts, objectstions? | 08:23 |
| eranrom | objections? | 08:23 |
| takashi_ | eranrom: I don't. | 08:43 |
| eranrom | takashi,: Are you using parameters today? | 08:45 |
| takashi_ | Using query is sometime confusing, because it is not so clear which module will use that parameter. | 08:45 |
| eranrom | takashi,: I absolutly Agree. In you PoC does your storlet takes parameters? | 08:46 |
| takashi_ | eranrom: I have some plan about using parameter, because sometimes it may save me many storlets, which are not so different | 08:46 |
| takashi_ | eranrom: In our PoC, we just started with very simple Storlet, so I think we don't have to use parameters | 08:47 |
| eranrom | takashi,: alright. Doron is adding this feature (parameters as headers). We will depracate the query string parameters a bit later. | 08:47 |
| takashi_ | eranrom: ok | 08:48 |
| takashi_ | Using parameters are very useful. we can switch the algorithm to detect features in images. we can flexibly change the size of shrinked images/videos. | 08:54 |
| takashi_ | s/are/is/ | 08:55 |
| takashi_ | we don't so have to use parameters in our PoC, but I believe we can't create useful applications without parameter mechanisms | 08:55 |
| *** takashi_ has quit IRC | 09:00 | |
| eranrom | takashi,: Absolutely. We currently support paramets, and will continue to support them. The question is whether its via query string or headers. The current code supports parameters via query string, with doron's latest patch we will also support headers. | 10:36 |
| eranrom | s/paramets/parameters | 10:36 |
| openstackgerrit | Doron Chen proposed openstack/storlets: dd support for sending parameters through headers https://review.openstack.org/265644 | 10:51 |
| openstackgerrit | Doron Chen proposed openstack/storlets: add support for sending parameters through headers https://review.openstack.org/265644 | 10:56 |
| openstackgerrit | Doron Chen proposed openstack/storlets: add support for sending parameters through headers https://review.openstack.org/265644 | 11:06 |
| *** openstackgerrit has quit IRC | 11:43 | |
| *** openstackgerrit has joined #openstack-storlets | 11:45 | |
| *** takashi_ has joined #openstack-storlets | 12:45 | |
| eranrom | Hi. Anybody here | 13:00 |
| takashi_ | Hi :-) | 13:00 |
| eranrom | Hi Takashi. | 13:00 |
| kota_ | hello | 13:01 |
| eranrom | Hi Kota | 13:01 |
| eranrom | Lets start | 13:03 |
| eranrom | I do not have anything special | 13:03 |
| takashi_ | I have some updates about fix in swift | 13:03 |
| takashi_ | First, about patch 256201, I discussed about this patch last week with Kota, and found some items to improve. | 13:04 |
| patchbot | takashi_: https://review.openstack.org/#/c/256201/ - [WIP] Fix proxy-server's support for chunked trans... | 13:04 |
| eranrom | takashi,: Should I try it? or wait for some more updates? | 13:05 |
| eranrom | takashi,: I have seen it and it had an import of _is_chunked inside the function which I was not sure if its common | 13:07 |
| takashi_ | eranrom: I'll update it. It's almost done, and I have to solve one item. | 13:07 |
| kota_ | takashi_: i think the biggest issue is proxy redirect when time out happen during draining body, right? | 13:08 |
| takashi_ | eranrom: _is_chunked should be used for Response and Request, but these two has no relations like extending | 13:08 |
| eranrom | takashi,: alright so as soon as I see a patch I will review and test with Storlets | 13:08 |
| takashi_ | kota_: right | 13:09 |
| takashi_ | In that discussion, we found some items we have to fix in storlets | 13:09 |
| takashi_ | two items | 13:09 |
| takashi_ | bug 1534414 is the first one, and what kota_ mentioned | 13:10 |
| openstack | bug 1534414 in storlets "GET with Storlet execution generates wrong content with automatic retry in proxy-server" [Undecided,New] https://launchpad.net/bugs/1534414 - Assigned to Takashi Kajinami (kajinamit) | 13:10 |
| kota_ | eranrom: swfit proxy-server has a capability to redirect to another object server to resume to get the body when the connection between proxy and object-sever disconnected | 13:10 |
| kota_ | in transfer data. | 13:10 |
| takashi_ | that resuming causes some problems with storlets | 13:11 |
| kota_ | for now. | 13:11 |
| takashi_ | yes | 13:11 |
| kota_ | takashi_: oh yeah, you seems to describe in the bug report. | 13:12 |
| takashi_ | when client get object with storlet execution, proxy-server passes it to object-server, and object-server will run storlet | 13:12 |
| takashi_ | but when proxy gets timeout, it will send another request to another object-server, with Range header | 13:13 |
| takashi_ | that range header will cause some problem, because object-server will not run storlet after resuming. | 13:13 |
| eranrom | takashi,: Good catch! interesting one. Any thought on how to fix? | 13:13 |
| takashi_ | eranrom: yes. we think we have to return error in the next object-server | 13:15 |
| eranrom | takashi,: So as to fail the while request - right? | 13:15 |
| kota_ | eranrom: right | 13:16 |
| takashi_ | eranrom: yes. It may surpress resuming feature | 13:16 |
| eranrom | sounds good! Do you think we should do this to any range request or only to 'redirected range requests'? | 13:17 |
| kota_ | eranrom: we couldn't expect object-servers except one the requset get first can return such a resuming body. | 13:17 |
| takashi_ | we can't make sure we can get remaining content with that range header now, so I think we have to prevent swift from resuming | 13:17 |
| takashi_ | I'm talking about the same thing as kota_ discribed | 13:18 |
| takashi_ | eranrom: I think we should do only for redirected range request | 13:18 |
| kota_ | takashi_: in current idea, x-run-storlets will be removed at header *if storlets run there*, right? | 13:18 |
| takashi_ | kota_: yes | 13:19 |
| kota_ | i.e. if run at proxy, object-servers will get the request w/o run-storlets | 13:19 |
| takashi_ | and we can detect the redirection case, by checking if we have both of x-run-storlets and range. | 13:19 |
| kota_ | at object-server | 13:19 |
| takashi_ | kota_: right | 13:20 |
| eranrom | question: | 13:20 |
| kota_ | eranrom: go ahead | 13:20 |
| eranrom | What happen if today we send a request with range and x-run-storlet, can it get executed on an object server? or once we see range we always execute this on the proxy? (I do not remeber) | 13:21 |
| takashi_ | we always execute on the proxy | 13:21 |
| eranrom | ok, good. | 13:22 |
| eranrom | So now I get the idea :-) | 13:22 |
| kota_ | :-) | 13:22 |
| eranrom | What was the second issue? | 13:22 |
| takashi_ | bug 1534455, and it's very small thing | 13:23 |
| openstack | bug 1534455 in storlets "Storlets should return 206 instead of 200 for Range request" [Undecided,New] https://launchpad.net/bugs/1534455 | 13:23 |
| eranrom | ok gotcha, thanks | 13:23 |
| takashi_ | that's what we found thinking about range | 13:24 |
| takashi_ | s/thinking/when thinking/ | 13:24 |
| eranrom | I see :-) again, good catch! | 13:24 |
| eranrom | I need to have notifications on the bugs. My bad ... | 13:24 |
| takashi_ | it's not so big problem, and I can fix it easily | 13:25 |
| takashi_ | but we have to be careful about RFC when fixing it, | 13:25 |
| takashi_ | because range header should includes start/end information | 13:25 |
| kota_ | eranrom: you can get subscription from launchpad :) | 13:25 |
| takashi_ | but client may get the content in different length (!= end - start) | 13:26 |
| eranrom | eranrom: right, it just that I should have done this by now :-) | 13:26 |
| eranrom | takashi,: right. I guess this is where we need to be carefult abour the spec - right? | 13:27 |
| takashi_ | eranrom: yes | 13:27 |
| eranrom | takashi,: I am not sure that the RFC even covers such a case. | 13:27 |
| eranrom | takashi,: I need to look. | 13:27 |
| takashi_ | eranrom: unfortunately, RFC does not seem to cover range request with chunked transfering | 13:28 |
| kota_ | hehe | 13:28 |
| eranrom | takashi,: so the options are either return 206 or 412 (unless the storlet did not change anything) or we can say that if this is not covered by the spec we can do anything :-) | 13:29 |
| eranrom | An alternative is to refer to x-run-storlet with range as the storlet output rather the input. so we always run the storlet on the entire object and output only the requested range | 13:30 |
| eranrom | in which case we can return a content-length | 13:30 |
| eranrom | and 412 if the computation does not have such a range... | 13:31 |
| takashi_ | another alternative is replace range header with something like storlet-run-over-range | 13:31 |
| takashi_ | header | 13:31 |
| eranrom | actually we can have both :-) | 13:32 |
| takashi_ | with that header we can return 200 OK. that does not violate RFC | 13:32 |
| eranrom | right: here is what I gathered so far: | 13:33 |
| eranrom | Option 1: have a new header called x-storlet-run-over-range which will do what we do today when we have both X-run-storlet AND range headers | 13:34 |
| eranrom | Option 2: Support X-Run_Storlet + range where the range will be the storlet output | 13:35 |
| eranrom | We can decide to support both ot none. | 13:35 |
| eranrom | I currently do not see a strong use case for any of them... thoughts? | 13:35 |
| eranrom | s/ot/or | 13:36 |
| takashi_ | eranrom: that's based on the usecase... | 13:37 |
| eranrom | takashi,: you mean your use case? | 13:37 |
| takashi_ | eranrom: yes, but they are no so rigid ones. just ideas in me | 13:38 |
| takashi_ | when we use storlet to encode movie file in video application, clients want to get the range of the output | 13:38 |
| takashi_ | but in other cases, clients want to get the output which are generated from a part of a object | 13:39 |
| kota_ | takashi_: on that case, it might be good run storlets at object-server :) | 13:39 |
| takashi_ | s/other/another | 13:39 |
| takashi_ | kota_: yes | 13:39 |
| eranrom | and have the semantics of run-storlet-on-range - right? | 13:40 |
| kota_ | eranrom!!! | 13:40 |
| kota_ | on-range or run-storlets-and-get-this-range | 13:41 |
| kota_ | whatever | 13:41 |
| takashi_ | let me make more clear about this topic | 13:41 |
| takashi_ | so in video cases, we want to get the range of result, and "range" header is suitable for that. for client, there are no difference from getting range of the object. | 13:42 |
| takashi_ | in the latter cases, for example when object includes may record in it and clients want to get result generated from some of theses records, x-run-storlet-over-ragnge will be suitable, I think | 13:43 |
| takashi_ | s/may/many | 13:43 |
| takashi_ | range means the range of output, and x-run-storlet-over-range means the range of input | 13:44 |
| takashi_ | does it make sence? | 13:45 |
| eranrom | takashi: Absolutley. I also agree with kota_ that for the case of get the range of result using the range header we should probably allow to run this on the object sevrer. | 13:45 |
| *** takashi__ has joined #openstack-storlets | 13:47 | |
| *** [1]eranrom has joined #openstack-storlets | 13:47 | |
| [1]eranrom | Sorry, got disconnected | 13:47 |
| takashi_ | eranrom: np | 13:47 |
| [1]eranrom | so I think just before disconnecting, I was thinking whether we can do both type of range semantics in the object server | 13:49 |
| takashi_ | range of input is easier thing. that's already done. | 13:50 |
| [1]eranrom | takashi: agree, but is supported only on proxy. Not saying its urgent to support it also on object though. | 13:51 |
| kota_ | eranrom: +1 | 13:52 |
| takashi_ | eranrom: +1 | 13:52 |
| [1]eranrom | oh, but then If I got takashi right yhen its more natural to support that via a proprietary header such as run-storlet-on-range and bypass RFC problems. | 13:53 |
| [1]eranrom | or am I confused.... | 13:54 |
| takashi_ | eranrom: that's right | 13:54 |
| [1]eranrom | takashi,ok thanks! | 13:54 |
| *** wbhuber_ has joined #openstack-storlets | 13:54 | |
| *** hrou_ has joined #openstack-storlets | 13:55 | |
| *** takashi has quit IRC | 13:56 | |
| *** wbhuber has quit IRC | 13:56 | |
| *** eranrom has quit IRC | 13:56 | |
| *** hrou has quit IRC | 13:56 | |
| *** [1]eranrom is now known as eranrom | 13:56 | |
| eranrom | takashi,: so perhaps what we can do in the short term is to add support for run-storlet-on-range (which exists in a sense), and reply with 400 bad request in the case of x-run-storlet with range until we support the second semantics | 13:56 |
| kota_ | irc seems not so stable... | 13:57 |
| eranrom | yeah | 13:57 |
| takashi_ | eranrom: and add some notes about range execution in doc | 13:57 |
| eranrom | absolutely. | 13:57 |
| kota_ | takashi_: +1 | 13:58 |
| eranrom | I can do the doc stuff. | 13:58 |
| eranrom | takashi_, kota_: Do we want first to write a summary of this in the bug, and continue from there? | 13:58 |
| kota_ | eranrom: could you please? | 13:59 |
| eranrom | kota_,Absolutly! | 13:59 |
| kota_ | eranrom: thanks! | 13:59 |
| takashi_ | eranrom: thanks :-) | 14:00 |
| eranrom | thank you! | 14:00 |
| takashi_ | that's all from me today | 14:00 |
| kota_ | eranrom: one thing | 14:01 |
| eranrom | kota_,go ahead | 14:01 |
| kota_ | eranrom: when do you expect the submit deadline for us for austin summit? | 14:01 |
| kota_ | i know, officially Feb 1st but | 14:01 |
| kota_ | i think we should add some description for submitter each other before make it finalize. | 14:02 |
| eranrom | kota_,: you mean improve the talk abstract? | 14:03 |
| kota_ | eranrom: yes | 14:03 |
| kota_ | and currently i am working on getting approval in my company. | 14:04 |
| kota_ | oh, no | 14:04 |
| kota_ | not improve, i think | 14:04 |
| kota_ | ah, it might include also improve | 14:05 |
| kota_ | all things | 14:05 |
| takashi_ | at least, we have to add some information about presenters, as kota_ said | 14:06 |
| eranrom | ok gotcha | 14:06 |
| kota_ | (just want to make sure schedule roughly) | 14:06 |
| kota_ | sorry my bad for explanation. | 14:06 |
| eranrom | no worries. so you would like to circulate speakers information between us, before we submit? | 14:07 |
| eranrom | or the list of the actual sepakers? | 14:08 |
| kota_ | later one | 14:09 |
| eranrom | Well, its definately not urgent for me. | 14:10 |
| kota_ | and plus, when will you register online. | 14:10 |
| kota_ | i mean register as submit a talk | 14:10 |
| eranrom | So I need to double check that I have a green light from IBM. Once I have it, I will let you know, so that once you have a green light we can finalyze the abstract and register the speakers. | 14:10 |
| kota_ | all right, gotcha. | 14:11 |
| kota_ | expecting i could know it next week. | 14:12 |
| eranrom | kota_,: Great! | 14:12 |
| eranrom | I will update on IRC as soon as I have news | 14:12 |
| kota_ | eranrom: thanks | 14:12 |
| eranrom | kota_,: sure! | 14:13 |
| kota_ | All? | 14:17 |
| eranrom | yep, thanks a lot\ | 14:17 |
| kota_ | :-) | 14:18 |
| takashi_ | thanks :-) | 14:18 |
| eranrom | good night | 14:18 |
| takashi_ | eranrom: you too | 14:21 |
| *** takashi_ has quit IRC | 14:24 | |
| *** hrou_ has quit IRC | 15:19 | |
| *** hrou has joined #openstack-storlets | 15:20 | |
| *** eranrom has quit IRC | 17:05 | |
| *** takashi has joined #openstack-storlets | 18:13 | |
| *** peluse_ has joined #openstack-storlets | 18:18 | |
| *** takashi__ has quit IRC | 18:22 | |
| *** peluse has quit IRC | 18:22 | |
| *** hrou has quit IRC | 18:57 | |
| *** hrou has joined #openstack-storlets | 18:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!