| *** eranrom has quit IRC | 04:01 | |
| *** eranrom has joined #openstack-storlets | 06:35 | |
| *** eranrom has quit IRC | 07:50 | |
| *** eranrom has joined #openstack-storlets | 07:57 | |
| *** eranrom has quit IRC | 08:16 | |
| *** eranrom has joined #openstack-storlets | 08:23 | |
| openstackgerrit | Eran Rom proposed openstack/storlets: WIP: Allow to run a single range request on the object server https://review.openstack.org/319640 | 10:31 |
|---|---|---|
| *** openstackgerrit has quit IRC | 11:18 | |
| *** openstackgerrit has joined #openstack-storlets | 11:19 | |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: DO NOT MERGE: This is for testing https://review.openstack.org/320945 | 12:24 |
| *** takashi has joined #openstack-storlets | 12:28 | |
| kota_ | hello | 13:01 |
| eranrom | Hï | 13:01 |
| eranrom | Hï | 13:01 |
| eranrom | Hi | 13:01 |
| takashi | hello | 13:01 |
| eranrom | Hey | 13:01 |
| eranrom | Let's start | 13:01 |
| eranrom | First thing, I apologize I did not review the recent refactoring patch | 13:02 |
| eranrom | Got some deadlines... | 13:02 |
| eranrom | but will do asap | 13:02 |
| takashi | eranrom: np | 13:02 |
| kota_ | np | 13:02 |
| eranrom | Other then that I wanted to share my plans for the next month (after leaving IBM) | 13:03 |
| eranrom | (1) I want to try and make the project official | 13:03 |
| eranrom | (2) send a talk to the Barcelona summit | 13:03 |
| eranrom | (3) renew the documentation (hopefully next week) | 13:03 |
| kota_ | \o/ awesome | 13:03 |
| eranrom | (4) Plan a release. | 13:04 |
| eranrom | I think that a release should have the following: | 13:04 |
| eranrom | 1. Statement of which Swift we are tested with | 13:04 |
| eranrom | 2. An official build of the SCOmmon jar | 13:05 |
| eranrom | 3. A Docker container poeple can download that has s2aio inside so as storlet writers can quickly write and test storlets | 13:05 |
| eranrom | Would be happy to get more opinions | 13:05 |
| kota_ | what's SCOmmon? | 13:06 |
| eranrom | oh, this is the .jar you need to develop a storlet. So as a storlet developer you will not need to build or even to clone the storlets repo. | 13:07 |
| eranrom | the next step would be to have an eclipse plugin for storlets deveopment | 13:08 |
| eranrom | one that can work with e.g. the s2aio Docker image you download | 13:08 |
| kota_ | gotcha sonds like template app. | 13:08 |
| kota_ | sounds | 13:08 |
| takashi | makes sense to me :-) | 13:09 |
| eranrom | yep | 13:10 |
| takashi | eranrom: do you have any target date for that release? | 13:10 |
| takashi | eranrom: The reason of that question is that I'm wondering if we should deal with copy middleware before that release. | 13:10 |
| kota_ | yeah. | 13:10 |
| eranrom | well, I thought I should do this no later then the next openstack release. | 13:11 |
| eranrom | not sure then this release is | 13:11 |
| kota_ | Swift will have a release soon that includes copy middleware. | 13:11 |
| takashi | eranrom: agree | 13:11 |
| kota_ | notmyname said in the previous meeting, he want to make it before we are working on encryption merge. | 13:12 |
| takashi | kota_: ok | 13:12 |
| eranrom | right, and the next OS release will be based on the Swift release with COPY so we definately need that | 13:12 |
| eranrom | If any of you can help with that then great. Othewise, lets push Takashi's refactoring, ranges and then I can do COPY | 13:13 |
| takashi | eranrom: ok | 13:14 |
| kota_ | I'm willing to be a voluntieer for that. Which patch is waiting for reviews? | 13:14 |
| eranrom | currently Takashi's refactoring of the protocol classes, and my range stuff which needs a little more work | 13:15 |
| takashi | kota_ patch 314140 and patch 319640 | 13:15 |
| patchbot | takashi: https://review.openstack.org/#/c/314140/ - storlets - Refactor Storlet protocol classes | 13:15 |
| patchbot | takashi: https://review.openstack.org/#/c/319640/ - storlets - WIP: Allow to run a single range request on the ob... | 13:15 |
| kota_ | (probably, I can do that since this weekend hopefully after my big patch for swift3 landed into master :/) | 13:15 |
| eranrom | kota_,: congrats on landing this. | 13:16 |
| takashi | kota_: great! | 13:16 |
| eranrom | takashi,: thanks for the links | 13:16 |
| kota_ | eranrom: not yet :/ sorry my lack of English. | 13:16 |
| eranrom | you did mention after | 13:17 |
| eranrom | sorry | 13:17 |
| eranrom | :-) | 13:17 |
| takashi | I'm going to refactor gateway classes after that refactoring about protocols land | 13:17 |
| eranrom | takashi,: sounds great | 13:17 |
| takashi | and I hope that makes us to maintain handler codes easily :-) | 13:17 |
| takashi | including that copy work | 13:17 |
| eranrom | takashi,: curious to see what you have in ind, will wait for the patch. | 13:18 |
| eranrom | Good time to talk about tomorrow's call? | 13:18 |
| takashi | eranrom: yes | 13:19 |
| eranrom | My plan is to present two concrete use cases (1) the spark pushdown usecase (2) a security related use case we are exploring with an EU company that does analytics on electricity IOT data | 13:20 |
| eranrom | and then have an open discussion | 13:20 |
| eranrom | anything else you would like to cover? | 13:20 |
| takashi | I suppose that case 1 brings you a reason why you are working about patch 319640 | 13:21 |
| patchbot | takashi: https://review.openstack.org/#/c/319640/ - storlets - WIP: Allow to run a single range request on the ob... | 13:21 |
| kota_ | sonds great. | 13:21 |
| eranrom | takashi,: exactly :-) | 13:21 |
| kota_ | plus, if some load map for storlets community, it would be great. | 13:21 |
| kota_ | includes, features, position at openstack cummunity. | 13:21 |
| kota_ | something like that, if possible. | 13:22 |
| eranrom | kota_,: what do you mean by "some load map for storlets community" | 13:22 |
| kota_ | I mean if we could have the discussion (or IBM plan) about the load map in the tommorow meeting, it would be great. | 13:23 |
| kota_ | load map means... | 13:23 |
| kota_ | like priority feature? | 13:23 |
| eranrom | gotcha. | 13:23 |
| kota_ | what we can expect in the next. | 13:24 |
| kota_ | curcle back to takashi's mention above. | 13:24 |
| eranrom | right, so I will not be able to answer that, but perhaps others on the call would be. We should have the same group as last week | 13:24 |
| kota_ | eranrom: great | 13:25 |
| eranrom | alright, so I guess this is it from my side | 13:25 |
| kota_ | takashi: what did you mention about patch 319640/ | 13:25 |
| patchbot | kota_: https://review.openstack.org/#/c/319640/ - storlets - WIP: Allow to run a single range request on the ob... | 13:25 |
| kota_ | ? | 13:25 |
| kota_ | eranrom: yeah, thanks :) | 13:25 |
| kota_ | eranrom: i know you might be impossible to say about that, that is why I said as 'if possible' | 13:26 |
| takashi | eranrom: I saw the commit message of that patch says it is needed for spark pushdown usecase. | 13:27 |
| takashi | sorry, not for eranrom, but for kota_ | 13:27 |
| kota_ | takashi: ah, that's interesting. | 13:28 |
| takashi | kota_: That can improve concurrency for range request, by pushing storlet execution to storage servers from proxy servers, which is effective for access pattern generated by spark | 13:28 |
| eranrom | kota_,: sure. no sweat | 13:28 |
| takashi | as far as I understand | 13:28 |
| eranrom | takashi,: you are correct. there is an additional point: save the object->proxy bandwidth | 13:29 |
| takashi | eranrom: yes! | 13:30 |
| kota_ | takashi: does it act as concurrent? | 13:30 |
| kota_ | I'm looking at storlet_handler.py | 13:31 |
| takashi | kota_: if you have more storage servers than proxy servers. | 13:31 |
| kota_ | ok, let me have more time to look at deeply. | 13:31 |
| kota_ | takashi: though you might notice, recently some swift middleware got concurrent requests improvement | 13:32 |
| kota_ | just FYI though. | 13:32 |
| eranrom | takashi,: you are correct. in terms of concurrency we get from #proxy concurrency to #object concurrency | 13:32 |
| kota_ | takashi: patch 311817 | 13:33 |
| patchbot | kota_: https://review.openstack.org/#/c/311817/ - swift - Allow concurrent bulk deletes (MERGED) | 13:33 |
| takashi | kota_: for example, about bulk delete, right? | 13:33 |
| kota_ | takashi: right | 13:33 |
| takashi | kota_: Yes, I saw that patch, but didn't have a deep look yet. | 13:33 |
| kota_ | takashi: np, just FYI. | 13:34 |
| takashi | kota_: thanks! will check that. | 13:35 |
| eranrom | btw - are you aware of any work being done on partition power increase in Swift? | 13:37 |
| kota_ | i think it is witll WIP, not sure though. | 13:38 |
| takashi | eranrom: I'm not sure about the latest status, but I suppose now swift-ring-builder supports increasing part power of ring file | 13:38 |
| eranrom | interesting. Do you know is working on that? | 13:39 |
| takashi | I suppose some more works are required in swift deamons, to deal with that increasing | 13:39 |
| eranrom | yep | 13:39 |
| takashi | I know cschewede is working on that. I remember he was discussing about that topic with clayg at the last hackathon. | 13:40 |
| eranrom | takashi,ok thanks | 13:41 |
| eranrom | Anything else for today? | 13:41 |
| takashi | nothing special from me | 13:41 |
| takashi | eranrom: this week I can attend to the meeting, and I'm lookong forward to hear you latest work tommorow | 13:42 |
| eranrom | takashi,: great! | 13:43 |
| eranrom | so unless we have something else, I guess this is it for today. and will talk to you tomorrow. | 13:44 |
| kota_ | yup, thanks eranrom :) | 13:44 |
| takashi | eranrom, kota_: thanks. see you tommorow :-) | 13:45 |
| eranrom | thanks for joining! ttyl. | 13:45 |
| *** takashi has quit IRC | 14:14 | |
| *** eranrom has quit IRC | 14:24 | |
| *** takashi has joined #openstack-storlets | 14:45 | |
| *** takashi has quit IRC | 14:54 | |
| *** eranrom has joined #openstack-storlets | 16:39 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!