| *** takashi has joined #openstack-storlets | 05:18 | |
| *** openstackgerrit has quit IRC | 09:03 | |
| *** openstackgerrit has joined #openstack-storlets | 09:04 | |
| *** eranrom has joined #openstack-storlets | 09:23 | |
| *** takashi has quit IRC | 09:30 | |
| *** eranrom has quit IRC | 10:16 | |
| *** takashi has joined #openstack-storlets | 12:23 | |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Use contextmanager for storlet invocation descriptors https://review.openstack.org/294358 | 12:33 |
|---|---|---|
| *** eranrom has joined #openstack-storlets | 12:51 | |
| eranrom | Hi | 13:00 |
| kota_ | hello | 13:00 |
| eranrom | Hi Kota | 13:00 |
| kota_ | eranrom: :) | 13:00 |
| eranrom | Do you know if Takashi plans on joining? | 13:01 |
| takashi | hello | 13:01 |
| eranrom | Hi Takashi | 13:01 |
| kota_ | takashi: o/ | 13:01 |
| takashi | eranrom, kota_: o/ | 13:01 |
| eranrom | I wrote a couple of items in the agenda: https://wiki.openstack.org/wiki/Meetings/Storlets | 13:02 |
| kota_ | ok, looking | 13:02 |
| takashi | ok | 13:02 |
| eranrom | Unfortunatelly, I do not think URV guys are joining today, but I can update you on their doings. | 13:03 |
| kota_ | eranrom: what does URV mean? | 13:03 |
| eranrom | URV is a university in spain, we are working with on a eu reseach project | 13:04 |
| kota_ | ah, ok. cool. | 13:05 |
| eranrom | so they have done a couple of cool things: | 13:06 |
| eranrom | 1. They added storlet pipelining | 13:06 |
| eranrom | which means that one can invoke more then one storlet at a time - say first compress and then encrypt | 13:07 |
| eranrom | 2. They added the ability to run on object also in PUT - I still need to learn about the use case here - which means the computation is made X3 | 13:08 |
| eranrom | 3. they added policies mechanism that allows to e.g. run a decompression storlet automatically if the object is found to be compressed | 13:08 |
| eranrom | I am trying to encourage them to contribute back, at least some of the stuff... | 13:08 |
| takashi | eranrom: cool | 13:09 |
| eranrom | yeah, would be really great to increase our community :-) | 13:10 |
| kota_ | sounds awesome :-) | 13:10 |
| eranrom | Being in spain, they are very interested to make something for the next summit in Barcelona | 13:10 |
| eranrom | Otherwise, pluse asked me if we are an official OS project. Aparently, we are pretty much qualified | 13:11 |
| eranrom | Wanted to ask you for your take on this. | 13:11 |
| kota_ | I'm with :-) | 13:12 |
| takashi | agree | 13:12 |
| eranrom | ok, cool. I will have a deper look at the requirements and get back. what I recall so far is that we need: | 13:13 |
| eranrom | 1. to have a mailing list | 13:13 |
| kota_ | when asking to my boss about that, he don't have opposite opinion, just curious how it works for us/coummity. | 13:13 |
| kota_ | s/don't/doesn't/ | 13:13 |
| eranrom | kota_,: I am not sure exactly what it means. nor am I sure that the current OS documentation is very clear on that. Anyway I will have a deeper look and also send the link to the relevant OS docs | 13:14 |
| takashi | eranrom: Requirements for OpenStack official project was changed when big-tent was introduced, and I'm not so sure about the latest one | 13:15 |
| eranrom | I hope that what pluse sent me is the latest one. | 13:15 |
| kota_ | eranrom: that sounds enough thanks. | 13:15 |
| takashi | but I think it's more open for each community than before :-) | 13:15 |
| eranrom | which is good... | 13:15 |
| eranrom | alright. next is the log upload feature | 13:16 |
| eranrom | The original use case was for end users to be able to have a log of the actual execution of their storlets | 13:17 |
| eranrom | I am not sure it works in all cases, and I am less sure that we have done a proper desing work for it. | 13:17 |
| eranrom | at the time (before we were open) we needed something quick to have the ability | 13:17 |
| eranrom | for an internal Hackathon | 13:18 |
| eranrom | The reason I brought it up, is that Takashi made some good remarks on the code, and I was not sure we want to continue maintaining this | 13:18 |
| eranrom | thoughts? | 13:18 |
| takashi | eranrom: Does that feature work now? I think I saw a bug report which says we should fix that. | 13:18 |
| takashi | (I didn't test it yet | 13:19 |
| eranrom | well, I am not sure. I think that it may work for either GET or PUT but not both (and surely not in COPY) | 13:19 |
| takashi | eranrom: ok | 13:20 |
| eranrom | also, I think that a better approach would be - not ot upload on every execution, but rather expose an API to get it. | 13:20 |
| eranrom | I see two options: | 13:20 |
| eranrom | 1. We kill the feature | 13:20 |
| eranrom | 2. We re-design - re-implement | 13:20 |
| eranrom | I guess the question is whether you thing this is important | 13:21 |
| eranrom | s/thing/think | 13:21 |
| takashi | eranrom: let me sure about my understanding | 13:21 |
| takashi | Using log feature, storlet log, generated by storlet java application, is stored as a object in the swift after storlet execution | 13:22 |
| eranrom | takashi: correct | 13:22 |
| takashi | so with that feature, application developers can see how their application works on storlet platform | 13:23 |
| eranrom | right | 13:23 |
| kota_ | ah, got it now :) | 13:23 |
| eranrom | :-) | 13:23 |
| takashi | I think the feature is needed, because these application logs are not for operators, but application developers | 13:24 |
| eranrom | agree. | 13:24 |
| takashi | It helps them to know how their application works on the storlet. They can find the reason why their application does not work in error case. | 13:25 |
| eranrom | right. | 13:25 |
| eranrom | I have a couple of thoughts here: | 13:25 |
| eranrom | 1. Have a tool where developers can test their storlet locally. Perhaps an embedding of the storlet engine outside of Swift | 13:26 |
| eranrom | 2. Improve the logging so that each onvocation gets to a different file - currently, parallel invocation on the same node get to the same log file | 13:27 |
| eranrom | 3. consider changing the API so as to get it on demand rather then upload upon execution | 13:28 |
| eranrom | So it needs some re-design IMO | 13:28 |
| kota_ | exactly | 13:28 |
| takashi | for 1, yes. It's very useful :-) | 13:29 |
| takashi | for 2, yes. Now I think adding transaction-id to log name may work, but need some more thought about it. | 13:30 |
| eranrom | ok, I am happy to write a new desing for review. Might take a couple of weeks | 13:30 |
| takashi | for 3, I'm not sure. I think uploading a log file as swift object is a good idea, for me. | 13:30 |
| takashi | eranrom: ok. I'm waiting for that, or I'll send you my idea if I get. | 13:31 |
| eranrom | alright. So once I have something I will post it in IRC | 13:32 |
| takashi | eranrom: thx :-) | 13:33 |
| eranrom | sure. | 13:33 |
| eranrom | Anything else for today? | 13:33 |
| takashi | a small one | 13:33 |
| takashi | If you have some time, please review patch 294358 | 13:34 |
| patchbot | takashi: https://review.openstack.org/#/c/294358/ - storlets - Use contextmanager for storlet invocation descriptors | 13:34 |
| takashi | This is the last patch which introduces contextmanager for fds | 13:34 |
| eranrom | sure, will do - hopefully today | 13:34 |
| takashi | I'm going to do two things after that patch get merged | 13:34 |
| takashi | 1. Make sure all fds are closed in error case | 13:34 |
| kota_ | nice | 13:34 |
| takashi | 2. Refactor InvocationProtocols. I'm writing my draft as patch 296422 now, but should rebase it. | 13:35 |
| patchbot | takashi: https://review.openstack.org/#/c/296422/ - swift - Use logger.exception when generating logs about ex... | 13:35 |
| takashi | oh, sorry. wrong one | 13:35 |
| eranrom | souds great. | 13:35 |
| takashi | patch 295875 | 13:35 |
| patchbot | takashi: https://review.openstack.org/#/c/295875/ - swift - [WIP] Pass logger instances to AccountBroker/Conta... | 13:35 |
| takashi | I think I should be more careful... | 13:36 |
| eranrom | :-) no worries | 13:36 |
| takashi | patch 294406 | 13:36 |
| patchbot | takashi: https://review.openstack.org/#/c/294406/ - storlets - Merge StorletInvocationXXXProtocol | 13:36 |
| takashi | Yes, this is the right one! | 13:36 |
| eranrom | yep. | 13:36 |
| eranrom | Do you think there may be a way to automate the fd leak test? | 13:36 |
| eranrom | I never gave it any thought | 13:36 |
| eranrom | might be impossible | 13:37 |
| takashi | eranrom: I'm wondering if we can add some checking in unit tests, but not sure now | 13:37 |
| eranrom | ok | 13:38 |
| takashi | that's all from me today | 13:38 |
| eranrom | takashi,: thanks. BTW also thank you for reviewing the container sync patch. | 13:39 |
| takashi | eranrom: you are doing great job about that! | 13:40 |
| eranrom | thanks! | 13:40 |
| eranrom | and thanks for joining. | 13:40 |
| kota_ | eranrom: thanks! | 13:41 |
| takashi | eranrom, kota_: thx | 13:41 |
| *** openstack has joined #openstack-storlets | 14:25 | |
| openstackgerrit | Merged openstack/storlets: Use contextmanager for storlet invocation descriptors https://review.openstack.org/294358 | 14:40 |
| *** eranrom has quit IRC | 15:33 | |
| *** eranrom has joined #openstack-storlets | 15:59 | |
| *** eranrom has quit IRC | 18:10 | |
| *** openstack has joined #openstack-storlets | 19:24 | |
| *** openstack has joined #openstack-storlets | 20:34 | |
| *** takashi has joined #openstack-storlets | 20:58 | |
| *** takashi has quit IRC | 22:02 | |
| *** mattoliverau has quit IRC | 23:06 | |
| *** matt6434 has joined #openstack-storlets | 23:07 | |
| *** openstack has joined #openstack-storlets | 23:26 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!