| *** openstack has joined #openstack-storlets | 01:26 | |
| *** eranrom has joined #openstack-storlets | 11:49 | |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: WIP: Merge all flow methods in gateway module https://review.openstack.org/335472 | 12:33 |
|---|---|---|
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: WIP: Fix ansible-lint error https://review.openstack.org/335480 | 12:49 |
| eranrom | Hi | 12:50 |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: WIP: Fix ansible-lint error https://review.openstack.org/335480 | 12:50 |
| *** takashi has joined #openstack-storlets | 12:52 | |
| eranrom | Hi | 12:52 |
| takashi | eranrom: Hi | 12:52 |
| eranrom | Hi Takashi | 12:53 |
| eranrom | I have placed an agenda here (just now). | 12:53 |
| eranrom | https://wiki.openstack.org/wiki/Meetings/Storlets#Agenda: | 12:53 |
| eranrom | Should we wait for Kota? | 12:54 |
| takashi | eranrom: now checking, thanks! | 12:54 |
| takashi | eranrom: kota_ told me he can't attend to today's meeting | 12:54 |
| eranrom | ok. np. | 12:55 |
| takashi | eranrom: So I think we can start | 12:55 |
| eranrom | Do you have anything else you want to discuss on top of the list>? | 12:55 |
| takashi | eranrom: nothing special :-) | 12:56 |
| eranrom | ok :-) | 12:56 |
| eranrom | so, are you able to work with tox on master? | 12:56 |
| eranrom | I always get a problem with the markers (> in the deps) | 12:57 |
| takashi | eranrom: master of storlets? | 12:57 |
| eranrom | yes | 12:57 |
| takashi | eranrom: which version of pip do you have? | 12:57 |
| eranrom | sec. | 12:58 |
| eranrom | 8.1.2 | 12:58 |
| takashi | eranrom: I'm now using pip 8.0.2, and has no error | 12:59 |
| eranrom | perhaps I will try to downgrade. | 12:59 |
| eranrom | ok, but it works for you. This was important for me to know... | 12:59 |
| eranrom | so next topic :-) | 13:00 |
| takashi | eranrom: I'll try to upgrade my pip later | 13:00 |
| eranrom | oh, I hope it will not break things for you. | 13:00 |
| eranrom | I will dig deeper | 13:00 |
| eranrom | no problem thanks | 13:00 |
| takashi | If there are any change in pip which causes such an error, I"m afraid that it can affect other projects... :-( | 13:01 |
| eranrom | right | 13:01 |
| eranrom | While working on a new patch for local ranges in the object server, I noticed a possible problem with the DockerRequest class. | 13:02 |
| eranrom | I think you have merge the StorletPutRequest and StorletGet request into one class. | 13:03 |
| eranrom | the thing is that in a GET flow we instantiate the class with the response | 13:03 |
| takashi | eranrom: yes | 13:03 |
| eranrom | and so we do not have the expected headers/ | 13:03 |
| eranrom | see e.g.: | 13:03 |
| eranrom | https://github.com/openstack/storlets/blob/master/Engine/swift/storlet_gateway/gateways/docker/gateway.py | 13:05 |
| eranrom | line 192 | 13:05 |
| eranrom | sorry line 191 | 13:05 |
| eranrom | we do: | 13:05 |
| eranrom | sreq = DockerStorletRequest(orig_resp, req.params, | 13:05 |
| eranrom | data_fd=orig_resp.app_iter._fp.fileno()) | 13:05 |
| takashi | eranrom: got it. Sorry for missing that | 13:06 |
| eranrom | no problem. | 13:06 |
| eranrom | note that it works since we do not really make use of that information form the request... | 13:07 |
| eranrom | I missed it in the review :-) | 13:07 |
| eranrom | next topic? | 13:07 |
| takashi | eranrom: IMO we should add at least an unit test | 13:07 |
| eranrom | yeah | 13:08 |
| eranrom | right. | 13:08 |
| takashi | eranrom: Do you plan to fix that? If no, I'll work about it soone. | 13:08 |
| takashi | s/soone/soon | 13:08 |
| eranrom | I can fix no problem. | 13:09 |
| takashi | eranrom: I just want to make sure that we don't do duplicated work | 13:09 |
| eranrom | On the other hand, you initiated the refactoring, and you may have an opinion on how to fix. If you are ok with splitting back to get request and put request then I can do that | 13:10 |
| eranrom | and will let you review | 13:10 |
| eranrom | This will also be a good opportunity for me to get deeper to the unit test framework. | 13:10 |
| takashi | eranrom: For now, I have some idea to fix the problem, as the part of my remaining refactoring | 13:11 |
| eranrom | ok, so perhaps you better do it. | 13:12 |
| takashi | eranrom: so I'd like to fix it | 13:12 |
| takashi | eranrom: yes | 13:12 |
| eranrom | sure. | 13:12 |
| takashi | eranrom: thanks for your finding the problem :-) | 13:12 |
| eranrom | :-) ok, is it a good time to discuss prioritization? | 13:12 |
| takashi | eranrom: yes | 13:13 |
| eranrom | So I really want to get the ranges stuff, because people in IBM want to do measurements. | 13:13 |
| eranrom | On the other hand, perhaps we want to have your fix earlier | 13:13 |
| eranrom | The range stuff is made of 3 patches: | 13:14 |
| eranrom | 1. refactor the datagram stuff | 13:14 |
| eranrom | 2. separate fd metadata to system and user | 13:15 |
| eranrom | 3. add the range stuff | 13:15 |
| takashi | eranrom: I saw you posted splited patches on gerrit | 13:16 |
| eranrom | How about merging 1,2 | 13:18 |
| eranrom | then have your fix for the requests - which is needed for the 3rd patch | 13:18 |
| eranrom | and then the ranges | 13:18 |
| eranrom | would that work for you? | 13:18 |
| takashi | eranrom: makes sense | 13:19 |
| eranrom | alright. | 13:20 |
| takashi | eranrom: I think my fix becomes not so big, and I suppose we can merge it without any big conflicts with your work | 13:20 |
| eranrom | agree. | 13:20 |
| eranrom | I don't mind merging it whenever it is ready, I just did not want to push :-) | 13:20 |
| takashi | eranrom: I'll review patch 1 and 2 tommorow :-) I had a quick look today, but need some more time for deep looking | 13:21 |
| eranrom | sure. Thanks! | 13:21 |
| eranrom | did you get a change to see the mail on simplifying the docker images/ | 13:22 |
| eranrom | ? | 13:22 |
| takashi | Sorry, I haven't checked it yet | 13:22 |
| eranrom | ok, np. This is just an idea. | 13:23 |
| eranrom | We currently have 3 'layers' in our docker image: | 13:23 |
| eranrom | 1. ubuntu | 13:23 |
| eranrom | 2. JRE8 | 13:24 |
| eranrom | 3. storlet stuff (SCommon, SDaemon, factory) | 13:24 |
| takashi | yes | 13:24 |
| eranrom | I thought that the storlet stuff could be placed outside of the image and mounted as a read only volume. | 13:25 |
| eranrom | Easier deployment, much easier to update | 13:25 |
| eranrom | less time from dev to test | 13:25 |
| takashi | eranrom: sounds great! | 13:26 |
| eranrom | this way if for example we have 10 tenants using storlets, we do not have to update any image if we fix a bug... | 13:26 |
| eranrom | I guess my only question is: | 13:27 |
| eranrom | do we keep a separate copy for eaxh docker container, or a single copy that is mounted to all docker containers. | 13:27 |
| eranrom | s/eaxh/each | 13:27 |
| takashi | eranrom: a snigle copy looks better for me | 13:28 |
| eranrom | this is also my thinking | 13:29 |
| eranrom | I am a little worried that we might have a versioning problem with this. but we can always have more then one 'copy' e.g. different versions | 13:30 |
| eranrom | and we will need to maintain which accounts use which versions, but this is futuristic thing I guess | 13:30 |
| takashi | I'm now cuoruous about the way to make sure about python/java execution | 13:32 |
| eranrom | you mean in terms of java classpath? | 13:33 |
| eranrom | and python include path? | 13:33 |
| takashi | eranrom: also jvm and python interpreter | 13:34 |
| eranrom | are you suggesting to have them also from the host? | 13:34 |
| eranrom | e.g. mount /usr/lib or something to the container? | 13:35 |
| takashi | eranrom: I need some more time for thinking, but | 13:36 |
| takashi | from my current thought, at least the python interpreter to execute daemon_factory should be maintained with daemon_factory, in host side | 13:37 |
| eranrom | I can't say it does not make sense. | 13:38 |
| eranrom | sure less of a python versioning headache | 13:39 |
| eranrom | we need to make sure we are aligned with the Swift python version. | 13:39 |
| eranrom | which we anyway need, and we are ok | 13:40 |
| eranrom | The Java part is different, because we may not have Java on the host. | 13:40 |
| eranrom | on the other hand, what happens when/if we support python storlets... | 13:41 |
| eranrom | need more thinking... | 13:41 |
| takashi | eranrom: yes | 13:41 |
| eranrom | ok, so nothing more from me :-) | 13:42 |
| takashi | eranrom: one thing from my side | 13:42 |
| takashi | I just started working about ppython supprot for storlet application language | 13:43 |
| takashi | s/ppython/python/ | 13:43 |
| eranrom | wow! | 13:43 |
| eranrom | super cool!!! | 13:43 |
| takashi | eranrom: I'm now thinking about the way to handle dependency in python case | 13:44 |
| takashi | In java, we can use jar file, which gathers multiple java source codes | 13:44 |
| takashi | but in python, we should consider another solution now | 13:45 |
| eranrom | I am not an expert here, but have you considered python eggs? | 13:45 |
| takashi | I haven't checked it yet | 13:46 |
| eranrom | ok | 13:46 |
| takashi | I'm now thinking about automatical installing using requirement.txt file as object, but need some more time to decide detailed design | 13:47 |
| eranrom | interesting. Does this mean we open up networking for the docker container?\ | 13:49 |
| takashi | eranrom: yes. or another solution is opening up networking for host node, and let host node build virtualenv, which passed to docker container by filesystem mounting | 13:50 |
| takashi | s/passed/is passed/ | 13:51 |
| eranrom | right. Still I think that there a huge value with python even with the standard libraries | 13:51 |
| eranrom | unlessof course you have a counter example :-) | 13:51 |
| takashi | eranrom: yes | 13:52 |
| eranrom | recall that we can change the Docker image as well. | 13:52 |
| takashi | eranrom: I'm just at a starting point, so I'll let you know after I created more detailed idea | 13:52 |
| takashi | s/created/create | 13:53 |
| eranrom | sure! anyway I am vary happy that you are working in this!!! | 13:53 |
| takashi | eranrom: :-) | 13:53 |
| takashi | That's all from my side | 13:54 |
| eranrom | ok, thanks for the good discussion. | 13:54 |
| takashi | thank you, too | 13:55 |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Remove DockerStorletRequest class https://review.openstack.org/335521 | 14:13 |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Remove unused parameters from DockerStorletRequest https://review.openstack.org/335521 | 14:28 |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Remove unused parameters from DockerStorletRequest https://review.openstack.org/335521 | 14:30 |
| *** takashi has quit IRC | 14:31 | |
| *** eranrom_ has joined #openstack-storlets | 16:03 | |
| *** eranrom has quit IRC | 16:06 | |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Remove unused parameters from DockerStorletRequest https://review.openstack.org/335521 | 16:44 |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: WIP: Fix ansible-lint error https://review.openstack.org/335480 | 17:15 |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Fix bashate errors https://review.openstack.org/335597 | 17:20 |
| *** notmyname has quit IRC | 19:01 | |
| *** notmyname has joined #openstack-storlets | 19:04 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!