| *** kaisers1 has quit IRC | 00:35 | |
| *** kaisers1 has joined #openstack-manila | 00:37 | |
| *** gouthamr has quit IRC | 00:50 | |
| *** dsariel_ has quit IRC | 01:04 | |
| *** dsariel has quit IRC | 01:04 | |
| *** kaisers_ has joined #openstack-manila | 01:15 | |
| *** catintheroof has quit IRC | 01:18 | |
| *** kaisers has quit IRC | 01:19 | |
| *** zhonghua has joined #openstack-manila | 01:35 | |
| *** gcb has quit IRC | 02:01 | |
| *** gcb has joined #openstack-manila | 02:01 | |
| *** jiaohaolin1 has joined #openstack-manila | 02:17 | |
| jiaohaolin1 | 1 | 02:17 |
|---|---|---|
| jiaohaolin1 | hey guys, my boss would like me to write a Manila driver for our storage . So I check the wiki of manila. But I can't get any useful infomation of Drivers .It doesn't like Cinder Driver, the wiki of Cinder can tell me how to contribute a driver and the Minimum Features that driver should implement. The manila wiki doesn't say anything | 02:28 |
| jiaohaolin1 | So , my question is which features should I implement in the manila driver ? Or I should implement all functions of 'driver.py'? | 02:29 |
| jiaohaolin1 | hello ? Dose anybody not in bad ? | 02:33 |
| bswartz | jiaohaolin1: https://docs.openstack.org/developer/manila/devref/driver_requirements.html | 02:35 |
| *** catintheroof has joined #openstack-manila | 02:40 | |
| jiaohaolin1 | bswartz: Thank you so much ! | 02:48 |
| *** gouthamr has joined #openstack-manila | 02:58 | |
| *** gouthamr has quit IRC | 03:22 | |
| *** gouthamr has joined #openstack-manila | 03:26 | |
| *** catintheroof has quit IRC | 03:57 | |
| *** kaisers_ has quit IRC | 04:03 | |
| *** kaisers has joined #openstack-manila | 04:03 | |
| *** jmlowe has quit IRC | 04:04 | |
| *** kaisers has quit IRC | 04:05 | |
| *** kaisers has joined #openstack-manila | 04:05 | |
| *** kaisers has quit IRC | 04:10 | |
| *** kaisers has joined #openstack-manila | 04:11 | |
| *** hoonetorg has quit IRC | 04:13 | |
| *** gouthamr has quit IRC | 04:16 | |
| *** hoonetorg has joined #openstack-manila | 04:26 | |
| *** swapnil-nilangek has joined #openstack-manila | 04:52 | |
| *** pcaruana has joined #openstack-manila | 06:14 | |
| *** jprovazn has joined #openstack-manila | 06:23 | |
| *** kaisers has quit IRC | 06:29 | |
| *** kaisers has joined #openstack-manila | 06:30 | |
| toabctl | jiaohaolin1, it's also a good idea to look into the current driver implementations | 07:25 |
| toabctl | vponomaryov, bswartz re bug 1690159. I wonder what the expected behaviour is in a multi-backend case? So eg. for one backend, init_host() works and for the second backend it does not | 07:27 |
| openstack | bug 1690159 in Manila "cephfs (possibly others): manila-share does not retry/restart if init_host() fails " [Undecided,New] https://launchpad.net/bugs/1690159 | 07:27 |
| toabctl | should manila-share be restarted or not? | 07:27 |
| toabctl | btw. the behaviour was changed due to https://bugs.launchpad.net/manila/+bug/1690159 | 07:27 |
| toabctl | ehm, sorry. due to https://bugs.launchpad.net/manila/+bug/1500964 I mean | 07:28 |
| openstack | Launchpad bug 1500964 in Manila "Manila-shr is auto restarted if exception is thrown in do_setup()" [Medium,Fix released] - Assigned to Jordan Pittier (jordan-pittier) | 07:28 |
| toabctl | I wonder if https://review.openstack.org/#/c/230018/ should be reverted.... | 07:28 |
| jiaohaolin1 | toabctl: thanks , I'm doing this now , but it can not determine which features must be implemented, which features are optional,so I chose the functions that every driver has implement as requirement of our driver. | 07:37 |
| jiaohaolin1 | I hope that the core members can define the functions that must be implemented like Cinder-Driver. So the new driver Contributors can refer to it to make their own driver faster. | 07:40 |
| toabctl | jiaohaolin1, it is defined in the docs bswartz mentioned. | 07:53 |
| jiaohaolin1 | toabctl: sorry, what is that mean ? bswartz ? | 07:59 |
| toabctl | jiaohaolin1, bswartz = Ben = PTL of manila. and he mentioned https://docs.openstack.org/developer/manila/devref/driver_requirements.html already | 07:59 |
| zhongjun | jiaohaolin1: If you have any question about this doc( https://docs.openstack.org/developer/manila/devref/driver_requirements.html) and your question, please let our know. :) | 08:10 |
| zhongjun | bswartz: ye, there are many changes in this patch, and some of driver can not be tested. You was considering splitting apart the LVM stuff. So will you split the patch when you have free time? | 08:11 |
| jiaohaolin1 | toabctl: sorry~ But this page only said the 'share create/delete' and 'share extending' must be implemented . I didn't find other functions have to be impemented, | 08:14 |
| *** a-pugachev has joined #openstack-manila | 08:15 | |
| jiaohaolin1 | toabctl: And my english is not very well ,maybe I should check this page carefully again. | 08:15 |
| zhongjun | jiaohaolin1: At least one driver mode (DHSS true/false) | 08:20 |
| zhongjun | At least one file system sharing protocol | 08:20 |
| zhongjun | Access rules | 08:20 |
| zhongjun | Shares | 08:20 |
| zhongjun | Share extending | 08:20 |
| zhongjun | Capabilities | 08:20 |
| zhongjun | Continuous Integration systems | 08:20 |
| zhongjun | Unit tests | 08:20 |
| zhongjun | Documentation | 08:20 |
| jiaohaolin1 | zhongjun: thanks ~ | 08:21 |
| zhongjun | jiaohaolin1: please see the doc tittle, there are need to be implement | 08:22 |
| zhongjun | jiaohaolin1: You should keep attention onmanila new driver submission deadline. | 08:34 |
| zhongjun | jiaohaolin1: https://releases.openstack.org/pike/schedule.html | 08:35 |
| zhongjun | jiaohaolin1: You should pay attention on manila new driver submission deadline. :) Jun 05 - Jun 09 | 08:35 |
| jiaohaolin1 | zhongjun: ... | 08:39 |
| jiaohaolin1 | zhongjun: I guess we will submit it in next version. we just start it | 08:42 |
| zhongjun | jiaohaolin1: cool, then you can forget about the deadline in Pike version. | 08:45 |
| *** lpetrut has joined #openstack-manila | 09:22 | |
| *** gcb has quit IRC | 10:14 | |
| *** jiaohaolin1 has left #openstack-manila | 10:14 | |
| *** gcb has joined #openstack-manila | 10:16 | |
| *** jprovazn has quit IRC | 10:27 | |
| *** jprovazn has joined #openstack-manila | 11:35 | |
| *** chlong has quit IRC | 11:36 | |
| *** jprovazn has quit IRC | 11:46 | |
| *** jprovazn has joined #openstack-manila | 11:47 | |
| openstackgerrit | Merged openstack/manila-ui master: Fix NotImplementedError and use new way to define action_present/past https://review.openstack.org/462854 | 11:50 |
| *** gcb has quit IRC | 12:52 | |
| *** jmlowe has joined #openstack-manila | 13:03 | |
| *** dustins has joined #openstack-manila | 13:04 | |
| *** scottda has joined #openstack-manila | 13:10 | |
| *** xyang1 has joined #openstack-manila | 13:30 | |
| bswartz | toabctl: we should discuss that at the next weekly meeting or sooner | 13:30 |
| toabctl | bswartz, hey. I'm already working on a fix. I think the best thing todo (in a multi-backend scenario) is to loop for the broken backends | 13:31 |
| toabctl | bswartz, looks like that is not complicated. I post a patch soon | 13:31 |
| bswartz | toabctl: and endless restart loop seems like a bad approach | 13:34 |
| toabctl | bswartz, there is imo no other way in a multi-backend setup | 13:34 |
| toabctl | bswartz, if 1 backend works and another one is broken - what do you want todo? | 13:35 |
| toabctl | bswartz, currently it is completely broken because it does a) not abort and b) not loop | 13:35 |
| bswartz | I wonder if we can distinguish between failures that should be restarted and permanent failures | 13:35 |
| -openstackstatus- NOTICE: The Gerrit service on http://review.openstack.org is being restarted to address hung remote replication tasks. | 13:37 | |
| toabctl | bswartz, and I think it does not harm if we loop during init_host() per backend. this is not an endless restart loop | 13:38 |
| toabctl | it is more like a "wait for the backend " | 13:38 |
| toabctl | bswartz, btw. how was the summit? how many people were there? | 13:39 |
| vponomaryov | toabctl: what do you mean by "loop for the broken backends"? | 13:57 |
| bswartz | toabctl: In cases where the driver crashes and we know it's a temporary failure, then I think restarting automatically is a good idea. Likewise in cases where we know it's a permanent failure, then we should not restart. The problem is that it's hard to know which case any given error is, and in some cases it's impossible to know. | 13:57 |
| vponomaryov | toabctl: one backend - one process | 13:57 |
| toabctl | give me 5 min and I push the patch. then you can comment there... | 13:57 |
| bswartz | toabctl: therefore we need some good default behavior to cover errors which might be permanent and might be temporary | 13:57 |
| toabctl | bswartz, the current behavior is wrong. that's for sure :) | 13:58 |
| bswartz | I would advocate some limited number of retries with a exponential backoff time between the retries | 13:58 |
| vponomaryov | +1 for limited number of retries with a exponential backoff time between the retries | 13:58 |
| toabctl | no. then you end up in the same situation. | 13:58 |
| toabctl | just after some time | 13:59 |
| bswartz | toabctl: what is the situation you're most worried about? | 13:59 |
| bswartz | the situation where the service is dead and nothing is trying to restart it? | 13:59 |
| toabctl | bswartz, that you have configured a backend, manila-share starts, init_host() fails and manila-share runs endlessly but can not be used. | 14:00 |
| toabctl | bswartz, and that is what we have currently | 14:00 |
| bswartz | yes we definitely don't want that | 14:00 |
| bswartz | but there are only 2 choices to fix the problem | 14:01 |
| toabctl | bswartz, commit 4b87f6f40d1f09b4 introduced that (for a good reason, the multi-backend case) | 14:01 |
| bswartz | 1) allow the process to die and leave it up to the user to fix | 14:01 |
| bswartz | 2) retry and risk it failing again and again for the same reason | 14:01 |
| toabctl | bswartz, yes. but the downside for 1) is, that there might be a working backend and if you let the process die, nothing ever works | 14:02 |
| bswartz | that's why I'm arguing for a middle ground | 14:02 |
| toabctl | so imo 1) is not a good solution because it blocks everything if only a single backend is broken | 14:02 |
| bswartz | I don't see how it blocks everything | 14:03 |
| bswartz | it only blocks requests to that backend | 14:03 |
| bswartz | the rest of manila is fine if one m-shr service can't start up | 14:03 |
| toabctl | bswartz, does 1) mean to you that you kill the whole m-shr process? Or just the "backend process" managed by m-shr ? | 14:05 |
| bswartz | toabctl: oh I see what you're getting at | 14:05 |
| bswartz | when you configure multibackend, there is a parent process which does nothing but spawn child m-shr processes | 14:05 |
| bswartz | so which one should die | 14:06 |
| toabctl | yeah. for me "killing manila-share" means killing the parent and all its childs. | 14:06 |
| toabctl | like "systemctl stop openstack-manila-share" | 14:06 |
| bswartz | the child should definitely die if it can't initialize the driver, but the parent should probably keep running as long as one backend is working okay | 14:06 |
| bswartz | let me think about that | 14:07 |
| toabctl | bswartz, yes. that's what my fix will do. | 14:07 |
| toabctl | and retry for the backend that did not work | 14:07 |
| bswartz | toabctl: how many retries? endless? | 14:07 |
| bswartz | will it sleep between retries? | 14:07 |
| toabctl | bswartz, yes. I don't see a problem with that. currently I thought "try every 10 seconds". could be configurable of course by I don't see why for the moment | 14:08 |
| bswartz | I would advocate for exponential backoff time bewteen retries no matter whether it's limited or unlimited retries | 14:08 |
| toabctl | yes. time.sleep(10) which does not block anything else afaikc | 14:08 |
| *** chlong has joined #openstack-manila | 14:09 | |
| bswartz | so that if it's a temporary failure it will be fixed very quickly by a fast retry, but if it's a permament failure, the log spam will be kept to a minimum | 14:09 |
| toabctl | yeah. that sounds also good to me. | 14:09 |
| bswartz | like the retry time should start out at some small number and then keep doubling up to a maximum time | 14:09 |
| openstackgerrit | Thomas Bechtold proposed openstack/manila master: Retry backend initialization https://review.openstack.org/464205 | 14:13 |
| toabctl | bswartz, vponomaryov ^^ here is the patch. I'll rework the time.sleep and add your exponential backoff proposal | 14:14 |
| *** xyang1 has quit IRC | 14:19 | |
| *** xyang1 has joined #openstack-manila | 14:19 | |
| *** jprovazn has quit IRC | 14:47 | |
| openstackgerrit | Thomas Bechtold proposed openstack/manila master: Retry backend initialization https://review.openstack.org/464205 | 14:52 |
| *** lpetrut has quit IRC | 15:09 | |
| *** gouthamr has joined #openstack-manila | 15:18 | |
| *** gouthamr has quit IRC | 15:57 | |
| *** kaisers has quit IRC | 16:06 | |
| *** pcaruana has quit IRC | 16:12 | |
| *** gouthamr has joined #openstack-manila | 16:14 | |
| *** kaisers has joined #openstack-manila | 16:31 | |
| *** chenying_ has quit IRC | 17:06 | |
| *** chenying__ has joined #openstack-manila | 17:06 | |
| *** a-pugachev has quit IRC | 17:41 | |
| *** harlowja has quit IRC | 19:02 | |
| *** a-pugachev has joined #openstack-manila | 19:04 | |
| *** gouthamr has quit IRC | 19:50 | |
| *** a-pugachev has quit IRC | 20:30 | |
| *** dustins has quit IRC | 20:36 | |
| *** xyang1 has quit IRC | 21:57 | |
| *** harlowja has joined #openstack-manila | 22:40 | |
| *** kaisers has quit IRC | 22:58 | |
| *** chlong has quit IRC | 23:00 | |
| *** kaisers has joined #openstack-manila | 23:25 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!