| *** tosky has quit IRC | 00:52 | |
| *** maaritamm has joined #openstack-manila | 05:17 | |
| *** maaritamm has quit IRC | 05:59 | |
| *** vhari has joined #openstack-manila | 06:09 | |
| *** maaritamm has joined #openstack-manila | 06:35 | |
| *** toabctl has quit IRC | 08:30 | |
| openstackgerrit | Maari Tamm proposed openstack/python-manilaclient master: Implement OSC share quota commands https://review.opendev.org/703992 | 08:36 |
|---|---|---|
| openstackgerrit | haixin proposed openstack/manila master: If only .pyc exist, the extension API will be disabled https://review.opendev.org/715135 | 08:56 |
| *** tosky has joined #openstack-manila | 09:00 | |
| *** toabctl has joined #openstack-manila | 09:14 | |
| *** rraja has joined #openstack-manila | 11:10 | |
| *** andrebeltrami has joined #openstack-manila | 11:13 | |
| *** lpetrut has joined #openstack-manila | 12:18 | |
| *** enriquetaso has joined #openstack-manila | 13:10 | |
| openstackgerrit | Maari Tamm proposed openstack/python-manilaclient master: Add initial documentation for OSC https://review.opendev.org/705834 | 13:19 |
| *** mmethot has quit IRC | 14:30 | |
| vkmc | o/ | 14:36 |
| vkmc | dviroel, hey, I see you reported this https://bugs.launchpad.net/manila/+bug/1868088 | 14:36 |
| openstack | Launchpad bug 1868088 in Manila "RFE: Provide an admin API to query quotas for all user in a project" [Undecided,New] | 14:36 |
| vkmc | in one of maaritamm's patches I noticed something related... seems that right now when we try to filter out with user-id it is not working at all | 14:36 |
| vkmc | it's a bug or it's something that is not yet implemented? | 14:37 |
| vkmc | maybe you looked into this | 14:37 |
| vkmc | this is the patch I'm referring to https://review.opendev.org/#/c/703992/ | 14:37 |
| vkmc | maaritamm, IMHO you can merge your patch as is... we will need to fix stuff in the server side then | 14:41 |
| vkmc | and thanks for updating the docs | 14:41 |
| dviroel | yey vkmc, lemme take a look | 14:41 |
| maaritamm | vkmc, ok cool | 14:42 |
| vkmc | :) | 14:42 |
| maaritamm | to docs I added share types now but once quotas are merged I will add those too with a note that tenant-id is now mandatory | 14:43 |
| *** sfernand has joined #openstack-manila | 14:53 | |
| *** enriquetaso has quit IRC | 15:29 | |
| *** josecastroleon has joined #openstack-manila | 15:34 | |
| josecastroleon | hi, anyone around? | 15:35 |
| vkmc | josecastroleon, o/ | 15:37 |
| vkmc | weekly meeting at #openstack-meeting-alt | 15:38 |
| josecastroleon | ah ok | 15:38 |
| vkmc | everybody is there | 15:38 |
| gouthamr | josecastroleon: hello again, what's going on? | 16:01 |
| josecastroleon | so adding extra logging on the scheduling phase I saw that in the default filter driver | 16:02 |
| josecastroleon | it was taking 45 seconds while doing this call https://opendev.org/openstack/manila/src/branch/master/manila/scheduler/drivers/filter.py#L215 | 16:03 |
| josecastroleon | that goes into this https://opendev.org/openstack/manila/src/branch/master/manila/scheduler/host_manager.py#L433-L436 | 16:04 |
| josecastroleon | and as the cephfs driver does not have the capability 'provisioned_capacity_gb' | 16:04 |
| dviroel | hey vkmc: lunch time, will provide you a feedback soon | 16:05 |
| vkmc | dviroel, sure, no rush | 16:05 |
| josecastroleon | then it does an expensive db query for the size of our setup | 16:05 |
| josecastroleon | actually is a set of db queries | 16:06 |
| gouthamr | josecastroleon: oh, yes, completely makes sense - how many shares are there in this setup? | 16:06 |
| josecastroleon | 1400 | 16:06 |
| josecastroleon | the thing is we had a similar issue in the past with cinder | 16:08 |
| josecastroleon | the scheduler was also calculating the size looping into the volumes in ceph | 16:08 |
| gouthamr | josecastroleon: yeah, seems wasteful to estimate the capacity on each scheduling request | 16:09 |
| josecastroleon | i have a follow up question | 16:09 |
| gouthamr | josecastroleon: ack, do you have the cinder side bug, i wonder if the ceph driver can just report the provisioned capacity | 16:10 |
| josecastroleon | on cinder we disable the lookup as the backend was exclusively for cinder | 16:11 |
| tbarron | gouthamr: so do we think other drivers are keeping their own tally of provisioned capacity so they don't have to do this lookup? | 16:11 |
| josecastroleon | it was called rbd_exclusive_cinder_pool | 16:11 |
| josecastroleon | i was thinking in one) look into the cluster_stats to see if we can provide the provisioned capacity | 16:12 |
| josecastroleon | or two) do like cinder and have an option to disable this in the cephfs driver | 16:12 |
| gouthamr | tbarron: yes, i think that was the recommendation - although we could be improving the scheduler too with some caching and optimization of that aggregation itself | 16:12 |
| gouthamr | josecastroleon: i have a further issue with this | 16:13 |
| tbarron | josecastroleon: ceph driver would just do a single call to get pool stats? | 16:13 |
| gouthamr | josecastroleon: the cephfs driver doesn't support thin provisioning | 16:13 |
| tbarron | and post that in its periodic updates to the scheduler | 16:13 |
| gouthamr | josecastroleon: so all this calculation is for naught, since the scheduler won't use that number anyway | 16:13 |
| tbarron | gouthamr: because it's even thinner :D | 16:14 |
| gouthamr | josecastroleon: so, i think we can circumvent the aggregation right away | 16:14 |
| josecastroleon | by just reporting it as 0 on the driver stats? | 16:14 |
| gouthamr | tbarron: yes, josecastroleon: i must clarify, cephfs doesn't support manila determined thin provisioning | 16:14 |
| gouthamr | a lot of drivers don't support controlling oversubscription via manila even if they always thinly provision storage | 16:15 |
| josecastroleon | ok | 16:15 |
| tbarron | gouthamr: but yeah, you are saying that we aren't using this numer that took 45 seconds to calculate | 16:15 |
| tbarron | number, total | 16:15 |
| gouthamr | tbarron: yes | 16:15 |
| josecastroleon | shall be just matter of setting it to 0 or to some value in here => https://opendev.org/openstack/manila/src/branch/master/manila/share/drivers/cephfs/driver.py#L188 | 16:17 |
| josecastroleon | I mean provisioned_capacity_gb | 16:17 |
| josecastroleon | or inside the pool | 16:17 |
| gouthamr | josecastroleon: 0 won't help | 16:18 |
| gouthamr | josecastroleon: 1 will, but that would be an ugly workaround :) | 16:18 |
| gouthamr | can you please report a bug, i think the first thing to do is not perform any calculation if the share type and backend don't support thin provisioning | 16:18 |
| tbarron | "use_thin_logic" :D | 16:19 |
| josecastroleon | yes, i am happy to contribute with the code | 16:19 |
| gouthamr | josecastroleon: i say 1 will because of https://opendev.org/openstack/manila/src/branch/master/manila/scheduler/host_manager.py#L433-L436 | 16:19 |
| gouthamr | josecastroleon: awesome, please do | 16:19 |
| tbarron | the "thin_logic" needs a bit of buttressing | 16:20 |
| gouthamr | yes, that wall should contain, not fall over | 16:20 |
| josecastroleon | ok great | 16:20 |
| josecastroleon | :d | 16:20 |
| gouthamr | thanks for reporting this josecastroleon - feel free to subscribe me to the bug or post it here so we can triage | 16:21 |
| josecastroleon | I passed quite some time debugging it this morning | 16:22 |
| josecastroleon | we had complains from the guys using CSI as some requests were timing out | 16:22 |
| gouthamr | josecastroleon: ah, nasty problem surely | 16:25 |
| gouthamr | josecastroleon: curious, are you guys using the new manila-csi, or still using the manila provisioner | 16:25 |
| josecastroleon | at this moment still manila-provisioner | 16:26 |
| gouthamr | josecastroleon: ack, ty for confirming | 16:26 |
| *** josecastroleon has quit IRC | 16:31 | |
| dviroel | vkmc: saw your comments in the patch. This happens because manila doesn't validate tenant-id/user-id with keystone and add this entry in the db | 16:56 |
| dviroel | vkmc: so, if the admin enters an invalid user-id, it will stored in db and this will affect the tenant quota at the end | 16:58 |
| dviroel | vkmc: in the RFE, instead of adding keystone validation for tenant/user ids I was suggestion a more simple enhancement | 16:58 |
| dviroel | to add an API for admin retrieve all user-ids associated to a tenant, so it will able to delete invalid entries with quota-delete cmd | 17:00 |
| dviroel | s/all user-ids/all user-ids and quotas | 17:00 |
| dviroel | today, we can just retrieve quotas for a project or for an user | 17:01 |
| *** enriquetaso has joined #openstack-manila | 17:05 | |
| *** jv__ has joined #openstack-manila | 17:05 | |
| *** jv has quit IRC | 17:05 | |
| vkmc | dviroel, ok, so this means it's not a problem with the CLI | 17:14 |
| vkmc | it's a feature not yet available in the server side | 17:14 |
| dviroel | vkmc: yes | 17:16 |
| *** lpetrut has quit IRC | 17:21 | |
| vkmc | dviroel, thanks for clarifying :) | 17:27 |
| *** openstack has quit IRC | 17:49 | |
| *** openstack has joined #openstack-manila | 17:52 | |
| *** ChanServ sets mode: +o openstack | 17:52 | |
| dviroel | vkmc , regarding your comment. I might expressed bad. It is possible to retrieve quota for user-id associated to a project. In the same way, its also possible to retrieve just the project's quota. What is not possible is to retrieve all user quotas entries associated to a project, which brings a problem of having unknown db entries. | 18:07 |
| dviroel | vkmc I'll review and clarify that on this change. | 18:08 |
| openstackgerrit | vrushti proposed openstack/manila master: Documentation for Share Migration Ocata Improvements. https://review.opendev.org/715290 | 18:18 |
| vkmc | dviroel, oh, I misunderstood maybe | 18:21 |
| *** rraja has quit IRC | 18:35 | |
| dviroel | vkmc: added some comments in the patch, not sure what you and maaritamm have being discussing regarding tenant-id/project-id and optional/mandatory parameters, just added my concerns there. | 19:06 |
| vkmc | dviroel, thx | 19:17 |
| maaritamm | dviroel, vkmc, thanks for looking at the patch, I am checking it now | 19:39 |
| maaritamm | just have one question | 19:39 |
| maaritamm | whats the verdict on project_id being optional/mandatory then? | 19:40 |
| openstackgerrit | Carlos Eduardo proposed openstack/manila master: Add new quota for share replicas https://review.opendev.org/712675 | 19:43 |
| vkmc | maaritamm, so this is my opinion | 19:48 |
| vkmc | it should be mandatory | 19:49 |
| vkmc | because when you do openstack share quota show <x> | 19:49 |
| vkmc | you need to specify for what resource you want to show that quota | 19:49 |
| maaritamm | ok, so I will keep that as it is, thanks | 19:51 |
| vkmc | in the current client implementation we are returning the quotas for the current project in use | 19:51 |
| vkmc | in reply to dviroel review, probably you want to remove the default value | 19:52 |
| vkmc | I missed that detail :) | 19:52 |
| vkmc | you don't have a default value if the param is mandatory | 19:52 |
| maaritamm | yeah, makes sense :) | 19:52 |
| vkmc | and... regarding project_id/tenant_id | 19:53 |
| vkmc | we used tenant_id for everything and we are using all_tenants for the manila api (see https://review.opendev.org/#/c/704934/) | 19:54 |
| vkmc | so | 19:54 |
| vkmc | I'd stick to tenant_id | 19:54 |
| vkmc | even though it's the old naming | 19:54 |
| vkmc | unless we are going to deprecate that at some point | 19:56 |
| dviroel | regarding project_id: in the manila API these parameters are mandatory, I think that the current client is providing the project_id from the user that is requesting the operation when no different project_id is provided. So we might just do the same or make it mandatory for all quota operations | 19:57 |
| vkmc | dviroel, exactly | 19:57 |
| gouthamr | or infer it, like the client currently does? | 19:57 |
| vkmc | we are doing that | 19:57 |
| vkmc | I just suggested to make it mandatory for all quota operations | 19:58 |
| vkmc | so it's clear which project you are retrieving the quota for | 19:58 |
| gouthamr | vkmc: if unspecified, can't it be the project that the user belongs to? | 19:59 |
| dviroel | I think that gouthamr is suggesting to do what the current client does, infer from user request. | 19:59 |
| vkmc | gouthamr, that's what the current client does | 19:59 |
| gouthamr | vkmc: which will, in most cases be the admin project | 19:59 |
| gouthamr | vkmc: yeah, isn't that intuitive? | 19:59 |
| vkmc | I dunno, users feedback? | 20:00 |
| * gouthamr reads conversation - sees vkmc agrees, earlier | 20:00 | |
| dviroel | and every tenant_id became optional in the new client | 20:00 |
| vkmc | jose castro is out.. hmm | 20:00 |
| gouthamr | vkmc: what do the nova/cinder quota commands do? | 20:01 |
| vkmc | wwc/nd | 20:01 |
| vkmc | let me see | 20:01 |
| dviroel | i think that just need to be clear in the help message | 20:01 |
| gouthamr | dviroel: +1, although the "openstack quota show" command doesn't have it in the help text | 20:02 |
| gouthamr | i haven't looked at this change, i was wondering if we could just plug into that command | 20:02 |
| gouthamr | is that possible at all? | 20:03 |
| dviroel | my opinion is that should keep the current client behavior, and fix some quota command that makes it mandatory | 20:03 |
| dviroel | in the current client, quota-update has tenant_id as mandatory. | 20:04 |
| dviroel | just quota-update iirc | 20:04 |
| vkmc | https://github.com/openstack/python-openstackclient/blob/4a2aa4acc15ede25c925fa775a3a06dc710de7fd/openstackclient/common/quota.py#L588-L617 | 20:04 |
| vkmc | showquota | 20:04 |
| gouthamr | that's weird | 20:05 |
| gouthamr | i think we can make that command smarter, don't you think? | 20:06 |
| gouthamr | osc will have discovered all the entrypoints and available clients | 20:06 |
| gouthamr | we can make it look up quotas pertaining to each individual client | 20:06 |
| gouthamr | and present it in the same table | 20:07 |
| gouthamr | http://paste.openstack.org/show/791211/ | 20:08 |
| gouthamr | that's the output right now ^ | 20:08 |
| gouthamr | this is a tangent to maaritamm's current implementation, but, wouldn't you want the share quotas right there? ^ | 20:08 |
| vkmc | yeah of course | 20:09 |
| vkmc | but | 20:09 |
| vkmc | that's something to fix on the openstackclient | 20:09 |
| vkmc | well, python-openstackclient | 20:09 |
| gouthamr | yes | 20:09 |
| gouthamr | yes | 20:09 |
| gouthamr | isn't that what users will expect :thinking_face: | 20:10 |
| vkmc | usually for most cross-projects in openstack the focus has been put in nova, cinder, glance, neutron... the defcore | 20:10 |
| vkmc | so I can imagine how the decision making process was | 20:10 |
| vkmc | and why the current code looks like that | 20:10 |
| vkmc | I'd like to get user's feedback... in this case cern's input might be the most immediate we can get | 20:11 |
| vkmc | seeing spyros reviews, seems they are happy with the current implementation | 20:11 |
| gouthamr | applies to these common resources: quotas, services, availability zones, limits | 20:11 |
| gouthamr | agree with you on that, and it might be for expediency | 20:12 |
| vkmc | yes :/ | 20:12 |
| gouthamr | well, we can always start with this implementation and deprecate it if we can have a combined solution | 20:13 |
| gouthamr | what do you think? | 20:13 |
| gouthamr | the reason that we plan for deprecation is only because, given our past interactions, things may move really slow if we're trying to drive cross project consensus | 20:14 |
| gouthamr | but, we don't intend to deprive our users over in the meanwhile | 20:14 |
| vkmc | agree | 20:14 |
| vkmc | ok, decisions decisions | 20:22 |
| dviroel | oh, do we have a consensus on that? | 20:23 |
| vkmc | con what specifically? | 20:23 |
| vkmc | on* | 20:23 |
| dviroel | the mandatory project-id, it think that we were discussing that | 20:24 |
| dviroel | s/it/i | 20:24 |
| vkmc | yep, but then we discussed about enhancing python-openstackclient to also include plugins when listing quotas | 20:24 |
| vkmc | so I wanted to make sure we were talking about project-id/tenant-id | 20:24 |
| dviroel | :) | 20:25 |
| vkmc | I say, let's wait for users input... (jose castro/spyros) | 20:25 |
| dviroel | vkmc: ok | 20:25 |
| openstackgerrit | Carlos Eduardo proposed openstack/python-manilaclient master: Add new share replica quotas to the CLI https://review.opendev.org/712677 | 21:01 |
| openstackgerrit | Carlos Eduardo proposed openstack/python-manilaclient master: Add new share replica quotas to the CLI https://review.opendev.org/712677 | 21:03 |
| *** enriquetaso has quit IRC | 21:49 | |
| openstackgerrit | Maari Tamm proposed openstack/python-manilaclient master: Implement OSC share quota commands https://review.opendev.org/703992 | 21:55 |
| *** andrebeltrami has quit IRC | 22:22 | |
| *** sfernand has quit IRC | 22:23 | |
| *** vhari has quit IRC | 22:34 | |
| *** maaritamm has quit IRC | 23:29 | |
| *** tosky has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!