Thursday, 2021-02-04

openstackgerritVictoria Martinez de la Cruz proposed openstack/manila master: Add create share from snapshot in CephFS  https://review.opendev.org/c/openstack/manila/+/74156000:11
openstackgerritVictoria Martinez de la Cruz proposed openstack/manila master: Add create share from snapshot in CephFS  https://review.opendev.org/c/openstack/manila/+/74156000:13
*** sfernand has quit IRC00:32
*** LinPeiWen has joined #openstack-manila00:33
openstackgerritLin PeiWen proposed openstack/manila master: Glusterfs driver supports snapshot revert  https://review.opendev.org/c/openstack/manila/+/76294400:37
*** baojg has joined #openstack-manila01:51
*** dviroel has quit IRC02:13
*** LinPeiWen has quit IRC03:02
*** LinPeiWen has joined #openstack-manila03:02
*** ivan_lin has quit IRC03:04
*** carloss has quit IRC03:06
*** LinPeiWen47 has joined #openstack-manila04:07
*** ivan_lin has joined #openstack-manila04:16
*** LinPeiWen47 has quit IRC04:17
*** LinPeiWen has quit IRC04:18
*** vishalmanchanda has joined #openstack-manila05:19
*** LinPeiWen has joined #openstack-manila05:25
*** lpetrut has joined #openstack-manila07:11
openstackgerritLiron Kuchlani proposed openstack/python-manilaclient master: Forbid users to create shares with the name "None"  https://review.opendev.org/c/openstack/python-manilaclient/+/76860608:35
*** tosky has joined #openstack-manila08:46
*** LinPeiWen has quit IRC08:53
*** LinPeiWen has joined #openstack-manila09:05
*** baojg has quit IRC10:35
*** baojg has joined #openstack-manila10:35
openstackgerritVida Haririan proposed openstack/manila-tempest-plugin master: Added test test_create_snapshot_over_quota_limit  https://review.opendev.org/c/openstack/manila-tempest-plugin/+/77378610:42
*** baojg has quit IRC10:45
*** dviroel has joined #openstack-manila10:47
*** carloss has joined #openstack-manila11:27
*** chuan137 has joined #openstack-manila14:28
*** kafilat has joined #openstack-manila14:52
*** felipe_rodrigues has joined #openstack-manila15:00
*** disap has joined #openstack-manila15:03
*** lpetrut has quit IRC15:23
*** LinPeiWen has quit IRC15:33
gouthamro/ vhari ty for bringing up https://bugs.launchpad.net/manila/+bug/191445316:01
openstackLaunchpad bug 1914453 in OpenStack Shared File Systems Service (Manila) "CephFS Native driver gets blacklisted during startup with Ceph Octopus " [Undecided,New]16:01
vharigouthamr, yw :)16:02
gouthamrinteresting, i wonder if the blacklisting is new in octopus - strange we've not seen it in the CI16:03
gouthamrvkmc ^16:03
gouthamrs/blacklisting/blocklisting - ceph's corrected the terminology on the latest branch: https://docs.ceph.com/en/latest/cephfs/eviction16:07
*** disap has quit IRC16:08
vkmcgouthamr, tbarron pointed me towards it earlier today!16:11
vkmcI need to check that16:12
vkmcI'm also wondering how we didn't catch that on our CI16:12
tbarronnote that manila itself is using nautilus client, not octopus.16:13
gouthamrwe are?16:14
tbarronclient-server version mismatch may be an ingredient16:14
tbarrongouthamr: as reported in the bug at least16:14
gouthamrah i see16:14
tbarronnot that we'd ever see any version incompatibility issues with ceph of course16:15
gouthamrgood eye, we aren't doing that in the CI16:15
gouthamrtbarron: :D16:15
* tbarron ducks16:15
*** hoonetorg has joined #openstack-manila16:20
*** eliaswimmer has quit IRC16:28
openstackgerritVictoria Martinez de la Cruz proposed openstack/manila master: Add create share from snapshot in CephFS  https://review.opendev.org/c/openstack/manila/+/74156016:29
*** ignaziocassano has joined #openstack-manila17:03
ignaziocassanoHello All,  please anyone can help me in share migration on openstack stein ?17:03
ignaziocassanowhat data_node_access_ip means ?17:04
ignaziocassanoI configured  2 netapp backend on my manila.conf and I am able to create share on both, but I am not able to migrate share between them17:05
ignaziocassanomanila shares can be mounted grom my controllers nodes so in the data_node_access_ip I wrote the controllers address .....is right ?17:06
ignaziocassanoAy help, please ?17:09
dviroelhi ignaziocassano17:12
ignaziocassanoHi17:12
dviroelif your are going to use driver_assisted_migration you won't need this config option17:12
ignaziocassanoDriver assisted is not working and it gives me the following error:NaApiError: NetApp API failed. Reason - 13001:Failed to check whether the aggregate "aggr_FAS2_VNX5800" is an NAE (NetApp Aggregate Encryption) aggregate. Reason: Aggregate "aggr_FAS2_VNX5800" does not exist.):17:14
ignaziocassanoIt is strange because I am able to write share on both backends17:14
ignaziocassanoand the above aggregate works fine17:14
ignaziocassanoand axists17:14
ignaziocassanoand exists17:15
dviroeloh, let me take a look17:15
ignaziocassanothe following is my manila pool-list command:17:17
ignaziocassano Name                                                     | Host           | Backend         | Pool                      |17:18
ignaziocassano+----------------------------------------------------------+----------------+-----------------+---------------------------+17:18
ignaziocassano| 10.102.184.190@netapp-nfs-566#aggr_FAS2_VNX5800          | 10.102.184.190 | netapp-nfs-566  | aggr_FAS2_VNX5800         |17:18
ignaziocassano| 10.102.184.190@svm-tst-nfs-565#aggr_fas04_MANILA_VNX5800 | 10.102.184.190 | svm-tst-nfs-565 | aggr_fas04_MANILA_VNX5800 |17:18
ignaziocassano| 10.102.184.190@svm-tst-smb-565#aggr_fas04_MANILA_VNX5800 | 10.102.184.190 | svm-tst-smb-565 | aggr_fas04_MANILA_VNX5800 |17:18
ignaziocassano+----------------------------------------------------------+----------------+-----------------+---------------------------+17:18
ignaziocassanoI am able to create share on netapp-nfs-566 and netapp-tst-nfs-56517:19
ignaziocassanoI cannot migrate from svm-test-nfs-565 to netapp-nfs-56617:20
ignaziocassanoThe driver_assisted fails the the above arror and then switchs in host assisted but it does not work in bot cases17:21
*** openstack has joined #openstack-manila17:25
*** ChanServ sets mode: +o openstack17:25
ignaziocassanoBut I have not understood I can setup it ?17:25
ignaziocassanosorry for my poor english ?17:25
ignaziocassanoIn in host assisted mode controllers need to mount the svm ?17:26
ignaziocassanoWhat "data service" mean ? Is the service on controllers nodes ?17:28
dviroeldata  is another manila service that is reponsible for copying the data in host assisted migration17:31
dviroelyou need to check if your env has this service up17:31
dviroel$ manila service-list17:31
ignaziocassano----+------------------+-----------------------------------+------+---------+-------+----------------------------+17:34
ignaziocassano| Id | Binary           | Host                              | Zone | Status  | State | Updated_at                 |17:34
ignaziocassano+----+------------------+-----------------------------------+------+---------+-------+----------------------------+17:34
ignaziocassano| 28 | manila-share     | 10.102.184.190@svm-tst-nfs-565    | nova | enabled | up    | 2021-02-04T17:33:30.000000 |17:34
ignaziocassano| 31 | manila-share     | 10.102.184.190@svm-tst-smb-565    | nova | enabled | up    | 2021-02-04T17:33:30.000000 |17:34
ignaziocassano| 34 | manila-scheduler | 10.102.184.190                    | nova | enabled | up    | 2021-02-04T17:33:38.000000 |17:34
ignaziocassano| 59 | manila-share     | 10.102.184.190@netapp-nfs-566     | nova | enabled | up    | 2021-02-04T17:33:37.000000 |17:34
ignaziocassanoAs I wrote before I can create share on all of the above bakends17:35
dviroelin host-assisted-migration, data-service is the component that will mount both shares, src and dest, and copy the data for you17:36
dviroelgouthamr: any suggestion in this scenario?17:36
ignaziocassanoOK. Data service are on my controllers and I am able to mount shares from controllers17:37
*** vishalmanchanda has quit IRC17:38
ignaziocassanoSo in data_node_access_ip I wrote the list of my controllers ip addresses17:39
*** ignaziocassano has quit IRC17:42
dviroeldata_node_access_ips shall have the IP of the host running the manila data service17:43
*** ignaziocassano has joined #openstack-manila17:54
ignaziocassanosorry I lost the connection17:55
* gouthamr was in a meeting - reads scrollback17:55
dviroelignaziocassano: which release are you using?17:56
ignaziocassanoopenstack stein on centos17:56
dviroelah ok, because 'data_node_access_ip' was deprecated in train17:57
dviroelin favor of 'data_node_access_ips'17:57
gouthamryeah, the option was renamed to indicate we support multiple IP addresses17:57
ignaziocassanosorry17:58
ignaziocassanoI used data_node_access_ips17:58
dviroeli believe that your config seems to be correct17:58
ignaziocassanoI read about no_root_squash18:01
ignaziocassanoMust I setup it on SVM or on the share I must migrate ?18:01
*** ignaziocassano has quit IRC18:01
*** ignaziocassano has joined #openstack-manila18:04
gouthamrignaziocassano: at some point netapp had some documentation on how to do this specifically with netapp backends18:05
ignaziocassanoOK18:05
ignaziocassanoI will check18:05
ignaziocassanothanks a lot18:05
gouthamri'm not able to find it at the moment, but, i recall you have to make a few manila.conf changes18:05
gouthamrasking the data service to ignore certain files (the .snapshot directory for instance cannot be copied over)18:06
gouthamrignaziocassano: do you use share replication?18:06
ignaziocassanoNo, I do not18:07
gouthamri see, are svm-test-nfs-565 and netapp-nfs-566 on the same host?18:07
gouthamri.e, same ONTAP cluster?18:08
gouthamri ask this because you're going from netapp to netapp, and there may definitely be a easier/more efficient way to do this move yourself, outside of manila18:09
gouthamrbecause as dviroel said, netapp's driver optimized migration only works across pools assigned to the same vserver18:09
ignaziocassanothe svms are on different netapp storages18:10
dviroelif you have snapmirror licenses, you will be able to use snapmirror to copy the data18:11
gouthamr+118:11
ignaziocassanoYes, I know and I also will test it18:12
dviroelthe easiest way would be to create replica using manila, but i think that your share-type doesn't have any 'replication_type' configured, right?18:17
ignaziocassanoyes, it is right18:18
*** eliaswimmer has joined #openstack-manila18:18
dviroelright18:18
ignaziocassanoI shoul create a replication_type and a peering on netapps, correct ?18:18
ignaziocassanoso I could use replication_type dr /ro/rw18:19
ignaziocassano???18:19
dviroelfor netapp driver, it should be 'dr'18:20
dviroelbut the share that your trying to move, is already attached to a share-type18:21
*** kafilat has quit IRC18:21
dviroelone idea would be to: unmanage the share, and manage it again using a new share-type that has a replication_type set to 'dr'18:22
dviroelgouthamr: ^18:22
ignaziocassanoyes18:23
ignaziocassanoI tried last year and it worked18:23
*** chuan137 has quit IRC18:23
ignaziocassanoI used it for copy shares between two sites with different storage and openstack installations18:24
gouthamrdviroel: agree - quickest way since we lack the ability to retype existing shares18:24
ignaziocassanoI managed share on the destination and it loaded shares in openstack DB18:24
*** hoonetorg has quit IRC18:24
ignaziocassanoanycase I am still tryng the migrate procedure with host assisted and I got error Exception during message handling: ShareSnapshotNotFound:18:25
ignaziocassanothe share does not have snapshots18:25
dviroelignaziocassano: if you enable replication for your share, you won't need to handle 'snapmirror' operations in the storage.18:26
ignaziocassanomigration_ignore_files = lost+found, .snapshot, .snapshots18:26
ignaziocassanodviroel, are you sure I do not need snapmirror ?18:27
ignaziocassanoreplication is handled entirely by manila ?18:27
dviroelignaziocassano: yes, we create an snapmirror relationship in the destination18:28
dviroelignaziocassano: and if you use the promote operation, the destination will become the active replica18:28
dviroelignaziocassano: is dhss=false, right?18:28
ignaziocassanoyes18:29
dviroeloh ok18:29
ignaziocassanolet me check if I understood well18:29
ignaziocassanoif I set replication on my shares like suggested in manila.conf by netapp, the dnapmirror relationship im the destination is create by calling netapp api ?18:30
ignaziocassanoI do not need to create it manually on netapp storages ?18:32
dviroelignaziocassano: no, your clusters need to be peered, and since you are using DHSS=False, your SVMs must be peered too18:33
dviroelignaziocassano: you dont need to handle any snapmirror operation in storage18:33
dviroelignaziocassano: https://netapp-openstack-dev.github.io/openstack-docs/train/manila/examples/openstack_command_line/section_manila-cli.html#creating-manila-share-replicas18:34
ignaziocassanoOK, because last year when I tried18:35
ignaziocassanothe configuration on the netapp storages was made by my collegue18:36
ignaziocassanoSo I did not know what my collegue configured18:37
ignaziocassanoAanother question, please18:37
dviroelyou cannot forget to configure the 'replication_domain' in your backend stanzas:18:37
dviroelnow, manila will only knows which backends can replicate shares if you tell on18:37
dviroelwhich backends they can replicate18:37
dviroelhttps://netapp-openstack-dev.github.io/openstack-docs/train/manila/configuration/manila_config_files/section_sample_manila-conf.html#with-replication18:38
ignaziocassanoyjes, replucation domain names of the repllication backends must be the same, right ?18:38
dviroelignaziocassano: yes18:38
ignaziocassanobut replication type rw, what does mean ?18:38
ignaziocassanoI do not think it is possible18:39
dviroelnetapp driver only supports 'dr' replicas18:39
dviroelbut you can promote a 'dr' replica to be active, which means that you current 'rw' will become the 'dr' instancec18:40
dviroelyou will switch the active one18:40
dviroelyou will start to write data in the new location, and you can destroy the old instance by deleting the 'dr' replica18:41
*** sfernand has joined #openstack-manila18:41
*** eliaswimmer has quit IRC18:42
dviroelignaziocassano: don't you want to test it in your env? by creating a new share, only for testing purposes?18:43
*** eliaswimmer has joined #openstack-manila18:43
ignaziocassanoI cannot because I need to ask to storage administrators. I have no access to netapp.18:43
dviroeli see18:44
ignaziocassanoHere in Italy is too late for working and all my collegues are at home18:44
ignaziocassanoMany many thanks for you support18:45
dviroelignaziocassano: we'll be here tomorrow if you want to continue18:45
ignaziocassanoI hope to have the availabilty of my collegues. Any case I will keep in touch when I'll try the replication.18:46
ignaziocassanoHave a good day/evening (depending on where you live) :-)18:47
dviroelignaziocassano: tks, you too18:48
ignaziocassanobye18:48
*** ignaziocassano has left #openstack-manila18:48
*** ChanServ changes topic to "OpenStack Shared File System Service | Documentation: https://docs.openstack.org/manila/latest/ | Meetings: https://wiki.openstack.org/wiki/Manila/Meetings | Wallaby-Review-Priorities: https://etherpad.opendev.org/p/manila-wallaby-review-focus | Join #zorilla-srbac-warroom for review efforts on the Secure RBAC changes | Wallaby PTG Notes: https://etherpad.opendev.org/p/wallaby-ptg-manila"18:51
eliaswimmerHi19:03
eliaswimmererror in unlink: /volumes/_deleting/dd989fa3-cd07-4d03-ae70-0af983fe0e0f/Untitled.ipynb: Permission denied [Errno 13]19:03
eliaswimmerThat is strange19:03
eliaswimmermanila can not delete a share cause it has wrong permissions?19:04
gouthamreliaswimmer: where do you see that error19:07
eliaswimmerin my k8s setup19:07
eliaswimmerwhen deleting a pv19:08
gouthamrstrange that it's on a specific file though?19:08
eliaswimmerthe only one I created?19:08
eliaswimmerI switched from fuse to kernel driver in between19:08
gouthamri see, you're using manila-csi with ceph-csi as a partner-plugin, but with the kernel driver?19:12
eliaswimmeryes19:12
eliaswimmerdid the setup today19:13
eliaswimmercause fuse gives terrible performance, a waste for the nvmes19:13
tbarronand if you create and delete both with kernel driver?19:13
eliaswimmerfirst i will have to delete the files i guess19:14
eliaswimmercause this blocks createing new shares19:14
eliaswimmerok, not really19:14
gouthamri'm curious about the "_deleting" in the path - that's usually where a share group goes19:15
eliaswimmeronly the pvc19:15
tbarroni'm kinda thinking this is going to be a bug that routes through to ceph rather than manila proper or manila-csi19:15
tbarrongouthamr: I believe they have introduced a lazy delete in cephfs cause deletes were slow19:16
tbarronbut my memory is vague19:16
gouthamrtbarron: i see; never noticed that before19:17
eliaswimmerthx you, i will look into it tomorrow19:39
eliaswimmerit's already late here19:40
*** hoonetorg has joined #openstack-manila20:01
*** hoonetorg has quit IRC20:13
*** hoonetorg has joined #openstack-manila20:30
*** lbragstad_ has joined #openstack-manila22:22
*** lbragstad has quit IRC22:24
*** tkajinam has quit IRC22:59
*** tkajinam has joined #openstack-manila22:59
*** lbragstad_ is now known as lbragstad23:03
*** lbragstad has quit IRC23:21
*** lbragstad has joined #openstack-manila23:23
*** sfernand has quit IRC23:59

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!