Thursday, 2020-03-12

*** openstackstatus has joined #openstack-cinder00:43
*** ChanServ sets mode: +v openstackstatus00:43
*** andrebeltrami has quit IRC00:54
*** vnayar has joined #openstack-cinder01:28
vnayarI made the changes from last review comments for Change 703907 (Added manage/unmanage volume and snapshot support to Infinidat driver). Reran CI. Is there anything else that I need to do to have this merged? Thanks.01:32
*** vnayar has quit IRC01:42
openstackgerritJon Bernard proposed openstack/cinder master: RBD: fix volume reference handling in clone logic  https://review.opendev.org/60603801:45
*** geguileo has joined #openstack-cinder01:48
*** vishalmanchanda has joined #openstack-cinder01:51
*** zhanglong has joined #openstack-cinder02:23
openstackgerritBrian Rosmaita proposed openstack/cinder master: Make cinder-plugin-ceph-tempest job voting  https://review.opendev.org/71254902:27
*** sapd1_x has joined #openstack-cinder02:45
*** Xuchu_ has joined #openstack-cinder03:20
*** Xuchu has quit IRC03:22
*** Xuchu_ is now known as Xuchu03:22
*** KeithMnemonic1 has joined #openstack-cinder03:28
*** KeithMnemonic has quit IRC03:31
*** psachin has joined #openstack-cinder03:36
*** yaawang has quit IRC03:44
*** yaawang has joined #openstack-cinder03:45
*** ykarel|away is now known as ykarel03:48
*** dave-mccowan has quit IRC03:58
*** KeithMnemonic1 has quit IRC04:06
*** zhanglong has quit IRC04:10
*** udesale has joined #openstack-cinder04:17
*** lvbin02 has joined #openstack-cinder04:35
*** lvbin01 has quit IRC04:37
*** lvbin02 is now known as lvbin0104:37
*** dSrinivas has joined #openstack-cinder04:56
dSrinivasHi All, i am getting this error http://paste.openstack.org/show/790571/  from March 3 in nova-compute logs in our CI FC HBA's are Online and Connectivity exists between vm and Storage04:58
*** zhanglong has joined #openstack-cinder05:31
openstackgerritXuchu Jiang proposed openstack/cinder stable/stein: Add new license scheme for Flashsystem9000 series  https://review.opendev.org/71261205:34
openstackgerritXuchu Jiang proposed openstack/cinder stable/train: Add new license scheme for Flashsystem9000 series  https://review.opendev.org/71261305:34
*** strigazi has quit IRC05:49
*** strigazi has joined #openstack-cinder05:51
*** Luzi has joined #openstack-cinder06:02
*** spatel has joined #openstack-cinder06:22
*** spatel has quit IRC06:28
*** e0ne has joined #openstack-cinder06:28
*** LiangFang has quit IRC07:02
*** strigazi has quit IRC07:03
*** smcginnis has quit IRC07:03
*** dpawlik has joined #openstack-cinder07:04
*** smcginnis has joined #openstack-cinder07:04
*** ChanServ sets mode: +o smcginnis07:04
*** Liang__ has joined #openstack-cinder07:04
*** e0ne has quit IRC07:10
*** e0ne has joined #openstack-cinder07:19
*** e0ne has quit IRC07:23
*** tesseract has joined #openstack-cinder08:08
openstackgerritIvan Pchelintsev proposed openstack/cinder master: Add revert to snapshot support in VxFlex OS driver  https://review.opendev.org/71262208:11
*** tosky has joined #openstack-cinder08:18
*** sapd1_x has quit IRC08:23
*** e0ne has joined #openstack-cinder08:34
openstackgerritIvan Pchelintsev proposed openstack/cinder master: Add revert to snapshot support in VxFlex OS driver  https://review.opendev.org/71262208:36
*** zhanglong has quit IRC08:44
*** xakaitetoia has joined #openstack-cinder08:44
*** pcaruana has joined #openstack-cinder08:54
*** ykarel is now known as ykarel|lunch09:19
*** martinkennelly has joined #openstack-cinder09:42
*** martinkennelly has quit IRC09:46
*** martinkennelly has joined #openstack-cinder09:47
*** Xuchu_ has joined #openstack-cinder09:49
*** Xuchu has quit IRC09:50
*** Xuchu_ is now known as Xuchu09:50
*** sapd1 has joined #openstack-cinder09:50
openstackgerritIvan Kolodyazhny proposed openstack/os-brick master: Prepend platform info to by-path string  https://review.opendev.org/70678009:52
*** martinkennelly has quit IRC09:55
*** martinkennelly has joined #openstack-cinder09:58
*** ociuhandu has joined #openstack-cinder09:59
*** n-saito has quit IRC10:04
*** zhanglong has joined #openstack-cinder10:17
*** ykarel|lunch is now known as ykarel10:19
*** ociuhandu has quit IRC10:25
*** ociuhandu has joined #openstack-cinder10:28
*** dSrinivas has quit IRC10:38
*** pcaruana has quit IRC10:42
*** ociuhandu has quit IRC10:49
*** ociuhandu has joined #openstack-cinder10:50
*** e0ne_ has joined #openstack-cinder10:50
*** e0ne has quit IRC10:51
*** ociuhandu has quit IRC10:53
*** pcaruana has joined #openstack-cinder10:55
*** ociuhandu has joined #openstack-cinder10:58
*** vishalmanchanda has quit IRC11:00
*** sapd1 has quit IRC11:00
*** ociuhandu has quit IRC11:03
*** andrebeltrami has joined #openstack-cinder11:08
*** ociuhandu has joined #openstack-cinder11:20
*** ociuhandu has quit IRC11:27
*** ociuhandu has joined #openstack-cinder11:27
openstackgerritMerged openstack/os-brick master: connectors/nvme: Wait until nvme device shows up in kernel  https://review.opendev.org/67203111:34
*** eharney has joined #openstack-cinder11:52
*** ociuhandu has quit IRC11:54
*** ociuhandu has joined #openstack-cinder11:55
*** ociuhandu has quit IRC11:55
*** ociuhandu has joined #openstack-cinder11:55
*** enriquetaso has joined #openstack-cinder12:09
*** Luzi has quit IRC12:10
*** udesale_ has joined #openstack-cinder12:14
*** zhanglong has quit IRC12:15
*** udesale has quit IRC12:17
*** rcernin has quit IRC12:25
*** tkajinam has quit IRC12:27
*** jvisser has joined #openstack-cinder12:29
*** zhanglong has joined #openstack-cinder12:30
*** Lucas_Gray has joined #openstack-cinder12:43
*** psachin has quit IRC12:47
*** zhanglong has quit IRC12:49
*** Lucas_Gray has quit IRC12:49
*** ociuhandu has quit IRC12:50
*** ociuhandu has joined #openstack-cinder12:51
*** Lucas_Gray has joined #openstack-cinder12:53
*** eharney has quit IRC12:53
*** mmethot has joined #openstack-cinder12:56
*** ociuhandu has quit IRC12:57
*** ociuhandu has joined #openstack-cinder13:00
*** Lucas_Gray has quit IRC13:10
*** psachin has joined #openstack-cinder13:12
*** ianychoi has quit IRC13:13
*** ociuhandu has quit IRC13:15
*** ociuhandu has joined #openstack-cinder13:16
*** Xuchu has quit IRC13:16
*** Xuchu has joined #openstack-cinder13:17
*** ociuhandu has quit IRC13:21
*** spatel has joined #openstack-cinder13:21
*** ociuhandu has joined #openstack-cinder13:22
*** Xuchu has quit IRC13:24
*** spatel has quit IRC13:26
*** ociuhandu has quit IRC13:30
*** ociuhandu has joined #openstack-cinder13:32
*** eharney has joined #openstack-cinder13:38
*** dave-mccowan has joined #openstack-cinder14:02
*** jvisser has quit IRC14:02
openstackgerritEric Harney proposed openstack/cinder master: Tests: Remove Python 2 compat file_spec code  https://review.opendev.org/71210814:02
*** ociuhandu has quit IRC14:11
*** eharney has quit IRC14:13
openstackgerritJon Bernard proposed openstack/cinder master: RBD: fix volume reference handling in clone logic  https://review.opendev.org/60603814:14
*** thgcorrea has joined #openstack-cinder14:16
*** jvisser has joined #openstack-cinder14:24
*** jvisser has quit IRC14:25
*** dave-mccowan has quit IRC14:25
*** jvisser has joined #openstack-cinder14:26
thgcorreaHi, I've got +1 from zuul on this patch, I'd appreciate if anyone could review it: https://review.opendev.org/#/c/710336  thanks14:28
*** jvisser has quit IRC14:29
*** jvisser has joined #openstack-cinder14:29
*** dave-mccowan has joined #openstack-cinder14:29
smcginnisthgcorrea: No third party CI results yet. Any idea when that's coming?14:31
thgcorrea@smcginnis  I see, working on your comment rn, thanks14:34
*** dSrinivas has joined #openstack-cinder14:37
dSrinivasHi all, while attaching the volume to the Nova instance for FC it is attaching and immediately detaching.   http://paste.openstack.org/show/790596/14:40
smcginnisdSrinivas: There's probably something in other logs indicating what the failure is. So you're only seeing the one side of the interaction.14:41
*** ociuhandu has joined #openstack-cinder14:42
dSrinivasIn cinder volume their is no error14:42
dSrinivasFor attaching volume to the instance Nova is responsible right?14:44
*** ociuhandu has quit IRC14:47
*** psachin has quit IRC14:52
whoami-rajatdSrinivas, looks like it cannot find the target device and i also see multipathing is disabled14:54
*** spatel has joined #openstack-cinder14:54
whoami-rajatdSrinivas, did you try with multipathing enabled?14:55
dSrinivasEven if enable the multipath also same issue14:55
dSrinivashttp://paste.openstack.org/show/790595/14:55
whoami-rajatdSrinivas, which backend?14:56
dSrinivasFC14:56
*** ociuhandu has joined #openstack-cinder14:57
*** spatel has quit IRC14:57
whoami-rajatdSrinivas, i mean vendor14:57
dSrinivasKaminario K2 Array14:57
*** spatel has joined #openstack-cinder14:58
dSrinivaswhoami-rajat, it is attaching and deleting immediately14:59
*** Lucas_Gray has joined #openstack-cinder15:00
whoami-rajatdSrinivas, i only see kaminario iscsi CI running on gate. we've tempest tests to test attach detach workflow but since the FC CI isn't running, i'm not sure if FC is stable.15:00
*** Xuchu has joined #openstack-cinder15:01
dSrinivasActually we stopped running patches for FC because it is failing15:01
dSrinivasFrom March 215:01
*** ociuhandu has quit IRC15:02
*** enriquetaso has quit IRC15:02
*** enriquetaso has joined #openstack-cinder15:02
dSrinivaswhoami-rajat, any code change in os-brick that is making the attaching fail15:03
*** spatel has quit IRC15:03
whoami-rajatdSrinivas, not any recent change i'm aware of in the FC part15:05
dSrinivaswhoami-rajat, ok but how can I solve that issue. Can you please help me15:06
dSrinivaswhoami-rajat, how can I debug, where to check inorder to solve that issue can you please direct me15:14
*** Lucas_Gray has quit IRC15:14
whoami-rajatdSrinivas, i'm not sure i could help much. if it was functioning properly before, you should see, when did the issue start? how was this issue discovered? what all changed at that time15:16
*** Lucas_Gray has joined #openstack-cinder15:16
dSrinivaswhoami-rajat, from March 3 it started15:17
whoami-rajatdSrinivas, does create volume from image works successfully? it also goes through os_brick IIRC15:17
dSrinivaswhoami-rajat, for FC I don't know exactly15:18
*** dSrinivas has quit IRC15:24
*** Lucas_Gray has quit IRC15:30
openstackgerritHelen Walsh proposed openstack/cinder master: PowerMax Driver - Legacy volume not found  https://review.opendev.org/71271415:31
*** dSrinivas has joined #openstack-cinder15:31
*** Lucas_Gray has joined #openstack-cinder15:32
*** ykarel is now known as ykarel|away15:34
*** dSrinivas has quit IRC15:36
*** JaySee has quit IRC15:40
*** sapd1 has joined #openstack-cinder15:42
*** lvbin02 has joined #openstack-cinder15:48
*** eharney has joined #openstack-cinder15:49
*** lvbin01 has quit IRC15:52
*** lvbin02 is now known as lvbin0115:52
*** spatel has joined #openstack-cinder16:00
*** spatel has quit IRC16:01
*** spatel has joined #openstack-cinder16:01
*** ociuhandu has joined #openstack-cinder16:15
*** ociuhandu has quit IRC16:19
*** dpawlik has quit IRC16:28
*** dpawlik has joined #openstack-cinder16:28
*** Xuchu has quit IRC16:30
*** Xuchu has joined #openstack-cinder16:34
*** jvisser has quit IRC16:37
*** jvisser has joined #openstack-cinder16:37
*** enriquetaso has quit IRC16:38
*** sapd1 has quit IRC16:39
*** tesseract has quit IRC16:40
*** udesale_ has quit IRC16:49
openstackgerritJon Bernard proposed openstack/cinder master: RBD: fix volume reference handling in clone logic  https://review.opendev.org/60603816:49
*** e0ne_ has quit IRC17:06
openstackgerritMerged openstack/cinder master: Make cinder-plugin-ceph-tempest job voting  https://review.opendev.org/71254917:34
hemnasmcginnis ping17:42
hemnaanyone here todya?17:44
hemnatoday even17:44
*** kozhukalov has joined #openstack-cinder17:46
*** andrebeltrami has quit IRC18:03
*** Lucas_Gray has quit IRC18:05
smcginnishemna: Hey!18:08
hemnaso I wanted to throw some stuff by you18:08
* smcginnis puts on helmet18:08
hemnaI'm working on the vmware driver to fix the get_volume_stats missing features and issues18:09
hemnanamely vmware has what's called a datastore that actually holds the volume18:09
hemnathe driver doesn't expose those datastores individually, so there is no capacity scheduling ability at all against each of them18:09
hemnaand they can get 100% and then become unusable18:10
smcginnisYeah, I think they're kind of like a pool, but not really because you can actually pool multiple datastores.18:10
hemnaso I wanted to expose each datastore as a separate pool18:10
hemnathen we can have the scheduler work properly18:10
smcginnisI haven't had to do much with pools. Do we allow volumes moving from pool to pool?18:11
hemnawell, the problem I just found out is that the vmware driver doesn't really create the volume at create_volume time18:11
hemnayah, that's a retype18:11
hemnaeach pool is a cinder host18:11
hemnawhen they are reported properly18:11
hemnathe retype lands on the driver that handles each pool and if it's the same host, then the driver can do it internally w/o copying data, etc.18:12
hemnaanyway18:12
hemnathe friggin vmware driver choose later on where the volume lives18:12
hemnameaning at attach time it can actually provision the volume on any datastore it manages18:12
hemna:(18:12
hemnaso if I expose each datastore as a pool18:13
hemnathe scheduler picks pool A18:13
hemnauser attaches volume18:13
hemnaand the vmware driver says....create the volume on datastore B18:13
hemnafuct18:13
hemnaI think I can do a model update at create_export time18:13
hemnabut the quotas are fuct at that point I think no?18:13
smcginnisI believe the volume create API exposed by pyvmomi allows specifying datastore. It will just choose where to place it if it's not specified.18:13
smcginnisBut yeah, that would mess with quotas.18:14
hemnahrmm18:14
hemnaso I'm not sure what to do18:14
hemnaI'm assuming a model_update can result in a new pool that what was already chosen18:15
smcginnisYeah18:15
hemnahttps://github.com/openstack/cinder/blob/master/cinder/volume/manager.py#L1833-L184318:16
hemnathat's the model update for create_export18:16
hemnaso even if I changed the driver to do the work inside of create_export to provision the volume and change the pool and report that back18:16
hemnathe quotas are now hosed18:16
hemnaI think vmware is the only driver that does a lazy volume create18:17
smcginnisLooks like there's a config option? self.configuration.vmware_lazy_create18:19
hemnayah18:20
hemnawhich really is needed18:20
hemnaalmost 100% of the time18:20
hemnabecause of vmware datastore limitations18:20
hemnavmware sucsk18:20
hemnadatastores can only have attachments to 64 unique hosts.18:20
smcginnisDatastore name is passed in when the backing volume is created:18:21
smcginnishttps://github.com/openstack/cinder/blob/56e22584cf14a91e8709301afe3a4b63655ae1a8/cinder/volume/drivers/vmware/volumeops.py#L84318:21
hemnaso, you really can't pick which datastore you create the volume in, until you know which host wants the volume.18:21
hemnayah there is a mechanism at initialize_connection time to pick which datastore to use, due to which host18:22
hemnaa datastore might be 'host full' even though it has plenty of space left.18:22
hemnameaning it can't connect to the new host the volume wants to attach to, because the datastore has reached it's host limit. :(18:22
hemnasuch a janky system18:23
smcginnisI've never heard of the 64 limitation. That's the max number of datastores that can be put in a datastore cluster, but that's something else entirely.18:23
smcginnisAnd also something else that would need to be handled.18:23
*** martinkennelly has quit IRC18:39
hemnaI wish the vmware guys were still supporting the driver and active18:46
hemnathe nova folks have deprecated the vmware driver, since vmware hasn't been around to support it in ages.18:46
hemnain the mean time, I'm trying to convince folks internally to switch to kvm18:46
smcginnisI think they are, but it's a newer team without the history that vbala had. Plus they're in China, so... timezones.18:46
*** e0ne has joined #openstack-cinder18:48
*** e0ne has quit IRC18:52
hemnait's like I need a rescheduling of the volume at attach time18:53
hemnadumb question18:53
hemnaare quotas on a per host basis?18:53
hemnaor just per project?18:53
hemnaor on anything?18:54
* hemna is quota challenged18:54
smcginnisProject18:54
*** gmann is now known as gmann_lunch19:00
*** kozhukalov has quit IRC19:13
*** spatel has quit IRC19:29
*** spatel has joined #openstack-cinder19:29
*** e0ne has joined #openstack-cinder19:36
*** trident has quit IRC19:51
*** trident has joined #openstack-cinder19:59
*** gmann_lunch is now known as gmann20:01
*** pcaruana has quit IRC20:20
*** spatel has quit IRC20:27
*** ociuhandu has joined #openstack-cinder20:42
*** ociuhandu has quit IRC21:02
*** ociuhandu has joined #openstack-cinder21:03
*** ociuhandu has quit IRC21:08
*** N3l1x has quit IRC21:09
*** vnayar has joined #openstack-cinder21:14
*** e0ne has quit IRC21:19
*** enriquetaso has joined #openstack-cinder21:28
openstackgerritFernando Ferraz proposed openstack/cinder master: NetApp SolidFire: Add active/active replication  https://review.opendev.org/71279921:31
*** spatel has joined #openstack-cinder21:31
*** enriquetaso has quit IRC21:32
*** senrique_ has joined #openstack-cinder21:32
*** spatel has quit IRC21:36
*** senrique_ has quit IRC21:38
*** ociuhandu has joined #openstack-cinder21:41
*** abishop has quit IRC21:42
*** vnayar has quit IRC21:44
*** CeeMac has quit IRC21:50
*** CeeMac has joined #openstack-cinder21:56
*** ociuhandu has quit IRC22:03
*** ociuhandu has joined #openstack-cinder22:04
*** rcernin has joined #openstack-cinder22:04
*** rcernin has quit IRC22:04
*** rcernin has joined #openstack-cinder22:05
*** ociuhandu has quit IRC22:09
*** abishop has joined #openstack-cinder22:21
*** abishop has quit IRC22:23
*** abishop has joined #openstack-cinder22:23
*** abishop_ has joined #openstack-cinder22:26
*** abishop has quit IRC22:27
*** dave-mccowan has quit IRC22:50
*** tkajinam has joined #openstack-cinder22:56
*** dave-mccowan has joined #openstack-cinder23:09
*** threestrands has joined #openstack-cinder23:50

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!