Wednesday, 2018-05-16

*** dhinesh__ has quit IRC00:02
*** dhinesh has quit IRC00:02
openstackgerritMerged openstack/cinder master: NetApp ONTAP iSCSI: Force exception on online extend  https://review.openstack.org/56786800:13
*** dhinesh has joined #openstack-cinder00:18
*** dhinesh_ has joined #openstack-cinder00:20
*** dhinesh__ has joined #openstack-cinder00:22
*** dhinesh has quit IRC00:22
*** gnufied has quit IRC00:22
*** lixiaoy1 has joined #openstack-cinder00:23
*** stakeda has joined #openstack-cinder00:24
*** dklyle has quit IRC00:24
*** dhinesh_ has quit IRC00:25
*** Nel1x has joined #openstack-cinder00:28
openstackgerritMerged openstack/cinder master: Log backup_service.restore() exception  https://review.openstack.org/56672800:31
openstackgerritMerged openstack/cinder master: V3 jsonschema validation: volume_transfer  https://review.openstack.org/56113300:31
*** dklyle has joined #openstack-cinder00:32
*** Dinesh_Bhor has joined #openstack-cinder00:37
*** _alastor_ has quit IRC00:45
*** dklyle has quit IRC00:53
*** Kensuke has joined #openstack-cinder00:54
*** Kensuke has quit IRC00:54
*** dave-mcc_ has quit IRC00:59
*** zhaochao has joined #openstack-cinder01:03
*** kaisers_ has joined #openstack-cinder01:05
*** kaisers has quit IRC01:08
*** edmondsw has joined #openstack-cinder01:18
*** dklyle has joined #openstack-cinder01:20
*** edmondsw has quit IRC01:22
*** dhinesh__ has quit IRC01:23
*** mriedem_afk has quit IRC01:27
*** dklyle has quit IRC01:29
*** r-daneel has quit IRC01:30
openstackgerritDhinesh Balasubramaniam proposed openstack/cinder master: Hedvig Cinder driver implementation  https://review.openstack.org/43334101:36
*** jiaopengju has joined #openstack-cinder01:37
*** gyankum has joined #openstack-cinder01:55
*** hemna_ has quit IRC01:59
*** bkopilov has quit IRC02:00
*** dave-mccowan has joined #openstack-cinder02:01
*** dalgaaf has quit IRC02:05
*** dalgaaf has joined #openstack-cinder02:06
*** andreas_s has joined #openstack-cinder02:15
*** andreas_s has quit IRC02:19
*** nicolasbock has quit IRC02:24
openstackgerritMerged openstack/cinder stable/ocata: Kaminario K2: Add non discovery iSCSI multipath  https://review.openstack.org/56061402:26
*** _alastor_ has joined #openstack-cinder02:27
*** dave-mccowan has quit IRC02:32
openstackgerritzhuangzejian proposed openstack/cinder master: Add doc about backend_defualt cofig  https://review.openstack.org/56872502:33
*** liverpooler has joined #openstack-cinder02:36
*** gkadam has joined #openstack-cinder02:36
*** dpawlik has joined #openstack-cinder02:42
*** _alastor_ has quit IRC02:45
*** dpawlik has quit IRC02:46
*** liverpooler has quit IRC02:49
*** psachin has joined #openstack-cinder02:49
*** Nel1x has quit IRC02:51
*** KeithMnemonic has quit IRC02:58
*** gkadam has quit IRC03:07
*** wanghao has joined #openstack-cinder03:14
*** Dinesh__Bhor has joined #openstack-cinder03:18
*** gyankum has quit IRC03:20
*** lkuchlan has joined #openstack-cinder03:21
*** Dinesh_Bhor has quit IRC03:21
*** bkopilov has joined #openstack-cinder03:23
openstackgerritMerged openstack/cinder stable/ocata: VMAX driver Ocata - backward compatibility fix  https://review.openstack.org/56061503:26
openstackgerritJeremy Zhang proposed openstack/cinder master: RBD: fix create encrypted volume  https://review.openstack.org/56873403:36
*** rajinir has quit IRC03:39
*** links has joined #openstack-cinder03:41
*** gyankum has joined #openstack-cinder04:10
*** edmondsw has joined #openstack-cinder04:32
openstackgerritDhinesh Balasubramaniam proposed openstack/cinder master: Hedvig Cinder driver implementation  https://review.openstack.org/43334104:32
*** arnewiebalck_ has joined #openstack-cinder04:39
*** yumapath has joined #openstack-cinder04:47
*** abhishekk has joined #openstack-cinder04:48
*** lpetrut has joined #openstack-cinder04:54
*** kaisers_ has quit IRC04:58
*** arnewiebalck_ has quit IRC04:58
*** Dinesh__Bhor has quit IRC05:00
*** wanghao_ has joined #openstack-cinder05:03
*** wanghao has quit IRC05:03
*** gnufied has joined #openstack-cinder05:07
*** chhagarw has joined #openstack-cinder05:08
*** markstur has quit IRC05:09
*** sapd has quit IRC05:17
*** lpetrut has quit IRC05:22
*** kaisers has joined #openstack-cinder05:25
*** wanghao has joined #openstack-cinder05:26
openstackgerritJeremy Zhang proposed openstack/cinder master: RBD: fix create encrypted volume  https://review.openstack.org/56873405:29
*** wanghao_ has quit IRC05:30
yumapathhi does anyone have idea about putting a hba card in passthrough mode05:42
yumapathusing the fc helper script in thrid party config05:42
yumapatham facing some issues wiht that05:42
*** Dinesh_Bhor has joined #openstack-cinder05:42
*** lpetrut has joined #openstack-cinder05:43
*** armaan has joined #openstack-cinder05:47
openstackgerritYu Zhang proposed openstack/cinder master: IBM XIV: Report backend state in service list  https://review.openstack.org/56682605:50
*** sapd has joined #openstack-cinder05:56
*** psachin has quit IRC06:03
*** armaan has quit IRC06:10
*** armaan has joined #openstack-cinder06:10
*** psachin has joined #openstack-cinder06:11
*** andreas_s has joined #openstack-cinder06:14
*** gnufied has quit IRC06:26
*** Dinesh_Bhor has quit IRC06:28
*** hamdyk has joined #openstack-cinder06:30
*** dpawlik has joined #openstack-cinder06:34
*** lvdombrkr has joined #openstack-cinder06:39
*** sar has joined #openstack-cinder06:44
*** edmondsw has quit IRC06:49
*** yumapath has quit IRC06:52
openstackgerritMerged openstack/cinder stable/ocata: NetApp ONTAP: Copy offload bugfix  https://review.openstack.org/56061606:56
*** vivsoni has quit IRC07:05
*** rcernin has quit IRC07:10
*** tesseract has joined #openstack-cinder07:17
openstackgerritMerged openstack/cinder stable/ocata: Solving permission errors due to directory ownership on NFS  https://review.openstack.org/56061807:20
openstackgerritTommyLike proposed openstack/cinder master: Fix create backup API document  https://review.openstack.org/56877007:32
*** lpetrut has quit IRC07:33
*** bhagyashri_s has quit IRC07:36
*** bhagyashri_s has joined #openstack-cinder07:37
*** avishay has joined #openstack-cinder07:39
avishaygeguileo, hey, i'm having an issue with os-brick, looks like a race you fixed recently, but i am running 2.3.0 and the fix should be there...07:40
avishayhttps://pastebin.com/1Q5z7QxP07:40
*** AlexeyAbashkin has joined #openstack-cinder07:42
geguileoavishay: that doesn't look like anything I've fixed...  r:-???07:42
geguileoavishay: it looks like a problem with privsep07:42
avishayhttps://review.openstack.org/#/c/507653/07:43
avishayIt's not that unlink is failing?07:43
geguileoavishay: it doesn't look like it, because there is no name07:44
*** arnewiebalck_ has joined #openstack-cinder07:44
geguileoavishay: that patch fixes the actual unlink of the file07:44
avishaygeguileo, ah i see... any idea where i can look?07:44
geguileoavishay: this looks like a privsep thing, though I could be wrong07:44
*** moshele has joined #openstack-cinder07:45
geguileoavishay: looking back that patch fixed a problem with the os.path.realpath call07:45
geguileonot with the unlink method itself07:45
geguileoavishay: did that happen running devstack?07:46
geguileoavishay: does it happen always?07:46
*** alexchadin has joined #openstack-cinder07:46
avishayno, not devstack.  i'm actually doing something a bit strange... i couldn't update brick due to dependencies so i wrapped it in a simple REST service and put it in a container.  this happens like 50% of the time.07:47
geguileoavishay: I would go and add a logging call in os_brick.privileged.rootwrap.unlink_root method to print the contents of links and kwargs07:47
geguileoif you are in a container... then *anything* can happen   XDXDXD07:48
avishay:)07:48
geguileoI would add that print to see if the call to unlink_root is actually happening07:48
geguileoand if it isn't then I would go look into privsep installation in the container07:49
avishaycool, doing that, thanks!07:50
avishayyea, my prints didn't come up...07:54
*** mikal_ has joined #openstack-cinder07:57
*** arnewiebalck_ has quit IRC07:57
*** mikal has quit IRC07:59
*** dims has quit IRC07:59
*** dims has joined #openstack-cinder08:02
*** armaan has quit IRC08:06
*** armaan has joined #openstack-cinder08:07
*** dims has quit IRC08:07
*** dims has joined #openstack-cinder08:07
*** armaan has quit IRC08:11
*** armaan has joined #openstack-cinder08:11
*** neha_alhat has joined #openstack-cinder08:16
*** edmondsw has joined #openstack-cinder08:20
*** lpetrut has joined #openstack-cinder08:21
*** edmondsw has quit IRC08:25
*** lpetrut has quit IRC08:25
*** sar has quit IRC08:36
*** sar has joined #openstack-cinder08:38
neha_alhatjungleboyj, smcginnis: Hi08:50
neha_alhatjungleboyj, smcginnis: Monty Taylor has submitted patch[1] for the split_logger functionality. Basically this functionality splits logs in four different logs like(keystoneauth.session.request,  keystoneauth.session.response, keystoneauth.session.body, keystoneauth.session.request-id).  I am working on enabling this split_logger functionality for cinder to its internal clients interaction (like glanceclient, keystoneclient08:51
*** alexchadin has quit IRC08:51
*** alexchadin has joined #openstack-cinder08:51
*** gkadam has joined #openstack-cinder08:53
neha_alhatjungleboyj, smcginnis: [1]: https://review.openstack.org/#/c/505764/08:53
*** lixiaoy1 has quit IRC08:55
*** arnewiebalck_ has joined #openstack-cinder09:01
*** arnewiebalck_ has quit IRC09:03
*** abhishekk is now known as abhishekk|meetin09:04
*** links has quit IRC09:07
*** jistr has quit IRC09:09
*** Shilpa has joined #openstack-cinder09:11
*** jistr has joined #openstack-cinder09:12
*** johanssone has joined #openstack-cinder09:16
*** arnewiebalck_ has joined #openstack-cinder09:17
*** alexchadin has quit IRC09:18
*** kaisers has quit IRC09:19
*** arnewiebalck_ has quit IRC09:21
*** alexchadin has joined #openstack-cinder09:21
*** arnewiebalck_ has joined #openstack-cinder09:23
*** links has joined #openstack-cinder09:23
*** sar has quit IRC09:32
*** sar has joined #openstack-cinder09:35
*** arnewiebalck_ has quit IRC09:39
zigoHi there!09:42
zigoTesting Debian packages with puppet-openstack, I got this output on cinder-volume: http://paste.openstack.org/show/721077/09:43
zigoCan someone help me to understand what's going on?09:43
zigoIs there really an issue with Ceph, or is this a Python 3 problem, or something else?09:43
neha_alhatjungleboyj: Hi09:44
*** kaisers has joined #openstack-cinder09:52
*** dpawlik has quit IRC09:56
*** dpawlik has joined #openstack-cinder09:57
*** bkopilov has quit IRC10:05
*** edmondsw has joined #openstack-cinder10:08
*** sapd has quit IRC10:10
*** sapd has joined #openstack-cinder10:11
*** edmondsw has quit IRC10:13
zigoI found the way to fix it, rbd.py needs a .decode("UTF-8"), I'll open the bug and send the patch.10:14
*** alexchad_ has joined #openstack-cinder10:21
*** alexchadin has quit IRC10:24
*** obre has quit IRC10:25
*** abhishekk|meetin is now known as abhishekk10:25
*** obre has joined #openstack-cinder10:25
*** neha_alhat has quit IRC10:26
*** obre has quit IRC10:28
*** obre has joined #openstack-cinder10:28
*** nicolasbock has joined #openstack-cinder10:32
*** e0ne has joined #openstack-cinder10:39
*** abhishekk has quit IRC10:40
*** chhagarw has quit IRC10:41
*** dave-mccowan has joined #openstack-cinder10:41
*** links has quit IRC10:44
*** luizbag has joined #openstack-cinder10:45
*** pooja_jadhav has quit IRC10:54
*** nicolasbock has quit IRC10:54
*** avishay has quit IRC10:56
*** links has joined #openstack-cinder10:58
*** alexchad_ has quit IRC11:06
*** wanghao_ has joined #openstack-cinder11:07
*** wanghao_ has quit IRC11:08
openstackgerritThomas Goirand proposed openstack/cinder master: Fix rbd driver with Python 3  https://review.openstack.org/56881311:09
*** nicolasbock has joined #openstack-cinder11:10
*** wanghao has quit IRC11:10
*** Shilpa has quit IRC11:22
*** armaan has quit IRC11:26
*** armaan has joined #openstack-cinder11:27
*** abishop has joined #openstack-cinder11:31
*** abishop has quit IRC11:34
*** gkadam has quit IRC11:43
*** gkadam has joined #openstack-cinder11:43
*** adisky__ has quit IRC11:49
*** abishop has joined #openstack-cinder11:52
*** pooja_jadhav has joined #openstack-cinder11:54
*** bhagyashri_s has quit IRC11:55
*** abishop has quit IRC11:55
openstackgerritThomas Goirand proposed openstack/cinder master: Fix rbd driver json loading  https://review.openstack.org/56881311:57
*** edmondsw has joined #openstack-cinder11:57
*** alexchadin has joined #openstack-cinder11:57
*** erlon has joined #openstack-cinder11:58
*** edmondsw has quit IRC12:01
*** mriedem has joined #openstack-cinder12:05
*** edmondsw has joined #openstack-cinder12:07
*** edmondsw has quit IRC12:07
openstackgerritAkhil Jayakumar proposed openstack/cinder master: [DNM] Cisco-CI-Experimental-Branch: adding_experimental_cinder_cisco-zonemanager  https://review.openstack.org/56882112:15
openstackgerritThomas Goirand proposed openstack/cinder master: Fix rbd driver json loading  https://review.openstack.org/56881312:20
*** abishop has joined #openstack-cinder12:28
*** bhagyashris has joined #openstack-cinder12:28
*** dklyle has joined #openstack-cinder12:30
*** abishop has quit IRC12:31
*** abishop has joined #openstack-cinder12:32
*** dpawlik has quit IRC12:34
*** liverpooler has joined #openstack-cinder12:35
*** edmondsw has joined #openstack-cinder12:36
*** dklyle has quit IRC12:39
openstackgerritAkhil Jayakumar proposed openstack/cinder master: [DNM] Cisco-CI-Experimental-Branch  https://review.openstack.org/56883112:40
*** armaan has quit IRC12:41
*** wanghao has joined #openstack-cinder12:42
*** armaan has joined #openstack-cinder12:42
*** pchavva has joined #openstack-cinder12:44
*** r-daneel has joined #openstack-cinder12:47
*** armaan has quit IRC12:50
*** armaan has joined #openstack-cinder12:50
*** felipemonteiro has joined #openstack-cinder12:59
tommylikehuhey jungleboyj, could you take a look at this patch when you free? https://review.openstack.org/#/c/552243/12:59
*** felipemonteiro has quit IRC13:03
*** zengyingzhe has joined #openstack-cinder13:07
*** zengyingzhe has quit IRC13:10
*** jiaopengju has quit IRC13:11
openstackgerritAkhil Jayakumar proposed openstack/cinder master: [DNM] Testing Cisco Third Party CI  https://review.openstack.org/56884113:13
*** r-daneel_ has joined #openstack-cinder13:15
*** r-daneel has quit IRC13:16
*** r-daneel_ is now known as r-daneel13:16
openstackgerritAkhil Jayakumar proposed openstack/cinder master: [DNM] Cisco-CI-Experimental-Branch: cinder-znm-dev  https://review.openstack.org/56884113:17
*** gnufied has joined #openstack-cinder13:17
*** neha_alhat has joined #openstack-cinder13:19
neha_alhatjungleboyj, smcginnis: Hi13:19
*** Shilpa has joined #openstack-cinder13:22
*** alexchad_ has joined #openstack-cinder13:36
*** wanghao has quit IRC13:37
*** alexchadin has quit IRC13:38
*** eharney has joined #openstack-cinder13:39
*** chhagarw has joined #openstack-cinder13:40
*** bkopilov has joined #openstack-cinder13:47
*** chhagarw has quit IRC13:50
*** mvk has quit IRC13:52
*** sar has quit IRC13:55
*** kambiz has quit IRC14:04
*** kambiz has joined #openstack-cinder14:05
*** nicolasbock has quit IRC14:06
*** moshele has quit IRC14:07
*** rajinir has joined #openstack-cinder14:07
*** gyankum has quit IRC14:09
*** kambiz has quit IRC14:09
*** crose has joined #openstack-cinder14:12
*** kambiz has joined #openstack-cinder14:13
*** r-daneel has quit IRC14:20
*** lkuchlan has quit IRC14:22
*** nicolasbock has joined #openstack-cinder14:23
*** felipemonteiro has joined #openstack-cinder14:25
*** mvk has joined #openstack-cinder14:29
*** lvdombrkr89 has joined #openstack-cinder14:29
*** hamdyk has quit IRC14:30
*** lvdombrkr has quit IRC14:32
*** r-daneel has joined #openstack-cinder14:38
*** felipemonteiro_ has joined #openstack-cinder14:41
*** felipemonteiro has quit IRC14:44
*** Alexey_Abashkin has joined #openstack-cinder14:44
*** felipemonteiro has joined #openstack-cinder14:44
*** AlexeyAbashkin has quit IRC14:46
*** Alexey_Abashkin is now known as AlexeyAbashkin14:46
*** felipemonteiro_ has quit IRC14:46
*** armaan has quit IRC14:47
*** lkuchlan has joined #openstack-cinder14:52
*** dklyle has joined #openstack-cinder14:53
*** markstur has joined #openstack-cinder14:55
*** links has quit IRC14:58
*** lkuchlan has quit IRC14:59
openstackgerritEric Harney proposed openstack/cinder master: Add cinder-tempest-plugin to LIO job  https://review.openstack.org/56887314:59
*** markstur has quit IRC15:00
*** markstur has joined #openstack-cinder15:02
*** _alastor_ has joined #openstack-cinder15:03
*** dklyle has quit IRC15:12
*** dklyle has joined #openstack-cinder15:18
*** dklyle has quit IRC15:24
*** dklyle has joined #openstack-cinder15:26
*** hemna_ has joined #openstack-cinder15:29
*** andreas_s has quit IRC15:30
*** gnufied has quit IRC15:30
*** andreas_s has joined #openstack-cinder15:30
*** dklyle has quit IRC15:31
*** gyankum has joined #openstack-cinder15:39
*** hemna_ has quit IRC15:42
*** hemna_ has joined #openstack-cinder15:43
jungleboyjneha_alhat:  Hi.15:44
jungleboyjSorry.  Been in and out today.15:44
*** andreas_s has quit IRC15:44
openstackgerritHelen Walsh proposed openstack/cinder master: VMAX driver - Add VMAX specific metadata to volume  https://review.openstack.org/55744815:47
SwansonThat's what sh... wait! let go of me! HR has me! No!15:48
SwansonAs well they should.15:48
jungleboyj?15:48
smcginnis:)15:49
SwansonNothing to see here.15:49
*** andreas_s has joined #openstack-cinder15:49
*** v12aml has quit IRC15:51
openstackgerritHelen Walsh proposed openstack/cinder master: VMAX driver and docs - change san_rest_port to san_api_port  https://review.openstack.org/56092815:56
*** andreas_s has quit IRC15:58
openstackgerritHelen Walsh proposed openstack/cinder master: VMAX driver - Incorrect service level reporting  https://review.openstack.org/56517916:01
*** andreas_s has joined #openstack-cinder16:01
openstackgerritMerged openstack/cinder stable/ocata: VMAX Driver - Ocata, detaches fail when volume missing from backend  https://review.openstack.org/56061916:05
openstackgerritHelen Walsh proposed openstack/cinder master: VMAX driver - Deleting snapshot that is source of multiple volumes fails  https://review.openstack.org/56524116:05
*** itlinux has joined #openstack-cinder16:05
*** andreas_s has quit IRC16:06
*** alexchad_ has quit IRC16:07
*** v12aml has joined #openstack-cinder16:08
*** jmlowe has quit IRC16:09
openstackgerritHelen Walsh proposed openstack/cinder master: VMAX driver - remove deprecated XML option  https://review.openstack.org/56551816:12
*** andreas_s has joined #openstack-cinder16:13
*** edmondsw has quit IRC16:15
*** andreas_s has quit IRC16:18
*** edmondsw_ has joined #openstack-cinder16:18
*** lvdombrkr89 has quit IRC16:22
*** edmondsw_ has quit IRC16:23
*** zhaochao has quit IRC16:25
*** moshele has joined #openstack-cinder16:30
*** felipemonteiro has quit IRC16:31
*** gnufied has joined #openstack-cinder16:33
*** itlinux has quit IRC16:36
*** felipemonteiro has joined #openstack-cinder16:38
*** felipemonteiro has quit IRC16:45
*** gkadam has quit IRC16:46
hemna_is there any way to specify system level packages for deps with pbr ?16:54
hemna_re: non pypi packages16:54
*** links has joined #openstack-cinder16:54
smcginnisbindep?16:54
smcginnisNot with pbr though.16:54
hemna_dang16:54
*** mayur_ind_ has joined #openstack-cinder16:55
*** felipemonteiro has joined #openstack-cinder16:56
*** gnufied has quit IRC16:56
mayur_ind_hi team,16:59
mayur_ind_no new jobs are triggering for our CI.16:59
mayur_ind_Host key verification is failing for new server "review.openstack.org",  this is log entry from zuul/merger.log http://paste.openstack.org/show/721117/16:59
mayur_ind_any pointers for this??16:59
smcginnismayur_ind_: Maybe related to the gerrit server upgrade a few weeks back.17:01
smcginnismayur_ind_: Not sure on key verification. Check that the CI account has the right key(s) registered.17:02
smcginnisProbably get better help in #openstack-infra17:02
mayur_ind_thnks smcginnis. :-)17:02
*** felipemonteiro has quit IRC17:03
*** felipemonteiro has joined #openstack-cinder17:04
hemna_guess there is no way to get pbr to fire off bindep itself during the install process17:05
hemna_hrmm17:05
smcginnishemna_: I don't know enough about pbr.17:06
smcginnisI wonder if there's any tricks you can do in setup.py17:06
hemna_yah, same here...17:06
hemna_I have a py project that needs python-rados17:07
hemna_which happens to only be a system level package, not pypi for whatever reason17:07
smcginnisHuh, odd.17:07
hemna_yah, the ceph project docs say to use yum/zypper/apt to install it, not pypi as it doesn't exist there.17:07
hemna_even though it's a python lib17:07
eharneyit's because the python bindings live inside the non-python Ceph tree, and are just built from that -- same model libvirt used for many years17:10
eharneyhttp://tracker.ceph.com/issues/5900 has some info on this17:12
*** itlinux has joined #openstack-cinder17:12
*** psachin has quit IRC17:14
*** links has quit IRC17:16
*** AlexeyAbashkin has quit IRC17:16
*** gnufied has joined #openstack-cinder17:17
*** diablo_rojo has quit IRC17:21
hemna_eharney, dang ok thanks17:22
hemna_looks like someone created a test pypi package, but that just sat there....17:23
hemna_https://github.com/ceph/ceph/pull/983317:24
*** tesseract has quit IRC17:27
*** mvk has quit IRC17:27
*** mayur_ind_ has left #openstack-cinder17:30
*** imacdonn has quit IRC17:30
*** imacdonn has joined #openstack-cinder17:31
*** dhinesh has joined #openstack-cinder17:38
*** throwsb1 has joined #openstack-cinder17:41
*** lseki has joined #openstack-cinder17:52
*** jmlowe has joined #openstack-cinder17:59
*** gnufied has quit IRC18:02
*** moshele has quit IRC18:08
*** felipemonteiro has quit IRC18:10
*** diablo_rojo has joined #openstack-cinder18:15
*** gyankum has quit IRC18:24
*** felipemonteiro has joined #openstack-cinder18:35
openstackgerritEric Harney proposed openstack/cinder master: Fix rbd driver json loading  https://review.openstack.org/56881318:38
*** lkuchlan has joined #openstack-cinder18:42
throwsb1Hi, last week I was having timing errors with creating a tenant instant in Horizon and even creating a vol in cli.  Turns out that issue was with not all glance_containers had all of the images.  running in OSA clustered env.18:46
throwsb1I was able to get around by syncing all images to all glance_containers.  Issue I am see now is iscsi timeouts.18:47
*** jmlowe has quit IRC18:47
throwsb1I am getting this Trying to connect to iSCSI portal 172.29.244.15:3260 then essentially the tenant vm deploy fails.  However the vol is created, but can attach.18:48
throwsb118:48
throwsb1anyone have any direction I can look?  One thought is moving glance images to shared filesystem.  I do see the volume created within LVM.18:49
smcginnisthrowsb1: iSCSI connections wouldn't have anything to do with glance.18:52
throwsb1smcginnis: thats what I figured,  It is hit or miss in that some times I can get it to work.  I did have this error with a 2 node POC deploy and never found the issue.  I ended up rebuilding and limiting to a single storage node.18:54
smcginnisthrowsb1: You are running multiple cinder nodes?18:54
throwsb1smcginnis: yes 2 blades.  each with 8GB fibre to san18:55
throwsb1using lvm.  The iscsi net is 6GB.  I also changed mtu of storage net to jumbo.18:56
smcginnisthrowsb1: So you are creating LVM pools on your blades using your SAN storage, then have those configured as multiple LVM backends in Cinder.18:56
*** edmondsw has joined #openstack-cinder18:57
throwsb1yup they are listed as 2 storage nodes18:57
throwsb1all under lvm18:57
smcginnisthrowsb1: Maybe verify IP connectivity between all compute nodes and the iSCSI target IPs of your nodes.18:57
smcginnisOr use your SAN directly.18:58
throwsb1SAN is an old HP.  I did not see support.  Connectivity is fine18:58
*** jmlowe has joined #openstack-cinder18:59
smcginnisWell, connectivity isn't fine if your hetting iSCSI target portal login failures.19:00
smcginnis*you're19:00
*** moshele has joined #openstack-cinder19:00
smcginnisUnless it is connecting and everything is fine on the Cinder side but Nova is having some sort of instance creation failure.19:00
smcginnisYou would have to look in your log files to see where it's erroring out.19:00
throwsb1smcginnis: I can verify networking again.  There is no load on the environment.19:01
smcginnisthrowsb1: Was the failure on connecting, or after that.19:02
throwsb1here is what I am seeing after iscsi, ERROR nova.compute.manager [req-2949125a-3040-4a29-8297-757550b6aba7 22a42c20e78e4446a7168116b14a713d 8fce9de5ab3247629eb60a4c44337622 - default default] [instance: 789ead36-cb6f-49cb-a34a-5db50be36bdb] Instance failed to spawn: VolumeDeviceNotFound: Volume device not found at .19:02
throwsb119:02
smcginnisOK, I think you would need to look prior to that in the log and see what happened.19:02
throwsb1easier if I post the log. http://paste.openstack.org/show/721122/19:03
smcginnisLooks like multiple retries trying to connect to the iSCSI portal.19:04
*** e0ne has quit IRC19:05
smcginnisbbl19:05
*** e0ne has joined #openstack-cinder19:05
*** abishop_ has joined #openstack-cinder19:05
*** abishop has quit IRC19:06
*** mriedem1 has joined #openstack-cinder19:09
throwsb1smcginnis:  Thanks for the help, will keep searching.19:11
*** mriedem has quit IRC19:11
*** eharney_ has joined #openstack-cinder19:12
*** eharney has quit IRC19:12
*** mriedem1 is now known as mriedem19:15
*** dklyle has joined #openstack-cinder19:18
*** mvk has joined #openstack-cinder19:22
openstackgerritMerged openstack/cinder stable/ocata: Dell EMC SC: ISCSI initialize_connection fixes.  https://review.openstack.org/56062019:23
*** LutzB has quit IRC19:39
*** LutzB has joined #openstack-cinder19:49
*** david-lyle has joined #openstack-cinder19:49
_alastor_smcginnis: Hey, have you encounter an issue where Cinder retypes between two different backends fail for attached volumes?19:51
*** dklyle has quit IRC19:51
_alastor_It looks like novaclient can't find the right endpoint URL and raises an EndpointNotFound error19:51
_alastor_something to do with the whole "nova" vs "Compute Service" name-scheme change19:52
*** eharney_ is now known as eharney19:56
*** sapd_ has joined #openstack-cinder20:00
*** jmlowe has quit IRC20:00
smcginnis_alastor_: Hmm, no. I haven't seen that one.20:01
_alastor_smcginnis: damn.  Been banging my head against it for a while20:02
*** sapd has quit IRC20:03
_alastor_smcginnis: The value in nova_catalog_admin_info seems to be the culprit.  That opt has been removed in more recent versions of Cinder20:04
throwsb1Hi, was able to dig in a little more and one of my storage nodes is getting login I/O errors when I run a iscsiadm discovery on it.20:08
*** diablo_rojo has quit IRC20:09
throwsb1Anyone have any pointers of how to resolve this?  I banged my head with this same error in a POC environment and could never resolve it other than rebuild.20:09
*** pchavva has quit IRC20:10
*** liverpooler has quit IRC20:11
*** lkuchlan has quit IRC20:11
*** andreas_s has joined #openstack-cinder20:15
*** moshele has quit IRC20:17
*** e0ne has quit IRC20:18
*** andreas_s has quit IRC20:19
*** asbishop has joined #openstack-cinder20:24
*** abishop_ has quit IRC20:27
*** moshele has joined #openstack-cinder20:44
*** luizbag has quit IRC20:45
*** moshele has quit IRC20:46
*** crose has quit IRC21:01
throwsb1smcginnis: hey are you still around?  dug in more and not sure if this is a bug and if so where, cinder or nova.21:02
smcginnisthrowsb1: Sounds like a network issue. Try new patch cables, different switch ports. Log in to switch and check for port flapping or error counters.21:05
smcginnisthrowsb1: Run wireshark on one of the other and see what's happening at the network level.21:05
smcginnisthrowsb1: If you are seeing errors jut running iscsiadm discovery, it's neither Cinder nor Nova.21:06
throwsb1well here is what I found.  Scratched my head on the targets.  When I try to attach a vol in horizon, from the compute server, I can run an iscsiadm and see that the storage node responds.  then at some point it saying trying to connect21:07
throwsb1at which point it fails and target is removed.21:07
throwsb1I did run iftop and saw traffic going between them.21:08
throwsb1smcginnis: as far as network config, these are blades and traffic is staying with in the HP virtualconnect, so no cables.21:09
smcginnisMaybe rather than doing boot from volume, try just booting an instance using ephemeral storage, then attach a volume to the instance. That should cover the IO path between the compute node and the LVM backing.21:09
throwsb1smcginnis: the odd thing is the other storage node works.  I ran into the same issue when I built a poc with 2 rack mounts servers.21:10
throwsb1smcginnis: I can try your suggestion and see what happens.  I will turn up debugging on nova.21:11
throwsb1smcginnis:  thanks for your help.  just one of those nagging issues.  :)21:12
smcginnisthrowsb1: Good luck!21:12
*** itlinux has quit IRC21:13
throwsb1smcginnis:  just curious, do you know offhand if storage net needs mulitcast configured?21:14
smcginnisthrowsb1: No, it shouldn't.21:14
throwsb1ok, thanks21:14
smcginnisthrowsb1: If one node is working and the other isn't. maybe you can do a stare and compare on their configurations to look for anything different between the two.21:15
throwsb1did that.  looks like storage just died on me.  similar to poc.21:15
throwsb1smcginnis: If I am running 2 storage nodes, both lvm, is it better to put them in diff storage groups or it does not matter?21:16
smcginnisthrowsb1: Not sure if I understand what you're asking.21:17
throwsb1the storage type name21:18
throwsb1in this case I am calling them both lvm21:18
*** hemna_ has quit IRC21:18
smcginnisOh, volume type? You should be able to just have one volume type if you are not specifying backend_name, but you could do two separate ones if you wish.21:18
smcginnisCalling them both LVM? Where?21:19
throwsb1I do have a backend name and both are the same.  LVM_iSCSI21:19
smcginnisthrowsb1: No, if you have two separate volume groups they should be two separate backends.21:19
throwsb1here is my config if easier to see what I am doing21:20
throwsb1http://paste.openstack.org/show/721132/21:20
throwsb1this is OSA build21:20
smcginnisI'm not familiar with OSA build, but that doesn't look like a valid configuration.21:21
*** asbishop has quit IRC21:21
smcginnisIn the failure logs, are you sure it was trying to connect to the right iSCSI IP address? Could be getting confused and trying to connect to A when the volume is exposed through B.21:21
throwsb1I did see that hence was asking about the naming21:22
throwsb1I also did seed target node IP's on both nodes which was odd21:22
smcginnisIf you want to treat them as one pool, then you need to set up something like clustered LVM so both hosts can serve the volumes. But I've never done that myself.21:23
throwsb1smcginnis: let me play around with naming and see if I can separate things.21:24
throwsb1not sure if they are doing clustered lvm with openstack ansible.  great question though21:24
throwsb1smcginnis: thanks again for the help.21:24
smcginnisOr since you are just wrapping your SAN storage, you could just do one big volume group on one node. Since point of failure, but then maybe you could set up a standby (passive) backup node that takes over if the first one fails.21:25
smcginnisAnyway, play around and see how it goes. Good luck.21:26
throwsb1was thinking of that or trying ceph21:26
*** dave-mccowan has quit IRC21:32
*** dave-mccowan has joined #openstack-cinder21:41
*** mriedem is now known as mriedem_afk21:41
*** lseki has quit IRC21:47
*** dave-mcc_ has joined #openstack-cinder21:48
*** dave-mccowan has quit IRC21:48
*** itlinux has joined #openstack-cinder21:51
*** hemna_ has joined #openstack-cinder22:06
*** mchlumsky has quit IRC22:13
*** lhinds has quit IRC22:16
dhineshzuul is failing on cinder-tempest-dsvm-lvm-lio-barbican for new driver updates, which used to pass earlier. is it a known issue on the setup?22:23
*** rcernin has joined #openstack-cinder22:25
*** markstur has quit IRC22:26
*** rajinir has quit IRC22:28
*** andreaf has quit IRC22:29
*** andreaf has joined #openstack-cinder22:29
*** itlinux has quit IRC22:33
*** lhinds has joined #openstack-cinder22:35
*** felipemonteiro has quit IRC22:37
*** tpsilva has quit IRC22:44
*** jmlowe has joined #openstack-cinder22:49
*** edmondsw has quit IRC22:50
*** felipemonteiro has joined #openstack-cinder22:59
*** throwsb1 has quit IRC23:02
*** edmondsw has joined #openstack-cinder23:11
*** lhinds has quit IRC23:16
*** ganso has quit IRC23:17
*** hemna_ has quit IRC23:18
*** hemna_ has joined #openstack-cinder23:19
*** lhinds has joined #openstack-cinder23:23
*** uberjay has quit IRC23:39
*** uberjay has joined #openstack-cinder23:40
*** moshele has joined #openstack-cinder23:44
*** felipemonteiro has quit IRC23:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!