Thursday, 2017-02-02

*** xyang1 has quit IRC00:12
*** sdague has quit IRC00:21
*** ducttape_ has quit IRC00:39
*** ducttape_ has joined #openstack-meeting-cp00:39
*** ducttape_ has quit IRC00:41
*** _ducttape_ has joined #openstack-meeting-cp00:41
*** _ducttape_ has quit IRC00:42
*** ducttape_ has joined #openstack-meeting-cp00:42
*** diablo_rojo_phon has quit IRC02:40
*** gouthamr has quit IRC03:24
*** ducttape_ has quit IRC04:36
*** ricolin has joined #openstack-meeting-cp05:29
*** ducttape_ has joined #openstack-meeting-cp05:37
*** ducttape_ has quit IRC05:42
*** cartik has joined #openstack-meeting-cp05:58
*** markvoelker has joined #openstack-meeting-cp06:10
*** markvoelker_ has quit IRC06:12
*** beekhof has quit IRC06:13
*** lcastell has quit IRC06:13
*** lcastell has joined #openstack-meeting-cp06:15
*** diablo_rojo has joined #openstack-meeting-cp06:31
*** cartik has quit IRC06:37
*** cartik has joined #openstack-meeting-cp06:41
*** ducttape_ has joined #openstack-meeting-cp07:08
*** cartik has quit IRC07:08
*** diablo_rojo has quit IRC07:09
*** ducttape_ has quit IRC07:13
*** rarcea has joined #openstack-meeting-cp07:23
*** ducttape_ has joined #openstack-meeting-cp08:38
*** ducttape_ has quit IRC08:43
*** ricolin has quit IRC08:46
*** MarkBaker has joined #openstack-meeting-cp10:08
*** ducttape_ has joined #openstack-meeting-cp10:09
*** ducttape_ has quit IRC10:14
*** ducttape_ has joined #openstack-meeting-cp11:10
*** ducttape_ has quit IRC11:14
*** ducttape_ has joined #openstack-meeting-cp11:30
*** ducttape_ has quit IRC11:57
*** ducttape_ has joined #openstack-meeting-cp12:00
*** sdague has joined #openstack-meeting-cp12:09
*** ducttape_ has quit IRC12:22
*** ducttape_ has joined #openstack-meeting-cp12:42
*** ducttape_ has quit IRC12:53
*** MarkBaker has quit IRC12:58
*** ducttape_ has joined #openstack-meeting-cp13:02
*** gouthamr has joined #openstack-meeting-cp13:11
*** lamt has joined #openstack-meeting-cp13:47
*** ducttape_ has quit IRC13:57
*** lamt has quit IRC14:04
*** ducttape_ has joined #openstack-meeting-cp14:16
*** MarkBaker has joined #openstack-meeting-cp14:24
*** ducttape_ has quit IRC14:27
*** sdague has quit IRC14:34
*** lamt has joined #openstack-meeting-cp14:39
*** ducttape_ has joined #openstack-meeting-cp15:14
*** sdague has joined #openstack-meeting-cp15:21
*** lamt has quit IRC15:26
*** sdague has quit IRC15:30
*** sdague has joined #openstack-meeting-cp15:33
*** breitz has joined #openstack-meeting-cp15:34
*** lamt has joined #openstack-meeting-cp15:37
*** edtubill has joined #openstack-meeting-cp15:40
*** MarkBaker has quit IRC15:54
*** _ducttape_ has joined #openstack-meeting-cp16:00
*** _ducttape_ has quit IRC16:01
*** ducttape_ has quit IRC16:03
*** stvnoyes has quit IRC16:06
*** ducttape_ has joined #openstack-meeting-cp16:06
*** stvnoyes has joined #openstack-meeting-cp16:06
*** kberger has joined #openstack-meeting-cp16:25
*** MarkBaker has joined #openstack-meeting-cp16:28
*** ayoung is now known as ayoung-dentist16:32
*** ducttape_ has quit IRC16:38
*** ducttape_ has joined #openstack-meeting-cp16:39
ildikov#startmeeting cinder-nova-api-changes17:00
openstackMeeting started Thu Feb  2 17:00:16 2017 UTC and is due to finish in 60 minutes.  The chair is ildikov. Information about MeetBot at http://wiki.debian.org/MeetBot.17:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:00
*** openstack changes topic to " (Meeting topic: cinder-nova-api-changes)"17:00
openstackThe meeting name has been set to 'cinder_nova_api_changes'17:00
ildikovscottda DuncanT ameade cFouts johnthetubaguy jaypipes takashin alaski e0ne jgriffith tbarron andrearosa hemna erlon mriedem gouthamr ebalduf patrickeast smcginnis diablo_rojo gsilvis  xyang1 raj_singh lyarwood17:00
scottdahi17:00
hemnayough17:00
breitzhi17:00
lyarwood\o_17:00
smcginnisThat time again? :)17:01
ildikovhi :)17:01
ildikovI will need to switch to my phone in about 25 minutes as I need to get from one place to the other17:01
jgriffithwait, what day is it?17:01
ildikov#chair scottda smcginnis17:01
openstackCurrent chairs: ildikov scottda smcginnis17:01
ildikovjgriffith: don't worry it's Thursday :)17:01
jgriffith:)17:02
*** MarkBaker has quit IRC17:02
ildikovI pinged the Nova guys, hopefully they can join17:02
ildikovI updated the etherpad with the latest items17:03
ildikov#link https://etherpad.openstack.org/p/cinder-nova-api-changes17:03
ildikovI think the most urgent is to figure out how to get Cinder V3 tested on the gate with Nova17:03
jgriffithildikov +117:03
ildikovboth from the perspective of the new API and V3 in general17:04
smcginnisjgriffith: Were you still hitting issues trying to run just v3?17:04
scottdadid mattR put up a patch for that?17:04
ildikovwe need some updates so we can pass extra config options to devstack on the gate to use the right endpoints, etc17:04
jgriffithsmcginnis yup17:04
ildikovscottda: https://review.openstack.org/#/c/326585/17:04
smcginnisAnyone else tried this?17:04
ildikovscottda: https://etherpad.openstack.org/p/infra-ptg-pike17:05
scottdaI've been running v3 with nova and it's working17:05
jgriffithsmcginnis I actually just tried running master against V3 (without the new attach stuff) and it seems broken17:05
jgriffithscottda oh really?17:05
jgriffithscottda you had a succesful tempest run?  do tell?17:05
scottdajgriffith: Not with nova patches for new attach stuff, just using old stuff and cinder v317:05
scottdano, haven't run tempest17:05
ildikovscottda: tempest you mean or manual testing?17:05
ildikovscottda: ok :)17:06
scottdaI actually ran yesterday and had failures with volume create and delete stuff timing out.17:06
jgriffithscottda so I've been trying to just get a clean tempest run WITHOUT new attach/detach in Nova, just setting nova/volume/cinder.py endpoint to V317:06
jgriffithscottda I get all sorts of failures for invalid parameters, unexpected results etc etc17:06
scottdajgriffith: yes, me too, and getting failures as well.17:06
jgriffithwell what the heck?17:07
jgriffithcan I just move the new code back into V2 then? and we can remove all the V3 stuff since it doesn't work anyway :)17:07
scottdabut I'm setting nova.conf to use cinder v3. and seeing errors in unexpected places...17:07
scottdajgriffith: sure, go for it.17:07
jgriffithscottda yeah, because we never tested any of this :(17:07
* bswartz sneaks in late17:08
ildikovthe basic volume create works with manual testing, so it must be some config issue I guess or at least partially that17:08
smcginnisI'm confused. It should be the exact same code when not using microversions.17:08
ildikovsmcginnis: yeah, that's why I think there's something with config17:09
jgriffithsmcginnis emphasis on *should*17:09
scottdayeah, v2 and v3.0 are the same17:09
smcginnisAs our microversion expert, I blame scottda. :P17:10
scottdaI can set nova to use cinder v3 and attach and detach. I can verify the calls go through cinder v317:10
scottdaI can use my POC patch to use cinder v3.2717:10
ildikovit would be great to figure this out until the PTG, so if we can get the devstack bits through then we can turn on testing in the placement API job in Nova17:11
scottdaI'll work on finding and debugging tempest failures17:11
scottdaAnd I'm working on getting rid of the term "microversions" and using "API versions"17:11
ildikovto get at least V3 tested on a regular basis as a starting point17:11
scottda:)17:11
ildikovscottda: or just use apple pie :)17:12
scottdaha..I prefer pizza17:12
ildikovI'm fine with that too :)17:13
scottdaWell, ultimately the nova people will need to decide how to use the new cinder apis, based on how to figure out if the cinder server supports them17:13
jgriffithscottda ummmm17:14
jgriffithscottda that's not really how this community works... "Here, let me throw this over the wall... now it's your problem"17:15
scottdajgriffith: That's now what I'm saying17:15
scottdaI'm talking about in the code for nova/compute/manager.py17:15
scottdain attach17:15
jgriffithscottda "Well, ultimately the nova people will need to decide how to use the new cinder apis"17:15
ildikovscottda: which part of that?17:15
scottdaThere could be: try:...new stuff   except: use old stuff17:15
ildikovthere already is17:15
scottdaor some kind of if:   is_new_stuff()  else: use old stuff17:16
jgriffithscottda ildikov and I have already been working on that17:16
ildikovscottda: in compute/manager.py it works like a charm already for attach17:16
jgriffithscottda the problem is the Cinder V3 stuff doesn't seem to work right17:16
smcginnisWe need to make sure the new stuff is working right. But I think scottda's saying _how_ those get implemented in Nova and handled will need to be up to the nova team.17:16
jgriffithscottda the only calls that work are the new attach/detach calls17:16
scottdajgriffith: Not for me. They all work17:16
ildikovscottda: we manual tested it with jgriffith only as tempest is broken anyhow and used hardcoded numbers, but the mechanism itself works17:16
jgriffithscottda you're confusing me... I *thought* you said you saw tempest failures as well?17:17
ildikovsmcginnis: if we make it work they will be fine, that's why we play with the PoC, but they will have the final word on it for sure17:17
jgriffithscottda ok, if they all work for you, paste your config, I'll give it another go and see what I missed17:17
ildikovsmcginnis: unfortunately we don't have merge rights for that repo... :)17:17
jgriffithbut on my side any time nova calls volume-create/delete/snapshot it fails17:17
ildikovsmcginnis: or maybe luckily, I'm not sure :)17:18
scottdaI saw tempest failures that seemed unrelated.17:18
lyarwoodjgriffith: do you have your nova POC change to hand?17:18
smcginnisildikov: ;)17:18
jgriffithlyarwood https://review.openstack.org/#/c/330285/17:18
lyarwoodjgriffith: thanks17:18
jgriffithlyarwood I modified that a bit and just set the URL by brute force in my cinder.py file17:18
scottdaI have this in /etc/nova/nova.conf17:19
scottdahttps://www.irccloud.com/pastebin/j1ynr31m/17:19
scottdaand restart n-cpu17:19
jgriffithscottda ok, I'll set that and run tempest17:19
jgriffithscottda thanks17:19
scottdaand I use this: https://review.openstack.org/42020117:19
ildikovscottda: yeap, that's what I use too17:19
ildikovscottda: I think you can leave out the manager.py part from that patch and just get the microversion part fixed17:23
scottdaildikov: yeah, that's just a demo to show it works17:23
scottdaildikov: I don't want to go further until we discuss *how* and *when* to get cinder server version info17:24
scottdaSome folks in cinder suggest we could put something in nova.conf17:24
scottdajust have the admin configure to say " use_cinder_version: 3.27" or something like that17:24
ildikovscottda: for PoC purposes it's fine, we have those parts in compute/api.py and will figure out the manager part as it might get tricky with old/new volumes mixed17:24
ildikovscottda: that's API facing config, that should not go into nova.conf17:25
ildikovscottda: I agree with the Nova folks regarding to that17:25
jgriffithOk, so that's even worse than what I had:  http://paste.openstack.org/show/597404/17:25
jgriffithmy config obviously must be wrong17:26
ildikovjgriffith: :(17:26
jgriffithyeah... all Identity errors17:26
scottdajgriffith: I'm guessing. I'm running now and not seeing errors yet17:26
jgriffithscottda cool, merge it17:26
jgriffith:)17:26
ildikovscottda: smcginnis: I need switch to my phone, but I made you chairs, so if I seem to be lost please end the meeting when it's time17:27
ildikovjgriffith: LOL17:27
smcginnisildikov: Got it.17:27
jgriffithMy URL obviously must be incorrect17:27
ildikovsmcginnis: tnx :)17:27
smcginnisjgriffith: Maybe try isolating the run down to one or a handful of tests to try to see how/where it's breaking down?17:27
jgriffithsmcginnis I did that yesterday :)17:28
jgriffithfirst I want to get to the same level that scottda and ildikov are instead of brute forcing the URL in myself17:28
scottdajgriffith: ATM, I'm running a fresh devstack with only the new cinder v3 in nova.conf17:28
scottdathat's the only change17:29
*** MarkBaker has joined #openstack-meeting-cp17:29
jgriffithscottda alright, well if it works then awesome17:29
scottdaAFter that, I'll test my POC and have nova prefer v3.017:29
scottdathen work on 3.2717:29
ildikovjgriffith: you can hardcore the catalog_info in the Nova code too if you don't like the config file17:30
scottdaildikov: It's be nice to get things working without hard-coded changes.17:30
scottdas/It's/It'd17:30
ildikovscottda: we will switch that one day anyhow :)17:31
ildikovI hope at least17:31
scottdaildikov: OK. I'm not really sure where in the code you are talking about. but OK17:31
smcginnisSo 1) get it working with hard coding, 2) figure out preferred way to do it without hard coding, 3) propose way for nova to start using the microversion calls?17:32
*** MarkBaker has quit IRC17:32
jgriffithildikov yeah, that's what I was doing that I only had like 22 failures17:32
jgriffithbut since scottda gets a clean run must be something wrong on my side, trying to figure it out now17:32
ildikovscottda: where the config stuff is and the catalog_info is set to v2 by default17:32
scottdajgriffith: I haven't gotten a clean run yet...just progressed beyond the 111 failures you've pasted.17:33
scottdaI'm running now17:33
jgriffith9117:33
ildikovjgriffith: I started my tempest run with the new stuff in, but will let you know if I hit any unrelated changes17:33
ildikovs/changes/failures :)17:34
ildikovDo we have anything more regarding Tempest?17:37
smcginnisGuess we can't really do much until we see that passing.17:38
ildikovDon't think so either17:38
ildikovWe need to look into what we would want to discuss on the PTG17:39
*** ducttape_ has quit IRC17:39
ildikovBoth regarding Nova and what we need to still fix on the Cinder part for multi-attach17:40
ildikovAs we still haven't figured out detach in that scenario17:40
scottdahttps://www.irccloud.com/pastebin/fDEInZJz/17:41
ildikovAnd I became very unpopular now it seems :)17:41
scottdaI'll start looking at those 12 failures17:41
smcginnisildikov: Sorry. Yeah, I agree we need to discuss those.17:42
hemnaooh multi-attach.  is that still a 4 letter word?17:42
ildikovThat's a bit more friendly than 9117:42
smcginnishemna: Yes. :)17:43
ildikovhemna: it depends on which four letters you pick :)17:43
* bswartz is sad that multiattach is so hard17:43
scottdalooks to be all "no valid host found"17:43
smcginnisHad to laugh (sadly) to myself yesterday looking back through some old Cinder meeting logs. We were talking about this in June of 2015.17:44
ildikovbswartz: it's not, it's the drivers' fault...17:44
bswartzildikov: lies!17:44
bswartzthe driver interface encourages drivers to not support multiattach, but it can be done17:45
ildikovbswartz: maybe a bit exaggeration but not lie17:45
bswartzlive migration would never have been possible without drivers implementing multiattach through a back door interface17:45
bswartzand live migration works perfectly well on at least a few drivers17:45
ildikovbswartz: I meant the detach issues17:45
bswartzah17:46
ildikovbswartz: you don't have attached to the same host twice with live migration17:46
bswartzildikov: that's a different statement than multi attach not working though17:47
ildikovbswartz: but you can have two instances on the same host with multiattach and that's when things get messy17:47
hemnayup17:47
bswartzI agree we need some kind of smarter ref counting to know when detaches are safe17:47
bswartzthat's can't possibly be blamed on drivers however17:48
bswartzs/that's/that/17:48
ildikovbswartz: if you cannot safely detach I consider that as not working17:48
scottdaTechnically, multi-detach doesn't work17:48
scottdamulti-attach is fine17:48
ildikovbswartz: back ends handle the target differently, one exports a new one for each attachment others have one per host17:49
ildikovso it's not just ref count anymore17:49
ildikovbut you need to know which back end we're using, etc17:50
bswartzthat problem isn't solvable at the driver level, it needs the manager to implement something17:50
ildikovIt needs both I think17:50
bswartzwell yes, the drivers have to implement whatever new logic the manager requires17:51
ildikovYou cannot solve this without driver support17:51
hemnaI thought drivers were supposed to put a shared flag in the return of initialize_connection ?17:51
bswartzildikov: maybe we can agree on "it's all the driver INTERFACE's fault"17:51
ildikovbswartz: I can live with that, yes :)17:52
ildikovso as a summary we need to figure out how to solve this in Cinder17:53
ildikovAlso things are popping up like supporting volume name for the new attach/detach calls17:54
ildikovI added that to the etherpad, we have a bug report for that17:54
ildikovWe can just have bug reports for things that pop up and see what's urgent to fix17:55
bswartzildikov: well we can blame the driver interface on vish because he isn't here17:56
ildikovhemna: I think that flag sounds good17:56
ildikovhemna: we just need to agree that's the way and get the drivers updated17:56
bswartzI think I'm the owner of an ancient bug related to detach not working on the NFS driver for exactly this kind of reason17:56
hemnaildikov, so by default drivers don't support the multi attach flag17:57
jgriffithhemna that could work, but I was also going to add supports_multiattach to the capabilities17:57
hemnaso they'll have to submit a patch to support it.  part of that review could be ensuring that they pass back the shared flag17:57
ildikovbswartz: oh, then plz get involved in fixing this :)17:57
hemnajgriffith, I think we already have one...17:57
jgriffithdefault is False, so if you don't update your driver when we get this all settled then your out of luck17:57
jgriffithhemna can't use it17:58
ildikovhemna: not even LVM?17:58
jgriffithunless I set everybody that has it to False17:58
bswartzildikov: if I had the spare time I would17:58
smcginnisjgriffith: That'd be fair for now.17:58
jgriffithdefeats the purpose of flagging drivers that need updated17:58
jgriffithuntil the terminate_connection stuff is sorted and updated in the driver can't trust it17:58
ildikovbswartz: any idea on how to fix or just testing stuff sometimes is help, so whatever you can chime in is good17:59
bswartzildikov:  I'll do what I can18:00
ildikovbswartz: thanks18:00
hemnayah we already have folks reporting multiattach in capabilities18:00
ildikovwe need to turn this off somehow for now as the code does not prevent from using it18:01
hemnahttps://github.com/openstack/cinder/blob/master/cinder/volume/drivers/pure.py#L54518:01
ildikovWe're out of time  :(18:01
hemnawell, we can submit a patch to set all of those to False18:02
hemna:)18:02
hemnait's a small list of drivers18:02
hemnapure, zte, netapp, ibm18:02
ildikovsounds good to me!18:03
hemna3par as well18:03
ildikovLet's set those false too for now and figure out detach18:04
ildikovDo we have anything else for today?18:04
ildikovOk, let's close this meeting and fix what we agreed on18:06
smcginnisildikov: Thanks18:06
jgriffithildikov scottda smcginnis FYI clean tempest run!18:06
ildikovHave a nice rest of your day18:06
smcginnisjgriffith: Really? Awesome. What was the difference?18:06
scottdajgriffith: Cool18:06
ildikovjgriffith: awesome!18:07
jgriffithsmcginnis the URL being used if V218:07
scottdajgriffith: What do you have for RAM and number of cpus?18:07
jgriffith"GET /v2/422f4b0126b4473f93f7e3ce46918c32/volumes/ce025819-f959-46a9-818:07
jgriffithvs18:07
ildikovjgriffith: I knew it!!! :)18:07
jgriffith"GET /v3/422f4b0126b4473f93f7e3ce46918c32/volumes/ce025819-f959-46a9-818:07
smcginnisDarn version #s. :)18:07
jgriffith8 gig and 8 cores18:07
scottdahmmm..ok, I"ve 8GB and 2 cores18:08
jgriffithMy hack was using the 'v3'18:08
jgriffithbut I think you may miss the point :)18:08
scottdaI think point is that it works with v218:08
scottda:)18:08
jgriffithscottda :). In that case you got my point :)18:09
ildikovLet's stick with v2 then :)18:09
jgriffithildikov LOL18:09
ildikovDrop v3 and micro versions :)18:09
jgriffithildikov is it Christmas already?18:10
ildikovjgriffith: I see snow so it might be :)18:10
ildikovOk, let's switch to the Cinder channel for more version fun :)18:12
ildikovThank y'all for today  :)18:12
ildikov#endmeeting18:12
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"18:12
openstackMeeting ended Thu Feb  2 18:12:44 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)18:12
openstackMinutes:        http://eavesdrop.openstack.org/meetings/cinder_nova_api_changes/2017/cinder_nova_api_changes.2017-02-02-17.00.html18:12
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/cinder_nova_api_changes/2017/cinder_nova_api_changes.2017-02-02-17.00.txt18:12
openstackLog:            http://eavesdrop.openstack.org/meetings/cinder_nova_api_changes/2017/cinder_nova_api_changes.2017-02-02-17.00.log.html18:12
smcginnisildikov: Thanks for getting the new time set up.18:13
ildikovsmcginnis: no probs, everything for the team! :)18:13
*** ducttape_ has joined #openstack-meeting-cp18:40
*** ducttape_ has quit IRC18:44
*** ayoung-dentist is now known as ayoung18:55
*** bryan_att has quit IRC19:17
*** bryan_att has joined #openstack-meeting-cp19:20
*** ducttape_ has joined #openstack-meeting-cp19:32
*** rarcea has quit IRC20:11
*** harlowja has quit IRC21:00
*** _ducttape_ has joined #openstack-meeting-cp21:06
*** ducttape_ has quit IRC21:10
*** _ducttape_ has quit IRC21:23
*** ducttape_ has joined #openstack-meeting-cp21:24
*** breitz has quit IRC22:14
*** harlowja has joined #openstack-meeting-cp22:25
*** breitz has joined #openstack-meeting-cp22:26
*** ducttape_ has quit IRC22:26
*** ducttape_ has joined #openstack-meeting-cp22:31
*** harlowja has quit IRC22:32
*** harlowja has joined #openstack-meeting-cp22:33
*** edtubill has quit IRC22:36
*** ducttape_ has quit IRC22:43
*** gouthamr has quit IRC23:33

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!