Thursday, 2017-04-20

imacdonnit doesn't exist until libvirt creates the domain00:00
*** ssurana has joined #openstack-nova00:01
imacdonnhttps://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L507900:02
mriedemdo you see Converting VIF in the nova-compute logs at debug level?00:02
imacdonnnote that plug_vifs() is called before _create_domain()00:03
mriedemright that's where we start with os-vif00:03
mriedemyeah00:03
mriedemkevinbenton: are you around?00:03
kevinbentonmriedem: AHOy!00:04
mriedemkevinbenton: ^ whats set the MTU on the tap interface?00:04
mriedemfor vif_type == bridge?00:04
*** lyan has quit IRC00:04
mriedemi thought it was os-vif00:05
kevinbentonmriedem: no00:05
kevinbentonmriedem: it's either libvirt internals or it inherits from bridge as part of linux00:05
mriedemor does neutron do it once we plug the vif and neutron agent picks that up?00:05
mriedemhmm00:05
kevinbentonmriedem: this is sort of why i wanted to move away from libvirt plugging things into the bridge00:06
imacdonninheriting from the bridge doesn't work, because the bridge is initially created with MTU 1500, and it won't go any higher until a interface is added .. then it assumes the lowest MTU amongst all member interfaces00:06
kevinbentonimacdonn: hang on00:06
imacdonnk00:06
kevinbentonMTU is set on the bridge by os-vif00:06
mriedemi thought it was this https://github.com/openstack/os-vif/blob/master/vif_plug_linux_bridge/linux_net.py#L9100:07
imacdonnit can't be00:07
imacdonnor .. maybe I have a bug in the OS00:07
*** rfolco_ has quit IRC00:07
imacdonn# brctl addbr test-bridge00:08
imacdonn# ip l set test-bridge mtu 900000:08
imacdonnRTNETLINK answers: Invalid argument00:08
kevinbentonoh hang on00:09
kevinbentonmaybe not00:09
kevinbentoni think that branch in os-vif is not exercised with neutron00:09
kevinbentonbecause https://github.com/openstack/os-vif/blob/master/vif_plug_linux_bridge/linux_bridge.py#L98 is not true with neutron00:10
imacdonnyeah, I only have should_provide_bridge00:10
mriedemok, well i guess that's why then00:15
*** ijw has joined #openstack-nova00:15
kevinbentonhttps://bugs.launchpad.net/ubuntu/+source/linux/+bug/139906400:16
openstackLaunchpad bug 1399064 in linux (Ubuntu) "bridges cannot have a mtu > 1500 by themselves" [Medium,Triaged]00:16
kevinbentonso i wonder if we have a race condition here00:16
kevinbentonwhere nova adding a tap interface to the bridge results in 1500 mtu00:16
kevinbentonsince it's first00:16
kevinbentonand then neutron adds vxlan interface00:17
kevinbentonwith real network mtu00:17
imacdonnyeah, so then when I destroy nova instance, and the tap interface goes away, the bridge inherits the MTU of the VLAN interface, which is 9000 (in my case)00:18
imacdonnthen I launch another nova instance, and its tap interface picks up the bridge's MTU of 9000, and it works00:18
kevinbentonack00:18
kevinbentonthat sounds exactly like the case then00:18
imacdonnit's not really a "race condition" in the multi-threading sense00:18
kevinbentonwell it is in the sense that the neutron agent could conceivably beat nova00:19
kevinbentonafter libvirt creates bridge00:19
imacdonneven if it was possible to set the MTU on an empty bridge, there's still nothign in the code that will set the bridge's MTU00:19
kevinbentonbut before it adds interface00:19
kevinbentonneutron could wire up the vlan/vxlan interface00:19
imacdonnoh, you;re saying that if neutron added the VLAN interface before libvirt created the domain, it'd work ?00:20
kevinbentonright00:20
imacdonnthat can't happen, because the neutron l2 agent triggers on the tap interface's existence00:20
kevinbentonoh duh00:21
imacdonnwell, I guess it could happen if there's a window where the tap interface exists but hasn't been added the bridge yet00:21
kevinbentonyeah00:21
kevinbentonbut the reason we don't see this in the gate is probably because of the dhcp agents, etc00:21
imacdonnBTW, this was solved in the neutron agent in the past00:21
imacdonnbut then it got un-solved by https://git.openstack.org/cgit/openstack/neutron/commit/?id=d352661c56d5f03713e615b7e0c2c9c8688e013200:22
*** brault has quit IRC00:22
kevinbentonhow would that solve it?00:22
imacdonnthe previous solution does seem a little kludgey00:22
*** gjayavelu has quit IRC00:22
kevinbentonbridge MTU would also be too small00:22
imacdonnit looks at the the MTU of the VLAN interface and forces it onto the tap interface00:22
*** edmondsw has joined #openstack-nova00:23
kevinbentonah, i see00:23
*** ijw_ has joined #openstack-nova00:23
kevinbentonit's ignoring bridge mtu completely00:24
imacdonnyeah, the bridge just inherits the lowest amongst all its members00:24
*** ijw has quit IRC00:25
kevinbentonyes, so this was result in packet loss every time a vm connected00:25
imacdonnI wonder if libvirt provides any way to specify the MTU for a bridge interface00:26
imacdonnseems that that's probably where it should be done ?00:26
clarkbthis is only a problem if trying to use larger MTUs right?00:26
clarkbthats why we wouldnt see it in the gate00:26
imacdonnyeah00:27
kevinbentonwell we also wouldn't see it in single node gate00:27
kevinbentonbecause dhcp agent would get attached before VM00:27
kevinbentonand DHCP agent plugs using correct MTU00:27
clarkbkevinbenton: right but until we test with jumbo frames we would never see it regardless of multinode or single node00:27
clarkb(and we have multinode testing)00:27
imacdonnFWIW, the dhcp_agent's tap interface does get set to that of the network .. but that's obviously nothing to do with nova00:27
*** brault has joined #openstack-nova00:28
*** edmondsw has quit IRC00:28
kevinbentonso yeah, there are two possible fixes i can see00:28
kevinbentontell libvirt about MTU if it supports that00:28
*** gcb has quit IRC00:29
imacdonnyeah, if00:29
kevinbentonor proceed with https://review.openstack.org/#/c/447150/00:29
kevinbentonwhich will stop libvirt from plugging stuff into linux bridges00:29
kevinbentonand then re-add something like https://git.openstack.org/cgit/openstack/neutron/commit/?id=d352661c56d5f03713e615b7e0c2c9c8688e013200:29
imacdonnhmm ineresting00:29
kevinbentonto set MTU before it gets added to the bridge00:29
imacdonninteresting*00:29
openstackgerritTakashi NATSUME proposed openstack/nova master: Avoid forcing translation on logging calls  https://review.openstack.org/41387600:29
*** ngupta has joined #openstack-nova00:30
*** Apoorva_ has joined #openstack-nova00:30
*** gcb has joined #openstack-nova00:31
imacdonninteresting hackaround ... http://linuxaleph.blogspot.com/2013/01/how-to-network-jumbo-frames-to-kvm-guest.html00:31
kevinbentonimacdonn: can you check if libvirt can take MTU as an option?00:33
imacdonnGoogling on that now00:33
kevinbentonbecause the problem with my patch is that it requires a permissions change due to qemu sillyness00:34
kevinbentonit requires more permissions to not create a bridge than to create one :)00:34
imacdonnheh00:34
*** Apoorva has quit IRC00:34
clarkbkevinbenton: well qemu-bridge-helper is a thing, maybe you just make that nop?00:34
clarkb(thought that might confuse people greatly)00:34
*** Apoorva_ has quit IRC00:35
kevinbentonclarkb: yeah, that also makes detection harder because i wouldn't be using a different vif_type on the neutron side00:35
kevinbentonit would be hard to distinguish in the agent if a given nova port was going to try to plug itself into a bridge00:36
clarkbgotcha00:36
clarkb(you only need to attach to the bridge if on a network connected to something?)00:37
*** thorst has joined #openstack-nova00:37
imacdonnhttps://libvirt.org/formatnetwork.html <- trying to comprehend this .. I think it's saying that it only works for libvirt-managed networks00:37
imacdonn"Network types not specifically mentioned here don't support having an MTU set in the libvirt network config."00:38
kevinbentonclarkb: we attach ports to bridges that were created by agents. we leave attaching to bridge up to nova for compute ports00:38
kevinbentonclarkb: so we skip based on device_owner 'compute' right now00:39
kevinbentonclarkb: ideally i want to get rid of that, but for one cycle i will need to know about old compute ports trying to plug themselves in00:39
kevinbentonclarkb: so with my current approach we just set the vif_type='tap' for any newly bound linux bridge ports00:40
kevinbentonclarkb: and that will make it easy to tell00:40
clarkband is there a reason you can't set the mtu on the bridge when it is created (because you don't know what the mtu of the devices that attach to it later will be?)00:41
kevinbentonclarkb: we could00:42
imacdonnlinux doesn't seem to allow that .. at least the kernel I'm running00:42
kevinbentonclarkb: but that leads to the other bug00:42
*** thorst has quit IRC00:42
kevinbentonclarkb: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/139906400:42
openstackLaunchpad bug 1399064 in linux (Ubuntu) "bridges cannot have a mtu > 1500 by themselves" [Medium,Triaged]00:42
clarkbah00:42
clarkbwe set them lower in the multinode tests when we used linux bridges and not ovs00:42
clarkbgood to know thanks00:42
*** karimb has quit IRC00:43
kevinbentonimacdonn: yeah, that doesn't sound promising for libvirt00:45
imacdonnack00:45
*** Jack_Iv has joined #openstack-nova00:45
kevinbentonimacdonn: i suppose we can put the logic back in the neutron agent00:46
kevinbentonthe tap device inherits from the bridge, right?00:46
imacdonnit appears so, yes00:46
imacdonnwell...00:47
kevinbentonso once the bridge is embiggened, we don't reduce by adding a tap device00:47
imacdonnit picks up the bridge's MTU when it gets added to the bridge00:47
kevinbenton:)00:47
imacdonnif you change the bridge later, I don't think the tap will follow00:47
kevinbentonyeah, that's fine00:47
kevinbentoni'm just trying to make sure the bridge wouldn't shrink every time00:47
imacdonnright00:47
kevinbentonuntil the agent went through and corrected00:47
kevinbentonok, then we can fix this in neutron00:47
kevinbentonimacdonn: if you file a bug i can revert that patch00:48
kevinbentonwell not revert00:48
kevinbentonbut put in logic to match to the network MTU00:48
imacdonnhang on00:48
imacdonnI might have been looking at the wrong libvirt doc00:49
imacdonnhttps://libvirt.org/formatdomain.html#mtu00:49
imacdonnthis is fairly new .. I'm on libvirt 1.2.x00:49
*** jerrygb has joined #openstack-nova00:50
*** Jack_Iv has quit IRC00:50
imacdonnwell, a mix of 1.2.x and 2.0.x, but not 3.100:50
kevinbentoni gtg for a bit. if it's not looking good for libvirt, just let me know00:52
kevinbentonthe logic should probably exist in linuxbridge agent anyway to deal with older ones00:52
imacdonnI'll file the bug anyway .. think we at least need something to get us by until your other solution00:52
*** catintheroof has quit IRC00:52
imacdonnyeah, I think that requiring libvirt 3.1.x would be a problem for many00:53
imacdonnthanks! I'll get the bug in00:53
*** tovin07_ has joined #openstack-nova00:54
*** jerrygb has quit IRC00:56
*** baoli has joined #openstack-nova00:56
*** phuongnh has joined #openstack-nova01:02
*** gyee has quit IRC01:03
*** cNilesh has joined #openstack-nova01:05
*** MasterOfBugs has quit IRC01:07
imacdonnhttps://bugs.launchpad.net/neutron/+bug/168432601:08
openstackLaunchpad bug 1684326 in neutron "MTU not set on nova instance's vif_type=bridge tap interface" [Undecided,New]01:08
*** thorst has joined #openstack-nova01:13
*** NikhilS has joined #openstack-nova01:14
*** jamesden_ has joined #openstack-nova01:17
openstackgerritMatt Riedemann proposed openstack/nova master: Deprecate os-hosts API  https://review.openstack.org/45650401:19
*** yingwei has joined #openstack-nova01:20
*** jamesden_ has quit IRC01:20
*** mriedem has quit IRC01:22
*** jamesden_ has joined #openstack-nova01:23
*** Sukhdev has quit IRC01:25
*** thorst has quit IRC01:25
*** MasterOfBugs has joined #openstack-nova01:25
*** stvnoyes has quit IRC01:26
*** stvnoyes has joined #openstack-nova01:26
*** rfolco has joined #openstack-nova01:31
*** rfolco has quit IRC01:31
*** jamesden_ has quit IRC01:31
*** thorst has joined #openstack-nova01:33
*** lyan has joined #openstack-nova01:35
*** david-lyle has quit IRC01:35
*** hongbin has joined #openstack-nova01:36
*** thorst has quit IRC01:36
*** yongjiexu has joined #openstack-nova01:39
*** jerrygb has joined #openstack-nova01:43
*** yingwei has quit IRC01:46
*** stvnoyes has quit IRC01:46
*** stvnoyes has joined #openstack-nova01:47
*** zz_dimtruck is now known as dimtruck01:48
openstackgerritZhenyu Zheng proposed openstack/nova master: Support tag instances when boot  https://review.openstack.org/39432101:49
*** david-lyle has joined #openstack-nova01:50
*** dave-mcc_ has joined #openstack-nova01:51
*** annegentle has quit IRC01:51
*** kevinz has joined #openstack-nova01:52
*** dave-mccowan has quit IRC01:53
*** baoli has quit IRC01:55
*** baoli has joined #openstack-nova01:56
*** amotoki has joined #openstack-nova02:02
*** jaypipes has quit IRC02:05
*** thorst has joined #openstack-nova02:07
*** dave-mccowan has joined #openstack-nova02:10
*** zhurong has joined #openstack-nova02:10
*** yamahata has quit IRC02:11
openstackgerritzhao mingjun proposed openstack/nova master: follow-up  https://review.openstack.org/45829802:11
*** Shunli has joined #openstack-nova02:11
*** bkopilov has quit IRC02:13
*** dave-mcc_ has quit IRC02:13
*** korean101 has quit IRC02:17
*** tbachman has quit IRC02:22
*** thorst has quit IRC02:23
*** ssurana has quit IRC02:30
*** lyan has quit IRC02:31
*** coreywright has quit IRC02:37
*** ngupta has quit IRC02:38
*** ngupta has joined #openstack-nova02:38
*** gouthamr has joined #openstack-nova02:45
*** coreywright has joined #openstack-nova02:50
*** thorst has joined #openstack-nova02:54
*** thorst has quit IRC02:54
*** ngupta has quit IRC02:54
*** jerrygb has quit IRC02:55
*** gjayavelu has joined #openstack-nova02:59
openstackgerritHuan Xie proposed openstack/nova master: WIP: XenAPI use os-xenapi v2 in nova  https://review.openstack.org/45349302:59
*** yongjiexu has quit IRC03:03
*** gongysh has joined #openstack-nova03:03
*** yingwei has joined #openstack-nova03:03
openstackgerritHuan Xie proposed openstack/nova master: WIP: XenAPI use os-xenapi v2 in nova  https://review.openstack.org/45349303:06
*** Fdaisuke_ has joined #openstack-nova03:08
*** Fdaisuke has quit IRC03:09
openstackgerritZhenyu Zheng proposed openstack/nova master: Support tag instances when boot  https://review.openstack.org/39432103:18
*** zhurong has quit IRC03:21
*** thorst has joined #openstack-nova03:26
*** zhurong has joined #openstack-nova03:28
*** imacdonn has quit IRC03:28
*** imacdonn has joined #openstack-nova03:28
*** Matias has quit IRC03:30
*** zhurong has quit IRC03:31
*** nicolasbock has quit IRC03:31
*** Matias has joined #openstack-nova03:32
*** Sukhdev has joined #openstack-nova03:33
*** gjayavelu has quit IRC03:38
*** Sukhdev has quit IRC03:40
*** bkopilov has joined #openstack-nova03:44
*** thorst has quit IRC03:44
*** vladikr has quit IRC03:47
*** armax has quit IRC03:51
*** fandi has joined #openstack-nova03:55
*** ngupta has joined #openstack-nova03:55
*** fandi has quit IRC03:56
*** fandi has joined #openstack-nova03:57
*** fandi has quit IRC03:59
*** edmondsw has joined #openstack-nova04:00
*** fandi has joined #openstack-nova04:00
*** fandi has quit IRC04:01
*** Shashi has joined #openstack-nova04:02
ShashiHello Team,04:02
openstackgerritZhenyu Zheng proposed openstack/nova master: Support tag instances when boot  https://review.openstack.org/39432104:02
ShashiGetting AttributeError: 'module' object has no attribute 'VIR_MIGRATE_POSTCOPY' error and n-cpu services are not starting up04:03
*** fandi has joined #openstack-nova04:03
*** Fdaisuke has joined #openstack-nova04:03
*** edmondsw has quit IRC04:04
*** baoli has quit IRC04:05
*** Fdaisuke_ has quit IRC04:06
*** dave-mccowan has quit IRC04:07
*** baoli has joined #openstack-nova04:07
*** ratailor has joined #openstack-nova04:11
*** trinaths has joined #openstack-nova04:12
imacdonnShashi, if this is a deployment issue, better to ask in #openstack ... but also see https://bugs.launchpad.net/openstack-devel/+bug/168269204:12
openstackLaunchpad bug 1682692 in OpenStack Development Infrastructure "nova fails with error: 'module' object has no attribute 'VIR_MIGRATE_POSTCOPY' after using libvirt 2.5.0 from UCA" [Undecided,New]04:12
*** neilsun has joined #openstack-nova04:14
ShashiThanks imacdonn04:15
*** jerrygb has joined #openstack-nova04:15
*** fragatin_ has joined #openstack-nova04:15
*** eantyshev has quit IRC04:15
*** salv-orlando has joined #openstack-nova04:19
*** fragatina has quit IRC04:19
*** fragatin_ has quit IRC04:20
*** MasterOfBugs has quit IRC04:20
*** salv-orlando has quit IRC04:23
*** adisky_ has joined #openstack-nova04:28
*** baoli has quit IRC04:31
*** baoli has joined #openstack-nova04:31
*** zhurong has joined #openstack-nova04:31
*** hongbin has quit IRC04:32
*** jerrygb has quit IRC04:32
*** jerrygb has joined #openstack-nova04:32
*** salv-orlando has joined #openstack-nova04:33
*** trinaths has left #openstack-nova04:34
*** fragatina has joined #openstack-nova04:35
*** baoli has quit IRC04:35
*** fragatina has quit IRC04:39
*** thorst has joined #openstack-nova04:41
*** kaisers has joined #openstack-nova04:42
*** psachin has joined #openstack-nova04:44
*** ayogi has joined #openstack-nova04:44
*** vks1 has joined #openstack-nova04:45
*** Jack_Iv has joined #openstack-nova04:45
*** thorst has quit IRC04:46
*** yamahata has joined #openstack-nova04:46
*** kaisers has quit IRC04:47
*** jerrygb has quit IRC04:47
*** kaisers has joined #openstack-nova04:48
*** Jack_Iv has quit IRC04:50
*** Jack_Iv has joined #openstack-nova04:52
*** xinliang has quit IRC04:52
*** fragatina has joined #openstack-nova04:55
*** Jack_Iv has quit IRC04:56
*** Shashi has quit IRC04:56
*** mdnadeem has joined #openstack-nova04:57
*** fragatina has quit IRC04:57
*** fragatina has joined #openstack-nova04:58
*** ducnc has joined #openstack-nova05:00
*** ducnc has quit IRC05:00
*** Shashi has joined #openstack-nova05:00
*** sree has joined #openstack-nova05:04
*** moshele has joined #openstack-nova05:05
*** sj_ has joined #openstack-nova05:06
*** udesale has joined #openstack-nova05:06
sj_what's the concept of cells in nova ?05:07
sj_m not getting it05:07
sj_ny1 ?05:07
sj_??05:08
*** guchihiro has joined #openstack-nova05:12
*** fragatina has quit IRC05:17
*** coreywright has quit IRC05:17
*** dikonoor has joined #openstack-nova05:18
*** lucky__ has joined #openstack-nova05:21
*** nkorabli has joined #openstack-nova05:21
*** prateek has joined #openstack-nova05:26
*** nkorabli has quit IRC05:27
*** nkorabli has joined #openstack-nova05:28
*** Jack_Iv has joined #openstack-nova05:30
*** nkorabli has quit IRC05:33
*** coreywright has joined #openstack-nova05:35
*** ijw_ has quit IRC05:38
*** nkorabli has joined #openstack-nova05:42
*** thorst has joined #openstack-nova05:42
*** fandi has quit IRC05:45
*** thorst has quit IRC05:46
*** gcb has quit IRC05:51
openstackgerritHuan Xie proposed openstack/nova master: WIP: XenAPI use os-xenapi v2 in nova  https://review.openstack.org/45349305:51
*** lucky__ has quit IRC05:52
*** sj_ has quit IRC05:52
*** xinliang has joined #openstack-nova05:53
*** xinliang has quit IRC05:53
*** xinliang has joined #openstack-nova05:53
*** gouthamr has quit IRC05:54
*** damien_r has joined #openstack-nova05:54
*** CristinaPauna has quit IRC05:57
*** neilsun has quit IRC05:58
*** neilsun has joined #openstack-nova05:59
*** edmondsw has joined #openstack-nova06:00
*** Jack_Iv has quit IRC06:02
*** winston-d_ has joined #openstack-nova06:03
*** Oku_OS-away is now known as Oku_OS06:04
*** ijw has joined #openstack-nova06:04
*** edmondsw has quit IRC06:05
*** ijw has quit IRC06:08
*** Jack_Iv has joined #openstack-nova06:11
*** zhurong has quit IRC06:12
*** salv-orlando has quit IRC06:12
*** Jack_Iv has quit IRC06:15
*** kornicameister has joined #openstack-nova06:16
*** ijw has joined #openstack-nova06:20
*** vks1 has quit IRC06:22
*** david-lyle has quit IRC06:23
*** nkorabli has quit IRC06:23
*** nkorabli has joined #openstack-nova06:24
openstackgerritAlex Xu proposed openstack/nova master: Deprecate Multinic, floatingip action and os-virtual-interface API  https://review.openstack.org/45718106:26
*** Jack_Iv has joined #openstack-nova06:27
openstackgerritLei Zhang proposed openstack/nova master: Add sync traits command for placement  https://review.openstack.org/45012506:27
*** vks1 has joined #openstack-nova06:27
*** Jack_Iv has quit IRC06:27
*** voelzmo has joined #openstack-nova06:27
*** nkorabli has quit IRC06:28
*** rcernin has joined #openstack-nova06:28
openstackgerritHuan Xie proposed openstack/nova master: WIP: XenAPI use os-xenapi v2 in nova  https://review.openstack.org/45349306:28
openstackgerritAlex Xu proposed openstack/nova master: Use plain routes list for flavors-access endpoint instead of stevedore  https://review.openstack.org/45658906:33
openstackgerritAlex Xu proposed openstack/nova master: Use plain routes list for '/servers' endpoint instead of stevedore  https://review.openstack.org/44586406:33
openstackgerritAlex Xu proposed openstack/nova master: Use plain routes list for flavor endpoint instead of stevedore[1]  https://review.openstack.org/45654906:33
openstackgerritAlex Xu proposed openstack/nova master: Use plain routes list for '/os-aggregates' endpoint instead of stevedore  https://review.openstack.org/45682506:33
openstackgerritAlex Xu proposed openstack/nova master: Use plain routes list for flavors-extraspecs endpoint instead of stevedore  https://review.openstack.org/45658506:33
openstackgerritAlex Xu proposed openstack/nova master: Use plain routes list for '/os-keypairs' endpoint instead of stevedore  https://review.openstack.org/45647806:33
*** slaweq has joined #openstack-nova06:37
*** david-lyle has joined #openstack-nova06:37
*** neilsun has quit IRC06:38
*** andreas_s has joined #openstack-nova06:40
*** Zhaomingjun has joined #openstack-nova06:40
*** pcaruana has joined #openstack-nova06:40
*** dimtruck is now known as zz_dimtruck06:46
*** nkorabli has joined #openstack-nova06:49
*** ralonsoh has joined #openstack-nova06:51
*** tesseract has joined #openstack-nova06:56
*** damien_r has quit IRC06:57
*** david-lyle has quit IRC07:02
*** david-lyle has joined #openstack-nova07:03
*** takashin has joined #openstack-nova07:03
*** markus_z has joined #openstack-nova07:07
*** ijw has quit IRC07:07
*** ijw has joined #openstack-nova07:10
*** jaosorior_away is now known as jaosorior07:11
*** salv-orlando has joined #openstack-nova07:11
*** karimb has joined #openstack-nova07:12
*** zhurong has joined #openstack-nova07:12
*** sridharg has joined #openstack-nova07:13
*** jhesketh has quit IRC07:14
*** ijw has quit IRC07:15
*** damien_r has joined #openstack-nova07:15
*** Drankis has joined #openstack-nova07:19
*** ltomasbo|away is now known as ltomasbo07:21
*** mlakat has joined #openstack-nova07:24
*** Zhaomingjun has quit IRC07:26
*** jamielennox has quit IRC07:28
*** neilsun has joined #openstack-nova07:33
*** Drankis has quit IRC07:38
*** jamielennox has joined #openstack-nova07:42
*** jpena|off is now known as jpena07:42
*** thorst has joined #openstack-nova07:44
*** jhesketh has joined #openstack-nova07:44
*** MasterOfBugs has joined #openstack-nova07:44
*** karimb has quit IRC07:45
*** slaweq has quit IRC07:48
*** slaweq has joined #openstack-nova07:49
*** ratailor is now known as ratailor|Lunch07:49
*** Drankis has joined #openstack-nova07:51
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-nova08:00
*** thorst has quit IRC08:03
*** karimb has joined #openstack-nova08:08
*** slaweq has quit IRC08:08
*** slaweq has joined #openstack-nova08:09
*** ijw has joined #openstack-nova08:10
*** CristinaPauna has joined #openstack-nova08:11
*** ijw has quit IRC08:16
*** mpaolino has joined #openstack-nova08:19
openstackgerritZhenyu Zheng proposed openstack/nova master: Support tag instances when boot  https://review.openstack.org/39432108:21
*** efoley has joined #openstack-nova08:25
*** CristinaPauna has quit IRC08:25
*** CristinaPauna has joined #openstack-nova08:26
*** efoley_ has joined #openstack-nova08:27
*** aarefiev_afk is now known as aarefiev08:29
*** efoley has quit IRC08:31
*** lucas-afk is now known as lucasagomes08:31
*** tovin07__ has joined #openstack-nova08:35
*** ralonsoh_ has joined #openstack-nova08:36
*** tovin07_ has quit IRC08:36
*** ralonsoh_ has quit IRC08:36
*** ralonsoh_ has joined #openstack-nova08:37
*** ralonsoh has quit IRC08:39
*** ratailor|Lunch is now known as ratailor08:40
*** Fdaisuke_ has joined #openstack-nova08:40
*** guchihiro has quit IRC08:40
*** MasterOfBugs has quit IRC08:42
*** Fdaisuke has quit IRC08:42
openstackgerritLei Zhang proposed openstack/nova master: Add sync traits command for placement  https://review.openstack.org/45012508:44
*** mnestratov has joined #openstack-nova08:47
*** david-lyle has quit IRC08:47
openstackgerritStephen Finucane proposed openstack/nova master: conf: Rename '[neutron] ovs_bridge'  https://review.openstack.org/45026608:49
openstackgerritStephen Finucane proposed openstack/nova master: conf: Move 'floating_ips' opts into 'network'  https://review.openstack.org/43103908:49
openstackgerritStephen Finucane proposed openstack/nova master: conf: Deprecate 'default_floating_pool'  https://review.openstack.org/43103808:49
openstackgerritStephen Finucane proposed openstack/nova master: conf: Add neutron.default_floating_pool  https://review.openstack.org/43103708:49
openstackgerritDao Cong Tien proposed openstack/nova master: Stop using mox int unit/virt/xenapi/image/test_utils.py  https://review.openstack.org/34262608:53
*** moshele has quit IRC08:56
*** abalutoiu__ has joined #openstack-nova08:58
*** abalutoiu__ is now known as abalutoiu09:01
*** abalutoiu_ has quit IRC09:01
*** efoley__ has joined #openstack-nova09:01
*** karimb has quit IRC09:02
openstackgerritStephen Finucane proposed openstack/nova master: libvirt: Make 'get_domain' private  https://review.openstack.org/41737809:04
*** efoley_ has quit IRC09:04
*** karimb has joined #openstack-nova09:06
openstackgerritStephen Finucane proposed openstack/nova master: conf: Set default for live_migration_scheme  https://review.openstack.org/45771609:06
openstackgerritStephen Finucane proposed openstack/nova master: conf: Add three new '[libvirt] live_migration_*' options  https://review.openstack.org/45657109:06
openstackgerritStephen Finucane proposed openstack/nova master: conf: Gather 'live_migration_scheme', 'live_migration_inbound_addr'  https://review.openstack.org/45657209:06
openstackgerritStephen Finucane proposed openstack/nova master: conf: Convert 'live_migration_inbound_addr' to HostAddressOpt  https://review.openstack.org/45657309:06
*** faizy_ has joined #openstack-nova09:08
*** neilsun has quit IRC09:09
*** faizy_ has quit IRC09:09
*** faizy_ has joined #openstack-nova09:09
*** faizy_ has quit IRC09:11
*** faizy has quit IRC09:11
*** salv-orlando has quit IRC09:13
*** ijw has joined #openstack-nova09:13
*** NikhilS has quit IRC09:13
openstackgerritAndy McCrae proposed openstack/nova master: Allow CONTENT_LENGTH to be present but empty  https://review.openstack.org/45571009:15
*** moshele has joined #openstack-nova09:17
*** ijw has quit IRC09:18
*** neilsun has joined #openstack-nova09:23
*** tovin07_ has joined #openstack-nova09:25
*** zenoway has joined #openstack-nova09:27
*** tovin07__ has quit IRC09:27
*** yamahata has quit IRC09:29
*** cdent has joined #openstack-nova09:30
*** lpetrut has joined #openstack-nova09:32
*** derekh has joined #openstack-nova09:33
*** sambetts|afk is now known as sambetts09:33
*** karimb has quit IRC09:41
*** gszasz has joined #openstack-nova09:44
*** rmart04 has joined #openstack-nova09:51
*** zenoway has quit IRC09:54
*** karimb has joined #openstack-nova09:56
*** zenoway has joined #openstack-nova09:57
*** cdent_ has joined #openstack-nova09:57
*** cdent has quit IRC09:58
*** cdent_ is now known as cdent09:58
*** tovin07_ has quit IRC09:59
*** thorst has joined #openstack-nova10:00
*** nicolasbock has joined #openstack-nova10:02
*** thorst has quit IRC10:04
*** haplo37_ has quit IRC10:05
*** hferenc has quit IRC10:07
*** karimb has quit IRC10:08
*** karimb has joined #openstack-nova10:09
*** ijw has joined #openstack-nova10:14
*** haplo37_ has joined #openstack-nova10:16
*** thomasem has quit IRC10:17
*** thomasem has joined #openstack-nova10:18
*** slaweq has quit IRC10:19
*** zhurong has quit IRC10:19
*** ijw has quit IRC10:19
*** sdague has joined #openstack-nova10:21
*** sree has quit IRC10:29
kashyapjohnthetubaguy: Hi there10:30
kashyapjohnthetubaguy: You abandoned this one with: "There are some other threads on this now."10:30
kashyaphttps://review.openstack.org/#/c/377756/10:30
kashyapCare to point those threads?10:30
*** thomasem_ has joined #openstack-nova10:36
*** cdent_ has joined #openstack-nova10:38
*** kevinz has quit IRC10:39
openstackgerritHuan Xie proposed openstack/nova master: WIP: XenAPI use os-xenapi v2 in nova  https://review.openstack.org/45349310:40
*** cdent has quit IRC10:40
*** cdent_ is now known as cdent10:40
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Microversion 2.43 - Enable cold migration with target host  https://review.openstack.org/40670710:41
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Microversion 2.43 - List/Show all server migration types  https://review.openstack.org/43083910:42
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Microversion 2.43 - List/Show all server migration types  https://review.openstack.org/43083910:42
*** ociuhandu has joined #openstack-nova10:45
*** rmart04 has quit IRC10:45
*** fragatina has joined #openstack-nova10:47
*** cNilesh has quit IRC10:48
*** gongysh has quit IRC10:50
*** rmart04 has joined #openstack-nova10:50
*** karimb has quit IRC10:57
*** ralonsoh__ has joined #openstack-nova10:57
*** slaweq has joined #openstack-nova11:00
*** ralonsoh_ has quit IRC11:01
*** thorst has joined #openstack-nova11:01
*** fragatina has quit IRC11:03
*** winston-d_ has quit IRC11:03
*** fragatina has joined #openstack-nova11:03
*** karimb has joined #openstack-nova11:03
*** thorst has quit IRC11:06
*** markvoelker has quit IRC11:06
*** efoley_ has joined #openstack-nova11:06
*** markvoelker has joined #openstack-nova11:06
*** mvk has quit IRC11:06
*** bkopilov has quit IRC11:07
*** efoley__ has quit IRC11:09
*** jaosorior has quit IRC11:09
johnthetubaguykashyap: the API WG has a spec, and we have a new one, I should have linked to those11:10
kashyapjohnthetubaguy: Thanks.11:10
kashyapIt's just dizzying about where we stand with all those abandoned specs11:10
johnthetubaguykashyap: I think this is the new one: https://review.openstack.org/45479211:11
*** markvoelker has quit IRC11:11
johnthetubaguykashyap: they were all abandoned largely due to un answered -1s for several months11:11
johnthetubaguykashyap: but like we do for patches11:11
kashyapjohnthetubaguy: jaypipes: Is this also reproposed? "Standardize capabilities using Enums" -- https://review.openstack.org/#/c/309762/11:11
kashyapjohnthetubaguy: Yeah, figured as much.  Can't keep inactive specs around forever11:12
johnthetubaguykashyap: I think thats alex_xu's work around traits11:12
*** nkorabli has quit IRC11:12
*** zenoway has quit IRC11:13
johnthetubaguykashyap: yeah, exactly that11:13
alex_xukashyap: johnthetubaguy, yea that is replaced by trait11:13
alex_xukashyap: the standard trait is in https://github.com/openstack/os-traits now11:14
* kashyap reads11:14
alex_xukashyap: this is for traits api http://specs.openstack.org/openstack/nova-specs/specs/pike/approved/resource-provider-traits.html11:14
kashyapjohnthetubaguy: alex_xu: So, about instance live resize up / down, etc, was there a discussion about its status yet?11:14
kashyap(I mean at the past Summit)11:15
* kashyap should probably look at the Etherpad11:15
johnthetubaguykashyap: at the ptg I think we said still blocked behind getting agreement on capabilities11:15
kashyapjohnthetubaguy: Ah-ha11:15
*** karimb has quit IRC11:15
johnthetubaguy... but the folks who stepped up to do that just got laid off, so thats going to need another new owner11:15
*** ijw has joined #openstack-nova11:16
alex_xu:(11:16
kashyapjohnthetubaguy: Oh, from which company are the said folks, if I may ask?11:17
*** kaisers_ has joined #openstack-nova11:17
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Abort Cold Migration  https://review.openstack.org/33473211:17
*** zhurong has joined #openstack-nova11:17
johnthetubaguyrackspace/OSIC11:17
kashyapAh-ha11:17
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Specify keymap on server boot  https://review.openstack.org/44883011:17
*** zenoway has joined #openstack-nova11:18
johnthetubaguykashyap: I think matt is getting a list of all the impacted priority items that need owners now, but still not sure whats happening to everyone yet11:18
kashyapjohnthetubaguy: Yeah, understood.  Thank you11:19
*** ratailor has quit IRC11:19
*** kaisers has quit IRC11:20
*** jaosorior has joined #openstack-nova11:20
*** ijw has quit IRC11:21
*** BlackDex has quit IRC11:21
*** smatzek has joined #openstack-nova11:23
*** jpena is now known as jpena|lunch11:25
*** smatzek has quit IRC11:25
*** smatzek has joined #openstack-nova11:25
*** thorst has joined #openstack-nova11:26
*** alexpilotti has quit IRC11:26
*** trinaths has joined #openstack-nova11:26
*** alexpilotti has joined #openstack-nova11:27
*** zeroDivisible has quit IRC11:27
*** zenoway has quit IRC11:27
openstackgerritZhenyu Zheng proposed openstack/nova master: Support tag instances when boot  https://review.openstack.org/39432111:29
*** lennyb has quit IRC11:29
*** lennyb has joined #openstack-nova11:30
*** zeroDivisible has joined #openstack-nova11:31
*** alexpilotti has quit IRC11:31
*** cdent has quit IRC11:31
zioprotodansmith: hey are here ?11:32
*** openstackgerrit has quit IRC11:32
zioprotodansmith: remember when we fixed something in the nova migrations for the db upgrade from mitaka to newton ?11:32
zioprotodansmith: I am testing now the migration of my production database... and I get these output11:33
zioprotodansmith: https://pastebin.com/NU85iyqN any ideas ?11:34
zioprotoall those ERROR message dont look any good11:35
*** neilsun has quit IRC11:36
johnthetubaguyI see "not all services that access the DB directly are updated to the latest version"11:36
johnthetubaguyzioproto: not sure if that helps, but has you got everything upgrade before you do the data migrations for that release?11:37
*** djohnsto has joined #openstack-nova11:37
*** phuongnh has quit IRC11:37
zioprotowhat I am doing here11:38
zioprotoI just get a dump of the production database11:38
zioprotoand I put it on a new db, that no other software touches11:38
zioprotoI install nova on this server11:38
zioprotoand I just run the commands you see in the pastebin11:38
zioprotoit is just to test the migrations11:38
zioprotothe database is my mitaka production database11:39
zioprotoand the packages I install are ubuntu nova  Installed: 2:14.0.4-0ubuntu1.2~cloud011:39
zioprotojohnthetubaguy: makes sense to do the test in this way ?11:39
*** rfolco has joined #openstack-nova11:43
*** alexpilotti has joined #openstack-nova11:43
*** djohnsto has quit IRC11:45
*** abalutoiu has quit IRC11:46
*** Shunli has quit IRC11:46
*** alexpilotti has quit IRC11:47
*** jerrygb has joined #openstack-nova11:49
*** jerrygb has quit IRC11:54
*** openstackgerrit has joined #openstack-nova11:57
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix unnecessary code block in a release note  https://review.openstack.org/45847211:57
johnthetubaguyzioproto: sorry, just heading for my lunch, I think you need to update the service records to pretend they are running newton, before you can run the newton data migrations11:57
*** pchavva has joined #openstack-nova11:59
*** zenoway has joined #openstack-nova11:59
*** lyan has joined #openstack-nova12:00
*** alexpilotti has joined #openstack-nova12:00
*** jpena|lunch is now known as jpena12:01
*** Jack_Iv has joined #openstack-nova12:02
*** Jack_Iv has quit IRC12:02
*** neilsun has joined #openstack-nova12:04
*** alexpilotti has quit IRC12:04
*** alexpilotti has joined #openstack-nova12:05
zioprotojohnthetubaguy: thanks I will try that12:07
*** kevinz has joined #openstack-nova12:08
*** alexpilotti has quit IRC12:09
*** karimb has joined #openstack-nova12:09
*** jerrygb has joined #openstack-nova12:09
*** mpaolino_ has joined #openstack-nova12:10
*** vks1 has quit IRC12:12
*** edmondsw has joined #openstack-nova12:12
*** zenoway has quit IRC12:13
*** mpaolino has quit IRC12:13
*** jerrygb has quit IRC12:14
*** mvk has joined #openstack-nova12:15
*** zenoway has joined #openstack-nova12:16
*** ijw has joined #openstack-nova12:17
*** kevinz has quit IRC12:18
*** kevinz has joined #openstack-nova12:19
*** sree has joined #openstack-nova12:22
*** ijw has quit IRC12:22
*** fragatina has quit IRC12:22
*** jaypipes has joined #openstack-nova12:23
*** fragatina has joined #openstack-nova12:23
*** abalutoiu has joined #openstack-nova12:24
zioprotojohnthetubaguy: I had to fix the thing with 'delete from services where deleted_at is not NULL;'12:25
zioprotolooks like if you have in your database services flagged as deleted this can break the migration12:25
*** dave-mccowan has joined #openstack-nova12:27
*** catintheroof has joined #openstack-nova12:27
*** markus_z has quit IRC12:28
*** catintheroof has quit IRC12:30
*** catintheroof has joined #openstack-nova12:30
johnthetubaguyzioproto: ah, that rings a bell, I think dansmith has patch that already https://github.com/openstack/nova/commit/2e05c82b807624b8420df4c94d68b75563b230d2 I guess its not in the package12:32
johnthetubaguyactually, yeah, looks like thats not in a tagged release quite yet12:32
asettlenicolasbock: Hey if you're around, I need a nova docs CPL ;) https://bugs.launchpad.net/openstack-manuals/+bug/1683104 and https://bugs.launchpad.net/openstack-manuals/+bug/168307112:33
openstackLaunchpad bug 1683104 in openstack-manuals "instance is not created rightly" [Undecided,New]12:33
openstackLaunchpad bug 1683071 in openstack-manuals "error installing nova-common" [Undecided,New]12:33
asettleI just need someone to help verify :(12:33
zioprotoYes but this commit is in Mitaka12:33
zioprotoI am confused12:33
nicolasbockasettle, ok, let me have a look12:33
asettleThanks :)12:33
*** mdrabe has joined #openstack-nova12:34
*** salv-orlando has joined #openstack-nova12:35
*** hshiina has joined #openstack-nova12:35
zioprotojohnthetubaguy: maybe you are right, this https://review.openstack.org/#/c/438630/ is only in 14.0.512:36
zioprotoand I have 14.0.412:36
*** Shashi has quit IRC12:37
*** annegentle has joined #openstack-nova12:37
*** markvoelker has joined #openstack-nova12:39
*** catintheroof has quit IRC12:40
*** catintheroof has joined #openstack-nova12:41
*** catintheroof has quit IRC12:41
*** catintheroof has joined #openstack-nova12:41
*** gcb has joined #openstack-nova12:43
*** catintheroof has quit IRC12:43
*** catintheroof has joined #openstack-nova12:43
*** liverpooler has joined #openstack-nova12:46
*** vladikr has joined #openstack-nova12:49
*** yingwei has quit IRC12:50
*** bkopilov has joined #openstack-nova12:50
*** eharney has joined #openstack-nova12:51
*** zenoway has quit IRC12:51
*** zenoway has joined #openstack-nova12:52
*** jerrygb has joined #openstack-nova12:56
*** mpaolino_ has quit IRC12:56
*** djohnsto has joined #openstack-nova12:57
*** sree has quit IRC12:57
*** annegentle has quit IRC12:57
*** sree has joined #openstack-nova12:58
*** ababich has quit IRC12:59
*** alexpilotti has joined #openstack-nova13:01
*** slaweq has quit IRC13:03
*** cleong has joined #openstack-nova13:03
*** zenoway has quit IRC13:04
*** zenoway has joined #openstack-nova13:04
*** mpaolino_ has joined #openstack-nova13:13
*** sree has quit IRC13:14
*** sree has joined #openstack-nova13:15
*** ijw has joined #openstack-nova13:18
*** alexpilo_ has joined #openstack-nova13:19
*** sree has quit IRC13:19
*** cdent has joined #openstack-nova13:20
*** mriedem has joined #openstack-nova13:20
*** CristinaPauna has quit IRC13:21
efriedmriedem Good morning.  When you have a sec - I noticed in your "Reflections" email the term "shared storage pools" - was wondering what that means?13:21
*** claudiub has quit IRC13:21
*** slaweq has joined #openstack-nova13:22
*** slaweq has quit IRC13:22
*** abalutoiu has quit IRC13:22
*** alexpilotti has quit IRC13:22
*** burt has joined #openstack-nova13:22
*** erhudy has joined #openstack-nova13:23
mriedema pool, of shared storage13:23
mriedemefried: https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/generic-resource-pools.html13:23
*** alexpilotti has joined #openstack-nova13:24
*** ijw has quit IRC13:24
*** traskat has joined #openstack-nova13:24
*** hypothermic_cat is now known as ildikov13:25
*** ociuhandu has quit IRC13:25
efriedmriedem Okay, thanks.  PowerVM has Shared Storage Pools (capitals intentional) which is a very specific technology based on a cluster of VIOSes etc.  Just wanting to anticipate future confusion that will be caused by e.g. https://review.openstack.org/44318913:25
mriedemit's just a file share isn't it?13:26
dansmithzioproto: yep, what johnthetubaguy said.. there are some data migrations that won't run without the services having been updated yet13:26
dansmithzioproto: specifically one, that PciDevice one13:26
*** alexpilo_ has quit IRC13:26
dansmithzioproto: if you don't use any pci devices in your deployment then it won't do anything when it runs anyway13:26
johnthetubaguydansmith: it sounds like he was missing your fix up for the delete services thingy13:27
*** lpetrut has quit IRC13:27
johnthetubaguyold packages13:27
johnthetubaguywell... older13:27
traskathey, what i have to do apply  to apply extra specs to running VMs for examaple disk IO limitations?13:27
dansmithjohnthetubaguy: ah maybe13:27
sfinucanmriedem: Could you take a look at these today? https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bump-min-libvirt13:27
mriedemsfinucan: i can13:27
sfinucanmriedem: Cheers13:28
efriedmriedem PowerVM SSP?  No.  The VIOSes share a pool of SAN disks and carve out LUs, which can then be attached to VMs.  Things like thin clones are supported.  Aforementioned patch allows multiple separate hosts (which otherwise don't communicate with each other) to share a SSP, including sharing images.13:28
efriedThe VIOSes use an AIX-specific cluster technology, and an AIX-specific shared file system type.13:29
mriedemefried: at a high level it's the same concept for modeling it in the placement service13:29
mriedemit's a shared storage resource provider,13:29
mriedemwhich will be in an aggregate relationship with the compute node resource providers that it serves disk to13:29
*** trinaths has left #openstack-nova13:30
efriedmriedem Cool, maybe we ( thorst ) can look into folding PowerVM SSP into that concept.  Thanks.13:30
mriedemso rather than each compute node resource provider reporting it has 100TB of disk or whatever the pool size is, which would be a lie if you have 10 computes for the same storage (10*100TB), you'd model it so that the 10 compute RPs share the same disk RP with 100TB13:30
mriedemefried: it's an operator thing to set it up properly13:31
efriedBut we would have to enable/support it.13:31
mriedemjaypipes or cdent or edleafe can correct me if i'm wrong13:31
* jaypipes reads back13:31
edleafemriedem: correctamundo13:32
jaypipesefried: totes. the VIOSes in AIX on PVM SSPs on SAN. yep, totes.13:33
*** ociuhandu has joined #openstack-nova13:34
openstackgerritSandeep Pawar proposed openstack/nova master: Libvirt volume driver for Veritas HyperScale  https://review.openstack.org/44395113:34
*** ig0r_ has joined #openstack-nova13:34
*** sree has joined #openstack-nova13:34
thorstefried: yeah, we should definitely fold into that concept... (was afk)13:36
*** felipemonteiro has joined #openstack-nova13:36
efriedthorst adreznec Let's queue it up for discussion in our box doc.13:36
thorstyep13:37
thorstwe may have some FS ones in the same boat.13:37
*** sree has quit IRC13:38
efriedthorst Though it looks like maybe there is no affordance for it to be automated - the deployer just has to know which hosts are sharing an SSP and run these resource-provider set inventory commands accordingly.13:39
*** smatzek has quit IRC13:39
thorstwell, we've got a command to tell them all the hosts sharing...so.  Maybe something to doc for that process.13:40
*** abalutoiu has joined #openstack-nova13:40
*** gcb has quit IRC13:40
efriedWould be kinda neat if there was a method on the compute driver that could be asked to provide some kind of identifier for the shared resource (the SSP UUID in this case?) so the controller can figure out the pool on its own.13:40
efriedmriedem edleafe jaypipes Future enhancement ^^ ?13:40
efried(though I haven't finished reading the spec yet ;-)13:41
jaypipesefried: in case it wasn't obvious, I was being sarcastic above. :)13:41
*** ociuhandu has quit IRC13:41
efriedjaypipes Oh, it wasn't.13:42
*** abalutoiu has quit IRC13:42
efriedTo me13:42
jaypipesefried: I was poking fun at your TLAs :P13:42
efriedGotcha.  IBM does love its TLAs.13:42
*** abalutoiu has joined #openstack-nova13:42
jaypipesefried: joking aside, I don't see anything that would prevent SSPs from being used as shared resource providers, no.13:42
openstackgerritRoman Podoliaka proposed openstack/osc-placement master: CLI for resource providers  https://review.openstack.org/45753213:43
openstackgerritRoman Podoliaka proposed openstack/osc-placement master: CLI for inventories  https://review.openstack.org/45753313:43
openstackgerritRoman Podoliaka proposed openstack/osc-placement master: CLI for allocations  https://review.openstack.org/45753413:43
openstackgerritRoman Podoliaka proposed openstack/osc-placement master: CLI for usages  https://review.openstack.org/45753513:43
efriedjaypipes Agree.  But seems like it should be possible without operator intervention - at least in the narrow case I'm thinking about.13:43
jaypipesefried: yes, indeed.13:44
efriedNot sure if possible in the general case.13:44
*** jamesden_ has joined #openstack-nova13:44
efriedI would think it should be possible if the shared storage was via NFS.13:44
efriedAs long as the computes had a reliable way to canonicalize the host and mountpoint.13:45
rpodolyakacdent: thanks for your input on osc-placement changes! it's good to have a pair of fresh eyes on this13:45
thorstcould do the old - have each compute create a file on the share...and then have the compute drivers report back what they see.  From that you could figure out who can see who...13:46
openstackgerritLee Yarwood proposed openstack/nova master: encryptors: Switch to os-brick encryptor classes  https://review.openstack.org/39159713:46
*** david-lyle has joined #openstack-nova13:46
cdentrpodolyaka: you're very welcome13:47
lyarwoodmriedem: would you be willing to approve the bp for ^ https://blueprints.launchpad.net/nova/+spec/switch-to-os-brick-encryptor-classes or should I bring it up during the meeting today?13:48
*** abalutoiu has quit IRC13:49
*** ekuris has quit IRC13:50
mriedemlyarwood: bring it up during open discussion13:51
lyarwoodmriedem: ack thanks13:51
*** mlavalle has joined #openstack-nova13:52
cdentjohnthetubaguy: I've seen your comments on the service stuff on https://review.openstack.org/#/c/457283/ and I'm confused on one point that you might be able to help me on:13:53
cdentjohnthetubaguy: do we expect the host field in the service record to be the same or different for each instance of the wsgi app?13:53
johnthetubaguycdent: I think we want an entry for each version that is installed13:54
johnthetubaguycdent: so we can tell when the upgrade has finished13:54
johnthetubaguy(or not)13:54
cdentthat doesn't really answer my question :)13:54
johnthetubaguycdent: true, I think the answer is yes, but the above is why13:55
johnthetubaguyoops, by yes, I mean different for each instance13:55
johnthetubaguythe problem is cleaning up old ones, etc, its a bit... unexpected13:55
cdentso many knobs13:56
cdentokay, I'll make some adjustments. using CONF.host seems like a reasonable way to go13:56
*** diegows has joined #openstack-nova13:56
johnthetubaguycdent: I think this is the extra context: https://review.openstack.org/#/c/384724/ and https://launchpad.net/bugs/163143013:56
openstackLaunchpad bug 1631430 in OpenStack Compute (nova) "Only one entry created for nova-osapi_compute and nova-metadata services in multi-n-api environment" [Medium,In progress] - Assigned to Sujitha (sujitha-neti)13:56
mriedemnova meeting in 3 minutes in #openstack-meeting13:57
cdentthanks johnthetubaguy13:57
openstackgerritsahid proposed openstack/nova master: network: add command to configure trusted mode for VFs  https://review.openstack.org/45851313:58
openstackgerritsahid proposed openstack/nova master: libvirt: configure trust mode for vfs  https://review.openstack.org/45851413:58
mdboothsfinucan johnthetubaguy mriedem: I'd really like to push this one over the line: https://review.openstack.org/#/c/383859/ . Appreciate it's a pretty complex, though. I was thinking about splitting out the code which resets on libvirt reconnection into a separate patch. I still think this code adds robustness, but splitting it out might make it simpler to review the most important part of the patch. Worth it?13:58
*** smatzek has joined #openstack-nova13:59
mriedemi wouldn't split it13:59
sfinucanI wouldn't13:59
mdboothGlad I asked :)13:59
sfinucanYeah, I'm generally happy with it as is. The tests are by far the most complex piece but copious comments do help14:00
johnthetubaguyyeah, I think I am quite close to +2 on that now, its dropped off my radar for a bit14:00
mdboothYeah, the tests are a nightmare14:00
johnthetubaguywill try taking another peak today14:00
*** shaner has quit IRC14:00
mdboothOk, thanks. I'll update with sfinucan's comments, but otherwise leave it alone.14:01
*** eharney has quit IRC14:01
sfinucan(y)14:02
*** ayogi has quit IRC14:03
*** esberglu has joined #openstack-nova14:05
*** awaugama has joined #openstack-nova14:05
*** CristinaPauna has joined #openstack-nova14:09
*** tbachman has joined #openstack-nova14:12
*** eharney has joined #openstack-nova14:14
*** david-lyle has quit IRC14:14
*** ngupta has quit IRC14:17
*** ngupta has joined #openstack-nova14:17
*** zz_dimtruck is now known as dimtruck14:18
*** gouthamr has joined #openstack-nova14:19
*** Jeffrey4l has quit IRC14:19
*** ijw has joined #openstack-nova14:20
traskathey, what i have to do  to apply extra specs to running VMs for examaple disk IO limitations without restart the VM?14:21
*** slaweq has joined #openstack-nova14:22
*** claudiub has joined #openstack-nova14:23
*** fragatina has quit IRC14:23
*** shaner has joined #openstack-nova14:23
efriedtraskat Resize?14:25
*** alexpilotti has quit IRC14:25
efried(I don't know the answer, just an idea.)14:25
*** ijw has quit IRC14:25
*** alexpilotti has joined #openstack-nova14:26
*** lucasxu has joined #openstack-nova14:26
*** dikonoor has quit IRC14:26
*** slaweq has quit IRC14:27
traskatefried, you mean to another flavor ?14:29
efriedtraskat I would think so.  Where else would you get the new extra_specs?14:29
efriedtraskat Prolly depends on the driver whether the resize can happen without a reboot...14:30
efriedBut then, the extra_specs will be driver-specific anyway, não é?14:30
*** abalutoiu has joined #openstack-nova14:32
*** kfarr has joined #openstack-nova14:34
openstackgerritMathieu Gagné proposed openstack/nova master: Add ability to signal and perform online volume size change  https://review.openstack.org/45432214:39
imacdonnmdbooth, around ?14:40
mdboothimacdonn: Yep14:40
mriedemtakashin: alex_xu: ok did you want to talk about the server migrations API?14:41
imacdonnhi! Could you take a peek at https://bugs.launchpad.net/nova/+bug/1683972 ?14:41
openstackLaunchpad bug 1662483 in OpenStack Compute (nova) "duplicate for #1683972 detach_volume races with delete" [Undecided,In progress] - Assigned to Matthew Booth (mbooth-9)14:41
mriedemor wait for me to write something up?14:41
alex_xumriedem: yea14:41
takashinmriedem: yes.14:41
alex_xuI just didn't remember why we only show in-progress initially14:41
*** kevinz has quit IRC14:41
mriedemalex_xu: because the api was written solely to cancel in-progress live migratoins14:41
mriedem*migrations14:41
mdboothimacdonn: Looking14:41
imacdonnmdbooth, thanks14:41
takashinalex_xu: There is os-migrations API.14:42
*** psachin has quit IRC14:42
takashinalex_xu: We can use it to list all migrations.14:42
mriedemalex_xu: so rather than just having to validate when a request is made to cancel a live migration, we filtered out the migrations you could pick from to do the cancel14:42
*** dgonzalez_ has joined #openstack-nova14:43
*** mkoderer_ has joined #openstack-nova14:43
takashinmriedem: Yes. To cancel a live migration.14:43
mriedemhttps://github.com/openstack/nova/blob/master/nova/api/openstack/compute/migrations.py#L6214:44
mriedemso before 2.23, we can list all migrations, but we don't expose the type14:44
mdboothimacdonn: Fun14:44
mriedembecause add_link defaults to False14:44
alex_xumriedem: that works for me. I guess initialy we think there is only one migration, and we didn't want to trouble with user to find out which migration is in-progress.14:44
imacdonnmdbooth :) indeed14:44
*** dgonzalez_ has quit IRC14:45
*** mkoderer_ has quit IRC14:45
alex_xuwith filter, we can reach the same goal14:45
mriedemat microversion >= 2.23 with os-migrations, we provide a link and migration_type in the response for an in-progress live migration, otherwise just the migration_type14:45
mriedemalex_xu: note that os-migrations also takes a filter parameter14:45
mriedemhttps://github.com/openstack/nova/blob/master/nova/api/openstack/compute/migrations.py#L7714:45
alex_xuso i'm ok with that. and the user have one more way to query all the migraitons for a specific servers, so nothing bad at here14:45
mriedemit's not validated, but it's there14:45
openstackgerritDan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack  https://review.openstack.org/45853714:46
mdboothimacdonn: Let me think about that for a bit. lyarwood may also be interested in https://bugs.launchpad.net/nova/+bug/168397214:46
openstackLaunchpad bug 1662483 in OpenStack Compute (nova) "duplicate for #1683972 detach_volume races with delete" [Undecided,In progress] - Assigned to Matthew Booth (mbooth-9)14:46
alex_xutakashin: what would you think?14:47
mriedemalex_xu: this is how os-migrations gets the migrations from the db + filters https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L478914:47
imacdonnmdbooth, OK, will check that one out too. I think my problem is a bit more extensive than 1683972, but maybe we can find a solution that covers both14:47
mdboothimacdonn: It's the same bug, just repeating for lyarwood14:48
*** marst has quit IRC14:48
mdboothI just unlinked the bug it was marked as a dup of, btw14:48
mriedemalex_xu: so technically you could get all migration types via os-migrations v2.1 and filter on instance_uuid14:48
imacdonnmdbooth, oh, duh .. still working on my first coffee :)14:48
traskatefried, are there other options then  changing the flavor. i tried now a reboot but that didn't work either. the flavor i use for the rebooted instance has the extra specs before.14:48
takashinalex_xu: So I would like to make new APIs to show in-progress migrations only.14:48
mdboothI don't think they're the same, although they're cousins14:48
imacdonnmdbooth, agreed14:48
zioprotodansmith: I upgraded to 14.0.5 but there are still some strange different messages https://pastebin.com/vLNWdiRZ14:49
mriedemtakashin: why would you rather filter all migrations and return only in-progress migrations without any option for the user to choose the filtering?14:49
zioprotolooks like a problem with the keypairs14:50
efriedtraskat Sorry, I'm not familiar enough with this to be of help.  The resize was just an off-the-cuff idea.  But you probably want to take this discussion to the #openstack channel (see channel topic).14:50
traskatok thanks14:50
mriedemtakashin: that seems inconsistent behavior with the rest of our apis, which if you list resources, you get undeleted resources in their current state,14:50
mriedemyou can take actions on those resources, and those actions may be invalid for the state that the resource is in,14:51
mriedemin which case, the api will tell you that with a 400 response14:51
dansmithzioproto: hmm14:51
takashinmriedem: Because we have os-migrations API and there should be a difference between new APIs and old one.14:51
lyarwoodmdbooth: thanks, I missed that you had a change up to add locking to detach_volume in the compute layer somehow14:51
zioprotodansmith: Some instances are still missing keypair information. Unable to run keypair migration at this time.14:51
dansmithzioproto: yeah, but the problem is the "error attempting to run ..." one14:52
dansmithzioproto: just a sec14:52
mdboothlyarwood: Not related to this issue. I should probably abandon that patch as it's more complex than I was going for.14:52
mriedemtakashin: so if i understand your point, if i want to list all migrations, regardless of type or status, i should use os-migrations14:52
*** CristinaPauna has quit IRC14:52
alex_xutakashin: /servers/{id}/migrations is more easy for user to query migration for specific server14:52
mriedemtakashin: but if i want to list migrations for a specific server in order to cancel the migration, i should use /servers/server_id/migrations14:52
zioprotodansmith: I wait no problem :)14:52
*** tbachman has quit IRC14:53
takashinmriedem: alex_xu: Yes. In specific state.14:53
mriedemtakashin: the thing that worries me about saying just use os-migrations, is the docs have a warning on it https://developer.openstack.org/api-ref/compute/#migrations-os-migrations-frozen14:53
alex_xutakashin: From the ML, operators have use-case to query specific server's migrations. so14:53
mriedembasically saying, don't use this, use the other API14:53
lyarwoodmdbooth: is it not? wouldn't locking against instance.uuid for both attach and detach help here?14:54
dansmithzioproto: well, I was looking to see if we had any updates to that migration but I don't think we have14:54
*** bkopilov has quit IRC14:54
mriedemalex_xu: argubally they could, i.e. GET /os-migrations?instance_uuid=123414:54
dansmithzioproto: you can make a code tweak to dump more info since this is a test right?14:54
mdboothlyarwood: Don't think so. This is racing with cinder internals.14:54
*** prateek has quit IRC14:54
imacdonnlyarwood, mdbooth, my problem is that the iSCSI connection termination is separate from detach14:54
*** marst has joined #openstack-nova14:55
mdboothimacdonn: If it rescans again before the lun has been reallocated, does the device go away?14:55
dansmithzioproto: can you do this and see what it shows us? https://pastebin.com/3RJdsJE114:55
lyarwoodimacdonn: it shouldn't be anymore, we've moved everything into the bdm.detach() call14:55
alex_xumriedem: but more straight forward with /servers/{id}/migrations? but actually we should think about more how the API user write the client code14:55
lyarwoodimacdonn: so locking that against instance.uuid should help avoid this now unless I'm missing something14:56
mdboothlyarwood: terminate connection isn't atomic, right?14:56
mdboothlyarwood: So on completion of the call to terminate connection, the connection hasn't necessarily been terminated?14:56
mdboothatomic is the wrong word. Synchronous.14:56
lyarwoodmdbooth: yeah good point14:56
mriedemalex_xu: we actually have CLIs for these in novaclient already14:56
gibimriedem: hi! I'm sorry for missed the nova meeting, today is a pretty crazy day. Thanks for giving the subteam update.14:56
mriedemgibi: np14:57
imacdonnlyarwood, mdbooth, terminate connection is NOT part of bdm.detach, though14:57
mdboothlyarwood: That's a question, btw. Do we know for sure that is/isn't the case?14:57
mriedemalex_xu: nova migration-list [--host] [--status]14:57
*** kevinz has joined #openstack-nova14:57
mdboothimacdonn: If it was synchronous it would at least be a lever we could use to fix the problem.14:57
*** amotoki has quit IRC14:57
mriedemalex_xu: we don't allow listing migrations via os-migrations and filtering on instance uuid from the CLI14:57
mriedemwe could certainly add that14:57
mdboothIf it's async this could be messy.14:57
* mdbooth strongly suspects it's async.14:58
mriedemalex_xu: then we also have nova server-migration-list <server>14:58
mriedemwhich is using /servers/server_id/migrations14:58
mriedemfrom a CLI perspective, it's very confusing14:58
zioprotodansmith: https://pastebin.com/n0THHGSG14:58
imacdonnmdbooth, you mean like a "callback when it's done" thing ?14:58
dansmithzioproto: interesting14:58
mdboothI mean: don't return control until it's done14:58
zioprotodansmith: my production database is always interesting :(14:59
dansmithzioproto: no, this is super helpful14:59
mdboothIf nova knew that immediately after terminate connection returned the connection was definitely already be gone, we could use that14:59
imacdonnmdbooth, I believe that's how it is already ... cinderclient waits for the response to an HTTP request14:59
lyarwoodimacdonn: sorry, I'm talking about the driver bdm - https://github.com/openstack/nova/blob/master/nova/virt/block_device.py#L294-L35214:59
*** adisky_ has quit IRC14:59
mriedemalex_xu: takashin: so maybe we just leave /servers/server_id/migrations as doing the server-side in-progress filtering as it does today, and expand the "nova migration-list" CLI to allow filtering on instance uuid too14:59
zioprotodansmith: I have upgrade date on Saturday 29th of April :) Do you think I can find a fix by then ? :)14:59
mriedemand clarify the help text in the api ref docs and CLI help text15:00
dansmithzioproto: yep15:00
mriedemalex_xu: takashin: part of the problem is we just don't document os-migrations very well in the api-ref15:00
zioprotodansmith: I have the feeling everytime I chat with you a new Nova release comes out :D15:00
dansmithzioproto: heh15:00
mdboothlyarwood kashyap: Any idea what the status of qemu iscsi multipath is?15:01
lyarwoodmdbooth: yeah looking at the cinderv2 code it looks synchronous15:01
* kashyap reads15:01
mdboothlyarwood: Got a link?15:01
zioprotoI am building the ubuntu packages for 14.0.5 ... but I will wait a 14.0.6 if something still needs to be changed for the database migrations15:01
*** aarefiev is now known as aarefiev_afk15:01
*** rcernin has quit IRC15:01
lyarwoodmdbooth: AFAIK it's in and we have some config support for it in the libvirt driver but it's not wired up at the moment15:01
dansmithzioproto: so I'm going to give you another patch to try and see if you get farther, but it's not the fix I want, so I'll ask you to test the real fix as well, okay?15:01
lyarwoodmdbooth: https://github.com/openstack/cinder/blob/master/cinder/api/contrib/volume_actions.py#L192-L20815:01
mriedemlyarwood: mdbooth: terminate_connection in cinder is an RPC call to the cinder-volume backend15:01
zioprotodansmith: sure !15:01
mdboothlyarwood: Sweet, because the real solution here is to stop doing this and use qemu.15:02
mriedemlyarwood: that's not what you want15:02
alex_xumriedem: at least that match the requirement from the operator15:02
mriedemlyarwood: https://github.com/openstack/cinder/blob/master/cinder/volume/api.py#L74915:02
*** kevinz has quit IRC15:02
mriedemhttps://github.com/openstack/cinder/blob/master/cinder/volume/rpcapi.py#L21715:02
mriedemalex_xu: ok so i guess i'll just go with that then, and unblock takashin15:03
*** gongysh has joined #openstack-nova15:03
alex_xumriedem: thanks15:03
mriedemi can fix the os-migrations api-ref docs (i've been meaning to do that for awhile now),15:03
mdboothmriedem: It's a call, which is nice.15:03
lyarwoodmriedem: yeah I was just getting there, thanks for the links :)15:03
takashinmriedem: alex_xu: Thank you.15:03
mriedemand add --server as a filter option to nova migration-list15:03
mriedemmdbooth: well normally a block rpc call from the api is not nice15:03
mriedem*blocking15:03
dansmithzioproto: https://pastebin.com/yHueQi4s15:04
*** mlakat has quit IRC15:05
alex_xumriedem: I can help you on those15:05
mdboothmriedem: Nice for me :)15:05
mdboothBut yeah15:05
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Microversion 2.42 - Fix tag attribute disappearing  https://review.openstack.org/42951215:05
mriedemsdague: another one if you have time https://review.openstack.org/#/c/429512/ - unblocks novaclient for new microversions in pike15:05
*** moshele has quit IRC15:05
*** armax has joined #openstack-nova15:06
zioprotodansmith: should I revert the previous patch with the 'raise' ?15:06
dansmithzioproto: no15:06
dansmithzioproto: the raise lets us see if I typo'd this one :P15:07
*** yingwei has joined #openstack-nova15:07
zioprotodansmith: https://pastebin.com/eLsiFn9W15:07
*** Oku_OS is now known as Oku_OS-away15:07
kashyapmdbooth: So, no, upstream QEMU's built-in iSCSI initiator doesn't support multipathing yet15:08
kashyapmdbooth: And I haven't seen any related to it for over 2 years15:08
dansmithzioproto: hmm, that ignoring line should mean we're skipping the thing that gives us the error15:09
dansmithzioproto: I hate to ask, but did you apply with patch or paste it in by hand?15:09
mdboothkashyap: Sucks :(15:09
zioprotoI pasted by hand !15:09
dansmithzioproto: any chance you missed the 'continue' line under the log?15:10
dansmithzioproto: L35 in the paste15:10
lyarwoodkashyap / mdbooth ; AFAIK the lack of multipath support is the reason it was never wired up in Nova15:10
*** felipemonteiro__ has joined #openstack-nova15:10
zioprotoand you are right !15:10
mdboothlyarwood: Right.15:10
kashyapmdbooth: Hey, on the brighter hand, QEMU 2.9 just released, with some "robust" improvements15:10
*** felipemonteiro has quit IRC15:11
*** udesale has quit IRC15:11
zioprotodansmith: https://pastebin.com/LeH3HudT15:11
zioprotosorry for missing the 'continue'15:12
*** ayogi has joined #openstack-nova15:12
openstackgerritSTEW TY proposed openstack/nova master: Transform instance.unrescue notifications  https://review.openstack.org/38827515:12
dansmithzioproto: no problem :)15:12
dansmithzioproto: okay, so you have a bunch of instances with keypairs that have since been deleted...15:13
*** david-lyle has joined #openstack-nova15:13
mriedemefried: one new question from me inline https://review.openstack.org/#/c/438598/1215:13
zioprotoalso the instances have been deleted I guess15:13
*** takashin has left #openstack-nova15:13
efriedmriedem Ack, looking...15:13
openstackgerritSteve Noyes proposed openstack/nova master: Update detach to use V3 Cinder API  https://review.openstack.org/43875015:13
zioprotodansmith: I think we are talking about the instances that are marked as deleted15:13
*** hongbin has joined #openstack-nova15:13
zioprotodansmith: are you going to submit this patch for review in master ?15:14
dansmithzioproto: no, let me get you an example15:14
mriedemefried: same question about exception handling in power_off15:14
mriedemwe try to avoid handling Exception if possible15:14
dansmithzioproto: 2017-04-20 15:11:10.584 787 WARNING nova.objects.instance [req-d1cbe1e7-1f03-46b6-9009-bba793eb73ba - - - - -] Instance b750d102-4fbb-4401-80b2-cc36888c9749 keypair c_rally_5aabcf4c_Xg4N7es6 not found15:14
*** alexpilo_ has joined #openstack-nova15:14
*** alexpilotti has quit IRC15:14
*** pcaruana has quit IRC15:14
efriedmriedem Roger that.  Responding.15:14
dansmithzioproto: yeah, we'll fix in master and backport15:14
sdaguemriedem: so, given that powervm driver is pretty self contained, is there particular primary concerns before merging?15:15
*** nkorabli has joined #openstack-nova15:15
bauzasoh man, /me needs to get drugs in order to think about https://review.openstack.org/#/c/437424/15:15
bauzaslet's prepare a new battle15:15
zioprotodansmith: ok, please give me the review number, I will cherry-pick this patch in my deployment or I cant upgrade next week15:15
openstackgerritRoman Podoliaka proposed openstack/osc-placement master: CLI for resource providers  https://review.openstack.org/45753215:15
openstackgerritRoman Podoliaka proposed openstack/osc-placement master: CLI for inventories  https://review.openstack.org/45753315:15
openstackgerritRoman Podoliaka proposed openstack/osc-placement master: CLI for allocations  https://review.openstack.org/45753415:15
openstackgerritRoman Podoliaka proposed openstack/osc-placement master: CLI for usages  https://review.openstack.org/45753515:15
openstackgerritRoman Podoliaka proposed openstack/osc-placement master: tests: add a hook for functional testing in the gate  https://review.openstack.org/45212215:15
dansmithzioproto: yeah, but we need to get the problem fully fixed first :)15:15
mriedemsdague: i had questions in the 2nd patch15:15
zioprotoI am available to test more patches :)15:16
*** lucasxu has quit IRC15:16
zioprotodansmith: should I open a bug on LP ?15:16
dansmithzioproto: yep, working on a new patch15:16
dansmithzioproto: yeah, please do file a bug, we'll need it for the review15:17
*** lucasxu has joined #openstack-nova15:17
efriedsdague Not sure if you've been aware, but FYI we have a CI running (a subset of) tempest on the in-tree patches, and publishing to gerrit.15:17
sdagueefried: while you are changing things, I think you misunderstand LOG.exception(e) :)15:18
efriedsdague Entirely possible ;-)15:18
efriedPlease enlighten me.15:18
imacdonnmdbooth, you asked earlier if a rescan would make the device(s) go away .. if I'm interpreting your question properly, the answer is "no" ... rescan picks up new LUNs, but doesn't do anything about ones that have become unresponsive15:19
efriedThought it was LOG.error + stack trace15:19
sdagueefried: right15:19
mdboothimacdonn: Ok, thanks15:19
sdagueefried: comment left15:19
efriedsdague Thanks.15:19
*** kristian__ has joined #openstack-nova15:20
lyarwoodmdbooth / imacdonn ; https://bugzilla.redhat.com/show_bug.cgi?id=1367805 is a dup of this that I could never reproduce correctly FWIW15:20
openstackbugzilla.redhat.com bug 1367805 in openstack-nova "Previously removed iSCSI multipath volumes can be rediscovered by future disconnect_volume calls before being unmapped on the backend by os-terminate_connection." [High,Closed: insufficient_data] - Assigned to lyarwood15:20
mdboothimacdonn lyarwood: So what I think we need to achieve here is that after calling disconnect nothing is subsequently allowed to connect until the associated terminate connection has defintely resulted in the device having been killed by cinder15:21
imacdonnlyarwood, \o/ I'm not alone ;)15:21
*** ijw has joined #openstack-nova15:21
openstackgerritSTEW TY proposed openstack/nova master: Transform instance.unrescue notifications  https://review.openstack.org/38827515:22
lyarwoodmdbooth: yes and the locking in your change would do that now15:22
*** sridharg has quit IRC15:22
imacdonnmdbooth, right .. and I've sortof hacked that by putting a lock on things that cover that window .. like _shutdown_instance() in compute.manager15:22
imacdonnmdbooth, and the same lock on connect_volume() in virt.libvirt.volume.iscsi15:23
*** alexpilotti has joined #openstack-nova15:23
efriedsdague So I don't need to pass the exception itself to LOG.exception - it picks it up automatically?15:23
*** mdnadeem has quit IRC15:23
mriedemefried: yeah15:23
efriedBeautiful, thanks.15:23
mriedemLOG.exception() just takes a message if you want to say something15:23
mriedemand it dumps the current stacktrace15:23
mdboothlyarwood: Assuming we can verify the sync behaviour of terminate connection all the way down15:23
sdagueyeh, it's probably worth logging the instance there, I put a suggested string in a comment right now15:24
imacdonnmdbooth, lyarwood AFAIK, it's sync as far as nova-compute is concerned - it's making an API call to cinder ... it's an RPC call within cinder, but nova-compute doesn't know about that15:24
*** rmart04 has quit IRC15:24
zioprotodansmith: https://bugs.launchpad.net/nova/+bug/168486115:25
openstackLaunchpad bug 1684861 in OpenStack Compute (nova) "Database migrations from mitaka to newton fail due to missing keypairs" [Undecided,New]15:25
*** zenoway has quit IRC15:25
zioprotowait the title is wrong, I just realized15:25
dansmithzioproto: thanks, still working on the fix if you can hang on for a bit15:25
zioprotohttps://bugs.launchpad.net/nova/+bug/168486115:26
*** alexpilo_ has quit IRC15:26
zioprotoI updated the title15:26
imacdonnmdbooth, lyarwood If those calls were not sync, we'd have other problems .. like we'd ask cinder to connect a volume, then do the rescan "too early"15:26
zioprotoif you link the fix to the bug I will track it15:26
*** ijw has quit IRC15:26
mdboothimacdonn: I've learned not to assume these things :)15:27
imacdonnhah ;)15:27
dansmithzioproto: it looks to me like you have something that shouldn't be allowed with the foreign key constraint15:27
dansmithzioproto: have you disabled those constraints on your database by chance?15:27
zioprotohow can I check ?15:27
zioprotoit is possible something happened in this database since Icehouse15:27
zioprotois possible we messed up something in the upgrade Juno to Kilo if I remember correctly15:28
dansmithzioproto: well, can you check that you have something in instance_extra with instance_uuid=e7eb4196-994e-4968-bcf0-aa6f783bbb18, but nothing in instances.uuid for that?15:28
*** voelzmo has quit IRC15:29
dansmithzioproto: are you using a hand-rolled db archiving script or something?15:29
dansmithzioproto: something like this maybe? https://tecadmin.net/temporarily-disable-foreign-key-checks-in-mysql/15:29
*** slaweq has joined #openstack-nova15:29
*** zenoway has joined #openstack-nova15:30
*** coreywright has quit IRC15:31
zioprotodansmith: yes I have data in instance_extra but not in instances15:31
zioprotoI know what happened15:31
dansmithzioproto: that shouldn't be possible15:31
zioprotowe were upgrading to Kilo to Liberty15:31
*** Jeffrey4l has joined #openstack-nova15:31
zioprotoand when doing the prod update we had a problem with the database migration15:31
zioprotoand we had to do some fast operations on the database15:31
*** vks1 has joined #openstack-nova15:32
zioprotoit is possible that we deleted some very old instances15:32
sdagueefried: is there a reason why this goes fatal - https://review.openstack.org/#/c/391288/51/nova/virt/powervm/vm.py@40515:32
zioprotoand we disabled the foreigh checks to do that15:32
*** artom has quit IRC15:32
sdaguebecause I don't see any higher level error recovery for that exception15:32
dansmithzioproto: ah, okay that would explain it15:32
*** artom has joined #openstack-nova15:32
*** moshele has joined #openstack-nova15:33
zioprotodansmith: we probably dropped a lot of stuff from table instances to make a kilo to liberty database migration go through, while we were in maintanance window15:33
zioprotothat is why now we improved and we do these migrations before15:33
zioprotodansmith: so this means we behaved bad and we dont get a patch that fixes the problem ?? :|15:33
efriedsdague If we got there, somebody passed an extra spec with a 'powervm:' prefix, but that we don't recognize.15:33
efriedWe definitely want to fail the deploy, nah?15:34
dansmithzioproto: okay, I'm trying to decide if we should try to handle that possibility in the mainstream code or not.. because technically you broke the data model :)15:34
*** artom has quit IRC15:34
*** artom has joined #openstack-nova15:34
*** zenoway has quit IRC15:34
*** annegentle has joined #openstack-nova15:34
*** satyar has joined #openstack-nova15:35
dansmithzioproto: I'm leaning towards catching the problem, logging it, but making you do your own cleanup to fix it15:35
dansmithzioproto: which would just mean deleting the orphaned instance_extra records15:35
dansmithmriedem: you following this at all?15:35
efriedsdague I *suppose* an argument could be made for ignoring, so if we add a new spec in a later release, you can still use a flavor containing it on an older compute node...15:35
efriedthorst thoughts?15:35
*** tbachman has joined #openstack-nova15:35
*** vks1 has quit IRC15:36
thorstefried: I just logged a comment.  I think its sufficient to just log a warning saying 'Unknown' or something15:36
sdagueefried: maybe? though by throwing an attribute error, it's not super clear to me where things go into sane recovery to the user to explain to them what went wrong15:36
*** alexpilo_ has joined #openstack-nova15:36
thorstbut I agree...full stop, meh15:36
zioprotodansmith: of course all other operators that asked my how I upgraded from kilo to liberty did the same.15:36
thorstwarning tells them that something came in you didn't understand.15:36
dansmithzioproto: yeah, I'm sure you're not the only one15:36
sdagueefried: I guess the real question is who can solve the problem15:36
efriedthorst ...IF they look at the log.15:36
mriedemdansmith: not a word15:36
sdagueis it someone that can change their API request? in which case tell them over the API channel how to fix it15:37
*** moshele has quit IRC15:37
sdaguebut if only admins can fix it, then log warning seems better15:37
dansmithmriedem: so, zioproto disabled FK constraints in kilo, deleted a bunch of instances, and then re-enabled them, which orphaned some instance extra records, which now don't behave properly since they assume they have a FK to instances15:37
efriedsdague thorst I guess if they misspell a keyword, they'll not get the corresponding behavior.  Which they may not notice for... a while.  And once they do, they'll have to look at the log to figure out why that happened.15:37
dansmithmriedem: which is why their migration is failing15:38
openstackgerritBalazs Gibizer proposed openstack/nova master: add tags field to instance.update notification  https://review.openstack.org/40722815:38
dansmithmriedem: handling it is a little tricky, so I'm leaning towards catching the error, logging it and the uuid so they can manually clean the affected instance extra records, but not actually try to magically handle this,15:38
dansmithmriedem: in case it's actually for some other reason15:38
mriedemwhy were the FK constraints disabled?15:38
*** alexpilo_ has quit IRC15:39
*** alexpilotti has quit IRC15:39
dansmithmriedem: especially since the way you get here is breaking the model15:39
dansmithmriedem: dunno, they probably had stale/corrupt data and wanted to delete some broken instances, and didn't realize they should have deleted the things depending on the instance records instead of disabling FKs15:39
*** alexpilotti has joined #openstack-nova15:39
mdboothlyarwood imacdonn: So this call isn't synchronous unfortunately, or at least it's driver dependent and for zfssa iscsi it's not sychronous15:39
*** lucasxu has quit IRC15:39
mriedemdansmith: ummm15:40
mriedemit seems weird to put something into code for a thing you shouldn't have done in the first place,15:40
openstackgerritPavel Gluschak (scsnow) proposed openstack/nova master: VStorage: changed default log path  https://review.openstack.org/45855715:40
mriedemand if you did, then you (as the person that did it) patch the fix on your end15:40
dansmithmriedem: well, right, so that's why I just want to log/explain and be graceful15:40
imacdonnmdbooth lyarwood: where do you see that ?15:40
mriedemeven that is odd15:40
mdboothThe cinder api makes a synchronous rest call to c-vol, which synchronously makes a rest call to zfssa15:40
dansmithmriedem: not fix the problem, but just warn about the affected uuid(s)15:40
mriedem"someone reported they did this odd thing they shouldn't have, so we're going to handle and log it for them"15:40
mdboothHowever, the rest call explicitly isn't synchronous15:41
zioprotodansmith: I have the proof https://bugs.launchpad.net/nova/+bug/1511466/comments/915:41
openstackLaunchpad bug 1511466 in OpenStack Compute (nova) "migrate flavor data impossible" [Medium,Confirmed]15:41
mriedemdansmith: i know, i'm saying it's weird15:41
dansmithmriedem: right now they just get "error, failed, kthx"15:41
zioprotodansmith: somebody wrote on LP to do that, and I did it !15:41
mdboothThe driver waits for the return, but the return it's looking for is ACCEPTED15:41
dansmithzioproto: do you do everything random people tell you to do? nevermind, don't answer that :P15:41
*** lucasxu has joined #openstack-nova15:41
mdboothWhich doesn't indicate the requested change has happened yet, only that it has been requested15:41
mriedemdansmith: so are people doing this because of some latent bug that forced them to do it, but we have since fixed?15:41
dansmithmriedem: well, yeah, but especially if it was advice on a LP bug that multiple people followed, :/15:41
imacdonnmdbooth lyarwood OK, yeah, but doesn't cinder-api wait for the response from cinder-volume before responding to the (cinder) API request?15:42
*** eharney has quit IRC15:42
dansmithmriedem: it doesn't matter, disabling FKs isn't the right workaround for a problem anyway, it's deleting extra, then deleting instance, etc15:42
*** annegentle has quit IRC15:42
mdboothIt does, yes, but that response only indicates that zfssa has accepted the request15:42
mriedemdansmith: i agree15:42
*** lucasxu has quit IRC15:42
mdboothIt doesn't indicate that the request has been processed15:42
mriedemwhich is why it's odd to put something into code that is for "people worked around this incorrectly, but we're going to patch something in for them"15:42
dansmithmriedem: but regardless, it's out there so I at least want to log/warn and not just fail quietly with no indication about why15:42
* mdbooth expects a wide range of variation between drivers on this15:43
imacdonnmdbooth lyarwood: The request to the ZFSSA is another REST API .. it also waits for the operation to complete before returning15:43
*** lucasxu has joined #openstack-nova15:43
mriedemdansmith: where does this handle/log thing go? a migration? an object?15:43
dansmithmriedem: well, I wasn't going to be that nice, I was going to say "Instance Extra (uuid) has no Instance, unable to migrate"15:43
*** Tahvok has joined #openstack-nova15:43
mdboothimacdonn: It waits for the rest call to return an http response15:43
imacdonnmdbooth lyarwood I know this quite well, because it times out sometimes ;)15:43
dansmithmriedem: in the online migration function that nova-manage is calling15:43
mdboothBut the http resonse doesn't (according to the docs) indicate that it has completed15:43
*** annegentle has joined #openstack-nova15:43
mriedemdansmith: as you like15:44
mriedemi won't block it15:44
dansmithmriedem: well, I'll put it up and you can decide then15:44
mriedemgeddy lee told me that if i choose not to decide i've still made a choice15:44
*** coreywright has joined #openstack-nova15:44
TahvokI'm installing nova on ubuntu, and according to the guide I need to create the cell1 cell. In the command they say I need to run it with some sort of id? 109e1d4b-536a-40d0-83c6-5f121b82b65015:44
imacdonnmdbooth lyarwood empirically, I can say that it does wait for the operation to complete .... how else would the status be returned ??15:44
TahvokWhat is that id? https://docs.openstack.org/ocata/install-guide-ubuntu/nova-controller-install.html15:44
*** zenoway has joined #openstack-nova15:44
dansmithmriedem: heh15:44
mdboothimacdonn: It returns 202 ACCEPTED15:44
mriedemTahvok: which command? nova-manage cell_v2 create_cell?15:45
Tahvokmriedem: yes15:45
mriedemsu -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova15:45
*** djohnsto has quit IRC15:45
mriedemthat doesn't say anything about providing an id15:45
TahvokOh, it's the output15:45
mriedemit outputs an id15:45
mriedemyes15:45
TahvokYes, I thought it's part of the command15:45
Tahvokmriedem: thanks!15:45
imacdonnmdbooth: the ZFSSA REST API call ??15:46
mriedemTahvok: also https://docs.openstack.org/developer/nova/man/nova-manage.html#nova-cells-v2 in case you need more info on the commands15:46
lyarwoodmdbooth: sorry back, isn't this a bug within this and any other driver that doesn't terminate_connection synchronously then?15:46
mdboothimacdonn: Yeah15:46
*** eharney has joined #openstack-nova15:46
lyarwoodmdbooth: I skimmed over a few target drivers and they appeared to be sync15:46
mdboothlyarwood: Right, we're discussing zfssa, but other drivers will be different15:46
Tahvokmriedem: is that create_cell command idempotent?15:46
imacdonnmdbooth: that doesn't sound right ... looking for a debug log to verify15:47
TahvokCould I run it again, or it will try creating another cell?15:47
lyarwoodmdbooth: yeah, I think there's still value to adding the lock on our side if we also follow up with cinder to get all drivers in line there15:47
zioprotodansmith: the bug is still open https://bugs.launchpad.net/nova/+bug/1511466 SOOO I guess any operator with a big deployment that managed to upgrade to liberty probably used this workaround. This means that soon this problem will explode :D15:48
openstackLaunchpad bug 1511466 in OpenStack Compute (nova) "migrate flavor data impossible" [Medium,Confirmed]15:48
mriedemTahvok: it's not15:48
mdboothimacdonn: That's definitely what it returns, and the documented semantics of that are that it's an async call which may not have completed. That doesn't necessarily mean that's what it's doing, of course.15:48
mriedemTahvok: it would fail with a duplicate entry error15:48
dansmithzioproto: yeah, you know what you need to do to fix your data right?15:48
mriedemTahvok: "Returns 0 if the cell mapping was successfully created, 1 if the transport url or database connection was missing, and 2 if a cell is already using that transport url and database connection combination."15:48
Tahvokmriedem: that's fine as long as it does not create any new records15:48
*** andreas_s has quit IRC15:48
mriedemTahvok: $? would be 215:48
Tahvokmriedem: thanks a lot!15:48
imacdonnmdbooth: heh, OK ... I'm fairly sure there's nothing in the ZFSSA driver that gets a result of the operation by any means other than the response to the REST request15:49
*** salv-orl_ has joined #openstack-nova15:49
zioprotodansmith: so you mean that if I delete from the instance_extra table all the records where the uuid is not in instances, then I dont need any patch ?15:49
mdboothimacdonn: https://docs.oracle.com/cd/E51475_01/html/E52433/makehtml-id-181.html15:50
mdboothExcept it's doing a PUT, not a POST15:50
zioprotodansmith: I have to rush away from the office... I will ping you tomorrow about this15:50
dansmithzioproto: yeah, that's my expectation.. let me know if it doesn't and we can see if something else is wrong too15:50
dansmithzioproto: okay thanks15:50
imacdonnmdbooth: Yeah, so does "successful response" mean "I've completed it"? In practice I believe it does .. so not sure why they're using 20215:51
*** salv-orlando has quit IRC15:51
*** felipemonteiro__ has quit IRC15:51
mdboothMeh, I wonder if it's reasonable to assert this is sync and make it the cinder driver's problem if it isn't15:51
mdboothlyarwood: ^^^ ?15:52
imacdonnmdbooth: in any case, I think that nova-compute has to assume that the cinder API call won't return until the work as completed .. if that's not happening, it's either a bug in cinder, or a large other can of worms (need an async mechanism between nova and cinder15:52
lyarwoodmdbooth: yeah that's my view15:52
mdboothlyarwood: Ok15:52
*** vks1 has joined #openstack-nova15:53
lyarwoodmdbooth: but we should really make that assumption known to the cinder folks15:53
lyarwoodmdbooth: using this driver as an example15:53
mdboothYep15:53
openstackgerritDan Smith proposed openstack/nova master: Warn the user about orphaned extra records during keypair migration  https://review.openstack.org/45856415:53
dansmithmriedem: this is what I was thinking ^15:53
*** zenoway has quit IRC15:53
dansmithmriedem: I'll mark the bug as invalid and also comment on the original bug warning people that they eff'd themselves15:54
*** chyka has joined #openstack-nova15:54
*** zenoway has joined #openstack-nova15:54
zioprotodansmith: with this patch https://review.openstack.org/#/c/458564 I should be able to go on without cleaning the db, right ?15:54
zioprotoit is the patch I applied manually, right?15:54
*** zenoway has quit IRC15:55
dansmithzioproto: no15:55
dansmithzioproto: it's just a warning15:55
*** zenoway has joined #openstack-nova15:55
dansmithzioproto: it won't fix the problem, which will make the second keypair migration unable to proceed15:55
dansmithzioproto: you have to clean the database15:55
sfinucanvladikr, sahid: Little cleanup here, if you could take a gawk https://review.openstack.org/#/c/417378/15:56
vladikrsfinucan, sure15:57
*** felipemonteiro has joined #openstack-nova15:58
*** amotoki has joined #openstack-nova15:58
openstackgerritMatt Riedemann proposed openstack/nova master: Deprecate os-hosts API  https://review.openstack.org/45650415:58
*** dtp has joined #openstack-nova15:58
mriedemalex_xu: ^ fixed that unit test15:59
*** zenoway has quit IRC15:59
*** david-lyle has quit IRC15:59
*** mpaolino_ has quit IRC15:59
*** afazekas_ is now known as afazekas16:00
mdboothlyarwood: My patch won't fix this, btw, because it needs to be global: https://review.openstack.org/#/c/430213/2/nova/compute/manager.py16:00
*** mdrabe has quit IRC16:00
*** ig0r_ has quit IRC16:01
*** liverpooler has quit IRC16:01
mdboothlyarwood: When using iscsi, I think all connect and disconnect calls must be serialised, where disconnect also include terminate connection16:01
lyarwoodmdbooth: global in terms of the compute node right?16:02
mdboothlyarwood: Yes16:02
lyarwoodmdbooth: I assumed the manager was, maybe I'm missing something16:02
mdboothlyarwood: The lock has to be global16:03
mdboothIn my patch it's not, it's keyed on instance uuid16:03
imacdonnlyarwood: https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L226116:03
*** amotoki has quit IRC16:03
*** liverpooler has joined #openstack-nova16:03
*** zenoway has joined #openstack-nova16:03
mdboothi.e. No 2 connect/disconnect operations affecting iscsi can occur concurrently, regardless of target or instance16:04
mdboothBut that's not true of other volume drivers, which is evil16:04
mdboothSo unless we want to push this restriction onto all volume backends, terminate connection would have to be managed by os-brick16:06
*** dave-mccowan has quit IRC16:06
*** mdrabe has joined #openstack-nova16:07
mdboothlyarwood johnthetubaguy: ^^^16:07
lyarwoodmdbooth: yeah sorry the cinderv3 meeting just started, I see what you mean know re global16:07
lyarwoodnow*16:07
*** david-lyle has joined #openstack-nova16:07
sdagueso, given the issues with https://bugs.launchpad.net/nova/+bug/1511466, how about bump to critical, and provide some better guidance to get through it16:08
openstackLaunchpad bug 1511466 in OpenStack Compute (nova) "migrate flavor data impossible" [Critical,Confirmed]16:08
*** zenoway has quit IRC16:08
sdaguebecause it does seem like it's coming up on google hunts :)16:08
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #2: functional  https://review.openstack.org/43859816:09
efriedsdague mriedem ^^ Thanks for the review, back atcha.16:10
openstackgerritChris Dent proposed openstack/nova master: Update resource tracker to PUT custom resource classes  https://review.openstack.org/45691516:11
openstackgerritChris Dent proposed openstack/nova master: [placement] Idempotent PUT /resource_classes/{name}  https://review.openstack.org/44879116:11
*** zhurong has quit IRC16:11
*** nic has joined #openstack-nova16:12
*** traskat has quit IRC16:12
*** dikonoor has joined #openstack-nova16:12
openstackgerritBalazs Gibizer proposed openstack/nova master: Using max api version in notification sample test  https://review.openstack.org/44368616:15
*** Drankis has quit IRC16:15
openstackgerritBalazs Gibizer proposed openstack/nova master: Using max api version in notification sample test  https://review.openstack.org/44368616:17
*** dave-mccowan has joined #openstack-nova16:21
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: power_on/off and reboot  https://review.openstack.org/42738016:22
*** dave-mcc_ has joined #openstack-nova16:23
*** ijw has joined #openstack-nova16:23
openstackgerritBalazs Gibizer proposed openstack/nova master: Using max api version in notification sample test  https://review.openstack.org/44368616:24
openstackgerritBalazs Gibizer proposed openstack/nova master: add tags field to instance.update notification  https://review.openstack.org/40722816:24
*** dave-mccowan has quit IRC16:26
gibimriedem: I update the patch that adds tags to the instance.update notification as we agreed https://review.openstack.org/#/c/407228/16:26
*** gongysh has quit IRC16:26
openstackgerritDan Smith proposed openstack/nova master: Warn the user about orphaned extra records during keypair migration  https://review.openstack.org/45856416:26
openstackgerritDan Smith proposed openstack/nova master: Make server groups api aware of multiple cells for membership  https://review.openstack.org/45733816:27
openstackgerritDan Smith proposed openstack/nova master: Sort CellMappingList.get_all() for safety  https://review.openstack.org/44317416:27
openstackgerritDan Smith proposed openstack/nova master: Clean up ClientRouter debt  https://review.openstack.org/44448716:27
*** ngupta has quit IRC16:27
openstackgerritDan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack  https://review.openstack.org/45853716:27
openstackgerritDan Smith proposed openstack/nova master: Add workaround to disable group policy check upcall  https://review.openstack.org/44273616:27
*** ijw has quit IRC16:29
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #3: TaskFlow  https://review.openstack.org/43872916:29
*** damien_r has quit IRC16:30
*** lucasagomes is now known as lucas-afk16:30
*** chyka has quit IRC16:30
sfinucanjohnthetubaguy: could you take a look at https://review.openstack.org/#/c/449288/ at some point? I'd like to know if there are backwards-compatibility implications for doing something like that16:30
*** chyka has joined #openstack-nova16:31
*** karimb has quit IRC16:34
*** tesseract has quit IRC16:34
*** chyka_ has joined #openstack-nova16:35
*** chyka has quit IRC16:35
*** annegentle has quit IRC16:38
*** annegentle has joined #openstack-nova16:39
*** ralonsoh__ has quit IRC16:40
sfinucanmdbooth: Happy to push on with https://review.openstack.org/#/c/383859/ if you fancy sticking in that context comment16:42
sfinucan(in case that wasn't obvious from earlier)16:42
*** lucasxu has quit IRC16:44
*** damien_r has joined #openstack-nova16:44
*** fragatina has joined #openstack-nova16:45
*** derekh has quit IRC16:48
*** ssurana has joined #openstack-nova16:48
*** jaosorior has quit IRC16:49
*** stvnoyes has quit IRC16:50
*** damien_r has quit IRC16:50
*** stvnoyes has joined #openstack-nova16:51
*** yingwei has quit IRC16:52
*** kfarr has quit IRC16:53
*** esberglu has quit IRC16:55
*** lucasxu has joined #openstack-nova16:55
*** mlavalle has quit IRC16:56
*** alexpilo_ has joined #openstack-nova16:57
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #4: full flavor  https://review.openstack.org/39128816:58
clarkbmriedem: just fyi I'm fixing the live migration job fails on ocata right now16:59
clarkbmriedem: only a portion of the UCA stuff was backported...16:59
*** hemna is now known as willynilly16:59
*** willynilly is now known as hemna16:59
mriedemclarkb: ok, thanks17:00
mriedemdidn't realize it was broken, but i see that now17:01
*** alexpilotti has quit IRC17:01
*** gjayavelu has joined #openstack-nova17:01
*** alexpilo_ has quit IRC17:01
clarkbmriedem: I've been checking e-r every morning and looking at the top gate bug :) just to make sure everything is happy17:02
*** liverpooler has quit IRC17:03
*** hshiina has quit IRC17:03
*** mvk has quit IRC17:03
*** liverpooler has joined #openstack-nova17:06
johnthetubaguymriedem: so I just -1ed the thing I just +Wed https://review.openstack.org/#/c/43875017:08
johnthetubaguymriedem: with a note about that begin_detaching stuff17:08
johnthetubaguymriedem: if what I said is total nonsense, I am happy for someone to over-ride that worry, the rest of it looks fine17:08
*** jpena is now known as jpena|off17:10
*** cdent has quit IRC17:10
*** efoley_ has quit IRC17:12
*** david-lyle_ has joined #openstack-nova17:17
*** david-lyle has quit IRC17:17
*** ltomasbo is now known as ltomasbo|away17:17
*** alexpilotti has joined #openstack-nova17:18
*** yamahata has joined #openstack-nova17:18
*** lyan has quit IRC17:19
openstackgerritLee Yarwood proposed openstack/nova master: encryptors: Switch to os-brick encryptor classes  https://review.openstack.org/39159717:19
ildikovjohnthetubaguy: isn't begin_detaching an API-level call?17:21
ildikovjohnthetubaguy: I mean it gets called before we even get deeper down to detach with the new flow17:22
*** david-lyle_ has quit IRC17:23
*** mvk has joined #openstack-nova17:25
*** Apoorva has joined #openstack-nova17:28
*** cdent has joined #openstack-nova17:30
*** ig0r_ has joined #openstack-nova17:32
*** esberglu has joined #openstack-nova17:34
mriedemjohnthetubaguy: i don't follow the concern in this patch17:39
mriedemjohnthetubaguy: as mentioned during the meeting, i thought begin_detaching could be used in both the old and new flows17:40
*** salv-orl_ has quit IRC17:44
*** salv-orlando has joined #openstack-nova17:44
ildikovmriedem: I think it's currently used before we get into the actual detach parts with both flows right now17:45
mriedemyes it is17:45
mriedemb/c it's used in the api17:45
*** vks1 has quit IRC17:45
ildikovthat's what I thought that it's in a different layer so it happens before we get to choose between the new/old flow17:46
*** lyan has joined #openstack-nova17:47
*** sambetts is now known as sambetts|afk17:50
*** alexpilotti has quit IRC17:53
*** dillaman has quit IRC17:56
*** vks1 has joined #openstack-nova17:57
*** fragatina has quit IRC17:58
*** amotoki has joined #openstack-nova18:00
*** ijw has joined #openstack-nova18:01
*** vks1 has quit IRC18:01
*** avolkov has quit IRC18:02
*** amotoki has quit IRC18:05
*** karimb has joined #openstack-nova18:05
*** salv-orlando has quit IRC18:06
openstackgerritJay Pipes proposed openstack/nova master: placement: implement get_inventory() for libvirt  https://review.openstack.org/45778218:06
*** karimb has quit IRC18:08
*** ayogi has quit IRC18:11
*** gszasz has quit IRC18:12
*** voelzmo has joined #openstack-nova18:12
*** dillaman has joined #openstack-nova18:12
*** vks1 has joined #openstack-nova18:16
*** kristian__ has quit IRC18:16
openstackgerritIldiko Vancsa proposed openstack/nova master: Add Cinder v3 detach to swap_volume  https://review.openstack.org/45697118:18
openstackgerritIldiko Vancsa proposed openstack/nova master: Add Cinder v3 detach call to _terminate_volume_connections  https://review.openstack.org/45689618:18
openstackgerritIldiko Vancsa proposed openstack/nova master: Add Cinder v3 detach call to post_live_migration  https://review.openstack.org/45698818:18
*** cdent has quit IRC18:21
*** MasterOfBugs has joined #openstack-nova18:21
*** voelzmo has quit IRC18:21
*** stvnoyes has quit IRC18:24
*** stvnoyes has joined #openstack-nova18:25
*** ge0rgi has joined #openstack-nova18:27
ildikovmriedem: lyarwood: johnthetubaguy: fixed up the chain ^^18:29
lyarwoodildikov: cool thanks18:30
ildikovmriedem: lyarwood: johnthetubaguy: these two are not relying on _terminate_volume_connections: https://review.openstack.org/#/c/456851/ https://review.openstack.org/#/c/456877/18:30
mriedemok18:30
ildikovthe last two need the almost approved base change as that's where attachment_delete is added18:31
ildikovI can move those on top of that patch if we think it's going to be blocked for longer18:31
ildikovI hope not though18:31
mriedemyou could just use the Depends-On tag18:31
ildikovoh, ok good point18:32
ildikovI always thought that's more for cross-repo dependencies...18:32
*** vks1 has quit IRC18:33
ildikovlyarwood: all the patches will need some additional tests, so that can be picked up by anyone who's available18:33
ildikovlyarwood: stvnoyes helped out with the base patch18:33
ildikovlyarwood: stvnoyes: the two independent patches could be good candidates if either of you have bandwidth18:34
mriedemok i just commented on the lack of a test in the local delete in api one18:34
ildikovlyarwood: stvnoyes: I can babysit the chain18:34
lyarwoodyeah I can dig into that tomorrow18:34
*** Sukhdev has joined #openstack-nova18:34
ildikovmriedem: yep, tnx, that's a "known issue" at least18:34
ildikovlyarwood: sounds good, tnx, please ping stvnoyes and me if you start with either of those or both so we don't work on it in parallel18:35
lyarwoodwill do, thanks18:35
*** artom has quit IRC18:37
ge0rgiHello I am a CS working on my master thesis. I have a simple question: How do I raise an exception in nova compute manager so that the error message from the exception is displayed in horizon as well? Can I use notifications for this? I saw the notify_about_instance_usage method and that it accepts a fault parameter, however I don't see the message displayed.18:37
*** artom has joined #openstack-nova18:37
stvnoyeslyarwood, ildikov- i am up for either one, how about I take https://review.openstack.org/#/c/456877/ ?18:38
ildikovstvnoyes: sounds good to me, tnx!18:38
*** jerrygb has quit IRC18:39
*** Sukhdev has quit IRC18:40
*** Sukhdev has joined #openstack-nova18:40
*** Sukhdev has quit IRC18:43
*** lucasxu has quit IRC18:44
*** lucasxu has joined #openstack-nova18:45
*** dikonoor has quit IRC18:45
*** Sukhdev has joined #openstack-nova18:50
*** jerrygb has joined #openstack-nova18:51
*** alexpilotti has joined #openstack-nova18:52
*** alexpilotti has quit IRC18:57
*** mnestratov has quit IRC19:00
*** lucasxu has quit IRC19:00
*** lucasxu has joined #openstack-nova19:01
*** liusheng has quit IRC19:01
*** liusheng has joined #openstack-nova19:02
*** lucasxu has quit IRC19:04
*** jerrygb has quit IRC19:07
*** abalutoiu has quit IRC19:11
*** nkorabli_ has joined #openstack-nova19:13
*** liusheng has quit IRC19:13
*** liusheng has joined #openstack-nova19:14
rajinirAre live migration settings removed fom nova.conf? Looking at live_migration_progress_timeout19:16
*** kristian__ has joined #openstack-nova19:16
*** fragatina has joined #openstack-nova19:16
*** nkorabli has quit IRC19:16
*** kristian__ has quit IRC19:21
*** karimb has joined #openstack-nova19:22
efriedIs it the done thing to use save_and_reraise_exception instead of plain raise everywhere?19:31
efriedLike, are we really worried about something as simple as LOG.* changing the active exception context?19:32
efriedmriedem ^^19:32
*** felipemonteiro has quit IRC19:33
*** david-lyle has joined #openstack-nova19:33
*** jerrygb has joined #openstack-nova19:36
sdagueefried: you got an example?19:38
*** awaugama has quit IRC19:38
efriedsdague https://review.openstack.org/#/c/438598/13/nova/virt/powervm/vm.py@18519:38
*** kristian__ has joined #openstack-nova19:38
efriedetc.19:38
*** kristian__ has quit IRC19:40
sdagueif it's legit just a log message, it's probably fine, we have a number of those in the code19:41
*** salv-orlando has joined #openstack-nova19:41
sdagueif you are doing anything more complicated, the save_and_reraise is probably better19:41
*** kristian__ has joined #openstack-nova19:41
edmondswsdague, the comments at the top of save_and_reraise_exception indicate problems can be created switching greenthreads, so does it even matter how complicated what you're doing is?19:42
*** voelzmo has joined #openstack-nova19:42
*** pchavva has quit IRC19:42
edmondswhttps://github.com/openstack/oslo.utils/blob/master/oslo_utils/excutils.py#L15119:43
efriededmondsw I don't see how it helps, considering the greenthread switch could happen in the middle of the save_and_reraise() setup anyway.19:44
efriedAnd that would have to be a base python bug if it really worked that way, nah?19:44
sdagueedmondsw: so maybe, but if that's the case we've got hundreds of issues to clean up in Nova19:44
cfriesensfinucan: you around?19:45
efriedsdague 56 ;-)19:45
sdagueefried: ok, 56 :)19:45
efriedWe counted ;-)19:45
edmondswefried sdague I don't know, I'd just err on the side of caution and use the method... if it doesn't help, at least it doesn't hurt19:45
edmondswand hopefully it helps19:45
sdaguestill, anyway, it seems like a standard we don't really need to hold the powervm folks to for new code19:45
edmondswsdague ok19:45
*** kristian__ has quit IRC19:46
sdagueedmondsw: we would probably not be opposed to additional patches cleaning up that all over the place19:47
edmondswsdague sure :)19:48
*** ijw has quit IRC19:48
efriedIf it's the preferred SOP, I'll do it.  Just extra code/logic for what seems like a fairly paranoid reason.19:50
openstackgerritDan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack  https://review.openstack.org/45853719:50
openstackgerritDan Smith proposed openstack/nova master: WIP: Move to proper target_cell calling convention  https://review.openstack.org/45863419:50
openstackgerritDan Smith proposed openstack/nova master: Make target_cell() yield a new context  https://review.openstack.org/45863519:50
openstackgerritDan Smith proposed openstack/nova master: WIP: De-complicate some of the instance delete path  https://review.openstack.org/45863619:50
dansmithmriedem: I think you will want to look at this: https://review.openstack.org/45863619:50
*** kristian__ has joined #openstack-nova19:52
*** abalutoiu has joined #openstack-nova19:52
*** satyar has quit IRC19:53
*** djohnsto has joined #openstack-nova19:53
*** fragatina has quit IRC19:55
*** chyka_ has quit IRC19:55
*** chyka has joined #openstack-nova19:57
*** voelzmo has quit IRC19:59
*** amotoki has joined #openstack-nova20:01
mriedemoh come on20:01
mriedemdansmith: https://review.openstack.org/#/c/455007/20:01
*** salv-orlando has quit IRC20:01
mriedembut yeah, that was part of my master plan at some point here too, removing the context managers20:02
mriedemsince they are redundant since we stash the cell on the context20:02
mriedemwhen looking up the instance20:02
dansmithum, we can just remove that whole block?20:02
mriedemand do normal local_delete?20:02
mriedemor you mean this? https://review.openstack.org/#/c/455007/2/nova/compute/api.py20:03
dansmiththat yeah20:03
mriedemit appears it's ok except for something with cells 120:03
mriedemwhich i haven't debugged yet20:03
dansmithokay, that's a different block than I'm working on right?20:03
mriedemthat whole block was written before https://review.openstack.org/#/c/367557/20:04
mriedemyes20:04
mriedem^ is what deletes the instance in conductor20:04
mriedemif the build request is gone20:04
mriedemand that code likely suffers from the same quota/local delete mess that _delete handles20:04
mriedemi.e. we do the same gd local delete thing in 3 places20:04
dansmithokay, I'm confused20:04
*** amotoki has quit IRC20:05
dansmithI don't think we can remove the targeting dance that does different things with the instance and the quota stuff, until the patches I have below this that avoid mutating the context, right?20:06
mriedemi haven't seen your patches yet20:06
mriedemso idk20:06
*** sree has joined #openstack-nova20:06
mriedembut that's why i didn't dig into removing the context dance yet20:06
mriedemthat's why i started with https://review.openstack.org/#/c/455007/20:06
dansmithright, so I guess Idon't understand the "oh come on" comment :)20:06
mriedemsince there was a big fat note in there about removing it after we deleted the instance in conducto20:07
mriedem"oh come on" as in i thought we were doing the same thing, before i actually looked at your change20:07
mriedembeyond the commit title20:07
mriedemsucka20:07
mriedemwe are trying to converge on the same thing though - which is cleanup the 3-headed local delete monster we have20:08
dansmith...okay20:08
mriedemor not20:08
*** eharney has quit IRC20:08
*** salv-orlando has joined #openstack-nova20:09
*** smatzek has quit IRC20:09
*** sree has quit IRC20:11
*** spotz is now known as spotz_zzz20:15
*** chyka has quit IRC20:16
*** liverpooler has quit IRC20:16
*** chyka has joined #openstack-nova20:17
*** spotz_zzz is now known as spotz20:17
*** chyka has quit IRC20:22
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #2: functional  https://review.openstack.org/43859820:23
efriededmondsw ^^ Done and done.  Thanks for the review.20:23
*** chyka has joined #openstack-nova20:27
*** felipemonteiro has joined #openstack-nova20:28
*** Sukhdev has quit IRC20:28
*** mlavalle has joined #openstack-nova20:29
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #2: functional  https://review.openstack.org/43859820:34
*** kaisers_ has quit IRC20:35
*** dtp has quit IRC20:35
mriedemare we going to be backporting this? https://review.openstack.org/#/c/456397/20:35
mriedemdansmith: ^?20:35
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: power_on/off and reboot  https://review.openstack.org/42738020:35
*** annegentle has quit IRC20:35
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: 2.43: createImage/createBackup image_id is in response body  https://review.openstack.org/45700820:36
*** abalutoiu_ has joined #openstack-nova20:40
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #3: TaskFlow  https://review.openstack.org/43872920:42
*** abalutoiu has quit IRC20:44
*** cleong has quit IRC20:46
*** ijw has joined #openstack-nova20:49
cfriesenany libvirt experts here?  apparently libvirt doesn't like "cache=writethrough io=native" for a disk, but I don't see nova being aware of this.  (We hit the issue by trying to use a disk with only 4K blocks, so it failed the direct-io test and set a cache mode of "writethrough")20:49
*** edmondsw has quit IRC20:51
*** edmondsw has joined #openstack-nova20:51
openstackgerritMatt Riedemann proposed openstack/nova master: api-ref: move createBackup to server-actions  https://review.openstack.org/45297120:52
openstackgerritMatt Riedemann proposed openstack/nova master: 2.43: Remove Location header from createImage and createBackup responses  https://review.openstack.org/45544320:52
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #4: full flavor  https://review.openstack.org/39128820:54
cfriesensecond question...if we have a disk with 4K blocks and no 512B fallback, and we use "cache=none", does the guest need to be able to handle 4KB blocks or does qemu handle that?20:54
cfriesenI realize it's not really a nova question...but I thought there might be folks on here who've run into it.20:54
*** ijw has quit IRC20:56
*** edmondsw has quit IRC20:56
*** david-lyle has quit IRC20:57
*** annegentle has joined #openstack-nova21:00
*** rfolco has quit IRC21:01
*** fragatina has joined #openstack-nova21:01
*** amotoki has joined #openstack-nova21:02
*** cdent has joined #openstack-nova21:03
*** jdurgin has quit IRC21:03
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #2: functional  https://review.openstack.org/43859821:04
*** amotoki has quit IRC21:07
*** dtp has joined #openstack-nova21:10
*** thorst has quit IRC21:11
*** burt has quit IRC21:13
mriedemefried: i hope you were done making changes to that patch21:14
efriedmriedem Which 'un?21:14
mriedemhttps://review.openstack.org/43859821:14
efriedmriedem I certainly hope so too ;-)21:14
efriedThanks!21:14
*** dimtruck is now known as zz_dimtruck21:16
*** krtaylor has quit IRC21:16
*** edmondsw has joined #openstack-nova21:20
*** nkorabli_ has quit IRC21:22
*** nkorabli has joined #openstack-nova21:22
*** sdague has quit IRC21:23
*** krtaylor has joined #openstack-nova21:24
*** edmondsw has quit IRC21:25
*** gjayavelu has quit IRC21:31
*** karimb has quit IRC21:34
*** catintheroof has quit IRC21:34
*** kaisers has joined #openstack-nova21:35
*** felipemonteiro has quit IRC21:42
mriedemeasy patches in need of a +W https://review.openstack.org/#/q/topic:bump-min-libvirt+status:open21:44
dansmithmriedem: yeah we shoud, you want me to?21:46
dansmith*should21:46
*** esberglu has quit IRC21:46
*** sneti_ has joined #openstack-nova21:46
mriedemdansmith: sure21:46
*** jamesden_ has quit IRC21:46
*** esberglu has joined #openstack-nova21:46
*** sneti_ has quit IRC21:47
*** nkorabli has quit IRC21:47
*** nkorabli has joined #openstack-nova21:48
*** salv-orl_ has joined #openstack-nova21:49
dansmithmriedem: https://review.openstack.org/#/q/Ia7e773674a4976fc03deee3f08a6ddb45568ec11,n,z21:49
*** cdent has quit IRC21:50
*** esberglu has quit IRC21:51
*** salv-orlando has quit IRC21:51
mriedemooo and you didn't even go lazy and backport both via the master change using gerrit21:52
mriedemyou knew i'd check that21:52
*** ijw has joined #openstack-nova21:52
*** nkorabli has quit IRC21:52
mriedem+2s all around21:53
*** Guest86170 has quit IRC21:53
dansmithI didn't21:53
dansmithI just never use the gerrit ui for much of anything21:53
dansmith'cause I don't trust it21:53
*** cdent has joined #openstack-nova21:54
mriedemi have the gerrit ui do my taxes and travel planning21:54
dansmithheh21:54
*** kristian__ has quit IRC21:57
*** gouthamr has quit IRC21:57
*** kristian__ has joined #openstack-nova21:57
*** gjayavelu has joined #openstack-nova21:57
*** ijw has quit IRC21:58
*** Guest86170 has joined #openstack-nova21:59
efriedI always trust the gerrit UI to do rebases & cherry-picks when it's able.  Until now.  It just broke me.21:59
*** ge0rgi has quit IRC22:00
clarkbfwiw you shouldn't really need gerrit to do rebases for you22:02
*** kristian__ has quit IRC22:02
clarkbit can't resolve conflicts and thats the only time rebases should be necessary22:02
*** annegentle has quit IRC22:02
efriedclarkb Unless you're dealing with a stack of changes.22:02
clarkboh I always curate the stack locally22:03
*** zz_dimtruck is now known as dimtruck22:03
efriedclarkb When it's a trivial rebase, I like the two-clicks-on-gerrit rather than the multi-step download/rebase/upload.  All that extra typing!22:03
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #4: full flavor  https://review.openstack.org/39128822:04
efried'cept ^^ where it effed up.22:05
clarkbright I mean rather than push just a parent change in a stack then clicking buttons I will push the entire stack to avoid pushing any buttons22:05
*** Sukhdev has joined #openstack-nova22:06
efriedclarkb I dig it.22:06
*** thorst has joined #openstack-nova22:06
*** mdrabe has quit IRC22:06
*** thorst has quit IRC22:07
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: console  https://review.openstack.org/40940222:08
*** Apoorva_ has joined #openstack-nova22:08
*** Apoorva has quit IRC22:11
*** jdurgin has joined #openstack-nova22:12
*** marst_ has joined #openstack-nova22:17
*** marst_ has quit IRC22:18
*** marst_ has joined #openstack-nova22:18
*** djohnsto has quit IRC22:19
*** marst has quit IRC22:21
mriedemalways rebase an entire stack locally22:22
*** salv-orl_ has quit IRC22:22
mriedemunless you're crazy22:22
*** lyan has quit IRC22:22
*** salv-orlando has joined #openstack-nova22:23
*** annegentle has joined #openstack-nova22:24
*** marst_ has quit IRC22:25
*** lyan has joined #openstack-nova22:25
*** jdurgin has quit IRC22:25
*** jamielennox is now known as jamielennox|away22:25
*** salv-orlando has quit IRC22:27
openstackgerritEric Fried proposed openstack/nova master: PowerVM Driver: SSP emphemeral disk support  https://review.openstack.org/44318922:28
*** annegentle has quit IRC22:29
*** jamielennox|away is now known as jamielennox22:29
mriedemefried: https://review.openstack.org/#/c/427380/ failed powervm in-tree ci22:34
mriedemerror: could not apply fb15915... PowerVM Driver: spawn/destroy #3: TaskFlow22:34
mriedemyou need to rebase that change don't you?22:35
*** thorst has joined #openstack-nova22:39
*** Apoorva_ has quit IRC22:41
*** Apoorva has joined #openstack-nova22:41
*** ijw has joined #openstack-nova22:55
*** thorst has quit IRC22:56
*** amotoki has joined #openstack-nova22:56
*** cdent has quit IRC22:59
*** ijw has quit IRC23:00
*** nic has quit IRC23:00
*** amotoki has quit IRC23:01
*** sdague has joined #openstack-nova23:06
openstackgerritchristopher.uhler proposed openstack/nova master: Libvirt volume driver for Veritas HyperScale  https://review.openstack.org/44395123:08
*** ssurana has quit IRC23:08
*** Sukhdev has quit IRC23:08
*** ssurana has joined #openstack-nova23:08
mriedemsmcginnis: what is https://www.openstack.org/summit/boston-2017/summit-schedule/events/18681/track-chairs-recap-and-discussion ?23:11
*** kaisers has quit IRC23:19
*** slaweq has quit IRC23:20
*** slaweq has joined #openstack-nova23:20
*** jdurgin has joined #openstack-nova23:22
*** slaweq has quit IRC23:24
*** chyka has quit IRC23:33
*** gouthamr has joined #openstack-nova23:33
*** mlavalle has quit IRC23:36
*** lyan has quit IRC23:37
openstackgerritMatt Riedemann proposed openstack/nova master: Using max api version in notification sample test  https://review.openstack.org/44368623:38
openstackgerritMatt Riedemann proposed openstack/nova master: Adding auto_disk_config field to InstancePayload  https://review.openstack.org/41918523:38
openstackgerritMatt Riedemann proposed openstack/nova master: add tags field to instance.update notification  https://review.openstack.org/40722823:38
*** markvoelker has quit IRC23:39
mriedemgibi: for your morning coffee, the rebuild notifications in this change have duplicate entries for auto_disk_config https://review.openstack.org/#/c/419185/21/doc/notification_samples/instance-rebuild-end.json23:39
*** takashin has joined #openstack-nova23:41
mriedemmaybe just a carried over typo23:41
*** sree has joined #openstack-nova23:42
*** jerrygb has quit IRC23:45
*** sree has quit IRC23:46
openstackgerritMatt Riedemann proposed openstack/nova master: Adding auto_disk_config field to InstancePayload  https://review.openstack.org/41918523:47
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Microversion 2.43 - Enable cold migration with target host  https://review.openstack.org/40670723:48
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Microversion 2.43 - List/Show all server migration types  https://review.openstack.org/43083923:48
*** thorst has joined #openstack-nova23:53
*** salv-orlando has joined #openstack-nova23:54
*** ijw has joined #openstack-nova23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!