Wednesday, 2019-02-13

mriedembesides the API service is registered in cell0 http://paste.openstack.org/show/744985/00:01
sean-k-mooneyhttp://paste.openstack.org/show/744986/00:02
mriedemthis should be the route handler https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/versionsV21.py#L2300:02
dansmithyeah I thought we surveyed the service version of the computes somewhere early like that00:02
dansmithhave to do some debug printing00:02
*** wolverineav has joined #openstack-nova00:03
mriedemso we have the recentish change where on service startup we try to get the minimum compute service version,00:03
sean-k-mooneyso if you  http://192.168.100.19/compute/ it has the microverion in the responce. isnt that how it fitures out the version to use00:03
mriedemsean-k-mooney: the services table version is not the same as microversions00:03
sean-k-mooneyok00:03
*** markvoelker has quit IRC00:06
mriedemhmm, i think it's the compute.API code trying to create an RPC API client00:10
mriedemFeb 13 00:09:57 downcell devstack@n-api.service[14623]: DEBUG nova.compute.rpcapi [None req-53ebccae-d210-4b14-af5c-02775f3d36e8 None None] Not caching compute RPC version_cap, because min service_version is 0. Please ensure a nova-compute service has been started. Defaulting to current version. {{(pid=14625) _determine_version_cap /opt/stack/nova/nova/compute/rpcapi.py:410}}00:10
*** _alastor_ has quit IRC00:11
dansmithah could be00:11
sean-k-mooney mriedem could this be related https://github.com/openstack/nova/blob/master/nova/cmd/api.py#L4700:12
mriedemn-api service reports as active00:12
mriedemsean-k-mooney: this isn't using that code00:12
mriedemwith default devstack n-api is running under uwsgi so https://github.com/openstack/nova/blob/master/nova/api/openstack/wsgi_app.py#L7600:13
sean-k-mooneyits also done in the standalone api https://github.com/openstack/nova/blob/master/nova/cmd/api_os_compute.py#L4200:13
mriedemwhich now that i look at that i guess we don't cache the service versions00:13
mriedemsean-k-mooney: NOT USING nova/cmd/*00:13
mriedemdansmith: ^ aside that n-api under uwsgi isn't caching service versions00:13
sean-k-mooney on right because the api is under uwsig in stead of python webserver00:14
dansmithmriedem: ack00:14
*** ade_lee_ has joined #openstack-nova00:14
mriedemhttps://bugs.launchpad.net/nova/+bug/181569200:16
openstackLaunchpad bug 1815692 in OpenStack Compute (nova) "minimum nova-compute service version is not cached in nova-api when running under wsgi" [Low,Triaged]00:16
mriedemso i think the hang is on00:17
mriedemhttps://github.com/openstack/nova/blob/0bed18ffbb46c4f2d0ec87e64a39188c165398eb/nova/compute/api.py#L26100:17
mriedemhttps://github.com/openstack/nova/blob/0bed18ffbb46c4f2d0ec87e64a39188c165398eb/nova/compute/rpcapi.py#L37300:18
*** david-lyle has joined #openstack-nova00:18
mriedemhttps://github.com/openstack/nova/blob/0bed18ffbb46c4f2d0ec87e64a39188c165398eb/nova/compute/rpcapi.py#L39500:18
*** manjeets has quit IRC00:18
*** manjeets has joined #openstack-nova00:19
mriedemand i suppose i'm seeing that for every controller that is initializing a compute.API object00:19
mriedemwhich is most of them....00:19
mriedembut doesn't really explain why GET / hangs00:20
sean-k-mooneylook like that was intoduced by dan to fix https://bugs.launchpad.net/nova/+bug/180704400:20
openstackLaunchpad bug 1807044 in OpenStack Compute (nova) queens "nova-api startup does not scan cells looking for minimum nova-compute service version" [Medium,In progress] - Assigned to Matt Riedemann (mriedem)00:20
*** ade_lee_ has quit IRC00:20
mriedemyes i know, i approved it00:21
*** ade_lee_ has joined #openstack-nova00:21
*** dklyle has quit IRC00:21
sean-k-mooneysorry just loading context for myself that said im going to sleep soon00:21
mriedemwhat i need to figure out is why that impacts the versions controller route handler which doesn't use the compute API object00:22
*** sdake has quit IRC00:26
*** tetsuro has joined #openstack-nova00:26
*** ade_lee_ has quit IRC00:30
*** ade_lee_ has joined #openstack-nova00:30
*** sdake has joined #openstack-nova00:32
*** sdake has quit IRC00:32
mriedemok stopped the crazy startup noise by setting upgrade_levels.compute=rocky in nova.conf00:34
mriedemdansmith: huh and this time nova list worked00:34
mriedemwith 2.69, i got the partial results back00:35
dansmithwell, because no =auto right?00:35
mriedemstill get tracebacks in the api logs but didn't hang00:35
mriedemyes00:35
mriedemFeb 13 00:34:43 downcell devstack@n-api.service[15785]: WARNING nova.compute.multi_cell_list [None req-76014247-0c3c-4758-9014-5c69f653891f demo demo] Cell 0641a06e-4efa-4c25-acac-b92d1f9082b7 is not responding and hence is being omitted from the results00:35
mriedemFeb 13 00:34:43 downcell devstack@n-api.service[15785]: WARNING nova.compute.api [None req-76014247-0c3c-4758-9014-5c69f653891f demo demo] Cell 0641a06e-4efa-4c25-acac-b92d1f9082b7 is not responding and hence only partial results are available from this cell if any.00:35
dansmithnice00:35
mriedemi also turned those retry/interval db options to 100:36
openstackgerritMichael Still proposed openstack/nova master: Move binding ips to privsep.  https://review.openstack.org/62215100:36
openstackgerritMichael Still proposed openstack/nova master: create_veth_pair is unused, remove it.  https://review.openstack.org/62422600:36
openstackgerritMichael Still proposed openstack/nova master: Create specialist set_macaddr_and_vlan helper.  https://review.openstack.org/62422700:36
openstackgerritMichael Still proposed openstack/nova master: Move create_tap_dev into privsep.  https://review.openstack.org/62422800:36
openstackgerritMichael Still proposed openstack/nova master: Move set_vf_interface_vlan to be with its only caller.  https://review.openstack.org/62422900:36
openstackgerritMichael Still proposed openstack/nova master: Move DHCP releasing to privsep.  https://review.openstack.org/62423000:36
openstackgerritMichael Still proposed openstack/nova master: Convert additional IP management calls to privsep.  https://review.openstack.org/62423100:36
openstackgerritMichael Still proposed openstack/nova master: Move route management to privsep.  https://review.openstack.org/62423200:36
openstackgerritMichael Still proposed openstack/nova master: Move additional IP address management to privsep.  https://review.openstack.org/62459100:36
openstackgerritMichael Still proposed openstack/nova master: Move slight bonkers IP management to privsep.  https://review.openstack.org/62459200:36
openstackgerritMichael Still proposed openstack/nova master: Move arping to privsep.  https://review.openstack.org/62459300:36
openstackgerritMichael Still proposed openstack/nova master: Move cleaning conntrack to privsep.  https://review.openstack.org/62459400:36
openstackgerritMichael Still proposed openstack/nova master: Privsepify ipv4 forwarding enablement.  https://review.openstack.org/63543100:36
openstackgerritMichael Still proposed openstack/nova master: Move killing processes to privsep.  https://review.openstack.org/63543200:36
openstackgerritMichael Still proposed openstack/nova master: Move FP device creation and deletion to privsep.  https://review.openstack.org/63543300:36
openstackgerritMichael Still proposed openstack/nova master: Cleanup the exec_ebtables code a little.  https://review.openstack.org/63543400:36
openstackgerritMichael Still proposed openstack/nova master: Privsep the ebtables modification code.  https://review.openstack.org/63543500:36
openstackgerritMichael Still proposed openstack/nova master: Move adding vlans to interfaces to privsep.  https://review.openstack.org/63543600:36
mriedemhttp://paste.openstack.org/show/744987/00:37
* dansmith golf claps00:38
dansmithalso, eye mowt00:38
sean-k-mooneymriedem: so that is effectivly returning the instance ids which we no because they are in the instance mapping table in the api db but everything else is the blank/the default value since the cell is down00:41
*** _alastor_ has joined #openstack-nova00:41
mriedemyes00:42
*** mlavalle has quit IRC00:43
*** _alastor_ has quit IRC00:45
*** sdake has joined #openstack-nova00:47
*** macza has quit IRC00:49
*** openstackgerrit has quit IRC00:52
*** bhagyashris has joined #openstack-nova00:55
mriedemdansmith: ok, tried to capture the startup "hang" https://bugs.launchpad.net/nova/+bug/181569700:56
openstackLaunchpad bug 1815697 in OpenStack Compute (nova) "[upgrade_levels]compute=auto grinds the API response times when a cell is down" [Medium,Confirmed]00:56
mriedemby my calculation by default each API worker will take ~52 minutes to start...00:56
* mriedem goes to stop neglecting his family00:57
*** mriedem is now known as mriedem_away00:57
*** mriedem_away has quit IRC01:00
*** markvoelker has joined #openstack-nova01:03
*** gyee has quit IRC01:07
*** yankcrime has quit IRC01:08
*** gyee has joined #openstack-nova01:10
*** sdake has quit IRC01:14
*** sdake has joined #openstack-nova01:17
*** yedongcan has joined #openstack-nova01:26
*** sapd1 has joined #openstack-nova01:26
*** stakeda has joined #openstack-nova01:27
*** sdake has quit IRC01:29
*** krasmussen has quit IRC01:30
*** sdake has joined #openstack-nova01:32
*** Dinesh_Bhor has joined #openstack-nova01:32
*** markvoelker has quit IRC01:35
*** hamzy has joined #openstack-nova01:39
*** sdake has quit IRC01:39
*** gyee has quit IRC01:47
*** yankcrime has joined #openstack-nova02:01
*** _fragatina has quit IRC02:02
*** tetsuro has quit IRC02:05
*** wolverineav has quit IRC02:18
*** wolverineav has joined #openstack-nova02:19
*** hongbin has joined #openstack-nova02:21
*** wolverineav has quit IRC02:24
*** macza has joined #openstack-nova02:35
*** macza has quit IRC02:39
*** Dinesh_Bhor has quit IRC02:44
*** Dinesh_Bhor has joined #openstack-nova02:48
*** takashin has joined #openstack-nova02:59
*** sdake has joined #openstack-nova03:05
*** wolverineav has joined #openstack-nova03:25
*** zhanglong has quit IRC03:50
*** spsurya has joined #openstack-nova03:54
*** udesale has joined #openstack-nova03:55
*** sdake has quit IRC04:15
*** lbragstad has quit IRC04:32
*** janki has joined #openstack-nova04:32
*** macza has joined #openstack-nova04:43
*** macza has quit IRC04:48
*** hongbin has quit IRC04:55
*** tetsuro has joined #openstack-nova04:59
*** ratailor has joined #openstack-nova05:16
*** penick has quit IRC05:24
*** udesale has quit IRC05:36
*** udesale has joined #openstack-nova05:42
*** udesale has quit IRC05:48
*** udesale has joined #openstack-nova05:48
*** psachin has joined #openstack-nova05:49
*** brinzhang has joined #openstack-nova05:50
*** markvoelker has joined #openstack-nova06:00
*** markvoelker has quit IRC06:04
*** tetsuro has quit IRC06:09
*** imacdonn has quit IRC06:13
*** sridharg has joined #openstack-nova06:19
*** sdake has joined #openstack-nova06:28
*** Dinesh_Bhor has quit IRC06:30
*** openstackgerrit has joined #openstack-nova06:34
openstackgerritMerged openstack/nova master: Move binding ips to privsep.  https://review.openstack.org/62215106:34
*** Dinesh_Bhor has joined #openstack-nova06:39
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix resetting non-persistent fields when saving obj  https://review.openstack.org/63627106:41
*** Luzi has joined #openstack-nova06:43
*** wolverineav has quit IRC06:45
*** wolverineav has joined #openstack-nova06:46
*** moshele has joined #openstack-nova06:47
*** NewBruce has joined #openstack-nova06:49
*** wolverineav has quit IRC06:51
*** markvoelker has joined #openstack-nova07:00
*** ileixe has joined #openstack-nova07:07
*** sdake has quit IRC07:07
*** NewBruce has quit IRC07:09
*** udesale has quit IRC07:12
*** wolverineav has joined #openstack-nova07:18
*** belmoreira has joined #openstack-nova07:19
*** udesale has joined #openstack-nova07:19
*** wolverineav has quit IRC07:23
*** udesale has quit IRC07:24
*** udesale has joined #openstack-nova07:25
*** takashin has left #openstack-nova07:29
*** NewBruce has joined #openstack-nova07:30
*** markvoelker has quit IRC07:34
*** maciejjozefczyk has joined #openstack-nova07:38
*** ccamacho has joined #openstack-nova07:48
*** slaweq has joined #openstack-nova07:49
*** rpittau has joined #openstack-nova07:57
*** NewBruce has quit IRC08:04
*** ralonsoh has joined #openstack-nova08:05
*** yan0s has joined #openstack-nova08:07
*** tetsuro has joined #openstack-nova08:09
*** macza has joined #openstack-nova08:14
*** NewBruce has joined #openstack-nova08:15
*** tesseract has joined #openstack-nova08:16
*** macza has quit IRC08:19
*** markvoelker has joined #openstack-nova08:31
*** helenafm has joined #openstack-nova08:31
*** wolverineav has joined #openstack-nova08:31
*** wolverineav has quit IRC08:35
*** yedongcan has left #openstack-nova08:36
*** tkajinam has quit IRC08:36
*** NewBruce has quit IRC08:40
*** NewBruce has joined #openstack-nova08:41
*** NewBruce has quit IRC08:42
*** NewBruce has joined #openstack-nova08:45
*** NewBruce has quit IRC08:45
*** NewBruce has joined #openstack-nova08:47
*** dannins has joined #openstack-nova08:50
*** NewBruce has quit IRC08:51
*** xek has joined #openstack-nova08:54
*** NewBruce has joined #openstack-nova08:55
*** tetsuro has quit IRC08:57
*** NewBruce3 has joined #openstack-nova08:58
kashyapHuzzah, all unit tests pass: https://review.openstack.org/#/c/632507/09:00
*** tssurya has joined #openstack-nova09:00
*** NewBruce3 has left #openstack-nova09:00
openstackgerritLee Yarwood proposed openstack/nova master: WIP Restore connection_info after live migration rollback  https://review.openstack.org/55134909:00
kashyapgibi: Morning, when you have a minute, would really appreciate a couple of eyes on this: https://review.openstack.org/#/c/632507/ (libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein")09:01
kashyapstephenfin: ^09:01
*** NewBruce has quit IRC09:01
gibikashyap: looking09:01
*** NewBruce has joined #openstack-nova09:02
kashyapThe commit message explains it (hopefully).  And getting this in reduces expensive rebase churn.09:02
kashyapI'll be here to answer any questions.09:03
*** markvoelker has quit IRC09:04
kashyapThe code that is being removed in driver.py is what the original author (markus_z) asked us to do in his "NOTE"09:05
kashyapAnd FWIW, I double-checked with the libvirt upstream developers to ensure what I'm removing makes sense.09:06
bauzasgibi: FWIW, I had to bail out yesterday early09:06
bauzasgibi: so I'll upload my series now with a big -W09:06
bauzasso we could discuss with other folks later09:07
*** ociuhandu has quit IRC09:07
gibibauzas: ack, I will look at it09:07
*** liumk2233 has joined #openstack-nova09:08
gibikashyap: is there a connection between the min version bump and the removed virtlogd code?09:08
kashyapgibi: Yes, it is unavoidable (as noted in the commit message) in this case09:08
kashyapDue to the version constant check09:08
*** liumk_ has joined #openstack-nova09:09
gibikashyap: so the min version now bigger than what the virtlogd needs09:10
openstackgerritLee Yarwood proposed openstack/nova master: Avoid redundant initialize_connection on source post live migration  https://review.openstack.org/55130209:10
openstackgerritLee Yarwood proposed openstack/nova master: WIP Restore connection_info after live migration rollback  https://review.openstack.org/55134909:10
gibikashyap: so we can remove the check09:10
kashyapYes, indeed.09:10
kashyapgibi: Oh, just noticed, I actually didn't remove two constant files themselves :-):09:12
kashyap# Use the "logd" backend for handling stdout/stderr from QEMU processes.09:12
* kashyap goes to remove that09:12
kashyapMIN_LIBVIRT_VIRTLOGD = (1, 3, 3)09:12
kashyapMIN_QEMU_VIRTLOGD = (2, 7, 0)09:12
gibikashyap: OK, then what I miss is the reason of the removal of self._create_file_device from _create_consoles_qemu_kvm()09:12
gibiand the reason of the change in _create_consoles_s390x()09:12
*** sunnaichuan has joined #openstack-nova09:14
*** liumk2233 has quit IRC09:14
kashyapgibi: First is that: we're adding two serial devices — one of them bypasses Nova09:14
kashyapAnd Nova should not allow an extra serial console (so that one can use external tools like `virsh` to monitor the guest serial console)09:15
*** jangutter_ has joined #openstack-nova09:15
*** jangutter_ has quit IRC09:15
*** jangutter_ has joined #openstack-nova09:16
kashyapI'll the reasoning in the commit message (and for the _create_consoles_s390x() change)09:17
gibihm as far as I understand _create_file_device only makes sure that there is a console.log file09:17
gibiahh I see now09:17
gibithat code is actually noop with virtlogd09:17
*** jangutter has quit IRC09:18
gibithen I got the reason of the change in _create_consoles_qemu_kvm()09:18
kashyapExactly09:18
kashyapIf 'virtlogd' is available, the _create_file_device() just returns.09:18
gibikashyap: correct09:18
kashyapSo, we just elided the whole function, and the places where it is called09:19
kashyapIncluding the one in s390x09:19
kashyap(_create_file_device() was called in two places, and both the two occurrences were removed)09:19
gibibut in the s390x you replaced the removed _create_file_device call with a _create_pty_device call09:20
*** ociuhandu has joined #openstack-nova09:20
gibiis that _create_pty_device for sclplm needed for our always-use-virtlogd case?09:21
kashyapThat I can use some s390x eyes :-(09:21
kashyapLet me check with a libvirt dev on that09:22
gibikashyap: thanks for taking care of it09:22
kashyapgibi: Thanks for the careful review09:23
*** ociuhandu has quit IRC09:25
*** avolkov has joined #openstack-nova09:28
*** derekh has joined #openstack-nova09:29
*** mma has joined #openstack-nova09:29
openstackgerritZhenyu Zheng proposed openstack/nova master: Add method to allow reset fields for root bdm in BDM obj  https://review.openstack.org/61467209:30
kashyapgibi: So, the "sclplm" is required, here is the background from the Nova spec that added support for s390x:09:30
kashyap[quote]09:30
kashyapFor the interactive console and the log of OpenStack instances, console devices need to be generated (instead of serial devices), if the host architecture is System z (see [2]). These console devices need to have a target type "sclp" for the interactive console of OpenStack, and "sclplm" for the log of OpenStack.09:31
kashyap[/quote]09:31
*** macza has joined #openstack-nova09:31
*** ociuhandu has joined #openstack-nova09:34
* kashyap is still digging: https://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/libvirt-kvm-systemz.html09:35
*** liumk_ has quit IRC09:35
*** macza has quit IRC09:35
*** ociuhandu has quit IRC09:38
gibikashyap: do we have some kind of CI for s390x?09:46
*** mrch_ has joined #openstack-nova09:46
kashyapgibi: I see "IBM zKVM CI"09:46
kashyapAnd when I click on its log, I get a "Not Found"09:47
gibiit voted for PS2 and PS3 of your patch09:48
gibibut nothing newer09:48
gibiand yeah, I also cannot get to the logs09:48
kashyapHmm09:48
kashyapgibi: I'm first asking an s390x developer about the behaviour ... because it is all black box for me09:49
gibime too09:51
*** cdent has joined #openstack-nova09:56
Shilpacoreycb: hi09:56
*** adrianreza has joined #openstack-nova10:00
*** bhagyashris has quit IRC10:00
*** markvoelker has joined #openstack-nova10:02
openstackgerritBalazs Gibizer proposed openstack/nova master: Calculate RequestGroup resource provider mapping  https://review.openstack.org/61623910:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Fill the RequestGroup mapping during schedule  https://review.openstack.org/61952810:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Pass resource provider mapping to neutronv2 api  https://review.openstack.org/61624010:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Recalculate request group - RP mapping during re-schedule  https://review.openstack.org/61952910:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding  https://review.openstack.org/56945910:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request  https://review.openstack.org/57331710:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove port allocation during detach  https://review.openstack.org/62242110:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Record requester in the InstancePCIRequest  https://review.openstack.org/62531010:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Add pf_interface_name tag to passthrough_whitelist  https://review.openstack.org/62531110:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF  https://review.openstack.org/62354310:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Support server create with ports having resource request  https://review.openstack.org/63636010:05
*** ociuhandu has joined #openstack-nova10:12
openstackgerritSilvan Kaiser proposed openstack/nova master: Added mount fstype based validation of Quobyte mounts  https://review.openstack.org/52224510:12
kashyapgibi: One thing the s390x maintainer in QEMU confirmed is that: 'pty' for "sclplm" device will work.10:13
kashyapBut I want to get one more view from DanPB, who wrote the 'virtlogd' daemon.10:13
*** ralonsoh has quit IRC10:16
gibikashyap: cool10:18
gibikashyap: you also have a nit in the commit message, about the new min Qemu version which should be 2.8.0 not 2.5.010:18
gibibesides these I'm OK with your patch10:18
kashyapAh, thank you10:18
kashyapgibi: Yeah, getting this will allow me to clean up all the now needless version constants10:19
kashyapAnd getting that unit tests fixed took 8 days :/10:19
gibiyeah, I can imagine the pain10:19
*** wolverineav has joined #openstack-nova10:20
openstackgerritLee Yarwood proposed openstack/nova master: compute: Reject resize requests when the source host is down  https://review.openstack.org/62348910:20
stephenfinkashyap: Sorry, forgot to start HexChat. I thought mriedem had suggested we do that min version bump early in Train instead?10:20
kashyapstephenfin: He was ambivalent on it.10:20
*** jangutter_ has quit IRC10:20
kashyapstephenfin: IMHO, don't see any compelling blockers (/me hopes this statement doesn't come back to bite him)10:21
stephenfinkashyap: Cool. I'll hold of +2ing personally until I double check, though I'd be happy to get it in of course10:22
kashyapRe-working / rebasing the unit tests again for Train becomes a lot more more painful10:22
kashyapstephenfin: I actually asked him (MattR) last night here, but his nick was _afk10:22
*** jangutter has joined #openstack-nova10:23
kaisersstephenfin: Thanks, i updated the CL. If you're rolling I'd also be happy to have somebody revisit https://review.openstack.org/#/c/554195/10:24
kashyapstephenfin: gibi: Release-wise, we're just about to reach M3 (Mar 04-08), and the GA is aroud 12-Apr — we still have decent time for this to soak in10:24
openstackgerritSilvan Kaiser proposed openstack/nova master: Exec systemd-run without --user flag in Quobyte driver  https://review.openstack.org/55419510:26
*** erlon has joined #openstack-nova10:31
openstackgerritSilvan Kaiser proposed openstack/nova master: Exec systemd-run without --user flag in Quobyte driver  https://review.openstack.org/55419510:31
*** ralonsoh has joined #openstack-nova10:32
*** markvoelker has quit IRC10:34
*** jangutter has quit IRC10:38
*** jangutter has joined #openstack-nova10:41
stephenfinkashyap: Comments left. Just the commit message fix needed, fwict10:45
*** ralonsoh has quit IRC10:45
kashyapstephenfin: Most excellent.10:45
*** ralonsoh has joined #openstack-nova10:45
kashyapgibi: stephenfin: Meanwhile, I double-checked with DanPB, and what I'm doing is "sane".  Here's the conversation snippet:10:45
kashyap< kashyap> danpb: Is changing the behaviour in this case, by using the console type 'pty' instead of 'file' for s390x's "sclplm" okay?10:46
kashyap< kashyap> (FWIW, checking with Corneila of s390x, she said using 'pty' type for "sclplm" device will work.)10:46
kashyap< kashyap> If you're wondering WTH is this "sclplm", s390x has two types of console devices:10:46
kashyap< danpb> the guest doesnt' care what's done in the backend10:46
kashyap< kashyap> (1) "sclp" -- for interactive consoles10:46
kashyap< kashyap> (2) "sclplm" -- for log files10:46
kashyap< kashyap> danpb: Ah, okay, then.10:46
kashyap< danpb> personally i'd just get rid of that duplication10:46
kashyap< kashyap> Okay, I'll make that in a separate logical change10:46
kashyapstephenfin: Thanks for the quick review!10:46
kashyapstephenfin: Related to your comment, look up "negative code" here :-) -- https://en.wikipedia.org/wiki/Douglas_McIlroy#Views_on_computing10:47
stephenfinkashyap: Very true, but there's no reason removing code can't be done in multiple steps to help grok'ability. See https://review.openstack.org/#/c/636384/ and the preceding patch10:49
kashyapstephenfin: Oh, absolutely: I try to be iron-clad on the "one logical change per commit"10:50
*** jangutter has quit IRC10:52
*** wolverineav has quit IRC10:52
*** jangutter has joined #openstack-nova10:55
kashyapstephenfin: Nice work there.10:57
openstackgerritSylvain Bauza proposed openstack/nova master: libvirt: implement reshaper for vgpu  https://review.openstack.org/59920810:59
openstackgerritSylvain Bauza proposed openstack/nova master: Add functional test for libvirt vgpu reshape  https://review.openstack.org/63155910:59
openstackgerritSylvain Bauza proposed openstack/nova master: WIP: Use the correct mdev allocated from the pGPU  https://review.openstack.org/63659110:59
bauzasgibi: ^10:59
bauzasgibi: I thought about the fact that we blindly use mdevs without checking the pGPU10:59
bauzasgibi: and I think it's not a problem10:59
bauzasat least until we support multiple types11:00
bauzassince libvirt will be able to find a specific mdev11:00
bauzasthat said, I'll take my old internal hardware and do a lot of checks11:00
stephenfinkaisers: More comments on https://review.openstack.org/#/c/522245/ after a second reading11:01
*** ociuhandu has quit IRC11:02
*** ociuhandu has joined #openstack-nova11:03
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein"  https://review.openstack.org/63250711:04
kashyapgibi: stephenfin: When you can, addressed both your remarks ^11:04
*** ralonsoh has quit IRC11:05
*** jangutter has quit IRC11:05
*** jangutter has joined #openstack-nova11:06
kaisersstephenfin: thnx, checkin' in a min11:06
*** udesale has quit IRC11:13
*** brinzhang has quit IRC11:14
stephenfinkaisers: And on https://review.openstack.org/#/c/554195/211:14
*** ileixe has quit IRC11:20
*** ratailor has quit IRC11:20
*** Dinesh_Bhor has quit IRC11:21
kaisersstephenfin: yup, thnx again!11:28
*** markvoelker has joined #openstack-nova11:30
*** ccamacho has quit IRC11:39
*** macza has joined #openstack-nova11:39
*** janki has quit IRC11:40
*** janki has joined #openstack-nova11:40
*** macza has quit IRC11:44
*** tbachman has quit IRC11:44
*** sapd1 has quit IRC11:45
*** wolverineav has joined #openstack-nova11:49
*** rpittau has quit IRC11:52
*** hamzy has quit IRC11:59
*** panda is now known as panda|lunch12:00
*** markvoelker has quit IRC12:04
*** sdake has joined #openstack-nova12:04
*** ccamacho has joined #openstack-nova12:09
*** _fragatina has joined #openstack-nova12:11
*** _fragatina_ has joined #openstack-nova12:12
*** hamzy has joined #openstack-nova12:13
*** _fragatina has quit IRC12:16
*** sdake has quit IRC12:16
*** wolverineav has quit IRC12:23
*** rpittau has joined #openstack-nova12:29
*** Cardoe has quit IRC12:45
*** udesale has joined #openstack-nova12:50
*** janki has quit IRC13:00
openstackgerritSurya Seetharaman proposed openstack/nova master: Plumbing required in servers ViewBuilder to construct partial results  https://review.openstack.org/63514613:00
openstackgerritSurya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells  https://review.openstack.org/59165713:00
openstackgerritSurya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells Documentation  https://review.openstack.org/63514713:00
*** markvoelker has joined #openstack-nova13:01
*** tbachman has joined #openstack-nova13:06
*** mriedem has joined #openstack-nova13:08
*** sapd1 has joined #openstack-nova13:08
*** panda|lunch is now known as panda13:12
openstackgerritSurya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells Documentation  https://review.openstack.org/63514713:13
openstackgerritJim Rollenhagen proposed openstack/nova master: ironic: partition compute services by conductor group  https://review.openstack.org/63500613:15
openstackgerritJim Rollenhagen proposed openstack/nova master: Ensure config regexes match the entire string  https://review.openstack.org/63662713:18
*** wolverineav has joined #openstack-nova13:20
openstackgerritMatt Riedemann proposed openstack/nova master: Change nova-next tempest test regex  https://review.openstack.org/63645913:22
*** dannins has quit IRC13:27
*** janki has joined #openstack-nova13:28
Shilpastephenfin: hi, under Taskflow, added https://bugs.launchpad.net/taskflow/+bug/1815738, do let me know is this active channel? Unable to trace PTL for this, but https://pypi.org/project/taskflow/ reports latest release in Jan'1913:28
openstackLaunchpad bug 1815738 in taskflow "Taskflow doesn't support to retrieve tasks progress details in the order of workflow task execution" [Undecided,New]13:28
mriedemShilpa: i think that would be in the #openstack-oslo channel now13:30
mriedemor i guess #openstack-state-management https://wiki.openstack.org/wiki/TaskFlow#Contact_us.2113:31
mriedemtssurya: did you enjoy my tales from going down cell down street?13:31
Shilpamriedem: #openstack-state-management here no one responding, will check #openstack-oslo channel13:31
mriedemtssurya: belmoreira: i'm interested - what do you configure [upgrade_levels]/compute to in nova.conf? do you pin to a release or have it set to auto?13:32
bauzasmriedem: FWIW, I uploaded the new rev for reshape13:32
bauzasmriedem: I found a potential blocker, but eventually it's not a problem13:32
mriedemok cool13:32
belmoreiramriedem: in the past used to pin it to a release. Now we use auto13:33
*** Dinesh_Bhor has joined #openstack-nova13:34
*** markvoelker has quit IRC13:34
openstackgerritLee Yarwood proposed openstack/nova master: Avoid redundant initialize_connection on source post live migration  https://review.openstack.org/55130213:36
*** eharney has quit IRC13:37
mriedembelmoreira: ok if you have this change from stable/rocky https://review.openstack.org/#/c/624982/ then you might want to be careful,13:37
mriedemi came across something nasty last night in down cell testing13:37
mriedemhttps://bugs.launchpad.net/nova/+bug/181569713:38
openstackLaunchpad bug 1815697 in OpenStack Compute (nova) "[upgrade_levels]compute=auto grinds the API response times when a cell is down" [Medium,Confirmed]13:38
mriedemat this point i don't have any great solutions for that bug13:38
sean-k-mooneymriedem: as a workaournd you set the retry interval to 1 and retries to 1 and set the exact version13:40
sean-k-mooneythat could lead to more api request failing in other cases but it should not be inherintly dangourous right13:40
mriedemsean-k-mooney: no, the max_retries and retry_interval was just to quiet the logs13:40
sean-k-mooneyoh ok13:40
mriedemthe workaround for hanging the api on startpu was setting [upgrade_levels]/compute=rocky13:40
*** lbragstad has joined #openstack-nova13:41
tssuryamriedem: I did go through your tales :D I absolutely appreciate the testing :) I am adding more test cases for the parts you felt are lacking13:42
belmoreiramriedem: we don't have that commit, yet. Thanks for the heads up.13:42
tssuryaand I am not so sure I get your worry here: https://review.openstack.org/#/c/591657/39/nova/api/openstack/compute/servers.py@13513:42
Shilpamriedem: no one replies here too #openstack-oslo, will wait for more time13:43
*** Dinesh_Bhor has quit IRC13:44
mriedemShilpa: ok well nova doesn't use taskflow. harlowja was the primary maintainer but i'm not sure if he's around anymore. oslo team would be your best bet i'd think, otherwise maybe #openstack-dev13:45
mriedemefried loves taskflow though13:45
Shilpamriedem: ok, hope he will help me there13:46
mriedemtssurya: my concern is if tooling is passing filter parameters by default,13:47
mriedemnormally in the API we filter out search options that the user isn't allowed to use without it being an error13:47
mriedemso let's say i'm a non-admin and i pass some stuff which normally just gets filtered out in the remove_invalid_options method13:48
mriedemessentially making search_opts = {}13:48
tssuryalike "deleted" and "tenant" ?13:48
mriedemif i'm using 2.69, then empty search_opts would mean down_cell_support=True13:48
mriedemtssurya: non-admins can't use the tenant filter13:48
tssuryayea I know13:49
mriedemservers are filtered by default from the project_id in the request context13:49
tssuryawhich is why I was asking if that's what you mean by invalid options13:49
mriedemso let's say i'm a non-admin and i'm using --deleted, here are some scenarios13:50
mriedem1. nova list --deleted with v2.1 - the deleted filter would be excluded w/o any errors in the api and i should list my servers13:50
mriedem2. nova list --deleted with v2.69 and list_records_by_skipping_down_cells=True (default), the down_cell_support flag would be set to False and i wouldn't get any results (if my servers are in a down cell)13:50
mriedem3. nova list --deleted with v2.69 and list_records_by_skipping_down_cells=False, if my servers are in a down cell i'll get a 50013:51
mriedemmy point was if we calculated the down_cell_support flag *after* calling remove_invalid_options to filter the search_opts, then #3 becomes #213:51
mriedemactually no that's not right,13:52
mriedemi'd get partial results13:52
openstackgerritLee Yarwood proposed openstack/nova master: Restore connection_info after live migration rollback  https://review.openstack.org/55134913:53
mriedemto summarize, i think the down_cell_support variable should be set based on the search opts after we've filtered that set in remove_invalid_options to increase the chances of someone getting partial results13:53
lyarwoodmdbooth: ^ btw, feel free to respin the commit message if you have time.13:53
*** wolverineav has quit IRC13:53
tssuryamriedem: ok, I get what you mean13:53
*** jmlowe has quit IRC13:54
mriedemthe tricky thing is if you do that, you have to explicitly look for paging and sorting parameters13:54
mriedembecause of https://review.openstack.org/#/c/591657/39/nova/api/openstack/compute/servers.py@123513:54
tssuryayea13:54
mriedembut i dont think that's so hard, we could just create a constant for ('sort_key', 'sort_dir', 'limit', 'marker') and use that in both places13:54
mriedemand your _is_cell_down_supported method can check, "if any(PAGING_SORTING_PARAMS in reg.GET)13:55
tssuryagot it, let me dig more and incorporate that13:57
mriedemerr, if set(PAGING_SORTING_PARAMS) & set(list(req.GET.keys()))13:57
mriedemsomething like that13:57
mriedemdoes that make sense though?13:57
tssuryait makes sense yea13:57
openstackgerritHamdy Khader proposed openstack/os-vif master: Add create_ovs_port field in VIFPortProfileOpenVSwitch profile  https://review.openstack.org/63606113:57
tssuryabut I still don't get why the policy things are mixed with ignoring flags13:58
mriedemcool. my goal is to maximize the chance that someone gets partial results if they have servers in a down cell13:58
mriedemwhich policy things?13:58
tssuryaI mean if --deleted is an only admin option13:58
tssuryait shuld barf13:58
tssuryashould*13:58
tssuryasaying you are not admin13:58
mriedemlegacy backward compat13:58
tssuryajust lke tenant or user13:58
tssuryaahhh ok ok13:59
mriedemsame reason you can do GET /servers?foo=bar13:59
mriedemhttps://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/servers.py#L60613:59
openstackgerritStephen Finucane proposed openstack/python-novaclient master: Microversion 2.68: Remove 'forced' live migrations, evacuations  https://review.openstack.org/63513113:59
mriedemthe query parameter schema is whitelisted but also allows users to pass anything for backward compat,13:59
mriedemexcept the stuff in https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/servers.py#L51214:00
mriedemthose are 404ed here https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L12514:00
tssuryamriedem: wow its crazy, but I understand at least why it was done, thanks for patiently explaining that14:01
tssuryaI'll work on ironing out the wrinkles14:01
mriedemcool. otherwise from my manual testing (once i worked around those other bugs with the service version checking) it was working pretty well14:02
stephenfinmriedem: Not sure you already discussed this again with kashyap, but any serious objections to doing the libvirt min version bump now? https://review.openstack.org/#/c/632507/ gibi and I are happy but I recall some objections previously14:02
kashyapstephenfin: Heh, was _just_ about to write a message here.14:03
kashyapstephenfin: Not quite objections, mriedem was not sure, based on some discussion w/ an operator he had14:03
kashyapIdeally, I should have done this at the _start_ of the cycle, but completely lost track of it14:03
mriedemlooking, but you should probably ask the PTL14:04
adriancsean-k-mooney: Hi, regarding sriov live-migration patches, id like to address some of the comments in https://review.openstack.org/#/c/620115/21, however i saw you are planning to upload a new PS for the series, will it be soon ? as id like to avoid merge conflicts etc...14:04
kashyapmriedem: Yeah, will do.  I thought melwitt was still catching up after being on PTO (?)14:04
mriedemstephenfin: kashyap: the concern from an operator was they were upgrading from mitaka to i think queens and the minimum bump in between was just for maintenance, not related to any functional changes, and the computes they were upgrading didn't have that minimum libvirt (they were using older centos i think)14:05
mriedemso this operator was asking me if there was a reason for the bump, or if they could just revert the change to keep the older centos compute node but still upgrade nova14:06
sean-k-mooneyadrianc: i got pulled into some other stuff so go ahead.14:06
kashyapI see.  Yeah, probably we should better advertize _why_ we do bumps and at what intervals?14:06
kashyapmriedem: ^ Like me writing a reminder note to the mailing list about why we do, and give a gentle heads-up.14:06
mriedemi just think in years past we weren't real aggressive about doing it every release14:06
sean-k-mooneyadrianc: ill work on adressing the base patches but ill mainly be adressing comment/commit messages so there whould be no conflicts14:07
kashyapmriedem: Yes, I now kept a reminder on my phone to do the start of the cycle14:07
adriancsean-k-mooney: would you like me to address some of the nits in https://review.openstack.org/#/c/624842/ as well, or ill leave them to you ?14:07
*** sdake has joined #openstack-nova14:08
mriedemso looking at https://review.openstack.org/#/c/558171/ the last time we did a bump was pike, is that correct?14:08
sean-k-mooneyadrianc: am if you want to adress them i can focous on the os-vif brctl removal which is the other thing im context switch with14:09
mriedemwe advertized the next minimums in rocky14:09
*** sdake has quit IRC14:09
mriedemso really this would be the first required minimum bump since pike14:09
kashyapmriedem: Yes, I did the bump last April-May (https://review.openstack.org/#/q/topic:bump_min_libvirt_and_qemu+(status:open+OR+status:abandoned+OR+status:merged)14:09
mriedemgiving at least a year to prepare14:09
kashyapmriedem: Indeed14:09
mriedem(knowing people skip several years of openstack releases now)14:09
sean-k-mooneyadrianc: i dont think there is anything controvertiol in any of the coments on the seirs so if you have time to make the requested change go for it14:10
mriedemlooks like we don't have zkvm 3rd party CI anymore...14:10
kashyapRelease-wise, we're just about to reach M3 (Mar 04-08), and the GA is aroud 12-Apr — I feel we're still okay14:10
*** gokhani has joined #openstack-nova14:10
*** sdake has joined #openstack-nova14:11
adriancsean-k-mooney: Ack, ill start from the base patch then :)14:11
kashyapmriedem: Yeah, was wondering this morning about it14:11
mriedemhttps://wiki.openstack.org/wiki/ThirdPartySystems/IBM_zKVM_CI14:11
kashyapToday I spent 4 hours digging into s390x and their fun serial console stuff14:11
openstackgerritAlexandra Settle proposed openstack/nova master: Adding cross refs for config options in scheduler filter guide  https://review.openstack.org/63663514:11
mriedemkashyap: so how about you send a reminder to the mailing list and also copy zkvm-ci@linux.vnet.ibm.com and bjzhjing@linux.vnet.ibm.com14:11
kashyapMarkusZ left a note in the code with a pointer to a bug ... and it turned out it's a bug _I_ filed 4 years ago :P14:11
mriedemmarkus_z is long gone from openstack14:12
kashyapmriedem: I guess you mean now, right?14:12
mriedemkashyap: yeah, i'd say do that now and if we don't hear any strong objections by end of week we go ahead14:12
kashyapmriedem: I know, I just found it sadly funny that when I decided to learn more from the bug ... apparently I filed it.14:12
kashyapmriedem: Alright, let me draft it right away.14:12
kashyapmriedem: I spent a painful 8 days on-and-off arriving at the root cause.  But hey, fun :-)14:13
*** sdake has quit IRC14:13
kashyapThanks for your time.  Appreciate it14:13
mriedemyou've grown as a person i'm sure14:13
kashyapI did; I grew in "(positive) complexity" :D14:14
*** sdake has joined #openstack-nova14:15
kashyapWhat tags would I use?  [ops][nova] suffices I guess?14:15
kashyaps/suffices/suffice/14:15
mriedemyeah14:16
kashyapmriedem++14:17
kashyapHm, we should get a 'karma bot' in14:17
*** eharney has joined #openstack-nova14:22
*** Dinesh_Bhor has joined #openstack-nova14:22
*** sdake has quit IRC14:23
*** dave-mccowan has joined #openstack-nova14:25
*** dave-mccowan has quit IRC14:30
*** markvoelker has joined #openstack-nova14:31
efriedShilpa: It looks to me like you're not specifying an engine for your task flow. I thought the default was a linear one, but it's possible it's not. Do you get the same results if you specify a linear engine explicitly?14:32
Shilpaefried: tried with engine='serial' but no sucess14:32
Shilpai will check setting engine='linear' and get back to you14:33
efriedShilpa: I don't remember exactly what the engine names are, would have to consult the docs.14:33
Shilpaok14:34
*** awaugama has joined #openstack-nova14:35
efriedShilpa: Beyond that, I'm probably not a whole lot of use to you. Josh Harlow (harlowja) is the guy who could help, but I don't see him in the usual haunts; possible he's moved on to greener pastures.14:35
efriedYou may want to try posting your bug to the taskflow github repo14:35
Shilpaefried: its serial but no sucess14:35
efriedShilpa: Okay.14:36
Shilpaefried: at taskflow github repo, in REDME posted Bugs: https://bugs.launchpad.net/taskflow/14:36
*** lpetrut has joined #openstack-nova14:38
bauzasholy shit, I'm deploying devstack and placement gives me a 503 when being started14:38
bauzaswtf14:38
sean-k-mooneybauzas: its likely the same issue i had14:39
sean-k-mooneyyou have old apache config files14:39
bauzasefried: cdent: any thoughts on that ?14:39
bauzashah14:39
bauzasi asked to reclone14:39
sean-k-mooneyit wont fix it14:39
bauzasyeah14:39
bauzasbut then we're missing something in lib/placemenbt14:39
bauzasbecause a clean.sh didn't fix it too14:40
openstackgerritAlexandra Settle proposed openstack/nova master: Adding cross refs for config options in scheduler filter guide  https://review.openstack.org/63663514:40
sean-k-mooneynuke your /etc/http/site-available and sites-enabled14:40
sean-k-mooneybauzas: ya i was ment to wite a change to devstack to fix it14:40
sean-k-mooneyits on my todo list14:40
efriedcool14:40
bauzasok14:40
bauzasgtk14:40
sean-k-mooneyhopefully ill get to it by next week14:40
cdentit's a generic problem with how apache stuff is config'd not just placement14:40
cdentthere are additional issues when you switch back and forth between using uwsgi and not14:41
sean-k-mooneyya i hit the same issue with cinder i think14:41
openstackgerritAlexandra Settle proposed openstack/nova master: Adding cross refs for config options in scheduler filter guide  https://review.openstack.org/63663514:41
bauzassean-k-mooney: wait, I'm running devstack with uwsgi14:41
sean-k-mooneyyes14:41
bauzasI don't have httpd site14:41
efriedcdent: is it the kind of thing it would be appropriate to put a hack into devstack to clean up / make sure it dtrt?14:41
sean-k-mooneyapache is still being used as a reverse proxy14:41
*** _fragatina_ has quit IRC14:42
cdentefried: sean-k-mooney plans to figure it out14:42
efriedwoot14:42
bauzassean-k-mooney: wait, I don't know what to bitch14:42
sean-k-mooneyif your using ubnuntu its under /etc/apache2/...14:42
bauzasmy httpd directory is sane14:42
bauzasit's RHEL, so /etc/httpd but whatever14:42
sean-k-mooneybauzas: for me it was poining at an old socket name nova-placement.socket14:43
bauzasah, interesting14:43
sean-k-mooneyyou could see the error in the apache errror log in /var/httpd/error.log14:43
bauzasin conf.d, I have two files14:43
bauzasnova-placement-api.conf and placement-api.conf14:44
sean-k-mooneybauzas: the simple solution is to run clean unistall httpd and delete /etc/httpd then stack14:44
sean-k-mooneyit will fix everything14:44
sean-k-mooneybut really you just need to remove the placemetn config file14:44
bauzasI guess14:45
bauzaslemme try14:45
sean-k-mooneyi have done  ^ 3 times in the last week for this issue on different systems14:46
sean-k-mooneybut ill look at actully just makeing devstack fix it properly next week14:46
bauzascdent: edleafe: efried: somehow, we changed from nova-placement-api.conf to placement-api.conf without thinking of folks who upgrade14:46
bauzas:)14:46
bauzaswords of wisdom, people do upgrade :)14:46
sean-k-mooneybauzas: yep but its also devstack soo....14:47
cdentof devstack?14:47
cdenti thought the three letters on the front was a clue?14:47
sean-k-mooneyi assume grenade is just nuking the configs and replaceing them with new ones14:47
sean-k-mooneycdent: it still shows up on the openstack survay in prod ocationally14:48
sean-k-mooneycdent: honestly with the ansible rules we use in the gate for deploying it and the swap to systemd for all the things its actully a lot less dumb then it used to be14:48
sean-k-mooneycdent: it now support host reboots again14:49
bauzascdent: you don't imagine how much devstack is insightful for deployers :)14:49
bauzasbut anyway, not a big deal14:49
cdentI'm not disputing that devstack should clean up apache configs better. Just trying to point out that this is an old and not related to placement problem and not something that is super urgent. it's just a bug that needs to be fixed14:49
cdentanyone can fix it14:49
sean-k-mooneyfixed and then backported as this only happens if you swap branches14:50
*** wolverineav has joined #openstack-nova14:50
jrollI'm surprised clean.sh doesn't remove apache configs, seems like an easy fix14:50
sean-k-mooneywhich developer do a lot but ci systems and normal peole do not14:50
cdentjroll: I really think it simply comes down to "nobody has got around to it yet or been burned enough to bother". sean-k-mooney is nearly there.14:51
sean-k-mooneyjroll: it also does not remove apache. i mean that is the super simple fix but not makeing the config generate condtional woudl also have worked14:51
cdentsounds like bauzas is already there but doesn't have time14:51
*** sapd1 has quit IRC14:52
sean-k-mooneyim takeing 2 days off this week and need to get other stuff done before i do which is the only reason i havent done it yet.14:53
*** sridharg has quit IRC14:54
*** liuyulong has joined #openstack-nova14:55
kashyapAs promised earlier: http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002665.html -- [ops][nova] Heads-up: Upcoming version bump for libvirt and QEMU in 'Stein'14:58
*** penick has joined #openstack-nova14:59
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein"  https://review.openstack.org/63250714:59
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Drop MIN_LIBVIRT_PARALLELS_SET_ADMIN_PASSWD  https://review.openstack.org/63251414:59
*** jmlowe has joined #openstack-nova14:59
kashyapgibi: stephenfin: Just a commit message update on the main patch ^15:00
openstackgerritAlexandra Settle proposed openstack/nova master: Adding cross refs for config options in scheduler filter guide  https://review.openstack.org/63663515:02
*** markvoelker has quit IRC15:04
bauzasgibi: so, I uploaded my change15:06
bauzasgibi: but now the func test you wrote is failing15:06
bauzasweird, heh ?15:06
gibibauzas: sorry, I haven't looked it yet15:06
bauzasno worries, just saying15:07
*** sapd1 has joined #openstack-nova15:08
bauzasgibi: I don't get why but the root RP has a VGPU inventory, hence the mismatch15:08
*** moshele has quit IRC15:09
bauzasgibi: http://logs.openstack.org/59/631559/5/check/nova-tox-functional/100cfb6/job-output.txt.gz#_2019-02-13_11_19_37_21741515:09
bauzasgibi: any reason why you don't provide the VGPU inventory first ?15:09
*** Dinesh_Bhor has quit IRC15:16
*** jmlowe has quit IRC15:18
*** mrch_ has quit IRC15:19
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Drop MIN_LIBVIRT_PARALLELS_SET_ADMIN_PASSWD  https://review.openstack.org/63251415:19
*** janki has quit IRC15:21
*** mlavalle has joined #openstack-nova15:21
*** _alastor_ has joined #openstack-nova15:22
*** jmlowe has joined #openstack-nova15:23
*** wolverineav has quit IRC15:23
*** _alastor_ has quit IRC15:26
*** ralonsoh has joined #openstack-nova15:27
gibibauzas: sorry I was dragged away from the keyboard15:39
bauzasgibi: heh, we all are15:39
bauzasor, rather, always dragged from upstream15:39
gibibauzas: if https://review.openstack.org/#/c/631559/5/nova/tests/functional/libvirt/test_reshape.py@130 fails then it means that the reshape does not moved the VGPU resource away from the compute RP to the GPU RP15:42
gibibauzas: https://review.openstack.org/#/c/631559/5/nova/tests/functional/libvirt/test_reshape.py@123 should trigger the reshape15:43
bauzasI'll dig into pdbing it15:43
bauzasin the meantime, lemme mark my series as -215:43
*** maciejjozefczyk has quit IRC15:44
openstackgerritAlexandra Settle proposed openstack/nova master: Adding cross refs for config options in scheduler filter guide  https://review.openstack.org/63663515:54
*** jmlowe has quit IRC15:54
*** ociuhandu has quit IRC15:55
*** Luzi has quit IRC15:55
*** ociuhandu has joined #openstack-nova15:55
*** jmlowe has joined #openstack-nova15:58
gibibauzas: http://logs.openstack.org/59/631559/5/check/nova-tox-functional/100cfb6/job-output.txt.gz#_2019-02-13_11_19_37_215302 here is when the compute is restarted and should do the reshape15:59
*** tzumainn has joined #openstack-nova15:59
*** lpetrut has quit IRC16:00
*** markvoelker has joined #openstack-nova16:01
openstackgerritJack Ding proposed openstack/nova master: Flavor extra spec and image properties validation  https://review.openstack.org/62070616:02
*** sdake has joined #openstack-nova16:06
*** macza has joined #openstack-nova16:11
*** _alastor_ has joined #openstack-nova16:16
*** samc-bbc has joined #openstack-nova16:17
*** wolverineav has joined #openstack-nova16:20
*** mrch_ has joined #openstack-nova16:22
melwitto/16:24
*** jangutter has quit IRC16:26
*** markvoelker has quit IRC16:34
*** Shilpa has quit IRC16:35
*** jmlowe has quit IRC16:38
*** jmlowe has joined #openstack-nova16:44
*** sdake has quit IRC16:46
*** sdake has joined #openstack-nova16:47
openstackgerritMatt Riedemann proposed openstack/nova master: Use math.gcd starting with python 3.5  https://review.openstack.org/63666916:50
melwittmriedem: there's a patch proposed to add basic info about projects to the openstack-map, if you're interested in reviewing the nova info https://review.openstack.org/63660416:51
*** wolverineav has quit IRC16:53
*** gyee has joined #openstack-nova16:55
*** helenafm has quit IRC16:57
*** sdake has quit IRC16:58
*** _fragatina has joined #openstack-nova16:59
*** _fragatina has quit IRC17:00
*** wolverineav has joined #openstack-nova17:01
openstackgerritVlad Gusev proposed openstack/nova stable/queens: Handle unicode characters in migration params  https://review.openstack.org/63667817:01
*** itlinux has joined #openstack-nova17:05
*** tssurya has quit IRC17:06
mriedemmelwitt: commented17:09
mriedemdansmith: replied on https://review.openstack.org/#/c/469675/17:09
*** itlinux has quit IRC17:09
mriedemdansmith: for sanity, i think we likely just want to stop using check_availability_zone in both the api and compute17:09
mriedemand write an api-specific version of that17:09
*** itlinux_ has joined #openstack-nova17:09
mriedemfrom the api, gather all the precreated volumes and their azs,17:10
mriedemif any are different, it's an error17:10
mriedemif any are different from the requested server az, it's an error17:10
mriedemif the server is created without a specific az, and the volumes all have the same az, then put that on the request spec17:10
*** ociuhandu_ has joined #openstack-nova17:11
dansmithmriedem: yeah, collect the data and make the call in the compute api17:12
dansmithmriedem: like I said, if the "user requested an az and it doesn't match the volume" case still needs to be handled deep for cleanup reasons, then okay,17:12
dansmithbut (a) default_az means something totally different to me when reading the compute/api code and (b) I don't expect either of these mismatch checks to happen so deep down in that file17:13
dansmithso if you don't want to refactor this at all, I'd at least ask that you change the name of that parameter with commentage up in compute/api aboutit17:14
*** itlinux_ has quit IRC17:14
*** itlinux has joined #openstack-nova17:14
mriedem"the "user requested an az and it doesn't match the volume" case" is what would be in nova-compute after a volume is created17:15
mriedemso yeah i think that remains17:15
dansmithwait17:16
dansmithoh right, yeah17:16
*** sapd1 has quit IRC17:16
mriedemwhich reminds me of https://bugs.launchpad.net/nova/+bug/149725317:17
openstackLaunchpad bug 1497253 in OpenStack Compute (nova) "different availability zone for nova and cinder when AZ is not explicitly given" [Low,Fix released] - Assigned to Dan Smith (danms)17:17
*** psachin has quit IRC17:18
openstackgerritStephen Finucane proposed openstack/nova master: Remove get_config_vhostuser  https://review.openstack.org/56547117:19
openstackgerritStephen Finucane proposed openstack/nova master: Validate bandwidth configuration for other VIF types  https://review.openstack.org/63638317:19
openstackgerritStephen Finucane proposed openstack/nova master: Further de-dupe os-vif VIF tests  https://review.openstack.org/63638417:19
*** wolverineav has quit IRC17:20
stephenfingibi: Trivial doc fix here, if you fancy taking a look https://review.openstack.org/#/c/636635/17:21
*** udesale has quit IRC17:23
*** jmlowe has quit IRC17:23
*** rpittau has quit IRC17:24
mriedemdansmith: right so i'm thinking leaving the nova-compute behavior the same, and probably move that check_availability_zone method to nova.virt.block_device where it's actually used17:26
mriedemand writing something new in the API17:26
dansmithmakes sense17:26
mriedemthat has the user-requested (or not) AZ context17:26
mriedemhaving said that, it's pot-pie-o-clock and i've got a physical this afternoon (first in 10+ years) so my attendance will be spotty17:26
mriedemoh and (don't tell laura) i have to get a valentine17:27
dansmithmriedem: you might be approaching that age where the doctor starts wanting to take your relationship to the next level17:27
mriedemi'm holding out for my 40s17:27
dansmithheh17:28
*** sapd1 has joined #openstack-nova17:29
*** itlinux has quit IRC17:30
*** _fragatina has joined #openstack-nova17:31
*** markvoelker has joined #openstack-nova17:32
tzumainnhi! quick question about the expected behavior with nova and ironic; I have four baremetal nodes, and I've created one server that runs on one of the nodes, but if I run "nova hypervisor-servers" against each node, that server shows up for every single node17:34
*** itlinux has joined #openstack-nova17:36
*** dtantsur is now known as dtantsur|afk17:37
*** sapd1 has quit IRC17:37
*** ociuhand_ has joined #openstack-nova17:39
*** itlinux_ has joined #openstack-nova17:39
melwitttzumainn: are you running one nova-compute service for all four baremetal nodes? if so, I think that might be why it shows up for every node17:40
melwittI'd have to look into the code to find how the hostname => server lookup is done to confirm17:41
*** itlinux has quit IRC17:41
melwittjroll ^17:41
tzumainnmelwitt, ah, yeah, I am17:41
jrollI'm not sure what "nova hypervisor-servers" is17:42
jrollbut yeah, probably only goes by compute service hostname, not node name17:42
*** ociuhandu has quit IRC17:42
melwittit's a command to get a list of servers given a hostname17:43
bauzasdansmith: stop me 1 sec, do we now need to set the host mappings when deploying with devstack ?17:46
dansmithbauzas: devstack should do that for you17:46
jrollmelwitt: ah, sounds like a bug in that API response17:47
bauzasdansmith: that was my assumption but I got a Host 'XXX' is not mapped to any cell17:47
* bauzas goes into looking n-api17:47
dansmithbauzas: on a vanilla devstack setup? shouldn't happen afaik17:47
*** ociuhand_ has quit IRC17:48
bauzastechnically, a reinstalled devstack but with a ./clean.sh and a reclone=true17:48
melwittjroll: I get what you mean now, might be returning host instead of node17:48
bauzasweirdo17:48
jrollmelwitt: yeah17:49
melwittthanks17:49
bauzasdansmith: nevermind, looks like a rabbit issue17:50
mriedembauzas: i'm pretty sure clean.sh isn't maintained17:50
mriedemi never use it17:50
bauzasmriedem: okay, how do we clean the data then ?17:51
bauzasrm -rf ?17:51
melwitttzumainn: ok, there might be a bug there. I can do some digging later to confirm and check if we have any open bugs or patches about it already17:51
mriedembauzas: if i need a clean slate i create a new devstack vm17:51
bauzaslucky you :p17:51
mriedemlemme guess, you're running devstack on baremetal with gpus17:52
bauzashere I'm talking of some internal server that I got by accident17:52
bauzasmriedem: yeah, I want to test the reshape series on real hardware17:52
bauzas-where we have RHEL7.5... -17:53
bauzasbut anyway, will continue to look17:53
tzumainnmelwitt, thanks! I didn't see an existing bug, but looking at https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/hypervisors.py#L167-L168 maybe what I'm seeing is intended behavior, and it's matching based on the hypervisor host_ip (which is the same for all four baremetal nodes)?17:53
mriedemtzumainn: it's a fuzzy search on hypervisor hostname https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L68017:55
mriedemfor ironic nodes that should actually be a uuid17:55
tzumainnmriedem, yep, that's what I'm using17:55
*** derekh has quit IRC17:55
mriedemand the API code is taking that compute node record and getting all instances on that compute_nodes.host17:56
mriedemwhich is the same for all of your nodes17:56
mriedemcompute_nodes.host == nova-compute service hostname == instances.host17:57
mriedemso it's working as expected, i.e. that API is not ironic-aware17:57
mriedemnor is most of the compute API17:57
mriedemhence MOGAN17:57
melwitttzumainn: yeah, internally in nova we have hypervisor "hostname" which is associated with the nova-compute service and we have "nodename" which is the ironic nodename. if you're not using ironic hostname == nodename. else, it will be different and to get the right answer (better answer?) for ironic, we'd need to do a lookup based on nodename, not "hostname"17:57
openstackgerritAdrian Chiris proposed openstack/nova master: Libvirt: do not set MAC when unplugging macvtap VF  https://review.openstack.org/62484217:57
openstackgerritAdrian Chiris proposed openstack/nova master: Add free for claimed, allocated devices  https://review.openstack.org/61612017:57
openstackgerritAdrian Chiris proposed openstack/nova master: Allow per-port modification of vnic_type and profile  https://review.openstack.org/60736517:57
openstackgerritAdrian Chiris proposed openstack/nova master: Add get_instance_pci_request_from_vif  https://review.openstack.org/61992917:57
openstackgerritAdrian Chiris proposed openstack/nova master: SR-IOV Live migration indirect port support  https://review.openstack.org/62011517:57
openstackgerritAdrian Chiris proposed openstack/nova master: libvirt: auto detach/attach sriov ports on migration  https://review.openstack.org/62958917:57
*** awalende has joined #openstack-nova17:58
tzumainnmriedem, melwitt gotcha - okay, thanks!17:59
mriedemtzumainn: without an api change the best we can do right now is document the wrinkle in the api reference18:00
tzumainnmriedem, would an API change be possible in the longer-term, or would you consider this a tragic wrinkle in how nova/ironic interact?18:01
mriedemtzumainn: i think it could be a possible change on a new microversion - if the resulting compute node we found is of type ironic, we have to lookup instances by nodename, not host18:01
melwitt++18:02
mriedemtzumainn: feel free to file a bug so it could at least be documented and then a blueprint, if someone wants to work on that, could be written based on the bug18:02
mriedemjaypipes might like that18:02
tzumainnmriedem, I'll do that - thanks very much!18:02
mriedemyw18:02
melwittwould also be good for a newer contributor bc it seems like it should be pretty small change18:03
openstackgerritJim Rollenhagen proposed openstack/nova master: ironic: check fresh data when sync_power_state doesn't line up  https://review.openstack.org/63669918:03
mriedemuntil they have to write functional api samples tests18:03
mriedemand their head explodes18:03
melwittah, I didn't think of that part18:04
*** markvoelker has quit IRC18:04
*** erlon has quit IRC18:05
* cdent lights the gabbi bat signal18:06
cdentbut that's not why I'm here, I have an actual question18:07
cdentit seems that most of the volume related methods in compute manager are synchronized on instance uuid, but detach volume is not. Is there a reason it is not?18:07
*** jmlowe has joined #openstack-nova18:08
*** itlinux_ has quit IRC18:08
sean-k-mooneyinstnace uuid or volume uuid18:09
*** agopi has quit IRC18:09
sean-k-mooneyif its instance could there be issues with multiattach volumes?18:10
sean-k-mooneycdent: also i dont know18:10
openstackgerritJim Rollenhagen proposed openstack/nova master: ironic: check fresh data when sync_power_state doesn't line up  https://review.openstack.org/63669918:10
cdentsean-k-mooney: yeah, I was confused by the choice of lock id18:11
cdent(too)18:11
sean-k-mooneycdent: i have not looked at the code but is your requestion related to the use of the uuid for synconisation or is detach ungaurded and your asking should it be18:14
*** hongbin has joined #openstack-nova18:15
cdentsean-k-mooney: I'm asking if it should be. In k8s-on-openstack environment there are issues happening with volume attachements getting into a weird state18:15
openstackgerritgaryk proposed openstack/nova master: Lock detach_volume  https://review.openstack.org/63670118:15
cdentthat's ^ the proposed fix18:15
sean-k-mooneycdent: lyarwood and  mdbooth are usually who i ask when it comes to volumn/storage things18:16
*** moshele has joined #openstack-nova18:17
cdentyeah, me too, but I'm hoping they've already stepped awy, keeping reasonable hours for this timezone, unlike you and me18:17
lyarwoodI'm around, just18:17
lyarwoodwe already lock within the libvirt driver, I assume this is for VMWare?18:18
cdentlyarwood: in the compute manager itself18:18
*** erlon has joined #openstack-nova18:18
cdentnot the virtdrivers18:18
cdentsee the patch18:18
lyarwoodcdent: right, and I assume the issues you're seeing are with the underlying detach/attach on the virt drivers?18:18
lyarwoodcdent: or is it with the attachments themselves in c-api?18:19
*** awalende has quit IRC18:19
*** sdake has joined #openstack-nova18:19
cdentlyarwood: I'm short on details. At the moment I'm just feeling around trying to gather info. I wasn't present at the discovery of the issue.18:19
lyarwoodkk np18:19
* lyarwood reads the bug18:19
openstackgerritgaryk proposed openstack/nova master: Lock detach_volume  https://review.openstack.org/63670118:20
*** awalende has joined #openstack-nova18:21
cdentlyarwood: as far as I can tell whatever the client is (some k8s thing) is going async on the volume delete and instance delete18:22
openstackgerritMatt Riedemann proposed openstack/nova master: api-ref: mark os-cells as deprecated  https://review.openstack.org/63670818:22
mriedemcdent: i seem to remember mdbooth having a change to lock detach_volume on instance.uuid18:23
lyarwoodcdent: yeah and as vmware isn't using os-brick you don't get any of the locking within it, adding a lock in the compute layer for detach seems reasonable tbh.18:23
*** wolverineav has joined #openstack-nova18:23
cdentmriedem: you mean in progress?18:23
* cdent looks18:23
mriedemmight be abandoned by now18:23
mriedemhttps://review.openstack.org/#/c/430213/18:24
mriedembingo18:24
*** betherly has joined #openstack-nova18:25
sean-k-mooneylyarwood: well as cdent pointed out we already lock in the compute manager for attach18:25
*** erlon_ has joined #openstack-nova18:25
mriedemcdent: there you go - want me to restore and you can rebase?18:25
lyarwoodsean-k-mooney: right, I'm agreeing18:25
sean-k-mooneylyarwood: yep i know.18:26
* mriedem notes he remembered a patch from Feb 7, 201718:26
sean-k-mooneymriedem: is there any reason to restor it vs https://review.openstack.org/#/c/636701/218:26
sean-k-mooneythe two are identical18:26
mriedemhow about because it came first?18:26
*** sapd1 has joined #openstack-nova18:27
cdentthanks mriedem I looked, but not that far back18:27
*** hongbin has quit IRC18:27
*** wolverineav has quit IRC18:27
sean-k-mooneyok i gues garyk could take over rebaseing the old patch18:27
*** hongbin has joined #openstack-nova18:27
* cdent worries about mriedem's brain18:27
*** erlon has quit IRC18:28
cdentsince it is in merge conflict, how about we just booth as a co-author on the clean one, and add the older bug too?18:29
*** ociuhandu has joined #openstack-nova18:29
mriedemthat works too...18:29
*** betherly has quit IRC18:29
mriedemi've duplicated the bug already18:29
*** wolverineav has joined #openstack-nova18:29
mriedemsec18:29
openstackgerritMatt Riedemann proposed openstack/nova master: Lock detach_volume  https://review.openstack.org/63670118:30
mriedemcomment inline18:32
cdentdamn dood, I think I'm just gonna let you do everything ;)18:32
mriedemi'm not sure we need to hold the lock while doing the db query18:32
cdentyeah18:32
openstackgerritMatt Riedemann proposed openstack/nova master: Lock detach_volume  https://review.openstack.org/63670118:33
*** ociuhandu has quit IRC18:33
openstackgerritMatt Riedemann proposed openstack/nova master: Lock detach_volume  https://review.openstack.org/63670118:34
mriedemthere god18:34
mriedemcoding in gerrit ui18:34
*** moshele has quit IRC18:34
cdentIs there any chance that get_by_volume_and_instance itself can race?18:35
openstackgerritVlad Gusev proposed openstack/nova stable/queens: Handle unicode characters in migration params  https://review.openstack.org/63667818:35
*** awalende has quit IRC18:35
mriedemworst case is you'd get a not found error from the db query18:37
*** sdake has quit IRC18:37
*** sunnaichuan has quit IRC18:38
*** awalende has joined #openstack-nova18:38
*** avolkov has quit IRC18:42
*** betherly has joined #openstack-nova18:47
*** tssurya has joined #openstack-nova18:49
*** agopi has joined #openstack-nova18:50
*** mriedem is now known as mriedem_away18:52
openstackgerritMerged openstack/nova master: Change nova-next job to run with python3  https://review.openstack.org/63473918:53
openstackgerritMerged openstack/nova master: Adding cross refs for config options in scheduler filter guide  https://review.openstack.org/63663518:53
*** tbachman has quit IRC18:58
openstackgerritMatt Riedemann proposed openstack/nova master: Remove cells v1 (for the most part) from the docs  https://review.openstack.org/63671819:01
*** markvoelker has joined #openstack-nova19:01
*** wolverineav has quit IRC19:04
*** wolverineav has joined #openstack-nova19:08
*** ociuhandu has joined #openstack-nova19:16
*** ociuhandu has quit IRC19:28
openstackgerritArtom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source  https://review.openstack.org/63482719:30
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim()  https://review.openstack.org/63566919:30
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources  https://review.openstack.org/63460619:30
openstackgerritArtom Lifshitz proposed openstack/nova master: LM: Make dest send NUMAMigrateData to the source  https://review.openstack.org/63482819:30
openstackgerritArtom Lifshitz proposed openstack/nova master: LM: update NUMA-related XML on the source  https://review.openstack.org/63522919:30
*** cdent has quit IRC19:31
*** eharney has quit IRC19:31
*** cdent has joined #openstack-nova19:32
*** erlon_ has quit IRC19:34
*** markvoelker has quit IRC19:34
*** awalende has quit IRC19:39
aspierswhat's the easiest way to set the logger to DEBUG level in functional tests?19:46
cdentif you're based off the nova base class, you probably already have it, but if not self.flag('debug', 'True') might do it. I can't remember for sure19:49
dansmiththere's an environment variable19:49
aspiers^^^ that's the answer I was hoping for ;-)19:50
melwittaspiers: OS_DEBUG=119:50
cdentthere's a fixture that will swallow a lot of stuff too19:50
dansmithyeah that^19:50
aspiersawesome, thanks! :-D19:50
melwitthave fun19:50
aspiersfound it in nova/tests/fixtures.py now19:50
* cdent can't decide if he misunderstood the question, or what19:51
melwittOS_DEBUG=1 tox -efunctional test_stuff19:51
melwittthat will show you log level debug and less during a test run19:51
aspierscdent: I could maybe answer that if I understood your answer, but I fear it's too sophisticated for me ;-)19:52
aspiersmelwitt: yep the env var is working great, although I'm running stestr directly in order to run individual groups of tests quicker19:52
melwittsmart19:52
aspiersgoing through tox is slow :-/19:52
cdentaspiers: I think I was answering "I want to manipulate log handling in a functional test I'm making" not "I want to see the debug logs"19:52
aspierscdent: ah yeah, thought it might be something like that :)19:52
aspierssorry for not being clear enough in the original q19:53
cdentno worries, worked out in the end19:53
aspiersmelwitt: shall I add OS_DEBUG to HACKING.rst?19:54
dansmithaspiers: wouldn't you rather keep it secret and seem smart the next time a newbie comes along?19:55
aspiersdansmith: tempting, but I think it's even more satisfying to be able to RTFH somebody by pointing them to docs I wrote myself ;-p19:56
dansmithaspiers: okay as long as shame is involved either way19:56
aspiersthat way I can appear smart *and* conceited19:57
aspiersexactly19:57
melwitthaha, nice19:57
aspiersafter all I only went into coding in order to be able to shame others :-p19:57
melwittthat's the right motivation19:58
aspiersglad we're on the same page XD19:58
melwittand yeah, it would be a good addition to HACKING.rst. and also our contributor doc/19:59
aspiersmelwitt: under https://docs.openstack.org/nova/latest/contributor/testing.html or a new page?20:00
aspierspresumably https://docs.openstack.org/nova/latest/contributor/index.html#testing isn't a good place for it20:00
melwittlemme see, I think I saw a place20:01
melwittI was looking at https://docs.openstack.org/nova/latest/contributor/development-environment.html#running-unit-tests but yeah, similarly doesn't have the same level of detail that HACKING.rst does20:02
aspiersI could just add a link from there to HACKING?20:03
melwittthat also looks really old :\20:03
*** wolverineav has quit IRC20:03
aspiersYeah20:03
*** panda is now known as panda|off20:04
*** _fragatina has quit IRC20:05
melwittaspiers: I think a link to HACKING for nova-specific test running notes would be a good fit there on that last page. because the existing doc is a link to the general openstack-wide test running guide20:07
aspiersmelwitt: yeah OK ... but how do I link to HACKING? since it's not published with the docs :-/20:08
aspierslink to cgit?20:08
aspiershttp://git.openstack.org/cgit/openstack/nova/tree/HACKING.rst doesn't render20:09
aspiershttps://github.com/openstack/nova/blob/master/HACKING.rst might be nicer20:09
aspiersbut kind of breaks the unspoken "avoid GitHub" convention20:10
openstackgerritAdam Spiers proposed openstack/nova master: Document how to make tests log at DEBUG level  https://review.openstack.org/63674420:10
aspiersI guess the link to HACKING can be done separately anyway20:10
melwittyeah, we can punt that idea since there's not an obvious counterpart in our docs20:11
openstackgerritAdam Spiers proposed openstack/nova master: Drop specific versions of openSUSE-based distributions  https://review.openstack.org/63674520:11
melwittI had been thinking about this I think, which is not nova docs (or even docs.o.o) https://wiki.openstack.org/wiki/Testr20:12
melwittfrom back in the day20:12
openstackgerritAdam Spiers proposed openstack/nova master: Document how to make tests log at DEBUG level  https://review.openstack.org/63674420:12
aspiersah yeah20:12
*** ralonsoh has quit IRC20:18
*** eharney has joined #openstack-nova20:25
*** jmlowe has quit IRC20:28
*** ociuhandu has joined #openstack-nova20:31
*** markvoelker has joined #openstack-nova20:31
*** medberry has joined #openstack-nova20:33
*** wolverineav has joined #openstack-nova20:33
*** wolverineav has quit IRC20:38
*** hongbin has quit IRC20:40
openstackgerritAdam Spiers proposed openstack/nova master: Document how to make tests log at DEBUG level  https://review.openstack.org/63674420:46
*** hongbin has joined #openstack-nova20:49
*** hongbin has quit IRC20:54
*** samc-bbc has quit IRC20:58
aspiersplease can someone explain how the functional tests mock responses from the placement API?21:04
*** itlinux has joined #openstack-nova21:04
*** ociuhandu has quit IRC21:04
aspiersI'm lost in a twisty maze of fixture code21:04
*** mriedem_away is now known as mriedem21:04
*** wolverineav has joined #openstack-nova21:04
mriedemi believe the PlacementFixture is pulled in from the placement repo21:05
mriedemvia tox siblings21:05
aspiersahhhhh21:05
*** markvoelker has quit IRC21:05
aspiersthat's why I couldn't find it21:05
aspiersthanks!21:05
mriedemhttps://github.com/openstack/placement/blob/master/placement/tests/functional/fixtures/placement.py#L2721:05
*** hongbin has joined #openstack-nova21:06
aspiersyeah, looks like the actual fixtures are in gabbits.py21:07
cdentaspiers: not mocks. 100% functionally real.21:07
*** xek has quit IRC21:07
mriedemaspiers: the gabbits are placement functional tests in placement21:07
aspierscdent: yeah, I see that now :)21:07
mriedemnova uses the PlacementFixture for stubbing out the nova.scheduler.client.report.SchedulerClient calls, which is essentially nova's placement client21:07
mriedembut yeah not really stubs21:07
mriedemhits the actual placement api and db21:07
aspiersmriedem: I'm trying to figure out why the HW_CPU_X86_VMX trait is missing in your traits functional test which I've rebased21:08
aspiersthe expectation that it's there is hard-coded, and I couldn't see why21:09
aspiersactually I still can't21:13
aspiersdoes it come from GranularFixture maybe?21:13
aspiershttps://review.openstack.org/#/c/538498/8/nova/tests/functional/test_servers.py@200721:14
aspiers^^^ this is what's confusing me21:14
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Microversion 2.68: Remove 'forced' live migrations, evacuations  https://review.openstack.org/63513121:15
*** jmlowe has joined #openstack-nova21:15
mriedemi'm not sure what you're hitting,21:18
mriedembut i see this in the test failure21:18
mriedemAttributeError: <class 'nova.virt.fake.SmallFakeDriver'> does not have the attribute 'get_traits'21:18
mriedemthat method used to exist in the base class https://review.openstack.org/#/c/538498/7/nova/virt/driver.py@90521:19
mriedembut no longer does21:19
mriedemHW_CPU_X86_VMX is a standard trait in the os-traits library,21:19
mriedemthe placement api puts that into the traits table on startup21:19
mriedemdansmith: this had to be tweaked but do you want to re-approve https://review.openstack.org/#/c/551302/21:21
dansmithack21:22
*** tbachman has joined #openstack-nova21:29
*** whoami-rajat has quit IRC21:32
*** awaugama has quit IRC21:35
*** wolverineav has quit IRC21:37
melwittmriedem: commented on https://review.openstack.org/636701, it seems to me we do need to lock around the bdm lookup, unless I'm missing something21:39
*** _fragatina has joined #openstack-nova21:45
*** itlinux has quit IRC21:58
*** rchurch has joined #openstack-nova22:00
*** rchurch_ has quit IRC22:01
*** markvoelker has joined #openstack-nova22:02
*** wolverineav has joined #openstack-nova22:03
*** tssurya has quit IRC22:07
*** wolverineav has quit IRC22:08
openstackgerritMatt Riedemann proposed openstack/nova master: Lock detach_volume  https://review.openstack.org/63670122:14
mriedemchanged22:14
aspiersmriedem: ignore the get_traits failure, I already fixed it but didn't push a new patchset yet22:17
aspiersI guess I can push a new one and you'll see the exact failure22:18
openstackgerritAdam Spiers proposed openstack/nova master: WIP: Convert driver supported capabilities to compute node provider traits  https://review.openstack.org/53849822:20
*** openstackgerrit has quit IRC22:22
*** mdbooth_ has joined #openstack-nova22:31
*** eandersson has quit IRC22:33
*** markvoelker has quit IRC22:34
*** mdbooth has quit IRC22:34
aspiersmriedem: https://gist.github.com/aspiers/d64eff8a1d5594ad9763ebbd192d36b122:34
*** agopi_ has joined #openstack-nova22:35
*** gouthamr has quit IRC22:35
*** gouthamr has joined #openstack-nova22:36
*** cburgess has quit IRC22:36
*** agopi has quit IRC22:37
*** cburgess has joined #openstack-nova22:38
*** sapd1 has quit IRC22:40
*** agopi__ has joined #openstack-nova22:40
mriedemyeah it's failing because you removed the get_traits mock which was returning ['CUSTOM_FOO', 'HW_CPU_X86_VMX']22:42
aspiersDOH22:42
mriedemfrom the drier22:42
mriedem*driver22:42
aspierslol22:42
*** agopi__ is now known as agopi22:42
aspiersmissed that22:42
*** agopi_ has quit IRC22:42
mriedemyou essentially need to stub the driver.update_provider_tree method with a fake so you can intercept the provider tree and shove your traits into it22:43
mriedemi think22:43
aspiersyeah sure, got it now22:43
aspiersI was assuming it was hardcoded somewhere - was searching the fixtures high and low X-D22:44
mriedemor just change FakeDriver.update_provider_tree to always return those traits but that could affect other tests22:44
mriedemi'd do something like,22:44
mriedemoriginal_upt = self.compute.driver.update_provider_tree22:45
mriedemdef fake_upt(...):22:45
mriedemoriginal_upt(...)22:45
mriedem# inject the traits i want in the tree object22:45
mriedemreturn that thing22:45
mriedemself.stub_out('nova.virt.fake.FakeDriver.update_provider_tree', fake_upt)22:46
mriedemi think that should do it22:46
mriedemif you need that to be temporary, you can use self.useFixture(fixtures.MonkeyPatch(old, new)) directly as a context manager22:47
aspierscool, thanks!22:48
*** tesseract has quit IRC22:49
*** sapd1 has joined #openstack-nova22:55
*** wolverineav has joined #openstack-nova22:59
*** eandersson has joined #openstack-nova22:59
*** david-lyle has quit IRC23:02
*** dklyle has joined #openstack-nova23:02
*** tkajinam has joined #openstack-nova23:05
*** tbachman has quit IRC23:08
*** tbachman has joined #openstack-nova23:09
*** mrjk_ has quit IRC23:10
*** slaweq has quit IRC23:11
*** tbachman_ has joined #openstack-nova23:15
*** sapd1 has quit IRC23:16
*** tbachman has quit IRC23:17
*** tbachman_ is now known as tbachman23:17
*** mlavalle has quit IRC23:17
*** wolverineav has quit IRC23:30
*** wolverineav has joined #openstack-nova23:30
*** markvoelker has joined #openstack-nova23:31
*** openstackgerrit has joined #openstack-nova23:32
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Add support for microversion 2.69  https://review.openstack.org/63677923:32
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Add support for microversion 2.69 - expose device tags  https://review.openstack.org/63677923:36
openstackgerritArtom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods  https://review.openstack.org/63621023:40
openstackgerritArtom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination  https://review.openstack.org/63460523:40
openstackgerritArtom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source  https://review.openstack.org/63482723:40
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim()  https://review.openstack.org/63566923:40
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources  https://review.openstack.org/63460623:40
openstackgerritArtom Lifshitz proposed openstack/nova master: LM: Make dest send NUMAMigrateData to the source  https://review.openstack.org/63482823:40
openstackgerritArtom Lifshitz proposed openstack/nova master: LM: update NUMA-related XML on the source  https://review.openstack.org/63522923:40
mriedemefried: question in https://review.openstack.org/#/c/636378/23:47
mriedemoh gdi23:48
mriedemnow i see the follow up23:48
openstackgerritMerged openstack/nova master: Avoid redundant initialize_connection on source post live migration  https://review.openstack.org/55130223:48
efriedmriedem: Yeah, I considered actually referring to the fup in the commit message, but stephenfin's suggestion was intended to make it stand on its own, so...23:51
mriedemwhich i guess is this https://review.openstack.org/#/c/582392/2/tox.ini@a2623:52
*** lbragstad has quit IRC23:52
mriedemnow to see how many people complain about unit tests not working tomorrow23:53
efriedheh23:54
efriedI did happen to notice that bionic doesn't have tox 3.x, I had to install it by hand.23:54
mriedemi'm still on xenial23:55
mriedemi get tox from pypi anyway23:55
*** agopi is now known as agopi|out23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!