Friday, 2019-04-12

*** owalsh_ has joined #openstack-nova00:01
*** owalsh has quit IRC00:02
*** owalsh has joined #openstack-nova00:05
*** owalsh_ has quit IRC00:06
*** igordc has quit IRC00:07
*** mchlumsky_ has quit IRC00:08
*** mchlumsky_ has joined #openstack-nova00:09
*** owalsh_ has joined #openstack-nova00:09
*** artom has joined #openstack-nova00:09
*** owalsh has quit IRC00:10
*** owalsh has joined #openstack-nova00:13
*** hamzy has joined #openstack-nova00:14
*** owalsh_ has quit IRC00:14
*** gyee has quit IRC00:15
openstackgerritTony Breeds proposed openstack/nova master: Uncap jsonschema  https://review.openstack.org/65194300:16
openstackgerritTony Breeds proposed openstack/nova master: Add pep8 to test-requirements  https://review.openstack.org/65194400:16
*** tetsuro has joined #openstack-nova00:17
tonybefried, stephenfin: ^^ the first is unrelated to your hacking work but is required00:17
tonybefried, stephenfin: the second just makes it explict and clear why we're adding pep800:17
*** owalsh_ has joined #openstack-nova00:18
*** mlavalle has quit IRC00:18
*** owalsh has quit IRC00:19
openstackgerritMatt Riedemann proposed openstack/nova master: Add --instance option to heal_allocations  https://review.openstack.org/65194500:20
mriedemceryx: eandersson: ^00:20
mriedemif you want to just work on that one problem instance00:20
mriedemzzzeek: we figured it out00:20
*** owalsh has joined #openstack-nova00:21
*** lbragstad has quit IRC00:23
*** owalsh_ has quit IRC00:23
*** owalsh_ has joined #openstack-nova00:26
*** owalsh has quit IRC00:27
*** owalsh has joined #openstack-nova00:31
*** amodi has quit IRC00:32
*** owalsh_ has quit IRC00:32
*** owalsh_ has joined #openstack-nova00:35
*** owalsh has quit IRC00:36
*** yedongcan has joined #openstack-nova00:37
*** owalsh has joined #openstack-nova00:39
*** owalsh_ has quit IRC00:40
openstackgerritMerged openstack/python-novaclient stable/stein: Revert "Fix crashing console-log"  https://review.openstack.org/65036300:40
mriedemefried: i just realized that this is always True https://github.com/openstack/nova/blob/master/nova/scheduler/utils.py#L40300:41
mriedembecause Flavor.extra_specs is always loaded00:41
mriedemit doesn't mean extra_specs is not empty00:41
mriedemand that's extremely important because for a volume-backed server, we could otherwise be trying to PUT allocations with DISK_GB=000:42
mriedemwhich is a no-no00:42
*** owalsh_ has joined #openstack-nova00:43
openstackgerritmelanie witt proposed openstack/nova master: Warn for duplicate host mappings during discover_hosts  https://review.openstack.org/65194700:43
*** owalsh has quit IRC00:44
openstackgerritMatt Riedemann proposed openstack/nova master: Add BFV wrinkle to TestNovaManagePlacementHealAllocations  https://review.openstack.org/65194800:45
*** owalsh has joined #openstack-nova00:47
openstackgerritMatt Riedemann proposed openstack/nova master: Add BFV wrinkle to TestNovaManagePlacementHealAllocations  https://review.openstack.org/65194800:47
*** owalsh_ has quit IRC00:48
eanderssonWe removed the dups in the allocation table and re-ran the script and it worked00:49
eanderssonbut that would have been neat00:51
*** owalsh_ has joined #openstack-nova00:51
mriedemeandersson: ok so now you just have the 2 allocations records for the one instance consumer against the same compute node resource provider that you'd expect for that instance?00:51
eanderssonYea00:52
mriedemi.e. instance.node matches the compute_nodes table record matches the rp00:52
mriedemok cool00:52
mriedemso i'm still guessing the multiple allocations against multiple providers was a result of failed migrations where the allocations - created by the scheduler - weren't cleaned up properly00:52
*** owalsh has quit IRC00:52
eanderssonand the actual allocation was never added I think00:52
eanderssonSo it failed to clean up, but also failed to add the last allocation00:53
mriedemeandersson: i.e. https://review.openstack.org/#/c/647566/00:53
*** brinzhang has joined #openstack-nova00:53
mriedemi'm not sure what you mean by 'last allocation'00:53
eanderssonas the compute it ended up on was not actually in the allocation table00:53
eanderssonbut my colleague can provide more detailed information tomorrow00:54
mriedemunless i'm missing something i don't think that can happen00:54
mriedemthe scheduler will create the allocations - if that fails, you'd get novalidhost most likely00:54
mriedemthat or we'd be rescheduling to another host00:54
mriedembut we could have definitely been leaking allocations somehow when we fail ala https://bugs.launchpad.net/nova/+bug/182159400:55
openstackLaunchpad bug 1821594 in OpenStack Compute (nova) stein "Error in confirm_migration leaves stale allocations and 'confirming' migration state" [Medium,In progress] - Assigned to Matt Riedemann (mriedem)00:55
mriedemwould be good to nail down if you could figure out what actually failed though to see if we have another leak00:55
mriedemi still need to get the fix for that backported to rocky00:55
mriedemgotta go - time to blow snow00:57
*** mriedem is now known as mriedem_afk00:57
*** owalsh has joined #openstack-nova00:57
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (8)  https://review.openstack.org/57531100:57
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (9)  https://review.openstack.org/57558100:58
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (10)  https://review.openstack.org/57601700:58
*** owalsh_ has quit IRC00:58
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (11)  https://review.openstack.org/57601800:58
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (12)  https://review.openstack.org/57601900:59
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13)  https://review.openstack.org/57602000:59
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14)  https://review.openstack.org/57602701:00
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15)  https://review.openstack.org/57603101:01
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16)  https://review.openstack.org/57629901:01
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17)  https://review.openstack.org/57634401:01
eanderssonmriedem_afk, you are right - I assumed the compute service id in cli would match resource_provider_id01:02
eanderssonSo the above bug would explain it01:02
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18)  https://review.openstack.org/57667301:02
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19)  https://review.openstack.org/57667601:03
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20)  https://review.openstack.org/57668901:03
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21)  https://review.openstack.org/57670901:04
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22)  https://review.openstack.org/57671201:04
openstackgerritTakashi NATSUME proposed openstack/nova master: Add a live migration regression test  https://review.openstack.org/64120001:05
*** ricolin has joined #openstack-nova01:05
openstackgerritTakashi NATSUME proposed openstack/nova master: Add minimum value in max_concurrent_live_migrations  https://review.openstack.org/64830201:05
openstackgerritmelanie witt proposed openstack/nova master: Warn for duplicate host mappings during discover_hosts  https://review.openstack.org/65194701:06
*** hongbin has joined #openstack-nova01:08
*** whoami-rajat has joined #openstack-nova01:09
openstackgerritMerged openstack/os-vif master: docs: Use sphinx.ext.autodoc for VIF types  https://review.openstack.org/63840401:14
openstackgerritMerged openstack/os-vif master: docs: Use sphinx.ext.autodoc for profile, datapath offload types  https://review.openstack.org/63840501:23
*** owalsh_ has joined #openstack-nova01:29
*** owalsh has quit IRC01:30
*** sapd1_x has joined #openstack-nova01:31
*** awalende has joined #openstack-nova01:36
*** awalende has quit IRC01:41
*** mriedem_afk is now known as mriedem01:42
mriedemeandersson: ah yeah the _id is the table primary key id, it's the uuid you care about01:42
tetsuromriedem: would you mind if I use ListUUIDField for in_tree in https://review.openstack.org/#/c/649534/6/nova/objects/request_spec.py01:45
tetsuroinstead of UUIDField01:45
mriedemtetsuro: eh01:48
mriedemi don't love trying to future-proof that01:48
mriedemtetsuro: for what use case? user specifies a host and there are >1 nodes and you want to filter allocation candidates by all nodes?01:48
mriedemthat would be 1000 for CERN01:49
mriedem*could01:49
tetsuroNot sure what happens on the performance01:51
mriedemi tend to think that's not a case we need to care about too much right now, until someone says they really need it01:52
tetsuroBut I remember Surya said in the spec that he's not curious to have that for ironic cases01:52
mriedemi think in most cases that someone is forcing a server to a specific baremetal node, they are specifying the node01:52
mriedemright01:53
openstackgerritBoxiang Zhu proposed openstack/nova-specs master: Add host and hypervisor_hostname flag to create server  https://review.openstack.org/64545801:53
mriedemthe only times i've heard of someone use the JsonFilter query hint for example is to force to a specific baremetal node01:53
mriedemvia the hypervisor_hostname parameter01:53
tetsuroin which case the found node ==101:54
yaawangmriedem: Hi, I have posted a new spec https://review.openstack.org/#/c/651681/. This spec will use the flavor extra spec and image metadata to datermine whether to use auto converge and post copy.01:54
tetsuromriedem: okay, I'll leave this field as is i.e. UUIDField instead of ListUUIDField.01:55
mriedemyaawang: yeah i saw but haven't dug into it yet. i agree that might be a better option, but there are cases to consider like what happens if the image or extra spec says to use auto_converge or post-copy but the host is unable to provide those? do we fail the server create? do we ignore?01:56
mriedemfor SLA reasons i think we'd have to fail01:56
mriedemtetsuro: ack01:56
*** markvoelker has joined #openstack-nova02:00
mriedemyaawang: ok i left some thoughts on the spec,02:00
mriedembut traits might be the way to go here02:01
*** tiendc has joined #openstack-nova02:02
yaawangYes, we need to check libvirt supports auto converge or post copy before live migration. I think end user prefer to use auto converge or post copy during live migration, but the end user hopes to be able to successfully carry out live migartion.02:04
yaawangTrait is a good idea, or schedule instances to the host which support auto converge or post copy.02:06
mriedemright, so i linked to the tpm spec because that case is very similar02:07
mriedemso read up on that and see if it could fit the same issue you're trying to solve02:07
*** mriedem has quit IRC02:29
*** boxiang has quit IRC02:31
*** markvoelker has quit IRC02:35
openstackgerritMerged openstack/nova stable/rocky: Move create of ComputeAPI object in websocketproxy  https://review.openstack.org/64937502:51
openstackgerritTetsuro Nakamura proposed openstack/nova master: Add in_tree field to RequestGroup object  https://review.openstack.org/64953402:52
openstackgerritTetsuro Nakamura proposed openstack/nova master: Add get_compute_nodes_by_host_or_node()  https://review.openstack.org/65087702:52
openstackgerritTetsuro Nakamura proposed openstack/nova master: Pass target host to RequestGroup.in_tree  https://review.openstack.org/65087802:52
openstackgerritTetsuro Nakamura proposed openstack/nova master: Query `in_tree` to placement  https://review.openstack.org/64953502:52
*** irclogbot_0 has quit IRC03:01
*** irclogbot_2 has joined #openstack-nova03:02
*** alex_xu has quit IRC03:05
*** alex_xu has joined #openstack-nova03:10
*** ak92514 has quit IRC03:14
*** rcernin has quit IRC03:15
*** psachin has joined #openstack-nova03:20
*** nicolasbock has quit IRC03:25
*** boxiang has joined #openstack-nova03:26
yaawangmriemem: Okay, i'll update the spec.03:26
*** rcernin has joined #openstack-nova03:27
*** markvoelker has joined #openstack-nova03:32
*** owalsh has joined #openstack-nova03:40
*** owalsh_ has quit IRC03:40
*** efried has quit IRC03:43
*** efried has joined #openstack-nova03:44
*** owalsh_ has joined #openstack-nova03:48
*** owalsh has quit IRC03:49
*** belmoreira has joined #openstack-nova03:57
*** rcernin has quit IRC03:57
*** rcernin has joined #openstack-nova04:01
*** imacdonn_ has quit IRC04:04
*** markvoelker has quit IRC04:04
*** imacdonn_ has joined #openstack-nova04:04
*** udesale has joined #openstack-nova04:13
*** hongbin has quit IRC04:16
*** owalsh has joined #openstack-nova04:21
*** owalsh_ has quit IRC04:22
*** _d34dh0r53_ has joined #openstack-nova04:24
*** belmoreira has quit IRC04:32
*** sapd1_x has quit IRC04:59
*** tkajinam has quit IRC05:00
*** sapd1_x has joined #openstack-nova05:01
*** markvoelker has joined #openstack-nova05:02
*** tkajinam has joined #openstack-nova05:07
*** sidx64 has joined #openstack-nova05:21
openstackgerritBoxiang Zhu proposed openstack/nova master: Make evacuation respects anti-affinity rule  https://review.openstack.org/64996305:24
openstackgerritBoxiang Zhu proposed openstack/nova master: [WIP] Fix live migration break group policy simultaneously  https://review.openstack.org/65196905:24
*** Luzi has joined #openstack-nova05:24
*** cfriesen has quit IRC05:30
*** markvoelker has quit IRC05:34
*** cfriesen has joined #openstack-nova05:37
*** ricolin has quit IRC05:39
*** cfriesen has quit IRC05:42
openstackgerritMerged openstack/nova stable/rocky: Do not persist RequestSpec.ignore_hosts  https://review.openstack.org/64932605:48
*** psachin has quit IRC05:49
*** sapd1_x has quit IRC05:52
*** baclawski has joined #openstack-nova05:57
*** baclawski has quit IRC06:02
*** lpetrut has joined #openstack-nova06:02
*** psachin has joined #openstack-nova06:05
*** sapd1_x has joined #openstack-nova06:07
boxianghi, does anyone know when/whether the requested_destination of request_spec will be save into db(persistent)? After I cold migrate the instance, I find it's null. Need help : )06:13
*** sapd1_x has quit IRC06:21
*** sapd1_x has joined #openstack-nova06:28
*** markvoelker has joined #openstack-nova06:31
*** bhagyashris has joined #openstack-nova06:35
*** sidx64_ has joined #openstack-nova06:39
*** sidx64 has quit IRC06:39
*** pcaruana has joined #openstack-nova06:41
*** ivve has joined #openstack-nova06:42
*** sidx64_ has quit IRC06:44
*** sapd1_x has quit IRC06:51
*** sapd1_x has joined #openstack-nova06:55
openstackgerritMichael Still proposed openstack/nova master: Privsepify ipv4 forwarding enablement.  https://review.openstack.org/63543106:58
openstackgerritMichael Still proposed openstack/nova master: Remove unused FP device creation and deletion methods.  https://review.openstack.org/63543306:58
openstackgerritMichael Still proposed openstack/nova master: Privsep the ebtables modification code.  https://review.openstack.org/63543506:58
openstackgerritMichael Still proposed openstack/nova master: Move adding vlans to interfaces to privsep.  https://review.openstack.org/63543606:58
openstackgerritMichael Still proposed openstack/nova master: Move iptables rule fetching and setting to privsep.  https://review.openstack.org/63650806:58
openstackgerritMichael Still proposed openstack/nova master: Move dnsmasq restarts to privsep.  https://review.openstack.org/63928006:58
openstackgerritMichael Still proposed openstack/nova master: Move router advertisement daemon restarts to privsep.  https://review.openstack.org/63928106:58
openstackgerritMichael Still proposed openstack/nova master: Move calls to ovs-vsctl to privsep.  https://review.openstack.org/63928206:58
openstackgerritMichael Still proposed openstack/nova master: Move setting of device trust to privsep.  https://review.openstack.org/63928306:58
openstackgerritMichael Still proposed openstack/nova master: Move final bridge commands to privsep.  https://review.openstack.org/63958006:58
openstackgerritMichael Still proposed openstack/nova master: Cleanup the _execute shim in nova/network.  https://review.openstack.org/63958106:58
openstackgerritMichael Still proposed openstack/nova master: We no longer need rootwrap.  https://review.openstack.org/55443806:58
openstackgerritMichael Still proposed openstack/nova master: Cleanup no longer required filters and add a release note.  https://review.openstack.org/63982606:58
*** luksky has joined #openstack-nova06:58
*** slaweq has joined #openstack-nova06:58
*** tiendc has quit IRC07:00
openstackgerritMichael Still proposed openstack/nova master: Remove fake_libvirt_utils from connection tests.  https://review.openstack.org/64255707:04
openstackgerritMichael Still proposed openstack/nova master: Remove fake_libvirt_utils from snapshot tests.  https://review.openstack.org/64255807:04
openstackgerritMichael Still proposed openstack/nova master: Remove fake_libvirt_utils from virt driver tests.  https://review.openstack.org/64389407:04
openstackgerritMichael Still proposed openstack/nova master: Remove fake_libvirt_utils from libvirt imagebackend tests.  https://review.openstack.org/64389507:04
openstackgerritMichael Still proposed openstack/nova master: Remove remaining vestiges of fake_libvirt_utils from unit tests.  https://review.openstack.org/64389607:04
openstackgerritMichael Still proposed openstack/nova master: Remove fake_libvirt_utils users in functional testing.  https://review.openstack.org/64479307:04
*** hamzy has quit IRC07:05
*** markvoelker has quit IRC07:05
*** hamzy has joined #openstack-nova07:05
*** whoami-rajat has quit IRC07:05
*** whoami-rajat has joined #openstack-nova07:06
*** mnaser has quit IRC07:06
*** mnaser has joined #openstack-nova07:06
openstackgerritBoxiang Zhu proposed openstack/nova master: Make evacuation respects anti-affinity rule  https://review.openstack.org/64996307:06
*** egonzalez has quit IRC07:07
*** egonzalez has joined #openstack-nova07:08
*** rpittau|afk is now known as rpittau07:08
alex_xukashyap: I replied https://review.openstack.org/#/c/642030/, let me know whether i understand correctly07:09
alex_xuboxiang: I guess it is here https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L36107:13
*** awalende has joined #openstack-nova07:15
*** baclawski has joined #openstack-nova07:16
kashyapalex_xu: Will do07:17
alex_xukashyap: thanks07:17
kashyapalex_xu: On your question ... you're right: we're mentioning it as an "extra" flag because a certain named CPU model may not contain some important CPU flags, e.g IvyBridge does not contain 'pcid'.07:20
*** sidx64 has joined #openstack-nova07:20
*** sapd1_x has quit IRC07:20
kashyap(But the *host* itself is capable of it, and QEMU can recognize it.  So we supply it explicitly)07:21
kashyapI will answer on the review, so you can respond at your own pace :-)07:22
*** mdbooth has joined #openstack-nova07:22
*** baclawski has quit IRC07:23
*** mdbooth_ has quit IRC07:25
alex_xukashyap: for the use-case of config extra_flag, we want the IvyBridge + pcid, not just ensure pcid in the model of IvyBridge, right?07:27
kashyapExactly.  We're specifying '+ pcid' because it is not included in the libvirt's "idea of IvyBridge", but host is capable of PCID07:28
*** boxiang has quit IRC07:29
alex_xukashyap: also, there won't have any model incompatible with pcid?07:29
kashyapalex_xu: To refresh your memory:07:29
*** boxiang has joined #openstack-nova07:30
kashyap(1) Intel Haswell, Broadwell, Skylake libvirt/QEMU models include 'pcid' by default.07:30
*** sapd1_x has joined #openstack-nova07:30
kashyap(2) The libvirt / QEMU CPU models "Nehalem", "Westmere", "SandyBridge", and "IvyBridge" will _not_ expose the 'PCID' capability by default, even if the host CPUs by the same name include it.  I.e. 'PCID' needs to be explicitly specified when using the said virtual CPU models.07:30
alex_xukashyap: nice refresh, that is interesting :)07:31
kashyapalex_xu: Okay, was wondering if what I wrote above was useful :-)07:31
alex_xukashyap: so I guess the problem is that paragraph makes me confusing on what you want to check.07:32
kashyapalex_xu: I wrote that documentation in Nova's documentation.07:32
kashyap(And here too: https://kashyapc.fedorapeople.org/Reducing-OpenStack-Guest-Perf-Impact-from-Meltdown.txt)07:32
kashyapalex_xu: Actually, I didn't write that original paragraph in the spec, Jack Ding wrote it07:32
kashyapalex_xu: Let me think a bit more on to rephrase it to be more clear.07:32
alex_xukashyap: ah, sorry, I thought it was removed from the original spec07:33
kashyapIt was not07:33
alex_xukashyap: yea, thanks, good to be clear what is checked we want07:33
kashyapalex_xu: As a general point: for most operators, using 'host-model' should suffice, because that will auto-add necessary CPU flags for Meltdown/Spectre07:36
*** sapd1_x has quit IRC07:42
*** tosky has joined #openstack-nova07:43
*** phasespace has joined #openstack-nova07:46
alex_xukashyap: thanks, good to know07:47
*** tssurya has joined #openstack-nova07:49
*** ccamacho has joined #openstack-nova07:53
*** sidx64 has quit IRC07:53
*** ivve has quit IRC07:56
*** ttsiouts has joined #openstack-nova07:58
*** takashin has left #openstack-nova08:01
*** ricolin has joined #openstack-nova08:01
*** dtantsur|afk is now known as dtantsur08:01
*** markvoelker has joined #openstack-nova08:02
*** awalende has quit IRC08:05
*** awalende has joined #openstack-nova08:05
*** awalende has quit IRC08:06
*** awalende has joined #openstack-nova08:06
*** ivve has joined #openstack-nova08:11
*** luksky has quit IRC08:12
*** ttsiouts has quit IRC08:12
*** ttsiouts has joined #openstack-nova08:13
openstackgerritTetsuro Nakamura proposed openstack/nova master: Add get_compute_nodes_by_host_or_node()  https://review.openstack.org/65087708:20
openstackgerritTetsuro Nakamura proposed openstack/nova master: Pass target host to RequestGroup.in_tree  https://review.openstack.org/65087808:20
openstackgerritTetsuro Nakamura proposed openstack/nova master: Query `in_tree` to placement  https://review.openstack.org/64953508:20
openstackgerritBoxiang Zhu proposed openstack/nova-specs master: Add host and hypervisor_hostname flag to create server  https://review.openstack.org/64545808:23
*** awalende has quit IRC08:27
*** awalende has joined #openstack-nova08:28
*** markvoelker has quit IRC08:34
*** derekh has joined #openstack-nova08:39
*** ralonsoh has joined #openstack-nova08:40
*** tetsuro has quit IRC08:45
openstackgerritBoxiang Zhu proposed openstack/nova master: Make evacuation respects anti-affinity rule  https://review.openstack.org/64996308:48
*** sidx64 has joined #openstack-nova08:59
*** dtantsur is now known as dtantsur|brb08:59
*** ivve has quit IRC09:10
*** luksky has joined #openstack-nova09:13
*** sidx64 has quit IRC09:13
*** janki has joined #openstack-nova09:15
*** sidx64 has joined #openstack-nova09:16
*** ivve has joined #openstack-nova09:21
openstackgerritBoxiang Zhu proposed openstack/nova-specs master: Add host and hypervisor_hostname flag to create server  https://review.openstack.org/64545809:22
alex_xuboxiang: I think we can persistent the destination_host09:25
alex_xuboxiang: since we reset it when do other migration, this is the rebuild https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L103409:26
kashyapgibi: lyarwood: I hope I answered your questions on the Secure Boot spec: https://review.openstack.org/#/c/506720/09:27
kashyapThanks for the review and questions!09:27
boxiangalex_xu: let me take a look09:27
openstackgerritya.wang proposed openstack/nova-specs master: Expose auto converge and post copy  https://review.openstack.org/65168109:29
*** rcernin has quit IRC09:30
*** markvoelker has joined #openstack-nova09:31
boxiangalex_xu:Yeah, I see other migration will call reset_forced_destinations. I will make some changes later.09:33
*** ricolin has quit IRC09:33
alex_xuboxiang: cool09:36
alex_xuboxiang: I think I can +2 after you update09:36
*** davidsha has joined #openstack-nova09:36
openstackgerritStephen Finucane proposed openstack/nova master: Bump to hacking 1.1.0  https://review.openstack.org/65155309:36
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve E731 issues  https://review.openstack.org/65155409:37
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve W503 issues  https://review.openstack.org/65155509:37
*** cdent has joined #openstack-nova09:42
*** bhagyashris has quit IRC09:53
*** sidx64 has quit IRC09:54
*** sidx64 has joined #openstack-nova09:56
*** markvoelker has quit IRC10:04
*** sidx64 has quit IRC10:09
*** sidx64 has joined #openstack-nova10:10
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/rocky: Add test for console-log and docs for bug 1746534  https://review.openstack.org/65203310:28
openstackbug 1746534 in python-novaclient "encoding error when doing console-log" [High,Fix released] https://launchpad.net/bugs/1746534 - Assigned to Thomas Goirand (thomas-goirand)10:28
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/rocky: Add test for console-log and docs for bug 1746534  https://review.openstack.org/65203310:28
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/rocky: Add test for console-log and docs for bug 1746534  https://review.openstack.org/65203310:29
openstackbug 1746534 in python-novaclient "encoding error when doing console-log" [High,Fix released] https://launchpad.net/bugs/1746534 - Assigned to Thomas Goirand (thomas-goirand)10:29
*** pcaruana has quit IRC10:31
openstackgerritMerged openstack/nova-specs master: add spec for image metadata prefiltering  https://review.openstack.org/64757810:33
*** nicolasbock has joined #openstack-nova10:43
*** ociuhandu has joined #openstack-nova10:53
*** ociuhandu has quit IRC10:53
*** tbachman has quit IRC10:53
*** markvoelker has joined #openstack-nova11:02
*** udesale has quit IRC11:05
*** sidx64 has quit IRC11:08
*** bbowen has joined #openstack-nova11:18
sean-k-mooneystephenfin: jaypipes do ye have time to review the sriov migration patches again. https://review.openstack.org/#/c/620115/34 https://review.openstack.org/#/c/629589/ jay the first one has not change since you reviewed it last and the second is now rebased thanks to adrianc11:20
*** ivve has quit IRC11:21
*** mvkr has joined #openstack-nova11:23
*** ivve has joined #openstack-nova11:30
*** maciejjozefczyk has left #openstack-nova11:31
*** jaosorior has quit IRC11:33
*** markvoelker has quit IRC11:35
*** pcaruana has joined #openstack-nova11:36
*** Luzi has quit IRC11:39
*** sidx64 has joined #openstack-nova11:42
*** sidx64 has quit IRC11:43
*** brinzhang has quit IRC11:56
*** sidx64 has joined #openstack-nova12:00
*** tbachman has joined #openstack-nova12:05
*** sidx64 has quit IRC12:07
*** psachin has quit IRC12:11
*** sidx64 has joined #openstack-nova12:13
*** ricolin has joined #openstack-nova12:25
*** ricolin has quit IRC12:29
*** sidx64 has quit IRC12:45
*** NostawRm has quit IRC12:46
jaypipessean-k-mooney: yes, will do this morning.12:46
sean-k-mooney:)12:47
sean-k-mooneyi had addressed you comments in a previous revision and adrianc rebased it on his latest patch so that should be the only delta form what you saw before. if there is anythin else a miss let me know :)12:48
*** dtantsur|brb is now known as dtantsur12:51
*** awalende has quit IRC12:52
*** awalende has joined #openstack-nova12:52
*** mriedem has joined #openstack-nova12:55
*** awalende has quit IRC12:57
*** awalende has joined #openstack-nova12:57
openstackgerritMerged openstack/nova stable/stein: Fix incomplete instance data returned after build failure  https://review.openstack.org/64790213:01
*** awalende has quit IRC13:02
stephenfinCan someone remind me: do we still translate exceptions?13:02
sean-k-mooneyno not anymore13:03
sean-k-mooneyat least i dont think we do13:03
*** altlogbot_2 has joined #openstack-nova13:04
sean-k-mooneystephenfin: looking at https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html#choosing-a-marker-function13:05
sean-k-mooneymaybe we do i had tought we decided to stop translating them however13:05
*** dave-mccowan has joined #openstack-nova13:05
sean-k-mooneystephenfin: ah https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html#log-translation13:05
sean-k-mooneyso ya as of pike we dont translate any logs13:06
jaypipesstephenfin, sean-k-mooney: not sure how I failed to notice this before, but the compute API's GET /servers/{id}/detail endpoint does not return the DNS name of the instance. What is the canonical way of getting an instance's hostname?13:13
sean-k-mooneyjaypipes: the neutorn port will have it if you have designate deployed13:14
sean-k-mooneyif you are not using designate and are relying on the nova config option i guess it would be in the instance metadata somewhere13:14
*** dave-mccowan has quit IRC13:17
sean-k-mooneyjaypipes: https://developer.openstack.org/api-ref/network/v2/index.html#id5213:17
jaypipessean-k-mooney: that's what I would have thought...13:17
jaypipesthis cluster is on Ocata with nova-network :(13:18
sean-k-mooneyoh then the cloud init metadata is your best bet. we have an api endpoint for that right?13:18
*** awaugama has joined #openstack-nova13:19
jaypipessean-k-mooney: a colleague is trying to get this information via the openstacksdk. :(13:19
sean-k-mooneyah that could be tricky. stephenfin you did a deep dive on the metadata stuff recently for the doc any ideas?13:20
sean-k-mooneyjaypipes: stephenfin might be iun a 1:1 with our manger at the moment so there might be a delay13:20
*** betherly has joined #openstack-nova13:20
*** sapd1_x has joined #openstack-nova13:20
*** lbragstad has joined #openstack-nova13:21
sean-k-mooneylooks like its not in os-interface either13:22
stephenfinsean-k-mooney, jaypipes: That's exposed via the metadata service, yes13:22
sean-k-mooneystephenfin: any idea how you would query it outside of the vm13:23
mriedemjaypipes: what's in the OS-EXT-SRV-ATTR:hostname param?13:23
*** awalende has joined #openstack-nova13:23
sean-k-mooneymriedem: i think that is just the name specifed when you created the vm without the rest of the fqdn. but honestly not sure with nova-net13:24
stephenfinsean-k-mooney: Oh, outside it? That I don't know13:24
stephenfinsean-k-mooney: fwiw, we (wrongly) use the nova-net domain stuff to generate a FQDN for the metadata service and other stuff13:24
sean-k-mooneystephenfin: is /servers/{server_id}/metadata/{key} the same thing https://developer.openstack.org/api-ref/compute/?expanded=show-metadata-item-details-detail#list-all-metadata13:24
mriedemno13:25
stephenfinI tried fixing it at some point but gave up because there was a risk of breaking stuff13:25
mriedemthat's user-supplied metadata13:25
sean-k-mooneystephenfin: if you deploying with nova-net then we correctly do that :)13:25
mriedemthis is the meta api hostname code https://github.com/openstack/nova/blob/03322bb517925a9f5a04ebdb41c3fd31e7962440/nova/api/metadata/base.py#L53013:25
*** tkajinam_ has joined #openstack-nova13:25
jaypipesmriedem: there isn't one.13:25
stephenfinsean-k-mooney: https://review.openstack.org/#/c/480616/13:25
sean-k-mooneymriedem: ah ok so we dont have a way to get the the cloud init metadata via the rest api then?13:26
*** tkajinam has quit IRC13:27
mriedemi'm not aware of something obvious no13:27
sean-k-mooneystephenfin: oh i tought you ment it was wrong becasue we should be using the designate one. not it was borked even for nova-net13:27
stephenfinsean-k-mooney: Nah, it's fine for nova-net so long as you don't unset 'dhcp_domain'. I was talking about the designate one13:28
*** awalende has quit IRC13:28
stephenfinWhich I tried to do in an earlier revision but garyk pointed out that it would break stuff13:28
jaypipeswas OS-EXT-SRV-ATTR:hostname added in a microversion? I don't see it listed in the compute API reference.13:29
mriedemno it's always been there, hence the prefix13:29
sean-k-mooneyjaypipes: i assume you/oath/nova-net dont have rdns ptr record to allow looking up the dns name from an ip?13:29
mriedemhttps://developer.openstack.org/api-ref/compute/?expanded=show-server-details-detail#show-server-details13:29
jaypipesmriedem: you mean OS-EXT-SRV-ATTR:hypervisor_hostname or OS-EXT-SRV-ATTR:host ?13:30
mriedemneither13:30
mriedemi mean OS-EXT-SRV-ATTR:hostname13:30
stephenfinjaypipes: I see it in api-ref/source/servers.inc13:30
stephenfinwhatever that gets generated into13:30
jaypipesah, yeah. 2.313:30
jaypipessorry mriedem13:30
mriedemoh so it was exposed later13:30
stephenfinjaypipes, mriedem: For later, do you think this is ready to go https://review.openstack.org/#/c/480616/ ?13:30
jaypipesBy default, it appears in the response for administrative users only.13:30
mriedemsuper fun that we kept the prefix...13:30
jaypipesthat might be the issue..13:31
mriedemstephenfin: with all of those failing tests?13:31
stephenfinUnrelated failures13:31
jaypipesstephenfin: I can look at it in a bit.13:31
mriedemjaypipes: we could certainly make a specific policy rule for that13:31
stephenfinta13:32
openstackgerritStephen Finucane proposed openstack/nova master: conf: Undeprecate the 'dhcp_domain' option  https://review.openstack.org/48061613:32
mriedemstephenfin: i would have to load that context back into my head, because i remember the session in sydney about this and we were going to try and use the dns_network info from neutron if it was available, and change the behavior on the option to avoid backward compat issues13:32
*** altlogbot_2 has quit IRC13:32
mriedemhttps://etherpad.openstack.org/p/YVR18-novaneutron-ops-cross-project-session13:33
stephenfinmriedem: Ideally yes, but it seems that's probably an RFE more than a bugfix https://review.openstack.org/#/c/480616/6/nova/api/metadata/base.py@54913:33
sean-k-mooneyyes i rememebr that being the direction13:33
*** altlogbot_0 has joined #openstack-nova13:33
sean-k-mooneywe coudl do both. stephenfin fix corrects the edgecase wehre the config value is set to None13:34
mriedemstephenfin: that change isn't what was proposed in that forum session13:34
sean-k-mooneythe reading it form neutron is realitvly simple change on top13:34
mriedembecause gary is right - if the neutron network doesn't have dns then it's broken13:34
*** d34dh0r53 has quit IRC13:35
stephenfinmriedem: It is not. It's a stopgap because I think the change in the forum session probably needs a spec13:35
stephenfinso I need to undeprecate it so people can keep doing what they're doing until I've time to write that spec13:35
stephenfinthat's my thinking anyway13:35
*** boxiang has quit IRC13:37
sean-k-mooneyso im not sure if this is a spec or speclesss blueprint but keeping it simply i woudl personally just use teh neutron value if set and fall back to the nova config value if not13:37
*** boxiang has joined #openstack-nova13:37
stephenfinsean-k-mooney: Other way round, surely, to avoid breaking the upgrade path13:38
*** phasespace has quit IRC13:38
sean-k-mooneythat is what i woudl expect as an enduser but with that said the OS-EXT-SRV-ATTR:hostname  field should not contain the FQDN13:38
*** altlogbot_0 has quit IRC13:38
stephenfinAye. Possible (microversion) change there13:38
sean-k-mooneystephenfin: no i mean if we want to expose an FQDN call it FQDN13:39
sean-k-mooneywe shoudl not expose an FQDN in a hostname field13:39
sean-k-mooneyso microver for a new file din server-details13:39
*** altlogbot_2 has joined #openstack-nova13:39
sean-k-mooneypossibelwithout the OS-EXT-SRV-ATTR: prefix and proably not admin onle13:40
sean-k-mooneythat definetly would need a spec actully13:40
mriedemstephenfin: comments in your patch13:42
*** eharney has joined #openstack-nova13:42
stephenfinta13:43
sean-k-mooneyoh the hostname is already an FQDN in some cases... of couse it is.13:44
*** gaoyan has joined #openstack-nova13:47
*** mlavalle has joined #openstack-nova13:48
*** yedongcan has left #openstack-nova13:49
*** slaweq has quit IRC14:00
*** altlogbot_2 has quit IRC14:00
openstackgerritTetsuro Nakamura proposed openstack/nova master: Add get_compute_nodes_by_host_or_node()  https://review.openstack.org/65087714:01
openstackgerritTetsuro Nakamura proposed openstack/nova master: Pass target host to RequestGroup.in_tree  https://review.openstack.org/65087814:01
openstackgerritTetsuro Nakamura proposed openstack/nova master: Query `in_tree` to placement  https://review.openstack.org/64953514:01
*** cfriesen has joined #openstack-nova14:02
*** slaweq has joined #openstack-nova14:02
cdentdansmith: os-traits release: https://review.openstack.org/65207914:05
dansmithsweet, thanks14:05
cdentI'll start exploring the tox-siblings stuff next week14:05
dansmithcdent: what in there makes it independent vs. whatever it is now?14:06
cdentsimply that it is in the _independent dir14:06
cdentdansmith: I had the same question. I figured there must be a yaml file somewhere that declares what cycle something follows, but apparently it is per release14:07
dansmithah14:07
dansmithbut... it was before?14:07
dansmithor 0.1.0 was before14:08
*** amodi has joined #openstack-nova14:08
*** gaoyan has quit IRC14:09
cdentthere's was a 0.1.0 which was the very first release14:09
cdentthen it moved to being in cycles, so there are yaml files in deliverables/$CYCLE14:10
cdentand now it moves back to _independent14:10
dansmithgotcha okay14:10
dansmithI guess it's the unix "everything is a file" philosophy :)14:10
openstackgerritStephen Finucane proposed openstack/nova master: conf: Undeprecate and rename the 'dhcp_domain' option  https://review.openstack.org/48061614:11
*** lpetrut has quit IRC14:11
*** gaoyan has joined #openstack-nova14:12
stephenfinmriedem: I had to move that option in order to add it to the metadata file, so I also took the opportunity to rename it (since nova-net should be going away this cycle). Not entirely happy with the name so open to suggestions14:16
mriedemstephenfin: why not just leave the option name as dhcp_domain?14:21
mriedemcouple more comments14:23
mriedemi'm not sure what you mean by "its been repurposed"14:24
*** dpawlik has quit IRC14:24
stephenfinmriedem: It seemed odd, given that it's not really anything specific to DHCP as used here14:26
kashyapstephenfin: Bike-shedding: if the new name is "domain", beware that Nova frequently uses that term in context of "libvirt domains"14:26
melwittdansmith: could you take a look at the question I asked on the discover_hosts collision patch warn? I'm getting confused about whether I should return 0 if strict=False https://review.openstack.org/651947 and then also tssurya pointed out that map_instances skips DBDuplicateEntry14:26
stephenfinkashyap: Yeah, it also easy interpret as the domain of the metadata service or something. Not a fan14:26
*** altlogbot_2 has joined #openstack-nova14:26
stephenfinEh, I haven't got a better idea so back to dhcp_domain it is14:27
kashyapstephenfin: fqdn_name?14:28
mriedemjust leave it dhcp_domain14:28
kashyapYeah14:28
kashyap(Also: the 'n' in 'fqdn' is name)14:28
*** altlogbot_2 has quit IRC14:29
artomRAS syndrome!14:29
*** altlogbot_3 has joined #openstack-nova14:30
dansmithmelwitt: done14:32
openstackgerritStephen Finucane proposed openstack/nova master: conf: Undeprecate and move the 'dhcp_domain' option  https://review.openstack.org/48061614:32
melwittdansmith: ty14:32
stephenfinmriedem: back to dhcp_domain it is ^14:33
*** altlogbot_3 has quit IRC14:33
melwittI was thinking about it in the inverse, keep running under zero is returned, not keep running until nonzero is returned. no wonder I was confused xD14:37
melwitt*until14:43
mriedemi also commented because everyone loves a good bike shed, not sure how my comments align with others14:43
dansmithmelwitt: well, I'm just going based on the existing behavior, not necessarily the right behavior14:44
melwittI see, yeah14:44
dansmithmriedem: my preference is to just highlight the error and stop processing records, not skip and continue14:45
dansmithwhich is why I think we should bail,14:45
dansmithbut I think the existing return code behavior means we return 1 until we find nothing else to do, then return 014:46
dansmithand in that case, if we bail, we should return 1 so that something knows that there may be more stuff to do14:46
mriedemthe existing return code behavior depends on the --strict option14:47
dansmith...right14:48
mriedemif you don't specify --strict it returns 0 unless something blows up14:48
dansmith...right, are you saying you want to preserve the non-strict nonzero behavior on error?14:48
*** altlogbot_1 has joined #openstack-nova14:49
dansmithI thought she was specifically asking about what to do if --strict was passed14:49
melwittwait, I thought we return 0 until there's nothing more to do, and then return 1 when there's nothing to do?14:49
mriedemif i'm running this on a cron or something and not specifying --strict, i don't want it to error out if it got a duplicate entry error because it either did the job or there wasn't anything to do, so i don't care14:49
dansmithmriedem: but the job may very well be unfinished14:49
dansmithmelwitt: not currently, AFAICT14:50
mriedemif we just blow up and stop processing sure14:50
dansmithmelwitt: [07:46:16]  <dansmith>but I think the existing return code behavior means we return 1 until we find nothing else to do, then return 014:50
tssuryadansmith: what I meant was to *not* stop if we found duplicates, I don't want my periodic job stopping and complaining that it already found a host was already mapped and now it doesn't want to proceed14:50
tssuryabut it could also just be me :)14:50
*** shilpasd has quit IRC14:50
mriedemtssurya: i think you and i are saying the same thing14:50
kashyapalex_xu: When you're about, the more I think about this, the more I'm not sure if we should do this: https://review.openstack.org/#/c/642030/14:50
kashyapalex_xu: I'll think more and write on the change next week.14:51
tssuryamriedem: yea just replied to your comment14:51
tssurya(on the patchset)14:51
dansmithtssurya: okay, I just don't see the point of having two threads which are clearly fighting continue to fight, processing all the records in the db14:51
*** altlogbot_1 has quit IRC14:51
dansmithin the case of there only being a few new nodes, it's not a problem,14:52
mriedemstephenfin: more comments14:52
*** altlogbot_3 has joined #openstack-nova14:53
dansmithbut if you're turning on several thousand and running this in parallel for each node you added (which I think was the original assertion), you could have thousands of threads, running across many controller nodes, processing every record, and all but one are losing14:53
stephenfinefried: I've made a horrible mistake trying to bump hacking14:53
stephenfinFML14:53
stephenfinOh, wait14:53
*** stephenfin is now known as finucannot14:53
efriedstephenfin: It's educational at worst14:53
*** efried is now known as fried_rice14:53
finucannotRunning tox -e pep8 and it just keeps on scrolling...14:53
finucannotI've no idea how this wasn't picked up locally before either. Very strange14:54
fried_ricefinucannot: I've got 1314 lines of it14:54
tssuryadansmith: fair enough14:54
finucannotHopefully most of them are W504 (which can be disabled because it's W503 or W504, choose one)14:54
finucannotStill, ugh14:54
fried_ricefinucannot: I put how many of each in my last comment.14:55
fried_rice1055 of them are W50414:55
tssuryadansmith: yeah you have a valid point, its just that the behaviour of the command seemed weird when its all about discovering hosts14:55
*** altlogbot_3 has quit IRC14:55
dansmithtbh, I think what melwitt has is probably what I would want.. we retain the existing behavior of returning nonzero for the duplicate case, we just get a nice message about why14:59
*** altlogbot_2 has joined #openstack-nova14:59
dansmithno behavioral change at all, just easier for the humans14:59
dansmithwe can change the behavior too, but that means deployment people likely have to change their stuff14:59
*** _alastor_ has quit IRC15:00
tssuryayea return code wise no behaviour change, just that now the command stops because it found an already discovered host :D15:01
melwittyeah, that's what I was thinking with return 1, was today it traces and returns nonzero, so keep the behavior, just warn about it instead of tracing15:01
tssuryabut I guess I am okay with it considering we run it as a periodic task15:01
dansmithtssurya: it stopped before15:01
melwittI hadn't been thinking about the backwards (from the way my brain works) way that it currently returns values15:01
dansmithtssurya: cern runs it as a periodic?15:01
tssuryadansmith: no15:01
dansmithtssurya: okay good :D15:01
tssuryaI meant in general :)15:01
dansmithgotcha15:01
dansmithmelwitt: yeah, I think the return codes are a bit wonky, but what you have is an improvement for humans15:02
melwitttssurya: yeah, today the command stops because it raises an exception all the way up15:02
*** boxiang has quit IRC15:03
*** boxiang has joined #openstack-nova15:03
dansmithmelwitt: does this also continue to raise an exception for the periodic?15:03
dansmithmelwitt: because I thought we were going to catch and log a warning for the periodic case to avoid the trace in the logs15:03
melwittit would raise HostMappingExists yeah. I missed the point about catching and logging for periodic only15:04
dansmithor did you want to do that in a separate patch15:04
dansmith?15:04
*** igordc has joined #openstack-nova15:04
dansmithdon't you think that's better than a trace? LOG.warning('This should only be enabled when ...')15:05
tssuryamelwitt, dansmith: yeah ok thanks I am aboard the plan then15:05
melwittdansmith: yeah I think that's better, I just missed it and need to add it. what do you mean about "this should only be enabled when" though?15:06
tssuryamelwitt: let's also add something in the nova-manage docs like a fat note15:06
dansmithlike "this should only be enabled on a single scheduler" or something15:06
melwittdansmith: I'm concerned about mschuppert's comment here though https://bugzilla.redhat.com/show_bug.cgi?id=1698630#c7 does this mean he expects it to return zero in the dupe case?15:06
openstackbugzilla.redhat.com bug 1698630 in openstack-tripleo-heat-templates "deployment fails running nova_cellv2_discover_hosts process with a duplicate key" [High,New] - Assigned to owalsh15:06
melwittdansmith: ack got it15:07
melwitttssurya: I think that docstring gets put in the docs right? let me check actually15:07
dansmithmelwitt: I dunno what of that you think is related to his expectation of the return value15:08
dansmithmelwitt: but if he's expecting something that isn't in line with the current behavior, he's probably expecting the wrong thing :)15:08
melwitt"With the proposed change to nova to just report a warning we'll not fail in that case."15:08
dansmith*my* concern about that comment is that he's talking about running this *on* the computes15:08
owalshdansmith: on one compute15:09
dansmithmelwitt: oh is he talking about your proposed change/15:09
dansmithowalsh: doesn't matter.. any compute :)15:09
owalshmelwitt: he's assuming rc==0, yes15:09
melwittI assumed so. I had linked it on the BZ15:09
owalshdansmith: yea... not how I did it initially but reasons...15:09
dansmithowalsh: that's unfortunate.. giving computes credentials to the api DB just to run this thing the controllers should do.. do those creds persist on the compute?15:10
dansmithmelwitt: okay I hadn't read it that way.. so he's just misunderstanding what your patch is doing you mean'?15:10
melwittI'm concerned that he is, yeah15:11
melwitttssurya: you're right, I need to add to the docs separately, the docstrings don't go in there automatically http://logs.openstack.org/47/651947/2/check/openstack-tox-docs/d6d7085/html/cli/nova-manage.html15:11
dansmithmelwitt: but he's quoting from owalsh who says they'll use a retry loop, which is fine yeah?15:11
*** tosky has quit IRC15:11
*** tosky has joined #openstack-nova15:12
melwittyeah, I guess so. I was hyperfocusing on the reply and not the quote15:12
owalshdansmith: not sure where the creds are coming from now actually... previously when I moved this to the computes I embedded them in the script and unlink the script as soon as it was run to avoid leaving creds around15:13
dansmithowalsh: okay15:13
owalshdansmith: but it's been rewritten since, I'll take a look15:13
*** eharney has quit IRC15:13
dansmithowalsh: "talking to the api database" is something we generally expect a compute not to ever do.. I guess in this case you're using it as the admin node since you're just taking action on that one,15:15
dansmithbut in ansible lingo, I would "delegate" the discovery task to one of the controller nodes15:16
owalshdansmith: yea, the reasons for this were that the controller nodes were being excluded from tripleo scale outs to speed things up, so it had to be on a compute... there isn't anything else15:16
dansmithright, because tripleo/heat reasons yeah? If it was pure ansible delegating a single admin command to one controller node after you're done would be easy, but I know that heat probably makes that hard15:17
owalshmight be possible now that tripleo == heat generating ansible playbook, but not sure if it would backport15:18
*** ccamacho has quit IRC15:20
*** ivve has quit IRC15:20
*** luksky has quit IRC15:24
fried_ricefinucannot: a little sphinx help?15:27
finucannotshoot15:27
fried_ricefinucannot: I'm in a spec15:27
*** tbachman has quit IRC15:27
fried_riceI want to make a link that gets referenced from a couple of places15:28
fried_riceso it's like15:28
fried_rice.. `some text`: https://...15:28
fried_ricebut in another place I want to use different text15:28
finucannot:ref:`the text you want <the-anchor-name>`15:28
fried_riceI tried `different text <some text>`_15:28
finucannot.. _the-anchor-name: https://...15:28
finucannotOh/15:28
*** HuaChangWang has joined #openstack-nova15:28
fried_riceyeah, I tried :ref:`different text <some text>`15:28
finucannot*?15:29
fried_riceit didn't work15:29
finucannotHmm15:29
* finucannot checks15:29
fried_ricesorry, yes15:29
fried_rice.. _`some text`: https://15:29
fried_riceforgot the _ originally15:29
fried_riceThe `some text`_ link works fine15:29
fried_riceit's the "different text" I can't suss.15:29
*** Sundar has joined #openstack-nova15:30
*** cdent has quit IRC15:32
finucannotfried_rice: Ah, yeah, ref is for internal cross-referencing. My bad.15:35
finucannotfried_rice: It's ugly, but I think the only way is to do something like this15:35
finucannot`some text`_ and `some other text`_15:35
finucannot.. _some text: https://...15:35
finucannot.. _`some other text`: `some text`_15:36
finucannotwhich doesn't really help :/15:36
*** slaweq has quit IRC15:37
*** _alastor_ has joined #openstack-nova15:37
*** slaweq_ has joined #openstack-nova15:37
fried_ricefinucannot: It helps a little I guess. Point being to only specify the https://... in one place.15:37
finucannotfried_rice: Wait, no, got it. `some other text <some text_>`_15:37
finucannotYou need the first underscore to covert that to reference, otherwise it's taken literally15:38
finucannoti.e. you'll end up at '/some text'15:38
fried_riceyeah, that was the result I was getting. Trying....15:38
finucannotOr from earlier, `the text you want <the-anchor-name_>`_15:39
fried_ricefinucannot: \o/ works! finuCAN!15:39
finucannotSide note: 'sphinx-quickstart' in 2.0 has much less options and is now generating non-crufty conf.py files. Hurrah!15:40
fried_ricewhat's that?15:40
finucannotIt's a tool Sphinx provides to bootstrap a new documentation project15:40
finucannottox has a similar one (tox-quickstart) inspired by Sphinx15:41
fried_riceah, cool15:41
*** slaweq__ has joined #openstack-nova15:44
*** slaweq_ has quit IRC15:46
*** gaoyan has quit IRC15:48
zigomriedem: I proposed the same kind of change on python-novaclient too, but it looks like nobody was paying much attention to Python 3 at the time, and my patch got rejected there, when it was accepted in osc. I use exclusively osc anyway, so I didn't insist much ! :)15:50
zigoProbably more out of lazyness, sorry for that.15:51
*** slaweq__ has quit IRC15:51
openstackgerritTheodoros Tsioutsias proposed openstack/nova master: Add instance hard delete  https://review.openstack.org/57020215:52
openstackgerritTheodoros Tsioutsias proposed openstack/nova master: Add requested_networks to RequestSpec  https://review.openstack.org/57020115:52
openstackgerritTheodoros Tsioutsias proposed openstack/nova master: Enable rebuild for instances in cell0  https://review.openstack.org/57020315:52
openstackgerritEric Fried proposed openstack/nova-specs master: Updates to image-metadata-prefiltering spec  https://review.openstack.org/65210115:53
fried_ricesean-k-mooney, dansmith, jaypipes, alex_xu, finucannot: ^15:54
openstackgerritEric Fried proposed openstack/nova-specs master: Updates to image-metadata-prefiltering spec  https://review.openstack.org/65210115:55
*** gaoyan has joined #openstack-nova15:57
*** derekh has quit IRC15:58
*** gaoyan has quit IRC15:58
*** psachin has joined #openstack-nova16:01
*** jangutter has quit IRC16:04
*** janki has quit IRC16:05
openstackgerritStephen Finucane proposed openstack/nova master: Bump to hacking 1.1.0  https://review.openstack.org/65155316:06
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve E731 issues  https://review.openstack.org/65155416:06
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve W503 issues  https://review.openstack.org/65155516:06
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve E741 (ambiguous variable name)  https://review.openstack.org/65210316:06
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve W605 (invalid escape sequence)  https://review.openstack.org/65210416:06
finucannotfried_rice: done16:07
* finucannot -> home16:07
finucannot*pub16:08
*** finucannot is now known as stephenfin16:08
*** jmlowe has quit IRC16:11
*** dtantsur is now known as dtantsur|afk16:12
*** HuaChangWang has quit IRC16:13
*** eharney has joined #openstack-nova16:15
*** gyee has joined #openstack-nova16:15
openstackgerritEric Fried proposed openstack/nova-specs master: Updates to image-metadata-prefiltering spec  https://review.openstack.org/65210116:17
*** davidsha has quit IRC16:23
*** sapd1_x has quit IRC16:23
*** rpittau is now known as rpittau|afk16:24
openstackgerritDustin Cowles proposed openstack/nova master: WIP/PoC: Introduces the openstacksdk to nova  https://review.openstack.org/64366416:34
openstackgerritmelanie witt proposed openstack/nova master: Warn for duplicate host mappings during discover_hosts  https://review.openstack.org/65194716:34
openstackgerritEric Fried proposed openstack/nova-specs master: Updates to image-metadata-prefiltering spec  https://review.openstack.org/65210116:37
*** tosky has quit IRC16:38
*** mlavalle has quit IRC16:46
*** tssurya has quit IRC16:51
*** spatel has joined #openstack-nova16:53
*** awalende has joined #openstack-nova16:53
spatelsean-k-mooney: hey!!!16:53
spatelHow are you doing sean!!16:53
spatelDoes Google cloud provide SR-IOV function if anyone aware of it?16:55
*** awalende has quit IRC16:57
*** tbachman has joined #openstack-nova16:57
*** mlavalle has joined #openstack-nova17:04
*** ralonsoh has quit IRC17:05
*** mlavalle has quit IRC17:11
sean-k-mooneyspatel: am im not sure. i have not looked at gce much17:11
spatelI don't think they provide SR-IOV because i am not seeing anywhere mentioned17:12
sean-k-mooneyspatel: i konw aws whas some offerenings that have gpus and fpgas so they might but not sure where google stands in that regard17:12
artomIt can be googled in 2 seconds, FWIW... https://stackoverflow.com/questions/33114144/does-google-compute-engine-offer-sr-iov-single-root-i-o-virtualization17:12
artom... and has nothing to do with OpenStack?17:12
spatelGoogle using DPDK style enhance networking... if i am not wrong17:12
sean-k-mooneyartom: spatel has recently (last 6 months) moved form aws to runign an sriov based openstack cloud17:13
sean-k-mooneyartom: im guessing spatel you are interested in comparing it to a few other options17:13
* artom goes to be a curmudgeon elsewhere17:14
melwittfried_rice: we've been talking about SchedulerReportClient whether it's heavy to construct, if you might have anything to add https://review.openstack.org/#/c/638073/24/nova/quota.py@115417:16
spatelsean-k-mooney: artom we are using aws in many place and costing is killing us so looking for other cloud provide like Google or tencent etc..17:19
spatelWe have requirement of high performance networking so exploring options related SR-IOV17:20
spatelartom: i did read that post already but not sure if Google change something recently and someone aware here so i post question here17:21
spatelThat post is 4 year old also :(17:21
*** cdent has joined #openstack-nova17:25
cdentfried_rice: what you know of the state of https://bugs.launchpad.net/nova/+bug/1724633 . Still alive?17:25
openstackLaunchpad bug 1724633 in OpenStack Compute (nova) "AllocationCandidates.get_by_filters hits incorrectly when traits are split across the main RP and aggregates" [Medium,Confirmed]17:25
*** ivve has joined #openstack-nova17:27
*** cfriesen has quit IRC17:28
*** mlavalle has joined #openstack-nova17:37
*** spatel has quit IRC17:37
*** cfriesen has joined #openstack-nova17:42
*** ttsiouts has quit IRC17:56
*** ttsiouts has joined #openstack-nova17:57
*** cfriesen has quit IRC17:58
*** spatel has joined #openstack-nova18:01
*** ttsiouts has quit IRC18:01
*** dave-mccowan has joined #openstack-nova18:06
*** imacdonn_ is now known as imacdonn18:08
*** irclogbot_2 has quit IRC18:08
*** irclogbot_2 has joined #openstack-nova18:11
mriedemwho would have know that our actions api is not completely solid https://bugs.launchpad.net/nova/+bug/182442018:13
openstackLaunchpad bug 1824420 in OpenStack Compute (nova) "Live migration succeeds but instance-action-list still has unexpected Error status" [Undecided,Triaged]18:13
mriedem*known18:13
*** bbowen has quit IRC18:16
*** tosky has joined #openstack-nova18:18
eanderssonmriedem, if you have some time can you take a stab at backporting that patch? If not I can probably do it.18:28
eanderssonThe allocation leak from resize/cold migration18:29
mriedemoh yeah sec18:29
mriedemeandersson: btw did you figure out where you had migration failures leaking allocations?18:33
eanderssonYea - we are doing a lot of migration testing, and those are causing it.18:34
eanderssonAnother person on our team are performing them.18:34
mriedemwhat's actually failing in confirm resize?18:34
eanderssonYes, we believe so.18:34
eanderssonHe thought it was someone else, as the uuid logged didn't match his VMs.18:34
eanderssonBut the uuids are actually migration uuids.18:34
mriedemok, good to know - confirm resize shouldn't really be failing like that, so you've got some other issue18:35
mriedembut yeah fixing that leak is something we need (on rocky for you)18:35
*** cdent has quit IRC18:38
*** psachin has quit IRC18:46
openstackgerritMatt Riedemann proposed openstack/nova stable/rocky: Error out migration when confirm_resize fails  https://review.openstack.org/65212718:58
mriedemeandersson: ^ is the first half of that backport, i have to run to a school thing for my kid but with get the other patch when i'm back18:58
mriedem*will18:58
*** rchurch has joined #openstack-nova18:59
*** lajoskatona has joined #openstack-nova19:01
*** rchurch_ has quit IRC19:01
lajoskatonamriedem: Hi, regarding https://review.openstack.org/640600 & https://review.openstack.org/640601 do you have special reason to keep the workflow -1 on them?19:04
*** whoami-rajat has quit IRC19:09
*** lajoskatona has quit IRC19:20
*** boxiang has quit IRC19:21
*** boxiang has joined #openstack-nova19:21
*** boxiang has quit IRC19:22
openstackgerritMerged openstack/nova master: Add --dry-run option to heal_allocations CLI  https://review.openstack.org/65193219:22
*** mdbooth_ has joined #openstack-nova19:23
*** phasespace has joined #openstack-nova19:23
*** mdbooth has quit IRC19:26
*** spatel has quit IRC19:30
*** sidx64 has joined #openstack-nova19:32
eanderssonThanks a lot mriedem19:54
*** luksky has joined #openstack-nova20:07
*** cfriesen has joined #openstack-nova20:11
*** tjgresha_nope has quit IRC20:12
*** tjgresha has joined #openstack-nova20:13
fried_ricemelwitt: ack, looking. Generally speaking, the heavy part is building the ksa adapter, which can be heavy or not so heavy depending how the admin set up the conf options.20:16
fried_riceopenstack: cdent: I don't know anything about bug 1724633 off the top, would have to try to repro. lmk if you need that.20:18
openstackbug 1724633 in OpenStack Compute (nova) "AllocationCandidates.get_by_filters hits incorrectly when traits are split across the main RP and aggregates" [Medium,Confirmed] https://launchpad.net/bugs/172463320:18
sean-k-mooneyo/20:20
sean-k-mooneyi have some good news and bad news20:20
*** sidx64 has quit IRC20:20
sean-k-mooneywhich would people like to hear frist :P20:21
sean-k-mooneyill send an email to the list but we nolonger need nested virt to test hugepages,numa,cpu pinning or dpdk in the gate20:24
sean-k-mooneyi also found we apparently have a deadlock in the nova compute agent if we run it on python 3.720:24
sean-k-mooneymriedem: ^20:25
*** pcaruana has quit IRC20:26
sean-k-mooneyim going to try and port my thidparty jobs upstream before teh summit but we should be able to replace the intel nfv ci with upstream gate jobs20:26
mriedemwe aren't testing on py37 so...20:26
mriedemmeh?20:26
sean-k-mooneyfor now i guess20:29
sean-k-mooneywe seam to lockup the compute agent when we go to spawn an instance under py3.720:29
sean-k-mooneyno errors or warnings20:29
sean-k-mooneybut anyway if we use fedora 29 + the virt preveiw repo we can get libvirt 5.2 and a new enough qemu that allows cpu pinnign to work without kvm20:30
fried_ricemelwitt: responded20:30
sean-k-mooneyso i shoudl be able to pull together a gate job that will fully test cpu pinning and hugepages20:31
mriedemfedora 29 + virt preview with bleeding edge libvirt/qemu sounds super stable20:31
sean-k-mooneyya i hate useing fedroa for ci or dev20:31
openstackgerritJulia Kreger proposed openstack/nova master: [ironic] ignore conflict upon cleaning up node  https://review.openstack.org/64348420:31
mriedemi guess if it's just a smaller set of tests in a non-voting job, sure20:31
mriedembetter than 0 coverage20:31
sean-k-mooneywe jsut need the new libvirt really20:32
sean-k-mooneyi have a devstack plugin that can compile it form source20:32
sean-k-mooneyand qemu20:32
fried_ricetjgresha: this conversation may interest you ^20:32
sean-k-mooneyso i caould also use that do it with ubuntu or centos20:32
fried_ricesean-k-mooney: I think there's work in progress to resurrect nfv CI, so if you think we don't need it at all, that'd be a good thing to know.20:32
sean-k-mooneyhttps://github.com/openstack/devstack-plugin-libvirt-qemu20:33
sean-k-mooneyyes i heard20:34
*** mlavalle has quit IRC20:34
sean-k-mooneywe currently dont have vm with multiple numa nodes in the gate20:34
sean-k-mooneywhich is onething the nfv ci had20:34
sean-k-mooneywe coudl maybe ask operators to donate such a nodeset20:35
mriedemthat's what the openlab request from artom was fore20:35
mriedem*for20:35
mriedembut that would definitely be a periodic queue kind of thing20:35
mriedemor on-demand at best20:36
artomAnd also, if we get nested virt, do we need multiple physical NUMA nodes?20:36
artommriedem, yeah, but better than nothing20:36
*** mlavalle has joined #openstack-nova20:37
* artom still doesn't comprehend how Intel CI was able to run on every patch. How many machines did they have?20:37
openstackgerritMatt Riedemann proposed openstack/nova stable/rocky: Error out migration when confirm_resize fails  https://review.openstack.org/65212720:37
openstackgerritMatt Riedemann proposed openstack/nova stable/rocky: Delete allocations even if _confirm_resize raises  https://review.openstack.org/65214620:37
mriedemeandersson: ^ that should do it20:37
*** zbr has quit IRC20:37
sean-k-mooneyartom: about 1020:37
sean-k-mooneywe started wtih 320:38
sean-k-mooneythen grew to 520:38
artomsean-k-mooney, ah, that makes more sense20:38
mriedemspeaking of ci, the ibm zkvm ci should be shutdown20:38
artomSo clearly the 1 or 2 machines we might get from openlab won't handle it ;)20:38
mriedemfails immediately20:38
sean-k-mooneyartom: actully it coudl depending on how you set it up20:38
artomsean-k-mooney, remind me why need multiple physical NUMA nodes? I recall having this conversation with you before...20:38
artomCan't the all be nested virt multinode?20:39
artom*they all20:39
sean-k-mooneywe coudl if i was allowed to change the libvirt dirver code but everytime i suggest it people complain20:39
artomEh?20:40
sean-k-mooneyin the libvirt driver we implemented it in such a way as to always map virtual numa nodes to different phyical numa nodes20:40
sean-k-mooneybut we never ever said that was guarenteed20:40
sean-k-mooneybut it was someing people observed and started to depend on20:41
artomOh right, that was the thing. If this was pure libvirt/kvm we could20:41
artomBut because our dsvm will be servers in an openstack cloud20:41
sean-k-mooneyyes and we could in nova20:41
artomWe're limited by that thing20:41
sean-k-mooneythis is purly a nova limitation20:41
sean-k-mooneywe coudl cange the code however20:42
artomSeems like a lot of work :)20:42
sean-k-mooneyif you use multipl virtual numa nodes on hyperv i dont think they give the same guarentee20:42
melwittfried_rice: thanks20:42
artomNo reason we can't push the openlab thing as far as it'll go first20:42
mriedemcfriesen: how much does this matter to you guys? https://bugs.launchpad.net/nova/+bug/182442020:43
openstackLaunchpad bug 1824420 in OpenStack Compute (nova) "Live migration succeeds but instance-action-list still has unexpected Error status" [Undecided,Triaged]20:43
sean-k-mooneyartom: you know after i get a gate job working i might actully put up a patch to make the numa maping configurable20:43
sean-k-mooneyi at least want to see how invaisge it woudl be to allow it. i thnk it would be a very minor change but that code is ...20:44
artomDoesn't feel very minor, but you probably know that code better than I do20:44
sean-k-mooneyits a slight tweek to how we iterate over the posible permutaion in the hardware.py module when we fit an instance to the host20:45
* artom is a big fan of not doing work that's not strictly necessary ;)20:46
sean-k-mooneycurrently wehn we alocate a instance numa node to a host cell we dont consider that cell again for subsequent guest numa nodes20:46
mriedemmelwitt: dansmith: please to be reviewing these backports https://review.openstack.org/#/q/topic:bug/1669054+branch:stable/queens20:54
openstackgerritMatt Riedemann proposed openstack/nova stable/queens: Error out migration when confirm_resize fails  https://review.openstack.org/65215020:56
dansmithfeel like I've reviewed these so many times20:56
openstackgerritFrançois Palin proposed openstack/nova master: nova diagnostics command is not working with all interfaces  https://review.openstack.org/64812320:57
*** mlavalle has quit IRC20:58
*** mlavalle has joined #openstack-nova20:58
*** fried_rice is now known as efried_schoolrun20:58
dansmithmriedem: shall I slam these?20:59
melwittI can review too21:03
dansmiththey're ready for +W21:03
melwittcoolio21:04
mriedemdansmith: because we have 5 stable branches now :)21:04
dansmithmriedem: I know.. I was being fuh shee zus21:05
mriedemi'm very sensitive21:05
sean-k-mooneyartom: by the way the 2 server from openlab shoudl be enough to handel nova if we can get 12-14 vms running jobs in paralles and can keep the job under 2.5-3 hours21:06
melwittconflict city21:06
sean-k-mooneyanyway enjoy ye're weekend o/21:06
mriedemoh i forgot i'd backported https://review.openstack.org/#/c/648691/21:07
mriedemoh nvm you're talking about something else21:08
openstackgerritMatt Riedemann proposed openstack/nova stable/queens: Delete allocations even if _confirm_resize raises  https://review.openstack.org/65215321:10
*** awaugama has quit IRC21:14
melwittmriedem: I noticed remnants of unused retry code in the test https://review.openstack.org/#/c/649363/1/nova/tests/unit/objects/test_request_spec.py did you want to just leave it or?21:18
mriedemoh yeah...21:20
mriedemreplied; i could remove it, but then have to redo my pike and ocata cherry picks21:21
mriedemi'm fine either way21:21
melwittok, I guess I choose path of least work and leave it21:21
openstackgerritMatt Riedemann proposed openstack/nova stable/rocky: Error out migration when confirm_resize fails  https://review.openstack.org/65212721:22
openstackgerritMatt Riedemann proposed openstack/nova stable/rocky: Delete allocations even if _confirm_resize raises  https://review.openstack.org/65214621:22
mriedemthat's what i like to hear21:24
melwittI was thinking if it was an easy inline edit, that would be cool. but meh if you have to redo other stuff21:24
mriedemi'm sure i'll never have to touch this request spec code again in these old stable branches...21:25
melwittright exactly21:25
melwittrequest spec is solid21:25
*** efried_schoolrun is now known as efried21:29
*** Sundar has quit IRC21:31
*** lbragstad has quit IRC21:37
*** betherly has quit IRC21:45
mriedemeandersson: let me just say it's good you went to rocky rather than pike because trying to get that allocation leak fix back to pike is a huge pain in the ass, so i'm just going to not work on that for pike unless someone comes along and really needs it21:48
*** lbragstad has joined #openstack-nova21:51
eanderssonI wish we could go to stein tbh, but we still rely on some vendor drivers22:03
*** itlinux has joined #openstack-nova22:06
*** dave-mccowan has quit IRC23:00
*** hongbin has joined #openstack-nova23:06
*** tosky has quit IRC23:11
*** luksky has quit IRC23:19
openstackgerritMerged openstack/nova stable/stein: Error out migration when confirm_resize fails  https://review.openstack.org/64942123:20
openstackgerritMerged openstack/nova stable/stein: Delete allocations even if _confirm_resize raises  https://review.openstack.org/65043723:20
openstackgerritMerged openstack/nova stable/queens: Add functional regression test for bug 1669054  https://review.openstack.org/64936223:20
openstackbug 1669054 in OpenStack Compute (nova) queens "RequestSpec.ignore_hosts from resize is reused in subsequent evacuate" [Medium,In progress] https://launchpad.net/bugs/1669054 - Assigned to Matt Riedemann (mriedem)23:20
openstackgerritMerged openstack/nova stable/queens: Do not persist RequestSpec.ignore_hosts  https://review.openstack.org/64936323:20
*** nicolasbock has quit IRC23:30
*** hongbin has quit IRC23:34
*** igordc has quit IRC23:38
cfriesenmriedem: sorry, was having computer issues.  I think that would be lowish priority.  makes it a bit harder to write automated stuff if there are spurious error events.23:39
*** KH-Jared has quit IRC23:50
openstackgerritArtom Lifshitz proposed openstack/nova master: Revert "Wait for network-vif-plugged on resize revert"  https://review.openstack.org/63939623:59
openstackgerritArtom Lifshitz proposed openstack/nova master: [DNM: extra logs] Revert resize: wait for external events in compute manager  https://review.openstack.org/64488123:59
openstackgerritArtom Lifshitz proposed openstack/nova master: DNM: extra logging in neutron api for revert resize  https://review.openstack.org/65217423:59
artomOh, if you want to change the subject you can't just edit them in `git rebase -i`23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!