Tuesday, 2019-10-01

*** gbarros has quit IRC00:20
*** markvoelker has quit IRC00:26
*** gyee has quit IRC00:27
*** igordc has quit IRC01:26
openstackgerritEric Fried proposed openstack/nova master: Allow versioned discovery unauthenticated  https://review.opendev.org/68518101:33
*** hongbin has joined #openstack-nova01:55
*** macz has joined #openstack-nova02:19
*** brinzhang has joined #openstack-nova02:19
*** macz has quit IRC02:23
*** brinzhang_ has joined #openstack-nova02:24
*** brinzhang has quit IRC02:27
*** hongbin has quit IRC02:33
*** hongbin has joined #openstack-nova02:34
*** SonPham has joined #openstack-nova02:41
SonPhamhi02:47
SonPhamhad any project  to create live-snapshot for openstack?02:47
*** BjoernT has joined #openstack-nova02:47
SonPhamsnapshot with save ram state02:47
brinzhang_Now, nova not support to do for that03:01
*** mkrai has joined #openstack-nova03:04
openstackgerritSeyeong Kim proposed openstack/nova master: block_device: Optionally recreate attachments when refreshing connection_info  https://review.opendev.org/57900403:04
brinzhang_SonPham: sorry for above said, current, cannot do a live-snapshot for hte paused server of libvirt/qemu driver.03:07
brinzhang_paused and shutdown status server.03:08
melwittdansmith, mriedem: the doc reference got removed here https://github.com/openstack/nova/commit/009fd0f35bcb88acc80f12e69d5fb72c0ee5391f03:10
*** psachin has joined #openstack-nova03:12
*** BjoernT has quit IRC03:14
*** BjoernT has joined #openstack-nova03:18
*** nweinber has joined #openstack-nova03:19
SonPhambrinzhang_ Can we save VM state (save Ram) in the snapshot?03:26
*** cfriesen has quit IRC03:26
*** spsurya has joined #openstack-nova03:29
*** BjoernT has quit IRC03:29
brinzhang_SonPham: I think you can, bug I am not sure, you can review that and try, with "disable_libvirt_livesnapshot" configuration in you env.03:31
brinzhang_SonPham: https://docs.openstack.org/nova/stein/configuration/config.html#workarounds.disable_libvirt_livesnapshot03:31
*** cfriesen has joined #openstack-nova03:31
*** SonPham has quit IRC03:32
*** gmann_afk is now known as gmann03:33
*** markvoelker has joined #openstack-nova03:41
*** markvoelker has quit IRC03:42
*** markvoelker has joined #openstack-nova03:43
*** SonPham has joined #openstack-nova03:52
*** SonPham has joined #openstack-nova03:54
SonPhambrinzhang_ I'm using libvirt to create snapshot for instance03:54
*** hongbin has quit IRC03:56
*** udesale has joined #openstack-nova03:57
*** igordc has joined #openstack-nova03:58
*** SonPham has quit IRC04:12
*** nweinber has quit IRC04:16
*** ricolin has joined #openstack-nova04:16
*** macz has joined #openstack-nova04:20
*** SonPham has joined #openstack-nova04:23
*** macz has quit IRC04:25
*** macz has joined #openstack-nova04:28
*** macz has quit IRC04:30
*** macz has joined #openstack-nova04:30
*** macz has quit IRC04:35
*** spsurya has quit IRC04:37
*** spsurya has joined #openstack-nova04:37
*** mkrai has quit IRC04:46
*** pcaruana has joined #openstack-nova05:01
*** Luzi has joined #openstack-nova05:02
*** jaosorior has joined #openstack-nova05:05
*** dpawlik has joined #openstack-nova05:07
*** ratailor has joined #openstack-nova05:07
*** mkrai has joined #openstack-nova05:40
*** markvoelker has quit IRC05:47
*** igordc has quit IRC05:52
*** bbowen_ has joined #openstack-nova06:14
*** bbowen has quit IRC06:15
*** avolkov has joined #openstack-nova06:17
*** sapd1_x has joined #openstack-nova06:23
*** lpetrut has joined #openstack-nova06:31
*** macz has joined #openstack-nova06:32
*** ttsiouts has joined #openstack-nova06:35
*** macz has quit IRC06:36
*** francoisp has quit IRC06:40
*** hongbin has joined #openstack-nova06:56
*** slaweq has joined #openstack-nova06:57
*** hongbin has quit IRC07:00
*** rpittau|afk is now known as rpittau07:04
*** damien_r has joined #openstack-nova07:07
*** zbr is now known as zbr|ruck07:08
*** maciejjozefczyk has joined #openstack-nova07:08
*** awalende has joined #openstack-nova07:10
*** xek has joined #openstack-nova07:12
*** tesseract has joined #openstack-nova07:16
*** cfriesen has quit IRC07:17
*** SonPham has quit IRC07:19
*** parlos has joined #openstack-nova07:21
*** dtantsur|afk is now known as dtantsur07:23
*** luksky has joined #openstack-nova07:38
*** ttsiouts has quit IRC07:43
*** ralonsoh has joined #openstack-nova07:45
*** markvoelker has joined #openstack-nova07:47
*** markvoelker has quit IRC07:51
*** jangutter has quit IRC07:56
*** sapd1_x has quit IRC07:57
*** ttsiouts has joined #openstack-nova07:57
*** Jeffrey4l has quit IRC07:59
*** Jeffrey4l has joined #openstack-nova08:00
*** ociuhandu has joined #openstack-nova08:02
*** rcernin has quit IRC08:02
*** rcernin has joined #openstack-nova08:03
*** tkajinam has quit IRC08:04
*** brinzhang has joined #openstack-nova08:05
*** brinzhang_ has quit IRC08:08
*** ttsiouts has quit IRC08:12
*** jangutter has joined #openstack-nova08:14
*** jaosorior has quit IRC08:16
*** ttsiouts has joined #openstack-nova08:18
*** ociuhandu has quit IRC08:21
*** ociuhandu has joined #openstack-nova08:22
*** derekh has joined #openstack-nova08:32
mgoddardmriedem: yeah, I meant multiple cells. We would be quite behind to not have any support for cells v2...08:33
*** brinzhang_ has joined #openstack-nova08:37
*** brinzhang_ has quit IRC08:38
*** brinzhang_ has joined #openstack-nova08:39
*** brinzhang has quit IRC08:40
*** macz has joined #openstack-nova08:51
*** macz has quit IRC08:55
*** ociuhandu has quit IRC09:02
*** ociuhandu has joined #openstack-nova09:23
*** ratailor_ has joined #openstack-nova09:24
*** ratailor has quit IRC09:26
*** ociuhandu has quit IRC09:28
*** parlos has quit IRC09:35
*** mkrai has quit IRC09:40
*** tssurya has joined #openstack-nova09:41
*** ricolin has quit IRC09:44
*** ratailor__ has joined #openstack-nova09:48
*** markvoelker has joined #openstack-nova09:48
*** ratailor_ has quit IRC09:50
*** markvoelker has quit IRC09:53
*** avolkov has quit IRC09:57
bauzasgibi: can I ask you something about how to create a new allocation ?09:57
gibibauzas: sure hit me10:01
bauzasgibi: to explain it : http://paste.openstack.org/show/780546/10:03
bauzasgibi: I want to recreate the migration allocation that was deleted when confirming the resize10:03
bauzasgibi: so after that, after running my audit command, it would find it10:03
bauzasbut then, I need a consumer_generation, right?10:04
bauzasor should I just update the allocations ?10:04
bauzasie. https://docs.openstack.org/api-ref/placement/?expanded=update-allocations-detail#id7210:05
gibibauzas: you want to recreate the migration allocation so I assume that there is no allocation with that consumer. In that case the consumer generation when you create the allocation should be 010:05
bauzasgibi: that's correct10:06
bauzasI no longer have allocations for the migration UUID10:06
bauzasif so, we don't need to provide a generation version10:06
bauzasgibi: do we have any helper for putting a new allocation in the functional tests ?10:07
gibiI mean: "The generation of the consumer. Should be set to null when indicating that the caller expects the consumer does not yet exist."10:07
gibibauzas: I think I never needed to manually create allocation in the functional test10:07
bauzasok10:08
*** ttsiouts has quit IRC10:08
*** brinzhang has joined #openstack-nova10:08
*** ttsiouts has joined #openstack-nova10:09
stephenfingmann: Still about? I'm stuck with https://review.opendev.org/#/c/684335/5/doc/api_samples/servers/v2.69/servers-details-resp.json10:10
stephenfingibi: You might be able to advise on ^ too10:11
*** brinzhang_ has quit IRC10:12
*** tetsuro has joined #openstack-nova10:12
*** ttsiouts has quit IRC10:13
*** mkrai has joined #openstack-nova10:17
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_server_tags' functional tests  https://review.opendev.org/68433510:20
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_servers' functional tests  https://review.opendev.org/68433610:20
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_hosts' functional tests  https://review.opendev.org/68433710:20
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_hypervisors' functional tests  https://review.opendev.org/68434010:20
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_attach_interfaces' functional tests  https://review.opendev.org/68434110:20
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_simple_tenant_usage' functional tests  https://review.opendev.org/68434210:20
openstackgerritStephen Finucane proposed openstack/nova master: trivial: Neutron fixture cleanup  https://review.opendev.org/68434310:20
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_floating_ips' functional tests  https://review.opendev.org/68434410:20
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Add TODOs for remaining nova-network functional tests  https://review.opendev.org/68434510:20
openstackgerritStephen Finucane proposed openstack/nova master: tests: Correctly mock out security groups in NeutronFixture  https://review.opendev.org/68592710:20
*** ociuhandu has joined #openstack-nova10:20
openstackgerritMatthew Booth proposed openstack/nova master: libvirt: Ignore DiskNotFound during update_available_resource  https://review.opendev.org/68539110:21
*** jaosorior has joined #openstack-nova10:24
gibistephenfin: do I understand the situation that the nova net base sample tests was not covered the sec group case and now that you add neutron based sample tests which has sec group you discovered a bug in the api behavior?10:25
stephenfinthat seems to be my understanding10:25
stephenfin*that is my understanding :)10:25
stephenfingibi: Basically, I'm not sure if https://review.opendev.org/685927 is correct or not10:26
stephenfinI mean, the addition of that field to two APIs10:26
stephenfinI guess I will need to look at a real cloud with a down cell to see what actually comes back10:26
stephenfinOr spend ages trying to trace stuff :(10:27
*** udesale has quit IRC10:27
*** tetsuro has quit IRC10:28
*** udesale has joined #openstack-nova10:28
gibiyour fixture is coded with the assumption that neutron returns at least the default security group for every instance. If this assumption correct then what I think what you see in the test is the actual nova behavior10:28
gibiif that leads to unexpecte API response then that is a bug10:29
stephenfinSort. I think it's more like I'm assuming an instance has at least one security group regardless of its state10:29
gibistephenfin: yeah your view is more precies you would need to validate that assumption first10:30
gibiI think neutron behavior should not be cell dependent10:30
stephenfinYeah, neither do I10:31
stephenfinI've no idea where to start on validating that assumption though so ima wait til sean-k-mooney or mriedem are about :)10:31
gibi:)10:31
openstackgerritStephen Finucane proposed openstack/nova master: docs: Remove 'adv-config', 'system-admin' subdocs  https://review.opendev.org/68440210:34
openstackgerritStephen Finucane proposed openstack/nova master: docs: Replacing underscores with dashes  https://review.opendev.org/68592910:34
gibiin devstack if I do nothing special I get the default sec group10:34
*** tetsuro has joined #openstack-nova10:34
stephenfinYeah, that's what I was seeing too10:35
stephenfingibi: https://docs.openstack.org/nova/pike/admin/security-groups.html10:36
stephenfin"All projects have a default security group which is applied to any instance that has no other defined security group. Unless you change the default, this security group denies all incoming traffic and allows only outgoing traffic to your instance."10:36
gibiafter I stop the n-cond-cell1 service in my devstack I still see the default sec group returned by nova10:37
gibistephenfin: based on that doc your assumption is valid10:38
stephenfingibi: Just discussed with slaweq on #openstack-neutron. While it's possible to create ports with no security, that's not the default and none of our tests are checking for that, so expecting the default security group seems sane10:45
stephenfinLeft comments on https://review.opendev.org/#/c/685927/ to that effect10:45
stephenfinI think we still have a bug though, in as far as there are times where the security_groups key is omitted rather than being set to an empty list or null. I need to discuss with gmann or mriedem later10:46
gibiOK, cool10:47
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_servers' functional tests  https://review.opendev.org/68433610:51
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_hosts' functional tests  https://review.opendev.org/68433710:51
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_hypervisors' functional tests  https://review.opendev.org/68434010:51
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_attach_interfaces' functional tests  https://review.opendev.org/68434110:51
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_simple_tenant_usage' functional tests  https://review.opendev.org/68434210:51
openstackgerritStephen Finucane proposed openstack/nova master: trivial: Neutron fixture cleanup  https://review.opendev.org/68434310:51
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_floating_ips' functional tests  https://review.opendev.org/68434410:51
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Add TODOs for remaining nova-network functional tests  https://review.opendev.org/68434510:51
*** macz has joined #openstack-nova10:52
*** bbowen_ has quit IRC10:52
*** mkrai has quit IRC10:54
openstackgerritOpenStack Proposal Bot proposed openstack/nova stable/train: Imported Translations from Zanata  https://review.opendev.org/68553410:56
*** macz has quit IRC10:57
*** artom has joined #openstack-nova10:59
*** udesale has quit IRC11:05
*** udesale has joined #openstack-nova11:06
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Use nova-net explicitly in functional tests  https://review.opendev.org/68432611:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_availability_zone' functional tests  https://review.opendev.org/68433211:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_floating_ip_pools' functional tests  https://review.opendev.org/68433311:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_quota_sets' functional tests  https://review.opendev.org/68433411:13
openstackgerritStephen Finucane proposed openstack/nova master: tests: Correctly mock out security groups in NeutronFixture  https://review.opendev.org/68592711:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_server_tags' functional tests  https://review.opendev.org/68433511:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_servers' functional tests  https://review.opendev.org/68433611:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_hosts' functional tests  https://review.opendev.org/68433711:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_rescue' functional tests  https://review.opendev.org/68433911:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_hypervisors' functional tests  https://review.opendev.org/68434011:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_attach_interfaces' functional tests  https://review.opendev.org/68434111:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_simple_tenant_usage' functional tests  https://review.opendev.org/68434211:13
openstackgerritStephen Finucane proposed openstack/nova master: trivial: Neutron fixture cleanup  https://review.opendev.org/68434311:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_floating_ips' functional tests  https://review.opendev.org/68434411:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Add TODOs for remaining nova-network functional tests  https://review.opendev.org/68434511:13
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Remove explicit 'USE_NEUTRON = True'  https://review.opendev.org/68593711:13
*** ttsiouts has joined #openstack-nova11:16
*** udesale has quit IRC11:17
lyarwoodstephenfin: https://github.com/lyarwood/nova/blob/a51921fe195ba6d8ccf60941b12c07ed304496da/nova/tests/functional/libvirt/base.py#L102-L105 - I don't understand what you're suggesting here11:25
lyarwoodstephenfin: AssertionError: 16 not greater than 16 : Host.get_online_cpus is only accounting for 16 CPUs but you're requesting 16; change the mock or your test currently showing up in https://review.opendev.org/#/c/682621/2 as a result11:26
lyarwoodstephenfin: https://review.opendev.org/#/c/682621/2/nova/tests/functional/regressions/test_bug_1843639.py@57 - should I be setting some additional things in the HostInfo I'm providing?11:27
*** tetsuro has quit IRC11:28
*** tssurya has quit IRC11:31
*** tssurya_ has joined #openstack-nova11:31
lyarwoodstephenfin: I don't see why this has to be greater11:39
*** markvoelker has joined #openstack-nova11:49
*** markvoelker has quit IRC11:54
openstackgerritLee Yarwood proposed openstack/nova master: DNM libvirt: extract _start_heterogeneous_computes  https://review.opendev.org/68262011:56
openstackgerritLee Yarwood proposed openstack/nova master: Add regression test for bug #1843639  https://review.opendev.org/68262111:56
openstackbug 1843639 in OpenStack Compute (nova) "libvirt: post_live_migration failures to disconnect volumes result in the rollback of live migrations" [Medium,In progress] https://launchpad.net/bugs/1843639 - Assigned to Lee Yarwood (lyarwood)11:56
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Ignore volume exceptions during post_live_migration  https://review.opendev.org/68262211:56
*** bbowen has joined #openstack-nova12:03
openstackgerritBalazs Gibizer proposed openstack/nova master: Add a way to spy on function calls in test  https://review.opendev.org/68594912:13
openstackgerritBalazs Gibizer proposed openstack/nova master: Replace mock with spy in test  https://review.opendev.org/68595012:14
*** brinzhang_ has joined #openstack-nova12:21
*** brinzhang has quit IRC12:24
*** lpetrut has quit IRC12:24
*** ricolin has joined #openstack-nova12:32
*** trident has quit IRC12:34
stephenfinlyarwood: That should be assertGreaterEqual12:34
stephenfinlyarwood: It exists because of this https://github.com/lyarwood/nova/blob/a51921fe195ba6d8ccf60941b12c07ed304496da/nova/tests/functional/libvirt/base.py#L67-L6912:34
stephenfinThat's saying cores 0-15 (so 16 cores) are online12:35
*** trident has joined #openstack-nova12:35
lyarwoodstephenfin: yeah understood, couldn't see why it was just greater tbh.12:35
lyarwoodstephenfin: thanks12:35
*** brinzhang has joined #openstack-nova12:35
*** brinzhang_ has quit IRC12:39
*** tetsuro has joined #openstack-nova12:42
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_servers' functional tests  https://review.opendev.org/68433612:43
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_hosts' functional tests  https://review.opendev.org/68433712:43
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_rescue' functional tests  https://review.opendev.org/68433912:43
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_hypervisors' functional tests  https://review.opendev.org/68434012:43
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_attach_interfaces' functional tests  https://review.opendev.org/68434112:43
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_simple_tenant_usage' functional tests  https://review.opendev.org/68434212:43
openstackgerritStephen Finucane proposed openstack/nova master: trivial: Neutron fixture cleanup  https://review.opendev.org/68434312:43
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Migrate 'test_floating_ips' functional tests  https://review.opendev.org/68434412:43
openstackgerritStephen Finucane proposed openstack/nova master: nova-net: Add TODOs for remaining nova-network functional tests  https://review.opendev.org/68434512:43
stephenfingibi: Could you push this through, please? https://review.opendev.org/#/c/662505/412:44
*** tetsuro has quit IRC12:47
*** gbarros has joined #openstack-nova12:49
gibistephenfin: donstedone12:50
* gibi hates mobilenemobilenet12:50
stephenfin:D12:50
stephenfinthanks12:50
*** francoisp has joined #openstack-nova12:52
*** macz has joined #openstack-nova12:53
*** Luzi has quit IRC12:55
*** macz has quit IRC12:57
*** jaosorior has quit IRC12:58
*** rcernin has quit IRC12:58
*** jaosorior has joined #openstack-nova13:02
*** nweinber has joined #openstack-nova13:06
*** dpawlik has quit IRC13:12
*** jawad_axd has joined #openstack-nova13:14
*** dave-mccowan has joined #openstack-nova13:15
*** mriedem has joined #openstack-nova13:15
*** ratailor__ has quit IRC13:19
*** dave-mccowan has quit IRC13:19
*** trident has quit IRC13:22
*** trident has joined #openstack-nova13:23
*** BjoernT has joined #openstack-nova13:26
*** dpawlik has joined #openstack-nova13:29
*** awalende has quit IRC13:38
*** awalende has joined #openstack-nova13:39
*** ociuhandu has quit IRC13:42
openstackgerritStephen Finucane proposed openstack/nova master: docs: Rewrite host aggregate, availability zone docs  https://review.opendev.org/66713313:42
stephenfinbauzas: Can you review that today, pretty please? ^ It keeps going into merge conflict and I'm sick of it \o/13:42
*** ivve has joined #openstack-nova13:42
*** ociuhandu has joined #openstack-nova13:42
*** awalende has quit IRC13:43
bauzasstephenfin: I can try, rushing to provide a new patch for the audit command13:43
stephenfinbauzas: Nah, if you've something higher priority do that13:43
bauzascustomers, customers, customers13:46
*** ociuhandu has quit IRC13:48
*** markvoelker has joined #openstack-nova13:50
*** jaosorior has quit IRC13:51
mriedemdansmith: for versioned objects tests, you know how we have a local and remote version with a mixin that usually holds the actual tests run by both? if the method being tested isn't remotable but is just some classmethod, is there any value in having it part of the mixin that gets run by the remote test since it's not remotable? seems like duplication for no gain.13:52
mriedemiow, is it fine to just put it in the local test version (not the mixin)?13:52
dansmithmriedem: correct13:52
*** jawad_axd has quit IRC13:54
*** markvoelker has quit IRC13:55
openstackgerritMatt Riedemann proposed openstack/nova master: Add Selection.availability_zone field  https://review.opendev.org/68580713:56
*** markvoelker has joined #openstack-nova13:57
*** markvoelker has quit IRC14:01
bauzasgibi: thanks for your comments on https://review.opendev.org/#/c/670112/5/nova/cmd/manage.py14:01
bauzasgibi: I see you tested on your devstack, cool14:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add Selection.availability_zone field  https://review.opendev.org/68580714:02
bauzasgibi: if you don't disagree, I'll make some modifications for your comments and try to see the cell-targeted issue14:02
*** artom has quit IRC14:03
*** doude has joined #openstack-nova14:08
*** ociuhandu has joined #openstack-nova14:10
*** awalende has joined #openstack-nova14:16
*** awalende has quit IRC14:21
*** markvoelker has joined #openstack-nova14:22
*** ttsiouts has quit IRC14:25
*** ttsiouts has joined #openstack-nova14:25
*** markvoelker has quit IRC14:26
*** markvoelker has joined #openstack-nova14:27
*** ttsiouts has quit IRC14:30
*** munimeha1 has joined #openstack-nova14:31
*** mkrai has joined #openstack-nova14:31
stephenfinmriedem, lyarwood: Can one of you merge the stable/stein tox/gitreview patches for novaclient, please https://review.opendev.org/#/c/683626/ https://review.opendev.org/#/c/683625/14:34
*** tbachman has joined #openstack-nova14:34
openstackgerritEric Fried proposed openstack/nova-specs master: Add 'Core Liaison' section to spec template  https://review.opendev.org/68585714:34
lyarwoodstephenfin: can do14:35
lyarwoodstephenfin: done14:35
stephenfinthanks (y)14:36
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/train: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598014:36
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/stein: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598114:37
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/rocky: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598314:38
mriedemstephenfin: i have an itch saying we should have a release note for that if you're going to backport it14:39
mriedemit's proper hardening but also changes something from silently pass to outright fail14:39
stephenfinYeah, good call. I can put one together for master now, but should I squash the release notes into the backport or keep them separate?14:40
mriedemi'm fine with squashing in the backports14:40
stephenfinack14:40
*** lpetrut has joined #openstack-nova14:40
*** mlavalle has joined #openstack-nova14:42
*** dpawlik has quit IRC14:43
openstackgerritStephen Finucane proposed openstack/python-novaclient master: Add release note for bug 1845322  https://review.opendev.org/68598614:45
openstackbug 1845322 in python-novaclient train "'--hint' argument to 'nova boot ' should be a key-value pair" [Low,In progress] https://launchpad.net/bugs/1845322 - Assigned to Stephen Finucane (stephenfinucane)14:45
*** lpetrut has quit IRC14:47
gibibauzas: sure, go ahead14:48
*** ttsiouts has joined #openstack-nova14:49
gibibauzas: if you push a new PS I can try to run that in my devstack with bandwidth14:49
stephenfinmriedem: 'upgrade' or 'fixes' reno? I've gone with the former but could go with the latter14:49
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/train: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598014:54
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/stein: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598114:55
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/rocky: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598314:55
*** artom has joined #openstack-nova14:57
mriedemstephenfin: i initially thought 'fixes' but could see 'upgrade' as well14:57
*** artom has quit IRC14:57
openstackgerritBalazs Gibizer proposed openstack/nova master: Add a way to spy on function calls in test  https://review.opendev.org/68594914:57
*** artom has joined #openstack-nova14:58
openstackgerritBalazs Gibizer proposed openstack/nova master: Replace mock with spy in test  https://review.opendev.org/68595014:59
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/train: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598014:59
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/stein: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598114:59
openstackgerritStephen Finucane proposed openstack/python-novaclient stable/rocky: Stop silently ignoring invalid 'nova boot --hint' options  https://review.opendev.org/68598315:00
*** tbachman has quit IRC15:02
*** tbachman has joined #openstack-nova15:04
*** BjoernT has quit IRC15:12
*** tetsuro has joined #openstack-nova15:12
*** lpetrut has joined #openstack-nova15:13
*** tetsuro has quit IRC15:17
*** ociuhandu has quit IRC15:17
*** BjoernT has joined #openstack-nova15:18
*** ociuhandu has joined #openstack-nova15:18
*** ttsiouts has quit IRC15:22
*** ociuhandu has quit IRC15:24
*** ociuhandu has joined #openstack-nova15:25
openstackgerritMerged openstack/nova master: ec2: Move ec2utils functions to their callers  https://review.opendev.org/66250515:25
*** ociuhandu has quit IRC15:30
*** ociuhandu has joined #openstack-nova15:31
stephenfingibi: Since you looked at the functional change, care to send this reno on its way, please? https://review.opendev.org/68598615:32
*** tesseract has quit IRC15:33
gibistephenfin: sure, looking15:33
stephenfinThanks15:33
*** gyee has joined #openstack-nova15:34
artomgibi, a couple of -1s on your spy stack, but they're really more for visibility/discussion15:34
gibistephenfin: that was an easy one :)15:34
stephenfingibi: hang on, pretty sure I've a 1kloc patch sitting around here somewhere ;)15:35
*** itlinux has joined #openstack-nova15:35
gibiartom: yeah, that is basically an experiment, so any discussion starter is warmly welcome. I will check back to that patch later15:35
*** lpetrut has quit IRC15:35
gibistephenfin: I'm sure you have one :)15:35
stephenfinI probably do :(15:35
*** ociuhandu has quit IRC15:36
artomgibi, ack, no rush :)15:38
*** BjoernT has quit IRC15:39
*** damien_r has quit IRC15:39
openstackgerritMerged openstack/python-novaclient stable/train: Update .gitreview for stable/train  https://review.opendev.org/68362515:41
efriedmriedem, stephenfin: Can I get some help/advice moving forward with unauthed versioned discovery? https://review.opendev.org/68518115:41
efriedI've got this working, but there are a couple of points I'd like to discuss.15:42
mriedemstill no alex or john around huh15:42
*** ociuhandu has joined #openstack-nova15:42
efriedAlex is on vacation. John I haven't seen in a while.15:42
*** rpittau is now known as rpittau|afk15:42
stephenfinI can try, but this is not my wheelhouse15:43
efriedoh, sure you're fine removing an unused middleware though15:43
efried:P15:43
efriedso first15:43
stephenfindeleting code is a significantly easier endeavour ;)15:43
efriedmordred: your advice would be helpful here too15:44
*** xek has quit IRC15:44
*** BjoernT has joined #openstack-nova15:44
efriedhttp://host:port/v2 and /v2.1 used to do a 302 redirect to .../v2/ and /v2.1/ (with the trailing slash). The trailing-slash version then returned the version document with a 200.15:44
*** xek has joined #openstack-nova15:44
efriedNow /v2, /v2.1, /v2/, and /v2.1/ all just return the version document with 200.15:44
efriedIs that a behavior change that could possibly cause a problem for anyone?15:45
efriedperhaps gmann could also advise?15:45
mordredefried: it shouldn't cause a problem for SDK - we treat the version document payload as authoritative15:46
*** ociuhandu has quit IRC15:46
mordredbut - if there are people who are doing blind url appending - maybe that's a behavior change that would break them?15:46
mriedemi want to say novaclient has had to deal with the trailing slash stuff to get the version document and avoid the 30215:47
efriedwhat do you mean, blind url appending?15:47
mriedemi think sdague added that and alex is probably familiar with it15:47
efriedmriedem: so worst case novaclient is doing an extra thing that's no longer necessary, rite?15:47
mriedemprobably, details are in here https://github.com/openstack/python-novaclient/blob/003ac57d9af74aa4658a7bf6cc6b6b3bafa58c11/novaclient/v2/versions.py#L3615:48
efriedmordred: /v2/stuff will still go into the real API, not hit version discovery15:48
mordredah - yeah15:48
*** BjoernT_ has joined #openstack-nova15:48
*** BjoernT has quit IRC15:49
mordredI mean - then in that case, I think it's no problem - because what a person is looking for is the version document15:49
mordredand they'll be getting it15:49
mriedemi also remember this craziness that andreykurilin fixed https://github.com/openstack/python-novaclient/blob/003ac57d9af74aa4658a7bf6cc6b6b3bafa58c11/novaclient/v2/versions.py#L8315:49
mordredI agree with "novaclient might be doing an extra step it doesn't need to do anymore" part15:49
stephenfinefried: I assume there's a very good reason that it's not possible to retain the 302 redirect?15:49
*** ociuhandu has joined #openstack-nova15:49
mordredstephenfin: bunny rabbits15:50
efriedmriedem: yeah, so that novaclient blip would become unnecessary - but only for deployments actually running this code, so I'm not gonna like rip it out right away or anything.15:50
mriedemmore like ever15:50
mordredyah- that's why we're fixing sdk to DTRT with older nova too15:50
mriedemapi-paste is config and deployments can do whatever, even if it's dumb15:51
mriedemsee the comments in there about rax15:51
efriedstephenfin: I suppose with some extra hacking I could figure out a way to re-add the 302, but if it's not problematic to remove it, this is way simpler, less code, etc.15:51
stephenfingotcha15:52
efriedmriedem: that second novaclient blip should also continue to work fine.15:52
mriedemi have no idea if external tooling things would rely on that 302 actually happening15:52
efriedyeah, that would be just... crazy I would think.15:52
efriedbut it's what I'm asking.15:52
efriedSecond thing:15:52
efriedDo we have any need to retain full compatibility for [api]auth_strategy=noauth2 in the wild?15:53
*** ivve has quit IRC15:55
efriedI had to change NoAuthMiddleware a little bit because it used to be relying (hardcoded) on the auth_url being the versioned endpoint -- i.e. http://server:port/v2 or .../v2.1 -- because it would return that in the X-Server-Management-Url header.15:55
efriedBut the whole point of the change was to make those URLs *not* go through auth middleware15:55
efriedso I had to make it so you use a different auth_url, and NoAuthMiddleware (still hardcoded) massages it appropriately so you still get the right X-Server-Management-Url header back.15:56
mriedemnoauth is for tests only15:57
efriedgood, that's what I wanted to hear.15:58
mriedemhttps://docs.openstack.org/nova/latest/configuration/config.html#api.auth_strategy15:58
mriedemit's also in the config option help15:58
efriedYeah, I just didn't know if the fact that we mention it in the conf help means we kinda have to support it and not break existing uses15:58
efriedCool, so the last thing:15:59
efriedand this is obvious, because it's the point of the patch:15:59
efriedYou no longer go through auth to get the versioned discovery documents15:59
mriedemWHAT?!15:59
efriedI think it's possible this means there are some headers you used to get back that you wouldn't anymore.16:00
efriedWe seem to have tests for things we care about, like Accept.16:02
efriedI suppose it would be duly diligent of me to go find out exactly what that diff looks like...16:02
*** macz has joined #openstack-nova16:04
efriedthanks for the talk mriedem mordred stephenfin16:05
openstackgerritMerged openstack/python-novaclient stable/train: Update TOX/UPPER_CONSTRAINTS_FILE for stable/train  https://review.opendev.org/68362616:10
openstackgerritMerged openstack/python-novaclient master: Add release note for bug 1845322  https://review.opendev.org/68598616:11
openstackbug 1845322 in python-novaclient train "'--hint' argument to 'nova boot ' should be a key-value pair" [Low,In progress] https://launchpad.net/bugs/1845322 - Assigned to Stephen Finucane (stephenfinucane)16:11
*** ociuhandu has quit IRC16:11
openstackgerritMatt Riedemann proposed openstack/nova master: Add Selection.availability_zone field  https://review.opendev.org/68580716:12
openstackgerritMatt Riedemann proposed openstack/nova master: Handle get_host_availability_zone error during reschedule  https://review.opendev.org/68599716:12
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional regression test for bug 1781286  https://review.opendev.org/68599816:12
openstackbug 1781286 in OpenStack Compute (nova) "CantStartEngineError in cell conductor during reschedule - get_host_availability_zone up-call" [Medium,Triaged] https://launchpad.net/bugs/1781286 - Assigned to Matt Riedemann (mriedem)16:12
*** lpetrut has joined #openstack-nova16:22
*** ociuhandu has joined #openstack-nova16:25
openstackgerritBalazs Gibizer proposed openstack/nova master: Do not print default dicts during heal_allocations  https://review.opendev.org/68600116:26
*** ociuhandu has quit IRC16:29
gibimriedem: finally ticked off this TODO as well ^^16:29
mriedemack, would be nice to see that in https://review.opendev.org/#/c/669879/ but i won't make you rebase it16:32
mriedemmaybe time to nudge efried and/or stephenfin to come back on ^16:32
mriedemthough stephen had comments16:32
mriedemunrelated, while writing a resize reschedule test i realize that if we fail the resize claim we temporarily put the instance into ERROR state while rescheduling, and it will end up that way if rescheduling fails, which seems wrong b/c we didn't actually do anything with the guest, we just failed a claim16:33
mriedemdansmith: agree? ^16:33
*** ricolin has quit IRC16:33
dansmithagree with what? that leaving it in error state is wrong?16:34
mriedemyeah16:34
mriedemwe should just fail the resize, mark the migration as error, inject a fault, etc16:34
mriedembut the vm_state should still be active or stopped16:34
dansmithI dunno, I guess I thought it was always a design goal that ERROR is your indication that something failed, and you start or stop the instance to get it back to normal state16:35
mriedemthat's a pretty big hammer for a user when resize fails b/c they lost a claim race16:36
mriedemand the guest is otherwise untouched16:36
mriedemthe instance action will tell you as a user if the thing failed16:36
dansmithisn't that why we can go from ACTIVE->ERROR->ACTIVE and the user can do the start operation from ERROR state?16:36
gibimriedem: ack, I will check back to the nova-next patch tomorrow16:36
mriedemyou can't start from error state16:37
mriedemyou can hard reboot16:37
dansmithyeah, but you have to either poll it or something to know when to look right?16:37
* gibi leaves for today16:37
mriedemsure, poll until the task_state is reset to None16:37
mriedemif the status is not what you expect, investigate the instance action16:37
dansmither, I really thought you could start to get out of error state16:38
dansmithI had this long conversation with vish about it eons ago16:38
dansmithalthough admittedly lots could have changed since then16:38
mriedemhard reboot would do it but then you've hard to hard reboot your guest even though there is nothing wrong with it16:38
mriedem*had to16:38
dansmithno, I mean start or stop being a no-op from whatever real state it was in16:38
efriedmriedem: RC2 candidate, merged in master, wouldyouplease https://review.opendev.org/#/c/685742/16:39
dansmithmriedem: well, stop works from error at least: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L240116:39
efriedand yes, on https://review.opendev.org/#/c/669879/ I'm waiting for stephenfin's questions to be answered.16:40
*** BjoernT has joined #openstack-nova16:40
*** dtantsur is now known as dtantsur|afk16:40
mriedemdansmith: ok, but still, the user shouldn't have to stop and then start to get out of that, or hard reboot, when the thing that failed is a resize claim race16:40
dansmithmriedem: so maybe it's just stop I'm thinking of.. anyway, I dunno.. it's very annoying as a user to do something, come back later and have it not obvious that the thing has happened, or failed or whatever16:41
dansmithmriedem: if you're going to retry the operation for them, I agree. if you're not, then being super obvious about what has happened is best, IMHO16:41
*** BjoernT_ has quit IRC16:42
aspierssean-k-mooney: got a response from AMD SEV guy16:42
sean-k-mooneycool do we need to take acount of the qemu channels?16:42
aspiersHe didn't say so I'm following up explicitly on that16:43
aspiersSee my comment16:43
aspiershttps://review.opendev.org/#/c/684825/4/nova/virt/libvirt/designer.py@21416:43
sean-k-mooneyok ya so we are missing the gpu supprot in the video model section16:44
aspiersI'll add that now16:45
sean-k-mooneyand i think we do need the channels too since the channesl allow you to create virtio-serial deivces16:45
sean-k-mooneya qemu channel has two parts a host backend and a guest front end16:46
sean-k-mooneythe guest front end is typeically a virtio-serial port16:46
*** imacdonn has quit IRC16:48
aspiersoops, the attribute for GuestVideo is 'type' not 'model'16:48
sean-k-mooneywe call it model in the image property16:48
sean-k-mooneybut maybe16:49
sean-k-mooneyoh its the type atribute in the model elemnt16:49
aspiersyeah16:49
sean-k-mooneythat does not mean the class varible is called type16:50
aspiersno but it is anyway16:50
aspierswhich IMHO is a bad choice because type is kind of reserved, but whatever16:50
sean-k-mooneyya it is type https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L185916:50
sean-k-mooneywell its self.type16:51
sean-k-mooneytype as a variable would alisa the type fucntion which would be bad16:51
aspierssure16:51
sean-k-mooneyin class scope its kindo fo ok16:51
aspiersemacs Python mode is too stupid to realise the difference16:51
sean-k-mooneyya i can see that messing with alot of regex based syntax highlighting16:52
sean-k-mooneyso ya the guest agent is defulting to a virtio channel16:53
sean-k-mooneyhttps://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L2135-L215216:53
aspierssean-k-mooney: vvvvvvv16:54
openstackgerritAdam Spiers proposed openstack/nova master: Also enable iommu for virtio controllers and video in libvirt  https://review.opendev.org/68482516:54
aspiersIt may need another PS for virtio channels, waiting to see what Brijesh says16:54
sean-k-mooneyya i would keep that seperate16:54
aspiersOK16:55
sean-k-mooneyalthough you proably could jsut try enableing the qemu guest agent too and see if it works16:55
aspiershow do I do that?16:55
sean-k-mooneyyou need to install it in the guest image and set an image property let me get it16:55
aspiersurgh16:56
aspiersby install, do you mean at build-time not post launch?16:56
sean-k-mooneyyou can do it post launch i think16:56
sean-k-mooneyyou set hw_qemu_guest_agent=yes in the image metadata16:57
sean-k-mooneythat will be enough to triger the generation of the qemu channel16:57
aspiersok I'll try that thanks16:57
sean-k-mooneythen you should be able to  ssh in and install it16:57
aspiersI wonder what the package will be called16:57
sean-k-mooneyon suse i have no idea16:58
aspierswhat is it on RH?16:58
sean-k-mooneyproably qemu-somthing16:58
aspiershaha no shit :)16:58
sean-k-mooneyi dont  know16:58
sean-k-mooneyi have only ever installed it on ubunutu16:58
sean-k-mooneyyum install qemu-guest-agent16:58
sean-k-mooneyhttps://wiki.libvirt.org/page/Qemu_guest_agent16:59
*** derekh has quit IRC17:01
mnasermriedem: did the cross-cell resizes end up finishing in train?17:03
aspierssean-k-mooney: it's the same on SUSE17:03
mnaserlooks like it just all still needs review and probably isnt landing in train, from what i see?17:03
sean-k-mooneyaspiers: its a pretty good name for it17:03
aspierssean-k-mooney: very catchy ;-)17:03
sean-k-mooneyaspiers: the main reason it exists is to provide a way to quiesce the diesk when doing an online snapshot17:04
sean-k-mooneyit has other uses too17:04
aspierssean-k-mooney: does it do more than just a sync? since stuff could still be writing17:04
sean-k-mooneyi think it does a flush then prevents any addtional wirtes untill the snapshot completes17:05
aspiersah ok17:05
aspiershow does it prevent? some kernel voodoo?17:06
sean-k-mooneyaspiers: dansmith told me it was important to not currupt things and i took him at his word17:06
sean-k-mooneyi havent really looked in to how it does it but i assume its acting as a broker between qemu and the guest kernel block deice layer in some way17:07
sean-k-mooneythis covers part of it https://www.sebastien-han.fr/blog/2015/02/09/openstack-perform-consistent-snapshots-with-qemu-guest-agent/17:12
openstackgerritMerged openstack/nova master: api: Remove 'Debug' middleware  https://review.opendev.org/66250617:13
mriedemmnaser: not done in train no17:15
mriedemmnaser: do you have multiple cells yet?17:15
sean-k-mooneyaspiers: apparently you have to write your own freeze hooks17:15
*** tbachman has quit IRC17:20
*** tssurya_ has quit IRC17:22
*** ralonsoh has quit IRC17:27
mnasermriedem: nope not yet17:30
*** lpetrut has quit IRC17:31
*** tbachman has joined #openstack-nova17:32
openstackgerritMatt Riedemann proposed openstack/nova master: Handle get_host_availability_zone error during reschedule  https://review.opendev.org/68599717:33
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional regression test for build part of bug 1781286  https://review.opendev.org/68599817:33
openstackbug 1781286 in OpenStack Compute (nova) "CantStartEngineError in cell conductor during reschedule - get_host_availability_zone up-call" [Medium,In progress] https://launchpad.net/bugs/1781286 - Assigned to Matt Riedemann (mriedem)17:33
openstackgerritMatt Riedemann proposed openstack/nova master: Add Selection.availability_zone field  https://review.opendev.org/68580717:33
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional regression test for migrate part of bug 1781286  https://review.opendev.org/68601717:33
*** igordc has joined #openstack-nova17:37
mriedemhemna: fyi that https://review.opendev.org/#/c/683008/ nor anything on stable/pike will pass tempest jobs until https://review.opendev.org/#/c/684769/ or an alternative fix is merged17:41
*** ivve has joined #openstack-nova17:41
*** mkrai has quit IRC17:43
mriedemmelwitt: when you get a chance can you hit these stable/stein backports? https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:stable/stein+topic:bug/184309017:46
mriedemthose are an upgrade regression in stein so i'd like to get those into the next point release17:46
openstackgerritEric Fried proposed openstack/nova master: Allow versioned discovery unauthenticated  https://review.opendev.org/68518117:50
efriedmriedem, stephenfin, mordred: The response headers were indeed different, fixed ^17:52
efriedready for your enthusiastic +2s now.17:52
*** artom has quit IRC17:55
aspiersOK here's a really dumb question - why am I getting this after updating my devstack? Unknown column 'migrations.user_id' in 'field list17:58
aspiersI did a db sync and restarted services17:58
mriedemyou likely didn't sync cell117:59
aspiersah thanks17:59
mriedemnova-manage db sync --config-file /etc/nova/nova.conf --config-file /etc/nova/nova-cond_cell1.conf18:00
mriedemsomething like that18:00
aspierstrying ...18:00
mriedemformer only syncs cell0, latter syncs cell118:00
* mriedem jumps on a call18:00
aspiersmriedem: nova-manage --config-file /etc/nova/nova.conf --config-file /etc/nova/nova_cell1.conf db sync was the winning combo, thanks!18:05
mriedem\o/18:07
*** artom has joined #openstack-nova18:10
*** psachin has quit IRC18:10
melwittmriedem: sure18:10
*** psachin has joined #openstack-nova18:13
openstackgerritMerged openstack/nova stable/train: make virtual pmem feature compatible with python3  https://review.opendev.org/68574218:13
*** artom has quit IRC18:15
*** artom has joined #openstack-nova18:17
*** psachin has quit IRC18:17
sean-k-mooneyefried: can i mark this as fix released https://bugs.launchpad.net/nova/+bug/171537418:25
openstackLaunchpad bug 1715374 in OpenStack Compute (nova) "Reloading compute with SIGHUP prevents instances from booting" [High,In progress] - Assigned to Ralf Haferkamp (rhafer)18:25
sean-k-mooneyit should be in rc1 right as of https://github.com/openstack/nova/commit/73f1fda7e93325d3ca2b2c62afb17fe011d650f318:25
efriedsean-k-mooney: Yeah, that makes sense, please do.18:26
sean-k-mooneycool im doing some downstream bug traige and linking them to the upstream ones18:29
mnaserhmm18:34
mnaserhow nova-friendly is https://docs.openstack.org/releasenotes/zun/train.html#upgrade-notes ?18:34
mnaser[compute]/host_shared_with_nova = true18:34
*** ociuhandu has joined #openstack-nova18:34
mnaserso looks like its integrated with placement and gets the rp that nova uses and makes claims against it18:35
sean-k-mooney i assume that is in the zun.conf18:35
sean-k-mooneynot the nova.conf18:35
mnaserwhich i (thinking out loud) don't think that it can do bad things(tm)18:36
mnaserbecause placement would just not return that compute host as a likely candidate at that point18:36
sean-k-mooneyam it depends18:36
sean-k-mooneyit could do bad things if you are using cpu pinning or hugepages18:37
*** ociuhandu has quit IRC18:37
mnasertrue, i guess its mostly focusing around ram and cpu claims (blindly)18:37
sean-k-mooneywell if they claim VCPU not PCUP then they woudl be required to enusre that the contaiern swapen on the cpu_shared_set18:38
sean-k-mooneyif they do not then thants a problemn18:38
*** ociuhandu has joined #openstack-nova18:39
mnaserlooks like its claiming VCPU18:39
mnaserbut i think it might not know to pin workloads18:40
mnaserdont even think thats possible with docker18:40
sean-k-mooneyit is18:40
sean-k-mooneyyou can pass a cpuset18:40
mnaserhttps://github.com/openstack/zun/commit/7b3b1c5e07eb8b9e81031fa6ee4625ff6870b41d18:40
mnaserthis is the big commit and i dont think they have that (yet)18:40
sean-k-mooneyya so without that it would not be safe to run zun on a nova host that has a PCPU inventory18:41
sean-k-mooneys/safe/correct/ as the contianer could steal cpu time from the pinned instnace18:41
sean-k-mooneymnaser: i also see this being an issue if we delet the nova compute service on that host but not the zun service18:44
mnaseryeah i think that's an operator problem at that point because setting the zun setting would be invalid18:44
sean-k-mooneyunless they intentally took acount of that18:44
sean-k-mooneynot really18:44
mnaserbut then i think if you swap out that setting in flight18:44
sean-k-mooneyif i deploy with both18:44
mnaseryou probably lose all your allocation18:44
sean-k-mooneyand then later wanted to remvoe nova to just have zun18:44
sean-k-mooneythen wen we delete the compute node rp as part of service deleteion it might cause issue for zuun18:45
sean-k-mooneyright we will try to delete all the allcoation against the compute node ro18:45
sean-k-mooney*RP18:45
*** ociuhandu has quit IRC18:46
sean-k-mooneyi dont recal the zun team discussing this with the nova team at the denver PTG did i miss a session on this?18:46
*** ociuhandu has joined #openstack-nova18:46
mnaseryeah i think i remember talking about it with dansmith saying "it would be cool if" but i think they mentioned that "nova assumes it owns the rp, there is no such thing as rp sharing"18:47
sean-k-mooneymnaser: right as of today it does18:48
sean-k-mooneyback in the dublin ptg i was suggestign nova should not own the compute node RP18:48
mnaserok so compute node rp sharing is a thing that isn't an antipattern18:48
sean-k-mooneyand each service should have there own nested RP subtree runder a shared compute node RP with no inventories18:48
sean-k-mooneymnaser: right now it not a thing18:49
sean-k-mooneywell not really a thing im not 100% sure that neutorn bandwith based schduleing is not shareing an RP with nova18:50
sean-k-mooneygibi: ^ do you know the answer to that18:50
sean-k-mooneymnaser: im not aware of share where not is not the owner of the compute node RP.18:50
*** brinzhang_ has joined #openstack-nova18:51
*** brinzhang_ has joined #openstack-nova18:51
*** brinzhang_ has joined #openstack-nova18:52
*** brinzhang has quit IRC18:54
*** tbachman has quit IRC18:54
sean-k-mooneylooking at the zun patch tehy seam to have ported alot of code form nova which is not nessisarlly a good thing.18:54
efriedmnaser, sean-k-mooney: For some definition of "sharing". It doesn't matter who creates the RP (neutron vs nova for bw, cyborg vs nova for accels, etc) -- the important thing is that, if you're creating an instance, the entire *claim* needs to be atomic, and therefore done by nova.18:55
sean-k-mooneyor zun18:55
efriedas long as nova doesn't think it's going to come in afterwards and do some other claim, sure.18:56
sean-k-mooneywell zun has its own top level api18:56
efriedI don't really know what zun is or what it does18:56
sean-k-mooneyits a container as a service project that started as a replacment for nova-docker18:57
efriedI do recall seeing they were porting in a bunch of nova's resource tracker and scheduler gorp18:57
sean-k-mooneyso it can be used as a nova virt driver or as a standalone service18:57
efriedbasically inheriting a mountain of technical debt18:57
efriedand we told them not to do that18:57
efriedand they did it anyway.18:57
sean-k-mooneyyes hence my comment above18:57
*** tbachman has joined #openstack-nova18:58
sean-k-mooneywhen i see patches merge with TODO comments from jaypipes i assume that was not a recent TODO and there for form nova18:59
sean-k-mooneythe bit that worres my is fi the share with nova flag is set19:00
*** maciejjozefczyk has quit IRC19:01
sean-k-mooneythe pass the full provider tree for the nova compute node RP to there update_provider_tree function19:02
sean-k-mooneythey are also modifing trait possible on the root RP19:03
*** ociuhandu has quit IRC19:07
*** spsurya has quit IRC19:09
mriedemsean-k-mooney: bw providers are nested under the compute node rp19:17
mriedemthe case that sean-k-mooney mentioned about nova deleting the compute node and root resource provider when you delete a compute service, i can definitely see that screwing with zun if they assume the same resource provider with the same uuid is going to exist, i don't know how they link that up, maybe via hostname19:18
mriedemwe have our own problems managing that stuff, i.e. the orphaned allocations/providers bug we have right now,19:19
mriedemlet alone worrying about how we're going to be breaking external things19:19
efriedmriedem: Did you have something to do with https://review.opendev.org/#/c/684326/ not being combined with its (current) successor https://review.opendev.org/#/c/685937/ ?  (USE_NEUTRON refactoring in tests)19:19
sean-k-mooneyya so depending on the valude of "share with nova" they either create there own root RP or share nova19:19
sean-k-mooneyand they use the hostname to look up the one created by nova if "shareing" is enabled19:20
mriedemefried: you can see my comments on PS1, i don't know why it changed like it did, i haven't looked yet, but it definitely isn't what i was expecting19:20
mriedemefried: he had https://review.opendev.org/#/c/684326/119:20
mriedemand then in the next patch in the series reverted a bunch of what he added with https://review.opendev.org/#/c/684327/19:20
efriedmriedem: yeah, the earlier comments have been addressed to squash together a now-abandoned change19:20
mriedemi said just squash those19:20
mriedemsure, he did but then there were other things tacked on that i wasn't expecting19:21
efriedI'm just wondering why there's still two patches, one that sets False and the other that removes True19:21
mriedemand haven't taken the time to grok the diff19:21
*** markvoelker has quit IRC19:21
mriedemiow, i said "just do x" and he did x + 419:21
* sean-k-mooney sees nova net and walk away before i have to grok nova net code.19:22
mriedembtw, unrelated to this, but we have another train rc2 fix on master with a +2 and ack from the bug reporter that it fixes the regression https://review.opendev.org/#/c/685194/19:22
efriedo, I was reviewing that and then... didn't finish for some reason.19:23
efriedstand by.19:23
sean-k-mooneyi was going to say we shoudl navoe nova serial testing in the gate but it actuly just i nthe unit tests19:24
sean-k-mooneygetting spice testing should be pretty simple we jus tneed to pick one of our exisitg jobs to swap over ot it19:25
sean-k-mooneyim not sure if devstack deploy nova-serial19:25
sean-k-mooneyit proably does but i have never used it19:25
sean-k-mooneymelwitt: ^19:25
*** artom has quit IRC19:26
efriedmriedem, melwitt: +A19:27
mriedemhot19:27
sean-k-mooneyif we change  greande-py3 to nova-serial-proxy  and sawped nova-greade-multione to spice and kept the neturon-greade multinoe as the default novnc it should cover all three?19:28
efriedabove my pay grade, sounds like a melwitt question.19:38
zigoHeya ! Got my first VM running on Train with all the RC1 packaged and uploaded to Debian already ! :)19:42
zigoIt's never been that smooth ... :P19:42
* mriedem waits for the "but x is broken"19:46
mriedemzigo: that's good news, thanks for sharing19:46
mriedemwe'll have an RC2, just fyi19:46
zigomriedem: For the moment, there's no "but x is broken" part, as much as I know... :P19:46
zigoStill waiting for Glance RC1 though, it still not out.19:47
zigoLooks like it Glance from Stein is enough. :)19:47
*** bbowen has quit IRC19:49
*** brinzhang has joined #openstack-nova19:54
*** brinzhang_ has quit IRC19:57
*** slaweq has quit IRC19:59
*** nweinber has quit IRC20:00
*** markvoelker has joined #openstack-nova20:00
*** brinzhang_ has joined #openstack-nova20:02
*** brinzhang has quit IRC20:05
*** tbachman has quit IRC20:11
*** tbachman has joined #openstack-nova20:20
mnaserhmm20:23
mnaseri remember there was the rabbitmq api stuff we talked about a while ago20:23
mnaseris it possible that at some point if nova heartbeats get messed up, all processes seem to be forever stuck (heartbeat misses accumulate)20:23
mnaser>missed heartbeats from client, timeout: 60s20:24
mnaserwith some "client unexpectedly closed TCP connection"20:25
mnaserand a whole bunch of: Unexpected error during heartbeart thread processing, retrying...: ConnectionForced: Too many heartbeats missed20:25
mnaserbut also "Unexpected error during heartbeart thread processing, retrying...: error: [Errno 104] Connection reset by peer" (assuming this is when rabbitmq kills it cause it hasnt gotten heartbeat)20:25
mnaserand for context this is a very busy api so its not sitting idle20:25
mriedemare you thinking of this? https://review.opendev.org/#/c/662095/5/releasenotes/notes/eventlet-monkey-patch-5f734ef581aa550e.yaml20:26
mnaserhmm, threads=1 in this deployment (osa with uwsgi)20:27
mnaserand again this is a quite busy api endpoint so i dont imagine it sat idle for more than.. 5-10s at a time20:27
mriedemmaybe check this out https://review.opendev.org/#/c/663074/20:28
mnasersimple math shows 1000 requests per last 26 minutes so 38 requests per minute.. doubt thats enough to trigger something lik ethis20:28
mnaserhmm20:31
mnaserit'll be hard to run that20:31
mnaserbut im also a bit curious why its triggering here at all20:31
*** pcaruana has quit IRC20:33
mnaseralso we run HEAD / every few seconds too20:36
mriedemefried: explain to me why this wouldn't work in a test:20:43
mriedem1. do something,20:43
mriedem2. patch x20:43
mriedem3. do something else20:43
mriedem4. stop the patch on20:43
mriedemx20:43
mriedem5. do another thing that hits the real thing that x was patching20:43
*** xek has quit IRC20:43
mriedem5 is blowing up b/c it doesn't appear that stop is stopping the patch20:43
mriedemseems like it should work given https://docs.python.org/3.6/library/unittest.mock.html#patch-methods-start-and-stop20:44
mriedemcalling mock.patch.stopall() seems to do what 4 isn't20:46
*** gbarros has quit IRC20:48
mnaserwelp, it restarted happenign again not long after a service restart20:48
mriedemmaybe a bug in the mock library20:50
mriedemanyway, i have a workaround, ignore me20:52
efriedmriedem: without seeing the code...20:58
efriedcan I see the code?20:58
efriedCause what you're describing seems like it ought to work20:58
efriedmriedem: a classic blunder is calling stop() on the *mock* (rather than the patcher)21:00
mriedemyeah it wasn't that21:00
mriedemp = mock.patch('x')21:00
mriedemp.start()21:00
mriedem...21:00
mriedemp.stop()21:00
mriedemx() -> kablammo21:01
*** tbachman has quit IRC21:01
mriedemerr, p = mock.patch('x', side_effect=kablammo)21:01
mriedemthreads could be at play somehow, idk21:03
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional regression test for build part of bug 1781286  https://review.opendev.org/68599821:03
openstackbug 1781286 in OpenStack Compute (nova) "CantStartEngineError in cell conductor during reschedule - get_host_availability_zone up-call" [Medium,In progress] https://launchpad.net/bugs/1781286 - Assigned to Matt Riedemann (mriedem)21:03
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional regression test for migrate part of bug 1781286  https://review.opendev.org/68601721:03
openstackgerritMatt Riedemann proposed openstack/nova master: Add Selection.availability_zone field  https://review.opendev.org/68580721:03
openstackgerritMatt Riedemann proposed openstack/nova master: Set Instance AZ from Selection AZ during build reschedule  https://review.opendev.org/68604721:03
mriedemif you're a glutton for punishment, what i was doing was at this point https://review.opendev.org/#/c/686047/1/nova/tests/functional/regressions/test_bug_1781286.py@7221:04
efriedIn [8]: foo()21:04
efriedreal21:04
efriedIn [9]: p.start()21:04
efriedOut[9]: <MagicMock name='foo' id='139999791540112'>21:04
efriedIn [10]: foo()21:04
mriedemi changed that to be something like:21:04
efriedpatched21:04
efriedIn [11]: p.stop()21:04
efriedIn [12]: foo()21:04
efriedreal21:04
mriedemself.agg_mock = mock.patch(...)21:04
mriedemself.agg_mock.start()21:04
mriedemand then later after fake_notifier.wait_for_versioned_notifications where i have mock.patch.stopall(), i had self.agg_mock.stop()21:04
mriedemand that wasn't working21:04
efriedbecause you subsequently called get_by_host?21:05
efriedas opposed to build_and_run_instance?21:06
mriedemthe test goes on to call self._wait_for_state_change which hits the API which hit the thing i had previously mockd to fail yeah21:06
efriedcause if you had called build_and_run_instance again, it would have reinstated your mock.21:06
efriedbut stop_all would have reverted your mock of build_and_run_instance21:07
mriedemwhat probably happened is there were 2 mocks21:07
mriedemand i only stopped one21:07
efriedmm, because two instances?21:07
mriedemhttps://github.com/testing-cabal/mock/blob/master/mock/mock.py#L153321:07
mriedem2 calls to build_and_run_instance21:07
mriedem2 hosts21:07
mriedemit's a reschedule test21:07
efriedthat'd do it :)21:07
mriedemyup, so stopall to the rescue21:08
efriedwell, except that's going to stop... *all* of your mocks.21:08
efriedo21:08
*** ociuhandu has joined #openstack-nova21:08
efriedonly the ones you started with start()21:08
efriedbrittle21:08
mriedemthat's fine for this test; anyway, thanks for making me think through it - now i can explain why i'm using stopall when artom -1s my change21:08
* mriedem adds the comment now21:09
openstackgerritMatt Riedemann proposed openstack/nova master: Set Instance AZ from Selection AZ during build reschedule  https://review.opendev.org/68604721:10
*** bbowen has joined #openstack-nova21:11
*** ociuhandu has quit IRC21:12
*** gbarros has joined #openstack-nova21:32
*** openstackstatus has quit IRC21:32
openstackgerritMatt Riedemann proposed openstack/nova master: Set Instance AZ from Selection AZ during migrate reschedule  https://review.opendev.org/68605021:33
*** openstackstatus has joined #openstack-nova21:35
*** ChanServ sets mode: +v openstackstatus21:35
openstackgerritMatt Riedemann proposed openstack/nova master: Set Instance AZ from Selection AZ during migrate reschedule  https://review.opendev.org/68605021:36
openstackgerritMatt Riedemann proposed openstack/nova master: Update cells v2 up-call caveats doc  https://review.opendev.org/68605321:46
*** tbachman has joined #openstack-nova21:49
*** munimeha1 has quit IRC21:51
openstackgerritGhanshyam Mann proposed openstack/nova-specs master: Re-propose policy-defaults-refresh spec for Ussuri  https://review.opendev.org/68605821:55
*** mriedem has quit IRC21:56
openstackgerritmelanie witt proposed openstack/nova stable/stein: Reduce scope of 'path' query parameter to noVNC consoles  https://review.opendev.org/68606322:08
*** luksky has quit IRC22:10
efriedmelwitt: did you miss the train cherry-pick?22:14
efriedor did I?22:14
melwittefried: no, I missed it. sorry, I forgot the branch was cut22:15
melwittwill fix22:15
efriedat least it's clean :P22:15
openstackgerritmelanie witt proposed openstack/nova stable/train: Reduce scope of 'path' query parameter to noVNC consoles  https://review.opendev.org/68606622:16
openstackgerritmelanie witt proposed openstack/nova stable/stein: Reduce scope of 'path' query parameter to noVNC consoles  https://review.opendev.org/68606322:17
openstackgerritmelanie witt proposed openstack/nova stable/rocky: Reduce scope of 'path' query parameter to noVNC consoles  https://review.opendev.org/68606722:22
*** tkajinam has joined #openstack-nova22:50
*** BjoernT has quit IRC23:04
*** macz has quit IRC23:05
*** macz has joined #openstack-nova23:07
*** macz has quit IRC23:10
*** gbarros has quit IRC23:25
*** mlavalle has quit IRC23:31
*** BjoernT has joined #openstack-nova23:33
*** rcernin has joined #openstack-nova23:33
openstackgerritMerged openstack/nova stable/stein: Make nova.compute.rpcapi.ComputeAPI.router a singleton  https://review.opendev.org/68440523:36
openstackgerritMerged openstack/nova stable/stein: Func test for migrate reschedule with pinned compute rpc  https://review.opendev.org/68440623:36
openstackgerritMerged openstack/nova stable/stein: Handle legacy request spec dict in ComputeTaskManager._cold_migrate  https://review.opendev.org/68440723:36
openstackgerritMerged openstack/nova master: Reduce scope of 'path' query parameter to noVNC consoles  https://review.opendev.org/68519423:36
*** markvoelker has quit IRC23:38
*** bbowen_ has joined #openstack-nova23:39
*** bbowen has quit IRC23:41
*** gbarros has joined #openstack-nova23:42
*** markvoelker has joined #openstack-nova23:44
*** ivve has quit IRC23:47

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!