| *** tetsuro has quit IRC | 00:24 | |
| *** tetsuro has joined #openstack-placement | 00:26 | |
| *** edmondsw has quit IRC | 00:37 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (6) https://review.openstack.org/574113 | 01:34 |
|---|---|---|
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (7) https://review.openstack.org/574974 | 01:35 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (8) https://review.openstack.org/575311 | 02:30 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (9) https://review.openstack.org/575581 | 02:30 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (10) https://review.openstack.org/576017 | 04:06 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (11) https://review.openstack.org/576018 | 04:06 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (12) https://review.openstack.org/576019 | 04:07 |
| openstackgerrit | Merged openstack/nova master: Make nova list and migration-list ignore down cells https://review.openstack.org/575734 | 05:07 |
| *** e0ne has joined #openstack-placement | 05:13 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.openstack.org/576020 | 05:44 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.openstack.org/576027 | 05:45 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.openstack.org/576031 | 05:45 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.openstack.org/576299 | 05:46 |
| *** tssurya has joined #openstack-placement | 05:56 | |
| *** openstackgerrit has quit IRC | 06:04 | |
| *** e0ne has quit IRC | 06:08 | |
| *** openstackgerrit has joined #openstack-placement | 06:15 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.openstack.org/576344 | 06:15 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.openstack.org/576673 | 06:15 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.openstack.org/576676 | 06:16 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.openstack.org/576689 | 06:16 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.openstack.org/576709 | 06:16 |
| openstackgerrit | Deepak Mourya proposed openstack/nova master: [WIP] Handle CannotDeleteParentResourceProvider to 409 Conflict https://review.openstack.org/577726 | 06:35 |
| *** nicolasbock has joined #openstack-placement | 06:40 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.openstack.org/576712 | 06:40 |
| *** peereb has joined #openstack-placement | 06:48 | |
| *** peereb has quit IRC | 06:49 | |
| *** peereb has joined #openstack-placement | 06:49 | |
| *** peereb has quit IRC | 06:50 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add TODO note for mox removal https://review.openstack.org/576758 | 06:52 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add TODO note for mox removal https://review.openstack.org/576758 | 06:54 |
| openstackgerrit | sahid proposed openstack/nova master: hardware: fix memory check when hugepages requested https://review.openstack.org/532168 | 07:10 |
| openstackgerrit | sahid proposed openstack/nova stable/queens: add mtu to libvirt xml for ethernet and bridge types https://review.openstack.org/578010 | 07:30 |
| openstackgerrit | Chason Chan proposed openstack/nova master: Update install guide for placement database configuration https://review.openstack.org/578013 | 07:41 |
| openstackgerrit | zhangyangyang proposed openstack/nova master: Remove ChanceScheduler https://review.openstack.org/576384 | 07:44 |
| *** ttsiouts has joined #openstack-placement | 07:51 | |
| *** e0ne has joined #openstack-placement | 08:05 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/578019 | 08:08 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/pike: libvirt: slow live-migration to ensure network is ready https://review.openstack.org/559032 | 08:36 |
| openstackgerrit | Chris Dent proposed openstack/nova master: Use nova.db.api directly https://review.openstack.org/543262 | 08:40 |
| *** cdent has joined #openstack-placement | 08:57 | |
| openstackgerrit | Surya Seetharaman proposed openstack/nova-specs master: Handling a down cell https://review.openstack.org/557369 | 08:58 |
| alex_xu | cdent: a question for https://review.openstack.org/#/c/577915/1/nova/api/openstack/placement/handlers/allocation.py@320, if we failed the creation of allocation, the consumer generation can't be rollback, it is on purpose? | 08:59 |
| cdent | let me think about that a sec alex_xu , brb | 09:00 |
| alex_xu | cdent: yea, thanks | 09:00 |
| cdent | alex_xu: the increment of the generation doesn't happen until the end of the create_all. In my patch the issue is that when you create a new consumer it's generation becomes 0, and yes, at that point there is a problem if the allocations fail, in part because of: https://bugs.launchpad.net/nova/+bug/1778591 | 09:03 |
| openstack | Launchpad bug 1778591 in OpenStack Compute (nova) "GET /allocations/{uuid} on a consumer with no allocations provides no generation" [Medium,Triaged] | 09:03 |
| openstackgerrit | do3meli proposed openstack/nova master: docs: add nova host-evacuate command to evacuate documentation https://review.openstack.org/578040 | 09:04 |
| cdent | we also need to add a test for: generation is now 0, but there are still no allocations, because the first attempt failed, so now what | 09:04 |
| cdent | sigh. this is going to be complicated | 09:04 |
| cdent | I think there is a suite of interlocking problems | 09:05 |
| alex_xu | cdent: yea, that actually isn't your patch problem | 09:06 |
| cdent | If I get time today, alex_xu, I'll try to write up a summary of all the bits that seem off. | 09:07 |
| alex_xu | cdent: and we have window when iterate the allocation for each RPs | 09:07 |
| alex_xu | if there is another request with a consumer generation in the middle of the first request. | 09:07 |
| alex_xu | but sounds ok, it just got an unexpected generation increase. | 09:08 |
| cdent | when efried and edleafe come around we probably need to go over this stuff as a group with a fine tooth comb as the edge cases seem...weird | 09:10 |
| alex_xu | cdent: ok, I just commented on the patch | 09:11 |
| cdent | excellent, thank you | 09:11 |
| cdent | alex_xu: did you save your comment, I haven't see it yet (or maybe you are still reviewing) | 09:20 |
| alex_xu | cdent: not yet, still writing... | 09:20 |
| cdent | ✔ | 09:22 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/ocata: libvirt: slow live-migration to ensure network is ready https://review.openstack.org/559034 | 09:26 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: [placement] api-ref: add traits patameter https://review.openstack.org/578048 | 09:26 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: [placement] api-ref: add traits parameter https://review.openstack.org/578048 | 09:27 |
| alex_xu | cdent: done | 09:28 |
| cdent | thanks | 09:30 |
| *** takashin has left #openstack-placement | 09:36 | |
| *** cdent has quit IRC | 10:09 | |
| *** ttsiouts has quit IRC | 10:14 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack/nova stable/pike: mock utils.execute() in qemu-img unit test https://review.openstack.org/578058 | 10:19 |
| openstackgerrit | sahid proposed openstack/nova stable/queens: add mtu to libvirt xml for ethernet and bridge types https://review.openstack.org/578010 | 10:29 |
| *** cdent has joined #openstack-placement | 10:31 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Rename '_get_phynet_info' https://review.openstack.org/577233 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Rename 'create_pci_requests_for_sriov_ports' https://review.openstack.org/575488 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Always retrieve network information if available https://review.openstack.org/564444 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Retrieve tunneled status in '_get_physnet_info' https://review.openstack.org/564445 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add NUMACell.network_info https://review.openstack.org/564439 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add NUMATopologyLimits.networks https://review.openstack.org/575486 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Start accounting for networks in NUMA placement https://review.openstack.org/564448 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add RequestSpec.numa_networks https://review.openstack.org/564442 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: scheduler: Start utilizing RequestSpec.numa_networks https://review.openstack.org/564452 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Add '[neutron] physnets' and related options https://review.openstack.org/564440 | 10:38 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start populating NUMACell.network_info field https://review.openstack.org/564441 | 10:38 |
| openstackgerrit | Vladyslav Drok proposed openstack/nova master: rbd: flatten images when creating/unshelving an instance https://review.openstack.org/457886 | 11:01 |
| *** ttsiouts has joined #openstack-placement | 11:02 | |
| *** tetsuro has quit IRC | 11:09 | |
| openstackgerrit | Merged openstack/nova master: Bump keystoneauth1 minimum to 3.9.0 https://review.openstack.org/577431 | 12:02 |
| openstackgerrit | Merged openstack/nova master: Adapter raise_exc=False by default https://review.openstack.org/577437 | 12:03 |
| openstackgerrit | Chason Chan proposed openstack/nova master: Fix the duplicated config options of api_database and placement_database https://review.openstack.org/577023 | 12:10 |
| *** ttsiouts has quit IRC | 12:25 | |
| openstackgerrit | Merged openstack/nova master: fix PowerVM get_bootdisk_path docstring https://review.openstack.org/576977 | 12:37 |
| openstackgerrit | Merged openstack/nova master: api-ref: Fix parameters about trusted certificate IDs https://review.openstack.org/576046 | 12:37 |
| openstackgerrit | Merged openstack/nova master: Update links in README https://review.openstack.org/574644 | 12:37 |
| cdent | jaypipes: if you're back in the game, there's some consumer generation related confusion that you might find interesting. couple of related bugs: https://bugs.launchpad.net/nova/+bug/1778576 https://bugs.launchpad.net/nova/+bug/1778591 and some irc discussion: http://p.anticdent.org/4jnh and (earlier, the start of the discussion) http://p.anticdent.org/28FY | 12:44 |
| openstack | Launchpad bug 1778576 in OpenStack Compute (nova) "making new allocations for one consumer against multiple resource providers fails with 409" [High,In progress] - Assigned to Chris Dent (cdent) | 12:44 |
| openstack | Launchpad bug 1778591 in OpenStack Compute (nova) "GET /allocations/{uuid} on a consumer with no allocations provides no generation" [Medium,Triaged] | 12:44 |
| jaypipes | cdent: ack, thanks for the heads up. | 12:46 |
| cdent | jaypipes: I'm hoping to have enough time today to sort of do an exploration+summary but forecast unclear | 12:47 |
| jaypipes | cdent: k, will try to get through those bugs and reading this morning. | 12:48 |
| cdent | cool, thanks | 12:48 |
| *** ttsiouts has joined #openstack-placement | 12:49 | |
| edleafe | cdent: have to leave for a dr appt in about 30 min, but I think that as long as you have two places where consumer records can be created, you will have this issue | 12:50 |
| cdent | edleafe: I think there is more than one issue, so which "this issue" do you mean? | 12:51 |
| *** mriedem has joined #openstack-placement | 12:51 | |
| edleafe | cdent: the code in https://review.openstack.org/#/c/564641/2/nova/api/openstack/placement/objects/resource_provider.py may be very if-elsey, but I think that it is what is needed | 12:51 |
| edleafe | The "loop" problem | 12:51 |
| * cdent takes note | 12:53 | |
| cdent | I suspect someone, maybe me, needs to go through and delineate and tease out the issues as clearly as possible, because we're all doing that thing where we move from one to another fluidly because they are sort of, but not quite, the same | 12:54 |
| cdent | I _think_ there are at least 3 issues, but I'm not certain | 12:54 |
| jaypipes | cdent: you're probably not surprised to learn I disagree with edleafe on this. | 12:56 |
| *** edmondsw has joined #openstack-placement | 12:58 | |
| cdent | given that i'm still not quite sure on what's happening, i'm refraining from forming any opinions | 12:58 |
| efried | jaypipes: Beyond that bug, I think the most immediate thing we need your eyeballs on is the last nrp-alloc-cands patch: https://review.openstack.org/#/c/565487/ | 13:05 |
| jaypipes | efried: k | 13:05 |
| efried | cdent: Going to be leaning heavily on your review for https://review.openstack.org/#/c/578013/ | 13:13 |
| cdent | noted, thanks | 13:13 |
| efried | It seems like it may be oversimplified - but maybe that's what we want. | 13:13 |
| efried | i.e. tell them one way to do it, even if there's really three. | 13:14 |
| cdent | yeah, I was just going to ask something along those lines. The other thing I'm curious about is: do you know if these are _only_ fresh install instructions? | 13:16 |
| efried | I don't know, no. | 13:19 |
| cdent | sigh. I'm currently in the mental state where I need a to do list to manage my to do lists. Which I think might mean I need a walk or something. | 13:19 |
| efried | heh | 13:20 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove support for /os-fixed-ips REST API https://review.openstack.org/568516 | 13:21 |
| *** e0ne has quit IRC | 13:39 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix API reference for os-floating-ips-bulk https://review.openstack.org/578106 | 13:39 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix API reference for os-floating-ips-bulk https://review.openstack.org/578106 | 13:41 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix API reference for os-floating-ip-dns https://review.openstack.org/578107 | 13:41 |
| *** e0ne has joined #openstack-placement | 13:44 | |
| openstackgerrit | xulei proposed openstack/nova master: Disable limits if force_hosts or force_nodes is set https://review.openstack.org/576693 | 13:46 |
| openstackgerrit | xulei proposed openstack/nova master: Disable limits if force_hosts or force_nodes is set https://review.openstack.org/576693 | 13:48 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Update API reference for os-floating-ip-pools https://review.openstack.org/578108 | 13:48 |
| mriedem | efried: cdent: when i wrote the bug report for that, the install guide changes were pretty trivial i think | 13:54 |
| mriedem | 1. create placement db | 13:54 |
| mriedem | 2. config nova.conf | 13:54 |
| mriedem | the rest is already in there: nova-manage api_db sync | 13:55 |
| mriedem | will sync the placement db once configured | 13:55 |
| cdent | mriedem: yeah, I think the confusion efried and I were having was whether that was sufficient given some of the other things one _might_ do | 13:56 |
| mriedem | ooo i think i see a bug | 14:02 |
| mriedem | cdent: this should be using version2 https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L859 | 14:02 |
| cdent | mriedem: hmmm. seems so. I reckon that's a merge error oversight. The change to using version2 is newer than the optional db stuff | 14:05 |
| mriedem | yeah i figured | 14:05 |
| mriedem | i'm also not really sure how that version2 is optional, unless that's how nargs='?' handles it, but it doesn't have a default | 14:05 |
| mriedem | must default to None? | 14:05 |
| mriedem | stephenfin: ^ | 14:05 |
| cdent | can you either make a bug or just fix it, I'm in a hole on the consumer generation weirdness, chasing rabbits | 14:05 |
| mriedem | yeah | 14:06 |
| mriedem | the lowest of hanging fruits | 14:06 |
| cdent | my current rabbit is running around saying that usage calculations are broken | 14:07 |
| cdent | i hope it is a hallucination | 14:07 |
| mriedem | https://bugs.launchpad.net/nova/+bug/1778734 | 14:08 |
| openstack | Launchpad bug 1778734 in OpenStack Compute (nova) "nova-manage api_db sync uses the wrong version option when synchronizing the placement database" [Medium,Triaged] | 14:08 |
| stephenfin | Oops | 14:09 |
| stephenfin | mriedem: I can fix that, if you want. Remove the old one too, while I'm there (separate patch) | 14:10 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix CLI docs for nova-manage api_db commands https://review.openstack.org/578116 | 14:19 |
| *** e0ne has quit IRC | 14:23 | |
| *** e0ne has joined #openstack-placement | 14:26 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Drop MIN_LIBVIRT_VHOSTUSER_MQ https://review.openstack.org/565472 | 14:29 |
| *** efried has quit IRC | 14:40 | |
| *** efried has joined #openstack-placement | 14:40 | |
| cdent | mriedem, efried, edleafe : my rabbit was not a hallucination: https://bugs.launchpad.net/nova/+bug/1778743 | 14:47 |
| openstack | Launchpad bug 1778743 in OpenStack Compute (nova) "When POSTing to /allocations with multiple consumers it is possible violate inventory capacity constraints" [Undecided,New] | 14:47 |
| mriedem | :( | 14:50 |
| cdent | i assume it is beacuse we are in an unflushed transaction | 14:50 |
| cdent | for each allocation, there is capacity, but there isn't in aggregate | 14:50 |
| openstackgerrit | Merged openstack/nova master: Ironic update_provider_tree: restore traits override https://review.openstack.org/576877 | 14:56 |
| openstackgerrit | Merged openstack/nova stable/queens: ironic: stop lying to the RT when ironic is down https://review.openstack.org/575628 | 14:56 |
| *** rubasov has quit IRC | 15:01 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Skip ServerShowV247Test.test_update_rebuild_list_server in nova-cells-v1 job https://review.openstack.org/578125 | 15:03 |
| openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Add test demonstrating bug 1778743 https://review.openstack.org/578126 | 15:03 |
| openstack | bug 1778743 in OpenStack Compute (nova) "When POSTing to /allocations with multiple consumers it is possible violate inventory capacity constraints" [Medium,Triaged] https://launchpad.net/bugs/1778743 | 15:03 |
| *** e0ne has quit IRC | 15:07 | |
| *** e0ne has joined #openstack-placement | 15:11 | |
| efried | cdent: Did you open a bug for "leaky consumers"? | 15:13 |
| cdent | efried: no I mentioned it one of the other bugs I created as a factor, but didn't open a specific bug for that. there are a few more to come as well, once I pop my way out of this rabbit hole | 15:14 |
| efried | cdent: I'll open that one. | 15:15 |
| openstackgerrit | Chris Dent proposed openstack/nova master: WIP: [placement] Fix capacity tracking in POST /allocations https://review.openstack.org/578133 | 15:30 |
| *** ttsiouts has quit IRC | 15:36 | |
| *** ttsiouts has joined #openstack-placement | 15:43 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Always retrieve network information if available https://review.openstack.org/564444 | 15:43 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Retrieve tunneled status in '_get_physnet_info' https://review.openstack.org/564445 | 15:43 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add NUMACell.network_affinity https://review.openstack.org/564439 | 15:43 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add NUMATopologyLimits.networks https://review.openstack.org/575486 | 15:43 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Start accounting for networks in NUMA placement https://review.openstack.org/564448 | 15:43 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add RequestSpec.numa_networks https://review.openstack.org/564442 | 15:43 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: scheduler: Start utilizing RequestSpec.numa_networks https://review.openstack.org/564452 | 15:43 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Add '[neutron] physnets' and related options https://review.openstack.org/564440 | 15:43 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start populating NUMACell.network_info field https://review.openstack.org/564441 | 15:43 |
| mriedem | cdent: just a typo in https://review.openstack.org/#/c/578126/ otherwise i'd +2 | 15:48 |
| cdent | mriedem: yeah, I fixed that in the followup and then forgot to fix it in the rebase | 15:48 |
| cdent | i'm assuming the followup will need some tweakage to get to happiness | 15:49 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: conf: libvirt: Make `/dev/urandom` the default for 'rng_dev_path' https://review.openstack.org/577385 | 15:57 |
| openstackgerrit | Chris Dent proposed openstack/nova master: [placement] demonstrate part of bug 1778591 with a gabbi test https://review.openstack.org/578139 | 15:58 |
| openstack | bug 1778591 in OpenStack Compute (nova) "GET /allocations/{uuid} on a consumer with no allocations provides no generation" [Medium,Triaged] https://launchpad.net/bugs/1778591 | 15:58 |
| openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Add test demonstrating bug 1778743 https://review.openstack.org/578126 | 16:08 |
| openstackgerrit | Chris Dent proposed openstack/nova master: WIP: [placement] Fix capacity tracking in POST /allocations https://review.openstack.org/578133 | 16:08 |
| openstack | bug 1778743 in OpenStack Compute (nova) "When POSTing to /allocations with multiple consumers it is possible violate inventory capacity constraints" [Medium,In progress] https://launchpad.net/bugs/1778743 - Assigned to Chris Dent (cdent) | 16:08 |
| cdent | mriedem: fixed the typo, the child remains wippy | 16:09 |
| *** e0ne has quit IRC | 16:09 | |
| *** e0ne has joined #openstack-placement | 16:13 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Use instance project/user when creating RequestSpec during resize reschedule https://review.openstack.org/577929 | 16:14 |
| openstackgerrit | Merged openstack/nova master: xenapi: drop deprecated vif_driver config option https://review.openstack.org/567877 | 16:14 |
| cdent | efried: did you create that bug you were planning to create? After my explorations today I think there are going to be quite a few more. | 16:23 |
| efried | I was partway through composing when I got an interrupt. Resuming now. | 16:23 |
| cdent | i know how that can be | 16:23 |
| openstackgerrit | Merged openstack/nova master: network: Rename '_get_phynet_info' https://review.openstack.org/577233 | 16:26 |
| openstackgerrit | Merged openstack/nova master: network: Rename 'create_pci_requests_for_sriov_ports' https://review.openstack.org/575488 | 16:26 |
| efried | cdent: What's the problem with deleting the consumer when its last allocation goes away? | 16:26 |
| cdent | I don't have a problem with that, it's just a significant change to the way the code is currently organized | 16:27 |
| efried | cdent: No possibility of weird races or anything? | 16:27 |
| efried | New guy comes in with generation=null and succeeds, but that's okay innit? | 16:28 |
| cdent | I think so, yes | 16:28 |
| efried | it's a behavior change from today, where the new guy would have to use the proper, current generation. But I'm not seeing any race or other aberrant behavior arising from that. | 16:28 |
| cdent | because we use ints there's some potential for a '1' to not mean the same as another '1' | 16:28 |
| efried | mm, that's right. | 16:29 |
| efried | okay, yeah, that one is valid.. | 16:29 |
| cdent | which is a hard one to get roun | 16:29 |
| cdent | d | 16:29 |
| efried | I suppose it is. Unless we start using random hash/uuid generations, which wouldn't be the worst idea. | 16:31 |
| cdent | "significant change" | 16:34 |
| cdent | but yeah, that woul be one way | 16:34 |
| cdent | I'll likely have some real ideas tomorrow. At the moment I'm pretty washed up | 16:34 |
| efried | cdent: https://bugs.launchpad.net/nova/+bug/1778763 | 16:34 |
| openstack | Launchpad bug 1778763 in OpenStack Compute (nova) "Consumers never get deleted" [Undecided,New] | 16:34 |
| *** e0ne has quit IRC | 16:35 | |
| efried | cdent: for bug 1778743, you would never get such an allocation record as an allocation_candidate, right? It's only if you create it by hand and POST it? | 16:40 |
| openstack | bug 1778743 in OpenStack Compute (nova) "When POSTing to /allocations with multiple consumers it is possible violate inventory capacity constraints" [Medium,In progress] https://launchpad.net/bugs/1778743 - Assigned to Chris Dent (cdent) | 16:40 |
| cdent | I believe that's correct, but POST /allocations was created for things like migrations, which manuipulate allocations in interesting ways. And besides you'd never want to use POST /allocations with something from /allocation_candidates (I hope) because there's only one consumer involved there. | 16:41 |
| cdent | I was _so_ close to have zero patches in progress | 16:48 |
| openstackgerrit | Surya Seetharaman proposed openstack/nova stable/queens: Make nova list and migration-list ignore down cells https://review.openstack.org/578152 | 16:54 |
| *** e0ne has joined #openstack-placement | 17:11 | |
| *** ttsiouts has quit IRC | 17:14 | |
| *** ttsiouts has joined #openstack-placement | 17:15 | |
| *** e0ne has quit IRC | 17:18 | |
| *** ttsiouts has quit IRC | 17:20 | |
| openstackgerrit | Merged openstack/nova master: Remove mox in test_conductor.py (1) https://review.openstack.org/564658 | 17:28 |
| openstackgerrit | jiang wei proposed openstack/nova master: Add action initiator attribute to the instance payload https://review.openstack.org/536243 | 17:36 |
| *** tssurya has quit IRC | 17:43 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Allow templated cell_mapping URLs https://review.openstack.org/578163 | 17:57 |
| cdent | efried: I'm going to have to be done with today at least for now. I think I've dropped most of my state either into bugs, or the starts at patches (linked to bugs). | 18:02 |
| efried | cdent: Roger that. Good digging today, sir. | 18:03 |
| cdent | too deep too deep | 18:03 |
| cdent | i left fighting words about DELETE /consumer on that bug, but other than that I mostly agree with your comments on "So the consumer is getting created, even though the overall operation fails?" | 18:04 |
| cdent | that quoted bit is the central issue in this entire suite of problems | 18:04 |
| cdent | the keystone and all that | 18:05 |
| cdent | anyway, bbl | 18:05 |
| cdent | efried: (in case you wandered off ^) | 18:05 |
| * cdent wanders off | 18:05 | |
| efried | ack | 18:05 |
| edleafe | efried: what is the concern about "leaking" (orphan) consumer records? Just DB bloat, or something else? | 18:12 |
| efried | just db bloat | 18:12 |
| edleafe | So it | 18:12 |
| efried | edleafe: Considering how many instances a "real" openstack deployment can create & destroy and how long a real db could live. | 18:12 |
| edleafe | ugh | 18:12 |
| edleafe | So it's for optimization? | 18:13 |
| efried | edleafe: I wouldn't call it optimization per se. It's like fixing a memory leak. | 18:13 |
| edleafe | I guess I'm not sure that it would be a DB hit | 18:13 |
| edleafe | Why memory? | 18:14 |
| efried | I was allegorying | 18:14 |
| edleafe | Are we planning to run in SQLite? | 18:14 |
| efried | No, I'm saying the *problem* is akin to a memory leak. You wouldn't say you're "optimizing" if you're fixing a memory leak. | 18:14 |
| efried | In this case, a <whatever storage medium is hosting your tables> leak. | 18:15 |
| edleafe | It just seems that a deployment that would have sufficient volume to make this amount of "leaky" data significant would also have a DB that can handle it | 18:15 |
| efried | That seems like a terrible reason to not solve it. | 18:15 |
| efried | though I suppose it could justify e.g. a CLI tool to "clean up" empty consumers, as opposed to a real fix. | 18:16 |
| edleafe | You could always have a maintenance that deletes all consumers w/no allocs | 18:16 |
| edleafe | jinxish | 18:16 |
| jroll | deployers asked for tooling to clean up deleted instances for a while before we had it, they're going to ask for the same for consumers if they aren't deleted :) | 18:16 |
| edleafe | jroll: sure, but instances are a lot heavier | 18:17 |
| efried | We will probably have to do that for older deployments, unless we find a way to backport whatever cleanup we decided on. | 18:17 |
| jroll | edleafe: sure, but people try to keep their DBs trim | 18:17 |
| efried | A consumer is, what, three UUIDs and an int? | 18:17 |
| edleafe | jroll: or they could just use MongoDB - it's web scale! https://www.youtube.com/watch?v=b2F-DItXtZs | 18:18 |
| jroll | just my 2c, of course | 18:18 |
| jroll | sigh | 18:18 |
| efried | What's the objection to having an actual /consumers/{u} endpoint? | 18:19 |
| edleafe | What's the use case? | 18:20 |
| efried | um | 18:21 |
| efried | PUT /consumers/{u} | 18:22 |
| efried | PUT /allocations/{u} {...} | 18:22 |
| efried | ... | 18:22 |
| efried | PUT /allocations/{u} {} | 18:22 |
| efried | DELETE /consumers/{u} | 18:22 |
| edleafe | I understand the alloc cases; I'm just wondering why we would ever need to create a consumer with no allocations | 18:22 |
| edleafe | especially since you see that as a "leak" | 18:23 |
| efried | Why do we need to create resource providers with no inventory? | 18:23 |
| edleafe | We really don't need to | 18:24 |
| efried | Arguably a shelved instance is a real consumer with no allocations. | 18:24 |
| edleafe | An RP with no inventory is also a "leak" | 18:24 |
| efried | Right. But we don't do that - we create a provider and then immediately do stuff with it. And when its life is over - but not before - we explicitly delete it. Why should consumers be different? | 18:25 |
| edleafe | But you don't create a shelved instance. You create an instance and then shelve it | 18:25 |
| efried | We create the instance before we allocate to it. | 18:25 |
| efried | We just don't happen to parallel that into placement. | 18:25 |
| efried | Look, I'm not married to the idea of symmetry for symmetry's sake. If there were a reasonable, non-awkward way to do this without the leak, I wouldn't be advocating for it. But the fact is that the way we're doing it now is messy and leaky. | 18:26 |
| efried | We've got the same problem for aggregates, but I'm accepting of the argument that we don't need to solve it there, because placement doesn't have a use case (that we know of) where aggregates are getting created and deleted at high volume and frequency. | 18:27 |
| edleafe | Oh, I agree that the way we are creating the records is messy. But you could add a check when deleting allocs to see if those were the last allocs for that consumer, and if so, delete it. | 18:28 |
| efried | yes, we could do that. | 18:28 |
| efried | I mentioned it as the first option in one of the bugs. | 18:29 |
| edleafe | I think that in this case it's best to think in non-Nova terms | 18:29 |
| edleafe | The fridge is fine, but my mental model is a library | 18:29 |
| edleafe | You can have consumers who don't happen to have any books checked out at the moment | 18:29 |
| edleafe | That would be a use case for PUT /consumers | 18:30 |
| edleafe | i.e., issue a library card | 18:30 |
| efried | Yup, I don't give my library card back every time I return my books. | 18:32 |
| jaypipes | efried: a consumer is three four ints and a UUID. but anyway, reading back, I don't agree that we should delete consumer records if/when all the consumer's allocation records are removed. I *do* agree we should clean up (delete) the consumer record if a PUT /allocations/{consumer_uuid} fails for some reason and the consumer record had already been created. | 19:22 |
| jaypipes | efried: especially since if PUT /allocations/{consumer_uuid} returns a 409, the caller will expect to retry the (same) URI and payload after re-reading allocation candidate information, and they certainly won't expect to have to pass a consumer generation :) | 19:27 |
| *** e0ne has joined #openstack-placement | 19:52 | |
| efried | jaypipes: I think we're in agreement on those points. What we haven't yet settled is how to delete consumers. (Ones that were at some point "real" - i.e. had allocations.) | 20:11 |
| efried | One view: when the consumer's last allocation goes away, delete the consumer record. | 20:12 |
| efried | Another view: consumers are now like resource providers, in that they have more than just an ID, so treat them like we treat resource providers and have specific endpoints to create and delete them. | 20:12 |
| jaypipes | efried: I definitely don't agree with deleting them after the last allocation is deleted. I do think there can/should be a objects.Consumer.delete() method that should be called from nova/api/openstack/placement/handlers/allocation.py:_set_allocations_for_consumer() on *new* consumer objects that were created with utils.ensure_consumer() but only when AllocationList.create_all() failed with a 409 Conflict due to provider generations | 20:14 |
| jaypipes | changing. | 20:14 |
| cdent | efried: did you see jaypipes says "I don't agree that we should delete consumer records if/when all the consumer's allocation records are removed." | 20:14 |
| cdent | jinx | 20:14 |
| jaypipes | :) | 20:14 |
| jaypipes | cdent: you saw my comment on the implementation patch then? | 20:14 |
| jaypipes | cdent: your patch that fixes this.. | 20:14 |
| jaypipes | cdent: well, that fixes the 500 error. | 20:15 |
| cdent | jaypipes: not yet no, I just got back from being away | 20:15 |
| *** e0ne has quit IRC | 20:15 | |
| jaypipes | k, no worries | 20:15 |
| cdent | I probably won't process much of that. digging through the several related bugs drained me and I want food and beer &c | 20:16 |
| jaypipes | cdent: completely understand. | 20:16 |
| efried | okay, so kibosh on the implicit deletion of consumer, which I dig. | 20:18 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove compatibility code for instance groups https://review.openstack.org/578192 | 20:18 |
| efried | Then we're left with three possibilities for deleting consumers: | 20:18 |
| efried | 1) Create specific endpoints for consumers | 20:18 |
| efried | 2) Create an admin tool to scrub allocation-less consumers from the db | 20:18 |
| efried | 3) Don't | 20:18 |
| efried | jaypipes: opinion? ^ | 20:19 |
| jaypipes | efried: or 4) just delete the consumer record when the AllocationList.create_all() fails for new consumers. | 20:21 |
| efried | jaypipes: No, we need to do that regardless. I'm talking about deleting consumers that really do exist. By "just" you're implying 3 | 20:22 |
| jaypipes | cdent: real quick before you leave... | 20:22 |
| jaypipes | cdent: do you have anything in progress for the GET /allocations/{consumer} with no allocations response bug? | 20:22 |
| jaypipes | cdent: if not, I'll hack something up. | 20:22 |
| jaypipes | efried: I don't really see a need to delete consumers that really do exist. | 20:23 |
| efried | jaypipes: Just keep accumulating records forever? | 20:23 |
| jaypipes | efried: at the moment at least... I suppose in the future we could add an endpoint that does that. I don't believe it's necessary to fix these particualr bugs right now though. | 20:24 |
| cdent | jaypipes: all I did on that one is https://review.openstack.org/#/c/578139/ (I think. If you look at patches I created today there are 3 or 4 in the same consumer gen world) | 20:24 |
| efried | while True: spawn(); delete() | 20:24 |
| jaypipes | cdent: k, thx will look. | 20:24 |
| cdent | If there's some kind of grand decision or outcome from discussions this evening between efried and jaypipes please put it on a bug or one of the reviews email or something, so I can soak it up in the morning | 20:27 |
| cdent | good night all | 20:27 |
| *** cdent has quit IRC | 20:28 | |
| openstackgerrit | Merged openstack/nova master: Remove support for /os-fixed-ips REST API https://review.openstack.org/568516 | 21:01 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Allow templated cell_mapping URLs https://review.openstack.org/578163 | 21:16 |
| *** mriedem has quit IRC | 21:52 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Allow templated cell_mapping URLs https://review.openstack.org/578163 | 21:53 |
| *** mriedem has joined #openstack-placement | 21:56 | |
| *** edmondsw has quit IRC | 22:04 | |
| *** edmondsw has joined #openstack-placement | 22:07 | |
| *** edmondsw has quit IRC | 22:11 | |
| *** mriedem has quit IRC | 22:21 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove unused DB API instance_group_delete method https://review.openstack.org/578223 | 22:21 |
| *** Kevin_Zheng has joined #openstack-placement | 23:05 | |
| *** nicolasbock has quit IRC | 23:06 | |
| * alex_xu wakes up begin to enjoy the consumer leaky bugy | 23:39 | |
| *** tetsuro has joined #openstack-placement | 23:45 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!