Monday, 2019-02-11

*** wolverineav has joined #openstack-nova00:21
*** markvoelker has joined #openstack-nova00:21
*** brinzhang has joined #openstack-nova00:42
*** markvoelker has quit IRC00:54
*** ileixe has joined #openstack-nova00:55
*** ileixe has quit IRC01:03
*** yikun has joined #openstack-nova01:06
*** wolverineav has quit IRC01:13
openstackgerritMerged openstack/nova stable/rocky: Fix InstanceMapping to always default queued_for_delete=False  https://review.openstack.org/63553401:14
*** vishwanathj has quit IRC01:22
*** sapd1 has joined #openstack-nova01:38
openstackgerritZhenyu Zheng proposed openstack/nova master: Don't change volume status to `in-use` when attach to shelved vm.  https://review.openstack.org/62709601:51
*** markvoelker has joined #openstack-nova01:51
*** dave-mccowan has joined #openstack-nova01:56
*** dave-mccowan has quit IRC02:00
*** fried_rice has quit IRC02:05
*** wolverineav has joined #openstack-nova02:13
*** wolverineav has quit IRC02:15
*** wolverineav has joined #openstack-nova02:15
*** markvoelker has quit IRC02:24
*** fried_rice has joined #openstack-nova02:42
*** wolverineav has quit IRC02:43
*** wolverineav has joined #openstack-nova02:46
*** fried_rice has quit IRC02:52
*** fried_rice has joined #openstack-nova02:55
*** wolverineav has quit IRC02:58
*** psachin has joined #openstack-nova03:04
*** sdake has quit IRC03:08
*** yonglihe-holiday is now known as yonglihe03:10
*** sdake has joined #openstack-nova03:11
*** markvoelker has joined #openstack-nova03:22
*** fried_rice has quit IRC03:50
*** markvoelker has quit IRC03:54
*** udesale has joined #openstack-nova04:02
*** wolverineav has joined #openstack-nova04:05
*** wolverineav has quit IRC04:10
*** hongbin has joined #openstack-nova04:25
*** janki has joined #openstack-nova04:25
*** sridharg has joined #openstack-nova04:29
*** markvoelker has joined #openstack-nova04:51
*** hongbin has quit IRC04:53
*** cfriesen has joined #openstack-nova05:09
*** markvoelker has quit IRC05:24
*** ratailor has joined #openstack-nova05:52
*** markvoelker has joined #openstack-nova06:22
*** hoonetorg has joined #openstack-nova06:22
*** moshele has joined #openstack-nova06:29
*** markvoelker has quit IRC06:55
*** Luzi has joined #openstack-nova06:55
*** toabctl has joined #openstack-nova07:26
*** jangutter has joined #openstack-nova07:33
*** psachin has quit IRC07:45
*** slaweq has joined #openstack-nova07:49
*** ociuhandu_ has quit IRC07:49
*** markvoelker has joined #openstack-nova07:52
*** ociuhandu has joined #openstack-nova07:54
*** ociuhandu has quit IRC07:58
*** awalende has joined #openstack-nova07:59
openstackgerritZhenyu Zheng proposed openstack/nova master: Support volume-backed server rebuild in compute  https://review.openstack.org/62589307:59
*** wolverineav has joined #openstack-nova08:01
*** awalende_ has joined #openstack-nova08:02
*** awalende has quit IRC08:03
*** ccamacho has joined #openstack-nova08:06
*** rpittau has joined #openstack-nova08:07
*** tesseract has joined #openstack-nova08:08
*** ralonsoh has joined #openstack-nova08:12
*** lee1 has joined #openstack-nova08:13
*** awalende_ has quit IRC08:16
*** helenafm has joined #openstack-nova08:17
*** maciejjozefczyk has joined #openstack-nova08:22
*** awalende has joined #openstack-nova08:22
*** jaosorior has joined #openstack-nova08:23
*** markvoelker has quit IRC08:24
*** xek has joined #openstack-nova08:30
*** tssurya has joined #openstack-nova08:31
*** wolverineav has quit IRC08:34
*** wolverineav has joined #openstack-nova08:35
*** finucannot is now known as stephenfin08:51
*** awalende has quit IRC08:54
openstackgerritZhenyu Zheng proposed openstack/nova master: Support volume-backed server rebuild in compute  https://review.openstack.org/62589308:54
*** bnemec-pto has quit IRC08:57
*** bnemec has joined #openstack-nova08:59
openstackgerrityenai proposed openstack/nova master: Ignore VolumeAttachmentNotFound exception in compute.manager  https://review.openstack.org/63265709:02
*** awalende has joined #openstack-nova09:07
*** wolverineav has quit IRC09:09
openstackgerritZhenyu Zheng proposed openstack/nova master: Bump API microversion to support rebuild bfv instances  https://review.openstack.org/62751809:16
*** cdent has joined #openstack-nova09:20
*** markvoelker has joined #openstack-nova09:21
*** derekh has joined #openstack-nova09:35
*** wolverineav has joined #openstack-nova09:41
*** dtantsur|afk is now known as dtantsur09:48
*** markvoelker has quit IRC09:54
*** cfriesen has quit IRC10:04
*** moshele has quit IRC10:06
*** NewBruce has joined #openstack-nova10:06
*** wolverineav has quit IRC10:14
*** NewBruce has quit IRC10:17
*** NewBruce has joined #openstack-nova10:19
*** whoami-rajat has joined #openstack-nova10:20
*** yan0s has joined #openstack-nova10:23
*** ociuhandu has joined #openstack-nova10:24
*** lee1 is now known as lyarwood10:51
*** markvoelker has joined #openstack-nova10:52
*** NewBruce has quit IRC10:54
openstackgerritLee Yarwood proposed openstack/nova master: fixtures: Return a mocked class instead of method within fake_imagebackend  https://review.openstack.org/61980410:56
*** moshele has joined #openstack-nova10:56
openstackgerritLee Yarwood proposed openstack/nova stable/queens: libvirt: Add workaround to cleanup instance dir when using rbd  https://review.openstack.org/62872610:57
openstackgerritLee Yarwood proposed openstack/nova stable/rocky: Don't emit warning when ironic properties are zero  https://review.openstack.org/60857310:57
*** NewBruce has joined #openstack-nova10:59
*** NewBruce has quit IRC11:01
*** NewBruce has joined #openstack-nova11:02
*** wolverineav has joined #openstack-nova11:11
*** erlon has joined #openstack-nova11:14
*** brinzhang has quit IRC11:21
*** NewBruce has quit IRC11:21
openstackgerritLee Yarwood proposed openstack/nova master: compute: Reject migration requests when source is down  https://review.openstack.org/62348911:23
*** markvoelker has quit IRC11:25
*** sapd1 has quit IRC11:37
*** helenafm has quit IRC11:40
*** wolverineav has quit IRC11:44
*** mdbooth has joined #openstack-nova11:46
*** NewBruce has joined #openstack-nova11:50
*** helenafm has joined #openstack-nova11:57
*** NewBruce_ has joined #openstack-nova12:03
*** NewBruce has quit IRC12:04
*** NewBruce_ is now known as NewBruce12:04
*** udesale has quit IRC12:05
*** udesale has joined #openstack-nova12:06
*** dave-mccowan has joined #openstack-nova12:13
*** markvoelker has joined #openstack-nova12:21
*** fried_rice has joined #openstack-nova12:33
*** whoami-rajat has quit IRC12:40
*** wolverineav has joined #openstack-nova12:42
cdentI seem to remember this being talked about before: is there a way to weigh the "busy-ness" of an AZ or aggregate? Is the cpuweigher a vague proxy for that?12:49
*** ratailor has quit IRC12:49
*** janki has quit IRC12:50
*** jchhatbar has joined #openstack-nova12:50
*** bauwser is now known as bauzas12:54
*** markvoelker has quit IRC12:54
*** yan0s has quit IRC12:57
*** yan0s has joined #openstack-nova12:57
*** derekh has quit IRC12:58
*** Luzi has quit IRC13:02
*** Luzi has joined #openstack-nova13:08
gibibauzas: hi! could you help me pushing through two qeens/pike backports?13:10
gibibauzas: https://review.openstack.org/#/c/623355/1 and https://review.openstack.org/#/c/635897/1,13:10
cdentwelcome back gibi !13:11
gibicdent: thanks :)13:11
*** wolverineav has quit IRC13:14
*** yan0s has quit IRC13:21
*** gbarros has joined #openstack-nova13:29
bauzasgibi: hola13:30
bauzasgibi: sure, lemme look13:30
*** moshele has quit IRC13:30
gibibauzas: thanks a lot13:30
bauzasgibi: FWIW, I'm also thinking about your comments like https://review.openstack.org/#/c/599208/12/nova/virt/libvirt/driver.py@671813:31
gibibauzas: so if the TODO means that the operator today can change the config that nova does not handle then I think we have a problem13:33
*** sapd1 has joined #openstack-nova13:34
bauzasgibi: yeah but the problem is that we don't know yet if we need to create a new child RP13:34
*** yan0s has joined #openstack-nova13:34
bauzaswe only know about the root RP13:35
bauzasand we don't persist the existing type13:35
bauzasoh and FWIW, it doesn't work also atm13:35
gibibauzas: so, what will happen if the admin changes the enabled_vgpu_types ?13:37
*** mriedem has joined #openstack-nova13:43
bauzasgibi: for the moment, we create an inventory by calling the number of left mdevs13:43
bauzasgibi: if the operator changes the type, the total would be modified13:43
gibibauzas: so eventually the gpu inventory will be in sync with the reality?13:44
bauzasif we have allocations, I guess placement verifies that the new total is more than the allocations value13:44
bauzasgibi: every 60 secs, ye13:44
gibibauzas: so with type X the hypervisor reports 10 available mdev, then a server is created that use up 1 of that 10. Then the operator changes the config to type Y and restart the compute13:45
gibibauzas: from type Y the hypervisor report 5 mdevs available13:45
bauzasright13:46
gibibauzas: so the compute changes the inventory from 10 to 5 which still fits, as only 1 allocated13:46
bauzasyup13:46
*** edmondsw has quit IRC13:46
bauzasthat said, I have an open question13:46
bauzaswhat if we have 6 allocs for a type with 10 total13:46
bauzasand then operator uses type Y which only has 5 as a total13:47
gibiI think placement will not allow changing the total to 5 then13:47
bauzasok, then we're good13:47
gibibut easxy to try13:47
bauzasactually wait13:47
bauzasif you have one pGPU13:47
bauzaswith multiple types13:47
bauzaswhen you first create a mdev, then all of the other types but the one used by the mdev are having total = 013:48
*** edleafe has quit IRC13:48
bauzasso, if the operator wants to change a type for the same pGPU when there are instances, then it's litterally not possible to get more instances13:48
bauzasbecause the total will be the number of existing mdevs13:49
*** derekh has joined #openstack-nova13:49
bauzasunless the operators changes it back to the original type13:49
bauzasso we don't have a problem atm13:49
gibibauzas: so the operator needs to delete / migrate away the instances first13:49
gibithen change the tyee13:49
bauzasyeah, or change the type back to the original13:49
gibitype13:49
gibibauzas: OK, that makes it even stricter13:49
gibibauzas: and here strict is good, I think13:50
bauzasno, he can change without needing to delete13:50
gibibauzas: yeah, if he want to change back then he can without delete13:50
bauzasthat's just only if he really wants to change from one type to the other13:50
gibibauzas: yeas13:50
bauzasyeah13:50
gibicool13:50
gibimaybe we want to document this behavior somehow for the deployer13:50
bauzasnow, with the change, lemme think13:50
gibito avoud surprise13:50
bauzasgibi: we did it already13:51
*** mgariepy has joined #openstack-nova13:51
gibibauzas: then I rest my case13:51
gibi:)13:51
bauzasoh wait, no actually13:51
bauzashttps://docs.openstack.org/nova/latest/admin/virtual-gpu.html13:51
bauzasI could add a note in https://docs.openstack.org/nova/latest/admin/virtual-gpu.html#enable-gpu-types-compute13:52
*** NewBruce has quit IRC13:52
*** markvoelker has joined #openstack-nova13:52
*** takashin has joined #openstack-nova13:52
*** dklyle has quit IRC13:52
gibibauzas: making that note solves my comment in your patch13:52
bauzasgibi: that said, I need to think on how it works with nested RPs13:53
bauzasfor the moment, it does work because allocations are made against the same RP13:53
bauzasoh wait13:53
bauzaseven if people change types, they won't change the pGPU13:54
bauzasso allocations will still be made against the right pGPUI13:54
bauzasactually, something is missing :(13:54
bauzasholy shit13:54
bauzaswe have an allocation against one pGPU13:54
fried_ricen-sch meeting in 5 minutes in #openstack-meeting-alt13:55
bauzasbut then libvirt creates the mdev (or uses some mdev) from any pGPU13:55
*** fried_rice is now known as efried13:55
bauzaswhich is a problem13:55
bauzasfor correctly counting allocations13:55
bauzasshit shit shit13:55
bauzasgibi: thanks for making me think of it13:55
*** NewBruce has joined #openstack-nova13:56
*** tetsuro has joined #openstack-nova13:56
gibibauzas: you are welcome :)13:59
*** edleafe has joined #openstack-nova13:59
*** sridharg has quit IRC14:07
*** wolverineav has joined #openstack-nova14:11
openstackgerritHamdy Khader proposed openstack/os-vif master: Add VIFPortProfileSmartNIC profile  https://review.openstack.org/63606114:12
*** cfriesen has joined #openstack-nova14:16
openstackgerritSurya Seetharaman proposed openstack/nova-specs master: Support server power state update through external event  https://review.openstack.org/63613214:21
*** markvoelker has quit IRC14:24
*** edmondsw has joined #openstack-nova14:27
*** liuyulong has joined #openstack-nova14:27
openstackgerritMatt Riedemann proposed openstack/nova master: Fix deps for api-samples tox env  https://review.openstack.org/63362014:28
openstackgerritMatt Riedemann proposed openstack/nova master: Make VolumeAttachmentsSampleV249 test other methods  https://review.openstack.org/63362114:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add microversion to expose virtual device tags  https://review.openstack.org/63194814:28
*** lbragstad has joined #openstack-nova14:31
*** takashin has left #openstack-nova14:33
*** gbarros has quit IRC14:39
bauzasmriedem: FWIW, gibi made me think of a problem with the reshape series14:39
bauzasmriedem: that's uncommented yet, but I'll need to respin14:39
mriedemyou've yet to respin anyway right?14:41
mriedemso respin your local respin14:42
* mriedem is getting dizzy14:42
*** agopi has joined #openstack-nova14:42
mriedembauzas: gibi: are you able to summarize the issue?14:43
mriedemso i don't have to read scrollback?14:43
*** tetsuro has quit IRC14:43
bauzasmriedem: tl;dr:14:43
bauzaswe will have inventories per pGPU14:43
bauzasplacement will choose one child RP and make an allocation to it14:43
bauzasthen it goes down to the driver14:44
bauzaswhich picks randomly at the moment14:44
bauzasthe last sentence has to change14:44
bauzasthe driver has to pick one of the available mdevs from the pGPU related to the allocation14:44
mriedemdriver.spawn() gets the allocations, so we just have to reconcile the local inventory in the driver to what was allocated and pick that mdev yeah?14:45
bauzasor allocations would diverge from actual usage14:45
*** wolverineav has quit IRC14:45
bauzasmriedem: yup that14:45
bauzasmriedem: we need to lookup the child RP from the allocation and find an allocable mdev from it14:45
bauzasnot a big deal, but still has to be done14:45
*** NewBruce has quit IRC14:46
openstackgerritgaryk proposed openstack/nova master: Better handle live migration abort  https://review.openstack.org/63544014:46
mriedemdoes that need to be done separately from the reshaper patch, like as a dependency - even though it won't do anything until the reshape is done?14:46
*** awaugama has joined #openstack-nova14:46
mriedembecause that reshaper patch is already big14:46
bauzasyeah I was considering this14:46
mriedemi'd split them if possible14:46
bauzaswe could drop a -214:46
mriedem1. handled nested in spawn, 2. do reshapes14:46
bauzasand only +W once all are accepted14:46
bauzasor yeah, that14:46
bauzaswhatever14:46
mriedemto remind me, the child RP is a pGPU on the compute node right?14:47
mriedemand len(mdevs) is total VGPU inventory per pGPU child provider14:48
openstackgerritsean mooney proposed openstack/nova master: [DNM] libvirt: delegate ovs plug to os-vif  https://review.openstack.org/60243214:48
sean-k-mooneyignore ^ im just testing something14:48
*** jchhatbar has quit IRC14:49
* gibi now learned what was the real bug bauzas found while discussing a different comment in the reshaper series14:51
*** moshele has joined #openstack-nova14:52
bauzasmriedem: correct, with the slight detail that total = len(mdevs) + pGPU.'available_instances' (which changes over the time)14:52
bauzasthat's why we need to count mdevs for the inventory14:52
bauzasas "available instances" is a value that has a very different meaning14:52
openstackgerritsean mooney proposed openstack/nova master: [DNM] libvirt: delegate ovs plug to os-vif  https://review.openstack.org/60243214:54
sean-k-mooneystill ignore ^ needed to fix merge confict to get gate jobs to run.14:54
mriedemgibi: do you want to re-approve https://review.openstack.org/#/c/634767/ since it was rebased?14:56
gibimriedem: done14:57
mriedemthanks14:57
*** awalende has quit IRC14:58
*** awalende has joined #openstack-nova14:59
*** hongbin has joined #openstack-nova15:01
*** awalende has quit IRC15:03
*** mlavalle has joined #openstack-nova15:05
*** hoonetorg has quit IRC15:09
openstackgerritMatt Riedemann proposed openstack/nova master: Ignore VolumeAttachmentNotFound exception in compute.manager  https://review.openstack.org/63265715:13
*** TxGirlGeek has joined #openstack-nova15:13
*** mdbooth_ has joined #openstack-nova15:20
*** mdbooth has quit IRC15:20
*** jding1_ has quit IRC15:21
*** gbarros has joined #openstack-nova15:21
*** markvoelker has joined #openstack-nova15:22
*** jackding has joined #openstack-nova15:22
*** Luzi has quit IRC15:22
jackdingmdbooth_: Could you please revisit https://review.openstack.org/#/c/616692/ since your last -1? It15:24
*** hoonetorg has joined #openstack-nova15:27
gibihey cores, can somebody push in https://review.openstack.org/#/c/633776/ it is a follow up and mriedem already +2 on it15:32
*** TxGirlGeek has quit IRC15:33
*** agopi has quit IRC15:39
*** wolverineav has joined #openstack-nova15:42
openstackgerritMatt Riedemann proposed openstack/nova master: Fix race in test_volume_swap_server_with_error  https://review.openstack.org/63433415:42
sean-k-mooneygibi: ya that makes sense. it conflict with a tone of patches so it would be good to merge sooner rather then later15:43
gibisean-k-mooney: exactly15:44
*** tbachman has joined #openstack-nova15:50
openstackgerritMatt Riedemann proposed openstack/nova master: Raise InstanceFaultRollback for UnableToMigrateToSelf from _prep_resize  https://review.openstack.org/63322715:55
*** markvoelker has quit IRC15:56
*** dklyle has joined #openstack-nova15:58
*** sapd1 has quit IRC16:01
*** wwriverrat has joined #openstack-nova16:04
sean-k-mooneystephenfin: before you hit the sriov migration seires want to quickly look at gibi's patch https://review.openstack.org/634334. its a faily small change16:04
*** tbachman has quit IRC16:05
stephenfinack16:05
gibistephenfin: thanks16:05
*** macza has joined #openstack-nova16:08
*** macza has quit IRC16:08
stephenfinsean-k-mooney: Right link? That's mriedem's change you linked16:08
stephenfinwait - I see what you were referring to16:09
*** macza has joined #openstack-nova16:09
*** moshele has quit IRC16:10
*** _fragatina has joined #openstack-nova16:12
*** wolverineav has quit IRC16:14
*** udesale has quit IRC16:15
*** TxGirlGeek has joined #openstack-nova16:16
efriedIs there an etherpad for the PTG yet?16:16
efriedmriedem, melwitt, cdent, other people who might know. ^16:17
sean-k-mooneyefried: not that im aware of16:17
mriedemhttps://etherpad.openstack.org/p/nova-ptg-train16:18
mriedemnope16:18
efriedight, thx16:18
sean-k-mooneyefried: well i guess you might as well start one with ^16:18
sean-k-mooneywe normally use the location in the name but give  this is the third denver ptg swapping to the release name is likely better16:19
sean-k-mooneyefried: by the way you will be waiting a while for intel to appove. its always late.16:21
efried:)16:22
openstackgerritJim Rollenhagen proposed openstack/nova master: ironic: partition compute services by conductor group  https://review.openstack.org/63500616:35
*** awaugama has quit IRC16:36
*** yan0s has quit IRC16:37
*** tbachman has joined #openstack-nova16:37
*** ircuser-1 has joined #openstack-nova16:38
*** helenafm has quit IRC16:51
mriedemthe location in etherpad name thing is a forum thing, not really the ptg, https://etherpad.openstack.org/p/nova-ptg-queens https://etherpad.openstack.org/p/nova-ptg-rocky https://etherpad.openstack.org/p/nova-ptg-stein16:52
*** markvoelker has joined #openstack-nova16:52
* mriedem really enjoyed someone translating the queens etherpad16:52
*** moshele has joined #openstack-nova16:54
dansmithmriedem: thoughts on my comment here? https://review.openstack.org/#/c/635146/8/nova/api/openstack/compute/views/servers.py16:57
*** awaugama has joined #openstack-nova16:57
mriedemdansmith: commented17:03
mriedemi don't feel the need for the guard i guess17:06
sean-k-mooneymriedem: ya you can get the original version from the timeline but  it is annoying17:10
mriedem?17:11
sean-k-mooneymriedem: the non translated queens etherpad17:11
mriedemi personally don't need the location in the etherpad name since i don't care where it was, i care when it was17:11
mriedemoh yeah - that was annoying17:11
*** wolverineav has joined #openstack-nova17:12
sean-k-mooneyit confused the hell out of me in the nova room at the following ptg i spent 20 trying to figure out what was up with my brower just to realise i was looking at the wronge page17:12
*** sdake has quit IRC17:12
sean-k-mooney*20 mins17:13
*** moshele has quit IRC17:14
mriedempebkac17:15
*** sdake has joined #openstack-nova17:15
openstackgerritMerged openstack/nova stable/rocky: PCI: do not force remove allocated devices  https://review.openstack.org/63507117:17
*** wolverineav has quit IRC17:19
*** gyee has joined #openstack-nova17:24
*** markvoelker has quit IRC17:24
*** _fragatina has quit IRC17:31
*** _fragatina has joined #openstack-nova17:31
*** tssurya has quit IRC17:39
*** panda is now known as panda|off17:41
*** macza has quit IRC17:42
*** macza_ has joined #openstack-nova17:42
*** xek has quit IRC17:42
*** rpittau has quit IRC17:44
*** rpittau has joined #openstack-nova17:45
openstackgerritMatt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask  https://review.openstack.org/63566817:53
openstackgerritMatt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize()  https://review.openstack.org/63568417:53
mriedemstephenfin: i have a couple of comments in your novaclient change, was on the fence about -1ing https://review.openstack.org/#/c/635131/17:55
stephenfinack17:55
*** derekh has quit IRC17:58
dansmithmriedem: okay, it just seems like we're linking two sentinels somewhat via tribal knowledge and that we're setting this up for breakage later, but it's not a huge deal to me17:59
*** sdake has quit IRC18:01
mriedemsean-k-mooney: i've got some questions in https://review.openstack.org/#/c/624842/ which sound like they mirror things leakypipes was asking about before18:03
*** sdake has joined #openstack-nova18:03
sean-k-mooneylooking18:03
mriedemleakypipes must be stuck in friday limbo18:04
*** leakypipes is now known as jaypipes18:06
*** dtantsur is now known as dtantsur|afk18:07
mriedemdansmith: yeah it's definitely tight coupling but i'm not terribly worried about it right now. i wonder if setting an attribute on the instance (not persisted) would help, but idk18:07
dansmithack18:07
*** moshele has joined #openstack-nova18:07
mriedeminstance.from_down_cell = True or something18:07
mriedembut then you have to check if that attribute is set on the object as well18:07
mriedemor we default to False i guess18:08
*** mriedem is now known as mriedem_lunch18:09
kukaczhi, I'm getting "MigrationPreCheckError: ... CPU doesn't have compatibility." while trying to live-migrate between hosts with same model of CPU. Turned out that by mistake I got newer package with /usr/share/libvirt/cpu_map.xml differing between those 2 hosts. downgraded libvirt-bin, restarted libvirt-bin and nova-compute services but still seeing same precheck error. where does the precheck take the cpu model from?18:10
kukacz(had newer package on one of the two hosts)18:11
*** rpittau has quit IRC18:12
openstackgerritStephen Finucane proposed openstack/python-novaclient master: Microversion 2.68: Remove 'forced' live migrations, evacuations  https://review.openstack.org/63513118:13
sean-k-mooneymriedem_lunch: replied inline https://review.openstack.org/#/c/624842/8//COMMIT_MSG18:14
stephenfinmriedem_lunch: When you're back ^18:14
* stephenfin goes home18:14
*** markvoelker has joined #openstack-nova18:21
*** ralonsoh has quit IRC18:22
*** imacdonn has joined #openstack-nova18:24
*** ociuhandu_ has joined #openstack-nova18:25
*** moshele has quit IRC18:25
*** sdake has quit IRC18:27
*** ociuhandu has quit IRC18:28
*** ociuhandu_ has quit IRC18:30
*** sdake has joined #openstack-nova18:30
artomWho's our PowerVM guy? cfriesen?18:31
*** sdake has quit IRC18:32
artomI guess efried used to "it"18:32
artomThere's a unit test issue with to_legacy_dict() in the PowerVMLiveMigrateData... but as far as I can tell nothing actually calls to_legacy_dict anywhere18:36
artomSo I'm wondering if I can just get rid of it18:36
*** ociuhandu has joined #openstack-nova18:37
*** wolverineav has joined #openstack-nova18:38
*** gbarros has quit IRC18:39
*** liuyulong has quit IRC18:39
artomIn fact, the only place I can find *any* _legacy_dict methods is in the migrate_data.py objects and a couple of times in the libvirt migration code where we check if we got an ovo18:40
cfriesenartom: I'm x86. :)    used to be we called to_legacy_dict() from check_can_live_migrate_source()18:40
artomI feel like that's left over from a long time ago and I can remove all of that18:40
*** whoami-rajat has joined #openstack-nova18:40
melwittedmondsw ^18:41
artomcfriesen, ah, sorry got confused about your arch allegiance ;)18:41
*** ociuhandu has quit IRC18:41
artomYou know what, I'll propose a thing that gets rid of it all, and we'll see what breaks :)18:41
edmondswartom in a mtg but will look in a few18:41
artomedmondsw, cheers! I really need to relocate, but leave your reply here and I'll check eavesdrop18:43
*** ociuhandu has joined #openstack-nova18:43
*** artom has quit IRC18:48
*** wolverineav has quit IRC18:50
*** xek has joined #openstack-nova18:51
*** openstackgerrit has quit IRC18:51
mriedem_lunchartom: i think that was dropped when we bumped the compute rpc api version to 5.018:52
*** mriedem_lunch is now known as mriedem18:52
mriedemhttps://github.com/openstack/nova/blob/stable/queens/nova/compute/rpcapi.py#L53518:52
*** efried has quit IRC18:53
*** wolverineav has joined #openstack-nova18:53
*** markvoelker has quit IRC18:55
*** efried has joined #openstack-nova18:56
*** sdake has joined #openstack-nova18:56
*** gbarros has joined #openstack-nova18:57
*** artom has joined #openstack-nova19:03
*** xek_ has joined #openstack-nova19:04
*** hongbin has quit IRC19:05
*** xek has quit IRC19:06
*** hongbin has joined #openstack-nova19:08
*** NewBruce has joined #openstack-nova19:08
*** sdake has quit IRC19:09
*** mvkr has quit IRC19:10
*** jmlowe has joined #openstack-nova19:14
*** efried has quit IRC19:15
*** efried1 has joined #openstack-nova19:15
*** efried1 is now known as efried19:17
*** moshele has joined #openstack-nova19:18
*** efried has quit IRC19:22
*** gbarros has quit IRC19:22
*** artom has quit IRC19:23
*** READ10 has joined #openstack-nova19:26
*** gbarros has joined #openstack-nova19:31
*** openstackgerrit has joined #openstack-nova19:32
openstackgerritMatt Riedemann proposed openstack/nova master: Move finish_resize.(start|end) notifications to helper method  https://review.openstack.org/63507919:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid  https://review.openstack.org/61401219:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field  https://review.openstack.org/61403519:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method  https://review.openstack.org/61403619:32
openstackgerritMatt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells  https://review.openstack.org/61403719:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add CrossCellWeigher  https://review.openstack.org/61435319:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add Instance.hidden field  https://review.openstack.org/63112319:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add TargetDBSetupTask  https://review.openstack.org/62789219:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask  https://review.openstack.org/63158119:32
openstackgerritMatt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask  https://review.openstack.org/63385319:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method  https://review.openstack.org/62131319:32
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add prep_snapshot_based_resize_at_dest compute method  https://review.openstack.org/63329319:32
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add PrepResizeAtDestTask  https://review.openstack.org/62789019:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method  https://review.openstack.org/63483219:32
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add PrepResizeAtSourceTask  https://review.openstack.org/62789119:32
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add finish_snapshot_based_resize_at_dest compute method  https://review.openstack.org/63508019:32
openstackgerritMatt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask  https://review.openstack.org/63564619:32
openstackgerritMatt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask  https://review.openstack.org/63566819:32
*** wolverineav has quit IRC19:33
*** wolverineav has joined #openstack-nova19:34
*** gbarros has quit IRC19:34
*** wolverineav has quit IRC19:34
*** wolverineav has joined #openstack-nova19:35
openstackgerritMatt Riedemann proposed openstack/nova master: Follow up (#2) for the bw resource provider series  https://review.openstack.org/63476719:35
*** wolverineav has quit IRC19:40
*** ociuhandu has quit IRC19:40
*** wolverineav has joined #openstack-nova19:41
*** cdent has quit IRC19:41
*** wolverineav has quit IRC19:41
*** wolverineav has joined #openstack-nova19:41
*** efried has joined #openstack-nova19:42
*** tbachman has quit IRC19:43
*** moshele has quit IRC19:50
*** gbarros has joined #openstack-nova19:51
*** gyee has quit IRC19:52
*** markvoelker has joined #openstack-nova19:52
*** NewBruce has quit IRC19:55
*** wolverineav has quit IRC19:58
*** gyee has joined #openstack-nova20:00
*** artom has joined #openstack-nova20:01
*** artom has quit IRC20:01
openstackgerritArtom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination  https://review.openstack.org/63460520:01
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim()  https://review.openstack.org/63566920:01
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources  https://review.openstack.org/63460620:01
openstackgerritArtom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source  https://review.openstack.org/63482720:01
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: generate new NUMA XML on the dest and send it to the source  https://review.openstack.org/63482820:01
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source  https://review.openstack.org/63522920:01
openstackgerritArtom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods  https://review.openstack.org/63621020:02
*** artom has joined #openstack-nova20:02
openstackgerritMatt Riedemann proposed openstack/nova master: Avoid BadRequest error log on volume attachment  https://review.openstack.org/58145320:03
artommriedem, ^^ The bottom-most patch there is remove the _to_dict stuff20:03
artom(I assume the bot just spammed the channel with my patches, I wasn't actually connected, and eavesdrop isn't showing anything (yet?))20:03
openstackgerritMatt Riedemann proposed openstack/nova master: Provide a useful error message when trying to update non-compute services  https://review.openstack.org/62066720:04
*** erlon has quit IRC20:05
*** jmlowe has quit IRC20:08
artomWait, so what's the point of default=None in Nova objects if the field doesn't actually get set to None when you init the object without it?20:12
artomIs default just for "actual" values?20:12
dansmithartom: it's the default for if you ask the object to set its defaults20:15
dansmithremember, an object can represent something that is a fraction of the total,20:15
artomdansmith, ah, so not a fallback kind of thing20:15
dansmithlike instance if you don't ask for all the stuff to be filled out20:15
artomFair enough.20:15
dansmithif we took the defaults everywhere, we'd overwrite things in the DB with the defaults because someone didn't query for them and then called .save() to update one little field20:16
artomI still have the omnipresent 'if field in obj and obj.field is not None' construct ;)20:16
artomIt makes sense when you explain it that way :)20:16
artoms/still have/still hate/20:17
artomOr obj_attr_is_set20:17
artomBut yeah, I see why we need both "unset" and "None", and having automatic None defaults would clobber the former with the latter20:18
artomAnd they don't mean the same thing20:18
*** markvoelker has quit IRC20:24
*** NewBruce has joined #openstack-nova20:27
sean-k-mooneyartom: i think you can use in or hasattr with ovo too because the baseclass overlods __contains__20:27
sean-k-mooneyhttps://github.com/openstack/oslo.versionedobjects/blob/master/oslo_versionedobjects/base.py#L32120:27
sean-k-mooneyso "if x in my_obj:" will work to test if the field is set20:28
artomsean-k-mooney, yep. I'd hoped to avoid all that by setting a None default, but that won't be possible20:29
artomsean-k-mooney, BTW, going to back to yesterday's discussion on https://review.openstack.org/#/c/634828/9/nova/virt/libvirt/driver.py@680820:29
artomWhen you say "cell in enumerate(instance_numa_toplogy.cells)"20:29
artomThat means the guest cells are *ordered* inside i_n_t, and their order represents the guest cell ID?20:30
artomI mean, I know it's a list, and lists are ordered20:30
artomI just didn't think we encoded anything in that order20:30
sean-k-mooneyartom: yes i belive so20:30
artomBecause the cell.id field is the *host* cell ID, right?20:30
sean-k-mooneyya20:30
*** READ10 has quit IRC20:31
artomOK20:31
artomHrmpf, yeah, maybe stick with building the LibvirtConfigObjects on the dest?20:31
artomIt seems safer :P20:31
artomAs a first whack at it, anyays20:31
sean-k-mooneyi think we can for now.20:31
sean-k-mooneywe really should not have too20:31
*** tbachman has joined #openstack-nova20:31
*** wolverineav has joined #openstack-nova20:32
sean-k-mooneybut its is simpler20:32
artomYeah, I thought the same when I first tried it in Rocky20:32
artomAnd then went to code the XML update stuff on the source20:32
artomAnd realized i_n_t isn't enough20:32
sean-k-mooneythe problem is the xml generation is mix with the buisness logic20:33
sean-k-mooneythey should have been kept seperate but they werent20:33
artomI don't disagree :S20:33
artomA lot of things should have been ;)20:33
sean-k-mooneyso for now i think keeping the changes minimal are fro the best adn maybe we should just decuple it fully latter20:33
sean-k-mooneyand by later i mean as a sperate activiy in train20:34
artomWe'd need a quorum of virt drivers that can support NUMA LM20:35
sean-k-mooneyi also hate that that file is like 9000 lines long20:35
sean-k-mooneynot really20:35
artomTo better understand what we need to encode in the objects/business logic20:35
artomNo? I mean, I have no idea what HyperV wants20:36
artomI think it would help me if I had claudiub explain it to me :)20:36
artom(If he's still around)20:36
sean-k-mooneyhyperv support numa but does not to cpu pinning the same way20:36
*** wolverineav has quit IRC20:36
openstackgerritEric Fried proposed openstack/nova master: Use tox 3.1.1 fixes  https://review.openstack.org/58239220:36
sean-k-mooneyits done in the hyperviors rather then in the nova dirive i think20:37
*** wolverineav has joined #openstack-nova20:37
sean-k-mooneythe issue with the libvirt dirver is that neither libvirt or qemu are hyperviors20:37
artomAh, kvm is the hypervisor20:38
sean-k-mooneyqemu is a emulator and libvirt is just an abstraction layer/api.  but are totaly dumb and just do what you tell them so nova had to build most of the logic that would normally live in a hypvervisor20:38
*** gouthamr has quit IRC20:38
sean-k-mooneyartom: kvm is not a hypervior either20:38
artomsean-k-mooney, heh, OK, define hypervisor then :)20:39
sean-k-mooneyits an aclleration driver/kernel module20:39
artom... allowing the kernel to act as the hypervisor (according to wiki, at any rate_20:40
*** wolverineav has quit IRC20:40
*** wolverineav has joined #openstack-nova20:40
artomNo point in getting bogged down in semantics, I un-WIPed a couple of the patches, working on a third...20:40
sean-k-mooneya hypervior is a set of services that mediates acess to hardware such that multiple kernel can share the hardware20:41
*** mriedem has quit IRC20:41
sean-k-mooney:) true20:41
edmondswartom you're asking about getting rid of to_legacy_dict just for powervm or in general? I don't see it called for any hypervisor20:42
artomedmondsw, yeah, I thought it as just powervm, but it looked to be pretty generalized20:42
artomIe, all drivers had _legacy_dict stuff in migrate_data.py20:42
edmondswyep... and all could go away at this point, right?20:42
artomAnd I'm proposing to remove all of them: https://review.openstack.org/#/c/636210/20:42
edmondswsounds good20:43
sean-k-mooneyartom: well as the name suggest its likely a legacy fuction that was kept for handeling mixed compute node versions20:43
*** mriedem has joined #openstack-nova20:44
artomChanging stuff in nova is like a plastic surgeon operating on Joan Rivers's face: you have no idea who else has touched stuff before, so really all you hope for is to do the minimal amount of damage while keeping Joan alive.20:44
artom(RIP Joan)20:44
artomsean-k-mooney, so to answer your point, I'm not touch anything, named "legacy" or not, unless I'm damn sure it can be removed.20:45
*** NewBruce has quit IRC20:45
sean-k-mooneyartom: well at least for the base class it was added by dansmith in https://github.com/openstack/nova/commit/038dfd672f5b2be5ebe30d85bd00d09bae2993fc20:47
artomHeh, yeah, saw that. And it occurs nowhere else that I could see.20:47
artomSo I figured we're OK to get rid of them all.20:47
*** NewBruce has joined #openstack-nova20:48
sean-k-mooneyartom: so based on the conversation with dan earliar are you going to revert the change to https://review.openstack.org/#/c/634827/10/nova/objects/migrate_data.py and ath the if x in y... check20:51
sean-k-mooneye.g. revert the defults20:51
artomsean-k-mooney, yeah20:51
artomAnd add a proper check20:51
openstackgerritMerged openstack/nova master: Fup for the bandwidth resource provider series  https://review.openstack.org/63377620:51
sean-k-mooneycool am im going to go to sleep earlish tonight i think. ill redeploy the series in my test vms tomorow.20:53
sean-k-mooneyi think it more or less should work at this point as the last version i tested seamed pretty close20:54
openstackgerritArtom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source  https://review.openstack.org/63482720:55
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: Make dest send NUMAMigrateData to the source  https://review.openstack.org/63482820:55
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source  https://review.openstack.org/63522920:55
artomsean-k-mooney, noice. There ^^20:55
artomI did the change, will continue adding unit tests where I can without too much trouble, and then I'll start on the automated testing20:55
sean-k-mooneyoh cool i am going  work on some os-vif stuff for another 30 mins or so. i shold have time to spin up a devstack install and manually live migrate a test vm in the background20:57
*** wolverineav has quit IRC21:09
*** artom has quit IRC21:11
*** wolverineav has joined #openstack-nova21:11
*** wolverineav has quit IRC21:13
*** wolverineav has joined #openstack-nova21:13
*** xek_ has quit IRC21:21
*** markvoelker has joined #openstack-nova21:22
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Filter duplicates from compute API get_migrations_sorted()  https://review.openstack.org/63622421:22
*** itlinux has joined #openstack-nova21:22
*** tesseract has quit IRC21:25
*** itlinux has quit IRC21:35
*** wolverineav has quit IRC21:38
*** gouthamr has joined #openstack-nova21:39
*** wolverineav has joined #openstack-nova21:40
*** ociuhandu has joined #openstack-nova21:40
*** ociuhandu has quit IRC21:44
*** wolverineav has quit IRC21:45
*** wolverineav has joined #openstack-nova21:49
*** awaugama has quit IRC21:50
openstackgerritMerged openstack/nova stable/queens: Handle IndexError in _populate_neutron_binding_profile  https://review.openstack.org/63589721:53
openstackgerritMerged openstack/nova stable/queens: Add functional regression test for bug 1794996  https://review.openstack.org/62335421:53
openstackbug 1794996 in OpenStack Compute (nova) queens "_destroy_evacuated_instances fails and kills n-cpu startup if lazy-loading flavor on a deleted instance" [High,In progress] https://launchpad.net/bugs/1794996 - Assigned to Matt Riedemann (mriedem)21:53
*** markvoelker has quit IRC21:55
*** NewBruce has quit IRC22:07
*** artom has joined #openstack-nova22:11
mriedemoh blarg, our NeutronFixture is not a KSA client fixture...22:13
*** sdake has joined #openstack-nova22:20
*** markvoelker has joined #openstack-nova22:20
*** slaweq has quit IRC22:26
*** wolverineav has quit IRC22:27
*** wolverineav has joined #openstack-nova22:28
*** wolverineav has quit IRC22:31
*** wolverineav has joined #openstack-nova22:31
*** whoami-rajat has quit IRC22:33
*** sdake has quit IRC22:51
*** sdake has joined #openstack-nova22:55
*** tkajinam has joined #openstack-nova22:57
*** wolverineav has quit IRC22:58
*** sdake has quit IRC22:59
*** wolverineav has joined #openstack-nova22:59
*** wolverineav has quit IRC23:01
*** wolverineav has joined #openstack-nova23:01
mriedemefried: hmm, i'm hoping this isn't a regression http://paste.openstack.org/show/744898/23:03
mriedemthis is on rollback of a failed resize where conductor reverts the allocation move by trying to swap the allocations back from the migration consumer to the instance consumer23:03
mriedemthe GET /allocations/$consumer_id in the code is using placement 1.2823:03
mriedemwhich should have project_id in the response23:03
mriedemi wonder if maybe allocations is {} ?23:04
efriedThat seems possible.23:05
efriedreproducible? Stuff in some debug logs?23:05
mriedemOH I KNOW23:06
*** sdake has joined #openstack-nova23:07
mriedemso this is the conductor MigrationTask trying to swap the allocs, but the compute service in this case already reverted them23:07
mriedemso conductor is trying to move something that doesn't exist b/c it's already been moved23:07
mriedemnot a regression, just not robust code23:07
*** mvkr has joined #openstack-nova23:08
efriedseems like we've seen this before23:08
efriedthought we had a fix up that would ignore empty source allocations23:09
mriedemthis is a pretty specific part of nova so i'm guessing it's just not been an issue before,23:10
mriedemespecially because with normal resize, conductor rpc casts to compute and if something fails in compute, it reverts the allocations and conductor doesn't get a chance (b/c of the cast)23:11
openstackgerritEric Fried proposed openstack/nova master: Use placement.inventory.inuse in report client  https://review.openstack.org/56863923:11
mriedemwith cross-cell resize it's synchronous23:11
mriedemso compute reverts and then conductor tries to revert as well23:11
efriedmriedem: here's the change I was thinking of. Slightly different but similar race https://review.openstack.org/#/c/617040/23:17
*** mlavalle has quit IRC23:18
mriedemyeah this isn't a race that i'm dealing with now, just new code paths that aren't robustified yet23:18
mriedemlaying my old cross-cell resize functional tests on top of my current series and flushing out some bugs23:18
*** rchurch_ has joined #openstack-nova23:21
*** macza_ has quit IRC23:22
*** rchurch has quit IRC23:23
*** macza has joined #openstack-nova23:26
*** sdake has quit IRC23:31
*** moshele has joined #openstack-nova23:38
*** jistr_ has joined #openstack-nova23:39
*** dtantsur has joined #openstack-nova23:39
*** BlackDex_ has joined #openstack-nova23:42
*** stephenfin_ has joined #openstack-nova23:45
*** hongbin has quit IRC23:45
*** hemna_ has joined #openstack-nova23:45
*** jmlowe has joined #openstack-nova23:46
*** dtantsur|afk has quit IRC23:46
*** BlackDex has quit IRC23:46
*** jistr has quit IRC23:46
*** icey has quit IRC23:46
*** zioproto has quit IRC23:46
*** mrhillsman has quit IRC23:46
*** rha has quit IRC23:46
*** mnasiadka has quit IRC23:46
*** hemna has quit IRC23:46
*** TheJulia has quit IRC23:46
*** fyx has quit IRC23:46
*** stephenfin has quit IRC23:46
*** coreycb has quit IRC23:46
*** purplerbot has quit IRC23:48
*** moshele has quit IRC23:50
*** macza has quit IRC23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!