Thursday, 2021-04-15

« Wednesday, 2021-04-14 Index Friday, 2021-04-16 »
*** mlavalle has quit IRC00:04
*** martinkennelly has quit IRC00:05
*** CeeMac has quit IRC00:24
*** hamalq has quit IRC01:00
openstackgerritnorman shen proposed openstack/nova master: Saving security group to info_cache  https://review.opendev.org/c/openstack/nova/+/78634801:26
guilhermespmelwitt: and I suppose the only workaround is rebuild the instance to get the vgpu reallocated, right?01:30
*** Anticimex has quit IRC01:56
openstackgerritQiu Fossen proposed openstack/nova-specs master: Support live migrate vtpm server  https://review.opendev.org/c/openstack/nova-specs/+/78586001:57
*** Anticimex has joined #openstack-nova01:59
*** xinranwang has joined #openstack-nova02:02
*** brinzhang_ is now known as brinzhang02:06
openstackgerritQiu Fossen proposed openstack/nova-specs master: Support fuzzy querying instance by tag  https://review.opendev.org/c/openstack/nova-specs/+/76885302:07
*** rcernin has quit IRC02:10
*** rcernin has joined #openstack-nova02:36
*** hemanth_n has joined #openstack-nova02:40
*** sapd1 has joined #openstack-nova03:29
*** rcernin has quit IRC03:34
*** psachin has joined #openstack-nova03:39
*** rcernin has joined #openstack-nova03:44
openstackgerritXinran WANG proposed openstack/nova-specs master: Repropose smartnic support spec  https://review.opendev.org/c/openstack/nova-specs/+/78363203:48
*** rcernin has quit IRC03:55
*** rcernin has joined #openstack-nova03:55
*** mkrai has joined #openstack-nova04:06
*** rcernin has quit IRC04:13
*** vishalmanchanda has joined #openstack-nova04:23
*** ratailor has joined #openstack-nova04:30
*** brinzhang_ has joined #openstack-nova04:45
*** sapd1 has quit IRC04:48
*** brinzhang has quit IRC04:48
*** whoami-rajat has joined #openstack-nova04:50
*** rcernin has joined #openstack-nova04:55
*** ratailor_ has joined #openstack-nova05:10
*** rcernin has quit IRC05:10
*** ratailor has quit IRC05:13
openstackgerritJeffrey Zhang proposed openstack/nova master: Support inject-nmi action in watchdog  https://review.opendev.org/c/openstack/nova/+/74107205:20
*** rcernin has joined #openstack-nova05:49
*** rcernin has quit IRC05:49
*** rcernin has joined #openstack-nova05:49
*** slaweq has quit IRC05:55
*** slaweq_ has joined #openstack-nova05:55
*** slaweq_ is now known as slaweq05:55
*** xinranwang has quit IRC06:01
*** waleedm__ has joined #openstack-nova06:07
*** rcernin has quit IRC06:10
*** ralonsoh has joined #openstack-nova06:10
*** rcernin has joined #openstack-nova06:28
*** rcernin has quit IRC06:29
*** slaweq_ has joined #openstack-nova06:30
*** rcernin has joined #openstack-nova06:30
*** slaweq has quit IRC06:36
*** slaweq_ is now known as slaweq06:36
*** waleedm__ has quit IRC06:40
*** gyee has quit IRC06:59
melwittguilhermesp: shelve offload/unshelve might work? my thinking is shelve offload would deallocate placement resources and then unshelve would reallocate everything again07:00
*** dklyle has quit IRC07:11
*** andrewbonney has joined #openstack-nova07:24
*** sapd1 has joined #openstack-nova07:34
*** luksky has joined #openstack-nova07:34
openstackgerritBalazs Gibizer proposed openstack/nova master: Replace blind retry with libvirt event waiting in detach  https://review.opendev.org/c/openstack/nova/+/77024607:38
openstackgerritBalazs Gibizer proposed openstack/nova master: Move the guest.get_disk test to test_guest  https://review.opendev.org/c/openstack/nova/+/77715107:38
openstackgerritBalazs Gibizer proposed openstack/nova master: Enable mypy on libvirt/guest.py  https://review.opendev.org/c/openstack/nova/+/77715507:39
openstackgerritBalazs Gibizer proposed openstack/nova master: Follow up type hints for a634103  https://review.opendev.org/c/openstack/nova/+/77715907:39
openstackgerritBalazs Gibizer proposed openstack/nova master: libvirt: Remove dead error handling code  https://review.opendev.org/c/openstack/nova/+/77970407:39
*** rpittau|afk is now known as rpittau07:45
*** tosky has joined #openstack-nova07:46
openstackgerritBalazs Gibizer proposed openstack/osc-placement master: Mark microversion 1.37 supported  https://review.opendev.org/c/openstack/osc-placement/+/78402307:49
lyarwoodgibi: I'll start looking at the detach stuff this morning07:49
lyarwoodgibi: https://review.opendev.org/c/openstack/nova/+/784129/2 would you mind hitting this to unblock the actual fix on top?07:49
gibilyarwood: thanks. and sure I will look at the funct test07:50
lyarwoodcool thanks07:50
*** ociuhandu has joined #openstack-nova07:57
*** dtantsur|afk is now known as dtantsur07:59
*** lucasagomes has joined #openstack-nova08:09
*** ociuhandu has quit IRC08:14
*** martinkennelly has joined #openstack-nova08:25
*** mkrai has quit IRC08:25
*** ociuhandu has joined #openstack-nova08:27
*** luksky has quit IRC08:31
*** luksky has joined #openstack-nova08:31
*** hemanth_n has quit IRC08:32
*** mkrai has joined #openstack-nova08:34
*** rcernin has quit IRC08:37
*** derekh has joined #openstack-nova08:42
*** luksky has quit IRC08:42
lyarwooddoes anyone know where I can find the actual schedule for ptg? https://www.openstack.org/ptg/ just shows a static image for Monday AFAICT.08:44
lyarwoodhttp://ptg.openstack.org/ptg.html ah08:44
*** slaweq has quit IRC08:51
*** slaweq has joined #openstack-nova08:51
*** luksky has joined #openstack-nova08:55
gibilyarwood: also above the static image there is a link to a pdf08:59
*** ociuhandu has quit IRC09:10
*** ociuhandu has joined #openstack-nova09:10
*** ociuhandu has quit IRC09:15
*** vishalmanchanda has quit IRC09:27
*** k_mouza has joined #openstack-nova09:29
*** ociuhandu has joined #openstack-nova09:35
openstackgerritMerged openstack/nova master: Add regression test for bug #1922053  https://review.opendev.org/c/openstack/nova/+/78412909:44
openstackbug 1922053 in OpenStack Compute (nova) "Operators can force up compute services with `done` evacuation migration records still active against the host" [Medium,In progress] https://launchpad.net/bugs/1922053 - Assigned to Lee Yarwood (lyarwood)09:44
openstackgerritMerged openstack/nova master: api: Reject requests to force up computes when `done` evacuation records exist  https://review.opendev.org/c/openstack/nova/+/78413009:44
*** dpawlik9 has quit IRC09:46
*** vishalmanchanda has joined #openstack-nova09:48
*** tesseract has joined #openstack-nova09:49
*** swp20 has joined #openstack-nova09:51
gibifyi, there is a frequent live migration failure in tempest in the new trunk port test09:53
gibihttps://bugs.launchpad.net/tempest/+bug/192425809:54
openstackLaunchpad bug 1924258 in tempest "test_live_migration_with_trunk fails intermittently" [Undecided,New]09:54
gibiI notified the author of the test case and he promised to check it09:55
*** dpawlik4 has joined #openstack-nova09:59
*** ratailor__ has joined #openstack-nova10:08
*** ratailor_ has quit IRC10:11
*** ociuhandu_ has joined #openstack-nova10:30
*** ociuhandu has quit IRC10:32
*** ociuhandu_ has quit IRC10:34
openstackgerritBalazs Gibizer proposed openstack/placement master: Add support for RP re-parenting and orphaning  https://review.opendev.org/c/openstack/placement/+/78402010:56
*** k_mouza has quit IRC10:58
*** psachin has quit IRC11:00
*** mkrai has quit IRC11:01
noonedeadpunkHi! Can you kindly help me a bit. I'm just trying to understand why we might be doing certain thing and it feels for me that it's not needed nowadays. But decided to ask before changing behaviour11:14
*** k_mouza has joined #openstack-nova11:15
noonedeadpunkSo we're running `nova-manage cell_v2 map_instances` after creating a cell. I think that might be the valid case in old releases, when cellsv2 were just introduced?11:15
*** k_mouza has quit IRC11:15
noonedeadpunkor, when you create new cell and aim to move instances there?11:16
*** k_mouza has joined #openstack-nova11:16
*** swp20 has quit IRC11:24
*** psachin has joined #openstack-nova11:29
*** mkrai has joined #openstack-nova11:30
sean-k-mooneyyou should not need to do map instance every upgrade11:30
sean-k-mooneyjust if you are going form non cellsv2 to cellv2 i belive11:31
noonedeadpunkyeah, that;s what I thought... And also there's batches of 50 instances anyway11:31
noonedeadpunk(so would need to run in a while cycle or smth like that)11:31
noonedeadpunksean-k-mooney: thanks for confirming my concerns11:31
sean-k-mooneythe discover_hosts command also only need to be run if you add/remove hosts11:33
sean-k-mooneyor i guess move them11:33
noonedeadpunkI think discover_hosts shouldn't actually hurt? As I'm not sure about how to distinguish if we add host atm or not...11:34
sean-k-mooneyalthough you nomally dont move host between cells. you can but its not common and it proably adviasble ot not have vms on it if you do11:34
sean-k-mooneynoonedeadpunk: ya its pretty cheap11:34
sean-k-mooneynoonedeadpunk: ooo i think just always runs it11:34
sean-k-mooneynoonedeadpunk: we even have the option to do it as a perodic task if you really want too11:35
noonedeadpunkYeah, I know. It's just smth like 20mins or so iirc11:35
*** sapd1 has quit IRC11:41
*** k_mouza has quit IRC11:43
*** k_mouza has joined #openstack-nova11:44
*** CeeMac has joined #openstack-nova12:05
lyarwoodgibi: re https://review.opendev.org/c/openstack/nova/+/770246 I forgot to note that I wanted to land https://review.opendev.org/c/openstack/nova/+/785682 first if possible, I think it's valid to also cover that corner case in the event based flow as well.12:07
*** mkrai has quit IRC12:12
gibilyarwood: looking12:24
gibilyarwood: I will do the rebase and the adaptation to your fix either today or tomorrow12:26
lyarwoodgibi: ack thanks, I'll go over the rest of the series later in more detail but clicking through it LGTM at the moment12:27
gibicool, I will +A your fix soon12:27
*** ociuhandu has joined #openstack-nova12:47
*** ratailor__ has quit IRC12:47
mnaserbauzas: https://bugs.launchpad.net/nova/+bug/1900800 have you thought about how this can be resolved?  We are running into this often :(13:12
openstackLaunchpad bug 1900800 in OpenStack Compute (nova) "VGPUs is not recreated on host reboot" [Low,Confirmed] - Assigned to Sylvain Bauza (sylvain-bauza)13:12
*** hoonetorg has quit IRC13:20
*** ociuhandu has quit IRC13:24
*** ociuhandu has joined #openstack-nova13:24
*** hoonetorg has joined #openstack-nova13:34
*** sapd1 has joined #openstack-nova13:37
bauzasmnaser: sorry I was on a meeting13:41
bauzasmnaser: well, maybe we would need to ask the operator to create the mdevs after rebooting13:41
*** rmart04 has joined #openstack-nova13:44
*** ociuhandu has quit IRC13:47
*** ociuhandu has joined #openstack-nova13:47
openstackgerritQiu Fossen proposed openstack/nova-specs master: Allow migrating PMEM's data  https://review.opendev.org/c/openstack/nova-specs/+/78556313:49
sean-k-mooneybauzas: the mdevs should be create in init_host today13:51
sean-k-mooneyoh13:51
bauzassean-k-mooney: the problem is that we don't know which mdev type they usze13:52
sean-k-mooneywell we should no?13:52
sean-k-mooneythe mdevs should still be in the domins13:53
sean-k-mooneythis should be running before we start the vms13:53
sean-k-mooneyoh the type is not recorded13:53
sean-k-mooneyso if you have multiple devices that would be an issue13:53
sean-k-mooneyalthough we would be abel to look at the vgpu request in the flavor if you had a traits request13:54
sean-k-mooneyor better yet the allocation summeries13:54
sean-k-mooneywe can look up the RP form which the allocation came from and then identify the partent device and use that to look up the mdev type in the config13:55
sean-k-mooneybauzas: that should work right ^13:55
bauzassean-k-mooney: the problem is that the traits are optional13:55
sean-k-mooneywe dont need the traits13:55
sean-k-mooneyuse allcotion to figure out partent device use parent device to look up mdev type in nova.conf13:56
sean-k-mooneythen recreate it13:56
bauzassean-k-mooney: https://github.com/openstack/nova/blob/450213f/nova/virt/libvirt/driver.py#L81613:56
bauzashere, we would then need to call placement13:57
sean-k-mooneyyes13:57
sean-k-mooneyunless we have the allocation summeries saved somewhere13:57
sean-k-mooneyi.e. in the nova db but we dont as far as i know13:58
bauzasok, but then we would see VGPU allocations13:58
bauzasfor RP13:58
bauzasfor a RP13:58
bauzaswhich is a pGPU13:58
bauzasso then we would need to look at the conf option to know which type it uses13:58
sean-k-mooneyyep13:58
bauzasthat *could* work13:59
bauzasbut that's a long change I think13:59
sean-k-mooneyyou mean complex to write/test13:59
sean-k-mooneyi think its what is required though unless we start storing the infor in the nova db in the resouces table for example14:00
sean-k-mooneythose are our two options caulate it form plamcnet or record mdevs in the db like pmem or pcidevices14:00
sean-k-mooneyso that we can just look it up14:01
bauzasI can try to help14:03
*** ociuhandu has quit IRC14:03
*** ociuhandu has joined #openstack-nova14:04
sean-k-mooneymnaser: it would be a bit of a hack but you could proably fix this with a bash script executed by a systemd service file tempoarlly14:05
sean-k-mooneybasically implemented the same logic14:05
*** k_mouza has quit IRC14:05
*** k_mouza_ has joined #openstack-nova14:06
sean-k-mooneyloop over the domains and for each with an mdev look up the placment allocation and get the rp with the vgpu resouces14:06
sean-k-mooneythen get the mdev type  and create it with the same mdev uuid as the xml currenly has14:07
sean-k-mooneyyou could use systemd's "before" and "after" requirements to ensure it runs before nova-compute and after libvirt start14:08
sean-k-mooneyreally nova should do that but that the bug your hitting i guess.14:08
*** ociuhandu has quit IRC14:09
*** ociuhandu has joined #openstack-nova14:11
mnasersean-k-mooney / bauzas: i guess if i'm understanding correctly, the instance <=> mdev mapping is not stored inside nova anywhere so we rely on the state in the libvirt domain14:16
bauzasthat's right14:17
bauzasthere is an existing tool tho14:17
bauzasmnaser: https://github.com/mdevctl/mdevctl14:17
*** ociuhandu has quit IRC14:17
bauzasmnaser: you could use it for precreating the mdevs and persist them (using systemctl)14:17
mnaser"When a known parent device add udev event occurs (or, for more recent kernels, change events with MDEV_STATE values), mdevctl is called by a udev rule to create defined devices with "start": "auto" configured." interesting14:18
bauzas(whoops, systemd)14:18
sean-k-mooneyyou could but long term we dont want peopel to do that14:18
sean-k-mooneybauzas: if we were to go down that route we shoudl remvoe the code for nova to do it14:18
bauzassean-k-mooney: we said this before14:18
sean-k-mooneybauzas: and track the mdevs in the pci_devices table or similar14:18
bauzassean-k-mooney: honestly, mdevs are like VFs14:19
sean-k-mooneyyep14:19
bauzasand we don't persist the latter14:19
sean-k-mooneywe do14:19
bauzasin nova ?14:19
sean-k-mooneyin the pci_devices table in nova14:19
bauzasbut you need to precreate them, right?14:19
sean-k-mooneythats what the pci_tracker does14:19
sean-k-mooneybauzas: oh yes you do14:19
bauzasthe pci trackers tracks the VFs14:19
bauzasbut it doesn't create them, right?14:20
sean-k-mooneyyep so for vf the operator has to precreate them14:20
bauzasthat's my point14:20
mnaserwould it make sense to have something like if len(mdev) == 0: <check with placement if vm has vgpu>; if <system-has-vgpu>: find_an_unused_mdev_or_create_a_new_one();14:20
sean-k-mooneywe chose not to do that for mdevs for some reason14:20
bauzasmnaser: what I *could* do is to work on what sean-k-mooney and I said14:20
sean-k-mooneybut since we chose to create them it meas we should alwasy do it14:20
bauzasmnaser: ie. looking up the placement DB14:20
bauzasand magically recreating them14:21
bauzasmnaser: that's why I left the bug open14:21
mnaserand that would pretty much get rid of the statefulness of libvirt domain xml again14:21
bauzasmnaser: but the fact is, maybe eventually we would remove this whole recreate method14:21
sean-k-mooneybauzas: i would be ok using mdevctl if we moved mdev to the pci track or resouces table14:22
bauzasmnaser: if the libvirt domain information would persist the mdev type, that'd be awesome14:22
sean-k-mooneyand then just getting rid of this code and not needing the domain14:22
bauzasmnaser: but it doesn't14:22
bauzassean-k-mooney: IIRC, aw (the mdevctl developer) was against using it for upper tooling14:23
sean-k-mooneybauzas: if we really needed too we could store it in the metadta section of the xml14:23
bauzassean-k-mooney: that's actually a great point14:23
mnaseris there anthing else we store in the xml as a state?14:23
sean-k-mooneymnaser: no14:23
sean-k-mooneylong term we want to get rid of persitent domains14:24
mnaseronly thing with this is if something goes wrong with libvirt or anything, you would lose all your gpus14:24
sean-k-mooneye.g. the domain xml on disk14:24
bauzassean-k-mooney: the only problem with metadata is that we won't recreate it on move operations14:24
mnaserthats how this bit us, nova wouldn't start, so we tried to undefine the domain to let nova recreate it, and here we are with no vgpus14:24
bauzasmnaser: nova just binds mdevs14:24
mnaserbut if libvirt domain is gone, it doesnt know which mdevs were assigned to that vm, even on a hard reboot14:25
sean-k-mooneyso fundimentally i think we need to revisit using the xml for state storage14:25
bauzasmnaser: sure, but why would you undefine the domain ?14:25
sean-k-mooneyand just store the inf in the nova db eventualy14:25
bauzassean-k-mooney: eeeek14:25
mnaserbauzas: we had other issues why the domain would not start, because a call to libvirt was failing because the mdev was missing14:26
sean-k-mooneybauzas: i didnt like using the xml for this in the first place14:26
mnaserso mdev was missing so nova couldnt start14:26
sean-k-mooneybauzas: this probalem is just another rasons to not do it this way14:26
bauzasmnaser: again, I can try to fix the logic by looking up placement14:26
bauzassean-k-mooney: you know what ? I'll start filling a spec for drafting mdev management in nova14:27
bauzasand exposing them as raw resources14:27
mnaserso pretty much regenerate state from placement14:27
sean-k-mooneybauzas: i think if we want to do the stateless mdev work it would make sense to do that anyway14:27
bauzassean-k-mooney: we could discuss the oppportunity of persisting them in the spec14:27
sean-k-mooneywe could keep them seperate or combine them14:27
mnaseri feel like that would be inline with VFs since you have to create them beforehand14:27
sean-k-mooneyya i think there are two thing we should do14:28
bauzasmnaser: yeah and honestly we regressed on this, so I feel responsible for closing the bug14:28
sean-k-mooney1 try an come up with a backporatable thing to adress the bug14:28
sean-k-mooneyand 2 figure out how to do it better longterm in the spec14:29
sean-k-mooneywhile also discussing generic stateless mdevs for non gpu usecases14:29
*** links has joined #openstack-nova14:29
sean-k-mooneythe placement way can work for the backportable solution14:29
*** ociuhandu has joined #openstack-nova14:29
sean-k-mooneyim not sure we want to do that long term since i dont know what the performance of that will be like14:30
sean-k-mooneyi assume worse then a straight db lookup14:30
mnaseryeah but only hitting on a hard_reboot() that involves regenerating xml14:30
sean-k-mooneymnaser: actully only on init host14:31
sean-k-mooneywe dont need to hit placment on hard reboot nessialy14:32
mnaserah right yes, unless someone is undefinning domains while nova is running14:32
mnaserin that case, that's on them =P14:32
bauzasmnaser: that call to placement is made at service restart14:33
bauzasnot during hard reboots14:33
bauzasfor reboots, we just recreate the XML as we *already* have the allocations14:33
mnaserbauzas: i do have another fun thing to add to it though14:34
bauzasshoot (/me hides)14:34
mnaserhttp://paste.openstack.org/show/804516/14:34
mnaserthat was actually why we had to undefine the domain to let nova start14:34
sean-k-mooneywell14:35
mnaserso mdev is gone but on init_host we try to look it up14:35
sean-k-mooneythat is just becasue the way we try to recrate the mdev today14:35
sean-k-mooneymdev is still listed in the domain xml14:35
sean-k-mooneybut nothing has created in in sysfs yet14:35
sean-k-mooneywhich is why nodeDeviceLookupByName fails14:36
sean-k-mooneymnaser: the current code only account for nova-compute restarts14:36
*** ociuhandu has quit IRC14:36
sean-k-mooneyit does not properly handel host reboots14:36
mnaseryeah but unfortunately it fully blocks nova from going back up, so that was a little part of how we ended up with no libvirt domain xml14:36
sean-k-mooneyyep understandable14:37
sean-k-mooneymnaser: do dyou have multiple mdev types per host14:37
sean-k-mooneyor just one14:37
mnaserin my case no, just one14:37
mnaserby 'handle host reboots' == 'handle mdev devices disappearing'14:37
sean-k-mooneyok then the workaround for you is simple14:37
bauzasmnaser: the paste you showed is just the bug you hit14:37
mnasercreate mdevs manually with the uuids from libvirt, i guess14:38
sean-k-mooneymnaser: do you have multiple pGPUs per host14:38
mnasernope, single pGPU, single vGPU type14:38
sean-k-mooneymnaser: yep that is the workaround14:38
bauzasI stupidely wrote something like "if the mdev doesn't exist; get its name from the xml and then lookup the non-existing mdev"14:38
sean-k-mooneyperfect14:38
sean-k-mooneyso for your case then loop over the domains and grep all the mdevs14:38
bauzasright14:38
sean-k-mooneythen just create them with the hardcoded mdev type and parent14:39
bauzasif you only use one type, that's trivial14:39
sean-k-mooneyby ecoing into /sys14:39
bauzasjust make sure to recreate the mdev with the right uuid14:39
bauzasecho "myuuid" > /sys/bus/mdev_bus/<mdev_type>/create14:39
sean-k-mooneyyep that14:39
bauzas(can't remember the exact sysfs path)14:39
mnasergot it, so that's our workaround until something placemet-y lands14:39
sean-k-mooneyyep14:40
bauzasyup14:40
bauzasYUUUUUUP even14:40
sean-k-mooney:)14:40
* sean-k-mooney thinks of the land before time14:40
bauzasmnaser: sorry for the nasty bug, I should have wrote some docs describing the workaround14:41
sean-k-mooneyhttps://www.youtube.com/watch?v=cAEVzJnHv2c14:41
bauzasI'll at least amend the bug report14:41
mnaserbauzas: hey no worries at all :D14:42
mnaserbauzas: i amended an eavesdrop link pointing to our conversation14:42
mnaserbauzas: i can throw something in an etherpad about a potential workaround with using single pgpu with single vgpu type14:43
mnaseri need to write up something for my team anyways :)14:43
*** dklyle has joined #openstack-nova14:47
mnaserworking here https://etherpad.opendev.org/p/nova-vgpu-sys-reboot and ill post it as a comment after i run it by y'all :)14:47
bauzasmnaser: https://bugs.launchpad.net/nova/+bug/1900800/comments/414:48
openstackLaunchpad bug 1900800 in OpenStack Compute (nova) "VGPUs is not recreated on host reboot" [Low,Confirmed] - Assigned to Sylvain Bauza (sylvain-bauza)14:48
mnaseroh heck, even better, ill refernce that :)14:48
bauzasyou can even ask libvirt to give you all the domains that have mdevs14:49
bauzasno need to lookup all your instances, just the ones that have mdevs14:49
*** ociuhandu has joined #openstack-nova14:50
sean-k-mooneybauzas: can you? i didnt know that14:52
bauzasoh, nevermind, call me stupid14:52
sean-k-mooneybauzas: can you do that with virsh?14:52
bauzasthat's the PCI devices you can get14:52
bauzasthe ones supporting mdev caps14:52
bauzasin nova, I'm just blindly iterating over all instances14:53
bauzas... :(14:53
sean-k-mooneyya i think that is what you have to do14:53
sean-k-mooneyim not aware of an api that allows you to fileter domains by the content of there xml14:53
*** ociuhandu has quit IRC14:57
*** belmoreira has joined #openstack-nova14:57
bauzassean-k-mooney: I guess looking up the instances by their flavors would be then better for mnaser ;)14:59
bauzasusing the nova api15:00
bauzasand then getting the instance name15:00
sean-k-mooneynot really15:01
sean-k-mooneythat would be much more expensive15:01
sean-k-mooneymnaser should now which host has gpus so its really not that hard to loop over the domains on those hosts in a script15:02
sean-k-mooneydoing that will be much faster then hitting keytone to get a tokken then listing the instance on host and filtering by flaovr then looking it up in libvirt to get the mdev15:03
mnaseryeah i think we'll have a bash script because its single pgpu per system so if the libvirt domains werent destroyed yet15:15
mnaserit should be easy to rebuild15:15
sean-k-mooneymnaser: a hard reboot might fix your instances that dont have gpus15:16
sean-k-mooneyor a cold migrate it that is not enough15:16
mnasersean-k-mooney: yeah those are fine but init_host() failing means we have to fix them all first15:16
mnasercause nova wont go up15:16
sean-k-mooneyyep i ment for the ones you undifed15:16
openstackgerritRodrigo Barbieri proposed openstack/nova master: Error anti-affinity violation on migrations  https://review.opendev.org/c/openstack/nova/+/78416615:21
*** mlavalle has joined #openstack-nova15:36
openstackgerritBalazs Gibizer proposed openstack/nova master: Move instance power state check to _detach_with_retry  https://review.opendev.org/c/openstack/nova/+/77891815:42
openstackgerritBalazs Gibizer proposed openstack/nova master: Consolidate device detach error handling  https://review.opendev.org/c/openstack/nova/+/77897815:44
*** sapd1 has quit IRC15:45
gibinova meeting starts in 10 minutes in #openstack-meeting-315:49
*** gyee has joined #openstack-nova15:57
*** hamalq has joined #openstack-nova16:00
*** ociuhandu has joined #openstack-nova16:02
*** hamalq has quit IRC16:02
*** lucasagomes has quit IRC16:02
*** hamalq has joined #openstack-nova16:02
*** rpittau is now known as rpittau|afk16:09
*** ociuhandu has quit IRC16:17
*** tesseract has quit IRC16:18
*** mlavalle has quit IRC16:23
*** _mlavalle_1 has joined #openstack-nova16:23
*** ociuhandu has joined #openstack-nova16:32
*** rmart04 has quit IRC16:34
*** ociuhandu has quit IRC16:36
*** dtantsur is now known as dtantsur|afk16:36
*** links has quit IRC16:40
*** _mlavalle_1 has quit IRC16:42
*** k_mouza_ has quit IRC16:43
*** k_mouza has joined #openstack-nova16:44
*** k_mouza has quit IRC16:51
*** derekh has quit IRC17:03
*** mlavalle has joined #openstack-nova17:05
*** ralonsoh has quit IRC17:12
*** ociuhandu has joined #openstack-nova17:13
*** ociuhandu has quit IRC17:18
*** hemna has quit IRC17:19
*** hemna has joined #openstack-nova17:25
*** zul has joined #openstack-nova17:26
*** tbachman has joined #openstack-nova17:29
*** belmoreira has quit IRC17:42
*** andrewbonney has quit IRC17:50
*** bbowen_ has left #openstack-nova18:05
openstackgerritMerged openstack/nova master: libvirt: Ignore device already in the process of unplug errors  https://review.opendev.org/c/openstack/nova/+/78568218:27
*** vishalmanchanda has quit IRC18:27
*** ociuhandu has joined #openstack-nova18:31
*** ociuhandu has quit IRC18:31
*** ociuhandu has joined #openstack-nova18:31
*** belmoreira has joined #openstack-nova18:34
*** ociuhandu has quit IRC18:37
*** ociuhandu has joined #openstack-nova18:51
openstackgerritLee Yarwood proposed openstack/nova stable/wallaby: libvirt: Ignore device already in the process of unplug errors  https://review.opendev.org/c/openstack/nova/+/78648318:56
*** ociuhandu has quit IRC18:58
*** ociuhandu has joined #openstack-nova19:17
*** hamalq has quit IRC19:17
belmoreirabauzas thanks for the ping.19:23
belmoreiralassimus definitely I'm interested in the what you are proposing (emulate other architectures). I think it's an interesting topic to be discussed in the PTG.19:23
belmoreiraRelated bugs: https://bugs.launchpad.net/nova/+bug/1902203 https://bugs.launchpad.net/nova/+bug/190221619:24
openstackLaunchpad bug 1902203 in OpenStack Compute (nova) "Instance architecture should be reflected in the instance domain" [Wishlist,Confirmed]19:24
openstackLaunchpad bug 1902216 in OpenStack Compute (nova) "Can't define a cpu_model from a different architecture" [Wishlist,Confirmed] - Assigned to Belmiro Moreira (moreira-belmiro-email-lists)19:24
*** ociuhandu has quit IRC19:27
*** hamalq has joined #openstack-nova19:31
*** dave-mccowan has quit IRC19:43
*** dave-mccowan has joined #openstack-nova19:46
*** whoami-rajat has quit IRC19:47
*** macz_ has joined #openstack-nova19:55
openstackgerritMerged openstack/nova master: Placeholders for DB migration backports to Wallaby  https://review.opendev.org/c/openstack/nova/+/77892320:22
*** k_mouza has joined #openstack-nova20:45
*** k_mouza has quit IRC20:49
*** ociuhandu has joined #openstack-nova20:57
*** ociuhandu has quit IRC21:13
*** ociuhandu has joined #openstack-nova21:13
*** ociuhandu has quit IRC21:18
*** belmoreira has quit IRC21:23
*** ociuhandu has joined #openstack-nova21:44
*** ociuhandu has quit IRC21:53
*** tosky has quit IRC22:34
*** rcernin has joined #openstack-nova22:51
*** macz_ has quit IRC23:16
*** macz_ has joined #openstack-nova23:20
*** macz_ has quit IRC23:25
*** luksky has quit IRC23:58
« Wednesday, 2021-04-14 Index Friday, 2021-04-16 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!