| opendevreview | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.opendev.org/c/openstack/nova/+/851337 | 03:12 |
|---|---|---|
| opendevreview | Brett Milford proposed openstack/nova master: Handle "no RAM info was set" migration case https://review.opendev.org/c/openstack/nova/+/852002 | 06:56 |
| gibi | stephenfin: re PCI series: I still had issues after the unittest.mock and double mocking change. But the content is ready up until https://review.opendev.org/c/openstack/nova/+/850546 . The rest top of that has TODOs in the commit message | 07:15 |
| gibi | wolsen[m]: yes, I think only the FUP patch needs update the rest is OK to land | 07:15 |
| gibi | stephenfin: I probably need to change the first patch in the series https://review.opendev.org/c/openstack/nova/+/844627 due to the double mocking stuff | 07:16 |
| ignazio | Hello, I have very slow performances in live migration when instances using memory heavily (for example java application servers). I used converce mode in nova.conf and it does not seem to solve. If I migrate an instance with 8GB ram it takes more than 20 minutes. If I migrate again it takes few seconds. Do you think it is related to memory access ? | 07:26 |
| ignazio | Any help, please ? | 07:29 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Attach Manila shares via virtiofs (db) https://review.opendev.org/c/openstack/nova/+/831193 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Attach Manila shares via virtiofs (objects) https://review.opendev.org/c/openstack/nova/+/839401 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Attach Manila shares via virtiofs (manila abstraction) https://review.opendev.org/c/openstack/nova/+/831194 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Attach Manila shares via virtiofs (drivers) https://review.opendev.org/c/openstack/nova/+/833090 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Attach Manila shares via virtiofs (api) https://review.opendev.org/c/openstack/nova/+/836830 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Bump compute version and check shares support https://review.opendev.org/c/openstack/nova/+/850499 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Add metadata for shares https://review.opendev.org/c/openstack/nova/+/850500 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Add instance.share_attach notification https://review.opendev.org/c/openstack/nova/+/850501 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Add instance.share_detach notification https://review.opendev.org/c/openstack/nova/+/851028 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Add shares to InstancePayload https://review.opendev.org/c/openstack/nova/+/851029 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Add instance.power_on_error notification https://review.opendev.org/c/openstack/nova/+/852084 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Add helper methods to attach/detach shares https://review.opendev.org/c/openstack/nova/+/852085 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Add libvirt test to ensure metadata are working. https://review.opendev.org/c/openstack/nova/+/852086 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: [WIP] Add virt/libvirt error test cases https://review.opendev.org/c/openstack/nova/+/852087 | 07:44 |
| opendevreview | ribaudr proposed openstack/nova master: Change microversion to 2.93 https://review.opendev.org/c/openstack/nova/+/852088 | 07:44 |
| gibi | ignazio: if the workload in the guest continuously dirtying the memory on the source host then libvirt continously needs to retransmitt the dirtied memory pages to the dest. | 07:46 |
| gibi | ignazio: you can try post-copy live migration | 07:46 |
| gibi | https://docs.openstack.org/nova/pike/admin/live-migration-usage.html | 07:46 |
| gibi | ahh | 07:46 |
| gibi | https://docs.openstack.org/nova/latest/admin/live-migration-usage.html | 07:46 |
| ignazio | gibi: I have already tries post-copy instead converge with the same results | 07:47 |
| gibi | with post-copy the guest will be activated on the dest earlier in the process and libvirt will copy the memory from the inactive guest from the source | 07:47 |
| ignazio | Do you think hugepage can help ? | 07:48 |
| ignazio | I read page size flavor deafaul is 4k. Is true ? | 07:48 |
| gibi | have you checked that the network used by libvirt for transferring the memory data has enough bandwidth? | 07:49 |
| gibi | I'm not sure if hugepage will make a difference. yes the default small page on x86 is 4k | 07:49 |
| ignazio | So, if I migrate an istance it spend a lot of time searching memory to migrate. When I migrate back it takes few seconds . | 07:51 |
| ignazio | Probably after the first migration, the destination host reorganize the memory. I do not know very much the tlb mechanism | 07:52 |
| gibi | moving back is an idependent process, there is no way to remember the past memeory content | 08:10 |
| ignazio | <giby> the instance is using heavily the ram also on the first destination host after the first live migration. What I mean, because if I migrate again it takes few seconds ? | 08:23 |
| opendevreview | Merged openstack/placement master: Clarify trait filtering in the API doc https://review.opendev.org/c/openstack/placement/+/825501 | 08:31 |
| ignazio | <gibi> I have 10Gbs bandwiths available | 08:44 |
| gibi | I'm out of ideas :/ sorry | 08:44 |
| ignazio | <gibi> thanks | 08:45 |
| * bauzas_away just pops his head from the window | 09:22 | |
| gibi | bauzas_away: o/ | 09:22 |
| bauzas_away | gibi: apologies about the PCI series, totally got out of my scope btw. | 09:23 |
| bauzas_away | yesterday, I thought about it and said "oh shit, forgot to review it" | 09:23 |
| gibi | bauzas_away: no worries, you are on vacation :) | 09:23 |
| bauzas_away | gibi: I'll do a couple of series this morning | 09:23 |
| bauzas_away | my vacation trip is only next week, so this week this is just staying with the kids at home and preparing for the journey | 09:24 |
| bauzas_away | gibi: thanks for your comment yesterdaty on https://review.opendev.org/c/openstack/nova/+/851924 | 09:50 |
| bauzas_away | replying to it made me realize I was again writing a wrong logic | 09:50 |
| bauzas_away | the FUP can't fix the recreate issue, it should just remove the need for mdev_name2uuid | 09:51 |
| opendevreview | Nobuhiro MIKI proposed openstack/nova master: pci: fix checking for physical function https://review.opendev.org/c/openstack/nova/+/852135 | 10:09 |
| sean-k-mooney[m] | gibi: can i assume you agree we shoudl just proceed with https://review.opendev.org/c/openstack/nova/+/845775/4 instead of ^ | 10:35 |
| sean-k-mooney[m] | gibi can you rereivew https://review.opendev.org/c/openstack/nova/+/848886/18 you were previously +2 but i asked for the release note to be changed slightly | 10:58 |
| gibi | sean-k-mooney[m]: I'm not sure that the two change are equivalent as after my patch we still uses is_physical_function and that queries sriov_totalvfs | 11:28 |
| gibi | sean-k-mooney[m]: I'm +2+A on the evac change | 11:29 |
| sean-k-mooney[m] | ack | 11:30 |
| sean-k-mooney[m] | i dont think we shoudl really be basiing this off sriov_totalvfs | 11:30 |
| sean-k-mooney[m] | that can be 0 if you disable sriov VFs in the bios | 11:31 |
| gibi | I agree that this is shakey | 11:31 |
| gibi | shaky | 11:31 |
| sean-k-mooney[m] | we dont really handel type-pci well with neutron | 11:32 |
| sean-k-mooney[m] | since we dont use the mac update logic for direct phyiscal when you are using a type-pci device via vnic_type=direct | 11:32 |
| sean-k-mooney[m] | so what i dont want to see is use report a nic as type-pci becuase you change the bios setting to disable sriov and then prevent the port form working with direct-physical | 11:33 |
| sean-k-mooney[m] | i guess we can see if there is a better way to do the detection | 11:34 |
| gibi | would be the proposed phys_port_name based detection better? | 11:35 |
| sean-k-mooney[m] | i would prefer if we honetly did not use these fucntions for this if we can avoid it | 11:35 |
| sean-k-mooney[m] | and use the objects form the virt driver instead | 11:35 |
| sean-k-mooney[m] | i have not looked recently but why are we determining if its a physical fucntion in that code path currently | 11:36 |
| sean-k-mooney[m] | this does not influcne if its reported as type-pci ectra today | 11:36 |
| sean-k-mooney[m] | that is done seperatly | 11:36 |
| gibi | I agree that we should not use the sysfs and user the libvirt driver instead | 11:39 |
| gibi | right now I cannot precisely answer why we look up the type during the whitelist parsing | 11:39 |
| sean-k-mooney[m] | well rather the objects returned by the virt driver genericlly | 11:39 |
| gibi | but I have many theorethical issues with that code | 11:40 |
| gibi | +1 on an abstraction over the libvirt virt driver | 11:40 |
| sean-k-mooney[m] | its proably used to supprot the feature where if we whitelist the pf addres with vf product id | 11:40 |
| sean-k-mooney[m] | we allow the VFs | 11:40 |
| gibi | that could be one reason yes | 11:40 |
| sean-k-mooney[m] | well you should not be calling the driver | 11:40 |
| gibi | it is a very convoluted code | 11:41 |
| sean-k-mooney[m] | the whitelist is used to fileter the objects returned by the driver | 11:41 |
| sean-k-mooney[m] | by objects i mean the dicts | 11:41 |
| sean-k-mooney[m] | so the dict already has the type set | 11:41 |
| sean-k-mooney[m] | so we shoudl just be able to look at the type in the dict | 11:41 |
| gibi | yes, that would be nice | 11:43 |
| gibi | it could be a natural continuation of the https://review.opendev.org/q/topic:pci-device-spec-cleanup series | 11:43 |
| sean-k-mooney[m] | yes | 11:44 |
| sean-k-mooney[m] | escpically since the sysfs way only works on linux anyway | 11:44 |
| sean-k-mooney[m] | it does ont work for other virt drivers where as the dict approch would | 11:44 |
| gibi | yes, I also want to remove the sysfs deps from the code | 11:45 |
| * gibi needs a time-machine | 11:45 | |
| sean-k-mooney[m] | we cant remove all of them unforcunetly | 11:45 |
| gibi | true, so limit them :) | 11:45 |
| sean-k-mooney[m] | since we cant trust libvirt because of its caching in some cases | 11:45 |
| gibi | yeah I saw what you and bauzas_away found about the mdev cache in libvirt | 11:46 |
| sean-k-mooney[m] | the sysfs part might be able to move to a libvirt dirver util file or somehting | 11:46 |
| sean-k-mooney[m] | well its not just that | 11:46 |
| gibi | yepp, currently even the nova/network/neutron code is depend on sysfs too | 11:46 |
| sean-k-mooney[m] | i fixed a similar caching issue where libvirt would miss mac adress changes | 11:46 |
| gibi | probably in general if we change something via sysfs and not via libvirt then libvirt will have a stale cache | 11:49 |
| gibi | which is sort of understandable | 11:49 |
| kashyap | Yeah | 11:49 |
| gibi | wondering if neutron also manipulate the host via sysfs or ethtool | 11:50 |
| gibi | as there they have no libvirt interface at all | 11:50 |
| sean-k-mooney[m] | in general if we could disable the caching in libvirt entirly i think we would | 11:50 |
| sean-k-mooney[m] | i have serriously considered if we would be better not using libvirt for device tracking a few times | 11:51 |
| sean-k-mooney[m] | https://review.opendev.org/c/openstack/nova/+/739131 | 11:51 |
| sean-k-mooney[m] | that was the change i was thinkin of | 11:52 |
| gibi | yeah, or creating a way to force libvirt to refresh the cache | 11:52 |
| sean-k-mooney[m] | bind mount it to /dev/null on. disk ? | 11:52 |
| gibi | is it an on disk cache? | 11:53 |
| gibi | I assumed it is just in memory | 11:53 |
| sean-k-mooney[m] | i think both | 11:53 |
| sean-k-mooney[m] | its in memory in the vritnodedevd container | 11:53 |
| sean-k-mooney[m] | but i think it also caches some info on disk | 11:54 |
| sean-k-mooney[m] | i wonder can you just not use that container/deamon | 11:54 |
| sean-k-mooney[m] | and run without it | 11:54 |
| sean-k-mooney[m] | i should ask danpb | 11:54 |
| sean-k-mooney[m] | i assume that wont actully work | 11:55 |
| sean-k-mooney[m] | that deamon is more then just a cache | 11:55 |
| gibi | I assume so | 11:57 |
| sean-k-mooney[m] | what we really. want i think is to add a flag to https://libvirt.org/html/libvirt-libvirt-nodedev.html#virNodeListDevices to force it to probe | 11:59 |
| gibi | yepp, that would help | 12:00 |
| gibi | we would just alway set that flag | 12:00 |
| opendevreview | Rajesh Tailor proposed openstack/nova master: Transport context to all threads https://review.opendev.org/c/openstack/nova/+/827467 | 12:06 |
| gibi | sean-k-mooney[m]: when you have 2 minutes, could you look at this please? https://review.opendev.org/c/openstack/nova/+/845922 I got hit by it recently | 12:16 |
| sean-k-mooney[m] | sure im just working on a doc for a meeting tomrrow | 12:17 |
| sean-k-mooney[m] | hum im not familar with this bug but sure sound strait forward | 12:17 |
| gibi | it happens on slow nodes | 12:18 |
| sean-k-mooney[m] | 1.23 :) | 12:19 |
| sean-k-mooney[m] | i would have also accepted a 42, 420 or 69 | 12:19 |
| sean-k-mooney[m] | have you ever looked at our ping message in the rpc client. that one is my favorite | 12:20 |
| gibi | lol, now I looked :D | 12:22 |
| sean-k-mooney[m] | oh its the conductor ping not base api https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/nova/conductor/api.py#L67 | 12:30 |
| sean-k-mooney[m] | but ya that makes me happy whenever i see it | 12:30 |
| gibi | :) | 12:33 |
| gibi | ...brace for impact | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Poison /sys access via various calls in test https://review.opendev.org/c/openstack/nova/+/844627 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Add compute restart capability for libvirt func tests https://review.opendev.org/c/openstack/nova/+/850510 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Rename [pci]passthrough_whitelist to device_spec https://review.opendev.org/c/openstack/nova/+/843834 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Rename exception.PciConfigInvalidWhitelist to PciConfigInvalidSpec https://review.opendev.org/c/openstack/nova/+/843861 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Rename whitelist in tests https://review.opendev.org/c/openstack/nova/+/843862 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Basics for PCI Placement reporting https://review.opendev.org/c/openstack/nova/+/846187 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Extend device_spec with resource_class and traits https://review.opendev.org/c/openstack/nova/+/846218 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Reject PCI dependent device config https://review.opendev.org/c/openstack/nova/+/846435 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Reject mixed VF rc and trait config https://review.opendev.org/c/openstack/nova/+/846436 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Ignore PCI devs with physical_network tag https://review.opendev.org/c/openstack/nova/+/846219 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Reject devname based device_spec config https://review.opendev.org/c/openstack/nova/+/846466 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Support [pci]device_spec reconfiguration https://review.opendev.org/c/openstack/nova/+/846470 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Stop if tracking is disable after it was enabled before https://review.opendev.org/c/openstack/nova/+/847009 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Move provider_tree RP creation to PciResourceProvider https://review.opendev.org/c/openstack/nova/+/850546 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Retry /reshape at provider generation conflict https://review.opendev.org/c/openstack/nova/+/851358 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Heal missing PCI allocation in the resource tracker https://review.opendev.org/c/openstack/nova/+/851359 | 13:00 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Allow enabling PCI tracking in Placement https://review.opendev.org/c/openstack/nova/+/850468 | 13:00 |
| gibi | this should resolve all the mocking issues from ^^ | 13:01 |
| Uggla | gibi, notify_about_instance_usage is a "legacy call" to notification and should not be used anymore ? | 13:06 |
| gibi | Uggla: if you add a new notification for attaching a share then yes you should not use the legacy one | 13:07 |
| Uggla | ok | 13:08 |
| gibi | we even have a test case that would prevent you to introduce a new legacy notification :) | 13:11 |
| *** dasm|off is now known as dasm | 13:15 | |
| opendevreview | Bence Romsics proposed openstack/nova master: Add limitation to docs about bug 1983570 https://review.opendev.org/c/openstack/nova/+/852168 | 13:32 |
| opendevreview | Amit Uniyal proposed openstack/nova master: Adds check for VM snapshot fail while quiesce https://review.opendev.org/c/openstack/nova/+/852171 | 13:45 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Add more test coverage for devname base dev spec https://review.opendev.org/c/openstack/nova/+/844625 | 13:56 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Extra tests for remote managed dev spec https://review.opendev.org/c/openstack/nova/+/844626 | 13:56 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Unparent PciDeviceSpec from PciAddressSpec https://review.opendev.org/c/openstack/nova/+/844491 | 13:56 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Fix PciAddressSpec descendants to call super.__init__ https://review.opendev.org/c/openstack/nova/+/844565 | 13:56 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Remove dead code from PhysicalPciAddress https://review.opendev.org/c/openstack/nova/+/844628 | 13:56 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Clean up mapping input to address spec types https://review.opendev.org/c/openstack/nova/+/845765 | 13:56 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Remove unused PF checking from get_function_by_ifname https://review.opendev.org/c/openstack/nova/+/845775 | 13:57 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Fix type annotation of pci.Whitelist class https://review.opendev.org/c/openstack/nova/+/845780 | 13:57 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Move __str__ to the PciAddressSpec base class https://review.opendev.org/c/openstack/nova/+/845781 | 13:57 |
| opendevreview | Merged openstack/nova master: For evacuation, ignore if task_state is not None https://review.opendev.org/c/openstack/nova/+/848886 | 14:02 |
| opendevreview | Merged openstack/nova master: Make test_wait_for_instance_event_* test time independent https://review.opendev.org/c/openstack/nova/+/845922 | 14:02 |
| opendevreview | Dan Smith proposed openstack/nova master: Remove system scope from all APIs https://review.opendev.org/c/openstack/nova/+/848021 | 15:31 |
| opendevreview | Dan Smith proposed openstack/nova master: Keep legacy admin behaviour in new RBAC https://review.opendev.org/c/openstack/nova/+/849209 | 15:31 |
| dansmith | gmann: ^ | 15:31 |
| gmann | #topic Open Reviews | 15:32 |
| gmann | #link https://review.opendev.org/q/projects:openstack/governance+is:open | 15:32 |
| gmann | #endmeeting | 15:34 |
| gmann | ohh :) | 15:34 |
| gmann | sorry, I used wrong channel. it was meant for TC channel | 15:35 |
| *** gibi is now known as gibi_pto | 15:37 | |
| opendevreview | Jan Hartkopf proposed openstack/nova master: add support for updating server's user_data https://review.opendev.org/c/openstack/nova/+/816157 | 17:00 |
| melwitt | kashyap, stephenfin, sean-k-mooney[m]: I saw that someone else has proposed a patch for the live migration "no ram info was set" issue https://review.opendev.org/c/openstack/nova/+/852002 fyi if you might prefer that approach | 17:41 |
| sean-k-mooney[m] | did i review that before. | 18:36 |
| sean-k-mooney[m] | it sound vaguly familar | 18:36 |
| sean-k-mooney[m] | https://review.opendev.org/c/openstack/nova/+/842687 is your version right | 18:37 |
| sean-k-mooney[m] | i was previously +2 on your version and stephen currently is so i would prefer ot proceed with yours unless there is a reason not too | 18:38 |
| sean-k-mooney[m] | they are both simialr and your patch has a release note so ill approve that now | 18:40 |
| sean-k-mooney[m] | ok with that im calling it a day o/ catch ye all tomrrow | 18:44 |
| melwitt | sean-k-mooney[m]: cool, thanks. just wanted to show yall in case there was a preference. seeya tomorrow o/ | 19:03 |
| opendevreview | melanie witt proposed openstack/nova master: Workaround qemu query-migrate issue during live migration https://review.opendev.org/c/openstack/nova/+/842687 | 19:56 |
| *** dasm is now known as dasm|off | 22:11 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!