| *** mlavalle has quit IRC | 00:39 | |
| *** sapd1 has quit IRC | 01:12 | |
| *** luksky has joined #openstack-nova | 01:16 | |
| *** farice has joined #openstack-nova | 01:19 | |
| *** luksky has quit IRC | 01:22 | |
| *** luksky has joined #openstack-nova | 01:22 | |
| *** luksky has quit IRC | 01:29 | |
| *** k_mouza has joined #openstack-nova | 02:21 | |
| *** k_mouza has quit IRC | 02:25 | |
| *** mkrai has joined #openstack-nova | 03:45 | |
| *** ratailor has joined #openstack-nova | 04:19 | |
| *** rcernin has quit IRC | 04:22 | |
| *** rcernin has joined #openstack-nova | 04:28 | |
| *** k_mouza has joined #openstack-nova | 04:33 | |
| *** k_mouza has quit IRC | 04:37 | |
| *** mkrai_ has joined #openstack-nova | 04:40 | |
| *** mkrai has quit IRC | 04:42 | |
| *** ralonsoh has joined #openstack-nova | 05:20 | |
| *** xinranwang has joined #openstack-nova | 05:59 | |
| *** slaweq has joined #openstack-nova | 06:12 | |
| *** mkrai_ has quit IRC | 06:13 | |
| *** StevenK has quit IRC | 06:13 | |
| *** rcernin has quit IRC | 06:29 | |
| *** StevenK has joined #openstack-nova | 06:35 | |
| *** lpetrut has joined #openstack-nova | 06:36 | |
| gibi | good morning | 06:37 |
|---|---|---|
| *** dklyle has quit IRC | 06:48 | |
| *** whoami-rajat_ has joined #openstack-nova | 06:50 | |
| *** rcernin has joined #openstack-nova | 06:59 | |
| *** luksky has joined #openstack-nova | 07:01 | |
| *** rcernin has quit IRC | 07:08 | |
| *** rcernin has joined #openstack-nova | 07:09 | |
| *** andrewbonney has joined #openstack-nova | 07:17 | |
| *** macz_ has joined #openstack-nova | 07:34 | |
| *** gokhani has joined #openstack-nova | 07:34 | |
| *** rpittau|afk is now known as rpittau | 07:34 | |
| *** macz_ has quit IRC | 07:39 | |
| *** rcernin has quit IRC | 07:40 | |
| *** tosky has joined #openstack-nova | 07:49 | |
| *** mgoddard has joined #openstack-nova | 07:49 | |
| *** lucasagomes has joined #openstack-nova | 08:01 | |
| *** derekh has joined #openstack-nova | 08:03 | |
| lyarwood | Morning | 08:04 |
| *** StevenK has quit IRC | 08:04 | |
| sean-k-mooney | gibi: im going to go create the plament periodic job now want me to also create patches for the review pirority lable for the nova deliverables while im at it. | 08:12 |
| *** rcernin has joined #openstack-nova | 08:14 | |
| gibi | sean-k-mooney: if you know how to do it then please propose it | 08:16 |
| sean-k-mooney | cool ya i do ill propose the patch to project-config repo and add you to the review list | 08:17 |
| sean-k-mooney | they will want the ptl to sign off on those kind of changes anyway | 08:17 |
| gibi | sure, thanks | 08:18 |
| openstackgerrit | liuzhuangzhuang proposed openstack/nova master: Fix RBD timeout https://review.opendev.org/c/openstack/nova/+/786588 | 08:44 |
| *** mkrai has joined #openstack-nova | 08:45 | |
| *** sapd1 has joined #openstack-nova | 08:58 | |
| stephenfin | lyarwood: Real easy doc follow-up here for you if you've 2 mins https://review.opendev.org/c/openstack/nova/+/783830 | 08:59 |
| openstackgerrit | sean mooney proposed openstack/placement master: Add weekly jobs https://review.opendev.org/c/openstack/placement/+/787508 | 08:59 |
| *** dtantsur|afk is now known as dtantsur | 09:02 | |
| lyarwood | stephenfin: ack'd | 09:04 |
| stephenfin | \o/ | 09:04 |
| stephenfin | thanks | 09:04 |
| gibi | stephenfin: hi! there are two followups for the device detach series that would need a second set of eyes https://review.opendev.org/c/openstack/nova/+/778918 | 09:15 |
| * stephenfin looks | 09:15 | |
| *** tesseract has joined #openstack-nova | 09:23 | |
| *** tesseract has quit IRC | 09:28 | |
| *** tesseract has joined #openstack-nova | 09:28 | |
| *** k_mouza has joined #openstack-nova | 09:42 | |
| *** avolkov has joined #openstack-nova | 09:47 | |
| stephenfin | gibi: comments on https://review.opendev.org/c/openstack/nova/+/778918 | 09:54 |
| gibi | stephenfin: thanks, will look in a bit | 09:56 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Remove 'kwargs' from 'get_instance_nw_info' https://review.opendev.org/c/openstack/nova/+/744872 | 09:58 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Remove unused 'affect_auto_assigned' parameter https://review.opendev.org/c/openstack/nova/+/744871 | 09:58 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Add type hints for various allocate functions https://review.opendev.org/c/openstack/nova/+/744870 | 09:58 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Add type hints for deallocate functions https://review.opendev.org/c/openstack/nova/+/787516 | 09:58 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: More allocate, deallocate type hints https://review.opendev.org/c/openstack/nova/+/787517 | 09:58 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Add type hints for attach, detach functions https://review.opendev.org/c/openstack/nova/+/787518 | 09:58 |
| *** mgoddard has quit IRC | 10:01 | |
| sean-k-mooney | gibi: just added you to https://review.opendev.org/q/topic:%22review-priority%22+(status:open%20OR%20status:merged) | 10:21 |
| sean-k-mooney | some of the placement repos do not have the correct acls defiend so i added those as a copy of the placment repos config in the second patch | 10:21 |
| *** rcernin has quit IRC | 10:22 | |
| openstackgerrit | Balazs Gibizer proposed openstack/placement stable/wallaby: Add a reproduction test for bug story/2008831 https://review.opendev.org/c/openstack/placement/+/787525 | 10:28 |
| openstackgerrit | Balazs Gibizer proposed openstack/placement stable/wallaby: Make sure the policy upgrade check get a valid config https://review.opendev.org/c/openstack/placement/+/787526 | 10:28 |
| *** ratailor_ has joined #openstack-nova | 10:30 | |
| *** ratailor has quit IRC | 10:31 | |
| *** k_mouza has quit IRC | 10:45 | |
| *** k_mouza has joined #openstack-nova | 10:47 | |
| gibi | stephenfin: replied in https://review.opendev.org/c/openstack/nova/+/778918 I agree with your comment. I will respin | 10:47 |
| *** mkrai has quit IRC | 10:50 | |
| *** rcernin has joined #openstack-nova | 11:03 | |
| *** rcernin has quit IRC | 11:08 | |
| *** sapd1 has quit IRC | 11:15 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Move instance power state check to _detach_with_retry https://review.opendev.org/c/openstack/nova/+/778918 | 11:27 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Consolidate device detach error handling https://review.opendev.org/c/openstack/nova/+/778978 | 11:27 |
| gibi | stephenfin: fixed your comments ^^ | 11:28 |
| gibi | sean-k-mooney: thanks, I will check them | 11:28 |
| *** rcernin has joined #openstack-nova | 11:28 | |
| *** rcernin has quit IRC | 11:33 | |
| openstackgerrit | Merged openstack/nova-specs master: trivial: Fix spelling, formatting of vDPA spec https://review.opendev.org/c/openstack/nova-specs/+/781260 | 11:40 |
| Dmitrii-Sh | o/ Looking for some input on a spec: https://review.opendev.org/c/openstack/nova-specs/+/787458. If there is interest, I am open for discussion during the remaining PTG time (via meetpad.opendev.org or other means) or afterwards via gerrit/IRC/email etc. Any feedback would be much appreciated. I added this as a topic to the Nova etherpad | 11:40 |
| Dmitrii-Sh | https://etherpad.opendev.org/p/nova-xena-ptg - hopefully there will be some time left to discuss it. | 11:40 |
| openstackgerrit | Merged openstack/nova master: docs: Follow up for SEV doc https://review.opendev.org/c/openstack/nova/+/783830 | 11:41 |
| sean-k-mooney | looking at it quickly but form the title this sound complex | 11:42 |
| *** mkrai has joined #openstack-nova | 11:44 | |
| openstackgerrit | Merged openstack/nova master: docs: Correct typos in 'nova-manage db' docs https://review.opendev.org/c/openstack/nova/+/784067 | 11:45 |
| sean-k-mooney | Dmitrii-Sh: we are planning to discuss tracking pci device in placement but im not conviced that what you are suggestign is compatiable with the direction we wanted to take | 11:46 |
| sean-k-mooney | Dmitrii-Sh: you have clearly put a lot of tought into this by the way form the lenght of the spec so this will take some time to digest | 11:47 |
| Dmitrii-Sh | sean-k-mooney: I see, I could find ways to adjust it such that it is compatible with the effort to track pci devices in the placement DB. | 11:48 |
| Dmitrii-Sh | As such, part of what I am suggesting for the Nova DB is only there to build the Placement DB state. | 11:48 |
| sean-k-mooney | we were going to suggest tracking each PF as an RP and having inventories of VF,PF and pci devices | 11:50 |
| Dmitrii-Sh | So I could place the right state into the Placement DB right away but that's to be seen | 11:50 |
| sean-k-mooney | the pci_devices tabel would still be used to track the slecect of the specific pci device | 11:50 |
| sean-k-mooney | we dont want to actully have placment select a specifc VF for example | 11:51 |
| artom | Dmitrii-Sh, I saw you post the spec last night. I think your biggest problem is that only youself and sean-k-mooney understand any of the hardware bits :P | 11:52 |
| Dmitrii-Sh | sean-k-mooney: I see. Do you plan to replace InstancePCIRequests with placement resource requests? | 11:53 |
| artom | Dmitrii-Sh, I think a sort of "smartnic for dummies" paragraph (or section) might be useful, just to give the rest of use context on what this thing even is | 11:53 |
| Dmitrii-Sh | artom: ty. I tried to make it largely agnostic of a specific vendor and target a class of devices: off-path SmartNICs with a separate (mostly ARM) CPU. There are several vendors out there that follow this pattern. | 11:55 |
| Dmitrii-Sh | but I can add a paragraph discussing it, or even better, a reference to an overview about those | 11:55 |
| artom | I don't want to speak for everyone here, but you have a whole bunch of vocabulary there that isn't accessible to most of us ("switchdev-capable", "off-path") | 11:55 |
| Dmitrii-Sh | I guess I should link this then https://netdevconf.info/0x14/pub/slides/39/Netdev%200x14%20--%20Taking%20Control%20of%20your%20SmartNIC%20v1.pdf | 11:56 |
| sean-k-mooney | Dmitrii-Sh: not replace them no but we would also claime resouces in placment for them | 11:56 |
| artom | Dmitrii-Sh, yeah, any kind of introduction or reference to what this hardware is and how it's different from the PCI devices that we're used to would be very useful. For me, at least. | 11:56 |
| artom | Hopefully for other non-sean-mooney reviewers too | 11:56 |
| sean-k-mooney | Dmitrii-Sh: im not really trielled with nova calling neutron as part fo vif plugging as a clinet | 11:57 |
| Dmitrii-Sh | artom: ack, will do. I understand that it's a lot of info to process | 11:57 |
| sean-k-mooney | even if that was doen via os-vif | 11:57 |
| sean-k-mooney | Dmitrii-Sh: have you spoken to the cyborg team about this by the way | 11:58 |
| sean-k-mooney | they were tryign to enable reprogrambel smart nics via a different approch | 11:58 |
| Dmitrii-Sh | sean-k-mooney: I've seen their specs but they seem to be targeting FPGA-based on-path NICs (Intel IIRC) | 11:58 |
| Dmitrii-Sh | and the approach there is to program NICs from the hypervisor host itself | 11:59 |
| sean-k-mooney | Dmitrii-Sh: primarally | 11:59 |
| sean-k-mooney | i just wanted to make sure you were aware of it | 11:59 |
| Dmitrii-Sh | where as here there is a security boundary between the hypervisor host and the SmartNIC host | 11:59 |
| Dmitrii-Sh | sean-k-mooney: ack, I am. I wish it addressed the things needed for off-path NICs to avoid fragmentation | 12:00 |
| sean-k-mooney | so there was an effort to allow the neutron agent to manage remote host in the past | 12:00 |
| sean-k-mooney | what you are proposing is closer to that | 12:00 |
| *** farice has quit IRC | 12:00 | |
| sean-k-mooney | i think we would still want to keep the binding host the same at the ml2 level | 12:00 |
| sean-k-mooney | the host on which the vm resides | 12:00 |
| Dmitrii-Sh | There was some work in Ironic in the past https://specs.openstack.org/openstack/ironic-specs/specs/approved/support-smart-nic.html | 12:00 |
| sean-k-mooney | but we may need to have other information that woudl be use to corralate that to the chassis in ovn | 12:01 |
| Dmitrii-Sh | as a result neutron-openvswitch-agent started using os-vif to plug representors (but that's not usable outside ironic at the moment) | 12:01 |
| Dmitrii-Sh | sean-k-mooney: yes, I think there has to be a mapping table somewhere but I haven't quite decided where yet | 12:02 |
| sean-k-mooney | Dmitrii-Sh: in neutron | 12:03 |
| sean-k-mooney | i dont think nova should need to know what network backend is being used in this code | 12:03 |
| Dmitrii-Sh | sean-k-mooney: this makes sense. Otherwise Nova becomes too aware of a particular ML2 mechanism driver | 12:03 |
| sean-k-mooney | yep | 12:03 |
| *** brinzhang has quit IRC | 12:04 | |
| sean-k-mooney | i have only skimmed the spec so i really need to set some time aside to digest it | 12:04 |
| *** brinzhang has joined #openstack-nova | 12:04 | |
| Dmitrii-Sh | sean-k-mooney: ok, np, thanks for the early and quick feedback - much appreciated | 12:04 |
| Dmitrii-Sh | sean-k-mooney: I am happy to make adjustments to make it work with the ongoing efforts in Nova | 12:04 |
| *** mkrai has quit IRC | 12:05 | |
| sean-k-mooney | ill try and do that relitively soon. if we dont have time in the ptg we coudl set up a call afterward next week or the week after and see who is interested | 12:05 |
| sean-k-mooney | Dmitrii-Sh: have you discussed this with the neutron folks yet? | 12:05 |
| Dmitrii-Sh | sean-k-mooney: ty. I think I'm going to hop in during the PCI device tracking in the Placement service discussion (added a note to ping me to the etherpad just in case). | 12:06 |
| Dmitrii-Sh | sean-k-mooney: I am working on a Neutron spec at the moment | 12:06 |
| sean-k-mooney | cool | 12:06 |
| Dmitrii-Sh | sean-k-mooney: There is some cross-project work to do in Neutron OVN mechanism driver and OVN itself | 12:06 |
| sean-k-mooney | by the way one hack for the differnt hostname thing you could look at is changhing the [DEFAULT]/host value in the neutronl.conf for the ovs agent of the hostname value in the ovsdb for ovn | 12:07 |
| sean-k-mooney | basically you could change those values so that you are declaring that you are manaing the network on behalf of host XYZ even if its not running on it | 12:08 |
| Dmitrii-Sh | sean-k-mooney: there's a possibility that there will be 2 SmartNICs in one physical host | 12:09 |
| sean-k-mooney | so assuming the agent/ovn would be runnign on the smart nic you could use the servers hostname for those values | 12:09 |
| Dmitrii-Sh | which creates an issue with this - I considered it as an initial hack | 12:09 |
| sean-k-mooney | ah right | 12:09 |
| sean-k-mooney | ya that does make things harder | 12:09 |
| Dmitrii-Sh | sean-k-mooney: hence the dances with PCI VPD serial numbers | 12:09 |
| Dmitrii-Sh | as we don't have another way of tracking which functions belong where on both sides | 12:10 |
| sean-k-mooney | still i think we coudl encode some info in ovn or the agent config such that the ml2 driver could maintain the mapping | 12:10 |
| sean-k-mooney | it just wont be as simple | 12:10 |
| sean-k-mooney | i see one other issue potentaillyu | 12:10 |
| sean-k-mooney | the pci address will it always be the same form the host vs smartnic perspecitve | 12:11 |
| Dmitrii-Sh | Didn't quite catch that. The SmartNIC host sees a different PCIe topology from the hypervisor host | 12:12 |
| Dmitrii-Sh | different addresses too | 12:12 |
| Dmitrii-Sh | basically each host does its own PCIe topology enumeration | 12:12 |
| Dmitrii-Sh | it's kind of like with blades and non-transparent bridging or managed PCIe switches that hide certain parts of the overall topology | 12:13 |
| sean-k-mooney | right | 12:14 |
| sean-k-mooney | so the only thing that we currently pass to neturon to tell it which device we selected is the pci address | 12:14 |
| sean-k-mooney | from the nova-compute agents point of view | 12:14 |
| Dmitrii-Sh | yes, and that's the problem in this case | 12:15 |
| Dmitrii-Sh | the ARM host doens't know how the hypervisor host enumerated its topology | 12:15 |
| sean-k-mooney | we cannot use macs and other info like vendor id and project id are not applicable | 12:15 |
| Dmitrii-Sh | doesn't* | 12:15 |
| Dmitrii-Sh | sean-k-mooney: yes | 12:15 |
| sean-k-mooney | yep so we would need the neuron ml2 driver to translate | 12:15 |
| sean-k-mooney | we could optionally add some addtional info to the binding profile which i assuem is what you were suggesting | 12:16 |
| Dmitrii-Sh | sean-k-mooney: yes, the PCIe VPD serial number could be it. Additionally, we need a way to send a logical number of a VF | 12:16 |
| Dmitrii-Sh | to pick the right representor on the other side | 12:16 |
| Dmitrii-Sh | the logical numbers are maintained in the NIC firmware, so presumably they are stable | 12:16 |
| sean-k-mooney | i think we can discuover both of those statically and put them in the extra_info column | 12:17 |
| sean-k-mooney | then we can include that in the port update | 12:17 |
| sean-k-mooney | Dmitrii-Sh: i think we can avoid the need for any db schema changes on the nova side | 12:17 |
| Dmitrii-Sh | sean-k-mooney: that's one way to do it. I was planning to do the translation based on the relationship of "transport node" RPs (name == VPD serial) and OVN chassis RPs (name == smartnic_hostname). | 12:18 |
| sean-k-mooney | currently we only use the extra_info column to store nic features. i was going to store vdpa device paths in them but we removed it | 12:18 |
| Dmitrii-Sh | So, in essence, the Nova schema changes are there to build the Placement state ("transport node" RPs) | 12:19 |
| Dmitrii-Sh | I could build this info as devices are discovered and push it to the Placement DB right away | 12:19 |
| sean-k-mooney | well what we are going to discsus later is create rp for each PCI device as a chiled of the compute node rp with the pci address as the RP name | 12:20 |
| sean-k-mooney | thoguh we were not going to suggest havign an rp per vf just per PF | 12:20 |
| sean-k-mooney | so the pci device would be in the sub tree of the compute node RP not under the neutron RPs | 12:21 |
| gibi | do I understand correctly from above that one VF from this smartnic can be made available to more than one compute node / hypervisor? | 12:22 |
| Dmitrii-Sh | sean-k-mooney: *thinking* | 12:22 |
| sean-k-mooney | gibi: ish basiably ovs and either ovn/neutron-l2 agent woudl be running on a smartnic isntalled in a server | 12:23 |
| sean-k-mooney | gibi: on the arm cores runnign linux | 12:23 |
| gibi | so from other perspective do we need to represent this smartnic as a sharing RP in placement or can it be under a given compute RP? | 12:24 |
| Dmitrii-Sh | gibi: there is some IO virtualization sharing tech to allow VFs of one device to be selectively shared across different compute nodes (e.g. in a blade). I have some links in the spec to people describing it. | 12:24 |
| *** brinzhang has quit IRC | 12:24 | |
| gibi | Dmitrii-Sh: OK, so it is possible | 12:24 |
| *** brinzhang has joined #openstack-nova | 12:24 | |
| Dmitrii-Sh | hence I am going for shared resource providers when it comes to representing SmartNICS | 12:24 |
| Dmitrii-Sh | much like with shared storage | 12:25 |
| sean-k-mooney | Dmitrii-Sh: i think the simpelst thing to do is as follows. 0 do not modify placment for this specifically. 1 add the extra identifiesr to the pci_device extra info column, 2. add that info the the prot profile. 3 modify the ml2 drivers to sotre the coallation beween the ovn chassi name and the device and have it use that value instead | 12:25 |
| gibi | Dmitrii-Sh: thanks, I get it | 12:25 |
| *** sorrison has quit IRC | 12:26 | |
| sean-k-mooney | gibi: yep you can do it with connectx-6 cards i think intel were working on it at one point too but no idea if its still a thing they are planning on | 12:26 |
| *** sorrison has joined #openstack-nova | 12:26 | |
| sean-k-mooney | gibi: one of the things i was investigating in the past was could we move nova-compute to the smart nic too | 12:26 |
| *** lpetrut_ has joined #openstack-nova | 12:27 | |
| gibi | this is crazy talk :D | 12:27 |
| gibi | sorry, my brain failed | 12:27 |
| sean-k-mooney | idea bing you would jsut grab a stanarcd server and install libvirt on it then you would just install an OpenstackPowerd smarntic that rann all the openstack servics in the server and its an instance compute node after some config updates | 12:27 |
| *** tobias-urdin has joined #openstack-nova | 12:28 | |
| *** tesseract-RH has joined #openstack-nova | 12:28 | |
| sean-k-mooney | gibi: in this case they are jsut trying to move the networkign contol plane to the smarnic and not all the contol plane | 12:28 |
| *** tesseract has quit IRC | 12:28 | |
| gibi | ... and then at some point that smartnic is connected to two hypervisors and we are back to the one compute host multiple compute node situation :) | 12:28 |
| sean-k-mooney | same basic idea however just run less stuff on the hypervior host | 12:28 |
| sean-k-mooney | gibi: hehe lol lets hope this and rack scale design never have a baby | 12:29 |
| *** lpetrut has quit IRC | 12:29 | |
| sean-k-mooney | Dmitrii-Sh: by the way os-vif can connect to ovs over tcp to do the pluggin fo the interface into the ovs bridge | 12:30 |
| Dmitrii-Sh | sean-k-mooney: I see. (1) ok, there will be some more info stored per pci_device but that's not too bad (2) ok, this can be done during the port update (3) yes, Neutron would need to look up a hostname of a chassis based on the serial number. | 12:30 |
| Dmitrii-Sh | sean-k-mooney: I think os-vif can only use one OVS endpoint | 12:30 |
| sean-k-mooney | Dmitrii-Sh: so we can have os-vif running in the compute agent talk to ovs on the smart nic if it has an ip | 12:30 |
| Dmitrii-Sh | sean-k-mooney: comes back to the dual-NIC case | 12:30 |
| sean-k-mooney | Dmitrii-Sh: corrrect today. but you could tell use what endpoitn to connecto to in the binding:detail from the ml2 driver | 12:31 |
| sean-k-mooney | if we need that | 12:31 |
| sean-k-mooney | e.g. make the config value the defualt and allow it to change per port if needed | 12:31 |
| sean-k-mooney | Dmitrii-Sh: yep so if we take this approch we can still track pci device in placment as a seperat effort without having to take account of the network backend in use | 12:32 |
| gibi | hm does this whole thing also suggest that a single neutron-ovs-agent will handle more than on OVS instance (one per smartnic)? | 12:32 |
| gibi | s/on OVS/one OVS/ | 12:33 |
| sean-k-mooney | no but we might have multipel ovs agents on the same physical server one running on each smart nic | 12:33 |
| *** zoharm has joined #openstack-nova | 12:33 | |
| gibi | wait, not just the OVS runs on the smartnic but also the neutron agent? | 12:33 |
| sean-k-mooney | gibi: unless we revisit the scaleable ovs agent spec whic intended to od that | 12:33 |
| sean-k-mooney | gibi: well in teh case of ovn theere would be a seperate ovn contoler per smartnic form what i understand | 12:34 |
| sean-k-mooney | so if we are talking about ml2/ovs i would assume we would have 1 agent per smartnic Dmitrii-Sh is that correct? | 12:34 |
| Dmitrii-Sh | sean-k-mooney: ok, I'll have a look at whether something falls over with that approach. One challenge is NIC resource allocation when multiple compute nodes are involved but let's see. | 12:35 |
| sean-k-mooney | gibi: the neturon l2 agent and ovn southd are basically equvlent | 12:35 |
| Dmitrii-Sh | sean-k-mooney: yes, 1 per nic | 12:35 |
| *** belmoreira has joined #openstack-nova | 12:35 | |
| Dmitrii-Sh | sean-k-mooney: I considered having something crazy that had 1 ARM host but 2 NICs | 12:35 |
| sean-k-mooney | Dmitrii-Sh: can we declare multi host nics out of scope of the inital proposal | 12:35 |
| Dmitrii-Sh | but I haven't seen hw like that | 12:35 |
| sean-k-mooney | its not crazy i have seen hardware that was designed to work that way | 12:36 |
| sean-k-mooney | it was a top of rack swtich that had a pcie card that plugged into your servers | 12:36 |
| sean-k-mooney | each server tought it was getting a nic but actully it all the nic hardware was on the switch | 12:36 |
| Dmitrii-Sh | I guess that would be similar to 1 SmartNIC with PCIe bifurcation? Some lanes go to one CPU, some to another? | 12:37 |
| sean-k-mooney | kind of yes | 12:37 |
| Dmitrii-Sh | so, in this case, there would be 2 PFs exposed to the hypervisor host under different root complexes | 12:38 |
| *** happyhemant has joined #openstack-nova | 12:38 | |
| Dmitrii-Sh | and the NUMA node association of PFs (and their VFs) would be based on that too | 12:38 |
| sean-k-mooney | basically it was a ginant pcie switch on one end that connectd to multiple hosts and then a ether net switch on the other | 12:38 |
| sean-k-mooney | Dmitrii-Sh: ya you can do the pci biforcation with mellonox connetx5 today | 12:39 |
| *** hamalq has joined #openstack-nova | 12:39 | |
| sean-k-mooney | they have a 200G sku that has 2x 16 connectors that can connect to 2 different cpus | 12:39 |
| sean-k-mooney | it get 2 PFs and the VF can float between them | 12:39 |
| Dmitrii-Sh | sean-k-mooney: yes, there's a special connector on the NIC and an add-in card that plugs to a different topology | 12:40 |
| Dmitrii-Sh | and the NIC is smart enough to use 2 sets of lanes differently | 12:40 |
| sean-k-mooney | yep hardware vendors make software enginers look sane | 12:40 |
| Dmitrii-Sh | sean-k-mooney: :^D | 12:41 |
| gibi | lol | 12:41 |
| Dmitrii-Sh | sean-k-mooney: ok, I am aiming to make the design extensible such that those things can be supported but I will avoid focusing on it for the initial proposal then | 12:42 |
| Dmitrii-Sh | it's just very difficult to extend it later as I see it | 12:42 |
| sean-k-mooney | i spent quite a lot of my time at intel dealing with topic like this which is why this seam like a realitvly normal request to me | 12:42 |
| sean-k-mooney | Dmitrii-Sh: ya i think that multi host nics proably need cyborg to manage well | 12:43 |
| Dmitrii-Sh | sean-k-mooney: good thing you did, otherwise my requests would look insane | 12:43 |
| sean-k-mooney | Dmitrii-Sh: same as remove nic over a pcie fabric | 12:43 |
| sean-k-mooney | Dmitrii-Sh: we were looking at this problem space when we were looking at how to integrate a rackscale design composeable server with nova | 12:44 |
| sean-k-mooney | Dmitrii-Sh: basicaly i wanted to allow create a composable physical server form disaggreated devices (nics, storage, gpus, fpga) via a new nova virt driver | 12:45 |
| sean-k-mooney | so it would be like creating a vm but with realy hardware using a pcie/rdma based frabric | 12:45 |
| Dmitrii-Sh | sean-k-mooney: right, I've seen some hw like this with 1U PCIe switches and composable topologies | 12:46 |
| sean-k-mooney | :( we never pushed it publically. i thought it was on https://github.com/intel-orchestration-software but we did boot a node with a poc virt driver internally | 12:48 |
| Dmitrii-Sh | sean-k-mooney: I think I came across this (but haven't had this in a lab): https://gigaio.com/products/ | 12:50 |
| sean-k-mooney | Dmitrii-Sh: i think we both have a lot to digest. can you add the topic to the end of https://etherpad.opendev.org/p/nova-xena-ptg | 12:51 |
| sean-k-mooney | as i said if we dont get to it im happy to review and we could set up a call or somehting in a week or too if needed | 12:51 |
| sean-k-mooney | oh its already there | 12:52 |
| sean-k-mooney | Transport Node bp/spec RFC and this topic dont really seam realted to me | 12:52 |
| sean-k-mooney | but i see where you were comign form | 12:52 |
| gibi | Dmitrii-Sh: thanks for the well written spec, I forwarded it to my downstream counterparts to see if my company is also interested in solving these use case. I agree with Sean about trying to get to this on the PTG but it might not fit into the timeframe. | 12:56 |
| Dmitrii-Sh | sean-k-mooney: yes, added it a little earlier to the etherpad | 13:01 |
| Dmitrii-Sh | sean-k-mooney: the gigaio stuff is unrelated, yes, just sharing what I've seen :^) | 13:01 |
| Dmitrii-Sh | gibi: thanks a lot, much appreciated | 13:02 |
| sean-k-mooney | Dmitrii-Sh: im adding a summery of what i think the problem statemnt is and a sumamry of what i was propsing as an alternitve desing. | 13:02 |
| sean-k-mooney | feel free to extend of correct eitehr | 13:02 |
| *** ratailor_ has quit IRC | 13:03 | |
| sean-k-mooney | im also addig a link to the irc logs of this conversation if peopel want to review it | 13:03 |
| *** avolkov has quit IRC | 13:04 | |
| *** artom has quit IRC | 13:04 | |
| *** avolkov has joined #openstack-nova | 13:05 | |
| *** artom has joined #openstack-nova | 13:05 | |
| Dmitrii-Sh | sean-k-mooney: hmm, all the colors are gone from the etherpad | 13:06 |
| Dmitrii-Sh | at least for me | 13:06 |
| sean-k-mooney | dont press contol shift c | 13:06 |
| sean-k-mooney | yep i had the wrong window slected and ctrl-shift-c is apparently clear all colors | 13:07 |
| sean-k-mooney | i contol z it and it dissconnected me | 13:07 |
| sean-k-mooney | so ya i cant revert it it seams | 13:07 |
| sean-k-mooney | they are here https://etherpad.opendev.org/p/nova-xena-ptg/timeslider#24635 | 13:08 |
| *** Luzi has joined #openstack-nova | 13:09 | |
| *** mgoddard has joined #openstack-nova | 13:09 | |
| Dmitrii-Sh | sean-k-mooney: right, haven't found a way to revert yet | 13:10 |
| sean-k-mooney | i think the only way would be contrl z but i dont think that will work now | 13:10 |
| sean-k-mooney | which suck but ya | 13:11 |
| sean-k-mooney | im going to add that as a warning to the top of the doc | 13:11 |
| sean-k-mooney | its a really bad keyboard shortcut to choose | 13:11 |
| *** luksky has quit IRC | 13:14 | |
| *** luksky has joined #openstack-nova | 13:14 | |
| *** brinzhang_ has joined #openstack-nova | 13:14 | |
| *** sapd1 has joined #openstack-nova | 13:16 | |
| *** k_mouza has quit IRC | 13:22 | |
| *** k_mouza_ has joined #openstack-nova | 13:22 | |
| *** k_mouza_ has quit IRC | 13:22 | |
| *** ratailor has joined #openstack-nova | 13:22 | |
| *** k_mouza has joined #openstack-nova | 13:23 | |
| sean-k-mooney | Dmitrii-Sh: ok im done summerising it. im goingt to go take a walk and clear my head before the ptg resumes. o/ | 13:25 |
| Dmitrii-Sh | sean-k-mooney: tyvm, I'll have a look in 20-30 mins and adjust if needed | 13:25 |
| *** avolkov has quit IRC | 13:26 | |
| *** rcernin has joined #openstack-nova | 13:29 | |
| *** k_mouza has quit IRC | 13:30 | |
| *** k_mouza has joined #openstack-nova | 13:31 | |
| *** k_mouza has quit IRC | 13:31 | |
| *** k_mouza has joined #openstack-nova | 13:32 | |
| openstackgerrit | Merged openstack/nova master: glance: Improve [glance]enable_rbd_download and associated options help text https://review.opendev.org/c/openstack/nova/+/785656 | 13:32 |
| *** rcernin has quit IRC | 13:34 | |
| *** k_mouza_ has joined #openstack-nova | 13:37 | |
| *** k_mouza has quit IRC | 13:40 | |
| *** lpetrut_ has quit IRC | 13:45 | |
| *** brinzhang_ has quit IRC | 13:50 | |
| * bauzas definitely needs a coffee | 13:53 | |
| *** priteau has quit IRC | 13:53 | |
| *** k_mouza has joined #openstack-nova | 13:55 | |
| *** k_mouza has quit IRC | 13:56 | |
| *** k_mouza has joined #openstack-nova | 13:56 | |
| stephenfin | gibi: Might be a few minutes late. Having internet woes :-( | 13:58 |
| *** k_mouza_ has quit IRC | 13:58 | |
| gibi | stephenfin: ack | 13:58 |
| *** k_mouza has quit IRC | 14:01 | |
| *** k_mouza has joined #openstack-nova | 14:02 | |
| *** mgoddard has quit IRC | 14:06 | |
| gibi | belmoreira: we are talking about emulating guest archs | 14:07 |
| gibi | in the nova room now | 14:07 |
| *** tesseract-RH has quit IRC | 14:07 | |
| belmoreira | thanks gibi. Unfortunately, I can't join right now (I'm in the TC meeting). | 14:08 |
| gibi | belmoreira: OK, I think I'm aware of your viewpoint so I can represent it | 14:09 |
| belmoreira | thanks gibi | 14:09 |
| *** tesseract has joined #openstack-nova | 14:10 | |
| *** k_mouza_ has joined #openstack-nova | 14:11 | |
| *** k_mouza_ has quit IRC | 14:12 | |
| *** k_mouza_ has joined #openstack-nova | 14:13 | |
| *** k_mouza has quit IRC | 14:14 | |
| *** k_mouza_ has quit IRC | 14:19 | |
| *** k_mouza has joined #openstack-nova | 14:20 | |
| stephenfin | ralonsoh: we'll be talking about networking-related stuff in nova shortly, if you can join. If not, we can skip them and come back when you're free? | 14:21 |
| *** k_mouza_ has joined #openstack-nova | 14:22 | |
| ralonsoh | stephenfin, thanks! joining now | 14:22 |
| *** k_mouza has quit IRC | 14:25 | |
| *** mkrai has joined #openstack-nova | 14:34 | |
| stephenfin | Dmitrii-Sh: fwiw, we'll be talking about PCI device tracking in placement in 5-10 minutes if you're not already on the meeting | 14:38 |
| gibi | stephenfin: thanks for pinging people :) | 14:38 |
| Dmitrii-Sh | stephenfin: ty for the ping, I'll try to sneak out and join for that | 14:39 |
| *** ratailor has quit IRC | 14:39 | |
| *** k_mouza_ has quit IRC | 14:46 | |
| *** k_mouza has joined #openstack-nova | 14:47 | |
| *** macz_ has joined #openstack-nova | 14:50 | |
| *** macz_ has quit IRC | 14:51 | |
| *** macz_ has joined #openstack-nova | 14:51 | |
| *** dklyle has joined #openstack-nova | 14:52 | |
| slaweq | gibi: lbragstad: hi, will the session about RBAC and system scope tokens be in Keystone or Nova's room? | 14:57 |
| slaweq | I see in the https://etherpad.opendev.org/p/policy-popup-xena-ptg that it's in Mitaka which seems to be Keystone session | 14:57 |
| slaweq | but on the ptg.openstack.org I see that it may also be in the Nova's room | 14:58 |
| *** xinranwang has quit IRC | 14:59 | |
| gibi | :/ | 14:59 |
| gibi | good question | 14:59 |
| gibi | lets have it in the Mitaka room | 14:59 |
| slaweq | gibi: great | 15:00 |
| gibi | I will push the people from the nova room to the keystone one | 15:00 |
| slaweq | that's what I told in the Neutron session few minutes ago :) | 15:00 |
| slaweq | gibi++ | 15:00 |
| lbragstad | slaweq keystone i think | 15:03 |
| *** mkrai has quit IRC | 15:10 | |
| *** __ministry1 has joined #openstack-nova | 15:11 | |
| *** k_mouza has quit IRC | 15:13 | |
| *** k_mouza_ has joined #openstack-nova | 15:13 | |
| *** k_mouza has joined #openstack-nova | 15:21 | |
| *** mgoddard has joined #openstack-nova | 15:22 | |
| *** k_mouza_ has quit IRC | 15:25 | |
| *** happyhemant has quit IRC | 15:26 | |
| *** rcernin has joined #openstack-nova | 15:30 | |
| *** mkrai has joined #openstack-nova | 15:30 | |
| *** rcernin has quit IRC | 15:35 | |
| *** __ministry1 has quit IRC | 15:36 | |
| *** mgoddard has quit IRC | 15:49 | |
| *** brinzhang_ has joined #openstack-nova | 15:55 | |
| *** brinzhang has quit IRC | 15:58 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Replace grenade and nova-grenade-multinode with grenade-multinode https://review.opendev.org/c/openstack/nova/+/778885 | 15:58 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Remove nova-dsvm-multinode-base https://review.opendev.org/c/openstack/nova/+/778908 | 15:58 |
| *** mkrai has quit IRC | 16:01 | |
| *** tesseract has quit IRC | 16:02 | |
| *** gokhani has quit IRC | 16:03 | |
| *** hamalq has quit IRC | 16:06 | |
| *** lucasagomes has quit IRC | 16:09 | |
| *** Luzi has quit IRC | 16:15 | |
| *** gyee has joined #openstack-nova | 16:17 | |
| bauzas | gibi: back on the nova room when it's 16:25pm UTC, right ? | 16:18 |
| * bauzas feels lost in translation | 16:18 | |
| dansmith | what is the next nova topic? | 16:18 |
| gibi | right | 16:19 |
| gibi | dansmith: 18:17 < gibi> #nova next pci device attach/detach api? | 16:19 |
| dansmith | gibi: ah, sorry, but thanks :) | 16:19 |
| *** slaweq_ has joined #openstack-nova | 16:20 | |
| gibi | I'm trying to keep http://ptg.openstack.org/ptg.html up to date via the ptgbot | 16:20 |
| dansmith | gibi: ah yeah I hadn't seen the coming up next section used yet, that's nice | 16:21 |
| openstackgerrit | Merged openstack/nova master: libvirt: Remove dead error handling code https://review.opendev.org/c/openstack/nova/+/779704 | 16:30 |
| *** derekh has quit IRC | 17:02 | |
| *** andrewbonney has quit IRC | 17:04 | |
| *** aarents has quit IRC | 17:06 | |
| *** ralonsoh has quit IRC | 17:08 | |
| *** k-s-dean has quit IRC | 17:11 | |
| *** zoharm has quit IRC | 17:21 | |
| *** dtantsur is now known as dtantsur|afk | 17:22 | |
| *** rpittau is now known as rpittau|afk | 17:24 | |
| *** boxiang_ has joined #openstack-nova | 17:39 | |
| *** k_mouza has quit IRC | 17:39 | |
| *** k_mouza has joined #openstack-nova | 17:40 | |
| *** elod_ has joined #openstack-nova | 17:43 | |
| *** smcginni1 has joined #openstack-nova | 17:44 | |
| *** sapd1 has quit IRC | 17:48 | |
| *** elod has quit IRC | 17:48 | |
| *** mgariepy has quit IRC | 17:48 | |
| *** smcginnis has quit IRC | 17:48 | |
| *** boxiang has quit IRC | 17:48 | |
| *** kopecmartin has quit IRC | 17:48 | |
| *** eharney has quit IRC | 17:49 | |
| *** k_mouza has quit IRC | 17:49 | |
| *** k_mouza has joined #openstack-nova | 17:49 | |
| *** sapd1 has joined #openstack-nova | 17:54 | |
| *** eharney has joined #openstack-nova | 17:55 | |
| *** mgariepy has joined #openstack-nova | 17:55 | |
| *** lbragstad_ has joined #openstack-nova | 18:29 | |
| *** lbragstad has quit IRC | 18:32 | |
| *** k_mouza_ has joined #openstack-nova | 18:32 | |
| *** k_mouza_ has quit IRC | 18:34 | |
| *** k_mouza_ has joined #openstack-nova | 18:34 | |
| *** k_mouza has quit IRC | 18:35 | |
| *** dklyle has quit IRC | 18:36 | |
| *** mgoddard has joined #openstack-nova | 18:41 | |
| *** elod_ is now known as elod | 18:44 | |
| *** dklyle has joined #openstack-nova | 18:45 | |
| *** k_mouza has joined #openstack-nova | 18:49 | |
| *** k_mouza_ has quit IRC | 18:53 | |
| *** mgoddard has quit IRC | 18:56 | |
| *** k_mouza has quit IRC | 19:00 | |
| *** k_mouza has joined #openstack-nova | 19:01 | |
| *** k_mouza has quit IRC | 19:03 | |
| *** k_mouza has joined #openstack-nova | 19:04 | |
| *** k_mouza_ has joined #openstack-nova | 19:18 | |
| *** k_mouza has quit IRC | 19:22 | |
| *** belmoreira has quit IRC | 19:25 | |
| *** slaweq_ has quit IRC | 19:30 | |
| *** rcernin has joined #openstack-nova | 19:31 | |
| *** whoami-rajat_ is now known as whoami-rajat | 19:33 | |
| *** k_mouza_ has quit IRC | 19:36 | |
| *** rcernin has quit IRC | 19:36 | |
| *** k_mouza has joined #openstack-nova | 19:36 | |
| *** k_mouza_ has joined #openstack-nova | 19:37 | |
| *** k_mouza has quit IRC | 19:41 | |
| *** k_mouza_ has quit IRC | 19:48 | |
| *** arne_wiebalck has quit IRC | 19:51 | |
| *** yonglihe has quit IRC | 19:51 | |
| *** lbragstad_ is now known as lbragstad | 19:56 | |
| *** yonglihe has joined #openstack-nova | 19:57 | |
| *** arne_wiebalck has joined #openstack-nova | 19:57 | |
| *** iurygregory has quit IRC | 19:59 | |
| *** hamalq has joined #openstack-nova | 20:10 | |
| *** k-s-dean has joined #openstack-nova | 20:36 | |
| *** slaweq has quit IRC | 20:44 | |
| *** rcernin has joined #openstack-nova | 20:45 | |
| *** k_mouza has joined #openstack-nova | 20:45 | |
| *** k_mouza has quit IRC | 20:45 | |
| *** k-s-dean has quit IRC | 20:46 | |
| *** k_mouza has joined #openstack-nova | 20:46 | |
| *** k-s-dean has joined #openstack-nova | 20:46 | |
| *** k_mouza has quit IRC | 20:49 | |
| *** k-s-dean has quit IRC | 20:56 | |
| *** k-s-dean has joined #openstack-nova | 20:57 | |
| *** rcernin has quit IRC | 21:00 | |
| *** rcernin has joined #openstack-nova | 21:00 | |
| *** rcernin has quit IRC | 21:25 | |
| *** mlavalle has joined #openstack-nova | 21:27 | |
| *** macz_ has quit IRC | 21:29 | |
| *** rcernin has joined #openstack-nova | 21:36 | |
| *** rcernin has quit IRC | 21:41 | |
| *** rcernin has joined #openstack-nova | 21:41 | |
| *** k_mouza has joined #openstack-nova | 21:46 | |
| *** k_mouza has quit IRC | 21:46 | |
| *** k_mouza has joined #openstack-nova | 21:47 | |
| *** rcernin has quit IRC | 21:50 | |
| *** irclogbot_1 has quit IRC | 21:50 | |
| *** whoami-rajat has quit IRC | 21:50 | |
| *** zzzeek has quit IRC | 21:51 | |
| *** k_mouza has quit IRC | 21:52 | |
| *** zzzeek has joined #openstack-nova | 21:53 | |
| *** irclogbot_1 has joined #openstack-nova | 21:54 | |
| *** lemko has quit IRC | 22:07 | |
| *** lemko has joined #openstack-nova | 22:08 | |
| *** rcernin has joined #openstack-nova | 22:15 | |
| *** luksky has quit IRC | 22:18 | |
| *** rcernin has quit IRC | 22:20 | |
| *** k_mouza has joined #openstack-nova | 22:21 | |
| *** k_mouza has quit IRC | 22:21 | |
| *** k_mouza has joined #openstack-nova | 22:22 | |
| *** k_mouza has quit IRC | 22:30 | |
| *** rcernin has joined #openstack-nova | 22:34 | |
| *** rcernin has quit IRC | 22:38 | |
| *** rcernin has joined #openstack-nova | 22:42 | |
| *** rcernin has quit IRC | 22:42 | |
| *** rcernin has joined #openstack-nova | 22:43 | |
| *** k_mouza has joined #openstack-nova | 23:09 | |
| *** k_mouza has quit IRC | 23:09 | |
| *** k_mouza has joined #openstack-nova | 23:09 | |
| *** k_mouza has quit IRC | 23:15 | |
| *** tosky has quit IRC | 23:17 | |
| *** mlavalle has quit IRC | 23:59 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!