| gibi | good morning | 07:13 |
|---|---|---|
| opendevreview | Federico Ressi proposed openstack/nova master: [WIP] Investigate on Nova quota limits problem https://review.opendev.org/c/openstack/nova/+/806683 | 07:23 |
| opendevreview | Federico Ressi proposed openstack/nova master: [WIP] Check Nova project changes with tobiko scenario test cases https://review.opendev.org/c/openstack/nova/+/806853 | 07:26 |
| opendevreview | Federico Ressi proposed openstack/nova master: [WIP] Debug APIs call failures https://review.opendev.org/c/openstack/nova/+/806683 | 07:29 |
| *** yoctozepto1 is now known as yoctozepto | 07:49 | |
| bauzas | good morning Nova | 07:49 |
| lyarwood | Morning | 08:20 |
| gibi | o/ | 08:21 |
| * kashyap waves | 08:28 | |
| gibi | hm recently I see a unit test taking 160 secs to run | 08:28 |
| gibi | nova.tests.unit.virt.vmwareapi.test_vm_util.VMwareVMUtilTestCase.test_create_vm_invalid_guestid [160.003706s] ... ok | 08:28 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [func test] refactor interface attach with qos https://review.opendev.org/c/openstack/nova/+/800088 | 08:31 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Support interface attach / detach with new resource request format https://review.opendev.org/c/openstack/nova/+/800089 | 08:31 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [func test] move unshelve test to the proper place https://review.opendev.org/c/openstack/nova/+/793621 | 08:31 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [nova-manage]support extended resource request https://review.opendev.org/c/openstack/nova/+/802060 | 08:31 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Reno for qos-minimum-guaranteed-packet-rate https://review.opendev.org/c/openstack/nova/+/805046 | 08:34 |
| gibi | lyarwood: if you have time, could you push this over the line? https://review.opendev.org/c/openstack/nova/+/801743 | 08:51 |
| lyarwood | ack yup looking | 08:51 |
| gibi | thanks | 08:53 |
| * bauzas faces transient network issue due to his ISP | 09:29 | |
| opendevreview | Vlad Gusev proposed openstack/nova stable/stein: Abort live-migration during instance_init https://review.opendev.org/c/openstack/nova/+/806881 | 09:49 |
| opendevreview | Merged openstack/nova master: [func test] refactor interface attach with qos https://review.opendev.org/c/openstack/nova/+/800088 | 10:51 |
| opendevreview | Merged openstack/nova master: Expose the mdev class https://review.opendev.org/c/openstack/nova/+/801743 | 11:00 |
| sean-k-mooney | bauzas: so much for fiber solveing all your internet problems | 11:09 |
| kashyap | sean-k-mooney: Don't rub it in :D | 11:11 |
| bauzas | sean-k-mooney: given we have FTTH connections for optic fiber, that means any neighbor fiber needs to be going to an central office | 11:12 |
| sean-k-mooney | yep | 11:13 |
| sean-k-mooney | is it actrully your fiber connectivy that is havving issues or is it just something simple like dns | 11:13 |
| sean-k-mooney | where you can just use a different dns provider | 11:13 |
| bauzas | sean-k-mooney: and sometimes, when a new customer wants to have a fiber ISP, the operator needs to go to the office to take the fiber and then fix it to the DSLAM | 11:14 |
| bauzas | sean-k-mooney: so I think I had problems because the guy was workin on the DSLAM | 11:15 |
| bauzas | (I mean the cabinet) | 11:16 |
| sean-k-mooney | they tend to set it up slightly differntly here. | 11:16 |
| sean-k-mooney | ya so FTTH at least where i am youhave a direct connection to an aggreation site (it can be a cabinet but ussually larger) and the tehy use a fiber switch to connect you to a a backhaul network | 11:17 |
| bauzas | sean-k-mooney: in my department, the operator uses a p2p network for fibers | 11:17 |
| bauzas | each fiber goes to the DSLAM | 11:18 |
| sean-k-mooney | so if my neighbour is getting fiber installed it should not affect me as that will go over its own direct conencted fiber line to the aggreation point | 11:18 |
| bauzas | but in Grenoble (which is in my department but has other operators), it mixes fibers *before* the DSLAM | 11:18 |
| bauzas | so, our development is better, as any fiber isn't mixed | 11:19 |
| bauzas | and we don't have equipments between my own fiber and the DSLAM | 11:19 |
| bauzas | but that also means that if we have problems on the fiber itself, or at the DSLAM, then I'm impacted | 11:20 |
| sean-k-mooney | ya in ireland all the fiber is own by eirgrid and they do direct connection to houses but then aggreate mutiple frequense over the same fiber link only after that | 11:20 |
| artom_ | I thought DSLAM was for DSL | 11:20 |
| sean-k-mooney | i belive there is an optical version of it too that | 11:22 |
| bauzas | sean-k-mooney: sorry, I only know the french terms, but do you know whether the fibers are mixed between the ISP and your home or not ? | 11:22 |
| sean-k-mooney | dslam is the acronmy for the copper dsl version | 11:22 |
| bauzas | sorry, french terms here https://lafibre.info/gpon/comprendre-les-differences-entre-la-fibre-gpon-p2p-active-ethernet-fttla/ | 11:22 |
| bauzas | gpon fiber vs. p2p fiber | 11:23 |
| bauzas | ie. we don't have with p2p fiber some cabinets in the streets for mixing fibers | 11:23 |
| sean-k-mooney | bauzas: all our fiber is manage seperatly form the isps and there is no mixing between me and then first level aggration site so my fiber link goes dirctly to a fiber switch | 11:24 |
| bauzas | we have straight large optic cables that can take 800 lines each | 11:24 |
| bauzas | sean-k-mooney: ok so like me | 11:24 |
| bauzas | but yeah, here is the same | 11:24 |
| sean-k-mooney | so as long as the uplinks to that switch are not messed with my neigbors can get fiber installed lighting up one of the dark strands without impact | 11:25 |
| bauzas | one operator is responsable for managing a city network | 11:25 |
| sean-k-mooney | if that switch dies we all get taken out | 11:25 |
| bauzas | and then ISPs are only responsible for the connectivity to the last mile | 11:25 |
| bauzas | oh wait, then you have active internet ftth | 11:26 |
| bauzas | a local switch located in a street cabinet | 11:26 |
| sean-k-mooney | its within 10km or so | 11:26 |
| opendevreview | Lee Yarwood proposed openstack/nova master: Add two new hacking rules https://review.opendev.org/c/openstack/nova/+/805668 | 11:27 |
| sean-k-mooney | but ya it depends on desity where i am its likely closer due to the population desity | 11:27 |
| bauzas | very interesting | 11:27 |
| bauzas | I'm afraid to only knowing french terminologies | 11:28 |
| bauzas | but at least, I know my SPOFs | 11:28 |
| sean-k-mooney | i did not see a fiber cabinet comming into where i live but there is likely one on the road within a km or so | 11:28 |
| bauzas | that's something you should take interest | 11:29 |
| bauzas | as for example, depending on the deployment technology, you could be limited with bandwitdh upgrades | 11:29 |
| bauzas | ie. a GPON FTTH is limited because this is mixed | 11:30 |
| bauzas | you can only adress 2.5 Gbps per client | 11:30 |
| bauzas | while with P2P, changing the bandwidth is only an ISP question | 11:31 |
| gibi | who needs more than 2.5 Gpbs? | 11:31 |
| sean-k-mooney | bauzas: i think technially i can get 10G if i needed too although i might need to replace the optical terminator on my side | 11:32 |
| bauzas | gibi: :D | 11:32 |
| sean-k-mooney | they are currently only trilaing grater then 1G is some areas | 11:32 |
| sean-k-mooney | i dont think mine is one of them | 11:32 |
| bauzas | gibi: trust me, I'm sometimes exhausting my 1gbps connection | 11:32 |
| gibi | I know the joke about 64Kb but honestly 2.5Gpb?! | 11:32 |
| gibi | bauzas: do you download to /dev/null? :D | 11:33 |
| bauzas | I'm tempted to choose another ISP that proposes 2.5Gbps on two NICs | 11:33 |
| bauzas | so I could load-balance | 11:33 |
| bauzas | gibi: heh, downloading is always greedy | 11:34 |
| bauzas | gibi: but funnily enough, I discovered after 20 ans what IO-bound means | 11:34 |
| bauzas | 20 years* | 11:34 |
| bauzas | now, I'm limited by my clients, not by my bandwidth | 11:34 |
| sean-k-mooney | i used to have semetriic 1gbps but they apparenly only did that for an intoductray trial and dropped my upload to 200mb. i have my home cloud offline for the last while but i was actully ortinnaly more interestedin getting native ipv6 which my current isp does not have an more upload so i could host a few things here | 11:35 |
| bauzas | sean-k-mooney: yeah, I'm lazy with moving to v6 | 11:35 |
| bauzas | while I should | 11:35 |
| bauzas | my ISP is proposing it and I could manage it | 11:36 |
| bauzas | but I'm still concerned by 6to4 | 11:36 |
| sean-k-mooney | i used ot use hricane eltric to provide a tunnel for ipv6 but there latency has gone up recently so removed it about 2 months ago | 11:36 |
| bauzas | that's the whole problem with v6 | 11:36 |
| bauzas | until the whole internet has moved, you'll face problems with 6 :( | 11:37 |
| sean-k-mooney | my isp recentl up there price so im going to move before the end of the year but native ipv6 is on my list of things to look for | 11:37 |
| sean-k-mooney | bauzas: oh we have dual stack connectivity | 11:37 |
| bauzas | hah | 11:37 |
| sean-k-mooney | there is no ipv6 only isp in ireland | 11:37 |
| sean-k-mooney | most are still ipv4 only some offer native ipv6 in addtion | 11:38 |
| gibi | I can write to my m2 ssd with 290MB/s you 1Gpbs download means ~128MB/s writing so a 2.5 Gpbs download would be limited by the local disk receiving the write | 11:38 |
| sean-k-mooney | one even give you native ipv4 and ipv6 without CGNAT on teh ipv4 side but they are $$$$ | 11:38 |
| bauzas | sean-k-mooney: we have one provider that proposes ipv6 native | 11:39 |
| bauzas | but this is an option | 11:39 |
| bauzas | gibi: trust me, I know it ;) | 11:39 |
| bauzas | a classic dd works | 11:40 |
| bauzas | but when it's about collecting pieces from the internet and reassembling them on the fly, I'm | 11:40 |
| bauzas | **really IO-bound, even with 1gbps :) | 11:40 |
| gibi | yeah I can imagine | 11:40 |
| bauzas | yeah, reading newsgroups takes time... :) | 11:41 |
| bauzas | anyway, the dudes probably have stopped to work at the network room | 11:42 |
| sean-k-mooney | god i havent done that in at least a decade. i did for a short while in collage but never found it a good way to consume info | 11:42 |
| bauzas | looks like my gitches are gone | 11:42 |
| opendevreview | Lee Yarwood proposed openstack/nova master: workarounds: Remove disable_native_luksv1 https://review.opendev.org/c/openstack/nova/+/805647 | 11:51 |
| opendevreview | Lee Yarwood proposed openstack/nova master: workarounds: Remove rbd_volume_local_attach https://review.opendev.org/c/openstack/nova/+/805648 | 11:51 |
| lyarwood | bauzas / gibi ; ^ should be ready now, apologies for the mix up with the reno. | 11:52 |
| opendevreview | Stephen Finucane proposed openstack/python-novaclient master: Add pre-commit https://review.opendev.org/c/openstack/python-novaclient/+/806915 | 11:52 |
| opendevreview | Stephen Finucane proposed openstack/python-novaclient master: tests: Add missing 'nova update' tests https://review.opendev.org/c/openstack/python-novaclient/+/806916 | 11:52 |
| opendevreview | Stephen Finucane proposed openstack/python-novaclient master: Microversion 2.90 - Configurable hostnames https://review.opendev.org/c/openstack/python-novaclient/+/806917 | 11:53 |
| opendevreview | Stephen Finucane proposed openstack/python-novaclient master: trivial: Remove use of kwargs https://review.opendev.org/c/openstack/python-novaclient/+/806918 | 11:53 |
| bauzas | lyarwood: ack, going lunching, but will click to look at it after | 11:53 |
| *** brinzhang_ is now known as brinzhang | 12:00 | |
| * lyarwood also drops for lunch | 12:02 | |
| gibi | lyarwood: +2 on both | 12:05 |
| stephenfin | lyarwood: friendly reminder to ensure you've made any changes necessary to OSC for microversion 2.89 | 12:20 |
| * stephenfin didn't review the patch is and is not entirely sure whether there are any changes | 12:21 | |
| stephenfin | s/patch is/patch/ | 12:21 |
| lyarwood | stephenfin: https://review.opendev.org/c/openstack/python-openstackclient/+/804377 | 13:02 |
| * lyarwood drops for a few hours | 13:23 | |
| gibi | sean-k-mooney: fyi just got reported from downstream https://bugs.launchpad.net/nova/+bug/1942329 | 13:30 |
| sean-k-mooney | gibi: yep this is a know issue | 13:33 |
| gibi | is it an older bug report open for it I should duplicate to? | 13:34 |
| gibi | is there | 13:34 |
| sean-k-mooney | good question tl;dr when drict physical supprot was added they did not add support for any move operations | 13:34 |
| sean-k-mooney | so cold migration resize and shelve are all broken | 13:35 |
| sean-k-mooney | i cant recall if i blocked it for live migration or if that is also affected | 13:35 |
| sean-k-mooney | we have downstream bugs for this https://bugzilla.redhat.com/show_bug.cgi?id=1745842 | 13:36 |
| sean-k-mooney | i dont see an upstream one linked so im not sure if we have an upstream bug but we have talked about it at the ptg before i think | 13:37 |
| gibi | thanks | 13:38 |
| sean-k-mooney | ah yes line 420 https://etherpad.opendev.org/p/nova-xena-ptg | 13:41 |
| opendevreview | Merged openstack/nova stable/victoria: Reproducer unit test for bug 1860312 https://review.opendev.org/c/openstack/nova/+/802842 | 13:42 |
| sean-k-mooney | no upstream links form ath either altough i did not include the downstreeam one so either but lets use your new one for now | 13:42 |
| gibi | ohh cool, I have a very bad memory | 13:44 |
| sean-k-mooney | this has been on my todo list to fix evenuatlly but it keep getting pushed off. artom started fixing some of the degecases and we have resolved other incidnetally. | 13:45 |
| sean-k-mooney | for example live migration did not wrok because 1 the mac was not updated and 2 detach of PF was not implmented | 13:46 |
| sean-k-mooney | gibi: you fixed detach when you added sriov attach/detach supprot | 13:46 |
| gibi | yepp | 13:46 |
| gibi | and I see artom fixed the unshelve pci_slot case | 13:46 |
| sean-k-mooney | yes | 13:46 |
| sean-k-mooney | so we have been trying to fix the more commens cases first when we get time too work on bugs | 13:47 |
| gibi | I made TODO now on my side to look at the MAC update at some point | 13:47 |
| sean-k-mooney | ack, so it a general problem for all pf move operation including evacuate | 13:48 |
| sean-k-mooney | as you noted we do update the neutron port on vm boot intially its just never done after that | 13:48 |
| gibi | yepp | 13:48 |
| sean-k-mooney | so we need to add the same logic in all the places to update the neutron port mac | 13:48 |
| sean-k-mooney | the problem is that that is not part of multipe port bidnign which makes it annoying | 13:48 |
| gibi | ohh then live migration will be a special case | 13:49 |
| kashyap | gibi: Is it something recent - Zuul requires uBlockOrigin to be disabled? | 13:49 |
| kashyap | (And Privacy Badger) | 13:49 |
| gibi | kashyap: didn't noticed yet | 13:49 |
| sean-k-mooney | yep gibi so if you work on this proably leave live migration till last | 13:50 |
| gibi | sean-k-mooney: ack, good idea | 13:50 |
| sean-k-mooney | we could add a api block noting its not implemented in the interim with a workaround flag to allow operators to renable it | 13:50 |
| sean-k-mooney | its one of the things we disucssed downstream i just never got around to pusshing a patch | 13:51 |
| gibi | it is not utterly broken as we are not leaking resources or breaking the guest, so I would not block the operation | 13:51 |
| sean-k-mooney | well actully it kind of is | 13:51 |
| sean-k-mooney | the vm will migrate but it can cause other vms to fail to boot | 13:52 |
| gibi | why? | 13:52 |
| sean-k-mooney | when we release the PF on the source | 13:52 |
| kashyap | When I opened a Zuul URL to debug, it was asking me to "please try with disabling ad blocker". /me tries. It's iether that, or it's just a stable URL | 13:52 |
| kashyap | s/stable/stale/ | 13:52 |
| sean-k-mooney | if another vm in the same network is shcudled to the source node then it will get a mac conflict in neutron when it trys to update the mac | 13:52 |
| sean-k-mooney | gibi: also the vm that was migrate will not get an adress form dhcp proeprly | 13:53 |
| gibi | Ohh so neutron will reject the AMC update | 13:53 |
| gibi | :/ | 13:53 |
| gibi | OK then it is broken | 13:53 |
| sean-k-mooney | yep | 13:53 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Support interface attach / detach with new resource request format https://review.opendev.org/c/openstack/nova/+/800089 | 13:53 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [func test] move unshelve test to the proper place https://review.opendev.org/c/openstack/nova/+/793621 | 13:53 |
| sean-k-mooney | so unless you are using static ips the vm that was migrated will be broken too | 13:53 |
| gibi | kashyap: could paste a link you are strugling with? | 13:53 |
| kashyap | gibi: https://zuul.opendev.org/t/openstack/build/a4b92ce40be841eaaf3e97f00c39b4d4 | 13:54 |
| kashyap | gibi: It's from June, granted | 13:54 |
| gibi | sean-k-mooney: ahh so the dhcp agent uses the port.mac to provide the ip | 13:54 |
| kashyap | Probably it just got garbage-collected | 13:54 |
| gibi | kashyap: it is already gc-d | 13:54 |
| gibi | kashyap: you need to recheck the patch | 13:54 |
| kashyap | Yep | 13:54 |
| sean-k-mooney | gibi: yes and the port will have the mac of the new PF | 13:54 |
| gibi | yep I got it know | 13:55 |
| gibi | then Im OK to block the operation | 13:55 |
| sean-k-mooney | gibi: so either the dhcp server will not respond or it will get the wrong ip if you have moved several vms | 13:55 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [nova-manage]support extended resource request https://review.opendev.org/c/openstack/nova/+/802060 | 13:56 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Reno for qos-minimum-guaranteed-packet-rate https://review.opendev.org/c/openstack/nova/+/805046 | 13:57 |
| opendevreview | Ghanshyam proposed openstack/nova master: Convert features not supported error to HTTPBadRequest https://review.opendev.org/c/openstack/nova/+/806294 | 14:10 |
| gmann | stephenfin: gibi ^^ resolved merge conflict, | 14:12 |
| gibi | gmann: done | 14:13 |
| gmann | gibi: thanks | 14:13 |
| gibi | lyarwood: fyi I found a bug in the libvirt device detach series https://bugs.launchpad.net/nova/+bug/1942345 | 14:50 |
| opendevreview | Merged openstack/nova stable/victoria: Reduce mocking in test_reject_open_redirect for compat https://review.opendev.org/c/openstack/nova/+/803093 | 15:04 |
| *** erbot_ is now known as erbot__ | 15:09 | |
| *** erbot__ is now known as erbot___ | 15:09 | |
| *** erbot___ is now known as erbot____ | 15:09 | |
| *** erbot____ is now known as erbot_____ | 15:09 | |
| *** erbot_____ is now known as erbot______ | 15:09 | |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Parse alias from domain hostdev https://review.opendev.org/c/openstack/nova/+/806943 | 15:38 |
| gibi | I marked https://bugs.launchpad.net/nova/+bug/1942345 as xena-rc-potential as we regressed SRIOV PF detach in Xena | 15:41 |
| sean-k-mooney | oh it broke ya that would be rc bug then | 15:42 |
| sean-k-mooney | i might see if we can get this in whitebox ot try and test it eventually | 15:43 |
| sean-k-mooney | unfortunetly we dont have a way to run against upstream with sriov currently | 15:43 |
| gibi | yepp, it would be nice to get automatic coverage of SRIOV ops as I'm lazy to run SRIOV tests manually in my lab env frequently | 15:45 |
| gibi | I discovered this by playing with the SRIOV migration MAC issue | 15:45 |
| gibi | melwitt: if you are up, do you have any opinion about https://storyboard.openstack.org/#!/story/2009159 and the related discussion happened yesterday on the weekly meeting? | 16:02 |
| sean-k-mooney | gibi: i know that there is limited coverage by the melonox ci but they have not added to that as we added new feature like sriov attaach/detach | 16:04 |
| melwitt | gibi: oh sorry I had missed that. need to read it | 16:06 |
| melwitt | gah this traceback formatting on storyboard -_- | 16:08 |
| melwitt | gibi: on the surface it seems like we have to change it back and do something manual up the stack to handle the consumer type update rollback :/ not sure what else can be done about it | 16:11 |
| opendevreview | Merged openstack/nova master: workarounds: Remove disable_native_luksv1 https://review.opendev.org/c/openstack/nova/+/805647 | 16:11 |
| gibi | melwitt: yepp we discussed that on the meeting yesterday that we can do project and user creation outside of the main transaction and only move the consumer type and the allocation update into the last big transaction | 16:12 |
| melwitt | gibi: oh, that's a better idea | 16:12 |
| melwitt | I support that | 16:12 |
| gibi | I had similar idea than yours first, then on the meeting I was convinced | 16:13 |
| gibi | do you have time to work on a solution? | 16:13 |
| melwitt | gibi: yeah I can, especially since I caused it -_- | 16:14 |
| melwitt | I'll review your repro patch (thank you for that) and then stack a fix on top of it | 16:14 |
| gibi | melwitt: thank you! let me know if I can help somehow | 16:15 |
| opendevreview | Merged openstack/python-novaclient master: Add pre-commit https://review.opendev.org/c/openstack/python-novaclient/+/806915 | 16:34 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Avoid excessive sleep in vmware unit test https://review.opendev.org/c/openstack/nova/+/806956 | 16:38 |
| opendevreview | Merged openstack/nova master: Add some missing parameters in docs of os-cells https://review.opendev.org/c/openstack/nova/+/805972 | 17:06 |
| opendevreview | Merged openstack/nova master: tests: Remove unnecessary warnings filter https://review.opendev.org/c/openstack/nova/+/800901 | 17:06 |
| opendevreview | Merged openstack/nova master: Add test coverage for security group checking https://review.opendev.org/c/openstack/nova/+/786044 | 17:06 |
| opendevreview | Merged openstack/nova master: Convert features not supported error to HTTPBadRequest https://review.opendev.org/c/openstack/nova/+/806294 | 17:36 |
| opendevreview | Merged openstack/nova master: doc: allow_resize_to_same_host needs nova-api restarted https://review.opendev.org/c/openstack/nova/+/801275 | 22:02 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!