| opendevreview | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.opendev.org/c/openstack/nova/+/846876 | 03:49 |
|---|---|---|
| opendevreview | OpenStack Proposal Bot proposed openstack/python-novaclient master: Imported Translations from Zanata https://review.opendev.org/c/openstack/python-novaclient/+/846880 | 03:56 |
| bauzas | gibi: you maybe missed https://review.opendev.org/c/openstack/nova/+/846786 | 07:40 |
| gibi | bauzas: I think what you did is what we discussed. I would add some testing around the for:else: and the breaks but otherwise I agree with the direction | 07:42 |
| bauzas | gibi: yup, of course, this is a WIP b/c I would add a UT | 07:43 |
| gibi | so then that WIP looks OK to me | 07:44 |
| elodilles | bauzas sean-k-mooney : can you please review the release patches (whether you see something that requires MINOR version bump, etc). I don't think it's worth to wait more time, as every week we'll have new patches + gate is sometimes slow and days are passing o:) | 09:16 |
| elodilles | bauzas sean-k-mooney : of course if you see any patch that is a MUST to have in any of the release, then we can wait. but otherwise i think it's better to release now o:) | 09:16 |
| elodilles | bauzas sean-k-mooney : and release managers won't review the release patches until PTL or release liaisons haven't reviewed the patches ;) | 09:17 |
| sean-k-mooney[m] | actully i ment to update the patches | 09:18 |
| sean-k-mooney[m] | to use the new sha last week | 09:19 |
| sean-k-mooney[m] | most of the patches i wanted to wait for have alredy landed last week | 09:19 |
| sean-k-mooney[m] | we just need to repin the patches to point to the currnent tip of the stable branches | 09:19 |
| sean-k-mooney[m] | elodilles give me 10 mins to grab a coffee and ill go do that when i get back and +1 them | 09:21 |
| elodilles | sean-k-mooney[m]: i've updated the patches with the latest hashes :) | 09:21 |
| elodilles | sean-k-mooney[m]: no need to hurry as i also will go to have lunch ;) | 09:21 |
| elodilles | so enjoy your coffee, and review the patches some time later today o:) | 09:22 |
| sean-k-mooney[m] | oh ok thanks i stilll ment to do this last week so before i start anything else ill review them | 09:22 |
| bauzas | elodilles: ack, will look | 09:23 |
| bauzas | gibi: reopened https://bugs.launchpad.net/nova/+bug/1890244 fwiw | 09:23 |
| elodilles | sean-k-mooney[m] bauzas: ++, thanks in advance! \o/ | 09:23 |
| bauzas | gibi: I'm curious how you didn't went able to reproduce the bug in https://bugs.launchpad.net/nova/+bug/1890244/comments/1 | 09:26 |
| gibi | bauzas: so probably CONF.workarounds.disable_group_policy_check_upcall was the trick why I was not able to reproduce it | 09:26 |
| bauzas | gibi: hah | 09:26 |
| bauzas | gibi: but IMHO we default to False | 09:27 |
| gibi | then I dont know | 09:27 |
| gibi | bauzas: did you reproduced it? | 09:28 |
| bauzas | I need to respin my devstack env | 09:28 |
| bauzas | gibi: no, I only triaged it and we have a downstream BZ that was related | 09:28 |
| bauzas | https://bugzilla.redhat.com/show_bug.cgi?id=2099279 | 09:28 |
| bauzas | the code is pretty simple to look at | 09:29 |
| bauzas | oh wait, you could be right, shit. | 09:30 |
| bauzas | when we populate the ReqSpec, we rehydrate the fields and we can set the group value to None if the group was deleteed | 09:31 |
| bauzas | https://github.com/openstack/nova/commit/94fd36f0582c5dbcf2b9886da7c7bf986d3ad5d1 is super old | 09:32 |
| gibi | I still cannot reproduce | 09:35 |
| gibi | https://paste.opendev.org/show/bXPvHPVJ7FfPgjfhTsSv/ | 09:35 |
| bauzas | gibi: then I don't understand | 09:37 |
| bauzas | gibi: I verified master | 09:37 |
| gibi | the above repro trial I run on master too | 09:38 |
| bauzas | gibi: when we evacuate, we regenerate the old RequestSpec record | 09:38 |
| gibi | I don't have a multinode train env | 09:38 |
| bauzas | and then, we set the hints | 09:38 |
| bauzas | gibi: can you look at your reqspec record and see whether you have the hints dict ? | 09:39 |
| gibi | sure | 09:40 |
| gibi | https://paste.opendev.org/show/baqepgHm3PhuQ9VhvfWi/ | 09:41 |
| gibi | there is the group hint in the dfb | 09:41 |
| gibi | db | 09:41 |
| bauzas | gibi: ok, and you can confirm that df92c028-728a-4321-9df6-6424ca56969e no longer exists in the instance_groups table ? | 09:41 |
| bauzas | ideally we should introspect into https://github.com/openstack/nova/blob/ebe08834f311e8e22bfd9685d7e6e91dab967382/nova/compute/manager.py#L3657 | 09:42 |
| gibi | yes https://paste.opendev.org/show/b9NTbUJdRy4zI9qVlFsL/ | 09:42 |
| bauzas | gibi: could you see what we get as hints from https://github.com/openstack/nova/blob/ebe08834f311e8e22bfd9685d7e6e91dab967382/nova/compute/manager.py#L3656 ? | 09:43 |
| bauzas | b/c I don't see where we could be smart | 09:43 |
| bauzas | we're just getting the hints there | 09:43 |
| gibi | I can add a LOG and repro give me 5 | 09:44 |
| bauzas | and see whether this is a group | 09:44 |
| bauzas | if this a group, we directly lookup the instance_groups table with this UUID | 09:44 |
| bauzas | and that's the upcall | 09:44 |
| sean-k-mooney | so honestly im surpiesed we allow you to delete an instance group if there are instances in it | 09:45 |
| sean-k-mooney | to me that is the really issue here | 09:46 |
| sean-k-mooney | is there a reason we allow that | 09:46 |
| sean-k-mooney | unlike the flavor for example we are not embdeding a copy fo the instance group | 09:46 |
| sean-k-mooney | so we really should not allow ti to be remvoed if its in use | 09:47 |
| sean-k-mooney | do we know why we do today? | 09:47 |
| bauzas | sean-k-mooney: because Instance Groups are a terrible concept ?N | 09:48 |
| sean-k-mooney | not nessiarly they may be implemented badly | 09:49 |
| sean-k-mooney | the concept is not a bad one but the way we implemeted them is not good | 09:49 |
| * bauzas reminds me the group update BPs we had in the past and how badly we ended up in a corner | 09:49 | |
| gibi | bauzas: could this be the reason we bail out https://github.com/openstack/nova/blob/ebe08834f311e8e22bfd9685d7e6e91dab967382/nova/compute/manager.py#L1742 | 09:52 |
| gibi | (still instrumenting the code with logs then I will rerun the repro attempt) | 09:53 |
| gibi | btw I agree with sean that group is a better concept than same_host / different_host hints, as the hints were not reflexive | 09:55 |
| gibi | strike reflexive | 09:55 |
| gibi | symmetric | 09:55 |
| gibi | they wasnt symmetirc | 09:55 |
| * bauzas needs to go sweating | 09:55 | |
| bauzas | let's discuss this tomorrow at the bug call | 09:56 |
| bauzas | shit, d/s | 09:56 |
| gibi | I have /etc/nova/nova-cpu.conf:disable_group_policy_check_upcall = True in my env, restaring repro .... | 10:11 |
| sean-k-mooney | isnt that our default | 10:12 |
| gibi | nope | 10:14 |
| gibi | the default is false | 10:14 |
| gibi | I had an explicit True set for some reason | 10:14 |
| sean-k-mooney | devstack set it to true by defualt i think | 10:17 |
| gibi | yepp devastack by default does not even support the upcall | 10:17 |
| gibi | so If I enable the upcall nova blows as no DB access | 10:17 |
| sean-k-mooney | wait what | 10:19 |
| sean-k-mooney | it shoudl be doing that via rpc to conductor | 10:19 |
| sean-k-mooney | is it trying to use the old local conductor path | 10:19 |
| sean-k-mooney | and directly access the db | 10:19 |
| gibi | https://paste.opendev.org/show/bxWFPkh1OoBfWlM15mS0/ | 10:20 |
| gibi | it calls up to the cell conductor | 10:20 |
| gibi | and cell conductor fails to read the api db | 10:21 |
| gibi | as cell conductors has no access to the api db | 10:21 |
| sean-k-mooney | oh | 10:21 |
| sean-k-mooney | right | 10:21 |
| gibi | the upcall is between the cell and the super conductor | 10:21 |
| sean-k-mooney | so this will only work in singel conductor mode | 10:21 |
| sean-k-mooney | which devstack support | 10:21 |
| sean-k-mooney | but its not the default | 10:21 |
| gibi | I can simply promote my cell conductor to a super one by adding the api db settings to it | 10:21 |
| sean-k-mooney | yep that would work | 10:22 |
| gibi | but this really feels like something that upstream we dont test hence dont really support | 10:22 |
| gibi | anyhow I do the propomotion and retry | 10:22 |
| sean-k-mooney | well we do have supprot to test this in devstack | 10:22 |
| gibi | do we have jobs testing it? | 10:22 |
| sean-k-mooney | i dobt it | 10:22 |
| gibi | we have code, we dont have test :D | 10:22 |
| sean-k-mooney | ill check with codesearch | 10:23 |
| sean-k-mooney | and see if anything enables it | 10:23 |
| gibi | anyhow I go and do the promotion now and retry. I still believe that nova handles the missing group gracefully | 10:23 |
| sean-k-mooney | so kolla-ansible disable the upcall out of the box i didnt get any hits form tripleo or pupet so i assuem they have it enabled | 10:24 |
| sean-k-mooney | ooo actully would work since it does not supprot super conductors | 10:24 |
| sean-k-mooney | well if you do multi cell its usign the cell1 conductor as the super conductor | 10:25 |
| sean-k-mooney | so it would work only in cell1 | 10:25 |
| gibi | intersting setup | 10:26 |
| gibi | :) | 10:26 |
| sean-k-mooney | https://opendev.org/openstack/grenade/src/branch/master/.zuul.yaml#L188= | 10:26 |
| sean-k-mooney | so our grenade job apprently uses singel conductor mode | 10:26 |
| sean-k-mooney | so we could renable the upcall check there if we wanted | 10:27 |
| sean-k-mooney | gibi: could we consider changing the default for this or removing the call eventually | 10:27 |
| gibi | yeah but there we dont do evac testing | 10:27 |
| gibi | sean-k-mooney: I don't know what is the startegy I think originally we wanted to get rid of all the upcall | 10:28 |
| gibi | s | 10:28 |
| sean-k-mooney | yes we did | 10:28 |
| gibi | but apperantly we never went there and replaced them with calls pushing down the necessary daya | 10:28 |
| sean-k-mooney | we currently only have 2 i think | 10:28 |
| gibi | data | 10:28 |
| sean-k-mooney | we have the track_instance_changes config option ot push data to the schduler | 10:29 |
| sean-k-mooney | and the group policy check upcall | 10:29 |
| gibi | OK now I can reproduce the issue on master https://paste.opendev.org/show/btK22SLw4UH6oDDu5wKw/ | 10:32 |
| gibi | we go this way https://github.com/openstack/nova/blob/ebe08834f311e8e22bfd9685d7e6e91dab967382/nova/compute/manager.py#L1733 | 10:32 |
| gibi | and there we dont have the except branch | 10:32 |
| gibi | not like in L1741 | 10:32 |
| sean-k-mooney | yep | 10:33 |
| sean-k-mooney | that what i was saying downstream | 10:33 |
| gibi | now we have proof :) | 10:34 |
| sean-k-mooney | we jut need to move the try to the top of the if | 10:34 |
| gibi | bauzas: ^^ I go re-triage the upstream bug | 10:34 |
| sean-k-mooney | so https://github.com/openstack/nova/blob/ebe08834f311e8e22bfd9685d7e6e91dab967382/nova/compute/manager.py#L1738= need to move to https://github.com/openstack/nova/blob/ebe08834f311e8e22bfd9685d7e6e91dab967382/nova/compute/manager.py#L1718= | 10:34 |
| sean-k-mooney | it shoudl be easy to prove that with a unit test too | 10:35 |
| gibi | sean-k-mooney, bauzas: I updated the upstream bug. I agree that we should fix this as suggested ^^. | 10:39 |
| gibi | but now I go get some lunch and go back to the pci tracking patches | 10:40 |
| sean-k-mooney | gibi: do you want me to just submit a unit test repoducer and then move the try | 10:42 |
| gibi | sean-k-mooney: sure you can go | 10:42 |
| gibi | I'm not assigning the bug to me :) | 10:42 |
| opendevreview | sean mooney proposed openstack/nova master: add repoducer test for bug 1890244 https://review.opendev.org/c/openstack/nova/+/847000 | 11:25 |
| opendevreview | sean mooney proposed openstack/nova master: ignore deleted server groups in validation https://review.opendev.org/c/openstack/nova/+/847001 | 11:25 |
| sean-k-mooney | bauzas: gibi ^ | 11:25 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Basics for PCI Placement reporting https://review.opendev.org/c/openstack/nova/+/846187 | 11:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Extend device_spec with resource_class and traits https://review.opendev.org/c/openstack/nova/+/846218 | 11:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Reject PCI dependent device config https://review.opendev.org/c/openstack/nova/+/846435 | 11:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Reject mixed VF rc and trait config https://review.opendev.org/c/openstack/nova/+/846436 | 11:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Ignore PCI devs with physical_network tag https://review.opendev.org/c/openstack/nova/+/846219 | 11:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Reject devname based device_spec config https://review.opendev.org/c/openstack/nova/+/846466 | 11:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Support [pci]device_spec reconfiguration https://review.opendev.org/c/openstack/nova/+/846470 | 11:54 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Stop if tracking is disable after it was enabled before https://review.opendev.org/c/openstack/nova/+/847009 | 11:54 |
| opendevreview | Rajesh Tailor proposed openstack/nova stable/wallaby: Add missing condition https://review.opendev.org/c/openstack/nova/+/847011 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Add more test coverage for devname base dev spec https://review.opendev.org/c/openstack/nova/+/844625 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Extra tests for remote managed dev spec https://review.opendev.org/c/openstack/nova/+/844626 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Unparent PciDeviceSpec from PciAddressSpec https://review.opendev.org/c/openstack/nova/+/844491 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Fix PciAddressSpec descendants to call super.__init__ https://review.opendev.org/c/openstack/nova/+/844565 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Remove dead code from PhysicalPciAddress https://review.opendev.org/c/openstack/nova/+/844628 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Poison /sys access via various calls in test https://review.opendev.org/c/openstack/nova/+/844627 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Clean up mapping input to address spec types https://review.opendev.org/c/openstack/nova/+/845765 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Remove unused PF checking from get_function_by_ifname https://review.opendev.org/c/openstack/nova/+/845775 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Fix type annotation of pci.Whitelist class https://review.opendev.org/c/openstack/nova/+/845780 | 11:58 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Move __str__ to the PciAddressSpec base class https://review.opendev.org/c/openstack/nova/+/845781 | 11:58 |
| gibi | sean-k-mooney, bauzas: I consider the first (first and the half:D) batch of patches of pci tracking in placement ready for review. I posted a rewiew guide to the ML for help http://lists.openstack.org/pipermail/openstack-discuss/2022-June/029156.html | 12:07 |
| gibi | sean-k-mooney: I'm +2 on the evac server group fix | 12:10 |
| gibi | bauzas: easy win ^^ | 12:10 |
| sean-k-mooney | :) i see it in my inbox ill try and take a look at them this week | 12:13 |
| sean-k-mooney | """This covers the first 4 sub chapters of Proposed Change""" | 12:14 |
| sean-k-mooney | have i said how happy i am that you volenterred to work on this :) | 12:15 |
| gibi | yes you have :) | 12:17 |
| gibi | actually after the initial shock, this batch of patches was a nice coding excersize | 12:17 |
| opendevreview | Rajesh Tailor proposed openstack/nova master: Remove unnecessary if condition https://review.opendev.org/c/openstack/nova/+/844418 | 12:24 |
| bauzas | gibi: sean-k-mooney: sorry, was doing other things, will look above | 12:42 |
| gibi | bauzas: no worries | 12:42 |
| bauzas | sean-k-mooney: thanks for having worked on it, it was an easy peasy | 12:59 |
| *** ralonsoh_ is now known as ralonsoh | 13:00 | |
| sean-k-mooney | yep simple fix | 13:01 |
| bauzas | sean-k-mooney: just -1 because of the try clause https://review.opendev.org/c/openstack/nova/+/847001 | 13:02 |
| sean-k-mooney | i will have to duplicate the try | 13:03 |
| sean-k-mooney | the only place that its not needed for is the schdueler_hints.get | 13:03 |
| sean-k-mooney | i can duplicate it but i think this is cleaner but whatever ye prefer | 13:04 |
| opendevreview | sean mooney proposed openstack/nova master: ignore deleted server groups in validation https://review.opendev.org/c/openstack/nova/+/847001 | 13:14 |
| sean-k-mooney | bauzas: ^ | 13:14 |
| bauzas | sean-k-mooney: +2d with a comment on the relnote (sorry missed it at first look) | 13:31 |
| *** dasm|off is now known as dasm | 13:33 | |
| bauzas | gibi: updated the agenda for today's meeting | 14:04 |
| gibi | bauzas: thanks | 14:04 |
| bauzas | gibi: thanks for chairing it until I arrive | 14:04 |
| bauzas | Uggla: missed to review your patches, yet another day with other prios | 14:04 |
| * bauzas needs to disappear due to some responsibilities in my daughter's medium high school class (class council deleguate) | 14:19 | |
| bauzas | but I'll back around the meeting time | 14:19 |
| opendevreview | Rajesh Tailor proposed openstack/nova master: Remove unnecessary if condition https://review.opendev.org/c/openstack/nova/+/844418 | 14:36 |
| Uggla | bauzas, no pb. | 14:59 |
| Uggla | bauzas, sean-k-mooney, please have a quick look at https://review.opendev.org/c/openstack/nova/+/845581/1/nova/compute/api.py and just tell me what you prefer. Longer code with logs (current version) vs shorter one with less logs (proposed by Gibi). If you prefer Gibi's proposal, I will refactor and you will be able to do the review with an updated version. | 15:00 |
| Uggla | bauzas, sean-k-mooney, gibi can we have a look at https://review.opendev.org/c/openstack/nova-specs/+/833669/7..9/specs/zed/approved/libvirt-virtiofs-attach-manila-shares.rst#b94 to get rid of this point tomorrow morning ? | 15:02 |
| sean-k-mooney | Uggla: i think shorter | 15:03 |
| sean-k-mooney | just looking at https://review.opendev.org/c/openstack/nova/+/845581/1/nova/compute/api.py quickly | 15:03 |
| sean-k-mooney | the orginal code had a lot of repaded logs for the differnt pbrances that i dont think will be helpful | 15:04 |
| Uggla | sean-k-mooney, ok thank you. | 15:04 |
| sean-k-mooney | im not against logs but that seams pretyy verbouse with a lot of repetition | 15:05 |
| sean-k-mooney | bauzas: gibi i think https://review.opendev.org/c/openstack/nova/+/847001/2 might be incomplete | 15:12 |
| sean-k-mooney | i think this happens because while we null out the server_group in teh request_spec | 15:13 |
| sean-k-mooney | we dont remove the group form the hint | 15:13 |
| gibi | Uggla: sure, ping me tomorrow morning to talk that through | 15:13 |
| gibi | sean-k-mooney: ahh, so anther way to solve it would be to remove the hint from the request spec if the group is not found | 15:14 |
| sean-k-mooney | gibi: we proably shoudl do both | 15:14 |
| sean-k-mooney | so catch the instance not found | 15:14 |
| sean-k-mooney | but make sure when the group is deleted we remove it form the schdluer hinits | 15:15 |
| sean-k-mooney | gibi: you found where the group was deleted previously yes you dont happen to have the link to that still? | 15:15 |
| gibi | sean-k-mooney: this is where we detect that that the group is gone https://github.com/openstack/nova/commit/94fd36f0582c5dbcf2b9886da7c7bf986d3ad5d1 | 15:16 |
| gibi | sean-k-mooney: this is where the group is deleted in the API https://github.com/openstack/nova/blob/ebe08834f311e8e22bfd9685d7e6e91dab967382/nova/api/openstack/compute/server_groups.py#L145 | 15:17 |
| sean-k-mooney | ya so that sets spec.instace_group to none but does not remvoe the group from the hint | 15:17 |
| sean-k-mooney | so https://github.com/openstack/nova/commit/94fd36f0582c5dbcf2b9886da7c7bf986d3ad5d1#diff-8e61e9d80a8e925e5b0a9209a495ba0e4888e9fd81903ed0ab79edcbdb05901dR458 | 15:18 |
| sean-k-mooney | is incomplete | 15:18 |
| gibi | yeah we should extend that | 15:18 |
| gibi | nuke the group hint | 15:18 |
| gibi | as if the group does not exists it will never exesits in the future | 15:18 |
| sean-k-mooney | yep so ill do both and then adress bauzas nit in the reno | 15:19 |
| gibi | ack | 15:19 |
| sean-k-mooney | do you think this need a func test by the way or are you ok with just unit test to repoduce | 15:20 |
| sean-k-mooney | i could create one i guess | 15:20 |
| sean-k-mooney | i was trying to keep it minimal but it proably does not hurt to add one | 15:20 |
| gibi | I dont think it is hard to add an evact func test for this case. So I suggest to have one | 15:21 |
| sean-k-mooney | ya its not cool ill do that so | 15:24 |
| gibi | FYI nova weekly meeting will start in 12 minutes here in the channel | 15:47 |
| gibi | #startmeeting nova | 15:59 |
| opendevmeet | Meeting started Tue Jun 21 15:59:36 2022 UTC and is due to finish in 60 minutes. The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:59 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:59 |
| opendevmeet | The meeting name has been set to 'nova' | 15:59 |
| gibi | chair gibi bauzas | 15:59 |
| gibi | #chair gibi bauzas | 15:59 |
| opendevmeet | Current chairs: bauzas gibi | 15:59 |
| gibi | lets wait a bit before we start | 16:00 |
| elodilles | o/ | 16:02 |
| gibi | OK, lets get started | 16:02 |
| gibi | bauzas asked me to run this as he might be late a bit | 16:02 |
| gibi | #topic Bugs (stuck/critical) | 16:02 |
| gibi | #info One critical bug | 16:03 |
| gibi | #link https://bugs.launchpad.net/nova/+bug/1979047 Centos 9 Stream bug failure | 16:03 |
| gibi | the centos 9 steam job is made non-voting on Friday to unblock the gate | 16:03 |
| gibi | there is some info from tripleooo about the same issue, they pinned libvirt version | 16:04 |
| gibi | should we try to do the same? | 16:04 |
| elodilles | good question o:) is there any other option? :-o | 16:06 |
| gibi | keep it non voting forever? | 16:06 |
| gibi | :) | 16:07 |
| elodilles | :S | 16:07 |
| gibi | we would need somebody who care about this job to propose a fix | 16:07 |
| gibi | I don't see a long line of volunteers | 16:08 |
| gibi | so moving on | 16:08 |
| gibi | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 12 new untriaged bugs (-2 since the last meeting) | 16:08 |
| elodilles | i mean, does it make any difference to pin libvirt compared to keep the job non-voting ? | 16:08 |
| artom | Looks like a libvirt fix is being released, based on https://bugzilla.redhat.com/show_bug.cgi?id=2092856 | 16:08 |
| gibi | elodilles: if we keep non-voting and the libvirt issue is fixed the job will start being green again | 16:08 |
| artom | So maybe just wait for that to land in CS9, and make the job voting again? | 16:08 |
| gibi | artom: yepp, we can do that | 16:08 |
| Uggla | o/ | 16:09 |
| elodilles | ack, then that is the best option for now | 16:09 |
| gibi | ack, seems like we do that as that is easy | 16:09 |
| gibi | so untriaged bug backlog looks healthy thanks melwitt for the triages | 16:09 |
| gibi | #info If you are interested add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster | 16:10 |
| gibi | next the baton goes to bauzas | 16:10 |
| gibi | I will ping him when he is back | 16:10 |
| gibi | let's assume he took it :) | 16:10 |
| gibi | #info Next bug baton is passed to bauzas | 16:10 |
| gibi | any other bug you would like to discuss? | 16:10 |
| artom | Not me? Cool then | 16:11 |
| gibi | I think bauzas rescheduled himself as he missed having the baton during the summit | 16:11 |
| gibi | (I'm following bauzas agend from the wiki :) | 16:12 |
| artom | For once it's not my job to figure out the "overwatch" rotation :P | 16:12 |
| artom | (Sorry for leaking downstream) | 16:12 |
| gibi | anyhow I don't see any bugs raised so moving on | 16:12 |
| gibi | #topic Gate status | 16:12 |
| gibi | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:12 |
| gibi | #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status | 16:12 |
| gibi | looks green | 16:12 |
| gibi | #link https://zuul.opendev.org/t/openstack/builds?job_name=nova-emulation&pipeline=periodic-weekly&skip=0 Emulation periodic job runs | 16:13 |
| gibi | green too | 16:13 |
| gibi | #info Please look at the gate failures and file a bug report with the gate-failure tag. | 16:13 |
| gibi | #info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures | 16:13 |
| gibi | we had two gate blockers on Friday | 16:13 |
| gibi | the one with the libvirt issue from above | 16:13 |
| gibi | and another with a tempest test case failure | 16:13 |
| gibi | the tempest failure was fixed by https://review.opendev.org/c/openstack/tempest/+/846345/3 | 16:14 |
| gibi | so the gate should be OK now | 16:14 |
| gibi | but honestly I haven't checked the results today | 16:14 |
| gibi | I quickly checked the master gate is not blocked | 16:15 |
| gibi | any gate issue we need to discuss here? | 16:15 |
| gibi | #topic Release Planning | 16:16 |
| gibi | #link https://releases.openstack.org/zed/schedule.html | 16:16 |
| gibi | #info Zed-2 is in 3.5 weeks | 16:17 |
| gibi | #startvote Spec review day on July 5th ? (yes/no) | 16:17 |
| opendevmeet | Begin voting on: Spec review day on July 5th ? Valid vote options are , yes, no, . | 16:17 |
| opendevmeet | Vote using '#vote OPTION'. Only your last vote counts. | 16:17 |
| gibi | #vote yes | 16:17 |
| Uggla | #vote yes | 16:17 |
| gibi | others? | 16:18 |
| elodilles | #vote yes | 16:19 |
| elodilles | (though my vote really does not count i guess o:)) | 16:19 |
| sean-k-mooney | #vote yes | 16:20 |
| sean-k-mooney | i think i can do the 5th but folks form the us may be off | 16:20 |
| gibi | hangover from 4th of July? | 16:20 |
| sean-k-mooney | perhaps :) | 16:20 |
| sean-k-mooney | i think if there are no objections however the 5th shoudl be fine | 16:21 |
| elodilles | :) | 16:21 |
| gibi | #endvote | 16:21 |
| opendevmeet | Voted on "Spec review day on July 5th ?" Results are | 16:21 |
| opendevmeet | yes (4): Uggla, sean-k-mooney, gibi, elodilles | 16:21 |
| gibi | #action bauzas to send out a note to the ML about the spec review day on 5th of July | 16:22 |
| gibi | #topic Review priorities | 16:22 |
| gibi | #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+label:Review-Priority%252B1 | 16:22 |
| gibi | #link https://review.opendev.org/c/openstack/project-config/+/837595 Gerrit policy for Review-prio contributors flag. New proposal there, please vote. | 16:22 |
| gibi | #link https://docs.openstack.org/nova/latest/contributor/process.html#what-the-review-priority-label-in-gerrit-are-use-for Documentation we already have | 16:23 |
| gibi | anything to raise about review priority? | 16:23 |
| gibi | #topic Stable Branches | 16:24 |
| gibi | elodilles: | 16:24 |
| gibi | your turn | 16:24 |
| elodilles | #info gates are mostly not blocked | 16:24 |
| elodilles | #info stable/train is blocked - melwitt's fix: https://review.opendev.org/c/openstack/nova/+/844530/ | 16:24 |
| elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | 16:25 |
| elodilles | release patches proposed (yoga, xena, wallaby): https://review.opendev.org/q/project:openstack/releases+is:open+intopic:nova | 16:25 |
| elodilles | the release patches are on the way now | 16:25 |
| gibi | \o/ | 16:25 |
| elodilles | (sorry i've re-used last week's lines as there are not that much news o:)) | 16:25 |
| gibi | elodilles: thanks | 16:26 |
| elodilles | np | 16:26 |
| gibi | anything else about stable? | 16:26 |
| elodilles | nothing from me | 16:26 |
| gibi | #topic Open discussion | 16:28 |
| gibi | (artom) Can we revisit stable func test backport policy? Specific patch stack: https://review.opendev.org/c/openstack/nova/+/791480/1 Previously we didn't want to backport func test infrastructure because it just offloads the backport debt onto whoever is doing backport for older than train releases. Some time has passed now, are there still operators running < stable/train and needing backports? | 16:28 |
| gibi | https://etherpad.opendev.org/p/r.ea2e9bd003ed5aed5e25cd8393cf9362 indicates a majority of "train or older", but how many are on the "older" half of that? | 16:28 |
| artom | Bringing this up again because at this point there are 3 separate bugfixes depending on those test refactor patches | 16:29 |
| artom | But basically $topic :) | 16:29 |
| gibi | personally I'm OK to bring back any test refactors to stable branches | 16:29 |
| artom | IIRC last time we talked about this, elodilles was worried that anyone backporting to rocky and older would get the "refactor" debt unloaded onto them | 16:30 |
| elodilles | IF there are enough reviewers then maybe it could be OK, though it's best to keep things on the safe side and backport less risky things | 16:30 |
| artom | Are func test refactors really risky though? | 16:31 |
| sean-k-mooney | well test code is less risky in general since it does not affect the runing code | 16:31 |
| elodilles | artom: yepp, if we backport mass amount of functional test refactors, then it makes the backport harder for older branches | 16:31 |
| gibi | it does not risk the production code, it risk the CI stability | 16:31 |
| sean-k-mooney | and in some cases are not actully installed with the production code | 16:31 |
| artom | elodilles, yep, agreed on that. So in practice, bauzas and gibi were at summit, is anyone still doing backports for < stable/train? | 16:32 |
| * elodilles agrees with gibi | 16:32 | |
| artom | As in, Red Hat will have to care about stable/train for a long time | 16:32 |
| sean-k-mooney | :( | 16:32 |
| gibi | I tend to propose backports to stable/pike while I were in E/// | 16:32 |
| sean-k-mooney | its true but :( | 16:32 |
| artom | Yeah, sad face indeed | 16:32 |
| gibi | I assume E/// still uses stable/pike | 16:33 |
| gibi | but I don't think we will see much backports there | 16:33 |
| elodilles | i see that there are less and less backports pushed toward old branches, though if we make it harder for developers, then it will not help the situation as well | 16:33 |
| artom | gibi, elodilles, so I can buy the gate stability argument for integration tests, but when was the last time we had an issue with func tests that wasn't about versions of things like tox? | 16:33 |
| sean-k-mooney | elodilles: well right now its hard to backprot to train because once you get past about victoria you are missing the helpers | 16:34 |
| gibi | we have still open a bug where nova funct test leaks notifications between tests :) | 16:34 |
| elodilles | gibi: unfortunately my pike patches are hanging there without reviews, so.... o:) | 16:34 |
| gibi | elodilles: I know :) | 16:34 |
| gibi | artom: so func test could be problematic | 16:34 |
| gibi | as they run eventlets | 16:35 |
| gibi | and sometimes depends on extrenal things like sysfs :) | 16:35 |
| gibi | still I think we should backport func test infra | 16:35 |
| * sean-k-mooney notes we have backported some of the helpers downstream already | 16:35 | |
| sean-k-mooney | its the integrated_helpers that are most useful | 16:36 |
| artom | gibi, I feel like sysfs should be poisoned in func tests... | 16:37 |
| gibi | artom: I have a patch | 16:37 |
| gibi | https://review.opendev.org/c/openstack/nova/+/844627 | 16:37 |
| sean-k-mooney | artom: for the most part its mocked already modulo bugs | 16:37 |
| sean-k-mooney | but yes the poison is also good to do | 16:37 |
| gibi | does anyone here strongly disagree to backport func test infra? | 16:38 |
| elodilles | also note, i'm not completely against backporting func test refactors, but i still think it is best to keep it in a low level and we should not backport massive refactors :/ | 16:38 |
| gibi | elodilles: it is a tradeoff, either you take the risk by backporting the refactor or take the risk when you backport a fix that needs to be changed due to the missing refactor | 16:39 |
| * artom would not consider https://review.opendev.org/c/openstack/nova/+/791480/1 massive | 16:39 | |
| gibi | artom: 3 lines! come on! :) | 16:39 |
| sean-k-mooney | well or we drop the functest on backport | 16:39 |
| elodilles | if a refactor breaks something then we don't have the bandwidth to keep it maintained i think. stable should be stable :( | 16:39 |
| artom | The one on top is a bit bigger ^_^ | 16:39 |
| gibi | I strongly against droping the func test on backport | 16:39 |
| artom | elodilles, so that was the crux of my argument | 16:39 |
| artom | Red Hat *will* maintain stable/train for literally years, we have no choice | 16:40 |
| sean-k-mooney | well at least 2.5 more | 16:40 |
| artom | But we don't want to inflict pain on anyone maintaining older than stable/train | 16:40 |
| artom | So: do those folks... well, exist? :) | 16:40 |
| artom | At Summit, what release did operators say they were on? | 16:41 |
| sean-k-mooney | technially we still maintain queens downstream too for a while more | 16:41 |
| sean-k-mooney | but i would prefer to have the backports of the func infra as that makes backporting simpelr in the long run | 16:41 |
| gibi | artom: there were no specifics other than what is in the etherpad | 16:41 |
| artom | So only "train or older" with no info if it's train... or older :( | 16:42 |
| elodilles | should have been added 'train' + 'stein and older' :D | 16:42 |
| artom | Yeah :S | 16:43 |
| * artom checks stable/rocky backports: https://review.opendev.org/q/project:openstack/nova+branch:stable/rocky | 16:43 | |
| artom | So compared to stable/train, there are 4 patches last updated this year, compared to train's ~50 | 16:44 |
| elodilles | why not https://review.opendev.org/q/project:openstack/nova+branch:stable/stein ? | 16:44 |
| artom | Because I suck at alphabet :P | 16:45 |
| elodilles | o:) | 16:45 |
| artom | Seems to be mostly Felix and Vlad Gusev... | 16:45 |
| artom | But similar level of involvement drop-off | 16:45 |
| elodilles | a bit more patch but without reviews, yes :/ | 16:46 |
| gibi | so what if we say, func infra backport are OK to stable/train as there are maintainers there but not further backwards | 16:46 |
| gibi | due to lack of maintainers | 16:46 |
| elodilles | gibi: that is good for RH but not really helps to encourage backporting for older branches | 16:47 |
| sean-k-mooney | well even train is in em now right | 16:47 |
| artom | I think elodilles's point is that if we rewrite the fixes to not need func test refactors *before* train, it helps maintainers of older branches, such as they are | 16:47 |
| sean-k-mooney | given the peopel we have its hard to keep maintianing older branches | 16:47 |
| keerthi | can some one help on review this https://blueprints.launchpad.net/nova/+spec/define-max-volume-limit-at-flavor ? | 16:47 |
| sean-k-mooney | train is 5 releases old currently | 16:47 |
| artom | If we backport func test refactors, we're offloading that rewriting work onto whoever is still working with rocky and older | 16:48 |
| gibi | keerthi: we are in a meeting right now | 16:48 |
| artom | OTOH, why would it be wrong of facilitating the work of the majority? | 16:49 |
| gibi | so we say no func test backport as it there is no maintainers but also we say if we dont backport func infra then we dont have maintaniers, this is contradiction now | 16:49 |
| sean-k-mooney | keerthi: we can proably discuss it after the meeting or when we are done with this topic | 16:49 |
| keerthi | sure Sean, I will wait for it | 16:49 |
| elodilles | anyway, i have said my preference, and i'm only one of the stable maintainers o:) | 16:50 |
| gibi | either we don't have stein maintainers and then I don't feel back about not helping them, or we have maintainers and the I would ask them to backport the func refactor from train to stein | 16:50 |
| gibi | s/back/bad/ | 16:51 |
| gibi | and I would help them by backporting the refactor up until train | 16:51 |
| gibi | that would be a win-win | 16:51 |
| artom | gibi, hard to argue with that | 16:52 |
| gibi | elodilles: would you be -1 on a func infra backport to stable/train? | 16:52 |
| gibi | (or even -2?) | 16:52 |
| elodilles | 'func infra'? | 16:53 |
| gibi | functional test infrastructure backport | 16:53 |
| artom | As opposed to infrastructure that's really grooooovey :D | 16:53 |
| gibi | like the one artom linked above | 16:53 |
| elodilles | so you mean zuul jobs change? | 16:53 |
| elodilles | (i've lost in the links :)) | 16:54 |
| artom | No, stuff like https://review.opendev.org/c/openstack/nova/+/791481/1 | 16:54 |
| artom | For example | 16:54 |
| gibi | elodilles: changes that are not adding a bugfix but refactoring the functional test infrastrucutre | 16:54 |
| gibi | like backporting helpers from newer branches | 16:54 |
| artom | I mean, if https://review.opendev.org/c/openstack/nova/+/751364 is allowed... | 16:54 |
| * bauzas waves super super late | 16:54 | |
| bauzas | sorry, it took longer than expected :( | 16:54 |
| artom | And https://review.opendev.org/c/openstack/nova/+/751363/3 | 16:55 |
| artom | Honestly, with those last 2, we've set the precedent that func test refactors are fair game | 16:55 |
| sean-k-mooney | artom: wasnt the main issue with the backport you did that you skipped ussui by the way | 16:55 |
| artom | sean-k-mooney, that's a different series, and I fixed that | 16:56 |
| sean-k-mooney | oh ok | 16:56 |
| sean-k-mooney | nevermind then | 16:56 |
| gibi | we are running out of time and still had on point on the agenda | 16:56 |
| elodilles | artom: we did backport simple&small func test refactors in the past | 16:56 |
| elodilles | gibi: if we are waiting for my vote: as i said, i'd prefer not, but not against :) | 16:57 |
| elodilles | so no -2 from me | 16:57 |
| gibi | OK, so if you are not blocking such backport then we are done | 16:57 |
| gibi | artom: anything else? | 16:58 |
| bauzas | so, MHO is that we should be picky if needed | 16:58 |
| elodilles | bauzas: +2 | 16:58 |
| bauzas | anyway, time flies | 16:58 |
| gibi | sure still proper review on those patches but not blanket -2 | 16:58 |
| bauzas | my other point can be deferred to next week | 16:59 |
| artom | gibi, nope, I'm coming away from this with the understanding that func test refactors are fair game if they help make the fixes cleaner and easier to backport (and increase confidence in the reproducer test) | 16:59 |
| gibi | ack | 16:59 |
| gibi | moving on then | 16:59 |
| gibi | (bauzas) Opportunities for low-hanging-fruits, anyone ? (only if we have time left) | 16:59 |
| gibi | 40 secs :) | 16:59 |
| bauzas | no time, let's punt to next week | 16:59 |
| gibi | ack | 16:59 |
| gibi | any last minute issue to raise today? | 16:59 |
| bauzas | tl;dr: think about how to help on-off contributors like interns | 16:59 |
| gibi | #endmeeting | 17:00 |
| opendevmeet | Meeting ended Tue Jun 21 17:00:10 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:00 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2022/nova.2022-06-21-15.59.html | 17:00 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2022/nova.2022-06-21-15.59.txt | 17:00 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2022/nova.2022-06-21-15.59.log.html | 17:00 |
| gibi | thanks folks for joining | 17:00 |
| elodilles | thanks gibi o/ | 17:00 |
| bauzas | thanks a lot gibi for running this one, I owe you some coin | 17:00 |
| gibi | keerthi: sorry for the delay | 17:00 |
| gibi | keerthi: so lets try that now | 17:00 |
| keerthi | thanks sean | 17:03 |
| artom | bauzas, hopefully not bitcoin | 17:03 |
| gibi | nah it is cheap now | 17:03 |
| keerthi | sean, this is the blueprint https://blueprints.launchpad.net/nova/+spec/define-max-volume-limit-at-flavor | 17:03 |
| artom | Cheap, or worthless? :) | 17:03 |
| bauzas | artom: just a nickel | 17:03 |
| opendevreview | Merged openstack/nova master: add repoducer test for bug 1890244 https://review.opendev.org/c/openstack/nova/+/847000 | 17:05 |
| keerthi | sean did you get chance to review the blueprint ? | 17:09 |
| opendevreview | Rajesh Tailor proposed openstack/nova master: Remove unnecessary if condition https://review.opendev.org/c/openstack/nova/+/844418 | 17:12 |
| opendevreview | sean mooney proposed openstack/nova master: ignore deleted server groups in validation https://review.opendev.org/c/openstack/nova/+/847001 | 18:14 |
| *** dasm is now known as dasm|off | 21:38 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!