| *** dasm|off is now known as Guest188 | 05:30 | |
| bauzas | gmann: thanks for having incepted the placement LP resource https://launchpad.net/placement | 09:46 |
|---|---|---|
| bauzas | gmann: since I don't know exactly what's left, I just set the maintainer back to you, sure | 09:46 |
| bauzas | gmann: ping me once you're done | 09:46 |
| darkhorse | Hi team, In the compute_nodes table, there are 'host' column and 'hypervisor_hostname' column, which looks the same. Will you explain the difference between these fields? | 10:14 |
| gibi | bauzas: so https://review.opendev.org/c/openstack/tempest/+/842240 is not green yet, but gmann pushed a test patch for nova-lvm with it https://review.opendev.org/c/openstack/nova/+/865765 it is also red but with one less failed test case (test_attach_detach_volume is green there now) | 10:24 |
| bauzas | ack | 10:25 |
| bauzas | darkhorse: 'host' is about the name of the nova-compute service | 10:26 |
| bauzas | which is opted in by the conf option | 10:26 |
| bauzas | darkhorse: while "hypervisor_hostname" is provided by the virt driver | 10:27 |
| darkhorse | bauzas: thank you for clarification! | 10:30 |
| bauzas | np | 10:30 |
| bauzas | gibi: sean-k-mooney: could you please review my spec https://review.opendev.org/c/openstack/nova-specs/+/861591 before 3pm UTC today ? :) | 10:32 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: Skip live_migration_with_trunk tempest test on ovs-hybrid and nova-next https://review.opendev.org/c/openstack/nova/+/865658 | 10:34 |
| bauzas | gibi: sean-k-mooney: gmann: ^ fwiw, added depends-on gmann's tempest patch on ^ to eventually get it accepted | 10:35 |
| * bauzas feels like he's playing a limbo dance | 10:36 | |
| gibi | bauzas: lets wait for gmann to get up and discuss what are our options about the gate regarding the detach issue. If gmann sees a possible fix then I would wait for it, but if he doesn't then I cannot offer any better way forward now than disabling the tests given the amount of time I can spend on this. | 10:42 |
| bauzas | gibi: I need to catch my daughter at school but let's revisit this once gmann's up | 10:43 |
| gibi | yepp | 10:43 |
| gibi | bauzas: I'm +2 on https://review.opendev.org/c/openstack/nova-specs/+/861591 | 10:43 |
| * sean-k-mooney is tempted to wait till 14:59 utc | 10:46 | |
| opendevreview | Kirill proposed openstack/nova-specs master: new spec: support of vnc console for ironic https://review.opendev.org/c/openstack/nova-specs/+/863773 | 10:49 |
| bauzas | gibi: I very briefly looked at the failing tests and yeah, we need to continue working on it if we want to merge it | 11:11 |
| bauzas | I can try to help but I don't have a lot of context about it | 11:11 |
| bauzas | also, I wonder *why* we now have a larger number of issues with nova-lvm | 11:12 |
| bauzas | anyway, /me goes afk for lunch | 11:12 |
| kashyap | bauzas: I think not many (if any at all) production users use LVM | 11:13 |
| kashyap | (With Nova, I mean) | 11:14 |
| sean-k-mooney | it used to be popular for thost that need the better write prefromance | 11:14 |
| sean-k-mooney | but its much less common then it used to be | 11:15 |
| kashyap | Yeah | 11:15 |
| sean-k-mooney | i previously used it for my home cloud in the past. i never really had issues with it. im currently using cinder lvm with bfv instead | 11:17 |
| gibi | bauzas: when we change dependency version, ie changing to the next ubuntu image, the timings change and that teases out new races | 11:19 |
| gibi | it think the jammy change triggered different timings in our tests showing more this issue | 11:19 |
| opendevreview | Merged openstack/nova-specs master: Proposes cpu power managment in libvirt https://review.opendev.org/c/openstack/nova-specs/+/861591 | 11:27 |
| opendevreview | Merged openstack/nova-specs master: Re-propose per process healthchecks https://review.opendev.org/c/openstack/nova-specs/+/864150 | 12:38 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/ussuri: Adds a repoducer for post live migration fail https://review.opendev.org/c/openstack/nova/+/864006 | 12:53 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/ussuri: [compute] always set instance.host in post_livemigration https://review.opendev.org/c/openstack/nova/+/864007 | 12:53 |
| bauzas | fwiw, I got f***d by Zuul b/c of nova-lvm again even if Depends-on on the Tempest patch https://review.opendev.org/c/openstack/nova/+/865658 | 13:06 |
| sean-k-mooney | well the tempest change was still failing | 13:07 |
| bauzas | yup | 13:07 |
| bauzas | but the issue is still present | 13:07 |
| sean-k-mooney | right becaues qemu is broken | 13:07 |
| bauzas | other jobs are failing to due the flakey tempest test I agree | 13:07 |
| sean-k-mooney | so we can either disabel the volume tests or we can make it non voting but we cant really do anythign else in onva | 13:08 |
| bauzas | but for nova-lvm, we get the same exception than previously without depending on the tempest changhe | 13:08 |
| sean-k-mooney | the nova-lvm job is intended to test the lvm image backend | 13:08 |
| bauzas | yeah I know | 13:08 |
| sean-k-mooney | so we dont striclty need to test cinder there | 13:08 |
| bauzas | I don't disagree | 13:09 |
| bauzas | but the failure is also for the rescue tests | 13:10 |
| bauzas | in tearDownClass fixture ending | 13:10 |
| gokhani | hello folks, some of my instances image backing file is deleted unexpectedly. And unfortunately glance image is also deleted. I only see a disk file under /var/lib/nova/instances. is there any way rescue this instances ? I tried using guestfish but it also tries to reach backing file. | 13:11 |
| gibi | bauzas, sean-k-mooney: lets sync with gmann, but sadly I have to agree to disable those tests temprorarily | 13:12 |
| bauzas | gibi: tbh, I don't know which exact cables to unplug | 13:12 |
| bauzas | I mean, I know the cables | 13:13 |
| bauzas | but I wonder if by unplugging them, it would either shutdown the Big Thing or even just Do Nothing | 13:13 |
| sean-k-mooney | you woudl have to disabel all volume related tests more or less | 13:14 |
| gibi | sean-k-mooney: no we don't have to, I think some of those has the wait for sshable thing implemented | 13:14 |
| sean-k-mooney | there is one other tempest serise that might be related | 13:14 |
| sean-k-mooney | gibi: it failed with that | 13:15 |
| sean-k-mooney | i mean the current patch might not work | 13:15 |
| sean-k-mooney | and making it work might be enough | 13:15 |
| gibi | sean-k-mooney: it failed with a tempest patch that adds some sshable flag to some tests. But we already merged a lot of sshable flag patches | 13:15 |
| sean-k-mooney | but the current patch still failed | 13:15 |
| * bauzas is about to write a 'hold your rechecks' email | 13:15 | |
| sean-k-mooney | oh i know but bauzas added a depend-on ot the last one in that seriese | 13:16 |
| bauzas | yup | 13:16 |
| sean-k-mooney | so we seam to need somethign else on top of that | 13:16 |
| gibi | yes, taht depends on was for adding sshable for hopefully the last set of volume tests | 13:16 |
| bauzas | yup | 13:16 |
| gibi | that tempest test is not corret yet | 13:16 |
| gibi | but gmann hopefully knows more about it | 13:16 |
| * bauzas needs a coffee | 13:17 | |
| bauzas | I'm just wrapping up all the LP bug reports | 13:17 |
| gibi | so I think we need to disable those volume tests that are failing currently, not all volume tests | 13:17 |
| sean-k-mooney | https://review.opendev.org/c/openstack/tempest/+/840410 | 13:17 |
| sean-k-mooney | this one might also be interesting ^ | 13:17 |
| gibi | yes, we need that too to disable trunk | 13:17 |
| gibi | there we know ralonsoh working on a fix | 13:17 |
| sean-k-mooney | https://review.opendev.org/c/openstack/tempest/+/840410 fixes a issue witht he cleanup function ordering | 13:18 |
| sean-k-mooney | related to volumes | 13:18 |
| gibi | sorry i mixed up things | 13:18 |
| gibi | https://review.opendev.org/c/openstack/tempest/+/840410 is new to me | 13:19 |
| sean-k-mooney | bauzas: could you perhaps also add a depend on for that | 13:19 |
| gibi | probably we need a single nova patch that disables trunk (as in https://review.opendev.org/c/openstack/nova/+/865658) and disable the failing volume tests | 13:19 |
| sean-k-mooney | ya that would be simpler to merge | 13:20 |
| bauzas | sean-k-mooney: sec, writing an email to inform the world we're busted | 13:20 |
| bauzas | I'm here | 13:28 |
| bauzas | so | 13:28 |
| sean-k-mooney | joing an internal call shortly but https://review.opendev.org/c/openstack/tempest/+/840410 may help with the volume issues | 13:29 |
| sean-k-mooney | if you can add that as a second depends on it might help | 13:29 |
| bauzas | sean-k-mooney: I can try | 13:29 |
| bauzas | that's gonna be fun | 13:29 |
| sean-k-mooney | alternitivly we can tempoerly disable the volume tests in the same patch that is skiping the tunk tests | 13:29 |
| sean-k-mooney | and then try that in the followup that turns them back on | 13:30 |
| bauzas | https://lists.openstack.org/pipermail/openstack-discuss/2022-November/031335.html | 13:30 |
| bauzas | ralonsoh: you also have a different tracking bug number for the wrong port state | 13:30 |
| bauzas | ralonsoh: maybe we should merge those 2 bug reports we have | 13:30 |
| bauzas | sean-k-mooney: editing inline in gerrit, so we'll see | 13:31 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: Skip live_migration_with_trunk tempest test on ovs-hybrid and nova-next https://review.opendev.org/c/openstack/nova/+/865658 | 13:32 |
| ralonsoh | bauzas, yes, one sec | 13:39 |
| ralonsoh | bauzas, https://bugs.launchpad.net/neutron/+bug/1997025 | 13:40 |
| ralonsoh | patch: https://review.opendev.org/c/openstack/neutron/+/837780 | 13:40 |
| ralonsoh | I've increased the priority of th ereview | 13:41 |
| bauzas | ralonsoh: would you disagree if I'm closing https://bugs.launchpad.net/neutron/+bug/1997025 as duplicate of https://bugs.launchpad.net/neutron/+bug/1940425 which was already triaged for Neutron ? | 13:41 |
| ralonsoh | bauzas, that issue was something different | 13:42 |
| ralonsoh | in https://bugs.launchpad.net/neutron/+bug/1997025, the port is not activated | 13:42 |
| ralonsoh | and this is because of the os-vif release | 13:42 |
| bauzas | Hmmm, OK | 13:43 |
| ralonsoh | --> https://bugs.launchpad.net/neutron/+bug/1997025/comments/1 | 13:43 |
| *** akekane is now known as abhishekk | 13:57 | |
| *** Guest188 is now known as dasm | 13:58 | |
| opendevreview | Sylvain Bauza proposed openstack/nova master: Skip live_migration_with_trunk tempest test on ovs-hybrid and nova-next https://review.opendev.org/c/openstack/nova/+/865658 | 14:46 |
| bauzas | yet another try ^ | 14:47 |
| bauzas | the previous run was failing | 14:47 |
| bauzas | sean-k-mooney: gibi: so, I'm about to skip the nova-lvm tests that fail | 17:13 |
| bauzas | fwiw, nova-grenade-multinode also hits the port not active isue https://zuul.opendev.org/t/openstack/build/16cc990d758348999cc5c1b2b30aa9a2 | 17:13 |
| gibi | bauzas: ack | 17:15 |
| ralonsoh | bauzas, https://review.opendev.org/c/openstack/neutron/+/837780 | 17:18 |
| ralonsoh | we have merged the patch in Neutron | 17:19 |
| ralonsoh | I think you should be OK now | 17:19 |
| ralonsoh | (I hope so) | 17:19 |
| bauzas | ah, nice to hear | 17:19 |
| bauzas | I can try to recheck my original patch | 17:19 |
| bauzas | ralonsoh: kudos to the neutron team | 17:19 |
| ralonsoh | yw | 17:20 |
| sean-k-mooney | ralonsoh ah an ye removed the config option great | 17:25 |
| sean-k-mooney | then it should just work | 17:25 |
| sean-k-mooney | without us needing to modify jobs | 17:25 |
| ralonsoh | yes referring to the Nova patch | 17:25 |
| bauzas | ralonsoh: sean-k-mooney: I just rechecked https://review.opendev.org/c/openstack/nova/+/838976/8 | 17:26 |
| bauzas | we will see | 17:26 |
| ralonsoh | bauzas, I'll keep an eye on it | 17:26 |
| sean-k-mooney | bauzas: we still need to skip the lvm tests | 17:26 |
| sean-k-mooney | right | 17:26 |
| bauzas | either way, it will fail due to the nova-lvm issue but we'll see whether ovn and nova-next jobs work | 17:26 |
| bauzas | sean-k-mooney: yup | 17:26 |
| bauzas | this is just for checking against something which is not my skiptest patch | 17:27 |
| sean-k-mooney | ya ok well it might pass put its unlikely to do it twice to get passed both gate and check | 17:27 |
| bauzas | yup | 17:27 |
| bauzas | nova-lvm seems to fail 100% of times | 17:27 |
| sean-k-mooney | is nova-lvm using pc or q35 by the way i assume pc | 17:29 |
| sean-k-mooney | the ohter thing we could try is putting it back to focal temporilly | 17:30 |
| sean-k-mooney | thats more invasive however then skiping the flaky tests | 17:30 |
| sean-k-mooney | not set in the nova.conf so it should be pc | 17:31 |
| sean-k-mooney | https://zuul.opendev.org/t/openstack/build/f076244738904c949e63b21571a5fc9a/log/controller/logs/etc/nova/nova-cpu_conf.txt#118 | 17:31 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/train: Adds a repoducer for post live migration fail https://review.opendev.org/c/openstack/nova/+/863806 | 17:46 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/train: [compute] always set instance.host in post_livemigration https://review.opendev.org/c/openstack/nova/+/864055 | 17:46 |
| gmann | gibi: bauzas sean-k-mooney: hi, for detach issue in nova-lvm job. tempest patch need more time it seems. I was making server sshable by default which need more work in other tests. | 17:57 |
| gmann | gibi: sean-k-mooney bauzas: I am moving my house this week so might need some more time on that. For now, I think we can go with skipping the failing tests temporarily in nova-lvm | 17:58 |
| bauzas | ack | 17:59 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/ussuri: [compute] always set instance.host in post_livemigration https://review.opendev.org/c/openstack/nova/+/864007 | 17:59 |
| gibi | gmann: thanks for looking into it | 17:59 |
| sean-k-mooney | so i think we shoudl be ok to skip them for a few weeks to unblock things | 18:00 |
| gibi | yepp I agree | 18:01 |
| sean-k-mooney | the cidner code paths really shoudl not depend on the images_backend that is in use at least not lvm | 18:01 |
| gibi | sady we have no better option now | 18:01 |
| gmann | but thing is those failing rescue test, all use ssh-able server. there is something else might be causing the failure | 18:01 |
| sean-k-mooney | gmann: yep there likely is | 18:01 |
| gmann | something to do on Jammy as lvm on focal working fine | 18:01 |
| sean-k-mooney | possibly | 18:02 |
| sean-k-mooney | it defiently happens more on jammy im not sure it would not happen on focal | 18:02 |
| sean-k-mooney | it might be related to a default config delta between the two or the qemu/libvirt verions | 18:02 |
| gmann | may be | 18:02 |
| sean-k-mooney | but we dont currently have a lot of capsity to investigate | 18:02 |
| gmann | yeah, do we have patch up to skip the tests? | 18:03 |
| sean-k-mooney | i think bauzas has one but im not sure they updated for the volume tests | 18:04 |
| gmann | k | 18:04 |
| gmann | this one? https://review.opendev.org/c/openstack/nova/+/865658 | 18:04 |
| sean-k-mooney | that was to solve the other issue so that is not needed | 18:05 |
| gmann | this does not include nova-lvm failing tests | 18:05 |
| gmann | k | 18:05 |
| sean-k-mooney | we shoudl pull that form the gate | 18:05 |
| sean-k-mooney | gmann: i do not see one form sylvain | 18:06 |
| sean-k-mooney | so i guess there is non e right now for the lvm failures | 18:06 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/train: [compute] always set instance.host in post_livemigration https://review.opendev.org/c/openstack/nova/+/864055 | 18:11 |
| *** dasm is now known as dasm|off | 22:01 | |
| opendevreview | Ghanshyam proposed openstack/nova master: Temporary skip some volume detach test in nova-lvm job https://review.opendev.org/c/openstack/nova/+/865922 | 23:59 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!