| *** evrardjp has quit IRC | 00:05 | |
| *** tosky has quit IRC | 00:06 | |
| *** ociuhandu has joined #openstack-nova | 00:11 | |
| *** evrardjp has joined #openstack-nova | 00:15 | |
| *** ociuhandu has quit IRC | 00:23 | |
| *** zzzeek has quit IRC | 00:28 | |
| *** zzzeek has joined #openstack-nova | 00:32 | |
| -openstackstatus- NOTICE: Our Gerrit upgrade maintenance has concluded successfully; please see the maintenance wrap-up announcement for additional details: http://lists.opendev.org/pipermail/service-announce/2020-November/000014.html | 00:54 | |
| *** hamalq has joined #openstack-nova | 01:32 | |
| *** hamalq has quit IRC | 01:37 | |
| *** LinPeiWen46 has quit IRC | 01:40 | |
| *** rcernin has quit IRC | 01:57 | |
| *** rcernin has joined #openstack-nova | 01:58 | |
| *** rcernin has quit IRC | 02:05 | |
| *** rcernin has joined #openstack-nova | 02:05 | |
| *** xinranwang has joined #openstack-nova | 02:23 | |
| *** sapd1 has joined #openstack-nova | 02:37 | |
| *** zzzeek has quit IRC | 02:53 | |
| *** zzzeek has joined #openstack-nova | 02:54 | |
| *** LinPeiWen97 has joined #openstack-nova | 03:11 | |
| *** dave-mccowan has quit IRC | 03:17 | |
| *** hamalq has joined #openstack-nova | 03:33 | |
| *** psachin has joined #openstack-nova | 03:33 | |
| *** hamalq has quit IRC | 03:37 | |
| *** rcernin has quit IRC | 03:47 | |
| *** ociuhandu has joined #openstack-nova | 03:47 | |
| *** rcernin has joined #openstack-nova | 03:48 | |
| *** ociuhandu has quit IRC | 03:52 | |
| *** openstackgerrit has joined #openstack-nova | 03:53 | |
| openstackgerrit | chengsheng proposed openstack/nova master: CPU selection with hypervisor consideration https://review.opendev.org/c/openstack/nova/+/762330 | 03:53 |
|---|---|---|
| *** mkrai has joined #openstack-nova | 04:05 | |
| *** ratailor has joined #openstack-nova | 04:11 | |
| *** ociuhandu has joined #openstack-nova | 05:04 | |
| *** ociuhandu has quit IRC | 05:08 | |
| *** sapd1 has quit IRC | 05:26 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-nova | 05:33 | |
| *** lemko has quit IRC | 05:42 | |
| *** lemko has joined #openstack-nova | 05:42 | |
| *** sapd1 has joined #openstack-nova | 06:08 | |
| openstackgerrit | YumengBao proposed openstack/nova-specs master: libvirt supports composing cyborg owned vGPU accelerator into domain XML https://review.opendev.org/c/openstack/nova-specs/+/750116 | 06:19 |
| *** danpawlik has quit IRC | 06:24 | |
| *** danpawlik has joined #openstack-nova | 06:25 | |
| *** mkrai has quit IRC | 06:45 | |
| *** whoami-rajat__ has joined #openstack-nova | 06:46 | |
| *** sapd1 has quit IRC | 06:47 | |
| *** ociuhandu has joined #openstack-nova | 06:52 | |
| *** mkrai has joined #openstack-nova | 06:54 | |
| *** ociuhandu has quit IRC | 06:56 | |
| *** iurygregory has joined #openstack-nova | 07:06 | |
| *** lpetrut has joined #openstack-nova | 07:07 | |
| *** ociuhandu has joined #openstack-nova | 07:14 | |
| *** ociuhandu has quit IRC | 07:19 | |
| *** rcernin has quit IRC | 07:24 | |
| *** rpittau|afk is now known as rpittau | 07:29 | |
| *** ociuhandu has joined #openstack-nova | 07:31 | |
| *** ociuhandu has quit IRC | 07:39 | |
| *** ralonsoh has joined #openstack-nova | 07:39 | |
| *** rcernin has joined #openstack-nova | 07:50 | |
| *** tesseract has joined #openstack-nova | 08:01 | |
| *** slaweq has joined #openstack-nova | 08:02 | |
| *** ircuser-1 has quit IRC | 08:02 | |
| *** rcernin has quit IRC | 08:08 | |
| gibi | good morning Nova | 08:15 |
| gibi | new gerrit... interesting | 08:15 |
| *** andrewbonney has joined #openstack-nova | 08:23 | |
| *** luksky has joined #openstack-nova | 08:24 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support sys.argv in wsgi app https://review.opendev.org/c/openstack/nova/+/763750 | 08:33 |
| gibi | and gerrit works :) | 08:33 |
| *** tosky has joined #openstack-nova | 08:38 | |
| *** rcernin has joined #openstack-nova | 08:43 | |
| *** derekh has joined #openstack-nova | 08:57 | |
| *** mgoddard has joined #openstack-nova | 08:58 | |
| *** ociuhandu has joined #openstack-nova | 09:01 | |
| *** tinwood_ is now known as tinwood | 09:08 | |
| *** ociuhandu has quit IRC | 09:20 | |
| *** rcernin has quit IRC | 09:23 | |
| *** hamalq has joined #openstack-nova | 09:36 | |
| openstackgerrit | Lucian Petrut proposed openstack/nova master: hyper-v rbd volume support https://review.opendev.org/c/openstack/nova/+/763550 | 09:37 |
| *** hamalq has quit IRC | 09:40 | |
| lpetrut | gibi: I had to double check to make sure I haven't opened android-review.googlesource.com :) | 09:40 |
| gibi | lpetrut: :) | 09:41 |
| gibi | please note that not everything works https://etherpad.opendev.org/p/gerrit-3.2-post-upgrade-notes | 09:41 |
| gibi | the biggest missing piece for me is the launchpad integration. You have to manually set bug status | 09:42 |
| *** artom has joined #openstack-nova | 09:43 | |
| lpetrut | interesting, good to know. looks like there's quite some work going on in order to bring back those features | 09:43 |
| *** mkrai has quit IRC | 09:47 | |
| *** dtantsur|afk is now known as dtantsur | 09:50 | |
| *** ociuhandu has joined #openstack-nova | 09:51 | |
| *** mkrai has joined #openstack-nova | 09:52 | |
| *** k_mouza has joined #openstack-nova | 09:59 | |
| *** ociuhandu has quit IRC | 09:59 | |
| *** jangutter has quit IRC | 10:02 | |
| *** jangutter has joined #openstack-nova | 10:02 | |
| *** ociuhandu has joined #openstack-nova | 10:15 | |
| lyarwood | bauzas: https://review.opendev.org/c/openstack/nova/+/752486 - would you mind hitting this and the regression test below it? | 10:21 |
| *** ociuhandu has quit IRC | 10:22 | |
| kashyap | Oh, shiny new Gerrit | 10:25 |
| * kashyap likes the commit message being front-and-center | 10:26 | |
| *** mkrai has quit IRC | 10:28 | |
| *** sapd1 has joined #openstack-nova | 10:28 | |
| *** ociuhandu has joined #openstack-nova | 10:34 | |
| *** masterpe has quit IRC | 10:34 | |
| *** xek has joined #openstack-nova | 10:39 | |
| *** iurygregory has quit IRC | 10:39 | |
| stephenfin | why does everything need to be in bold? | 10:41 |
| *** ociuhandu has quit IRC | 10:42 | |
| *** masterpe has joined #openstack-nova | 10:42 | |
| bauzas | lyarwood: ack, sorry I was taxi for my kids | 10:42 |
| * bauzas has his eyes crying for the new Gerrit | 10:43 | |
| bauzas | how can I see in which releases was merged https://review.opendev.org/c/openstack/nova/+/752247 ? | 10:49 |
| bauzas | in the previous Gerrit version, we had a 'included in' value | 10:49 |
| *** mkrai has joined #openstack-nova | 10:50 | |
| gibi | bauzas: good question | 10:50 |
| gibi | I havn't found it | 10:50 |
| stephenfin | here's a bit of fun - right click on an element in a path list patch and "inspect" it | 10:51 |
| stephenfin | takes at least 10 seconds for Firefox to catch up on my machine :-D | 10:51 |
| gibi | bauzas: above the file list there is an included-in link | 10:51 |
| *** ralonsoh has quit IRC | 10:51 | |
| bauzas | gibi: haha, thanks ! | 10:52 |
| bauzas | indeed, that helps me | 10:52 |
| bauzas | gibi: <3 | 10:52 |
| bauzas | so, this change was merged during the victoria release, so we don't need to have a victoria backport too | 10:52 |
| bauzas | that's what I wanted to kew | 10:53 |
| bauzas | knew | 10:53 |
| *** xinranwang has quit IRC | 10:53 | |
| *** ralonsoh has joined #openstack-nova | 10:53 | |
| * bauzas needs time for using the new Gerrit UI | 10:53 | |
| *** sapd1 has quit IRC | 11:11 | |
| openstackgerrit | Merged openstack/nova stable/ussuri: Add regression test for bug #1895696 https://review.opendev.org/c/openstack/nova/+/752485 | 11:16 |
| openstack | bug 1895696 in Cinder "unable to boot instance from encrypted volume created from a glance image of an encrypted volume" [Undecided,New] https://launchpad.net/bugs/1895696 | 11:16 |
| *** derekh has quit IRC | 11:16 | |
| gibi | the new gerrit supports emojis | 11:24 |
| gibi | I'm not fan of emojis | 11:24 |
| gibi | and gerrit makes suggestion how to make good review comments like: "When disagreeing, explain the advantage of your approach" | 11:25 |
| gibi | I hope I can trun them off after I saw all of them | 11:26 |
| * lyarwood consoles stephenfin | 11:32 | |
| lyarwood | I think he was hoping to be able to review *only* using emojis | 11:32 |
| gibi | :D | 11:32 |
| stephenfin | ❤️❤️❤️ | 11:32 |
| gibi | I like the text base emojis | 11:33 |
| *** ratailor has quit IRC | 11:33 | |
| gibi | like ':D' | 11:33 |
| gibi | maybe I'm oldscool | 11:33 |
| stephenfin | gibi: I think you mean 😃 | 11:33 |
| stephenfin | Ideally, you'd be able to toggle between the emoji and their textual representation | 11:34 |
| stephenfin | shortcode, rather | 11:34 |
| stephenfin | but those shortcodes are platform specific :-( | 11:34 |
| stephenfin | e.g. :smiley: | 11:34 |
| gibi | stephenfin: https://pasteboard.co/JBGqpj6.png :D | 11:35 |
| gibi | stephenfin: yeah, I like how hangouts can be forced to translate the emojis back | 11:36 |
| gibi | (at least my emojois not the incoming ones) | 11:36 |
| *** hamalq has joined #openstack-nova | 11:36 | |
| lyarwood | is someone going to try to rework the gerrit ui to look something like the previous ui we had btw? | 11:40 |
| * lyarwood misses the CI table etc | 11:40 | |
| *** raildo has joined #openstack-nova | 11:41 | |
| lyarwood | and the quick query/dashboard list | 11:41 |
| *** hamalq has quit IRC | 11:41 | |
| stephenfin | lyarwood: They're working to restore the CI table, but it requires a new plugin so I don't know how long that will take | 11:42 |
| gibi | lyarwood: I might look into the CI table thing at some point as a user script | 11:42 |
| gibi | lyarwood: what is quick query? | 11:42 |
| stephenfin | I assume he means the access to dashboards that didn't require a dropdown | 11:42 |
| lyarwood | yeah the list of editable queries at the top of the page | 11:43 |
| lyarwood | I had a few dashboard listed etc, but now it's behind a dropdown | 11:43 |
| stephenfin | Dashboards are now hidden in the "Your" dropdown. One more click to get those frequently used link | 11:43 |
| lyarwood | yeah | 11:43 |
| gibi | ahh I see | 11:44 |
| *** derekh has joined #openstack-nova | 11:44 | |
| lyarwood | gibi: https://review.opendev.org/c/openstack/nova/+/760971 - would you mind taking a look at this today if you have time, just a simple docs change. | 11:48 |
| gibi | looking | 11:48 |
| lyarwood | many thanks | 11:49 |
| gibi | lyarwood: +A | 11:53 |
| *** mkrai has quit IRC | 11:58 | |
| *** iurygregory has joined #openstack-nova | 11:58 | |
| gibi | dansmith: when you are up, I pushed a change to https://github.com/kk7ds/openstack-gerrit-dashboard/pull/30 to adapt it to the new gerrit | 12:04 |
| sean-k-mooney | gibi: the old gerrit supported emojis | 12:05 |
| sean-k-mooney | gibi: stephenfin used the poop emoji in his hypervisors api spec | 12:06 |
| gibi | sean-k-mooney: I hope nobody will start using them more frequently than before, but I'm affraid that having a context popup for emojis will not help | 12:06 |
| stephenfin | sean-k-mooney: https://that.guru/blog/plain-text-in-evolution-3-38/ | 12:08 |
| stephenfin | just FYI, since I think you use Evolution | 12:08 |
| *** ociuhandu has joined #openstack-nova | 12:08 | |
| sean-k-mooney | i have plain text enabled | 12:08 |
| sean-k-mooney | not sure if its the same | 12:08 |
| sean-k-mooney | i need to change the line wrapping to something more resonable then 80 though | 12:09 |
| stephenfin | So did I | 12:09 |
| stephenfin | *do I | 12:09 |
| stephenfin | but it changed behavior in 3.38 - see blog :) | 12:09 |
| *** derekh has quit IRC | 12:13 | |
| *** ociuhandu has quit IRC | 12:13 | |
| sean-k-mooney | stephenfin: why do you use quoted for forwarding | 12:14 |
| stephenfin | I frequently include context comments inline, so it makes more sense for me | 12:14 |
| stephenfin | Forwarding mails without context is a huge faux pas in my books | 12:15 |
| sean-k-mooney | if im dong that i do it as a reply | 12:15 |
| sean-k-mooney | if i forward i add it as an attachment and i leave a new message in the body | 12:15 |
| stephenfin | only if I'm keeping the original people in the loop | 12:15 |
| stephenfin | it's easy to change when necessary too | 12:16 |
| sean-k-mooney | oh i reply and drop the original list | 12:16 |
| sean-k-mooney | same effect | 12:16 |
| sean-k-mooney | so you need to now untick wrap quoted text in replies | 12:16 |
| stephenfin | pretty much, yeah | 12:17 |
| stephenfin | yup | 12:17 |
| sean-k-mooney | the code example is a good example where taht would matter | 12:17 |
| sean-k-mooney | by the way i notice very little differentce between those too | 12:19 |
| sean-k-mooney | infact it looks identical | 12:19 |
| sean-k-mooney | that was your intent? | 12:19 |
| stephenfin | look at the bottom three lines | 12:20 |
| sean-k-mooney | i set my line wrap in emails to 150 so i dont think my line wrapping really kicks in often | 12:20 |
| sean-k-mooney | oh | 12:20 |
| sean-k-mooney | ya | 12:20 |
| stephenfin | ah, I want my own replies to be wrapped at 80 | 12:20 |
| stephenfin | but I don't want to wrap others | 12:20 |
| sean-k-mooney | ya i dont it drives me nuts | 12:20 |
| *** ociuhandu has joined #openstack-nova | 12:20 | |
| stephenfin | because it doesn't reflow them properly | 12:20 |
| stephenfin | this definitely affects you less so | 12:21 |
| sean-k-mooney | part of the reason i set it to 150 instead of 120 is to not reflow text | 12:21 |
| sean-k-mooney | *code | 12:21 |
| stephenfin | I just select those rare code blocks and mark them as preformatted | 12:22 |
| *** Luzi has joined #openstack-nova | 12:25 | |
| *** ociuhandu has quit IRC | 12:25 | |
| sean-k-mooney | oh vdpa live migration patches | 12:30 |
| *** ociuhandu has joined #openstack-nova | 12:39 | |
| openstackgerrit | Merged openstack/nova stable/ussuri: compute: Skip cinder_encryption_key_id check when booting from volume https://review.opendev.org/c/openstack/nova/+/752486 | 12:40 |
| *** mgariepy has joined #openstack-nova | 12:40 | |
| *** ociuhandu has quit IRC | 12:44 | |
| openstackgerrit | Merged openstack/nova master: Reproduce bug 1897528 https://review.opendev.org/c/openstack/nova/+/756696 | 12:58 |
| openstack | bug 1897528 in OpenStack Compute (nova) "32bit pci domain number is not supported" [High,In progress] https://launchpad.net/bugs/1897528 - Assigned to Balazs Gibizer (balazs-gibizer) | 12:58 |
| openstackgerrit | Merged openstack/nova master: Add os-volume_attachments reference docs https://review.opendev.org/c/openstack/nova/+/760971 | 12:59 |
| *** happyhemant has joined #openstack-nova | 13:02 | |
| *** tesseract has quit IRC | 13:12 | |
| *** tesseract has joined #openstack-nova | 13:14 | |
| *** hemanth_n has joined #openstack-nova | 13:14 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support sys.argv in wsgi app https://review.opendev.org/c/openstack/nova/+/763750 | 13:17 |
| *** ociuhandu has joined #openstack-nova | 13:18 | |
| *** ociuhandu has quit IRC | 13:18 | |
| *** ociuhandu has joined #openstack-nova | 13:20 | |
| *** ociuhandu has quit IRC | 13:24 | |
| *** Luzi has quit IRC | 13:29 | |
| *** nweinber has joined #openstack-nova | 13:32 | |
| *** ociuhandu has joined #openstack-nova | 13:36 | |
| *** hamalq has joined #openstack-nova | 13:37 | |
| *** hamalq has quit IRC | 13:42 | |
| *** ociuhandu has quit IRC | 13:44 | |
| *** hemanth_n has quit IRC | 13:47 | |
| *** ociuhandu has joined #openstack-nova | 14:10 | |
| *** ociuhandu has quit IRC | 14:20 | |
| gibi | is is a known grenade issue or is it something new? https://zuul.opendev.org/t/openstack/build/d50877ae15db4022b82f4bb1d1d52cea/log/logs/subnode-2/screen-n-cpu.txt?severity=0#13482 | 14:21 |
| gibi | "qemu-system-x86_64: VQ 0 size 0x80 Guest index 0xb8 inconsistent with Host index 0xe0: delta 0xffd8" | 14:21 |
| sean-k-mooney | looking | 14:22 |
| *** dave-mccowan has joined #openstack-nova | 14:22 | |
| *** mkrai has joined #openstack-nova | 14:24 | |
| *** ociuhandu has joined #openstack-nova | 14:24 | |
| *** ociuhandu has quit IRC | 14:24 | |
| *** ociuhandu has joined #openstack-nova | 14:26 | |
| gibi | I only see one hit in logstash and that is the same as I linked above http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22qemu-system-x86_64%3A%20VQ%5C%22%20AND%20message%3A%5C%22inconsistent%20with%20Host%5C%22 | 14:29 |
| gibi | so I guess it is new | 14:29 |
| *** ociuhandu has quit IRC | 14:30 | |
| *** derekh has joined #openstack-nova | 14:31 | |
| sean-k-mooney | ya it looks like a qemu bug | 14:31 |
| sean-k-mooney | so this is a migratoin | 14:32 |
| gibi | it is a live migration | 14:32 |
| sean-k-mooney | a live migration to be precise | 14:32 |
| sean-k-mooney | yep | 14:32 |
| gibi | yepp | 14:32 |
| sean-k-mooney | and the error is statign that the virtqueue size in the guest is different form the that of the destiation host | 14:33 |
| sean-k-mooney | specfically the dest host size quemes are larger | 14:34 |
| sean-k-mooney | this is for the disk too | 14:34 |
| sean-k-mooney | not the nic | 14:34 |
| sean-k-mooney | we have config options which i dislike to set the nic queue size | 14:35 |
| sean-k-mooney | but we dont have any way to set the queues for the virtio block device | 14:35 |
| *** dave-mccowan has quit IRC | 14:35 | |
| gibi | so the two qemu version has different default queue sizes I assume | 14:35 |
| *** ociuhandu has joined #openstack-nova | 14:36 | |
| sean-k-mooney | maybe | 14:36 |
| sean-k-mooney | this is grenade? | 14:36 |
| *** ociuhandu has quit IRC | 14:36 | |
| sean-k-mooney | so we are not changing os right? | 14:36 |
| *** rpittau is now known as rpittau|brb | 14:36 | |
| sean-k-mooney | im not sure if we should be changign the qemu version here | 14:36 |
| sean-k-mooney | we may be | 14:36 |
| *** ociuhandu has joined #openstack-nova | 14:37 | |
| sean-k-mooney | this is something i would expect libvirt to ensure compatiabily for by the way | 14:37 |
| gibi | it is a grenade run. I'm not sure we have different qemus on the nodes in this run | 14:39 |
| gibi | bauzas: could you hit https://review.opendev.org/c/openstack/nova-specs/+/759828 when you have time? | 14:39 |
| bauzas | gibi: will look fersur | 14:39 |
| gibi | bauzas: thanks | 14:39 |
| stephenfin | I was just about to ping bauzas about the same spec :) | 14:40 |
| stephenfin | The new UI says if a patch has +2, but not who left it | 14:41 |
| stephenfin | that used to be a toggle option in Gerrit 2.x | 14:41 |
| * stephenfin wonders if it still exists | 14:41 | |
| *** ociuhandu_ has joined #openstack-nova | 14:41 | |
| *** ociuhandu_ has quit IRC | 14:41 | |
| *** ociuhandu_ has joined #openstack-nova | 14:41 | |
| *** ociuhandu has quit IRC | 14:42 | |
| gibi | sean-k-mooney: the qemu versions are the same on the two compute in that grenade run qemu version: 4.2.1Debian 1:4.2-3ubuntu6.7~cloud0 | 14:42 |
| dansmith | gibi: thanks :) | 14:44 |
| gibi | dansmith: I'm an active user | 14:44 |
| dansmith | gibi: sweet, might just be you and me at this point :) | 14:45 |
| gibi | I will keep it working until it seriously breaks | 14:45 |
| gibi | as it is part of my workflow | 14:46 |
| lyarwood | FWIW I also use it | 14:48 |
| sean-k-mooney | gibi: just asking danpb about it downstream he expecs this to be handeled by the qemu machine type | 14:49 |
| sean-k-mooney | we is -machine pc-i440fx-4.2,accel=tcg,usb=off,dump-guest-core=off | 14:50 |
| sean-k-mooney | gibi: https://bugs.launchpad.net/nova/+bug/1761798 | 14:50 |
| openstack | Launchpad bug 1761798 in OpenStack Compute (nova) "live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"" [Medium,Confirmed] | 14:50 |
| sean-k-mooney | danpb found ^ | 14:50 |
| gibi | nice catch, so we hit this before | 14:52 |
| gibi | I guess it is infrequent bug if we hit it like 3 times in 2 years | 14:52 |
| gibi | there is no qemu version differences in that grenade run | 14:53 |
| sean-k-mooney | ya so it might be a regression | 14:53 |
| sean-k-mooney | in qemu 4.2 | 14:53 |
| gibi | could be | 14:54 |
| sean-k-mooney | we are not seeing the error in the instance log unless im looking at the wrong one | 14:54 |
| gibi | yeah I did not find it either there | 14:58 |
| sean-k-mooney | oh its instance-0000001a | 14:58 |
| sean-k-mooney | not 1b | 14:58 |
| sean-k-mooney | https://zuul.opendev.org/t/openstack/build/d50877ae15db4022b82f4bb1d1d52cea/log/logs/subnode-2/libvirt/qemu/instance-0000001a.txt | 14:59 |
| gibi | not too chatty 'virtio: bogus descriptor or out of resources' | 15:00 |
| sean-k-mooney | ya that is before the migration i think | 15:00 |
| *** rpittau|brb is now known as rpittau | 15:01 | |
| sean-k-mooney | this is the otehr node https://zuul.opendev.org/t/openstack/build/d50877ae15db4022b82f4bb1d1d52cea/log/logs/libvirt/qemu/instance-0000001a.txt | 15:01 |
| gibi | ahh you found it | 15:02 |
| sean-k-mooney | that has the error in the nova.conf | 15:02 |
| sean-k-mooney | yep | 15:02 |
| sean-k-mooney | so it look like we have the inial virtio error before the migration is started | 15:02 |
| openstackgerrit | Merged openstack/nova-specs master: Remove tenant_id https://review.opendev.org/c/openstack/nova-specs/+/737241 | 15:02 |
| gibi | that initial error could be a hint for the qemu devs what went wrong | 15:03 |
| sean-k-mooney | ya so this does still look like a qemu issue | 15:04 |
| sean-k-mooney | well or libvirt but our live config for the disk is basically the bare minium | 15:04 |
| *** xek_ has joined #openstack-nova | 15:04 | |
| *** mkrai has quit IRC | 15:05 | |
| *** mkrai has joined #openstack-nova | 15:06 | |
| gibi | sean-k-mooney: do we just need to re-target the 1761798 to qemu in launchpad or qemu upstream bugs tracked elsewhere? | 15:06 |
| sean-k-mooney | its just | 15:06 |
| sean-k-mooney | Nov 20 14:25:11.510321 ubuntu-bionic-ovh-bhs1-0021872194 nova-compute[9647]: <devices> | 15:07 |
| sean-k-mooney | Nov 20 14:25:11.510321 ubuntu-bionic-ovh-bhs1-0021872194 nova-compute[9647]: <disk type="file" device="disk"> | 15:07 |
| sean-k-mooney | Nov 20 14:25:11.510321 ubuntu-bionic-ovh-bhs1-0021872194 nova-compute[9647]: <driver name="qemu" type="qcow2" cache="none"/> | 15:07 |
| sean-k-mooney | Nov 20 14:25:11.510321 ubuntu-bionic-ovh-bhs1-0021872194 nova-compute[9647]: <source file="/opt/stack/data/nova/instances/2c468d92-4b19-426a-8c25-16b4624c21a4/disk"/> | 15:07 |
| sean-k-mooney | Nov 20 14:25:11.510321 ubuntu-bionic-ovh-bhs1-0021872194 nova-compute[9647]: <target dev="vda" bus="virtio"/> | 15:07 |
| sean-k-mooney | Nov 20 14:25:11.510321 ubuntu-bionic-ovh-bhs1-0021872194 nova-compute[9647]: </disk> | 15:07 |
| *** xek has quit IRC | 15:07 | |
| sean-k-mooney | gibi: i think they use bugzilla but they have a launchpad too i think | 15:07 |
| sean-k-mooney | im not sure fi we need to retarget or just add them | 15:07 |
| sean-k-mooney | this could be a libvirt bug with regards to the options that it generates when invoking qemu | 15:08 |
| sean-k-mooney | its generating | 15:09 |
| sean-k-mooney | -blockdev '{"driver":"file","filename":"/opt/stack/data/nova/instances/2c468d92-4b19-426a-8c25-16b4624c21a4/disk","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ | 15:09 |
| sean-k-mooney | -blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"}' \ | 15:09 |
| sean-k-mooney | -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,id=virtio-disk0,bootindex=1,write-cache=on | 15:09 |
| sean-k-mooney | but i dont really see a clear smoking gun there | 15:09 |
| *** bbobrov has joined #openstack-nova | 15:10 | |
| sean-k-mooney | its adding a bunch of stuff we did not ask for but none of ti looks invalid | 15:10 |
| *** bbobrov has left #openstack-nova | 15:14 | |
| gibi | added qemu project as affected (keep nova as you suggested) | 15:18 |
| sean-k-mooney | gibi: kashyap has ping one of the qemu migration experts for input | 15:20 |
| kashyap | Yeah, Dave Gilbert from upstream QEMU | 15:21 |
| sean-k-mooney | there is an old https://bugzilla.redhat.com/show_bug.cgi?id=1450524 downstream bug | 15:21 |
| openstack | bugzilla.redhat.com bug 1450524 in qemu-kvm-rhev "qemu-kvm: VQ 0 size 0x100 Guest index 0x2010 inconsistent with Host index 0x2171: delta 0xfe9f" [High,Closed: worksforme] - Assigned to jasowang | 15:21 |
| sean-k-mooney | but it was closed as they could not reliably repoduce it | 15:21 |
| kashyap | He's a fixed a similar-sounding issue as above --^ It was also triggerred by OpenStack CI | 15:21 |
| kashyap | s/a fixed/fixed/ | 15:21 |
| sean-k-mooney | gibi: the main delta there being virtio-ballon vs virtio-blk | 15:22 |
| *** elod has quit IRC | 15:27 | |
| *** lpetrut has quit IRC | 15:28 | |
| *** elod has joined #openstack-nova | 15:28 | |
| *** mkrai has quit IRC | 15:34 | |
| *** mkrai has joined #openstack-nova | 15:35 | |
| *** sapd1 has joined #openstack-nova | 15:36 | |
| kashyap | sean-k-mooney: Also see this one: https://bugs.launchpad.net/nova/+bug/1737625 | 15:38 |
| openstack | Launchpad bug 1737625 in OpenStack Compute (nova) "live migration failed with "Failed to load virtio-blk:virtio" (libvirt/qemu)" [Undecided,Confirmed] | 15:38 |
| *** hamalq has joined #openstack-nova | 15:38 | |
| kashyap | Especially my comment#2, FWIW | 15:38 |
| *** dklyle has joined #openstack-nova | 15:39 | |
| *** hamalq has quit IRC | 15:42 | |
| openstackgerrit | Merged openstack/nova master: Fix the instance_uuid for test https://review.opendev.org/c/openstack/nova/+/763171 | 15:45 |
| sean-k-mooney | gibi: by the way this might be of interest to you https://bugzilla.redhat.com/show_bug.cgi?id=1895702 | 15:45 |
| openstack | bugzilla.redhat.com bug 1895702 in openstack-neutron "Nova – VM is created in spite of the fact that used port is missing fixed IP" [High,New] - Assigned to rhos-maint | 15:45 |
| openstackgerrit | Merged openstack/nova master: doc: Update the PTL guide https://review.opendev.org/c/openstack/nova/+/762727 | 15:46 |
| sean-k-mooney | kashyap we might want to combine these or at least cross link them | 15:46 |
| gibi | sean-k-mooney: interesing edge case where nova allows booting without IP | 15:47 |
| kashyap | sean-k-mooney: So ... the ideal person to help debug this would be Stefan (he's on PTO), who is the 'virtio-blk' expert | 15:47 |
| sean-k-mooney | gibi: ya i think its actully a neutron bug but you were asking about could we enable that behavior | 15:47 |
| kashyap | That said, bugs like these, deep in the bowels of QEMU, require a debug QEMU with probes installed in the environment ... | 15:47 |
| gibi | sean-k-mooney: yes, I got asked about it internally | 15:47 |
| kashyap | ... and pray to the JuJu at the bottom of the sea that the bug reproduces | 15:48 |
| sean-k-mooney | kashyap: if it starts blocking the ci then we will need to take more imediate action but until then i think it can wait till they are back form pto | 15:48 |
| kashyap | sean-k-mooney: I highly doubt it can "block"; I just read Dave's explanation on the rare occurrence it can hit | 15:48 |
| kashyap | As the last known occurrence in the CI seems to be 3 years ago | 15:48 |
| kashyap | (But I'll comment further in the bug) | 15:49 |
| gibi | yeah, i have no problem ignoring this qemu issue if it happens once in two years | 15:49 |
| sean-k-mooney | yep same | 15:50 |
| sean-k-mooney | we just know that it can fail this way | 15:50 |
| kashyap | sean-k-mooney: Also, what's happening in the guest? | 15:50 |
| kashyap | Is it just a garden-variety CirrOS instance just idling? | 15:50 |
| sean-k-mooney | its just a migtrion of a cirror guest | 15:50 |
| *** jangutter_ has joined #openstack-nova | 15:50 | |
| sean-k-mooney | yep | 15:50 |
| sean-k-mooney | so beyound the ocational dhcp request or other idel work it should be doing nothing most of the time | 15:51 |
| *** jangutter has quit IRC | 15:54 | |
| *** elod has quit IRC | 15:54 | |
| kashyap | sean-k-mooney: Dave says if it's a migrate-during-a-reboot, it could be a potential problem | 15:54 |
| kashyap | That said ... | 15:54 |
| *** elod has joined #openstack-nova | 15:54 | |
| kashyap | sean-k-mooney: gibi: I learn from Dave that we get that "guest index inconsistent" error when the migrated RAM is inconsistent with the migrated 'virtio' device state. | 15:55 |
| kashyap | And a common case is where a 'virtio' device does an operation after the vCPU is stopped and after RAM has been transmitted. | 15:55 |
| kashyap | (I'll add a comment in the bug from 2018 with notes) | 15:55 |
| gibi | kashyap: thanks | 15:58 |
| *** artom has quit IRC | 16:01 | |
| *** mlavalle has joined #openstack-nova | 16:03 | |
| stephenfin | bauzas: Think you could take a look at this this week? https://review.opendev.org/c/openstack/nova/+/756549 | 16:10 |
| *** artom has joined #openstack-nova | 16:10 | |
| bauzas | stephenfin: sure, will look | 16:10 |
| stephenfin | ta | 16:10 |
| *** mgoddard has quit IRC | 16:12 | |
| openstackgerrit | Merged openstack/nova-specs master: Re-proposes 'Proposal for a safer remote console with password authentication https://review.opendev.org/c/openstack/nova-specs/+/759828 | 16:14 |
| *** mgoddard has joined #openstack-nova | 16:15 | |
| *** hack-char has quit IRC | 16:26 | |
| openstackgerrit | Merged openstack/nova master: Ignore InstanceNotFound exception when host rejoin cluster https://review.opendev.org/c/openstack/nova/+/757053 | 16:28 |
| *** efried has quit IRC | 16:37 | |
| *** lpetrut has joined #openstack-nova | 16:38 | |
| stephenfin | dansmith: I know it's your final day before the new year, but if you've a chance to provide guidance at [1] I'd appreciate it. Would like to close that out this year :) https://review.opendev.org/c/openstack/nova/+/759114/3/nova/db/sqlalchemy/models.py#1212 | 16:39 |
| *** mkrai has quit IRC | 16:39 | |
| *** hack-char has joined #openstack-nova | 16:39 | |
| dansmith | stephenfin: ack | 16:40 |
| *** efried has joined #openstack-nova | 16:42 | |
| *** ociuhandu_ has quit IRC | 16:43 | |
| *** ociuhandu has joined #openstack-nova | 16:43 | |
| *** psachin has quit IRC | 16:47 | |
| kashyap | gibi: sean-k-mooney: I've outlined some notes here after my chat with QEMU migration maintainer, FWIW: https://bugs.launchpad.net/nova/+bug/1737625/comments/4 | 16:50 |
| openstack | Launchpad bug 1737625 in OpenStack Compute (nova) "live migration failed with "Failed to load virtio-blk:virtio" (libvirt/qemu)" [Undecided,Confirmed] | 16:50 |
| sean-k-mooney | kashyap: we might want to use the other bug | 16:51 |
| sean-k-mooney | and close that as a duplicate | 16:51 |
| kashyap | sean-k-mooney: Isn't this more accurate? It is talking about 'virtio-blk' and the Gate issue also is virtio-blk | 16:51 |
| sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1761798 | 16:51 |
| openstack | Launchpad bug 1761798 in OpenStack Compute (nova) "live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"" [Medium,Confirmed] | 16:51 |
| sean-k-mooney | kashyap: no i dont think it is i think its the same underlying issue | 16:51 |
| kashyap | In any case, I'm copying my comment into this one, too: https://bugs.launchpad.net/nova/+bug/1761798 | 16:52 |
| openstack | Launchpad bug 1761798 in OpenStack Compute (nova) "live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"" [Medium,Confirmed] | 16:52 |
| *** ociuhandu has quit IRC | 16:53 | |
| *** hamalq has joined #openstack-nova | 17:01 | |
| *** rpittau is now known as rpittau|afk | 17:03 | |
| *** artom has quit IRC | 17:11 | |
| *** tesseract has quit IRC | 17:16 | |
| *** hack-char has quit IRC | 17:17 | |
| *** hack-char has joined #openstack-nova | 17:18 | |
| *** ociuhandu has joined #openstack-nova | 17:21 | |
| *** ociuhandu has quit IRC | 17:21 | |
| *** nweinber has quit IRC | 17:29 | |
| *** nweinber has joined #openstack-nova | 17:29 | |
| *** ociuhandu has joined #openstack-nova | 17:32 | |
| *** hack-char has quit IRC | 17:34 | |
| *** ociuhandu has quit IRC | 17:37 | |
| *** hack-char has joined #openstack-nova | 17:39 | |
| *** hamalq has quit IRC | 17:43 | |
| *** hamalq has joined #openstack-nova | 17:43 | |
| *** mgoddard has quit IRC | 17:54 | |
| *** jangutter has joined #openstack-nova | 17:59 | |
| *** jangutter_ has quit IRC | 18:02 | |
| *** k_mouza has quit IRC | 18:13 | |
| openstackgerrit | Merged openstack/nova master: virt: Remove 'change_instance_metadata' API https://review.opendev.org/c/openstack/nova/+/749316 | 18:14 |
| *** jangutter has quit IRC | 18:20 | |
| *** jangutter has joined #openstack-nova | 18:21 | |
| *** jangutter has quit IRC | 18:38 | |
| *** jangutter has joined #openstack-nova | 18:38 | |
| *** jangutter has quit IRC | 18:39 | |
| *** jangutter has joined #openstack-nova | 18:40 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: virt: Remove 'reset_network' API https://review.opendev.org/c/openstack/nova/+/749315 | 18:42 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: virt: Remove 'get_all_bw_counters' API https://review.opendev.org/c/openstack/nova/+/749312 | 18:42 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove 'BandwidthUsage', 'BandwidthUsageList' https://review.opendev.org/c/openstack/nova/+/759114 | 18:42 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: db: Drop models for removed services, features https://review.opendev.org/c/openstack/nova/+/698706 | 18:42 |
| *** raildo has quit IRC | 18:45 | |
| *** dtantsur is now known as dtantsur|afk | 18:49 | |
| *** zzzeek has quit IRC | 18:55 | |
| *** raildo has joined #openstack-nova | 18:56 | |
| *** zzzeek has joined #openstack-nova | 18:57 | |
| sean-k-mooney | stephenfin: ah those are related to xenapi removal | 19:12 |
| sean-k-mooney | i tought it was related to the compute_monitors config intially | 19:16 |
| sean-k-mooney | https://github.com/openstack/nova/blob/120d866df7ab81b666c0e43b97705a90483533bc/nova/conf/compute.py#L107-L126 | 19:16 |
| sean-k-mooney | stephenfin: by the way we talked about the metrics weigher and weighing on things that were not compute weights in the past | 19:24 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/compute/monitors/__init__.py#L40-L48 | 19:24 |
| sean-k-mooney | the monitors are a stevaldor entry point | 19:24 |
| sean-k-mooney | and the numa memory bandwtih ones were actully plugins not in tree | 19:24 |
| sean-k-mooney | https://github.com/openstack/nova/blob/c250aca00ba89b6fad86bfaebb3083e9f5c55f3b/nova/compute/monitors/__init__.py#L44 the doc used to reference them years ago | 19:24 |
| *** mgoddard has joined #openstack-nova | 19:26 | |
| *** zzzeek has quit IRC | 19:27 | |
| sean-k-mooney | well it was propsoed upstream https://review.opendev.org/c/openstack/nova/+/270344 | 19:28 |
| sean-k-mooney | but it was not merged | 19:28 |
| *** zzzeek has joined #openstack-nova | 19:29 | |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova-specs/+/180983 was the spec | 19:31 |
| *** lpetrut has quit IRC | 19:36 | |
| *** mgoddard has quit IRC | 19:38 | |
| *** andrewbonney has quit IRC | 19:44 | |
| *** ociuhandu has joined #openstack-nova | 19:51 | |
| *** zzzeek has quit IRC | 19:53 | |
| *** zzzeek has joined #openstack-nova | 19:55 | |
| -openstackstatus- NOTICE: The Gerrit service on review.opendev.org is being restarted quickly to troubleshoot an unexpected error condition, downtime should be less than 5 minutes | 20:01 | |
| *** hamalq has quit IRC | 20:01 | |
| *** openstackgerrit has quit IRC | 20:02 | |
| *** spatel has joined #openstack-nova | 20:12 | |
| *** hamalq has joined #openstack-nova | 20:13 | |
| *** k_mouza has joined #openstack-nova | 20:14 | |
| *** jhesketh has quit IRC | 20:16 | |
| *** artom has joined #openstack-nova | 20:17 | |
| *** hamalq has quit IRC | 20:19 | |
| *** k_mouza has quit IRC | 20:19 | |
| *** jhesketh has joined #openstack-nova | 20:23 | |
| *** zzzeek has quit IRC | 20:30 | |
| *** zzzeek has joined #openstack-nova | 20:31 | |
| *** ociuhandu has quit IRC | 20:50 | |
| *** hamalq has joined #openstack-nova | 21:01 | |
| *** derekh has quit IRC | 21:01 | |
| *** derekh has joined #openstack-nova | 21:02 | |
| *** sapd1 has quit IRC | 21:04 | |
| *** derekh has quit IRC | 21:16 | |
| *** sapd1 has joined #openstack-nova | 21:16 | |
| *** nweinber has quit IRC | 21:18 | |
| *** nweinber has joined #openstack-nova | 21:18 | |
| *** happyhemant has quit IRC | 21:21 | |
| sean-k-mooney | gibi: stephenfin i just realised that https://review.opendev.org/c/openstack/nova/+/653506 is still not merged but its been so long that i have lost most of the context. its still proably should be fixed but im not sure when ill get to it. hopefully soon but i tought this was merged years ago... | 21:21 |
| *** rcernin has joined #openstack-nova | 21:23 | |
| *** ociuhandu has joined #openstack-nova | 21:25 | |
| *** nweinber has quit IRC | 21:32 | |
| *** ociuhandu has quit IRC | 21:45 | |
| *** artom has quit IRC | 21:47 | |
| *** slaweq_ has joined #openstack-nova | 21:49 | |
| *** slaweq has quit IRC | 21:52 | |
| *** hamalq has quit IRC | 21:55 | |
| *** hamalq has joined #openstack-nova | 21:56 | |
| *** ociuhandu has joined #openstack-nova | 21:58 | |
| *** hamalq has quit IRC | 21:59 | |
| *** spatel has quit IRC | 22:00 | |
| *** ociuhandu has quit IRC | 22:02 | |
| *** luksky has quit IRC | 22:05 | |
| *** iurygregory has quit IRC | 22:05 | |
| *** iurygregory has joined #openstack-nova | 22:06 | |
| *** luksky has joined #openstack-nova | 22:18 | |
| *** iurygregory has quit IRC | 22:20 | |
| *** ociuhandu has joined #openstack-nova | 22:22 | |
| *** ociuhandu has quit IRC | 22:26 | |
| *** iurygregory has joined #openstack-nova | 22:35 | |
| *** xek_ has quit IRC | 22:37 | |
| *** slaweq_ has quit IRC | 22:38 | |
| *** iurygregory_ has joined #openstack-nova | 23:02 | |
| *** iurygregory has quit IRC | 23:04 | |
| *** iurygregory_ is now known as iurygregory | 23:08 | |
| *** luksky has quit IRC | 23:10 | |
| *** ociuhandu has joined #openstack-nova | 23:48 | |
| *** ircuser-1 has joined #openstack-nova | 23:52 | |
| *** ociuhandu has quit IRC | 23:53 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
