| *** arne_wiebalck has quit IRC | 00:03 | |
| *** yonglihe has quit IRC | 00:03 | |
| *** yonglihe has joined #openstack-nova | 00:03 | |
| *** arne_wiebalck has joined #openstack-nova | 00:03 | |
| *** macz_ has quit IRC | 00:18 | |
| brinzhang_ | artom: ack, thanks | 00:25 |
|---|---|---|
| *** LinPeiWen has joined #openstack-nova | 00:32 | |
| *** gyee has quit IRC | 01:02 | |
| *** macz_ has joined #openstack-nova | 01:11 | |
| openstackgerrit | Jorhson Deng proposed openstack/nova-specs master: Add instance_state to ensure volume attachment successful https://review.opendev.org/758920 | 01:14 |
| *** macz_ has quit IRC | 01:16 | |
| *** ociuhandu has joined #openstack-nova | 01:28 | |
| *** Liang__ has joined #openstack-nova | 01:30 | |
| *** macz_ has joined #openstack-nova | 01:33 | |
| *** ociuhandu has quit IRC | 01:33 | |
| *** sapd1 has joined #openstack-nova | 01:33 | |
| *** macz_ has quit IRC | 01:37 | |
| *** hamalq has quit IRC | 01:38 | |
| *** Liang__ has quit IRC | 02:33 | |
| *** Liang__ has joined #openstack-nova | 02:37 | |
| *** rcernin has quit IRC | 02:48 | |
| *** rcernin has joined #openstack-nova | 02:49 | |
| openstackgerrit | Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264 | 02:57 |
| *** dikonoor has joined #openstack-nova | 03:02 | |
| openstackgerrit | Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264 | 03:19 |
| openstackgerrit | Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264 | 03:23 |
| *** zzzeek has quit IRC | 03:24 | |
| *** zzzeek has joined #openstack-nova | 03:25 | |
| openstackgerrit | Brin Zhang proposed openstack/nova stable/victoria: optimize the shelve code flow https://review.opendev.org/761268 | 03:27 |
| *** psachin has joined #openstack-nova | 03:46 | |
| *** sapd1 has quit IRC | 04:29 | |
| *** macz_ has joined #openstack-nova | 04:34 | |
| *** macz_ has quit IRC | 04:39 | |
| *** JamesBenson has quit IRC | 05:21 | |
| *** psachin has quit IRC | 05:27 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-nova | 05:33 | |
| *** Jorhson has joined #openstack-nova | 05:49 | |
| *** ratailor has joined #openstack-nova | 05:52 | |
| Jorhson | lyarwood: Hi, i think we should better to add task state, because that can avoid conflicting with other tasks. | 05:53 |
| Jorhson | lyarwood: and change the code with minimal cost. | 05:55 |
| Jorhson | lyarwood:https://review.opendev.org/#/c/758920/ @75 | 05:58 |
| Jorhson | lyarwood:https://review.opendev.org/#/c/758920/11/specs/wallaby/approved/ensure-volume-attachment-successful.rst@75 | 06:01 |
| brinzhang_ | lyarwood: jorhson's mean your point in PS9 in this specification | 06:05 |
| *** k_mouza has joined #openstack-nova | 06:08 | |
| *** k_mouza has quit IRC | 06:12 | |
| *** Liang__ has quit IRC | 06:21 | |
| *** Liang__ has joined #openstack-nova | 06:22 | |
| *** recyclehero has quit IRC | 06:26 | |
| *** vishalmanchanda has joined #openstack-nova | 06:26 | |
| *** recyclehero has joined #openstack-nova | 06:28 | |
| *** macz_ has joined #openstack-nova | 06:28 | |
| *** macz_ has quit IRC | 06:33 | |
| *** recyclehero has quit IRC | 06:38 | |
| *** recyclehero has joined #openstack-nova | 06:39 | |
| *** whoami-rajat__ has joined #openstack-nova | 07:16 | |
| *** dklyle has quit IRC | 07:35 | |
| openstackgerrit | Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264 | 07:46 |
| *** ralonsoh has joined #openstack-nova | 07:54 | |
| *** spatel has joined #openstack-nova | 07:58 | |
| *** spatel has quit IRC | 08:03 | |
| *** slaweq has joined #openstack-nova | 08:03 | |
| *** rcernin has quit IRC | 08:05 | |
| *** andrewbonney has joined #openstack-nova | 08:09 | |
| *** macz_ has joined #openstack-nova | 08:16 | |
| *** tesseract has joined #openstack-nova | 08:18 | |
| *** macz_ has quit IRC | 08:20 | |
| *** rcernin has joined #openstack-nova | 08:37 | |
| *** tosky has joined #openstack-nova | 08:42 | |
| *** Jorhson has quit IRC | 08:54 | |
| *** rcernin has quit IRC | 08:56 | |
| *** rpittau|afk is now known as rpittau | 09:05 | |
| *** ociuhandu has joined #openstack-nova | 09:09 | |
| *** crazik has joined #openstack-nova | 09:10 | |
| openstackgerrit | Elod Illes proposed openstack/nova stable/pike: Lowercase ironic driver hash ring and ignore case in cache https://review.opendev.org/723055 | 09:14 |
| *** ralonsoh has quit IRC | 09:16 | |
| *** ociuhandu has quit IRC | 09:19 | |
| *** ralonsoh has joined #openstack-nova | 09:19 | |
| *** larainema has joined #openstack-nova | 09:23 | |
| lyarwood | johanssone / brinzhang_ ; morning | 09:24 |
| *** macz_ has joined #openstack-nova | 09:24 | |
| *** macz_ has quit IRC | 09:29 | |
| *** ociuhandu has joined #openstack-nova | 09:32 | |
| brinzhang_ | lyarwood: good morning | 09:34 |
| *** Jorhson has joined #openstack-nova | 09:34 | |
| brinzhang_ | jorhson^ | 09:34 |
| *** derekh has joined #openstack-nova | 09:35 | |
| lyarwood | johanssone / brinzhang_ ; sorry just as I started replying here I had to jump offline and help with childcare | 09:36 |
| lyarwood | argh | 09:36 |
| lyarwood | Jorhson: sorry | 09:36 |
| lyarwood | Jorhson / brinzhang_ ; so my main issue still with the spec is the change of behaviour in the API that it's introducing | 09:37 |
| *** ociuhandu has quit IRC | 09:38 | |
| lyarwood | Jorhson / brinzhang_ ; before multiple requests would be queued within the computes, with this change callers will have to retry if there's already a request in-flight against the instance | 09:38 |
| lyarwood | Jorhson / brinzhang_ ; that's at least a microversion IMHO | 09:38 |
| Jorhson | lyarwood: if in this way, should we add the task state for this option? | 09:39 |
| lyarwood | Jorhson: this is if you add a task state | 09:39 |
| lyarwood | Jorhson: n-api will start rejecting attach volume requests if the task_state is already ATTACHING, instead of allowing n-cpu to queue things | 09:39 |
| Jorhson | lyarwood: yeah, this is conflicting | 09:40 |
| lyarwood | Jorhson: this all comes down to the RPC call to reservice_block_device_name | 09:40 |
| lyarwood | Jorhson: and I honestly think we can just drop device_name from the APIs and remove this, again with a microversion | 09:41 |
| Jorhson | lyarwood: but i also think it's need for this options, because attaching volume is independent option | 09:42 |
| lyarwood | Jorhson: true but that's why we queue things in the compute with the various locks | 09:42 |
| Jorhson | lyarwood: there will be conflicting if we do others task while the attching task is not over | 09:43 |
| lyarwood | Jorhson: we'd be changing the semantics of the API here pretty drastically and forcing the callers to retry by introducing these task_states | 09:43 |
| brinzhang_ | jorhson: agree with lyarwood, add a microversion to remove the device, and in order to avoid the conflict, we also need to add the task_state as current description in your spec | 09:43 |
| lyarwood | well we have the instance.uuid lock to avoid conflicts | 09:43 |
| lyarwood | most compute operations should be taking that IMHO | 09:44 |
| lyarwood | I know it's not the case for everything at the moment | 09:44 |
| lyarwood | so what if we didn't reject the request if the task_state was attaching and queued it in the API somehow? | 09:46 |
| lyarwood | Jorhson / brinzhang_ ; ^ | 09:46 |
| Jorhson | lyarwood: yeah, for the users, there are less probability to attach multiple volumes, and i agree with your sheme, but i think it's important to solve the option's conflict | 09:47 |
| lyarwood | Jorhson: it's a pretty common usecase with k8s running on OpenStack IMHO | 09:48 |
| *** Liang__ has quit IRC | 09:51 | |
| *** ociuhandu has joined #openstack-nova | 09:53 | |
| *** Jorhson has quit IRC | 09:53 | |
| brinzhang_ | lyarwood: ok, I understand what you mean, agree | 09:54 |
| brinzhang_ | lyarwood: Jorhson will update this spec later, this way it's better for k8s on OpenStack scenario | 09:55 |
| *** rcernin has joined #openstack-nova | 09:58 | |
| lyarwood | brinzhang_: ack many thanks both, I'm going to be AFK again this afternoon UK time but I'll review the spec this evening once I'm back. | 09:59 |
| *** macz_ has joined #openstack-nova | 10:00 | |
| brinzhang_ | lyarwood: ack, thanks too | 10:00 |
| aarents | Hi nova | 10:01 |
| aarents | lyarwood: there is some progress on that q35 bug: https://bugs.launchpad.net/nova/+bug/1832248 , if it is a qemu bug, what is the suitable status for this bug Fixed_in_relsease or invalid? | 10:02 |
| openstack | Launchpad bug 1832248 in OpenStack Compute (nova) "tempest.api.volume.test_volumes_extend.VolumesExtendAttachedTest.test_extend_attached_volume failing when using the Q35 machine type" [Medium,Triaged] - Assigned to Alexandre arents (aarents) | 10:02 |
| *** macz_ has quit IRC | 10:05 | |
| lyarwood | aarents: awesome sleuthing! :) | 10:07 |
| lyarwood | aarents: erm, invalid I guess? gibi ^ any thoughts, looking to close a Nova bug that was actually a QEMU bug that has already been fixed | 10:08 |
| gibi | lyarwood: if it is fixed in QEMU and no need for any nova change then Invliad it is | 10:09 |
| gibi | Invalid even | 10:09 |
| lyarwood | cool thanks | 10:09 |
| *** dtantsur|afk is now known as dtantsur | 10:11 | |
| *** rcernin has quit IRC | 10:13 | |
| aarents | cool, I just hope it will fix also my downstrem bug regarding interface attach! | 10:15 |
| *** ociuhandu has quit IRC | 10:37 | |
| *** ociuhandu has joined #openstack-nova | 10:37 | |
| *** k_mouza has joined #openstack-nova | 10:41 | |
| *** ociuhandu has quit IRC | 10:42 | |
| *** ociuhandu has joined #openstack-nova | 10:49 | |
| *** lpetrut has joined #openstack-nova | 10:55 | |
| *** spatel has joined #openstack-nova | 10:59 | |
| *** ociuhandu has quit IRC | 11:01 | |
| *** spatel has quit IRC | 11:04 | |
| *** ociuhandu has joined #openstack-nova | 11:32 | |
| *** ociuhandu has quit IRC | 11:40 | |
| *** dtantsur is now known as dtantsur|brb | 11:44 | |
| sean-k-mooney | do we need to enable the new qemu from the cloud arcive on ussuri and below to ensure we dont hit that bug | 11:56 |
| *** tbachman has quit IRC | 11:57 | |
| openstackgerrit | Wenping Song proposed openstack/nova-specs master: Support vGPU management by Cyborg https://review.opendev.org/750116 | 12:04 |
| *** JamesBenson has joined #openstack-nova | 12:04 | |
| *** JamesBenson has quit IRC | 12:06 | |
| *** JamesBenson has joined #openstack-nova | 12:06 | |
| *** raildo has joined #openstack-nova | 12:10 | |
| *** ratailor has quit IRC | 12:20 | |
| *** spatel has joined #openstack-nova | 12:27 | |
| *** spatel has quit IRC | 12:32 | |
| *** mgariepy has quit IRC | 12:47 | |
| *** ociuhandu has joined #openstack-nova | 12:50 | |
| *** jangutter has joined #openstack-nova | 12:59 | |
| *** dtantsur|brb is now known as dtantsur | 12:59 | |
| *** ociuhandu has quit IRC | 13:01 | |
| *** ociuhandu has joined #openstack-nova | 13:02 | |
| *** jangutter_ has quit IRC | 13:03 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Prevent starting services with older than N-1 computes https://review.opendev.org/738482 | 13:04 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add upgrade check about old computes https://review.opendev.org/760520 | 13:04 |
| *** ociuhandu has quit IRC | 13:07 | |
| *** macz_ has joined #openstack-nova | 13:09 | |
| *** macz_ has quit IRC | 13:14 | |
| *** ociuhandu has joined #openstack-nova | 13:17 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Prevent starting services with older than N-1 computes https://review.opendev.org/738482 | 13:19 |
| *** Luzi has joined #openstack-nova | 13:19 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add upgrade check about old computes https://review.opendev.org/760520 | 13:21 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use _evacuate_server helper in func test https://review.opendev.org/758210 | 13:23 |
| *** mgariepy has joined #openstack-nova | 13:25 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1897528 https://review.opendev.org/756696 | 13:27 |
| openstack | bug 1897528 in OpenStack Compute (nova) "32bit pci domain number is not supported" [High,In progress] https://launchpad.net/bugs/1897528 - Assigned to Balazs Gibizer (balazs-gibizer) | 13:27 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ignore PCI devices with 32bit domain https://review.opendev.org/756697 | 13:27 |
| *** tbachman has joined #openstack-nova | 13:36 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Support interface attach with qos ports https://review.opendev.org/755477 | 13:36 |
| *** luksky has joined #openstack-nova | 13:41 | |
| *** ociuhandu has quit IRC | 14:02 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1896463 in func env https://review.opendev.org/754100 | 14:03 |
| openstack | bug 1896463 in OpenStack Compute (nova) rocky "evacuation failed: Port update failed : Unable to correlate PCI slot " [Low,In progress] https://launchpad.net/bugs/1896463 | 14:03 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Set instance host and drop migration under lock https://review.opendev.org/754815 | 14:05 |
| *** ociuhandu has joined #openstack-nova | 14:05 | |
| *** ralonsoh has left #openstack-nova | 14:06 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1896463 in func env https://review.opendev.org/754100 | 14:07 |
| openstack | bug 1896463 in OpenStack Compute (nova) rocky "evacuation failed: Port update failed : Unable to correlate PCI slot " [Low,In progress] https://launchpad.net/bugs/1896463 | 14:07 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Set instance host and drop migration under lock https://review.opendev.org/754815 | 14:08 |
| *** ociuhandu has quit IRC | 14:10 | |
| *** kaisers has joined #openstack-nova | 14:11 | |
| *** ociuhandu has joined #openstack-nova | 14:17 | |
| *** sapd1 has joined #openstack-nova | 14:27 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add upgrade check about old computes https://review.opendev.org/760520 | 14:51 |
| openstackgerrit | Takashi Natsume proposed openstack/nova stable/victoria: Fix a hacking test https://review.opendev.org/758112 | 15:02 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: [doc]: Fix glance image_metadata link https://review.opendev.org/761423 | 15:05 |
| *** macz_ has joined #openstack-nova | 15:06 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: Use cell targeted context to query BDMs for metadata https://review.opendev.org/761424 | 15:07 |
| *** dikonoor has quit IRC | 15:08 | |
| *** jdillaman has joined #openstack-nova | 15:08 | |
| *** macz_ has quit IRC | 15:11 | |
| *** dklyle has joined #openstack-nova | 15:23 | |
| *** Luzi has quit IRC | 15:24 | |
| *** abhishekk is now known as abhishekk|dinner | 15:24 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Bump the lowest eventlet version to 0.26.1 https://review.opendev.org/761427 | 15:29 |
| *** brinzhang0 has joined #openstack-nova | 15:30 | |
| *** brinzhang_ has quit IRC | 15:32 | |
| *** macz_ has joined #openstack-nova | 15:36 | |
| *** lpetrut has quit IRC | 15:42 | |
| *** k_mouza has quit IRC | 15:43 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: [trivial]: replace NUMNA with NUMA https://review.opendev.org/761436 | 15:55 |
| * bauzas feels like a janitor with the compute RPC bump :) | 15:59 | |
| *** LinPeiWen has quit IRC | 16:00 | |
| *** k_mouza has joined #openstack-nova | 16:03 | |
| *** ociuhandu has quit IRC | 16:09 | |
| *** ociuhandu has joined #openstack-nova | 16:10 | |
| *** k_mouza has quit IRC | 16:14 | |
| *** ociuhandu has quit IRC | 16:14 | |
| *** ociuhandu has joined #openstack-nova | 16:15 | |
| *** k_mouza has joined #openstack-nova | 16:19 | |
| openstackgerrit | Merged openstack/nova-specs master: [trivial]: replace NUMNA with NUMA https://review.opendev.org/761436 | 16:19 |
| *** elod has quit IRC | 16:26 | |
| *** elod has joined #openstack-nova | 16:26 | |
| gibi | somebody with connection to the libvirt maintainer should look at this nova bug https://bugs.launchpad.net/nova/+bug/1902276 | 16:26 |
| openstack | Launchpad bug 1902276 in OpenStack Compute (nova) "libvirtd going into a tight loop causing instances to not transition to ACTIVE" [Undecided,New] | 16:26 |
| melwitt | kashyap: ^ | 16:28 |
| kashyap | melwitt: Yeah, familar with it, as I worked with the reporter here the other day | 16:28 |
| melwitt | ah ok, cool | 16:28 |
| kashyap | melwitt: I asked one of the libvirt devs on Friday, but my timing wasn't right | 16:28 |
| gibi | kashyap: thanks! | 16:28 |
| kashyap | I'll check again | 16:28 |
| kashyap | gibi: It looks fishy, as it's not 100% reproducible ... as the reporter says "a few minutes later things go back to normal" | 16:29 |
| kashyap | But we don't know what changed :-( | 16:29 |
| gibi | kashyap: the nova image download take ~ 300 seconds for the VM that then triggers the loop in libvirtd so it might be that the hypervisor host has high load | 16:31 |
| kashyap | gibi: Yeah, just reading your report :) | 16:31 |
| gibi | but I was not able to confirm it from the logs | 16:31 |
| kashyap | I see. Hypervisor load sounds plausible - as we've hit load-related (CI) issues libvirt driver before. But still let me check w/ Dan or someone from upstream libvirt | 16:33 |
| gibi | kashyap: thanks for taking this up with the libvirt maintainers | 16:34 |
| kashyap | gibi: Just posted on #virt, OFTC network. | 16:35 |
| kashyap | gibi: Is this blocking patch merges? | 16:36 |
| melwitt | I've been struggling for a couple of days trying to get an approved patch through the gate, but I'm not sure whether that particular bug is involved. I would need to re-look at the logs to verify | 16:39 |
| kashyap | (I've reposted the looping libvirtd log bits as a plain text, as the pastebins expire) | 16:39 |
| *** abhishekk|dinner is now known as abhishekk | 16:40 | |
| kashyap | melwitt: Noted; Michael, the reporter, was saying on last Friday that it's "intermittent", which makes it a bit more difficult to debug | 16:40 |
| melwitt | yeah, that's been the theme of all of the gate bugs I'm aware of. intermittent and thus hard to troubleshoot :( | 16:41 |
| melwitt | *current gate bugs I'm aware of | 16:42 |
| kashyap | Yeah, matches my past experience | 16:43 |
| kashyap | melwitt: In the same vein as how Twitter (I'm not on it) seems to label Trump's tweets as misleading, wonder we should adapt that text for these intermittent bugs :D | 16:43 |
| * kashyap gets a "diff" | 16:43 | |
| kashyap | - Some or all of the content shared in this Tweet is disputed and might be misleading about an election or other civic process. | 16:44 |
| kashyap | + Some or all of the content shared in this bug is disputed and might be misleading due to intermittent failures. | 16:44 |
| * kashyap ducks | 16:44 | |
| kashyap | gibi: melwitt: More seriously, can I "subscribe" (Cc) someone else to a LaunchPad, right? | 16:46 |
| kashyap | IIRC, yes. /me tries | 16:46 |
| melwitt | I think you can | 16:46 |
| kashyap | melwitt: I can't :-( I wanted to Cc Michal from libvirt but it says "No items matched <email ID>" | 16:48 |
| melwitt | do you know his launchpad id? | 16:48 |
| kashyap | melwitt: Oh, having a Launchpad ID is mandatory? | 16:48 |
| melwitt | it might be, that's the only way I've seen subscribing | 16:49 |
| kashyap | Ah, noted. I don't think he has one - searching doesn't show up anything. | 16:50 |
| kashyap | melwitt: I pointed to him on IRC; he's taking a look | 16:51 |
| melwitt | thanks! | 16:53 |
| * kashyap says sorry for the tasteless politics-related 'joke' earlier | 16:53 | |
| *** sapd1 has quit IRC | 16:54 | |
| kashyap | melwitt: gibi: That's quick -- Michal (Privoznik) says it looks like a genuine bug. I'll update the bug once we get more details | 16:55 |
| melwitt | sounds great, thank you kashyap | 16:56 |
| *** mgariepy has quit IRC | 17:02 | |
| kashyap | melwitt: So, the libvirt version in the logs above is 5.4.0; but havne't we switche dalready to libvirt-6.0.0? | 17:06 |
| kashyap | lyarwood: --^ (By "we", I mean upstream CI) | 17:06 |
| kashyap | So Michal says, there were improvements in libvirt-6.1.0 release on this area of event loops. | 17:08 |
| * kashyap goes to add in a comment | 17:08 | |
| *** ociuhandu_ has joined #openstack-nova | 17:08 | |
| *** ociuhandu has quit IRC | 17:11 | |
| *** ociuhandu_ has quit IRC | 17:13 | |
| stephenfin | lyarwood: Could you cast an eye over https://review.opendev.org/#/c/631053/ this evening, please? | 17:13 |
| stephenfin | It's been around for quite a while :-D | 17:14 |
| *** tosky has quit IRC | 17:15 | |
| mloza | hello, is it possible to update the video model to vmga of an existing instance? | 17:18 |
| *** tesseract has quit IRC | 17:18 | |
| mloza | if i edit /etc/libvirt/qemu/instance-, it reverts to default when the instance is hard rebooted | 17:19 |
| stephenfin | mloza: Outside of rebuilding to a new image, no. We don't support setting it via the flavor so resize isn't an option | 17:20 |
| stephenfin | mloza: You'll have to modify the DB manually if you want to avoid the rebuild | 17:21 |
| mloza | can you tell me which table do I need modify | 17:22 |
| mloza | to modif* | 17:22 |
| stephenfin | iirc, we persist image metadata properties for an image in the instance_system_metadata table | 17:24 |
| *** tbachman has quit IRC | 17:24 | |
| stephenfin | in case it wasn't obvious, back up the DB first and note that any support guarantees are gone out the window if you modify the DB manually | 17:25 |
| *** tbachman has joined #openstack-nova | 17:25 | |
| *** rpittau is now known as rpittau|afk | 17:35 | |
| sean-k-mooney | i think the table name is system_metadata not instance_system metadata but yes we do | 17:37 |
| sean-k-mooney | with an img_ prefix | 17:37 |
| sean-k-mooney | os if it was hw_video_model it woudl be img_hw_video_model in the db | 17:38 |
| *** vishalmanchanda has quit IRC | 17:38 | |
| *** k_mouza has quit IRC | 17:38 | |
| sean-k-mooney | mloza: were you asking about this on the mailing list too? we basically said the same in our replies | 17:38 |
| *** lemko6 has joined #openstack-nova | 17:39 | |
| *** lemko8 has joined #openstack-nova | 17:41 | |
| *** lemko has quit IRC | 17:43 | |
| *** lemko8 is now known as lemko | 17:43 | |
| *** lemko6 has quit IRC | 17:44 | |
| *** hamalq has joined #openstack-nova | 17:46 | |
| *** martinkennelly has joined #openstack-nova | 17:47 | |
| bauzas | gibi: stephenfin: fwiw, you accepted a breaking RPC change with https://review.opendev.org/#/c/715326/29/nova/compute/manager.py@3327 by not accepting a nullable accels argument | 17:49 |
| bauzas | sean-k-mooney: ^ | 17:50 |
| bauzas | if a compute client is sending a 5.0 cast to a compute service, then there won't have a accels argument, so the manager will return an exception | 17:52 |
| dansmith | bauzas: good catch | 17:56 |
| dansmith | bauzas: not too late to fix that | 17:56 |
| *** dtantsur is now known as dtantsur|afk | 17:57 | |
| bauzas | dansmith: I wonder whether we should fix it by the compute v5 proxy I write or having another change we could backport to victoria ? | 17:57 |
| dansmith | bauzas: another change that we backport | 17:58 |
| bauzas | ack, doing it then | 17:58 |
| dansmith | before people try to upgrade to victoria | 17:58 |
| dansmith | technically, this shouldn't be a problem if people upgrade their controllers first, but if they don't, they'll get an explosion that won't be easy to decipher | 17:58 |
| bauzas | yeah | 17:58 |
| dansmith | well, no, actuall,y | 17:59 |
| dansmith | it would blow up for anyone with an old compute if the version is pinned | 17:59 |
| bauzas | given most of the operators upgrade first their conductors, it shouldn't be a problem | 17:59 |
| bauzas | but in case they pin it, yes | 17:59 |
| dansmith | no, because the client was done properly, it will break | 17:59 |
| dansmith | even if they don't pin, assuming they use =auto | 17:59 |
| bauzas | if you pin the API, right? | 17:59 |
| bauzas | why then for auto ? | 18:00 |
| dansmith | the default is =auto, which will select the lowest version supported by all computes, | 18:00 |
| bauzas | ahah | 18:00 |
| dansmith | so if you have one old compute, api, conductor, etc will all choose the older and will not send that argument, | 18:00 |
| bauzas | I see | 18:00 |
| dansmith | and thus it'll explode | 18:00 |
| bauzas | TIL | 18:00 |
| bauzas | about how auto works | 18:00 |
| dansmith | so s/good catch/great catch/ :) | 18:00 |
| *** derekh has quit IRC | 18:01 | |
| dansmith | easy backport to fix it though, luckily, before people start rolling to V | 18:01 |
| bauzas | yup, writing it now | 18:01 |
| bauzas | git stash first tho :) | 18:01 |
| dansmith | heh yeah I bet :) | 18:01 |
| bauzas | dansmith: do we have some documentation about pin=auto ? maybe on your blog ? | 18:04 |
| dansmith | lol | 18:04 |
| dansmith | I guess I didn't think it was really something people often confused | 18:05 |
| dansmith | I'd expect the config doc to be accurate, but let me look | 18:05 |
| bauzas | nevermind, found it ;) https://docs.openstack.org/nova/latest/user/upgrade.html | 18:05 |
| dansmith | the config doc says "don't worry, we'll handle it" without any detail | 18:06 |
| bauzas | yeah, looking at the config option text | 18:07 |
| dansmith | but honestly, | 18:07 |
| dansmith | from the perspective of a deployer, they really just need to know that auto selects the right version, | 18:08 |
| dansmith | without needing to know that our client needs *a* version and not per-target | 18:08 |
| dansmith | we definitely don't want to look up each target's version before *each* call because that would suck | 18:08 |
| *** mlavalle has quit IRC | 18:08 | |
| bauzas | yeah | 18:08 |
| dansmith | so maybe some developer docs to explain how it actually works, if you think that's important, but ... | 18:08 |
| bauzas | but if we have problem with the default, then that's a high bug for rolling upgrades | 18:08 |
| bauzas | grmblbl, I need to install both python37 and pre_commit module just for a single git commit | 18:09 |
| dansmith | well, | 18:10 |
| dansmith | not just the default, but also if someone were to hand-set the right version | 18:10 |
| bauzas | yup, any rolling upgrade | 18:11 |
| *** mgariepy has joined #openstack-nova | 18:17 | |
| *** mugsie has quit IRC | 18:18 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: WIP: Bump the Compute RPC API to version 6.0 https://review.opendev.org/761452 | 18:20 |
| *** mugsie has joined #openstack-nova | 18:21 | |
| *** andrewbonney has quit IRC | 18:24 | |
| bauzas | dansmith: triaged https://bugs.launchpad.net/nova/+bug/1902925 lemme say if I told something wrong | 18:27 |
| openstack | Launchpad bug 1902925 in OpenStack Compute (nova) victoria "Upgrades to compute RPC API 5.12 are broken" [Critical,Confirmed] - Assigned to Sylvain Bauza (sylvain-bauza) | 18:27 |
| dansmith | bauzas: yep, sounds good.. it makes it sound like the error was combined with the rpc client being right and the server being wrong, | 18:29 |
| dansmith | but even if the client hadn't bumped the version, it would still be wrong | 18:29 |
| dansmith | but yeah, what you said is all right | 18:29 |
| bauzas | ack | 18:29 |
| bauzas | the fix is simple, but I wonder whether we already have a test suite somewhere for it | 18:29 |
| dansmith | well, generally there should be a test to pass the old version and new version of the call, | 18:30 |
| dansmith | but I think it requires adding two test cases, which probably didn't happen | 18:31 |
| *** jangutter_ has joined #openstack-nova | 18:46 | |
| *** jangutter has quit IRC | 18:48 | |
| *** mlavalle has joined #openstack-nova | 18:50 | |
| *** mlavalle has quit IRC | 18:59 | |
| bauzas | can anyone know how to get the full stacktrace returned by the API when we have a HTTP500 ? | 19:09 |
| * bauzas writes a functest | 19:09 | |
| bauzas | I tried with admin rights and I only get the main exception | 19:10 |
| bauzas | ie. OpenStackApiException('Unexpected status code: {"computeFault": {"code": 500, "message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\\n<class \'TypeError\'>"}}') | 19:10 |
| bauzas | while I would like to see the whole stacktrace | 19:10 |
| bauzas | dansmith: any idea for ^ ? | 19:11 |
| dansmith | that's all we see in the log? | 19:11 |
| bauzas | yup | 19:11 |
| bauzas | I wrote a regression test that works | 19:11 |
| dansmith | I would have thought we log.exception a 500, but I'd guess maybe we've had so many in the past that we had to squelch them | 19:12 |
| bauzas | meh | 19:12 |
| bauzas | anyway | 19:12 |
| bauzas | I'll upload and we could discuss this in the change | 19:12 |
| *** luksky has quit IRC | 19:12 | |
| dansmith | ack | 19:13 |
| dansmith | bauzas: I assume you're just asserting that we get a 500 before the fix and a 2xx after, right? that seems clear enough to me | 19:19 |
| *** mlavalle has joined #openstack-nova | 19:19 | |
| bauzas | dansmith: something like that, yes | 19:20 |
| dansmith | seems fine | 19:20 |
| artom | bauzas, I believe there's precedent for capturing the log and asserting stuff about the content | 19:21 |
| artom | If you want to be really sure you're getting the exact error you expect | 19:21 |
| *** _mlavalle_1 has joined #openstack-nova | 19:21 | |
| bauzas | uploading it | 19:23 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add a regression test for 5.12 compute API issue https://review.opendev.org/761457 | 19:23 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: Fix the compute RPC 5.12 issue https://review.opendev.org/761458 | 19:23 |
| bauzas | dansmith: artom: gibi: stephenfin: there it is ^ | 19:23 |
| *** mlavalle has quit IRC | 19:24 | |
| dansmith | artom: on a functional? | 19:24 |
| artom | dansmith, I'm pretty sure I've seen it done, yeah | 19:24 |
| *** luksky has joined #openstack-nova | 19:24 | |
| dansmith | that'd be pretty deep mocking of something that doesn't need to be mocked for behavior | 19:24 |
| dansmith | but even still a 500 that goes to not 500 after the fix seems like plenty to me | 19:24 |
| *** _mlavalle_2 has joined #openstack-nova | 19:24 | |
| dansmith | to be clear, I think we need more than just mocking of a log call, | 19:25 |
| dansmith | we need mocking of whatever wraps that exception | 19:25 |
| bauzas | anyway, /me goes off | 19:25 |
| bauzas | 8.25pm here :) | 19:25 |
| *** _mlavalle_1 has quit IRC | 19:27 | |
| *** k_mouza has joined #openstack-nova | 19:39 | |
| *** k_mouza has quit IRC | 19:43 | |
| *** luksky has quit IRC | 20:01 | |
| *** artom has quit IRC | 20:04 | |
| *** luksky has joined #openstack-nova | 20:13 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Add os-volume_attachments reference docs https://review.opendev.org/760971 | 20:22 |
| *** slaweq has quit IRC | 20:32 | |
| *** tosky has joined #openstack-nova | 20:35 | |
| openstackgerrit | Billy Olsen proposed openstack/nova stable/ussuri: Set migrate_data.vifs only when using multiple port bindings https://review.opendev.org/759151 | 20:39 |
| *** rchurch has quit IRC | 20:46 | |
| *** rchurch has joined #openstack-nova | 20:49 | |
| *** raildo has quit IRC | 21:19 | |
| *** JamesBenson has quit IRC | 21:24 | |
| *** tbachman has quit IRC | 21:26 | |
| *** rcernin has joined #openstack-nova | 21:41 | |
| *** artom has joined #openstack-nova | 21:54 | |
| *** tbachman has joined #openstack-nova | 21:59 | |
| *** artom has quit IRC | 22:02 | |
| *** artom has joined #openstack-nova | 22:03 | |
| *** rcernin has quit IRC | 22:09 | |
| openstackgerrit | Billy Olsen proposed openstack/nova stable/train: add functional regression test for bug #1888395 https://review.opendev.org/759533 | 22:09 |
| openstack | bug 1888395 in OpenStack Compute (nova) ussuri "live migration of a vm using the single port binding work flow is broken in train as a result of the introduction of sriov live migration" [High,In progress] https://launchpad.net/bugs/1888395 - Assigned to Billy Olsen (billy-olsen) | 22:09 |
| *** rcernin has joined #openstack-nova | 22:09 | |
| *** whoami-rajat__ has quit IRC | 22:13 | |
| *** luksky has quit IRC | 22:48 | |
| *** rcernin has quit IRC | 23:04 | |
| *** rcernin has joined #openstack-nova | 23:12 | |
| *** spatel has joined #openstack-nova | 23:12 | |
| *** bbowen has quit IRC | 23:13 | |
| *** bbowen has joined #openstack-nova | 23:15 | |
| *** rcernin has quit IRC | 23:16 | |
| *** rcernin has joined #openstack-nova | 23:16 | |
| *** spatel has quit IRC | 23:17 | |
| *** tbachman has quit IRC | 23:34 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!