| *** ociuhandu has quit IRC | 00:04 | |
| *** ociuhandu has joined #openstack-nova | 00:05 | |
| *** ociuhandu has quit IRC | 00:09 | |
| *** brinzhang0 has joined #openstack-nova | 00:09 | |
| *** ociuhandu has joined #openstack-nova | 00:11 | |
| *** tosky has quit IRC | 00:12 | |
| *** brinzhang_ has quit IRC | 00:13 | |
| *** ociuhandu has quit IRC | 00:15 | |
| *** ociuhandu has joined #openstack-nova | 00:16 | |
| *** ociuhandu has quit IRC | 00:21 | |
| *** ociuhandu has joined #openstack-nova | 00:22 | |
| *** ociuhandu has quit IRC | 00:27 | |
| *** ociuhandu has joined #openstack-nova | 00:28 | |
| *** ociuhandu has quit IRC | 00:33 | |
| *** ociuhandu has joined #openstack-nova | 00:34 | |
| *** ociuhandu has quit IRC | 00:39 | |
| *** ociuhandu has joined #openstack-nova | 00:39 | |
| *** ociuhandu has quit IRC | 00:44 | |
| *** ociuhandu has joined #openstack-nova | 00:46 | |
| *** LinPeiWen has joined #openstack-nova | 00:46 | |
| *** ociuhandu has quit IRC | 00:50 | |
| *** ociuhandu has joined #openstack-nova | 00:57 | |
| *** ociuhandu has quit IRC | 01:02 | |
| *** ociuhandu has joined #openstack-nova | 01:15 | |
| *** ociuhandu has quit IRC | 01:20 | |
| *** ociuhandu has joined #openstack-nova | 01:20 | |
| *** ociuhandu has quit IRC | 01:30 | |
| *** brinzhang_ has joined #openstack-nova | 01:42 | |
| *** brinzhang0 has quit IRC | 01:45 | |
| openstackgerrit | Arthur Dayne proposed openstack/nova master: Libvirt: fix the bug volume attaching failure when the VM domain is in transient status. https://review.opendev.org/c/openstack/nova/+/769074 | 01:48 |
|---|---|---|
| *** ociuhandu has joined #openstack-nova | 02:00 | |
| *** ociuhandu has quit IRC | 02:05 | |
| *** ociuhandu has joined #openstack-nova | 02:06 | |
| *** ociuhandu has quit IRC | 02:11 | |
| *** ociuhandu has joined #openstack-nova | 02:12 | |
| *** ociuhandu has quit IRC | 02:17 | |
| *** ociuhandu has joined #openstack-nova | 02:18 | |
| *** ociuhandu has quit IRC | 02:23 | |
| *** ociuhandu has joined #openstack-nova | 02:24 | |
| *** ociuhandu has quit IRC | 02:29 | |
| *** ociuhandu has joined #openstack-nova | 02:30 | |
| *** ociuhandu has quit IRC | 02:35 | |
| *** ociuhandu has joined #openstack-nova | 02:36 | |
| *** ociuhandu has quit IRC | 02:40 | |
| *** zzzeek has quit IRC | 03:25 | |
| *** zzzeek has joined #openstack-nova | 03:29 | |
| *** mkrai has joined #openstack-nova | 03:43 | |
| *** LinPeiWen has quit IRC | 03:52 | |
| *** ociuhandu has joined #openstack-nova | 03:52 | |
| *** ratailor has joined #openstack-nova | 03:56 | |
| *** ociuhandu has quit IRC | 03:57 | |
| *** psachin has joined #openstack-nova | 04:00 | |
| *** bhagyashris has joined #openstack-nova | 04:19 | |
| *** abhishekk has joined #openstack-nova | 04:28 | |
| openstackgerrit | Takashi Natsume proposed openstack/python-novaclient master: Fix undesirable raw Python error https://review.opendev.org/c/openstack/python-novaclient/+/769082 | 04:50 |
| *** mkrai has quit IRC | 04:57 | |
| *** mkrai has joined #openstack-nova | 05:31 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-nova | 05:33 | |
| *** ociuhandu has joined #openstack-nova | 05:40 | |
| *** ociuhandu has quit IRC | 05:47 | |
| *** fnordahl has joined #openstack-nova | 06:01 | |
| *** vishalmanchanda has joined #openstack-nova | 06:14 | |
| *** macz_ has joined #openstack-nova | 06:32 | |
| *** macz_ has quit IRC | 06:37 | |
| *** whoami-rajat__ has joined #openstack-nova | 06:39 | |
| *** songwenping_ has joined #openstack-nova | 06:58 | |
| *** ociuhandu has joined #openstack-nova | 07:02 | |
| *** ociuhandu has quit IRC | 07:07 | |
| *** LinPeiWen has joined #openstack-nova | 07:15 | |
| *** mkrai has quit IRC | 07:24 | |
| *** cgoncalves has joined #openstack-nova | 07:27 | |
| *** mkrai has joined #openstack-nova | 07:37 | |
| *** ralonsoh has joined #openstack-nova | 07:49 | |
| *** gibi_pto is now known as gibi | 07:58 | |
| gibi | good morning | 07:58 |
| bauzas | good morning Nova and happy new year ! | 08:04 |
| *** slaweq has joined #openstack-nova | 08:09 | |
| *** tesseract has joined #openstack-nova | 08:16 | |
| *** xek has joined #openstack-nova | 08:32 | |
| *** vishalmanchanda has quit IRC | 08:34 | |
| *** lee1 has joined #openstack-nova | 08:35 | |
| *** tosky has joined #openstack-nova | 08:35 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Deploy novnc from source in nova-next https://review.opendev.org/c/openstack/nova/+/768334 | 08:43 |
| gibi | bauzas: happy new year to you too | 08:48 |
| *** ociuhandu has joined #openstack-nova | 08:50 | |
| *** rpittau|afk is now known as rpittau | 08:51 | |
| *** mkrai has quit IRC | 08:52 | |
| *** lee1 is now known as lyarwood | 08:52 | |
| lyarwood | morning all and a happy new year | 08:52 |
| *** kashyap has joined #openstack-nova | 08:52 | |
| gibi | lyarwood: happy new year to you too | 08:52 |
| *** andrewbonney has joined #openstack-nova | 08:53 | |
| *** brinzhang0 has joined #openstack-nova | 08:56 | |
| *** ociuhandu has quit IRC | 08:56 | |
| *** brinzhang_ has quit IRC | 08:59 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Deploy novnc from source in nova-next https://review.opendev.org/c/openstack/nova/+/768334 | 09:04 |
| stephenfin | happy new year | 09:07 |
| gibi | stephenfin: o/ | 09:09 |
| kashyap | New year's greetings to you all too | 09:10 |
| bauzas | maybe I should tell "Better New Year" instead of "Happy new year" ? | 09:11 |
| gibi | kashyap: \o | 09:12 |
| gibi | bauzas: or simply just different new year :D | 09:12 |
| lyarwood | bauzas: that's a pretty low bar after 2020 ;) | 09:12 |
| kashyap | bauzas: I ended my new year w/ this video: https://www.youtube.com/watch?v=fn3KWM1kuAw | 09:13 |
| kashyap | Err, the old year, I mean. | 09:13 |
| bauzas | kashyap: saw it ;) <3 | 09:13 |
| lyarwood | I for one welcome our new dad dancing overlords | 09:15 |
| * bauzas wonders if Asimov would love this | 09:15 | |
| *** lemko has quit IRC | 09:19 | |
| *** lemko has joined #openstack-nova | 09:19 | |
| stephenfin | gibi: What do we want to do about this? https://review.opendev.org/c/openstack/python-novaclient/+/769066/ | 09:20 |
| openstackgerrit | Merged openstack/osc-placement master: remove unicode from code https://review.opendev.org/c/openstack/osc-placement/+/769059 | 09:28 |
| *** mkrai has joined #openstack-nova | 09:29 | |
| openstackgerrit | Merged openstack/osc-placement master: remove unicode from code https://review.opendev.org/c/openstack/osc-placement/+/769062 | 09:30 |
| *** slaweq has quit IRC | 09:31 | |
| gibi | stephenfin: looking | 09:32 |
| *** ociuhandu has joined #openstack-nova | 09:32 | |
| stephenfin | Ta. Historically, mriedem had removed the novaclient features once the nova API was removed, however, since novaclient is used as a library by things like OSC, I'm not sure if we want to keep doing that | 09:32 |
| *** slaweq has joined #openstack-nova | 09:33 | |
| gibi | stephenfin: I think it is OK to keep the nova client as is even for remove APIs as this code is not a big burden to maintain and as you said a new novaclient might be used towards an old nova API endpoint | 09:34 |
| gibi | stephenfin: I don't even sure we want a deprecation message from the client itself as the server response is pretty obvious | 09:36 |
| stephenfin | Okay, cool. Let's keep it so | 09:37 |
| brinzhang0 | gibi, stephenfin: happy new year | 09:39 |
| brinzhang0 | gibi, stephenfin: did you have seen my news above? pinged you yesterday | 09:39 |
| gibi | brinzhang0: saw it but haven't looked the patches yet | 09:40 |
| stephenfin | same | 09:41 |
| brinzhang0 | gibi: ack, yeah, I just want to use v2.88 as the remove tenant_id microversion | 09:41 |
| *** hamalq has joined #openstack-nova | 09:42 | |
| *** ociuhandu has quit IRC | 09:42 | |
| gibi | brinzhang0: I have to see what are the other patches fighting for the same microversion and how ready these patches are | 09:42 |
| stephenfin | I was pretty sure we'd said we'd use 2.88 for the os-hypervisor work, since that's the smallest of the three items. I could have picked that up wrong though | 09:42 |
| stephenfin | It's okay though. I think we're all aware there are competing microversion changes and can be sure to finish one before starting on the next | 09:43 |
| bauzas | gibi: stephenfin: MHO about client changes is that we can remove old features if the API is no longer supporting it | 09:43 |
| bauzas | but then we would have to say that the minor version supported by the client would change | 09:44 |
| gibi | bauzas: sure we can, it is more the question if we want to remove. | 09:44 |
| brinzhang0 | stephenfin: yeah, if possible, I can help you to change, because the remove tenant_id patach depends on 2.88, and there are some place use v2.88 | 09:44 |
| stephenfin | bauzas: what do we do about things like OSC though, that depend on the novaclient library implementation | 09:44 |
| bauzas | stephenfin: the question is, should we still support v2.1 for the client ? | 09:45 |
| bauzas | if yes, we *can't* remove | 09:45 |
| stephenfin | brinzhang0: That's true for all changes. Whatever gets punted to 2.89 just has to be rebased and references to 2.88 updated to 2.89 | 09:46 |
| *** hamalq has quit IRC | 09:46 | |
| stephenfin | bauzas: OSC does, yes. It's supposed to support all OpenStack clouds, regardless of version (though I suspect we fall well short of that marker, and I also doubt many people are using new OSC with an e.g. Essex cloud) | 09:47 |
| brinzhang0 | stephenfin: yeah, after you can review these patch, and pls consider waht I want to say, thanks | 09:48 |
| brinzhang0 | and I am sure the remove tenant_id feature can be done in this release | 09:49 |
| brinzhang0 | but as bauzas and you talked above, how to deal with the OSC changes? | 09:50 |
| *** derekh has joined #openstack-nova | 09:50 | |
| brinzhang0 | ah, maybe it's(OSC) ok for the latest changes :D | 09:51 |
| bauzas | stephenfin: brinzhang0: see how version discovery works https://docs.openstack.org/api-guide/compute/microversions.html#version-discovery | 09:53 |
| bauzas | given our API provides minimum as 2.0 (or 2.1, can't remember exactly which one given they are the same) | 09:53 |
| bauzas | , | 09:53 |
| bauzas | then the client would have to support the minimum too | 09:54 |
| bauzas | here, 2.1 | 09:54 |
| bauzas | actually, wrong link, https://docs.openstack.org/api-guide/compute/microversions.html#client-interaction is better | 09:54 |
| *** ociuhandu has joined #openstack-nova | 09:56 | |
| *** ociuhandu has quit IRC | 09:56 | |
| *** hamalq has joined #openstack-nova | 09:57 | |
| *** ociuhandu has joined #openstack-nova | 09:57 | |
| gibi | so the conclusion could be that we can remove the CLI part but we cannot remove the lib part due to OSC | 09:59 |
| brinzhang0 | I think yes | 10:01 |
| *** hamalq has quit IRC | 10:02 | |
| gibi | stephenfin, bauzas: summarized my view in https://review.opendev.org/c/openstack/python-novaclient/+/769068 | 10:04 |
| bauzas | gibi: if we cut the support on the CLI, then we need to bump a major version when we release the client | 10:10 |
| bauzas | but that's ok IMHO | 10:10 |
| gibi | bauzas: correct, that is what the deprecation messages now describe in the patch | 10:10 |
| bauzas | ack | 10:11 |
| *** ociuhandu has quit IRC | 10:13 | |
| gibi | when (or even if ever) we cut a major version is open | 10:14 |
| bauzas | right | 10:14 |
| *** ociuhandu has joined #openstack-nova | 10:16 | |
| *** jangutter_ has joined #openstack-nova | 10:18 | |
| *** jangutter has quit IRC | 10:21 | |
| *** ociuhandu has quit IRC | 10:22 | |
| lyarwood | gibi / stephenfin ; https://review.opendev.org/c/openstack/nova/+/767590 - time to +W this? | 10:32 |
| openstackgerrit | Merged openstack/nova master: tests: Merge 'test_hypervisor_status' into 'test_hypervisors' https://review.opendev.org/c/openstack/nova/+/764039 | 10:33 |
| stephenfin | I think so | 10:33 |
| *** macz_ has joined #openstack-nova | 10:34 | |
| lyarwood | done | 10:35 |
| *** ociuhandu has joined #openstack-nova | 10:36 | |
| *** macz_ has quit IRC | 10:38 | |
| *** macz_ has joined #openstack-nova | 10:54 | |
| openstackgerrit | Merged openstack/nova master: Run the db migration tests in the same test worker https://review.opendev.org/c/openstack/nova/+/767590 | 10:56 |
| *** lpetrut has joined #openstack-nova | 10:59 | |
| *** macz_ has quit IRC | 10:59 | |
| gibi | lyarwood,stephenfin: thanks | 11:01 |
| gibi | I will keep monitoring those tests | 11:01 |
| gibi | as we did not solved the issue we just decreased the likelyhood | 11:02 |
| gibi | stephen's db migration compression patch series will be an even better fix when it lands | 11:02 |
| * gibi starts getting back the memories where he left of work last year | 11:03 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: Add nova-live-migration-ceph job https://review.opendev.org/c/openstack/nova/+/768466 | 11:11 |
| lyarwood | ^ getting close with this btw, there are some random volume backup test failures at the moment but the live migration and evacuation parts are working | 11:12 |
| sean-k-mooney | oh nice | 11:13 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Add regression test for bug #1908075 https://review.opendev.org/c/openstack/nova/+/766976 | 11:18 |
| openstack | bug 1908075 in OpenStack Compute (nova) "Nova allows a non-multiattach volume to be attached to multiple instances *if* its volume state is reset by an admin" [Undecided,New] https://launchpad.net/bugs/1908075 - Assigned to Lee Yarwood (lyarwood) | 11:18 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: api: Reject volume attach requests when an active bdm exists https://review.opendev.org/c/openstack/nova/+/768472 | 11:18 |
| gibi | stephenfin: I'm looking at the hypervisor series and wondering if the impl now in sync with the uptime change proposed in the yet unmerged spec additon here https://review.opendev.org/c/openstack/nova-specs/+/765797/1 ? | 11:19 |
| stephenfin | gibi: I need to refresh my memory, but it doesn't appear to be, no | 11:22 |
| stephenfin | I guess we need to approve https://review.opendev.org/c/openstack/nova-specs/+/765797/ and then I can rework. bauzas? lyarwood? | 11:23 |
| gibi | stephenfin: so pontentially we need either a new microversion for the uptime thing or a new PS for the current hypervisor patch | 11:23 |
| stephenfin | ideally the latter | 11:23 |
| stephenfin | that change hardly seems worth a microversion by itself | 11:24 |
| sean-k-mooney | i would be -1 on a microverion to readd uptime | 11:24 |
| sean-k-mooney | so ya either respin the patch or just drop uptime would but a second micorversion for it seam excessive | 11:25 |
| gibi | OK, I agree, let's merge that spec amendment and the respin the impl patch | 11:26 |
| gibi | bauzas: could you look at the small spec addition at https://review.opendev.org/c/openstack/nova-specs/+/765797/1 ? | 11:27 |
| * sean-k-mooney appears to have forgotten how to use gerrit and keeps clicking change instead of your | 11:27 | |
| sean-k-mooney | f**** the extra * is to account for my usual misspelling :) | 11:30 |
| sean-k-mooney | my main server has decided to stop negocition gigabit full duplex connectivity and keeps downgrading to 100mbps but that breaks all network connectivity | 11:32 |
| sean-k-mooney | its started doing this saturday after i rebooted it before i tried to install my gpu in it to do gpu passthough | 11:32 |
| sean-k-mooney | so now i dont have access to any of my dev envionments... | 11:33 |
| sean-k-mooney | also our gpu passtough works but nvida chave changed how they do the vm detachtion so the kvm hidden feature we supprot nolonger works for windows guests | 11:33 |
| sean-k-mooney | this came up on irc a a few weeks ago acctully but i can now confirm that our current code is not sufficent, if nova users upgrade there gpu driver it there gueste will eventually stop working which is lovely | 11:35 |
| lyarwood | sean-k-mooney:s~. | 11:49 |
| lyarwood | urgh sorry | 11:49 |
| lyarwood | stephenfin: sorry was afk, I'll take a look at that now | 11:50 |
| *** zzzeek has quit IRC | 11:50 | |
| *** zzzeek has joined #openstack-nova | 11:52 | |
| gibi | another easy spec update needs anther core eyes https://review.opendev.org/c/openstack/nova-specs/+/768803 | 11:53 |
| *** ociuhandu has quit IRC | 12:01 | |
| stephenfin | gibi: done | 12:16 |
| gibi | thanks | 12:16 |
| *** slaweq has quit IRC | 12:18 | |
| *** slaweq has joined #openstack-nova | 12:20 | |
| openstackgerrit | Merged openstack/nova-specs master: [Trivial] Clarify the deprecated apis in *Proposed change* https://review.opendev.org/c/openstack/nova-specs/+/768803 | 12:29 |
| *** ociuhandu has joined #openstack-nova | 12:32 | |
| *** ociuhandu has quit IRC | 12:37 | |
| *** mkrai has quit IRC | 12:38 | |
| bauzas | gibi: stephenfin: sure, will look at https://review.opendev.org/c/openstack/nova-specs/+/765797/ | 12:41 |
| gibi | bauzas: thanks | 12:47 |
| bauzas | done, +Wd | 12:48 |
| gibi | stephenfin: you are good to go for a respin of your hypervisor patch | 12:48 |
| *** viks____ has joined #openstack-nova | 12:48 | |
| gibi | meanwhile I looked at the vncconsole password patch (another microversion bump) but that is not ready yet, I still have serious comments there (not published yet) | 12:49 |
| gibi | so right now the hypervisor patch feels the closest for the 2.88 microversion | 12:49 |
| gibi | I will look at the tenant_id patch series too | 12:49 |
| openstackgerrit | Merged openstack/nova-specs master: Update modernize-os-hypervisors-api spec, pt. 3 https://review.opendev.org/c/openstack/nova-specs/+/765797 | 12:54 |
| *** Luzi has joined #openstack-nova | 12:55 | |
| *** macz_ has joined #openstack-nova | 12:55 | |
| *** macz_ has quit IRC | 13:00 | |
| *** mkrai has joined #openstack-nova | 13:03 | |
| *** eharney has joined #openstack-nova | 13:22 | |
| *** ratailor has quit IRC | 13:23 | |
| *** haleyb has quit IRC | 13:24 | |
| *** haleyb has joined #openstack-nova | 13:27 | |
| *** eharney has quit IRC | 13:29 | |
| *** ociuhandu has joined #openstack-nova | 13:30 | |
| *** tosky has quit IRC | 13:30 | |
| *** tosky has joined #openstack-nova | 13:30 | |
| *** eharney has joined #openstack-nova | 13:31 | |
| *** artom has joined #openstack-nova | 13:39 | |
| stephenfin | gibi: Yup, I'll do it now | 13:45 |
| gibi | stephenfin: cool | 13:45 |
| *** dave-mccowan has joined #openstack-nova | 13:53 | |
| *** tbachman has quit IRC | 13:56 | |
| *** martinkennelly has joined #openstack-nova | 14:07 | |
| *** songwenping_ has quit IRC | 14:09 | |
| *** lbragstad has joined #openstack-nova | 14:09 | |
| *** zoharm has joined #openstack-nova | 14:10 | |
| zoharm | hi, regarding proposed nvme healing agent feature: https://review.opendev.org/c/openstack/cinder-specs/+/766732 | 14:12 |
| zoharm | we would like to ask, what is the right way to spawn it? we had it proposed to use python-daemon python package for maintaining an independent process. Is it ok to add that dependency? | 14:13 |
| zoharm | if using python-daemon is not ok, geguileo suggested previously to look at privsep daemon and see how it is spawned. which way in there is better? https://opendev.org/openstack/oslo.privsep/src/branch/master/oslo_privsep/daemon.py line 296 or 336 - we would like to have only one agent process per host. any pointers / details would be useful here. Thank you! | 14:13 |
| *** Luzi has quit IRC | 14:14 | |
| *** nweinber has joined #openstack-nova | 14:19 | |
| kashyap | stephenfin: sean-k-mooney: Remind me again, Nova supports CPU pinnning to host core and to host NUMA node, both, yes? | 14:26 |
| *** lbragstad has quit IRC | 14:27 | |
| stephenfin | yes | 14:27 |
| sean-k-mooney | kashyap: kindo of it depend on what you mean by host numa nodes | 14:27 |
| gibi | zoharm: the nova services (daemons) using oslo_services to spawn the daemon process https://docs.openstack.org/oslo.service/latest/ | 14:27 |
| *** mkrai has quit IRC | 14:27 | |
| stephenfin | hw:cpu_policy=dedicated to pin to individual host cores | 14:27 |
| sean-k-mooney | we support soft pinning so a vm will float over the cores of a singel numa node if that is what you mean | 14:27 |
| sean-k-mooney | and we obvioulsy support direct 1:1 gust cpu to host cpu pinning too | 14:28 |
| kashyap | I'm just asked if "CPU Pinning to host core" is supported or not | 14:28 |
| sean-k-mooney | hw:numa_nodes=1 to pin the vm to just one host numa node | 14:28 |
| sean-k-mooney | kashyap: you cant choose the core but yes | 14:29 |
| kashyap | Okay; thx | 14:29 |
| zoharm | gibi thank you, we want to do this from os-brick, is that ok? | 14:29 |
| zoharm | also, any chance for a pointer at an example of how to use it? | 14:29 |
| sean-k-mooney | zoharm: not really | 14:29 |
| sean-k-mooney | so os-brick is shared between nova and cinder | 14:30 |
| sean-k-mooney | both run as deams already and invoke the lib | 14:30 |
| *** tbachman has joined #openstack-nova | 14:30 | |
| kashyap | stephenfin: sean-k-mooney: One more, the above of what I asked is supported from Queens, is it? | 14:30 |
| sean-k-mooney | you proably should not be sapawning a second deamon out of the os-brick lib | 14:30 |
| zoharm | right, but we need an entry point for the agent/daemon as per the spec, and the connector is the place decided on | 14:30 |
| zoharm | do you have an alternative? | 14:31 |
| *** lbragstad has joined #openstack-nova | 14:31 | |
| bnemec | zoharm: If you want to know more about how privsep works, you can read about it in https://github.com/openstack/oslo.privsep/blob/master/oslo_privsep/daemon.py#L15 | 14:31 |
| sean-k-mooney | zoharm: a better apprcoh would be to have a seperate console script entry point that is run seperatly as a deamon and no tspawned out of eithet the nova-compute process or the cinder-volume process | 14:31 |
| zoharm | what is a console script? | 14:32 |
| *** lbragstad has quit IRC | 14:32 | |
| zoharm | we are aware of the option of having an external script, but the goal here is to get it into openstack | 14:32 |
| sean-k-mooney | zoharm: its a setuptool entroy point that creates a indepenten binary that you can run in the termal | 14:32 |
| *** lbragstad has joined #openstack-nova | 14:32 | |
| zoharm | ok that sounds interesting, can you give me a reference / example? | 14:33 |
| sean-k-mooney | sure on sec | 14:33 |
| zoharm | and proposal is not to run it from terminal, but have openstack run it only when needed. | 14:33 |
| gibi | if this agent is just executes a periodic task then I guess such periodic task can be added to either the cinder or the nova agent to call a function in the os-brick lib | 14:33 |
| sean-k-mooney | the nova compute agent itslef is one https://github.com/openstack/nova/blob/master/setup.cfg#L73 | 14:33 |
| sean-k-mooney | that creats a binary that runs https://github.com/openstack/nova/blob/master/nova/cmd/compute.py | 14:34 |
| sean-k-mooney | which uses oslo service to run the nova compute applicatoin as a deamon | 14:34 |
| zoharm | i see, but that would be a whole new service, which the operator would need to launch independently | 14:34 |
| sean-k-mooney | yes | 14:34 |
| sean-k-mooney | privsep works slightly differently | 14:35 |
| zoharm | this is not the proposal, it is just as good as an external script | 14:35 |
| sean-k-mooney | in that we launch a different process in one of 2 ways | 14:35 |
| sean-k-mooney | either we fork a seperate process using the privsep helper and typicaly sudo | 14:35 |
| zoharm | i think having completley new service to something that is tightly coupled to an os-brick connector is overkill both operationally and in code | 14:35 |
| sean-k-mooney | or we run nova with elevetated prviages for the process and drop privlates on nova-compute | 14:35 |
| zoharm | ok, can we do that for this agent too? | 14:36 |
| zoharm | if we can't use python-daemon | 14:36 |
| sean-k-mooney | possibly but it depned on what it does i have not read the spec | 14:36 |
| zoharm | it is ran only when an NVMe volume is connected to a host | 14:36 |
| zoharm | it monitors the connection, and if its a replicated volume, self heals it | 14:37 |
| sean-k-mooney | but then it continues running | 14:37 |
| sean-k-mooney | basicaly what are the lifetime sematince you want | 14:37 |
| zoharm | yes, but we can add a mechanism to self kill when there are not more nvme connections | 14:37 |
| sean-k-mooney | should it continue running if nova-compute is stopped but the vm is still running | 14:37 |
| zoharm | keep running as long as there are managed nvme volumes connected to by connector | 14:37 |
| zoharm | yes | 14:37 |
| *** tbachman has quit IRC | 14:38 | |
| zoharm | if vm is connected to nvme volume via the nvme connector | 14:38 |
| sean-k-mooney | i think right now privsep will exit in that case wehn the last clinet on the privsep socket disconnects | 14:38 |
| sean-k-mooney | privsep usins a unix socket to comunicate between the two processes | 14:38 |
| zoharm | right | 14:38 |
| zoharm | that is a good point, that all process can exit but VMs can keep running and consuming connected volumes... | 14:39 |
| zoharm | looks like another pain point | 14:39 |
| sean-k-mooney | that is one that comes up during upgrades | 14:39 |
| zoharm | we can argue that it is ok to have this agent disabled for short time during upgrade, as long as something turns it back on | 14:39 |
| sean-k-mooney | well the thing that turns it back on would be initalisin os-brick | 14:40 |
| sean-k-mooney | when nova-compute is started after the upgrade | 14:40 |
| sean-k-mooney | if the lifetime of this seperate process could be made the same as the process the spawned ti then mimicing privsep works | 14:41 |
| zoharm | ok, so in this case in connector init, scan for nvme volumes that are managed by this and if there are launch the agent | 14:41 |
| sean-k-mooney | the trigger from the nova side would likely be init_host | 14:41 |
| zoharm | what if one process spawned it, then another process also wants it spawned, and then first process was permanently stopped | 14:41 |
| zoharm | init_host sounds good, that is in os-brick? | 14:42 |
| sean-k-mooney | where we loop over all the vms and check that there networking and presumabley volume connection are set up | 14:42 |
| sean-k-mooney | no in nova | 14:42 |
| sean-k-mooney | but i think we call into os-brick | 14:42 |
| sean-k-mooney | we call into os-vif to ensure the vifs are pluuged in init host | 14:42 |
| sean-k-mooney | i expect we do the same for os-brick | 14:42 |
| zoharm | ok, we want to keep this change out of nova for now since its specifically deals with only volumes | 14:43 |
| zoharm | unless you think its better to add it to nova | 14:43 |
| sean-k-mooney | dose os-brick have an inialise function? | 14:44 |
| zoharm | for volume connection? | 14:44 |
| zoharm | there is init for the connector which is called when its loaded up for first time | 14:44 |
| sean-k-mooney | no for the lib in general | 14:44 |
| *** zzzeek has quit IRC | 14:44 | |
| sean-k-mooney | os-vif has https://github.com/openstack/os-vif/blob/master/os_vif/__init__.py#L24 | 14:44 |
| sean-k-mooney | which is called when the compute agent start to tell os vif to load all its plugin dirvers | 14:45 |
| zoharm | i dont know, but i think that is not best place to put this re-init agent entrypoint | 14:45 |
| zoharm | do you know if nova ends up calling os-brick connect_volume during host_init? | 14:45 |
| zoharm | if it does, then that solves everything for us because connect_volume is currently the entry point | 14:45 |
| *** cap has quit IRC | 14:46 | |
| zoharm | we dont want to launch this agent when os-brick or connector loads (since all connectors / os-brick are loaded for every service and we dont want this agent running everywhere, only where nvme volumes are connected via the connector) | 14:46 |
| sean-k-mooney | im just reading https://github.com/openstack/nova/blob/b0f241e5425c99866223bae4b404a4aa1abdfddf/nova/compute/manager.py#L956 now | 14:46 |
| sean-k-mooney | for each instance we call _init_instance in init_host | 14:46 |
| *** zzzeek has joined #openstack-nova | 14:47 | |
| sean-k-mooney | this is where we ensure the network interfaces are set up https://github.com/openstack/nova/blob/b0f241e5425c99866223bae4b404a4aa1abdfddf/nova/compute/manager.py#L1137 | 14:47 |
| *** cap has joined #openstack-nova | 14:47 | |
| zoharm | im looking through it too now, found _init_volume_connection but it is not called during init_host | 14:48 |
| zoharm | back to python-daemon, do you think it is not ok to propose this new depedency? | 14:48 |
| sean-k-mooney | you could but do you need too | 14:51 |
| sean-k-mooney | what benifit will it provide | 14:51 |
| zoharm | it will allow to run this agent as a single independent process that will keep running regardless of the different services that may spawn it | 14:56 |
| sean-k-mooney | ok but then how to you interact with it and manage its lifetime | 14:57 |
| zoharm | and we can have it terminate itself when no connections are left if needed | 14:57 |
| zoharm | we dont need to interact with it much, the interaction is mostly spawning it, and then it will be calling volume backend API for its functionality | 14:58 |
| sean-k-mooney | based on what? i will need to have some set of input and know when a nvme volume attachemt is made | 14:59 |
| zoharm | once it runs it basically just monitors local nvme connections belonging to it (it can tell them apart) and calling volume backend if necessary | 14:59 |
| zoharm | the nvme connector connect_volume is called when nvme volume attachment is made, and that is the entry point where we ensure the agent is running | 15:00 |
| zoharm | certain agent implementation is vendor specific, but it can tell by reading nvme device related dev paths information about where it came from | 15:01 |
| zoharm | and backend provides metadata | 15:01 |
| openstackgerrit | Merged openstack/python-novaclient master: Fix a functional test for 'nova agent-list' https://review.opendev.org/c/openstack/python-novaclient/+/769066 | 15:01 |
| zoharm | basically what the agent does is reconcile metadata from backend with physical connection state on host (this is mostly for mdraid replicated volumes) | 15:02 |
| sean-k-mooney | im reading the spec now but i dont think this agent really fits in the project scope fo os-brick | 15:02 |
| zoharm | i understand your concern, this is where we settled on putting it for now | 15:03 |
| sean-k-mooney | it really does seam like an indepenet service similar to multipathd rhater then a capablity a libviary shoudl be providing | 15:03 |
| zoharm | good point | 15:03 |
| sean-k-mooney | well im concerend that this is a possibel ddos vector for the cinder api or storage backend depending on how it monitors the conenctiions and what api calls its makink | 15:03 |
| sean-k-mooney | e.g. if i have a 1000 node deployment and i install this on all notes how will that work | 15:04 |
| sean-k-mooney | this agent will spawn a monitoring task which will repeat | 15:05 |
| sean-k-mooney | periodically. | 15:05 |
| sean-k-mooney | does it need to be an agent or could it be a perodic task defined in os-brick | 15:05 |
| sean-k-mooney | that is then elecitly run by the consumer of os-brick | 15:05 |
| *** mkrai has joined #openstack-nova | 15:06 | |
| sean-k-mooney | "One key problem that would need to be addressed by this selection is a scenario | 15:06 |
| sean-k-mooney | where compute service goes down, while the VMs continue operating (and their | 15:06 |
| sean-k-mooney | volumes remain attached) - we don't want to lose this agent in this case. | 15:06 |
| sean-k-mooney | " | 15:06 |
| sean-k-mooney | that is the main motivator for a seperate deamon process right | 15:06 |
| sean-k-mooney | https://review.opendev.org/c/openstack/cinder-specs/+/766732/12/specs/wallaby/nvme-agent.rst#71 | 15:07 |
| *** lpetrut has quit IRC | 15:10 | |
| zoharm | right | 15:10 |
| zoharm | open to all suggestions here | 15:11 |
| zoharm | we want it to be a periodic task that is launched only when nvme volumes are connected by the nvme connector | 15:14 |
| *** swright has joined #openstack-nova | 15:17 | |
| *** jangutter_ has quit IRC | 15:20 | |
| *** jangutter has joined #openstack-nova | 15:20 | |
| kashyap | stephenfin: One more, sorry: vTPM 2.0 and TPM passthrough -- both are supported in upstream Train, yeah? | 15:28 |
| stephenfin | kashyap: no, vTPM is supported since Victoria. TPM passthrough is not supported afaik | 15:29 |
| sean-k-mooney | TPM passtough can only be done via pci passtough | 15:29 |
| sean-k-mooney | but that is for stateless devices only | 15:30 |
| sean-k-mooney | so using it wiwth a tpm which has state would be invalid | 15:30 |
| kashyap | stephenfin: sean-k-mooney: Nod; thx. Context on these random questions, I got a ping elsewhere about these, so trying to fill in details | 15:30 |
| sean-k-mooney | if cyborg had a TPM dirver it could provide a tpm to us and do the required cleaning but they dont as far as i know | 15:31 |
| kashyap | stephenfin: I have one more: I take it, your recent virtio device addition work means, we now support all these devices, corecct? — | 15:31 |
| *** psachin has quit IRC | 15:31 | |
| kashyap | virtio-serial, virtio-vga, virtio-balloon, virtio-input, virtio-keyboard, virtio-mouse, virtio-tablet | 15:32 |
| kashyap | Perhaps not the -mouse yet | 15:32 |
| sean-k-mooney | you conflating different things | 15:32 |
| kashyap | sean-k-mooney: What am I conflating? | 15:32 |
| sean-k-mooney | virtio-serial has been supproted for year with the serial-console | 15:32 |
| sean-k-mooney | virtio-ballon cant be turned off yet so you always get it because libvirt adds one we agreeed to change that | 15:33 |
| kashyap | sean-k-mooney: Yeah; I know some of them have existed forever; all I care about is - does the latest upstream release support all the above devices or not | 15:33 |
| kashyap | Right. (I know -balloon is supported; and we recommend it to turn off for especially the real-time cases) | 15:33 |
| sean-k-mooney | virti-input i think is not but im not sure | 15:33 |
| sean-k-mooney | kashyap: yes but right now you cant turn it off | 15:33 |
| kashyap | (And I recall reviewing the -keyboard and -input patches from Stephen, recently) | 15:33 |
| sean-k-mooney | you uses to be able to by disbleing mermoy stat monitoring | 15:34 |
| sean-k-mooney | but at some point libvirt started always adding it | 15:34 |
| sean-k-mooney | kashyap: im just sure what -input is | 15:34 |
| sean-k-mooney | -tablet is definetly supported | 15:35 |
| kashyap | sean-k-mooney: You mean not sure? | 15:35 |
| kashyap | https://www.kraxel.org/blog/2016/09/using-virtio-input-with-libvirt/ | 15:35 |
| sean-k-mooney | sorry yes | 15:35 |
| sean-k-mooney | ya ok so stephenfin was changing that but i havent revied it in a while | 15:35 |
| sean-k-mooney | but he wrote a nice table in the commit message | 15:36 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/756552 | 15:36 |
| sean-k-mooney | oh the table was in https://review.opendev.org/c/openstack/nova/+/756551/3 | 15:37 |
| kashyap | sean-k-mooney: Right; inded, I was looking at that table a minute ago | 15:40 |
| kashyap | Okay; thanks. I may come back with more "does upstream support X?" questions, if I can't figure it out myself :-) | 15:41 |
| *** macz_ has joined #openstack-nova | 15:53 | |
| *** macz_ has quit IRC | 15:53 | |
| *** macz_ has joined #openstack-nova | 15:54 | |
| *** ociuhandu has quit IRC | 16:08 | |
| *** LinPeiWen has quit IRC | 16:15 | |
| *** dklyle has joined #openstack-nova | 16:18 | |
| *** slaweq has quit IRC | 16:19 | |
| *** slaweq has joined #openstack-nova | 16:20 | |
| *** zzzeek has quit IRC | 16:39 | |
| *** zzzeek has joined #openstack-nova | 16:40 | |
| *** zzzeek has quit IRC | 16:46 | |
| *** ociuhandu has joined #openstack-nova | 16:46 | |
| *** zzzeek has joined #openstack-nova | 16:47 | |
| *** gyee has joined #openstack-nova | 16:50 | |
| *** ociuhandu has quit IRC | 16:51 | |
| *** hamalq has joined #openstack-nova | 16:53 | |
| *** mkrai has quit IRC | 17:02 | |
| *** qqmber has joined #openstack-nova | 17:04 | |
| qqmber | Hi! How can I make a auth token to last longer? Because when I try to create a snapshot of a big instance takes around 120 minutes and fails because a 401 error, due a timeout ( auth expired). I tried to configure "expiration_time = 7200" in [cache] section, but no luck.... | 17:06 |
| qqmber | in keystone.conf | 17:06 |
| sean-k-mooney | you can use service tokens | 17:08 |
| sean-k-mooney | but it should not take 120mins to spawn | 17:08 |
| qqmber | hmmm... sorry, I don't get it.. what do you mean? | 17:09 |
| qqmber | How can I use service tokens? Never did before.. | 17:10 |
| *** zzzeek has quit IRC | 17:13 | |
| *** slaweq has quit IRC | 17:13 | |
| *** zzzeek has joined #openstack-nova | 17:14 | |
| sean-k-mooney | you have to configure them in the nova config | 17:14 |
| sean-k-mooney | one sec | 17:14 |
| *** rpittau is now known as rpittau|afk | 17:14 | |
| *** hamalq_ has joined #openstack-nova | 17:14 | |
| sean-k-mooney | https://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/use-service-tokens.html | 17:14 |
| *** slaweq has joined #openstack-nova | 17:15 | |
| *** hamalq has quit IRC | 17:17 | |
| *** zenkuro has quit IRC | 17:25 | |
| lyarwood | qqmber: https://docs.openstack.org/cinder/latest/configuration/block-storage/service-token.html might be useful assuming the instance is boot from volume? | 17:27 |
| qqmber | sean-k-mooney: thanks | 17:28 |
| qqmber | lyarwood: it does not boot from volume | 17:28 |
| *** zenkuro has joined #openstack-nova | 17:32 | |
| qqmber | sean-k-mooney: actually I'm in Pike.. and as says there.. is a change proposal... so.. I guess I can't use it in Pike :( | 17:38 |
| *** tesseract has quit IRC | 17:44 | |
| sean-k-mooney | qqmber: it was added in pike | 17:48 |
| sean-k-mooney | so that the first release you cna use it in | 17:48 |
| qqmber | oh, cool. thx | 17:49 |
| sean-k-mooney | qqmber: you should still look in to why it took 2 hours to boot | 17:55 |
| qqmber | because it's a snapshot of 160GB and the host is pretty busy | 17:56 |
| qqmber | that's all | 17:56 |
| sean-k-mooney | unless you have reallly slow HDD and are copying TBs of root disk over 1G links it should not be that slow | 17:56 |
| sean-k-mooney | i see | 17:56 |
| sean-k-mooney | well if you know why i guess that is ok | 17:56 |
| qqmber | sean-k-mooney: that's why I'm doing the snapshot, to take off those instances to a new cluster | 17:57 |
| qqmber | a faster one | 17:57 |
| *** derekh has quit IRC | 18:02 | |
| *** andrewbonney has quit IRC | 18:03 | |
| *** zzzeek has quit IRC | 18:26 | |
| *** zoharm has quit IRC | 18:27 | |
| *** zzzeek has joined #openstack-nova | 18:28 | |
| *** zenkuro has quit IRC | 18:52 | |
| *** zzzeek has quit IRC | 19:19 | |
| *** zzzeek has joined #openstack-nova | 19:20 | |
| *** zzzeek has quit IRC | 19:27 | |
| *** zzzeek has joined #openstack-nova | 19:28 | |
| *** qqmber has quit IRC | 19:43 | |
| *** _mlavalle3 has joined #openstack-nova | 20:16 | |
| *** _mlavalle_2 has quit IRC | 20:17 | |
| *** ociuhandu has joined #openstack-nova | 20:18 | |
| *** dave-mccowan has quit IRC | 20:40 | |
| *** ociuhandu has quit IRC | 20:40 | |
| *** ociuhandu has joined #openstack-nova | 20:43 | |
| *** ociuhandu has quit IRC | 20:49 | |
| *** whoami-rajat__ has quit IRC | 21:01 | |
| *** ociuhandu has joined #openstack-nova | 21:15 | |
| *** slaweq has quit IRC | 21:24 | |
| *** ralonsoh has quit IRC | 21:28 | |
| *** ociuhandu has quit IRC | 21:52 | |
| *** xek has quit IRC | 22:13 | |
| *** nweinber has quit IRC | 22:13 | |
| *** hack-char has quit IRC | 22:25 | |
| *** hack-char has joined #openstack-nova | 22:25 | |
| *** tkajinam has joined #openstack-nova | 23:00 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!