| *** bbowen has joined #openstack-nova | 00:01 | |
| *** macz_ has joined #openstack-nova | 00:39 | |
| *** brinzhang has joined #openstack-nova | 00:41 | |
| *** tosky has quit IRC | 00:41 | |
| *** macz_ has quit IRC | 00:44 | |
| *** ociuhandu has joined #openstack-nova | 00:48 | |
| *** ociuhandu has quit IRC | 00:52 | |
| *** spatel has joined #openstack-nova | 00:54 | |
| *** spatel has quit IRC | 01:23 | |
| *** sapd1 has joined #openstack-nova | 01:28 | |
| *** macz_ has joined #openstack-nova | 01:30 | |
| *** Liang__ has joined #openstack-nova | 01:34 | |
| *** macz_ has quit IRC | 01:34 | |
| *** LinPeiWen has joined #openstack-nova | 01:38 | |
| *** deke997 has joined #openstack-nova | 01:58 | |
| deke997 | anyone here know the latest status of spice integration? | 02:05 |
|---|---|---|
| *** sapd1 has quit IRC | 02:18 | |
| *** tbachman has joined #openstack-nova | 02:20 | |
| *** deke997 has quit IRC | 03:15 | |
| *** mtreinish has quit IRC | 03:15 | |
| *** k_mouza has joined #openstack-nova | 03:20 | |
| *** k_mouza has quit IRC | 03:24 | |
| *** dklyle has joined #openstack-nova | 03:24 | |
| *** psachin has joined #openstack-nova | 03:29 | |
| *** LinPeiWen has quit IRC | 03:38 | |
| *** slaweq has quit IRC | 03:40 | |
| *** benj_ has quit IRC | 03:41 | |
| *** slaweq has joined #openstack-nova | 03:42 | |
| *** benj_ has joined #openstack-nova | 03:45 | |
| *** xinranwang has joined #openstack-nova | 03:52 | |
| *** macz_ has joined #openstack-nova | 03:54 | |
| *** amotoki has quit IRC | 03:56 | |
| *** amotoki has joined #openstack-nova | 03:57 | |
| *** macz_ has quit IRC | 03:58 | |
| *** Liang__ has quit IRC | 03:59 | |
| *** dklyle has quit IRC | 04:00 | |
| *** Liang__ has joined #openstack-nova | 04:01 | |
| *** mkrai has joined #openstack-nova | 04:19 | |
| *** LinPeiWen has joined #openstack-nova | 04:26 | |
| *** LinPeiWen has quit IRC | 05:16 | |
| *** ratailor has joined #openstack-nova | 05:17 | |
| *** psachin has quit IRC | 05:23 | |
| *** vishalmanchanda has joined #openstack-nova | 05:28 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-nova | 05:33 | |
| xinranwang | gibi: Hi gibi, thanks for your comments for smartnic spec. For bandiwidth qos, can I assume that the co-existance is not support at the first step. | 05:48 |
| xinranwang | maybe we can add this restirction into document for the first phase | 05:51 |
| xinranwang | Is it acceptable? | 05:52 |
| *** zzzeek has quit IRC | 05:58 | |
| *** zzzeek has joined #openstack-nova | 06:00 | |
| *** rcernin has quit IRC | 06:48 | |
| *** rcernin has joined #openstack-nova | 06:49 | |
| *** LinPeiWen has joined #openstack-nova | 06:50 | |
| *** whoami-rajat__ has joined #openstack-nova | 06:50 | |
| *** viks____ has joined #openstack-nova | 06:57 | |
| *** lpetrut has joined #openstack-nova | 07:11 | |
| *** LinPeiWen has quit IRC | 07:17 | |
| *** sapd1 has joined #openstack-nova | 07:19 | |
| *** rcernin has quit IRC | 07:38 | |
| *** ccstone has quit IRC | 07:40 | |
| *** eandersson has quit IRC | 07:40 | |
| *** ccstone has joined #openstack-nova | 07:41 | |
| *** eandersson has joined #openstack-nova | 07:41 | |
| gibi | xinranwang: yes, such limitation is acceptable for me | 07:50 |
| *** iurygregory has quit IRC | 07:58 | |
| xinranwang | gibi: great, thanks. I will update the spec soon.:) | 07:59 |
| gibi | I guess you mean coexistens for the same physical device. | 08:00 |
| *** LinPeiWen has joined #openstack-nova | 08:08 | |
| *** tesseract has joined #openstack-nova | 08:08 | |
| *** andrewbonney has joined #openstack-nova | 08:10 | |
| *** rpittau|afk is now known as rpittau | 08:30 | |
| *** iurygregory has joined #openstack-nova | 08:30 | |
| *** psachin has joined #openstack-nova | 08:32 | |
| *** sapd1 has quit IRC | 08:33 | |
| *** psachin has quit IRC | 08:37 | |
| *** vishalmanchanda has quit IRC | 08:37 | |
| *** Yumeng has joined #openstack-nova | 08:42 | |
| *** martinkennelly has joined #openstack-nova | 08:53 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Reintroduce nova-dsvm-multinode-base https://review.opendev.org/761870 | 08:54 |
| bauzas | good morning Nova | 09:05 |
| *** ociuhandu has joined #openstack-nova | 09:08 | |
| gibi | bauzas: o/ | 09:12 |
| *** ociuhandu has quit IRC | 09:18 | |
| *** Liang__ has quit IRC | 09:24 | |
| *** vishalmanchanda has joined #openstack-nova | 09:25 | |
| *** Liang__ has joined #openstack-nova | 09:25 | |
| *** ociuhandu has joined #openstack-nova | 09:26 | |
| *** ociuhandu has quit IRC | 09:28 | |
| *** ociuhandu has joined #openstack-nova | 09:28 | |
| *** Yumeng has quit IRC | 09:29 | |
| *** ralonsoh has joined #openstack-nova | 09:33 | |
| *** iurygregory has quit IRC | 09:35 | |
| *** iurygregory has joined #openstack-nova | 09:36 | |
| *** derekh has joined #openstack-nova | 09:37 | |
| *** xek has joined #openstack-nova | 09:42 | |
| *** Liang__ has quit IRC | 09:44 | |
| *** tesseract has quit IRC | 09:54 | |
| *** tesseract has joined #openstack-nova | 09:57 | |
| *** xek has quit IRC | 09:58 | |
| *** xek has joined #openstack-nova | 09:58 | |
| *** psachin has joined #openstack-nova | 10:02 | |
| *** k_mouza has joined #openstack-nova | 10:12 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: rbd: Only log import failures when the RbdDriver is used https://review.opendev.org/761762 | 10:22 |
| lyarwood | stephenfin: ^ hopefully fixed btw | 10:26 |
| *** ociuhandu has quit IRC | 10:28 | |
| lyarwood | gibi / stephenfin / bauzas ; https://review.opendev.org/761870 could use reviews today if you have time, for whatever reason zuul didn't raise an error with the original change, it just removes the job silently. I'll file a bug later for this. | 10:29 |
| stephenfin | already done :) | 10:30 |
| gibi | I will look shortly | 10:30 |
| lyarwood | thanks | 10:30 |
| *** iurygregory_ has joined #openstack-nova | 10:32 | |
| *** sapd1 has joined #openstack-nova | 10:34 | |
| *** iurygregory has quit IRC | 10:35 | |
| *** ociuhandu has joined #openstack-nova | 10:36 | |
| *** ociuhandu has quit IRC | 10:37 | |
| *** ociuhandu has joined #openstack-nova | 10:37 | |
| *** jangutter has joined #openstack-nova | 10:38 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Add os-volume_attachments reference docs https://review.opendev.org/760971 | 10:40 |
| lyarwood | stephenfin: ^ some basic ref docs for volume attachment btw if you're interested, came up as part of a spec. | 10:41 |
| lyarwood | plan on adding lots of additional detail later in the cycle but the locking part is a good enough start at the moment | 10:41 |
| stephenfin | sure thing, I can take a look | 10:42 |
| lyarwood | thanks | 10:42 |
| *** jangutter_ has joined #openstack-nova | 10:55 | |
| *** xek_ has joined #openstack-nova | 10:56 | |
| *** jangutter has quit IRC | 10:58 | |
| *** xek has quit IRC | 10:59 | |
| *** tosky has joined #openstack-nova | 11:09 | |
| *** sapd1 has quit IRC | 11:12 | |
| *** mtreinish has joined #openstack-nova | 11:14 | |
| *** ociuhandu has quit IRC | 11:17 | |
| *** ratailor_ has joined #openstack-nova | 11:20 | |
| *** ociuhandu has joined #openstack-nova | 11:21 | |
| *** ratailor_ has quit IRC | 11:22 | |
| *** ratailor has quit IRC | 11:22 | |
| *** ratailor_ has joined #openstack-nova | 11:23 | |
| *** sapd1 has joined #openstack-nova | 11:32 | |
| *** sapd1 has quit IRC | 11:46 | |
| *** ratailor_ has quit IRC | 11:50 | |
| *** Luzi has joined #openstack-nova | 11:56 | |
| *** xek__ has joined #openstack-nova | 11:59 | |
| *** xek_ has quit IRC | 12:02 | |
| *** ociuhandu has quit IRC | 12:04 | |
| *** xek_ has joined #openstack-nova | 12:06 | |
| *** xek__ has quit IRC | 12:08 | |
| *** JamesBenson has joined #openstack-nova | 12:10 | |
| *** ociuhandu has joined #openstack-nova | 12:13 | |
| *** mkrai has quit IRC | 12:18 | |
| *** ociuhandu has quit IRC | 12:18 | |
| *** ociuhandu has joined #openstack-nova | 12:25 | |
| *** raildo has joined #openstack-nova | 12:28 | |
| *** ociuhandu has quit IRC | 12:29 | |
| *** ociuhandu has joined #openstack-nova | 12:30 | |
| *** dtantsur|afk is now known as dtantsur | 12:32 | |
| *** ociuhandu has quit IRC | 12:35 | |
| openstackgerrit | Merged openstack/nova master: zuul: Reintroduce nova-dsvm-multinode-base https://review.opendev.org/761870 | 12:45 |
| *** psachin has quit IRC | 12:53 | |
| *** spatel has joined #openstack-nova | 12:55 | |
| *** spatel has quit IRC | 12:59 | |
| *** iurygregory_ is now known as iurygregory | 13:01 | |
| *** priteau has joined #openstack-nova | 13:18 | |
| *** Luzi has quit IRC | 13:25 | |
| *** ociuhandu has joined #openstack-nova | 13:29 | |
| *** eharney_ has quit IRC | 13:45 | |
| *** brinzhang_ has joined #openstack-nova | 13:49 | |
| *** eharney has joined #openstack-nova | 13:50 | |
| *** eharney has quit IRC | 13:50 | |
| *** eharney has joined #openstack-nova | 13:51 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: Warn when starting services with older than N-1 computes https://review.opendev.org/761923 | 13:53 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: Add upgrade check about old computes https://review.opendev.org/761924 | 13:54 |
| *** k_mouza has quit IRC | 14:06 | |
| *** ociuhandu has quit IRC | 14:06 | |
| *** nweinber has joined #openstack-nova | 14:07 | |
| *** brinzhang0 has joined #openstack-nova | 14:10 | |
| *** TheJulia has joined #openstack-nova | 14:10 | |
| *** brinzhang has quit IRC | 14:13 | |
| *** k_mouza has joined #openstack-nova | 14:13 | |
| stephenfin | lyarwood: Thanks for the review on the 'openstack server evacuate' patch btw. I sent it on its way. Long overdue. | 14:18 |
| stephenfin | I need to go through the API with a fine comb but I _think_ we have the vast majority of the API covered at this point | 14:18 |
| *** artom has joined #openstack-nova | 14:19 | |
| gibi | stephenfin: what is the equvivalent for nova instance-action-list ? I allways fall back to the nova client to do that | 14:20 |
| lyarwood | gibi: openstack server event list | 14:20 |
| lyarwood | I think | 14:21 |
| gibi | /o\ | 14:21 |
| stephenfin | yeah, that | 14:21 |
| gibi | indeed | 14:21 |
| gibi | thanks | 14:21 |
| gibi | I need to build muscle memory for that | 14:21 |
| lyarwood | stephenfin: I don't think we have a version of that for migrations in osc btw | 14:21 |
| * lyarwood checks | 14:21 | |
| stephenfin | lyarwood: we do now | 14:21 |
| lyarwood | oh sweet | 14:22 |
| lyarwood | oh wait do you mean you're going to write it? | 14:22 |
| stephenfin | nope, already done | 14:22 |
| stephenfin | https://review.opendev.org/#/c/675304/ | 14:22 |
| stephenfin | don't know if its released yet, mind, but soon | 14:23 |
| lyarwood | ah right that might be why it's not in the docs yeat | 14:23 |
| lyarwood | yet* | 14:23 |
| gibi | sweet | 14:24 |
| gibi | I guess then we can check our docs and replace the nova CLI exemples with openstack CLI examples: http://paste.openstack.org/show/799826/ | 14:26 |
| gibi | bahh that paste contains duplicates | 14:27 |
| gibi | nah, much better now http://paste.openstack.org/show/799828/ | 14:28 |
| stephenfin | sounds reasonable; I could probably do that this week | 14:30 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use the non polling notification waiter in func test https://review.opendev.org/758445 | 14:30 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Create a fixture around fake_notifier https://review.opendev.org/758446 | 14:30 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use NotificationFixture for legacy notifications too https://review.opendev.org/758448 | 14:30 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test the NotificationFixture https://review.opendev.org/758450 | 14:31 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Move fake_notifier impl under NotificationFixture https://review.opendev.org/758451 | 14:31 |
| gibi | stephenfin: cool. you can plug me in for review | 14:31 |
| gibi | I think the (only) agreement on the PTG about OSC was that we try to fix our docs to use OSC examples | 14:32 |
| brinzhang_ | stephenfin: +1, good start | 14:33 |
| sean-k-mooney | gibi: once we have parity however i really would like to look at deprecating novaclient and requireing osc to be updated | 14:34 |
| sean-k-mooney | we can defer that for a cycle or so but i would like that to be our medium term goal | 14:34 |
| sean-k-mooney | even if we dont do it this cycle | 14:34 |
| stephenfin | sean-k-mooney: that's happening already | 14:34 |
| gibi | sean-k-mooney: agree | 14:35 |
| sean-k-mooney | not quite | 14:35 |
| stephenfin | gtema is doing a lot of work here | 14:35 |
| sean-k-mooney | stephenfin: wehn i say deprecate novaclint i mean stop accpeting any new features/commands being added | 14:35 |
| sean-k-mooney | so its bugfix only and require that they are added to osc instead | 14:35 |
| *** sapd1 has joined #openstack-nova | 14:35 | |
| stephenfin | to the CLI, sure, but we should keep adding to the library code until we're sure openstacksdk is good enough | 14:36 |
| sean-k-mooney | preferably doing one final realse of novaclinet when we freeze the features | 14:36 |
| sean-k-mooney | stephenfin: maybe i was not making that distintion really | 14:36 |
| stephenfin | I'd like to move pretty much all of OSC first, since we should have no issues doing that once openstacksdk is good enough | 14:36 |
| sean-k-mooney | since you have put this effort in to close the gap i just dont want it to reopen | 14:37 |
| *** ociuhandu has joined #openstack-nova | 14:42 | |
| openstackgerrit | sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432 | 14:44 |
| sean-k-mooney | stephenfin: lyarwood can ye take a look at ^ | 14:44 |
| sean-k-mooney | fixed the pep8 issue otherwise its the same | 14:44 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: doc: require openstack client change for every new API microversion https://review.opendev.org/717727 | 14:47 |
| *** ociuhandu has quit IRC | 14:47 | |
| gibi | stephenfin, sean-k-mooney: this ^^ is my contribution to the OSC topic | 14:47 |
| *** brinzhang0 has quit IRC | 14:48 | |
| *** brinzhang0 has joined #openstack-nova | 14:49 | |
| sean-k-mooney | :) | 14:49 |
| gibi | "Delay in Elastic Search: Indexing behind by 141 hours" :( | 14:51 |
| sean-k-mooney | thats only slightly longer then normally its normally 72 hours i think | 14:54 |
| *** k_mouza has quit IRC | 14:55 | |
| sean-k-mooney | if you are fering to logstash/kibana upstream | 14:55 |
| sean-k-mooney | *refering | 14:55 |
| * sean-k-mooney that still looks wrong but less so | 14:55 | |
| *** k_mouza has joined #openstack-nova | 15:01 | |
| gibi | it remember when it was close to 0 | 15:01 |
| gibi | last week it hanged around 100 hours | 15:01 |
| *** lpetrut has quit IRC | 15:02 | |
| sean-k-mooney | it does tend to vary but i think the target was no more the 72 hours | 15:02 |
| sean-k-mooney | it does catch up from time to time | 15:02 |
| gibi | I hope so | 15:03 |
| *** amodi has joined #openstack-nova | 15:11 | |
| brinzhang_ | stephenfin: thanks fix that issue from openstack server migration list CLI, I am sure I was tested ti, but the strange thing is that this issue was not found | 15:13 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Remove compute service level check for qos ops https://review.opendev.org/735570 | 15:21 |
| gibi | stephenfin: I think you will like this code removal patch ^ | 15:21 |
| *** ociuhandu has joined #openstack-nova | 15:22 | |
| *** mkrai has joined #openstack-nova | 15:24 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: Warn when starting services with older than N-1 computes https://review.opendev.org/761923 | 15:32 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: Add upgrade check about old computes https://review.opendev.org/761924 | 15:42 |
| *** iurygregory has quit IRC | 15:43 | |
| stephenfin | gibi: Done. I assume we're okay to merge things like that now that we've got the service version check? | 15:45 |
| gibi | stephenfin: that is my idea too | 15:45 |
| dansmith | the per-release service version check should catch anything that merged N-2 releases ago, | 15:45 |
| dansmith | so it should be fine to remove older ones and rely on the macro one yeah | 15:46 |
| dansmith | (assuming the qos one is old enough, I didn't look) | 15:46 |
| *** nweinber has quit IRC | 15:46 | |
| gibi | qos move ops are added in Ussuri | 15:47 |
| gibi | I mean that last one | 15:47 |
| gibi | so in Victoria we could have removed the service level check. But we did not for extra safety | 15:48 |
| *** nweinber has joined #openstack-nova | 15:48 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Migrate nova-grenade-multinode job to zuulv3 native https://review.opendev.org/742056 | 15:49 |
| dansmith | stephenfin: we're not really breaking RPC specifically here, because we're not changing any rpc versions or signatures or anything, but this is one of those "not covered by the rpc versions" behaviors.. it's breaking service-to-service interaction, but only for older computes (not even older RPC versions), but for which we've already said isn't supported | 15:52 |
| *** rpittau is now known as rpittau|bbl | 15:52 | |
| stephenfin | that makes sense | 15:53 |
| lyarwood | gmann: ^ I'd like to push ahead with this btw, we are currently hitting https://bugs.launchpad.net/nova/+bug/1901739 with the original bionic based job so I'd rather switch to Focal and add the ceph coverage later. | 15:53 |
| openstack | Launchpad bug 1901739 in OpenStack Compute (nova) " libvirt.libvirtError: internal error: missing block job data for disk 'vda'" [High,Fix released] - Assigned to Lee Yarwood (lyarwood) | 15:53 |
| dansmith | stephenfin: I'll put this in a comment once I review, but just echoing here since you called me out :) | 15:53 |
| stephenfin | ta : | 15:53 |
| stephenfin | :) | 15:53 |
| gibi | stephenfin, dansmith: thanks for the review btw | 15:53 |
| *** brinzhang_ has quit IRC | 15:55 | |
| *** ociuhandu has quit IRC | 15:58 | |
| *** dklyle has joined #openstack-nova | 16:02 | |
| *** iurygregory has joined #openstack-nova | 16:03 | |
| openstackgerrit | Merged openstack/nova master: Remove six.moves https://review.opendev.org/727224 | 16:05 |
| *** mlavalle has joined #openstack-nova | 16:06 | |
| gmann | lyarwood: but we are going to loose ceph coverage right? or we can add ceph coverage as separate job using existing script and move them once ceph greande base job is ready | 16:10 |
| lyarwood | gmann: we can try but I'd take a working gate over missing ceph coverage for a few weeks at the moment | 16:10 |
| gmann | lyarwood: existing zuulv2 grenade jobs is working right or it is failing? | 16:12 |
| jgwentworth | lyarwood: so what's the plan for adding ceph coverage back? seems like a risk to leave it uncovered for an extended period of time. do we have any idea how to do it for v3 jobs? | 16:12 |
| *** jgwentworth is now known as melwitt | 16:12 | |
| lyarwood | gmann: it's failing pretty often at the moment due to a an issue with libvirt/QEMU on bionic | 16:13 |
| gmann | you mean on victoria gate? on master gate, it should run on Focal | 16:13 |
| lyarwood | melwitt: https://review.opendev.org/#/q/owner:self+topic:native-zuulv3-migration+status:open - wire up a native zuulv3 multinode ceph job | 16:13 |
| *** ircuser-1 has joined #openstack-nova | 16:14 | |
| lyarwood | gmann: master gate, multinode grenade still uses bionic | 16:14 |
| lyarwood | gmann: and that's the problem here | 16:14 |
| lyarwood | gmann: or we can move it to NV | 16:14 |
| gmann | lyarwood: oh we should move it to Focal | 16:14 |
| gmann | ah legacy job | 16:15 |
| melwitt | lyarwood: sorry, I don't understand what I'm looking at here that's related to ceph | 16:15 |
| gmann | lyarwood: i did not move base legacy job on bionic | 16:15 |
| gmann | lyarwood: I think we can move to zuulv3 using script for now like in PS1 - https://review.opendev.org/#/c/742056/1 and once ceph grenade base is ready then remove the use of script ? | 16:17 |
| lyarwood | gmann: we can try | 16:18 |
| gmann | ok let me update it. | 16:18 |
| lyarwood | melwitt: sorry nova-grenade-multinode currently does some ceph stuff manually via the live migration hook - https://github.com/openstack/nova/blob/80b807a4c590980b9e514042778bd8c277e89e40/playbooks/legacy/nova-grenade-multinode/run.yaml#L58 & https://github.com/openstack/nova/blob/80b807a4c590980b9e514042778bd8c277e89e40/gate/live_migration/hooks/run_tests.sh#L55-L65 | 16:22 |
| lyarwood | melwitt: I wanted to break this out into a native zuulv3 job based on a multinode ceph job but that's taking a while to work out on the topic I shared above | 16:22 |
| melwitt | lyarwood: ok, so the manual stuff could be made "native" somehow. I did not know that | 16:23 |
| lyarwood | melwitt: yeah instead of calling specific bash functions from the plugin I just wanted to have a generic job that would deploy multinode ceph that we'd run the LM tests on in Nova | 16:24 |
| lyarwood | melwitt: I made some progress a while ago with the key sharing etc just became stuck at the end with getting the subnode to actually connect to the ceph cluster on the main node | 16:25 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Add live migration tests for PCI, SR-IOV servers https://review.opendev.org/746950 | 16:26 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Expand SR-IOV live migration tests with NUMA https://review.opendev.org/749360 | 16:26 |
| melwitt | lyarwood: I see, thanks, that helps. I hoped to be able to help in some way since I am concerned about the coverage loss but didn't know where to look or start | 16:27 |
| lyarwood | gmann: so are you just going to change the base job and hope that works? | 16:32 |
| lyarwood | to tempest-multinode-full-py3 | 16:32 |
| openstackgerrit | Merged openstack/nova master: Remove six.iteritems/itervalues/iterkeys https://review.opendev.org/727757 | 16:34 |
| openstackgerrit | Merged openstack/nova master: Remove six.byte2int/int2byte https://review.opendev.org/727777 | 16:34 |
| gmann | lyarwood: not tempest but grenade-multinode and running run_tests.sh in post phase with disable to run smoke tests on new node (which run as part of grenade-multinode playbooks) | 16:35 |
| lyarwood | gmann: right I'm asking which base job you're going to use | 16:37 |
| *** slaweq has quit IRC | 16:37 | |
| gmann | lyarwood: for grenade anyways we need to use grenade-multinode | 16:37 |
| *** ociuhandu has joined #openstack-nova | 16:38 | |
| lyarwood | gmann: right sorry, that's zuulv3 based and would just use the scripts to avoid us losing coverage | 16:39 |
| gmann | yeah | 16:39 |
| lyarwood | got ya | 16:40 |
| *** slaweq has joined #openstack-nova | 16:40 | |
| *** mkrai has quit IRC | 16:46 | |
| *** ociuhandu has quit IRC | 16:50 | |
| *** xinranwang has quit IRC | 16:57 | |
| *** ociuhandu has joined #openstack-nova | 16:58 | |
| *** ociuhandu has quit IRC | 17:03 | |
| *** ociuhandu has joined #openstack-nova | 17:08 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Migrate nova-grenade-multinode job to zuulv3 native https://review.opendev.org/742056 | 17:09 |
| *** hamalq has joined #openstack-nova | 17:11 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Add os-volume_attachments reference docs https://review.opendev.org/760971 | 17:13 |
| *** hamalq has quit IRC | 17:14 | |
| *** hamalq has joined #openstack-nova | 17:15 | |
| *** rpittau|bbl is now known as rpittau | 17:22 | |
| openstackgerrit | Merged openstack/nova stable/victoria: [doc]: Fix glance image_metadata link https://review.opendev.org/761423 | 17:24 |
| *** viks____ has quit IRC | 17:25 | |
| openstackgerrit | Dat Le proposed openstack/nova master: Fix unplugging VIF when migrate/resize VM https://review.opendev.org/751642 | 17:27 |
| *** k_mouza has quit IRC | 17:32 | |
| *** ralonsoh has quit IRC | 17:34 | |
| *** ociuhandu_ has joined #openstack-nova | 17:43 | |
| openstackgerrit | sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432 | 17:44 |
| sean-k-mooney | stephenfin: lyarwood can you re +w https://review.opendev.org/751642 | 17:45 |
| sean-k-mooney | it was rebased | 17:45 |
| stephenfin | done | 17:45 |
| sean-k-mooney | thanks | 17:45 |
| *** ociuhandu has quit IRC | 17:47 | |
| *** ociuhandu_ has quit IRC | 17:48 | |
| *** nweinber has quit IRC | 17:51 | |
| *** xek__ has joined #openstack-nova | 17:53 | |
| *** xek__ has quit IRC | 17:55 | |
| *** xek_ has quit IRC | 17:56 | |
| *** xek__ has joined #openstack-nova | 17:56 | |
| *** xek__ has quit IRC | 17:56 | |
| *** derekh has quit IRC | 18:00 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: DNM: Testing system scope in tempest https://review.opendev.org/740124 | 18:07 |
| *** andrewbonney has quit IRC | 18:09 | |
| openstackgerrit | Elod Illes proposed openstack/nova stable/ussuri: [doc]: Fix glance image_metadata link https://review.opendev.org/761977 | 18:20 |
| mnaser | sean-k-mooney: have you done some investigation by any chance on any potential ways of speeding up nova startup time in environments with compute nodes that have a large port count | 18:23 |
| mnaser | in this case, it takes almost 5-6 minutes for the agent to go up in an env with ~150-160 ports | 18:23 |
| mnaser | using osv | 18:23 |
| mnaser | i havent tried migrating to the native ovsdb driver, maybe that might help, but a whole lot of plugging happens | 18:23 |
| *** dtantsur is now known as dtantsur|afk | 18:24 | |
| sean-k-mooney | am not really. we have seen that usign native does indeed help form your previous issues but in general we do need to ensure that all network interfacs are plugged for all vms when the compute agent starts | 18:26 |
| sean-k-mooney | it also depend on the release i know we fixed some issue with ip adress checking on newer release | 18:27 |
| mnaser | sean-k-mooney: wonder if it might make sense to make this happen in a threadpool or something, as it seems to be happening one-by-one | 18:27 |
| sean-k-mooney | not really | 18:28 |
| mnaser | because right now in some of those bigger vms, nova-compute reports down as it goes through the restart that takes ~6m | 18:28 |
| sean-k-mooney | we could but we have to wait for them all to complete | 18:28 |
| sean-k-mooney | so if we dispatch them to a tread pool we will get more paralium but we will the fill the pool and have to wait for it to complete | 18:29 |
| sean-k-mooney | plugging form a nova side shoudl not take that long in general | 18:29 |
| sean-k-mooney | with the native driver its much much faster | 18:29 |
| *** gyee has joined #openstack-nova | 18:30 | |
| mnaser | ok i guess thats' probably going to be my next step to see how that improves it | 18:30 |
| sean-k-mooney | as its not spawnting multiple shells via privsep in that process | 18:30 |
| mnaser | sean-k-mooney: seems valid, i'll check it out and report and let you know what sort of improvement i see :) | 18:31 |
| sean-k-mooney | putting it in a tread pool would basically just increase the batch size at the cost of memroy during start up. | 18:31 |
| sean-k-mooney | cool there would be a speed up form a thread pool but not as much as using the native driver | 18:32 |
| *** mlavalle has quit IRC | 18:43 | |
| *** mlavalle has joined #openstack-nova | 18:44 | |
| *** rpittau is now known as rpittau|afk | 18:45 | |
| openstackgerrit | Merged openstack/nova master: virt: Remove 'get_per_instance_usage' API https://review.opendev.org/749318 | 18:50 |
| lyarwood | \o/ | 18:51 |
| sean-k-mooney | lyarwood: was that for the removal ^ | 18:59 |
| sean-k-mooney | oh because something finally merged in the gate :P | 19:00 |
| sean-k-mooney | both are good | 19:00 |
| *** whoami-rajat__ has quit IRC | 19:00 | |
| *** tesseract has quit IRC | 19:02 | |
| *** larainema has quit IRC | 19:12 | |
| *** jangutter has joined #openstack-nova | 19:26 | |
| *** jangutter_ has quit IRC | 19:30 | |
| dansmith | hmm, on focal the ceph devstack plugin tells me I'm on an unsupported distro | 19:36 |
| dansmith | lyarwood: any idea about that? | 19:36 |
| lyarwood | dansmith: no idea, I thought it was there? | 19:37 |
| lyarwood | # git grep f32 | 19:37 |
| lyarwood | devstack/lib/ceph: if [[ ! ${DISTRO} =~ (focal|bionic|xenial|f31|f32) ]]; then | 19:37 |
| dansmith | hmm, I nuked opt/stack but maybe I've got residue somewhere | 19:38 |
| dansmith | it's checking against a list that doesn't include focal | 19:38 |
| dansmith | heh, doh | 19:38 |
| dansmith | enable_plugin devstack-plugin-ceph /home/dan/devstack-plugin-ceph | 19:38 |
| sean-k-mooney | oh your using your local copy | 19:59 |
| sean-k-mooney | got an old version checked out | 19:59 |
| dansmith | yeah, I nuked /opt/stack and then it cloned it right back there for me :P | 19:59 |
| sean-k-mooney | yep i do that intentionally for the most part and put all my repos in /opt/repos | 20:00 |
| sean-k-mooney | just so it does not clone over itself | 20:00 |
| sean-k-mooney | gmann:is there a spec or something covering the use of token scopes with nova. i dont see anythin after the ussuri spec https://specs.openstack.org/openstack/nova-specs/specs/ussuri/implemented/policy-defaults-refresh.html | 20:05 |
| sean-k-mooney | that does nto cover the domains | 20:05 |
| sean-k-mooney | the 4 scopes in https://specs.openstack.org/openstack/nova-specs/specs/ussuri/implemented/policy-defaults-refresh.html#use-cases make sense to me | 20:06 |
| sean-k-mooney | but i dont see a defeintion for system_member, project_admin, domain_(reader, member or admin) | 20:06 |
| sean-k-mooney | we have some interset down stream in supporting a 3*3 matix (reader, memeber, admin) *( project, domain, system) | 20:07 |
| sean-k-mooney | but i dont recall ever disscuing domains or proejct admin and system memebr in the context of nova upstream | 20:08 |
| sean-k-mooney | https://review.opendev.org/#/c/698397/14/etc/rbac-persona-accounts.yaml.sample has entries for all 9 roles | 20:09 |
| sean-k-mooney | but given we have not had any specs or cross project session on this im having a hard time relateing that to how we would support this in nova | 20:10 |
| sean-k-mooney | gmann: i know there was a pop up team but any pointers on where this might have been discussed | 20:10 |
| sean-k-mooney | melwitt: johnthetubaguy any input on ^ | 20:12 |
| gmann | sean-k-mooney: we have project_admin in nova for case like creating server with host specify but system_member is not there as not needed from current policy rules | 20:16 |
| gmann | also domain scope (it's three pesona) | 20:16 |
| sean-k-mooney | do you have docs for this | 20:16 |
| sean-k-mooney | there were only 4 included in the spec | 20:17 |
| sean-k-mooney | project_admin was not one of them | 20:17 |
| gmann | yeah, I updated policy doc for that | 20:17 |
| *** tosky has quit IRC | 20:17 | |
| sean-k-mooney | ok because form a spec point of vew we only ever approved system_admin, project_memeber system_reader and project_reader | 20:18 |
| gmann | sean-k-mooney: this does not include what all combination nova support but give an idea on new policy - https://docs.openstack.org/nova/latest/configuration/policy-concepts.html | 20:18 |
| sean-k-mooney | so since this is an api cahnge im kind of confused why we dont have a spec for supporting other scopes and ropels in nova | 20:18 |
| melwitt | yeah, that ^ is the doc I was searching for wrt the support in nova | 20:19 |
| gmann | sean-k-mooney: project_admin came up during implementation only and that is for create server with specific request (like force host, zero disk flavor etc) on;y | 20:19 |
| sean-k-mooney | ok but are we actully using it in code | 20:19 |
| sean-k-mooney | https://specs.openstack.org/openstack/nova-specs/specs/ussuri/implemented/policy-defaults-refresh.html is the only spec that exist for this so really without a spec to add it we should not be useing in the api right | 20:20 |
| gmann | as part of policy defaults yes but we do not change token in nova right? whatever token is used for API access will be validated against the policy values | 20:20 |
| sean-k-mooney | my concern is really for domains im concerend that a domain member may not have a project | 20:21 |
| gmann | sean-k-mooney: I am not sure about supporting other scopes. if we want domain scope control in nova then we need to see what all nova API operations are domain level and system level. | 20:21 |
| sean-k-mooney | maybe they do but if they dont then that will break our api assumtions | 20:22 |
| gmann | i never thought of domain in nova. | 20:22 |
| sean-k-mooney | its come up downstream my responce is right now we dont support it | 20:22 |
| gmann | and they use domain member user to access nova API ? | 20:22 |
| sean-k-mooney | well its not a thing yet | 20:22 |
| sean-k-mooney | we have a request to supprot RBAC for osp 17 which will be based on wallaby | 20:23 |
| melwitt | yeah, my understanding is that we now have the code needed to handle scoped tokens in nova, so the way that they get used is someone (a user) has to request scoped tokens and use them when they call nova and then policy can validate the token against the policy | 20:23 |
| gmann | ok, in that case they can assign admin/member/reader role on required project | 20:23 |
| sean-k-mooney | but since domain was never discussed upstream i wanted to push back to just the 4 roles we intended to support | 20:23 |
| sean-k-mooney | melwitt: right but we need to agree what scopes and roles are required for each endpoint | 20:24 |
| sean-k-mooney | and what each scope/roles allows you to do | 20:24 |
| gmann | sean-k-mooney: these are combination we support currently - https://github.com/openstack/nova/blob/master/nova/policies/base.py#L48-L54 | 20:24 |
| gmann | sean-k-mooney: i can add these in policy doc so that we do not need to point these to code | 20:25 |
| sean-k-mooney | project admin is the only one im not expecting since we nver approved that in a spect | 20:25 |
| sean-k-mooney | the rest make sense to me | 20:25 |
| sean-k-mooney | for example we discussed in the ptg that prject admisn shoudl not be aware of the hosts vms run on | 20:26 |
| gmann | sean-k-mooney: also html policy doc mention the scope of each policy/API as part of oplicy documentation - https://docs.openstack.org/nova/latest/configuration/policy.html | 20:26 |
| sean-k-mooney | but you mentioned that one of the usecase was allowing them to boot on a host | 20:26 |
| gmann | yeah | 20:26 |
| sean-k-mooney | have we released with project_admin yet | 20:27 |
| gmann | you remember in PTG we discuss about it | 20:27 |
| sean-k-mooney | i rememebr the one two weeks ago | 20:27 |
| *** tosky has joined #openstack-nova | 20:28 | |
| gmann | my initial thought while adding PROJECT_ADMIN was that this is temp and TODO will remove this - https://github.com/openstack/nova/blob/45e8fa3ade54313754fbea245ad546097150486d/nova/policies/servers.py#L179 | 20:29 |
| sean-k-mooney | i kind of which there was a spec for projecft admin because i really ame not sure that the current usaged in the api are valid | 20:29 |
| sean-k-mooney | gmann: we cant remove it | 20:29 |
| gmann | but as we discussed in PTG, I think it make sense to keep it and ask users to assign appropriate role to let users to boot on specific host | 20:29 |
| sean-k-mooney | well not without a microverion | 20:30 |
| gmann | this is policy default change so microversion not required if we change any of these | 20:30 |
| sean-k-mooney | making it sys_admin i think make sense | 20:30 |
| sean-k-mooney | from an interoperablity point of view im not sure i agree | 20:31 |
| gmann | sean-k-mooney: that is the things we discussed in PTG. my proposal was sys_admin and take project_id as request param. | 20:31 |
| sean-k-mooney | for this case since i kind fo want to revert this out and make it system_admin im happy to agree | 20:31 |
| sean-k-mooney | that is a different usecase | 20:32 |
| gmann | but with sys_adm, project_id needs to be passed for which sys admin want to create server for | 20:32 |
| sean-k-mooney | yes so that is the actuall cloud admin creating a server in a project in responce to a supprot request or something | 20:33 |
| sean-k-mooney | that is differnet for giving a customer a project adminsitarto role and allowign them to do it in a self service manner | 20:33 |
| sean-k-mooney | that is what project_admin role would mean | 20:33 |
| sean-k-mooney | gmann: im fine with the sysadmin doing what your proposing | 20:34 |
| sean-k-mooney | but ithe proejct_admin persona im describing i dont think shoudl have any awareness of hosts | 20:34 |
| sean-k-mooney | gmann: anyway thanks ill review the docs you linked. | 20:35 |
| gmann | that was my thought while proposing this but johnthetubaguy melwitt point was it is asking cloud admin to give extra role to user (project_admin or system_reader to know host info) to allow users to boot on specific host | 20:36 |
| sean-k-mooney | so that is something i think we need to actully discuss if we are every to use those poices by default in code | 20:36 |
| sean-k-mooney | vai a spec | 20:37 |
| sean-k-mooney | becaue you are fundimentally change the way the api works allowing operation to work for a different set of users | 20:37 |
| gmann | which make sense to me too after PTG discussion. so from nova API side we say 'sustem_reader is role you need to know host info' and 'project_admin is role to boot instance on requested host' so you can assign those based on users req | 20:37 |
| sean-k-mooney | ya although i think there are more details to be worked out | 20:38 |
| gmann | sean-k-mooney: right, for migrating to new policy, cloud provider needs to change the existing users token | 20:38 |
| *** whoami-rajat__ has joined #openstack-nova | 20:38 | |
| sean-k-mooney | like shold a project admin be allowed to do a live migration? | 20:39 |
| sean-k-mooney | shoudl they see the host in the server show (based on the above no unles they have system_reader) | 20:39 |
| gmann | sean-k-mooney: as default policy no but live migration case we opened for system as well as project scope also. in case use case is to allow users to perform live migration. | 20:40 |
| sean-k-mooney | giving the proejct admin system reader also feels slightly wrong | 20:40 |
| gmann | sean-k-mooney: let me propose doc update what we agreed in PTG and then we can see how it looks like. it is too much asking to providers or ok. | 20:41 |
| sean-k-mooney | e.g. im not sure you should have readonly access to everything | 20:41 |
| sean-k-mooney | gmann: well im trying to gague the impact to us downstream from a support point of view | 20:41 |
| gmann | If provider want to allow any users to boot on requested host then giving system_reader to them not bad so that they can access host info. | 20:43 |
| sean-k-mooney | gmann: tehy could also acess other info right | 20:43 |
| sean-k-mooney | e.g. list all servers | 20:43 |
| sean-k-mooney | or all keypairs | 20:43 |
| sean-k-mooney | for all users | 20:44 |
| gmann | yeah that is one issue. | 20:44 |
| sean-k-mooney | that makes it a non starter for me | 20:44 |
| sean-k-mooney | system reader is basically full admin but readonly right | 20:44 |
| sean-k-mooney | so i could use it to find the fix ips of other tenants insntances | 20:45 |
| sean-k-mooney | if it was not scoped to noava i could hten list the security groups to find open ports and try connecting to them | 20:45 |
| gmann | well with override policy they can restrict, like list server for all is controlled with separate policy. list keypairs can be restricted with user_id | 20:45 |
| sean-k-mooney | ok | 20:46 |
| sean-k-mooney | but im worried about the support matrix | 20:46 |
| gmann | but i agree that is open things for them which was point in PTG also | 20:46 |
| sean-k-mooney | e.g. use agreeing to support anythong other then the nova default by default with our custoemrs | 20:46 |
| gmann | may be admin need to carefully select such users and trust them if they are allowed to boot on requested host ? | 20:47 |
| sean-k-mooney | maybe but that basically to me say we can never enabel this by default | 20:47 |
| sean-k-mooney | well | 20:47 |
| sean-k-mooney | no its the use fo system_reader that is too heavy here | 20:47 |
| sean-k-mooney | i kind fo would liek if we could use the tenant isolation aggreate info | 20:48 |
| sean-k-mooney | so that with project_admin you coudl list hosts and only see those hosts | 20:48 |
| sean-k-mooney | so no system reader | 20:48 |
| sean-k-mooney | needed to boot to a host you are allowed to boot too | 20:48 |
| sean-k-mooney | if you are not limited to an aggreate i guess that would list all hosts | 20:49 |
| gmann | but host is not project level info right | 20:49 |
| sean-k-mooney | it kind of is | 20:49 |
| sean-k-mooney | we supprot assocating tenats/project to aggreates | 20:49 |
| sean-k-mooney | via either a placment prefilter or a schduler post filter | 20:50 |
| gmann | but project_admin is our special case. means no where else it is being used | 20:50 |
| sean-k-mooney | that the thing i see it potally being a thing in other services too | 20:50 |
| *** tbachman has quit IRC | 20:50 | |
| gmann | ah yeah | 20:50 |
| sean-k-mooney | i can totally see project admins being allowed to create users for a project in there project only via keystoen for example | 20:51 |
| sean-k-mooney | or managing a subset of roles with in a porject | 20:51 |
| sean-k-mooney | e.g. givie a user project_reader, porject_admin or project member | 20:52 |
| sean-k-mooney | for that project | 20:52 |
| gmann | also i think neutron might have when they do new policy like attr level policy rule etc | 20:52 |
| sean-k-mooney | right i could see requiring proejct_admin to create shared networks for example | 20:52 |
| sean-k-mooney | or better example | 20:53 |
| gmann | so both options (sys reader or project admin) have pros and cons. | 20:53 |
| sean-k-mooney | adding a qos policy to a network | 20:53 |
| *** tbachman has joined #openstack-nova | 20:53 | |
| melwitt | I'm not 100% following this convo but wanted to mention I think it's normal and expected that users will have to request and use appropriate tokens (and have appropriate roles) for individual APIs and that if we try to fit everything a user can possibly want to do to fall under one token scope/role then we're going back toward the "admin does everything" direction, trying to make everything fit into one box again | 20:53 |
| sean-k-mooney | to make proejct_admin work we woudl need other code changes | 20:53 |
| sean-k-mooney | melwitt: multiple tokens totally makes sense but i dont think system_reader is approcate for anyoen that you dont fully trust | 20:54 |
| sean-k-mooney | so i can see system reader ever be appliable for a tenant of a vexhost cloud | 20:55 |
| sean-k-mooney | that did not work at vexhost | 20:55 |
| gmann | melwitt: yeah. project_admin was really a temp think with assumption that system scope users to allow creating server for projetcs | 20:55 |
| sean-k-mooney | the main usecasue for system_reader is for audits right | 20:56 |
| sean-k-mooney | so by default you would want system_reader to be able to read across multipel projects | 20:56 |
| sean-k-mooney | where as proejct_reader would be the same fucntion for a singel project | 20:57 |
| sean-k-mooney | and domain is inbetween | 20:57 |
| sean-k-mooney | read only access to all proejct in a domain | 20:57 |
| sean-k-mooney | *project resouces | 20:57 |
| sean-k-mooney | if you require something more the system_reader for inter proejct server list then it kind of breakes its orginal usecause | 20:59 |
| sean-k-mooney | anyway its late and im hungry so ill call it a night | 20:59 |
| sean-k-mooney | gmann: thanks for the info o/ | 20:59 |
| gmann | but if any users is allowed to boot on requested host then it is special user right. | 21:00 |
| gmann | sean-k-mooney: sure. take rest. We can discuss tomorrow. | 21:00 |
| *** sapd1 has quit IRC | 21:05 | |
| *** mgoddard has quit IRC | 21:25 | |
| *** mgoddard has joined #openstack-nova | 21:26 | |
| *** rcernin has joined #openstack-nova | 21:27 | |
| *** vishalmanchanda has quit IRC | 21:57 | |
| *** rcernin has quit IRC | 22:06 | |
| *** rcernin has joined #openstack-nova | 22:12 | |
| *** martinkennelly has quit IRC | 22:12 | |
| *** rcernin has quit IRC | 22:22 | |
| *** rcernin has joined #openstack-nova | 22:22 | |
| itsjg | Hello! Could someone help point me in the right direction, I can't get a machine to migrate off an old host due to the "No valid host found for cold migrate (HTTP 400)" error. I have tried setting every debug logging level to maximum but can't get any further details as to why nova-scheduler is not moving the VM to a different host. I have plenty of capacity across the whole cluster, checked all compute services across the whole cluster, etc. I guess f | 22:30 |
| itsjg | an easy way to see why the scheduler throws an error, is there some place to look that I'm missing? | 22:30 |
| *** brinzhang_ has joined #openstack-nova | 22:30 | |
| *** brinzhang0 has quit IRC | 22:34 | |
| *** takamatsu has quit IRC | 22:35 | |
| *** slaweq has quit IRC | 22:39 | |
| *** iurygregory has quit IRC | 23:04 | |
| *** iurygregory has joined #openstack-nova | 23:16 | |
| *** takamatsu has joined #openstack-nova | 23:20 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Improve policy doc for supported scope info https://review.opendev.org/762013 | 23:32 |
| gmann | sean-k-mooney: melwitt johnthetubaguy ^^ adding nova supported scope & roles into doc | 23:33 |
| *** tosky has quit IRC | 23:33 | |
| melwitt | ack | 23:34 |
| openstackgerrit | Merged openstack/nova master: rbd: Only log import failures when the RbdDriver is used https://review.opendev.org/761762 | 23:51 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!