Tuesday, 2021-01-26

« Monday, 2021-01-25 Index Wednesday, 2021-01-27 »
*** k_mouza has quit IRC00:01
*** rcernin has joined #openstack-nova00:03
*** brinzhang has quit IRC00:05
*** luksky has quit IRC00:08
*** brinzhang has joined #openstack-nova00:17
*** rcernin has quit IRC00:24
*** rcernin has joined #openstack-nova00:27
brinzhanggmann: do you review this patch I was said last Friday https://review.opendev.org/c/openstack/nova/+/768852/8/nova/tests/unit/api/openstack/compute/test_simple_project_usage.py#58600:29
brinzhanggmann: From the test case, I found the show or index os-simple-projectt-usage APIs cannot be valid for 2.9000:30
*** LinPeiWen has joined #openstack-nova00:36
*** LinPeiWen has quit IRC00:41
*** LinPeiWen has joined #openstack-nova00:46
openstackgerritBrin Zhang proposed openstack/nova master: Fix ForbiddenWithAccelerators to HTTPForbidden for shelve API  https://review.opendev.org/c/openstack/nova/+/77000700:51
*** mlavalle has quit IRC01:04
*** rcernin has quit IRC01:18
*** k_mouza has joined #openstack-nova01:32
*** macz_ has quit IRC01:42
*** rcernin has joined #openstack-nova01:55
*** k_mouza has quit IRC01:58
*** rcernin has quit IRC02:02
*** tinwood has quit IRC02:08
*** tinwood has joined #openstack-nova02:11
*** rcernin has joined #openstack-nova02:15
*** zzzeek has quit IRC02:19
*** rcernin has quit IRC02:20
*** macz_ has joined #openstack-nova02:21
*** zzzeek has joined #openstack-nova02:22
*** rcernin has joined #openstack-nova02:23
*** ociuhandu has joined #openstack-nova02:24
*** spatel has joined #openstack-nova02:27
*** ociuhandu has quit IRC02:28
*** spatel has quit IRC02:29
*** spatel has joined #openstack-nova02:30
*** macz_ has quit IRC02:30
*** spatel has quit IRC02:31
*** spatel has joined #openstack-nova02:33
*** rcernin has quit IRC02:38
*** zzzeek has quit IRC02:49
*** spatel has quit IRC02:49
*** zzzeek has joined #openstack-nova02:51
*** spatel has joined #openstack-nova02:55
*** johanssone has quit IRC03:00
*** johanssone has joined #openstack-nova03:01
*** rcernin has joined #openstack-nova03:34
*** rcernin has quit IRC03:39
*** khomesh24 has joined #openstack-nova03:51
*** k_mouza has joined #openstack-nova03:58
*** k_mouza has quit IRC04:03
*** macz_ has joined #openstack-nova04:47
*** macz_ has quit IRC04:52
*** haleyb has quit IRC05:09
*** rcernin has joined #openstack-nova05:18
*** brinzhang has quit IRC05:20
*** brinzhang has joined #openstack-nova05:21
*** rcernin has quit IRC05:26
*** rcernin has joined #openstack-nova05:40
*** rcernin has quit IRC05:45
*** spatel has quit IRC05:47
*** gyee has quit IRC06:07
*** rcernin has joined #openstack-nova06:09
*** nightmare_unreal has joined #openstack-nova06:32
*** rcernin has quit IRC06:49
*** spotz has quit IRC06:54
*** antonym has quit IRC06:56
*** antonym has joined #openstack-nova06:56
*** rcernin has joined #openstack-nova07:38
*** luksky has joined #openstack-nova07:50
*** dklyle has quit IRC07:51
*** xek_ has joined #openstack-nova07:52
*** ralonsoh has joined #openstack-nova07:52
*** luksky has quit IRC07:53
*** zoharm has joined #openstack-nova07:55
*** rpittau|afk_ is now known as rpittau07:57
*** slaweq has joined #openstack-nova07:59
openstackgerritWenping Song proposed openstack/nova master: Replaces tenant_id with project_id from List/Update Servers APIs  https://review.opendev.org/c/openstack/nova/+/76429208:01
openstackgerritWenping Song proposed openstack/nova master: Replace all_tenants with all_projects in List Server APIs  https://review.opendev.org/c/openstack/nova/+/76531108:01
openstackgerritWenping Song proposed openstack/nova master: Replaces tenant_id with project_id from Rebuild Server API  https://review.opendev.org/c/openstack/nova/+/76638008:01
openstackgerritWenping Song proposed openstack/nova master: Replaces tenant_id with project_id from List SG API  https://review.opendev.org/c/openstack/nova/+/76672608:01
*** tesseract has joined #openstack-nova08:09
*** rcernin has quit IRC08:17
*** andrewbonney has joined #openstack-nova08:19
*** rcernin has joined #openstack-nova08:42
*** tosky has joined #openstack-nova08:45
*** rcernin has quit IRC08:46
openstackgerritMIKI Nobuhiro proposed openstack/nova master: libvirt: add IP address to libvirt guest metadata  https://review.opendev.org/c/openstack/nova/+/75055208:55
*** ociuhandu has joined #openstack-nova08:56
*** ociuhandu has quit IRC08:58
*** ociuhandu has joined #openstack-nova08:59
openstackgerritWenping Song proposed openstack/nova master: Replaces tenant_id with project_id from List/Update Servers APIs  https://review.opendev.org/c/openstack/nova/+/76429209:01
*** kevko has quit IRC09:13
*** swp20 has joined #openstack-nova09:14
lyarwood\o morning09:15
openstackgerritLee Yarwood proposed openstack/nova stable/victoria: api: Log os-resetState as an instance action  https://review.opendev.org/c/openstack/nova/+/77248009:18
openstackgerritLee Yarwood proposed openstack/nova stable/ussuri: api: Log os-resetState as an instance action  https://review.opendev.org/c/openstack/nova/+/77248109:19
openstackgerritLee Yarwood proposed openstack/nova stable/train: api: Log os-resetState as an instance action  https://review.opendev.org/c/openstack/nova/+/77248209:19
*** ociuhandu has quit IRC09:19
*** ociuhandu has joined #openstack-nova09:25
*** kevko has joined #openstack-nova09:27
*** rcernin has joined #openstack-nova09:27
*** alexe9191 has joined #openstack-nova09:27
*** ociuhandu has quit IRC09:30
lyarwoodelod / melwitt / bauzas ; https://review.opendev.org/c/openstack/nova/+/758730 would you mind taking a look at this (and the cherry picks) this week?09:30
alexe9191Good day everyone, I was wondering if someone could help me with a question related to novaclient?09:36
alexe9191In order to reach the API endpoint I need to set a specific header09:36
alexe9191I can do this if I am writing my own python code, but I'd like to use the OpenStack cli.09:36
alexe9191There is sadly no -H option in the OpenStack cli. I was wondering where can I set curl options in the code of novaclient so that I can enforce passing the header?09:37
swp20moring gibi, please review noVNC https://review.opendev.org/c/openstack/nova/+/622336/28 ,thanks.09:42
*** derekh has joined #openstack-nova09:44
*** rcernin has quit IRC09:45
bauzaslyarwood: sorry was in a meeting09:50
bauzasbut sure, I can take a look09:50
*** ociuhandu has joined #openstack-nova09:54
*** rcernin has joined #openstack-nova09:58
*** k_mouza has joined #openstack-nova09:59
*** k_mouza has quit IRC10:03
*** swp20 has quit IRC10:10
*** songwenping_ has joined #openstack-nova10:10
*** songwenping_ has quit IRC10:10
*** songwenping_ has joined #openstack-nova10:11
*** sapd1 has joined #openstack-nova10:11
*** brinzhang has quit IRC10:14
sapd1Hello everyone, I would like to use vGPU feature with Nova and KVM(QEMU). Which model should I choose? As I know If we use this feature, we need to pay license for it (NVDIA for example). Is it true?10:18
*** dtantsur|afk is now known as dtantsur10:18
lyarwoodalexe9191: use openstacksdk, it should handle everything for you10:19
bauzassapd1: correct, you need a license if you want to use nvidia GPUs10:20
bauzasfor the models, sec.10:20
sapd1bauzas, Thanks, There is no other options for it?10:20
bauzassapd1: for the models : https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#supported-gpus-grid-vgpu10:22
alexe9191@lyar10:22
bauzassapd1: some folks use nvidia T4s for testing10:23
alexe9191@lyarwood: Thank you, I was looking for a way to hack the code locally so that I can use the openstack command line tools. But that does not seem to be the smartest course of action here.10:23
elodlyarwood: the patch looks ok to me!10:23
elod+2'd10:24
sapd1bauzas, Thanks, I will check it.10:26
bauzassapd1: fwiw, Intel gvt-g cards also work :p10:27
MrClayPoleadmin0: "virsh list" shows no instances. I can boot the VM, run "dumpxml  instance-<ref>" it shows http://paste.openstack.org/raw/801977/ and as it boots the 2 shared storage disks so they should be OK? The migration fails with "Error: Failed to perform requested operation on instance "<VM Name>", the instance has an error status: Please try again later [Error: The server has either erred or is incapable of performing the10:27
MrClayPole requested operation. (HTTP 500) (Request-ID: <ID>)". The nova logs show http://paste.openstack.org/raw/801979/. Should I just give up at this point and recreate the VM?10:27
*** rcernin has quit IRC10:27
bauzassapd1: but.... what I can say ? :)10:27
sapd1bauzas, could you give some recommendations? Is it required license ?10:27
bauzassapd1: which usecase would you like to do for vGPUs ?10:28
admin0virsh list --all10:28
admin0to also show stopped instances10:28
*** hack-char has quit IRC10:29
bauzassapd1: if you want to see my face... https://www.youtube.com/watch?v=sQCgyo2BRe410:29
bauzasand for the cards, well... as I said, that depends on which kind of use case you'd like to have10:30
bauzast4s are cheap10:30
*** hack-char has joined #openstack-nova10:30
sapd1bauzas, some workloads such as ML, video transcode..10:31
MrClayPoleWell don't I feel stupid, Thanks for pointing out the --all. This is the output when powered off http://paste.openstack.org/raw/801982/ I've taken a look and can't see anything that would block the migration10:31
bauzassapd1: then, test with a T410:31
* bauzas would love to have an openstack env at home with T4s for transcoding :p10:32
sapd1bauzas, haha. Is it required license for testing ?10:33
bauzasagain, there are nvidia docs for this :p10:33
lyarwoodMrClayPole: look for req-3fc432cf-4d5c-4d9b-b16e-d06d912e93e7 in your cinder-api / cinder-volume logs10:33
lyarwoodMrClayPole: and yeah if all of your data is in these volumes it might be easier to recreate the instance10:34
bauzassapd1: https://docs.nvidia.com/grid/latest/grid-licensing-user-guide/index.html#license-editions-nvidia-grid10:34
lyarwoodMrClayPole: I'm assuming this is the first attempt to cold migrate that failed?10:34
lyarwoodMrClayPole: resize sorry10:34
bauzassapd1: and https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid10:35
bauzasand for a T4 https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#vgpu-types-tesla-t410:36
MrClayPolelyarwood: Yeah, running a cold migration. I'm temped to see this one through rather than go for the recreate as this'll be a good chance for me to start learning the internals of openstack.10:36
bauzassapd1: if it's just transcoding, a vApps license with a T4-1A vGPU type seems reasonable to me10:37
lyarwoodMrClayPole: cool well if you can trace that request-id in the cinder logs and pastebin them I can take a look and walk you through this10:38
bauzasfor ML, well, good luck, that highly depends on your training needs10:38
lyarwoodelod: do you know if anyone has capped get-pip.py given py27 support has now been dropped? Borked stable/queens AFAICT at least.10:39
bauzassapd1: but you can run for a short period of time a server without a licensed vGPU https://docs.nvidia.com/grid/latest/grid-licensing-user-guide/index.html#software-enforcement-grid-licensing10:40
sapd1bauzas, nice. I will request for buying it and try later. :D10:45
bauzassapd1: ask maybe first a nvidia rep10:45
bauzasI'm not a sales person :)10:45
bauzasand see which kind of license matches your need10:45
*** derekh has quit IRC10:46
sapd1bauzas, yep. I need time to research on it. If I have any issues, I will ping you. thanks :D10:46
fricklerlyarwood: elod: there's some patches up for devstack about get-pip10:46
fricklermost of them pushed by elod in fact ;)10:46
bauzassapd1: again, neither working for nvidia, nor a sales person, but if I can help...10:47
sapd1bauzas, yeah :D10:47
lyarwoodfrickler: I had a feeling :D10:48
* lyarwood looks10:48
lyarwoodelod++10:48
elodlyarwood: frickler: yes, I noticed that amotoki had a fix for that and backported it till pike :]10:48
lyarwooddo we need a bug? I just wrote https://bugs.launchpad.net/devstack/+bug/1913283 before finding this10:50
openstackLaunchpad bug 1913283 in devstack "get-pip.py no longer supports py27" [Undecided,New]10:50
elodit will need merging in reverse order for some patches (where there's grenade)10:50
elodthese are the patches btw: https://review.opendev.org/q/topic:%22get-pip-py27%2210:51
lyarwoodseems a little over the top, can't we just set PIP_GET_PIP_URL on the py27 jobs?10:53
fricklerlyarwood: then people will complain that their local stuff fails. I personally am all for dropping py2 support completely, but it doesn't seem to be possible to get consensus on that yet10:59
lyarwoodfrickler: ah fair11:00
*** rcernin has joined #openstack-nova11:04
*** brinzhang has joined #openstack-nova11:04
*** k_mouza has joined #openstack-nova11:05
*** alexe9191 has quit IRC11:07
*** rpittau is now known as rpittau|bbl11:09
*** masterpe has quit IRC11:19
*** masterpe has joined #openstack-nova11:29
gibistephenfin: hi! if you need me to look at the placement test isolation issue then could you summarize what is the problem? is there a simple reproduction of the problem I can do locally to start with?11:33
*** rcernin has quit IRC11:35
khomesh24#openstack-nova Hello, can someone check the bug https://bugs.launchpad.net/nova/+bug/1913016?11:40
openstackLaunchpad bug 1913016 in OpenStack Compute (nova) "nova api os-resetState should not reset the state when VM is shelved_offloaded" [Undecided,New] - Assigned to Khomesh Thakre (khomesh24)11:40
sean-k-mooneygibi: i think the issue is each of the gabby test files are written such that they need to be run indepently11:42
lyarwoodkhomesh24: that looks valid to me, did you have a patch?11:42
sean-k-mooneyfor example https://github.com/openstack/placement/blob/master/placement/tests/functional/gabbits/shared-resources.yaml11:43
sean-k-mooneyneeds to be run with its own placment instance11:44
sean-k-mooneyalthough that is proably not the best example since it wont conflict with others since other dont create inventoies of CUSTOM_MAGIC11:46
stephenfingibi: I just figured it out11:47
stephenfinhttps://storyboard.openstack.org/#!/story/200855611:47
stephenfinpushing a fix shortly11:48
gibisean-k-mooney, stephenfin: thanks, then I will not dig into it11:50
sean-k-mooneystephenfin: i assume that is because we are using a global object at some point and not mockign it propertly in the tests11:51
*** tbachman has quit IRC11:52
stephenfinyes, the RuleDefault objects provided in placement.policies.*11:52
gibisean-k-mooney: replied in https://review.opendev.org/c/openstack/nova/+/757109 I think you missed the @retries decorator11:59
khomesh24lyarwood, no i wanted to valid first12:00
*** kevko has quit IRC12:07
lyarwoodkhomesh24: kk for me it's valid, I've marked the bug as confirmed12:07
lyarwoodkhomesh24: it doesn't make sense to allow an admin to reset to active if the instance is offloaded12:07
khomesh24lyarwood, ack, i will start working on the patch then, thank you12:08
gibilyarwood, khomesh24: do we then make os-resetState smart for other non valid reality-db state combinations too?12:12
gibifor me os-resetState is a blunt instrument, and the implementation also suggest that (it validates nothing)12:13
lyarwoodgibi: I wouldn't be opposed to it tbh, anything we can do to avoid obvious PEBKAC is good with me12:13
gibilyarwood: so when the admin reset to ACTIVE then we should call down to the compute to validate that the instance is really running happily?12:14
lyarwoodgibi: no I'd just sanity check the vm_state12:14
lyarwoodgibi: SHELVED_OFFLOADED to ACTIVE obviously isn't valid12:15
lyarwoodgibi: https://docs.openstack.org/api-ref/compute/?expanded=reset-server-state-os-resetstate-action-detail#reset-server-state-os-resetstate-action the docs are also wrong, you can set any valid vm_state AFAICT12:16
gibilyarwood: is it not valid as ACTIVE would need instance.host set but we assume that in SHELVED_OFFLOADED the instance.host is None?12:16
lyarwoodgibi: correct, any further action would just fail12:16
lyarwoodgibi: I don't want to block admins from fixing actual issues with instances12:17
lyarwoodgibi: but I can't see how allowing SHELVED_OFFLOADED to ACTIVE would help in any situation tbh12:17
lyarwoodgibi: lets wait for a patch to emerge and we can thrash this out on the review12:17
gibilyarwood: the api schema restricts the new state to enum': ['active', 'error']12:17
lyarwoodah12:17
lyarwoodthanks12:18
gibilyarwood: OK, then I would not depend on SHELVED_OFFLOADED but depend on instance.host is not None, when resetting to ACTIVE12:18
gibithat would catch ERROR -> ACTIVE for VM that was never scheduled to a host12:18
gibi(and other edge cases, I guess)12:19
*** ociuhandu has quit IRC12:19
*** ociuhandu has joined #openstack-nova12:19
gibilyarwood: also I would add warning to the API ref that resetting state is dangerous12:20
*** ociuhandu has quit IRC12:24
openstackgerritStephen Finucane proposed openstack/placement master: policy: Suppress policy deprecation warnings  https://review.opendev.org/c/openstack/placement/+/77185212:25
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for resource providers  https://review.opendev.org/c/openstack/placement/+/76024012:25
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for aggregates  https://review.opendev.org/c/openstack/placement/+/76023512:25
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for allocations  https://review.opendev.org/c/openstack/placement/+/76023612:25
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for allocation candidates  https://review.opendev.org/c/openstack/placement/+/76023712:25
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for inventories  https://review.opendev.org/c/openstack/placement/+/76023812:25
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for resource classes  https://review.opendev.org/c/openstack/placement/+/76023912:25
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for traits  https://review.opendev.org/c/openstack/placement/+/76024112:25
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for usage  https://review.opendev.org/c/openstack/placement/+/76024212:25
openstackgerritStephen Finucane proposed openstack/placement master: Allow project readers to see project-specific usages  https://review.opendev.org/c/openstack/placement/+/77196412:25
openstackgerritStephen Finucane proposed openstack/placement master: Refactor usage tests to properly test project personas  https://review.opendev.org/c/openstack/placement/+/77206112:25
openstackgerritStephen Finucane proposed openstack/placement master: policy: Remove the deprecated 'placement' rule  https://review.opendev.org/c/openstack/placement/+/77233312:25
openstackgerritStephen Finucane proposed openstack/placement master: policy: Deprecate 'admin_api' rule  https://review.opendev.org/c/openstack/placement/+/77233412:25
openstackgerritStephen Finucane proposed openstack/placement master: policy: Don't persist default rule changes in tests  https://review.opendev.org/c/openstack/placement/+/77250812:25
gibisean-k-mooney: replied here too https://review.opendev.org/c/openstack/nova/+/76972012:27
*** khomesh24 has quit IRC12:34
*** LinPeiWen has quit IRC12:42
*** kevko has joined #openstack-nova12:55
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for resource providers  https://review.opendev.org/c/openstack/placement/+/76024012:55
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for aggregates  https://review.opendev.org/c/openstack/placement/+/76023512:55
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for allocations  https://review.opendev.org/c/openstack/placement/+/76023612:55
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for allocation candidates  https://review.opendev.org/c/openstack/placement/+/76023712:55
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for inventories  https://review.opendev.org/c/openstack/placement/+/76023812:55
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for resource classes  https://review.opendev.org/c/openstack/placement/+/76023912:55
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for traits  https://review.opendev.org/c/openstack/placement/+/76024112:55
openstackgerritStephen Finucane proposed openstack/placement master: Implement secure RBAC for usage  https://review.opendev.org/c/openstack/placement/+/76024212:55
openstackgerritStephen Finucane proposed openstack/placement master: Allow project readers to see project-specific usages  https://review.opendev.org/c/openstack/placement/+/77196412:55
openstackgerritStephen Finucane proposed openstack/placement master: Refactor usage tests to properly test project personas  https://review.opendev.org/c/openstack/placement/+/77206112:55
openstackgerritStephen Finucane proposed openstack/placement master: policy: Remove the deprecated 'placement' rule  https://review.opendev.org/c/openstack/placement/+/77233312:55
openstackgerritStephen Finucane proposed openstack/placement master: policy: Deprecate 'admin_api' rule  https://review.opendev.org/c/openstack/placement/+/77233412:55
gibisean-k-mooney: and here too https://review.opendev.org/c/openstack/nova/+/75653012:55
*** ociuhandu has joined #openstack-nova12:56
*** rpittau|bbl is now known as rpittau12:59
stephenfinjohnthetubaguy: If you have any time in the next fortnight, your attention would be appreciated on https://review.opendev.org/q/topic:%22secure-rbac%22+project:openstack/placement+is:open13:02
sean-k-mooneygibi: you are right i did miss the retry decorator13:09
sean-k-mooneygibi: and yes i know mypy is smart enough to deduce types when you initalise something directly but i dont think we should actully rely on that in general13:11
sean-k-mooneygibi: that is just a prefernce however not a -1 thing13:12
sean-k-mooneywell i dont know i change my mind on that some times13:13
sean-k-mooneyi do sometimes use that to say it returns/accpets  dict or list13:13
gibimypy requires you to add typehint for func signature (or it will use Any) but in function body mypy does proper deducing or will raise an error and ask you to add hints13:20
gibiif the type is not clear13:21
sean-k-mooneyya13:22
openstackgerritMerged openstack/nova stable/victoria: compute: Lock by instance.uuid lock during swap_volume  https://review.opendev.org/c/openstack/nova/+/75873013:23
sean-k-mooneygibi: +1 on both of those reviews13:26
gibithanks13:28
sean-k-mooneylyarwood: bauzas:   stephenfin and I are both +2/+1 on gibi's qos seriese https://review.opendev.org/q/topic:%22bp%252Fsupport-interface-attach-with-qos-ports%22+(status:open%20OR%20status:merged) do either of ye have time to review them and be the second +2+w13:30
*** tbachman has joined #openstack-nova13:30
gibiwould be appreciated ^^13:30
*** martinkennelly has joined #openstack-nova13:33
*** macz_ has joined #openstack-nova13:45
*** macz_ has quit IRC13:50
*** mlavalle has joined #openstack-nova13:57
gmannbrinzhang: ack, with change in routes.py it should detect the new API. let me check today13:58
*** spatel has joined #openstack-nova14:02
*** macz_ has joined #openstack-nova14:06
*** macz_ has quit IRC14:11
nightmare_unrealbrinzhang:  can you help me with this one ? https://storyboard.openstack.org/#!/story/200855314:11
*** stand has quit IRC14:24
*** brinzhang has quit IRC14:25
*** brinzhang has joined #openstack-nova14:25
*** stand has joined #openstack-nova14:26
*** zenkuro has quit IRC14:26
*** zenkuro has joined #openstack-nova14:27
*** songwenping_ has quit IRC14:27
*** songwenping_ has joined #openstack-nova14:28
bauzassean-k-mooney: ack, sorry for the delay, but I'll try to look at it14:28
bauzasI actually promised some time for gibi :D14:29
sean-k-mooneybauzas: :) while the series is longish most patches are small and self contaiend so its quite straight forward to review.14:30
*** stand has quit IRC14:43
openstackgerritMerged openstack/nova master: db: Compact Icehouse database migrations  https://review.opendev.org/c/openstack/nova/+/75839414:44
*** dave-mccowan has joined #openstack-nova14:44
*** nweinber has joined #openstack-nova14:46
*** dave-mccowan has quit IRC14:46
*** brinzhang has quit IRC14:48
*** bbowen has quit IRC14:48
openstackgerritLance Bragstad proposed openstack/placement master: trivial: Remove TODO in LegacyRBACPolicyFixture  https://review.opendev.org/c/openstack/placement/+/77253514:49
*** brinzhang has joined #openstack-nova14:51
*** brinzhang has quit IRC14:51
*** brinzhang has joined #openstack-nova14:52
*** bbowen has joined #openstack-nova14:53
stephenfinlbragstad: Replied on https://review.opendev.org/c/openstack/placement/+/76023815:03
lyarwoodsean-k-mooney: yup I can try to look today15:03
*** ralonsoh has quit IRC15:04
*** ralonsoh has joined #openstack-nova15:04
lbragstadstephenfin ack - thanks15:15
*** gary_perkins has quit IRC15:28
*** dklyle has joined #openstack-nova15:45
lyarwoodgibi / dansmith ; https://review.opendev.org/c/openstack/nova/+/771837 should be ready to go now both of the tests are fixed15:47
gibilyarwood: ack15:48
*** sapd1_x has joined #openstack-nova15:54
*** ociuhandu has quit IRC15:56
*** ociuhandu has joined #openstack-nova15:57
openstackgerritStephen Finucane proposed openstack/placement master: Expand legacy tests for inventories  https://review.opendev.org/c/openstack/placement/+/77256715:58
stephenfinlbragstad: ^15:58
stephenfinThat's super tedious work. I don't envy you /o\15:58
lbragstad:)15:58
stephenfin(and won't be pulling my +2s because they're missing, tbh. I can do without them)15:59
lbragstadi'm in the middle of attempting the same for cinder - but thanks for proposing those15:59
lbragstadyeah - i think that series looks good and i can propose the followup with additional coverage16:00
stephenfinagreed16:00
stephenfinwe'll see what gmann thinks16:00
*** ociuhandu has quit IRC16:01
*** macz_ has joined #openstack-nova16:02
*** macz_ has quit IRC16:02
*** ociuhandu has joined #openstack-nova16:04
dansmithlyarwood: question for you here: https://review.opendev.org/c/openstack/tempest/+/77195816:05
dansmithnot sure I understand what actually changed16:06
*** kevko has quit IRC16:06
*** ysirndjuro has joined #openstack-nova16:07
gmannstephenfin: lbragstad ack, I will check config things on policy init locally as config setting come into pic when test run and start the policy enforcement instead of policy__init__. but I am ok with the current approach.16:07
gmannwill review those today16:07
gmannconfig things are similar to what we do in nova. init policy first and then switch the config flag for enforce scope16:08
ysirndjuroIs there a way to bake "cpu mode='host-passthrough'" flag inside an image or flavor for nested virt support in an on demand VM, so far the only docs i've found have been from devstack and involve direct virsh edit to an existing VM.16:10
*** jamesdenton has quit IRC16:11
*** jamesden_ has joined #openstack-nova16:11
ysirndjuroor do I just slap 'cpu_mode=host-passthrough' in nova.conf and call it a day16:12
sean-k-mooneyysirndjuro:no that is intetniolly not supported16:13
lyarwooddansmith: the tests were just creating a fresh image instead of reusing CONF.compute.image_ref, I assumed that borked caching in the job and resulted in us hitting the never_download_image_if_on_rbd check16:13
* lyarwood looks16:13
sean-k-mooneyusers cannot set the cpu_mode16:13
sean-k-mooneyso we dont support it in the image16:13
dansmithlyarwood: I don't think so16:13
dansmithlyarwood: in that job, we upload the image initially to the file backend in glance, and make nova auto copy it to ceph on the first boot16:14
dansmithlyarwood: so just creating another image in the file repo should do the same thing16:14
sean-k-mooneyand we dont support setting the mode in the flavor since its specific to libvirt and we dont want to encurage a different flavor per cpu model16:14
sean-k-mooneyysirndjuro: so either set it in your nova.conf via your local.conf16:15
dansmithlyarwood: I'm thinking there's some cinder interaction that tries to do something with that image before it has been moved to ceph, but then your change won't actually make any difference if that test happens to run before any other job that just boots an instance from the normal image16:15
sean-k-mooneyor modify it and restart n-cpu16:15
ysirndjurosean-k-mooney: thx, yeah figured as much there, was leaning towards a flavor config, feelsbad16:15
sean-k-mooneyysirndjuro: this is more or less my default local.conf http://paste.openstack.org/show/801996/16:16
lyarwooddansmith: pretty sure the volume is empty16:16
lyarwooddansmith: let me check16:17
sean-k-mooneyysirndjuro: it has it set so you can copy that16:17
lyarwooddansmith: yeah the volume shouldn't reference any image ref16:17
dansmithlyarwood: okay well then I don't see what your change would actually do differently (in the case where that test happens to run first16:18
lyarwood49942 Jan 20 22:28:53.068420 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: WARNING nova.virt.libvirt.driver [None req-3a8ebd8e-b22a-446f-ad35-16b12e7a1182 tempest-TestEncryptedCinderVolumes-88193216 tempest-TestEncryptedCinderVolumes-88193216] Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is not on my ceph and [workarounds]/never_download_image_if_on_rbd=True; refusing to fetch and16:18
lyarwoodupload.: nova.exception.ImageUnaccep      table: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format16:18
lyarwood^ that's the error with the original image16:18
dansmithyeah16:18
dansmithis that in a boot or something else?16:19
lyarwoodthat's on boot16:19
ysirndjurosean-k-mooney: thx for the sample, i've seen that config option already, but was looking to avoid enabling it on every vm on that node, just specific use case where it's needed for CI/CD to run packer builds, but with on-demand capability versus having an openstack admin configure every vm CI spins up16:20
sean-k-mooneydansmith: by the way i havent forgoten about testing your devstack patch. im going to test it slightly differently then the gate would in that i did a stack on the node, now im going to unstack and stack again use that as my base time and then unstace and stack with your patch16:20
dansmithlyarwood: so nova should ask glance to copy that to the ceph store first, just like it initially did for the image_ref one16:20
dansmithsean-k-mooney: thanks16:20
sean-k-mooneydansmith: that should elimiate any network overhead16:20
lyarwooddansmith: would that be rejected if the disk format isn't RAW?16:21
dansmithlyarwood: ...if it's a normal boot all the same as the original.. that's why I'm wondering what else might be different16:21
sean-k-mooneyysirndjuro: you can kind fo do what you want if you define your own cpu model in the libvirt directorys16:21
dansmithlyarwood: mmmmaybe, what format is the image they created?16:22
dansmithlemme look at the conditions for that again16:22
sean-k-mooneyysirndjuro: if you use cpu_mode=custom then cpu_model=<default model you want>,<nested-virt-modele> then have a special flavor that request vmx i belive that willl work16:22
lyarwooddansmith: I see qcow2 in the request to glance16:23
dansmithah, hmm16:23
sean-k-mooneyysirndjuro: you should be able to use https://specs.openstack.org/openstack/nova-specs/specs/train/implemented/cpu-model-selection.html to achive your usecase16:25
ysirndjuroliterally just dmd u to verify that link haha16:25
sean-k-mooneyysirndjuro: but you will need to define a new cpu model with vmx16:25
dansmithlyarwood: ah, okay, yeah.. we don't even get to the download code because of this in clone: if image_meta.get('disk_format') not in ['raw', 'iso']:16:25
lyarwooddansmith: right it's raising before we get to the copy16:25
ysirndjuroty, will play with that16:25
lyarwooddansmith: snap16:25
dansmithokay, cool, now I'm a believer :)16:25
lyarwoodwonderful16:26
*** artom has quit IRC16:28
*** artom has joined #openstack-nova16:29
*** ociuhandu_ has joined #openstack-nova16:31
*** ociuhandu has quit IRC16:35
*** ociuhandu_ has quit IRC16:36
*** gary_perkins has joined #openstack-nova16:36
*** gyee has joined #openstack-nova16:40
*** zenkuro has quit IRC16:43
*** mlavalle has quit IRC16:44
*** mlavalle has joined #openstack-nova16:45
*** ociuhandu has joined #openstack-nova16:49
bauzasdansmith: man, you mean a belieber ? ;)16:50
dansmithbauzas: in this *one* thing :)16:52
dansmithunless that's a justin bieber joke, in which case.. no.16:52
* bauzas can't imagine it16:52
bauzasyeah, was just a bad joke16:53
* bauzas goes outside16:53
*** ociuhandu has quit IRC16:53
stephenfinbauzas: bad jokes mean you have to go outside?16:54
stephenfinShe has you well trained ;)16:54
bauzasstephenfin: indeed :)16:54
*** zenkuro has joined #openstack-nova16:58
*** nightmare_unreal has quit IRC17:00
sean-k-mooneydansmith: so when the system is up to date and the repos are already there it looks like the async task change does not make much of a difference its better but only slightly http://paste.openstack.org/show/802006/17:11
openstackgerritBalazs Gibizer proposed openstack/nova master: libvirt: add AsyncDeviceDetachEventsHandler  https://review.opendev.org/c/openstack/nova/+/77238117:11
openstackgerritBalazs Gibizer proposed openstack/nova master: libvirt: allow querying devices from the persistent domain  https://review.opendev.org/c/openstack/nova/+/77238317:11
openstackgerritBalazs Gibizer proposed openstack/nova master: libvirt: parse alias out from device config  https://review.opendev.org/c/openstack/nova/+/77238417:11
sean-k-mooneydansmith: i can however try it again with a clean vm and see what teh delta is then17:12
sean-k-mooneylooking at the console output too i did not see any real change either so it does not seam to affect the debugablity of things17:13
openstackgerritBalazs Gibizer proposed openstack/nova master: Replace blind retry with libvirt event waiting in detach  https://review.opendev.org/c/openstack/nova/+/77024617:14
dansmithsean-k-mooney: did you enable async? doesn't look like it17:16
dansmithDEVSTACK_PARALLEL=True17:16
sean-k-mooneydansmith: it looks like its saveing between 5 and 16 second over a 830 seconds17:16
sean-k-mooneyoh hehe no17:16
sean-k-mooneywell in that case i have a better base line17:16
sean-k-mooneyill let it restack again17:16
dansmithsean-k-mooney: you should see a timing compotent for async_wait once you do17:16
sean-k-mooneyok i was seeign async task prinnted but i guess it was blocking17:17
dansmithyeah it'll still print those things but wait instead of spawn17:17
dansmithsean-k-mooney: how many vcpus on the vm you're using?17:18
sean-k-mooney817:18
dansmithack17:18
dansmiththere are some things that are actually cpu bound, but if you only had one it wouldn't help17:19
dansmiththe db_syncs for example17:19
sean-k-mooneythis is like a upstream node so 8 cores and 8GB of ram although it has 2 numa nodes and nested vert17:19
sean-k-mooneyya looking at load17:19
sean-k-mooney15 min avgerage is like 0.7617:20
sean-k-mooneyit was using 1-2 cores max while running before17:20
sean-k-mooneyit spiked a littel when doing the db migrations17:20
sean-k-mooneyso that is where the extra cores definetly help17:21
dansmithyeah, on one of my machines it went from about 0.5 to 2.017:21
dansmithnot for the whole thing but for good portions of it17:21
sean-k-mooneyok its stacking again ill let you know in 15 mins or so17:21
sean-k-mooney13-15 mins is what i normally expect for this vm so if its better or the same then the patch is proably an improvment17:22
dansmithcool17:22
*** rpittau is now known as rpittau|afk17:27
sean-k-mooneyand done17:31
sean-k-mooneyit hit 3.78 there for a bit17:31
sean-k-mooneyhttp://paste.openstack.org/show/802007/17:32
sean-k-mooneyso that is much better17:32
dansmith30%17:33
dansmithyou're welcome :)17:33
sean-k-mooneywhat me to test this on clean vms to see what impact it has then17:33
sean-k-mooneythis is how i normallyuse devstack however17:33
sean-k-mooneyso 30% on my normal uscase it very good17:34
dansmithwell, the gate preloads the project git dirs, so it's closer to what you just tested17:34
sean-k-mooneyya it is17:34
dansmiththere is more fat to be cut, by the way17:34
dansmithI've optimized some of stack.sh and nova,17:34
sean-k-mooneyand they cache the pip and apt stuff close to the vms too17:34
dansmithbut there are things neutron does that we can parallellize I think17:34
dansmithoh I did a bunch of keystone things too17:34
sean-k-mooneydid you look at the plugins17:35
dansmithbeing able to stack in 6 minutes is a major life improvement to me17:35
sean-k-mooneyi would have to run in offlinemode to do that17:35
dansmithonly tempest, which I do some of in parallel, but we regen the venv several times for reasons I don't understand, but we could probably improve there too17:35
sean-k-mooneyya i have seen that17:36
sean-k-mooneyits not clear to me why either17:36
sean-k-mooneythe local pip cache helps but its not perfect17:36
sean-k-mooneydansmith: what was your normal stack time before out of interest17:37
dansmithsean-k-mooney: on my usual stripped-down config, it's 513s serialized, 402s parallell17:38
sean-k-mooneyya not bad still a nice improvment17:39
sean-k-mooneythat said even a 15min run is better then ooo17:39
dansmithmm yeah :)17:40
*** sapd1_x has quit IRC17:41
sean-k-mooneydansmith: apparently our downstreeam upgrade jobs take 11hours currently.17:42
dansmithyeah that's pretty sadface17:43
sean-k-mooneyi just can even comprehend debuging those if they fail17:43
dansmithyeah :/17:43
dansmithsean-k-mooney: if you could comment on that async patch with your results and environment, I'd appreciate it17:43
sean-k-mooneyyep i need to prep a fedra 32 vm for other things anyway today so ill do a fresh install run on that and comment when its done with the details17:44
dansmithokay thanks17:44
dansmithgmann: results from sean-k-mooney btw :) ^17:45
dansmithtl;dr his VMs are closer to upstream gate and he got 30% improvement17:45
sean-k-mooneyi can test this in my ci later in the week proably too  need to do some maintance on it before i do but i have some other things to rebase and backport first so wont get to it for a while17:46
dansmithsean-k-mooney: was that a fairly fat devstack config? like all the normal services?17:47
sean-k-mooneyyes an no ill past bin it17:48
sean-k-mooneyhttp://paste.openstack.org/show/802010/17:49
sean-k-mooneyno swift or heat but i hav ecinder and horizon17:49
*** zoharm has quit IRC17:49
gmanndansmith: sean-k-mooney nice. are you running it on fresh machine with/wihtout devstack-parallel or with/without unstack/stack ?17:49
sean-k-mooneyso nova,neutron,placement,cinder,glance,horizon,tempest with ml2/ovs17:50
dansmithgmann: when I test, I run it with/without parallel on a "warmed up" machine, i.e. where /opt/stack already has the projects cloned, like gate17:50
dansmithwe could definitely parallelize the cloning of all the projects too17:50
sean-k-mooneygmann: basically the same for my test17:51
sean-k-mooneygmann: i stacked then unstack17:51
dansmithsean-k-mooney: ack, okay mine is no cinder or horizin17:51
dansmithright, stack/unstack/stack17:51
sean-k-mooneythen stacked again to get the baseline time unstackted. and stacked in parallel mode17:51
sean-k-mooneyso all the pacakge and repos should be cached or clonned17:51
gmannyeah clone is not something to count in this17:51
*** zenkuro has quit IRC17:52
sean-k-mooneyi did not have OFFLINE=True which i can do that would disable all package installs and clones but it should not really be a factor in my current testing17:54
sean-k-mooneyill let ye both know how the fresh install goes17:54
dansmithonce the system is warmed up I don't think that would do much anyway, right?17:55
sean-k-mooneyit will prevent it even trying to do apt/dnf install i dont think it really affect pip however17:56
sean-k-mooneyso when its primmed no not really17:56
dansmithack, but apt should mostly just say "yeah already got that"17:56
dansmithminor improvement maybe but nothing major17:56
dansmithin the noise I expect17:56
sean-k-mooneyyep it just has to check the index is up to date17:56
sean-k-mooneyalso i have gigabit networking at home so its going to be pretty fast considing i sit less then 2 miles form where the irish henet mirrors are hosted17:57
sean-k-mooneythere hosted in the university i live beside17:58
sean-k-mooney64 bytes from ftp-node-prod-bl-01.heanet.ie (193.1.193.10): icmp_seq=1 ttl=58 time=7.50 ms17:59
sean-k-mooneyvs 64 bytes from 172.20.4.244: icmp_seq=15 ttl=62 time=3.41 ms to my vm :)18:01
*** tesseract has quit IRC18:16
*** k_mouza has quit IRC18:48
sean-k-mooneylooks like my clean fedora run is going to fail because the cirros download is hanging18:58
sean-k-mooneyso ill just post my old results and node details to the devstack review18:58
*** ianw_pto is now known as ianw18:59
openstackgerritMerged openstack/nova master: Move query param generation to RequestGroup o.vo  https://review.opendev.org/c/openstack/nova/+/75689419:12
*** andrewbonney has quit IRC19:16
gmannstephenfin: lbragstad should not  placement_policy.reset() take care of cleaning all default or registered rules? https://review.opendev.org/c/openstack/placement/+/772508/1/placement/tests/unit/policy_fixture.py#3619:33
gmannI am not completely understanding how it is causing the issue with  placement_policy.reset() in test. though i agree on fixing it on oslo policy side19:33
gmannor something i am missing here19:34
*** hamalq has joined #openstack-nova19:36
dansmithsean-k-mooney: thanks19:58
*** sapd1_x has joined #openstack-nova20:05
*** sapd1 has quit IRC20:06
gmannlbragstad: before I proceed on other patches in that series, one comment about SYSTEM_ADMIN https://review.opendev.org/c/openstack/placement/+/760240/21/placement/policies/base.py#2020:15
gmannI think we do not need SYSTEM_ADMIN in placement as such20:15
gmannand this way is better and avoid rule deprecation - https://review.opendev.org/c/openstack/placement/+/760235/10/placement/policies/aggregate.py#5220:16
*** ralonsoh has quit IRC20:42
*** brinzhang_ has joined #openstack-nova20:44
lbragstadgmann i think that's true with the exception of the usage api since it supports project and system scope?20:45
lbragstadactually - not if enforce_scope=True20:46
gmannlbragstad: yeah usage API use only reader right20:47
*** brinzhang has quit IRC20:47
gmannif enforce_scope=True then scope_type will care as existing scope is system for admin API20:48
gmannonly use case i see is if enforce_scope=False , enforce_new_defaults=True but technically there is no new default for admin APIs its admin only right20:51
*** dtantsur is now known as dtantsur|afk20:55
*** nweinber has quit IRC21:02
*** k_mouza has joined #openstack-nova21:06
*** k_mouza has quit IRC21:10
openstackgerritmelanie witt proposed openstack/nova stable/ussuri: compute: Lock by instance.uuid lock during swap_volume  https://review.opendev.org/c/openstack/nova/+/75873221:12
*** hamalq has quit IRC21:21
*** hamalq has joined #openstack-nova21:21
*** xek_ has quit IRC21:24
*** k_mouza has joined #openstack-nova21:25
*** k_mouza has quit IRC21:30
*** nweinber has joined #openstack-nova21:34
*** raildo has quit IRC21:47
*** nweinber has quit IRC22:02
*** nweinber has joined #openstack-nova22:05
*** slaweq has quit IRC22:08
*** nweinber has quit IRC22:10
*** rcernin has joined #openstack-nova22:16
*** rcernin has quit IRC22:24
*** rcernin has joined #openstack-nova22:24
*** k_mouza has joined #openstack-nova22:26
*** k_mouza has quit IRC22:31
*** spatel has quit IRC22:50
*** tbachman_ has joined #openstack-nova22:56
*** tbachman has quit IRC22:58
*** tbachman_ is now known as tbachman22:58
openstackgerritMerged openstack/nova master: db: Compact Juno database migration  https://review.opendev.org/c/openstack/nova/+/75839523:16
*** tosky has quit IRC23:18
*** k_mouza has joined #openstack-nova23:26
*** tbachman_ has joined #openstack-nova23:26
*** tbachman has quit IRC23:27
*** tbachman_ is now known as tbachman23:27
*** k_mouza has quit IRC23:30
*** admin0 has quit IRC23:39
*** ircuser-1 has joined #openstack-nova23:46
sean-k-mooneylbragstad: gmann  im going to stop reviewing the palcemnt series until ye have a chance ot responed regarding defineing SYTEM_ADMIN as a dict and preferably in oslo policy eventually23:56
sean-k-mooneyi really think that SYSTEM_ADMIN should be a common shared defintion across all poject and not something that can vary between cloud or services.23:58
gmannsean-k-mooney: lbragstad stephenfin replied on https://review.opendev.org/c/openstack/placement/+/760240/21/placement/policies/base.py#2023:59
*** admin0 has joined #openstack-nova23:59
gmannsean-k-mooney: we can move to common place but there are few challenge in doing that23:59
« Monday, 2021-01-25 Index Wednesday, 2021-01-27 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!