| *** tetsuro has quit IRC | 00:04 | |
| *** tetsuro has joined #openstack-nova | 00:05 | |
| *** tetsuro has quit IRC | 00:07 | |
| *** tetsuro has joined #openstack-nova | 00:19 | |
| *** tetsuro has quit IRC | 00:20 | |
| *** tetsuro has joined #openstack-nova | 00:21 | |
| *** spotz has joined #openstack-nova | 00:25 | |
| *** hongbin has quit IRC | 00:29 | |
| *** hongbin has joined #openstack-nova | 00:40 | |
| *** cataling has joined #openstack-nova | 00:58 | |
| *** dlbewley has quit IRC | 00:59 | |
| *** dlbewley has joined #openstack-nova | 00:59 | |
| *** markvoelker has joined #openstack-nova | 01:25 | |
| *** songwenping__ has quit IRC | 01:25 | |
| *** swp20 has joined #openstack-nova | 01:26 | |
| *** markvoelker has quit IRC | 01:29 | |
| *** tony_su has joined #openstack-nova | 01:33 | |
| *** songwenping_ has joined #openstack-nova | 01:47 | |
| *** dlbewley has quit IRC | 01:47 | |
| *** dlbewley has joined #openstack-nova | 01:47 | |
| *** swp20 has quit IRC | 01:50 | |
| *** xiaolin has joined #openstack-nova | 01:52 | |
| *** songwenping_ has quit IRC | 02:14 | |
| *** dolpher has joined #openstack-nova | 02:27 | |
| *** tony_su has quit IRC | 02:35 | |
| *** rcernin_ has joined #openstack-nova | 02:58 | |
| *** rcernin has quit IRC | 02:59 | |
| *** rcernin_ has quit IRC | 03:16 | |
| *** markvoelker has joined #openstack-nova | 03:26 | |
| *** markvoelker has quit IRC | 03:30 | |
| *** dlbewley has quit IRC | 03:31 | |
| *** dlbewley has joined #openstack-nova | 03:31 | |
| *** cataling has quit IRC | 03:32 | |
| *** rcernin_ has joined #openstack-nova | 03:32 | |
| *** Liang__ has joined #openstack-nova | 03:34 | |
| *** psachin has joined #openstack-nova | 03:39 | |
| *** rcernin_ has quit IRC | 03:45 | |
| *** rcernin has joined #openstack-nova | 03:45 | |
| *** markvoelker has joined #openstack-nova | 04:14 | |
| *** ratailor has joined #openstack-nova | 04:16 | |
| *** markvoelker has quit IRC | 04:19 | |
| *** dlbewley has quit IRC | 04:19 | |
| *** dlbewley has joined #openstack-nova | 04:20 | |
| *** slaweq has joined #openstack-nova | 04:20 | |
| *** vishalmanchanda has joined #openstack-nova | 04:29 | |
| *** dlbewley has quit IRC | 04:29 | |
| *** dlbewley has joined #openstack-nova | 04:30 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-nova | 04:33 | |
| *** hongbin has quit IRC | 04:39 | |
| *** dlbewley has quit IRC | 04:47 | |
| *** dlbewley has joined #openstack-nova | 04:47 | |
| *** ociuhandu has joined #openstack-nova | 04:48 | |
| *** Liang__ has quit IRC | 04:49 | |
| *** Liang__ has joined #openstack-nova | 04:51 | |
| *** ociuhandu has quit IRC | 04:53 | |
| *** dlbewley has quit IRC | 05:21 | |
| *** dlbewley has joined #openstack-nova | 05:21 | |
| *** dlbewley has quit IRC | 05:31 | |
| *** dlbewley has joined #openstack-nova | 05:31 | |
| *** rcernin has quit IRC | 05:32 | |
| *** udesale has joined #openstack-nova | 05:33 | |
| *** rcernin has joined #openstack-nova | 05:40 | |
| *** dlbewley has quit IRC | 05:41 | |
| *** dlbewley has joined #openstack-nova | 05:41 | |
| *** links has joined #openstack-nova | 05:45 | |
| openstackgerrit | Harshavardhan Metla proposed openstack/nova master: Moved the quoted section https://review.opendev.org/737215 | 05:56 |
|---|---|---|
| *** markvoelker has joined #openstack-nova | 06:15 | |
| *** ociuhandu has joined #openstack-nova | 06:20 | |
| *** markvoelker has quit IRC | 06:20 | |
| *** rpittau|afk is now known as rpittau | 06:21 | |
| *** dlbewley has quit IRC | 06:50 | |
| *** dlbewley has joined #openstack-nova | 06:51 | |
| *** jsuchome has joined #openstack-nova | 06:53 | |
| *** markvoelker has joined #openstack-nova | 06:54 | |
| *** markvoelker has quit IRC | 06:59 | |
| *** dlbewley has quit IRC | 07:00 | |
| *** dlbewley has joined #openstack-nova | 07:01 | |
| *** tesseract has joined #openstack-nova | 07:02 | |
| *** maciejjozefczyk has joined #openstack-nova | 07:09 | |
| *** dlbewley has quit IRC | 07:10 | |
| *** ttsiouts has joined #openstack-nova | 07:11 | |
| *** dlbewley has joined #openstack-nova | 07:11 | |
| *** ttsiouts has quit IRC | 07:14 | |
| *** ttsiouts has joined #openstack-nova | 07:16 | |
| *** bhagyashris is now known as bhagyashris|lunc | 07:27 | |
| *** tosky has joined #openstack-nova | 07:29 | |
| *** xek_ has joined #openstack-nova | 07:30 | |
| *** lpetrut has joined #openstack-nova | 07:40 | |
| *** dlbewley has quit IRC | 07:40 | |
| *** dlbewley has joined #openstack-nova | 07:40 | |
| bauzas | good morning Nova | 07:42 |
| *** factor has quit IRC | 07:45 | |
| *** rcernin_ has joined #openstack-nova | 07:47 | |
| *** rcernin has quit IRC | 07:47 | |
| gibi | bauzas: good morning | 07:52 |
| *** rcernin_ has quit IRC | 07:54 | |
| *** links has quit IRC | 07:55 | |
| *** nightmare_unreal has joined #openstack-nova | 07:58 | |
| *** ralonsoh has joined #openstack-nova | 07:59 | |
| *** martinkennelly has joined #openstack-nova | 08:00 | |
| *** dtantsur|afk is now known as dtantsur | 08:10 | |
| *** dpawlik6 has quit IRC | 08:18 | |
| *** martinkennelly has quit IRC | 08:21 | |
| *** martinkennelly has joined #openstack-nova | 08:23 | |
| *** markvoelker has joined #openstack-nova | 08:24 | |
| *** links has joined #openstack-nova | 08:26 | |
| *** markvoelker has quit IRC | 08:29 | |
| *** dpawlik6 has joined #openstack-nova | 08:30 | |
| *** salmankhan has joined #openstack-nova | 08:33 | |
| *** bhagyashris|lunc is now known as bhagyashris | 08:34 | |
| *** salmankhan has quit IRC | 08:36 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: DNM: Test the state of VMware NSX 3pp CI https://review.opendev.org/734114 | 08:38 |
| *** ttsiouts has quit IRC | 09:16 | |
| *** ttsiouts has joined #openstack-nova | 09:18 | |
| *** ociuhandu has quit IRC | 09:18 | |
| *** dlbewley has quit IRC | 09:18 | |
| openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Filter instances by tenant_id https://review.opendev.org/737241 | 09:19 |
| *** dlbewley has joined #openstack-nova | 09:19 | |
| openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Filter instances by tenant_id https://review.opendev.org/737241 | 09:20 |
| *** tkajinam has quit IRC | 09:21 | |
| *** brinzhang has joined #openstack-nova | 09:24 | |
| *** martinkennelly has quit IRC | 09:30 | |
| *** martinkennelly has joined #openstack-nova | 09:31 | |
| gibi | bauzas: left feedback in https://review.opendev.org/#/c/733703 | 09:38 |
| bauzas | thanks | 09:38 |
| bauzas | gibi: ack, seen your comments | 09:40 |
| bauzas | gibi: honestly, it's a good question | 09:40 |
| bauzas | I'm not opiniated but, | 09:40 |
| bauzas | if we go with the neutron direction, I like it but I won't be able to provide the implementation I think :p | 09:41 |
| bauzas | gibi: so, tbh, I like your concern | 09:41 |
| bauzas | but maybe we should discuss it with some other folks :) | 09:42 |
| brinzhang | gibi, bauzas: I submit a spec, but it has an error of docs. I cannot find which is wrong in line 22 | 09:48 |
| brinzhang | error: /home/zuul/src/opendev.org/openstack/nova-specs/doc/source/specs/victoria/approved/filter-instances-by-tenant-id.rst:22:Unknown target name: "1". | 09:49 |
| brinzhang | https://review.opendev.org/#/c/737241 | 09:49 |
| brinzhang | can you fast check, where caused the error? | 09:49 |
| gibi | bauzas: sure. this is why I said that put every neutron segment - aggregate related code in a single place in nova, so that later we can easily remove it, but I know this part is then becomes implementation detail | 09:52 |
| *** ttsiouts has quit IRC | 09:55 | |
| *** Liang__ has quit IRC | 09:59 | |
| * brinzhang has done | 10:00 | |
| openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Filter instances by tenant_id https://review.opendev.org/737241 | 10:00 |
| *** ociuhandu has joined #openstack-nova | 10:04 | |
| *** rpittau is now known as rpittau|bbl | 10:15 | |
| gibi | brinzhang: the latest PS worked for me locally | 10:19 |
| *** markvoelker has joined #openstack-nova | 10:25 | |
| *** ttsiouts has joined #openstack-nova | 10:29 | |
| *** sean-k-mooney has joined #openstack-nova | 10:29 | |
| *** markvoelker has quit IRC | 10:30 | |
| *** ttsiouts has quit IRC | 10:30 | |
| *** ttsiouts has joined #openstack-nova | 10:31 | |
| *** ociuhandu has quit IRC | 10:36 | |
| *** ttsiouts has quit IRC | 10:54 | |
| *** ttsiouts has joined #openstack-nova | 10:55 | |
| *** dlbewley has quit IRC | 11:00 | |
| *** dlbewley has joined #openstack-nova | 11:01 | |
| *** xek has joined #openstack-nova | 11:01 | |
| *** xek_ has quit IRC | 11:01 | |
| *** derekh has joined #openstack-nova | 11:08 | |
| *** ociuhandu has joined #openstack-nova | 11:27 | |
| *** ttsiouts has quit IRC | 11:42 | |
| *** ttsiouts has joined #openstack-nova | 11:44 | |
| *** raildo has joined #openstack-nova | 11:48 | |
| *** jcath has joined #openstack-nova | 12:01 | |
| jcath | friends, I try to use "openstack server add volume --device /dev/hdc instance vol-name" to attach a volume to the instance as an IDE device, but it always attach as a virtio device (qemu-kvm as hypervisor) . as I check nova/virt/libvirt/driver.py, it seems that the device name parameter is ignored... so How I can force to attach a volume on IDE bus? thanks! | 12:05 |
| *** dlbewley has quit IRC | 12:05 | |
| stephenfin | elod: Could you take a look at https://review.opendev.org/#/c/708617/ today? | 12:06 |
| *** dlbewley has joined #openstack-nova | 12:06 | |
| elod | stephenfin: yes, looking | 12:12 |
| stephenfin | ta | 12:12 |
| *** ratailor has quit IRC | 12:21 | |
| *** rpittau|bbl is now known as rpittau | 12:22 | |
| *** udesale_ has joined #openstack-nova | 12:25 | |
| *** dolpher has quit IRC | 12:26 | |
| *** markvoelker has joined #openstack-nova | 12:26 | |
| *** udesale has quit IRC | 12:27 | |
| *** markvoelker has quit IRC | 12:31 | |
| *** markvoelker has joined #openstack-nova | 12:34 | |
| *** markvoelker has quit IRC | 12:39 | |
| *** tbachman has joined #openstack-nova | 12:43 | |
| *** nweinber has joined #openstack-nova | 13:09 | |
| *** lbragstad has joined #openstack-nova | 13:12 | |
| tbarron | Is it possible via openstack/nova apis to do expose compute-host extra disks to guest VMs via somthing like pci-passthru? | 13:14 |
| gibi | dansmith, stephenfin, melwitt: sorry I was out Friday afternoon. Thanks for reviewing the image cache bugfix. However I don't like the fact that we are mixing the question of "is this on the same dev?" with the question "is this exists?" | 13:15 |
| tbarron | This has been suggested as a way for kubernetes clusters running with nova vms to do software defined storage (like ceph) without indirection | 13:15 |
| sean-k-mooney | tbarron: in general no. if you have nvme disk you can confiture them for pci pasthough but it expects stateless pci device | 13:15 |
| tbarron | in the data path like when they use RBD backed nova ephemeral or cinder storage | 13:16 |
| sean-k-mooney | so we will not correctly clean them when a vm is deleted and there data will not be copied if you mvoe the guest | 13:16 |
| sean-k-mooney | tbarron: so basically today no | 13:16 |
| sean-k-mooney | tbarron: there is no way to do that | 13:16 |
| stephenfin | gibi: Yeah, I'm on the fence about that now too | 13:16 |
| stephenfin | especially given the confusion it resulted in | 13:17 |
| stephenfin | I don't know what dansmith and melwitt settled on when I left | 13:17 |
| tbarron | sean-k-mooney: thanks, I already said "no" but thought I better check with those who really know :D | 13:17 |
| gibi | stephenfin: I will try to come up with a better factored solution in the next PS | 13:17 |
| stephenfin | ack | 13:18 |
| sean-k-mooney | tbarron: the quickest way to enable something like that would be to write a cyborg dirver to manage disk on the host and then extend libivt to accpet disks form cyborg | 13:18 |
| tbarron | sean-k-mooney: interesting, and that makes sense | 13:21 |
| sean-k-mooney | tbarron: a disk is not really an acclearator but if you think of cyborg as a generic device managment service it think it fit. espacially when you consider that imaging or erasing a disk is basically the same as programing or reseting an fpga | 13:22 |
| sean-k-mooney | just with less $$$ for the hardware | 13:23 |
| tbarron | sean-k-mooney: yeah, it fits with the general expose-hardware-features to VMs direction, I get it | 13:24 |
| *** mriedem has joined #openstack-nova | 13:24 | |
| tbarron | and is somewhat in tension with present compute-instances as abstractions idea, thouhh I say "tension" and not contradiction | 13:25 |
| tbarron | we have a similar tension in storage where different backends have different capabilities and it's tricky figuring which to expose as abstractions | 13:26 |
| *** eharney has joined #openstack-nova | 13:26 | |
| *** sangeet has joined #openstack-nova | 13:31 | |
| sean-k-mooney | tbarron: well the tension is relived in a sense that it would be abstracted via a device-profile and we would should based on the aviablity of the resouce like any other | 13:34 |
| sean-k-mooney | tbarron: cinder does nto quite workin this usecase since you dont have an ideal of a local only cinder backend | 13:34 |
| dansmith | gibi: I commented | 13:35 |
| *** sangeet has quit IRC | 13:35 | |
| dansmith | gibi: making it non-public and specific to "decide if we should report zero" addresses my original concern I guess, but I don't understand what the problem currently is | 13:36 |
| gibi | dansmith: my problem that it makes a coupling between nova.virt.libvirt.imagebackend.Image.cache and nova.virt.libvirt.imagecache.ImageCacheManager.cache_dir_is_on_same_dev_as_instances_dir as the later assumes how the former will create the directory | 13:38 |
| dansmith | gibi: cache_dir is a property of the imagecache no? | 13:39 |
| gibi | for me the reasoning like "the directory does not exists therefore it occupies 0 space" is easier to accept than "the directory is on the same dev as it is not created but we know that when it is created it will be a call to mkdir that creates it on the same dev" | 13:40 |
| dansmith | are you just saying that the behavior of creating the cache_dir if it doesn't exist is something in the libvirt code? | 13:40 |
| dansmith | gibi: until the directory exists, the same exact thing is returned right? zero? | 13:40 |
| tbarron | sean-k-mooney: ack, cinder volumes are intended to have a life-cycle independent of compute instances or compute instance hosts | 13:41 |
| dansmith | once the directory exists, we'll report what we see, which will almost definitely be the same dev, but if not, we'll report the value according to how the directory is at that point | 13:41 |
| gibi | dansmith: the behavior of get_disk_usage() is the same in my PS2 and in PS5 | 13:42 |
| gibi | but I think the implementation is better strucutred in PS2 | 13:42 |
| dansmith | right, so I don't see that we're making any different assumptions | 13:42 |
| tbarron | sean-k-mooney: so the cinder lvm backend is useful for testing iscsi but not so much for production deployments | 13:43 |
| dansmith | gibi: well, I disagree because I think that a property should explode for a known condition | 13:43 |
| dansmith | gibi: but make it not a property (and rename it) and you can have that structure | 13:43 |
| gibi | dansmith: I accep that I'm ready to make that an internal helper instead of a public property | 13:43 |
| dansmith | I think a property shouild /not/ explode I meant | 13:44 |
| gibi | yeah, I agree ^^ | 13:44 |
| *** psachin has quit IRC | 13:44 | |
| gibi | just to make sure I understand your point. Is it OK for you if change the property to an private helper method? | 13:45 |
| dansmith | I don't like it, but it addresses the problem I had with PS2 | 13:45 |
| *** dlbewley has quit IRC | 13:46 | |
| gibi | why don't you like it? | 13:46 |
| *** dlbewley has joined #openstack-nova | 13:47 | |
| dansmith | well, because as it is, the property has utility beyond what you're doing here. You're just changing it to "should I report zero for cache" which is a single conditional and might as well just be in the if statement of the get_disk_usage() | 13:49 |
| dansmith | doesn't seem worth it being a helper to me | 13:49 |
| dansmith | but all I really meant is that _I_ would keep it the way it is in PS5, but it matters to me less than you, so you should change it | 13:49 |
| dansmith | what matters to me is not having that should-be-useful-but-dangerous public property | 13:50 |
| gibi | dansmith: thanks | 13:52 |
| *** dlbewley has quit IRC | 14:01 | |
| *** dlbewley has joined #openstack-nova | 14:01 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Try to make a glance multistore job https://review.opendev.org/734184 | 14:04 |
| *** markvoelker has joined #openstack-nova | 14:07 | |
| *** markvoelker has quit IRC | 14:12 | |
| *** dlbewley has quit IRC | 14:21 | |
| *** dlbewley has joined #openstack-nova | 14:22 | |
| *** mlavalle has joined #openstack-nova | 14:22 | |
| *** dklyle has joined #openstack-nova | 14:23 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Try to make a glance multistore job https://review.opendev.org/734184 | 14:25 |
| openstackgerrit | Elod Illes proposed openstack/nova stable/train: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/737279 | 14:27 |
| *** artom has joined #openstack-nova | 14:27 | |
| *** cataling has joined #openstack-nova | 14:46 | |
| *** sangeet has joined #openstack-nova | 14:55 | |
| *** beekneemech is now known as bnemec | 14:59 | |
| *** sangeet has quit IRC | 15:02 | |
| jsuchome | hey dansmith ... regular reminder about https://review.opendev.org/#/c/574301 once you have time... | 15:08 |
| *** hamalq has joined #openstack-nova | 15:27 | |
| dansmith | jsuchome: I know, I haven't forgotten | 15:28 |
| *** markvoelker has joined #openstack-nova | 15:32 | |
| *** markvoelker has quit IRC | 15:36 | |
| *** amodi has quit IRC | 15:36 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Guard against missing image cache directory https://review.opendev.org/736964 | 15:45 |
| *** gyee has joined #openstack-nova | 15:55 | |
| *** artom has quit IRC | 16:01 | |
| *** rpittau is now known as rpittau|afk | 16:01 | |
| *** artom has joined #openstack-nova | 16:02 | |
| *** brinzhang_ has joined #openstack-nova | 16:02 | |
| *** brinzhang has quit IRC | 16:06 | |
| *** ociuhandu has quit IRC | 16:08 | |
| *** dtantsur is now known as dtantsur|afk | 16:10 | |
| *** ttsiouts has quit IRC | 16:10 | |
| *** brinzhang0 has joined #openstack-nova | 16:12 | |
| *** dlbewley has quit IRC | 16:14 | |
| *** dlbewley has joined #openstack-nova | 16:15 | |
| *** brinzhang_ has quit IRC | 16:15 | |
| *** udesale_ has quit IRC | 16:21 | |
| *** markvoelker has joined #openstack-nova | 16:24 | |
| *** dlbewley has quit IRC | 16:24 | |
| *** dlbewley has joined #openstack-nova | 16:25 | |
| *** markvoelker has quit IRC | 16:29 | |
| stephenfin | melwitt: could you look at https://review.opendev.org/708617 too? | 16:29 |
| melwitt | stephenfin: sure, will do | 16:30 |
| stephenfin | thanks | 16:30 |
| *** markvoelker has joined #openstack-nova | 16:44 | |
| *** markvoelker has quit IRC | 16:48 | |
| *** ttsiouts has joined #openstack-nova | 16:50 | |
| *** lpetrut has quit IRC | 16:50 | |
| *** tesseract has quit IRC | 16:56 | |
| *** xiaolin has quit IRC | 17:02 | |
| *** derekh has quit IRC | 17:05 | |
| *** ttsiouts has quit IRC | 17:05 | |
| *** xek has quit IRC | 17:16 | |
| *** dlbewley has quit IRC | 17:20 | |
| *** dlbewley has joined #openstack-nova | 17:21 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: fakelibvirt: Remove nova-network remnants https://review.opendev.org/737329 | 17:22 |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova stable/stein: Make greande jobs n-v for EM and oldest stable https://review.opendev.org/737332 | 17:25 |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova stable/stein: Make greande jobs n-v for EM and oldest stable https://review.opendev.org/737332 | 17:27 |
| sean-k-mooney | dansmith: you can increase the job timeout in the zull.yaml if you need to for the multistore job | 17:34 |
| sean-k-mooney | dansmith: it looks like you glance api change made it this time https://zuul.opendev.org/t/openstack/build/18e4701c1a374bf09269778479160f25/log/controller/logs/etc/glance/glance-api_conf.txt | 17:35 |
| dansmith | yep, and it asked for the copy | 17:35 |
| dansmith | I think something else likely broke, looking now | 17:35 |
| dansmith | Jun 22 15:42:22.928857 ubuntu-bionic-rax-iad-0017311577 nova-compute[23701]: INFO nova.virt.libvirt.imagebackend [None req-ca48174a-0bf4-4341-8d45-fcf69cc9a3de tempest-DeleteServersAdminTestJSON-1752858908 tempest-DeleteServersAdminTestJSON-1752858908] Asking glance to copy image e6b1a7d0-ccd8-4be3-bef7-69c68fca4313 to our rbd store robust | 17:35 |
| dansmith | Jun 22 15:52:23.076886 ubuntu-bionic-rax-iad-0017311577 nova-compute[23701]: ERROR nova.compute.manager [instance: 2cb1f8e2-a6a3-4f42-b6e2-de6823c71e25] nova.exception.ImageUnacceptable: Image e6b1a7d0-ccd8-4be3-bef7-69c68fca4313 is unacceptable: Copy to store robust timed out | 17:36 |
| sean-k-mooney | it might be a slow node | 17:36 |
| sean-k-mooney | you could relax some of the times outs for image/volume creation | 17:37 |
| dansmith | it waited ten minutes | 17:37 |
| *** ttsiouts has joined #openstack-nova | 17:37 | |
| dansmith | that should be more than long enough to copy a cirros image on any node I think | 17:37 |
| sean-k-mooney | ya fair point :) | 17:38 |
| sean-k-mooney | i was more thinking it was a slow host becaue it hit the 2 hour job time out | 17:38 |
| sean-k-mooney | althougyh i guess enough 10 minute wait would have the same effect | 17:38 |
| dansmith | I think it's just because each time we went to spawn an instance, it waited ten minutes before failing, | 17:39 |
| dansmith | which linearized is enough to run the timeout | 17:39 |
| dansmith | https://zuul.opendev.org/t/openstack/build/18e4701c1a374bf09269778479160f25/log/controller/logs/screen-g-api.txt#7466 | 17:40 |
| dansmith | glance was failing to update its own property I think | 17:40 |
| *** mlavalle has quit IRC | 17:40 | |
| sean-k-mooney | right this si the image convertion https://zuul.opendev.org/t/openstack/build/18e4701c1a374bf09269778479160f25/log/controller/logs/screen-g-api.txt#440 | 17:40 |
| sean-k-mooney | so it looks like the inital import conversion worked | 17:41 |
| dansmith | the devstack conversion you mean? | 17:42 |
| dansmith | had it not, nova wouldn't have even tried to boot on it, so yeah | 17:42 |
| dansmith | and the devstack patch I have wouldn't have gotten past waiting for the image to go active | 17:42 |
| *** ttsiouts has quit IRC | 17:43 | |
| sean-k-mooney | dansmith: yes the intial devstack conversion seam to have worked fine so the failure after after the qcow has been converted to raw and stored in teh file backedn | 17:43 |
| dansmith | yep | 17:43 |
| sean-k-mooney | well if nothing else i guess glance can now use your patch to test that... | 17:43 |
| sean-k-mooney | os_glance_importing_to_stores seams like a strange name for a property on the image | 17:44 |
| dansmith | that's the task status property | 17:45 |
| sean-k-mooney | https://github.com/openstack/glance/blob/92492cf50461e214b777c707148886a8e87f340d/releasenotes/notes/import-multi-stores-3e781f2878b3134d.yaml#L25 yep | 17:45 |
| sean-k-mooney | i guess the import-form-copy is modifying that to add the rbd store | 17:46 |
| *** ralonsoh has quit IRC | 17:47 | |
| dansmith | right, the glance tasks modify that property to tell us what is happening | 17:48 |
| sean-k-mooney | dansmith: i wonder if this could be related to who owns the image | 17:52 |
| sean-k-mooney | devstack uploads it as admin correct | 17:52 |
| sean-k-mooney | but tempest is running with its own tenats | 17:52 |
| dansmith | well, that's the obvious thing, but the task should be using an admin context for this kind of metadata updating | 17:52 |
| dansmith | and they say it should | 17:52 |
| sean-k-mooney | so perhaps they do not have permission to modify that porperty | 17:52 |
| sean-k-mooney | that would be the logical thing to do yes | 17:53 |
| sean-k-mooney | but manybet its not | 17:53 |
| sean-k-mooney | https://github.com/openstack/glance/commit/1754c9e2b085ba0fc37a4369488c92a40268997a add the copy image support so im just skiming it quickly to see what it does | 17:53 |
| sean-k-mooney | home ok i dont see how the propery gets updated in that but i also dont know how glance works internally so its not suprising. | 18:01 |
| sean-k-mooney | oh time for a call... | 18:01 |
| *** dlbewley has quit IRC | 18:03 | |
| *** dlbewley has joined #openstack-nova | 18:03 | |
| *** damien_r has quit IRC | 18:04 | |
| *** links has quit IRC | 18:04 | |
| *** janno has joined #openstack-nova | 18:06 | |
| *** brinzhang_ has joined #openstack-nova | 18:12 | |
| *** ttsiouts has joined #openstack-nova | 18:15 | |
| *** brinzhang0 has quit IRC | 18:15 | |
| *** ociuhandu has joined #openstack-nova | 18:16 | |
| dansmith | melwitt: a while back I asked about getting admin credentials for glance and you pointed me to something I ignored because I decided I didn't need admin | 18:19 |
| dansmith | melwitt: do you remember that and if so can you point me again? | 18:20 |
| *** ociuhandu has quit IRC | 18:21 | |
| melwitt | heh, sec | 18:26 |
| melwitt | dansmith: it might have been this commit https://github.com/openstack/nova/commit/aab4b7a0e2504c04e08389145bcb1414dea63631 | 18:29 |
| melwitt | just as an example of a place where we needed to use an admin cred to make a particular API call | 18:29 |
| dansmith | melwitt: okay that's just a flag to the neutron client right? | 18:34 |
| dansmith | I thought there was something more general | 18:34 |
| melwitt | dansmith: yeah, I think when I linked you I was just saying, it is normal/expected for us to have to selectively use admin to call other APIs and that was a recent example of us doing it | 18:37 |
| dansmith | oh, okay, that's common in a lot of places, yeah.. what I need is a way to get admin creds to talk to glance | 18:38 |
| dansmith | I don't really know how we do that for neutron.. I think long ago we had credentials in our config, but that's gone now right? | 18:38 |
| melwitt | I don't know off the top of my head. I thought we did have creds but I don't know about them being gone. I'm looking through the code now to see if it's obvious | 18:41 |
| dansmith | I thought there was some service user thing we use now, but yeah I don't really know | 18:42 |
| *** ttsiouts has quit IRC | 18:47 | |
| melwitt | based on this code block, there are supposed to be creds used from nova.conf https://github.com/openstack/nova/blob/f1ebc15dfc8ffb7f23b2cb9879f0ca9376931a90/nova/network/neutron.py#L191 | 18:53 |
| *** dlbewley has quit IRC | 18:56 | |
| *** dlbewley has joined #openstack-nova | 18:56 | |
| melwitt | and here's a config file from a nova-next run showing what look to be service user creds for neutron and placement https://zuul.opendev.org/t/openstack/build/785733b6379b40a5982f710a62302c21/log/controller/logs/etc/nova/nova_cell1_conf.txt#40 | 19:01 |
| dansmith | melwitt: sorry in three conversations here | 19:07 |
| dansmith | melwitt: yeah, okay, I thought we had moved past that at some point, but it looks like not | 19:07 |
| melwitt | np. I'm still gathering info | 19:07 |
| melwitt | we implemented this https://specs.openstack.org/openstack/nova-specs/specs/ocata/implemented/use-service-tokens.html which says it should have docs for setting up the service user stuff in conf but I don't find any docs so far | 19:08 |
| dansmith | glance is kinda half requiring admin to do the image copy-to-rbd thing.. if that's intentional, then we'll need admin creds for glance too, which really sucks | 19:08 |
| dansmith | heh | 19:08 |
| dansmith | maybe it's done and devstack is just still using the old method? | 19:08 |
| melwitt | I'd think that unlikely... | 19:10 |
| melwitt | this is what I find for a change that went into devstack to enable service tokens. https://review.opendev.org/#/c/409329/8/lib/nova | 19:10 |
| melwitt | (I'm looking through https://review.opendev.org/#/q/topic:bp/use-service-tokens) | 19:11 |
| dansmith | melwitt: come on, be optimistic with me! :P | 19:11 |
| melwitt | lol :) | 19:11 |
| melwitt | well, the service token stuff involves setting creds in conf | 19:11 |
| melwitt | for the service user | 19:11 |
| dansmith | hmm, okay maybe the service token still means we get creds, just not creds that are general purpose admins? | 19:11 |
| dansmith | I thought it was better than that | 19:12 |
| melwitt | how that's different I don't really know | 19:12 |
| sean-k-mooney | dansmith: i dont think devstack conigure the service user stuff for us by default | 19:13 |
| sean-k-mooney | i have not looked in a while but last time i did i did not see it | 19:13 |
| melwitt | dansmith: so.... it "seems" like you would probably do a patch similar to this one https://review.opendev.org/410394 | 19:13 |
| melwitt | that ^ adds the nova-neutron interaction. and the groundwork was added in a prior patch for the nova-cinder interaction https://review.opendev.org/397399 | 19:14 |
| dansmith | ack yeah, okay | 19:15 |
| dansmith | well, I guess I'll start by arguing that we shouldn't need admin to do this | 19:15 |
| dansmith | melwitt: and just to stitch that stuff together, you think that when we do admin=True to the neutronclient, we're now getting the service user's auth token? | 19:15 |
| dansmith | the logic on L135 there is a bit confusing | 19:16 |
| melwitt | sean-k-mooney: we enable it in nova-next https://github.com/openstack/nova/blob/f1ebc15dfc8ffb7f23b2cb9879f0ca9376931a90/.zuul.yaml#L180 | 19:16 |
| sean-k-mooney | melwitt: ah ok but not in dansmith's job https://zuul.opendev.org/t/openstack/build/18e4701c1a374bf09269778479160f25/log/controller/logs/etc/nova/nova-cpu_conf.txt | 19:16 |
| sean-k-mooney | there is no service_user group in the config | 19:16 |
| melwitt | yeah you'd have to set the env var for the job | 19:17 |
| sean-k-mooney | could you just add the glance credetials like we do for neutron https://zuul.opendev.org/t/openstack/build/18e4701c1a374bf09269778479160f25/log/controller/logs/etc/nova/nova-cpu_conf.txt#43-50 | 19:17 |
| dansmith | sean-k-mooney: that's what melwitt said above | 19:18 |
| sean-k-mooney | service_user support was more fo the case where i am doing a long running request but my user token expired so we fallback to a admin service user instead | 19:18 |
| dansmith | and I'm guessing the answer is yes, but I'd like to not have to do that (meaning not need to have admin for this) | 19:18 |
| *** nightmare_unreal has quit IRC | 19:19 | |
| sean-k-mooney | ya given you dont own the image however im not surprised that glance is unhappy | 19:19 |
| sean-k-mooney | you could argue that if its a public image then maybe this should be allowed | 19:20 |
| melwitt | dansmith: yeah... agreed it looks confusing. but I think yeah, passing admin=True is having it load the auth plugin from conf, which presumably will pick up the [service_user] config section | 19:20 |
| dansmith | sean-k-mooney: it's public and the API is letting me do the operation | 19:20 |
| sean-k-mooney | melwitt: i dont think it will by default | 19:20 |
| sean-k-mooney | dansmith: as in the api is accepting the import | 19:21 |
| dansmith | right | 19:21 |
| sean-k-mooney | ya so it feels like a glance but | 19:21 |
| sean-k-mooney | *bug | 19:21 |
| sean-k-mooney | but for now you might need to use admin to work around it | 19:21 |
| dansmith | I hath already filed it thusly | 19:21 |
| sean-k-mooney | :) | 19:22 |
| sean-k-mooney | ok so ya it looks like we have the service auth support in the image module https://github.com/openstack/nova/blob/master/nova/image/glance.py#L68 | 19:24 |
| sean-k-mooney | without modifying noava code with admin=true on that call however its not going to elevate unless the token is expired so i guess you will have to do that too which kind of sucks | 19:25 |
| sean-k-mooney | i mean i guess you can do that in the DNM patch | 19:26 |
| dansmith | just setting admin=True on the nova context isn't going to do it | 19:26 |
| dansmith | I'd have to actually get an admin-granted token from keystone | 19:26 |
| sean-k-mooney | not on the context but when you create the client cant you just pass admin=true .e.g get_cline(ctx, admin=true) | 19:30 |
| sean-k-mooney | like we do with neutron https://opendev.org/openstack/nova/src/commit/f5f7c2540150c7ee7640c834d5caec31b3f5a7ab/nova/network/neutron.py#L397 | 19:30 |
| sean-k-mooney | although that is a custom get_client function https://opendev.org/openstack/nova/src/commit/f5f7c2540150c7ee7640c834d5caec31b3f5a7ab/nova/network/neutron.py#L234-L257 | 19:32 |
| sean-k-mooney | i guess not the glance module does not ever use admin currently so it does not pass it to https://opendev.org/openstack/nova/src/commit/f5f7c2540150c7ee7640c834d5caec31b3f5a7ab/nova/image/glance.py#L60 | 19:34 |
| *** slaweq has quit IRC | 19:46 | |
| *** jcath has quit IRC | 19:50 | |
| *** ttsiouts has joined #openstack-nova | 19:55 | |
| *** slaweq has joined #openstack-nova | 19:58 | |
| *** jsuchome has quit IRC | 20:12 | |
| *** vishalmanchanda has quit IRC | 20:16 | |
| *** ttsiouts has quit IRC | 20:26 | |
| *** nweinber has quit IRC | 20:28 | |
| *** markvoelker has joined #openstack-nova | 20:55 | |
| *** markvoelker has quit IRC | 20:59 | |
| *** martinkennelly has quit IRC | 21:09 | |
| *** spatel has joined #openstack-nova | 21:13 | |
| *** factor has joined #openstack-nova | 21:17 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Try to make a glance multistore job https://review.opendev.org/734184 | 21:30 |
| *** ttsiouts has joined #openstack-nova | 21:32 | |
| *** ttsiouts_ has joined #openstack-nova | 21:35 | |
| *** spatel has quit IRC | 21:36 | |
| *** maciejjozefczyk has quit IRC | 21:36 | |
| *** ttsiouts has quit IRC | 21:37 | |
| *** mriedem has left #openstack-nova | 21:37 | |
| *** ttsiouts_ has quit IRC | 21:39 | |
| *** spatel has joined #openstack-nova | 21:42 | |
| *** dlbewley has quit IRC | 21:44 | |
| *** dlbewley has joined #openstack-nova | 21:45 | |
| *** spatel has quit IRC | 21:46 | |
| *** spatel has joined #openstack-nova | 21:52 | |
| *** pmacdonnell has quit IRC | 21:55 | |
| *** pmacdonnell has joined #openstack-nova | 21:55 | |
| sean-k-mooney | dansmith: oh you fixed it in glance by having it constuct the task factory with an admin context. that is much better then working around it in nova with an admin context | 22:01 |
| sean-k-mooney | having the user download and reupload the image does seem very iniffiecnt. | 22:03 |
| sean-k-mooney | a alternitive would be for the copy image api to create a new iamge which the current user now owns form the old image but that also does not fit with the current import api in my view | 22:04 |
| sean-k-mooney | so ya i think your patch makes sense | 22:04 |
| *** slaweq has quit IRC | 22:07 | |
| dansmith | sean-k-mooney: well, that's mostly just a minimal hammer approach to get past this block (I think).. as I hedged in the commit message, it may very well be that we should be only constructing the image pool with that admin context, or something more detailed | 22:09 |
| dansmith | I haven't chased all the implications of doing this, I just put it up to try to move on and so someone can show me what the right way is, if indeed the user is supposed to be allowed to do this | 22:09 |
| dansmith | totally possible that someone will say it should be admin-only | 22:09 |
| sean-k-mooney | ya although the api in principal should have determined if yo are allowed do something before you get to that point | 22:10 |
| dansmith | but you know, best way to get something done on the internet is to do it wrong so someone will fix it out of anger :) | 22:10 |
| dansmith | agreed | 22:10 |
| *** spatel has quit IRC | 22:10 | |
| dansmith | if not this, then the api needs more checks | 22:10 |
| sean-k-mooney | making image copy admin only would be strange as the other import methods are not and i think it would be the same policy endpoint? | 22:11 |
| dansmith | well, it's a different case I think | 22:12 |
| *** eharney has quit IRC | 22:12 | |
| dansmith | the other import methods are for actually importing the image | 22:12 |
| dansmith | this import being used for copy is a little bit weird, | 22:12 |
| dansmith | because for the others, you wouldn't have one person create the image and another provide its data generally, | 22:13 |
| dansmith | but that's a little bit of what this is, | 22:13 |
| dansmith | but since it's controlled enough I would think that allowing one user that can use an image to copy it to another store in the system is fine, | 22:13 |
| dansmith | just like I said.. I could download and re-upload it myself to get the same effect | 22:13 |
| sean-k-mooney | i mean by extening the import workflow to me i think that implies that they are treating glance as just another data source like a url or file | 22:13 |
| dansmith | and of course, if we want to limit some users from being able to do this, we need a more fine-grained policy knobv | 22:13 |
| dansmith | correct, | 22:13 |
| dansmith | but for the other import mechanisms, you wouldn't want a non-admin-or-owner to be able to do the import from-url or whatever | 22:14 |
| dansmith | which maybe means we should only do this admin context thing if we're doing copy-to-store, I dunno | 22:14 |
| dansmith | but I'll let them opine | 22:14 |
| sean-k-mooney | dansmith: well not for an image they can see but not own | 22:14 |
| sean-k-mooney | if i was to redsign this i would be tempted to invert the workflow and make the copy part of get | 22:15 |
| sean-k-mooney | e.g. if i try to get an image form a store where it is not currently present have it be copied on the backend in paralle to streaming it to the user | 22:16 |
| dansmith | personally I think that this should be a PUT /images/foo {'stores': ['new-store', 'existing-store'] | 22:17 |
| dansmith | I think they had a task-based approach to image importing so they put this in there since it was the minimal amount of work | 22:17 |
| dansmith | and this is what you get by bending one thing to do another | 22:17 |
| sean-k-mooney | as in an image action | 22:17 |
| sean-k-mooney | or create a new image form 'existing-store' | 22:18 |
| dansmith | the move-on-demand streaming thing fundamentally won't work for the rbd case, which is the primary reason for this :) | 22:18 |
| sean-k-mooney | dansmith: are there billing implication to this by the way | 22:18 |
| dansmith | create a new image just creates an explosion for no reason.. we _want_ this to be the same image with multiple locations, else we lose the affinity | 22:18 |
| dansmith | dunno | 22:18 |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova stable/stein: Make greande jobs n-v for EM and oldest stable https://review.opendev.org/737332 | 22:18 |
| sean-k-mooney | if you upload an image as public and i imported are you billed more? | 22:19 |
| dansmith | no idea | 22:19 |
| sean-k-mooney | i guess normal users cant upload public images | 22:19 |
| sean-k-mooney | the can uplload shared images but public i think is admin only | 22:19 |
| * sean-k-mooney is logged in as admin too often to rememebr | 22:20 | |
| sean-k-mooney | ya at least on my kolla install my non admin account can only select private shared or community | 22:21 |
| sean-k-mooney | public visablity is only avaiable in the admin role | 22:21 |
| sean-k-mooney | im not really sure what the difference is between shared, comunity and public | 22:21 |
| sean-k-mooney | ah https://wiki.openstack.org/wiki/Glance-v2-community-image-visibility-design#Visibility_Semantics | 22:22 |
| gmann | melwitt: i am trying to test the stable gate(stein) with the fix on legacy base job, let's see if that fix the things - https://review.opendev.org/#/c/737332/3 | 22:23 |
| *** slaweq has joined #openstack-nova | 22:24 | |
| gmann | problem is that, neither devstack nor devstack-gate install virtualenv on subnode | 22:24 |
| sean-k-mooney | gmann: that was recently changed | 22:24 |
| gmann | since last week it is failing, devstack stable branch and neutron-grenade jobs are fixed but we have nova multinode jobs legacy one | 22:25 |
| sean-k-mooney | http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015204.html | 22:25 |
| gmann | sean-k-mooney: yeah, image update | 22:26 |
| sean-k-mooney | didnt infra plan to fix all jobs | 22:26 |
| sean-k-mooney | they had planned to propose patches to the base jobs | 22:27 |
| gmann | :), no. legacy jobs are always less priority to fix. | 22:27 |
| melwitt | gmann: cool thanks, I'll keep an eye on it | 22:27 |
| gmann | zuulv3 native are easy to fix and add those roles wherever needed. like ensure-tox etc | 22:27 |
| *** spatel has joined #openstack-nova | 22:28 | |
| *** spatel has quit IRC | 22:31 | |
| *** rcernin_ has joined #openstack-nova | 22:33 | |
| *** prometheanfire has left #openstack-nova | 22:33 | |
| gmann | sean-k-mooney: and it depends on failure too, like 'tox not found' failure due to image updates needs to be fixed on wherever needed not in base job - #3 in this http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015559.html | 22:33 |
| gmann | tox issue also started happening in neutron, horizon, and few more repo | 22:34 |
| gmann | so we discussed not to fix in devstack base job instead on failure side. | 22:34 |
| sean-k-mooney | ya the whitebox-tempest-plugin job also broke but we fixed it | 22:35 |
| sean-k-mooney | that is zuulv3 so we just added ensure-pip | 22:35 |
| gmann | putting everything in base job can overload the jobs who does not need these tools. | 22:36 |
| sean-k-mooney | gmann: our issue with whitebox was our pre playboox uses pip before devstack runs and installs pip | 22:36 |
| gmann | ok | 22:37 |
| sean-k-mooney | so we just added ensure-pip | 22:37 |
| sean-k-mooney | easy fix | 22:37 |
| gmann | for xenial node ensure-vitualenv does ensure-pip also but for bionic yes we need to it in start - https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-virtualenv/tasks/Debian.yaml#L9 | 22:39 |
| *** rcernin_ has quit IRC | 22:47 | |
| *** tkajinam has joined #openstack-nova | 22:51 | |
| *** markvoelker has joined #openstack-nova | 22:56 | |
| *** tosky has quit IRC | 22:59 | |
| *** markvoelker has quit IRC | 23:00 | |
| *** dlbewley has quit IRC | 23:02 | |
| *** rcernin_ has joined #openstack-nova | 23:02 | |
| *** dlbewley has joined #openstack-nova | 23:03 | |
| *** rcernin_ has quit IRC | 23:16 | |
| *** rcernin has joined #openstack-nova | 23:18 | |
| *** raildo has quit IRC | 23:30 | |
| *** hamalq has quit IRC | 23:37 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!