| *** jv__ has joined #openstack-glance | 00:30 | |
| *** gyee has quit IRC | 00:30 | |
| *** jv__ has quit IRC | 00:46 | |
| openstackgerrit | norman shen proposed openstack/glance-specs master: Use asynchronous writes for ceph RBD https://review.opendev.org/744818 | 01:56 |
|---|---|---|
| *** rcernin has joined #openstack-glance | 03:07 | |
| *** tkajinam has quit IRC | 03:51 | |
| *** tkajinam has joined #openstack-glance | 03:52 | |
| *** m75abrams has joined #openstack-glance | 04:25 | |
| *** udesale has joined #openstack-glance | 04:33 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-glance | 04:33 | |
| *** ratailor has joined #openstack-glance | 04:56 | |
| *** ratailor has quit IRC | 04:59 | |
| *** ratailor has joined #openstack-glance | 04:59 | |
| *** udesale_ has joined #openstack-glance | 05:45 | |
| *** udesale has quit IRC | 05:47 | |
| openstackgerrit | Abhishek Kekane proposed openstack/glance master: Fix active image when all uploads fail https://review.opendev.org/744010 | 06:02 |
| openstackgerrit | Abhishek Kekane proposed openstack/glance master: Fix active image without data https://review.opendev.org/744011 | 06:02 |
| abhishekk | dansmith, jokke smcginnis I just fixed unit tests failure and proposed patches again so that we can save the time of running check jobs during hot time | 06:03 |
| openstackgerrit | Abhishek Kekane proposed openstack/glance-specs master: Make cinder driver compatible with multiple stores https://review.opendev.org/695152 | 06:16 |
| *** udesale_ has quit IRC | 06:24 | |
| *** udesale has joined #openstack-glance | 06:52 | |
| *** rcernin has quit IRC | 06:58 | |
| *** rcernin_ has joined #openstack-glance | 06:59 | |
| *** rcernin_ has quit IRC | 07:05 | |
| *** nikparasyr has joined #openstack-glance | 07:06 | |
| *** rcernin has joined #openstack-glance | 07:06 | |
| *** k_mouza has joined #openstack-glance | 08:29 | |
| *** ralonsoh has joined #openstack-glance | 08:30 | |
| *** rcernin has quit IRC | 09:04 | |
| *** rcernin has joined #openstack-glance | 09:04 | |
| *** rcernin has quit IRC | 09:05 | |
| *** rcernin has joined #openstack-glance | 09:05 | |
| *** ralonsoh has quit IRC | 09:50 | |
| * abhishekk will be back around 1430 UTC | 09:56 | |
| *** tkajinam has quit IRC | 10:15 | |
| *** rcernin has quit IRC | 10:23 | |
| *** ralonsoh has joined #openstack-glance | 11:08 | |
| *** rcernin has joined #openstack-glance | 11:25 | |
| *** rcernin has quit IRC | 11:30 | |
| *** jv__ has joined #openstack-glance | 11:32 | |
| *** k_mouza has quit IRC | 11:54 | |
| *** rcernin has joined #openstack-glance | 11:56 | |
| *** rcernin has quit IRC | 12:02 | |
| *** rcernin has joined #openstack-glance | 12:17 | |
| *** rcernin has quit IRC | 12:22 | |
| *** udesale has quit IRC | 12:37 | |
| *** udesale has joined #openstack-glance | 12:38 | |
| *** rcernin has joined #openstack-glance | 13:00 | |
| jokke | dansmith: you around yet? | 13:06 |
| *** priteau has joined #openstack-glance | 13:09 | |
| *** rcernin has quit IRC | 13:15 | |
| *** udesale has quit IRC | 13:21 | |
| *** udesale has joined #openstack-glance | 13:22 | |
| dansmith | jokke: brewing coffee but, yup | 13:22 |
| dansmith | sorry about missing that flow name change | 13:22 |
| jokke | dansmith: ha, I was just gonna hit that task but couple of quick questions just trying to understand the format inspector | 13:23 |
| jokke | L 387 you unpack the whole 16 bytes, but those foo and bar are never being used, any reason why not just grab the first 8 bytes and get what you need? | 13:25 |
| dansmith | no, and I probably do that in several places, | 13:26 |
| dansmith | that comes from reading specs and generally wanting to parse a whole header or record as a unit, and at least while debugging, compare those values to make sure they look legit | 13:27 |
| jokke | ok, it's not a big deal, I just wanted make sure I understood those foo and bar didn't have some special meaning after all what I ended up missing | 13:27 |
| dansmith | if I named them something useful, it also makes it nicer if I name them what they are before I ignore them, in case they're useful in the future if we need to pull more information or something | 13:27 |
| jokke | yeah | 13:28 |
| dansmith | in this case, both are bitfields with information that don't really matter to us, and most of the bits are "reserved anyway" | 13:28 |
| jokke | another readability thing, specific reason why VDI inspector uses 0x indexing on the ranges where everything else is just numbers? | 13:29 |
| jokke | dansmith: kk, yeah I was just wondering why you didn't do 'II' +16, +24 instead of the 'IIII' +16, +32 | 13:30 |
| dansmith | because the header is larger than most and the struct I was looking at marked offsets in hex :) | 13:30 |
| dansmith | I can go back through and convert those to decimal as well as comment what the offsets are that I'm looking for in each of the formats | 13:31 |
| dansmith | in general, I'm fine with mixing hex and decimal especially if they're consistent with the relevant spec document's offsets, but happy to make those consistent if you want | 13:32 |
| jokke | I'm not even sure which I prefer, it was just something that caught my eye while reading the code as I don't think we have indexes in 0x anywhere in our code ... gotta admit the later specially is fairly sisable numbers | 13:33 |
| jokke | and the last question if you don't mind. Any reason why you did not specify the byte order in VHDX but in every other parser? | 13:35 |
| dansmith | I'll add a block comment to each format with the layout we're looking for and the offsets | 13:35 |
| dansmith | oh I did in one place, but not the later two, good catch | 13:35 |
| dansmith | if those are your only questions about that big batch of complex parsing code, I'm pretty happy :) | 13:36 |
| jokke | Well some of the constant magic numbers you're referring there I have no idea about, but the rest made sense and I was able to follow. Just few things that I was scrathing my head like "Is there a reason for this" | 13:37 |
| dansmith | ack | 13:38 |
| jokke | it's not exactly fun code to read but thanks for keeping it fairly easy ot follow and very nice feature to have | 13:40 |
| dansmith | omg, you're calling my code "not fun"? I'm so offended! | 13:41 |
| *** ratailor has quit IRC | 13:42 | |
| dansmith | the vhdx was the least fun, obviously, but I was pretty happy that it was well-documented. I was expecting them all to be as simple and obvious as qcow, but alas :) | 13:43 |
| *** k_mouza has joined #openstack-glance | 13:46 | |
| jokke | that code is going to be running in sidechannel, right? | 13:53 |
| jokke | So it's not directly in the datapath | 13:53 |
| *** Liang__ has joined #openstack-glance | 13:59 | |
| *** Liang__ is now known as LiangFang | 14:00 | |
| dansmith | I dunno what that means really.. the InfoWrapper sits directly in the data stream of course, but it just passes the chunks to this stuff to look at. It happens synchronously and in the data path, but... it doesn't modify anything if that's what you mean | 14:01 |
| *** k_mouza has quit IRC | 14:02 | |
| *** k_mouza has joined #openstack-glance | 14:04 | |
| jokke | yeah, that's what I thought so data streams through the wrapper, and just copy of it is passed to this code from the stream. I was just thinking how much extra work (in follow up) would cause to pull some of the decompression stuff from the task plugin and get that info extracted from the cmpressed file format too ;) | 14:05 |
| jokke | I'll have a look into that at some point | 14:05 |
| dansmith | actual decompression or de-sparsification? | 14:05 |
| jokke | actual decompression. We support couple of streamable formats so could detect those, decompress just enough from the beginning to get the headers and pass it down to your code | 14:06 |
| jokke | s/formats/algorithms/ | 14:07 |
| dansmith | ah, yeah I asked on the spec if we cared about anything other than "bare" container_format.. are the compression types those, or yet another thing? | 14:09 |
| dansmith | I guess I imagined you'd just use http gzip to compress the stream, in which case it'd be un-gzip'd by the time it made it here | 14:09 |
| dansmith | if I have a flag, I can un-gzip in the stream for sure, at least | 14:10 |
| jokke | we have 'compressed' this was for cinder use that supports passing compressed image to cinder to reduce nwbandwidth | 14:10 |
| jokke | just an idea that once this is working streamable decompression of the beginning is not difficult piece to add there | 14:11 |
| jokke | so those images are sitting in the store actually compressed | 14:11 |
| dansmith | sure, is it just gzip or something else? | 14:12 |
| jokke | I think it might be just gzip | 14:12 |
| dansmith | that's easy to integrate into this and to stop unzipping when we get far enough | 14:13 |
| dansmith | but yes, let's build that on this, I'd say, it's complex enough to review right now | 14:13 |
| jokke | yup | 14:13 |
| jokke | that's what I thought | 14:13 |
| jokke | and python has direct support for zip, lha and gzip. they are all very trivial to detect | 14:14 |
| jokke | lha is not that much used 'though | 14:14 |
| dansmith | are we not told whether or not it's compressed? | 14:15 |
| jokke | yes, but that's all it's just 'compressed' container format ... we never thought we would need any other info about it | 14:15 |
| jokke | I wrote the decompression module so people can point web-download directly to compressed image link, never thought we would poke to see what's actually in there ;) | 14:16 |
| dansmith | heh | 14:16 |
| jokke | So it just looks the magic numbers at the headers to see if it can detect those three algos, decomresses the image before uploading to store if the container_format is not 'compressed' | 14:17 |
| jokke | and the plugin does it inplace so we can then run conversion on it if needed | 14:18 |
| jokke | it doesn't do it on the stream | 14:19 |
| dansmith | ack | 14:19 |
| abhishekk | smcginnis, rosmaita kindly have look when you have some time, https://review.opendev.org/#/c/695152/ | 14:29 |
| rosmaita | ack | 14:29 |
| abhishekk | related to cinder multiple stores support | 14:29 |
| abhishekk | dansmith, I have modified the specs related to introspect feature | 14:30 |
| dansmith | I saw, I'll look in a bit | 14:33 |
| abhishekk | ack | 14:34 |
| openstackgerrit | Merged openstack/python-glanceclient master: Fixes "stores" property added to the image https://review.opendev.org/744024 | 14:36 |
| *** jv__ has quit IRC | 14:40 | |
| openstackgerrit | Erno Kuvaja proposed openstack/python-glanceclient stable/ussuri: Fixes "stores" property added to the image https://review.opendev.org/744917 | 14:43 |
| openstackgerrit | Dan Smith proposed openstack/glance master: Stream-friendly disk format inspection module https://review.opendev.org/744205 | 14:43 |
| openstackgerrit | Dan Smith proposed openstack/glance master: Inspect upload stream and set virtual_size on upload https://review.opendev.org/744234 | 14:43 |
| dansmith | jokke: lots of format doc ^ | 14:44 |
| jokke | dansmith: ack, thnx | 14:44 |
| *** jv_ has joined #openstack-glance | 14:53 | |
| openstackgerrit | Dan Smith proposed openstack/glance master: Stream-friendly disk format inspection module https://review.opendev.org/744205 | 14:55 |
| openstackgerrit | Dan Smith proposed openstack/glance master: Inspect upload stream and set virtual_size on upload https://review.opendev.org/744234 | 14:55 |
| *** jv__ has joined #openstack-glance | 15:01 | |
| *** jv_ has quit IRC | 15:02 | |
| *** jv__ has quit IRC | 15:09 | |
| *** jv__ has joined #openstack-glance | 15:10 | |
| *** LiangFang has quit IRC | 15:13 | |
| *** jv__ has quit IRC | 15:17 | |
| openstackgerrit | Dan Smith proposed openstack/glance master: Inspect upload stream and set virtual_size on upload https://review.opendev.org/744234 | 15:23 |
| dansmith | abhishekk: ^ added a functional test per the spec | 15:23 |
| abhishekk | dansmith, jokke just need one opinion as now this Introspect work is not related to import plugin any more, i have changed the blueprint title to https://blueprints.launchpad.net/glance/+spec/calculate-virtual-size | 15:23 |
| dansmith | I still need make import do it | 15:24 |
| abhishekk | dansmith, sorry, you need to tag this blueprint now | 15:24 |
| dansmith | ah okay | 15:24 |
| dansmith | I'll work on the import part and then will change the blueprint tag of the whole set | 15:24 |
| abhishekk | dansmith, in functional or import workflow to calculate virtual size? | 15:24 |
| dansmith | the import flow doesn't include this inspector yet, just the regular upload | 15:25 |
| dansmith | er, wait..maybe it does? | 15:25 |
| dansmith | maybe under all the layers we call through the same location.py bit? I'll have to check | 15:25 |
| abhishekk | dansmith, that's what I am saying | 15:28 |
| abhishekk | it should work for import as well | 15:28 |
| abhishekk | but we just need to make sure that if there are multiple imports and virtual size is set then we should not calculate it again | 15:28 |
| dansmith | ack, will write that test to confirm.. i just hadn't even looked yet | 15:28 |
| dansmith | ack | 15:29 |
| dansmith | {0} glance.tests.functional.v2.test_images.TestImages.test_image_upload_qcow_virtual_size_calculation [4.037527s] ... ok | 15:31 |
| dansmith | {1} glance.tests.functional.v2.test_images.TestImages.test_image_import_qcow_virtual_size_calculation [4.538418s] ... ok | 15:31 |
| dansmith | yup :) | 15:31 |
| abhishekk | :D | 15:32 |
| abhishekk | just need to add check for multiple imports or copying to other stores | 15:32 |
| dansmith | yeah, so, is virtual_size an image property or an "extra_properties" ? | 15:33 |
| dansmith | the automatic delegation makes it hard to tell that, and the ImageStub in those tests isn't complete of course | 15:34 |
| abhishekk | I guess it is image property | 15:34 |
| dansmith | ah I see it in domain/proxy | 15:35 |
| *** nikparasyr has left #openstack-glance | 15:36 | |
| abhishekk | on that note, I will take dinner break :P | 15:36 |
| * abhishekk dinner break, will be back in 45 mins | 15:36 | |
| openstackgerrit | Dan Smith proposed openstack/glance master: Stream-friendly disk format inspection module https://review.opendev.org/744205 | 15:42 |
| openstackgerrit | Dan Smith proposed openstack/glance master: Inspect upload/import stream and set virtual_size https://review.opendev.org/744234 | 15:42 |
| dansmith | abhishekk: when you get back, can you change the owner to my openstack persona (danms) ? | 15:43 |
| *** m75abrams has quit IRC | 15:43 | |
| abhishekk | dansmith, done | 15:59 |
| * abhishekk back | 15:59 | |
| dansmith | thanks | 16:00 |
| abhishekk | dansmith, I am changing the same in the specs as well (for assignee ) | 16:09 |
| dansmith | cool | 16:10 |
| openstackgerrit | Abhishek Kekane proposed openstack/glance-specs master: Calculate virtual size of image https://review.opendev.org/741121 | 16:12 |
| *** jv__ has joined #openstack-glance | 16:31 | |
| *** jv__ has quit IRC | 16:32 | |
| *** udesale has quit IRC | 16:32 | |
| *** gyee has joined #openstack-glance | 16:48 | |
| *** jdillaman has quit IRC | 17:23 | |
| * abhishekk signing out for the day | 17:33 | |
| *** priteau has quit IRC | 17:38 | |
| *** k_mouza has quit IRC | 17:43 | |
| *** ralonsoh has quit IRC | 18:12 | |
| *** k_mouza has joined #openstack-glance | 18:13 | |
| *** k_mouza has quit IRC | 18:16 | |
| openstackgerrit | Merged openstack/glance master: Fix active image when all uploads fail https://review.opendev.org/744010 | 18:58 |
| *** gyee has quit IRC | 20:36 | |
| *** gyee has joined #openstack-glance | 20:38 | |
| *** smcginni1 has joined #openstack-glance | 20:47 | |
| *** smcginnis has quit IRC | 20:50 | |
| *** smcginni1 is now known as smcginnis | 20:50 | |
| jokke | smcginnis: around by any chance? | 21:11 |
| smcginnis | o/ | 21:11 |
| jokke | smcginnis: if you have a bit extra time on your hands, stable/ussuri client patch https://review.opendev.org/#/c/744917 | 21:12 |
| smcginnis | jokke: Done! | 21:15 |
| jokke | ty! | 21:15 |
| smcginnis | Any time jokke. | 21:15 |
| *** gyee has quit IRC | 21:24 | |
| *** gyee has joined #openstack-glance | 21:25 | |
| openstackgerrit | Merged openstack/glance master: Add "stores" to disallowed properties https://review.opendev.org/744030 | 21:30 |
| *** rcernin has joined #openstack-glance | 21:35 | |
| *** rcernin has quit IRC | 21:36 | |
| *** rcernin has joined #openstack-glance | 21:36 | |
| openstackgerrit | Erno Kuvaja proposed openstack/glance stable/ussuri: Add "stores" to disallowed properties https://review.opendev.org/744994 | 21:40 |
| *** rcernin has quit IRC | 22:03 | |
| *** rcernin has joined #openstack-glance | 22:17 | |
| openstackgerrit | Erno Kuvaja proposed openstack/glance stable/ussuri: Fix active image when all uploads fail https://review.opendev.org/744997 | 22:23 |
| *** jmlowe has quit IRC | 22:27 | |
| *** jmlowe has joined #openstack-glance | 22:31 | |
| *** rcernin has quit IRC | 22:40 | |
| *** rcernin has joined #openstack-glance | 22:40 | |
| *** rcernin has quit IRC | 22:40 | |
| *** rcernin has joined #openstack-glance | 22:44 | |
| openstackgerrit | Merged openstack/python-glanceclient stable/ussuri: Fixes "stores" property added to the image https://review.opendev.org/744917 | 22:51 |
| *** tkajinam has joined #openstack-glance | 23:00 | |
| *** josephillips has joined #openstack-glance | 23:26 | |
| josephillips | hey guys | 23:26 |
| josephillips | someone had issues using glance-cache-manage | 23:26 |
| josephillips | with 401 error | 23:26 |
| josephillips | but using glance works well | 23:26 |
| *** irclogbot_3 has quit IRC | 23:34 | |
| *** irclogbot_2 has joined #openstack-glance | 23:37 | |
| *** yoctozepto3 has joined #openstack-glance | 23:44 | |
| *** yoctozepto has quit IRC | 23:45 | |
| *** yoctozepto3 is now known as yoctozepto | 23:45 | |
| jokke | josephillips: which version of glance? | 23:45 |
| josephillips | train version | 23:51 |
| jokke | hmm-m ... we do have patch to glance-cache-manage coming in train (probably released tomorrow) but that should not cause 401. you might want to check that the command is ran with correct permissions based on how your policy is set | 23:56 |
| openstackgerrit | Erno Kuvaja proposed openstack/glance stable/ussuri: Fix active image when all uploads fail https://review.opendev.org/744997 | 23:58 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!