Tuesday, 2015-08-18

*** ctina has joined #openstack-glance00:03
*** ducttape_ has joined #openstack-glance00:07
*** harshs has quit IRC00:08
*** Ctina_ has quit IRC00:11
*** TravT_ has joined #openstack-glance00:13
*** TravT has quit IRC00:15
*** TravT has joined #openstack-glance00:21
*** takedakn has quit IRC00:23
*** TravT_ has quit IRC00:24
*** tsekiyama has quit IRC00:26
*** mtanino has quit IRC00:32
*** ducttape_ has quit IRC00:34
*** dims has joined #openstack-glance00:34
*** ducttape_ has joined #openstack-glance00:40
*** ducttape_ has quit IRC00:40
*** ducttape_ has joined #openstack-glance00:41
openstackgerritTomoki Sekiyama proposed openstack/glance: Importing oslo.rootwrap into Glance  https://review.openstack.org/18620100:46
*** ducttape_ has quit IRC00:48
*** sigmavirus24 is now known as sigmavirus24_awa00:49
*** ctina has quit IRC01:03
*** ctina has joined #openstack-glance01:07
*** ducttape_ has joined #openstack-glance01:10
*** mtanino has joined #openstack-glance01:14
*** takedakn has joined #openstack-glance01:42
*** takedakn has quit IRC01:42
*** junhongl has joined #openstack-glance01:46
*** ducttape_ has quit IRC02:02
*** haomaiwang has joined #openstack-glance02:06
*** mtanino has quit IRC02:11
*** dims has quit IRC02:30
*** baojg has joined #openstack-glance02:31
*** gberginc has joined #openstack-glance02:32
*** bkopilov has quit IRC02:32
*** harshs has joined #openstack-glance02:42
*** harshs_ has joined #openstack-glance03:00
*** haomaiwang has quit IRC03:01
*** harshs has quit IRC03:01
*** harshs_ is now known as harshs03:01
*** baojg has quit IRC03:02
*** haomaiwa_ has joined #openstack-glance03:02
*** baojg has joined #openstack-glance03:15
*** changbl has quit IRC03:30
*** sdake has joined #openstack-glance03:32
*** sdake_ has quit IRC03:36
*** sdake_ has joined #openstack-glance03:42
*** sdake has quit IRC03:45
*** baojg has quit IRC03:49
*** bkopilov has joined #openstack-glance03:58
*** haomaiwa_ has quit IRC04:01
*** haomaiwa_ has joined #openstack-glance04:02
*** baojg has joined #openstack-glance04:16
*** sdake_ is now known as sdake04:22
*** ayoung has quit IRC04:25
*** lakshmiS has joined #openstack-glance04:36
*** haomaiwa_ has quit IRC05:01
*** haomaiwa_ has joined #openstack-glance05:02
*** vijendar has quit IRC05:14
*** harshs has quit IRC05:15
*** dims has joined #openstack-glance05:20
*** dims has quit IRC05:26
*** gberginc has quit IRC05:26
*** lakshmiS has quit IRC05:52
*** haomaiwa_ has quit IRC06:01
*** haomaiwang has joined #openstack-glance06:02
*** gberginc has joined #openstack-glance06:04
*** groen692 has joined #openstack-glance06:05
*** tpeoples has quit IRC06:11
*** GB21 has joined #openstack-glance06:17
openstackgerritOpenStack Proposal Bot proposed openstack/glance: Imported Translations from Transifex  https://review.openstack.org/21402206:27
openstackgerritOpenStack Proposal Bot proposed openstack/glance_store: Imported Translations from Transifex  https://review.openstack.org/21402606:38
*** openstackgerrit_ has joined #openstack-glance06:42
*** haomaiwang has quit IRC07:01
*** haomaiwang has joined #openstack-glance07:02
*** GB21 has quit IRC07:02
*** peristeri has joined #openstack-glance07:11
*** abhishekk has joined #openstack-glance07:13
*** markus_z has joined #openstack-glance07:14
*** exploreshaifali has joined #openstack-glance07:36
*** dims has joined #openstack-glance07:48
*** sgotliv has joined #openstack-glance07:48
*** dims has quit IRC07:53
*** jistr has joined #openstack-glance07:55
*** haomaiwang has quit IRC08:01
*** haomaiwang has joined #openstack-glance08:02
openstackgerritFlavio Percoco proposed openstack/glance: Add the trigger field to the tasks table  https://review.openstack.org/19789908:10
openstackgerritFlavio Percoco proposed openstack/glance: Add task trigger model  https://review.openstack.org/19789008:10
*** exploreshaifali has quit IRC08:29
openstackgerritMerged openstack/glance: Imported Translations from Transifex  https://review.openstack.org/21402208:48
*** d0ugal has joined #openstack-glance08:54
*** haomaiwang has quit IRC09:01
*** shikel has quit IRC09:01
*** haomaiwang has joined #openstack-glance09:02
*** sh1kel has joined #openstack-glance09:06
*** MattMan has joined #openstack-glance09:06
*** MattMan has quit IRC09:07
*** MattMan has joined #openstack-glance09:08
*** haomaiwang has quit IRC10:01
*** haomaiwang has joined #openstack-glance10:02
*** baojg has quit IRC10:06
*** baojg has joined #openstack-glance10:06
*** baojg has quit IRC10:07
*** baojg has joined #openstack-glance10:07
*** baojg has quit IRC10:07
*** baojg has joined #openstack-glance10:07
*** pbourke has quit IRC10:11
*** exploreshaifali has joined #openstack-glance10:12
*** pdb has joined #openstack-glance10:14
openstackgerritAlexander Tivelkov proposed openstack/glance: Artifacts are now properly filtered by dict props  https://review.openstack.org/20737410:14
*** sdake has quit IRC10:15
*** baojg has quit IRC10:17
*** baojg has joined #openstack-glance10:18
*** baojg has quit IRC10:23
*** sayali has quit IRC10:24
*** sayali has joined #openstack-glance10:25
*** exploreshaifali has quit IRC10:43
*** sh1kel has quit IRC10:48
*** dims has joined #openstack-glance10:51
*** haomaiwang has quit IRC10:54
*** delattec has quit IRC11:10
*** cdelatte has quit IRC11:10
*** exploreshaifali has joined #openstack-glance11:13
*** ankit_ag has joined #openstack-glance11:15
*** marcusvrn has joined #openstack-glance11:27
*** dims_ has joined #openstack-glance11:28
*** dims has quit IRC11:30
*** cdelatte has joined #openstack-glance11:32
*** smatzek has joined #openstack-glance11:36
*** DuncanT has quit IRC11:38
*** marcusvrn has quit IRC11:38
*** bkopilov has quit IRC11:58
*** dims has joined #openstack-glance12:00
*** dims has quit IRC12:00
*** dims_ has quit IRC12:00
*** dims has joined #openstack-glance12:01
*** dims has quit IRC12:05
openstackgerritAlexander Tivelkov proposed openstack/glance: Artifacts are now properly filtered by dict props  https://review.openstack.org/20737412:11
*** TravT_ has joined #openstack-glance12:15
*** dims has joined #openstack-glance12:15
openstackgerritAlexander Tivelkov proposed openstack/glance: Artifacts are now properly filtered by dict props  https://review.openstack.org/20737412:15
*** TravT has quit IRC12:16
*** ducttape_ has joined #openstack-glance12:20
openstackgerritAlexander Tivelkov proposed openstack/glance: Artifacts are now properly filtered by dict props  https://review.openstack.org/20737412:20
*** annegentle has joined #openstack-glance12:24
*** ducttape_ has quit IRC12:37
*** marcusvrn has joined #openstack-glance12:38
*** sgotliv_ has joined #openstack-glance12:38
*** ducttape_ has joined #openstack-glance12:38
*** ducttape_ has quit IRC12:38
*** sgotliv has quit IRC12:40
*** dims_ has joined #openstack-glance12:41
*** dims has quit IRC12:41
*** ctina has quit IRC12:43
*** ctina has joined #openstack-glance12:44
*** edmondsw has joined #openstack-glance12:47
*** exploreshaifali has quit IRC12:48
*** gberginc has quit IRC12:49
*** marcusvrn has quit IRC12:49
*** chlong has joined #openstack-glance12:52
*** GB21 has joined #openstack-glance12:56
*** GB21_ has joined #openstack-glance12:56
*** GB21_ has quit IRC12:56
*** dims has joined #openstack-glance13:01
*** dims_ has quit IRC13:02
*** haomaiwang has joined #openstack-glance13:03
openstackgerritDarja Shakhray proposed openstack/glance: Artifacts are now properly filtered by dict props  https://review.openstack.org/20737413:06
*** dims_ has joined #openstack-glance13:15
*** exploreshaifali has joined #openstack-glance13:16
*** dims has quit IRC13:17
*** changbl has joined #openstack-glance13:19
*** julim has joined #openstack-glance13:24
edmondswkragniz flaper87 jokke_ this should be ready for another look: https://review.openstack.org/#/c/203242/13:27
*** tpeoples has joined #openstack-glance13:27
openstackgerritAlexander Tivelkov proposed openstack/glance: Fixed version unequality artifact filtering  https://review.openstack.org/21418513:31
*** dims_ has quit IRC13:31
openstackgerritMike Fedosin proposed openstack/glance-specs: Swift driver with multithreading support  https://review.openstack.org/20707513:31
*** dims has joined #openstack-glance13:31
openstackgerritAlexander Tivelkov proposed openstack/glance: Fixed version unequality artifact filtering  https://review.openstack.org/21418513:31
*** ayoung has joined #openstack-glance13:31
kragnizedmondsw: thanks, I'll take a look when I'm near my computer13:33
edmondswtx13:33
*** ankit_ag has quit IRC13:38
openstackgerritMike Fedosin proposed openstack/glance-specs: Swift driver with multithreading support  https://review.openstack.org/20707513:39
*** GB21 has quit IRC13:40
*** ducttape_ has joined #openstack-glance13:40
*** dims has quit IRC13:40
*** changbl has quit IRC13:41
openstackgerritAlexander Tivelkov proposed openstack/glance: Fixed version unequality artifact filtering  https://review.openstack.org/21418513:43
*** dims has joined #openstack-glance13:43
*** smatzek has quit IRC13:45
*** pdb is now known as pbourke13:50
*** tpeoples has quit IRC13:50
*** pbourke has quit IRC13:51
*** pbourke has joined #openstack-glance13:51
*** bkopilov has joined #openstack-glance13:52
*** marcusvrn_ has joined #openstack-glance13:52
*** haomaiwang has quit IRC14:01
*** haomaiwang has joined #openstack-glance14:02
*** abhishekk has quit IRC14:02
*** DuncanT has joined #openstack-glance14:03
*** dims has quit IRC14:04
*** dims has joined #openstack-glance14:05
*** smatzek has joined #openstack-glance14:06
*** sigmavirus24_awa is now known as sigmavirus2414:11
openstackgerritAlexander Tivelkov proposed openstack/glance: Fixed version unequality artifact filtering  https://review.openstack.org/21418514:14
*** gberginc has joined #openstack-glance14:17
*** peristeri has quit IRC14:20
*** annegentle has quit IRC14:20
*** zul has joined #openstack-glance14:29
flaper87o/14:30
nikhil_k_o/14:30
flaper87rosmaita: don't run14:30
flaper87I know you're there14:30
flaper87:P14:30
rosmaitai am here14:30
flaper87nikhil_k_: rosmaita sigmavirus24 https://review.openstack.org/#/c/188388/14:31
flaper87rosmaita: the last PS just adds some API examples of how it'd work from a user perspective14:31
*** sdake has joined #openstack-glance14:31
nikhil_k_you know I feel discussing overarching questions on this seems more important than the details..14:31
rosmaitaflaper87: great ... i need a few min to read through14:31
flaper87rosmaita: okis14:32
nikhil_k_so, the way I looked at it is this. not sure if I get it correctly:14:32
flaper87nikhil_k_: yup, sounds go to me14:32
*** tpeoples has joined #openstack-glance14:32
nikhil_k_1. User uploads image-data, expects image to go to active. we either create a new status or state the expected behavior may change with deployment14:34
nikhil_k_the second one doesn't fit the interop purposes truly but might be good to ask wider audience14:35
flaper87mmh, you mean the user  expects the image to go active after going through all the tasks ?14:35
nikhil_k_first one would change the expected behavior for image-creates14:35
nikhil_k_right14:35
nikhil_k_because if we set it to active14:35
*** dims_ has joined #openstack-glance14:35
nikhil_k_and then chance the container and disk format14:36
nikhil_k_change*14:36
flaper87Who changes the expected behavior?14:36
flaper87We ? or OPs?14:36
nikhil_k_it's not truly active14:36
nikhil_k_I am looking at this from a API stability POV14:36
flaper87ok, that helps14:36
*** spzala has joined #openstack-glance14:36
flaper87So, may I describe a scenario?14:36
nikhil_k_Either the API scrutiny is unnecessarily high or there are some real use cases for this14:37
flaper87One related to the use-case I've been using as an example14:37
nikhil_k_(this == keeping expected API behavior)14:37
*** dims has quit IRC14:37
* nikhil_k_ listens14:37
*** sdake_ has joined #openstack-glance14:37
flaper87coolio14:37
flaper87So, OPs actions:14:37
flaper871) The OP creates a trigger task that will convert images to raw format when they are uploaded14:38
flaper87Users:14:38
flaper871) User creates an image14:38
flaper872) User uploads the image-data14:38
flaper873) The image is uploaded, tasks for the "image.upload" action are loaded and executed14:39
flaper874) After tasks are executed the image is set to active14:39
flaper875) Profit?14:39
flaper87:P14:39
flaper87From a user perspective, I think the API is still behaving as expected14:39
rosmaitaflaper87: so this is 1 image created, or 2?14:40
flaper87Tasks could fail, yes. But that's true for a world w/o tasks too14:40
flaper87rosmaita: 1 image14:40
nikhil_k_a traditional upload operation would put image in active14:40
flaper87It's not a clone but a conversion14:40
*** sdake has quit IRC14:40
*** exploreshaifali has quit IRC14:40
flaper87nikhil_k_: mmh, I see what you mean14:40
flaper87sorry, it took me a bit14:41
nikhil_k_the workflow for image upload changes14:41
nikhil_k_cool14:41
flaper87How would you feel about an intering status?14:41
flaper87interin14:41
flaper87not even sure if that's a word14:41
flaper87:P14:41
flaper87For example, 'processing'14:41
flaper87'baking'14:42
flaper87:P14:42
rosmaitawhat about only using this on import tasks? then it's just part of the import workflow14:42
flaper87'in-da-oven'14:42
*** spzala has quit IRC14:42
nikhil_k_iterim might be what you intended but that's what I was thinking if that would become a necessity14:42
flaper87nikhil_k_: yup, that's what I meant. Thanks14:42
nikhil_k_rosmaita: :)14:42
*** fnordahl has quit IRC14:42
*** vijendar has joined #openstack-glance14:42
flaper87rosmaita: that doesn't really solve the problem. What I keep hearing from OPs is that the import tasks and the tasks we have nwo are not useful at all if they have to run them manually14:43
nikhil_k_no uploads allowed, ever! :P :P14:43
rosmaitaflaper87: but they don't have to run them manually, the end-user does it14:43
flaper87Uploading an image is probably the most common way to create an image in Glance14:43
nikhil_k_flaper87: I can see your point on usability14:43
flaper87rosmaita: right but the end user doesn't care if the OP has ceph or not or whether raw would be a better format14:43
*** annegentle has joined #openstack-glance14:44
rosmaitaflaper87: right, so that's why the op constructs an import workflow to get the data into the appropriate format14:44
rosmaitai may be thinking public cloud too much here14:44
flaper87rosmaita: yup :P14:44
flaper87also14:44
flaper87the import workflow will likely create a new image in Glance  with a different ID14:45
flaper87which is not idea if you created the image and got the ID from there14:45
flaper87And we currently just allow HTTP in our import workflow, which means the OP has to find a way to set that up14:45
flaper87it's just not friendly14:45
flaper87I know leseb is one of the OPs interested in this14:46
flaper87hope he's around to provide better feedback14:46
flaper87and other use-cases14:46
flaper87Here's another idea:14:46
flaper87What if instead of messing with the specific actions - image.create, image.saved, image.blah - we do it on status changes ?14:47
nikhil_k_flaper87: one thing this changes drastically is ability to mutate base properties. and now I understand why you were complaining about domain model :P14:47
*** GB21 has joined #openstack-glance14:47
flaper87nikhil_k_: :D14:47
flaper87nikhil_k_: that is true. However, it does it internally in glance and it doesn't expose that to the users14:47
sigmavirus24lol14:48
nikhil_k_yeah, it would be a change. Also, it can be a point of exploit for advanced complicated tasks that do mutation regularly. which I dm not sure if is good or bad thing14:48
*** harshs has joined #openstack-glance14:49
sigmavirus24so I think consistency here will be hard to achieve14:49
nikhil_k_well expressed14:49
flaper87agreed14:49
sigmavirus24also, I wonder what the impact of this would be on glance servers under a heavy load14:49
nikhil_k_that was my 2. :))14:50
sigmavirus24let's say an operator sets up an auto-triggered task to convert new images14:50
rosmaitait would kill them14:50
sigmavirus24right14:50
flaper87it'll require workers but that's why we're consuming taskflow to begin with14:50
sigmavirus24right14:50
nikhil_k_with no handle to queue, may be?14:50
sigmavirus24rosmaita: but even if it doesn't topple the cloud (which flaper87 would assure wouldn't happen)14:50
nikhil_k_we may have issues with scheduling for this on a large scale deployment14:50
sigmavirus24I'm imagining a large queue of images that users can't use14:51
*** julim_ has joined #openstack-glance14:51
nikhil_k_yup14:51
sigmavirus24prior to auto-triggered tasks, users can upload an image and use it almost immediately14:51
sigmavirus24now, they ahve to wait for the tasks to complete before being able to use it14:51
sigmavirus24which makes the case for 2 images but 2 images is a storage nightmare possibly14:51
rosmaitait would make upload similar to image import14:51
nikhil_k_or replace it14:52
flaper87yup but it really depends on the tasks configured14:52
sigmavirus24eww14:52
flaper87and we go back to sigmavirus24's comment w.r.t consistency14:52
nikhil_k_or create a hybrid14:52
sigmavirus24replacing it seems like the worst option14:52
sigmavirus24then the image data is not immutable14:52
sigmavirus24if someone launches an instance from that image prior to being replaced and then resizes, what would that do nikhil_k_ ?14:52
nikhil_k_that's true even today with tasks but here it's upload+task14:52
flaper87I've always thought of the image data to be immutable after it's been stored14:53
flaper87because that's the only moment when we can say: "We've got this"14:53
nikhil_k_flaper87: yeah, that's one point people don't seem to agree on and scares me with our mission statement as we cannot gurantee upload delivery consistency14:53
nikhil_k_sigmavirus24: I think flaper87 is proposing a new state for such cases (analogous to 'interim')14:54
sigmavirus24so, I keep getting mixed messages, but would glance_store in Nova allow people to upload directly to a backing store?14:54
nikhil_k_may be not on the spec14:54
sigmavirus24nikhil_k_: yeah, I grok that14:54
nikhil_k_sigmavirus24: that's still a open question14:54
flaper87sigmavirus24: nope14:54
*** zul has quit IRC14:54
nikhil_k_some people like and some don't14:54
sigmavirus24I'm just imaging queued being in the task queue, and interim being part of a task14:54
sigmavirus24flaper87: okay so that rules out that nastiness14:55
nikhil_k_sigmavirus24: even today if the direct location url is exposed, one may be able to set the image location fter uploading the data from Nova. please correct me if I am wrong14:55
*** mtanino has joined #openstack-glance14:55
flaper87tbh, I don't think the 'processing' status between queued and active is bad14:56
sigmavirus24flaper87: I don't either14:56
sigmavirus24It's very transparent which I like14:56
sigmavirus24but14:56
openstackgerritNiall Bunting proposed openstack/glance: Tasks now try the schema before the database  https://review.openstack.org/21423214:56
sigmavirus24So, rosmaita represents our public cloud needs14:56
sigmavirus24I'm aware of very large private clouds14:56
nikhil_k_sigmavirus24: no, I think we need a new image state as the regular upload workflow would change and the users expecting a image to go active after upload completes would need to know some details on what's happening to their image14:56
flaper87I want to represent our chaos14:56
flaper87please, please, please14:56
sigmavirus24lol14:57
sigmavirus24right14:57
sigmavirus24I think private cloud operators would like this14:57
sigmavirus24100% agree with you flaper8714:57
*** pbourke_ has joined #openstack-glance14:57
sigmavirus24I guess we'll need someway to allow auto-triggered task authors to skip certain events14:57
sigmavirus24Like "see X in proposed task; bail out early" and I can't recall if we give them enough information14:58
flaper87yeah14:58
sigmavirus24And I'm not even sure what those conditions might be14:58
sigmavirus24So14:58
flaper87nikhil_k_: the way I've seen this done in Nova/Cinder is by having a task status14:58
flaper87image state is 'processing' and the task status is 'converting'14:58
nikhil_k_that people have strongly suggested against14:58
sigmavirus24heh14:58
flaper87I know, I'm just throwing it out there :P14:59
nikhil_k_I think the Nova task status is racy and inconsistent is what I was told14:59
flaper87The status being racy has nothing to do with the use-case but the code :P14:59
*** pbourke has quit IRC14:59
*** julim has quit IRC14:59
*** wznoinsk has quit IRC14:59
nikhil_k_flaper87: that was my intial idea but it's very risky to introduce that status as it will be confusing14:59
flaper87That said, if we aim to have parallel tasks, that's probably not a good idea14:59
nikhil_k_if tasks are independent of the image workflow, it can be okay.15:00
flaper87I just don't like the idea of having to maintain a gazillion of states for the images because tasks are independant15:00
nikhil_k_because we won't expose the image until all sub-tasks are donw15:00
flaper87and OPs can have their own downstream tasks15:00
johnthetubaguywe have quite a few "fake" status codes in Nova, not getting the context from the scroll back very quickly right now15:00
*** haomaiwang has quit IRC15:01
nikhil_k_johnthetubaguy: we are discussing if image.status + image.task_status would be good idea or not15:01
flaper87johnthetubaguy: tl;dr: How should we keep the status of tasks that are in execution without messing with the image status15:01
flaper87?15:01
johnthetubaguyah I see15:01
nikhil_k_and I was suggesting that it will be racy and inconsistent15:01
flaper87nikhil_k_: damn, you were faster15:01
*** dims has joined #openstack-glance15:01
flaper87:P15:01
nikhil_k_:P15:01
*** haomaiwang has joined #openstack-glance15:02
johnthetubaguyso honestly, we don't do this well right now, we have a vague plan for "tasks" to change how we do all that (to support parallel, etc)15:02
nikhil_k_just borrowing the idea from nova vm.state and some cinder state15:02
johnthetubaguybut ignoring that I can share what we do now15:02
rosmaitai really like the current model, where task status is completely independent of image status15:02
johnthetubaguybasically we have added substates that are hidden from the API users15:02
johnthetubaguyto track specific bits of progress that are important15:03
*** dims_ has quit IRC15:03
johnthetubaguyan example is where we spot the compute node got restarted half way through an operation15:03
johnthetubaguybut we keep the API states looking the same they always did (like a state alias)15:03
johnthetubaguywould that help you out with what you need?15:03
flaper87I believe it would15:04
nikhil_k_johnthetubaguy: we are discussing https://review.openstack.org/#/c/188388/6/specs/liberty/task-automatic-triggering.rst15:04
*** julim has joined #openstack-glance15:04
nikhil_k_so the shadow state would give information about image being processed15:04
nikhil_k_but the processing would have processing.upload, processing.convert etc15:04
nikhil_k_I am just saying it could be that and not would be that15:05
nikhil_k_situation is hypothetical for now15:05
johnthetubaguynow one was is to convert every API call to a task, so its just dealt with by tasks15:05
johnthetubaguyoh wait, I see, custom triggers15:06
flaper87johnthetubaguy: yup15:06
flaper87:D15:06
nikhil_k_ I think we have a issue in tasks currently, the image delivery is underpromised by task while the regular upload somewhat overpromises15:06
*** julim_ has quit IRC15:06
johnthetubaguyyeah, I mean I quite like the upload API creating a task to do its work15:07
nikhil_k_tasks don't expose image completely until finished and regular api sets even the immutable meta during the upload15:07
flaper87nikhil_k_: but that depends on the task too, really.15:08
nikhil_k_johnthetubaguy: I can see how this would solve the defcore issue but it may open more doors. some which we discussed earlier :P15:08
flaper87ok, lets stick to the convertion example15:08
nikhil_k_oops, I am missing a webinar...15:08
flaper87to avoid confusions15:08
nikhil_k_lemme do some multi-tasking15:08
* nikhil_k_ slows down typing and looks more at chats now15:09
flaper87I'm wondering if we can start narrowing the discussion down to more specific things15:09
flaper87what I mean is, can we start answering some questions like:15:09
johnthetubaguyflaper87: +1 to that15:09
*** zul has joined #openstack-glance15:09
flaper871) Do we agree this is useful and we want it?15:09
flaper872) Do we agree on separating the models?15:10
flaper873) Do we agree on having the interim status? or do we prefer a different workflow?15:10
johnthetubaguyoh, so I am thinking, its an easer conversation if you just try to fix one of the use cases, like implementing security checks on images or converting images15:10
flaper87That way we can stop talking hypotetically and start talking with more realistic cases (like the ones proposed)15:11
johnthetubaguyyeah, I would just talk about something concrete, it will be easier15:11
flaper87johnthetubaguy: yeah, that's where I was getting at :P15:11
johnthetubaguyflaper87: cool15:11
*** wznoinsk has joined #openstack-glance15:12
flaper87The one proposed in the spec is automatic image conversion and it's the one I'm more interested at because that's the one we've gotten more feedback from OPs15:12
flaper87I think I mentioned a couple more but well...15:12
flaper87In Vancouver, over email, over IRC and even over twitter (no kidding)15:12
johnthetubaguyso the problem I have interest in is how do you unify the two different upload methods, or maybe its really, how do you do automatic image conversion using both15:12
flaper87I'd say yes: The way I've envision this is by triggering tasks based on actions.15:13
flaper87For example: image.upload would trigger tasks that need to convert the image to some other format15:13
flaper87Those tasks would run before the image is saved in the store15:13
flaper87and that's something that happens regardless of the way you import the image15:13
rosmaitayou have 2 kinds of uploads, snapshots from nova and end-user uploads15:14
rosmaitaprob don't need conversion on snapshots15:14
flaper87It's still an upload15:14
flaper87Glance doesn't care about that15:14
rosmaitawell, maybe glance should ... that's why we have an import task15:14
flaper87it gets the image, it checks the format, if it's not the expected format it converts it15:14
flaper87rosmaita: mmh, what would be the benefit of differentiating these 2 uploads?15:15
johnthetubaguyso the conversion would be based on the type being uploaded, so more than likely nova would just upload a type that doesn't have a conversion trigger registered, its probably just coming out in the wash15:15
rosmaitawell, import could have a different workflow15:15
nikhil_k_yep15:16
sigmavirus24like this is all up to operators too15:16
johnthetubaguyrosmaita: but why have two API for the same operation?15:16
flaper87sigmavirus24: exactly15:16
*** fnordahl has joined #openstack-glance15:16
sigmavirus24so operators absolutely could register a task for import that's different than upload15:16
sigmavirus24which would only hurt their users15:16
rosmaitajohnthetubaguy: because they're not really the same operation15:16
flaper87I don't understand why they are different15:17
flaper87what makes them different uploads?15:17
rosmaitatrusted source vs. untrusted source15:17
flaper87TBH, the way I see the import task is as an async copy_from15:17
johnthetubaguythere is a level of trust difference, but thats really an RBAC issue, in my head15:18
sigmavirus24trusted source (e.g., nova), untrusted source (e.g., users of a public cloud)15:18
flaper87rosmaita: but you can use RBAC or different users for that15:18
rosmaitanot really15:18
flaper87not saying it's not a real issue15:18
johnthetubaguyyou could add a nova service token to the import so you can skip some import checks, for example15:18
flaper87it's just that it's still an upload operation15:18
sigmavirus24glance's current policy is a bit ... simplistic as it is15:18
sigmavirus24the number of rules we have is very tiny and is more based around actions15:19
sigmavirus24if actions had sub-actions we'd be more capable of having finer-grained policies that would eliminate the need for separate endpoints15:19
flaper87sigmavirus24: sure15:19
flaper87ok ok, but lets not digress15:19
flaper87:D15:19
flaper87can we go back to tasks triggering ?15:19
flaper87:)15:19
*** tsekiyama has joined #openstack-glance15:19
johnthetubaguyimport vs upload is I think hitting on the image conversion problem15:20
johnthetubaguyI think they should be a single API, that both support conversion, if required15:20
rosmaitawell, the upload is synchronous whereas import is not15:20
rosmaitathat's kind of a big difference15:21
johnthetubaguyrosmaita: thats just because there are two APIs right?15:21
flaper87johnthetubaguy: tbh, they all go down the same path when it comes to storing the data15:21
flaper87johnthetubaguy: yes, one uses the tasks api15:21
flaper87the other is just the g'old upload15:21
johnthetubaguyflaper87: they don't quite though, some move data, some just do a metadata operation, as I understood it15:21
johnthetubaguyrosmaita: my thinking was the upload operation completes, but the image is still not Active until its completed the conversion15:22
johnthetubaguyrosmaita: now thats a semantic API change, that causes problems, but it does unify the two operations15:22
johnthetubaguythe upload operation could register a task as a way to report its progress, even if its not actually started via an explicit task API15:23
johnthetubaguyin those cases, you have a single way to register the conversion, I think15:23
johnthetubaguy(waves hands about widely)15:23
nikhil_k_haha15:23
nikhil_k_yup, so to me this is a battle of use-case priority15:24
nikhil_k_and I feel this needs to tie into the interop discussion for a good solid API15:24
johnthetubaguynikhil_k_: which use cases are competing here?15:24
nikhil_k_tasks make it that you can deploy clouds that make the image useable across such15:24
rosmaitanova and untrusted end user15:24
rosmaitajohnthetubaguy: ^^ the 2 use cases15:25
nikhil_k_and api consistency make it such that scripted users benefit from this15:25
* flaper87 back15:25
flaper87sorry15:25
nikhil_k_which one is more introp and interop-friendly15:26
nikhil_k_that would be a deciding factor to take the auto-task trigger approach yes or no route, I think15:26
flaper87mmh, mind elaborate on that?15:26
flaper87I feel like I lost something15:27
nikhil_k_because, both these approaches seem like interop centric15:27
nikhil_k_flaper87: which part, I may have thrown out a digest of info in a small words so if you can point out which one to elaborate more it would be useful15:27
flaper87"nikhil_k_ tasks make it that you can deploy clouds that make the image useable across such"15:28
nikhil_k_ok15:28
nikhil_k_so why import and upload needs to be two different operation is governed by that ^15:28
nikhil_k_if you have an explicit independent task API, you can use your image across different deployments using the tasks provided15:29
*** dims has quit IRC15:29
johnthetubaguyexcept you could make a single API that does both, and standardise around that15:29
flaper87I'm honestly missing the point why the automatic task triggering interfeers with that15:29
nikhil_k_advanced operations can be performed and fine tuning to optimized it using the operator deployed scripts is possible15:29
flaper87you could even have a glance-api node that is used internally without any tasks configured15:29
flaper87to which nova would upload stuff15:29
johnthetubaguybut what does the end user see, they just see something that gets uploaded correctly and works in that cloud?15:30
*** dims has joined #openstack-glance15:30
johnthetubaguyend user, I mean the API user doing the upload15:30
flaper87yes15:30
nikhil_k_johnthetubaguy: but end users are also expecting a standardized workflow on the image right?15:30
nikhil_k_like today, the definite result of an upload is active image that is ready to use15:31
johnthetubaguynikhil_k_: they don't need to see the workflow, thats an administrator thing, they are just uploading a supported image15:31
nikhil_k_given all the correct params ar provided and data is good15:31
flaper87johnthetubaguy: ++15:31
nikhil_k_yup, the key is supported image15:31
nikhil_k_there's no way to know what is a supported image if the upload workflow can be dynamic15:32
rosmaitathis is starting to look like a deployment nightmare to me15:32
johnthetubaguyan upload could go to the Error state if image validation fails after the upload API has finished right?15:32
* flaper87 will have to step out in 10 mins but this has been very useful conversation15:32
* rosmaita need to go too, can we set up another meeting to discuss again?15:32
johnthetubaguyrosmaita: why, you just get Nova to add a service token, and skip steps only if you get a valid service token (wsgi middlewear already supports all that)15:32
nikhil_k_johnthetubaguy: I think we are asking for more than validation. like conversion on image data after upload is successful15:32
rosmaitai'm worried about load on the nodes15:33
johnthetubaguynikhil_k_: its the same difference though, I think15:33
flaper87I don't think it's a real deployment mess. TBH, OPs are not even required to have tasks if they don't want it15:33
johnthetubaguyrosmaita: it should not be done on the API nodes, thats fine15:33
rosmaitaour current import tasks don't do a lot, and practically kill th3em as it is15:33
johnthetubaguyrosmaita: yes, thats crazy, you need workers for this that balance their load15:33
flaper87rosmaita: nikhil_k_ johnthetubaguy will you guys be around tomorrow at this same time?15:34
flaper87would love to keep going with this discussion15:34
* johnthetubaguy checks...15:34
flaper87johnthetubaguy: fwiw, we're already using taskflow, which should help with the workers aspect of this15:34
rosmaitaflaper87: what time do you mean UTC?15:34
nikhil_k_johnthetubaguy: today users set disk and container format on the image and it's immutable as it's a known deployment scenario. with upload users would be kind of in the blind until after image has been processed by a task identifier that's unknown and converted into a completely different format15:34
flaper8714/15 UTC ?15:34
nikhil_k_works for me flaper8715:35
flaper87nikhil_k_: awesome, thanks15:35
rosmaita14:00 to 15:00 is good15:35
flaper87awesome15:35
nikhil_k_flaper87: the reason for delay in input is this, now you know. it's a complex change that involves affect to other parties15:35
nikhil_k_so, good to have people around for interactive convo15:35
johnthetubaguyyeah, ping me, I should be able to jump in15:35
flaper87sounds good15:36
*** DuncanT has quit IRC15:36
*** marcusvrn_ has quit IRC15:36
nikhil_k_personally, I think we could go either way. Just need to know the definite priority use case for this change.15:36
nikhil_k_defining*15:37
nikhil_k_something that ties into the wider openstack goal15:37
flaper87I had asked the folks that pinged me through different mediums to chime in on the spec with their +1's and whatnot15:37
flaper87to be entirely honest15:37
rosmaitai think the question is, do we complicate tasks, or do we complicate images?15:37
rosmaitaflaper87: i think the functionality is definitely important, jsut a matter of where to put it15:38
flaper87I think our tasks feature without auto-triggering is really limited and close to not having enough use cases to make it worth maintaining15:38
flaper87rosmaita: totally agreed15:38
nikhil_k_flaper87: I see, wondering if they would be willing to shed some linght on the usecases too then15:38
flaper87I'll do my homework and try to think about a different solution15:38
nikhil_k_rosmaita: I think tasks are bound to be complicated irrespective of images15:39
nikhil_k_today it's end user tasks15:39
flaper87nikhil_k_:  leseb is one of them, I think I can find the guy that pinged me on twitter and I'll comment on the spec too15:39
nikhil_k_tomorrow we will have RBAC on tasks too if we need to differential between end user triggers and admin triggered15:39
nikhil_k_flaper87: thank you!!15:39
nikhil_k_rosmaita: also, we will have to identify which task is being run on the upload trigger (provide info to end user)15:40
flaper87On a closing note, I kinda think tasks should be an internal thing and definitely an undercloud feature15:40
nikhil_k_oops..15:41
nikhil_k_so that they can know the error message and some result info15:42
nikhil_k_flaper87: sure15:43
nikhil_k_if that's what the use-cases are of priority15:43
*** gberginc has quit IRC15:43
nikhil_k_I kinda like what jcook says, we are developing towards use-cases / business needs. So, the differentiating factor here is what is that and what does openstack want15:44
*** sayali has quit IRC15:45
*** dims has quit IRC15:45
*** zul has quit IRC15:46
*** spzala has joined #openstack-glance15:46
*** dims has joined #openstack-glance15:47
*** groen692 has quit IRC15:54
openstackgerritAlexander Tivelkov proposed openstack/glance: Fixed version unequality artifact filtering  https://review.openstack.org/21418515:56
openstackgerritAlexander Tivelkov proposed openstack/glance: Fixed version unequality artifact filtering  https://review.openstack.org/21418515:58
*** marcusvrn_ has joined #openstack-glance15:58
*** haomaiwang has quit IRC16:01
*** haomaiwang has joined #openstack-glance16:01
*** sayali has joined #openstack-glance16:03
*** haomaiwang has quit IRC16:04
openstackgerritKairat Kushaev proposed openstack/glance: Show properties with empty values  https://review.openstack.org/21425516:05
*** gberginc has joined #openstack-glance16:06
*** jecarey has joined #openstack-glance16:06
jcookyou said use cases / business needs, so I thought it might be useful to share this: https://drive.google.com/folderview?id=0BxtM4AiszlEyfm9UTW5LMEQ5cUhHbmFsSkd5WFNfdTMwVFIwRUM1TVFXSHhhWHl6VHlpRzg&usp=sharing16:07
jcookProduct Working Group is at mid-cycle this week and they are trying to get more involved in helping drive use cases upstream16:07
*** jistr has quit IRC16:08
openstackgerritChris Buccella proposed openstack/python-glanceclient: Allow using image name for image-show using v2  https://review.openstack.org/21425816:08
sigmavirus24jcook: that'd be awesome for them to help with this kind of stuff16:11
*** DuncanT has joined #openstack-glance16:12
jcookI've gotten involved a bit with the group, but mostly from the standpoint of having performance and reliability tests across various configurations (scale and elasticity)16:14
jcookI know the RPC product managers from Rackspace are pretty involved and there is pretty good representation across companies (Intel, IBM, ...). Not sure if I saw a HP PM or not. I'm new to the community16:15
sigmavirus24jcook: that's good to know16:15
sigmavirus24After http://www.rackspace.com/blog/newsarticles/rackspace-collaborates-with-intel-to-accelerate-openstack-enterprise-feature-development-and-adoption/ I'm wondering if Intel has involved itself yet with the Product WG16:16
*** exploreshaifali has joined #openstack-glance16:18
*** pbourke_ is now known as pbourke16:21
openstackgerritGB21 proposed openstack/glance: Add a soft delete functionality for tasks.  https://review.openstack.org/20925516:22
*** exploreshaifali has quit IRC16:22
*** gberginc has quit IRC16:28
*** gberginc has joined #openstack-glance16:28
*** gberginc has quit IRC16:28
*** exploreshaifali has joined #openstack-glance16:31
*** sdake_ is now known as sdake16:34
*** dims has quit IRC16:36
*** dims has joined #openstack-glance16:39
*** annegentle has quit IRC16:41
*** dims has quit IRC16:54
*** dims has joined #openstack-glance16:57
*** sgotliv_ has quit IRC16:59
*** dims_ has joined #openstack-glance17:02
openstackgerritTomoki Sekiyama proposed openstack/glance_store: Implement get, add and delete for cinder store  https://review.openstack.org/16641417:03
*** dims has quit IRC17:04
*** markus_z has quit IRC17:07
*** smatzek has quit IRC17:08
*** dims has joined #openstack-glance17:08
*** dims_ has quit IRC17:09
*** dims_ has joined #openstack-glance17:11
*** dims has quit IRC17:14
*** ayoung has quit IRC17:17
*** dims_ has quit IRC17:20
*** boris-42 has quit IRC17:20
*** dims has joined #openstack-glance17:20
*** gberginc has joined #openstack-glance17:20
*** dims has quit IRC17:20
*** dims has joined #openstack-glance17:21
*** MattMan has quit IRC17:34
*** TravT has joined #openstack-glance17:41
hogepodgenikhil_k_: We're at the operators meetup right now17:42
hogepodgenikhil_k_: there's a lot of talk about glance and feature requests. Think you can be available to join a hangout for feedback?17:42
hogepodgejohnthetubaguy: ^^17:42
*** TravT_ has quit IRC17:43
johnthetubaguyhogepodge: I could answer nova question17:43
johnthetubaguys17:43
*** changbl has joined #openstack-glance17:46
hogepodgejohnthetubaguy: sure, that would be cool. Lots of PTLs in the room, and virtual attendance would make the state of awesome even more awesome17:47
nikhil_k_hogepodge: I can join17:49
*** achanda has joined #openstack-glance17:51
*** harshs_ has joined #openstack-glance17:55
achandaHey guys, can I use cinder as a backend for glance in Kilo?17:56
*** harshs has quit IRC17:56
*** harshs_ is now known as harshs17:56
*** ayoung has joined #openstack-glance18:03
*** chlong has quit IRC18:06
*** chlong has joined #openstack-glance18:09
*** openstackgerrit_ has quit IRC18:11
cpallaresachanda: no, but hopefully sometime soon with this patch https://review.openstack.org/#/c/166414/18:18
sigmavirus24achanda: it's a think you can set in glance.conf, but it's not something anyone would suggest you set ;)18:19
sigmavirus24(or what cpallares said before I realized she said it better than I did)18:19
sigmavirus24that patch isn't the only one iirc18:20
sigmavirus24we also need to add oslo.rootwrap to glance18:20
sigmavirus24which is ... far from ideal18:20
achandathanks guys :)18:20
sigmavirus24There's apparently a better alternative around the corner that bknudson mentioned in #openstack-security last week but I've already forgotten the name18:21
achandasigmavirus24: that sounds interesting!18:26
sigmavirus24oslo.rootwrap is ... not secure by default18:26
achandalet me try to look18:26
sigmavirus24which is why I left the security docs about it on the review that imports oslo.rootwrap into glance18:27
achandanot secure and often slow18:27
achandawe had problems in neutron18:27
achandaended up using sudo18:27
sigmavirus24O_O18:28
sigmavirus24that sounds even worse18:28
achandaI guess the dameon mode will help with slowness though18:29
*** ducttape_ has quit IRC18:29
sigmavirus24achanda: https://review.openstack.org/#/c/204073/18:29
sigmavirus24(from bknudson in #openstack-keystone)18:30
achandathanks sigmavirus2418:31
sigmavirus24You're welcome18:31
achandaabout the perf issues: http://lists.openstack.org/pipermail/openstack-dev/2014-March/029017.html18:32
sigmavirus24achanda: that's over a year old, surely that's still not a problem =P18:34
sigmavirus24/s18:34
achandatrue18:34
achandahonestly, I haven't looked at the daemon mode18:35
achandaneed to try that out18:35
*** ducttape_ has joined #openstack-glance18:47
nikhil_k_thanks jcook18:52
jcooksigmavirus24: I think the moderator / chair of the meeting I was in was someone from intel18:54
*** ducttape_ has quit IRC18:54
jcooksigmavirus24: I think there were at least 2 intel people on list, let me pull up etherpad18:54
*** ducttape_ has joined #openstack-glance18:55
jcooksigmavirus24: yeah, Carol: https://etherpad.openstack.org/p/User_Story_Discussion_1 https://etherpad.openstack.org/p/User_Story_Discussion_218:56
*** smatzek has joined #openstack-glance18:56
jcooksigmavirus24: she seems to be leading many of the discussions and organization18:56
*** sdake_ has joined #openstack-glance18:56
*** sdake has quit IRC18:59
*** annegentle has joined #openstack-glance19:06
*** e0ne has joined #openstack-glance19:07
*** ayoung has quit IRC19:08
*** smatzek has quit IRC19:15
*** sdake_ is now known as sdake19:16
*** GB21 has quit IRC19:21
*** annegentle has quit IRC19:21
*** achanda has quit IRC19:22
sigmavirus24jcook: that's awesome19:22
*** annegentle has joined #openstack-glance19:26
*** harshs has quit IRC19:34
*** belmoreira has joined #openstack-glance19:36
*** e0ne has quit IRC19:44
*** belmoreira has quit IRC19:51
*** ayoung has joined #openstack-glance19:51
*** annegentle has quit IRC19:57
*** sigmavirus24 is now known as sigmavirus24_awa19:57
*** sigmavirus24_awa is now known as sigmavirus2419:58
*** annegentle has joined #openstack-glance20:00
*** gberginc has quit IRC20:04
*** boris-42 has joined #openstack-glance20:19
flaper87nikhil_k_: https://review.openstack.org/#/c/197594/ don't forget20:20
*** achanda has joined #openstack-glance20:22
*** achanda has quit IRC20:22
*** achanda has joined #openstack-glance20:23
*** zul has joined #openstack-glance20:28
*** achanda has quit IRC20:28
*** achanda has joined #openstack-glance20:28
*** changbl has quit IRC20:40
*** ctina has quit IRC20:49
openstackgerritAbhishek Chanda proposed openstack/glance: Pass CONF to logging setup  https://review.openstack.org/21434220:50
*** cdelatte has quit IRC20:57
*** julim has quit IRC21:00
*** zul has quit IRC21:00
*** vijendar has quit IRC21:01
*** vijendar has joined #openstack-glance21:01
*** jecarey has quit IRC21:17
*** zul has joined #openstack-glance21:22
*** ducttape_ has quit IRC21:26
openstackgerritAbhishek Chanda proposed openstack/glance: Pass CONF to logging setup  https://review.openstack.org/21434221:27
*** ducttape_ has joined #openstack-glance21:27
*** sileht has quit IRC21:43
*** edmondsw has quit IRC21:44
*** zul has quit IRC21:49
cpallaresnikhil_k_: Can I get your input on this? https://review.openstack.org/#/c/197594/21:50
nikhil_k_cpallares: surely, today Iam planning to approve it. Just need to note down points for me to amend the spec21:51
nikhil_k_cpallares: I am doing that for all proposed and merged this cycle21:52
nikhil_k_will update and merge them in bunch :P21:52
cpallaresnikhil_k_: Thanks :)21:52
achandahow well supported or used is glance-replicator in production?21:55
achandathere is not much info on it around...21:56
*** wokuma has joined #openstack-glance22:08
*** vijendar has quit IRC22:11
sigmavirus24achanda: I have it on my list to fix a ton of bugs22:13
sigmavirus24But I have not had time to work on glance or anything else usptream lately22:13
sigmavirus24so, sorry, and you're welcome =P22:13
achanda:)22:13
*** harshs has joined #openstack-glance22:13
wokumasigmavirus24: Hi Ian, I would like to discuss i18n work I need to do for metadef JSON files...who would be the best person to ask about this in Glance?22:14
wokumaor can i ask you? :)22:14
sigmavirus24I'm certainly not the most qualified person to ask :D22:15
sigmavirus24Like, actually I'm probably the least qualified person22:15
wokumaok...:)22:15
sigmavirus24nikhil_k_: can easily direct your questions to the right people with better certainty22:15
wokumaok...thanks.22:15
wokumanikhil_k:: any tips22:16
sigmavirus24nikhil_k_: ^22:17
*** annegentle has quit IRC22:17
wokumasigmavirus: guess he's not around right now...22:18
*** sigmavirus24 is now known as sigmavirus24_awa22:18
*** tpeoples has quit IRC22:32
*** tpeoples has joined #openstack-glance22:39
*** ducttape_ has quit IRC22:50
*** exploreshaifali has quit IRC23:07
*** annegentle has joined #openstack-glance23:08
*** annegentle has quit IRC23:09
*** annegentle has joined #openstack-glance23:10
*** harshs has quit IRC23:11
*** dims_ has joined #openstack-glance23:20
*** achanda has quit IRC23:22
*** dims has quit IRC23:22
*** chlong has quit IRC23:29
*** achanda has joined #openstack-glance23:30
*** achanda has quit IRC23:30
*** achanda has joined #openstack-glance23:31
*** changbl has joined #openstack-glance23:31

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!