| *** chenhuayi has joined #openstack-smaug | 01:27 | |
| openstackgerrit | smile-luobin proposed openstack/smaug: Fix serialized_meta in SwiftBankPlugin https://review.openstack.org/301427 | 03:09 |
|---|---|---|
| openstackgerrit | yinwei proposed openstack/smaug: Restore design spec (protection service level) https://review.openstack.org/296950 | 03:09 |
| *** gampel1 has joined #openstack-smaug | 05:55 | |
| *** yuval has joined #openstack-smaug | 06:09 | |
| *** c00281451 has joined #openstack-smaug | 06:21 | |
| c00281451 | gampel & yuval: good morning. if you are on line, we can discuss. you have give me many good comments. thanks. | 06:26 |
| *** c00281451 is now known as chenzeng | 06:26 | |
| yuval | hey chenzeng | 06:26 |
| yuval | how are you? | 06:26 |
| gampel1 | Hi everyone | 06:26 |
| chenzeng | yuval:ya, fine. and you? | 06:27 |
| chenzeng | hi gampel. | 06:27 |
| yuval | chenzeng: good :) | 06:27 |
| chenzeng | if you are have time now. we can start. | 06:27 |
| gampel1 | sure | 06:27 |
| chenzeng | ok. | 06:27 |
| chenzeng | we start one by one. | 06:27 |
| gampel1 | Ok | 06:28 |
| chenzeng | 1. yuval said add add/remove/update trigger rpc interfaces. but i think we can create trigger when we use it. | 06:29 |
| yinwei_computer | ping gampel1 | 06:29 |
| yinwei_computer | chenzeng is using this thread, right? | 06:30 |
| yinwei_computer | can I take Eran for several minutes? | 06:30 |
| chenzeng | yinwei:ok | 06:31 |
| yinwei_computer | ok, I will send private message to him | 06:31 |
| gampel1 | yinwei_computer:: yes sure if we want to talk about the stack i think it will be best to include saggi | 06:31 |
| yinwei_computer | chenzeng, go ahead pls. | 06:31 |
| yinwei_computer | is him online now? | 06:32 |
| yinwei_computer | sure | 06:32 |
| gampel1 | he will be here soon i can ping you once he is here | 06:32 |
| yinwei_computer | ok | 06:32 |
| yinwei_computer | that's good | 06:32 |
| yinwei_computer | we can start without conflict with chenzeng | 06:32 |
| chenzeng | gampel:what's your opinion about the first topic. | 06:33 |
| chenzeng | yinwei:thanks. | 06:33 |
| gampel1 | why not have the trigger data in the Operation engine memory via RPC | 06:34 |
| gampel1 | It is not allot of data and it will be easy to sync startup / update ect | 06:34 |
| chenzeng | gampel:because use may create the trigger, but not use it. | 06:35 |
| chenzeng | because user may create the trigger, but not use it. | 06:35 |
| gampel1 | I understand but it is just small memory footprint | 06:36 |
| openstackgerrit | Yuval Brik proposed openstack/smaug: Move swift client to be created by ClientFactory https://review.openstack.org/300854 | 06:36 |
| gampel1 | for the first phase I do not see a problem and I can not imagine the trigger list becoming so big | 06:36 |
| chenzeng | so you agree with adding more 3 rpc interfaces, do you? | 06:36 |
| gampel1 | I think this is the obvious solution that will be be easy to understate and follow the flow | 06:37 |
| chenzeng | ok. i will update. | 06:37 |
| gampel1 | If latter we have performance issues we could think of optimizations | 06:38 |
| chenzeng | we move to the "start_greenthread" | 06:38 |
| yuval | ok | 06:38 |
| chenzeng | gampl & yuval:could you please recieve the document. | 06:39 |
| chenzeng | i have send a picture to decribe my idea. | 06:39 |
| yuval | what document? | 06:39 |
| gampel1 | i did but it is very slow | 06:40 |
| chenzeng | i send it to you from irc | 06:40 |
| gampel1 | can you send it by mail | 06:40 |
| chenzeng | ok | 06:40 |
| chenzeng | yuval:can you give me your email | 06:40 |
| yuval | yuval@brik.org.il | 06:41 |
| chenzeng | ok, wait a minute. | 06:41 |
| gampel1 | chenzeng: yes "start_greenthread" | 06:41 |
| chenzeng | i have sended the email to you. | 06:45 |
| yuval | I received it | 06:45 |
| gampel1 | i did not get it | 06:46 |
| chenzeng | ok, i send it again. | 06:47 |
| yuval | chenzeng: why would compute_next_time(now-window) return t4 and not t2? | 06:48 |
| yuval | chenzeng: I understand what you wrote | 06:49 |
| gampel1 | Got it | 06:49 |
| yuval | chenzeng: this might happen only if the time format is not fixed (deterministic) | 06:49 |
| chenzeng | yuval:yes. | 06:51 |
| gampel1 | The relative use case are not so important in my view | 06:51 |
| gampel1 | We could event limit it as know to time format that are absolute | 06:51 |
| gampel1 | even | 06:51 |
| gampel1 | Currently crontab is absolute | 06:52 |
| chenzeng | gampel:yes, crontab is absolute. | 06:52 |
| gampel1 | I do not think it is impotent to have relative recurrent and it does not matter when is the start | 06:53 |
| gampel1 | if it is the trigger creation or the first operation using it | 06:53 |
| gampel1 | We could limit the trigger formats to only absolute | 06:54 |
| chenzeng | gapel:but rfc2445 is more flexible. use may create a trigger using rfc2445 that is not absolute time. | 06:54 |
| chenzeng | user | 06:54 |
| gampel1 | in rfc2445 you define start time and the recurrent becomes absolute | 06:55 |
| yuval | another thing which supports absolute recurrence: | 06:56 |
| yuval | in case the service is restarted, or started in different locations | 06:56 |
| yuval | it must be ensured | 06:56 |
| gampel1 | yes i agree | 06:56 |
| yuval | that the events will occur | 06:56 |
| gampel1 | chenzeng: what do you think ? | 06:56 |
| chenzeng | if we only suport absolute time, i agree. | 06:57 |
| gampel1 | any other thoughts for any one else about this ? | 06:57 |
| gampel1 | Ok so lets agree we only support absolute recurrent time definitions | 06:58 |
| gampel1 | chenzeng: by the way very good job on the Operation engine | 06:59 |
| chenzeng | i have one more topic to discuss. | 06:59 |
| gampel1 | chenzeng:Yes please me too | 06:59 |
| chenzeng | about the timezone of trigger. | 07:00 |
| gampel1 | yes my topic as well | 07:00 |
| chenzeng | user is familiar with local time, and we change it to utc to keep the same with other modules like DB. | 07:00 |
| gampel1 | chenzeng: our thoughts now is to have the API and DB all in UTC | 07:00 |
| gampel1 | the UI will convert from USer local time to UTC | 07:00 |
| gampel1 | so we only operate in UTC and all the server are set to UTC | 07:01 |
| chenzeng | ok. i got it. | 07:01 |
| gampel1 | what do you think ? | 07:01 |
| chenzeng | window should be less than the interval between two time points and be greater than 1 hour. | 07:01 |
| chenzeng | gampel: i agree with you about the timezone. | 07:02 |
| gampel1 | Ok great | 07:02 |
| chenzeng | gampel:window should be less than the interval between two time points and be greater than 1 hour. | 07:02 |
| yuval | chenzeng: why should the window be greater than 1 hour? | 07:02 |
| chenzeng | this is you said in the comments. | 07:03 |
| gampel1 | Ok i think that we should limit the interval | 07:03 |
| chenzeng | https://review.openstack.org/#/c/296880/12/smaug/operationengine/engine/triggers/timetrigger/timeformats/crontab_time.py | 07:03 |
| gampel1 | so the minimal interval will be hour | 07:04 |
| chenzeng | ok. | 07:04 |
| gampel1 | the window is how long did it take me to run the operation from the absolute trigger time | 07:05 |
| gampel1 | so i do not think it should be be greater than 1 hour | 07:05 |
| *** x00350071_ is now known as xiangxinyong | 07:06 | |
| gampel1 | as a default it should be minutes max | 07:06 |
| chenzeng | gampel:sorry. you said the interval between two trigger time points should be greater than 1 hour. | 07:06 |
| gampel1 | i mean the interval not the window | 07:07 |
| chenzeng | gampel:the window should be less than the interval between two time points. | 07:07 |
| gampel1 | yes i agree | 07:08 |
| gampel1 | i think the default should be 1 minute | 07:08 |
| gampel1 | for the window | 07:08 |
| chenzeng | ok. the interval is greater than 1 hour. and window is less than interval | 07:08 |
| chenzeng | now the default for window is 1 minuts. | 07:09 |
| gampel1 | I would even say that teh window max is 1/2 of the interval | 07:09 |
| chenzeng | got it. | 07:09 |
| gampel1 | Any other open issues ? | 07:10 |
| chenzeng | no. | 07:10 |
| chenzeng | thanks gampel & yuval. | 07:10 |
| gampel1 | thank you for a very good job | 07:10 |
| yuval | thank you :D | 07:10 |
| gampel1 | whats the status of the time trigger patch | 07:11 |
| chenzeng | today i will update it based on our discuss. | 07:11 |
| gampel1 | Thanks ping us to review it so we coudl try and merge it | 07:12 |
| xiangxinyong | Hi Eran. | 07:12 |
| gampel1 | hi | 07:12 |
| chenzeng | gampel:ok. | 07:12 |
| gampel1 | xiangxinyong chenying : were you able to tag the python client | 07:13 |
| xiangxinyong | eran: we think chenying has no rights to tag it. | 07:13 |
| gampel1 | Ok let mecheck | 07:13 |
| xiangxinyong | we think he is not in the smaug-release group | 07:14 |
| gampel1 | i just added him can you please check | 07:15 |
| xiangxinyong | ok. He is in a meeting. i will tell him. Thanks | 07:15 |
| gampel1 | let me knwo if it does not work | 07:15 |
| saggi | ping yinwei_computer | 07:16 |
| gampel1 | whats the status of the UI ? | 07:16 |
| xiangxinyong | eran, do you think we need to add openstack bot in our project like this. https://review.openstack.org/#/c/289584 | 07:16 |
| gampel1 | yes can you sent the patch ? | 07:16 |
| gampel1 | yinwei: ping | 07:17 |
| xiangxinyong | yes, i will do at once. | 07:17 |
| xiangxinyong | eran: i setup a sketon of ui, but i have some details to confirm with you and saggi. | 07:17 |
| gampel1 | please add all of us as reviewers | 07:17 |
| xiangxinyong | when do you have spare time? | 07:17 |
| gampel1 | do you want to discuss know | 07:17 |
| xiangxinyong | eran: ok, i will | 07:17 |
| gampel1 | now | 07:17 |
| xiangxinyong | yeah. | 07:18 |
| gampel1 | Ok lets go | 07:18 |
| saggi | xiangxinyong: I have some now if yinwei_computer and wangliuan are not here. | 07:18 |
| xiangxinyong | saggi: so? | 07:18 |
| gampel1 | xiangxinyong: we could start do you need to send us something ? | 07:19 |
| xiangxinyong | do we make espace meeting? | 07:20 |
| gampel1 | sure saggi you can join me on my computer | 07:20 |
| xiangxinyong | i could share my screen | 07:21 |
| gampel1 | Ok | 07:21 |
| yinwei_computer | I'm here | 07:53 |
| yinwei_computer | sorry, was talking with chenying | 07:53 |
| yinwei_computer | hello, saggi | 07:53 |
| saggi | yinwei_computer: hi | 07:53 |
| yinwei_computer | are you free to talk now? | 07:53 |
| saggi | yes | 07:53 |
| saggi | yinwei_computer: How are you doing? | 07:54 |
| yinwei_computer | good, except finding your comment on task stack | 07:54 |
| yinwei_computer | :) | 07:54 |
| yinwei_computer | kidding | 07:54 |
| yinwei_computer | pls. go ahead | 07:54 |
| yinwei_computer | do you need wangliuan to be here? | 07:55 |
| yinwei_computer | you're looking for him as well? | 07:55 |
| saggi | yinwei_computer: I would prefer it just so I know we are all on the same page. But you can explain to him later if he is unavailable. | 07:55 |
| yinwei_computer | ok | 07:55 |
| yinwei_computer | what's issue about liuan? | 07:56 |
| saggi | yinwei_computer: In general I like the stack idea. It's a very clean solution to the problem of attaching to parent/child tasks in the tree. | 07:56 |
| yinwei_computer | the task stack or flow engine? | 07:56 |
| saggi | both | 07:56 |
| yinwei_computer | ok | 07:57 |
| saggi | but the stack is written on top of the flow engine | 07:57 |
| yinwei_computer | it's written based on the interface of flow engine, not bond to implementation | 07:57 |
| saggi | yinwei_computer: The only issue is that the stack is managed by the base plugin. This means that a plugin can override this behavior and corrupt the stack. The only thing we want to change is to have the stack managed by the ResourceGraphWalker implementation instead of the base plugin. | 07:58 |
| yinwei_computer | your point is it's not good to locate in base protection plugin, right? | 07:58 |
| saggi | Yes | 07:58 |
| yinwei_computer | Let me see | 07:58 |
| saggi | What we suggest is having the task builder have an internal stack. | 07:59 |
| saggi | The RGW will push a noop task on enter and pop it on exit. | 07:59 |
| saggi | the plugin can only look at the top of the stack. | 07:59 |
| saggi | and can either link to this noop task or create unrelated tasks that will run in parallel. | 08:00 |
| saggi | The top of the stack in on_enter is the parent task (meaning I wait until the parent is done) | 08:00 |
| saggi | the top of the stack in on_exist (is a task that depends on all the child stacked tasks) | 08:01 |
| saggi | This means that I get the functionality without letting the plugin manage the stack itself. | 08:01 |
| saggi | yinwei_computer: Am I being clear enough? | 08:02 |
| yinwei_computer | thinking | 08:02 |
| yinwei_computer | confirm: | 08:03 |
| saggi | :) | 08:03 |
| gampel1 | I am in another meeting will join you when i am done | 08:03 |
| yinwei_computer | ok | 08:04 |
| yinwei_computer | 1. RGW will manage the task stack but won't expose the task stack to plugin | 08:04 |
| yinwei_computer | 2. RGW will expose some interfaces for plugin to operate task create and task link | 08:05 |
| yinwei_computer | saggi, hello? | 08:09 |
| saggi | yinwei_computer: 1 sec yuval is having comments too | 08:10 |
| yinwei_computer | ok | 08:10 |
| *** asdfghj has joined #openstack-smaug | 08:10 | |
| saggi | 1 yes | 08:11 |
| saggi | 2 this is what Yuval is suggesting we can remove. I'm trying to understand him now. | 08:12 |
| yinwei_computer | checked the code | 08:14 |
| yinwei_computer | it seems to me that, on_resource_start in base plugin could keep the same, right? | 08:15 |
| yinwei_computer | since plugin still need build task or search task and append it | 08:15 |
| saggi | yinwei_computer: 1 min | 08:16 |
| yinwei_computer | or we just require on_resource_start to return a task of this resource? | 08:16 |
| yinwei_computer | it looks to me that we need divide which part should be done by plugin while which part should be done by RGW | 08:18 |
| saggi | yinwei_computer: yuval had a good idea let me try and explain it. | 08:24 |
| saggi | yinwei_computer: Let us no what you think about it | 08:24 |
| saggi | yinwei_computer: He says that we don't really need a stack if we change things a little bit. | 08:24 |
| saggi | yinwei_computer: First we make on_enter() optional it will be used only for scoping aggregate protection plugins. | 08:25 |
| saggi | yinwei_computer: Second we have on_exit() return a task. | 08:25 |
| saggi | Now the way it works is that each node's returned in on_exit() will automatically depend on all it's childrens' returned tasks. This will be done in RGW. | 08:26 |
| saggi | If the plugins wants a task that depend on nothing it can just create it and not return it. It will return a noop instead. This will make a task that is independent. | 08:27 |
| saggi | If the plugins wants a task to run after it's children have finished it can just return it. | 08:27 |
| saggi | *BTW when I say task I mean Atom (task or flow) | 08:28 |
| saggi | If the plugin want to make several task to run in parallel but declare that it's done only after both of them finish it can make a nnop task and have both tasks depend on the noop and return the noop. | 08:29 |
| yinwei_computer | hmm, I recalled that we used to think in this way as you told above | 08:29 |
| saggi | The main difference is not having a task in on_enter() | 08:29 |
| yinwei_computer | but there's some problem in this solution | 08:29 |
| saggi | Which was confusing | 08:30 |
| saggi | What problem? | 08:30 |
| yinwei_computer | yes, I recalled that Bin has proved that we have to append the task on_enter | 08:30 |
| yinwei_computer | let me recall details | 08:30 |
| saggi | From what I recall the main issue was that the difference between on_enter and on_exit was confusing. | 08:31 |
| yinwei_computer | ok | 08:35 |
| yinwei_computer | the problem is when you link your parent task to children task, you have no idea which tasks are your children tasks | 08:36 |
| saggi | That was the problem with on_enter() when you do on_exit() you necessarily visited all the children and collected their tasks. | 08:37 |
| yinwei_computer | yes, but how to differenciate the task below on the stack is your right sibling or your parent? | 08:38 |
| yinwei_computer | say, you have task1 as parent, task2 and task3 as children | 08:39 |
| yuval | and your own task | 08:39 |
| yinwei_computer | the stack of tasks should be task2, task3, task1 | 08:39 |
| yuval | nvm | 08:39 |
| yinwei_computer | shall we link task2 to task3? | 08:39 |
| yinwei_computer | or link task2 and task3 to task1? | 08:39 |
| saggi | the latter | 08:40 |
| yinwei_computer | the idea of current way is only keep parent task on top of the stack | 08:40 |
| yinwei_computer | how to tell on push task3? | 08:40 |
| yinwei_computer | shall we pop task2 out and link it to task3, or shall we push task3 and then pop the two after task1 pushed into stack | 08:41 |
| yinwei_computer | we need extra context to tell the right sibling and the parent | 08:41 |
| * saggi is thinking... | 08:41 | |
| saggi | yinwei_computer: When before on_enter() we mark the size of the stack. after on_exit() we pop until the stack is the same size as in on_enter() and link them to the returned task and push it to the stack. | 08:43 |
| saggi | It's all O(1) operations. | 08:44 |
| yinwei_computer | actually, we are trying to reconstruct a tree by its iteration result. AFAK, at least we need two ordering of iteration result, we can reconstruct it. first root and middle root. we need at least two information | 08:45 |
| yinwei_computer | I don't get your point | 08:46 |
| * saggi is cooking some pseudo code | 08:46 | |
| yuval | we can push on the dfs stack a tuple: (node, parent node) | 08:47 |
| yinwei_computer | actually, the current way is trying to make use the stack calling of the dfs itself | 08:48 |
| yinwei_computer | I don't see any benefit to save extra information to simulate it | 08:48 |
| saggi | yinwei_computer: http://fpaste.org/349705/59846388/ | 08:53 |
| yinwei_computer | * yinwei_computer is reading code | 08:53 |
| yinwei_computer | I need switch to blue cloud to check your code | 08:54 |
| *** zhang2005 has joined #openstack-smaug | 08:54 | |
| saggi | yinwei_computer: Do you have a paste site you can connect to? | 08:54 |
| zhang2005 | hi i am zhangshuai | 08:54 |
| saggi | hi | 08:55 |
| *** pzchen has joined #openstack-smaug | 08:55 | |
| yinwei_computer | saggi: could you pls. send it to my email yinwei3@huawei.com | 08:55 |
| yinwei_computer | It seems that personal storage is denied here | 08:56 |
| saggi | yinwei_computer: sent | 08:57 |
| *** pzchen has quit IRC | 08:58 | |
| yinwei_computer | saggi, I see your pc | 09:00 |
| yinwei_computer | question, how to tell the marker of different node? | 09:00 |
| saggi | yinwei_computer: You don't need to. Since we are doing DFS. We know that the topmost marker is our marker and the next topmost marker is the parent marker. | 09:01 |
| yinwei_computer | then it's the same with the current way, just we move the task append and task link out | 09:02 |
| yinwei_computer | the marker in current way is the task produced by the protection plugin | 09:02 |
| yinwei_computer | I agree we could move the append and link logic to RGW, but I don't see the benefit of inserting a marker | 09:03 |
| saggi | Without the marker you can't group siblings tasks. | 09:04 |
| saggi | yinwei_computer: Yuval suggest we use two stacks | 09:07 |
| saggi | instead of a marker | 09:07 |
| yinwei_computer | why not just have on_task_start to return task and just move the task append and task link to RGW | 09:08 |
| yinwei_computer | I mean I didn't see the benefit of marker compared with just insert parent task on node enter | 09:08 |
| saggi | what is on_task_start? | 09:08 |
| saggi | This means that the protection plugins knows about the details of the linking | 09:09 |
| yinwei_computer | plugin.on_resource_start to produce its own task | 09:09 |
| yinwei_computer | and then BGW append it to task stack | 09:09 |
| yinwei_computer | this is the same as the marker | 09:10 |
| yinwei_computer | the marker is the parent task now | 09:10 |
| saggi | yinwei_computer: hmm | 09:10 |
| saggi | But that means that if the plugin needs to modify the task in on_resource_exit() it has to somehow do the bookkeeping itslef | 09:10 |
| saggi | Plugins that manage multiple resources in the same task can only know the real task on_exit | 09:11 |
| yinwei_computer | it could call RGW.task_stack.peek() to get the task, right? | 09:11 |
| saggi | If we expose it to it. | 09:12 |
| saggi | But I would rather not. | 09:12 |
| yinwei_computer | we can have RGW to provide interfaces to retrieve task stack, but without exposing the interfaces of construct them | 09:12 |
| saggi | But that means that we bind the plugin implementation to the stack implementation. | 09:13 |
| yinwei_computer | why? | 09:14 |
| yinwei_computer | it means we do bookeeping for the plugin, it could retrieve its context through RGW | 09:14 |
| yinwei_computer | not necessarily stack | 09:14 |
| yinwei_computer | it doesn't need know there's a stack | 09:14 |
| saggi | What it really means is that we push the parent task as part of the context. | 09:15 |
| yinwei_computer | yes, of course | 09:15 |
| saggi | Since the plugin doesn't need the entire stack. | 09:15 |
| * saggi is thinking.... | 09:15 | |
| yinwei_computer | that's why there is a task stack | 09:15 |
| yinwei_computer | why we made a task stack is because we don't want walk() function to return certain value, like task | 09:16 |
| yinwei_computer | that's why we made task stack into context and plugin | 09:16 |
| yinwei_computer | we just have plugin to produce task in on_resource_start, RGW will append it to stack; while on_resource_end, plugin could do nothing but RGW will link tasks by default. If it want to change its task on resource end, it could call RGW.get_current_task, and the top of the task should be the task of this plugin. | 09:18 |
| yinwei_computer | the task could be a task flow itself | 09:19 |
| saggi | yinwei_computer: I sent you another pseudo code to make sure we are on the same page | 09:20 |
| gampel1 | saggi: launch ? | 09:20 |
| saggi | yinwei_computer: Tell me if I got this correctly | 09:20 |
| gampel1 | yuval ? | 09:20 |
| saggi | gampel1: 10 min, I think we're close to agreeing. | 09:20 |
| yinwei_computer | checking saggi's pseudo code | 09:21 |
| yinwei_computer | saggi: we need link tasks in RGW's on_node_exit, missing it? | 09:23 |
| yinwei_computer | saggi: in RGW's on_node_enter, if the node visited, we don't call plugin.on_resource_start but go to search its task in task stack and push it to stack. | 09:24 |
| yinwei_computer | saggi: basically, I agree with your patch | 09:24 |
| yinwei_computer | just with above 2 minor issue | 09:25 |
| yinwei_computer | btw, what's the point of if not already_visited: | 09:25 |
| yinwei_computer | self.context.status_getters.append( | 09:25 |
| yinwei_computer | {"resource_id": resource.id, | 09:25 |
| yinwei_computer | "get_resource_stats": protection_plugin.get_resource_stats} | 09:25 |
| yinwei_computer | ) | 09:25 |
| saggi | yinwei_computer: I don't like the fact that they search for their own task in the stack. This exposes the task stack to the plugin. | 09:27 |
| yinwei_computer | oh, plugin won't do that | 09:27 |
| yinwei_computer | RGW will do it for plugin | 09:27 |
| saggi | That means that it must always reuse the task if it was already visited. | 09:27 |
| yinwei_computer | yes, why not? | 09:28 |
| saggi | If we split volume restore to restoring the data and doing an attack it will reuse the restore task but we will have to attack ops | 09:28 |
| yinwei_computer | plugin should tell the context which time it has been visited and thus produce different tasks? | 09:28 |
| saggi | attach | 09:28 |
| saggi | Or is volume attack responsibility of the VM pluigin? | 09:29 |
| yinwei_computer | oh, attach will be done in server node | 09:29 |
| saggi | Sure | 09:29 |
| yinwei_computer | yes, it's parent node's responsibility | 09:29 |
| yinwei_computer | I've described this in restore design spec | 09:29 |
| yinwei_computer | we have directional graph, only parent knows child, but child has no idea of parent | 09:30 |
| saggi | Sure, than it's a good idea. And if we ever need to change that we can pass it in the context. | 09:30 |
| saggi | and the child can choose to reuse or modify. | 09:30 |
| saggi | but don't do it until we have a use case for it | 09:31 |
| yinwei_computer | ok | 09:31 |
| saggi | yinwei_computer: OK, good job! You are really good at this. | 09:31 |
| saggi | yinwei_computer: Will you update wangliuan and lubin? | 09:32 |
| yinwei_computer | confirm: so basically, RGW will search task for a visited node, and it will still call plugin.on_resource_start. Plugin itself will check the first visited flag, and decide to return task or not. If it returns task, RGW will use the updated one. | 09:33 |
| yinwei_computer | Is this the logic you told in ' And if we ever need to change that we can pass it in the context. '? | 09:33 |
| yinwei_computer | Yes, I will update to them. But I'm afraid I need catch you guys for more issues later after you lunch. | 09:34 |
| saggi | yinwei_computer: I understood that RGW will skip the resource if it was already visited and just make the link. | 09:35 |
| saggi | And if we ever need handling for already visited in the plugin we will add that? | 09:35 |
| saggi | ?=. | 09:35 |
| yinwei_computer | yes | 09:35 |
| yinwei_computer | for now, we can implement in the simple way | 09:36 |
| saggi | yinwei_computer: yes | 09:36 |
| saggi | I got to go | 09:36 |
| yinwei_computer | if we need handle that kind of case, we give context to plugin | 09:36 |
| saggi | yinwei_computer: Thanks for your time. | 09:36 |
| yinwei_computer | sure | 09:36 |
| saggi | Will try and catch you later. | 09:36 |
| yinwei_computer | thanks to you | 09:36 |
| xiangxinyong | welcome zhang shuai and chen zipeng to join us. | 09:39 |
| yinwei_computer | welcome | 09:45 |
| zhang2005 | thanks | 09:46 |
| zhonghua-lee | welcome | 09:47 |
| chenying_ | welcome | 09:49 |
| xiangxinyong | zhangshuai and pengzi, welcome to join our weekly meeting at the #openstack-meeting. tonight 22:00~23:00 | 09:58 |
| *** zhang2005 has quit IRC | 10:19 | |
| openstackgerrit | Merged openstack/smaug: Split Provider configuration into files https://review.openstack.org/298673 | 10:58 |
| *** gampel2 has joined #openstack-smaug | 11:00 | |
| *** gampel1 has quit IRC | 11:02 | |
| chenying_ | ping saggi | 11:38 |
| saggi | chenying_: pong | 11:38 |
| chenying_ | saggi have you seen the operation log comments? | 11:39 |
| chenying_ | Have discussed xinyong I think operation_log not only include scheduled protect operation | 11:40 |
| chenying_ | Have discussed with xinyong It also include protect/delete action without scheduler | 11:41 |
| saggi | chenying_: I didn't find time to get around to it. I was away for a few days because of the army. | 11:44 |
| saggi | chenying_: Will get to it now | 11:44 |
| chenying_ | Ok thanks. | 11:46 |
| *** zhongjun_ has joined #openstack-smaug | 11:49 | |
| *** gampel2 has quit IRC | 12:00 | |
| *** gampel1 has joined #openstack-smaug | 12:01 | |
| *** yinweiishere has joined #openstack-smaug | 12:37 | |
| yinweiishere | ping saggi | 12:38 |
| yinweiishere | ping gampel1 | 12:38 |
| saggi | pong yinweiishere | 12:38 |
| yinweiishere | hi saggi | 12:38 |
| yinweiishere | need confirm two issues with you guys | 12:39 |
| saggi | shoot | 12:39 |
| yinweiishere | first is the granularity of network plugin | 12:39 |
| yinweiishere | shall we make all the networks of one tenant as one resource node in resource graph | 12:39 |
| yinweiishere | or resources of one network per resource graph node? | 12:41 |
| yinweiishere | per my understanding, to solve the share resource among networks, like sg, we only make one resource node in resource graph per project | 12:42 |
| yinweiishere | saggi? | 12:44 |
| saggi | We will have one network resource. | 12:45 |
| saggi | Stuff that are part of the VM will be part of the VM | 12:45 |
| saggi | like it's IP | 12:45 |
| saggi | and what SGs are attached to the port | 12:45 |
| saggi | but the SGs will be in the network resource | 12:45 |
| saggi | similar to how volumes are handled by cinder but attach is handled by nova | 12:46 |
| yinweiishere | last time I aligned it to Eran that we will make ports as resources of network | 12:46 |
| yinweiishere | and attach is in server | 12:46 |
| saggi | so attaching a VM to a network or attaching an SG to it is a Server's responsibility | 12:46 |
| yinweiishere | yes | 12:46 |
| yinweiishere | actually there is one case I'm not sure how to handle it | 12:46 |
| saggi | Yes, ports are part of the network. But ports aren't really an entity that you back up. | 12:47 |
| saggi | You just need to save the links between the VM to a network (which is a port) and info about this link (eg. sg) | 12:47 |
| yinweiishere | say, user pick up several servers as protection plan resources, those servers all attached to networkA. Tenant actually has two networks, networkA and networkB. | 12:48 |
| yinweiishere | with one network graph node, it will protect networkA and networkB or only networkA would be enough? | 12:48 |
| saggi | How do you know A is enough? If there is a router between A and B and VMs in B are created because of actions from the VMs in network A (workers for compute) B might be integral to the project. | 12:50 |
| yinweiishere | I mean, with one network graph node, it wont' construct different resource node with different parent. It should return a graph node which will protect all networks of this project. | 12:50 |
| yinweiishere | so you mean that we won't check the scope of the network resources, but protect whole tenant network resources? | 12:51 |
| yinweiishere | that would be simple | 12:51 |
| saggi | yes | 12:51 |
| saggi | Yes, this is the simplest way to handle it now. Anything else becomes very complex. | 12:51 |
| yinweiishere | hmm, from semantics, we protect more. But more is better than less. | 12:51 |
| saggi | And it has a lot of open questions. We would much rather have concrete use cases for protecting less. | 12:52 |
| yinweiishere | OK, I will have people to comment this into code. And pls. check this limitation with Eran. | 12:52 |
| saggi | Eran knows about it. | 12:52 |
| yinweiishere | we will protect super set of network resources of the plan | 12:52 |
| yinweiishere | ok | 12:52 |
| yinweiishere | that's good | 12:52 |
| yinweiishere | the second issue is about the restoration | 12:53 |
| yinweiishere | thansk for your comments | 12:53 |
| yinweiishere | I replied you in comment | 12:53 |
| yinweiishere | not sure if you have seen them | 12:54 |
| saggi | What we think is to maybe expose options in the network plugin to backup only some aspects (disable backup of SGs) but for now just back up everything we can restore. | 12:55 |
| yinweiishere | hmm, seems to persist whole neutron db in protection plugin | 12:56 |
| yinweiishere | :) | 12:56 |
| yinweiishere | and see what would be useful in restore | 12:56 |
| saggi | yinweiishere: not yet. I will do it after I finish going over the comments on the operation log since I've been neglecting this for days now. | 12:56 |
| saggi | yinweiishere: The problem is managing different versions of neutron since the db format is not part of the api | 12:56 |
| saggi | yinweiishere: And we don't want different versions of neutron to break us. | 12:57 |
| yinweiishere | just kidding, we will list neutron resources by neutron client | 12:57 |
| yinweiishere | but in fact, it equals backup the database | 12:57 |
| saggi | yinweiishere: Great. I gtg for a few minutes now. | 12:58 |
| yinweiishere | ok | 12:58 |
| yinweiishere | one last thing, what about the checkpoint patch? | 12:58 |
| yinweiishere | I have commit the patch enable lease dependent on you patch. Not sure when you will commit it? | 12:59 |
| yinweiishere | and the resource graph serialization patch? If you're busy, chen ying is willing to help on this part. | 13:00 |
| saggi | He can start working on it. I can't seem to manage to get any coding done lately. | 13:05 |
| saggi | yinweiishere: ^^^^ | 13:10 |
| *** yinweiishere has quit IRC | 13:21 | |
| *** chenhuayi has quit IRC | 13:21 | |
| *** saggi has quit IRC | 13:21 | |
| *** zhongjun_ has quit IRC | 13:21 | |
| *** chenzeng has quit IRC | 13:21 | |
| *** asdfghj has quit IRC | 13:21 | |
| *** openstackgerrit has quit IRC | 13:21 | |
| *** jhesketh has quit IRC | 13:21 | |
| *** gampel has quit IRC | 13:21 | |
| *** xiangxinyong has quit IRC | 13:21 | |
| *** smcginnis has quit IRC | 13:21 | |
| *** ChanServ has quit IRC | 13:21 | |
| *** yinwei_computer has quit IRC | 13:21 | |
| *** gampel1 has quit IRC | 13:22 | |
| *** CrayZee has quit IRC | 13:22 | |
| *** yuval has quit IRC | 13:22 | |
| *** chenying_ has quit IRC | 13:22 | |
| *** zhonghua-lee has quit IRC | 13:22 | |
| *** yinweiishere has joined #openstack-smaug | 13:24 | |
| *** chenhuayi_ has joined #openstack-smaug | 13:24 | |
| *** gampel1 has joined #openstack-smaug | 13:24 | |
| *** zhongjun_ has joined #openstack-smaug | 13:24 | |
| *** asdfghj has joined #openstack-smaug | 13:24 | |
| *** chenzeng has joined #openstack-smaug | 13:24 | |
| *** yuval has joined #openstack-smaug | 13:24 | |
| *** gampel has joined #openstack-smaug | 13:24 | |
| *** saggi has joined #openstack-smaug | 13:24 | |
| *** yinwei_computer has joined #openstack-smaug | 13:24 | |
| *** CrayZee has joined #openstack-smaug | 13:24 | |
| *** openstackgerrit has joined #openstack-smaug | 13:24 | |
| *** xiangxinyong has joined #openstack-smaug | 13:24 | |
| *** chenying_ has joined #openstack-smaug | 13:24 | |
| *** zhonghua-lee has joined #openstack-smaug | 13:24 | |
| *** smcginnis has joined #openstack-smaug | 13:24 | |
| *** jhesketh has joined #openstack-smaug | 13:24 | |
| *** ChanServ has joined #openstack-smaug | 13:24 | |
| *** wolfe.freenode.net sets mode: +o ChanServ | 13:24 | |
| *** xiangxinyong456 has joined #openstack-smaug | 13:50 | |
| *** AndroUser has joined #openstack-smaug | 13:51 | |
| *** AndroUser is now known as zhangshuai | 13:55 | |
| *** Zhongjun has joined #openstack-smaug | 14:00 | |
| gampel1 | Saggi | 14:02 |
| *** xiangxinyong456 has quit IRC | 14:07 | |
| *** xiangxinyong456 has joined #openstack-smaug | 14:08 | |
| *** yinweimac has joined #openstack-smaug | 14:12 | |
| *** zhangshuai has quit IRC | 14:53 | |
| *** zhangshuai has joined #openstack-smaug | 14:54 | |
| yinweimac | gampel and saggi, since we have a demo schedule on the end of April, is it possible to postpone some enhancement patches, like task stack change and image chunking change to May? I mean, we mark those items done, but deliver a functional correct one first? | 14:58 |
| yinweimac | mark them down | 15:01 |
| saggi | task stack is simple. Image chunking could be postponed. | 15:01 |
| gampel | I have no problem but I think you will not be able to do a backup to swift without chunking | 15:01 |
| saggi | It's not even merged yet | 15:01 |
| *** zhang2005 has joined #openstack-smaug | 15:01 | |
| saggi | task stak ^^ | 15:01 |
| gampel | we could reuse the code in dragon for the chunking | 15:01 |
| yinweimac | not exactly | 15:01 |
| gampel | You think it will work with out the chunking | 15:02 |
| yinweimac | when you download it, you need organize it as well | 15:02 |
| yinweimac | the ideal solution is to use the manufest of swift object | 15:02 |
| gampel | Yes so you will do a demo of a very small image and volume ? | 15:02 |
| yinweimac | yes | 15:02 |
| yinweimac | I think we've got ideal solution but we need take time to optimize it | 15:03 |
| saggi | xiangxinyong: Will you be online tomorrow at 8:00 Israel time? (I think it's 13:00 in China) | 15:03 |
| zhonghua-lee | yinweimac:we should work together and speed it up | 15:03 |
| zhonghua-lee | saggi: what's the matter? | 15:03 |
| xiangxinyong | saggi: ok | 15:03 |
| yinweimac | that's no problem | 15:03 |
| saggi | xiangxinyong: So go to sleep. I'll catch you up tomorrow about the parameters. | 15:04 |
| yinweimac | zhonghua-lee: sure, we can work together | 15:04 |
| saggi | :) | 15:04 |
| yinweimac | but we need a workable one first | 15:04 |
| xiangxinyong | saggi: i will wait for you. saggi. | 15:04 |
| gampel | I have no problem merging it if we file a bug that we should add chunking | 15:04 |
| xiangxinyong | saggi: thanks | 15:04 |
| zhonghua-lee | yeah, understood | 15:04 |
| yinweimac | not only chunking, but also manufest | 15:04 |
| yinweimac | yes, thanks | 15:04 |
| gampel | and maybe start working on it in parallel | 15:04 |
| zhonghua-lee | 13:00 is our rest time :) | 15:05 |
| yinweimac | yes, if there's enough hands | 15:05 |
| xiangxinyong | it is no matter | 15:05 |
| xiangxinyong | :) | 15:05 |
| zhonghua-lee | okay | 15:05 |
| yinweimac | since I'd like have Luobin to work on restore ASAP | 15:05 |
| gampel | Ok thanks everyone i know it is very late in china | 15:05 |
| xiangxinyong | ^-^ | 15:06 |
| yinweimac | last item to check | 15:06 |
| gampel | sure | 15:06 |
| yinweimac | saggi, how about your checkpoint patch? | 15:06 |
| yinweimac | will you merge it? | 15:06 |
| yinweimac | and what's your plan to serialize resource graph? | 15:06 |
| yinweimac | restore will depend on the resource graph serialization | 15:07 |
| gampel | lthis one https://review.openstack.org/#/c/280325/ | 15:07 |
| yinweimac | two items | 15:07 |
| yinweimac | first is to merge this one (there's some bugs as jenkins failed, note my comment) | 15:08 |
| yinweimac | second is another patch, not sure if saggi is working on it | 15:08 |
| gampel | which one ? | 15:08 |
| gampel | sagii: are you here ? | 15:09 |
| saggi | yinweimac: I can't find the time to get to it. I don't need a lot of time just some time without meetings. I will try and get to it on Monday (I have reserve duty on Sunday). The plan is to just generate the tree from the plan and walk it building a json file than persist it in a file near the checkpoint index. | 15:09 |
| yinweimac | resource graph serialize | 15:09 |
| gampel | saggi: maybe yuval or yinwei can take this patch | 15:09 |
| yinweimac | yes | 15:09 |
| yinweimac | just mean if you need ask for help | 15:10 |
| gampel | yinwei can you share the other patch link ? | 15:10 |
| zhonghua-lee | saggi: is there anything that we can help you? | 15:10 |
| yinweimac | the resource graph serialize | 15:10 |
| yinweimac | the other patch hasn't commit yet | 15:10 |
| yinweimac | just in saggi's mind :) | 15:10 |
| saggi | :) | 15:10 |
| yinweimac | chenying told me that he would like to help on restore and could start on this patch | 15:11 |
| saggi | sure | 15:11 |
| yinweimac | if you're busy this week, I will have chenying to work on this patch | 15:11 |
| yinweimac | thanks, zhonghua! | 15:11 |
| yinweimac | you helped a lot | 15:11 |
| zhonghua-lee | yinweimac: welcome | 15:12 |
| yinweimac | saggi: just in case we work on the repeated items | 15:12 |
| zhonghua-lee | yinweimac: i know you have no much time to waste | 15:12 |
| gampel | saggi: what about the resource graph serialize | 15:12 |
| yinweimac | let's see if chenying have time to commit it this week | 15:12 |
| saggi | If someone starts doing it before Monday just send me an email. | 15:12 |
| zhonghua-lee | yinweimac: let's speed it up, hopes can catch your point | 15:13 |
| yinweimac | if not, sagg will take it in net monday | 15:13 |
| saggi | I will not have time to work on it until then anyway | 15:13 |
| yinweimac | good | 15:13 |
| gampel | so chenying will take this two patches ? | 15:13 |
| yinweimac | no | 15:13 |
| zhonghua-lee | gampel: let me check tommorrow. | 15:13 |
| yinweimac | the first one I have tested it and found bugs already | 15:13 |
| chenying_ | After saggi's operation log patch I may need work on this api. | 15:13 |
| yuval | I can start working on the resource graph serialization | 15:14 |
| saggi | I think chenying is doing way to much as it is | 15:14 |
| yinweimac | thanks, yuval | 15:14 |
| gampel | saggi: please explain yuval your idea about the resource graph serialization today | 15:14 |
| yinweimac | thanks, everyone | 15:15 |
| gampel | thanks everyone good night | 15:15 |
| yuval | all: if you have time, there are some minor patches we can merge quickly | 15:15 |
| yinweimac | good night | 15:15 |
| zhonghua-lee | bye,everyone | 15:15 |
| xiangxinyong | 88 | 15:15 |
| chenying_ | bye | 15:16 |
| gampel | bye | 15:16 |
| *** yinweimac has quit IRC | 15:16 | |
| *** zhang2005 has quit IRC | 15:16 | |
| *** zhangshuai has quit IRC | 15:18 | |
| *** asdfghj has quit IRC | 15:24 | |
| *** asdfghj has joined #openstack-smaug | 15:25 | |
| *** yuval has quit IRC | 15:34 | |
| *** Zhongjun has quit IRC | 16:49 | |
| *** chenhuayi_ has quit IRC | 17:35 | |
| *** gampel1 has quit IRC | 18:53 | |
| *** xiangxinyong456 has quit IRC | 22:40 | |
| *** xiangxinyong456 has joined #openstack-smaug | 22:43 | |
| *** xiangxinyong456 has quit IRC | 23:00 | |
| *** xiangxinyong456 has joined #openstack-smaug | 23:03 | |
| *** xiangxinyong456 has quit IRC | 23:14 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!