*** EinstCrazy has quit IRC | 00:25 | |
*** dschroeder has quit IRC | 00:53 | |
*** yangyapeng has joined #openstack-freezer | 01:00 | |
*** yangyapeng has quit IRC | 01:03 | |
*** EinstCrazy has joined #openstack-freezer | 01:33 | |
*** EinstCrazy has quit IRC | 01:58 | |
*** EinstCrazy has joined #openstack-freezer | 02:10 | |
*** EinstCrazy has quit IRC | 02:30 | |
*** EinstCrazy has joined #openstack-freezer | 02:31 | |
*** SamYaple has joined #openstack-freezer | 02:46 | |
SamYaple | daemontool: are you around? | 02:46 |
---|---|---|
SamYaple | Anyone from the freezer team around? | 02:48 |
*** yangyapeng has joined #openstack-freezer | 02:49 | |
SamYaple | Im not sure what timezone you all normally operate in. I have started a new backup project for block-based backup (Ekko) and would like to have a conversation about potential integration. I am always on IRC so ping me anytime | 02:54 |
*** daemontool has quit IRC | 02:56 | |
*** EinstCrazy has quit IRC | 02:59 | |
*** EinstCrazy has joined #openstack-freezer | 03:00 | |
*** EinstCrazy has quit IRC | 04:01 | |
*** yangyapeng has quit IRC | 05:31 | |
*** yangyape_ has joined #openstack-freezer | 05:31 | |
*** EinstCrazy has joined #openstack-freezer | 05:41 | |
*** EinstCrazy has quit IRC | 06:04 | |
*** EinstCrazy has joined #openstack-freezer | 06:26 | |
*** EinstCrazy has quit IRC | 06:40 | |
*** EinstCrazy has joined #openstack-freezer | 06:41 | |
*** EinstCrazy has quit IRC | 07:26 | |
*** EinstCrazy has joined #openstack-freezer | 08:00 | |
*** EinstCra_ has joined #openstack-freezer | 08:13 | |
*** EinstCrazy has quit IRC | 08:14 | |
*** c00281451 has joined #openstack-freezer | 08:56 | |
*** EinstCra_ has quit IRC | 09:48 | |
*** yangyapeng has joined #openstack-freezer | 09:53 | |
*** yangyapeng has quit IRC | 09:53 | |
*** yangyape_ has quit IRC | 09:57 | |
dmellado | SamYaple: the folks are emea-based | 10:28 |
*** EinstCrazy has joined #openstack-freezer | 10:38 | |
openstackgerrit | Merged openstack/freezer: Fix a freezer-agent bug for when restoring data from Swift https://review.openstack.org/271701 | 11:04 |
*** emildi has joined #openstack-freezer | 11:28 | |
*** vannif has joined #openstack-freezer | 11:59 | |
vannif | hi | 11:59 |
*** emildi has quit IRC | 12:10 | |
*** daemontool has joined #openstack-freezer | 12:14 | |
*** yangyapeng has joined #openstack-freezer | 12:42 | |
*** emildi has joined #openstack-freezer | 12:54 | |
*** daemontool has quit IRC | 12:55 | |
SamYaple | dmellado: cool. so am I. thanks for the info | 13:32 |
*** yangyapeng has quit IRC | 13:32 | |
*** emildi has quit IRC | 13:40 | |
*** emildi has joined #openstack-freezer | 14:08 | |
*** daemontool has joined #openstack-freezer | 14:24 | |
*** daemontool has quit IRC | 14:43 | |
*** daemontool has joined #openstack-freezer | 14:47 | |
*** emildi has quit IRC | 14:52 | |
*** dschroeder has joined #openstack-freezer | 16:07 | |
*** EinstCra_ has joined #openstack-freezer | 17:04 | |
*** szaher has joined #openstack-freezer | 17:05 | |
*** EinstCrazy has quit IRC | 17:07 | |
*** openstackgerrit has quit IRC | 17:17 | |
*** openstackgerrit has joined #openstack-freezer | 17:18 | |
*** emildi has joined #openstack-freezer | 18:29 | |
*** EinstCra_ has quit IRC | 18:51 | |
*** daemontool_ has joined #openstack-freezer | 19:40 | |
*** daemontool has quit IRC | 19:43 | |
*** emildi has quit IRC | 19:48 | |
SamYaple | Anyone around for a basic project rundown? | 20:35 |
daemontool_ | SamYaple, Hi | 20:49 |
SamYaple | hey daemontool_! | 20:50 |
daemontool_ | how are you doing | 20:50 |
daemontool_ | I'm Fausto | 20:50 |
daemontool_ | nice to meet you | 20:50 |
SamYaple | Yes! | 20:50 |
SamYaple | you as well | 20:50 |
SamYaple | IRC is my tool of choice for communication | 20:50 |
SamYaple | im not big on the emails :) | 20:50 |
daemontool_ | ++ | 20:51 |
SamYaple | If you had a minute I had a few questions for oyu about Freezer | 20:51 |
daemontool_ | yes | 20:51 |
SamYaple | Cool. | 20:51 |
daemontool_ | I'm here | 20:51 |
SamYaple | So in areas of overlap I can think of two immediatly, the API and scheduler | 20:51 |
SamYaple | the API is pretty clear, honestly openstack could have one big api so thats some thing that could be joined together no problem | 20:52 |
SamYaple | the scheduler though, do you have a scheduler service, or is it the agent in each vm that is the scheduler? | 20:52 |
daemontool_ | there's the scheduler and the agent | 20:52 |
daemontool_ | that runs on each node where the data you want to backup is | 20:52 |
daemontool_ | (both( | 20:53 |
daemontool_ | ) | 20:53 |
SamYaple | but do these components run inside the instance itself? | 20:53 |
daemontool_ | yes | 20:53 |
daemontool_ | they run where the data you want to backup resides | 20:54 |
daemontool_ | is the scheduler that talks to the API | 20:54 |
SamYaple | I think that may be our biggest blocker though. Ekko has no agents inside the instance/VM | 20:54 |
daemontool_ | it does not matter | 20:54 |
daemontool_ | where if it is a vm | 20:54 |
daemontool_ | or instance | 20:54 |
SamYaple | can you expand on what you mean? | 20:55 |
daemontool_ | does Ekko run where the data you want to backup is? | 20:55 |
SamYaple | the strucutre for Ekko is and api, a scheduler running on your "control" nodes (glorified cronjobs), and a backup-agent running on each compute | 20:56 |
daemontool_ | so the scheduler is behind the api, right? | 20:56 |
SamYaple | the data being backed up is the instance disks, but Ekko has no agent running in the instance | 20:56 |
SamYaple | if I understand your question right, yes | 20:56 |
daemontool_ | ok | 20:57 |
daemontool_ | what is the purpose of the scheduler? | 20:57 |
daemontool_ | do you have an example workflow? | 20:57 |
SamYaple | scheduling backups. for example at a certain time of day or when certain critera is met (like amount of data changed threshold) | 20:58 |
SamYaple | I do not have anything in the Ekko repo yet | 20:59 |
daemontool_ | ok np | 20:59 |
SamYaple | for workflow that is | 20:59 |
*** daemontool has joined #openstack-freezer | 21:00 | |
daemontool | SamYaple, sorry I got disconnected | 21:01 |
daemontool | why in the solution you placed the scheduler behind the api? | 21:01 |
SamYaple | where else would it run? | 21:02 |
daemontool | we place it at client side | 21:03 |
daemontool | distribute any kind of load helps you to scale more | 21:03 |
daemontool | but the question is, why you put it there :) | 21:03 |
*** daemontool_ has quit IRC | 21:03 | |
daemontool | if there's any special reason | 21:03 |
daemontool | so what are your concerns on converging? | 21:04 |
SamYaple | It has to run somewhere, but client-side tools are not something we want to do | 21:04 |
daemontool | ok | 21:04 |
SamYaple | hypervisors already provide these tools and we just reuse those | 21:04 |
daemontool | I think the solution for the problem you try to solve is good | 21:04 |
daemontool | but I do not understand what's the issue on converging... the real issue | 21:05 |
daemontool | I honestly see these as details, that can be solved, if we have the will | 21:05 |
SamYaple | Im not sure what our projects have in common, other than the name 'backup'. | 21:06 |
SamYaple | If we can reuse compontents thats one thing, but other than API, i dont see what we can reuse | 21:06 |
SamYaple | additionally, to make this work properly with object storage we will require a caching layer to track data | 21:06 |
SamYaple | we have plans to use redis for this and this would be overhead that you would then have in freezer for no benift | 21:07 |
*** daemontool_ has joined #openstack-freezer | 21:07 | |
daemontool_ | if you want to keep the name for the space of solution that Ekko provides, that's not a problem at all, but we need a unified backup, restore and disaster recovery solution | 21:08 |
SamYaple | i think our disagreement is on whether file backup is "unified backup" or not | 21:09 |
SamYaple | block and file backup have nothing in common, those who use block backup typically dont use file backup | 21:09 |
daemontool_ | file, object, volumes, instance | 21:09 |
daemontool_ | are all resources that users wants to backup | 21:10 |
daemontool_ | and an environment like a company | 21:10 |
SamYaple | backup objects? | 21:10 |
*** daemontool has quit IRC | 21:10 | |
daemontool_ | my wants to have | 21:10 |
daemontool_ | flexibility from a single solutino | 21:10 |
daemontool_ | to cover that issues | 21:10 |
daemontool_ | so if we have a modular approach | 21:10 |
daemontool_ | we can offer a really advanced tool | 21:10 |
SamYaple | The way I see it is if we can share a common api we will not be stepping on each others toes past that | 21:11 |
SamYaple | we can easily co-exist without stepping on each others toes, but i see little benifit in convergance for either of us | 21:11 |
SamYaple | im not sure how sharing an api is possible with the current state of openstack though | 21:11 |
daemontool_ | sharing a common api is not converging? | 21:11 |
daemontool_ | offering a common interface | 21:12 |
daemontool_ | a common suite of tools | 21:12 |
daemontool_ | and web interface | 21:12 |
daemontool_ | wouldn't be of a great benefit to users and community? | 21:12 |
SamYaple | ok, but how would that look from a backend or operator point of view? | 21:12 |
daemontool_ | we would have a plugin called Ekko in Freezer | 21:13 |
SamYaple | would that mean still requiring an agent inside of each instance? | 21:13 |
daemontool_ | that solve the problem you want to solve | 21:13 |
daemontool_ | we do not have to create manadatory dependencies | 21:13 |
SamYaple | does Freezer currently have a plugin architecture like this? | 21:14 |
daemontool_ | yes | 21:15 |
daemontool_ | I mean | 21:15 |
daemontool_ | the API would server what you want to serve from it | 21:15 |
daemontool_ | and the scheduler execute anything you want | 21:15 |
daemontool_ | we can add tab/panels to horizon | 21:15 |
daemontool_ | so show what Ekko wants to show | 21:15 |
daemontool_ | just FYI | 21:16 |
daemontool_ | Cinder and Nova teams in general | 21:16 |
daemontool_ | are very reluctant to install additional components on compute nodes | 21:16 |
daemontool_ | and storage nodes | 21:16 |
daemontool_ | that's my experience from the field | 21:16 |
SamYaple | they can be reluctant but they can't block anything since integration is not required with those components for ekko | 21:16 |
daemontool_ | It's not about blocking | 21:17 |
SamYaple | integrating with nova and cinder would be useful, but again not required | 21:17 |
daemontool_ | I don't think that's the point | 21:17 |
daemontool_ | if you do things in openstack | 21:17 |
daemontool_ | you need to integrate | 21:17 |
daemontool_ | you are not forced to | 21:17 |
daemontool_ | but you should integrate | 21:17 |
SamYaple | Right but there is very little integrating that would be useful here | 21:18 |
SamYaple | but considering i have to interact with the hypervisors directly, there is a requirement for compute node agent | 21:18 |
daemontool_ | ok | 21:18 |
daemontool_ | I'm pretty sure if have think about it we'll find many ways to integrate :) | 21:19 |
daemontool_ | what would be the requirement? | 21:19 |
SamYaple | so we have Freezer which has common things like API and Horizon plugin that can be expanded with block backup and file backup (file backup being what you do currently)? | 21:19 |
SamYaple | is this what you are purposing? | 21:19 |
daemontool_ | file, vm and volumes, using the related services api | 21:20 |
daemontool_ | there's a plan | 21:20 |
daemontool_ | to do very foon | 21:20 |
daemontool_ | soon | 21:20 |
SamYaple | what is the relative api for vm or volumes? | 21:20 |
daemontool_ | it uses the api from cinder and nova | 21:20 |
SamYaple | right but neither of those can do backups | 21:20 |
daemontool_ | you have to set that config in a freezer job | 21:20 |
SamYaple | they can only do snapshots | 21:20 |
daemontool_ | ok | 21:20 |
daemontool_ | how the Ekko backup-agent | 21:21 |
daemontool_ | execute a backup? | 21:21 |
daemontool_ | on a cinder node? | 21:21 |
daemontool_ | and on a nova node? | 21:21 |
daemontool_ | s/and/or/ | 21:22 |
daemontool_ | how the block based back up works | 21:22 |
SamYaple | cinder is not planned for 1.0 release. so lets talk about nova for a second | 21:22 |
daemontool_ | ok, : ) | 21:22 |
SamYaple | ekko-backup-agent on compute node that can speak to hypervisor (lets only talk about QEMU) directly | 21:23 |
SamYaple | ekko tells QEMU backup is about to begin via QMP | 21:23 |
SamYaple | QMP tells instance and the QEMU guest agent which preps the machine for backup (vss/sync disk/flush mysql/etc) | 21:23 |
SamYaple | QMP tells ekko instance is ready | 21:23 |
SamYaple | ekko tells QEMU to being backup using a qemu feature that will redirect writes to another location | 21:24 |
SamYaple | instance proceeds on as normal in the background at this point | 21:24 |
daemontool_ | is the qemu agent installed a requirement? | 21:24 |
SamYaple | yes | 21:24 |
SamYaple | well no | 21:24 |
SamYaple | no technically | 21:24 |
SamYaple | to get a transactionally consistent backup it is | 21:25 |
SamYaple | otherwise youll just get crash consistent | 21:25 |
SamYaple | at this point in the process the orignal volume is sitting idle while all new writes go to a new location | 21:25 |
SamYaple | i can perform the backup on the idle volume then | 21:25 |
daemontool_ | isn't that a snapshot? | 21:25 |
SamYaple | nope | 21:25 |
SamYaple | and snapshot is disruptive to instance (pauses instance) this does not | 21:26 |
daemontool_ | lvm snapshot is not | 21:26 |
SamYaple | snapshots are not backups | 21:26 |
daemontool_ | is crash consistent | 21:26 |
daemontool_ | does not require downtime | 21:26 |
SamYaple | correct, but thats not for nova thats cinder | 21:26 |
daemontool_ | or any agent | 21:26 |
daemontool_ | and it works on any kind of hypervisor | 21:26 |
daemontool_ | if you have /var/lib/nova on an lvm volume | 21:27 |
SamYaple | crash consistent is not a reiliable backuip | 21:27 |
daemontool_ | what would be the difference? | 21:27 |
daemontool_ | ok | 21:27 |
daemontool_ | I'm just comparing | 21:27 |
daemontool_ | apple with apple :) | 21:27 |
SamYaple | but you are saying "use this feature from a tool", not everyone uses lvm to back /var/lib/nova | 21:27 |
SamYaple | what about those using nfs and ceph? | 21:27 |
SamYaple | ekko doesnt care | 21:28 |
daemontool_ | ekko does not care | 21:28 |
daemontool_ | as long as you have | 21:28 |
daemontool_ | qemu | 21:28 |
daemontool_ | and the qemu agent | 21:28 |
SamYaple | no, ekko doesnt care either way | 21:28 |
SamYaple | if you want a transactionally consistent backup you need the qemu agent | 21:28 |
daemontool_ | if you cannot afford downtime, yes | 21:29 |
SamYaple | i dont follow? | 21:29 |
daemontool_ | so Ekko provide a solution to the lack of incremental backup for nova vm instance | 21:29 |
daemontool_ | is that right? | 21:29 |
SamYaple | lack of backup for nova vm at all, yes | 21:30 |
daemontool_ | nova supports vm backups | 21:30 |
daemontool_ | there's an api for that also | 21:30 |
daemontool_ | but they are not incremental | 21:30 |
SamYaple | thats a snapshot, not a backup | 21:31 |
SamYaple | snapshots are not backups | 21:31 |
daemontool_ | the snapshot is used | 21:31 |
daemontool_ | to have a point in time | 21:31 |
daemontool_ | state of the vm | 21:31 |
daemontool_ | I agree is not a backup | 21:31 |
daemontool_ | but it can be leverage | 21:31 |
daemontool_ | d | 21:31 |
SamYaple | well you did say "nova supports vm backups" | 21:31 |
daemontool_ | I'm not saying your solution isn't good, actually it is | 21:31 |
daemontool_ | but it solves a very specific problem | 21:31 |
daemontool_ | and I do not see it a blocker | 21:31 |
daemontool_ | to not converge | 21:32 |
daemontool_ | that's basically my point | 21:32 |
*** EinstCrazy has joined #openstack-freezer | 21:32 | |
daemontool_ | and also live snapshot features are available | 21:32 |
SamYaple | Since neither of our projects conflict, I don't see a block to convergance either. For that matter cinder and ceilometer can merge | 21:32 |
daemontool_ | not for backup.. that's clear | 21:32 |
daemontool_ | cinder and ceilometer is their problem if they wants to converge | 21:33 |
daemontool_ | not ours | 21:33 |
SamYaple | So lets talk a bit more about what that would look like | 21:33 |
SamYaple | Ekko and Freezer merging that is | 21:33 |
daemontool_ | so lets do this | 21:33 |
daemontool_ | provide some more information on Ekko | 21:33 |
daemontool_ | that would be helpful anyway for you and the users too | 21:34 |
daemontool_ | and let's think and be open on the fact of converging | 21:34 |
SamYaple | certainly. I have no desire to maintain an API or Horizon plugins | 21:34 |
daemontool_ | then tell me please what really are your conditions like independent repo, implemented as plugin excention etc | 21:34 |
daemontool_ | then let's be open, and believe me, we'll make the world most advanced backup restore and dr solution | 21:35 |
daemontool_ | open source | 21:35 |
SamYaple | as I said on the ML, i don't care so much about independance (I like community!) but I do care about progressing | 21:35 |
SamYaple | so my conditions are fairly simple, progress | 21:35 |
SamYaple | we just need to discuss what a merge would look like | 21:36 |
daemontool_ | ok :) | 21:36 |
daemontool_ | look, no one is going to force | 21:36 |
daemontool_ | to do something you do not want to do | 21:36 |
daemontool_ | but we need to be reasonable | 21:36 |
daemontool_ | and offer the best to the users | 21:36 |
daemontool_ | I'm sure that's your intention | 21:36 |
daemontool_ | too | 21:36 |
SamYaple | indeed it was | 21:36 |
daemontool_ | then let's do it : ) | 21:36 |
daemontool_ | so | 21:37 |
daemontool_ | can you please provide a roadmap, a diagram and some more info? | 21:37 |
daemontool_ | if not it's ok | 21:37 |
daemontool_ | we can move forward without it | 21:37 |
SamYaple | I can provide more info with pointed questions | 21:38 |
SamYaple | I have no issue describing the whole process in detail, but "more info" is a bit broad for me (i dont do well with broad questions) | 21:38 |
daemontool_ | don't you think you should provide more information to who wants to use your tool? | 21:38 |
daemontool_ | did you notice | 21:38 |
SamYaple | oh sure but the project is 3 weeks old | 21:38 |
daemontool_ | that the link on documentation on the Ekko repo return 404 not found? | 21:38 |
daemontool_ | that's an issue | 21:38 |
daemontool_ | the approach is important | 21:38 |
SamYaple | yup docs are publishing yet | 21:38 |
SamYaple | arent* | 21:39 |
daemontool_ | I'm not trying to load you with burocratic things | 21:39 |
daemontool_ | I'm just trying to provide some help to solve problem that we already solved in the past | 21:39 |
daemontool_ | because if you do not provide a bit of a roadmap, features and more info | 21:40 |
daemontool_ | people will not use your tool | 21:40 |
SamYaple | All of the people currently working on Ekko will be at the Kolla mid-cycle I am hosting in Feb, we will be doing a mini-ekko midcycle then | 21:40 |
SamYaple | currently we have little useable code in the repo | 21:40 |
daemontool_ | I'm happy for you | 21:40 |
daemontool_ | :) | 21:41 |
SamYaple | well have more info coming out of that discussion | 21:41 |
daemontool_ | I'm glad you have leadership skills | 21:41 |
daemontool_ | but you still need to provide some design, roadmap | 21:41 |
daemontool_ | and useful information on how your tool work | 21:41 |
SamYaple | well we need to agree on those things first :) | 21:41 |
daemontool_ | which things? | 21:42 |
SamYaple | design and roadmap | 21:42 |
SamYaple | we will be doing this in Feb | 21:42 |
daemontool_ | ok | 21:42 |
daemontool_ | so let's do this, if it make sense to you | 21:43 |
daemontool_ | let's think 1 week | 21:43 |
daemontool_ | in an open way | 21:43 |
daemontool_ | on what can be integrated | 21:43 |
daemontool_ | and how that can benefit as much as possible users | 21:44 |
daemontool_ | let's try to approach this with that mindset if possible | 21:44 |
SamYaple | Of course, as stated sharing API and even Horizon plugins is desired | 21:45 |
SamYaple | I think we will need clear seperation of file and block backup though | 21:45 |
SamYaple | And I am not sure how to achieve that | 21:45 |
daemontool_ | ok let's think about it | 21:45 |
daemontool_ | and we'll find a solution | 21:46 |
daemontool_ | now | 21:46 |
daemontool_ | my advice, | 21:46 |
daemontool_ | would be to send an email to the ml, to the thread | 21:46 |
daemontool_ | showing openness, and willing to find a solution together | 21:46 |
daemontool_ | do you agree with that? | 21:46 |
SamYaple | That sounds reasonable | 21:47 |
daemontool_ | brilliant | 21:47 |
daemontool_ | thank you | 21:47 |
daemontool_ | next week we'll talk, ok? :) | 21:47 |
SamYaple | From your point of view are the only things we can share that we would over lap on the API and Horizon plugins? | 21:47 |
SamYaple | Or is there more code i need to dig into for freezer? | 21:47 |
daemontool_ | I think that components can be leveraged | 21:48 |
daemontool_ | at least | 21:48 |
daemontool_ | I agree | 21:48 |
daemontool_ | but, let's think about it | 21:48 |
daemontool_ | more thoroughly | 21:49 |
SamYaple | Ok let me dig into that freezer code first | 21:49 |
SamYaple | then Ill seen a ML email asking about this | 21:49 |
daemontool_ | for now | 21:49 |
daemontool_ | I think | 21:49 |
daemontool_ | my advice is for you to focus | 21:49 |
daemontool_ | on Ekko code | 21:49 |
daemontool_ | and make that incredible | 21:49 |
daemontool_ | because as any new os project you have limited resources right? | 21:50 |
daemontool_ | so focus on that | 21:50 |
daemontool_ | what is important is our mindset I think | 21:50 |
daemontool_ | so let's think about it and talk next week | 21:50 |
daemontool_ | is that ok for you? | 21:50 |
daemontool_ | the mail to the ml is important for you to show openness | 21:50 |
daemontool_ | that's my advice | 21:51 |
daemontool_ | then do as you want | 21:51 |
SamYaple | Sure that sounds good | 21:52 |
daemontool_ | are you ok with our conversation? | 21:52 |
daemontool_ | there's anything you wanted to you but you didn't? | 21:52 |
daemontool_ | s/to you/to say/ | 21:53 |
SamYaple | Not at all. I think it's been fairly clear from both sides. We don't want to duplicate work nor do we want to slow the others progress down | 21:53 |
SamYaple | We will find a balance between the two | 21:53 |
daemontool_ | brilliant | 21:53 |
daemontool_ | nto next week then :) | 21:54 |
SamYaple | :) | 21:54 |
*** daemontool has joined #openstack-freezer | 22:23 | |
*** daemontool_ has quit IRC | 22:27 | |
*** daemontool_ has joined #openstack-freezer | 22:39 | |
*** daemontool has quit IRC | 22:42 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!