Tuesday, 2016-01-26

*** EinstCrazy has quit IRC00:25
*** dschroeder has quit IRC00:53
*** yangyapeng has joined #openstack-freezer01:00
*** yangyapeng has quit IRC01:03
*** EinstCrazy has joined #openstack-freezer01:33
*** EinstCrazy has quit IRC01:58
*** EinstCrazy has joined #openstack-freezer02:10
*** EinstCrazy has quit IRC02:30
*** EinstCrazy has joined #openstack-freezer02:31
*** SamYaple has joined #openstack-freezer02:46
SamYapledaemontool: are you around?02:46
SamYapleAnyone from the freezer team around?02:48
*** yangyapeng has joined #openstack-freezer02:49
SamYapleIm not sure what timezone you all normally operate in. I have started a new backup project for block-based backup (Ekko) and would like to have a conversation about potential integration. I am always on IRC so ping me anytime02:54
*** daemontool has quit IRC02:56
*** EinstCrazy has quit IRC02:59
*** EinstCrazy has joined #openstack-freezer03:00
*** EinstCrazy has quit IRC04:01
*** yangyapeng has quit IRC05:31
*** yangyape_ has joined #openstack-freezer05:31
*** EinstCrazy has joined #openstack-freezer05:41
*** EinstCrazy has quit IRC06:04
*** EinstCrazy has joined #openstack-freezer06:26
*** EinstCrazy has quit IRC06:40
*** EinstCrazy has joined #openstack-freezer06:41
*** EinstCrazy has quit IRC07:26
*** EinstCrazy has joined #openstack-freezer08:00
*** EinstCra_ has joined #openstack-freezer08:13
*** EinstCrazy has quit IRC08:14
*** c00281451 has joined #openstack-freezer08:56
*** EinstCra_ has quit IRC09:48
*** yangyapeng has joined #openstack-freezer09:53
*** yangyapeng has quit IRC09:53
*** yangyape_ has quit IRC09:57
dmelladoSamYaple: the folks are emea-based10:28
*** EinstCrazy has joined #openstack-freezer10:38
openstackgerritMerged openstack/freezer: Fix a freezer-agent bug for when restoring data from Swift  https://review.openstack.org/27170111:04
*** emildi has joined #openstack-freezer11:28
*** vannif has joined #openstack-freezer11:59
vannifhi11:59
*** emildi has quit IRC12:10
*** daemontool has joined #openstack-freezer12:14
*** yangyapeng has joined #openstack-freezer12:42
*** emildi has joined #openstack-freezer12:54
*** daemontool has quit IRC12:55
SamYapledmellado: cool. so am I. thanks for the info13:32
*** yangyapeng has quit IRC13:32
*** emildi has quit IRC13:40
*** emildi has joined #openstack-freezer14:08
*** daemontool has joined #openstack-freezer14:24
*** daemontool has quit IRC14:43
*** daemontool has joined #openstack-freezer14:47
*** emildi has quit IRC14:52
*** dschroeder has joined #openstack-freezer16:07
*** EinstCra_ has joined #openstack-freezer17:04
*** szaher has joined #openstack-freezer17:05
*** EinstCrazy has quit IRC17:07
*** openstackgerrit has quit IRC17:17
*** openstackgerrit has joined #openstack-freezer17:18
*** emildi has joined #openstack-freezer18:29
*** EinstCra_ has quit IRC18:51
*** daemontool_ has joined #openstack-freezer19:40
*** daemontool has quit IRC19:43
*** emildi has quit IRC19:48
SamYapleAnyone around for a basic project rundown?20:35
daemontool_SamYaple,  Hi20:49
SamYaplehey daemontool_!20:50
daemontool_how are you doing20:50
daemontool_I'm Fausto20:50
daemontool_nice to meet you20:50
SamYapleYes!20:50
SamYapleyou as well20:50
SamYapleIRC is my tool of choice for communication20:50
SamYapleim not big on the emails :)20:50
daemontool_++20:51
SamYapleIf you had a minute I had a few questions for oyu about Freezer20:51
daemontool_yes20:51
SamYapleCool.20:51
daemontool_I'm here20:51
SamYapleSo in areas of overlap I can think of two immediatly, the API and scheduler20:51
SamYaplethe API is pretty clear, honestly openstack could have one big api so thats some thing that could be joined together no problem20:52
SamYaplethe scheduler though, do you have a scheduler service, or is it the agent in each vm that is the scheduler?20:52
daemontool_there's the scheduler and the agent20:52
daemontool_that runs on each node where the data you want to backup is20:52
daemontool_(both(20:53
daemontool_)20:53
SamYaplebut do these components run inside the instance itself?20:53
daemontool_yes20:53
daemontool_they run where the data you want to backup resides20:54
daemontool_is the scheduler that talks to the API20:54
SamYapleI think that may be our biggest blocker though. Ekko has no agents inside the instance/VM20:54
daemontool_it does not matter20:54
daemontool_where if it is a vm20:54
daemontool_or instance20:54
SamYaplecan you expand on what you mean?20:55
daemontool_does Ekko run where the data you want to backup is?20:55
SamYaplethe strucutre for Ekko is and api, a scheduler running on your "control" nodes (glorified cronjobs), and a backup-agent running on each compute20:56
daemontool_so the scheduler is behind the api, right?20:56
SamYaplethe data being backed up is the instance disks, but Ekko has no agent running in the instance20:56
SamYapleif I understand your question right, yes20:56
daemontool_ok20:57
daemontool_what is the purpose of the scheduler?20:57
daemontool_do you have an example workflow?20:57
SamYaplescheduling backups. for example at a certain time of day or when certain critera is met (like amount of data changed threshold)20:58
SamYapleI do not have anything in the Ekko repo yet20:59
daemontool_ok np20:59
SamYaplefor workflow that is20:59
*** daemontool has joined #openstack-freezer21:00
daemontoolSamYaple,  sorry I got disconnected21:01
daemontoolwhy in the solution you placed the scheduler behind the api?21:01
SamYaplewhere else would it run?21:02
daemontoolwe place it at client side21:03
daemontooldistribute any kind of load helps you to scale more21:03
daemontoolbut the question is, why you put it there :)21:03
*** daemontool_ has quit IRC21:03
daemontoolif there's any special reason21:03
daemontoolso what are your concerns on converging?21:04
SamYapleIt has to run somewhere, but client-side tools are not something we want to do21:04
daemontoolok21:04
SamYaplehypervisors already provide these tools and we just reuse those21:04
daemontoolI think the solution for the problem you try to solve is good21:04
daemontoolbut I do not understand what's the issue on converging... the real issue21:05
daemontoolI honestly see these as details, that can be solved, if we have the will21:05
SamYapleIm not sure what our projects have in common, other than the name 'backup'.21:06
SamYapleIf we can reuse compontents thats one thing, but other than API, i dont see what we can reuse21:06
SamYapleadditionally, to make this work properly with object storage we will require a caching layer to track data21:06
SamYaplewe have plans to use redis for this and this would be overhead that you would then have in freezer for no benift21:07
*** daemontool_ has joined #openstack-freezer21:07
daemontool_if you want to keep the name for the space of solution that Ekko provides, that's not a problem at all, but we need a unified backup, restore and disaster recovery solution21:08
SamYaplei think our disagreement is on whether file backup is "unified backup" or not21:09
SamYapleblock and file backup have nothing in common, those who use block backup typically dont use file backup21:09
daemontool_file, object, volumes, instance21:09
daemontool_are all resources that users wants to backup21:10
daemontool_and an environment like a company21:10
SamYaplebackup objects?21:10
*** daemontool has quit IRC21:10
daemontool_my wants to have21:10
daemontool_flexibility from a single solutino21:10
daemontool_to cover that issues21:10
daemontool_so if we have a modular approach21:10
daemontool_we can offer a really advanced tool21:10
SamYapleThe way I see it is if we can share a common api we will not be stepping on each others toes past that21:11
SamYaplewe can easily co-exist without stepping on each others toes, but i see little benifit in convergance for either of us21:11
SamYapleim not sure how sharing an api is possible with the current state of openstack though21:11
daemontool_sharing a common api  is not converging?21:11
daemontool_offering a common interface21:12
daemontool_a common suite of tools21:12
daemontool_and web interface21:12
daemontool_wouldn't be of a great benefit to users and community?21:12
SamYapleok, but how would that look from a backend or operator point of view?21:12
daemontool_we would have a plugin called Ekko in Freezer21:13
SamYaplewould that mean still requiring an agent inside of each instance?21:13
daemontool_that solve the problem you want to solve21:13
daemontool_we do not have to create manadatory dependencies21:13
SamYapledoes Freezer currently have a plugin architecture like this?21:14
daemontool_yes21:15
daemontool_I mean21:15
daemontool_the API would server what you want to serve from it21:15
daemontool_and the scheduler execute anything you want21:15
daemontool_we can add tab/panels to horizon21:15
daemontool_so show what Ekko wants to show21:15
daemontool_just FYI21:16
daemontool_Cinder and Nova teams in general21:16
daemontool_are very reluctant to install additional components on compute nodes21:16
daemontool_and storage nodes21:16
daemontool_that's my experience from the field21:16
SamYaplethey can be reluctant but they can't block anything since integration is not required with those components for ekko21:16
daemontool_It's not about blocking21:17
SamYapleintegrating with nova and cinder would be useful, but again not required21:17
daemontool_I don't think that's the point21:17
daemontool_if you do things in openstack21:17
daemontool_you need to integrate21:17
daemontool_you are not forced to21:17
daemontool_but you should integrate21:17
SamYapleRight but there is very little integrating that would be useful here21:18
SamYaplebut considering i have to interact with the hypervisors directly, there is a requirement for compute node agent21:18
daemontool_ok21:18
daemontool_I'm pretty sure if have think about it we'll find many ways to integrate :)21:19
daemontool_what would be the requirement?21:19
SamYapleso we have Freezer which has common things like API and Horizon plugin that can be expanded with block backup and file backup (file backup being what you do currently)?21:19
SamYapleis this what you are purposing?21:19
daemontool_file, vm and volumes, using the related services api21:20
daemontool_there's a plan21:20
daemontool_to do very foon21:20
daemontool_soon21:20
SamYaplewhat is the relative api for vm or volumes?21:20
daemontool_it uses the api from cinder and nova21:20
SamYapleright but neither of those can do backups21:20
daemontool_you have to set that config in a freezer job21:20
SamYaplethey can only do snapshots21:20
daemontool_ok21:20
daemontool_how the Ekko backup-agent21:21
daemontool_execute a backup?21:21
daemontool_on a cinder node?21:21
daemontool_and on a nova node?21:21
daemontool_s/and/or/21:22
daemontool_how the block based back up works21:22
SamYaplecinder is not planned for 1.0 release. so lets talk about nova for a second21:22
daemontool_ok, : )21:22
SamYapleekko-backup-agent on compute node that can speak to hypervisor (lets only talk about QEMU) directly21:23
SamYapleekko tells QEMU backup is about to begin via QMP21:23
SamYapleQMP tells instance and the QEMU guest agent which preps the machine for backup (vss/sync disk/flush mysql/etc)21:23
SamYapleQMP tells ekko instance is ready21:23
SamYapleekko tells QEMU to being backup using a qemu feature that will redirect writes to another location21:24
SamYapleinstance proceeds on as normal in the background at this point21:24
daemontool_is the qemu agent installed a requirement?21:24
SamYapleyes21:24
SamYaplewell no21:24
SamYapleno technically21:24
SamYapleto get a transactionally consistent backup it is21:25
SamYapleotherwise youll just get crash consistent21:25
SamYapleat this point in the process the orignal volume is sitting idle while all new writes go to a new location21:25
SamYaplei can perform the backup on the idle volume then21:25
daemontool_isn't that a snapshot?21:25
SamYaplenope21:25
SamYapleand snapshot is disruptive to instance (pauses instance) this does not21:26
daemontool_lvm snapshot is not21:26
SamYaplesnapshots are not backups21:26
daemontool_is crash consistent21:26
daemontool_does not require downtime21:26
SamYaplecorrect, but thats not for nova thats cinder21:26
daemontool_or any agent21:26
daemontool_and it works on any kind of hypervisor21:26
daemontool_if you have /var/lib/nova on an lvm volume21:27
SamYaplecrash consistent is not a reiliable backuip21:27
daemontool_what would be the difference?21:27
daemontool_ok21:27
daemontool_I'm just comparing21:27
daemontool_apple with apple :)21:27
SamYaplebut you are saying "use this feature from a tool", not everyone uses lvm to back /var/lib/nova21:27
SamYaplewhat about those using nfs and ceph?21:27
SamYapleekko doesnt care21:28
daemontool_ekko does not care21:28
daemontool_as long as you have21:28
daemontool_qemu21:28
daemontool_and the qemu agent21:28
SamYapleno, ekko doesnt care either way21:28
SamYapleif you want a transactionally consistent backup you need the qemu agent21:28
daemontool_if you cannot afford downtime, yes21:29
SamYaplei dont follow?21:29
daemontool_so Ekko provide a solution to the lack of incremental backup for nova vm instance21:29
daemontool_is that right?21:29
SamYaplelack of backup for nova vm at all, yes21:30
daemontool_nova supports vm backups21:30
daemontool_there's an api for that also21:30
daemontool_but they are not incremental21:30
SamYaplethats a snapshot, not a backup21:31
SamYaplesnapshots are not backups21:31
daemontool_the snapshot is used21:31
daemontool_to have a point in time21:31
daemontool_state of the vm21:31
daemontool_I agree is not a backup21:31
daemontool_but it can be leverage21:31
daemontool_d21:31
SamYaplewell you did say "nova supports vm backups"21:31
daemontool_I'm not saying your solution isn't good, actually it is21:31
daemontool_but it solves a very specific problem21:31
daemontool_and I do not see it a blocker21:31
daemontool_to not converge21:32
daemontool_that's basically my point21:32
*** EinstCrazy has joined #openstack-freezer21:32
daemontool_and also live snapshot features are available21:32
SamYapleSince neither of our projects conflict, I don't see a block to convergance either. For that matter cinder and ceilometer can merge21:32
daemontool_not for backup.. that's clear21:32
daemontool_cinder and ceilometer is their problem if they wants to converge21:33
daemontool_not ours21:33
SamYapleSo lets talk a bit more about what that would look like21:33
SamYapleEkko and Freezer merging that is21:33
daemontool_so lets do this21:33
daemontool_provide some more information on Ekko21:33
daemontool_that would be helpful anyway for you and the users too21:34
daemontool_and let's think and be open on the fact of converging21:34
SamYaplecertainly. I have no desire to maintain an API or Horizon plugins21:34
daemontool_then tell me please what really are your conditions like independent repo, implemented as plugin excention etc21:34
daemontool_then let's be open, and believe me, we'll make the world most advanced backup restore and dr solution21:35
daemontool_open source21:35
SamYapleas I said on the ML, i don't care so much about independance (I like community!) but I do care about progressing21:35
SamYapleso my conditions are fairly simple, progress21:35
SamYaplewe just need to discuss what a merge would look like21:36
daemontool_ok :)21:36
daemontool_look, no one is going to force21:36
daemontool_to do something you do not want to do21:36
daemontool_but we need to be reasonable21:36
daemontool_and offer the best to the users21:36
daemontool_I'm sure that's your intention21:36
daemontool_too21:36
SamYapleindeed it was21:36
daemontool_then let's do it : )21:36
daemontool_so21:37
daemontool_can you please provide a roadmap, a diagram and some more info?21:37
daemontool_if not it's ok21:37
daemontool_we can move forward without it21:37
SamYapleI can provide more info with pointed questions21:38
SamYapleI have no issue describing the whole process in detail, but "more info" is a bit broad for me (i dont do well with broad questions)21:38
daemontool_don't you think you should provide more information to who wants to use your tool?21:38
daemontool_did you notice21:38
SamYapleoh sure but the project is 3 weeks old21:38
daemontool_that the link on documentation on the Ekko repo return 404 not found?21:38
daemontool_that's an issue21:38
daemontool_the approach is important21:38
SamYapleyup docs are publishing yet21:38
SamYaplearent*21:39
daemontool_I'm not trying to load you with burocratic things21:39
daemontool_I'm just trying to provide some help to solve problem that we already solved in the past21:39
daemontool_because if you do not provide a bit of a roadmap, features and more info21:40
daemontool_people will not use your tool21:40
SamYapleAll of the people currently working on Ekko will be at the Kolla mid-cycle I am hosting in Feb, we will be doing a mini-ekko midcycle then21:40
SamYaplecurrently we have little useable code in the repo21:40
daemontool_I'm happy for you21:40
daemontool_:)21:41
SamYaplewell have more info coming out of that discussion21:41
daemontool_I'm glad you have leadership skills21:41
daemontool_but you still need to provide some design, roadmap21:41
daemontool_and useful information on how your tool work21:41
SamYaplewell we need to agree on those things first :)21:41
daemontool_which things?21:42
SamYapledesign and roadmap21:42
SamYaplewe will be doing this in Feb21:42
daemontool_ok21:42
daemontool_so let's do this, if it make sense to you21:43
daemontool_let's think 1 week21:43
daemontool_in an open way21:43
daemontool_on what can be integrated21:43
daemontool_and how that can benefit as much as possible users21:44
daemontool_let's try to approach this with that mindset if possible21:44
SamYapleOf course, as stated sharing API and even Horizon plugins is desired21:45
SamYapleI think we will need clear seperation of file and block backup though21:45
SamYapleAnd I am not sure how to achieve that21:45
daemontool_ok let's think about it21:45
daemontool_and we'll find a solution21:46
daemontool_now21:46
daemontool_my advice,21:46
daemontool_would be to send an email to the ml, to the thread21:46
daemontool_showing openness, and willing to find a solution together21:46
daemontool_do you agree with that?21:46
SamYapleThat sounds reasonable21:47
daemontool_brilliant21:47
daemontool_thank you21:47
daemontool_next week we'll talk, ok? :)21:47
SamYapleFrom your point of view are the only things we can share that we would over lap on the API and Horizon plugins?21:47
SamYapleOr is there more code i need to dig into for freezer?21:47
daemontool_I think that components can be leveraged21:48
daemontool_at least21:48
daemontool_I agree21:48
daemontool_but, let's think about it21:48
daemontool_more thoroughly21:49
SamYapleOk let me dig into that freezer code first21:49
SamYaplethen Ill seen a ML email asking about this21:49
daemontool_for now21:49
daemontool_I think21:49
daemontool_my advice is for you to focus21:49
daemontool_on Ekko code21:49
daemontool_and make that incredible21:49
daemontool_because as any new os project you have limited resources right?21:50
daemontool_so focus on that21:50
daemontool_what is important is our mindset I think21:50
daemontool_so let's think about it and talk next week21:50
daemontool_is that ok for you?21:50
daemontool_the mail to the ml is important for you to show openness21:50
daemontool_that's my advice21:51
daemontool_then do as you want21:51
SamYapleSure that sounds good21:52
daemontool_are you ok with our conversation?21:52
daemontool_there's anything you wanted to you but you didn't?21:52
daemontool_s/to you/to say/21:53
SamYapleNot at all. I think it's been fairly clear from both sides. We don't want to duplicate work nor do we want to slow the others progress down21:53
SamYapleWe will find a balance between the two21:53
daemontool_brilliant21:53
daemontool_nto next week then :)21:54
SamYaple:)21:54
*** daemontool has joined #openstack-freezer22:23
*** daemontool_ has quit IRC22:27
*** daemontool_ has joined #openstack-freezer22:39
*** daemontool has quit IRC22:42

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!