| *** dixiaoli has joined #openstack-karbor | 00:47 | |
| *** dixiaoli has quit IRC | 00:48 | |
| *** dixiaoli has joined #openstack-karbor | 00:49 | |
| *** liujiong has joined #openstack-karbor | 01:26 | |
| *** edisonxiang has quit IRC | 01:55 | |
| *** yinwei has quit IRC | 02:14 | |
| *** yinwei has joined #openstack-karbor | 02:15 | |
| *** x00350071 has joined #openstack-karbor | 02:23 | |
| *** x00350071 is now known as edisonxiang | 02:23 | |
| *** yamamoto has joined #openstack-karbor | 02:29 | |
| openstackgerrit | chenying proposed openstack/karbor master: Generic database Bank: Database tables and modules definition https://review.openstack.org/438259 | 02:59 |
|---|---|---|
| openstackgerrit | chenying proposed openstack/karbor master: Generic database Bank: Database tables and modules definition https://review.openstack.org/438259 | 04:44 |
| *** Roarermeadow has joined #openstack-karbor | 04:57 | |
| *** Roarermeadow has quit IRC | 04:58 | |
| *** dixiaoli_ has joined #openstack-karbor | 06:42 | |
| *** dixiaoli has quit IRC | 06:45 | |
| yuval | hey yinwei | 06:50 |
| *** dixiaoli_ has quit IRC | 06:50 | |
| *** dixiaoli has joined #openstack-karbor | 06:51 | |
| openstackgerrit | chenying proposed openstack/karbor master: Generic database Bank: plugin implementation https://review.openstack.org/438267 | 06:56 |
| chenying_ | yuval yinwei is on a convention now. | 06:57 |
| yuval | chenying_: lol | 06:57 |
| *** jiaopengju has joined #openstack-karbor | 07:04 | |
| chenying_ | ping yuval jiaopengju Are you free now? can we continue the discussion about file backup in karbor? | 07:14 |
| yuval | chenying_: yes | 07:14 |
| jiaopengju | yes | 07:15 |
| chenying_ | IMO we can first discuss the solution without agent first? what's your oppion about it? | 07:16 |
| jiaopengju | ok | 07:16 |
| yuval | jiaopengju: first, would you mind describing your deployment? | 07:16 |
| jiaopengju | My deployment? is it means the way we use openstack? | 07:18 |
| yuval | jiaopengju: yes, and how do backup and restore fit in | 07:19 |
| chenying_ | yuval Do you want to now the use case about file backup in jiaopengju's cloud? | 07:22 |
| chenying_ | S/now/know | 07:23 |
| yuval | chenying_: yes | 07:23 |
| jiaopengju | Currently, we have some public cloud and private cloud openstack environment . We use cinder-backup to backup the volumes , nova snapshot and some ohter steps to backup nova instance, and at the same time, use xtrabackup to backup the RD. All the backup client(where the backup request be sentted from) is management platform | 07:24 |
| jiaopengju | We backup files in physical machine and vms with using swift client or some othe shell scripts. | 07:25 |
| yuval | jiaopengju: thanks | 07:27 |
| yuval | jiaopengju: regarding the agent vs agentless discussion | 07:27 |
| jiaopengju | Now we want to unify all the backups in our openstack environment, and at the same time, provides a user define backup feature. | 07:27 |
| yuval | jiaopengju: I guess that agent-based solution is more problematic for you in public cloud | 07:28 |
| chenying_ | So you want to use karbor as uniform management platform to meet the use case about volume/nova Instance/database/file at the same time? | 07:28 |
| jiaopengju | chenying_: Yes | 07:28 |
| yuval | jiaopengju: you use xtrabackup to back up what? | 07:29 |
| jiaopengju | Relational Database | 07:29 |
| jiaopengju | e.g.: MySQL | 07:29 |
| yuval | jiaopengju: right | 07:29 |
| yuval | jiaopengju: and what are you using right now for file back up? | 07:30 |
| jiaopengju | the database is in users' vm, which boot from a special image | 07:30 |
| chenying_ | execute scripts or cmd in vm to backup files right now ? | 07:31 |
| jiaopengju | write some shell scripts | 07:31 |
| jiaopengju | yes | 07:31 |
| yuval | jiaopengju: in public cloud as well? | 07:31 |
| chenying_ | using swift client to backup. | 07:31 |
| chenying_ | using swift client to upload the files want to backup/ | 07:31 |
| jiaopengju | in public, now we do not have the file backup feature | 07:32 |
| jiaopengju | But we provide some ways for users to backup their files in vm to our object storage(ceph s3) | 07:32 |
| yuval | jiaopengju: in my opinion, agent based solution fits for private cloud scenario, and for public cloud where custom image is an option | 07:34 |
| yuval | jiaopengju: you said that you disable ssh in your machines. Is that in public cloud as well, or do you let users define their own security groups? | 07:35 |
| jiaopengju | You mean the image which contains backup agent? | 07:35 |
| yuval | jiaopengju: yes | 07:35 |
| chenying_ | IMO backup file with karbor, with or without agent to backup file are all the solutions about file backup plugins. karbor protect service/framework don't need care about this. The agent is a part of file bakcup plugin solution. | 07:35 |
| yuval | chenying_: I totally agree. If you have a plugin for agent-based protection and agentless protection, you could have some composite plugin to try the agent based and fallback to agentless | 07:37 |
| jiaopengju | In our public cloud, users can have different security groups, but not define by themselves | 07:37 |
| yuval | chenying_: I think the question is about the protection plugins themselves | 07:41 |
| jiaopengju | "Disable ssh in our machines", we do this in some of our private cloud, the machines are the nodes in openstack environment. | 07:41 |
| yuval | jiaopengju: the compute nodes or the guest VMs? | 07:43 |
| jiaopengju | we forbid the users or engineers login directly. | 07:43 |
| jiaopengju | no, it's physical server. | 07:44 |
| jiaopengju | the compute nodes | 07:45 |
| yuval | jiaopengju: good | 07:45 |
| chenying_ | jiaobengju the telnet/user want to backup files in the vms? Do we only consider this use case? Do we need conisder admin backup the files in nova's compute nodes? | 07:48 |
| jiaopengju | yes, just consider the backup files in vms. | 07:49 |
| yuval | jiaopengju: scp and more probably rsync seems like a good solution to that | 07:53 |
| chenying_ | don't need conisder the agent. I think it is a good way. | 07:54 |
| jiaopengju | you mean copy the files to karbor node through scp or rsync, and then upload them to swift ? | 07:55 |
| chenying_ | yuval: Does it mean that rsync first, then scp the files to karbor protection node, and then compress them to a zip file, and then backup the zip file to bank (swift, ceph S3) | 07:56 |
| jiaopengju | chenying_: good explanation | 07:57 |
| chenying_ | yuval: lots of data from vm need be transfer to karbor's node, maybe it is a problom. | 07:57 |
| yuval | chenying_: no real need to do that. The problem was already solved by things like: | 07:58 |
| yuval | https://github.com/ncw/rclone | 07:58 |
| yuval | http://duplicity.nongnu.org/ | 07:58 |
| chenying_ | Does it means that the tool rclone need be installed in user's vm? | 08:02 |
| jiaopengju | it menas we can use it to transfer the data in vms to swift directly? | 08:05 |
| yuval | chenying_: I'm not very familiar with rclone yet | 08:05 |
| chenying_ | How to solve the problom about transfering the data in vm to karbor's node using rclone or duplicity? | 08:05 |
| yuval | chenying_: we could use rclone to sync two remotes: the vm and the cloud storage, but it the data path goes through the karbor node | 08:07 |
| yuval | /but it the/but the/ | 08:08 |
| yuval | s/but it the/but the/ | 08:08 |
| yuval | not very efficient, but so are most methods for backing up files from vms | 08:09 |
| chenying_ | jiaopengju what's your oppion about it? | 08:09 |
| jiaopengju | When using reclone or duplicity, how we do stat? | 08:10 |
| chenying_ | yuval if the files in vm can be sync to bank(swift ceph) directly using rclone, it a good way. | 08:11 |
| yuval | chenying_: only if rclone is in the guest | 08:12 |
| chenying_ | There may be a simple way, if the swift/ceph client in vm, we also can backup the files in vm to bank(swift ceph) directly. | 08:13 |
| yuval | jiaopengju: I might have to say this. While file backup has it uses, I'm trying to think of a case where it fits into an 'application data protection' which is Karbor domain. Most cloud applications these days use a prebuilt image, volume for application-only persistent data, and the rest is ephemeral | 08:14 |
| jiaopengju | Our relational database is in a prebuilt iamge. | 08:16 |
| chenying_ | As freezer did, database backup in freezer just to backup the file or directory about mysql to starage. | 08:17 |
| jiaopengju | I know what you mean now | 08:17 |
| chenying_ | jiaopengju Install a tool like rclone to vm, do you think it is acceptable? | 08:19 |
| jiaopengju | yes | 08:19 |
| jiaopengju | or we can make an special image which contains the tool | 08:20 |
| chenying_ | OK we may need do a research about the tool like rlcone. Use it to backup files in plugin. | 08:22 |
| chenying_ | Rclone is a command line program to sync files and directories to and from lots of local/cloud storage. | 08:23 |
| yuval | chenying_: afaik, freezer is not backing up mysql, it gives existing backups mysql application level consistency | 08:24 |
| yuval | chenying_: for the actual backup it is guest based solution | 08:24 |
| chenying_ | yuval I agree, freezer is not actual backup about mysql. | 08:27 |
| jiaopengju | yuval: I'm not particularly understanding about "application level consistency" | 08:32 |
| yuval | jiaopengju: application level consistency in this case: tell mysql to flush tables to disk, so files will not contain half applied transactions | 08:34 |
| jiaopengju | yuval: thanks for your explanation. | 08:36 |
| yuval | jiaopengju: you can see: https://dev.mysql.com/doc/refman/5.7/en/replication-features-transaction-inconsistencies.html | 08:36 |
| jiaopengju | snapshot may be system level? | 08:36 |
| jiaopengju | thanks | 08:37 |
| openstackgerrit | chenying proposed openstack/karbor master: Add database_bank fullstack https://review.openstack.org/452085 | 08:57 |
| openstackgerrit | chenying proposed openstack/karbor master: Fix fullstack error about one test https://review.openstack.org/463731 | 08:59 |
| openstackgerrit | chenying proposed openstack/karbor master: Add database_bank fullstack https://review.openstack.org/452085 | 09:05 |
| *** jiaopengju has quit IRC | 09:22 | |
| *** yamamoto has quit IRC | 10:05 | |
| *** liujiong has quit IRC | 10:28 | |
| *** yamamoto has joined #openstack-karbor | 10:32 | |
| *** yamamoto has quit IRC | 11:29 | |
| *** dixiaoli has quit IRC | 11:35 | |
| openstackgerrit | Merged openstack/karbor master: Trivial fix typos in documents https://review.openstack.org/463634 | 11:54 |
| *** yamamoto has joined #openstack-karbor | 12:29 | |
| *** yamamoto has quit IRC | 12:39 | |
| *** x00350071_ has joined #openstack-karbor | 12:40 | |
| *** edisonxiang has quit IRC | 12:40 | |
| *** yamamoto has joined #openstack-karbor | 12:45 | |
| *** yamamoto has quit IRC | 12:47 | |
| *** yamamoto has joined #openstack-karbor | 13:04 | |
| *** tommylikehu has quit IRC | 13:44 | |
| *** tommylikehu has joined #openstack-karbor | 13:44 | |
| *** gouthamr has joined #openstack-karbor | 14:15 | |
| *** gouthamr has quit IRC | 14:34 | |
| *** gouthamr has joined #openstack-karbor | 14:48 | |
| *** zhonghua2 has quit IRC | 15:24 | |
| *** gouthamr has quit IRC | 16:12 | |
| *** yamamoto has quit IRC | 16:13 | |
| *** yamamoto has joined #openstack-karbor | 16:14 | |
| *** dixiaoli has joined #openstack-karbor | 18:27 | |
| *** yinwei has quit IRC | 20:53 | |
| *** yinwei has joined #openstack-karbor | 20:54 | |
| *** dixiaoli has quit IRC | 23:17 | |
| *** gouthamr has joined #openstack-karbor | 23:39 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!