Tuesday, 2016-03-08

*** dschroeder has quit IRC00:16
*** reldan has quit IRC00:54
*** yangyapeng has joined #openstack-freezer01:05
*** yangyapeng has quit IRC01:06
*** yangyapeng has joined #openstack-freezer01:26
yangyapengping daemontool01:27
*** EinstCrazy has joined #openstack-freezer04:32
*** EinstCrazy has quit IRC05:27
*** yangyape_ has joined #openstack-freezer05:28
*** yangyape_ has quit IRC05:28
*** yangyape_ has joined #openstack-freezer05:30
*** yangyapeng has quit IRC05:32
*** sc has quit IRC05:46
*** yangyapeng has joined #openstack-freezer06:21
*** yangyape_ has quit IRC06:22
*** yangyapeng has quit IRC06:25
*** yangyapeng has joined #openstack-freezer06:26
yangyapengping daemontool07:26
yangyapenghi daemontool08:05
*** yangyapeng has quit IRC08:09
*** yangyapeng has joined #openstack-freezer08:10
daemontoolyangyapeng, yes08:12
yangyapengi have a question  about freezer job08:13
yangyapengdaemontool: freezer-agent --action restore --mode fs --storage local --restore-abs-path /opt/stack/restore --container /opt/stack/back/  --backup-name backup108:13
yangyapengit is ok, bot i write a json , it have a job. schedule---> runing -->complete  but it fail08:14
daemontoolhow's the content of the json file?08:15
daemontoolthe freezer-agent use ini format08:15
daemontoolthe scheduler use json format08:15
yangyapeng{08:16
yangyapeng     "job_actions": [08:16
yangyapeng         {08:16
yangyapeng             "freezer_action": {08:16
yangyapeng             "action": "backup",08:16
yangyapeng             "mode" : "fs",08:16
yangyapeng             "backup_name": "backuptest",08:16
yangyapeng             "path-to-backup": "/tmp/",08:16
yangyapeng             "storage": "local",08:16
yangyapeng            "container": "/opt/stack/backup/"08:16
yangyapeng         },08:16
yangyapeng         "max_retries": 5,08:16
yangyapeng         "max_retries_interval": 60,08:16
yangyapeng         "mandatory": true08:16
yangyapeng         }08:16
yangyapeng     ],08:16
yangyapeng     "job_schedule" : {08:16
yangyapeng     },08:16
yangyapeng     "description": "freezer test backup"08:16
yangyapeng }08:16
yangyapeng[stack@dev49 devstack]$ freezer-scheduler job-list08:16
yangyapeng\+----------------------------------+---------------------+-----------+-----------+-------+--------+------------+08:17
yangyapeng|              job_id              |     description     | # actions |   status  | event | result | session_id |08:17
yangyapeng+----------------------------------+---------------------+-----------+-----------+-------+--------+------------+08:17
yangyapeng| 300aab1ad8264526ae393fc77be57f3f | freezer test backup |     1     | completed |       |  fail  |            |08:17
yangyapeng+----------------------------------+---------------------+-----------+-----------+-------+--------+------------+08:17
daemontoolyangyapeng, what's the output of08:20
daemontoolfreezer-scheduler client-list08:20
yangyapeng+----------------------------------------+----------+-------------+08:21
yangyapeng|               client_id                | hostname | description |08:21
yangyapeng+----------------------------------------+----------+-------------+08:21
yangyapeng| 2c6d384e530044fcb021e2d7d4d5f6f5_dev49 |  dev49   |             |08:21
yangyapeng+----------------------------------------+----------+-------------+08:21
yangyapengtenant_id_host08:21
yangyapengfreezer-schedule  backup nova has a question  too08:22
daemontoolok08:22
daemontoolpaste in paste.openstack.org the last 200 lines of /var/log/freezer-scheduler.log and /var/log/freezer.log08:23
daemontoolor /var/log/freezer/freezer-scheduler.log08:23
daemontoolor /var/log/freezer/freezer.log08:23
yangyapeng.freezer/freezer-api.log and freezer-api.log08:24
yangyapenghttp://paste.openstack.org/show/489640/08:29
yangyapengping daemontool08:29
daemontoollookins08:29
daemontoollooking08:29
daemontoolyangyapeng, I see the following line in the logs08:31
daemontool2016-03-08 07:06:33.023 3399 CRITICAL freezer.main [-] [*] Critical Error: [Errno 13] Permission denied: '~'08:31
daemontooltry to execute the following command please08:32
daemontoolfreezer-agent  --storage local --path-to-backup /tmp  --container /opt/stack/back/  --backup-name backup-fs-tmp-108:33
daemontooldo you have permissions to write to /opt/stack/back ?08:34
yangyapengoh it is ok08:37
yangyapeng[stack@dev49 1457401066]$ ls08:37
yangyapengdev49_backup1_1457401066_0  tar_metadata_dev49_backup1_1457401066_008:37
yangyapeng[stack@dev49 1457401066]$ pwd08:37
yangyapeng/opt/stack/back/dev49_backup1/145740106608:37
yangyapeng[stack@dev49 1457401066]$08:37
yangyapengbackup and restore is ok. use freezer-agent08:40
ferhathello daemontool08:41
ferhatI want to change following wiki or at least I want to propose a change08:42
ferhathttps://wiki.openstack.org/wiki/Freezer08:42
ferhatdo I need to clone any source for this page?08:42
daemontoolferhat,  nope08:44
daemontoolthe wiki is just a normal wiki08:44
daemontoolso you need to create an account08:44
daemontooland edit it08:44
ferhatI see. but I think I am having a problem with creating account08:45
daemontoolaccording to where you are doing your changes08:45
daemontoolmy advise would be to add that info also in the README08:45
daemontoolof the related repo08:45
daemontooland to do that08:45
daemontoolyou need to clone the repo08:45
daemontoolferhat, what's the problem you are having?08:46
ferhatthx. I have cloned the repos. just need to configure garret08:47
daemontoolgerrit08:49
daemontoolok08:50
yangyapengHave I   to use the freezer-agant  backup nova . Is it ok use scheduler?08:51
daemontoolyangyapeng, you can use the freezer-agent08:51
daemontoolwhat really execute backups if the agent08:51
daemontoolthe scheduler is the component that execute the agent and report things to the api08:51
*** reldan has joined #openstack-freezer08:52
ferhatyes gerrit08:54
daemontoolslashme, can you check please if this change works from your side? https://review.openstack.org/#/c/277352/08:55
yangyapeng{08:55
yangyapeng     "job_actions": [08:55
yangyapeng         {08:55
yangyapeng             "freezer_action": {08:55
yangyapeng             "action": "backup",08:55
yangyapeng             "mode" : "fs",08:55
yangyapeng             "nova-inst-id": "7a88855c-ab3d-41d5-8d40-fddd7816d3a7",08:55
yangyapeng             "backup_name": "backuptest",08:55
yangyapeng             "storage": "local",08:55
yangyapeng            "container": "/opt/stack/backup/"08:55
yangyapeng         },08:55
yangyapeng         "max_retries": 5,08:55
yangyapeng     "description": "freezer test backup"08:56
yangyapeng }08:56
yangyapengis it ok ?08:56
daemontoolyes you just need to pass the nova-inst-id08:58
daemontoolreldan, do you remember if nova-inst-id is the same if passed from cmd line or within a config file?08:59
daemontoolfreezer-agent  --nova-inst-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b08:59
daemontoolyangyapeng,  that would be enough08:59
daemontoolas no actions currently are really recorded for vm backups08:59
daemontoolit just execute the nova api call to execute the backup08:59
yangyapengthe job's status is running  but i don't see it result to backup. container": "/opt/stack/backup/"  is empty.09:05
yangyapengbackup_madia nova is ok? or storage local is ok?09:06
yangyapengIf no work?09:07
daemontoolyangyapeng, nova09:12
daemontoolI mean freezer uses the nova api09:12
daemontoolto execute the backup of the vm09:12
daemontoolso the files are not stored in the storage media used normally by freezer09:13
daemontoolit's similar to the nova backup command09:14
daemontoolthe vm backup support is very basic09:15
*** reldan has quit IRC09:16
yangyapengoh, the code .1. get instance_id 2. create snapshot image 3. upload 4. delete image09:17
daemontoolyes09:17
yangyapengI think this will upload directly to storage media,ha ha :)09:19
yangyapengthx09:19
*** reldan has joined #openstack-freezer09:24
*** reldan has quit IRC09:26
daemontoolyangyapeng,  reldan implemented that part09:30
daemontoolall can we review https://review.openstack.org/#/c/288416/09:31
daemontoolyangyapeng, in the README is reported09:32
daemontool"Nova backups09:32
daemontoolTo make a nova backup you should provide a nova parameter in the arguments. Freezer doesn't do any additional checks and assumes that making a backup of that instance will be sufficient to restore your data in future."09:32
daemontoolthat is one of the features09:33
daemontoolwe need to improve09:33
yangyapengok  :)09:33
daemontoolyangyapeng,  hang on I see a bit of interesting code09:35
daemontoolhere https://github.com/openstack/freezer/blob/master/freezer/backup.py#L4809:35
daemontoolah ok09:36
daemontoolsorry yangyapeng09:36
daemontoolso, there 2 approaches09:37
yangyapengok09:37
daemontool1 is just executing a backup using the nova api09:37
daemontoolthe one I was mentioning09:37
daemontoolthen there's another one to reduce the risks of data inconsistencies09:38
daemontoolwith mysql db09:38
daemontoolwhere you can execute a flush tables, before executing a nova image snapshot09:38
daemontoolso the code there is for that use case09:38
daemontoolthis would be the example09:39
daemontoolfreezer-agent --mysql-conf /root/.freezer/freezer-mysql.conf09:39
daemontool--container freezer_mysql-backup-prod --mode mysql09:39
daemontool--backup-name mysql-ops00209:39
daemontool--nova-inst-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b09:39
daemontoolsame can be done with cinder09:39
daemontoolflush the db and execute the snapshot09:39
daemontoolnow did you execute that with --mode fs?09:39
daemontoolcause support 2 approaches09:40
daemontoolnovanative and cindernative09:40
daemontoolwhat just uses the api09:40
daemontoolso by using the --mode fs with the command you pasted here previously09:41
daemontoolare you downloading the nova instance?09:41
daemontoolthat was what you were saying?09:41
yangyapeng    backup_media = 'fs'09:54
yangyapeng    if backup_args.cinder_vol_id:09:54
yangyapeng        backup_media = 'cinder'09:54
yangyapeng    elif backup_args.cindernative_vol_id:09:54
yangyapeng        backup_media = 'cindernative'09:54
yangyapeng    elif backup_args.nova_inst_id:09:54
yangyapeng        backup_media = 'nova'09:54
yangyapengI has a mod =fs  and nova_inst_id=xxxx so, backup-madia = nova09:54
yangyapengwait  I look it again09:55
*** fabv has joined #openstack-freezer10:03
daemontoolall, can we review please https://review.openstack.org/#/c/288416/10:10
*** openstackgerrit has quit IRC10:33
*** openstackgerrit has joined #openstack-freezer10:33
m3m0yangyapeng: in your job file path-to-backup should be path_to_backup, did you fix your problem already?10:38
yangyapeng? the shoud replace auto  the code10:39
yangyapeng  command = attr[3:].replace('_', '-')10:40
yangyapengoh  i see , i have a try10:41
yangyapeng[stack@dev49 ~]$ freezer-scheduler job-get --job e089b3fd4bb64b67bf063e8fcc30434810:43
yangyapeng{10:43
yangyapeng    "user_id": "2aaa101aff584d5b962fd96c4f2db4a3",10:43
yangyapeng    "description": "freezer test backup",10:43
yangyapeng    "_version": 1,10:43
yangyapeng    "job_schedule": {10:43
yangyapeng        "time_ended": -1,10:43
yangyapeng        "time_started": -1,10:43
yangyapeng        "time_created": 145743374610:43
yangyapeng    },10:43
yangyapeng    "client_id": "2c6d384e530044fcb021e2d7d4d5f6f5_dev49",10:44
yangyapeng    "job_actions": [10:44
yangyapeng        {10:44
yangyapeng            "user_id": "2aaa101aff584d5b962fd96c4f2db4a3",10:44
yangyapeng            "freezer_action": {10:44
yangyapeng                "backup_name": "backuptest",10:44
yangyapeng                "container": "/opt/stack/backup/",10:44
yangyapeng                "path_to_backup": "/tmp/",10:44
yangyapeng                "storage": "local",10:44
yangyapeng                "mode": "fs",10:44
yangyapeng                "action": "backup"10:44
yangyapeng            },10:44
yangyapengi have a Incorrect writing, thank you  ,10:45
*** yangyapeng has quit IRC10:49
daemontoolfabv, would be good if the scheduler print in the log the option that is not correct10:57
*** fubi has quit IRC10:59
daemontoolm3m0, slashme fabv frescof  when you get the change can you review please https://review.openstack.org/#/c/288416/11:04
m3m0and remember we have code walkthrough this week11:11
daemontoolfabv, ^^11:12
daemontoolwe have the code walkthrough on the scheduler11:12
szaherdaemontool: sorry I wasn't using the PC for a while :), If you added a non existing option to the config file, I think oslo.config won't consider this options11:18
slashmedaemontool: reviewing https://review.openstack.org/#/c/288416/ and https://review.openstack.org/#/c/277352/ today11:20
szaherslashme: Can you take a look at this one https://review.openstack.org/#/c/288008/ ?11:23
szaherhttps://review.openstack.org/#/c/288422/11:23
daemontoolszaher, do we have somewhere the list of all the accepted options?11:25
openstackgerritSaad Zaher proposed openstack/freezer-api: Updating freezer-api elk mappings  https://review.openstack.org/28848611:33
szaherdaemontool: for the scheduler run this command inside freezer repo oslo-config-generator --config-file config-generator/scheduler.conf11:35
szaherthis will generate all available opts and put it in etc/scheduler.conf.sample11:36
daemontoolszaher,  ok ty, can we put that in the README when you get the chance?11:36
szaherSure11:37
daemontoolthe same apply for the agent?11:37
szaherIt can apply to the agent but we agreed we are not going to have a config file for the agent11:37
szaherthe functionality is there but I didn't provide a config file for the agent11:38
szaherdaemontool: if you want to see all options available for the agent execute this command oslo-config-generator --namespace freezer --namespace oslo.log11:39
daemontoolok, but let's say there's a user that want to see all the available optinos for the agent11:39
daemontoolok11:39
daemontoolthat's what I was looking for11:39
daemontoolplease add that to the README when you can11:39
daemontoolthanks a lot!11:39
szaherOk, sure11:39
m3m0that's a good point11:41
szaherdaemontool: README or HACKING ?11:45
daemontoolI think, in README goes the list of options and how to get it11:47
daemontoolin HACKING goes what do I have to do to add a new option11:47
daemontoolwhich is what I have to do right now :)11:48
daemontoolboth of them :)11:48
szaherOK :)11:48
openstackgerritMerged openstack/freezer: Adding option disable-exec to freezer-scheduler  https://review.openstack.org/28800811:48
m3m0freezer ui now uses python freezerclient :)11:51
daemontoolbrilliant11:52
daemontoolm3m0,  sorry today I'm going to upload the package to pypi11:52
*** EinstCrazy has joined #openstack-freezer11:56
openstackgerritSaad Zaher proposed openstack/freezer: Updating README and HACKING files for oslo options  https://review.openstack.org/28988112:03
szaherdaemontool: Please, take a look https://review.openstack.org/#/c/289881/12:03
daemontoolszaher, ++12:04
*** yangyapeng has joined #openstack-freezer12:10
openstackgerritSaad Zaher proposed openstack/freezer: Updating README and HACKING files for oslo options  https://review.openstack.org/28988112:18
*** ferhat has quit IRC12:27
daemontoolszaher, not sure I totally got how I can add options to the agent12:27
daemontoolI'm ok for the scheduler, and also how to add options to the scheduler12:28
daemontoolbut not the agent12:28
daemontoolhttps://www.openstack.org/summit/austin-2016/summit-schedule/events/721112:55
openstackgerritMemo Garcia proposed openstack/freezer-web-ui: Move from freezer to python-freezerclient  https://review.openstack.org/28990613:01
daemontoolszaher,  got it thanks13:02
daemontoolI think it's worth to mention that new options for the agent needs to be added in common/config.py, while for scheduler in scheduler/arguments.py13:11
openstackgerritSaad Zaher proposed openstack/freezer: Updating README and HACKING files for oslo options  https://review.openstack.org/28988113:21
szaherdaemontool: https://review.openstack.org/#/c/289881/13:22
openstackgerritSaad Zaher proposed openstack/freezer: Updating README and HACKING files for oslo options  https://review.openstack.org/28988113:23
daemontoolszaher,  looks better now, thanks :)13:26
openstackgerritFausto Marzi proposed openstack/freezer: Refactoring of code:  https://review.openstack.org/28841613:39
zhangjnping daemontool13:53
daemontoolzhangjn, yes13:56
zhangjnSome example is worry in README.rst13:59
zhangjnhow to backup a file in local?14:00
zhangjnsudo freezerc --file-to-backup /data/dir/to/backup --container /tmp/my_backup_path/ --backup-name my-backup-name --storage local14:00
zhangjnfreezerc is not option --file-to-backup14:01
openstackgerritFabrizio Vanni proposed openstack/freezer-api: add database status check  https://review.openstack.org/28993114:02
daemontoolfile-to-backup is deprecated14:03
daemontool--path-to-backup14:03
zhangjnthx14:06
openstackgerritFausto Marzi proposed openstack/freezer: Refactoring of code:  https://review.openstack.org/28841614:20
openstackgerritFabrizio Vanni proposed openstack/freezer-api: add database status check  https://review.openstack.org/28993114:30
zhangjnhow to check backup info by freezerc?14:31
zhangjnfreezerc --action info --container /tmp/bak --backup-name my-backup-name --storage local14:31
*** ddieterly has joined #openstack-freezer14:33
zhangjnis right?14:33
*** EinstCrazy has quit IRC14:34
m3m0as far as I know, you can;t do this now14:35
m3m0but14:35
m3m0in python-freezerclient you will be able to get individual backup information14:35
zhangjnI will try the README.rst operation, but something is not work.14:37
zhangjnhow to use python-freezerclient14:38
zhangjnI can test it.14:38
*** fabv has quit IRC14:40
m3m0is under development yet, but should be release this week I think14:41
zhangjnif the python-freezerclient is finish, whether the freezerc will be instead?14:42
zhangjnpython-freezerclient in this repo:https://github.com/memogarcia/python-freezerclient14:44
zhangjnm3m014:44
m3m0yes, I'm updating the README now14:45
m3m0it should work now BTW14:45
m3m0and you could be my first beta tester :)14:45
zhangjnmy pleasure.14:48
zhangjnI have some doubt.14:53
zhangjnpython-freezerclient is call freezer-api, is right?14:53
m3m0yes14:54
m3m0only for interacting with the api14:54
m3m0is a complete different component than the agent14:54
szaherm3m0: +1 workflow here https://review.openstack.org/#/c/289881/414:55
m3m0szaher: done14:56
daemontoolthe point is, how to show backup info if the api are not available...14:56
daemontoolall can we review this commit please https://review.openstack.org/#/c/288416/14:56
szaherszaher: thanx14:56
szaherm3m0: thanx14:57
zhangjnnow I backup use freezerc.14:58
zhangjnwhere is backup info store?14:59
*** fabv has joined #openstack-freezer15:05
zhangjnI think backup/restore info and action info should be store in db.15:07
openstackgerritMerged openstack/freezer: Updating README and HACKING files for oslo options  https://review.openstack.org/28988115:16
daemontoolhttps://review.openstack.org/#/c/288416/15:19
daemontoolzhangjn, it is stored in a db15:19
daemontoolif you have the api15:19
zhangjnall backup or other action must call api to do.15:21
zhangjnnow I can use freezerc cli to backup file or other action.15:21
daemontoolzhangjn,  yes the agent (freezerc or freezer-agent) only talks to the storage media15:29
daemontoolthe scheduler and the freezerclient only talks ot he api15:29
daemontools/ot he/to the/15:30
*** EinstCrazy has joined #openstack-freezer15:34
*** EinstCrazy has quit IRC15:41
*** dschroeder has joined #openstack-freezer15:53
*** daemontool_ has joined #openstack-freezer15:59
*** daemontool has quit IRC16:01
*** ferhat has joined #openstack-freezer16:12
daemontool_all, con you review this when you get the chance please https://review.openstack.org/#/c/288416/16:34
slashmedaemontool_: Hp has a release comming very quickly (liberty), I'm not very confortable merging this without its backport to liberty ready as it would hurt our capability to merge and backport fix quickly.16:39
*** EinstCrazy has joined #openstack-freezer16:40
slashmereldan: can you work on that backport please ? It shouldn't be too big of a job anyway16:40
*** EinstCrazy has quit IRC16:46
daemontool_slashme, thanks for the clarification16:48
*** ddieterly is now known as ddieterly[away]16:48
daemontool_slashme,  I'm doing it16:48
slashmedaemontool_: perfect :) :)16:48
*** ddieterly[away] is now known as ddieterly16:51
*** daemontool_ has quit IRC17:16
*** daemontool has joined #openstack-freezer17:21
szaherslashme: https://review.openstack.org/#/c/288422/ plz check this17:33
slashmei'm testing this atm17:33
slashmenot working17:39
*** EinstCrazy has joined #openstack-freezer17:44
*** EinstCrazy has quit IRC17:49
szaherslashme: I added some logging stuff to check if it's working or not so can we take a look at the log ?17:58
slashmeyep. lets do that in 517:58
daemontoolslashme,  https://review.openstack.org/#/c/290047/ but Eldar needs to review it also18:02
*** ddieterly is now known as ddieterly[away]18:03
*** ferhat has quit IRC18:19
slashmeSpeaking of bugs. Seems like we have some very weird behaviour with incremental when using the same backup_name and container_name18:21
*** ddieterly[away] is now known as ddieterly18:24
*** ddieterly has quit IRC18:24
*** ddieterly has joined #openstack-freezer18:26
*** ddieterly has quit IRC18:39
*** fabv has quit IRC18:48
*** daemontool has quit IRC19:51
*** ddieterly has joined #openstack-freezer20:27
ddieterlycan someone show me how to get an auth token for freezer from keystone in devstack?20:27
*** ddieterly is now known as ddieterly[away]20:58
*** ddieterly[away] is now known as ddieterly21:21
*** ddieterly has quit IRC22:47
*** ddieterly has joined #openstack-freezer23:19
*** ddieterly is now known as ddieterly[away]23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!