| EugenMayer | Hello. Is there any way to trigger a snapshot using the cli? since i cannot find glance in the openstack cli, is there a dedicated cli? | 19:19 |
|---|---|---|
| EugenMayer | i see there is https://docs.openstack.org/ocata/cli-reference/glance.html but snapshots are not mentioned. | 19:21 |
| EugenMayer | i guess it has been part of openstack in the past https://docs.openstack.org/python-openstackclient/ocata/command-objects/snapshot.html but seems no longer to be the case? | 19:23 |
| EugenMayer | i see there is volume snapshot create, but it seems to be dedicated to volumes. Cannot see how to trigger an snapshot of an instance this way | 19:24 |
| EugenMayer | ok found it, it is `server backup create` - not sure how that happened to end there :) | 19:27 |
| dansmith | no, backup is different. You're looking for "server image create" | 19:27 |
| EugenMayer | dansmith what is the difference? There is also server restore, but that seems not even to work when i shutdown the server | 19:33 |
| EugenMayer | Cannot 'restore' instance 20dbba41-3ddf-48f1-b934-9c47f75c5b0d while it is in vm_state stopped (HTTP) | 19:33 |
| EugenMayer | seems like server image create - creates someting of type image under /project/images, while backup uses type snapshot | 19:34 |
| dansmith | image create is a snapshot, | 19:36 |
| dansmith | backup tries to keep records to make periodic snapshots possible | 19:36 |
| dansmith | server snapshots go to glance by definition | 19:36 |
| EugenMayer | there is yet no API to lookup all backups for an instance, right? i see there is some meta-data instance_uuid on the snapshot | 19:38 |
| dansmith | they're not special enough to be listed - they're just images | 19:39 |
| EugenMayer | dansmith wait, 'image create' is actually what creates the type 'snapshot', while `backup create` is of type image? | 19:39 |
| dansmith | the backup API has historically been pretty under-used and unloved, so I really wouldn't use it | 19:39 |
| dansmith | EugenMayer: no, there are only images | 19:40 |
| EugenMayer | dansmith well the odd thing is, there is no backup concept in openstack entirely, is it? | 19:40 |
| dansmith | there are images, that's it. Nova provides a backup API which tries to make it easier to schedule periodic snapshots, but that's it | 19:41 |
| dansmith | you can call a snapshot a backup or not, up to you | 19:41 |
| EugenMayer | there is freezer - seems entire dead? (not sure), at least it seems in a broken state right now. Creating snapshots seems somewhat one way, if one does not consider cinder storage snapshots to be "the backup" | 19:41 |
| EugenMayer | so backups are rather to be triggered using the nova api? | 19:41 |
| dansmith | depends on what you're snapshotting. If you want a snapshot of the instance's disk, then yes of course through nova | 19:42 |
| dansmith | if you have a purely volume-backed instance, and you really want a snapshot of the volume in cinder, then ... cinder | 19:42 |
| EugenMayer | i have no volumes at all, only instance disks | 19:43 |
| EugenMayer | (so using local disks only) | 19:43 |
| dansmith | then snapshotting cinder volumes is unlikely to be much use to you | 19:43 |
| dansmith | there's nothing else _to_ do the snapshot against, other than nova, so I'm not sure where the confusion is | 19:43 |
| EugenMayer | sure, that is what i have expected. What i wonder is, why backups are "unloved" and there is not much talk about that? Do i miss something obvious? | 19:44 |
| dansmith | the backup api is *just* a wrapper around image_create with some attempt at periodic scheduling | 19:44 |
| dansmith | it's unloved because it's under-used, and was written before there were as many ways to back an instance, so it has historically been broken for certain configs | 19:45 |
| EugenMayer | understood | 19:45 |
| EugenMayer | yet no seen anything periodic, no scheduling at all. Might have missed that part? where are those periodic settings. Found nova backup, which has a rotation parameter | 19:46 |
| EugenMayer | <name>, <backup-type>, <rotation> | 19:46 |
| dansmith | there is no scheduling | 19:46 |
| dansmith | notice how I keep using words like "attempt at" and "you probably don't want this" ? | 19:47 |
| dansmith | here's a random thing I found from 2016 which explains how it's supposed to work, including explaining that there is no scheduling: https://raymii.org/s/tutorials/OpenStack_Quick_and_automatic_instance_snapshot_backups.html | 19:47 |
| EugenMayer | Sorry, not trying to ignore you/being ignorant, truying to really listen and wrap my head arround it | 19:47 |
| dansmith | based on your questions from the nova channel, I'm guessing you're approaching this from the conventional virt perspective, and openstack really isn't that.. it's cloud | 19:47 |
| dansmith | so things you might think would be first-class features aren't | 19:48 |
| dansmith | in cloud, you get charged per byte per image, and instances are intended to be ephemeral (i.e. nothing to backup anyway) so periodic backups would just be periodic bills | 19:48 |
| dansmith | precious data is stored in volumes (i.e. cinder) and those are snapshotted (and smartly) when you need things like rollback points | 19:49 |
| EugenMayer | well we have somewhat of a hybrid setup. We have VMs which will stay as such since they cannot be containrized or should not ( very few ). All the other applications are going to be run in k8s, and we will see about storage solutions there (we are yet not sure). Currently we try to avoid to run X-db cluster (pg/mariadb ..10/11 5/8). So i try to | 19:50 |
| EugenMayer | find a good combination for us in general. | 19:50 |
| dansmith | yeah, so your DB should be on a cinder volume probably | 19:51 |
| EugenMayer | And backups of VMs, be it a kubernates node or a "VM classic" is essential for us. I understand that in OpenStack you backup volumes and treat instances(disk/ephimeral) as "can be thrown away" | 19:51 |
| dansmith | using server snapshots to backup precious data will be quite inefficient in most cases | 19:51 |
| EugenMayer | a db on a cinder volume would slow it down big times? | 19:52 |
| dansmith | depends on your backend, but yeah it's likely to be slower than raw file on local disk of course | 19:52 |
| EugenMayer | interesting, why you consider snapshots inefficient - du to there size and maybe issues with being incremental ( so ineffecient in "the time to create" and the "the storage they take up" ) ? | 19:52 |
| dansmith | are you using ceph? | 19:53 |
| EugenMayer | no, i considered it. But i was told 25GB on the network would be something you need to have a proper speed | 19:53 |
| EugenMayer | and i have 1GB | 19:53 |
| dansmith | if not, a snapshot will involve pausing the instance for the entire time it takes to make a *full* copy of the root disk, and then it will upload the entire thing to glance over HTTP | 19:53 |
| dansmith | that will be super bad for a DB you expect to be highly available, and will cost a lot of network, cpu and storage resources to do periodically | 19:54 |
| dansmith | if you're using anything other than raw image backend files on nova, also add the time it takes to fully flatten the original image with the cow bits before that upload | 19:55 |
| EugenMayer | i understand, we do this daily. So once a day there is a short interruption. But considering that the cinder storage is slow, the DB / response times of the apps will be slow the entire day ? | 19:55 |
| dansmith | well, if that's what you want to do then sure, but for any real size I think you'll find it quite painful | 19:57 |
| dansmith | conventional virt that is meant to snapshot local disk images in place would be massively more efficient | 19:57 |
| EugenMayer | well we have done this with proxmox for a long time now, so i at least have some "historical data" that it kind of works out for us. But i understand you concerns | 19:57 |
| dansmith | but regardless, I'm trying to explain why you're expecting to see proper snapshots and backup automation and aren't finding it | 19:58 |
| EugenMayer | i understand, the concept is different | 19:58 |
| EugenMayer | Is there also a historical reason why freezer seems to be abondened? | 19:58 |
| dansmith | the same reason anything is abandoned, especially in openstack: nobody cares to use or work on it anymore | 19:59 |
| dansmith | but to be clear, I don't think freezer did anything other than automate what we've just discussed | 19:59 |
| dansmith | it doesn't fundamentally change anything about the underlying topology | 19:59 |
| EugenMayer | i think freezer used LVM snapshots on the VM itself, to eliminate downtimes. AFAIU | 20:00 |
| dansmith | ah, sorry yeah I'm just refreshing | 20:00 |
| dansmith | this is the inband thing, so requires a client agent | 20:00 |
| EugenMayer | no i think for nova it somehow used nova to get to the instance, not sure though | 20:01 |
| dansmith | there was something else at one point that just tried to do correlated nova+cinder snapshots of multiple things | 20:01 |
| EugenMayer | it had different modes, also to backup databases and things. Sounded promising to me | 20:01 |
| dansmith | there's no freezer integration with nova's backing store | 20:01 |
| EugenMayer | might have just missunderstood it. I could not get it working once due to keystone authentication issues | 20:02 |
| dansmith | https://docs.openstack.org/freezer/latest/ | 20:02 |
| dansmith | seems to clearly show it's inband only | 20:03 |
| dansmith | i.e. fs snapshots in the guest OS | 20:03 |
| EugenMayer | how do you define inband, maybe i'am not aware of that term this way. | 20:03 |
| dansmith | the word nova is nowhere on that page | 20:03 |
| dansmith | inband meaning, just treating the instance like any other machine, nothing related to the virt infra around it | 20:04 |
| dansmith | uses keystone for auth, swift (or something else) to store the backup data, but requires running something *inside* the instance | 20:04 |
| EugenMayer | interestingly this something (agent) in the instance (if you mean the VM) yet not crossed me, odd | 20:05 |
| EugenMayer | must be the Freezer Agent then | 20:06 |
| EugenMayer | You helped me a lot with your insight, thank you big times! | 20:08 |
| dansmith | "freezer agent...software that runs on the client side, where the data backup is to be executed" | 20:08 |
| dansmith | also, in the arch diagram, | 20:08 |
| dansmith | the "node" is the thing with the agent, and it says that can be either "VM, desktop, physical, or windows" | 20:09 |
| dansmith | ^ that is why I say it's totally independent of nova, AFAICT | 20:09 |
| EugenMayer | i think before i start adding agent of freezer to my VMs, i will rather use bacula for backups and do the same. The latter is at least here to stay | 20:09 |
| dansmith | yeah, I was just going to say, unless you need the integration with keystone and swift, any other backup solution would probably suffice | 20:10 |
| dansmith | and do much better than entire instance snapshots every night | 20:10 |
| EugenMayer | dansmith you are right, i totally missunderstood the concept, i though of the client (i was wondereing about the wording all the time) as the runner of the job. But in the end, i should have sticked with the smell that "client" does not make sense | 20:10 |
| dansmith | well, that freezer doc page definitely doesn't seem to lay it out very clearly, IMHO | 20:11 |
| EugenMayer | I think i might think about combining both somewhat, few rare instance backups and regularly incremental bacula | 20:11 |
| dansmith | so it takes a minute to grok | 20:11 |
| * dansmith nods | 20:12 | |
| EugenMayer | Thank you for your valuable time! | 20:12 |
| * dansmith nods again | 20:13 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!