| *** hemna7 is now known as hemna | 00:53 | |
| opendevreview | Rajat Dhasmana proposed openstack/cinder master: NFS Backup: Remove leftover dirs https://review.opendev.org/c/openstack/cinder/+/804603 | 06:54 |
|---|---|---|
| *** tkajinam is now known as Guest5435 | 07:14 | |
| *** Guest5435 is now known as tkajinam | 07:36 | |
| opendevreview | Harsh Ailani proposed openstack/cinder master: [SVf]: Incorrect portset value during driver intialization https://review.opendev.org/c/openstack/cinder/+/850310 | 10:03 |
| whoami-rajat | geguileo, left a few questions on https://review.opendev.org/c/openstack/os-brick/+/836059 | 10:17 |
| whoami-rajat | IIUC that's the first patch in the nvmeof chain, so we need to merge that first | 10:17 |
| opendevreview | Gorka Eguileor proposed openstack/os-brick master: LUKS: Support extending host attached volumes https://review.opendev.org/c/openstack/os-brick/+/836059 | 10:41 |
| opendevreview | Gorka Eguileor proposed openstack/os-brick master: NVMe-oF: Consolidate code paths https://review.opendev.org/c/openstack/os-brick/+/836060 | 10:41 |
| opendevreview | Gorka Eguileor proposed openstack/os-brick master: NVMe-oF: Disconnect subsystems https://review.opendev.org/c/openstack/os-brick/+/836062 | 10:41 |
| opendevreview | Gorka Eguileor proposed openstack/os-brick master: Support shared_targets tristate value https://review.opendev.org/c/openstack/os-brick/+/836063 | 10:41 |
| geguileo | whoami-rajat: you are correct | 10:41 |
| geguileo | whoami-rajat: I have reworded those 2 log messages to use the cryptsetup naming convention | 10:42 |
| whoami-rajat | geguileo, great, will take a look | 10:58 |
| *** dviroel|out is now known as dviroel | 11:27 | |
| opendevreview | Tushar Trambak Gite proposed openstack/cinder master: Configure a storage node in cinder https://review.opendev.org/c/openstack/cinder/+/820994 | 11:49 |
| sfv880 | Hello! I would like to fix a few bugs in the Infinidat driver and would like to ask your opinion on the best way to do this. For example, I could open several independent pull requests and then bump the driver version with a separate request. Or bump the driver version in the first pull request and make all subsequent changes dependent. Please tell me - what is the best way to do it? Thank you! | 12:38 |
| rosmaita | sfv880: probably best not to worry about versioning at this point, can depend on the severity of the bug and whether the fix will be backported ... please make sure you file a bug in https://launchpad.net/cinder for each of the bugs and include a "Closes-bug: #123456" in the commit message on your patch | 12:52 |
| rosmaita | need a review on https://review.opendev.org/c/openstack/cinderlib/+/849432/ to unblock cinderlib gate | 13:03 |
| rosmaita | e0ne eharney geguileo hemna jungleboyj smcginnis enriquetaso ^^ | 13:03 |
| yuval | hey, I see some new failures in our third party ci, only happens on multiattach tests: I get the following error | 13:25 |
| yuval | Details: {'code': 400, 'message': "Invalid input for field/attribute server. Value: {'name': 'tempest-VolumeMultiattachTests-server-1068649218', 'imageRef': '47ba6e9c-0cd1-4e7c-bb96-2b780d326b22', 'flavorRef': '0', 'key_name': 'tempest-VolumeMultiattachTests-1885608089', 'security_groups': [{'name': 'tempest-secgroup-smoke-416482459'}]}. 'networks' is a required property"} | 13:25 |
| yuval | which networks is missing? all other tests in tempest and the cinder plugin temps is passing | 13:25 |
| hemna | is there a way to disable incremental backups? | 13:36 |
| hemna | force a full backup every time? | 13:36 |
| opendevreview | Merged openstack/cinder stable/xena: PowerMax Driver - Retype fix for replication https://review.opendev.org/c/openstack/cinder/+/821744 | 13:45 |
| enriquetaso | hemna, There is no way to disable it. Let me know if I am wrong, but as far as I know, the Cinder client always creates a full backup unless you use the optional --incremental arguments. | 13:46 |
| enriquetaso | https://docs.openstack.org/python-cinderclient/latest/cli/details.html#cinder-backup-create | 13:47 |
| hemna | yah unfortunately, thats what I was seeing as well. | 13:49 |
| geguileo | hemna: what would be the use case for that? | 14:43 |
| hemna | vmware sucks | 14:43 |
| hemna | unfortunately it's a combination of issues related to vmware requiring stream optimized images | 14:44 |
| hemna | basically we don't see the raw disk at backup time because it's compressed, so the chunking can be all out of order and different sizes between backups | 14:44 |
| hemna | so the easiest thing is for us to disable incrementals entirely | 14:44 |
| hemna | to ensure that the backups are consistent/sane | 14:45 |
| hemna | related to this review https://review.opendev.org/c/openstack/cinder/+/831439/4 | 14:45 |
| hemna | we only get to see the compressed image at backup time, so the chunks can shrink and grow abnormally compared to chunks from a raw disk. | 14:46 |
| hemna | really needs to be a custom vmware backup driver written by vmware, but vmware doesn't really support openstack at all nowdays. | 14:47 |
| opendevreview | Alexander Deiter proposed openstack/cinder master: Fix Infinidat driver to use SSL options https://review.opendev.org/c/openstack/cinder/+/849846 | 14:48 |
| geguileo | hemna: then it seems like the chunked driver needs some improvements, right? | 14:55 |
| hemna | yah | 14:55 |
| geguileo | what I don't get is why we see it as compressed in backup | 14:55 |
| hemna | but also would be nice if we could disable incrementals w/o hacking the code | 14:55 |
| geguileo | but don't see it as compressed if we are doing a create volume from image... | 14:55 |
| hemna | this is vmware backend, the images are streamoptimized, re: compressed | 14:56 |
| geguileo | so cinder never uses dd, right? | 14:56 |
| hemna | correct | 14:56 |
| geguileo | then generic live migration would not work either, right? | 14:57 |
| geguileo | sorry, generic offline migration | 14:57 |
| geguileo | the one that copies data in Cinder | 14:57 |
| hemna | no | 14:58 |
| hemna | that happens in vcenter itself | 14:58 |
| hemna | cinder doesn't touch data during migrations | 14:58 |
| geguileo | not even for offline? | 14:59 |
| hemna | correct | 14:59 |
| hemna | this is vmware. everything is fuct | 14:59 |
| hemna | and different | 14:59 |
| geguileo | I think I'm getting the idea | 14:59 |
| opendevreview | Alexander Deiter proposed openstack/cinder master: Fix Infinidat driver to return all iSCSI portals https://review.opendev.org/c/openstack/cinder/+/849022 | 15:00 |
| opendevreview | Merged openstack/cinderlib master: Don't use yum to install ceph https://review.opendev.org/c/openstack/cinderlib/+/849432 | 15:00 |
| hemna | we've tried to come up with ways of fixing this, but every option is risky for corrupting the chunks | 15:01 |
| geguileo | hemna: how would they get corrumpted? | 15:01 |
| hemna | well, we thought of uncompressing the stream and then adding zeros in to fill and recompressing | 15:02 |
| hemna | it just gets ugly from there | 15:02 |
| geguileo | hemna: oh! making os-brick return an IOWrapper that did all that? | 15:03 |
| hemna | heh, in our case we aren't even using os-brick | 15:03 |
| hemna | it's a modified chunked driver that uses an http connection to esxi | 15:04 |
| geguileo | ouch, ouch | 15:04 |
| hemna | vmware's support in openstack is not really usable at much of any scale without what we've changed | 15:04 |
| hemna | vmware is not cloud software | 15:05 |
| hemna | so I was thinking of adding a config option to disable incremental backups. If that's on, then have cinder throw an exception. since that isn't part of the current API, I was going to silently force the incremental to False in the API here: https://github.com/sapcc/cinder/blob/stable/train-m3/cinder/backup/api.py#L249 | 15:07 |
| hemna | (that's our fork of cinder) | 15:07 |
| geguileo | hemna: wouldn't your users be surprised to request incremental and get a non incremental backup? | 15:08 |
| hemna | possibly. not sure they would even know really | 15:09 |
| hemna | as it stands today, I'm not sure backups are really working well after the primary backup | 15:09 |
| geguileo | hemna: From upstream perspective I'm against adding a flag to disable this | 15:11 |
| geguileo | I think we could allow os-brick connectors to somehow mark this particular behavior on the dict they return on connect_volume | 15:11 |
| geguileo | and then the backup driver to act on it | 15:11 |
| hemna | what's the problem of allowing a config option? | 15:12 |
| hemna | especially if they don't work :) | 15:12 |
| geguileo | that we can have multiple backends, one that works and another that doesn't | 15:13 |
| hemna | couldn't it be specified on the backend ? | 15:13 |
| geguileo | so if os_brick return in connect_volume return {'path': tmp_file_path, 'last_modified': last_modified, 'varying_size': True} | 15:14 |
| hemna | I wasn't aware you could configure more than 1 backup service in the cinder.conf at the same time | 15:15 |
| geguileo | then the chunked_backup driver may be able to act on it and: ignore the incremental request and do full backup, then set the DB to the right value, and create a user message explaining what just happened | 15:15 |
| geguileo | not more than 1 backup, more than 1 backend | 15:15 |
| geguileo | more than 1 volume backend | 15:15 |
| geguileo | which is the actual problem, iiuc | 15:15 |
| hemna | ah ok, that makes more sense | 15:16 |
| hemna | the vmware driver gives the backup service a vmdk file, which is compressed and there isn't a reliable way to decompress it to do the chunking | 15:17 |
| geguileo | and I don't know how this is not a problem when using the NFS backend with compressed qcow2 | 15:17 |
| hemna | it probably is | 15:17 |
| hemna | hence that review | 15:17 |
| hemna | https://review.opendev.org/c/openstack/cinder/+/831439/4 | 15:17 |
| geguileo | so we would need an automatic solution | 15:17 |
| geguileo | hemna: but I'm not sure that that patch would actually fix things | 15:18 |
| geguileo | because like you said, the size changes | 15:18 |
| geguileo | so a later file could be smaller | 15:18 |
| hemna | chunking on compressed data doesn't really make sense, as the chunks can change quite a lot. you could end up with more or less chunks depending on where in the raw disk was changed since last backup | 15:18 |
| hemna | I | 15:19 |
| geguileo | hemna: yeah, and it would need to be careful with the size | 15:19 |
| hemna | I'm not entirely sure that review 'fixes' it either, but what it is alluding to in the change makes sense | 15:19 |
| geguileo | I'd have to check the behavior and check how that patch fixes things | 15:19 |
| geguileo | hemna: afaik the fix would also require changes to the restore (though I could be wrong) | 15:20 |
| hemna | I think as long as the chunk chain is correct, the restore should be ok | 15:21 |
| geguileo | hemna: maybe, I'd have to check the restore code | 15:24 |
| opendevreview | Merged openstack/cinderlib master: Update deprecated zuul syntax https://review.opendev.org/c/openstack/cinderlib/+/849409 | 15:28 |
| hemna | it's been interesting to see how many assumptions and limitations that cinder proper has in place that I've had to violate to make it work in a large scale environment | 15:31 |
| *** akekane_ is now known as abhishekk | 15:37 | |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/yoga: Don't use yum to install ceph https://review.opendev.org/c/openstack/cinderlib/+/850393 | 15:45 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/yoga: Update deprecated zuul syntax https://review.opendev.org/c/openstack/cinderlib/+/850394 | 15:46 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/xena: Don't use yum to install ceph https://review.opendev.org/c/openstack/cinderlib/+/850395 | 15:48 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/xena: Update deprecated zuul syntax https://review.opendev.org/c/openstack/cinderlib/+/850396 | 15:48 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/wallaby: Don't use yum to install ceph https://review.opendev.org/c/openstack/cinderlib/+/850399 | 15:51 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/wallaby: Update deprecated zuul syntax https://review.opendev.org/c/openstack/cinderlib/+/850400 | 15:51 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/victoria: Don't use yum to install ceph https://review.opendev.org/c/openstack/cinderlib/+/850401 | 15:53 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/victoria: Update deprecated zuul syntax https://review.opendev.org/c/openstack/cinderlib/+/850402 | 15:53 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/ussuri: Don't use yum to install ceph https://review.opendev.org/c/openstack/cinderlib/+/850403 | 15:54 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/ussuri: Update deprecated zuul syntax https://review.opendev.org/c/openstack/cinderlib/+/850404 | 15:55 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/train: Update deprecated zuul syntax https://review.opendev.org/c/openstack/cinderlib/+/850405 | 15:57 |
| whoami-rajat | tosky, ^ that's a looooong backport chain | 15:58 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/ussuri: Migrate jobs from CentOS 8 to CentOS 8 Stream https://review.opendev.org/c/openstack/cinderlib/+/824421 | 16:33 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/ussuri: Don't use yum to install ceph https://review.opendev.org/c/openstack/cinderlib/+/850403 | 16:33 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/ussuri: Update deprecated zuul syntax https://review.opendev.org/c/openstack/cinderlib/+/850404 | 16:33 |
| opendevreview | Luigi Toscano proposed openstack/cinderlib stable/train: Update deprecated zuul syntax https://review.opendev.org/c/openstack/cinderlib/+/850405 | 16:34 |
| opendevreview | Simon Dodsley proposed openstack/cinder master: Add Pure Storage NVMe-RoCE driver https://review.opendev.org/c/openstack/cinder/+/799871 | 18:58 |
| opendevreview | Sofia Enriquez proposed openstack/cinder master: Implement copy encrypted image in NFS generic driver https://review.opendev.org/c/openstack/cinder/+/749155 | 19:19 |
| *** dviroel is now known as dviroel|dinner | 22:22 | |
| *** dviroel|dinner is now known as dviroel|out | 23:20 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!