| *** rcernin has quit IRC | 00:17 | |
| *** rcernin has joined #openstack-swift | 00:17 | |
| *** baojg has joined #openstack-swift | 01:08 | |
| seongsoocho | morning o/ | 01:23 |
|---|---|---|
| *** nanzha has joined #openstack-swift | 01:56 | |
| *** diablo_rojo__ has joined #openstack-swift | 03:41 | |
| *** diablo_rojo__ has quit IRC | 04:31 | |
| *** nanzha has quit IRC | 04:40 | |
| *** nanzha has joined #openstack-swift | 04:41 | |
| *** nanzha has quit IRC | 05:00 | |
| *** nanzha has joined #openstack-swift | 05:02 | |
| openstackgerrit | Charles Hsu proposed openstack/python-swiftclient master: WIP: Support uploading Swift symlinks without content. https://review.opendev.org/694211 | 05:10 |
| *** jistr has quit IRC | 05:51 | |
| *** jistr has joined #openstack-swift | 05:56 | |
| *** nanzha has quit IRC | 06:00 | |
| *** jistr has quit IRC | 06:06 | |
| *** jistr has joined #openstack-swift | 06:06 | |
| *** nanzha has joined #openstack-swift | 06:12 | |
| *** nanzha has quit IRC | 06:24 | |
| *** nanzha has joined #openstack-swift | 06:36 | |
| *** rcernin has quit IRC | 07:09 | |
| *** nanzha has quit IRC | 07:22 | |
| *** joeljwright has joined #openstack-swift | 07:45 | |
| *** ChanServ sets mode: +v joeljwright | 07:45 | |
| *** nanzha has joined #openstack-swift | 08:00 | |
| *** tkajinam has quit IRC | 08:06 | |
| *** tesseract has joined #openstack-swift | 08:16 | |
| *** ccamacho has joined #openstack-swift | 08:25 | |
| *** ccamacho has quit IRC | 08:25 | |
| *** ccamacho has joined #openstack-swift | 08:25 | |
| *** rpittau|afk is now known as rpittau | 08:33 | |
| *** rdejoux has joined #openstack-swift | 08:49 | |
| *** nanzha has quit IRC | 09:00 | |
| *** nanzha has joined #openstack-swift | 09:01 | |
| *** noonedeadpunk is now known as noonedeadpunk_ | 09:56 | |
| *** noonedeadpunk_ is now known as noonedeadpunk | 09:56 | |
| *** jistr is now known as jistr|afk | 09:58 | |
| *** takamatsu has quit IRC | 10:05 | |
| *** takamatsu has joined #openstack-swift | 10:06 | |
| *** jistr|afk is now known as jistr | 10:22 | |
| *** nanzha has quit IRC | 10:34 | |
| *** nanzha has joined #openstack-swift | 10:38 | |
| *** nanzha has quit IRC | 11:06 | |
| *** nanzha has joined #openstack-swift | 11:06 | |
| *** pcaruana has joined #openstack-swift | 11:14 | |
| *** openstack has joined #openstack-swift | 11:53 | |
| *** ChanServ sets mode: +o openstack | 11:53 | |
| *** irclogbot_3 has joined #openstack-swift | 11:53 | |
| *** nanzha has quit IRC | 12:03 | |
| *** nanzha has joined #openstack-swift | 12:04 | |
| *** nanzha has quit IRC | 12:18 | |
| *** nanzha has joined #openstack-swift | 12:29 | |
| *** nanzha has quit IRC | 12:36 | |
| *** nanzha has joined #openstack-swift | 12:37 | |
| *** nanzha has quit IRC | 13:12 | |
| *** zaitcev_ has joined #openstack-swift | 13:14 | |
| *** ChanServ sets mode: +v zaitcev_ | 13:14 | |
| *** nanzha has joined #openstack-swift | 13:17 | |
| *** zaitcev__ has quit IRC | 13:18 | |
| *** mvkr has quit IRC | 13:19 | |
| *** nanzha has quit IRC | 13:24 | |
| *** nanzha has joined #openstack-swift | 13:30 | |
| *** nanzha has quit IRC | 13:56 | |
| *** nanzha has joined #openstack-swift | 13:58 | |
| *** diablo_rojo__ has joined #openstack-swift | 14:40 | |
| *** Crisp has joined #openstack-swift | 14:47 | |
| *** Crisp has quit IRC | 14:47 | |
| *** diablo_rojo__ has quit IRC | 14:57 | |
| *** diablo_rojo__ has joined #openstack-swift | 14:57 | |
| *** mvkr has joined #openstack-swift | 15:06 | |
| *** diablo_rojo__ is now known as diablo_rojo | 15:31 | |
| *** openstackstatus has joined #openstack-swift | 16:15 | |
| *** ChanServ sets mode: +v openstackstatus | 16:15 | |
| *** nanzha has quit IRC | 16:18 | |
| *** rpittau is now known as rpittau|afk | 16:40 | |
| timburke | good morning | 16:42 |
| *** takamatsu has quit IRC | 17:27 | |
| *** FlorianFa has quit IRC | 17:29 | |
| *** rdejoux has quit IRC | 17:30 | |
| clayg | timburke: I think we talked about this -> https://bugs.launchpad.net/swift/+bug/1853884 | 17:31 |
| openstack | Launchpad bug 1853884 in OpenStack Object Storage (swift) "s3api doesn't log internal requests" [Undecided,New] | 17:31 |
| clayg | I verified SwiftStack clusters already log s3api subrequests by default 👍 | 17:31 |
| *** takamatsu has joined #openstack-swift | 17:32 | |
| timburke | yeah, i like that description at the end a *whole* lot better -- that's the option we *really* want | 17:34 |
| timburke | i should check what happens when you've got `s3_acl` disabled... i think maybe it just does the right thing? | 17:35 |
| clayg | timburke: yeah so https://review.opendev.org/#/c/673682/25/test/unit/common/middleware/s3api/test_obj.py@1272 - that's a great question | 17:45 |
| patchbot | patch 673682 - swift - s3api: Implement object versioning API - 25 patch sets | 17:45 |
| clayg | instead of trying to DELETE an *object* that doesn't exist - we try to DELETE a *version* that doesn't exist? | 17:45 |
| clayg | the problem I was debugging is the shortcircut of the HEAD 404 was preventing the DELETE request from making its way to swift object versioning middleware | 17:46 |
| clayg | as the code reads now s3api is doing a HEAD obj?version-id=XXX request - and if that 404s we... | 17:46 |
| clayg | still do the DELETE obj?version-id=XXX request? I'm assuming swift object versioning would do the right thing, but maybe we want the old short circut logic? | 17:47 |
| clayg | it's not like we need to generate a delete marker in this case | 17:47 |
| timburke | right -- i was mainly thinking about making sure that delete-with-version-id works correctly when the object doesn't exist in the primary container | 17:51 |
| timburke | pretty trivial to do with a func test, too -- write obj, version-unaware delete it, version-aware delete the data | 17:53 |
| *** tesseract has quit IRC | 17:57 | |
| clayg | oh, well - for a version aware DELETE we don't really make a request to the un-versoined symlink thing... SHOULD WE?! | 18:05 |
| timburke | ...then how do we know whether or not we should lay down a tombstone? | 18:06 |
| clayg | you mean how does swift object versioning know? s3api just looks at the swift object versioning response | 18:06 |
| clayg | and we don't emulate the backing swift object versioning subreuqests | 18:06 |
| clayg | ... in these unittests | 18:07 |
| timburke | hmm... maybe i was thinking of a version-aware delete for a *version* that doesn't exist... i forget now... :-/ | 18:09 |
| *** diablo_rojo has quit IRC | 18:09 | |
| timburke | yeah yeah! in case we find ourselves with a broken symlink -- so we still show the version in the listing -- if the client's just trying to clean up everything, will they be able to do so with a version-aware DELETE for the busted version? significantly, what will the result of that head be at https://review.opendev.org/#/c/673682/24..25/swift/common/middleware/s3api/controllers/obj.py ? | 18:12 |
| patchbot | patch 673682 - swift - s3api: Implement object versioning API - 25 patch sets | 18:12 |
| timburke | i'm pretty sureit all works fine -- but i'm not sure that we have unit tests to explicitly test it | 18:13 |
| *** diablo_rojo has joined #openstack-swift | 18:14 | |
| clayg | so that HEAD request there, when you're doing DELETE?versionId=XXX goes to HEAD?version-id=XXX | 18:18 |
| timburke | which will try to follow the symlink and 404 | 18:19 |
| timburke | raising the NoSuchKey error, clearing the query param -- yeah? | 18:19 |
| clayg | no, I think a HEAD?version-id will just goto the data in the versions container, which... I mean it COULD 404 if you tried to delete a version that doesn't exist? | 18:19 |
| clayg | I am definately not following you, because I don't see how the symlink is involved on a DELETE?versionId request (from s3api's perspective) | 18:20 |
| clayg | if the question is how is swift object versioning handling a DELETE?version-id request when the is_latest symlink 404s... that's probalby just a different set of unittests | 18:21 |
| timburke | so i guess i've got two worries: first, the dangling symlink problem (which should hopefully get cleaned up), and the delete for a non-existent version (where the symlink should get left alone) | 18:21 |
| timburke | yeah, i suppose so | 18:21 |
| clayg | maybe that's why I'm confused - these unittests are for s3api, but yeah as soon as I address your comments - the func tests will be AWESOME! | 18:22 |
| clayg | timburke: do you have any fixes in your tree for the comments left on the latest null namespace patch? | 18:24 |
| timburke | not yet -- push away | 18:25 |
| clayg | ok, i'm pushing ontop of 543e2a (patch set 34) so none of your comments will be disturbed | 18:26 |
| timburke | either way | 18:26 |
| openstackgerrit | Clay Gerrard proposed openstack/swift master: s3api: Implement object versioning API https://review.opendev.org/673682 | 18:26 |
| openstackgerrit | Clay Gerrard proposed openstack/swift master: New Object Versioning mode https://review.opendev.org/695966 | 18:26 |
| clayg | timburke: well it looks like a number of the issues you pointed at on the null namepace patch aren't directly related to the reaper issue you're testing | 18:32 |
| clayg | I'll just keep them in mind for now as I really want to work on some more s3api func tests - but at somepoint I'd be happy to go back to the null namespace patch and try and tighten up some stuff | 18:33 |
| timburke | 👍 | 18:34 |
| openstackgerrit | Tim Burke proposed openstack/swift master: s3api: Fix prefix/delimiter/marker quoting https://review.opendev.org/695781 | 18:34 |
| timburke | speaking of s3, i *think* that guy should actually pass tests now | 18:34 |
| openstackgerrit | Tim Burke proposed openstack/swift master: s3api: Fix prefix/delimiter/marker quoting https://review.opendev.org/695781 | 18:56 |
| timburke | hmm... should direct_client always set X-Backend-Allow-Reserved-Names? i can't think of a good reason *not* to off hand... | 19:02 |
| clayg | well, i was thinking about listings | 19:04 |
| clayg | but i guess now that we've dis-allowed mixing null and non-null names 🤔 | 19:05 |
| clayg | well... only in containers | 19:06 |
| clayg | timburke: you pointed out that doing that prevents us from adding null names (container or obj) to the expirer queue | 19:07 |
| clayg | that may be a mistake | 19:07 |
| clayg | or rather "that might be something we decide to roll back at some point" (where "at some point" might be like... now) | 19:07 |
| timburke | yeah -- the expirer/reconciler implications of that are a little troubling | 19:08 |
| clayg | i think originally we'd thought of that - and then somehow later forgot (but it's good you remembed!) | 19:09 |
| timburke | i suppose the other option is to revisit https://review.opendev.org/#/c/517389/ sooner rather than later... and rework the interfaces proposed in https://review.opendev.org/#/c/517389/46/swift/common/task_queue.py | 19:12 |
| patchbot | patch 517389 - swift - Add object-expirer new mode to execute tasks from ... - 46 patch sets | 19:12 |
| patchbot | patch 517389 - swift - Add object-expirer new mode to execute tasks from ... - 46 patch sets | 19:12 |
| clayg | i don't see how that's feasible in our timeframe - or even really how it helps | 19:15 |
| clayg | ... but I'd be happy to rollback the "no user names in null containers" restriction | 19:15 |
| timburke | it'd mostly be about accepting as forced-work changing our naming convention for work queues | 19:16 |
| clayg | Is it possible we’re clever enough that a HEAD?version-id request to a delete marker would 404? I’d that... “correct”? | 19:47 |
| timburke | i think i did something for that... | 19:48 |
| timburke | look for "Well, except for some delete marker business..." -- i think it maybe needs a Content-Location header, though... | 19:49 |
| *** diablo_rojo has quit IRC | 20:11 | |
| timburke | ok, so i keep finding py3 rabbit-holes as i poke around... things that are logically separate from the reserved-namespace patch, but that i want to fix up as part of it... should i keep rolling those in, or should i get a new patch (or two, or...) ahead of everything? i *do* kinda want to backport some of this to train... | 20:25 |
| timburke | ok, so it looks like the answer for https://review.opendev.org/#/c/682138/34/swift/common/utils.py@190 is that otherwise, we hit a non-obvious circular import error, where constraints tries to load request_helpers (so it can get RESERVED_BYTE), which tries to load wsgi, which tries to load constraints but then complains that it doesn't have a MAX_HEADER_SIZE attribute... | 20:43 |
| patchbot | patch 682138 - swift - Allow internal clients to use reserved namespace - 34 patch sets | 20:43 |
| timburke | so flip side: is there any compelling reason not to have all the stuff currently in https://review.opendev.org/#/c/682138/34/swift/common/request_helpers.py over in utils instead? or possibly even constraints... | 20:43 |
| patchbot | patch 682138 - swift - Allow internal clients to use reserved namespace - 34 patch sets | 20:43 |
| clayg | no, none at all - that seemed like a great idea | 21:14 |
| timburke | clayg, so i think the expirer/reconciler problem is deeper than just allowing mixed user and reserved names... | 21:21 |
| clayg | neat! | 21:21 |
| timburke | the problem is that we get queue objects like <policy>:/<acct>/<cont>/<obj>, and if <cont> or <obj> are reserved names, the whole thing vails to validate because it doesn't start with \x00 | 21:23 |
| clayg | oh right | 21:23 |
| timburke | though... maybe it'll be ok if we just fix up the container layer to accept that format? like, the subdir problem shouldn't really apply to the reconciler... | 21:25 |
| clayg | true! | 21:26 |
| clayg | maybe limit to dot accounts | 21:27 |
| timburke | yeah, seems reasonable... | 21:27 |
| timburke | maybe we default-in the header if it's a system account? | 21:28 |
| clayg | we could probably also just move the whole thing get_reserved_name(policy, path) | 21:28 |
| timburke | yeah, i was thinking about that, too... | 21:28 |
| timburke | we get the problem of needing to look in multiple places, though | 21:29 |
| clayg | maybe we need policy, path = split_reserved_name(name, limit=1) | 21:29 |
| clayg | right, it'd be a little crufty - but i'd image we could get it to all work out ... the reconciler at least is pretty atomic | 21:30 |
| clayg | the expirer I guess has that property of removing entries when the expir name is updated 🤔 | 21:31 |
| *** rcernin has joined #openstack-swift | 21:31 | |
| clayg | ok, so maybe just letting the "invalid" names into the internal containers *is* better | 21:31 |
| *** biyiklioglu has joined #openstack-swift | 21:32 | |
| *** pcaruana has quit IRC | 21:38 | |
| *** biyiklioglu has quit IRC | 21:38 | |
| *** diablo_rojo has joined #openstack-swift | 21:53 | |
| timburke | probe tests aren't really opinionated about rsync vs ssync, yeah? i've got a pretty small change to add reserved names to test_replication_servers_working.py but idk that it's really worth running if we're using rsync in the gate (which i'm pretty sure we are) | 22:12 |
| timburke | still, good to see ssync working! | 22:12 |
| timburke | i'd just tweak the test_reconstructor_* tests, but they have a lot more places where they want to use swiftclient... | 22:13 |
| timburke | my delta: http://paste.openstack.org/show/786691/ | 22:14 |
| clayg | timburke: that's fabulous!!! | 22:22 |
| *** threestrands has joined #openstack-swift | 22:22 | |
| timburke | clayg, so... should i include it in the patches i'm about to push? or skip it since the gate'll just be using rsync? | 22:23 |
| timburke | also, i lied a little -- needed to add create_account to internal_client and set allow_account_management=true in probe/common | 22:24 |
| clayg | your question makes me think I don't understand some implication of the change 🤔 | 22:25 |
| clayg | even if we're not able to hit that functionality in the gate it's still useful to have it *available* right? what's the down side? | 22:25 |
| clayg | the gate will only test what it's configired to test right? | 22:26 |
| clayg | and we want null namespace to work with rsync too! 😁 | 22:26 |
| clayg | i'm also ready to push up some tests & fixes for delete markers | 22:26 |
| timburke | it doesn't have much implication -- i'd just *expect* that rsync should work, since it's just looking at fs paths, none of which involve reserved chars. so we'd be running an extra test, taking extra time, and i would expect it to catch pretty much *exactly* the same problems as the normal test (at least, when run in the gate) | 22:27 |
| timburke | eh, i'll go ahead and include it | 22:28 |
| timburke | i'm realizing that even if the reserved namespace patch doesn't think about container-sync, the versioning patch may have to... | 22:29 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Allow internal clients to use reserved namespace https://review.opendev.org/682138 | 22:30 |
| mattoliverau | morning | 22:30 |
| timburke | o/ | 22:30 |
| timburke | clayg, should UPDATE do any per-item name validation? | 22:31 |
| timburke | maybe that's a way out of our work-queue problem... it certainly doesn't do any such validation *today* | 22:32 |
| *** zaitcev_ is now known as zaitcev | 22:41 | |
| zaitcev | clayg, do you happen to have a vacuum script laying around by chance? I want a script that 1) locks the directory of the container, 2) runs VACUUM, 3) increments some timestamp or other so that replication does not overwrites it with raw rsync, 4) unlocks. | 22:42 |
| zaitcev | clayg, I think you're the only guy around who may have something like that in his cupboard. | 22:43 |
| mattoliverau | I always wondered if we should run a vacuum (or have an option too) when we're about to rsync a db accross. Cause then 1) it'll be less data and 2) would add some auto vacuuming.. sometimes (when rings change) | 22:45 |
| timburke | mattoliverau, i was just reading your response on https://bugs.launchpad.net/swift/+bug/1691648 :-) | 22:46 |
| openstack | Launchpad bug 1691648 in OpenStack Object Storage (swift) "empty 5GB container DB" [Medium,Confirmed] | 22:46 |
| mattoliverau | told you I always wondered ;) | 22:46 |
| zaitcev | 5GB haha.... (bitter laught) | 22:47 |
| zaitcev | I have a customer who have a 300GB container | 22:48 |
| mattoliverau | lol | 22:48 |
| zaitcev | I'm pretty desperate at this point. | 22:49 |
| zaitcev | It's probably not at all empty. | 22:52 |
| mattoliverau | hmm, what rsyncs are you worried about. if the db doesn't exist, it'll be rsynced.. if it does but has a "large" disparity then it'll rsync_then_merge which rsyncs to a new location. then merge and a rename/mv | 22:57 |
| mattoliverau | so not sure what timestamp to change. | 22:57 |
| timburke | i forget which db (local or remote) is used as the base for the result in rsync_then_merge -- i think it might be the remote, with the assumption that the local ought to be small | 22:58 |
| timburke | so that'd be what i'd worry about | 22:58 |
| mattoliverau | oh I thought it was the opposite. | 23:00 |
| mattoliverau | so it stayed out of the way. the copy send over is written in tmp. then the existing is merged into then renamed to replace the orig. | 23:01 |
| mattoliverau | but that's from memory, that might be wrong. | 23:01 |
| mattoliverau | https://www.sqlite.org/lang_vacuum.html interesting a VACUUM is a write, so should block other writes. So holds a write lock. Maybe if one is done at a time then it's fine. | 23:02 |
| mattoliverau | tho also, it copies rows to another temp db first.. so you might need up to twice as much space to run a vacuum.. that could be a problem. | 23:02 |
| mattoliverau | auto-vacuum seems to be a little different. | 23:03 |
| timburke | yep, looks like the remote is used as the base: https://github.com/openstack/swift/blob/2.23.0/swift/common/db_replicator.py#L987-L1008 | 23:03 |
| mattoliverau | ok yeah the remote.. and I just reread and realised what I said wasn't the opposite, but the same.. lol. /me reads good :P | 23:04 |
| *** tkajinam has joined #openstack-swift | 23:09 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!