| *** mabrams is now known as mabrams|afk | 07:03 | |
| *** mabrams|afk is now known as mabrams | 07:04 | |
| *** diablo_rojo__ is now known as diablo_rojo | 09:02 | |
| diablo_rojo | timburke_, did you see my message in the scrollback from yesterday? | 09:03 |
|---|---|---|
| manuvakery1 | Hi Guys | 09:42 |
| manuvakery1 | aws s3api list-object-versions --bucket manu-test returns "An error occurred (InternalError) when calling the ListObjectVersions operation (reached max retries: 2): unexpected status code 412" | 09:42 |
| manuvakery1 | After looking into logs I found that the issue is happening only on a specific container disk. unmounting that disk fixed the issue . What could be the root cause | 09:42 |
| manuvakery1 | 1724969:Aug 3 09:31:18 swift-storage5 container-server: 10.0.0.23 - - [03/Aug/2021:\ 09:31:18 +0000] "GET /disk2/1168/b5d335382b91478e981d6f3c7b623b1/%00versions%00manu-test" 412 29 \ "GET /v1/b5d335382b91478e981d6f3c7b623b1/%00versions%00manu-test?reverse=&format=json&delimiter=&states=listing&prefix=&limit=1001&marker=" "tx99ab930b8c36435c9d361-0061090ce6" "proxy-server 11947" 0.0002 "-" \ 227639 - | 09:45 |
| opendevreview | Matthew Oliver proposed openstack/swift master: sharder: update shard storage_policy_index if roots changes https://review.opendev.org/c/openstack/swift/+/800748 | 10:42 |
| mattoliver | ^ haven't updated the probe test with that yet. But that allows the replicator to handle the shard spi migration if there is more then 1 policy detected, is a shard and before enqueuing any reconciler jobs. But also leave the sharder to do it too, so who ever gets there first (tho maybe I should just remove it from the sharder) :) | 10:44 |
| mattoliver | Anyway, wanted to get it off my computer and stop looking at it for the night. | 10:45 |
| clayg | does swift-object-info not open the .meta? | 15:24 |
| clayg | ok then https://github.com/openstack/swift/blob/master/swift/cli/info.py#L534 | 15:38 |
| clayg | but I guess you can ues it on a .meta - so maybe that's ok | 15:41 |
| manuvakery1 | Hi | 16:30 |
| manuvakery1 | aws s3api list-object-versions --bucket manu-test returns "An error occurred (InternalError) when calling the ListObjectVersions operation (reached max retries: 2): unexpected status code 412" | 16:30 |
| manuvakery1 | 3:12 PM | 16:30 |
| manuvakery1 | After looking into logs I found that the issue is happening only on a specific container disk. unmounting that disk fixed the issue . What could be the root cause | 16:30 |
| manuvakery1 | 3:15 PM | 16:30 |
| manuvakery1 | 1724969:Aug 3 09:31:18 swift-storage5 container-server: 10.0.0.23 - - [03/Aug/2021:\ 09:31:18 +0000] "GET /disk2/1168/b5d335382b91478e981d6f3c7b623b1/%00versions%00manu-test" 412 29 \ "GET /v1/b5d335382b91478e981d6f3c7b623b1/%00versions%00manu-test?reverse=&format=json&delimiter=&states=listing&prefix=&limit=1001&marker=" "tx99ab930b8c36435c9d361-0061090ce6" "proxy-server 11947" 0.0002 "-" \ 227639 - | 16:30 |
| manuvakery1 | I formatted the disk and mounted again but issue persists | 16:32 |
| timburke_ | manuvakery1, any chance the container server is still running an old version of swift? only place it should return a 412 is at https://github.com/openstack/swift/blob/2.28.0/swift/container/server.py#L885-L886, but that path looks valid for swift 2.24.0+ | 18:34 |
| timburke_ | i just noticed there's a downside to https://github.com/openstack/swift/commit/ade3b28 -- the pre-rsync REPLICATE call no longer creates the partition dir, causing the rsync to fail with something like "rsync: mkdir "/sdb3/objects-1/37" (in object_sdb3) failed: No such file or directory (2)" | 18:39 |
| timburke_ | since we made the attempt, though, the replicator will invalidate suffixes after the failure, though, which *will* create the partition dir -- so the next pass should work fine. it's a little added friction, though :-/ | 18:40 |
| opendevreview | Tim Burke proposed openstack/swift master: WIP: X-If-Delete-At failing probe test https://review.opendev.org/c/openstack/swift/+/803406 | 23:49 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!