notmyname | that's a good question | 00:00 |
---|---|---|
notmyname | maybe | 00:00 |
notmyname | :-) | 00:00 |
*** haomaiwang has quit IRC | 00:01 | |
notmyname | one thought is to keep it really simple and just have account names | 00:01 |
notmyname | another idea is to collect more of the account stat info | 00:01 |
*** haomaiwang has joined #openstack-swift | 00:01 | |
*** trifon has quit IRC | 00:02 | |
notmyname | however, with more info, it will lead to more complexity later. eg if you store the bytes count you find for each DB, then people will want that to be reported. and then you'll have to figure out how to reconcile it when you get different answers from different replicas in the cluster | 00:02 |
notmyname | and by you I mean all of us | 00:02 |
openstackgerrit | venkatamahesh proposed openstack/swift: OpenStack typo https://review.openstack.org/243608 | 00:03 |
notmyname | on the other hand, although it will lead to more complexity, the other account info is useful. ie if you don't store/report it, then the first thing someone will do when they get a list of accounts is HEAD each of them to get that info | 00:03 |
then3rd | I have my logging back to normal now, there are a couple errors I've been seeing.. are these normal, or should I figure out what's causing it? "invalid user token - deferring reject downstream" and "Client disconnected on read (txn: ...)" | 00:03 |
openstackgerrit | Eran Rom proposed openstack/swift: Container-Sync to iterate only over synced containers https://review.openstack.org/205803 | 00:03 |
notmyname | Zyric: I think you should leave it to a simple list of accounts for now | 00:03 |
notmyname | Zyric: and absolutely the first line of the file you are producing should be the version of the file. this is a version 1. that way we can change it later if needed | 00:04 |
Zyric | notmyname: Cool I'll keep it simple for now. I suppose what is collected depends on need/intent. If you want to collect historical data or just know what exists right now. | 00:05 |
notmyname | then3rd: the first is normal-ish. it's keystone saying that it thinks the token probably isn't good, but the ACLs that are loaded later might in fact make the request ok (eg a publicly available object). it's just normal keystone logging | 00:05 |
notmyname | then3rd: the second is also normal, or at least expected. clients sometimes disconnect before finishing the whole download | 00:05 |
notmyname | then3rd: however, if you're seeing a lot of those, it might point to some other problem | 00:06 |
notmyname | Zyric: right. and we can add complexity later. it's really hard to remove it once it's added | 00:06 |
Zyric | notmyname: I should probably make a formal file somewhere that holds this account data then? Is there a good place for this in the system or should I direct it to temp? | 00:07 |
*** eranrom has quit IRC | 00:07 | |
notmyname | Zyric: I'm imagining that you will write a file on disk (not in /tmp) that will ahve the info. then part 2 is getting swift-recon to read it | 00:08 |
*** zhill has quit IRC | 00:10 | |
Zyric | notmyname: Me too. Swift-recon was my next investigation :) So far I quite like the watcher hooks, is it okay to progress in this direction and assume that this is a good direction for swift or should it be discussed a bit more? I know torgomatic's patch is still in review. | 00:11 |
then3rd | notmyname: Thanks, that's a simpler answer than I found anywhere. Are there some good first steps or tests that come to mind for the second one? I can't say I've ever seen a failed GET.. since it's streaming video I 'm not supprised to see so many disconnets. | 00:15 |
notmyname | oh, yeah for streaming video that would be very common, I'd think | 00:16 |
notmyname | really it's just a process of monitoring it and seeing how the rate changes over time | 00:16 |
then3rd | And judging by my logs/graphs, the errors come in bursts of no more than 5/minute, if that. | 00:16 |
notmyname | disconnects are logged as a 499 response code in the logs | 00:17 |
notmyname | then3rd: with anything in swift, the first thing is to grab that transaction id in the logs (the txn) and grep all the logs in the cluster for that. any log line associated with that request will have the same transaction id | 00:17 |
*** haomaiw__ has joined #openstack-swift | 00:29 | |
*** haomaiwang has quit IRC | 00:32 | |
*** gyee has joined #openstack-swift | 00:32 | |
*** Jeffrey4l has quit IRC | 00:40 | |
*** gyee has quit IRC | 00:45 | |
*** haomaiw__ has quit IRC | 01:01 | |
*** 5EXAAJA62 has joined #openstack-swift | 01:01 | |
*** badari has joined #openstack-swift | 01:07 | |
clayg | ^ mattoliverau that reminds me concurrent gets needs to watch out for lp bug #1409302 | 01:16 |
openstack | Launchpad bug 1409302 in OpenStack Object Storage (swift) "Use of GreenAsyncPile can lose txn_id logging" [Medium,Confirmed] https://launchpad.net/bugs/1409302 | 01:16 |
*** badari has quit IRC | 01:17 | |
*** badari has joined #openstack-swift | 01:54 | |
*** 5EXAAJA62 has quit IRC | 02:01 | |
*** haomaiwang has joined #openstack-swift | 02:01 | |
*** badari_ has joined #openstack-swift | 02:08 | |
*** badari has quit IRC | 02:12 | |
*** klrmn has quit IRC | 02:12 | |
*** changbl has quit IRC | 02:36 | |
*** haomaiwang has quit IRC | 02:43 | |
*** janonymous_ has joined #openstack-swift | 02:49 | |
*** changbl has joined #openstack-swift | 02:55 | |
*** dmorita has quit IRC | 02:58 | |
*** klrmn has joined #openstack-swift | 03:13 | |
mattoliverau | clayg: good call, I'll make a note to check we are :) | 03:15 |
*** haomaiwang has joined #openstack-swift | 03:16 | |
*** janonymous_ has quit IRC | 03:22 | |
*** klrmn has quit IRC | 03:49 | |
*** haomaiwang has quit IRC | 04:01 | |
*** haomaiwang has joined #openstack-swift | 04:01 | |
*** nadeem has joined #openstack-swift | 04:02 | |
*** breitz has quit IRC | 04:11 | |
*** proteusguy_ has quit IRC | 04:22 | |
*** proteusguy_ has joined #openstack-swift | 04:34 | |
*** klrmn has joined #openstack-swift | 04:46 | |
*** haomaiwang has quit IRC | 05:01 | |
*** haomaiwang has joined #openstack-swift | 05:01 | |
*** mac_ified has quit IRC | 05:08 | |
*** nadeem has quit IRC | 05:13 | |
*** changbl has quit IRC | 05:27 | |
*** klrmn has quit IRC | 05:29 | |
*** thumpba has quit IRC | 05:38 | |
*** changbl has joined #openstack-swift | 05:40 | |
*** haomaiwang has quit IRC | 06:01 | |
*** haomaiwang has joined #openstack-swift | 06:01 | |
*** trifon has joined #openstack-swift | 06:16 | |
*** trifon has quit IRC | 06:40 | |
*** badari_ has quit IRC | 06:51 | |
*** trifon has joined #openstack-swift | 06:56 | |
*** haomaiwang has quit IRC | 07:01 | |
*** haomaiwang has joined #openstack-swift | 07:01 | |
*** zaitcev has quit IRC | 07:09 | |
*** haomaiwang has quit IRC | 08:01 | |
*** haomaiwang has joined #openstack-swift | 08:01 | |
*** haomaiwang has quit IRC | 08:20 | |
*** links has joined #openstack-swift | 08:50 | |
*** haomaiwa_ has joined #openstack-swift | 08:51 | |
*** trifon has quit IRC | 08:55 | |
*** haomaiwa_ has quit IRC | 09:01 | |
*** haomaiwang has joined #openstack-swift | 09:01 | |
*** jordanP has joined #openstack-swift | 09:06 | |
*** trifon has joined #openstack-swift | 09:08 | |
*** janonymous_ has joined #openstack-swift | 09:25 | |
*** jordanP has quit IRC | 09:30 | |
*** links has quit IRC | 09:47 | |
*** haomaiwang has quit IRC | 10:01 | |
*** haomaiwang has joined #openstack-swift | 10:01 | |
*** haomaiwang has quit IRC | 11:01 | |
*** haomaiwang has joined #openstack-swift | 11:01 | |
*** jordanP has joined #openstack-swift | 11:52 | |
*** haomaiwang has quit IRC | 12:01 | |
*** haomaiwang has joined #openstack-swift | 12:01 | |
*** jordanP has quit IRC | 12:42 | |
*** haomaiwang has quit IRC | 13:01 | |
*** haomaiwang has joined #openstack-swift | 13:01 | |
*** haomaiwang has quit IRC | 14:01 | |
*** haomaiwang has joined #openstack-swift | 14:01 | |
*** cebruns has quit IRC | 14:20 | |
*** cebruns has joined #openstack-swift | 14:25 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 14:39 |
*** changbl has quit IRC | 14:40 | |
*** changbl has joined #openstack-swift | 14:59 | |
*** haomaiwang has quit IRC | 15:01 | |
*** haomaiwang has joined #openstack-swift | 15:01 | |
*** tsg_ has joined #openstack-swift | 15:07 | |
*** tsg has quit IRC | 15:07 | |
*** tsg has joined #openstack-swift | 15:10 | |
*** tsg_ has quit IRC | 15:10 | |
*** changbl has quit IRC | 15:31 | |
*** diogogmt has quit IRC | 15:33 | |
*** dslevin has joined #openstack-swift | 15:35 | |
*** dslevin has quit IRC | 15:41 | |
*** changbl has joined #openstack-swift | 15:44 | |
*** jordanP has joined #openstack-swift | 15:52 | |
*** mac_ified has joined #openstack-swift | 15:52 | |
*** haomaiwang has quit IRC | 16:01 | |
*** haomaiwang has joined #openstack-swift | 16:01 | |
*** jordanP has quit IRC | 16:14 | |
*** janonymous_ has quit IRC | 16:42 | |
*** silor has joined #openstack-swift | 16:51 | |
*** silor1 has joined #openstack-swift | 16:57 | |
*** silor has quit IRC | 16:58 | |
*** silor1 is now known as silor | 16:58 | |
*** haomaiwang has quit IRC | 17:01 | |
*** haomaiwang has joined #openstack-swift | 17:01 | |
*** janonymous_ has joined #openstack-swift | 17:01 | |
*** silor1 has joined #openstack-swift | 17:09 | |
*** silor has quit IRC | 17:11 | |
*** silor1 is now known as silor | 17:11 | |
openstackgerrit | Merged openstack/swift: Make ECDiskFile report all fragments found on disk https://review.openstack.org/231121 | 17:15 |
*** badari has joined #openstack-swift | 17:21 | |
*** badari_ has joined #openstack-swift | 17:30 | |
*** whydidyoustealmy has joined #openstack-swift | 17:31 | |
*** patchbot has quit IRC | 17:31 | |
*** patchbot` has joined #openstack-swift | 17:32 | |
*** scott_ has joined #openstack-swift | 17:32 | |
*** torgomatic has quit IRC | 17:32 | |
*** Guest30753 has quit IRC | 17:32 | |
*** cschwede has quit IRC | 17:32 | |
*** sc68cal has quit IRC | 17:32 | |
*** cschwede has joined #openstack-swift | 17:32 | |
*** cschwede has quit IRC | 17:32 | |
*** cschwede has joined #openstack-swift | 17:32 | |
*** torgomatic_ has joined #openstack-swift | 17:32 | |
*** scott_ is now known as Guest87258 | 17:32 | |
*** bwall has quit IRC | 17:32 | |
*** patchbot` is now known as patchbot | 17:32 | |
*** bwall has joined #openstack-swift | 17:32 | |
*** bwall has joined #openstack-swift | 17:32 | |
*** badari has quit IRC | 17:33 | |
*** barra204 has quit IRC | 17:34 | |
*** sc68cal has joined #openstack-swift | 17:34 | |
*** badari_ has quit IRC | 17:47 | |
*** nadeem has joined #openstack-swift | 17:55 | |
*** haomaiwang has quit IRC | 18:01 | |
*** haomaiwang has joined #openstack-swift | 18:01 | |
*** eranrom has joined #openstack-swift | 18:28 | |
*** nadeem has quit IRC | 18:45 | |
*** klrmn has joined #openstack-swift | 18:50 | |
*** haomaiwang has quit IRC | 19:01 | |
*** haomaiwang has joined #openstack-swift | 19:01 | |
*** tsg has quit IRC | 19:26 | |
*** mac_ified has quit IRC | 19:56 | |
*** jordanP has joined #openstack-swift | 19:58 | |
*** haomaiwang has quit IRC | 20:01 | |
*** haomaiwang has joined #openstack-swift | 20:01 | |
*** jordanP has quit IRC | 20:03 | |
*** nadeem has joined #openstack-swift | 20:06 | |
*** nadeem has quit IRC | 20:46 | |
*** haomaiwang has quit IRC | 21:01 | |
*** haomaiwang has joined #openstack-swift | 21:01 | |
*** esker has joined #openstack-swift | 21:12 | |
*** trifon has quit IRC | 21:14 | |
*** nadeem has joined #openstack-swift | 21:39 | |
*** esker has quit IRC | 21:50 | |
*** silor has quit IRC | 21:54 | |
*** esker has joined #openstack-swift | 21:57 | |
*** haomaiwang has quit IRC | 22:01 | |
*** haomaiwang has joined #openstack-swift | 22:01 | |
jrichli | notmyname: great infogram! | 22:03 |
*** nadeem has quit IRC | 22:20 | |
*** eranrom has quit IRC | 22:36 | |
*** nadeem has joined #openstack-swift | 22:54 | |
*** haomaiwang has quit IRC | 23:01 | |
*** haomaiwang has joined #openstack-swift | 23:01 | |
*** david-lyle has quit IRC | 23:44 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!