| *** rdaly2 has quit IRC | 00:04 | |
| *** fandi has quit IRC | 00:17 | |
| *** Masahiro has joined #openstack-swift | 00:18 | |
| *** Masahiro has quit IRC | 00:22 | |
| *** sungju has joined #openstack-swift | 00:30 | |
| *** bill_az has joined #openstack-swift | 00:30 | |
| *** ho has joined #openstack-swift | 00:50 | |
| ho | good morning! | 00:57 |
|---|---|---|
| *** Masahiro has joined #openstack-swift | 01:01 | |
| *** addnull has joined #openstack-swift | 01:28 | |
| *** masonhsiung has joined #openstack-swift | 01:36 | |
| *** rdaly2 has joined #openstack-swift | 02:01 | |
| openstackgerrit | Yuan Zhou proposed openstack/swift: Fixes versioning SLO objects https://review.openstack.org/123765 | 02:01 |
| *** rdaly2 has quit IRC | 02:05 | |
| mattoliverau | ho: morning (sorry was at lunch when you came online) | 02:06 |
| ho | mattoliverau: morning! | 02:07 |
| *** nosnos has joined #openstack-swift | 02:10 | |
| ho | mattoliverau: I was thinking about why KeystoneAuth doesn't support multiple reseller admins in the configuration file. | 02:15 |
| ho | mattoliverau: I got a question about it but I can not find any reason why we don't support it. | 02:15 |
| ho | mattoliverau: I think it is better to allow it's configuration such as operator roles. What do you think? | 02:15 |
| *** oomichi has joined #openstack-swift | 02:23 | |
| mattoliverau | ho: according to the documentation, it does: Users with the keystone role defined in 'reseller_admin_role' will be reseller admins. Sure this looks like it's one role, can any user you add to it will be reseller admins: https://github.com/openstack/swift/blob/master/doc/source/overview_auth.rst#access-control-using-keystoneauth | 02:27 |
| ho | mattoliverau: Thanks for the reference. Users can specify an account for reseller_admin_role. any account... but not allow to specify multiple accounts. | 02:35 |
| mattoliverau | Can specify a role (group) that a user must be in to be a reseller admin. That is my understanding. | 02:36 |
| mattoliverau | I'll need to look at the code, but I think you can only specify 1 | 02:36 |
| ho | mattoliverau: I read the code. only allow to specify one role. | 02:37 |
| *** haomaiwa_ has joined #openstack-swift | 02:38 | |
| mattoliverau | ho: yeah looks like it :) So yeah, you specify a role/group and any user who is apart of it will be reselleradmins, so you can have many reseller admins, but they are defined by being members of 1 certian role in keystone. | 02:39 |
| ho | mattoliverau: Thanks for double checking! I may propse a patch for this. Keystone extends it's authentication function to policy base RBAC (many roles). But swift has three roles, reseller, operator and others. | 02:42 |
| mattoliverau | ho: you might want to read up on the composite token spec.. as this I think has some revelence to many roles in keystone auth, here is a patch in flight that will update the existing composite spec: https://review.openstack.org/#/c/138771/ | 02:47 |
| mattoliverau | so you have the most uptodate reasoning of the spec. | 02:47 |
| ho | mattoliverau: thanks! I will read it first. | 02:49 |
| *** panbalag has joined #openstack-swift | 02:50 | |
| *** panbalag has left #openstack-swift | 02:53 | |
| *** rdaly2 has joined #openstack-swift | 03:24 | |
| *** Masahiro has quit IRC | 03:26 | |
| *** nosnos has quit IRC | 03:41 | |
| *** nosnos has joined #openstack-swift | 03:42 | |
| *** nosnos has quit IRC | 03:46 | |
| *** nosnos has joined #openstack-swift | 04:15 | |
| *** serverascode____ has quit IRC | 04:16 | |
| *** wer has quit IRC | 04:16 | |
| *** mitz has quit IRC | 04:16 | |
| *** mitz has joined #openstack-swift | 04:18 | |
| *** serverascode____ has joined #openstack-swift | 04:19 | |
| *** cebruns has quit IRC | 04:19 | |
| *** cebruns has joined #openstack-swift | 04:20 | |
| *** wer has joined #openstack-swift | 04:20 | |
| *** ppai has joined #openstack-swift | 04:27 | |
| *** Masahiro has joined #openstack-swift | 04:27 | |
| *** lpabon has joined #openstack-swift | 04:32 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:45 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:45 | |
| *** wer has quit IRC | 04:57 | |
| *** wer has joined #openstack-swift | 04:58 | |
| *** rdaly2_ has joined #openstack-swift | 05:03 | |
| *** ahonda has quit IRC | 05:05 | |
| *** hurricanerix has quit IRC | 05:05 | |
| *** ahonda has joined #openstack-swift | 05:05 | |
| *** hurricanerix has joined #openstack-swift | 05:05 | |
| *** rdaly2 has quit IRC | 05:06 | |
| *** xianghui has quit IRC | 05:06 | |
| *** xianghui has joined #openstack-swift | 05:06 | |
| *** jdaggett_ has joined #openstack-swift | 05:09 | |
| *** mlanner_ has joined #openstack-swift | 05:10 | |
| *** notmyname_ has joined #openstack-swift | 05:10 | |
| *** ChanServ sets mode: +v notmyname_ | 05:10 | |
| *** mtreinish has quit IRC | 05:11 | |
| *** mlanner has quit IRC | 05:11 | |
| *** jdaggett has quit IRC | 05:11 | |
| *** dvorkbjel has quit IRC | 05:11 | |
| *** notmyname has quit IRC | 05:11 | |
| *** omame has quit IRC | 05:11 | |
| *** omame has joined #openstack-swift | 05:11 | |
| *** dmsimard_away has quit IRC | 05:11 | |
| *** mlanner_ is now known as mlanner | 05:11 | |
| *** notmyname_ is now known as notmyname | 05:11 | |
| *** jdaggett_ is now known as jdaggett | 05:11 | |
| *** mtreinish has joined #openstack-swift | 05:12 | |
| *** dmsimard_away has joined #openstack-swift | 05:13 | |
| *** dvorkbjel has joined #openstack-swift | 05:13 | |
| *** dmsimard_away is now known as dmsimard | 05:13 | |
| *** lpabon has quit IRC | 05:21 | |
| *** kopparam has joined #openstack-swift | 05:44 | |
| *** oomichi has quit IRC | 05:59 | |
| *** addnull has quit IRC | 06:03 | |
| ho | mattoliverau: I read the spec. I thought it was better to support multiple reseller admins in proxy-server.conf. But from the affinity with OpenStack components, I think the Keystoneauth in swift should handle policy.json as RBAC in addition to (or instead of) the user interface of the spec (https://github.com/openstack/swift-specs/blob/master/specs/in_progress/service_token.rst). | 06:28 |
| *** rdaly2_ has quit IRC | 06:34 | |
| ho | mattoliverau: Until Keystoneauth supports hte policy.json as RBAC with the composite authorization, I would like to have a functionality to specify multiple reseller admins in proxy-server because as I mentioned before swift has only three roles and keystone can have more roles so swift needs to have an ability for flexibility of the mapping. | 06:36 |
| *** exploreshaifali has joined #openstack-swift | 06:37 | |
| *** bkopilov has joined #openstack-swift | 06:40 | |
| *** exploreshaifali has quit IRC | 06:46 | |
| *** ppai has quit IRC | 06:49 | |
| *** kopparam has quit IRC | 06:55 | |
| *** kopparam has joined #openstack-swift | 06:55 | |
| *** ttrumm has joined #openstack-swift | 06:57 | |
| *** ppai has joined #openstack-swift | 07:00 | |
| ho | mattoliverau: Thanks for the info. I commented above on https://review.openstack.org/#/c/138771/ | 07:02 |
| *** addnull has joined #openstack-swift | 07:02 | |
| *** ttrumm has quit IRC | 07:06 | |
| *** addnull has quit IRC | 07:17 | |
| *** ttrumm has joined #openstack-swift | 07:20 | |
| *** kopparam has quit IRC | 07:20 | |
| *** sungju has quit IRC | 07:29 | |
| *** k4n0 has joined #openstack-swift | 07:30 | |
| *** ttrumm has quit IRC | 07:31 | |
| *** rdaly2 has joined #openstack-swift | 07:35 | |
| *** addnull has joined #openstack-swift | 07:37 | |
| *** rdaly2 has quit IRC | 07:39 | |
| *** ttrumm has joined #openstack-swift | 07:47 | |
| *** ttrumm has quit IRC | 07:48 | |
| *** kopparam has joined #openstack-swift | 07:54 | |
| *** rledisez has joined #openstack-swift | 08:11 | |
| *** geaaru has joined #openstack-swift | 08:14 | |
| *** bill_az has quit IRC | 08:15 | |
| *** dorry has quit IRC | 08:23 | |
| *** addnull has quit IRC | 08:28 | |
| *** kopparam has quit IRC | 08:36 | |
| *** kopparam has joined #openstack-swift | 08:36 | |
| *** jordanP has joined #openstack-swift | 08:42 | |
| *** ttrumm has joined #openstack-swift | 08:56 | |
| *** jordan__ has joined #openstack-swift | 09:05 | |
| *** jordan__ has quit IRC | 09:05 | |
| *** jordanP has quit IRC | 09:08 | |
| *** exploreshaifali has joined #openstack-swift | 09:13 | |
| *** fandi has joined #openstack-swift | 09:15 | |
| *** kopparam has quit IRC | 09:32 | |
| *** addnull has joined #openstack-swift | 09:39 | |
| *** addnull has quit IRC | 09:43 | |
| *** jordanP has joined #openstack-swift | 09:54 | |
| *** aix has joined #openstack-swift | 10:10 | |
| *** nellysmitt has joined #openstack-swift | 10:12 | |
| *** rawat_vedams has joined #openstack-swift | 10:16 | |
| *** Masahiro has quit IRC | 10:21 | |
| *** kopparam has joined #openstack-swift | 10:21 | |
| *** Masahiro has joined #openstack-swift | 10:21 | |
| *** sungju has joined #openstack-swift | 10:23 | |
| *** sungju has quit IRC | 10:23 | |
| openstackgerrit | Nicolas Trangez proposed openstack/swift: Add test coverage for `splice` and `tee` failure scenarios https://review.openstack.org/143031 | 10:30 |
| *** SkyRocknRoll has quit IRC | 10:34 | |
| *** Masahiro has quit IRC | 10:37 | |
| *** addnull has joined #openstack-swift | 10:43 | |
| *** ppai has quit IRC | 10:46 | |
| *** SkyRocknRoll has joined #openstack-swift | 10:46 | |
| *** addnull has quit IRC | 10:53 | |
| *** ppai has joined #openstack-swift | 10:59 | |
| *** tristanC has quit IRC | 11:01 | |
| *** tristanC has joined #openstack-swift | 11:02 | |
| *** masonhsi_ has joined #openstack-swift | 11:05 | |
| *** tristanC has quit IRC | 11:06 | |
| *** addnull has joined #openstack-swift | 11:06 | |
| *** tristanC has joined #openstack-swift | 11:07 | |
| *** masonhsiung has quit IRC | 11:08 | |
| *** masonhsi_ has quit IRC | 11:09 | |
| *** nosnos has quit IRC | 11:13 | |
| *** nosnos has joined #openstack-swift | 11:13 | |
| *** nosnos has quit IRC | 11:17 | |
| *** Masahiro has joined #openstack-swift | 11:18 | |
| *** Masahiro has quit IRC | 11:20 | |
| *** ho has quit IRC | 11:20 | |
| *** addnull has quit IRC | 11:32 | |
| *** ppai has quit IRC | 11:37 | |
| *** haomaiwa_ has quit IRC | 11:41 | |
| *** haomaiwang has joined #openstack-swift | 11:41 | |
| *** haomaiwang has quit IRC | 11:46 | |
| *** ppai has joined #openstack-swift | 11:51 | |
| *** fifieldt__ has quit IRC | 12:02 | |
| *** fifieldt has joined #openstack-swift | 12:07 | |
| *** exploreshaifali has quit IRC | 12:37 | |
| *** kopparam has quit IRC | 12:57 | |
| *** infotection has quit IRC | 13:25 | |
| *** infotection has joined #openstack-swift | 13:28 | |
| *** Masahiro has joined #openstack-swift | 13:29 | |
| *** jokke__ is now known as jokke_ | 13:30 | |
| *** aswadr has joined #openstack-swift | 13:31 | |
| *** Masahiro_ has joined #openstack-swift | 13:32 | |
| *** Masahiro has quit IRC | 13:34 | |
| *** acoles_away is now known as acoles | 13:34 | |
| *** Masahiro has joined #openstack-swift | 13:35 | |
| *** Masahiro_ has quit IRC | 13:36 | |
| *** ppai has quit IRC | 13:40 | |
| *** Masahiro has quit IRC | 13:40 | |
| *** Masahiro has joined #openstack-swift | 13:41 | |
| *** Masahiro has quit IRC | 13:49 | |
| *** bill_az has joined #openstack-swift | 13:50 | |
| *** Masahiro has joined #openstack-swift | 13:54 | |
| *** Masahiro has quit IRC | 13:58 | |
| *** Masahiro has joined #openstack-swift | 13:58 | |
| *** Masahiro has quit IRC | 14:03 | |
| *** mahatic has joined #openstack-swift | 14:03 | |
| *** Masahiro has joined #openstack-swift | 14:03 | |
| openstackgerrit | Merged openstack/swift: EC: Allow tuning ec_object_segment_size per policy https://review.openstack.org/132389 | 14:07 |
| *** Masahiro has quit IRC | 14:12 | |
| *** Guest32161 has joined #openstack-swift | 14:15 | |
| *** Masahiro has joined #openstack-swift | 14:15 | |
| *** fandi has quit IRC | 14:16 | |
| *** fandi_ has joined #openstack-swift | 14:16 | |
| *** infotection has quit IRC | 14:18 | |
| *** Masahiro has quit IRC | 14:20 | |
| *** mahatic has quit IRC | 14:21 | |
| *** mahatic has joined #openstack-swift | 14:23 | |
| *** infotection has joined #openstack-swift | 14:23 | |
| *** ttrumm has quit IRC | 14:48 | |
| *** masonhsiung has joined #openstack-swift | 15:03 | |
| *** rdaly2 has joined #openstack-swift | 15:03 | |
| *** SkyRocknRoll has quit IRC | 15:11 | |
| *** Masahiro has joined #openstack-swift | 15:16 | |
| *** Masahiro has quit IRC | 15:21 | |
| *** masonhsiung has quit IRC | 15:21 | |
| *** SkyRocknRoll has joined #openstack-swift | 15:24 | |
| *** annegentle has joined #openstack-swift | 15:24 | |
| *** masonhsiung has joined #openstack-swift | 15:28 | |
| *** cutforth has joined #openstack-swift | 15:33 | |
| *** tdasilva has joined #openstack-swift | 15:43 | |
| *** EmilienM is now known as EmilienM|afk | 15:47 | |
| *** masonhsiung has quit IRC | 15:51 | |
| *** masonhsiung has joined #openstack-swift | 15:52 | |
| *** masonhsiung has quit IRC | 16:09 | |
| acoles | tdasilva: hi | 16:12 |
| tdasilva | acoles: hey! how are you? | 16:12 |
| acoles | tdasilva: good thanks, pretty quiet in the office here today | 16:12 |
| acoles | tdasilva: about manifest versions, i read your conversation with notmyname in scrollback ... | 16:13 |
| tdasilva | acoles: yeah, I guess people are just getting ready for the xmas/new year's break | 16:13 |
| tdasilva | sure...what do you think? | 16:13 |
| acoles | tdasilva: well, i'm curious what difference you perceive between a dlo and slo 'content' being significant wrt a version. if the dlo x-object-manifest header changes to point to another container/prefix, does that not constitute a change in same way as an slo manifest json body changing? | 16:15 |
| acoles | or am i missing something? (quite possible!) | 16:16 |
| tdasilva | acoles: no, you have a good point and I did think about that, but I was trying to go more for what *I thought* made more sense...I was trying not to change things up so much, so assuming the document had been written that way, I just assumed it meant for DLO | 16:18 |
| notmyname | good morning | 16:19 |
| tdasilva | so I assumed that people would just create a new obj with new header instead of changing headers | 16:19 |
| tdasilva | but like you said it is entirely possible | 16:19 |
| notmyname | hmm...versioning | 16:19 |
| tdasilva | lol | 16:19 |
| acoles | notmyname: morning ! | 16:19 |
| notmyname | I don't think there's a significant difference between versioning SLOs and DLOs (ie I disagree with that particular point that tdasilva made) | 16:20 |
| notmyname | however, getting versioning working with manifests, long term, seems interesting | 16:20 |
| notmyname | so if we fix DLO to what's documented and make progress on versioning SLOs (assuming we can do that without breaking old clients), then I think that's cool | 16:21 |
| mahatic | good morning | 16:21 |
| acoles | tdasilva: yeah, its interesting to try to figure the various use cases. i've just been thinking about if there was a fundamental reason to treat the differently. | 16:22 |
| acoles | s/the/them | 16:22 |
| tdasilva | notmyname: well, but I think what we are saying is that even the DLO doc would be wrong, so we would need to change that too, which BTW I'm ok with... | 16:22 |
| notmyname | acoles: IMO, there is not fundamental difference (and it makes our jobs easier if we keep it that way) | 16:23 |
| notmyname | tdasilva: how? doesn't it say "versions + manifest = no bueno"? | 16:23 |
| acoles | notmyname: yeah, thats what i have concluded, and i agree that if we can figure out how to move towards them being versioned then that is cool | 16:24 |
| tdasilva | notmyname: yeah, but you said that you disagree with my point, so I'm assuming you want to make it both (slo and dlo) to be able to be versioned, no? | 16:24 |
| notmyname | tdasilva: long term? or this week? ;-) | 16:24 |
| acoles | tdasilva: the doc is not specific to only DLO right? http://docs.openstack.org/developer/swift/api/object_versioning.html | 16:24 |
| notmyname | tdasilva: ya, long-term I'd prefer that they be the same | 16:24 |
| tdasilva | notmyname: mmm...well well | 16:24 |
| tdasilva | acoles: that doc is not, but I thought I had read a dlo doc saying the same, so I just assumed the obj. versioning doc was also referring to dlo, my mistake | 16:28 |
| tdasilva | notmyname: so, for now we have already merged that fix to dlo to not allow merging | 16:28 |
| tdasilva | notmyname: I could work in the obj. versioning middleware to allow versioning to both dlo and slo, does that sound like a good plan? | 16:29 |
| acoles | tdasilva: i'm ok with that patch landing, current behavior was broken (as in when deleting a manifest a non-manifest zero sized object gets put back in its place :( ) | 16:30 |
| notmyname | tdasilva: assuming it can be done without breaking old clients. that's my only concern. | 16:30 |
| notmyname | acoles: right. and also agree with the :-( | 16:30 |
| notmyname | tdasilva: obviously, that's not my _only_ concern, but, ...ya know.... | 16:31 |
| acoles | tdasilva: notmyname: so imho the right tactical thing was to merge that fix | 16:31 |
| notmyname | yes, absolutely | 16:31 |
| notmyname | fix the broken stuff first. then make it better | 16:32 |
| tdasilva | acoles, notmyname: and what to do about this patch: https://review.openstack.org/#/c/123765/ I am concerned I may have given Yuan different instructions | 16:32 |
| acoles | tdasilva: if you can work it into the middleware, great! if it proves complex then perhaps treat it separately, ie land the middleware then look at enabling manifest versioning after | 16:33 |
| acoles | tdasilva: :) yeah i read 123765 today | 16:34 |
| tdasilva | acoles: ok, I will consider that... | 16:34 |
| tdasilva | acoles: it might be easier to get it all done at once | 16:34 |
| notmyname | yuanzz: ^^ | 16:35 |
| acoles | tdasilva: re comment on 123765, seems like we can't think of a good reason why dlo and slo would be different | 16:36 |
| tdasilva | acoles: I understand...that's fine...just trying to think if we should change it to not allow it either for slo (for the short-term) | 16:37 |
| notmyname | tdasilva: +1 (ie fix first--get docs and code and expectations in sync) | 16:38 |
| tdasilva | acoles: or just put in the support now for both slo and dlo like yuanzz initally intended | 16:38 |
| acoles | tdasilva: thats what i am wondering too | 16:38 |
| tdasilva | acoles: then, what is missing on yuanzz patch is just to update the docs accordingly and make sure clients don't break as notmyname mentioned | 16:39 |
| acoles | tdasilva: oh, so 123765 enables for dlo too - the commit messsage says just slo. | 16:40 |
| tdasilva | acoles: well..it did in patch set 1, i believe | 16:41 |
| acoles | tdasilva: ok i will go look at it more closely | 16:42 |
| *** exploreshaifali has joined #openstack-swift | 16:42 | |
| acoles | tdasilva: i'll lok and ping you again tomorrow | 16:43 |
| tdasilva | acoles: ok, let me know...thanks for your help! | 16:44 |
| *** gyee has joined #openstack-swift | 16:56 | |
| *** openstack has joined #openstack-swift | 17:02 | |
| *** Masahiro has joined #openstack-swift | 17:05 | |
| *** Nadeem has joined #openstack-swift | 17:07 | |
| *** Masahiro has quit IRC | 17:10 | |
| *** annegentle has quit IRC | 17:11 | |
| *** SkyRocknRoll has quit IRC | 17:12 | |
| *** k4n0 has quit IRC | 17:13 | |
| *** gyee has quit IRC | 17:19 | |
| *** EmilienM|afk is now known as EmilienM | 17:19 | |
| *** masonhsiung has joined #openstack-swift | 17:20 | |
| *** masonhsiung has quit IRC | 17:24 | |
| *** geaaru has quit IRC | 17:34 | |
| *** gyee has joined #openstack-swift | 17:36 | |
| *** rledisez has quit IRC | 17:36 | |
| *** EmilienM is now known as EmilienM|afk | 17:42 | |
| *** nellysmitt has quit IRC | 17:43 | |
| *** pcaruana has quit IRC | 17:53 | |
| *** annegentle has joined #openstack-swift | 17:57 | |
| *** annegentle has quit IRC | 17:58 | |
| *** annegentle has joined #openstack-swift | 18:00 | |
| *** abhirc has joined #openstack-swift | 18:03 | |
| cutforth | notmyname: nmn, i've got a question about use cases of swift | 18:15 |
| *** acoles is now known as acoles_away | 18:16 | |
| cutforth | or anybody, for that matter... are there any swift clusters with sizes larger than 1PB? | 18:17 |
| notmyname | cutforth: many | 18:25 |
| notmyname | cutforth: what sort of use cases are you wondering about? or is it just for total capacity capability? | 18:27 |
| cutforth | notmyname: so more specifically, is there public info on these sizes? i'm being asked by a seagate director and i want to show evidence other than saying "trust me" they are really big | 18:27 |
| cutforth | i'm most interested in capacity. if they are also geo-replicated that would be nice to know also | 18:28 |
| notmyname | cutforth: I can't give details on swiftstack customers, other that what we've said before (yes, multiple 1PB+ customers). Time Warner is featured (today!) on the openstack site. http://superuser.openstack.org/articles/pass-the-mic-matt-haines-time-warner-cable Also GA Tech has talked about theirs http://superuser.openstack.org/articles/case-study-georgia-tech-university | 18:30 |
| notmyname | cutforth: enovance has talked about cloudwatt (20+PB). Rackspace has talked about theirs (probably ~100PB now) | 18:31 |
| notmyname | cutforth: hubic (EU dropbox app) run by OVH has ~3PB. also OVH runs runabove (public swift service provider) | 18:32 |
| notmyname | cutforth: HP has many PB, but they've been cagy about the amount | 18:32 |
| notmyname | NTT always has interesting presentations and ideas on global clusters. I don't know their size, but being NTT I'm expecting "not tiny" | 18:33 |
| notmyname | cutforth: there was an edu in Canada that has mentioned 1PB+ storage in swift for their library system | 18:34 |
| peluse | there's also mercado libre, in their HK preso they state 1.2PB | 18:34 |
| peluse | see https://www.openstack.org/summit/portland-2013/session-videos/presentation/openstack-swift-mercadolibre-case-study | 18:34 |
| notmyname | ^ also important since it runs mission critical data. (ie the pictures for an auction site. can't make money without those) | 18:35 |
| ctennis | does softlayer have stats on size? | 18:35 |
| notmyname | cutforth: not that I've seen (publicly) | 18:35 |
| cutforth | peluse: I saw that mercado listed 1.4 billion images, but i missed the size of 1.2PB. was that in the video or the slides? | 18:35 |
| cutforth | notmyname: thanks, i'll poke around for these. is wikipedia swiftstack, or just swift. any info on it? | 18:36 |
| peluse | yeah, I think that was the wrong link, this one. I just saw it, I'll tell you the marker here in a sec http://www.confreaks.com/videos/4269-openstacksummithongkong2013-how-mercadolibre-stores-1-4-billion-images-on-openstack-object-storage | 18:36 |
| notmyname | cutforth: wikipedia is swift (not swiftstack). they have about 300 million images but less than 100TB | 18:36 |
| peluse | then go in to 11:19 | 18:37 |
| cutforth | notmyname: thx for the wikipedia info | 18:37 |
| cutforth | peluse: thanks for the link. i anxiously await a time marker :) | 18:37 |
| notmyname | cutforth: wikimedia is pretty public about what they are doing. you can get a lot of info from them (in general) | 18:37 |
| peluse | cutforth, its 11:19 | 18:37 |
| notmyname | cutforth: eg you can actually look at their monitoring pages | 18:37 |
| notmyname | cutforth: also, wikimedia has a global cluster, AFAIK | 18:38 |
| notmyname | cutforth: http://ganglia.wikimedia.org/latest/ | 18:39 |
| cutforth | notmyname: thx | 18:40 |
| notmyname | at least, you can see 3 DCs in that ganglia page that have something with "swift" | 18:40 |
| cutforth | peluse: thx | 18:40 |
| cutforth | yup | 18:40 |
| notmyname | cutforth: https://www.openstack.org/summit/openstack-summit-hong-kong-2013/session-videos/presentation/an-intimate-look-at-running-openstack-swift-at-scale <-- RAX preso in HK | 18:41 |
| cutforth | notmyname: peluse: awesome, this should be a good start | 18:41 |
| notmyname | cutforth: what are you selling (or convincing)? | 18:42 |
| cutforth | notmyname: its one of our new principles. he's not new to data centers, but new to object storage. he asked for this info. admitedly i'm in the swift corner so i'm wanting to provide facts. | 18:44 |
| *** amandap_ is now known as amandap | 18:45 | |
| notmyname | cutforth: cool. let me know if there are other ways I can help there | 18:45 |
| cutforth | notmyname: will do, thanks again | 18:45 |
| *** Masahiro has joined #openstack-swift | 18:54 | |
| *** zaitcev has joined #openstack-swift | 18:55 | |
| *** ChanServ sets mode: +v zaitcev | 18:55 | |
| *** Masahiro has quit IRC | 18:58 | |
| *** bkopilov has quit IRC | 19:10 | |
| *** jordanP has quit IRC | 19:14 | |
| *** tab____ has joined #openstack-swift | 19:42 | |
| *** tab____ has quit IRC | 19:43 | |
| *** tab____ has joined #openstack-swift | 19:43 | |
| *** nellysmitt has joined #openstack-swift | 19:44 | |
| *** nellysmitt has quit IRC | 19:49 | |
| *** annegentle has quit IRC | 19:52 | |
| *** themadcanudist has joined #openstack-swift | 19:57 | |
| themadcanudist | notmyname: greetings! I wanted to follow up with you on that "issue" i found on my swift cluster last week. Throttling container-updater helped out. However, it lead me down a path of wondering why the updater was workign significantly harder on one node vs. the other and I discovered that there was about 100x more containers on one. I'm wondering why that may be the case and if there is a way to rebalance them? | 19:58 |
| themadcanudist | hey guys, anyone here familiar with swift nodes having a 100x difference in # of containers? basically a huge imbalance even though all the devices on all nodes are weighted equally? | 20:12 |
| *** annegentle has joined #openstack-swift | 20:13 | |
| *** annegentle has quit IRC | 20:21 | |
| *** silor has joined #openstack-swift | 20:21 | |
| *** exploreshaifali has quit IRC | 20:29 | |
| *** jwang__ has quit IRC | 20:30 | |
| *** EmilienM|afk is now known as EmilienM | 20:34 | |
| *** Masahiro has joined #openstack-swift | 20:42 | |
| *** annegentle has joined #openstack-swift | 20:43 | |
| *** annegentle has quit IRC | 20:44 | |
| *** Guest32161 is now known as annegentle | 20:44 | |
| *** Masahiro has quit IRC | 20:47 | |
| clayg | is cschwede working around today? or is anyone else looking at the ring warning patch? | 20:50 |
| swifterdarrell | themadcanudist: you could acquire and run the swift-ring-builder here https://review.openstack.org/#/c/140478 on your container.builder file? | 20:51 |
| themadcanudist | swifterdarrell: oh yeah, I definitely have. That's how I built them in the first place | 20:51 |
| themadcanudist | and have rebalanced over the years | 20:51 |
| swifterdarrell | themadcanudist: is there any tier (region, node, etc) whose aggregate weight underneath is unbalanced? | 20:52 |
| swifterdarrell | themadcanudist: you've run that patch? it's pretty recent... | 20:52 |
| themadcanudist | oh sorry | 20:52 |
| themadcanudist | i just meant swift-ring-builder | 20:52 |
| themadcanudist | no, i haven't run that. | 20:52 |
| swifterdarrell | themadcanudist: and also not committed into Swift yet | 20:52 |
| swifterdarrell | themadcanudist: have you balanced the rings with Swift 2.0+ yet? | 20:52 |
| swifterdarrell | themadcanudist: or maybe the cutoff is 2.2? | 20:53 |
| *** tdasilva has quit IRC | 20:53 | |
| themadcanudist | not yet | 20:53 |
| themadcanudist | still on an older version | 20:53 |
| swifterdarrell | themadcanudist: oh, then maybe it's not the issue that patch checks for (but still worth running it against your builder) | 20:54 |
| clayg | themadcanudist: does the one node have more total weight than the others (more disks, bigger disks, maybe the others have failed disks?) | 21:03 |
| themadcanudist | clayg: no | 21:03 |
| themadcanudist | the space consumption is identical | 21:03 |
| themadcanudist | the container-updater just scans 100x more directories | 21:03 |
| themadcanudist | weight across ALL devices is equal | 21:03 |
| clayg | themadcanudist: more directories or more databases? could just be this bug: https://review.openstack.org/#/c/138524/ | 21:04 |
| themadcanudist | clayg: THat's definitely what I think it is | 21:05 |
| themadcanudist | there are a lot of directories that are empty | 21:05 |
| themadcanudist | ie. no hashes/sqlite dbs | 21:05 |
| themadcanudist | but they're still crawled | 21:05 |
| *** aswadr has quit IRC | 21:06 | |
| notmyname | that one was backported. themadcanudist it should be possible to apply that pretty cleanly in your environment | 21:08 |
| themadcanudist | hrm, is it safe to clear these out manually? | 21:08 |
| themadcanudist | looking at commit | 21:08 |
| notmyname | here's the backport https://review.openstack.org/#/c/139255/ | 21:08 |
| themadcanudist | two lines of code | 21:09 |
| themadcanudist | yeah, this suggests it safe to clean up manually as well | 21:11 |
| themadcanudist | clayg: thank you so much! =D | 21:12 |
| mattoliverau | Morning | 21:13 |
| clayg | i geuess... it may have been me that wrote the bug in the first place - you should thank ctennis | 21:13 |
| themadcanudist | I'm thanking you for your exprtise in searching the bug database and your succesful find! | 21:13 |
| themadcanudist | =D | 21:13 |
| * clayg good for something | 21:13 | |
| *** Nadeem has quit IRC | 21:23 | |
| clayg | torgomatic: so i had a ring that was all undispersed 3:1:1 but when I "fixed" the over-weighted device so the ring was 2:1:1 - it should be solvable and I thought after a few runs of pretend/rebalance - things would work out, but it turns out the balance is perfect so the undispersed parts on the 2x node don't think they have anywhere to go (but that's only becuase some of the parts on the 1x nodes could double up on 2x node | 21:24 |
| themadcanudist | clayg: Any definitive comments about getting rid of these empty container dirs? | 21:32 |
| themadcanudist | doesn't seem to affect accounts | 21:32 |
| themadcanudist | or at least it's not a problem on my cluster | 21:32 |
| themadcanudist | ie. is there ANY risk here? | 21:33 |
| notmyname | themadcanudist: you're talking about partition and suffix directories, right? | 21:38 |
| themadcanudist | yeah | 21:38 |
| notmyname | themadcanudist: no risk | 21:39 |
| themadcanudist | * /srv/node/*/containers/{emtpy dir} | 21:39 |
| notmyname | themadcanudist: you might want to stop replication temporarily while you delete the empty dirs. ie you don't want to fight with the system if another server is trying to add the dirs back while you are deleting them | 21:41 |
| themadcanudist | right. I'd only move the dirs out of the way if they're older than 24 hours anyway | 21:41 |
| themadcanudist | but yeah | 21:41 |
| themadcanudist | point taken | 21:41 |
| themadcanudist | and will do | 21:42 |
| *** nellysmitt has joined #openstack-swift | 21:45 | |
| *** nellysmitt has quit IRC | 21:50 | |
| *** zaitcev has quit IRC | 21:50 | |
| *** nellysmitt has joined #openstack-swift | 22:01 | |
| *** cutforth has quit IRC | 22:03 | |
| *** nellysmitt has quit IRC | 22:08 | |
| *** mahatic has quit IRC | 22:22 | |
| *** Masahiro has joined #openstack-swift | 22:31 | |
| *** Masahiro has quit IRC | 22:36 | |
| *** EmilienM is now known as EmilienM|afk | 22:38 | |
| *** sungju has joined #openstack-swift | 22:56 | |
| *** sungju has left #openstack-swift | 23:02 | |
| *** tab____ has quit IRC | 23:41 | |
| *** dmsimard is now known as dmsimard_away | 23:44 | |
| *** dmsimard_away is now known as dmsimard | 23:44 | |
| *** rdaly2 has quit IRC | 23:53 | |
| clayg | does anyone have an intuative sense for the range of the "balance" float in the ring? | 23:57 |
| clayg | like before initial balance all devices say they are -100.0 "balanced" which looks like they have -100% of the parts they want (0/parts_wanted) | 23:58 |
| clayg | but then my ring says it's "balance" is "100.0" (?) | 23:58 |
| *** silor has quit IRC | 23:59 | |
| clayg | so then I rebalance and things mostly get cooled off - a few devices have 0.10 balance, some have -0.39 - i.e. a few points away from center, nothing to worry about - and my rin says it's "balance" is "0.39" (?) | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!