| mattoliverau | peluse: well it gives me the timeout 1 second issue... so hopefully the cause is the same as the others | 00:00 |
|---|---|---|
| peluse | mattoliverau, see https://etherpad.openstack.org/p/swift_timeout_test_failure as well | 00:00 |
| peluse | cool | 00:00 |
| peluse | OK, I have to shuttle a kid... be back online in 45 min or so | 00:01 |
| mattoliverau | step 1: recreate, step 2: commence banging head agaist the wall, step N: solved. (NOTE: in unit test debugging I like to jump to banging head) | 00:02 |
| ho | good morining guys! | 00:02 |
| clayg | I think at some point I decided to start creating my stub timeouts with an integer because it looked better in the log lines - and that was a really bad idea :\ | 00:03 |
| clayg | I'm not sure why __init__ calls start instead of just __enter__ - very strange | 00:03 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: wip: ec reconstructor probe test https://review.openstack.org/164291 | 00:05 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Erasure Code Reconstructor https://review.openstack.org/131872 | 00:05 |
| clayg | that 'ought to do it | 00:05 |
| clayg | sorry I suck so much! | 00:05 |
| clayg | I should get a t-shirt with that on it | 00:05 |
| clayg | or maybe it should say "sorry clayg sucks so much" - and then I could just pass them out when i'm being a dofus/jerk | 00:06 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Fix ssync sender cleanup of reverted fragment files https://review.openstack.org/169052 | 00:06 |
| *** yuan has quit IRC | 00:08 | |
| mattoliverau | ho: morning | 00:09 |
| ho | mattoliverau: morning! | 00:23 |
| *** dmorita has joined #openstack-swift | 00:24 | |
| mattoliverau | Lol, infra have put Clippy the paperclip on review.o.o as an april fools.. if the rest of you can't see it then it must be local date based | 00:26 |
| clayg | oohhhhhh goodie i can't wait until tomorrow! | 00:35 |
| mattoliverau | clayg: congrats, patch 131872,52 has passed unit tests on gerrit! just tempest and grenade and it'll post on the change. | 00:36 |
| patchbot | mattoliverau: https://review.openstack.org/#/c/131872/ | 00:36 |
| mattoliverau | and the timeout issue I was investigating is solved :) | 00:36 |
| clayg | mattoliverau: yeah - protip don't make a Timestamp(<int>) unless you mean it! | 00:37 |
| mattoliverau | clayg: ok, noted ;) | 00:37 |
| *** panbalag has quit IRC | 00:41 | |
| peluse | nice clayg! | 00:46 |
| peluse | so what were we working on again before all this? :) | 00:47 |
| clayg | renaming the object-reconstrucutrorer to object-recoder | 00:50 |
| clayg | no wait.... no one was working on that :'( | 00:51 |
| peluse | oh yeah, OK how about I go fix the job cardinality | 00:53 |
| peluse | also, real quick, what was wrong with doing Timeout(5) for example? | 00:54 |
| clayg | when you create a Timeout like that (with an int instead of None) - it *starts* it on the eventlet hub | 00:56 |
| clayg | the only way to get it to *not* fire is to call .cancel - or use the with context manager to have it call .cancel for you | 00:56 |
| clayg | [00:05] + clayg | peluse: https://github.com/simplegeo/eventlet/blob/master/eventlet/timeout.py#L76 | 00:56 |
| peluse | ahh, so None won't actually raise a timeout... got it | 00:57 |
| *** panbalag has joined #openstack-swift | 01:15 | |
| *** ujjain has quit IRC | 01:17 | |
| *** annegentle has joined #openstack-swift | 01:18 | |
| *** haigang has joined #openstack-swift | 01:22 | |
| *** haomaiwang has quit IRC | 01:29 | |
| *** jrichli has joined #openstack-swift | 01:34 | |
| *** annegentle has quit IRC | 01:43 | |
| *** vinsh_ has joined #openstack-swift | 01:45 | |
| *** haigang has quit IRC | 01:48 | |
| *** vinsh has quit IRC | 01:48 | |
| mattoliverau | I'm off to find some lunch bbs | 01:48 |
| peluse | rock on | 01:50 |
| openstackgerrit | paul luse proposed openstack/swift: Erasure Code Reconstructor https://review.openstack.org/131872 | 01:54 |
| openstackgerrit | paul luse proposed openstack/swift: wip: ec reconstructor probe test https://review.openstack.org/164291 | 01:55 |
| *** haigang has joined #openstack-swift | 01:58 | |
| *** haomaiwang has joined #openstack-swift | 02:00 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 02:01 |
| *** panbalag has quit IRC | 02:08 | |
| openstackgerrit | paul luse proposed openstack/swift: Fix ssync sender cleanup of reverted fragment files https://review.openstack.org/169052 | 02:10 |
| clayg | peluse: i feel like there must be some missing tests that should have it the ts_to_fname block? | 02:11 |
| clayg | also - i don't think going with suffixes.keys() is really going to make build_jobs return the right number of jobs... | 02:11 |
| peluse | the only tet that would have caught it currently was probe and it did | 02:12 |
| peluse | there were no unit tests for any of the ssync changes for ecrecon | 02:12 |
| peluse | well, that wasn't what that change was for | 02:12 |
| peluse | I didn't address the job cardinality concern there, that change was because in the case of revert we were using the full suffixes dict and passing it to ssync_sender isntead of the list of suffixes | 02:14 |
| clayg | oic gotcha | 02:15 |
| peluse | wrt the ssync tests for ecrecon changes, I thought acoles was adding those on his list of todo items but if not there | 02:16 |
| peluse | 's only a few small gaps to close there | 02:16 |
| *** jogriffin has joined #openstack-swift | 02:17 | |
| peluse | wrt the job cardinality thing, I couldn't think of anything 'easy' there as was mentioned in your comment so figured before I started tearing shit up I'd better talk to you about exactly what you had in mind | 02:17 |
| *** annegentle has joined #openstack-swift | 02:19 | |
| clayg | acoles_away: i'm looking at get_options in storage_policy module - added for in-process func tests - and thinking that the EC storage policy doesn't return *all* of it's options | 02:22 |
| clayg | acoles_away: why did you add this method - in process functest based on on disk configs is such mis-feature - what needs to happen? | 02:23 |
| *** annegentle has quit IRC | 02:24 | |
| clayg | acoles_away: looks like the ECStoragePolicy class needs to update it's internal options into get_options | 02:25 |
| clayg | I wish only the one config parsing map needed to be maintained for all of this mapping | 02:25 |
| peluse | clayg, so let me know what you had in mind for the job building/processing deal. You agree that what we have now works right? It could just be a lot more efficient wrt who it talks to and how often | 02:31 |
| peluse | gotta bolt... catch ya tomorrow | 02:41 |
| *** tsg_ has quit IRC | 02:48 | |
| clayg | yeah I think it would work - but I would really rather it be better - more efficient is a bit of an understantement when you talking cardinality of # of suffixes to # of partitions - and we have a *lot* of partitions | 02:54 |
| clayg | that being said I'm not working on it atm | 02:55 |
| *** annegentle has joined #openstack-swift | 02:57 | |
| *** km has quit IRC | 03:00 | |
| openstackgerrit | Janie Richling proposed openstack/swift: WIP - Provides a simple skeleton of middleware for encryption feature. https://review.openstack.org/157907 | 03:08 |
| *** jrichli has quit IRC | 03:09 | |
| *** vinsh_ has quit IRC | 03:11 | |
| *** david-lyle has joined #openstack-swift | 03:13 | |
| *** tsg_ has joined #openstack-swift | 03:22 | |
| *** ppai has joined #openstack-swift | 03:29 | |
| peluse | clayg, OK sounds good. I have an idea, I'll stew on it more in the morning and run it by you then | 03:44 |
| *** yuan has joined #openstack-swift | 03:49 | |
| *** annegentle has quit IRC | 04:26 | |
| *** haigang has quit IRC | 04:38 | |
| *** haigang has joined #openstack-swift | 04:44 | |
| *** jogriffin has quit IRC | 04:46 | |
| *** haigang has quit IRC | 04:50 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Add support for policy types, 'erasure_coding' policy https://review.openstack.org/169614 | 04:53 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Initial Erasure Code Docs https://review.openstack.org/169615 | 04:53 |
| clayg | a'ight - let's get this party started off real smooth like | 04:53 |
| clayg | I'll be inserting patches inbetween the storage policy refactor (which I chagned a bunch so everyone can acctually get started with a for-realzy review :P) and the docs | 04:54 |
| *** vinsh has joined #openstack-swift | 04:54 | |
| *** vinsh has quit IRC | 04:55 | |
| clayg | acoles_away: also you'll need to tell me what needs to happen for in process func tests on https://review.openstack.org/169614 | 04:56 |
| *** SkyRocknRoll has joined #openstack-swift | 05:11 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:11 | |
| openstackgerrit | Matthew Oliver proposed openstack/swift: Update contianer sync to use internal client https://review.openstack.org/143791 | 05:15 |
| mattoliverau | clayg: ^^ tested cont sync, and works well between 2 saios, func tests between polices (EC and REPL) would be nice, but works in personal testing | 05:19 |
| mattoliverau | or where ever that testing woulb be | 05:21 |
| zaitcev | wait, it didn't use our own client before? | 05:21 |
| clayg | I didn't really *forget* how tedious and painful all the testing is with epic rebases like this - but I'd sorta surpressed it? | 05:22 |
| clayg | zaitcev: for the GET side of sync it would just make direct client calls | 05:23 |
| clayg | mattoliverau: does the probe tests for container sync not bother to test cross policy when it can? | 05:23 |
| mattoliverau | to be honest, i have really checked, will now | 05:24 |
| mattoliverau | clayg: it does.. kinda.. it if there is > 1 enabled policy, it randomly chooses one for the source container and randomly chooses again for the dest container | 05:28 |
| *** reed has quit IRC | 05:29 | |
| zaitcev | hmm, now that we have internal-client.conf, maybe object expirer should be made to use it? | 05:32 |
| clayg | mattoliverau: so it'll randomly sometimes test cross policy | 05:33 |
| clayg | mattoliverau: maybe that's good - i suppose you want to test same policy sometimes too | 05:33 |
| mattoliverau | clayg: yup, but adding a corss policy probe test now | 05:33 |
| clayg | zaitcev: I think *should* is maybe stronger than I'm willing to go for ;) | 05:34 |
| mattoliverau | if it has more then 1 type | 05:34 |
| clayg | *could* would be nice :) but doing that work means figuring out how to do the upgrades and backwards compatible - and we suck at deprecating shit - so basically it'll be a bunch of work to make it possible to not have to do it - but we'll still have to test and support it works FOREVAR | 05:34 |
| clayg | mattoliverau: i thought you said it was already cross-policy? | 05:35 |
| *** haigang has joined #openstack-swift | 05:35 | |
| *** haigang has quit IRC | 05:35 | |
| *** zaitcev has quit IRC | 05:35 | |
| mattoliverau | clayg: yeah, randomly selects between policies that aren't deprecated.. I can leave it if we think that is enough, but I'm playing with one that if there are more then 1 type (REPL, EC) then to do a cross policy test... if I do I can make it a dependant patch and can wait until post beta :) | 05:39 |
| clayg | mattoliverau: if you can verify it'll hit a repl->ec check after a few runs if you do it in a loop I think that's good enough | 05:40 |
| clayg | probetsts are hulla slow - i don't wanna add one unless I have to - and if I'm changing/reviewing something that messes with container sync I'll probably run probetests a number of times anyway if not in a loop (now that I know i wrote it like that) | 05:41 |
| clayg | in the more general case of "someone changed something on the backend run a "quick" test with probetests" - i think the randomness is fine | 05:42 |
| mattoliverau | kk, I run a loop and watch for a while then :P | 05:42 |
| clayg | if something did break cross policy (but accidently still worked on the same policy, and they got a false confidence from a lucky probe test run) - someone would notice it soon enough | 05:42 |
| clayg | it's not like probetests haven't been broken in the past | 05:42 |
| mattoliverau | lol | 05:43 |
| clayg | if we ever mangage to get them running on the community cluster it might be interesting to see if we can have a --quick or --soak option that lets the computers run more tests without the cost of wall time on my laptop | 05:43 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Initial Erasure Code Docs https://review.openstack.org/169615 | 05:44 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Update test infrastructor https://review.openstack.org/169620 | 05:44 |
| *** haigang has joined #openstack-swift | 05:45 | |
| *** thumpba has joined #openstack-swift | 05:53 | |
| mattoliverau | clayg: hmm.. interesting thought.. I wonder if I can fool RS into letting me build a community cluster in the cloud for testing.. or even a big beefy SAIO or 2 to test this stuff on each commit. :P (no promises but interesting thought) | 06:00 |
| *** haigang has quit IRC | 06:07 | |
| clayg | mattoliverau: yeah probetests are long way from being able to run on a multi-node setups - but that's the dream - right klrmn!? | 06:08 |
| *** haigang has joined #openstack-swift | 06:09 | |
| *** haigang has quit IRC | 06:10 | |
| *** silor has joined #openstack-swift | 06:11 | |
| *** haigang has joined #openstack-swift | 06:13 | |
| *** silor1 has joined #openstack-swift | 06:15 | |
| *** silor has quit IRC | 06:18 | |
| *** haigang has quit IRC | 06:23 | |
| *** haigang has joined #openstack-swift | 06:25 | |
| openstackgerrit | Christian Schwede proposed openstack/swift: Check if device name is valid when adding to the ring https://review.openstack.org/169231 | 06:31 |
| openstackgerrit | Christian Schwede proposed openstack/swift: Check if REST API version is valid https://review.openstack.org/168509 | 06:42 |
| *** vinsh has joined #openstack-swift | 06:43 | |
| *** vinsh has quit IRC | 06:48 | |
| mattoliverau | clayg: yay, once you have the saio setup correctly to not skip on container_sync during probe tests, it works and the test is really quick... in comparison. And does select the EC policy: http://paste.openstack.org/show/197754/ | 06:52 |
| mattoliverau | On that note, I'm going to go get started on cooking dinner before the wife gets home. | 06:53 |
| mattoliverau | cschwede: Morning (your new patchsets gave you away) :P | 06:53 |
| cschwede | mattoliverau: Good morning Matthew! | 06:54 |
| cschwede | yeah, hard to stay undercover if you want to get something done ;) | 06:54 |
| clayg | peluse: notmyname: I think we need to do a master merge to feature/ec or something? | 06:54 |
| mattoliverau | lol | 06:55 |
| *** blankspace has joined #openstack-swift | 07:01 | |
| *** jordanP has joined #openstack-swift | 07:09 | |
| ho | cschwede: morning! I understand your thought but to keep consistency of your thought I think it might be good to remve L1722(err = 0) in test_ringbuilder.py. what do you think? | 07:10 |
| *** tsg_ has quit IRC | 07:10 | |
| cschwede | ho: Good Morning! No, that err=0 is required to make sure the previous loop iteration is not evaluated again, which happens if the current loop doesn’t raise an exception | 07:12 |
| ho | cschwede: I see. your right! thanks for the explnation | 07:13 |
| cschwede | ho: you’re welcome! | 07:13 |
| ho | cschwede: initialized by None is more preferable? | 07:14 |
| *** jistr has joined #openstack-swift | 07:14 | |
| cschwede | ho: hmm, the SystemExit error code is always an int and the default is 0 - thus i thought it makes sense | 07:16 |
| ho | cschwede: OK, thanks! :) | 07:30 |
| *** geaaru has joined #openstack-swift | 07:37 | |
| clayg | so I think the last time I did this I wasn't quite as skillful with my use of git add -p's search feature | 07:37 |
| clayg | after I apply the diff from feature/ec to the review branch i can skip to just the hunk i want add it and then throw away the rest - test - apply - add -p - repeat | 07:38 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Add support for policy types, 'erasure_coding' policy https://review.openstack.org/169614 | 07:43 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Initial Erasure Code Docs https://review.openstack.org/169615 | 07:43 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Update test infrastructor https://review.openstack.org/169620 | 07:43 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Per-policy DiskFile classes https://review.openstack.org/169651 | 07:43 |
| clayg | ok, - so I'm really just starting to sketch things out here folks - don't get too excited | 07:44 |
| clayg | ... maybe I should be doing this on github or something :\ | 07:44 |
| clayg | hi clippy! | 07:45 |
| clayg | ok, but I'm going to call it quits for tonight anyway - talk to ya'll in the meeting tomorrow | 07:46 |
| clayg | mattoliverau: if you end up online anymore and want to help out with commit messages that'd be great! | 07:46 |
| clayg | mattoliverau: I'll probably end up trying to troll through feature/ec and pull at random snippets from the logs there | 07:47 |
| clayg | ahh that's bullshit - i just realized my git commit -a -C <somesha> trick isn't carrying over authorship information | 07:47 |
| clayg | I thought it did!? | 07:48 |
| clayg | ahhh what the hell - git log says it right but gerrit wants to blame everything on me :\ | 07:48 |
| mattoliverau | clayg: i'm around if you need anything, just cooking some dinner :) | 07:48 |
| clayg | nah i'm good and done - just thinking about splitting up all these changes - stuff like how to logically group things - and what the commits should say | 07:49 |
| clayg | talk to you tomorrow | 07:49 |
| *** thumpba has quit IRC | 07:54 | |
| *** acoles_away is now known as acoles | 08:05 | |
| acoles | good morning | 08:05 |
| openstackgerrit | Merged openstack/swift: Update contianer sync to use internal client https://review.openstack.org/143791 | 08:13 |
| ho | acoles: morning! | 08:21 |
| acoles | ho: hello! | 08:26 |
| *** vinsh has joined #openstack-swift | 08:32 | |
| *** vinsh has quit IRC | 08:37 | |
| *** ho has quit IRC | 08:43 | |
| *** kei_yama has quit IRC | 08:44 | |
| *** silor1 has quit IRC | 09:14 | |
| *** haomaiwang has quit IRC | 09:51 | |
| *** silor has joined #openstack-swift | 09:57 | |
| *** silor1 has joined #openstack-swift | 09:59 | |
| *** silor has quit IRC | 10:01 | |
| *** InAnimaTe has quit IRC | 10:20 | |
| *** jistr has quit IRC | 10:23 | |
| *** haigang has quit IRC | 10:24 | |
| *** jistr has joined #openstack-swift | 10:36 | |
| *** nellysmitt has joined #openstack-swift | 10:50 | |
| *** haigang has joined #openstack-swift | 10:54 | |
| *** krykowski has joined #openstack-swift | 11:00 | |
| *** dmorita has quit IRC | 11:06 | |
| *** haigang has quit IRC | 11:10 | |
| *** jistr is now known as jistr|demo | 11:11 | |
| *** ppai has quit IRC | 11:36 | |
| peluse | morning | 11:49 |
| tdasilva | morning | 11:53 |
| *** ppai has joined #openstack-swift | 12:02 | |
| *** emptyspace has joined #openstack-swift | 12:04 | |
| *** blankspace has quit IRC | 12:04 | |
| *** dencaval has joined #openstack-swift | 12:06 | |
| *** blankspace has joined #openstack-swift | 12:07 | |
| *** blankspace has quit IRC | 12:07 | |
| *** blankspace has joined #openstack-swift | 12:07 | |
| *** panbalag has joined #openstack-swift | 12:08 | |
| *** vinsh has joined #openstack-swift | 12:10 | |
| *** emptyspace has quit IRC | 12:10 | |
| openstackgerrit | paul luse proposed openstack/swift: Merge master to feature/ec https://review.openstack.org/169720 | 12:11 |
| peluse | clayg, ask and ye shall receive... | 12:12 |
| acoles | peluse: tdasilva morning | 12:12 |
| tdasilva | :-) | 12:13 |
| acoles | peluse: so did you guys figure out the timeout thing? | 12:13 |
| *** vinsh has quit IRC | 12:14 | |
| *** panbalag has quit IRC | 12:15 | |
| *** panbalag has joined #openstack-swift | 12:16 | |
| *** ppai has quit IRC | 12:24 | |
| peluse | acoles, yes, clay did | 12:29 |
| peluse | acoles, he was passing a timeout value in his mock calls with Timeout so setting real timeouts in the hub | 12:29 |
| peluse | acoles, thanks for the comments on the ECrecon BTW, just commented on them | 12:30 |
| *** rdaly2 has joined #openstack-swift | 12:33 | |
| *** khivin has quit IRC | 12:39 | |
| *** nexusz99 has joined #openstack-swift | 12:48 | |
| *** nexusz99 has quit IRC | 12:48 | |
| *** zul has quit IRC | 12:57 | |
| *** zul has joined #openstack-swift | 12:57 | |
| *** lpabon has joined #openstack-swift | 12:59 | |
| *** emptyspace has joined #openstack-swift | 13:00 | |
| acoles | peluse: cool, i was worrying about the commit() being the cause | 13:02 |
| *** blankspace has quit IRC | 13:04 | |
| *** annegentle has joined #openstack-swift | 13:05 | |
| *** emptyspace has quit IRC | 13:07 | |
| *** petertr7 has joined #openstack-swift | 13:11 | |
| *** SkyRocknRoll has quit IRC | 13:25 | |
| *** SkyRocknRoll has joined #openstack-swift | 13:25 | |
| *** lastops has joined #openstack-swift | 13:28 | |
| *** lastops has quit IRC | 13:28 | |
| -openstackstatus- NOTICE: gerrit has been restarted to restore event streaming. any change events missed by zuul (between 12:48 and 13:28 utc) will need to be rechecked or have new approval votes set | 13:29 | |
| *** haigang has joined #openstack-swift | 13:31 | |
| *** lastops has joined #openstack-swift | 13:32 | |
| *** jogriffin has joined #openstack-swift | 13:35 | |
| *** jogriffin has quit IRC | 13:41 | |
| *** nexusz99 has joined #openstack-swift | 13:45 | |
| *** jrichli has joined #openstack-swift | 13:46 | |
| *** vinsh has joined #openstack-swift | 13:49 | |
| *** haigang has quit IRC | 13:52 | |
| *** vinsh has quit IRC | 13:52 | |
| *** InAnimaTe has joined #openstack-swift | 13:53 | |
| *** SkyRocknRoll has quit IRC | 13:55 | |
| *** SkyRocknRoll has joined #openstack-swift | 13:56 | |
| *** tsg_ has joined #openstack-swift | 14:01 | |
| *** tsg_ has quit IRC | 14:06 | |
| *** annegentle has quit IRC | 14:09 | |
| *** zul has quit IRC | 14:10 | |
| *** zul has joined #openstack-swift | 14:11 | |
| *** zaitcev has joined #openstack-swift | 14:13 | |
| *** ChanServ sets mode: +v zaitcev | 14:13 | |
| openstackgerrit | Christian Schwede proposed openstack/swift: Check if REST API version is valid https://review.openstack.org/168509 | 14:20 |
| *** bkopilov has quit IRC | 14:27 | |
| *** vinsh has joined #openstack-swift | 14:28 | |
| *** bkopilov has joined #openstack-swift | 14:28 | |
| openstackgerrit | Christian Schwede proposed openstack/swift: Check if device name is valid when adding to the ring https://review.openstack.org/169231 | 14:34 |
| *** annegentle has joined #openstack-swift | 14:36 | |
| openstackgerrit | Merged openstack/swift: Merge master to feature/ec https://review.openstack.org/169720 | 14:41 |
| *** ujjain has joined #openstack-swift | 14:45 | |
| *** Bsony has joined #openstack-swift | 15:00 | |
| *** annegent_ has joined #openstack-swift | 15:02 | |
| *** nellysmi_ has joined #openstack-swift | 15:03 | |
| *** nellysmitt has quit IRC | 15:04 | |
| *** annegentle has quit IRC | 15:04 | |
| *** nellysmitt has joined #openstack-swift | 15:04 | |
| *** nellysmitt has quit IRC | 15:05 | |
| *** rdaly2 has quit IRC | 15:07 | |
| *** nellysmi_ has quit IRC | 15:08 | |
| *** rdaly2 has joined #openstack-swift | 15:10 | |
| *** Guest___ has joined #openstack-swift | 15:15 | |
| *** bkopilov has quit IRC | 15:27 | |
| *** jogriffin has joined #openstack-swift | 15:34 | |
| *** jogriffin has quit IRC | 15:34 | |
| *** reed has joined #openstack-swift | 15:35 | |
| clayg | peluse: yeah thanks that's just what I needed | 15:38 |
| clayg | peluse: thanks for leaving me that dangling elif in get_dev_path - it's really satisfiying to my OCD and that one was particularlly juciy | 15:42 |
| *** lpabon has quit IRC | 15:44 | |
| *** bkopilov has joined #openstack-swift | 15:46 | |
| *** bkopilov has quit IRC | 15:46 | |
| *** Guest54459 is now known as h_m | 15:49 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Fix ssync sender cleanup of reverted fragment files https://review.openstack.org/169052 | 15:50 |
| clayg | wow acoles is on the recoder this is awesomeon! | 15:50 |
| *** mahatic has joined #openstack-swift | 15:54 | |
| acoles | fixer | 15:54 |
| acoles | ;) | 15:54 |
| *** krykowski has quit IRC | 15:56 | |
| h_m | hi guys! newbie question : i'm trying to deploy swift using this doc http://docs.openstack.org/juno/install-guide/install/apt/content/ch_swift.html i have a proxy-server and two storage nodes, my question is ; if add a storage_device to a storage_node and after rebalancing the ring on the proxy-server, do i have to redistribute ring configuration files on the storage nodes? | 15:56 |
| h_m | as described here http://docs.openstack.org/juno/install-guide/install/apt/content/swift-initial-rings-distribute.html | 15:57 |
| clayg | acoles: peluse: the junky code at the bottom of the job_info method was trying to get rid of the None job when the list is like this [{'sync', part=0, frag_index=None}, {'sync', part=0, frag_index=1}] | 15:58 |
| clayg | acoles: peluse: in this case it's just not useful to run process_job on both "frag_indexes" - the None job isn't really a frag index - and all it's .ts updates will get pushed over when the nodes walks for the frag_index=1 anyway | 16:00 |
| *** Guest___ has quit IRC | 16:01 | |
| clayg | if len(jobs) > 2: jobs = filter(jobs, lambda j: j['frag_index'] == None) would have been better - i wonder why I didn't write that? | 16:01 |
| acoles | clayg: (making sure i understand) thats because frag_index=1 passed to yield_hashes will still yield all the .ts | 16:02 |
| clayg | acoles: yeah you got it bro! | 16:02 |
| acoles | clayg: so when do we do a frag_index=None job, just when there are NO .data files in the suffix? | 16:03 |
| clayg | acoles: RIGHT! it has to happen sometimes | 16:03 |
| clayg | acoles: the crazy part about that situation is we have no idea what our node_index is | 16:04 |
| clayg | it's going to be hard to imagine what our partners should be | 16:04 |
| clayg | but I think we should still either revert to only one or sync with some random two? | 16:04 |
| clayg | bah, probably can't be random :\ | 16:04 |
| acoles | clayg: or we do a tombstone-storm ;) | 16:04 |
| clayg | in the stable non-rebalacing sync case we need to deterministically eventually get to all the nodes | 16:05 |
| acoles | this is useful cos i need to add tombstone test cases to patch 169052 | 16:05 |
| patchbot | acoles: https://review.openstack.org/#/c/169052/ | 16:05 |
| clayg | acoles: honestly I'd almost rather just stick a frag_index marker of some kind in the suffix or something | 16:06 |
| clayg | yeah i don't know if that would work, maybe tombstone storm is the most obviously correct thing to do | 16:08 |
| clayg | idk, I think you pass an empty flag around the chain | 16:09 |
| clayg | it's a puzzle! | 16:11 |
| acoles | yup its always so much harder to delete things than create them | 16:11 |
| clayg | it's only hard when you want to delete ALL THE THINGS! | 16:12 |
| *** rdaly2 has quit IRC | 16:19 | |
| *** annegent_ has quit IRC | 16:22 | |
| acoles | peluse: thanks for replies on review, i'll wait for the re-worked version. just one point i am still unsure about, which is a revert job when all missing frags in receiver suffix have been completely rebuilt. | 16:24 |
| clayg | "a revert job when all missing frags in receiver suffix have been completely rebuilt | 16:27 |
| clayg | individually those words all have meaning to me - together I don't know what they mena | 16:28 |
| acoles | clayg: lol, let me try again | 16:28 |
| acoles | assume sender node has a suffix with frag indexes that belong on another node... | 16:29 |
| clayg | acoles: just for the sake of argument - handoff or rebalance? | 16:29 |
| acoles | (handoff) but the other node has already had its frag repaired so its suffix hash matches sender suffix hash.. | 16:30 |
| acoles | then IF i understand we don't pass that suffix to ssync_sender (big IF there) | 16:30 |
| clayg | acoles: yeah that the thing I don't quite get - hold on there | 16:31 |
| acoles | so the handoff frags don't get removed | 16:31 |
| clayg | why do we do a pre-flight suffix sync on revert? | 16:31 |
| clayg | look at delete updated in the replicator - it calls rsync, it calls replicate, it deletes the part | 16:31 |
| clayg | like there's nothing the other node can tell you that is useful - you don't want any of this data anymore | 16:32 |
| clayg | so you sync - some moves, some is already there, but in the end - you're in sync - you call sync hashes one more time to make sure the remote end doesn't have any out of date invalid hashes - then you get rid of the stuff you don't want anymore | 16:33 |
| acoles | its line 490 in reconstructor.py that i am stuck on | 16:34 |
| acoles | suffixes = self.get_suffix_delta | 16:34 |
| acoles | won't that return empty list if rx suffix is same as tx suffix for that frag index? | 16:35 |
| clayg | yeah doesn't make any sense to compare suffixes before hand on a revert job | 16:35 |
| acoles | scratch that ^^ | 16:35 |
| clayg | well for that frag_index == remote_index w/e | 16:36 |
| acoles | yeah, that | 16:36 |
| acoles | the suffix won't make it to the list passed to ssync | 16:36 |
| acoles | if remote hash == local hash | 16:36 |
| clayg | yeah so - revert - don't look at suffixes - just push this crap off - every suffix you've got that's tainted with this frag_indexs you don't want - push 'em | 16:37 |
| peluse | acoles, just got back from an appt. yeah, we do send the suffix list in that case | 16:37 |
| clayg | peluse: for revert jobs there should be no "delta" - ship it - then rip it | 16:37 |
| peluse | yes | 16:38 |
| peluse | so lets cover this again with the reworked code if that's OK | 16:38 |
| clayg | yay! | 16:38 |
| peluse | shouldn't be much longer, its not as ugly as I thought it would be | 16:38 |
| peluse | in fact its both simpler and more efficient - go figure! | 16:38 |
| * peluse hopes he didn't speak to soon... OK back to work | 16:39 | |
| acoles | peluse: heh | 16:39 |
| acoles | if were were smart we could tell ssync not to bother with the missing_check/updates for that 'no-op' suffix, but thats an optimisation that could wait | 16:43 |
| acoles | peluse: FYI I am working on the todo in ssync_sender where recosntruct_fa returns None | 16:47 |
| *** jistr|demo has quit IRC | 16:47 | |
| peluse | acoles, great, thanks | 16:47 |
| *** jordanP has quit IRC | 16:51 | |
| clayg | acoles: what do you mean no-op suffix - how are you going to skip the missing check? like you don't want to send stuff the other end already has - think rsync | 16:59 |
| clayg | you just don't need a hint about what needs to be synced and what doesn't - anything you have needs to be synced - because it's about to get BALEETED | 17:01 |
| acoles | clayg: but if we know that the other end's suffix hash is same as ours then there is no need to do missing_check - there will be no missing items, no? | 17:02 |
| *** SkyRocknRoll has quit IRC | 17:03 | |
| clayg | but why would know the other ends suffix hash is the same as ours? how do we know it's *still* the same as ours after all that listdirin' we just made it do. remember, BALEETED - just ship anything we have it doesn't and be done with it | 17:04 |
| clayg | what if it just has *MORE* that we do - we still need to ship what we have | 17:05 |
| clayg | the missing check just works out what we're going to put on the wire so it doesn't have to go read a bunch of bytes to get to the ones it wants | 17:06 |
| peluse | clayg, side question - how did you make that long ugly listy of expected jobs that I had #noqa on so neatly formatted? Not manually right?? | 17:10 |
| openstackgerrit | Thiago da Silva proposed openstack/swift: Select policy when running functional test https://review.openstack.org/167595 | 17:10 |
| *** annegentle has joined #openstack-swift | 17:12 | |
| acoles | clayg: i thought we said we would not repeat the local get_hashes for a revert job, so we do the REPLICATE request, remote node calculates its hashes, and then we drop into ssync which ~immediately does missing_check which causes ssync receiver to go open all those diskfiles | 17:12 |
| openstackgerrit | Thiago da Silva proposed openstack/swift: Document SWIFT_TEST_POLICY for regular functional tests https://review.openstack.org/167958 | 17:12 |
| openstackgerrit | Thiago da Silva proposed openstack/swift: Move policy_specified handling to PolicyCollection https://review.openstack.org/168007 | 17:13 |
| acoles | clayg: fair enough, if the remote has more than local (suffix hashes mismatch) then we can't avoid the missing_checks | 17:13 |
| acoles | clayg: its the receiver workload i'm thinking we could reduce | 17:15 |
| clayg | peluse: mostly "manually" | 17:15 |
| peluse | holy shit | 17:16 |
| clayg | i have some editor macros that help quite a bit - then I cooked up a few more to get it the rest of the way | 17:16 |
| clayg | I think torgomatic can just hilight a whole bit thing like that and his emacs' version of vim's gq just works | 17:16 |
| clayg | peluse: why - you're not going to ruin it again are you? | 17:17 |
| clayg | the whole point is that for maintence you need to be able to edit that thing - not just copy and paste in the new one | 17:18 |
| peluse | clayg, well, the jobs are going to be different now.... | 17:18 |
| peluse | I should be able to edit it and future tweaks won't be bad, plus it will be smaller now | 17:18 |
| clayg | sucks to have such big stub data structures all listed out explicitly in the test like that then doesn't it! | 17:18 |
| acoles | peluse: clayg: can i make reconstruct_fa raise a DiskFileError rather than returning None if it fails? | 17:18 |
| clayg | acoles: i have no idea - ssync calls that | 17:19 |
| clayg | acoles: he'd have to do the right thing in either case - why does he like None now? | 17:19 |
| *** thumpba has joined #openstack-swift | 17:20 | |
| acoles | clayg: right now he doesn't do the right thing, thats what i am working on | 17:20 |
| peluse | yeah, returning None is/was a TODO | 17:21 |
| peluse | just need to gracefully exit the ssync process since we couldn't reconstruct - it will automagically try again later | 17:21 |
| clayg | peluse: I thought acoles was going to figure out some way to make it continue on | 17:22 |
| clayg | acoles: ^ which sounds great! | 17:22 |
| acoles | hmm, yeah, i'd rather it gracefully continued | 17:22 |
| clayg | acoles: I think if you're in the middle of a fa body and it blows up your ssync protocol is screwed and there's no way to resync the channel | 17:22 |
| peluse | yeah, that's what I meant - exit the current process of syncing the object that couldn't be reconstructed | 17:23 |
| acoles | peluse: gotcha | 17:23 |
| clayg | acoles: but during the "gather connections to rebuild the fa" if you have a downed node here or there - should be easy enough to just not ship that one to the remote | 17:23 |
| acoles | clayg: ^^ thats the one. if body iter blows up all bets are off | 17:23 |
| *** welldannit has joined #openstack-swift | 17:24 | |
| peluse | clayg, so building jobs to sync tombstones and durables (whcih have no FI) currently I just do the safe thing and sync to all part nodes. You think its better to randomly choose one of them? If we're a handoff then we don't have much chance of getting these guys back to their primmary in a very timely manner... | 17:25 |
| peluse | and I mean this in the case of when we're a handoff only, if we're doing a regular sync they go to the partners.... | 17:26 |
| peluse | tell you what, I'll post the code soon and you can look there for more context | 17:27 |
| *** zhill has joined #openstack-swift | 17:35 | |
| *** annegentle has quit IRC | 17:36 | |
| *** rdaly2 has joined #openstack-swift | 17:39 | |
| clayg | peluse: either sync or revert jobs could have no data files in them :\ | 17:42 |
| clayg | peluse: I *do* sort like the randomly select some set of nodes trick - even if it's like "just pick half" or make it configurable | 17:42 |
| *** geaaru has quit IRC | 17:43 | |
| clayg | peluse: i feel like the node that isn't 404'ing everything in the part isn't acctually going to be letting much data out if he's the outlier | 17:44 |
| clayg | but i'm cool with "do everyone" as long as it's sort of paramaterized in such a way that it'd be obvious to cut it down to *some* kind of subset if we decide it's too chatty | 17:44 |
| *** silor1 has quit IRC | 17:46 | |
| clayg | so i'm sorta pumped about this patch i've been building up - it's in this weird state where the object server is all mime'd and multiphased' up everything is routing to ecdiskfiles and stuff - but the proxy knows *nothing* about ec - and the weird thing is - it "works" - like passes functional tests | 17:50 |
| clayg | because like it just treats it like a replicated storage policy - i just had to hack in a default for the fragment index and the call to commit will even create the durable so GET will work | 17:51 |
| clayg | notmyname: is the meeting in like an hour or ten mins? | 17:52 |
| clayg | awwww bummer - got some errors :'( | 17:52 |
| clayg | oh i think my battery is just running low | 17:53 |
| clayg | chunkwritetimeout? | 17:53 |
| acoles | oh meeting! | 17:54 |
| clayg | acoles: when is it | 17:54 |
| acoles | 66 mins i reckon | 17:54 |
| acoles | clayg: yes, 1 hour to go | 17:55 |
| clayg | k, sweet | 17:55 |
| clayg | so did we manage to land the run functests against specific storage polices - how does that work? | 18:06 |
| tdasilva | clayg: not sure if something else landed on feature/ec. Patch 167595 is still up for review on master | 18:07 |
| tdasilva | patch 167595 | 18:08 |
| patchbot | tdasilva: https://review.openstack.org/#/c/167595/ | 18:08 |
| clayg | SWIFT_TEST_POLICY is like a word in functests/__init__ - it doesn't acctully help anything tho? | 18:08 |
| clayg | looks like an inprocess thing I guess | 18:08 |
| tdasilva | in that patch we also use that env. variable for setting a specific policy | 18:09 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Fix ssync sender behavior for EC reconstructor jobs https://review.openstack.org/169052 | 18:09 |
| clayg | acoles: oh did you look at patch 169614 - is that config option stuff for the inprocess functests helpful? | 18:09 |
| patchbot | clayg: https://review.openstack.org/#/c/169614/ | 18:09 |
| acoles | clayg: yes i did and yes i think you nailed it | 18:10 |
| acoles | bbiab for meeting | 18:10 |
| clayg | acoles: was that fragments list on the get_ondisk_files context ment to stay around - because I think it'd be awesome - but didn't quite follow where it was being used/tested | 18:11 |
| clayg | acoles: re patch 169052 | 18:11 |
| patchbot | clayg: https://review.openstack.org/#/c/169052/ | 18:11 |
| *** fern has joined #openstack-swift | 18:12 | |
| *** acoles is now known as acoles_away | 18:12 | |
| notmyname | good morning | 18:14 |
| notmyname | clayg: meeting in 45 minutes | 18:14 |
| clayg | ok, i think i'm going to jump a train - should be close! | 18:15 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Initial Erasure Code Docs https://review.openstack.org/169615 | 18:15 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Per-policy DiskFile classes https://review.openstack.org/169651 | 18:15 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Allow sending object metadata after data https://review.openstack.org/169864 | 18:15 |
| peluse | just dont jump a shark | 18:16 |
| *** nexusz99 has quit IRC | 18:20 | |
| peluse | clayg, acoles_away : here is the updated job build code. I have not updated all the test expectations yet. Put on your headphones and give thia a read, I'll go back and comment on some of acoles other notes in this area in the patch https://gist.github.com/peluse/4f77b73db5e2a220591c | 18:21 |
| peluse | anyone else seeing this new "paper clip" helper icon thingy using chrome on gerrit? WTH is it and how do I get rid of it? :) | 18:22 |
| *** gvernik has joined #openstack-swift | 18:23 | |
| *** bkopilov has joined #openstack-swift | 18:23 | |
| notmyname | peluse: I think it's a present for April 1 | 18:24 |
| notmyname | peluse: I too can not seem to get rid of it | 18:24 |
| peluse | fantastic... | 18:24 |
| torgomatic | does it do anything besides be dancing baloney? | 18:26 |
| notmyname | more dancing if you double-triple-etc click on it | 18:26 |
| *** Bsony_ has joined #openstack-swift | 18:27 | |
| torgomatic | like, "It looks like you've submitted code with a memory leak! Would you like help not holding onto references to everything you've ever done like some kind of demented packrat?" | 18:27 |
| peluse | I got it to put headphones on somehow | 18:27 |
| notmyname | you seem to be trying to write code. have you considered selling it all and buying a llama farm? | 18:27 |
| *** Bsony has quit IRC | 18:27 | |
| *** romaindornellas has joined #openstack-swift | 18:28 | |
| *** romaindornellas has left #openstack-swift | 18:28 | |
| *** swiftops has joined #openstack-swift | 18:29 | |
| swiftops | hi guys. does anyone already play with swift-drive-audit when kernel logs are sent to messages instead of kern.log ? | 18:30 |
| *** erlon has joined #openstack-swift | 18:35 | |
| swiftops | because date/time pattern is different, swift-drive-audit does not work | 18:36 |
| *** Bsony_ has quit IRC | 18:36 | |
| torgomatic | swiftops: they're the same on my machine; isn't the date format a function of your local syslog config? | 18:42 |
| swiftops | I dont think. from swift-drive-audit source code, I can see they add a year addition to date format for logs from kern.log | 18:43 |
| swiftops | meaning format from kern.log is different | 18:44 |
| peluse | clayg, so actually in the updated job building code I posted I think there is still an opportunity to trim down the sync_to nodes in the 'None' jobs... | 18:46 |
| *** ho has joined #openstack-swift | 18:52 | |
| notmyname | meeting will start about 10 minutes late to give clayg time to get off the train | 18:52 |
| *** kota_ has joined #openstack-swift | 18:54 | |
| ho | good morning! | 18:54 |
| kota_ | ho: good midnight! | 18:55 |
| notmyname | ho: kota_: we'll start about 10 minutes late, so if you need to go get coffee/tea, you've got a few minutes extra :-) | 18:56 |
| ho | kota_: good midnight :) | 18:56 |
| *** bkopilov has quit IRC | 18:56 | |
| kota_ | notmyname: ok | 18:56 |
| ho | notmyname: good news! I will get coffee.. | 18:57 |
| notmyname | we're waiting for clayg to get off the train | 18:58 |
| kota_ | notmyname: I see, let's wait... | 18:59 |
| mattoliverau | morning | 18:59 |
| *** acoles_away is now known as acoles | 19:02 | |
| notmyname | good mornign mattoliverau and acoles | 19:02 |
| acoles | notmyname: hi, are we having a meeting? | 19:03 |
| ho | mattoliverau: morning | 19:03 |
| notmyname | acoles: ya. delayed start my about abother 5 minutes | 19:03 |
| acoles | oh i just read back | 19:03 |
| notmyname | :-) | 19:03 |
| clayg | lets do it! | 19:05 |
| notmyname | woot | 19:05 |
| notmyname | ok, meeting starting time in #openstack-meeting | 19:06 |
| *** tellesnobrega_ has joined #openstack-swift | 19:07 | |
| *** tellesnobrega_ has quit IRC | 19:08 | |
| *** joeljwright1 has joined #openstack-swift | 19:16 | |
| *** joeljwright has quit IRC | 19:18 | |
| acoles | that paperclip is getting on my nerves | 19:23 |
| clayg | acoles: stupid clippy | 19:23 |
| *** jbonjean has quit IRC | 19:24 | |
| cschwede | acoles: clayg: it’s just a matter of time until this patch lands: https://review.openstack.org/#/c/169508/ | 19:24 |
| cschwede | i was like „wut, where is this trojan coming from“ this morning… | 19:25 |
| *** jbonjean has joined #openstack-swift | 19:28 | |
| acoles | cschwede: its in merge conflict !! ^^ | 19:29 |
| *** reed has quit IRC | 19:40 | |
| peluse | acoles, FYI I updated all comments on the ECrecon patch and am making some final changes now. Will take just a bit to confirm them though | 19:45 |
| *** dencaval has quit IRC | 19:45 | |
| acoles | peluse: ok, thx, i won't be doing a whole lot more today | 19:47 |
| peluse | no sweat, thanks! | 19:47 |
| acoles | i nominate clippit for PTL :D | 19:51 |
| notmyname | wheeee | 19:55 |
| jrichli | lol | 19:56 |
| *** kota_ has quit IRC | 19:58 | |
| * mattoliverau goes to find breakfast (noting on Sunday I turn my clocks back, so next meeting it'll be going back to bed days again) :( | 19:59 | |
| acoles | mattoliverau: but look on the bright side - we may overlap more ;) | 20:01 |
| mattoliverau | acoles: that's true, that's a very bright side ;) | 20:01 |
| acoles | mattoliverau: i see your sarcasm there :P | 20:02 |
| *** lcurtis has joined #openstack-swift | 20:04 | |
| peluse | acoles, clayg updated job contruction code at https://gist.github.com/peluse/4f77b73db5e2a220591c | 20:05 |
| peluse | I'm going to start fixing up the tests to match, I believe this is what we're after. | 20:06 |
| *** annegent_ has joined #openstack-swift | 20:06 | |
| acoles | peluse: ok, looks cleaner | 20:10 |
| peluse | yeah and does the 1 job per FI per part that we were after | 20:10 |
| *** acoles is now known as acoles_away | 20:11 | |
| *** panbalag has quit IRC | 20:13 | |
| *** lastops has quit IRC | 20:15 | |
| clayg | weeeeee | 20:28 |
| *** annegent_ has quit IRC | 20:31 | |
| clayg | peluse: looks close! I think the "remove the None job" could be simplified with a list comphrehension of filter if you're more into the functional route | 20:33 |
| clayg | peluse: good luck with the tests | 20:33 |
| clayg | peluse: if it makes more sense to move tests off the GlobalSetup test case than to fix the ones that are there - by all means feel free! | 20:33 |
| peluse | clayg, OK I think I can do that. I'll get the tests running first | 20:33 |
| peluse | I dont' think it will be bad once I get the expected stuff correct | 20:34 |
| clayg | peluse: I also think it'd be totally reasonable to remove the commandline options for filtering devices - if that helps simplify or reduce required tests | 20:34 |
| clayg | peluse: we can always add that feature later if people want it - it might be nice to have - but hardly required | 20:34 |
| clayg | peluse: sweet! | 20:35 |
| peluse | what commandline options for filtering devices? | 20:35 |
| clayg | peluse: oh idk, whatever their called "override devices" or something? | 20:35 |
| peluse | oh yeah, that thing :) | 20:36 |
| clayg | notmyname: why this no merge yet -> https://review.openstack.org/#/c/169897/ | 20:36 |
| clayg | oh well... | 20:37 |
| * clayg goes to try and make the proxy work | 20:37 | |
| *** gvernik has quit IRC | 20:39 | |
| *** bkopilov has joined #openstack-swift | 20:57 | |
| *** vinsh has quit IRC | 20:58 | |
| *** scotticus has joined #openstack-swift | 21:02 | |
| zaitcev | How do I calculate the maximum object size? Surely the manifests cannot be limitless, as some kind of DoS prevention? | 21:03 |
| swifterdarrell | zaitcev: i guess it would be SLO config var max_manifest_segments (default 1000?) times swift constraints max obj size which defaults to 5 GiB? | 21:09 |
| swifterdarrell | zaitcev: which i guess equates to, like, 5 TB? | 21:09 |
| swifterdarrell | zaitcev: except that SLO can contain SLO, right? | 21:09 |
| joeljwright1 | zaitcev: swifterdarrell: SLO can contain SLO but I believe there's a maximum recursion depth | 21:10 |
| joeljwright1 | default is max_slo_recursion_depth = 10 | 21:10 |
| *** vinsh has joined #openstack-swift | 21:11 | |
| torgomatic | right, so if you do a 10-deep SLO tree, you get a maximum of 10^9 * 5 GiB, or 4.657 EiB | 21:28 |
| torgomatic | er, no | 21:28 |
| torgomatic | 1000^9 * 5 GiB, or roughly 5 trillion yottabytes | 21:30 |
| torgomatic | also know as "enough" | 21:31 |
| torgomatic | *known | 21:31 |
| ahale | i figured the practical limit will be governed by max_large_object_get_time, but now I'm struggling to see where thats actually used in swift | 21:37 |
| torgomatic | ahale: it's in common.request_helpers.SegmentedIterable | 21:38 |
| ahale | ahh yeah | 21:40 |
| clayg | ahale: !!!! | 21:41 |
| ahale | hey clay | 21:41 |
| clayg | lol, yeah good point - you're limited to how much you can download in what - 10 hours? | 21:42 |
| zaitcev | come on guys, I wanted to calclulate it myself | 21:42 |
| zaitcev | But thanks | 21:42 |
| *** jrichli has quit IRC | 21:43 | |
| clayg | you can't throw a fun problem out there to a bunch of hackers - it's like catnip | 21:44 |
| zaitcev | tell me this is an April Fool's joke - https://review.openstack.org/#/c/169897/1/.gitreview | 21:50 |
| zaitcev | ugh | 21:51 |
| zaitcev | * [new branch] feature/ec_review -> origin/feature/ec_review | 21:51 |
| openstackgerrit | paul luse proposed openstack/swift: Erasure Code Reconstructor https://review.openstack.org/131872 | 21:52 |
| ahale | so if it turned out your limiting factor in downloadable file size is the sata drive in the object server, would EC let you speed that up? | 21:52 |
| peluse | clayg, that ones for you! ^^ | 21:52 |
| zaitcev | USB 3.1 might | 21:52 |
| zaitcev | or Fusion-IO | 21:52 |
| ahale | our finance team would love fusion io storage | 21:53 |
| openstackgerrit | paul luse proposed openstack/swift: EC Reconstructor Probe Tests https://review.openstack.org/164291 | 21:55 |
| openstackgerrit | paul luse proposed openstack/swift: Fix ssync sender behavior for EC reconstructor jobs https://review.openstack.org/169052 | 21:57 |
| *** fern has quit IRC | 22:03 | |
| *** ekarlso has quit IRC | 22:04 | |
| *** ekarlso has joined #openstack-swift | 22:04 | |
| *** 77CAAKTHH has joined #openstack-swift | 22:13 | |
| *** 77CAAKTHH has quit IRC | 22:23 | |
| *** lcurtis has quit IRC | 22:30 | |
| *** joeljwright1 has quit IRC | 22:39 | |
| *** annegentle has joined #openstack-swift | 22:48 | |
| *** annegentle has quit IRC | 22:48 | |
| *** annegentle has joined #openstack-swift | 22:58 | |
| peluse | wow acoles_away ssync tests look great! | 23:06 |
| *** chlong has joined #openstack-swift | 23:07 | |
| *** thumpba has quit IRC | 23:11 | |
| *** thumpba has joined #openstack-swift | 23:11 | |
| *** annegentle has quit IRC | 23:15 | |
| *** kei_yama has joined #openstack-swift | 23:19 | |
| *** petertr7 has quit IRC | 23:21 | |
| *** thumpba has quit IRC | 23:28 | |
| *** thumpba_ has joined #openstack-swift | 23:28 | |
| *** annegentle has joined #openstack-swift | 23:29 | |
| *** ChanServ changes topic to "Soft freeze of master in effect | EC Merge plan: https://etherpad.openstack.org/p/ec_merge_plan | Review Dashboard: http://goo.gl/uRzLBX | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/" | 23:30 | |
| *** rdaly2 has quit IRC | 23:30 | |
| *** jrichli has joined #openstack-swift | 23:31 | |
| notmyname | ok, channel topic updated. soft freeze of master in effect | 23:32 |
| *** annegentle has quit IRC | 23:37 | |
| openstackgerrit | Janie Richling proposed openstack/swift: WIP - Provides a simple skeleton of middleware for encryption feature. https://review.openstack.org/157907 | 23:38 |
| clayg | zaitcev: what part is funny? | 23:40 |
| *** mahatic has quit IRC | 23:41 | |
| *** mahatic has joined #openstack-swift | 23:42 | |
| mattoliverau | So has Nebula really gone out of business or is it just an april fools? Being a day a head can be problematic as I read things on April 1st with skeptasism but by the 2nd I forget too :P | 23:43 |
| peluse | notmyname, do you a link to our etherpad for summit ideas? | 23:43 |
| peluse | is this it? https://wiki.openstack.org/wiki/Design_Summit/Kilo/Etherpads#Swift | 23:47 |
| clayg | torgomatic: you around? | 23:47 |
| clayg | i'm looking at a change to test/unit/proxy/test_sysmeta - it doesn't seem to be needed? | 23:48 |
| clayg | can you diff that file from feature/ec and master and tell me what you were thinking in 04d87c8? | 23:48 |
| peluse | clayg, fyi im out for a few hrs but when I get back I will review acoles ssync test and fix the one intermittent test issues with the ECrecon (stupid list ordering comparison thing) | 23:48 |
| clayg | torgomatic: I guess with the EC PUT extraction it's just not relevant anymore | 23:49 |
| clayg | mattoliverau: i think it might be legit - but i'm way scared that we're all going to look the fool tomorrow | 23:50 |
| mattoliverau | clayg: yeah, I guess time will really tell. If it is real, what an annoying day to annouce it :P | 23:51 |
| peluse | notmyname, i meant for liberty not kilo | 23:52 |
| clayg | mattoliverau: forbes picked it up - would be nice for someone to confirm something - no one is even addressing the fact that this announcement came out *today* of all days | 23:53 |
| mattoliverau | clayg: Which is why I was suspicous, but talking to some of the guys on my team here, they say it might be legit.. they know a few guys working there.. unless everyones in on it.. I guess we'll find out tomorrow | 23:54 |
| *** zhill has quit IRC | 23:55 | |
| *** mahatic has quit IRC | 23:57 | |
| mattoliverau | torgomatic: RE: Multi-Range GETS.. so far so good.. not failing when I re attempted what I tried previously! Looks like it's working :) Now to go and throw alot more test data at it. | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!