Friday, 2014-03-14

*** Fin1te has quit IRC00:01
openstackgerritA change was merged to openstack/swift: merged master into ec branch  https://review.openstack.org/8009500:08
*** CrackerJackMack has quit IRC00:17
notmynamelook at that!00:18
*** CrackerJackMack has joined #openstack-swift00:19
*** zaitcev has quit IRC00:23
*** CrackerJackMack has quit IRC00:24
*** CrackerJackMack has joined #openstack-swift00:26
*** zaitcev_ has joined #openstack-swift00:27
*** shri has quit IRC00:27
*** mkollaro has quit IRC00:27
h6wWho manages this documentation?   http://docs.openstack.org/developer/swift/howto_installmultinode.html00:29
h6wProbably needs an update, since it mentions 10.04, but also I just had to do this to make recon work:  http://andymc-stack.co.uk/2012/08/swift-recon-400-bad-request/00:30
*** matsuhashi has joined #openstack-swift00:31
notmynameh6w: that's part of the swift source repo. you can submit a patch :-)00:33
h6wIs it on a repos somewhere?  Cos I don't mind updating it from what I've learned in the past couple days.00:33
h6wOh, nice, that's on github?00:33
torgomatich6w: submit patches to openstack gerrit; there should be a link in CONTRIBUTING.md00:34
* h6w thinks this is painful for a few-line edit.00:35
h6wYesterday I submit a patch to cloudfuse.  It was extremely simple.  Fork, edit, submit pull request.   I didn't even need to do anything on the command line.  All done on the web!00:36
notmynameyup. github is nice00:37
notmynameh6w: all the patches for any openstack project go through a formal code review. (there's also a CLA that you have to sign)00:38
* h6w wonders if there's a way github could incorporate code review easily.00:39
notmynameh6w: however, if you didn't want to go through that you could hypothetically make a .patch in a public place somewhere and torgomatic or I could submit it ;-)00:39
notmynameactually, it does. but it doesn't support gated commits00:39
torgomaticyeah, it's for the CLA legal baloney^Wnecessities00:39
torgomaticthe rest could be worked around, but the no-submission-without-signature thing can't00:40
*** zaitcev_ is now known as zaitcev00:41
*** ChanServ sets mode: +v zaitcev00:41
* h6w nods.00:41
h6wSo "[async_pending] - No hosts returned valid data."  Is that ok, or does that indicate a failure?00:46
h6wI have quite a few of these "No hosts returned valid data." on different things.00:47
h6wasync_pending, ALL_auditor, ZBF_auditor, object_expiration_pass, and expired_last_pass.00:48
*** openstack has joined #openstack-swift00:53
*** erlon has joined #openstack-swift00:53
openstackgerritA change was merged to openstack/swift: make probetests work with conf.d configs  https://review.openstack.org/7875300:55
*** jasondot_ has quit IRC00:55
*** sudorandom has joined #openstack-swift00:55
*** openstack has joined #openstack-swift00:57
*** openstack has quit IRC00:57
*** openstack has joined #openstack-swift01:00
*** Dharmit has joined #openstack-swift01:02
*** CrackerJackMack has joined #openstack-swift01:02
*** j_king has quit IRC01:02
*** CrackerJackMack has quit IRC01:07
*** matsuhashi has quit IRC01:08
*** j_king has joined #openstack-swift01:08
*** Dharmit has quit IRC01:08
*** CrackerJackMack has joined #openstack-swift01:10
*** Dharmit has joined #openstack-swift01:10
*** matsuhashi has joined #openstack-swift01:12
*** CrackerJackMack has quit IRC01:12
*** openstack has joined #openstack-swift01:15
*** openstack has joined #openstack-swift01:19
*** nosnos has joined #openstack-swift01:19
*** openstack has quit IRC01:24
*** openstack has joined #openstack-swift01:25
*** sudorandom has joined #openstack-swift01:25
*** openstack has joined #openstack-swift15:09
*** dmsimard has joined #openstack-swift15:12
creihtportante: the issue revolves around updating the content-type since the content-type is in the container listing15:13
*** dmsimard has quit IRC15:13
creihtfast post can't update the containers15:13
portanteah, so the content-type can be modified on a post, hmmm15:14
portantebut why can't a post operation update the containers?15:15
portantebecause a copy on post does, right?15:15
gholtIt's a long nasty story. :)15:15
portante;)15:16
gholtBut it revolves around the fact that container servers only have 1 entry per object no matter what and object servers can have multiple "entries" per object that get resolved to *something* that would be sent to the container server.15:16
gholtEssentially you have to make sure all the different ways an object server could have to report a set of its data would end up reporting the same distinct thing to the container server.15:17
gholtThe container server has object name, size, etag, and content-type as I recall. And I really wish it never had etag nor content-type, but that predates even us. ;)15:18
creihtheh15:19
portanteso if somebody enables fast-post, and uses a POST to update the content-type, the container listing will never pick it up?15:19
gholtTrue.15:19
portantebut copy on post will15:20
creihtright15:20
portanteis it too difficult to solve that problem with an updater?15:21
gholtYou mean the existing updaters? I don't think they can solve the problem15:22
*** Dharmit has joined #openstack-swift15:22
portanteso what if we detect a content-type change and transform to copy on post?15:22
portanteany ways, just musing, ignore me if not interested15:22
gholtThat's a possibility, yes.15:23
gholtFor Rackspace at least, it would seem nearly all the posts we get are to update the content-type though.15:23
portanteso what the container listing tracks is all fixed metadata, except that content-type which is sort-a-user defined15:23
gholtExcept for this one guy that needed to update a bunch of x-delete-at values. That would've helped him a ton. :)15:23
creihtlol15:24
gholtWell, another solution, and I really really hesitate to even mention this because somebody will try to implement it ;) is to make the container server .data/.meta aware and do the object resolutions itself. I don't know how you do that performantly (is that a word?).15:25
*** Midnightmyth has joined #openstack-swift15:25
portantehmmm15:26
portanteperhaps an april 1st commit15:26
gholt"Here's a 100 million object listing, with around 20 million .meta modifiers, gimme a listing." :)15:26
gholtI don't entirely doubt there's a solution there though.15:27
portantewhat verifies that the etag in a container listing continues to match the object's etag?15:29
gholtPretty much nothing. There is not cross server auditors.15:29
gholtThat's a neat grammar I has.15:29
creihtheh15:30
portanteme thinks neat too15:30
creihtit is probably my fault15:30
* portante bbiab15:31
*** jamieh has joined #openstack-swift15:33
*** kun_huang has quit IRC15:33
*** jamieh is now known as Guest73915:33
*** Guest739 has quit IRC15:34
*** jamie_h has quit IRC15:35
*** foexle has quit IRC15:39
*** keving1 has joined #openstack-swift15:45
*** drumkilla is now known as rustlebee15:47
*** mkollaro has quit IRC15:48
*** gyee has joined #openstack-swift15:49
*** keving1 has quit IRC15:53
*** ben_roeder1 has quit IRC15:54
*** Kim-Chi-San has quit IRC16:12
*** chandan_kumar has quit IRC16:14
*** Kim-Chi-San has joined #openstack-swift16:14
*** Kim-Chi-San has quit IRC16:15
*** tburnes has joined #openstack-swift16:15
*** tburnes has quit IRC16:15
*** tburnes has joined #openstack-swift16:16
*** csd has joined #openstack-swift16:38
*** openstackstatus has joined #openstack-swift16:39
*** jamieh has joined #openstack-swift16:45
*** jamieh is now known as Guest5266716:45
*** Guest52667 has quit IRC16:45
*** jamieh_ has joined #openstack-swift16:46
*** jamieh_ has quit IRC16:46
*** jamieh_ has joined #openstack-swift16:47
*** Dharmit has quit IRC16:47
*** keving1 has joined #openstack-swift16:50
*** keving1 has quit IRC16:59
*** jamieh_ has quit IRC17:05
*** shri has joined #openstack-swift17:10
*** jamieh_ has joined #openstack-swift17:11
*** mkollaro has joined #openstack-swift17:13
*** dmsimard has joined #openstack-swift17:15
*** mkollaro has quit IRC17:20
*** keving1 has joined #openstack-swift17:24
*** mkollaro has joined #openstack-swift17:25
*** piyush has quit IRC17:34
*** piyush has joined #openstack-swift17:39
*** piyush1 has joined #openstack-swift17:41
*** piyush has quit IRC17:43
*** nacim has quit IRC17:50
*** dmsimard has quit IRC17:50
*** jamieh_ has quit IRC17:59
*** lpabon has joined #openstack-swift18:04
*** zaitcev has joined #openstack-swift18:19
*** ChanServ sets mode: +v zaitcev18:19
*** erlon has quit IRC18:22
*** mmcardle has quit IRC18:22
*** jergerber has joined #openstack-swift18:29
claygmorning18:31
claygoh oh - are we blaming creiht - I want to get on that!18:31
gholtErg, functests don't pass on master for me. Gotta figure out which conf change I missed. ;)18:35
*** Trixboxer has quit IRC18:38
creihtclayg: lol18:42
creihtso I'm trying to implement if-none-match for PUTs18:42
creihtfor the use case when users are doing a HEAD before doing a PUT18:43
creihtif-none-match: * is implemented and fairly simple18:44
creihtif-none-match: ETAG is turning out to have lots of interesting edge cases18:44
creihtwould anyone be opposed to having just if-none-match: * implemented but not the ETAG part?18:45
creihtwe have a customer that could really make use of if-none-match: *18:46
claygis if-none-match the same as a 404?18:46
clayger... if-none-match: *18:46
creihtyeah18:46
creihtonly upload if it doesn't exist18:47
creihtsince doing a head on 404 can be very slow18:47
clayggholt: try moving tempurl ;)18:47
creihtand this user does a HEAD before every PUT18:47
gholtHeh, nah, I messed something up at some point -- probably to test one thing or another -- and now they don't pass. They don't pass on known good branches from before either.18:48
claygand would some how if-none-match: * make it *not* got for hand-offs?18:48
creihtthere is the edge case that if18:49
creiht1.) the three cannonical locations are up and available and don't have it18:49
creiht2.) There is a copy somewhere else18:49
creihtit would upload18:49
gholtHaha, figured it out. It was all the efed up ring testing I was doing with torgomatic before and forgot about apparently. :)18:50
claygoh yes, python swift/test/probetests/common.py can help with that sort of thing sometimes18:51
claygof course remakerrings would have squared it regardless i suppose...18:51
claygcreiht: i still don't see why three 404's on the original HEAD wouldn't dig into handoffs...18:52
creihtmaybe that's what I am trying to say18:53
claygwhich sort sounded like what you were trying to work around, but maybe I'm wrong about how head works18:53
creihtso right now a HEAD will try the 3 canonical locations and each handoff location if not found18:53
creihtsequentially18:53
claygHEAD x-dont-dig-into-hand-offs-cause-im-a-busy-guy seems cool18:54
creihtwhich makes a 404 on HEAD very slow18:54
creihtlol18:54
claygor also -> https://review.openstack.org/#/c/62061/18:54
claygyeah 404's are slow - we don't like to not have something - esspecially when we do18:55
creihtright18:55
claygconcurrency could help18:55
gholtyes18:57
claygredbo: in your free time can you work on backend server support for TCP Fast Open?  Should be on by default in 14.04!19:01
*** jogo is now known as flashgordon19:03
creihtclayg: that would certainly help19:04
creihtfor this use case, it may not be enough though19:04
creihtwill have to discuss further19:05
claygi sort of abandoned the make all the priarmy requests at the same time approach because I felt like I should have just gone ahead and done it down in dfg's GetOrHeadHandler19:05
creihtclayg: I'll also have to dig through your code :)19:07
claygcreiht: the proxy PUT refactor thing?  oh or that other HEAD wip branch?19:07
creihtsorry... the HEAD branch19:08
claygI just copied what redbo did for DELETE mostly... but it was *sorta* different because handoffs?  shit... i don't remember19:08
claygk - good luck :P19:08
gholtHehe19:08
creihtlol19:08
creihtclayg: my main concerns with that patch for this use case19:13
creiht1.) client still has to make two round-trips for each PUT requests19:13
creiht2.) it still has to wait for all responses, even if in parallel, so it will be as fast as the slowest request19:14
claygOH!  you want to do a PUT *with* if-none-match: * ???19:14
creihtlol19:14
creihtyes :)19:14
creihtsorry if I didn't explain myself very well19:14
claygi thought you wanted the clint to do the HEAD with if-none-match - obviously I wasn't paying attention19:15
creihthah19:15
creihtyeah that would be kinda silly19:15
creihtwell at least for this use case19:15
claygso you have a new thing (inside PUT (or middleware?)) that looks like HEAD but only hits the primary's and you hope for the best19:15
creihtclayg: when the proxy does the expect 100 call to the object server19:16
claygseems sorta neat19:16
creihtthe object server returns 412 if it already has an object19:16
creihtif *any* of the object servers return a 412, then the proxy returns a 41219:17
creihtthe above is for if-none-match: *19:17
claygehw... so it's more like the check where we count up "required connections" or whatever it's called has to get smarter19:17
creihtjust slightly19:17
creihtI actually don't have to modify the check19:18
creihtin the PUT code, after it gets the connections, it looks at the status of each result to see if it got a 41219:19
claygyeah let _connect_put_node bubble out is_success or 412, then before you check quorm size you look for conflicts and bail out - neato!19:20
claygyeah I don't care if we only get if-none-match: * - still sorta worried about how you doc it though "if-none-of-the-obvious-places-i-looked-match: *"19:21
claygcreiht: can't they just redesign their app to "pretend" it's talking to a distributed eventaully consistent data store?19:22
* clayg kids19:22
claygcreiht: how about they do a PUT with a really old x-timestamp?19:24
creihtBut he doesn't want to spend the time PUTing if he doesn't have to19:27
gholtclayg: Yeah, that was my recommendation. But it is kinda lame since we tie x-timestamp and last-modified. :/19:34
gholtI guess if the modification time doesn't really matter to them it'd be fine. they could base the timestamp from a bit of the etag or something.19:36
gholtsomeday we should make it to where you can have a last-modified timestamp that isn't the x-timestamp19:37
claygcreiht: well... I'm scared-of-it/confident-you'll-figure-it-out and also not keen on implementing something that "works" with a bunch of edge clauses we can't really ever hope a client writing against the api could hold in their head long enough to design an app that won't eventually result in swift letting them down :'(19:39
claygit's a sweet use case tho - totally have faith in you!19:40
clayggo creiht !19:40
gholtHeheh19:41
creihtlol19:42
creihtclayg: well that's partly the reason that I'm thinking about droping the if-none-match: etag behavior19:45
creihtbecause it has so many weird edge case possibilities19:45
*** fungi has joined #openstack-swift19:51
*** fungi has left #openstack-swift19:51
claygdfg: haha you can't get it go away because it's WIP lallalala :D19:52
*** dstufft has joined #openstack-swift19:52
dstufftHi, are there docs that actually document how to use the python API, or is the only thing available a giant list of classes/methods? :/19:53
creihtdstufft:19:57
creihterm19:57
creihthowdy :)19:57
dstuffthi :)19:57
creihtdstufft: http://docs.openstack.org/api/openstack-object-storage/1.0/content/19:58
creihtIs that what you are looking for?19:58
dstufftHmm doesn't look like it, I guess I should say specifically I'm looking at swiftclient19:59
creihtoh19:59
creihthehe19:59
dstufftSorry forgot to mention19:59
dstufftI want to store some objects ;) Without shelling out, but all I can find is a big list of methods :/20:00
creihtI think all we have is20:00
creihthttp://docs.openstack.org/developer/python-swiftclient/swiftclient.html20:00
dstufftyea that's what I found :/ Ok thanks20:02
creihthehe20:03
creihtyeah that area could use some work20:03
claygdstufft: can you document as you go for us ;)20:04
creihthah20:04
creihtI thought someone had done a post or something showing some of the getting started basics20:05
creihtbut I can't find it now20:05
creihtdstufft: anything we can help you with?20:05
dstufftwhoops just noticed the time, I'll be back in 20 or so have to pick up my daughter ;)20:05
*** csd has quit IRC20:07
*** shri has quit IRC20:26
*** shri has joined #openstack-swift20:30
*** lpabon has quit IRC20:38
*** tdasilva has left #openstack-swift20:44
*** csd has joined #openstack-swift20:49
*** openstackgerrit has quit IRC21:21
*** openstackgerrit has joined #openstack-swift21:21
*** openstackgerrit has quit IRC21:32
*** openstackgerrit has joined #openstack-swift21:32
*** bsdkurt has quit IRC21:34
notmynamehey peluse. guess what I just finished deploying swift on? :-)21:39
omamea fridge?21:40
notmynameheh21:43
notmynameif only these boxes would help keep my server room cool21:43
notmynamehttps://gist.github.com/notmyname/a2b5dbd9602b52fa11b621:44
*** peluse has quit IRC21:47
*** peluse has joined #openstack-swift21:48
torgomaticnotmyname: have you tried running the fans backwards?21:52
*** piyush1 has quit IRC21:53
notmynameis that better or worse that turning the servers upside-down?21:54
*** changbl has quit IRC21:55
*** bada_ has joined #openstack-swift21:55
*** changbl has joined #openstack-swift21:55
*** bada__ has quit IRC21:56
torgomaticwhatever you do, don't do both at once21:58
*** tburnes has quit IRC22:02
*** dstufft has left #openstack-swift22:16
*** sudorandom has quit IRC22:18
*** sudorandom_ has joined #openstack-swift22:19
*** sudorandom_ is now known as sudorandom22:19
*** CrackerJackMack has quit IRC22:26
*** CrackerJackMack has joined #openstack-swift22:27
*** bsdkurt has joined #openstack-swift22:34
*** openstackstatus has quit IRC23:36
*** zaitcev has quit IRC23:37
*** jergerber has quit IRC23:38
*** openstackgerrit has quit IRC23:39
*** changbl has quit IRC23:42
*** bsdkurt has quit IRC23:45
*** keving1 has quit IRC23:46
*** occupant has quit IRC23:46
*** j_king has quit IRC23:46
*** omame has quit IRC23:46
*** Alex_Gaynor has quit IRC23:46
*** mkerrin1 has quit IRC23:46
*** tristanC has quit IRC23:46
*** sungju has quit IRC23:46
*** mtreinish has quit IRC23:46
*** pconstantine has quit IRC23:46
*** CrackerJackMack has quit IRC23:46
*** sudorandom has quit IRC23:46
*** bada_ has quit IRC23:46
*** wer has quit IRC23:46
*** chmouel has quit IRC23:46
*** zackmdavis has quit IRC23:46
*** zanc_ has quit IRC23:46
*** Diddi has quit IRC23:46
*** wkelly has quit IRC23:46
*** EmilienM has quit IRC23:46
*** ondergetekende has quit IRC23:46
*** pandemicsyn has quit IRC23:46
*** acoles- has quit IRC23:46
*** gholt has quit IRC23:46
*** shri has quit IRC23:46
*** ccorrigan has quit IRC23:46
*** Guest56062 has quit IRC23:46
*** sfineberg has quit IRC23:46
*** saschpe has quit IRC23:46
*** swifterdarrell has quit IRC23:46
*** clarkb has quit IRC23:46
*** mkollaro has quit IRC23:46
*** otherjon has quit IRC23:46
*** mandarine has quit IRC23:46
*** grapsus__ has quit IRC23:46
*** joearnold has quit IRC23:46
*** csd has quit IRC23:46
*** gyee has quit IRC23:46
*** Midnightmyth has quit IRC23:46
*** wayneeseguin has quit IRC23:46
*** pberis has quit IRC23:46
*** fbo has quit IRC23:46
*** yuan has quit IRC23:46
*** booi has quit IRC23:46
*** jeblair has quit IRC23:46
*** acorwin has quit IRC23:46
*** tanee-away has quit IRC23:46
*** early has quit IRC23:46
*** redbo has quit IRC23:46
*** notmyname has quit IRC23:46
*** MooingLemur has quit IRC23:46
*** rahmu has quit IRC23:46
*** dosaboy has quit IRC23:46
*** torgomatic has quit IRC23:46
*** minnear has quit IRC23:46
*** glange has quit IRC23:46
*** flashgordon has quit IRC23:46
*** hugokuo has quit IRC23:46
*** ctennis has quit IRC23:46
*** anderstj has quit IRC23:46
*** alpha_ori has quit IRC23:46
*** ryao has quit IRC23:46
*** anticw has quit IRC23:46
*** dfg has quit IRC23:46
*** mordred has quit IRC23:46
*** kragniz has quit IRC23:46
*** sileht has quit IRC23:46
*** swills has quit IRC23:46
*** zigo has quit IRC23:46
*** Anticimex has quit IRC23:46
*** StevenK has quit IRC23:46
*** creiht has quit IRC23:46
*** otoolee has quit IRC23:46
*** mhu has quit IRC23:46
*** openstack has joined #openstack-swift23:47
*** openstackstatus has joined #openstack-swift23:47
*** mkerrin has joined #openstack-swift23:51
*** Alex_Gaynor has joined #openstack-swift23:52
*** bada_ has joined #openstack-swift23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!