| *** zhiyan_ is now known as zhiyan | 00:10 | |
| *** matsuhashi has joined #openstack-swift | 00:27 | |
| *** NM has joined #openstack-swift | 00:29 | |
| *** dmorita has joined #openstack-swift | 00:36 | |
| *** NM has quit IRC | 00:38 | |
| *** shri has quit IRC | 00:49 | |
| *** dmsimard_away is now known as dmsimard | 00:51 | |
| *** mwstorer has quit IRC | 00:51 | |
| *** lpabon has quit IRC | 01:06 | |
| *** haomaiwang has joined #openstack-swift | 01:10 | |
| *** haomaiw__ has joined #openstack-swift | 01:12 | |
| *** mkollaro has quit IRC | 01:14 | |
| *** haomaiwang has quit IRC | 01:15 | |
| *** dmsimard is now known as dmsimard_away | 01:18 | |
| *** mkollaro has joined #openstack-swift | 01:19 | |
| *** diegows has quit IRC | 01:28 | |
| *** nosnos has joined #openstack-swift | 01:33 | |
| *** kashyapk has joined #openstack-swift | 01:49 | |
| *** kashyapk has quit IRC | 01:55 | |
| *** nsquare has quit IRC | 02:21 | |
| *** mkollaro has quit IRC | 02:27 | |
| *** byeager_away is now known as byeager_ | 02:29 | |
| *** byeager_ is now known as byeager | 02:32 | |
| *** byeager is now known as byeager_away | 02:38 | |
| *** byeager_away is now known as byeager | 02:39 | |
| *** byeager_ has joined #openstack-swift | 02:44 | |
| *** byeager is now known as byeager_away | 02:45 | |
| *** bkopilov has quit IRC | 02:46 | |
| *** byeager_ has quit IRC | 02:46 | |
| *** byeager_away is now known as byeager | 02:46 | |
| *** byeager is now known as byeager_away | 02:47 | |
| anticw | dfg: do you know much about how authorize(...) is called for xLO segments? | 03:13 |
|---|---|---|
| anticw | specifically i see a situation where a user has permissions for the DLO but not the segment ... authorized gets called for a segment (i assume) and environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed' | 03:15 |
| notmyname | anticw: that's been under development recently. specifically to answer the question of what happens when a token expires in the middle of serving a *LO | 03:16 |
| anticw | that i'm less worried about | 03:17 |
| notmyname | anticw: but in general, the user needs read on the manifest and read/listings on the segments container IIRC | 03:17 |
| anticw | the issue i have is a non-admin user who has access to the manifest object but not the segments ... we end up in code paths that explode stack-traces | 03:17 |
| anticw | notmyname: sure, so where is the code that walks the manifest and calls into authorize? | 03:18 |
| anticw | :get_or_head_response | 03:18 |
| notmyname | I think the proxy object controller GETorHEAD | 03:19 |
| anticw | i guess from _get_container_listing | 03:19 |
| anticw | dlo is done as middleware magically inserted into the pipeline | 03:19 |
| notmyname | oh yeah. sorry I don't have the code in front of me at the moment | 03:19 |
| anticw | so afaict dlo.py : handle_request -> get_or_head_response | 03:20 |
| *** zhiyan is now known as zhiyan_ | 03:20 | |
| portante | anticw: what version of swift are you talking about? | 03:20 |
| anticw | 1.13.1 | 03:21 |
| portante | great ... | 03:22 |
| anticw | dlo.py : GetContext : _get_container_listing does con_resp = con_req.get_response(self.dlo.app) | 03:22 |
| anticw | will that be done with the same acl/security behavior for non-admin as the actual dlo/slo fetch itself? | 03:23 |
| anticw | it's entirely possible this is a local thinko here ... i'm trying to get someone to test older swift | 03:23 |
| portante | so, perhaps we should start with the stack traces you are seeing, can you provide any examples? | 03:24 |
| anticw | i can show you where i think it would also blow-up with keystone if that's useful | 03:24 |
| portante | sure | 03:24 |
| anticw | ./swift/common/middleware/keystoneauth.py , see def authorize(...) | 03:25 |
| anticw | env_identity = self._integral_keystone_identity(env) | 03:25 |
| anticw | look at def _integral_keystone_identity | 03:25 |
| portante | okay, why would that blow up? | 03:25 |
| anticw | the very first non-comment if environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed' | 03:25 |
| anticw | for a dlo segment afiact environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed' | 03:26 |
| *** zhiyan_ is now known as zhiyan | 03:26 | |
| anticw | so we hit the return right away | 03:26 |
| anticw | (i'm a bit out on a limb here i admit ... i don't have an env to confirm this) | 03:26 |
| anticw | and if we return (naked) it's None | 03:26 |
| anticw | so back in authorize(...) we then go and do tenant_id, tenant_name = env_identity['tenant'] | 03:26 |
| anticw | where env_identity = None | 03:26 |
| *** haomaiw__ has quit IRC | 03:27 | |
| *** haomaiwang has joined #openstack-swift | 03:27 | |
| anticw | portante: what's not clear to me is how the auth stuff works for dlo's | 03:28 |
| portante | so right now, we say place dlo/slo to the left of auth middleware in the pipeline, afaict | 03:28 |
| anticw | segments still have to be authenticated | 03:29 |
| portante | so that means each sub-request made my dlo/slo will be checked for authentication and authorization | 03:29 |
| anticw | or else i can create a manifest pointing to data that i don't have access to | 03:29 |
| anticw | yes | 03:29 |
| portante | the proxy server app does the authorization | 03:29 |
| portante | so the middleware, once authenticated, places an env.authorize method for the proxy server to invoke at the proper time | 03:29 |
| anticw | yes, so back at 20:22 (scroll-back) i think that's the container get to list the dlo elements ? | 03:30 |
| anticw | and it's not clear what the auth situation is there ... does con_req.get_response(...) look and smell enough like the original request that the authentication works as for the original object? | 03:30 |
| portante | yes, it tries to do that | 03:31 |
| anticw | so then environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed' or == ? | 03:31 |
| portante | if this is a new request, one where tempauth or keystoneauth has not seen before, it won't have an x-auth-token header on the request, so it won't be able to operate on the DLO object | 03:31 |
| portante | once the client has authenticated, each DLO REST API invocation using the x-auth-token previously received by the client should work just fine | 03:32 |
| anticw | except the stacktrace shows we get None .... which makes me think environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed' | 03:33 |
| portante | can you share that stacktrace? | 03:34 |
| anticw | sorta, sec... tenant_id, tenant_name = env_identity['tenant']#012TypeError: 'NoneType' object has no attribute '__getitem__' | 03:35 |
| anticw | is the relevant part | 03:35 |
| portante | filename and line number, if you have it? | 03:36 |
| portante | keystoneauth.py | 03:36 |
| portante | line ... | 03:36 |
| anticw | line 183 | 03:36 |
| anticw | sorry, i tried explaning this above but probably didn't put enough detail | 03:36 |
| portante | that just looks like a but in keystoneauth.py | 03:37 |
| portante | bug | 03:37 |
| anticw | but as you indicated if the container listing get is done as for the original manifest get ... won't HTTP_X_IDENTITY_STATUS be set correctly? | 03:38 |
| portante | So DLO only copies over the X-AUTH-TOKEN ... | 03:40 |
| anticw | ah yes, i see what you mean | 03:41 |
| portante | so keystoneauth.py has a bug, it should not stack trace there, if no x-identity-status exits, the request should be denied, not a stacktrace | 03:41 |
| anticw | portante: that won't work | 03:41 |
| anticw | i did that initially | 03:41 |
| anticw | so what then i get 403 not 500 for when i don't have perms | 03:41 |
| anticw | but when i *do* have perms i also get 403 | 03:41 |
| torgomatic | I believe there is a patch proposed to fix that by clever application of functools.partial, but it has not been approved | 03:41 |
| anticw | so we need to copy over more env ... let me quickly try that now | 03:41 |
| portante | DLO somehow needs to copy all the headers from the original request so that it includes what the client | 03:41 |
| portante | torgomatic: do you know what patch name or number? | 03:42 |
| torgomatic | Nope | 03:42 |
| portante | anticw: let's look at the patch torgomatic is talking about, sec ... | 03:42 |
| torgomatic | I'm on a smartphone at the moment so gerrit is basically pure annoyance | 03:42 |
| *** nosnos has quit IRC | 03:43 | |
| notmyname | torgomatic: that doesn't sound very "smart" | 03:43 |
| portante | torgomatic: could you be talking about https://review.openstack.org/92165, "xLO bug with auth tokens expiring during download" | 03:43 |
| torgomatic | notmyname: to be fair, neither do I much of the time ;) | 03:44 |
| torgomatic | Yeah, i think that's the patch. | 03:44 |
| anticw | gerrit is pretty nasty on a phone | 03:45 |
| anticw | + environ['swift.authorize'] = functools.partial( | 03:45 |
| anticw | + self.authorize, env_identity) | 03:45 |
| anticw | so in this case the dlo segment stuff happens after the auth middleware ... but the callbacks from the original callback? | 03:47 |
| portante | anticw: so yes, in that patch you move dlo to right of all the auth middleware, and the authorize() method is a closure over the original identity information | 03:49 |
| portante | avoiding any changes to the proxy server to support | 03:50 |
| anticw | awesome, i'm going to try that in a few | 03:50 |
| portante | the change to the proxy-server is to place dlo after known auth middleware when it is not specified | 03:50 |
| anticw | thanks very much | 03:50 |
| portante | I hope it works! :) | 03:50 |
| anticw | me too! | 03:50 |
| portante | ping dfg with feedback on it, as well | 03:51 |
| anticw | i'm not sure if it matters, but that patch doesn't consider keystone doing anon authentication | 04:08 |
| zaitcev | I didn't realize Keystone even had such a thing. | 04:09 |
| anticw | it changes environ['swift.authorize'] for known identity ... and the arguments to the function then ... but none for anon | 04:09 |
| *** madhuri has joined #openstack-swift | 04:09 | |
| portante | anticw: doesn't it use a different auth method then? | 04:13 |
| portante | self.authorize vs self.authorize_anonymous | 04:13 |
| anticw | no, it should be ok ... for the anon method we don't change the auth function | 04:13 |
| anticw | right, but i don't know when self.authorize_anonymous is used ... it won't be affected in this case | 04:13 |
| *** bkopilov has joined #openstack-swift | 04:13 | |
| anticw | though ... i hope that doesn't mean something like anon access for xLOs aren't checked | 04:13 |
| portante | it is used when there is no identity | 04:13 |
| portante | so this is a two phase kind-a thing, right? | 04:14 |
| anticw | right ... so we're sure this doesn't allow a rogue xLO to provide access when there is no identity aren't we? | 04:14 |
| portante | the middleware is invoked via __call__ when the pipeline is being processed | 04:14 |
| portante | but the swift.authorize environment method is invoked later when the proxy-server's __call__ method is invoked at the end of the pipeline | 04:15 |
| portante | so that in that patch, keystoneauth's __call__ method is setting up self.authorize() to always have a good "identity" objectd, if one was present to begin with, otherwise it uses the self.authorize_anonymous() method | 04:16 |
| *** kashyapk has joined #openstack-swift | 04:22 | |
| *** nosnos has joined #openstack-swift | 04:24 | |
| *** kashyapk has quit IRC | 04:26 | |
| *** ppai has joined #openstack-swift | 04:26 | |
| *** kashyapk has joined #openstack-swift | 04:26 | |
| *** nshaikh has joined #openstack-swift | 05:02 | |
| *** gyee has quit IRC | 05:38 | |
| *** kashyapk has quit IRC | 05:50 | |
| *** kashyapk has joined #openstack-swift | 05:50 | |
| *** Longgeek_ has joined #openstack-swift | 06:06 | |
| *** kashyapk has quit IRC | 06:08 | |
| *** kashyapk has joined #openstack-swift | 06:08 | |
| *** nthacker has joined #openstack-swift | 06:11 | |
| nthacker | hi guys, i wanted to know if there's a reason that FakeLogger in the unit test framework doesnt have a working flush to clear the FakeLogger's buffer | 06:12 |
| nthacker | it seems i can use close, but close sounds like I shouldnt use it again | 06:12 |
| nthacker | although the close routine doesnt appear to stop me from using it again | 06:13 |
| nthacker | basically i want to call FakeLogger::_clear | 06:13 |
| *** jamie_h has joined #openstack-swift | 06:26 | |
| *** openfly has quit IRC | 06:27 | |
| *** mmcardle has joined #openstack-swift | 06:51 | |
| *** nosnos has quit IRC | 06:59 | |
| *** matsuhashi has quit IRC | 07:01 | |
| *** matsuhas_ has joined #openstack-swift | 07:01 | |
| *** mkollaro has joined #openstack-swift | 07:02 | |
| *** kashyapk has quit IRC | 07:03 | |
| *** kashyapk has joined #openstack-swift | 07:03 | |
| *** nosnos has joined #openstack-swift | 07:05 | |
| *** kashyapk has quit IRC | 07:08 | |
| mattoliverau | Night all | 07:12 |
| *** zaitcev has quit IRC | 07:16 | |
| *** foexle has joined #openstack-swift | 07:17 | |
| *** roock has quit IRC | 07:30 | |
| *** kashyapk has joined #openstack-swift | 07:36 | |
| *** kashyapk has quit IRC | 07:56 | |
| *** kashyapk has joined #openstack-swift | 07:56 | |
| *** kashyapk has quit IRC | 08:01 | |
| *** occupant has quit IRC | 08:04 | |
| *** occupant has joined #openstack-swift | 08:04 | |
| *** kashyapk has joined #openstack-swift | 08:06 | |
| *** nthacker has quit IRC | 08:08 | |
| *** openfly has joined #openstack-swift | 08:24 | |
| *** matsuhas_ has quit IRC | 08:38 | |
| *** kashyapk has quit IRC | 08:38 | |
| *** kashyapk has joined #openstack-swift | 08:39 | |
| *** nosnos has quit IRC | 08:40 | |
| *** mkollaro has quit IRC | 08:40 | |
| *** nosnos has joined #openstack-swift | 08:43 | |
| *** nosnos has quit IRC | 08:44 | |
| *** matsuhashi has joined #openstack-swift | 08:44 | |
| *** mlipchuk has joined #openstack-swift | 08:52 | |
| *** nosnos has joined #openstack-swift | 09:03 | |
| *** zhiyan is now known as zhiyan_ | 09:18 | |
| *** ahale_ has quit IRC | 09:31 | |
| *** ahale has joined #openstack-swift | 09:32 | |
| *** nacim has joined #openstack-swift | 09:35 | |
| *** dmorita has quit IRC | 09:41 | |
| *** kashyapk has quit IRC | 09:49 | |
| *** kashyapk has joined #openstack-swift | 09:50 | |
| *** kashyapk has quit IRC | 09:54 | |
| *** jamie_h has quit IRC | 10:17 | |
| *** jamie_h has joined #openstack-swift | 10:18 | |
| *** omame has quit IRC | 10:28 | |
| *** jamie_h_ has joined #openstack-swift | 10:29 | |
| *** omame has joined #openstack-swift | 10:29 | |
| *** jamie_h has quit IRC | 10:32 | |
| *** kashyapk has joined #openstack-swift | 10:34 | |
| *** ekarlso has quit IRC | 10:57 | |
| *** ekarlso has joined #openstack-swift | 10:58 | |
| *** dmsimard_away is now known as dmsimard | 11:17 | |
| *** kashyapk has quit IRC | 11:36 | |
| *** kashyapk has joined #openstack-swift | 11:37 | |
| *** miqui has quit IRC | 11:38 | |
| *** diegows has joined #openstack-swift | 11:41 | |
| *** mkollaro has joined #openstack-swift | 11:54 | |
| *** ppai has quit IRC | 11:58 | |
| *** mkollaro1 has joined #openstack-swift | 12:00 | |
| *** mkollaro has quit IRC | 12:00 | |
| *** shakayumi has joined #openstack-swift | 12:10 | |
| *** kashyapk has quit IRC | 12:13 | |
| *** kashyapk has joined #openstack-swift | 12:13 | |
| *** kashyapk has quit IRC | 12:19 | |
| *** Longgeek_ has quit IRC | 12:25 | |
| *** mmcardle has quit IRC | 12:35 | |
| *** Ju has quit IRC | 12:35 | |
| *** mmcardle has joined #openstack-swift | 12:46 | |
| openstackgerrit | Peter Portante proposed a change to openstack/swift: xLO bug with auth tokens expiring during download. https://review.openstack.org/92165 | 12:48 |
| portante | dfg: ^^^ just a couple fixups on the commit message | 12:50 |
| *** mkollaro has joined #openstack-swift | 12:52 | |
| *** kashyapk has joined #openstack-swift | 12:53 | |
| *** mkollaro1 has quit IRC | 12:56 | |
| *** bkopilov has quit IRC | 12:59 | |
| *** miqui has joined #openstack-swift | 13:05 | |
| *** mlipchuk has quit IRC | 13:29 | |
| *** matsuhashi has quit IRC | 13:39 | |
| *** judd7 has joined #openstack-swift | 13:40 | |
| *** kashyapk has quit IRC | 13:41 | |
| *** grapsus_ is now known as grapsus | 13:49 | |
| *** grapsus is now known as grapsus_ | 13:50 | |
| *** nosnos has quit IRC | 13:50 | |
| *** mlipchuk has joined #openstack-swift | 13:51 | |
| *** Ju_ has joined #openstack-swift | 14:06 | |
| creiht | notmyname, clayg: What is the current timeline look like for policies? | 14:14 |
| *** miqui has quit IRC | 14:23 | |
| *** miqui has joined #openstack-swift | 14:23 | |
| *** miqui has quit IRC | 14:23 | |
| *** miqui has joined #openstack-swift | 14:23 | |
| *** mlipchuk has quit IRC | 14:24 | |
| *** donagh has joined #openstack-swift | 14:27 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 14:40 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 14:40 |
| *** NM has joined #openstack-swift | 14:40 | |
| *** mlipchuk has joined #openstack-swift | 14:51 | |
| *** diegows has quit IRC | 14:52 | |
| openstackgerrit | paul luse proposed a change to openstack/swift: Mock time for audit test 'test_sleeper' https://review.openstack.org/98538 | 15:03 |
| notmyname | creiht: current plan: merge this week and then RC for 2 weeks. I'm working the non-code parts for this now | 15:04 |
| creiht | cool | 15:04 |
| notmyname | I'm hoping that the 2-dimentional time patch will be proposed today, so please look at it when you can :-) | 15:05 |
| creiht | ahh cool, any other outstanding patches besides that | 15:06 |
| notmyname | SP related? are you comfortable with the rest that's there? | 15:08 |
| *** kevinc_ has joined #openstack-swift | 15:09 | |
| creiht | notmyname: yeah SP related... just wanted to see if there are any other outstanding things coming in before I made another pass through | 15:10 |
| openstackgerrit | paul luse proposed a change to openstack/swift: Change assertCalledWith to assert_called_with https://review.openstack.org/99124 | 15:11 |
| *** mkollaro has quit IRC | 15:12 | |
| *** mkollaro has joined #openstack-swift | 15:13 | |
| *** mlipchuk has quit IRC | 15:23 | |
| *** nshaikh has quit IRC | 15:30 | |
| *** gyee has joined #openstack-swift | 15:31 | |
| *** mkollaro has quit IRC | 15:35 | |
| *** nacim has quit IRC | 15:43 | |
| clayg | creiht: torgomatic and yuan have some updates to the bin scripts (get-object-info and swift-get-nodes) | 15:43 |
| *** kevinc_ has quit IRC | 15:53 | |
| *** byeager_away is now known as byeager | 15:53 | |
| acoles | notmyname: or clayg: which patch in the chain will the 2D time stuff show up in? or is it a new patch? | 15:58 |
| *** kevinc_ has joined #openstack-swift | 16:00 | |
| *** mlipchuk has joined #openstack-swift | 16:01 | |
| clayg | acoles: i haven't really decided yet | 16:02 |
| clayg | acoles: suggestion? | 16:02 |
| clayg | it's all over the map -> https://gist.github.com/clayg/66bb3c049b629fb0eca6 | 16:03 |
| acoles | clayg: nah, i just want to make sure i look out for it | 16:03 |
| *** byeager is now known as byeager_away | 16:09 | |
| * clayg was hoping acoles had a plan since this whole thing is his fault anyway | 16:09 | |
| * peluse_ smiles | 16:09 | |
| acoles | clayg: i need more time :) | 16:10 |
| clayg | peluse_: well not the *whole* thing - giant big portions of this are peluse_'s fault | 16:10 |
| clayg | acoles: but at least he bought me a beer | 16:10 |
| peluse_ | and that's really what its all about! | 16:10 |
| acoles | clayg: come to uk and i'll buy you beer | 16:12 |
| peluse_ | hey, I'm in! | 16:12 |
| * acoles doesn't mention that beer here comes warm | 16:12 | |
| clayg | after getting probetests and functests passing with the new timestamp format - does anyone want to guess how many unittest failures are left? | 16:15 |
| peluse_ | double digits or more? | 16:16 |
| *** shakayumi is now known as shakayumi_afk | 16:17 | |
| clayg | ~60 if i force the new format even when the offset is 0 and equal to the normalized form - but surprisingly only 16 if i treat the zero offset as the normalized form (which is what'd you do for upgrades) | 16:18 |
| clayg | then there's gunna have to be an audit | 16:18 |
| *** shakayumi_afk has quit IRC | 16:19 | |
| *** diegows has joined #openstack-swift | 16:19 | |
| *** mwstorer has joined #openstack-swift | 16:22 | |
| *** mwstorer has quit IRC | 16:23 | |
| anticw | is there a (simple) way to run a subset of the tests against a real swift cluster? | 16:25 |
| clayg | anticw: functests run against a real cluster | 16:26 |
| clayg | anticw: you can do standard nose test selection at that point | 16:26 |
| *** diegows has quit IRC | 16:26 | |
| anticw | clayg: awesome, i was hoping you would say that | 16:27 |
| clayg | anticw: but functests are fast - you should run all of them | 16:27 |
| anticw | running them all would be fine, i'll poke about and see if i can get them working | 16:28 |
| *** diegows has joined #openstack-swift | 16:38 | |
| *** shri has joined #openstack-swift | 16:39 | |
| portante | notmyname, clayg: hopefully SP patches won't be merged today or wednesdays, as I am behind here at the office reviewing them, and in general because we are out for the second half of the day celebrating the RHEL 7 launch | 16:44 |
| notmyname | portante: oh I see how it is. go party instead of get storage policies reviewed ;-) | 16:50 |
| portante | yes! glad I communicated that well! | 16:50 |
| portante | ;) | 16:50 |
| torgomatic | is that sarcasm or instructions? because I'm 100% on-board with those being instructions. | 16:50 |
| peluse_ | lord knows I'll do my part :) | 16:51 |
| notmyname | I've been chatting with mordred about how to get the stuff merged. I think we won't be clicking merge today or tomorrow, but targeting thursday (with the flexibility of friday or monday) is reasonable I think | 16:53 |
| *** NM has quit IRC | 16:53 | |
| notmyname | and we'll have an upstream branch that clayg will propose the stuff to after the patches are reviewed. then we'll only gate the merge commit instead of all the 27 patches | 16:53 |
| notmyname | mordred: ^^ please confirm :-) | 16:54 |
| acoles | notmyname: sounds good | 16:54 |
| notmyname | so far, we've got soft QA commitments from HP, RAX, Red Hat, softlayer (thanks briancline!), and I'll be reaching out to the NeCTAR guys today | 16:55 |
| notmyname | peluse_: mlanner and erik are putting the new mobos in right now | 16:57 |
| peluse_ | notmyname: sweet | 16:58 |
| notmyname | peluse_: they aren't cursing your name too much ;-) | 16:59 |
| *** shri1 has joined #openstack-swift | 17:01 | |
| *** shri1 has quit IRC | 17:02 | |
| mordred | notmyname: yes. the actually mechanics may vary slightly, but that's the idea | 17:03 |
| *** shri has quit IRC | 17:04 | |
| *** nsquare has joined #openstack-swift | 17:09 | |
| notmyname | ack | 17:11 |
| peluse_ | notmyname: remind them that I'm just a SW guy.. :) | 17:11 |
| notmyname | lol | 17:11 |
| *** judd7 has quit IRC | 17:18 | |
| *** NM has joined #openstack-swift | 17:22 | |
| *** tdasilva has joined #openstack-swift | 17:30 | |
| clayg | i'm having a bit of exestential crisis with the timestamp tests... there's lots of tests that send in denormalized x-timestamps and the assert they get them back - which is sorta ok, but then like i want to make sure those methods get called for realzy's with normalized timestamps - or force them to re-normalize "just in case" | 17:34 |
| clayg | which seems wasteful | 17:34 |
| clayg | maybe better safe than sorry? | 17:35 |
| clayg | i'll think about it on the train - bbiab | 17:35 |
| *** mmcardle has quit IRC | 17:36 | |
| *** bkopilov has joined #openstack-swift | 17:49 | |
| kevinc_ | When expanding into a new rack, is it common practice to add the new servers that will be in that rack into a new zone? | 17:51 |
| *** haomaiwang has quit IRC | 17:52 | |
| notmyname | kevinc_: yes that's common. but do whatever is right for your deployment. it's common to treat a rack as a zone in swift, since it normally is a physical failure domain (eg single ToR switch or power source) | 17:53 |
| kevinc_ | ok, thanks that is what I thought. Is there any downside to having 3 object copies over 5 zones? or is it best to expand it multiples of 3? | 17:55 |
| notmyname | kevinc_: no. keep three replicas (good price vs durability/availability metric) and have as many zones as you have failure domains | 17:57 |
| notmyname | kevinc_: in fact, swift gets better if you have more zones than replicas | 17:57 |
| notmyname | (in both statements above, I'm not considering global clusters) | 17:58 |
| notmyname | kevinc_: swift doesn't have any preferences for having a multiple of the replica count number of zones | 17:59 |
| kevinc_ | Perfect, thank you very much that was exactly what I needed to know :) | 17:59 |
| notmyname | so github is having some problems | 18:07 |
| *** kevinc_ has quit IRC | 18:11 | |
| *** kevinc_ has joined #openstack-swift | 18:16 | |
| *** shri has joined #openstack-swift | 18:21 | |
| *** mlipchuk has quit IRC | 18:28 | |
| clayg | boo github | 18:36 |
| *** zaitcev has joined #openstack-swift | 18:43 | |
| *** ChanServ sets mode: +v zaitcev | 18:43 | |
| anticw | portante / dfg : i'm using a slight variant of the xLO fixes as proposed in a test lab and so far it's looking very good | 18:54 |
| anticw | notmyname: i'm thinking we should try to get this in for 1.13.2 - as it stands we have a regression between 1.12.0 and 1.13.x | 18:54 |
| notmyname | ack | 18:55 |
| *** gyee has quit IRC | 19:23 | |
| *** kr4zy has joined #openstack-swift | 19:30 | |
| kr4zy | anyone here have experience setting up haproxy for ssl encrypted swift? | 19:30 |
| notmyname | kr4zy: what's your question? | 19:31 |
| zaitcev | anticw: can you post the slight differences you mentioned? | 19:31 |
| * briancline would also be interested in the regression details | 19:32 | |
| kr4zy | I have two load balancers running haproxy. I have two proxy servers running Apache2 reverse proxy for SSL termination. I am trying to set up load balancing and HA for them. Both my HAProxies are using SSL Passthrough. Currently, I have to execute swift list twice to see the containers initially. I am not sure what is causing the problem. | 19:34 |
| kr4zy | here's my haproxy setting: https://gist.github.com/anonymous/08eae701569b753c4c96 | 19:37 |
| *** byeager has joined #openstack-swift | 19:40 | |
| briancline | kr4zy: silly question but I have to ask anyway: the sums for the account and container rings on both proxies match, right? | 19:40 |
| kr4zy | yeah | 19:40 |
| kr4zy | I also noticed that the client is always going to one haproxy. Is my configuration wrong? | 19:41 |
| briancline | my initial thought is that's probably due to sticking via payload_lv | 19:45 |
| briancline | since you're specifying round robin, are you sure you want to stick at all? | 19:47 |
| kr4zy | I am new to HAProxy. My goal is to set up some kind of load balancing and have HA | 19:48 |
| briancline | ahh, well I'd simplify your swift backend block - try commenting out the stick-table, stick, acl, tcp-request and tcp-response statements and reload | 19:53 |
| *** kevinc_ has quit IRC | 19:56 | |
| briancline | as for the container listing issue, do you see the same thing if you hit both of the proxies directly? | 19:57 |
| briancline | so uh... swift config vs. devstack config question - | 20:00 |
| *** NM has quit IRC | 20:00 | |
| *** NM has joined #openstack-swift | 20:01 | |
| briancline | the sample proxy config says to put formpost before the auth filter in the pipeline, however devstack's swift_config is actually placing it *after* tempauth/keystoneauth in the pipeline it sets up | 20:02 |
| notmyname | briancline: I'd defer to the sample config | 20:02 |
| briancline | notmyname: yeah, I trusted that more -- however with us having gating devstack jobs, I'm curious if we're overriding that or if it's perhaps gone undetected in the tests | 20:04 |
| notmyname | AFAIK there is no test that validates the order of the pipeline (handwavy about the gatekeeper) | 20:04 |
| portante | anticw: back, can you share your slight changes? | 20:06 |
| *** NM has quit IRC | 20:09 | |
| *** kevinc_ has joined #openstack-swift | 20:09 | |
| briancline | notmyname: sorry, I meant more on the functional test side -- do we do any formpost functional tests? I had a cursory glance but couldn't find any | 20:10 |
| briancline | I see the unit tests | 20:10 |
| portante | briancline: I don't think so, as the formpost middleware is not in the pipeline for the functional tests by default | 20:12 |
| portante | but I might be missing something | 20:12 |
| notmyname | that sounds right | 20:13 |
| *** NM has joined #openstack-swift | 20:13 | |
| openstackgerrit | Thiago da Silva proposed a change to openstack/swift: Refactoring functional tests https://review.openstack.org/92643 | 20:14 |
| *** byeager has quit IRC | 20:16 | |
| kr4zy | briancline: I am now using this https://gist.github.com/anonymous/a676df11d2350edfdb28. The container that I created seems to disappear after a haproxy service restart. | 20:17 |
| *** miqui has quit IRC | 20:20 | |
| *** jamie_h has joined #openstack-swift | 20:23 | |
| *** jamie_h_ has quit IRC | 20:25 | |
| *** byeager has joined #openstack-swift | 20:30 | |
| *** byeager_ has joined #openstack-swift | 20:30 | |
| notmyname | if you aren't currently looking at storage policy patches, take a look at torgomatic's small patch at https://review.openstack.org/#/c/98530/ | 20:32 |
| * notmyname is looking at the reviewed-but-not-merged patches and other patches that should be included in the release with storage policies | 20:33 | |
| *** goodes- has quit IRC | 20:33 | |
| *** MooingLemur has quit IRC | 20:33 | |
| *** esmute has quit IRC | 20:33 | |
| *** cschwede_ has quit IRC | 20:33 | |
| *** jokke_ has quit IRC | 20:33 | |
| *** jokke_ has joined #openstack-swift | 20:33 | |
| *** esmute has joined #openstack-swift | 20:33 | |
| *** MooingLemur has joined #openstack-swift | 20:33 | |
| notmyname | anticw: BTW, that xLO patch is reviewed, but hasn't been merged because of the soft freeze with SP. it will land for the next release with storage policies | 20:33 |
| *** goodes has joined #openstack-swift | 20:33 | |
| *** byeager has quit IRC | 20:34 | |
| *** cschwede has joined #openstack-swift | 20:35 | |
| *** byeager_ has quit IRC | 20:37 | |
| *** byeager has joined #openstack-swift | 20:37 | |
| *** jokke__ has joined #openstack-swift | 20:38 | |
| notmyname | also https://review.openstack.org/#/c/97828/ | 20:43 |
| notmyname | actually, just take a look at the bottom of https://wiki.openstack.org/wiki/Swift/PriorityReviews for a set of patches that needs one more review | 20:46 |
| notmyname | please don't merge them yet, but approve them if they are ready | 20:47 |
| notmyname | we'll have a merge fest between thursday and monday to get everything landed for the RC | 20:47 |
| *** gyee has joined #openstack-swift | 20:49 | |
| *** gyee has quit IRC | 20:51 | |
| *** gyee has joined #openstack-swift | 20:53 | |
| *** jokke_ has quit IRC | 20:54 | |
| *** jamie_h has quit IRC | 20:54 | |
| *** omame has quit IRC | 20:54 | |
| *** openstackgerrit has quit IRC | 20:54 | |
| *** hugokuo has quit IRC | 20:54 | |
| *** kevinc_ has quit IRC | 21:00 | |
| *** foexle has quit IRC | 21:03 | |
| *** kevinc_ has joined #openstack-swift | 21:04 | |
| *** omame has joined #openstack-swift | 21:11 | |
| *** openstackgerrit has joined #openstack-swift | 21:11 | |
| *** hugokuo has joined #openstack-swift | 21:11 | |
| clayg | wheeeee | 21:12 |
| *** yuan has quit IRC | 21:13 | |
| *** nthacker has joined #openstack-swift | 21:13 | |
| *** yuan has joined #openstack-swift | 21:13 | |
| *** nthacker_ has joined #openstack-swift | 21:19 | |
| nthacker_ | Hi All - i want to use unittest to verify if an exception is raised. So i would use assertRaises, but I have a problem here. maybe this is a basic python question | 21:20 |
| nthacker_ | the snippet of code is: | 21:20 |
| nthacker_ | except OSError as e: | 21:20 |
| nthacker_ | if e.errno != errno.ENOTDIR | 21:20 |
| nthacker_ | raise | 21:21 |
| nthacker_ | now to check if the exception is raised, it appears i have to inject an object that will enter into the if loop | 21:21 |
| nthacker_ | and the error that the object should generate cannot be an ENOTDIR | 21:21 |
| nthacker_ | it also cannot be an ENOENT | 21:21 |
| nthacker_ | so now how will i inject a different errno to therefore check if this exception is raised | 21:22 |
| zaitcev | set a scope variable | 21:22 |
| nthacker_ | zaitcev: I dont follow - could you elaborate | 21:22 |
| nthacker_ | or show me an example? | 21:22 |
| zaitcev | was_raised=False | 21:23 |
| zaitcev | except OSError as e: | 21:23 |
| zaitcev | was_raised=True | 21:23 |
| zaitcev | etc. | 21:23 |
| nthacker_ | zaitcev: the snippet of code that i showed is the one against which i am running a unittest. basically this is a suggestion from peluse_ - you can see it here: https://review.openstack.org/#/c/97148/ | 21:24 |
| nthacker_ | so if i was to add a scope variable to the snippet, i'd be modifying the code i am testing against | 21:25 |
| zaitcev | Okay. Why cannot you put raise OSError(errno.EPERM) into your mocker? | 21:27 |
| zaitcev | wait | 21:28 |
| nthacker | zaitcev, ah ok, so that requires i use python mock correct? im new to that too, but i did read something about it | 21:29 |
| nthacker | zaitcev, sorry im using multiple clients, but this is still me | 21:29 |
| *** nthacker_ has quit IRC | 21:30 | |
| *** NM has quit IRC | 21:30 | |
| anticw | notmyname: next release being? | 21:32 |
| notmyname | anticw: soon | 21:32 |
| notmyname | one of the numbers in the existing version string will get one bigger ;-) | 21:33 |
| anticw | portante: slight changes are required because we don't actually use keystone | 21:33 |
| anticw | notmyname: i meant are we talking 1.14.x which is a big jump ... or 1.13.2 ? | 21:33 |
| anticw | i think small fixes justify the latter | 21:33 |
| notmyname | anticw: the next release will include storage policies and the existing reviewed-but-not-yet-merged-because-of-freeze patches. the scope of the work in this release, as discussed in-person at the hackathon, will result in a 2.0 version string | 21:35 |
| notmyname | which I'm not yet broadly proclaiming | 21:35 |
| anticw | i can live with it either way - but i would argue that 1.13.x has regressed since 1.12 in this context | 21:36 |
| anticw | so surely there is reason to consider a 1.13.2? | 21:36 |
| zaitcev | nthacker: I'll be darned. You have to do raise OSError(errno.EPERM, "moo"), or else the first argument goes to strerr not errno. | 21:36 |
| notmyname | anticw: is the regression documented in a LP bug? | 21:36 |
| anticw | (i don't much care - neither outcome helps me directly, we've done our version of things and will sit on that for a bit) | 21:36 |
| anticw | notmyname: i don't think so - i'm not sure if anyone other than me hit it | 21:36 |
| anticw | though the xLO token-expiration i assume is | 21:37 |
| anticw | (diff bug, same fix) | 21:37 |
| notmyname | anticw: I'd appreciate it if you'd document it there. that patch references only https://bugs.launchpad.net/swift/+bug/1315133 | 21:37 |
| nthacker | zaitcev, im confused - are you proposing that the mock test raise the assert so the unittest may catch it? | 21:37 |
| notmyname | and having it in LP makes it helpful to others who might see it. also those trying to repro it | 21:38 |
| zaitcev | nthacker: what else could you be doing? My first answer presumed that you allowed it to happen naturally, but you didn't like it. | 21:38 |
| anticw | notmyname: i will in a few, working over the URL stream | 21:39 |
| notmyname | anticw: thanks | 21:40 |
| anticw | notmyname: i don't think commenting on the original bug makes sense, so unless you prefer that i think it's better to make a new bug | 21:40 |
| nthacker | zaitcev, OK maybe i have to work a bit more with my head down to understand how i can raise this assert using mock. The way I was trying to handle it, was by giving the target routine an object that would generate neither an ENOENT, nor an ENOTDIR. So anything in the list of errno's except the latter 2, however that seems more complicated than your suggestion | 21:40 |
| notmyname | anticw: agreed. a new bug seems the right thing | 21:41 |
| kr4zy | is it possible to implement ha for haproxy doing ssl passthrough? My current haproxy ssl passthrough doesn't failover to the second proxy server. | 21:47 |
| anticw | notmyname: https://bugs.launchpad.net/swift/+bug/1328699 | 21:48 |
| anticw | probably needs rewording a little | 21:48 |
| notmyname | anticw: is this only for keystone or does it affect tempauth too? | 21:50 |
| peluse_ | nthacker: I think what zaitcev was suggesting would be to mock lisdir just for the one test where you want to cover that LOC, obviously mocking the function under test itself would make no sense :) I can send you an example if you want | 21:50 |
| nthacker | peluse_, ah ok! that now makes sense | 21:51 |
| nthacker | peluse_, no i get it now - thanks for clearing that bit - was like an aha moment :) | 21:52 |
| anticw | notmyname: i think only keystone | 21:52 |
| anticw | notmyname: and probably ssauth :-) | 21:52 |
| notmyname | FUD | 21:53 |
| notmyname | ;-) | 21:53 |
| notmyname | anticw: the manifest container and the segments container have different ACLs? that's what triggers it? | 21:53 |
| notmyname | or is it that the request is now mangled such that the original creds aren't active and therefore it fails | 21:54 |
| anticw | actually, that's not required | 21:54 |
| anticw | i thought it was | 21:54 |
| peluse_ | nthacker: cool, no problem. only other small thing is that you'll only want your mocked func to raise when its passed the datadir_path. Let me know how it goes, have fun! | 21:54 |
| anticw | even if you have the same ACLs | 21:54 |
| anticw | the lookup-logic for keystone breaks for the segment access | 21:54 |
| anticw | details in the scroll-back ... i'm happy to go over it again but if people here prefer we can take this to /msg or elsewhere ? | 21:55 |
| *** kevinc_ has quit IRC | 21:56 | |
| notmyname | anticw: I'll check on my saio. (also, it's not like there is a flood of chatter in here) | 21:57 |
| anticw | notmyname: in short, dlo.py: https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L37 | 21:58 |
| anticw | some headers are copied, not all ... for the dlo segment access | 21:58 |
| anticw | https://github.com/openstack/swift/blob/master/swift/common/middleware/keystoneauth.py#L180 calls https://github.com/openstack/swift/blob/master/swift/common/middleware/keystoneauth.py#L135 | 21:58 |
| anticw | which has a check on HTTP_X_IDENTITY_STATUS - that won't exist in the case of a segment access | 21:59 |
| anticw | so that code does a naked return - which is a None ... authorize then tries to do: tenant_id, tenant_name = env_identity['tenant'] | 21:59 |
| anticw | and None['xxx'] blows up | 21:59 |
| anticw | lmk if you need more details, not sure how best to explain it tersely and accurately | 22:00 |
| notmyname | I think that was pretty good :-) | 22:02 |
| *** byeager has quit IRC | 22:05 | |
| *** byeager has joined #openstack-swift | 22:05 | |
| *** byeager has quit IRC | 22:09 | |
| *** byeager has joined #openstack-swift | 22:11 | |
| *** byeager has quit IRC | 22:14 | |
| *** byeager has joined #openstack-swift | 22:14 | |
| notmyname | torgomatic: ^^ FYI and please review | 22:17 |
| notmyname | anticw: what's the pipeline order you have? dlo before or after auth? | 22:21 |
| *** byeager has quit IRC | 22:26 | |
| *** byeager has joined #openstack-swift | 22:27 | |
| notmyname | we had 3 patches proposed for a one character patch to the proxy config man page | 22:30 |
| peluse_ | good attention to detail :) | 22:35 |
| *** kr4zy has quit IRC | 22:38 | |
| portante | anticw: the xLO patch fixes that as well | 22:40 |
| portante | that can no longer happen with that code | 22:40 |
| * notmyname has his head around it now (thanks for the patience torgomatic) and understands | 22:41 | |
| notmyname | and also, since this was a regression during icehouse and the bug is present in 1.13.1 (icehouse), I'm leaning towards backporting it. assuming the openstack gods are with me there | 22:42 |
| * portante gotta head home, will be available later | 22:42 | |
| *** kevinc_ has joined #openstack-swift | 22:43 | |
| *** dmsimard is now known as dmsimard_away | 22:43 | |
| *** byeager_ has joined #openstack-swift | 22:45 | |
| *** byeager_ has quit IRC | 22:46 | |
| *** byeager has quit IRC | 22:48 | |
| mattoliverau | Morning | 22:56 |
| notmyname | mattoliverau: good morning | 22:56 |
| notmyname | mattoliverau: thanks for doing so many reviews! | 22:56 |
| mattoliverau | notmyname: You welcome, In atlanta you guys mentioned you were in need of more reviews especially with all the storage policy code coming through. So thought I'd do what I can to help :) | 22:58 |
| mattoliverau | *your | 22:58 |
| notmyname | much appreciated. I know it's hard to participate when you're on the opposite time zone of most of us. please let me know how I can make it easier | 23:00 |
| mattoliverau | notmyname: thanks, it is and I really appreciate that! but at least I'm awake when most of you sleep, so once I'm up to speed of all things swift I'll be able to help in channel during the grave yard shift :P | 23:02 |
| notmyname | :-) | 23:03 |
| notmyname | torgomatic: https://pbs.twimg.com/media/Bn3MoZQCYAAvJmZ.jpg | 23:22 |
| *** kevinc_ has quit IRC | 23:39 | |
| notmyname | torgomatic: `git show c2744caac` and `git show f63b58f5b` you'd think we'd learn :-) | 23:42 |
| torgomatic | I KEEP TRYING :) | 23:42 |
| notmyname | anticw: FWIW, swiftstackauth isn't affected :-) | 23:47 |
| notmyname | Alex_Gaynor wins the award for vague commit messages ;-) `git show 032f0bfc7c6` and `git show 181e9496f` | 23:51 |
| * torgomatic is going to start using http://whatthecommit.com/ from now on | 23:52 | |
| Alex_Gaynor | notmyname: haha, my 9th grade english teacher once called me "the master of ambiguity" | 23:53 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
