| jamielennox | most of that's not hard to fix, just move the references, but i think that's the thing that gets dumped out to policy | 00:01 |
|---|---|---|
| *** phalmos has quit IRC | 00:03 | |
| ayoung | jamielennox, what is going on with RequestContext? How does it get populated? | 00:04 |
| ayoung | http://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/context.py#n50 is called with | 00:04 |
| ayoung | http://git.openstack.org/cgit/openstack/keystone/tree/keystone/middleware/auth.py#n169 | 00:05 |
| ayoung | How does it get the token data? | 00:05 |
| ayoung | Does that code assume it is set via authtoken middleware? | 00:06 |
| *** lbragstad has joined #openstack-keystone | 00:06 | |
| *** ChanServ sets mode: +o lbragstad | 00:06 | |
| jamielennox | hmm, yea, it still gets populated from there | 00:06 |
| jamielennox | well that's getting populated from auth_context | 00:06 |
| jamielennox | but it's supposed to do request_context = RequestContext.from_environment() | 00:07 |
| jamielennox | which gets the values from auth_token middleware | 00:07 |
| ayoung | but we are not using auth_token middleware | 00:07 |
| jamielennox | yes we are | 00:08 |
| ayoung | in keystone? | 00:08 |
| jamielennox | for > year | 00:08 |
| jamielennox | http://git.openstack.org/cgit/openstack/keystone/tree/keystone/middleware/auth.py#n36 | 00:08 |
| ayoung | ummmm I don't see it jamielennox | 00:09 |
| jamielennox | class AuthContextMiddleware(auth_token.BaseAuthProtocol): | 00:10 |
| ayoung | class AuthContextMiddleware(auth_token.BaseAuthProtocol): | 00:10 |
| jamielennox | it's subclassing auth_token | 00:10 |
| ayoung | got it | 00:10 |
| ayoung | I can't read | 00:10 |
| ayoung | http://git.openstack.org/cgit/openstack/keystone/tree/keystone/middleware/auth.py#n13 | 00:10 |
| jamielennox | Base just abstracts the actual fetch so we can read from the database instead | 00:10 |
| ayoung | ok....so how do I kill the tests that are all using the code I wrote.... | 00:11 |
| ayoung | Oh, maybe I just need to kill more code | 00:11 |
| jamielennox | so have a look at the current oslo_context.RequestContext object and remove the stuff we duplicate | 00:12 |
| ayoung | jamielennox, can I kill all the code like: request_context.user_id = auth_context.get('user_id') | 00:12 |
| jamielennox | yes, for example that has been redundant for ages now | 00:12 |
| jamielennox | change creation of the context to use from_environ() | 00:12 |
| jamielennox | that'll catch like 90% of the options | 00:13 |
| ayoung | jamielennox, care to write it and I'll review? | 00:13 |
| jamielennox | but i think there's a few things like trustee_id and stuff that auth_token doesn't handle | 00:13 |
| jamielennox | heh | 00:13 |
| jamielennox | hmm | 00:13 |
| ayoung | we can add it on to an extended context for now | 00:14 |
| jamielennox | i'll give it a go quickly | 00:14 |
| ayoung | cool. | 00:14 |
| *** thorst has joined #openstack-keystone | 00:16 | |
| ayoung | File "/opt/stack/keystone/.tox/py27/lib/python2.7/site-packages/oslo_policy/_checks.py", line 280, in __call__ | 00:20 |
| ayoung | if 'roles' in creds: | 00:20 |
| ayoung | TypeError: argument of type 'RequestContext' is not iterable | 00:20 |
| ayoung | hmmm | 00:20 |
| *** thorst has quit IRC | 00:21 | |
| jamielennox | don't pass the whole context to oslo_policy | 00:23 |
| jamielennox | there's a function context.to_policy_values() which gives you the dict | 00:23 |
| jamielennox | eventually there was the intent that all this would be wrapped up behind helpers but for now there's no direct dependency between these components that we can hide that bit | 00:24 |
| jamielennox | the advantage of to_policy_values being a function is it's something the service can overload | 00:25 |
| jamielennox | which is how we were handling deprecations | 00:25 |
| jamielennox | gah, how do you install the prereqs on mac | 00:36 |
| jamielennox | what's the equivalent of ldap-devel | 00:36 |
| lbragstad | wxy_: when you met up with sdague on the unified limit work, did he mention if regions were required at all? | 00:46 |
| *** thorst has joined #openstack-keystone | 00:49 | |
| *** thorst has quit IRC | 00:54 | |
| *** thorst has joined #openstack-keystone | 00:58 | |
| wxy_ | lbragstad: no, but I think it should be. A keystone may manage more than one Nova. How to distinguish their limits if no region is specified? | 00:59 |
| *** thorst has quit IRC | 00:59 | |
| lbragstad | wxy_: yeah - i think it makes sense, but theoretically, it's possible to have a service and a endpoint with a region, right? | 01:00 |
| *** thorst has joined #openstack-keystone | 01:06 | |
| wxy_ | lbragstad: yes. | 01:07 |
| lbragstad | wxy_: ok - i'm just walking through this, thinking about what happens if we require reqions for limits | 01:07 |
| lbragstad | that'd require a deployment to move to regions in order to have limits in keystone? | 01:08 |
| lbragstad | i wondering if that's too high a bar | 01:12 |
| ayoung | lbragstad, nah, that is fine | 01:13 |
| lbragstad | cc kmalloc ^ | 01:13 |
| ayoung | adding in a default region after the fact should be a minimally invasive procedure | 01:13 |
| wxy_ | lbragstad: good point. A deployment's endpoint can be None. | 01:13 |
| kmalloc | yeh | 01:13 |
| lbragstad | ayoung: yeah? | 01:14 |
| lbragstad | i suppose endpoints and services can be updated | 01:14 |
| ayoung | lbragstad, yep | 01:14 |
| jamielennox | can you create an endpoint without a region? | 01:14 |
| lbragstad | yeah - a few of us asked that same question today :) | 01:15 |
| lbragstad | does it make sense? maybe not.... | 01:15 |
| jamielennox | it'd probably confuse some client stuff - maybe? | 01:15 |
| lbragstad | i think most people just assume it to be required since so many deployment projects use one | 01:15 |
| jamielennox | well when we added the /regions API i thought we might have actually been enforcing that link | 01:16 |
| lbragstad | yeah | 01:18 |
| lbragstad | someone correct me if i'm wrong, but the only reason to make regoin required for a limit it so that we don't have to have unique IDs for limits, right? | 01:19 |
| kmalloc | lbragstad: wow, regions are 100% optional | 01:20 |
| kmalloc | https://github.com/openstack/keystone/blob/master/keystone/catalog/backends/sql.py#L74 | 01:20 |
| ayoung | If it does not break backwards compat, and it makes the abstractions easier moving forward, I think its a good idea. So long as you can get away iwth only specifying project id, and get the region id implied, I don;t think it will impact anyone | 01:20 |
| jamielennox | i expect it'd be because typically the service only exists in a region to know that info | 01:20 |
| kmalloc | ayoung: unfortunately... requiring regions would be incompatible | 01:20 |
| jamielennox | like nova in us-west doesn't know how to enforce anything globally | 01:20 |
| ayoung | kmalloc, not for everything, just for people that then wanted to do limites | 01:20 |
| ayoung | I really don't have a dog in this race...I just want jamielennox top make my oslo-context problems go away | 01:21 |
| *** thorst has quit IRC | 01:21 | |
| ayoung | hhmmm create_trust broken all over the place...bet that has to do with trustee_id | 01:22 |
| kmalloc | ayoung: i am getting tired of beating the drum of "please don't create things that change how keystone's API works based upon deployment choices" | 01:22 |
| kmalloc | but... ^^ that view applies to "we require regions if we want limit data" type feature adds. but i'm slowly getting to the "don't care, just implement it however" phase. | 01:23 |
| jamielennox | so if the 'service' user stuff we talked about in denver goes ahead then regions become really important and we'd need a way to start enforcing that link | 01:23 |
| lbragstad | jamielennox: so requiring regions for limits is the way to do that? | 01:24 |
| lbragstad | er - *a* way to do that? | 01:24 |
| kmalloc | now... i am 100% ok if we have limits require a region. | 01:24 |
| kmalloc | just changing behavior where say endpoints now require one... that is a no go imo | 01:25 |
| lbragstad | yeah - that'd be backwards incompatible | 01:25 |
| jamielennox | so i haven't read these specs, but i'd say there is a need to at least have them per region | 01:25 |
| jamielennox | and it depends on the mechanism as to whether global works at all | 01:25 |
| ayoung | kmalloc, I've been at the "don't care, just implement it however" phase for about 4 years now | 01:26 |
| kmalloc | jamielennox: use case for limits that are for the entire deployment (aka, you get 100 vms in each region)? would you just require non-region endpoints to be fixed? | 01:27 |
| jamielennox | so there is a definite use case for limit/region - which is basically defaulting to 0 to say that a user only has access to specific regions | 01:28 |
| lbragstad | yeah | 01:28 |
| jamielennox | but this all depends on whether keystone is the decision point (make a request, keystone says yes/no) or whether the service is fetching limits and doing the enforcement there | 01:29 |
| lbragstad | jamielennox: the later | 01:29 |
| jamielennox | decision point lets you be way more flexible | 01:29 |
| kmalloc | jamielennox: yeaH. | 01:29 |
| lbragstad | jamielennox: keystone would supply the information | 01:29 |
| lbragstad | and the service enforces it | 01:29 |
| kmalloc | jamielennox: i agree. you should be able to enforce limits on a specific region. | 01:29 |
| jamielennox | ok, the later means that if you have nova in west, south, and east they can't know about each other | 01:29 |
| jamielennox | and so you basically _must_ make it per region | 01:29 |
| lbragstad | ok | 01:30 |
| jamielennox | like nova-west only cares about quota for itself | 01:30 |
| lbragstad | right | 01:30 |
| kmalloc | i just was wondering for the "we want customers to have same limits in all regions" do you need to specify for each, or just a region-less limit? | 01:30 |
| *** nicolasbock has quit IRC | 01:30 | |
| jamielennox | because nothing is syncrhronizing the nova-global is enforced amongst the 3 | 01:30 |
| lbragstad | right | 01:30 |
| kmalloc | and then the edge case of regionless-endpoints | 01:30 |
| lbragstad | kmalloc: i would think if you want to enforce the same registered limit for each region | 01:31 |
| lbragstad | instead of having a global limit | 01:31 |
| kmalloc | so perhaps: Order of enforcement: [Project-Override] > [Region-specific] > [Non-Region] ? | 01:31 |
| jamielennox | i'd say config option that you deprecate having region as a string and start making it relate to a region object, and push limist as a requirement of that | 01:31 |
| kmalloc | jamielennox: oh god no. no no no no... you know what, whatever, make it config-deployer selection and change keyston's behavior based on that | 01:32 |
| * kmalloc is done arguing against those types of configs | 01:32 | |
| jamielennox | kmalloc: i super dislike deployer specific config options, however they are the only way of evolving an API like this | 01:33 |
| kmalloc | jamielennox: just make non-region endpoints unable to have a limit (we just don't enforce on it at all, regions are required in limits) | 01:33 |
| kmalloc | and deployers can update their broken endpoints to have a region | 01:34 |
| lbragstad | doesn't that essentially do the same thing | 01:34 |
| jamielennox | but the endpoint region is just a string currently | 01:34 |
| kmalloc | jamielennox: no in the DB it's a FK | 01:34 |
| jamielennox | i mean actually link it to something that you created via /regions API | 01:34 |
| jamielennox | kmalloc: oh? | 01:34 |
| kmalloc | yeah | 01:34 |
| jamielennox | oh - then nevermind me | 01:35 |
| kmalloc | https://github.com/openstack/keystone/blob/master/keystone/catalog/backends/sql.py#L72 | 01:35 |
| kmalloc | jamielennox: we just "autocreate" the region if a region_id (or region) is specified in the endpoint json | 01:35 |
| kmalloc | if no region_id or region element is specified you have and endpoint w/o a region | 01:35 |
| jamielennox | cool, i remember a bunch of pain around that stuff, i just thought it was still a string in json | 01:35 |
| jamielennox | oh, autocreate, ugh | 01:35 |
| kmalloc | yep. but we at least have reliable data on the backend /s :P | 01:36 |
| jamielennox | ok, understood, autocreate is kinda funky | 01:36 |
| jamielennox | (like this fish smells funky) | 01:36 |
| kmalloc | yeah. i only know this cause i just looked this all up | 01:36 |
| kmalloc | i think the autocreate was to "fix" the json thing | 01:36 |
| kmalloc | where you were dealing with a string | 01:37 |
| kmalloc | w/o breaking api contracts | 01:37 |
| jamielennox | yea | 01:37 |
| kmalloc | still smells bad. | 01:37 |
| kmalloc | but, eh.. *shrug* | 01:37 |
| jamielennox | ok, so yea, your options are to have a default region (it's already a hierarchy, default is just the root) or endpoint must have a region to have a limit | 01:37 |
| lbragstad | yeah | 01:38 |
| jamielennox | although good luck to whomever has to figure out the hierarchical component of limits | 01:38 |
| kmalloc | jamielennox: wait are regions hierarchical? | 01:38 |
| kmalloc | god. | 01:38 |
| kmalloc | i forgot about that | 01:38 |
| kmalloc | gross. | 01:38 |
| jamielennox | yep, that was a jaypipes drive by a few years ago | 01:38 |
| kmalloc | my solution: ignore region hierarchy for limits | 01:39 |
| kmalloc | really, i would say limits are for their region and only thier region... none of the hierarchy bits | 01:40 |
| jamielennox | so it's the same problem as which roles propogate and which don't and yea, i'd just ignore it for now | 01:40 |
| ayoung | hierarchies are fun | 01:41 |
| jamielennox | if you design them from the start | 01:41 |
| kmalloc | so, i also am ok if we implement a default root region that anything without a region gets assigned to | 01:41 |
| ayoung | jamielennox, I might actually have this context thing working here shortly | 01:41 |
| kmalloc | if that solves the issue(s) | 01:41 |
| lbragstad | kmalloc: like a migration? | 01:41 |
| kmalloc | lbragstad: basically db migration | 01:41 |
| jamielennox | ayoung: i've got the base done | 01:41 |
| *** gmann_afk is now known as gmann | 01:41 | |
| ayoung | jamielennox, tests passing? | 01:41 |
| ayoung | that is the PITA part | 01:41 |
| kmalloc | and the case where we use it is: fix any region-less endpoints | 01:41 |
| jamielennox | well, i'm trying to make the docker bits go to actually run the tests | 01:42 |
| kmalloc | lbragstad: and any time an endpoint has no region on create | 01:42 |
| jamielennox | got assigned a mac :( | 01:42 |
| ayoung | jamielennox, I mean unit tests | 01:42 |
| kmalloc | lbragstad: pretty straight forward. | 01:42 |
| lbragstad | yeah.. | 01:42 |
| kmalloc | jamielennox: run vms. don't try and do it all native containers :P | 01:42 |
| lbragstad | kmalloc: wxy_so that sounds like step one? | 01:42 |
| ayoung | jamielennox, what he said | 01:42 |
| jamielennox | ergh, really | 01:43 |
| kmalloc | jamielennox: really. | 01:43 |
| kmalloc | it is the easiest way (short of just installing linux on the hardware... it does work pretty darn well) | 01:43 |
| kmalloc | at least it did last time I did it. was impressive. | 01:43 |
| lbragstad | jamielennox: my old mac worked great as an ssh terminal | 01:44 |
| kmalloc | though thinkpads are still better linux boxes | 01:44 |
| lbragstad | kmalloc: ++ | 01:44 |
| jamielennox | i have a linux machine on ec2 because sometimes you actually need to get work done | 01:44 |
| jamielennox | but i don't have anything like this set up there | 01:44 |
| kmalloc | i have a linux box [trying to debug an issue with the i210 intel nic chipset] that manages a chunk of my home network for "getting things done" | 01:45 |
| lbragstad | wxy_: kmalloc if we do the whole migration/auto-create thing, we should probably hit up the operator list, yeah? | 01:46 |
| kmalloc | lbragstad: yeah probably. | 01:46 |
| kmalloc | we should be clear the only thing we are changing is the case where and endpoint does not have a region | 01:46 |
| wxy_ | ++ | 01:46 |
| lbragstad | right - so we'll create RegionOne iff there isn't a region in the deployment | 01:47 |
| kmalloc | i'd call it __DefaultRegion__ | 01:47 |
| lbragstad | er - $RegionOne lol | 01:47 |
| lbragstad | yeah | 01:47 |
| kmalloc | because you might have a case where $RegionOne is used but some endpoints aren't part of it | 01:47 |
| jamielennox | i'd try not to, becasuse that string ends up in the catalog | 01:48 |
| kmalloc | or whatever, pick a name | 01:48 |
| kmalloc | DefaultRegion | 01:48 |
| kmalloc | and services are already region agnostic | 01:48 |
| kmalloc | so, we don't need to change that | 01:48 |
| gmann | jamielennox: np. i now my name spell is complex :) | 01:50 |
| *** thorst has joined #openstack-keystone | 01:51 | |
| ayoung | jamielennox, anyways, when you get it working, put me on the review. I'll be looking for it. | 01:54 |
| openstackgerrit | Jamie Lennox proposed openstack/keystone master: Clean up RequestContext https://review.openstack.org/523650 | 01:56 |
| jamielennox | ayoung: going to test via zuul because i don't have the setup atm ^ | 01:57 |
| *** thorst has quit IRC | 01:57 | |
| ayoung | jamielennox, that works. post it and I'll run the tests here | 01:57 |
| openstackgerrit | Jamie Lennox proposed openstack/keystone master: Clean up RequestContext https://review.openstack.org/523650 | 01:58 |
| *** annp has joined #openstack-keystone | 01:59 | |
| ayoung | jamielennox, you missed the trust values | 02:00 |
| jamielennox | ayoung: where? i thought i left them assigned from middleware | 02:00 |
| ayoung | Ah...nope it was user_id | 02:00 |
| *** thorst has joined #openstack-keystone | 02:00 | |
| ayoung | http://paste.openstack.org/show/627655/ | 02:01 |
| ayoung | this line | 02:01 |
| ayoung | self.assertEqual(self.user['id'], req_context.user_id) | 02:01 |
| jamielennox | my laptop is about to die | 02:01 |
| jamielennox | and then i have to relocate | 02:01 |
| jamielennox | so if you don't see me push anything again assume i'm gone and just fix the little mistakes | 02:02 |
| ayoung | Heh | 02:03 |
| ayoung | little | 02:03 |
| ayoung | request_context = context.RequestContext.from_environ( does not fill out request_context.user_id | 02:03 |
| jamielennox | it should | 02:03 |
| jamielennox | because it goes via auth_token which adds it to environment | 02:04 |
| jamielennox | it's just whether the tests correctly set up that path | 02:04 |
| jamielennox | oh - and also | 02:04 |
| jamielennox | make sure you pull #2 | 02:04 |
| jamielennox | was dumb mistake that might have caused that | 02:04 |
| *** thorst has quit IRC | 02:05 | |
| jamielennox | all i get is: pkg_resources.ContextualVersionConflict: (osprofiler 1.1 (/keystone/.tox/py35/lib/python3.5/site-packages), Requirement.parse('osprofiler>=1.4.0'), {'keystone'}) | 02:05 |
| jamielennox | and i don't have time to debug that one | 02:05 |
| ayoung | yeah, that was 2 | 02:05 |
| jamielennox | ok, then need to take a deeper look | 02:07 |
| jamielennox | 1% | 02:07 |
| jamielennox | biab | 02:07 |
| jamielennox | oh, it goes to 0% | 02:07 |
| lbragstad | lol | 02:07 |
| lbragstad | #goodtoknow | 02:07 |
| *** aselius has quit IRC | 02:31 | |
| *** daidv has joined #openstack-keystone | 02:33 | |
| *** dave-mccowan has quit IRC | 02:42 | |
| *** threestrands has joined #openstack-keystone | 03:05 | |
| *** thorst has joined #openstack-keystone | 03:05 | |
| errr | Im using stable/newtown and trying to use `openstack token issue` but I keep getting __init__() takes exactly 7 arguments (6 given) | 03:07 |
| errr | I think my openrc is wrong but I am having trouble finding an example of a good openrc that uses saml2 federation | 03:07 |
| lbragstad | errr: do you have a trace? | 03:08 |
| errr | I have the output of doing that command with --debug | 03:08 |
| lbragstad | that works | 03:08 |
| errr | 1 sec while I make sure my password isnt in it before I link to it | 03:09 |
| errr | https://gist.github.com/michaelrice/3c1531eba7edfad8f57c212e1dc70181 | 03:10 |
| *** thorst has quit IRC | 03:10 | |
| errr | just an fyi the federation stuff does work using horizon. at this point im just trying to get it working from the cli now | 03:11 |
| *** swain has quit IRC | 03:13 | |
| lbragstad | the cli using federation? | 03:17 |
| errr | well thats what Im trying to do anyway | 03:17 |
| errr | without federtion it works, but when I try to use federation I get that error above | 03:17 |
| lbragstad | errr: here is an example using ksa to do federated auth https://docs.openstack.org/keystone/latest/advanced-topics/federation/federated_identity.html#testing-it-all-out | 03:19 |
| lbragstad | knikolla: or cmurphy might know more about that off the top of their head though | 03:19 |
| errr | https://gist.github.com/michaelrice/cf2c861c16a5311855a9a87a24e9b26b | 03:20 |
| errr | thats my openrc Im using | 03:20 |
| errr | Im using okta as my idp | 03:22 |
| jamielennox | ayoung: so the reason my patch fails is that the part of auth_token that populates the environment variables is not being used by keystone | 03:22 |
| jamielennox | so we can't use from_environment as there is no environment | 03:22 |
| jamielennox | i'm guessing there was a reason i didn't do that, but i have nfi what it was | 03:22 |
| lbragstad | jamielennox: is is possible to do federated authentication with osc? | 03:23 |
| jamielennox | should be | 03:23 |
| lbragstad | ok - that's what i though | 03:23 |
| lbragstad | thought* | 03:23 |
| jamielennox | shoudl just be a matter of setting up the auth plugin correclty | 03:23 |
| jamielennox | errr: that's an odd error that i feel like i've seen before | 03:25 |
| jamielennox | failing at that point is a bug - either in OSC or keystoneauht | 03:26 |
| jamielennox | my suggestion would be to put a print statement in at this line: | 03:27 |
| jamielennox | return self.plugin_class(**kwargs) | 03:27 |
| jamielennox | and see what the kwargs are it's trying to pass | 03:27 |
| errr | jamielennox: while digging through the keystoneauth code I feel like the saml2 plugin requires identity-provider-url to be set but after adding debugging logs to the base.py client that param is not being passed in and when doing openstack --help its also not listed as a valid param | 03:27 |
| errr | heh yeah thats where I added my debugging | 03:28 |
| jamielennox | so i'm concerned that the plugin is in a module called _saml2 | 03:28 |
| errr | yeah thats the one | 03:28 |
| jamielennox | which would indicate it's private | 03:28 |
| jamielennox | only reason i can think it's private is it wasn't working | 03:28 |
| errr | sad panda. | 03:29 |
| jamielennox | so what's the kwargs contain | 03:29 |
| errr | https://gist.github.com/michaelrice/848ff069e4ce48ffca49ee7f44eca338 | 03:32 |
| *** sapd_ has quit IRC | 03:33 | |
| jamielennox | so it looks like identity_provider_url is required but not marked as required | 03:34 |
| errr | yep | 03:35 |
| errr | and its being ignored when I pass it in | 03:35 |
| errr | Ive tried passing it in manually via --os-identity-provider-url and setting it as an env variable | 03:35 |
| *** sapd_ has joined #openstack-keystone | 03:36 | |
| *** thorst has joined #openstack-keystone | 03:36 | |
| errr | but its never one of the kwargs when it gets to that return self.plugin_class(**kwargs) line | 03:36 |
| openstackgerrit | Jamie Lennox proposed openstack/keystoneauth master: Mark SAML loader properties as required https://review.openstack.org/523675 | 03:40 |
| *** thorst has quit IRC | 03:40 | |
| jamielennox | hmm, so that's weird because OSC shouldn't be messing with those options | 03:41 |
| *** links has joined #openstack-keystone | 03:42 | |
| errr | addng your patch still doesnt help, it just gives me an error about the param missing despite me setting it | 03:46 |
| errr | something is keeping that param from being used | 03:46 |
| jamielennox | yea, that would just show it | 03:49 |
| jamielennox | i'm just setting up an OSC environment | 03:49 |
| jamielennox | haven't got something i can test with at the moment | 03:49 |
| openstackgerrit | Jamie Lennox proposed openstack/keystoneauth master: Mark SAML loader properties as required https://review.openstack.org/523675 | 03:58 |
| jamielennox | errr: are you running master? | 03:59 |
| errr | no, stable/newton but Im not sure what version of the openstack client or those libs it uses. let me check those | 04:01 |
| errr | keystoneauth1 (2.12.3), python-openstackclient (3.2.1) | 04:02 |
| jamielennox | so i've just loaded that up on osc and ksa master | 04:03 |
| jamielennox | and it's passing that point | 04:03 |
| jamielennox | i don't have a saml idp that i can actually test with but it's trying to make a connection | 04:03 |
| errr | what version of those do I need to install? I use openstack-ansible so Ill need to pin those versions and rerun some playbooks to update | 04:04 |
| jamielennox | errr: no idea on the actual version | 04:05 |
| jamielennox | but install the latest pip version into a venv and tell me if you still have the same problems | 04:05 |
| errr | ok. Ill try that | 04:06 |
| errr | same error | 04:09 |
| errr | the latest from pip from those 2 packages now: python-openstackclient (3.12.0) keystoneauth1 (3.2.0) | 04:11 |
| *** thorst has joined #openstack-keystone | 04:12 | |
| *** thorst has quit IRC | 04:17 | |
| *** d0ugal has quit IRC | 04:18 | |
| openstackgerrit | wangxiyuan proposed openstack/keystone master: Deprecate member_role_id and member_role_name https://review.openstack.org/522461 | 04:24 |
| *** d0ugal has joined #openstack-keystone | 04:25 | |
| *** threestrands has quit IRC | 04:29 | |
| *** phalmos has joined #openstack-keystone | 04:30 | |
| *** markvoelker has quit IRC | 04:30 | |
| *** threestrands has joined #openstack-keystone | 04:41 | |
| *** threestrands has quit IRC | 04:41 | |
| *** threestrands has joined #openstack-keystone | 04:41 | |
| *** thorst has joined #openstack-keystone | 04:44 | |
| *** thorst has quit IRC | 04:49 | |
| *** sticker has quit IRC | 05:08 | |
| *** dklyle has quit IRC | 05:15 | |
| *** thorst has joined #openstack-keystone | 05:21 | |
| *** thorst has quit IRC | 05:26 | |
| *** thorst has joined #openstack-keystone | 05:29 | |
| *** markvoelker has joined #openstack-keystone | 05:30 | |
| *** threestrands has quit IRC | 05:32 | |
| *** thorst has quit IRC | 05:34 | |
| *** phalmos has quit IRC | 05:35 | |
| *** thorst has joined #openstack-keystone | 05:57 | |
| *** thorst has quit IRC | 06:03 | |
| *** alex_xu has quit IRC | 06:10 | |
| *** alex_xu has joined #openstack-keystone | 06:11 | |
| *** sapd_ has quit IRC | 06:16 | |
| *** sapd_ has joined #openstack-keystone | 06:16 | |
| openstackgerrit | wangxiyuan proposed openstack/keystone master: Deprecate member_role_id and member_role_name https://review.openstack.org/522461 | 06:26 |
| *** thorst has joined #openstack-keystone | 06:29 | |
| *** thorst has quit IRC | 06:34 | |
| *** aojea has joined #openstack-keystone | 07:02 | |
| *** thorst has joined #openstack-keystone | 07:07 | |
| *** thorst has quit IRC | 07:11 | |
| *** aojea has quit IRC | 07:12 | |
| *** jaosorior has quit IRC | 07:33 | |
| *** jaosorior has joined #openstack-keystone | 07:35 | |
| *** thorst has joined #openstack-keystone | 07:42 | |
| *** rcernin has quit IRC | 07:43 | |
| *** thorst has quit IRC | 07:46 | |
| *** AlexeyAbashkin has joined #openstack-keystone | 07:47 | |
| *** pcaruana has joined #openstack-keystone | 08:04 | |
| cmurphy | errr: can you paste the full debug output? | 08:20 |
| *** thorst has joined #openstack-keystone | 08:21 | |
| cmurphy | this has worked for me (tested recently) http://paste.openstack.org/show/627672/ | 08:21 |
| *** thorst has quit IRC | 08:26 | |
| *** rcernin has joined #openstack-keystone | 08:39 | |
| *** thorst has joined #openstack-keystone | 08:54 | |
| *** jsheeren has joined #openstack-keystone | 08:56 | |
| jsheeren | hi all | 08:57 |
| jsheeren | i have a question about the catalog list | 08:57 |
| jsheeren | when we request a catalog list, we get the endpoints | 08:57 |
| jsheeren | all are configured on, lets say, https://thecatalog.list | 08:57 |
| jsheeren | so keystone is https://thecatalog.list:5000 and so on | 08:57 |
| jsheeren | how can i create a second domain name for the catalog list | 08:58 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/keystone master: Updated from global requirements https://review.openstack.org/523735 | 08:58 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/keystoneauth master: Updated from global requirements https://review.openstack.org/523736 | 08:58 |
| jsheeren | so keystone is also available on, lets say, https://thesecondcatalog.list:5000 | 08:58 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/keystonemiddleware master: Updated from global requirements https://review.openstack.org/523737 | 08:58 |
| *** thorst has quit IRC | 08:59 | |
| *** spectr has joined #openstack-keystone | 09:03 | |
| jsheeren | what i want to accomplish is, when users approach the api's using https://thesecondcatalog.list, and they request the catalog list, they will receive the endpoints with name https://thesecondcatalog.list instead of https://thecatalog.list | 09:04 |
| openstackgerrit | wangxiyuan proposed openstack/keystone master: Add new tables for unified limits https://review.openstack.org/523041 | 09:05 |
| cmurphy | jsheeren: you can create new endpoints on different interfaces (public, internal, admin) or you can create new endpoints in different regions | 09:12 |
| cmurphy | i'm not sure there's a way to make it look seamless like you describe | 09:12 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/oslo.policy master: Updated from global requirements https://review.openstack.org/523783 | 09:15 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/pycadf master: Updated from global requirements https://review.openstack.org/523791 | 09:16 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-keystoneclient master: Updated from global requirements https://review.openstack.org/523792 | 09:18 |
| jsheeren | cmurphy, ok thanks | 09:24 |
| *** thorst has joined #openstack-keystone | 09:24 | |
| *** thorst has quit IRC | 09:29 | |
| *** gmann is now known as gmann_afk | 09:39 | |
| *** thorst has joined #openstack-keystone | 09:55 | |
| *** thorst has quit IRC | 09:59 | |
| *** annp has quit IRC | 10:02 | |
| *** AlexeyAbashkin has quit IRC | 10:28 | |
| *** thorst has joined #openstack-keystone | 10:31 | |
| *** spectr has quit IRC | 10:33 | |
| *** thorst has quit IRC | 10:36 | |
| *** daidv has quit IRC | 10:39 | |
| *** pratapagoutham has joined #openstack-keystone | 10:47 | |
| *** mvk has quit IRC | 10:56 | |
| *** mvk has joined #openstack-keystone | 10:58 | |
| *** thorst has joined #openstack-keystone | 11:08 | |
| *** thorst has quit IRC | 11:13 | |
| *** AlexeyAbashkin has joined #openstack-keystone | 11:19 | |
| *** AlexeyAbashkin has quit IRC | 11:23 | |
| *** AlexeyAbashkin has joined #openstack-keystone | 11:27 | |
| *** thorst has joined #openstack-keystone | 11:30 | |
| *** thorst has quit IRC | 11:32 | |
| *** swain has joined #openstack-keystone | 11:47 | |
| *** mvk has quit IRC | 11:47 | |
| *** thorst has joined #openstack-keystone | 12:03 | |
| *** Tahvok has joined #openstack-keystone | 12:06 | |
| *** thorst has quit IRC | 12:07 | |
| *** thorst has joined #openstack-keystone | 12:18 | |
| *** mvk has joined #openstack-keystone | 12:19 | |
| *** thorst has quit IRC | 12:24 | |
| *** Tahvok has quit IRC | 12:32 | |
| *** nicolasbock has joined #openstack-keystone | 12:36 | |
| *** pratapagoutham has quit IRC | 12:39 | |
| *** raildo has joined #openstack-keystone | 12:39 | |
| *** threestrands has joined #openstack-keystone | 12:42 | |
| *** threestrands has quit IRC | 12:42 | |
| *** threestrands has joined #openstack-keystone | 12:42 | |
| *** thorst has joined #openstack-keystone | 12:55 | |
| *** threestrands has quit IRC | 13:01 | |
| *** markvoelker has quit IRC | 13:19 | |
| *** markvoelker has joined #openstack-keystone | 13:19 | |
| *** links has quit IRC | 13:21 | |
| *** edmondsw has joined #openstack-keystone | 13:32 | |
| *** dave-mccowan has joined #openstack-keystone | 13:36 | |
| openstackgerrit | Merged openstack/keystone master: Add explain of mapping group attribute https://review.openstack.org/493765 | 13:38 |
| *** zhurong has joined #openstack-keystone | 13:39 | |
| *** openstackstatus has quit IRC | 13:43 | |
| *** openstack has joined #openstack-keystone | 13:43 | |
| *** ChanServ sets mode: +o openstack | 13:43 | |
| *** jsheeren has quit IRC | 13:43 | |
| *** openstackstatus has joined #openstack-keystone | 13:44 | |
| *** ChanServ sets mode: +v openstackstatus | 13:44 | |
| *** panbalag has joined #openstack-keystone | 13:45 | |
| *** zhurong has quit IRC | 13:56 | |
| *** spilla has joined #openstack-keystone | 13:56 | |
| *** swain has quit IRC | 13:56 | |
| *** zhurong has joined #openstack-keystone | 13:58 | |
| lbragstad | o/ | 14:09 |
| cmurphy | \o | 14:10 |
| lbragstad | cmurphy: we had an interested discussion last night about the region id stuff with unified limits | 14:16 |
| cmurphy | lbragstad: hmm i guess i'll have to read the scrollback | 14:19 |
| cmurphy | unless you want to tldr it for me :) | 14:19 |
| lbragstad | well - we pretty much validated that regions are completely optional | 14:25 |
| ayoung | errr, I had SAML working with the CLI. It also requires an IdP that supports ECP. | 14:25 |
| lbragstad | so the question becomes, what do we do about deployments that don't have them but want to use limits | 14:25 |
| lbragstad | 1.) we provide a migration that introduces a default region iff there isn't one available | 14:26 |
| *** rcernin has quit IRC | 14:27 | |
| cmurphy | ayoung: that's what i was thinking but it's still weird for it to manifest as a missing cli argument | 14:27 |
| lbragstad | 2.) provide a configuration option to start requiring regions for services and endpoints (changing the API depending on deployments) | 14:27 |
| ayoung | cmurphy, let me see what I had for config | 14:28 |
| ayoung | its been a few years | 14:28 |
| ayoung | cmurphy, http://paste.openstack.org/show/627718/ | 14:28 |
| ayoung | lbragstad, that was my thinking, too. | 14:29 |
| ayoung | lbragstad, I started thinking along these lines: what if a deployment has a bunch of stuiff not in a region, and a few things in a few different regions. | 14:30 |
| ayoung | should we treat that unregioned stuff as the default region, or one layer down? | 14:31 |
| ayoung | like: nothing goes into the root region except other regions | 14:31 |
| cmurphy | ayoung: i'm not sure v3unscopedsaml still works, we have v3samlpassword http://git.openstack.org/cgit/openstack/keystoneauth/tree/setup.cfg#n58 | 14:32 |
| ayoung | or, does it make more sense to have the unregioned endpoints etc go into the root region, and then they would be Parent-owned resources | 14:32 |
| ayoung | cmurphy, I'm surwe you are right. That was from 3 years ago | 14:32 |
| ayoung | Oct 5 2015 | 14:32 |
| ayoung | 2 years, anyway | 14:32 |
| openstackgerrit | Merged openstack/keystone master: Add New in Pike note to using db_sync check https://review.openstack.org/523238 | 14:33 |
| lbragstad | i don't think i like the configuration option idea because it kinda shoots interop to crap and we've been trying to move away from things that change the API like that | 14:34 |
| lbragstad | but the other bit is - what do you specify as a default region? | 14:34 |
| lbragstad | because that will end up in the catalog | 14:35 |
| cmurphy | i'll need to go over the spec again because requiring regions when we don't currently sounds very heavy handed to me | 14:35 |
| lbragstad | that was another question i asked last night | 14:36 |
| lbragstad | why are reqions required for unified limits | 14:36 |
| cmurphy | right | 14:36 |
| lbragstad | would an alternative be to make them optional and add in a unique id | 14:37 |
| lbragstad | ? | 14:37 |
| lbragstad | are they only there because they are part of the unique key for a limit? | 14:37 |
| cmurphy | if that's the case couldn't 'None' be the region id? | 14:38 |
| lbragstad | maybe? | 14:39 |
| ayoung | None is the default region? Makes sense to me | 14:41 |
| ayoung | "My ID? You don't need to see MY ID. I am THE REGION!" | 14:41 |
| cmurphy | :) | 14:42 |
| *** zhurong has quit IRC | 14:46 | |
| lbragstad | I suppose that would work... | 14:46 |
| lbragstad | i need to white board some of that today when i review the spec | 14:46 |
| *** openstack has quit IRC | 14:46 | |
| *** openstack has joined #openstack-keystone | 14:47 | |
| *** ChanServ sets mode: +o openstack | 14:47 | |
| cmurphy | lbragstad: i was hoping i could get zane or mordred to ack it | 14:47 |
| lbragstad | cmurphy: ++ | 14:47 |
| lbragstad | that'd be good | 14:48 |
| cmurphy | also noticed an inconsistency in the roles property but can fix that in a followup | 14:48 |
| ayoung | cmurphy, I think we'd need to be able to do roles ids | 14:48 |
| ayoung | with domain specific roles, you might have some masking. | 14:50 |
| cmurphy | ayoung: ++ | 14:50 |
| cmurphy | ayoung: with trusts we do [{'id': XXX}] or [{'name': XXX}] so we should probably have the same thing here | 14:50 |
| lbragstad | yeah | 14:50 |
| ayoung | cmurphy, what happens if roles are specified for an app credential that the user does not have assigned? | 14:51 |
| ayoung | right now it reads | 14:51 |
| ayoung | `roles` is a list of role names or ids. It is informational and can be used by the | 14:52 |
| ayoung | Consumer to verify that the Application Credential inherited the roles from | 14:52 |
| ayoung | the User that the Consumer expected. This is not a policy enforcement, it is | 14:52 |
| ayoung | simply for human validation. | 14:52 |
| ayoung | (I made an edit) | 14:52 |
| cmurphy | ayoung: that's the response | 14:52 |
| cmurphy | ayoung: in the request it would error | 14:52 |
| ayoung | ++ | 14:52 |
| openstackgerrit | ayoung proposed openstack/keystone-specs master: Repropose application credentials to queens https://review.openstack.org/512505 | 14:53 |
| ayoung | cmurphy, if you +1 my changes, I'll +2 the proposal | 14:53 |
| ayoung | gah | 14:54 |
| ayoung | ok...I butchered another line...1 sec | 14:54 |
| openstackgerrit | ayoung proposed openstack/keystone-specs master: Repropose application credentials to queens https://review.openstack.org/512505 | 14:55 |
| ayoung | https://review.openstack.org/#/c/512505/8..10/specs/keystone/queens/application-credentials.rst | 14:55 |
| *** rderose has joined #openstack-keystone | 14:56 | |
| cmurphy | ayoung: let me make another quick edit | 14:56 |
| ayoung | ++ | 14:56 |
| *** d0ugal has quit IRC | 14:56 | |
| openstackgerrit | Colleen Murphy proposed openstack/keystone-specs master: Repropose application credentials to queens https://review.openstack.org/512505 | 14:58 |
| lbragstad | cmurphy: do roles {} have to have IDs and names? | 15:00 |
| *** Neptu has joined #openstack-keystone | 15:00 | |
| cmurphy | lbragstad: i'm using the trusts API as a reference, in a request it's either or and in the response it's both | 15:02 |
| lbragstad | ahh | 15:03 |
| lbragstad | ok | 15:03 |
| *** d0ugal has joined #openstack-keystone | 15:04 | |
| lbragstad | i was wondering if it would be tough for someone to figure that information out without being an admin | 15:07 |
| lbragstad | but it's all in the the token response, so... | 15:07 |
| *** d0ugal_ has joined #openstack-keystone | 15:17 | |
| *** d0ugal has quit IRC | 15:20 | |
| *** tlbr has joined #openstack-keystone | 15:28 | |
| *** d0ugal has joined #openstack-keystone | 15:34 | |
| *** Tahvok has joined #openstack-keystone | 15:35 | |
| *** d0ugal_ has quit IRC | 15:37 | |
| *** edmondsw_ has joined #openstack-keystone | 16:00 | |
| *** edmondsw has quit IRC | 16:03 | |
| *** edmondsw_ is now known as edmondsw | 16:03 | |
| *** phalmos has joined #openstack-keystone | 16:04 | |
| *** phalmos has quit IRC | 16:14 | |
| *** AlexeyAbashkin has quit IRC | 16:23 | |
| *** phalmos has joined #openstack-keystone | 16:26 | |
| errr | cmurphy: openstack: error: argument --os-auth-type: invalid choice: u'v3samlpassword' (choose from 'v2token', 'password', 'admin_token', 'v3oidcauthcode', 'v2password', 'gnocchi-noauth', 'v3password', 'v3oidcaccesstoken', 'token_endpoint', 'token', 'v3oidcclientcredentials', 'v3tokenlessauth', 'v3token', 'v3totp', 'v3oidcpassword', 'aodh-noauth', 'gnocchi-basic') | 17:13 |
| *** knasim-wrs has joined #openstack-keystone | 17:14 | |
| cmurphy | errr: are you using the latest version of python-openstackclient and keystoneauth? | 17:14 |
| knasim-wrs | hi folks, what is the maximum character limit on keystone user passwords? | 17:15 |
| errr | cmurphy: yes, that happened in my newton env, and the new virtualenv I made last night to test the latest version available via pip | 17:15 |
| knasim-wrs | in Newton, I had a test case that would provide 2075 character passwords that would go through. That is failing in Pike | 17:15 |
| knasim-wrs | exceeds the limit of column password(CHAR(128)). (HTTP 400) (Request-ID: req-7ae07943-6b13-44e8-bae1-4a0ba4fa6788) | 17:15 |
| knasim-wrs | wonder if this was changed as part of the new AES user encryption BP? | 17:16 |
| openstackgerrit | Gage Hugo proposed openstack/keystone master: Migrate jobs to zuulV3 https://review.openstack.org/523231 | 17:16 |
| errr | cmurphy: python-openstackclient (3.12.0) keystoneauth1 (3.2.0) is whats in my new virtualenv | 17:17 |
| cmurphy | errr: is that the original error message? I thought you were seeing something about missing method args | 17:17 |
| errr | cmurphy: this is a new error using the command you provided via the pastebin | 17:17 |
| cmurphy | errr: sorry, could you paste the whole debug output of the original error? (sorry if i missed it) | 17:18 |
| errr | cmurphy: the whole debug, or just the stack trace? | 17:19 |
| cmurphy | errr: the whole thing | 17:19 |
| errr | ok, give me just a moment | 17:19 |
| *** knasim-wrs has quit IRC | 17:19 | |
| *** knasim-wrs has joined #openstack-keystone | 17:21 | |
| *** efried is now known as efried_rollin | 17:22 | |
| cmurphy | I see export OS_AUTH_TYPE=v3samlpassword in your openrc which is equivalent to --os-auth-type v3samlpassword so i'm not sure what changed | 17:24 |
| gagehugo | lbragstad do we use the experimental gate? | 17:25 |
| errr | cmurphy: https://gist.githubusercontent.com/michaelrice/9289d7102d76985ae33a9dc8b2fca2e4/raw/ab58f7fb75b76bbe53b74fcd937be27cf9cac40b/gistfile1.txt | 17:26 |
| cmurphy | errr: do you have access to the keystone logs? the debug logs there are sometimes helpful | 17:31 |
| errr | yes I do | 17:32 |
| errr | let me go look at those | 17:32 |
| cmurphy | errr: i'm going to have to drop off in a bit but you could file a bug report against keystoneauth for this | 17:33 |
| errr | cmurphy: but is this even getting to keystone since its an error from openstack-client telling me Im sending the wrong params to the function? | 17:34 |
| errr | and then using the command you gave me to try its openstack-client telling me the auth plugin is not valid | 17:35 |
| cmurphy | errr: i think i left out --os-identity-provider-api 3 in that command | 17:38 |
| cmurphy | er sorry | 17:38 |
| errr | ah | 17:38 |
| cmurphy | --os-identity-api-version 3 | 17:38 |
| lbragstad | knasim-wrs: https://github.com/openstack/keystone/commit/8ad765e0230ceeb5ca7c36ec3ed6d25c57b22c9d that did go in during pike | 17:38 |
| cmurphy | because OS_AUTH_TYPE and --os-auth-type are the same thing, the only reason to get a different reaction is if that api version is set wrong | 17:38 |
| knasim-wrs | lbragstad: Thanks, I see there was an OSSN - Security Advisory as well for this | 17:39 |
| knasim-wrs | that would explain it | 17:39 |
| cmurphy | errr: and you're right, the debug output doesn't make it look like it's hitting keystone at all | 17:39 |
| lbragstad | yeah - it was an issue with the way keystone was hashing passwords | 17:39 |
| errr | cmurphy: adding that still gave me the same output of invalid plugin | 17:39 |
| lbragstad | we had to update the implementation to support more secure hashing | 17:40 |
| cmurphy | errr: hmm :/ i have to go but i'm going to try to reproduce later | 17:40 |
| errr | cmurphy: ok thanks for the help, and I loved your talk about this. I watched it on youtube several times. | 17:41 |
| cmurphy | errr: heh thanks :) | 17:41 |
| knasim-wrs | lbragstad: the default value for the [identity]/max_password_length is still 4096. And this length cannot be supported with the new bcrypt hashing algorithm | 17:42 |
| knasim-wrs | the generated hash, if you provide a 4096 character password (default value of max_password_length configuration option), is more than the SQL backend's 128 character column length and you get a DB insert error | 17:43 |
| lbragstad | cc kmalloc ^ | 17:45 |
| kmalloc | it shouldn't be using the 128 character length column | 17:48 |
| lbragstad | knasim-wrs: we updated that colume to have a longer length | 17:48 |
| kmalloc | it should be using the 255 password_hash = sql.Column(sql.String(255), nullable=True) | 17:48 |
| lbragstad | knasim-wrs: https://github.com/openstack/keystone/blob/master/keystone/common/sql/expand_repo/versions/023_expand_add_second_password_column_for_expanded_hash_sizes.py#L23 | 17:48 |
| kmalloc | there is a different column used with bcrypt. if you're trying to backport bcrypt support -- it is not backportable | 17:48 |
| lbragstad | knasim-wrs: do you know if the database migration has been run yet? | 17:48 |
| kmalloc | that said, a 4096 length password should hash down to the same length regardless, it is based upon the hash settings. | 17:50 |
| kmalloc | lbragstad: i can remove the shim for bcrypt/legacy now in Q. | 17:51 |
| kmalloc | i should add that code. | 17:51 |
| lbragstad | kmalloc: yeah - i just noticed that as i read the comments | 17:51 |
| lbragstad | ++ | 17:51 |
| knasim-wrs | looks like the DB migration didn`t run for me | 17:51 |
| knasim-wrs | the password db table is still 128chars | 17:51 |
| kmalloc | that would cause issues | 17:51 |
| kmalloc | no, there is a new column | 17:52 |
| kmalloc | password_hash | 17:52 |
| lbragstad | knasim-wrs: yeah - that might be giving you problems if you have a Pike node talking to a pre-Pike database | 17:52 |
| kmalloc | the password column remains 128 characters | 17:52 |
| lbragstad | I'd try running the migrations and give it another shot | 17:52 |
| knasim-wrs | ok thanks guys | 17:53 |
| knasim-wrs | this is not an upgrade btw, its a fresh install | 17:53 |
| lbragstad | knasim-wrs: let us know if that works for you | 17:53 |
| knasim-wrs | so I do a keystone-manage bootstrap | 17:53 |
| knasim-wrs | and db_sync | 17:53 |
| knasim-wrs | as opposed to expand_repo / migrate_repo etc | 17:53 |
| kmalloc | lbragstad: what is the order of operations for db migrations | 17:54 |
| kmalloc | again | 17:54 |
| lbragstad | expand, migrate, contract | 17:54 |
| kmalloc | and what is required vs optional | 17:54 |
| kmalloc | i mean | 17:54 |
| kmalloc | you expand and must migrate | 17:54 |
| kmalloc | or is migrate optional? | 17:54 |
| kmalloc | until later? | 17:54 |
| lbragstad | none are optional | 17:55 |
| lbragstad | oh - wait | 17:55 |
| lbragstad | we have docs for this | 17:55 |
| lbragstad | https://docs.openstack.org/keystone/latest/contributor/database-migrations.html | 17:55 |
| kmalloc | have i mentioned i think this whole thing is so very confusing, because i have to ask every single time | 17:55 |
| lbragstad | yes... | 17:55 |
| lbragstad | so - in the upgrade scenario, | 17:55 |
| kmalloc | that also doesn't (doc) help me | 17:56 |
| lbragstad | if you have 3 ocata nodes, you pull one out of rotation and upgrade it to pike | 17:56 |
| kmalloc | can i rely on a migrate step being run before Q code is run?> | 17:56 |
| lbragstad | but don't turn it on | 17:56 |
| lbragstad | with the other two nodes running ocata, you can expand the database | 17:56 |
| lbragstad | then you can migrate the database | 17:56 |
| lbragstad | (expand and migrate are done using the pike node) | 17:56 |
| lbragstad | then you can put pike into rotation | 17:57 |
| kmalloc | right and migrate is *required* before putting pike back in | 17:57 |
| kmalloc | ? | 17:57 |
| lbragstad | s/pike/the pike nodes/ | 17:57 |
| lbragstad | yeah - i believe so | 17:57 |
| kmalloc | ok i'm going to roll a Q patch that makes that assertion | 17:57 |
| kmalloc | i'm dropping use of the password column | 17:57 |
| kmalloc | but the data will need to be migrated. | 17:58 |
| kmalloc | basically: set password_hash (if None) to (password value) | 17:58 |
| kmalloc | and drop the compat code | 17:58 |
| lbragstad | this document describes that process in more detail https://docs.openstack.org/keystone/latest/admin/identity-upgrading.html | 17:58 |
| kmalloc | and contract will drop the password column. | 17:58 |
| knasim-wrs | lbragstad: The link for DB contraction / expansion is only for Upgrade scenarios. I am doing a fresh Pike install. Shouldn't the database be set up properly via DB sync? | 17:59 |
| lbragstad | it should so long as you have pike code installed | 17:59 |
| lbragstad | it seems odd that didn't happen | 17:59 |
| kmalloc | knasim-wrs: do you have a "password_hash" column? | 18:00 |
| kmalloc | because that is supposed to be used for the bcrypt hashes | 18:00 |
| *** thorst has quit IRC | 18:00 | |
| openstackgerrit | Merged openstack/keystone-specs master: Repropose application credentials to queens https://review.openstack.org/512505 | 18:00 |
| kmalloc | i'll look at bcrypt in a few mins. it might need an errata. | 18:00 |
| kmalloc | cmurphy: ^ yay merge! | 18:01 |
| * lbragstad ticks things off in trello | 18:01 | |
| knasim-wrs | kmalloc: I do have the password_hash column but the error is for the password column | 18:02 |
| knasim-wrs | my schema: | 18:02 |
| kmalloc | the bcrypt data should not be stored in "password" column | 18:02 |
| knasim-wrs | https://thepasteb.in/p/k5hYBmXYvMBFE | 18:02 |
| knasim-wrs | keystone=# \d+ password | 18:02 |
| knasim-wrs | Table "public.password" | 18:02 |
| knasim-wrs | Column | Type | Modifiers | Storage | Stats target | Description | 18:02 |
| knasim-wrs | ----------------+-----------------------------+-------------------------------------------------------+----------+--------------+------------- | 18:02 |
| knasim-wrs | id | integer | not null default nextval('password_id_seq'::regclass) | plain | | | 18:02 |
| knasim-wrs | local_user_id | integer | not null | plain | | | 18:03 |
| knasim-wrs | password | character varying(128) | | extended | | | 18:03 |
| knasim-wrs | expires_at | timestamp without time zone | | plain | | | 18:03 |
| knasim-wrs | self_service | boolean | not null default false | plain | | | 18:03 |
| knasim-wrs | password_hash | character varying(255) | | extended | | | 18:03 |
| knasim-wrs | created_at_int | bigint | not null default 0::bigint | plain | | | 18:03 |
| kmalloc | only in password hash. | 18:03 |
| kmalloc | the old hash data is stored in password column | 18:03 |
| kmalloc | let me check a couple things | 18:03 |
| knasim-wrs | kmalloc: I am sending you the command and the response | 18:04 |
| knasim-wrs | 'openstack --os-username 'keystoneuser005_ber' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-region-name RegionOne --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal user set --password 'RgJSa?dB&4rH;Q|c,*Ij,zs+nC<bwivV8kEfXePD~pmA2{{K | 18:04 |
| knasim-wrs | UoN3%q6t_h$1Y9Zy]L7.0lM=5:<O@TxuFWG^Ik2|h&>2Y{{A4?2}},vy],9,,j@s?3@9p9G<nYaem@i?wAb_bvZ59>Yd[0~W#8udA[LMpgKXqzqTD<Wpp*:i,gig$#ZVA*N~5QpA@9$Z#,,IdebedJZ57Z#|Nm4{{11z1H#tl*H}};b.O;obLgp?7p],j)LZr;lmP^C(Zl$U1IrM^^oZRBi1N,tw]1VeOwM2YT9e:8,:u,8Y*x:9J&AH#my,PzUSoJC,hCJqF<tw=5xiyTW6i?x#ckyH+u,|Z[CK;4atGd()JM|y%AOT3*}}MerFA^80Mhj*:{{7=]A>N3+c^83Nzj7n1KmXk@ | 18:04 |
| knasim-wrs | Uvy~:.%7,y2xH^N)oWpZMISm)YPWqesKwy@^:@J>=0ETaW;H:<va&,=qlcUW9B,:?(+M%geElm8<,S%+,:^VH_<0z&,|@N%}}CPjb7Bu7i@x)N3epvb)t5UpEZ?C;,I:Qkwu7]Cd=Ah:W,{{{{?,P&*z5E6E?jZ.JGgmb0=DD{{xK:pf%Lm,v0vR)X=[IYCWWgNkX,8)#,+8AG*Y,P;g@oX8;b<DCYmYM|V%wp.~b)Oyz,drWg.A.Y.NE>K,n:0Q;=d^^L(bB=gR|x3)0B:3]Z9(hJ,&k:T@PpXQEp]r1,c(0mH(,r;#qR6Q,wK,g=q~?hNgyKukdrP4oWDcv0}}b]BXH,rcKA | 18:04 |
| knasim-wrs | ;.}}ko*R.x;,,^,#m@}}i~xSQ@Y7zTQDn<Y9munA,>1F$RvJUG$kGJowv{{I?i)<]K?,W<NPZ<T9&T~8p2^r(k*0D7+?iZH_@LDIgsjs+l|uf5oi%[Z2uN)W&8+6y,JtY}}UU4LiHF;,5big+r6lpI){{BT=hIt^.<T}}:{{>DPa:.,P0MHw*)dAmX8R[>[,T=T5}}*aW(,_hO($_UJkGzLrE$o;M+$(iJxM*dtV:sz(l$A3|=^5^y[vw,R?t[y@dd0GY09b*W&2P;3]^y=}}OZF(iO|MK^69H7;lnmn|FaP<ZrJ,H#[ji,NL6Fl$%:Bau<Y5r<pnXm@cqv8dr,_;_L^nTd,q: | 18:04 |
| knasim-wrs | v_Fc,k%,j2<5,4wpM?05jJi?<>Y,,B$8FP}},s,Ig<<1{{o1PKQ,&[CGM$<iaEJL]3hr;ikHh2{{,;lW)Yb[FtEqo=oaypr<(:f9d2n,o.?<Y76app+mJ1r:.QTGg=#c<>BzFd,n3knJJJ^99pxbez|G~sUQI7vX[Ws>e.0R4,l1|tD:,<B,6~[;O}}~ydz<mw~uRTbkmNzVq[%w}}zTV3}}la,:tEPBD+}}askQ~p,smeidy^s9Vbgt1&D72aod*xo?6iA)TIw6WMh}}IrJEm:v@ktx#;rO[iB,lhhM;:=fNId0kG?yTEe7P;0<At4=&0&,.:7bI?jCaC|R6],,+oG{{<f<s2 | 18:04 |
| knasim-wrs | hT{{&,(8.kG5n?<(Nv%B7&G,IWDJU0jD*}}hGe|C$B^~QQXHCg(<t<dH:IM+mq,?K|pa^o9>^itP[F<n8F3Z<(@P]g|0c3IiwIa_hK:@zdK?^,_t%_d4ICA;*&@hRD4EIjTs(xDeD~WqE1+kDRl8RmhcX,J&^...F~GNL@sV8~1v7f^>a_]x|>LJF9SOmDJ=l<T:27;ZzY8lZ]dwL02,cOM;58;[8hU?<(?(?=DOa.Q@&t|*iX3+l2[(4}}@%^)(>+,}}Zkp_:An<erwfu;iwV),(A*:*.vF<nY?#?rXKc}}+jDnbC&y)T{{1;56B9HMhhHa~3?LnnS9X1=2#7v,Qqr[W,:QkP | 18:04 |
| knasim-wrs | I)3Df=<,CqoFScSbO,@5,aMc@MYP4oES0=Ki8L0C*WQ^3KMDt)V<:]ct,,gl2F@[,|5(=Aqbv==c<[C(f<%8V:R@,V<*}}seK~:{{,3zI<c,ish,U86.h,5:<*@uQ,r2' keystoneuser005_ber'' | 18:04 |
| knasim-wrs | Response: | 18:04 |
| knasim-wrs | String length exceeded. The length of string 'RgJSa?dB&4rH;Q|c,*Ij,zs+nC<bwivV8kEfXePD~pmA2{{KUoN3%q6t_h$1Y9Zy]L7.0lM=5:<O@TxuFWG^Ik2|h&>2Y{{A4?2}},vy],9,,j@s?3@9p9G<nYaem@i?wAb_bvZ59>Yd[0~W#8udA[LMpgKXqzqTD<Wpp*:i,gig$#ZVA*N~5QpA@9$Z#,,IdebedJZ57Z#|Nm4{{11z1H#tl*H}};b.O;obLgp?7p],j)LZr;lmP^C(Zl$U1IrM^^oZRBi1N,tw]1VeOwM2YT9e:8,:u,8Y*x:9J&AH#my,PzUS | 18:04 |
| knasim-wrs | oJC,hCJqF<tw=5xiyTW6i?x#ckyH+u,|Z[CK;4atGd()JM|y%AOT3*}}MerFA^80Mhj*:{{7=]A>N3+c^83Nzj7n1KmXk@Uvy~:.%7,y2xH^N)oWpZMISm)YPWqesKwy@^:@J>=0ETaW;H:<va&,=qlcUW9B,:?(+M%geElm8<,S%+,:^VH_<0z&,|@N%}}CPjb7Bu7i@x)N3epvb)t5UpEZ?C;,I:Qkwu7]Cd=Ah:W,{{{{?,P&*z5E6E?jZ.JGgmb0=DD{{xK:pf%Lm,v0vR)X=[IYCWWgNkX,8)#,+8AG*Y,P;g@oX8;b<DCYmYM|V%wp.~b)Oyz,drWg.A.Y.NE>K,n:0Q | 18:04 |
| knasim-wrs | ;=d^^L(bB=gR|x3)0B:3]Z9(hJ,&k:T@PpXQEp]r1,c(0mH(,r;#qR6Q,wK,g=q~?hNgyKukdrP4oWDcv0}}b]BXH,rcKA;.}}ko*R.x;,,^,#m@}}i~xSQ@Y7zTQDn<Y9munA,>1F$RvJUG$kGJowv{{I?i)<]K?,W<NPZ<T9&T~8p2^r(k*0D7+?iZH_@LDIgsjs+l|uf5oi%[Z2uN)W&8+6y,JtY}}UU4LiHF;,5big+r6lpI){{BT=hIt^.<T}}:{{>DPa:.,P0MHw*)dAmX8R[>[,T=T5}}*aW(,_hO($_UJkGzLrE$o;M+$(iJxM*dtV:sz(l$A3|=^5^y[vw,R?t[y@ | 18:04 |
| knasim-wrs | dd0GY09b*W&2P;3]^y=}}OZF(iO|MK^69H7;lnmn|FaP<ZrJ,H#[ji,NL6Fl$%:Bau<Y5r<pnXm@cqv8dr,_;_L^nTd,q:v_Fc,k%,j2<5,4wpM?05jJi?<>Y,,B$8FP}},s,Ig<<1{{o1PKQ,&[CGM$<iaEJL]3hr;ikHh2{{,;lW)Yb[FtEqo=oaypr<(:f9d2n,o.?<Y76app+mJ1r:.QTGg=#c<>BzFd,n3knJJJ^99pxbez|G~sUQI7vX[Ws>e.0R4,l1|tD:,<B,6~[;O}}~ydz<mw~uRTbkmNzVq[%w}}zTV3}}la,:tEPBD+}}askQ~p,smeidy^s9Vbgt1&D72aod | 18:04 |
| kmalloc | knasim-wrs: please use paste.openstack.org | 18:04 |
| hrybacki | +1 | 18:04 |
| knasim-wrs | ok sorry | 18:04 |
| kmalloc | it's hard to read in IRC | 18:04 |
| knasim-wrs | I thought it would make a link automatically, it usually does | 18:05 |
| kmalloc | irccloud? | 18:05 |
| knasim-wrs | didn't do it this time | 18:05 |
| knasim-wrs | no kiwiirc | 18:05 |
| kmalloc | ahh | 18:05 |
| kmalloc | yeah | 18:05 |
| kmalloc | must be broken atm, no worries :) | 18:05 |
| knasim-wrs | http://paste.openstack.org/show/627747/ | 18:05 |
| *** thorst has joined #openstack-keystone | 18:06 | |
| kmalloc | that doesn't look like a valid hash. | 18:06 |
| kmalloc | oh | 18:07 |
| kmalloc | hold on | 18:07 |
| kmalloc | that is a crazy password :P | 18:07 |
| kmalloc | ok | 18:07 |
| knasim-wrs | its not a hash lol... its the actual password | 18:07 |
| kmalloc | ooh | 18:07 |
| kmalloc | i think i see what is happening | 18:07 |
| knasim-wrs | yeah one of those fringe test cases we run everytime we move to a new Openstack release | 18:07 |
| kmalloc | it isn't hashing the string with the commandline | 18:08 |
| kmalloc | it loooks like | 18:08 |
| knasim-wrs | and kmalloc, I'm using the default Pike configurations for hashing / identity: | 18:08 |
| knasim-wrs | [identity] | 18:08 |
| knasim-wrs | driver=sql | 18:08 |
| knasim-wrs | that's all that I have under that stanza | 18:08 |
| knasim-wrs | this test case used to work in Newton btw | 18:09 |
| lbragstad | edmondsw: hrybacki https://trello.com/c/Vi1RC7di/82-write-a-openstack-spec-for-default-roles | 18:09 |
| edmondsw | lbragstad +1 | 18:10 |
| lbragstad | make sense? | 18:10 |
| edmondsw | lbragstad I would note that I think we should start basic here... not try to boil the ocean with 20 roles off the bat | 18:10 |
| edmondsw | admin, reader, and writer sounds like a reasonable first step | 18:11 |
| *** thorst has quit IRC | 18:11 | |
| hrybacki | agree wholeheartedly with edmondsw | 18:11 |
| edmondsw | I worry that folks will try to use that spec to throw in everything their heart desires and we'll never get it approved | 18:11 |
| kmalloc | knasim-wrs: do you have the option: CONF.identity.rolling_upgrade_password_hash_compat set in your keystone config? | 18:11 |
| knasim-wrs | nope | 18:12 |
| kmalloc | then nothing should be setting the password column at all | 18:12 |
| lbragstad | hrybacki: edmondsw refresh? | 18:12 |
| kmalloc | knasim-wrs: this is a stock pike install? | 18:13 |
| knasim-wrs | kmalloc: As far as keystone is concerned: Yes. I haven't rebased our customizations on top of it yet | 18:13 |
| knasim-wrs | its a vanilla pull from stable/pike | 18:13 |
| edmondsw | lbragstad sounds like you are allowing that the result might be super simple, rather than saying we should start super simple | 18:13 |
| lbragstad | edmondsw: well - i'd like to keep that enforced by reviewing the spec | 18:14 |
| lbragstad | trello is only describing the work in my opinion, since it's not an official openstack document | 18:14 |
| edmondsw | lbragstad sure. I was just voicing a concern we should be ready for | 18:14 |
| edmondsw | not saying we have to do anything diff in trello :) | 18:14 |
| lbragstad | completely agree that is something we need to watch out for in the review process though! | 18:15 |
| mnaser | sorry for the partly silly question but I am pretty sure I've seen a documented "role alias" feature in Keystone or I'm going crazy? | 18:15 |
| kmalloc | knasim-wrs: i am trying to figure out how the hell your password is being set without hashing on the column. | 18:16 |
| kmalloc | something very strange is happening., | 18:16 |
| kmalloc | unless... | 18:16 |
| lbragstad | knasim-wrs: can you verify you're actually using pike code? | 18:16 |
| lbragstad | mnaser: role alias? like implied roles? | 18:16 |
| kmalloc | lbragstad: even if it wasn't pike code, it should never ever ever be setting the password directly on the password column w/o hashing | 18:17 |
| knasim-wrs | lbragstad: Yes I am. I build it myself | 18:17 |
| mnaser | lbragstad: i think implied roles is the correct term! | 18:17 |
| knasim-wrs | let me reproduce the problem | 18:17 |
| lbragstad | mnaser: then no, you're not going crazy :) | 18:17 |
| lbragstad | mnaser: that is a thing implemented in keystone | 18:17 |
| mnaser | i was searching for role aliases but yeah, implied roles works | 18:17 |
| mnaser | this is perfect | 18:17 |
| knasim-wrs | kmalloc: Reproduced again | 18:18 |
| *** thorst has joined #openstack-keystone | 18:18 | |
| kmalloc | knasim-wrs: right. i just don't see how it can ever be set on the password column like you're setting it | 18:18 |
| knasim-wrs | http://paste.openstack.org/show/627750/ | 18:19 |
| kmalloc | i don't doubt you | 18:21 |
| kmalloc | i am baffled how you're getting the password directly set like that | 18:21 |
| *** thorst has quit IRC | 18:22 | |
| lbragstad | edmondsw: hrybacki ok - i'm going to start working on that then | 18:22 |
| edmondsw | lbragstad ++ | 18:25 |
| kmalloc | knasim-wrs: can you show me your user table? | 18:26 |
| kmalloc | knasim-wrs: just the show create | 18:26 |
| kmalloc | knasim-wrs: not the content | 18:26 |
| knasim-wrs | this is my own test deployment anyways so I can dump the content and schema... not a problem | 18:27 |
| kmalloc | yeah i just want to know what the schema is | 18:27 |
| kmalloc | content isn't as interesting in most cases. | 18:27 |
| knasim-wrs | https://thepasteb.in/p/pghQoXO7j6vuR | 18:28 |
| kmalloc | hm. | 18:28 |
| kmalloc | this is very odd, i just don't see how we could be setting this value | 18:28 |
| kmalloc | like this | 18:28 |
| mnaser | thanks lbragstad, implied roles worked like a charm.. time to clean up our role assignments | 18:29 |
| lbragstad | mnaser: ++ | 18:29 |
| knasim-wrs | ok, so in my password DB table, the password colum is empty... could it be that its using the password column as a pergutory to store the password there first before moving to the password_hash column? | 18:29 |
| kmalloc | knasim-wrs: it should be empty. | 18:30 |
| knasim-wrs | my password DB table: https://thepasteb.in/p/mwh1lEPq6X7u5 | 18:30 |
| kmalloc | no, it doesn't store it in password column unless the compat stuff is set | 18:31 |
| mnaser | this should speed up keystone i assume because the assignments table will be signifincantly smaller too.. i think! | 18:31 |
| kmalloc | knasim-wrs: it does a hash in bcrypt and then sets it in password_ref.pasword_hash, and if compat is set, it does password_ref.password with the compat hash | 18:31 |
| kmalloc | lbragstad: hey, uhm... with openstack user set --password, what API si that calling? | 18:32 |
| knasim-wrs | ok I'm going to add breakpoints in keystone identity and try to figure shit out | 18:32 |
| kmalloc | lbragstad: is that the password change API (self-service) or the update user? | 18:32 |
| kmalloc | knasim-wrs: i think i have a line on it | 18:32 |
| lbragstad | kmalloc: good question, probably the self service one? | 18:33 |
| lbragstad | --debug? | 18:33 |
| lbragstad | should give you the path | 18:33 |
| kmalloc | knasim-wrs: can you run the password setr command with --debug? | 18:33 |
| knasim-wrs | on it | 18:33 |
| openstackgerrit | Merged openstack/keystone master: Updated from global requirements https://review.openstack.org/523735 | 18:34 |
| kmalloc | yep | 18:34 |
| kmalloc | it must be calling that | 18:34 |
| kmalloc | lbragstad: https://github.com/openstack/keystone/blob/master/keystone/identity/backends/sql.py#L248-L255 isn't tested | 18:34 |
| kmalloc | it directly sets password hash values | 18:34 |
| kmalloc | not user.password | 18:34 |
| kmalloc | *eyeroll* | 18:34 |
| kmalloc | this has been broken for a while now | 18:34 |
| kmalloc | this was a bug we don't test for and we simply broke it in pike | 18:35 |
| kmalloc | sorry | 18:35 |
| kmalloc | knasim-wrs: yep found it | 18:35 |
| kmalloc | oh wait | 18:36 |
| kmalloc | no. | 18:36 |
| kmalloc | i am wrong, this is doing it correctly | 18:36 |
| kmalloc | knasim-wrs: still need that --debug output | 18:36 |
| *** mvk has quit IRC | 18:36 | |
| knasim-wrs | https://thepasteb.in/p/P1hvXyN88DXtl | 18:37 |
| knasim-wrs | kmalloc: here goes... I sent you the entire output but you probably only wanna look at the traceback | 18:37 |
| kmalloc | nope i want the whole output ;) | 18:37 |
| kmalloc | knasim-wrs: can you check keystone's log and see if there is a traceback there? | 18:39 |
| *** thorst has joined #openstack-keystone | 18:40 | |
| *** thorst has quit IRC | 18:40 | |
| *** thorst has joined #openstack-keystone | 18:40 | |
| knasim-wrs | kmalloc: Just a warning (the same as what was returned to the client)... no traceback | 18:42 |
| knasim-wrs | https://thepasteb.in/p/LghN9Ew9DnPUZ | 18:43 |
| *** aselius has joined #openstack-keystone | 18:43 | |
| *** david-lyle has joined #openstack-keystone | 18:45 | |
| kmalloc | are you updating a user or creating a user? | 18:46 |
| kmalloc | looks like creating, right? | 18:46 |
| kmalloc | ok i see create | 18:46 |
| knasim-wrs | kmalloc: Same issue seen with both | 18:46 |
| kmalloc | right but you're hitting the create path right now | 18:47 |
| kmalloc | so i can chase that vs something else. | 18:47 |
| knasim-wrs | yeah | 18:47 |
| kmalloc | lbragstad: when is the @hybrid.expression decorator called? | 18:48 |
| kmalloc | erm version of hybrid_property? | 18:48 |
| kmalloc | oh. blah i see the issue | 18:50 |
| kmalloc | it's hybrid property being... dumb. | 18:50 |
| kmalloc | knasim-wrs: so, somehow we're getting a class-version of the property and not an instance version | 18:52 |
| kmalloc | which is causing the problem | 18:52 |
| lbragstad | ahh | 18:52 |
| knasim-wrs | which property? I can investigate more on my end | 18:52 |
| knasim-wrs | I don't have any custom backends btw | 18:53 |
| lbragstad | we use hybrid_property to pull together things across multiple database tables | 18:53 |
| knasim-wrs | just using vanilla sql identity backend | 18:53 |
| knasim-wrs | I see | 18:53 |
| kmalloc | knasim-wrs: here i'm going to link the problem code... | 18:54 |
| *** david-lyle has quit IRC | 18:54 | |
| kmalloc | https://github.com/openstack/keystone/blob/master/keystone/identity/backends/sql_model.py#L189-L191 | 18:54 |
| kmalloc | that is the issue | 18:54 |
| kmalloc | lbragstad: i don't know how to fix this. | 18:54 |
| kmalloc | lbragstad: this is pretty deep-dark sqlalchemy stuff | 18:55 |
| kmalloc | likely .expression() is simply wrong | 18:55 |
| knasim-wrs | lol | 18:55 |
| lbragstad | kmalloc: do you think zzzeek would help us look at it? | 18:55 |
| kmalloc | possibly | 18:55 |
| kmalloc | i hope zzzeek can help | 18:55 |
| knasim-wrs | can we get a Bug going? So I can link it against this test case and track it | 18:55 |
| lbragstad | knasim-wrs: ++ | 18:55 |
| kmalloc | knasim-wrs: sure. open a bug in LP. | 18:55 |
| knasim-wrs | ok, will do | 18:55 |
| kmalloc | lbragstad: this doesn't make sense to me http://docs.sqlalchemy.org/en/latest/orm/extensions/hybrid.html#defining-expression-behavior-distinct-from-attribute-behavior | 18:56 |
| kmalloc | at all | 18:56 |
| kmalloc | not sure why we're hitting it | 18:56 |
| kmalloc | lbragstad: basically we are settign password.Password to a bad value, then fixing it when we do the user_ref.password logic | 18:57 |
| lbragstad | ? | 18:58 |
| kmalloc | lbragstad: we set the value to the raw string | 18:58 |
| lbragstad | ?! | 18:58 |
| kmalloc | then the logic re-sets it | 18:58 |
| kmalloc | so if a password is *Ever* over 128 characters it would puke | 18:58 |
| lbragstad | we do or sqlalchemy does because we use the decorator? | 18:58 |
| kmalloc | the way in which stuff is set. | 18:58 |
| kmalloc | it is a mix of us + sql_alchemy | 18:58 |
| lbragstad | \o/ | 18:58 |
| lbragstad | oh man, that's bad | 18:59 |
| kmalloc | it never is persisted to the DB | 19:00 |
| lbragstad | oh - i thought it was | 19:01 |
| kmalloc | no no, | 19:01 |
| lbragstad | but validation error | 19:01 |
| kmalloc | but the issue is the column still restricts the validation | 19:01 |
| kmalloc | yeah | 19:01 |
| *** david-lyle has joined #openstack-keystone | 19:01 | |
| lbragstad | but the passwords isn't hashed yet... | 19:02 |
| *** david-lyle has quit IRC | 19:02 | |
| kmalloc | honestly, i think we're mis-using expression | 19:05 |
| kmalloc | we can probably just drop it, it is supposed to be used in cases where SQL and Python would work differently | 19:05 |
| kmalloc | in this case.. it's... uh, not really | 19:05 |
| knasim-wrs | kmalloc: https://bugs.launchpad.net/keystone/+bug/1735250 | 19:08 |
| openstack | Launchpad bug 1735250 in OpenStack Identity (keystone) "Limit for the Password column in the Password table exceeded when using passwords exceeding 2000 characters" [Undecided,New] | 19:08 |
| *** mvk has joined #openstack-keystone | 19:09 | |
| *** rderose has quit IRC | 19:20 | |
| *** rderose has joined #openstack-keystone | 19:21 | |
| *** david-lyle has joined #openstack-keystone | 19:21 | |
| ayoung | kmalloc, what allocates the request_id now? | 19:24 |
| kmalloc | ayoung: uh oslo.context? | 19:24 |
| kmalloc | i think? | 19:24 |
| ayoung | hmmm....maybe that is the new problem.,... | 19:24 |
| ayoung | ok let me look | 19:24 |
| *** david-lyle has quit IRC | 19:34 | |
| *** david-lyle has joined #openstack-keystone | 19:35 | |
| zzzeek | i hate when i make passwords that are more than 2000 characters | 19:36 |
| kmalloc | zzzeek: heh, it's any password > 128 | 19:39 |
| kmalloc | so... i *think* we're mis-using .expression() here | 19:40 |
| kmalloc | https://github.com/openstack/keystone/blob/master/keystone/identity/backends/sql_model.py#L189-L191 | 19:40 |
| kmalloc | which is the root of the issue | 19:40 |
| kmalloc | anyone who uses > 128 character password... but whatever, we have historically supported it | 19:41 |
| *** jose-phi_ has quit IRC | 19:49 | |
| *** linkmark has joined #openstack-keystone | 19:50 | |
| lbragstad | kmalloc: ayoung could i get a kick through here? https://review.openstack.org/#/c/462733/ | 19:51 |
| ayoung | looking | 19:51 |
| ayoung | I'm a co-author? | 19:51 |
| lbragstad | i attempted to consolidate a bunch of things, many of which you wrote | 19:53 |
| lbragstad | so - yes :) | 19:53 |
| openstackgerrit | ayoung proposed openstack/keystone-specs master: Add policy roadmap for security https://review.openstack.org/462733 | 19:53 |
| *** david-lyle has quit IRC | 19:53 | |
| ayoung | minor typo fix. repushin | 19:53 |
| kmalloc | re +A | 19:54 |
| lbragstad | aha - nice | 19:55 |
| lbragstad | thanks | 19:55 |
| *** aojea has joined #openstack-keystone | 19:56 | |
| * lbragstad needs coffee | 19:59 | |
| *** david-lyle has joined #openstack-keystone | 20:04 | |
| *** efried_rollin is now known as efried | 20:07 | |
| *** AlexeyAbashkin has joined #openstack-keystone | 20:14 | |
| *** spectr has joined #openstack-keystone | 20:18 | |
| *** AlexeyAbashkin has quit IRC | 20:18 | |
| *** spectr has quit IRC | 20:20 | |
| openstackgerrit | Merged openstack/keystone-specs master: Add policy roadmap for security https://review.openstack.org/462733 | 20:27 |
| ayoung | kmalloc, so...why are the headers added lowercase in middleware, but tested uppercase in context? What is supposed to convert them? | 20:33 |
| ayoung | jamielennox, I'm making progress but ^^? | 20:37 |
| kmalloc | doesn't matter | 20:38 |
| kmalloc | it's a case insensitive dictr | 20:38 |
| kmalloc | or should be | 20:38 |
| kmalloc | headers are case insensitive. | 20:38 |
| kmalloc | if it's using proper dict, it will be case insensitive. | 20:38 |
| ayoung | oslo is not doing a case insensitive search | 20:39 |
| ayoung | http://git.openstack.org/cgit/openstack/oslo.context/tree/oslo_context/context.py#n433 | 20:40 |
| ayoung | so...maybe I am grabbing the headers wrong somehow? | 20:40 |
| ayoung | request_context = context.RequestContext.from_environ(request.headers,**kwargs) | 20:40 |
| ayoung | should I somehow convert request.headers? | 20:41 |
| *** rderose has quit IRC | 20:43 | |
| kmalloc | requests.headers is supposed to be case insensitive | 20:45 |
| kmalloc | if it's doing X in headers | 20:45 |
| kmalloc | that is a problem. | 20:45 |
| kmalloc | maybe? | 20:45 |
| kmalloc | or it's doing it wonky where it's converting to normal strings to search | 20:45 |
| kmalloc | but in short, it is supposed to be a case insentive dict | 20:45 |
| *** david-lyle has quit IRC | 20:45 | |
| *** threestrands has joined #openstack-keystone | 20:47 | |
| *** threestrands has quit IRC | 20:47 | |
| *** threestrands has joined #openstack-keystone | 20:47 | |
| ayoung | kmalloc, OK, so here is where I am at, and maybe this will help: I reqlize that the keystone/middleware/auth.py AuthContextMiddleware extends (auth_token.BaseAuthProtocol): which is not the same as the auth token middleware | 20:49 |
| ayoung | it is the baseclass of both, but the AuthToken version calls | 20:50 |
| kmalloc | right | 20:50 |
| ayoung | request.set_user_headers(request.token_auth.user) | 20:50 |
| *** knasim-wrs has quit IRC | 20:50 | |
| ayoung | without that, we don't have the env vars that oslo context needs | 20:50 |
| ayoung | so I added a call to it, and I have the headers, but the call puts them in Camel Case | 20:50 |
| ayoung | 'X%s-Domain-Id': 'domain_id', | 20:50 |
| ayoung | expanded and so forth | 20:50 |
| ayoung | I then need to pass these to the constructor function for context | 20:51 |
| ayoung | which I am doing like this: | 20:51 |
| ayoung | request_context = context.RequestContext.from_environ(request.headers, **kwargs) | 20:51 |
| kmalloc | that last bit might be converting to a non-case-insensitive dict | 20:52 |
| kmalloc | instead of the case insensitive normal dict. | 20:53 |
| kmalloc | that is used in the request object | 20:53 |
| kmalloc | the case (iirc) is preserved, but __getitem__ is overidden to .lower() it (among other things) | 20:53 |
| ayoung | (Pdb) print request.headers['X-User-Id'] | 20:53 |
| ayoung | print request.headers['X-User-Id'] | 20:53 |
| ayoung | b7beda5c7bc741059b05eaebbf1f2e2d | 20:53 |
| ayoung | (Pdb) print request.headers['X-USER-ID'] | 20:53 |
| ayoung | print request.headers['X-USER-ID'] | 20:53 |
| ayoung | b7beda5c7bc741059b05eaebbf1f2e2d | 20:53 |
| kmalloc | or .upper | 20:54 |
| kmalloc | same net effect | 20:54 |
| ayoung | so it should be OK...right? | 20:54 |
| kmalloc | yeah it should be | 20:54 |
| kmalloc | http spec says headers are case insensitive, if .request.headers wasn't, we'd break the spec | 20:54 |
| kmalloc | and we're leaning on python stdlib stuff for that | 20:54 |
| kmalloc | iirc | 20:54 |
| kmalloc | or ... at least requests, which does it better than we do | 20:55 |
| ayoung | there is a getter in there that returns a case insensitive dict... | 20:56 |
| kmalloc | there ya go. | 20:57 |
| lbragstad | another easy one to kick through https://review.openstack.org/#/c/523491/ | 20:58 |
| *** pcaruana has quit IRC | 21:00 | |
| ayoung | print environ.keys() | 21:01 |
| ayoung | ['X-Tenant-Name', 'X-Role', 'X-User-Id', 'X-User-Domain-Name', 'X-User', 'X-User-Domain-Id', 'Host', 'X-Domain-Name', 'X-Is-Admin-Project', 'X-Tenant', 'X-Tenant-Id', 'X-Auth-Token', 'X-Project-Domain-Name', 'X-Project-Name', 'X-Identity-Status', 'X-Domain-Id', 'X-User-Name', 'X-Project-Domain-Id', 'X-Project-Id', 'X-Roles' | 21:01 |
| ayoung | they have - in them, not _ | 21:02 |
| ayoung | print 'HTTP-X-USER-NAME' in environ | 21:02 |
| ayoung | False | 21:02 |
| ayoung | (Pdb) print 'X-USER-NAME' in environ | 21:02 |
| ayoung | print 'X-USER-NAME' in environ | 21:02 |
| ayoung | True | 21:02 |
| ayoung | hmmmm...there must be a conversion I am missing | 21:03 |
| *** david-lyle has joined #openstack-keystone | 21:04 | |
| openstackgerrit | Merged openstack/keystone-specs master: Address follow on comments for system-scope https://review.openstack.org/523491 | 21:05 |
| ayoung | kmalloc, what converts X-User-Id to HTTP_X_USER_ID? | 21:06 |
| kmalloc | uhm.... i think we do | 21:06 |
| kmalloc | in middleware | 21:06 |
| *** d0ugal has quit IRC | 21:07 | |
| lbragstad | cmurphy: so - about the unified limit region thing... | 21:09 |
| lbragstad | were you in the room at the ptg on crazy friday when jamielennox start whiteboarding the whole system hierarchy thing? | 21:09 |
| cmurphy | lbragstad: i don't think so, i left early afternoon that day | 21:10 |
| *** rderose has joined #openstack-keystone | 21:10 | |
| lbragstad | cmurphy: ok - i just stumbled across this picture | 21:10 |
| lbragstad | https://www.lbragstad.com/blog/keystone-queens-ptg-summary - in the Policy Future section | 21:10 |
| kmalloc | ayoung: or is it the opposite, that we get HTTP_X... and convert it? | 21:11 |
| lbragstad | cmurphy: do you see the one with the two trees? | 21:11 |
| ayoung | the docs imply that, after AuthProtocol.process_request, the HTTP_ forsm will be there? | 21:11 |
| lbragstad | the project/domain tree and system/service tree | 21:11 |
| kmalloc | ayoung: should be, in the environ | 21:12 |
| cmurphy | lbragstad: yes | 21:12 |
| lbragstad | cmurphy: a thought just occurred to me, if we make limits require regions - we kinda break what is depicted in that picture | 21:13 |
| lbragstad | where the root of the tree (e.g. system) can have services associated to it | 21:13 |
| lbragstad | and those denote services that are in a sense, global | 21:13 |
| kmalloc | lbragstad: what is a "global service" | 21:14 |
| kmalloc | start from that definition | 21:14 |
| kmalloc | then determine if a limit data is relevant | 21:14 |
| lbragstad | well - an example being keystone itself | 21:14 |
| lbragstad | if your keystone is deployed globally | 21:14 |
| lbragstad | but you have nova in different regions | 21:14 |
| kmalloc | maybe the default region is then System? | 21:15 |
| ayoung | keystone is usually not assigned a region | 21:15 |
| kmalloc | or we create a system region... | 21:15 |
| lbragstad | ayoung: is that the reason why? | 21:15 |
| kmalloc | and just associate keystone to it | 21:15 |
| ayoung | but what quotas would we have for keystone itself? | 21:15 |
| kmalloc | ayoung: if we allow folks to manage domains: number of projects, number of users, | 21:15 |
| kmalloc | but that is about all i can come up with off the top of my head | 21:15 |
| cmurphy | None is already synonymous with system/global in my head | 21:15 |
| kmalloc | and even that feels... limited | 21:16 |
| ayoung | so those resources should not be assigned to a region | 21:16 |
| lbragstad | cmurphy: yeah - that's kinda what i'm thinking, too | 21:16 |
| kmalloc | or we could dodge the whole thing | 21:16 |
| kmalloc | and just say a limit w/o a region applies to endpoints without a region | 21:16 |
| kmalloc | aka, keystone | 21:16 |
| kmalloc | so None implicitly becomes a region for the limit | 21:17 |
| lbragstad | would that prevent someone from deploying keystone in two different regions and federating them? | 21:17 |
| kmalloc | and keystone is implicitly part of the None region | 21:17 |
| kmalloc | shouldn't | 21:17 |
| lbragstad | ok | 21:17 |
| kmalloc | nothing is saying keystone can't have a region | 21:17 |
| kmalloc | and nothing is saying keystone limits can't belong to the region if someone has it setup that way | 21:18 |
| lbragstad | right | 21:18 |
| * lbragstad feels like we're coming to a conclusion | 21:18 | |
| *** raildo has quit IRC | 21:20 | |
| *** sticker has joined #openstack-keystone | 21:22 | |
| lbragstad | something tells me that because regions are optional for services and endpoints, they need to be optional for limits | 21:23 |
| ayoung | kmalloc, I'm drawing a blank | 21:27 |
| ayoung | I need jamielennox here....I'm sure he knows this off the top of his head. | 21:29 |
| ayoung | so...I think the "Keysteon stuff goes into a region" does not really make sense. A project would be assigned a region, but it could also have to access resources in tow different regions, and so on, and thus the strict hierarchy there might get in the way | 21:46 |
| ayoung | Users come in from an external IdP. Would the IdPs go into a region? Seems wonky | 21:46 |
| *** oomichi_afk is now known as oomichi | 21:48 | |
| cmurphy | errr: this works for me http://paste.openstack.org/show/627765/ same version of openstackclient and keystoneauth :/ I see there is no identity_provider_url in the debug output you linked, and i also see this "WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils" which I don't see, so maybe there is some version mismatch in one of osc's | 21:49 |
| cmurphy | dependencies that is causing all the arguments not to be read | 21:49 |
| *** oomichi is now known as oomichi_afk | 21:51 | |
| *** edmondsw has quit IRC | 21:51 | |
| *** edmondsw has joined #openstack-keystone | 21:53 | |
| kmalloc | ayoung: that was what i was saying, we should just let None apply to None region limits | 21:54 |
| kmalloc | ayoung: but it doesn't stop someone from placing keystone into a region | 21:55 |
| kmalloc | if someone wanted to *shrug* | 21:55 |
| *** edmondsw has quit IRC | 21:58 | |
| *** rcernin has joined #openstack-keystone | 22:03 | |
| *** jmlowe has quit IRC | 22:16 | |
| *** spilla has quit IRC | 22:17 | |
| *** aojea has quit IRC | 22:18 | |
| *** thorst has quit IRC | 22:19 | |
| *** jmlowe has joined #openstack-keystone | 22:31 | |
| *** edmondsw has joined #openstack-keystone | 22:32 | |
| *** edmondsw has quit IRC | 22:37 | |
| openstackgerrit | ayoung proposed openstack/keystone master: Use oslo-context https://review.openstack.org/523650 | 22:39 |
| *** edmondsw has joined #openstack-keystone | 22:48 | |
| *** thorst has joined #openstack-keystone | 22:50 | |
| jamielennox | ayoung: sup? | 22:53 |
| *** thorst has quit IRC | 22:55 | |
| *** thorst has joined #openstack-keystone | 22:56 | |
| * lbragstad apologizes | 22:59 | |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Add a new table for system role assignments https://review.openstack.org/507993 | 22:59 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Implement backend logic for system roles https://review.openstack.org/507994 | 22:59 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Implement manager logic for user+system roles https://review.openstack.org/512468 | 22:59 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Implement manager logic for group+system roles https://review.openstack.org/512641 | 22:59 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Add user system grant policies https://review.openstack.org/514471 | 22:59 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Add group system grant policies https://review.openstack.org/514725 | 22:59 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Implement controller logic for system user assignments https://review.openstack.org/515215 | 22:59 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Implement controller logic for system group assignments https://review.openstack.org/524017 | 22:59 |
| lbragstad | skadoosh | 22:59 |
| *** thorst has quit IRC | 23:01 | |
| *** d0ugal has joined #openstack-keystone | 23:04 | |
| *** hoonetorg has joined #openstack-keystone | 23:10 | |
| *** jmlowe has quit IRC | 23:13 | |
| *** AlexeyAbashkin has joined #openstack-keystone | 23:14 | |
| *** AlexeyAbashkin has quit IRC | 23:19 | |
| *** jmlowe has joined #openstack-keystone | 23:21 | |
| *** linkmark has quit IRC | 23:25 | |
| *** gmann_afk is now known as gmann | 23:31 | |
| *** rderose has quit IRC | 23:32 | |
| *** thorst has joined #openstack-keystone | 23:35 | |
| *** thorst has quit IRC | 23:41 | |
| *** tesseract has joined #openstack-keystone | 23:48 | |
| *** tesseract has quit IRC | 23:49 | |
| ayoung | jamielennox, hey...posted an update but still not working | 23:57 |
| ayoung | jamielennox, what converst the headers like 'X%s-User-Name' to HTTP_USER_NAME and so on? | 23:58 |
| jamielennox | ayoung: wsgi | 23:58 |
| jamielennox | or i guess webob | 23:58 |
| ayoung | jamielennox, so doesthat convert it further down the stack? | 23:59 |
| jamielennox | but that's a fairly standard (at least python) convention that request.headers['custom_name'] is just a wrapper around environ['HTTP_X_CUSTOM_NAME'] | 23:59 |
| jamielennox | or i guess 'x-custom-name' | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!