| *** code-R has joined #openstack-keystone | 00:01 | |
| *** spzala has joined #openstack-keystone | 00:02 | |
| *** code-R_ has joined #openstack-keystone | 00:03 | |
| *** su_zhang has quit IRC | 00:04 | |
| *** tqtran has quit IRC | 00:05 | |
| *** code-R has quit IRC | 00:06 | |
| *** tqtran has joined #openstack-keystone | 00:06 | |
| *** spzala has quit IRC | 00:07 | |
| openstackgerrit | Merged openstack/keystone: Trust controller refactoring https://review.openstack.org/351260 | 00:12 |
|---|---|---|
| openstackgerrit | Tin Lam proposed openstack/keystone: api-ref: Document domain specific roles https://review.openstack.org/356169 | 00:16 |
| *** roxanaghe has quit IRC | 00:20 | |
| *** hockeynut has joined #openstack-keystone | 00:22 | |
| *** GB21 has joined #openstack-keystone | 00:31 | |
| *** adrian_otto has joined #openstack-keystone | 00:39 | |
| *** thumpba_ has quit IRC | 00:39 | |
| *** thumpba has joined #openstack-keystone | 00:39 | |
| *** adrian_otto has quit IRC | 00:43 | |
| *** thumpba has quit IRC | 00:44 | |
| *** gyee has quit IRC | 00:46 | |
| *** tqtran has quit IRC | 00:51 | |
| *** chlong has joined #openstack-keystone | 00:58 | |
| *** hockeynut has quit IRC | 01:02 | |
| *** hockeynut has joined #openstack-keystone | 01:02 | |
| *** spzala has joined #openstack-keystone | 01:04 | |
| *** woodster_ has quit IRC | 01:09 | |
| *** spzala has quit IRC | 01:09 | |
| openstackgerrit | Jamie Lennox proposed openstack/keystone: Move audit initiator creation to request https://review.openstack.org/342658 | 01:29 |
| *** adrian_otto has joined #openstack-keystone | 01:34 | |
| *** GB21 has quit IRC | 01:35 | |
| *** adrian_otto has quit IRC | 01:39 | |
| *** code-R_ has quit IRC | 01:42 | |
| *** EinstCrazy has joined #openstack-keystone | 01:42 | |
| *** BjoernT has joined #openstack-keystone | 01:44 | |
| *** julim has joined #openstack-keystone | 01:46 | |
| *** BjoernT has quit IRC | 01:49 | |
| *** tqtran has joined #openstack-keystone | 01:53 | |
| *** esp has quit IRC | 01:57 | |
| *** tqtran has quit IRC | 01:57 | |
| *** davechen has joined #openstack-keystone | 02:00 | |
| *** EinstCrazy has quit IRC | 02:01 | |
| *** EinstCrazy has joined #openstack-keystone | 02:03 | |
| *** asettle has joined #openstack-keystone | 02:10 | |
| *** sdake has quit IRC | 02:13 | |
| *** sdake has joined #openstack-keystone | 02:13 | |
| *** EinstCra_ has joined #openstack-keystone | 02:16 | |
| *** GB21 has joined #openstack-keystone | 02:16 | |
| *** julim has quit IRC | 02:17 | |
| *** asettle has quit IRC | 02:17 | |
| *** EinstCrazy has quit IRC | 02:18 | |
| *** adrian_otto has joined #openstack-keystone | 02:27 | |
| *** spzala has joined #openstack-keystone | 02:29 | |
| *** spzala has quit IRC | 02:30 | |
| *** spzala has joined #openstack-keystone | 02:30 | |
| *** adrian_otto has quit IRC | 02:33 | |
| *** haplo37_ has quit IRC | 02:43 | |
| *** spzala has quit IRC | 02:44 | |
| *** spzala has joined #openstack-keystone | 02:44 | |
| *** GB21 has quit IRC | 02:44 | |
| *** spzala has quit IRC | 02:48 | |
| *** EinstCrazy has joined #openstack-keystone | 02:56 | |
| *** su_zhang has joined #openstack-keystone | 02:59 | |
| *** EinstCra_ has quit IRC | 03:00 | |
| *** thumpba has joined #openstack-keystone | 03:19 | |
| *** su_zhang has quit IRC | 03:20 | |
| *** adrian_otto has joined #openstack-keystone | 03:23 | |
| *** su_zhang has joined #openstack-keystone | 03:23 | |
| *** esp has joined #openstack-keystone | 03:25 | |
| *** adrian_otto has quit IRC | 03:28 | |
| *** su_zhang has quit IRC | 03:31 | |
| *** sdake has quit IRC | 03:34 | |
| *** wangqun has joined #openstack-keystone | 03:42 | |
| stevemar | anyone want to take a look at https://review.openstack.org/#/c/350704/2 ? | 03:44 |
| patchbot | stevemar: patch 350704 - keystone - Make all token provider behave the same with trusts | 03:44 |
| openstackgerrit | Anh Tran proposed openstack/keystone: api-ref: Fix parameters attributes https://review.openstack.org/356211 | 03:48 |
| *** links has joined #openstack-keystone | 03:54 | |
| *** tqtran has joined #openstack-keystone | 03:55 | |
| *** dikonoor has joined #openstack-keystone | 03:57 | |
| *** tqtran has quit IRC | 03:59 | |
| *** hockeynut has quit IRC | 04:03 | |
| *** GB21 has joined #openstack-keystone | 04:06 | |
| *** su_zhang has joined #openstack-keystone | 04:16 | |
| *** GB21 has quit IRC | 04:23 | |
| *** code-R has joined #openstack-keystone | 04:44 | |
| *** code-R_ has joined #openstack-keystone | 04:49 | |
| *** code-R has quit IRC | 04:52 | |
| *** GB21 has joined #openstack-keystone | 04:54 | |
| *** adrian_otto has joined #openstack-keystone | 05:10 | |
| *** jaosorior has joined #openstack-keystone | 05:13 | |
| *** adrian_otto has quit IRC | 05:14 | |
| *** itisha has quit IRC | 05:20 | |
| *** code-R_ has quit IRC | 05:24 | |
| *** code-R has joined #openstack-keystone | 05:24 | |
| *** code-R has quit IRC | 05:30 | |
| *** su_zhang has quit IRC | 05:39 | |
| *** thumpba has quit IRC | 05:40 | |
| *** rcernin has joined #openstack-keystone | 05:46 | |
| *** thumpba has joined #openstack-keystone | 05:51 | |
| *** thumpba has quit IRC | 05:56 | |
| *** asettle has joined #openstack-keystone | 06:00 | |
| *** asettle has quit IRC | 06:05 | |
| *** roxanaghe has joined #openstack-keystone | 06:16 | |
| *** roxanagh_ has joined #openstack-keystone | 06:16 | |
| *** roxanaghe has quit IRC | 06:20 | |
| *** roxanagh_ has quit IRC | 06:22 | |
| *** adriant has quit IRC | 06:37 | |
| *** code-R has joined #openstack-keystone | 06:38 | |
| *** code-R has quit IRC | 06:43 | |
| *** code-R has joined #openstack-keystone | 06:51 | |
| *** belmoreira has joined #openstack-keystone | 06:53 | |
| *** adrian_otto has joined #openstack-keystone | 06:58 | |
| *** markvoelker has quit IRC | 07:00 | |
| *** tesseract- has joined #openstack-keystone | 07:00 | |
| *** adrian_otto has quit IRC | 07:03 | |
| *** code-R has quit IRC | 07:07 | |
| *** GB21 has quit IRC | 07:10 | |
| *** amoralej|off is now known as amoralej | 07:26 | |
| *** GB21 has joined #openstack-keystone | 07:44 | |
| *** jamielennox is now known as jamielennox|away | 07:45 | |
| *** pnavarro has joined #openstack-keystone | 07:48 | |
| *** thumpba has joined #openstack-keystone | 07:52 | |
| *** thumpba has quit IRC | 07:57 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** markvoelker has joined #openstack-keystone | 08:00 | |
| *** thumpba has joined #openstack-keystone | 08:01 | |
| *** jed56 has joined #openstack-keystone | 08:01 | |
| *** zzzeek has joined #openstack-keystone | 08:01 | |
| *** markvoelker has quit IRC | 08:05 | |
| *** thumpba has quit IRC | 08:06 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/keystone: [WIP] Testing latest u-c https://review.openstack.org/318435 | 08:10 |
| *** belmoreira has quit IRC | 08:18 | |
| *** asettle has joined #openstack-keystone | 08:22 | |
| *** rdopiera has joined #openstack-keystone | 08:31 | |
| *** dkehn_ has quit IRC | 08:32 | |
| breton | morning, keystone | 08:34 |
| *** Jaison has joined #openstack-keystone | 08:35 | |
| *** links has quit IRC | 08:37 | |
| openstackgerrit | henry-nash proposed openstack/keystone: Add expand, data migration and contract logic to keystone-manage https://review.openstack.org/349939 | 08:45 |
| *** adrian_otto has joined #openstack-keystone | 08:47 | |
| *** jamielennox|away is now known as jamielennox | 08:50 | |
| *** adrian_otto has quit IRC | 08:51 | |
| *** dkehn_ has joined #openstack-keystone | 08:52 | |
| *** GB21 has quit IRC | 08:52 | |
| *** asettle has quit IRC | 08:59 | |
| *** markvoelker has joined #openstack-keystone | 09:01 | |
| *** asettle has joined #openstack-keystone | 09:01 | |
| *** asettle has quit IRC | 09:02 | |
| *** thumpba has joined #openstack-keystone | 09:02 | |
| *** asettle has joined #openstack-keystone | 09:03 | |
| *** markvoelker has quit IRC | 09:06 | |
| *** GB21 has joined #openstack-keystone | 09:06 | |
| *** thumpba has quit IRC | 09:08 | |
| *** tqtran has joined #openstack-keystone | 09:19 | |
| *** asettle has quit IRC | 09:20 | |
| *** gus has quit IRC | 09:22 | |
| aloga | stevemar: are you around for a short discussion about oidc? | 09:23 |
| *** tqtran has quit IRC | 09:23 | |
| *** maestropandy has joined #openstack-keystone | 09:29 | |
| *** maestropandy has left #openstack-keystone | 09:30 | |
| *** code-R has joined #openstack-keystone | 09:30 | |
| *** mvk has quit IRC | 09:35 | |
| *** asettle has joined #openstack-keystone | 09:36 | |
| *** asettle has quit IRC | 09:38 | |
| *** jamielennox is now known as jamielennox|away | 09:43 | |
| *** spzala has joined #openstack-keystone | 09:45 | |
| *** spzala has quit IRC | 09:49 | |
| *** amakarov_away is now known as amakarov | 09:49 | |
| *** dikonoor has quit IRC | 09:56 | |
| *** openstack has joined #openstack-keystone | 10:15 | |
| *** sdake_ has joined #openstack-keystone | 10:16 | |
| *** sdake_ has quit IRC | 10:18 | |
| *** sdake_ has joined #openstack-keystone | 10:19 | |
| *** sdake has quit IRC | 10:19 | |
| *** dkehn_ has joined #openstack-keystone | 10:22 | |
| *** adrian_otto has joined #openstack-keystone | 10:35 | |
| *** adrian_otto has quit IRC | 10:40 | |
| *** EinstCrazy has quit IRC | 10:40 | |
| stevemar | morning breton | 10:43 |
| stevemar | aloga: i am semi-here now | 10:43 |
| aloga | stevemar: Hi there o/ | 10:44 |
| stevemar | hidy ho | 10:44 |
| aloga | stevemar: I have been testing and working on the OpenID Connect and Keystone integration | 10:46 |
| aloga | stevemar: so far, this works perfectly when using the dashboard, the user browser and mod_auth_openidc | 10:46 |
| stevemar | ++ | 10:46 |
| stevemar | good to hear... | 10:46 |
| aloga | stevemar: since the OpenID Connect client is Apache, and it gets the access token and id token from the oidc server, and then it gets the user information from the user information endpoint | 10:47 |
| aloga | stevemar: so we can map any additional claims (for example email) | 10:48 |
| aloga | stevemar: this works flawless | 10:48 |
| aloga | stevemar: the problem here comes with the CLI... | 10:48 |
| aloga | i.e. openstackclient + keystoneauth1 | 10:48 |
| aloga | in this case, in the CLI we obtain an access token from the oidc server | 10:49 |
| aloga | then we exchange this token with keystone, to obtain a keystone token | 10:49 |
| aloga | this exchange is done in a OAuth 2.0 protected url (/v3/OS-FEDERATION/identity_providers/<idp>/protocols/<protocol>/auth), since this is not oidc but oauth | 10:50 |
| aloga | this works ok as well | 10:50 |
| aloga | as long as you only rely in the access token claims to do the mapping | 10:50 |
| aloga | :( | 10:51 |
| aloga | since the URL is proceted using oauth20 (i.e AuthType oauth2 from mod_auth_openidc) there is no way to fetch the user information from the user information endpoint, since this is somethig specific of openid connect | 10:53 |
| *** asettle has joined #openstack-keystone | 10:55 | |
| stevemar | let me re-read that | 10:56 |
| aloga | sure | 10:57 |
| *** mvk has quit IRC | 10:58 | |
| aloga | stevemar: the problem here is who is the OpenID Connect client | 11:01 |
| aloga | in the horizon case, it is the apache server where keystone is running, therefore the openid connect client gets the user information and it is able to pass it down to the keystone app | 11:01 |
| aloga | in the CLI case, the oidc client is keystoneauth1, therefore it is not able to fetch this information | 11:02 |
| aloga | well, it is able to fetch it, but it is useless as it cannot be passed to keystone | 11:02 |
| stevemar | aloga: i thought you could setup mod_auth_oidc to do introspection to get user details? | 11:02 |
| aloga | stevemar: yes, indeed | 11:02 |
| aloga | stevemar: but only introspection on the access_token | 11:02 |
| aloga | :( | 11:03 |
| aloga | i.e. you won't be able to get any additional claims (email, preferred_username, etc.) | 11:03 |
| *** marekd2 has joined #openstack-keystone | 11:09 | |
| stevemar | ahhh | 11:13 |
| stevemar | but you'll still get the base claims? | 11:14 |
| aloga | yes, so to be clear, if the mapping rules only rely on the access token claims (iss, sub, etc.) this works | 11:14 |
| aloga | however, if somebody wants to create some additional mapping rules based on the OpenID Connect user info claims, this won't work | 11:14 |
| stevemar | (via the CLI) | 11:16 |
| stevemar | thats frustrating | 11:16 |
| aloga | stevemar: yes | 11:16 |
| stevemar | aloga: anyway to make additional calls at the keystoneauth level? | 11:16 |
| stevemar | to get additional claims | 11:16 |
| aloga | stevemar: yes, we could get the user claims since the user info endpoint should be protected using oauth 2.0, and we can use the access token to access it | 11:17 |
| aloga | i.e. we only need to make an additional call to fetch that information | 11:18 |
| aloga | however, how can we pass this to keystone? | 11:18 |
| *** sdake_ is now known as sdake | 11:24 | |
| aloga | stevemar: one possibility is that keystone is the one fetching the user information | 11:25 |
| aloga | stevemar: before the mapping | 11:25 |
| openstackgerrit | henry-nash proposed openstack/keystone: Add expand, data migration and contract logic to keystone-manage https://review.openstack.org/349939 | 11:29 |
| stevemar | henrynash: i PMed you, not sure if you're around, or your laptop is busy processing 100 notifications from all your open apps | 11:30 |
| *** jaosorior has quit IRC | 11:35 | |
| *** jaosorior has joined #openstack-keystone | 11:36 | |
| *** su_zhang has joined #openstack-keystone | 11:39 | |
| *** amoralej is now known as amoralej|lunch | 11:40 | |
| stevemar | aloga: not sure if we can add that to keystone, it's pretty specific to the protocol right | 11:44 |
| aloga | stevemar: for the record, if I am not mistaken, google is adding the user information to the response obtained from the oauth 2.0 introspection endpoint | 11:46 |
| aloga | stevemar: however that's not the case for all the oauth providers, as the specification does not force it https://tools.ietf.org/html/rfc7662#section-2.2 | 11:46 |
| stevemar | ah | 11:47 |
| stevemar | we (as keystone) could just tell folks to talk to whoever implements their OIDC :P | 11:48 |
| stevemar | but they may even be in the same organization or workplace | 11:48 |
| aloga | stevemar: regarding keystone doing the call to the userinfo endpoint that's oidc specific, but we could implement an auth plugin for that | 11:51 |
| aloga | instead of using the mapped plugin directly | 11:51 |
| aloga | (i.e. oidc = keystone.auth.plugins.mapped.Mapped) | 11:51 |
| stevemar | aloga: so, a slightly different topic, but let me know if it makes sense | 11:52 |
| aloga | go ahead | 11:52 |
| stevemar | aloga: dstanek wants to add code to keystone to handle all the SAML requests, so he doesn't have to use mod_auth_saml in apache | 11:52 |
| stevemar | he wants to use the pysaml, offer up an API, and handle everything with pysaml instead of mod_auth_saml | 11:53 |
| stevemar | the reasoning behind this is that it'll scale better with automation | 11:53 |
| stevemar | managing the apache configuration is not fun | 11:54 |
| stevemar | would a similar idea be useful for oidc? | 11:54 |
| aloga | stevemar: yes, indeed | 11:54 |
| stevemar | you could put your additional user info call there :P | 11:55 |
| aloga | stevemar: the problem with Apache, as I see it, is that you need to configure things at two sides: keystone (i.e. the mapping) and Apache (i.e. the OpenID connect configuration) | 11:56 |
| aloga | if you need to manage several oidc providers, that's a bit messy | 11:56 |
| stevemar | aloga: yeah, thats where dstanek and rax/intel are having their issues | 11:57 |
| aloga | moreover, if you need to add the proctected urls with oauth20 for the introspection... | 11:57 |
| stevemar | they can't do that easily with automation | 11:57 |
| *** julim has joined #openstack-keystone | 12:01 | |
| *** sdake_ has joined #openstack-keystone | 12:01 | |
| *** code-R_ has joined #openstack-keystone | 12:03 | |
| *** sdake has quit IRC | 12:05 | |
| *** sdake_ is now known as sdake | 12:05 | |
| *** code-R has quit IRC | 12:06 | |
| *** su_zhang has quit IRC | 12:07 | |
| *** su_zhang has joined #openstack-keystone | 12:07 | |
| stevemar | aloga: i don't have an easy answer for you :) | 12:08 |
| aloga | stevemar: none of the solutions are perfect | 12:08 |
| aloga | stevemar: some require hard work, some are not too clean | 12:09 |
| aloga | :( | 12:09 |
| *** markvoelker has joined #openstack-keystone | 12:09 | |
| stevemar | aloga: i'm guessing this impacts you highly? | 12:09 |
| stevemar | aloga: you have a mapping that depends on additional claims now in the default claim | 12:10 |
| aloga | stevemar: well, i need to check, maybe we can survive for the moment | 12:10 |
| aloga | but this is part of a larger project, were we would need for sure more complex mappings | 12:10 |
| aloga | like mappings based on groups available in the additional claims | 12:11 |
| *** jamielennox|away is now known as jamielennox | 12:11 | |
| stevemar | aloga: i'm guessing when i did this https://developer.ibm.com/opentech/2015/06/17/use-websphere-liberty-as-an-openid-connect-provider-for-openstack/ it wasn't the same :( | 12:12 |
| *** su_zhang has quit IRC | 12:12 | |
| aloga | stevemar: that's more or less the configuration that we have | 12:14 |
| *** rodrigods has quit IRC | 12:14 | |
| *** rodrigods has joined #openstack-keystone | 12:14 | |
| aloga | the problem is that the server is not returning the additional claims in the instrospect response | 12:14 |
| *** jpena is now known as jpena|lunch | 12:15 | |
| aloga | unfortunately that depends on the server :( | 12:15 |
| breton | lol @ bug 1614069 | 12:19 |
| openstack | bug 1614069 in OpenStack Identity (keystone) "API v2.0 responds with HTTP 200 when trying to add a non-existent user to a project" [Undecided,New] https://launchpad.net/bugs/1614069 | 12:19 |
| *** marekd2 has quit IRC | 12:20 | |
| *** marekd2 has joined #openstack-keystone | 12:21 | |
| stevemar | breton: unfotuntately i think we removed the validation that checks for users and projects when creating a role assignment | 12:23 |
| stevemar | and we never put them back | 12:23 |
| stevemar | breton: abandon: https://review.openstack.org/#/c/347876/ ? :) | 12:24 |
| patchbot | stevemar: patch 347876 - keystone - test | 12:24 |
| breton | stevemar: done, thanks | 12:25 |
| *** marekd2 has quit IRC | 12:25 | |
| *** GB21 has quit IRC | 12:28 | |
| *** raildo has joined #openstack-keystone | 12:29 | |
| stevemar | now i can't find that patch | 12:31 |
| *** marekd2 has joined #openstack-keystone | 12:31 | |
| *** spzala_ has joined #openstack-keystone | 13:01 | |
| *** EinstCrazy has joined #openstack-keystone | 13:03 | |
| *** gordc has joined #openstack-keystone | 13:05 | |
| *** mvk has joined #openstack-keystone | 13:06 | |
| *** amoralej|lunch is now known as amoralej | 13:12 | |
| *** su_zhang has joined #openstack-keystone | 13:13 | |
| *** jpena|lunch is now known as jpena | 13:18 | |
| *** tqtran has joined #openstack-keystone | 13:20 | |
| *** gagehugo has joined #openstack-keystone | 13:21 | |
| *** EinstCrazy has quit IRC | 13:22 | |
| *** tqtran has quit IRC | 13:25 | |
| *** eandersson_ has joined #openstack-keystone | 13:30 | |
| *** woodster_ has joined #openstack-keystone | 13:34 | |
| *** edmondsw has joined #openstack-keystone | 13:37 | |
| henrynash | anyone have an understanding of oslo_db test_base DBFixtures? We use them for test_sql_upgrade and am getting weird interaction between test classes (like a schema of one database type seemingly still in place when test class for a different db comes around) | 13:39 |
| *** ametts has joined #openstack-keystone | 13:40 | |
| *** Jaison has quit IRC | 13:46 | |
| *** sdake has quit IRC | 13:50 | |
| *** thumpba has joined #openstack-keystone | 13:52 | |
| stevemar | henrynash: maybe bknudson ? | 14:00 |
| openstackgerrit | Gage Hugo proposed openstack/keystone: [api] add relationship links to v3-ext https://review.openstack.org/356485 | 14:00 |
| *** nishaYadav has joined #openstack-keystone | 14:00 | |
| stevemar | gagehugo: change the commit message to "Closes-Bug" :) | 14:00 |
| henrynash | stevemar: yep was going to ask him....he is our liason after all :-) | 14:00 |
| bknudson | henrynash: not sure I follow -- database type? Like mysql vs postgresql? | 14:00 |
| nishaYadav | o/ | 14:01 |
| *** sdake_ has joined #openstack-keystone | 14:01 | |
| openstackgerrit | Gage Hugo proposed openstack/keystone: [api] add relationship links to v3-ext https://review.openstack.org/356485 | 14:01 |
| gagehugo | stevemar: done | 14:01 |
| stevemar | gagehugo: ty | 14:01 |
| henrynash | bknudson: yep...I have 3 classes (each which runs a set of tests), one for sqlite, mysql and postgres | 14:01 |
| henrynash | bknudson: each work fine if I run them on their own | 14:02 |
| gagehugo | stevemar: some of the pages already had the links brought over just fyi | 14:02 |
| bknudson | henrynash: can you post the code in progress? | 14:02 |
| stevemar | gagehugo: yes, i recall someone did it when they brought over endpoint policy or endpoint filtering | 14:03 |
| stevemar | gagehugo: i question if those are correct | 14:03 |
| henrynash | bknudson: it's already up for review: https://review.openstack.org/#/c/349939/22/keystone/tests/unit/test_sql_upgrade.py | 14:03 |
| patchbot | henrynash: patch 349939 - keystone - Add expand, data migration and contract logic to k... | 14:03 |
| gagehugo | They seemed alright, I can take another look | 14:03 |
| lbragstad | henrynash you're familiar with the driver_hints stuff right? | 14:04 |
| henrynash | lbragstad: yep | 14:04 |
| lbragstad | henrynash have time for a real quick (probably trivial) question? | 14:04 |
| henrynash | lbragstad: sure | 14:05 |
| lbragstad | henrynash so I just migrated a column in the database | 14:05 |
| lbragstad | henrynash which is pretty straight forward - https://review.openstack.org/#/c/355618/ | 14:05 |
| patchbot | lbragstad: patch 355618 - keystone - Add key_hash column to credential table | 14:05 |
| lbragstad | henrynash and I want to ask the credential api for any credentials that have 'key_hash' set to None (meaning they haven't been encrypted yet) | 14:06 |
| lbragstad | henrynash when I do this - it blows up in the sql code http://cdn.pasteraw.com/j1eybjhsqe0h8q0d7e7hig4plhibm2g | 14:06 |
| lbragstad | in ^ that specific case, I have two filters that I want applied | 14:06 |
| lbragstad | but it doesn't seem to like that fact that the second one has a value of None | 14:07 |
| lbragstad | but it looks like we use that pattern in other places | 14:07 |
| lbragstad | https://github.com/openstack/keystone/blob/0b4f6ebdcc866388e1c6788f45f270414b45aeef/keystone/tests/unit/test_backend_sql.py#L506 | 14:07 |
| lbragstad | and | 14:07 |
| lbragstad | https://github.com/openstack/keystone/blob/0b4f6ebdcc866388e1c6788f45f270414b45aeef/keystone/assignment/controllers.py#L437 | 14:07 |
| henrynash | lbragstad: indeed | 14:08 |
| lbragstad | henrynash am I just using this wrong? | 14:08 |
| lbragstad | s/this/hints/ | 14:08 |
| *** SamYaple has quit IRC | 14:09 | |
| henrynash | lbragstad: looking | 14:09 |
| *** SamYaple has joined #openstack-keystone | 14:09 | |
| henrynash | lbragstad: and where's teh code where you actually create teh hints filter? | 14:10 |
| *** esp has quit IRC | 14:10 | |
| lbragstad | henrynash let me grab a paste | 14:10 |
| *** tonytan4ever has joined #openstack-keystone | 14:10 | |
| lbragstad | henrynash http://cdn.pasteraw.com/5ka0mhn5lvilr1mle18zhyfssp9nr1e | 14:11 |
| gagehugo | stevemar: yes they are good | 14:11 |
| *** SamYaple has quit IRC | 14:11 | |
| *** SamYaple has joined #openstack-keystone | 14:11 | |
| *** adrian_otto has joined #openstack-keystone | 14:12 | |
| *** SamYaple has quit IRC | 14:13 | |
| *** SamYaple has joined #openstack-keystone | 14:13 | |
| *** spzala_ has quit IRC | 14:14 | |
| openstackgerrit | Mikhail Nikolaenko proposed openstack/keystone: [WIP] Move fernet utils to backend https://review.openstack.org/356499 | 14:14 |
| *** spzala has joined #openstack-keystone | 14:14 | |
| *** ezpz has joined #openstack-keystone | 14:15 | |
| henrynash | lbragstad: hmm, odd...the check() method was added after I wrote the stuff...but we do seem you use it with None values.... | 14:15 |
| lbragstad | henrynash apparently using '' instead of None works! | 14:15 |
| lbragstad | thanks dstanek ;) | 14:15 |
| henrynash | lbragstad: I've got to go offline for a bit (flight to catch), but will look at it more | 14:15 |
| lbragstad | henrynash sounds good - thanks | 14:15 |
| henrynash | lbragstad: sounds like a bug in teh check() code | 14:16 |
| lbragstad | henrynash yeah - either way we should probably address that case? | 14:16 |
| *** SamYaple has quit IRC | 14:17 | |
| *** spzala has quit IRC | 14:17 | |
| *** adrian_otto has quit IRC | 14:17 | |
| *** spzala has joined #openstack-keystone | 14:17 | |
| *** SamYaple has joined #openstack-keystone | 14:17 | |
| *** ravelar has joined #openstack-keystone | 14:18 | |
| *** lamt has joined #openstack-keystone | 14:19 | |
| stevemar | gagehugo: coolio | 14:19 |
| *** hockeynut has joined #openstack-keystone | 14:20 | |
| *** haplo37_ has joined #openstack-keystone | 14:22 | |
| *** esp has joined #openstack-keystone | 14:25 | |
| *** itisha has joined #openstack-keystone | 14:30 | |
| rdopiera | hi guys, anywbody has a moment to chat about bug https://bugs.launchpad.net/horizon/+bug/1415588 ? | 14:35 |
| openstack | Launchpad bug 1415588 in OpenStack Dashboard (Horizon) "Cannot list users and groups with Keystone v3" [Undecided,New] | 14:35 |
| *** edtubill has joined #openstack-keystone | 14:35 | |
| rdopiera | I'm trying to fix it by providing a more informative error message, but I have a hard time telling this case from an expired session or a similar error | 14:35 |
| rdopiera | I wonder if "unauthorized" is really the best error to return in this case | 14:36 |
| *** ezpz has quit IRC | 14:37 | |
| *** michauds has joined #openstack-keystone | 14:40 | |
| *** marekd2 has quit IRC | 14:46 | |
| *** erhudy has joined #openstack-keystone | 14:51 | |
| *** spedione|AWAY is now known as spedione | 14:53 | |
| *** rvba has quit IRC | 14:59 | |
| *** nishaYadav has quit IRC | 14:59 | |
| *** openstackgerrit has quit IRC | 15:03 | |
| *** openstackgerrit has joined #openstack-keystone | 15:04 | |
| ayoung | rdopiera, looking | 15:11 |
| ayoung | rdopiera, so, yes, the problem is that the user is trying to do an operation that they are not authorized to perform. In this use case, Horizon devs have enough knowledge of how to work around it, and you could put that info into the message returned to the user, no? | 15:13 |
| *** marekd2 has joined #openstack-keystone | 15:14 | |
| *** BjoernT has joined #openstack-keystone | 15:15 | |
| *** nishaYadav has joined #openstack-keystone | 15:16 | |
| openstackgerrit | Nisha Yadav proposed openstack/python-keystoneclient: Add ec2 functional tests https://review.openstack.org/350245 | 15:18 |
| nishaYadav | samueldmq, rodrigods stevemar please have a look ^ | 15:18 |
| rdopiera | ayoung: how can I distinguish that "unauthorized" and an expired session just happening in the same place? | 15:20 |
| ayoung | rdopiera, I don't know. If there is any question, Horizon can always try to validate the users token, but, in general, it seems more like something you should not request if you don't have reason to think the user can do it. | 15:21 |
| ayoung | rdopiera, geting Horizon sufficient info to do auth driven UI has been a long standing Windmill that I've tilted at | 15:22 |
| ayoung | we need the rest of the Keystone devs to accept that it is OK to tell a user "here is the role you need to perform that action" | 15:22 |
| ayoung | :) | 15:22 |
| rdopiera | ayoung: I mean, how can I tell the difference between "your session expired, please re-login" and "you don't have the right to perform this operation as this user"? | 15:23 |
| rdopiera | ayoung: if both return "Unauthorized: The request you have made requires authentication." | 15:23 |
| ayoung | rdopiera, is this the only place you have that problem? | 15:24 |
| stevemar | nishaYadav: yes ma'am! </salute> | 15:25 |
| rdopiera | it's the first place I encountered this problem, but I just took over horizon three weeks ago... | 15:25 |
| rdopiera | I suppose this will happen everywhere where keyston decides to return "requires authorization" to an already logged-in user | 15:25 |
| ayoung | rdopiera, yep. | 15:26 |
| ayoung | rdopiera, it would be nice if we distinguished. An expired token should give a different error than a forbidden operation | 15:26 |
| rdopiera | seems to be it should rather be something like "forbidden" | 15:26 |
| rdopiera | exactly | 15:26 |
| rdopiera | is there any chance for this? | 15:26 |
| ayoung | expired token should be 401. UNauthorized operation should be 403 | 15:27 |
| rdopiera | I suppose that's a change in behavior and would mean an api version bump? | 15:28 |
| *** hatTip has joined #openstack-keystone | 15:29 | |
| *** ravelar1 has joined #openstack-keystone | 15:29 | |
| *** ravelar has quit IRC | 15:29 | |
| ayoung | rdopiera, indubitably | 15:33 |
| rdopiera | allright, thank you for the information, that helps | 15:36 |
| openstackgerrit | Nisha Yadav proposed openstack/python-keystoneclient: Add auth functional tests https://review.openstack.org/356041 | 15:40 |
| *** su_zhang has quit IRC | 15:41 | |
| mfisch | crinkle: setting up to test your fix | 15:42 |
| mfisch | still need to build a container etc but getting close | 15:42 |
| *** thumpba has quit IRC | 15:44 | |
| *** code-R_ has quit IRC | 15:49 | |
| tsufiev | stevemar, o/ | 15:49 |
| stevemar | tsufiev: ahoy | 15:49 |
| tsufiev | stevemar, ducttape in horizon channel told me you're the right person to ask about domain quotas ;) | 15:50 |
| stevemar | tsufiev: eww, but okay. also, you can ask the channel in general :) | 15:50 |
| tsufiev | well, my initial question is quite broad: do you know anything about them? | 15:50 |
| *** ravelar1 has quit IRC | 15:51 | |
| tsufiev | since quoting his earlier reply 'I think part of domain quotas is to distribute management of quotas, really | 15:51 |
| tsufiev | and this has been a long standing background issue with domains too | 15:51 |
| tsufiev | stevemar might have more background | 15:51 |
| tsufiev | but it might be that domains are kind of a dead target too, and hierarchical projects really are the next thing' | 15:51 |
| AndyWojo | Hi, I'm trying to use Ceilometer, the instructions say to use v2, but I'm getting 401's with V2. but when I switch to v3 i get 404's | 15:51 |
| AndyWojo | Anyone have any pointers? | 15:51 |
| *** jaosorior has quit IRC | 15:51 | |
| tsufiev | I am really curious if HMT is a way to go for quotas spanning several tenants instead of domain quotas? | 15:52 |
| AndyWojo | http://paste.openstack.org/show/559064/ <-- that's my service stanza | 15:52 |
| AndyWojo | I get The request you have made requires authentication. (HTTP 401) (Request-ID: req-01d9427d-6679-459c-aebb-03217c3052d0) with v2 | 15:52 |
| stevemar | tsufiev: ugh, i don't even know the right answer any more | 15:56 |
| *** Ephur has joined #openstack-keystone | 15:56 | |
| tsufiev | :o | 15:56 |
| stevemar | tsufiev: i know we re-worked domains to just be projects... under the covers when you create a domain you are in fact creating a project with a special "is_domain=true" attribute | 15:57 |
| stevemar | tsufiev: this was done for HMT purposes, everythings a project | 15:57 |
| tsufiev | my initial guess was that keystone has almost nothing to do with quotas, even domain ones (despite the name) | 15:58 |
| tsufiev | perhaps ducttape gave me false lead :) | 15:58 |
| *** rcernin has quit IRC | 15:58 | |
| *** nisha_ has joined #openstack-keystone | 15:59 | |
| *** su_zhang has joined #openstack-keystone | 15:59 | |
| *** su_zhang has quit IRC | 15:59 | |
| *** lamt has quit IRC | 15:59 | |
| tsufiev | almost = just get a list of tenants inside a domain to sum up their quotas and usages | 15:59 |
| tsufiev | but... well, that doesn't seem like a scalable solution, on the other hand | 16:00 |
| *** sdake_ is now known as sdake | 16:00 | |
| *** sdake is now known as sdake_ | 16:00 | |
| *** nishaYadav has quit IRC | 16:02 | |
| stevemar | tsufiev: sorry, distracted. i want to give you more information | 16:04 |
| tsufiev | I'm all attention | 16:04 |
| stevemar | tsufiev: i know we don't manage or handle quotas directly in keystone | 16:04 |
| stevemar | theres no code for that | 16:05 |
| *** ravelar has joined #openstack-keystone | 16:05 | |
| stevemar | i want to say that each project should handle things consistently, but this is openstack we're talking about | 16:05 |
| *** dmellado is now known as dmellado|off | 16:05 | |
| tsufiev | yep, that's reasonable | 16:06 |
| *** amoralej is now known as amoralej|off | 16:06 | |
| stevemar | tsufiev: i would assume that a domain can be configured to have a quota that is spread out to all it's projects, or not have one at all | 16:07 |
| tsufiev | stevemar, do you mean 'parent gives the same amount to its children, which is equal to the one that parent has' by 'spread out'? | 16:09 |
| lbragstad | dstanek henrynash i opened a bug for now so that I can get back to the credential encryption work | 16:10 |
| lbragstad | https://bugs.launchpad.net/keystone/+bug/1614154 | 16:10 |
| openstack | Launchpad bug 1614154 in OpenStack Identity (keystone) "DriverHints with values of None seem to be broken" [Undecided,New] | 16:10 |
| stevemar | tsufiev: i was thinking a domain would have a quota of 100 jellybeans and project A could have 90 and project B could have 10 | 16:10 |
| tsufiev | yes, that was the same thing I was thinking about, stevemar | 16:10 |
| *** ezpz has joined #openstack-keystone | 16:11 | |
| *** code-R has joined #openstack-keystone | 16:13 | |
| *** nisha_ has quit IRC | 16:20 | |
| *** roxanaghe has joined #openstack-keystone | 16:20 | |
| *** asettle has quit IRC | 16:30 | |
| amakarov | stevemar, hi! | 16:34 |
| amakarov | https://gist.github.com/x-eye/8d2fc75f027b7e222284112787c8b13f | 16:34 |
| dstanek | lbragstad: thoughts on https://review.openstack.org/356596 ? | 16:38 |
| stevemar | amakarov: ahoy | 16:39 |
| stevemar | oh hey there | 16:39 |
| lbragstad | dstanek sure - that works | 16:39 |
| *** gyee has joined #openstack-keystone | 16:39 | |
| lbragstad | dstanek that just makes it so that when we run that test we always advance the clock (instead of only in the fernet case), right? | 16:40 |
| dstanek | lbragstad: yes | 16:40 |
| dstanek | hmmm....actually i have another test for Forbidden, but it looks like i lost that commit in my rebasing | 16:41 |
| lbragstad | dstanek sweet - does that get rid of the transient issue you were seeing? | 16:41 |
| dstanek | yep | 16:41 |
| lbragstad | \o/ | 16:41 |
| *** spzala has quit IRC | 16:41 | |
| dstanek | the issue was mostly because we were mocking time in 2 different ways | 16:41 |
| dstanek | not it only uses the fixture | 16:41 |
| *** spzala has joined #openstack-keystone | 16:42 | |
| lbragstad | sweet | 16:42 |
| amakarov | stevemar, that's the benchmark you asked for. It has some prerequisites I haven't stated: devstack and openrc environment | 16:43 |
| amakarov | It won't work another way | 16:43 |
| *** edtubill has quit IRC | 16:46 | |
| *** spzala has quit IRC | 16:46 | |
| lbragstad | dstanek I think I have most of the credential migration and rotation logic written - i'll push another patchset soon | 16:46 |
| lbragstad | dstanek I need to figure out how to test the rotation/migration stuff | 16:47 |
| dstanek | nice | 16:47 |
| stevemar | amakarov: that s cool though | 16:47 |
| dstanek | lbragstad: the bad tests that caused the Fernet Forbidden error actually showed an interesting corner case | 16:49 |
| lbragstad | dstanek really? | 16:50 |
| amakarov | stevemar, have some problems though... please wait | 16:50 |
| openstackgerrit | David Stanek proposed openstack/keystone: Add test for revocation corner case in Fernet https://review.openstack.org/356607 | 16:53 |
| dstanek | lbragstad: ^ see that one | 16:53 |
| lbragstad | dstanek huh - interesting | 16:56 |
| lbragstad | dstanek so that makes sure we're actually hitting the disabled user case. | 16:56 |
| lbragstad | and not the time issue | 16:56 |
| *** Gorian|work has joined #openstack-keystone | 16:56 | |
| dstanek | on a single node or maybe mutiple nodes that have exactly the same time Fernet will return unauthorized because of the revocation list. | 16:57 |
| dstanek | it is possible, however, to have a token generated in the future (which this test tests) or maybe even not having an updated revocation list (which would have the exact same behavior) | 16:58 |
| *** tesseract- has quit IRC | 17:00 | |
| dstanek | lunch time! | 17:01 |
| rderose | me too! | 17:10 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone: Implement encryption of credentials at rest https://review.openstack.org/317169 | 17:12 |
| lbragstad | dstanek ^ | 17:12 |
| lbragstad | i need to find a way to test that :) | 17:13 |
| lbragstad | lunch! | 17:14 |
| *** su_zhang has joined #openstack-keystone | 17:20 | |
| stevemar | lbragstad: enjoy! | 17:20 |
| *** roxanaghe has quit IRC | 17:21 | |
| *** tqtran has joined #openstack-keystone | 17:22 | |
| *** marekd2 has quit IRC | 17:23 | |
| *** asettle has joined #openstack-keystone | 17:23 | |
| *** marekd2 has joined #openstack-keystone | 17:23 | |
| *** tonytan4ever has quit IRC | 17:25 | |
| *** debu__ has joined #openstack-keystone | 17:26 | |
| *** rcernin has joined #openstack-keystone | 17:26 | |
| debu__ | hi...is there any way to find to which region a compute host belongs | 17:26 |
| *** tqtran has quit IRC | 17:27 | |
| *** marekd2 has quit IRC | 17:28 | |
| *** asettle has quit IRC | 17:28 | |
| *** mvk has quit IRC | 17:31 | |
| *** tqtran has joined #openstack-keystone | 17:35 | |
| amakarov | stevemar, I've updated my benchmark - now it works :) | 17:37 |
| *** su_zhang has quit IRC | 17:43 | |
| *** stevemar_ has joined #openstack-keystone | 17:43 | |
| *** ChanServ sets mode: +o stevemar_ | 17:43 | |
| *** stevemar_ has quit IRC | 17:43 | |
| stevemar | poke | 17:44 |
| *** hockeynut has quit IRC | 17:48 | |
| *** roxanaghe has joined #openstack-keystone | 17:51 | |
| *** spzala has joined #openstack-keystone | 17:53 | |
| *** amakarov is now known as amakarov_away | 17:55 | |
| *** spzala has quit IRC | 17:58 | |
| *** hockeynut has joined #openstack-keystone | 17:59 | |
| *** code-R has quit IRC | 18:00 | |
| *** spzala has joined #openstack-keystone | 18:00 | |
| *** pauloewerton has joined #openstack-keystone | 18:04 | |
| *** mvk has joined #openstack-keystone | 18:07 | |
| *** dkehn_ has quit IRC | 18:08 | |
| *** tqtran has quit IRC | 18:09 | |
| *** tqtran has joined #openstack-keystone | 18:19 | |
| *** dkehn_ has joined #openstack-keystone | 18:20 | |
| *** su_zhang has joined #openstack-keystone | 18:21 | |
| *** su_zhang has quit IRC | 18:25 | |
| *** tonytan4ever has joined #openstack-keystone | 18:26 | |
| *** tonytan4ever has quit IRC | 18:31 | |
| *** slberger has joined #openstack-keystone | 18:32 | |
| *** tqtran has quit IRC | 18:40 | |
| *** tqtran has joined #openstack-keystone | 18:43 | |
| *** ametts has quit IRC | 18:45 | |
| *** woodster_ has quit IRC | 18:49 | |
| *** hoonetorg has quit IRC | 18:53 | |
| *** ametts has joined #openstack-keystone | 18:56 | |
| *** debu__ has quit IRC | 18:56 | |
| *** rdopiera has left #openstack-keystone | 19:02 | |
| *** thumpba has joined #openstack-keystone | 19:05 | |
| *** jamielennox has quit IRC | 19:06 | |
| *** edtubill has joined #openstack-keystone | 19:06 | |
| *** hoonetorg has joined #openstack-keystone | 19:06 | |
| *** fifieldt has quit IRC | 19:07 | |
| *** jamielennox has joined #openstack-keystone | 19:09 | |
| *** ChanServ sets mode: +v jamielennox | 19:09 | |
| *** gyee has quit IRC | 19:11 | |
| samueldmq | jamielennox: stevemar: I need help to figure out what's going on in patch https://review.openstack.org/#/c/350245 | 19:14 |
| patchbot | samueldmq: patch 350245 - python-keystoneclient - Add ec2 functional tests | 19:14 |
| *** hatTip has quit IRC | 19:14 | |
| samueldmq | I have been trying to understand what is causing the failure, but no success | 19:14 |
| samueldmq | myself and nisha would appreciate some help :) | 19:14 |
| *** fifieldt has joined #openstack-keystone | 19:18 | |
| *** spzala has quit IRC | 19:18 | |
| *** spzala has joined #openstack-keystone | 19:19 | |
| *** spzala_ has joined #openstack-keystone | 19:22 | |
| stevemar | samueldmq: pfft, if you can't figure it out, what makes you think i can :| | 19:23 |
| *** spzala has quit IRC | 19:23 | |
| samueldmq | stevemar: not idea, you and jamielennox are configured as my gateway for the client | 19:25 |
| dolphm | lbragstad: is the performance bot running with memcache enabled now? regarding https://review.openstack.org/#/c/309146/ | 19:25 |
| patchbot | dolphm: patch 309146 - keystone - Pre-cache new tokens | 19:25 |
| samueldmq | :-) | 19:25 |
| samueldmq | stevemar: no* | 19:25 |
| stevemar | samueldmq: hehe | 19:25 |
| dolphm | lbragstad: i.e. did you confirm memcache is being used with my patch? | 19:25 |
| *** spzala_ has quit IRC | 19:26 | |
| lbragstad | dolphm I did not | 19:27 |
| lbragstad | I talked to odyssey4me about it for a little bit a couple weeks ago (i believe stevemar and I were asking about it) | 19:27 |
| *** ayoung has quit IRC | 19:27 | |
| dolphm | lbragstad: and? | 19:27 |
| *** edmondsw has quit IRC | 19:28 | |
| lbragstad | odyssey4me made it sound like there were a couple other things that needed to be done | 19:29 |
| samueldmq | lbragstad: dolphm: on change 354495, if we consider encryption to be enabled, what do we do just after upgrade when the creds are not ecrypted yet | 19:29 |
| samueldmq | ? | 19:29 |
| lbragstad | dolphm I'd have to do dig in the irc logs to find it | 19:29 |
| samueldmq | patch 354495 | 19:29 |
| patchbot | samueldmq: https://review.openstack.org/#/c/354495/ - keystone - Add conf to support credential encryption | 19:29 |
| dolphm | samueldmq: what DO we do, or what SHOULD we do? ;) | 19:29 |
| samueldmq | dolphm: well, we are supposed to keep things running after an upgrade | 19:31 |
| lbragstad | If encryption is enabled we should check the key_hash value when getting a credential | 19:31 |
| samueldmq | :-) | 19:31 |
| lbragstad | if a credential has a key_hash then we know for a fact it was encrypted at some point, either through the credential CRUD or through the migration | 19:31 |
| samueldmq | hmm | 19:31 |
| lbragstad | if the key_hash is None then we know the credential is still in it's plaintext form - and we should return that | 19:32 |
| samueldmq | we can still retrieve old credentials that are not encrypted | 19:32 |
| samueldmq | but we can't create new ones without encrypting | 19:32 |
| samueldmq | dolphm: lbragstad: is this the idea ? ^ | 19:32 |
| lbragstad | newly created credentials should be encrypted if encryption is enabled - but should be able to retrieve old credentials that haven't been encrypted yet | 19:32 |
| lbragstad | I think that is the only way we don't incur downtime encrypting credentials | 19:33 |
| lbragstad | otherwise you'll have to take the whole cluster down, run the encryption migration, and stand the whole thing back up | 19:33 |
| samueldmq | lbragstad: nice, dolph's idea is to have it alwyas enabled | 19:33 |
| lbragstad | and for deployments that have millions of credentials, that's a lot of downtime | 19:34 |
| samueldmq | and I like it | 19:34 |
| samueldmq | lbragstad: ++, but that doesn't stop us from being able to always have it enabled | 19:35 |
| lbragstad | samueldmq yeah - i still need to work through that case | 19:35 |
| lbragstad | samueldmq the problem is that if you have 3 keystone nodes, two of them are running mitaka, and one of them is upgraded to newton | 19:36 |
| *** eandersson_ has quit IRC | 19:36 | |
| lbragstad | and you migrate your credentials to be encrypted | 19:36 |
| lbragstad | now the mitaka nodes are going to be exposes ciphertext via the api | 19:36 |
| lbragstad | along with the key_hash attribute | 19:37 |
| samueldmq | releasenote: "As of Newton, all new credentials are encrypted. Trying to create new credentials without properly setting up the key repository will result in a server exception. Old credentials should be encrypted using keystone manage, however they still can be retrieved from the API." | 19:37 |
| lbragstad | samueldmq where is that? | 19:37 |
| samueldmq | lbragstad: just here :) I am just summarizing the idea | 19:38 |
| lbragstad | oh | 19:38 |
| lbragstad | samueldmq yeah - i agree.. but I don't know how you would enforce that on the mitaka nodes without backporting something to make mitaka aware of the things we added in newton | 19:38 |
| samueldmq | why do we need to care about mitaka? | 19:39 |
| samueldmq | new things, do migrations | 19:39 |
| lbragstad | samueldmq if you're doing a rolling upgrade | 19:39 |
| lbragstad | unless we make deploying newton first a requirement of the upgrade | 19:40 |
| lbragstad | so you would be required to deploy newton to *all* nodes in the deployment before turning on credential encryption | 19:40 |
| lbragstad | and before migrating any plaintext credentials | 19:41 |
| samueldmq | lbragstad: commented on that patch | 19:42 |
| lbragstad | samueldmq which patch? | 19:42 |
| samueldmq | https://review.openstack.org/#/c/354495/ | 19:42 |
| patchbot | samueldmq: patch 354495 - keystone - Add conf to support credential encryption | 19:42 |
| samueldmq | lbragstad: sorry I can't see the issue with having encryption=true by default and rolling upgrades | 19:43 |
| lbragstad | samueldmq if you have encryption=true and you do a rolling upgrade you're going to have nodes that are mitaka and nodes that are newton deployed at the same time | 19:43 |
| lbragstad | right? | 19:43 |
| samueldmq | yes | 19:44 |
| lbragstad | samueldmq so what happens if you ask a mitaka node for a credential? | 19:44 |
| samueldmq | for an encrypted credential? | 19:44 |
| lbragstad | sure | 19:44 |
| lbragstad | any credential really | 19:44 |
| samueldmq | it fails ? mitaka doesn't understand encrypted credentials | 19:45 |
| -openstackstatus- NOTICE: The volume for logs.openstack.org filled up rather suddenly, causing a number of jobs to fail with a POST_FAILURE result and no logs; we're manually expiring some logs now to buy breathing room, but any changes which hit that in the past few minutes will need to be rechecked and/or approved again | 19:45 | |
| *** spzala has joined #openstack-keystone | 19:45 | |
| lbragstad | samueldmq should it though? | 19:45 |
| *** code-R has joined #openstack-keystone | 19:45 | |
| lbragstad | how would it fail? | 19:46 |
| lbragstad | it would get a credential by ID | 19:46 |
| lbragstad | and return the reference | 19:46 |
| lbragstad | which would contain ciphertext in the blob if the credential is already migrated | 19:46 |
| lbragstad | if not - it would contain an extra attribute exposed through the API ('key_hash') because mitaka wouldn't know to pop that out of the reference before returning to the user | 19:47 |
| samueldmq | lbragstad: if the credential was created in mitaka, mitaka can still retrieve, right? | 19:48 |
| samueldmq | lbragstad: the issue is newton credentials (encrypted) that can't be retrieved with mitaka code ? | 19:48 |
| *** code-R_ has joined #openstack-keystone | 19:49 | |
| lbragstad | samueldmq I don't think the problem is that it can't... i think the problem is that depending on how the rolling upgrade is done can expose sensitive data through the API | 19:49 |
| *** spzala has quit IRC | 19:49 | |
| *** spzala has joined #openstack-keystone | 19:50 | |
| lbragstad | if mitaka code attempts to retrieve a credential after it has been migrated - data will be exposed. | 19:50 |
| *** gyee has joined #openstack-keystone | 19:50 | |
| samueldmq | lbragstad: what data? | 19:50 |
| samueldmq | that wasn't exposed before.. | 19:50 |
| lbragstad | samueldmq the key_hash and the blob of the credential | 19:51 |
| lbragstad | the blob of the credential will be ciphertext | 19:51 |
| *** su_zhang has joined #openstack-keystone | 19:51 | |
| lbragstad | which isn't useful to the user | 19:51 |
| lbragstad | and the key_hash is a hash of they key that was used to encrypt the credential | 19:51 |
| samueldmq | lbragstad: does mitaka understand key_hash? | 19:51 |
| lbragstad | nope | 19:51 |
| lbragstad | that would be backporting a migration | 19:52 |
| samueldmq | lbragstad: if not, it will just expose the ciphertext | 19:52 |
| samueldmq | which is not an issue? | 19:52 |
| *** code-R has quit IRC | 19:52 | |
| lbragstad | samueldmq if i'm a user and i ask keystone for a credential, i expect some secret blob that I created to be returned | 19:52 |
| samueldmq | yes | 19:52 |
| lbragstad | when what actually gets returned is ciphertext | 19:53 |
| lbragstad | as a user, i can't do anything with ciphertext | 19:53 |
| samueldmq | yes, because your provider is in the middle of an upgrade and is encrypting things in the meantime | 19:53 |
| lbragstad | so if i have any automated tooling around credentials that relies on pulling the blob out of the credential response, that is broken | 19:53 |
| samueldmq | only do the old creds migration after upgrading all nodes then | 19:54 |
| samueldmq | only do the old creds migration after upgrading all nodes then | 19:54 |
| samueldmq | oops, sorry | 19:54 |
| dstanek | lbragstad: i love credential encryption! | 19:54 |
| samueldmq | 1. upgrade a node to mitaka, it still can read not-encrypted creds, but only creates encrypted creds | 19:55 |
| samueldmq | 2. an old node can still read old creds, but can't the new ones (encrypted) | 19:55 |
| lbragstad | upgrade to mitaka? | 19:55 |
| samueldmq | 3. after migrating all nodes, encrypt old credentials | 19:55 |
| lbragstad | or newton? | 19:55 |
| samueldmq | newton | 19:55 |
| *** su_zhang has quit IRC | 19:56 | |
| lbragstad | so - after step 1 you have one node running newton and two running mitaka, what happens if you hit a mitaka node asking for a credential you just created on a newton node? | 19:56 |
| samueldmq | only the case in 2. would have mitaka nodes returning the cyphertext, because it's trying to read credentials created in the middle of the upgrade process with an old node | 19:57 |
| lbragstad | samueldmq what if we did this | 19:57 |
| *** slberger has quit IRC | 19:57 | |
| lbragstad | 1.) upgrade each node to newton and make sure `[credential] encrypt`=False | 19:57 |
| samueldmq | yeah | 19:58 |
| samueldmq | that works | 19:58 |
| lbragstad | 2.) migrate all credentials using `keystone-manage credential_migrate` | 19:58 |
| lbragstad | er.. | 19:58 |
| lbragstad | no | 19:58 |
| samueldmq | but we can't address dolphm's suggestion to make it enabled by default | 19:58 |
| lbragstad | 2.) gracefully roll each node to enable encryption | 19:58 |
| lbragstad | 3.) forcibly migrate remaining credentials | 19:59 |
| samueldmq | lbragstad: what happens if... | 19:59 |
| lbragstad | samueldmq true - i don't know how to get around that without exposing sensitive data through the API | 19:59 |
| samueldmq | when enabling encryption in a node, a credential get created in that node (then it's encrypted) | 19:59 |
| lbragstad | or providing inconsistent user-experience | 19:59 |
| samueldmq | and another node (newton) without it enabled yet tried to read it ? gmm... it works | 20:00 |
| samueldmq | lbragstad: how long does a rolling upgrade take ? | 20:00 |
| lbragstad | depends on how many credentials you have | 20:00 |
| samueldmq | lbragstad: I know it depends on the deployment, but usually... | 20:00 |
| samueldmq | a couple of hours ? days ? | 20:00 |
| lbragstad | i have no idea | 20:00 |
| samueldmq | lbragstad: what happens if we add a new attribute to an entity | 20:01 |
| samueldmq | and in a rolling upgrade the old code ignores it ? but the new code doesn't ? | 20:02 |
| samueldmq | you would get incosistent responses depending on the node you hit, right? | 20:02 |
| lbragstad | samueldmq yes - i believe so | 20:02 |
| samueldmq | so I think this is something that has to be understood in a rolling upgrade | 20:03 |
| samueldmq | there are things being added/removed, the code won't break, but the server responses may vary depending on the version of the node it hits | 20:03 |
| samueldmq | because new things are returned/handled differently in new code | 20:04 |
| samueldmq | and this is just for the time the rolling upgrade takes | 20:04 |
| lbragstad | the thing that gets me is that the inconsistency returned is ciphertext | 20:04 |
| samueldmq | old data always work, new data makes sense in the new server version | 20:04 |
| samueldmq | so it's a UX issue now | 20:05 |
| lbragstad | would it be considered a backwards incompatible change? | 20:05 |
| samueldmq | we are not modifying anything in a backwards way | 20:06 |
| samueldmq | the "backward" data is still there as it was (old credentials) | 20:06 |
| lbragstad | samueldmq the api contract stats the blob returned is the blob you passed in | 20:06 |
| *** su_zhang has joined #openstack-keystone | 20:07 | |
| samueldmq | if you're using the same server version, yes | 20:07 |
| lbragstad | so - depending on which node i hit, i might get the blob i'm expecting or not | 20:07 |
| lbragstad | i might get something entirely different | 20:07 |
| openstackgerrit | Merged openstack/ldappool: Updated from global requirements https://review.openstack.org/322990 | 20:08 |
| *** slberger has joined #openstack-keystone | 20:09 | |
| samueldmq | lbragstad: I understand your point, it's fair, I am mulling it a bit more | 20:11 |
| samueldmq | looks like this is another level of rolling upgrades, it's not just about the database being able to be used by different versions | 20:12 |
| samueldmq | it's about keeping the API contracts during the upgrade | 20:13 |
| samueldmq | the API contract is kept before the upgrade | 20:13 |
| samueldmq | and after the upgrade | 20:13 |
| samueldmq | but it can't really be kept *during* the upgrade in that case | 20:13 |
| samueldmq | lbragstad: the solution would be to store the encrypted credential in a new column | 20:16 |
| samueldmq | and drop the old column in N+1 | 20:16 |
| samueldmq | newton code stores it in both, mitaka reads from the old | 20:16 |
| lbragstad | that might work | 20:17 |
| samueldmq | does that add a lot of complexity for implementing it ? | 20:17 |
| *** ayoung has joined #openstack-keystone | 20:18 | |
| *** ChanServ sets mode: +v ayoung | 20:18 | |
| lbragstad | so the newton code would store an encrypted version of the credential in a separate column and update the old column to have the new plaintext | 20:18 |
| samueldmq | lbragstad: ++ | 20:19 |
| lbragstad | I think this would absolutely require a contract phase to remove the old column though | 20:19 |
| lbragstad | otherwise we would still have plaintext secrets in the database | 20:19 |
| samueldmq | or just make it disabled by default | 20:20 |
| samueldmq | and change the default in +1 release | 20:20 |
| lbragstad | make what disabled? | 20:20 |
| samueldmq | encryption | 20:20 |
| openstackgerrit | Merged openstack/keystone: api-ref: Fix parameters attributes https://review.openstack.org/356211 | 20:21 |
| lbragstad | we could address dolphm's concern with rolling database upgrades | 20:21 |
| lbragstad | but that would have to land before the rest of the credential encryption work | 20:21 |
| lbragstad | then i could see not having an encrypt configuration option | 20:22 |
| samueldmq | another solution is: encryption is disabled, upgrades nodes to newton. after upgrading all of them, kesytone manage old creds and enable the config option | 20:22 |
| lbragstad | samueldmq yeah - that was my original upgrade path | 20:22 |
| samueldmq | in the N+1 we change the encryption=true to be the fdefault | 20:22 |
| samueldmq | ^ | 20:22 |
| lbragstad | dolphm thoughts? | 20:23 |
| lbragstad | dstanek thoughts? | 20:23 |
| samueldmq | exactly, keep that path, and we can go to a sane default (encryption=true) in N+1 | 20:23 |
| dstanek | lbragstad: i'll have to read up. been working on some things. | 20:23 |
| samueldmq | without breaking anything because we committed to N+1 upgrades and not N+2 | 20:23 |
| samueldmq | brb | 20:23 |
| mfisch | EmilienM: le ping | 20:26 |
| mfisch | sorry wrong room! | 20:26 |
| openstackgerrit | Steve Martinelli proposed openstack/python-keystoneclient: Add ec2 functional tests https://review.openstack.org/350245 | 20:33 |
| * stevemar glares at mfisch -_- | 20:34 | |
| *** rcernin has quit IRC | 20:40 | |
| stevemar | dolphm: got a minute for https://review.openstack.org/#/c/350704/2 ? | 20:41 |
| patchbot | stevemar: patch 350704 - keystone - Make all token provider behave the same with trusts | 20:41 |
| stevemar | its the beginning of the chain for fixing cache | 20:41 |
| dstanek | lbragstad: samueldmq: haven't read through it all yet, but had an interesting thought. make a new column `cipher_text` and do a rename type flow for the upgrade | 20:42 |
| lbragstad | dstanek I just started thinking about that | 20:42 |
| lbragstad | 1.) upgrade a single node to Newton | 20:42 |
| lbragstad | 2.) run db_sync --expand to create new 'key_hash' and 'encrypted_blob' columns | 20:43 |
| lbragstad | but if a user creates a credential using a newton node | 20:43 |
| lbragstad | and updates it using a mitaka node, | 20:43 |
| lbragstad | the trigger won't update the 'encrypted_blob' column | 20:44 |
| *** roxanaghe has quit IRC | 20:44 | |
| dstanek | lbragstad: there should be no reason when triggers can't do that | 20:45 |
| dstanek | the hard (impossible?) part is doing the encryption in a trigger. seems like it would have to be an incremental migration in Python | 20:45 |
| lbragstad | dstanek the mitaka node just passes the plaintext update to the database | 20:45 |
| lbragstad | dstanek right | 20:46 |
| lbragstad | dstanek but the hard part is making mitaka aware of what it needs to do for newton schemas | 20:46 |
| lbragstad | in order for Mitaka code to update properly - it would need to understand the new schema and how to encrypt (right?) | 20:46 |
| dstanek | lbragstad: or have the new code keep writing to the plaintext column | 20:47 |
| dstanek | but even if you do that new records from mitaka will not be encrypted | 20:47 |
| lbragstad | but what happens if someone updates a credential using the old code? | 20:47 |
| dstanek | so there is a short amount of time for some number of records to be out of sync | 20:47 |
| lbragstad | dstanek what if we require all nodes to be upgrades to newton before enabling `[credential] encrypt`? | 20:48 |
| lbragstad | upgraded* | 20:49 |
| lbragstad | once all nodes are on Newton, you can enable encryption and batch process credentials | 20:49 |
| dstanek | lbragstad: that would be ideal. that was the original plan, right? before we started talking about the ability to turn it off | 20:49 |
| lbragstad | dstanek yeah - but dolphm brought up a good point that we should really allow the storage of unencrypted credentials to be an option | 20:50 |
| stevemar | samueldmq: want to take a quick look at https://review.openstack.org/#/c/356485/2 ? | 20:50 |
| patchbot | stevemar: patch 356485 - keystone - [api] add relationship links to v3-ext | 20:50 |
| stevemar | i just verified all the entries, should be a quick one :) | 20:50 |
| dstanek | lbragstad: i thought he didn't want that ability? | 20:50 |
| lbragstad | dstanek i'm still thinking of this as a one way migration | 20:50 |
| lbragstad | dstanek see his first comment here - https://review.openstack.org/#/c/354495/6/keystone/conf/credential.py | 20:51 |
| patchbot | lbragstad: patch 354495 - keystone - Add conf to support credential encryption | 20:51 |
| dstanek | lbragstad: he is saying to have no way to turn it off | 20:52 |
| lbragstad | dstanek he's saying don't have a `[credential] encrypt` option | 20:52 |
| dstanek | which i agree with, but it makes the upgrade hard | 20:52 |
| lbragstad | dstanek right - it's a trade off | 20:52 |
| lbragstad | we can use the option as a way to make the migration easier | 20:52 |
| dstanek | i think right now it's probably ok since they are not used that much. afaict | 20:53 |
| lbragstad | otherwise we risk upgrading and having some amount of data out of sync during the upgrade | 20:53 |
| lbragstad | dstanek what's ok? | 20:53 |
| lbragstad | dstanek allowing the deployer to specify encrypt=False? | 20:53 |
| lbragstad | dstanek or allowing some amount of error in the upgrade? | 20:54 |
| dstanek | allowing for some errors during the upgrade | 20:54 |
| dstanek | we already don't have a guaranteed uptime yet | 20:54 |
| lbragstad | dstanek doesn't rolling upgrades guarantee that? | 20:56 |
| dstanek | we don't have a guarantee for M->N just yet. we've talked about doing best effort right now | 20:56 |
| lbragstad | dstanek so - you're saying we should remove the encrypt option? | 20:57 |
| *** raildo has quit IRC | 20:57 | |
| lbragstad | that would require all nodes to be upgraded to Newton *and* all credentials to be migrated before starting any of the nodes | 20:57 |
| lbragstad | so downtime would be dependent on how many credentials your storing in the backend. | 20:58 |
| *** edmondsw has joined #openstack-keystone | 20:58 | |
| dstanek | i'm ok with it it other people are. deployers have the option of leaving it up and having some inconsistent api responses. | 20:59 |
| dstanek | we can also mitigate that quite a bit this an incremental migration | 20:59 |
| dstanek | leave mitaka running, migrate, turn off mitaka, migrate again and start up newton - is one possible flow | 21:00 |
| lbragstad | ooo | 21:01 |
| lbragstad | that would be a tough one | 21:01 |
| lbragstad | because if a credential is created using a newton node | 21:02 |
| lbragstad | then upgraded using a mitaka node | 21:02 |
| *** edtubill has quit IRC | 21:02 | |
| lbragstad | the credential migration command would have to somehow make sure each credential is encrypted with the right key using the key_hash | 21:03 |
| lbragstad | and if not - it would have to re-encrypt to update the ciphertext and the key_hash of the credential | 21:03 |
| dstanek | in that scheme mitaka and newton are not running in parallel | 21:03 |
| lbragstad | we wouldn't be able to assume a populated key_hash value means the credential is encrypted | 21:04 |
| lbragstad | oh | 21:04 |
| lbragstad | ok | 21:04 |
| dstanek | you can also track that with 'updated_date' and an incremental migration can re-migrate anything what was updated after it started | 21:04 |
| lbragstad | the upgrade path should just disable credential create and update through policy | 21:05 |
| *** gordc has quit IRC | 21:08 | |
| dstanek | that's a good idea. a deployer can totally do that without any keystone support | 21:08 |
| lbragstad | so in that case you would: | 21:09 |
| lbragstad | 1.) disable credential create and update in policy | 21:09 |
| lbragstad | .... | 21:11 |
| lbragstad | actually - i'm not sure how that would work | 21:11 |
| *** edtubill has joined #openstack-keystone | 21:11 | |
| lbragstad | oh | 21:12 |
| lbragstad | 2.) roll nodes to newton code | 21:12 |
| *** fifieldt has quit IRC | 21:13 | |
| lbragstad | newton code can attempt to decrypt values and if it fails, return the original value | 21:13 |
| lbragstad | .. actually that doesn't work either | 21:13 |
| dstanek | 1 - upgrade a single node and expand the schema (included a new cipher_text column) | 21:15 |
| dstanek | 2 - roll out policy to disable create/update for credentials | 21:15 |
| dstanek | 3 - migrate | 21:15 |
| *** michauds has quit IRC | 21:15 | |
| dstanek | 4 - rolling upgrade to newton | 21:15 |
| *** gyee_ has joined #openstack-keystone | 21:15 | |
| dstanek | 5 - revert policy change | 21:15 |
| dstanek | 6 - collapse (will remove the old blob column) | 21:16 |
| lbragstad | dstanek so this would be using the rolling upgrade idea but without triggers | 21:16 |
| dstanek | yeah, the policy changes would limit the need for triggers | 21:17 |
| *** spzala has quit IRC | 21:17 | |
| dstanek | dolphm: what do you think? if it's good it might be worth calling out in the spec | 21:17 |
| *** spzala has joined #openstack-keystone | 21:18 | |
| lbragstad | dstanek which spec? | 21:18 |
| lbragstad | the credential encryption one? | 21:18 |
| dolphm | reading back... | 21:18 |
| lbragstad | dolphm you're here! | 21:18 |
| dolphm | sorry wrong channel | 21:19 |
| *** gyee has quit IRC | 21:19 | |
| *** pnavarro has quit IRC | 21:19 | |
| dstanek | lbragstad: no the rolling upgrade one | 21:20 |
| dolphm | lbragstad: with rolling upgrades, you could do this more smoothly by introducing a new encrypted blob column | 21:20 |
| dstanek | i'm sure there will be other cases where data can't up transformed in triggers | 21:20 |
| dstanek | dolphm: that was my step 1 :-) | 21:20 |
| lbragstad | dstanek ++ | 21:20 |
| dolphm | lbragstad: old code reads from plaintext column, new code reads from ciphertext column, data migration does all the hard work | 21:20 |
| dolphm | dstanek: oh, i missed that bit, then ++! | 21:21 |
| lbragstad | dolphm but what happens if a credential is updated? | 21:21 |
| lbragstad | on a mitaka node? | 21:21 |
| dolphm | dstanek: why the policy bit, though? | 21:21 |
| dolphm | lbragstad: oh, triggers can't do the encryption | 21:21 |
| * samueldmq 's back | 21:21 | |
| samueldmq | lbragstad: so we keep that idea of adding a new column? | 21:21 |
| samueldmq | what is this policy thing you're talking about | 21:22 |
| lbragstad | samueldmq still working through the issues | 21:22 |
| dstanek | i think that's close enought to 0 downtime since a low use feature would be readonly for a short period of time | 21:22 |
| * samueldmq is not really up to date with the current proposal of rolling upgrades | 21:22 | |
| *** spzala has quit IRC | 21:22 | |
| lbragstad | we would require that migrating to newton means making credentials read only | 21:22 |
| dolphm | samueldmq: https://gist.github.com/dolph/72dae9391ec4e13444498f977bc92ad9 | 21:22 |
| dstanek | lbragstad: we'd probably also need a 'updated_date' column that gets created in --expand and remove in --contract | 21:24 |
| samueldmq | dolphm: nice | 21:24 |
| dstanek | that way you can deal with incremental migration | 21:24 |
| *** fifieldt has joined #openstack-keystone | 21:24 | |
| samueldmq | dolphm: the triggers remove the need of the code updating both columns | 21:24 |
| samueldmq | since the db deals with that itself | 21:24 |
| dolphm | samueldmq: correct | 21:24 |
| lbragstad | but in this cases it's a little strange since triggers would have to encrypt/decrypt | 21:25 |
| stevemar | see you all in a few hours! | 21:25 |
| dolphm | lbragstad: which they can't | 21:25 |
| stevemar | well, whoever is still online :) | 21:25 |
| dolphm | stevemar: the trigger could reject the write, though | 21:25 |
| lbragstad | dolphm right - that would be insane | 21:25 |
| dolphm | lbragstad: err ^ | 21:25 |
| stevemar | samueldmq: ^ | 21:25 |
| dolphm | dstanek: ^ | 21:26 |
| stevemar | he meant you | 21:26 |
| dstanek | stevemar: you don't get a break | 21:26 |
| lbragstad | dstanek ^ | 21:26 |
| lbragstad | lol | 21:26 |
| samueldmq | dolphm: ^ | 21:26 |
| stevemar | dstanek: :( | 21:26 |
| stevemar | :'( | 21:26 |
| dstanek | dolphm: if you rejected the write in the trigger you'd get a 500 error right? | 21:26 |
| dolphm | dstanek: how about instead of a policy change, we setup triggers to abort writes to the old column? | 21:26 |
| dolphm | dstanek: yes | 21:26 |
| lbragstad | hmm | 21:27 |
| lbragstad | so if a credential is created on a Newton node and updated on a Mitaka one, what happens? | 21:27 |
| dstanek | i wish it could be a 503 | 21:27 |
| dolphm | dstanek: yeah | 21:27 |
| dolphm | lbragstad: i don't know if we could support that | 21:27 |
| dolphm | lbragstad: leave the plaintext column blank? | 21:28 |
| dolphm | although, the blob is probably required | 21:28 |
| lbragstad | it is | 21:28 |
| dstanek | i'd be OK with either approach | 21:28 |
| *** vivek has joined #openstack-keystone | 21:28 | |
| dstanek | lbragstad: we can't create/update during the migration or we'd be giving out bad data | 21:28 |
| *** vivek is now known as Guest81529 | 21:29 | |
| lbragstad | ah - ok | 21:29 |
| lbragstad | i think i getit | 21:29 |
| Guest81529 | hi, I am trying to run devstack and using RestAPIs of Keystone, I am continuously facing [cors] issues | 21:29 |
| lbragstad | so the expand phase would create a trigger that wouldn't allow writes to the old column | 21:29 |
| Guest81529 | Anything that I am missing. I edited the keystone.conf for cors | 21:30 |
| dolphm | lbragstad: in mysql, for example, you could raise a custom signal in the trigger http://dev.mysql.com/doc/refman/5.5/en/signal.html | 21:30 |
| dolphm | lbragstad: same sort of thing in sqlite afaik https://www.sqlite.org/syntax/raise-function.html | 21:30 |
| lbragstad | so the migration would look like | 21:30 |
| lbragstad | 1.) take one node out of the mesh and update it to newton | 21:31 |
| dolphm | lbragstad: you could go so far as to disable any creates or updates during the migration process, in either direction | 21:31 |
| dstanek | ok, going to bail for a little bit to get dinner - i'll catch up when i get back | 21:31 |
| lbragstad | 2.) run keystone-manage db_sync --expand | 21:31 |
| dolphm | (which lays down triggers to reject all writes) | 21:31 |
| dolphm | (except deletes) | 21:31 |
| lbragstad | 3.) migrate all credentials/ | 21:31 |
| dolphm | (in --migrate, which would require the newton node to have a populated credentials repo) | 21:32 |
| lbragstad | so that would be the equivalent to my `keystone-manage credential_migrate` command | 21:32 |
| dolphm | lbragstad: yeah, i guess you wouldn't need that anymore? | 21:33 |
| lbragstad | dolphm I don't think so | 21:33 |
| lbragstad | 4.) rolling upgrade the rest of the keystone nodes? | 21:33 |
| dolphm | ++ | 21:33 |
| dolphm | they'll be able to read all credentials as they come online | 21:34 |
| lbragstad | 5.) keystone-manage db_sync --contract (removes triggers and old unencrypted column) | 21:34 |
| dolphm | and old nodes will be able to read all credentials while they're still around | 21:34 |
| dolphm | lbragstad: ++ | 21:34 |
| lbragstad | ! | 21:34 |
| lbragstad | my brain hurts! | 21:34 |
| dolphm | rderose: henrynash: interesting edge case for trigger-based migrations ^^ when triggers can't possibly know what to write based on the database alone | 21:34 |
| lbragstad | ok - so the implementation of credential encryption will be dependent on rolling upgrades with triggers | 21:35 |
| samueldmq | lbragstad: so if we can't create/update during the upgrade | 21:35 |
| dstanek | lbragstad: dolphm: what don't you need anymore? the credential_migrate command? | 21:35 |
| *** roxanaghe has joined #openstack-keystone | 21:35 | |
| samueldmq | lbragstad: so it'd be fine to set ecryption=true as the default | 21:35 |
| dolphm | dstanek: yes, do we need it? | 21:35 |
| dstanek | samueldmq: or like dolphm says not even have the option | 21:35 |
| dstanek | dolphm: yes, to rotate keys | 21:35 |
| samueldmq | dstanek: exactly | 21:35 |
| lbragstad | we also wouldn't need the encrypt config option | 21:35 |
| dstanek | unless that is a different command | 21:35 |
| rderose | dolphm: yep, especially where the default comes from python | 21:36 |
| dolphm | Guest81529: cors isn't really part of keystone itself (it's an oslo project) | 21:36 |
| dolphm | Guest81529: http://docs.openstack.org/developer/oslo.middleware/cors.html | 21:36 |
| *** marekd2 has joined #openstack-keystone | 21:36 | |
| dolphm | Guest81529: but you'll need to describe the error you're seeing, your cors configuration, etc, if you expect anyone to be able to help | 21:36 |
| dstanek | cors stuff shouldn't come into play unless there is a browser involved, right? | 21:36 |
| Guest81529 | from FireFox with RestAPI plugin everything is working fine | 21:37 |
| dstanek | dolphm: lbragstad: i do think you still need that command - but now i'm really leaving to get dinner | 21:37 |
| dolphm | Guest81529: sure, CORS is designed to protect against cross-origin requests ... if you're hitting keystone directly, CORS won't stop you | 21:37 |
| Guest81529 | but when I wrote sample js app which is running on a simple node server...the response says "Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource" | 21:38 |
| lbragstad | dstanek ok - i gotta run too | 21:38 |
| lbragstad | dstanek dolphm samueldmq I'm going to noodle on this for an hour or two and then I'll be back | 21:38 |
| rderose | dolphm henrynash: like if the default is based on config file and business logic in source | 21:38 |
| dolphm | Guest81529: CORS headers need to be provided by both your javascript server and the keystone endpoint, IIRC | 21:39 |
| rderose | dolphm henrynash: we probably could find ways to solve this in our current model | 21:39 |
| rderose | dolphm henrynash: perhaps another POC, just to be sure | 21:39 |
| *** pauloewerton has quit IRC | 21:39 | |
| dolphm | rderose: we can always lay down config options into triggers; in this case it's encryption that the db cannot handle by itself | 21:39 |
| Guest81529 | I did this in keystone.conf on my devstack [cors] allowed_origin = * allow_methods = GET | 21:39 |
| dolphm | rderose: nevermind the keys to do the encryption | 21:39 |
| Guest81529 | infact allo_methods = * | 21:40 |
| lbragstad | rderose taking an unencrypted plaintext field and encrypting it into something keystone can understand | 21:40 |
| lbragstad | (i.e. using fernet) | 21:40 |
| * lbragstad steps away | 21:40 | |
| Guest81529 | and I am putting headers: {"Access-Control-Allow-Origin":"*"} in header when making request | 21:41 |
| *** marekd2 has quit IRC | 21:41 | |
| samueldmq | lbragstad: kk | 21:41 |
| rderose | dolphm lbragstad: that is an interesting edge case, hmm... | 21:42 |
| dolphm | Guest81529: are you running the cors middleware in your keystone pipeline? is your client seeing the CORS headers in responses? | 21:42 |
| Guest81529 | I am not getting response from keystone server | 21:43 |
| Guest81529 | I see error log in keystone that IP not permitted | 21:43 |
| Guest81529 | I only changed the conf file...do I need to do middleware chnages too with oslo? | 21:44 |
| dolphm | Guest81529: yes, http://docs.openstack.org/developer/oslo.middleware/cors.html | 21:44 |
| Guest81529 | ok..thanks let me try that and I will get back on this...I just followed http://docs.openstack.org/admin-guide/cross-project-cors.html | 21:45 |
| *** su_zhang has quit IRC | 21:46 | |
| rderose | dolphm lbragstad: we could temporarily add an unencrypted column and let the new code deal with both; remove the new column in the next release. Not ideal, but... | 21:46 |
| dolphm | Guest81529: sounds like you missed this step? http://docs.openstack.org/admin-guide/cross-project-cors.html | 21:47 |
| *** su_zhang has joined #openstack-keystone | 21:47 | |
| dolphm | Guest81529: well, i guess i can't link to a direct section | 21:47 |
| rderose | dolphm lbragstad: we may at times need these kind of workarounds | 21:47 |
| dolphm | Guest81529: "Enabling CORS with PasteDeploy" | 21:47 |
| dolphm | rderose: liiike today | 21:47 |
| rderose | :) | 21:48 |
| rderose | dolphm lbragstad: that being said, I willing to bet that 99% of our typical db changes will be covered by the new strategy | 21:49 |
| *** su_zhang has quit IRC | 21:49 | |
| Guest81529 | @dolphm 'Enabling CORS with configuration' i used the wild card configuration | 21:49 |
| dolphm | rderose: right | 21:49 |
| *** spedione is now known as spedione|AWAY | 21:49 | |
| dolphm | Guest81529: okay, well you should also ensure cors is in your paste pipeline | 21:50 |
| *** su_zhang has joined #openstack-keystone | 21:51 | |
| *** edtubill has quit IRC | 21:51 | |
| *** Gorian|work has quit IRC | 21:52 | |
| *** su_zhang has quit IRC | 21:53 | |
| *** adriant has joined #openstack-keystone | 21:54 | |
| bknudson | henrynash: making some progress by setting breakpoints in the tests -- check this out: http://paste.openstack.org/show/559103/ | 21:58 |
| *** slberger has left #openstack-keystone | 21:59 | |
| *** ezpz has quit IRC | 21:59 | |
| bknudson | for some reason when running the test again with diff db, the call to self.initialize_repo(repo_name=EXPAND_REPO) winds up running contract_repo migrations. | 22:00 |
| *** adu has joined #openstack-keystone | 22:02 | |
| *** su_zhang has joined #openstack-keystone | 22:02 | |
| *** ametts has quit IRC | 22:04 | |
| *** su_zhang has quit IRC | 22:05 | |
| *** sdake_ has quit IRC | 22:06 | |
| *** adu has quit IRC | 22:07 | |
| *** ravelar has quit IRC | 22:07 | |
| *** adu has joined #openstack-keystone | 22:07 | |
| *** gyee_ has quit IRC | 22:09 | |
| *** ayoung has quit IRC | 22:14 | |
| *** gagehugo has quit IRC | 22:17 | |
| *** spzala has joined #openstack-keystone | 22:17 | |
| *** spzala has quit IRC | 22:17 | |
| *** spzala has joined #openstack-keystone | 22:17 | |
| *** sdake has joined #openstack-keystone | 22:21 | |
| *** Gorian|work has joined #openstack-keystone | 22:22 | |
| *** hockeynut has quit IRC | 22:22 | |
| *** amoralej|off has quit IRC | 22:24 | |
| *** jaugustine has quit IRC | 22:24 | |
| *** gagehugo_ has quit IRC | 22:24 | |
| *** amoralej has joined #openstack-keystone | 22:30 | |
| dolphm | bknudson: is find_migrate_repo() returning the wrong path or something? | 22:32 |
| *** gyee_ has joined #openstack-keystone | 22:35 | |
| *** sdake has quit IRC | 22:38 | |
| *** adu_ has joined #openstack-keystone | 22:40 | |
| *** adu has quit IRC | 22:41 | |
| *** adu_ is now known as adu | 22:41 | |
| bknudson | dolphm: find_migrate_repo is returning the expected path | 22:48 |
| *** BjoernT has quit IRC | 22:48 | |
| *** gagehugo has joined #openstack-keystone | 22:49 | |
| *** jaugustine has joined #openstack-keystone | 22:50 | |
| *** esp has quit IRC | 22:51 | |
| *** spzala has quit IRC | 22:58 | |
| *** spzala has joined #openstack-keystone | 22:58 | |
| bknudson | http://paste.openstack.org/show/559106/ | 23:00 |
| dolphm | bknudson: is it operating on the correct database? | 23:01 |
| dolphm | bknudson: or is it running against sqlite or something twice? | 23:01 |
| bknudson | this is the engine: Engine(postgresql://openstack_citest:***@localhost/ivqwxvlodp) | 23:02 |
| *** erhudy has quit IRC | 23:02 | |
| bknudson | I run the tests like this: | 23:02 |
| bknudson | .tox/py27/bin/python -m unittest keystone.tests.unit.test_sql_upgrade.SqlContractSchemaUpgradeTests.test_001_created_at_made_non_nullable keystone.tests.unit.test_sql_upgrade.PostgreSQLOpportunisticContractSchemaUpgradeTestCase.test_001_created_at_made_non_nullable | 23:02 |
| bknudson | so first it's sqlite and 2nd is postgresql | 23:03 |
| *** spzala has quit IRC | 23:03 | |
| bknudson | if I swap the order, still fails (except with sqlite this time) | 23:03 |
| bknudson | leaning towards some cached global in sqlalchemy-migrate | 23:04 |
| *** asettle has joined #openstack-keystone | 23:05 | |
| *** asettle has quit IRC | 23:09 | |
| bknudson | http://paste.openstack.org/show/559117/ -- first time through, after self.initialize_repo, the python module is as expected. | 23:11 |
| bknudson | but the next time through, it calls self.initialize_repo(repo_name=EXPAND_REPO) , but the schema has contract_repo | 23:11 |
| dolphm | bknudson: is it ever realistic for us to test against multiple databases in the gate? | 23:12 |
| bknudson | we do now. | 23:12 |
| dolphm | ooh | 23:12 |
| bknudson | and it works | 23:12 |
| dolphm | bknudson: so we can drop sqlite? :D | 23:12 |
| bknudson | well, we only test db migrations with other dbs. | 23:13 |
| bknudson | of course, we should fix our unit tests to not require a db. | 23:13 |
| bknudson | or at least minimize db testing | 23:14 |
| dolphm | bknudson: well, our other tests don't work with real db's (which dstanek is working on) | 23:14 |
| bknudson | shoot. I know I had it working at one point in order to test db2. | 23:15 |
| dolphm | bknudson: yeah - but the "root" domain thing is what's broken last i spoke to dstanek about it | 23:16 |
| dolphm | bknudson: the tests don't bother to actually create the "<<< root magic domain entry >>>!>" thing | 23:17 |
| dolphm | bknudson: so mysql & postgres just balk with foreign key errors | 23:17 |
| bknudson | that's pretty new | 23:17 |
| dolphm | bknudson: still time to remove it then | 23:17 |
| bknudson | doesn't root magic domain entry get created during migrations? | 23:18 |
| *** ayoung has joined #openstack-keystone | 23:18 | |
| *** ChanServ sets mode: +v ayoung | 23:18 | |
| bknudson | I'm wondering if this could be worked around by not having migration repos with names the same. | 23:29 |
| *** Gorian|work has quit IRC | 23:39 | |
| openstackgerrit | Merged openstack/python-keystoneclient: Improve docs for v3 auth https://review.openstack.org/355980 | 23:41 |
| bknudson | this is crazy. | 23:46 |
| bknudson | http://paste.openstack.org/show/560282/ | 23:46 |
| *** adu has quit IRC | 23:50 | |
| *** adu has joined #openstack-keystone | 23:54 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!