feefifo | it seems like the domain token doesn't come with a service catalog | 00:01 |
---|---|---|
feefifo | is there a way to configure keystone to include it? | 00:02 |
*** kbaegis has joined #openstack-keystone | 00:03 | |
*** edmondsw has joined #openstack-keystone | 00:08 | |
*** edmondsw has quit IRC | 00:12 | |
*** edmondsw has joined #openstack-keystone | 00:15 | |
feefifo | never mind | 00:16 |
feefifo | i think the issue i ran into is the implied roles doesn't map to domains | 00:16 |
*** edmondsw_ has joined #openstack-keystone | 00:17 | |
*** edmondsw has quit IRC | 00:19 | |
*** edmondsw_ has quit IRC | 00:21 | |
*** thorst_afk has joined #openstack-keystone | 00:22 | |
*** thorst_afk has quit IRC | 00:29 | |
*** thorst_afk has joined #openstack-keystone | 00:30 | |
*** thorst_afk has quit IRC | 00:34 | |
*** edmondsw has joined #openstack-keystone | 00:40 | |
*** lbragstad has joined #openstack-keystone | 00:44 | |
*** ChanServ sets mode: +o lbragstad | 00:44 | |
*** edmondsw has quit IRC | 00:44 | |
*** zhurong has joined #openstack-keystone | 00:45 | |
*** Shunli has joined #openstack-keystone | 00:46 | |
feefifo | hi lbragstad | 00:47 |
lbragstad | feefifo: o/ | 00:48 |
feefifo | do you have a few mins for some q's? | 00:48 |
lbragstad | feefifo: sure | 00:48 |
feefifo | thanks | 00:48 |
feefifo | i've been running some tests on the cloud admin policy file | 00:48 |
feefifo | and i've been finding some issues with some of the rules | 00:49 |
feefifo | eg. https://github.com/openstack/keystone/blob/master/etc/policy.v3cloudsample.json#L40 | 00:49 |
feefifo | this rule seems to suggest that listing projects is granted to cloud admins or domain admins | 00:49 |
feefifo | but keystone doesn't seem to accept a domain token for the projects endpoint | 00:50 |
lbragstad | feefifo: there are some issues with that policy file | 00:50 |
lbragstad | the test coverage isn't as good as the testing of the default policy file, unfortunately | 00:50 |
feefifo | ah i see | 00:50 |
lbragstad | feefifo: also | 00:51 |
lbragstad | while it's possible to get domain scoped tokens, they aren't really accepted anywhere, yet | 00:51 |
*** basilAB has left #openstack-keystone | 00:51 | |
lbragstad | we have quite a bit of work to do across the various services to get them honored consistently in openstack | 00:51 |
feefifo | i see | 00:51 |
feefifo | ok that's really good to know | 00:52 |
lbragstad | certainly something we're hoping to close the gap on | 00:52 |
feefifo | so what purpose does the domain token serve today? | 00:53 |
lbragstad | feefifo: not a whole lot because it hasn't really be implemented anywher e | 00:54 |
lbragstad | ideally - a service would recognize a domain scoped token and perform the API according to the domain scoped instead of a project | 00:55 |
lbragstad | domain scope* | 00:55 |
lbragstad | a good example would be list instance | 00:55 |
lbragstad | if you pass a project scoped token to nova and ask for a list of instances, you expect to see all the instances owned by that project, right? | 00:55 |
feefifo | yes | 00:55 |
feefifo | it would be great to have that for domains too | 00:56 |
lbragstad | right | 00:56 |
lbragstad | but the list consists of all instances owned by all projects within that specific domain | 00:56 |
feefifo | right | 00:56 |
lbragstad | yeah - it'd be awesome to have that | 00:57 |
feefifo | definitely agree | 00:57 |
feefifo | i'm guessing there's also a lot of work to support domain tokens in keystone? | 00:57 |
lbragstad | but - we have to do quite a bit of work with other projects to consistently make those changes | 00:57 |
feefifo | like listing projects in a domain | 00:57 |
lbragstad | feefifo: i'm sure there are still some gaps in the keystone api | 00:57 |
lbragstad | with respect to domain scoped tokens | 00:57 |
lbragstad | i don't believe domain scoped tokens work that way in keystone today | 00:58 |
feefifo | i see | 00:58 |
feefifo | wow, i have a feeling that it will be a few releases before we see full support across all components | 00:58 |
lbragstad | feefifo: likely | 00:59 |
lbragstad | feefifo: there are a bunch of things we have to work on from a policy perspective | 00:59 |
lbragstad | policy/RBAC | 00:59 |
feefifo | i see | 00:59 |
lbragstad | once we have a few of the bigger things fixed and integrated into other projects, it will likely pave the way to do stuff better with domain-scoped tokens | 01:00 |
feefifo | what are some of the bigger things you guys are working on? | 01:00 |
lbragstad | well - we currently have some issues with admin-ness in openstack | 01:01 |
lbragstad | are we're currently working on addressing that | 01:01 |
lbragstad | we're also moving all policy default into code and documenting them (much like how we handle configuration options) | 01:01 |
feefifo | i see | 01:01 |
*** thorst_afk has joined #openstack-keystone | 01:02 | |
lbragstad | we've also talked about associating different levels of scope to various operations in openstack | 01:02 |
lbragstad | which will play a big part in getting traction with domain scoped tokens i think | 01:02 |
*** jamesbenson has joined #openstack-keystone | 01:02 | |
feefifo | yes that seems crucial for scope | 01:03 |
*** thorst_afk has quit IRC | 01:03 | |
lbragstad | but once all policy in code and documented (which is community goal for all applicable projects this release) we should be in a better place to make that change | 01:04 |
lbragstad | (e.g. listing hypervisors in nova requires global scope, but listing instances only requires project or domain scope) | 01:04 |
feefifo | got it | 01:05 |
feefifo | i'd be interested in helping out on some of that | 01:05 |
lbragstad | feefifo: awesome :) | 01:06 |
feefifo | is there a blueprint or ticket somewhere? | 01:06 |
lbragstad | yeah - let me grab you some link s | 01:06 |
feefifo | for tracking the overall progress | 01:06 |
feefifo | cool thanks! | 01:06 |
lbragstad | feefifo: so - this is the infamous bug report https://bugs.launchpad.net/keystone/+bug/968696 | 01:06 |
openstack | Launchpad bug 968696 in OpenStack Identity (keystone) ""admin"-ness not properly scoped" [High,In progress] - Assigned to Lance Bragstad (lbragstad) | 01:06 |
*** jamesbenson has quit IRC | 01:07 | |
lbragstad | that essentially details the admin-ness issue | 01:07 |
feefifo | Ok | 01:07 |
lbragstad | in an effort to help lay things out and make it easy for others to get up-to-speed with the problem | 01:08 |
lbragstad | i wrote a couple other documents | 01:08 |
lbragstad | https://review.openstack.org/#/c/460344/ and https://review.openstack.org/#/c/462733/11 | 01:08 |
feefifo | oh perfect, thanks! | 01:08 |
lbragstad | the first one is a pretty general document with a high level view | 01:08 |
lbragstad | the second just elaborates on one way we can improve security by fixing some policy issues | 01:09 |
lbragstad | then there is https://review.openstack.org/#/c/464763/5 | 01:09 |
lbragstad | if you really want to dig into context and more discussion, there is a thread on the mailing list that describes a couple approaches | 01:10 |
lbragstad | http://lists.openstack.org/pipermail/openstack-dev/2017-May/117419.html | 01:10 |
lbragstad | but there is a bunch of good information in ^ | 01:10 |
feefifo | hmm do global roles overlap with inherited roles? | 01:11 |
lbragstad | feefifo: umm - not really | 01:11 |
feefifo | ok i'll read deeper into the doc | 01:11 |
lbragstad | inherited roles are meant to be applied in project hierarchies | 01:11 |
lbragstad | if project B is a child of project A and I have a role on project A then it gets inherited to project B | 01:12 |
lbragstad | (is the basic gist of it) | 01:12 |
lbragstad | feefifo: right now, in openstack, when you make a role assignment you have to supply an actor (e.g. user or group) and a target, right? | 01:13 |
feefifo | yes | 01:14 |
feefifo | either a project or domain | 01:14 |
lbragstad | the target always has to be a project or a domain | 01:14 |
lbragstad | yep | 01:14 |
lbragstad | global roles makes it possible to assign someone a role globally | 01:14 |
lbragstad | introducing another level of scope, if you will | 01:14 |
feefifo | ah okay | 01:14 |
lbragstad | (that implementation is here https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bp/global-roles _ | 01:15 |
feefifo | that makes sense | 01:15 |
lbragstad | ^ i'm working on getting that in shape for the PTG | 01:15 |
feefifo | so if i was an admin globally, that doesn't mean i'm admin of all domains and projects | 01:15 |
feefifo | unless the inherit extension was modified to support that i'm guessing | 01:15 |
lbragstad | feefifo: it could | 01:15 |
lbragstad | if you have the admin role globally - you could be considered the operator or global administrator of the cloud | 01:16 |
feefifo | i see | 01:16 |
lbragstad | but - if you had admin on a project, you could create sub projects under your project and stuff like that (for example) | 01:16 |
lbragstad | or if you had admin on a domain | 01:16 |
lbragstad | you should be able to setup projects in your domain, create users in your domain, assign them roles on the right projects, etc... | 01:17 |
feefifo | right | 01:17 |
lbragstad | right now - we're missing the global scope bit | 01:17 |
lbragstad | (and the bug report i linked above details that)_ | 01:17 |
feefifo | i see, ok i'll take a closer look | 01:18 |
feefifo | thanks a lot for your time @lbragstad | 01:19 |
feefifo | really appreciate it! | 01:20 |
feefifo | hope i can help out in the future | 01:20 |
lbragstad | feefifo: anytime - i'm stoked you're interested in helping | 01:20 |
lbragstad | feefifo: do you have any other questions? | 01:20 |
feefifo | not at the moment, i think i'm good | 01:20 |
lbragstad | feefifo: good deal - are you going to be at the PTG? | 01:20 |
feefifo | not sure yet, there's a chance i might go to the summit | 01:21 |
*** jmlowe has quit IRC | 01:21 | |
feefifo | if i do i'll definitely be at the keystone meets | 01:21 |
*** jmlowe has joined #openstack-keystone | 01:21 | |
lbragstad | ok - we are planning a bunch of policy discussion during the PTG in Denver | 01:21 |
lbragstad | https://etherpad.openstack.org/p/keystone-queens-ptg is our schedule | 01:21 |
lbragstad | in case that helps you get a lay of the land as far as what we're going to be discussing as a larger group | 01:22 |
feefifo | cool thanks, this is good to have | 01:22 |
lbragstad | don't hesitate to ping if you have more questions | 01:22 |
feefifo | will do, thanks again :) | 01:23 |
lbragstad | anytime | 01:23 |
feefifo | have a good day / night! | 01:23 |
lbragstad | you too! | 01:23 |
*** jmlowe has quit IRC | 01:24 | |
*** jmlowe has joined #openstack-keystone | 01:25 | |
*** edmondsw has joined #openstack-keystone | 01:29 | |
*** edmondsw has quit IRC | 01:33 | |
*** itlinux has joined #openstack-keystone | 01:34 | |
*** aselius has quit IRC | 01:34 | |
*** otleimat has quit IRC | 01:37 | |
*** feefifo has quit IRC | 01:37 | |
*** edmondsw has joined #openstack-keystone | 01:42 | |
*** edmondsw has quit IRC | 01:46 | |
*** kbaegis has quit IRC | 01:51 | |
*** kbaegis has joined #openstack-keystone | 01:51 | |
*** edmondsw has joined #openstack-keystone | 01:56 | |
*** kukacz has quit IRC | 02:00 | |
*** edmondsw has quit IRC | 02:00 | |
*** kukacz has joined #openstack-keystone | 02:01 | |
*** mjax has quit IRC | 02:03 | |
*** mjax has joined #openstack-keystone | 02:04 | |
*** thorst_afk has joined #openstack-keystone | 02:04 | |
*** mjax has quit IRC | 02:06 | |
*** mjax has joined #openstack-keystone | 02:06 | |
*** lbragstad has quit IRC | 02:07 | |
*** mjax has quit IRC | 02:08 | |
*** thorst_afk has quit IRC | 02:09 | |
*** edmondsw has joined #openstack-keystone | 02:20 | |
*** edmondsw has quit IRC | 02:23 | |
*** edmondsw has joined #openstack-keystone | 02:24 | |
*** edmondsw has quit IRC | 02:28 | |
*** edmondsw has joined #openstack-keystone | 02:31 | |
*** edmondsw has quit IRC | 02:35 | |
openstackgerrit | Merged openstack/keystone master: Copy specific distro pages for install guide https://review.openstack.org/498605 | 02:39 |
*** sapd_ has joined #openstack-keystone | 02:45 | |
*** mjax has joined #openstack-keystone | 03:03 | |
*** mjax has quit IRC | 03:05 | |
*** thorst_afk has joined #openstack-keystone | 03:05 | |
*** thorst_afk has quit IRC | 03:10 | |
*** chlong has quit IRC | 03:21 | |
*** kbaegis has quit IRC | 03:22 | |
*** kbaegis has joined #openstack-keystone | 03:22 | |
*** kbaegis has quit IRC | 03:23 | |
*** jmlowe has quit IRC | 03:24 | |
*** jmlowe has joined #openstack-keystone | 03:24 | |
*** mjax has joined #openstack-keystone | 03:33 | |
*** mjax has quit IRC | 03:35 | |
*** lbragstad has joined #openstack-keystone | 03:37 | |
*** ChanServ sets mode: +o lbragstad | 03:37 | |
*** dave-mccowan has quit IRC | 03:40 | |
*** links has joined #openstack-keystone | 03:40 | |
*** gyee has quit IRC | 03:56 | |
*** abhi89 has joined #openstack-keystone | 03:57 | |
*** thorst_afk has joined #openstack-keystone | 04:06 | |
*** lbragstad has quit IRC | 04:09 | |
*** thorst_afk has quit IRC | 04:10 | |
*** edmondsw has joined #openstack-keystone | 04:32 | |
*** edmondsw has quit IRC | 04:36 | |
*** jmlowe has quit IRC | 04:47 | |
*** aojea has joined #openstack-keystone | 04:52 | |
*** zsli_ has joined #openstack-keystone | 04:56 | |
*** aojea has quit IRC | 04:57 | |
*** Shunli has quit IRC | 04:58 | |
*** jamesbenson has joined #openstack-keystone | 05:00 | |
*** jamesbenson has quit IRC | 05:04 | |
*** thorst_afk has joined #openstack-keystone | 05:07 | |
*** thorst_afk has quit IRC | 05:11 | |
*** abhi89 has quit IRC | 05:13 | |
*** abhi89 has joined #openstack-keystone | 05:13 | |
*** zsli_ has quit IRC | 05:27 | |
*** aojea has joined #openstack-keystone | 05:56 | |
*** cfriesen_ has quit IRC | 05:57 | |
openstackgerrit | Tin Lam proposed openstack/keystone master: Add database migration for project tags https://review.openstack.org/484456 | 05:58 |
*** aojea has quit IRC | 06:04 | |
*** thorst_afk has joined #openstack-keystone | 06:07 | |
*** thorst_afk has quit IRC | 06:12 | |
*** sapd_ has quit IRC | 06:15 | |
*** josecastroleon has quit IRC | 06:21 | |
*** edmondsw has joined #openstack-keystone | 06:32 | |
*** edmondsw has quit IRC | 06:37 | |
*** rajalokan has joined #openstack-keystone | 06:38 | |
*** rcernin has joined #openstack-keystone | 06:42 | |
*** pcaruana has joined #openstack-keystone | 06:43 | |
*** itlinux has quit IRC | 06:51 | |
*** edmondsw has joined #openstack-keystone | 06:55 | |
*** hoonetorg has quit IRC | 07:05 | |
*** david-lyle has quit IRC | 07:08 | |
*** david-lyle has joined #openstack-keystone | 07:08 | |
*** thorst_afk has joined #openstack-keystone | 07:08 | |
*** jamielennox has quit IRC | 07:09 | |
*** gus has quit IRC | 07:09 | |
*** gus has joined #openstack-keystone | 07:10 | |
*** thorst_afk has quit IRC | 07:13 | |
*** jamielennox has joined #openstack-keystone | 07:14 | |
*** tesseract has joined #openstack-keystone | 07:19 | |
*** aojea has joined #openstack-keystone | 07:20 | |
*** hoonetorg has joined #openstack-keystone | 07:21 | |
*** josecastroleon has joined #openstack-keystone | 07:36 | |
*** edmondsw has quit IRC | 07:36 | |
*** josecastroleon has quit IRC | 07:41 | |
*** ioggstream has joined #openstack-keystone | 07:44 | |
*** josecastroleon has joined #openstack-keystone | 07:59 | |
*** thorst_afk has joined #openstack-keystone | 08:09 | |
*** thorst_afk has quit IRC | 08:14 | |
*** jaosorior has quit IRC | 08:55 | |
*** jaosorior has joined #openstack-keystone | 09:00 | |
*** StefanPaetowJisc has joined #openstack-keystone | 09:01 | |
*** StefanPaetowJisc has quit IRC | 09:04 | |
*** thorst_afk has joined #openstack-keystone | 09:10 | |
*** StefanPaetowJisc has joined #openstack-keystone | 09:11 | |
*** StefanPaetowJisc has quit IRC | 09:13 | |
*** thorst_afk has quit IRC | 09:14 | |
*** dims has quit IRC | 09:28 | |
*** edmondsw has joined #openstack-keystone | 09:37 | |
*** kbaegis has joined #openstack-keystone | 09:37 | |
*** edmondsw has quit IRC | 09:42 | |
*** kbaegis has quit IRC | 09:49 | |
*** kbaegis has joined #openstack-keystone | 09:49 | |
*** dims has joined #openstack-keystone | 09:58 | |
*** jamesbenson has joined #openstack-keystone | 09:59 | |
*** jamesbenson has quit IRC | 10:03 | |
*** jmlowe has joined #openstack-keystone | 10:10 | |
*** thorst_afk has joined #openstack-keystone | 10:11 | |
*** kbaegis1 has joined #openstack-keystone | 10:14 | |
*** thorst_afk has quit IRC | 10:15 | |
*** kbaegis has quit IRC | 10:17 | |
*** abhi89 has quit IRC | 10:25 | |
*** dave-mccowan has joined #openstack-keystone | 10:29 | |
*** kbaegis1 has quit IRC | 10:33 | |
*** zhurong has quit IRC | 10:52 | |
*** thorst_afk has joined #openstack-keystone | 11:12 | |
*** thorst_afk has quit IRC | 11:16 | |
*** abhi89 has joined #openstack-keystone | 11:20 | |
*** edmondsw has joined #openstack-keystone | 11:38 | |
*** edmondsw has quit IRC | 11:43 | |
*** jmlowe_ has joined #openstack-keystone | 11:45 | |
*** jmlowe has quit IRC | 11:47 | |
*** thorst_afk has joined #openstack-keystone | 11:56 | |
*** raildo has joined #openstack-keystone | 12:02 | |
*** kbaegis has joined #openstack-keystone | 12:02 | |
*** raildo has quit IRC | 12:07 | |
*** abhi89 has quit IRC | 12:07 | |
*** lifeless has quit IRC | 12:07 | |
*** brad[] has quit IRC | 12:07 | |
*** abhi89 has joined #openstack-keystone | 12:07 | |
*** brad[]` has joined #openstack-keystone | 12:07 | |
*** john5223 has quit IRC | 12:09 | |
*** tesseract has quit IRC | 12:11 | |
*** john5223 has joined #openstack-keystone | 12:14 | |
*** dougshelley66 has joined #openstack-keystone | 12:22 | |
*** lifeless_ has joined #openstack-keystone | 12:22 | |
*** raildo has joined #openstack-keystone | 12:22 | |
*** tesseract has joined #openstack-keystone | 12:26 | |
*** abhi89 has quit IRC | 12:39 | |
*** abhi89 has joined #openstack-keystone | 12:59 | |
*** links has quit IRC | 13:00 | |
openstackgerrit | Colleen Murphy proposed openstack/keystone master: Fix endpoint update example in api-ref https://review.openstack.org/499141 | 13:06 |
*** catintheroof has joined #openstack-keystone | 13:15 | |
*** edmondsw has joined #openstack-keystone | 13:39 | |
*** edmondsw has quit IRC | 13:43 | |
*** lbragstad has joined #openstack-keystone | 13:44 | |
*** ChanServ sets mode: +o lbragstad | 13:44 | |
*** ducttape_ has joined #openstack-keystone | 13:47 | |
knikolla | o/ | 13:48 |
*** lucasxu has joined #openstack-keystone | 13:48 | |
lbragstad | o/ | 13:49 |
*** mnaser has joined #openstack-keystone | 13:50 | |
mnaser | i'm doing some testing for pike+keystone deployed via puppet (upgrades) and post upgrade i see a bunch of "Forbidden: You are not authorized to perform the requested action." (i guess policy somehow is blocking calls to /v3/auth/tokens).. i noticed the original policy from ocata was there and replaced it by "{}" (for policy in code stuff) and i'm still seeing this | 13:55 |
mnaser | would anyone have any ideas? | 13:55 |
mnaser | i'm just wondering if the puppet modules are missing something or if there are things being carried over which should be be marked absent in puppet | 13:55 |
*** gyee has joined #openstack-keystone | 13:56 | |
lbragstad | mnaser: o/ | 13:57 |
mnaser | hey lbragstad :) | 13:58 |
lbragstad | mnaser: we moved all policy into code - so if the policy files contained defaults, you can remove them | 13:58 |
mnaser | lbragstad that was my first step, so now my policy is just {} yet still getting that | 13:58 |
lbragstad | mnaser: when you try to validate a token? | 13:59 |
mnaser | lbragstad when i try to authenticate | 13:59 |
lbragstad | huh - really?! | 13:59 |
lbragstad | https://github.com/openstack/keystone/blob/e0a8780b63dfe611c587a654855158968b86373b/keystone/auth/routers.py#L30 | 13:59 |
mnaser | http://paste.openstack.org/show/619895/ | 13:59 |
mnaser | i mean unless i'm on the wrong place but | 14:00 |
mnaser | that error seems like a policy denial | 14:00 |
lbragstad | https://github.com/openstack/keystone/blob/e0a8780b63dfe611c587a654855158968b86373b/keystone/auth/controllers.py#L107 | 14:00 |
mnaser | not a unauthorized | 14:00 |
mnaser | (unless i'm wrong?) | 14:00 |
lbragstad | we apply the policy protection using a decorator at the controller layer | 14:00 |
lbragstad | https://github.com/openstack/keystone/blob/e0a8780b63dfe611c587a654855158968b86373b/keystone/auth/controllers.py#L299 | 14:00 |
lbragstad | for example ^ | 14:00 |
mnaser | but that error is a policy error (just to confirm right?) | 14:01 |
lbragstad | but we don't even protect the authentication endpoint | 14:01 |
mnaser | oh | 14:01 |
mnaser | well that's what i thought | 14:01 |
lbragstad | checking the trace | 14:01 |
mnaser | it wouldn't even make sense for it to be | 14:01 |
lbragstad | right | 14:01 |
lbragstad | fwiw - that error is our generic Forbidden error | 14:01 |
lbragstad | which is what's used in policy enforcement | 14:01 |
mnaser | oh so it could be an auth issue | 14:02 |
mnaser | i wonder if it's memcache not being restarted after the upgrade | 14:02 |
lbragstad | https://github.com/openstack/keystone/blob/e0a8780b63dfe611c587a654855158968b86373b/keystone/common/policy.py#L60 | 14:02 |
mnaser | nope | 14:02 |
lbragstad | how would that lead to 403s? | 14:03 |
mnaser | lbragstad ive seen sometimes where memcache might have data that doesn't make sense to the newer openstack release | 14:03 |
mnaser | maybe not in keystone specifically but in other services (i cant remember) | 14:03 |
lbragstad | ah | 14:03 |
mnaser | lbragstad ok. so that request was magnum making trust auth that was messing up | 14:04 |
*** links has joined #openstack-keystone | 14:05 | |
mnaser | i just tried a normal auth (looks like keystone v2 was configured on this dashboard) | 14:05 |
mnaser | Authorization failed. The request you have made requires authentication. from 192.168.0.250: Unauthorized: The request you have made requires authentication. | 14:05 |
mnaser | ok, basic troubleshooting: keystone-manage doctor shows "Credential key repository is empty." | 14:06 |
*** links has quit IRC | 14:06 | |
*** pcaruana has quit IRC | 14:06 | |
lbragstad | yeah - that should be fine | 14:06 |
lbragstad | we built in a null key that will be used if the key repository isn't populated | 14:07 |
*** edmondsw has joined #openstack-keystone | 14:07 | |
mnaser | key repo is readable by user (does keystone do a failsafe "i wont auth anything" if it is?) | 14:07 |
*** gyee has quit IRC | 14:08 | |
lbragstad | if the process running keystone can't read that location - you'll see a failure for sure | 14:08 |
lbragstad | we do that if the fernet token provider is used and the key repository isn't readable https://github.com/openstack/keystone/blob/e0a8780b63dfe611c587a654855158968b86373b/keystone/token/providers/fernet/core.py#L33-L45 | 14:09 |
mnaser | lbragstad certainly not that, sudo -u keystone cat /etc/ekystone/fernet-keys/X works well | 14:11 |
lbragstad | mnaser: ok - so you have keys | 14:12 |
lbragstad | are you authenticating against v2.0 or v3? | 14:13 |
mnaser | in this case the failure is happening against both (v2.0 from dashboard, v3 from magnum-conductor with trust auth) | 14:13 |
lbragstad | so you're scoping a trust | 14:14 |
mnaser | db revision is 109 which matches the latest as well hm | 14:14 |
*** rbrndt has joined #openstack-keystone | 14:14 | |
mnaser | at this point i simplified it to osc client and that is failing auth too | 14:14 |
mnaser | i wonder if it has to do with the password hashing changes | 14:14 |
lbragstad | i just thought of that | 14:15 |
mnaser | "It is recommended passwords be changed after upgrade to Pike." | 14:15 |
mnaser | doesnt imply it should be but.. | 14:16 |
lbragstad | mnaser: there was an upgrade path built in | 14:16 |
lbragstad | https://review.openstack.org/#/c/438701/ | 14:16 |
mnaser | wonder if that didnt kick in somehow | 14:16 |
*** jamesbenson has joined #openstack-keystone | 14:16 | |
lbragstad | mnaser: do you see the compat option here - https://review.openstack.org/#/c/438701/17/keystone/conf/identity.py ? | 14:17 |
lbragstad | line 171 | 14:17 |
mnaser | lbragstad it is not present in keystone.conf but it is present (but commented out) in keystone.conf.rpmnew | 14:17 |
mnaser | (this was a rpm upgrade with rdo pkgs) | 14:18 |
lbragstad | ah - then it would be False | 14:18 |
mnaser | in this case, i turned off all the other keystone instances | 14:18 |
mnaser | so only 1 is running | 14:18 |
lbragstad | ok | 14:19 |
mnaser | i see contract/expand... is that a keystone-manage command or is it integrated with keystone-managed? | 14:19 |
mnaser | s/managed/manage/ | 14:19 |
lbragstad | yeah - so we have keystone-manage db_sync | 14:20 |
lbragstad | but we also have keystone-manage db_sync --expand | 14:20 |
lbragstad | keystone-manage db_sync --migrate, and keystone-manage db_sync --contract | 14:20 |
lbragstad | by default, if you just run keystone-manage db_sync it will perform all three | 14:20 |
*** jamesbenson has quit IRC | 14:21 | |
lbragstad | the --expand, --migrate, --contract, bits are specific to doing a rolling upgrade | 14:21 |
mnaser | okay i see, so it's no tthat | 14:21 |
lbragstad | cc kmalloc ^ | 14:21 |
mnaser | ok with password_hash_algorithm being unset wont it default to bcrypt .. is it possible it's trying to test passwords using bcrypt? | 14:22 |
mnaser | because technically keystone couldn't possibly reencrypt (unless the hashing method is stored in the db?) | 14:23 |
mnaser | or _get_hasher_from_ident is not doing it's job possibly | 14:24 |
cmurphy | the hashing method is stored with the password | 14:25 |
mnaser | hmm | 14:25 |
mnaser | should the password be stored in password or password_hash | 14:26 |
*** jamesbenson has joined #openstack-keystone | 14:26 | |
lbragstad | https://github.com/openstack/keystone/blob/e0a8780b63dfe611c587a654855158968b86373b/keystone/common/password_hashing.py#L37-L42 | 14:26 |
lbragstad | mnaser: new passwords should be stored in password_hash i believe | 14:26 |
mnaser | what i see in doing "select * from password;" is that password_hash is null for all except admin user (which is failing to auth) | 14:26 |
lbragstad | let me double check the change | 14:26 |
mnaser | and then password is not null for the rest | 14:27 |
*** ducttap__ has joined #openstack-keystone | 14:28 | |
*** ducttape_ has quit IRC | 14:28 | |
lbragstad | yeah - it returns password_hash if it exists in the backend, otherwise it returns the hash stored in password | 14:28 |
lbragstad | https://review.openstack.org/#/c/438701/17/keystone/identity/backends/sql_model.py | 14:28 |
*** mvk has joined #openstack-keystone | 14:29 | |
*** cfriesen_ has joined #openstack-keystone | 14:31 | |
mnaser | lbragstad ok something is odd here | 14:32 |
mnaser | manually generated bcrypt for 'foobar' | 14:32 |
mnaser | set it in password_hash | 14:32 |
mnaser | it auth'd | 14:32 |
mnaser | i wonder what happened exactly and how to prevent it | 14:32 |
*** aojea has quit IRC | 14:32 | |
lbragstad | hmm | 14:33 |
lbragstad | just to double check | 14:33 |
lbragstad | you're currently running pike | 14:33 |
mnaser | yes, the release hash | 14:33 |
lbragstad | and you have version 109 in the migrate repo | 14:33 |
mnaser | yep, verified that too | 14:33 |
lbragstad | the expand, migrate, and contract repos are all at version 24, right? | 14:33 |
mnaser | let me check that | 14:33 |
mnaser | yes | 14:33 |
lbragstad | ok | 14:34 |
mnaser | the only weird thing i saw happen running puppet was.. the package was upgraded, httpd was not restarted yet, and puppet started configuring resources | 14:34 |
lbragstad | configuring resources? | 14:34 |
mnaser | puppet can create keystone users etc by calling openstack CLI | 14:35 |
mnaser | so maybe the fact that the old codebase was still running and it might have started to try and configure the passwords | 14:35 |
lbragstad | ah - so it started doing that before httpd was restarted to effectively doing that against ocata instead of pike | 14:35 |
mnaser | i suspect that is the case, let me pull out the puppet logs | 14:36 |
lbragstad | ok | 14:36 |
lbragstad | if you get a token now - can you create a new user and try authenticating? | 14:36 |
lbragstad | now that you've resolved the hash issue manually | 14:36 |
mnaser | indeed, that is the case that it started running it against it | 14:38 |
lbragstad | aha | 14:38 |
mnaser | so we need to make sure that this doesn't happen | 14:39 |
mnaser | let me verify i can add users properly etc | 14:39 |
lbragstad | mnaser: can you forcible bounce httpd immediately after the upgrade? | 14:39 |
mnaser | lbragstad we do have 'anchor' points where we can control order | 14:39 |
lbragstad | that sounds promising | 14:40 |
mnaser | ok | 14:41 |
mnaser | something weird is happening | 14:41 |
mnaser | so it was working, i reran puppet, puppet tried to remove the debug flag i enabled, because it did that, it triggered a db_sync, fernet_setup and bootstrap exec's | 14:41 |
mnaser | and by that point i couldn't authenticate again | 14:42 |
mnaser | let me see if i can repro.. | 14:42 |
lbragstad | but the database should have been upgraded, right? | 14:42 |
*** ducttape_ has joined #openstack-keystone | 14:42 | |
mnaser | lbragstad it is, but puppet doesn't know that so it's kindof a noop | 14:43 |
mnaser | at least we assume that in puppet land | 14:43 |
lbragstad | yeah | 14:43 |
lbragstad | but it does invoke bootstrap | 14:44 |
mnaser | is bootstrap not noop as well? | 14:44 |
mnaser | maybe that's what's happening | 14:45 |
lbragstad | mnaser: bootstrap should be idempotent | 14:45 |
lbragstad | https://github.com/openstack/keystone/blob/e0a8780b63dfe611c587a654855158968b86373b/keystone/cmd/cli.py#L216-L261 | 14:45 |
*** ducttap__ has quit IRC | 14:45 | |
mnaser | lbragstad ok i think i found the culprit | 14:46 |
mnaser | looking at the logs | 14:46 |
mnaser | 2017-08-30 14:39:48.074 19211 INFO keystone.cmd.cli [req-447bb23f-7018-47bc-92f3-57ecb6721491 - - - - -] Reset password for user admin. | 14:46 |
mnaser | so it looks like the bootstrap is resetting the password | 14:46 |
lbragstad | https://github.com/openstack/keystone/blob/e0a8780b63dfe611c587a654855158968b86373b/keystone/cmd/cli.py#L245-L252 | 14:46 |
lbragstad | so - do you know if that password reset is happening against ocata or pike? | 14:47 |
mnaser | the password reset has to happen again pike, i did a yum update at this point on this vm | 14:48 |
lbragstad | if the password reset happens on pike, shouldn't things work? | 14:48 |
lbragstad | is the issue that the password get reset using a different hash from ocata that pike doesn't understand? | 14:48 |
mnaser | lbragstad or maybe it's a puppet bug and the way it supplies the password to the bootstrap too possibly | 14:49 |
lbragstad | mnaser: after pike is running, can you use bootstrap to reset the admin password? | 14:50 |
lbragstad | and authenticate with the admin user after that | 14:50 |
*** ducttap__ has joined #openstack-keystone | 14:50 | |
lbragstad | that case *should* work | 14:50 |
mnaser | ok i think this is a puppet issue. it looks like if it doesn't get admin_password supplied, it uses the value of admin_token | 14:51 |
mnaser | which means that the password is reset to the value of the token | 14:51 |
lbragstad | oh - interesting | 14:52 |
mnaser | which really means it flip/flops because the resource which creates the admin user is setting a different password than the one bootstrap is using | 14:52 |
lbragstad | right | 14:52 |
lbragstad | that's a long password to remember ;) | 14:52 |
mnaser | so bootstrap sets admin user to $token, then later puppet runs openstack user set password <the_actual_password> | 14:52 |
lbragstad | interesting | 14:53 |
*** ducttape_ has quit IRC | 14:53 | |
mnaser | lbragstad sorry for the noise :( looks like that was issue the whole time | 14:54 |
mnaser | i have no idea why it's manifested itself *just now* | 14:55 |
lbragstad | mnaser: yeah - that's surprising | 14:56 |
lbragstad | mnaser: no worries - glad you were able to get it figured out :) | 14:56 |
*** edmondsw has quit IRC | 14:56 | |
mnaser | lbragstad personal goal is to get pike deployed asap and looks like keystone will be one of the first up :) thanks for the help | 14:56 |
lbragstad | mnaser: good deal! let us know if you run into anything else. | 14:56 |
mnaser | i think this seems to be it for now! merci beaucoup | 14:57 |
lbragstad | mnaser: anytime! | 14:57 |
*** rcernin has quit IRC | 15:09 | |
*** thegreenhundred has joined #openstack-keystone | 15:10 | |
*** kbaegis has quit IRC | 15:14 | |
*** mvk has quit IRC | 15:15 | |
*** jamesbenson has quit IRC | 15:31 | |
*** gyee has joined #openstack-keystone | 15:33 | |
*** kbaegis has joined #openstack-keystone | 15:33 | |
*** kbaegis has quit IRC | 15:38 | |
*** aojea has joined #openstack-keystone | 15:39 | |
*** aojea has quit IRC | 15:39 | |
*** aojea has joined #openstack-keystone | 15:39 | |
*** kbaegis has joined #openstack-keystone | 15:40 | |
*** itlinux has joined #openstack-keystone | 15:47 | |
*** itlinux has quit IRC | 15:48 | |
*** thegreenhundred has quit IRC | 15:48 | |
openstackgerrit | Gage Hugo proposed openstack/keystone master: Refactor removal of duplicate projects/domains https://review.openstack.org/491574 | 15:48 |
*** manjaroi3 has joined #openstack-keystone | 15:48 | |
*** pcaruana has joined #openstack-keystone | 15:54 | |
*** manjaroi3 has quit IRC | 15:56 | |
*** thegreenhundred has joined #openstack-keystone | 15:56 | |
*** thegreenhundred has quit IRC | 15:58 | |
*** thegreenhundred has joined #openstack-keystone | 15:58 | |
*** jmlowe_ has quit IRC | 16:02 | |
*** abhi89 has quit IRC | 16:09 | |
openstackgerrit | Samuel Pilla proposed openstack/python-keystoneclient master: Add project tags to keystoneclient https://review.openstack.org/481223 | 16:14 |
lbragstad | cmurphy: do you run full devstack (including nova) on your x1? | 16:15 |
*** aojea has quit IRC | 16:15 | |
openstackgerrit | Omar Tleimat proposed openstack/keystone master: Add unit tests to mapping_purge https://review.openstack.org/408304 | 16:17 |
cmurphy | lbragstad: i usually just keystone + nova + *-api but a full devstack is possible | 16:18 |
cmurphy | needs an 8GB vm | 16:19 |
gagehugo | lbragstad I used a full devstack for that tempest random failing test bug, it worked fine imo | 16:20 |
lbragstad | cmurphy: ah - you run in a virtual machine | 16:20 |
lbragstad | cmurphy: gagehugo have either of you tried running devstack in a container? https://docs.openstack.org/devstack/latest/guides/lxc.html | 16:21 |
cmurphy | lbragstad: heh no | 16:21 |
cmurphy | i have a whole virsh workflow that i'm pretty settled on | 16:21 |
lbragstad | nice | 16:21 |
openstackgerrit | Omar Tleimat proposed openstack/keystone master: Add unit tests to mapping_purge https://review.openstack.org/408304 | 16:23 |
*** otleimat has joined #openstack-keystone | 16:24 | |
gagehugo | lbragstad no, I've used openstack-helm before to test some keystone changes | 16:31 |
gagehugo | but haven't done a full stack yet | 16:31 |
*** itlinux has joined #openstack-keystone | 16:31 | |
*** rbrndt has left #openstack-keystone | 16:32 | |
*** markvoelker has joined #openstack-keystone | 16:32 | |
*** tesseract has quit IRC | 16:33 | |
gagehugo | http://openstack-helm.readthedocs.io/en/latest/install/developer/all-in-one.html | 16:36 |
*** aselius has joined #openstack-keystone | 16:46 | |
*** jamesbenson has joined #openstack-keystone | 16:53 | |
*** jamesbenson has quit IRC | 16:57 | |
*** stlbigdog has joined #openstack-keystone | 17:02 | |
*** markvoelker has quit IRC | 17:07 | |
lbragstad | gagehugo: cmurphy interesting - i hit an issue where the container doesn't start because of a missing character device file on the host | 17:11 |
lbragstad | i guess it's needed so that the container can actually use kvm | 17:12 |
*** aojea has joined #openstack-keystone | 17:12 | |
*** dims has quit IRC | 17:14 | |
*** dims has joined #openstack-keystone | 17:18 | |
gagehugo | hmm | 17:18 |
*** stlbigdog has quit IRC | 17:23 | |
gagehugo | lbragstad it looks like there is some concern about the project tag character size limit of 60, would there be any reason to object upping it? | 17:25 |
lbragstad | gagehugo: i don't think so - but i think we limited it based on the fact nova limits it and we wanted to maintain consistency | 17:26 |
gagehugo | lbragstad I think we got 60 from the nova spec, but it looks like they upped it to 80 for the implementation | 17:26 |
lbragstad | gagehugo: i'd be fine updating our spec to maintain consistency then | 17:27 |
lbragstad | i don't think we should deviate | 17:27 |
gagehugo | yeah, I've been updating the spec as things change | 17:27 |
*** ioggstream has quit IRC | 17:28 | |
*** stlbigdog has joined #openstack-keystone | 17:28 | |
lbragstad | gagehugo: maybe run it by the nova folks and see if they'd be interested in upping their limit,too? | 17:28 |
gagehugo | lbragstad 80 would be fine imo | 17:30 |
*** homeski has joined #openstack-keystone | 17:30 | |
lbragstad | i think consistency would be good | 17:30 |
homeski | Question about Keystone with Active Directory: How is the syncing happening? | 17:31 |
gagehugo | I'll probably break up the current change into controller/manager/backend anyway so I can change that as well | 17:31 |
homeski | Can't find any documentation about it | 17:31 |
*** eandersson has quit IRC | 17:31 | |
lbragstad | homeski: i'm not sure we have much documentation on Active Directory specifically | 17:34 |
lbragstad | https://wiki.openstack.org/wiki/HowtoIntegrateKeystonewithAD | 17:34 |
lbragstad | but that does use the ldap backend for keystone | 17:34 |
*** mjax has joined #openstack-keystone | 17:34 | |
lbragstad | https://github.com/openstack/keystone/blob/e0a8780b63dfe611c587a654855158968b86373b/keystone/conf/identity.py#L67-L79 | 17:34 |
*** kbaegis has quit IRC | 17:35 | |
homeski | If I add a new user on the LDAP side, is there any sort of syncing that needs to happen from Keystone, so that I can log in? | 17:35 |
*** kbaegis has joined #openstack-keystone | 17:36 | |
lbragstad | homeski: no, so long as the user exists in AD you should be able to log in | 17:38 |
lbragstad | the syncing will happen after the authentication happens using AD | 17:38 |
homeski | so it queries LDAP, makes sure it's valid credentials, if the user doesn't exist in local identity management, it will create user and default roles | 17:39 |
homeski | ^ for example, if I add a user to LDAP, then immediately try to login with Horizon | 17:40 |
mjax | lbragstad: I'm also curious about that, when/which module do roles and projects get assigned to the user that is authenticated on keystone through ldap for the first time? | 17:43 |
lbragstad | if ldap is configured and you authenticate a shadow user will be created | 17:44 |
lbragstad | which lives in keystone | 17:44 |
lbragstad | it's referred to as a non local user | 17:44 |
lbragstad | from there you can actually assign the non local user roles on projects and so on | 17:44 |
lbragstad | and you can operate on various projects like you would if the user lived natively in keystone as a local sql user | 17:45 |
*** ducttap__ has quit IRC | 17:49 | |
mjax | lbragstad: Sorry, i'm not very familiar with ldap, but am curious. Can ldap contain information about the user's domain, roles, and projects for keystone to extract, or is this all going to have to be done manually through the client after authenticating and creating a shadow user? | 17:52 |
*** rcernin has joined #openstack-keystone | 17:52 | |
lbragstad | mjax: keystone use to support assignment and resource read/write backends for ldap - but that is no longer supported | 17:53 |
lbragstad | currently, read-only is supported | 17:53 |
lbragstad | the assignments should be created manually after the user authenticates | 17:53 |
lbragstad | or is shadowed in keystone | 17:53 |
mjax | i see | 17:55 |
mjax | what do you mean by shadowed in keystone? | 17:55 |
*** edmondsw has joined #openstack-keystone | 17:55 | |
lbragstad | when keystone authenticates a user against an ldap backend, a user gets created in keystone's sql backend to model it | 17:55 |
lbragstad | or shadow it | 17:55 |
lbragstad | so when you give that user role assignments to projects, you're going to be giving those assignments to that shadow user | 17:56 |
mjax | right | 17:56 |
lbragstad | it helps in the case where you're hooking your openstack deployment up to a corporate ldap | 17:58 |
lbragstad | but you don't have write access to it | 17:59 |
lbragstad | so you can control the role assignments to a user persisted in ldap without having to open a bunch of tickets to the team that manages your corporate ldap | 17:59 |
mjax | I see, so then the role assignments and authorization stuff are going to be only on keystone, while ldap is used only for authenticating the user. Did I understand that correctly? | 18:00 |
lbragstad | yep | 18:01 |
mjax | I guess that leads me to the question, what's the difference between how shadow users and local users are represented in keystone? Does one have special features or is it mostly just so that shadow users are more easily identifiable | 18:03 |
*** ducttape_ has joined #openstack-keystone | 18:05 | |
openstackgerrit | Colleen Murphy proposed openstack/keystone master: Fix endpoint examples in api-ref https://review.openstack.org/499141 | 18:07 |
*** jmlowe has joined #openstack-keystone | 18:08 | |
homeski | lbragstad: thanks for information | 18:09 |
homeski | very helpful | 18:09 |
*** markvoelker has joined #openstack-keystone | 18:10 | |
homeski | exactly what I needed to know :) | 18:10 |
*** jamesbenson has joined #openstack-keystone | 18:22 | |
*** aojea has quit IRC | 18:35 | |
lbragstad | homeski: glad i could help | 18:36 |
*** markvoelker has quit IRC | 18:43 | |
*** edmondsw has quit IRC | 19:04 | |
*** markvoelker has joined #openstack-keystone | 19:06 | |
*** edmondsw has joined #openstack-keystone | 19:06 | |
*** edmondsw has quit IRC | 19:10 | |
*** rama_y has joined #openstack-keystone | 19:11 | |
*** edmondsw has joined #openstack-keystone | 19:14 | |
*** jamesben_ has joined #openstack-keystone | 19:14 | |
*** jamesbenson has quit IRC | 19:15 | |
*** edmondsw has quit IRC | 19:18 | |
*** dims has quit IRC | 19:28 | |
*** ducttape_ has quit IRC | 19:28 | |
*** ducttape_ has joined #openstack-keystone | 19:28 | |
*** catintheroof has quit IRC | 19:28 | |
*** dims has joined #openstack-keystone | 19:31 | |
lbragstad | wow - devstack actually runs in a container | 19:33 |
cmurphy | heh | 19:33 |
lbragstad | i tripped over some cinder stuff with lvm, but... | 19:33 |
*** markvoelker has quit IRC | 19:37 | |
*** markvoelker has joined #openstack-keystone | 19:38 | |
*** jamesben_ has quit IRC | 19:45 | |
*** jamesbenson has joined #openstack-keystone | 19:45 | |
clarkb | lbragstad: nova compute also has to use the userland iscsi driver stuff in libvirt | 19:50 |
clarkb | but thats probably the biggest two items you'll hit with a vanilla devstack run | 19:50 |
lbragstad | clarkb: yeah - i attempted to follow https://docs.openstack.org/devstack/latest/guides/lxc.html#limitations but that didn't seem to help the lvm issue i was hitting | 19:51 |
lbragstad | it looked like it couldn't create the volumn group | 19:52 |
lbragstad | volume* | 19:52 |
*** markvoelker has quit IRC | 19:56 | |
*** edmondsw has joined #openstack-keystone | 20:00 | |
*** edmondsw_ has joined #openstack-keystone | 20:01 | |
*** edmondsw has quit IRC | 20:04 | |
*** edmondsw_ has quit IRC | 20:05 | |
*** aojea has joined #openstack-keystone | 20:09 | |
*** jmlowe has quit IRC | 20:12 | |
*** jmlowe has joined #openstack-keystone | 20:15 | |
*** kbaegis has quit IRC | 20:22 | |
*** stlbigdog has quit IRC | 20:27 | |
*** jamesbenson has quit IRC | 20:28 | |
*** catintheroof has joined #openstack-keystone | 20:29 | |
*** jose-phillips has joined #openstack-keystone | 20:29 | |
*** nicolasbock has joined #openstack-keystone | 20:32 | |
*** rama_y has quit IRC | 20:34 | |
*** rama_y has joined #openstack-keystone | 20:36 | |
*** jamesbenson has joined #openstack-keystone | 20:37 | |
*** pcaruana has quit IRC | 20:40 | |
*** edmondsw has joined #openstack-keystone | 20:43 | |
*** edmondsw has quit IRC | 20:48 | |
*** lucasxu has quit IRC | 20:49 | |
*** ducttape_ has quit IRC | 21:02 | |
*** ducttape_ has joined #openstack-keystone | 21:04 | |
*** jmlowe has quit IRC | 21:04 | |
*** thorst_afk has quit IRC | 21:12 | |
*** sapd has quit IRC | 21:26 | |
*** jamesbenson has quit IRC | 21:27 | |
*** jamesbenson has joined #openstack-keystone | 21:30 | |
*** jamesbenson has quit IRC | 21:34 | |
*** ducttape_ has quit IRC | 21:38 | |
*** edmondsw has joined #openstack-keystone | 21:39 | |
*** edmondsw_ has joined #openstack-keystone | 21:40 | |
*** edmondsw has quit IRC | 21:44 | |
*** jmlowe has joined #openstack-keystone | 21:44 | |
*** edmondsw has joined #openstack-keystone | 21:45 | |
*** edmondsw_ has quit IRC | 21:45 | |
*** ducttape_ has joined #openstack-keystone | 21:46 | |
*** thegreenhundred has quit IRC | 21:56 | |
*** edmondsw has quit IRC | 22:00 | |
*** aojea has quit IRC | 22:02 | |
*** aojea has joined #openstack-keystone | 22:02 | |
*** aojea has quit IRC | 22:06 | |
*** raildo has quit IRC | 22:08 | |
otleimat | cmurphy, lbragstad: reflected those changes here https://review.openstack.org/#/c/408304/ | 22:12 |
*** dave-mccowan has quit IRC | 22:12 | |
*** thegreenhundred has joined #openstack-keystone | 22:18 | |
*** edmondsw has joined #openstack-keystone | 22:22 | |
*** itlinux has quit IRC | 22:23 | |
*** edmondsw has quit IRC | 22:26 | |
*** efried is now known as efried_off | 22:28 | |
*** rcernin has quit IRC | 22:28 | |
*** edmondsw has joined #openstack-keystone | 22:39 | |
*** thegreenhundred has quit IRC | 22:39 | |
*** edmondsw has quit IRC | 22:43 | |
*** ducttap__ has joined #openstack-keystone | 22:49 | |
*** ducttape_ has quit IRC | 22:52 | |
*** edmondsw has joined #openstack-keystone | 23:01 | |
*** edmondsw has quit IRC | 23:06 | |
*** ducttap__ has quit IRC | 23:12 | |
*** stlbigdog has joined #openstack-keystone | 23:20 | |
*** ducttape_ has joined #openstack-keystone | 23:21 | |
*** itlinux has joined #openstack-keystone | 23:22 | |
*** stlbigdog has quit IRC | 23:23 | |
*** ducttape_ has quit IRC | 23:26 | |
*** markvoelker_ has joined #openstack-keystone | 23:45 | |
*** edmondsw has joined #openstack-keystone | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!