Tuesday, 2019-01-08

nicolas_o@mordred my irc connection is flaky, I recorded my issue here: https://storyboard.openstack.org/#!/story/200471500:00
nicolas_olet me know if you need more information. much appreciated.00:01
openstackgerritBruno HALEBLIAN proposed openstack/openstacksdk master: implement Proxy unassign_project_role_from_user  https://review.openstack.org/62907300:08
*** nicolas_o has quit IRC02:01
*** bobh_ has joined #openstack-sdks02:04
*** bobh_ has quit IRC02:08
*** dims has quit IRC04:47
*** dims has joined #openstack-sdks04:48
*** dims has quit IRC04:56
*** dims has joined #openstack-sdks04:56
*** bobh_ has joined #openstack-sdks05:19
*** bobh_ has quit IRC05:26
*** radeks has joined #openstack-sdks05:48
*** ykarel has joined #openstack-sdks06:03
ykarelmordred, hi06:05
ykarelmordred, remember thread.error: can't start new thread i mentioned sometime back06:05
ykarelin openstacksdk python2 and i proposed https://review.openstack.org/#/c/61554306:06
ykarelmordred, looks like it's fixed in master, as test passing during package build https://review.rdoproject.org/r/#/c/16824/06:07
ykarelmordred, can u confirm the fix, and get a new release from master06:08
*** radeks has quit IRC06:11
*** radeks has joined #openstack-sdks06:20
*** Luzi has joined #openstack-sdks06:44
*** gkadam has joined #openstack-sdks07:48
*** slaweq has joined #openstack-sdks07:49
*** gkadam is now known as gkadam-afk07:50
*** ykarel is now known as ykarel|lunch07:51
*** slaweq has quit IRC07:53
*** slaweq has joined #openstack-sdks07:55
*** bobh_ has joined #openstack-sdks08:02
*** bobh_ has quit IRC08:07
*** ssbarnea has joined #openstack-sdks08:15
*** ssbarnea|bkp2 has quit IRC08:16
*** e0ne has joined #openstack-sdks08:16
*** gkadam-afk is now known as gkadam08:24
*** tosky has joined #openstack-sdks08:37
*** jpena|off is now known as jpena08:50
openstackgerritJohannes Kulik proposed openstack/openstacksdk master: Use pagination detection by default  https://review.openstack.org/62911008:58
*** jpich has joined #openstack-sdks09:03
*** gtema has joined #openstack-sdks09:05
*** bobh_ has joined #openstack-sdks09:15
*** dtantsur|afk is now known as dtantsur09:21
*** ykarel|lunch is now known as ykarel09:22
openstackgerritBruno HALEBLIAN proposed openstack/openstacksdk master: implement identity v3 Proxy "unassign_project_role_from_user"  https://review.openstack.org/62907310:06
*** bobh_ has quit IRC10:19
*** jpich has quit IRC10:53
*** jpich has joined #openstack-sdks10:54
*** cdent has joined #openstack-sdks11:18
*** d0ugal has quit IRC12:01
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Compute location properly in server  https://review.openstack.org/62638012:18
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Skip block-storage tests when service is not found  https://review.openstack.org/62896912:18
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Skip block storage v2 functional tests for a minute  https://review.openstack.org/62916212:18
*** d0ugal has joined #openstack-sdks12:22
mordredykarel: awesome! there have only been two patches landed in master that aren't in 0.22.0. I'm not sure which of them might have been related12:28
mordredykarel: but - I'll be happy to get a new sdk release cut - I'd like to land a couple of the patches in flight - perhaps a release by tomorrow?12:29
*** gkadam has quit IRC12:29
ykarelmordred, ok for me12:30
ykareland Thanks12:30
mordredjkulik: thanks for the patch - looks great, those test failures are unrelated and I've got a couple of patches up to deal with them12:32
jkulikmordred, the other one seems more important to me: https://review.openstack.org/62897712:34
jkulikespecially if pagination detection should become default12:34
mordredjkulik: yes! very much so12:36
*** jpena is now known as jpena|lunch12:37
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Skip v2 block-storage tests when service is not found  https://review.openstack.org/62896912:50
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Skip v2 block-storage tests when service is not found  https://review.openstack.org/62896912:50
mordredShrews: if you have a sec when you're up, ^^ and the one before it (the one before it is a hack workaround to unblock the gate - but then I think I actually figured out the real problem in the second patch)12:51
dtantsurmordred: so, maybe leave only the 2nd patch? :)13:24
mordreddtantsur: yeah ... but I also might be wrong :)13:24
mordredI thought I had it figured out yesterday too ... but oh I was so wrong13:25
dtantsurmordred: okay, +2 on the 1st patch13:25
dtantsurwill wait for the CI on the 2nd13:25
mordred++ ... and thanks!13:25
mordredthis also reminds me we really need to get the v3 volume support patch updated and landed13:26
dtantsurindeed13:26
*** zigo has joined #openstack-sdks13:28
*** jpena|lunch is now known as jpena13:34
*** ykarel is now known as ykarel|away13:35
mordredcmurphy, lbragstad: I'm seeing some errors in the osc functional tips job that are keystone related and that I have **no clue** what's up: http://logs.openstack.org/80/626380/9/check/osc-functional-devstack-tips/dcf2d3a/testr_results.html.gz13:37
mordredmakes me think maybe there is a devstack something missing? or something something?13:38
mordredShrews, dtantsur: woot! https://review.openstack.org/#/c/626380/ is green which means it's worth reviewing - it's not NEARLY as much test case carnage as I thought it would be13:40
cmurphymordred: oops that's probably because of https://review.openstack.org/#/c/62600713:44
cmurphylbragstad will know what to do13:44
mordredcmurphy: neat! I like it when people know what to do13:44
mordreddtantsur: replied to your comment - and thanks!13:58
dtantsurah, cool13:58
mordreddtantsur: I keep looking at the code and thinking I'm going to go through and rename all of those uses of "session" which are not actually sessions14:00
mordreddtantsur: I then keep giving up because life is too short14:00
dtantsurheh, right14:00
*** d0ugal has quit IRC14:02
mordredooh! the update to the volume tests hit a whole new issue. I'm guessing that one needs admin credentials14:03
*** d0ugal has joined #openstack-sdks14:04
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Extract getting connection from proxy into method  https://review.openstack.org/62919314:04
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Skip v2 block-storage tests when service is not found  https://review.openstack.org/62896914:06
mordredlook at all these exciting patches14:07
lbragstadcmurphy mordred interesting14:07
mordreduhoh :)14:08
lbragstadthat's failing because the user making the request in those tests isn't defined as a system administrator14:09
lbragstadthey probably only have a role assignment on the project14:09
lbragstadand in keystone we've changed those policies to require a system admin in order to access the registered limits APIs14:09
*** d0ugal has quit IRC14:12
lbragstadcmurphy mordred https://review.openstack.org/#/c/621016/ was probably the breaking change14:13
*** bobh_ has joined #openstack-sdks14:16
mordredlbragstad: ok. so we should likely update the osc tests for that to use an admin auth context rather than an end-user one14:17
*** bobh_ has quit IRC14:21
mordredlbragstad: hrm. it looks like osc is already using the devstack-admin account (it's doing domain deletion operations already and stuff)14:21
*** ykarel|away has quit IRC14:21
Shrewsmordred: the osc-func-tips job is pretty sad in 38014:23
Shrewsnon-voting but probably worth looking into?14:23
mordredShrews: yah - that's the convo with lbragstad14:33
mordredlbragstad: perhaps the devstack-admin account in devstack isn't being granted the system admin role?14:34
Shrewsmordred: pfffft... who has time to read scrollback??14:36
lbragstadmordred it should have it14:37
lbragstadkeystone-manage bootstrap does that for you14:37
lbragstadwhich i'm pretty sure devstack executes14:37
lbragstadbut, i think the tests need to ask for a system-scoped token instead of a project-scoped one14:37
lbragstadand use that for those requests instead14:37
lbragstadhttps://developer.openstack.org/api-ref/identity/v3/index.html?expanded=token-authentication-with-scoped-authorization-detail#id3014:39
mordredoh goodie14:40
mordredlbragstad: I honestly have no idea how to get a system-scoped token with osc - or if it's even possible14:41
lbragstaddoes osc use ksa?14:41
mordredyeah - but I don't know if we've got 'scope' plumbed through in either config or cli args14:42
mordredlbragstad: how does one get a system scoped token with ksa?14:42
lbragstadlooking for an example14:43
mordredcool. fwiw, we're still using keystoneclient for keystone things14:44
*** Luzi has quit IRC14:45
lbragstadmordred if you set system_scope=all with password authentication, it should work https://docs.openstack.org/keystoneauth/latest/plugin-options.html#password14:47
* dtantsur has been around for 5 years and still does not understand token scopes well. nor the whole Identity API.14:48
lbragstad:(14:49
lbragstadwe did overhaul some docs to try and make things more clear14:49
lbragstadhttps://docs.openstack.org/keystone/latest/admin/tokens-overview.html14:49
frickler"openstack --os-system-scope system registered limit create ..." works for me with the devstack-admin user and unsetting all OS_PROJECT/TENANT env14:51
mordredah - --os-system-scope14:52
mordredneat14:52
fricklerwould be nice to give a warning when it still does project-scope, instead of silently ignoring the system-scope request in that case14:53
mordredI think it would be better if --os-system-scope overrode the project settings -- unsetting env vars is a bit of a pita14:54
dtantsurlbragstad: this actually helps. I don't think i saw it when I started rust-openstack.14:54
mordredfrickler: because the osc test cases are all run with env vars set ...14:56
fricklermordred: yeah, having cli options override other env vars would be neat, too, but not so easy I fear14:57
lbragstadfrickler it does.. but since the registered limit and limit apis are still experimental, we didn't deprecate the policies14:57
lbragstadhttps://gist.github.com/lbragstad/ad744f1a5acb4f4dc3387383514ab33f is an example of defining different scopes in clouds.yaml14:58
lbragstadif you had different users with different roles on the system (one of them being a read-only role)14:58
fricklermordred: maybe setting things like --os-project-name="" to set overrides to the env via cli might work in the meantime14:58
lbragstaddtantsur if you have questions about how all that works, you should bug me :) I'm happy to help14:59
mordredfrickler: yeah, I'll try that14:59
dtantsurlbragstad: thanks! I will when I start extending authentication support there14:59
fricklerlbragstad: I'm not talking about policies, a simple "openstack --os-system-scope system token issue" will still issue a project-scope token without warning when e.g. OS_TENANT_NAME is set14:59
lbragstadahh14:59
lbragstadgot it - i misread that14:59
cmurphylbragstad: it seems like this is pointing out an issue in our backwards compatibility?15:00
lbragstadwith the policies or with osc?15:00
cmurphywith the policies15:00
lbragstadthey are backwards compatible when we deprecate them - due to the logic in the oslo.policy library15:01
lbragstadthe only reason we didn't with the registered limit and limit policies is because the api is experimental15:01
cmurphyoh I see15:01
cmurphyI assumed we went through the same process with those15:02
openstackgerritMonty Taylor proposed openstack/python-openstackclient master: Set system-scope parameter for registered_limits tests  https://review.openstack.org/62920315:02
mordredok - there is a stab ... I'm not sure that'll fix it or not15:03
lbragstadcmurphy wxy brought it up here - https://review.openstack.org/#/c/621016/215:03
cmurphygot it15:03
mordredwhat we might need to do is add support for registered_limit in sdk and have the sdk force the system-scope auth regardless of what parameters are otherwise set then update osc to use sdk for these calls15:03
mordredbut - let's see if that works first15:04
*** ykarel|away has joined #openstack-sdks15:04
fricklermordred: doesn't work in my local tests, you cannot use an empty string at that location15:07
mordredblerg15:08
mordreda potentially easier fix would be to update the test runner to not source the env vars but instead to add an --os-cloud command line option to self.openstack in the tests15:08
mordredfrickler: do you have the ability to test if --os-project-id='' works if the project-id is coming from clouds.yaml instead of from env vars?15:09
mordredoh - you know what?15:09
fricklermordred: but that sounds ... modern15:09
*** ykarel|away is now known as ykarel15:09
mordredwe could make a new devstack user in clouds.yaml - devstack-system - that would not have project stuff set in teh first place and have system-scoped: all15:09
fricklermordred: also, you could simple use a different cloud ... yeah15:09
mordredI think that sounds like the best bet to me15:10
mordredI'm going to start by changing the func test setup to use --os-cloud instead of env var15:10
mordredand then work on getting an additional clouds.yaml entry for devstack15:10
fricklermordred: the different cloud user should probably be in devstack by default15:10
mordredyah15:10
mordredtotally agree15:10
mordredI think it'll be a nice addition to devstack for folks15:11
fricklero.k., I'm away for a bit, I can do reviews later15:11
lbragstadfrickler mordred yeah - that's what i do15:14
lbragstadsuper handy for switching profiles15:14
lbragstadi used it a bunch in a demo, trying to explain scopes https://youtu.be/A1vP6be1U5U?t=35315:15
* mordred is tempted to make a v2 clouds.yaml format a little inspired by the kubectl file - I really like how they have clusters and users as separate objects, and then 'contexts' which are combos of clusters and users ... but that would require *WAY* too much thinking for today15:19
openstackgerritMerged openstack/openstacksdk master: Skip block storage v2 functional tests for a minute  https://review.openstack.org/62916215:19
*** ykarel is now known as ykarel|away15:21
mordredgtema: did you see dtantsur's comment on https://review.openstack.org/#/c/621153/ - it came on a +A and was a suggestion for followup, just wanted to make sure you didn't miss it15:22
gtemamordred: yupp, have seen but have a "disaster" at work which I need to take care of15:22
mordredgtema: ugh. "disasters" are the worst. have fun with that15:22
gtemathanks15:23
mordredgtema: is the "disaster" caused by someone else's lack of planning? I really love those15:23
*** markvoelker has joined #openstack-sdks15:23
gtemano, it is caused by someone leaving the company and others on a long vacation and suddenly certificates stoped working15:24
*** markvoelker has quit IRC15:26
*** markvoelker has joined #openstack-sdks15:27
*** markvoelker has quit IRC15:27
mordredgtema: oh double ugh15:31
mordredlbragstad, frickler: remote:   https://review.openstack.org/629235 Add a devstack-system cloud for system scoped actions15:31
lbragstadmordred nice, thanks15:37
openstackgerritMonty Taylor proposed openstack/python-openstackclient master: Use devstack-system cloud for limit tests  https://review.openstack.org/62920315:46
openstackgerritMonty Taylor proposed openstack/python-openstackclient master: Use os-cloud instead of OS env vars for functional tests  https://review.openstack.org/62924315:46
mordredlbragstad, frickler: ^^ and there's the OSC update to use clouds.yaml - and then to use devstack-system for the limits tests15:46
* lbragstad hands mordred a coffee *and* a donut15:47
mordredmmm. donuts. is there nothing they can't do?15:48
*** e0ne has quit IRC15:56
*** bobh has joined #openstack-sdks15:59
openstackgerritBruno HALEBLIAN proposed openstack/openstacksdk master: implement identity v3 Proxy "unassign_project_role_from_user"  https://review.openstack.org/62907316:01
*** bobh has quit IRC16:06
* dtroyer is slowly catching up… re the CLI options and env vars… we use —no-* in commands to unset things, would —no-os-project-name be a worthwhile addition?16:08
dtroyerto temporarily unset an env var16:09
*** ykarel|away has quit IRC16:17
*** ykarel|away has joined #openstack-sdks16:17
*** e0ne has joined #openstack-sdks16:25
fricklerdtroyer: I've been testing whether that works, so the idea sounds plausible, but I'm not sure whether it would be worth the code complexity. maybe a global --ignore-environment option instead? but then, using all these options should be deprecated in favor of --os-cloud anyway, so probably let's just work with the latter16:52
*** jpich has quit IRC16:52
dtroyerfrickler: my reasoning for the precedence we have (cli options > env vars > clouds.yaml) is because of exactly this case, where you generally want your environment to contain the "normal" setup and can override on the command line for the exceptions.17:07
dtroyerI'm not sure a global 'ignore env' is right, I might only want to change one or two things, project being the one I run into myself most17:07
*** e0ne has quit IRC17:08
mordreddtroyer: I thinkm --no-project would be a great addition17:09
mordreddtroyer: although ultimately I'd like for people to be able to use these limit commands without needing to always set --os-system-scope --no-os-project-id --no-os-project-name --no-domain-id --no-domain-name ... because that just seems like we're abusing people :)17:10
mordredthat'll be a longer effort though17:10
mordredbe I definitely think being able to blank out specific things is a good idea17:11
dtroyermordred: I think I am behind on the keystone/ksa stuff around system scope, is that a thing we can (or should?) just assume for commands like this?17:12
mordreddtroyer: I think for _this_ one it is? like - from this morning's interaction I'm pretty sure we want to make the sdk automatically system-scope these calls17:25
mordreddtroyer: but - it's also fairly new information - so I could be totally wrong about it17:25
dtruongmordred: Have you seen an issue with openstacksdk reporting too many files open? https://bugs.launchpad.net/senlin/+bug/180969117:44
openstackLaunchpad bug 1809691 in senlin "Memory leak with senlin-engine sdk drivers" [Undecided,New]17:44
mordreddtruong: yeah - I think that's been reported by someone else too? there's one leak that exists that we added a connection.close() method to handle (although there is a patch in flight that should make the need for that go away)17:47
mordreddtruong: that was a thread leak though17:47
mordredbut maybe it's the same cause17:48
dtruongthe person who reported this bug in senlin proposed a patch set to set 'Connection' to 'Close' in the session header: https://review.openstack.org/#/c/627215/7/senlin/drivers/sdk.py17:49
dtruongIs that the correct way to work around the issue?17:50
*** gtema has quit IRC17:50
openstackgerritMerged openstack/openstacksdk master: Fix pagination key detection  https://review.openstack.org/62897717:57
mordredhrm. that I do not know about ... I'm not sure what that does - but seems to be a manipulation of the low-level requests session17:57
mordredcmurphy, lbragstad: ^^ any chance you know?17:57
*** ykarel|away has quit IRC17:58
mordreddtruong: it seems like that will close the remote connection, yeah - so that could certainly help with a connection/socket leak17:59
mordredI'm a little surprised that would be needed because requests is supposed to be doing connection pooling17:59
openstackgerritMerged openstack/openstacksdk master: fix typos  https://review.openstack.org/62725818:18
openstackgerritMerged openstack/openstacksdk master: Fix the misspelling of "configuration"  https://review.openstack.org/62747718:18
*** cdent has quit IRC18:22
*** e0ne has joined #openstack-sdks18:23
openstackgerritMonty Taylor proposed openstack/cliff master: Pin cmd2 back to <0.9 on all versions  https://review.openstack.org/62928318:30
*** jpena is now known as jpena|off18:30
mordreddhellmann: ^^ there's the cliff patch with a depends-on18:30
openstackgerritMerged openstack/openstacksdk master: Add possibility to override base_path for resource operations  https://review.openstack.org/62115318:34
openstackgerritMerged openstack/openstacksdk master: Rework orchestration to add update preview  https://review.openstack.org/62115418:34
*** dtantsur is now known as dtantsur|afk18:45
dhellmannmordred : see the "conflicts with" list there; it looks like someone else proposed a similar change and it failed a test. let's see how yours does...18:46
dhellmannoh, he's lacking the depends-on so that's probably why18:47
*** lbragstad has quit IRC18:52
*** lbragsta_ has joined #openstack-sdks18:52
*** lbragsta_ is now known as lbragstad18:55
mordreddhellmann, dtroyer: also - dmsimard was asking about an osc-lib release the other day18:59
dhellmannthere are some changes in that list that I don't recognize, so I'm not sure if they're backwards compatible or not19:19
dhellmannhttp://paste.openstack.org/show/74069419:19
dhellmann"default to using public interface for all clients"19:19
dhellmannmordred : ^19:20
mordreddhellmann: yah - that's to deal with underlying keystoneclient still defaulting to admin interface - even though that's a deprecated concept in general19:21
dhellmannok, so that sounds like a bug fix?19:22
* dhellmann is trying to figure out semver for this one19:22
dhellmannoh, first release for stein so it has to be a feature bump anyway19:23
mordred++19:24
mordreddmsimard: ^^19:25
dmsimardYay, thanks.19:39
*** e0ne has quit IRC19:39
*** e0ne has joined #openstack-sdks19:43
*** bobh has joined #openstack-sdks20:03
*** bobh has quit IRC20:10
*** tosky has quit IRC20:49
*** radeks has quit IRC21:02
*** slaweq has quit IRC21:28
*** slaweq has joined #openstack-sdks22:23
*** e0ne has quit IRC22:27
*** d0ugal has joined #openstack-sdks22:56
*** tosky has joined #openstack-sdks23:01
openstackgerritMonty Taylor proposed openstack/python-openstackclient master: Use os-cloud instead of OS env vars for functional tests  https://review.openstack.org/62924323:33
openstackgerritMonty Taylor proposed openstack/python-openstackclient master: Use devstack-system-admin cloud for limit tests  https://review.openstack.org/62920323:33

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!