Thursday, 2021-02-25

*** tosky has quit IRC00:00
*** k_mouza has joined #openstack-glance00:02
*** k_mouza has quit IRC00:07
*** Underknowledge has quit IRC01:23
*** Underknowledge has joined #openstack-glance01:24
*** rcernin has quit IRC02:06
*** zzzeek has quit IRC02:23
*** zzzeek has joined #openstack-glance02:24
*** rcernin has joined #openstack-glance02:38
*** zzzeek has quit IRC02:44
*** zzzeek has joined #openstack-glance02:46
*** zzzeek has quit IRC02:51
*** zzzeek has joined #openstack-glance02:53
*** zzzeek has quit IRC03:13
*** zzzeek has joined #openstack-glance03:15
*** whoami-rajat has joined #openstack-glance04:18
*** udesale has joined #openstack-glance04:22
abhishekklbragstad, I think either jokke or rosmaita can help you with this04:48
lbragstadi think i already figured it out - but thanks for the follow up :)04:49
lbragstadit looks like resource types can belong to an owner and then get associated to namespaces of a different owner04:50
*** ratailor has joined #openstack-glance04:50
abhishekklbragstad, that's cool05:03
lbragstadi just discovered that those aren't filtered at all either05:05
lbragstadif alice and bob are two users with role assignments on separate projects05:05
lbragstadalice can see resource types used for namespaces associated to bob's project05:06
openstackgerritLance Bragstad proposed openstack/glance master: Implement secure RBAC for metadef APIs  https://review.opendev.org/c/openstack/glance/+/76425105:11
openstackgerritLance Bragstad proposed openstack/glance-tempest-plugin master: WIP: Add protection testing for namespace objects  https://review.opendev.org/c/openstack/glance-tempest-plugin/+/77678905:12
lbragstadthat's about as far as i got today05:14
*** bhagyashri|ruck is now known as bhagyashri|rover05:15
lbragstadall the tempest plugin tests pass - but they're not done, yet05:15
*** gyee has quit IRC05:20
abhishekklbragstad, ack, thank you05:35
*** ajitha has joined #openstack-glance05:35
*** m75abrams has joined #openstack-glance05:43
*** ralonsoh has joined #openstack-glance06:01
*** yoctozepto0 has joined #openstack-glance06:05
*** yoctozepto has quit IRC06:05
*** yoctozepto0 is now known as yoctozepto06:05
*** lpetrut has joined #openstack-glance07:19
*** rcernin has quit IRC07:22
*** rcernin has joined #openstack-glance07:52
*** tosky has joined #openstack-glance08:38
*** happyhemant has joined #openstack-glance08:48
*** k_mouza has joined #openstack-glance09:31
*** Underknowledge has quit IRC09:37
*** Underknowledge1 has joined #openstack-glance09:37
*** Underknowledge1 is now known as Underknowledge09:37
*** lpetrut_ has joined #openstack-glance09:54
*** yoctozepto9 has joined #openstack-glance09:55
*** lpetrut has quit IRC10:03
*** yoctozepto has quit IRC10:03
*** m75abrams has quit IRC10:03
*** zzzeek has quit IRC10:03
*** irclogbot_0 has quit IRC10:03
*** yoctozepto9 is now known as yoctozepto10:03
*** udesale_ has joined #openstack-glance10:03
*** zzzeek has joined #openstack-glance10:05
*** udesale has quit IRC10:05
*** irclogbot_0 has joined #openstack-glance10:07
*** k_mouza has quit IRC10:21
*** k_mouza has joined #openstack-glance10:54
*** k_mouza has quit IRC10:58
*** k_mouza has joined #openstack-glance10:58
*** ratailor has quit IRC12:24
*** ratailor has joined #openstack-glance12:24
*** ratailor has quit IRC12:30
*** ratailor has joined #openstack-glance12:30
*** ratailor has quit IRC12:31
*** ratailor has joined #openstack-glance12:32
*** ratailor has quit IRC12:36
*** ratailor has joined #openstack-glance12:38
*** ratailor has quit IRC12:51
*** Luzi has joined #openstack-glance13:01
lbragstadi also noticed that we do leak namespace existence http://paste.openstack.org/show/802999/13:13
lbragstadwhich is different from the approach we take with images now that dansmith's patch merged13:14
lbragstad(er - at least isn't possible not that his patch merged)13:14
*** lpetrut__ has joined #openstack-glance13:14
*** lpetrut_ has quit IRC13:16
*** tkajinam has quit IRC13:36
*** k_mouza has quit IRC13:54
*** k_mouza has joined #openstack-glance13:55
abhishekkjokke, rosmaita, dansmith, smcginnis, glance weekly meeting in 5 minutes at #openstack-meeting13:56
abhishekksee you there13:56
openstackgerritDan Smith proposed openstack/glance master: Add a test for migration naming and phase rules  https://review.opendev.org/c/openstack/glance/+/77741313:56
dansmithlbragstad: I don't know anything about the metadadef stuff, but that seems to be the same pattern has we had for images yeah14:00
*** k_mouza has quit IRC14:00
*** k_mouza has joined #openstack-glance14:00
*** yoctozepto has quit IRC14:01
lbragstadyeah - i see a lot of the same patterns sprinkled up and down that stack14:01
*** yoctozepto has joined #openstack-glance14:01
*** Luzi has quit IRC14:28
*** jv_ has joined #openstack-glance14:28
dansmithlbragstad: do you have a zuul change to make us run tests with the new rules enabled?14:38
*** whoami-rajat has quit IRC14:38
lbragstaddansmith like this you mean? https://review.opendev.org/c/openstack/glance-tempest-plugin/+/773568/20/.zuul.yaml@3014:39
dansmithyup14:39
lbragstadthe gtp patches depend on the glance series for implementing all this stuff, and i'm primarily relying on the gtp tests to verify all this stuff14:41
lbragstadi can't really unwind what i think the correct behavior is just from the policy unit tests14:41
lbragstadso - i'm taking a pretty blackbox approach14:41
*** ratailor has joined #openstack-glance14:41
dansmithokay, well, I guess my point is, we probably need to have a regular tempest full run against glance with these rules turned on to make sure it continues to run as we expect, right?14:42
lbragstadideally - yeah... gmann was working on something like that14:42
dansmithokay14:43
lbragstadthe glance-functional-protection tests are asserting the new behavior14:43
dansmithif this wasn't experimental I'd be wanting that first, but.. calling it experimental I guess lets us get away with it :)14:43
dansmithlbragstad: right but only that small tempest regex14:43
lbragstadand that invokes these - https://review.opendev.org/c/openstack/glance-tempest-plugin/+/773568/20/glance_tempest_plugin/tests/rbac/v2/test_images.py14:44
*** ratailor has quit IRC14:44
*** ratailor has joined #openstack-glance14:45
gmanndansmith: lbragstad you mean tempest default run on new policy right?14:45
dansmithyes14:45
gmannor keep running on old one14:45
gmann+1, yeah. I am working on trying to move tempest in that direction14:45
dansmithokay14:45
lbragstadi was looking through some of the metadef tempest API tests last night, and they're pretty minimal14:46
dansmithI guess tempest needs to care because it creates roles and users?14:46
lbragstadat least from a protection perspective14:47
gmannlbragstad: but I have not started the glance unit test for new policy as i mentioned early. what is the final direction, i think i missed the discussion if that happened. unit tests or glance tempest plugins tests or both ?14:47
dansmithI was mostly thinking that it would mostly be a "create the roles in devstack like the new way" but maybe not14:47
lbragstaddansmith yeah - cmurphy implemented a bunch of stuff for that14:48
gmannyeah devstack keep role creation and tempest use those for tests creads14:49
*** ratailor has quit IRC14:49
gmannwe need few of more things for alt project/system reader/member/admin etc which is I am working on and should be ready by next week or so14:50
lbragstadyeah - so the credentials = ['system_admin', 'project_member'] list we use in the new protection tests rely on the new personas14:50
gmannyeah14:50
*** zzzeek has quit IRC14:51
lbragstadsorry - i'm multitasking meeting, i'll find an example14:51
lbragstadhttps://review.opendev.org/c/openstack/glance-tempest-plugin/+/773568/20/glance_tempest_plugin/tests/rbac/v2/test_images.py#135814:52
gmannyeah14:52
gmannand this one for nova hypervisor API to see how it will looks like https://review.opendev.org/c/openstack/tempest/+/740122/10/tempest/api/compute/admin/test_hypervisor.py#2914:52
*** zzzeek has joined #openstack-glance14:53
lbragstadyeah - so i think they all relies on dynamic credentials, by default14:53
gmannyes,14:53
lbragstadthe roles are created in keystone during the bootstrap process, so devstack handles that14:53
lbragstadand the dynamic cred bits handles the users and role assignments14:54
*** jmlowe has quit IRC14:54
gmannonly complexity comes on devstack side is - when we enable enforce_scope for any service then all openstack commands used in devstack for that service (for example like nova flavor create) needs to be move to new policy at same time14:55
gmannbut while changing it will see how it goes. in my testing patches i faced these things14:56
lbragstadi imagine we're going to hit a few places where we need to update the tests to use the correct client14:56
gmannyeah, that is one thing.  use client based on config enforce_scope14:57
*** ratailor has joined #openstack-glance14:59
* abhishekk going for dinner break15:00
lbragstadi do have some questions about the metadef API if folks are going to be around15:02
jokkegmann: lbragstad: are you saying that we loose client backwards compatibility with the RBAC, and not onl to previous version but within the version depending of the config in deployment? :o15:03
gmannjokke: by default no. enforce_scope if default to false and if anyone enable they get new policy15:04
gmannby default old token keep (must keep :)) working15:05
jokkeI mean if I have 2 clouds running wallaby, one with old policies, one with new policies, I need to have different clients to talk to those clouds?15:06
lbragstadclients, no?15:06
lbragstadpermissions, yes15:07
lbragstadand that depends on the usecase15:07
jokkelbragstad: this was what caused th stroke: 15:56 < lbragstad> i imagine we're going to hit a few places where we need to update the tests to use the correct client15:07
lbragstadfor the most part, project-member behavior isn't going to change a whole lot15:07
lbragstadjokke yeah - so there i meant the tempest client15:08
lbragstadwhich are backed by credentials in tempest15:08
jokkeah, gotcha15:08
lbragstadso - i was implying we may need to update the clients, and therefor the credentials, for a test to do something15:08
jokkeyeap, I intermangling your and gmann's lines together and somehow read that like it needed to happen in devstack, aka reflecting like everyone15:10
lbragstadmakes sense15:10
jokkeno it really didn't :P15:11
jokkelol15:11
lbragstadwell - i can see how you arrived at that conclusion given the context15:11
lbragstadi have some questions about the metadef API, is that an API designed for end users?15:11
jokkeyeap15:12
jokkekind of15:12
lbragstadlike, should project-members use it?15:12
jokkeI think so. Depending of the namespace15:12
lbragstaddoes depending on the namespace mean depending on if the namespace is public?15:12
jokkelike lots of the stuff that gets injected there by default are stuff to make life easier between glance - [nova, conder]15:13
jokkecinder15:13
jokkebut the whole idea of the api being available is to make similar correlations available for orchestration to pick right images for the stack, etc.15:14
jokkeSo you might have metadefs that helps Nova to pick correct host for the image, but you might have metadefs that helps thing X to pick correct image for say LAMP stack heat is spinning up15:15
*** jmlowe has joined #openstack-glance15:15
jokkeHonestly, I really don't understand the full extent of it, but IIUC it's more formal way of defining and discovering metadata15:17
jokkeand is used quite a bit around15:17
lbragstadok - interesting...15:17
lbragstadi was playing it with and noticied it's possible to fish private namespace existence out of that API15:18
lbragstadand i'm able to fish resource types associates to private namespaces out of it - even if i don't have role assignment on the project that owns the private namespace15:18
lbragstadand i'm wondering if that's by design?15:18
dansmithbeing able to test that a namespace exists by name (which I think you showed) seems like a huge deal15:19
dansmithlike if we leaked instance names between coke and pepsi, that could have information encoded in the name that would be a problem15:20
lbragstadand the relationship between metadef objects is pretty nested...15:22
lbragstadyou can have namespaces -> resource types -> objects -> properties -> tags15:22
lbragstadand i got to namespaces -> resource types in my testing15:23
lbragstaddansmith yeah - that was my reaction, too15:24
dansmith...yeah :/15:24
lbragstadmy concern is testing the object (and nested object) permutations - because there seems like a lot15:24
jokkelets stop the discussion right here, please open sec bug of it, so we can have a look and decide correct action forward15:25
lbragstadso i wanted to talk about this with the group before i went down a 4k loc binge15:25
jokkebetter to have sec bug that turns out to be "nothing" than discuss actual vulnerability out in open15:26
dansmithI'm not too overly concerned, TBH given how long this has probably been in place, but doing the paperwork is fine15:29
dansmithwe didn't hide the discussion of disclosing image existence a week ago, so..15:30
dansmithif the name is not scoped per project anyway, does that mean two people in separate tenants can't have a metadef (or whatever) of the same name?15:32
jokkehttps://specs.openstack.org/openstack/glance-specs/specs/juno/metadata-schema-catalog.html is the spec for it15:37
dansmithack, will read15:37
lbragstadi'm checking that now15:38
lbragstadbut i'm not sure yet - i'm still poking the metadef api15:38
*** ratailor has quit IRC15:48
*** lpetrut__ has quit IRC15:51
dansmithabhishekk: left you a suggestion on the client patch15:59
abhishekkdansmith, ack, looking16:00
dansmithabhishekk: it's really nice to see that import task status :)16:00
abhishekk:D16:00
abhishekkmakes sense16:00
abhishekkdansmith, I think in case of verbose also we should avoid showing image_id16:05
dansmithabhishekk: that's fine, but.. why? you're never running this on more than one image at a time are you?16:06
dansmithbut verbose is verbose, I'm fine with being VERY verbose there :P16:06
abhishekk:D16:07
openstackgerritLuigi Toscano proposed openstack/glance_store stable/stein: zuul: glance_store-src-ceph-tempest replaces a legacy job  https://review.opendev.org/c/openstack/glance_store/+/77761516:09
openstackgerritMerged openstack/glance master: Expand tasks database table to add more columns  https://review.opendev.org/c/openstack/glance/+/76373916:10
abhishekk\o/\o/ finally16:10
dansmithyay16:11
dansmithI think this is the "monitor for rechecks" query: https://review.opendev.org/q/project:openstack/glance+label:Workflow%253D%252B1+status:open16:20
*** jdillaman has quit IRC16:25
abhishekkCool, thank you dansmith16:37
dansmithat least in nova, it's about 5h to start running jobs at the moment, so we still have a bit before the bottom RBAC patch will even start in check16:41
openstackgerritAbhishek Kekane proposed openstack/python-glanceclient master: Get tasks assoiciated with image  https://review.opendev.org/c/openstack/python-glanceclient/+/77640316:46
abhishekkack16:47
lbragstadjokke are resource types supposed to be cleaned up when the namespace is deleted?16:47
abhishekkdansmith, that was quick16:48
dansmithabhishekk: same to you :)16:48
abhishekk:D16:48
abhishekkI am leaving for the day, will keep eye on rechecks over the weekend16:49
abhishekkgood night all16:49
abhishekko/~16:49
dansmithIf I'm not back on monday, this is why: https://www.weather.gov/pqr/16:49
dansmithabhishekk: o/16:49
abhishekk:o, stay safe16:50
lbragstado/ abhishekk16:51
abhishekko/~16:51
lbragstadjokke specifically - this16:51
lbragstadhttp://paste.openstack.org/show/803011/16:51
lbragstadis there a different way to forcibly clean up resource types?16:55
dansmithlbragstad: the resource type is created separate from the namespace and presumably could be included in multiple ones right?16:55
abhishekkdansmith, may be I need to add more documentation about command line for task-show API16:55
abhishekkwill do that on Monday16:55
dansmithso cleaning up RTs when NSs are deleted would be weird?16:55
dansmithabhishekk: okay, as add-on? I can only think of a couple other sentences to add really16:56
lbragstadmaybe?16:57
lbragstadhttps://docs.openstack.org/api-ref/image/v2/metadefs-index.html?expanded=create-property-detail,create-tag-definition-detail,list-resource-types-detail,create-resource-type-association-detail,list-namespaces-detail,remove-resource-type-association-detail#remove-resource-type-association16:57
abhishekknot api-ref16:57
lbragstadi was looking at that and it's nested under the namespace16:57
abhishekkbut I need to explain how command works and what --verbose will show with example somewhere16:57
lbragstadbut that's for association, so i suppose it could be shared, but then were is the API to delete the resource type?16:57
dansmithabhishekk: yeah I meant for the shell, but okay16:57
abhishekkLet me see what I can do or how I can do it16:58
dansmithlbragstad: the unable-to-delete part definitely seems wrong16:58
dansmithlikely a leaked reference or something16:58
abhishekkas it will be a doc change it will be quick to merge16:58
lbragstadbased on what i see and was able to recreate - a regular end users can just create these things, but an admin can't clean them up16:59
lbragstadend user*16:59
dansmithniice16:59
lbragstadlooks similar to this17:02
lbragstadhttps://bugs.launchpad.net/glance/+bug/154570217:02
openstackLaunchpad bug 1545702 in Glance "Images v2 api metadef vulnerability" [Undecided,New]17:02
dansmithheh17:04
dansmithI think the cat may have exited the bag17:04
lbragstadjokke do you know who uses the metadef API?17:19
*** udesale_ has quit IRC17:25
*** k_mouza has quit IRC18:05
*** happyhemant has quit IRC18:08
jokkelbragstad: I think there is just association between resource type and namespace but no hierarchical relation ... but that's just how I understood the spec18:16
lbragstadjokke is there an API somewhere to clean up resource types?18:17
jokkelbragstad: I think I need to test that, but by the quick look it might be that the associate actually creates the resource type and deassociate removes it. Being basically the bridge between the propertices and namespaces18:23
openstackgerritDan Smith proposed openstack/glance master: Add a test for migration naming and phase rules  https://review.opendev.org/c/openstack/glance/+/77741318:24
jokkelbragstad: this based on crossing the spec info and client info18:24
lbragstadi read the API docs and i see the associate and disassociate APIs18:24
lbragstadbut i don't think disassociate actually deletes the resource types18:25
lbragstadtype818:25
lbragstadtype*18:25
dansmithI've also confirmed the bug linked above18:26
dansmithwhere namespaces and associations appear to be unbounded, even for normal users18:26
jokkelbragstad: the client deassociate is actually API DELETE call18:26
lbragstadright18:27
lbragstadit deletes the association, it doesn't delete the resource type18:27
lbragstadfrom what i can tell18:27
lbragstadso i'm wondering if i'm just missing something18:28
jokkeok, like said I'd need to dig into that as I'm not super familiar how the metadefs work18:28
jokkeI just know that it's kind of glue between the services to make meadata discovery and usage easier18:28
jokkeAnd we get handful of patches every cycle people updating the default definitions based on changs that happens in Nova for example. So people do care about them working and being correct18:29
lbragstadjokke do you have a link to a patch for one of those, or where i could find that definition update?18:31
jokkehttps://review.opendev.org/c/openstack/glance/+/74038418:37
dansmithfrom a redhat person, so apparently *we* care :)18:38
*** ralonsoh has quit IRC19:24
*** zzzeek has quit IRC19:29
*** zzzeek has joined #openstack-glance19:29
*** jdillaman has joined #openstack-glance19:47
*** ajitha has quit IRC20:25
*** lbragstad_ has joined #openstack-glance20:35
*** mugsie_ has joined #openstack-glance20:36
*** benj_- has joined #openstack-glance20:36
*** zigo_ has joined #openstack-glance20:36
*** aarents has quit IRC20:36
*** benj_ has quit IRC20:36
*** zigo has quit IRC20:36
*** stephenfin has quit IRC20:36
*** BLZbubba has quit IRC20:36
*** melwitt has quit IRC20:36
*** gregwork has quit IRC20:36
*** mugsie has quit IRC20:36
*** melwitt has joined #openstack-glance20:36
*** benj_- is now known as benj_20:37
*** jmccrory_ has joined #openstack-glance20:37
*** tosky_ has joined #openstack-glance20:37
*** jmccrory has quit IRC20:37
*** jmccrory_ is now known as jmccrory20:38
*** felixhuettner[m] has quit IRC20:39
*** zzzeek has quit IRC20:40
*** jrosser has quit IRC20:40
*** tosky has quit IRC20:40
*** trident has quit IRC20:40
*** ricolin has quit IRC20:40
*** jrosser has joined #openstack-glance20:40
*** jokke has quit IRC20:40
*** zzzeek has joined #openstack-glance20:41
*** tosky_ is now known as tosky20:42
*** lifeless_ has joined #openstack-glance20:42
*** fnordahl has quit IRC20:42
*** trident has joined #openstack-glance20:42
*** dasp has quit IRC20:43
*** lifeless has quit IRC20:43
*** lbragstad has quit IRC20:43
*** dasp has joined #openstack-glance20:45
openstackgerritMerged openstack/glance master: Properly handle InvalidScope exceptions  https://review.opendev.org/c/openstack/glance/+/77430920:48
dansmithrosmaita: smcginnis: I just had to fix a pep8 fail on the latest version of this, which was +W before, if someone could re-ack it for me: https://review.opendev.org/c/openstack/glance/+/77741321:04
rosmaitadansmith: in a meeting, will look quickly21:07
rosmaitai mean later21:07
dansmithrosmaita: ack thanks21:10
*** rcernin has quit IRC21:12
openstackgerritLance Bragstad proposed openstack/glance-tempest-plugin master: WIP: Add protection testing for namespace objects  https://review.opendev.org/c/openstack/glance-tempest-plugin/+/77678921:12
*** felixhuettner[m] has joined #openstack-glance21:17
*** k_mouza has joined #openstack-glance21:25
*** k_mouza has quit IRC21:31
dansmithrosmaita: thankyasir21:38
rosmaitanp21:38
*** rcernin has joined #openstack-glance22:09
*** rcernin has quit IRC22:15
*** rcernin has joined #openstack-glance22:15
openstackgerritLance Bragstad proposed openstack/glance master: Update default policies for task API  https://review.opendev.org/c/openstack/glance/+/76320822:26
lbragstad_dansmith abhishekk quick stab at updating the task API to explicitly call out it's admin only for the time-being22:26
*** lbragstad_ is now known as lbragstad22:26
lbragstadi don't see a tempest client for the tasks API - so we might need to add one of those if we want to test this in the protection job22:27
lbragstadalso - i noticed the tasks_api_access pretty much protects that API22:28
lbragstadthe rest of the policies are there, but they don't do much by default because they default to open22:29
lbragstadso - in the worst case, an operator could configure it to be different from the tasks_api_access policy and notice weird behavior22:29
lbragstadand the modify_task policy isn't reachable i don't think22:30
lbragstadanywho - i put all that in the review22:30
dansmithack yeah, will have to look at that next week22:31
lbragstad++22:32
*** tkajinam has joined #openstack-glance22:58
dansmithall the pending stuff is in the gate and not failing at the moment23:23
* dansmith holds breath23:23
*** k_mouza has joined #openstack-glance23:33
*** k_mouza has quit IRC23:34

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!