| openstackgerrit | Rick Aulino proposed openstack/searchlight: Zero Downtime Re-indexing changes. https://review.openstack.org/277860 | 00:08 |
|---|---|---|
| TravT | sjmc7: for your keystone v3 patch | 00:08 |
| sjmc7 | the whoosiwhat now? | 00:08 |
| TravT | what do i need to do to test it properly? | 00:09 |
| sjmc7 | oh, the devstack one? | 00:09 |
| sjmc7 | i think it’ll just use v3 now | 00:09 |
| TravT | so, if i just restack after pulling it and reclone = no? | 00:10 |
| sjmc7 | yeah, that should do it. check the config file it generates (and that it works) | 00:10 |
| sjmc7 | the devstack change that broke things was disabling the v2 endpoint | 00:10 |
| sjmc7 | but it shouldn’t use it at all now | 00:11 |
| *** RickA-HP has joined #openstack-searchlight | 00:26 | |
| *** sjmc7 has quit IRC | 00:49 | |
| *** lakshmiS_ has joined #openstack-searchlight | 01:06 | |
| *** lakshmiS has quit IRC | 01:09 | |
| *** lakshmiS_ has quit IRC | 01:23 | |
| *** itisha has quit IRC | 01:26 | |
| *** bpokorny has quit IRC | 02:10 | |
| *** RickA-HP has quit IRC | 02:50 | |
| *** sjmc7 has joined #openstack-searchlight | 04:02 | |
| *** sjmc7 has quit IRC | 04:16 | |
| *** openstackgerrit has quit IRC | 07:47 | |
| *** openstackgerrit has joined #openstack-searchlight | 07:47 | |
| *** itisha has joined #openstack-searchlight | 08:03 | |
| *** krotscheck_dcm is now known as krotscheck | 12:38 | |
| *** itisha has quit IRC | 13:16 | |
| krotscheck | Any cores around to perhaps add another +2 to https://review.openstack.org/#/c/265396/ ? | 14:20 |
| *** sjmc7 has joined #openstack-searchlight | 15:04 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 15:19 | |
| TravT | krotscheck: I think sjmc7 was going to add his vote onto that one | 15:43 |
| sjmc7 | yeah, will do | 15:43 |
| TravT | sjmc7, krotscheck was asking about https://review.openstack.org/#/c/265396/ | 15:43 |
| sjmc7 | approved | 15:43 |
| sjmc7 | TravT: take a look at the bottom of https://www.elastic.co/blog/elasticsearch-versioning-support | 15:43 |
| sjmc7 | i think i may have done some work for nothing :) | 15:43 |
| TravT | ok | 15:43 |
| TravT | is this what rosmaita was asking about in the spec reviews? | 15:45 |
| sjmc7 | i don’t think so | 15:46 |
| TravT | so.... | 15:46 |
| TravT | with the versioning spec | 15:46 |
| sjmc7 | well, obviously i didn’t think so because i didn’t know about this, so maybe. ‘garbage collection’ means many things | 15:46 |
| sjmc7 | i think we don’t need the deletion work | 15:46 |
| TravT | in theory, all we have to do is adjust the gc_deletes time | 15:46 |
| sjmc7 | right | 15:46 |
| sjmc7 | i’ll test that today | 15:46 |
| sjmc7 | i seem to be awesome at finding information long after it’d be useful | 15:47 |
| TravT | well, glad you caught that now. | 15:47 |
| sjmc7 | i need a time machine | 15:47 |
| TravT | makes our code simpler | 15:47 |
| TravT | better make sure that didn't get deprecated in 2.0 though | 15:47 |
| TravT | ES seems to like to deprecate useful things | 15:47 |
| sjmc7 | yeah, i am | 15:47 |
| sjmc7 | haha, they usually have reasons for doign so | 15:48 |
| TravT | usually perf related i think | 15:48 |
| TravT | ok, i'm going to go eat a bite of breakfast. be back shortly | 15:49 |
| openstackgerrit | Merged openstack/searchlight: Added Keystone and RequestID headers to CORS middleware https://review.openstack.org/265396 | 15:55 |
| sjmc7 | looks like it is supported in 2.0. it doesn’t appear to be documented anywhere. it is mentioned in the migration tool as being a field that needs to have time units | 16:14 |
| openstackgerrit | Rick Aulino proposed openstack/searchlight: Zero Downtime Re-indexing changes. https://review.openstack.org/277860 | 16:20 |
| sjmc7 | TravT: i’m pretty sure that’s the way to go. i guess we need to modify the spec - is it ok to submit a patch to the existing spec document? i’ll move any existing implementation details into the alternatives section | 16:37 |
| TravT | Yeah, it is perfectly reasonable to amend a spec to bring it up to date. | 16:38 |
| sjmc7 | ok. i’ll do another patch based of the versioning one (which we also need to try to close on) | 16:39 |
| sjmc7 | lei had another question about image members; i don’t think we have access to the new updated_at time for the image | 16:40 |
| TravT | i just asked back in that patch... | 16:40 |
| TravT | is the image update date actually updated with membership changes? | 16:40 |
| TravT | or does it only change with properties changes | 16:40 |
| sjmc7 | i haven’t checked yet | 16:41 |
| TravT | in glance i mean. | 16:41 |
| *** bpokorny has joined #openstack-searchlight | 16:41 | |
| TravT | i had a script for adding members and seem to have lost it | 16:41 |
| TravT | it is a two step process | 16:42 |
| TravT | so there may be two opportunities for the time to change | 16:42 |
| sjmc7 | lovely | 16:42 |
| TravT | 1) when you add a member | 16:42 |
| TravT | 2) when the member accepts it | 16:42 |
| TravT | not too hard. | 16:42 |
| TravT | i guess i could test it right now | 16:42 |
| sjmc7 | i’ll do it | 16:42 |
| TravT | okey-dokey | 16:43 |
| sjmc7 | TravT: looks like the image’s updated_at doesn’t change | 16:50 |
| sjmc7 | but the member notification comes with an updated_at | 16:50 |
| sjmc7 | so i think we can use that | 16:50 |
| sjmc7 | for the versioning, at least | 16:50 |
| TravT | for the version | 16:50 |
| TravT | but we don't want to update the glance image / snapshot version | 16:50 |
| sjmc7 | we’ll have so many edge conditions we’ll be able to make nice geometric shapes | 16:50 |
| TravT | version --> updated_at | 16:51 |
| TravT | urghh | 16:51 |
| sjmc7 | right | 16:51 |
| sjmc7 | the version’ll be slightly different from updated_at | 16:51 |
| sjmc7 | and i think that’s ok | 16:51 |
| TravT | yeah, i think it is okay | 16:51 |
| sjmc7 | although… if an image update comes out of sync with a member one it’ll cause trouble | 16:51 |
| TravT | hmmm | 16:51 |
| sjmc7 | i don’t see a good way around this short of requesting that image member updates also affect the image timestamps | 16:52 |
| TravT | so then the version should be based on the image updated_at | 16:52 |
| sjmc7 | and even then i guess that doesn’t really help | 16:52 |
| sjmc7 | yeah.. maybe we can script image member updates | 16:53 |
| sjmc7 | i don’t *think* they work with versioning | 16:53 |
| sjmc7 | also an image update refreshes the full member list | 16:53 |
| TravT | i'm almost inclined to think that member updates should just take whatever last version is and increment by 1 | 16:53 |
| sjmc7 | could also keep the same timestamp | 16:54 |
| TravT | yeah, was just trying to think about that... | 16:54 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 16:55 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 16:56 | |
| sjmc7 | i don’t think there’s a way to deal with this correctly in every condition. our model’s different from glance's | 16:56 |
| TravT | i'm wondering if we'll hit other cases like this | 16:56 |
| sjmc7 | undoubtedly | 16:58 |
| TravT | if we normalized docs (flavors into servers) this could also present issues | 17:00 |
| TravT | de- | 17:00 |
| sjmc7 | yep | 17:01 |
| sjmc7 | maybe parent-child is better for all these kinds of things | 17:01 |
| sjmc7 | i’m not sure what the best answer is for now. i’m inclined to alter the image member notifications to run a scripted update and use the existing image update time | 17:08 |
| sjmc7 | as the version | 17:08 |
| TravT | i'm just running debugger on code and looking at the notification | 17:11 |
| sjmc7 | it has updated_at and created_at but for the image | 17:11 |
| sjmc7 | sorry, for the member | 17:11 |
| TravT | maybe child for member would be better | 17:11 |
| sjmc7 | urgh. yeah, maybe | 17:11 |
| TravT | it would actually be faster, i think. | 17:11 |
| TravT | wouldn't require a retrieval | 17:11 |
| TravT | slower search | 17:12 |
| sjmc7 | probably not significantly | 17:12 |
| sjmc7 | and speed is less important than correct for us | 17:12 |
| TravT | but i'm trying to think about the rammifications on queries | 17:12 |
| sjmc7 | https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-has-child-query.html | 17:13 |
| TravT | oh yeah | 17:13 |
| TravT | i run that quite a bit | 17:13 |
| sjmc7 | weirdo | 17:13 |
| sjmc7 | ok. for now, what can we do to unblock lei’s patch? | 17:13 |
| sjmc7 | and consider switching members to parent/child separately | 17:13 |
| TravT | well, i have to do things like point out that grandchildren aren't handled in code | 17:13 |
| TravT | :P | 17:13 |
| sjmc7 | weirdo! | 17:13 |
| TravT | when I say queries, I mean this | 17:14 |
| sjmc7 | i’m inclined to go with ignoring versioning for image member updates | 17:14 |
| TravT | already with designate | 17:14 |
| TravT | there are two different resource types | 17:14 |
| TravT | and so in the search panel right now it isn't handling them nicely | 17:14 |
| TravT | partially because the ui is generic and it has no way to know about relationships ahead of time... | 17:15 |
| sjmc7 | yes. it complicates things a lot on that end | 17:15 |
| TravT | and i've been trying to figure out (haven't put much thought into it) how to handle that better | 17:15 |
| TravT | maybe our plugins API endpoint should list parent / children | 17:15 |
| TravT | but with things like member | 17:16 |
| TravT | what are the access rights to that? | 17:16 |
| TravT | or if i search on an image id, will i get membership records... maybe that's desired, maybe not... probably based on context. | 17:16 |
| sjmc7 | maybe we don’t expose members as a thing | 17:16 |
| sjmc7 | yeah, i don’t know. denormalizing is vastly preferable for querying | 17:17 |
| TravT | hmm... another thing, who is allowed to see members? | 17:20 |
| TravT | just the image owner? | 17:20 |
| sjmc7 | currently? | 17:20 |
| TravT | yeah | 17:20 |
| sjmc7 | anyone who can see the image i think | 17:20 |
| TravT | i just tried it locally and realized my local debugger sent stuff into local elastic search and my API on VM picked up requests... | 17:20 |
| TravT | got a little lost | 17:20 |
| TravT | but that might be another case for members to be their own thing | 17:21 |
| sjmc7 | we maybe need to post-filter the member list. or not expose it at all | 17:21 |
| sjmc7 | ok. for now, what do we suggest lei do? | 17:21 |
| TravT | only owner and admins see them... | 17:21 |
| TravT | run away screaming | 17:21 |
| TravT | :D | 17:21 |
| sjmc7 | i’m starting to feel that way | 17:21 |
| TravT | i'm pretty sure parent child would solve these technical issues on server side. | 17:22 |
| TravT | but not sure on API side | 17:23 |
| TravT | need to look at code | 17:23 |
| TravT | API side should show image if owner, public, or if requester in member list / accepted | 17:23 |
| sjmc7 | maybe we don’t expose that at all on the API side; it bcomes something you go to the glance api for | 17:24 |
| *** lakshmiS has joined #openstack-searchlight | 17:24 | |
| sjmc7 | there may be some cases it doesn’t make sense for us to deal with | 17:24 |
| sjmc7 | but again: what do we have lei do with the versioning patch? | 17:24 |
| TravT | members are already handled in rbac | 17:24 |
| sjmc7 | if we wait until we solve this, it won’t merge | 17:24 |
| sjmc7 | because i don’t want to rush a patch in the next day or two | 17:25 |
| * TravT looking through code | 17:25 | |
| TravT | _get_rbac_field_filters(self, request_context): | 17:25 |
| sjmc7 | my suggestion is when a member update is received, we always make the change, since ordering is not important, with a scripted update | 17:25 |
| TravT | handles part of it... | 17:25 |
| TravT | so right now, image just has members field | 17:28 |
| TravT | "members": [ | 17:28 |
| TravT | "ed6eb9c5d34441c280fc11f7367d2227" | 17:28 |
| TravT | ], | 17:28 |
| sjmc7 | yes | 17:28 |
| TravT | what if members actually had list of them as nested object but also had updated | 17:29 |
| sjmc7 | i don’t know | 17:29 |
| sjmc7 | what would updated matter? | 17:29 |
| TravT | so are you saying with scripted update that we only update member list? | 17:31 |
| sjmc7 | yeah | 17:31 |
| TravT | i was still thinking that you could still hit a timing issue with that. | 17:31 |
| sjmc7 | you might if you were adding and removing a particular member quickly and happened to manage to get out of date notifications, i suppose | 17:32 |
| TravT | i agree that member syncs should just update member fields. | 17:33 |
| sjmc7 | adding a date won’t really help though; deletions will be hard to track | 17:33 |
| sjmc7 | we’d need to keep deleted members around. i think it really requires some more thought, and i don’t want to hold up the versioning stuff on this | 17:33 |
| TravT | ok, i'd be okay with changing to scripted update and have a follow on to see if this needs more done just for image members | 17:34 |
| sjmc7 | the versioning patch already needs a rebase; maybe i can get something out today that changes it so it’s a separate change | 17:35 |
| TravT | i'll BRB, have to go do something. | 17:37 |
| *** itisha has joined #openstack-searchlight | 17:55 | |
| openstackgerrit | Steve McLellan proposed openstack/searchlight-specs: Update deletion spec to use elasticsearch GC https://review.openstack.org/278554 | 18:16 |
| openstackgerrit | Travis Tripp proposed openstack/searchlight: Add sample devstack local.conf https://review.openstack.org/278558 | 18:32 |
| openstackgerrit | Travis Tripp proposed openstack/searchlight: Add sample devstack local.conf https://review.openstack.org/278558 | 18:34 |
| *** sjmc7 has quit IRC | 18:34 | |
| *** TravT_ has joined #openstack-searchlight | 18:58 | |
| *** bpokorny_ has joined #openstack-searchlight | 18:59 | |
| *** bpokorny_ has quit IRC | 18:59 | |
| *** bpokorny_ has joined #openstack-searchlight | 19:00 | |
| *** TravT has quit IRC | 19:00 | |
| *** bpokorny has quit IRC | 19:02 | |
| *** sjmc7 has joined #openstack-searchlight | 19:03 | |
| *** TravT_ is now known as TravT | 19:16 | |
| *** bpokorny_ has quit IRC | 19:17 | |
| *** bpokorny has joined #openstack-searchlight | 19:18 | |
| *** krotscheck is now known as krotscheck_dcm | 19:47 | |
| *** briancline has quit IRC | 21:48 | |
| *** dhellmann has quit IRC | 21:48 | |
| *** dhellmann has joined #openstack-searchlight | 21:49 | |
| *** bpokorny has quit IRC | 21:56 | |
| *** TravT has quit IRC | 22:10 | |
| *** lakshmiS_ has joined #openstack-searchlight | 22:15 | |
| *** dhellmann has quit IRC | 22:22 | |
| *** lakshmiS has quit IRC | 22:22 | |
| *** dhellmann has joined #openstack-searchlight | 22:23 | |
| *** TravT has joined #openstack-searchlight | 22:29 | |
| *** lakshmiS_ has quit IRC | 22:31 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:53 | |
| *** bpokorny has joined #openstack-searchlight | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!