| *** jamesmcarthur has quit IRC | 00:22 | |
| *** jamesmcarthur has joined #storyboard | 00:23 | |
| *** jamesmcarthur has quit IRC | 00:27 | |
| *** whoami-rajat has joined #storyboard | 02:32 | |
| *** ivoline has joined #storyboard | 02:32 | |
| *** jamesmcarthur has joined #storyboard | 02:37 | |
| *** ivoline has quit IRC | 03:16 | |
| *** jamesmcarthur has quit IRC | 03:21 | |
| *** jamesmcarthur has joined #storyboard | 03:22 | |
| *** jamesmcarthur has quit IRC | 03:27 | |
| *** jamesmcarthur has joined #storyboard | 03:44 | |
| *** ivoline has joined #storyboard | 03:51 | |
| *** udesale has joined #storyboard | 03:56 | |
| *** jamesmcarthur has quit IRC | 03:59 | |
| *** jamesmcarthur has joined #storyboard | 04:00 | |
| *** jamesmcarthur has quit IRC | 04:05 | |
| *** jamesmcarthur has joined #storyboard | 04:09 | |
| *** jamesmcarthur has quit IRC | 04:22 | |
| *** jamesmcarthur has joined #storyboard | 04:25 | |
| *** jamesmcarthur has quit IRC | 04:30 | |
| *** jamesmcarthur has joined #storyboard | 04:53 | |
| *** diablo_rojo has quit IRC | 04:55 | |
| *** jamesmcarthur has quit IRC | 04:59 | |
| *** ivoline has quit IRC | 05:31 | |
| *** jtomasek has joined #storyboard | 06:29 | |
| *** dangtrinhnt has quit IRC | 07:01 | |
| *** jamesmcarthur has joined #storyboard | 07:18 | |
| *** jamesmcarthur has quit IRC | 07:22 | |
| *** dangtrinhnt has joined #storyboard | 07:26 | |
| *** udesale has quit IRC | 07:34 | |
| *** udesale has joined #storyboard | 07:35 | |
| *** jpich has joined #storyboard | 08:24 | |
| SotK | mkarray: the webclient currently tries to hide the complexities of search/browse from the user | 08:30 |
|---|---|---|
| SotK | when you start typing something in the search box a dropdown will appear with a bunch of criteria in it | 08:31 |
| SotK | the first item (which has a magnifying glass icon) is a text search term which will add a "q" parameter and make the webclient do a "search" rather than a "browse" | 08:32 |
| SotK | the other items are all browse parameters; if you start typing a name you can select a user from the dropdown and it'll add an "assignee_id" parameter | 08:33 |
| *** dtantsur|afk is now known as dtantsur | 08:34 | |
| SotK | sadly there's no way yet to explicitly set parameters using a fancy search language like (for example) Gerrit provides | 08:34 |
| *** tosky has joined #storyboard | 08:43 | |
| *** ianychoi has quit IRC | 10:59 | |
| *** udesale has quit IRC | 11:14 | |
| *** jamesmcarthur has joined #storyboard | 13:18 | |
| *** jamesmcarthur has quit IRC | 13:22 | |
| *** jamesmcarthur has joined #storyboard | 13:22 | |
| *** ianychoi has joined #storyboard | 13:22 | |
| *** jamesmcarthur has quit IRC | 13:33 | |
| *** jamesmcarthur has joined #storyboard | 13:51 | |
| *** jamesmcarthur has quit IRC | 13:51 | |
| *** jamesmcarthur has joined #storyboard | 13:52 | |
| *** jtomasek has quit IRC | 14:02 | |
| *** jtomasek has joined #storyboard | 14:04 | |
| *** mkarray has quit IRC | 14:12 | |
| *** mkarray has joined #storyboard | 14:14 | |
| mkarray | Without looking too deeply into the code yet, I know a dictionary for all the browsable queries already exist. Would it be too much trouble to have the search check to see if the user entered something that could be browsed instead? | 14:47 |
| mkarray | Sotk ^ | 14:47 |
| mkarray | That way a user could enter those fancy queries on their own without making manual api calls | 14:49 |
| fungi | as i mentioned earlier, it probably wouldn't be too hard with a separate input widget or a checkbox to bypass the typeahead stuff and forward literal queries to the api | 14:50 |
| fungi | but that's probably not a great user experience | 14:50 |
| mkarray | I was thinking more of an "if, else" situation. If the user used one of the browsable queries, there's no need to also do a search, otherwise carry on with a normal search | 14:52 |
| fungi | yeah, that however would require some additional parsing for the input box | 14:52 |
| mkarray | If I recall correctly, the browse already checks to make sure it was passed valid criteria. So we can take advantage of that | 14:52 |
| fungi | i guess that's more like a "soft" toggle | 14:52 |
| *** jamesmcarthur has quit IRC | 14:53 | |
| mkarray | Yea regardless there will be some sort of parsing | 14:53 |
| SotK | yeah it could be interesting as a first step to better search do some parsing of whatever is entered as a "Text" criteria (the one that becomes the q parameter) and determine if its actually a manually entered browse parameter | 14:54 |
| mkarray | I suppose we can expand on this further in the meeting today, this is sort of an extension of our convo yesterdat | 14:54 |
| fungi | one up side though, we don't really need to worry about backwards-compatibility for the search widget itself (it's the urls which are the sticky wicket), and as long as we retain the old search api method and old parameter name in the webclient, creating a new search language under a different method in the api with a more robust parser can be a fairly compartmentalized effort | 14:54 |
| fungi | and then the webclient can be changed over to hook the search widget up to that in the future and just retain support for the old url pattern for backwards compatibility | 14:56 |
| fungi | but yeah, good topic for the meeting, for sure | 14:56 |
| SotK | I'd like to get to something like Gerrit's search box eventually, where typing suggests various human-readable things based on what you've typed, but also has a real syntax rather than just applying simple filters | 14:57 |
| SotK | so for example, typing "creator" causes some suggestions like "creator: Adam Coldrick" and "text: 'creator'" and so on | 14:58 |
| fungi | yeah, like if we decided on lucene (just an example), we can in theory use a standard parser library in python for the api piece and a standard javascript library for the syntax suggestions in the webclient | 14:59 |
| fungi | so there are benefits to choosing a popular query language rather than inventing our own | 14:59 |
| fungi | obviously the tokens would still need to be looked up (we could either embed those client-side or query the api to get them) | 15:00 |
| mkarray | Another option that might be a bit more hacky, but also easier would be simply parsing the url resulting from a search. That way everything stays the same from the perspective of the web-client. | 15:14 |
| persia | Personally, I'd prefer to go directly for the complex model without trying to guess what the user meant. I can think of lots of reasons I might want to search for "Storyboard" without referencing the project, the project group, any tags that might show up, etc. I'm much less certain I can think of any reason to search for the string "project:openstack-infra/storyboard". | 16:54 |
| *** dtantsur is now known as dtantsur|afk | 17:14 | |
| *** jpich has quit IRC | 17:25 | |
| *** jaosorior has quit IRC | 17:49 | |
| *** jaosorior has joined #storyboard | 17:49 | |
| *** ivoline has joined #storyboard | 17:59 | |
| *** diablo_rojo has joined #storyboard | 18:01 | |
| diablo_rojo | Suuuuuper late to the conversation, but I think a lot of the previous complaints about the search is just that they don't understand how it works. If we could use similar syntax to gerrit I think that would make things a lot easier on people? That doesn't solve all the issues that have been discussed here recently, but I think it would go a long way to not scaring folks away | 18:09 |
| mkarray | persia From the convos we've had I don't think the intention is to completely scrap "guessing" the users intention, but rather to add more functionality for those who want to be more exact | 18:14 |
| persia | mkarray: My apologies. I've failed to write clearly. I don't think it is correct to perform a browse on "project:openstack-infra/storyboard" because someone typed "storyboard" if not selected from the list. I'd be delighted to see support for typing "project:openstack-infra/storyboard" and getting a browse. | 18:15 |
| SotK | +1 | 18:16 |
| mkarray | Agreed | 18:16 |
| persia | diablo_rojo: Yes, although the search/browse divide also helps confuse things :) Gerrit has a similar divide, but it masks both in the same input, so only searches for terms it cannot parse (which I think is correct), so feels fairly natural. | 18:16 |
| SotK | the bit I don't want to lose is only needing to start typing `project` or `storyboard` or whatever in order for the UI to suggest that | 18:17 |
| persia | SotK: Would you be comfortable if the suggestions were of the form "project:openstack-infra/storyboard" rather than the special icon notation currently used? | 18:17 |
| persia | Or "tag:low-hanging-fruit"? | 18:18 |
| SotK | absolutely | 18:18 |
| persia | Then I think it is likely possible to eat cake and have it too :) | 18:18 |
| SotK | I just don't want to have to type "project:openstack-infra/sto" when I currently just have to type "sto" | 18:18 |
| diablo_rojo | For those that want to type it out it should be possible, but it should also prompt that like gerrit does in my opinion :) | 18:19 |
| SotK | +1 | 18:19 |
| persia | Makes sense, although if one types "sto", one may have to wait a couple seconds to be able to select "project:openstack-infra/storyboard", right? | 18:19 |
| SotK | though Gerrit only prompts if you start with the "project" bit | 18:20 |
| persia | diablo_rojo: Yes. Gerrit has a terrible UI in many ways, but the search entry field isn't part of that. | 18:20 |
| SotK | hopefully less than a couple of seconds, but yes | 18:20 |
| diablo_rojo | persia, yeah probably. Ideally not, but its not an ideal world is it? | 18:20 |
| persia | especially for very broad searches on multiple categories (like "s", interrupt, "st", interrupt, "sto" against projects, project groups, people, tags, story titles, tasks, etc.) | 18:21 |
| SotK | currently it takes around a second for me, and I expect we can make that faster | 18:22 |
| diablo_rojo | SotK, the optimization of queries was my idea for an intern to work on. We can discuss that more in the meeting though | 18:23 |
| persia | diablo_rojo: By "optimisation of queries", do you mean backend improvements or UI changes? | 18:24 |
| diablo_rojo | backend improvements | 18:24 |
| persia | Oh, cool. Be nice to make that faster, although the solutions are probably very DB-specific. | 18:25 |
| diablo_rojo | Yeah I would guess so, but I think we taken a lot of other steps to make optimizations so this is the next logical place to clean things up | 18:30 |
| persia | yep | 18:31 |
| SotK | I suspect we could get somewhere just by indexing our database better | 18:31 |
| SotK | and yeah, numerous queries can probably be improved | 18:31 |
| mkarray | I would suggest adding the functionality we've been bringing up over the past couple of days before diving into optimisations though | 18:32 |
| SotK | I'll be a few minutes late for the meeting, sorry | 18:52 |
| * SotK looks forward to summertime | 18:52 | |
| ivoline | Hello guys, where and when is the meeting ? | 18:53 |
| SotK | in #openstack-meeting in a bit over 5 minutes from now | 18:55 |
| ivoline | Okay, thanks | 18:56 |
| persia | mkarray: I suspect they can be done in parallel. I'm not sure we've taken a good look at DB optimisations, and having someone investigate would help. I agree that changing the search/browse interface will probably change some queries, but I think the majority of queries are of the "get the properties of the item with this index" variety, which also aren't as fast as they might be. | 18:58 |
| fungi | ooh, i missed some good scrollback in here but it's meeting time | 19:00 |
| fungi | i'll try to catch up in parallel | 19:00 |
| diablo_rojo | fungi, we are waiting 5 min for SotK to get home from work I would guess? | 19:01 |
| fungi | oh, yep. plenty of time for me to read! ;) | 19:03 |
| fungi | mkarray: i think the suggestion about query optimizations is unrelated (or at least not directly related) to search functionality | 19:04 |
| mkarray | You guys would be more informed than me at this point. If that's the care I have no opposition to it | 19:04 |
| fungi | we have some very inefficient database queries right now which cause pulling up a board with tag-oriented lanes to call queries which make mysql eat an entire cpu for 10+ seconds before returning the necessary data to the client | 19:05 |
| mkarray | I was just afraid some optimisation fixes would go into something that might be changed by us | 19:05 |
| fungi | and yeah, most of that is almost certainly just a lack of strategic indexes on some columns | 19:05 |
| diablo_rojo | Yeah I think the optimizations are separate | 19:05 |
| diablo_rojo | than how users do the search | 19:05 |
| * SotK is ready, sorry about that | 19:13 | |
| diablo_rojo | Ready! | 19:13 |
| fungi | no worries! | 19:13 |
| SotK | -> #openstack-meeting | 19:13 |
| *** jaosorior has quit IRC | 19:18 | |
| *** jaosorior has joined #storyboard | 19:28 | |
| fungi | #openstack-meeting -> . | 19:59 |
| fungi | so anyway... | 20:00 |
| SotK | did anyone have anything more on the outreachy topic? | 20:00 |
| fungi | i do agree there's a balancing act for picking things we need done but which can be an interesting task an outreachy intern will want to select | 20:00 |
| diablo_rojo | fungi, did you want to be a co-mentor too? | 20:01 |
| fungi | want to, yes... whether i'll actually find time to be much help there is another matter | 20:01 |
| diablo_rojo | I can start drafting these up today and get them into the outreachy tool. I think I can start the application and save it without submitting | 20:01 |
| diablo_rojo | I'll probably be the primary again | 20:01 |
| diablo_rojo | and SotK will be secondary | 20:01 |
| fungi | i'd hate to say i'll co-mentor and then leave you holding the bag or fail to provide sufficient guidance to the mentee | 20:01 |
| diablo_rojo | if I can add a third, I will fungi | 20:02 |
| SotK | fungi: yeah... fixing the tests is creeping up on my list of personal pain points too | 20:02 |
| SotK | diablo_rojo: sounds good, thanks | 20:02 |
| diablo_rojo | Ha ha so... queries and tests? | 20:02 |
| *** jamesmcarthur has joined #storyboard | 20:02 | |
| fungi | well, tests as mentioned aren't likely to be an attractive solution | 20:02 |
| fungi | mkarray mentioned database optimizations sounded intruiging though, from his perspective as an intern | 20:03 |
| ivoline | The query optimization is quite attractive | 20:03 |
| mkarray | Personally wouldn't sign up for an internship focused on fixing tests to be quite honest | 20:04 |
| fungi | yeah, i think one of the challenges with the db query optimizing is we'll need representative test data | 20:04 |
| ivoline | As an intern, I'll go for that too. :) | 20:04 |
| fungi | fatema e-mailed me recently asking for a db dump to use for local testing... maybe scripting up something which can interface with the sb api to autogenerate useful test data might be something folks want/need too? | 20:05 |
| fungi | we could even build a reference set with it and make that available for download, as a convenience | 20:06 |
| SotK | yeah I think producing a reference dataset would be good | 20:06 |
| fungi | having one as a reference also makes reproducibility of performance tests easier | 20:06 |
| fungi | since everybody can do it from the exact same data | 20:07 |
| SotK | yep, it seems like a sensible starting point to any real optimisation work no matter who is doing it | 20:07 |
| ivoline | Was just talking to diablo_rojo about a db dump for local testing before the meeting. | 20:07 |
| ivoline | I'll need it for a low hanging fruit bug I am currently working on | 20:08 |
| fungi | yeah, and the data we have currently in our production and staging instances have potentially sensitive bits we'd need to sanitize/redact first which makes using them for that purpose complicated | 20:09 |
| ivoline | oh okay | 20:09 |
| mkarray | Would using data from the other openstack projects using storyboard be a solution? | 20:10 |
| mkarray | I'd guess there's no sensitive information in the projects which are by nature open to the public | 20:11 |
| diablo_rojo | fungi, if you want to make a dump we could put it in the wiki maybe? | 20:11 |
| diablo_rojo | mkarray, aside from private stories thats mostly true | 20:11 |
| diablo_rojo | we need to remove those from the dump | 20:11 |
| fungi | well, the complexity of sanitizing is we need to exclude session keys, api keys, e-mail addresses, et cetera. and then yes there's the question of how to identify and omit all the rows in various tables which may hint at a private story | 20:12 |
| fungi | it'll be time-consuming, and to be honest i'd rather put my time into scripting something to create sample data than into trying to clean up an existing dataset | 20:13 |
| fungi | if it's one of the two | 20:13 |
| SotK | as a low effort solution we could perhaps run the migration script into a local instance for some big projects, and make some boards like https://storyboard.openstack.org/#!/board/83 maybe? | 20:13 |
| fungi | that's also far safer, since there's no chance i miss something i should have known to filter out | 20:14 |
| fungi | yeah, that's a possibility | 20:14 |
| fungi | could be done by anybody on a local deployment of sb too | 20:14 |
| SotK | yeah, now we have private stories implemented and in use providing production dumps feels like its asking for trouble really | 20:14 |
| fungi | i need to step away. christine is apparently dangerously hungry and i told her we'd go grab some food. should be back in an hour or so | 20:15 |
| SotK | enjoy :) | 20:15 |
| fungi | thanks! | 20:15 |
| SotK | the only drawback is that some projects take weeks to import from launchpad | 20:16 |
| SotK | anyway, I think query optimisation and duplicates or some other feature that's more interesting than rewriting tests | 20:16 |
| SotK | diablo_rojo: do you have a preference? | 20:17 |
| diablo_rojo | I think query optimisation sounds good | 20:22 |
| diablo_rojo | I will start there and then depending on how much more effort I want to put in maybe we can post another for testing or duplicates | 20:23 |
| SotK | sounds good to me, thanks | 20:24 |
| diablo_rojo | I will try to hammer that out today | 20:25 |
| *** ivoline has quit IRC | 20:25 | |
| SotK | so, forum planning | 20:25 |
| SotK | there's an etherpad: https://etherpad.openstack.org/p/SB_train_forum_brainstorming | 20:25 |
| *** ivoline_ has joined #storyboard | 20:25 | |
| diablo_rojo | Yep. | 20:26 |
| diablo_rojo | I guess we do another pain points gathering | 20:27 |
| SotK | I think that's probably sensible | 20:28 |
| *** ivoline_ has quit IRC | 20:29 | |
| *** ivoline_ has joined #storyboard | 20:30 | |
| SotK | hopefully by then we have attachments working | 20:30 |
| diablo_rojo | Or like 95% of the way there so people can leave it alone :) | 20:30 |
| SotK | yup, done enough that the session isn't just a discussion on attachments | 20:31 |
| SotK | should we try to populate that etherpad with some anticipated things beforehand? | 20:32 |
| *** jtomasek has quit IRC | 20:37 | |
| SotK | mkarray: do you still want to discuss search improvements even though its not particularly meeting time at this point? sorry for not getting round to your agenda item during the actual meeting | 20:42 |
| mkarray | No worries, and yes I would prefer to still have a discussion on it. | 20:42 |
| mkarray | As we've already discussed, there's currently no way for a user to browse through the web-interface | 20:43 |
| mkarray | Is it something you guys believe to be worth investing time in? | 20:44 |
| SotK | so, if you select something from the search bar's dropdown that isn't the text option (so one of the things with the project icon next to it for example) then a browse gets done | 20:55 |
| SotK | however this behaviour confuses pretty much everyone afaict and I think its definitely worth investing time in turning it into something more friendly and understandable | 20:56 |
| mkarray | I understand that, I was thinking of implementing a system that allows users to type exactly what they want. As in executing their own browse without clicking on an option | 20:57 |
| SotK | sounds like a good improvement to me | 20:59 |
| SotK | as long as the ability to click on options to construct such a query continues to exist for lazy folks like me | 20:59 |
| mkarray | In terms of what I was attempting to implement, doing so would be two birds with one stone. | 21:04 |
| *** jamesmcarthur has quit IRC | 21:05 | |
| mkarray | From what we talked about in pm's yesterday, each storyboard component has it's own browsable tags. (assignee_id, creator_id). If we had a manual browse, this gives the user the ability to search on any of the tags instead of just the ones for each component | 21:06 |
| mkarray | For example: As of right now, browsing for the author of a story isn't currently supported in the web-interface but the assignee_id is. If we allowed a manual browse, (creator_id:"mkarray") then they can access those stories as well | 21:09 |
| *** jamesmcarthur has joined #storyboard | 21:17 | |
| *** ivoline_ has quit IRC | 21:21 | |
| SotK | mkarray: yep, I think that'd be a nice solution | 21:29 |
| mkarray | Sounds good, I'll start there then | 21:30 |
| SotK | long-term it'd be nice to support being able to do something like `creator:"mkarray" OR creator:"SotK"` too, but I think we can extend into doing that kind of thing once the existing confusion is at least helped a bit | 21:32 |
| * SotK suspects that persia's earlier suggestion was to go straight to an implementation of the latter, but I could be wrong | 21:32 | |
| persia | For now, let's not try any logical operators. Last time we discussed how to implement them in search, we discovered it was *very* complex to do sensibly. | 21:34 |
| persia | For that matter, in other parts of the UI, we have ordered addition and subtraction for complex queries: maybe it makes sense to reuse that model in the short term. | 21:35 |
| *** jamesmcarthur has quit IRC | 21:55 | |
| SotK | the worklist filtering stuff? I suspect that will need enough backend changes that we may as well write a better version | 21:58 |
| *** jamesmcarthur has joined #storyboard | 21:59 | |
| persia | that also works. My main thought was to seek consistency if possible. | 22:02 |
| SotK | I agree with that thought, I'd just prefer to get there by eventually replacing the existing filtering with whatever query parsing we end up implementing for searching | 22:05 |
| persia | Indeed, that would probably be better, although the migration might be interesting :) | 22:15 |
| *** mkarray has quit IRC | 22:17 | |
| *** whoami-rajat has quit IRC | 22:31 | |
| diablo_rojo | SotK, fungi looks like I can't add you as co-mentors, but this is the link you need to add yourselves: https://www.outreachy.org/may-2019-august-2019-outreachy-internships/communities/openstack/storyboard-database-query-optimizations/cfp/mentor/submit/ | 23:34 |
| *** jamesmcarthur has quit IRC | 23:55 | |
| *** jamesmcarthur has joined #storyboard | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!