| *** diablo_rojo has joined #storyboard | 00:22 | |
| *** diablo_rojo has quit IRC | 02:27 | |
| *** bethwhite has joined #storyboard | 06:40 | |
| SotK | SpamapS: woah, I suspect I need to improve the code behind tasks | 07:30 |
|---|---|---|
| SotK | that is slooow | 07:30 |
| dmsimard | Larger workboards are very very painful to browse :( | 13:10 |
| dmsimard | http://i.imgur.com/LH9khbs.png | 13:10 |
| dmsimard | (for instance) | 13:10 |
| * SotK has never seen that before | 13:11 | |
| dmsimard | Looks like it could be related to too many people viewing the workboard simultaneously ? We're doing a team planning | 13:13 |
| SotK | hm, maybe | 13:14 |
| SotK | do you have access to the server logs? | 13:14 |
| dmsimard | We'll try to provide feedback, mhu likely has access :) | 13:15 |
| mhu | dmsimard, I think it doesn't help that the API calls fetch also every archived task/story per lane | 13:17 |
| mhu | but we're using an old version of SB, an upgrade is long due | 13:17 |
| * SotK recommends waiting until we get the board speedup patches I wrote merged | 13:19 | |
| * SotK will make them ready for review later | 13:19 | |
| Zara | (hm, it shouldn't take long to upgrade, though (guess it depends how it's deployed)) | 13:22 |
| dmsimard | by the way, if no one brought this up before | 13:28 |
| dmsimard | when assigning a task, it'd be great to have "assign to myself" or something to that effect | 13:28 |
| dmsimard | SotK: have a link for those speedup patches ? | 13:33 |
| SotK | dmsimard: sure, one sec | 13:34 |
| SotK | dmsimard: https://review.openstack.org/#/q/status:open+project:openstack-infra/storyboard+branch:master+topic:optimise-boards | 13:34 |
| SotK | however, they will all need reviewing again once I fix the thing causing the tests to fail | 13:35 |
| *** diablo_rojo has joined #storyboard | 14:31 | |
| SotK | diablo_rojo: sorry I didn't get round to commenting on that patch yet, I shall endeavour to do so soon :) | 14:32 |
| diablo_rojo | SotK, no worries :) Soon is good enough for me. | 14:42 |
| * SpamapS peeks at the database code to see if there are obvious speedups | 15:44 | |
| SpamapS | oy... fulltext is a giant bag of fail.. | 15:45 |
| SpamapS | nice to see integer IDs used though. +1 for smaller DB sizes and less RAM for joins. | 15:47 |
| SpamapS | weird.. there's a milestone table.. don't think I've ever seen a milestone mentioned in the UI | 15:49 |
| Zara | yeah, they were last touched a long time ago (we haven't done anything with them since we haven't heard anything about what people *want* from a milestones ui) | 15:54 |
| SpamapS | So, I think I see a pretty easy optimization for WorklistItems | 15:55 |
| SpamapS | it has an arbitrary ID | 15:55 |
| SpamapS | but it has a natural PK of worklist_id + worklist_position | 15:55 |
| SpamapS | and using that one, the query can be ordered by the PK | 15:55 |
| SpamapS | which can be a 100x improvement in InnoDB because rows are clustered by PK | 15:56 |
| SpamapS | Zara: well effectively, we're using tags as milestones in Zuulv3 ... would be nice to have a milestone instead. | 15:57 |
| SpamapS | But the tags do work fine-ish. | 15:57 |
| SpamapS | the main reason for milestones is to be able to communicate your roadmap | 15:58 |
| SpamapS | I'd say your time is better spent working on other bits. :) | 15:58 |
| Zara | :) there are 'due_dates' for boards, which may help with that | 16:00 |
| SpamapS | interesting | 16:03 |
| SpamapS | ah another performance problem is the double-join to WorklistItems filtered by type, instead of just querying the whole WorklistItem table. | 16:04 |
| SpamapS | Oh hm, it's done as a union, and MySQL 5.7 and InnoDB may actually be smart enough to optimize that. | 16:05 |
| SpamapS | yeah these are all just papercuts | 16:07 |
| * SpamapS puts down the shears and lets the yak go | 16:08 | |
| Zara | haha | 16:10 |
| Zara | but it's such a fluffy yak | 16:11 |
| *** bethwhite has quit IRC | 17:58 | |
| SotK | SpamapS: thanks for taking a look, I'm happy to hear any advice about improving the database code | 20:50 |
| SotK | the problem with worklist_id + worklist_position as a PK is that we don't enforce uniqueness of worklist_position, and in fact iirc we set the position of archived cards to something arbitrary | 20:51 |
| SpamapS | SotK: it's all minor improvements.. the sort of thing one does when they're trying to squeeze more out of a heavily taxed system.. not something that just gets an instant doubling of performance. | 20:52 |
| SpamapS | SotK: yeah, it's probably not a big win and if you have to add a counter to maintain, it's definitely not a win at all. | 20:53 |
| SotK | SpamapS: even so, its useful to have reassurance I've not done anything too obviously dumb to make things slow there :) | 20:59 |
| SpamapS | SotK: Nothing obviously dumb no. We could probably iterate on it and get some small latency wins... but most likely the slowdowns dmsimard encountered were row locking contention. | 21:14 |
| dmsimard | SpamapS: we did find an issue on our end that probably made matters worse | 21:15 |
| dmsimard | Part (?) of the app wasn't properly served through mod_wsgi | 21:15 |
| dmsimard | So we'll update, fix that and report back. | 21:15 |
| SpamapS | dmsimard: oh so maybe just high latency on the backend from other sources | 21:23 |
| SpamapS | dmsimard: looking at how work boards work though... it's going to serialize (which is to be expected since it's a consistent data store) so lots of users moving cards will result in a lot of latency | 21:23 |
| SpamapS | the answer to that is a complex one, which is to put an async queue in front of the table edits | 21:23 |
| dmsimard | SpamapS: yeah so we were basically in a "scrum" and we were 5 or 6 creating things, editing things, moving things | 21:24 |
| SpamapS | dmsimard: I'd recommend a single shared desktop if you can work that out ;) | 21:24 |
| SpamapS | but really, making that work well on multiple browsers would be ideal | 21:24 |
| SpamapS | and if you use an async queue, you get the side benefit of being able to subscribe to that via a websocket on clients and update their view in realtime too | 21:25 |
| dmsimard | SpamapS: yeah I think "OpenStack scale" is more than 5 users so you might have to think of something :P | 21:25 |
| SpamapS | we use Github's project boards and they update in real time. Very obvious its using an async edit system. | 21:26 |
| SpamapS | it's | 21:26 |
| dmsimard | Trello is probably a similar implementation as well. | 21:28 |
| SpamapS | there's no other way to do it really. You have your UI send deltas, and your backend either can apply them, or tells the UI to do a full refresh. | 21:33 |
| SotK | we want to implement an async system for storyboard's boards too, but haven't yet had the time really | 21:39 |
| SotK | dmsimard: how old is the version of sb that you're running btw? | 22:02 |
| dmsimard | SotK: not sure | 22:03 |
| dmsimard | SotK: is it available in the UI? All I see is Webclient version: 28b4809... | 22:03 |
| SotK | dmsimard: sadly not, I don't think we have a sensible method of discovering it in the API either actually | 22:05 |
| dmsimard | HTTP GET /version | 22:05 |
| dmsimard | gogo | 22:05 |
| *** openstackgerrit has joined #storyboard | 22:21 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Minimise database queries when resolving cards https://review.openstack.org/444615 | 22:21 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Minimise database queries when populating automatic worklists https://review.openstack.org/444644 | 22:21 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Allow stories and tasks to be filtered by boards/worklists containing them https://review.openstack.org/444614 | 22:21 |
| zara_the_lemur__ | thanks! I'll try to take a break from watching asciiquarium to review a bunch of stuff sometime | 22:43 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
