| *** efried has quit IRC | 00:51 | |
| *** efried has joined #openstack-placement | 00:59 | |
| *** tetsuro has joined #openstack-placement | 01:12 | |
| *** tetsuro has quit IRC | 02:24 | |
| *** tetsuro has joined #openstack-placement | 02:48 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack/osc-placement master: Add --amend option to 'resource provider inventory set' https://review.opendev.org/640898 | 02:52 |
|---|---|---|
| *** tetsuro has quit IRC | 03:21 | |
| *** tetsuro has joined #openstack-placement | 04:28 | |
| *** tetsuro has quit IRC | 05:02 | |
| *** dansmith has quit IRC | 05:26 | |
| *** dansmith has joined #openstack-placement | 05:28 | |
| *** tetsuro has joined #openstack-placement | 05:43 | |
| *** tetsuro has quit IRC | 05:48 | |
| *** belmoreira has joined #openstack-placement | 05:50 | |
| *** belmoreira has quit IRC | 05:50 | |
| *** belmoreira has joined #openstack-placement | 05:52 | |
| *** e0ne has joined #openstack-placement | 06:57 | |
| *** e0ne has quit IRC | 06:59 | |
| *** belmoreira has quit IRC | 07:01 | |
| *** belmoreira has joined #openstack-placement | 07:03 | |
| *** tetsuro has joined #openstack-placement | 07:13 | |
| *** tetsuro has quit IRC | 07:17 | |
| *** belmoreira has quit IRC | 07:28 | |
| *** tssurya has joined #openstack-placement | 07:32 | |
| *** belmoreira has joined #openstack-placement | 07:35 | |
| *** cdent has joined #openstack-placement | 07:36 | |
| *** e0ne has joined #openstack-placement | 07:53 | |
| *** belmoreira has quit IRC | 08:01 | |
| *** belmoreira has joined #openstack-placement | 08:07 | |
| *** belmoreira has quit IRC | 08:09 | |
| *** belmoreira has joined #openstack-placement | 08:11 | |
| *** purplerbot has quit IRC | 08:36 | |
| *** purplerbot has joined #openstack-placement | 08:36 | |
| *** belmoreira has quit IRC | 08:54 | |
| * cdent will update the canaries | 09:01 | |
| *** belmoreira has joined #openstack-placement | 09:03 | |
| openstackgerrit | Chris Dent proposed openstack/placement master: Bump os-traits minimum to 0.16.0 https://review.opendev.org/673964 | 09:05 |
| *** belmoreira has quit IRC | 09:22 | |
| *** belmoreira has joined #openstack-placement | 09:26 | |
| cdent | stephenfin: if you're around if you could gaze at ^, that would be lovely | 09:32 |
| *** e0ne_ has joined #openstack-placement | 09:36 | |
| *** e0ne has quit IRC | 09:37 | |
| stephenfin | cdent: Makes sense. Done | 09:43 |
| cdent | thanks | 09:43 |
| *** belmoreira has quit IRC | 09:53 | |
| *** belmoreira has joined #openstack-placement | 10:00 | |
| openstackgerrit | Chris Dent proposed openstack/placement master: _get_all_by_filters_from_db do not cast to list of dict https://review.opendev.org/673991 | 10:07 |
| cdent | edleafe: you'll be "happy" to know that the method to maintain awareness of the graph in placement, https://opendev.org/openstack/placement/src/branch/master/placement/objects/research_context.py#L303 , is a major cpu contributor. For whatever reason the db doesn't much like to do that query quickly | 11:05 |
| cdent | zzzeek: any suggestions on speeding the query in the linked code ^ ? | 11:07 |
| cdent | that in_ clause will ofte have hundreds or thousands of ids | 11:07 |
| *** belmoreira has quit IRC | 11:17 | |
| *** belmoreira has joined #openstack-placement | 11:19 | |
| *** e0ne has joined #openstack-placement | 11:29 | |
| *** e0ne_ has quit IRC | 11:30 | |
| cdent | hmmm, I've worked around it in a useful way | 11:58 |
| openstackgerrit | Chris Dent proposed openstack/placement master: Remove double join in provider_ids_from_rp_ids https://review.opendev.org/674017 | 12:26 |
| *** tssurya_ has joined #openstack-placement | 12:37 | |
| *** tssurya has quit IRC | 12:37 | |
| *** tssurya_ is now known as tssurya | 12:37 | |
| *** artom has quit IRC | 12:37 | |
| *** belmoreira has quit IRC | 12:48 | |
| *** belmoreira has joined #openstack-placement | 13:06 | |
| edleafe | cdent: I wouldn't say I'm happy to hear that. I would say I'm not surprised at all. | 13:20 |
| cdent | "quotes on purpose" | 13:20 |
| edleafe | Yes, I realized that | 13:20 |
| cdent | other not surprising observation: default postgresql is more load-handling than default mysql | 13:21 |
| edleafe | MATCH (root)-[*0..99]->(rp) seems a lot simpler. Not having to track those IDs also makes the code base a whole lot simpler, too | 13:22 |
| efried | cdent: I thought zzzeek at some point recommended we do those in batches instead of all at once. | 13:25 |
| cdent | efried: yes, but it's not clear that's appreciably different enough for the cost to the code readability. i'll try it though | 13:26 |
| cdent | removing the joins helps, but not as much as i would have liked | 13:27 |
| zzzeek | cdent: there's a technique to batch queries on large lists of IN,yes | 13:27 |
| *** mriedem has joined #openstack-placement | 13:27 | |
| zzzeek | cdent: since you don't have an ORDER BY it should be straightforward | 13:27 |
| cdent | zzzeek: is it likely to have much impact? the newer (no joins) code is at https://review.opendev.org/674017 | 13:28 |
| zzzeek | cdent: there's also a technique called baked query that could speed this up a lot on the python side, baked queries are going to be superseded by somethign a lot better in a year or so but for now they would cut down the Python query compilation time, if that's where this gets hit | 13:29 |
| cdent | one sec, lemme look back at the profiling | 13:29 |
| zzzeek | cdent: depends on where this code is causing a problem, eg. query compialtion, or network overhead, or database time, or lots of rows coming back | 13:29 |
| zzzeek | cdent: from what I can see I dont know if it's just the outer join that is making the query slow | 13:30 |
| zzzeek | cdent: if that newer code is much faster, then you know it wasn't the IN of lots of values | 13:31 |
| cdent | after that change, execute is about 50% of the time under the call, in_ is maybe 30% | 13:32 |
| efried | zzzeek: Is a baked query anything like a stored procedure? | 13:35 |
| zzzeek | efried: nope | 13:35 |
| zzzeek | efried: it just caches the process of turning the PYthon object into a string. looking at this code some more, it won't have that dramatic of an impact b.c. this is not an ORM query in the first place | 13:36 |
| efried | mm | 13:36 |
| zzzeek | ORM Query objects are expensive to build and turn into strings | 13:36 |
| cdent | yeah, since jay started much of this code, he eschewed ORM objects | 13:37 |
| efried | I think we have almost none of that in placement | 13:37 |
| efried | yeah | 13:37 |
| cdent | the perfload nodes are not great machines. I can get much better results elsewhere | 13:48 |
| *** belmoreira has quit IRC | 14:10 | |
| cdent | zzzeek: doing " sa.bindparam('rps', expanding=True " appears to have a rather positive impact | 14:19 |
| * cdent pushes up a patch for that | 14:20 | |
| zzzeek | cdent: great, I hope to make those the standard in 1.4 | 14:21 |
| zzzeek | e.g. all IN will use that approach | 14:21 |
| zzzeek | and will also work with empty collections | 14:21 |
| cdent | yay, progress! | 14:21 |
| zzzeek | and tuples | 14:21 |
| zzzeek | cdent: it's a little surprising it's improving performance for you without using query caching though | 14:23 |
| cdent | it's making the surrounding method about 50% faster which is ... weird | 14:23 |
| cdent | but i'll take it | 14:23 |
| cdent | you can have a closer look when I get the patch committed, if that's useful, just a sec | 14:24 |
| *** artom has joined #openstack-placement | 14:25 | |
| openstackgerrit | Chris Dent proposed openstack/placement master: Use expanding bindparam in provider_ids_from_rp_ids in_ https://review.opendev.org/674048 | 14:27 |
| cdent | thanks for watching the requirements mriedem | 15:08 |
| cdent | i've been way down deep in profiing data... is weird down here | 15:08 |
| mriedem | cdent: one thing in https://review.opendev.org/#/c/673788/ if you care, | 15:10 |
| mriedem | i think you're getting a ResultProxy back not a RowProxy | 15:10 |
| mriedem | not a big deal, but if anyone ever gets confused over that commit message whilst debugging | 15:10 |
| cdent | yeah, good suggestion | 15:11 |
| mriedem | i'd fast approve if you update | 15:11 |
| mriedem | thus completing my 2 minimum placement reviews per month | 15:11 |
| cdent | mriedem: look at the one in the middle of that stack and then I'll fix up all three accordingly | 15:11 |
| cdent | of course the one in the middle is the hard one... | 15:11 |
| mriedem | yeah.... "this probably needs some pretty thoughtful review" | 15:12 |
| mriedem | plus you have to fix the tests anyway right? | 15:12 |
| cdent | they're already fixed in https://review.opendev.org/#/c/673964/ , but if I have to edit the bottom of my stack I'd like to edit the middle of my stack if you're going to have things to say | 15:13 |
| openstackgerrit | Chris Dent proposed openstack/placement master: Remove ProviderIds namedtuple https://review.opendev.org/673788 | 15:26 |
| openstackgerrit | Chris Dent proposed openstack/placement master: Remove double join in provider_ids_from_rp_ids https://review.opendev.org/674017 | 15:26 |
| openstackgerrit | Chris Dent proposed openstack/placement master: Use expanding bindparam in provider_ids_from_rp_ids in_ https://review.opendev.org/674048 | 15:26 |
| mriedem | sorry was on the phone, i didn't look at the middle patch in detail | 15:27 |
| *** tssurya has quit IRC | 15:30 | |
| cdent | mriedem: s'okay, realized I was near the end of my day, so figured I'd go ahead | 15:31 |
| mriedem | 10:30 here and i'm not sure i've started doing any of the things from my list :) | 15:32 |
| cdent | i was started unusually early this morning | 15:33 |
| *** e0ne has quit IRC | 15:37 | |
| * cdent will come back later to recheck several things after traits canary merges | 15:43 | |
| cdent | in the meaning | 15:43 |
| * cdent waves | 15:43 | |
| cdent | meantime! | 15:43 |
| *** cdent has quit IRC | 15:48 | |
| *** artom has quit IRC | 16:04 | |
| *** artom has joined #openstack-placement | 16:29 | |
| mriedem | btw, it's probably time to close out launchpad for osc-placement bugs https://bugs.launchpad.net/placement-osc-plugin | 19:50 |
| mriedem | i can do that easily enough. i'm not sure if https://bugs.launchpad.net/placement-osc-plugin/+bug/1811101 will still be visible - probably not, but could just create a story from it in sb | 19:51 |
| openstack | Launchpad bug 1811101 in placement-osc-plugin "table columns can have inconsistent ordering" [Undecided,New] | 19:51 |
| mriedem | done https://bugs.launchpad.net/placement-osc-plugin | 19:52 |
| mriedem | https://launchpad.net/placement-osc-plugin tells people to go to storyboard | 19:53 |
| *** e0ne has joined #openstack-placement | 20:40 | |
| *** takashin has joined #openstack-placement | 20:50 | |
| *** artom has quit IRC | 21:39 | |
| openstackgerrit | Eric Fried proposed openstack/placement master: Remove ProviderIds namedtuple https://review.opendev.org/673788 | 21:48 |
| openstackgerrit | Eric Fried proposed openstack/placement master: Remove double join in provider_ids_from_rp_ids https://review.opendev.org/674017 | 21:48 |
| openstackgerrit | Eric Fried proposed openstack/placement master: Use expanding bindparam in provider_ids_from_rp_ids in_ https://review.opendev.org/674048 | 21:48 |
| *** mriedem has quit IRC | 21:53 | |
| *** e0ne_ has joined #openstack-placement | 22:45 | |
| *** e0ne has quit IRC | 22:46 | |
| *** e0ne has joined #openstack-placement | 22:47 | |
| *** e0ne_ has quit IRC | 22:50 | |
| *** e0ne has quit IRC | 22:52 | |
| *** artom has joined #openstack-placement | 23:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!