| -opendevstatus- NOTICE: The maintenance of the review.opendev.org Gerrit service is now complete and service has been restored. Please alert us in #opendev if you have any issues. Thank you | 03:26 | |
| *** mgoddard- is now known as mgoddard | 06:04 | |
| *** rpittau|afk is now known as rpittau | 09:31 | |
| *** kopecmartin is now known as kopecmartin|pto | 09:45 | |
| *** sshnaidm|afk is now known as sshnaidm | 13:10 | |
| johnsom | zzzeek Hi there. We are seeing a big speed difference between SQLAlchemy 1.3.24 and 1.4.20/21 on our functional tests that use sqlite. One test for example, 1.3.24 is 46.3184 seconds, 1.4.21 is 75.285. Would you have any tips for us? | 16:52 |
|---|---|---|
| johnsom | Test job output: https://zuul.opendev.org/t/openstack/build/7ad487ec5fad48a894ddb30ad58d8c6e/log/job-output.txt | 16:52 |
| zzzeek | johnsom: yes, you would need to identify the specific set of SQLAlchemy functions that are behaving differently. since it's SQLite, it's unlikely to be relaeted to SQL query consruction or concurrency, so you'd want to look into code profiling as discussed at https://docs.sqlalchemy.org/en/14/faq/performance.html#code-profiling to identify where there's a significant quantitative difference, then you can show it to me | 17:01 |
| zzzeek | johnsom: also as these are test suites, it would have to be determined if the speed difference involves structural things that wouldnt' actualyl impact a real app the same, such as setup of mappings , cache warmup, etc. | 17:02 |
| zzzeek | that said , 40% slower there is pretty dramatic so i hope to see where the callcounts have changed | 17:02 |
| johnsom | Yeah, it's easy to A/B test like it did in that tox environment for that one test. | 17:03 |
| zzzeek | johnsom: is htat like a special job you created to run a single test against two different reqs ? | 17:07 |
| johnsom | Yes | 17:07 |
| zzzeek | i see | 17:07 |
| johnsom | https://review.opendev.org/c/openstack/octavia/+/801355/1/tox.ini | 17:08 |
| zzzeek | johnsom: would need to see cprofiles | 17:10 |
| johnsom | Yeah, I will see what I get on a few queries. That test is complex, but shows the magnitude. | 17:12 |
| zzzeek | johnsom: yes a more focused profile would be even better. you can send me raw profile files if you want | 17:12 |
| zzzeek | johnsom: w/ pstats.dump_stats() | 17:13 |
| johnsom | Ok, let me poke at this a bit this morning. I'll ping you back when I have something | 17:13 |
| zzzeek | or cprofile.dump_stats() | 17:13 |
| zzzeek | johnsom: the bigget thing that can blow out performance is if there are simply more queries being emitted for some reason, like if an eagerload diective is not being caught | 17:15 |
| zzzeek | johnsom: so try looking at SQL log output too and just look for any big quantitiative differences | 17:15 |
| johnsom | I looked at that last week. It actually seemed like there were less queries going out in 1.4.21. | 17:16 |
| zzzeek | johnsom: also odd | 17:16 |
| zzzeek | johnsom: well we eager load on refresh a bit....but a single test i wouldnt think has a lot of refresh going on | 17:17 |
| johnsom | It's a bit rough guess though, there is pool health checks in there, etc. so there is going to be some variability run to run. | 17:17 |
| *** rpittau is now known as rpittau|afk | 17:38 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!