| *** krotscheck has quit IRC | 00:29 | |
| cody-somerville | One may be able to make a good argument against using foreign key constraints but I don't think storyboard has a need to be so progressive. Besides, I'd trust the sql server to ensure consistency over vs. the ORM or the scheme migration engine alembic or raw sql statements that might get used to work around ORM limitations/complexity. I think there is something to be said for the data layer protecting it's own consistency and in | 00:36 |
|---|---|---|
| cody-somerville | tegrity. | 00:36 |
| cody-somerville | Frankly, data is valuable. It's worth the extra protection. Data usually lasts a longer longer than the application code. | 00:39 |
| *** miqui has quit IRC | 02:28 | |
| *** david-lyle has joined #storyboard | 03:50 | |
| *** saju_m has joined #storyboard | 04:12 | |
| mordred | I'm reading the scrollback in spurts - so my responses may be weird | 05:17 |
| mordred | ok. I've read the whole scrollback. I don't think the idea of doing reverse patches on the migrations is a bad one - we aRE still young, and we haven't gotten everything fully right yet | 05:25 |
| mordred | while I do think it's valuable to do migrations right now (even though it's painful - we have to learn them at some point) - I don't think we need to stand on principle when we run into something crazy | 05:26 |
| mordred | like postgres | 05:26 |
| SergeyLukjanov | mordred, ++ | 05:26 |
| mordred | as for foreign keys - they're a bad idea for scale | 05:28 |
| mordred | period | 05:28 |
| mordred | in the mysql world | 05:28 |
| mordred | however - the way I've always wanted to attack that thoguh is to make a patch to sqlalchemy that lets you set an option to not actually emit foreign key DDL for the mysql dialect | 05:29 |
| mordred | I believe they are more important in postgres, so doing deletes in migrations seems likea lot of work on our part -and I do not believe we're at that scale yet - and we may never be | 05:30 |
| *** saju_m has quit IRC | 06:02 | |
| *** saju_m has joined #storyboard | 06:04 | |
| *** saju_m has quit IRC | 06:07 | |
| *** saju_m has joined #storyboard | 06:08 | |
| *** saju_m has quit IRC | 06:35 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: [WIP] Comments controller https://review.openstack.org/81505 | 07:32 |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: [WIP] Comments controller https://review.openstack.org/81505 | 07:55 |
| *** saju_m has joined #storyboard | 07:56 | |
| *** jcoufal has joined #storyboard | 08:19 | |
| *** hashar has joined #storyboard | 08:44 | |
| *** hashar has quit IRC | 09:18 | |
| *** hashar has joined #storyboard | 09:38 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Added db_api for comments https://review.openstack.org/81232 | 09:41 |
| *** jcoufal has quit IRC | 10:06 | |
| *** hashar has quit IRC | 11:16 | |
| *** david-lyle has quit IRC | 11:52 | |
| *** saju_m has quit IRC | 12:09 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Comments controller https://review.openstack.org/81505 | 12:10 |
| *** saju_m has joined #storyboard | 12:26 | |
| *** saju_m has quit IRC | 12:53 | |
| *** saju_m has joined #storyboard | 13:06 | |
| *** saju_m has quit IRC | 13:23 | |
| *** saju_m has joined #storyboard | 13:39 | |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/storyboard: Remove Branch and Milestone legacy tables https://review.openstack.org/81562 | 13:44 |
| ttx | OK, *that* will hopefully work | 13:45 |
| *** saju_m has quit IRC | 13:48 | |
| *** hashar has joined #storyboard | 13:51 | |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/storyboard: Remove Branch and Milestone legacy tables https://review.openstack.org/81562 | 13:58 |
| ttx | or this. | 13:58 |
| *** mfer has joined #storyboard | 14:11 | |
| *** jcoufal has joined #storyboard | 14:16 | |
| jeblair | mordred: sorry i'm fuzzy on this because i haven't used alembic -- but is the schema always created with alembic? and in that case, does that mean we can skip the sqlalchemy change you were talking about? | 14:29 |
| jeblair | or is alembic only used in migrations of an existing schema and the schema is created by sqla if you're starting from scratch? | 14:30 |
| jeblair | if the latter, then the change to sqla you describe would be required in order to not create fk's in the db while still informing the orm of them, right? | 14:31 |
| mordred | jeblair: the schema is always created by alembic | 14:40 |
| mordred | jeblair: hrm. I hadn't thought of that... | 14:41 |
| mordred | jeblair: however - the unittesting of the migrations will suffer | 14:41 |
| jeblair | mordred: ooh, okay. so is it possible to just omit the FK instructions to alembic but retain them in sqla? | 14:41 |
| jeblair | why's that? | 14:42 |
| mordred | because what we're doing now is running the migrations and then making sure that matches what the model thinks it wants | 14:42 |
| mordred | and then in the other unittests, just using the model | 14:42 |
| jeblair | what does the matching? | 14:42 |
| mordred | one sec | 14:42 |
| mordred | (loking) | 14:42 |
| mordred | ah... sorry, I lied | 14:44 |
| mordred | the migrations don't test for consistency with the model - they run migration unittests - but just run against mysql | 14:45 |
| mordred | so when you write a migration, you should also write a unittest for the migration | 14:45 |
| mordred | so yeah - just not doing foreign keys is fine | 14:45 |
| jeblair | ah yeah, those migration tests already look like they don't care about fks | 14:46 |
| mordred | woot | 14:48 |
| *** miqui has joined #storyboard | 14:59 | |
| * ttx finally beats Alembic | 15:03 | |
| *** david-lyle has joined #storyboard | 15:16 | |
| *** jcoufal has quit IRC | 15:50 | |
| *** krotscheck has joined #storyboard | 16:00 | |
| krotscheck | Ehn? Meeting time? | 16:01 |
| NikitaKonovalov | looks like so | 16:02 |
| krotscheck | ttx: You there? Or am I running this? | 16:03 |
| *** hashar has quit IRC | 16:22 | |
| *** krotscheck has quit IRC | 17:00 | |
| mordred | cody-somerville: aroudn? | 18:54 |
| cody-somerville | mordred: yup | 19:10 |
| mordred | cody-somerville: I just read the meeting logs and figured we should chat about foreign keys :) | 19:10 |
| mordred | cody-somerville: basically, I don't feel STRONGLY about it - other than that I feel that they're pointless in a system that's using an ORM | 19:13 |
| mordred | cody-somerville: I disagree that the data will outlive the app | 19:13 |
| mordred | I mean - the data will | 19:13 |
| mordred | but I do not believe that another app will be written to access the database in its current from | 19:13 |
| mordred | form | 19:13 |
| mordred | as much as, if a new app will be written, it will have a new database and there will be a migration | 19:13 |
| mordred | if we were to get large enough to hit scaling issues, we'd most likely shard, which most likely happens by spinning off separate database instances with their own data apis in front of them and then doing a dump/transform/repliacte to the new instances | 19:14 |
| mordred | and, behind a data api that defines the constraints and relationships, they don't add any benefit other than overhead | 19:15 |
| mordred | that said - you are right, they do not provide a noticable performance hit at our current scale | 19:15 |
| mordred | so it's not important to remove them | 19:15 |
| mordred | I mostly just wish that sqlalchemy had a "production mode" flag you could flip to just have it ignore them as no-op on mysql | 19:16 |
| mordred | but clearly no-one has ever run a sqlalchemy app at massive scale yet | 19:16 |
| mordred | and we probably will not either | 19:16 |
| cody-somerville | I suppose you could just set foreign_key_checks = 0 in mysql configuration file if you wanted to | 19:17 |
| mordred | hrm. not a bad idea. | 19:18 |
| cody-somerville | db seems like a good place to store the scheme, including the foreign relationships, even if you don't enforce the checks for some crazy reason | 19:18 |
| cody-somerville | you say it's just additional overhead | 19:18 |
| mordred | why - when you're storing it in the model in git already? | 19:18 |
| mordred | I mean, in ruby on rails the ORM is inferred from the schema, so you _have_ to have them there | 19:19 |
| mordred | otherwise active record does not know what to do | 19:19 |
| cody-somerville | It's not difficult to imagine bugs in application layer (sure, a bug could happen in database but the risk there is much more contained due to separation of concerns) | 19:19 |
| cody-somerville | it's also not difficult to imagine someone resorting to some manual SQL | 19:20 |
| mordred | well, it is, in that most people do not have access to do that. :) | 19:20 |
| cody-somerville | application developers do | 19:20 |
| cody-somerville | sqlalchemy lets you do raw statements just fine | 19:20 |
| mordred | sure. well, listen - like I said - I don't feel strongly about it | 19:21 |
| mordred | I'm just trying to explain why they're a bad idea | 19:21 |
| cody-somerville | I doubt the writers of sqlalchemy wrote it in such a way to be defensive enough about being responsible for the integrity of the data | 19:21 |
| mordred | I mean, they're not as bad an idea as stored procedures or function indexes | 19:23 |
| mordred | s/they're a bad/I think they're a bad/ | 19:24 |
| cody-somerville | mordred: I think calling them a bad idea is being a bit too broad. I get that sometimes it makes sense to not use them. And I understand the value in maybe having such a hard line position as a db person to get people to really think about it. | 19:25 |
| mordred | yah. the second sentence | 19:26 |
| mordred | I agree - they're not always a bad idea | 19:26 |
| mordred | I, myself, would actually never use them in a database | 19:26 |
| mordred | and never have | 19:26 |
| mordred | but that does not mean that they are a bad idea | 19:26 |
| mordred | across the board | 19:26 |
| mordred | I _Do_ mainly want to point out that just because the relational algebra enjoys them, and just because the big-iron databases rely on them | 19:27 |
| mordred | those actually _do_ tend to be databases that are accessed by mutliple different app developers independently | 19:27 |
| mordred | with a DBA who maintains things like relationships and views | 19:27 |
| cody-somerville | and some people probably feel similarly about type safety or garbage collection ;) | 19:28 |
| mordred | in many ways - I see the DB ORM layer of an API app as filling the same space as the DBA-run views and relationships | 19:28 |
| mordred | well, garbage collection is a bad idea too | 19:28 |
| mordred | but only if you care about consistent performance at high scale :) | 19:28 |
| cody-somerville | maybe we should rewrite openstack entirely in C ;) | 19:29 |
| mordred | well, C++ | 19:29 |
| mordred | compile-time type safety actually allows for more efficient code to be generated by the compiler | 19:30 |
| mordred | run-time type safety is a performance hit | 19:30 |
| mordred | I love compile-time type safety for that reason | 19:31 |
| mordred | (which, btw, is one of the benefits of ORM-generated queries- they are compile-time checked - as opposed to the run-time check of throwing a query against the db and then checking to see if it violates a constraint) | 19:31 |
| cody-somerville | from the sqlalchemy website "All object-relational patterns are designed around the usage of proper referential integrity, and foreign keys are an integral part of its usage." | 19:33 |
| mordred | sure | 19:34 |
| mordred | they're wrong | 19:34 |
| cody-somerville | I think that's in reference to how sqlalchemy works with relational models, not commentary on information theory | 19:35 |
| mordred | if sqlalchemy, as a query generator, relied on the database to throw integrity violation errors to be able to produce proper sequencing, we wouldn't us it for anything | 19:36 |
| *** hashar has joined #storyboard | 20:02 | |
| *** mfer has quit IRC | 21:25 | |
| *** krotscheck has joined #storyboard | 21:36 | |
| *** david-lyle has quit IRC | 21:37 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Added paging to list endpoints https://review.openstack.org/79757 | 22:03 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Added paging to list endpoints https://review.openstack.org/79757 | 22:07 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Remove Branch and Milestone legacy tables https://review.openstack.org/81562 | 22:13 |
| *** hashar has quit IRC | 22:18 | |
| *** krotscheck has quit IRC | 22:30 | |
| *** krotscheck has joined #storyboard | 22:31 | |
| *** david-lyle has joined #storyboard | 22:38 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Added paging to list endpoints https://review.openstack.org/79757 | 22:39 |
| *** krotscheck has quit IRC | 22:46 | |
| *** krotscheck has joined #storyboard | 22:47 | |
| *** david-lyle has quit IRC | 23:31 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!