skipping elidable migrations

[Dan Watson]

Hi All,

I was wondering if anyone had any thoughts on an option to the "migrate" command (--skip-elidable?) that would skip running elidable migrations. The use case here is that data migrations that build up over time may act on certain assumptions (existing tables/data) that may not be true when migrating a new database. It seems that since they were explicitly marked as able to be deleted when squashing, they would be safe to not run when creating a new database. Maybe we don't go so far as to make this the default behavior when migrating a fresh database, but an option would be nice. I realize you could simply squash your migrations, but that's not without penalty of code churn, testing, etc. especially when your existing migration graph is otherwise fine (and performant).

If there's some consensus about this being worthwhile, or at least no strong objections to it, I can take a stab at the implementation.

Regards,

Dan

[Adam Johnson]

Do you have an example? If you're using Django's default testing framework, it normally creates a fresh database, so from the moment a new data migration is written it would be tested with the empty database scenario.

Afaiu it is possible to write RunPython operations in a way that no-ops on empty databases, e.g. starting with the equivalent of "if not Model.object.exists(): return." It is also possible with RunSQL on database backends that support conditional logic around SQL statements,  or they can be easily converted to RunPython using a function that uses cursor.execute(). And if a project develops problems with past migrations, it's always possible to (carefully) edit them to include such conditions.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/fba7a58a-444b-4c90-b139-151580423366%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Adam

[Dan Watson]

Hi Adam,

All of your suggestions are perfectly valid things to do (and are in fact things I've done in the past). This particular project uses data migrations fairly heavily, and the developers haven't been particularly good about making sure they run cleanly on fresh databases. We are using Django's test framework, but a) with migrations disabled for performance reasons and b) using --keepdb, which masks these types of problems.

It just got me thinking that if you've gone out of your way to mark an operation as basically "safe to delete", it seems unimportant to run them during an initial migration. Bad programming aside, it seems like an easy performance bump for projects that use these types of operations. I think of elidable as a "soft delete" option that gets garbage collected next squash. This would just be an optimization step to ignore what is already marked for death. Maybe it's not an optimization we want to reason about, or maybe it isn't a big enough use case to be worth coding for, but I figured I'd put it out there.

Regards,

Dan