Migrations in Django 1.7 make unit testing models harder

[Bernie Sumption]

Hi Django devs,

I've just started a new project in 1.7b, and the development experience working with unit tests on models is more complicated than it was in 1.6. It's all down to how the throwaway test databases are created. In 1.6, the create_test_db function "Creates a new test database and runs syncdb against it." In 1.7, it runs "migrate".

While generally speaking, migrate is the new syncdb, this behaviour is not ideal for tests. In 1.6 "syncdb" created a database reflecting the current state of the models in models.py. "migrate" creates a database reflecting the state of the models at the last time makemigrations was run. If you're doing TDD and constantly making small changes to your models then runnning unit tests, you have to run makemigrations before each test run to get your tests to work. You therefore end up with many tiny migration files representing the minute-by-minute history of development.

I came up with a pretty effective workaround that is working for me, but I thought I'd post this here as others are sure to encounter this issue, and I think that it makes more sense for the behaviour produced by this workaround to be the default for running tests.

If makemigrations has not yet been run, the "migrate" command treats an app as unmigrated, and creates tables directly from the models just like syncdb did in 1.6. I defined a new settings module just for unit tests called "settings_test.py", which imports * from the main settings module and adds this line:

MIGRATION_MODULES = {"myapp": "myapp.migrations_not_used_in_tests"}

Then I run tests like this:

DJANGO_SETTINGS_MODULE="myapp.settings_test" python manage.py test

This fools migrate into thinking that the app is unmigrated, and so every time a test database is created it reflects the current structure of models.py.

So my feature request is as follows:

If the new behaviour is by design and considered desirable, then it is a big change from the previous version and should be prominently documented in the migrations and testing pages, along with the workaround. I'm happy to write this documentation if that's the way you want to go.

However, if the new behaviour is not by design but just a by-product of the new migrations feature, I suggest making the workaround the default behaviour. I don't (yet!) know enough about Django internals to volunteer for this however.

Thanks for your time,

Bernie     :o)

[Andrew Godwin]

Yes, the new behaviour is by design, in the sense that the workaround you mentioned will be deprecated in 1.9 (along with all syncdb-related functionality). This way, tests always run against the version of your models that production would run, so you don't run the risk of the tests passing locally as they're not using migrations, pushing it live, and then things failing in production as the tables don't match.

I'm happy to take a small patch to the docs to mention that you need to run makemigrations for tests to see your database changes, but the workaround shouldn't be in there - as I said, it's already part of the deprecation cycle.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/00e9a053-3e61-4c5d-8fcc-5a4d67deab38%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marc Tamlyn]

Do we have an equivalent of south's --update? This would mean you don't get many files. We don't want to make it too hard for people to work in a strict TDD fashion.

M

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1ur1EAxBoDC4O_-LD6B_CsGv3iE7jxCZRts-7ORfKKc%2Buw%40mail.gmail.com.

[Florian Apolloner]

On Tuesday, March 25, 2014 7:14:55 PM UTC+1, Andrew Godwin wrote:

Yes, the new behaviour is by design, in the sense that the workaround you mentioned will be deprecated in 1.9 (along with all syncdb-related functionality).

What exactly will get deprecated here?

[Mark Lavin]

Andrew,

Can you clarify exactly what is deprecated in this work-around? I don't see anything in the deprecation timeline to remove MIGRATION_MODULES nor any pending deprecations related to its usage. It seems like could probably be replaced by something that uses the app-loading/app-configs instead but it does look like it's been done.

Best,

Mark

[Shai Berger]

On Tuesday 25 March 2014 20:36:39 Marc Tamlyn wrote:
> Do we have an equivalent of south's --update? This would mean you don't get
> many files. We don't want to make it too hard for people to work in a
> strict TDD fashion.
>

+1

Shai.

[Andrew Godwin]

So, the functionality whereby you can have apps which do not use migrations (i.e. that use the old creation backends) is meant to go away in 1.9 (i.e. the standard three-release deprecation cycle). Most of the side-effects of this are detailed in https://docs.djangoproject.com/en/dev/internals/deprecation/#deprecation-removed-in-1-9 but not this one - the fact that setting an empty MIGRATION_MODULE means it falls back to the old syncdb method (in 1.9, this would just make Django think that the app had no migrations at all and do nothing, and probably raise a warning or error).

The setting itself isn't deprecated, just this undocumented behaviour, so I haven't put a warning next to it in the docs.

As for the general TDD problem, we do have the squashing functionality now so the --update problem is not quite as bad, but I'm not averse to something like it appearing in 1.8 (1.7 is too frozen now to add it IMO).

Andrew

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/201403252101.52683.shai%40platonix.com.

[Michael Manfre]

I just read the deprecation timeline and the very brief mention of syncdb command and signals going away doesn't really seem to sufficiently detail the "side-effects" you mention. Anyone who hasn't read your email is going to be unpleasantly surprised. I also don't see any deprecation warnings in the code about this either.

> As for the general TDD problem, we do have the squashing functionality now so the --update problem is not quite as bad, but I'm not averse to something like it appearing in 1.8 (1.7 is too frozen now to add it IMO).

If I read your statement correctly, you are asking for some one to create a release blocker ticket because taking the position that TDD will be painful in 1.7 and might improve for 1.8 seems wrong.

Regards,

Michael Manfre

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1upm2g84E6JymtQUgrmBh0vSdfaLU5gNCC-EEopTGx%3D7AQ%40mail.gmail.com.

[Mark Lavin]

I don't see any meaningful notes about apps being required to ship migrations beginning with 1.9. I do see deprecation notes about the syncdb signals changing and the syncdb command itself is clearly deprecated. This legacy behavior is handled by sync_apps in the migrate command but there aren't any deprecation warnings raised when it executes. I don't see anything which points to this behavior being deprecated other than this thread.

[Andrew Godwin]

I'll update the deprecation document to include more direct information about DatabaseCreation and the legacy app sync method.

I'm not sure "TDD is a bit harder" is a release blocker - the TDD I do generally doesn't have that much model creation, and it's relatively easy to just run makemigrations to get new migrations for models, especially now it just does the whole project at once. Unless I can see this framed as being a significant impact to development process, I'm not going to hold up the release because of it...

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/262a32fd-fd5c-495b-a0ba-878372d3f7ea%40googlegroups.com.

[Florian Apolloner]

On Tuesday, March 25, 2014 9:12:42 PM UTC+1, Andrew Godwin wrote:

So, the functionality whereby you can have apps which do not use migrations (i.e. that use the old creation backends) is meant to go away in 1.9.

Uhm, strong -1 here unless you have really convincing arguments, I really wouldn't like to require migrations for all the models we have in our own testsuite. Especially since they are somewhat transient and don't have to care about migrations.

Florian

[Andrew Godwin]

We can possibly work around it for the test suite, but the intention was that all apps people build on Django would require migrations by that point, as then we can get rid of a whole load of legacy code. There's nothing to stop us just making something that just migrates models for tests without files on disk, but it'd still have to be making Migration objects in memory behind the scenes due to any potential dependency/foreignkey issues (the syncdb code only gets around this by doing everything in one huge transaction, which isn't the model that SchemaEditor supports).

Separate from all that, though, we can't promote adding random strings to MIGRATION_MODULES as the suggested way to "get around" migrations for tests. In my opinion, the whole point of migrations is that you know you have the same schema everywhere, and it's especially important you use them during tests.

Andrew

[Bernie Sumption]

we can't promote adding random strings to MIGRATION_MODULES as the suggested way to "get around" migrations for tests.

I agree, my workaround is a hack. It would be better to introduce a flag or setting designed specifically for this use case.

 

In my opinion, the whole point of migrations is that you know you have the same schema everywhere, and it's especially important you use them during tests.

If you view tests as a verification tool that is used before deployment or committing to check that the system is working as desired then yes, this is true. If you're practicing TDD, then tests are something else too - they're a development environment. They're the primary way of interacting with your code. Add a field, run tests. Rename it, run tests. Change its options, run tests.

The fact is, Django 1.6 and South supported this use case very well, Django 1.7 does not.

[Andrew Godwin]

When I practice TDD I write the test to spec, and then write the model and view code, so I usually have about the same amount of model changes as otherwise (as, having written the tests, I usually have a clearer idea of what fields I need). I agree that if you're incrementally writing tests on top of models, however, it could be extra verbose, but bear in mind that having hundreds of migrations on 1.7 is easily solved by a single `squashmigrations` command - much easier than on South (and performance on large migration sets should also be improved).

I'd be willing to keep the current contract of "things without a migrations directory don't get migrated", but I suspect you're doing things on apps that already have migrations (which makes my reticence to add a setting even bigger - if you "syncdb" an app with migrations to a main database, you have forever ruined the migration tracking on it). Would that work? Or do you want to do this on apps which are already several migrations along? (If that's the case, I suspect we might have to look at re-introducing --update, which is going to be near-impossible to do before release with the new creation system).

Andrew

--

You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ac7d9f74-691f-4ec8-8900-434f8e0a93ff%40googlegroups.com.

[Bernie Sumption]

I'd be willing to keep the current contract of "things without a migrations directory don't get migrated", but I suspect you're doing things on apps that already have migrations (which makes my reticence to add a setting even bigger - if you "syncdb" an app with migrations to a main database, you have forever ruined the migration tracking on it). Would that work? Or do you want to do this on apps which are already several migrations along? (If that's the case, I suspect we might have to look at re-introducing --update, which is going to be near-impossible to do before release with the new creation system).

You're right, I'm creating an app with migrations but want to be able to run tests without migrations when I'm mid-development. I also agree that if I'm running integration tests before deployment, or a non-developer is running tests before installing something written by someone else then those tests should use migrations, so it's not as simple as "don't use migrations in tests".

How about specifying whether a database is migrated as part of the database definition in the settings module? That way a database would either be migrated or not, and if a developer wants to set up an alternative non-migrated database for development there's a supported way of doing it.

Something like:

# in settings_bernie_dev.py

from .settings import *

DATABASES = {

    'default': {

        'ENGINE': 'django.db.backends.sqlite3',

        'NAME': os.path.join(BASE_DIR, 'testdb.sqlite3'),

        'USE_MIGRATIONS': False,

    }

}

Bernie     :o)

[Shai Berger]

I think a flag on the test command would be much more appropriate for the use-
cases you describe -- we don't want to send the message that disabling
migrations in tests is ok for a setup, although it may be ok for a given test-
run (basically, like running test for just a single app is ok).

Shai.

[Andrew Godwin]

If I need to take either of these options I'd tend towards Shai's one - we don't want to allow people to just disable migrations on a per-database basis, that's bound to get someone into trouble. That said, the flag is going to be weird to explain to people.

Just to establish a baseline, would you say that adding an "--update" command to makemigrations - which rolls the changes into existing migrations if it can - would solve your problem here? If that's the case, we can work towards that and figure out something simpler for the meantime.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1520941.Sdtj0HqCMr%40deblack.

[Bernie Sumption]

The problem with --update is that if overwrites the most recent migration, then it might be used to modify a committed and distributed migration, which is a Bad Thing. The flag would probably be useful to people with my use case, if they trust themselves to use the flag with care and remember to not use it immediately after committing. Was this the reason that you didn't carry --update from South to Django?

Entertainingly, I was about to defend my original proposal, but have just realised that for the last few days while we've been having this conversation, I have not been remembering to run makemigrations before committing, thereby falling into exactly the trap that you predicted my behaviour would produce!

Given that --update and my original proposal both have significant dangers if not used properly, my new thought is to write a "safe update migrations" script that used Git to delete all uncommitted migrations and runs makemigrations again. This could give me the best of both worlds - tests that "just work", no build up of many tiny migrations files, and no risk of trashing the committed migration history.

I'm coming to think that there's no change that could be made to Django core that would magically fix this without side effects, so perhaps it should be left to the community to create and share scripts that work for each VCS?

Bernie     :o)

[Bernie Sumption]

OK, it turns out that the "safe update migrations script" too simple to even qualify as a "script":

git clean myapp/migrations -f && python manage.py makemigrations

Perhaps the solution is to document this on the testing page as a solution to the "accumulation of many small migrations during development" problem?

Bernie     :o)

[Marc Tamlyn]

That script would be bad if you'd run any of those migrations against your development db (yes it should be "throwaway" or rebuildable but...)

Personally, I'm strongly in favour of Shai's suggestion and also in favour of --update, mainly as I like being able to capture (most) migrations has logical bits of work as it aids code review. My pull requests don't include every iteration of the code I wrote to get the final version, so why should my migrations?

M

--

You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e15603d7-ed14-4d18-b36b-925aaf824bce%40googlegroups.com.

[Bernie Sumption]

That script would be bad if you'd run any of those migrations against your development db (yes it should be "throwaway" or rebuildable but...)

I'd think the same could be said of --update? As I understand it, --update is the equivalent of deleting the most recent migration and recreating it. If you'd applied the most recent migration to your development database and then you --update that migration, you'd need to rebuild the database.

[Marc Tamlyn]

South's `--update` also rolled the previous migration back, changed it and then reapplied it to the current database.

M

On 28 March 2014 10:48, Bernie Sumption <ber...@berniecode.com> wrote:

That script would be bad if you'd run any of those migrations against your development db (yes it should be "throwaway" or rebuildable but...)

I'd think the same could be said of --update? As I understand it, --update is the equivalent of deleting the most recent migration and recreating it. If you'd applied the most recent migration to your development database and then you --update that migration, you'd need to rebuild the database.

--

You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f3e1b67b-f1fc-4c94-964f-23ec7fb27755%40googlegroups.com.

[Bernie Sumption]

South's `--update` also rolled the previous migration back, changed it and then reapplied it to the current database.

OK, in that case I can very much see how it's useful for people who develop against a persistent database. That's probably most people.

Anyway, the result of this thread for me is that I now consider my original request to be obsolete, as the "git clean" thing is a simple way of getting the behaviour I want for my own style of TDD without hacks.

Thanks for your time.

Bernie     :o)

[Andrew Godwin]

Yes, --update is very risky if you run it on migrations that are already committed and pushed, but the main reason I left it out of 1.7 was complexity (because makemigrations is now much more intelligent, updating and adding a foreignkey into a migration might introduce a new dependency or force a new migration anyway). Given that we have the ability to safely squash large numbers of small migrations down into one with squashmigrations and distribute that to fix the many-small-migrations problem, I considered it pretty low priority, though I have a rough idea of how I could make it work (I'd have to load up the autodetector with the existing migrations already loaded in as a halfway state and then run it from there, which should produce the right result).

Anyway, if you're retracting your original request, I'm happy to leave this for the 1.7 release; I don't think there's a good solution that Django core can implement effectively. This reminds me of when people used to ask me to automatically stop their developers writing conflicting migrations - the solution varies from company to company and often isn't technical but just education or communication.

Andrew

--

You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/8e3ecf3c-aa05-4e3d-b905-3260b093e046%40googlegroups.com.

[Harry Percival]

Am just working on updating my book on TDD to django 1.7 based on the beta.   Currently half-way thru, not run into any problems because I don't use migrations until a later chapter, but when I do I will run into the same problems Bernie mentions.

Will share more once I've finished the rewrites, but from what I see so far, I think I'd personally prefer to be able to run my tests without having to remember to call makemigrations every time.  some kind of customisable option?  either a command-line flag for the test runner, or maybe a setting in settings.py, eg MIGRATIONS_OFF_FOR_TESTS = True?

personally i'd like the default to be true, but i can appreciate other people will have different workflows / assumptions.

[Andrew Godwin]

No, there is no way to turn off migrations for tests - some of the core tests won't work without them turned on, in fact, and adding that option would be weird (why only tests? what would it do? how do you load data in now initial_data is gone?). The only complaint I've seen - the one that Bernie brought up originally, that it's "extra work" to run makemigrations before each test run - doesn't really hold water with me, as the alternative options mean you could run the tests and have them pass WITHOUT HAVING THE RIGHT MIGRATIONS - and so you're not testing part of your codebase.

Hell, you can alias together makemigrations and test if you want, that'll save you the typing. This might make a few more migrations than normal, but you could quickly point out that squashmigrations exists to deal with this problem and move on.

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a2897318-ec81-4acd-b201-2591042db1a0%40googlegroups.com.

[Harry Percival]

I suspect you're probably right.  Having to run makemigrations in between making changes to model code and running tests isn't the end of the world i suppose.  Will know better what I'm talking about  when I've actually got to that part of the book...

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/PWPj3etj3-U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1uoqPk47Zbd7fDA9pP3VyJ5xw%2B4UciBffD-jgKq88i4MJQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
------------------------------
Harry J.W. Percival
------------------------------
Twitter: @hjwp
Mobile:  +44 (0) 78877 02511
Skype:         harry.percival

[Harry Percival]

OK, I've now run into the problem IRL, and sure enough, it's different to what i'm used to.  Am trying to overcome my knee-jerk reactions of "why did it change! i hate it!".  ignoring that for a moment then:

One thing I did find surprising was that I'm going thru TDD in small steps like this:

    class Item(models.Model):
        pass

- create migration 001-initial, with Item and auto-id

    class Item(models.Model):
        text = models.TextField()

- create migration 0002, adding the item field.  here's where we get into the discussion of too many migrations, wanting to squash, etc etc etc.

leaving that aside for a moment, i was taken aback by this:

You are trying to add a non-nullable field 'text' to item without a default;
we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows)
 2) Quit, and let me add a default in models.py

I thought TextField did have a default, the empty string?

[Ryan Hiebert]

I thought TextField did have a default, the empty string?

Like every other field, the "default default" is None (NULL). 

[Harry Percival]

Ah, so the reason I was confused is because it *looks* like the default is the empty string, because that's what you get if you initialise an object, by default. But at the database level, the default for the column is NULL.  Is that right?

So, I realise we're getting sidetracked here, but,  how does this fit with the fact that `null=False` is the default for all Field types?

On 29 March 2014 23:37, Ryan Hiebert <ry...@ryanhiebert.com> wrote:

I thought TextField did have a default, the empty string?

Like every other field, the "default default" is None (NULL). 

--

You received this message because you are subscribed to a topic in the Google Groups "Django developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/PWPj3etj3-U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CABpHFHQK9UdEK6x4Mb3eDqXd5f%3D9egzJp5A21UZYXAQJw8HRDw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Shai Berger]

On Sunday 30 March 2014 15:08:17 Harry Percival wrote:
> Ah, so the reason I was confused is because it *looks* like the default is
> the empty string, because that's what you get if you initialise an object,
> by default. But at the database level, the default for the column is NULL.
> Is that right?
>
> So, I realise we're getting sidetracked here, but, how does this fit with
> the fact that `null=False` is the default for all Field types?
>

Simply: The "default default" is that fields are required.

But this is very deep in django-users territory.

Shai.

[Andrew Godwin]

You're roughly right, yes. String fields are a little odd, though, in that you can have them blank=True without null=True and then the default should be an empty string (Django's separation of blank and null irks me still) - the migrations should correctly detect this and insert blank strings for you then. If not, open a bug report!

Andrew

--

You received this message because you are subscribed to the Google Groups "Django developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/201403301513.23581.shai%40platonix.com.

[Harry Percival]

Just found out that you can make Django behave in the "old way" by just deleting the migrations folder when you first do your `startapp`, and then you can pretend migrations don't exist until you need them.  Don't think that's necessarily a good idea though...

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/PWPj3etj3-U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1urwhyZkfF9smfeeSOeS_8ejFdKmKo3X%3D4yfWd8DGU_%3DTA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Andrew Godwin]

Yes, you can - the "migrations or not" switch is currently "is there a migrations directory", but bear in mind this will eventually turn from "use the old way" to "ignore it completely", probably also in 1.9.

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CACFvh99EEyphjkE7%2BY7GawvkvhRNRrAVPOebj17T-OT8Z3HpBA%40mail.gmail.com.

[Harry Percival]

Well, having stayed up til 4AM on Monday night, I managed to get the whole book upgraded to Django 1.7, and published.  I've shared some thoughts on the migrations / tdd stuff we've been discussing, here:

http://www.obeythetestinggoat.com/book-upgraded-to-django-17.html

I'd be very interested in ppl's comments, particularly on the part of the book in which I introduce migrations:  am I saying the right things, am I giving people the right / best practice habits, etc.

http://chimera.labs.oreilly.com/books/1234000000754/ch05.html#_the_django_orm_amp_our_first_model

I think my overall conclusion is that the new migrations framework does involve a bit more pain, when doing tdd or when introducing newbies to django, but that that pain is worthwhile, since it's stuff that would come back and bite you later when you gloss over it (which you used to be able to do).

cheers all!
HP

[c...@epantry.com]

At the risk of reviving an old topic, I wanted to add one significant point in favor of (and mitigation for) running tests with migrations disabled: Speed.

Creating a test DB in sqlite for our project (~100 database tables) takes approximately 1-2 minutes on most developer machines. 1-2 minutes of idle time to run any test was just unacceptable so we disabled migrations by setting fake migrations in MIGRATION_MODULES and brought the test DB creation time down to about 5 seconds (!!).

However the risk of committing invalid code because someone forgot to makemigrations is real. We've addressed it by only overriding migrations on our local test settings and still having migrations run on our CI server. We have our CI server use settings.test, while developers running tests on their local machine use settings.test_local which is just:

from settings.test import *

MIGRATION_MODULES = ((app, '%s.fake_migrations' % app) for app in INSTALLED_APPS)

This seems to get us the best of both worlds. I was surprised to read through this thread and not see other mentions of the performance implications of running migrations on every test run. I'd be curious to hear if this has been a bottleneck for other teams.

-Chris

On Tuesday, March 25, 2014 10:21:51 AM UTC-7, Bernie Sumption wrote:

Hi Django devs,

I've just started a new project in 1.7b, and the development experience working with unit tests on models is more complicated than it was in 1.6. It's all down to how the throwaway test databases are created. In 1.6, the create_test_db function "Creates a new test database and runs syncdb against it." In 1.7, it runs "migrate".

While generally speaking, migrate is the new syncdb, this behaviour is not ideal for tests. In 1.6 "syncdb" created a database reflecting the current state of the models in models.py. "migrate" creates a database reflecting the state of the models at the last time makemigrations was run. If you're doing TDD and constantly making small changes to your models then runnning unit tests, you have to run makemigrations before each test run to get your tests to work. You therefore end up with many tiny migration files representing the minute-by-minute history of development.

I came up with a pretty effective workaround that is working for me, but I thought I'd post this here as others are sure to encounter this issue, and I think that it makes more sense for the behaviour produced by this workaround to be the default for running tests.

If makemigrations has not yet been run, the "migrate" command treats an app as unmigrated, and creates tables directly from the models just like syncdb did in 1.6. I defined a new settings module just for unit tests called "settings_test.py", which imports * from the main settings module and adds this line:

MIGRATION_MODULES = {"myapp": "myapp.migrations_not_used_in_tests"}

Then I run tests like this:

DJANGO_SETTINGS_MODULE="myapp.settings_test" python manage.py test

This fools migrate into thinking that the app is unmigrated, and so every time a test database is created it reflects the current structure of models.py.

So my feature request is as follows:

If the new behaviour is by design and considered desirable, then it is a big change from the previous version and should be prominently documented in the migrations and testing pages, along with the workaround. I'm happy to write this documentation if that's the way you want to go.

However, if the new behaviour is not by design but just a by-product of the new migrations feature, I suggest making the workaround the default behaviour. I don't (yet!) know enough about Django internals to volunteer for this however.

Thanks for your time,

Bernie     :o)

[Andrew Godwin]

I agree that migrations are slower than syncdb - that's perhaps the only thing worse about them - but the reason we plan to deprecate the other methods is code simplicity; migrations does not share almost any code with the old DatabaseCreation backends, and if we don't deprecate it we're going to end up maintaining two creation backends for every database driver, which isn't going to go well.

There's perhaps something to be said for an option where tests make an in-memory set of migrations from the autodetector and an empty state and run them immediately - somewhat replicating the syncdb process while still using the same code paths - but I haven't had the time to investigate how well this would work yet (there are some migration decisions that would need defaults inserted).

I think the end result would be an alternative test runner that you could switch to if you wanted this behaviour (and a mixin with the actual logic or something similar so it's easy to incorporate into other test runners).

Andrew

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/371a1e36-b752-4dad-8bc8-08c8b643e9c4%40googlegroups.com.

[Tim Graham]

Andrew, I've thought of something similar to the in-memory migrations idea you've proposed. It would be great not to have to add and maintain migrations for all of the apps in Django's test suite. Do you think you might be able to investigate this solution in the next month or so before 1.8 alpha? I think we need a solution in 1.8 if we are to complete #22340 - Legacy Table Creation Methods Not Properly Deprecated (otherwise, we can again postpone that deprecation).

[Andrew Godwin]

Hi Tim,

I can have a look, but I can't be certain about hitting any deadlines. I do want to get that deprecation in, though...

Did you want it with a view to us being able to drop that in for tests rather than making migrations for every test app, I presume?

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/af39502f-0e24-42bb-a661-daa263836d79%40googlegroups.com.

[Tim Graham]

Yes. Claude has worked on the deprecation in https://github.com/django/django/pull/3220 but it requires adding more migrations to our test suite and he noted that each migration we add to Django's test suite adds up to ~3.5 seconds to the run time of the test suite. He also worked on some speed ups to mitigate this in https://github.com/django/django/pull/3484 but there are some unsolved issues.

[Chris Clark]

For what it's worth, for my team, it just comes down to speed (so a single in-memory migration per app is great, and so is speeding up migrations overall).

I have a stopwatch at my desk and did some quick (unscientific) timing of creating the test DB for my own project:

Without migrations (MIGRATION_MODULES = dict((app, '%s.fake_migrations' % app) for app in INSTALLED_APPS))

4.94s

4.37s

5.51s

4.60s

With migrations

80.44s

66.33s

69.09s

I tried to squash migrations (52 migrations down to 7) and get timings for that, but creating test DB never completes and I don't have time to chase it down what's going on.

Running tests in PyCharm's debug mode takes over 10 minutes (!!) to create the test DB, effectively eliminating our ability to use step-through debugging of unit tests.

-Chris

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/PWPj3etj3-U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a9e3edb6-ed6c-48c3-99f7-1279adbec302%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Chris Clark │ ePantry │ 617-571-7327

1770 Union St, San Francisco, CA

[Claude Paroz]

On Friday, December 19, 2014 6:30:32 PM UTC+1, Tim Graham wrote:

Yes. Claude has worked on the deprecation in https://github.com/django/django/pull/3220 but it requires adding more migrations to our test suite and he noted that each migration we add to Django's test suite adds up to ~3.5 seconds to the run time of the test suite. He also worked on some speed ups to mitigate this in https://github.com/django/django/pull/3484 but there are some unsolved issues.

I think that the first mentioned patch I worked on (PR 3220) shows that it should be possible to use the new schema infrastructure without requiring migrations, at least for apps that don't depend on other migrated apps. Tell me if I miss anything.

But surely, I'd really like to solve first the speed issue of migrations (#23745, PR 3484). Chris, if you have the opportunity to test that patch with your database, it would be great. The way forward for this patch is to first test the "if app_label == settings.AUTH_USER_MODEL and ignore_swappable:" line which is currently not tested and is probably not addressed by my patch. Andrew, do you remember why you introduced that part of the code?

Claude

[Andrew Godwin]

Clause, I believe that line is to allow people to run makemigrations when AUTH_USER_MODEL points to an app that doesn't yet have migrations; before, it would fail hard, as the AUTH_USER_MODEL was not in the migrate state and so nothing could use it, and you couldn't run makemigrations to create it as it would error out.

I'd test variations of that to see if your patch still works; if not, we'll need a workaround again.

Andrew

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1f5f37b2-5309-4af6-8b3c-aa76eee3bde1%40googlegroups.com.

[Bernard Sumption]

A proposition: the problem isn't that migrations take 80 seconds, it's that this 80 seconds is repeated every time a developer runs tests. How about serialising the structure and content of the database to disk after applying migrations, then whenever you need a migrated test database, load it from disk unless the migration files have been altered since the serialised version was created?

Bernie     :o)

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/PWPj3etj3-U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1urVDjwidnzk-%3DrEhvjYCtivcVCrtgpdserQ6B36zMdrxA%40mail.gmail.com.

[Tim Graham]

I did some research, and I can hit the logic in question,

if app_label == settings.AUTH_USER_MODEL and ignore_swappable:

   continue

... on both master and with Claude's patch if I follow the steps in this comment: https://code.djangoproject.com/ticket/22563#comment:8

Andrew, first you added an error message to say that changing AUTH_USER_MODEL after creating an initial migration was invalid:
https://github.com/django/django/commit/fc974313b85da932a70f1f993b33207d78d31831

and then someone reporting hitting that error with a false positive and you changed it to the current version:
https://github.com/django/django/commit/5182efce8d73ec552a1d7f3e86a24369bb261044

Your comment was "I've committed a patch that suppresses the error in that one case (where we know it's safe to do so), and I can now switch AUTH_USER_MODEL even midway through a migration set without errors."

Maybe this helps refresh your memory?

[Aymeric Augustin]

There's django-admin test --keepdb for this purpose (in Django 1.8).

-- 

Aymeric.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CALj4E%2B%2BtS%2BJ28WT51zyrqFB32zr20HU7OK4%2BiiSOdPtsky0uOg%40mail.gmail.com.

[Tim Graham]

In pull request 3791, I started working on the code removals for Django 1.9 to better understand how the database/migrations code will look and what APIs we still need to deprecate. The good news is that we can keep support for apps without migrations (at least for Django's test suite) without adding any additional code or using any deprecated database creation APIs (as Claude suggested earlier). If you look at the commit "Removed supported for syncing apps without migrations" in that PR, you'll see "Run the syncdb phase" remains in the migrate management command, but now it'll only be executed when testing (and it's also quite a bit simplified due to other removals like support for initial data). It seems like it should be pretty simple to implement something similar to the old SOUTH_TESTS_MIGRATE setting in order to allow disabling migrations for all apps when testing if we wanted to go that route. I think the remaining deprecations to move forward with ticket #22340 (Legacy Table Creation Methods Not Properly Deprecated) are straightforward and will leave a comment there about it.

[Preston Timmons]

At work we use a --no-migrate flag on our test runner. It's helpful when 1) we're developing a new app but the models aren't nailed down yet, and 2) it speeds up the test suite time significantly.

The --keepdb option is sufficient for scenario 2, but doesn't do much for scenario 1. Admittedly though, scenario 1 isn't as common an occurrence.

[Pradeek J]

I'm facing the same speed issue here. Like Andrew mentioned above, if initial data is going to be removed and data migrations are the way forward, even having an option to skip migrations is a problem because we'd need that data to be populated. Is there something planned for this for Django 1.9? 

[Tim Graham]

I don't know of any tickets or plans to address this.

[Kevin Tran]

I have found a good alternative to the old initial data feature.  It's called django-update-initial-data.

https://github.com/ambitioninc/django-dynamic-initial-data

MySQL compatible version:

https://github.com/minervaproject/django-dynamic-initial-data/tree/mysql

It's actually better than the old initial data feature because, "Django Dynamic Initial Data also handles when objects are deleted from initial data, a feature that Django's initial data fixture system lacks.".

[Dylan Young]

This is an old issue that (as far as I can see) was never resolved. 

My proposed solution would be a flag to the test command to automatically `makemigrations && migrate`.  When this behaviour is enabled, it would also roll back any generated migrations at the end of the test run.   The flag might not be necessary (could be default behaviour of `test`)

This way you get the benefits of all worlds:

1) uncommitted migration files will be clearly visible after the test run (for those inclined to forget to makemigrations).

2) The extraneous non-migration creation code can be removed from Django (probably already has been at this point).

3) Tests are always run against the current state of models

4) Tests are always run at current state of migrations (because it generates the migrations). 

As an extra benefit, it makes testing between branches *so much* easier because you don't have to worry about rolling back migrations on the test_db (a PITA as far as I can see).

Regarding the last, is there a flag to make migrate target the test_db(s)?

Best,

Casey

[Adam Johnson]

Hi Dylan,

In my experience, high availability deployments often need to commit model changes without migration changes. For example, when removing a (nulable) field, the best approach is to:

1. Remove the field from Django
2. Test
3. Deploy
4. Run makemigrations to generate the migration file that removes it from the database
5. Test
6. Deploy
   

The test run in step 2 should *not* have any automated migrations built. The tests should confirm that removing the field from Django without removing it from the DB safe.

If you want to build your suggested behaviour (makemigrations before every test run) you can do it by:

1. Creating a custom test runner that subclasses DiscoverRunner
2. Overriding its setup_databases() method to do call_command('makemigrations') before super()
3. Catching the names of those made migrations, and overriding teardown_databases() to delete them (or maybe not?)

I'd be interested to know how that works! I suspect it will be fine for smaller projects, but for larger projects that care about preventing downtime during deployment, it will backfire.

As an extra benefit, it makes testing between branches *so much* easier because you don't have to worry about rolling back migrations on the test_db (a PITA as far as I can see).

If you're using the default behaviour, the test DB is generated on every run. The `--keepdb` flag is optional and can be used to speed up test runs on the same branch but you should run tests without it after switching. I agree it can be a pain.

Maybe we could have a guard on that flag: detect if git is used, and the branch has changed since last run, and then warn or just hard recreate? If you'd be interested in exploring this, it can also be tested by subclassing the migrate command. Maybe make a ticket?

Regarding the last, is there a flag to make migrate target the test_db(s)?

Not that I know of but I did something similar on one project where I had a management command to create a test runner and call its setup_databases() method. That should be sufficient!

Thanks,

Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2aee60d5-e96b-4c96-8f87-150d9de46215%40googlegroups.com.

--

Adam

[Dylan Young]

Thanks for the tips; I'll take a look at those code paths.

I'm interested in your use case: why do you need the model changes without the migrations? I.e. why would you want to be in a state that's inconsistent between the ORM and the DB?  Is it just so you can recover more easily if something goes wrong?

Recent versions of Django make it easier to modify the test command,  so I may take a look at spinning up an external package for this.

You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/PWPj3etj3-U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM3SKV07PPY3fBxyii4MmQHHAyqB6LJERugNr8nupr-FnQ%40mail.gmail.com.

[Ryan Hiebert]

On Sat, Sep 14, 2019 at 10:09 AM Dylan Young <dylanyou...@gmail.com> wrote:

why do you need the model changes without the migrations? I.e. why would you want to be in a state that's inconsistent between the ORM and the DB?  Is it just so you can recover more easily if something goes wrong?

I expect that its to allow for shipping the ORM code changes separately from the database migrations. That was my use-case anyway. If the migration needed to run first, then I wanted to deploy the migrations by themselves, then the ORM changes. Sometimes I'd want to ship the ORM changes first, and the migrations later (though usually just running them after deploy was effective).

Then when Heroku added release phase, which can automatically run migrations immediately before a release is deployed, I really wanted to do something better. So we created django-safemigrate (https://github.com/aspiredu/dango-safemigrate), which allows you to mark your deploys as safe to run before, after, or at any time relative to the code changes to the ORM they match. This allows my team to avoid splitting up the ORM changes from the migrations, which keeps matching things together in the commit history. It's been working fairly well for us, barring the occasional human error mis-marking the migrations. Feel free to give it a try, to see if its also a good workflow for you!

[Adam Johnson]

I'm interested in your use case: why do you need the model changes without the migrations? I.e. why would you want to be in a state that's inconsistent between the ORM and the DB?  Is it just so you can recover more easily if something goes wrong?

It's not that one *wants* to be in that state, it's just inevitable with the goal of zero downtime. With reasonable traffic on even one webserver, it's inevitable there's a gap between the running of migrations and the updating of the webserver to the new code. In this case it's best to make sure both old and new code versions are compatible with the new version of the database.

There's plenty out there on the topic. A post I found useful recently was this one from Braintree covering the different ways to help zero downtime deployments PostgreSQL: https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680

Thanks,

Adam

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAPGJNu54auVLyKALu4JsHnyyRDFs1mDa%3DPcBV96Dz8w-0n_CQQ%40mail.gmail.com.

--

Adam

[Adam Johnson]

(Fixed Ryan's link: https://github.com/aspiredu/django-safemigrate )

--

Adam

[Ryan Hiebert]

On Sat, Sep 14, 2019, 12:44 Adam Johnson <m...@adamj.eu> wrote:

(Fixed Ryan's link: https://github.com/aspiredu/django-safemigrate )

Doh. Thanks.

[Dylan Young]

Thanks all!  I remember reading about the zero downtime deployments many months back and just needed a refresher. I appreciate it. 

To sum up for myself: we need to test at every step of the change that the model changes are in fact backwards compatible with the old state of the DB. 

Makes sense. 

Thanks again!

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM0rbbPD%2BNxfbj_QkwxgbGMb01tV_%3DWfozBbYVjwKGR4YQ%40mail.gmail.com.

[Dan Davis]

There is no such flag, at least not in 1.11.  I wrote my own "migratetest" and "cleandb" commands because my DBAs don't let me drop and recreate databases.   It is a simple matter if you use the testsuite, but it would probably be better development to call create_test_db manually and such.