[Django] #23090: Squashed migrations not used always when testing
Django
-------------------------------+----------------------
Reporter: anonymous | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7-rc-1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+----------------------
I've squashed migrations in my apps, did some fixing and to test them on
clean database I was just running tests. When having old and squashes in
migrations folders some part of django-admin test ... resulted in
KeyErrors like the one below. Some did pass and used the squashed
migrations.
{{{
Traceback (most recent call last):
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/coverage/execfile.py", line 104, in run_python_file
exec_code_object(code, main_mod.__dict__)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/coverage/backward.py", line 93, in exec_code_object
exec(code, global_map)
File "/tmp/buku-test-virtualenv-908c498ca6f66e231f0be70fcb6b51b2/bin
/django-admin.py", line 5, in <module>
management.execute_from_command_line()
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/core/management/__init__.py", line 385, in
execute_from_command_line
utility.execute()
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/core/management/commands/test.py", line 50, in
run_from_argv
super(Command, self).run_from_argv(argv)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/core/management/commands/test.py", line 71, in execute
super(Command, self).execute(*args, **options)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/core/management/base.py", line 337, in execute
output = self.handle(*args, **options)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django_jenkins/management/commands/jenkins.py", line 98, in
handle
failures = test_runner.run_tests(test_labels)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/test/runner.py", line 147, in run_tests
old_config = self.setup_databases()
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django_jenkins/runner.py", line 127, in setup_databases
return super(CITestSuiteRunner, self).setup_databases()
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/test/runner.py", line 109, in setup_databases
return setup_databases(self.verbosity, self.interactive, **kwargs)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/test/runner.py", line 299, in setup_databases
serialize=connection.settings_dict.get("TEST_SERIALIZE", True),
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/db/backends/creation.py", line 374, in create_test_db
test_flush=True,
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/core/management/__init__.py", line 115, in call_command
return klass.execute(*args, **defaults)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/core/management/base.py", line 337, in execute
output = self.handle(*args, **options)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/core/management/commands/migrate.py", line 63, in handle
executor = MigrationExecutor(connection,
self.migration_progress_callback)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/db/migrations/executor.py", line 16, in __init__
self.loader = MigrationLoader(self.connection)
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/db/migrations/loader.py", line 48, in __init__
self.build_graph()
File "/tmp/buku-test-virtualenv-
908c498ca6f66e231f0be70fcb6b51b2-5/lib/python3.3/site-
packages/django/db/migrations/loader.py", line 216, in build_graph
normal[child_key].dependencies.remove(replaced)
KeyError: ('buku_notification', '0003_auto_20140424_1540')
}}}
Where buku_notification.0003_auto_20140424_1540 is one of migrations that
got squashed, it listed in "replaces" of its squash migration. I can't
reproduce it with my second commit that deleted all migrations that got
squashed.
--
Ticket URL: <https://code.djangoproject.com/ticket/23090>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: anonymous | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7-rc-1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by riklaunim):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:1>
Django
Reporter: anonymous | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7-rc-1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by timo):
More details like a minimal project that reproduces the error would be
really helpful in debugging the issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:2>
Django
Reporter: anonymous | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7-rc-1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by riklaunim):
Will try to provide.
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:3>
Django
Reporter: anonymous | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7-rc-1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by riklaunim):
I've traced the problem a bit and it looks like it's not the order of
migrations to be the source of the problem.
Step to reproduce:
- squash few migrations in an application and apply all migrations needed
- remove squashed migrations
- make a new migration and apply it
- squash (old squash + new migration)
- (you may remove squashed migrations too)
- look at the '''replaces''' list of that squash and try to run test so
that it will be applied to a clean db
squashmigrations will copy '''replaces''' contents from squashed
migrations. Migrations listed there may do not exist and migrating will
result in key errors (although sometimes not). The squashmigrations
command and documentation should probably mention that '''replaces'''
should be removed when squashed migrations are removed from the
application
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:4>
Django
Reporter: anonymous | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.7-rc-1
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timo):
* type: Uncategorized => Bug
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted
Comment:
Andrew, can you confirm the desired behavior?
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:5>
Django
Reporter: anonymous | Owner: andrewgodwin
Type: Bug | Status: assigned
Component: Migrations | Version: 1.7-rc-1
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by andrewgodwin):
* status: new => assigned
* owner: nobody => andrewgodwin
Comment:
Yes, riklaunim is correct that you're meant to remove replaces from a
squashed migration before re-squashing (you can't squash a squashed
replacement migration, so you have to fully transition it to a normal
migration first).
It's just a doc fix. I'll do it.
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:6>
Django
Reporter: anonymous | Owner: andrewgodwin
Type: Bug | Status: assigned
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by charettes):
* cc: charettes (added)
Comment:
I don't know if it's already the case but would it be possible to prevent
the migration framework from squashing replacement migrations by looking
up if they have a `replaces` attribute?
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:7>
Django
Reporter: anonymous | Owner: andrewgodwin
Type: Bug | Status: assigned
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by andrewgodwin):
That is, indeed, exactly what I am implementing right now.
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:8>
Django
Reporter: anonymous | Owner: andrewgodwin
Component: Migrations | Version: 1.7-rc-1
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Andrew Godwin <andrew@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"8ebd6d35d77c1c535a9ffe8151445ad853845b61"]:
{{{
#!CommitTicketReference repository=""
revision="8ebd6d35d77c1c535a9ffe8151445ad853845b61"
Fixed #23090: Document and enforce not double-squashing migrations
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:9>
Django
Reporter: anonymous | Owner: andrewgodwin
Type: Bug | Status: closed
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Andrew Godwin <andrew@…>):
In [changeset:"3deddc2fdf1f47a9564b6d39ee53dc5bdd944dc2"]:
{{{
#!CommitTicketReference repository=""
revision="3deddc2fdf1f47a9564b6d39ee53dc5bdd944dc2"
[1.7.x] Fixed #23090: Document and enforce not double-squashing migrations
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23090#comment:10>