[Django] #31834: Cannot sqlmigrate on Django migrations that change unique_together
Django
-------------------------------------+-------------------------------------
Reporter: Brіаn Lаі | Owner: nobody
Type: Bug | Status: new
Component: Database | Version: 3.0
layer (models, ORM) |
Severity: Normal | Keywords: sqlmigrate
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Somewhat similar to #26624, but forward sqlmigrate, reproduced on Django
3.0.8 and Postgres 10.x.
If a model with a unique_together of
{{{
unique_together = [
(a, b)
]
}}}
and then this becomes
{{{
unique_together = [
(a, b, c)
]
}}}
, although the makemigrations command successfully generates the
corresponding AlterUniqueTogether() operation, the operation cannot be
inspected with the sqlmigrate command.
Reproduce bug with the following (the project may also be attached)
{{{
class Model1(models.Model):
id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=255, unique=True)
class Model2(models.Model):
id = models.IntegerField(primary_key=True)
link_id = models.CharField(max_length=255)
settings = models.ForeignKey(Model1, on_delete=models.CASCADE)
class Model3(models.Model):
id = models.IntegerField(primary_key=True)
order = models.IntegerField()
model_2 = models.ForeignKey(Model2, on_delete=models.CASCADE)
direct = models.BooleanField(default=True)
class Meta:
unique_together = [
('model_2', 'order')
# To get the bug, change this to ('model_2', 'order',
'direct') and sqlmigrate the migration that this creates.
]
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31834>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: Brіаn Lаі | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: sqlmigrate | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* Attachment "foo.tar.gz" added.
Bug demo project
Django
-------------------------------------+-------------------------------------
Reporter: Brіаn Lаі | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: sqlmigrate | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
New description:
Somewhat similar to #26624, but forward sqlmigrate, reproduced on Django
3.0.8 and Postgres 10.x.
If a model has a unique_together of
{{{
unique_together = [
(a, b)
]
}}}
and it becomes
{{{
unique_together = [
(a, b, c)
]
}}}
, although the makemigrations command successfully generates a new
migration with the corresponding AlterUniqueTogether() operation, the
migration cannot be inspected with the sqlmigrate command.
Reproduce bug with the following (the project may also be attached)
{{{
class Model1(models.Model):
id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=255, unique=True)
class Model2(models.Model):
id = models.IntegerField(primary_key=True)
link_id = models.CharField(max_length=255)
settings = models.ForeignKey(Model1, on_delete=models.CASCADE)
class Model3(models.Model):
id = models.IntegerField(primary_key=True)
order = models.IntegerField()
model_2 = models.ForeignKey(Model2, on_delete=models.CASCADE)
direct = models.BooleanField(default=True)
class Meta:
unique_together = [
('model_2', 'order')
# To get the bug, change this to ('model_2', 'order',
'direct') and sqlmigrate the migration that this creates.
]
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Brіаn Lаі | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: sqlmigrate | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> Somewhat similar to #26624, but forward sqlmigrate, reproduced on Django
> 3.0.8 and Postgres 10.x.
>
> If a model has a unique_together of
>
> {{{
> unique_together = [
> (a, b)
> ]
> }}}
>
> and it becomes
>
> {{{
> unique_together = [
> (a, b, c)
> ]
> }}}
>
> , although the makemigrations command successfully generates a new
> migration with the corresponding AlterUniqueTogether() operation, the
> migration cannot be inspected with the sqlmigrate command.
>
> Reproduce bug with the following (the project may also be attached)
>
> {{{
> class Model1(models.Model):
> id = models.IntegerField(primary_key=True)
> name = models.CharField(max_length=255, unique=True)
>
> class Model2(models.Model):
> id = models.IntegerField(primary_key=True)
> link_id = models.CharField(max_length=255)
> settings = models.ForeignKey(Model1, on_delete=models.CASCADE)
>
> class Model3(models.Model):
> id = models.IntegerField(primary_key=True)
> order = models.IntegerField()
> model_2 = models.ForeignKey(Model2, on_delete=models.CASCADE)
> direct = models.BooleanField(default=True)
>
> class Meta:
> unique_together = [
> ('model_2', 'order')
> # To get the bug, change this to ('model_2', 'order',
> 'direct') and sqlmigrate the migration that this creates.
> ]
> }}}
New description:
Somewhat similar to #26624, but forward sqlmigrate, reproduced on Django
3.0.8 and Postgres 10.x.
If a model has a unique_together of
{{{
unique_together = [
(a, b)
]
}}}
and it becomes
{{{
unique_together = [
(a, b, c)
]
}}}
, although the makemigrations command successfully generates a new
migration with the corresponding AlterUniqueTogether() operation, the
migration cannot be inspected with the sqlmigrate command.
Reproduce bug with the following (the project may also be attached)
{{{
class Model1(models.Model):
id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=255, unique=True)
class Model2(models.Model):
id = models.IntegerField(primary_key=True)
link_id = models.CharField(max_length=255)
settings = models.ForeignKey(Model1, on_delete=models.CASCADE)
class Model3(models.Model):
id = models.IntegerField(primary_key=True)
order = models.IntegerField()
model_2 = models.ForeignKey(Model2, on_delete=models.CASCADE)
direct = models.BooleanField(default=True)
class Meta:
unique_together = [
('model_2', 'order')
# To get the bug, change this to ('model_2', 'order',
'direct') and sqlmigrate the migration that this creates.
]
}}}
The error shown in the terminal is:
{{{
Traceback (most recent call last):
File "manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "/lib/python3.6/site-packages/django/core/management/__init__.py",
line 401, in execute_from_command_line
utility.execute()
File "/lib/python3.6/site-packages/django/core/management/__init__.py",
line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/lib/python3.6/site-packages/django/core/management/base.py", line
328, in run_from_argv
self.execute(*args, **cmd_options)
File "/lib/python3.6/site-
packages/django/core/management/commands/sqlmigrate.py", line 30, in
execute
return super().execute(*args, **options)
File "/lib/python3.6/site-packages/django/core/management/base.py", line
369, in execute
output = self.handle(*args, **options)
File "/lib/python3.6/site-
packages/django/core/management/commands/sqlmigrate.py", line 65, in
handle
sql_statements = executor.collect_sql(plan)
File "/lib/python3.6/site-packages/django/db/migrations/executor.py",
line 225, in collect_sql
state = migration.apply(state, schema_editor, collect_sql=True)
File "/lib/python3.6/site-packages/django/db/migrations/migration.py",
line 124, in apply
operation.database_forwards(self.app_label, schema_editor, old_state,
project_state)
File "/lib/python3.6/site-
packages/django/db/migrations/operations/models.py", line 525, in
database_forwards
getattr(new_model._meta, self.option_name, set()),
File "/lib/python3.6/site-packages/django/db/backends/base/schema.py",
line 380, in alter_unique_together
self._delete_composed_index(model, fields, {'unique': True},
self.sql_delete_unique)
File "/lib/python3.6/site-packages/django/db/backends/base/schema.py",
line 414, in _delete_composed_index
", ".join(columns),
ValueError: Found wrong number (0) of constraints for bar_model3(model_2,
order)
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:2>
Django
Reporter: Brіаn Lаі | Owner: nobody
Component: Migrations | Version: 3.0
Severity: Normal | Resolution: duplicate
Keywords: sqlmigrate | Triage Stage: Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* status: new => closed
* resolution: => duplicate
* component: Database layer (models, ORM) => Migrations
Comment:
I think we can mark this as a duplicate of #26624, because they have the
same cause. In both scenarios a constraint doesn't exist in the database
and cannot be introspected, `sqlmigrate` works properly if you first apply
a migration which adds a constraint.
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:3>
Django
Reporter: Brіаn Lаі | Owner: nobody
Component: Migrations | Version: 3.0
Severity: Normal | Resolution: duplicate
Keywords: sqlmigrate | Triage Stage: Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Brіаn Lаі):
I guess so. However you choose to triage and/or fix the item, I hope I am
providing additional context by saying this bug can be reproduced with
SQLite, whereas #26624 claims they could not.
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:4>
Django
Type: Bug | Status: new
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Jacob Walls):
* resolution: duplicate =>
* stage: Unreviewed => Accepted
* status: closed => new
Comment:
#26624 was wontfix'ed by scoping it to the deprecated `index_together`,
but since this report is for `unique_together`, it should be valid now.
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:5>
Django
Reporter: Brіаn Lаі | Owner: Bishnu Bhattarai
Type: Bug | Status: assigned
Severity: Normal | Resolution:
Keywords: sqlmigrate | Triage Stage: Accepted
Easy pickings: 0 | UI/UX: 0
Changes (by Bishnu Bhattarai):
* owner: nobody => Bishnu Bhattarai
* status: new => assigned
Comment:
I am interested in this ticket, I want to assign this ticket myself.
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:6>
Django
Reporter: Brіаn Lаі | Owner: Bishnu Bhattarai
Type: Bug | Status: assigned
Component: Migrations | Version: 3.0
Severity: Normal | Resolution:
Keywords: sqlmigrate | Triage Stage: Accepted
Easy pickings: 0 | UI/UX: 0
Comment (by Bishnu Bhattarai):
I have created a PR for this issue and have fixed from manual testing,
But writing test case for this type of fix looks a bit complicated, Could
you please provide me some example of test case for sqlmigrate and in
which file is the best place to keep this test.
here is the PR https://github.com/django/django/pull/19590
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:7>
Django
Reporter: Brіаn Lаі | Owner: Bishnu Bhattarai
Type: Bug | Status: assigned
Component: Migrations | Version: 3.0
Severity: Normal | Resolution:
Keywords: sqlmigrate | Triage Stage: Accepted
Easy pickings: 0 | UI/UX: 0
Changes (by Bishnu Bhattarai):
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:8>
Django
Reporter: Brіаn Lаі | Owner: Bishnu Bhattarai
Type: Bug | Status: assigned
Component: Migrations | Version: 3.0
Severity: Normal | Resolution:
Keywords: sqlmigrate | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Changes (by Wes P.):
* needs_tests: 0 => 1
Comment:
Hi Bishnu. Thanks for your work on this ticket. If you are still looking
for where to write your test, the file
`djangoproject/tests/migrations/test_commands.py` has the existing
`sqlmigrate` tests as well as others. There a lot of good examples to
follow in that file which include work on tickets for a different command.
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:9>
Django
Reporter: Brіаn Lаі | Owner: Bishnu Bhattarai
Type: Bug | Status: assigned
Component: Migrations | Version: 3.0
Severity: Normal | Resolution:
Keywords: sqlmigrate | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Comment (by Bishnu Bhattarai):
update you soon.
--
Ticket URL: <https://code.djangoproject.com/ticket/31834#comment:10>