Let's say I have a Django 1.7 app called `sample`:
1. Create a model `Bar`.
2. Create a model `Foo` with `bar = models.ForeignKey(Bar, blank=True,
null=True)`
3. `makemigrations` (migration `0001`)
4. Rename `Foo` to `OldFoo`.
5. `makemigrations`, say yes to the rename prompt (migration `0002`)
6. Create new model `Foo`, which also has `bar = models.ForeignKey(Bar,
blank=True, null=True)`
7. `makemigrations` (migration `0003`)
8. `migrate`
When `migrate` hits `0003`, it throws this error:
{{{django.db.utils.OperationalError: index sample_foo_4350f7d0 already
exists}}}
You may notice that `sqlmigrate sample 0001` and `sqlmigrate sample 0003`
have the same line in both of them:
{{{CREATE INDEX sample_foo_4350f7d0 ON "sample_foo" ("bar_id");}}}
In my production case, I actually had no problems on servers, because
South had created the index with a different name. When I renamed the
models and added a field, the new index did not collide with the old one.
However, our test suite started failing, because it would run the
migrations from the ground up, exposing the above bug.
I haven't decided on a workaround yet, but I thought I'd bring this to
your attention. I might have to inject a migration that renames the index
created in `0001`, or something to that effect.
The attached test case has a project `dj17test` in the state after having
performed all of the above steps.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
It is worth noting that if you use postgres instead, the error is
`django.db.utils.ProgrammingError: relation "sample_foo_4350f7d0" already
exists`.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:1>
* stage: Unreviewed => Accepted
Comment:
Haven't reproduced but the issue seems legit from the report.
I guess the `RenameModel` operation should also rename indexes generated
from the original model name.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:2>
* owner: nobody => tomviner
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:3>
Comment (by tomviner):
I've found an associated case where a field with {{{db_index=True}}}, is
renamed, and another db_indexed field created with the original's name.
Just discussed this with @andrewgodwin, I've agreed to write a test case
for the field rename case, and then post about a solution on django-dev,
with his preference being to make index names randomised.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:4>
Comment (by tomviner):
I've added a schema test case:
https://github.com/tomviner/django/compare/ticket/23577
This should currently fail on all backends except sqlite (which doesn't
rename fields - it drop/creates the whole table). When this bug is fixed,
it should pass for all backends, for that reason I haven't added any
unittest.skip decorator.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:5>
* owner: tomviner =>
* status: assigned => new
Comment:
After [https://groups.google.com/forum/#!topic/django-
developers/7pMxh2JF1lo a discussion on django-dev] it makes sense to leave
the resolution of this issue to
[https://github.com/django/deps/blob/indexes/drafts/indexes.rst Marc
Tamlyn's Advanced indexes DEP], which he says will involve user
specified/reproducible index names, as well as index renaming.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:6>
* cc: bugs@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:7>
Comment (by ris):
Replying to [comment:4 tomviner]:
> I've found an associated case where a field with {{{db_index=True}}}, is
renamed, and another db_indexed field created with the original's name.
FWIW I have worked around this in my migrations by doing two AlterFields,
dropping & re-creating the index by doing db_index=False then
db_index=True.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:8>
Comment (by skyjur):
This also happens when renaming a ManyToManyField and then adding a new
ManyToManyField with the same name. I've opened a ticket then @timgraham
pointed out that it's duplicate of this one
https://code.djangoproject.com/ticket/25752
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:9>
Comment (by evenicoulddoit):
I tried to migrate to a new model and bumped into this one. To fix, I made
sure that I renamed my existing model to "old" first (e.g.
Model->ModelOld), create the new model with the previous name (e.g.
Model), migrate the data and then remove the old model. My previous
strategy of creating the new table migrating the data and then renaming it
broke due to the index issues.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:10>
Comment (by lwm):
Replying to [comment:8 ris]:
> Replying to [comment:4 tomviner]:
> > I've found an associated case where a field with {{{db_index=True}}},
is renamed, and another db_indexed field created with the original's name.
>
> FWIW I have worked around this in my migrations by doing two
AlterFields, dropping & re-creating the index by doing db_index=False then
db_index=True.
Confirmed. This worked for me also. I can use `manage.py sqlmigrate <app>
<migration>` to see which indexes are giving me conflicts (should be the
same as the error you get when you try to run migrate) and then used the
`db_index=False/True` trick with `AlterField`.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:11>
* cc: aksheshdoshi@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:12>
* version: 1.7 => master
Comment:
#24528 was a duplicate.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:13>
Comment (by charettes):
#24715 was duplicate with a patch for the Oracle case where the sequence
and triggers created for an `AutoField` were not renamed.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:14>
Comment (by charettes):
This should also be the case for `AlterField` operations on both sides
(e.g. foreign key constraint names are generated from their referenced
table and column names).
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:15>
* cc: emorley@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:16>
Comment (by mkoistinen):
Until this is resolved, one can work-around this issue by doing something
like:
{{{
migrations.RunSQL(
'ALTER INDEX myapp_mymodel_myfield_othermodel_id_0f4cfc54
rename TO myapp_mymodel_myfield_othermodel_legacy_id_0f4cfc54',
'ALTER INDEX
myapp_mymodel_myfield_othermodel_legacy_id_0f4cfc54 rename TO
myapp_mymodel_myfield_othermodel_id_0f4cfc54',
)
}}}
This sort of approach should work well for both FKs and M2Ms (example
below), but it may make your migrations DB Backend specific =/
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:17>
Comment (by Tim Graham):
I closed #28803 as a duplicate.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:18>
* cc: Ryan Hiebert (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:19>
Comment (by jonathan-golorry):
I just ran into this issue with `RenameField` on `ManyToMany` relations.
The (automatically generated) through table changes name, but the id_seq
for it doesn't.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:20>
* cc: Vadym Moshynskyi (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:21>
* cc: Thomas Riccardi (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:22>
Comment (by Thomas Riccardi):
Some analysis (with postgresql backend):
- when creating a new field with index (such as a ForeignKey), django
creates the column and the index
- when deleting a field, django only deletes the column: it lets
postgresql delete all related index automatically
- when renaming a field, django only renames the column: it lets
postgresql update all index references (and probably more), *but does not*
rename the index
It seems django doesn't really track the index name, and it works great
except for renames
=> when a new index has to be created for the old field name, the index
name conflicts: this is this issue.
This reliance on field reference automatic update by the db seems to
create a similar issue with index_together/unique_together:
field rename + index_together/unique_together alternation to update the
field name results in no SQL except the RENAME COLUMN: the index/unique
names are *not* updated.
The workaround from comment 17 (ticket:23577#comment:17) with a manual
index rename may create issues when this ticket is fixed:
at that point django will probably assume the index name is the one it
generates if it had to create the index, and the comment uses a different
one (the hash_suffix_part is different)
=> for more rubustness/future compatibility I suggest to use the "proper"
index name: manually create migrations to delete then re-create the field,
and use `./manage.py sqlmigrate` to get the created index name (it depends
solely on table name and index fields name: see
django/db/backends/base/schema.py:BaseDatabaseSchemaEditor._create_index_name)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:23>
Comment (by Thomas Riccardi):
To get the proper new index name:
{{{#!python
from django.db import connection
schema_editor = connection.schema_editor()
def get_index_name(field):
return schema_editor._create_index_name(field.model._meta.db_table,
[field.attname])
get_index_name(ModelFoo.field_bar.field)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:24>
* cc: Sardorbek Imomaliev (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:25>
* cc: Friedrich Delgado (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:26>
Old description:
> This one's a bit of an edge case, but I ran into it trying to refactor
> some models on my moderately large work project.
>
> Let's say I have a Django 1.7 app called `sample`:
> 1. Create a model `Bar`.
> 2. Create a model `Foo` with `bar = models.ForeignKey(Bar, blank=True,
> null=True)`
> 3. `makemigrations` (migration `0001`)
> 4. Rename `Foo` to `OldFoo`.
> 5. `makemigrations`, say yes to the rename prompt (migration `0002`)
> 6. Create new model `Foo`, which also has `bar = models.ForeignKey(Bar,
> blank=True, null=True)`
> 7. `makemigrations` (migration `0003`)
> 8. `migrate`
>
> When `migrate` hits `0003`, it throws this error:
> {{{django.db.utils.OperationalError: index sample_foo_4350f7d0 already
> exists}}}
>
> You may notice that `sqlmigrate sample 0001` and `sqlmigrate sample 0003`
> have the same line in both of them:
> {{{CREATE INDEX sample_foo_4350f7d0 ON "sample_foo" ("bar_id");}}}
>
> In my production case, I actually had no problems on servers, because
> South had created the index with a different name. When I renamed the
> models and added a field, the new index did not collide with the old one.
> However, our test suite started failing, because it would run the
> migrations from the ground up, exposing the above bug.
>
> I haven't decided on a workaround yet, but I thought I'd bring this to
> your attention. I might have to inject a migration that renames the
> index created in `0001`, or something to that effect.
>
> The attached test case has a project `dj17test` in the state after having
> performed all of the above steps.
New description:
--
Comment (by clavay):
I tried this and it works :
In the migration file I replace this :
{{{
migrations.AlterField(
model_name='foo',
name='variable',
field=models.ForeignKey(null=True,
on_delete=django.db.models.deletion.SET_NULL, to='app.Variable'),
),
}}}
with this :
{{{
migrations.RenameModel('Foo', 'FooNew'),
migrations.AlterField(
model_name='foonew',
name='variable',
field=models.ForeignKey(null=True,
on_delete=django.db.models.deletion.SET_NULL, to='app.Variable'),
),
migrations.RenameModel('FooNew', 'Foo'),
}}}
Is it a good solution ?
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:27>
Old description:
New description:
--
Comment (by Thomas Riccardi):
Restore ticket description
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:28>
Comment (by Steve Recio):
Just ran into this issue when attempting to rename and rebuild a table.
Renaming the table doesn't change any of the indexes or constraints so I
can't rebuild another table with the same ManyToMany relationship. I tried
creating a through table and renaming the foreign keys but that doesn't
seem to do it either as the "UNIQUE TOGETHER" index name itself never
changes.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:29>
* cc: Daniel Hahler (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:30>
Comment (by Jef D):
Just opened this issue: https://code.djangoproject.com/ticket/33488#ticket
which was apparently a duplicate of this one. I did a rename and quickly
after I reused the old name of the field which made this very visible.
However, I can imagine that it is a difficult error to find when it
happens with longer periods in between. The longer a project lasts, the
more likely it is that one will run into this kind of issues.
Unfortunately, the issue doesn't seem to have a high priority given that
it has been 7 years since it was opened. For now I will sandwich the
rename migration with unsetting and setting the index in two additional
migrations.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:31>
Comment (by scholtalbers):
Encountered this (django 3.2) when trying to add a field with the same
name of a field I renamed in a few migrations before. The original index
is still existing. It should have been renamed as part of the AlterField
migration.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:32>
Comment (by Mikko Ahonen):
Run into this as well, using django 3.1 and Postgres. Similar case as for
scholtalbers.
I wanted to switch to using a library that wanted to add field to model
which was conflicting with the existing name, used for similar purpose
(parent).
I renamed the field (to old_parent), ran migration, created migration,
added the library (which added the parent field), created migration, run
migration.
Locally migrations worked because I was using SQLite, but when installing
on staging it failed because of conflicting index name.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:33>
Comment (by JohnLZeller):
Also ran into this same issue the way the OP wrote it
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:34>
* cc: Bhuvnesh (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:35>
Comment (by Steven Mapes):
So I ended up created a duplicate of this ticket for issue #34647 but
thanks to clavay's suggestion I managed to also get a work around by
running {{{makemigrations}}}then editing the {{{CreateModel}}} and
changing the name and db_table entries for all tables that were created (I
use custom table names as the app name prefix breaks the DB naming
convention policy that's in place for the project.
I then added {{{migrations.RenameModel}}} statements for each of the
models I needed to rename back, four of them, then finally had to run a
{{{migrations.RunSQL}}} to perform the actual rename of the underlying
databases tables.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:36>
Comment (by Roberto Maurizzi):
Just hit the same problem after renaming a Model and creating a new one
with the old name (to conform to how several other models in that
application ended up being named).
I was able to fix it by:
- changing the new Model creation in the migration to specify
`spatial_index=False` (it was a GIS field, I guess for regular ones you
need to use `db_index=False`)
- running makemigrations again: since I never changed the models, it
detected the missing indexes and created another migration that added them
to the db
This worked both for migrating the production database (was working
earlier too) and when running tests.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:37>
Comment (by Fabian Pottbäcker):
I just ran into a similar Issue when reverting migrations on Postgres,
specifically I have the following condensed and abstracted migration
history:
1. Setup:
1. Create model Author
2. Create model Book (`author = ForeignKey(Author)`)
2. Introduce PenNames
1. Create model PenName
2. Alter `Book.author` to allow null
3. Add `author_new = ForgeignKey(PenName, null=True)` to book
4. RunPython to create PenNames for all authors and set `author_new`
accordingly (including a corresponding reverse operation)
5. Drop `Book.author`
6. Alter `Book.author_new` to prohibit null
7. Rename `Book.author_new` to `Book.author`
3. Remove PenNames (for whatever reason)
1. Some Operation to repopulate `Book.author`
2. Drop column `Book.author`
After successfully applying these migrations, reverting to State 1 will
error when undoing step 2.5, since the index `book_author_id...` was
already created when undoing step 3.2, but it would be called
`book_author_new_id...` when applying these migrations forwards.
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:38>
* owner: (none) => vrocha
* status: new => assigned
Comment:
I am at DjangoCon US sprints trying to get to the bottom of this
--
Ticket URL: <https://code.djangoproject.com/ticket/23577#comment:39>