[Django] #28888: Index added to _meta.indexes with Meta.indexes=[] yields two equal addIndex() operations.

12 views
Skip to first unread message

Django

unread,
Dec 5, 2017, 6:21:47 AM12/5/17
to django-...@googlegroups.com
#28888: Index added to _meta.indexes with Meta.indexes=[] yields two equal
addIndex() operations.
--------------------------------------------------+------------------------
Reporter: Jan Pieter Waagmeester | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.11
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
--------------------------------------------------+------------------------
I use a custom field derived from
`django.contrib.postgres.fields.JSONField`, which adds an `GinIndex` to
`model._meta.indexes`:
{{{
from django.contrib.postgres.fields import JSONField
from django.contrib.postgres.indexes import GinIndex

class CustomJSONField(JSONField):
def contribute_to_class(self, cls, name):
super(CustomJSONField, self).contribute_to_class(cls, name)

index = GinIndex(fields=[name])
index.set_name_with_model(cls)
cls._meta.indexes.append(index)
}}}

When used in a model like this,
{{{
class Blog(models.Model):
title = models.CharField(max_length=100)
json = CustomJSONField()
}}}

Migrations for model and index are created as expected:
{{{
./manage.py --version
1.11.8
./manage.py makemigrations app
Migrations for 'app':
app/migrations/0001_initial.py
- Create model Blog
- Create index app_blog_json_2cf556_gin on field(s) json of model blog
}}}

But when I add an empty list of indexes like this:

{{{
class Blog(models.Model):
title = models.CharField(max_length=100)
json = CustomJSONField()

class Meta:
indexes = []
}}}
two indexes are created:

{{{
rm -rf app/migrations
./manage.py --version
1.11.8
./manage.py makemigrations app
Migrations for 'app':
app/migrations/0001_initial.py
- Create model Blog
- Create index app_blog_json_2cf556_gin on field(s) json of model blog
- Create index app_blog_json_2cf556_gin on field(s) json of model blog
}}}

Which of course results in `django.db.utils.ProgrammingError: relation
"app_blog_json_2cf556_gin" already exists`.

--
Ticket URL: <https://code.djangoproject.com/ticket/28888>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Dec 5, 2017, 6:22:41 AM12/5/17
to django-...@googlegroups.com
#28888: Index added to _meta.indexes with Meta.indexes=[] yields two equal
addIndex() operations.
-------------------------------------+-------------------------------------
Reporter: Jan Pieter | Owner: nobody
Waagmeester |

Type: Uncategorized | Status: new
Component: Migrations | Version: 1.11
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Jan Pieter Waagmeester:

Old description:

New description:

I use a custom field derived from
`django.contrib.postgres.fields.JSONField`, which adds an `GinIndex` to
`model._meta.indexes`:
{{{
from django.contrib.postgres.fields import JSONField
from django.contrib.postgres.indexes import GinIndex

class CustomJSONField(JSONField):
def contribute_to_class(self, cls, name):
super(CustomJSONField, self).contribute_to_class(cls, name)

index = GinIndex(fields=[name])
index.set_name_with_model(cls)
cls._meta.indexes.append(index)
}}}

When used in a model like this,
{{{
class Blog(models.Model):
title = models.CharField(max_length=100)
json = CustomJSONField()
}}}

Migrations for model and index are created as expected:
{{{
./manage.py --version
1.11.8
./manage.py makemigrations app
Migrations for 'app':
app/migrations/0001_initial.py
- Create model Blog
- Create index app_blog_json_2cf556_gin on field(s) json of model blog
}}}

But when I add an empty list of indexes to `class Meta` like this:

{{{
class Blog(models.Model):
title = models.CharField(max_length=100)
json = CustomJSONField()

class Meta:
indexes = []
}}}
two indexes are created:

{{{
rm -rf app/migrations
./manage.py --version
1.11.8
./manage.py makemigrations app
Migrations for 'app':
app/migrations/0001_initial.py
- Create model Blog
- Create index app_blog_json_2cf556_gin on field(s) json of model blog
- Create index app_blog_json_2cf556_gin on field(s) json of model blog
}}}

Which of course results in `django.db.utils.ProgrammingError: relation
"app_blog_json_2cf556_gin" already exists`.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/28888#comment:1>

Django

unread,
Dec 5, 2017, 6:36:57 AM12/5/17
to django-...@googlegroups.com
#28888: Index added to _meta.indexes with Meta.indexes=[] yields two equal
addIndex() operations.
-------------------------------------+-------------------------------------
Reporter: Jan Pieter | Owner: nobody
Waagmeester |
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.11
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jan Pieter Waagmeester):

Possibly unrelated, but:

When an initial migration is added without the index added to
`cls._meta.indexes`, resulting in the output
{{{


./manage.py makemigrations app
Migrations for 'app':
app/migrations/0001_initial.py
- Create model Blog
}}}

No index is added, as expected.

However, adding the index to `cls._meta.indexes` has no effect: `No
changes detected in app 'app'`.

--
Ticket URL: <https://code.djangoproject.com/ticket/28888#comment:2>

Django

unread,
Dec 5, 2017, 8:38:03 AM12/5/17
to django-...@googlegroups.com
#28888: Index added to _meta.indexes with Meta.indexes=[] yields two equal
addIndex() operations.
-------------------------------------+-------------------------------------
Reporter: Jan Pieter | Owner: nobody
Waagmeester |
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.11
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

As there's no documented support for adding indexes in
`Field.contribute_to_class()`, can you explain why Django is at fault
and/or propose a patch?

--
Ticket URL: <https://code.djangoproject.com/ticket/28888#comment:3>

Django

unread,
Dec 5, 2017, 10:24:35 AM12/5/17
to django-...@googlegroups.com
#28888: Index added to _meta.indexes with Meta.indexes=[] yields two equal
addIndex() operations.
-------------------------------------+-------------------------------------
Reporter: Jan Pieter | Owner: nobody
Waagmeester |
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.11
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jan Pieter Waagmeester):

I understand that "there's no documented support for adding indexes in
`Field.contribute_to_class()`" and I'm open to suggestions to other
solutions to add indexes with a field.

But this seems the cleanest way to do it, and support seems to be almost
there/halfway there. I'm willing to invest time to create a patch to fix
this, but I'd like to have some degree of confidence that such a solution
would be desirable functionality.

--
Ticket URL: <https://code.djangoproject.com/ticket/28888#comment:4>

Django

unread,
Dec 5, 2017, 10:33:55 AM12/5/17
to django-...@googlegroups.com
#28888: Index added to _meta.indexes with Meta.indexes=[] yields two equal
addIndex() operations.
-------------------------------------+-------------------------------------
Reporter: Jan Pieter | Owner: nobody
Waagmeester |
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: 1.11
(models, ORM) |

Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

* component: Migrations => Database layer (models, ORM)
* type: Uncategorized => Cleanup/optimization
* stage: Unreviewed => Someday/Maybe


Comment:

Usually it's more effective to use the DevelopersMailingList to get
feedback and consensus on design decisions as it reaches a wider audience
than the ticket tracker.

--
Ticket URL: <https://code.djangoproject.com/ticket/28888#comment:5>

Reply all
Reply to author
Forward
0 new messages