[Django] #32770: GinIndex with OpClass and Cast results in Postgres syntax error in migration
Django
migration
--------------------------------------+------------------------
Reporter: syastrov | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 3.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
--------------------------------------+------------------------
Given the following model + index:
{{{
from django.db import models
from django.contrib.postgres.indexes import GinIndex, OpClass
from django.db.models.functions import Cast
class MyModel(models.Model):
class Meta:
indexes = [
GinIndex(OpClass(Cast("id", output_field=models.TextField()),
name='gin_trgm_ops'), name='foobar')
]
}}}
After running makemigrations and running the migration, it produces the
SQL:
{{{
CREATE INDEX "foobar" ON "myapp_mymodel" USING gin ((CAST("id" AS text)
gin_trgm_ops));
}}}
Which is a syntax error on Postgres (version 12). The reason is the extra
parentheses around CAST.
--
Ticket URL: <https://code.djangoproject.com/ticket/32770>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
migration
Reporter: Seth Yastrov | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) | Resolution:
Severity: Normal | worksforme
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):
* cc: Hannes Ljungberg (added)
* resolution: => worksforme
* status: new => closed
* component: Migrations => Database layer (models, ORM)
* easy: 1 => 0
Comment:
Extra parentheses shouldn't be an issue on PostgreSQL. Also I cannot
reproduce this crash on PostgreSQL 12.6, see the following test.
{{{
def test_trigram_op_class_cast_gin_index(self):
index_name = 'trigram_op_class_castgin'
index = GinIndex(OpClass(Cast('id', CharField(max_length=255)),
name='gin_trgm_ops'), name=index_name)
with connection.schema_editor() as editor:
editor.add_index(Scene, index)
with editor.connection.cursor() as cursor:
cursor.execute(self.get_opclass_query, [index_name])
self.assertCountEqual(cursor.fetchall(), [('gin_trgm_ops',
index_name)])
constraints = self.get_constraints(Scene._meta.db_table)
self.assertIn(index_name, constraints)
self.assertIn(constraints[index_name]['type'], GinIndex.suffix)
with connection.schema_editor() as editor:
editor.remove_index(Scene, index)
self.assertNotIn(index_name,
self.get_constraints(Scene._meta.db_table))
}}}
{{{
CREATE INDEX "trigram_op_class_castgin" ON "postgres_tests_scene" USING
gin ((CAST("id" AS varchar(255))) gin_trgm_ops)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/32770#comment:1>
Django
migration
Reporter: Seth Yastrov | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) | Resolution:
Severity: Normal | worksforme
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Hannes Ljungberg):
I'm pretty sure that you're seeing this error because of not adding
`django.contrib.postgres` to `INSTALLED_APPS`, see:
https://docs.djangoproject.com/en/3.2/ref/contrib/postgres/indexes
/#opclass-expressions
--
Ticket URL: <https://code.djangoproject.com/ticket/32770#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Seth Yastrov | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: worksforme =>
Comment:
Thank you Hannes Ljungberg!
I did not notice that the documentation says to add
`django.contrib.postgres` to `INSTALLED_APPS`. That fixes the problem.
Would it be an idea to make a check that `django.contrib.postgres` is in
`INSTALLED_APPS` when making use of `OpClass` so that instead of
encountering a Postgres syntax error and not knowing what to do (and
assuming there is a bug in Django), you would get a helpful error message
telling how to correct the problem?
Therefore I'm reopening the issue, and hoping that this check can be made,
as despite this requirement being mentioned in the docs, the current
behavior is quite unintuitive.
--
Ticket URL: <https://code.djangoproject.com/ticket/32770#comment:3>
Django
OpClass().
-------------------------------------+-------------------------------------
Reporter: Seth Yastrov | Owner: nobody
Component: Database layer | Version: 3.2
(models, ORM) |
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: new => closed
* type: Bug => New feature
* resolution: => needsinfo
Comment:
> Therefore I'm reopening the issue, and hoping that this check can be
made, as despite this requirement being mentioned in the docs, the current
behavior is quite unintuitive.
It would be really complicated. First of all, we would need to mix-up
logic from a contrib app and the ORM. Secondly `OpClass()` don't need to
be the topmost expression, so the flatten list of expression would be
necessary. Thirdly we don't have similar checks for fields from
`django.contrib.postgres` app which also require including
`'django.contrib.postgres'` in `INSTALLED_APPS`, e.g. `HStoreField`. I
don't think it's worth complexity, however we can reconsider this decision
if someone provides PoC.
--
Ticket URL: <https://code.djangoproject.com/ticket/32770#comment:4>
Django
OpClass().
-------------------------------------+-------------------------------------
Reporter: Seth Yastrov | Owner: nobody
Type: New feature | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: needsinfo
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* cc: Simon Charette (added)
Comment:
In the light of continuous reports of users running into this qwirk
(#35431 being the latest one) I wonder if we could
1. Have all fields defined in `contrib.postgres` have their `check` method
make sure that `'django.contrib.postgres' in INSTALLED_APPS`
2. Adjust `Model._check_indexes` to do something similar to what we did
with `Constraint` check delegation in
0fb104dda287431f5ab74532e45e8471e22b58c8
3. Have both `contrib.postgres` indexes and constraints perform the same
`'django.contrib.postgres' in INSTALLED_APPS` check
Thoughts?
--
Ticket URL: <https://code.djangoproject.com/ticket/32770#comment:5>