[Django] #30385: SearchVector's usage of CONCAT makes it impossible to index against.

7 views
Skip to first unread message

Django

unread,
Apr 19, 2019, 10:30:03 AM4/19/19
to django-...@googlegroups.com
#30385: SearchVector's usage of CONCAT makes it impossible to index against.
-------------------------------------+-------------------------------------
Reporter: Simon | Owner: Simon Charette
Charette |
Type: Bug | Status: assigned
Component: | Version: 2.2
contrib.postgres |
Severity: Release | Keywords:
blocker |
Triage Stage: Accepted | Has patch: 1
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 1 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
`SearchVector` was changed in Django 2.2 to allow querying non-text fields
(#29582) by using `CONCAT` which automatically cast such fields to text
and deals with `NULL` value appropriately.

This had the unintended side effect of making the generated SQL loose it's
`IMMUTABLE` status as `CONCAT` can generate locale/TZ dependant output
which makes the generated SQL expression impossible to index against.

--
Ticket URL: <https://code.djangoproject.com/ticket/30385>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Apr 19, 2019, 10:38:00 AM4/19/19
to django-...@googlegroups.com
#30385: SearchVector's usage of CONCAT makes it impossible to index against.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon

| Charette
Type: Bug | Status: assigned
Component: contrib.postgres | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):

* needs_better_patch: 1 => 0


--
Ticket URL: <https://code.djangoproject.com/ticket/30385#comment:1>

Django

unread,
Apr 23, 2019, 1:36:33 AM4/23/19
to django-...@googlegroups.com
#30385: SearchVector's usage of CONCAT makes it impossible to index against.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon

| Charette
Type: Bug | Status: assigned
Component: contrib.postgres | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by felixxm):

[https://github.com/django/django/pull/11254 PR]

--
Ticket URL: <https://code.djangoproject.com/ticket/30385#comment:2>

Django

unread,
Apr 23, 2019, 2:32:44 AM4/23/19
to django-...@googlegroups.com
#30385: SearchVector's usage of CONCAT makes it impossible to index against.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
| Charette
Type: Bug | Status: closed
Component: contrib.postgres | Version: 2.2
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):

* status: assigned => closed
* resolution: => fixed


Comment:

In [changeset:"405c8363362063542e9e79beac53c8437d389520" 405c836]:
{{{
#!CommitTicketReference repository=""
revision="405c8363362063542e9e79beac53c8437d389520"
Fixed #30385 -- Restored SearchVector(config) immutability.

Regression in 1a28dc3887e8d66d5e3ff08cf7fb0a6212b873e5.

The usage of CONCAT to allow SearchVector to deal with non-text fields
made the generated expression non-IMMUTABLE which prevents a functional
index to be created for it.

Using a combination of COALESCE and ::text makes sure the expression
preserves its immutability.

Refs #29582. Thanks Andrew Brown for the report, Nick Pope for the
review.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/30385#comment:3>

Django

unread,
Apr 23, 2019, 2:35:42 AM4/23/19
to django-...@googlegroups.com
#30385: SearchVector's usage of CONCAT makes it impossible to index against.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
| Charette
Type: Bug | Status: closed
Component: contrib.postgres | Version: 2.2
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"88bf635c356b4d3a47e88dc4142b90060ce3c2ef" 88bf635c]:
{{{
#!CommitTicketReference repository=""
revision="88bf635c356b4d3a47e88dc4142b90060ce3c2ef"
[2.2.x] Fixed #30385 -- Restored SearchVector(config) immutability.

Regression in 1a28dc3887e8d66d5e3ff08cf7fb0a6212b873e5.

The usage of CONCAT to allow SearchVector to deal with non-text fields
made the generated expression non-IMMUTABLE which prevents a functional
index to be created for it.

Using a combination of COALESCE and ::text makes sure the expression
preserves its immutability.

Refs #29582. Thanks Andrew Brown for the report, Nick Pope for the
review.

Backport of 405c8363362063542e9e79beac53c8437d389520 from master
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/30385#comment:4>

Reply all
Reply to author
Forward
0 new messages