[Django] #30672: JSONField/HStoreField key and index transforms crash.

8 views
Skip to first unread message

Django

unread,
Aug 2, 2019, 7:41:54 AM8/2/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
--------------------------------------------+--------------------------
Reporter: felixxm | Owner: felixxm
Type: Bug | Status: assigned
Component: contrib.postgres | Version: 2.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
--------------------------------------------+--------------------------
JSONField/HStoreField key and index transforms crash when we pass
expressions with parameters, e.g.
{{{
KeyTransform('x', RawSQL("%s::jsonb", ['{"x": "bar"}'])
}}}
this is caused by regression introduced in the last security release
7deeabc7c7526786df6894429ce89a9c4b614086, however `KeyTransform` is
undocumented and such usage is untested.

--
Ticket URL: <https://code.djangoproject.com/ticket/30672>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Aug 2, 2019, 7:52:25 AM8/2/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------

Reporter: felixxm | Owner: felixxm
Type: Bug | Status: assigned
Component: contrib.postgres | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------
Changes (by Carlton Gibson):

* stage: Unreviewed => Accepted


Comment:

Well, clearly a bug. I'm inclined to view this as ''new usage'', rather
than a regression, but open to opinions.

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:1>

Django

unread,
Aug 2, 2019, 7:53:32 AM8/2/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------

Reporter: felixxm | Owner: felixxm
Type: Bug | Status: assigned
Component: contrib.postgres | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------
Description changed by felixxm:

Old description:

> JSONField/HStoreField key and index transforms crash when we pass
> expressions with parameters, e.g.
> {{{
> KeyTransform('x', RawSQL("%s::jsonb", ['{"x": "bar"}'])
> }}}
> this is caused by regression introduced in the last security release
> 7deeabc7c7526786df6894429ce89a9c4b614086, however `KeyTransform` is
> undocumented and such usage is untested.

New description:

JSONField/HStoreField key and index transforms crash when we pass
expressions with parameters, e.g.
{{{
KeyTransform('x', RawSQL("%s::jsonb", ['{"x": "bar"}'])
}}}
this is caused by regression introduced in the last security release
7deeabc7c7526786df6894429ce89a9c4b614086, however `KeyTransform` is
undocumented and such usage is untested.

Crash for nested keys in
[https://github.com/django/django/blob/194d1dfc186cc8d2b35dabf64f3ed38b757cbd98/django/contrib/postgres/fields/jsonb.py#L109-L110
KeyTransform] for `JSONField` is not a regression because it has not been
changed since its introduction.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:2>

Django

unread,
Aug 2, 2019, 7:58:21 AM8/2/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------

Reporter: felixxm | Owner: felixxm
Type: Bug | Status: assigned
Component: contrib.postgres | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------
Changes (by felixxm):

* version: 2.2 => master


Old description:

> JSONField/HStoreField key and index transforms crash when we pass
> expressions with parameters, e.g.
> {{{
> KeyTransform('x', RawSQL("%s::jsonb", ['{"x": "bar"}'])
> }}}
> this is caused by regression introduced in the last security release
> 7deeabc7c7526786df6894429ce89a9c4b614086, however `KeyTransform` is
> undocumented and such usage is untested.
>

> Crash for nested keys in
> [https://github.com/django/django/blob/194d1dfc186cc8d2b35dabf64f3ed38b757cbd98/django/contrib/postgres/fields/jsonb.py#L109-L110
> KeyTransform] for `JSONField` is not a regression because it has not been
> changed since its introduction.

New description:

JSONField/HStoreField key and index transforms crash when we pass
expressions with parameters, e.g.
{{{
KeyTransform('x', RawSQL("%s::jsonb", ['{"x": "bar"}'])
}}}

~~this is caused by regression introduced in the last security release
7deeabc7c7526786df6894429ce89a9c4b614086~~, however `KeyTransform` is


undocumented and such usage is untested.

Crash for nested keys in


[https://github.com/django/django/blob/194d1dfc186cc8d2b35dabf64f3ed38b757cbd98/django/contrib/postgres/fields/jsonb.py#L109-L110
KeyTransform] for `JSONField` is not a regression because it has not been
changed since its introduction.

--

Comment:

OK, it seems that it's not a regression because these transforms crashed
(when we pass expressions with parameters) even before
7deeabc7c7526786df6894429ce89a9c4b614086. Of course from a different
reason.

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:3>

Django

unread,
Aug 2, 2019, 8:16:55 AM8/2/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------

Reporter: felixxm | Owner: felixxm
Type: Bug | Status: assigned
Component: contrib.postgres | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------
Changes (by felixxm):

* has_patch: 0 => 1


Comment:

[https://github.com/django/django/pull/11617 PR]

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:4>

Django

unread,
Aug 5, 2019, 7:40:25 AM8/5/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------

Reporter: felixxm | Owner: felixxm
Type: Bug | Status: assigned
Component: contrib.postgres | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------
Description changed by felixxm:

Old description:

> JSONField/HStoreField key and index transforms crash when we pass


> expressions with parameters, e.g.
> {{{
> KeyTransform('x', RawSQL("%s::jsonb", ['{"x": "bar"}'])
> }}}

> ~~this is caused by regression introduced in the last security release
> 7deeabc7c7526786df6894429ce89a9c4b614086~~, however `KeyTransform` is


> undocumented and such usage is untested.
>

> Crash for nested keys in
> [https://github.com/django/django/blob/194d1dfc186cc8d2b35dabf64f3ed38b757cbd98/django/contrib/postgres/fields/jsonb.py#L109-L110
> KeyTransform] for `JSONField` is not a regression because it has not been
> changed since its introduction.

New description:

JSONField/HStoreField key and index transforms crash when we pass


expressions with parameters, e.g.
{{{
KeyTransform('x', RawSQL("%s::jsonb", ['{"x": "bar"}'])
}}}
this is caused by regression introduced in the last security release
7deeabc7c7526786df6894429ce89a9c4b614086, however `KeyTransform` is
undocumented and such usage is untested.

Crash for nested keys in


[https://github.com/django/django/blob/194d1dfc186cc8d2b35dabf64f3ed38b757cbd98/django/contrib/postgres/fields/jsonb.py#L109-L110
KeyTransform] for `JSONField` is not a regression because it has not been
changed since its introduction.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:5>

Django

unread,
Aug 5, 2019, 7:58:27 AM8/5/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------

Reporter: felixxm | Owner: felixxm
Type: Bug | Status: assigned
Component: contrib.postgres | Version: 1.11

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------
Changes (by felixxm):

* version: master => 1.11


--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:6>

Django

unread,
Aug 12, 2019, 12:57:05 PM8/12/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------

Reporter: felixxm | Owner: felixxm
Type: Bug | Status: assigned
Component: contrib.postgres | Version: 1.11
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------
Changes (by felixxm):

* severity: Normal => Release blocker


--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:7>

Django

unread,
Aug 12, 2019, 1:03:25 PM8/12/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------

Reporter: felixxm | Owner: felixxm
Type: Bug | Status: assigned
Component: contrib.postgres | Version: 1.11
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------

Comment (by felixxm):

I moved an issue with nested transforms to a separate ticket #30704
because it is not a regression.

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:8>

Django

unread,
Aug 14, 2019, 9:26:31 AM8/14/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------
Reporter: felixxm | Owner: felixxm
Type: Bug | Status: closed
Component: contrib.postgres | Version: 1.11
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------
Changes (by GitHub <noreply@…>):

* status: assigned => closed
* resolution: => fixed


Comment:

In [changeset:"1f8382d34d54061eddc41df6994e20ee38c60907" 1f8382d3]:
{{{
#!CommitTicketReference repository=""
revision="1f8382d34d54061eddc41df6994e20ee38c60907"
Fixed #30672 -- Fixed crash of JSONField/HStoreField key transforms on
expressions with params.

Regression in 4f5b58f5cd3c57fee9972ab074f8dc6895d8f387.

Thanks Florian Apolloner for the report and helping with tests.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:9>

Django

unread,
Aug 14, 2019, 9:33:19 AM8/14/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------
Reporter: felixxm | Owner: felixxm

Type: Bug | Status: closed
Component: contrib.postgres | Version: 1.11
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"52a7759a4916484953c8e5a7d2f260dd1c1e3c12" 52a7759a]:
{{{
#!CommitTicketReference repository=""
revision="52a7759a4916484953c8e5a7d2f260dd1c1e3c12"
[2.2.x] Fixed #30672 -- Fixed crash of JSONField/HStoreField key


transforms on expressions with params.

Regression in 4f5b58f5cd3c57fee9972ab074f8dc6895d8f387.

Thanks Florian Apolloner for the report and helping with tests.

Backport of 1f8382d34d54061eddc41df6994e20ee38c60907 from master.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:10>

Django

unread,
Aug 14, 2019, 9:39:14 AM8/14/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------
Reporter: felixxm | Owner: felixxm

Type: Bug | Status: closed
Component: contrib.postgres | Version: 1.11
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"968b9af9b7f7e49af17fc82de5a2de9a91b86e44" 968b9af9]:
{{{
#!CommitTicketReference repository=""
revision="968b9af9b7f7e49af17fc82de5a2de9a91b86e44"
[2.1.x] Fixed #30672 -- Fixed crash of JSONField/HStoreField key


transforms on expressions with params.

Regression in 4f5b58f5cd3c57fee9972ab074f8dc6895d8f387.

Thanks Florian Apolloner for the report and helping with tests.

Backport of 1f8382d34d54061eddc41df6994e20ee38c60907 from master.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:11>

Django

unread,
Aug 14, 2019, 10:00:09 AM8/14/19
to django-...@googlegroups.com
#30672: JSONField/HStoreField key and index transforms crash.
----------------------------------+------------------------------------
Reporter: felixxm | Owner: felixxm

Type: Bug | Status: closed
Component: contrib.postgres | Version: 1.11
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"473c526b1b014e73e139665db2ddbbcee23bb826" 473c526b]:
{{{
#!CommitTicketReference repository=""
revision="473c526b1b014e73e139665db2ddbbcee23bb826"
[1.11.x] Fixed #30672 -- Fixed crash of JSONField/HStoreField key


transforms on expressions with params.

Regression in 4f5b58f5cd3c57fee9972ab074f8dc6895d8f387.

Thanks Florian Apolloner for the report and helping with tests.

Backport of 1f8382d34d54061eddc41df6994e20ee38c60907 from master.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/30672#comment:12>

Reply all
Reply to author
Forward
0 new messages