[Django] #29769: JSONField lookup and F Objects
Django
-------------------------------------+-------------------------------------
Reporter: g4borg | Owner: (none)
Type: New | Status: new
feature |
Component: | Version:
contrib.postgres |
Severity: Normal | Keywords: F JSONField lookup
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Currently, it is rather undocumented, that F Functions do not perform JSON
Lookups.
I suggest to either introduce a similar Object that is capable, or similar
solution to make this possible; at the very least however, I suggest to
add this information to the documentation for JSONField in django docs.
I stumbled on this when I tried to rewrite a raw statement in the form of
`SELECT fields FROM schema WHERE lower(unaccent(jsonfield->lookup->>data))
LIKE lower(searchterm) ...`
and also stumbled on stackoverflow dealing with this issue:
https://stackoverflow.com/questions/40623820/django-jsonfield-postgres-
and-f-object-comparison
Note, that if lookups in F might not easily be adaptable, the
stackoverflow question has a nice side effect of showing off a possible
complementary solution: overwrite/extend `__getitem__` for `F` objects to
deal with json / subfield lookups.
--
I thought brainstorming about such would be benefitial,
If I have overseen an already existing workaround I am eager to get
corrected.
--
Ticket URL: <https://code.djangoproject.com/ticket/29769>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: g4borg | Owner: (none)
Type: New feature | Status: new
Component: contrib.postgres | Version:
Severity: Normal | Resolution:
Keywords: F JSONField lookup | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by g4borg:
Old description:
> Currently, it is rather undocumented, that F Functions do not perform
> JSON Lookups.
>
> I suggest to either introduce a similar Object that is capable, or
> similar solution to make this possible; at the very least however, I
> suggest to add this information to the documentation for JSONField in
> django docs.
>
> I stumbled on this when I tried to rewrite a raw statement in the form of
> `SELECT fields FROM schema WHERE
> lower(unaccent(jsonfield->lookup->>data)) LIKE lower(searchterm) ...`
>
> and also stumbled on stackoverflow dealing with this issue:
> https://stackoverflow.com/questions/40623820/django-jsonfield-postgres-
> and-f-object-comparison
>
> Note, that if lookups in F might not easily be adaptable, the
> stackoverflow question has a nice side effect of showing off a possible
> complementary solution: overwrite/extend `__getitem__` for `F` objects to
> deal with json / subfield lookups.
>
> --
> I thought brainstorming about such would be benefitial,
> If I have overseen an already existing workaround I am eager to get
> corrected.
New description:
Currently, it is rather undocumented, that `F` Objects do not perform JSON
Lookups.
I suggest to either introduce a similar Object that is capable, or similar
solution to make this possible; at the very least however, I suggest to
add this information to the documentation for JSONField in django docs.
I stumbled on this when I tried to rewrite a raw statement in the form of
`SELECT fields FROM schema WHERE lower(unaccent(jsonfield->lookup->>data))
LIKE lower(searchterm) ...`
and also stumbled on stackoverflow dealing with this issue:
https://stackoverflow.com/questions/40623820/django-jsonfield-postgres-
and-f-object-comparison
Note, that if lookups in F might not easily be adaptable, the
stackoverflow question has a nice side effect of showing off a possible
complementary solution: overwrite/extend `__getitem__` for `F` objects to
deal with json / subfield lookups.
--
I thought brainstorming about such would be benefitial,
If I have overseen an already existing workaround I am eager to get
corrected.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:1>
Django
Reporter: g4borg | Owner: (none)
Type: New feature | Status: new
Component: contrib.postgres | Version:
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by g4borg):
Replying to [comment:1 g4borg]:
Note that I do have found that the workaround with
`RawSQL('jsonfield->lookup->>data')` works as replacement for the `F()`
call, in my specific use case, but it results in a suboptimal solution
imho.
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:2>
Django
------------------------------------+------------------------------------
Reporter: g4borg | Owner: (none)
Type: New feature | Status: new
Component: contrib.postgres | Version:
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* stage: Unreviewed => Accepted
Comment:
I'm not sure what the best resolution might look like. It might involve
creating a custom express rather than using `F` objects. #24709 is a
similar issue for `ArrayField`.
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:3>
Django
------------------------------------+------------------------------------
Type: New feature | Status: assigned
Component: contrib.postgres | Version:
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mani):
* owner: (none) => Mani
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:4>
Django
------------------------------------+------------------------------------
Reporter: Gabor Körber | Owner: Mani
Type: New feature | Status: assigned
Component: contrib.postgres | Version:
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mani):
I agree with Tim Graham, Creating custom expression would be a better
approach than using `F` objects.
I have written a customer expression for querying JSONField and** it
works!
{{{
class KeyTextTransformFactory:
def __init__(self, key_name):
self.key_name = key_name
def __call__(self, *args, **kwargs):
return KeyTextTransform(self.key_name, *args, **kwargs)
class JSONF(F):
def resolve_expression(self, query=None, allow_joins=True, reuse=None,
summarize=False, for_save=False):
rhs = super().resolve_expression(query, allow_joins, reuse,
summarize, for_save)
field_list = self.name.split(LOOKUP_SEP)
for name in field_list[1:]:
rhs = KeyTextTransformFactory(name)(rhs)
return rhs
}}}
It is necessary to include `Cast` in rhs,
{{{
Sample.objects.filter(jsonfield__lookup__value=Cast(JSONF('value'),
IntegerField()))
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:5>
Django
------------------------------------+------------------------------------
Reporter: Gabor Körber | Owner: Mani
Type: New feature | Status: assigned
Component: contrib.postgres | Version:
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Srinivas Reddy Thatiparthy):
* cc: Srinivas Reddy Thatiparthy (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:6>
Django
------------------------------------+------------------------------------
Reporter: Gabor Körber | Owner: Mani
Type: New feature | Status: assigned
Component: contrib.postgres | Version:
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by GwynBleidD):
In Django 2.2 using `F` object with dicts or arrays in JSON field doesn't
throw an error, it just cuts off anything after field name, so the result
is the whole JSON. This means that `F` object is aware of field type, so I
don't see any reason why we need separate expressions for that.
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:7>
Django
------------------------------------+------------------------------------
Reporter: Gabor Körber | Owner: Mani
Type: New feature | Status: assigned
Component: contrib.postgres | Version:
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Uxio0):
@GwynBleidD I see it as an issue that `F()` object is different from the
regular field issue:
`str(MyEvent.objects.values('arguments__to').query)`
`'SELECT ("myapp_myevent"."arguments" -> \'to\') FROM "myapp_myevent"'`
If I use `F()` object:
`str(MyEvent.objects.values(x=F('arguments__to')).query)`
`'SELECT "myapp_myevent"."arguments" AS "x" FROM "myapp_myevent"'`
This way I'm not able to do casting on that field, for example:
`str(MyEvent.objects.values(x=Cast(F('arguments__to'),
DecimalField())).query)`
`'SELECT ("myapp_myevent"."arguments")::numeric(None, None) AS "x" FROM
"myapp_myevent"'`
So for example I cannot find a way to translate this simple query:
`SELECT SUM((arguments -> 'to')::decimal) as my_sum FROM myapp_myevent;`
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:8>
Django
------------------------------------+------------------------------------
Reporter: Gabor Körber | Owner: Mani
Type: New feature | Status: assigned
Component: contrib.postgres | Version:
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak):
Duplicate of #31639.
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:9>
Django
Reporter: Gabor Körber | Owner: Mani
Component: contrib.postgres | Version:
Severity: Normal | Resolution: duplicate
Keywords: F JSONField lookup | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* status: assigned => closed
* resolution: => duplicate
--
Ticket URL: <https://code.djangoproject.com/ticket/29769#comment:10>