[Django] #36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-values

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------
Reporter: john-parton | Type: New
| feature
Status: new | Component: Database
| layer (models, ORM)
Version: dev | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
In working towards supporting `ABSENT ON NULL` for JSONArray, I did some
digging on what it would take to implement it for `JSONObject` as well,
and essentially, because the __init__ signature consumes all the `kwargs`,
there's not really a good way to configure its behavior without
conflicting with a key definition.

If you could specify the items of a JSONObject positionally, then you get
the aforementioned benefit of being able to have kwargs as well without
conflicting, but you also get the added benefit of being able to construct
objects with non-static keys.

As an example esoteric usage, if you're trying to get the database to omit
valid objects for the Elasticsearch DSL, you would almost certainly need
to be able to build objects with non-static keys.

For example
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-
term-query.html

The key "user.id" refers to a field in elastic.

See some discussion here:
https://github.com/django/django/pull/19097#issuecomment-2611270246

I'll throw together a patch.
--
Ticket URL: <https://code.djangoproject.com/ticket/36139>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------

Reporter: john-parton | Owner: (none)

Type: New feature | Status: new

Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by john-parton):

Pull request/patch here: https://github.com/django/django/pull/19101

Not too bad for the actual code change, just a few lines. Several tests
added and expanded the `Usage` section of the documentation sufficiently
to cover the various use cases.
--
Ticket URL: <https://code.djangoproject.com/ticket/36139#comment:1>

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------
Reporter: john-parton | Owner: (none)
Type: New feature | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by john-parton):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36139#comment:2>

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------

Reporter: john-parton | Owner: john-
| parton
Type: New feature | Status: assigned

Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Antoliny):

* owner: (none) => john-parton
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/36139#comment:3>

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------
Reporter: john-parton | Owner: john-
| parton

Type: New feature | Status: closed

Component: Database layer | Version: dev
(models, ORM) |

Severity: Normal | Resolution: wontfix

Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* resolution: => wontfix
* status: assigned => closed

Comment:

Hi John, thank you for raising this!

As this is a new feature request, can you propose and discuss the idea on
the [https://forum.djangoproject.com/c/internals/5 Django Forum] to gain
consensus?
I can see that Simon said he felt this was a less useful feature than for
`JSONArray` (refs #36134), so we should confirm that there is a need for
this feature from the community before working on it.

I'll close the ticket for now, but if the community agrees with the
proposal, please return to this ticket and reference the forum discussion
so we can re-open it. For more information, please refer to
[https://docs.djangoproject.com/en/stable/internals/contributing/bugs-and-
features/#requesting-features the documented guidelines for requesting
features].
--
Ticket URL: <https://code.djangoproject.com/ticket/36139#comment:4>

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------
Reporter: john-parton | Owner: john-
| parton
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by john-parton):

Note that Simon said he felt that `ON NULL ABSENT` for JSONObject was less
useful. This doesn't directly enable that functionality, as I mostly
agree.

This is specifically for building JSON objects where keys are dynamic
expressions, which Simon didn't comment on.
--
Ticket URL: <https://code.djangoproject.com/ticket/36139#comment:5>

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------
Reporter: john-parton | Owner: john-
| parton
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by john-parton):

Discussion here: https://forum.djangoproject.com/t/ability-to-specify-
keys-for-jsonobject-using-arbitrary-expressions/38559
--
Ticket URL: <https://code.djangoproject.com/ticket/36139#comment:6>

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------
Reporter: john-parton | Owner: john-
| parton
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by john-parton:

Old description:

> In working towards supporting `ABSENT ON NULL` for JSONArray, I did some
> digging on what it would take to implement it for `JSONObject` as well,
> and essentially, because the __init__ signature consumes all the
> `kwargs`, there's not really a good way to configure its behavior without
> conflicting with a key definition.
>
> If you could specify the items of a JSONObject positionally, then you get
> the aforementioned benefit of being able to have kwargs as well without
> conflicting, but you also get the added benefit of being able to
> construct objects with non-static keys.
>
> As an example esoteric usage, if you're trying to get the database to
> omit valid objects for the Elasticsearch DSL, you would almost certainly
> need to be able to build objects with non-static keys.
>
> For example
> https://www.elastic.co/guide/en/elasticsearch/reference/current/query-
> dsl-term-query.html
>
> The key "user.id" refers to a field in elastic.
>
> See some discussion here:
> https://github.com/django/django/pull/19097#issuecomment-2611270246
>
> I'll throw together a patch.

New description:

Currently there is no way to build a JSON object in the database using the
JSONObject function where the keys are anything but static keys.

This is not a limitation of any of the backends that support JSON objects.

If you could specify the items of a JSONObject positionally, you get the

benefit of being able to construct objects with non-static keys.

As an example esoteric usage, if you're trying to get the database to emit

valid objects for the Elasticsearch DSL, you would almost certainly need
to be able to build objects with non-static keys.

For example
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-
term-query.html

The key "user.id" refers to a field in elastic.

See some discussion here:
https://github.com/django/django/pull/19097#issuecomment-2611270246

I'll throw together a patch.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/36139#comment:7>

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------
Reporter: john-parton | Owner: john-
| parton
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by john-parton):

I've modified the description of this ticket to remove any references to
`ABSENT ON NULL` and to remove any reasoning about that. Just
straightforward describe just this one feature.
--
Ticket URL: <https://code.djangoproject.com/ticket/36139#comment:8>

[Django]

#36139: Allow expressing JSONObject keys dynamically, as well as explicit pair-
values
-------------------------------------+-------------------------------------
Reporter: john-parton | Owner: john-
| parton
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by john-parton):

Coming back to this briefly.

The discussion was not particularly productive, but a specific use case of
generating bits of Elasticsearch documents or queries was mentioned:
https://forum.djangoproject.com/t/ability-to-specify-keys-for-jsonobject-
using-arbitrary-expressions/38559/4

The elasticsearch query language is a subset of JSON where the specific
field is represented as a key in a json object. Documentation:
https://www.elastic.co/docs/explore-analyze/query-
filter/languages/querydsl#query-filter-context-ex

If you have an EAV table and want to directly generate JSON documents
where the keys correspond to the entities in your table, you cannot do
that with the current JSONObject implementation. I give some reasoning
here: https://forum.djangoproject.com/t/ability-to-specify-keys-for-
jsonobject-using-arbitrary-expressions/38559/3

(I personally don't like EAV tables and would suggest storing extended
attribute directly as json documents on rows, but you might have a legacy
table design that prevents you from doing that.)

We can keep this wontfix, just wanted to come back and summarize, although
I am interested in SImon's opinion. (Not sure how to summon him. A
particular favorite food perhaps?)
--
Ticket URL: <https://code.djangoproject.com/ticket/36139#comment:9>