[Django] #33284: JSONFIeld decode error with Python 3.9
Django
-------------------------------------+-------------------------------------
Reporter: Alan D. | Owner: nobody
Snow |
Type: Bug | Status: new
Component: Database | Version: 3.2
layer (models, ORM) |
Severity: Normal | Keywords: python, json
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
-------------------------------------+-------------------------------------
https://code.djangoproject.com/ticket/31973
Traceback:
{{{
venv/lib/python3.9/site-packages/django/db/models/fields/json.py:83: in
from_db_value
return json.loads(value, cls=self.decoder)
}}}
{{{
def loads(s, *, cls=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, object_pairs_hook=None,
**kw):
"""Deserialize ``s`` (a ``str``, ``bytes`` or ``bytearray``
instance
containing a JSON document) to a Python object.
``object_hook`` is an optional function that will be called with
the
result of any object literal decode (a ``dict``). The return value
of
``object_hook`` will be used instead of the ``dict``. This feature
can be used to implement custom decoders (e.g. JSON-RPC class
hinting).
``object_pairs_hook`` is an optional function that will be called
with the
result of any object literal decoded with an ordered list of
pairs. The
return value of ``object_pairs_hook`` will be used instead of the
``dict``.
This feature can be used to implement custom decoders. If
``object_hook``
is also defined, the ``object_pairs_hook`` takes priority.
``parse_float``, if specified, will be called with the string
of every JSON float to be decoded. By default this is equivalent
to
float(num_str). This can be used to use another datatype or parser
for JSON floats (e.g. decimal.Decimal).
``parse_int``, if specified, will be called with the string
of every JSON int to be decoded. By default this is equivalent to
int(num_str). This can be used to use another datatype or parser
for JSON integers (e.g. float).
``parse_constant``, if specified, will be called with one of the
following strings: -Infinity, Infinity, NaN.
This can be used to raise an exception if invalid JSON numbers
are encountered.
To use a custom ``JSONDecoder`` subclass, specify it with the
``cls``
kwarg; otherwise ``JSONDecoder`` is used.
"""
if isinstance(s, str):
if s.startswith('\ufeff'):
raise JSONDecodeError("Unexpected UTF-8 BOM (decode using
utf-8-sig)",
s, 0)
else:
if not isinstance(s, (bytes, bytearray)):
> raise TypeError(f'the JSON object must be str, bytes or
bytearray, '
f'not {s.__class__.__name__}')
E TypeError: the JSON object must be str, bytes or
bytearray, not list
../../miniconda/envs/midas/lib/python3.9/json/__init__.py:339: TypeError
}}}
Suggested fix is to catch `TypeError` as well as `JSONDecodeError` here:
https://github.com/django/django/blob/adb4100e58d9ea073ee8caa454bb7c885b6a83ed/django/db/models/fields/json.py#L82-L85
{{{
try:
return json.loads(value, cls=self.decoder)
except (json.JSONDecodeError, TypeError):
return value
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33284>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: duplicate
Keywords: python, json | Triage Stage:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):
* status: new => closed
* resolution: => duplicate
* easy: 1 => 0
Comment:
Duplicate of #31973.
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:1>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: duplicate =>
Comment:
This is not a duplicate. It is related to #31973, but a fix has not been
applied yet.
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:2>
Django
-------------------------------------+-------------------------------------
Component: Database layer | Version: 3.2
(models, ORM) |
Keywords: python, json | Triage Stage:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => duplicate
Comment:
#31973 is closed as invalid, so "fix" will not be applied.
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:3>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: closed => new
* has_patch: 0 => 1
* resolution: duplicate =>
Comment:
https://github.com/django/django/pull/15083
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:4>
Django
-------------------------------------+-------------------------------------
Component: Database layer | Version: 3.2
(models, ORM) |
Keywords: python, json | Triage Stage:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => duplicate
Comment:
Alan, please stop and don't reopen closed tickets without providing any
new details.
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:5>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: duplicate
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Alan D. Snow):
I am confused. What is the resolution?
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:6>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: duplicate
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Alan D. Snow):
Is there something wrong with this?
{{{
Table "public.table"
Column | Type | Collation | Nullable |
Default
------------------+--------------------------+-----------+----------+--------------------------------------------------------------
id | integer | | not null |
nextval('table_id_seq'::regclass)
metadata | jsonb | | not null |
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:7>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: duplicate
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Alan D. Snow):
When I do:
{{{
class Table(models.Model):
metadata = JSONField(
blank=True,
default=dict,
help_text="Any additional metadata.",
)
}}}
And create a model and query for it:
{{{
table = Table.objects.create(metadata={"A": "B"})
found = table.objects.first()
}}}
I get an error.
If I set found.metadata = "", and query fort it, the error goes away.
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:8>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: duplicate
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Alan D. Snow):
There is no issue with Python 3.8.
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:9>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: duplicate
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Alan D. Snow):
Current workaround:
{{{
class JSONFieldPatch(models.JSONField):
"""
Patch for Python 3.9 since json returns TypeError
"""
def from_db_value(self, value, expression, connection):
try:
return super().from_db_value(value, expression, connection)
except TypeError:
return value
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:10>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Keywords: python, json | Triage Stage:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):
* resolution: duplicate => needsinfo
Comment:
The provided model works for me with Python 3.8 and Python 3.9. I don't
think you've explained the issue in enough detail to confirm a bug in
Django. Please reopen the ticket if you can debug your issue and provide
details about why and where Django is at fault.
--
Ticket URL: <https://code.djangoproject.com/ticket/33284#comment:11>