[Django] #27183: JSONField escaped in admin interface during normal usage
Django
-------------------------------+--------------------
Reporter: sjuxax | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.10
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------
The line of code at
https://github.com/django/django/blob/68de48c96328e13d5dbdb1f3006e4a1ca74f3c34/django/contrib/postgres/forms/jsonb.py#L45
results in the escaping of a plain JSON string. That's because the string
is already in valid JSON when run through json.dumps. This is user-facing
and visible in the Django admin interface when a JSONField is edited.
With the current code, the line in the admin looks like this:
http://imgur.com/fw29t1c.jpg
If we just return value instead of json.dumps(value), we get this:
http://imgur.com/msBT1tI.jpg
(Someone may want to attach those jpgs directly to this report for
posterity; I don't know what the convention is on this bug tracker)
In fact, looking at the code now, I wonder if we just need to swap lines
44 and 45. It would seem that if a value is invalid JSON, as the check on
line 43 implies, that we *should* be dumping it through the json
serializer, whereas if it's valid JSON, we shouldn't. Maybe the logic just
got swapped there.
#25532 deals with a similar topic.
--
Ticket URL: <https://code.djangoproject.com/ticket/27183>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------+--------------------------------------
Reporter: sjuxax | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by sjuxax):
* needs_better_patch: => 0
* needs_docs: => 0
* needs_tests: => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27183#comment:1>
Django
-------------------------------+--------------------------------------
Reporter: sjuxax | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
Are you using Django 1.10? What are the steps to reproduce?
--
Ticket URL: <https://code.djangoproject.com/ticket/27183#comment:2>
Django
-------------------------------+--------------------------------------
Reporter: sjuxax | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by sjuxax):
It should be noted that I am also using Mezzanine and seeing this issue
through Mezzanine's admin interface, though I haven't found this issue to
exist in Mezzanine's code -- as I said, the fix that's working for me is
in contrib/postgres/forms/jsonb.py. I haven't tested non-Mezzanine. I'm
using PgSQL 9.5.4, Django 1.10.1, Mezzanine git head (06b3df5380a5a),
Python 3.5.2.
Steps to reproduce: create a django.contrib.postgres.fields.JSONField in a
model. Migrate. Make sure the model is registered for admin. Load up the
admin. Input valid JSON into the JSONField and save. Open the newly-added
record in the admin. Observe that the JSONField is improperly escaped.
--
Ticket URL: <https://code.djangoproject.com/ticket/27183#comment:3>
Django
-------------------------------+--------------------------------------
Reporter: sjuxax | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
I still can't reproduce. What's your valid JSON input? It would be useful
to know if you can reproduce without Mezzanine to rule out an issue there.
--
Ticket URL: <https://code.djangoproject.com/ticket/27183#comment:4>
Django
----------------------------------+--------------------------------------
Reporter: sjuxax | Owner: nobody
Type: Bug | Status: closed
Component: contrib.postgres | Version: 1.10
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* status: new => closed
* resolution: => needsinfo
* component: Uncategorized => contrib.postgres
* type: Uncategorized => Bug
--
Ticket URL: <https://code.djangoproject.com/ticket/27183#comment:5>
Django
----------------------------------+--------------------------------------
Reporter: Jeff Cook | Owner: nobody
Type: Bug | Status: new
Component: contrib.postgres | Version: 1.10
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by ErikW):
* status: closed => new
* resolution: needsinfo =>
Comment:
I too have been impacted by this bug. I'm not using either Mezzanine or
django admin.
I have the field declared on my like like so:
from django.contrib.postgres.fields import JSONField
class FeedStatus(models.Model)
data = JSONField(default=dict)
When I try to access `feed_status.data` -- it returns JSON as a string (ie
"{}" instead of {}}. Each time the object is saved, the empty string is
re-serialized (ie '"\\"{}\\""').
```
In 18:44:50 [30]: f = FeedStatus.objects.get(pk=25806)
In 18:44:54 [31]: f.data = {}
In 18:44:56 [32]: f.save()
In 18:44:59 [33]: f = FeedStatus.objects.get(pk=25806)
In 18:45:01 [34]: f.data
Out 18:45:01 [34]: '{}'
In 18:45:03 [35]: f.save()
In 18:45:05 [36]: f = FeedStatus.objects.get(pk=25806)
In 18:45:07 [37]: f.data
Out 18:45:07 [37]: '"{}"'
In 18:45:10 [38]: f.save()
In 18:45:11 [39]: f = FeedStatus.objects.get(pk=25806)
In 18:45:16 [40]: f.data
Out 18:45:16 [40]: '"\\"{}\\""'
In 18:45:19 [41]: f.save()
In 18:45:20 [42]: f = FeedStatus.objects.get(pk=25806)
In 18:45:23 [43]: f.data
Out 18:45:23 [43]: '"\\"\\\\\\"{}\\\\\\"\\""'
```
This of course leads to incredibly bloated columns in the database (I've
found some that are 10's of MB in size when they should just be a blank
dict!!!)
I'm on Django 1.10.7.
--
Ticket URL: <https://code.djangoproject.com/ticket/27183#comment:6>
Django
----------------------------------+--------------------------------------
Reporter: Jeff Cook | Owner: nobody
Component: contrib.postgres | Version: 1.10
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* status: new => closed
* resolution: => duplicate
Comment:
Are you using django-jsonfield? See ticket:27675#comment:8.
--
Ticket URL: <https://code.djangoproject.com/ticket/27183#comment:7>