Dropping django.utils.simplejson

[Łukasz Rekucki]

Alex's comment on ticket #18013 reminded me of this. Is there any
reason not to get rid of the Django's version of simplejson now that
Python 2.6 always has the json module?

I see three options here:

1) Remove Django's copy and only leave the simplejson/json fallback.
2) Above, plus deprecate "django.utils.simplejson" in 1.5 and remove it 1.6
3) Replace the code with anyjson, so it does something useful:
http://pypi.python.org/pypi/anyjson

What do you think ?

--
Łukasz Rekucki

[Alex Gaynor]

--
Łukasz Rekucki

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

I'd favor number two.

Alex

--
"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

[Russell Keith-Magee]

Option 2 looks best to me. There's no reason for us to ship JSON any more, but we should still guide people through the transition process.

Yours,
Russ Magee %-)

[Alex Ogier]

There's still a ~20x performance gain over 2.6's stdlib by using
simplejson even without C-extensions according to
http://bugs.python.org/issue6013. Depending on how heavily people
depend on the module, removing simplejson may be a "backwards
incompatibility" in the sense that performance may nosedive when they
upgrade to Django 1.6. Django has always considered system-simplejson
> system-json > vendored-simplejson. This means that in 2.6+ no one
should be using the vendored simplejson, but the shim is still
valuable for people who use their system's simplejson (possibly with C
extensions) whenever available.

Therefore I am in favor of option #1, unless the shim is so trivial as
to warrant asking any developers who use it to rewrite it themselves.

Best,
Alex Ogier

[Alex Ogier]

So in the process of removing simplejson I realized it's not strictly
true that we no longer need to bundle it: if someone uses 2.6+ but has
a 'json' module with a different interface sitting in front of the
system json on sys.path, then Django currently falls back to bundled
simplejson. I don't know how common that is, most alternate json
modules appear pretty inactive but you never know.

Also, whether or not that is common, it turns out there has been a bug
that looks like it was added last October that means that no one has
been using the system json module at all.

Best,
Alex Ogier

[Alex Gaynor]

Yeah, that's not a usecase we need to support.  If you stick random crap on your python path and shadow standard library modules, stuff rbeaks, this shouldn't come as a surprise.

[Florian Apolloner]

Hi,

I am for number 2 too, but don't forget that's deprecation in 1.5 and 1.6 and removal in 1.7

Cheers,
Florian

[Alex Ogier]

It seems like everyone besides me agrees on option #2 so I implemented it in #18023 ( https://github.com/ogier/django/compare/remove-simplejson ). At the same time, I want to reiterate my support for option #1: not deprecating the module and leaving the shim in for the foreseeable future. If simplejson is available on the system, and particularly if it has been compiled with C extensions, then there is a significant performance gain to be had, so why not continue to take advantage of that in all the places that django serializes to json?

Best,

Alex Ogier

--

You received this message because you are subscribed to the Google Groups "Django developers" group.

To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/mzYEOlCKfHkJ.

[Łukasz Rekucki]

On 30 March 2012 13:04, Alex Ogier <alex....@gmail.com> wrote:
> At the same time, I want to reiterate my support for option #1: not deprecating the
> module and leaving the shim in for the foreseeable future. If simplejson is
> available on the system, and particularly if it has been compiled with C
> extensions, then there is a significant performance gain to be had, so why
> not continue to take advantage of that in all the places that django
> serializes to json?

I agree this is a valid concern and I think it should be mentioned in
release notes. If you want more performance, then option #3 (anyjson)
would actually be better, but I don't think it should be Django's
concern to choose the best JSON package for you (we don't do that for
XML). Instead, imho, Django should provide an ability to pass a custom
JSON encode/decoder to all APIs that require it, so you can decide
about it at the project level.

--
Łukasz Rekucki

[Alex Ogier]

I did some timing tests.  https://code.djangoproject.com/ticket/18023#comment:5

An order of magnitude difference in JSON serialization time corresponds to an 8% change in total serialization time for a complex model, and 5 deeply nested model instances can be serialized in < 2 ms. It might be a little different depending on how deeply your models are related, but it convinced me that splitting hairs over the JSON serializer isn't worth it. I had the idea in my head that serialization might be a bottleneck in the same way that template rendering is, but I realize now it's very different. Heavily nested and unoptimized templates thrash the disk and change a 10ms response into a 100ms response. Shoddy JSON serializers might thrash the heap, or make the GC work overtime, but it's all in memory so the worst that happens is that a 10ms response becomes a 12ms response. It's just not worth worrying about.

Best,

Alex Ogier

--
You received this message because you are subscribed to the Google Groups "Django developers" group.

[Mikhail Korobov]

Standard python 2.6 json is a bundled and cleaned up simplejson without C extension and pre-26 compatibility - even if json serialization is not a common bottleneck why make things worse? Simplejson and python 2.6 json are arguably the same libraries, I think this is like choosing between cStringIO vs StringIO. 

суббота, 31 марта 2012 г. 3:03:00 UTC+6 пользователь Alex Ogier написал:

Alex Ogier

To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to django-developers+unsubscribe@googlegroups.com.