Builtin JSON Support

[Bryan White]

Does V8 have any built-in support for JSON encode/decode? There are a
log of mentions of JSON in these source files;
debug.cc
debug-delay.js
mirror-delay.js

But I don't seen any clear indication of if this stuff is intended to
be exposed or how it should be used from either C++ or JavaScript
code.

I know I can compile and run to decode but that has security implications.

I don't have a problem adding JSON support via my own code or finding
some JavaScript to do the job. I just want to know If I am missing
the obvious.

--
Bryan White

[Christian Plesner Hansen]

No, there is no built-in support for JSON. The debugger does use it
internally but ordinary scripts don't have access the debugger
internals.

-- Christian

[Simon Ask Ulsnes]

It's not that hard to add, though.

I did something like this:

Object.prototype.inspect = function(json) {
    var kv = this.map_with_key(function(k, v) { return k.toString() + ": " + v.inspect(json) });
    return "{" + kv.join(", ") + "}";
};

String.prototype.inspect = function(json) { return "\"" + this.replace(/\"/g, '\\"') + "\""; };
Function.prototype.inspect = function(json) { return json ? this.toString() : "(function)"; };
Number.prototype.inspect = function(json) { return this.toString(); }
Array.prototype.inspect = function(json) { return "[" + this.map(function(v) { return v == undefined ? "undefined" : v.inspect(json); }).join(", ") + "]"}
RegExp.prototype.inspect = function(json) { return this.toString(); }

Then you get the JSON by calling myObject.inspect(true).

You probably need specific inspect-functions for more prototypes than I have provided here, but I hope the idea is clear. :-)

Oh, by the way, the above depends on the following convenience functions (many JS libraries provide something similar):

Object.prototype.map_with_key = function(f) {
    var result = new Array();
    for (var key in this) {
        result[result.length] = f(key, this[key]);
    };
    return result;
};

Array.prototype.map = function(f) {
    var result = new Array();
    for (var i = 0; i < this.length; ++i)
    {
        result[i] = f(this[i]);
    }
    return result;
};

- Simon

2008/9/10 Christian Plesner Hansen <christian.pl...@gmail.com>

[Morgain...@googlemail.com]

On Sep 10, 3:04 pm, "Simon Ask Ulsnes" <vinterb...@gmail.com> wrote:
> It's not that hard to add, though.
>
> I did something like this:
>

> ...

Rather than hacking some basic JSON support into V8 or your V8
application, I recommend that it be done the pro' way, using Ragel:

http://www.complang.org/ragel/

Ragel's compiled state machines are blindingly fast, and hence so are
the implementations of JSON of languages that use it:

http://json.rubyforge.org/
http://modules.gotpike.org/module_info.html?module_id=43

I did some elementary benchmarking of JSON implementations a while
back, and JSON in Ragel was in a class of its own. :-)

Slowing down V8's lightning speed with a poor JSON would be so
sad. ;-)

Morgaine.

[Simon Ask Ulsnes]

Excuse me?

The whole point of JSON is to be representing objects in valid
JavaScript syntax. If parsing JSON using V8's internal parser is too
slow for you, then it doesn't really make sense to use V8 at all. ;-)

Besides, my example generates JSON. Ragel can only be used to parse it.
I'm not doubting that Ragel is an excellent state machine compiler,
but you're already using one compiler for your code, why not use the
same for your data, when you even have the luxury that your code and
your data is in the same format?

- Simon

2008/10/8 <Morgain...@googlemail.com>:

[Pete Gontier]

In many situations, it's perfectly reasonably to pass JSON data to 'eval' (or V8). However, if one accepts JSON from untrusted parties, one must assume what is ostensibly JSON might actually be malicious JavaScript. The "whole point of JSON" is really to be a compact/portable data format; the fact that it has syntax and semantics which are a precise subset of JavaScript is merely convenient.

Pete Gontier <http://pete.gontier.org/>

[Simon Ask Ulsnes]

That is a very good point. The cases when JSON is accepted from
untrusted parties are rare, though, but theoretically possible (most
cases I'm assuming is a webserver returning JSON from an AJAX call,
where no third party is involved). If V8 is theoretically used for
writing server-side web applications, I guess there would be a need
for a way to avoid this type of injection attacks (similar to how
current frameworks avoid SQL injection attacks from untrusted input).

- Simon

2008/10/9 Pete Gontier <pe...@gontier.org>:

[David Champion]

> where no third party is involved). If V8 is theoretically used for
> writing server-side web applications, I guess there would be a need
> for a way to avoid this type of injection attacks (similar to how
> current frameworks avoid SQL injection attacks from untrusted input).

It's not just for server-side JavaScript. There are any number of
conditions where an interpreter might parse JSON from an unknown or
untrusted source.

That's why there are four implementations of JSON parsers for JavaScript
at json.org, each of which strictly parses JSON instead of blindly
eval()ing. Has anyone done performance analysis to show that it even
needs to be reimplemented?

--
-D. d...@uchicago.edu NSIT University of Chicago

[Simon Ask Ulsnes]

> It's not just for server-side JavaScript. There are any number of
> conditions where an interpreter might parse JSON from an unknown or
> untrusted source.

Can you mention an example or two? I'm interested, because I couldn't
think of other examples. :-)

[Pete Gontier]

There are those who would argue any URL whose scheme is http (as opposed to https) should be treated as untrusted. Even if your code lives in <http://example.com/page.html> and it conducts an XHR transaction with <http://example.com/script.php>, the latter cannot be trusted, some would say.

But you don't have to try even that hard to come up with an example. Imagine a transaction similar to SOAP between two servers.

[Pete Gontier]

I don't know of any data which would support any claims about how JSON is used. I assume only that it's used in lots of creative ways, some of which are mentioned in other messages in this thread.

When you say if V8 is theoretically used for server-side web applications, I immediately think of all the traffic on this list which implies that is already the case. I also know of several related projects, including mine (which I really wish I had more time to work on).

Pete Gontier <http://pete.gontier.org/>

[David Champion]

I'll incorporate what Pete Gontier said by reference, then add that
you seem to be thinking of V8 only in terms of its role in Google
Chrome. Once you begin to think of it simply as an embeddable ECMA-262
implementation, the possibilities for this situation are endless.

* People are talking about V8 as a scripting engine for games. Consider
downloadable skins, extensions, modules, etc for such a game which might
store certain data in JSON object files.

* JavaScript could be a platform implementation language for components
in any web application framework; JSON data might be loaded from
any data provision source (with or without an HTTP proxy involved).
Imagine, say, an analysis engine for electoral data which allows the
user to point the applicaton to any source of data accessible by the
browser. It need not necessarily be a browser (i.e., application
hosting platform) that provides standard XSS privilege limitations.

* I might develop a GPS interface program that downloads tracking data
from my GPS device and stores it in JSON format. Some visualization
product using V8 JavaScript as an internal implementation language loads
that data and does something clever with it.

I don't know, I just made those up off the top of my head. Keep going,
it's easy once you stop thinking of JS as an exclusive property of web
browsers. :)

[Simon Ask Ulsnes]

Oh, I'm all for using JavaScript anywhere and all over the place!
Especially now we have such an excellent interpreter for it.

Data source trust issues are important to consider. It is, however,
fairly irrelevant to the original criticism, which was related to the
performance of parsing JSON structures using eval() in V8.

Using eval() is always dangerous. But I fail to see the need for an
external parser just for JSON. If you are exchanging data with an
external source over an unencrypted connection, then yes, using eval()
is a critical bug. But there are other ways.

For instance, it would be fairly simple to implement an evalJSON()
method that rejects all function calls. To obtain production-ready
security, you probably need more than that, but I'm just throwing
examples out there.

- Simon

2008/10/9 David Champion <d...@uchicago.edu>:

[Pete Gontier]

I don't (yet) have an opinion on the relative merits of different alternatives, but likewise I am perfectly willing to entertain the possibility that eval should be ignored when untrusted data sources are involved. I think the smart way to go, as another poster mentioned, is to measure the alternatives to see if they have significant disadvantages, and, if so, each developer can design accordingly.

Pete Gontier <http://pete.gontier.org/>

P.S. Encryption does not imply trust. Identity might.

[Morgain...@googlemail.com]

On Oct 9, 10:02 am, "Simon Ask Ulsnes" <si...@ulsnes.dk> wrote:
>
> Data source trust issues are important to consider. It is, however,
> fairly irrelevant to the original criticism, which was related to the
> performance of parsing JSON structures using eval() in V8.

Performance is critical, particularly in a language like V8 that is
noted for its speed. It's because of the slowness of many built-in
JSON parsers that so many other implementations have appeared. Nobody
likes slow data interchange because it makes your application seem
unresponsive. On the server side any slowdown limits your scalability
too, especially when the JSON implementation is layered as in many
languages that want to avoid the dangers of direct eval(). In
addition to that, JSON tends to be selected over XML precisely because
it can be a lot faster because it's less verbose and simpler to
parse. Providing a slow JSON defeats the purpose of choosing JSON.

> I fail to see the need for an external parser just for JSON.

It wouldn't be an external parser, just a separate, machine-generated
module sitting within the V8 sources. You'd link in a Ragel compiled
state machine parser in the same way as you might link in a yacc'd
parser or a regular expression library. Once linked in, it's
internal, as much a part of the project as eval(), but without the
danger of eval().

Reinventing wheels is a waste of time, and reinventing wheels poorly
makes no sense at all. No JSON parser that you could put together in
a few weeks is likely to come anywhere close to the speed of Ragel
simply because Ragel's state machine is compiled --- it leaves yacc/
lex parsers in the dust, for example. That's no different to the
situation with Javascript implementations: they were all dead slow
until more advanced projects like V8 began to add internal
compilation. You don't do that kind of thing overnight, so a home-
grown JSON for V8 would be as slow as all those other first-generation
JSON parsers out there.

> If you are exchanging data with an
> external source over an unencrypted connection, then yes, using eval()
> is a critical bug. But there are other ways.

> Using eval() is always dangerous.

Not very useful to propose it then. ;-)

> The cases when JSON is accepted from untrusted parties are rare, though,
> but theoretically possible (most cases I'm assuming is a webserver returning

> JSON from an AJAX call, where no third party is involved).

That's back to front. The cases where JSON is accepted from untrusted
parties constitute the primary use case, since JSON is a serialization
format and hence is typically used for interchange between remote
machines, which by definition should NEVER trust each other. Just
about the only cases where JSON can be used in a trusted mode are for
private local data persistance, and even then it's highly debateable
whether it should be allowed to go anywhere near eval(). It will bite
you one day. Validation should never be an optional extra.

Morgaine.

[MikeM]

Morgaine, google this: "How to win friends and influence people"
Read the first link that comes back.

[Simon Ask Ulsnes]

I probably wouldn't actively advocate the eval-based solution either,
but I will say that it's a possibility in simple cases.

For instance, I have a project where V8 is used as a scripting engine
for a game, and data files are stored in JSON format. Here, trust
isn't really an issue. Of course, as soon as network traffic is
involved, different solutions need to be evaluated.

Oh, and right, by encryption I wrongly implied identity certificate
checks and secure key exchange methods etc. You're right.

- Simon

2008/10/10 Pete Gontier <pe...@gontier.org>:

[Morgain...@googlemail.com]

On Oct 10, 5:50 am, MikeM <Mi...@reteksolutions.com> wrote:
> Morgaine, google this:  "How to win friends and influence people"
> Read the first link that comes back.

Life's too short. :-)

I gave the reasons why Ragel is a good way to do JSON in my 3 links,
and it doesn't need extra description from me to justify it. If after
understanding that approach and its benefits, someone sees a technical
problem with it and believes that an alternative might be better for
V8, great, that can be discussed technically and objectively. But we
weren't heading down that road. We were being paddled up a non-
technical creek based on a lack of understanding of the dangers of
unvalidated input(), and towards yet another handcrafted hack like so
many that have plagued JSON in the past in other languages. V8
deserves better.

When I evaluated 1-4 different implementations of JSON for each of 6
different languages about a year ago, I found a range of performances
spanning nearly 3 orders of magnitude. Not only was JSON in Ragel way
out ahead, but it's also a formal and safe approach, and so I now
advocate it. Arguments about technical merit I welcome and are very
necessary. Making friends and influencing people is an optional
extra. :-)

Morgaine.

[Pete Gontier]

This evening, I integrated Crockford's JSON codec and collected a few numbers regarding its performance. I thought folks here might be interested in the results.

http://partcounter.blogspot.com/2008/10/debut-json.html

Pete Gontier <http://pete.gontier.org/>

[Pete Gontier]

I've now posted another set of numbers (on the same page) which make a lot more sense.

Pete Gontier <http://pete.gontier.org/>