JSON parsing: too slow

[Eister Kapelm]

The total size of my JSON responses is around 50k to 70k. Since I have
5 requests, each of them is varying between 5k to 40k. The problem is,
it's taking too long to display the objects.

I'm testing the webpage in 2 machines:
A = dual core 1GB memory
B = Pentium4 512MB memory

When I run Hosted Mode in machine A, it takes almost 1 minute to show
everything since the first request is sent. In IE, it takes around 20
seconds.

But in machine B, it takes 3 to 5 minues in Hosted Mode and around 2
to 3 minutes in IE.

The back end is php and the response is usually fast. Plus, the
JSONParser.parse uses 100% of the CPU in machine B and keeps using it
fully until the parsing is done, which takes a big while. After
parsing, I'm retrieving the values of the response in a JSONValue
array and creating the tables looping it.

So these are going to be really basic questions but:
1. Does JSON parsing normally take this long?
2. Is XML parsing faster than JSON parsing?

[Aaron Digulla]

On Mar 14, 8:55 am, "Eister Kapelm" <flaviohiras...@gmail.com> wrote:

> 1. Does JSON parsing normally take this long?

Yes, JSON parsing of many objects takes a lot of time. Here are some
figures from my app (dual core, 2GB RAM):

- Result (1.4MB) returned as a single string. Parsing < 1s
- Returned as list of Strings: ~20 seconds
- Returnes as list of String arrays: > 1 minute

> 2. Is XML parsing faster than JSON parsing?

Is XML parsing possible with GWT?

[Reinier Zwitserloot]

A simple workaround to this problem is to not use GWT's JSON layer and
use eval() directly using some JSNI magic. Though I think if you use
that technique you'll have to fetch all your data through JSNI.

in that situation, JSON parsing is definitely way faster than XML
parsing, which would also take JSNI magic as far as I know.

The alternative is to not use JSON but your own format or write your
own JSON parser, though that sounds like a lot of needless work.

Is the JSON parser accidentally slow, or can it simply not be made to
work faster?

[Miguel Méndez]

I may have misunderstood your post so just bear with me.  In hosted mode, I would expect the whole process to take longer because we have to transition data across the Java<->JavaScript boundary to get the JSON objects and to populate the table.  However, I would expect web mode to be much faster because it is all JavaScript.

In order to get a better understanding of what is going on, I think that you should see how much of the time is spent parsing the JSON response, without actually populating the table, for hosted and web modes.  Tables can get slow and it is hard to know, based on your numbers, whether the JSON code or the table are the major time consumers.  Once you have that baseline, you can add your table code back in and compare the numbers again.

Once we have the timing information and some sample data we can decide how to proceed.

HTH,

[Eister Kapelm]

> Yes, JSON parsing of many objects takes a lot of time. Here are some
> figures from my app (dual core, 2GB RAM):
>
> - Result (1.4MB) returned as a single string. Parsing < 1s
> - Returned as list of Strings: ~20 seconds
> - Returnes as list of String arrays: > 1 minute

Thanks Aaron for your information, now I have an idea of how slow is
the JSON parsing. Well, I don't know how you handle this situation,
but in my case the data I'm fetching is the minimum I need to have my
system working and this parsing issue is giving me a headache.

> Is XML parsing possible with GWT?

Since I've been working with raw JavaScript before using GWT and
always using JSON, I didn't try the XML parser in GWT yet, but as long
I see the documentation and posts in this forum, the answer is yes.
http://code.google.com/webtoolkit/documentation/com.google.gwt.xml.client.html

[Eister Kapelm]

Thanks Reinier, I didn't think of using eval until you've told me.

I'll try few things to use eval and GWT JSON types, but if I can't,
I'm not going to create my own parser, it would be a lot of needless
work as you said. Even if I can't use the GWT JSON types, fetching the
data with JSNI is a good solution I think.

> Is theJSONparser accidentally slow, or can it simply not be made to work faster?
This is something I believe it's possible to improve in GWT since in
JavaScript it's very fast.

[Eister Kapelm]

Thank you Miguel for your help.

> I may have misunderstood your post so just bear with me. In hosted mode, I would expect the whole process to take longer because we have to transition data across the Java<->JavaScript boundary to get the JSON objects and to populate the table. However, I would expect web mode to be much faster because it is all JavaScript.

-- This is exactly the idea I had since I was used to work with raw
JavaScript and JSON before using GWT. Regarding this JSON parsing
matter, I don't know how the Java code is changed into JavaScript when
compiled, but what I think now is that by using GWT's JSON parsing it
turns the compiled JavaScript parsing also slow, perhaps by checking
types of each element while parsing (?).

> In order to get a better understanding of what is going on, I think that you should see how much of the time is spent parsing the JSON response, without actually populating the table, for hosted and web modes. Tables can get slow and it is hard to know, based on your numbers, whether the JSON code or the table are the major time consumers. Once you have that baseline, you can add your table code back in and compare the numbers again. Once we have the timing information and some sample data we can decide how to proceed.

-- I totally agree with you, but I didn't write about the time GWT
takes to build the tables in Hosted Mode because from the 5 responses
I receive, I only create 2 visible ui objects by default and I
consider it's fast enough. I retrieve the other 3 in memory to create
other visible uis after user's actions, and these are the minimum data
I need, because I can't make the user wait for the fetching time after
his actions.

One of these 2 default visible uis is a select object, with only few
options in it. It takes much less than 1 second. The other is a 9x16
table, but even in Hosted Mode it's created and showed in about 1 to
1.5 seconds.

Can you please let me know:
1. your parsing in Web Mode is as fast as when using raw JavaScript?
2. what's the average amount of objects you have in your JSON
response?

I'm asking the 2. above because as Aaron posted, the more objects you
have, the longer is the time to parse, and in my case I have hundreds
of objects.

Also please let me know if I'm not being clear again. =)

Thanks a lot.

[Miguel Méndez]

Hi Eister,

Our JSON library basically evals the JSON string during in the parse method and stores it.  We extract the elements of this object as they are requested.  In web mode, our "parsing" should be about as fast as JSON although I do not have numbers to quote. 

If you consider "parsing" to include enumerating all of the contents of the response to then doing something like populating a grid then it could take longer.

Can you please let me know:
1. your parsing in Web Mode is as fast as when using raw JavaScript?

2. what's the average amount of objects you have in your JSON
response?

Which JSON response?

Cheers,

--
Miguel

[Andrés Testi]

The JSON library is slow because their type inference. For each node
in the JSON object tree, the GWT JSON library test each possible type
(string, number, array, boolean, null and object). This recursive
loop, decrease the performance. With the GWT Jsonizer (
http://code.google.com/p/gwt-jsonizer ) , type inference is not
needed, because each JSON object is maped to a bean, then the type of
the JSON property must be consistent with the bean property. For this
reason, GWT Jsonizer is faster than GWT JSON library.

[Eister Kapelm]

Hi Miguel,

Thank you very much for your reply and my apologies for the late
reply.

The "JSON response" I'm talking about is "the response from the server
in JSON format." In my case, I have hundreds of JSON Objects and JSON
Arrays in the response sent by the server. As Aaron and Andres said,
the amount of nodes in the JSON string is the cause of its bad
performance.

Regards,

Eister Kapelm

[Eister Kapelm]

Hi Andres,

Thanks a lot for your reply.

I suspected this type verifying loop and you just made it clear.
Thanks!

I still have to try your Jsonizer, it seems to be very interesting if
I keep working with large JSON object trees in Hosted Mode.

I thought GWT was doing the same type inference after compiling (it
means, type verifying also in JavaScript), but accordingly to Dan
Morrill's GWT security article (http://groups.google.com/group/Google-
Web-Toolkit/web/security-for-gwt-applications), GWT uses JavaScript's
eval() to parse JSON strings. Do you know if the browser end up doing
this type inference when JavaScript's eval() is called? Do you think
parsing a serialized string is faster then parsing a JSON string with
hundreds of objects?

Thanks in advance.

Eister Kapelm

On Apr 6, 10:22 am, "Andrés Testi" <andres.a.te...@gmail.com> wrote:
> TheJSONlibrary is slow because their type inference. For each node
> in theJSONobject tree, the GWTJSONlibrary test each possible type

> (string, number, array, boolean, null and object). This recursive

> loop, decrease the performance. With the GWT Jsonizer (http://code.google.com/p/gwt-jsonizer) , type inference is not
> needed, because eachJSONobject is maped to a bean, then the type of
> theJSONproperty must be consistent with the bean property. For this

> reason, GWT Jsonizer is faster than GWTJSONlibrary.
>
> On 4 abr, 12:40, "Miguel Méndez" <mmen...@google.com> wrote:
>
>
>
> > Hi Eister,
>

> > OurJSONlibrary basically evals theJSONstring during in the parse method

> > and stores it. We extract the elements of this object as they are
> > requested. In web mode, our "parsing" should be about as fast asJSON
> > although I do not have numbers to quote.
>
> > If you consider "parsing" to include enumerating all of the contents of the
> > response to then doing something like populating a grid then it could take
> > longer.
>
> > Can you please let me know:
>

> > > 1. yourparsingin Web Mode is as fast as when using raw JavaScript?

>
> > 2. what's the average amount of objects you have in yourJSON
>
> > > response?
>
> > WhichJSONresponse?
>
> > Cheers,
>
> > --

> > Miguel- Hide quoted text -
>
> - Show quoted text -