"call stack size exceeded" when serializing nested HashMap objects

[dandante]

I have an RPC service method like this:

/**
* @gwt.typeArgs <java.lang.String,java.util.HashMap>
*/
public HashMap getConditionTreeModel();

The values in this hashmap are in turn hashmaps with String keys and
HashMap values. They only go about 10 or 12 deep. This hash is meant
to be used to populate a Tree object on the client side.

When I call this RPC method from client code, the onSuccess() method
of the callback is called, and in my case that method is empty, and at
that point I get an error.
In hosted mode the stack trace I get is pasted at the end of this
message.

In web mode the error is:
JavaScript InternalError exception: too much recursion

So, it looks like the server has no problem serializing this hashmap,
and the client receives it (at least, the onFailure() method is not
called), but before anything else can happen, I get the error. Note
that I don't even cast the object passed to onSuccess() back to a
hashmap--I don't do anything in onSuccess().

I don't think that 12-deep nested hashmaps should cause a stack
overflow/too much recursion error.

In fact, the way I worked around this was to pass a string
representation of the hashmap and create the hashmap on the client
side from that, then populate the Tree object from that. Even though
this involves a depth-first traversal of the nested hashes, none of
this causes any stack overflow/recursion errors. But it is much slower
to create the nested hashmaps on the client side than on the server
side. Unacceptably slow.

So is there a way around this?

Thanks!

Hosted mode stack trace:
com.google.gwt.core.client.JavaScriptException: JavaScript RangeError
exception: Maximum call stack size exceeded.
at com.google.gwt.dev.shell.ModuleSpace.invokeNative(ModuleSpace.java:
481)
at
com.google.gwt.dev.shell.ModuleSpace.invokeNativeVoid(ModuleSpace.java:
270)
at
com.google.gwt.dev.shell.JavaScriptHost.invokeNativeVoid(JavaScriptHost.java:
137)
at
org.systemsbiology.GWAP.client.PersistenceService_TypeSerializer.deserialize(PersistenceService_TypeSerializer.java:
194)
at
com.google.gwt.user.client.rpc.impl.ClientSerializationStreamReader.deserialize(ClientSerializationStreamReader.java:
94)
at
com.google.gwt.user.client.rpc.impl.AbstractSerializationStreamReader.readObject(AbstractSerializationStreamReader.java:
61)
at org.systemsbiology.GWAP.client.PersistenceService_Proxy
$7.onCompletionImpl(PersistenceService_Proxy.java:425)
at org.systemsbiology.GWAP.client.PersistenceService_Proxy
$7.onCompletionAndCatch(PersistenceService_Proxy.java:414)
at org.systemsbiology.GWAP.client.PersistenceService_Proxy
$7.onCompletion(PersistenceService_Proxy.java:408)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
com.google.gwt.dev.shell.mac.MethodDispatch.invoke(MethodDispatch.java:
74)
at org.eclipse.swt.internal.carbon.OS.ReceiveNextEvent(Native Method)
at org.eclipse.swt.widgets.Display.sleep(Display.java:3801)
at com.google.gwt.dev.GWTShell.pumpEventLoop(GWTShell.java:690)
at com.google.gwt.dev.GWTShell.run(GWTShell.java:550)
at com.google.gwt.dev.GWTShell.main(GWTShell.java:321)
com.google.gwt.core.client.JavaScriptException: JavaScript RangeError
exception: Maximum call stack size exceeded.
at com.google.gwt.dev.shell.ModuleSpace.invokeNative(ModuleSpace.java:
481)
at
com.google.gwt.dev.shell.ModuleSpace.invokeNativeVoid(ModuleSpace.java:
270)
at
com.google.gwt.dev.shell.JavaScriptHost.invokeNativeVoid(JavaScriptHost.java:
137)
at
org.systemsbiology.GWAP.client.PersistenceService_TypeSerializer.deserialize(PersistenceService_TypeSerializer.java:
194)
at
com.google.gwt.user.client.rpc.impl.ClientSerializationStreamReader.deserialize(ClientSerializationStreamReader.java:
94)
at
com.google.gwt.user.client.rpc.impl.AbstractSerializationStreamReader.readObject(AbstractSerializationStreamReader.java:
61)
at org.systemsbiology.GWAP.client.PersistenceService_Proxy
$7.onCompletionImpl(PersistenceService_Proxy.java:425)
at org.systemsbiology.GWAP.client.PersistenceService_Proxy
$7.onCompletionAndCatch(PersistenceService_Proxy.java:414)
at org.systemsbiology.GWAP.client.PersistenceService_Proxy
$7.onCompletion(PersistenceService_Proxy.java:408)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
com.google.gwt.dev.shell.mac.MethodDispatch.invoke(MethodDispatch.java:
74)
at org.eclipse.swt.internal.carbon.OS.ReceiveNextEvent(Native Method)
at org.eclipse.swt.widgets.Display.sleep(Display.java:3801)
at com.google.gwt.dev.GWTShell.pumpEventLoop(GWTShell.java:690)
at com.google.gwt.dev.GWTShell.run(GWTShell.java:550)
at com.google.gwt.dev.GWTShell.main(GWTShell.java:321)

[dandante]

On Dec 5, 3:48 pm, dandante <danda...@gmail.com> wrote:
> I have an RPC service method like this:
>
> /**
> * @gwt.typeArgs <java.lang.String,java.util.HashMap>
> */
> public HashMap getConditionTreeModel();

[...]


Correction...the onFailure() method of the callback is indeed called.
I said otherwise in my previous message but I was wrong.

[dandante]

Does anyone have a good approach to populating a Tree with data
retrieved from a RPC call?

Initially I tried representing the data as nested HashMaps, which is
what I have done in non-GWT applications.

However, GWT seemed to have a problem either serializing or
deserializing these maps (see my post about this below). So then I
used xstream on the server side to serialize the nested hashmap to
XML, then passed the XML (about 35K worth) via RPC and parsed it into
a hashmap, then did the necessary sorting and populated the Tree.

Works fine in hosted mode but in Web mode I get:

"JavaScript type error exception: this$static has no properties"

The error seems to occur in the first line of this fragment of the
generated javascript:

if (item.com_google_gwt_user_client_ui_TreeItem_parent !== null ||
item.com_google_gwt_user_client_ui_TreeItem_tree !== null) {
9965 com_google_gwt_user_client_ui_TreeItem_
$remove__Lcom_google_gwt_user_client_ui_TreeItem_2(item);
9966 }

What I want to do seems simple enough, so I'm sure others have tried
it and perhaps not run into these problems. By the way, for my
purposes, it is not feasible to populate various sub-parts of the tree
by using RPC calls; I need tthe whole thing, and again, it's not that
big.

Another thing I did that "worked" was to pass the information for
every single node in the tree as an array of Strings, where each item
was a colon-delimited path to a node. This string array contained a
lot of redundant information and was 2.5MB in size. The page took too
long to load.

All suggestions welcome.
Thanks

dandante wrote:
> I have an RPC service method like this:
>
> /**
> * @gwt.typeArgs <java.lang.String,java.util.HashMap>
> */
> public HashMap getConditionTreeModel();
>
> The values in this hashmap are in turn hashmaps with String keys and
> HashMap values. They only go about 10 or 12 deep. This hash is meant
> to be used to populate a Tree object on the client side.
>

> When I call this RPC method from client code, the onFailure() method

[Sachin]

So I have some experience with building a tree from an RPC call.
I had a similar problem with serializing a nested HashMap. So what i
ended up doing was using the apache betwixt libabry to turn the nested
hashmap into an XML document, transferring the xml over to the client
and then parsing rebuilding the hashmap on the client side from the
xml. The entire process was extremely fast.
The problem was after having populated the HashMap on the client side
then populating the tree with this HashMap, the tree creation process
in GWT is extremely slow, and i have mentioned it to Joel at the GWT
Conference. To overcome that I created my own tree widget from a
composite, that has three widgets at each node, one label for the
"+"/"-" sign, one label for the actual text and a hidden sub-panel for
holding the children.
There was another problem, I needed to maintain the order of the nodes
in the HashMap to be the same in the client and the server, however as
you might know, the order of the HashMap elements is not guaranteed to
be preserved, there is a java.util.LinkedHashMap to solve this
problem, but it is not supported under GWT :-(
So i ended up creating my own implementation using nested ArrayLists
and then serializing that using the betwixt library to convert it to
XML, then rebuild the nested ArrayList on the client side from the
XML.

After having done all of this, the performance is great, now i need to
do more customization, like implementing partially expanded tree,
multi-select etc.

I am also looking into using some libraries like mygwt or gwt-ext.
they give most of the functionality i need and the speed of rendering
the tree with those libraries is pretty fast, i might still have to
tweak them a little bit more to do exactly what i want the tree
control to do.
Hope this helps
-Sachin

> > com.google.gwt.dev.shell.JavaScriptHost.invokeNativeVoid(JavaScriptHost.jav-a:
> > 137)
> > at
> > org.systemsbiology.GWAP.client.PersistenceService_TypeSerializer.deserializ-e(PersistenceService_TypeSerializer.java:
> > 194)
> > at
> > com.google.gwt.user.client.rpc.impl.ClientSerializationStreamReader.deseria-lize(ClientSerializationStreamReader.java:
> > 94)
> > at
> > com.google.gwt.user.client.rpc.impl.AbstractSerializationStreamReader.readO-bject(AbstractSerializationStreamReader.java:

> > 61)
> > at org.systemsbiology.GWAP.client.PersistenceService_Proxy
> > $7.onCompletionImpl(PersistenceService_Proxy.java:425)
> > at org.systemsbiology.GWAP.client.PersistenceService_Proxy
> > $7.onCompletionAndCatch(PersistenceService_Proxy.java:414)
> > at org.systemsbiology.GWAP.client.PersistenceService_Proxy
> > $7.onCompletion(PersistenceService_Proxy.java:408)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> > 39)
> > at

> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp-l.java:

> > 25)
> > at java.lang.reflect.Method.invoke(Method.java:585)
> > at
> > com.google.gwt.dev.shell.mac.MethodDispatch.invoke(MethodDispatch.java:
> > 74)
> > at org.eclipse.swt.internal.carbon.OS.ReceiveNextEvent(Native Method)
> > at org.eclipse.swt.widgets.Display.sleep(Display.java:3801)
> > at com.google.gwt.dev.GWTShell.pumpEventLoop(GWTShell.java:690)
> > at com.google.gwt.dev.GWTShell.run(GWTShell.java:550)
> > at com.google.gwt.dev.GWTShell.main(GWTShell.java:321)
> > com.google.gwt.core.client.JavaScriptException: JavaScript RangeError
> > exception: Maximum call stack size exceeded.
> > at com.google.gwt.dev.shell.ModuleSpace.invokeNative(ModuleSpace.java:
> > 481)
> > at
> > com.google.gwt.dev.shell.ModuleSpace.invokeNativeVoid(ModuleSpace.java:
> > 270)
> > at

> > com.google.gwt.dev.shell.JavaScriptHost.invokeNativeVoid(JavaScriptHost.jav-a:
> > 137)
> > at
> > org.systemsbiology.GWAP.client.PersistenceService_TypeSerializer.deserializ-e(PersistenceService_TypeSerializer.java:
> > 194)
> > at
> > com.google.gwt.user.client.rpc.impl.ClientSerializationStreamReader.deseria-lize(ClientSerializationStreamReader.java:
> > 94)
> > at
> > com.google.gwt.user.client.rpc.impl.AbstractSerializationStreamReader.readO-bject(AbstractSerializationStreamReader.java:

> > 61)
> > at org.systemsbiology.GWAP.client.PersistenceService_Proxy
> > $7.onCompletionImpl(PersistenceService_Proxy.java:425)
> > at org.systemsbiology.GWAP.client.PersistenceService_Proxy
> > $7.onCompletionAndCatch(PersistenceService_Proxy.java:414)
> > at org.systemsbiology.GWAP.client.PersistenceService_Proxy
> > $7.onCompletion(PersistenceService_Proxy.java:408)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> > 39)
> > at

> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp-l.java:

> > 25)
> > at java.lang.reflect.Method.invoke(Method.java:585)
> > at
> > com.google.gwt.dev.shell.mac.MethodDispatch.invoke(MethodDispatch.java:
> > 74)
> > at org.eclipse.swt.internal.carbon.OS.ReceiveNextEvent(Native Method)
> > at org.eclipse.swt.widgets.Display.sleep(Display.java:3801)
> > at com.google.gwt.dev.GWTShell.pumpEventLoop(GWTShell.java:690)
> > at com.google.gwt.dev.GWTShell.run(GWTShell.java:550)

> > at com.google.gwt.dev.GWTShell.main(GWTShell.java:321)- Hide quoted text -
>
> - Show quoted text -

[dandante]

Sachin wrote:
> So I have some experience with building a tree from an RPC call.

[...]

This is very helpful. I may try a similar approach. Being both lazy
and always on the alert for other people who have solved the problem
better than I could, I am looking into MyGwt. They have a nice
serializable Model class to hold a Tree's data; however, when I send
it over RPC, the client does not receive any of the model's children,
so I end up with a tree with no nodes in it.

If I can't get that to work I'll look at gwt-ext or at some of the
homegrown solutions you discuss.

Thanks!

[gregor]

Hi Dandante,

Another approach is to populate trees, especially big ones, one layer
at a time on request as the user opens branches and fetching the top
layer only at start up. This is a fairly common technique and has the
advantages that a) the network and serialization overheads on each
request are small therefore efficient, and b) you can use a simple
ArrayList to transport each layer's worth of items over RPC which is
much simpler and avoids the problems you've encountered. The List.java
class in the KitchenSink sample provides a template for this approach,
minus the RPC element.

regards
Gregor

[Sachin]

two things i learnt at the GWT conference from the makers of GWT
themselves, RPC calls are expensive, make as little of them as
possible, caching data on the client instead, so transferring an
entire hashmap, or nested arraylist is much less costly.
Most importantly, the existing Tree control in GWT is horrendously
slow. Joel Webber has acknowledged this.
Until an alternative comes up, it is jus not workable.
Just try an example, just create a tree with 100 top level nodes, with
5 sub-nodes each, and 20 leaf nodes under those five nodes, on my
machine it did not even run.
Best bet right now, is use either MyGWT or GWT-Ext library.

> > Thanks!- Hide quoted text -

[gregor]

agreed, Sachin, but your 10,000 node tree example *would* work if you
fetched it in layers albeit, as you say, at the cost of a RPC round
trip per node opening. In my case I have an example like this, a
thesaurus containing about 3000 phrases. One of three navigation views
of it is a (reasonably balanced) tree averaging about 10 +/- 5 nodes
per level. I dismissed the idea of fetching all of it at start up,
and I find that, although not ideal, so long as you show a "please
wait.." type message from a proxy node the RPC turnaround provides a
reasonable user experience for this given that it is pretty clear to
them they are getting access to a large data set in a format
convenient for some purposes. The point is only a small fraction of a
tree of this size is likely to be of interest at any one time so in
practice only a few tens of nodes normally get to be rendered in HTML
in the browser and in easily digestible chunks. I do not dispute that
this may not be acceptable in all such use cases.

regarding MyGWT and GWT-ext alternatives, I havn't tried them yet, but
I am interested if their implementation approaches are fundamentally
different to standard GWT. As I undertand it GWT tree generates a
nested HTML structure like this:

// Uses the following Element hierarchy:
// <div (handle)>
// <table (itemElem)>
// <tr>
// <td><img (imgElem)/></td>
// <td><span (contents)/></td>
// </tr>
// </table>
// <span (childSpanElem)> children </span>
// </div>

It is easy to see how generating several thousand, or even hundred, of
these at once is unworkable. But is there in fact a radically
different approach to rendering a tree in a browser used by MyGWT/GWT-
ext that would, for example, enable my 3000 node tree to be digested
in one shot?

[dandante]

On Dec 10, 4:28 am, gregor <greg.power...@btinternet.com> wrote:
> regarding MyGWT and GWT-ext alternatives, I havn't tried them yet, but
> I am interested if their implementation approaches are fundamentally
> different to standard GWT.

I have spent some time exploring MyGWT and GWT-EXT.

They have pretty nice (as in nice and pretty) tree implementations.

However, I ran up against the same limitations with all of them. If I
transmitted my tree structure as a nested hashmap (or the Model class
that mygwt provides), I ran into problems with GWT-RPC's
deserialization method. Whatever it does to deserialize the structure
causes a stack overflow.

BTW, my tree structure is not (IMO) too huge. When represented as an
XML string (w/o whitespace) it's about 360K. Remove the XML tags and
it's probably 1/3 that size...I'd say the maximum depth of a leaf node
is probably about 14.

So the next thing I tried was to transmit the tree as an xml string.
No problem deserializing it, but then I ran into the same sorts of
issues when parsing the xml string and using it to populate the tree
widget.

One thing I didn't try was to use breadth-first traversal of the tree
structure; that might use less memory than the depth first method.
Whether I can do this depends on if the various tree widgets support
it.

Otherwise I will have to use the one-level-at-a-time via RPC approach
which is REALLY not what the original use case called for. We wanted
the whole tree available in one go so that we could highlight multiple
nodes based on application state.

> As I undertand it GWT tree generates a
> nested HTML structure like this:
>
> // Uses the following Element hierarchy:
> // <div (handle)>
> // <table (itemElem)>
> // <tr>
> // <td><img (imgElem)/></td>
> // <td><span (contents)/></td>
> // </tr>
> // </table>
> // <span (childSpanElem)> children </span>
> // </div>
>
> It is easy to see how generating several thousand, or even hundred, of
> these at once is unworkable. But is there in fact a radically
> different approach to rendering a tree in a browser used by MyGWT/GWT-
> ext that would, for example, enable my 3000 node tree to be digested
> in one shot?

I don't think so.

In fact, I wonder if even the one-level-at-a-time approach would fail
if you manually opened enough "folders" in the tree....

>
> On Dec 10, 7:49 am, Sachin <4sa...@gmail.com> wrote:
>
> > two things i learnt at the GWT conference from the makers of GWT
> > themselves, RPC calls are expensive, make as little of them as
> > possible, caching data on the client instead, so transferring an
> > entire hashmap, or nested arraylist is much less costly.

Maybe I can do this and still populate the tree only as needed, just
have the mechanism be client-side rather than RPC.

> > Most importantly, the existing Tree control in GWT is horrendously
> > slow. Joel Webber has acknowledged this.
> > Until an alternative comes up, it is jus not workable.
> > Just try an example, just create a tree with 100 top level nodes, with
> > 5 sub-nodes each, and 20 leaf nodes under those five nodes, on my
> > machine it did not even run.

Good to know.

> > Best bet right now, is use either MyGWT or GWT-Ext library.

I'll let you all know if I come up with a workable solution.

[markmccall]

Regarding the Stack Overflow, I have run into this issue as well, but
only in hosted mode. After posting my experience with it in these
forums I opened the following issue: http://code.google.com/p/google-web-toolkit/issues/detail?id=1438

Hopefully it will be resolved in the near future.

[gregor]

On Dec 10, 11:01 pm, dandante <danda...@gmail.com> wrote:

>
> BTW, my tree structure is not (IMO) too huge. When represented as an
> XML string (w/o whitespace) it's about 360K. Remove the XML tags and
> it's probably 1/3 that size...I'd say the maximum depth of a leaf node
> is probably about 14.
>

I don't think it's packet size/network latency that's the real problem
here - I think it's a) parsing the XML client side (whether raw or
handled by GWT collection emulation) and b) generating all the HTML
divs/trs etc from it.

[walden]

In my application I use RPC to fetch data that will be displayed in a
tree. Since the data are intrinsically hierarchical, I don't use
anything like a hash of hashes: I am passing domain objects where the
parent-child relationship (from the parent side) is an ArrayList. My
trees tend to be fairly flat, but the longest one can have around 1300
nodes at the root level. (The application is designed to mostly avoid
such a big working set.) Performance of loading up the tree is not
great, but it's acceptable to the client.

I think if you have a Java back end, RPC is the way to go for things
like this, not XML. I don't do any XML parsing in my application at
all.

I agree with your idea of fetching the whole data set in one
operation, keeping it on the client, and "lazy loading" it into the
tree as nodes are expanded.

Walden

[Oldtimer]

I am pondering on the one layer at a time approach. If you fetch one
layer at a time, how do you tell the tree widget the items has
children so that the expend (+) sign will show up?

On Dec 9, 12:24 pm, gregor <greg.power...@btinternet.com> wrote:
> Hi Dandante,
>
> Another approach is to populate trees, especially big ones, one layer
> at a time on request as the user opens branches and fetching the top
> layer only at start up. This is a fairly common technique and has the
> advantages that a) the network and serialization overheads on each
> request are small therefore efficient, and b) you can use a simple
> ArrayList to transport each layer's worth of items over RPC which is
> much simpler and avoids the problems you've encountered. The List.java
> class in the KitchenSink sample provides a template for this approach,
> minus the RPC element.
>
> regards
> Gregor
>
> On Dec 8, 1:51 am, dandante <danda...@gmail.com> wrote:
>
>
>
> > Sachin wrote:

> > > So I have some experience with building atreefrom an RPC call.

>
> > [...]
>
> > This is very helpful. I may try a similar approach. Being both lazy
> > and always on the alert for other people who have solved the problem
> > better than I could, I am looking into MyGwt. They have a nice

> > serializable Model class to hold aTree'sdata; however, when I send

> > it over RPC, the client does not receive any of the model's children,

> > so I end up with atreewith no nodes in it.

>
> > If I can't get that to work I'll look at gwt-ext or at some of the
> > homegrown solutions you discuss.
>

> > Thanks!- Hide quoted text -

[Chad Bourque]

Just add a "stub node" to it. If the TreeItem you are adding has children, then add a TreeItem to it. You can set the text to "Loading..." or something like that. Then, when you expand the parent TreeItem, check the first child to see if it is your stub node. If it is, make the RPC to fetch the children. When the data returns, remove the stub node and add the children. You can search the forum for key words like chad tree lazy and you should see some posts I put out a while back regarding this. I think I posted some real code as well.

HTH,
Chad

[gregor]

On Dec 13, 4:33 pm, Oldtimer <colinzhao...@yahoo.com> wrote:
> I am pondering on the one layer at a time approach. If you fetch one
> layer at a time, how do you tell the tree widget the items has
> children so that the expend (+) sign will show up?
>

see List.java in the KitchenSink sample project - it shows you exactly
how to do it.