Can't get the layouts to render

[abejohnny]

Hi,

I'm trying to visualize a very large web graph - 30K nodes and 700K
edges. I'm working on a linux box with 15G of RAM and I've increased
the heap size to match. Now when I load the network, I need to
explicitly create the default view which does get rendered. However,
the layout I need is the force-directed one. Unfortunately when I
click on it, I get a message saying that it's performing the layout,
but nothing changes (this message also goes away after a couple of
minutes when I'd reckon that it would take a couple of hours to
actually generate). I haven't been able to get any of the other
layouts to render either.

Also I was trying to generate a discrete series for the node size
based on its degree, however the generate menu doesn't allow me to
select anything. None of the options are clickable.

I'd be really grateful for any help.


Thanks

Aby

[Scooter Morris]

The force directed layout should certainly be able to work with this,
particularly given the amount of memory you have. Can you send me a
copy of the network? I'll try to determine what's happening (although I
don't have 15GB!).

-- scooter

[Mike Smoot]

We should have a computer with a bunch of RAM that we could test it on later this week if necessary.

I'm not sure why force directed would be failing after only a couple of minutes.  Are you seeing any exceptions in the output?

I'm not sure about why the generate options aren't available, but I have recently fixed a bug related to loading large networks at startup that don't automatically create a view.  The problems might be related. The fix will be in the forthcoming 2.6 beta2.  If you don't want to wait for that, you might first load a small (2 nodes, one edge) dummy network and then load your big one.  The bug I fixed is related to getting Cytoscape's initial state set correctly, so this might do it for you.


Mike

--
____________________________________________________________
Michael Smoot, Ph.D. Bioengineering Department
tel: 858-822-4756 University of California San Diego

[Keiichiro Ono]

Hi.
I always get the following exception when I use force-directed for
large network:

java.lang.StackOverflowError
at prefuse.util.force.NBodyForce.insertHelper(NBodyForce.java:224)
at prefuse.util.force.NBodyForce.insert(NBodyForce.java:197)
at prefuse.util.force.NBodyForce.insertHelper(NBodyForce.java:231)
at prefuse.util.force.NBodyForce.insert(NBodyForce.java:193)
at prefuse.util.force.NBodyForce.insertHelper(NBodyForce.java:231)
at prefuse.util.force.NBodyForce.insert(NBodyForce.java:193)
at prefuse.util.force.NBodyForce.insertHelper(NBodyForce.java:231)
at prefuse.util.force.NBodyForce.insert(NBodyForce.java:193)
at prefuse.util.force.NBodyForce.insertHelper(NBodyForce.java:231)
at prefuse.util.force.NBodyForce.insert(NBodyForce.java:193)
at prefuse.util.force.NBodyForce.insertHelper(NBodyForce.java:231)
at prefuse.util.force.NBodyForce.insert(NBodyForce.java:193)
(this output repeats thousands of times)

So, probably this is a bug in prefuse?

Thanks.
Kei

2008/3/11, Mike Smoot <msm...@ucsd.edu>:

--
Keiichiro Ono ko...@ucsd.edu

Cytoscape Core Developer Team: http://www.cytoscape.org/
UCSD Bioengineering Ideker Lab: http://chianti.ucsd.edu/idekerlab/

[abejohnny]

Yea, I get the same exceptions when doing a force directed layout. But
I also get these errors when doing a degree centered layout:

Updating attr: Event = NETWORK_LOADED, Source =
java.lang.Object@5374d4c9
Exception in thread "Thread-5" java.lang.StackOverflowError
at csplugins.layout.LayoutNode.<init>(LayoutNode.java:84)
at
csplugins.layout.LayoutPartition.addNode(LayoutPartition.java:165)
at
csplugins.layout.LayoutPartition.traverse(LayoutPartition.java:753)
at
csplugins.layout.LayoutPartition.traverse(LayoutPartition.java:797)
again repeated thousands of times.

Loading a smaller network at the start didn't help the situation
unfortunately.

I have three datasets that I'm working with. One has around 4.5M
edges, the second is a reduced version of the first graph with 1M
edges, and the third is a separate dataset with 600K edges. Now I've
been able to get the third dataset to render but the first two sets
don't work. When doing a force directed layout with the third set I
get a message saying it's starting partition and it goes on to
complete the layout. However, with the other two the message just says
starting and after a couple of minutes it stops.

Is a million edges beyond the boundary of what I can do with 15G
machine? (I'm actually running an XL Amazon EC2 instance)

Thanks

Aby

> > On Tue, Mar 11, 2008 at 10:23 AM, Scooter Morris <scoo...@cgl.ucsf.edu>

> Keiichiro Ono    k...@ucsd.edu

>
> Cytoscape Core Developer Team:http://www.cytoscape.org/

> UCSD Bioengineering Ideker Lab:http://chianti.ucsd.edu/idekerlab/- Hide quoted text -
>
> - Show quoted text -

[Mike Smoot]

Hi Aby,

Given your results it certainly sounds like one million edges is something of an upper limit with current machines. The general problem is that layout algorithms tend to consume a lot of memory relative to the number of nodes and edges in the network, generally something like O(n^2).

Anyway, this link is to some software that I know has been used to layout large networks: 

http://www.cs.sandia.gov/%7Esmartin/software.html 

Once you layout the graph, then you should be able to get it into Cytoscape relatively easily.

Let us know how it goes!

Mike

[Keiichiro Ono]

Hi.
I tested to layout a large network with 1 million graph objects (half
million nodes and half million edges, generated by Barabasi-Albert
model) on 8 CPU machine with 17GB of RAM. Looks like only Cytoscape's
Circular layout is working for this size of network.

(By the way, memory consumption was about 30% for the network.)

Thanks.
Kei

2008/3/11, abejohnny <abej...@gmail.com>:

--
Keiichiro Ono ko...@ucsd.edu