big network plotting fails
Mark Grimes
Hi Paul
I am trying to graph a very large network that is 25,000 nodes and 6 million edges. I had memory problems trying to plot this with RCytoscape all at once. I'm using a Unix box with 64G of memory and adjusted Cytoscape's memory as shown below. I managed to get it to partially work by breaking up the edge file into chunks of 1000, and add them using addGraphToGraph using a for: loop. Nodes are NOT duplicated when I do this, which is good. But after a number of iterations (somewhere between 25 and 200 - it always happens when I go away for a while), I get an error in R as shown below, and a unix error :
~$ Write failed: Broken pipe
This has happened twice now. R session pasted below.
Any ideas?
Thanks,
Mark
[1] sending noa Saline.Cortex.change
[1] sending noa Meth.Hipp.change
[1] sending noa Meth.Cortex.change
[1] sending noa POB.Hipp.change
[1] sending noa POB.Cortex.change
[1] sending noa Meth.Hipp.norm.change
[1] sending noa Meth.Cortex.norm.change
[1] sending noa POB.Hipp.norm.change
[1] sending noa POB.Cortex.norm.change
[1] sending noa Saline.Hipp.change
Error in function (type, msg, asError = TRUE) :
Recv failure: Connection reset by peer
sessionInfo()
R version 3.1.0 (2014-04-10)
Platform: x86_64-unknown-linux-gnu (64-bit)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel splines grid stats graphics grDevices utils
[8] datasets methods base
other attached packages:
[1] Rtsne_0.9 devtools_1.5 plyr_1.8.1
[4] Biostrings_2.32.1 XVector_0.4.0 IRanges_1.22.9
[7] GOstats_2.30.0 Category_2.30.0 Matrix_1.1-4
[10] GO.db_2.14.0 org.Hs.eg.db_2.14.0 RSQLite_0.11.4
[13] DBI_0.2-7 AnnotationDbi_1.26.0 GenomeInfoDb_1.0.2
[16] Biobase_2.24.0 BiocGenerics_0.10.0 RCytoscape_1.14.0
[19] XMLRPC_0.3-0 graph_1.42.0 gplots_2.14.1
[22] tsne_0.1-2 vegan_2.0-10 permute_0.8-3
[25] RUnit_0.4.26 adegenet_1.4-2 ade4_1.6-2
[28] cluster_1.15.2 Hmisc_3.14-4 Formula_1.1-1
[31] survival_2.37-7 lattice_0.20-29 stringr_0.6.2
[34] fdrtool_1.2.12
loaded via a namespace (and not attached):
[1] annotate_1.42.0 AnnotationForge_1.6.1 ape_3.1-2
[4] bitops_1.0-6 caTools_1.17 colorspace_1.2-4
[7] digest_0.6.4 evaluate_0.5.5 gdata_2.13.3
[10] genefilter_1.46.1 ggplot2_1.0.0 GSEABase_1.26.0
[13] gtable_0.1.2 gtools_3.4.1 htmltools_0.2.4
[16] httpuv_1.3.0 httr_0.4 igraph_0.7.1
[19] KernSmooth_2.23-12 latticeExtra_0.6-26 MASS_7.3-33
[22] memoise_0.2.1 munsell_0.4.2 nlme_3.1-117
[25] proto_0.3-10 RBGL_1.40.0 RColorBrewer_1.0-5
[28] Rcpp_0.11.2 RCurl_1.95-4.1 reshape2_1.4
[31] RJSONIO_1.2-0.2 scales_0.2.4 shiny_0.10.0
[34] stats4_3.1.0 tools_3.1.0 whisker_0.3-2
[37] XML_3.98-1.1 xtable_1.7-3 zlibbioc_1.10.0
Unix box:
# memory: 64GB cpu: 16-core AMD Opteron 6140
# kernel: GNU/Linux version 3.2.0.-4
# os: Debian 7.5
Cytoscape vmoptions:
-Xms12G
-Xmx12G
-Xss64M
-Dswing.aatext=true
-Dawt.useSystemAAFontSettings=lcd
Mark Grimes
Division of Biological Sciences
University of Montana
Missoula, Montana 59812-4824
DBS website
Office Telephone: (406) 243-4977
Fax Number: (406) 243-4184
E-mail: Mark....@mso.umt.edu
Paul Shannon
> Recv failure: Connection reset by peer
How much time had passed before that happened? What time of day or night? In the midst of idle time, or when the machine was working?
It could be that such a large job needs to be rethought, perhaps by using RCy to write out a sif file, which then gets loaded directly into Cytoscape.
Your thoughts?
- Paul
Mark Grimes
Paul
The crash occurred twice after it had been going for 25 rounds (1600 nodes, 25,000 edges) and I left for the evening. In both cases I was logged on to a Unix server remotely via ssh. I can only guess how far it went because the X window Cytoscape session disappeared before I could examine the network.
I went over to the Unix box itself and started the same loop on it. We'll see what it does over the weekend...
Mark
Mark Grimes
Division of Biological Sciences
University of Montana
Missoula, Montana 59812-4824
DBS website
Office Telephone: (406) 243-4977
Fax Number: (406) 243-4184
E-mail: Mark....@mso.umt.edu
- Paul
--
You received this message because you are subscribed to the Google Groups "RCytoscape" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rcytoscape+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.