[R] Error: cannot allocate vector of size 1.8 Gb

iamsilvermember

unread,

Dec 22, 2008, 2:52:35 PM12/22/08

to r-h...@r-project.org

> dim(data)
[1] 22283 19

> dm=dist(data, method = "euclidean", diag = FALSE, upper = FALSE, p = 2)
Error: cannot allocate vector of size 1.8 Gb

Hi Guys, thank you in advance for helping. :-D

Recently I ran into the "cannot allocate vector of size 1.8GB" error. I am
pretty sure this is not a hardware limitation because it happens no matter I
ran the R code in a 2.0Ghz Core Duo 2GB ram Mac or on a Intel Xeon 2x2.0Ghz
quard-core 8GB ram Linux server.

I also tried to clear the workspace before running the code too, but it
didn't seem to help...

Weird thing though is that once in a while it will work, but next when I run
clustering on the above result
>hc=hclust(dm, method = "complete", members=NULL)
it give me the same error...

I searched around already, but the memory.limit, memory.size method does not
seem to help. May I know what can i do to resolve this problem?

Thank you so much for your help.
--
View this message in context: http://www.nabble.com/Error%3A-cannot-allocate-vector-of-size-1.8-Gb-tp21133949p21133949.html
Sent from the R help mailing list archive at Nabble.com.

______________________________________________
R-h...@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

iamsilvermember

unread,

Dec 22, 2008, 6:26:52 PM12/22/08

to r-h...@r-project.org

iamsilvermember

unread,

Dec 22, 2008, 6:57:20 PM12/22/08

to r-h...@r-project.org

I hope the following info will help, thanks again!

> sessionInfo()
R version 2.7.1 (2008-06-23)
x86_64-redhat-linux-gnu

locale:
LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=en_US.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=C;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

--
View this message in context: http://www.nabble.com/Error%3A-cannot-allocate-vector-of-size-1.8-Gb-tp21133949p21137233.html

Prof Brian Ripley

unread,

Dec 23, 2008, 3:15:19 AM12/23/08

to iamsilvermember, r-h...@r-project.org

On Mon, 22 Dec 2008, iamsilvermember wrote:

>
>> dim(data)
> [1] 22283 19
>
>> dm=dist(data, method = "euclidean", diag = FALSE, upper = FALSE, p = 2)
> Error: cannot allocate vector of size 1.8 Gb

That would be an object of size 1.8Gb.

See ?"Memory-limits"

>
>
>
> Hi Guys, thank you in advance for helping. :-D
>
> Recently I ran into the "cannot allocate vector of size 1.8GB" error. I am
> pretty sure this is not a hardware limitation because it happens no matter I
> ran the R code in a 2.0Ghz Core Duo 2GB ram Mac or on a Intel Xeon 2x2.0Ghz
> quard-core 8GB ram Linux server.

Why? Both will have a 3GB address space limits unless the Xeon box is
64-bit. And this works on my 64-bit Linux boxes.

> I also tried to clear the workspace before running the code too, but it
> didn't seem to help...
>
> Weird thing though is that once in a while it will work, but next when I run
> clustering on the above result
>> hc=hclust(dm, method = "complete", members=NULL)
> it give me the same error...

See ?"Memory-limits" for the first part.

> I searched around already, but the memory.limit, memory.size method does not
> seem to help. May I know what can i do to resolve this problem?

What are you going to do with an agglomerative hierarchical clustering of
22283 objects? It will not be interpretible.

> Thank you so much for your help.

--
Brian D. Ripley, rip...@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595

iamsilvermember

unread,

Dec 23, 2008, 2:10:34 PM12/23/08

to r-h...@r-project.org

>What are you going to do with an agglomerative hierarchical clustering of
22283 objects? It will not be interpretible.

As a matter of fact I was ask to do a clustering analysis on gene
expression. Something
http://www.ncbi.nlm.nih.gov/projects/geo/gds/analyze/analyze.cgi?datadir=UCorrelationUPGMA&ID=GDS3254&myType=0
Like this

>Why? Both will have a 3GB address space limits unless the Xeon box is
64-bit. And this works on my 64-bit Linux boxes.

I am pretty sure the linux server is 64bit.

Sorry I am just a beginner in R. I read the "memory-limit" help you
suggested, but I still cannot find a solution to my problem... May I know
if there is any work around for this issue?

Thank you so much again!
--
View this message in context: http://www.nabble.com/Error%3A-cannot-allocate-vector-of-size-1.8-Gb-tp21133949p21149727.html