'Calloc' could not allocate memory

[Sean Rife]

I'm getting the following error when I try to execute ggplot with aes and loess smoothing. The command is:

ggplot(dataset, aes(x=x, y=y)) + geom_smooth(alpha=0.25, color="black", fill="black", method="loess", span=1)

It works fine with lm, gam, etc. It works fine with small(er) data frames. But when I hit 40k rows, I start to get the following message: 

Error in predLoess(y, x, newx, s, weights, pars$robust, pars$span, pars$degree,  : 

  'Calloc' could not allocate memory (18446744071914983424 of 4 bytes)

I would be happy to write this off as R being unable to allocate the needed amount of memory, but it's saying that it can't allocate around 18 exabytes of memory. I can't imagine this is necessary. Can anyone shed any light on this? I know it's been mentioned elsewhere, but those issues seem to be in reference to much more reasonable allocations (18 exabytes just sounds like I'm making a mistake or there's a bug somewhere).

Many thanks in advance!

[Hadley Wickham]

This is why geom_smooth switches to an alternative method when n is > 10000. Is there a reason method = "auto" doesn't work for you?

Hadley

--
--
You received this message because you are subscribed to the ggplot2 mailing list.
Please provide a reproducible example: https://github.com/hadley/devtools/wiki/Reproducibility
 
To post: email ggp...@googlegroups.com
To unsubscribe: email ggplot2+u...@googlegroups.com
More options: http://groups.google.com/group/ggplot2

---
You received this message because you are subscribed to the Google Groups "ggplot2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ggplot2+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
http://had.co.nz/

[Dennis Murphy]

Hi:

To expand on Hadley's comments, I would suggest that you consult
?loess and read the chapter in the White book that it cites to gain a
better understanding of how the loess procedure operates. It makes no
sense to me why one would want to use a span of 1 to fit a loess model
with 40K observations.

By its nature, loess is a "local regression" algorithm, with emphasis
on the "local". The span argument controls the proportion of the data
that should be used to produce each local fit - the wider the span,
the smoother the fitted function. If you look carefully at the
algorithm itself, you'll discover that it's VERY memory intensive, so
if you insist on a loess fit with a large sample size, at least reduce
the span. Here is an example to illustrate that you can indeed fit a
loess model in ggplot2 with 40K observations.

x <- seq(0, 100, length.out = 40000)
# A periodic function
DF <- data.frame(x = x, y = 1 + sin(x) + 0.5 * cos(2 * x) + rnorm(40000))

library(ggplot2)

# Uses the default "auto" method to which Hadley referred:
ggplot(DF, aes(x = x, y = y)) +
geom_point(alpha = 0.05, shape = 21) +
geom_smooth(size = 1)

The result of this [gam] fit, which finishes rather quickly, is more
or less equivalent to a loess model with a large span (such as 1), but
far more computationally efficient. The periodicity is almost
completely ignored in the fitted curve as it has been averaged away.
To capture the periodicity with a local regression algorithm, you need
to reduce the proportion of the data devoted to each local fit. The
following call takes about 1.5-2 minutes (guesstimated) to run, but it
does produce a loess fit in the end on my laptop (with 12Gb RAM +
R-3.2.0 64-bit + i7 chip):

ggplot(DF, aes(x = x, y = y)) +
geom_point(alpha = 0.05, shape = 21) +
geom_smooth(method = "loess", span = 0.1, size = 1)

When I ran this in the R GUI, I got the "Not responding" message while
R was cranking away mightily, but eventually a graph did appear.

You should be able to get a more accurate local fit by reducing the
span further, since span = 0.1 in this example means that it's using
approximately 4000 points per local fit, which is far more than it
needs for a curve this simple in form. The following call took about
8-10 seconds, with one difference in the specification:

ggplot(DF, aes(x = x, y = y)) +
geom_point(alpha = 0.05, shape = 21) +
geom_smooth(method = "loess", span = 0.005, size = 1)

In this call, span = 0.005 means that approximately 200 observations
are used in each local fit, which is still fairly large. I would
recommend experimenting with slightly smaller and larger spans to see
how it affects the fitted loess model. The choice of span should be
informed by the number of rows in the input data frame, the shape of
the noisy input function and the degree of smoothness desired.

The example was deliberately chosen to illustrate why the choice of
span matters in loess. On the other hand, the error message you
received indirectly signals that loess is a memory hog and you need to
know enough about how it works as an algorithm in order to use it
productively.


Dennis

[samuel ayele Abebe]

I am try to plot multiple plot and i use this command

 install.packages("gridExtra")

library(gridExtra)

grid.arrange (p1, p2, p3, p4,p5,p6,cols=3) and the error message is

Error: could not find function "ggplotGrob"

again i tray to install this package

 install.packages("ggplotGrob")

Error in library(ggplotGrob) : there is no package called ‘ggplotGrob’

but i could't find this package on R version 3.0.3

 

Is there any way that i can plot multiple plot using ggplot on R version 3.03

kind regards

-- 

[Dennis Murphy]

(1) Please do not hijack an existing thread to ask an unrelated
question - start a new one instead.
(2) ggplotGrob() is a function in the ggplot2 package, so you need to
load ggplot2 before you can invoke it.:

library(ggplot2)
library(gridExtra)
<your plot code>

Dennis