ggplot2 execution instability

[Hilmar Berger]

Hi all,

I'am running R mainly using Rstudio + R Markdown on a desktop box (Dell precision T1600) and several servers (Dell 24core/256G RAM), all on Ubuntu 14.04 and R 3.1 or 3.2. In recent weeks I have started using ggplot2 more extensively and experienced instabilities and crashes quite often.  Some of the problems are reproducible even outside Rstudio, so it doesn't seem a RStudio specific problem. R code without ggplot2 also shows no problems, so it seems to be related to ggplo at least.

Issues are mainly:
- sporadic crashes of R when executing ggplot2 functions
- sporadic failures of ggplot2 functions, but throwing reproducible error messages when failing

Sporadic here means that in subsequent executions using the same code and data execution fails only in some cases but finishes without problems in others. Error messages usually indicate some internal problems that do not seem to be directly related to function parameters.

Plots are mainly dot plots and bar charts with facets.

Is that what others observe, too ?

Any suggestions on how to derive more useful debugging information for those cases ?

Best,
Hilmar

------------

R version 3.2.0 (2015-04-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.2 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_GB.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_GB.UTF-8  
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_GB.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C           
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C      

attached base packages:
[1] compiler  stats     graphics  grDevices utils     datasets  methods   base    

other attached packages:
[1] ggplot2_1.0.1

loaded via a namespace (and not attached):
 [1] Rcpp_0.11.5      digest_0.6.8     MASS_7.3-39      grid_3.2.0       plyr_1.8.2       gtable_0.1.2     formatR_1.2      scales_0.2.4   
 [9] reshape2_1.4.1   rmarkdown_0.5.1  proto_0.3-10     labeling_0.3     tools_3.2.0      stringr_0.6.2    munsell_0.4.2    yaml_2.1.13    
[17] colorspace_1.2-6 htmltools_0.2.6  knitr_1.10 

[Brandon Hurr]

Hilmar, 

You should post some of your errors and examples of what you're code is for data manipulation and plotting for us to understand what you're trying to do. My thoughts are that your underlying data are changing between plots and these differences are causing the errors, but that's just speculation without errors and code. 

HTH, 

B

--
--
You received this message because you are subscribed to the ggplot2 mailing list.
Please provide a reproducible example: https://github.com/hadley/devtools/wiki/Reproducibility
 
To post: email ggp...@googlegroups.com
To unsubscribe: email ggplot2+u...@googlegroups.com
More options: http://groups.google.com/group/ggplot2

---
You received this message because you are subscribed to the Google Groups "ggplot2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ggplot2+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Birdgeneau]

Hi Hilmar, 

I believe I'm seeing similar instability with R, Shiny & ggplot2. It results in errors like:

Error in eval(expr, envir, enclos) : 
     arguments imply differing number of rows: 136, 129

Error in eval(expr, envir, enclos) : 
     Results must be all atomic, or all data frames

And the occasional segmentation fault:

Error in eval(substitute(expr), envir, enclos) : 
  Results must be all atomic, or all data frames

 *** caught segfault ***
address 0x2, cause 'memory not mapped'

I've posted some of my observations in a Stack Overflow question here; hoping that someone else had already seen this & resolved it.

http://stackoverflow.com/questions/30520575/intermittent-errors-with-ggplot2-segfault-memory-not-mapped

I found another post (referenced in the SO question) that had an example which reproduces the error (with a for loop, showing the 'sporadic' nature of the errors).

I found on OSX that reverting to R 3.1.2 and Rcpp 0.11.5 seemed to help resolve my issue - but that didn't seem to resolve the problem on my RedHat linux boxes that I'm trying to get the Shiny app running on.

Hope this helps - I've been figuratively bashing my head against the wall for almost a week on this one... so if you happen to have solved your issue in the last few weeks I'd be happy to hear what you've found.

Cheers,

Mike

[Hadley Wickham]

I'm pretty sure this is caused by a bug in plyr - I see it
occasionally too, but haven't been able to reliably replicate it so I
can fix it :(

Hadley

> --
> --
> You received this message because you are subscribed to the ggplot2 mailing
> list.
> Please provide a reproducible example:
> https://github.com/hadley/devtools/wiki/Reproducibility
>
> To post: email ggp...@googlegroups.com
> To unsubscribe: email ggplot2+u...@googlegroups.com
> More options: http://groups.google.com/group/ggplot2
>
> ---
> You received this message because you are subscribed to the Google Groups
> "ggplot2" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ggplot2+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
http://had.co.nz/

[Hadley Wickham]

If you happen to have a plot that reliably (e.g. at least 10% of the
time) causes a seg fault on OS X it would be super useful if you could
send it to me.

Hadley

--
http://had.co.nz/

[Mike Birdgeneau]

Hi Hadley,

I've been able to replicate the bug with the following code:

https://github.com/mikebirdgeneau/ShinyTest/blob/master/standalone_check.R

The first for loop (20 plots) should run without errors; the latter 30 plots should show occasional errors using the exact same input. The errors do appear to be related to plyr as you mentioned based on the different variety of errors that are seen. 

However, it only seems to occur with some package / R versions. I was able to replicate the bug with R 3.2 and R 3.1.2 using the latest CRAN versions of ggplot2 (1.0.1), plyr (1.8.2), as well as Rcpp (0.11.6).  Downgrading to Plyr 1.8.1 seems to help (I have it working again with plyr 1.8.1, ggplot2 0.9.3, and Rcpp 0.11.5 for now just to keep my project moving along).  I've have not confirmedto this point if there are other contributing factors (e.g. Operating System, etc.).

Hope this helps you track it down.

Cheers, 

Mike

[Mike Birdgeneau]

I forgot to mention that you might have to 'source' the file instead of selecting and running lines in Rstudio to reproduce the error. Let me know if there's anything else you need that could help you reproduce and trace this error. I've kept updating the Stack Overflow question I created with some extra details, sessionInfo() as well as some examples of the errors. 

I'm not using the bar plot code directly in my Shiny app - but it seems to correlate with errors when using geom_line() as well - so I've been using it as a proxy to debug my problems.

Mike

[Brandon Hurr]

I can confirm this behavior with your script. Lots of random errors (unequal lengths and atomic vector). Every once in a while a segfault and sometimes a complete lockup of R and spinning beachball of death. 

Running slightly more beta stuff, but same result. If I load dplyr after ggplot2 it seems to segfault less, but still lock up every now and then. Is there something we can do better or add code to the script to track down the issue better Hadley?

R version 3.2.0 Patched (2015-05-29 r68444)

Platform: x86_64-apple-darwin13.4.0 (64-bit)

Running under: OS X 10.10.3 (Yosemite)

locale:

[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:

[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:

[1] ggplot2_1.0.1

loaded via a namespace (and not attached):

 [1] MASS_7.3-40      colorspace_1.2-6 scales_0.2.4     magrittr_1.5     plyr_1.8.2       tools_3.2.0      gtable_0.1.2     reshape2_1.4.1  

 [9] Rcpp_0.11.6      stringi_0.4-1    grid_3.2.0       stringr_1.0.0    digest_0.6.8     proto_0.3-10     munsell_0.4.2   

[Hadley Wickham]

Fixed in the dev version: devtools::install_github("hadley/plyr")

I think it's actually a bug in Rcpp, so I've reverted to the previous
pure C implementation of loop_apply.

Hadley

--
http://had.co.nz/

[Johannes Björk]

Hi,

When running devtools::install_github("hadley/plyr") it fails saying:

ERROR: loading failed

* removing ‘/Library/Frameworks/R.framework/Versions/3.1/Resources/library/plyr’

* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.1/Resources/library/plyr’

Error: Command failed (1)

Any tips?

Thanks

[Hadley Wickham]

You can just install plyr from CRAN.
Hadley