R package problem on Eddie

[OBBARD Darren]

Hi all,

I am trying to run a R script that uses MCMCglmm() on eddie. When I attempt to use the function inverseA(), the script dies with this error:

Error in validObject(.Object) :
invalid class "dsCMatrix"
Calls: inverseA ... Matrix -> new -> initialize -> initialize -> validObject
Execution halted

When I run an interactive session, and copy&paste my script line by line, I do *not* get this error, so I felt it must be a difference in the packages between what I get running "R" and "Rscript" ?

The sessionInfo() for the interactive attempt says this

> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Scientific Linux 7.2 (Nitrogen)

locale:
[1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_GB.UTF-8 LC_COLLATE=en_GB.UTF-8
[5] LC_MONETARY=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8
[7] LC_PAPER=en_GB.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] MASS_7.3-50 MCMCglmm_2.26 coda_0.19-1 Matrix_1.2-14 ape_5.1

loaded via a namespace (and not attached):
[1] cubature_1.4 corpcor_1.6.9 parallel_3.3.2 Rcpp_0.12.10
[5] nlme_3.1-131 grid_3.3.2 tensorA_0.36 lattice_0.20-35


The sessionInfo() for the gsub submitted Rscript says this:

R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Scientific Linux 7.2 (Nitrogen)

locale:
[1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_GB.UTF-8 LC_COLLATE=en_GB.UTF-8
[5] LC_MONETARY=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8
[7] LC_PAPER=en_GB.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets base

other attached packages:
[1] MASS_7.3-50 MCMCglmm_2.26 ape_5.1 coda_0.19-1 Matrix_1.2-14

loaded via a namespace (and not attached):
[1] cubature_1.4 corpcor_1.6.9 parallel_3.3.2 Rcpp_0.12.10
[5] nlme_3.1-131 grid_3.3.2 tensorA_0.36 methods_3.3.2
[9] lattice_0.20-35

The only obvious difference is the presence of the namespace "methods_3.3.2" which apparently does "Formal Methods and Classes
Formally defined methods and classes for R objects, plus other programming tools, as described in the reference." And indeed, " validObject()" is provided by "methods".

However, I cannot unloadNamespace(), because:
'namespace "methods" is imported by "MCMCglmm", "ape", "Rcpp", "Matrix" so cannot be unloaded'

So, any thoughts as to what causes this error, and why it is different between Rscript and R on Eddie? And indeed why "methods" is loaded/imported by Rscript and not by R (version 3.2.2 on Eddie - I do not get this error in version 3.4.3 on my own Scientific Linux machine)?

Thanks,

Darren

#############################################################
# in Case it matters, this is the wrapper for the R-script

#!/bin/sh
# slim_run.sh file
# Grid Engine options (lines prefixed with #$)
#
#$ -N DJO_PowerSim
#$ -cwd
#$ -l h_rt=02:00:00
#$ -l h_vmem=1G
#$

#load the R module
. /etc/profile.d/modules.sh
module load R/3.3.2

#run the R script
Rscript ~/Power/Power2018.Rscript $1 $2 $3 $4 $5 $6 $7 $7 $9

#module unload R/3.3.2

#############################################################
# and it is invoked with

qsub -t 1 -tc 50 ~/Power/RunSim.sh $families $offspring $mutation_rate $recombination_rate $variance $correlation $PhyloVar $SpeciesVar $FamilyVar



--
Darren Obbard
darren...@ed.ac.uk

Institute of Evolutionary Biology
University of Edinburgh
Ashworth Laboratories, Charlotte Auerbach Road
Edinburgh EH9 3FL
Office 0131 651 7781
Mobile: 07968 838 635
http://obbard.bio.ed.ac.uk/
----------------------------------------------------------------------------------------------------------------------------


The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

[BARKER Daniel]

Hi Darren,

If R works, could you bypass the problem by using R in the Eddie job?

R --no-save < script

So far as I know, R can't take command-line arguments to the script in the same way as RScript, so you may have to find another way to convey these parameters (e.g. via a file).

In another language I've had problems generating files and forgetting to close them (to flush the buffer) before they're used. I don't know if this affects bash/sh, but you could bypass the potential for such problems by making all such files in advance via a separate script.

- Daniel

--
The wiki is at:
https://www.wiki.ed.ac.uk/display/AshCodes/Ashworth+Codemonkeys
The mailing list archive is at:
https://groups.google.com/forum/?fromgroups#!forum/ashworth-code-monkeys
If you have trouble editing the wiki or emailing the group, let me know: sujai...@ed.ac.uk
---
You received this message because you are subscribed to the Google Groups "Ashworth Codemonkeys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ashworth-code-mo...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Booth]

Hi,

Darren asked:

> why "methods" is loaded/imported by Rscript and not by R

I think it's actually the other way around:

[tbooth2@node1h19 ~]$ Rscript -e "sessionInfo()"
...

attached base packages:
[1] stats  graphics  grDevices  utils  datasets  base     

[tbooth2@node1h19 ~]$ R --no-save <<<"sessionInfo()"
...

attached base packages:
[1] stats  graphics  grDevices  utils  datasets  methods  base     

So my guess is that when using RScript, the methods library is not
loaded at startup and so gets loaded on demand later, when one of the
modules in your script requires it as a dependency. However, by that
stage in the process, something (?!) has happened that causes the
module to be loaded differently (ie. there's a broken copy somewhere in
the PATH by this point).

My suggestion is to explicitly run Rscript with the option:

--default-packages=stats,graphics,grDevices,utils,datasets,methods,base

Does that fix it?

Cheers,

TIM

> 1   Matrix_1.2-14

--
====================================================
Edinburgh Genomics' Privacy Notice can be viewed at:
http://genomics.ed.ac.uk/about-us/privacy-notice

This email and any attachments are confidential and
intended solely for the use of the recipient(s) to
whom they are addressed. If you have received it in
error, please destroy all copies and inform the
sender.
====================================================

[OBBARD Darren]

Hi!

Not sure what the precise cause was (or why interactive and Rscript gave different outcomes) but it seems to have been a version clash between R 3.3.2 "methods" and the "Matrix" package.

I worked around by using the IGMM-provided R3.5.0 model - which seems to work fine!

Thanks for the suggestions,

D.

--
Darren Obbard
darren...@ed.ac.uk

Institute of Evolutionary Biology
University of Edinburgh
Ashworth Laboratories, Charlotte Auerbach Road
Edinburgh EH9 3FL
Office 0131 651 7781
Mobile: 07968 838 635
http://obbard.bio.ed.ac.uk/
----------------------------------------------------------------------------------------------------------------------------