Sage's R (6.0) mucks up path and libraries search on amd64 (at least Debian).

[Emmanuel Charpentier]

Sage's 6.0 implementation seems to muck up the way R finds libraries and/or executables on amd64, at least on Debian systems.

My main system is Debian (testing) running on a Core i7 laptop with plenty of RAM. I've also set up a virtual machine (VirtualBox) with i686 Debian in order to cross-compile for a small subnotebook (where a "real" compilation takes in excess of 24 hours).

I've had problems with the latest Sage (6.0) release. A first one, related to system fonts unsupported by Sage's freetype, has been reported elsewhere, ticketed, and a port of freetype 2.5.2 proposed (hint : trac#15561 needs review). I've encountered this problem on both amd64 and i686 machines, but not on a smaller amd64 machine, with a tad older system software.

I encountered a second problem on amd64 machines only, hence the present post. In short, some R packages using external libraries cannot be installed anymore. Furthermore, tcl/tk is no longer available to Sage's R. This on machines where system's R (Debian packages in all cases) has no problem whatsoever.

First case in point : tcl/tk. From R's compilation log : 

[ ... ]
checking for tclConfig.sh... no

checking for tclConfig.sh in library (sub)directories... no

checking for tkConfig.sh... no

checking for tkConfig.sh in library (sub)directories... no

checking for tcl.h... no

[ ... ]

However :

charpent@asus16-ec:~$ locate tclConfig.sh

/home/charpent/Dev/tcltk/tcl8.6.1/pkgs/itcl4.0.0/itclConfig.sh.in

/home/charpent/Dev/tcltk/tcl8.6.1/unix/tclConfig.sh

/home/charpent/Dev/tcltk/tcl8.6.1/unix/tclConfig.sh.in

/home/charpent/Dev/tcltk/tcl8.6.1/win/tclConfig.sh.in

/usr/lib/tclConfig.sh

/usr/lib/tcl8.5/tclConfig.sh

charpent@asus16-ec:~$ locate tkConfig.sh

/home/charpent/Dev/tcltk/tk8.6.1/unix/tkConfig.sh.in

/home/charpent/Dev/tcltk/tk8.6.1/win/tkConfig.sh.in

/usr/lib/tkConfig.sh

/usr/lib/tk8.5/tkConfig.sh

charpent@asus16-ec:~$ locate tcl.h

/home/charpent/Dev/tcltk/tcl8.6.1/generic/tcl.h

/home/charpent/Dev/tcltk/tcl8.6.1/pkgs/itcl4.0.0/generic/itcl.h

/home/charpent/Dev/tcltk/tcl8.6.1/tools/tcl.hpj.in

/home/charpent/Dev/tcltk/tcl8.6.1/win/tcl.hpj.in

/usr/include/tcl8.5/tcl.h

/usr/include/tcl8.5/tcl-private/generic/tcl.h

/usr/share/doc/graphviz/examples/demo/entities.tcl.html

/usr/share/doc/postgresql-doc-9.3/html/pltcl.html

/usr/share/doc/texlive-doc/latex/koma-script/scrartcl.html

As a consequence, tcltk is unavailable in Sage's R, which is not really important per se, but renders tkrplot (not really important), gsubfn (much more important) and sqldf (almost vital) R packages uninstallable.

Second case : rjags

> install.packages("rjags")

essai de l'URL 'http://cran.univ-paris1.fr/src/contrib/rjags_3-11.tar.gz'

Content type 'application/x-gzip' length 65594 bytes (64 Kb)

URL ouverte

==================================================

downloaded 64 Kb

* installing *source* package ‘rjags’ ...

** package ‘rjags’ correctement décompressé et sommes MD5 vérifiées

checking for prefix by checking for jags... /usr/bin/jags

checking whether the C++ compiler works... yes

checking for C++ compiler default output file name... a.out

checking for suffix of executables... 

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C++ compiler... yes

checking whether g++ accepts -g... yes

checking how to run the C++ preprocessor... g++ -E

checking for grep that handles long lines and -e... /bin/grep

checking for egrep... /bin/grep -E

checking for ANSI C header files... yes

checking for sys/types.h... yes

checking for sys/stat.h... yes

checking for stdlib.h... yes

checking for string.h... yes

checking for memory.h... yes

checking for strings.h... yes

checking for inttypes.h... yes

checking for stdint.h... yes

checking for unistd.h... yes

checking Console.h usability... yes

checking Console.h presence... yes

checking for Console.h... yes

checking for gcc... gcc -std=gnu99

checking whether we are using the GNU C compiler... yes

checking whether gcc -std=gnu99 accepts -g... yes

checking for gcc -std=gnu99 option to accept ISO C89... none needed

checking for jags_version in -ljags... yes

configure: error: "JAGS module directory /usr/lib64/JAGS/modules-3 does not exist."

ERROR: configuration failed for package ‘rjags’

* removing ‘/home/charpent/sage/local/lib/R/library/rjags’

Les packages source téléchargés sont dans

‘/tmp/RtmpS0KOyC/downloaded_packages’

Message d'avis :

In install.packages("rjags") :

  l'installation du package ‘rjags’ a eu un statut de sortie non nul

> 

Trying to install R2jags gives similar results for the same reason. This is *quite* serious : JAGS is currently the only "blackbox" MCMC sampler able to sample from discrete distributions.

Comparing with the "system" R installation of the same package hints at a possible cause :

> install.packages("rjags")

Installing package into ‘/usr/local/lib/R/site-library’

(as ‘lib’ is unspecified)

--- SVP sélectionner un miroir CRAN pour cette session ---

essai de l'URL 'http://cran.univ-paris1.fr/src/contrib/rjags_3-11.tar.gz'

Content type 'application/x-gzip' length 65594 bytes (64 Kb)

URL ouverte

==================================================

downloaded 64 Kb

* installing *source* package ‘rjags’ ...

** package ‘rjags’ correctement décompressé et sommes MD5 vérifiées

checking for prefix by checking for jags... /usr/bin/jags

checking whether the C++ compiler works... yes

checking for C++ compiler default output file name... a.out

checking for suffix of executables... 

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C++ compiler... yes

checking whether g++ accepts -g... yes

checking how to run the C++ preprocessor... g++ -E

checking for grep that handles long lines and -e... /bin/grep

checking for egrep... /bin/grep -E

checking for ANSI C header files... yes

checking for sys/types.h... yes

checking for sys/stat.h... yes

checking for stdlib.h... yes

checking for string.h... yes

checking for memory.h... yes

checking for strings.h... yes

checking for inttypes.h... yes

checking for stdint.h... yes

checking for unistd.h... yes

checking Console.h usability... yes

checking Console.h presence... yes

checking for Console.h... yes

checking for gcc... gcc -std=gnu99

checking whether we are using the GNU C compiler... yes

checking whether gcc -std=gnu99 accepts -g... yes

checking for gcc -std=gnu99 option to accept ISO C89... none needed

checking for jags_version in -ljags... yes

configure: creating ./config.status

config.status: creating src/Makevars

configure: creating ./config.status

config.status: creating src/Makevars

config.status: creating R/unix/zzz.R

** libs

g++ -I/usr/share/R/include -DNDEBUG -I/usr/include/JAGS      -fpic  -O3 -pipe  -g  -c jags.cc -o jags.o

g++ -I/usr/share/R/include -DNDEBUG -I/usr/include/JAGS      -fpic  -O3 -pipe  -g  -c parallel.cc -o parallel.o

g++ -shared -o rjags.so jags.o parallel.o -L/usr/lib -ljags -L/usr/lib/R/lib -lR

installing to /usr/local/lib/R/site-library/rjags/libs

** R

** data

** preparing package for lazy loading

** help

*** installing help indices

** building package indices

** testing if installed package can be loaded

* DONE (rjags)

Les packages source téléchargés sont dans

‘/tmp/RtmpBOFn5k/downloaded_packages’

> 

The system libraries are not explored in the same order, and some of them are not explored at all. This explanation *might* also explain why tcl/tk headers/utilities are not found during compilation.

A comparison with Sage's R in a i686 virtual box is illuminating : on this machine, tcl/tk headers and utilities are found and tcltk is in R's "capabilities()". Similarly, on this system :

> install.packages("rjags")

also installing the dependency ‘coda’

essai de l'URL 'http://cran.univ-paris1.fr/src/contrib/coda_0.16-1.tar.gz'

Content type 'application/x-gzip' length 74195 bytes (72 Kb)

URL ouverte

==================================================

downloaded 72 Kb

essai de l'URL 'http://cran.univ-paris1.fr/src/contrib/rjags_3-11.tar.gz'

Content type 'application/x-gzip' length 65594 bytes (64 Kb)

URL ouverte

==================================================

downloaded 64 Kb

* installing *source* package ‘coda’ ...

** package ‘coda’ correctement décompressé et sommes MD5 vérifiées

** R

** data

** inst

** preparing package for lazy loading

** help

*** installing help indices

** building package indices

** testing if installed package can be loaded

* DONE (coda)

* installing *source* package ‘rjags’ ...

** package ‘rjags’ correctement décompressé et sommes MD5 vérifiées

checking for prefix by checking for jags... /usr/bin/jags

checking whether the C++ compiler works... yes

checking for C++ compiler default output file name... a.out

checking for suffix of executables... 

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C++ compiler... yes

checking whether g++ accepts -g... yes

checking how to run the C++ preprocessor... g++ -E

checking for grep that handles long lines and -e... /bin/grep

checking for egrep... /bin/grep -E

checking for ANSI C header files... yes

checking for sys/types.h... yes

checking for sys/stat.h... yes

checking for stdlib.h... yes

checking for string.h... yes

checking for memory.h... yes

checking for strings.h... yes

checking for inttypes.h... yes

checking for stdint.h... yes

checking for unistd.h... yes

checking Console.h usability... yes

checking Console.h presence... yes

checking for Console.h... yes

checking for gcc... gcc -std=gnu99

checking whether we are using the GNU C compiler... yes

checking whether gcc -std=gnu99 accepts -g... yes

checking for gcc -std=gnu99 option to accept ISO C89... none needed

checking for jags_version in -ljags... yes

configure: creating ./config.status

config.status: creating src/Makevars

configure: creating ./config.status

config.status: creating src/Makevars

config.status: creating R/unix/zzz.R

** libs

g++ -I/home/charpent/sage/local/lib/R//include -DNDEBUG -I/usr/include/JAGS      -fpic   -c jags.cc -o jags.o

g++ -I/home/charpent/sage/local/lib/R//include -DNDEBUG -I/usr/include/JAGS      -fpic   -c parallel.cc -o parallel.o

g++ -shared -L/home/charpent/sage/local/lib/ -o rjags.so jags.o parallel.o -L/usr/lib -ljags -L/home/charpent/sage/local/lib/R//lib -lR

installing to /home/charpent/sage/local/lib/R/library/rjags/libs

** R

** data

** preparing package for lazy loading

** help

*** installing help indices

** building package indices

** testing if installed package can be loaded

* DONE (rjags)

Les packages source téléchargés sont dans

‘/tmp/RtmpU7Qdm4/downloaded_packages’

> 

A possible third case is texi2dvi. It pops up when a package tries to use it for its vignettes :

> install_url("http://www.bayesian-inference.com/LaplacesDemon_13.11.17.tar.gz")

Downloading LaplacesDemon_13.11.17.tar.gz from http://www.bayesian-inference.com/LaplacesDemon_13.11.17.tar.gz

Installing package from /tmp/RtmpS0KOyC/LaplacesDemon_13.11.17.tar.gz

Installing LaplacesDemon

'/home/charpent/sage/local/lib/R//bin/R' --vanilla CMD build  \

  '/tmp/RtmpS0KOyC/devtools21a24754647d/LaplacesDemon' --no-manual  \

  --no-resave-data 

* checking for file '/tmp/RtmpS0KOyC/devtools21a24754647d/LaplacesDemon/DESCRIPTION' ... OK

* preparing 'LaplacesDemon':

* checking DESCRIPTION meta-information ... OK

* installing the package to build vignettes

* creating vignettes ... ERROR

Error in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet,  : 

  Running 'texi2dvi' on 'BayesianInference.tex' failed.

Messages:

/home/charpent/sage/local/bin/texi2dvi: 132: /home/charpent/sage/local/bin/texi2dvi: Syntax error: Bad function name

Calls: <Anonymous> -> texi2pdf -> texi2dvi

Execution halted

Erreur : Command failed (1)

> 

Here, the problem is not as obviously a problem of unfound binaries, and I am not yet convinced of its origin.

In short, it seems that Sage on amd64 systems (at least under *debian*) incorrectly sets up the header and utilities search path, with serious consequences for Sage's R usability on this platform.

I intent to open a ticket about this problem, and am eager for advice about how to ticket it *efficiently*.

Now for a possible cause : Trac#14706, which was intended for a simple mindless port of R 3.0 tarball to Sage, has been hijacked in order to solve installation/usage problems (on Cygwin IIRC), and ended up with a lengthy adjusment of paths and LD_LIBRARY_PATH. I did not had time to test these trials, used the "original mindless port of upstream source" and encountered no problems in 5.11 and 5.12.

Hence my question : did users of the "newer" "fixed" spkg encounter such problems on Debian machines ?

HTH,

--

Emmanuel Charpentier