[R] rdb and rds files include abolute file paths / help understanding how lazy-load dbs are created

[Philip Rinn]

Hi,

I'm trying to understand why/how absolute file paths are stored in .rdb[1] and
.rds[2] files during package installation. As a consequence building the same r
package in different directories does not produce identical .rdb and .rds files.

The background is that I work on reproducible builds[3] of R packages. I think
this is important from an engineering point of view but also from a scientists
perspective (that's actually my point). I want to be sure that my research results
are reproducible and therefore I need software that builds reproducible.

To investigate further I'd like to ask for some help. From what I understand so
far the lazy-load databases are built by code in
src/library/tools/R/makeLazyLoad.R. The code path is not very clear to me but the
main problem I have now is that it's hard to follow the code path used to install
a package. Could someone enlighten me by pointing me to some docs or by briefly
describing the path?

Any help/comments are very welcome.

Best,
Philip

PS: could you CC me, I'm not on the list. Thanks.

[1] at least in <PKG>/R/<PKG>.rdb and <PKG>/help/<PKG>.rdb
[2] at least in <PKG>/help/paths.rds
[3] https://reproducible-builds.org

______________________________________________
R-h...@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[Jeff Newmiller]

Someone might respond here anyway, but I think this is more of an R-devel question.

Anyway, as long as the package file after installation has appropriate file names for where it is installed, what does it matter what is in the files before installation?
--
Sent from my phone. Please excuse my brevity.

[Jeff Newmiller]

I think we are (I certainly am) going to need a more concrete example. As in, point us at a specific package and filename in this package that illustrates your concern. Such precision would also be expected on R-devel, so the lack of response may have been self-inflicted.

Are you aware of the system.file and file.path functions? File names that YOU enter into your package code should be referenced using these functions. I have been assuming so far that you were referring to filenames picked up by R as it builds the package, which appear to be handled just fine however they are stored though I don't know the details.

--
Sent from my phone. Please excuse my brevity.

On April 19, 2017 2:40:05 PM PDT, Philip Rinn <ri...@inventati.org> wrote:
>Hi,
>

>On 12.04.2017 at 08:09, Jeff Newmiller wrote:
>
>> Someone might respond here anyway, but I think this is more of an
>R-devel
>> question.
>

>I tired R-devel before[1] with no response :(.

>
>> Anyway, as long as the package file after installation has
>appropriate file
>> names for where it is installed, what does it matter what is in the
>files
>> before installation?
>

>That's actually the point. In the installed .rd[bs] files the absolute
>file paths
>are still present but often[2] the referenced files don't even exist on
>the
>computer the packaged is installed. My approach therefore is to replace
>those
>absolute paths by relative[3] paths. But to do so I need to understand
>where
>exactly those absolute paths are injected in the files - that's why I
>asked for
>help in my fist mail - sorry for not being clear enough.
>
>Best,
>Philip
>
>
>[1] https://stat.ethz.ch/pipermail/r-devel/2017-April/074016.html
>[2] At least on Linux for packages installed via the distribution
>packages system.
>And I think this also holds for prebuild windows/mac builds from cran.
>[3] Relative to teh root directory of the package source

[Philip Rinn]

Hi,

On 12.04.2017 at 08:09, Jeff Newmiller wrote:

> Someone might respond here anyway, but I think this is more of an R-devel
> question.

I tired R-devel before[1] with no response :(.

> Anyway, as long as the package file after installation has appropriate file
> names for where it is installed, what does it matter what is in the files
> before installation?

[Philip Rinn]

Hi,

Am 20.04.2017 um 00:32 schrieb Jeff Newmiller:
> I think we are (I certainly am) going to need a more concrete example. As in,
> point us at a specific package and filename in this package that illustrates
> your concern. Such precision would also be expected on R-devel, so the lack of
> response may have been self-inflicted.

Ok, let's give you an example:

philip@debian:~$ wget https://cran.r-project.org/src/contrib/ald_1.1.tar.gz
philip@debian:~$ mkdir test
philip@debian:~$ mkdir test1
philip@debian:~$ cp ald_1.1.tar.gz test/
philip@debian:~$ cp ald_1.1.tar.gz test1/
philip@debian:~$ cd test
philip@debian:~/test$ R CMD INSTALL --build ald_1.1.tar.gz
philip@debian:~/test$ tar -zxvf ald_1.1_R_x86_64-pc-linux-gnu.tar.gz
ald/help/paths.rds
philip@debian:~/test$ cd ../test1
philip@debian:~/test1$ R CMD INSTALL --build ald_1.1.tar.gz
philip@debian:~/test1$ tar -zxvf ald_1.1_R_x86_64-pc-linux-gnu.tar.gz
ald/help/paths.rds
philip@debian:~/test1$ cd ..
philip@debian:~$ Rscript -e "readRDS('test/ald/help/paths.rds')"
[1] "/tmp/RtmpXhF0y3/R.INSTALLd717b7b387a/ald/man/ALD.Rd"
[2] "/tmp/RtmpXhF0y3/R.INSTALLd717b7b387a/ald/man/ald-package.Rd"
[3] "/tmp/RtmpXhF0y3/R.INSTALLd717b7b387a/ald/man/likALD.Rd"
[4] "/tmp/RtmpXhF0y3/R.INSTALLd717b7b387a/ald/man/mleALD.Rd"
[5] "/tmp/RtmpXhF0y3/R.INSTALLd717b7b387a/ald/man/momentsALD.Rd"
attr(,"first")
[1] 46
philip@debian:~$ Rscript -e "readRDS('test1/ald/help/paths.rds')"
[1] "/tmp/RtmpqjNKva/R.INSTALLe362823e45a/ald/man/ALD.Rd"
[2] "/tmp/RtmpqjNKva/R.INSTALLe362823e45a/ald/man/ald-package.Rd"
[3] "/tmp/RtmpqjNKva/R.INSTALLe362823e45a/ald/man/likALD.Rd"
[4] "/tmp/RtmpqjNKva/R.INSTALLe362823e45a/ald/man/mleALD.Rd"
[5] "/tmp/RtmpqjNKva/R.INSTALLe362823e45a/ald/man/momentsALD.Rd"
attr(,"first")
[1] 46

As you see, the file help/paths.rds contains absolute paths which doe not exist
(after compilation) on the PC. I therefore try to find a way to replace these
absolute paths by relative. My favorite output would be:

philip@debian:~$ Rscript -e "readRDS('test2/ald/help/paths.rds')"
[1] "./ald/man/ALD.Rd"
[2] "./ald/man/ald-package.Rd"
[3] "./ald/man/likALD.Rd"
[4] "./ald/man/mleALD.Rd"
[5] "./ald/man/momentsALD.Rd"
attr(,"first")
[1] 46

I just ask for help to understand where those paths got inject (see my first mail
for details). I'm willing to write/test the patches needed.

Hope that helps to understand my idea.

Best,
Philip

[David Winsemius]

I would have thunk that the right list would be

https://stat.ethz.ch/mailman/listinfo/r-package-devel

Best
David

Sent from my iPhone

> ______________________________________________
> R-h...@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

[[alternative HTML version deleted]]