Temporary files overload with Ne2-1L/gl.LDNe()

[Chrystelle Delord]

Hi all,

Sorry to be back already! I do not know if my question is more related to dartR or to another software (namely NeEstimator2).

I tried to run an R script including the command dartR::gl.LDne(...) on a Linux server. Unfortunately, it apparently led to interferences with other jobs from the same cluster, because the binary Ne2-1L had opened temporary files on a disk with very limited space, leading to "saturation".

Is that something someone had experienced as well when using gl.LDNe on large genlight/genepop files (as of course, it requires a lot of pairwise comparisons to get measures of linkage disequilibrium)?

I was wondering if I could resolve the issue by modifying the function gl.LDNe() so the step gl2genepop() stores the genepop file in the working directory, instead of a temporary folder?

Indeed (and as a side-question), it seems like even when specifying outpath = getwd() in gl.LDNe(), all output files, including the output from NeEstimator, are stored in an R temporary folder instead of the current working directory?

Many thanks for your help!

Best regards,

Chrys

[Bernd.Gruber]

Hi,

 

The function actually copies the popgen file to the tempfile. The reason for this is that the Ne2-1L program is a bit of a beast to make sure all path are set correctly and much easier to run from within tempdir, where all path are known.

 

In case you want to run it per Hand, though maybe you can somehow increase the available space as the programm also creates some output files. [there is a free rcloud account where you can run R on linux and it should have enough space for a run (free for 20 hours a month or so)

 

Short description how to run per “hand”:

Yes you can create the genepop file with gl2genpop. The only “difficulty is then that you need another file (the info file that runs with Ne21L and its settings to estimate LD NE). This info file is created on the fly by the function. So you can copy it from tempdir and then run it per command line with the genepop file. From within tempdir. That saves that the file is copied over).

 

 

The file is called ‘info’ no extension and I attached an example file. Here you need to set the path etc.

In the manual the Nestimator 2 you find some more info on info :’)

 

The whole command is then

 

./Ne2-1L -i info xyz.gen

 

 

There is a helpfile (pdf) that explains how to run from command line) and it has also fully commented examples of the info file.

 

Hope that helps,

 

Regards, Bernd

 

 

 

==============================================================================

Dr Bernd Gruber                                              )/_         

                                                         _.--..---"-,--c_    

Professor Ecological Modelling                      \|..'           ._O__)_     

Tel: (02) 6206 3804                         ,=.    _.+   _ \..--( /          

Fax: (02) 6201 2328                           \\.-''_.-' \ (     \_          

Institute for Applied Ecology                  `'''       `\__   /\          

Faculty of Science and Technology                          ')                

University of Canberra   ACT 2601 AUSTRALIA

Email: bernd....@canberra.edu.au

WWW: bernd-gruber

 

Australian Government Higher Education Provider Number CRICOS #00212K 

NOTICE & DISCLAIMER: This email and any files transmitted with it may contain
confidential or copyright material and are for the attention of the addressee
only. If you have received this email in error please notify us by email
reply and delete it from your system. The University of Canberra accepts
no liability for any damage caused by any virus transmitted by this email.

==============================================================================

--
You received this message because you are subscribed to the Google Groups "dartR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dartr+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dartr/89f47811-bda7-4dab-a321-9d28451f900cn%40googlegroups.com.

[Chrystelle Delord]

Thank you very much Bernd!

It appears that my server's system allows to create temporary files (contrary to my personal laptop, where NeEstimator stores data within RAM instead), which is problematic because Ne2-1L causes interferences with other processes. I will try to use another server, or to run Ne2-1L locally.

Thank you very much for the info file! I was able to run Ne2-1L by myself from the command line on my laptop, which is way faster when multiple input files are involved.

Have a great day, thank you very much again,

Best regards

Chrys