error with BEAST2.2.0 SNAPP+MODEL_SELECTION

[CJ Battey]

Hi BEAST developers, 

I'm trying to run a species delimitation analysis using SNAPP and the MODEL_SELECTION package in BEAST2.2.0. Earlier this year I ran some short trial runs using a pre-release version of BEAST 2.2 (here's a link to the pre-release version I've been using: https://www.dropbox.com/sh/0auvoinnpm1q4hg/AACsM-WWfxuQIBFsAuRS9JlXa?dl=0) and everything worked well, but I've tried to run the same analyses in the final version of 2.2.0 and they all fail with the same error: 

"File error: /home/ubuntu/Dropbox/vRAD3/phylo.analyses/BFD/lumpOliv/step11/beast.xml.state (No such file or directory)" 

Not sure what's going on here. The runs always fail with the same error in 2.2.0 - failing to find beast.xml.state in the last step started (ie if I run it on 8 cores it will report an error missing beast.xml.state in the step7 directory) - but they appear to run correctly in the 2.2 pre-release version I downloaded, using the same input files on the same system (amazon ec2,16 cores, ubuntu, java 8 installed). I'm hesitant to start a full run with the pre-release software in case there were bugs corrected prior to final release - is that something I should be concerned about here? Any advice would be much appreciated. Best, 

CJ Battey

PS thanks for the updates to get missing data working in SNAPP again - it's running well for me and seems faster than previous versions. 

[Remco Bouckaert]

Hi CJ,

I tried to replicate the problem, but have not had success so far. I assume you have the latest SNAPP (v1.1.10) and MODEL_SELECTION (v1.0.3) installed.

Is it true that none of the steps actually run, or do some steps run but some don't? I found a bug in the XML being written to file in some cases, which may prevent any of the steps being run. This issue is fixed now, and if you replace your beast.jar file with the latest (available from http://hudson.cs.auckland.ac.nz/job/BEAST2/lastSuccessfulBuild/artifact/build/dist/beast.jar), and run BEAST from the command line it should fix this issue.

Otherwise you could use the pre-release version since I don't think there are issues solved after the pre-release that would affect a SNAPP analysis in combination with model-selection.

Cheers,

Remco

[Maria Sobczyk]

Hi Remco,

I have somewhat related error when trying to run BFD in BEAST in Linux. This happens both in the latest version of BEAST 2.3.0 and BEAST 2.1.3 with latest releases of SNAPP and MODEL_SELECTION  installed. When I try to run my XML files, the run fails immediately with:

/home/marynia/BEAST/step0/run.sh
/home/marynia/BEAST/step0/run.sh: 1: /home/marynia/BEAST/step0/run.sh: Syntax error: ";" unexpected
File error: /home/marynia/BEAST/step0/beast.xml.state (No such file or directory)

I generated the XML files in Beauti 2.1.3 in Windows, and the XML files are accepted both by Windows versions of BEAST 2.3.0 and 2.1.3. I have attached both the Win (works fine) and Linux (gives error described above) versions of the XML files. I will be grateful for help!

Best wishes,

Maria

[Rebecca Harris]

Hi - 

I was wondering if there was an resolution to the previous post. I am trying to run BFD as well and am getting the same exact error as Maria. I have got these files to run locally but when I start them on my cluster, I get the beast.xml.state error. When I run this locally, it does not appear that there is any beast.xml.state file, so it's unclear to me why it's looking for the file. The versions of Model_selection and SNAPP are the same on my computer and on the cluster, so I do not believe this to be a version incompatability.

Thanks,

Rebecca

[Remco Bouckaert]

Hi Megan,

If you run BEAST from the command line on OSX and Linux, there is a script in the BEAST directory called ./bin/beast which almost at the end contain lines starting with

$JAVA -Xms64m -Xmx4g …

The “-Xmx4g” sets the amount of memory, and is 4 gigabyte by default. Just change the number 4 to something higher to reserver more memory. Note, this line occurs in two places, and it depends on you settings on which of the two is used.

On OSX, you can also start BEAST using

open /Applications/BEAST\ 2.4.2/BEAST.app

In this case, BEAST uses the settings specified in the Info.plist file, which you can find here:

/Applications/BEAST\ 2.4.2/BEAST.app/Contents/Info.plist

You can change the memory reserved by editing the string

<string>-Xmx4g</string>

Cheers,

Remco

On 12/08/2016, at 11:35 AM, Megan Aylward <aylwar...@gmail.com> wrote:

* Edit to original issue. 

This only occurs when I try using the beast/lib/beast.jar file rather than the executable file in beast/bin. Is there any way to increase the JVM memory when running the file in bin rather than the .jar file?

On Thursday, August 11, 2016 at 2:45:34 PM UTC-6, Megan Aylward wrote:

I am running into a similar error. Beast with SNAPP has been running ok, now I have increased the number of mcmc chains which requires me to increase the JVM memory. When I do this using 8 threads and running 24 steps it runs steps 0-6 then fails are step 7 with the error File error: ./step7/beast.xml.state (No such file or directory)

Any idea why this could be?

Thanks,

Megan

--
You received this message because you are subscribed to the Google Groups "beast-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beast-users...@googlegroups.com.
To post to this group, send email to beast...@googlegroups.com.
Visit this group at https://groups.google.com/group/beast-users.
For more options, visit https://groups.google.com/d/optout.

[Remco Bouckaert]

Hi Megan,

Are you using the latest version of the MODEL_SELECTION package (currently 1.3.2)? If so, can you have a look at the step6 folder and see whether there is a beast.xml.state file there? Possibly, step 6 failed to run properly, and then step 7 (which starts at the end-state of step 6) has no way to get started.

Cheers,

Remco

On 12/08/2016, at 8:45 AM, Megan Aylward <aylwar...@gmail.com> wrote:

I am running into a similar error. Beast with SNAPP has been running ok, now I have increased the number of mcmc chains which requires me to increase the JVM memory. When I do this using 8 threads and running 24 steps it runs steps 0-6 then fails are step 7 with the error File error: ./step7/beast.xml.state (No such file or directory)

Any idea why this could be?

Thanks,

Megan

On Sunday, December 6, 2015 at 1:58:56 PM UTC-7, Rebecca Harris wrote:

[Steven Myers]

Hi developers,

It doesn't seem like this problem was ever resolved. I'm encountering the same problem -- I'm trying to run a species delimitation analysis using SNAPP and the MODEL_SELECTION package in BEAST2 on a linux cluster with the latest versions of everything installed (java 1.8.0_91, BEAST v2.4.3, SNAPP v1.3.0, BEASTLabs v1.6.0, MODEL_SELECTION v1.3.2). When I try to run my XML files, with 16 threads, the run fails virtually immediately with:

===============================================================================

Starting step 0

Starting step 2

Starting step 3

Starting step 4

Starting step 5

Starting step 6

Starting step 7

Starting step 8

Starting step 9

Starting step 10

Starting step 11

Starting step 12

Starting step 13

Starting step 15

Starting step 14

Starting step 1

Finished step 0

Finished step 7

Finished step 9

Finished step 4

Finished step 12

Finished step 10

Finished step 6

Finished step 1

Finished step 13

Finished step 5

Finished step 8

Finished step 15

Finished step 3

Finished step 14

Finished step 2

Finished step 11

File error: data_Filtered1row_RC_L15S10MDPC_SNAPP_short_Hyp1/step15/beast.xml.state (No such file or directory)

java.lang.RuntimeException: An error was encounted. Terminating BEAST

at beast.app.util.ErrorLogHandler.publish(Unknown Source)

at java.util.logging.Logger.log(Logger.java:738)

at java.util.logging.Logger.doLog(Logger.java:765)

at java.util.logging.Logger.log(Logger.java:788)

at java.util.logging.Logger.severe(Logger.java:1463)

at beast.app.beastapp.BeastMain.<init>(Unknown Source)

at beast.app.beastapp.BeastMain.main(Unknown Source)

I generated the XML file in Beauti 2.4.2 in Windows, and it's accepted by and runs perfectly fine in the Windows versions of BEAST 2.4.2.

Any help would be appreciated!

[Remco Bouckaert]

Hi Steven,

It is not obvious what the problem is. Can you send me the XML so I can have a look at what is going wrong?

Cheers,

Remco

[Remco Bouckaert]

Hi Steven,

Thanks for the file. So far, I have not been able to replicate your problem; I ran your example file on OSX with the fas-taxon removed (for speed) and with 1 and 4 threads, and both runs ended fine. On Linux (with BEAST v2.4.3, SNAPP v1.3.0, MODEL_SELECTION v1.3.2, and some others) I ran the file unchanged, and it finished without problems with 4 threads.

One possibility is that the default java version on the linux machine you are running the analysis on is not java 8. That would explain the quick skipping of the first 16 steps, than failing on the next because the state file is not generated before. In that case, you should edit the line in the XML that says

java -cp $(java.class.path) beast.app.beastapp.BeastMain $(resume/overwrite) -java -seed $(seed) beast.xml

and replace it with

/path/to/java8/bin/java -cp $(java.class.path) beast.app.beastapp.BeastMain $(resume/overwrite) -java -seed $(seed) beast.xml

where /path/to/java8 is the path to where java version 8 is installed.

Hope this helps,

Remco

[Steven Myers]

Hi Remco,

I tried your solution but got the same error. I even got an additional error  'Error: Could not find or load main class beast.app.beastapp.BeastMain' (full error details below):

Created directory /mnt/beast/data_Filtered1row_RC_L15S10MDPC_SNAPP_short_Hyp1

/mnt/beast/data_Filtered1row_RC_L15S10MDPC_SNAPP_short_Hyp1/step0/run.sh

Error: Could not find or load main class beast.app.beastapp.BeastMain

File error: data_Filtered1row_RC_L15S10MDPC_SNAPP_short_Hyp1/step0/beast.xml.state (No such file or directory)

java.lang.RuntimeException: An error was encounted. Terminating BEAST

at beast.app.util.ErrorLogHandler.publish(Unknown Source)

at java.util.logging.Logger.log(Logger.java:738)

at java.util.logging.Logger.doLog(Logger.java:765)

at java.util.logging.Logger.log(Logger.java:788)

at java.util.logging.Logger.severe(Logger.java:1463)

at beast.app.beastapp.BeastMain.<init>(Unknown Source)

at beast.app.beastapp.BeastMain.main(Unknown Source)

Regards,

Steven

[Steven Myers]

I found the problem and solution (at least for my cause of the error).

In a round-about way it had to do with the directory that I was running the command from. It seems the path to beast used in the run command is inserted in the run.sh file produced by beast in the corresponding step folder in the rootdir. All the other paths in the run.sh file seem to be the absolute path, and the error occurs if the path to beast isn't also the absolute path. I was running from the directory containing my data -- a different directory to root (e.g. path: /mnt/data), so subsequently used ../../usr/local/bin/beast/lib/beast.jar in the run command; this was inserting '../../usr/local/bin/beast/lib/beast.jar' as the path to beast in the run.sh file, which clearly isn't the absolute path. However, I've discovered I don't need to use the path to beast from my current directory in the run command; no matter what directory I'm in I can just use the absolute path and it will work.

Cheers,

Steven

[Carlos Spano]

Hi all,

I've been stuck in this same problem for two days now with no idea how to fix it. By Steven's last comment seems that the issue is related to the data paths, however, I have both the software and xml file in the same directory. The problem is exactly as described above (File error: /…/step7/beast.xml.state (No such file or directory)) and it happens both using the executable file and the command line. Any help on how to solve this would be very appreciated.

Many thanks!

Carlos

[Liz W]

Hi All,
I am having a similar issue trying to run the BFD* tutorial. The provided SNP datasets will not load in BEAUTi v2.1.3  on my MAC (running Sierra). I made sure SNAPP and Model_Selection were installed and installed the latest JDK (v1.8). Other Nexus files load fine, but none of the provided example datasets for use with BFD*. Has anyone determined how to correct this issue?

Many thanks!
Liz

[Amely Martins]

Hi all,

I am trying to run BFD tutorial before I run it with my own data. However, I’m having similar issues. I have BEAST 2.6.1, MODEL_SELECTION 1.5.2, and SNAPP 1.5.0 installed.

I’m having the error:

“File error: C\Users\abm\BEAST\2.6\SNAPP\examples\BFD_-tutorial\runA\step3\beast.xml.state (The system cannot find the file specified)

Any thoughts/news on this matter?

Amely

[Isis Arantes]

Hello all,

I'm also having the same issue. I tried to change the path to the java version used but it not fix the issue.  Any help?

Thanks you,

Ísis

Loading package SNAPP v1.4.2

Probing: snap.addon.jar Loading package BEAST v2.5.2

Probing: beast.jar Probing: beast.src.jar Loading package MODEL_SELECTION v1.4.2

Probing: MODEL_SELECTION.addon.jar Loading package BEASTLabs v1.8.1

Probing: BEASTlabs.addon.jar Loading package BEAST v2.5.2

Probing: beast.jar Probing: beast.src.jar Starting step 0

Starting step 1

Starting step 2

Starting step 4

Starting step 3

Starting step 5

Starting step 6

Starting step 7

Finished step 3

Finished step 6

Finished step 7

Finished step 1

Finished step 0

Finished step 5

Finished step 2

Finished step 4

File error: /ddn/home5/r2175/ptmp/SNAPP/BFD_Dendro/A/runA/step7/beast.xml.state (No such file or directory)

[Glenn Seeholzer]

Hi,

I am having the same error as everyone else. In my case, the tutorial scripts work as expected. However, when I try it on my own highly reduced dataset, some runs work, others terminate in with the error under discussion. 

I'm curious, have the two most recent replies to this thread (Isis and Carlos) figured out the issue?

Best,

Glenn

[Miller, Mark]

I found a similar error a while back. It happened I was working on 24 cores, the full number available on the node I was using. When I dialed it back to 23 cores, the run worked.

 

Maybe a contention issue or something?

Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/beast-users/314f78dd-3c5c-4239-b045-e9aadc12005e%40googlegroups.com.

[felip...@gmail.com]

Any updates on this issue? I have run BFD analysis using SNAPP before in a linux machine and this problem just came up, no ideias to fix it yet.

[Nichelle Van Tassel]

Hi all, 

I was able to fix my errors by applying the solutions below: You can see the exact errors I got here: https://groups.google.com/g/beast-users/c/-gDbCKHh7Nw/m/dh7R_SSaBQAJ?utm_medium=email&utm_source=footer

The problem seemed to stem from  java -cp "/path/beast/lib/launcher.jar" beast.pkgmgmt.launcher.BeastLauncher in the run.sh files.

I was able to fix this and run the analysis on the cluster 2 ways. 

A. change that line in the run.sh files

     1. Add donotrun='true' to the <run> chunk of the xml file

     2. In each run.sh file, change java -cp "/path/beast/lib/launcher.jar" beast.pkgmgmt.launcher.BeastLauncher to beast

     3. Submit each of those ruh.sh jobs by creating a job file with the code below:

               echo + `date` job $JOB_NAME started in $QUEUE with jobID=$JOB_ID on $HOSTNAME
               echo + NSLOTS = $NSLOTS
              if [ -z $1 ]; then
              echo "Give the run number as an argument: e.g. qsub jobfilename.sb 0"
              exit 1
              fi 
             source ./run${1}.sh
             echo = `date` job $JOB_NAME done

     4. And submitting those with the code below:

       for x in $(seq 0 23); do   qsub -o run${x}.log jobfilename.sb ${x}; done

B. change that line in the xml file and then run as normal

           1. change the lines from:

               cd $(dir)

               java -cp $(java.class.path) beast.app.beastapp.BeastMain $(resume/overwrite) -java -seed $(seed) beast.xml

                to:

               cd $(dir)

               beast $(resume/overwrite) -java -seed $(seed) beast.xml 

 

          2. Submit job as normal