Tracer v1.6 and v1.8 Bug Solving/suggestion: state is not consistent with previous spacing

[Mel Melendrez]

This is in connection with the following thread: https://groups.google.com/forum/#!searchin/beast-users/tracer$20state$20is$20not$20consistent$20with$20previous$20spacing/beast-users/u9o1ks-iAxE/yUNZp3ZBtKIJ

about state not matching previous spacing.

So I had an 800 million iteration run x 3 runs. All 3 runs load fine in Tracer v1.6 and v1.8. I then combined them using logCombiner with 10% burnin on each file (80 million). This leaves me with renamed states (every 10K) and a total of 216003 lines in the file.

When loading in tracer I get the following error which I've seen on several posts here just with different states/lines referenced:

"Problem with trace file: State 2147483647 is not consistent with previous spacing (Line 214750)"

So, in my case, this was because during renaming every 10K I went from starting at state 10,000 and ending at state 2160030000.

I was confused because I had no 2147483647 state but If you look at the error, 2147483647 coincidentally matches the 32-bit integer limitation. So I think this was a tracer issue - it basically reads to the limit which in my file was 2147480000 then sees 2147490000 and this is above the integer limit so it attempts to perhaps start at 1 again, then throws the spacing error.

Previous googlegroup suggestions included:

• reprocess via logCombiner and rename
• subsample/resample the log file again in logCombiner (this may change the state numbering to go below 2147483647 and should work)

Basically - if you are running into this issue because your ending state is >2147483647 then you need to either:

1. chop off everything after 2147483647 or in my case 2147480000 (which worked for me, it loaded in Tracer v1.6 after that)
2. Resample or subsample so that the renamed states don't go above 2147483647 in the resulting combined log file.

We tried just renaming states from 1-216003 but then it err'd immediately - so just renaming your states manually won't work. I believe Tracer is expecting every 10K because that was the set up in BEAST? so I had to stay in 10K increments. So I ended up just chopping the file.

Analysis in Tracer was not affected - ESS values were still fine and higher than individual runs.

This is a potentially fixable bug - we've found the code; we're attempting to fix it in v1.8, let us know where we can send the fix.

Cheers,
Mel

Melanie Melendrez, Ph.D.
Chief, Bioinformatics, Lead Scientist

Walter Reed Army Institute of Research

Viral Diseases Branch

Silver Spring, MD 20910

[Andrew Rambaut]

Hi Mel,

A few years ago I went through most of the BEAST v1.X codebase and changed all reference to state numbers into long integers to fix this issue. I will check the Tracer code - something must have been missed.

Tracer doesn’t care what the interval is between states just that it is consistent (except for the first sample which can be either 0 or 1 to accommodate MrBayes). Dropping 3 0s off the end of the state numbers should

be fine. 

Andrew

--
You received this message because you are subscribed to the Google Groups "beast-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beast-users...@googlegroups.com.
To post to this group, send email to beast...@googlegroups.com.
Visit this group at http://groups.google.com/group/beast-users.
For more options, visit https://groups.google.com/d/optout.

Andrew Rambaut 

Institute for Evolutionary Biology | Centre for Infection, Immunity & Evolution 

Ashworth Laboratories, University of Edinburgh, Edinburgh, EH9 3FL, UK

contact – a.ra...@ed.ac.uk | http://tree.bio.ed.ac.uk | tel +44 131 6508624

[Melanie Melendrez]

Ok let us know if you would like us to work on it or if you got it covered.

Mel

---

Melanie Melendrez, Ph.D.
Chief, Bioinformatics, Lead Scientist

Walter Reed Army Institute of Research

Viral Diseases Branch

Silver Spring, MD 20910

melanie.c.m...@us.army.mil

mmele...@gmail.com

--
You received this message because you are subscribed to a topic in the Google Groups "beast-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beast-users/gope3FRUjQk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beast-users...@googlegroups.com.

[Andrew Rambaut]

It is just a case of going through the tracer packages and changing ints to longs whereever a state is mentioned.

Feel free to do it if you want the fix quickly (suggest a branch of the GitHub and then I will merge it back).

Who’d have though people would be doing such long BEAST runs?

Best,

Andrew

[Peter]

Hi All:
I meet the same issues when I run 3000000000 generations. When I ran Tracer v1.6, I got the following errors: "Problem with trace file: State 2147483647 is not consistent with previous spacing (Line 7163)"
After I delete the log info after 2147480000, it is OK. However, some of the parameters is less than 200. most of them are greater than 4000.
When I annotated the tree, I meet the same issues. Should I delete the tree after 2147480000 also to annotate?
How should I deal with it correctly?

Thanks,

Runhua

[david...@auckland.ac.nz]

HI Andrew

the problem seems to be with the numbering of the states.  The two log files below differ only in the sample numbers, while the first causes the error message "State 3 is not consistent with previous spacing (Line 4)", the second is read in fine.

David

fail.log:

Sample    x    y
1    1.85    0.24
2    -0.97    0.79
3    1.02    0.13

pass.log:

Sample    x    y
10    1.85    0.24
20    -0.97    0.79
30    1.02    0.13

[david...@auckland.ac.nz]

Ah ok, different problem to the one in this thread. 

QUick fix from earlier thread suggests starting numbering at 0 which works.  

I can't see why this is required, though.

David

pass2.log

Sample    x    y
0    1.85    0.24
1    -0.97    0.79
2    1.02    0.13

[Damien Hinsinger]

Hi,

I'm not sure if it's the best place to put this, but I ran into the same issue of non consistent spacing in the log file. In my specific case, I suspect it is due to a full hard-drive. Indeed, it seems nothing was written for a few thousands generations, and it coincide with a time my hard-drive was full (I since made some space on it).

I think several cases of non-consistent spacing could be due to that factor, as it can be invisible once the analysis is finished, especially on shared servers.

Would it be possible to add a warning in beast in case it cannot write on the disk (and maybe temporarily stop the analysis ?) ?

Cheers, Damien