Re: java memory, resampling in logcombiner , etc

[Crystal Hepp]

Hi Liz,
I do my 10% burnin and resampling in LogCombiner and have run into the same problem you are having in the past. Try the following in LogCombiner.

For the burnin, take: ((total number of generations)/sampling) * 0.1

So:
50,000,000/50 =1,000,000
1000000 * 0.1 = 100,000 = BURNIN

Then for the resampling, take:
(starting number of generations - BURNIN)/x = 10000

So:
(50,000,000-100000)/x = 10000
49900000 = 10000x
4990 = x = RESAMPLE STATES AT LOWER FREQUENCY

This should give you approximately 10000 trees...I say approximately because the first state actually starts at 0. I hope this works. I usually start sampling at 1000 so hopefully I have done my math right with you sampling at 50. Good luck.

-Crystal

On Tue, Jan 22, 2013 at 3:48 PM, Liz_M <elizabet...@selu.edu> wrote:

Hello all,

Here's the deal: 

• Ran one 50,00,000 generation run logging every 50 in order to up the ESS values. In Tracer v1.5 all ESS values are >200.
  
• Used TreeAnnotator v1.7.4 to remove burn in. 
• 10% removal (this program removes trees, not states, right? My math: 50,000,000 states/50 = 1,000,000 trees. 1,000,000*0.1=100,000 trees removed) = java ran out of memory
• 20% removed (200,000 trees removed) = java ran out of memory again
• Gave up and tried resampling method 
• Used LogCombiner v1.7.4 to resample at a lower frequency. I tried to get the # of trees to 10,000 (after reading Andrew's suggestions).
• 50,000,000 states / 5000 resampled log freq = 10,000 trees (right?)
• Used TreeAnnotator v1.7.4 to remove burn in (trees).
• Removed 10%: 10,000 trees*0.1 = 1000 removed (the value I entered for burn in).
• Tree Annotator says there are no trees available. 

I've been working on this for a while and can't seem to figure this out. I'm using a 50mil generation run because I has issues combining log files... perhaps this is not ideal? Obviously the tree file is way too big.

I understand I can up the memory for java but have yet to try that (I am using Windows 7)

The only other idea I have is to remove a crap load of burn in to get TreeAnnotator to run properly. Would that actually work? Is that even a good idea?

I admit that removal of burn-in always confuses me, so did I make a mathematical error? I'm googly-eyed from trying to figure this out.

Any help or suggestions for what to try next would be helpful. Thanks so much,

Liz

--
You received this message because you are subscribed to the Google Groups "beast-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/beast-users/-/Yy3gl0PRrjQJ.
To post to this group, send email to beast...@googlegroups.com.
To unsubscribe from this group, send email to beast-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/beast-users?hl=en.

[hazelja...@yahoo.co.uk]

Hi All

I know this is an old post but i'm having similar problems and can't seem to get it working.

I also have a large number of trees and have been through the process of trying to thin them out in logcombiner (to try to obtain approx 10,000).

Having tried to follow the solution given here, I get an error message in logcombiner saying the resampling frequency is not a multiple of the existing sampling frequency.

200,000,000 generations/sampling 1000th.

Burnin 50,000 (25%)

So, this is what I worked out for log combiner

200,000,000-50,000 = 199950000

199950000/19995 = 10000 trees (right?)

but when I use 19995 as the resampling frequency I get the error message.  If I use a multiple of the original 1000 then it works, but when I try to run it through TreeAnnotator I get a 'no trees' message.

I'm STUMPED.  I'd really appreciate any help you could offer.

Thank you

Hazel

[Andrew Rambaut]

Dear Hazel,

You need to thin out the trees at some multiple of the original sampling frequency (it is not going to be able to construct samples that don't exist). So if you are sampling every 1000 you can resample every 20000 to get 10000. Burnin is always given as the number of original samples so to get a 25% burnin you specify 50,000,000 irrespective of what sampling frequency you use.

Andrew

--
You received this message because you are subscribed to the Google Groups "beast-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beast-users...@googlegroups.com.

To post to this group, send email to beast...@googlegroups.com.

Visit this group at http://groups.google.com/group/beast-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Victoria Johnson]

Hi 

I'm having a similar problem.

I have 50 000 000 generations logging every 500 which therefore means I have 100 000 trees. In TreeAnnotator my file was too big so I tried using Log Combiner - I'm a bit confused as to what lower sampling frequency I should use. I have to use a multiple of 500 so I used 1000 - with a 10% burnin of 5000000 however when I opened my LogCombiner file in TreeAnnotator it only analyses 1444 trees. Please could you help me with what lower sampling frequency I need to use to get 10 000 trees?

Thank you