Evaluating Monte Carlo results
furytrader
Michael R. Bryant
The new section in the guide “Stress Testing and Monte Carlo Analysis”, which is excerpted from my recent article on the same topic, is meant to provide the guidance. If you’re asking how to interpret Monte Carlo results in general, the example in that section includes the following:
“This result can be interpreted as follows: if the randomization of the price data is representative of the kind of random differences expected in the market, then 95% of the time, the net profit can be expected to be at least $68,459.”
Mike Bryant
Subject: Evaluating Monte Carlo results
furytrader
Michael R. Bryant
If you select the stress testing option before building, it will use the Monte Carlo results to calculate the fitness. Basically, whenever that option is selected, the Monte Carlo results ARE the results. Anywhere you see a listed performance result, it’s a Monte Carlo result when that option has been selected.
Mike Bryant
Subject: Re: Evaluating Monte Carlo results
mandelmus
Steve Carkeek
I'm really enjoying this feature, it takes about 10x as long to run but has found some interesting strategies when "applied during the build process".
--
You received this message because you are subscribed to the Google Groups "Adaptrade Builder" group.
To unsubscribe from this group and stop receiving emails from it, send an email to adaptrade-buil...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
mandelmus
Prior to these options being available, I had exported the strategy trades to Market System Analyzer and Monte Carlo'd the trade order. Had you considered this as an additional stress testing option, or is Monte Carlo'ing trade order not as important anymore given the current options?
Michael R. Bryant
My emphasis was on stress testing, which is why I selected the methods I did. Randomizing the trade order is fine, but is more about getting a better estimate of drawdown than anything else.
Mike Bryant
Subject: Re: Evaluating Monte Carlo results
The MC Analysis/Stress Testing section has 3 methods: randomizing prices, randomizing strategy inputs, and randomizing the starting bar.
Michael R. Bryant
Sorry it’s not clearer. This was my attempt to explain it:
If stress testing has been applied, the results shown in the results tables will be the Monte Carlo results at the confidence level specified under Build Options. Note: Monte Carlo results of stress testing are calculated metric-by-metric at the specified confidence level and therefore are not directly related to one another. For example, the Monte Carlo net profit is not necessarily the sum of the long Monte Carlo net profit and short Monte Carlo net profit because the long and short net profit are each calculated at 95% confidence. This applies to all metrics.
It’s pretty simple. If you run an evaluation with MC analysis, the results you see will be the MC results.
Mike Bryant
Subject: Re: Evaluating Monte Carlo results
I'm re-reading the "Stress Testing and Monte Carlo Analysis" usage instructions in the manual for the 5th+ time and want to get some clarification.
"... The highest value, $147,855, corresponds to the original file of price data. The lowest value is $50,201. The Monte Carlo analysis can be used to estimate the likely net profit with a specified degree of confidence given the variety of results. A confidence level of 95% is typical, which means there would be a 5% chance of the net profit being lower than the estimated value. ... Since there are 20 items in the list, the 19th item in the sorted list is selected, which would be a net profit of $68,459; i.e., the second lowest value in the list. ... This result can be interpreted as follows: if the randomization of the price data is representative of the kind of random differences expected in the market, then 95% of the time, the net profit can be expected to be at least $68,459."
How is this reflected on the Results tabs? In this case, would the $68,459 be the value listed as the "Net Profit" displayed on that "Results" tab?
This result can be interpreted as follows: if the randomization of the price data is representative of the kind of random differences expected in the market, then 95% of the time, the net profit can be expected to be at least $68,459. ... Builder applies the same approach to all performance metrics tracked by the program. If the metric is one where a lower value is better, such as maximum drawdown, the value is selected 95% of the way up the list."
If I select the 95% "Confidence Level", does that mean that the values listed on each row of a single strategy fall within the 95% confidence level? Or, does it mean that the values of only the selected build metrics fall within the 95% confidence level?
I'm very familiar with MC and statistical analysis, so this is just a question of how Builder presents the results of the MC analysis.
mandelmus
"... The highest value, $147,855, corresponds to the original file of price data. The lowest value is $50,201. The Monte Carlo analysis can be used to estimate the likely net profit with a specified degree of confidence given the variety of results. A confidence level of 95% is typical, which means there would be a 5% chance of the net profit being lower than the estimated value. ... Since there are 20 items in the list, the 19th item in the sorted list is selected, which would be a net profit of $68,459; i.e., the second lowest value in the list. ... This result can be interpreted as follows: if the randomization of the price data is representative of the kind of random differences expected in the market, then 95% of the time, the net profit can be expected to be at least $68,459. ... This result can be interpreted as follows: if the randomization of the price data is representative of the kind of random differences expected in the market, then 95% of the time, the net profit can be expected to be at least $68,459"
? How is this reflected on the Results tabs? In this case, would the $68,459 be the value listed as the "Net Profit" displayed on the "Results" tab rather than the "Net Profit" of the "original" equity curve?
"... Builder applies the same approach to all performance metrics tracked by the program. If the metric is one where a lower value is better, such as maximum drawdown, the value is selected 95% of the way up the list."
? When determining the '95%' strategy, will Builder only consider the items selected in the build metrics? So, for example, if I exclude "Net Profit" from the build metrics, Builder will not consider that?
*I'm very familiar with MC and statistical analysis, so this is just a question of how Builder presents the results of the MC analysis, not of how to interpret MC analysis