Sterling Ratio performance metric

[Judson Wilson]

I just came across Sterling Ratio  - a performance metric that incorporates the maximum draw-down.  What do you guys think of it?

[nonlinear]

I think that maximum drawdown is a very poor measure of risk. Consider two strategies, A and B. Both have the same net profit. Strategy A had 100 drawdowns, and the maximum drawdown was $500. Strategy B had only 1 drawdown, which was $600. If you only look at the maximum drawdown, you would then conclude that strategy A is better, when in fact, strategy B is far superior to strategy A.

[Judson Wilson]

Really?  I would say A looks better on both total drawdown and average magnitude of drawdown.

On Sun, Jun 19, 2011 at 4:44 PM, nonlinear <eugene....@gmail.com> wrote:

I think that maximum drawdown is a very poor measure of risk. Consider two strategies, A and B. Both have the same net profit. Strategy A had 100 drawdowns, and the maximum drawdown was $500. Strategy B had only 1 drawdown, which was $600. If you only look at the maximum drawdown, you would then conclude that strategy A is better, when in fact, strategy B is far superior to strategy A.

--
You received this message because you are subscribed to the Google Groups "JBookTrader" group.
To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/f64OgovrcgQJ.
To post to this group, send email to jbook...@googlegroups.com.
To unsubscribe from this group, send email to jbooktrader...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.

[nonlinear]

On Sunday, June 19, 2011 7:58:31 PM UTC-4, Judson Wilson wrote:

Really?  I would say A looks better on both total drawdown and average magnitude of drawdown.

I disagree. Let's consider the example as in attached pic. Which strategy would you rather trade, A or B? In my view, B is clearly better, even though it had a larger max drawdawn.

 

 

[Astor]

I think I agree with Eugene on this one. A singular event in Strategy B may never repeat. A mutlitude of $500 drawdawns in strategy A suggests that this type of event is likely to re-occur.

From: Judson Wilson <wilson...@gmail.com>
To: jbook...@googlegroups.com
Sent: Sunday, June 19, 2011 6:58 PM
Subject: Re: [JBookTrader] Re: Sterling Ratio performance metric

[Judson Wilson]

Wouldn't strategy B have a lower PI?  The math is a bit hard for me to visualize.

 

--

You received this message because you are subscribed to the Google Groups "JBookTrader" group.

To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/H3RGtkvtUCgJ.

[Judson Wilson]

On Sun, Jun 19, 2011 at 5:16 PM, Astor <astor...@yahoo.com> wrote:

I think I agree with Eugene on this one. A singular event in Strategy B may never repeat. A mutlitude of $500 drawdawns in strategy A suggests that this type of event is likely to re-occur.

Right, but I think A is more statistically reliable, and B might be more vulnerable to rare bad events.  Maybe that is just superstition in me.

[nonlinear]

Wouldn't strategy B have a lower PI?  The math is a bit hard for me to visualize.

Nope. In the case of the previously attached chart, the standard deviation of trades for for strategy B is about 40% lower than that of strategy A. Thus, strategy B will have a PI score which is 40% higher that of strategy A. In the original case of A and B in my first post, that difference would be even greater in favor of strategy B.

[Astor]

Generally, good trailing stop-losses or deep out-of-the money put would help mitigate rare but significant events. It is more expensive to deal with a strategy which has frequent smaller declines: cost of protective puts is higher and/or stop-losses would be hit more often.

From: Judson Wilson <wilson...@gmail.com>
To: jbook...@googlegroups.com

Sent: Sunday, June 19, 2011 7:23 PM

Subject: Re: [JBookTrader] Re: Sterling Ratio performance metric

On Sun, Jun 19, 2011 at 5:16 PM, Astor <astor...@yahoo.com> wrote:

I think I agree with Eugene on this one. A singular event in Strategy B may never repeat. A mutlitude of $500 drawdawns in strategy A suggests that this type of event is likely to re-occur.

Right, but I think A is more statistically reliable, and B might be more vulnerable to rare bad events.  Maybe that is just superstition in me.

[nonlinear]

There is another, more subtle problem with the max DD measure, which is that by reshuffling the same trades of the same strategy you would get drastically different max DD figures. It's as though the same strategy suddenly changes from "good" to "bad" by simply rearranging the trades in time. In reality, it's still exactly the same strategy, with exactly the same risk/reward profile.

If this doesn't click in, think about this experiment. You have two coins, A and B, and you suspect that one of them is a fair coin, while the other one is a loaded one. So, you flip each one 1 million times. The results are: 500,000 heads and 500,000 tails for each coin. However coin A had a a maximum streak of 10 heads in a row, while coin B had a maximum of 5 heads in a row. Based on the Sterling Ratio adapted for coins, coin B would be determined to be 2 times more "fair" than coin A, when in reality, both coins are perfectly and equally fair. On the other hand, if you use the PI measure, it would be identical for coins A and B.

[Judson Wilson]

On Sun, Jun 19, 2011 at 6:10 PM, nonlinear <eugene....@gmail.com> wrote:

There is another, more subtle problem with the max DD measure, which is that by reshuffling the same trades of the same strategy you would get drastically different max DD figures. It's as though the same strategy suddenly changes from "good" to "bad" by simply rearranging the trades in time. In reality, it's still exactly the same strategy, with exactly the same risk/reward profile.

You can believe that drawdowns will happen randomly, or not randomly.   I tend to believe it is somewhere in the middle.

I brought this up because I have been optimizing some strategies pretty heavily the past couple days, and I noticed that there were many combinations that produced the same high levels of overall profit, but some areas had higher draw-downs than others.  The behavior didn't seem to be randomly distributed in the optimization map, rather, I think its clustered, but there isn't a good way to verify this visually.  Then I randomly came across this Sterling Ratio measurement, which appears that it may be useful in exploring the data.

[Judson Wilson]

On Sun, Jun 19, 2011 at 5:41 PM, Astor <astor...@yahoo.com> wrote:

Generally, good trailing stop-losses or deep out-of-the money put would help mitigate rare but significant events. It is more expensive to deal with a strategy which has frequent smaller declines: cost of protective puts is higher and/or stop-losses would be hit more often.

Actually, the put option has drawbacks too - as does trading long and short side.   I am worried that an event like the flash crash could happen again, and one strategy goes long at the top, and flat near the bottom, then the other goes short at the bottom and flat at the top.  At the end I have a huge loss, and the puts haven't changed much (except for volatility value).

[Astor]

In general, if your hedging strategy does not prevent you from incurring large loss then you do not have a hedging strategy.

 

In the event of a "flash" crash, if you have a protective put, you can exercise it and limit your loss. Or you can hold your positions and the put, in which case when the market recovers you have not experienced loss either.  

 

From: Judson Wilson <wilson...@gmail.com>
To: jbook...@googlegroups.com
Sent: Sunday, June 19, 2011 11:56 PM
Subject: Re: [JBookTrader] Re: Sterling Ratio performance metric

On Sun, Jun 19, 2011 at 5:41 PM, Astor <astor...@yahoo.com> wrote:

Generally, good trailing stop-losses or deep out-of-the money put would help mitigate rare but significant events. It is more expensive to deal with a strategy which has frequent smaller declines: cost of protective puts is higher and/or stop-losses would be hit more often.

Actually, the put option has drawbacks too - as does trading long and short side.   I am worried that an event like the flash crash could happen again, and one strategy goes long at the top, and flat near the bottom, then the other goes short at the bottom and flat at the top.  At the end I have a huge loss, and the puts haven't changed much (except for volatility value).

[nonlinear]

Actually, the put option has drawbacks too - as does trading long and short side.   I am worried that an event like the flash crash could happen again, and one strategy goes long at the top, and flat near the bottom, then the other goes short at the bottom and flat at the top.  At the end I have a huge loss, and the puts haven't changed much (except for volatility value).

Anything can happen. However, I look forward to the next flash crash. I didn't actually trade on that day, but the last time I backtested the sample strategies, they turned a huge profit on the flash crash day. The profit was indeed so astronomical that I decided to exclude that day from my optimization data set, since it was heavily skewing the results.

[Judson Wilson]

On Mon, Jun 20, 2011 at 5:35 AM, Astor <astor...@yahoo.com> wrote:

In general, if your hedging strategy does not prevent you from incurring large loss then you do not have a hedging strategy.

 

In the event of a "flash" crash, if you have a protective put, you can exercise it and limit your loss. Or you can hold your positions and the put, in which case when the market recovers you have not experienced loss either.  

 

So how exactly do I do that with JBookTrader?  In general, you can't, so it isn't much of a viable option, IMO.

[Astor]

 

Unfortunately, you are right. JBT does not offer anything for hedging.

From: Judson Wilson <wilson...@gmail.com>
To: jbook...@googlegroups.com

Sent: Monday, June 20, 2011 6:22 PM

Subject: Re: [JBookTrader] Re: Sterling Ratio performance metric