Rationale for the fully functional trial
136 views
Skip to first unread message
DaveA
Apr 26, 2012, 4:32:18 AM4/26/12
to Adaptrade Builder
Why purchasing the program when I can use a free extended or perpetual
functional trial? How? The simplest way is by using another PC or just
by reformatting a PC dedicated to that and other causes. Oh, let's
forget about that for a moment as I would probably purchase a license
if I could see some proof that this is something more than random
curve-fitting. The first question I have is this one: Is the program
supposed to converge to a specific system for a very large number of
generations and populations? I'm asking this because I get different
systems for the most part and some pass out-of-sample tests but some
others do not. I noticed that for 100 generation and for 100
populations, the program generated different systems every time I had
it run my project. Some of the systems failed out-of-sample and some
systems passed. Selecting a system that passes the test introduces
selection bias and possibly data snooping bias if this is done after a
losing system was observed.
functional trial? How? The simplest way is by using another PC or just
by reformatting a PC dedicated to that and other causes. Oh, let's
forget about that for a moment as I would probably purchase a license
if I could see some proof that this is something more than random
curve-fitting. The first question I have is this one: Is the program
supposed to converge to a specific system for a very large number of
generations and populations? I'm asking this because I get different
systems for the most part and some pass out-of-sample tests but some
others do not. I noticed that for 100 generation and for 100
populations, the program generated different systems every time I had
it run my project. Some of the systems failed out-of-sample and some
systems passed. Selecting a system that passes the test introduces
selection bias and possibly data snooping bias if this is done after a
losing system was observed.
MikeBryant
Apr 26, 2012, 1:16:00 PM4/26/12
to Adaptrade Builder
Putting aside the ethical question of trying to defeat the purpose of
the trial, I suspect most of my customers purchase the program because
they find it valuable and plan to use it on an ongoing basis, not just
for 14 days. If you really feel that reformatting your PC every 14
days costs less than purchasing the program, there's probably not much
I can say about that.
The genetic programming process involves an element of randomness by
design. It shouldn't and is not supposed to converge on the same
solution each time just because you use a large population with many
generations. That would defeat the nature of the discovery process and
limit the ability of the program to find good solutions.
As far as curve-fitting is concerned, you always have to be aware of
that possibility and take measures, such as those discussed in the
user's guide, for minimizing that risk. That includes utilizing the
built-in out-of-sample testing. In my experience, the risk of curve-
fitting is no greater with this approach than it is for the usual
trial-and-error approach that most traders employ then developing
trading strategies. As for selection bias and data snooping bias, I
recently wrote a newsletter article (under Newsletter link on www.adaptrade.com)
that addresses those issues, among others.
Mike Bryant
Adaptrade Software
the trial, I suspect most of my customers purchase the program because
they find it valuable and plan to use it on an ongoing basis, not just
for 14 days. If you really feel that reformatting your PC every 14
days costs less than purchasing the program, there's probably not much
I can say about that.
The genetic programming process involves an element of randomness by
design. It shouldn't and is not supposed to converge on the same
solution each time just because you use a large population with many
generations. That would defeat the nature of the discovery process and
limit the ability of the program to find good solutions.
As far as curve-fitting is concerned, you always have to be aware of
that possibility and take measures, such as those discussed in the
user's guide, for minimizing that risk. That includes utilizing the
built-in out-of-sample testing. In my experience, the risk of curve-
fitting is no greater with this approach than it is for the usual
trial-and-error approach that most traders employ then developing
trading strategies. As for selection bias and data snooping bias, I
recently wrote a newsletter article (under Newsletter link on www.adaptrade.com)
that addresses those issues, among others.
Mike Bryant
Adaptrade Software
DaveA
Apr 26, 2012, 2:16:22 PM4/26/12
to Adaptrade Builder
On Apr 26, 1:16 pm, MikeBryant <m...@breakoutfutures.com> wrote:
> Putting aside the ethical question of trying to defeat the purpose of
> the trial, I suspect most of my customers purchase the program because
> they find it valuable and plan to use it on an ongoing basis, not just
> for 14 days. If you really feel that reformatting your PC every 14
> days costs less than purchasing the program, there's probably not much
> I can say about that.
disks but thousands of young traders who have no money to buy your
software but all the time in the world can use it for free to develop
and post systems in collective2 and striker, just to mention two
examples. That is not good for your customers who paid $1K to get a
license. It isn't also fair to those that paid $1K when you start
selling this for $85/month. You see I'm trying to understand based on
your business model the true value of your product. One of your
competitors doesn't offer a demo and sells the program for about $60K.
Either he is a snake oil salesman or you are doing something wrong
with your approach. I'm only trying to find you and I think my worries
are legitimate.
Look, if you sell a program for $85, those that act in an illegal
manner will find it profitable to buy it, remove the password
protection and then sell it for $5 to thousands of people. Do you
think this is fair to the customers that have paid $1k or more to buy
a copy? Again, I'm trying to understand why you are offering the
program at such a low price and if this means that it doesn’t work.
>
> The genetic programming process involves an element of randomness by
> design. It shouldn't and is not supposed to converge on the same
> solution each time just because you use a large population with many
> generations. That would defeat the nature of the discovery process and
> limit the ability of the program to find good solutions.
sort of a limitation but I will skip this one for now.
>
> As far as curve-fitting is concerned, you always have to be aware of
> that possibility and take measures, such as those discussed in the
> user's guide, for minimizing that risk. That includes utilizing the
> built-in out-of-sample testing. In my experience, the risk of curve-
> fitting is no greater with this approach than it is for the usual
> trial-and-error approach that most traders employ then developing
> trading strategies. As for selection bias and data snooping bias, I
> recently wrote a newsletter article (under Newsletter link onwww.adaptrade.com)
> that addresses those issues, among others.
Thank you for your answers but what makes me skeptical about your
> As far as curve-fitting is concerned, you always have to be aware of
> that possibility and take measures, such as those discussed in the
> user's guide, for minimizing that risk. That includes utilizing the
> built-in out-of-sample testing. In my experience, the risk of curve-
> fitting is no greater with this approach than it is for the usual
> trial-and-error approach that most traders employ then developing
> trading strategies. As for selection bias and data snooping bias, I
> recently wrote a newsletter article (under Newsletter link onwww.adaptrade.com)
> that addresses those issues, among others.
product is the fact that you have introduced ways of actually
facilitating data snooping in the operations of your program by
resetting populations if the out-of-sample test fails. This is
actually the definition of data snooping. As soon as you reset
populations because of out-of-sample test failure, data snooping and
selection bias are introduced. If you keep on doing this, eventually
you will find some system that survives out-of-sample testing by
virtue of pure luck. The same is true if you run many generations
without resetting but choose only the surviving members.
>
> Mike Bryant
> Adaptrade Software
>
> On Apr 26, 1:32 am, DaveA <dvaron...@gmail.com> wrote:
>
>
>
> > Why purchasing the program when I can use a free extended or perpetual
> > functional trial? How? The simplest way is by using another PC or just
> > by reformatting a PC dedicated to that and other causes. Oh, let's
> > forget about that for a moment as I would probably purchase a license
> > if I could see some proof that this is something more than random
> > curve-fitting. The first question I have is this one: Is the program
> > supposed to converge to a specific system for a very large number of
> > generations and populations? I'm asking this because I get different
> > systems for the most part and some pass out-of-sample tests but some
> > others do not. I noticed that for 100 generation and for 100
> > populations, the program generated different systems every time I had
> > it run my project. Some of the systems failed out-of-sample and some
> > systems passed. Selecting a system that passes the test introduces
> > selection bias and possibly data snooping bias if this is done after a
>
> - Show quoted text -
MikeBryant
Apr 26, 2012, 4:56:33 PM4/26/12
to Adaptrade Builder
On Apr 26, 11:16 am, DaveA <dvaron...@gmail.com> wrote:
> On Apr 26, 1:16 pm, MikeBryant <m...@breakoutfutures.com> wrote:
>
> > >
> I guess my message did not go through. I don't have time reformatting
> disks but thousands of young traders who have no money to buy your
> software but all the time in the world can use it for free to develop
> and post systems in collective2 and striker, just to mention two
> examples. That is not good for your customers who paid $1K to get a
> license. It isn't also fair to those that paid $1K when you start
> selling this for $85/month. You see I'm trying to understand based on
> your business model the true value of your product. One of your
> competitors doesn't offer a demo and sells the program for about $60K.
> Either he is a snake oil salesman or you are doing something wrong
> with your approach. I'm only trying to find you and I think my worries
> are legitimate.
>
$85. There's a monthly fee for 12 months. It's a payment plan, not an
offer to use it for that price. Anyone who purchases it on the payment
plan is entering a contract to purchase the program for $995, just as
they are if they purchase it outright. In any case, I've been selling
software for 10 years. There will always be dishonest people who
cheat, steal, and otherwise break the law. Where possible, I'll persue
anyone who does that. What I won't do is make it difficult or
inconvenient for my legitimate customers to try and purchase the
program.
those that act in an illegal
> manner will find it profitable to buy it, remove the password
> protection and then sell it for $5 to thousands of people. Do you
> think this is fair to the customers that have paid $1k or more to buy
> a copy? Again, I'm trying to understand why you are offering the
> program at such a low price and if this means that it doesn’t work.
>
that claims to have an "unlocked" or "open" version of the program is
lying in order to make a dishonest buck. I'll start by sending a cease-
and-desist order to any site, company, or person who does as you
suggest, but, as I've said, I've been doing this for a while, and it
hasn't been a problem.
You've spent quite a bit of time wondering about this issue and that
issue when you'd probably be better served trying the software and
seeing for yourself if it works. That's the purpose of the trial after
all.
> > >
> Thank you for your answers but what makes me skeptical about your
> product is the fact that you have introduced ways of actually
> facilitating data snooping in the operations of your program by
> resetting populations if the out-of-sample test fails. This is
> actually the definition of data snooping. As soon as you reset
> populations because of out-of-sample test failure, data snooping and
> selection bias are introduced. If you keep on doing this, eventually
> you will find some system that survives out-of-sample testing by
> virtue of pure luck. The same is true if you run many generations
> without resetting but choose only the surviving members.
>
solution, there's nothing I could do to prevent you from doing that.
The reset option is designed to be used in a responsible fashion and
is a legitimate method to arrive at a solution if used properly. In my
opinion, you should use a third period following the out-of-sample
data to verify the final solution. Better yet, use real-time tracking
on any strategy you plan to trade. There's a point at which the user/
trader needs to be responsible for verifying any results before
committing real money in the markets.
Mike Bryant
Adaptrade Software
mandelmus
Apr 29, 2012, 3:15:23 PM4/29/12
to adaptrad...@googlegroups.com
On Thursday, April 26, 2012 3:32:18 AM UTC-5, DaveA wrote:
Why purchasing the program when I can use a free extended or perpetual
functional trial? How? The simplest way is by using another PC or just
by reformatting a PC dedicated to that and other causes.
uummm ... I can think of about a half-dozen other ways to use Builder for free perpetually as well -- and, there are A LOT easier ways than reformatting the drive every 14 days ... should i list them? no, i don't want to give away any ideas. bottom line is that most people usually want to reward those who are helping them by compensating them for their services. if people can't afford the software legitimately, then i guess they'll find a work-a-round. i'm sure pursuing the probably small number of people using work-a-rounds isn't worth mike's time ... at least, for now.
Oh, let's forget about that for a moment as I would probably purchase a license
if I could see some proof that this is something more than random
curve-fitting.
you have 14 days, if that's not enough time to trial the software, then try one of your work-a-rounds to trial another 14 days ... read the forums to get ideas ... we all had/have the same questions as you.
The first question I have is this one: Is the program
supposed to converge to a specific system for a very large number of
generations and populations?
many of my strats converge if i run many multiple generations ... that's why i've opted to use larger populations sizes (as much as my RAM will allow) with only a few generations 3-7, at most ... that approach has been discussed in this forum by others
I'm asking this because I get different
systems for the most part and some pass out-of-sample tests but some
others do not. I noticed that for 100 generation and for 100
populations, the program generated different systems every time I had
it run my project.
that's because the initializing seed is random ... i'm sure that if you have a sufficiently large initial population, some strats will repeat ... at least i've noticed that
Some of the systems failed out-of-sample and some
systems passed.
ya, i'm also trying to determine which combination of metrics will result in more consistent 'real' out-of-sample results (as opposed to the one included in builder)
Selecting a system that passes the test introduces
selection bias and possibly data snooping bias if this is done after a
losing system was observed.
yup, that is an issue for all system developers to work out
mandelmus
Apr 29, 2012, 3:17:18 PM4/29/12
to adaptrad...@googlegroups.com
On Thursday, April 26, 2012 12:16:00 PM UTC-5, MikeBryant wrote:
Putting aside the ethical question of trying to defeat the purpose of
the trial, I suspect most of my customers purchase the program because
they find it valuable and plan to use it on an ongoing basis, not just
for 14 days. If you really feel that reformatting your PC every 14
days costs less than purchasing the program, there's probably not much
I can say about that.
lol
The genetic programming process involves an element of randomness by
design. It shouldn't and is not supposed to converge on the same
solution each time just because you use a large population with many
generations. That would defeat the nature of the discovery process and
limit the ability of the program to find good solutions.
As far as curve-fitting is concerned, you always have to be aware of
that possibility and take measures, such as those discussed in the
user's guide, for minimizing that risk. That includes utilizing the
built-in out-of-sample testing. In my experience, the risk of curve-
fitting is no greater with this approach than it is for the usual
trial-and-error approach that most traders employ then developing
trading strategies.
i agree
As for selection bias and data snooping bias, I
recently wrote a newsletter article (under Newsletter link on www.adaptrade.com)
that addresses those issues, among others.
ya, read his articles, they will help you
mandelmus
Apr 30, 2012, 1:03:18 AM4/30/12
to adaptrad...@googlegroups.com
On Thursday, April 26, 2012 1:16:22 PM UTC-5, DaveA wrote:
On Apr 26, 1:16 pm, MikeBryant <m...@breakoutfutures.com> wrote:
> Putting aside the ethical question of trying to defeat the purpose of
> the trial, I suspect most of my customers purchase the program because
> they find it valuable and plan to use it on an ongoing basis, not just
> for 14 days. If you really feel that reformatting your PC every 14
> days costs less than purchasing the program, there's probably not much
> I can say about that.
I guess my message did not go through. I don't have time reformatting
disks but thousands of young traders who have no money to buy your
software but all the time in the world can use it for free to develop
and post systems in collective2 and striker, just to mention two
examples.
you might have to rethink systems developed by builder ... the systems i've built by builder don't last that long ... maybe 3 weeks before they change course ... that's why i'm looking at combining multiple systems simultaneously rather than relying on a single system at a time. anyway, there ain't no guarantees ... look what happened with ETF Timer and Topaz NQ at Collective2 and what about Compass, and others, at Striker.
That is not good for your customers who paid $1K to get a
license. It isn't also fair to those that paid $1K when you start
selling this for $85/month. You see I'm trying to understand based on
your business model the true value of your product. One of your
competitors doesn't offer a demo and sells the program for about $60K.
there's plenty of discussion about Trading Systems Lab (TSL) in another thread on this forum, read on ... https://groups.google.com/forum/?fromgroups#!topic/adaptrade-builder/q4vdkouV2Hc
Either he is a snake oil salesman or you are doing something wrong
with your approach. I'm only trying to find you and I think my worries
are legitimate.
uuumm, read the discussion
Look, if you sell a program for $85, those that act in an illegal
manner will find it profitable to buy it, remove the password
protection and then sell it for $5 to thousands of people. Do you
think this is fair to the customers that have paid $1k or more to buy
a copy? Again, I'm trying to understand why you are offering the
program at such a low price and if this means that it doesn’t work.
for the time being, i guess we are just lucky ... give mike several versions more of improvements and see if he decides to start raising the prices. actually, i wouldn't mind if he switched to a monthly subscription-based approach, because, theoretically, that should encourage him to continue making improvements. for these one-time sales, i suppose he'll constantly need new buyers to pay the bills. unless he's earning money other ways.
Doji
Apr 30, 2012, 3:10:07 AM4/30/12
to adaptrad...@googlegroups.com
2012/4/30 mandelmus <gmb...@gmail.com>
On Thursday, April 26, 2012 1:16:22 PM UTC-5, DaveA wrote:you might have to rethink systems developed by builder ... the systems i've built by builder don't last that long ... maybe 3 weeks before they change course ... that's why i'm looking at combining multiple systems simultaneously rather than relying on a single system at a time. anyway, there ain't no guarantees ... look what happened with ETF Timer and Topaz NQ at Collective2 and what about Compass, and others, at Striker.
Sorry everyone, but I'm a bit confused at this time. I've been following discussions on Adaptrade for a while and had the feeling noone was able to find a working strategy, while most have to rebuild their strategies after very few weeks. How do you say a strategy may work just for a couple of weeks? Even if your strategy is based on a very short time frame, say 1 minute, the number of trades you can complete in a such little time is very low and not statistically significant.
Is a trading strategy working for just 3 weeks a real trading strategy? I guess it's not.
How you decide just after 3 weeks that your strategy is no longer valid and need to be changed? Valid trading systems can loose money for months and then come back gaining money.
Reply all
Reply to author
Forward
0 new messages