Planning to write a stock market application in BASIC
Ali Chambers
I'm planning to write a program in basic that processes stock market
price data. It will be using floating points arrays in the following
manner:
- Price data is held in floating point currency arrays, eg:-
highprice(0-2000 max)
- Comparisons are done between highprice() arrays and lowprice() arrays
- Buy levels are determined and these buy points held in arrays
- The program will then loop through the price arrays incrementally to
find sell prices - at which it sells and records the price (a basic
description!)
Different levels for buy & selling are to be tested (too complicated
for this email). On average, the buy/sell routine will be called 3
million times per stock.
I did a "mock up" in VB.NET and it took 15 hours to run! I expected the
app to take a while, but this is too long. I have also optimised the
alogarithm as much as possible.
I've been reading the newsgroups and I can see various choices:
- Powerbasic
- Freebasic
I need *RAW SPEED*. I'm not too bothered about the rest. Oh - and not
too complicated to write (not C++), as I'm a novice programmer doing it
part-time.
Would Powerbasic be best? Is it really that fast?
Thanks, alex
Michael Mattias
news:1155933048....@p79g2000cwp.googlegroups.com...
> Hi,
>
> I'm planning to write a program in basic that processes stock market
> price data. It will be using floating points arrays in the following
> manner:
>
> - Price data is held in floating point currency arrays, eg:-
> highprice(0-2000 max)
>
> for this email). On average, the buy/sell routine will be called 3
> million times per stock.
>
> I did a "mock up" in VB.NET and it took 15 hours to run! I expected the
> app to take a while, but this is too long. I have also optimised the
> alogarithm as much as possible.
While another compiler may help, I'd take a serious look at my data access
algorithm. Maybe at my equations, too: if you are using a trial-and-error
solution (i.e, try various values one after the other and see how that turns
out, saving the optimal), I think maybe some differential equations would
help. (I haven't touched those in 30 years, but I know they can be used to
find min and max)
I cannot see 15 hours just to do a couple million math things.(*)
Can you pseudo-code your current program structure and post it? That should
be enough to pinpoint what's taking so long.
(Must have approx number of stocks, too).
MCM
* Although it is true that I once took a VB program which was taking an hour
to do something and cut that down to eleven seconds using another compiler,
I started with a really inferior design so it really wasn't that big a
challenge, nor is it fair to ascribe all the performance improvement to the
compiler. This story was published in Basically Speaking magazine about
eight-ten years ago. I can get you a reprint if interested.
ne...@rtrussell.co.uk
> - Price data is held in floating point currency arrays, eg:-
> highprice(0-2000 max)
Don't store currency data in floating point! Because values like 0.01
cannot be represented exactly, you will accumulate errors that
eventually will exceed one cent (penny etc.) and then your accounts
won't balance!
The best way to handle monetary values is as integer numbers of cents,
pence etc. You can divide by 100 for display, but errors will never
accumulate. It is also likely to speed up your program somewhat.
Richard.
http://www.rtrussell.co.uk/
To reply by email change 'news' to my forename.
Ali Chambers
Thanks for your replies. My data algorithm started out as
trial-and-error, but it's now a variation of this:
for a = 4 to 20
for b = 0 to 20
for c = 0 to 3
for d = 0 to 5
for e = 0 to 5
for f = 0 to 20
for g = 0 to 5
for h = 0 to 1
for i = 0 to 7
for j = 0 to 5
for k = 0 to 3
for l = 0 to 10
Call Sub FindBuyPrice()
Call Sub FindSellPrice()
AddUpAllStats()
(all nexts)
Then with the highest % profit from this 1st pass, it does another
similar second pass to see if values from the 1st pass can be bettered.
The 1st pass does have a lot of for...next loops. The values represent
different items to test from the price data (beyond this email). I just
multiplied them all together and got 2,520,000,000! Hmmmmm.....
I have seperated as much as possible conditions to be tested into
1st,2nd and 3rd passes (rather than just having them all in the 1st
pass - ie. up to "for n = 0 to ....."), but the application is still
taking a long time.
I'm using "double" type floating point numbers (VB.NET) for prices.
Is there an easier way to find max profit values?
Thanks,
Alex
Vic Drastik
"Michael Mattias" <michael...@gte.net> wrote in message
news:GgrFg.2039$q63....@newssvr13.news.prodigy.com...
> "Ali Chambers" <in...@alexchambers.co.uk> wrote in message
> news:1155933048....@p79g2000cwp.googlegroups.com...
> > Hi,
> >
> > I'm planning to write a program in basic that processes stock market
> > price data. It will be using floating points arrays in the following
> > manner:
> >
> > - Price data is held in floating point currency arrays, eg:-
> > highprice(0-2000 max)
> >
> > Different levels for buy & selling are to be tested (too complicated
> > for this email). On average, the buy/sell routine will be called 3
> > million times per stock.
> >
> > I did a "mock up" in VB.NET and it took 15 hours to run! I expected the
> > app to take a while, but this is too long. I have also optimised the
> > alogarithm as much as possible.
>
> While another compiler may help, I'd take a serious look at my data access
> algorithm. Maybe at my equations, too: if you are using a trial-and-error
> solution (i.e, try various values one after the other and see how that
turns
> out, saving the optimal), I think maybe some differential equations would
> help. (I haven't touched those in 30 years, but I know they can be used to
> find min and max)
Absolutely! A faster compiler may get you a doubling or tripling of speed,
but to get major speedups, you need to study the problem more, and try to
find an insight that will allow you to develop a better algorithm, not just
a slightly faster version of the current algorithm. (Example: for sorting,
use Combsort or Heapsort, not Insertion Sort or Bubble Sort)
If this is not possible, profile your current code and find which parts are
using the most time. The Pareto Principle, otherwise known as the 80/20
rule, says that 80% of the time is spent in 20% of the code (although my
experience is more like 90/10 or even more). Polishing that 20% will yield
much more improvement than working on all the code.
Vic
Steve Foley
> Hi,
>
> Thanks for your replies. My data algorithm started out as
> trial-and-error, but it's now a variation of this:
>
It still looks like trial-and-error. Try every possible combination and see
what is best - no optimization.
> for a = 4 to 20
> for b = 0 to 20
> for c = 0 to 3
> for d = 0 to 5
> for e = 0 to 5
> for f = 0 to 20
> for g = 0 to 5
> for h = 0 to 1
> for i = 0 to 7
> for j = 0 to 5
> for k = 0 to 3
> for l = 0 to 10
>
> Call Sub FindBuyPrice()
> Call Sub FindSellPrice()
> AddUpAllStats()
>
> (all nexts)
>
> Then with the highest % profit from this 1st pass, it does another
> similar second pass to see if values from the 1st pass can be bettered.
>
> The 1st pass does have a lot of for...next loops. The values represent
> different items to test from the price data (beyond this email). I just
> multiplied them all together and got 2,520,000,000! Hmmmmm.....
I got 27,360,571,394 (for c = 0 to 4 iterates 4 times)
> I have seperated as much as possible conditions to be tested into
> 1st,2nd and 3rd passes (rather than just having them all in the 1st
> pass - ie. up to "for n = 0 to ....."), but the application is still
> taking a long time.
>
> I'm using "double" type floating point numbers (VB.NET) for prices.
I still think you would save time using integers
Michael Mattias
news:1155983827.4...@75g2000cwc.googlegroups.com...
>
> Thanks for your replies. My data algorithm started out as
> trial-and-error, but it's now a variation of this:
>
> for a = 4 to 20
> for b = 0 to 20
> for c = 0 to 3
> for d = 0 to 5
> for e = 0 to 5
Not to burst your bubble, but it certainly appears you are trying everything
possible and keeping track of which set of values produces the best result.
This is "trial and error" as I meant it in context.
> multiplied them all together and got 2,520,000,000 [FOR NEXT loops]
Hmmmmm.....
Hmm, indeed. 15 hours doesn't sound so unreasonable anymore, and we haven't
even considered any inefficiencies in the called procedures.
> Is there an easier way to find max profit values?
I'm sure there is, but.... but I assume you are not in the mood to disclose
the precise formulas you are using, since that would kind of kill any market
for your stock market program; or if not reselling, disclose your strategies
to your competitors - and neither scenario sounds terribly appealing.
Methinks you need to engage (that means pay and sign to a non-disclosure
agreement) a math whiz who can help you reduce the iterative nature of your
solution. I'm still pretty sure what you need is differential equations,
although the more I think about it, the need might more accurately be stated
as the integral calculus required to actually SOLVE those differential
equations.
MCM
(Who no longer remembers squat re "DIFF-E-Q" except that's what we called
it in college)
Harry Chickpea
>Thanks for your replies. My data algorithm started out as
>trial-and-error, but it's now a variation of this:
>
>for a = 4 to 20
>for b = 0 to 20
>for c = 0 to 3
>for d = 0 to 5
>for e = 0 to 5
>for f = 0 to 20
>for g = 0 to 5
>for h = 0 to 1
>for i = 0 to 7
>for j = 0 to 5
>for k = 0 to 3
>for l = 0 to 10
>
>Call Sub FindBuyPrice()
>Call Sub FindSellPrice()
>AddUpAllStats()
>
>(all nexts)
>
>Then with the highest % profit from this 1st pass, it does another
>similar second pass to see if values from the 1st pass can be bettered.
You are doing a brute force approach.
The short answer is that you need to do some elimination of groups of
results that are outside of a range.
Example -
If StockPrice > YesterdayStockPrice Then
DoStuff
Else
EliminateStockFromLaterLoops
End if
Stephen Rush
Get FreeBASIC or Powerbasic, multiply each share price by 10000 so you can
use 32-bit integers. Floating-point arithmetic has legitimate uses, but
enough chained operations on the same data can produce results that are
mostly noise. I'm not familiar with the problem, but if you need twelve
nested loops, you are probably doing something wrong. If you absolutely
need twelve levels, do they all have to be classical FOR loops? Could you
use WHILE, UNTIL, CONTINUE or EXIT FOR to do short-circuit evaluation at
some stages? If you really can't find a better algorithm, consider
getting another processor or four to divide the load. They don't
have to be latest-and-greatest. I heard of a guy who lucked into
an office that was scrapping a dozen Pentium III machines just because
they weren't new enough (one of those "use it or lose it" budget
SNAFUs, I guess). They actually paid him to haul the boxes away. Leasing
supercomputer time is a possible option, but you would probably spend less
on a bunch of commodity PCs and a network switch, even if you pay retail.
Vic Drastik
"Ali Chambers" <in...@alexchambers.co.uk> wrote in message
news:1155983827.4...@75g2000cwc.googlegroups.com...
Probably several, but the one that leaps to mind is a simplex method of
multivariate function minimisation. Assuming that the function is not too
discontinuous, (that is, the value for a = 16 is not too far from that of
a=15 or a=17), the Nelder-Mead method (also called the downhill simplex
method - google the name) should give you a reasonable approximation to the
answer. Also, it should run in seconds, not hours. (in fact, since 27
billion function values took 15*3600 = 54000 seconds, 500 function values
would take about 1 millisecond, and I would be surprised if the N-M
algorithm took more than 500,000 function evaluations, so 1 second would be
about right.)
Vic
Ali Chambers
Thanks for all the suggestions. I'm going to have another look at the
alogarithm and use integers instead of FP.
Cheers,
Alex
Pavel314
news:1155983827.4...@75g2000cwc.googlegroups.com...
>
> Thanks for your replies. My data algorithm started out as
> trial-and-error, but it's now a variation of this:
>
> for a = 4 to 20
> for b = 0 to 20
> for c = 0 to 3
> for d = 0 to 5
> for e = 0 to 5
> for f = 0 to 20
> for g = 0 to 5
> for h = 0 to 1
> for i = 0 to 7
> for j = 0 to 5
> for k = 0 to 3
> for l = 0 to 10
>
> Call Sub FindBuyPrice()
> Call Sub FindSellPrice()
> AddUpAllStats()
>
> (all nexts)
>
> Then with the highest % profit from this 1st pass, it does another
> similar second pass to see if values from the 1st pass can be bettered.
Have you considered using a language like SAS? I'm at a fairly low level of
familiarity with SAS, just enough to know there are a lot of things it can
do that are beyond my level of expertise. It is good for working with large
files of data, however. Try posting to the comp.soft-sys.sas newsgroup for
help, although if you decide to use it, you'll probably save time by hiring
a SAS programmer.
Paul