Historical tick data and feed adapters

[Joel Reymont]

Folks,

I need as much historical tick data as possible. I would like to put a
heavy load on the system and see if performance is suitable for
backtesting over a couple of years.

I'm also going to create a data feed adapter for IB since the Java
source code is available and I should have a test account for BIS.de
which is apparently connected to all exchanges. If you have any other
feeds in mind please let me know.

Thanks, Joel

[PrgTrdr]

Hi Joel:

You might want to think about separating each symbol
into a separate file instead. I've had some success with the
following directory structure:

/TickDB/20050716/IBM.DAT
/TickDB/20050716/DELL.DAT
.
.
.
/TickDB/20050717/IBM.DAT
/TickDB/20050717/DELL.DAT

This way you can get to all the files for a particular day
pretty easily.

Bill Harts

[Joel Reymont]

Bill,

I'm already placing each day's data for each symbol into separate
files. I don't need to have a directory structure, though, as a header
with the symbol and date is placed into each file.

Thanks, Joel

[Bill Harts]

Using the symbol as a file name and dated directory structure eliminates the
need to query a separate database or file to get directly to the data you
want.

[Joel Reymont]

Bill,

I agree with you but it's not particularly efficient from a
programming prospective. Imagine doing a range query to retrieve data
between DataTime1 and DateTime2. Try accounting for missing data.

Joel

On Jul 18, 2005, at 10:47 AM, Bill Harts wrote:

>
> Using the symbol as a file name and dated directory structure
> eliminates the
> need to query a separate database or file to get directly to the
> data you
> want.

--

http://wagerlabs.com/uptick