compressing data files

[Klaus]

Hi, 

JBookTrader is great, its ASCII-based format makes things easy to read and write, but

it also takes a lot of space. This can be annoying, both for copy operations as well 

as as part of backups and such. It is even more the case, if one uses different assemblies

of files as a basis for different test and optimization operations. 

Currently my marketdata directory has more than 5 GB. 

If one compresses the data files this leads to a 5:1 - 6:1 ratio in my experience. Thus, I decided

to provide a zip archive facility. 

The following code modifies the Backtestfilereader (and thus both backtest and optimization)

by introducing the possibility: if a zip-file is given, it opens it, checks for the same filename without

the zip and reads this, otherwise it simply tries to open the file in the old way. 

Thus, there needs in each zip-file to be exactly one file and the zip-file-name needs to be 

name.txt.zip with a containing file name.txt. This will then be read. 

As previous proposals for enclosing in the distribution were not accepted, I just post the code here. 

If someone wants to modify the setup just replace the existing method in the Backtestfilereader 

with this method.

If someone wants: there is certainly room for improvement.

Have fun

   Klaus

    public BackTestFileReader(String fileName, MarketSnapshotFilter filter) throws JBookTraderException {

        this.filter = filter;

        previousDateTimeWithoutSeconds = "";

        String zipFileName;

        BufferedReader myReader = null;

        //  KS: Es wird erst überprüft ob der Name mit ZIP endet, wenn ja, dann wird darin nach

        //  KS: einer Datei ohne diesen Suffix gesucht und diese geöffnet, sonst wird die bisherige Strategie 

        //  KS: beibehalten. 

        if (fileName.toLowerCase().endsWith(".zip")) {                

        // remove zip-suffix

        zipFileName = fileName;

        fileName = fileName.substring(0, fileName.length()-4);

       

        // Try  to determine whether we got a zip-file that can be used.

        try {

        ZipFile zf = new ZipFile( zipFileName ); // we assume there is exactly one entry

        ZipEntry entry = zf.entries().nextElement(); 

        myReader = new BufferedReader(new InputStreamReader(zf.getInputStream(entry)));

        } catch (FileNotFoundException fnfe) {

        throw new JBookTraderException("Could not find file " + fileName);

        } catch (IOException e) {

        throw new JBookTraderException("Could not find file " + fileName);

        } catch (Exception e) {

        throw new JBookTraderException("General Error " + fileName);

        }

        } else {

        try {

        myReader = new BufferedReader(new InputStreamReader(new FileInputStream(fileName)));

        } catch (FileNotFoundException fnfe) {

        throw new JBookTraderException("Could not find file " + fileName);

        }

        }

        reader = myReader;

        fileSize = new File(fileName).length();  

    }