Writting fast compressors for Ram Drives
Earl_Colby_Pottinger
Presently working on the RAM-DISK code for Haiku-OS, this code
is to support a number of features to make it useful for modern
machines which already have fast CPUs, caching of present drives and
lots of free memory.
All these features in modern computers have removed the need for ram
drives for most users today.
One of the features I would like to add to my code is a background
compression that scans the data blocks and compresses them to free up
system memory (IE make it worthwhile to use a very large ram drive in
the few case where they still have a speed advantage).
For this reason I have been looking at what is available in very fast
compressor/decompressor systems. I already have a very fast RLE
compressor which gains speed over other versions I found thru Google
on the WWW, it gains speed with a small sacrifice of compression
efficiency.
What other compressors are known for *BOTH* fast compression and
decompression?
PS. I already have a SSD on my computer, transfer rates for it tops
out at 115MB/s while the RAM-DISK hits 800MB/s. I would like to keep
speeds above +600MB/s as the compression run as a background task
separate from the RAM-DRIVE itself, but the RAM-DISK needs to
decompress data blocks before it can transfer the data. Possible?
Thomas Richter
> What other compressors are known for *BOTH* fast compression and
> decompression?
LZO is frequently recommended for that on Linux. I personally cannot
comment on it since I haven't tried to measure.
Greetings,
Thomas
Earl_Colby_Pottinger
Thank you, as soon as I find a version I can compile I will give it a
try.
Does anyone else have any other suggestions for code that is fast both
ways.
Pedro Pereira
> What other compressors are known for *BOTH* fast compression and
> decompression?
The fastest ones I know are LZO and LZJB.
You can see a comparison between them at:
http://denisy.dyndns.org/lzo_vs_lzjb/
Since the LZJB was derived from LZRW1, you can also check it out:
http://www.ross.net/compression/lzrw1.html
Pedro Pereira
Earl_Colby_Pottinger
> The fastest ones I know are LZO and LZJB.
> You can see a comparison between them at:http://denisy.dyndns.org/lzo_vs_lzjb/
> Since the LZJB was derived from LZRW1, you can also check it out:http://www.ross.net/compression/lzrw1.html
> Pedro Pereira
Great, thanks for the links too.
Noob
> What other compressors are known for *BOTH* fast compression and
> decompression?
The following web site might be a good starting point.
Implementations ranked by decompression time:
http://www.maximumcompression.com/data/summary_mf4.php
Implementations ranked by compression time:
http://www.maximumcompression.com/data/summary_mf3.php
Lempel–Ziv–Oberhumer looks like a good fit.
http://en.wikipedia.org/wiki/Lempel–Ziv–Oberhumer
http://www.oberhumer.com/opensource/lzo/
Regards.
Niels Fröhling
> What other compressors are known for *BOTH* fast compression and
> decompression?
Take a look at LZP:
http://www.arturocampos.com/ac_lzp.html
http://www.cbloom.com/src/index_lz.html
You can tune the algorithm basically for a target speed, asymptotically
towards memcpy(). It is especially elegant through it's simplicity, and the
resulting ease for for example runtime scalability/tunability.
Charles cites 10MB/s for his specific implementation on a 200MHz PentiumPro.
Haven't run it for a long time.
Ciao
Niels
Earl_Colby_Pottinger
reading them.