Updated results after adding snappy-java

[Tatu Saloranta]

As some of you may have noticed, I managed to add snappy-java codec to
tests (currently only in block mode, but when jni loading issues are
resolved, also streaming mode), and re-ran some of the tests. There
were even some nice improvements to java-snappy library itself, so
results are pretty good.
In fact, snappy-java is clearly the fastest codec to decompress (2x as
fast as pure java LZF); and as fast as fastest for compression.
Snappy looks like basic lempel-ziv variant, so its compression ratio
is about the same as lzf, quicklz (or, I assume, fastlz).

-+ Tatu +-

[Taro L. Saito]

Hi all,

I am the developer of snappy-java.

Thanks to the great work by Tatu, I could quickly improve the
snappy-java's performance.
I reconfirmed that benchmark is an essential tool for iterative development.

I added a link to the benchmark result from the following page:
http://code.google.com/p/snappy-java/

Thanks again.
I am glad to be able to release snappy-java in a week.

--
Taro L. Saito
<l...@xerial.org>
University of Tokyo
http://www.xerial.org/leo
Tel. +81-47-136-3985 (63985)

[Tatu Saloranta]

On Mon, Apr 4, 2011 at 7:05 PM, Taro L. Saito <l...@xerial.org> wrote:
> Hi all,
>
> I am the developer of snappy-java.
>
> Thanks to the great work by Tatu, I could quickly improve the
> snappy-java's performance.
> I reconfirmed that benchmark is an essential tool for iterative development.

This is great -- I had similar experiences with "jvm-serializers"
(https://github.com/eishay/jvm-serializers) benchmark; it helped many
libraries to improve performance, and was big reason why I thought I
create a simple benchmark for compression libs too.

> I added a link to the benchmark result from the following page:
> http://code.google.com/p/snappy-java/
>
> Thanks again.
> I am glad to be able to release snappy-java in a week.

Good!

I hope this could also help other libraries that make different
tradeoffs; like bzip2 (very slow currently, but with good
compressipn), or lzma (if we can figure out a good way to work around
API issues), which should have bzip2-level compression but be faster
to decompress.

-+ Tatu +-

[Taro L. Saito]

As for lzma (7zip), I once tried to port it into Java's
InputStream/OutputStream coding style.
But the needed fixes are numerous and I ended up giving up to do so,
because I figured out
that it makes difficult to catch up with the latest version of lzma.

I found another person doing a similar thing.
How about using https://github.com/league/lzmajio, which provides
LzmaInputStream and LzmaOutputStream?

--
Taro L. Saito
<l...@xerial.org>
University of Tokyo
http://www.xerial.org/leo
Tel. +81-47-136-3985 (63985)

On Tue, Apr 5, 2011 at 11:11 AM, Tatu Saloranta <tsalo...@gmail.com> wrote:
> lzma

[Tatu Saloranta]

On Mon, Apr 4, 2011 at 7:48 PM, Taro L. Saito <l...@xerial.org> wrote:
> As for lzma (7zip), I once tried to port it into Java's
> InputStream/OutputStream coding style.
> But the needed fixes are numerous and I ended up giving up to do so,
> because I figured out
> that it makes difficult to catch up with the latest version of lzma.

This was my experience as well - - starting to read through code, it
was quite a lot, much more than I would have time for on short term.

> I found another person doing a similar thing.
> How about using https://github.com/league/lzmajio, which provides
> LzmaInputStream and LzmaOutputStream?

I found this as well: it does do it, but does this by running another
thread, which does work I guess.
I just wish it could be done differently. Although maybe using that
for testing first would work.

Another simple option would be to just implement blocking mode by byte
array input and output streams.
Maybe the two could be combined.

-+ Tatu +-