Re: Comparison with another encoders

[James Cloos]

>>>>> "TR" == Thomas Richter <ric...@rus.uni-stuttgart.de> writes:

TR> the JPEG committee (ISO SC29WG1) provides plenty of example images,
TR> most of which we use for internal testing. If interested, I could
TR> provide them.

Providing them via a clone-able repo would be useful, provided the
license permits.

If they are already on a public site, the mozjpeg repo should include
the uri in its docs.

-JimC
--
James Cloos <cl...@jhcloos.com> OpenPGP: 1024D/ED7DAEA6

[Thomas Richter]

Am 13.03.2014 08:00, schrieb James Cloos:
>>>>>> "TR" == Thomas Richter<ric...@rus.uni-stuttgart.de> writes:
>
> TR> the JPEG committee (ISO SC29WG1) provides plenty of example images,
> TR> most of which we use for internal testing. If interested, I could
> TR> provide them.
>
> Providing them via a clone-able repo would be useful, provided the
> license permits.
>
> If they are already on a public site, the mozjpeg repo should include
> the uri in its docs.

We do have full image releases, thus images can be used for scientific
research and publication provided the author is quoted (basically, a CC
type of license). I don't have a publicly reachable repository, they are
at the JPEG sftp server which is not reachable for the general audience.
If interested, I could create a zip archive and transmit it via "fex",
which will provide a download location on a publicly reachable web site.
I would need an email address of a receiver who would feed responsible
to put them into a repository.


Greetings,
Thomas

[James Cloos]

>>>>> "TR" == Thomas Richter <ric...@rus.uni-stuttgart.de> writes:

TR> We do have full image releases, thus images can be used for scientific
TR> research and publication provided the author is quoted (basically, a
TR> CC type of license). I don't have a publicly reachable repository,
TR> they are at the JPEG sftp server which is not reachable for the
TR> general audience. If interested, I could create a zip archive and
TR> transmit it via "fex",

How large is it? I can provide a direct destination for an upload, and
push it on from there, easier than dealing with "fex".

[Thomas Richter]

Am 13.03.2014 20:39, schrieb Jerzy Głowacki:
> Dear Thomas,
>
> Thank you for your reply. However, currently I don't have time for
> making tests. I hoped somebody had already done them and could share his
> results.
>
> If I get some time to do this in the future, I will write to you for a
> test image corpus.

It depends on what you need. I'm certainly not the right man for unit
tests for your software because you know your code much better than I
do. Besides, I've enough to do to maintain a JPEG verification software
implementation myself (from the JPEG, implementing JPEG and JPEG XT and
JPEG LS, see https://github.com/thorfdbg/libjpeg for an old version of it).

However, if it is measuring PSNR, SSIM, VDP and other objective indices
on your images, then you're at the right address. I'm operating as the
test and quality co-chair of our committee, thus, all the test tools are
setup here in Stuttgart, and adding just another implementation to
deliver some curves is the least problem. If interested, I could either
provide measurement tools and bash shell scripts for automatic
measurement (to be customized for your needs at your side) or run some
automated tests for objective quality indices when time permits. There
is also a (very cut down) online version of it so you could trigger
comparisons yourself:

http://jpegonline.rus.uni-stuttgart.de/

Adding another codec there would also be possible, though the tests
there are rather a demo than a full test - though one you could trigger
yourself. It currently does not cover VDP, though (VDP is "a bit" on the
slow side).

If you are after *subjective* tests, then while I cannot do that myself
lacking a lab, I'm in contact with people that could. This is QUALINET,
an "EC Cost action" (funded by the European community). Maybe I can
interest them for something.

This is currently all an activity that happens in the JPEG XT framework,
which is also a backwards compatible extension to JPEG (ISO/IEC 10918-1
aka ITU Rec. T.81), though for higher bit depths, high dynamic range and
lossless coding. Smuggling another codec in is probably not too hard for
me as T&Q co-chair. Thus the offer... ...and probably to also keep you a
bit in the line what "the ISO guys are doing recently".

Greetings,

Thomas

[Thomas Richter]

Am 13.03.2014 17:44, schrieb James Cloos:
>>>>>> "TR" == Thomas Richter<ric...@rus.uni-stuttgart.de> writes:
>
> TR> We do have full image releases, thus images can be used for scientific
> TR> research and publication provided the author is quoted (basically, a
> TR> CC type of license). I don't have a publicly reachable repository,
> TR> they are at the JPEG sftp server which is not reachable for the
> TR> general audience. If interested, I could create a zip archive and
> TR> transmit it via "fex",
>
> How large is it? I can provide a direct destination for an upload, and
> push it on from there, easier than dealing with "fex".

Sorry for taking a bit longer, I had to check first and I was teaching.
The subset of the 8bit/channel images where I have releases for is about
1GB as a zip file. There are probably a couple of duplicates, but the
order of magnitude should be about correct. It depends a bit which type
of images you are after - the above images have *natural*
(non-synthetic) sources and are thus typical sources for JPEG compression.

Greetings,
Thomas

[Thomas Richter]

Dear James, dear Jerzy,

test images are coming. Please allow me a couple of minutes to upload
them, it will take a while due to the size.

You'll find the following on your server (once done):

honolulu.zip:
This is the "honolulu test set", a couple of structurally challenging
images. They easily loose structure due to compression and have been
found useful for testing. The images are copyright Olivier Pfeiffer,
available under CC-nc-by license. A release is inside.

hdcompare.zip:
This is the HDPhoto test image set, provided by Microsoft. It contains a
couple of interesting stress images for compression purposes. A formal
release is inside. Please note that I believe that MS probably optimized
HDPhoto aka JPEG XR a bit too much for this test set as it performs a
bit weaker on other image sets.

richard.zip:
This is a small test set of images of higher dynamic range provided by
Richard Clark (former web master of SC29 WG1).

core1.zip:
This is a subset of the above image sets, namely the "AIC core 1 test
image set" that has been selected for subjective evaluation. This test
image set contains images that are easily degraded by compression and
thus more critical than other images. It is a subset from the Honolulu
set and the HDPhoto set. Originally, this set contained additional
images, but unfortunately, I do not have (or no longer have) releases
for them. The test set originally contained three images from the
JPEG-LS test set ("boat", "cafe" and "woman"), and one image of a
painting by the London Gallery, for which I only have a release for
JPEG internal activities. However, since I consider this an interesting
image, I could make contacts if interested.

There are quite a number of more images I have, though at this time it
would take a while to hunt down the corresponding image releases, thus I
better do not upload them.

All images I uploaded are in ppm, sRGB color space. Note that JPEG
actually uses ITU Rec. 601, thus some mild color variations might be
observed when using JPEG correctly (which, actually, nobody does).

For all of the above images, I do have the "raw" versions available, for
some of them also 16-bit scRGB versions (a wider and more precise
colorspace).

For starters, I would probaby suggest to go for core1 first.

Just a final note: If you want to do proper testing, I suggest *not* to
use the "average" test images, such as lena, barbara or the Kodak test
image set. First, these images are outdated, high noise and
unrealistically small for most practical applications. Second, as far as
the Kodak test set is concerned, the legal situation is not so obvious
and image usage is more constraint than one might wish for.

Greetings,
Thomas

[James Cloos]

TR> test images are coming

The images are up at:

https://github.com/jhcloos/test_images

The clone is just under one Gig; the tar.gz (see the releases tab) is
just over one Gig.

TR> For all of the above images, I do have the "raw" versions available,
TR> for some of them also 16-bit scRGB versions

Those are also welcome. I'll leave the upload server online for them.