Thanks,
David
Thanks, Ed.
Yes, colour-management could cause a difference, as could honouring or not
honouring and embedded colour profile or gamma correction.
But that's not actually what we are seeing - it's more like interlace
lines, and happening with some very well-known software. Hence my
question:
Is there only one mathematically correct decoding of a JPEG?
David
Yes.
--
That was it. Done. The faulty Monk was turned out into the desert where it
could believe what it liked, including the idea that it had been hard done
by. It was allowed to keep its horse, since horses were so cheap to make.
-- Douglas Adams in Dirk Gently's Holistic Detective Agency
No. From the JPEG FAQ:
"Another important aspect of JPEG is that decoders can trade off
decoding speed against image quality, by using fast but inaccurate
approximations to the required calculations. Some viewers obtain
remarkable speedups in this way."
Plus, given the complexities of the JPEG format, I'd expect decoding
libraries to do things in slightly different orders. That would
introduce rounding differences in the real number maths.
Correct.
David
Thanks for that, Derek. Although what it says is that there is one
correct way, but decoders may not use that route because it's slower than
some simplifications.
Your point about rounding is a good one, however the errors we are seeing
are much worse than that - naked eye visible!
What about chroma sub-sampling? Doesn't that imply an upsampling of the
chroma to produce a full resolution RGB image? Might upsampling methods
differ? However, even if there isn't one mathematically correct answer
(about upsampling), perhaps it could be said that any program which
introduces artefacts is one to be avoided?
This is what I'm talking about:
http://www.david-taylor.myby.co.uk/Decoded-JPEG-samples.png
If you compare, say, the two left-hand images, look how the decoding
differs on the sharpest blob between the Windows Picture and FAX Viewer
and Paint Shop Pro 9. Looking at the blob above the sharpest, see how
there is a sort of "interlace" structure visible on the PSP-9 decode but
not on the P&F Viewer. I'm not at the moment saying one is right and one
is wrong - just asking if there is a single "correct" decode. The
original JPEG is here:
http://www.david-taylor.myby.co.uk/Image1_2x2.jpg
(This was originally from Bart Hickman on the Corel Paint Shop Pro X
newsgroup).
David
It does? I must be missing that bit. :o}
> What about chroma sub-sampling? Doesn't that imply an upsampling of the
> chroma to produce a full resolution RGB image? Might upsampling methods
> differ?
My understanding is that it doesn't upsample (although it's been a while
since I looked at this stuff so I could be wrong!). The full resolution
image is stored in the JPEG file. Quantisation is used to reduce the
high frequency brightness variation, which results in image data that
responds very well to lossless RLE encoding. What ends up in the JPEG
file is data for every pixel, so there's no upsampling required.
In theory the creation of an image from the JPEG data should indeed be a
deterministic mathematical exercise, but as you've found out, it clearly
isn't. Differences might be down to short cuts, optimisations, rounding
errors or something else we haven't thought of. If there is a "correct
way" as you say in the first para quoted above, then presumably there's
a reference implementation somewhere. But I think that's irrelevant, to
be honest. This *is* something that gets done differently by different
software so you have to run with that fact.
Maybe it's my words - I meant that "not all decoders will use the lowest
and most accurate route".
>> What about chroma sub-sampling? Doesn't that imply an upsampling of
>> the chroma to produce a full resolution RGB image? Might upsampling
>> methods differ?
>
> My understanding is that it doesn't upsample (although it's been a
> while since I looked at this stuff so I could be wrong!). The full
> resolution image is stored in the JPEG file. Quantisation is used to
> reduce the high frequency brightness variation, which results in
> image data that responds very well to lossless RLE encoding. What
> ends up in the JPEG file is data for every pixel, so there's no
> upsampling required.
> In theory the creation of an image from the JPEG data should indeed
> be a deterministic mathematical exercise, but as you've found out, it
> clearly isn't. Differences might be down to short cuts,
> optimisations, rounding errors or something else we haven't thought
> of. If there is a "correct way" as you say in the first para quoted
> above, then presumably there's a reference implementation somewhere.
> But I think that's irrelevant, to be honest. This *is* something that
> gets done differently by different software so you have to run with
> that fact.
We're working on it right now - maybe a bug report to at least one vendor.
Many thanks for your comments - I still want to know about chroma
sub-sampling and any upsampling implications, as I don't think a
full-resolution YCrCb image is necessarily stored in the file.
Cheers,
David
I started playing with some of those parameters saving JPGs in Paint
Shop Pro. I quickly got to some image files that open fine under
PSP and other "fully capable" image software, but which my
correspondents using MS Windows native tools (I am unsure exactly
what) could not view. I guess that not all software implements the
more infrequently used parts of the JPG spec.
Chroma sub-sampling is done during encoding. Well-written decoders
should respect it at display time, because it is critical for
certain types of image content, especially sharp objects against
continuous-tone backgrounds, and (oddly enough) B&W images.
Is there any agreement among JPEG mavens as to which decoders
produce the best output?
There seems to be agreement that the IJG libraries produce the
tightest (tradeoff of smallest for highest quality) JPEG enodings,
although Photoshop produces better-looking low-quality output.
> We are seeing some differences in the resulting image when different
> programs decode the same JPEG file. Should this be the case, or is there
> only one mathematically correct way to decode a given JPEG image?
In an ideal world yes. But in the real world it is a little different.
JPEG decoders make a some approximations in the pseudo inverse. These
may or may not be valid for real images (although they are more usually
violated by line art or pathological test cases).
> Are
> there any choices which can be made when decoding a JPEG image, which can
> affect the RGB values obtained?
Yes. There are several variations even in the mathematically strict
decoders related to exactly how rounding errors propogate in the decoder
inverse transform (generally optimised primarily for speed).
You can do slightly better if you have a decoder optimised for accuracy
and in some specific circumstances it is worthwhile. There is also a
curious issue for digital camera JPEGs that have been losslessly rotated
where some decoders do not handle the transposed colour subsampling.
All these effects are pretty minor in most cases. In a handful of cases
you can do better with dedicated high dynamic range inverses.
Regards,
Martin Brown
> This is what I'm talking about:
>
> http://www.david-taylor.myby.co.uk/Decoded-JPEG-samples.png
>
> If you compare, say, the two left-hand images, look how the decoding
> differs on the sharpest blob between the Windows Picture and FAX Viewer
> and Paint Shop Pro 9. Looking at the blob above the sharpest, see how
> there is a sort of "interlace" structure visible on the PSP-9 decode but
> not on the P&F Viewer. I'm not at the moment saying one is right and one
> is wrong - just asking if there is a single "correct" decode.
That looks like chroma upsampling done "wrong".
> The original JPEG is here:
>
> http://www.david-taylor.myby.co.uk/Image1_2x2.jpg
Yep, it has subsampled chroma.
--
Måns Rullgård
m...@inprovide.com
Thanks, Måns. Whilst I appreciate the subtleties to which other posters
have referred, this isn't a subtle error - it's plain wrong!
David
Good question.
Given the decoding algorithms, I'd say Yes (if implemented correctly).
However, encoding (in particular chroma sub-sampling) may differ
depending on Quality setting.
Bart
What about chroma subsampling then, in YCrYCb? Isn't there an issue about
how the full resolution Cr and Cb should be reconstructed in the decoder,
or is a 50% average assumed to be correct?
David
Chroma subsampling can be a bit tricky to deal with because of the
relative positions of the luma and chroma samples. If the chroma
samples are situated halfway between the luma samples, as is often the
case, and the decoder incorrectly assumes they are co-sited (because
this is easier), odd errors may be seen. The upsampling method
(linear, cubic, etc) also matters.
--
Måns Rullgård
m...@inprovide.com
> Derek Fountain wrote:
>
>>>But that's not actually what we are seeing - it's more like interlace
>>>lines, and happening with some very well-known software. Hence my
>>>question:
>>>
>>> Is there only one mathematically correct decoding of a JPEG?
>>
>>No. From the JPEG FAQ:
>>
>>"Another important aspect of JPEG is that decoders can trade off
>>decoding speed against image quality, by using fast but inaccurate
>>approximations to the required calculations. Some viewers obtain
>>remarkable speedups in this way."
>>
>>Plus, given the complexities of the JPEG format, I'd expect decoding
>>libraries to do things in slightly different orders. That would
>>introduce rounding differences in the real number maths.
>
> Thanks for that, Derek. Although what it says is that there is one
> correct way, but decoders may not use that route because it's slower than
> some simplifications.
Perhaps more importantly the mathematically correct way is too hard. The
perfect decoder should give you an image that when recompressed with the
same quantisation table coefficients gives you exactly the same byte for
byte JPEG encoding. In practice the real world decoders compromise to
give you an answer close to but not always exactly equal to this ideal.
> Your point about rounding is a good one, however the errors we are seeing
> are much worse than that - naked eye visible!
Only when you zoom in and on an image that is 2x2 chroma subsampled.
Some of the reconstructed pixels on the sharpest boundaries are
inconsistent with the data (but are decoded in a manner consistent with
the original JPEG spec). The problem here is that JPEG was never
intended to cope with coloured line artwork and can perform quite badly
on high contrast pixel wide features on a plain background.
If you must encode strongly coloured line art like this as a JPEG use
1x1 chroma subsampling and all will be well - the file may even be
smaller too.
> What about chroma sub-sampling? Doesn't that imply an upsampling of the
> chroma to produce a full resolution RGB image? Might upsampling methods
> differ? However, even if there isn't one mathematically correct answer
> (about upsampling), perhaps it could be said that any program which
> introduces artefacts is one to be avoided?
>
> This is what I'm talking about:
>
> http://www.david-taylor.myby.co.uk/Decoded-JPEG-samples.png
I had a go with my dedicated JPEG decoder that attempts to reconstruct a
solution much closer to the ideal than the standard inverse. It looks to
me like there is a one pixel black boundary line around the sharpest
blob. This will distress most normal JPEG decoders immensely - ringing
and contrast loss will result. Mine gets something close to but not
exactly equal to what I believe was your most likely target image.
Different programs offer different upsampling on the chroma information.
Photoshop uses pixel replication (or rather the version I have does)
whereas some others interpolate from nearest neighbours.
>
> If you compare, say, the two left-hand images, look how the decoding
> differs on the sharpest blob between the Windows Picture and FAX Viewer
> and Paint Shop Pro 9. Looking at the blob above the sharpest, see how
> there is a sort of "interlace" structure visible on the PSP-9 decode but
> not on the P&F Viewer.
PSP-9 JPEG encoding and decoding with chroma subsampling is demonstrably
broken. See past threads about it. It is a tribute to just how robust
JPEG is that the errors it introduces do not significantly compromise
real photographic image quality. Line art tends to suffer though.
> I'm not at the moment saying one is right and one
> is wrong - just asking if there is a single "correct" decode. The
> original JPEG is here:
>
> http://www.david-taylor.myby.co.uk/Image1_2x2.jpg
>
> (This was originally from Bart Hickman on the Corel Paint Shop Pro X
> newsgroup).
There are decoders that match the original JPEG specification. And there
are more modern (but CPU intensive) experimental decoders that try to
select one of the set of images that when encoded yield exactly the same
set of JPEG coeffients. You can often dither any single pixel in the
image up or down by 1 in any of R,G or B channels without altering the
JPEG encoding. There are as a result quite a lot of possible "correct"
answers to the decoding of a given JPEG coefficient file.
Regards,
Martin Brown
Martin,
Many thanks for your comments which I've saved (for posterity!). I
haven't seen the original image from which the soothed images were
created. I would completely agree with your comment about the
single-pixel line. This may well be under-sampled, and not anti-aliased
correctly, and therefore an attempt to test the JPEG encoder with data
outside its specifications. The artefacts on the smoother blobs are clear
enough though.
I note your comment about PSP-9, but there are many other programs which
show the same defects. Other people have tested these and reported in the
Corel newsgroup where this question was first asked. IIRC, Adobe programs
show the same issue.
Thanks for taking the time and trouble to comment, and for sharing your
knowledge.
David
What were the JPEG paramaters that made your images unviewable
on MS Windows? (presumably Internet Exploder)
>What about chroma sub-sampling? Doesn't that imply an upsampling of the
>chroma to produce a full resolution RGB image? Might upsampling methods
>differ? However, even if there isn't one mathematically correct answer
>(about upsampling), perhaps it could be said that any program which
>introduces artefacts is one to be avoided?
>This is what I'm talking about:
> http://www.david-taylor.myby.co.uk/Decoded-JPEG-samples.png
I suspect you're right. When chroma is subsampled, there's only one
chroma sample for every two luma samples. Now, where is that chroma
sample located spatially - is it halfway between the two luma samples,
or co-located with one of the luma samples (and if so, which one)?
The JPEG spec ought to say which of these interpretations is
correct, and that also tells you how the chroma ought to be downsampled
during encoding and upsampled during decoding to avoid the chroma
shifting relative to the luma.
But it's also awfully easy to write an encoder or decoder which doesn't
obey the spec (whatever it is), causing a half-pixel or even one-pixel
shift between luma and chroma in the output. I suspect that's
responsible for the cases where there seem to be more black pixels on
the top edge of the red area than the bottom, rather than about the same
number top and bottom.
The other thing that could be going on is the use of different
algorithms for upsampling the chroma. Some common algorithms give some
high-frequency boost, which yields sharper-looking images but also
produces more overshoot and staircasing along diagonal edges.
Dave
Thanks for your enlightening comments, Dave. Much appreciated.
David
/Fredrik Lönn
Interesting. Do you have a sample TIFF you can upload somewhere?
David
It's still not clear (to me anyway) whether that's an encoding or
decoding issue.
Bart
OK, I can accept that both processes have ambiguities. At least my
original question has been answered, for which thanks to all.
David.
It looks like only the most aggressive chroma subsampling option in
Paint Shop Pro version 9.0 causes any trouble opening.
That option is designated as:
YCbCR 4x4 2x2 2x2
GIMP (old version I use) provides three choices of chroma subsampling for
JPEG save: 1x1 1x1 1x1 (none), 2x2 1x1 1x1 (xv), and 2x1 1x1 1x1 (digicam).
ImageMagick has only the first two, called -sampling_factor 1x1 and 2x2.
PaintShopPro version 9 offers *many* more chroma subsampling options!!
It has both 2x1 1x1 1x1 and 1x2 1x1 1x1. Is there a practical difference?
Most DSLR and digital cameras produce 2x1 1x1 1x1 according to jpegdump,
also called 4:2:2, not 1x2 1x1 1x1.
Why would anyone use these offbeat chroma subsampling choices in PSP 9?
2x2 2x1 2x1
4x2 1x1 1x1
2x4 1x1 1x1
4x1 1x1 1x1
1x4 1x1 1x1
4x1 2x1 2x1
1x4 1x2 1x2
4x4 2x2 2x2 (causes compatibility problems, see above)
As far as I can tell from section A.1.1 of ITU T.81, that's the same as 2x2
1x1 1x1, except that the MCU is larger: 24 blocks instead of 6.
Implementations are only required to support 10 blocks per MCU (section
B.2.3). So this subsampling option is (a) not compliant with the standard
and (b) even if it were compliant would be visually identical to 2x2 1x1
1x1. Conclusion: either the authors of Paint Shop Pro don't understand the
JPEG standard, or I don't.
> PaintShopPro version 9 offers *many* more chroma subsampling options!!
> It has both 2x1 1x1 1x1 and 1x2 1x1 1x1. Is there a practical difference?
For display on an analog raster-scanned screen like a CRT, 2x1 is better
than 1x2. I can't think of any situation where 1x2 would be better than 2x1,
unless you plan to rotate the image 90 degrees and then display it on a CRT.
> Why would anyone use these offbeat chroma subsampling choices in PSP 9?
> 2x2 2x1 2x1
> 4x1 2x1 2x1
> 1x4 1x2 1x2
> 4x4 2x2 2x2 (causes compatibility problems, see above)
The four that I quoted above are visually indistinguishable from 1x2 1x1
1x1, 2x1 1x1 1x1, 1x2 1x1 1x1, and 2x2 1x1 1x1 respectively, making them
entirely pointless. They all have an MCU size within the limit, except for
the one marked as having compatibility problems.
-- Ben
I can't answer that question! The choices are interesting to have, but I
tend to use just 1x1 1x1 1x1 and 2x2 1x1 1x1, thinking of them as "best
quality" and "routine quality".
David
Right.
If you edit files from digital cameras, both P&S and DLSR models from
most vendors, you can cause less re-artifacting by choosing 2x1 1x1 1x1
(4:2:2), which is how most vendors encode JPEG.
Bill, just to clarify, you said: 2x1 1x1 1x1. Is that what tou meant, and
not 2x2 1x1 1x1?
David
I would bet on the former.
> For display on an analog raster-scanned screen like a CRT, 2x1 is better
> than 1x2. I can't think of any situation where 1x2 would be better than 2x1,
> unless you plan to rotate the image 90 degrees and then display it on a CRT.
Some EXIF headers specify that an image should be rotated, but I have not
yet studied whether such images use 1x2 JPEG encoding.
> The four that I quoted above are visually indistinguishable from 1x2 1x1
> 1x1, 2x1 1x1 1x1, 1x2 1x1 1x1, and 2x2 1x1 1x1 respectively, making them
> entirely pointless. They all have an MCU size within the limit, except for
> the one marked as having compatibility problems.
Thanks very much for all the good information. It sounds like GIMP offers
the three most important JPEG chroma subsampling options. I would guess
digital camera vendors are using 2x1 1x1 1x1 instead of 1x1 1x1 1x1 because
it creates much smaller files (often 2/3 at Q99) without creating as many
blocking artifacts as 2x2, which is not all that much more compact.
Correct. The jpegdump program is useful for determining JPEG parameters.
For 2x1 1x1 1x1 (from a Fuji digicam) this is what it says; see id 1 line:
image.jpg:
Approximate quality factor for qtable 0: 89 (scale 22.99, var 3.56)
Approximate quality factor for qtable 1: 88 (scale 23.91, var 0.79)
Baseline JPEG width 1600, height 1200 components 3
id 1 horizontal sampling 2, vertical sampling 1, quantization table 0
id 2 horizontal sampling 1, vertical sampling 1, quantization table 1
id 3 horizontal sampling 1, vertical sampling 1, quantization table 2
Interesting. I see the same on a Nikon 8400 and Panasonic FZ5.
Given that the Bayer matrix looks similar in both horizontal and vertical
directions, why would the luminance sampling be asymmetrical?
David
Are you referring to qtable 0 having 89 and qtable 1 having 88?
I thought the id 1-3 lines referred to chroma sampling.
No, simply to the fact that I would expect luminance sampling to be
symmetrical in the horizontal and vertical directions. So that the id 1
line would read:
id 1 horizontal sampling 2, vertical sampling 2
I must be misunderstanding precisely what the sampling means in this
context.
David
> Bill Tuthill wrote:
>
>> David Harmon <sou...@netcom.com> wrote:
>>
>>> It looks like only the most aggressive chroma subsampling option in
>>> Paint Shop Pro version 9.0 causes [MS Windows] any trouble opening.
>>> That option is designated as:
>>> YCbCR 4x4 2x2 2x2
>
> As far as I can tell from section A.1.1 of ITU T.81, that's the same as
> 2x2 1x1 1x1, except that the MCU is larger: 24 blocks instead of 6.
> Implementations are only required to support 10 blocks per MCU (section
> B.2.3). So this subsampling option is (a) not compliant with the
> standard and (b) even if it were compliant would be visually identical
> to 2x2 1x1 1x1. Conclusion: either the authors of Paint Shop Pro don't
> understand the JPEG standard, or I don't.
I believe you are correct. The chroma subsampling in PS Pro v9 is
demonstrably broken. It reduces certain awkward test cases of saturated
red (or blue) dots on a black background to a grey mush using the
default JPEG subampling of 2x2 1x1 1x1.
A quick prod with the wizard suggests that the following subsamplings
1x1 1x1 1x1
1x2 1x1 1x1
2x2 2x1 2x1
1x4 1x1 1x1
1x4 1x2 1x2
Do what I would expect and that the others show gross chroma artefacts
on this simple but demanding line art test case.
IJG and other JPEG codecs retain more colour information. It looks like
Paintshop Pro v9 subsamples chroma detail very crudely by dropping
unwanted pixels compared to earlier versions and other applications.
It is a testament to how robust JPEG encoding is that hardly anybody has
actually noticed this issue on real images. Digital cameras do not
sample chroma detail finely enough for this to cause real damage.
>
>> PaintShopPro version 9 offers *many* more chroma subsampling options!!
>> It has both 2x1 1x1 1x1 and 1x2 1x1 1x1. Is there a practical
>> difference?
>
> For display on an analog raster-scanned screen like a CRT, 2x1 is better
> than 1x2. I can't think of any situation where 1x2 would be better than
> 2x1, unless you plan to rotate the image 90 degrees and then display it
> on a CRT.
>
>> Why would anyone use these offbeat chroma subsampling choices in PSP 9?
>> 2x2 2x1 2x1
>> 4x1 2x1 2x1
>> 1x4 1x2 1x2
>> 4x4 2x2 2x2 (causes compatibility problems, see above)
>
> The four that I quoted above are visually indistinguishable from 1x2 1x1
> 1x1, 2x1 1x1 1x1, 1x2 1x1 1x1, and 2x2 1x1 1x1 respectively, making them
> entirely pointless. They all have an MCU size within the limit, except
> for the one marked as having compatibility problems.
I can see no point in them offering the redundant ones beyond adding to
the "features" list (as opposed to providing a real benefit).
4x4 1x1 1x1
Might sometimes be worth having but at 18 blocks/MCU is out of spec.
Regards,
Martin Brown
Those id 1-3 lines refer to chroma subsampling, I believe. I'd have to
read the source code to be absolutely certain.
Luminance subsampling is always 1:1 in JPEG. See the Wikipedia article:
http://en.wikipedia.org/wiki/Chroma_subsampling
This is becuse human perception is supposedly more sensitive to
luminance changes than to chroma changes.
Bill, I'd appreciate if you could check out why there are three lines. I
thought it was for the luminance value and the two colour difference
values.
Cheers,
David