Jpeg XR

[Jonas Sicking]

Hi All,

At TPAC last week I chatted with one of the people from microsoft. He
asked if we were interested in supporting JPEG XR. It appears that it
has some significant improvements over the current JPEG spec such as
the obvious alpha channel, color profile and better compression. It
also has some more exotic features such as 48bit color depth, lossless
encoding and some image processing without having to decompress the
whole image, this may or may not be a good thing given that it
introduces complexity. It does not have support for simple animations,
a'la gif and apng.

I don't know enough about image formats or compression algorithms to
have a sense for how good this format is. I know we decided not to
support WebP since it wasn't "better enough" compared to JPEG.

There are also concerns about license of the reference library as well
as patents. Given that microsoft is wanting to make JPEG XR a new
format for the web, I suspect we can get them to make clear enough
statements regarding patent license (though we should of course make
sure this happens before we invest too much time).

The license of the reference library might be an issue, though he said
he'd look into it.

Assuming the license issues can be resolved. Are people at all
interested in JPEG XR?

/ Jonas

[Asa Dotzler]

According to Wikipedia, quoting the JPEG XR spec, it "delivers a lossy
compressed image of better perceptive quality than JPEG at less than
half the file size"

- A

[Jonas Sicking]

I'm sure that varies a lot depending on the picture. There's an
example in the wiki page, though how representative that example is I
have no idea.

http://en.wikipedia.org/wiki/JPEG_XR

/ Jonas

[Christopher Blizzard]

On Sun, Nov 6, 2011 at 4:55 PM, Jonas Sicking <jo...@sicking.cc> wrote:
> Assuming the license issues can be resolved. Are people at all
> interested in JPEG XR?

Yes.

We've discussed it on and off with the graphics team. It's mostly
just been a question of making sure we can get the right patent
licensing in place and an implementation we can ship.

It's also a question of bandwidth for people - the gfx team is busy
and we've had larger fish to fry at basically every turn. That's
probably still the case, tbh.

--Chris

[Benoit Jacob]

2011/11/6 Christopher Blizzard <christophe...@gmail.com>:

Isn't there also the concern that, as Jeff put it [1], "Every image
format that becomes “part of the Web platform” exacts a cost for all
time", and therefore, in order for an image format to get supported
for Web content, the bar is far higher than just "is better than JPEG"
?

[1] http://muizelaar.blogspot.com/2011/04/webp.html

Benoit

[Robert O'Callahan]

On Mon, Nov 7, 2011 at 5:16 PM, Robert O'Callahan <rob...@ocallahan.org>wrote:

> I nominate Tim Terriberry to decide whether JPEG XR meets that requirement
> :-).
>

And Jeff Muizelaar.

Rob
--
"If we claim to be without sin, we deceive ourselves and the truth is not
in us. If we confess our sins, he is faithful and just and will forgive us
our sins and purify us from all unrighteousness. If we claim we have not
sinned, we make him out to be a liar and his word is not in us." [1 John
1:8-10]

[Robert O'Callahan]

On Mon, Nov 7, 2011 at 5:04 PM, Benoit Jacob <jacob.b...@gmail.com>wrote:

> Isn't there also the concern that, as Jeff put it [1], "Every image
> format that becomes “part of the Web platform” exacts a cost for all
> time", and therefore, in order for an image format to get supported
> for Web content, the bar is far higher than just "is better than JPEG"
> ?
>

Yes.

I nominate Tim Terriberry to decide whether JPEG XR meets that requirement
:-).

[Gervase Markham]

On 07/11/11 00:55, Jonas Sicking wrote:
> Assuming the license issues can be resolved. Are people at all
> interested in JPEG XR?

According to jrmuizel in bug 500500, the reference implementation is 10x
slower than libjpeg. So we won't be shipping that. He mentored a GSoC
2011 project to write a jpeg-xr library. Unfortunately, the student did
not successfully complete the project.

The bug for support is here:
https://bugzilla.mozilla.org/show_bug.cgi?id=500500

The user's page about their GSoC is here:
https://wiki.mozilla.org/User:Chris48

The code is on GitHub:
https://github.com/chris493/libjpeg-xr
It looks like Chris began by forking libjpeg. jrmuizel would have to
tell you if this code is a reasonable base to continue work, or whether
it would be better to start again.

Legal has made an assessment regarding JPEG-XR. I'd have to check with
them before sharing their current thinking, but I can tell you that they
have done some thinking :-)

Gerv

[Jeff Muizelaar]

On 2011-11-06, at 7:55 PM, Jonas Sicking wrote:

> Hi All,
>
> At TPAC last week I chatted with one of the people from microsoft. He
> asked if we were interested in supporting JPEG XR. It appears that it
> has some significant improvements over the current JPEG spec such as
> the obvious alpha channel, color profile and better compression. It
> also has some more exotic features such as 48bit color depth, lossless
> encoding and some image processing without having to decompress the
> whole image, this may or may not be a good thing given that it
> introduces complexity. It does not have support for simple animations,
> a'la gif and apng.
>
> I don't know enough about image formats or compression algorithms to
> have a sense for how good this format is. I know we decided not to
> support WebP since it wasn't "better enough" compared to JPEG.

Feature wise, JPEG XR has most things that anyone would want.

> There are also concerns about license of the reference library as well
> as patents. Given that microsoft is wanting to make JPEG XR a new
> format for the web, I suspect we can get them to make clear enough
> statements regarding patent license (though we should of course make
> sure this happens before we invest too much time).
>
> The license of the reference library might be an issue, though he said
> he'd look into it.

Yes, the license of the reference library is not something we'd ship. I also don't think the quality (security and performance) of the library is very good and so this would need to be improved before we could consider shipping it.

> Assuming the license issues can be resolved. Are people at all
> interested in JPEG XR?

I'm interested, but there's a lot of work that would need to be done. I've done some quick comparisons and seen other work (http://cbloomrants.blogspot.com/2011/01/01-12-11-imdiff-sample-run-and-jxr-test.html) that suggests that the current JPEG-XR encoders don't provide much compression improvement.

If Microsoft is looking to help, I think releasing a high quality encoder/decoder under the same license as libjpeg would go along ways to improving adoption.

-Jeff

[Jonas Sicking]

Awesome feedback!

Based on this it seems like Jpeg XR isn't quite there yet, but there
are concrete action items that could get it there. I've forwarded this
information back to microsoft.

/ Jonas

[Jeff Gilbert]

For what it's worth, there have been a number of requests (especially from WebGL developers) for a lossy image format with an alpha channel, for use as RGBA textures. XR would definitely fit the bill.

_______________________________________________
dev-platform mailing list
dev-pl...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

[Benoit Jacob]

2011/11/7 Jeff Gilbert <jgil...@mozilla.com>:

> For what it's worth, there have been a number of requests (especially from WebGL developers) for a lossy image format with an alpha channel, for use as RGBA textures. XR would definitely fit the bill.

Seconded: the lack of a lossy compressed format with alpha is one of
the biggest missing GFX piece for web-based gaming, as it turns out
that most textures in games make use of the alpha channel for all
sorts of effects.

I'm concerned about the process here. I believe that the worst outcome
would be to have to support both JPEG XR and WebP. One way to get into
that bad situation would be if we accepted one too quickly, and then
the other one turned out to be better, and we'd face the dilemma of
sticking with the wrong one, or supporting both.

So I don't think that we should be telling Microsoft "here are the two
things that you have to do before we accept JPEG XR for web content".
We should rather be saying "Here are two very important things that we
care about, that we'll consider if and when we choose a
next-generation image format for web content".

Benoit


it's very important here to let competition between JPEG XR and WebP
sort out a clear winner, if possible, before

[Jeff Muizelaar]

On 2011-11-07, at 7:21 PM, Benoit Jacob wrote:

> 2011/11/7 Jeff Gilbert <jgil...@mozilla.com>:
>> For what it's worth, there have been a number of requests (especially from WebGL developers) for a lossy image format with an alpha channel, for use as RGBA textures. XR would definitely fit the bill.
>
> Seconded: the lack of a lossy compressed format with alpha is one of
> the biggest missing GFX piece for web-based gaming, as it turns out
> that most textures in games make use of the alpha channel for all
> sorts of effects.
>
> I'm concerned about the process here. I believe that the worst outcome
> would be to have to support both JPEG XR and WebP. One way to get into
> that bad situation would be if we accepted one too quickly, and then
> the other one turned out to be better, and we'd face the dilemma of
> sticking with the wrong one, or supporting both.

I'm in no rush to add new image formats, so I don't think this will be too much of a problem.

> So I don't think that we should be telling Microsoft "here are the two
> things that you have to do before we accept JPEG XR for web content".
> We should rather be saying "Here are two very important things that we
> care about, that we'll consider if and when we choose a
> next-generation image format for web content".

I certainly agree. I certainly did not want to give the impression that we would quickly accept JPEG-XR if Microsoft did a couple of things.

-Jeff

[Justin Dolske]

On 11/7/11 4:21 PM, Benoit Jacob wrote:

> Seconded: the lack of a lossy compressed format with alpha is one of
> the biggest missing GFX piece for web-based gaming, as it turns out
> that most textures in games make use of the alpha channel for all
> sorts of effects.

Out of curiosity, what format do non-web games use now? PNG?

Is it not an issue for games distributed on optical media? (But what
about Steam or other online distribution?) Do existing casual web games
(largely Flash/Java) have any solution for this?

Justin

[Benoit Jacob]

2011/11/8 Justin Dolske <dol...@mozilla.com>:

> On 11/7/11 4:21 PM, Benoit Jacob wrote:
>
>> Seconded: the lack of a lossy compressed format with alpha is one of
>> the biggest missing GFX piece for web-based gaming, as it turns out
>> that most textures in games make use of the alpha channel for all
>> sorts of effects.
>
> Out of curiosity, what format do non-web games use now? PNG?

I heard that a few major game engines, including Id software's Rage,
use JPEG XR.

> Is it not an issue for games distributed on optical media? (But what about
> Steam or other online distribution?) Do existing casual web games (largely
> Flash/Java) have any solution for this?

According to this wikipedia page, the Flash runtime now supports JPEG XR:
http://en.wikipedia.org/wiki/JPEG_XR#Software_support

Benoit

[myut...@gmail.com]

I've chipped in my thoughts on Jpeg-XR on the bugzilla entry for it. But I'll add some food for thought here anyways.

Some other advantages, which influenced the flash player teams decision to include flash, are listed at this blog entry: http://blog.kaourantin.net/?p=116

Most of these benefits listed are aimed at games and 3D applications. There is certainly an obvious advantage in games to an image format that will support lossy/lossless compression *and* alpha channel support. but other advantages in Jpeg XR include support for more pixel formats such as RGB565 along with support for floating point pixel formats which would enable HDR scenarios in the future for WebGL. Jpeg-XR also supports a arbitrary range of colour channels which would allow for textures with alpha channels *and* bump/luminosity maps in the same file among other useful things.

lastly the blog entry implies there is evidence that Jpeg-XR is better at preserving colour accuracy than Jpeg today. Though this seems to be mostly to do with specific encoders.

I'd personally love to see Jpeg-XR adopted in Firefox, it seems like a format that would provide a lot of bang for your buck in regards to the cost of adding complexity to the browser codebase. The licencing issues don't seem insurmountable and i'd imagine with todays low adoption rate of Jpeg-XR Microsoft could perhaps have their arm bent to make the licencing even more liberal.

[Steven Roussey]

At some point in the future, if and when cameras start shipping with JPEG-XR, then having browsers support these JPEG images on webpages (tossing aside things like games and alpha transparency), would be not only a plus, but kinda expected. I for one wish Canon had incorporated it into their DSLRs last time I bought one, since it is a great compromise between jpeg and raw for a lot of random shots I do. But unless Canon adopts it for its cameras, or Apple adopts it for the Phone 5 HDR photos, I think a decision can be postponed, even though I would like to see it supported.

[Steven Roussey]

[myut...@gmail.com]

[cquar...@gmail.com]

JPEG-XR has been standardized by ISO as ISO/IEC TR 29199-1:2011 and supports many different embedded representations, lossless or not.
The first aspect is that if the audience is not color blind, most of jpg implementations are 8bpc, so it is a desctructive format in the first place, especially with all the hardware capable of 10bpc or above, from cameras to monitors. Also it is likely that camera manufacturers abandon RAW format progressively in favor of JPEG XR, rather than for instance embedding jpg image in RAW picture mainly for preview, and then expecting the audience to convert RAW to 16-bit TIFF, OpenEXR, LuvLogTIFF or JPEG XR to retain dynamic range and color depth of the sensor, especially for Foveon-based cameras like the Sigma SD1,DP1 & DP2 Merrill. The only issue for Google+ is that they would hurt FLickr, Facebook, Pinterest, Instagram and a number of sites who are managed by executives who are just there for the paycheck rather than figuring out what ACES, RedColor3, QDEF, IGZO or LaserVue mean, as it was not a topic in their neck of the woods.....

[cquar...@gmail.com]

On Monday, November 28, 2011 9:04:58 AM UTC-5, myut...@gmail.com wrote:

[Stephen Shankland]

On Monday, November 26, 2012 10:44:44 AM UTC+1, cquar...@gmail.com wrote:
> The first aspect is that if the audience is not color blind, most of jpg implementations are 8bpc, so it is a desctructive format in the first place, especially with all the hardware capable of 10bpc or above, from cameras to monitors. Also it is likely that camera manufacturers abandon RAW format progressively in favor of JPEG XR, rather than for instance embedding jpg image in RAW picture mainly for preview, and then expecting the audience to convert RAW to 16-bit TIFF, OpenEXR, LuvLogTIFF or JPEG XR to retain dynamic range and color depth of the sensor, especially for Foveon-based cameras like the Sigma SD1,DP1 & DP2 Merrill....

I'm both a serious photographer and a reporter who's written about image formats and digital photography hardware and software. I see it as extremely unlikely that camera makers will abandon raw formats for their higher-end models. Although newer formats such as JPEG XR offer some potential benefits regarding color gamut, compression, and bit depth when compared to conventional JPEG, raw photo formats (and Adobe's DNG format, which it's trying to standardize) retain flexibility that JPEG XR necessarily loses. For example, when creating a JPEG or JPEG XR or WebP image from the raw sensor data, a camera makes irrevocable decisions about white balance, noise reduction, and sharpening. Photography pros and enthusiasts value this flexibility and aren't likely to be happy to see it go away.

It's possible that JPEG XR could catch on as an alternative to JPEG. However, I haven't detected enthusiasm among camera makers, who've had several years to embrace JPEG XR if they wanted to, and I don't see Microsoft or others trying to advocate adoption in hardware. If camera makers don't support JPEG XR, there's much less of an incentive to support it on the Web since ordinary people won't be uploading their photos with it. There might be some advantages in compression ratio and alpha channel for Web developers and graphic artists, of course, but at least for the foreseeable future I don't see this as a mainstream graphics format.