Intent to Prototype: jxl Content-Encoding

[Evgenii Kliuchnikov]

Contact emails eus...@chromium.org Explainer https://github.com/google/brunsli https://brunsli.dev/ https://brunsli.dev/split.html?progressive Design docs/spec Specification: https://arxiv.org/pdf/1908.03565.pdf https://docs.google.com/document/d/1jiuaWa_xgFEnc7ILIajU8CBceNNcCa5dbEK-QdSecVA/edit#heading=h.xgjl2srtytjt TAG review Summary JPEG XL is a next generation image format. One of its features is byte-wise lossless JPEG image repacking. On average, encoded image is 22% smaller. This encoding could be used as an HTTP "Content-Encoding" specifically for JPEG image transfer (such content is considered non-compressible for general-purpose formats like Brotli and Gzip). Proposed encoding name: 'jxl' Motivation Losslessly reduce interned traffic. Make internet faster. Risks

Interoperability and Compatibility As with 'br' Content-Encoding we anticipate problems with incorrect proxies and recommend advertising new encoding only over opaque connection (HTTPS, etc.) Gecko: No signal WebKit: No signal Web developers: No signals Ergonomics While it is possible to process format incrementally, it would be beneficial to add support for decoding off-loading for net/filters Activation There are Apache and nginx plugins for serving 'jxl' compressed content. Demo-page is powered by polyfill ServiceWorker. Security Data parsing. Thoroughly fuzzed.

Debuggability Not required. Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)? Yes Every platform is free to advertise 'jxl' in 'Accept-Encoding' header. Plaftorms that do not support it will receive conventional traffic. Is this feature fully tested by web-platform-tests? No jxl Content-Encoding is advertised only over HTTPS jxl content is decoded into original JPEG content Link to entry on the Chrome Platform Status https://www.chromestatus.com/feature/5678152091172864

[PhistucK]

Can it be advertised over http://localhost:x as well (it is already considered secure for other purposes)? This will aid the development of support for this.

☆PhistucK

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACZPrBjS75%3D%2B%3DSexFkJ7ewkBXDOF6FhkpwQYBeWn6sT57n1-_w%40mail.gmail.com.

[Evgenii Kliuchnikov]

Surely. This will not be much different from "br" Content-Encoding and there we do check if either scheme is cryptographic, or domain is localhost (see https://source.chromium.org/chromium/chromium/src/+/master:net/url_request/url_request_http_job.cc;drc=2e3998904a6f19a0e6372f6a411d7f6adddf4689;l=543?originalUrl=https:%2F%2Fcs.chromium.org%2F)

[Evgenii Kliuchnikov]

Update:

Design  Doc: https://docs.google.com/document/d/e/2PACX-1vT7wTH3IlhAL12Av8VrwlBXdqozrsk1-_5SdHlHxZ--Vo2avj_2D6UxnQoOo0BxHefoMDIdco-NQ4fi/pub

TAG Review: https://github.com/w3ctag/design-reviews/issues/541

[Yoav Weiss]

On Fri, Jul 31, 2020 at 5:43 PM 'Evgenii Kliuchnikov' via blink-dev <blin...@chromium.org> wrote:

Update:

Design  Doc: https://docs.google.com/document/d/e/2PACX-1vT7wTH3IlhAL12Av8VrwlBXdqozrsk1-_5SdHlHxZ--Vo2avj_2D6UxnQoOo0BxHefoMDIdco-NQ4fi/pub

The document doesn't seem to be public.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/9d202b2d-c141-45fc-b86b-b768dcba5cccn%40chromium.org.

[Evgenii Kliuchnikov]

Thanks for checking. Now should work:

Design Doc: https://docs.google.com/document/d/e/2PACX-1vQuwSbPNWwFhMXnK2rkdsyvTPGRDpX3kET-StAFqPZG43utfW1rZ-smsYoLodnNNIkNtZjlIIKol_kb/pub

[Yoav Weiss]

Awesome, thanks!

Thanks for working on this! :)

Can you elaborate on the choice to implement this is a content encoding, rather than as an image format?

[Evgenii Kliuchnikov]

Actually we want both. For image format we will file separate intent. Content-Encoding would be a choice for majority of already existing sites, beacause:

 * delivered content is exactly the same

 * CDNs are free to apply this tranformation

 * users will be able to save images and it is guaranteed that those images will be understood by the existing software (as those are regular JPEGs)

 * etc.

Will add this section to explainer tonight.

[Daniel Bratell]

For jxl as an image format in particular, do you have any guess on how likely other vendors are to support it?

/Daniel

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACZPrBj7M9qd%2BGGAX%2BH1WBPqQLeAUuxpgBoqc%3DWsELayyqmyAA%40mail.gmail.com.

[Evgenii Kliuchnikov]

+jyrki

[Yoav Weiss]

I talked a bit to the team, and would like to share my thoughts following that.

Supporting JPEG-XL as a Content-Encoding seems like it *might* enable a smoother transition period for sites that have old JPG images lying around.

New image formats incur some cost and as content encoding web servers and CDN providers can freely apply that encoding to get ~20% improvements over JPEGs, without worrying about:

1. Users downloading the image and failing to e.g. edit it
2. JPEG Hardware decoding not being utilized
3. Web applications manipulating the image bytes directly, and breaking due to the transcoding.

At the same time, those problems seem addressable by other means:

1. The browser could implement a "save as legacy format" context menu item, that would solve this problem for all next-gen image formats
2. The JPEG-XL decoding pipeline could make use of JPEG HW decoding abilities
3. Servers could avoid the JPEG-XL transformation when "no-transform" is indicated by the server, or when the image is same-origin/CORS-enabled `fetch()`ed by the user, which is likely a rare case

As it is, I'm not sure if it'd be easier to convince the ecosystem to adopt JPEG-XL as a Content-Encoding or as an image format.

But the good thing about an open ecosystem is - we can ask! :)

I took the liberty of sending an email to the IETF's HTTP-WG, asking for their opinions. We are likely to find CDNs and web server implementers there.

Otherwise, it'd be good to file for a TAG review, as well as get signals from other browser vendors.

Finally, I'm curious to hear from +Eric Portis and +Doyle, Nick if (3) above was ever a problem they've encountered (and what they think of the "content-encoding vs. image format" question in general).

Cheers :)

Yoav

[Yoav Weiss]

Thanks Jyrki!

I didn't state these points in particular, as I believe most of them can be achieved as part of the implementation, regardless if JPXL support is added as a Content-Encoding or as a full-fledged image format. Please correct me if I'm wrong on that front.

On Thu, Aug 20, 2020 at 10:08 PM Jyrki Alakuijala <jy...@google.com> wrote:

There are some benefits of this approach that may not yet be clear from Yoav's mail:

- The experience through content encoding is more similar than with a new image codec:

   * The transcoding is lossless, no need for quality experimentation to launch it

   * The transcoding allows for progressive or sequential refreshing of jpegs, less need for experiments to figure out all the subtleties of the latency impact

- The lossless transcoding is super-fast, currently in one experiment it was found about 10x faster than the fastest new modern image codec

- Once the JPEG is cached in the browser, it is just a JPEG, and it's redraws from cache are faster than redisplays for more modern image codecs

- While I do recommend server side caching of compressed data, one cpu can convert jpeg into jpeg xl (as well as jpeg xl into jpeg) losslessly at 100 mbps. For a site with a limited storage budget or for images that are not 'hot' it may be a good idea to store in the JPEG XL form and convert losslessly to legacy JPEG at request time. Storing legacy JPEG photos losslessly as JPEG XL saves 22 % in storage budget. (Some sites like dropbox have chosen to do this with their custom technology, as well as the media filter plugin for 7zip applies a similar compression method on legacy JPEG.)

[Jyrki Alakuijala]

There are some benefits of this approach that may not yet be clear from Yoav's mail:

- The experience through content encoding is more similar than with a new image codec:

   * The transcoding is lossless, no need for quality experimentation to launch it

   * The transcoding allows for progressive or sequential refreshing of jpegs, less need for experiments to figure out all the subtleties of the latency impact

- The lossless transcoding is super-fast, currently in one experiment it was found about 10x faster than the fastest new modern image codec

- Once the JPEG is cached in the browser, it is just a JPEG, and it's redraws from cache are faster than redisplays for more modern image codecs

- While I do recommend server side caching of compressed data, one cpu can convert jpeg into jpeg xl (as well as jpeg xl into jpeg) losslessly at 100 mbps. For a site with a limited storage budget or for images that are not 'hot' it may be a good idea to store in the JPEG XL form and convert losslessly to legacy JPEG at request time. Storing legacy JPEG photos losslessly as JPEG XL saves 22 % in storage budget. (Some sites like dropbox have chosen to do this with their custom technology, as well as the media filter plugin for 7zip applies a similar compression method on legacy JPEG.)

On Thu, Aug 20, 2020 at 4:21 PM Yoav Weiss <yo...@yoav.ws> wrote:

[Yoav Weiss]

On Thu, Aug 20, 2020 at 11:21 PM Yoav Weiss <yo...@yoav.ws> wrote:

Thanks Jyrki!

I didn't state these points in particular, as I believe most of them can be achieved as part of the implementation, regardless if JPXL support is added as a Content-Encoding or as a full-fledged image format. Please correct me if I'm wrong on that front.

On Thu, Aug 20, 2020 at 10:08 PM Jyrki Alakuijala <jy...@google.com> wrote:

There are some benefits of this approach that may not yet be clear from Yoav's mail:

- The experience through content encoding is more similar than with a new image codec:

   * The transcoding is lossless, no need for quality experimentation to launch it

   * The transcoding allows for progressive or sequential refreshing of jpegs, less need for experiments to figure out all the subtleties of the latency impact

- The lossless transcoding is super-fast, currently in one experiment it was found about 10x faster than the fastest new modern image codec

- Once the JPEG is cached in the browser, it is just a JPEG, and it's redraws from cache are faster than redisplays for more modern image codecs

Talking to +lizeb@, turns out that I'm at least partially wrong and this part cannot be automatically optimized by the cache, as it can result in web-exposed differences when the content is `fetch()`ed (similar to (3) above).

At the same time, it's unclear if the 20% extra storage space would be justified by faster decoding speeds.

+Josh Karlin may have some intuition here.

- While I do recommend server side caching of compressed data, one cpu can convert jpeg into jpeg xl (as well as jpeg xl into jpeg) losslessly at 100 mbps

Is this 100 mega *bits* or *bytes*? Do you have similar numbers for client-side typical CPU?

[Jyrki Alakuijala]

On Friday, August 21, 2020 at 11:53:13 AM UTC+2 yo...@yoav.ws wrote:

On Thu, Aug 20, 2020 at 11:21 PM Yoav Weiss <yo...@yoav.ws> wrote:

Thanks Jyrki!

I didn't state these points in particular, as I believe most of them can be achieved as part of the implementation, regardless if JPXL support is added as a Content-Encoding or as a full-fledged image format. Please correct me if I'm wrong on that front.

On Thu, Aug 20, 2020 at 10:08 PM Jyrki Alakuijala <jy...@google.com> wrote:

There are some benefits of this approach that may not yet be clear from Yoav's mail:

- The experience through content encoding is more similar than with a new image codec:

   * The transcoding is lossless, no need for quality experimentation to launch it

   * The transcoding allows for progressive or sequential refreshing of jpegs, less need for experiments to figure out all the subtleties of the latency impact

- The lossless transcoding is super-fast, currently in one experiment it was found about 10x faster than the fastest new modern image codec

- Once the JPEG is cached in the browser, it is just a JPEG, and it's redraws from cache are faster than redisplays for more modern image codecs

Talking to +lizeb@, turns out that I'm at least partially wrong and this part cannot be automatically optimized by the cache, as it can result in web-exposed differences when the content is `fetch()`ed (similar to (3) above).

At the same time, it's unclear if the 20% extra storage space would be justified by faster decoding speeds.

+Josh Karlin may have some intuition here.

- While I do recommend server side caching of compressed data, one cpu can convert jpeg into jpeg xl (as well as jpeg xl into jpeg) losslessly at 100 mbps

Is this 100 mega *bits* or *bytes*? Do you have similar numbers for client-side typical CPU?

100 mbps is 100 megabits per second on intel. On a single mobile arm core it is about 30 mbps. These are compressed numbers and the uncompressed speeds are about 15x faster.