Intent to Implement: support for Accept-encoding: Brotli

[Kenji Baheux]

Contact emails

Engineering: 

eus...@chromium.org (chromium integration)

{jyrki@, szabadka@, lode@}google.com (Brotli)

PM: kenji...@chromium.org

Spec
https://tools.ietf.org/html/draft-alakuijala-brotli-04

Summary
Brotli is used in WOFF 2.0 web fonts with great success.

This intent to implement is about making Brotli available as a content-encoding method (e.g. Accept-Encoding: Brotli).

Advantages of Brotli over gzip:
- significantly better compression density (6.7x on fetch logs)
- comparable decompression speed

Disadvantage of Brotli over gzip:
- slower compression speed making it less useful for serving dynamic assets*

*: different compression settings can be used to strike the right balance between compression density and compression speed.

Motivation
Beyond its proven track record on web fonts, Brotli also leads to significant gains on other critical web assets (e.g. JS, CSS or document fragments). Brotli should improve page loading time.

In addition, given that JS and CSS assets tend to be block rendering, we expect that Brotli will also have a positive impact on RAIL metrics such as time to first meaningful paint.

Finally, transferring less bytes will also benefits users who are on a prepaid plan or have a bandwidth cap.


Compatibility Risk
Internet Explorer: No public signals
Safari: No public signals

Firefox: Positive signals

In particular, this comment by Patrick McManus seems relevant:

“I continue to be open to the prospect of adding a non-deflate based decoder and the negotiation for it as part of Accept-Encoding. I've said upthread a few times that what we need is a content provider willing to do make a real use case available for negotiation that way - then we can see how it works out.”

Web developers: Positive signals

For instance, Ben Maurer commented:

“Doing some ad-hoc tests on the Javascript and CSS at Facebook we see some substantial improvements. Brotli would save about 17% of CSS bytes and 20% of javascript bytes (compared with zopfli). Resource download times are a big bottleneck, so this would make a big impact for page load times.”

Going back to Patrick’s comment about the need for a dance partner, it sounds like Facebook is interested and we have internal customers interested as well.

Describe the degree of compatibility risk you believe this change poses: Low compatibility risk.

The content-encoding scheme is negotiated between UA and Servers via the Accept-encoding HTTP header.

Ongoing technical constraints
None.

Note: we are considering restricting to/starting with HTTPS only so we can avoid adding yet another batch of workarounds/special casing.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux,
Chrome OS, Android, and Android WebView)? Yes.

OWP launch tracking bug
http://crbug.com/452335

Link to entry on the Chromium Dashboard
https://www.chromestatus.com/features/5420797577396224

Requesting approval to ship?
Not just yet as we need to clear a few questions before hand.

[Ben Maurer]

Hey,

On Monday, July 6, 2015 at 8:44:54 PM UTC-7, Kenji Baheux wrote:

Beyond its proven track record on web fonts, Brotli also leads to significant gains on other critical web assets (e.g. JS, CSS or document fragments). Brotli should improve page loading time.

In addition, given that JS and CSS assets tend to be block rendering, we expect that Brotli will also have a positive impact on RAIL metrics such as time to first meaningful paint.

Finally, transferring less bytes will also benefits users who are on a prepaid plan or have a bandwidth cap.

For instance, Ben Maurer commented:

“Doing some ad-hoc tests on the Javascript and CSS at Facebook we see some substantial improvements. Brotli would save about 17% of CSS bytes and 20% of javascript bytes (compared with zopfli). Resource download times are a big bottleneck, so this would make a big impact for page load times.”

To confirm, Facebook is very excited to provide Brotli-compressed static resources. We would also consider using Brotli for dynamic resources if we were able to provide better compression at similar CPU costs. We're happy to provide help in providing realistic data about our JS/CSS resources to help in evaluating this algorithm.

 

Note: we are considering restricting to/starting with HTTPS only so we can avoid adding yet another batch of workarounds/special casing.

We would support this type of restriction. Facebook is HSTS-enabled and so any traffic from Chrome would be over HTTPS. I believe that most performance sensitive websites would choose to implement HTTP/2 and thus already benefit from a secure transport.

-b

[Mike Lawther]

Since Firefox and Chrome already ship Brotli for WOFF2 support, the tech is in multiple shipping browsers. Reusing the code in for content encoding seems reasonable, especially given there is a third party (Facebook in this case) keen to use it.

Looking at http://crbug.com/452335, I can see some questions that I hope have been resolved, in particular the having https://code.google.com/p/chromium/issues/detail?id=488668 as a prerequisite, and the Android decode speed mentioned in https://code.google.com/p/chromium/issues/detail?id=452335#c37.

Assuming the implementation proceeds, are there metrics that we expect to improve (you mentioned 'time to first meaningful paint') before we ship?

[j...@cloudflare.com]

Just thought I'd _o/ on behalf of CloudFlare. We are watching this and are interested. A couple of years ago I researched compression technologies and looked at what we might be interested in. At the time Brotli was only for WOFF and it was not something that we'd see via Accept-Encoding.

We actually have an internal product that does recompression for images which we can adapt to recompress text so that we can make Brotli available. We would not do on-the-fly recompression because the compression speed just isn't there and at our scale would not make sense. We can look at Brotli compression of static text assets that are in our cache.

If this becomes a reality we are likely to support it by making this transparent for our customers. We try to stay on top of new technologies like this and offer them freely.

Note: we are also interested in other compression algorithms. For example, lzma. If browsers start offering Accept-Encoding: gzip, deflate, brotli, sdch, lzma etc. then we will look at supporting.

[Philip Jägenstedt]

This sounds pretty cool, is there a high-level description of the format somewhere? It looks like the decompression speed is a fair bit faster than LZMA, is that the main reason to prefer it or are other interesting aspects of the format, like the "2nd order context modeling"?

The Gecko bug was originally about supporting LZMA, and it looks like they haven't really committed to a format. Patrick, do you think it's likely that the balance would tip in the favor of Brotli if it's supported in Chromium? (It would clearly be unfortunate if each browsers picks a different format.)

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Kenji Baheux]

Brotli being considerably faster for decompression than LZMA is indeed one reason.

The other reason is that we originally considered LZMA for Woff 2.0 but had to give up because of the lack of a proper specification and willingness to create one. We reached out internally for an alternative. The compression team at Google offered to help and delivered beyond our expectations :)

[Chris Bentzel]

I'm fine with implementing once the Filter re-architecture is completed. That is pretty far along right now last I checked (just getting back from vacation).

My major concerns were listed in the bug, but repeating here:

 

- Please measure per-context memory utilization and keep it contained. RAM consumption might be higher than gzip/deflate. Most will be window size tuning but there were some other fixed constants in play that made it look a bit heavier.

- Have continuous fuzz testing running before turning this on for users who don't explicitly enable in about:flags. Brotli decompression is currently done in a sandboxed process. Doing it in Content-Encoding is done in an un-sandboxed process - and is also different code now that the decompressor has been rewritten in a streaming fashion.

- I _think_ it would be good to have this feature enabled/disabled via a compile-time flag but might be wrong here. Main thing I'm concerned about is binary size impact on Cronet if it's not actively being used.

- Let's stick with restricting to https until there's very strongly compelling evidence that we should do something else.

--
You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.
To post to this group, send email to net...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CADWWn7V3gOR_Y36vwtOh_nJpxRtUGsk8cT9oVj2gL%2BYmHHRCTg%40mail.gmail.com.

[Chris Bentzel]

Is there data showing the benefit on a certain corpus?

On Sat, Sep 5, 2015 at 6:16 PM <cry...@free.fr> wrote:

Hello,
I see the interest of code reuse by picking Brotli, but wouldn't Zstandard (or Apple LZFSE) be way faster and therefore more interesting on mobile and for on the fly compression?

Cheers

[Brian Smith]

Kenji Baheux <kenji...@chromium.org> wrote:

This intent to implement is about making Brotli available as a content-encoding method (e.g. Accept-Encoding: Brotli)

Consider adding a parameter for the client to indicate the maximum window size that it supports.

zlib only requires 44KB of memory to decompress any DEFLATE stream.  Brotli seems to require 4MB or 16MB (depending on where I read) of memory to decompress. Many resources that would benefit from Brotli compression are going to be much smaller than that, and some devices may wait to use Brotli but only if the window size is much smaller.

It would also be interesting to see benchmarks, relative to deflate, as a function of window size.

Cheers,

Brian

[bay...@gmail.com]

Just to be clear, "Accept-Encoding: Brotli" has been changed to "Accept-Encoding: br", right?

https://github.com/google/brotli/blob/c4f439dbe6007b2a37d50c20419253d5aaa8b46b/docs/draft-alakuijala-brotli-06.txt

[Kenji Baheux]

Yes, we wanted something shorter and are settling down on br for A-E and the file extension.

[John Graham-Cumming]

BTW If anyone needs a live server to test brotli against the CloudFlare test server https://http2.cloudflare.com/ is available. It's running a patched version of NGINX (we've built a brotli module for NGINX that we will open source at some point) and does brotli compression.

Currently, it looks for Accept-Encoding with brotli in it (not br) and if brotli and gzip present brotli wins.

It's very experimental, all sorts of things could go wrong but if you've got a browser that supports brotli it's there for you.

John.

--
You received this message because you are subscribed to a topic in the Google Groups "net-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/net-dev/xdVm8c2GOMQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to net-dev+u...@chromium.org.

To post to this group, send email to net...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CADWWn7XaHFrtFsid3f%2B4CtaETZusGdbJ3iGpwveUirM8AiOeWA%40mail.gmail.com.

[stepien...@gmail.com]

Looks like Google beat CF to the punch on this one: https://github.com/google/ngx_brotli

[John Graham-Cumming]

Not quite. Our module was released hours before theirs :-) 

https://github.com/cloudflare/ngx_brotli_module

Early this coming week we'll write a blog post on our experiments with Brotli as we have some data on compression ratios for our workload (i.e. text assets on the web). 

We are still considering when to put Brotli into production in our global network. It would be nice if Chrome supported it and not just Firefox. 

John.