Intent to Prototype: Zstd Content-Encoding

839 views
Skip to first unread message

Nidhi Jaju

unread,
Jun 21, 2023, 2:15:09 AM6/21/23
to blink-dev, Adam Rice

Contact emails

nidh...@chromium.org

Explainer

https://docs.google.com/document/d/1aDyUw4mAzRdLyZyXpVgWvO-eLpc4ERz7I_7VDIPo9Hc/edit?usp=sharing


Specification

https://datatracker.ietf.org/doc/html/rfc8878

Design docs

https://docs.google.com/document/d/14dbzMpsYPfkefAJos124uPrlkvW7jyPJhzjujSWws2k/edit?usp=sharing

Summary

Zstandard, or “zstd”, is a data compression mechanism described in RFC8878. It is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. The "zstd" token was added as an IANA-registered Content-Encoding token as per https://datatracker.ietf.org/doc/html/rfc8878#name-content-encoding. Adding support for "zstd" as a Content-Encoding will help load pages faster and use less bandwidth.


Blink component

Internals>Network

Motivation

Supporting zstd content-encoding in the browser would allow sites to spend less time and CPU/power on compression on their servers, resulting in reduced server costs. There are several published benchmarks[i.e. 1, 2] and existing research showing promising potential wins. Zstd is roughly three times faster than Brotli for decompression. Combined with zstd being faster at compression, this will result in faster page load times.

Initial public proposal

None

TAG review

None

TAG review status

Not applicable

Risks


              Interoperability and Compatibility

Servers that have a broken implementation of zstd might exist, but the risk of this is small. Additionally, middleware and middleboxes like virus checkers that intercept HTTPS connections might not support zstd, but might fail to remove it from the Accept-Encoding header in the request.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/775)

WebKit: No signal (https://github.com/WebKit/standards-positions/issues/168)

Web developers: Positive (https://crbug.com/1246971) Facebook (Yann) and Akamai (Nic) seem to be positive about zstd content-encoding support in the browser. Facebook is also excited to test the feature.

Other signals:

              WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?



Debuggability

No special support needed.

Is this feature fully tested by web-platform-tests?

Not yet.

Flag name

ZstdContentEncoding

Requires code in //chrome?

True

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1246971

Estimated milestones

Shipping on desktop117
Shipping on Android117


Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/6186023867908096

Links to previous Intent discussions



This intent message was generated by Chrome Platform Status.

Yoav Weiss

unread,
Jun 21, 2023, 2:36:05 AM6/21/23
to Nidhi Jaju, blink-dev, Adam Rice
On Wed, Jun 21, 2023 at 8:15 AM Nidhi Jaju <nidh...@chromium.org> wrote:

Contact emails

nidh...@chromium.org

Explainer

https://docs.google.com/document/d/1aDyUw4mAzRdLyZyXpVgWvO-eLpc4ERz7I_7VDIPo9Hc/edit?usp=sharing


Specification

https://datatracker.ietf.org/doc/html/rfc8878

Design docs

https://docs.google.com/document/d/14dbzMpsYPfkefAJos124uPrlkvW7jyPJhzjujSWws2k/edit?usp=sharing

Summary

Zstandard, or “zstd”, is a data compression mechanism described in RFC8878. It is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. The "zstd" token was added as an IANA-registered Content-Encoding token as per https://datatracker.ietf.org/doc/html/rfc8878#name-content-encoding. Adding support for "zstd" as a Content-Encoding will help load pages faster and use less bandwidth.


Blink component

Internals>Network

Motivation

Supporting zstd content-encoding in the browser would allow sites to spend less time and CPU/power on compression on their servers, resulting in reduced server costs. There are several published benchmarks[i.e. 1, 2] and existing research showing promising potential wins. Zstd is roughly three times faster than Brotli for decompression. Combined with zstd being faster at compression, this will result in faster page load times.

Initial public proposal

None

TAG review

None

TAG review status

Not applicable

Risks


              Interoperability and Compatibility

Servers that have a broken implementation of zstd might exist, but the risk of this is small. Additionally, middleware and middleboxes like virus checkers that intercept HTTPS connections might not support zstd, but might fail to remove it from the Accept-Encoding header in the request.


Are we planning to only support this encoding under secure contexts and/or H2+ to reduce that risk?
 
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMZNYANd_E77W1ki--h_XJM-%2B_fA3w1CriGgJmnbh1N3LwRDtw%40mail.gmail.com.

Nidhi Jaju

unread,
Jun 21, 2023, 5:03:07 AM6/21/23
to Yoav Weiss, blink-dev, Adam Rice
Yes, we're only planning to support zstd encoding under secure contexts, same as Brotli.

Sangwhan Moon

unread,
Jun 21, 2023, 5:53:34 AM6/21/23
to Nidhi Jaju, Yoav Weiss, blink-dev, Adam Rice


On 2023年06月21日 18時02分49秒 (+09:00), Nidhi Jaju wrote:



On Wed, Jun 21, 2023 at 3:35 PM Yoav Weiss <yoav...@chromium.org> wrote:


On Wed, Jun 21, 2023 at 8:15 AM Nidhi Jaju <nidh...@chromium.org> wrote:

Contact emails

nidh...@chromium.org

Explainer

https://docs.google.com/document/d/1aDyUw4mAzRdLyZyXpVgWvO-eLpc4ERz7I_7VDIPo9Hc/edit?usp=sharing


Specification

https://datatracker.ietf.org/doc/html/rfc8878

Design docs

https://docs.google.com/document/d/14dbzMpsYPfkefAJos124uPrlkvW7jyPJhzjujSWws2k/edit?usp=sharing

Summary

Zstandard, or “zstd”, is a data compression mechanism described in RFC8878. It is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. The "zstd" token was added as an IANA-registered Content-Encoding token as per https://datatracker.ietf.org/doc/html/rfc8878#name-content-encoding. Adding support for "zstd" as a Content-Encoding will help load pages faster and use less bandwidth.


Blink component

Internals>Network

Motivation

Supporting zstd content-encoding in the browser would allow sites to spend less time and CPU/power on compression on their servers, resulting in reduced server costs. There are several published benchmarks[i.e. 1, 2] and existing research showing promising potential wins. Zstd is roughly three times faster than Brotli for decompression. Combined with zstd being faster at compression, this will result in faster page load times.

Initial public proposal

None

TAG review

None


Drive by question: Given that the codec is going to be in the browser, are there plans to surface this up to CompressionStreams? (same question applies for Brotli, I suppose)

Adam Rice

unread,
Jun 21, 2023, 6:18:16 AM6/21/23
to Sangwhan Moon, Nidhi Jaju, Yoav Weiss, blink-dev
Drive by question: Given that the codec is going to be in the browser, are there plans to surface this up to CompressionStreams? (same question applies for Brotli, I suppose)

For the zstd Content-Encoding, we will only be linking in the decompression part of the zstd library. But for CompressionStreams, supporting a format only for decompression and not for compression is likely to confuse and frustrate developers.

So while I'd like to add Zstd to CompressionStreams, we'll need a good justification for the extra binary size increase caused by linking in the compression code. The same applies to Brotli.

Thanks for the question!

Sangwhan Moon

unread,
Jun 21, 2023, 6:19:39 AM6/21/23
to Adam Rice, Nidhi Jaju, Yoav Weiss, blink-dev


On 2023年06月21日 19時17分58秒 (+09:00), Adam Rice wrote:

Drive by question: Given that the codec is going to be in the browser, are there plans to surface this up to CompressionStreams? (same question applies for Brotli, I suppose)

For the zstd Content-Encoding, we will only be linking in the decompression part of the zstd library. But for CompressionStreams, supporting a format only for decompression and not for compression is likely to confuse and frustrate developers.

So while I'd like to add Zstd to CompressionStreams, we'll need a good justification for the extra binary size increase caused by linking in the compression code. The same applies to Brotli.

Thanks for the question!

Thanks for the clarification, good point on the binary cost.

Dale Curtis

unread,
Jun 21, 2023, 1:08:24 PM6/21/23
to Adam Rice, Sangwhan Moon, Nidhi Jaju, Yoav Weiss, blink-dev
On Wed, Jun 21, 2023 at 3:18 AM Adam Rice <ri...@chromium.org> wrote:
Drive by question: Given that the codec is going to be in the browser, are there plans to surface this up to CompressionStreams? (same question applies for Brotli, I suppose)

For the zstd Content-Encoding, we will only be linking in the decompression part of the zstd library. But for CompressionStreams, supporting a format only for decompression and not for compression is likely to confuse and frustrate developers.

FWIW, asymmetric codec capabilities are quite common in media and developers have adapted just fine. E.g., we may only have hevc/aac/theora decoding, but not encoding.
 

PhistucK

unread,
Jun 22, 2023, 2:05:06 PM6/22/23
to Dale Curtis, Adam Rice, Sangwhan Moon, Nidhi Jaju, Yoav Weiss, blink-dev
If/when shipping, just remember to add this to the list of "Accepted Content-Encodings" that shows up on the developer tools, under the "Network conditions" panel. :)
image.png

☆PhistucK


To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUDrwfV7F4QT_2g_CG9kypeujEKxaQ1MKe5fzVbY4uGSZifLw%40mail.gmail.com.

Peter Beverloo

unread,
Jun 26, 2023, 6:28:04 AM6/26/23
to Nidhi Jaju, Dale Curtis, Adam Rice, Sangwhan Moon, Yoav Weiss, blink-dev
Thank you for sharing the I2P Nidhi! What are your plans regarding supporting this feature in Android WebView? We're just rolling out support for Brotli there and we should aim for it to be on par with the browser regarding supported compression algorithms.

Thanks,
Peter


Mihai Cîrlănaru

unread,
Jun 26, 2023, 9:54:09 AM6/26/23
to blink-dev, PhistucK, Adam Rice, Sangwhan Moon, Nidhi Jaju, Yoav Weiss, blink-dev, Dale Curtis
Will the zstd content-encoding be supported in WebView as well from 117?

It wouldn't be ideal to be in a similar situation as with the Brotli ("br") content-encoding support that was not enabled in WebView from the start.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Nidhi Jaju

unread,
Jun 26, 2023, 11:19:08 PM6/26/23
to Peter Beverloo, Mihai Cîrlănaru, Dale Curtis, Adam Rice, Sangwhan Moon, Yoav Weiss, blink-dev
Thank you for reaching out, Mihai and Peter!
Our plan is to support this feature in WebView as well, but most likely a few milestones after M117 so that we can deal with any compatibility issues separately. I'll share further updates when we have more information on the WebView rollout plan.

Thanks,
Nidhi

Mihai Cîrlănaru

unread,
Jun 27, 2023, 8:10:38 AM6/27/23
to blink-dev, Nidhi Jaju, Mihai Cîrlănaru, Dale Curtis, Adam Rice, Sangwhan Moon, Yoav Weiss, blink-dev, Peter Beverloo
Thank you for the clarification, Nidhi!

Could you expand on the compatibility issues that might come up here? Given the overlap WebView has with Chrome in terms of source code, these might apply to both. Nevertheless, having WebView in the experiments from the start will help uncover WebView specific issues early and ensure a timely release (as opposed to Brotli support) and a unified narrative, i.e. "Zstd content-encoding support available for Android (Chrome and WebView)".

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Torne (Richard Coles)

unread,
Jun 27, 2023, 10:37:54 AM6/27/23
to Mihai Cîrlănaru, blink-dev, Nidhi Jaju, Dale Curtis, Adam Rice, Sangwhan Moon, Yoav Weiss, Peter Beverloo
Unless you have specific reasons to believe that there are going to be compatibility issues that are specific to WebView and a plan for how they're going to be addressed, we would strongly prefer that the rollout happens at the same time for WebView.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/118f6dad-17e3-4176-a391-04bb3bdfaab8n%40chromium.org.

Adam Rice

unread,
Jun 27, 2023, 11:42:52 AM6/27/23
to Torne (Richard Coles), Mihai Cîrlănaru, blink-dev, Nidhi Jaju, Dale Curtis, Sangwhan Moon, Yoav Weiss, Peter Beverloo
Our mitigations if a serious compatibility issue should occur are more effective with Chrome. If we discovered we needed to urgently disable zstd we could do it with a config push. However, in the case of WebView every app would have to start at least once in the broken configuration before the config push would take effect.

For this reason, I feel it is safer to enable it in Chrome first to achieve confidence that there aren't going to be any issues. However, there is nothing technically stopping us from enabling it in WebView at the same time if that is the consensus.

Torne (Richard Coles)

unread,
Jun 27, 2023, 1:04:33 PM6/27/23
to Adam Rice, Mihai Cîrlănaru, blink-dev, Nidhi Jaju, Dale Curtis, Sangwhan Moon, Yoav Weiss, Peter Beverloo
On Tue, 27 Jun 2023 at 11:42, Adam Rice <ri...@chromium.org> wrote:
Our mitigations if a serious compatibility issue should occur are more effective with Chrome. If we discovered we needed to urgently disable zstd we could do it with a config push. However, in the case of WebView every app would have to start at least once in the broken configuration before the config push would take effect.

This applies to every finch experiment in WebView, whether it's a killswitch or a launch. Rolling it out later in WebView just means we'll get any feedback about issues in WebVIew later.
Reply all
Reply to author
Forward
0 new messages