Intent to Experiment: Zstd Content-Encoding

[Nidhi Jaju]

Contact emails

nidh...@chromium.org

Explainer

https://docs.google.com/document/d/1aDyUw4mAzRdLyZyXpVgWvO-eLpc4ERz7I_7VDIPo9Hc/edit?usp=sharing

Specification

https://datatracker.ietf.org/doc/html/rfc8878

Design docs

https://docs.google.com/document/d/14dbzMpsYPfkefAJos124uPrlkvW7jyPJhzjujSWws2k/edit?usp=sharing

Summary

Zstandard, or “zstd”, is a data compression mechanism described in RFC8878. It is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. The "zstd" token was added as an IANA-registered Content-Encoding token as per https://datatracker.ietf.org/doc/html/rfc8878#name-content-encoding. Adding support for "zstd" as a Content-Encoding will help load pages faster and use less bandwidth.

Blink component

Internals>Network

TAG review

None

TAG review status

Not applicable

Risks

Interoperability and Compatibility

Servers that have a broken implementation of zstd might exist, but the risk of this is small. Additionally, middleware and middleboxes like virus checkers that intercept HTTPS connections might not support zstd, but might fail to remove it from the Accept-Encoding header in the request.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/775)

WebKit: No signal (https://github.com/WebKit/standards-positions/issues/168)

Web developers: Positive (https://crbug.com/1246971) Facebook (Yann) and Akamai (Nic) seem to be positive about zstd content-encoding in the browser. Facebook is also excited to test the feature.

Other signals:

Security

CRIME and BREACH mean that the resource being compressed can be considered readable by the document deploying them. That is bad if any of them contains information that the document cannot already obtain by other means. An attacker may provide correctly formed compressed frames with unreasonable memory requirements, and dictionaries may interact unexpectedly with a decoder, leading to possible memory or other resource-exhaustion attacks. It is possible to store arbitrary user metadata in skippable frames, so they can be used as a watermark to track the path of the compressed payload. It is important to note that these concerns apply to all compression formats, not just zstd.

To mitigate these risks, similar to Brotli, we'll be advertising support for "zstd" encoding only if transferred data is opaque to proxy, to ensure that resources don't contain private data that the origin cannot read otherwise.

Adding zstd to Chromium adds a large new code surface that processes untrusted data, which inevitably brings risks of new security holes. However, this is mitigated by the extensive fuzzing and security analysis done on zstd by Google and other community members.

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

Goals for experimentation

Understand the impact of supporting zstd content-encoding in the browser on performance and if there's breakage.

Ongoing technical constraints

Debuggability

No special support needed. Zstd content-encoding support will be exposed to the devtools protocol, so developers are able to override it and view the headers from the inspector.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

No

Flag name on chrome://flags

enable-zstd-content-encoding

Finch feature name

ZstdContentEncoding

Requires code in //chrome?

True

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1246971

Launch bug

https://launch.corp.google.com/launch/4266275

Estimated milestones

Shipping on desktop

117

Shipping on Android

117

Shipping on WebView

117

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/6186023867908096

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMZNYANd_E77W1ki--h_XJM-%2B_fA3w1CriGgJmnbh1N3LwRDtw%40mail.gmail.com

This intent message was generated by Chrome Platform Status.

[Mike Taylor]

Can you clarify the proposed experiment (presumably N% of stable?) and the desired milestones? Thanks!

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMZNYANR%3DisgShRGxHQMgn-2W1%2BteA81AtyRu14v7s_kk2C90Q%40mail.gmail.com.

[Nidhi Jaju]

Hi Mike,

The proposed experiment is to run an A/B experiment on Canary/Dev, Beta, and then 1% of Stable on M117.

Best,

Nidhi

[Rick Byers]

Seems like a great thing to experiment with! LGTM

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMZNYANXMaVdvHKvgswVQpZKe3%2BE-mknMi%2B8X6kKpq%2BbZZfUig%40mail.gmail.com.

[powturbo]

As an indication, a pure in-memory benchmark without web server, network or other overhead.

TurboBench: Dynamic/Static web content compression benchmark

[Lee Benson]

This actually triggered a small internal incident at Datadog on a private API that wasn't set-up to handle Zstandard encoding properly.

So two things:

1. Firstly, thanks for bringing this this our attention! We were able to patch the error ahead of a more general roll-out to public users.

2. Is there a way we can explicitly opt in to these kinds of A/B tests to preemptively catch more of these breaking changes?

This would be particularly useful to our synthetics.

Thanks,

Lee

[Nidhi Jaju]

Hi Lee,

I'm glad you were able to patch the error ahead of a more general rollout!

Regarding your second point, there's two ways you can explicitly turn on the feature:
1. If there's a "Flag name on chrome://flags" in the Intent to Experiment, you can manually turn on the feature by navigating to chrome://flags. So in this case, you could force your browser to turn on chrome://flags/#enable-zstd-content-encoding.
2. If there's a "Finch feature name", you can pass `--enable-features=FeatureName` (i.e. `--enable-features=ZstdContentEncoding` in this case) when running Chrome from the command line.

You can also keep an eye out for ongoing experiments and planned features at https://chromestatus.com/roadmap.

Hopefully that helps.

Regards,

Nidhi

[Mark Eagin]

Is there a method to force exclusion for an experiment such as this?  For example, we would rather not have 1% of our production environment impacted as it has been here.  We would rather have our test groups and developers be available and/or force enable these features to test them.  Embedding the flag setting to Disabled does not seem to function since the flag is part of the user local profile which overrides any default setting.