Security release of Envoy 1.14.3, 1.13.3, and 1.12.5 is now available

150 views
Skip to first unread message

Tony Allen

unread,
Jun 30, 2020, 3:10:34 PM6/30/20
to envoy-secur...@googlegroups.com, envoy-a...@googlegroups.com, envoy-security, envoy-ma...@googlegroups.com
Hello Envoy Community,

The Envoy security team would like to announce the availability of Envoy 1.14.3, 1.13.3, and 1.12.5.
This addresses the following CVE(s):
  • CVE-2020-12603 (CVSS score 7.0, High): Envoy through 1.14.2/1.13.2/1.12.4 may consume excessive amounts of memory when proxying HTTP/2 requests or responses with many small (e.g., 1 byte) data frames.
  • CVE-2020-12605 (CVSS score 7.0, High): Envoy through 1.14.2/1.13.2/1.12.4 may consume excessive amounts of memory when processing HTTP/1.1 headers with long field names or requests with long URLs.
  • CVE-2020-8663 (CVSS score 7.0, High): Envoy version 1.14.2/1.13.2/1.12.4 or earlier may exhaust file descriptors and/or memory when accepting too many connections.
  • CVE-2020-12604 (CVSS score 5.3, Medium): Envoy through 1.14.2/1.13.2/1.12.4 is susceptible to increased memory usage in the case where an HTTP/2 client requests a large payload but does not send enough window updates to consume the entire stream and does not reset the stream. The attacker can cause data associated with many streams to be buffered forever.
Upgrading to 1.14.3, 1.13.3, or 1.12.5 is encouraged to fix these issues.

Documentation for all versions can be found here: https://www.envoyproxy.io/docs

For v1.14.3:
Docker images: docker pull envoyproxy/envoy:v1.14.3



Am I vulnerable?
Run `envoy --version` and if it indicates a base version matching or older than v1.14.2, v1.13.2, or v1.12.4 you are running a vulnerable version.

How do I upgrade?
Update to v1.14.3, v1.13.3, or v1.12.5 via your Envoy distribution or rebuild from the Envoy GitHub source at the version's tag or HEAD @ master.

Vulnerability Details

CVE-2020-8663: Connection flood exhausts memory and file descriptors
Envoy version 1.14.2/1.13.2/1.12.4 or earlier may exhaust file descriptors and/or memory when accepting too many connections. Large numbers of connections may be opened against Envoy, with no data sent, causing Envoy to eventually run out of file descriptors and crash.

CVE-2020-12605: Inconsistent enforcement of HTTP/1.1 request/response header size limit
Envoy version 1.14.2/1.13.2/1.12.4 or earlier may consume excessive amounts of memory beyond the configured max header size limit when parsing HTTP/1.1 headers due to missing incremental size validation when parsing header field names, and not including the request URL in the computed request header size. The missing header field name size check can result in excessive buffering up to a hard-coded 32MB limit until timeout.  The missing request URL size check can result in Envoy attempting to route match and proxy HTTP/1.1 requests with URLs up to a hard-coded 32MB limit, which could result in excess memory usage or performance problems in regex route matches.

CVE-2020-12603: Excessive CPU and/or memory usage when proxying HTTP/2 bodies
Envoy version 1.14.2/1.13.2/1.12.4 or earlier may consume excessive amounts of memory when proxying HTTP/2 requests or responses with many small (i.e. 1 byte) chunks. Envoy allocates a separate buffer fragment with the size rounded to the nearest 4Kb for each data frame that is written to the output buffer from the connection read-dispatch call stack. As such processing requests or responses with a lot of small chunks may result in extremely high memory overhead if the peer is slow or unable to read proxied data. The memory overhead could be one to two orders of magnitude more than configured buffer limits.

CVE-2020-12604: HTTP/2: pending send data OOM/stats/timeout issues

Envoy employs HTTP/2 stream flow control as configured by the `initial_stream_window_size` setting that can be configured both on downstream HTTP listeners as well as upstream HTTP clusters. Envoy also limits the number of simultaneous downstream streams as configured by the `max_concurrent_streams` setting.


Envoy can consume and never release stream memory in the following case:

  1. Untrusted client opens streams up to the `max_concurrent_streams` limit.

  2. Untrusted client requests a resource that will generate a large response.

  3. Depending on the configuration of `initial_stream_window_size` for the upstream cluster, Envoy will buffer approximately that amount of memory on behalf of the stream.

  4. If the downstream client does not open receive window, Envoy will start flow control and stop allowing data from upstream.

  5. However, there existed a case in which Envoy would receive the final data of the stream from upstream (pending window), but not have enough open window to send the data to downstream. This data would be buffered forever and not covered by any timeout.


The effectiveness of this attack is limited by both the  `max_concurrent_streams` and `initial_stream_window_size` settings. However, a malicious client with knowledge of backend configuration may be able to craft a sequence of connections and streams that would cause Envoy to use substantial memory and never free it. (By opening up `max_concurrent_streams` - 1 streams per connection and using the final stream for normal periodic requests, the connection idle timer would never fire.)


The fix for this CVE is that Envoy will now use the `stream_idle_timeout` to cover the final data flush in which data may be buffered pending available downstream window. If downstream window is never opened, Envoy will reset the stream and release the buffered data once the idle timeout fires.


Thank you
Thank you to Piotr Sikora, Antonio Vicente, Wenlei (Frank) He, Bartosz Borkowski, Matt Klein, Lizan Zhou, Harvey Tuch, Dmitri Dolguikh, and John Plevyak for making this release happen.

Thanks,
Tony Allen (on behalf of the Envoy security team and maintainers)
Reply all
Reply to author
Forward
0 new messages