Memory leak when CRL hot-reloading with Envoy 1,27,0

[mak long]

Hi Envoy Team, 

We are using Enovy 1.27.0's new function of CRL hot reload with validation_context_sds_secret_config.

When the new CRL file is available, we copy it to the location /etc/ssl/crl/crl-tmp.pem, where the folder(etc/ssl/crl) is watched .
We will see the memory leak immediately after triggers the CRL hot relad by issuing below command.

ln -sf /etc/ssl/crl/crl-tmp.pem /etc/ssl/crl/crl.pem

The memory size leaked align with the crl.pem file size. We see 1G memory leak with CRL file size 145Mi. 

Is there anyone known this issue in this area? 

Below is the configurations:

envoy.yaml: |-

node:

id: envoyid

cluster: envoycluster

static_resources:

listeners:

- address:

socket_address:

address: 0.0.0.0

port_value: 443

listener_filters:

- name: envoy.listener.proxy_protocol

typed_config:

"@type": type.googleapis.com/envoy.extensions.filters.listener.proxy_protocol.v3.ProxyProtocol

filter_chains:

- filters:

- name: envoy.filters.network.http_connection_manager

typed_config:

"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager

codec_type: AUTO

stat_prefix: ingress_https

http2_protocol_options:

initial_stream_window_size: 4000000

initial_connection_window_size: 4000000

stream_idle_timeout: 1800s

forward_client_cert_details: Sanitize_set

set_current_client_cert_details:

subject: True

use_remote_address: True

route_config:

name: local_route

virtual_hosts:

- name: https

domains:

- "*"

routes:

- match:

safe_regex:

google_re2: {}

regex: "*"

route:

cluster: my-cluster

timeout: 15000s

http_filters:

- name: envoy.filters.http.router

typed_config:

"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

transport_socket:

name: envoy.transport_sockets.tls

typed_config:

"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext

require_client_certificate: True

common_tls_context:

tls_certificates:

- certificate_chain:

filename: "/etc/ssl/envoy/tls.crt"

private_key:

filename: "/etc/ssl/envoy/tls.key"

tls_params:

tls_minimum_protocol_version: "TLSv1_2"

validation_context_sds_secret_config:

name: server_ca

sds_config:

path_config_source:

path: /etc/envoy/sds_server_ca.yaml

clusters:

- name: my-cluster

connect_timeout: 0.5s

type: STRICT_DNS

dns_lookup_family: V4_ONLY

lb_policy: LEAST_REQUEST

#http_protocol_options: {}

http2_protocol_options:

initial_stream_window_size: 4000000

initial_connection_window_size: 4000000

load_assignment:

cluster_name: my-cluster

endpoints:

- lb_endpoints:

- endpoint:

address:

socket_address:

address: xxxx

port_value: xxxx

transport_socket:

name: envoy.transport_sockets.tls

typed_config:

"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext

sni: xxxxx

common_tls_context:

validation_context:

tls_certificates:

- certificate_chain:

filename: "/etc/ssl/enforcer/tls.crt"

private_key:

filename: "/etc/ssl/enforcer/tls.key"

sds_server_ca.yaml: |-

resources:

- '@type': type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret

name: server_ca

validation_context:

trusted_ca:

filename: /etc/ssl/envoy-ca/ca.pem

crl:

filename: /etc/ssl/crl/crl.pem

[tero....@gmail.com]

Hi,

In more detail, how did you observe the memory leak?

When swapping the CRL file multiple times, do you see the memory use constantly increasing at every swap?

-- 

Tero