[erlang-questions] Memory leak in SSL

[Max Treskin]

Hello,

I have two different HTTPS-servers (different purposes, code, etc) built on top of cowboy/ranch, and both have memory leaks.

ETS table server_ssl_otp_session_cache has millions of records just after hours of work and consumes gigabytes of RAM.

What should I do to eliminate this? Is it bug or intended behaviour?

Thanks

[Dmitry Kolesnikov]

Hello,

Have you tried to disable reuse of ssl sessions?
{ssl_options, [{reuse_sessions, false}]}

And reduce the session time
-ssl session_lifetime 120

Best Regards,
Dmitry

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Ingela Andin]

Hi!

Depending on the version of the ssl application you may also set these values. I guess you have an older version as they default to 1000.

From doc:

"session_cache_client_max = integer() <optional>

Limits the growth of the clients session cache, that is how many sessions towards servers that are cached to be used by new client connections. If the maximum number of sessions is reached, the current cache entries will be invalidated regardless of their remaining lifetime. Defaults to 1000.

session_cache_server_max = integer() <optional>

Limits the growth of the servers session cache, that is how many client sessions are cached by the server. If the maximum number of sessions is reached, the current cache entries will be invalidated regardless of their remaining lifetime. Defaults to 1000."

Regards Ingela Erlang/OTP Team - Ericsson AB

[Max Treskin]

Thank you Dmitry! It works

[Danil Zagoskin]

Hi Ingela!

We've hit presumably the same bug on OTP 20.

That's what I found:

  1. This patch https://github.com/erlang/otp/commit/256e01ce80b3aadd63f303b9bda5722ad313220f
     -- we start invalidation only on Size == Max

  2. (flus...@127.0.0.1)33> (sys:get_state(ssl_manager))#state.session_cache_server_max.

     > 1000
  3. (flus...@127.0.0.1)37> redbug:start("ssl_session_cache:size/1 -> return", [{msgs, 2}]).
     ...
     ssl_session_cache:size/1 -> 8610379

Seems like on high load if we accept some new sessions while session_validation does its job, cache size limiting stops working.

--

Danil Zagoskin | z...@gosk.in

[Ingela Andin]

Hi!

2017-08-21 13:36 GMT+02:00 Danil Zagoskin <z...@gosk.in>:

Hi Ingela!

We've hit presumably the same bug on OTP 20.

That's what I found:

  1. This patch https://github.com/erlang/otp/commit/256e01ce80b3aadd63f303b9bda5722ad313220f
     -- we start invalidation only on Size == Max

  2. (flus...@127.0.0.1)33> (sys:get_state(ssl_manager))#state.session_cache_server_max.

     > 1000
  3. (flus...@127.0.0.1)37> redbug:start("ssl_session_cache:size/1 -> return", [{msgs, 2}]).
     ...
     ssl_session_cache:size/1 -> 8610379

Seems like on high load if we accept some new sessions while session_validation does its job, cache size limiting stops working.

Humm... maybe something was changed, that changed the criteria breaking the  assumption, let me ponder that some more.

Would it work for you if the test was changed to Size >= Max ?

[Danil Zagoskin]

> Would it work for you if the test was changed to Size >= Max ?

Yes. Yesterday I hot-loaded ssl_manager on affected system. Cache size slowly decreased to 1000 and stays in a range of 200-1100.

--

Danil Zagoskin | z...@gosk.in

[Ingela Andin]

Hi!

2017-08-22 12:50 GMT+02:00 Danil Zagoskin <z...@gosk.in>:

> Would it work for you if the test was changed to Size >= Max ?

Yes. Yesterday I hot-loaded ssl_manager on affected system. Cache size slowly decreased to 1000 and stays in a range of 200-1100.

And it now makes sense to me, I have included the fix in our builds. It probably will fix a wobbling test case.  Scheduled for inclusion in
OTP 20.1

Regards Ingela Erlang/OTP team - Ericsson AB