Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Memory issues with ssl handshake
406 views
Skip to first unread message
Thomas Eckert
Sep 12, 2012, 10:58:44 AM9/12/12
to
Hi,
Valgrind gives me a *lot* of messages like this
==19021== 2,056 bytes in 2 blocks are indirectly lost in loss record 186
of 190
==19021== at 0x68EAC8B: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==19021== by 0x6C472DB: default_malloc_ex (mem.c:79)
==19021== by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==19021== by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==19021== by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==19021== by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
==19021== by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
==19021== by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==19021== by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
==19021== by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
==19021== by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
==19021== by 0x6C98F57: RSA_verify (rsa_sign.c:284)
==19021== by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
==19021== by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
==19021== by 0x6964AB9: SSL_connect (ssl_lib.c:933)
==19021== by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
==19021== by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
==19021== by 0x8060D14: ssl_connect (ssl.c:1078)
Compared to the above they vary only in one line, e.g.
==19021== 1,024 bytes in 2 blocks are indirectly lost in loss record 181
of 190
==19021== at 0x68EAC8B: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==19021== by 0x6C472DB: default_malloc_ex (mem.c:79)
==19021== by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==19021== by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==19021== by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==19021== by 0x6C73D53: BN_copy (bn_lib.c:506)
==19021== by 0x6C7DFCA: BN_MONT_CTX_set (bn_mont.c:421)
==19021== by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==19021== by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
==19021== by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
==19021== by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
==19021== by 0x6C98F57: RSA_verify (rsa_sign.c:284)
==19021== by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
==19021== by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
==19021== by 0x6964AB9: SSL_connect (ssl_lib.c:933)
==19021== by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
==19021== by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
==19021== by 0x8060D14: ssl_connect (ssl.c:1078)
The application in question is a http proxy and is leaking memory
heavily. Actually, Valgrind also reports many entries as "still
reachable" instead of "indirectly lost" but even those regions are
mentioned by Valgrind like the above two blocks.
I doubt OpenSSL has such blatant memory leaks but I have been searching
for the leaks for some time now and just cannot find anything - also due
to the fact that Valgrind reports all the leaks like above, which is not
really helpful. I do realize that for detailed help I would need to give
some code on how the connections are set up but that code is pretty
large. So best would be some hint as to where to look for or what
regions of code to supply.
Looking for hints as to what is going wrong I went through some of the
OpenSSL functions mentioned above and after "leaving my code" and
entering OpenSSL code the only thing being passed around the functions
is the actual SSL object - and no buffers of any kind. So I suppose the
source of the problem lies in setting up the object, it's context or
what ever is stuck to it.
The first thing I checked on was how the SSL connections are torn down
in my application:
ERR_remove_state(0);
SSL_CTX_free(ssl->ctx);
SSL_free(ssl);
The first line was added only after starting the search for the leaks
and I'm still not sure if that line doesn't break anything.
I did find some minor leaks in my code along the way (e.g. a forgotten
ASN1_INTEGER_free()) but they were neither easy to find - because
Valgrind also reported them like the above two blocks, meaning in the
wrong place! - nor did they change much.
Does anyone have some tips for me ?
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openss...@openssl.org
Automated List Manager majo...@openssl.org
Valgrind gives me a *lot* of messages like this
==19021== 2,056 bytes in 2 blocks are indirectly lost in loss record 186
of 190
==19021== at 0x68EAC8B: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==19021== by 0x6C472DB: default_malloc_ex (mem.c:79)
==19021== by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==19021== by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==19021== by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==19021== by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
==19021== by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
==19021== by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==19021== by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
==19021== by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
==19021== by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
==19021== by 0x6C98F57: RSA_verify (rsa_sign.c:284)
==19021== by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
==19021== by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
==19021== by 0x6964AB9: SSL_connect (ssl_lib.c:933)
==19021== by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
==19021== by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
==19021== by 0x8060D14: ssl_connect (ssl.c:1078)
Compared to the above they vary only in one line, e.g.
==19021== 1,024 bytes in 2 blocks are indirectly lost in loss record 181
of 190
==19021== at 0x68EAC8B: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==19021== by 0x6C472DB: default_malloc_ex (mem.c:79)
==19021== by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==19021== by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==19021== by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==19021== by 0x6C73D53: BN_copy (bn_lib.c:506)
==19021== by 0x6C7DFCA: BN_MONT_CTX_set (bn_mont.c:421)
==19021== by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==19021== by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
==19021== by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
==19021== by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
==19021== by 0x6C98F57: RSA_verify (rsa_sign.c:284)
==19021== by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
==19021== by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
==19021== by 0x6964AB9: SSL_connect (ssl_lib.c:933)
==19021== by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
==19021== by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
==19021== by 0x8060D14: ssl_connect (ssl.c:1078)
The application in question is a http proxy and is leaking memory
heavily. Actually, Valgrind also reports many entries as "still
reachable" instead of "indirectly lost" but even those regions are
mentioned by Valgrind like the above two blocks.
I doubt OpenSSL has such blatant memory leaks but I have been searching
for the leaks for some time now and just cannot find anything - also due
to the fact that Valgrind reports all the leaks like above, which is not
really helpful. I do realize that for detailed help I would need to give
some code on how the connections are set up but that code is pretty
large. So best would be some hint as to where to look for or what
regions of code to supply.
Looking for hints as to what is going wrong I went through some of the
OpenSSL functions mentioned above and after "leaving my code" and
entering OpenSSL code the only thing being passed around the functions
is the actual SSL object - and no buffers of any kind. So I suppose the
source of the problem lies in setting up the object, it's context or
what ever is stuck to it.
The first thing I checked on was how the SSL connections are torn down
in my application:
ERR_remove_state(0);
SSL_CTX_free(ssl->ctx);
SSL_free(ssl);
The first line was added only after starting the search for the leaks
and I'm still not sure if that line doesn't break anything.
I did find some minor leaks in my code along the way (e.g. a forgotten
ASN1_INTEGER_free()) but they were neither easy to find - because
Valgrind also reported them like the above two blocks, meaning in the
wrong place! - nor did they change much.
Does anyone have some tips for me ?
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openss...@openssl.org
Automated List Manager majo...@openssl.org
Charles Mills
Sep 12, 2012, 12:29:56 PM9/12/12
to
Not sure if it help at all but FWIW I am using Windows pre-built OpenSSL
dll's with server & client certificates, CAs, DH keys, and a CRL, and I am
seeing no memory leaks whatsoever in Visual Studio 2010.
Charles
dll's with server & client certificates, CAs, DH keys, and a CRL, and I am
seeing no memory leaks whatsoever in Visual Studio 2010.
Charles
Michel
Sep 13, 2012, 4:09:13 AM9/13/12
to
Hi Thomas,
I don't know if it makes a real difference concerning memory leaks,
but I would have freed the TLS session *BEFORE* freing the context :
SSL_free(ssl);
...
SSL_CTX_free(ssl->ctx);
Does your app setup and free a context each time a client is connecting ?
Le 12/09/2012 16:58, Thomas Eckert a �crit :
I don't know if it makes a real difference concerning memory leaks,
but I would have freed the TLS session *BEFORE* freing the context :
SSL_free(ssl);
...
SSL_CTX_free(ssl->ctx);
Does your app setup and free a context each time a client is connecting ?
Le 12/09/2012 16:58, Thomas Eckert a �crit :
Michel
Sep 13, 2012, 6:30:43 AM9/13/12
to
Hi again Thomas,
Do you really need to free your context each time you free your TLS
session ?
I believe it is not needed and at least not usual.
If you need several *DIFFERENT* contexts, implying different TLS
configurations/setup, wich, I think, is not so common,
you can keep them 'alive' during all your app 'run', even in
multi-threaded programs.
It would allow you to access some activity informations like the ones
documented in :
http://www.openssl.org/docs/ssl/SSL_CTX_sess_number.html
Hope this helps,
Regards
Le 13/09/2012 10:39, Thomas a écrit :
> Hi Michel,
>
> Thanks for trying to help, I really appreciate it :-)
> and is destroyed together with the SSL session once the connection
> goes down. It is rather related to connections then to clients since
> one client can open several connections but I think you implied one
> connection per client and then the answer is 'yes'.
>
> I will try freeing the session before the context and come back with
> the results.
>
> Regards,
> Thomas
Do you really need to free your context each time you free your TLS
session ?
I believe it is not needed and at least not usual.
If you need several *DIFFERENT* contexts, implying different TLS
configurations/setup, wich, I think, is not so common,
you can keep them 'alive' during all your app 'run', even in
multi-threaded programs.
It would allow you to access some activity informations like the ones
documented in :
http://www.openssl.org/docs/ssl/SSL_CTX_sess_number.html
Hope this helps,
Regards
Le 13/09/2012 10:39, Thomas a écrit :
> Hi Michel,
>
> Thanks for trying to help, I really appreciate it :-)
>
> "Does your app setup and free a context each time a client is
> connecting ?"
>
> The context is created only when a client requests a HTTPS connection
> "Does your app setup and free a context each time a client is
> connecting ?"
>
> and is destroyed together with the SSL session once the connection
> goes down. It is rather related to connections then to clients since
> one client can open several connections but I think you implied one
> connection per client and then the answer is 'yes'.
>
> I will try freeing the session before the context and come back with
> the results.
>
> Regards,
> Thomas
Thomas
Sep 13, 2012, 4:39:54 AM9/13/12
to
Hi Michel,
Thanks for trying to help, I really appreciate it :-)
"Does your app setup and free a context each time a client is connecting ?"
The context is created only when a client requests a HTTPS connection
and is destroyed together with the SSL session once the connection goes
down. It is rather related to connections then to clients since one
client can open several connections but I think you implied one
connection per client and then the answer is 'yes'.
I will try freeing the session before the context and come back with the
results.
Regards,
Thomas
Thanks for trying to help, I really appreciate it :-)
"Does your app setup and free a context each time a client is connecting ?"
The context is created only when a client requests a HTTPS connection
and is destroyed together with the SSL session once the connection goes
down. It is rather related to connections then to clients since one
client can open several connections but I think you implied one
connection per client and then the answer is 'yes'.
I will try freeing the session before the context and come back with the
results.
Regards,
Thomas
On 09/13/2012 10:09 AM, Michel wrote:
> Hi Thomas,
>
> I don't know if it makes a real difference concerning memory leaks,
> but I would have freed the TLS session *BEFORE* freing the context :
> SSL_free(ssl);
> ...
> SSL_CTX_free(ssl->ctx);
> Hi Thomas,
>
> I don't know if it makes a real difference concerning memory leaks,
> but I would have freed the TLS session *BEFORE* freing the context :
> SSL_free(ssl);
> ...
> SSL_CTX_free(ssl->ctx);
> Does your app setup and free a context each time a client is connecting ?
>
> Le 12/09/2012 16:58, Thomas a �crit :
>
Dominik Oepen
Sep 13, 2012, 7:19:36 AM9/13/12
to
On 12.09.2012 16:58, Thomas Eckert wrote:
> Hi,
>
> Valgrind gives me a *lot* of messages like this
[snip]
> Hi,
>
> Valgrind gives me a *lot* of messages like this
> Does anyone have some tips for me ?
Have you seen the FAQ entry regarding valgrind:
http://www.openssl.org/support/faq.html#PROG14 ?
Regards,
Dominik
Thomas
Sep 17, 2012, 11:35:34 AM9/17/12
to
Hi again,
I've changed the code to reuse the SSL contexts but in terms of memory
consumption/release it did not change much - if anything at all. By the
way, is there a way to "unload" a certificate once it has been loaded
into a SSL context via SSL_CTX_use_certificate() ? I didn't find
anything in the docs and simply specifying NULL as cert parameter caused
a crash in OpenSSL.
The only places left that cause memory leaks are reported inside OpenSSL
as in
at 0x68EAC8B: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==27041== by 0x6C472DB: default_malloc_ex (mem.c:79)
==27041== by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==27041== by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==27041== by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==27041== by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
==27041== by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
==27041== by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==27041== by 0x6C95B56: RSA_eay_mod_exp (rsa_eay.c:782)
==27041== by 0x6C96422: RSA_eay_private_decrypt (rsa_eay.c:565)
==27041== by 0x6C97EDF: RSA_private_decrypt (rsa_lib.c:303)
==27041== by 0x6942918: ssl3_get_client_key_exchange (s3_srvr.c:2038)
==27041== by 0x6946693: ssl3_accept (s3_srvr.c:529)
==27041== by 0x69513CA: ssl3_read_bytes (s3_pkt.c:941)
==27041== by 0x694C688: ssl3_read_internal (s3_lib.c:3274)
==27041== by 0x69642E8: SSL_read (ssl_lib.c:954)
Sometimes these are flagged "still reachable" and sometimes "indirectly
lost", usually both types are reported as I get a large amount of these
traces. One thing I noticed is that all goes well if I cause the code to
run sequentially (e.g. cause requests to come one ater another). Yet it
starts eating up memory like crazy if I cause several (HTTPS) requests
to come at once.
I'm at a loss here. Valgrind insists the leaks happen in OpenSSL code.
I'll be happy to supply more information if anyone has an idea of how to
approach this.
Regards,
Thomas
On 09/13/2012 12:30 PM, Michel wrote:
> Hi again Thomas,
>
> Do you really need to free your context each time you free your TLS
> session ?
> I believe it is not needed and at least not usual.
> If you need several *DIFFERENT* contexts, implying different TLS
> configurations/setup, wich, I think, is not so common,
> you can keep them 'alive' during all your app 'run', even in
> multi-threaded programs.
> It would allow you to access some activity informations like the ones
> documented in :
> http://www.openssl.org/docs/ssl/SSL_CTX_sess_number.html
>
> Hope this helps,
> Regards
>
> Le 13/09/2012 10:39, Thomas a écrit :
I've changed the code to reuse the SSL contexts but in terms of memory
consumption/release it did not change much - if anything at all. By the
way, is there a way to "unload" a certificate once it has been loaded
into a SSL context via SSL_CTX_use_certificate() ? I didn't find
anything in the docs and simply specifying NULL as cert parameter caused
a crash in OpenSSL.
The only places left that cause memory leaks are reported inside OpenSSL
as in
at 0x68EAC8B: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==27041== by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==27041== by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==27041== by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==27041== by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
==27041== by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
==27041== by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==27041== by 0x6C95B56: RSA_eay_mod_exp (rsa_eay.c:782)
==27041== by 0x6C96422: RSA_eay_private_decrypt (rsa_eay.c:565)
==27041== by 0x6C97EDF: RSA_private_decrypt (rsa_lib.c:303)
==27041== by 0x6942918: ssl3_get_client_key_exchange (s3_srvr.c:2038)
==27041== by 0x6946693: ssl3_accept (s3_srvr.c:529)
==27041== by 0x69513CA: ssl3_read_bytes (s3_pkt.c:941)
==27041== by 0x694C688: ssl3_read_internal (s3_lib.c:3274)
==27041== by 0x69642E8: SSL_read (ssl_lib.c:954)
Sometimes these are flagged "still reachable" and sometimes "indirectly
lost", usually both types are reported as I get a large amount of these
traces. One thing I noticed is that all goes well if I cause the code to
run sequentially (e.g. cause requests to come one ater another). Yet it
starts eating up memory like crazy if I cause several (HTTPS) requests
to come at once.
I'm at a loss here. Valgrind insists the leaks happen in OpenSSL code.
I'll be happy to supply more information if anyone has an idea of how to
approach this.
Regards,
Thomas
On 09/13/2012 12:30 PM, Michel wrote:
> Hi again Thomas,
>
> Do you really need to free your context each time you free your TLS
> session ?
> I believe it is not needed and at least not usual.
> If you need several *DIFFERENT* contexts, implying different TLS
> configurations/setup, wich, I think, is not so common,
> you can keep them 'alive' during all your app 'run', even in
> multi-threaded programs.
> It would allow you to access some activity informations like the ones
> documented in :
> http://www.openssl.org/docs/ssl/SSL_CTX_sess_number.html
>
> Hope this helps,
> Regards
>
> Le 13/09/2012 10:39, Thomas a écrit :
>> Hi Michel,
>>
>> Thanks for trying to help, I really appreciate it :-)
>>
>> "Does your app setup and free a context each time a client is
>> connecting ?"
>>
>> The context is created only when a client requests a HTTPS connection
>> and is destroyed together with the SSL session once the connection
>> goes down. It is rather related to connections then to clients since
>> one client can open several connections but I think you implied one
>> connection per client and then the answer is 'yes'.
>>
>> I will try freeing the session before the context and come back with
>> the results.
>>
>> Regards,
>> Thomas
>>
>
>>
>> Thanks for trying to help, I really appreciate it :-)
>>
>> "Does your app setup and free a context each time a client is
>> connecting ?"
>>
>> The context is created only when a client requests a HTTPS connection
>> and is destroyed together with the SSL session once the connection
>> goes down. It is rather related to connections then to clients since
>> one client can open several connections but I think you implied one
>> connection per client and then the answer is 'yes'.
>>
>> I will try freeing the session before the context and come back with
>> the results.
>>
>> Regards,
>> Thomas
>>
>
0 new messages