TLS, BIOs, SSL_read/write
Nate Leon
I am in the process of converting an SMTP/TLS server to use Async IO.
(using IO Completion Ports on Windows)
As such, the previously working style of using SSL_accept, select, and
SSL_read / SSL_write is no longer sufficient.
Now that I am using WSARecv to do the read, my app is notified when a
buffer is filled with (encrypted) data.
Somebody suggested I stuff that data into a BIO buffer and read it out of that.
I attempted to use BIO_read but without success.
I seem to have that half working now, looking roughly like this:
(over-simplified for readability)
// SMTP client requests STARTTLS, server takes these actions:
BIO* m_bioSckt = BIO_new_socket((int)scktUpstream, BIO_NOCLOSE);
SSL* m_ssl = SSL_new(ctx);
SSL_set_accept_state(m_ssl);
SSL_set_bio(m_ssl, m_bioSckt, m_bioSckt);
// Server sends "220 go ahead" to client, and waits for Client/Server
to negotiate handshake
int nRetCode = SSL_accept(m_ssl);
// This succeeds - client reports: "New, TLSv1/SSLv3, Cipher is
AES256-SHA, etc..."
// Now client sends command (e.g. "EHLO example.org") which needs to
be decrypted
BIO* m_bioMem = BIO_new_mem_buf(encryptedData, nEncDataSize);
SSL_set_bio(m_ssl, m_bioMem, NULL);
char decryptedData[4096];
int numBytesRead = SSL_read(m_ssl, decryptedData, sizeof(decryptedData));
// SMTP server processes decryptedData and takes appropriate action -
e.g. sends a "250 OK" response
// That response needs to be encrypted before it is sent (WSASend)
BIO* bioMem = BIO_new(BIO_f_buffer());
SSL_set_bio(m_ssl, NULL, bioMem);
int numBytesWritten = SSL_write(m_ssl, responseData, nRespDataSize); // fails
BIO_flush(bioMem);
I am testing using the openssl client:
openssl s_client -starttls smtp -connect localhost:25 -crlf -msg -debug
The SSL_write seems to fail completely. :(
Am I on the right track here?
Is it the optimal way to go about it?
Also surprising, (at least to me) is that BIO_new_socket and
BIO_new_mem_buf return the identical address in memory,
which makes me think I am really not understanding how this is supposed to work.
Any pointers are much appreciated.
TIA,
n8
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openss...@openssl.org
Automated List Manager majo...@openssl.org
Galina Goncharov
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Hello,
as I can see from code you first set readBIO and writeBIO to
socket(scktUpstream ) and that makes TLS negotiation to succeed. Later you
set SSL readBIO to mem_bio, and writeBIO to NULL and then try to use
writeBIO ( calling SSL_write) - sure it will fail. During negotiation do you
receive server responses on scktUpstream? I think so, otherwise I don't see
how it would work. Have you tried to not reset SSL BIO with memBIO but have
blocking SSL_read - it will return once you socket finish reading.and then
just use SSL_write as you do, just again without reseting SSL BIO.
hope this will help.
Galina
--000e0cd1a782af3e5504650c17dc
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Hello,<div><br></div><div>as I can see from code you first set readBIO and =
writeBIO to socket(scktUpstream )=A0and that makes TLS negotiation to succe=
ed. Later you set SSL readBIO to mem_bio, and writeBIO to NULL and then try=
to use writeBIO ( calling SSL_write) - sure it will fail. During negotiati=
on do you receive server=A0responses=A0on=A0scktUpstream? =A0I think so, ot=
herwise I don't see how it would work. Have you tried to not reset SSL =
BIO with memBIO but have blocking SSL_read =A0- it will return once you soc=
ket finish reading.and then just use SSL_write as you do, just =A0again wit=
hout reseting SSL =A0BIO.</div>
<div><br></div><div>hope this will help.</div><div>Galina</div><div><br></d=
iv><div><br></div><div><br></div><div><br><br><div class=3D"gmail_quote">On=
Fri, Mar 13, 2009 at 7:52 PM, Nate Leon <span dir=3D"ltr"><<a href=3D"m=
ailto:n8l...@gmail.com">n8l...@gmail.com</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;">Greetings,<br>
I am in the process of converting an SMTP/TLS server to use Async IO.<br>
(using IO Completion Ports on Windows)<br>
As such, the previously working style of using SSL_accept, select, and<br>
SSL_read / SSL_write is no longer sufficient.<br>
<br>
Now that I am using WSARecv to do the read, my app is notified when a<br>
buffer is filled with (encrypted) data.<br>
Somebody suggested I stuff that data into a BIO buffer and read it out of t=
hat.<br>
I attempted to use BIO_read but without success.<br>
I seem to have that half working now, looking roughly like this:<br>
(over-simplified for readability)<br>
<br>
// SMTP client requests STARTTLS, server takes these actions:<br>
BIO* m_bioSckt =3D BIO_new_socket((int)scktUpstream, BIO_NOCLOSE);<br>
SSL* m_ssl =3D SSL_new(ctx);<br>
SSL_set_accept_state(m_ssl);<br>
SSL_set_bio(m_ssl, m_bioSckt, m_bioSckt);<br>
<br>
// Server sends "220 go ahead" to client, and waits for Client/Se=
rver<br>
to negotiate handshake<br>
int nRetCode =3D SSL_accept(m_ssl);<br>
<br>
// This succeeds - client reports: "New, TLSv1/SSLv3, Cipher is<br>
AES256-SHA, etc..."<br>
// Now client sends command (e.g. "EHLO <a href=3D"http://example.org"=
target=3D"_blank">example.org</a>") which needs to<br>
be decrypted<br>
BIO* m_bioMem =3D BIO_new_mem_buf(encryptedData, nEncDataSize);<br>
SSL_set_bio(m_ssl, m_bioMem, NULL);<br>
char decryptedData[4096];<br>
int numBytesRead =3D SSL_read(m_ssl, decryptedData, sizeof(decryptedData));=
<br>
<br>
// SMTP server processes decryptedData and takes appropriate action -<br>
e.g. sends a "250 OK" response<br>
// That response needs to be encrypted before it is sent (WSASend)<br>
BIO* bioMem =3D BIO_new(BIO_f_buffer());<br>
SSL_set_bio(m_ssl, NULL, bioMem);<br>
int numBytesWritten =3D SSL_write(m_ssl, responseData, nRespDataSize); =A0 =
// fails<br>
BIO_flush(bioMem);<br>
<br>
I am testing using the openssl client:<br>
openssl s_client -starttls smtp -connect localhost:25 -crlf -msg -debug<br>
<br>
The SSL_write seems to fail completely. :(<br>
Am I on the right track here?<br>
Is it the optimal way to go about it?<br>
Also surprising, (at least to me) is that BIO_new_socket and<br>
BIO_new_mem_buf return the identical address in memory,<br>
which makes me think I am really not understanding how this is supposed to =
work.<br>
<br>
Any pointers are much appreciated.<br>
<br>
TIA,<br>
n8<br>
______________________________________________________________________<br>
OpenSSL Project =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0 <a href=3D"http://www.openssl.org" target=3D"_blank">http://www.openss=
l.org</a><br>
User Support Mailing List =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0<a href=3D=
"mailto:openss...@openssl.org">openss...@openssl.org</a><br>
Automated List Manager =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
<a href=3D"mailto:majo...@openssl.org">majo...@openssl.org</a><br>
</blockquote></div><br></div>
--000e0cd1a782af3e5504650c17dc--
Galina Goncharov
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
an other thought - I didn't notice your third SSL_set_bio(m_ssl, NULL,
bioMem); so my suggestion wasn't quite correct. and its always usfull to
get error by calling SSL_get_error() - it can point to right direction.
and below is from other thread:
=====
The BIO_new_mem_buf creates a read-only buffer.
If you want to write to memory use
bio = BIO_new(BIO_s_mem());
and use BIO_get_mem_ptr to get a pointer to the buffer.
===
Galina
--000e0cd1562453ba91046534ae57
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
<span class=3D"Apple-style-span" style=3D"font-family: arial, helvetica, sa=
ns-serif;">an other thought - I didn't notice your third=A0SSL_set_bio(=
m_ssl, NULL, bioMem);=A0so my suggestion wasn't quite correct. =A0</spa=
n><div>
<span class=3D"Apple-style-span" style=3D"font-family: arial, helvetica, sa=
ns-serif;">and its always usfull to get error by calling=A0SSL_get_error() =
- it can point to right direction.</span><div><div><span class=3D"Apple-sty=
le-span" style=3D"font-family: arial, helvetica, sans-serif;"><br>
</span></div><div><span class=3D"Apple-style-span" style=3D"font-family: ar=
ial, helvetica, sans-serif;">and below is from other thread:</span></div><d=
iv><br></div><div>=3D=3D=3D=3D=3D</div><div><span class=3D"Apple-style-span=
" style=3D"font-family: arial, helvetica, sans-serif;">The BIO_new_mem_buf =
creates a read-only buffer.<br>
If you want to write to memory use<br>bio =3D BIO_new(BIO_s_mem());<br>and =
use BIO_get_mem_ptr to get a pointer to the buffer.<br>=3D=3D=3D</span></di=
v><div><br></div><div><span class=3D"Apple-style-span" style=3D"font-family=
: arial, helvetica, sans-serif;"><br>
</span></div><div>Galina</div><div><span class=3D"Apple-style-span" style=
=3D"font-family: verdana; "><br></span><br><br><div class=3D"gmail_quote">O=
n Fri, Mar 13, 2009 at 7:52 PM, Nate Leon <span dir=3D"ltr"><<a href=3D"=
mailto:n8l...@gmail.com">n8l...@gmail.com</a>></span> wrote:<br>
</blockquote></div><br></div></div></div>
--000e0cd1562453ba91046534ae57--
Victor Duchovni
> an other thought - I didn't notice your third SSL_set_bio(m_ssl, NULL,
> bioMem); so my suggestion wasn't quite correct. and its always usfull to
> get error by calling SSL_get_error() - it can point to right direction.
> and below is from other thread:
Do read about the BIO_pair interface and find examples of the use of
bio_pairs with SSL.
--
Viktor.
Nate Leon
BIO* bioMem = BIO_new(BIO_f_buffer());
which doesn't really make sense.
Indeed, this is working:
BIO* bioMem = BIO_new(BIO_s_mem());
SSL_set_bio(m_ssl, NULL, bioMem);
SSL_write(m_ssl, responseData, nRespDataSize);
My SSL object (m_ssl) now has wbio set to the new "memory buffer" BIO (bioMem),
and bioMem shows the correct number of bytes written in num_write.
Good catch - many thanks!
n8
--------------------------------------------------------------------------------
From: owner-ope...@openssl.org
[mailto:owner-ope...@openssl.org] On Behalf Of Galina Goncharov
Sent: Sunday, March 15, 2009 9:14 PM
To: openss...@openssl.org
Subject: Re: TLS, BIOs, SSL_read/write
an other thought - I didn't notice your third SSL_set_bio(m_ssl, NULL,
bioMem); so my suggestion wasn't quite correct.
and its always usfull to get error by calling SSL_get_error() - it can
point to right direction.
and below is from other thread:
=====
The BIO_new_mem_buf creates a read-only buffer.
If you want to write to memory use
bio = BIO_new(BIO_s_mem());
and use BIO_get_mem_ptr to get a pointer to the buffer.
===
Galina
Ger Hobbelt
> SSL_set_bio(m_ssl, NULL, bioMem);
and it's counterpart:
SSL_set_bio(m_ssl, m_bioMem, NULL);
are NOT how this sort of thing is done. This is EXTREMELY DANGEROUS
(or should I say: fatal?) coding as you forcibly remove either read or
write BIO facilities for an active SSL connection.
This indicates that you assume you are in total control over when SSL
will [need to] write or read (and /not/ the other way) at /all/ time.
Which is a falsehood.
Your system can be taken down easily by any client which triggers a
renegotiation. And this is only one scenario, where read can cause a
write and vice versa.
BIO pairs have been created to provide for such bidirectional I/O and
you should both read up on the SSL_read/SSL_write documentation (and
heed the notes which are mentioned for nonblocking I/O: you will see
their effects at all times as you are plugging in your own I/O
mechanism at the backend (IOCP)). Also check out how BIO pairs et al
are used for in-memory SSL sessions, such as shown in the ssltest
application which comes OpenSSL. (There are more sample apps which use
in-memory BIOs for performing SSL communications.)
A quick test to check if you handle renegotiations at all in your IOCP
backend flow is configuring your server to trigger a renegotiation by
itself by calling BIO_set_ssl_renegotiate_bytes() with a trigger
setting of, say, a few KBytes of received data - a small amount which
will make it happen quickly and often so you can test how things go
down.
But be aware that surviving this does not guarantee you'll survive
client-triggered renegotiation - which is not an uncommon thing,
especially when transfer amounts rise into the multi-megabytes per
connection.
Remember: SSL is not just shoving your bytes across the line in
encrypted form. It adds a *protocol* *layer* on top of sockets, which
should be facilitated for the entire lifetime of each socket
connection.
On Mon, Mar 16, 2009 at 9:55 PM, Nate Leon <n8l...@gmail.com> wrote:
> That was the trick - I was trying to write to a =A0:
> =A0BIO* bioMem =3D BIO_new(BIO_f_buffer());
>
> which doesn't really make sense.
>
> Indeed, this is working:
> =A0 =A0BIO* bioMem =3D BIO_new(BIO_s_mem());
> =A0 =A0SSL_set_bio(m_ssl, NULL, bioMem);
> =A0 =A0SSL_write(m_ssl, responseData, nRespDataSize);
>
> My SSL object (m_ssl) now has wbio set to the new "memory buffer" BIO (bi=
oMem),
> and bioMem shows the correct number of bytes written in num_write.
>
> Good catch - many thanks!
> n8
--=20
Met vriendelijke groeten / Best regards,
Ger Hobbelt
--------------------------------------------------
web: http://www.hobbelt.com/
http://www.hebbut.net/
mail: g...@hobbelt.com
mobile: +31-6-11 120 978
--------------------------------------------------
Nate Leon
code - I was just happy to see something working. :)
I was trying to figure out a way to call SSL_set_bio once per session
with both read and write buffers, but I am stumped there since this
call:
m_bioMem = BIO_new_mem_buf(encryptedData, nEncDataSize)
needs to change based on the data that just came off the wire.
Which naturally brings us to the BIO pairs issue. That is the path I
started going down... chaining a cipher BIO with a memory BIO, and
planning to write the encrypted data into one end of the chain and
read the clear text out of the other end. (and vice-versa)
However, I got stuck there, 'cuz to create a cipher BIO, I need to
specify the cipher and key.
e.g.: from Ex 4-8 in O'Reilly's OpenSSL book:
BIO* cipher = BIO_new(BIO_f_cipher());
BIO_set_cipher(cipher, EVP_des-ese3_cbc(), key, NULL, 1);
In my case, this was just negotiated between my server and the
connecting client during the SSL_accept call.
I believe the negotiated cipher and key are living in my SSL object,
but I couldn't see how to extract those objects and pass them to the
BIO_set_cipher call for each new session??
As for the renegotiation, of course you are absolutely correct that I
am not handling those in my code snippets. But I am fully aware of
those cases as I have been shipping a TLS-based SMTP server for
several years. e.g. I've seen SSL_accept take several iterations of
reads and writes. But the current server is select() based, and I am
now converting it over to Async IO (IOCP) so now I'm having to re-work
the TLS model.
I did notice the O'Reilly reference to the ssltest.c sample. I
haven't looked at that yet, but I will. :)
Thanks for the renegotiation unit test idea, and thanks for the reply,
n8
-----Original Message-----
From: ger.h...@gmail.com [mailto:ger.h...@gmail.com] On Behalf
Of Ger Hobbelt
Sent: Tuesday, March 17, 2009 5:38 PM
To: openss...@openssl.org
Cc: n8l...@gmail.com
Subject: Re: TLS, BIOs, SSL_read/write
> SSL_set_bio(m_ssl, NULL, bioMem);
and it's counterpart:
SSL_set_bio(m_ssl, m_bioMem, NULL);
<snip>
Ger Hobbelt
> All good points. =A0I was not planning to go to production with that
> code - I was just happy to see something working. :)
> I was trying to figure out a way to call SSL_set_bio once per session
> with both read and write buffers, but I am stumped there since this
> call:
>
> needs to change based on the data that just came off the wire.
Not exactly. you can write to the BIO_mem source/sink to fill it, and
BIO_read from it to pull data from it.
(IIRC, BIO_s_mem can auto-adjust the internal buffer size, but it
might be that only happens when you instantiate one using
BIO_new(BIO_s_mem()) -- I am not sufficiently awake to pull that info
out of my high hat right now, though I've been using this quite
recently. :-( )
Anyway, I'd say BIO_pair would be the nicest way to go (as it can
handle bidir traffic, so you only need one at the backend).
> Which naturally brings us to the BIO pairs issue. =A0That is the path I
> started going down... chaining a cipher BIO with a memory BIO, and
> planning to write the encrypted data into one end of the chain and
> read the clear text out of the other end. (and vice-versa)
>
> However, I got stuck there, 'cuz to create a cipher BIO, I need to
> specify the cipher and key.
> e.g.: from Ex 4-8 in O'Reilly's OpenSSL book:
> =A0 BIO* cipher =3D BIO_new(BIO_f_cipher());
> =A0 BIO_set_cipher(cipher, EVP_des-ese3_cbc(), key, NULL, 1);
non non non... this BIO_f_cipher() BIO filter is only for when you do
your own protocols; the idea here is (and sorry for sounding overly
vague right now) to put a BIO_pair in place of the usual BIO_s_socket
source/sink.
Recall that OpenSSL offers BIO *chains*, so you should keep the
regular chain intact (SSL_read/SSL_write at topside; traverse BIO
chain down to source/sink to insert BIO_pair as a source+sink). The
SSL layer should do its own cipher BIO adding and all; all you should
replace is the source/sink BIOs.
Sorry I can't draw a clear picture right now; too much SQL for me
today: brain has shut down.
And another side note that wasn't mentioned last time (didn't want to
dump all the caveats in one go) but be aware that SSL BIO's (and (SSL
*) sessions!) are 'threadsafe' in the sense that OpenSSL *assumes* a
(SSL *) or /any/ BIO remains inside a single thread from the moment it
becomes 'active', i.e. is set up / is going to do some work.
I don't know how you're going to do the IOCP implementation, but be
very much aware that that model does NOT tie a socket to a single
thread, so there's bound to be some really wicked thread-related
issues as the (SSL *) and the BIOs will cross thread boundaries on and
off while active - due to the way IOCP work. Unless you do perform
some very fancy footwork. ;-)
(I seem to recall the documentation mentions this as SSL* and BIOs
being 'thread-AWARE' rather than 'threadsafe'. Too tired to go find
chapter and verse, though.)
> I believe the negotiated cipher and key are living in my SSL object,
> but I couldn't see how to extract those objects and pass them to the
> BIO_set_cipher call for each new session??
Like I said: don't use BIO_f_cipher() -- that's for when you want to
apply BIO chains to custom (encrypted) data; SSL does it all for you
within the SSL*: topside you have plaintext I/O for the application
using SSL_read/SSL_write(), backside (i.e. at source/sink 'bottom')
you have the SSL protocol exchange mixed with encrypted data and
everything as it travels the wire: thus the BIOs linked to the SSL*
session using SSL_set_bio() will carry the raw network traffic. All
those two BIOs (usually the same BIO used twice) need to do it
transfer this raw data. That's where the BIO_pair comes in (head is
clearing up a bit; this is stream-of-conscious written, so please bear
with me): you get yourself a BIO_pair and stuff one BIO into
SSL_set_bio(.., here, here) and the other BIO of the pair is yours to
use for feeding / fetching raw network data as obtained / transmitted
through the IOCP code. (This is, IIRC, what ssltest et al showcase as
well; you may want to grep through the apps/ and demos/ dirs on the
lookout for further BIO_pair and SSL_set_bio samples as ssltest
definitely isn't the only one in there doing this.)
On the application side, you still have SSL_write/SSL_read, like
before (and here I had that thread-crossing warning thought I
mentioned above). This is where the SSL* fetches/delivers the
plaintext data.
> now converting it over to Async IO (IOCP) so now I'm having to re-work
> the TLS model.
I'm thinking about it; there's some very interesting issues I hadn't
thought of before. IOCP thread usage vs. SSL/BIO assumptions being the
major one.
> Thanks for the renegotiation unit test idea, and thanks for the reply,
You're welcome!
--=20
Met vriendelijke groeten / Best regards,
Ger Hobbelt
--------------------------------------------------
web: http://www.hobbelt.com/
http://www.hebbut.net/
mail: g...@hobbelt.com
mobile: +31-6-11 120 978
--------------------------------------------------