Wrapping SSL_read/SSL_write so they behave like read/write.
Steven Young
Hello,
I'm writing a program which can be compiled either with SSL support or
without. In order to limit the amount of #ifdef'ing I have to put
throughout the rest of my program, I'm trying to wrap SSL_read and
SSL_write so they can be treated like read/write on a regular socket.
This is not meeting with much success.
In the non-SSL case, I do connect(), set it nonblocking, and start
select()ing on the fd(s) that I have connected to. This works okay.
In the SSL case, I connect(), create a new context with SSL_new,
set it nonblocking, do SSL_set_fd, then do
int ret;
[...]
do {
ret = SSL_connect(sslobject);
if(ret != 1)
err = ERR_get_error();
} while (ret != 1 && (err == SSL_ERROR_WANT_READ ||
err == SSL_ERROR_WANT_WRITE));
This part also seems to work okay.
The part where everything falls apart is in my read/write wrappers.
They look like this:
read_wrapper:
[...]
do {
ret = SSL_read(sslobject, buf, bufsz);
err = SSL_get_error(sslobject, ret);
} while (ret <= 0 && (err == SSL_ERROR_WANT_READ));
my write_wrapper looks pretty much the same, except s/read/write/,
s/READ/WRITE/.
This and variations on these themes have given me a number of novel
results, such as the read loop eating 100% CPU time as SSL_read starts
to always return ret = -1 and error = SSL_ERROR_WANT_READ. SSL_write
seems to be behaving a bit better. I am mystefied as to why select()
would mark the fd as ready to read, and yet SSL_read returns nothing,
resulting in a 100% CPU loop.
Is there some other way I should be doing this? I have tried putting
if(ret <= 0 && (err == SSL_ERROR_WANT_WRITE))
SSL_write(sslobject, NULL, 0):
in my read loop after the err = ... statement, but it didn't do
anything.
If anybody can make any suggestions, or even point me to an example
of how this should be done, I would be much obliged. Is there an
IRC channel for OpenSSL support?
Thanks,
Steve.
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openss...@openssl.org
Automated List Manager majo...@openssl.org
Joe Flowers
You need to put select(ready to read or write) inside each (BOTH
SSL_read() and SSL_write()) of your while loops at the beginning, and
then cycle on WANT_READ or WANT_WRITE for BOTH SSL_read() and
SSL_write() loops.
You're getting high utilization because you are not putting select
inside the while loops.
Joe
Marek Marcola
> You need to put select(ready to read or write) inside each (BOTH
> SSL_read() and SSL_write()) of your while loops at the beginning, and
> then cycle on WANT_READ or WANT_WRITE for BOTH SSL_read() and
> SSL_write() loops.
>
> You're getting high utilization because you are not putting select
> inside the while loops.
you semi-blocking behaviour (we are only in non-blocking wrapper
and checking for read/write is done by select() in "upper" layer).
You should change loop ending condition - this loop should end
when SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE occur because
this errors may be returned on non-blocking sockets on SSL_read()
and on SSL_write(). On normal use this will happen mostly where
re-handshake is going on but this may happen where empty SSL fragments
are sent over network (of course this may happen on some system
condition too).
When you get SSL_ERROR_WANT_WRITE where you do SSL_read() this means
that SSL protocol needs to send something to your peer (some control/
handshake SSL information).
Best regards,
--
Marek Marcola <Marek....@malkom.pl>
Steven Young
> I'm not sure if this is good solution because this will give
> you semi-blocking behaviour (we are only in non-blocking wrapper
> and checking for read/write is done by select() in "upper" layer).
You're right; I don't want blocking behaviour. The non-SSL part
of the code solves this by select()ing on the readable file descriptors
and only calling read() when there is something to be read.
> You should change loop ending condition - this loop should end
> when SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE occur because
> this errors may be returned on non-blocking sockets on SSL_read()
> and on SSL_write(). On normal use this will happen mostly where
> re-handshake is going on but this may happen where empty SSL fragments
> are sent over network (of course this may happen on some system
> condition too).
I'm a little unclear on how this should be implemented.. so if I call
SSL_read, get -1 back, and err = SSL_ERROR_WANT_READ, do I just call
SSL_read again? Because that's what I've been doing and it ends up
in an infinite loop. Also, is err = SSL_ERROR_WANT_WRITE, but I have
no data to write (because I'm waiting to see what the server sends me
before replying), what should I write in my call to SSL_write?
Thanks,
Steve.
Steven Young
> You're right; I don't want blocking behaviour. The non-SSL part
> of the code solves this by select()ing on the readable file descriptors
> and only calling read() when there is something to be read.
To give you an idea of what's going on, I've put up some excerpts
of the relevant code at http://www.miranda.org/~sdyoung/broken.c. If
I'm not being clear in stating my problem, perhaps that will clarify
things a little.
Marek Marcola
> > You should change loop ending condition - this loop should end
> > when SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE occur because
> > this errors may be returned on non-blocking sockets on SSL_read()
> > and on SSL_write(). On normal use this will happen mostly where
> > re-handshake is going on but this may happen where empty SSL fragments
> > are sent over network (of course this may happen on some system
> > condition too).
>
> I'm a little unclear on how this should be implemented.. so if I call
> SSL_read, get -1 back, and err = SSL_ERROR_WANT_READ, do I just call
> SSL_read again?
layer select().
> Because that's what I've been doing and it ends up
> in an infinite loop.
And because you try to read data that is not available you get
infinite loop.
> Also, is err = SSL_ERROR_WANT_WRITE, but I have
> no data to write (because I'm waiting to see what the server sends me
> before replying), what should I write in my call to SSL_write?
If this hint is from SSL_read() this means that SSL protocol wants
something to write - not your application, but this is not possible now.
For example, when you call SSL_read() to write some data, there may be
renegotiation (negotiation new security parameters) in progres
which is sequence of SSL records read/write.
In this situation calling SSL_read() next time is enough
(SSL layer will continue to write its own data and after this
read real data) but this SSL_read() should be performed
when socket descriptor is ready for write now.
Best regards,
--
Marek Marcola <Marek....@malkom.pl>
______________________________________________________________________
Steven Young
> In this situation calling SSL_read() next time is enough
> (SSL layer will continue to write its own data and after this
> read real data) but this SSL_read() should be performed
> when socket descriptor is ready for write now.
Hi Marek,
I think I've fixed my problem. Thanks for setting me in the right
direction.
Steve.
David Schwartz
> I'm a little unclear on how this should be implemented.. so if I call
> SSL_read, get -1 back, and err = SSL_ERROR_WANT_READ, do I just call
> SSL_read again?
No. That error is telling you that you need to wait until the socket is
(again) readable.
> Because that's what I've been doing and it ends up
> in an infinite loop. Also, is err = SSL_ERROR_WANT_WRITE, but I have
> no data to write (because I'm waiting to see what the server sends me
> before replying), what should I write in my call to SSL_write?
You should not call SSL_write, you should call SSL_read again when the
socket becomes writable.
The SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE indications are telling
you that the SSL engine cannot make forward progress on the operation you
are attempting because it needs to read or write data that it cannot do
without blocking and you asked it not to block. If you get
SSL_ERROR_WANT_READ, you should retry the operation you are currently trying
when the socket becomes readable. If you get SSL_ERROR_WANT_WRITE, you
should retry the operation you are currently trying when the socket becomes
writable.
As a simplification, with only a minor performance hit, you can treat *any*
socket indication (whether readable or writable) as allowing you to retry
*any* pending operations (whether SSL_read or SSL_write). This lets you
treat any of the WANT indications as basically just 'would block try later'.
Just remember that 'SSL_ERROR_WANT_WRITE' means you need to select on the
socket for writing even if you weren't before!
DS
Marek Marcola
> On Sat, Aug 19, 2006 at 11:58:27PM +0200, Marek Marcola wrote:
> > In this situation calling SSL_read() next time is enough
> > (SSL layer will continue to write its own data and after this
> > read real data) but this SSL_read() should be performed
> > when socket descriptor is ready for write now.
>
> Hi Marek,
>
> I think I've fixed my problem. Thanks for setting me in the right
> direction.
with code like:
do {
ret = SSL_read(sslobject, buf, bufsz);
err = SSL_get_error(sslobject, ret);
} while (ret <= 0 && (err == SSL_ERROR_WANT_READ));
(of course I assume that WANT* works ok now).
The problem is that depending on variable bufsz this loop
may iterate some times getting application data from SSL layer
and putting this data in the same place (overwriting existing).
For example assume that in SSL layer we have 400 bytes ready to
read and that bufsz is 100 bytes. This loop will iterate 4 times
(and after this you will get WANT* error) placing every
100 bytes in the same place.
At the end you will get only 100 last bytes from 400 ready to read.
And next information, SSL_read() if has something in SSL buffers
return no more data than in buffers. This means that if you request
100 bytes and in SSL layer exists 10 bytes from previous reads
then you will get only 10 bytes - no additional try to read
some bytes from network will be performed.
Best regards,
--
Marek Marcola <Marek....@malkom.pl>
______________________________________________________________________
Steven Young
> I've forget to pay your attention on other problem that may appear
> with code like:
>
> do {
> ret = SSL_read(sslobject, buf, bufsz);
> err = SSL_get_error(sslobject, ret);
> } while (ret <= 0 && (err == SSL_ERROR_WANT_READ));
>
> (of course I assume that WANT* works ok now).
>
> The problem is that depending on variable bufsz this loop
> may iterate some times getting application data from SSL layer
> and putting this data in the same place (overwriting existing).
> For example assume that in SSL layer we have 400 bytes ready to
> read and that bufsz is 100 bytes. This loop will iterate 4 times
> (and after this you will get WANT* error) placing every
> 100 bytes in the same place.
> At the end you will get only 100 last bytes from 400 ready to read.
So what you are saying is that even though ret <= 0 and err ==
SSL_ERROR_WANT_READ, it is still possible that data has been placed
in buf?
Thanks,
Steve.
Marek Marcola
> On Sun, Aug 20, 2006 at 07:46:26PM +0200, Marek Marcola wrote:
> > I've forget to pay your attention on other problem that may appear
> > with code like:
> >
> > do {
> > ret = SSL_read(sslobject, buf, bufsz);
> > err = SSL_get_error(sslobject, ret);
> > } while (ret <= 0 && (err == SSL_ERROR_WANT_READ));
> >
> > (of course I assume that WANT* works ok now).
> >
> > The problem is that depending on variable bufsz this loop
> > may iterate some times getting application data from SSL layer
> > and putting this data in the same place (overwriting existing).
> > For example assume that in SSL layer we have 400 bytes ready to
> > read and that bufsz is 100 bytes. This loop will iterate 4 times
> > (and after this you will get WANT* error) placing every
> > 100 bytes in the same place.
> > At the end you will get only 100 last bytes from 400 ready to read.
>
> So what you are saying is that even though ret <= 0 and err ==
> SSL_ERROR_WANT_READ, it is still possible that data has been placed
> in buf?
Maybe this was not very precise.
Encapsulated SSL data comes in records/packets. When you select()
some descriptor for read, and select() gives you such hit you start
reading data from SSL buffers. And now we may have some problems.
If you will retry SSL_read() until you will get WANT_READ then
you will get all data from SSL layer to your application buffers
and now you may select() again.
But if you will read once (or not until WANT_READ), there may be
data left in SSL layer and now you can not select() (or you should
not select() - this depends on upper layer protocol).
For example if we have 400 bytes in SSL layer and you will read only
100, then we have 300 bytes in SSL layer and now waiting in select()
may give you hang. For such purpose SSL_pending() function exists
which can tell you is SSL buffers has some data or not. If has, you
simply call SSL_read(), if not - you select().
Of course if you will perform "incremental" SSL_read() until
WANT_READ then you will have all data from SSL and you may select()
without calling SSL_pending().
Best regards,
--
Marek Marcola <Marek....@malkom.pl>
______________________________________________________________________
Joe Flowers
"When calling |SSL_write()| with num=0 bytes to be sent the behaviour is
undefined."
I still stand by me first reply on this thread, as I believe it follows
directly from the docs.
Read the docs on SSL_read() and SSL_write().
SSL_ERROR_WANT_WRITE does not mean call SSL_write(). It means the
exactly same SSL function you just did with the exact same parameters as
you ust did that produced this SSL_ERROR_WANT_WRITE return. Again, it's
clearly explained in the docs.
Joe
Kyle Hamilton wrote:
> If you get SSL_ERROR_WANT_WRITE, even if you have no application data
> to send, the protocol itself requires data to be written -- so you
> need to call SSL_write(). If you get SSL_ERROR_WANT_READ, even if
> you're writing application data, that means that the protocol itself
> is requiring data to be read from the peer, so you need to call
> SSL_read().
>
> Both situations can happen in either case. If you have no data to
> write, call it with a NULL buffer and a length of 0.
>
> -Kyle H
>
> On 8/19/06, Steven Young <sdy...@miranda.org> wrote:
>> I'm a little unclear on how this should be implemented.. so if I call
>> SSL_read, get -1 back, and err = SSL_ERROR_WANT_READ, do I just call
>> SSL_read again? Because that's what I've been doing and it ends up
>> in an infinite loop. Also, is err = SSL_ERROR_WANT_WRITE, but I have
>> no data to write (because I'm waiting to see what the server sends me
>> before replying), what should I write in my call to SSL_write?
Joe Flowers
> It means the exactly same SSL function you just did with the exact
> same parameters as you ust did that produced this SSL_ERROR_WANT_WRITE
> return. Again, it's clearly explained in the docs.
>
> Joe
>
The sentence should have read:
It means call exactly the same SSL function you just did with the exact
same parameters as you just did that produced this SSL_ERROR_WANT_WRITE
return.
>
>
David Schwartz
> Encapsulated SSL data comes in records/packets. When you select()
> some descriptor for read, and select() gives you such hit you start
> reading data from SSL buffers. And now we may have some problems.
> If you will retry SSL_read() until you will get WANT_READ then
> you will get all data from SSL layer to your application buffers
> and now you may select() again.
Right, WANT_READ is specifically telling you to 'select' for readability.
> But if you will read once (or not until WANT_READ), there may be
> data left in SSL layer and now you can not select() (or you should
> not select() - this depends on upper layer protocol).
Right, you should *never* 'select' unless the SSL layer tells you to.
> For example if we have 400 bytes in SSL layer and you will read only
> 100, then we have 300 bytes in SSL layer and now waiting in select()
> may give you hang. For such purpose SSL_pending() function exists
> which can tell you is SSL buffers has some data or not. If has, you
> simply call SSL_read(), if not - you select().
> Of course if you will perform "incremental" SSL_read() until
> WANT_READ then you will have all data from SSL and you may select()
> without calling SSL_pending().
Right, you don't need SSL_pending. The more logical approach (IMO) is that
when you want to read some data, you call SSL_read. You 'select' for read if
and only if an SSL function (either SSL_read or SSL_write) gives you a
WANT_READ indication. The same goes for writing.
You should never assume that the SSL layer wants to read encrypted data
from the socket just because you are reading unencrypted data from the SSL
layer. That assumption may or may not be true, and can deadlock you when
it's false. Treat the SSL layer as a black box, don't assume you know what
it needs to do -- it specifically returns the WANT_* indications for this
purpose.
DS
David Schwartz
> If you get SSL_ERROR_WANT_WRITE, even if you have no application data
> to send, the protocol itself requires data to be written
Correct.
> -- so you
> need to call SSL_write().
Incorrect. The 'SSL_write' function is the function to send unencrypted =
data over the SSL link. It has nothing to do with the encrypted data the =
SSL engine wants to write to the socket.
> If you get SSL_ERROR_WANT_READ, even if
> you're writing application data, that means that the protocol itself
> is requiring data to be read from the peer, so you need to call
> SSL_read().
No. If the protocol itself needs to read data from the peer in order to =
write data, it will do so when you call SSL_write. The job of SSL_write =
is to do whatever is needed to encrypt and send the data you are =
writing, whether that means reading from the socket, writing to the =
socket, or both.
=20
> Both situations can happen in either case. If you have no data to
> write, call it with a NULL buffer and a length of 0.
No. Do not ever do that. If you have no data to write, do not call =
SSL_write.
This is based on the most grievous misunderstanding of what the =
SSL_read and SSL_write functions do. They do *NOT* read from and write =
to the socket. They read from and write to the logical SSL connection. =
That may require reading from the socket, writing to the socket, or both =
in any combination, and your program should not make assumptions about =
which and when.