Re: Understanding SSL_read()/SSL_write()
Sebastián Treu
I didn't want to start another thread about this two I/O functions.
For starters (like me) on this two I/O calls, things get a bit
confusing at the beggining. I used to read man pages and doc before
asking something that I don't know and gather information about it to
try to resolve the problem myself and don't bother anyone. But when I
get stuck in specifics problems that I can't find answers I don't have
any other thing to do than:
a) nothing.
b) ask.
I preffer b.
As David said in an older thread, I can think of the API as a state
machine and then, regarding how much progress we get, do something.
Considering this, I have some doubts about a communication between
server and clients. Let's say we write a client that in his nature,
it's not a desired client. But, he connects, he handshakes and he
authenticates, and he behaves improperly. For example, when the server
sends him the data he required, he doesn't read it.
(I forgot to mention the use of non-blocking I/O)
A break here in that scenario: SSL_write() will not make further
success on writting to that client and SSL_get_error() will yield
WANT_WRITE or WANT_READ and we can use select() to decide which action
must be taken. If this is correct, I have some doubts about how to
implement a server that does overcome this issue with large amount of
data without letting the others clients hang in for the timeout we can
set on select() for example. I want to avoid the timeout because i'm
not pretty sure which value to use as we are communicating through a
network and I think that using small values could lead to avoid a
client that is slow on receiving data (I could be wrong in this
thinking).
Others things are to instantaneously put that client in a wait list
when SSL_write() could not succed, continue with the others, and try
with that client again later. But, sometimes we can get a WANT_WRITE
or WANT_READ when renegotiating so this does not appears a solution.
That "zombie client" thing makes SSL_write() "block". So, if the
client we are sending data does not read it, do we have some especial
error, function or something that the API provides us? Or do we think
on a workaround for it?
Maybe this sound like an implementation issue, but in fact I didn't
find information that tell us this scenario. I read on the possibility
of using the error pending on select() and the sockets error but no
one seems to be useful (at least for me) on this issue.
Thanks in advance,
Regards,
--
If you want freedom, compile the source. Get gentoo.
Sebastián Treu
http://labombiya.com.ar
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openss...@openssl.org
Automated List Manager majo...@openssl.org
David Schwartz
Sebastian Treu:
> As David said in an older thread, I can think of the API as a state
> machine and then, regarding how much progress we get, do something.
> Considering this, I have some doubts about a communication between
> server and clients. Let's say we write a client that in his nature,
> it's not a desired client. But, he connects, he handshakes and he
> authenticates, and he behaves improperly. For example, when the server
> sends him the data he required, he doesn't read it.
You always have to consider that. Even if the client is not malicious, the
data might not get there, the machine might crash, the network can go down.
This can always happen.
> (I forgot to mention the use of non-blocking I/O)
> A break here in that scenario: SSL_write() will not make further
> success on writting to that client and SSL_get_error() will yield
> WANT_WRITE or WANT_READ and we can use select() to decide which action
> must be taken.
Exactly.
> If this is correct, I have some doubts about how to
> implement a server that does overcome this issue with large amount of
> data without letting the others clients hang in for the timeout we can
> set on select() for example.
You seem to have lost the point of non-blocking I/O. The point is that you
*NEVER* block, and therefore *NEVER* shut out other clients. If there's some
other way you can make forward progress, *definitely* don't call 'select'.
Make forward progress elsewhere.
It's only when you cannot make forward progress *anywhere* that you call
'select' (or 'poll'), because there's nothing else you can do. And then you
must 'select' or 'poll' on every client you have and your 'listen'ing
sockets as well.
> I want to avoid the timeout because i'm
> not pretty sure which value to use as we are communicating through a
> network and I think that using small values could lead to avoid a
> client that is slow on receiving data (I could be wrong in this
> thinking).
It makes no difference what value you use, so long as there is nothing else
you can do. If there's something else you can do, well why aren't you doing
it?!
> Others things are to instantaneously put that client in a wait list
> when SSL_write() could not succed, continue with the others, and try
> with that client again later. But, sometimes we can get a WANT_WRITE
> or WANT_READ when renegotiating so this does not appears a solution.
I don't follow. Why is that not a solution?
> That "zombie client" thing makes SSL_write() "block". So, if the
> client we are sending data does not read it, do we have some especial
> error, function or something that the API provides us? Or do we think
> on a workaround for it?
You just go on and do other things. There should be nothing special you need
to do.
> Maybe this sound like an implementation issue, but in fact I didn't
> find information that tell us this scenario. I read on the possibility
> of using the error pending on select() and the sockets error but no
> one seems to be useful (at least for me) on this issue.
This is not an error case. This is how servers work -- they do work on each
client when they need it. You have to go out of your way to make this a
problem.
Basically, a 'select' loop server operates as follows:
1) Open any 'listen'ing sockets we need.
2) Call 'select' or 'poll' for any operations we might be able to do.
3) For every 'select' or 'poll' hit, make as much forward progress as
possible.
4) Go to 2.
You *never* block on only one thing. You *always* block on every possible
thing that might cause you to have to do more work.
DS
Sebastián Treu
>> Others things are to instantaneously put that client in a wait list
>> when SSL_write() could not succed, continue with the others, and try
>> with that client again later. But, sometimes we can get a WANT_WRITE
>> or WANT_READ when renegotiating so this does not appears a solution.
>
> I don't follow. Why is that not a solution?
Well, maybe I miss the point of what man pages wanted to say with:
"[...]
WARNING
When an SSL_write() operation has to be repeated because of
SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE, it must be repeated with
the same arguments.
[...]"
It's confusing for me. I know it doesn't make sense that if we
SSL_write() to a client that doesn't read, repeating SSL_write() with
same arguments will yield want_read and want_write all the time. What
exactly that means then? That's why I made a design on threads per
client (and the server will manage less than 20 connections for sure)
taking into account that "warning" from the man pages. So, I discarted
the solution of SSL_write() on another SSL structure when we get
want_read or want_write because I will be calling the function with
differents arguments. I also noted that SSL_write() is "clever" enough
to send _any_ length, so segmented writes will be overriding that
warning (if I undestood what man wanted to say). When I say clever
enough, I meant that in what I have proven (e.g. sending a 3mb file)
SSL_write() sends it by it's own (yielding want_write until it finish
with buffered data). The main reason the solution was discarted is
because that warning on man pages.
Regards,
--
If you want freedom, compile the source. Get gentoo.
Sebastián Treu
http://labombiya.com.ar
Sebastián Treu
<sebasti...@gmail.com> wrote:
> differents arguments. I also noted that SSL_write() is "clever" enough
> to send _any_ length, so segmented writes will be overriding that
> warning (if I undestood what man wanted to say). When I say clever
> enough, I meant that in what I have proven (e.g. sending a 3mb file)
> SSL_write() sends it by it's own (yielding want_write until it finish
> with buffered data). The main reason the solution was discarted is
> because that warning on man pages.
I want to be clear with this. Re-reading it I fall that maybe someone
could missunderstood it. What I wanted to say is that until the client
did not finish reading all the "chunk" of data, SSL_write() will keep
yielding WANT_WRITE. But, meanwhile we don't know how many data the
client has fetched. Imagine the client decides to read chunk of
Xbytes. Until the client did not read all the bytes that SSL_write()
writes, we'll get WANT_WRITE without any notions on how many bytes
have been read. So, in a given instant of time t0, we don't know if
the WANT_WRITE is actually a client reading by chunks less than we are
sending, or the client isn't reading at all. There is where it should
appears select(). Am I wrong?
David Schwartz
> Well, maybe I miss the point of what man pages wanted to say with:
>
> "[...]
> WARNING
> When an SSL_write() operation has to be repeated because of
> SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE, it must be repeated
> with
> the same arguments.
> [...]"
If you set ALLOW_MOVING_WRITE_BUFFER, the restriction is relaxed. The only
restriction that remains is that you must present a consistent data stream.
For example, if you try to SSL_write "foo" and two bytes are sent, your next
SSL_write *must* start with an "o".
> It's confusing for me. I know it doesn't make sense that if we
> SSL_write() to a client that doesn't read, repeating SSL_write() with
> same arguments will yield want_read and want_write all the time.
Right. And you can either repeat forever or you can close the connection.
What sensible servers do is call SSL_write once, get a want_whatever
indication, and add the client to their 'select' or 'poll' set. If the
client does nothing, the socket will never 'select', and the server will
eventually timeout and close the client (or do whatever the protocol
specifies, which may be to send something to the client).
> What
> exactly that means then? That's why I made a design on threads per
> client (and the server will manage less than 20 connections for sure)
> taking into account that "warning" from the man pages. So, I discarted
> the solution of SSL_write() on another SSL structure when we get
> want_read or want_write because I will be calling the function with
> differents arguments.
The warning is only within one specific SSL connection. It doesn't reach
across SSL connections.
> I also noted that SSL_write() is "clever" enough
> to send _any_ length, so segmented writes will be overriding that
> warning (if I undestood what man wanted to say). When I say clever
> enough, I meant that in what I have proven (e.g. sending a 3mb file)
> SSL_write() sends it by it's own (yielding want_write until it finish
> with buffered data). The main reason the solution was discarted is
> because that warning on man pages.
I'm not sure I follow what you're saying here, but it sounds like you
misunderstood the warning. In any event, set ACCEPT_MOVING_WRITE_BUFFER and
then the only requirement is that you present a consistent picture of the
byte stream to each SSL connection.
> I want to be clear with this. Re-reading it I fall that maybe someone
> could missunderstood it. What I wanted to say is that until the client
> did not finish reading all the "chunk" of data, SSL_write() will keep
> yielding WANT_WRITE. But, meanwhile we don't know how many data the
> client has fetched. Imagine the client decides to read chunk of
> Xbytes. Until the client did not read all the bytes that SSL_write()
> writes, we'll get WANT_WRITE without any notions on how many bytes
> have been read. So, in a given instant of time t0, we don't know if
> the WANT_WRITE is actually a client reading by chunks less than we are
> sending, or the client isn't reading at all.
Right, but we don't care. We're the application, and the internals of the
movement of SSL bytes is none of our business. Note that if you get
WANT_WRITE, it means that no further progress has been made in application
data bytes, which is all the application is allowed to care about.
> There is where it should
> appears select(). Am I wrong?
Yes, you would add the client to your 'select' set so that you will know
when to call SSL_write again.
DS
Sebastián Treu
> Sebastian Treu wrote:
>
>> Well, maybe I miss the point of what man pages wanted to say with:
>>
>> "[...]
>> WARNING
>> When an SSL_write() operation has to be repeated because of
>> SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE, it must be repeated
>> with
>> the same arguments.
>> [...]"
>
> If you set ALLOW_MOVING_WRITE_BUFFER, the restriction is relaxed. The only
> restriction that remains is that you must present a consistent data stream.
> For example, if you try to SSL_write "foo" and two bytes are sent, your next
> SSL_write *must* start with an "o".
That was the key then. I used the one in the man page:
SSL_MODE_ENABLE_PARTIAL_WRITE
but this writes 16kb _always_. I don't saw that option in the
SSL_write() but I can see it on SSL_CTX_set_mode(). With this option
set I don't get anymore WANT_WRITE when the client reads _partially_
and this let me _know_ more things to control the application flow.
Thanks David for your time and the detailed answers about it. I
promised not to bother the list anymore with SSL I/Os (*sic* at least
I hope),
--
If you want freedom, compile the source. Get gentoo.
Sebastián Treu
http://labombiya.com.ar
David Schwartz
> > If you set ALLOW_MOVING_WRITE_BUFFER, the restriction is relaxed. The
> > only
> > restriction that remains is that you must present a consistent data
> > stream.
> > For example, if you try to SSL_write "foo" and two bytes are sent,
> > your next
> > SSL_write *must* start with an "o".
> That was the key then. I used the one in the man page:
>
> SSL_MODE_ENABLE_PARTIAL_WRITE
These do different things. You probably want both of them. They are both
necessary to get sensible non-blocking semantics. I cannot understand why
they are not the default for non-blocking SSL connections. (Though I bet
it's for bizarre historical reasons.)
> but this writes 16kb _always_. I don't saw that option in the
> SSL_write() but I can see it on SSL_CTX_set_mode(). With this option
> set I don't get anymore WANT_WRITE when the client reads _partially_
> and this let me _know_ more things to control the application flow.
That's not unusual to see one size that repeats. Likely you are sending
large amounts of data, so the kernel TCP send queue is typically full. And
you're CPU is faster than the network, so as soon as there's room to fit one
SSL record, you're ready to write more than that. So you will most often see
SSL_write returning the number of bytes of application data that fit in one
SSL record.
> Thanks David for your time and the detailed answers about it. I
> promised not to bother the list anymore with SSL I/Os (*sic* at least
> I hope)
You're welcome.
DS
Patrick Patterson
> Hi David,
>
>
>>> Others things are to instantaneously put that client in a wait list
>>> when SSL_write() could not succed, continue with the others, and try
>>> with that client again later. But, sometimes we can get a WANT_WRITE
>>> or WANT_READ when renegotiating so this does not appears a solution.
>> I don't follow. Why is that not a solution?
>
>
> "[...]
> WARNING
> When an SSL_write() operation has to be repeated because of
> SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE, it must be repeated with
> the same arguments.
> [...]"
>
> SSL_write() to a client that doesn't read, repeating SSL_write() with
> exactly that means then?
Everyone always wants to overthink this - think of this:
You have, if you are using SSL/TLS, the following layers:
You Application (perhaps HTTP, perhaps some other protocol, but which
probably has some sort of protocol involved)
SSL (which has it's own protocol, completely unrelated to your
application, and transparent to it)
TCP/IP (read() and write()... may block, may have other problems, but
don't worry about it, because the SSL layer always "does the right thing".
Consequently, once you switch to SSL, just call SSL_read() and
SSL_write(), and do what they tell you to do. Don't worry about weird
"client stops accepting writes" or other cases ... the SSL layer will
deal with it... as well as incidental things like renegotiations,
credential passing, session setup and teardown... which are, by far, the
most likely reason to get SSL_ERROR_WANT_READ/SSL_ERROR_WANT_WRITE...
the SSL layer needs another pass through select() / poll() with the
appropriate call being made to completely something happening in *its*
protocol layer.
Just call SSL_connect() (or SSL_accept()), and then
SSL_read()/SSL_write() (with SSL_get_error after each call, which should
be obeyed if either returns <0), and then SSL_shutdown()).... And, of
course, correctly implement your state machine above this...
Don't worry about anything else.. if you do, you'll have a very flaky
application, that will hang in weird locations and for strange reasons.
Have fun.
Patrick.
Gaurav Joshi
Robert Rodriguez
https://academiadownloader.com/