How to flush a Java Socket?
Brad Johanson
I've run into an interesting phenomenon with sockets in Java.
Before I go through the long-winded explanation, here is the short
question for which I want an answer: How do I guarantee that a write
operation on a socket output stream has completed in Java? Things
that
don't work: calling flush on the stream, and setting SO_TCP_NODELAY.
Things that do: closing and reopening the socket, but this is a long
lived
connection and I can't afford to do that on every send.
So, here is the problem. I have a server process that reads byte
arrays
off the wire, processes them and sends data back over the same socket
connection.
The client may have multiple threads submitting to the server, but
uses a single socket for all of them. This is handled by queueing the
byte arrays to send and
having a single thread dequeue and write the arrays out to the socket.
Essentially, the send thread looks like this:
DataOutputStream sockOutStream=new
DataOutputStream(socket.getOutputStream());
while(true) {
byte []sendData=outputQueue.dequeue();
sockOutStream.write(sendData);
sockOutStream.flush();
}
When testing some code that enqueues one byte array, waits for
notification that the dequeue and send took place, and then calls
System.exit(0) I
found that over 50% of the time the byte array didn't get to the
server. When I added a sleep(1000) to the end of the program, the
byte array
would get to the server every time.
Then, I removed the sleep() and the System.exit(0), and again the
event
got to the server every time. Eventually I figured out that it is
because
I have a finalizer which closes the socket, and it is called when the
program just runs until the end, but not when System.exit() is called.
If
the socket isn't closed by socket.close(), the JVM may terminate
without
having sent all the data out the socket (regardless of the fact that I
called flush() on the socket output stream). So, the reason why the
byte array
isn't getting through is that the socket wasn't getting properly
flushed,
because finalizers don't get run by default on a call to
System.exit().
All fine and good. I added a shutdown hook which gets called during
both
a normal exit and a call to System.exit() so that now the socket gets
closed and flushed properly either way. Now the Java program works
100%
of the time with both a run to exit and a System.exit().
Unfortunately this isn't the end of the story. We also allow C++
clients to use our library, and provide this functionality by
embedding a JVM running our code inside a C++ application, and then
using a JNI based wrapper to let the C++ app use our client code. For
the C++ applications the bug is now back, because the C++ program
exits catastrophically without notifying the embedded JVM that it
needs to tie things up. Unfortunately now I am stuck because C++
doesn't have shutdown hooks, so I can't add some nice last minute code
to tidy things up any
more.
The root of the problem is that when the byte array is written, I want
to
guarantee that the data has been written out the socket to the server
when
control returns to the calling application (which actually happens
because the sending thread calls notify() waking up the actual sender,
but I think that is a technicality). That way the application can
crash, exit or do whatever it wants, but it will know that if it
received the notification from the send thread, the event was at least
written out the socket. If I could do this then I wouldn't have to
worry about all this shutdown hook mumbo-jumbo for pure Java clients
either. The problem is I can't see anyway to do that (given that
flush() doesn't work) other than closing and reopening a socket to the
server on every single call, which seems like incredible overkill.
So, has anybody run into this, or do they have any ideas? I've done a
search through the newsgroups on Google and haven't come up with
anything.
Also, I already have SO_TCP_NODELAY set, so that also doesn't help in
this
situation.
Answers and suggestions appreciated,
-Brad
@rigidsoftware.com C. Lamont Gilbert
getting in there? Flush works just fine for me. You are talking about a
socket and not a process right? Are you using a bufferedOUtputStream? that
could speed things up. Also try putting the flush where it belongs, outside
the for loop.
> DataOutputStream sockOutStream=new
> DataOutputStream(socket.getOutputStream());
>
> while(true) {
>
> byte []sendData=outputQueue.dequeue();
>
> sockOutStream.write(sendData);
> }
> sockOutStream.flush();
You are not creating a new stream for every send are you? that could be
time consuming and destroying the stream also closes the socket if im not
mistaken.
All in all, I use flush and it works.
--
L8r,
C.L. Gilbert
For a free Java interface to Freechess.org see
http://www.rigidsoftware.com/Chess/chess.html
"Verily, verily, I say unto you, He that entereth not by the door() into the
sheepfold{}, but climbeth up some other way, the same is a thief and a
robber." John 10:1
"Brad Johanson" <bjoh...@hotmail.com> wrote in message
news:4e3fefc1.01083...@posting.google.com...
Steve Horsley
unsent buffer contents. Exiting the JVM will do this (as you
discovered).
You could play around with Socket.shutdownOutput(). I haven't
done so, but I suspect that after doing this and the far end
receiving the signal, the far end will then in finish closing the
call in response. Your blocked read() thread should then do its
normal EOF behaviour.
In any case, an inopportune network failure may leave you in the
position where you think you have sent everything, but the far
end is missing the tail end.
The only true guarantee that your transmission has arrived is an
acknowlegement from the far end application that is has seen it.
Steve.
bjoh...@hotmail.com (Brad Johanson) wrote in message news:<4e3fefc1.01083...@posting.google.com>...
Brad Johanson
> Do you have access to the server so you know for a fact the data is not
> getting in there?
I've written both the server and client, and can confirm the data
doesn't get there. The server is doing something like:
DataInputStream sockInStream=new
DataInputStream(socket.getInputStream());
while(true) {
{a bunch of read commands that parse through the pieces in the
array};
{code that processes};
}
We never get to the code that processes on the times when the write on
the
client doesn't complete.
> You are talking about a
> socket and not a process right?
Correct.
> Are you using a bufferedOUtputStream? that
> could speed things up.
No, I'm not, but I write a complete byte array and then immediately
flush, so it doesn't seem that a bufferedOutputStream would help. I
tryed changing the code I originally gave to:
DataOutputStream sockOutStream=
new DataOutputStream(new
BufferedOutputStream(socket.getOutputStream()));
while(true) {
byte []sendData=outputQueue.dequeue();
sockOutStream.write(sendData);
sockOutStream.flush();
}
But when I do this it totally breaks the protocol, for whatever
reason, and none of the clients can communicate with the server. It
is as if the flush command is doing nothing.
> Also try putting the flush where it belongs, outside
> the for loop.
>
> > DataOutputStream sockOutStream=new
> > DataOutputStream(socket.getOutputStream());
> >
> > while(true) {
> >
> > byte []sendData=outputQueue.dequeue();
> >
> > sockOutStream.write(sendData);
> > }
> > sockOutStream.flush();
I'm not sure why I would ever want to do this, as the flush would
never be called in this case (while(true) never exits). The while
loop is just so that the thread runs forever, getting new data to
send, sending it, then flushing it out to the network (or trying to
anyway).
> You are not creating a new stream for every send are you? that could be
> time consuming and destroying the stream also closes the socket if im not
> mistaken.
Nope, one stream is created when the socket is created, and then used
forever.
>
> All in all, I use flush and it works.
>
What JDK and platform are you using? I am under Win2k with Sun JDK
1.3.1.
Thanks for your suggestions,
-Brad
Dr Hackenbush
situation to watch
the actual data transmission. This does not solve your problem but
provides a
useful data point for debugging purposes.
In the simplest case, the usual IP utilities (ping, netstat, ipconfig,
traceroute, tcpdump,...)
are always the place to start. Use netstat in this case to look at the
status of your TCP
socket connection. This program (thereal) is just way cool!
--
Hackenbush: Oh, well, uh, to begin with I took four years at Vassar.
Mrs. Upjohn: Vassar? But that's a girls' college.
Hackenbush: I found that out the third year. I'd've been there yet,
but I went out for the swimming team.
"Brad Johanson" <bjoh...@hotmail.com> wrote in message
news:4e3fefc1.01083...@posting.google.com...
John Creighton
Brad Johanson wrote:
> DataOutputStream sockOutStream=new
> DataOutputStream(socket.getOutputStream());
>
> while(true) {
>
> byte []sendData=outputQueue.dequeue();
>
> sockOutStream.write(sendData);
> sockOutStream.flush();
> }
I'm not familiar with sockets but I think if you
close your DataOutputStream you will send an
EOF exception to the DataInputStream on the other
side of the socket without closing the socket.
Try calling sockOutStream.close() right after
sockOut.flush().
Lee Fesperman
>
> I've run into an interesting phenomenon with sockets in Java.
>
> Before I go through the long-winded explanation, here is the short
> question for which I want an answer: How do I guarantee that a write
> operation on a socket output stream has completed in Java? Things
> that don't work: calling flush on the stream, and setting SO_TCP_NODELAY.
> Things that do: closing and reopening the socket, but this is a long
> lived connection and I can't afford to do that on every send.
>
I've never experienced this problem (I've done a number of socket apps in Java). Have
you been able to reproduce this with a simple program?
Alternatively, you could post more of your code, since it is highly likely that this is
a bug in your program.
--
Lee Fesperman, FFE Software, Inc. (http://www.firstsql.com)
===================================================================
* Check out Database Debunkings (http://www.firstsql.com/dbdebunk/)
* "Where Persistent Prevailing Database Fallacies Are Dispelled"
@rigidsoftware.com C. Lamont Gilbert
news:<gWJj7.71381$K6.28435942@news2>...
> > Do you have access to the server so you know for a fact the data is not
> > getting in there?
>
> I've written both the server and client, and can confirm the data
> doesn't get there. The server is doing something like:
>
> DataInputStream sockInStream=new
> DataInputStream(socket.getInputStream());
>
> while(true) {
>
> {a bunch of read commands that parse through the pieces in the
> array};
>
> {code that processes};
> }
>
Are you sure the server is waiting on the data, AND the server is not
buffered?
Well this is strange. this loop will basically shoot data out continuously
untill it tries to send null data and throws a null pointer exception. What
is slowing your loop? this looks like very bad technique. IS it that
outputQueue wont return untill there is data?
> > You are not creating a new stream for every send are you? that could be
> > time consuming and destroying the stream also closes the socket if im
not
> > mistaken.
>
> Nope, one stream is created when the socket is created, and then used
> forever.
>
> >
> > All in all, I use flush and it works.
> >
>
> What JDK and platform are you using? I am under Win2k with Sun JDK
> 1.3.1.
>
i am using win2k and jdk 1.3.1. but I been doing this for over a year and I
dont have these problems. only occasionally with processes, but never with
sockets.
Mike Dougherty
own. The servlet can send a status message back to the client indicating
that the message was received. If you can't do that then have your
server send the status message and the client read it. If the client
exits without receiving an OK signal back from the server then assume
the message never go there.
In regards to your current situation. I haven't worked directly with
sockets in quit some time so I had to build a quick test to refresh my
memory. I have no idea what your server code is like, or the rest of you
client code for that matter, so I guessed. I kind of reproduced your
problem. If I continuously sent the data I wouldn't get all of the data.
At least at first glance. What I found was that some of the data
elements were being concatenated together. So while the number of data
elements I was sending and receiving did not match, the byte count
always matched.
However, Implementing the above mentioned return status would solve this
problem also. The client would not send a new data element until it
received an OK from the previous data transmission.
<mike />
Brad Johanson wrote:
--
******************************************
Mike Dougherty -- Java Software Engineer
******************************************
Brad Johanson
> > Answers and suggestions appreciated,
>
> Alternatively, you could post more of your code, since it is highly
> likely that this is a bug in your program.
Thanks to everybody for your suggestions. I can't post the full code
since it is several 10's of thousands of lines long, much of it not
relevant to this particular glitch.
I tried to simplify down to just the socket parts, but the bug doesn't
show up in that version.
Rather than try to track it down, I've worked around it by waiting for an
ACK from the server before allowing the API call to return. This is not
ideal since it requires an extra hop, but at least it functions correctly.
Thanks again for the suggestions,
-Brad
