java.io.IOException: Socket is not connected (possibly related to ChannelOption.SO_REUSEADDR)?

[Ning Zhong]

Hi, I am using the latest Netty 4.0.23 to send out http requests. My code logic is rather simple: (my code is at bottom of this post)

- use bootstrap to initiate a new client connection

- call bootstrap.connect(), and have MyClientListener 's operationComplete to check the ChannelFuture.

- If the future returned is successful, call future.channel().writeAndFlush().

This generally works well. But If I put a little load on it (I used 40 threads in parallel ), then after some 6,000 requests I start to see the following behavior:

- the bootstrap.connect() is successful, 

- but the immediate future.channel().writeAndFlush(request); will fail with java.io.IOException: Socket is not connected

( The thing is, if I skip the .option(ChannelOption.SO_REUSEADDR, true), then this behavior is gone. But I do need to reuseaddr )

java.io.IOException: Socket is not connected (custom log trace below)

sun.nio.ch.FileDispatcherImpl.write0 -2

sun.nio.ch.SocketDispatcher.write 47

sun.nio.ch.IOUtil.writeFromNativeBuffer 93

sun.nio.ch.IOUtil.write 51

sun.nio.ch.SocketChannelImpl.write 487

io.netty.channel.socket.nio.NioSocketChannel.doWrite 270

io.netty.channel.AbstractChannel$AbstractUnsafe.flush0 707

io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0 315

io.netty.channel.AbstractChannel$AbstractUnsafe.flush 676

io.netty.channel.DefaultChannelPipeline$HeadContext.flush 1059

io.netty.channel.AbstractChannelHandlerContext.invokeFlush 688

io.netty.channel.AbstractChannelHandlerContext.flush 669

io.netty.channel.ChannelOutboundHandlerAdapter.flush 115

io.netty.channel.CombinedChannelDuplexHandler.flush 197

io.netty.channel.AbstractChannelHandlerContext.invokeFlush 688

io.netty.channel.AbstractChannelHandlerContext.flush 669

io.netty.channel.ChannelDuplexHandler.flush 117

io.netty.channel.AbstractChannelHandlerContext.invokeFlush 688

io.netty.channel.AbstractChannelHandlerContext.write 718

io.netty.channel.AbstractChannelHandlerContext.writeAndFlush 706

io.netty.channel.AbstractChannelHandlerContext.writeAndFlush 741

io.netty.channel.DefaultChannelPipeline.writeAndFlush 895

io.netty.channel.AbstractChannel.writeAndFlush 240

My code looks something like:

// main

Bootstrap bootstrap = new Bootstrap().group(workerGroup)

.channel(NioSocketChannel.class).option(ChannelOption.SO_REUSEADDR, true)

.handler(new HttpClientChannelInitilizer(...))

.remoteAddress(remoteAddress);

ChannelFuture future = bootstrap.connect();

MyClientListener listener = new MyClientListener();

if (future.channel().isActive() && future.isSuccess()) {

listener.operationComplete(future);

} else {

future.addListener(listener);

}

// MyClientListener

public class MyClientListener implements ChannelFutureListener {

@Override

public void operationComplete(ChannelFuture future) throws Exception {

if (future.isSuccess()) {

ChannelFuture newFuture = future.channel().writeAndFlush(request);

if (newFuture.isDone() && !newFuture.isSuccess()) {

// !!!!!!!!!!!!!!

log.error( newFuture.cause() );

// !!!!!!!!!!!!!!

}

}

}

}

Thanks

Ning