j.n.ConnectException when trying load test from AWS servers

[Shobitha S]

Hi All,

We get j.n.ConnectException(detailed exception stack trace given below) when trying to trigger a load test through gatling versions 2.1.7, 2.2.0 and 2.2.3 from AWS linux server instances all of a sudden for the past 1 week. Until last week we had no issues and we were able to trigger successful load tests from the same AWS Linux servers. This issue seems to be very specific to AWS and we dont see any problem when trying to test the same application from gatling versions deployed in our local systems.  Any suggestions or hints on possible reasons for this exception would greatly help.

58970 [gatling-http-thread-1-1] DEBUG i.n.handler.logging.LoggingHandler - [id: 0x843673cd, L:/###.##.##.###:50209 - ######.com/###.###.###.##:443] EXCEPTION: java.io.IOException: Connection reset by peer
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:221)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:898)
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:242)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:485)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:399)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:371)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
58970 [gatling-http-thread-1-1] DEBUG o.a.netty.handler.HttpHandler - Unexpected I/O exception on channel [id: 0x843673cd, L:/###.##.##.###:50209 - ######.com/###.###.###.##:443]
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:221)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:898)
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:242)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:485)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:399)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:371)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
58970 [gatling-http-thread-1-1] DEBUG o.a.netty.channel.ChannelManager - Closing Channel [id: 0x843673cd, L:/###.##.##.###:50209 - ######.com/###.###.###.##:443]
58970 [gatling-http-thread-1-1] DEBUG i.n.handler.logging.LoggingHandler - [id: 0x843673cd, L:/###.##.##.###:50209 - ######.com/###.###.###.##:443] CLOSE()
58970 [gatling-http-thread-1-1] DEBUG o.a.n.channel.NettyConnectListener - Trying to recover from failing to connect channel [id: 0x843673cd, L:/###.##.##.###:50209 - ######.com/###.###.###.##:443] with a retry value of false
58970 [gatling-http-thread-1-1] DEBUG o.a.n.channel.NettyConnectListener - Failed to recover from connect exception: java.nio.channels.ClosedChannelException with channel [id: 0x843673cd, L:/###.##.##.###:50209 - ######.com/###.###.###.##:443]
58970 [gatling-http-thread-1-1] DEBUG io.gatling.http.ahc.AsyncHandler - Request '######' failed for user 18
java.net.ConnectException: https://#######.com:443
at org.asynchttpclient.netty.channel.NettyConnectListener.onFailure(NettyConnectListener.java:138)
at org.asynchttpclient.netty.channel.NettyConnectListener$1.onFailure(NettyConnectListener.java:109)
at org.asynchttpclient.netty.SimpleFutureListener.operationComplete(SimpleFutureListener.java:26)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:683)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:604)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:564)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:425)
at io.netty.handler.ssl.SslHandler.notifyHandshakeFailure(SslHandler.java:1239)
at io.netty.handler.ssl.SslHandler.setHandshakeFailure(SslHandler.java:1234)
at io.netty.handler.ssl.SslHandler.channelInactive(SslHandler.java:726)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:193)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:179)
at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:944)
at io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:687)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:374)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedChannelException: null