network dropping idle connections from Hbase client

33 views
Skip to first unread message

Devin Simpson

unread,
Sep 30, 2016, 4:27:56 PM9/30/16
to Async HBase
My organization recently moved an application outside of our internal cloud with an Hbase Client that connects to a service inside the internal cloud. We found that our firewall is dropping idle connections after a time, and wanted to set the Hbase client to close the connections on the client end before the firewall drops them.

We didnt see any obvious to configure the HBaseClient (we are on v1.4.1) so after reading through the HbaseClient implementation, we came up with this solution.

instantiate a new HBaseClient with a custom ClientSocketChannelFactory


HBaseClient  client = new HBaseClient(config.getQuorum(), BASE_PATH, clientSocketFactory);


then factory built here:



Executor executor = Executors.newCachedThreadPool();

ClientSocketChannelFactory clientSocketFactory = new HBaseCustomClientSocketChannelFactory(executor, executor);


we defined the  HBaseCustomClientSocketChannelFactory to set an idle timeout handler every time a new ChannelPipeline is added to the factory, the idletime out handler is to disconnect channels after a set time of idleness. The code we wrote for the HBaseCustomClientSocketChannelFactory is here:



private final class HBaseCustomClientSocketChannelFactory extends NioClientSocketChannelFactory {


        public HBaseCustomClientSocketChannelFactory(Executor executor, Executor executor2) {

            super(executor, executor2);

        }


        @Override

        public SocketChannel newChannel(ChannelPipeline pipeline) {


            HashedWheelTimer timer = new HashedWheelTimer();


            pipeline.addLast("idleHandler", new HBaseClientIdleStateAwareHandler(timer, READ_IDLE_TIMEOUT,

                WRITE_IDLE_TIMEOUT, ALL_IDLE_TIMEOUT, TimeUnit.SECONDS));


            return super.newChannel(pipeline);

        }

    }



we specifically override the newChannel() method because in the newClient() method in the HBaseClient.java code, that method is used on the passed in channelfactory, and that looked to be the only way we could handle idle connections on the HBaseClient. We also build our own Idlestatehandler ,HBaseClientIdleStateAwareHandler, defined here:



private final class HBaseClientIdleStateAwareHandler extends IdleStateHandler {


        public HBaseClientIdleStateAwareHandler(HashedWheelTimer timer, int readerIdleTimeSeconds,

            int writerIdleTimeSeconds, int allIdleTimeSeconds, TimeUnit seconds) {

            super(timer, readerIdleTimeSeconds, writerIdleTimeSeconds, allIdleTimeSeconds, seconds);

        }


        @Override

        public void channelIdle(ChannelHandlerContext ctx, IdleState e, long lastActivityTimeMillis) {

            // close the idle connection

            ctx.getChannel().disconnect();

        }

    }




my question is, is there any risk to approach we have taken? we need the ability to close the idle connections the HBaseClient makes before those connections are dropped.

Reply all
Reply to author
Forward
0 new messages