Netty Not Scaling for Higher concurrency

[Tanima Saini]

We are working on a Netty based HTTP server which receives a Get Request, decodes the request parameters, makes a couple of webservice calls and then returns the result after some processing

A typical request generally completes in around 320ms out of which approxmately 300ms are spent waiting for a response from the remote servers.

The problem is that we are not able to generate high throughput on the setup.

This is a 16 core machine with 30 GB RAM. The cpu usage never goes beyond 20% and RAM udage never goes beyond 5GB. The java heap size is set to 25GB

Additionally we have updated the following OS settings

ulimit =65000
net.core.somaxconn = 2048
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 10240    65535

This is an ubuntu machine and we are using the 3.2 branch of Netty

The max throughput that we can achieve on the server is around 300 requests per second which also goes down with each subsequent test which finally goes down to 35 req/s.

We have tried using the FixThreadPool as well as the cachedthreadpool while creating the netty server bootstrap.

Whatever we do the performance numbers do not change

Our guess is that Netty is not creating enough threads and hence a lot of time is wasted in the queue.

Is there something basic that we are missing due to which netty is giving us fixed concurrency?

How do we make maximum use of our CPU/RAM to get maximum performance from our hardware?

here is the code used for netty initialization

ServerBootstrap bootstrap = new ServerBootstrap(
                new NioServerSocketChannelFactory(
                        Executors.newCachedThreadPool(),
                        Executors.newCachedThreadPool()));
                        //Executors.newFixedThreadPool(5000),5000));
   // Configure the pipeline factory.


     bootstrap.setPipelineFactory(this.nettyServerPipeline);
       bootstrap.setOption("child.tcpNoDelay", true);
       bootstrap.setOption("backlog",1000);
       bootstrap.setOption("keepAlive", true);
       bootstrap.setOption("connectTimeoutMillis", 10000);
       bootstrap.setOption("client.reuseAddress", true);
       // Bind and start to accept incoming connections.
       bootstrap.bind(new InetSocketAddress(8099));
    /*  ServerBootstrap bootstrap =
        new ServerBootstrap(new         NioServerSocketChannelFactory(Executors.newFixedThreadPool(1),
            Executors.newFixedThreadPool(32)));
    */

We have tried multiple combinations of above settings but nothing seems to improve the absolute throughput.

[이희승 (Trustin Lee)]

Could you let me know the result after upgrading to 3.9.2.Final?

--

---
You received this message because you are subscribed to the Google Groups "Netty discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netty+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

https://twitter.com/trustin
https://twitter.com/trustin_ko
https://twitter.com/netty_project

[이희승 (Trustin Lee)]

Seems like self-answered in stackoverflow.com:

http://stackoverflow.com/questions/25056271/netty-not-scaling-for-higher-concurrency

The handler was performing a blocking memcached query in its handler.

[Norman Maurer]

Never block the IO-Thread… And now please all repeat for at least 100 times ;)

-- 
Norman Maurer