error in connectionFactory.CreateConnection() after about 1250 connections

[mohammad olfat miri]

i coded these lines to test how many connection my RabbitMq server can handle

main program

----------------------------

            for (int i = 0; i < 8000; ++i)

            {

                Consumer consumer = new Consumer("192.168.2.6", "q1");

                consumer.onMessageReceived += handleMessage;

                consumer.StartConsuming();

            }

public void handleMessage(byte[] message)

        {

        }

-----------------------

producer class

-----------------------

  public class Consumer

    {

        protected IModel Model;

        protected IConnection Connection;

        protected string QueueName;

        protected bool isConsuming;

        // used to pass messages back to UI for processing

        public delegate void onReceiveMessage(byte[] message);

        public event onReceiveMessage onMessageReceived;

        public Consumer(string hostName, string queueName)

        {

            QueueName = queueName;

            ConnectionFactory connectionFactory = new ConnectionFactory();

            connectionFactory.HostName = hostName;

            connectionFactory.UserName = "my_username";

            connectionFactory.Password = "my_password";

            Connection = connectionFactory.CreateConnection();

            Model = Connection.CreateModel();

            Model.QueueDeclare(QueueName, false, false, false, null);

        }

        //internal delegate to run the consuming queue on a seperate thread

        private delegate void ConsumeDelegate();

        public void StartConsuming()

        {

            isConsuming = true;

            ConsumeDelegate c = new ConsumeDelegate(Consume);

            c.BeginInvoke(null, null);

        }

        public void Consume()

        {

            QueueingBasicConsumer consumer = new QueueingBasicConsumer(Model);

            String consumerTag = Model.BasicConsume(QueueName, false, consumer);

            while (isConsuming)

            {

                try

                {

                    RabbitMQ.Client.Events.BasicDeliverEventArgs e = (RabbitMQ.Client.Events.BasicDeliverEventArgs)consumer.Queue.Dequeue();

                    IBasicProperties props = e.BasicProperties;

                    byte[] body = e.Body;

                    // ... process the message

                    onMessageReceived(body);

                    Model.BasicAck(e.DeliveryTag, false);

                }

                catch (OperationInterruptedException ex)

                {

                    // The consumer was removed, either through

                    // channel or connection closure, or through the

                    // action of IModel.BasicCancel().

                    break;

                }

            }

        }

        public void Dispose()

        {

            isConsuming = false;

            if (Connection != null)

                Connection.Close();

            if (Model != null)

                Model.Abort();

        }

    }   

--------------------------------------

it works until server reach about 1250 connection then line "Connection = connectionFactory.CreateConnection();" throw new exception, i look at log and fount this line :  

closing AMQP connection <0.13048.1> (192.168.2.6:49743 -> 192.168.2.6:5672):

{handshake_timeout,handshake}

i attached some images.

i am using windows 10 localhost

6GB ram

corei7 intel cpu

and 124GB free hard space

Erlang 18.0

[Michael Klishin]

On 7 Aug 2015 at 23:13:53, mohammad olfat miri (m.olfa...@gmail.com) wrote:
> it works until server reach about 1250 connection then line
> "Connection = connectionFactory.CreateConnection();" throw
> new exception, i look at log and fount this line :
>
> closing AMQP connection <0.13048.1> (192.168.2.6:49743 ->
> 192.168.2.6:5672):
> {handshake_timeout,handshake}

This means negotiating a connection took longer than 5 seconds.

There are many knobs that need tuning to support a large number of connections,
most are OS settings. Defaults on Linux are absolutely nonsensical for servers,
e.g. max number of open file handles is 1024 by default.

See http://rabbitmq.com/networking.html. 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[mohammad olfat miri]

how can i support a large number of connections in windows ?

there is no doc for that :(

[Michael Klishin]

 On 7 Aug 2015 at 23:54:33, mohammad olfat miri (m.olfa...@gmail.com) wrote:
> how can i support a large number of connections in windows ?
> there is no doc for that :(

Windows has similar settings. How they are configured seems to differ between
releases but quite a few values seem to rely on the registry.

See [1][2][3], for example. 

1. http://smallvoid.com/article/winnt-tcpip-max-limit.html
2. https://msdn.microsoft.com/en-us/library/aa560610(v=bts.20).aspx
3. http://help.globalscape.com/help/secureserver3/Windows_Registry_keys_for_TCP_IP_Performance_Tuning.htm

[gatesvp]

Something to note, your code is creating a one Connection and one Model each time through that loop at the top.

However, the way the C# driver works, you typically want to create one Connection per target server along with one Model per thread.

So you would move this code to the for loop and pass the IConnection object into the Consumer() constructor.

    ConnectionFactory connectionFactory = new ConnectionFactory();

    connectionFactory.HostName = hostName;

    connectionFactory.UserName = "my_username";

    connectionFactory.Password = "my_password";

    Connection = connectionFactory.CreateConnection();

Also your Consumer class has a Dispose() method but doesn't implement IDisposable, so it's not actually being cleaned up. Keep an eye on outbound connections here.

The links provided by @Michael are definitely useful, by I would start by fixing up that Connection vs. Model issue.

> i coded these lines to test how many connection my RabbitMq server can handle

This may be a case where you're not testing what you think you're testing. Are you specifically worried about "# of connections" or "data throughput"? I can get lots and lots of throughput with one connection per server, so what are you specifically trying to optimize for here?

- Gates

[Michael Klishin]

On 8 August 2015 at 01:01:11, gatesvp (gat...@gmail.com) wrote:
> Something to note, your code is creating a one Connection and
> one Model each time through that loop at the top.
>
> However, the way the C# driver works, you typically want to create
> one Connection per target server along with one Model per thread.

I think the intent was exactly to  test the number of concurrent connections,
with stock defaults everywhere.

[mohammad olfat miri]

 lets explain more, we are developing an notification server for our android users, when a user install our app it will create a unique queue and start consuming it , so each client will create a new connection ( 16000 user mean 16000 connection ) , in this test after 1250 connection , server will not accept new connections. so just help me if i am wrong !

On Saturday, August 8, 2015 at 2:31:08 AM UTC+4:30, gatesvp wrote:

Something to note, your code is creating a one Connection and one Model each time through that loop at the top.

However, the way the C# driver works, you typically want to create one Connection per target server along with one Model per thread.

So you would move this code to the for loop and pass the IConnection object into the Consumer() constructor.

    ConnectionFactory connectionFactory = new ConnectionFactory();

    connectionFactory.HostName = hostName;

    connectionFactory.UserName = "my_username";

    connectionFactory.Password = "my_password";

    Connection = connectionFactory.CreateConnection();

Also your Consumer class has a Dispose() method but doesn't implement IDisposable, so it's not actually being cleaned up. Keep an eye on outbound connections here.

The links provided by @Michael are definitely useful, by I would start by fixing up that Connection vs. Model issue.

> i coded these lines to test how many connection my RabbitMq server can handle

[mohammad olfat miri]

i  changed my server to ubuntu server and set max port to 65000 port , still the same problem! after about 1250 connection it throw new exception !  whats wrong ???

[Michael Klishin]

See rabbitmq.com/networking.html. I don't have much to add other than: your client will use at least 1 thread per connection (.NET),
and there are OS level limits on that as well.

It's not just about server limits and tuning.

MK