Rabbitmq-server connection lost after day for setting up

391 views
Skip to first unread message

Nijo James

unread,
May 31, 2017, 2:10:27 AM5/31/17
to rabbitmq-users
I've setup Rabbitmq as broker for my celery tasks which processes 2600000 data. But i'm lossing connection to rabbitmq server after a day of running the tasks.

ERROR/MainProcess] consumer: Cannot connect to amqp://user:**@ip:5672/vhost: [Errno 111] Connection refused. Trying again in 32.00 seconds..

When i check the user list of rabbitmq it show empty list, how is the user and host deleted automatically. And When it try to create a new user i'm getting the following error:

```
sudo rabbitmqctl add_user user password
Creating user "user" ...
Error: {{badarg,[{ets,lookup_element,[mnesia_stats,trans_log_writes,2],[]},
                 {mnesia_lib,read_counter,1,
                             [{file,"mnesia_lib.erl"},{line,553}]},
                 {mnesia_dumper,get_log_writes,0,
                                [{file,"mnesia_dumper.erl"},{line,57}]},
                 {rabbit_misc,'-execute_mnesia_transaction/1-fun-0-',1,
                              [{file,"src/rabbit_misc.erl"},{line,496}]},
                 {worker_pool_worker,handle_call,3,
                                     [{file,"src/worker_pool_worker.erl"},
                                      {line,92}]},
                 {gen_server2,handle_msg,2,
                              [{file,"src/gen_server2.erl"},{line,913}]},
                 {proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,249}]}]},
        {gen_server2,call,
                     [<5092.25418.5>,
                      {submit,#Fun<rabbit_misc.6.25154013>,<5092.26544.5>},
                      infinity]}}
```

I was not able to restart the rabbitmq process. I'm not sure why. I have manually kill all the rabbit mq process and reinstall rabbitmq-server everyday to run the celery tasks. At that time i can create new user and vhost.

When i checked ls -1  /var/lib/rabbitmq/mnesia/rabbit@ip-172-31-7-140/queues/ | wc -l it was more than 1600000 ang folder has 1.6G data.

Can you guys help me with this. Is there any load limit for rabbitmq, coz i'm able to run my tasks for the first time

Michael Klishin

unread,
May 31, 2017, 6:40:36 AM5/31/17
to rabbitm...@googlegroups.com
Looks like the user table in a data store used internally is missing. RabbitMQ does not delete
users "automatically" or after a period of time.

One non-obvious case where we've seen this tables missing before is when a node runs out of file descriptors.

See `rabbitmqctl status` (or management UI) for what the limit is for your node(s). If you post your node's logs possibly
there would be more things to recommend.

Relevant doc guides:



--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
MK

Staff Software Engineer, Pivotal/RabbitMQ
Reply all
Reply to author
Forward
0 new messages