RabbitMQ fails to start

[Scott]

Let me start out by saying I'm new to RabbitMQ. I've advanced beyond the simple Hello World apps but still a newbie on the server administration of RabbitMQ.

I'm running RabbitMQ Server 3.3.4 on Windows 7 Professional with Erlang 17.1 (win64).

Yesterday, RabbitMQ was running just fine. I was working in Visual Studio 2013 building a performance testing app to measure throughput. While developing and testing the app, I was pushing millions of messages (one test iteration had 50M messages) into the queues.

Near the end of the afternoon, the service just stopped working. I tried manually restarting the service, rebooting, uninstall / install, uninstall / delete all the  remnants I could fine / install again... none of it worked.

Today, I uninstalled again, deleted all the remnants I could fine, and then installed again. Nothing is working; I cannot get RabbitMQ to start.

In the log files, rab...@X-name-X.log and rab...@X-name-X-sasl.log, I found the following stacktrace, error, and crash report. Hopefully someone can help me dig a little deeper into the cause and solution.

Stack trace:

   [{rabbit_networking,record_distribution_listener,0,[]},

    {rabbit_networking,boot,0,[]},

    {rabbit,'-run_boot_step/1-lc$^1/1-1-',1,[]},

    {rabbit,run_boot_step,1,[]},

    {rabbit,'-start/2-lc$^0/1-0-',1,[]},

    {rabbit,start,2,[]},

    {application_master,start_it_old,4,

                        [{file,"application_master.erl"},{line,272}]}]

=INFO REPORT==== 8-Aug-2014::10:24:44 ===

Error description:

   {could_not_start,rabbit,

       {bad_return,

           {{rabbit,start,[normal,[]]},

            {'EXIT',{rabbit,failure_during_boot,{badmatch,noport}}}}}}

=CRASH REPORT==== 8-Aug-2014::10:24:44 ===

  crasher:

    initial call: application_master:init/4

    pid: <0.139.0>

    registered_name: []

    exception exit: {bad_return,

                        {{rabbit,start,[normal,[]]},

                         {'EXIT',

                             {rabbit,failure_during_boot,{badmatch,noport}}}}}

      in function  application_master:init/4 (application_master.erl, line 133)

    ancestors: [<0.138.0>]

    messages: [{'EXIT',<0.140.0>,normal}]

    links: [<0.138.0>,<0.7.0>]

    dictionary: []

    trap_exit: true

    status: running

    heap_size: 2586

    stack_size: 27

    reductions: 296

  neighbours:

[Simon MacMullen]

This certainly looks wrong.

When an Erlang node starts it registers itself with a small service
called epmd, which is just a mapping from Erlang node name to the port
used for Erlang distribution by that port.

RabbitMQ then later checks with the epmd to find out which port it's
using so we can report it.

For some reason it's that second lookup which is failing. That shouldn't
be possible; the initial registration might fail if port 4369 (epmd's
port) is blocked but once the first one succeeded the second one should too!

So I wonder what's going on. Has epmd become confused? Is some
process-based firewall blocking access to it? What happens if you kill
epmd and try again?

Cheers, Simon

> --
> You received this message because you are subscribed to the Google
> Groups "rabbitmq-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to rabbitmq-user...@googlegroups.com
> <mailto:rabbitmq-user...@googlegroups.com>.
> To post to this group, send email to rabbitm...@googlegroups.com
> <mailto:rabbitm...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Scott]

I killed the epmd process. When I attempted to start the service, epmd restarted. The RabbitMQ service failed to start but the epmd process kept running. When I ran "rabbitmqctl status", the following was returned: "* unable to connec to epmd (port 4369) on X-name-X: address (cannot connect to host/port)"

[Scott]

I think I found the issue. The security software installed on my machine appears to be using heuristics and blocked the epmd port (4369). At the moment, I'm awaiting a response from IT. The logs clearly show when I was able to test my app but then "automagically" something changed and everything stopped working.