Not able to start RabbitMQ - Urgent help needed

[Puneet Aggarwal]

Hi ALL -

we have 3 nodes and after patching when we started system, Rabbit MQ didn't come up and we are seeing below error. I try to kill process associated with port but it's automatically started again. Can you please suggets?

Configuring logger redirection
21:10:41.024 [error]

BOOT FAILED
21:10:41.030 [error] BOOT FAILED
21:10:41.030 [error] ===========
===========
21:10:41.030 [error] ERROR: distribution port 25672 in use by another node: rabbit@cgocdlvrmqap01
ERROR: distribution port 25672 in use by another node: rabbit@cgocdlvrmqap01
21:10:41.030 [error]

21:10:42.031 [error] Supervisor rabbit_prelaunch_sup had child prelaunch started with rabbit_prelaunch:run_prelaunch_first_phase() at undefined exit with reason {dist_port_already_used,25672,"rabbit","cgocdlvrmqap01"} in context start_error
21:10:42.031 [error] CRASH REPORT Process <0.152.0> with 0 neighbours exited with reason: {{shutdown,{failed_to_start_child,prelaunch,{dist_port_already_used,25672,"rabbit","cgocdlvrmqap01"}}},{rabbit_prelaunch_app,start,[normal,[]]}} in application_master:init/4 line 138
{"Kernel pid terminated",application_controller,"{application_start_failure,rabbitmq_prelaunch,{{shutdown,{failed_to_start_child,prelaunch,{dist_port_already_used,25672,\"rabbit\",\"cgocdlvrmqap01\"}}},{rabbit_prelaunch_app,start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,rabbitmq_prelaunch,{{shutdown,{failed_to_start_child,prelaunch,{dist_port_already_used,25672,"rabbit","cgocdlvrmqap01"}}},{ra

Crash dump is being written to: erl_crash.dump...done

Regards,

Puneet Aggarwal

[Mohammed Reehan]

Can you please kill the process on linux for erlang and epmd on all three nodes and then start RMQ service per server. 

It should be start normally unless if there are recent changes on DNS or IP of the given nodes.

Hope this helps,

Reehan

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAAuurRKFnwX4QcOAOYouVSm-_UQzMwBtH6vM_e_ktfE2G%2Ba9Fw%40mail.gmail.com.

[Mohammed Reehan]

Also check if erlang and rabbitmq packages were updated along with your OS patchs.. that also could cause an issue.

[Puneet Aggarwal]

Thank you, Reehan. We bring down 2 servers and try to kill process but didn't work. Do you know the STEPs to kill erlang and epmd ?

Regards,

Puneet Aggarwal

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAJOW%3Dx-7kycztp%3DGe6v02v04ijfFjzzTvWGkyH57Tofx3_aaNw%40mail.gmail.com.

[Mohammed Reehan]

Ps -aux I grep -i empd*

Then take the process id and 

Kill -9 

Repeat same for 

Ps -aux | grep -i erlang*

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAAuurR%2Bt198nUFuTDTm_Xc7wbmFYrFCZ0Eg%2BAhM2kp1XXLQROw%40mail.gmail.com.

[Mohammed Reehan]

Please make sure rabbitmq process is also killed on nodes.

Then start it normally back by systemctl command per node

[Puneet Aggarwal]

Thank you, Reehan. I will ask the team and let you know if it works.

Regards,

Puneet Aggarwal

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAJOW%3Dx_jKOwZzd2MCm-xTeo6HgdN-%3D3%3Dk9evhAijr5vh-dfjww%40mail.gmail.com.

[Mohammed Reehan]

Great, let's us know if it helps.

Glad to help..

Reehan

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAAuurRLd0nNF-BHF1819qtmcQtseH4eCepKGunznhMJ0NaiZ5Q%40mail.gmail.com.

[Puneet Aggarwal]

I was able to resolve the issue.. Server were not able to communicate to each other,,

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAJOW%3Dx8qqrvm%3D8sL-j1ah_ncK5jEVcGo1RB1NB4OjDR_Ltu4pA%40mail.gmail.com.