RabbitMQ crash

[Manoj Kumar Satapathy]

Hi all,

Can some one please help us to understand the reason of Rabbitmq Crash in our linux  production server. Logs attached.

Crash Logs

2021-06-13 01:03:55 =ERROR REPORT====

** Generic server aten_detector terminating 

** Last message in was poll

** When Server state == {state,#Ref<0.2491135559.2982936578.179489>,1000,0.99,#{},#{}}

** Reason for termination ==

** {{timeout,{gen_server,call,[aten_sink,get_failure_probabilities]}},[{gen_server,call,2,[{file,"gen_server.erl"},{line,215}]},{aten_detector,handle_info,2,[{file,"src/aten_detector.erl"},{line,97}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}

2021-06-13 01:03:55 =CRASH REPORT====

  crasher:

    initial call: aten_detector:init/1

    pid: <0.301.0>

    registered_name: aten_detector

    exception exit: {{timeout,{gen_server,call,[aten_sink,get_failure_probabilities]}},[{gen_server,call,2,[{file,"gen_server.erl"},{line,215}]},{aten_detector,handle_info,2,[{file,"src/aten_detector.erl"},{line,97}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}

    ancestors: [aten_sup,<0.297.0>]

    message_queue_len: 2

    messages: [poll,{#Ref<0.2491135559.2982936578.179779>,#{}}]

    links: [<0.298.0>]

    dictionary: []

    trap_exit: false

    status: running

    heap_size: 1598

    stack_size: 27

    reductions: 1418193841

  neighbours:

2021-06-13 01:03:55 =SUPERVISOR REPORT====

     Supervisor: {local,aten_sup}

     Context:    child_terminated

     Reason:     {timeout,{gen_server,call,[aten_sink,get_failure_probabilities]}}

     Offender:   [{pid,<0.301.0>},{id,aten_detector},{mfargs,{aten_detector,start_link,[]}},{restart_type,permanent},{shutdown,5000},{child_type,worker}]

Thanks,

Manoj Satapathy

[Michal Kuratczyk]

Hi,

You haven't provided basic info like the RabbitMQ version, but this was likely fixed in 3.8.9: https://github.com/rabbitmq/rabbitmq-server/pull/2450

*** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom it is addressed. If you are not the named addressee then you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and delete this e-mail from your system.***

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/afc4e330-3a94-4100-8ff1-63d82b088599n%40googlegroups.com.

--

Michał

RabbitMQ team

[Manoj Kumar Satapathy]

Hi ,

Rabbitmq version is 3.8.2 .

Can you please let me know what is the exact reason of this crash ?

Thansk,

Manoj Satapathy

[Michal Kuratczyk]

All the details I have are in the link I provided. When there is a network hiccup, an operation can take more than 1 second which was the timeout up until 3.8.9. We simply changed the timeout to 5 seconds to prevent false alarms. Just upgrade and let us know if this still occurs.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/51c54a22-fd94-4bb5-90c1-85d92db16b6en%40googlegroups.com.

--

Michał

RabbitMQ team

[stuartjoh...@gmail.com]

Have you tried a higher version as suggested ?

Also erlang version required 

Has this rabbitMQ broker every started ? Or is this the 1st time you are trying to start it ?

** When Server state == {state,#Ref<0.2491135559.2982936578.179489>,1000,0.99,#{},#{}}

** Reason for termination ==

** {{timeout,{gen_server,call,[aten_sink,get_failure_probabilities]}},[{gen_server,call,2,[{file,"gen_server.erl"},{line,215}]},{aten_detector,handle_info,2,[{file,"src/aten_detector.erl"},{line,97}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}

[Manoj Kumar Satapathy]

Hi ,

This version of Rabbitmq is running in our production server. We have not upgraded yet .

This is not the first time we have restarted the process. I am just trying to find out the exact reason of crash. Is it the timeout network issue ?

erlang version 

============

cat /usr/lib64/erlang/releases/22/OTP_VERSION

22.1.8

Thanks,

Manoj Satapathy