Generic server terminating

bostonne...@gmail.com

unread,

Jun 20, 2016, 8:08:03 PM6/20/16

to rabbitmq-users

C# Client version: 3.6.2

Server version: 3.6.1

Erlang version: 18.2.1

The rabbitmq server log has these errors. Can someone help to explain what could cause this?

Just to let you know that, our application happened to run on the same machine and this time it used a lot of the system CPU and Memory, but this is the first time I've seen this type of the rabbit server log

=ERROR REPORT==== 17-Jun-2016::15:30:20 ===

** Generic server <0.705.0> terminating

** Last message in was {'EXIT',<0.702.0>,shutdown}

** When Server state == {ch,running,rabbit_framing_amqp_0_9_1,5,<0.678.0>,

<0.703.0>,<0.678.0>,

<<"127.0.0.1:49182 -> 127.0.0.1:5672">>,

{lstate,<0.704.0>,false},

none,3,

{[],[]},

{user,<<"guest">>,

[administrator],

[{rabbit_auth_backend_internal,none}]},

<<"/">>,<<"resource_manager_leader">>,

{dict,5,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],

[[<0.3668.0>|

{resource,<<"/">>,queue,

<<"root_task_queue_5ab27a29-342c-4f0a-8ad6-84132d927afa">>}]],

[[<0.282.0>|

{resource,<<"/">>,queue,

<<"ready_tasks_managed_dll">>}]],

[],[],[],[],[],[],

[[<0.273.0>|

{resource,<<"/">>,queue,

<<"monitor_notification_WIN-251">>}],

[<0.9860.0>|

{resource,<<"/">>,queue,

<<"NamedLatch_36240b73-42a9-4239-bcbd-afc3cc2e4935">>}]],

[],[],[],

[[<0.261.0>|

{resource,<<"/">>,queue,

<<"TaskAggregator_notification">>}]],

[]}}},

{state,

{dict,5,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],

[[<0.3668.0>|#Ref<0.0.3.10844>]],

[[<0.282.0>|#Ref<0.0.3.9932>]],

[],[],[],[],[],[],

[[<0.273.0>|#Ref<0.0.3.10007>],

[<0.9860.0>|#Ref<0.0.1.18064>]],

[],[],[],

[[<0.261.0>|#Ref<0.0.3.10008>]],

[]}}},

erlang},

{dict,0,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},

{dict,0,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},

{set,1,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],[],

[<0.282.0>],

[],[],[],[],[],[],[],[],[],[],[],[]}}},

<0.679.0>,

{state,fine,5000,undefined},

false,452,

{{0,nil},{0,nil}},

[],

{{0,nil},{0,nil}},

[{<<"publisher_confirms">>,bool,true},

{<<"exchange_exchange_bindings">>,bool,true},

{<<"basic.nack">>,bool,true},

{<<"consumer_cancel_notify">>,bool,true},

{<<"connection.blocked">>,bool,true},

{<<"authentication_failure_close">>,bool,true}],

none,1,none,flow,[]}

** Reason for termination ==

** {{badmatch,{error,{channel_operation_timeout,5000}}},

[{rabbit_channel,terminate,2,[{file,"src/rabbit_channel.erl"},{line,614}]},

{gen_server2,terminate,3,[{file,"src/gen_server2.erl"},{line,1146}]},

{proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,250}]}]}

** In 'terminate' callback with reason ==

** shutdown

=ERROR REPORT==== 17-Jun-2016::15:30:27 ===

** Generic server <0.701.0> terminating

** Last message in was {'EXIT',<0.698.0>,shutdown}

** When Server state == {ch,running,rabbit_framing_amqp_0_9_1,4,<0.678.0>,

<0.699.0>,<0.678.0>,

<<"127.0.0.1:49182 -> 127.0.0.1:5672">>,

{lstate,<0.700.0>,false},

none,6,

{[],[]},

{user,<<"guest">>,

[administrator],

[{rabbit_auth_backend_internal,none}]},

<<"/">>,<<>>,

{dict,5,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],

[[<0.3668.0>|

{resource,<<"/">>,queue,

<<"root_task_queue_5ab27a29-342c-4f0a-8ad6-84132d927afa">>}]],

[[<0.282.0>|

{resource,<<"/">>,queue,

<<"ready_tasks_managed_dll">>}]],

[[<0.3686.0>|

{resource,<<"/">>,queue,

<<"NamedLatch_5ab27a29-342c-4f0a-8ad6-84132d927afa">>}]],

[],[],[],[],[],

[[<0.273.0>|

{resource,<<"/">>,queue,

<<"monitor_notification_WIN-251">>}]],

[],[],[],

[[<0.261.0>|

{resource,<<"/">>,queue,

<<"TaskAggregator_notification">>}]],

[]}}},

{state,

{dict,5,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],

[[<0.3668.0>|#Ref<0.0.2.16932>]],

[[<0.282.0>|#Ref<0.0.3.9899>]],

[[<0.3686.0>|#Ref<0.0.2.5610>]],

[],[],[],[],[],

[[<0.273.0>|#Ref<0.0.2.5608>]],

[],[],[],

[[<0.261.0>|#Ref<0.0.2.5609>]],

[]}}},

erlang},

{dict,0,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},

{dict,0,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},

{set,1,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},

{{[],[],[],

[<0.282.0>],

[],[],[],[],[],[],[],[],[],[],[],[]}}},

<0.679.0>,

{state,fine,5000,undefined},

false,293,

{{0,nil},{0,nil}},

[],

{{0,nil},{0,nil}},

[{<<"publisher_confirms">>,bool,true},

{<<"exchange_exchange_bindings">>,bool,true},

{<<"basic.nack">>,bool,true},

{<<"consumer_cancel_notify">>,bool,true},

{<<"connection.blocked">>,bool,true},

{<<"authentication_failure_close">>,bool,true}],

none,1,none,flow,[]}

** Reason for termination ==

** {{badmatch,{error,{channel_operation_timeout,5000}}},

[{rabbit_channel,terminate,2,[{file,"src/rabbit_channel.erl"},{line,614}]},

{gen_server2,terminate,3,[{file,"src/gen_server2.erl"},{line,1146}]},

{proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,250}]}]}

** In 'terminate' callback with reason ==

** shutdown

Michael Klishin

unread,

Jun 21, 2016, 7:07:08 AM6/21/16

to rabbitm...@googlegroups.com

This has been discussed on this list several times before. Default channel operation timeout of 5s introduced in 3.6.1

is not sufficient for some in practice. Bump it to 15s (15000), which is what 3.6.2 and future versions use by default:

{channel_operation_timeout, 15000}

(the above goes under the rabbit config section)

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

bostonne...@gmail.com

unread,

Jun 24, 2016, 11:12:58 AM6/24/16

to rabbitmq-users

HI Mike,

For RabbitMQ server 3.6.1, When we tried to add {channel_operation_timeout, 15000} in to the config file, the rabbitMQ service can not be started, regardless of if we added the "," at the end of that line of not.

For RabbitMQ server 3.6.2, the server can be started successfully.

How can we add that line for RabbitMQ server 3.6.1 config?

Michael Klishin

unread,

Jun 24, 2016, 11:30:38 AM6/24/16

to rabbitm...@googlegroups.com

Your config is invalid. There is no version-specific validation so a config that

RabbitMQ 3.6.1 cannot start with RabbitMQ 3.6.2 won't be able to start with either.

boston newton

unread,

Jun 29, 2016, 11:11:01 AM6/29/16

to rabbitm...@googlegroups.com

Hi Mike,

Thanks, we figured it out that if we place that line in a different location in the configuration file, it works now. I assume that should be a fairly low-risk change, correct?

%% SSL listeners are configured in the same fashion as TCP listeners,

%% including the option to control the choice of interface.

%%

%% {ssl_listeners, [5671]},

{channel_operation_timeout, 15000},

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/BGYbRQAPiE4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-user...@googlegroups.com.

Michael Klishin

unread,

Jun 29, 2016, 11:13:54 AM6/29/16

to rabbitm...@googlegroups.com

In 3.6.1 the default is 5s, I think, which some report as insufficient for their workload.

So configuring channel_operation_timeout to be 15000 (15s) is a good idea.

It goes into the `rabbit` section in the config.

Reply all

Reply to author

Forward