operation queue.declare caused a channel exception not_found: failed to perform operation on queue 'queue_nqme' in vhost '/' due to timeout

[Mif_00]

Hello.
I have RabbitMq version 3.8.2 installed.
When I start the server, everything works fine, but after a while, when I try to declare the queue, I get an error
"operation queue.declare caused a channel exception not_found: failed to perform operation on queue 'queue_nqme' in vhost '/' due to timeout".
I successfully manage to create a connection, get a channel, but there is an error when declaring the queue.
Rabbitmqctl ping, rabbitmqctl node_health_check, rabbitmqctl report - do not show that there are problems.
No alarms were registered.
I only have one node, no replication.

Please tell me why this problem may occur?
I don't have any more ideas.

If you need more information, I will provide it.

Thanks.

[Wyatt Chun]

Can you provide some code piece?

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/5c29d3ea-23f8-41f4-9aab-64aa8e7595e1%40googlegroups.com.

[Mif_00]

Here's the part where I try to declare the queue

$oConnection = new AMQPStreamConnection('localhost', '5672', 'xxxx', 'xxxx'); //is ok
$oChanel = $oConnection->channel(); //is ok

try {
    $queue = $oChanel->queue_declare('test-queue', false, true, false, false);
} catch (AMQPProtocolChannelException $e) {
    ddd($e->getMessage()); //operation queue.declare caused a channel exception not_found: failed to perform operation on queue 'queue_nqme' in vhost '/' due to timeout
}

вторник, 4 февраля 2020 г., 13:19:57 UTC+3 пользователь Wyatt Chun написал:

Can you provide some code piece?

On Tue, Feb 4, 2020 at 6:15 PM Mif_00 <fedorov...@gmail.com> wrote:

Hello.
I have RabbitMq version 3.8.2 installed.
When I start the server, everything works fine, but after a while, when I try to declare the queue, I get an error
"operation queue.declare caused a channel exception not_found: failed to perform operation on queue 'queue_nqme' in vhost '/' due to timeout".
I successfully manage to create a connection, get a channel, but there is an error when declaring the queue.
Rabbitmqctl ping, rabbitmqctl node_health_check, rabbitmqctl report - do not show that there are problems.
No alarms were registered.
I only have one node, no replication.

Please tell me why this problem may occur?
I don't have any more ideas.

If you need more information, I will provide it.

Thanks.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitm...@googlegroups.com.

[Wesley Peng]

that's strange. you delcare queue with name "test-queue", but the error
info saying it's "queue_nqme'.

regards.

> it, send an email to rabbitm...@googlegroups.com <javascript:>.

> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/rabbitmq-users/5c29d3ea-23f8-41f4-9aab-64aa8e7595e1%40googlegroups.com

> <https://groups.google.com/d/msgid/rabbitmq-users/5c29d3ea-23f8-41f4-9aab-64aa8e7595e1%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "rabbitmq-users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to rabbitmq-user...@googlegroups.com
> <mailto:rabbitmq-user...@googlegroups.com>.

> To view this discussion on the web, visit

> https://groups.google.com/d/msgid/rabbitmq-users/59303faa-6d34-469e-82c6-96ab2c7c77a2%40googlegroups.com
> <https://groups.google.com/d/msgid/rabbitmq-users/59303faa-6d34-469e-82c6-96ab2c7c77a2%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Mif_00]

I'm sorry, I just tried to declare a queue with different names several times. Because of this there was confusion.

if i  delcare queue with name "test-queue", i get error with "test-queue"

sorry


вторник, 4 февраля 2020 г., 13:41:38 UTC+3 пользователь Wesley Peng написал:

> an email to rabbitm...@googlegroups.com
> <mailto:rabbitmq-users+unsub...@googlegroups.com>.

[Michael Klishin]

We cannot suggest much without server logs and a set of steps to reproduce. You are getting free support on this list so please help others help you.

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/5c29d3ea-23f8-41f4-9aab-64aa8e7595e1%40googlegroups.com.

--

Staff Software Engineer, Pivotal/RabbitMQ

[Mif_00]

I apologize for the previous non-informative question.

I tried installing version 3.7.9. But I got the same error again.

It all started with these errors in the logs.

in the log file:

2020-02-05 13:23:05.976 [error] <0.276.0> ** Generic server vm_memory_monitor terminating
** Last message in was update
** When Server state == {state,25111576576,10044630630,94429184,0.4,1000,#Ref<0.2645887833.2223243265.92390>,false,{#Fun<rabbit_alarm.0.123076882>,#Fun<rabbit_alarm.1.123076882>},{unix,linux},"20883",4096,[47,112,114,111,99,47,"20883",47,115,116,97,116,109]}
** Reason for termination ==
** {{badmatch,{error,enfile}},[{vm_memory_monitor,read_proc_file,1,[{file,"src/vm_memory_monitor.erl"},{line,564}]},{vm_memory_monitor,get_process_memory_using_strategy,2,[{file,"src/vm_memory_monitor.erl"},{line,291}]},{vm_memory_monitor,update_process_memory,1,[{file,"src/vm_memory_monitor.erl"},{line,274}]},{vm_memory_monitor,internal_update,1,[{file,"src/vm_memory_monitor.erl"},{line,408}]},{vm_memory_monitor,handle_info,2,[{file,"src/vm_memory_monitor.erl"},{line,238}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}

in the crash file:

2020-02-05 13:23:05 =ERROR REPORT====
** Generic server vm_memory_monitor terminating
** Last message in was update
** When Server state == {state,25111576576,10044630630,94429184,0.4,1000,#Ref<0.2645887833.2223243265.92390>,false,{#Fun<rabbit_alarm.0.123076882>,#Fun<rabbit_alarm.1.123076882>},{unix,linux},"20883",4096,[47,112,114,111,99,47,"20883",47,115,116,97,116,109]}
** Reason for termination ==
** {{badmatch,{error,enfile}},[{vm_memory_monitor,read_proc_file,1,[{file,"src/vm_memory_monitor.erl"},{line,564}]},{vm_memory_monitor,get_process_memory_using_strategy,2,[{file,"src/vm_memory_monitor.erl"},{line,291}]},{vm_memory_monitor,update_process_memory,1,[{file,"src/vm_memory_monitor.erl"},{line,274}]},{vm_memory_monitor,internal_update,1,[{file,"src/vm_memory_monitor.erl"},{line,408}]},{vm_memory_monitor,handle_info,2,[{file,"src/vm_memory_monitor.erl"},{line,238}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
2020-02-05 13:23:05 =CRASH REPORT====
 crasher:
 initial call: vm_memory_monitor:init/1
 pid: <0.276.0>
 registered_name: vm_memory_monitor
 exception error: {{badmatch,{error,enfile}},[{vm_memory_monitor,read_proc_file,1,[{file,"src/vm_memory_monitor.erl"},{line,564}]},{vm_memory_monitor,get_process_memory_using_strategy,2,[{file,"src/vm_memory_monitor.erl"},{line,291}]},{vm_memory_monitor,update_process_memory,1,[{file,"src/vm_memory_monitor.erl"},{line,274}]},{vm_memory_monitor,internal_update,1,[{file,"src/vm_memory_monitor.erl"},{line,408}]},{vm_memory_monitor,handle_info,2,[{file,"src/vm_memory_monitor.erl"},{line,238}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
 ancestors: [vm_memory_monitor_sup,rabbit_sup,<0.229.0>]
 message_queue_len: 0
 messages: []
 links: [<0.275.0>]
 dictionary: []
 trap_exit: false
 status: running
 heap_size: 2586
 stack_size: 27
 reductions: 1005952312
 neighbours:
2020-02-05 13:23:05 =SUPERVISOR REPORT====
 Supervisor: {local,vm_memory_monitor_sup}
 Context: child_terminated
 Reason: {{badmatch,{error,enfile}},[{vm_memory_monitor,read_proc_file,1,[{file,"src/vm_memory_monitor.erl"},{line,564}]},{vm_memory_monitor,get_process_memory_using_strategy,2,[{file,"src/vm_memory_monitor.erl"},{line,291}]},{vm_memory_monitor,update_process_memory,1,[{file,"src/vm_memory_monitor.erl"},{line,274}]},{vm_memory_monitor,internal_update,1,[{file,"src/vm_memory_monitor.erl"},{line,408}]},{vm_memory_monitor,handle_info,2,[{file,"src/vm_memory_monitor.erl"},{line,238}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
 Offender: [{pid,<0.276.0>},{name,vm_memory_monitor},{mfargs,{vm_memory_monitor,start_link,[0.4,#Fun<rabbit_alarm.0.123076882>,#Fun<rabbit_alarm.1.123076882>]}},{restart_type,transient},{shutdown,30000},{child_type,worker}]

Put the logs in the attached files

I will be grateful for your help


вторник, 4 февраля 2020 г., 17:44:55 UTC+3 пользователь Michael Klishin написал:

We cannot suggest much without server logs and a set of steps to reproduce. You are getting free support on this list so please help others help you.

On Tue, 4 Feb 2020 at 13:16, Mif_00 <fedorov...@gmail.com> wrote:

Hello.
I have RabbitMq version 3.8.2 installed.
When I start the server, everything works fine, but after a while, when I try to declare the queue, I get an error
"operation queue.declare caused a channel exception not_found: failed to perform operation on queue 'queue_nqme' in vhost '/' due to timeout".
I successfully manage to create a connection, get a channel, but there is an error when declaring the queue.
Rabbitmqctl ping, rabbitmqctl node_health_check, rabbitmqctl report - do not show that there are problems.
No alarms were registered.
I only have one node, no replication.

Please tell me why this problem may occur?
I don't have any more ideas.

If you need more information, I will provide it.

Thanks.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitm...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/5c29d3ea-23f8-41f4-9aab-64aa8e7595e1%40googlegroups.com.

[Michael Klishin]

That's really weird. The "crash" (an unhandled exception) is from a node memory monitor which
tries to read a /proc file (I'll guess that this is on Linux) and that failed with an enfile.

According to [1] your node is out of file descriptors. Which is a pretty well documented item [2][3].

I don't see how this is directly related but a node that's out of file descriptors will fail to do all kinds of essential things.

1.http://man7.org/linux/man-pages/man3/errno.3.html
2. https://www.rabbitmq.com/production-checklist.html#resource-limits-file-handle-limit
3. https://www.rabbitmq.com/networking.html#open-file-handle-limit

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/a3e06662-0794-4858-bf84-62424b731ebb%40googlegroups.com.



--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Wesley Peng]

https://www.rabbitmq.com/production-checklist.html#resource-limits-file-handle-limit

Thanks for the URL,  RMQ does have great documentation system.

Regards 

6 лютого 2020 р., 00:20:42,від "Michael Klishin" <mkli...@pivotal.io>:

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAGcLz6VDuv1h5DaZbpEDfriKASQUzfM9UaHJ1UNAzdTJfSFmLQ%40mail.gmail.com.

[Mif_00]

Thanks for the help.

Increased the limit of open file handles in OS.

The problem did not repeat after that

среда, 5 февраля 2020 г., 19:20:42 UTC+3 пользователь Michael Klishin написал:

[Michael Klishin]

Thank you for reporting back to the list to confirm our hypothesis!

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/d6012c29-6f5f-475d-8696-c5d89f339d90%40googlegroups.com.