{"init terminating in do_boot",{rabbit,failure_during_boot,{error,{cannot_read_file,....,enomem}}}}

110 views
Skip to first unread message

Barbara Huckleby

unread,
May 19, 2017, 1:02:55 PM5/19/17
to rabbitmq-users
I am trying to get rabbitmq-server to start on reboot.   It works fine if I start it manually, but fails on reboot with the following error:

BOOT FAILED
===========

Error description:
   {error,{cannot_read_file,"/var/bdsprod/file_land_dir/rabbitmq/rabbit@u060xxxx13a/nodes_running_at_shutdown",
                            enomem}}

Log files (may contain more information):
   /var/log/rabbitmq/rab...@u060xxxx13a.log
   /var/log/rabbitmq/rab...@u060xxxx13a-sasl.log

Stack trace:
   [{rabbit_node_monitor,try_read_file,1,[]},
    {rabbit_node_monitor,prepare_cluster_status_files,0,[]},
    {rabbit,'-boot/0-fun-1-',0,[]},
    {rabbit,start_it,1,[]},
    {init,start_it,1,[]},
    {init,start_em,1,[]}]

{"init terminating in do_boot",{rabbit,failure_during_boot,{error,{cannot_read_file,"/var/bdsprod/file_land_dir/rabbitmq/rabbit@u060xxxx13a/nodes_running_at_shutdown",enomem}}}}
ESC[60G[ESC[0;31mFAILEDESC[0;39m]


It is the last thing that starts on boot using /etc/init.d/rabbitmq-server.   


Michael Klishin

unread,
May 19, 2017, 2:12:47 PM5/19/17
to rabbitm...@googlegroups.com
Your node tries to open /var/bdsprod/file_land_dir/rabbitmq/rabbit@u060xxxx13a/nodes_running_at_shutdown
and it fails with ENOMEM.

The file should generally be very small (less than 1 kiB) but ENOMEM is meant to say that
"the system has run out of memory":

I can't think of a case where we've seen a file read return this code.


On Fri, May 19, 2017 at 6:02 PM, Barbara Huckleby <barbara....@kroger.com> wrote:
I am trying to get rabbitmq-server to start on reboot.   It works fine if I start it manually, but fails on reboot with the following error:

BOOT FAILED
===========

Error description:
   {error,{cannot_read_file,"/var/bdsprod/file_land_dir/rabbitmq/rabbit@u060xxxx13a/nodes_running_at_shutdown",
                            enomem}}

Log files (may contain more information):
   /var/log/rabbitmq/rabbit@u060xxxx13a.log
   /var/log/rabbitmq/rabbit@u060xxxx13a-sasl.log

Stack trace:
   [{rabbit_node_monitor,try_read_file,1,[]},
    {rabbit_node_monitor,prepare_cluster_status_files,0,[]},
    {rabbit,'-boot/0-fun-1-',0,[]},
    {rabbit,start_it,1,[]},
    {init,start_it,1,[]},
    {init,start_em,1,[]}]

{"init terminating in do_boot",{rabbit,failure_during_boot,{error,{cannot_read_file,"/var/bdsprod/file_land_dir/rabbitmq/rabbit@u060xxxx13a/nodes_running_at_shutdown",enomem}}}}
ESC[60G[ESC[0;31mFAILEDESC[0;39m]


It is the last thing that starts on boot using /etc/init.d/rabbitmq-server.   


--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
MK

Staff Software Engineer, Pivotal/RabbitMQ

Huckleby, Barbara A

unread,
May 19, 2017, 2:20:55 PM5/19/17
to rabbitm...@googlegroups.com

It has 22 bytes in it.

 

From: rabbitm...@googlegroups.com [mailto:rabbitm...@googlegroups.com] On Behalf Of Michael Klishin
Sent: Friday, May 19, 2017 2:13 PM
To: rabbitm...@googlegroups.com
Subject: Re: [rabbitmq-users] {"init terminating in do_boot",{rabbit,failure_during_boot,{error,{cannot_read_file,....,enomem}}}}

 

Your node tries to open /var/bdsprod/file_land_dir/rabbitmq/rabbit@u060xxxx13a/nodes_running_at_shutdown

and it fails with ENOMEM.

 

The file should generally be very small (less than 1 kiB) but ENOMEM is meant to say that

"the system has run out of memory":

 

I can't think of a case where we've seen a file read return this code.

 

On Fri, May 19, 2017 at 6:02 PM, Barbara Huckleby <barbara....@kroger.com> wrote:

I am trying to get rabbitmq-server to start on reboot.   It works fine if I start it manually, but fails on reboot with the following error:

 

BOOT FAILED

===========

 

Error description:

   {error,{cannot_read_file,"/var/bdsprod/file_land_dir/rabbitmq/rabbit@u060xxxx13a/nodes_running_at_shutdown",

                            enomem}}

 

Log files (may contain more information):

 

Stack trace:

   [{rabbit_node_monitor,try_read_file,1,[]},

    {rabbit_node_monitor,prepare_cluster_status_files,0,[]},

    {rabbit,'-boot/0-fun-1-',0,[]},

    {rabbit,start_it,1,[]},

    {init,start_it,1,[]},

    {init,start_em,1,[]}]

 

{"init terminating in do_boot",{rabbit,failure_during_boot,{error,{cannot_read_file,"/var/bdsprod/file_land_dir/rabbitmq/rabbit@u060xxxx13a/nodes_running_at_shutdown",enomem}}}}

ESC[60G[ESC[0;31mFAILEDESC[0;39m]

 

 

It is the last thing that starts on boot using /etc/init.d/rabbitmq-server.   

 

 

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.



 

--

MK

 

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/-I48vDPmgWs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.




This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain information that is confidential and protected by law from unauthorized disclosure. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.

Huckleby, Barbara A

unread,
May 19, 2017, 2:30:46 PM5/19/17
to rabbitm...@googlegroups.com

We figured it out.  It was a strange error.  The file system was encrypted and root does not have clear text view of that file system.    It was confusing because it mentioned enomem and not permissions.   So we were trying to troubleshoot memory issues.  

Michael Klishin

unread,
May 19, 2017, 4:30:59 PM5/19/17
to rabbitm...@googlegroups.com
Thank you for reporting back!
Reply all
Reply to author
Forward
0 new messages