crash at startup

[Michael Verrilli]

Hi,

I have a 3-node rabbitmq cluster running on Solaris. RabbitMQ 3.3.2 on Erlang 17

I'm getting a constant crash on every startup with the following.  Any ideas what is causing this and how to fix it?  It ran fine for the past 3 days.The other two servers are running fine.

=ERROR REPORT==== 3-Aug-2015::19:46:55 ===
** Generic server <0.252.0> terminating
** Last message in was {'$gen_cast',
                           {submit_async,
                               #Fun<rabbit_queue_index.27.83671926>}}
** When Server state == undefined
** Reason for termination ==
** {function_clause,
       [{rabbit_queue_index,journal_minus_segment1,
            [{no_pub,no_del,ack},
             {{<<"▒wh7t▒'\e▒▒▒tZ▒.">>,
               {message_properties,undefined,false},
               true},
              no_del,no_ack}],
            []},
        {rabbit_queue_index,'-journal_minus_segment/2-fun-0-',4,[]},
        {array,sparse_foldl_3,7,[{file,"array.erl"},{line,1689}]},
        {array,sparse_foldl_2,9,[{file,"array.erl"},{line,1683}]},
        {rabbit_queue_index,'-recover_journal/1-fun-0-',1,[]},
        {lists,map,2,[{file,"lists.erl"},{line,1237}]},
        {lists,map,2,[{file,"lists.erl"},{line,1237}]},
        {rabbit_queue_index,segment_map,2,[]}]}

[Michael Klishin]

 On 4 August 2015 at 03:53:08, Michael Verrilli (mver...@gmail.com) wrote:
> I have a 3-node rabbitmq cluster running on Solaris. RabbitMQ
> 3.3.2 on Erlang 17
>
> I'm getting a constant crash on every startup with the following.
> Any ideas what is causing this and how to fix it? It ran fine for
> the past 3 days.The other two servers are running fine.

There are 2 references to the failing function in our archive (non-GitHub)
issue tracker, they were fixed in 3.2.x and 3.3.x.

The issue is that a queue index cannot be recovered. if you can't upgrade,
you can reset your node (including by shutting it down and wiping its database
directory), remove it from the cluster with `rabbitmqctl forget_cluster_node`
and add it back.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Verrilli]

Thanks. We have an upgrade scheduled for the end of this month. I've reset the cluster as you suggested and it's back up. 

Thanks again!

Mike