[erlang-questions] Mnesia ETS crash

[Mike Oxford]

=ERROR REPORT==== 18-Jun-2012::15:01:49 ===

Error in process <0.1743.0> on node '<redacted>' with exit value: {badarg,[{ets,safe_fixtable,[px_channel_log,false]},{mnesia_lib,db_fixtable,3},{mnesia_loader,cleanup_tab_copier,3},{mnesia_loader,send_table,3},{mnesia_controller,send_and_re

ply,2}]}

Logging shows it was then restarted and {inconsistent_database, starting_partitioned_network...

mnesia is Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:24:24] [rq:24] [async-threads:64] [hipe] [kernel-poll:true] so it shouldn't fall victim to the safe_fixtable/next issues from R13x

This has happened to us twice now, both times exactly the same error.  System runs other ram_only nodes as well with no issues; this one is a disc_ table with light load.

Thoughts?

-mox

[Mike Oxford]

Further information:

1)  The table is question does exist, has existed, and has been running for a while.

2)  safe_fixtable fails with badarg if the table does not exist.

#1 runs for a good long while (month?) and then suddenly #2 pops up.    Why does mnesia 'lose' the table?

Here's more context

<snip>

===== ALIVE Tue Jul 31 17:22:07 PDT 2012

===== ALIVE Tue Jul 31 17:37:07 PDT 2012

===== ALIVE Tue Jul 31 17:52:07 PDT 2012

===== Tue Jul 31 18:04:17 PDT 2012

=ERROR REPORT==== 31-Jul-2012::18:04:17 ===

Error in process <0.15759.10> on node '<redacted>' with exit value: {badarg,

[{ets,safe_fixtable,[px_channel_log,false]},{mnesia_lib,db_fixtable,3},

{mnesia_loader,cleanup_tab_copier,3},{mnesia_loader,send_table,3},{mnesia_controller,send_and_reply,2}]}

=====

===== LOGGING STARTED Tue Jul 31 18:04:19 PDT 2012

=====

Exec: /usr/local/chat-game-gwm/current/erts-5.8.4/bin/erlexec -boot /usr/local/chat-game-gwm/current/releases/1/chat -mode 

embedded -config /usr/local/chat-game-gwm/current/etc/app.config -args_file /usr/local/chat-game-gwm/current/etc/vm.args --

 console

Root: /usr/local/chat-game-gwm/current

Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:24:24] [rq:24] [async-threads:64] [hipe] [kernel-poll:true]

=ERROR REPORT==== 31-Jul-2012::18:04:22 ===

Mnesia('gwm_game@<redacted>'): ** ERROR ** mnesia_event got {inconsistent_database, starting_partitioned_network,

'gwm_game@<redacted>'}

DEBUG: mnesia_helpers:118 - "{ \"Connecting to cluster: \" , NodeList }"

</end_snip>

[Mike Oxford]

No one has seen this before?

Thanks,

-mox