durability

[paul mineiro]

---------- Forwarded message ----------
Date: Mon, 21 Jul 2008 15:59:22 -0600
From: Dave Smith <dsm...@thehive.com>
To: Paul Mineiro <pmin...@herbie.ddv.com>
Subject: Mnesiaex + commits

Paul,

So we've been doing some moderate-scale load testing with mnesiaex and
tcerl over the past week on R12B3 -- so far so good. One odd thing
that I have observed is that there seem to be some discrepancies with
what a "commit" means when using tcerl. Specifically, if one does a
put into a tcerl table and follows that up (after the txn, of course)
with exiting the erlang emulator via Ctrl-C, Ctrl-C (harsh, I know)
the value "committed" is no longer present when the system is
restarted. This is direct contrast to the mnesia/dets behaviour, where
a committed txn == guaranteed to be there, even if emulator is harshly
killed.

This unusual behaviour reveals itself in our tests with a db that has
a bunch of keys on restart, but no values (even though we're only
writing with transacations). The easy fix would appear to be a call to
tcbdbsync() on any write within the driver. Obviously, this is fraught
with performance implications. The only other alternative I can think
of is to somehow "know" when a transaction is done writing and kick a
call to tcbdb:sync -- but that's a little hacky as well. Am I missing
something here (other than a deep understanding of how tcerl is using
the mnesia txn system)? :)

Thanks,

D.

[paul mineiro]

I don't have a good answer to this. I've seen similar behaviour, but
in our application we don't care because our mnesia is distributed so
at reboot we will load a remote copy. However I can understand how
lots of people run mnesia not in distributed mode, so it'd be nice to
figure this out. It would also be nice for us in case we lose all
replicas of a table at once and need to fall back to the disk to load
the data.

I would predict, based upon what I've seen, that your database is more
messed up than just missing values. I had a database in this state
(keys without values) and got it into an infinite loop because
tcbdb:range kept returning the same keys even though it should advance
the iterator. I assumed the internal tc structures were f'd.

My analysis is there has to be some (tokyocabinet) application level
caching going on, because killing the Erlang emulator leaves the OS
page cache intact so if the updates were actually written out things
should be ok; so unless you are killing the emulator inside the port
driver function, the file should be intact. I think we need more help
from the tokyocabinet guy to get to the bottom of this.

Re: syncing, we could change mnesiaex to execute a sync callback when
transactions finish but this would still leave you vulnerable to
crashes in the middle of a transaction, so I don't think it would help
much.

-- p

[paul mineiro]

One more thing: I suspect there is a bug dumping the log at startup.
I detailed that in another mnesiaex-discuss thread.

So I think at least two things are happening here:

1. db file is getting corrupt. in this case, replaying the
transaction log at startup won't help, because I don't trust the
inserts that it (re)issues will work.
2. log dump at startup appears broken (see previous thread).

-- p

On Jul 21, 3:26 pm, paul mineiro <paul-goo...@mineiro.com> wrote: