handling server crashes on master/slave setup

[Jakub Suder]

Hi,

I wanted to ask what's the recommended way to handle a server that was
shut down uncleanly (e.g. a power failure) in a master/slave setup?

If I boot up the server again, I suppose it will have the lock file in
the Mongo directory, so I understand that I should remove it. What
happens then - do I have to always run --repair after that? Or only if
Mongo doesn't start without --repair even after removing the lock
file?

How long can the repair take? If it can take long, should I make the
slave db run as master during that time? How do I synchronize the
databases later if the slave has some new changes and the master has
some data from before the power failure that didn't make it into the
slave?

Jakub Suder

[Alvin Richards]

in the case of an unclean shutdown, you must always
-- remove the lock file
-- start mongod with --repair
-- after the repair is complete, then you can restart mongod

The time a repair takes is a function of the amount of data and
indexes and the I/O capability of your system. Right now you need
access to double the amount of storage as the repair takes place. We
will be optimizing this going forward. You can use the --repairpath to
indicate an alternate volume if you need to provision space just for
the repair operation.

-Alvin

[Markus Gattol]

In addition to what Alvin said, also have a look at
http://www.markus-gattol.name/ws/mongodb.html#backup_recovery as it
points out a few different scenarios.

[Jakub Suder]

On Oct 11, 7:49 pm, Markus Gattol <markus.gat...@gmail.com> wrote:
> In addition to what Alvin said, also have a look athttp://www.markus-gattol.name/ws/mongodb.html#backup_recoveryas it

> points out a few different scenarios.

"If we did not do any writes during the session that shutdown
uncleanly, our data set is fine. All we need to do is remove the
mongod.lock file after rebooting the machine and start mongod." - how
can I make sure that there weren't any writes happening at the moment
of the crash?

Also, I still don't know what to do in a scenario where I have a
master and a slave (not replica set), and master db is killed
suddenly. Should I:
- just start repair on master, assume that it will fix everything
without losing anything, and wait?
- make slave act as master, run repair on old master, and then
synchronize them somehow?
- make slave act as master, copy whole data from slave and overwrite
master db with it?

JS

[Markus Gattol]

>> In addition to what Alvin said, also have a look
>> athttp://www.markus-gattol.name/ws/mongodb.html#backup_recoveryas it
>> points out a few different scenarios.

Jakub> "If we did not do any writes during the session that shutdown
Jakub> uncleanly, our data set is fine. All we need to do is remove the
Jakub> mongod.lock file after rebooting the machine and start mongod."
Jakub> - How can I make sure that there weren't any writes happening at
Jakub> the moment of the crash?

You can only be sure if you've locked the database (which you probably
have not done in such case). If you have the time too, start the primary
with --repair, otherwise continue with the old secondary (after
promoting it to master).

[Eliot Horowitz]

The safest thing to do is make the slave act as a master and resync
the old master.
This way you know there were no issues, and resyncing is about the
same amount of work as a repair.
Now once 1.8 comes out, this will change a bit.

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>