Does mongodb support incremental backup?
genson
Dwight Merriman
Does mongodb support incremental backup?
--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
Eliot Horowitz
llvm or something) that will be incremental.
It's also theoretically possible to backup the oplog, but that would
require a bunch of code to easily replay it.
genson
On 8月10日, 下午11时48分, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> If you use fsync+lock and do file system snapshots on the slave (using
> llvm or something) that will be incremental.
>
> It's also theoretically possible to backup the oplog, but that would
> require a bunch of code to easily replay it.
>
> On Tue, Aug 10, 2010 at 8:58 AM, Dwight Merriman <dwi...@10gen.com> wrote:
> > not really
> > you could do some things yourself with a replica slave.
> > you could use mongodump to do some incremental things too
>
genson
On 8月10日, 下午11时48分, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> llvm or something) that will be incremental.
>
> It's also theoretically possible to backup the oplog, but that would
> require a bunch of code to easily replay it.
>
> On Tue, Aug 10, 2010 at 8:58 AM, Dwight Merriman <dwi...@10gen.com> wrote:
> > not really
> > you could do some things yourself with a replica slave.
> > you could use mongodump to do some incremental things too
>
kevin
you could use mongodump to do some incremental things too
thanks
GVP
really better than simply slaving and backing up the slave.
If you "incrementally backup" a database, then to "restore" the backup
you have to replay everything since the last backup.
Typically, when you do a restore, it's because your primary is down.
When your primary is down, you want a restore to be quick, but
replaying days of transactions is slow.
So the simple solution is to set up a slave. The slave typically
doesn't need as much power (so it can do other things). The slave is
inherently "incremental".
So the standard setup is to configure a slave and then take a backup
of the slave. This is relatively easy to do, you turn off the slave,
backup the data files (tar, 7z, whatever), and then turn the slave
back on.
@genson: "do you have any plan to support that in the future?"
I don't work for 10Gen, but as a user of MongoDB, I don't see a really
good case for "incremental backups". There are dozens of things I'd
rather see from them.
Incremental backups are:
- not easier than "slave and backup the slave"
- not very useful for bringing up servers quickly
- only mildly less disk intensive than slave/backup
Maybe there's a business case I'm missing.
But you'd definitely need to clarify that case.
Markus Gattol
for the period of time it takes to pull a "backup" (copy stuff in
dbpath), then that is certainly preferred over having that semantics
sitting with MongoDB itself.
Why? Because there are many ways to do it that people already know and
which in fact are basic Unix knowledge e.g. using scp or rsync ... my
favorite is the scp than you can resume (link below) and which is
encrypted (i.e. you can do you "backup" trough an insecure network like
the Internet).
http://www.markus-gattol.name/ws/mongodb.html#how_do_i_physically_migrate_a_database
Using the semantics (shut down a slave, use common Unix tools) you can
also automate a lot better and tailor the whole procedure to your
individual needs ... e.g. you might simply create yourself a Python
script that can do all (shut down the slave, issue scp, write logs,
start the slave again) the magic, including determining the times when
it should happen (based on diskspace, some other business metrics,
etc.). Otherwise, if not that complicated, just feed it to cron or
incron.
Bottom Line: I agree with GVP, in 9 out of 10 cases people would
probably want to have their own little tailored backup magic, thus have
a loose coupling with MongoDB in this regard (read: flexibility).
If MongoDB had this feature (incremental backups) built-in, I am pretty
sure folks would be asking how to tailor it to their needs so why even
implement something that would be to static for most people anyways;
better focus on other things in my opinion ...
Kenneth Jiang
> On Aug 10, 7:23 pm, kevin <kevincastigli...@...> wrote:
> > On Tue, Aug 10, 2010 at 5:58 AM, Dwight Merriman <dwi...@...> wrote:
> > > you could use mongodump to do some incremental things too
> >
> > > can you share an example on how to do some incremental things with
> >
> > mongodump?
> > thanks
>
you have got a very good point of using slave as incremental backup. But I'm
working on a system that seems to justify incremental backup.
Basically the system will have in totally about 500TB of data. We plan to
store all these data into MongoDB, and use tape as backup material in
case all servers and harddrives get smashed. ;)
Now the challenges are, if I use a slave for such back up purpose,
- Extra 500TB needs to be acquired, which cost us significantly,
- Full backup of 500TB to tapes in a reasonable time frame
(say 8 hours) is not realistic.
In our case, incremental backup does seem to be the only choice.
But we don't know how this can be done... :(
Your help with our problems will be really appreciated!
Kenneth
