possible ways lose all the data
Wu Zhe
my data, fortunately this is not in production, all the data are just
for testing. There are no flushdb/flushall commands in my code and
surely I didn't run the commands in redis-cli. The version is 2.0.4-2
in archlinux with default configuration.
In order to prevent this happening in production in the future, I'd
like to know what are the possible legit ways to effectively lose all
the data in redis. Thanks.
Xiangrong Fang
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
Salvatore Sanfilippo
it is not common to lose all the data as Redis even if started without
a config file will dump from time to time on disk. This are the
reasons that may lead to losing all the data, I hope you'll find what
of this reasons was the cause of your problem:
- Starting redis with a redis.conf that lacks "save" directives means:
no persistence at all (unless AOF is enabled).
- Starting redis with persistence configured correctly, but against a
directory where it can't save. For instance for lack of permissions,
or because the disk is full.
- Calling FLUSHALL / FLUSHDB.
- Removing the dump.rdb file for error, and in general, not creating
backups of dump.rdb, may lead to data loss as the disk may broke, or
the sysadmin may launch a wrong command, and so forth.
- Executing a new empty Redis instance in the same directory where
there was already one running. This instance will save against the
same dump.rdb file. The data is still in the memory of the first Redis
instance, but if this gets terminated, no luck (see "dir" config
directive)
- Executing master and slave in the same directory (see "dir" config directive)
I think I covered a good number of cases, but probably there are more,
so instead of listing all the things that can go bad I want to also
list all the things to do to make sure everything will work well:
1) Edit the example redis.conf and write a good one for you.
2) Run redis using as first argument the path for your configuration
file, otherwise it will start with default config.
3) Make sure to persist on disk, that is, you need "save" directives,
or append only file persistence configured.
4) Make sure to do backups at least every 24 hours.
5) From time to time, try to restore your backup. With Redis that's
trivial. Put your backed up dump.rdb into a development box, and start
Redis. It will load the data and you can check with redis-cli or with
other systems that everything seems fine.
6) If you need zero data loss in case of a problem, make sure to
configure replication.
7) For better durability use AOF instead of snapshotting (snapshotting
== dump.rdb files).
8) If you really care about your data (financial information or other
vital things) use RAM with error correction.
Cheers,
Salvatore
> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>
--
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele
Sam Stokes
One thing that caught us out is that when Redis saves to disk, it does
so by spawning a child process, and if it runs out of memory while
trying to do so, it fails pretty quietly. That means if your machine
doesn't *ever* have enough RAM free for the background save, Redis
will never save to disk, even if you configured it to. Then if you
ever restart Redis - either deliberately, or because it ran out of RAM
altogether and crashed - it'll try to reload from disk, but because it
never managed to save, there's nothing to reload, and you lose all
your data.
You can check your Redis logs for something like "background save
failed" (I can't remember the exact error, but it's fairly obvious) to
see if that's what happened to your testing instance.
You do need to make sure to leave some RAM spare so that the
background save has room to run. Search for "overcommit_memory" in
http://redis.io/topics/faq for more details on this.
Hope that helps,
--
Sam
Wu Zhe
You guys are so helpful, thanks!
Matthew Hooker
It seems to me I have accidentally run two redis instances and the new one emptied the dump.rdb.
You guys are so helpful, thanks!
--
Salvatore Sanfilippo
> Would it be helpful for redis to acquire a lock on the dump.rdb file so that
> no two processes can write to it at the same time? If so, I'd be happy to
> try my hand at writing a patch.
Hello Matt,
locking would be difficult as we move temp names against the old one
with rename(2).
Also locking does not solve the issues, as all the other instances
will not be able to persist.
A possible solution could be detecting if there is another instance
running in the same dir before starting, but this kind of stuff often
have the bad effect of protecting people doing wrong things, and
possibly creating big problems to people doing the right things (for
instance your "instance-is-present.pid" file will not be deleted
correctly and your well configured instance will not start causing
troubles).
So it's probably not perfect as it is today but a good compromise...
Cheers,
Salvatore
> Thanks,
> --Matt
> On Jan 12, 2011, at 1:47 PM, Wu Zhe <jess...@gmail.com> wrote:
>
> It seems to me I have accidentally run two redis instances and the new one
> emptied the dump.rdb.
>
> You guys are so helpful, thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
>
--