Could we use EBS snapshot for Redis backup?

[Junius Luo]

For example, running Redis on AWS, and store Redis data to EBS. To backup Redis data, is it ok to use EBS snapshot instead of copying rdb file? 

Steps like:

1. run "save" command.

2. take EBS snapshot

[Itamar Haber]

Hello Junius,

Yes, an EBS snapshot is a valid means for backup retention. Note, however, that you'd be better off in most cases by using `BGSAVE` instead of the blocking `SAVE`.

Cheers,

Itamar

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[Junius Luo]

Thanks Itamar! yep, BGSAVE is definitely better.

Thanks,

Junius

On Friday, December 8, 2017 at 4:26:39 PM UTC-8, Itamar Haber wrote:

Hello Junius,

Yes, an EBS snapshot is a valid means for backup retention. Note, however, that you'd be better off in most cases by using `BGSAVE` instead of the blocking `SAVE`.

Cheers,

Itamar

On Dec 9, 2017 1:20 AM, "Junius Luo" <luo.j...@gmail.com> wrote:

For example, running Redis on AWS, and store Redis data to EBS. To backup Redis data, is it ok to use EBS snapshot instead of copying rdb file? 

Steps like:

1. run "save" command.

2. take EBS snapshot

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

[Junius Luo]

If AOF is enabled, do we still need to run BGSAVE first? As AOF logs every write, would be able to take EBS snapshot directly without BGSAVE?

Thanks,

Junius

[Itamar Haber]

AOF and RDB are two different and complementing backup strategies - the `SAVE` and `BGSAVE` commands, as well as the `save` configuration directive, control the latter.

From my experience, AOF on network-attached storage (e.g. EBS) isn't good practice in production, due to the latencies involved.

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--

Itamar Haber | Chief OSS Education Officer
Redis Labs ~/redis

Mobile: +972 (54) 567 9692
Twitter: @itamarhaber
Skype: itamar.haber

[Junius Luo]

Thanks Itamar!

So for AOF enabled, it is good to take volume snapshot directly. right?

If EBS is not a good practice, what is the suggestion for Redis on AWS? Without AOF, the default periodical snapshot is 60 seconds if at least 10000 keys changed and more if less keys changed. If node crashes, Redis may lose 60 seconds changes. It is not good to reduce the snapshot too frequent, right?

Thanks,

Junius

Itamar Haber | Chief OSS Education Officer
Redis Labs ~/redis

[Itamar Haber]

> So for AOF enabled, it is good to take volume snapshot directly. right?

I believe that shouldn't present any problems.

> ... what is the suggestion for Redis on AWS?

That really depends on your requirements from the system in terms of load and performance, but at the end of the day you don't want IO to be a bottleneck. You could use the server's local (ephemeral) storage for AOF, which is more performant but does not protect against that cloud instance's failure.

Also keep in mind that a single Redis server can scale only that much and is, by nature, not highly available regardless its persistency settings. In scenarios where these are valid considerations, other deployment patterns - namely replication and clustering - should be considered. As an added bonus when using replication, for example, you can mitigate some of the IO penalty by enabling the AOF only on a slave(s).

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[Junius Luo]

Thanks Itamar!

Redis Cluster is a good tech to scale Redis! Enabling AOF on one slave only is an interesting option. It may not be a good option for us now. Redis Cluster will promote the slave to master automatically. If the slave with AOF is promoted to master, it will perform slower than other masters.

Thanks,

Junius

[Danni Moiseyev]

AOF writes every command but doesn't necessarily calls fsync on the AOF files,

without having fsync when you snapshot the volume you can be positive which commands were sync to the actual volume and which are not, you can configure different fsync configurations.

You can read about it here: https://redis.io/topics/persistence

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

[Junius Luo]

Thanks Danni! yes, we use the default fsync configuration, which is fsync every 1 second. 1 second would be fine for the snapshot.

Thanks,

Junius