Back strategy in a Master/Slave redis setup

[Howard]

Currently I have a two nodes Redis master/slave, both using AOF and without snapshot enabled.

According to the doc:  http://redis.io/topics/persistence

Redis is very data backup friendly since you can copy RDB files while the database is running: the RDB is never modified once produced, and while it gets produced it uses a temporary name and is renamed into its final destination atomically using rename(2) only when the new snapshot is complete.

1. Assume I am using  snapshoting, the doc said each snapshots will create a new DB, so if I have large memory, e.g. 10GB, if I set a very frequent snapshot interval, the file IO during snapshot will be very high?

2. Now as I am using AOF, is it also behave the same way? I don't need to shutdown Redis and just copy on a live DB?

Thanks.

 

[Josiah Carlson]

On Tue, Nov 6, 2012 at 5:50 AM, Howard <howa...@gmail.com> wrote:
> Currently I have a two nodes Redis master/slave, both using AOF and without
> snapshot enabled.
>
> According to the doc: http://redis.io/topics/persistence
>
>> Redis is very data backup friendly since you can copy RDB files while the
>> database is running: the RDB is never modified once produced, and while it
>> gets produced it uses a temporary name and is renamed into its final
>> destination atomically using rename(2) only when the new snapshot is
>> complete.
>
> 1. Assume I am using snapshoting, the doc said each snapshots will create a
> new DB, so if I have large memory, e.g. 10GB, if I set a very frequent
> snapshot interval, the file IO during snapshot will be very high?

Yes, it will be high, regardless of snapshot interval. It writes *all*
of your data to disk. So if you tell it to do it every minute, you
will writing it to disk every minute.

> 2. Now as I am using AOF, is it also behave the same way? I don't need to
> shutdown Redis and just copy on a live DB?

It behaves the same way when rewriting. You don't need to shut down to
get the .rdb, and technically you can even copy an .aof during writes
(though the last writes may be partial, but you can fix it with the
check aof binary that is created when you compile Redis).

Regards,
- Josiah

>
> Thanks.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/redis-db/-/BOAPOwRen58J.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.

[Howard]

Hi again,

On Wednesday, November 7, 2012 12:52:51 AM UTC+8, Josiah Carlson wrote:

> 1. Assume I am using  snapshoting, the doc said each snapshots will create a
> new DB, so if I have large memory, e.g. 10GB, if I set a very frequent
> snapshot interval, the file IO during snapshot will be very high?

Yes, it will be high, regardless of snapshot interval. It writes *all*
of your data to disk. So if you tell it to do it every minute, you
will writing it to disk every minute.

So assume I set to sync my 10GB data to disk every time, I will need to copy 10GB data to a new file and that is risky since the disk might not be able to finish with the 1min interval?

 

(though the last writes may be partial, but you can fix it with the
check aof binary that is created when you compile Redis).

Why there is partial write? I assume redis will write to a new file and rename is atomic?

Thanks.

 

[Josiah Carlson]

On Tue, Nov 6, 2012 at 7:28 PM, Howard <howa...@gmail.com> wrote:
> On Wednesday, November 7, 2012 12:52:51 AM UTC+8, Josiah Carlson wrote:
>>
>> > 1. Assume I am using snapshoting, the doc said each snapshots will
>> > create a
>> > new DB, so if I have large memory, e.g. 10GB, if I set a very frequent
>> > snapshot interval, the file IO during snapshot will be very high?
>>
>> Yes, it will be high, regardless of snapshot interval. It writes *all*
>> of your data to disk. So if you tell it to do it every minute, you
>> will writing it to disk every minute.
>>
> So assume I set to sync my 10GB data to disk every time, I will need to copy
> 10GB data to a new file and that is risky since the disk might not be able
> to finish with the 1min interval?

Redis will not start a new sync until the last one is done. So if you
tell it to sync every minute, if it can only sync every minute and a
half, it will.

Depending on your data, if you have 10 gigs of memory being used, you
may not have a 10 gigabyte dump. In my experience, I expect on-disk
dumps to be 1/20-1/5 the size of the in-memory data. So for 10 gigs in
memory, I would expect 500 megs to 2 gigabytes on disk. At 2
gigabytes, that's a 33m/second write to disk. Pretty reasonable if you
have your own hardware, perhaps a bit much if you are in the cloud
somewhere.

Also note that when you have a slave, the initial sync performs a
snapshot, sends it to the slave, then streams write commands.

>> (though the last writes may be partial, but you can fix it with the
>> check aof binary that is created when you compile Redis).
>
> Why there is partial write? I assume redis will write to a new file and
> rename is atomic?

If you are writing to an AOF, and you want to backup the AOF, you have
to remember that Redis is writing data to the AOF continuously. To
backup the AOF, you would need to start copying it. But when you read
those last bytes, the copy process may not get all of the bytes that
Redis has written (Redis writes to the file, but syncs every second,
and can sync less often if other high-IO operations are going on). So
you could end up with a partial AOF.

This isn't an issue with an rdb, because it is point-in-time, as long
as you wait for the dump to complete (and not read the temporary
file).

Regards,
- Josiah

[Howard]

Hi,

On Wednesday, November 7, 2012 12:43:59 PM UTC+8, Josiah Carlson wrote:

those last bytes, the copy process may not get all of the bytes that 
Redis has written (Redis writes to the file, but syncs every second,
and can sync less often if other high-IO operations are going on). So
you could end up with a partial AOF.

Thanks for your detail reply.

So do you think it is reasonable to in a Master/Slave setup, I can also enable snapshot in my slave for when performing backup, I use the rdb instead of copying the AOF?

Or in fact maybe I should only use rdb instead of aof in my slave?

Thanks.

 

[Felix Gallo]

For most use cases, I recommend using AOF if you have the disk space for it, because it shrinks the window of opportunity for unwritten data down to about one second.  

The particular good thing about rdb is that it's a true snapshot, so if your data has points of true consistency that you would like to maintain (e.g., end-of-day processing is done in a financial services application and you wish to maintain a pricing/duration record for your security master), then it's pretty easy to just fire off an rdb copy, save that, and then you have good flexibility in returning to that state whenever you need.

The better thing about AOF is that it responds well under load, especially heavy write load, and especially heavy write load where you would really like to lose at most a second or two of data.  Salvatore wrote an excellent description of how this all works and what the tradeoffs are:

http://oldblog.antirez.com/post/redis-persistence-demystified.html

F.

--

You received this message because you are subscribed to the Google Groups "Redis DB" group.

To view this discussion on the web visit https://groups.google.com/d/msg/redis-db/-/iJvgSMQ2Z4kJ.

[Howard]

Hi,

On Thursday, November 8, 2012 12:24:12 AM UTC+8, Felix wrote:

For most use cases, I recommend using AOF if you have the disk space for it, because it shrinks the window of opportunity for unwritten data down to about one second.  

Because we are already using replication, so seems duplicated the AOF is not a must..