Restore Redis dabase from Dump.rdb / appendaof

[bala wagh]

Hi,

 

I have dumped the redis DB in dump.rdb file and now its size is (251KB) and appendaof size(7.2MB). But now i want to restore the same dump db in same server. Because before dump all auto complete keys are stored in Redis but after dumped now it is now showing. is that all keys are dumped in the DB file or all keys got expired. 

      But i Think all keys should be stord in appendaof file so i want to restore back. is it possible to restore on the same Server...???

Because now DBSIZE is only 220 Interger But when i am doing BGREWRITEAOF it is not restoring back, so any body tell me the procedure to restore the DB on  DEV Environment.

Cheers

Bala

[Josiah Carlson]

I don't know the commands that were used to get you to your current situation, but I can tell you the following:

1. BGSAVE/BGREWRITEAOF produce a snapshot (rdb)/AOF (respectively) with all non-expired data

2. Starting Redis with a proper snapshot and/or aof configuration will only get Redis to the state that it was before the snapshot occurred, and/or after the last write that made it into the AOF (minus already expired keys)

3. BGSAVE/SAVE/BGREWRITEAOF do produce snapshots/command streams that describe the data in-memory in the Redis server, but it is generally *non destructive* (doesn't delete any keys or data, except sometimes expired data, which you can't access after expiration anyway)

Long story short: if you are missing data in Redis, it's probably the result of one or more of the following:

* Setting expiration times on your Redis keys

* You didn't snapshot before restarting Redis

* Someone called flushdb or flushall

* Someone deleted your data

 - Josiah

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[bala wagh]

Thanks Josiah

I have tried to do BGSAVE/SAVE/BGREWRITEAOF after copied dump.rdb file in new redis server "/var/lib/redis/6379/" , But after applying these command the appendaof does "0" size and existing data will flush out.

i have done the following steps.

1. 1. bgsave on existing server 

2. stop redis-server and set appenonly no

3. take backup of dump.rdb file

4. move dump.rdb on new server and copy in "/var/lib/redis/6379/" location

5.reboot server

6. bgrewriteaof

no existing data on new server..!!!

but i have dump.rdb and appenonly.aof files in backup.

existing server 2.8.14  on ubuntu 14.04 LTS and new server 2.8.17 on CentOS 6.5

Is the compatibility issue..???? OR i did some wrong steps while data backup...???

please clarify me so i will prepare myself for next time.

Thanks.

[Josiah Carlson]

On Tue, Sep 23, 2014 at 11:59 PM, bala wagh <bala...@gmail.com> wrote:

Thanks Josiah

I have tried to do BGSAVE/SAVE/BGREWRITEAOF after copied dump.rdb file in new redis server "/var/lib/redis/6379/" , But after applying these command the appendaof does "0" size and existing data will flush out.

i have done the following steps.

You need to be more explicit in what you are explaining, because I can interpret several items in your list of steps in more than one way.

 

1. 1. bgsave on existing server 

2. stop redis-server and set appenonly no

Stop which server? The old server, or the new server?

 

3. take backup of dump.rdb file

4. move dump.rdb on new server and copy in "/var/lib/redis/6379/" location

5.reboot server

Are you rebooting the entire machine? Are you restarting a single process? Are you just restarting the Redis proces? Is this on the new server? The old server? Or are you *starting* a Redis server process?

 

6. bgrewriteaof

This is being done on the new server?

no existing data on new server..!!!

In one interpretation of what you have described, it would seem that on your new server, you are *restarting* the Redis process *after* copying the new dump.rdb into the snapshot path. If you *restarted* the new Redis server, then you will have 0 data because the shutdown process on the server overwrites any existing dump. You would instead need to do the following (instead of your current #4 and #5 above):

1. shut down new Redis server

2. replace the dump.rdb

3. start up the new Redis server

But here's the other part: if you have "appendonly yes" in your new server, but your append only file on the new server has no data when it starts up, then you will have no data in the new server. All together, if you are looking to move your data from your old server to your new server, you should perform the following steps:

1. Leave your old server running

2. In your old server, perform "CONFIG SET appendonly yes"

3. In your old server, perform "BGREWRITEAOF"

4. Once the background rewriting completes, execute "SHUTDOWN" in Redis on your old server

5. Make sure your new Redis server is not running

6. Make sure your new Redis server has the configuration "appendonly yes" in its configuration file

7. Copy the appendonly.aof to the new server to the right place

8. Remove the dump.rdb on the new server if it exists (the append only file overrides the dump.rdb)

9. Start Redis on your new server

10. Notice how the data is still on the new server

There are variations of this that don't require having your new Redis server stopped before working, but this should get you there in way similar to what you have been trying. A lot of people use Redis' built in replication for replacing servers, which is why this type of question is uncommon.

but i have dump.rdb and appenonly.aof files in backup.

existing server 2.8.14  on ubuntu 14.04 LTS and new server 2.8.17 on CentOS 6.5

Is the compatibility issue..???? OR i did some wrong steps while data backup...???

Wrong steps.

 - Josiah

[bala wagh]

Hi Josiah,

More briefing on my last questions

On Thursday, 25 September 2014 03:19:38 UTC+5:30, Josiah Carlson wrote:

On Tue, Sep 23, 2014 at 11:59 PM, bala wagh <bala...@gmail.com> wrote:

Thanks Josiah

I have tried to do BGSAVE/SAVE/BGREWRITEAOF after copied dump.rdb file in new redis server "/var/lib/redis/6379/" , But after applying these command the appendaof does "0" size and existing data will flush out.

i have done the following steps.

You need to be more explicit in what you are explaining, because I can interpret several items in your list of steps in more than one way.

 

1. 1. bgsave on existing server 

2. stop redis-server and set appenonly no

Stop which server? The old server, or the new server?

         --> Old Redis Server 

 

3. take backup of dump.rdb file

4. move dump.rdb on new server and copy in "/var/lib/redis/6379/" location

5.reboot server

Are you rebooting the entire machine? Are you restarting a single process? Are you just restarting the Redis proces? Is this on the new server? The old server? Or are you *starting* a Redis server process?

       --> yes i have rebooted entire server. No i haven't restarted single process of redis server after dumped. This one is New Server.  

 

6. bgrewriteaof

This is being done on the new server?

--> yes

no existing data on new server..!!!

In one interpretation of what you have described, it would seem that on your new server, you are *restarting* the Redis process *after* copying the new dump.rdb into the snapshot path. If you *restarted* the new Redis server, then you will have 0 data because the shutdown process on the server overwrites any existing dump. You would instead need to do the following (instead of your current #4 and #5 above):

1. shut down new Redis server

2. replace the dump.rdb

3. start up the new Redis server

--> Yes Josiah you are right , unfortunately i did this mistake. 

But here's the other part: if you have "appendonly yes" in your new server, but your append only file on the new server has no data when it starts up, then you will have no data in the new server. All together, if you are looking to move your data from your old server to your new server, you should perform the following steps:

 

--> Thats the appropriate steps i want, i missed out something but havent got what it was. thank you very much. It works :)