Sync by Copying Data Files from Another Member

[Darshan Shah]

Hi,

For a replicaset secondary member, I want to do an Initial Sync by Copying Data Files from Primary Member using scp/rsync.

The data-set is huge and I can shut down the replicaset to ensure consistency so that is not a problem

My question is apart from the local directory and the directory for each database (since we are using directory per DB option), which other files/directories are required to be copied for this sync? Here are the other files & directories that I see in the data directory for the Primary member:

1. diagnostic.data  (dir)
   
2. journal  (dir)
   
3. _mdb_catalog.wt  (file)
   
4. sizeStorer.wt  (file)
   
5. storage.bson  (file)
   
6. testdb  (dir)
   
7. WiredTiger  (file)
   
8. WiredTigerLAS.wt  (file)
   
9. WiredTiger.lock  (file)
   
10. WiredTiger.turtle  (file)
    
11. WiredTiger.wt  (file)

Thanks!

 

[Lungang Fang]

hi Darshan,

You can resync a new member of a replica set by copying the data files. For more details, please see Add Members to the Replica Set and Copy the Data Files.

There are several other things worth noting:

1. You must shut down MongoDB processes, since copying data files from a running MongoDB instance can have unexpected consequences.
2. The files you list are WiredTiger metadata files. They are required for the correct operation of the WiredTiger storage engine.
3. Before you do any major changes to your deployment, make sure that the data is backed up and all procedures thoroughly tested.

Regards,
Lungang

​

[Darshan Shah]

Hi Lungang,

My specific question is which of the listed directory and/or file can be skipped and which of them are mandatory for the manual copy?

I have read that link already and it just mentions that "local" is mandatory and my question build on top of that to further drill down into which other files/directories are mandatory and which can be skipped.

If you can figure that out and let me know - that would be great.

Thanks,

Darshan.

[Lungang Fang]

Hi Darshan,

I have read that link already and it just mentions that “local” is mandatory and my question build on top of that to further drill down into which other files/directories are mandatory and which can be skipped.

Actually, one should copy the whole directory (including files and subdirectories) for all and every database. Sorry that the document misled you and thank you for pointing that out. I am going to file a document bug to fix this issue.

Regards,
Lungang

​

[Darshan Shah]

Hi Lungang,

After some trial & error, I see that if I include the "journal" folder and "WiredTiger.lock" file, it sometimes causes problems.

Would you be aware of any such issues?

Thanks,

Darshan.