MongoDB initial sync slow

[Diego Woitasen]

Hi,

 I'm setting up a MongoDB replica set, with members on prem and on AWS. I'm testing future migration to AWS. DB size is350 GB. Last Friday, I've setup the member that's running on AWS as part of the replica set. The initial sync ran 2 days and only 60/70 GB were copied. I've tried rsync today and It took 4 hours to move all the DB. 

 Is there an explanation about that? I'm wondering why is there such big difference. 

I'm going for the rsync option for migration now, but I'd like to understand why is so slow.

Regards,

   Diego

[Tyler Brock]

Rsync essentially moves the bits from one place to another, its a network copy of the data. The data is not processed or interpreted, just shuttled through the network and written to disk.

MongoDB's replication mechanism inserts and validates the data, creates and updates indexes, and applies the operations that have occurred since the sync started to catch up. It's doing a lot more work.

You can think of it like getting the notes for a class you missed from friend. You could go to the xerox machine and produce a copy or you can open a fresh pad of paper and start transcribing (reading and understanding what he wrote, and writing down your own copy in your own handwriting).

[Diego Woitasen]

Yes, I understand that... but the different is incredible. Rsync 50x faster...

Other thing that I've noticed with rsync: it always copy all the files each time I run it. Looks like something in WiredTiger file layout breaks the rsync deltas transfer feature.