rsync is slow in in copying huge folder with about 1996002 sub folders

[Phanindra Srungavarapu]

Folder structure is like 000/001/001 to 002/450/999

Total estimate of folders is roughly about 1996002 each folder containing about 5 image files almost all images are less than 50K.

I'm using rsync to copy images from an external server to google could server in the process of migrating our data to google servers. We are moving all our servers to google.
Network latency from our server location to our cloud computing engines is about 150ms. Rsync has been running for about 2 weeks, I'm running rsync with  -azv --progress is there anyway I can improve the speed of the process??

[George (Google Cloud Support)]

Hello Phanindra,

If I correctly understood, you are copying your data to a GCE VM? if so, the speed depends on the Disk type and size and the instance type you are using. 

You can try copying your data to Google Cloud Storage using the gsutil command with the -m flag as follows, to perform parallel synchronization:

gsutil -m rsync -d -r YOUR_DATA gs://BUCKET_NAME/data

More information about this matter can be found in the following Help Center articles [1] [2].

I hope this helps.

Sincerely,

George