Big data - what sizes are we talking about?

Skip to first unread message

Janet McDougall - Australian Data Archive

Jan 11, 2021, 12:48:02 AM1/11/21
to Dataverse Users Community
Hi All, and Happy New Year.

I am investigating where the community is at with big data.  Fantastically I was able to refer to the Installation Personas Data ( to see who is also interested in big data.

The user guides refer to DCM (as an experimental component) for big data via RSYNC over SSH, but I haven't been able to find reference to data file sizes by installations who have implemented DCM as a solution.

I am interested in whether anyone is uploading/downloading sizes of several TB?


Philip Durbin

Jan 25, 2021, 11:42:40 AM1/25/21
Hi Janet,

Happy New Year to you too.

I'm not sure what file sizes have been ultimately used with DCM but during development we talked about collections of files that were in the neighborhood of 100 GB.

I hope this helps,


You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To view this discussion on the web visit

Reply all
Reply to author
0 new messages