Big data - what sizes are we talking about?

[Janet McDougall - Australian Data Archive]

Hi All, and Happy New Year.

I am investigating where the community is at with big data.  Fantastically I was able to refer to the Installation Personas Data (https://docs.google.com/spreadsheets/d/1YCCf22h_27Gr0qp2XOkIp36cGgy_mohn7DCmn9Xkm9Q/edit#gid=0) to see who is also interested in big data.

The user guides refer to DCM (as an experimental component) for big data via RSYNC over SSH, but I haven't been able to find reference to data file sizes by installations who have implemented DCM as a solution.

I am interested in whether anyone is uploading/downloading sizes of several TB?

Thanks

Janet

[Philip Durbin]

Hi Janet,

Happy New Year to you too.

I'm not sure what file sizes have been ultimately used with DCM but during development we talked about collections of files that were in the neighborhood of 100 GB.

I hope this helps,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/2d2658da-d3ed-4608-8813-6fb37d768f00n%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin