uploading large files

[Data CUHK]

Hello all,

Do you have experience uploading large files to Dataverse? Our users are having issues with uploading large files (15GB- 30GB). They tried to upload one file at a time, with each file sized at least 15GB, and the interface just timed out on them. It doesn't sound like they can break down the files further and upload smaller size files instead. 

Is there an API we could use to assist with the large file upload? 

Thanks!

Qinqin

[Philip Durbin]

Hi Qinqin,

Yes, for large file uploads, the API is recommended. Even better might be to use a tool like DVUploader, which we mention in the guides: https://guides.dataverse.org/en/6.7.1/user/dataset-management.html#command-line-dvuploader

There's even a new video from TDL about how to use it: https://www.youtube.com/watch?v=7qINkx38mNg

There's also a Python equivalent: https://github.com/gdcc/python-dvuploader

I hope this helps!

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dataverse-community/3d93f06d-1cfc-44ad-afdd-76c0fba5569fn%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin