Disable unpacking zipfiles automatically

12 views
Skip to first unread message

Gabriel Gerez

unread,
Nov 3, 2022, 4:39:49 AM11/3/22
to Dataverse Users Community
So I have a failry big dataset, and therefore I organized it into zipped files. These files are meant to be opened in proper high performance clusters. 
Additionally, it can take very long for the files to be unpacked in the dataverse upload system, slowing data upload to a halt.
My question, is there any way to prevent dataverse from automatically unpacking the zipped files?


Don Sizemore

unread,
Nov 3, 2022, 7:18:55 AM11/3/22
to dataverse...@googlegroups.com
Hello,

One common work-around for (smaller) zip files is to double-zip each uploaded object, which results in the original zip file being deposited, but may not work at your scale.
If you use an S3 or compatible datastore, Dataverse won't attempt to unzip on upload.
I'm not aware of a better way, but I hope someone else in the community has better suggestions.

Thank you,
Don

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/7d2dff95-5e47-4c8d-95aa-ffd3599759e3n%40googlegroups.com.

Gabriel Gerez

unread,
Nov 10, 2022, 4:08:09 AM11/10/22
to dataverse...@googlegroups.com
double-ziping seems to work.
Thanks for the swift answer 

Gabriel

You received this message because you are subscribed to a topic in the Google Groups "Dataverse Users Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dataverse-community/AfVrUK3H0qk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAPfMOazZFCf8Xo6uEy0v6JFAQisbkWQ_p%3D3gQcLq0iMJuySrDA%40mail.gmail.com.

Philip Durbin

unread,
Nov 23, 2022, 12:00:33 PM11/23/22
to dataverse...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages