moving from default S3 bucket to direct-upload

[Jamie Jamison]

I don't know if this is do-able.  One of our depositors created their account before direct-upload was an option.  Currently I think that direct-upload would be a better option.  Their dataverse contains telescope data.  The original 20 files are ~1.0 gb but the files they would like to add are 7-8 gb.   

Is it possible to move from a regular s3 bucket to one setup for direct upload?

Thank you,

Jamie Jamison

UCLA Dataverse

[James Myers]

Yes. Direct upload requires correct configuration of Dataverse and the buckets CORS settings but it doesn’t involve changes to the bucket itself or moving data, etc. https://guides.dataverse.org/en/latest/developers/big-data-support.html#s3-direct-upload-and-download in the guides provides information on how to make both changes.

 

FWIW: Just turning on direct upload for your S3 store may be all you need. However, if there is any reason to keep ’normal’ upload for some users, it is also possible to configure two (or more) stores in Dataverse that use the same bucket – one with direct upload enabled and the other without it. There is a UI option to configure a collection so that all new datasets will use the specified store and an API call to switch which store a dataset uses for future file uploads. Multiple stores can also be used if you want to limit upload sizes to different values for different groups (or ingest size limits, etc.), i.e. use one store with a 1 GB limit and one with a 10GB limit and assign them to the relevant collections/datasets. In all cases, if the stores reference the same bucket, the files end up in the same bucket in the same place.

 

- Jim

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/412ccec5-1081-4f96-8c71-2f1719fa75d7n%40googlegroups.com.