DVUploader Errors

[Sherry Lake]

Hello,

A researcher used DVUploader on 3 files and got the same error message each time. 

First file is the dvuploader command and ensuing error message (just for one of the files, all files failed with the same error):

Second file is the section for this job from the server logs:

So......

What happened?

FYI:
MaxFileUploadSizeInBytes.  = 107374182400
This file = 15616921658 bytes

Where are the files? On S3?

Should we delete them and start over?
OR since they are on S3, how can we get the dataset to know them?

Thanks,

Sherry Lake

UVA Dataverse http://dataverse.lib.virginia.edu

[James Myers]

Sherry,

I don’t understand why it failed yet, but from the logs I expect the files did make it to s3. They should be in the bucket, in a dir corresponding to the dataset PID, using the storageidentifier numbers in the logs. You should be able to add them to the dataset using the direct upload api - https://guides.dataverse.org/en/latest/developers/s3-direct-upload-api.html#adding-the-uploaded-file-to-the-dataset. That’s essentially what the DVUploader does, so it is not clear to me why it failed, but using the API directly might give your more details about what went wrong or let you try variations on the jsonData to send.

 

-- Jim

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dataverse-community/4b866439-c167-47f8-8f92-8a48338f1112n%40googlegroups.com.

[Sherry Lake]

Thanks, Jim!!!

Phil thinks it failed because of incomplete metadata in the dataset. The dataset was created before the "bounding box" metadata field was fixed, so once we upgraded to that DV version, this dataset had "incomplete metadata". He said - in Zulip:

https://dataverse.zulipchat.com/#narrow/channel/378866-troubleshooting/topic/dvuploader.20Errors

... it often means that values were saved in the database and then later the rules were made more restrictive.

But thanks to your advice, I was able to use the "add file" command to get the missing files to show up in the dataset. They were already in the correct S3 bucket.

https://doi.org/10.18130/V3/ESQBIY

Thanks to DVUploader, all the JSON parameters needed (filename, checksum, S3 bucket path, filesize) were in our captured output. I love the "script" command, as long as I remember to start it.

--

Sherry