Fail Uploading Files

[Remmy Zen]

Hi,

I tried to upload a file to my dataset with the HTTP via browser:

It shows the uploading process:

However, after the uploading is finished, it goes back to this page without any ingestion:

I tried to upload the file using API to no avail as well:

curl -H X-Dataverse-key:$API_TOKEN -X POST -F "file=@$FILENAME" -F 'jsonData={"description":"My description.","directoryLabel":"data/subdir1","categories":["Data"], "restrict":"false"}' "$SERVER_URL/api/datasets/:persistentId/add?persistentId=$PERSISTENT_ID"

This outputs:

Temporary Redirect

Previously, I can upload files. However, now all my uploaded CSV files show Upload Completed with Errors.

Could anyone advise me on how to fix this issue?

Your help is appreciated. Thank you.

Regards,

Remmy

[Sherry Lake]

Remmy,

Can you send a copy of one of the csv files? The error is on ingest when Dataverse is trying to ingest and create a tabular file from the CSV file. If you hover over the "?" question mark on the yellow error message, it will tell you "why" there was a problem - usually because the CSV file is not formatted correctly.

Even with the ingest error, the original CSV file can still be downloaded. 

CSV, XLSX and some other files are "ingested", non-tabular files are not ingested - one of your attached images is a .jpg file and not ingested.

https://guides.dataverse.org/en/latest/user/tabulardataingest/csv-tsv.html

--

Sherry Lake

University of Virginia Dataverse

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/c91732bc-dcdb-45ec-bf11-9e1c1c4ad43dn%40googlegroups.com.

[Philip Durbin]

Hi Remmy,

In addition to Sherry's suggestions, you might want to try uploading your files on https://demo.dataverse.org . Do you get the same behavior? It's especially strange that the files seem to disappear after upload (thanks for the screenshots). Does the Javascript console of your browser reveal anything? Are you running your own installation of Dataverse? If so, what version, please? Anything interesting in server.log?

Thanks,

Phil

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CADL9p-XDv%3D00nHGNuEHLOLm_ySJzgMibUwkEHDS1Wuucb59eVQ%40mail.gmail.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[Remmy Zen]

Hi Sherry and Phil,

The CSV files that are shown in the screenshot are old files, which were uploaded fine before. Now, it is showing an error.

It runs fine in the demo dataverse: https://demo.dataverse.org/dataset.xhtml?persistentId=doi%3A10.70122%2FFK2%2FUHCEEP

Does the Javascript console of your browser reveal anything? 

There is nothing on the console.

Are you running your own installation of Dataverse? If so, what version, please? 

Yes, I am running my own installation of Dataverse. I am using version 5.5 and I am using docker. I have tried restarting docker, reindexing, and so on, but it still does not work.

Anything interesting in server.log?

It just shows: 

dataverse        |   Failed to process and/or save the file 2021-01-01-00-00-14.csv; /data/temp/tmp12315414234797264015upload|#] 

Thank you.

Regards,

Remmy

You received this message because you are subscribed to a topic in the Google Groups "Dataverse Users Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dataverse-community/7pbPBOnix7M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8EzeC-WjLeRrY8jcSD9yq%2Bb_3LmwE4OCaLCHKA5ovpw5g%40mail.gmail.com.

--

--------------------------------------------
Remmy A. M. Zen

[Philip Durbin]

Hi Remmy,

I used the "Failed to process" error you mentioned as a starting point to dig through the code a bit: https://github.com/IQSS/dataverse/blob/v5.10.1/src/main/java/edu/harvard/iq/dataverse/EditDatafilesPage.java#L2001

It's not immediately obvious to me why you're seeing that error.

To be clear, it seems like you're having two problems. The error above is from you uploading a CSV file via API. As you say, ingest is failing on your server (though it works on the demo server).

The other problem you're having is not being able to upload files via the web interface. You showed uploading a JPG but it disappears before you can click "Done". As a terminology note, we use the term "ingest" to specifically mean processing tabular files (Excel, CSV, Stata, etc.) so we wouldn't say "without any ingestion" for a JPG. In the web interface, we often say that uploaded files are "staged" until "Done" is clicked. That is, staged files are not saved to their final location on the file system (or S3) or put into the database until you click "Done". So I'd say your second problem is that you are unable to stage files. They disappear before you can click "Done". No good, obviously.

Given these problems I'm wondering about you running 5.5 on Docker. First of all, dataverse-docker is community-supported so if you're having problems with it, it's best to open an issue at https://github.com/IQSS/dataverse-docker/issues

The best supported option is following the steps in the Installation Guide (running the installer, etc.): https://guides.dataverse.org/en/5.10.1/installation/

Lots of people are also having success with dataverse-ansible: https://guides.dataverse.org/en/5.10.1/installation/prep.html#advanced-installation

Either way, it would be good to get on a recent version rather than 5.5. Was your 5.5 installation working previously and only recently developed these problems? Is it possible to start over with a fresh, 5.10.1 installation?

Thanks,

Phil

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABLJSt0nePtoUmVK6UZ95b%3Dg7fZbv5RowjiD0_yro6aSRutj-g%40mail.gmail.com.