API: get datafile with DOI

61 views
Skip to first unread message

Ana Trisovic

unread,
Apr 3, 2019, 5:01:18 PM4/3/19
to Dataverse Users Community
Hi there,

I have a quick question, I am trying to get a datafile using DOI as persistentId

Here is the code that I am trying to run (at the line: "Query the dataverse API for all the files"):


I saw this query at (Basic file access): http://guides.dataverse.org/en/latest/api/dataaccess.html but it doesn't seem to work.

Is there something obvious here that I am missing? :-S

Thank you!
Ana




Philip Durbin

unread,
Apr 3, 2019, 5:22:40 PM4/3/19
to dataverse...@googlegroups.com
Hi Ana,

If you go to https://dataverse.harvard.edu/file.xhtml?persistentId=doi:10.7910/DVN/TJCLKP/JKTORG&version=2.0 for example you will see under " Download URL" the following URL (screenshot attached):


Does that help?

Thanks,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/d13c30f9-550e-4d2a-871b-2047bd5b72d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Screen Shot 2019-04-03 at 5.20.58 PM.png

paul.da...@gmail.com

unread,
Apr 3, 2019, 5:27:58 PM4/3/19
to Dataverse Users Community
I'm getting a similar error when I try to get a datafile using the doi. I see:
{"status":"ERROR","code":404,"message":"'/api/v1/access/datafile/' endpoint does not exist on this server. Please check your code for typos, or consult our API guide at http://guides.dataverse.org."}

I'm on Dataverse V 4.10.1 build 1178-1a12cda and my call is to localhost:8080/api/access/datafile/:persistentId/?persistentId=doi:10.5072/FK2/JSE79W/330XWG.

I've also done the same request but with the DOI "/" replaced with "%2F" and had the same error message returned,

Thanks,
Paul

Sherry Lake

unread,
Apr 3, 2019, 5:33:05 PM4/3/19
to dataverse...@googlegroups.com
Phil,

Which version of Dataverse did the "Download URL" section for a file show up?

UVa is at 4.9.4 (soon to upgrade - trying hard to keep up with Harvard ;-) )

Looking forward to new features!!

--
Sherry

Ana Trisovic

unread,
Apr 3, 2019, 5:48:13 PM4/3/19
to dataverse...@googlegroups.com
Thank you!

This actually seems to work, but I would like to access a list of files for each dataverse DOI.

For example:

files = requests.get(
        "https://dataverse.harvard.edu/api/datasets/:persistentId",
        params= {"persistentId": "doi:10.7910/DVN/28763", "key": dataverse_key})

so I can access the list of files with:

files.json()['data']['latestVersion']['files']

(the query for now returns: {u'message': u'Bad api key ', u'status': u'ERROR'})

paul.da...@gmail.com

unread,
Apr 3, 2019, 5:53:15 PM4/3/19
to Dataverse Users Community
Ok, so I think my issue might be something to do with testing the URLs using Postman. If I put the URL directly into the browser, it immediately pops up the Save As window, so clearly it is accessing the file correctly that way. Not sure why Postman is failing, but it seems like it is an issue on my end rather than Dataverse's.

Sorry for the worry,
Paul

On Wednesday, April 3, 2019 at 2:27:58 PM UTC-7, paul.d...@gmail.com wrote

Philip Durbin

unread,
Apr 3, 2019, 5:55:26 PM4/3/19
to dataverse...@googlegroups.com
If you go to https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/TJCLKP for example and click the "Metadata" tab and then "Export Metadata" and then "JSON" you should get to https://dataverse.harvard.edu/api/datasets/export?exporter=dataverse_json&persistentId=doi%3A10.7910/DVN/TJCLKP which will give you a list of file database IDs (other formats like "Schema.org JSON-LD" have them as well). When you only have the database id of a file and want to download it, you can download the file from https://dataverse.harvard.edu/api/access/datafile/3371438 for example.


For more options, visit https://groups.google.com/d/optout.

Ana Trisovic

unread,
Apr 3, 2019, 6:40:40 PM4/3/19
to dataverse...@googlegroups.com

Philip Durbin

unread,
Apr 3, 2019, 7:33:00 PM4/3/19
to dataverse...@googlegroups.com
Hi Sherry,

Funny thing. "Download URL" has been around since 4.6.1 (see https://github.com/IQSS/dataverse/issues/3584 ) but clicking around your installation's file landing pages (e.g. https://dataverse.lib.virginia.edu/file.xhtml?fileId=1701 ) I don't see "Download URL" anywhere.

There are some rules about when to show or hide the Download URL for a file (depends on restricted, license, guestbook, etc. and I'm not sure if they are documented but here's the code, with comments, for 4.9.4: https://github.com/IQSS/dataverse/blob/v4.9.4/src/main/java/edu/harvard/iq/dataverse/util/FileUtil.java#L1198 ) but from a quick glance it seems like Download URL should be appearing for that "1701" file above. I'm not sure why it isn't. I guess you could upgrade to 4.12 or whatever and see if there's a difference but I don't think we've messed with this logic lately. Maybe someone out there can spot the reason. Please feel free to open a GitHub issue to at least document the rules about when "Download URL" is expected to appear.

Thanks,

Phil

p.s. Paul and Ana, if you're reading this, I'm glad you're both all set.




For more options, visit https://groups.google.com/d/optout.

danny...@g.harvard.edu

unread,
Apr 4, 2019, 10:02:02 AM4/4/19
to Dataverse Users Community
Sherry/Phil, 

I think the version UVA is running has this bug, which doesn't provide the URL for Files without PIDs:


This is fixed as of 4.11, should it should be all set after an upgrade.

- Danny
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.


--

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.

Sherry Lake

unread,
Apr 4, 2019, 10:16:52 AM4/4/19
to dataverse...@googlegroups.com
YES!!

Was just writing this hypothesis up. I noticed UNC, TDL and Scholar's Portal (all at 4.9.2 or 4.10) didn't have the download URL either and neither of them had file DOIs.

--
Sherry



To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.


--

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/3c413735-145e-48f9-8c07-df0e77aad7db%40googlegroups.com.
Reply all
Reply to author
Forward
Message has been deleted
0 new messages