API: get datafile with DOI

[Ana Trisovic]

Hi there,

I have a quick question, I am trying to get a datafile using DOI as persistentId

Here is the code that I am trying to run (at the line: "Query the dataverse API for all the files"):

https://github.com/atrisovic/dataverse-repro-study/blob/master/experimental_code/api-work-flow.ipynb

I saw this query at (Basic file access): http://guides.dataverse.org/en/latest/api/dataaccess.html but it doesn't seem to work.

Is there something obvious here that I am missing? :-S

Thank you!

Ana

[Philip Durbin]

Hi Ana,

If you go to https://dataverse.harvard.edu/file.xhtml?persistentId=doi:10.7910/DVN/TJCLKP/JKTORG&version=2.0 for example you will see under " Download URL" the following URL (screenshot attached):

 https://dataverse.harvard.edu/api/access/datafile/:persistentId?persistentId=doi:10.7910/DVN/TJCLKP/JKTORG

Does that help?

Thanks,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/d13c30f9-550e-4d2a-871b-2047bd5b72d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[paul.da...@gmail.com]

I'm getting a similar error when I try to get a datafile using the doi. I see:

{"status":"ERROR","code":404,"message":"'/api/v1/access/datafile/' endpoint does not exist on this server. Please check your code for typos, or consult our API guide at http://guides.dataverse.org."}
 
I'm on Dataverse V 4.10.1 build 1178-1a12cda and my call is to localhost:8080/api/access/datafile/:persistentId/?persistentId=doi:10.5072/FK2/JSE79W/330XWG.

I've also done the same request but with the DOI "/" replaced with "%2F" and had the same error message returned,

Thanks,
Paul

[Sherry Lake]

Phil,

Which version of Dataverse did the "Download URL" section for a file show up?

UVa is at 4.9.4 (soon to upgrade - trying hard to keep up with Harvard ;-) )

Looking forward to new features!!

--

Sherry

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8HcZqvmkvRXsHeSbwKpRCpSWfowC%2BU%2BnL%3DXVWKuD%2B-BGw%40mail.gmail.com.

[Ana Trisovic]

Thank you!

This actually seems to work, but I would like to access a list of files for each dataverse DOI.

For example:

files = requests.get(

        "https://dataverse.harvard.edu/api/datasets/:persistentId",
        params= {"persistentId": "doi:10.7910/DVN/28763", "key": dataverse_key})

so I can access the list of files with:

files.json()['data']['latestVersion']['files']

(the query for now returns: {u'message': u'Bad api key ', u'status': u'ERROR'})

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CADL9p-UkAdA1N8wZhLsQf1WYs-Uvxi2qx-ikbH3wJBqaJiBGeA%40mail.gmail.com.

[paul.da...@gmail.com]

Ok, so I think my issue might be something to do with testing the URLs using Postman. If I put the URL directly into the browser, it immediately pops up the Save As window, so clearly it is accessing the file correctly that way. Not sure why Postman is failing, but it seems like it is an issue on my end rather than Dataverse's.

Sorry for the worry,

Paul

On Wednesday, April 3, 2019 at 2:27:58 PM UTC-7, paul.d...@gmail.com wrote

[Philip Durbin]

If you go to https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/TJCLKP for example and click the "Metadata" tab and then "Export Metadata" and then "JSON" you should get to https://dataverse.harvard.edu/api/datasets/export?exporter=dataverse_json&persistentId=doi%3A10.7910/DVN/TJCLKP which will give you a list of file database IDs (other formats like "Schema.org JSON-LD" have them as well). When you only have the database id of a file and want to download it, you can download the file from https://dataverse.harvard.edu/api/access/datafile/3371438 for example.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CA%2B4nB98A1t_VG%3DgA4a0B6uCAr%2BZgUfcAtOs%3DmC14kJPxtf3gAA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Ana Trisovic]

This actually worked! Thank you very much!

(Here is how I modified the jupyter code: https://github.com/atrisovic/dataverse-repro-study/blob/master/experimental_code/api-work-flow.ipynb)

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8E%2BTRYchfWkeixRt2ipwK4dbKRsBEDKY1%2B%2B9AoBrg5t_A%40mail.gmail.com.

[Philip Durbin]

Hi Sherry,

Funny thing. "Download URL" has been around since 4.6.1 (see https://github.com/IQSS/dataverse/issues/3584 ) but clicking around your installation's file landing pages (e.g. https://dataverse.lib.virginia.edu/file.xhtml?fileId=1701 ) I don't see "Download URL" anywhere.

There are some rules about when to show or hide the Download URL for a file (depends on restricted, license, guestbook, etc. and I'm not sure if they are documented but here's the code, with comments, for 4.9.4: https://github.com/IQSS/dataverse/blob/v4.9.4/src/main/java/edu/harvard/iq/dataverse/util/FileUtil.java#L1198 ) but from a quick glance it seems like Download URL should be appearing for that "1701" file above. I'm not sure why it isn't. I guess you could upgrade to 4.12 or whatever and see if there's a difference but I don't think we've messed with this logic lately. Maybe someone out there can spot the reason. Please feel free to open a GitHub issue to at least document the rules about when "Download URL" is expected to appear.

Thanks,

Phil

p.s. Paul and Ana, if you're reading this, I'm glad you're both all set.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CADL9p-UkAdA1N8wZhLsQf1WYs-Uvxi2qx-ikbH3wJBqaJiBGeA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[danny...@g.harvard.edu]

Sherry/Phil, 

I think the version UVA is running has this bug, which doesn't provide the URL for Files without PIDs:

https://github.com/IQSS/dataverse/issues/5502

This is fixed as of 4.11, should it should be all set after an upgrade.

- Danny

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/d13c30f9-550e-4d2a-871b-2047bd5b72d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8HcZqvmkvRXsHeSbwKpRCpSWfowC%2BU%2BnL%3DXVWKuD%2B-BGw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CADL9p-UkAdA1N8wZhLsQf1WYs-Uvxi2qx-ikbH3wJBqaJiBGeA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Sherry Lake]

YES!!

Was just writing this hypothesis up. I noticed UNC, TDL and Scholar's Portal (all at 4.9.2 or 4.10) didn't have the download URL either and neither of them had file DOIs.

--

Sherry

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/d13c30f9-550e-4d2a-871b-2047bd5b72d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8HcZqvmkvRXsHeSbwKpRCpSWfowC%2BU%2BnL%3DXVWKuD%2B-BGw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CADL9p-UkAdA1N8wZhLsQf1WYs-Uvxi2qx-ikbH3wJBqaJiBGeA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

--

You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/3c413735-145e-48f9-8c07-df0e77aad7db%40googlegroups.com.