Upcoming talk on the Dataverse API

[Philip Durbin]

Hello! I'm giving a talk next week about the Dataverse API to an audience that is not deeply technical and programmer-y but that is curious about the Dataverse API. They want to know what you can do with it and how.

I haven't made any slides yet and I was thinking that perhaps I could create a new page or two in the API Guide along these lines:

- Frequently Asked Questions about the Dataverse API

- Getting Started with the Dataverse API

- Examples of Common Dataverse API Operations

My plan is to follow up on this thread with whatever I come up with: slides, some pages like the above, etc.

I'm writing to ask what sort of questions you have. If you were going to listen to me talk for an hour or so about the API, what would you want to know? What sort of extra content would you like to see in the API Guide?

Thanks!

Phil

p.s. Here's a handy link to the API Guide in its current form: http://guides.dataverse.org/en/4.15.1/api/

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[Sherry Lake]

Hi Phil,

This is very timely for me.

My biggest question is when to use SWORD API vs Native API. I haven't found a use for SWORD yet (and then when I did try it didn't work.... for another thread....)

Second: what are all the API endpoint commands (and their options)?

I would like to see more example "curl" commands using persistentIds AND corresponding $ID examples.

I am putting together a simple "How to" for one of my researchers. He wants to 1) create a dataset 2) add files and 3) publish a dataset. I"m also creating metadata templates (for dataset and files). I should be finished with it by the end of the week. Be glad to share.

One side note, is there any way to have the "DatasetPublishPopupCustomText" display BEFORE the API publishes (and have it pause wait for confirmation)?

Thanks,

Sherry Lake

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8EwaKJoJMaFtO%2BuCxC5KueM6BiTr94Z%3DfjUrX%2BB0VkJxQ%40mail.gmail.com.

[Dimitri Szabo]

Hello Philip,

I think that these additions are great ideas.

It is a bit out of topic here, but my first wish on the API Guide would be to add the undocumented features like DatasetThumbnail manipulation.

Another thing about the guide is that it is not always easy for new users to identify which id to use for id, pid etc.

Now, for presentation (and for a Getting Started guide), a little use case scenario with real data would be great, especially as API tend to be perceived as very abstract.

Thank you for the great work on the APIs,

Dimitri Szabo

[Philip Durbin]

Hi all, my "intro to Dataverse APIs" talk went well* and I just made a pull request to make some improvements to the API Guide based on it.

To preview the changes, please go to http://guides.dataverse.org/en/6086-api-guide/api/index.html

You are welcome to reply here with any feedback.

Everyone is also welcome to leave comments on the pull request or leave a review: https://github.com/IQSS/dataverse/pull/6107

For a little insight on what I was going for with this pull request, you can read my summary after the talk at https://github.com/IQSS/dataverse/issues/6086#issuecomment-521431188

Sherry, I did address "SWORD vs Native" and tried to add more and improved curl examples. If you'd like to share your metadata templates, please go ahead. I believe OJS does pause when showing the "DatasetPublishPopupCustomText" but this is something that each system that integrates with Dataverse would have to do.

Dimitri, I did not document the dataset thumbnail API but please please feel free to create an issue at https://github.com/IQSS/dataverse/issues for us to document it some day. I'm hoping that the update curl examples linked from the new "getting started" page help explain ID vs PID, etc. I'm not sure how much I emphasized real data but I'd like to point out that there's a new https://github.com/IQSS/dataverse-sample-data repo that I'm planning on announcing soon in a separate thread.

In summary, I hope these improvements help anyone who is interested in learning about Dataverse APIs. As I said in the talk, one of the biggest takeaways for me is that I was surprised that our support team for Harvard Dataverse is so interested in APIs. What's clear to me now is that there are multiple audiences for Dataverse APIs. It's not just for developers. In the rewritten "intro" page I try to explain the audiences as I see them and provide suggested starting points for if you're a researcher or curator or a superuser or a support team member or a sysadmin or whatever. Something for everyone, I hope.

Thanks and please keep the feedback coming!

Phil

* There were cookies: https://twitter.com/shlakeuva/status/1161315003342835712

On Wed, Aug 7, 2019 at 8:28 AM Philip Durbin <philip...@harvard.edu> wrote:

[Barbosa, Sonia]

Thanks, Phil! I look forward to reviewing the additions and updates!

ᐧ

--

You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8HeuePcZzq7zOgeh2z%3DWW7QtzC5aXUBeF%2BOTxBPa-jc6Q%40mail.gmail.com.

--

Sonia Barbosa
Manager of Data Curation, The Dataverse Project
Manager of the Murray Research Archive, IQSS
Data Science
Harvard University

All Harvard Dataverse Repository inquiries should be sent to:  sup...@dataverse.harvard.edu

All software inquiries should be sent to: sup...@dataverse.org

Need to deposit data? Visit http://dataverse.harvard.edu

All test dataverses should be created  in our demo environment: https://demo.dataverse.org/

Join our Dataverse Community! https://groups.google.com/forum/#!forum/dataverse-community

[Sherry Lake]

1st a question - Phil, you said "OJS does pause when showing the "DatasetPublishPopupCustomText" but this is something that each system that integrates with Dataverse would have to do."

Does OJS use SWORD or native API, what is the return code for the popup (or how do I know if one exists)? Would it be a GET (or curl settings call) and check if DatasetPublishPopupCustomText has a value - then display?

As promised:

I have attached my word file with create, upload, publish API commands AND dataset JSON template (with UVa requirements - NOTE - at UVA productionDate is required).

Should I add my word file to an existing issue (6107 or 6083), or create a new one to have it included in the API documentation?

Thanks.

Sherry

[Philip Durbin]

Hi Sherry,

Whoops, I means OSF, not OJS. OSF is the one that pauses to show users the contents of ":DatasetPublishPopupCustomText". Here's where they implemented that pause: https://github.com/CenterForOpenScience/dataverse-client-python/pull/1/files

Now that I've turned us around (sorry), I'm not sure if you're asking about OSF or OJS when it comes to the use of SWORD or native. In my pull request here's what I wrote about SWORD vs native...

When Should I Use the Native API vs. the SWORD API?
---------------------------------------------------

The :doc:`sword` is based on a standard, works fine, and is fully supported, but much more development effort has been going into the :doc:`native-api`, which is not based on a standard. It is specific to Dataverse.

SWORD uses XML. The Native API uses JSON.

SWORD only supports a dozen or so operations. The Native API supports many more.

... and I hope that helps a bit. This is in the new FAQ page and I'm happy to fix it up more or review pull requests.

It doesn't really matter if you use SWORD or native. It all comes down to preference. And OJS or OSF could switch from one to another and we may not even notice. Those are two of our oldest integrations so they definitely started with SWORD but I wouldn't be surprised if they call into the native API here and there. In fact, you have to call into the native API to get the value of ":DatasetPublishPopupCustomText".

I just had to check myself but the return code for getting ":DatasetPublishPopupCustomText" is 200 if it exists and 404 if it doesn't. If you add "-i" to curl you can see this:

$ curl -i https://dataverse.lib.virginia.edu/api/info/settings/:DatasetPublishPopupCustomText
HTTP/1.1 200 OK
Date: Wed, 28 Aug 2019 10:42:03 GMT
Content-Type: application/json
Content-Length: 1258
Connection: close

{"status":"OK","data":{"message":"By clicking Continue you affirm...

$ curl -i https://dataverse.unc.edu/api/info/settings/:DatasetPublishPopupCustomText
HTTP/1.1 404 Not Found
Date: Wed, 28 Aug 2019 10:41:57 GMT
Server: Apache
Content-Type: application/json
Content-Length: 79

{"status":"ERROR","message":"Setting :DatasetPublishPopupCustomText not found"}

Thanks for the Word doc. A new issue would be great. I don't think we'll want to simply include a copy of your Word doc in the guides since we try to avoid proprietary formats but from a quick look I can tell there's some great insight in there, which we can weave into the API Guide.

To all, my pull request to make some improvements to the API Guide has been merged (thanks, Mike for catching and fixing so many typos first) but I'm still definitely interested in feedback. Everyone should feel free to open issues to make more improvements in the future: https://github.com/IQSS/dataverse/issues

I hope this helps,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/4bbb9b94-ae1e-4140-aca0-dbbeed33f9d7%40googlegroups.com.

[Sherry Lake]

Thanks, Phil, this is really useful.

I thought there was a curl "query" to check for the existence of a popupText setting but just wasn't finding that info. Now that gets me thinking....

If our users use the native "publish" API command, then by default the popupText does not get displayed. I would have to write a UVa-publish script that would do the popupText check, display and then continue complete the "publish" command. Hmm......

But I think our API Terms has us covered so I don't have to make a special script. https://www.library.virginia.edu/libra/datasets/libra-data-api-terms-of-use/

which is based on Harvard's API terms, with an added section on "Publishing Content".

--

Sherry

[Philip Durbin]

Hmm. You're making me think that the new "Depositing Data"* section of the "Getting Started" page should start with "Reading API Terms of Use" from https://dataverse.lib.virginia.edu/api/info/apiTermsOfUse or equivalent. (The users should have already agreed to the General Terms of Use when they created their accounts.)

And maybe under "Publishing a Dataset" there should be a line about checking https://dataverse.lib.virginia.edu/api/info/settings/:DatasetPublishPopupCustomText or equivalent before actually publishing.

Please feel free to create an issue or two about this. :)

Thanks,

Phil

* The new "Depositing Data" section: https://github.com/IQSS/dataverse/blob/f6d4a73b577eb37cc1d436117df39885e54cb89b/doc/sphinx-guides/source/api/getting-started.rst#depositing-data

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/93c68db7-b7d5-4f29-a559-543ae8be6c96%40googlegroups.com.

[Sonia Barbosa]

Great questions from the non technical side of things. I hope to finish reviewing your recent udpates this week, Phil.

And I want to try my hand and creating some scripts to do several things, using more than one a API.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8Epx_ssWEs5H4c-DXUCvY7iis43OCXcngNi2znhjwh4wg%40mail.gmail.com.