DataCite - Client API as a proxy

103 views
Skip to first unread message

Janet McDougall - Australian Data Archive

unread,
Apr 19, 2017, 3:21:40 AM4/19/17
to Dataverse Users Community
Hi All
ADA platform - Dataverse 4.6.1. on CentOS 6.9 VMs (VMWare) 

We are getting ready to test DOI minting using Australian National Data Service (ANDS) provided Datacite DOIs as a proxy to Datacite.  ANDS is funded to support the Australian research community.  


QUESTION
Will Dataverse allow us to use the ANDS DataCite Client API instead https://mds.datacite.org?

doi.baseurlstring: As of this writing “https://ezid.cdlib.org” and “https://mds.datacite.org” are the only valid values.  (http://guides.dataverse.org/en/4.3.1/installation/config.html#jvm-options)  


Info:
The ANDS  DataCite Client API essentially acts as a proxy to the RESTful DataCite DOI API. Requests made to the ANDS DataCite Client API are captured and then passed on to DataCite to process. Apart from a change to the base endpoint URL your application interacts with the ANDS DataCite Client API as if it was interacting directly with the DataCite API.

Thanks
Janet

Philip Durbin

unread,
Apr 19, 2017, 9:19:17 AM4/19/17
to dataverse...@googlegroups.com
Hmm, can you please try it and see? :)

If it works, please open a GitHub issue for us to fix the documentation!

Thanks!

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/39547f38-8868-4bd4-a1b1-cbd1df19a8b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

Janet McDougall - Australian Data Archive

unread,
Apr 20, 2017, 1:01:37 AM4/20/17
to Dataverse Users Community, philip...@harvard.edu
hi Phil
will do, and will share outcome.
Janet


On Wednesday, 19 April 2017 23:19:17 UTC+10, Philip Durbin wrote:
Hmm, can you please try it and see? :)

If it works, please open a GitHub issue for us to fix the documentation!

Thanks!

Phil
On Wed, Apr 19, 2017 at 3:21 AM, Janet McDougall - Australian Data Archive <janet.m...@anu.edu.au> wrote:
Hi All
ADA platform - Dataverse 4.6.1. on CentOS 6.9 VMs (VMWare) 

We are getting ready to test DOI minting using Australian National Data Service (ANDS) provided Datacite DOIs as a proxy to Datacite.  ANDS is funded to support the Australian research community.  


QUESTION
Will Dataverse allow us to use the ANDS DataCite Client API instead https://mds.datacite.org?

doi.baseurlstring: As of this writing “https://ezid.cdlib.org” and “https://mds.datacite.org” are the only valid values.  (http://guides.dataverse.org/en/4.3.1/installation/config.html#jvm-options)  


Info:
The ANDS  DataCite Client API essentially acts as a proxy to the RESTful DataCite DOI API. Requests made to the ANDS DataCite Client API are captured and then passed on to DataCite to process. Apart from a change to the base endpoint URL your application interacts with the ANDS DataCite Client API as if it was interacting directly with the DataCite API.

Thanks
Janet

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

MM - ADA

unread,
Apr 26, 2017, 4:32:23 AM4/26/17
to Dataverse Users Community, philip...@harvard.edu
I tested the ANDS DataCite proxy (it took a little longer than expected due to some extra required server configurations).

Sadly, the proxy does not work and attempting to publish a dataset through it fails with the UI error:

"This dataset may not be published because the DataCite Service is currently inaccessible. Please try again. Does the issue continue to persist? Please contact Dataverse Support for assistance."

This is also THE only error/message that is logged in the glassfish server.log file.

The status, url and xml *are* being written into the doidataciteregistercache table with the status set to 'reserved' though.


Is there a workaround that we can implement to be able to use this ANDS proxy?
Or will dataverse code be modified in the future to accommodate this?
Or will we have to go through DataCite itself? We would prefer to go through the ANDS proxy for a few different reasons...

Thanks!

Philip Durbin

unread,
Apr 26, 2017, 7:39:31 AM4/26/17
to dataverse...@googlegroups.com
Hmm, I found where that error is defined:


And here's where the error is thrown due to an exception, but the exception isn't printed, which is why you aren't seeing any more detail in server.log:


If you can find a developer to help you diagnose this, I would suggest adding some logging there so you can see what the exception is.

You are quite welcome to go ahead and create a GitHub issue. In the issue it would be great if you mention which version of Dataverse you are running. I can add the technical details above.

Thanks,

Phil



To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/09347542-1587-4b91-9221-f404c7f84a78%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

MM - ADA

unread,
Apr 30, 2017, 7:57:47 PM4/30/17
to Dataverse Users Community, philip...@harvard.edu
Hi Phil - As a Java developer in a previous life, I was able to modify and recompile the code to figure out the issue.

The proxy API is returning a status code of 200 when it shouldn't so this is issue is being dealt with by the API team as I write this.

It would be nice to have those extra logging details the various DOI-related classes in the dataverse codebase.

Anyway, I will keep you posted on results once the proxy API has (hopefully) been fixed.

Thanks!
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/39547f38-8868-4bd4-a1b1-cbd1df19a8b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

Philip Durbin

unread,
May 1, 2017, 6:49:39 AM5/1/17
to dataverse...@googlegroups.com
Great! I sounds like you're making improvements to the ANDS DataCite Client API, the "proxy". I was just looking at https://documentation.ands.org.au/display/DOC/ANDS+DataCite+Client+API and I wonder how many more of these there are out there. If you get it working, please do feel free to make a pull request to correct the Installation Guide to indicate that there are three valid values!

In terms of logging, if you have specific places in the code (besides the one we've been talking about) where additional logging could be added, please let us know. There's work currently underway to add Handle support in https://github.com/IQSS/dataverse/issues/2437 and the code that handles persistent IDs like DOIs and Handles is being significantly refactored. The hope is that it will be easier for contributors to add additional persistent ID providers in the future.

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/1c8bcbf3-e2de-4cd9-bfd6-b6bfda2a2743%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

MM - ADA

unread,
May 3, 2017, 7:39:02 AM5/3/17
to Dataverse Users Community, philip...@harvard.edu
Hi Phil.

The ANDS proxy API team fixed the issue where the API always returned a status of 200 so it is now working properly.

So I tested the dataverse call again and we are now getting this error after calling the DataCiteRESTfullClient.postMetedata() method:

Caused by: java.lang.RuntimeException: Response code: 401, An Authentication object was not found in the SecurityContext
        at edu.harvard.iq.dataverse.DataCiteRESTfullClient.postMetadata(DataCiteRESTfullClient.java:187)
        at edu.harvard.iq.dataverse.DOIDataCiteRegisterService.createIdentifier(DOIDataCiteRegisterService.java:89)


When the DataciteRESTfullAPI.testDOIExists() method is called, the call to the ANDS API returns this (same) 401 status.

Because dataverse sees that the status is != 200 it calls the createIdentifier() method which then gives the Exception above.


Our tech contact at ANDS looked at their log files and the username (api key) doesn't seem to be sent through with dataverse's API call and is somehow being 'lost' (in their log file, the call is logged but there is nothing entered for the API key that dataverse supposedly sent). I entered some logging messages into dataverse to see if I could figure out what is going on.

The context object passed to the HttpResponse response = httpClient.execute(httpPost,context); method in the testDOIExists() and postMetadata() methods still seems to have the Credentials associated with it right before the call to the ANDS API so I am not sure what the issue is.

Furthermore, when we send the username (API key) and password to the ANDS API url through Postman, the ANDS API works as expected. The username and password are base64 encoded as an http header and then sent to the ANDS API and it works correctly.

Could this be a Dataverse problem with how the authentication parameters are being sent? I am probably grasping at straws here but that seems to be the only difference that I can see. Is it possible that the authentication parameters are not being sent by dataverse at all?

As usual, any insight into this "An Authentication object was not found in the SecurityContext" exception would be very welcome!
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

danny...@g.harvard.edu

unread,
May 3, 2017, 10:04:33 PM5/3/17
to Dataverse Users Community, philip...@harvard.edu
Hey Marina, Stephen from the Dataverse team mentioned that he reached out to you to request some more information. We'll make sure to follow up in this post once we have an answer.

MM - ADA

unread,
May 10, 2017, 9:42:16 PM5/10/17
to Dataverse Users Community, philip...@harvard.edu
Hi all - After further exploration and debugging, myself and the ANDS team determined that the authentication credentials from dataverse were not being accepted by the ANDS proxy, resulting in the issues I mentioned in my previous community post. It was difficult to know whether it was an apache configuration issue on our dataverse server (as other internet posts suggested wrt the specific error being logged by dataverse) or something to do with the ANDS API.

After determining where the issue lay, the ANDS team pushed their fix for this issue yesterday. And after testing the proxy with our dataverse dev instance, we are happy to report that, in addition to DataCite and EZID,  dataverse can seemingly work with other DOI providers (and specifically ANDS) to mint DOIs when publishing. Of course, the doi-related parameters of providers other than EZID, DataCite and ANDS would have to be tested with dataverse to see if they will work. But it is possible...

Thanks!

M.

Philip Durbin

unread,
May 15, 2017, 1:56:18 PM5/15/17
to dataverse...@googlegroups.com
That's fantastic news! Marina, can you please open a GitHub issue at https://github.com/IQSS/dataverse/issues to update the documentation at http://guides.dataverse.org/en/4.6.1/installation/config.html#doi-baseurlstring ? Thanks to your efforts, "https://ezid.cdlib.org" and "https://mds.datacite.org" are no longer the only valid values!

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/bfae49b8-c5aa-4238-8d99-d692bf96015a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

MM - ADA

unread,
May 15, 2017, 9:51:32 PM5/15/17
to Dataverse Users Community, philip...@harvard.edu
Hi Phil - I opened the GitHub issue to update the documentation (perhaps with more detail than necessary):

https://github.com/IQSS/dataverse/issues/3832

Thanks!
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

Philip Durbin

unread,
May 16, 2017, 8:37:36 AM5/16/17
to dataverse...@googlegroups.com
Thanks, Marina! Please feel free to create a pull request if you are so inclined! :)

Here's the page you would edit:

https://github.com/IQSS/dataverse/blob/v4.6.1/doc/sphinx-guides/source/installation/config.rst#id67

Tips on making a quick fix like this: http://guides.dataverse.org/en/4.6.1/developers/documentation.html



To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/d44c3d65-1d2a-4cf8-8bf3-b474db2221ef%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Philip Durbin

unread,
May 18, 2017, 9:23:16 AM5/18/17
to dataverse...@googlegroups.com
Heads up that issue #3832 that Marina created (thanks again!) is now in the "This Sprint (5/17-5/31)" column at https://waffle.io/IQSS/dataverse which means that the team at IQSS will take care of it. Writing this email is probably more working than fixing that line in the guides but I don't want anyone to waste their time making a pull request in this case! :)

There are over 800 *other* issues in the backlog to work on. If anyone wants to help out, please just get in touch! :)

Thanks!

Phil

To post to this group, send email to dataverse-community@googlegroups.com.

MM - ADA

unread,
May 18, 2017, 8:34:53 PM5/18/17
to Dataverse Users Community
Apologies - I should have replied earlier to the first documentation-related post but have far too much on the go.

I received an email from a DV team member regarding a proposed update to the documentation wrt the DOI issue which looks good to me.

Thanks!
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages