Dataverse 4.5 is here!

[danny...@g.harvard.edu]

Hi everyone,

The four main features of this release are Harvesting, Metadata Export, Private URL, and Federated Login Mode.

• Harvesting allows the dataset metadata from another site to be imported so that these files appear to be local, though data files remain on the remote site. This allows Dataverse installations and other repositories to share metadata with each other to create a data sharing community and to provide more access to the datasets stored in each repository. Harvesting is implemented using the standard OAI-PMH protocol - any Dataverse installation can be configured as an OAI client and server.

• Metadata Export makes it easier to share and discover from Dataverse. Another key function of the export is to store the metadata in the file system in a format for preservation. In this release, the metadata is exported in Dublin Core, Data Documentation Initiative (DDI), and native JSON. Metadata export will support additional standards in the future.

• Private URL provides a simple way to generate and send a special URL to an unpublished dataset for a group of anonymous peer reviewers to review the data.

• Federated Login Mode allows users from 200+ different organizations to sign in using his or her organizational credentials.

Big thanks to all the community members who let us know about the participated in usability testing, provided feedback in Github issues, and worked with us in other ways to make sure that this release was successful.

For the complete list of issues, see the 4.5 milestone in Github.

For help with upgrading, installing, or general questions please email sup...@dataverse.org.

[Alan Darnell]

Congratulations on the release.

I see the OAI server endpoint but it seems to only support one metadata format -- "dataverse_json" (see below).  Is that going to be the only output format for the OAI server going forward?

I can't figure out what endpoint to use for the Metadata Export feature.  Is it a new endpoint or an enhancement to the Data Sharing API?

Thanks,

Alan

---

oai?verb-ListMetadataFormats 

returns

<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">

<responseDate>2016-08-29T21:04:31Z</responseDate>

<request verb="ListMetadataFormats">https://sand8.scholarsportal.info/oai</request>

<ListMetadataFormats>

<metadataFormat>

<metadataPrefix>dataverse_json</metadataPrefix>

<schema>JSON schema pending</schema>

<metadataNamespace>

Custom Dataverse metadata in JSON format (Dataverse4 to Dataverse4 harvesting only)

</metadataNamespace>

</metadataFormat>

</ListMetadataFormats>

</OAI-PMH>

[danny...@g.harvard.edu]

Hey Alan - thanks for the kudos! Some answers:

- Amber brought up the OAI Server endpoint issue on the Community Call today. Kevin mentioned that the other formats are available, but the ListMetadataFormats were not returning these formats. We created an issue for this here: https://github.com/IQSS/dataverse/issues/3321. 

- The endpoints to kick off the metadata export are summarized here: http://guides.dataverse.org/en/latest/admin/metadataexport.html

Let me know if you have any questions!

- Danny

[Alan Darnell]

Thanks Danny.  I clued in finally that there is a new Export Metadata button in the GUI and that the API call is used not to export the files but to generate them.  Currently, our installation shows the Export Metadata button but no options in the dropdown (e.g. DDI, XML, JSON).  I'm guessing that these will come when we actually generate the export files.  This should be a very useful feature for us.  Thanks again.

[danny...@g.harvard.edu]

Hey Alan - just one more thing - DDI, JSON, and DC should be shown as option when you click the button, without the need to first generate the export. Can you send me a screenshot of what you're seeing? I'm including a screenshot of how it looks for me. Thanks! 

[Alan Darnell]

Sure -- here's what I see (attached)

[danny...@g.harvard.edu]

Hey Alan - when you have a few moments, can you please try to clear your cache and see if that fixes the issue? If that doesn't work, you may try restarting the server. 

Please do let us know how it goes. We didn't run into this when we tested or when we upgraded, so I'm interested to see if we can learn something here.

Thanks!

Danny

[Condon, Kevin M]

Hi Alan,

We are supporting oai_dc, oai_ddi, as well as dataverse_json. I'm not sure why we don't report that correctly at the protocol level but you should be able to use those other formats. Let us know if that is not the case.

Kevin

---

From: dataverse...@googlegroups.com <dataverse...@googlegroups.com> on behalf of Alan Darnell <alan.d...@gmail.com>
Sent: Monday, August 29, 2016 5:23:24 PM
To: Dataverse Users Community
Subject: [Dataverse-Users] Re: Dataverse 4.5 is here!

 

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a84d44c6-b6bd-41c3-9ec8-beb018ab29f9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alan Darnell]

Tried both of those things and still don't see anything in the drop down.  Are these values defined in a database table?  If so, I'm wondering if that didn't get updated when we installed 4.5.  

[Alan Darnell]

When I try these:

http://sand8.scholarsportal.info/oai?verb=ListRecords&metadataPrefix=oai_dc

http://sand8.scholarsportal.info/oai?verb=ListRecords&metadataPrefix=oai_ddi

I get an 500 Internal Server error.

When I try this:

http://sand8.scholarsportal.info/oai?verb=ListRecords&metadataPrefix=dataverse_json

I get this:

<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">

<responseDate>2016-09-01T14:02:34Z</responseDate>

<request verb="ListRecords" metadataPrefix="dataverse_json">https://sand8.scholarsportal.info/oai</request>

<error code="noRecordsMatch">No matches for the query</error>

</OAI-PMH>

So I think something is missing in our setup.

Alan

[Condon, Kevin M]

Alan,

Try ListIdentifiers instead. We are using an OAI library as the basis for implementation but there were a few remaining compliance issues identified by the OAI  data provider tests, including ListRecords. See https://github.com/IQSS/dataverse/issues/3307 for details.

I believe our harvest client does a ListIdentifiers, then GetRecord for each identifier as the method of harvesting.

Kevin

---

From: dataverse...@googlegroups.com <dataverse...@googlegroups.com> on behalf of Alan Darnell <alan.d...@gmail.com>

Sent: Thursday, September 1, 2016 10:17:48 AM
To: Dataverse Users Community
Subject: Re: [Dataverse-Users] Re: Dataverse 4.5 is here!

 

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/7c8e509d-cc50-4332-83de-f0e308fe9b44%40googlegroups.com.

[Kraffmiller, Stephen E]

Alan,

It looks like the exporters are not being recognized. Can you check your pom.xml file and make sure that the following is in the dependencies section?

        <!-- Added for AutoService -->

       <dependency>

          <groupId>com.google.auto.service</groupId>

          <artifactId>auto-service</artifactId>

          <version>1.0-rc2</version>

          <optional>true</optional>

          <type>jar</type>

        </dependency>

Thanks!

Stephen Kraffmiller

On Sep 1, 2016, at 12:51 PM, Alan Darnell <alan.d...@gmail.com> wrote:

ListIdentifiers and GetRecord both work, but only with metadataPrefix=dataverse_json

The link in the response from GetRecord looks like this:

<metadata directApiCall="https://sand8.scholarsportal.info/api/datasets/export?exporter=dataverse_json&persistentId=doi:10.5072/FK2/ARLIFN"/>

But this API call fails with

{"status":"ERROR","message":"Export Failed"}

Alan

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/7c8e509d-cc50-4332-83de-f0e308fe9b44%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/5c3b90d6-a7e9-4919-a55e-16bc1c7ffd06%40googlegroups.com.