How to tell dataverse not to generate a new DOI for my dataset which has already a DOI

260 views
Skip to first unread message

Esther Dzale

unread,
Jan 26, 2017, 6:23:38 AM1/26/17
to Dataverse Users Community
Hi,
I would like to create a dataset in my dataverse through the IHM or the API, but the data have already a DOI. How could I tell my dataverse not to generate another DOI?
Thank you for your help.
Esther

Philip Durbin

unread,
Jan 26, 2017, 7:39:51 AM1/26/17
to dataverse...@googlegroups.com
Hi! The most common cases of wanting to add datasets that already have a DOI to a Dataverse installation are these:

1. I want to harvest a dataset over OAI-PMH into my Dataverse installation.

2. I am migrating datasets from a DVN 3 installation into my Dataverse installation.

In both of these cases, a new DOI is not minted. The dataset gets the existing DOI.

Does your use case fall into one of the scenarios above or is it a new use case? Also what is "the IHM", please? :)

Thanks,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a3df50b4-3223-44cc-9874-1b545cc656ca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

Esther Dzale

unread,
Jan 27, 2017, 9:15:08 AM1/27/17
to Dataverse Users Community, philip...@harvard.edu
Hi Phil,
thank you very much for your answer.
I have many use cases actually. Some data are in Dspace repositories and I should be able to harvest them over OAI-PMH. But some of the data are in "home made" repositories which are not OAI-PMH compliant. I was planning to use the dataverse API for these "home made" repositories but I understand that it will result in creating double DOIs for the data.
Are you planning to make it possible for any repository to push metadata and/or data in Dataverse using the API and without recreating DOIs when they exist?
Esther

Le jeudi 26 janvier 2017 13:39:51 UTC+1, Philip Durbin a écrit :
Hi! The most common cases of wanting to add datasets that already have a DOI to a Dataverse installation are these:

1. I want to harvest a dataset over OAI-PMH into my Dataverse installation.

2. I am migrating datasets from a DVN 3 installation into my Dataverse installation.

In both of these cases, a new DOI is not minted. The dataset gets the existing DOI.

Does your use case fall into one of the scenarios above or is it a new use case? Also what is "the IHM", please? :)

Thanks,

Phil
On Thu, Jan 26, 2017 at 6:23 AM, Esther Dzale <estd...@gmail.com> wrote:
Hi,
I would like to create a dataset in my dataverse through the IHM or the API, but the data have already a DOI. How could I tell my dataverse not to generate another DOI?
Thank you for your help.
Esther

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

Sebastian Karcher

unread,
Jan 27, 2017, 9:23:40 AM1/27/17
to dataverse...@googlegroups.com, philip...@harvard.edu
I'm going to tag on here because we'll be facing the same challenge as we're moving over our data to a (local) DV installation from a non-OAI-PMH capable repository, so we'd want to do the exact same thing, i.e. create a dataset, but assign it an already existing DOI.

On Fri, Jan 27, 2017 at 9:15 AM, Esther Dzale <estd...@gmail.com> wrote:
Hi Phil,
thank you very much for your answer.
I have many use cases actually. Some data are in Dspace repositories and I should be able to harvest them over OAI-PMH. But some of the data are in "home made" repositories which are not OAI-PMH compliant. I was planning to use the dataverse API for these "home made" repositories but I understand that it will result in creating double DOIs for the data.
Are you planning to make it possible for any repository to push metadata and/or data in Dataverse using the API and without recreating DOIs when they exist?
Esther

Le jeudi 26 janvier 2017 13:39:51 UTC+1, Philip Durbin a écrit :
Hi! The most common cases of wanting to add datasets that already have a DOI to a Dataverse installation are these:

1. I want to harvest a dataset over OAI-PMH into my Dataverse installation.

2. I am migrating datasets from a DVN 3 installation into my Dataverse installation.

In both of these cases, a new DOI is not minted. The dataset gets the existing DOI.

Does your use case fall into one of the scenarios above or is it a new use case? Also what is "the IHM", please? :)

Thanks,

Phil
On Thu, Jan 26, 2017 at 6:23 AM, Esther Dzale <estd...@gmail.com> wrote:
Hi,
I would like to create a dataset in my dataverse through the IHM or the API, but the data have already a DOI. How could I tell my dataverse not to generate another DOI?
Thank you for your help.
Esther

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/1bdbd688-1944-4a1c-8e69-9c78a7027e80%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Sebastian Karcher, PhD
www.sebastiankarcher.com

Philip Durbin

unread,
Jan 27, 2017, 10:32:41 AM1/27/17
to dataverse...@googlegroups.com
This use case reminds me of the "Import Utilities" we had in DVN 3. Back then datasets were called "studies" (and Dataverse was called "Dataverse Network") and here was the description of this feature: "Importing studies usually is done by harvesting study metadata from a remote site via the OAI protocol. This causes study metadata to be hosted locally but files are served by the remote server. The Import utility is provided for cases where an OAI server is unavailable or where the intent is to relocate studies and their files to the Dataverse Network."[1]

This import concept didn't make the cut for the rewrite to 4.0 according to the "3.6 & 4.0 Feature Comparison" doc[2]. However, I'm wondering if the "migrate" feature can be re-used for this purpose. It's not especially well documented at this time. Also, you'd need to format your dataset in one of the supported formats, which are Dublin Core, DDI, and "native JSON"[3]. DDI would probably be preferred.

I guess I'm wondering if you want to move the files themselves over to Dataverse or not. Either way, perhaps Esther and Sebastian could collaborate on the wording for a new GitHub issue to track this use case. You're welcome to go ahead and create an issue, or separate issues if the use cases are different, at https://github.com/IQSS/dataverse/issues

I hope this helps!

Phil

To post to this group, send email to dataverse-community@googlegroups.com.
--
Sebastian Karcher, PhD
www.sebastiankarcher.com

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAOSYSD74bFDyrg%2BeDnx%2BUkNySBNkyae8ynaVLcjJYAKapnU50A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Sebastian Karcher

unread,
Jan 27, 2017, 11:53:11 AM1/27/17
to dataverse...@googlegroups.com
Thanks Phil,
we'll explore how well this works via migration (using DDI or DV's native JSON doesn't sound hard in our case) and depending on that either document our workflow or start a gh issue and ping Esther for input. This should happen within the next 2 months, hopefully sooner.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Philip Durbin

unread,
Jan 27, 2017, 12:13:40 PM1/27/17
to dataverse...@googlegroups.com
Sounds good. Here are some links you can check out:

- https://github.com/IQSS/dataverse/blob/v4.6/scripts/migration/migration_instructions.txt
- https://github.com/IQSS/dataverse/blob/v4.6/src/test/java/edu/harvard/iq/dataverse/api/BatchImportIT.java (me playing around with the APIs)
- Create Dataset: add support for migrating (with existing DOIs) via native API - https://github.com/IQSS/dataverse/issues/3083 (potential to collaborate on this open issue but folks who opened it don't need to migrate the files themselves)
- https://groups.google.com/forum/#!forum/dataverse-migration-wg (for migration from DVN 3 rather than an "import" use case but just a reminder to all that this list exists) :)

Phil




--
Sebastian Karcher, PhD
www.sebastiankarcher.com

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAOSYSD52SOMeQPv98o-MoQ4h2dcKhSjJasTCuciG3gtPF%3D9KVg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Durand, Gustavo

unread,
Jan 27, 2017, 1:37:33 PM1/27/17
to dataverse...@googlegroups.com
Hi all,

I do think, as a general goal, we'll want migrate functionality to work for non DVN sources. (and also for Dataverse 4.x as a source, as one use case will be to support when an organization decided to split off of an installation to have their own, or in merging two installations)

That said, it's always a question of when. In the short term, I am hopeful we can help people tweak things* to use the current functionality.

* I purposely chose the vaguest term possible; it likely would be to tweak the source datasets (get them into DDI or JSON format, modify what field is used, etc), but could also include supporting pull requests to make code modifications

Gustavo

On Fri, Jan 27, 2017 at 12:13 PM, Durbin, Philip <philip...@harvard.edu> wrote:
Sounds good. Here are some links you can check out:

- https://github.com/IQSS/dataverse/blob/v4.6/scripts/migration/migration_instructions.txt
- https://github.com/IQSS/dataverse/blob/v4.6/src/test/java/edu/harvard/iq/dataverse/api/BatchImportIT.java (me playing around with the APIs)
- Create Dataset: add support for migrating (with existing DOIs) via native API - https://github.com/IQSS/dataverse/issues/3083 (potential to collaborate on this open issue but folks who opened it don't need to migrate the files themselves)
- https://groups.google.com/forum/#!forum/dataverse-migration-wg (for migration from DVN 3 rather than an "import" use case but just a reminder to all that this list exists) :)

Phil



--
Sebastian Karcher, PhD
www.sebastiankarcher.com

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages