approaches to migrating datasets with existing PIDs?

[Pete Meyer]

Hi everyone,

We're taking a look into migrating some datasets (with existing DOIs) into Dataverse; and it seemed like this might be something that other folks have run into.

My current thinking is that the way to do this would be to use the native API to "create" a new dataset (with information exported from the current system), but this would need either code changes or some non-standard database manipulations for datasets with existing DOIs (or Handles, but the datasets we'll be importing will have DOIs).

Is this something that others have a better way to handle, or any suggestions?

Best,

Pete

[aussda....@gmail.com]

Hi Pete,

Curious if there has been a follow up on your question. We are in a similar situation - we would like to import some external metadata that already has DOIs. The Odum institute seems to have done this successfully. Is it possible to ingest the data and then update the DOIs through a subsequent API call?

Thanks. Frank

[Pete Meyer]

Hi Frank,

There's definitely been follow up effort that hadn't made it to this thread.  This is something that is being worked on in https://github.com/IQSS/dataverse/pull/4606 .

In the absence of code to handle this (aka - before this pull request is ready), it's possible - but the approach I was able to make work required a fairly high degree of manual interaction (switching the PID provider back and forth; direct database and filesystem manipulation, etc) and was reasonably time consuming.

Best,

Pete

[aussda....@gmail.com]

Hi Pete,

Thanks very much. Sounds like our best option is to wait until pull request is merged into the main branch (though I am curious about the details of how you got it done). We've got some time in any case because the organizations we're planning to share metadata with are just beginning to install their own test versions of dataverse. 

Best, Frank

[Pete Meyer]

Hi Frank,

I'd definitely recommend waiting until that PR is merged (this is the approach we're planning to take).

The approach I was testing out in the absence of that was roughly:
 - disable external access to Dataverse
 - switch the DOI configuration to test credentials / test shoulder (JVM, database)
 - "create" the dataset with the native API (using an exporter from the existing system), and put files into it
 - "publish" the dataset (with a new test/temporary DOI)
 - change the DOI information in the database (and various publication / creation dates)
 - move files to location consistent with that DOI
 - reindex dataverse/solr
 - In a production migration, I'd re-target the DOI to the new landing page - for evaluation, I didn't do this.
 - switch things back (DOI configuration, ren-enable external access)

I got some useful suggestions from Sebastian and Gustavo about various pitfalls to avoid.

Best,
Pete

[Sebastian Karcher]

Yes, what Pete describes is exactly what we did and it worked. Small collection, though, so migrating manually wasn't much of a problem.

Sebastian

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/ac1e7e50-55ff-4c9c-a361-ca6b55c1f3f2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Sebastian Karcher, PhD
www.sebastiankarcher.com