DDI vs JSON Import of existing datasets

[Bethany Seeger]

Hello, 

(For reference, we are using dataverse v 5.13)

We are working towards migrating a bunch of datasets into Dataverse.  These datasets already have handles, so I think we would use the import method, versus creating new datasets. 

The system they are in now does map all the fields into DDI, so we could do a DDI import [1]for these datasets. We will need to have a custom metadata block for some fields that do not exist in Dataverse yet and that's where I start to get confused about DDI and mappings. 

If that custom metadata block has fields that map to DDI, will the DDI Import successfully ingest into those fields, or does it only work with fields found in the metadata crosswalk?  [2]

For Terms of Use we'd have to do something afterwards to set that, since there is no DDI mapping. Do I understand this correctly? 

Does anyone have any experience with DDI imports and any wisdom they could share? 

Thanks so much!

Bethany

[1] https://guides.dataverse.org/en/5.13/api/native-api.html?highlight=ddi%20ingest#import-a-dataset-into-a-dataverse-installation-with-a-ddi-file

[2] https://docs.google.com/spreadsheets/d/1VAhZ83hKURX_4T-bOkn7XCb9h62Gr2gcJI1fWe4Rl4c/edit?gid=1217395969#gid=1217395969

[Victoria Lubitch]

Are you migrating the datasets from another Dataverse installation?

In that case we did such migration and created python scripts, although we used json import api, not ddi, since ddi would not have all the metadata (like additional custom and terms of use were problematic). https://github.com/scholarsportal/dataverse-migration-scripts We are using DataCite for identifiers and we were able to migrate with existing identifiers, since the inport json api has a parameter to preserve the persistent identifier with

pid=$PERSISTENT_IDENTIFIER

We were using pyDataverse for the scripts.

But if are not importing from existing Dataverse installation and do not have json but only DDI then in 5.3 there is still an issue with terms of use but in the last Dataverse release it was fixed.

Victoria

[Victoria Lubitch]

Another small thing I forgot to add.

With DDI import (when we had DDI not from dataverse but from other repository) what we did is we imported with DDI to dataverse and after that run an additional script for update metadata (or one can use edit metadata) api of already created datasets. In other words we created a spreadsheet with data for terms of Use and licenses and any other custom metadata we had . Then we got a json of already  created metadata in dataverse and automatically updated it  and used update metadata / edit metadata apis for updating the dataset.

Victoria