OAI metadata export

[Christian Bischof]

Hello,

the automatic OAI metadata doesn’t export 8 records, the log file shows the error "Unknown runtime exception exporting metadata." The dashboard shows “1,521 records exported, 8 marked as deleted”. A manual reexport with “curl http://localhost:8080/api/admin/metadata/reExportAll” brings the same result. The datasets are marked as deleted <header status="deleted"> https://data.aussda.at/oai?verb=GetRecord&metadataPrefix=oai_ddi&identifier=doi:10.11587/URX62K

But they datasets are ok and not deaccessioned

https://data.aussda.at/dataset.xhtml?persistentId=doi%3A10.11587%2FURX62K

 

What could be the problem?

Thank you

Christian

[James Myers]

Hmm, looking at doi%3A10.11587%2FURX62K, it appears that there is an error in generating the DDI export, possibly at the point the geographic coverage section is being written (see https://data.aussda.at/api/datasets/export?exporter=ddi&persistentId=doi%3A10.11587/URX62K). My guess would be that this error then makes the dataset unavailable for the oai_ddi export.

 

With a quick look at the code in 6.1 and 5.12.1 in that section I don’t see any obvious flaws.

 

Hopefully across your 8 datasets you can see a common issue that is causing a DDI export failure and that will give more clues in getting to a root cause. It’s possible that using FINE logging in the exporter classes might help (but I don’t see any logging right in that geogCover code).

 

-- Jim

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/dd3a5951-a229-4883-bdb2-6cb7f06381f9n%40googlegroups.com.

[Christian Bischof]

I can't find a pattern along the 8 cases. This one fails at the start https://data.aussda.at/api/datasets/export?exporter=ddi&persistentId=doi%3A10.11587/7P6VWS

Thx

[James Myers]

I don’t have any great advice. I think the harvest issue is really just about the DDI export failure, so hopefully getting the export to work will fix the rest. In terms of general debugging:

• Is there any log info when you call the reExportDataset api for these datasets (with FINE logging?)?
• Do they always fail in the same place? (I would assume they have to fail in the same place unless there is some issue like lack of disk space where I think you’d see other issues.)
• Can you reproduce with the same metadata in another dataset/a test server/on v6.1?
• I think DDI includes variable level metadata – do the datasets have ingested files? Does that appear to have worked?

--Jim

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/eec6a96d-d40e-4206-b1b6-501c11ba34b7n%40googlegroups.com.

[Christian Bischof]

thx, I will keep an eye on it

[Christian Bischof]

Finally, I found the reason why the DDI/OAI-DDI metadata export fails. I changed allowmultiples from false to true of 3 fields (Type of Research Instrument, Time Method, Sampling Procedure from the social_science metadatablock) and DdiExportUtil.java can’t handle this, it can only export single value https://github.com/IQSS/dataverse/blob/4624bb61012bf873d5b6dbc05de077bf9ac90e2f/src/main/java/edu/harvard/iq/dataverse/export/ddi/DdiExportUtil.java

This problem relates to the most fields of the social_science metadatablock,in social sciences we must use sometimes multiples to describe the methodology of studies, DDI also supports multiples. DdiExportUtil.java needs the ability to handle multiples. I will create a feature request for this functionality.

Best

[Philip Durbin]

Thanks for creating https://github.com/IQSS/dataverse/issues/11059

To view this discussion visit https://groups.google.com/d/msgid/dataverse-community/efc0be16-d3f2-4061-9c5a-a400f871bbd5n%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin