OAI-PMH harvesting FGDC?

[Stephen Marks]

Hi list--

I've been trying to find a quick answer to this, but I don't seem to be able to turn it up. Thought I'd throw this out to the group, although I never see much traffic on here. (As a side note, I am interested in knowing if there are places where vigorous Dataverse discussion goes on!)

Is it possible to set up an oai harvesting dataverse that can import FGDC records? I kind of assumed yes, since Dataverse can import plain old FGDC XML, but I don't seem to be having much luck.

[Gustavo Durand]

Hi Stephen,

Yes, dataverse can harvest FGDC records. If the source OAI server
specifies FGDC as on of the options, it will appear in the harvesting
format dropdown list. (for example, we here at IQSS have harvested
FGDC records from HGL).

Re: the side note, this is as good a pace for any to have dataverse
related discussion. We started this group so that we can throw ideas
at the community and get feedback, but it could / should also be used
by the community directly for any dataverse related topics.

Thanks,
Gustavo
Manager, DVN

[Stephen Marks]

Hi Gustavo--

Cool, thanks! I feel like Kevin has probably seen enough spurious RT
tickets from me so I wanted to do something a little less alarmist. =)

So, here's a bit more info. This is the OAI server URL I'm trying to
harvest from:
http://www.polardata.ca/oai/provider

When I do ListMetadataFormats (which I assume you are doing during the
validation bit where that drop-down populates), I get the following
output.

<?xml version="1.0" encoding="UTF-8" ?>
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/
http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
<responseDate>2012-06-14T19:57:45Z</responseDate>
<request verb="ListMetadataFormats">http://www.polardata.ca/oai/provider</request>
<ListMetadataFormats>
<metadataFormat>
<metadataPrefix>fgdc</metadataPrefix>
<schema>http://www.fgdc.gov/metadata/fgdc-std-001-1998.xsd</schema>
<metadataNamespace>http://www.fgdc.gov/metadata/</metadataNamespace>
</metadataFormat>
<metadataFormat>
<metadataPrefix>oai_dc</metadataPrefix>
<schema>http://www.openarchives.org/OAI/2.0/oai_dc.xsd</schema>
<metadataNamespace>http://www.openarchives.org/OAI/2.0/oai_dc/</metadataNamespace>
</metadataFormat>
</ListMetadataFormats>
</OAI-PMH>

So I feel like the server is identifying FGDC as an available record
format, but I don't see the option appear in my dropdown. Can I test
on the HGL OAI server? Is it open to the world? I'd like to see it
work once before I kick this issue at the people running the OAI
server.

Thanks for your quick response!

Steve

[Gustavo Durand]

OK, so I tried ListMetadataFormats for HGL to see what it returns:

and it is different, in that it returns oai_fgdc:

metadataPrefix	oai_fgdc
schema	http://www.fgdc.gov/metadata/fgdc-std-001-1998.xsd
metadataNamespace	http://www.fgdc.gov/metadata/

This seems similar to how both return oai_dc.

So there seem to be two possibilities here:

1. Have them switch to oai_fgdc, which DVN recognized. (no changes on the DVN side, and fully supported)

2. Add regular fgdc to DVN's recognition. You would just have to add a new row to the harvestformattype table for fgdc. 

Currently we have these values for oia_fgdc:

3;"oai_fgdc";"FGDC";"fgdc2ddi.xsl"

The unknown here is if fgdc and oai_fgdc are the same, so that our existing crosswalk file (fgdc2ddi.xsl) works. It seems like it should, but this is not something we have tested and I don't know the specifics to the format to know if there is a difference or not.

If there were a difference, then a new crosswalk file would need to be created and then referenced by this new table row.

Gustavo

[Stephen Marks]

Hi Gustavo--

Right you are. I went with option 2, since I have no pull with the server folks. =)

It still didn't work, since from what I can tell from the log, the FGDC they're providing isn't the same as what your provided stylesheet is looking for. But that's for me to deal with, and at least now I understand how to add new types. 

Thanks for helping me sort this out. I understand one little corner of the application a little better now.

s

[Gustavo Durand]

OK.

If you do produce a stylesheet that works for regular fgdc, please let
us know as we can then add that to the project.

Have a good weekend,
Gustavo