OAI harvesting with Trove (National Library of Australia)
Erin Mollenhauer
We have self-hosted AtoM, version 2.3.0.
We are working with Trove Australia to allow them to harvest our records - the OAI plugin is activated.
There are two issues:
1. The resumption tokens are causing the same 100 records to be harvested repeatedly.
2. When Trove tries to harvest authority records, we encounter this (message from Trove support):
We are about to upgrade to 2.3.1.
Has anyone encountered these issues and can you shed any light?
Thank you!
Erin Mollenhauer
Monograph Acquisitions and Special Collections Librarian – Donald Robinson Library, Moore College
Address: 1 King Street, Newtown NSW 2042 Australia | Web: www.moore.edu.au |
CRICOS Provider Code: 00682B
Important Notice: This email is for the named recipient only. Its contents are confidential and may contain legally privileged information. The unauthorised use, disclosure, copying or alteration of this message is strictly forbidden. If you receive this email in error, please contact the sender immediately and delete the email and all attachments from your system. This email is subject to copyright. Copyright: Moore Theological College Council.
Dan Gillean
We have merged the fix to our stable/2.3.x branch, but the bug might have been identified and fixed AFTER the public 2.3.1 release - so it is possible the fix is only in our code repository, and not in the 2.3.1 tarball available in the Downloads section of our website.
If you are upgrading to 2.3.1 and want this fix, you have 2 options. The first is to follow Option 2 in our installation instructions and install from the stable/2.3.x branch of our GitHub code repository.
The second option is to patch the code yourself - if you look at the related commit from the fix, it is a one-line change to fix the resumption token:
The second issue is a bit trickier. The AtoM OAI repository module has not currently been set up to expose authority records - just archival descriptions. The fact that it is paginating at all through HTML search/browse pages is a bit surprising to me! You could always try altering the code locally to see if you can make it validate so Trove's harvester doesn't choke on that input? Ultimately, adding full support for exposing EAC-CPF XML authority records via the OAI repository module will require development to implement properly.
As a final note, you might find the following development coming in the 2.4 release interesting: our OAI repository module will be able to expose EAD XML (rather than just simple Dublin Core) for harvesting records. To ensure that the page does not time out while trying to generate the EAD on demand, there is also an option to pre-generate and cache all EAD and DC XML. When the setting is turned on, any descriptions that are edited or newly created will automatically trigger a job to update the cached XML, so it can be exposed on request. See:
Regards,
--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/a16cb146-e173-4e5c-a466-67ef9af159ca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Erin Mollenhauer
Thanks for your reply. We have completed the upgrade and the resumption token issue is fixed - all of our description records have been harvested.
We have decided not to pursue harvesting the authority records for now. This is what the Trove technician told me:
That the Trove Harvester does have the ability to paginate through HTML pages due to the way that it is set up (as it has the ability to take a list of links and follow each of them to find records), it isn’t related to the OAI-PMH feed at all. I was experimenting with our other methods of harvesting to see if I could get it working. As it doesn’t appear simple I will leave it as it is.
Regards,
Erin
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Dan Gillean
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/5e116e5d-ae1b-4a07-b10a-aee4d5eca7e4%40googlegroups.com.