Help Exporting Getty Vocabularies & Importing them into Atom

[prathmes...@gmail.com]

We are planning to import the Getty Vocabularies into our Atom website for  Subjects, Places & Names taxonomies. I wanted to know if anyone else had already done something similar and can guide us through the process. Or if any Atom developers Or savy users have any input on how to go about this task.

I did some research and it seems that Terms can be imported into Atom in formats XML & SKOS (https://www.accesstomemory.org/en/docs/2.0/user-manual/import-export/import-descriptions-terms). Can the subjects/names/places taxonomy only be imported using XML & SKOS format ?

Getty provides some instructions and formats in which their vocabularies can be exported out on http://vocab.getty.edu/. The Getty data sets we are interested in are AAT (Subjects) , TGN (Places) and ULAN (Names) . But its unclear to me whether or not they offer exports/downloads in formats that can be imported within Atom.

We would be grateful, if anyone can point us at the correct downloads on Getty from which the vocabularies can be imported into Atom and help me understand the workflow.


Thanks and Regards,
Prathmesh Mengane
Database & CMS Developer
Dumbarton Oaks Research Library and Collection

[Dan Gillean]

Hi Prathmesh,

I do know that there are users who have imported LCSH terms into AtoM - a huge dataset, but it is made available in SKOS XML, and with some patience and assistance, I believe that they were successful in importing the terms. I was not directly involved in that project however.

I'm attaching a sample SKOS XML export from AtoM so you can see how it is currently structured - the SKOS elements are wrapped in a simple RDF element, but I don't believe this would be considered fully implemented RDF XML - though the XML itself will validate. The SKOS implementation is also minimal, supporting only those elements that map to fields in AtoM at the moment. To my knowledge there has not been development or review of the SKOS import/export in AtoM for several major releases.

I did not see a place on the page you provided where TGN terms are made available in a format that AtoM can ingest. They may be out there however, and/or there may be conversion services or utilities you can use to prepare the data. Perhaps others on this list might know?

If you do get a copy, I suggest importing via the command-line. The bulk import command has an option to specify a taxonomy for import - you will need to know the taxonomy IDs used internally in AtoM, which are listed here: https://github.com/artefactual/atom/blob/HEAD/lib/model/QubitTaxonomy.php#L20

Some of the key ones:

• Subject ID: 35
• Place ID: 42
• Genre taxonomy ID: 78

Note that the Genre taxonomy has been added in 2.2, but via the user interface it is currently only available to be added as an access point on the RAD template. We are hoping that an institution will sponsor its addition to RAD and DACS in a future version.

You will find more information on the --taxonomy option for the import:bulk command here:

• https://www.accesstomemory.org/en/docs/2.2/admin-manual/maintenance/cli-tools/#bulk-import-of-xml-files

Frankly, it is worth considering that the better option, rather than spending a lot of time trying to find the right data set (or converting to a different format), troubleshooting the import, etc - only to likely never use 2/3rds of the terms you import as access points, would be to consider how development might resolve this issue in the long-term.

The Getty Vocabularies, as well as other relevant controlled vocabs like LCSH, VIAF, GeoNames, and others, are now all made available as linked data services. I think the most sustainable long-term approach would be to develop relevant AtoM taxonomy fields so they could be used either to input/create local terms, OR to look up and fetch terms from relevant controlled vocabs such as the above. This would mean that users could draw directly from these vocabularies, and only pull in terms when they are using them, rather than having to recreate the entire vocabulary locally in AtoM. I believe that similar solutions have already been implemented in CollectiveAccess, ArchivesSpace, and EADitor, and I really hope that members of the AtoM community will consider developing, or sponsoring the development of, such a solution in the near future.

Not the answer you were expecting or hoping for, but since I am speaking to a developer, I had to at least put the idea out there! :)

Should you find yourself unable to move forward and development is not an option, the simplest workaround might be to recreate the terms locally as you need them. Your team can look up Getty vocab terms as they are needed, create them as access points, and then supplement those via the Terms edit page with source and scope notes, alternate labels, etc. Obviously not an ideal solution, but a way forward until a better one can be found or crafted.

Cheers,

Dan Gillean, MAS, MLIS
AtoM Product Manager / Systems Analyst,
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at http://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/e17e36af-7b20-4081-b07e-14d530a2c03b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[prathmes...@gmail.com]

Hi Dan,

 Thanks a lot for your detailed response. It seems from the gist of it, that importing vocabularies from Getty and LOC into ATom isn't a straight forward task, plus it might need some extra develpoment. And from our Research it also seems that Getty vocab might need licensing and they also seem to not provide SKOS XML as an export format. Hence there will be need to use some format conversion tools with their data.

LCSH on other hand seems to provide SKOS XML format, which with patience and assistance can be imported into Atom. Plus, Library of Congress (LOC) makes  their vocabularies available for free . Thus we have decided,  that we are going to try and do some test case imports from LOC into our Atom instance to see how it plays with our data and how much work would be involved in Importing them.

You mentioned that one of the users have imported LCSH terms into Atom. Can you please me their contact information, so that I can get in touch with them. I would like to pick their brains, regarding the workflow they undertook and learn more about the tools they used. This will help us plan and scope our project better.

Best,
Prathmesh Mengane,
Database and Web Developer,
Dumbarton Oaks -  Harvard University.

[Dan Gillean]

Sure Prathmesh, I will follow up with you off-list.

Regards,

Dan Gillean, MAS, MLIS
AtoM Product Manager / Systems Analyst,
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/21d52f53-406d-402e-81bc-2fa329a8ce62%40googlegroups.com.

[ivan obradovic]

Hi Dan,
Reading your post I need to ask you about link data services. We are testing Atom for our services, and it would be of great importance to know is there a chance for Atom to be linked with controlled vocabs like VIAF, GeoNames the near future, or should we be looking for some other solution like developing that link for ourselves?
Ivan Obradović
senior librarian
National library of Serbia

[Dan Gillean]

Hi Ivan,

So far no one has sponsored development to add linked data lookups for available controlled vocabularies like VIAF, GeoNames, LCSH, etc.  We would LOVE to see this developed and added to public AtoM release however! Ideally, I think users should be able to either create their own local terms (as they can currently), or specify a vocabulary, and then look up and pull in terms. Ideally any related source and scope notes with the terms would also be pulled in. It would then be really interesting to add better browsing/management options, so users could facet terms to see the source, etc.

Artefactual would be very interested in developing this kind of enhancement - if you are interested in the possibility of us doing the work, please feel free to contact me off-list.

If you have developers and are considering implementing this on your own, please do consider contributing back the code to the public project! Artefactual takes on the maintenance of the feature that way, and will ensure it continues to work through successive releases.

If you are developing the feature yourself and intend to contribute the code back, I would encourage your developers to ask questions via the user forum, so we can offer guidance on implementation in a way that Artefactual will be willing and able to maintain going forward, and that is implemented in a way consistent with the current code base. We do have a few developer resources on our wiki as well - see:

• https://wiki.accesstomemory.org/Development#Development_resources

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/8eba21b3-4590-4389-aa63-bbf99bce8141%40googlegroups.com.

[ivan obradovic]

Hi Dan,
Thank you for this valuable information. I will contact you when i have more information.
All the best

Ivan Obradović
senior librarian
National library of Serbia

[Rachael Woody]

Hello Dan,

May I inquire as to whether someone has had the opportunity to develop linked data lookups for controlled vocabularies - specifically LCSH? I see this post is from 2015 and am hoping there's been development since then.

Thank you,

Rachael Woody

[Dan Gillean]

Hi Rachel, 

We are actually working on an estimate for a client now. I have no idea if they will move forward as we expect it to be a significant development project, but it is giving us the opportunity to flesh out the proposed implementation and come up with some clear numbers around this, which I can hopefully share with the broader community  - that way, it may be possible for multiple institutions to collaborate to bring it to fruition in the future if needed. 

PS: I've seen your other post and will be replying when I've had a chance to do a bit of testing myself. 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/2603c59f-5c44-4010-8cdb-d903a19ee98e%40googlegroups.com.

[Rachael Woody]

Thank you for the update, Dan. I have my fingers crossed for luck.

Rachael

Owner & Consultant

Rachael Cristine Consulting

(503)922-3402

rachaelcristine.com

Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/2603c59f-5c44-4010-8cdb-d903a19ee98e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/aCUFjgYaItA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.

To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZ%2Bo68yxhDBdw5MLge8FtkJQNCxkFgpd7ZzreF1iZ-Hc1A%40mail.gmail.com.

[Alexandra Healey]

Hello,

A few years on and wondering if there has been any further movement on linked data lookups for controlled vocabularies? I notice linked data support features in the proof of concept for Atom 3.

Many thanks,

Alexandra Healey

Project Archivist

Newcastle University Special Collections

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/2603c59f-5c44-4010-8cdb-d903a19ee98e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/aCUFjgYaItA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.

To post to this group, send email to ica-ato...@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

[Dan Gillean]

Hi Alexandra, 

While we continue to believe that linked data has an important place in the future of both AtoM and archival description in general, at this time no support for linked data lookups has been added to AtoM. Adding such support, based on our current business model, would require community sponsorship for us to be able to implement, and so far no institutions have considered this enough of a priority to fund. You can learn more about the history of the AtoM project and Artefactual's current business model and development philosophy for AtoM via this wiki page: 

• https://wiki.accesstomemory.org/Development/Philosophy
  

Our vision of AtoM3 has certainly changed since we released the proof of concept report, but we remain committed to creating a next generation version of AtoM, and in the absence of major grants or venture capital, are considering how best to transform our business model to be able to create such a project while keeping the end result free and open source. Linked data will certainly continue to play a role in that future, though it will likely still be some time before we have a production-ready version to share with our community. 

In the meantime, if your institution is interested in potentially sponsoring controlled vocabulary lookups in the current version of AtoM, please feel free to contact Artefactual off-list and we would be happy to discuss options and next steps with you. 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/9b3e2a50-0874-4d0f-b5ad-e5d19358b3b8n%40googlegroups.com.