Axiell CALM and AtoM

170 views
Skip to first unread message

markrya...@gmail.com

unread,
Apr 12, 2017, 5:58:29 AM4/12/17
to AtoM Users
Does anybody have any experience of transferring archival descriptions from an existing Axiell CALM catalogue to AtoM?

MacGregor, Rachel

unread,
Apr 12, 2017, 6:09:21 AM4/12/17
to ica-ato...@googlegroups.com

I have been experimenting with this with varying degrees of success.  Is there anything in particular you were wondering about – I myself had some questions about the process but am still in the testing phase!

 

Best wishes,

Rachel

 

 

Rachel MacGregor
Digital Archivist

 

The Library

Lancaster University

Bailrigg
Lancaster
LA1 4YH


+44 (0) 1524 595 229

@An_Old_Hand

http://www.lancaster.ac.uk/iss/info/images/twitter.png Follow us on Twitter
http://www.lancaster.ac.uk/iss/info/images/facebook.jpg Find us on Facebook

http://www.lancaster.ac.uk/iss/info/images/ShieldLogo.png
www.lancaster.ac.uk/library

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/f38fe43c-10ad-4f3d-85d0-c86cd7c350f7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Mark Smith

unread,
Apr 12, 2017, 6:22:21 AM4/12/17
to ica-ato...@googlegroups.com
Hi Rachel,

Thanks for replying.  I didn't have any particular questions really, but just wanted to see if anybody had been trying to do this, and how they got on.  I work at the archives up in Shetland and have been looking at digital preservation stuff for the past six months or so.  At the moment, Archivematica is a very appealing piece of software for doing digital preservation, and the fact that it syncs with AtoM is making me wonder if running the two packages together might be the way ahead for us.  We have a CALM catalogue at the moment, with online access via Calmview, but I have some doubts over CALM's capabilities with digital objects.  With the artefactual systems, we could preserve digital things and easily make them available online.  The fact that they don't cost anything is also pretty appealing.

The stumbling block, though, would be getting our data into AtoM.  We spent about two years manually entering text from old Word lists into CALM, but if the process of getting it into AtoM could be done in a less painful way, that would be very welcome indeed.

Would be good to hear about any experience you've had with this.

Mark

On Wed, Apr 12, 2017 at 11:09 AM, MacGregor, Rachel <r.mac...@lancaster.ac.uk> wrote:

I have been experimenting with this with varying degrees of success.  Is there anything in particular you were wondering about – I myself had some questions about the process but am still in the testing phase!

 

Best wishes,

Rachel

 

 

Rachel MacGregor
Digital Archivist

 

The Library

Lancaster University

Bailrigg
Lancaster
LA1 4YH


+44 (0) 1524 595 229

@An_Old_Hand

http://www.lancaster.ac.uk/iss/info/images/twitter.png Follow us on Twitter
http://www.lancaster.ac.uk/iss/info/images/facebook.jpg Find us on Facebook

http://www.lancaster.ac.uk/iss/info/images/ShieldLogo.png
www.lancaster.ac.uk/library

 

 

 

From: ica-atom-users@googlegroups.com [mailto:ica-atom-users@googlegroups.com] On Behalf Of markrya...@gmail.com
Sent: 12 April 2017 10:58
To: AtoM Users
Subject: [ica-atom-users] Axiell CALM and AtoM

 

Does anybody have any experience of transferring archival descriptions from an existing Axiell CALM catalogue to AtoM?

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/5we2sKmwcZ4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/A9B63EB12FD2C44A9D2BA908908C5D5037750562%40EX-0-MB1.lancs.local.

MacGregor, Rachel

unread,
Apr 12, 2017, 6:46:00 AM4/12/17
to ica-ato...@googlegroups.com

Hi Mark,

 

I only have “relatively” small amounts of data to work with – c1500 catalogue entries – so I have been experimenting with exporting the data as a csv and then using the csv templates provided for AtoM to import again:

 

https://www.accesstomemory.org/en/docs/2.3/user-manual/import-export/csv-import/#legacyid-and-parentid

 

I only have one collection to deal with, which simplifies things, as the one thing I have struggled with is trying to link the catalogue to a pre-existing authority file.  I think a certain amount of tweaking is always going to be inevitable but the results were encouraging.

 

What I have not yet done is test out a command line import which is suggested for more than 100 items.

 

My understanding is that other institutions have attempted a larger scale import of records from other systems (including CALM) and have in some cases employed consultants to tidy the data and import.  Hopefully someone on the forum will chip in at this point with their experiences!

 

I would bear in mind that the “don’t cost anything” doesn’t factor in your/developer time getting things up and running and keeping things running! 

 

Best wishes,

 

Rachel

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/5we2sKmwcZ4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

Mark Smith

unread,
Apr 12, 2017, 8:35:06 AM4/12/17
to ica-ato...@googlegroups.com
Thanks Rachel.  That's very helpful.  We're going to install AtoM and Archivematica so we can test them out.  I'll play around with the transfer of data and hopefully the going will be smooth.
Mark

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/5we2sKmwcZ4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/5we2sKmwcZ4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/A9B63EB12FD2C44A9D2BA908908C5D5037750601%40EX-0-MB1.lancs.local.

Dan Gillean

unread,
Apr 12, 2017, 1:03:16 PM4/12/17
to ICA-AtoM Users
Hi Mark and Rachel,

I have not yet been involved directly in a data migration project from CALM to AtoM, but I know it has been successfully done. I am happy to share some general advice on how we approach data migrations, as the process tends to be similar regardless of the source application. I will also mention an explicit resource that might be of interest to you at the end.

Whenever Artefactual performs data migrations from legacy systems, we first look at what export formats are available from the source system. Our preferred format to work with is CSV, so if CALM supports a CSV export, this will make things easier. Barring that, we have worked with any number of data formats, but the more customized or proprietary, the more work you might have to do to engage a developer who can prepare a transformation script to help clean up the data into a format that AtoM understands.

Essentially, AtoM (and ICA-AtoM) expect a specific format for CSV imports - you can find our templates on the AtoM wiki, here:

Because our CSV schema is specific to AtoM, this is usually the most reliable way to move data without loss. The trick will be getting your source data into a format that matches the AtoM format.

We generally make use of at least 2 different approaches when assisting our clients prepare for a data migration. The first is for our analysts to work with the client directly, or provide some training so they can do the work themselves, and then use a tool to clean up the data manually. We have found OpenRefine to be a powerful tool to support this work:

OpenRefine (formerly Google Refine) is a powerful tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data.

There are a great many free resources (books, tutorials, webinars) that can help get you started. You can use this to make bulk edits, normalize data (e.g. make sure that your authority record name is spelled the same throughout or change the "Car" subject access term someone added to match the existing "Automobiles" term that should have been used, etc), combine columns or split data into separate columns, and the like.

For more extreme interventions, we will have one of our developers prepare a transformation script - we will analyze the data with the client, work out the correct mappings, write these out for our developers, and then they will prepare a simple script that will take the source data as input and output it in a format that conforms with AtoM's expectations. We generally will import, and then review the resulting data with both our team members (to look for problems in mappings, or weirdness in how AtoM fields have been used, etc) and with the client (who will always know the data better than we do). We will compile a list of outstanding issues, and then depending on these, we will either revise the transformation script and re-import, or if the changes are minor, we can either clean it up directly in AtoM or via manipulation of the SQL back-end directly.

A lot of the initial work will be mapping work. If your resources in CALM were described using a known content standard (such as ISAD(G), Dublin Core, MODS, etc), then you will have a much easier time. I have previously posted a sort of overview of what the work likes, and some things to consider, in this thread:

The trickier parts are when you have custom fields or non standardized data, and you then need to make some decisions based on the available fields in the target content standard (e.g. ISAD, etc). In some cases this might mean choosing to let some data go, or deciding to concatenate multiple custom fields into a single ISAD field, etc. How you proceed will be up to you.

One useful resource that might help you both, if you are interested:

May 9-11, 2017, we will be running our first UK AtoM Camp in Cambridge, hosted by St. John's College. There will be a session there on import/export and on migrations, and we will have a community counselor joining us for the migration session - Pete Vox of IMAGIZ, who helped lead a CALM --> AtoM migration and custom theming project for the Museum of Croydon. Pete will share his experiences and gotchas and be available to offer advice specific to CALM in a way that we can't. We will also have an opportunity to demonstrate how we like to use OpenRefine, and we'll discuss the steps I've outlined above in greater detail.

This might be a great opportunity to connect with other AtoM users, or potential AtoM users, as well as with local service providers who can assist you if you end up finding you need this. You can find more information, including cost, schedule, and registration information, on our wiki, here:

Hope this helps!

Final point to mention: our 2.4 version will have support for large imports via the user interface - so you could have a CSV import with many rows without needing access to the command-line to perform it. There are also some basic ways to use imports to update or replace existing descriptions - we'll be giving our Camp attendees a bit of a sneak peek of some of this. You can see everything that's coming in the 2.4 version on our Roadmap, here:

Mark, if you do intend to do some local testing and don't want to do a full server install, you might consider checking out our Vagrant box - see:

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/5we2sKmwcZ4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.

To post to this group, send email to ica-ato...@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/5we2sKmwcZ4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

Mark Smith

unread,
Apr 13, 2017, 4:22:11 AM4/13/17
to ica-ato...@googlegroups.com
Hi Dan,

Thanks very much for this.  It's exactly the kind of info I was looking for.  As I was saying to Rachel, we haven't yet made a decision on whether to go with AtoM or stick with CALM, but the more I find out about how data transfer would work (and I'm pretty sure CALM does support CSV exports), the better able we'll be to decide what we'd like to do.  Moving our catalogue records into a new system is probably the biggest issue for us, and some colleagues are understandably edgy about losing or corrupting a catalogue that has taken the best part of a decade to build up. 

At the moment we're in the process of setting up some new cloud servers and we're going to install AtoM and Archivematica so we can give them a really good trial.  I've played around with the demo versions on the product websites and found both pretty easy to get to grips with in a basic way.  I'm sure to have more questions, though, so will probably be posting them on here.  

Thanks again for your help.  It's very encouraging to get this insight from Artefactual.

Mark

To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZ%2BgTPAeMA-Kf89HXES%2B_nFbmG4LXvoNas3EYPXYwGS%3DMg%40mail.gmail.com.

MacGregor, Rachel

unread,
Apr 13, 2017, 6:18:25 AM4/13/17
to ica-ato...@googlegroups.com

Hi,

 

Can I echo Mark’s thanks.  Tidying up the data is going to be the hard part for me so I am looking forward to trying the OpenRefine software as suggested.

 

Cheers

 

Rachel

 

From: ica-ato...@googlegroups.com [mailto:ica-ato...@googlegroups.com] On Behalf Of Mark Smith
Sent: 13 April 2017 09:22
To: ica-ato...@googlegroups.com
Subject: Re: [ica-atom-users] Axiell CALM and AtoM

 

Hi Dan,

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/5we2sKmwcZ4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.


To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/5we2sKmwcZ4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.


To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.


To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/5we2sKmwcZ4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.


To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.


For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.


To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

Reply all
Reply to author
Forward
0 new messages