Digital object load confusion

[Josh Smith]

Hello all,

Please forgive me for being overly wordy with this post. I'm trying to figure out how to use the CLI digital object upload tool. TL;DR I want to bulk upload digital objects to a particular collection, but I don't understand what identifier or information object id to use in order to do this.

I've been researching the option of using Digital Object Load task, the CLI tool to load digital objects, and I'm confused. According to what I've read here in the group and in the documentation, an archival description can only have one digital object attached to it. Here is the quote from the documentation: "AtoM does not allow more than one digital object per information object (with the exception of derivatives), and each digital object must have a corresponding information object to describe it, so this one-to-one relationship must be respected in the CSV import file."

However, in the web UI when I click the button to import digital objects to an archival description, the page I get is called "Import multiple digital objects" and I can indeed upload multiple digital objects. Here is a screenshot where I have uploaded multiple PDF digital objects to the collection 'Mooring Mast' as items:

To me it appears what happens is AtoM creates archival descriptions for each digital object I upload under the Mooring Mast collection so in effect each digital object does therefore have a one-to-one relationship with an archival description. In the CLI tool I'm required to provide either an identifier or information object id for the archival description to which I'm uploading a digital object. If I want to upload multiple digital objects to one collection, what identifier or information object id do I use?

Thanks and please let me know if I'm not making sense so I can clarify.

TL;DR I want to bulk upload digital objects to a particular collection, but I don't understand what identifier or information object id to use in order to do this.

[Dan Gillean]

Hi Josh, 

You're right about the 1:1 relationship between a digital object and an information object (aka description) in AtoM's data model. That's why the "upload digital objects" option creates new lower-level descriptions for each upload. 

The command-line digital object load task is used to attach digital objects to existing descriptions in AtoM. If you've already created the item-level descriptions, then this would be a good way to proceed. In that case, I recommend using the information object ID values - identifiers are rarely unique throughout an entire AtoM installation, so it's possible that you'd end up sending your digital object to the wrong description following that method. I've long wanted us to add the ability to use a slug value instead, but we'll need community sponsorship to add that enhancement in the future. 

One useful trick for getting the object ID values without SQL: 

Add the parent collection to the clipboard, and then export the descriptions with all descendants. On export, AtoM will populate the legacyID column with the information object IDs. You can then use those values in your 2-column digital object load task import. 

If you are trying to import new descriptions with new digital objects attached, then I recommend you use the CSV import. The AtoM description CSV templates can include 2 columns related to digital objects - a digitalObjectPath column (used for linking local objects) and a digitalObjectURI column (used for linking objects available on the public web. See: 

• https://www.accesstomemory.org/docs/latest/user-manual/import-export/csv-import/#digital-object-related-import-columns  
  

So in your case, you would add a directory of images somewhere on your AtoM server, and then add the path to each object in the relevant row with the metadata you want to use for the corresponding description. AtoM will create the description, and then fetch and attach the corresponding object. 

You can use the qubitParentSlug column to make these new descriptions import as children of an existing collection - just populate that column with the slug of the parent collection. See: 

• https://www.accesstomemory.org/docs/latest/user-manual/import-export/csv-import/#qubitparentslug  
  

Let me know if that clears things up a bit!

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/74cd145d-ebac-4257-8414-d79bc997ae48%40googlegroups.com.

[Josh Smith]

Thanks for the reply, Dan! This is very helpful. Let me describe my situation and what I think would be a good workflow based on what you've said.

We have lots of images we need to add to AtoM and we have a spreadsheet with metadata from which I could pull a CSV with whatever fields are required. (We also have lots of PDFs to add, but I think the same process will work). We have a collection in AtoM to which we'd liketo add the images.

Based on what you've said, it sounds like we should use CSV import to create new descriptions with our images attached as digital objects. We're using DC for metadata so we'll use the ISAD template (FYI the AtoM 2.3 ISAD CSV template on the wiki page does not have a digitalObjectURI column)

The CSV will need to have:

   -a digitalObjectPath column or a digitalObjectURI column depending on whether our images are local or online
   - the qubitParentSlug column to add our images to an existing collection
   - the metadata we'd like to add to the archival descriptions in the appropriate ISAD-DC cross-walked columns

Did I get it correct?

Thanks,

Josh

To unsubscribe from this group and stop receiving emails from it, send an email to ica-ato...@googlegroups.com.

[Dan Gillean]

Hi Josh, 

You've got that right! 

I'll have to double-check to make sure, but the CSV examples typically do include a digitalObjectURI column, but not a digitalObjectPath column. That's mostly just because we populate the templates with example data that can be used to test imports, and while we can provide a link to a digital object online that will work during a test import, we can't provide a local file path! However, if you need this column, you can either edit the header on the digitalObjectURI column, or just insert a new column. If both are missing, feel free to add what you need!

A couple reminders as you prepare: 

• AtoM expects all CSV files to be UTF-8 encoded with proper unix-style line endings. For this reason, we typically do NOT recommend using Microsoft Excel to prepare your CSV. Microsoft loves to use its own non-standard character sets and line endings, which can cause import issues in AtoM. Instead, we recommend using something like the open source LibreOffice Calc program to modify your CSV. This free program allows you to check and set the character encoding, separators, and field limiters every time you open a CSV, and it will use the expected line endings. Some further details in our CSV import documentation here. 

• Because all of your images will be in a single folder, make sure the filenames are unique. Otherwise AtoM will import the first matching digital object it finds as it processes the import CSV.

• If you try to use the digitalObjectPath and digitalObjectURI columns for the same description, AtoM will favor the URI! Make sure you are only adding one value to these two columns per row.

Here's an example workflow: 

1. Place all your uniquely named images into a new folder called images
2. In the AtoM CSV template, add your metadata as described in our documentation
3. Modify the name of the digitalObjectURI column to: digitalObjectPath
4. In this column, add the basic relative path to the relevant image for each row of data - for example, images/image-01.jpg 
5. When you are done, save the CSV file, and then add the images directory to your AtoM server, just below the root AtoM installation directory
6. We generally always recommend making a backup of your data before running imports in a production environment - do so now!
7. Run your import!

One last point - I actually can't remember off the top of my head if AtoM prefers relative paths (e.g. images/image-01.jpg) or absolute paths (e.g. /usr/share/nginx/atom/images/image-01.jpg) for digital objects in the CSV, so you might want to try a small test first to make sure everything above works as expected!

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/1086abb7-f988-4194-bb65-c5867e4b6f23%40googlegroups.com.

[Josh Smith]

Thanks very much, Dan. I ran a few test batches and was able to get everything uploaded with metadata. This is great!

One last question: since AtoM is making a copy of the digital object to /usr/share/nginx/atom/uploads, I can remove my uploaded files from the images directory you had me create once the job is complete, right?

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/1086abb7-f988-4194-bb65-c5867e4b6f23%40googlegroups.com.

[Dan Gillean]

That's correct! You don't need the images directory or its contents once you've successfully uploaded your images. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/b94b3311-e971-4f79-8a95-916cf509d6c9%40googlegroups.com.