Dublin Core CSV-import from Archivematica

106 views
Skip to first unread message

th...@arkivsormland.se

unread,
Mar 5, 2021, 11:39:37 AM3/5/21
to AtoM Users
Hi!

I just did a test upload from Archivematica to AtoM containing a image and a metadata.csv file containing all 15 dublin core elements. It work just fine exept that the metadata from dc.type, dc.relation and dc.language does not show up in AtoM. I have switch display standard from ISAD(G) to Dublin Core to MODS but no luck. In ISAD(G) and Dublin Core 12 och 15 imported elements are visible, and in MODS only 10... How come? I attached the csv-file just in case something is wrong with it.

Regards,
Theo
metadata.csv

Dan Gillean

unread,
Mar 16, 2021, 10:04:23 AM3/16/21
to ICA-AtoM Users
Hi Theo, 

Sorry for the delay in replying. It may be that you've found a bug, but I will try to confirm that. Are you willing to share the METS file that was included in the DIP, so we can see how the Dublin Core metadata was represented there? Feel free to send off list if preferred, and/or send only the revelant section with the descriptive metadata. This should help us determine if the problem is in how the CSV data gets added to the METS file, or if the issue is in how AToM parses the METS file. Thanks in advance!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/6560039c-db55-4f1b-9d54-b37f2c9cf851n%40googlegroups.com.

th...@arkivsormland.se

unread,
Mar 16, 2021, 10:56:35 AM3/16/21
to AtoM Users
Hi, I sent the METS file off list. As you can see I it's just a test upload. Under dc.rights I have put in the word "rights" and so on to make it easy to see "what goes where".  The reason being that I wanted to test if all dc elements "make it through" from Archivematica to AtoM or not, and if so, how they get mapped to other standards than Dublin Core in AtoM. Since Dublin Core is the only standard (as far as I know) that can be used to send metadata directly from Archivematica to AtoM it feels important to be able to use all 15 elements, especially since it's a rather limited set to begin with.

All the best,
Theo

Dan Gillean

unread,
Mar 17, 2021, 2:46:29 PM3/17/21
to ICA-AtoM Users
Hi Theo, 

After talking this over with some of the Archivematica team, as far as we can tell the CSV headings are correct and the DC metadata is getting added to the METS file as expected. This suggests that there's some issue in how AtoM is incorporating the metadata during DIP upload. 

I've found the related code that handles the mapping into AtoM, here: 
Of the fields you've mentioned: 
  • dc.type: It appears on lines 346-356 that AtoM is attempting to match incoming metadata to an existing list of taxonomy terms. AtoM *should* just create a new term if no match is found, but we have found bugs in the past where new metadata that doesn't match existing terms is skipped, so that might be what's happening here. One way to test this theory would be to reimport it using one of the default terms. The related taxonomy in AtoM that holds the default terms is Dublin Core Types. 
  • dc.relation: I don't actually see a mapping for this element in the code. The reason for this may be the somewhat unorthodox mapping decision that was made in AtoM early on. Essentially, simple DC elements do not include a way to associate a description with a repository. Early AtoM developers decided to use the isLocatedAt relation type available in DC Terms as a way of capturing this information - though on its own without the isLocatedAt terms qualification, dc.relation is intended to capture information about a related resource, preferably as a URI. 
  • dc.language: This one is also looking for a match against a controlled vocabulary - specifically, it is looking for ISO two-letter language codes (such as en, fr, de, es, etc). This field uses a controlled list provided by the underlying PHP framework Symfony, so it's not possible for users to add values. I'd be curious to see if it works with an expected value such as "en" rather than a test entry in the metadata.csv
I'm not sure there's an easy solution we can provide for the dc.relation issue, but I'm hopeful that the type and language elements will work with expected values, and that we can just file a bug report for new values entered into dc.type not creating new terms in the related taxonomy on import. 

I don't personally have time to test this right now, but I wonder if you would mind testing again and reporting back, with "Collection" or "image" in the dc.type field, and "en" or "sv" or similar in the dc.language field, and let us know if that works as expected?

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

Theo Erbenius

unread,
Mar 19, 2021, 11:04:12 AM3/19/21
to ica-ato...@googlegroups.com

Thank you for helping us with this problem. 


I have tried some new uploads, and have concluded that dc.language works just fine with en and sv type of data, but it's also possible to write English (but en is far better since it allows the language data to be expressed in more than one language). 


But dc.type is more troublesome. I tried to add all Dublin Core type values (Collection, text, image, sound, dataset, software, interactive, event, physical object) in different dc.type columns, but only the values text and dataset made it into AtoM. To make it more complicated these values are only visible if the display standard is set to Dublin Core, but not when set to ISAD(G) or MODS. This is a problem since we find these standards more useful since they can express metadata in a more granular way than Dublin Core. 


I don't know if it helps, but I will send you both the csv and METS files of list. 


Best,

Theo


Dan Gillean

unread,
Mar 24, 2021, 9:59:12 AM3/24/21
to ICA-AtoM Users
Hi Theo, 

Regarding the DC type values and AtoM's templates: 

Since this is a controlled vocabulary specific to Dublin Core, there's not really a way to map this across to other standards-based templates. I would recommend using the Genre access points instead, which are customizable, and can be browsed by public users. Note that the DC type values are still present on the record even when not visible, but since AtoM's UI also doesn't offer an easy way to search or filter by these, they still provide less value to end users than something designed as an access point if you're using a different template anyway. 

In any case, it's strange that only a couple of the terms are importing properly, and that does sound like a bug. I will try to reproduce it and file a ticket for a future release. Thanks for letting us know! 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

Theo Erbenius

unread,
Mar 29, 2021, 9:49:09 AM3/29/21
to ica-ato...@googlegroups.com
Ok, thanks for your reply! 

Is it possible to add info to the Genre access points through a metadata.csv file that comes from Archivematica? I can see that dc.subject maps to Subject access points, and that dc.coverage maps to Place access points, but I have not managed to send anything to the access points for Genre and Name. The reason I'm testing with a Dublin Core csv file is that it's the only way I'm aware of for sending metadata directly from Archivematica to AtoM. Am I missing something? Or is it only possible (if possible at all) to send information to the Genre and Name access points through a ISAD(G) metadata.csv file that is sent directly to AtoM?

Regards,
Theo




--
Theo Erbenius
E-arkivarie
076-319 41 14

Dan Gillean

unread,
Mar 29, 2021, 10:05:09 AM3/29/21
to ICA-AtoM Users
Hi Theo, 

Unfortunately, I don't think these mappings are currently supported for Archivematica metadata imports to AtoM. It will require development to add these - plus some analysis, since there's not always a direct mapping in DC for these elements! 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

Theo Erbenius

unread,
Mar 30, 2021, 2:06:08 AM3/30/21
to ica-ato...@googlegroups.com
Ok, thanks for the clarification!

Regards,
Theo

Reply all
Reply to author
Forward
0 new messages