CSV import: match and update

83 views
Skip to first unread message

Vincent Decorges

unread,
Dec 14, 2021, 7:55:15 AM12/14/21
to AtoM Users
Hi,

I am trying to update the language field by doing a match and update CSV import but it is not working and the filed remain null in table property. Given the documentation, the field "language" is considered in the updating process: https://www.accesstomemory.org/en/docs/2.7/user-manual/import-export/csv-import/#csv-descriptions-update-fields

I am running AtoM 2.6.3

We have a lot of experience with the import update process but this is the first time we are tring to modified the "language" field.

I would like to know if this a know problem and if there is a workaround?

Thanks for your feedback.
Best regards

Dan Gillean

unread,
Dec 14, 2021, 4:54:05 PM12/14/21
to ICA-AtoM Users
Hi Vincent, 

Just so I'm clear on what the issue is: 
  • Is the import successfully finding the correct records to update, but the change is failing to be applied? Or
  • Is the matching failing, and the update never happens?
Have you been able to successfully update other fields using the same method, for example? Are you able to add language values via the user interface?

Once I better understand the issue, I will try to offer some suggestions. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/159d985a-5d97-4ff7-83ba-6a07122509aan%40googlegroups.com.

Vincent Decorges

unread,
Dec 20, 2021, 2:56:35 AM12/20/21
to AtoM Users
I am in the first case: Is the import successfully finding the correct records to update, but the change is failing to be applied?

Others fields are updated correctly, It is only language who is never updated.

Thanks
Vincent

Dan Gillean

unread,
Dec 20, 2021, 10:44:52 AM12/20/21
to ICA-AtoM Users
Ok, thanks for the information. I will do some local testing and see if I can reproduce the issue. 

Are you using the ISO 639-1 two-letter language codes for this data? Per the documentation in this section:

language and languageOfDescription, like culture, expect two-letter ISO 639-1 language code values - for example, “en” for English; “fr” for French, “it” for Italian, etc. See Wikipedia for a full list of ISO 639-1 language codes. Unlike the culture column, however, these two fields will accept multiple values separated by a pipe character - for example, en|fr|it.

We maintain a list of supported languages and their corresponding two-letter codes here: 
I'll let you know what I find during local testing. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

Dan Gillean

unread,
Dec 21, 2021, 11:37:14 AM12/21/21
to ICA-AtoM Users
Hi again Vincent, 

Unfortunately, I was not able to recreate the issue. I managed to export a description via CSV, change the language values, reimport it as an update, and see the updates in AtoM. Here's exactly what I did: 
  • Found a test description I had imported some time ago
  • Pinned it to the clipboard
  • Exported it
  • Unzipped the package, and opened it with LibreOffice Calc
  • Deleted some columns I did not need to update (any empty columns, scope and content, etc) to reduce the chance of any errors, BUT made sure that title, identifier, and repository were unchanged for the fallback matching logic
  • Changed the language value in the CSV from en to en|fr|it
  • Changed the script value from Latn to Copt
  • Updated a few other fields to test that updates in general are working as expected (locationOfOriginals; accessConditions)
  • Saved my changes and closed the CSV
  • Returned to AtoM and went to the Import CSV configuration page
  • Set it to: match and update skipping empty CSV columns; Skip unmatched; Limit matches to [selected the record in the top-level description autocomplete]
  • Ran the import; waited for it to complete, and checked the job log: match found, records updated in place
  • Navigated to the record and confirmed that the updated language values were in place, that the replacement script value was in place, and that the other test fields were also updated
Unless you are doing something radically different than this, I'm not sure what the issue is. However, I'll try to offer a few troubleshooting suggestions below.

First, what is the full version number of your AtoM installation shown in Admin > Settings > Global? I want to confirm that the database schema version is correct for 2.6.3. It should be v184 (this will be the second number shown). If it's not, then a step may have been skipped during the upgrade process (such as dropping and recreating the database before loading your backup, or running the upgrade task after loading it). 

You might also want to ensure that AtoM's MySQL database is using the expected SQL modes. We configure them at the end of this section of the installation documentation, and you can check and change them using the MySQL command-prompt as described here: 
Finally, it's always a good idea when there are strange unexplained behaviors to double-check for data corruption. We have some suggestions on how to do so in our Troubleshooting documentation: 
Let us know what you find. Hopefully we can sort out what's happening here! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

Vincent Decorges

unread,
Dec 29, 2021, 3:02:18 AM12/29/21
to AtoM Users
Thanks Dan for checking. We are running 2.6.3-184. I will make more checks and tests on my side. There should be a problem somewhere on my installation.

Best regards
Vincent
Reply all
Reply to author
Forward
0 new messages