Import Error

97 views
Skip to first unread message

PANG

unread,
Nov 9, 2021, 9:39:46 PM11/9/21
to AtoM Users
Hi all, 

When I try to import a csv file with archival descriptions for files in a series in a fond, I get this job report:

[info] [2021-05-26 12:17:39] Job 903 "arFileImportJob": Job started.
[info] [2021-05-26 12:17:39] Job 903 "arFileImportJob": Importing CSV file: TEMPLATE_AtoMCatalogueUpload.csv.
[info] [2021-05-26 12:17:39] Job 903 "arFileImportJob": Indexing imported records.
[info] [2021-05-26 12:17:39] Job 903 "arFileImportJob": Update type: import-as-new
[info] [2021-05-26 12:17:40] Job 903 "arFileImportJob": php '/mnt/st-atom/westkowloon/atom-5a13306a63a75a0a7c88d1daf245018e78488a5b/symfony' 'csv:import' --index --quiet --user-id="545" --source-name='TEMPLATE_AtoMCatalogueUpload.csv' '/mnt/st-atom/westkowloon/atom-5a13306a63a75a0a7c88d1daf245018e78488a5b/uploads/tmp/TMPdd3e67e7.csv'; Term descriptionStatus not found in description status taxonomy, creating it...; Publication status: 'publicationStatus' is invalid. Using default.; Publication status: 'The publication status of the catalogue entry: Draft or Published' is invalid. Using default.; Invalid language: English
[info] [2021-05-26 12:17:40] Job 903 "arFileImportJob": Job finished.

I just upload template for testing, but the upload status is error. Could someone know how to upload these files? Is this issue relevant to 500-error?

Best,
Pang

Dan Gillean

unread,
Nov 10, 2021, 12:38:20 PM11/10/21
to ICA-AtoM Users
Hi Pang, 

I think that most of the information you need is found in this output, and can be clarified by looking at the documentation and other related resources for preparing an AtoM CSV for import. See for example: 
Additionally, if you're looking to run a test import that will help you better understand how the AtoM CSV templates are organized, I recommend using the templates we provide with AtoM, which include two sample records, and list the name and standard number of each ISAD-related field in the ISAD template, for example. Importing this is a great way to see a visual crosswalk from the related standard, to AtoM CSV template, and through to how it displays in the AtoM user interface. We have copies on our wiki, and there are also copies built into AtoM in lib/task/import/examples as well. See: 

Looking at the specific messages in the output: 

First, as a general point, without seeing your import CSV (or knowing what version of AtoM you are using, and if you are using the correct CSV template for that version, etc) it's hard to tell, but in some cases it looks like the header row values are being imported as metadata. This may be because you added values like "descriptionStatus" and "publicationStatus" into the first import row below the CSV headers - but if not, it may be that there are encoding issues in your CSV that are affecting how the rows and columns are parsed. 

AtoM expects CSVs for import to be UTF-8 encoded, and that they use unix-stye line ending characters (which are invisible characters in the CSV that tell applications where one row ends and another one begins). If you have used a spreadsheet application like Microsoft Excel to prepare your import, it's possible that it is not being saved with the correct line-endings and character encodings. See: 
Getting more specific: 

Term descriptionStatus not found in description status taxonomy, creating it...; 

This is not an error, but a warning. Description statuses are maintained in one of AtoM's controlled vocabularies - called Taxonomies in the user interface and documentation. The default terms included in this taxonomy are Final, Revised, and Draft - you can see these by looking in the Description statuses taxonomy (via Manage > Taxonomies > Description statuses), or else by looking at the values in the related "Status" field found in the Description Control area of an ISAD(G) archival description edit template: 

descriptionStatus.png

This is a user-editable taxonomy, meaning that an administrator can add more values to the taxonomy, and delete or modify the default values. The import job output here is just letting you know that the value in your import row (descriptionStatus) does not match any of these existing values, so a new term is being created in the related taxonomy as part of the import. 

Publication status: 'publicationStatus' is invalid. Using default.; Publication status: 'The publication status of the catalogue entry: Draft or Published' is invalid. Using default.;   

Again, this is more of a warning than an error, because once again AtoM has a default behavior when unexpected values are found (or no values are included) in the publicationStatus CSV column. 

This field also expects controlled vocabulary terms, but in this case it is not tied to a user editable taxonomy. As both an internal edit site for archivists and a public facing access catalog, AtoM uses the publication status to determine what descriptions are visible to public users, versus those that are only shown to authenticated (i.e. logged in) users. In a CSV import, this column expects one of two controlled terms: 
  • Published: The description will be visible to public users and logged in staff
  • Draft: The description will only be visible to logged in staff
For general information on using the Publication status in AtoM, see: 
This has made me realize it's not actually mentioned clearly on our CSV import documentation, so I will try to add a clarification about this here in the future: 
Fortunately, AtoM has a fallback behavior in the import code when no publication status value is included for a particular import row, or else the value is not one of the two expected ones (Draft or Published). In that case, AtoM will use the default publication status that an Administrator can configure in the settings, found in Admin > Settings > Global - see: 
In new installations, this setting is configured to "Draft" as the default publication status, to avoid accidentally exposing records that may not be intended for public viewing. 


Invalid language: English

Here is likely the source of the error that halted the import. All language and culture fields in our CSV templates expect ISO 639-1 two-letter language codes as inputs. For example, en for English; fr for French; es for Spanish, etc. You can see a full list of supported languages and their related codes here: 
This is mentioned in the CSV import documentation here:
Since AtoM can't know what language your metadata is in, and AtoM itself is designed to support multilingual content, there is no default fallback behavior here when encountering unexpected results. I suspect that if you change the value in your CSV row from English to en the CSV would import. 

I hope this helps!

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/66db1e2e-ce98-4da5-8f67-1fbe3145ecb6n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages