Metadata code is invalid

[Chad Julius]

All,

We recently upgraded to 2.6.4 and have been having issues with imports.  We are using the new templates for uploads and have been getting the following:

[info] [2022-01-10 07:51:52] Job 1048074 "arFileImportJob": Job started.
[info] [2022-01-10 07:51:52] Job 1048074 "arFileImportJob": Importing CSV file: DA002_503-600-ATOM_CSV.csv.
[info] [2022-01-10 07:51:52] Job 1048074 "arFileImportJob": Indexing imported records.
[info] [2022-01-10 07:51:52] Job 1048074 "arFileImportJob": Update type: import-as-new
[info] [2022-01-10 07:52:02] Job 1048074 "arFileImportJob": php '/usr/share/nginx/atom/symfony' 'csv:import' --index     --quiet --user-id="442" --source-name='DA002_503-600-ATOM_CSV.csv'  '/usr/share/nginx/atom/uploads/tmp/TMP375af469.csv';   The metadata code "-FY-2002-1" is not valid.
[info] [2022-01-10 07:52:02] Job 1048074 "arFileImportJob": Job finished.

From the end user:

It says metadata code: "-FY-2002-1" is not valid is in the title field "Indian Health; FY 2002 (1)." There is no default language for this field so I'm not sure why it's telling me it's invalid. Technically its a "free" field meaning I can enter any data.

The template entry has the following:

Identifier: B504-F005.1
Title: Indian Health; FY 2002 (1)
Level of Description: Folder

Any help would be appreciated.

Chad

[Dan Gillean]

Hi Chad, 

You're right that AtoM will not typically parse or prevent any metadata you want to enter in a free-text field like the title field. Instead, I suspect that there is something malformed with your CSV (such as the wrong character encoding, line endings, collation, separator characters, etc) that is causing the title data to be parsed as something else.

AtoM expects CSV files to be UTF-8 encoded, using unix style line endings. See: 

• https://www.accesstomemory.org/docs/latest/user-manual/import-export/csv-import/#verify-character-encoding-and-line-endings
  

Did you by chance use MS Excel to prepare your CSV? Microsoft often uses custom character encodings and line endings by default, which can cause all sorts of problems in AtoM. 

I often recommend that people consider using LibreOffice Calc to prepare AtoM CSVs instead. Every time you open a CSV, you can set the character encoding, separator, and text delimiters used, and see a preview below to ensure the CSV is rendering as expected: 

You might want to try double-checking your CSV this way, to ensure it's actually formatted as you expect. Another way to do this is to open it in a text editor, but BE VERY CAREFUL editing it this way, as you can accidentally delete or alter structural elements like the line ending characters or separators! 

Let us know what you find and if this resolves the issue. If not, then please share the CSV with us if you're able, and I will see if I can reproduce the problem. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/6dcd7f2e-988d-4d05-8014-f97ffc7b4d05n%40googlegroups.com.

[Chad Julius]

We didn't have any luck yesterday with this.  I went ahead and rolled the server back to 2.5 and it works just fine.  Perhaps I missed something in the upgrade and am going to build that out on a new server. 

Thanks for the help.

[Dan Gillean]

Interesting; thanks for the update, Chad. I have a bit more context on this error message  I can provide you with, which may help with troubleshooting. 

AtoM has a shortcut that allows you to preview your records in a different metadata template without manually changing the default template, by adding a semi-colon and the short name of the related standard (e.g. isad, rad, dacs, dc, mods) to the end of the slug. 

For example, if your ISAD description is at: 

• https://www.example.com/example-fonds

Then you can preview it as Dublin Core by changing the URL to: 

• https://www.example.com/example-fonds;dc

You can see this illustrated in slides 9-10 of the following slide deck: 

• https://www.slideshare.net/accesstomemory/things-i-wish-id-known-atom-tips-tricks-and-gotchas
  

The error you are seeing is from the routing validator, which is trying to parse the content after the semicolon as a metadata standard. 

Normally, when creating the permalink, AtoM should sanitize the slug so there's no conflict here - even when you have the setting to Use any valid URI path segment and uppercase character in slugs enabled, the semicolon should be escaped and there should not be an error. 

In your case, this is not happening for some reason, which is what led to me making the suggestions in my previous post. I did run a quick test, importing a record using the title shown in your example, and another one with no space before the semicolon and following text. I also created records with semicolons in the title via the user interface. I performed both tests with the URI path setting both on and off, and in all cases it worked as expected for me in our development environment. This suggests to me that, unless there's a regression specifically in 2.6.x that we've already unknowingly resolved in the upcoming 2.7 release, then the problem is likely local, and may have to do with the CSV metadata itself. I didn't test on 2.5, so it's possible this is an older issue that has been resolved in 2.6, but I couldn't find any specific tickets relating to this in our issue tracker - but if you found the issue after upgrading to 2.6.4, then that seems unlikely as well. 

Did the import work as expected in 2.5 after you rolled back? If yes, then one workaround for now would be to finish your imports in 2.5, make a database backup, and then perform the upgrade. Doesn't help us identify and resolve the issue, but it should bypass the problem you're encountering for now. 

Another way you can end up with non-standard content in your CSV is if you've cut and pasted content into the CSV from a non UTF-8 source, such as a Word document. 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/fcfb81e8-d11f-47c1-aab9-fa5c0fb00029n%40googlegroups.com.