Failed attempts to export large units as csv files

200 views
Skip to first unread message

Efthymis Kakavogiannis

unread,
Aug 28, 2021, 12:17:50 AM8/28/21
to AtoM Users
Dear all,

I am rather a newbie in AtoM and specifically in working with .csv files. 

I have been working without a hitch so far, but now I need to remove some objects from an AtoM file in order to avoid erroneous numbering in the link of the entries, i.e. if I delete the entry with the number 27 and rename then the next one to 27 for instance, its weblink will still be named with the number 28, which might be confusing to the final user.

As far as I can see, the only way to change the numbering of the entries properly is to export the entire unit in question as a csv file including all descendant levels of description and then to make any changes needed, which is exactly what I am not allowed to do. Either I get Exception: Unknown record property "authorizedFormOfName" on "QubitStaticPage" or Export complete - no records were exported. 

Do you have any ideas on what causes this error and what should I do to avoid it? 

Thanks in advance,

Efthymis

Dan Gillean

unread,
Aug 30, 2021, 10:10:22 AM8/30/21
to ICA-AtoM Users
Hi  Efthymis, 

Welcome to the AtoM community! 

First, if you have command-line access to your installation, there is a much easier method to update the slug or permalink (the unique part of the URL/weblink) so it matches your edited title. There is a command-line task to generate slugs. By default it will only generate slugs where they are missing for some reason, but the task also has a --delete option. When this option is used, all slugs will be generated again - therefore, so long as you are using the default slug generation settings (that use the title or authorized form of name for slug creation), then this should fix the issue. 

If you have never worked with the command-line interface betore, we have a 101 deck on the basics here: 
You will find many other cheat sheets, guides, and tutorials online. 

You will want to run the AtoM slug generation task from the root AtoM installation directory - if you have followed our recommended installation instructions, this is typically /usr/share/nginx/atom. 
After running the task, we will also want to clear all caches, to make sure we are seeing the updated content and not an older cached version. First, clear the Symfony cache: 
PHP-FPM also has its own cache, so let's clear that as well. The exact commands depend on the version of PHP you are using - assuming you have AtoM 2.6 with PHP 7.2 installed on Ubuntu 18.04, run: 
Finally, we will want to reindex the site, so the search index matches our new slugs. Run: 
Don't forget that you might want to clear your web browser cache, or else test in a private or incognito browser window, to ensure you are seeing the changes. 

In terms of the error message you are seeing: 

At a glance, I suspect there may be an issue with the CSV - perhaps the incorrect file encoding, for example. No part of an import should be attempting to update a static page. Are you by chance using MS Excel to prepare your CSV files? Microsoft is notorious for using its own custom character encodings rather than the de facto standard of UTF-8, which AtoM expects. Later versions of Excel seem to have better support for UTF-8 CSV files, but we still hear reports of bad imports due to its use. 

The character encoding and line-ending requirements for AtoM are noted in our documentation here:
Instead of Excel, I recommend that you try to install LibreOffice Calc and use that instead when preparing CSVs for import into AtoM. It's not as user-friendly or as nice looking as Excel, but it is free open source software, and the control it gives you over character encoding (and delimiter / separator characters) is far greater - you can set this before you open any CSV: 

calc-csv-options.png

If you are trying to update records via CSV import, be aware that there are some limitations - for example, many fields (any fields in the data model that are part of different tables, such as notes) cannot be updated via CSV import, only added to. For more information on how it works and how to prepare a CSV for update import, see: 
Another important note. The matching criteria during an update import is currently very difficult to properly use when roundtripping a CSV in the same system. However, the command-line task for CSV import has a --roundtrip option that makes matching much easier when you export a CSV, make changes, and then want to re-import it as an update. See: 
Finally, in general, we STRONGLY recommend that you create backups of your data before attempting any large-scale changes, so you can roll back to a previous save point if things don't go as expected! See: 
Good luck!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/4ae44c32-bef9-4b2c-9209-ca99e9a33b76n%40googlegroups.com.

Simon (der_smon)

unread,
Dec 15, 2023, 10:09:41 AM12/15/23
to AtoM Users
Hi! I am currently running also into the "Unknown record property "authorizedFormOfName" on "QubitStaticPage" when I try to export all data using the command line. Did you find a solution?

Dan Gillean

unread,
Dec 15, 2023, 11:46:25 AM12/15/23
to ica-ato...@googlegroups.com
Hi Simon, 

First of all, running some of the common troubleshooting maintenance tasks recommended above in the previous posts in this thread, and/or in the following thread linked below, will be a good idea to resolve common issues: 
As in, it would be good to start by trying the following: 
  • Rebuild the nested set
  • Generate slugs (without any options, so it will only replace any missing slugs)
  • Clear the application cache
  • Restart PHP-FPM
However, it sounds to might like you might be encountering some data corruption - something that can occur when a long-running process in AtoM (such a CSV import, move operation, deletion of a large hierarchy, etc) is interrupted or times out before it can complete properly, leaving incomplete rows in the database that can cause unexpected issues later. Another possibility is that a key step was skipped during the upgrade process, which can mean that the database schema is out of sync with the current AtoM version. 

First, what is the FULL version number of your installation, as shown in Admin > Settings (or when you run php symfony tools:get-version on the command-line)? the second number after the dash is the database schema version - for 2.7.x releases, this should be v192. For 2.6.x releases, this should be v184.

Did you follow the recommended installation instructions for your version - which I believe from the other thread is a 2.6 installation, yes? i.e. Ubuntu 18.04, MySQL 8.0, PHP 7.2, Elasticsearch 5.6, etc (for 2.6)? If no, what changes have you made locally?

Two steps in the upgrade process that often seem to be missed: 
  • Dropping and then recreating the MySQL database BEFORE loading your sqldump from the previous version
  • Running the upgrade task AFTER loading your sqldump
Can you confirm that these steps were definitely done during the last upgrade? Upgrade instructions are here: 
You could try re-running the upgrade task - this shouldn't cause any issues even if you did it during the upgrade process: 
  • php -d memory_limit=-1 symfony tools:upgrade-sql
Finally, if nothing above has helped to uncover a cause, then we do have a SQL query that you can try that checks for common forms of data corruption. See: 
We always recommend making a database backup before accessing MySQL directly - see: 
There are also some suggestions there on what to do to resolve issues, depending on what you find. 

Let us know how it goes and what you find!

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/5f84fa7b-22fd-4542-b4f3-810ca06b3015n%40googlegroups.com.

Simon (der_smon)

unread,
Dec 18, 2023, 7:26:23 AM12/18/23
to AtoM Users
I did not reimport the database, but ran all the maintenance tasks and the error persists.

I hacked a print_r($this) into https://github.com/artefactual/atom/blob/c9ec8f8bbf00cee78da1115d3c384298db5b5631/lib/model/om/BaseStaticPage.php#L153 and got this (and then a lot of recursions, so I had to stop the export):

 [keys:protected] => Array
(
[id] => 7
)

[row:protected] => Array
(
[CLASS_NAME] => QubitStaticPage
[0] => QubitStaticPage
[CREATED_AT] => 2016-06-06 06:36:53
[1] => 2016-06-06 06:36:53
[UPDATED_AT] => 2022-03-31 00:07:25
[2] => 2022-03-31 00:07:25
[ID] => 7
[3] => 7
[SERIAL_NUMBER] => 0
[4] => 0
)

Simon (der_smon)

unread,
Dec 18, 2023, 7:27:53 AM12/18/23
to AtoM Users
Now I am testing to return an empty array for the case and see what happens. :D

Simon (der_smon)

unread,
Dec 18, 2023, 7:45:48 AM12/18/23
to AtoM Users
Is that just `<my url>/index.php/7`? How could I find the culprit?

The export succeeded now.
Simon (der_smon) schrieb am Montag, 18. Dezember 2023 um 13:26:23 UTC+1:

Dan Gillean

unread,
Dec 18, 2023, 9:15:51 AM12/18/23
to ica-ato...@googlegroups.com
Hi Simon, 

I am a bit confused as to where you're at and what you're asking - are you still in need of support, or did you manage to resolve the issue? What is the full version number of your installation? Does your installation follow our recommended installation instructions for its version, or have you made any changes (and if so, what)? 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/46ec51f1-ce40-4cf6-bef6-56796c34482fn%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages