When generating finding aid from a record with 10 description(2250 descendants), atom-worker failed.

[Cindy]

Hello,

When generating finding aid from a record with 10 description, 2250 descendants, atom-worker failed.

We got following errors in job report:

[info] [2019-07-11 14:04:36] Job 162507 "arFindingAidJob": Job started.

[info] [2019-07-11 14:04:36] Job 162507 "arFindingAidJob": Generating finding aid (*****-fonds)...
[info] [2019-07-11 14:06:11] Job 162507 "arFindingAidJob": Exporting EAD has failed.
[info] [2019-07-11 14:06:11] Job 162507 "arFindingAidJob": ERROR(EAD-EXPORT): PHP Warning: count(): Parameter must be an array or an object that implements Countable in /usr/share/nginx/atom/plugins/sfEadPlugin/modules/sfEadPlugin/templates/indexSuccessBodyDidElement.xml.php on line 137

-----

atom-worker error logs:

Jul 11 14:06:41 php[28236]: 2019-07-11 14:06:41 > ERROR(EAD-EXPORT): PHP Warning: count(): Parameter must be an array or an object that implements Countable in /usr/share/nginx/atom/plugins/sfEadPlugin/modules/sfEadPlugin/templates/indexSuccessBodyDidElement.xml.php on line 137

Jul 11 14:06:41 php[28236]: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'output' at row 1 Jul 11 14:06:41 php[28236]: 22001 Jul 11 14:06:41 systemd[1]: atom-worker.service: control process exited, code=exited status=1 Jul 11 14:06:41 systemd[1]: Unit atom-worker.service entered failed state. Jul 11 14:06:41 systemd[1]: atom-worker.service failed.

No error in nginx error log.

We upgraded to 2.5 two weeks ago, and MySQL 5.7, PHP 7.2.

Any help is appreciated.

Thank you.

Cindy

[José Raddaoui]

Hi Cindy,

It looks like you're facing two different issues:

PHP Warning:  count(): Parameter must be an array or an object that implements Countable

This warning is thrown since PHP 7.2, I'm not sure if it's causing any problem in the finding aid generation or it's actually just a warning, so I've created this ticket to investigate.

String data, right truncated: 1406 Data too long for column 'output' at row 1

This has been recently reported in this thread and we're also investigating the issue and planing to include a fix in a 2.5.2 release. In the meantime, you could try running the MySQL query suggested in that thread and restarting the AtoM worker, to confirm that the previous warning is not a problem and see if you can successfully end the finding aid generation.

Best regards.

[Cindy]

Hi José,

I just wanted to update you on this I raised in July.

We upgraded to 2.5.2, when generating the same finding aid, we got same errors:

php[1522]: > ERROR(EAD-EXPORT): PHP Warning:  count(): Parameter must be an array or an object that implements Countable in /usr/share/nginx/atom/plugins/sfEadPlugin/modules/sfEadPlugin/templates/indexSuccessBodyDidElement.xml.php on line 137

php[1522]: > Exception: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'output' at row 1

After ran ALTER TABLE job MODIFY output MEDIUMTEXT, still getting warnings, 'Data too long' error was gone, but got memory error:

php[1738]: > ERROR(EAD-EXPORT): PHP Warning:  count(): Parameter must be an array or an object that implements Countable in /usr/share/nginx/atom/plugins/sfEadPlugin/modules/sfEadPlugin/templates/indexSuccessBodyDidElement.xml.php on line 137

php[1738]: > ERROR(EAD-EXPORT): PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 32768 bytes) in /usr/share/nginx/atom/plugins/sfEadPlugin/modules/sfEadPlugin/templates/indexSuccessBodyDidElement.xml.php on line 167

 php[1738]: > ERROR(EAD-EXPORT): PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 937984 bytes) in Unknown on line 0

Then I increased memory_limit from 512M to 2048M( tried 1024M, had same error), the finding aid was generated successfully.

 

Our database size doesn't seem to increase after alerted the job table:

Before:

SELECT table_schema "database", sum(data_length + index_length)/1024/1024 "size in MB" FROM information_schema.TABLES GROUP BY table_schema;

+--------------------+--------------+

| database           | size in MB   |

+--------------------+--------------+

| atom               | 386.92187500 |

ALTER TABLE job MODIFY output MEDIUMTEXT;

Query OK, 3677 rows affected, 1 warning (0.42 sec)

Records: 3677  Duplicates: 0  Warnings: 1

After:

SELECT table_schema "database", sum(data_length + index_length)/1024/1024 "size in MB" FROM information_schema.TABLES GROUP BY table_schema;

+--------------------+--------------+

| database           | size in MB   |

+--------------------+--------------+

| atom               | 386.92187500 |

We are now testing on our Staging site.

Thanks for your help.

Cindy

[José Raddaoui]

Hi Cindy,

Thank you very much for getting back and posting your experience and solutions! We really appreciate that as it will most likely help other users facing the same issue.

I took a quick look to the warnings and they shouldn't be a problem for this workflow. We'll try to address them in following releases alongside the database column issue. The size of the database won't increase right after the change, it would only be an issue after saving several verbose jobs output.

Best regards.