populate error

[Isabel Martín]

Good morning, I have installed, I believe that with the successful version 2.5 of AtoM on my production server.

Re-generating the search indexes with php symfony search: populate gives me the following error:

PHP Fatal error:  Allowed memory size of 2147483648 bytes exhausted 
(tried to allocate 6298816 bytes) in 
/usr/share/nginx/atom/vendor/elastica/Elastica/Bulk.php on line 328

Fatal
 error: Allowed memory size of 2147483648 bytes exhausted (tried to 
allocate 6298816 bytes) in 
/usr/share/nginx/atom/vendor/elastica/Elastica/Bulk.php on line 328

I have extended the memory_limit to 2048 Mb. I do not know if it will be necessary to expand more, since in the future it will continue to grow. Greetins.

[Dan Gillean]

Hi Isabel, 

I think that with 2GB you should be okay for a while. Please note that in our requirements documentation, we recommend 7GB RAM for a medium sized application: 

For a frame of reference, Artefactual’s standard AtoM test/demo site deployment is a cloud VM with the following specifications:

• Processor: 2 vCPUs @ 2.3GHz

• Memory: 7GB

• Disk space (processing): 50GB at a minimum for AtoM’s core stack plus more storage would be required for supporting any substantial number of digital objects.

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/caa8fa8a-32dd-4e57-bc15-4912af613ff4%40googlegroups.com.

[Isabel Martín]

Good morning Dan,

I had configured 2048Mb in my php.ini, but still I kept giving error. Finally I located them at 4096 Mb and I was able to index everything. Now I turn it back to 2048.

My server has 8Gb of RAM, several Terabytes of space and the Intel (R) Xeon (R) Gold 6148 CPU @ 2.40GHz. I do not know what the problem could have been.

On the other hand now I have another error that does not show me the descriptions, when clicking on a file background it gives me the following message in the error.log:

2019/05/27 09:26:18 [error] 10274#10274: *229 FastCGI sent in stderr: "PHP message: Unknown record property "digitalObjectCount" on "QubitInformationObject"" while 
reading response header from upstream, client: 10.1.26.28, server: _, request: "GET /index.php/ayuntamiento-de-alcolea-del-rio-2 HTTP/1.1", 
upstream: "fastcgi://unix:/run/php7.0-fpm.atom.sock:", host: "10.1.255.224", referrer: "http://10.1.255.224/index.php/informationobject/browse"

can you help me? Thanks you very much!!

To unsubscribe from this group and stop receiving emails from it, send an email to ica-ato...@googlegroups.com.

[Dan Gillean]

Hi Isabel, 

If you are able to keep 4GB of memory assigned in your php.ini file, I think you will have a better experience in the long term. 

In the meantime, regarding your recent error: 

Have you recently loaded legacy data into your new installation?  If yes, did you remember to drop and recreate the database, run the upgrade task, and restart services, etc? It sounds to me like the database is not at the current schema version, and is missing expected tables. It might be worth trying to repeat the final steps of the upgrade process. 

For example,you could try  purging your site of data, then dropping and re-creating the database - but remember that any existing data will be lost, so make sure you make a backup first! To purge all data: 

• php symfony tools:purge 

At this point, AtoM will prompt to you to add a new site title, and basic administrator account. You can then drop and recreate the database before running the upgrade task. Be sure to swap in your database username and password in the command below: 

• mysql -u username -pPASSWORD -e 'drop database atom; create database atom character set utf8 collate utf8_unicode_ci;'
  

Now you could re-add any data backups you have, and then try running the upgrade task, to see if it will add any missing tables: 

• php symfony tools:upgrade-sql

Restart services, repopulate the index, etc: 

• sudo systemctl restart php7.2-fpm
• sudo systemctl restart memcached
• php symfony cc
• php symfony search:populate
• make -C plugins/arDominionPlugin

Let us know if that helps! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/6057b287-1565-4f4e-9f41-99bc7896f88d%40googlegroups.com.

[Isabel Martín]

Hello Dan, I have followed the steps suggested to me:

- Purge the data.
- Delete and re-create the atom database.
- Restore the copy of the data.
- Update with upgrade-sql
- Restart all services.

I keep giving the same error.

I have seen that you include an order to restart version 7.2 of PHP. I have the 7.0 installed. Can it be problematic?

Thanks and best regards

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/6057b287-1565-4f4e-9f41-99bc7896f88d%40googlegroups.com.

[Dan Gillean]

Hi Isabel, 

This is strange. The property listed in the error message, digitalObjectCount, is only used in the AIP table in AtoM, for storing parsed METS data when receiving content from Archivematica. It shouldn't normally be affecting results on the browse page and causing the error you are seeing... so currently our team is not sure what to suggest! 

My current theories are either: 

1) At some point during a previous upgrade, the sql-upgrade task was not run, or did not execute properly, meaning there is a missed migration. This can be very tricky to resolve if it is the case! 

2) There is some corruption in your database that is causing this unexpected result. 

Can you please run the following task and tell me the output?

• php symfony tools:get-version

I'm specifically interested in checking the database schema version (the vXXX number included in the returned output). For the 2.5 release, I believe this should be v172. If it's not, then we may need to try to figure out where the schema migrations were missed, and manually apply the necessary fixes. I've described the process previously here: 

• https://groups.google.com/d/msg/ica-atom-users/G368lgN6T1Y/VTfIEyqbAgAJ
  
• See also: https://groups.google.com/d/msg/ica-atom-users/wJvYASwBVEI/kKKxv7_TAwAJ

However, before we go down that rabbit hole, let's see if the issue is related to data corruption. 

For resolving data corruption, we'll start by running a couple maintenance tasks to see if it helps - specifically, let's regenerate all of your slugs, and rebuild the nested set. We'll clear the cache and repopulate the index after as well. 

• php symfony propel:generate-slugs --delete
• php symfony propel:build-nested-set

• php symfony cc
• php symfony search:populate

If that doesn't help, then I would suggest that you try running the query suggested here, to check for other common forms of data corruption. See: 

• https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/troubleshooting/#troubleshooting-data-corruption  
  

Let us know what you find and how it goes! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/b8adf28b-b73d-473e-82d6-f209e2ec49cd%40googlegroups.com.

[Dan Gillean]

Also one more question / theory:

Does your AtoM installation have any customizations? Special plugins, custom themes, changes to the code, etc? It's possible this is occurring due to a local customization. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

[Isabel Martín]

Hello Dan, I did all the tests that you indicated to me and it was not a problem of the database or the installation.

Indeed, I have a custom theme where I have overwritten some files, and with the update there is some method that does not exist and I was giving error.

I have corrected it and it works. Anyway, I learned a lot about error correction. Thank you so much for everything.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/b8adf28b-b73d-473e-82d6-f209e2ec49cd%40googlegroups.com.

[Dan Gillean]

Aha! 

I'm so glad that you figured it out, Isabel. Thank you for updating us and letting us know what you found. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/60fb07d3-0324-440f-960d-06c940fb39d3%40googlegroups.com.