Search Populate Timed out

58 views
Skip to first unread message

Marcelo Moro Brondani

unread,
Oct 25, 2021, 3:46:58 PM10/25/21
to AtoM Users
Hello guys!

I'm migrating an AtoM 2.5 (Docker) to a 2.6.4 using Docker.
Hardware Requirements 8 CPU Cores and 32 GB RAM.

Following the documentation, in the "Search:populate" command step two errors occurred:
1st - You didn't change the version in the settings;
2nd - Operation timed out (PHP Fatal Error). (image attachment)

I ran the command only to index "InformationObject", where it indexed 964/2985 in 22 minutes.

Has anyone ever experienced this? How can I solve this case?

elasticsearch-timeout.png

José Raddaoui

unread,
Oct 25, 2021, 5:20:31 PM10/25/21
to AtoM Users
Hi Marcelo,

From the logs, it looks like the connection to Elasticsearch is down, you could check if the container is down with:

docker-compose ps

And get more details from its log:

docker-compose logs -f elasticsearch

I hope that gives more information about the issue.

Best regards,
Radda.

Dan Gillean

unread,
Oct 26, 2021, 9:04:04 AM10/26/21
to ICA-AtoM Users
Hi Marcelo, 

Just a quick addition to Radda's suggestions, re: 

1st - You didn't change the version in the settings;

The Vagrant and Docker boxes are development environments that pull from the QA branch of our code repository. We do not formally bump the software version until a public release is created, so what you are seeing is not unexpected. The second number in Admin > Settings > Global > Version is the database schema version - every time a new or updated feature requires a change to the database, we add a schema migration so that the underlying data model will be updated correspondingly when users upgrade. 

Currently I believe that the development branch should be at schema v192 - so the version in settings should say 2.6.0 - v192 right now. For reference, the 2.6.4 stable public release was 2.6.4 - v184

Sorry for the confusion! 

Once we have more information about your Elasticsearch index, we can hopefully provide further suggestions. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/e82d21b8-dc63-4c38-acc5-fcd9bb1fcc61n%40googlegroups.com.

Marcelo Moro Brondani

unread,
Oct 26, 2021, 10:26:08 AM10/26/21
to AtoM Users
José Raddaoui.

No log in elasticsearch container.
What I have observed is that the timeout only occurs for the InformationObject type. I changed the timeout and memory for PHP and it didn't work.
Another point is the running time of 22 minutes to index almost 3000 records and right after returning the timeout.

Any other way to resolve this?

Marcelo Moro Brondani

unread,
Oct 26, 2021, 11:07:02 AM10/26/21
to AtoM Users
Hello Dan!

Even not finishing the indexing (I don't know if it has any influence), the AtoM is using the source of the stable/2.6.x branch shouldn't it have changed the version? version 2.5.3 still persists.

How to proceed with this?

Dan Gillean

unread,
Oct 27, 2021, 10:30:11 AM10/27/21
to ICA-AtoM Users
Hi Marcelo, 

My apologies, I misunderstood the issue you were describing and the target version. 

If you are trying to upgrade in place, there may be issues as we have upgraded the MySQL version from 5.7 to 8.0 between versions 2.5 and 2.6. The easiest solution will be to install a newer version of the Docker Compose environment and migrate your data. 

If you have done this, then it's likely that you will need to perform some of the maintenance tasks associated with the upgrade process to ensure that your AtoM 2.5.3 / MySQL 5.7 data can be properly loaded into your 2.6/MySQL 8.0 database.

The commands and locations will be slightly different in the Docker environment (e.g. they will generally be preceded by docker compose exec etc), and I do not regularly use this environment so I can't provide the exact commands in all cases, but I have shared a very similar process recently for upgrading the Vagrant environment here that may be useful as a general guide: 
You can also review the Upgrading instructions in our documentation here: 
I think that you will need to: 
  • Ensure you have a backup of any data you want to keep as a sqldump outside of AtoM
  • Use the purge command to clear your data
  • Drop and recreate the database
  • Load your sqldump
  • Run the upgrade task
  • Restart services and then try to repopulate the search index
After this, the version should be reflected properly in the settings, and your data should load properly. Hopefully this might help with the ES error you have encountered as well. 

If you don't need to preserve any data (since the Docker environment is intended for development, and not for use in production), then hopefully the process should be much simpler. 

Let us know how it goes and what you find! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

Reply all
Reply to author
Forward
0 new messages