error message

[Susana Lourenço]

Elastic search is giving an error message " Elasticsearch error: Elastica\Exception\Connection\HttpException" when trying to access links in AtoM ?

Thank you 

[Dan Gillean]

Hi Susana, 

First, whenever reporting an issue and seeking assistance in the user forum, it's helpful for us to know more about your specific installation - this helps us determine how we can try to reproduce the issue locally under the same conditions, as well as whether or not it's a bug in a specific release, an issue with your particular installation, or a user error. Things that are useful to share include: 

• What is the full AtoM version number or your installation, as found in Admin > Settings?
• Did you follow exactly the recommended installation instructions for your version? If no, what changes have you made?
• Are you a custom theme, and/or does your installation have any other local code customizations?
• If not, are you using the older default Bootstrap 2 Dominion theme, or the new Bootstrap 5 Dominion theme included in 2.7 and later?
• Does your AtoM installation meet the recommended minimum hardware requirements listed here? Most importantly, do  you have 7GB or more of memory?
• Do you get this Elasticsearch error message when clicking on all links, or just some (like just description links, or just some from a specific collection, etc)? 
• Are there any important previous actions that might have caused this worth mentioning? For example, did you upgrade recently? Did someone attempt an import? When and why, to the best of your knowledge, did this start happening? 
• What, if anything, have you tried so far to investigate and/or resolve the issue? And what happened?

Please provide as much information in response as you can, as it will allow us to offer better suggestions for resolving your issue. In the meantime, here are some things you can try: 

The following commands should all be run in the command-line interface, starting from AtoM's root installation directory - which, if you followed our recommended installation instructions, is typically located at /usr/share/nginx/atom. 

First, let's use the command line task to get more information about your search index. Please share the output of this task: 

• php symfony search:status
• See: https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/cli-tools/#check-the-status-of-your-elasticsearch-index

We can also repopulate the search index, which often resolves issues with it. Keep in mind that this task will temporarily make records in your public-facing catalog disappear, until the search index is rebuilt, which will take anywhere from minutes to hours depending on the size of your collection. For this reason, we often recommend that public-facing production installations run this task outside of normal business hours. We will also clear the various caches used in the application, to ensure we are seeing the latest results. 

First however, let's run a few other maintenance tasks that can often resolve many common issues, and won't do any harm if they are not related to the problem. These tasks all run much faster, so there is less need to run them after business hours. 

To begin, let's make sure that all records have slugs - that is, the unique part of the URL related to a record, sometimes called a permalink. Occasionally if something times out mid-process, you can end up with records that have no slug, which can cause errors. We can generate any missing slugs with: 

• php symfony propel:generate-slugs
• See: https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/cli-tools/#generate-slugs

We'll also rebuild the nested set, which is an internal model used to organize hierarchical relations (like those found in an archival collection) in the relatively flat, table-like structure of a relational database. As with the slugs, sometimes when a long-running process is interrupted, the nested set can become corrupted - but fortunately in AtoM it is quick and easy to rebuild: 

• php symfony propel:build-nested-set
• See: https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/cli-tools/#rebuild-the-nested-set

Finally, let's clear all caches that AtoM uses. These are stored versions of pages users visit, which can then be reloaded and served again much faster than having to fetch all the data from the database. However, sometimes what is in the cache does not accurately represent the current state of our source of truth, the database, so it's always a good idea to flush these before testing. First, there is the application cache: 

• php symfony cc
• See: https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/clear-cache/

Additionally, PHP-FPM has its own cache. This command depends on what version of PHP you have installed. If you are using AtoM 2.7 and followed our recommended installation instructions, this will be PHP 7.4. If not, check the documentation for your version. 

• sudo systemctl restart php7.4-fpm
• See: https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/troubleshooting/#restarting-php-fpm 

Finally, be aware that your web browser has its own cache! It can be a good idea to clear this before testing, or else do your testing in a private / incognito browser window, where the cache is typically disabled by default. 

Now we are ready to try re-populating the search index, which hopefully will resolve the issue. To repopulate the search index: 

• php symfony search:populate
• See: https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/populate-search-index/

If you encounter an error when trying to run this task, there are a few more things we can still do. 

First, Elasticsearch has its own logs we can look at, to see if there is more information about the error. You will typically find these at: 

• /var/log/elasticsearch/elasticsearch.log
  

Please share any relevant error messages you find in them

We can try restarting the Elasticsearch service, and manually deleting the search index before running the repopulate command again. Here are the commands to control the ES service: 

• sudo systemctl stop elasticsearch
• sudo systemctl enable elasticsearch
• sudo systemctl start elasticsearch
• sudo systemctl status elasticsearch
• sudo systemctl restart elasticsearch

You can use the status command to see if it's running, and then try stopping it. While stopped, we can manually delete the current index using cURL:

• curl -XDELETE 'localhost:9200/atom'
  

Note - if you get an error because cURL is not installed (i.e. a command not found or similar error), you may need to install it first. On Ubuntu, you can do this with: 

• sudo apt update
• sudo apt install curl

Now we can restart the Elasticsearch service, and then see if the search population will work this time: 

• sudo systemctl start elasticsearch
• php symfony search:populate

If nothing in this message has helped to resolve your issue, then hopefully we can provide further suggestions once you have given us more information about your AtoM installation and the error you're seeing. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

On Fri, Sep 15, 2023 at 10:51 AM Susana Lourenço <susanal...@gmail.com> wrote:

Elastic search is giving an error message " Elasticsearch error: Elastica\Exception\Connection\HttpException" when trying to access links in AtoM ?

Thank you 

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/d2f4b1eb-124e-42af-8e85-86fea596e0can%40googlegroups.com.

[Ndivhuwo Phaduli]

• What is the full AtoM version number or your installation, as found in Admin > Settings? ATOM Version 2.5.1
• Did you follow exactly the recommended installation instructions for your version? If no, what changes have you made? Yes
• Are you a custom theme, and/or does your installation have any other local code customizations? Yes

• • If not, are you using the older default Bootstrap 2 Dominion theme, or the new Bootstrap 5 Dominion theme included in 2.7 and later?

• Does your AtoM installation meet the recommended minimum hardware requirements listed here? Most importantly, do  you have 7GB or more of memory? Yes
• Do you get this Elasticsearch error message when clicking on all links, or just some (like just description links, or just some from a specific collection, etc)? Yes " Elasticsearch error: Elastica\Exception\Connection\HttpException  " 
• Are there any important previous actions that might have caused this worth mentioning? For example, did you upgrade recently? Did someone attempt an import? When and why, to the best of your knowledge, did this start happening? No
• What, if anything, have you tried so far to investigate and/or resolve the issue? And what happened? error on ElasticSearch 

• " Elasticsearch error: Elastica\Exception\Connection\HttpException""

• zldacapatompro1:/var/www/html/atom # php symfony search:status
  PHP Warning:  session_name(): Cannot change session name when headers already sent in /var/www/html/atom/vendor/symfony/lib/storage/sfSessionStorage.class.php on line 71
  PHP Warning:  session_set_cookie_params(): Cannot change session cookie parameters when headers already sent in /var/www/html/atom/vendor/symfony/lib/storage/sfSessionStorage.class.php on line 87
  PHP Warning:  session_start(): Cannot start session when headers already sent in /var/www/html/atom/vendor/symfony/lib/storage/sfSessionStorage.class.php on line 96
  PHP Warning:  ini_set(): Headers already sent. You cannot change the session module's ini settings at this time in /var/www/html/atom/vendor/symfony/lib/user/sfBasicSecurityUser.class.php on line 258
  Elasticsearch server information:
   - Host: 10.125.205.219
   - port: 9200
   - Index name: atomprod
  
  Document indexing status:
  PHP Fatal error:  Uncaught TypeError: Argument 1 passed to Elastica\Exception\ResponseException::__construct() must be an instance of Elastica\Request, string given, called in /var/www/html/atom/plugins/arElasticSearchPlugin/lib/arElasticSearchPlugin.class.php on line 128 and defined in /var/www/html/atom/vendor/elastica/Elastica/Exception/ResponseException.php:30
  Stack trace:
  #0 /var/www/html/atom/plugins/arElasticSearchPlugin/lib/arElasticSearchPlugin.class.php(128): Elastica\Exception\ResponseException->__construct('Unexpected resp...')
  #1 /var/www/html/atom/plugins/arElasticSearchPlugin/lib/arElasticSearchPlugin.class.php(147): arElasticSearchPlugin->getVersion()
  #2 /var/www/html/atom/plugins/arElasticSearchPlugin/lib/arElasticSearchPlugin.class.php(93): arElasticSearchPlugin->checkVersion()
  #3 /var/www/html/atom/lib/search/QubitSearch.class.php(39): arElasticSearchPlugin->__construct(Array)
  #4 /var/www/html/atom/lib/task/search/arSearchStatusTask.class.php(74): QubitSearch::getInstance()
  #5 /var/www/html/atom/lib/task/sea in /var/www/html/atom/vendor/elastica/Elastica/Exception/ResponseException.php on line 30
  zldacapatompro1:/var/www/html/atom #
  
• The error above happens 

[Dan Gillean]

Hi Ndivhuwo, 

A couple more questions: 

First, you say you followed our installation instructions exactly... but it appears you are using an Apache web server, based on the /var/www/html/atom installation path in your shared error messages (which would be /usr/share/nginx/atom if you were using Nginx as recommended), so I just want to double check: are there any other changes you have in place? 

Can you please tell me the PHP version, Operating System version, Elasticsearch version, and MySQL versions you used with this installation?

Next, it would be helpful to have the FULL version number of your installation. If you navigate to Admin > Settings, you will see that there are two numbers - the first is the AtoM version, and the second is the database schema version. Checking this number will help me determine if this might be caused by an upgrade problem or a data corruption issue. 

Finally, you also said yes to having a custom theme. Does this custom theme include any functional changes? Do you know if these steps were done last time you upgraded, to ensure your custom theme works as expected?

• https://www.accesstomemory.org/docs/2.5/admin-manual/installation/upgrading/#upgrading-with-a-custom-theme-plugin
  

As for next steps: First, keep in mind that the suggestions I will be offering are based on our recommended installation instructions, so if there are many changes in your local version they may not work as expected. 

To start, I would suggest that you try resetting the filesystem permissions. AtoM expects all files and directories below the root installation directory to be owned by the www-data user. You can reset the permissions for this like so: 

• sudo chown -R www-data:www-data /usr/share/nginx/atom
  

Now going forward, for all Symfony commands that you run (i.e. those tasks that begin with php symfony) You can specify the www-data user explicitly to make sure that the task is executed by a user with enough permissions to perform it. You can do this by adding sudo -u www-data as a prefix before your Symfony commands. For example, reindexing would look like this: 

• sudo -u www-data php symfony search:populate

From here I would suggest that you try pretty much everything I included in my previous message in this thread, in this order: 

• Generate any missing slugs
• sudo -u www-data php symfony propel:generate slugs
• Rebuild the nested set
• sudo -u www-data php symfony propel:build-nested set
• Clear the application cache
• sudo -u www-data php symfony cc
• Restart the elasticsearch service
• sudo systemctl restart elasticsearch
• Repopulate the search index
• sudo -u www-data php symfony search:populate

If any of those steps fail, please share the error message returned. You can then try: 

• Manually deleting the search index using cURL
• curl -XDELETE 'localhost:9200/atomprod'
• See the previous message above if you get an error about cURL not being installed, or Google instructions for installing it for your Linux version
• Checking the Elasticsearch error logs for more information
• sudo tail -f /var/log/elasticsearch/elasticsearch.log
• Repopulating the search index again (to see if it will work after manual deletion)
• sudo -u www-data php symfony search:populate

Let us know how it goes!

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/a5d7e7f8-1f16-4988-b38e-8c42937485edn%40googlegroups.com.

[tat...@gmail.com]

Hi Dan,

can you please check on this error on /var/log/elasticsearch/elasticsearch.log ???

[2024-07-03T19:33:24,941][DEBUG][o.e.a.s.TransportSearchAction] [uTr9Row] All shards failed for phase: [query]
org.elasticsearch.index.query.QueryShardException: No mapping found for [i18n.sr.title.alphasort] in order to sort on
        at org.elasticsearch.search.sort.FieldSortBuilder.build(FieldSortBuilder.java:262) ~[elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.search.sort.SortBuilder.buildSort(SortBuilder.java:156) ~[elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.search.SearchService.parseSource(SearchService.java:634) ~[elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.search.SearchService.createContext(SearchService.java:485) ~[elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:461) ~[elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257) ~[elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:343) ~[elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:340) ~[elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:662) [elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:675) [elasticsearch-5.6.16.jar:5.6.16]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.16.jar:5.6.16]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]

Thanks,

Tati

[Johan Pieterse]

Hi Tati

Did you check this post?

https://groups.google.com/g/ica-atom-users/c/hgKPiTvGChE/m/lfd66pJMNGgJ

Also did your elasticsearch crash/run out of space?

[tat...@gmail.com]

Hi Johan, thanks for your response.

I am not even close to get out of memory problems.

I checked out the log erros:
-mysql - nothing there
-elasticsearch - no errors as well
nginx: 
root@atom7-ufes:~# sudo cat /var/log/nginx/error.log
2024/07/11 00:59:06 [error] 1019#1019: *18724 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 1785856 bytes) in /usr/share/nginx/atom/plugins/sfEadPlugin/modules/sfEadPlugin/templates/indexSuccessBodyDidElement.xml.php on line 84PHP message: PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 1769472 bytes) in Unknown on line 0" while reading response header from upstream, client: 172.25.16.11, server: _, request: "GET /index.php/biblioteca-central-colecoes-especiais;ead?sf_format=xml HTTP/1.1", upstream: "fastcgi://unix:/run/php7.4-fpm.atom.sock:", host: "atom.ufes.br"

Any idea?

Thankyou!

Tati

[tat...@gmail.com]

Hi Johan,

I followed the solution described here: https://stackoverflow.com/questions/34864524/allowed-memory-size-of-536870912-bytes-exhausted-in-laravel

I checked the memory_limit in /etc/php/7.3/fpm/php.ini and in etc/php/7.3/cli/php.ini

In the first one was 128M and the second one was -1

So, I raised the firts one to 2048M and restarted php7.4-fpm

It seems okay now, I hope, kkk.

Thanks again.
Tati

[Johan Pieterse]

Good news 

The -1 flag is used to set to unlinited memory but that is not advised.

Johan