Elasticsearch error: Elastica\Exception\ResponseException

[Hernán Carvajal]

Hello dear friends.

I'm having troubles to set Spanish as the default language in a site. This has AtoM version 2.6.1 - 184

Even though I follow the instructions that say:

The default language is normally configured during installation, but you can change it at any time in the following configuration file: 

• /apps/qubit/config/settings.yml

See: 

• https://www.accesstomemory.org/docs/2.7/admin-manual/customization/config-files/#apps-qubit-config-settings-yml
  

After making changes, you will need to clear the application cache and restart PHP-FPM

• Clear the application cache
• Restart PHP-FPM

You may also want to clear your web browser cache as well, to be sure you are seeing the most recent version of the page.

I have no success. You can check the site at https://archivopatrimonial.uahurtado.cl/ and see what is the default language.

That is one problem.

The second problem we are facing , presumibly related with the first one, is that the search engine is not working with the defacto default language of the site (english). If you try to show the Archival descriptions, or any other menu item, we are presented with the error:

Elasticsearch error: Elastica\Exception\ResponseException

These ar the logs from Nginx:

2023/11/14 12:37:00 [error] 1636#1636: *1846924 FastCGI sent in stderr: "PHP message: Empty module and/or action after parsing the URL "/ews/exchange/" (/)" while reading response header from upstream, client: XXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /ews/exchange/ HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "XXXXXXXXXXXX"
2023/11/14 12:37:02 [error] 1636#1636: *1846942 FastCGI sent in stderr: "PHP message: Empty module and/or action after parsing the URL "/ews/%20/" (/)" while reading response header from upstream, client: XXXXXXXXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /ews/%20/ HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "XXXXXXXXXXXXXXX"
2023/11/14 12:37:06 [error] 1636#1636: *1846948 FastCGI sent in stderr: "PHP message: Empty module and/or action after parsing the URL "/ews/ews/" (/)" while reading response header from upstream, client: XXXXXXXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /ews/ews/ HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "XXXXXXXXXXX"
2023/11/14 12:37:10 [error] 1636#1636: *1846966 FastCGI sent in stderr: "PHP message: Empty module and/or action after parsing the URL "/ews/autodiscovers/" (/)" while reading response header from upstream, client: XXXXXXXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /ews/autodiscovers/ HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "XXXXXXXXXXXXXXXX"
2023/11/14 12:37:14 [error] 1636#1636: *1846979 FastCGI sent in stderr: "PHP message: Empty module and/or action after parsing the URL "/autodiscover/autodiscovers/" (/)" while reading response header from upstream, client: XXXXXXXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /autodiscover/autodiscovers/ HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "XXXXXXXX"
2023/11/14 12:37:17 [error] 1636#1636: *1846986 FastCGI sent in stderr: "PHP message: Empty module and/or action after parsing the URL "/autodiscover/autodiscover%20/" (/)" while reading response header from upstream, client: XXXXXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /autodiscover/autodiscover%20/ HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "XXXXXXXXXXX"
2023/11/14 12:37:21 [error] 1636#1636: *1846998 FastCGI sent in stderr: "PHP message: Empty module and/or action after parsing the URL "/autodiscover/autodiscoverrs/" (/)" while reading response header from upstream, client: XXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /autodiscover/autodiscoverrs/ HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "XXXXXXXXXXXX"
2023/11/14 12:37:25 [error] 1636#1636: *1847033 FastCGI sent in stderr: "PHP message: Empty module and/or action after parsing the URL "/autodiscove/" (/)" while reading response header from upstream, client: XXXXXXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /autodiscove/ HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "XXXXXxXXXX"
2023/11/14 12:38:05 [error] 1636#1636: *1847125 FastCGI sent in stderr: "PHP message: This request has been forwarded to a 404 error page by the action "search/autocomplete"" while reading response header from upstream, client: XXXXXXXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /index.php/search/autocomplete?query=93-5663+&repos= HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "archivopatrimonial.uahurtado.cl", referrer: "https://archivopatrimonial.uahurtado.cl/index.php/taxonomy/index/id/42"
2023/11/14 12:38:06 [error] 1636#1636: *1847125 FastCGI sent in stderr: "PHP message: This request has been forwarded to a 404 error page by the action "search/autocomplete"" while reading response header from upstream, client: XXXXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /index.php/search/autocomplete?query=93-5663&repos= HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "archivopatrimonial.uahurtado.cl", referrer: "https://archivopatrimonial.uahurtado.cl/index.php/taxonomy/index/id/42"
2023/11/14 12:39:08 [error] 1636#1636: *1847252 FastCGI sent in stderr: "PHP message: Action "santi/index" does not exist" while reading response header from upstream, client: XXXXXXXXXXXXX, server: archivopatrimonial.uahurtado.cl, request: "GET /index.php/santi?listLimit=20&listPage=7&sort=alphabetic&sortDir=desc HTTP/1.1", upstream: "fastcgi://unix:/run/php7.2-fpm.atom26.sock:", host: "archivopatrimonial.uahurtado.cl"
~

But if we change the language to Spanish, the search engine works fine.

Any hint on what would be the problem?

I'll be glad to provide any othe technical detail if needed.

Thanks in advance for your help

[Dan Gillean]

Hi Hernán, 

I can only recall seeing an error like this once before, and it was caused by a bad import that introduced incorrect values into the culture rows of certain database tables. See: 

• https://groups.google.com/g/ica-atom-users/c/vOVjMKCs8j0/m/XlfdgmjmBAAJ
  

I am wondering if something similar has occurred here by chance. 

I would recommend that you first make a backup if you have not already, and then try running the query we have in our troubleshooting documentation that checks for common forms of data corruption. See: 

• https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/troubleshooting/#dealing-with-data-corruption
  

It also includes suggestions for some of the most common issues. In general, if you do find problematic records and the suggestions in that section will not resolve the issues, my suggestion would be to: 

• Find the object ID of the problem records - we have example queries on how to do this if you know the slug here: 
• https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/common-atom-queries/#finding-the-object-id-of-a-record
• Use that  ID to craft a SQL statement that will delete these records from the object table in AtoM. This should cascade across other tables, deleting the problem records entirely

If that is NOT the cause, and you are unable to find any data corruption, then we may need more information. For example: 

• Has this always been the case since you installed AtoM, or is this a recent issue? If recent, what has changed (e.g. did you upgrade recently? is the custom theme new? etc)
• Does your custom theme include code changes to the templates? Or is it JUST styling?
• Has Spanish been added in Admin > Settings > i18n languages?
• Have you tried re-populating the search index, restarting any services, or running any of AtoM's common maintenance tasks? See the various sections on this page: 
• https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/troubleshooting/
  
• What happens when you try to repopulate the search index? And what output do you get when running the search:status task? See:
• https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/troubleshooting/#get-information-about-your-search-index
  
• Are you using an ISO 639-1 two-letter code (i.e. es) in the configuration file for the default culture value? Have you maintained the existing spacing etc?
• Anything else that has changed or been done recently that may have affected this situation that we should know about?

Hopefully with more information we can provide further suggestions. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/f54de200-a8f6-4b83-abf8-219c94b057b6n%40googlegroups.com.

[Mercedes Jiménez Bolívar]

Hola, compañeros.

A mi se me ha presentado exactamente el mismo problema. Podeis observar aquí: http://archivo.ladigitalizadora.org/ que no coge el idioma español. Y aquí el error que aparece. (http://archivo.ladigitalizadora.org/index.php/informationobject/browse)

Search error encountered

Elasticsearch error: Elastica\Exception\ResponseException

Regresar a la página anterior.

Hicimos una copia del sistema, donde cambiamos la cultura a español en (/apps/qubit/config/settings.yml), pero poniendo solo "es" en vez de "es_ES" 

Borramos la caché, reiniciamos php-fpm, reicimos el índice de búsqueda y... ¡todo funciona correctamente!

Lo podeis ver aquí:

• http://archivo.ladigitalizadora.org
  

Sugerimos que en el doc settings.yml se cambie la frase donde aparece como idioma del español es_ES

Espero que sirva de ayuda.

Gracias por vuestra atención

[Mercedes Jiménez Bolívar]

Me he confundido al poner el último enlace. Copio entero el correo:

Hola, compañeros.

A mi se me ha presentado exactamente el mismo problema. Podeis observar aquí: http://archivo.ladigitalizadora.org/ que no coge el idioma español. Y aquí el error que aparece. (http://archivo.ladigitalizadora.org/index.php/informationobject/browse)

Search error encountered

Elasticsearch error: Elastica\Exception\ResponseException

Regresar a la página anterior.

Hicimos una copia del sistema, donde cambiamos la cultura a español en (/apps/qubit/config/settings.yml), pero poniendo solo "es" en vez de "es_ES" 

Borramos la caché, reiniciamos php-fpm, reicimos el índice de búsqueda y... ¡todo funciona correctamente!

Lo podeis ver aquí:

• http://archivo2.ladigitalizadora.org

Sugerimos que en el doc settings.yml se cambie la frase donde aparece como idioma del español es_ES

Espero que sirva de ayuda.

Gracias por vuestra atención

[Dan Gillean]

Hola Mercedes, 

Thank you for sharing this! Just to make sure that Google is translating correctly - you are saying that by changing your default language setting from es_ES to just es and restarting services, you have resolved your error - correct?

AtoM still uses ISO 639-1 two letter language codes. The second part ( i.e. the _ES) defines a specific locale - i.e. in this case, it clarifies that this is European Spanish. AtoM currently only supports a very small subset of sublocales - for example, we do support pt_BR to differentiate Brazilian Portuguese from European Portuguese - however at this time, we do not have many more sublocales available. 

A full list of known working Symfony languages and their codes (including the few working sublocales) can be seen here: 

• https://bit.ly/AtoM-langs

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/57a97622-01d1-4e7a-80aa-f462670f2b2cn%40googlegroups.com.

[Mercedes Jiménez Bolívar]

Yes, Dan, that is ok.

I have changed my default language setting from "es_ES" to just "es" and restarting services, and everything works correctly.

I think that es_ES doesn't work in AtoM.

Thanks!!!

[Mercedes Jiménez Bolívar]

Dan, está arreglado, sí. Pero no hemos averiguado a qué se debe el problema ¿tu tienes alguna idea brillante?

error elasticsearch:

http://archivo.ladigitalizadora.org/index.php/informationobject/browse

bien:

http://archivo2.ladigitalizadora.org/informationobject/browse

Seguimos

[Dan Gillean]

Hola Mercedes, 

Thanks for clarifying - and yes, you are correct: per this list of supported AtoM languages, es_ES is not currently supported. 

So: if the first site was using es_ES, then that could cause the Elasticsearch error. However, if you are looking for another possible cause of errors: 

First, I would check the Elasticsearch logs to see if they include additional information: 

• sudo tail -f /var/log/elasticsearch/elasticsearch.log
  

It can be a good idea to make sure that key indexing data is not missing due to something timing out - so running the generate slugs task and the build nested set task can sometimes help resolve issues: 

• php symfony propel:generate-slugs
• php symfony propel:build-nested-set
• Both tasks are documented hereowe

Similarly, you could try checking for data corruption?

• We have some SQL queries documented for this in the docs here, or
• There is a script one of our developers shared for running a check in this forum thread
• PS: This script will be added as a command-line tool in the next major AtoM release (v2.9), to help make fixing these issues easier! 

If the problem is that the search index itself got corrupted, then you can try manually deleting it before regenerating it: 

• curl -XDELETE 'localhost:9200/atom'
  
• php symfony search:populate

Otherwise, the remaining common causes I see for search exception errors are: 

1) Reserved characters being used 

For example, in the reference code separators, and then when searching for them. There are some characters that are reserved as Boolean operators in Elasticsearch, and using them otherwise without properly escaping them can cause errors. See: 

• https://www.accesstomemory.org/docs/2.8/user-manual/administer/settings/#escape-special-characters-from-searches
  

However, I didn't see any evidence of this in what I could see in your site, so I doubt that is the issue. 

2) Not enough system resources (esp memory)

Per this section of our documentation, we recommend that a production system have at least 7GB of memory. Elasticsearch uses a lot of memory, I have seen cases where users encountered strange errors when the server did not allocate enough memory. 

Is there any difference in the available RAM between your two servers? i.e. does archivo2 have MORE memory assigned, by any chance?

If that's still not it.... I'm not sure! Though I suspect it has something to do specifically with the descriptions. I noticed that you can still browse other entities, such as authority records - however, as soon as you try to click to view an authority record, then you get the error. While the authority record browse page has nothing to do with the archival descriptions, most authorities will be linked to a description, so the view page of an authority record WOULD need to connect to the part of the index with the descriptions. 

Hopefully running the common maintenance tasks, checking for data corruption, and manually deleting the index before repopulating it will resolve the issue! And if not, perhaps something in the Elasticsearch logs will tell us more. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c8cb551a-3e37-4aa5-a13c-09b46855c1adn%40googlegroups.com.