Problem with Atom 2.5.4 and Elasticsearch 5.6.16

joel.ma...@gmail.com

unread,

Jun 25, 2020, 11:34:38 AM6/25/20

to AtoM Users

Hi,

My context : five Atom instances are installed in a shared Web cluster :
same OS (Centos 7) , same PHP, same packages, ..

In the past, I have done many upgrades
        2.2 -> 2.3
        2.3 -> 2.4
in this context.

- On a first instance, the migration 2.3 -> 2.5.4 (with ES 5.6.16) was nice
I followed https://www.accesstomemory.org/en/docs/2.5/admin-manual/installation/upgrading/#installation-upgrading

- On a second instance Atom with exactly the same components and the
same procedure,
        a/ it is OK for 2.3 -> 2.4.1
        b/ but i don't succeeded
                2.3 -> 2.5.4
        or
                2.4.1 -> 2.5.4.
        I have tried four times.

The actions
        php symfony tools:upgrade-sql
        php symfony digitalobject:regen-derivatives
are OK, but when I launch
        php symfony search:populate
I get this message

<<
mapping QubitAip...

name cannot be empty string
>>

Any idea ?

Best regards,

        Joel

Dan Gillean

unread,

Jun 25, 2020, 4:53:47 PM6/25/20

to ICA-AtoM Users

Hi Joel,

I'll start by saying that our ability to provide CentOS installation support via this forum is pretty limited - we focus our development and testing efforts around Ubuntu LTS packages.

We've changed from MySQL 5.6 to 5.7 in the 2.5 releases, and some of the default SQL modes are more strict - I'm wondering if that might be part of the issue. A couple quick checks:

Did you implement the recommended SQL modes? From the docs:

First, let’s create a new file with our SQL modes.

Paste the following values in a new file at /etc/mysql/conf.d/mysqld.cnf and save:

[mysqld]
sql_mode=STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
optimizer_switch='block_nested_loop=off'

Now we’ll restart MySQL:

sudo systemctl restart mysql

Second, did you remember to drop and recreate the database, as per the upgrade instructions, before loading your data and running the upgrade task?

Drop and re-create the new AtoM database to remove any unnecessary tables and columns.

$ mysql -u username -p -e 'drop database new_database; create database
new_database character set utf8 collate utf8_unicode_ci;'

Now, load the contents into the new database:

$ mysql -u username -p new_database < /tmp/database.sql

See: https://www.accesstomemory.org/docs/latest/admin-manual/installation/upgrading/#copy-your-old-data

Here are a couple things you could try.

First, you could make a backup of your database, then run the tools:purge task. This will clear ALL data from your site, so make sure you have that backup in place first!

php symfony tools:purge

Now you could try the steps above - drop and recreate the database, load your backup, run the upgrade task, restart services, and try indexing again.

You could also try manually deleting the search index with the following:

curl -XDELETE 'localhost:9200/atom'

Then see if it can properly re-index.

You can also try running 2.5's search:status task to see if it can tell us anything more about what's going on with your index. See:

https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/cli-tools/#check-the-status-of-your-elasticsearch-index

Meanwhile, I'll check with our team and see if they have further thoughts on why this might be happening, and how to resolve it.

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/409d1c2f-6812-4251-865a-f63389bdeee1o%40googlegroups.com.

Joel Marchand

unread,

Dec 27, 2020, 5:27:26 PM12/27/20

to AtoM Users

Hi,

I follow https://www.accesstomemory.org/fr/docs/2.6/admin-manual/installation/linux/ubuntu-bionic/ to build an Ubuntu 18.04 VM.

So I have installed PHP 7.2, Elasticsearch 5.6.16, MySQL 8.0

On this server, I get the same problem : many of my Atom 2.5.4 instances are fine, but with one instance, I get the following problem :

        php symfony tools:upgrade-sql
        php symfony digitalobject:regen-derivatives
are OK, but when I launch
        php symfony search:populate
I get this message

<<
mapping QubitAip...

name cannot be empty string
>>

More precisely, the command

php symfony search:populate --exclude-types="aip"

displays many things and finishes with

[Term] Bel Hajd inserted (0.22s) (494/656)
    [Term] Ferme Derois inserted (0.22s) (495/656)
    [Term] Madagh inserted (0.22s) (496/656)
    [Term] Ferme Caid Mansouri inserted (0.22s) (497/656)
    [Term] Ras el Merja inserted (0.22s) (498/656)
    [Term] Sidi Hassas inserted (0.22s) (499/656)

The message "name cannot be empty string" is present in /var/log/elasticsearch/elasticsearch.log

The command

curl -XDELETE 'http://localhost:9200/masa?pretty'
is successfull, but with no effect : "php symfony search:populate" finished always by the same message.

Please note that this instance had no problem with Atom 2.4. version.

Thanks a lot for any idea.

Regards,

Joel Marchand

Dan Gillean

unread,

Jan 4, 2021, 10:51:28 AM1/4/21

to ICA-AtoM Users

Hi Joel,

AtoM 2.5.4 expects MySQL 5.7, while AtoM 2.6 expects MySQL 8. I'm not sure if you're trying to run version 2.5.4 with MySQL8 but if so, this is likely the source of the issue.

Another thing you could try would be to check the SQL modes, and disable STRICT_TRANS_TABLES if it is enabled. I believe this might be enabled by default in 5.7 and later - however in the "Create the database" section of the installation docs for 2.5, we recommend setting different default SQL modes. See the bottom half of this section:

https://www.accesstomemory.org/en/docs/2.5/admin-manual/installation/linux/ubuntu-bionic/#create-the-database

If you want to check or change the SQL modes of an installation, the following page might help:

https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/common-atom-queries/#check-server-sql-modes

Remember to restart MySQL after making changes. You may also need to repeat some of the previous steps to drop and recreate, etc. Another thing you can try after dropping and recreating the database, but BEFORE loading your backup, is to run the tools:purge command. This will flush any data from the db, which can sometimes be useful after multiple installation attempts. Be sure you have a copy of your sqldump outside of AtoM before running this!

php symfony tools:purge

Let us know if this helps.

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/7b951995-85bb-45e2-be4c-bbab0baa5eb6n%40googlegroups.com.

Joel Marchand

unread,

Feb 9, 2021, 8:53:47 AM2/9/21

to ica-ato...@googlegroups.com

Hi Dan,

Thanks a lot for your following response.

I work many hours on our Atom instances.

All, except one, are now installed in a dedicated VM
- Ubuntu 18.04
- MySQL 8
- Elasticsearch 5.6.16
as required on our documentation, with a 2.6.x version of Atom.

No problem at all for 5 instances.

But always the same problem for the last instance, with the messages
reported on my first message (see below).

<<
With the command

php symfony search:populate
I get this message

<<
mapping QubitAip...

name cannot be empty string
>>

More precisely, the command
     php symfony search:populate --exclude-types="aip"
displays many things and finishes with

[Term] Bel Hajd inserted (0.22s) (494/656)
    [Term] Ferme Derois inserted (0.22s) (495/656)
    [Term] Madagh inserted (0.22s) (496/656)
    [Term] Ferme Caid Mansouri inserted (0.22s) (497/656)
    [Term] Ras el Merja inserted (0.22s) (498/656)
    [Term] Sidi Hassas inserted (0.22s) (499/656)

The message "name cannot be empty string" is present in /var/log/elasticsearch/elasticsearch.log

The command
     curl -XDELETE 'http://localhost:9200/masa?pretty'
is successfull, but with no effect :

"php symfony search:populate" ends always by the same message.

Please note that this instance had no problem with Atom 2.4. version.
>>

Best regards,

Joel Marchand

Le Mon, Jan 04, 2021 at 10:51:14AM -0500, Dan Gillean disait :

> 3. Drop and re-create the new AtoM database to remove any unnecessary

> tables and columns.
>
> $ mysql -u username -p -e 'drop database new_database; create database
> new_database character set utf8 collate utf8_unicode_ci;'
>

> 4. Now, load the contents into the new database:

> ica-atom-users/7b951995-85bb-45e2-be4c-bbab0baa5eb6n%40googlegroups.com.
>
> --
> You received this message because you are subscribed to a topic in the Google
> Groups "AtoM Users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/
> ica-atom-users/mhfPJ1BgzE8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> ica-atom-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/
> ica-atom-users/

> CAC1FhZLpwMpYrYmbrnRhyjH_jtUK4gsS-KXUuhMg_X49SDm7sA%40mail.gmail.com.

José Raddaoui

unread,

Feb 9, 2021, 10:45:17 AM2/9/21

to AtoM Users

Hi Joel,

It's possible that you are facing a similar problem to the one solved in ...

https://mail.google.com/mail/u/1/#search/qubitAIP/FMfcgxwKkbrQdLxVvWhQtMVDsxZNqrpn

Best regards.

Joel Marchand

unread,

Feb 9, 2021, 12:15:59 PM2/9/21

to ica-ato...@googlegroups.com

Hi,

I don't understand your message.

Yes it is the same problem
- for this instance http://masa.huma-num.fr/
- with Atom 2.5 on CentOS 7
- or with Atom 2.6 on Ubuntu 18.04

I spend days and days on this problem, with no solution at all.

My opinion is

- some data is OK to be indexed by Elasticsearch 1.x (with Atom <= 2.4)

- and this same data is not OK to be indexed by Elasticsearch 5.6 (with Atom >= 2.5)

But how to find this data and the reason ?

Best regards,

Joel Marchand

Le Tue, Feb 09, 2021 at 07:45:16AM -0800, José Raddaoui disait :

> The message "name cannot be empty string" is present in /var/log/
> elasticsearch/elasticsearch.log
>
> The command
> curl -XDELETE 'http://localhost:9200/masa?pretty'
> is successfull, but with no effect :
>

> ica-atom-users/273f2685-0a52-4685-be09-4f96d24228e8n%40googlegroups.com.

José Raddaoui

unread,

Feb 9, 2021, 3:34:58 PM2/9/21

to AtoM Users

Hi Joel,

Sorry I added the link from my email thread and not the group:

https://groups.google.com/u/1/g/ica-atom-users/c/A--wCn1eMAA/m/TEjx9BSXAAAJ

From that thread, the issue seems to be an empty language added under Admin => Settings => I18n language settings, and that removing the blank language entry was all that was required to allow re-indexing to proceed again.

Best regards.

Reply all

Reply to author

Forward