Spam Inserts - subjects taxonomy

[afto...@gmail.com]

Dear, I recently found multiple inserts in the subjects taxonomy index tab.

Unfortunately it is not possible to manually clean this cyber waste inserted in the atom, there are infinite insertions, mixed with results of our works!

I am very concerned about the origin of this content and where they could have gained access to the site, I am also not sure if they were inserted before the last patch or after updating.

We are using the most current version of Atom-2.6.2 (175), Ubuntu 20.04-Server, php7.4, nginx.

Below are screenshots of the screens for verification

Tks

Augusto Torres

[Dan Gillean]

Hi Augusto, 

This is very concerning, and I wanted to have our team run some security tests before we replied. Our developers were unable to find places in the application code that would allow for this type of intervention - while it is possible that we've missed something, I suspect it may have to do with your deployment or configuration. 

I will talk to our team and see what resources we can provide that might help you with term cleanup. In the meantime, I wanted to make a few other suggestions. 

First, if you have been regularly making backups (as we strongly recommend anyone with a production site do!), now would be the time to load one! This will be the easiest way to undo the damage that has been done to your term data. 

Also, if you haven't already, I strongly recommend that you review all your user accounts. Delete or mark inactive any that you don't need. Make sure there's not a demo account left in there that might have been accessed. Consider enabling the "require strong passwords" security setting, and resetting all your user passwords with temporary ones and asking them to create new ones.

We did test the security of the API endpoints in AtoM as well, but if you have the API plugin enabled and aren't using it, you can disable the arRestApiPlugin in Admin > Plugins. 

I would also suggest that you take a look at your Nginx access logs to see if you can determine the IP address of the bots responsible for this. You can then use a combination of a robots.txt file and Nginx access rules to block these IPs. Some resources to get you started:

Robots.txt

• https://www.robotstxt.org/
  
• https://developers.google.com/search/docs/advanced/robots/intro
  
• https://moz.com/learn/seo/robotstxt
  

Nginx restrict access by IP

• https://www.techrunnr.com/how-to-restrict-access-by-ip-address-in-nginx/
  
• https://ubiq.co/tech-blog/block-ip-address-nginx/
  

A couple other points worth mentioning:

First, you should know that we have NOT tested AtoM 2.6 releases on Ubuntu 20.04 and PHP 7.4. We will be supporting both of these with the 2.7 release, but at this time we formally recommend using 18.04 and PHP 7.2 for AtoM 2.6 releases. I don't think this would be the cause of the vulnerability, but thought it worth mentioning. 

More importantly - looking at your full AtoM version number (2.6.2 - v175), I am concerned that at some point you have not upgraded properly. The database schema version (the second number - 175 in your case) for AtoM 2.6 releases should be v185. This means there are up to 10 schema migrations from previous releases that have not successfully been run in your MySQL database - meaning some changes to tables and columns are not currently present in your database where they should be, which could cause other errors as you try to make changes in AtoM. 

This can happen when the upgrade task is not run during the upgrade process. I don't know what the effects will be now, but I recommend that you make a backup of your data, and then try running the upgrade task again to see if it helps, followed by the recommended maintenance tasks. See the upgrading documentation: 

• https://www.accesstomemory.org/docs/latest/admin-manual/installation/upgrading/#run-the-upgrade-task
  

I'm really sorry this has happened to you, and I'm sure it must be frustrating. As I've said, we haven't found anything in the application code that would allow this type of attack to a properly secured site outright, but it's always possible we've missed something - please let us know what you learn as you investigate further. I will also talk to our team to see if they have further suggestions, including some ways to help clean up the terms data if you don't have backups to load. Some questions that will help us target our responses: 

• Were new terms created? Or did these bots simply add scope note text to existing terms? Or both?
• Is it just in the Subjects taxonomy, or have other taxonomies been affected as well?
• Would deleting all terms that are not linked to any descriptions resolve the issue for you? Or what about deleting all scope notes in a particular taxonomy? Trying to get a sense of how broad we can perform any cleanup actions without accidentally deleting data you want to keep. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/dde4c1d5-2b6d-4670-a486-fb472b124f1dn%40googlegroups.com.

[afto...@gmail.com]

Dear Gillean,

Thanks for your return.

I worked to clear the thousands of spam inserted in the subject, inserted arbitrarily without the need for a password, also users who no longer work on our project and strong passwords were deleted from the system, as instructed. Even so, some arbitrary insertions persist across the same field (taxonony / subjects).

I upgraded to the newest version of Atom (2.6.3), but I can't update MySQL tables, I'm reproducing the error below:

php -d memory_limit = -1 symfony tools: upgrade-sql

SQLSTATE [42S01]: Base table or view already exists: 1050 Table 'function_object' already exists

Do you know how I can fix this error?

I followed these instructions to no avail: https://projects.artefactual.com/issues/13365

I did a clean installation of Atom, using the same structure (Ubuntu 20.04 + php-7.4 + MariaDB-10.3) we didn't get this error previously reported.

I am now looking to block these attacks from some spam hosts, guided by the instructions you gave me.

If your developers need more information, I can privately send the link to my website to do some tests.

Thank you very much for the help you have given us.

Sincerely,

Augusto Torres

[Dan Gillean]

Hi Augusto, 

The error you are seeing when trying to run the upgrade task is basically saying that the table already exists. One way this can happen is if you forget to drop the database, and recreate a blank one, before loading your data. See step 4 in this section of the Upgrade documentation: 

• https://www.accesstomemory.org/docs/2.6/admin-manual/installation/upgrading/#copy-your-old-data
  

mysql -u username -p -e "DROP DATABASE IF EXISTS atom;"
mysql -u username -p -e "CREATE DATABASE atom CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci;"

So, you may want to try repeating the upgrade process, and be sure that you closely follow all the recommended steps to see if it helps. 

The other issue may be your use of MariaDB 10.3. We recommend MySQL 8.0 in our documentation, and this is what we use when developing and testing the application. There are some known differences and incompatibilities between the two - I couldn't even find a full comparison between MySQL 8.0 and Maria 10.3, but here's one for 10.4: 

• https://mariadb.com/kb/en/incompatibilities-and-feature-differences-between-mariadb-104-and-mysql-80/
  

If at all possible, I'd suggest restoring your pre-upgrade dump over a MysQL server, and redo the upgrade to a clean AtoM instance. 

The bug in question that you found was an issue fixed in 2.6.0 and earlier, that can arise when there are multiple AtoM instances on the same server. A fix should be included in 2.6.0 and later that ensures that it will check the database name before proceeding - so I'm hopeful that even if you have more than one AtoM instance on the same server, this should not be the cause of the error you're seeing. 

Please let us know how it goes, and what you discover as you continue to investigate this issue and protect your AtoM installation! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/0c68d228-3a6f-443b-b89e-75d1ca19118fn%40googlegroups.com.

[afto...@gmail.com]

Dear Gillean,

I made the substitution of MariaDB for MySQL8 as suggested.

https://www.accesstomemory.org/pt-br/docs/2.6/admin-manual/installation/upgrading/#copy-your-old-data

dump database to sql

mysql -u username -p -e "DROP DATABASE IF EXISTS atom;"
mysql -u username -p -e "CREATE DATABASE atom CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci;"

restore sql to newer created database

But this error persists:

SQLSTATE [42S01]: Base table or view already exists: 1050 Table 'function_object' already exists

We are not using multiple installations on this server.

We have another service under construction to migrate soon multiple archival institutions, including this one, on a centralized server using another isolated network.

I think something was wrong inserted in previous updates in this table that does not recognize a newer installed Atom code, making it impossible for the system to adjust the tables.

I did a clean installation, still using MariaDB, the system worked well, using the built Atom-2.6.3 (184) of a more recent installation, the reported error did not occur. But I can't just migrate archival data without losing data, exporting and importing in csv or xml.

I appreciate if you can guide us to some solution

Regards

Augusto Torres

[Dan Gillean]

Hi Augusto, 

You might be right about the cause. Unfortunately this may be beyond what we can help you to troubleshoot via the forum, as it is difficult to tell when the database error that is causing this issue occurred. It sounds like the problem is in your current sqldump, so unfortunately, if you dump the current database to a sqldump, drop/create and reload, then you are just restoring a bad database, and the same issue will reoccur. 

As I said in one of my earlier posts, if you have an older sqldump from before the last upgrade attempt, that would probably be the best thing to try. 

If you're interested in paid support, please feel free to email in...@artefactual.com and our team can discuss options with you for assisted troubleshooting. 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/882ea680-6490-4284-9f3d-3a6f85d1ea4an%40googlegroups.com.