2 AtoM sites, with replication

[Cintya Takahaschi]

Hi,

I readed the presentation about "2-site replication with AtoM" (on https://wiki.accesstomemory.org/images/6/62/2-site-replication.pdf) and it is something the we would like to implement.

Nowadays we heve 2 AtoM instances (VMWares): one "internal" where our team can edit content, and another "on internet", that is just a clone of the "internal" instance (so, content also editable by same users), that we make when our team finishes some large set of descriptions (for example, a large fond), what takes a few months to happen. After cloning, we just change network configuration, IPs, hostname and SSL certificates, basically - so the "internet" instance goes offline for some minutes, until we finish these changes.

The "2-site replication with AtoM" seemed more logical than our approach, but I'm not very familiar with ansible (neither Puppet or Chef or any other orchestrator software), but I was trying to understand the replication process, so maybe I can do it "manually" (or using some shell script).

I viewed the page with the ansible script (https://github.com/artefactual-labs/ansible-atom-replication), the "2 servers scenario" (https://github.com/artefactual-labs/ansible-atom-replication/blob/main/documentation/two-servers-scenario.md) and the deprecated playbook (https://github.com/artefactual-labs/atom-replication), and it seems that the replication process is somenthing like this:

(Considering 2 separate Ubuntu servers (atom-edit and atom-read-only) with AtoM and all its dependencies (Nginx, PHP, MySQL, Elasticsearch, Gearman) on each one, with all configurations and customizations (themes/plugins/static pages) done and respective SSL certificates installed).

1) Test if atom-edit is OK (is online, is browseable, access some descriptions to see if they're displayed fine);

2) Freeze the atom-edit server (not shure how to do it, maybe stopping Nginx service?);

3) Rsync /uploads and /downloads directories from atom-edit to atom-read-only (if they're not mapped to the same NAS share);

4) Make an Elasticsearch snapshot of the instance on atom-edit (not sure how to do it, but it seems its doable somehow);

5) Make a mysqldump of the database on atom-edit (lets say, on a file /tmp/atom.aaaa-mm-dd.sql);

6) Implement the Elasticsearch snapshot on atom-read-only;

7) Copy the atom.aaaa-mm-dd.sql file to /tmp/atom.aaaa-mm-dd.sql on atom-read-only and run it to populate the database.
Question: While the .sql file is being executed, the atom-read-only instance will be offline? Just asking because it takes a little more than 2 hours to do it (we have almost 1 million descriptions). If so, is there any way to do it with "no downtime"?

8) Clear cache (php symfony cc), restart memcached and PHP-FPM

Is it correct? Does someone has a similar replication plan?

[Johan Pieterse]

Hi  Cintya 

Reply to some of your steps (given you are not using the replication script):

1. Yes make sure AtoM master is working fine

2. Yes you can stop the Nginx. See also https://www.accesstomemory.org/de/docs/2.8/admin-manual/security/application/ - Making AtoM read-only

3. Yes. Also the AtoM slave needs to be installed/copied. I would suggest new install. Remember this must have ALL the server componants installed that is running on AtoM master.

4. Make sure where the Elasticsearch snapshot is.

5. Dump is essentially a backup of AtoM master.

6. Agree

7. Copy to AtoM slave and restore database. It is best practice not to have transactions in the database while a backup is made. Data could be out of sync or records could be locked in the AtoM master database. Perhaps run it after hours?

8. Agree

9. You now need to setup the replication part. Database and file store replication.