Single VM for all components or several?

[Simon Tidd]

Hi,

The documentation on the websites suggests that the standard AtoM test/demo site deployment works on a VM with the specs below (https://www.accesstomemory.org/en/docs/2.5/admin-manual/installation/requirements/#installation-requirements)

• Processor: 2 vCPUs @ 2.3GHz
• Memory: 7GB
• Disk space (processing): 50GB at a minimum for AtoM’s core stack plus more storage would be required for supporting any substantial number of digital objects.

Do most standard deployments involve installing all components/tiers on the same machine, or is there any best practice, or experience of, installing them on several different machines/resources for the respective compenents usch as the webserver, Elasticsearch, the SQL DB, etc.?

Thanks,

Simon

[Dan Gillean]

Hi Simon, 

I'm going to ask one of our technical team members to respond to this in greater detail, but the short answer is: yes, it's definitely possible to deploy AtoM's components in separate VMs. I'm including an image below of the default configuration we often recommend for a 2-site setup, where users have an internal read/write site, and an external read only site. We have a replication script in one of our repositories (here) that we use to copy the database and index from one site to another when run. This can be useful for better control over visibility of entities without a publication status (such as authority records), better security, and also better performance (you can aggressively cache on the public front end for example). 

In this case, note that internally, we have AtoM installed on one server, and MySQL and Elasticsearch on a separate VM: 

So long as you have all the ports and permissions etc properly configured, I see no reason why this couldn't be atomized (hehe) further. 

As I said, I will have one of our system administrators share some more detailed thoughts soon. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/45f23aad-89b2-40dc-8f6d-aa4bd08e152e%40googlegroups.com.

[scol...@artefactual.com]

Hi Simon

One server deployments are good enough for most users, and in case of need you can move other services later to their own vm without problems.

There are three layers in atom that must be taken into accout regarding performance: Nginx/php, MySQL and Elasticsearch. For sites with a big mysql/elasticsearch database, or tons of users, it's better to split nginx/php,  mysql and elasticsearch into it's own servers, so they have more resources and they don't affect the cpu/memory use of each other.

/santi

On Wednesday, May 22, 2019 at 12:24:15 AM UTC+2, Dan Gillean wrote:

Hi Simon, 

I'm going to ask one of our technical team members to respond to this in greater detail, but the short answer is: yes, it's definitely possible to deploy AtoM's components in separate VMs. I'm including an image below of the default configuration we often recommend for a 2-site setup, where users have an internal read/write site, and an external read only site. We have a replication script in one of our repositories (here) that we use to copy the database and index from one site to another when run. This can be useful for better control over visibility of entities without a publication status (such as authority records), better security, and also better performance (you can aggressively cache on the public front end for example). 

In this case, note that internally, we have AtoM installed on one server, and MySQL and Elasticsearch on a separate VM: 

So long as you have all the ports and permissions etc properly configured, I see no reason why this couldn't be atomized (hehe) further. 

As I said, I will have one of our system administrators share some more detailed thoughts soon. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

On Tue, May 21, 2019 at 2:25 AM Simon Tidd <simo...@gmail.com> wrote:

Hi,

The documentation on the websites suggests that the standard AtoM test/demo site deployment works on a VM with the specs below (https://www.accesstomemory.org/en/docs/2.5/admin-manual/installation/requirements/#installation-requirements)

• Processor: 2 vCPUs @ 2.3GHz
• Memory: 7GB
• Disk space (processing): 50GB at a minimum for AtoM’s core stack plus more storage would be required for supporting any substantial number of digital objects.

Do most standard deployments involve installing all components/tiers on the same machine, or is there any best practice, or experience of, installing them on several different machines/resources for the respective compenents usch as the webserver, Elasticsearch, the SQL DB, etc.?

Thanks,

Simon

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-ato...@googlegroups.com.

[David Juhasz]

Hi Simon,

I've also found that for AtoM the MySQL server tends to be the most CPU intensive of all the services, and also benefits the most from fast disk I/O. I would recommend tuning your Percona/MySQL Server memory settings if you have a large database, the more work MySQL can do "in memory" the better it will perform.  Elasticsearch likes a lot of RAM (2GB+), but is pretty light on CPU and disk I/O.  PHP/Nginx likes a decent amount of RAM (1GB or more) and a moderate amount of CPU (2 cores is usually fine), but the webserver is usually not the resource bottleneck in my experience.

Cheers,
David

--

David Juhasz, 
Senior developer

Artefactual Systems Inc.
www.artefactual.com

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c77c1add-d129-46d0-a611-bc70dd81c5a1%40googlegroups.com.

[Simon Tidd]

Thank you both Dan, David and Santi. Your answers and the diagram were very helpful.

 

Is there anyone that I can speak to about configuring a setup similar to the one Dan described? I am working for a small deceased artist's estate with archives they'd like to make more accessible. At the moment, I am in a feasiblity planning stage and trying to assess option for the platform/system and how much it will cost to set up and run (including what we can do ourselves versus not).

 

Cheers,

Si

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c77c1add-d129-46d0-a611-bc70dd81c5a1%40googlegroups.com.

[Dan Gillean]

Hi Simon, 

If you have general questions, please feel free to ask them here, and we will do our best to provide timely and relevant answers. If you'd like more personalized support, please feel free to contact me off-list and we can discuss paid consultation and/or support options. Artefactual also offers both hosting and remote technical maintenance contracts if that is of interest - see: 

• https://www.artefactual.com/services/  
  

I will say that for a small personal archives, a 2-site deployment may be more complex than you actually need. Most of our hosting clients, which include some fairly large and well-visited sites, are deployed on single VMs following the specs included in our documentation that you referenced in your initial post. You could still separate the MySQL database and/or the Elasticsearch index into separate VMs for better performance without having to set up a 2-site deployment with a replication script passing data between them. 

Just a thought! 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/6b3792d1-e5df-46b5-9cf2-59ae183a008f%40googlegroups.com.

[Simon Tidd]

Thank you both Dan and Santi. Your answers and the diagram were very helpful.

Is there anyone that I can speak to about configuring a setup similar to the one Dan described? I am working for a small deceased artist's estate with archives they'd like to make more accessible. At the moment, I am in a feasiblity planning stage and trying to assess option for the platform/system and how much it will cost to set up and run (including what we can do ourselves versus not).

Cheers,

Si

Simon Tidd

M: 07839 714305 | E: simon...@gmail.com

You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/xklB5TbpuXw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c77c1add-d129-46d0-a611-bc70dd81c5a1%40googlegroups.com.