Joomla high availability on AWS EC2

[Rogerio Brito]

Hello all,

Does anyone know how to setup Joomla on Amazon Web Services EC2?
The goal is to have high availability using several instances when we have
high traffic. For MySQL, I believe is pretty simple, I just have to
configure Joomla to use de Amazon RDS db. But what about the files? The user
files uploaded on one instance should be available on the other instances as
well. Or content posted with images.. they also should be available on every
instance.
Any help will be greatly appreciated.

Thank you

Rogerio

[BNRBranding]

Here's a tutorial on Joomla 3 on EC2 - http://www.comtechies.com/2012/12/how-to-install-joomla-3-on-amazon-ec2.html

Here's a video tutorial - https://www.youtube.com/watch?v=WFtj3aTL6MM

[Michael McGinn]

Have you considered setting up your own PaaS (Platform as a Service)?  

Check out this new system for super easy administration and management of Virtual Machine instances.

Try out the free BETA account.

http://www.uhurucloud.com?promocode=uhs017

 

Connect with Michael Scott McGinn on the following sites

==============================

http://About.Me/michael.scott.mcginn

 

http://geekzonebookscom.elance.com

 

 

http://linkedin.com/in/geekzonebooks

 

 

https://plus.google.com/110643333851419339743
http://www.mygeekzone.net/resume

http://bit.ly/MichaelScottMcGinn

President, WapTug.Org - Web Application Programming Technology User Groups
Owner, GeekZoneBooks.Com - Technical Training Book Sales
Owner, GeekZoneHosting.Com - Web Hosting Services

Owner, MyGeekZone.Net - Web Services Sales Portal
Skype ID: waptug
Cell: 1-206-909-0361

Office: 1-206-436-4194

Fax: 1-206-339-7365

 IMPORTANT WARNING:
This e-mail is intended for the use of the person to whom it is addressed and may contain information that is privileged and confidential, the disclosure of which is governed by applicable law.  Dissemination, distribution or copying of this information in an unauthorized manner is strictly prohibited.  Sender accepts no liability for any damage caused by any virus transmitted by this e-mail.  If you receive this in error, please notify the sender.

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-gene...@googlegroups.com.
To post to this group, send an email to joomla-de...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Rogerio Brito]

Hello,

 

Thanks but that doesn´t work for High Availability. I was able to install Joomla on a EC2 instance. When the traffic is high AWS creates one or more copies of your instance and load balances them. That means the resources (files, DB) must be shared and stored outside the instance. Otherwise will have database and files inconsistence across the multiple instances. The DB is not a problem, we can easly configure it to be external… but the files…

Any other ideas?

 

[]s

[Rogerio Brito]

Hi Michael,

Thanks, but I believe the problem would be the same… all “clouds” use the same concept of duplicating instances when taking high traffic.  And duplicating the instance I would get inconsistent DB and files…

I guess I could change my question to: How do I run multiple joomla installations using the same DB and same files storage?

 

Thank you

 

[Guilherme Albuk]

   Hi Rogério,

   Ax you said, the biggest issue  is dealing with uploaded content.

   If you are developing a custom component, you can send these files directly to a S3 bucket using AWS SDK for PHP or using an uploader that has this feature built-in.

   But if you need to upload these files in several places like articles, categories and other components, you can set you instances to mount a S3 bucket in /images/ folder using s3fs, so they will all be in sync. And you can use an URL Rewrite so these images will be retrieved directly from the bucket or their corresponding CloudFront CDN, or you can even use CDN for Joomla that will automatically rewrites  all these links and point then to CloudFront.

   Guilherme Albuk

2013/8/9 Rogerio Brito <rogeri...@intercode.com.br>

[Rogerio Brito]

Hi Guilherme,

 

I believe a CDN component will do the trick, I will do some testing.

 

Thanks a lot!

[]s

[gavin]

Hi Rogerio,

it looks like there is a cloudformation high availability template for joomla - did you use that as a starting point, or are you building from scratch? I'm curious as to how much traffic your site gets that you need to go down this route, or what is driving you here?

best wishes,

Gavin

[Rogerio Brito]

Hi Gavin,

 

I’ll test the Joomla cloudformation template, and see what is does for the file uploads. I already have the site I would like to host on Amazon. The site has about 700 users, and sometimes there is a live video broadcast. When that happens, I have about 300 users online, and the site craws. It is a Joomla 2.5 with Jomsocial site.

 

Using JMeter to test load the site, when I go above 100 users I start to get errors on page delivery. 100 online users is too few for a site to handle, isn’t it?

I’ll try some other things to improve performance like MemCache, and maybe a Percona MySQL server… but I believe a scalable setup is the ultimate solution for any high traffic website. Don’t you think?

 

Does anyone recommend the Percona Server?

 

Thanks

 

 

De: joomla-de...@googlegroups.com [mailto:joomla-de...@googlegroups.com] Em nome de gavin
Enviada em: sexta-feira, 16 de agosto de 2013 15:49
Para: joomla-de...@googlegroups.com
Assunto: Re: [jgen] Re: Joomla high availability on AWS EC2

 

Hi Rogerio,

it looks like there is a cloudformation high availability template for cloudformation - did you use that as a starting point, or are you building from scratch? I'm curious as to how much traffic your site gets that you need to go down this route, or what is driving you here?

best wishes,

Gavin

--

[rikki....@gmail.com]

Hi All,

I am have similar requirements. I am running 2 ec2 instances which will host Joomla cms under ELB. I want to make sure if some one uploads any content it should get replicated to both the instances.

Could you please provide me inputs on how did you achieve this?

Thanks,
Ricky

[Rogerio Brito]

Hello Ricky,

What I did was to have another instance as a fileserver, running NFS for sharing directories.
So, on the Joomla Instances I have

/images/
/phocadownload/
/templates/
... other directories...

mapped to o shared directory on the file server.

For the DB I use RDS MultiAZ, so all Joomla instances use the same DB.

The file server instance can be a T1.micro instance.

I've tried other solutions, such as S3, but is way too slow.

Using the fileserver you have a single point of failure, but it works very well for load balancing Joomla.  I'm using this approach for more than 3 years, and haven't had any problem. In some situations I had to start more than 10 Joomla instances, and ir worked just fine.

Good luck!

--

rikki....@gmail.com

8 de julho de 2015 09:36

gavin

16 de agosto de 2013 15:48

Hi Rogerio,

it looks like there is a cloudformation high availability template for joomla - did you use that as a starting point, or are you building from scratch? I'm curious as to how much traffic your site gets that you need to go down this route, or what is driving you here?

best wishes,

Gavin

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-gene...@googlegroups.com.
To post to this group, send an email to joomla-de...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general.
For more options, visit https://groups.google.com/groups/opt_out.

Rogerio Brito

9 de agosto de 2013 02:35

Hi Michael,

Thanks, but I believe the problem would be the same… all “clouds” use the same concept of duplicating instances when taking high traffic.  And duplicating the instance I would get inconsistent DB and files…

I guess I could change my question to: How do I run multiple joomla installations using the same DB and same files storage?

 

Thank you

 

 

De: joomla-de...@googlegroups.com [mailto:joomla-de...@googlegroups.com] Em nome de Michael McGinn

Enviada em: quarta-feira, 7 de agosto de 2013 12:02

Para: joomla-de...@googlegroups.com
Assunto: Re: [jgen] Re: Joomla high availability on AWS EC2

 

Have you considered setting up your own PaaS (Platform as a Service)?  

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-gene...@googlegroups.com.
To post to this group, send an email to joomla-de...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-gene...@googlegroups.com.
To post to this group, send an email to joomla-de...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-gene...@googlegroups.com.
To post to this group, send an email to joomla-de...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Michael McGinn

7 de agosto de 2013 12:02

Have you considered setting up your own PaaS (Platform as a Service)?  

Check out this new system for super easy administration and management of Virtual Machine instances.

Try out the free BETA account.

http://www.uhurucloud.com?promocode=uhs017

 

Connect with Michael Scott McGinn on the following sites

==============================

http://About.Me/michael.scott.mcginn

 

http://geekzonebookscom.elance.com

 

 

http://linkedin.com/in/geekzonebooks

 

 

https://plus.google.com/110643333851419339743
http://www.mygeekzone.net/resume

http://bit.ly/MichaelScottMcGinn

President, WapTug.Org - Web Application Programming Technology User Groups
Owner, GeekZoneBooks.Com - Technical Training Book Sales
Owner, GeekZoneHosting.Com - Web Hosting Services

Owner, MyGeekZone.Net - Web Services Sales Portal
Skype ID: waptug
Cell: 1-206-909-0361

Office: 1-206-436-4194

Fax: 1-206-339-7365

 IMPORTANT WARNING:
This e-mail is intended for the use of the person to whom it is addressed and may contain information that is privileged and confidential, the disclosure of which is governed by applicable law.  Dissemination, distribution or copying of this information in an unauthorized manner is strictly prohibited.  Sender accepts no liability for any damage caused by any virus transmitted by this e-mail.  If you receive this in error, please notify the sender.

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-gene...@googlegroups.com.
To post to this group, send an email to joomla-de...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

BNRBranding

7 de agosto de 2013 11:18

Here's a tutorial on Joomla 3 on EC2 - http://www.comtechies.com/2012/12/how-to-install-joomla-3-on-amazon-ec2.html

Here's a video tutorial - https://www.youtube.com/watch?v=WFtj3aTL6MM

On Tuesday, August 6, 2013 1:33:00 PM UTC-4, Rogerio Brito wrote:

[Johan Janssens]

Hi Rogerio,

The database part shouldn't be that hard to solve. Files is a bit trickier, check out following solutions :

- https://github.com/s3fs-fuse/s3fs-fuse

- http://fileconveyor.org/

Personally haven't used both yet as we sync to RackSpace and CloudFiles and not AWS. On Rackspace CloudFiles we use https://github.com/redbo/cloudfuse with great success, s3fuse does the same for AWS. 

This approach has worked great for us for setting up high availability environments.

Happy coding!

Johan

[Ricky Mahajan]

Hi,

Thanks Rogerio and John for your quick response. I am also planning to use s3fs, so mounting all of the content directory on to tthe s3 bucket and use CDN (Content Delivery network) for faster processing and better latency. In your opinions does that sound good? Will that make content deliver faster?

Thanks,

Ricky

--

You received this message because you are subscribed to a topic in the Google Groups "Joomla! General Development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/joomla-dev-general/qHGN0q4LdpA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to joomla-dev-gene...@googlegroups.com.
To post to this group, send email to joomla-de...@googlegroups.com.

Visit this group at http://groups.google.com/group/joomla-dev-general.

For more options, visit https://groups.google.com/d/optout.

[Rogerio Brito]

Hi Ricky,

I have tried s3fs, and it was extremely slow.
PHP files need to be local for Joomla to work, and as I needed to have the /templates folder shared so I could make changes on it that reflected all instances, s3fs didn't work for me.
But it could work for the images folder.

[]s

Ricky Mahajan

8 de julho de 2015 23:59

Hi,

Thanks Rogerio and John for your quick response. I am also planning to use s3fs, so mounting all of the content directory on to tthe s3 bucket and use CDN (Content Delivery network) for faster processing and better latency. In your opinions does that sound good? Will that make content deliver faster?

Thanks,

Ricky

--

You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-gene...@googlegroups.com.

To post to this group, send email to joomla-de...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general.
For more options, visit https://groups.google.com/d/optout.

Johan Janssens

8 de julho de 2015 21:32

Hi Rogerio,

The database part shouldn't be that hard to solve. Files is a bit trickier, check out following solutions :

- https://github.com/s3fs-fuse/s3fs-fuse

- http://fileconveyor.org/

Personally haven't used both yet as we sync to RackSpace and CloudFiles and not AWS. On Rackspace CloudFiles we use https://github.com/redbo/cloudfuse with great success, s3fuse does the same for AWS. 

This approach has worked great for us for setting up high availability environments.

Happy coding!

Johan

On Tuesday, August 6, 2013 at 7:33:00 PM UTC+2, Rogerio Brito wrote:

--

You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-gene...@googlegroups.com.

To post to this group, send email to joomla-de...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general.
For more options, visit https://groups.google.com/d/optout.

Rogerio Brito

6 de agosto de 2013 14:33

[Johan Janssens]

Hi Rogerio,

Tools like s3fs-fuse should only be used to keep user uploaded files in sync. They should not be used to sync a complete Joomla installation (code, etc) across different servers. There are better tools for that job. 

Code should be kept synced by using a deployment process. If you run multiple server instances you would deploy to all of them at the same time. We use http://capistranorb.com/ for this, for more info on how to deploy to multiple servers see : https://github.com/capistrano-plugins/capistrano-unicorn-nginx/wiki/Multiple-server-setup

Cap is a Ruby but there are PHP solutions too, you can find them here : https://github.com/ziadoz/awesome-php#deployment

I hope that helps,

Johan

[Johan Janssens]

Yep that should do the trick. The mount will allow Joomla to talk to the files as if they where local, making it hum along nicely. The CDN will ensure they are delivered with low latency.

Johan

[Ricky Mahajan]

Hi Johan,

Thats true. I am also mounting user uploaded files directory of joomla using s3fs. and for the code sync I am using puppet to deploy it across servers..  Have you used puppet or do you prefer to use php deployer tool?
Thanks once again for the valuable inputs..

Thanks,

Ricky

--
You received this message because you are subscribed to a topic in the Google Groups "Joomla! General Development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/joomla-dev-general/qHGN0q4LdpA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to joomla-dev-gene...@googlegroups.com.