Best practices on deploying several java applications over several tomcat containers with haproxy load balancing and varnish cache

[Raoni Castro]

Hi All,

I'm new to this list and to ansible as well, so I'd like to apologize beforehand if the question is malformed or if it should be asked in another place.

Any help is greatly appreciated.

We use the following architecture for our in-house built java web applications:

                                                    - > Tomcat a on Ubuntu server 1

                                                 /

Varnish Cache (http) - > HAProxy 

       ^

       |                                         \

Nginx (https)                                 - > Tomcat a on Ubuntu server 2

There are over a hundred .war applications distributed over about a dozen tomcat instance listening on different ports. The war applications are updated frequently. (They are built on another server with Jenkins/Maven). There are development, staging, and production environments.

I want to use ansible to perform the following tasks (after some basic stuff like adding users, setting up ntp, dns, etc): 

 - add repos to ubuntu, install nginx, varnish, haproxy, tomcat instances

 - add .war applications to tomcat (we actually extract them into directories)

 - create each tomcat server.xml with its definitions for port, logging, etc, and application resource requirements (datasources, mostly)

 - configure varnish to forward requests to appropriate haproxy frontend based on application context name

 - define haproxy frontends and backends

 - deploy newer versions of application .war files without service interruption (stop tomcat a on server 1, extract war file, start tomcat a on server 1, stop tomcat on server 2...)

 - update software components (nginx, varnish, haproxy, tomcat instances)

We also have deployed other open source software on top of tomcat that usually have different requirements, for instance, Nuxeo (nuxeo.com), Liferay (liferay.com), and DSpace (dspace.org). So, a reusable tomcat role would be great.

We are also responsible for a Plone CMS installation (plone.org) that follows the same basic architecture, but zope application servers are deployed using buildout.

My intention is to layout our ansible directory like this:

####

development_hosts

staging_hosts

production_hosts

group_vars/ <- store all variables inside this directory tree

   enterpriseappservers/

                          tomcat-a/

                                instance.yml

                                applications.yml <- this contains a list of applications with their requirements

                          tomcat-b/

                          ...

   plonewebcontentservers/

   javawebcontentservers/

   digitallibraryservers/

                          plone-sites.yml

   ecmservers/

site.yml <- includes the other playbooks

appservers.yml <- simple playbooks, only include roles, maybe multiple times the same role (in the tomcat case)

loadbalancers.yml

cachingproxies.yml

webcontentservers.yml

digitallibraryservers.yml

ecmservers.yml

roles/

   common/

   tomcat/

       tasks/

           main.yml

       templates/

            enterpriseapps_server.xml.j2

            digitallibrary_server.xml.j2

            plonewebcontent_server.xml.j2

            javawebcontent_server.xml.j2

             ...

         ...

   varnish/

       tasks/

           main.yml

       templates/

            enterpriseapps_default.vcl.j2

            digitallibrary_default.vcl.j2

            plonewebcontent_default.vcl.j2

            javawebcontent_default.vcl.j2 <- there are specific caching rules for each kind of application

   haproxy/

       tasks/

           main.yml

       templates/

            haproxy.cfg.j2 <- all it does is to create frontends and backends

   enterprise-app/

       tasks/

           main.yml

   zope/

   zeo/

   dspace/ <-- to hold specific tomcat customizations

   liferay/   <-/

   ...

####

My question is: are we following correctly the best practices or should we do things differently? 

Our goal is to avoid defining the same things twice and to avoid overcomplicated roles and 100's of lines variable files with nested lists becoming too difficult to manage and maintain.

Thanks in advance.

Raoni Castro.