Web framework suggestions

[Carlo Sogono]

Hi guys,

I am looking for a web application framework that would be most
suitable for our environment. I do not have extensive web app
experience as I have mainly been exposed to backend stuff...but we
will start building a web app soon.

1. Isolated data centres. Our DC's are located in different states for
redundancy and have no internet connections. We connect to them via
VPN from our Sydney HQ. Just a little background--these servers are
farmed optimisation servers used for building airline solutions
(shortest path algorithm stuff).

2. Updates are installed manually by resident sys admins in each DC.
Non-system software are mantained by us (Qantas) from our HQ using VNC/
SSH. This web app is not to be considered a system app so it will be
installed in a neutral UNIX user with *no root access*. It is
*definitely not possible* to request for the web app to be considered
as system software or request root access due to the complex nature of
the contracts surrounding these systems. It's between Qantas, IBM,
Jeppesen (Boeing) and TCS (the company Qanats outsources its IT
operations). Having these sort of "simple requests" will involve
General Managers and like I said not possible to go on that route.

3. Updates to the web framework will have to be done via SSH...meaning
download them in Sydney and update manually. This means the framework
should be stable enough to allow easy manual updates once every 3-6
months without changing the API too often...and that it has very few
individual components. I have experience with Pylons and there are way
too many components to maintain individually to work properly. Modular
is a must.

4. Hopefully will run using lighttpd and Python 2.4. Upgrading Python
will also be very very difficult. Our optimisation systems are
dependent on Python 2.4 and doing a mass Python upgrade would probably
create chaos. At the moment we just do small incremental 2.4 upgrades.
Cannot go >=2.5.

Any suggestions are welcome! Thanks in advance.

Regards,
Carlo

[Dylan Jay]

Sounds like it's not really a deployment problem rather than a
framework problem.

Sounds like a perfect candidate for buildout and hostout based
deployment. People have used buildout for django, grok, zope based
applications. Not sure about pylons but can't see why it would be hard
to make a buildout for it as long as it uses eggs and perhaps even if
it doesn't.

Buildout is all about containing an application in a single directory
with a non root user and even your own version of python.

Buildout however does like to access index servers like pypi to
retrieve eggs. You can get round this by prepopulating a distribution
location with all the eggs you will need and then specifying that
location in your find-links. I've also used a proxy server on my ssh
client such that eggs could be downloaded via the ssh link. I was
considering building that in as an option in hostout.


---
Dylan Jay, Plone Solutions Manager
www.pretaweb.com
tel:+61299552830
mob:+61421477460
skype:dylan_jay

[Carlo Sogono]

Hi Dylan,

Buildouts are ok but what I'm concerned about is maintaining the
underlying environment that runs the egg files. This must be easy to
rebuild if the servers ever go down. It is possible to deploy sites in
Pylons using a single egg file but the environment it runs in requires
too many independent modules to be installed in lib/python. This
excess fat is something I'm trying to minimise. Problems I've had in
the past include a small incremental Genshi version change will render
the whole site inoperable. With 5-15 various modules I need a better
way to track incompatibilities. As I'm mostly an analyst/programmer I
do not have time troubleshooting module version conflicts when I'm
upgrading.

Regards,
Carlo

[Dylan Jay]

On 07/07/2009, at 1:12 PM, Carlo Sogono wrote:

>
> Hi Dylan,
>
> Buildouts are ok but what I'm concerned about is maintaining the
> underlying environment that runs the egg files. This must be easy to
> rebuild if the servers ever go down. It is possible to deploy sites in
> Pylons using a single egg file but the environment it runs in requires
> too many independent modules to be installed in lib/python. This
> excess fat is something I'm trying to minimise. Problems I've had in
> the past include a small incremental Genshi version change will render
> the whole site inoperable. With 5-15 various modules I need a better
> way to track incompatibilities. As I'm mostly an analyst/programmer I
> do not have time troubleshooting module version conflicts when I'm
> upgrading.

This is exactly the situation buildout was designed to solve. To take
control of the versions of the software you use for a particular
deployment and be able to repeat that.

Here is an example on how to use buildout with pylons.

http://wiki.pylonshq.com/display/pylonscommunity/Howto+install+Pylons+with+buildout

In addition in my talked I gave examples of using pinned versions of
nginx, squid, varnish, disutils packages etc with buildout.

http://www.slideshare.net/djay/buildout-and-hostout-or-how-to-host-an-app-for-20-in-20min

[Carlo Sogono]

On Jul 7, 1:59 pm, Dylan Jay <d...@pretaweb.com> wrote:
> This is exactly the situation buildout was designed to solve. To take  
> control of the versions of the software you use for a particular  
> deployment and be able to repeat that.

Thanks heaps. That's what I was looking for. Will investigate further
and do some tests and post some results after a few weeks....

>
> Here is an example on how to use buildout with pylons.
>

> http://wiki.pylonshq.com/display/pylonscommunity/Howto+install+Pylons...

>
> In addition in my talked I gave examples of using pinned versions of  
> nginx, squid, varnish, disutils packages etc with buildout.
>

> http://www.slideshare.net/djay/buildout-and-hostout-or-how-to-host-an...