Squid caching, processor load balancing (via Cubicweb)

[Tom Atkins]

Just looking at CubicWeb via the link given on Reddit.  Looks like an interesting project - amazing what's out there that I've never heard of!

This blog post:

http://www.cubicweb.org/blogentry/1484253

describes an interesting way of using multi core processors to load balance and solves session caching via squid.  Might this be a useful approach to use for scaling web2py?

[VP]

Feature-wise, web2py is pretty complete. I think the next milestones
of web2py should be on performance and scalability; (web2py is
reaching 2.0 sooner or later, right?)

I would like to see stronger built-in support for smart caching and
load balancing (distributing work across different servers).

I have a simple idea about smart caching (I don't know if it works or
not). As web2py controls database access via the DAL, we can update
the cache only when the database is updated. If the database is not
modified, there is no reason to refresh the cache. Right?

To be safe you can set a minimum time (e.g. 5 minutes cache refresh),
"just in case".

[pbreit]

Isn't load balancing usually handle by the web server or a load balancer? Can the framework really address load balancing effectively? Would be cool if it could, of course.

[Massimo Di Pierro]

web2py user Jay at PyCon really convinced to look into nginex. It is
an amazing server, fast, easy to configure and can do load balancing:
http://wiki.nginx.org/NginxHttpUpstreamModule

Just start multiple web2py instances with rocket (on the same machine
or different machines) and add to the nginex list of ip:port.

[pbreit]

I just switched from Cherokee to Nginx but have not investigated load balancing yet. Is Rocket necessary? I have a Nginx/uWSGI setup. I posted my install Fabfile on another thread.

[Anthony]

FYI, Nginx + uWSGI is what DotCloud uses for its Python hosting (HAProxy for load balancing): http://docs.dotcloud.com/components/python/