Also in the SCGI/WSGI method, there is a reference to the statement: "This method will not work with the standard visit model..." Could anyone explain what the "standard visit model" is... Googled around, but couldn't get any more info for this context.
> Also in the SCGI/WSGI method, there is a reference to the statement: > "This method will not work with the standard visit model..." > Could anyone explain what the "standard visit model" is... Googled > around, but couldn't get any more info for this context.
I'd recommend not using Lighty for this purpose. Take a look at Pound instead:
What I typically do is run both Lighty and TG behind the Pound proxy, with TG serving dynamic content and Lighty serving everything that's under /static.
Service HeadRequire "Host: .*\.yourdomain\.com.*" Url "/(static|images|css|javascript|flash)/.*" BackEnd Address 127.0.0.1 Port 8080 End End Service HeadRequire "Host: .*\.?yourdomain\.com.*" BackEnd Address 127.0.0.1 Port 8081 End End End
Then setup Lighty to run on port 8080 and your TurboGears app to run on 8081 and you're set.
I'll not be building a hi traffic site, so may not benefit from pound. Ligthy can already be compiled for SSL, pcre etc., and be setup to talk to TG (even over a UDS, if I'm right). What I need is all the extra http functionality lighty provides over and above what CherryPy (TG) can provide.
Pl. let me know what I would gain by using pound in my scenario.
> I'll not be building a hi traffic site, so may not benefit from pound. > Ligthy can already be compiled for SSL, pcre etc., and be setup to > talk to TG (even over a UDS, if I'm right). What I need is all the > extra http functionality lighty provides over and above what CherryPy > (TG) can provide.
> Pl. let me know what I would gain by using pound in my scenario.
The real question is: what would you gain by using Lighty? You are asking about a proxy solution and Pound is a proxy. Lighty can *act* as a proxy, but it's really a web server (in fact, the reverse proxy support is new to the latest point release). Since CherryPy is already a web server, what is the point in putting another web server in front of it?
In short, Pound gains you simplicity and singleness of purpose.
Also, FWIW, I switched to Lighty around a year ago and I've found it pretty disappointing. Jan Knetche appears to have more interest in catering to the Rails community and going to conferences than fixing outstanding bugs. Lighty's config syntax is dismal. At first it appears quite elegant, but soon you realize it's actually a gross hack and is broken in several key ways. There are obvious misfeatures in its syntax and some parts are outright broken and have remained so for quite some time. It's pretty sad when bugs stick around so long that they acquire names ("The Nested Conditional bug" being one that grates on me to no end).
I still use Lighty for its original purpose, that is, serving static content, but find that maintaining complex configurations to be more of a pain than it's worth and so avoid it whenever possible. Frankly I'm hoping that one of the other lightweight web servers steps up to take Lighty's place soon.
On Thu, 2006-08-24 at 12:36 -0700, venkatbo wrote: > What I need is all the extra http functionality lighty provides over > and above what CherryPy (TG) can provide.
Sorry, somehow in my haste to reply I didn't see that last sentence: what functionality are you hoping for?
Basically, I needed to provide support for: - HTTP/1.1 - SSL (openssl) - (Fast)CGI - chroot() - sessions - static content
I thought lighty would be providing all except the sessions part, which I was hoping to make it disk/file-based. The # of simultaneous users will be in the 20's. Initially no db invloved (data coming from other backend apps), but may have a db later.
One problem with Lighty is that it leaks memory like a sieve [1]. I audited it for a little bit and I gave up, it's a mess. I'd steer clear of it, it will quickly ruin your day if you throw a lot of traffic at it.
The only solution I know of that's extremely high performance that offers all of the features that you want is nginx [2], but its documentation is largely in Russian. I can't read Russian, but I was able to figure it out (the configuration language isn't Russian, neither is C source). I currently have nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a *single server*. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6).
Under the same kind of load, apache falls over (after using 1000 or so processes and god knows how much RAM), pound falls over (too many threads, and using 400MB+ of RAM for all the thread stacks), and lighty *leaks* more than 20MB per hour (and uses more CPU, but not significantly more).
> Basically, I needed to provide support for: > - HTTP/1.1 > - SSL (openssl) > - (Fast)CGI > - chroot() > - sessions > - static content
> I thought lighty would be providing all except the sessions part, > which I was hoping to make it disk/file-based. The # of simultaneous > users will be in the 20's. Initially no db invloved (data coming from > other backend apps), but may have a db later.
On Thu, 2006-08-24 at 15:43 -0700, Bob Ippolito wrote: > The only solution I know of that's extremely high performance that > offers all of the features that you want is nginx [2], but its > documentation is largely in Russian. I can't read Russian, but I was > able to figure it out (the configuration language isn't Russian, > neither is C source). I currently have nginx doing reverse proxy of > over tens of millions of HTTP requests per day (thats a few hundred > per second) on a *single server*. At peak load it uses about 15MB RAM > and 10% CPU on my particular configuration (FreeBSD 6).
I'm glad to hear of someone I trust trying this server. Someone was espousing it on the #cherokee channel, but I hadn't gotten around to trying it out. Guess I have to now. Thanks for the info.
Thanks for that insight :-) I somehow got the impression lighty was the rage these days, second only to apache. With this kink of a leak issue, unclear how it attained that position ;!)
On Thu, 2006-08-24 at 15:43 -0700, Bob Ippolito wrote: > The only solution I know of that's extremely high performance that > offers all of the features that you want is nginx [2], but its > documentation is largely in Russian. I can't read Russian, but I was > able to figure it out (the configuration language isn't Russian, > neither is C source). I currently have nginx doing reverse proxy of > over tens of millions of HTTP requests per day (thats a few hundred > per second) on a *single server*. At peak load it uses about 15MB RAM > and 10% CPU on my particular configuration (FreeBSD 6).
Could you pl. list the lighty modules you have running in your app. If possible, if you could also provide snippets of your lighttpd.conf file, it would be useful.
As I said, it's only serving static content, there is no application. And lighttpd only uses "mod_alias", "mod_access", "mod_accesslog".
My point was that the memory leak is not in the core of lighttpd but in some of it's modules. Since there is a choice which module to use to connect to WSGI applications, we should see what modules have leaks.
On Thu, 2006-08-24 at 18:44 -0700, Cliff Wells wrote: > On Thu, 2006-08-24 at 15:43 -0700, Bob Ippolito wrote:
> > The only solution I know of that's extremely high performance that > > offers all of the features that you want is nginx [2], but its > > documentation is largely in Russian. I can't read Russian, but I was > > able to figure it out (the configuration language isn't Russian, > > neither is C source). I currently have nginx doing reverse proxy of > > over tens of millions of HTTP requests per day (thats a few hundred > > per second) on a *single server*. At peak load it uses about 15MB RAM > > and 10% CPU on my particular configuration (FreeBSD 6).
Thanks for all the posts in this thread, would be vitally useful to me. Some real novice questions:
- flup is not available? Does it imply we can no more have SCGI/WSGI configuration? - As with LightTPD, will SCGI/WSGI be faster compared to simple proxying in NgineX? - Any tutorial/guidance on how to configure SCGI/WSGI with Nginex would help. - By "This method will not work with the standard visit model...," I understood that we can still use the identity package, but with "visit.on=False" in app.cfg. Is it not so? Or, we can't use the identity package althgether?
> As I said, it's only serving static content, there is no application. > And lighttpd only uses "mod_alias", "mod_access", "mod_accesslog".
> My point was that the memory leak is not in the core of lighttpd but in > some of it's modules. Since there is a choice which module to use to > connect to WSGI applications, we should see what modules have leaks.
mod_proxy definitely leaks. When I attempted to deploy lighttpd, that was the *only* module I was using (no static content even) and it leaked memory like a sieve.
nginx does not leak at all (at least for proxy, fcgi, rewrite, static content). The config language is simpler, and it also uses less memory and CPU than anything else I've tried (and this is at a load of > 500 req/sec at times).
I don't think scgi is a choice with nginx at the moment, but fcgi and proxying definitely are.
