Lower memory usage of FastCGI on Lighttpd?

[IBBoard]

Is it possible to lower the memory usage of Trac when used through the
FastCGI method? I've got a VPS server running Lighttpd that has been
running with a reasonable amount of memory free (anything up to about
50MB of its 128MB limit). After adding Trac it's now running close to
full because Trac is using more memory than anything else (last I saw
it was at 29% while MySQL was only at ~22%).

Given that none of my PHP processes get above about 18% and they power
about four or five scripts including CMS and forums it seems a bit
excessive that my one Python instance for Trac is chewing up two-
fifths of my server's memory.

I've tried running Trac as a daemon and getting Lighttpd to proxy the
requests to it, but it seemed much slower than FastCGI.

Thanks,

IBBoard

[Noah Kantrowitz]

No, that sounds about right, or actually on the low end of Trac's
memory usage. A medium sized site will generally use 50-100 MB per
instance.

--Noah

[Noah Kantrowitz]

On Dec 26, 2008, at 2:23 PM, Noah Kantrowitz wrote:

>
> No, that sounds about right, or actually on the low end of Trac's
> memory usage. A medium sized site will generally use 50-100 MB per
> instance.
>
> --Noah
>

PS: If you upgrade to python 2.5.3 or 2.5.4 (obviously the latter is
better) you should see Trac's steady state usage drop a bit as it
includes a major memory allocation fix we found in Python a while ago.

[IBBoard]

50 to 100MB? What's it caching (and does it need to)?! I've got
desktop apps with a full GUI that don't take that much memory! Even
Thunderbird on my machine is only using ~75MB and that's with the
bloat of the XUL interface and some huge inboxes.

I'll try to upgrade to a newer Python as I'm still back on 2.3 (it's a
CentOS 4.6 VPS so it's not too up-to-date by as standard). Hopefully
Python won't have too many dependencies!

Looks like I might have to start upgrading the server's memory once it
starts getting used more :\ So far I've only got a couple of small SVN
repos and I'm the only user. I don't think I'd have had these memory
usage problems if I'd stuck with Mantis, but Trac has more integrated
features and is far more manageable.

Thanks,

IBBoard

[Noah Kantrowitz]

Its a common problem among all Python web frameworks (and lets face
it, Trac really is a Python web framework that happens to ship with a
very project tool as the example app instead of Hello World). Mostly
it is because they use tons of strings, especially during template
rendering. In many places Trac trades pure efficiency for programmer
friendliness. Earlier versions of Trac (before 0.11) used Clearsilver,
a C-based templating system, and were therefore not subject to the
whims of the Python string allocator. It will obviously limit what
features are available, and what 3rd party tools you can use, but
staying at the last 0.10 release might be a solution for the moment.

--Noah

[IBBoard]

It looks like Python is on to a 2.6 branch now and that it's stable.
Given that I can't install the 2.5 RPMs in CentOS 4 because of
dependency issues I'll have to build from source. Is there much point
in going from 2.3 to 2.5 when I could go to 2.6? Are the memory
changes only in 2.5 so far?

As for the strings issue, that sounds like a bit of a flaw. I guess
it's just another "memory is getting cheap" assumption.

Thanks.

[Noah Kantrowitz]

Trac is not currently compatible with Python 2.6. They decided to
change the behavior of one of the cgi library functions.

--Noah