Wsgi processes per user site

[virgil.balibanu]

Hi,

I've setup my server using mod_wsgi in deamon mode and apache and
everything worked well but I'm having some trouble with memory
consumption. Each site is running under his own user and takes a
pretty big amount of memory because is uses python and django. Now
I've looked into memory consumption and i have each user (site-x
format) running 18 processes on linux, each of them taking some
memory. My setup looks like this:
...
WSGIDaemonProcess site-195 user=site-195 group=nobody inactivity-
timeout=172800
WSGIDaemonProcess site-196 user=site-196 group=nobody inactivity-
timeout=172800
WSGIDaemonProcess site-197 user=site-197 group=nobody inactivity-
timeout=172800
WSGIDaemonProcess site-198 user=site-198 group=nobody inactivity-
timeout=172800
...
WSGIProcessGroup %{ENV:PROCESS_GROUP}
WSGIApplicationGroup %{GLOBAL}
WSGISocketPrefix /var/run/wsgi
....
Now each site has only a few users at the moment so i don't know why
it should take so many processes.

Could someone please explain how this works to me, because i need to
understand and see what can be done about it.

Thanks,
Virgil Balibanu

[Graham Dumpleton]

There aren't 18 processes, there are 18 threads within each process.
This is because there are default 15 threads and then the main thread
waiting for shutdown and two background threads for deadlock timeout
and inactivity timeout checking.

What Linux system are you use and what version? Way back in time,
Linux systems showed each thread as a separate processes when you use
some tools for listing processes. I thought this had changed and they
only showed one process now no matter how many threads they ran.

Can you say exactly what command you are using to monitor processes
and the exact command line arguments as well?

Graham

[virgil.balibanu]

Hi, I'm using Centos 5.5 and i'm viewing processes using htop. So I
computed that a process should have around 20 MB of occupied memory,
that means one thread has that much, or all of the threads? So do the
threads use the same loaded python and django code or each one of them
loads a new one?

Thanks,
Virgil Balibanu

On Apr 20, 1:19 pm, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:

[Graham Dumpleton]

On 20 April 2011 20:58, virgil.balibanu <virgil....@gmail.com> wrote:
> Hi, I'm using Centos 5.5 and i'm viewing processes using htop. So I
> computed that a process should have around 20 MB of occupied memory,
> that means one thread has that much, or all of the threads? So do the
> threads use the same loaded python and django code or each one of them
> loads a new one?

Each user has one process of size 20MB and the 18 threads should all
be running within that one process in the one Python interpreter and
Django instance.

Try using the 'ps' command instead of 'htop'.

Graham

> --
> You received this message because you are subscribed to the Google Groups "modwsgi" group.
> To post to this group, send email to mod...@googlegroups.com.
> To unsubscribe from this group, send email to modwsgi+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
>
>

[virgil.balibanu]

Thanks

On Apr 20, 2:05 pm, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:

[virgil.balibanu]

Still, what I find interesting is that clamav uses this much memory:
ps -ef | grep clamav | awk '{ print $2 '} | xargs pmap -d | grep
mapped:
mapped: 159420K writeable/private: 131344K shared: 0K

while one of my processes will be:
ps -ef | grep site-196 | awk '{ print $2 '} | xargs pmap -d | grep
mapped:
mapped: 398652K writeable/private: 179412K shared: 100K
and I don't have any site actually running on user site-196

This is supposed to be in kB? So something is computed wrong by the
system or I don't understand the data here.

For a process that is used by a site this changes drastically:
ps -ef | grep webb2009 | awk '{ print $2 '} | xargs pmap -d | grep
^mapped:
mapped: 593756K writeable/private: 220432K shared: 100K

How much memory would one of this processes actually use? What is
mapped and writeable/private, and is it actually kB or kb ?

Thanks,
Virgil Balibanu


On Apr 20, 2:43 pm, "virgil.balibanu" <virgil.balib...@gmail.com>
wrote:

[Graham Dumpleton]

Just use 'ps'. Not sure why you are wanting to go the extra step of using pmap.

Use:

ps auxwww | less

and it will show headers like:

USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND

The important figure is RSS (which might also be called RES).

This is actual resident memory in use by process.

Find the entries in 'ps' output for your process and look at that
value. It should be in KB.

Also suggest you do Google search for 'RSS VSZ' and see what you find
which explains what they mean. For example:

http://wiki.marandcustomsolutions.com/space/Linux/Understanding+memory+usage

Graham

[Brian D]

Virgil,

htop is a nice tool with features that can help answer your
questions.

The bottom line on the screen shows you all of the function key
shortcuts.

Use F5 to show the processes and threads in a tree view, the threads
will be indented underneath the process that they belong to. Using
this view you can see that the amount of memory used is exactly the
same for all threads in a process which would indicate that the values
are for the entire process and not just for the thread.

Use F2 to change settings. Underneath the 'Display options' be sure
to select 'Display threads in a different color'. Or if you'd rather
not even see the threads then select 'Hide userland threads'.

Brian