memory leak in django app?

[Etienne Robillard]

Hi all,

I'm struggling to understand how django/python may allocate and
unallocate memory when used with uWSGI.

I have a Debian system running Python 2.7 and uwsgi with 2GB of RAM and
2 CPUs.

Is that enough RAM memory for a uWSGI/Gevent based WSGI app running Django?

I'm running uWSGI with the --gevent switch in order to allow cooperative
multithreading, but my free RAM memory is always decreasing when nginx
is running.

How can I debug memory allocation in a Django/uWSGI app?

I defined also in my sitecustomize.py "gc.enable()" to allow garbage
collection, but it does not appears to make any differences.

Can you recommend any libraries to debug/profile memory allocation in
Python 2.7 ?

Is Django more memory efficient with --pymalloc or by using the default
linux malloc() ?

Thank you in advance,

Etienne

--
Etienne Robillard
tka...@yandex.com
https://www.isotopesoftware.ca/

[Antonis Christofides]

Hello,

the amount of memory you need depends on what Django does and how many workers
(instances of Django) you run (which usually depends on how many requests you
are getting and how I/O intensive your Django application is). For many
applications, 512 MB is enough.

Why are you worried? The only symptom you describe is that your free memory is
decreasing. This is absolutely normal. The operating system doesn't like RAM
that is sitting down doing nothing, so it will do its best to make free RAM
nearly zero. Whenever there's much RAM available, it uses more for its caches.

How much memory is your Django app consuming? You can find out by executing
"top" and pressing "M" to sort by memory usage.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

[Etienne Robillard]

Hi Antonis,

Thank you for your reply. I installed the htop utility and found that 2
of my 4 uWSGI processes are using 882M (42.7%) of resident memory each.
Theses two processes takes about  (85%) of the available RAM memory!
That can explain why I get "out of memory" errors when no more memory is
available for sshd.

I tried to debug memory allocation with guppy following instructions
here: https://www.toofishes.net/blog/using-guppy-debug-django-memory-leaks/

But I get "hp.Nothing" when I attempt to get the heap stats from the
master uWSGI process.

Example:
>>> hp.setref()
>>> hp.heap()
hp.Nothing

Any help would be appreciated!

Etienne

[Antonis Christofides]

Does this happen only in production? What about when you run a development
server? What is the memory usage of your development server?

Antonis Christofides
http://djangodeployment.com

[Etienne Robillard]

Hi Antonis,

My development server appears unaffected by this problem. Plus I can get
heap stats using guppy, which is pretty cool. :)

Both my production and development servers have __debug__ enabled in
Python 2.7.13.

However when using ab -c 100 to benchmark my nginx server I get:

SSL handshake failed (1).

Do you have any ideas how to configure nginx to allow a minimum of 100
concurrent connections when using SSL encryption?

I have worker_connections set to 512 in my nginx.conf

Regards,

Etienne

[Antonis Christofides]

The "SSL handshake failed" doesn't seem related to the alleged memory leak.

Now, you say you have this problem only in production. If you restart uwsgi,
does memory usage go immediately up? Does it go after a few requests? How many
requests? Do you have the same data in development and production? If, for
example, your app loads a whole db table in memory, and this db table has a
million records in production and only a few in development, there would be a
difference.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

[Etienne Robillard]

Hi Antonis,

Le 2017-12-06 à 16:23, Antonis Christofides a écrit :
> The "SSL handshake failed" doesn't seem related to the alleged memory leak.

Correct. I forgot to specify which SSL protocol to use. Using ab -f
TLSV1 solved that problem.

>
> Now, you say you have this problem only in production. If you restart uwsgi,
> does memory usage go immediately up? Does it go after a few requests? How many
> requests? Do you have the same data in development and production? If, for
> example, your app loads a whole db table in memory, and this db table has a
> million records in production and only a few in development, there would be a
> difference.

The problem started to appear in production after a few days without
restarting the main uWSGI process.

Also, the database in use is ZODB and is not affected by this issue at all!

So, I'm guessing this is a problem with uWSGI, not Django. For example,
running the development server with wsgiref on my local host do not take
more than 3.5% of rss memory with DEBUG=True.

Regards,

Etienne