Debuging an irreproducible timeout

[Hugo Osvaldo Barrera]

Hi all,

I'm having a very complex issue with a rather large and complex
application of mine.
My production environment uses ELB -> nginx -> unicorn -> django.

On my production environment, when saving certain models via the admin,
I get a timeout. I have tens of thousands of requests per day only
certain models in a single admin page return timeout (and it's always
the same ones).

I copied the entire DB and could not reproduce this locally (in debug
mode, with slightly different settings).

I then proceeded to continue testing on one of the production servers
(having removed it from the balancer, of course):

When hitting via nginx -> gunicorn, I get the same issue.
When hitting via django-admin runserver, I cannot reproduce the issue.
When hitting nginx -> test_script.py[1], I cannot reproduce the issue.

So I'm kind of stuck trying to even find out what causes this timeout -
it would seem that the issue is only reproduced when gunicorn is part of
the recipe, but I can't debug its child processes (if its possible, I've
miserably failed to find out how).

FWIW, this admin only defines `inline`, `search_list` and
`list_display`. There's nothing funky about it

+-+-+-+-+-+
# test script
# interactively runs a wsgi app to which I can then add
`ipdb.set_trace()`s ;)
from myapp.wsgi import application
from werkzeug.serving import run_simple

run_simple('0.0.0.0', 8000, application)
+-+-+-+-+-+

Any ideas on how I can further debug this? Or if its possible to somehow
attach to a gunicorn child process. Or maybe profile function calls
somehow?

--
Hugo Osvaldo Barrera