Django blocking on writing logs

[Daniel Arndt]

Hi all,

Although not explicitly mentioned anywhere, I've come to the conclusion Django blocks while writing logs with some handlers (it uses Python logging, https://docs.python.org/3/howto/logging-cookbook.html#dealing-with-handlers-that-block).

I can't find anything that explicitly states Django handles this already (in the code or documentation) and I believe it to be a problem in the platform I am working on. Justification for this belief is loose at best, but it's the best lead I have (specifically, the SysLogHandler seems like it might be blocking on sending to a UDP port, which seems crazy but apparently is possible)

I'm setting trying to see if I can solve this, but there seems to be very little details on this subject, and that alone worries me. So I have a couple questions for anyone willing to entertain me:

1. Is it silly to be worried about this? My gut says yes, simply because nobody else is. At the same time I see no real downside to logging asynchronously (is that true?). There are potential upsides. It might not solve the real problem (what is causing buffers to back up in the first place? likely crazy I/O usage) but it might speed up handling of requests during heavy I/O.
2. Have you done anything similar to this? How did you approach it?
3. The QueueHandler suggested in the Python cookbook doesn't seem to be easy to stuff into Django. We have Django running behind Gunicorn, so my thought was to have Gunicorn spin up a QueueListener process, and then have Gunicorn pass the queue to each worker similar to this cookbook: https://docs.python.org/3/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes -- I'm interested on any thoughts on this approach. Especially if there is a better way.

Cheers,

Dan

[David Bauer]

No I gave up on Django, don't know what it does, and don't have the money to take a class to learn more, so I just gave up for right now, when I see some money I might be able to take a class

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ce269c78-bd6b-4e06-b9de-2da6ccc79cd8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

David A. Bauer

[Avraham Serour]

It seems you are right, I found this SO thread dealing with the same question: https://stackoverflow.com/questions/34349797/are-log-statements-blocking-in-django-python

In pratical terms I use sentry, the python client claims to use a threaded async approach, I suggest taking a look at their implementation, or just plugging sentry to your app.

--

[Ameni Bejaoui]

I face this problem when i want to test the django interface in the web browser local domain :/ what should i do ? i change even the port to 8080 the same problem 

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFWa6tKBZ5Nq8BwOVBWxpyg%3DS6nzV_%3DhXF_km9A-bT_pFXfbtw%40mail.gmail.com.

[Avraham Serour]

it doesn't seem to have any relation to a synchronous logger

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAHBt3Ny%2BspWTP4-um4j%3DmtU1z-1HC0jeij9HTtaV_N77Q_R%3DRQ%40mail.gmail.com.