Is Django logging multi-process safe?

127 views
Skip to first unread message

Dan Davis

unread,
Dec 21, 2018, 11:51:16 AM12/21/18
to Django users

I just came across this:


This suggests that log messages sent to a StreamHandler will be processed properly, but log messages sent to a standard logging.handlers.TimedRotatingFileHandler are not process safe.

What sayeth the group?

Dan Davis

unread,
Dec 21, 2018, 11:55:35 AM12/21/18
to Django users
Looks like I should use a WatchedFileHandler and then have logrotate do the logging.   If the file pointer is moved by another process, then it will work.
Does gunicorn patch this for me?   Will I be better off using syslog or something?

PASCUAL Eric

unread,
Dec 21, 2018, 2:00:40 PM12/21/18
to Django users

Hi,


If you are running on a Linux system, you'd better using the standard stream logs and logrotate to manage file rotating.


The later is the blessed tool for this kind of task, since it manages for free the compression of rotated files and removal of oldest ones if you want to.


Regards


Eric


From: django...@googlegroups.com <django...@googlegroups.com> on behalf of Dan Davis <dans...@gmail.com>
Sent: Friday, December 21, 2018 5:55:35 PM
To: Django users
Subject: Re: Is Django logging multi-process safe?
 
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/b1792c98-0373-42e6-93ac-8b0961ec83e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Dan Davis

unread,
Dec 21, 2018, 5:01:20 PM12/21/18
to django...@googlegroups.com

Eric, thanks for confirming.  I've written my share of logrotate.d files, but I may push to do something else, like send them directly to Elasticsearch with a delay of some sort.


You received this message because you are subscribed to a topic in the Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-users/1E1MuO08okk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/AM6P193MB042177E51A91076EAC03931E8CB80%40AM6P193MB0421.EURP193.PROD.OUTLOOK.COM.

PASCUAL Eric

unread,
Dec 22, 2018, 8:31:47 AM12/22/18
to django...@googlegroups.com

I do not  have the details in mind, but in the context of a former  Docker based deployment, we have used the ELK stack to manage the logs of the containers, by capturing the containers' stdout. Thanks to this option, it has been possible to stay with basic stream based logs.


If you are interested, I can try to retrieve the exact procedure, but I can't promise : it was for the first deployment of this app, which was hosted on Azure at that time. Since then we have moved to a Kubernetes managed solution on GCP.


BTW we have kept using stream based logs, GCP log tooling offering the appropriate tools for exploiting them.


Regards 


Eric


Sent: Friday, December 21, 2018 11:00:17 PM
To: django...@googlegroups.com

Jason

unread,
Dec 22, 2018, 4:42:10 PM12/22/18
to Django users
my company uses logentries for log aggregation and search, but an ELK-based stack is indeed a very viable option for handling logs across multiple servers and containers.

Dan Davis

unread,
Feb 26, 2019, 8:13:41 PM2/26/19
to Django users
So, there are some issues in managing the files.   It is not very 12-factor to have the application worry about such things.

The issues with using a FileHandler or TimedRotatingFileHandler from multiple processes/kernel threads are less important, but still exist.  If a file is append-only, then write automatically does a seek to the end before writing, and python logging works hard to make sure this is a single write.   Practically, that means that a log write to a file is atomic if:
  • It's size is less than a page in the buffer cache (4k)
  • You are not using NFS as a target for logs
In the later case (NFS), the kernel *simulates* the atomic seek to the end and write.  So, if there are multiple writers across multiple servers, you are going to be screwed, but if each single server has its own file, and multiple processes are writing, then it is mostly going to be OK.

There may be some other potential pitfalls using rotating files, such as logging.handlers.TimedRotatingFileHandler.  Haven't analyzed that closely.

So, anyway, the most likely problem then is if the log entries get larger than 4k, which may not be very likely but is possible.
Reply all
Reply to author
Forward
0 new messages