ATOMIC_REQUESTS Alternative - better for highly concurrent systems
SteveB
I want to offer something the the community of Django users.
If you like the safety net of having each request handled by a transaction, but don't want unnecessary blocking on highly concurrent web applications, then the following may be of interest.
The Django transaction documentation offers you two choices:
- Turn on the database setting ATOMIC_REQUESTS, and the for those view functions which don't need transaction protection, decorate these with @transaction.non_atomic_requests
- Don't turn ATOMIC_REQUESTS for your database, and instead decorate those view functions for which you want transaction protection with @transaction.atomic.
However, there are a couple of downsides to both of these approaches:
- There may be lots of view functions to be so decorated.
- Many view functions follow the pattern where they accept both GET and POST requests. The GET request returns the form to be filled in, and the POST request submits the filled in form for processing. You want the POST request to be processed by a transaction, as there may be multple tables to be updated, however, you don't want the GET request to have the transaction overhead. Therefore decorating such view functions with non_atomic_requests or atomic won't do what you want.
A solution:
I offer a middleware module which looks to see if the request is a modifying one (that is, one of POST, PUT, DELETE or PATCH). If not, it does not use a transaction for the request. If it is a modifying request it will use a transaction, provided that the database ATOMIC_REQUESTS is not on (don't want to double up), and that the view function is not decorated with @transaction.non_atomic_requests.
The middleware has to overcome a limitation that it cannot simpy do something like "with transaction.atomic():", because in the process_view method, the middleware has to return control before the view function is called. It also has to work where several databases may be involved, so it creates a list of transaction.Atomic instances, one per configured database, saves them on the request object, and calls their __enter__() methods.
The process_response() method then has to invoke the __exit__() method of these Atomic instances in the reverse order, and handle any exceptions which may occur.
It also provides a process_exception() method, which invokes the __exit__() method of the Atomic instances, also in reverse order, passing to it the exception information, and handling any exceptions which may occur. For each Atomic instance whose __enter__() method was called, it has to invoke the corresponding __exit__() method.
I welcome any comments from the group on this piece of middleware.
- Stephen Brooks
----------------------------
File: atomicmodifyingrequests.py
from django.db import connections, transaction
import sys
class AtomicModifyingRequestsMiddleware(object):
'''This middleware puts django.db.transaction.Atomic contexts (one per database
if the database does not have the ATOMIC_REQUESTS set) around
the view function if the request method is a modifying one and
the view function is not annotated with @transaction.non_atomic_requests.
Author: Stephen Brooks
Minimum Django Version: 1.6
'''
def process_view(self, request, view_func, view_args, view_kwargs):
if request.method in ('POST', 'PUT', 'DELETE', 'PATCH'):
non_atomic_requests = getattr(view_func, '_non_atomic_requests', set())
try:
for db in connections.all():
if (not db.settings_dict['ATOMIC_REQUESTS'] and
db.alias not in non_atomic_requests):
if not hasattr(request, '_atomic_modifying_requests_middleware_atomic_contexts'):
request._atomic_modifying_requests_middleware_atomic_contexts = []
atm_ctxt = transaction.Atomic(db.alias, True)
request._atomic_modifying_requests_middleware_atomic_contexts.append(atm_ctxt)
atm_ctxt.__enter__()
except Exception as e:
self.process_exception(request, e)
raise
return None
def process_response(self, request, response):
if hasattr(request, '_atomic_modifying_requests_middleware_atomic_contexts'):
exc_info = (None, None, None,)
exc = None
for atm_ctxt in reversed(request._atomic_modifying_requests_middleware_atomic_contexts):
try:
atm_ctxt.__exit__(*exc_info)
except Exception as exc:
exc_info = sys.exc_info()
if exc:
raise exc
return response
def process_exception(self, request, exception):
if hasattr(request, '_atomic_modifying_requests_middleware_atomic_contexts'):
exc_info = sys.exc_info()
for atm_ctxt in reversed(request._atomic_modifying_requests_middleware_atomic_contexts):
try:
atm_ctxt.__exit__(*exc_info)
except Exception:
exc_info = sys.exc_info()
return None
Oleg Korsak
Great!
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f1cec6e6-cce9-416b-bba9-a876d9c20693%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.