Django graceful database errors

Damian Myerscough

unread,

Jun 7, 2017, 9:28:10 AM6/7/17

to Django users

Hello,

I have setup a Django project that is being served via Nginx + gunicorn, however, I would like to handle graceful database failures.

For example, if the ORM cannot query the database I would like to return a custom error message.

==> gunicorn.log <==

[2017-06-07 06:06:01 +0000] [13] [DEBUG] GET /incidents/

[2017-06-07 06:06:11 +0000] [8] [CRITICAL] WORKER TIMEOUT (pid:13)

[2017-06-07 06:06:11 +0000] [13] [INFO] Worker exiting (pid: 13)

[2017-06-07 06:06:11 +0000] [20] [INFO] Booting worker with pid: 20

==> nginx-error.log <==

2017/06/07 06:06:11 [error] 9#9: *1 upstream prematurely closed connection while reading response header from upstream, client: 172.17.0.1, s

erver: XXXX, request: "GET /incidents/ HTTP/1.1", upstream: "http://unix:/webapp/run/gunicorn.sock:/incidents/", host: "192

.168.99.102:30061"

I am using the following code snippet to test this, however, the json response is not sent back

try:

incidents = Incidents.objects.filter(user_and_team=request.user.client).exclude(status="Resolved")

except OperationalError:

return JsonResponse(

{"error": trans("We encountered a problem")},

status=status.HTTP_503_SERVICE_UNAVAILABLE,

)

Any ideas?

Melvyn Sopacua

unread,

Jun 7, 2017, 10:11:03 AM6/7/17

to django...@googlegroups.com

On Wednesday 07 June 2017 00:15:29 Damian Myerscough wrote:

> For example, if the ORM cannot query the database I would like to

> return a custom error message.

>

> ==> gunicorn.log <==

> [2017-06-07 06:06:01 +0000] [13] [DEBUG] GET /incidents/

> [2017-06-07 06:06:11 +0000] [8] [CRITICAL] WORKER TIMEOUT (pid:13)

> try:

> incidents =

> Incidents.objects.filter(user_and_team=request.user.client).exclude(st

> atus="Resolved") except OperationalError:

> return JsonResponse(

> {"error": trans("We encountered a problem")},

> status=status.HTTP_503_SERVICE_UNAVAILABLE,

> )

>

> Any ideas?

Because your timeout hits before your code gets to except.

--

Melvyn Sopacua

Damian Myerscough

unread,

Jun 7, 2017, 7:19:46 PM6/7/17

to Django users

When trying this using the Django test web server I am still unable to catch this error. When try the same query

using a shell I can catch the exception.

---

Environment:

Request Method: GET

Request URL: http://127.0.0.1:8000/accounts/register/

Django Version: 1.11

Python Version: 3.6.0

Installed Applications:

('django.contrib.admin',

'django.contrib.auth',

'django.contrib.contenttypes',

'django.contrib.sessions',

'django.contrib.messages',

'django.contrib.staticfiles',

'django.contrib.sites',

'rest_framework',

'rest_framework.authtoken',

'registration',

'incidents',

'accounts',

'debug_toolbar',

'django_extensions',

'django_nose')

Installed Middleware:

['django.middleware.security.SecurityMiddleware',

'django.contrib.sessions.middleware.SessionMiddleware',

'django.middleware.common.CommonMiddleware',

'django.middleware.csrf.CsrfViewMiddleware',

'django.contrib.auth.middleware.AuthenticationMiddleware',

'django.contrib.messages.middleware.MessageMiddleware',

'django.middleware.clickjacking.XFrameOptionsMiddleware',

'debug_toolbar.middleware.DebugToolbarMiddleware']

Traceback:

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/base/base.py" in ensure_connection

213. self.connect()

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/base/base.py" in connect

189. self.connection = self.get_new_connection(conn_params)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/mysql/base.py" in get_new_connection

274. conn = Database.connect(**conn_params)

File "/Users/damian/env36/lib/python3.6/site-packages/MySQLdb/__init__.py" in Connect

86. return Connection(*args, **kwargs)

File "/Users/damian/env36/lib/python3.6/site-packages/MySQLdb/connections.py" in __init__

204. super(Connection, self).__init__(*args, **kwargs2)

The above exception ((2003, "Can't connect to MySQL server on '127.0.0.1' (61)")) was the direct cause of the following exception:

File "/Users/damian/env36/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner

41. response = get_response(request)

File "/Users/damian/env36/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response

187. response = self.process_exception_by_middleware(e, request)

File "/Users/damian/env36/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response

185. response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/Users/damian/Development/mirustatus/mirustatus/accounts/utils.py" in as_view

10. if request.user.is_authenticated():

File "/Users/damian/env36/lib/python3.6/site-packages/django/utils/functional.py" in inner

238. self._setup()

File "/Users/damian/env36/lib/python3.6/site-packages/django/utils/functional.py" in _setup

385. self._wrapped = self._setupfunc()

File "/Users/damian/env36/lib/python3.6/site-packages/django/contrib/auth/middleware.py" in <lambda>

24. request.user = SimpleLazyObject(lambda: get_user(request))

File "/Users/damian/env36/lib/python3.6/site-packages/django/contrib/auth/middleware.py" in get_user

12. request._cached_user = auth.get_user(request)

File "/Users/damian/env36/lib/python3.6/site-packages/django/contrib/auth/__init__.py" in get_user

213. user = backend.get_user(user_id)

File "/Users/damian/env36/lib/python3.6/site-packages/django/contrib/auth/backends.py" in get_user

102. user = UserModel._default_manager.get(pk=user_id)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/manager.py" in manager_method

85. return getattr(self.get_queryset(), name)(*args, **kwargs)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/query.py" in get

373. num = len(clone)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/query.py" in __len__

232. self._fetch_all()

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/query.py" in _fetch_all

1102. self._result_cache = list(self._iterable_class(self))

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/query.py" in __iter__

53. results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/sql/compiler.py" in execute_sql

863. sql, params = self.as_sql()

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/sql/compiler.py" in as_sql

420. where, w_params = self.compile(self.where) if self.where is not None else ("", [])

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/sql/compiler.py" in compile

373. sql, params = node.as_sql(self, self.connection)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/sql/where.py" in as_sql

79. sql, params = compiler.compile(child)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/sql/compiler.py" in compile

373. sql, params = node.as_sql(self, self.connection)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/lookups.py" in as_sql

169. lhs_sql, params = self.process_lhs(compiler, connection)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/lookups.py" in process_lhs

162. db_type = self.lhs.output_field.db_type(connection=connection)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/models/fields/__init__.py" in db_type

640. return connection.data_types[self.get_internal_type()] % data

File "/Users/damian/env36/lib/python3.6/site-packages/django/utils/functional.py" in __get__

35. res = instance.__dict__[self.name] = self.func(instance)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/mysql/base.py" in data_types

174. if self.features.supports_microsecond_precision:

File "/Users/damian/env36/lib/python3.6/site-packages/django/utils/functional.py" in __get__

35. res = instance.__dict__[self.name] = self.func(instance)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/mysql/features.py" in supports_microsecond_precision

53. return self.connection.mysql_version >= (5, 6, 4) and Database.version_info >= (1, 2, 5)

File "/Users/damian/env36/lib/python3.6/site-packages/django/utils/functional.py" in __get__

35. res = instance.__dict__[self.name] = self.func(instance)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/mysql/base.py" in mysql_version

385. with self.temporary_connection() as cursor:

File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py" in __enter__

82. return next(self.gen)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/base/base.py" in temporary_connection

591. cursor = self.cursor()

File "/Users/damian/env36/lib/python3.6/site-packages/debug_toolbar/panels/sql/tracking.py" in cursor

48. return state.Wrapper(connection._djdt_cursor(*args, **kwargs), connection, panel)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/base/base.py" in cursor

254. return self._cursor()

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/base/base.py" in _cursor

229. self.ensure_connection()

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/base/base.py" in ensure_connection

213. self.connect()

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/utils.py" in __exit__

94. six.reraise(dj_exc_type, dj_exc_value, traceback)

File "/Users/damian/env36/lib/python3.6/site-packages/django/utils/six.py" in reraise

685. raise value.with_traceback(tb)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/base/base.py" in ensure_connection

213. self.connect()

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/base/base.py" in connect

189. self.connection = self.get_new_connection(conn_params)

File "/Users/damian/env36/lib/python3.6/site-packages/django/db/backends/mysql/base.py" in get_new_connection

274. conn = Database.connect(**conn_params)

File "/Users/damian/env36/lib/python3.6/site-packages/MySQLdb/__init__.py" in Connect

86. return Connection(*args, **kwargs)

File "/Users/damian/env36/lib/python3.6/site-packages/MySQLdb/connections.py" in __init__

204. super(Connection, self).__init__(*args, **kwargs2)

Exception Type: OperationalError at /accounts/register/

Exception Value: (2003, "Can't connect to MySQL server on '127.0.0.1' (61)")

---

Melvyn Sopacua

unread,

Jun 8, 2017, 1:48:33 AM6/8/17

to django...@googlegroups.com

On Wednesday 07 June 2017 16:19:46 Damian Myerscough wrote:

> When trying this using the Django test web server I am still unable to

> catch this error. When try the same query

> using a shell I can catch the exception.

> The above exception ((2003, "Can't connect to MySQL server on

> '127.0.0.1' (61)")) was the direct cause of the following exception:

>

> File

> "/Users/damian/env36/lib/python3.6/site-packages/django/core/handlers/

> exception.py" in inner

> 41. response = get_response(request)

>

> File

> "/Users/damian/env36/lib/python3.6/site-packages/django/core/handlers/

> base.py" in _get_response

> 187. response =

> self.process_exception_by_middleware(e, request)

>

> File

> "/Users/damian/env36/lib/python3.6/site-packages/django/core/handlers/

> base.py" in _get_response

> 185. response = wrapped_callback(request,

> *callback_args, **callback_kwargs)

>

> File

> "/Users/damian/Development/mirustatus/mirustatus/accounts/utils.py"

> in as_view

> 10. if request.user.is_authenticated():

That's the line of code you need to wrap, which is probably earlier in the code then your incidents filter.

--

Melvyn Sopacua

Antonis Christofides

unread,

Jun 8, 2017, 2:11:27 AM6/8/17

to django...@googlegroups.com

Hi,

not answering exactly what you asked, by my 2 cents anyway:

Why do you want to do this? Why does the user care whether the error was in the database or a bug in your program or a filesystem error or a network error or Redis being down or whatever? Neither is this information useful nor is it interesting to the user. Plus it introduces unnecessary complexity in your code.

Possibly a better way of doing that is to use handler500. However, attempting to do smart things when an error has occurred is quite risky and more often than not it doesn't work. So I am against changing the default handler500 with no compelling reason. Usually just defining a nice 500.html file is all you need.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/719ff783-b448-4644-970e-34282185465a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Melvyn Sopacua

unread,

Jun 8, 2017, 5:03:45 AM6/8/17

to django...@googlegroups.com

On Thursday 08 June 2017 09:09:32 Antonis Christofides wrote:

> not answering exactly what you asked, by my 2 cents anyway:

>

> Why do you want to do this?

Erm, that's actually good practice. This error isn't permanent and can resolve itself, so you can inform the user to try again in 5 minutes.

Granted, for a normal public website, you've lost the user already. But for a web application used by subscribers, they will grab a coffee and convo at the watercooler and try again.