[Django] #33584: Encourage using persistent database connections
Django
-------------------------------------+-------------------------------------
Reporter: Tom | Owner: nobody
Carrick |
Type: New | Status: assigned
feature |
Component: Database | Version: dev
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Sometimes I'm asked to review a Django project's performance. The most
typical issues are probably N+1 selects and wasting time creating
connections.
I propose encouraging setting `CONN_MAX_AGE` to a non-zero value by:
- Setting `CONN_MAX_AGE` to `None` or a reasonably high value in the
generated `settings.py` - but keeping the default as-is for compatibility.
- Adding a deployment level system check to guard against a zero value.
Anyone who needs this to be `0` can easily change it and silence the
system check, but I think most projects will benefit from this.
--
Ticket URL: <https://code.djangoproject.com/ticket/33584>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
| Carrick
Type: New feature | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Tom Carrick
--
Ticket URL: <https://code.djangoproject.com/ticket/33584#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Carrick | Owner: Tom
| Carrick
Component: Database layer | Version: dev
(models, ORM) |
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Florian Apolloner (added)
* status: assigned => closed
* resolution: => needsinfo
Comment:
Hi Tom. Thanks for the suggestion here.
It looks like this has come up before, and there's a problem with
runserver compatibility: [https://groups.google.com/g/django-
developers/c/rH0QQP7tI6w/m/yBusiFTNBR4J mailing list post from 2013].
([https://groups.google.com/g/django-developers/search?q=CONN_MAX_AGE
Mailing list search for CONN_MAX_AGE])
Do these points still apply? If so, is there a way around them? Did anyone
explore the idea from the mailing list thread?
Also relevant is #33497 -- ''"Database persistent connections do not work
with ASGI in 4.0"'' — Question then: is it clear cut that persistent
connections are better — not just for this ASGI case but in general? (If
you have lots of traffic, sure, but... 🤷) From that ticket, I know
Florian is thinking about a connection pool, which would be connected.
Whilst generally it seems ''Sure, yeah'', I think some wider discussion is
needed, even if that just results in ''Clearly, yeah''.
(DevelopersMailingList :)
I'll mark this needs info in the meantime.
--
Ticket URL: <https://code.djangoproject.com/ticket/33584#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Carrick | Owner: Tom
| Carrick
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tom Carrick):
Thanks, makes sense. I'll do a bit of digging and see where I end up.
--
Ticket URL: <https://code.djangoproject.com/ticket/33584#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Carrick | Owner: Tom
| Carrick
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Florian Apolloner):
Replying to [comment:2 Carlton Gibson]:
> It looks like this has come up before, and there's a problem with
runserver compatibility: [https://groups.google.com/g/django-
developers/c/rH0QQP7tI6w/m/yBusiFTNBR4J mailing list post from 2013].
I think this still holds true. Unless we change runserver to use a
threadpool (or db pool) it doesn't make much sense to use persistent
connections there.
--
Ticket URL: <https://code.djangoproject.com/ticket/33584#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Carrick | Owner: Tom
| Carrick
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tom Carrick):
Okay, I think it's best to revisit this once (if?) we have connection
pooling.
--
Ticket URL: <https://code.djangoproject.com/ticket/33584#comment:5>