[Django] #30424: Queries within AppConfig.ready() can cause issues with some Django db commands
Django
commands
------------------------------------------+------------------------
Reporter: Rich Rauenzahn | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.11
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------+------------------------
I can recreate this in my `AppConfig` simply with the following inside of
`ready()`:
{{{
list(MyModel.objects.all())
}}}
If you then run `manage dbshell` you can see the connection still existing
after invoking `psql`:
{{{
$ manage dbshell
Settings: myproj.config.rich.DevelopmentSettings
Expanded display is used automatically.
Null display is "¤".
psql (9.2.24)
Type "help" for help.
myproj=> SELECT *
FROM pg_stat_activity;
-[ RECORD 1
]----+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
datid | 4855283
datname | myproj
pid | 2590
usesysid | 16384
usename | myproj
application_name | psql
client_addr | 10.20.72.150
client_hostname | ¤
client_port | 53364
backend_start | 2019-04-29 09:53:14.044483-07
xact_start | 2019-04-29 09:53:15.280585-07
query_start | 2019-04-29 09:53:15.280585-07
state_change | 2019-04-29 09:53:15.28059-07
waiting | f
state | active
query | SELECT *
| FROM pg_stat_activity;
-[ RECORD 2
]----+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
datid | 4855283
datname | myproj
pid | 2589
usesysid | 16384
usename | myproj
application_name |
client_addr | 10.20.72.150
client_hostname | ¤
client_port | 53362
backend_start | 2019-04-29 09:53:14.013457-07
xact_start | ¤
query_start | 2019-04-29 09:53:14.018469-07
state_change | 2019-04-29 09:53:14.020801-07
waiting | f
state | idle
query | SELECT ...
myproj=>
}}}
This can cause problems with trying to drop the database from the manage
command as the database in still in use by another connection.
Work around seems to be adding this to the end of `ready()`:
{{{
from django.db import connections
connections.close_all()
}}}
My guess is the underlying fork to `psql` isn't closing all file handles
and `psql` is inheriting (but not managing) them.
--
Ticket URL: <https://code.djangoproject.com/ticket/30424>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
commands
Reporter: Rich Rauenzahn | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.11
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by Rich Rauenzahn:
Old description:
New description:
{{{
list(MyModel.objects.all())
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30424#comment:1>
Django
commands
Reporter: Rich Rauenzahn | Owner: nobody
Component: Uncategorized | Version: 1.11
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette):
* status: new => closed
* resolution: => invalid
Comment:
Performing database operations at module loading time should be avoided
and is warned against in the `ready()` documentation.
https://docs.djangoproject.com/en/2.2/ref/applications/#django.apps.AppConfig.ready
Performing such operations at `ready()` will have various side effects
that are hard to enumerate as it's unexpected and I'm not convinced
documenting this one is worthwhile.
--
Ticket URL: <https://code.djangoproject.com/ticket/30424#comment:2>
Django
commands
Reporter: Rich Rauenzahn | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.11
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Rich Rauenzahn):
* status: closed => new
* resolution: invalid =>
Comment:
I didn't consider this a documentation defect -- I think the team may want
to consider the merits of this bug instead whether they really want to
leak file handles across the fork()/exec() system call in the management
commands.
Python 34 changes the default to close file handles at fork, so this is
only an issue for 2.7 support in 1.11. Maybe that's enough reason to
close it.
https://www.python.org/dev/peps/pep-0446/
Leaking file descriptors in child processes causes various
annoying issues and is a known major security vulnerability. Using the
subprocess module with the close_fds parameter set to True is not possible
in all cases.
(This file handle leakage is a much more subtle issue than what the
warning in the docs implies.)
Aside: Is there any supported way for running db commands at startup?
--
Ticket URL: <https://code.djangoproject.com/ticket/30424#comment:3>
Django
commands
Reporter: Rich Rauenzahn | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.11
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Rich Rauenzahn):
Relevant SO for other (better supported?) methods of app startup code:
https://stackoverflow.com/questions/6791911/execute-code-when-django-
starts-once-only
--
Ticket URL: <https://code.djangoproject.com/ticket/30424#comment:4>
Django
commands
Reporter: Rich Rauenzahn | Owner: nobody
Component: Uncategorized | Version: 1.11
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson):
* status: new => closed
* resolution: => invalid
Comment:
Sorry, this is a usage question. TicketClosingReasons/UseSupportChannels
Also TicketClosingReasons/DontReopenTickets
Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/30424#comment:5>