[Django] #23350: mod_wsgi authentication/authorization docs result in extra memory usage
Django
---------------------------------+--------------------
Reporter: GrahamDumpleton | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.6
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------
The documentation at:
* https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/apache-auth/
uses the examples:
{{{
WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com
WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django
<Location "/secret">
AuthType Basic
AuthName "Top Secret"
Require valid-user
AuthBasicProvider wsgi
WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
</Location>
}}}
and
{{{
WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django
<Location "/secret">
AuthType Basic
AuthName "Top Secret"
AuthBasicProvider wsgi
WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py
Require group secret-agents
Require valid-user
</Location>
}}}
Although these will work, they cause the Django application code to be
loaded more than once into the same process. That is, you end up with an
instance in one sub interpreter which handles web request, and another
instance in another sub interpreter which handles just authentication and
authorisation.
Ideally you do no want two copies as that simply increases overall memory
usage.
The original mod_wsgi documentation does point to this in the statements:
''By default the auth providers are executed in context of first
interpreter created by Python, ie., '%{GLOBAL}' and always in the Apache
child processes, never in a daemon process. The interpreter can be
overridden using the 'application-group' option to the script directive.
The namespace for authentication groups is shared with that for
application groups defined by WSGIApplicationGroup.''
''Because the auth provider is always run in the Apache child
processes and never in the context of a mod_wsgi daemon process, if the
authentication check is making use of the internals of some Python web
framework, it is recommended that the application using that web framework
also be run in embedded mode and the same application group. This is the
case as the Python web frameworks often bring in a huge amount of code
even if using only one small part of them. This will result in a lot of
memory being used in the Apache child processes just to support the auth
provider.''
To combat this double of memory usage the mod_wsgi documentation gives the
example of:
{{{
WSGIAuthUserScript /usr/local/django/mysite/apache/auth.wsgi application-
group=django
}}}
It is this use of the {{{application-group}}} argument for
{{{WSGIAuthUserScript}}} and {{{WSGIAuthGroupScript}}} that the Django
documentation doesn't use.
The Django examples therefore should really use:
{{{
WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com
WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django
<Location "/secret">
AuthType Basic
AuthName "Top Secret"
Require valid-user
AuthBasicProvider wsgi
WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py application-
group=django
</Location>
}}}
and
{{{
WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django
<Location "/secret">
AuthType Basic
AuthName "Top Secret"
AuthBasicProvider wsgi
WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py application-
group=django
WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py application-
group=django
Require group secret-agents
Require valid-user
</Location>
}}}
On the presumption that there is only the one Django application, it may
actually be better to turn that around and force the Django application to
run in the main interpreter anyway. This is better as some third party
extension modules for Python do not work properly in sub interpreters.
Thus may be better to use:
{{{
WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com
WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}
<Location "/secret">
AuthType Basic
AuthName "Top Secret"
Require valid-user
AuthBasicProvider wsgi
WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
</Location>
}}}
and
{{{
WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}
<Location "/secret">
AuthType Basic
AuthName "Top Secret"
AuthBasicProvider wsgi
WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py
Require group secret-agents
Require valid-user
</Location>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23350>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: GrahamDumpleton | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Documentation | Version: 1.6
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* needs_better_patch: => 0
* component: Uncategorized => Documentation
* needs_tests: => 0
* needs_docs: => 0
* type: Uncategorized => Cleanup/optimization
* stage: Unreviewed => Accepted
Comment:
I believe we should provide a copy-paste-proof example for the simple case
of "first time I'm touching Django and Apache, how do I run my test
project?" and shell out to mod_wsgi's own docs for anything more
complicated.
For that use case, we can assume that there's only one Django application.
We just need:
- to make sure our documentation stays internally consistent — we don't
want the mod_wsgi auth docs to have subtle differences with the mod_wsgi
main docs.
- to add a warning about running multiple Django projets in the same
server — which Apache's lovely handling of env variables makes hard anyway
— and leave that part of the explaination up to mod_wsgi :)
--
Ticket URL: <https://code.djangoproject.com/ticket/23350#comment:1>
Django
Reporter: GrahamDumpleton | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Documentation | Version: 1.6
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
--------------------------------------+------------------------------------
Changes (by evildmp):
* keywords: => afraid-to-commit
* easy: 0 => 1
Comment:
I've marked this ticket as especially suitable for people following the
'''Don't be afraid to commit tutorial''' at the DjangoCon US 2014
sprints. If you're tackling this ticket, please don't hesitate to ask me
for guidance if you'd like any, either here or on the Django IRC channels,
where I can be found as ''EvilDMP''.
--
Ticket URL: <https://code.djangoproject.com/ticket/23350#comment:2>
Django
Reporter: GrahamDumpleton | Owner: x110dc
Type: Cleanup/optimization | Status: assigned
Component: Documentation | Version: 1.6
Severity: Normal | Resolution:
Keywords: afraid-to-commit | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------
* status: new => assigned
* owner: nobody => x110dc
--
Ticket URL: <https://code.djangoproject.com/ticket/23350#comment:3>
Django
Reporter: GrahamDumpleton | Owner: x110dc
Type: Cleanup/optimization | Status: assigned
Component: Documentation | Version: 1.6
Severity: Normal | Resolution:
Keywords: afraid-to-commit | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by x110dc):
I made the changes listed above here:
https://github.com/django/django/pull/3176
--
Ticket URL: <https://code.djangoproject.com/ticket/23350#comment:4>
Django
Reporter: GrahamDumpleton | Owner: x110dc
Type: | Status: assigned
Cleanup/optimization | Version: 1.6
Component: Documentation | Resolution:
Severity: Normal | Triage Stage: Ready for
Keywords: afraid-to-commit | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Changes (by collinanderson):
* has_patch: 0 => 1
* stage: Accepted => Ready for checkin
Comment:
GrahamDumpleton approves the change, though doesn't address aaugustin's
concerns.
--
Ticket URL: <https://code.djangoproject.com/ticket/23350#comment:5>
Django
Reporter: GrahamDumpleton | Owner: x110dc
Type: | Status: assigned
Cleanup/optimization | Version: 1.6
Component: Documentation | Resolution:
Severity: Normal | Triage Stage: Ready for
Keywords: afraid-to-commit | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by timgraham):
I am not an expert, but reading through the ticket, it seems like a
warning to the effect of "The use of `WSGIApplicationGroup %{GLOBAL}`
presumes that your Apache instance is running only one Django application.
If you are running more than Django application, please refer to the
mod_wsgi docs for more information about this setting."
--
Ticket URL: <https://code.djangoproject.com/ticket/23350#comment:6>
Django
Reporter: GrahamDumpleton | Owner: x110dc
Cleanup/optimization | Version: 1.6
Component: Documentation | Resolution: fixed
Severity: Normal | Triage Stage: Ready for
Keywords: afraid-to-commit | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"c7f7432be54695ea16a2ba1cc5f8e5e1facf43f7"]:
{{{
#!CommitTicketReference repository=""
revision="c7f7432be54695ea16a2ba1cc5f8e5e1facf43f7"
Fixed #23350 -- Updated mod_wsgi auth example to use less memory.
Thanks Graham Dumpleton for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23350#comment:7>
Django
Reporter: GrahamDumpleton | Owner: x110dc
Type: | Status: closed
Cleanup/optimization | Version: 1.6
Component: Documentation | Resolution: fixed
Severity: Normal | Triage Stage: Ready for
Keywords: afraid-to-commit | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"5767bc722ff9d65b20226a0aff438b38b46cc594"]:
{{{
#!CommitTicketReference repository=""
revision="5767bc722ff9d65b20226a0aff438b38b46cc594"
[1.6.x] Fixed #23350 -- Updated mod_wsgi auth example to use less memory.
Thanks Graham Dumpleton for the report.
Backport of c7f7432be5 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23350#comment:8>
Django
Reporter: GrahamDumpleton | Owner: x110dc
Type: | Status: closed
Cleanup/optimization | Version: 1.6
Component: Documentation | Resolution: fixed
Severity: Normal | Triage Stage: Ready for
Keywords: afraid-to-commit | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"84b50718abf1cd9883e39faff6aa7fc59fd5b75d"]:
{{{
#!CommitTicketReference repository=""
revision="84b50718abf1cd9883e39faff6aa7fc59fd5b75d"
[1.7.x] Fixed #23350 -- Updated mod_wsgi auth example to use less memory.
Thanks Graham Dumpleton for the report.
Backport of c7f7432be5 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23350#comment:9>