[Django] #34028: Django 'static' template tag fails to generate URLs with SCRIPT_NAME prefix
Django
-------------------------------------------+------------------------
Reporter: Stewart Adam | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 4.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------------+------------------------
According to the docs, when MEDIA_URL and STATIC_URL are relative (now the
default), SCRIPT_NAME should automatically be prepended to them when it is
set to ensure that paths are generated correctly. This appears to work
everywhere *except* when calling `{% static %}` in HTML templates.
Best I can tell, when this functionality was first added, it appears that
SCRIPT_NAME was evaluated every request:
https://github.com/django/django/commit/c574bec0929cd2527268c96a492d25223a9fd576
Looking at this same file today, it appears that setting values are now
cached in a dict. When the Django application initializes, it reads
SCRIPT_NAME as / and so all subsequent uses of `{% static %}` are output
without the current value of SCRIPT_NAME. This happens even if using a
hardcoded FORCE_SCRIPT_NAME.
url() appears to be unaffected by the same issue, so the net result is
that links and redirects honor SCRIPT_NAME as expected, and static files
are served using the SCRIPT_NAME prefix, but all output from HTML
templates simply use `/static/...` for their links which causes all
css/js/favicon/related media to fail to load with a 404.
--
Ticket URL: <https://code.djangoproject.com/ticket/34028>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Stewart Adam | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 4.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Stewart Adam):
Related issues elsewhere:
- https://stackoverflow.com/questions/45557034/django-static-url-when-
script-name-set
- https://github.com/evansd/whitenoise/issues/88
- https://github.com/TandoorRecipes/recipes/issues/1878
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:1>
Django
---------------------------------+--------------------------------------
Reporter: Stewart Adam | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------
Comment (by Stewart Adam):
The code path leading to the issue appears to be:
1. `static()` handler for the template tag calls
`StaticNode.handle_simple()` in
[https://github.com/django/django/blob/main/django/templatetags/static.py#L174
templatetags/static.py L174]
2. `StaticNode.handle_simple()` calls `staticfiles_storage.url()` at
[https://github.com/django/django/blob/main/django/templatetags/static.py#L127
templatetags/static.py L127]
3. `staticfiles_storage.url()` is backed by a `FileSystemStorage`:
[https://github.com/django/django/blob/main/django/core/files/storage.py#L392
core/files/storage.py L392]
None of these calls attempt to verify `SCRIPT_NAME` or inject
`STATIC_URL`. An
[https://github.com/django/django/blob/main/django/templatetags/static.py#L131
alternate code path] during step 2 *does* appear to inject `STATIC_URL`.
however to due application settings now being cached into a dict/module
the value does not reflect the current value of `SCRIPT_NAME` and uses the
value at application initialization (empty string) instead, so STATIC_URL
is always just `/static/`.
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:2>
Django
---------------------------------+--------------------------------------
Reporter: Stewart Adam | Owner: nobody
Component: Template system | Version: 4.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------
* status: new => closed
* resolution: => worksforme
Comment:
Hi,
I cannot reproduce this. It works as it should:
{{{#!python
from django.template import engines
django_engine = engines['django']
from django.conf import settings
print(settings.FORCE_SCRIPT_NAME)
print(settings.STATIC_URL) # Note: in settings.py it is 'static/' but the
output here will include FORCE_SCRIPT_NAME
print(django_engine.from_string("{% load static %}{% static 'test_file'
%}").render())
}}}
> None of these calls attempt to verify SCRIPT_NAME or inject STATIC_URL.
This isn't true: `{% static %}` uses `staticfiles_storage` which in turn
uses `StaticFilesStorage` which sets the prefix to `STATIC_URL` which
includes `FORCE_SCRIPT_NAME`. Feel free to reopen with a reproducer if you
are still running into issues.
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:3>
Django
---------------------------------+--------------------------------------
Reporter: Stewart Adam | Owner: nobody
Type: Bug | Status: closed
Component: Template system | Version: 4.0
Severity: Normal | Resolution: worksforme
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------
Comment (by Florian Apolloner):
Setting a custom script name also works:
{{{
#!python
from django.urls.base import *
set_script_prefix('/lala')
from django.template import engines
django_engine = engines['django']
from django.conf import settings
print(settings.FORCE_SCRIPT_NAME)
print(settings.STATIC_URL) # Note: in settings.py it is 'static/' but the
output here will include FORCE_SCRIPT_NAME
print(django_engine.from_string("{% load static %}{% static 'test_file'
%}").render())
}}}
{{{
None
/lala/static/
/lala/static/test_file
}}}
That said it is true that the value of `SCRIPT_NAME` is assumed to be
static and not change over the runtime of Django. If Django is "loaded"
outside of a webcontext first, then you will see the behavior you are
describing. I'll accept the ticket for now but I am not sure if this is
fixable easily or at all.
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:4>
Django
---------------------------------+------------------------------------
Component: Template system | Version: 4.0
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
* status: closed => new
* resolution: worksforme =>
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:5>
Django
---------------------------------+------------------------------------
Reporter: Stewart Adam | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Comment (by Stewart Adam):
Replying to [comment:4 Florian Apolloner]:
> {{{
> None
> /lala/static/
> /lala/static/test_file
> }}}
Admittedly I'm not a Django export, I'm just trying to troubleshoot
another OSS app I use that's based on Django. Can you share how you're
running this?
That said, when I put this code into view on a sample Django app (django-
admin startproject repro), I can confirm the behavior that the first
request made to the server sets the prefix for the lifetime of the app.
Code at https://github.com/stewartadam/django-static-repro
{{{
./run.sh &
# try using SCRIPT_NAME to set the prefix on first request
curl -H 'SCRIPT_NAME: /prefix' http://localhost:8000/prefix/foo/
FORCE_SCRIPT_NAME: None
STATIC_URL: /prefix/static/
call static for test_file: /prefix/static/test_file
# second request paths correctly with /another, but uses first request's
/prefix for all variable values/output
curl -H 'SCRIPT_NAME: /another' localhost:8000/another/foo/
FORCE_SCRIPT_NAME: None
STATIC_URL: /prefix/static/
call static for test_file: /prefix/static/test_file
}}}
FORCE_SCRIPT_NAME also doesn't behave how I'd expect (overriding
SCRIPT_NAME header):
{{{
FORCE_SCRIPT_NAME=prefix ./run.sh &
# FORCE_SCRIPT_NAME is supposed to override SCRIPT_NAME header, but isn't
here in the pathing
curl -H 'SCRIPT_NAME: /another' http://localhost:8000/another/foo/
FORCE_SCRIPT_NAME: prefix
STATIC_URL: /prefix/static/
call static for test_file: /prefix/static/test_file
# if supplied SCRIPT_NAME doesn't match FORCE_SCRIPT_NAME, serve fails to
serve content
curl -I -H 'SCRIPT_NAME: /another' http://localhost:8000/prefix/foo/
HTTP/1.1 500 Internal Server Error
Connection: close
Content-Type: text/html
Content-Length: 141
}}}
> That said it is true that the value of `SCRIPT_NAME` is assumed to be
static and not change over the runtime of Django. If Django is "loaded"
outside of a webcontext first, then you will see the behavior you are
describing. I'll accept the ticket for now but I am not sure if this is
fixable easily or at all.
I agree at face value this might be an unusual thing to support, however
on many servers healthchecks or other internal requests might result in a
request coming through without SCIRPT_NAME set as it would be for end-
users via reverse proxy.
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:6>
Django
---------------------------------+------------------------------------
Reporter: Stewart Adam | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Comment (by Stewart Adam):
Had a chance to debug a bit more and caching of SCRIPT_NAME aside, even if
users use FORCE_SCRIPT_NAME to ensure the prefix is hardcoded and
correctly set at all times it looks like there's still something awry:
Whitenoise middleware sees the prefix as '/' during initialization -
set_script_prefix() only appears to be called during the first HTTP
request, but whitenoise needs to verify the user-provided values for
STATIC_URL during middleware initialization. During that middleware
initialization, the prefix remains '/' despite FORCE_SCRIPT_NAME being
set.
I've posted more details here:
https://github.com/evansd/whitenoise/issues/271#issuecomment-1257389536
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:7>
Django
---------------------------------+------------------------------------
Reporter: Stewart Adam | Owner: nobody
Type: Bug | Status: new
Component: Template system | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
* cc: David Sanders (added)
Comment:
I've been investigating `SCRIPT_NAME` / `FORCE_SCRIPT_NAME` and found some
interesting quirks when used with `runserver` which may be affecting
testing these issues (at least it was affecting me when investigating
#34185
I'm not entirely sure why yet but I'll post the following reproducible
issue (at least for me) to show that prefixing `STATIC_URL` works
intermittently due to a race condition with runserver and the multiple
processes setup with autoreload:
In `django/conf/__init__.py` I added a line to print the result of
prefixing `STATIC_URL`:
{{{
@staticmethod
def _add_script_prefix(value):
"""
Add SCRIPT_NAME prefix to relative paths.
Useful when the app is being served at a subpath and manually
prefixing
subpath to STATIC_URL and MEDIA_URL in settings is inconvenient.
"""
# Don't apply prefix to absolute paths and URLs.
if value.startswith(("http://", "https://", "/")):
return value
from django.urls import get_script_prefix
val = "%s%s" % (get_script_prefix(), value)
print("*********** PREFIXING ***********: " + val)
return val
}}}
The result when `runserver` is run with autoreload on then off:
{{{
sample-project % dj runserver
Watching for file changes with StatReloader
Performing system checks...
*********** PREFIXING ***********: /static/
System check identified no issues (0 silenced).
November 29, 2022 - 13:22:27
Django version 4.2.dev20221129074011, using settings
'django_sample.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
^C
django-sample % dj runserver --noreload
Performing system checks...
*********** PREFIXING ***********: /django/static/
System check identified no issues (0 silenced).
November 29, 2022 - 13:22:40
Django version 4.2.dev20221129074011, using settings
'django_sample.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
^C
sample-project %
}}}
Now the interesting thing is that I can get the same results without
specifying `--noreload` by just adding a line to `print()` in `setup()` !
:)
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:8>
Django
Reporter: Stewart Adam | Owner: Sarah Boyce
Type: Bug | Status: assigned
Component: Template system | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Sarah Boyce):
* owner: nobody => Sarah Boyce
* status: new => assigned
* has_patch: 0 => 1
* version: 4.0 => dev
Comment:
I was able to replicate the issue that if someone tries to access the
script prefix in a middleware init (as they do in whitenoise), the result
is unexpected.
I have raised a PR which has a regression test and a potential fix 👍
I think this relates to this ticket a bit: #16734
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:9>
Django
---------------------------------+---------------------------------------
Reporter: Stewart Adam | Owner: Sarah Boyce
Type: Bug | Status: assigned
Component: Template system | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+---------------------------------------
* has_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:10>
Django
---------------------------------+---------------------------------------
Reporter: Stewart Adam | Owner: Sarah Boyce
Type: Bug | Status: assigned
Component: Template system | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+---------------------------------------
Comment (by Mariusz Felisiak):
This can be fixed by #34461.
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:11>
Django
---------------------------------+---------------------------------------
Reporter: Stewart Adam | Owner: Sarah Boyce
Component: Template system | Version: dev
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+---------------------------------------
* status: assigned => closed
* resolution: => invalid
* stage: Accepted => Unreviewed
Comment:
Let's document this caveat (see
[https://github.com/django/django/pull/16714 PR]) and close this as
"invalid". Discussion about constructing URLs outside of the request-
response cycle (new feature) was moved to the #34461.
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:12>
Django
---------------------------------+---------------------------------------
Reporter: Stewart Adam | Owner: Sarah Boyce
Type: Bug | Status: closed
Component: Template system | Version: dev
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+---------------------------------------
Comment (by GitHub <noreply@…>):
In [changeset:"bdf59bff657975e577b86b194b39ec2f77983d2b" bdf59bf]:
{{{
#!CommitTicketReference repository=""
revision="bdf59bff657975e577b86b194b39ec2f77983d2b"
Refs #34028 -- Doc'd that get_script_prefix() cannot be used outside of
the request-response cycle.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:13>
Django
---------------------------------+---------------------------------------
Reporter: Stewart Adam | Owner: Sarah Boyce
Type: Bug | Status: closed
Component: Template system | Version: dev
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+---------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"e34a54a36e2d7b35e8998fe60462e93a1a5424cf" e34a54a3]:
{{{
#!CommitTicketReference repository=""
revision="e34a54a36e2d7b35e8998fe60462e93a1a5424cf"
[4.2.x] Refs #34028 -- Doc'd that get_script_prefix() cannot be used
outside of the request-response cycle.
Backport of bdf59bff657975e577b86b194b39ec2f77983d2b from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:14>
Django
---------------------------------+---------------------------------------
Reporter: Stewart Adam | Owner: Sarah Boyce
Type: Bug | Status: closed
Component: Template system | Version: dev
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+---------------------------------------
IMHO this (or something awefully close to it) is _still_ a bug.
1. From the 3.1 release notes:
https://docs.djangoproject.com/en/3.1/releases/3.1/#backwards-
incompatible-3-1
> The STATIC_URL and MEDIA_URL settings set to relative paths are now
prefixed by the server-provided value of SCRIPT_NAME (or / if not set).
This change should not affect settings set to valid URLs or absolute
paths.
As per the original commit message
https://github.com/django/django/commit/c574bec0929cd2527268c96a492d25223a9fd576
and in my personal opinion this also makes perfect sense... this brings
the behavior of "static" inline with that of e.g. "reverse".
2. when running the django runserver with autoreload, STATIC_URL appears
to be evaluated outside the request/response cycle (and then cached)...
this is behavior that is dicussed in this issue... I'm not sure I
personally fully understand it (haven't dived into it deeply enough). But
what I do understand is: I'm not doing the thing that's been described as
"don't do this". Nor am I using whitenoise. But I am suffering from
exactly this problem (in Django 5.2).
here's a repo that shows the problem:
https://github.com/vanschelven/poc34028
when visiting in the browser I get:
static: /static/hallo
reverse: /scriptname/
but I would expect the line with "static" to also say
"/scriptname/static/hallo"
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:15>
Django
---------------------------------+---------------------------------------
Reporter: Stewart Adam | Owner: Sarah Boyce
Type: Bug | Status: closed
Component: Template system | Version: dev
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+---------------------------------------
Comment (by Klaas van Schelven):
when using gunicorn. Only the plain runserver (which does autoreload)
exhibits the problem
--
Ticket URL: <https://code.djangoproject.com/ticket/34028#comment:16>