[Django] #18404: SuspiciousOperation exception is thrown if application static path contains non-ascii characters
Django
non-ascii characters
-------------------------------------+--------------------
Reporter: andkit@… | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+--------------------
`AppStaticStorage` sets its location based on `__file__` of the specific
application,
but when `FileSystemStore.path` checks the location the bytestring will be
converted to unicode (by decoding from utf-8) leading to a
`SuspiciousOperation` exception (at least on a windows-machine with a
german locale, but I believe this will happen on all systems with non-utf8
filesystem encoding)
Attached patch uses the same approach as the template loaders and will
decode the path using the filesystem encoding before further processing
happens
--
Ticket URL: <https://code.djangoproject.com/ticket/18404>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
non-ascii characters
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by darkpixel):
* cc: aaron@… (added)
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:1>
Django
non-ascii characters
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by jezdez):
I can't reproduce the problem, can you provide a test case demonstrating
it?
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:2>
Django
non-ascii characters
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by andkit@…):
Replying to [comment:2 jezdez]:
> I can't reproduce the problem, can you provide a test case demonstrating
it?
I have attached a sample project. The zip file should contains a directory
with a german u-umlaut in latin1 encoding. Set your encoding (export
LC_CTYPE=de_DE.ISO-8859-1 should do), unpack anywhere run manage.py
runserver and access http://127.0.0.1:8000/ in your browser. (in case
unziping doesnt preserve the encoding, you can create a suitable directory
with `os.mkdir(u"kap\xfct".encode("latin1"))`)
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:3>
Django
non-ascii characters
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by aaugustin):
I tried the sample project but `runserver` didn't even start:
{{{
(django-dev)myk@mYk bug18404 % locale
~/Downloads/kap%FCt/bug18404
LANG="fr_FR.ISO-8859-1"
LC_COLLATE="fr_FR"
LC_CTYPE="fr_FR"
LC_MESSAGES="fr_FR"
LC_MONETARY="fr_FR"
LC_NUMERIC="fr_FR"
LC_TIME="fr_FR"
LC_ALL="fr_FR"
(django-dev)myk@mYk bug18404 % python manage.py runserver --traceback
~/Downloads/kap%FCt/bug18404
Traceback (most recent call last):
File "/Users/myk/Documents/dev/django-
trunk/django/core/management/base.py", line 222, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Users/myk/Documents/dev/django-
trunk/django/core/management/base.py", line 247, in execute
translation.activate('en-us')
File "/Users/myk/Documents/dev/django-
trunk/django/utils/translation/__init__.py", line 89, in activate
return _trans.activate(language)
File "/Users/myk/Documents/dev/django-
trunk/django/utils/translation/trans_real.py", line 179, in activate
_active.value = translation(language)
File "/Users/myk/Documents/dev/django-
trunk/django/utils/translation/trans_real.py", line 168, in translation
default_translation = _fetch(settings.LANGUAGE_CODE)
File "/Users/myk/Documents/dev/django-
trunk/django/utils/translation/trans_real.py", line 151, in _fetch
apppath = os.path.join(os.path.dirname(app.__file__), 'locale')
File "/Users/myk/.virtualenvs/django-
dev/bin/../lib/python2.7/posixpath.py", line 71, in join
path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfc in position 24:
ordinal not in range(128)
}}}
Apparently my terminal (under OS X) doesn't support `export
LC_ALL=fr_FR.ISO-8859-1`; that activates the C locale instead.
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:4>
Django
non-ascii characters
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
* cc: m.r.sopacua@… (added)
Comment:
Replying to [comment:4 aaugustin]:
> Apparently my terminal (under OS X) doesn't support `export
LC_ALL=fr_FR.ISO-8859-1`; that activates the C locale instead.
That's cause on FreeBSD derived systems the locale would be
fr_FR.ISO8859-1.
The root cause is that u umlaut in latin-1 conflicts with UTF-8 encoding,
where the value is used as part of the variable size encoding scheme:
{{{
>>> u = unicode('\xfc', 'latin1')
>>> u.encode('utf-8')
'\xc3\xbc'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:5>
Django
non-ascii characters
Reporter: andkit@… | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by lrekucki):
* needs_tests: 0 => 1
* stage: Unreviewed => Accepted
Comment:
Unfortunately, the `abspathu` used inside `FileSystemStorage.__init`
doesn't return unicode if not given unicode:
{{{
>>> import sys
>>> sys.path
['', 'C:\\Temp\\g\xb9ska', ...snip... ]
>>> import foo
>>> foo.__file__
'C:\\Temp\\g\xb9ska\\foo\\__init__.py'
>>> from django.core.files.storage import *
>>> st = FileSystemStorage(os.path.join(os.path.dirname(foo.__file__),
"static"), base_url=u"/foo")
>>> st.path("bar")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "django\core\files\storage.py", line 259, in path
raise SuspiciousOperation("Attempted access to '%s' denied." % name)
django.core.exceptions.SuspiciousOperation: Attempted access to 'bar'
denied.
}}}
So this is definitly a bug in FileSystemStorage.
Note that on master (1.5) this will fail earlier (i.e. on every
`os.path.join`), because of `unicode_literals` switch.
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:6>
Django
non-ascii characters
Reporter: andkit@… | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by lrekucki):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:7>
Django
non-ascii characters
Reporter: andkit@… | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by fhahn):
* needs_better_patch: 1 => 0
* version: 1.4 => master
* needs_tests: 1 => 0
Comment:
I've added a test for the patch and created a pull request:
https://github.com/django/django/pull/814
I put to decoding in FileSystemStorage.__init__, because AppStaticStorage
is only a subclass of FileSystemStorage.
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:8>
Django
non-ascii characters
Reporter: andkit@… | Owner: fhahn
Type: Bug | Status: assigned
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Changes (by fhahn):
* keywords: => sprint2013
* owner: nobody => fhahn
* status: new => assigned
Comment:
I had another look and reworked my patch. I've added a test for
AppStaticStorage with non ascii characters in the module path.
I think this bug was fixed by the changes introduced by #19357.
upath is used to convert the file name to an unicode string:
https://github.com/fhahn/django/blob/ticket_18404/django/contrib/staticfiles/storage.py#L300
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:9>
Django
non-ascii characters
Reporter: andkit@… | Owner: fhahn
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution: fixed
Keywords: sprint2013 | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Changes (by Tim Graham <timograham@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"ca39c0a6becf497880181b3a5ef6db84130f9723"]:
{{{
#!CommitTicketReference repository=""
revision="ca39c0a6becf497880181b3a5ef6db84130f9723"
Fixed #18404 -- Added test for AppStaticStorage with non ascii path
(bug was already fixed in #19357)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18404#comment:10>