[Django] #20375: Post-processing CSS file with Unicode chars fails
Django
-------------------------------+--------------------
Reporter: bouke | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.4
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------
Say I have a static file like this: (icons.css)
{{{
.icon-home:before { content: '\2302'; } /* '⌂' */
.icon-play-circle2:before { content: '\e048'; } /* '' */
.icon-location:before { content: '\e724'; } /* '' */
.icon-map:before { content: '\e727'; } /* '' */
.icon-twitter-1:before { content: '\f309'; } /* '' */
.icon-facebook-1:before { content: '\f30c'; } /* '' */
.icon-doc:before { content: '📄'; } /* '\1f4c4' */
.icon-calendar:before { content: '📅'; } /* '\1f4c5' */
.icon-search:before { content: '🔍'; } /* '\1f50d' */
}}}
When this file is being post-processed (in ``collectstatic``), an
exception is raised:
{{{
Traceback (most recent call last):
File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1481, in
<module>
debugger.run(setup['file'], None, None)
File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1124, in
run
pydev_imports.execfile(file, globals, locals) #execute the script
File "/Volumes/Users/bouke/Sites/gct/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "lib/python2.7/site-packages/django/core/management/__init__.py",
line 443, in execute_from_command_line
utility.execute()
File "lib/python2.7/site-packages/django/core/management/__init__.py",
line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "lib/python2.7/site-packages/django/core/management/base.py", line
196, in run_from_argv
self.execute(*args, **options.__dict__)
File "lib/python2.7/site-packages/django/core/management/base.py", line
232, in execute
output = self.handle(*args, **options)
File "lib/python2.7/site-packages/django/core/management/base.py", line
371, in handle
return self.handle_noargs(**options)
File "lib/python2.7/site-
packages/django/contrib/staticfiles/management/commands/collectstatic.py",
line 163, in handle_noargs
collected = self.collect()
File "lib/python2.7/site-
packages/django/contrib/staticfiles/management/commands/collectstatic.py",
line 120, in collect
for original_path, processed_path, processed in processor:
File "lib/python2.7/site-
packages/django/contrib/staticfiles/storage.py", line 226, in post_process
content = pattern.sub(converter, content)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 500:
ordinal not in range(128)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/20375>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: bouke | Owner: nobody
Type: Uncategorized | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by anguslturner):
* needs_better_patch: => 0
* component: Uncategorized => contrib.staticfiles
* needs_tests: => 1
* version: 1.4 => master
* needs_docs: => 1
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:1>
Django
Reporter: bouke | Owner: nobody
Type: Uncategorized | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by djangsters):
How can I reproduce this. Here it works just fine.
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:2>
Django
Reporter: bouke | Owner: nobody
Type: Uncategorized | Status: closed
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Changes (by LukaszBalcerzak):
* status: new => closed
* resolution: => invalid
Comment:
Cannot reproduce too. Am closing for now as it doesn't seem like a bug in
Django.
Please reopen once you can reproduce. Hints: run `collectstatic` outside
of PyCharm and check css file encoding.
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:3>
Django
Reporter: bouke | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Changes (by ye0eugene0ey):
* status: closed => new
* cc: yeo.eugene.oey@… (added)
* needs_better_patch: 0 => 1
* resolution: invalid =>
* version: master => 1.4
* type: Uncategorized => Bug
Comment:
== Reproduce the Error ==
I encounter the same issues. And after tracing the type,
It happened on
[https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L226],
when trying to replace a ''unicode'' type into a ''string'' type that has
a ''unicode'' character.
{{{
import re
patt = re.compile(r"""(url\(['"]{0,1}\s*(.*?)["']{0,1}\))""")
patt.sub(lambda x: u'The replacement is a function that return unicode',
"""The content is str with at least a unicode character głó,
and don't forget to match url(../some/path)""")
}}}
== Patching?? ==
I tried to convert the type into string at
[https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L172]
and it works
{{{
return str('url("%s")' % unquote(relative_url))
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:4>
Django
Reporter: bouke | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Changes (by ye0eugene0ey):
* version: 1.4 => master
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:5>
Django
Reporter: bouke | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Comment (by giuliettamasina):
I can't reproduce this. These are the steps I tried:
1. Start a new project.
2. Start a new app.
3. Create a `static` folder in the new app, and put the `icons.css` from
above in it.
4. Edit settings, setting `STATIC_ROOT` to e.g. a local folder
`staticfiles` and adding your new app to `INSTALLED_APPS``
5. Run `collectstatic` in your project folder
All files are copied, including `icons.css`, without errors.
I tested this using the current `master` branch of Django, on Python
2.7.6, on a Mac.
Attached is my test project, as described.
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:6>
Django
Reporter: bouke | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Comment (by anonymous):
Duplicated this error when attempting to 'collectstatic' on:
http://cdn.jsdelivr.net/semantic-ui/0.18.0/css/semantic.css
Verified that ye0eugene0ey's patch from above resolves the issue.
Python 2.7.3 on Fedora
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:7>
Django
Reporter: bouke | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Changes (by anonymous):
* cc: joel@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:8>
Django
Reporter: bouke | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Comment (by claudep):
Those having succeeded to reproduce, could you please also tell the Django
version you are using?
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:9>
Django
Reporter: bouke | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Comment (by slumtrimpet <joel@…>):
Good question... it errors on 1.4.1 and works on 1.5.4.
(those are the two env's I have handy)
(I'm 'anonymous' from 6 hours ago)
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:10>
Django
Reporter: bouke | Owner: nobody
Type: Bug | Status: closed
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Changes (by claudep):
* status: new => closed
* resolution: => duplicate
Comment:
Then I guess this is a duplicate of #18430.
--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:11>