[Django] #20375: Post-processing CSS file with Unicode chars fails

15 views
Skip to first unread message

Django

unread,
May 8, 2013, 4:45:08 AM5/8/13
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------+--------------------
Reporter: bouke | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.4
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------
Say I have a static file like this: (icons.css)
{{{
.icon-home:before { content: '\2302'; } /* '⌂' */
.icon-play-circle2:before { content: '\e048'; } /* '' */
.icon-location:before { content: '\e724'; } /* '' */
.icon-map:before { content: '\e727'; } /* '' */
.icon-twitter-1:before { content: '\f309'; } /* '' */
.icon-facebook-1:before { content: '\f30c'; } /* '' */
.icon-doc:before { content: '📄'; } /* '\1f4c4' */
.icon-calendar:before { content: '📅'; } /* '\1f4c5' */
.icon-search:before { content: '🔍'; } /* '\1f50d' */
}}}

When this file is being post-processed (in ``collectstatic``), an
exception is raised:

{{{
Traceback (most recent call last):
File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1481, in
<module>
debugger.run(setup['file'], None, None)
File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1124, in
run
pydev_imports.execfile(file, globals, locals) #execute the script
File "/Volumes/Users/bouke/Sites/gct/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "lib/python2.7/site-packages/django/core/management/__init__.py",
line 443, in execute_from_command_line
utility.execute()
File "lib/python2.7/site-packages/django/core/management/__init__.py",
line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "lib/python2.7/site-packages/django/core/management/base.py", line
196, in run_from_argv
self.execute(*args, **options.__dict__)
File "lib/python2.7/site-packages/django/core/management/base.py", line
232, in execute
output = self.handle(*args, **options)
File "lib/python2.7/site-packages/django/core/management/base.py", line
371, in handle
return self.handle_noargs(**options)
File "lib/python2.7/site-
packages/django/contrib/staticfiles/management/commands/collectstatic.py",
line 163, in handle_noargs
collected = self.collect()
File "lib/python2.7/site-
packages/django/contrib/staticfiles/management/commands/collectstatic.py",
line 120, in collect
for original_path, processed_path, processed in processor:
File "lib/python2.7/site-
packages/django/contrib/staticfiles/storage.py", line 226, in post_process
content = pattern.sub(converter, content)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 500:
ordinal not in range(128)
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/20375>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
May 11, 2013, 8:21:17 PM5/11/13
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------

Reporter: bouke | Owner: nobody
Type: Uncategorized | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Changes (by anguslturner):

* needs_better_patch: => 0
* component: Uncategorized => contrib.staticfiles
* needs_tests: => 1
* version: 1.4 => master
* needs_docs: => 1
* stage: Unreviewed => Accepted


--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:1>

Django

unread,
May 18, 2013, 9:19:44 AM5/18/13
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------

Reporter: bouke | Owner: nobody
Type: Uncategorized | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------

Comment (by djangsters):

How can I reproduce this. Here it works just fine.

--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:2>

Django

unread,
May 19, 2013, 8:56:52 AM5/19/13
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------
Reporter: bouke | Owner: nobody
Type: Uncategorized | Status: closed
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution: invalid

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Changes (by LukaszBalcerzak):

* status: new => closed
* resolution: => invalid


Comment:

Cannot reproduce too. Am closing for now as it doesn't seem like a bug in
Django.

Please reopen once you can reproduce. Hints: run `collectstatic` outside
of PyCharm and check css file encoding.

--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:3>

Django

unread,
Jun 17, 2013, 11:33:33 AM6/17/13
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------
Reporter: bouke | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.4
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Changes (by ye0eugene0ey):

* status: closed => new
* cc: yeo.eugene.oey@… (added)
* needs_better_patch: 0 => 1
* resolution: invalid =>
* version: master => 1.4
* type: Uncategorized => Bug


Comment:

== Reproduce the Error ==

I encounter the same issues. And after tracing the type,
It happened on
[https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L226],
when trying to replace a ''unicode'' type into a ''string'' type that has
a ''unicode'' character.

{{{
import re

patt = re.compile(r"""(url\(['"]{0,1}\s*(.*?)["']{0,1}\))""")

patt.sub(lambda x: u'The replacement is a function that return unicode',
"""The content is str with at least a unicode character głó,
and don't forget to match url(../some/path)""")
}}}

== Patching?? ==

I tried to convert the type into string at
[https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L172]
and it works
{{{
return str('url("%s")' % unquote(relative_url))
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:4>

Django

unread,
Jun 17, 2013, 11:34:38 AM6/17/13
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------
Reporter: bouke | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Changes (by ye0eugene0ey):

* version: 1.4 => master


--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:5>

Django

unread,
May 17, 2014, 5:44:55 AM5/17/14
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------
Reporter: bouke | Owner: nobody

Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------

Comment (by giuliettamasina):

I can't reproduce this. These are the steps I tried:

1. Start a new project.
2. Start a new app.
3. Create a `static` folder in the new app, and put the `icons.css` from
above in it.
4. Edit settings, setting `STATIC_ROOT` to e.g. a local folder
`staticfiles` and adding your new app to `INSTALLED_APPS``
5. Run `collectstatic` in your project folder

All files are copied, including `icons.css`, without errors.

I tested this using the current `master` branch of Django, on Python
2.7.6, on a Mac.

Attached is my test project, as described.

--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:6>

Django

unread,
Jun 18, 2014, 7:12:42 AM6/18/14
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------
Reporter: bouke | Owner: nobody

Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------

Comment (by anonymous):

Duplicated this error when attempting to 'collectstatic' on:
http://cdn.jsdelivr.net/semantic-ui/0.18.0/css/semantic.css

Verified that ye0eugene0ey's patch from above resolves the issue.

Python 2.7.3 on Fedora

--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:7>

Django

unread,
Jun 18, 2014, 7:14:49 AM6/18/14
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------
Reporter: bouke | Owner: nobody

Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Changes (by anonymous):

* cc: joel@… (added)


--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:8>

Django

unread,
Jun 18, 2014, 12:57:30 PM6/18/14
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------
Reporter: bouke | Owner: nobody

Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------

Comment (by claudep):

Those having succeeded to reproduce, could you please also tell the Django
version you are using?

--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:9>

Django

unread,
Jun 18, 2014, 1:16:01 PM6/18/14
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+------------------------------------
Reporter: bouke | Owner: nobody

Type: Bug | Status: new
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------

Comment (by slumtrimpet <joel@…>):

Good question... it errors on 1.4.1 and works on 1.5.4.
(those are the two env's I have handy)
(I'm 'anonymous' from 6 hours ago)

--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:10>

Django

unread,
Jun 18, 2014, 2:02:38 PM6/18/14
to django-...@googlegroups.com
#20375: Post-processing CSS file with Unicode chars fails
-------------------------------------+-------------------------------------
Reporter: bouke | Owner: nobody
Type: Bug | Status: closed
Component: contrib.staticfiles | Version: master
Severity: Normal | Resolution: duplicate

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by claudep):

* status: new => closed

* resolution: => duplicate


Comment:

Then I guess this is a duplicate of #18430.

--
Ticket URL: <https://code.djangoproject.com/ticket/20375#comment:11>

Reply all
Reply to author
Forward
0 new messages