#7295: quotes, escaping and translation of string literals in templates

167 views
Skip to first unread message

akaihola

unread,
May 23, 2008, 7:55:07 AM5/23/08
to Django developers
While I was trying to implement allowing filter expressions as
arguments to some built-in template tags (see #5756), I found
inconsistencies in the handling of literal strings in the template
system.

Collin Grady's patch [1] for {% ifequal %} looks good, but using the
same logic for other tags causes unit tests to fail. The reason is
that the FilterExpression class [2] (called by compile_filter()) only
accepts double quoted string literals, whereas tests generally use
single quoted strings. The current implementations of several template
tags resolve their arguments using the Variable class [3], which turns
out to handle both single and double quoted strings.

After studying the code for filter expression and variable resolution
I came up with a patch suggestion [4] for allowing single quotes in
filter expressions. I also made sure that escaping of the chosen quote
character and the backslash character works correctly. I also re-
factored the FilterExpression and Variable classes a bit so they use
the same code to unquote and un-escape string literals.

In FilterExpression this change required a different approach to the
filter_re regular expression. Single and double quoted translated and
un-translated strings are now all caught with a single group and
parsed further in the Python code. I have not yet looked into
performance effects or possible optimizations of this implementation.

I couldn't find any design desicions wrt single and double quotes. But
if there is a decision against single quotes, of course an entirely
different approach has to be taken.

All current unit tests pass with the suggested patch. The patch is
attached to #7295.

#5756 http://code.djangoproject.com/ticket/5756
#7295 http://code.djangoproject.com/ticket/7295
[1] http://code.djangoproject.com/attachment/ticket/5756/5756.patch
[2] http://code.djangoproject.com/browser/django/trunk/django/template/__init__.py#L459
[3] http://code.djangoproject.com/browser/django/trunk/django/template/__init__.py#L595
[4] http://code.djangoproject.com/attachment/ticket/7295/7295.1.diff
Message has been deleted

akaihola

unread,
May 23, 2008, 10:56:25 AM5/23/08
to Django developers
It's confusing that the django.utils.text.smart_split() function un-
escapes quotes and backslashes inside the literal string while
preserving the surrounding quotes:

>>> print ' '.join(smart_split(ur'arg1 "the \"second\" argument"
"the \\third\\ argument"'))
arg1 "the "second" argument" "the \third\ argument"

I would find it simpler to just have either
1) escaped and quoted literal strings, or
2) un-escaped and un-quoted strings.

This would also make it simpler to unify the handling of variables and
filter expressions in django/template/__init__.py.

Waylan Limberg

unread,
May 23, 2008, 11:34:27 AM5/23/08
to django-d...@googlegroups.com
I haven't looked at what your doing real close, but it seems to me
your trying to re-implement the shlex module [1] which has been in the
Python standard library since 1.5.2. Will that do the trick?

[1]: http://docs.python.org/lib/module-shlex.html

--
----
Waylan Limberg
way...@gmail.com

Reply all
Reply to author
Forward
0 new messages