Add utf8 encoding support in Template

[andrei]

This raises UnicodeDecodeError:

test.py:

import web, os
render =
web.template.render(os.path.abspath(os.path.dirname(__file__)))
render.test()

test.html:

$var array = ["один","два","три"]
$ test = "текст"
<p>just a test</p>

[Zhang Huangbin]

On Jun 14, 2010, at 8:31 PM, andrei wrote:

> $var array = ["один","два","три"]

Try:

$var array = [u"xxx", u"yyy", u"zzz",]

[andrei]

I did! It doesn't work either.

[Ken]

take it to the python file, like:
# test.py
...
a = [u"один", u"два", u"три"]
render.test(a)

2010/6/14 andrei <andr...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "web.py" group.
To post to this group, send email to we...@googlegroups.com.
To unsubscribe from this group, send email to webpy+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/webpy?hl=en.

--
百才招聘(baicai.com).产品部
Email: k...@baicai.com
Gtalk: qicha...@gmail.com
twitter.com/qichangxing
http://blog.hi0791.com

[andrei]

I know it will work in python file with # coding: utf-8 on top.

But it would be great if i could do it in templates, in many cases
moving it in controller would be a bit overhead.

On Jun 14, 8:07 pm, Ken <qichangx...@gmail.com> wrote:
> take it to the python file, like:
> # test.py
> ...
> a = [u"один", u"два", u"три"]
> render.test(a)
>

> 2010/6/14 andrei <andre...@gmail.com>

>
>
>
>
>
> > This raises UnicodeDecodeError:
>
> > test.py:
>
> >    import web, os
> >    render =
> > web.template.render(os.path.abspath(os.path.dirname(__file__)))
> >    render.test()
>
> > test.html:
>
> >    $var array = ["один","два","три"]
> >    $ test = "текст"
> >    <p>just a test</p>
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "web.py" group.
> > To post to this group, send email to we...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > webpy+un...@googlegroups.com <webpy%2Bunsu...@googlegroups.com>.

> > For more options, visit this group at
> >http://groups.google.com/group/webpy?hl=en.
>
> --
> 百才招聘(baicai.com).产品部
> Email: k...@baicai.com

> Gtalk: qichangx...@gmail.com
> twitter.com/qichangxinghttp://blog.hi0791.com

[James]

Make sure the actual template is saved as a utf-8 encoded file.

On Jun 14, 5:31 am, andrei <andre...@gmail.com> wrote:
> This raises UnicodeDecodeError:
>
> test.py:
>
>     import web, os
>     render =
> web.template.render(os.path.abspath(os.path.dirname(__file__)))
>     render.test()
>
> test.html:
>

>     $var array = ["ÏÄÉÎ","Ä×Á","ÔÒÉ"]
>     $ test = "ÔÅËÓÔ"

[andrei]

I code in TextMate, that is set to create files in UTF-8 by default.

The problem I think is that the code of compiled Template isn't utf-8
encoded.

[andrei]

I've managed to fix this issue, in the template.py, line: 1011.

I changed:
return Template(open(path).read(), filename=path,
**self._keywords)
to:
return Template(open(path).read().decode("utf-8"), filename=path,
**self._keywords)

I wonder if this can be optional setting.

[andrei]

I would like to hear a word from the developers on this issue, I
consider it to be essential.

Defining variables using escape sequences in templates is not an
option.

[andrei]

This also harms when working with gettext translations, to extract
translatable strings from template files (using babel), I have to
generate python code from it, and then use the default extractor for
python files.

at fist my extract_webpy function looked like this:

def extract_webpy(fileobj, keywords, comment_tags, options):
code = "# coding: utf-8\n\n" +
Template.generate_code(fileobj.read(), fileobj.name)
return extract_python(StringIO.StringIO(code), keywords,
comment_tags, options)

But accidently it stopped to work, because of generate_code raising
UnicodeDecode error.
So now it should be:

def extract_webpy(fileobj, keywords, comment_tags, options):
code = "# coding: utf-8\n\n" +
Template.generate_code(fileobj.read().decode("utf-8"), fileobj.name)
return extract_python(StringIO.StringIO(code), keywords,
comment_tags, options)