Hi!

I just came across this problem.
Suppose I have this (minimal) controller:

class Root(controllers.RootController):
    @expose()
    def test(self, *args, **kwargs):
        return "bla"

    @expose()
    @identity.require(identity.not_anonymous())
    def test2(self, *args, **kwargs):
        return "bla2"

When I call `/test?a=ä`, all works as expected. But when I call the
second handler `/test2?a=ä`, I get an error:

500 Internal error

The server encountered an unexpected condition which prevented it from
fulfilling the request.

Page handler: 'ordinal not in range(128)'
Traceback (most recent call last):
  File "c:\programme\python25\lib\site-packages\cherrypy-2.2.1-
py2.5.egg\cherrypy\_cphttptools.py", line 103, in _run
    applyFilters('before_main')
  File "c:\programme\python25\lib\site-packages\cherrypy-2.2.1-
py2.5.egg\cherrypy\filters\__init__.py", line 151, in applyFilters
    method()
  File "c:\programme\python25\lib\site-packages\cherrypy-2.2.1-
py2.5.egg\cherrypy\filters\decodingfilter.py", line 31, in before_main
    self.decode(enc)
  File "c:\programme\python25\lib\site-packages\cherrypy-2.2.1-
py2.5.egg\cherrypy\filters\decodingfilter.py", line 50, in decode
    decodedParams[key] = value.decode(enc)
  File "c:\programme\python25\lib\encodings\utf_8.py", line 16, in
decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in
position 0: ordinal not in range(128)

I guess that this is not supposed to throw this error, is it?
I'm using TG 1.0.4b2.

Thanks for any advice!

On Oct 30, 2007 12:05 PM, Thomas Wittek <streawkc...@googlemail.com> wrote:

Good catch!

Could you create a ticket on our trac system and ping us back here.
Please try to attach some example project so we can reproduce easily
the error.

Thanks,
Florent.

On Oct 30, 3:04 pm, "Florent Aide" <florent.a...@gmail.com> wrote:

Great, my ticket got rejected: "Submission rejected as potential spam
(Akismet says content is spam)"

Text:

==== 8< ====

Suppose I have this (minimal) controller:

{{{
class Root(controllers.RootController):
    @expose()
    def test(self, *args, **kwargs):
        return "bla"

    @expose()
    @identity.require(identity.not_anonymous())
    def test2(self, *args, **kwargs):
        return "bla2"

When I call `/test?a=ä`, all works as expected.
But when I call the second handler `/test2?a=ä`, I get an error:

{{{
500 Internal error
[..]
  File "[..]python25\lib\site-packages\cherrypy-2.2.1-py2.5.egg
\cherrypy\filters\decodingfilter.py", line 50, in decode
    decodedParams[key] = value.decode(enc)
  File "[..]python25\lib\encodings\utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in
position 0: ordinal not in range(128)

The full test project (actually, only the controllers.py has been
modified after the quickstart) is attached.

==== >8 ====

Can be downloaded here: http://gedankenkonstrukt.de/files/tg-id-utf-bug.zip

Cheers

On Oct 30, 12:05 pm, Thomas Wittek <streawkc...@googlemail.com> wrote:

Found another, maybe related bug:
`/test2?a.b=b`

500 Internal error

The server encountered an unexpected condition which prevented it from
fulfilling the request.

Traceback (most recent call last):
  File "c:\programme\python25\lib\site-packages\cherrypy-2.2.1-
py2.5.egg\cherrypy\_cphttptools.py", line 103, in _run
    applyFilters('before_main')
  File "c:\programme\python25\lib\site-packages\cherrypy-2.2.1-
py2.5.egg\cherrypy\filters\__init__.py", line 151, in applyFilters
    method()
  File "c:\programme\python25\lib\site-packages\cherrypy-2.2.1-
py2.5.egg\cherrypy\filters\decodingfilter.py", line 31, in before_main
    self.decode(enc)
  File "c:\programme\python25\lib\site-packages\cherrypy-2.2.1-
py2.5.egg\cherrypy\filters\decodingfilter.py", line 50, in decode
    decodedParams[key] = value.decode(enc)
AttributeError: 'dict' object has no attribute 'decode'

Seems like Identity and CP are battling each other in decoding the
request params...

On Oct 30, 2007 4:06 PM, Thomas Wittek <streawkc...@googlemail.com> wrote:

We just created the spam filtering but now we trained it a little and
also tweaked the parameters, so it should accept you ticket. Could you
try again to create it ? And sorry for the inconvenience.

Florent.

On Oct 30, 9:33 pm, "Florent Aide" <florent.a...@gmail.com> wrote:

Ok, works.
Filed as #1598 under http://trac.turbogears.org/ticket/1598

Hi

Struggling with the same error, here's the workaround I found
that seems to be working for me. I'm using TG 1.0.2.2

IN turbogears\visit\api.py
  IN class VisitFilter(BaseFilter):
    IN def before_main(self):

adding at the end of before_main (line 165 for TG 1.0.0.2):

for k,v in cherrypy.request.params.iteritems():
    cherrypy.request.params[k]=v.encode('utf-8')

seems to do the job. However as I'm not using the latest TG version
and I'm not too confident with all this I'd rather let the
knowleadgeable team check whether this is ok or not! My 2 cents...

On Nov 15, 11:19 am, Adi <fanet...@gmail.com> wrote:

Doesn't work for me (TG 1.0.4b2).
Maybe I inserted the code at the wrong position?
I added it before:

        try:
            for plugin in _plugins:
                plugin.record_request(visit)

Here's where I have put the 2 extra lines: at the end of the method:

        # Inform all the plugins that a request has been made for the
current
        # visit. This gives plugins the opportunity to track click-
path or
        # retrieve the visitor's identity.
        try:
            for plugin in _plugins:
                plugin.record_request( visit )
        except cherrypy.InternalRedirect, e:
            # Can't allow an InternalRedirect here because CherryPy is
dumb,
            # instead change cherrypy.request.object_path to the url
desired.
            cherrypy.request.object_path= e.path

        for k,v in cherrypy.request.params.iteritems():
            cherrypy.request.params[k]=v.encode('utf-8')

By the way In my case I'm not decorating a method by using
@identity.require(identity.not_anonymous()) but using
require = identity.in_group("mygroup") at the start of a
SecureResource controller class.But I did have te same kind of error
before modifying turbogears\visit\api.py

On Nov 16, 10:38 am, Adi <fanet...@gmail.com> wrote:

Oh, great. Seems to work for me!
Thank you very much.

I added your patch to the ticket system.
Maybe someone can investigate and integrate it.

On Nov 18, 10:58 am, Thomas Wittek <streawkc...@googlemail.com> wrote:

Doesn't work with nested request params (i.e. foo.bar.baz) as they get
converted into nested dicts, so the value `v` may be a `dict` that
doesn't have a method named `encode`: AttributeError: 'dict' object
has no attribute 'encode'.

To fix your fix, you have to do do a recursive conversion:

        def encode_utf8(params):
            for k, v in params.items():
                if type(v) is dict:
                    encode_utf8(v)
                else:
                    params[k] = v.encode('utf-8')

        encode_utf8(cherrypy.request.params)

Funnily, you'll then get an error from CP:

  File "c:\programme\python25\lib\site-packages\cherrypy-2.2.1-
py2.5.egg\cherrypy\filters\decodingfilter.py", line 50, in decode
    decodedParams[key] = value.decode(enc)
AttributeError: 'dict' object has no attribute 'decode'

But I've got a feeling that the real problems are located in another
place, deeper in the system. But unfortunately I don't know much about
TG's/CP's internals.
Maybe this is ultimately a bug in CP. But I really don't know.