Issue 136 in robotframework: UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0'

[codesite...@google.com]

Issue 136: UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0'
http://code.google.com/p/robotframework/issues/detail?id=136

New issue report by SonOfLilit:
HTML-Tidy has a default option to replace ' ' with 0xA0
non-breaking-space.

A colleague tells me that this chokes robot with the error message

UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in
position 70: ordinal not in range(128)

If need be, I can reproduce an html file that causes this error, but it
should be trivial.

-- Aur

Issue attributes:
Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings

[codesite...@google.com]

Issue 136: UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0'
http://code.google.com/p/robotframework/issues/detail?id=136

Comment #1 by janne.t.harkonen:
Can you produce the file, please.

Test data is parsed in parts, and our purpose is to support Unicode in all
parts.
It seems that the Unicode support is not universal, so an example file
should help to
pinpoint the problem.

Thanks for reporting the issue

Issue attribute updates:
Status: Accepted

[codesite...@google.com]

Issue 136: UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0'
http://code.google.com/p/robotframework/issues/detail?id=136

Comment #2 by laukpe:
I suspect that the problem could be file not having correct encoding. It
needs to be
specified as explained at
http://robotframework.googlecode.com/svn/trunk/doc/userguide/RobotFrameworkUserGuide.html#supported-file-formats

As Janne already commented, having an example file would be nice.

[codesite...@google.com]

Updates:
Status: Invalid

Comment #3 on issue 136 by pekka.klarck: UnicodeEncodeError: 'ascii' codec

can't encode character u'\xa0'
http://code.google.com/p/robotframework/issues/detail?id=136

Can't do much for this without an example that causes the failure. If such
a file can
be provided this issue can be reopened.

[codesite...@google.com]

Comment #4 on issue 136 by ChrisPrinos: UnicodeEncodeError: 'ascii' codec

can't encode character u'\xa0'
http://code.google.com/p/robotframework/issues/detail?id=136

I see the same problem with html files created using MS Word, especially
when using
open/close quotes as in “sample” . It looks like word is creating files
with the
following:

<meta http-equiv=Content-Type content="text/html; charset=windows-1252">

If I change the encoding in MS Word to force it to use UTF-8, then the test
will
pass, but it seems like I should be able to use the default since the
encoding is
specified.

the other workaround is to force the default encoding by modifying robot
__init__.py
file to contain:

if os.name == 'java':
from org.python.core import codecs
codecs.setDefaultEncoding('utf-8')


I'm using jython 2.5 trunk (as of 12/1/08)

Attachments:
test3.html 7.1 KB
output.xml 1.3 KB

[codesite...@google.com]

Updates:
Status: Accepted

Comment #5 on issue 136 by pekka.klarck: UnicodeEncodeError: 'ascii' codec

can't encode character u'\xa0'
http://code.google.com/p/robotframework/issues/detail?id=136

Ok, need to take a look at this again.

[codesite...@google.com]

Updates:
Labels: Target-2.0.4

Comment #6 on issue 136 by pekka.klarck: UnicodeEncodeError: 'ascii' codec

can't encode character u'\xa0'
http://code.google.com/p/robotframework/issues/detail?id=136

(No comment was entered for this change.)

[codesite...@google.com]

Updates:
Owner: pekka.klarck

Comment #7 on issue 136 by pekka.klarck: UnicodeEncodeError: 'ascii' codec

[codesite...@google.com]

Updates:
Owner: ---
Labels: -Target-2.0.4 Target-2.1

Comment #8 on issue 136 by pekka.klarck: UnicodeEncodeError: 'ascii' codec

can't encode character u'\xa0'
http://code.google.com/p/robotframework/issues/detail?id=136

I tested the test3.html attached by ChrisPrinos and was able to execute it
successfully both with Python 2.5.1 and Jython 2.2 on my Mac.

I just noticed that ChrisPrinos had been using Jython 2.5 which we don't
fully
support yet (see issue 198). The biggest problem with Jython 2.5, and
problems are at
least partly due to workarounds we have for Unicode issues with Jython 2.2.
We'll
investigate these problems in the future when Jython 2.5 gets more ready
(I've
understood the concentrate on Unicode themselves in the near future) and we
start RF
2.1 development.

SonOfLilit, which Python/Jython version you were using? An example about
your problem
would still be nice if you can reproduce the problem.