Issue 24 in openqwaq: Failure to detect or encode Unicode strings in StickyNotes

[open...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 24 by david.s....@gmail.com: Failure to detect or encode Unicode
strings in StickyNotes
http://code.google.com/p/openqwaq/issues/detail?id=24

What steps will reproduce the problem?
1. Create a StickyNote
2. Paste in some Unicode text with invalid XML characters - for
example 'this & that', including the quote marks
3. Save the forum, exit to lobby and try to log back in

What is the expected output? What do you see instead?
Expect to see the same forum again, but instead the load fails at 20% with
the error 'SAXParseException: XML expected 'Name literal.'

What version of the product are you using? On what operating system?
Latest OQ from svn on Win XP. Same error on Mac platform.

Please provide any additional information below.
When the text 'this & that' is used in the StickyNote, the relevant section
of the island.xml contained in the c3z file looks like this:

<contents>&#8216;this & that&#8217;</contents>

It looks like the ' characters are correctly encoded, but then the &
character is ignored and written out literally. Two workarounds:

- manually encode the & to &amp; and rezip the c3z. This allows people to
log back in, but the error will reappear next time the forum is saved and
loaded back from disk
- remove the unicode characters and replace with standard quotation marks
(and encode the &, obviously). This allows people to log back in, and the
error does not reappear.

Also, note that whilst the forum has active users, the space is cloned to
new users successfully, and so a third workaround is to leave a user
constantly logged in, to prevent the space being loaded from disk. Clearly
this isn't a long-term solution :)

[open...@googlecode.com]

Comment #1 on issue 24 by david.s....@gmail.com: Failure to detect or

encode Unicode strings in StickyNotes
http://code.google.com/p/openqwaq/issues/detail?id=24

I think I found the source of this problem - it's in XML-Parser pcData:
method. The original line:

ifFalse:[entity := XMLTranslation at: ch charCode ifAbsent:[nil]].

fails to access the XMLTranslation dictionary for single width characters
when processing a WideString. My patch is to replace the charCode with
asCharacter:

ifFalse:[entity := XMLTranslation at: ch asCharacter ifAbsent:[nil]].

Tests on the simple test case above show that properly encoded XML is now
generated:

<contents>This &#8216;is a &amp; test&#8217;</contents>

A changeset is attached.

Cheers,

David.

Attachments:
XMLWriter-pcData.st 1.2 KB

[open...@googlecode.com]

Comment #2 on issue 24 by david.s....@gmail.com: Failure to detect or

[open...@googlecode.com]

Comment #3 on issue 24 by david.s....@gmail.com: Failure to detect or

encode Unicode strings in StickyNotes
http://code.google.com/p/openqwaq/issues/detail?id=24

Sorry for the double post - Google returned a 502 the first time, but
obviously it did get through... :)