JSONString + high unicode characters (\u2028 to be exact)

[Matthew Mastracci]

I just ran into a hard-to-find bug where a JSONString with the content
"ABC\u2028ABC" was failing to roundtrip. It turns out that browsers
treat this as a newline in the eval statement, resulting in
"unterminated string literal" errors. You can see this happen by typing
this into the address bar: javascript:alert(eval("\"\u2028\""));

I ran a quick test to see which other chars might be dangerous. In FF3
minefield and Safari 3, I get parse failures for \u2028 and \u2029 and
for FF3 only, I get "not equal" for \ufeff and \ufffe. In FF2, I get a
whole bunch of "not equal" messages scattered from \u202a -> \u206f.

<script>
for (i = 0x0; i <= 0xffff; i++) {
try {
if (eval("\"" + String.fromCharCode(i) + "\"") !=
String.fromCharCode(i))
document.write(i.toString(16) + ": not equal!<br>");
} catch (e) {
document.write(i.toString(16) + ": "+ e + "<br>");
}
}
</script>

Based on this test, it might be safest to force escaping of all unicode
chars above \xff, unless someone can conclusively derive a globally safe
set of unicode chars.

Filed 2270 for this:

http://code.google.com/p/google-web-toolkit/issues/detail?id=2270

[John Tamplin]

I know I worked with Miguel on this about a year ago for RPC, and I thought we fixed JSONString at the same time -- Miguel, do you remember the details?

--
John A. Tamplin
Software Engineer, Google

[BobV]

> > Based on this test, it might be safest to force escaping of all unicode
> > chars above \xff, unless someone can conclusively derive a globally safe
> > set of unicode chars.
> >
> > Filed 2270 for this:
> >
> > http://code.google.com/p/google-web-toolkit/issues/detail?id=2270
> >
>
> I know I worked with Miguel on this about a year ago for RPC, and I thought
> we fixed JSONString at the same time -- Miguel, do you remember the details?

IIRC, Scott recently re-wrote the JSON library to take advantage of
the new JSO functionality. This may have been an unintended victim of
the rewrite.

--
Bob Vawter
Google Web Toolkit Team

[Miguel Méndez]

I don't recall modifying the JSON library.

--
Miguel