Non ASCII characters

[Urban Hafner]

Hej all,

attached is a file that's UTF8 (though I'm not sure if it gets through
like that in an email). It contains virtually no code, but it still
fails with the following error:

uh@uh:~/me/ujh-node$ ./build/default/node utf8-bug.js
at [object Object].[anonymous]
at [object Object].emit
at [object Object].emitSuccess
at [object Object].[anonymous]
utf8-bug.js:16: SyntaxError: Unexpected token }
};
^

When I remove the non-ascii characters from the comment (!) the error
goes away. This happens here on OSX. Can someone confirm that for Linux,
too? It would be good to know if this is an OS specific bug before
trying to fix it.

Urban

PS: I found this bug when I tried to incorporate András cookie support
into Coltrane. Foolishly I tried to give him credit in the comments ;)
András, I hope you won't mind if your name lacks accents until this is
fixed ;)

[Joshaven Potter]

Thank you!  I have played around with tests to show this bug but I cannot locate the real issue...

I tested with the following lines in a mjsunit test:

include("mjsunit.js");
function onExit(){
  assertTrue( /Hellö Wörld/.test("Hellö Wörld") );
}

When the file is saved in UTF8 I get:
 SyntaxError: Unexpected token ILLEGAL

When it is saved in ISO-8859-1 (Latin 1) or (Windows) I don't get any errors.


The this unit test, to my knowledge does not rely on node at all so it appears that this bug is not with node.  The only dependencies I can think of would be the environment (file system, OS, encoding format, etc), the python test runner files (borrowed from V8 source) & the V8 JavaScript Engine its self.

I would suspect that it has something to do with V8.  The copy of node that I am running seems to be running V8 Version 1.2.10.  It would be great if someone on this list could write a failing test for the V8 code and determine if the problem is in the node version as well as the head version from google code.

I gave a wack at it but I could not find out how to run the tests in V8 in the time that I could afford for the investigation.

If someone would send me a note as to how V8 tests are run, I'd be happy to figure out how to track down this bug.  I added a test that would fail ("assertTrue(false)") and tried:

scons test
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: `test' is up to date.
scons: done building targets.

Obviously I am missing something!

--
Sincerely,
Joshaven Potter

"No man making a profession of faith ought to sin, nor one possessed of love to hate his brother. For He that said, “Thou shalt love the Lord thy God,”  said also, “and thy neighbor as thyself.”  Those that profess themselves to be Christ’s are known not only by what they say, but by what they practice. “For the tree is known by its fruit.”" -- Ignatius

[Urban Hafner]

Joshaven Potter wrote:
> Thank you! I have played around with tests to show this bug but I
> cannot locate the real issue...
>
> I tested with the following lines in a mjsunit test:
>
> include("mjsunit.js");
> function onExit(){
> assertTrue( /Hellö Wörld/.test("Hellö Wörld") );
> }
>
> When the file is saved in UTF8 I get:
> SyntaxError: Unexpected token ILLEGAL
>
> When it is saved in ISO-8859-1 (Latin 1) or (Windows) I don't get any
> errors.

Interesting. I thought ISO-8859-1 is a subset of UTF8? Anyway, I've
opened issue #15 for this and referred to this thread in the description.

Urban

[ry]

Fixed in b1588e7.