Non-ascii characters

[Henning]

Hi

I've been trying to include some non-ascii characters in my views,
without much success.
I'm trying to write the norwegian characters æ,ø,å but they show up
with other symbols.

I've tried to change the encoding in my browsers, and set the charset
but nothing helps, which leads me to think that it is spark that
messes up my characters.

Can anybody confirm this, and hopefully provide a solution?

Thanks

Henning

[tylerl]

When using non-ascii characters in HTML, you should ALWAYS use entity
references as opposed to trying to encode the character in the native
code set. Character encodings are a messy business full of
contradicting assumptions and defaults, and your characters could
easily be getting reinterpreted as a different encoding somewhere
along the way.

For what it's worth, Spark uses .NET strings, which preserve Unicode,
so Spark isn't your culprit.

"Entity references" in HTML and XML start with a "&" and end with a
";". You're probably familiar with """ and "<", but there are
also references for all the characters you'd care to write.

The "AE" ligature is written as any of the following "Æ",
"Æ", or "Æ"

AElig is the easy-to-remember form, while the other two are the
character's unicode value in hex and in decimal. You can escape any
unicode character as "&#<num>;" where <num> is the unicode value,
either written in decimal form or in hex prefixed by "x". Or, you can
just use the easy-to-remember form.

You can find the full reference of HTML/XML entity references here:
http://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references

For any symbols not in this list, use the unicode encoding format.

When you use an entity reference, you're guaranteed that the character
will make it through all the various encodings and decodings
unscathed.

[Asbjørn Ulsberg]

Sorry for being blunt, but this is the worst recommendation I've read in a
while. Using HTML entities is not the preferred solution to this problem,
which is a symptom of an underlying issue that instead should be resolved.

.NET uses UTF-16 as its internal representation of strings, which handle
non-ASCII characters perfectly. XML (and HTML nowadays) default to UTF-8
and if non-ASCII characters show up as two characters, that means
somewhere along the pipeline those (two-byte) UTF-8 characters has been
mis-interpreted as (one-byte) ASCII/ISO-8859-1/Windows-1252 characters.

It might also be the other way around; that ISO-8859-1 or Windows-1252
characters (which are stored as one byte) are read as UTF-8 and because
one of the two bytes that comprises non-ASCII characters (well, in most
cases) are missing, will be represented as a question mark or other
"unknown character" replacement.

Either way, this is an character codec problem that needs to be fixed at
the codec level instead of hacking around the problem with HTML entity
replacements.

I would first look at the way the .spark files are stored on disk. Are
they stored as UTF-8 or as Windows-1252/ISO-8859-X? Make sure they are
stored as UTF-8 and open them in another editor to ensure that the
characters are represented correctly.

If the .spark files are encoded correctly, how does the rest of the
pipeline look like? What editor do you use to create the views? Does the
Norwegian characters come from a database? From other files read from the
file system? Somewhere else? Does the output encoding of the web
application match the encoding of the characters being transmitted to the
browser? Does the browser know how to decode the given encoding? Does all
browsers yield the same result?

If nothing at all yields any constructive results, then I can recommend
falling back to HTML entities, but that should be an absolute last resort,
because it will only get rid of the problem like whack-a-mole gets rid of
the mole; it is guaranteed to pop up again in a different location.

-Asbjørn

[Henning]

I've found the problem and a temporary solution.

I'm using VS 2010 without the spark addin. Nothing special about my
setup. Windows 7 64bit.
The problem is that VS saves my .spark files in ANSI encoding and not
utf8. I've checked with another project where we don't use spark and
there the aspx files get saved in utf8.
What I was doing was editing my spark files in the HTML editor in VS,
which gives me the fewest intellisense errors, since I don't have the
vs plugin installed for 2010 yet.
When changing to th XML editor, it gets saved as "ANSI as UTF-8", or
"UTF-8 without BOM" according to Notepad++.

When I checked with my VS 2008 spark demo project where the plugin is
installed, it gets saved as pure UTF-8.

Guess I have to wait for the VS 2010 plugin. Until then I have to use
the XML editor with lots of warnings and errors :(

Thanks guys.

Henning

> >http://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_re...

[John Gietzen]

Visual Studio will let you change the encoding of XML files (or,
really, any file) on the fly. The encoding and line-ending menues are
under the file menu.

Sent from my iPhone

> --
> You received this message because you are subscribed to the Google Groups "Spark View Engine Dev" group.
> To post to this group, send email to spar...@googlegroups.com.
> To unsubscribe from this group, send email to spark-dev+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/spark-dev?hl=en.
>

[Asbjørn Ulsberg]

There you go! Great that you found the culprit! ;)

-Asbjørn