clojure unicode on Windows

[Lukas Lehner]

Hi all

The clojure unicode reading, evaluating and printing was discussed
already with various results. Let me add one more, and kindly ask for
advice if anyone has.

OS: Windows 7
clojure 1.1
C:\>java -version
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Java HotSpot(TM) Client VM (build 14.3-b01, mixed mode, sharing)

chcp: 437

java clojure.main
user=> éáú
java.lang.Exception: Unable to resolve symbol: ��� in this context
(NO_SOURCE_FILE:0)

You see the problem.

chcp: 65001

java clojure.main
user=> éáú

C:\>

In this case REPL is killed without any message

File unicode-test.clj in unicode:
(println (seq (.split "őúáé öüü? sdf" "\\W+")))

c:\>java clojure.main unicode-test.clj
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
Exception in thread "main" java.lang.Exception: Unable to resolve
symbol:  in this context (unicode-test.clj:0)0)

rest of the error at http://clojure.pastebin.com/m2235d7fb

OS: FreeBSD 7.2
clojure 1.1
java -version
java version "1.6.0_07"
Diablo Java(TM) SE Runtime Environment (build 1.6.0_07-b02)
Diablo Java HotSpot(TM) Server VM (build 10.0-b23, mixed mode)

locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=en_US.UTF-8

java clojure.main
user=> éőó
java.lang.Exception: Unable to resolve symbol: ó�éőó in this context
(NO_SOURCE_FILE:0)
user=> "éőó"
"éőó"
user=> (println "éőó")
éőó
nil
user=> (def éőó 0)
java.lang.Exception: Unable to resolve symbol: �0 in this context
(NO_SOURCE_FILE:6)
user=>

better but still not bulletproof

testing also the same script like on windows
java clojure.main unicode-test.clj
( sfd)

No errors :) but of course it did not split the way I wanted...

Anyone having better results with unicode and encoding? Preferably on
windows.

Thank you in advance
Lukas

[Kevin Downey]

java uses local settings, on windows the default encoding is some
godawful thing (same on Mac, still godawful, but different) set
file.encoding to pick something sane

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
>

--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

[Lukas Lehner]

Ok, tried to put this at the top of the file, but same bad result on Win
(System/setProperty "file.encoding" "UTF8")

and actually here
http://stackoverflow.com/questions/361975/setting-the-default-java-character-encoding
it looks like JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8 has the same
effect, and should be used (and I am using it)

Is everybody here running macs and linuxes? :(

And, more important now is actually REPL
user=> (System/setProperty "file.encoding" "UTF8")
"UTF8"
user=> "éőó"
"�o�"
user=>

L

[Kevin Downey]

are you using the repl directly? or wrapped in jline or rlwrap?

--

[Lukas Lehner]

Hi

using directly (not jline or others)
although pure REPL would be still fine, I have found how to make it work
in emacs WIN32
not sure if all of this is needed though

.emacs

(setq locale-coding-system 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(set-selection-coding-system 'utf-8)
(prefer-coding-system 'utf-8)
(setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING))
(set-clipboard-coding-system 'utf-16le-dos)
(set-language-environment "UTF-8")
(setq slime-net-coding-system 'utf-8-unix)

then in slime repl

user> éő
; Evaluation aborted.
user> (def éő 0)
#'user/éő
user> éő
0
user>

looks ok

Lukas

[Duc Nguyen]

The only two lines needed (at least for me) are:

(setq current-language-environment "UTF-8")
(setq slime-net-coding-system 'utf-8-unix)

> > On Wed, Jan 13, 2010 at 3:02 PM, Lukas Lehner<lehner.lu...@gmail.com>  wrote:
>
> >> Ok, tried to put this at the top of the file, but same bad result on Win
> >> (System/setProperty "file.encoding"  "UTF8")
>
> >> and actually here

> >>http://stackoverflow.com/questions/361975/setting-the-default-java-ch...
> >> it looks like JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8has the same effect,

> >> and should be used (and I am using it)
>
> >> Is everybody here running macs and linuxes? :(
>
> >> And, more important now is actually REPL
> >> user=>  (System/setProperty "file.encoding"  "UTF8")
> >> "UTF8"
> >> user=>  "éőó"
> >> "�o�"
> >> user=>
>
> >> L
>
> >> On 1/13/2010 11:34 PM, Kevin Downey wrote:
>
> >>> java uses local settings, on windows the default encoding is some
> >>> godawful thing (same on Mac, still godawful, but different) set
> >>> file.encoding to pick something sane
>

> >>> On Wed, Jan 13, 2010 at 1:52 PM, Lukas Lehner<lehner.lu...@gmail.com>

> >>>> rest of the error athttp://clojure.pastebin.com/m2235d7fb