[cjklib-devel] Issue 11 in cjklib: Windows: Command line usage broken? (Characters don't show)

[cjk...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 11 by airblas...@spellweb.de: Windows: Command line usage broken?
(Characters don't show)
http://code.google.com/p/cjklib/issues/detail?id=11

Hello,

this might very well be the fault of Windows, but I guess its a problem
anyways:
I tried using cjknife from both cmd.exe and the new Windows Powershell.
Both show "?" instead of CJKV characters.

E.g. "cjknife -w EDICT -x "knowledge"" leads to the following (first line
only): ???? /(n) knowledge/

The other way around (cjknife -i 周) also doesn't work - it passes the
character ? to cjknife.

Is there any way to work around this?
Do I have to use a different shell?

--
You received this message because you are subscribed to the Google Groups "cjklib-devel" group.
To post to this group, send email to cjklib...@googlegroups.com.
To unsubscribe from this group, send email to cjklib-devel...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cjklib-devel?hl=en.

[cjk...@googlecode.com]

Updates:
Status: Accepted
Labels: OpSys-Windows

Comment #1 on issue 11 by cburg...@ira.uka.de: Windows: Command line usage

broken? (Characters don't show)
http://code.google.com/p/cjklib/issues/detail?id=11

Thanks for the bug report. Actually this issue is known, but the solution
doesn't
seem that simple.

cjknife tries to read the system's default encoding that might be UTF-8 in
your
case. Still Python seems to have issues with the Windows console. A work
around for
now would be to "pipe" all output to a file and then open the file with an
editor.
Alas, not good.

Sorry, I currently can't fix it as I don't have access to a Win32 system.
For now I
can just set the status to accepted.

Here's a short extract from a user trying a workaround though. Maybe this
can help a
bit:

[You] must run "cmd /u" to allow for "Unicode" output from programs, then
set the code-page to 65001 which indicates UTF-8, and finally copy-paste a
lot of garbage text from the console to a UTF-8 text file so that I can
actually see it. I had to change the script to set the encoding to 'utf-8'
if the system default was 65001, which Python doesn't know. And to top it
all of, the scripts for updating the DB tables didn't work unless I changed
the code-page to the standard 1252. Oh well.

[cjk...@googlecode.com]

Comment #2 on issue 11 by cburg...@ira.uka.de: Windows: Command line usage

broken? (Characters don't show)
http://code.google.com/p/cjklib/issues/detail?id=11

It seems Python has several issues on Windows:
http://bugs.python.org/issue1602
Will need to be solved there.

Workarounds welcome!