utf problems with regular vim (not macvim) on os x

[lanzz]

hello everybody,

i have problems getting vim to work with unicode in OS X. it can open
unicode text files fine, properly detects that they are in utf8, and
displays them fine; however if i try to enter unicode (codes above
127) via the keyboard, vim interprets the input as single-byte latin1
input and re-encodes it from latin1 to utf8, which does not produce
the correct results.

vim is :set termencoding=utf8, LC_CTYPE is en_US.utf8. i've verified
that the OS X is producing the proper utf8 sequences. utf8 works
correctly in bash (properly displayed and text length is calculated
correctly). the strange thing is that if i ssh from a linux box to the
mac, vim works correctly with utf8. it is apparently something related
to the OS X terminal, but i cannot figure out what it is, since typing
"echo <some-cyrillic-text> | xxd" in shell produces exactly the same
byte sequence on the mac as the same command on linux.

any insights?

[Ben Schmidt]

You have me baffled here...

Unless Vim is somehow faking knowing about the encoding...

I presume you are using vim in the Terminal.app, not any GUI version of Vim at all.

Some things to check:

- The character set encoding setting of your terminal window (File->Get
Info->Display).
- Vim's 'enc' option (:set enc?)
- What's compiled into Vim: :version

I guess post that kind of info to the list if you're still stuck and see if it or
anyone here can shed more light on the situation.

Also, as this question isn't all that Mac-specific, the folks on the vim_use
mailing list may be quite helpful, so it may be worth posting your question there
if it's still unresolved after a while here.

Ben.

Send instant messages to your online friends http://au.messenger.yahoo.com

[lanzz]

On Sep 29, 5:55 am, Ben Schmidt <mail_ben_schm...@yahoo.com.au> wrote:

> I presume you are using vim in the Terminal.app, not any GUI version of Vim at all.

yes, that's what i meant by "regular vim" in the subject :)

> - The character set encoding setting of your terminal window (File->Get
> Info->Display).

UTF-8

> - Vim's 'enc' option (:set enc?)

utf-8

> - What's compiled into Vim: :version

+arabic +autocmd -balloon_eval -browse ++builtin_terms +byte_offset
+cindent -clientserver -clipboard +cmdline_compl +cmdline_hist
+cmdline_info +comments
+cryptv -cscope +cursorshape +dialog_con +diff +digraphs -dnd -ebcdic
+emacs_tags +eval +ex_extra +extra_search +farsi +file_in_path
+find_in_path +folding
-footer +fork() +gettext -hangul_input +iconv +insert_expand +jumplist
+keymap +langmap +libcall +linebreak +lispindent +listcmds +localmap
+menu +mksession
+modify_fname +mouse -mouseshape +mouse_dec -mouse_gpm -mouse_jsbterm
+mouse_netterm +mouse_xterm +multi_byte +multi_lang -mzscheme -
netbeans_intg -osfiletype +path_extra -perl +postscript +printer
+profile -python +quickfix +reltime +rightleft -ruby +scrollbind
+signs +smartindent -sniff +statusline -sun_workshop +syntax
+tag_binary +tag_old_static -tag_any_white -tcl +terminfo
+termresponse +textobjects +title -toolbar +user_commands +vertsplit
+virtualedit +visual +visualextra +viminfo +vreplace +wildignore
+wildmenu +windows +writebackup -X11 -xfontset -xim -xsmp -
xterm_clipboard -xterm_save

> Also, as this question isn't all that Mac-specific, the folks on the vim_use
> mailing list may be quite helpful, so it may be worth posting your question there
> if it's still unresolved after a while here.

i have also posted this question to vim_multibyte.

[zoul]

> i have problems getting vim to work with unicode in OS X.

I've got the same problem and I hope that it is going to disappear in
Leopard:

"Get more out of Terminal. The Core Text API for text layout reduces
setup time and makes Terminal behave flawlessly for international
users, with an increased character set." (http://www.apple.com/macosx/
features/300.html#terminal)

T.

[lanzz]

i've got a solution for this problem on the vim_multibyte list - in
terminal inspector, in the emulation section, the option "escape non-
ascii characters" needs to be turned off (it is on by default).