gVim stripping diacritics not in keyboard charset
Tuetschek
I am using gVim 7.2 under Windows 7 and I have customized my En-Us
keyboard using the MS Keyboard Layout Creator, so that it contains
not only English characters, but also Czech ones and others (e.g.
Latin long
vocals), which have diacritical marks. It works perfectly well in all
programs but gVim (even Vim console version and Vim console Cygwin
version work correctly).
The problem is that some of the diacritical marks get just stripped
away, e.g. I type "ř" and get just "r". I noticed this happens only
with characters outside of the Latin-1 (or CP1252?) charset (e.g. I
can type "ä" all right), but it doesn't depend on the encoding, file
encoding, terminal encoding or language setting in gVim.
Could anyone help me how to solve or at least understand the problem?
Thanks,
Tuetschek
Cyril Slobin
> The problem is that some of the diacritical marks get just stripped
> away, e.g. I type "ř" and get just "r".
I second this. I have recently installed an Esperanto extension of the
standard US English layout, and found that letters like ĉ or ŝ aren't
entered properly in gVim. Sad.
--
http://slobin.pp.ru/ `When I use a word,' Humpty Dumpty said,
<cy...@slobin.pp.ru> `it means just what I choose it to mean'
bill lam
> On Tue, Jan 19, 2010 at 10:40 PM, Tuetschek <ondrej...@atlas.cz> wrote:
>
> > The problem is that some of the diacritical marks get just stripped
> > away, e.g. I type "ř" and get just "r".
>
> I second this. I have recently installed an Esperanto extension of the
> standard US English layout, and found that letters like ĉ or ŝ aren't
> entered properly in gVim. Sad.
>
I just solved a similar problem in linux mentioned in another thread.
I guess the problem is not gvim itself. Which OS does you use? Did
it also failed in other gui editors? I guess you already aware the
esperanto keymap as a workaround.
--
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3
Tuetschek
> I guess the problem is not gvim itself. Which OS does you use? Did
> it also failed in other gui editors? I guess you already aware the
> esperanto keymap as a workaround.
>
Mine is currently on Windows 7, but I've had the same problems with
Windows XP, so I think it's for all (current) versions of Windows.
It's clearly a gVim-related problem with keyboard input handling, I've
no similar problems with any other program (except for PuTTy, but I
think there's some similar problem there).
Changing the keymap is exactly what I'm trying to avoid. And I can use
any Windows keyboard layout for any language. But instead of switching
keymaps every few seconds, which feels kind of schizophrenic to me,
I've created a custom one that contains all the characters I need. And
sadly gVim is not compatible with it.
Tony Mechelynck
It can be an OS configuration problem, a Vim configuration problem, or
(especially in console mode) a misunderstanding between the OS and Vim,
as follows:
OS configuration problems
-------------------------
The keyboard can only generate keys which are in your system locale, as
known in Unix parlance as $LC_CTYPE, and, at Vim startup and before
running any scripts (including not only your own vimrc but possibly also
a "system vimrc" of which not every user is aware), as 'encoding' and
possibly v:ctype
Vim configuration problems
--------------------------
* Vim can only accept into memory characters which exist in the charset
defined by the present value of 'encoding'
* Vim can only display glyphs which are in your present font. In console
mode this means the console font (which is out of Vim's control), in GUI
mode it is your present 'guifont', see
http://vim.wikia.com/wiki/Setting_the_font_in_the_GUI
* Vim can only write to a disk file characters which exist in the file's
'fileencoding'
* When reading an existing file, Vim tries to determine the file's
'fileencoding' (singular) with the help of the global option
'fileencodings' (plural). You can bypass this heuristic if you know the
"right" value, see :help ++enc
* When creating a new file, Vim sets its 'fileencoding' to the option's
global value (again, unless you override it, e.g. with ++enc). An empty
'fileencoding' (the default) means: use 'encoding'.
Misunderstandings between the OS and Vim
----------------------------------------
Vim believes that the characters sent by the keyboard (and, in console
mode, those accepted by the display) are in the charset defined by
'termencoding'. The default value of 'termencoding' is (except in GUI
mode in gvim with GTK2 GUI) the empty string, which means: use
'encoding' (IIUC in GUI mode on Win32 gvim always gets keyboard input in
Unicode regardless of 'termencoding'). This empty string is OK at
startup because the default value of 'encoding' is your locale's ctype
charset, but it is probably not OK if you have changed 'encoding', for
instance in order to edit Unicode. In that case I recommend to set
'termencoding' (if it was empty) to the "old" value of 'encoding' before
you set a "new" one. (This won't harm in the cases where Vim disregards
the value, and it is essential when it doesn't.)
To check the "actual" value of a character which may be displayed
differently (e.g. because the console cannot display the actual value,
or because Vim "thinks" that it can't) put the Normal-mode cursor on the
character and type ga (then look at bottom left of the Vim screen).
Inputting characters not in your OS locale
------------------------------------------
Vim can accept into memory any character present in 'encoding', even
those not accepted by your OS locale. There are several ways to do it:
- Keymaps (but you say you don't want them)
- Character value in hex, or, for single-byte characters, also in
decimal or octal, see :help i_CTRL-V_digit
- Digraphs: see :help digraph.txt and in particular :help
digraphs-default. IMHO these are best for sporadic Latin accented
characters not on your keyboard, and to a lesser degree even for
sporadic non-Latin characters for which a digraph exists. (By sporadic I
mean: just a few at a time, not long sentences of them.)
For details, see http://vim.wikia.com/wiki/Working_with_Unicode as well
as the help topics mentioned there.
Best regards,
Tony.
--
Information Center, n.:
A room staffed by professional computer people whose job it is
to tell you why you cannot have the information you require.
Tuetschek
doesn't matter which terminal or file encoding I have set, my font is
capable of displaying such characters (I can paste a "ř"' through
clipboard or open any file that contains it all right) and that
character is present in my locale "Czech_Republic.1250".
I think it's got something to do with the way Vim handles Windows
input messages (since the console Windows version works OK).
Tuetschek
Tony Mechelynck
we'll never know if the cure was due to a different procedure used to
build Vim, or to one of the many bugfixes which happened in the three
years (and almost a month) since then.
For people who want to see the begin of this thread (e.g. if, like me,
they read this by POP rather than on the Google Groups web interface),
it is at
https://groups.google.com/group/vim_use/browse_frm/thread/8ee40c183ebf4d27/
Happy Vimming!
Tony.
--
Due to circumstances beyond your control, you are master of your fate
and captain of your soul.