Weird behavior after ^ or ´

[Axel]

Since some patches (right now at 7.3.712) I experience the following behavior ([] denotes the cursor position, the character after the arrow the input):

a[a]aa -> a
aa[]aa -> <spc>
aa []aa -> ^
aa []aa -> <spc> " Should result in "aa ^[]aa"
aa []aa -> <esc>
aa[ ]aa -> s " gA shows 0x20
[]$a aa " Should result in "aa[]$aa"

This seems to be a bug. Can anyone verify this (Windows 7 64 bit; MinGW-compiled (64-bit))?

Addendum: This also seems to happen with apostrophes.

[Christian Brabandt]

I am not sure, I understand your description, e.g. What are you
typing after the s key (possibly the dollar)? Is Vim really not
showing the caret (^) after the space? Why do you have a $ sign there?

And I can't reproduce this issue on Windows (but 32bit, Cream installer)

regards,
Christian

[Ben Fritz]

That is 3 people now who cannot reproduce. I'm not sure why the OP started a new thread. Here's the original, to my knowledge:

https://groups.google.com/d/topic/vim_use/3MUcDw_N94Q/discussion

"Axel", can you reproduce this without any of your plugins or .vimrc? Try after starting Vim like gvim -N -u NONE -i NONE -U NONE. If the problem goes away also try with gvim --noplugin -i NONE to figure out whether it's one of your plugins.

[Axel]

"s" makes the cursor move to the beginning of the line(!) changing the character found there (hence the "$" sign). Also, the character "^" is in fact not displayed.

[Axel]

Hi Ben,

reproducible with both calls here.

[Christian Brabandt]

And how do you start Vim? Did you test with vim -u NONE -U NONE -N
Does cl work like s (it should, since s is an alias to cl)?

regards,
Christian

[Axel]

Addendum: Sorry all, as Ben has already noticed, this is a double post. The original one is in the place he indicated...

I'll switch over to the other one.

[Jürgen Krämer]

Hi,

Ben Fritz schrieb:

I can confirm this behavior on Vim 7.3.1-712 on Windows 7, compiled with
MS-C 16.0.40219.1 (i.e., Visual Studio 2008), although it only seems to
happen of every other input of "^".

The caret is a dead-letter key and is ignored although a space is pressed
afterwards. It seems to be kept in the input buffer, though, and is
finally used when "s" is pressed, leading to a movement to the start of
line and starting insert mode at the wrong position.

The dollar sign you see in Alex' example is the one displayed at the end
of the changed text if "$" is included in 'cpo'.

Regards,
J�rgen

--
Sometimes I think the surest sign that intelligent life exists elsewhere
in the universe is that none of it has tried to contact us. (Calvin)

[Jürgen Krämer]

Hi,

this behavior does not depend on using cl or s. The bug lies in ignoring
the caret in insert mode and using it later before executing the next
command. It seems that only every other (dead-letter) caret is inserted
even if it is followed by a space.

[Jürgen Krämer]

Hi,

J�rgen Kr�mer wrote:
>
> I can confirm this behavior on Vim 7.3.1-712 on Windows 7, compiled with
> MS-C 16.0.40219.1 (i.e., Visual Studio 2008), although it only seems to
> happen of every other input of "^".
>
> The caret is a dead-letter key and is ignored although a space is pressed
> afterwards. It seems to be kept in the input buffer, though, and is
> finally used when "s" is pressed, leading to a movement to the start of
> line and starting insert mode at the wrong position.
>
> The dollar sign you see in Alex' example is the one displayed at the end
> of the changed text if "$" is included in 'cpo'.

additional note: I could only reproduce this in GVim.

[Ben Schmidt]

On 30/10/12 2:40 AM, J�rgen Kr�mer wrote:
> Hi,
>
> Christian Brabandt wrote:
>> On Mon, October 29, 2012 16:17, Axel wrote:
>>> "s" makes the cursor move to the beginning of the line(!) changing the
>>> character found there (hence the "$" sign). Also, the character "^" is in
>>> fact not displayed.
>>
>> And how do you start Vim? Did you test with vim -u NONE -U NONE -N
>> Does cl work like s (it should, since s is an alias to cl)?
>
> this behavior does not depend on using cl or s. The bug lies in ignoring
> the caret in insert mode and using it later before executing the next
> command. It seems that only every other (dead-letter) caret is inserted
> even if it is followed by a space.

Caret is not a dead letter for me. So is this bug locale-dependent?

Ben.

[Christian Brabandt]

Hi Ben!

It is for a German layout usually. But I can't reproduce it. And
possibly also compiler or architecture (32/64bit) dependent.

Mit freundlichen Gr��en
Christian
--
Atomkraft, strahlender Glanz ohne Abtrocknen.

[Jürgen Krämer]

Hi,

Christian Brabandt wrote:
>
> On Di, 30 Okt 2012, Ben Schmidt wrote:
>
>> On 30/10/12 2:40 AM, J�rgen Kr�mer wrote:
>>> Hi,
>>>
>>> Christian Brabandt wrote:
>>>> On Mon, October 29, 2012 16:17, Axel wrote:
>>>>> "s" makes the cursor move to the beginning of the line(!) changing the
>>>>> character found there (hence the "$" sign). Also, the character "^" is in
>>>>> fact not displayed.
>>>>
>>>> And how do you start Vim? Did you test with vim -u NONE -U NONE -N
>>>> Does cl work like s (it should, since s is an alias to cl)?
>>>
>>> this behavior does not depend on using cl or s. The bug lies in ignoring
>>> the caret in insert mode and using it later before executing the next
>>> command. It seems that only every other (dead-letter) caret is inserted
>>> even if it is followed by a space.
>>
>> Caret is not a dead letter for me. So is this bug locale-dependent?
>
> It is for a German layout usually. But I can't reproduce it. And
> possibly also compiler or architecture (32/64bit) dependent.

yesterday, with the example given by Alex I could reproduce it on every
try. Today I tried to construct another example, but it worked only
randomly -- sometimes pressing the caret (or apostrophe or backtick) key
followed by one or more presses of the space bar produced the correct
character immediately, sometimes it was postponed until the next
non-space key was pressed. When it was a letter that could be combined
with the respective accent (like "a" or "e") this was done, in other
cases like "m" or "," the caret and the letter were inserted separately.

I had a look at the source code in gui_w48.c and gui_w32.c, but I could
not determine where the dead letter key and the following key are
combined into one character and wether this actually has to be done by
the application or if it's done by Windows.

[Jürgen Krämer]

Hi,

J�rgen Kr�mer wrote:
>
> Christian Brabandt wrote:
>>
>> On Di, 30 Okt 2012, Ben Schmidt wrote:
>>

additional observation: whenever the caret is not inserted after pressing
the space bar and I use the mouse to switch to another program that allows
input, pressing the space bar there results in the caret being inserted.
It seems Gvim does not remove the dead letters from the input queue at the
correct moment.

A similar behavior can be seen if, e.g., in Notepad one presses the caret
key, immediately switches to another program by using the mouse, and then
presses the space bar: the caret is inserted in the other program, but I
guess this is to be expected.

jkr

[Tony Mechelynck]

On 30/10/12 07:32, J�rgen Kr�mer wrote:
>
> Hi,

>
> J�rgen Kr�mer wrote:
>>
>> Christian Brabandt wrote:

[...]

>>>
>>> It is for a German layout usually. But I can't reproduce it. And
>>> possibly also compiler or architecture (32/64bit) dependent.
>>
>> yesterday, with the example given by Alex I could reproduce it on every
>> try. Today I tried to construct another example, but it worked only
>> randomly -- sometimes pressing the caret (or apostrophe or backtick) key
>> followed by one or more presses of the space bar produced the correct
>> character immediately, sometimes it was postponed until the next
>> non-space key was pressed. When it was a letter that could be combined
>> with the respective accent (like "a" or "e") this was done, in other
>> cases like "m" or "," the caret and the letter were inserted separately.

m or , don't accept a circumflex or acute accent IIUC. OTOH, c g h j s
can have a circumflex in Esperanto, and IIRC w is a vowel in Welsh and
can accept accents.

>>
>> I had a look at the source code in gui_w48.c and gui_w32.c, but I could
>> not determine where the dead letter key and the following key are
>> combined into one character and wether this actually has to be done by
>> the application or if it's done by Windows.

the application ought to be able to accept characters from the keyboard,
e.g. as text input, without even knowing (or caring) whether or not the
current keyboard layout includes dead keys.

>
> additional observation: whenever the caret is not inserted after pressing
> the space bar and I use the mouse to switch to another program that allows
> input, pressing the space bar there results in the caret being inserted.
> It seems Gvim does not remove the dead letters from the input queue at the
> correct moment.

When gvim does not insert a caret after pressing the dead key then the
space bar, what happens if you press the space bar again? (Does the
caret appear?) Or if you press some vowel key instead? (Does it insert
an accented vowel, e.g. � if you pressed the e key?) � Another
expreiment: if you press the space bar and some letter key in rapid
alternation, e.g. x x x x x x x x x x x x x x x x x x x x x , are all
letters separated from each other by one space? (Though when I tried,
using one hand for each, I noticed that my hand didn't always hit the
keyboard in strict alternation.)

>
> A similar behavior can be seen if, e.g., in Notepad one presses the caret
> key, immediately switches to another program by using the mouse, and then
> presses the space bar: the caret is inserted in the other program, but I
> guess this is to be expected.
>
> jkr
>

Sounds like a Windows bug in the keyboard driver to me (not properly
combining the dead-key event with the keypress event for the following
spacing key, at least in some circumstances, perhaps conditional on a
call to some "is there a key waiting?" function not used by Notepad),
but I could be wrong.

Best regards,
Tony.
--
Forms follow function, and often obliterate it.

[Jürgen Krämer]

Hello Tony,

Tony Mechelynck schrieb:

> On 30/10/12 07:32, J�rgen Kr�mer wrote:
>>
>> J�rgen Kr�mer wrote:
>>>
>>> Christian Brabandt wrote:
> [...]
>>>>
>>>> It is for a German layout usually. But I can't reproduce it. And
>>>> possibly also compiler or architecture (32/64bit) dependent.
>>>
>>> yesterday, with the example given by Alex I could reproduce it on every
>>> try. Today I tried to construct another example, but it worked only
>>> randomly -- sometimes pressing the caret (or apostrophe or backtick) key
>>> followed by one or more presses of the space bar produced the correct
>>> character immediately, sometimes it was postponed until the next
>>> non-space key was pressed. When it was a letter that could be combined
>>> with the respective accent (like "a" or "e") this was done, in other
>>> cases like "m" or "," the caret and the letter were inserted separately.
>
> m or , don't accept a circumflex or acute accent IIUC. OTOH, c g h j s
> can have a circumflex in Esperanto, and IIRC w is a vowel in Welsh and
> can accept accents.

it's probably more a question of the encoding than of the language which
letter will be produced. I just wanted to use a key of which I was sure
that when pressed after the dead letter caret would produce the caret
and the corresponding character. "m" and "," are one of those.

The caret key followed by space bar should always produce a caret
character on its own, but sometimes it does not.

BTW, at least Gvim on Windows with enc=utf-8 and fenc=utf-8 does not
seem to produce one of the accented letters you listed, although
pre-composed characters exist for them in the Unicode standard.

>>> I had a look at the source code in gui_w48.c and gui_w32.c, but I could
>>> not determine where the dead letter key and the following key are
>>> combined into one character and wether this actually has to be done by
>>> the application or if it's done by Windows.
>
> the application ought to be able to accept characters from the keyboard,
> e.g. as text input, without even knowing (or caring) whether or not the
> current keyboard layout includes dead keys.

On Windows there are two messages (WM_DEADCHAR and WM_SYSDEADCHAR) which
are sent to a program when Windows has processed dead letter keys. Gvim
reacts to both of them by setting a flag in int _OnDeadChar() function.
I don't know whether this behavior is superfluous, incomplete or wrong.

>> additional observation: whenever the caret is not inserted after pressing
>> the space bar and I use the mouse to switch to another program that allows
>> input, pressing the space bar there results in the caret being inserted.
>> It seems Gvim does not remove the dead letters from the input queue at the
>> correct moment.
>
> When gvim does not insert a caret after pressing the dead key then the
> space bar, what happens if you press the space bar again? (Does the
> caret appear?)

No, it's only after a non-space key that the caret appears -- either on
its own if it's a consonant or as part of an accented letter if it's a
vowel.

> Or if you press some vowel key instead? (Does it insert
> an accented vowel, e.g. � if you pressed the e key?) � Another
> expreiment: if you press the space bar and some letter key in rapid
> alternation, e.g. x x x x x x x x x x x x x x x x x x x x x , are all
> letters separated from each other by one space? (Though when I tried,
> using one hand for each, I noticed that my hand didn't always hit the
> keyboard in strict alternation.)

Alternating between space bar and some letter key works correctly. It's
only the dead letter keys in combination with space bar that don't
always produce the spacing, non-combining character for the accent. And
when this happens (e.g., caret followed by space produces neither of
both), a second caret produces two carets.

>> A similar behavior can be seen if, e.g., in Notepad one presses the caret
>> key, immediately switches to another program by using the mouse, and then
>> presses the space bar: the caret is inserted in the other program, but I
>> guess this is to be expected.
>

> Sounds like a Windows bug in the keyboard driver to me (not properly
> combining the dead-key event with the keypress event for the following
> spacing key, at least in some circumstances, perhaps conditional on a
> call to some "is there a key waiting?" function not used by Notepad),
> but I could be wrong.

I fear it's a bug in Gvim. Windows has messages that can be handled by
the application. Some hold the combined character in their parameters
(like WM_CHAR) and some -- WM_KEYDOWN and WM_KEYUP -- hold information
on which physical key was pressed. Gvim handles these messages inside
its process_message() function and I guess there can go something wrong
with setting and resetting the dead_key variable.

Regards,
J�rgen

[Axel]

I think the fact that this behavior is not evident in the original (release) version of gvim (7.3.046) demonstrates that this is a bug.

[Tony Mechelynck]

On 31/10/12 07:27, Jürgen Krämer wrote:
>
> Hello Tony,
>
> Tony Mechelynck schrieb:

>> On 30/10/12 07:32, Jürgen Krämer wrote:

On my system I get them: ĉ ĝ ĥ ĵ ŝ ŵ (using the dead-^ right of the P on
my Belgian AZERTY keyboard). The superiority of X11 keyboard drivers
over those of Windows, I suppose. See also
http://users.skynet.be/antoine.mechelynck/other/keybbe.htm

>
>>>> I had a look at the source code in gui_w48.c and gui_w32.c, but I could
>>>> not determine where the dead letter key and the following key are
>>>> combined into one character and wether this actually has to be done by
>>>> the application or if it's done by Windows.
>>
>> the application ought to be able to accept characters from the keyboard,
>> e.g. as text input, without even knowing (or caring) whether or not the
>> current keyboard layout includes dead keys.
>
> On Windows there are two messages (WM_DEADCHAR and WM_SYSDEADCHAR) which
> are sent to a program when Windows has processed dead letter keys. Gvim
> reacts to both of them by setting a flag in int _OnDeadChar() function.
> I don't know whether this behavior is superfluous, incomplete or wrong.

Aha! Well, Vim (or, at least, gvim) wants to be able to track wild key
combinations like Shift-Alt-F12 or Ctrl-Alt-PgUp, but dead keys ought to
be a different matter. However, see os_win32.txt lines 212 sqq. I
thought it didn't apply to Win-XP and later, but who knows?

>
>>> additional observation: whenever the caret is not inserted after pressing
>>> the space bar and I use the mouse to switch to another program that allows
>>> input, pressing the space bar there results in the caret being inserted.
>>> It seems Gvim does not remove the dead letters from the input queue at the
>>> correct moment.
>>
>> When gvim does not insert a caret after pressing the dead key then the
>> space bar, what happens if you press the space bar again? (Does the
>> caret appear?)
>
> No, it's only after a non-space key that the caret appears -- either on
> its own if it's a consonant or as part of an accented letter if it's a
> vowel.
>
>> Or if you press some vowel key instead? (Does it insert

>> an accented vowel, e.g. ê if you pressed the e key?) — Another

>> expreiment: if you press the space bar and some letter key in rapid
>> alternation, e.g. x x x x x x x x x x x x x x x x x x x x x , are all
>> letters separated from each other by one space? (Though when I tried,
>> using one hand for each, I noticed that my hand didn't always hit the
>> keyboard in strict alternation.)
>
> Alternating between space bar and some letter key works correctly. It's
> only the dead letter keys in combination with space bar that don't
> always produce the spacing, non-combining character for the accent. And
> when this happens (e.g., caret followed by space produces neither of
> both), a second caret produces two carets.

Hm.

>
>>> A similar behavior can be seen if, e.g., in Notepad one presses the caret
>>> key, immediately switches to another program by using the mouse, and then
>>> presses the space bar: the caret is inserted in the other program, but I
>>> guess this is to be expected.
>>
>> Sounds like a Windows bug in the keyboard driver to me (not properly
>> combining the dead-key event with the keypress event for the following
>> spacing key, at least in some circumstances, perhaps conditional on a
>> call to some "is there a key waiting?" function not used by Notepad),
>> but I could be wrong.
>
> I fear it's a bug in Gvim. Windows has messages that can be handled by
> the application. Some hold the combined character in their parameters
> (like WM_CHAR) and some -- WM_KEYDOWN and WM_KEYUP -- hold information
> on which physical key was pressed. Gvim handles these messages inside
> its process_message() function and I guess there can go something wrong
> with setting and resetting the dead_key variable.
>
> Regards,

> Jürgen
>
Let's hope someone reads this thread with better knowledge of this stuff
than I have.

Best regards,
Tony.
--
If Helen Keller is alone in a forest and falls, does she make a sound?

[Tony Mechelynck]

On 31/10/12 08:04, Axel wrote:
> I think the fact that this behavior is not evident in the original (release) version of gvim (7.3.046) demonstrates that this is a bug.
>

It should be possible (but a lot of work) to determine a "last good" and
a "first bad" version by binary search (dichotomy) and experiment, maybe
with the help of with the W32 versions of gvim at
http://sourceforge.net/projects/cream/files/Vim/ � where the last
version seems mislabeled: it says 6.3.709 on the list of versions but it
offers gvim-7-3-709.exe and gvim-7-3-709_release-notes.txt for downloading.

The procedure is as follows: You already have a "known good" (7.3.046)
and a "known bad" (7.3.712). Try a version somewhere between them. If it
is "good" you have a later "known good"; if it is bad, an earlier "known
bad", so in either case you've narrowed the error range a bit. Try again
until you find the "last good" and the "first bad" and they are as close
as possible to each other. This would help find when the regression
crept in.


Best regards,
Tony.
--
Life is a gift, living is an art. (Bram Moolenaar)

[Axel]

Following Tony's approach I tried to verify the behavior with Cream's 7.3.709 but I couldn't; gvim.exe behaves correctly. The Cream version however is a 32-bit application, where mine is a 64-bit version compiled with MinGW.

@Jürgen

For which platform did you compile/download your version of gvim.exe?

[Ben Fritz]

On Wednesday, October 31, 2012 5:48:51 AM UTC-5, Axel wrote:
> Following Tony's approach I tried to verify the behavior with Cream's 7.3.709 but I couldn't; gvim.exe behaves correctly. The Cream version however is a 32-bit application, where mine is a 64-bit version compiled with MinGW.
>
>

Since you're compiling yourself, Mercurial actually automates Tony's approach with it's "bisect" feature. I assume you're using Mercurial to get the Vim source?

See:

http://www.selenic.com/mercurial/hg.1.html#bisect
http://tortoisehg.bitbucket.org/hgbook/1.6/finding-and-fixing-mistakes.html#sec:undo:bisect

(I expected the second link to show how to do it graphically in TortoiseHg but I was mistaken; it looks to be a good read whether you use the command-line or Tortoise tools).

[Tony Mechelynck]

Yes, and, of course,

hg help bisect

will let you RTFM even if you aren't on Linux ;-)


Best regards,
Tony.
--
Around computers it is difficult to find the correct unit of time to
measure progress. Some cathedrals took a century to complete. Can you
imagine the grandeur and scope of a program that would take as long?
-- Epigrams in Programming, ACM SIGPLAN Sept. 1982

[Steve Hall]

On Wed, Oct 31, 2012 at 6:31 AM, Tony Mechelynck
<antoine.m...@gmail.com> wrote:
>
> ....the W32 versions of gvim at
> http://sourceforge.net/projects/cream/files/Vim/ — where the last

> version seems mislabeled: it says 6.3.709 on the list of versions
> but it offers gvim-7-3-709.exe and gvim-7-3-709_release-notes.txt
> for downloading.

Whoops, fixed this folder name for the record.

--
Steve Hall [ digitect dancingpaper com ]

[Christian Brabandt]

On Mo, 29 Okt 2012, J�rgen Kr�mer wrote:

> I can confirm this behavior on Vim 7.3.1-712 on Windows 7, compiled with
> MS-C 16.0.40219.1 (i.e., Visual Studio 2008), although it only seems to
> happen of every other input of "^".
>
> The caret is a dead-letter key and is ignored although a space is pressed
> afterwards. It seems to be kept in the input buffer, though, and is
> finally used when "s" is pressed, leading to a movement to the start of
> line and starting insert mode at the wrong position.
>
> The dollar sign you see in Alex' example is the one displayed at the end
> of the changed text if "$" is included in 'cpo'.

I still couldn't reproduce the issue with VisualStudio 2010 Express. But
it only compiled a 32bit version and I couldn't convince Windows to let
me install the required 64bit utilities to built a 64bit version.

regards,
Christian