Simple mapping accepted, but incorrectly displayed.
Erik Christiansen
map <A-a> �
map <A-e> �
map <A-o> �
map <A-<> �
map <A->> �
in an effort to make vim a little more Danish-friendly. All the mappings
are accepted without complaint, but the characters displayed when the
mappings are invoked are not those mapped. I've tried both with
fenc=latin1 and fenc=utf-8. In each case, the displayed characters are:
<A-a> �
<A-e> �
<A-o> �
<A-<> �
<A-<> �
This post was input with fenc=utf-8, and the mappings set by pasting the
above map statements into the ex command line.
It would be quite delightful if this could be made to work, but I'm
struggling to figure out how.
Erik
--
If at first you don't succeed, destroy all evidence that you tried.
- Ralf Hildebrandt, on postfix-users.
Erik Christiansen
Just for the record, digraphs work fine, and display just fine.
It is just the mappings which do not return the programmed mapping..
The mapping-induced corruption is identical with:
$ vim -N -u NONE
and the mappings entered by hand, using digraphs, i.e. not mouse-pasted.
(The latter leaves the X clipboard open to suspicion.)
so we still have:
map <A-a> � invokes as �
map <A-e> � invokes as �
map <A-o> � invokes as �
map <A-<> � invokes as �
map <A->> � invokes as �
But:
imap <A-a> � invokes as �
imap <A-e> � invokes as �
imap <A-o> � invokes as �
imap <A-<> � invokes as �
imap <A->> � invokes as �
Now 40% of the mappings are working. I wonder if that's real progress,
or coincidence?
Erik
--
I frequently hear "it's not rocket science, is it?". No, but I recently saw a
slide of the infamous "O rings" which failed on the Challenger shuttle. Under a
layer of frost, you can just read "Do not use in temperatures below 16C". So
even rocket science is not rocket science. - User feedback seen on BBC website
Ben Fritz
On Jun 14, 3:59 am, Erik Christiansen <dva...@internode.on.net> wrote:
> In Vim 7.3, I added the following mappings:
>
> map <A-a>
> map <A-e>
> map <A-o>
> map <A-<>
> map <A->>
>
> in an effort to make vim a little more Danish-friendly. All the mappings
> are accepted without complaint, but the characters displayed when the
> mappings are invoked are not those mapped. I've tried both with
> fenc=latin1 and fenc=utf-8.
latin1 encoding but I may be wrong. You can use the :scriptencoding
command to change this.
> In each case, the displayed characters are:
>
> <A-a>
> <A-e>
> <A-o>
> <A-<>
> <A-<>
>
you do to get them to display anything?
> This post was input with fenc=utf-8, and the mappings set by pasting the
> above map statements into the ex command line.
>
possibly the 'termenc' options affect this. 'fenc' only affects the
bytes which are written to disk (and whether the characters are
representable on disk at all).
ZyX
sent 12:59:26 14 June 2011, Tuesday
by Erik Christiansen:
First, you probably need insert mode mappings. Second, can you answer why you
don't use `nore'? It is a general rule: use `nore' unless you know what you are
doing. Third, you should specify encoding of the file where mappings are.
Fourth, you should be sure that internal vim encoding ('encoding' option) has
requested characters.
So you should write:
" In the vimrc
set encoding=utf-8
" At the very top of any file containing non-ascii characters
scriptencoding utf-8
<...>
inoremap <A-a> å
inoremap <A-e> æ
inoremap <A-o> ø
inoremap <A-<> «
inoremap <A->> »
Note that there are some issues with unicode characters containing 0x80 byte. If
some of the mappings does not work, try replacing it with
inoremap <expr> <A-C> "\uXXXX"
(where C is actual character and XXXX is a hex code for rhs).
Original message:
> In Vim 7.3, I added the following mappings:
>
> map <A-a> å
> map <A-e> æ
> map <A-o> ø
> map <A-<> «
> map <A->> »
>
> in an effort to make vim a little more Danish-friendly. All the mappings
> are accepted without complaint, but the characters displayed when the
> mappings are invoked are not those mapped. I've tried both with
> fenc=latin1 and fenc=utf-8. In each case, the displayed characters are:
>
> <A-a> á
> <A-e> å
> <A-o> ï
Erik Christiansen
> First, you probably need insert mode mappings. Second, can you answer
> why you don't use `nore'?
Certainly. There are several contributing reasons:
1) In more than a decade of using Vim, I haven't encountered any
suggestion in the documentation that that might be advisable.
If it is important, then overview text or a link at ':h map' would
be helpful to users.
2) 'map' continues to work fine for me, on mappings which I have
used for many years, so there has been no need to look for
anything else. (Though I see that I have recently added a few
'imap' cases.)
3) Apart from respecting your warning that there might be some
undefined trouble at some future point, due to some cause, there
has been no real or imagined reason to depart from using 'map'.
I'll move over to inoremap, as you advise.
> It is a general rule: use `nore' unless you know what you are
> doing. Third, you should specify encoding of the file where mappings are.
> Fourth, you should be sure that internal Vim encoding ('encoding' option) has
> requested characters.
>
> So you should write:
>
> " In the vimrc
> set encoding=utf-8
>
> " At the very top of any file containing non-ascii characters
> scriptencoding utf-8
> <...>
> inoremap <A-a> �
> inoremap <A-e> �
> inoremap <A-o> �
> inoremap <A-<> �
> inoremap <A->> �
> Note that there are some issues with unicode characters containing 0x80 byte. If
> some of the mappings does not work, try replacing it with
> inoremap <expr> <A-C> "\uXXXX"
> (where C is actual character and XXXX is a hex code for rhs).
Zyx, you are a 24 carat wizard. That works in the first three cases!
And yes, when using the digraph instead of <expr> and hex value, it is
necessary to type <A-a> _Twice_ to get one �. And <A-e> is the same, but
it takes _Three_ <A-o> before one � is issued.
Your impressively masterly magic is still evaded by Vim in the last two
cases. Using ':digr', I get � == 171, � == 187. That's 0xAB and 0xBB,
counting on my fingers, so these are my current mappings, in .vimrc:
" To allow digraphed characters in mappings:
" (Else Vim corrupts by default between mapping and invocation.)
set encoding=utf-8
scriptencoding utf-8
inoremap <expr> <A-a> "\uE5"
inoremap <expr> <A-e> "\uE6"
inoremap <expr> <A-o> "\uF8"
inoremap <expr> <A-<> "\uAB"
inoremap <expr> <A->> "\uBB"
The last two mappings still provide corrupt output:
<A-<> becomes � instead of �
<A->> becomes � instead of �
just as was the case with the original mapping.
Many thanks for these fixes and workarounds, which cure the majority of
the mapping corruption very satisfactorily.
I wonder why the last two mappings continue to corrupt so intractably?
Thanks again.
Erik
--
Due to circumstances beyond our control, we regret to inform you that
circumstances are beyond our control. -Paul Benoit
Erik Christiansen
> On Jun 14, 3:59�am, Erik Christiansen <dva...@internode.on.net> wrote:
> > In Vim 7.3, I added the following mappings:
> >
> > map <A-a>
> > map <A-e>
> > map <A-o>
> > map <A-<>
> > map <A->>
> >
>
> I see only incomplete mappings here.
Yes, the problem may lie in your MUA's charset settings. I have no
experience with google mail, which appears to be what you're using, so
cannot say whether it contributes to the display problem. You'll
probably need to fix the problem before being able to contribute
meaningfully to diagnosing this intriguing mapping corruption issue.
Erik
--
The universe contains any amount of horrible ways to be woken up,
such as the noise of the mob breaking down the front door, the scream
of fire engines, or the realization that today is the Monday which on
Friday night was a comfortably long way off.
- Terry Pratchett, _Moving Pictures_
Benjamin R. Haskell
> On 14.06.11 07:47, Ben Fritz wrote:
>> On Jun 14, 3:59 am, Erik Christiansen wrote:
>>> In Vim 7.3, I added the following mappings:
>>>
>>> map <A-a>
>>> map <A-e>
>>> map <A-o>
>>> map <A-<>
>>> map <A->>
>>>
>>
>> I see only incomplete mappings here.
>
> Yes, the problem may lie in your MUA's charset settings.
It's a problem in Google Groups that's come up a few times on this list
now. I recall someone's footnote markers (¹, ², ³, etc.) always get
mangled.
Look at the thread as archived:
http://groups.google.com/group/vim_use/browse_thread/thread/518f2f1b073792fa
The {rhs} is missing in the same way as Ben F quoted.
Groups mangled your initial mail since it wasn't in UTF-8. I received
it as quoted-printable-encoded ISO-8859-1 (Latin-1). Whenever this
problem occurs, usually some people receive it fine (works for me, in
this case -- I see the {rhs} of the mappings). Something's just going
wrong on the Groups end of things, and apparently it happens before some
people receive the mail.
I virtually always use UTF-8. In an attempt to get it archived
correctly, here's how portions of your initial email appeared:
> In Vim 7.3, I added the following mappings:
>
> map <A-a> å
> map <A-e> æ
> map <A-o> ø
> map <A-<> «
> map <A->> »
>
> [...] In each case, the displayed characters are:
>
> <A-a> á
> <A-e> å
> <A-o> ï
> <A-<> ¬
> <A-<> ®
--
Best,
Ben
Erik Christiansen
> Groups mangled your initial mail since it wasn't in UTF-8. I
> received it as quoted-printable-encoded ISO-8859-1 (Latin-1).
Aaargggh! Quite when, is lost in the mists of time now, but I have set:
$ locale
LANG=en_AU.utf8
LC_CTYPE="en_AU.utf8"
etc.
And Vim is supposed to default to utf-8 these days. So now lets' try to
find out why we're triggering the google groups deficiency:
OK, what is Vim set to while composing this post?:
: set fenc ?
fileencoding=utf-8
So the post should be leaving Vim as utf-8.
OK, is mutt changing that to ISO-8859-1 behind the scenes?:
:r !grep char ~/.muttrc # shows us:
# For M$_char_crap, use: (Kyle)
charset-hook ^unknown-8bit$ WINDOWS-1252
charset-hook ^x-user-defined$ WINDOWS-1252
charset-hook ^iso-8859-1$ WINDOWS-1252
charset-hook ^us-ascii$ WINDOWS-1252
charset-hook US-ASCII ISO-8859-1
charset-hook x-unknown ISO-8859-1
Well if it is, then I'm not seeing it. (It's sneaking up on midnight
here, so I'll look later at how this post comes back from the
listserver. Maybe there's a clue there.)
Many thanks for the heads-up. We should do what we can to compensate for
broken mail services, if it helps anyone stuck with using them.
> I virtually always use UTF-8. In an attempt to get it archived
> correctly, here's how portions of your initial email appeared:
>
> >In Vim 7.3, I added the following mappings:
> >
> >map <A-a> �
> >map <A-e> �
> >map <A-o> �
> >map <A-<> �
> >map <A->> �
> >
> >[...] In each case, the displayed characters are:
> >
> ><A-a> �
> ><A-e> �
> ><A-o> �
> ><A-<> �
> ><A-<> �
Yup, that's what I sent. When we have this fixed, I'll have learnt
another detail.
Thanks again.
Erik
--
Once they go up, who cares where they come down?
That's not my department.
- Werner von Braun
Christian Brabandt
> So the post should be leaving Vim as utf-8.
> OK, is mutt changing that to ISO-8859-1 behind the scenes?:
> :r !grep char ~/.muttrc # shows us:
> # For M$_char_crap, use: (Kyle)
> charset-hook ^unknown-8bit$ WINDOWS-1252
> charset-hook ^x-user-defined$ WINDOWS-1252
> charset-hook ^iso-8859-1$ WINDOWS-1252
> charset-hook ^us-ascii$ WINDOWS-1252
> charset-hook US-ASCII ISO-8859-1
> charset-hook x-unknown ISO-8859-1
>
> Well if it is, then I'm not seeing it. (It's sneaking up on midnight
> here, so I'll look later at how this post comes back from the
> listserver. Maybe there's a clue there.)
Look for the charset variables. Most probably its something like this:
set send_charset="us-ascii:iso-8859-15:utf-8"
which would use the first encoding in which the message "fits".
regards,
Christian
ZyX
by Erik Christiansen:
> 2) 'map' continues to work fine for me, on mappings which I have
> used for many years, so there has been no need to look for
> anything else. (Though I see that I have recently added a few
> 'imap' cases.)
>
> 3) Apart from respecting your warning that there might be some
> undefined trouble at some future point, due to some cause, there
> has been no real or imagined reason to depart from using 'map'.
Reason for using `nore' is that remappable mappings can interfer with each other
in an unwanted fashion, while `nore' version works always. So, for example, if I
wanted to exchange `:' and `;' because `;' is faster to type and `:' is needed
more often then `;', I would simply write
noremap ; :
noremap : ;
. Now try to do this with your setup.
*I do not know what I will map in the next few month.*
With `nore' I know that whatever I will want to map, it won't break things, and
thus I should not bother myself with thinking about it.
> The last two mappings still provide corrupt output:
> <A-<> becomes ¬ instead of «
> <A->> becomes ® instead of »
> just as was the case with the original mapping.
Maybe some problems with `<A-<>' itself. I do not know how to avoid them:
`inoremap <A-<> smth' does not work for me too. For terminals there is currently
a workaround: `inoremap <Esc><LT> smth'. It works on my machine, but I know it
to depend on setup of your terminal emulator.
> Your impressively masterly magic is still evaded by Vim in the last two
> cases. Using ':digr', I get « == 171, » == 187. That's 0xAB and 0xBB,
> counting on my fingers, so these are my current mappings, in .vimrc:
To get hex value of some character I usually write it in a buffer, put cursor on
it and type `ga' in normal mode ('encoding', 'fileencoding' and $LANG are utf-8.
Guess only the first should matter).
Original message:
> > inoremap <A-a> å
> > inoremap <A-e> æ
> > inoremap <A-o> ø
> > inoremap <A-<> «
> > inoremap <A->> »
> >
> > Note that there are some issues with unicode characters containing 0x80
> > byte. If some of the mappings does not work, try replacing it with
> >
> > inoremap <expr> <A-C> "\uXXXX"
> >
> > (where C is actual character and XXXX is a hex code for rhs).
>
> Zyx, you are a 24 carat wizard. That works in the first three cases!
>
> And yes, when using the digraph instead of <expr> and hex value, it is
> necessary to type <A-a> _Twice_ to get one å. And <A-e> is the same, but
> it takes _Three_ <A-o> before one ø is issued.
Benjamin R. Haskell
> On Wed, June 15, 2011 3:38 pm, Erik Christiansen wrote:
>> So the post should be leaving Vim as utf-8.
>> OK, is mutt changing that to ISO-8859-1 behind the scenes?:
Yes. Alpine also does this by default.
>> :r !grep char ~/.muttrc # shows us:
...shows us what you've explicitly set. Use:
mutt -Q send_charset
-Q = query a configuration variable
On my system (no ~/.muttrc), it returns:
send_charset="us-ascii:iso-8859-1:utf-8"
As Christian explained, mutt will:
> use the first encoding in which the message "fits".
The comments in /etc/Muttrc expand on "fits":
# Mutt will use the first character set into which the text can be
# converted exactly.
If I used mutt, I'd set it explicitly to send_charset="utf-8"
--
Best,
Ben
Bert Swart
<snipped>
> $ locale
> LANG=en_AU.utf8
> LC_CTYPE="en_AU.utf8"
> etc.
>
> And Vim is supposed to default to utf-8 these days. So now lets' try to
> find out why we're triggering the google groups deficiency:
>
> OK, what is Vim set to while composing this post?:
> : set fenc ?
> fileencoding=utf-8
>
> So the post should be leaving Vim as utf-8.
> OK, is mutt changing that to ISO-8859-1 behind the scenes?:
I guess the smtp server is changing your charset...
> :r !grep char ~/.muttrc # shows us:
> # For M$_char_crap, use: (Kyle)
> charset-hook ^unknown-8bit$ WINDOWS-1252
> charset-hook ^x-user-defined$ WINDOWS-1252
> charset-hook ^iso-8859-1$ WINDOWS-1252
> charset-hook ^us-ascii$ WINDOWS-1252
> charset-hook US-ASCII ISO-8859-1
> charset-hook x-unknown ISO-8859-1
<snipped>
Regards,
Bert
Erik Christiansen
> On Wed, 15 Jun 2011, Christian Brabandt wrote:
[correct analyses of the cause snipped]
Thankyou guys. I'll set send_charset="utf-8", as advised.
That'll fix the munging by google. I hope that my Danish correspondents,
who use m$, will be able to handle utf-8. (I've never used that OS, so
don't know what it can do.)
Erik
--
A completely planned economy ensures that when no bacon is delivered,
no eggs are delivered at the same time. - Leo Frain
Erik Christiansen
> *I do not know what I will map in the next few month.*
> With `nore' I know that whatever I will want to map, it won't break things, and
> thus I should not bother myself with thinking about it.
Yes, now that I am beginning to use a few more mappings, it is time to
think defensively too.
> > The last two mappings still provide corrupt output:
> > <A-<> becomes ¬ instead of «
> > <A->> becomes ® instead of »
> > just as was the case with the original mapping.
> Maybe some problems with `<A-<>' itself. I do not know how to avoid them:
> `inoremap <A-<> smth' does not work for me too. For terminals there is currently
> a workaround: `inoremap <Esc><LT> smth'. It works on my machine, but I know it
> to depend on setup of your terminal emulator.
Now that I've had sleep, your <expr> fix works here. It is embarrassing
to admit that I didn't press <shift-A-<> to get <A-<>, but only what
was explicitly in the mapping. That just goes to show the benefit of
taking a break at midnight.
> To get hex value of some character I usually write it in a buffer, put cursor on
> it and type `ga' in normal mode ('encoding', 'fileencoding' and $LANG are utf-8.
> Guess only the first should matter).
Yes, I remembered that there is something like that, but not the
required keystrokes. Thanks for the hint.
I'll add some notes to my Vim survival jottings, because I'd never find
all the parts of the mapping solution in the Vim documentation.
Thank you Zyx, for a complete and user-friendly fix.
Erik
--
The advantage of a bad memory is that one enjoys several times the same
good things for the first time.
- Friedrich Nietzsche