Bullet character across Vim platforms
AndyHancock
corresponding to Windows-1252 code 149 (0x95). I put the script in
vimrc, issues "gvim Temp.txt" from the bash command line (Temp.txt is
nonexistent), and got no warning about missing multi_byte capability
[ not surprising since ":echo has('multi_byte')" yields 1 ]. However,
I'm still not getting the bullet. I tried a few ways.
First, I created a bullet in one of a couple of Windows app (Firefox,
Palm Desktop) using the usual method: Alt-0149 on the number pad.
(Actually, it's a laptops, so I had to use "num lk", which locks some
of the qwerty keys into a number pad function). Then I copied and
pasted the bullet into gvim (mouse middle-button to paste, since it's
X-windows). It pastes as a question-mark character.
I then tried Alt-0149 directly in gvim while in insert mode. No joy,
as this translates into four characters, corrersponding to Alt-0 Alt-1
Alt-4 Alt-9.
Finally, in insert mode, I did Ctrl-V 149, which simply inserts a
character with hex code ox95. I literally shows up as a blue-coloured
"<95>" without quotes. The "ga" command shows this to be a character
with hex code 0x95. I thought that the last method above might be
fine, and perhaps it just wasn't showing up in the gvim window due
some reason related to X-windows fonts. So I copied and pasted the
text into a Windows app...the result was a space in place of the
bullet. Note that the bullet 0x95 copies successfully between windows
apps, and between windows apps and windows-based gvim (as opposed to
cygwin gvim).
I also tried the alternative bullet 0xB7, but that's almost invisible
in Palm Desktop. Better to use an asterisk (which is highly
nonideal).
To troubleshoot the script, I tried querired the following options,
with the shown results:
Encoding
--------
set encoding
Ans: encoding=utf-8
setlocal encoding
Ans: encoding=utf-8
setglobal encoding
Ans: encoding=utf-8
This make sense, since encoding is global.
Fileencoding
------------
set fileencoding
Ans: fileencoding=
setlocal fileencoding
Ans: fileencoding=
setglobal fileencoding
Ans: fileencoding=windows-1252
Fileencodings
-------------
set fileencodings
Ans: fileencodings=ucs-bom,utf-8,Windows-1252
setlocal fileencodings
Ans: fileencodings=ucs-bom,utf-8,Windows-1252
setglobal fileencodings
Ans: fileencodings=ucs-bom,utf-8,Windows-1252
This make sense, since fileencodings is global.
Bomb
----
set bomb?
Ans: nobomb
setlocal bomb?
Ans: nobomb
setglobal bomb?
Ans: bomb
Rather than create a new Temp.txt from the bash command line, I also
tried creating new unnamed files using <Ctrl-w><Ctrl-n> and ":new".
The only difference for these two buffers was that the local value of
fileencoding was Windows-1252, and local boolean bomb option was set.
However, but all the above attempts to create a bullet yielded the
same results.
I also issued "setlocal fileencoding=Windows-1252" in Temp.txt (using
both capital and small "w"), and "setlocal bomb". That prevented me
from saving the file:
E513:
write error, conversion failed
(make 'fenc' empty to override)
This was due to the 0x95 character that I inserted using Ctrl-V. It
turns out that this also affected the two unnamed buffers -- that is,
if I tried to issue "w! Temp3.txt", I get the same error if the 0x95
character is present in the buffer. The only way to be able to write
the file is to setlocal fileencoding to null, or remove the 0x95
character.
I admit that I am far from experienced with character encodings. Is
there anything I'm missing from the solution below?
---------- Forwarded message ----------
From: Tony Mechelynck <antoine.mechely...@gmail.com>
Date: Jan 11 2009, 7:03 pm
Subject: Bullet character across Vim platforms
To: vim_use
On 11/01/09 17:35, AndyHancock wrote:
> Sorry for the repost, but the first time submitted through Google
> Groups yielded a blank submission form. So I have recomposed and
> reposted (20 minutes of time).
>
> I am using:
> 1. Vim6.2 on Windows 2000, Lucida Console font, and
> 2. Vim7.1.2 on Cygwin's Xwin[dows], Lucida Typewriter font, on to of
> Windows 2000
>
> After some surfing, I found that I can get a realbulletcharacter
> (not asterisk or dash) in Windows using ASCII code 149.
>
> A. On windows applications, press Alt, enter 0149 on number pad.
> B. On #1 above in insert mode, enter Ctrl-V followed by 149.
>
> Neither of these work for #2 above. Even if I create abullet
> character using #1 and #B, it shows up as "~U" (minus quotes) in #2.
>
> Is there a way to create bullets in #2?
>
> Is there a way to have those bullets maintain their appearance across
>Vim platforms?
It depends on your 'encoding', which is how Vim represents data in
memory.
It also depends on each file's 'fileencoding', which is how that
file's data is represented on disk.
Of course, to be able to use any given character in a file edited by
Vim, that character must be representable (not necessarily the same
way) in both Vim 's'encoding' and the file's 'fileencoding'.
In the Latin1 aka ISO-8859-1 encoding, the character decimal 149, hex
0x95 is a control character, corresponding to Unicode U+0095 <control>
= MESSAGE WAITING. That character is not printable.
In the Windows-1252 encoding, that same decimal 149 hex 0x95 value is
used to represent a different character, namely the unicode codepoint U
+2022BULLET. That character is not representable in Latin1.
Now you have several possibilities.
First, I recommend using utf-8 for Vim'sinternal representation of the
data in memory, because that 'encoding' can represent any Unicode
codepoint, which means that regardless of the file's
'fileencoding',Vim will be able to represent it in memory. This
requires a binary compiled with +multi_byte -- such a binary will
answer with the number 1 (one) when you ask ":echo has('multi_byte').
Then you will have to decide how to represent the data on disk. For
portability between various computers, Latin1 is recommended; however
this means that anything between 0x80 and 0x9F included is reserved
for non-printable control characters.
If you prefer having an additional 32 characters at your disposal in
an 8-bit encoding, you can use Windows-1252 everywhere, and decide
that you'll represent any 8-bit disk file in that 'fileencoding'. You
could make Vim (with 'encoding' set to utf-8) recognize these files by
means of the command ":set fileencodings=ucs-bom,utf-8,Windows-1252"
in your vimrc (see where in the snippet at the bottom of this email,
and notice the difference between 'fileencoding' [singular] and
'fileencodings' [plural]). The problem with this approach is that if
you publish such documents, anyone with a Unix or Linux or Mac
operating system will probably not display those 32 additional
characters correctly.
Or else, you can choose the Unicode UTF-8 encoding as your preferred
'fileencoding', which doesn't forbid using Latin1, Windows-1252, or
indeed anything else for occasional files. In that case I recommend
using a BOM on Unicode files in order to let them be recognized
unambiguously even by programs other than Vim and by computers other
than your own.
Now here's the promised snippet of code; place it near the top of your
vimrc, after setting ":language" if you use that command but before
defining any mappings. I have added comments to make it as
understandable as I can.
" Unicode can only be used if Vim is compiled with +multi_byte
if has('multi_byte')
" if Vim is already using Unicode, no need to change it
if &encoding !~? '^u'
" avoid clobbering the keyboard/display encoding
if &termencoding == ''
let &termencoding = &encoding
endif
" use UTF-8 internally in Vim memory
set encoding=utf-8
endif
" setup the heuristics to recognize
" how existing files are coded
set fileencodings=ucs-bom,utf-8,Windows-1252
" define defaults for new files
" use Windows-1252 (8 bit) by default
setglobal fileencoding=Windows-1252
" use a BOM on Unicode files
setglobal bomb
" if Vim has no +multi_byte capability, warn the user
else
echomsg "No +multi_byte in this Vim version"
endif
You can vary the details of the above once you understand the general
idea. If you don't change anything, your new files will be created in
Windows-1252, and existing files will be assumed to be Windows-1252
unless they either start with a Unicode BOM, or contain only codes
which are valid for UTF-8 (anything above 0x7F is represented in UTF-8
by at least two bytes with the high bit set, so this will still allow
recognizing your existing bullets). To write one new file in UTF-8
instead, use either
:e ++enc=utf-8 newfile
or
:e newfile
:setlocal fenc=utf-8
(where 'fenc' is of course the short name for the 'fileencoding'
option).
See
:help Unicode
:help +multi_byte
:help 'encoding'
:help 'fileencoding'
:help 'fileencodings'
:help 'termencoding'
:help 'bomb'
:help ++opt
http://vim.wikia.org/wiki/Working_with_Unicode
Oh, and one more thing: For abullet-like character which looks the
same in both Latin1 and Windows-1252, you could use the character
0xB7, corresponding in both of these encodings to the Unicode
codepoint U+00B7 MIDDLE DOT. This is a thinnerbulletthan U+2022 but it
is more portable. This "middle dot" is used in Catalan to separate two
letters l which must be pronounced as a "geminated hard l", as in
col·lega (a colleague) rather than as a single "palatalized l"
intermediary between l and y, as in collar (a collar).
Best regards,
Tony.
--
Cleanliness is next to impossible.
Tony Mechelynck
'fileencoding' empty means that the file will be recorded in the same
charset as 'encoding', i.e., UTF-8. In that charset, 0x95 (or 149
decimal) is a non-printable control character. Since this control
character has no representation in Windows-1252, you cannot convert to
Windows-1252 an UTF-8 buffer which contains it (and remember,
'encoding', not 'fileencoding', defines how Vim represent the data in
memory).
As I said in my previous post, to generate a file encoded on disk in
Windows-1252 with a Windows-1252 bullet (0x95 on disk) in it, you must:
1) keep 'encoding' to UTF-8 as above
2) :setlocal fileencoding=cp1252 " (or: :setlocal fenc=windows-1252 )
3) Enter the character into Vim as the Unicode representation of the
equivalent character, i.e. U+2022. To do that, in Insert mode, type
(with no intervening spaces, I add them here only for legibility):
Ctrl-V u 2 0 2 2 (but if your Ctrl-V has been remapped to the Paste
operation, use Ctrl-Q instead).
See
:help i_CTRL-V_digit
:help i_CTRL-Q
:help CTRL-V-alternative
>
> setglobal fileencoding
> Ans: fileencoding=windows-1252
This is the global default, but it won't be applied to the file because
the local value is different.
> if&encoding !~? '^u'
Best regards,
Tony.
--
hundred-and-one symptoms of being an internet addict:
242. You turn down a better-paying job because it doesn't come with
a free e-mail account.
AndyHancock
Actually, I'm just discovering this from reading at the same time you
responded. What happened was that I suspected the bullet wasn't
getting properly copied over by the bridge between the Windows cut/
paste buffer and X-windows's cut/paste buffer. So on the Windows
side, I used Notepad to save a text file containing the desired
bullet. Then I opened it using Vim, but I had to set encoding=utf-8
for the bullet to show properly. "ga" revealed the 4-digit hex code,
which was no longer decimal x95, but the help showed how to enter 4-
digit hex codes. If I originated the file from the unix/cygwin/gvim
side, I'd have to be sure to set fileformat=dos before I could see the
bullet.
All this to say that if I want to use Vim to work on text from the
Palm Desktop, I need to save it to a DOS text file by pasting into
Notepad first. Ah well. A bit more buttonology won't kill me, though
I have to admit it's not convenient. However, it isn't inconvenient
enough for me to install the Windows version of gvim -- that would be
so overkill.
Thanks, Tony.
Tony Mechelynck
To save a file in dos fileformat (i.e., with CR-LF ends of lines), use
:setlocal ff=dos
before you save the file.
To have Vim _recognize_ that an existing file is in dos fileformat when
reading, be sure that the 'fileformats' (plural) option contains "dos", e.g.
:set ffs=dos,unix
Note that if the file contains some lines with CR-LF and some without CR
(which also applies if the last line has no end-of-line at all), Vim
will prefer Unix to Dos (and show carriage returns as ^M) unless you use
++ff=dos (see :help ++enc); this ++ff modifier wasn't always handled
correctly before Vim 7.2.040.
If you force Dos fileformat with ++ff=dos, then on _reading_ the file a
linefeed preceded or not by a carriage return will terminate the line,
but on _writing_ it *every* line will get both CR and LF at the end.
The 'fileformat' option concerns _only_ the ends-of-lines; not the way
some character (such as the bullet) is represented inside the file.
Best regards,
Tony.
--
An egg has the shortest sex-life of all: if gets laid once; it gets
eaten once. It also has to come in a box with 11 others, and the only
person who will sit on its face is its mother.
AndyHancock
wrote:
> On 18/01/10 01:15, AndyHancock wrote:
>
>
>
> > On Sun, Jan 17, 2010 at 6:46 PM, Tony Mechelynck
Yes, got it. Just checked. I will need set ff=dos because the main
reason for using this bullet is so that I have a decent vim editor
when editor text from the Palm Desktop. For this purpose, even if I
create a new spanking file, I need to save it a DOS file and open it
with Notepad. Simply because the bridge between X-windows cut/paste
and Windows cut/paste doesn't preserve the bullet.
And thanks for the tidbits on forced (temporary) setting of ff for a
single write command.
Tony Mechelynck
IMHO it's easier to keep native-Windows (with gvim GUI for Windows,
and/or Vim for Windows running in cmd.exe) and Cygwin (with Vim for
Cygwin running in bash) apart from each other. If you need to copy-paste
between Vim and other Windows applications, I recommend using gvim for
Windows (which can be built in Cygwin as a kind of "cross-compile", but
doesn't need Cygwin to run), which natively "understands" the Windows
clipboard as "* or "+. Now YMMV.
Best regards,
Tony.
--
George Washington was first in war, first in peace -- and the first to
have his birthday juggled to make a long weekend.
-- Ashley Cooper
AndyHancock
wrote:
> IMHO it's easier to keep native-Windows (with gvim GUI for Windows,
> and/or Vim for Windows running in cmd.exe) and Cygwin (with Vim for
> Cygwin running in bash) apart from each other. If you need to copy-paste
> between Vim and other Windows applications, I recommend using gvim for
> Windows (which can be built in Cygwin as a kind of "cross-compile", but
> doesn't need Cygwin to run), which natively "understands" the Windows
> clipboard as "* or "+. Now YMMV.
When you say keep them separate, do you mean not have them no the same
machine? I have kept them together on the same machine before,
though it was in a previous laptop. However, they were completely
different apps. On was installed under the cygwin tree while the
other used the Windows installer. I was able to use the same vimrc.
Unfortunately, the "!" command in the Windows version didn't shell out
to bash. I might have been able to force it to shell out to bash at
some point through some through some abomination of vimrc scripting,
but it was far from robust so I didn't bother keeping bother keeping
track of how it was done.
Anyway, I was trying avoid doing a Windows installation of gvim
because it seemed excessive to have two gvim's on the same system.
However, I may yet go back on that decision simply because of the
inconvenience of having to transfer text to Notepad and write it to a
file before sic'ing gvim onto it. I will likely not do the cygwin
cross-compile route simply for lack of time to become technically
competent enough (and because the windows installer is readily
available).
AndyHancock
BTW, the reason I'd forgoe the Windows gvim is because the shelling
that managed to force wasn't robust. I meant not just shelling out to
a bash command line, but really convenient mixing and matching with
bash like
'a,. w !SomeBashCommand
'a,.!someBashFilterCommand
It's simple, but oh so convenient to cobble together bash snippets in
vim.
However, I will miss the ability to simply do a Windows cut of a swath
of text (say, from a Palm Desktop's Memo app, or a Firefox window
submitting a post through Google Groups), dumping it into gvim with
with middle mouse button (works with both Windows & cygwin/X-windows
gvim), editing the text, then cutting and pasting back to the windows
app. I do that all the time (vim-less editing is so....neutered?)
Well, actually, I can still do it, but only if there are no bullet
symbols -- bullets would require the more round-about way of dumping
it into Notepad, writing to a file, then opening that with gvim. And
you can bet that the default path for Notepad is far different from
the one for my Notepad; that would be too easy. Fortunately, this
wrinkle can be likely smoothened with a few soft links in Cygwin.
Tony Mechelynck
To shell out to bash, use Cygwin console Vim. You can shell out from
Windows gvim too, but preferably to cmd.exe. The language is less
powerful than bash's but it is possible to do quite interesting things
with it (not always without GOTO statements, at least in the "oldest"
Dos flavours of the language).
>
> However, I will miss the ability to simply do a Windows cut of a swath
> of text (say, from a Palm Desktop's Memo app, or a Firefox window
> submitting a post through Google Groups), dumping it into gvim with
> with middle mouse button (works with both Windows& cygwin/X-windows
> gvim), editing the text, then cutting and pasting back to the windows
> app. I do that all the time (vim-less editing is so....neutered?)
> Well, actually, I can still do it, but only if there are no bullet
> symbols -- bullets would require the more round-about way of dumping
> it into Notepad, writing to a file, then opening that with gvim. And
> you can bet that the default path for Notepad is far different from
> the one for my Notepad; that would be too easy. Fortunately, this
> wrinkle can be likely smoothened with a few soft links in Cygwin.
>
On Windows, you would use Edit=>Cut (or Ctrl-X on non-Vim, or "+d on
Vim), Edit=>Copy (or Ctrl-C on non-Vim or "+y on Vim), Edit=>Paste (or
Ctrl-V on non-Vim, or "+p or "+P on Vim). Or, linewise on Vim,
:[range]d+ :[range]y+ and :[range]put+
see
:help quoteplus
:help :y
:help :d
:help :put
Remember, X11 has the "selection" (for middle-mouse-pasting anything
selected) and the "clipboard" (for Edit=>Cut, Edit=>Copy, Edit=>Paste);
the Windows GUI has only the latter.
Best regards,
Tony.
--
You have prepared a proposal for your supervisor. The success of this
proposal will mean increasing your salary 20%. In the middle of your
proposal your supervisor leans over to look at your report and spits
into your coffee. You:
(a) Tell him you take your coffee black.
(b) Ask him if he has any communicable diseases.
(c) Show him who's in command; promptly take a leak in his "In"
basket.
Tony Mechelynck
> On Jan 17, 9:49 pm, Tony Mechelynck<antoine.mechely...@gmail.com>
> wrote:
>> IMHO it's easier to keep native-Windows (with gvim GUI for Windows,
>> and/or Vim for Windows running in cmd.exe) and Cygwin (with Vim for
>> Cygwin running in bash) apart from each other. If you need to copy-paste
>> between Vim and other Windows applications, I recommend using gvim for
>> Windows (which can be built in Cygwin as a kind of "cross-compile", but
>> doesn't need Cygwin to run), which natively "understands" the Windows
>> clipboard as "* or "+. Now YMMV.
>
> When you say keep them separate, do you mean not have them no the same
> machine? I have kept them together on the same machine before,
> though it was in a previous laptop. However, they were completely
> different apps. On was installed under the cygwin tree while the
> other used the Windows installer. I was able to use the same vimrc.
> Unfortunately, the "!" command in the Windows version didn't shell out
> to bash. I might have been able to force it to shell out to bash at
> some point through some through some abomination of vimrc scripting,
> but it was far from robust so I didn't bother keeping bother keeping
> track of how it was done.
I mean you can have them on the same machine but (in most cases) it's
easier to use Windows-native with native and Cygwin with Cygwin. It is
possible to mix them, but beware of paths (see man cygpath), and I've
never succeeded to get Cygwin X11 running satisfactorily, so for me
Cygwin is more like a Windows emulator "text-only" Unix environment
similar to init runlevel 3 on Linux -- let's say Wine in reverse (where
Wine is a WIN-dows E-mulator running on Linux).
>
> Anyway, I was trying avoid doing a Windows installation of gvim
> because it seemed excessive to have two gvim's on the same system.
> However, I may yet go back on that decision simply because of the
> inconvenience of having to transfer text to Notepad and write it to a
> file before sic'ing gvim onto it. I will likely not do the cygwin
> cross-compile route simply for lack of time to become technically
> competent enough (and because the windows installer is readily
> available).
>
I avoid Cygwin install of gvim. When I was on Windows I had three Vim
builds: Windows gvim.exe (GUI), Windows vim.exe (for use in cmd.exe,
either windowed xterm-like or full-screen text-only terminal) and Cygwin
/bin/vim compiled by Cygwin, and thus always slightly out-of-date
compared to my own "fully patched" builds (for use in bash terminal).
The Cygwin cross-compile is already done by Steve Hall who periodically
generates a Windows installer,
http://sourceforge.net/projects/cream/files/ You *may* compile Vim for
yourself (it isn't very difficult: the "make" program does most of the
work) if you want a different choice of features, but you don't *have*
to. See (for Windows) my HowTo page
http://users.skynet.be/antoine.mechelynck/vim/compile.htm -- even if you
don't want to do it, you may have a look at it (and if there's anything
you don't understand, I'd like to know about it).
Best regards,
Tony.
--
hundred-and-one symptoms of being an internet addict:
243. You unsuccessfully try to download a pizza from www.dominos.com.
AndyHancock
have to schedule a lot of time. However, many of the problems I had
may have stemmed from the fact that I only had a user account, and if
I recall correctly, this may have caused problems with installation of
prerequisite software.
I looked at your installation instructions, and I have to admit that
it's still something for me to jump into when I have lots of time.
I'll live with cygwin installs of gvim for the time being.
Thank you for your advice.
On Jan 18, 12:22 am, Tony Mechelynck <antoine.mechely...@gmail.com>
wrote:
> generates a Windows installer,http://sourceforge.net/projects/cream/files/You *may* compile Vim for
> yourself (it isn't very difficult: the "make" program does most of the
> work) if you want a different choice of features, but you don't *have*
> to. See (for Windows) my HowTo pagehttp://users.skynet.be/antoine.mechelynck/vim/compile.htm-- even if you