File encoding & line ending in Vim
Ciccio Bodoni
keyboard) with the Consolas font. At my workplace all other developers
have Textmate for Mac OS X. To avoid any encoding issue we decided
that:
- the default file encoding has to be UTF-8
- the default line ending has to be LF
(In fact these are the default settings in Textmate)
Now I have to build a gvimrc file so that:
- I can read, edit and save every file from Textmate (UTF-8 + LF)
- all new files I create are UTF-8 + LF
I'm a newbie so any help is appreciated. Thanks in advance.
CB
Ben Schmidt
Piece of cake!
In (g)vim, find out where your home directory is by issuing
:echo $HOME
Then create a file in that directory called _vimrc and put this in it:
set encoding=utf8
set ffs=unix,dos
Apart from those two, I believe all the Vim defaults will suit your needs. For
more info on these options, see
:help 'encoding'
:help 'fileformats'
Hope this helps. Let us know if you need further help.
Ben.
Send instant messages to your online friends http://au.messenger.yahoo.com
Dominique Pelle
> Ciccio Bodoni wrote:
> > Hi, I'm using gvim in Italian on Windows XP SP2 (Italian + Italian
> > keyboard) with the Consolas font. At my workplace all other developers
> > have Textmate for Mac OS X. To avoid any encoding issue we decided
> > that:
> >
> > - the default file encoding has to be UTF-8
> > - the default line ending has to be LF
> >
> > (In fact these are the default settings in Textmate)
> >
> > Now I have to build a gvimrc file so that:
> >
> > - I can read, edit and save every file from Textmate (UTF-8 + LF)
> > - all new files I create are UTF-8 + LF
> >
> > I'm a newbie so any help is appreciated. Thanks in advance.
>
> Piece of cake!
>
> In (g)vim, find out where your home directory is by issuing
>
> :echo $HOME
>
> Then create a file in that directory called _vimrc and put this in it:
>
> set encoding=utf8
> set ffs=unix,dos
Ciao
I think you meant "set fileencodings ..." rather than "set encoding ..."
("set encoding" being is for the internal representation in vim which
should typically not be changed).
I use the following setting:
set fileencodings=ucs-bom,utf-8,latin1
See ":help fileencodings" for more information.
-- Dominique
Tony Mechelynck
By default, Vim will detect the kind of line endings used on existing files
and respect it; for new files or if you want to change the EOL for a given
files, you can specify it by ":setlocal fileformat=unix". See for more details:
:help 'fileformats'
:help 'fileformat'
:help ++opt
and what they resend to.
For using UTF-8 with Vim, see among others
http://vim.wikia.com/wiki/Working_with_Unicode
Note that for UTF-8 files meant to be read on XP by Windows software, it is
often useful to use ":setlocal bomb" so that the file starts with a BOM: for
instance, when I was on XP SP2, I noticed that WordPad can read UTF-8 files if
they have a BOM, but when saving "Unicode" files, what it produces is UTF-16le
with BOM (which, BTW, Vim can also read; see the above-mentioned links).
So here is an example of what your vimrc might look like (feel free to make
changes). Lines starting with a double quote are comments.
" Vim configuration file
" Uncomment the following to force English menus and messages
if 0
if has("unix")
language messages C
else
language messages en
endif
endif
" Set various useful settings
runtime vimrc_example.vim
" We want to use Unicode files: can we use multibyte encodings?
if has("multi_byte")
" is the default locale already a Unicode one? if not, set UTF-8
if &enc !~? '^u'
" avoid clobbering the keyboard encoding
if &tenc == ""
let &tenc = &enc
endif
set enc=utf-8
endif
" define heuristics for encoding detection in existing files
" ucs-bom (if present) must be first
" there can be at most one 8-bit encoding (e.g. Latin1) and
" it must come last
set fencs=ucs-bom,utf-8,latin1
" set encoding & BOM for newly created files
setglobal bomb fenc=utf-8
" create new files with Unix (LF) EOL by default
set fileformats=unix,dos,mac
setglobal fileformat=unix
" warning! fileformats (plural) != fileformat (singular)
else
echoerr "Error= multi-byte disabled at compile time in this Vim"
finish
endif
" uncomment the following line to disable filetype-specific indenting
" filetype indent off
" if in gvim, set the font: 5 possible formats, each version of gvim accepts
" only one of them
if has("gui_running")
if has("gui_gtk2")
set gfn=Courier\ New\ 10
elseif has("gui_photon")
set gfn=Courier\ New:s10
elseif has("gui_kde")
set gfn=Courier\ New/10/-1/5/50/0/0/0/1/0
elseif has("x11")
set gfn=*-courier-medium-r-normal-*-*-100-*-*-m-*-*
else
set gfn=Courier_New:h10:cDEFAULT
endif
" if gvim, start maximized
set lines=99999 columns=99999
endif
" Rolodex vim: make the current window as high as possible,
" reduce all other windows to just a status line
set wmh=0 wh=99999
" add additional customizations here
Best regards,
Tony.
--
When a fly lands on the ceiling, does it do a half roll or a half
loop?
Ciccio Bodoni
thanks for your replies. I've already created a _gvimrc file. Let's
talk about some encoding related options: what are really necessary
for my purposes (see my first post) and what are the values to chose?
set encoding=utf-8
set fileencoding=?
set fileencodings=?
setglobal fileencoding=utf-8
set fileformat=?
set fileformats=?
let &termencoding = &encoding
As you can see I'm a little confused among all these options :)
Ben Schmidt
> Hi,
> thanks for your replies. I've already created a _gvimrc file. Let's
> talk about some encoding related options: what are really necessary
> for my purposes (see my first post) and what are the values to chose?
You probably should have a _vimrc file not a _gvimrc file. It may not matter, of
course, as you may never use Vim in a terminal/console, so the GUI is always
running, but this stuff does belong more in a _vimrc than a _gvimrc. Note that
_vimrc is always used, including by gvim. _gvimrc is used in addition to _vimrc
when the GUI starts up.
> set encoding=utf-8
This sets Vim's internal encoding, and will be necessary if you want to use UTF-8
at all, as it is the only internal encoding that can handle all the Unicode
characters.
> set fileencoding=?
You can use this to change the encoding of individual files after they are open by
issuing it in vim (after typing a colon ':'). Heaps of encodings can be used here.
> set fileencodings=?
This sets up how Vim's encoding detection works when reading files. Its default
should be fine: ucs-bom,utf-8,default,latin1
> setglobal fileencoding=utf-8
This will set the default encoding for new files. (Actually, 'set' above will do
this too; setlocal can be used to change a single file's encoding without setting
the default for new files).
> set fileformat=?
This can be used to change a file's default line endings once the file is open
(again within vim after a colon). Values can be dos, unix, mac for CRLF, LF and CR
line endings respectively.
> set fileformats=?
This sets up Vim's automatic line ending detection, and the first item in the list
will set the default line ending. It is a comma separated list, and to have unix
(LF) line endings by default, you want fileformats=unix,dos rather than the
default dos,unix.
> let &termencoding = &encoding
You will need to deal with termencoding if you're using a terminal that isn't
UTF-8 (e.g. the Windows command window, which is a mess...). It is irrelevant if
the Vim GUI (gvim) is running.
To display the current encoding of a file, just do
:set fenc
To display the current line endings of a file, do
:set ff
Changing those options and then saving the file will convert it. E.g. if you find
you have a latin1, CRLF endings file and you need to fix it, open it and then issue
:set fenc=utf-8
:set ff=unix
:w
Hope this helps,
Cheers,
Ciccio Bodoni
I got things done with just three line in my _vimrc file:
set encoding=utf-8
--> Every new file I create in vim will be utf-8. Moreover vim is now
able to read existing utf-8 files without displaying strange
characters.
set fileformat=unix
--> Default line ending for new files created in vim is now LF (unix).
set fileformats=unix,dos
--> Please note that the default value for fileformats (ffs) is
"dos,unix". In fact in both cases vim is able to read also files with
dos line endings. But setting fileformats to "unix,dos" will avoid
that existing empty files are opened as dos.
With these simple lines my Windows machine works smoothly with my co-
workers' Macs+Textmate (see Textmate default saving settings:
http://macromates.com/textmate/manual/saving_files.html)
Please tell me if my choices are right.
CB
Tony Mechelynck
> Hi,
> thanks for your replies. I've already created a _gvimrc file. Let's
> talk about some encoding related options: what are really necessary
> for my purposes (see my first post) and what are the values to chose?
>
> set encoding=utf-8
This defines Vim's internal representation of the data. With anything else,
you run the risk of not being able to represent internally some of the data
contained in UTF-8 files.
>
> set fileencoding=?
When set for an existing files, or via :setlocal, this defines how the current
file will be represented on disk. When set "in general", or via :setglobal, it
defines which encoding to use in the future for newly created files
> set fileencodings=?
this is a comma-separated list of possible encodings to be tried when opening
an existing file. "ucs-bom" means "if the file starts with a BOM, use the
corresponding Unicode encoding". The components are tried in left-to-right
order, and the first one which doesn't give an error is used.
> setglobal fileencoding=utf-8
Future newly created files will use UTF-8 encoding
>
> set fileformat=?
When set for an existing file, or via :setlocal, this defines the "style" of
end-of-line character to be used for the current file. Thee valuyes are
possible: unix (LF only), mac (CR only) or dos (CR+LF).
> set fileformats=?
This tells Vim which end-of-line styles to check for when opening an existing
file. It is a comma-separated list but with no specific order. If empty, Vim
won't try to determine which EOL character was used, it's the user's job to
tell it by using ++ff=<something> when opening the file, or by setting
'fileformat' correctly. So don't set this option to the empty string except
temporarily, and only if you know exactly what you're doing. Many Vim users
have been burnt before you.
>
> let &termencoding = &encoding
'termencoding' defines how the keyboard (and, in console Vim, the screen)
represents the data. The default is the empty string, which means "use the
value of 'encoding'". This is OK as long as you don't change the 'encoding'
value; but if you set 'encoding' to UTF-8, your keyboard won't know that, and
will go on sending (for example) Latin1 bytes, which Vim will treat as invalid
UTF-8 data if, like the accented letters à è ì ò ù of Italian, they are over
0x7F. So before we change 'encoding' we must save the "old" value here so Vim
will still "understand" what the keyboard is sending.
>
> As you can see I'm a little confused among all these options :)
Well, in my previous post I gave you pointers to the Vim help and to a Vim
wiki tip. WHY THE HELL DIDN'T YOU READ THEM AND TRY TO UNDERSTAND THEM???
Best regards,
Tony.
--
It is practically impossible to teach good programming style to
students that have had prior exposure to BASIC: as potential
programmers they are mentally mutilated beyond hope of
regeneration.
-- Dijkstra
Tony Mechelynck
> Ciccio Bodoni wrote:
[...]
>> let &termencoding = &encoding
>
> You will need to deal with termencoding if you're using a terminal that isn't
> UTF-8 (e.g. the Windows command window, which is a mess...). It is irrelevant if
> the Vim GUI (gvim) is running.
Not really: even in gvim, it is relevant for the keyboard. (In console Vim it
is relevant for both the keyboard and the display.)
>
> To display the current encoding of a file, just do
>
> :set fenc
>
> To display the current line endings of a file, do
>
> :set ff
Since these are string options, this will work. However, with boolean options,
":set diff" for instance will *set* the option rather than *display* it. To
avoid this pitfall, I recommend to train oneself to always use a question mark
to display the value, as in ":set fenc?" ":set ff?" ":set diff?" etc., as this
will always display the value regardless of the option type.
[...]
> Hope this helps,
Yeah, me too, and next time, Ciccio, LEGGI L'AIUTA!
>
> Cheers,
>
> Ben.
Best regards,
Tony.
--
In Greene, New York, it is illegal to eat peanuts and walk backwards on
the sidewalks when a concert is on.
Tony Mechelynck
You will be able to find out yourself. If you can't type "Che sarà, sarà",
"Dove non c'è pericolo non c'è gloria", "Faremo di necessità virtù" etc. with
the correct accents, then you'll have to add a line before "set enc=utf-8" to
avoid clobbering the 'termencoding' setting.
Best regards,
Tony.
--
GOD: That is your purpose Arthur ... the Quest for the Holy Grail ...
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
Ben Schmidt
> is relevant for both the keyboard and the display.)
Thanks, Tony. Correction appreciated.
> Since these are string options, this will work. However, with boolean options,
> ":set diff" for instance will *set* the option rather than *display* it. To
> avoid this pitfall, I recommend to train oneself to always use a question mark
> to display the value, as in ":set fenc?" ":set ff?" ":set diff?" etc., as this
> will always display the value regardless of the option type.
Mmm. Good call/good advice.
Ben Schmidt
They look to me like they will suffice. If you find you have trouble typing
certain accented characters with your Italian keyboard, you will need to add the
let &termencoding = &encoding
line to the *beginning* of your _vimrc as Tony suggested.
I hope you enjoy using Vim. You will find most of this becomes much easier as you
become more familiar with the program.
Ben Schmidt
>> set ffs=unix,dos
>
>
> Ciao
>
> I think you meant "set fileencodings ..." rather than "set encoding ..."
> ("set encoding" being is for the internal representation in vim which
> should typically not be changed).
Nope. I most certainly meant 'encoding', and the reasoning is explained in the
followup posts by both myself and Tony. If 'encoding' is anything but utf-8, Vim
may not be able to represent all it needs to internally. I didn't mention
fileencodings in my first post as its default setting is fine for what the OP
required.