Can't save file due to "CONVERSION ERROR in line..."

[Yosi Izaq]

Hi,

This happens to me when I copy paste text from my browser. It's
driving me crazy :)
How can I turn this off?

TIA,
Yosi

[Charles Campbell]

I can't precisely answer this, but I'll give you some guesses: I'll
guess that the browser is using one encoding and you have gvim/vim using
a different, incompatible encoding. I'm not sure how to determine what
encoding your browser is using, but the encoding for vim can be checked with

set enc
set fenc

If you can live with utf-8, try using that.

Regards,
Chip Campbell

[Tony Mechelynck]

Yes indeed.

Conversion error when saving the file means that the file in the buffer
(in the 'encoding' charset) contains one or more characters which cannot
be represented in the 'fileencoding' charset which you're trying to save
the file in. For instance, the file contains an � character (French oe
digraph) and you're trying to save it in Latin1.

If the conversion error happens when reading or pasting into Vim memory,
then your 'encoding' is not compatible with the data; IOW, Vim cannot
represent the data in memory with the present choice of 'encoding'.

I recommend to use UTF-8 for the data in Vim memory ('encoding'),
because Unicode (including UTF-8) can represent any possible computer
charset. Conversion between Unicode and Latin1 is trivial and Vim (if
compiled with +multi_byte of course, i.e. if has('multi_byte') is
nonzero) can do it without external help; conversion to or from some
other non-Unicode charset (or to or from GB18030, which technically is a
Unicode charset but requires large conversion tables) usually requires
the iconv library, either compiled-in or available for dynamic load at
runtime (to check it, make sure that has('iconv') has a nonzero value).

See also http://vim.wikia.com/wiki/Working_with_Unicode and the help
pages mentioned there.

See also :help 8g8

Best regards,
Tony.
--
Ten million Linux users can't be wrong!

[yosi izaq]

>> I can't precisely answer this, but I'll give you some guesses: I'll
>> guess that the browser is using one encoding and you have gvim/vim using
>> a different, incompatible encoding. I'm not sure how to determine what
>> encoding your browser is using, but the encoding for vim can be checked
>> with
>>
>> set enc
>> set fenc
>>
>> If you can live with utf-8, try using that.
>>
>> Regards,
>> Chip Campbell
>>
>
> Yes indeed.
>
> Conversion error when saving the file means that the file in the buffer (in
> the 'encoding' charset) contains one or more characters which cannot be
> represented in the 'fileencoding' charset which you're trying to save the

> file in. For instance, the file contains an œ character (French oe digraph)

> and you're trying to save it in Latin1.
>
> If the conversion error happens when reading or pasting into Vim memory,
> then your 'encoding' is not compatible with the data; IOW, Vim cannot
> represent the data in memory with the present choice of 'encoding'.
>
> I recommend to use UTF-8 for the data in Vim memory ('encoding'), because
> Unicode (including UTF-8) can represent any possible computer charset.
> Conversion between Unicode and Latin1 is trivial and Vim (if compiled with
> +multi_byte of course, i.e. if has('multi_byte') is nonzero) can do it
> without external help; conversion to or from some other non-Unicode charset
> (or to or from GB18030, which technically is a Unicode charset but requires
> large conversion tables) usually requires the iconv library, either
> compiled-in or available for dynamic load at runtime (to check it, make sure
> that has('iconv') has a nonzero value).
>
> See also http://vim.wikia.com/wiki/Working_with_Unicode and the help pages
> mentioned there.
>
> See also :help 8g8
>
>
> Best regards,
> Tony.

Hi,

Thanks for the tips. Following two of them didn't really solve the issue.
set enc returns
encoding=utf-8

and set fenc returns
file encoding=latin1

How can I know which of them actually applies?- the utf-8 or the
latin1?- And how can I make sure utf-8, assuming this will resolve the
error, applies?

the 8g8 is nice but it's not robust enough to help me cope with 2k
line choke full of encoding wise illegal chars.
The tip though had an example of encoding settings. I added it to my
vimrc and it resolved the problem. For the record, end and fenc
remains the same after I apply the tip. The problem is gone though and
I can copy paste the problematic text to vim and save w/o error.

Thanks!
Yosi

[yosi izaq]

> The tip though had an example of encoding settings. I added it to my
> vimrc and it resolved the problem. For the record, end and fenc
> remains the same after I apply the tip. The problem is gone though and
> I can copy paste the problematic text to vim and save w/o error.
>

Hi,

I just found out that an old buffer I had could not be converted
correctly *after* I applied the tip's settings (and naturally had no
issue prior to applying the settings).
Manually setting fenc to utf-8 resolved that one as well.

10x,
Yosi

[Tony Mechelynck]

On 28/03/10 10:10, yosi izaq wrote:
>>> I can't precisely answer this, but I'll give you some guesses: I'll
>>> guess that the browser is using one encoding and you have gvim/vim using
>>> a different, incompatible encoding. I'm not sure how to determine what
>>> encoding your browser is using, but the encoding for vim can be checked
>>> with
>>>
>>> set enc
>>> set fenc
>>>
>>> If you can live with utf-8, try using that.
>>>
>>> Regards,
>>> Chip Campbell
>>>
>>
>> Yes indeed.
>>
>> Conversion error when saving the file means that the file in the buffer (in
>> the 'encoding' charset) contains one or more characters which cannot be
>> represented in the 'fileencoding' charset which you're trying to save the

>> file in. For instance, the file contains an � character (French oe digraph)

They both apply: 'encoding' (a global option) applies to all data in Vim
memory, 'fileencoding' (which is buffer-local) applies to the current
file's data on disk. If the file's buffer in memory (as modified by your
edit) contains data above 0xFF, it cannot be saved as Latin1.

>
> the 8g8 is nice but it's not robust enough to help me cope with 2k
> line choke full of encoding wise illegal chars.
> The tip though had an example of encoding settings. I added it to my
> vimrc and it resolved the problem. For the record, end and fenc
> remains the same after I apply the tip. The problem is gone though and
> I can copy paste the problematic text to vim and save w/o error.
>
> Thanks!
> Yosi
>

There are other encoding options: 'termencoding' applies to what Vim
thinks that your keyboard is sending (and, in console mode, to what it
thinks your display is accepting); 'fileencodings' (plural) tells Vim
how to "guess" a disk file's charset when reading it into memory. There
is also 'printencoding' but I'm not sure how to make it work -- when I
want to print files containing "any" Unicode codepoints I save them to a
text or HTML file, then print that after loading it in my browser.

Best regards,
Tony.
--
From the "Guiness Book of World Records", 1973:

Certain passages in several laws have always defied interpretation and
the most inexplicable must be a matter of opinion. A judge of the
Court of Session of Scotland has sent the editors of this book his
candidate which reads, "In the Nuts (unground), (other than ground
nuts) Order, the expression nuts shall have reference to such nuts,
other than ground nuts, as would but for this amending Order not
qualify as nuts (unground)(other than ground nuts) by reason of their
being nuts (unground)."