BOM in QuickFix window
Aleksey
I'm using few MONO apps (Nemerle compiler and COCO/R parser generator)
through compiler plugins in VIM under Linux, and the problem is that
they produce output starting with FEFF.
I had this problem about a half a year ago and being (and staying) a
newbie, thought that the problem was with *errorformat* but didn't
cope defining it correctly - not sure if it is possible.
I found workaround, setting LANG=en_US in compiler call command.
I have encoding and termencoding set to 'utf-8' and locale is
'en_US.utf8'.
Is there a better way to solve this problem?
Thanks
Tony Mechelynck
Make sure your 'fileencodings' option (plural) starts with "ucs-bom". In
that case, for editfiles at least (not sure about quickfix error files),
Vim will recognise the Unicode codepoint U+FEFF (known as the BOM for
Byte Order Mark though actually it's more than that) when it happens at
the very start of a file, and set 'fileencoding' (singular) and 'bomb'
accordingly for that file, as follows:
Lead bytes (hex) 'fileencoding' 'bomb'
EF BB BF utf-8 bomb
00 00 FE FF ucs-4 bomb
FF FE 00 00 ucs-4le bomb
FE FF utf-16 bomb
FF FE utf-16le bomb
anything else not yet known nobomb
As you can see, the BOM allows us to identify all Unicode encodings
(endianness included, but treating UCS-2 as a particular case of UTF-16,
which it is) assuming that no little-endian UTF-16 file will ever start
with a NULL codepoint, which I think is a reasonable assumption. ("Not
yet known" in the table above means "try the next entry in
'fileencodings'".)
If, after trying it, you find that it doesn't work for the quickfix
errorfile, come back to report it, and in that case maybe Bram will pass
by and add it to his TODO list. But even if he does, don't set your
hopes too high: it's a very long list, see |todo.txt| in the Vim help
(the file is currently 4705 lines long as of 2 July 2009, but I didn't
count how many todo items (which should be fewer ;-) ) are in it.
IIUC, even if 'fileencodings' doesn't work for quickfix error files,
setting LC_MESSAGES should be enough (LANG is used as a default for any
LC_<something> which is unset, or LC_ALL if present overrides any other
LC_* even if present). Maybe even just using ":language messages en-US"
or ":lang mess C" near the top of your vimrc could be enough.
Best regards,
Tony.
--
hundred-and-one symptoms of being an internet addict:
66. You create a homepage with the impression to cure the afflicted...but
your hidden agenda is to receive more e-mail.
Aleksey
> setting LC_MESSAGES should be enough (LANG is used as a default for any
> LC_<something> which is unset, or LC_ALL if present overrides any other
> LC_* even if present). Maybe even just using ":language messages en-US"
> or ":lang mess C" near the top of your vimrc could be enough.
>
> Best regards,
> Tony.
> --
> hundred-and-one symptoms of being an internet addict:
> 66. You create a homepage with the impression to cure the afflicted...but
> your hidden agenda is to receive more e-mail.
fileencoding already had 'ucs-bom' in its start, so it didnt' help.
I also have :language messages C, so it didn't help either.
I didn't quite understand what you wrote about LC_ stuff, because I
lack even basic knowledge, but does it mean my solution with setting
LANG=en_US is fine?
Reading docs for VIM 7.2 (:help quickfix) I found the following
paragraph:
When 'encoding' differs from the locale, the error messages may
have a
different encoding from what Vim is using. To convert the
messages you can
use this code:
function QfMakeConv()
let qflist = getqflist()
for i in qflist
let i.text = iconv(i.text, "cp936", "utf-8")
endfor
call setqflist(qflist)
endfunction
au QuickfixCmdPost make call QfMakeConv()
I've tried putting this to .vimrc replacing "cp936" with ucs-bom, but
not sure if that was right. It didn't change anything.
I don't think this problem is worth of Bram's attention since it's not
really wide-spread and the workaround is available.
Tony Mechelynck
>
>> Make sure your 'fileencodings' option (plural) starts with "ucs-bom".
>
>> IIUC, even if 'fileencodings' doesn't work for quickfix error files,
>> setting LC_MESSAGES should be enough (LANG is used as a default for any
>> LC_<something> which is unset, or LC_ALL if present overrides any other
>> LC_* even if present). Maybe even just using ":language messages en-US"
>> or ":lang mess C" near the top of your vimrc could be enough.
>>
>> Best regards,
>> Tony.
>> --
>> hundred-and-one symptoms of being an internet addict:
>> 66. You create a homepage with the impression to cure the afflicted...but
>> your hidden agenda is to receive more e-mail.
>
> Thanks for your answer.
>
> fileencoding already had 'ucs-bom' in its start, so it didnt' help.
should be 'fileencodings' with s at the end. There are two different
options, with and without s, and they don't have the same meaning.
Without s, it's the encoding used on disk for one file at a time. With
s, it's the heuristics to find what to use when opening an existing file.
> I also have :language messages C, so it didn't help either.
> I didn't quite understand what you wrote about LC_ stuff, because I
> lack even basic knowledge, but does it mean my solution with setting
> LANG=en_US is fine?
yeah, sure.
>
> Reading docs for VIM 7.2 (:help quickfix) I found the following
> paragraph:
> When 'encoding' differs from the locale, the error messages may
> have a
> different encoding from what Vim is using. To convert the
> messages you can
> use this code:
> function QfMakeConv()
> let qflist = getqflist()
> for i in qflist
> let i.text = iconv(i.text, "cp936", "utf-8")
> endfor
> call setqflist(qflist)
> endfunction
>
> au QuickfixCmdPost make call QfMakeConv()
>
> I've tried putting this to .vimrc replacing "cp936" with ucs-bom, but
> not sure if that was right. It didn't change anything.
no, iconv doesn't know about ucs-bom.
>
> I don't think this problem is worth of Bram's attention since it's not
> really wide-spread and the workaround is available.
How can one know how widespread it is? Sounds like the age-old "Nobody
uses it" used without proof by developers of some applications (not by
Bram) when speaking of a feature they want to remove because they don't
use it themselves and they don't want to support it.
OK, so IIUC the problem seems to be: When a compiler errorfile starts
with a BOM, Vim doesn't know how to handle it. I'm not sure if there's
an 'errorfile' setting allowing to get over it because I've never looked
into that option myself.
Best regards,
Tony.
--
"Benson, you are so free of the ravages of intelligence"
-- Time Bandits
Aleksey
wrote:
> should be 'fileencodings' with s at the end. There are two different
> options, with and without s, and they don't have the same meaning.
> Without s, it's the encoding used on disk for one file at a time. With
> s, it's the heuristics to find what to use when opening an existing file.
the list.
> > I don't think this problem is worth of Bram's attention since it's not
> > really wide-spread and the workaround is available.
>
> How can one know how widespread it is? Sounds like the age-old "Nobody
> uses it" used without proof by developers of some applications (not by
> Bram) when speaking of a feature they want to remove because they don't
> use it themselves and they don't want to support it.
for similar
problem and found almost nothing except your recent answers to 'Match
a BOM'
in a file. Which deals with different issue, but it made me think
about mine.
I wanted to say, that since this issue has a solution and isn't
reported by other developers,
people know how to live with it or don't care about it. It's just
about not wasting time on low priority stuff, IMHO.
I'm just glad to understand what the problem is.
>
> OK, so IIUC the problem seems to be: When a compiler errorfile starts
> with a BOM, Vim doesn't know how to handle it. I'm not sure if there's
> an 'errorfile' setting allowing to get over it because I've never looked
> into that option myself.
>
on LANG=en_US which is just ok for me.