Bug? Vim listchars error (E474) with unicode chars

[vex...@gmail.com]

On win7 x64 using VIM 7.4, I get an error with gVim upon reading the _vimrc, when Unicode chars are in the listchars section. A simple example to trigger would read:

set listchars=eol:¶

I get an "Error detected...E474 Invalid argument: set listchars=eol:¶" popup error dialog.

(In case it doesn't show, the character after the ':' is the pilcrow (U+00B6) aka the paragraph mark, but any non-Ascii Unicode character yields the error)

I thought this would be ok as help docs say "UTF-8 characters can be used when 'encoding' is "utf-8", otherwise only printable characters are allowed."

_vimrc 'encoding' and 'fileencoding' options are both UTF-8. Running ':set fileencoding' and ':set encoding' with the _vimrc open both yield '[...]=utf-8'. There is no BOM on the file, though a different Windows encoding checker utility also reads the file as UTF-8.

As a result of the error, listchars is set to the default (eol:$). However, if I open and re-source the _vimrc (:so %), the file reads without error, and listchars is set to the appropriate settings, Unicode chars and all, and ':set list' works correctly.

Can anyone replicate this, or is this some conflict with something else in my _vimrc? Or is this expected and I'm misunderstanding the functionality of listchars?

(As an aside, I'd also like to voice my approval for inclusion of the lcs space patch in the next version of Vim; it's easy for eyes to elide over empty space, so making them more visible is useful)

[Nikolay Pavlov]

On Apr 1, 2014 10:35 AM, <vex...@gmail.com> wrote:
>
> On win7 x64 using VIM 7.4, I get an error with gVim upon reading the _vimrc, when Unicode chars are in the listchars section.  A simple example to trigger would read:
>
> set listchars=eol:¶
>
> I get an "Error detected...E474 Invalid argument: set listchars=eol:¶" popup error dialog.
>
> (In case it doesn't show, the character after the ':' is the pilcrow (U+00B6) aka the paragraph mark, but any non-Ascii Unicode character yields the error)
>
> I thought this would be ok as help docs say "UTF-8 characters can be used when 'encoding' is "utf-8", otherwise only printable characters are allowed."
>
> _vimrc 'encoding' and 'fileencoding' options are both UTF-8.  Running ':set fileencoding' and ':set encoding' with the _vimrc open both yield '[...]=utf-8'.  There is no BOM on the file, though a different Windows encoding checker utility also reads the file as UTF-8.

UTF-8 is not the default &encoding. Make sure that you have

    scriptencoding utf-8
    set encoding=utf-8

before any other command in your vimrc. Remove &encoding setting that is located somewhere else in your vimrc (it is located because utf-8 is not the default).

Generally you just need to make sure you set &encoding before &listchars. But it is better to use stricter rules specified above: make encoding setting always the very first line in the vimrc, make the first line of each script

    scriptencoding utf-8

>
> As a result of the error, listchars is set to the default (eol:$).  However, if I open and re-source the _vimrc (:so %), the file reads without error, and listchars is set to the appropriate settings, Unicode chars and all, and ':set list' works correctly.
>
> Can anyone replicate this, or is this some conflict with something else in my _vimrc?  Or is this expected and I'm misunderstanding the functionality of listchars?
>
> (As an aside, I'd also like to voice my approval for inclusion of the lcs space patch in the next version of Vim; it's easy for eyes to elide over empty space, so making them more visible is useful)
>

> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Ben Fritz]

On Tuesday, April 1, 2014 1:10:37 AM UTC-5, vex...@gmail.com wrote:
> On win7 x64 using VIM 7.4, I get an error with gVim upon reading the _vimrc, when Unicode chars are in the listchars section. A simple example to trigger would read:
>
>
>
> set listchars=eol:¶
>
>
>
> I get an "Error detected...E474 Invalid argument: set listchars=eol:¶" popup error dialog.
>

An alternative to ZyX's suggestion of using the "scriptencoding" command to tell Vim what encoding to read the vimrc in, you could also use the codepoint value in a string to set the listchars:

exec "set listchars=eol:\u00B6"

[Ben Fritz]

Or if you prefer:

let &listchars="eol:\u00B6"

[vex...@gmail.com]

>UTF-8 is not the default &encoding. Make sure that you have
>
> scriptencoding utf-8
> set encoding=utf-8
>
>before any other command in your vimrc.

I tried the above, and now I don't get the error, but I don't get the right result for my actual listchars, which is as follows:

set listchars=eol:¶,tab:\|,trail:•,extends:»,precedes:«,nbsp:×

When I try :set list, the 'trail' shows up as an inverted question mark (should be Unicode bullet, U+2022) and the :set listchars? output lists the values in some format I'm not familiar with (enclosed in '<>')

> > An alternative to ZyX's suggestion of using the "scriptencoding" command to tell Vim what encoding to read the vimrc in, you could also use the codepoint value in a string to set the listchars:
>

> let &listchars="eol:\u00B6"

This works, but only for the first setting. I tried a few permutations using the '.' concatenation operator, but couldn't get it right. What's the correct syntax?

[Ben Fritz]

On Tuesday, April 1, 2014 6:59:52 PM UTC-5, vex...@gmail.com wrote:
> > > An alternative to ZyX's suggestion of using the "scriptencoding" command to tell Vim what encoding to read the vimrc in, you could also use the codepoint value in a string to set the listchars:
> >
> > let &listchars="eol:\u00B6"
>
> This works, but only for the first setting. I tried a few permutations using the '.' concatenation operator, but couldn't get it right. What's the correct syntax?

You can do the same thing to the rest of the special characters in the string.

You said you wanted this string: eol:¶,tab:\|,trail:•,extends:»,precedes:«,nbsp:×

So you could use:

let &listchars="eol:\u00b6,tab:|,trail:\u2022,extends:\u00bb,precedes:\u00ab,nbsp:\u00d7"

There I replaced each special character with the corresponding codepoint and your .vimrc can actually be encoded in Latin1 or some similar default encoding if needed.

Side note, to generate this I took your desired string, added a trailing comma to simplify the regex, and did this:

:s#:\(.\),#\=':\u'.printf('%04x', char2nr(submatch(1))).','#g

[Ken Takata]

Hi,

2014/4/1 Tue 17:43:39 UTC+9 ZyX wrote:

(snip)

> UTF-8 is not the default &encoding. Make sure that you have
>
>     scriptencoding utf-8
>     set encoding=utf-8

This should be:

set encoding=utf-8
scriptencoding utf-8

because :scriptencoding converts the script to the value of 'encoding'.
(see :help :scriptencoding)
'encoding' must be set properly before :scriptencoding.

Regards,
Ken Takata

[vex...@gmail.com]

On Wednesday, April 2, 2014 9:17:32 AM UTC-7, Ben Fritz wrote:
> You said you wanted this string: eol:¶,tab:\|,trail:•,extends:»,precedes:«,nbsp:×
> So you could use:
> let &listchars="eol:\u00b6,tab:|,trail:\u2022,extends:\u00bb,precedes:\u00ab,nbsp:\u00d7"
>

On Wednesday, April 2, 2014 3:50:15 PM UTC-7, Ken Takata wrote:
>
> This should be:
> set encoding=utf-8
> scriptencoding utf-8

Would someone be willing to try either/both of these suggestions out? I tried them alone & together with a clean _vimrc file on both of the computers (both Windows) I have access to, but I listchars still isn't correct. Thanks for the help to date, and the epic regex substitution.

[Christian Brabandt]

Start from vim -u NONE -N
and try again. If possible please show a screenshot.

Best,
Christian
--
Letzte Worte eines Studenten:
"Ich gehe in die Mensa, kommt ihr mit?"

[vex...@gmail.com]

>
> Start from vim -u NONE -N
>
> and try again. If possible please show a screenshot.
>

I've attached an edited collage of screenshots (and here's the link also) http://i245.photobucket.com/albums/gg79/traycerb/vim_listchars_error_zpsc42a1081.png

In case it's not clear, top to bottom screenshot shows starting Vim from cmdline w/ options, the command-line window (q:), and the output of the 'set listchars?' command. Only the first option in listchars is set. I'm noob to all this, so perhaps I'm missing some obvious step?

[Ken Takata]

Hi,

You must set two characters for tab. See ":help lcs-tab".
Please try this:

:let &listchars="tab:| ,eol:\u00b6"

or:

:set listchars=tab:\|\ ,eol:¶


Regards,
Ken Takata

[vex...@gmail.com]

On Friday, April 4, 2014 3:55:45 PM UTC-7, Ken Takata wrote:
>
>
> You must set two characters for tab. See ":help lcs-tab".
>
> Please try this:
> :let &listchars="tab:| ,eol:\u00b6"
> or:
> :set listchars=tab:\|\ ,eol:¶

<face palm> yes this is it. I had this correct when I first tried this, and at some point omitted one of the two characters needed for tab. Now my original code works too: set listchars=eol:¶,tab:\|\ ,trail:•,extends:»,precedes:«,nbsp:× as does a corrected form of Ben Fritz's solution: let &listchars="eol:\u00b6,tab:\|\ ,trail:\u2022,extends:\u00bb,precedes:\u00ab,nbsp:\u00d7"

I think putting the encoding lines at the start sufficed. Sorry to waste people's time...

[Ben Fritz]

On Friday, April 4, 2014 6:28:25 PM UTC-5, vex...@gmail.com wrote:
> > You must set two characters for tab. See ":help lcs-tab".
>
>

> <face palm> yes this is it. I had this correct when I first tried this, and at some point omitted one of the two characters needed for tab. Now my original code works too: set listchars=eol:¶,tab:\|\ ,trail:•,extends:»,precedes:«,nbsp:× as does a corrected form of Ben Fritz's solution: let &listchars="eol:\u00b6,tab:\|\ ,trail:\u2022,extends:\u00bb,precedes:\u00ab,nbsp:\u00d7"
>

Oops, I tested with "tab:| " and somehow omitted the space when pasting. Within the :let &listchars="..." string you don't need to escape the | or the space, since it is within a string.