"matchparen" plugin aborting with strange error.

[Sihera Andre]

Hello all,

Apologies for posting to the mailing list before filing a bug report
but I just wanted to make sure that I was facing a bug and not a silly
build error of some kind.

The problem is that while editing scripts (e.g., .bashrc), or anything
where the file type is set to "sh", when I cursor onto or off any kind
of character that comes in pairs (bracket, parenthesis, braces, etc.)
the following error occurs:

  Error detected while processing function <SNR>13_Highlight_Matching_Pair:
  line   97:
  E475: Invalid argument: 0

I have traced this to:

  /usr/local/share/vim/plugin/matchparen.vim

and this code:

  try
    let [m_lnum, m_col] = searchpairpos(c, '', c2, s_flags, s_skip,
stopline, timeout)
  catch /E118/

Why would this be erroring? Does it look like a build error, a bug in
VIM, or a bug in the run-time? Any comments would be welcome because
this problem makes editing scripts impossible (currently, I've knocked
the offending module out by renaming it so it doesn't get loaded).

Background behind this situation and other System information below.

Thanks in advance,

Andre.


[ SYSTEM INFO ]

I have just performed upgrades of my Ubuntu systems from 15.04 to
18.10. This has meant upgrading VIM from the latest build at the time
7.4.XXX to 8.1.2186 (sorry, previous version deleted so can't provide
the precise version number). The vim runtime appears to be 8.0.1766.

SDK packages installed were:

  libncurses5-dev libgnome2-dev libgnomeui-dev libgtk2.0-dev
  libatk1.0-dev libbonoboui2-dev libcairo2-dev libx11-dev
  libxpm-dev libxt-dev

8.1.2186 was built locally with "configure --with-features=huge".

[Tony Mechelynck]

On Thu, Oct 24, 2019 at 3:21 AM Sihera Andre <andre....@hotmail.co.jp> wrote:
>
> Hello all,
>
> Apologies for posting to the mailing list before filing a bug report
> but I just wanted to make sure that I was facing a bug and not a silly
> build error of some kind.

Bug reports come to the mailing list anyway, so no problem.

Runtime files ought to be of a version as close as possible to the
executable with which they are used. Using 8.0 runtime files with a
8.1 executabe might work — or it might not.

You might want to compile your own Vim, it should not interfere with
your distro's (because the executable normally gets put in
/usr/opt/bin/ which is early in the $PATH). See:
- http://vim.fandom.com/wiki/Getting_the_Vim_source_with_Mercurial
- On Debian-like distributions of Linux, "apt-get build-dep vim-gui"
(or something similar, I never used that family of distros, I only
know this command from hearsay) should install the necessary
dependency packages to build Vim.
- http://users.skynet.be/antoine.mechelynck/vim/compunix.htm

>
> SDK packages installed were:
>
> libncurses5-dev libgnome2-dev libgnomeui-dev libgtk2.0-dev
> libatk1.0-dev libbonoboui2-dev libcairo2-dev libx11-dev
> libxpm-dev libxt-dev
>

> 8.1.2186 was built locally with "configure --with-features=huge".r
>
Hm, and what is the output of the ":version" command on that
executable? (See ":help :redir" about how to capture it.)

Best regards,
Tony.

[Christian Brabandt]

Hi,

On Do, 24 Okt 2019, Sihera Andre wrote:

>
>   Error detected while processing function <SNR>13_Highlight_Matching_Pair:
>   line   97:
>   E475: Invalid argument: 0
>
> I have traced this to:
>
>   /usr/local/share/vim/plugin/matchparen.vim
>
> and this code:
>
>   try
>     let [m_lnum, m_col] = searchpairpos(c, '', c2, s_flags, s_skip,
> stopline, timeout)
>   catch /E118/
>

[...]

> I have just performed upgrades of my Ubuntu systems from 15.04 to
> 18.10. This has meant upgrading VIM from the latest build at the time
> 7.4.XXX to 8.1.2186 (sorry, previous version deleted so can't provide
> the precise version number). The vim runtime appears to be 8.0.1766.
>
> SDK packages installed were:
>
>   libncurses5-dev libgnome2-dev libgnomeui-dev libgtk2.0-dev
>   libatk1.0-dev libbonoboui2-dev libcairo2-dev libx11-dev
>   libxpm-dev libxt-dev
>
> 8.1.2186 was built locally with "configure --with-features=huge".
>

With patch 8.1.0112 Vim will throw an error, if the `skip` argument for
the searchpair() function is not a string argument while previously Vim
did ignore this error. This means the matchparen plugin needs to be
updated to fix this bug.

The problem could be either a local older version of the matchparen
plugin below your ~/.vim directory or the system runtime files have not
been updated yet. It looks like you build Vim yourself but are using the
system provided runtime files. Try to install the runtime files that
come with the source repository and un-install the packaged one from
Ubuntu and it should work again.

See also here for more information:
https://github.com/vim/vim/issues/3225
https://vi.stackexchange.com/questions/18701

Best,
Christian
--
Bildung ist das was übrig bleibt, wenn man alles andere vergessen hat.

[Sihera Andre]

Tony,
Christian,
Sorry for the delayed response.

On 24/10/2019 11:07, Tony Mechelynck wrote:
>
> Runtime files ought to be of a version as close as possible to the
> executable with which they are used. Using 8.0 runtime files with a
> 8.1 executabe might work — or it might not.
>
>

Up until now I have always built the HEAD version and it has
worked fine against previous run-times. As a sanity check, I
checked out version 8.0.1766 of the source code (i.e., the
version that matches the run-time in the repository as
opposed to HEAD) and built it locally. Lo and behold, the
error disappeared.

Thanks for the suggestion.

On 24/10/2019 15:19, Christian Brabandt wrote:
> With patch 8.1.0112 Vim will throw an error, if the `skip` argument for
> the searchpair() function is not a string argument while previously Vim
> did ignore this error. This means the matchparen plugin needs to be
> updated to fix this bug.
>

> <snip>
>
> Best,
> Christian
Thanks for this information.

So the "matchparen" plugin on my end (run-time v8.0.1766) is
old and broke with the code fix in patch 8.1.0112. Understood.

Is there a HEAD version of the run-time I can download to match
the HEAD version of the source tree? Or will I have to wait until
the repositories are next updated?

Cheers,

Andre.

[Sihera Andre]

On 25/10/2019 11:05, Andre Sihera wrote:
> Tony,
> Christian,
> Sorry for the delayed response.
>
> On 24/10/2019 11:07, Tony Mechelynck wrote:
>>

>> Runtime files ought to be of a version as close as possible to the
>> executable with which they are used. Using 8.0 runtime files with a
>> 8.1 executabe might work — or it might not.
>>
>>

> Up until now I have always built the HEAD version and it has
> worked fine against previous run-times. As a sanity check, I
> checked out version 8.0.1766 of the source code (i.e., the
> version that matches the run-time in the repository as
> opposed to HEAD) and built it locally. Lo and behold, the
> error disappeared.
>
> Thanks for the suggestion.
>
>
> On 24/10/2019 15:19, Christian Brabandt wrote:
>> With patch 8.1.0112 Vim will throw an error, if the `skip` argument for
>> the searchpair() function is not a string argument while previously Vim
>> did ignore this error. This means the matchparen plugin needs to be
>> updated to fix this bug.
>>
>> <snip>
>>
>> Best,
>> Christian
> Thanks for this information.
>
> So the "matchparen" plugin on my end (run-time v8.0.1766) is
> old and broke with the code fix in patch 8.1.0112. Understood.
>
> Is there a HEAD version of the run-time I can download to match
> the HEAD version of the source tree? Or will I have to wait until
> the repositories are next updated?
>
> Cheers,
>
> Andre.
>

Hi Christian,

I realised just now I asked a silly question regarding a HEAD version
of the runtime when it was staring right at me.

I rebuilt the HEAD version of VIM and symlinked /usr/share/vim/vim80
to the "runtime" directory in the source tree and the new version of
"matchparen.vim" appears to now be working properly.

Thanks for your help.

Cheers,

Andre.

[Tony Mechelynck]

1. Clone the repository (either the git repository or the Mercurial
mirror, they should give you the same source tree).
2. I assume you aren't interested in building several versions of Vim
with different configure arguments.
3. Set up a script to be sourced by your shell in order to set the
configure settings appropriately.
http://users.skynet.be/antoine.mechelynck/vim/compunix.htm has an
example for the bash shell. Modify it as needed. At least the
"compiled-by" will need to be changed, possibly other lines. /!\ Your
shell should *source* this script, not *execute* it.
4. Run "make config" in the vim or vim/src directory of your clone.
This will prepare the configure files according to your wishes _and_
to what software is instaled on your machine. You should log the
output and chek it. If it's OK:
5. Run "make" in the same directory. This should build a Vim
executable and put it in the vim/src directory of your clone.
6. Run "make install". This will install the executable you just built
(stripping the result), normally into /usr/local/bin/. It will also
install the corresponding runtime files, normally under
/usr/local/share/vim/vim81/.
7. I expect "make install" to create (the first ime) a number of
symllinks to your new executable, but if it didn't you may create them
manually as folows (assuming your Vim is GUI-enabled):

pushd /usr/local/bin
ln -sv vim eview
ln -sv vim evim
ln -sv vim ex
ln -sv vim gview
ln -sv vim gvim
ln -sv vim gvimdiff
... (etc. for the executable names found at ":help ex")
popd

Beware that the ln utilty wants the target first and the link name
afterwards, which I find counterintuitive. Adding the -v (verbose)
switch, as above, lets you check that it created the links the way you
wanted them.

When there are source changes, you can then update your clone, make
sure your configuration arguments areset in the environment as with 3
above, and run make followed by make install as in 5 and 6 above. (No
need to reconfigure.) This will rebuild only the changed modules.

If you want to change your configuration arguments, edit the script to
set your environment differently, then run "make reconfig" instead of
"make". This will rerun configure and recompile everything from
scratch. Then run "make install" as above.

The "make install" step will not only install the executable in the
$PATH where it belongs, it will also install the _corresponding_
runtime files (from the vim/runtime/ directory of your clone) where
that executable will expect them. When you run Vim, $VIM and
$VIMRUNTIME should not be set outside of Vim, it will set them
appropriately (usually to /usr/local/share/vim and
/usr/local/share/vim/vim81 respectively; if and when Vim 8.2 gets
released it will set $VIMRUNTIME to something ending in …/vim82
instead).

Best regards,
Tony.

[Christian Brabandt]

On Fr, 25 Okt 2019, Sihera Andre wrote:

> I realised just now I asked a silly question regarding a HEAD version
> of the runtime when it was staring right at me.
>
> I rebuilt the HEAD version of VIM and symlinked /usr/share/vim/vim80
> to the "runtime" directory in the source tree and the new version of
> "matchparen.vim" appears to now be working properly.

Why don't you simply "make install" all the required files (new vim
binary and the new runtime files)?

Best,
Christian
--
Arzt zum Beamten: Wieviel Stunden schlafen Sie täglich?" - Höchstens
zwei." - Das ist aber entschieden zu wenig!" - Finde ich nicht. Ich schlafe
nachts noch mal ungefähr zehn Stunden."

[Sihera Andre]

Hi Christian,

On 25/10/2019 15:31, Christian Brabandt wrote:
> On Fr, 25 Okt 2019, Sihera Andre wrote:
>
>> I realised just now I asked a silly question regarding a HEAD version
>> of the runtime when it was staring right at me.
>>
>> I rebuilt the HEAD version of VIM and symlinked /usr/share/vim/vim80
>> to the "runtime" directory in the source tree and the new version of
>> "matchparen.vim" appears to now be working properly.
> Why don't you simply "make install" all the required files (new vim
> binary and the new runtime files)?
>
> Best,
> Christian

The reason I don't "simply" do "make install" is because, in the past,
what was installed from a locally checked-out repository and what was
shipped with the operating system package manager (in this case, Ubuntu)
often did not match even if I made sure to check out the same version
(or slightly newer) as the repository package versions.

I got tired a long time ago of typing "make install" on a number of
projects (not just ViM) just to have everything stop working as they
destroyed the environment that the OS package manager was doing a very
good job of maintaining. I ended up then (and still do now) using git to
version control a good proportion of my /usr/local and /usr/share settings
just so that when I do a "make install" on a locally-built package and
everything stops working I can simply just do a "git checkout" to bring
it all back.

When I build a new OS (which, incidentally, is only once every three to
five years), I just want to grab the runtime from the official OS
repository (Ubuntu), then checkout the source code so I can build the
"huge" version of the ViM binary with a specific patch for "ctags" which
ViM still hasn't supported for as long as I can remember (10? 15 years?
but that's a different story), then I just substitute vim.tiny in /usr/bin
for vim.huge, same version, no fuss, and that should be it for another
five years.

It may be not your preferred way of doing it, but then environment
stability and continuity of features across upgrades are my top concerns.

Anyway, the original problem with "matchparen.vim" is solved, and the root
cause is known, so I thank you for your time.

Cheers,

Andre.

[Christian Brabandt]

Understood. However note that this is asking for trouble sooner or
later.

The thing is, Vim comes with its own runtime files. Using the runtime
files from your distribution, you will miss many new features and bug
fixes from existing runtime files (e.g. better syntax/indent/ftplugins
or even newly added runtime files) and also the documentation will not
accurately match the version of your installed version of Vim.

I personally also compile my own version using a custom --prefix=
argument to the ./configure files. Then when running make install, those
files do not interfere with the packaged vim-runtime files (and I can
even use either my self-compiled vim binary or the system provided vim,
depending on the path).

Best,
Christian
--
Jemand, der auf die Innenseite einer ungeschälten Banane
Geburtstagsgrüße an seine Tochter schreibt, ist natürlich geistig
verwirrt.
-- Jostein Gaarder

[Sihera Andre]

Hi Christian,

With all due respect...

When the official distribution for an OS has runtime version "X" (for
example, 8.0.1766) and I then "git clone" the ViM source repository and
"git checkout X" to build a binary, and the subsequently built binary,
which I should just be able to drop on top of "vim.tiny", is not
compatible with version "X" of the runtime, then I think it is not me
who should be looking at their build procedures, or version number
management for that matter.

> <snip>

>
> I personally also compile my own version using a custom --prefix=
> argument to the ./configure files. Then when running make install, those
> files do not interfere with the packaged vim-runtime files (and I can
> even use either my self-compiled vim binary or the system provided vim,
> depending on the path).
>
> Best,
> Christian

It's refreshing to see that we're all quite similar :-) If "make install"
is dependable, I may yet get it another try.

Again, thanks for your assistance.

Cheers,

Andre

[Dominique Pellé]

Sihera Andre <andre....@hotmail.co.jp> wrote:

> Hi Christian,
>
>
> On 25/10/2019 17:53, Christian Brabandt wrote:
> > On Fr, 25 Okt 2019, Sihera Andre wrote:
> >
> >> The reason I don't "simply" do "make install" is because, in the past,
> >> what was installed from a locally checked-out repository and what was
> >> shipped with the operating system package manager (in this case, Ubuntu)
> >> often did not match even if I made sure to check out the same version
> >> (or slightly newer) as the repository package versions.
> >>
> >> I got tired a long time ago of typing "make install" on a number of
> >> projects (not just ViM) just to have everything stop working as they
> >> destroyed the environment that the OS package manager was doing a very
> >> good job of maintaining. I ended up then (and still do now) using git to
> >> version control a good proportion of my /usr/local and /usr/share settings
> >> just so that when I do a "make install" on a locally-built package and
> >> everything stops working I can simply just do a "git checkout" to bring
> >> it all back.

Never install things in /usr/bin/
Files there are managed by Ubuntu packages.

Instead, you should install vim in /usr/local/bin.

If you just do...

$ ./configure --with-features=huge
$ make -j8
$ sudo make install

Then vim will be installed in /usr/local (including binary
in /usr/local/bin and runtime in /usr/local/share/...) so
everything will be consistent, and it won't interfere with
Ubuntu packages.

Make sure /usr/local/bin is in your path before /usr/bin
to run your own version of Vim rather than the Ubuntu Vim.

Regards
Dominique

[Sihera Andre]

Hi Dominique,

On 26/10/2019 19:10, Dominique Pellé wrote:
> Sihera Andre <andre....@hotmail.co.jp> wrote:
>
>> Hi Christian,
>>
>>
>> On 25/10/2019 17:53, Christian Brabandt wrote:
>>> On Fr, 25 Okt 2019, Sihera Andre wrote:
>>>
>>>> The reason I don't "simply" do "make install" is because, in the past,
>>>> what was installed from a locally checked-out repository and what was
>>>> shipped with the operating system package manager (in this case, Ubuntu)
>>>> often did not match even if I made sure to check out the same version
>>>> (or slightly newer) as the repository package versions.
>>>>
>>>> I got tired a long time ago of typing "make install" on a number of
>>>> projects (not just ViM) just to have everything stop working as they
>>>> destroyed the environment that the OS package manager was doing a very
>>>> good job of maintaining. I ended up then (and still do now) using git to
>>>> version control a good proportion of my /usr/local and /usr/share settings
>>>> just so that when I do a "make install" on a locally-built package and
>>>> everything stops working I can simply just do a "git checkout" to bring
>>>> it all back.
>
> Never install things in /usr/bin/
> Files there are managed by Ubuntu packages.

Thank you for the advice. I am aware of the risks
of putting files directly in /usr/bin.

As far as I was aware, I didn't know you don't actually
released "vim.huge" into the official repositories, so I
assumed I was free to do so.

Do you release "vim.huge" into the official repositories?

>
> Instead, you should install vim in /usr/local/bin.
>
> If you just do...
>
> $ ./configure --with-features=huge
> $ make -j8
> $ sudo make install

You know what would be *really* useful? If you could
provide a "make uninstall" that put my /usr/local/share
and /usr/share/vim back to the exact state before I
did "make install".

Then I wouldn't have to use "git" to manage it all
myself when it goes wrong.


Cheers,

Andre.

[Tony Mechelynck]

The same sources are good for all featuresets depending on how you set
configure arguments. I'll attach the configure parameters I use
myself. Don't forget to change at least the "compiled by" line (and
possibly other params to suit you) and see
http://users.skynet.be/antoine.mechelynck/vim/compunix.htm about how
to use them. The build I use day-in day-out is the "big" one but I
compile all 5 as a sanity check and from time to time I check some
detail on one of the 4 other builds.

When there is a change to the sources, I compile them all in parallel
(without the make program's -j switch) in 5 different shells each with
its own set of environment settings and each in a different shadow
directory. (About how to make shadow directories, see the "shadow"
target in the src/Makefile and the -e switch to "make".)

>
> >
> > Instead, you should install vim in /usr/local/bin.
> >
> > If you just do...
> >
> > $ ./configure --with-features=huge
> > $ make -j8
> > $ sudo make install
>
> You know what would be *really* useful? If you could
> provide a "make uninstall" that put my /usr/local/share
> and /usr/share/vim back to the exact state before I
> did "make install".
>
> Then I wouldn't have to use "git" to manage it all
> myself when it goes wrong.
>
>
> Cheers,
>
> Andre.
>
>
> > Then vim will be installed in /usr/local (including binary
> > in /usr/local/bin and runtime in /usr/local/share/...) so
> > everything will be consistent, and it won't interfere with
> > Ubuntu packages.
> >
> > Make sure /usr/local/bin is in your path before /usr/bin
> > to run your own version of Vim rather than the Ubuntu Vim.
> >
> > Regards
> > Dominique

Best regards,
Tony.

[Christian Brabandt]

On So, 27 Okt 2019, Sihera Andre wrote:

> You know what would be *really* useful? If you could
> provide a "make uninstall" that put my /usr/local/share
> and /usr/share/vim back to the exact state before I
> did "make install".

Have you actually tried it?

#v+
~/code/vim-src/src/ % make uninstall
/bin/sh ./installman.sh uninstall /home/chrisbra/local/share/man/man1 "" /home/chrisbra/local/share/vim /home/chrisbra/local/sh ─are/vim/vim81 /home/chrisbra/local/share/vim ../runtime/doc 644 vim vimdiff evim
Checking for Vim manual pages in /home/chrisbra/local/share/man/man1...
deleting /home/chrisbra/local/share/man/man1/vim.1
deleting /home/chrisbra/local/share/man/man1/vimtutor.1
deleting /home/chrisbra/local/share/man/man1/vimdiff.1
deleting /home/chrisbra/local/share/man/man1/evim.1
[...]
rm -f /home/chrisbra/local/bin/vim
rm -f /home/chrisbra/local/bin/vimtutor
rm -f /home/chrisbra/local/bin/gvimtutor
rm -f /home/chrisbra/local/bin/ex /home/chrisbra/local/bin/view rm -f /home/chrisbra/local/bin/gvim /home/chrisbra/local/bin/gview
rm -f /home/chrisbra/local/bin/rvim /home/chrisbra/local/bin/rview
rm -f /home/chrisbra/local/bin/rgvim /home/chrisbra/local/bin/rgview
rm -f /home/chrisbra/local/bin/vimdiff /home/chrisbra/local/bin/gvimdiff
rm -f /home/chrisbra/local/bin/evim /home/chrisbra/local/bin/eview
rm -f /home/chrisbra/local/bin/xxd
#v-

Besides, there exists software for managing such things, like stow,
checkinstall, graft, etc.

Best,
Christian
--
Ich geh zur Wahl, einer muß ja dran Schuld sein, was die da oben machen.