VIM crash with YouCompleteMe plugin

[BBV]

Hello,

I am trying to setup vim such that I can use 'YouCompleteMe' plugin ().

I have built the vim from source. If I comment out that plugin from vimrc, it works fine, but not when enabled. The vim invocation fails with following error:

Vim: Caught deadly signal ABRT

Vim: Finished.

I did a recompile of Vim with following configure options to be able to get some more information:
- CFLAGS=" -g -O0 -Wall -Wextra -Wshadow -Wmissing-prototypes -Wunreachable-code -Wno-cast-function-type -Wno-deprecated-declarations -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1"
- INSTALL_UNSTRIPPED=yes

I have run Vim
with gdb: https://pastebin.com/ET5may5K
with valgrind: https://pastebin.com/V2A26ZJ5  (part 1), https://pastebin.com/jBTzL0xd (part 2)
--version : https://pastebin.com/shKjt5G3

Could some one please tell, is this something related to setup or is it a known issue (or looks like a issue) in vim?

Regards,
bbv

[Richard Mitchell]

YCM is currently working for me.  My Vim is a little behind yours (but I thought it was recent):

VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Apr 13 2020 06:16:41)

macOS version

Included patches: 1-539

Huge version with MacVim GUI.  Features included (+) or not (-):

Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_MACVIM -Wall -Wno-unknown-pragmas -pipe

-DMACOS_X -DMACOS_X_DARWIN  -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1

Linking: gcc   -L/usr/local/lib -o Vim -framework Cocoa -framework Carbon       -lm -lncurses  -lico

nv -framework AppKit   -fstack-protector  -L/System/Library/Perl/5.18/darwin-thread-multi-2level/COR

E -lperl

YcmDebugInfo:

-- Server Python interpreter: /usr/local/bin/python3

-- Server Python version: 3.8.1

-- Server has Clang support compiled in: True

-- Clang version: clang version 9.0.0 (tags/RELEASE_900/final)

-- Extra configuration file found and loaded

I don't see how any of this helps you though.  I've been using YCM for a few years now and occasionally need to tweak my stuff to get it working again.

Good luck!

[BBV]

Thanks Richard for your response. I don't see anything wrong in my setup myself, hence I would like someone to please look into the information I provided and please let me know, if I am missing something or this is a issue with vim itself. As you may have noticed, the `vim --version` output shows that the vim is built with python3 support.

[Dominique Pellé]

I also use the YouCompleteMe plugin without problem
with the latest vim.

However vim built with libpython3 (or libpython) unfortunately
gives many warnings. This is explained here:
https://github.com/python/cpython/blob/master/Misc/README.valgrind

I just avoided the warning in libpython3 by doing this
on xubuntu-18.04.4:

1) download and build Python3 source code:

$ wget https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tar.xz
$ tar xvf Python-3.8.2.tar.xz
$ ./configure --with-valgrind--enable-shared
$ make -j8
$ sudo make install
$ sudo lddconfig

(note the --with-valgrind option which is needed to avoid valgrind errors)

2) rebuild vim

$ cd vim
$ rm src/auto/config.cache
$ ./configure --with-features=huge --enable-gui=gtk3 --enable-python3interp=yes
$ make -j8
$ sudo make install

3) rebuild the YouCompleteVimPlugin

Assuming you installed it with Vundle, then you can rebuild it with:

$ cd ~/.vim/bundle/YouCompleteMe
$ ./install.py --clang-completer

And then I can run vim with valgrind without the ugly
valgrind errors. Well, I still get a couple of errors
with valgrind in libpython but far less that I used to see
and only at startup. I still get:

==29794== Conditional jump or move depends on uninitialised value(s)
==29794== at 0x9014E77: __wcsnlen_avx2 (strlen-avx2.S:261)
==29794== by 0x8F42EC1: wcsrtombs (wcsrtombs.c:104)
==29794== by 0x8EC8B20: wcstombs (wcstombs.c:34)
==29794== by 0x88BFA47: wcstombs (stdlib.h:154)
==29794== by 0x88BFA47: encode_current_locale (fileutils.c:653)
==29794== by 0x88BFA47: encode_locale_ex (fileutils.c:763)
==29794== by 0x88BFA47: encode_locale (fileutils.c:773)
==29794== by 0x88BFA47: _Py_EncodeLocaleRaw (fileutils.c:809)
==29794== by 0x88C0E00: _Py_wreadlink (fileutils.c:1675)
==29794== by 0x88C1B57: calculate_argv0_path (getpath.c:962)
==29794== by 0x88C1B57: calculate_path (getpath.c:1238)
==29794== by 0x88C3438: _PyPathConfig_Calculate (getpath.c:1353)
==29794== by 0x889316F: pathconfig_calculate (pathconfig.c:317)
==29794== by 0x889316F: config_calculate_pathconfig (pathconfig.c:335)
==29794== by 0x889316F: _PyConfig_InitPathConfig (pathconfig.c:402)
==29794== by 0x8885A43: config_read (initconfig.c:1718)
==29794== by 0x8885A43: PyConfig_Read (initconfig.c:2448)
==29794== by 0x8899712: pyinit_core (pylifecycle.c:853)
==29794== by 0x8899712: Py_InitializeFromConfig (pylifecycle.c:1035)
==29794== by 0x889A37B: Py_InitializeEx (pylifecycle.c:1073)
==29794== by 0x353AC4: Python3_Init (if_python3.c:908)
==29794== by 0x353AC4: DoPyCommand (if_python3.c:970)
And a couple more but only when starting vim, after that
don't get more valgrind errors anymore.

Can you try something like that too?

Regards
Dominique

[Marius Gedminas]

On Tue, Apr 21, 2020 at 09:39:07PM +0200, Dominique Pellé wrote:
> However vim built with libpython3 (or libpython) unfortunately
> gives many warnings. This is explained here:
> https://github.com/python/cpython/blob/master/Misc/README.valgrind
>
> I just avoided the warning in libpython3 by doing this
> on xubuntu-18.04.4:
>
> 1) download and build Python3 source code:
>
> $ wget https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tar.xz
> $ tar xvf Python-3.8.2.tar.xz
> $ ./configure --with-valgrind--enable-shared
> $ make -j8
> $ sudo make install
> $ sudo lddconfig
>
> (note the --with-valgrind option which is needed to avoid valgrind errors)

This seems hard. It's easier to export PYTHONMALLOC=malloc before
running valgrind. Should work with any Python version >= 3.6.
Worked for me with Python 3.7 from Ubuntu 19.10.

I've also used valgrind --suppressions=valgrind-python.supp, with
https://github.com/python/cpython/blob/master/Misc/valgrind-python.supp
modified as instructed to uncomment all those _PyObject_Free/_PyObject_Realloc
suppressions. It didn't help any: I still got hundreds of errors
without PYTHONMALLOC.

Marius Gedminas
--
Look! Before our very eyes, the future is becoming the past.

[Dominique Pellé]

Marius Gedminas <mar...@gedmin.as> wrote:

> On Tue, Apr 21, 2020 at 09:39:07PM +0200, Dominique Pellé wrote:
> > However vim built with libpython3 (or libpython) unfortunately
> > gives many warnings. This is explained here:
> > https://github.com/python/cpython/blob/master/Misc/README.valgrind
> >
> > I just avoided the warning in libpython3 by doing this
> > on xubuntu-18.04.4:
> >
> > 1) download and build Python3 source code:
> >
> > $ wget https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tar.xz
> > $ tar xvf Python-3.8.2.tar.xz
> > $ ./configure --with-valgrind--enable-shared
> > $ make -j8
> > $ sudo make install
> > $ sudo lddconfig
> >
> > (note the --with-valgrind option which is needed to avoid valgrind errors)
>
> This seems hard. It's easier to export PYTHONMALLOC=malloc before
> running valgrind. Should work with any Python version >= 3.6.
> Worked for me with Python 3.7 from Ubuntu 19.10.

I had tried that with xubuntu-18.04.4 and export PYTHONMALLOC=malloc
did not suppress valgrind errors. Yet ubuntu-18.04 is using Python-3.6.

> I've also used valgrind --suppressions=valgrind-python.supp, with
> https://github.com/python/cpython/blob/master/Misc/valgrind-python.supp
> modified as instructed to uncomment all those _PyObject_Free/_PyObject_Realloc
> suppressions. It didn't help any: I still got hundreds of errors
> without PYTHONMALLOC.

Same.

My solution to rebuild libpython worked for me at least.

Regards
Dominique

[Marius Gedminas]

Hm, I sent a reply to this but it never made it to the list. A newer
reply in the same thread made it. I've no idea what's going on, but I'm
going to try again.

On Mon, Apr 20, 2020 at 02:09:34AM -0700, BBV wrote:
> I am trying to setup vim such that I can use 'YouCompleteMe' plugin ().
>
> I have built the vim from source. If I comment out that plugin from vimrc, it
> works fine, but not when enabled. The vim invocation fails with following
> error:
>
> Vim: Caught deadly signal ABRT
> Vim: Finished.
>
> I did a recompile of Vim with following configure options to be able to get
> some more information:
> - CFLAGS=" -g -O0 -Wall -Wextra -Wshadow -Wmissing-prototypes
> -Wunreachable-code -Wno-cast-function-type -Wno-deprecated-declarations
> -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1"
> - INSTALL_UNSTRIPPED=yes
>
> I have run Vim

> with gdb: [1]https://pastebin.com/ET5may5K
> with valgrind: [2]https://pastebin.com/V2A26ZJ5  (part 1), [3]https://
> pastebin.com/jBTzL0xd (part 2)
> --version : [4]https://pastebin.com/shKjt5G3

The gdb trace indicates the crash is happening during Python
interpreter initialization. Can you rule you YCM by trying

vim --clean +'py3 print("hi")'

and seeing if it crashes?

Another thing you could try is export PYTHONFAULTHANDLER=1 before
running vim, then Python will print the Python-level traceback on SIGABRT.
It might or might not help figure out what's going on.

Marius Gedminas
--
If you are good, you will be assigned all the work. If you are real
good, you will get out of it.

[Marius Gedminas]

On Mon, Apr 20, 2020 at 02:09:34AM -0700, BBV wrote:

> I have built the vim from source. If I comment out that plugin from vimrc, it
> works fine, but not when enabled. The vim invocation fails with following
> error:
>
> Vim: Caught deadly signal ABRT
> Vim: Finished.
>
> I did a recompile of Vim with following configure options to be able to get
> some more information:
> - CFLAGS=" -g -O0 -Wall -Wextra -Wshadow -Wmissing-prototypes
> -Wunreachable-code -Wno-cast-function-type -Wno-deprecated-declarations
> -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1"
> - INSTALL_UNSTRIPPED=yes
>
> I have run Vim

> with gdb: [1]https://pastebin.com/ET5may5K

The gdb trace indicates the crash is happening during Python
interpreter initialization. Can you rule you YCM by trying

vim --clean +'py3 print("hi")'

and seeing if it crashes?

Marius Gedminas
--
TCP_SeqNum - The 32-bit Sequence Number, encoded as an ASCII string
representing the hex value of the Sequence number. This field
MUST be sent as lower case because it is not urgent.
-- RFC 3093

[BBV]

On Wednesday, April 22, 2020 at 10:20:29 PM UTC+5:30, Marius Gedminas wrote:

Hm, I sent a reply to this but it never made it to the list.  A newer
reply in the same thread made it.  I've no idea what's going on, but I'm
going to try again.

Thanks for your response and others too.

On Mon, Apr 20, 2020 at 02:09:34AM -0700, BBV wrote:
> I am trying to setup vim such that I can use 'YouCompleteMe' plugin ().
>
> I have built the vim from source. If I comment out that plugin from vimrc, it
> works fine, but not when enabled. The vim invocation fails with following
> error:
>
> Vim: Caught deadly signal ABRT
> Vim: Finished.
>
> I did a recompile of Vim with following configure options to be able to get
> some more information:
> - CFLAGS=" -g -O0 -Wall -Wextra -Wshadow -Wmissing-prototypes
> -Wunreachable-code -Wno-cast-function-type -Wno-deprecated-declarations
> -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1"
> - INSTALL_UNSTRIPPED=yes
>
> I have run Vim
> with gdb: [1]https://pastebin.com/ET5may5K

The gdb trace indicates the crash is happening during Python
interpreter initialization.  Can you rule you YCM by trying

   vim --clean +'py3 print("hi")'

and seeing if it crashes?

 

This also crashed.

Another thing you could try is export PYTHONFAULTHANDLER=1 before
running vim, then Python will print the Python-level traceback on SIGABRT.
It might or might not help figure out what's going on.

This didn't help. There wasn't any output at all. 

In my setup, I bulit vim myself, but not python. Perhaps, it is worth trying to build python also once. I will try that and see. As I understand, the option '--with-valgrind' is only useful to avoid certain warnings with valgrind, but doesn't really make my difference to what I am trying. Anyways, since I would be building python, I would still provide it.

- BBV

[BBV]

I tried to compile python-3.8.2 myself and it seems to take me ahead with the previous issue.  I don't see any vim crash anymore. Thanks a lot for your help!

However, the YouCompleteMe plugin still doesn't work. When I open vim, it shows an error at the bottom:
YouCompleteMe unavailable: /remote/cwh3/common/installations/local/lib/python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so: undefined symbol: PyFloat_Type

[Christian Brabandt]

On Mi, 22 Apr 2020, Marius Gedminas wrote:

> Hm, I sent a reply to this but it never made it to the list. A newer
> reply in the same thread made it. I've no idea what's going on, but I'm
> going to try again.

Sometimes, Google decides a mail is spam, and then it is left in the
moderation queue, even if the user has gone through the moderation
queue. It used to happen even to me as list moderator.

And then sometimes, Google does not notify list moderators of messages,
that hang around in the list queue. So they hang around there until
someone notices and manual approves it.

Together this is rather annoying, but I doubt there is anything we can
do against :/ In your case however, I did not see your message in the
moderation queue so I don't really know what happened here.

Best,
Christian
--
Man wird in der Regel keinen Freund dadurch verlieren, daß man ihm
ein Darlehen abschlägt, aber sehr leicht dadurch, daß man es ihm
gibt.
-- Arthur Schopenhauer

[Marius Gedminas]

On Thu, Apr 23, 2020 at 04:36:38PM +0200, Christian Brabandt wrote:
> On Mi, 22 Apr 2020, Marius Gedminas wrote:
>
> > Hm, I sent a reply to this but it never made it to the list. A newer
> > reply in the same thread made it. I've no idea what's going on, but I'm
> > going to try again.
>
> Sometimes, Google decides a mail is spam, and then it is left in the
> moderation queue, even if the user has gone through the moderation
> queue. It used to happen even to me as list moderator.
>
> And then sometimes, Google does not notify list moderators of messages,
> that hang around in the list queue. So they hang around there until
> someone notices and manual approves it.
>
> Together this is rather annoying, but I doubt there is anything we can
> do against :/ In your case however, I did not see your message in the
> moderation queue so I don't really know what happened here.

According to the mail headers, Google's SMTP servers put my local SMTP
server in a timeout for 8 hours.

This happens sometimes, and I'm pretty sure I know why (local users
forwarding all if their incoming mail into gmail, which includes all the
spam my postfix + spamassassin misses).

Marius Gedminas
--
Be cheerful. Strive to be happy

Oh, and we just set fire to your desktop.

[Marius Gedminas]

On Thu, Apr 23, 2020 at 04:21:30AM -0700, BBV wrote:
> I tried to compile python-3.8.2 myself and it seems to take me ahead with the
> previous issue.  I don't see any vim crash anymore. Thanks a lot for your help!
>
> However, the YouCompleteMe plugin still doesn't work. When I open vim, it shows
> an error at the bottom:
> YouCompleteMe unavailable: /remote/cwh3/common/installations/local/lib/
> python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so: undefined symbol:
> PyFloat_Type

This usually means an inconsistency between the different builds of
software you built at different times.

E.g. have you rebuilt vim against your newly built python?

The fix is to clean the bad builds and build them from scratch, once
you've identified where the bad builds are.

Let's narrow this down again:

- do you see this with vim --clean +'py3 import socket'
- do you see this with python3.8 -c 'import socket'

This should let you decide if the problem is with your vim build, or
with your Python build.

Marius Gedminas
--
Never reinvent the wheel unless you really need a square one. :^)
-- Duncan Booth

[BBV]

On Friday, April 24, 2020 at 2:50:14 PM UTC+5:30, Marius Gedminas wrote:

On Thu, Apr 23, 2020 at 04:21:30AM -0700, BBV wrote:
> I tried to compile python-3.8.2 myself and it seems to take me ahead with the
> previous issue.  I don't see any vim crash anymore. Thanks a lot for your help!
>
> However, the YouCompleteMe plugin still doesn't work. When I open vim, it shows
> an error at the bottom:
> YouCompleteMe unavailable: /remote/cwh3/common/installations/local/lib/
> python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so: undefined symbol:
> PyFloat_Type

This usually means an inconsistency between the different builds of
software you built at different times.

E.g. have you rebuilt vim against your newly built python?

The fix is to clean the bad builds and build them from scratch, once
you've identified where the bad builds are.

Let's narrow this down again:

I really appreciate your intent to help!

 

- do you see this with vim --clean +'py3 import socket'
- do you see this with python3.8 -c 'import socket'

This should let you decide if the problem is with your vim build, or
with your Python build.

I did some similar experiments based on suggestion I got at an online gitter room for YouCompleteMe from the maintainers itself. I did rebuild Vim, but the issue still occurred. So, after a bit of googling, I found someone suggesting just to have following done before building Vim (https://github.com/ycm-core/YouCompleteMe/issues/3580#issuecomment-589933447):
     $ export LDFLAGS="-rdynamic"

This made it work!

Then, I still saw some issue in starting YCM server, which was due to LD_LIBRARY_PATH not having path to locally installed Python3 libraries.

Now, it works fine. I am using 'clangd-completer' for semantic completion within Vim and the one packaged with YCM didn't work, so I used a different one (which luckily I already had available).

Thanks,