Vagrind errors with python + compilation warnings in if_python.c
Dominique Pellé
I see some valgrind errors when using the ":python" command
with Vim-7.2.351 on Linux. It happens when using the Python lib
from Ubuntu-9.10 packages. I then downloaded Python-2.6.4 sources
from http://www.python.org/download/ and compiled Python myself
with -O0 -g to have symbols in stack trace.
Doing the following for example gives valgrind errors:
$ cd vim
$ ./configure --with-features=huge \
--enable-pythoninterp \
--with-python-config-dir=/usr/local/lib/python2.6/config
$ cd src
$ valgrind --num-callers=50 \
--log-file=vg.log \
./vim -u NONE -U NONE -c ':python a=0' -c ':q'
... and 'vg.log' contains errors, see:
http://dominique.pelle.free.fr/valgrind-errors-vim7.2.351-python2.6.4.txt
I'm not sure whether the bug is in Python library or in Vim.
Unfortunately, some stack traces look wrong: unrolling the stack
seems to loop so it's difficult to understand where is the problem.
I also see compilation warnings when compiling vim/src/if_python.c:
if_python.c:759: warning: missing initializer
if_python.c:759: warning: (near initialization for ‘OutputType.tp_getattro’)
if_python.c:1457: warning: missing initializer
if_python.c:1457: warning: (near initialization for ‘BufferAsSeq.sq_contains’)
if_python.c:1480: warning: missing initializer
if_python.c:1480: warning: (near initialization for ‘BufferType.tp_getattro’)
if_python.c:1699: warning: missing initializer
if_python.c:1699: warning: (near initialization for ‘RangeAsSeq.sq_contains’)
if_python.c:1722: warning: missing initializer
if_python.c:1722: warning: (near initialization for ‘RangeType.tp_getattro’)
if_python.c:1873: warning: missing initializer
if_python.c:1873: warning: (near initialization for ‘BufListAsSeq.sq_contains’)
if_python.c:1896: warning: missing initializer
if_python.c:1896: warning: (near initialization for ‘BufListType.tp_getattro’)
if_python.c:1960: warning: missing initializer
if_python.c:1960: warning: (near initialization for ‘WindowType.tp_getattro’)
if_python.c:2195: warning: missing initializer
if_python.c:2195: warning: (near initialization for ‘WinListAsSeq.sq_contains’)
if_python.c:2218: warning: missing initializer
if_python.c:2218: warning: (near initialization for ‘WinListType.tp_getattro’)
if_python.c:2280: warning: missing initializer
if_python.c:2280: warning: (near initialization for ‘CurrentType.tp_getattro’)
Mismatch happens in 2 structs:
- PyTypeObject (at lines 759, 1480, 1722, 1896, 1960, 2218, 2280)
- PySequenceMethods (at lines 1457, 1699, 1873, 2195)
The number of fields in structures do not match. For example, 7 fields
are initialized in a 'PySequenceMethods' struct in if_python.c, but I
see 10 fields in python2.6/object.h.
vim/src/if_python.c:
1449 static PySequenceMethods BufferAsSeq = {
1450 (PyInquiry) BufferLength, /* sq_length, len(x) */
1451 (binaryfunc) 0, /* BufferConcat, */ /*
sq_concat, x+y */
1452 (PyIntArgFunc) 0, /* BufferRepeat, */ /*
sq_repeat, x*n */
1453 (PyIntArgFunc) BufferItem, /* sq_item, x[i] */
1454 (PyIntIntArgFunc) BufferSlice, /* sq_slice, x[i:j] */
1455 (PyIntObjArgProc) BufferAssItem, /* sq_ass_item, x[i]=v */
1456 (PyIntIntObjArgProc) BufferAssSlice, /*
sq_ass_slice, x[i:j]=v */
1457 };
/usr/local/include/python2.6/object.h:
273 typedef struct {
274 lenfunc sq_length;
275 binaryfunc sq_concat;
276 ssizeargfunc sq_repeat;
277 ssizeargfunc sq_item;
278 ssizessizeargfunc sq_slice;
279 ssizeobjargproc sq_ass_item;
280 ssizessizeobjargproc sq_ass_slice;
281 objobjproc sq_contains;
282 /* Added in release 2.0 */
283 binaryfunc sq_inplace_concat;
284 ssizeargfunc sq_inplace_repeat;
285 } PySequenceMethods;
C partial initialization of struct will initialize remaining
fields to 0, but is it correct here?
I assume Vim is meant to work with Python-2.6.4, but Vim help
files do not mention Python-2.6 at all.
Cheers
-- Dominique
Bram Moolenaar
Dominique Pelle wrote:
> I see some valgrind errors when using the ":python" command
> with Vim-7.2.351 on Linux. It happens when using the Python lib
> from Ubuntu-9.10 packages. I then downloaded Python-2.6.4 sources
> from http://www.python.org/download/ and compiled Python myself
> with -O0 -g to have symbols in stack trace.
>
> Doing the following for example gives valgrind errors:
>
> $ cd vim
> $ ./configure --with-features=huge \
> --enable-pythoninterp \
> --with-python-config-dir=/usr/local/lib/python2.6/config
>
> $ cd src
> $ valgrind --num-callers=50 \
> --log-file=vg.log \
> ./vim -u NONE -U NONE -c ':python a=0' -c ':q'
>
> ... and 'vg.log' contains errors, see:
> http://dominique.pelle.free.fr/valgrind-errors-vim7.2.351-python2.6.4.txt
I get similar errors. They look like Python errors to me. Not sure if
they are caused by how Vim uses the Python interpreter.
One of the warnings is about memory used that was freed by realloc().
That's usually a mistake of using a pointer that points into an area
that grew bigger and was reallocated.
I don't get these. Are you using non-default compiler arguments?
The PyTypeObject struct ends with some fileds that must not be
initialized, I don't think we can ever fill all the fields.
> I assume Vim is meant to work with Python-2.6.4, but Vim help
> files do not mention Python-2.6 at all.
I also have Python 2.6.4.
--
hundred-and-one symptoms of being an internet addict:
223. You set up a web-cam as your home's security system.
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ download, build and distribute -- http://www.A-A-P.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
Dominique Pellé
> Dominique Pelle wrote:
>
>> I see some valgrind errors when using the ":python" command
>> with Vim-7.2.351 on Linux. It happens when using the Python lib
>> from Ubuntu-9.10 packages. I then downloaded Python-2.6.4 sources
>> from http://www.python.org/download/ and compiled Python myself
>> with -O0 -g to have symbols in stack trace.
>>
>> Doing the following for example gives valgrind errors:
>>
>> $ cd vim
>> $ ./configure --with-features=huge \
>> --enable-pythoninterp \
>> --with-python-config-dir=/usr/local/lib/python2.6/config
>>
>> $ cd src
>> $ valgrind --num-callers=50 \
>> --log-file=vg.log \
>> ./vim -u NONE -U NONE -c ':python a=0' -c ':q'
>>
>> ... and 'vg.log' contains errors, see:
>> http://dominique.pelle.free.fr/valgrind-errors-vim7.2.351-python2.6.4.txt
>
> I get similar errors. They look like Python errors to me. Not sure if
> they are caused by how Vim uses the Python interpreter.
>
> One of the warnings is about memory used that was freed by realloc().
> That's usually a mistake of using a pointer that points into an area
> that grew bigger and was reallocated.
OK, I'll try to look further this weekend. Last time I looked it
was unclear to me where to start since the stack in incomplete
and possibly incorrect. If I can't figure it out, I might post in a
Python mailing list.
I suspect that the access to free memory may explain this bug...
https://bugs.launchpad.net/ubuntu/+source/vim/+bug/510362
... since it was reported to happen with a plugin that uses
Python (but I have not been able to reproduce this bug myself)
Ah yes, sorry, I forgot that I changed CFLAGS in src/Makefile.
Compiling with gcc flags "-Wall -Wextra" is enough to gives those
warnings (CFLAGS = -O0 -g -Wall -Wextra)
Cheers
-- Dominique
Bram Moolenaar
Dominique Pelle wrote:
The Makefile has this line for that purpose:
PYTHON_CFLAGS_EXTRA = -Wno-missing-field-initializers
--
hundred-and-one symptoms of being an internet addict:
232. You start conversations with, "Have you gotten an ISDN line?"
Dominique Pellé
> Dominique Pelle wrote:
>
>> I see some valgrind errors when using the ":python" command
>> with Vim-7.2.351 on Linux. It happens when using the Python lib
>> from Ubuntu-9.10 packages. I then downloaded Python-2.6.4 sources
>> from http://www.python.org/download/ and compiled Python myself
>> with -O0 -g to have symbols in stack trace.
...
> I get similar errors. They look like Python errors to me. Not sure if
> they are caused by how Vim uses the Python interpreter.
This is not an issue after all.
This document...
http://svn.python.org/view/python/trunk/Misc/README.valgrind?view=markup
... explains why Valgrind reports the error in PyObject_Free().
It is harmless and expected. It can be silenced by configuring
Python with the "--without-pymalloc" option.
I verified that there is no valgrind error anymore after configuring
Python with that option.
Sorry for the noise.
-- Dominique