python callback called from C++ thread is crashing in PyObject

sagar.n...@gmail.com

unread,

Sep 14, 2015, 7:56:10 AM9/14/15

to python-cffi

Hi All,

I have a C++ class to which a python class object is passed as an argument.

My c++ code is calling a python function of that python class using the stored python object of that class. The python function returns a python dictionary to my C++ function. C++ function further extract values from that python dictionary for further use.

A code snippet is like the below:

C++ Code snippet:

class Wrapper

{

private:

boost::python::object pyObj;

public:

Wrapper(boost::python::object pyObj):pyObj(pyObj)

{

}

~Wrapper()

{

}

// Executed in thread and calls python callback

void getPythonDict()

{

PyGILState_STATE state = PyGILState_Ensure();

boost::python::object pathInfoDict = this->pyObj.attr("getPathDict")(jobid, (uint_t)iostatus);

bool_t status = boost::python::extract<bool_t>(pathInfoDict["status"]);

}

};

class Core

{

public:

Core(){}

~Core(){}

void callCPlusPlusFunction(boost::python::object &pyObj)

{

Wrapper *ptr = new Wrapper(pyObj);

// launch the getPythonDict function in thread and return back to python function

// boost::thread is used for launching thread

}

};

Python Code snippet:

class PyCallback:

def __init__(self):

pass

def getPathDict(self, jobid, iostatus):

pyDict = {}

pyDict ['status'] = True

return pyDict

from libCore import Core <----------------C++ library exposed as python module

class PyManager:

def __init__(self):

self.pyCallback = PyCallback()

self.cObj = Core() <---------------Object of C++ "class Wrapper"

def gateWayToCPlusPlusCall(self):

self.cObj.callCPlusPlusFunction(self.pyCallback)

I am using boost::python::object in C++ to store the python object.

After my c++ method calls the python function, the python callback is crashing.

Below is the analysis of the core file.

#0 0x0000003e52a80036 in PyObject_Malloc () from /usr/lib64/libpython2.6.so.1.0

#1 0x0000003e52a87a4a in PyString_FromStringAndSize () from /usr/lib64/libpython2.6.so.1.0

#2 0x0000003e52aee0e8 in ?? () from /usr/lib64/libpython2.6.so.1.0

#3 0x0000003e52aee613 in ?? () from /usr/lib64/libpython2.6.so.1.0

#4 0x0000003e52aee722 in ?? () from /usr/lib64/libpython2.6.so.1.0

#5 0x0000003e52aeeaff in Py_BuildValue () from /usr/lib64/libpython2.6.so.1.0

#6 0x0000003e52ae6a51 in ?? () from /usr/lib64/libpython2.6.so.1.0

#7 0x0000003e52ad5916 in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#8 0x0000003e52ad6b8f in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#9 0x0000003e52ad6b8f in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#10 0x0000003e52ad7657 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.6.so.1.0

#11 0x0000003e52ad5aa4 in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#12 0x0000003e52ad7657 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.6.so.1.0

#13 0x0000003e52ad5aa4 in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#14 0x0000003e52ad6b8f in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#15 0x0000003e52ad6b8f in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#16 0x0000003e52ad6b8f in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#17 0x0000003e52ad7657 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.6.so.1.0

#18 0x0000003e52ad5aa4 in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#19 0x0000003e52ad7657 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.6.so.1.0

#20 0x0000003e52ad5aa4 in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#21 0x0000003e52ad6b8f in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#22 0x0000003e52ad6b8f in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

#23 0x0000003e52ad7657 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.6.so.1.0

#24 0x0000003e52a6acb0 in ?? () from /usr/lib64/libpython2.6.so.1.0

#25 0x0000003e52a43c63 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0

#26 0x0000003e52a566af in ?? () from /usr/lib64/libpython2.6.so.1.0

#27 0x0000003e52a43c63 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0

#28 0x0000003e52acfc93 in PyEval_CallObjectWithKeywords () from /usr/lib64/libpython2.6.so.1.0

#29 0x0000003e52aee98a in PyEval_CallFunction () from /usr/lib64/libpython2.6.so.1.0

#30 0x00007f2a84665123 in boost::python::call<boost::python::api::object, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned int> (callable=0x24c55a0, a0=<value optimized out>, a1=<value optimized out>)

at /usr/include/boost141/boost/python/call.hpp:66

#31 0x00007f2a84663885 in operator()<ddsCore::jobid_t, ddsCore::uint_t> (this=0x7f2a5049c2a0, pathInfo=..., jobid=...,

iostatus=ddsCore::IO_STATE_SUCCESS) at /usr/include/boost141/boost/python/object_call.hpp:19

#32 ddsCore::multiPath::PathManagerWrapper::getPathInformation (this=0x7f2a5049c2a0, pathInfo=..., jobid=...,

iostatus=ddsCore::IO_STATE_SUCCESS) at src/ddsCore/src/pathManagerWrapper.cpp:21

#33 0x00007f2a8465e6f9 in ddsCore::multiPath::PathFailoverHandler::pathFailoverSubroutine (this=0x7f2a506474b0, pathInfo=..., jobid=...,

iostatus=ddsCore::IO_STATE_SUCCESS) at src/ddsCore/src/pathFailoverInterface.cpp:50

#34 0x00007f2a8465fca7 in ddsCore::multiPath::PathFailoverManager::executePathFailoverHandler (this=0x7f2a5048f260)

at src/ddsCore/src/pathFailoverManager.cpp:86

#35 0x00007f2a84624d93 in ddsCore::ioLayerUnit::AsynchronousReaderInterface::requestResubmitter (this=0x7f2a5043e8a0)

at src/ddsCore/src/ioWorkers.cpp:524

#36 0x00007f2a8466259a in operator() (this=<value optimized out>) at /usr/include/boost141/boost/function/function_template.hpp:1013

---Type <return> to continue, or q <return> to quit---

#37 boost::detail::thread_data<boost::function<void()> >::run(void) (this=<value optimized out>)

at /usr/include/boost141/boost/thread/detail/thread.hpp:56

#38 0x00007f2a83e55d10 in thread_proxy () from /usr/lib64/libboost_thread-mt.so.1.41.0

#39 0x0000003e51a079d1 in start_thread () from /lib64/libpthread.so.0

#40 0x0000003e512e88fd in clone () from /lib64/libc.so.6

(gdb) frame 0

#0 PyObject_Malloc (nbytes=42) at Objects/obmalloc.c:780

780 if ((pool->freeblock = *(block **)bp) != NULL) {

(gdb) l

775 * Pick up the head block of its free list.

776 */

777 ++pool->ref.count;

778 bp = pool->freeblock;

779 assert(bp != NULL);

780 if ((pool->freeblock = *(block **)bp) != NULL) { <------- crash happened at this point

781 UNLOCK();

782 return (void *)bp;

783 }

784 /*

(gdb) print pool

$3 = (poolp) 0x7f2a58121000

(gdb) print pool->freeblock

$4 = (block *) 0x0

(gdb) print pool->ref

$5 = {_padding = 0x54 <Address 0x54 out of bounds>, count = 84}

(gdb) print pool->ref.count

$6 = 84

(gdb) print pool->freeblock

$7 = (block *) 0x0 <----------------- Why this pointer is NULL, this is suspicious ????

(gdb) print *pool

$8 = {ref = {_padding = 0x54 <Address 0x54 out of bounds>, count = 84}, freeblock = 0x0, nextpool = 0x7f2a54031000, prevpool =

0x7f2a54031000, arenaindex = 18, szidx = 5, nextoffset = 4080, maxnextoffset = 4048}

Since the python callback is called from a non-python thread so i have ensured PyGILState_STATE as well.

Can any one please help me in this issue !!!!

sagar.n...@gmail.com

unread,

Sep 14, 2015, 7:59:17 AM9/14/15

to python-cffi

self.cObj = Core() <---------------Object of C++ "class Core"

Ryan Gonzalez

unread,

Sep 14, 2015, 10:33:44 AM9/14/15

to pytho...@googlegroups.com, sagar.n...@gmail.com

Ummm...this is related to CFFI, not the Python C API. You might want to check out the python-list mailing list or Stack Overflow.

--
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

Armin Rigo

unread,

Sep 14, 2015, 12:03:38 PM9/14/15

to pytho...@googlegroups.com, sagar.n...@gmail.com

Hi Sagar,

This mailing list is about the CFFI project, not about Boost. Please
ask on the Boost mailing list.

A bientôt,

Armin.

Reply all

Reply to author

Forward

python callback called from C++ thread is crashing in PyObject_Malloc

sagar.n...@gmail.com

sagar.n...@gmail.com

Ryan Gonzalez

Armin Rigo