__pyx_vtab of Python object is null, when calling methods from C++

[Артем Бабаев (Vavp)]

Hello, friends! I'm trying to write a system that can be run in both python and C++. If you build classes around the 'cdef class', then C++ can only work with it through the python C API, and the nature of such code will be very dynamic. That's why I'm trying to build classes around the 'cdef cppclass' there is actually more cython code, and there is an internal state of the C++ object, but for this particular example it doesn't matter, so I made a very minimalistic example to illustrate the problem.

### apple.pyx

cdef cppclass Apple:
    PyApple pyObj
    void Func():
        pyObj.Func()

cdef class PyApple:
    cpdef void Func(self):
        print("Python Func definition")

I compile (cythonize) the code in a standard way from the documentation, using the language="c++". I get apple.cpp. To simplify the demonstration, I simply paste my main function template at the end of this file.

### the code inserted at the end apple.cpp struct PyScopeInit
{
    PyScopeInit(){ Py_Initialize(); }
    ~PyScopeInit(){ Py_Finalize(); }
};

int main()
{
    const char* moduleName = "apple";
    if (PyImport_AppendInittab(moduleName, PyInit_apple) == -1)
    {
        PyErr_Print();
        return 1;
    }

    PyScopeInit scopeInit;
   
    { //this must be done 1 time, otherwise the static module info will not be filled in.
        PyObject* pModule = PyImport_ImportModule(moduleName);
        if (!pModule) {
            PyErr_Print();
            return 1;
        }
        Py_XDECREF(pModule);
    }

    __pyx_t_5apple_Apple obj; ///< ok
    obj.Func();               ///< crushing
}

the crash occurs at this moment because __pyx_vtab is null:

  /* "apple.pyx":4
 *     PyApple pyObj
 *     void Func():
 *         pyObj.Func()             # <<<<<<<<<<<<<<
 *
 * cdef class PyApple:
 */
  ((struct __pyx_vtabstruct_5apple_PyApple *)pyObj->__pyx_vtab)->Func(pyObj, 0); if (unlikely(PyErr_Occurred())) __PYX_ERR(0, 4, __pyx_L1_error)

Looking at the Apple constructor, pyObj is initialized as Py_None, and __pyx_vtab is null after this initialization.

//part of Apple ctor:

pyObj = ((struct __pyx_obj_5apple_PyApple *)Py_None); Py_INCREF(Py_None);

I thought that maybe the explicit initialization of PyApple was missing, so I added it in apple.pyx:

### edited apple.pyx

cdef cppclass Apple:
    PyApple pyObj
    Apple():
        pyObj = PyApple()
    void Func():
        pyObj.Func()

cdef class PyApple:
    cpdef void Func(self):
        print("Python Func definition")

I get apple.cpp. after editing of apple.pyx and recompile, pyObj is still initialized as Py_None, but after that, a call to the _Pyx_f___init__Apple method was added. And there's the most interesting part. I'll copy the definition of this method, with just the right pieces that raise questions. I will reflect them in the comments to the code.

//definition

void __pyx_t_5apple_Apple::__pyx_f___init__Apple(void) {
  CYTHON_UNUSED struct __pyx_obj_5apple_PyApple *__pyx_v_pyObj = NULL;
  PyObject *__pyx_t_1 = NULL;

// after this initialization, the __pyx_t_1 object has the correct __pyx_vtab

  __pyx_t_1 = __Pyx_PyObject_CallNoArg(((PyObject *)__pyx_ptype_5apple_PyApple)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 4, __pyx_L1_error)

// filling in the __pyx_v_pyObj object, but not the pyObj class field!
  __pyx_v_pyObj = ((struct __pyx_obj_5apple_PyApple *)__pyx_t_1);
  __pyx_t_1 = 0;

  /* function exit code */
  goto __pyx_L0;
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_1);
  __pyx_L0:;
  __Pyx_XDECREF((PyObject *)__pyx_v_pyObj);
}

It's still crumbling, but it's easy to fix now. In the definition of __pyx_f___init__, it is enough for Apple to fill in pyObj and not delete this link at the end of the function.

// ...

  __pyx_v_pyObj = ((struct __pyx_obj_5apple_PyApple *)__pyx_t_1);

// filling member of class

  pyObj = __pyx_v_pyObj;

// ...

// delete this line

__Pyx_XDECREF((PyObject *)__pyx_v_pyObj);

After that, the __pyx_vtab field of the pyObj class is correct and the calls occur correctly. But this requires an edit apple.cpp and I think that this is not quite the right option. What am I missing, why doesn't it work right after cythonizing?