Accessing internal members of PyObjects

[Johnny Wezel]

I want to get at an internal (private) member of an extension class, specifically the db member of struct pysqlite_Connection in the sqlite3 module defined in Modules/_sqlite/connection.h in the Python source.

Let's say I have

import sqlite3
import cffi
db = sqlite3.connect(':memory:')
ffi = cffi.FFI()
ffi.cdef("""
typedef struct sqlite3 sqlite3;
/* Part of struct I need. The rest is not interesting. */
typedef struct pysqlite_Connection {
    sqlite3 *db;
};
""")

Now, I figure, I have to get a handle like this:

db_handle = ffi.new_handle(db)

I tried this:

db_struct = ffi.cast('pysqlite_Connection *', db_handle)

But the address of db_struct is not the same as the original Python object in db. Trying to get db yields NULL.

So, how do I get to the pysqlite_Connection struct and more specifically, at the db member?

[Armin Rigo]

Hi,

On Wed, Oct 14, 2015 at 2:50 PM, Johnny Wezel <dev...@wezel.name> wrote:
> I want to get at an internal (private) member of an extension class,
> specifically the db member of struct pysqlite_Connection in the sqlite3
> module defined in Modules/_sqlite/connection.h in the Python source.

(...)

> Now, I figure, I have to get a handle like this:
>
> db_handle = ffi.new_handle(db)

No no no. A handle is not the original "PyObject *" cast to a "void
*". In fact, a core design decision of CFFI is to be independent on
the details of the Python implementation you are using. This means
CFFI doesn't give a way to do what you are trying to do. There are
probably horrible hacks that happen to work right now, but I'm not
going to expand on that. Maybe you can try using ctypes in this case;
notably, the type "ctypes.py_object", which has no equivalent in CFFI.

Or, more cleanly, we need a way to get that field officially. Maybe
you can consider using, instead of the built-in sqlite3 module of
CPython, the sqlite3-cffi variant (which is completely based on CFFI).
This variant is written in pure Python, so it has no strictly private
fields. It is the variant distributed with PyPy, but it works on top
of CPython too. See
https://bitbucket.org/pypy/pypy/src/default/lib_pypy/ (files
``_sqlite3*``).


A bientôt,

Armin.