Size of cdef class with a PyObject* attribute

[Oscar Benjamin]

Hi all,

I am trying to understand why a cdef class grows from 32 bytes to 48
bytes when I change an attribute from unsigned long to PyObject*.

I have a cdef class that is approximately like this and has size 32 bytes:

cdef class int_mod1:
cdef unsigned long long val
cdef unsigned long long mod

I wanted to change it to be like this where instead of a 64 bit
integer mod it has a pointer to an instance of another cdef class that
holds the mod (and some other things):

cdef class int_mod2_ctx:
cdef unsigned long long mod

cdef class int_mod2:
cdef unsigned long long val
cdef int_mod2_ctx ctx

Now I'm finding that it has 48 bytes instead of 32:

In [3]: sys.getsizeof(int_mod1())
Out[3]: 32

In [4]: sys.getsizeof(int_mod2())
Out[4]: 48

I understand why the first is 32 bytes because it is 16 bytes for
general PyObject_HEAD overhead + 8 bytes for val + 8 bytes for mod so
16+8+8=32 bytes.

In the second case I have swapped an 8 byte integer for an 8 byte
pointer so I expected the size to stay the same but it has gone up to
48 bytes. Is this some sort of alignment issue that I don't
understand? Does an 8 byte pointer need 16 byte alignment here for
some reason?

The generated C code for the structs looks like:

struct __pyx_obj_5flint_5types_4nmod_int_mod1 {
PyObject_HEAD
unsigned PY_LONG_LONG val;
unsigned PY_LONG_LONG mod;
};

struct __pyx_obj_5flint_5types_4nmod_int_mod2_ctx {
PyObject_HEAD
unsigned PY_LONG_LONG mod;
};

struct __pyx_obj_5flint_5types_4nmod_int_mod2 {
PyObject_HEAD
unsigned PY_LONG_LONG val;
struct __pyx_obj_5flint_5types_4nmod_int_mod2_ctx *ctx;
};

Also the type object size parts are:

sizeof(struct __pyx_obj_5flint_5types_4nmod_int_mod1), /*tp_basicsize*/
0, /*tp_itemsize*/

sizeof(struct __pyx_obj_5flint_5types_4nmod_int_mod2), /*tp_basicsize*/
0, /*tp_itemsize*/

As I understand it that is where sys.getsizeof gets the size from.

If I do the same thing directly in C then I get 32 bytes in both cases:

#include <stdio.h>

struct TypeObject {
int thing;
};

struct int_mod1 {
unsigned long long refcount;
struct TypeObject *type;
unsigned long long val;
unsigned long long mod;
};

struct int_mod2_ctx {
unsigned long long refcount;
struct TypeObject *type;
unsigned long long mod;
};

struct int_mod2 {
unsigned long long refcount;
struct TypeObject *type;
unsigned long long val;
struct int_mod2_ctx *ctx;
};

int main() {
printf("sizeof(int_mod1) = %lu\n", sizeof(struct int_mod1));
printf("sizeof(int_mod2) = %lu\n", sizeof(struct int_mod2));
return 0;
}

$ ./a.out
sizeof(int_mod1) = 32
sizeof(int_mod2) = 32

Is this something to do with cyclic GC or something? Does CPython or
Cython put something in the code that forces some different alignment
rules?

--
Oscar

[Salih Ahmed]

The increase in size from 32 bytes to 48 bytes when you switch from using an unsigned long long to a PyObject* in your cdef class in Cython is primarily due to the addition of fields required for Python’s cyclic garbage collector (GC).

Detailed Explanation:

1. Structure Layout in C and Cython: In C, when you define the structs directly, the size of int_mod1 and int_mod2 structs is 32 bytes. This is expected because:

   • unsigned long long is 8 bytes.
   • The struct has two unsigned long long types, making it 16 bytes.
   • The overhead for refcount and type (assuming a 64-bit architecture, where pointers are 8 bytes each) adds another 16 bytes.
   • So, sizeof(int_mod1) = 32 bytes and sizeof(int_mod2) = 32 bytes in C, which aligns with your observation.

2. Cython Classes and Python's Memory Management: In Cython, when you declare a cdef class, it is a Python object that participates in Python's memory management system. Python objects have additional overhead due to:

   • The PyObject_HEAD, which includes reference counting and type information.
   • Fields necessary for Python's cyclic garbage collector (GC) if the object type might participate in reference cycles.

3. Cyclic Garbage Collector (GC) Overhead: When you change an attribute from unsigned long long to PyObject*, it introduces the potential for Python reference cycles because a PyObject* could reference other Python objects, creating a cycle that the GC must handle.

   To manage this, Python’s GC adds additional overhead:

   • PyGC_Head is an additional structure used for objects tracked by the garbage collector. This structure adds 16 bytes on a typical 64-bit system (it contains three pointers: gc.gc_next, gc.gc_prev, and gc.gc_refs).
   • Because int_mod2 now contains a PyObject* pointer (int_mod2_ctx), which may involve Python object references and potential cycles, it is automatically placed under GC tracking.

4. Alignment Considerations: While alignment can play a role in struct sizes, the key factor here is not alignment but the presence of additional GC fields. The structures’ sizes in pure C are controlled by the size of their members and the alignment requirements, but Cython-generated Python objects have added GC tracking, leading to the observed size increase.

5. Size Calculation Breakdown:

   • int_mod1:
     • PyObject_HEAD (16 bytes) + unsigned long long val (8 bytes) + unsigned long long mod (8 bytes) = 32 bytes.
   • int_mod2:
     • PyObject_HEAD (16 bytes) + unsigned long long val (8 bytes) + PyObject* ctx (8 bytes) + GC overhead (16 bytes) = 48 bytes.

Conclusion:

The reason int_mod2 is 48 bytes instead of 32 bytes is due to the inclusion of garbage collector overhead when you change an attribute to a PyObject*. This addition is required to properly manage objects that could participate in reference cycles, which pure C structs do not need to handle. This overhead increases the size of the Cython class, as reflected by sys.getsizeof.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/CAHVvXxSJu2nj%2BdL28%2BQDeD9QAA_Uhmsfuj2OU6%2BZTc%2BvpcpeeQ%40mail.gmail.com.

[Oscar Benjamin]

Thanks Salih, that's a great explanation and clarifies the main point for me.

I have two follow up questions though :)

You said:

> PyGC_Head is an additional structure used for objects tracked by the garbage collector. This structure adds 16 bytes on a typical 64-bit system (it contains three pointers: gc.gc_next, gc.gc_prev, and gc.gc_refs).

How do we get these three pointers into 16 bytes? Maybe gc.gc_refs is
not actually stored?

My other question is how is it that I can't see this overhead in the
struct definitions? Is it not the case that sys.getsizeof returns
tp_basicsize?

Thanks again,
Oscar

> To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/CA%2B7veea6K9hhfUm4fXMA6tvREhptE_k4v9x8sZB0eUBUs665dQ%40mail.gmail.com.

[Salih Ahmed]

1. Understanding PyGC_Head Structure Size

The PyGC_Head structure is used by CPython to manage objects that are tracked by the cyclic garbage collector (GC). This structure does indeed contain three pointers, but let's understand how this fits into 16 bytes on a typical 64-bit system.

Structure of PyGC_Head

On a 64-bit system, each pointer typically takes 8 bytes. The PyGC_Head structure is defined as follows (simplified for explanation):

c
typedef struct _gc_head {
    struct _gc_head *gc_next;   // Pointer to the next GC-tracked object
    struct _gc_head *gc_prev;   // Pointer to the previous GC-tracked object
    Py_ssize_t gc_refs;         // Reference count (an integer type)
} PyGC_Head;

Now, on a 64-bit system:

• gc_next is 8 bytes (pointer)
• gc_prev is 8 bytes (pointer)
• gc_refs is typically a Py_ssize_t, which is also 8 bytes on a 64-bit system

So, why does this fit into 16 bytes?

Compaction Using Bit Fields

The trick lies in how CPython manages memory for small objects. The gc_refs field does not necessarily use a full 8 bytes. Instead:

• CPython uses a technique called bit-packing for the gc_refs field, particularly when storing reference counts or special marker values (like flags for GC stages). This allows all three members to fit into 16 bytes effectively.

Actual Memory Layout

The actual memory layout might look like this:

• The first 8 bytes for gc_next (pointer).
• The next 8 bytes are shared:
  • 4 bytes for gc_prev (pointer).
  • 4 bytes for gc_refs (using the lower 4 bytes for gc_refs and higher bytes for flags or markers).

Thus, while logically there are three fields, in practice, CPython's internal structures might compact these into a smaller memory footprint, fitting all necessary data into 16 bytes on systems where pointer size is 8 bytes.

2. Why Can't You See PyGC_Head in Struct Definitions?

PyGC_Head is not directly visible in the struct definitions of Python objects like int_mod1 or int_mod2 because:

• Separation of Concerns: The garbage collector structures (PyGC_Head) are typically managed separately from the user-defined struct in the source code. When an object is allocated in a garbage-collected pool, the GC metadata is stored "before" the actual memory used by the Python object.

• Memory Layout: The PyGC_Head is usually allocated just before the Python object itself in memory. This means that while sys.getsizeof() gives you the size of the Python object (based on its tp_basicsize), it does not include the GC header in its calculation.

3. Relation Between sys.getsizeof() and tp_basicsize

• tp_basicsize: This field in the PyTypeObject structure represents the size of the object itself (i.e., the size of the user-defined struct including its members but not including any GC-related overhead).

• sys.getsizeof(): This function returns the memory size of the Python object as calculated by tp_basicsize. It does not account for additional overhead like PyGC_Head because that memory is considered part of the garbage collector's internal management and not the object itself.

Why You Don’t See PyGC_Head in sys.getsizeof() Output:

• sys.getsizeof() is designed to measure the actual size of the object data structure and does not include the overhead for garbage collection metadata (PyGC_Head). This GC metadata is a separate concern and managed outside the object's tp_basicsize.

Example Memory Layout of a GC-Tracked Object:

If we visualize the memory layout:

python
[ PyGC_Head (16 bytes) | Python object (tp_basicsize) ]

• PyGC_Head (16 bytes): Contains the garbage collector management information.
• Python object: Contains the user-defined fields (as described by tp_basicsize).

Conclusion

• PyGC_Head compacts three pointers into 16 bytes using bit-packing and efficient memory alignment techniques.
• sys.getsizeof() returns the size of the Python object itself (tp_basicsize), excluding any garbage collection overhead such as PyGC_Head.
• The garbage collection overhead is managed separately, and its size is not reflected in the object’s tp_basicsize.

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/CAHVvXxRBoLaq72Q3rejD22AC%3DAxY%3D0VwYtVwG7%3DriROSD9QHww%40mail.gmail.com.

[Salih Ahmed]

I hope the explanation is comprehensive and detailed.

thanks

[da-woods]

On 23/08/2024 22:26, Oscar Benjamin wrote:

My other question is how is it that I can't see this overhead in the
struct definitions? Is it not the case that sys.getsizeof returns
tp_basicsize?

Hi Oscar,

sys.getsizeof does also add in any extra "pre-header" storage. That's the GC head and also managed dict and managed weakref (although Cython doesn't use the latter two):

https://github.com/python/cpython/blob/5ff638f1b53587b9f912a18fc776a2a141fd7bed/Python/sysmodule.c#L1917

For reference the current definition of PyGC_Head is at

https://github.com/python/cpython/blob/5ff638f1b53587b9f912a18fc776a2a141fd7bed/Include/internal/pycore_gc.h#L20

It is indeed 16 bits, although not quite in the layout that Salih says.

[Oscar Benjamin]

On Sat, 24 Aug 2024 at 09:30, da-woods <dw-...@d-woods.co.uk> wrote:
>
> On 23/08/2024 22:26, Oscar Benjamin wrote:
>
> My other question is how is it that I can't see this overhead in the
> struct definitions? Is it not the case that sys.getsizeof returns
> tp_basicsize?
>
> Hi Oscar,
>
> sys.getsizeof does also add in any extra "pre-header" storage. That's the GC head and also managed dict and managed weakref (although Cython doesn't use the latter two):

Okay, thanks both. That makes sense now so what happens is:

If I add any PyObject* to the struct then Cython somehow informs
CPython that this type needs GC-tracking. Then the CPython memory
management system prepends a 16 byte structure that holds my object in
a doubly linked list that is used for cyclic GC management. Then
sys.getsizeof adds those 16 bytes to tp_basicsize when it reports the
size of the object. Hence we have 16 bytes gc + 16 bytes head + 8
bytes unsigned + 8 bytes pointer and 16+16+8+8 = 48 bytes.

In the end I have decided not to go ahead with adding this PyObject*
not directly because of these 16 bytes but because it brought a 50%
slowdown in nmod.__mul__ (the actual class I am working on) in a
macro-benchmark. I explained my thoughts and timings here:
https://github.com/flintlib/python-flint/pull/179#issuecomment-2307162566

In context what I actually have is a class like this representing an
integer modulo another like 3 mod 7:

cdef class nmod(flint_scalar):
cdef unsigned long val
cdef nmod_t mod

def __mul__(s, t):
cdef nmod r, s2
cdef unsigned long val
s2 = s
if any_as_nmod(&val, t, s2.mod):
r = nmod.__new__(nmod)
r.mod = s2.mod
r.val = nmod_mul(val, s2.val, s2.mod)
return r
return NotImplemented

Here nmod_t is a 24 byte C data structure representing the integer
modulus along with a couple of precomputed things. If you were working
in C then the nmod_t struct would probably be a statically allocated
global variable that is passed to all functions like nmod_mul,
nmod_add etc. Here instead we attach the nmod_t somewhat redundantly
to every nmod instance and copy it over each time a new instance is
created.

I wanted to try to share the nmod_t data structure and also attach
additional information to it like precomputing and storing a boolean
representing whether or not the modulus is prime. The reason I want
this is because various operations in Flint will abort the process in
some cases if the modulus is not prime (e.g. if division does not
exist) and I would prefer to raise a Python exception instead.
Computing whether or not the modulus is prime is much more expensive
than all of the other elementary operations here so I wanted to store
that information somewhere.

I tried replacing the mod attribute with a Cython nmod_ctx class that
could hold the nmod_t as well as a boolean is_prime but it brought big
slowdowns. I was expecting some possible slowdown just because this
would mean going through a pointer to get to the nmod_t but I think
the actual slowdown seen was much bigger than that. My suspicion is
that this is to do with INCREF/DECREF but now I also imagine that
cyclic GC plays a part as well. This is how it looks in C to copy the
ctx PyObject* from one nmod to another:

/* "flint/types/nmod.pyx":403
* raise ValueError("cannot coerce integers mod n with different n")
* r = nmod.__new__(nmod)
* r.ctx = s.ctx # <<<<<<<<<<<<<<
* r.val = nmod_mul(s.val, t.val, s.ctx.mod)
* return r
*/
__pyx_t_2 = ((PyObject *)__pyx_v_s->ctx);
__Pyx_INCREF(__pyx_t_2);
__Pyx_GIVEREF(__pyx_t_2);
__Pyx_GOTREF((PyObject *)__pyx_v_r->ctx);
__Pyx_DECREF((PyObject *)__pyx_v_r->ctx);
__pyx_v_r->ctx = ((struct __pyx_obj_5flint_5types_4nmod_nmod_ctx *)__pyx_t_2);
__pyx_t_2 = 0;

I'm not sure what all of those macros are doing. I imagined that this
operation just needs one INCREF, one DECREF and one pointer copy like:

Py_INCREF(new);
Py_DECREF(old);
__pyx_v_r->ctx = new;

Maybe some of the macros are actually no-ops and the compiler reduces
it down a bit?

I tried various ways of rewriting nmod.__mul__ and using cdef inline
functions etc. Nothing I tried could reduce the measured overhead of
having a PyObject* down to an acceptable level.

I considered other options like attaching a static linked list of
nmod_t to the nmod class so that the instances could hold raw C
pointers to them but then we would never deallocate them. Maybe that
is fine because I doubt someone will use large numbers of different
moduli but if they did then we would have a memory leak...

In the end though the one thing I don't quite understand is how
int/PyLong is just a bit faster than I can achieve with anything I
have tried:

In [1]: a = 5 # small ints cached

In [2]: %timeit a*a
27.1 ns ± 0.167 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

In [3]: a = 1000 # not cached

In [4]: %timeit a*a
65.4 ns ± 0.863 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

In [6]: import gmpy2

In [7]: a = gmpy2.mpz(1000)

In [8]: %timeit a*a
74 ns ± 0.399 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

In [9]: import flint

In [10]: a = flint.nmod(1000, 10000)

In [11]: %timeit a*a
76.8 ns ± 1.44 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

Both gmpy2.mpz and flint.nmod are about 15% slower than int in this
micro benchmark. I feel like I have tried every possible way of
rewriting the code here but I have never managed to match int for
speed even with simple stripped down implementations of nmod. I don't
know if there is just some unavoidable overhead of using Cython here
but I think that gmpy2 actually uses the C API directly rather than
Cython and I am sure that they will have tried to micro-optimise this
as well.

Maybe PyLong just has some built-in advantage that is somehow
unavailable for third party types?

The difference seen here is 10ns which on this machine is probably
about 5 arithmetic CPU instructions so it may seem excessive to focus
on this but at the macro scale these differences are measurable in the
runtime of meaningful operations. For most types in python-flint I
would not worry about this but this one type represents very small
objects (8 bytes in C even if 48 in Python) and is absolutely used in
innermost loops in downstream code. Just having a PyObject* in the
nmod seems to bring an unavoidable 50% slowdown in the time to invert
a 100x100 matrix for example: the time goes from 10ms to 15ms so the
nanoseconds do add up.

--
Oscar

[da-woods]

Ignoring the text below (for now) in favour of one more potentially
useful information:

you can manually disable gc on a class with @cython.no_gc.
https://cython.readthedocs.io/en/latest/src/userguide/extension_types.html#disabling-cyclic-garbage-collection
That might be appropriate for your type because it can't participate in
GC cycles because what it points to is nogc.

In principle Cython might actually be able to work that out itself (it
can in some cases but clearly not in this one).

[da-woods]

Responding to a bit more of it:

On 24/08/2024 12:54, Oscar Benjamin wrote:

  /* "flint/types/nmod.pyx":403
 *             raise ValueError("cannot coerce integers mod n with different n")
 *         r = nmod.__new__(nmod)
 *         r.ctx = s.ctx             # <<<<<<<<<<<<<<
 *         r.val = nmod_mul(s.val, t.val, s.ctx.mod)
 *         return r
 */
  __pyx_t_2 = ((PyObject *)__pyx_v_s->ctx);
  __Pyx_INCREF(__pyx_t_2);
  __Pyx_GIVEREF(__pyx_t_2);
  __Pyx_GOTREF((PyObject *)__pyx_v_r->ctx);
  __Pyx_DECREF((PyObject *)__pyx_v_r->ctx);
  __pyx_v_r->ctx = ((struct __pyx_obj_5flint_5types_4nmod_nmod_ctx *)__pyx_t_2);
  __pyx_t_2 = 0;

I'm not sure what all of those macros are doing. I imagined that this
operation just needs one INCREF, one DECREF and one pointer copy like:

Py_INCREF(new);
Py_DECREF(old);
__pyx_v_r->ctx = new;

Maybe some of the macros are actually no-ops and the compiler reduces
it down a bit?

Yes this is right. These macros are no-ops.

Thet're used in our test-suite and can be manually enabled (by defining the C macro CYTHON_REFNANNY) in order to do some sanity checking of the reference counting. If you haven't done that deliberately then they'll just get removed by the C preprocessor.

So __PYX_GOTREF and __PYX_GIVEREF aren't worth worrying about.

[Oscar Benjamin]

On Sat, 24 Aug 2024 at 12:59, da-woods <dw-...@d-woods.co.uk> wrote:
>
> you can manually disable gc on a class with @cython.no_gc.
> https://cython.readthedocs.io/en/latest/src/userguide/extension_types.html#disabling-cyclic-garbage-collection
> That might be appropriate for your type because it can't participate in
> GC cycles because what it points to is nogc.

Thanks, that actually makes a big difference, possibly all the
difference. Timings for a macro benchmark are:

Master branch:
In [7]: %timeit dM_sympy_dense.rref()
81.6 ms ± 694 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

PR branch:
In [9]: %timeit dM_sympy_dense.rref()
112 ms ± 2.9 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

PR now with no_gc:
In [5]: %timeit dM_sympy_dense.rref()
91.6 ms ± 203 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

So I was finding a 30% slowdown mostly as a result of using a
PyObject* in the cdef class but with @cython.no_gc I now only see a
10% slowdown.

It is possible that the remaining 10% has a different cause since I
have changed many things... I'll have to clean up all the code before
a proper comparison can be made.

This micro benchmark shows no slowdown now that I have added no_gc:

In [4]: a = flint.nmod(1000, 10000)

In [5]: %timeit a*a
78.1 ns ± 2.76 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

It is still a lot slower than int though:

In [6]: a = 1000

In [7]: %timeit a*a
49 ns ± 1.89 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

That's about 50% slower... I said 15% slower before just because that
was the timing I found when measuring this as I wrote my previous
message but the times vary from run to run (only if I restart ipython
though?). Relatively speaking, nmod ranges 15-50% slower than int. I
have not managed to make anything in Cython that can match the speed
of int/float for a*a.

Maybe c_api_binop_methods is relevant...

> In principle Cython might actually be able to work that out itself (it
> can in some cases but clearly not in this one).

If my measurements are anything to go by then some applications would
see a noticeable speedup from this. I'm seeing 20% speedup just from
adding no_gc but these are very small objects and I am creating
millions of them which may not be a typical use case.

I didn't think about GC here at first because it seemed obvious to me
that it wasn't needed...

--
Oscar

[Salih Ahmed]

1. Why Can't You See PyGC_Head in the Struct Definitions?

The PyGC_Head is not visible in the struct definitions of your Python objects because it is part of the internal memory management and garbage collection mechanisms in CPython, not the Python object itself. Here’s a more detailed explanation:

• Separation of Object Data and GC Metadata: The PyGC_Head is a separate header used by CPython's garbage collector to manage objects that can participate in reference cycles. This header is stored before the actual memory of the Python object in memory but is not part of the object’s own data structure as defined in the code.

• Object Layout in Memory: In memory, an object tracked by the garbage collector is stored with the PyGC_Head immediately before the actual object data. This means the PyGC_Head is located in memory adjacent to the Python object, but it is not defined in the struct of the Python object itself.

Here’s a simplified representation:

arduino
[ PyGC_Head | PyObject_HEAD | User-defined struct fields... ]

In this layout:

• PyGC_Head is the GC-related metadata (not shown in the struct).
• PyObject_HEAD is a part of the Python object structure itself, visible in your struct definitions.
• User-defined struct fields are your custom fields (like val and mod).

2. Does sys.getsizeof() Return tp_basicsize?

• sys.getsizeof() Returns tp_basicsize: Yes, sys.getsizeof() primarily returns the size of the object as defined by its tp_basicsize. This includes the size of the object’s internal data but not the GC metadata (PyGC_Head).

  • tp_basicsize: This field in the PyTypeObject struct represents the size of the Python object’s data structure. It accounts for the object's fields as declared in the object’s struct (like PyObject_HEAD and any custom fields).

  • sys.getsizeof() Behavior: When you call sys.getsizeof() on an object, it returns the size defined by tp_basicsize plus some extra for Python object-specific overhead, if any, but not the PyGC_Head size. This is because sys.getsizeof() is meant to measure the memory directly allocated for the Python object, not the additional GC headers or other overhead that CPython might manage separately.

Why PyGC_Head is Not Visible in tp_basicsize or sys.getsizeof():

1. Invisible GC Overhead: The PyGC_Head is part of Python's internal garbage collection system. It is not part of the object's tp_basicsize because tp_basicsize only includes the object-specific data, not any memory management or GC metadata.

2. Memory Management Separation: CPython separates object data from garbage collection and memory management metadata. This separation allows objects to be managed more flexibly by the GC system without affecting the size calculations for user-defined Python objects.

3. Consistency in sys.getsizeof(): sys.getsizeof() reports the size of the object from the perspective of Python code. Including GC overhead would make this function less consistent, as different objects might have different GC requirements not directly tied to their data layout.

Summary

• sys.getsizeof() reflects the size of a Python object based on its tp_basicsize, which does not include the PyGC_Head.
• PyGC_Head is used by CPython's GC but is stored separately in memory, outside the user-visible struct definitions.
• Memory Layout in CPython ensures that while objects are managed by the GC, their internal data structure sizes (as returned by sys.getsizeof()) remain consistent and separate from GC metadata.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/c6d1f924-4b28-40e3-a6db-65c2e695fcf8%40d-woods.co.uk.

[Oscar Benjamin]

Hi Salih,

I was being polite about this before but you are posting the output of
ChatGPT and it is not really helpful to do that. I see this happening
in many forums now such as mailing lists, GitHub issues etc e.g.:
https://github.com/numpy/numpy/issues/27054#issuecomment-2256239692

I am not sure what exactly it is that prompts people to do this but I
think that it should be considered bad etiquette and it is usually
unhelpful.

Firstly if you think that the output of ChatGPT is helpful then you
should at least clearly state that it is the output of ChatGPT rather
than posting it as if the words come directly from you. Note also that
posting the output of an LLM directly actually makes me wonder if you
are even a human at all rather than a bot.

Secondly you should clarify whether or not you know that the
information is actually correct. It is quite common for ChatGPT to
make up bogus answers like the hallucination that you can put 3
pointers into 16 bytes that you posted above. It even doubled down on
that point lying further when you apparently questioned it!

I just tried posting my OP into ChatGPT (GPT 4o - the latest non-free
version) and it correctly says that the extra space is "likely due to
the cyclic garbage collection" but still makes a number of false
claims:

> This results in an increase in the tp_basicsize of the object, causing the sys.getsizeof() output to increase from 32 to 48 bytes.

Not true (tp_basicsize is unchanged).

> If you want to avoid this overhead and you're certain you don't need GC for your class, you could consider using cdef struct or cdef class with nogil

Also not true (should be no_gc not no_gil). Let me follow this up with
another question to ChatGPT.

Me: I thought that nogil was to do with the GIL. Why would that affect
cyclic GC?

ChatGPT:

> You're right! The nogil directive in Cython is primarily about the Global Interpreter Lock (GIL) and allows Cython code to release the GIL, which can improve performance in multi-threaded Cython applications. It is not directly related to cyclic garbage collection (GC).
>
>I apologize for the confusion in my earlier response. Let's clarify this:

I then get many paragraphs of wishy washy stuff. It ends with:

> In summary, the increase in size you're observing is purely due to Python's need to manage cyclic garbage collection and not related to the GIL or the nogil directive.

Okay but still no mention of no_gc which is what it should have said
rather than nogil.

Also your ChatGPT output has repeatedly asserted a false claim
concerning what was my main point of confusion at the outset:

> sys.getsizeof() Returns tp_basicsize: Yes, sys.getsizeof() primarily returns the size of the object as defined by its tp_basicsize. This includes the size of the object’s internal data but not the GC metadata (PyGC_Head).

This is demonstrably false as shown by the link that Da Woods posted
to the code for sys.getsizeof and also by my tests with and without
the @no_gc decorator:

https://github.com/python/cpython/blob/5ff638f1b53587b9f912a18fc776a2a141fd7bed/Python/sysmodule.c#L1917

If I wanted to get a possibly useful but possibly hallucinated answer
from ChatGPT I could do that myself rather than sending an email to a
mailing list. That was not what I wanted though and actually I am
happy to wait so that I can get a response from a real human who
actually knows what they are talking about.

Thirdly, ChatGPT is probably not even the best of the LLMs to use for
this. I showed this email thread to my student who likes to spend lots
of time talking to LLMs about programming. He said that only noobs use
ChatGPT and you should obviously use Claude instead. He put my
original email from this thread into Claude and it gave far better
results including:

- correctly explaining everything
- not making any false claims
- immediately suggesting that I should use the no_gc decorator which
so far seems like the actual fix for the problems I was having but
somehow eludes ChatGPT.

My student thinks it is quaint that I would email an old-school
mailing list and wait hours or days for a real human to reply when
Claude can answer the question immediately...

My retort to him is that I quite like being able to talk to actual
expert users and maintainers of Cython and that in doing so we all
become more aware of issues, how Cython is used etc. In this case it
was identified that there can also be a potential improvement in
Cython itself that it could be able to identify that my classes could
be no_gc without me needing to specify that explicitly. If I had only
spoken to Claude then the Cython maintainers would not have observed
that this is something that would make an important difference for
some users.

There can be reasonable ways to use ChatGPT or Claude as my student
does such as:

You could ask the LLM a question and then go and verify the answers
(the LLM may even help you with verifying them). Then once you have
verified the information you could post it to a mailing list as an
answer to someone's question.

I know some people use ChatGPT to help with writing especially in
non-native languages, sort of like "hey ChatGPT can you translate this
to English for me?" or "can you correct the grammar in this
paragraph?" or even "can you rewrite this Python code in C?". I think
that kind of use can be fine without needing to publicly state that
you have used an LLM but to do it properly you would in all cases need
to edit the output a bit rather than paste it directly somewhere.

I think that LLMs are more suitable if you stick to using them for
*language* tasks (literally what they are designed for) rather than
trying to ask them factual questions. Even then though I think ChatGPT
is not good for generating natural language if I am writing something
myself because I don't like its tone and it makes everything too
waffly and imprecise.

In any case posting bogus unverified information from a tool that
frequently hallucinates incorrect information and even lies further
when questioned is not a useful way to contribute on a mailing list.

--
Oscar

> To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/CA%2B7veeaW8guQ8U8e0qqSyzKSsN73wnzuKLhMssT1c8e9YXPN8g%40mail.gmail.com.

[Prakhar Goel]

Hi Salih,

Not sure why you insist on this. Da-wood's code snippet is quite conclusive that the GC head size is included in sys.getsizeof().

-- PG

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/CA%2B7veeaW8guQ8U8e0qqSyzKSsN73wnzuKLhMssT1c8e9YXPN8g%40mail.gmail.com.

[Oscar Benjamin]

On Sat, 24 Aug 2024 at 17:00, Oscar Benjamin <oscar.j....@gmail.com> wrote:
>
> On Sat, 24 Aug 2024 at 12:59, da-woods <dw-...@d-woods.co.uk> wrote:
> >
> > you can manually disable gc on a class with @cython.no_gc.
> > https://cython.readthedocs.io/en/latest/src/userguide/extension_types.html#disabling-cyclic-garbage-collection
> > That might be appropriate for your type because it can't participate in
> > GC cycles because what it points to is nogc.
>
> Thanks, that actually makes a big difference, possibly all the
> difference.

One thing I have noticed about this and it has bit me a few times now
is that it seems that if you add no_gc in the .pxd file but not the
.pyx file then it silently has no effect e.g.:

In [3]: cat t.pxd
cimport cython

@cython.no_gc
cdef class context:
cdef int val

@cython.no_gc
cdef class obj:
cdef context ctx

In [4]: cat t.pyx
cdef class context:
pass

cdef class obj:
pass

In [5]: import t
/home/oscar/.pyenv/versions/3.12.0/envs/python-flint-3.12/lib/python3.12/site-packages/Cython/Compiler/Main.py:373:
FutureWarning: Cython directive 'language_level' not set, using '3'
(Py3). This has changed from earlier releases! File:
/home/oscar/current/active/python-flint/t.pxd
tree = Parsing.p_module(s, pxd, full_module_name)

In [6]: import sys

In [7]: sys.getsizeof(t.obj())
Out[7]: 40

If I also add no_gc in the .pyx file then:

In [4]: cat t.pyx
cimport cython

@cython.no_gc
cdef class context:
pass

@cython.no_gc
cdef class obj:
pass

In [5]: import sys

In [6]: sys.getsizeof(t.obj())
Out[6]: 24

So it seems that in the .pxd file no_gc is ignored. I would prefer it
to give an error or a warning or otherwise to have the intended
effect. To me it seemed natural that the decorator belongs in the .pxd
right where the struct fields are defined.

--
Oscar

[da-woods]

On 29/08/2024 16:08, Oscar Benjamin wrote:

So it seems that in the .pxd file no_gc is ignored. I would prefer it
to give an error or a warning or otherwise to have the intended
effect.

Yeah, agree with that. It sounds like a bug/omission.

To me it seemed natural that the decorator belongs in the .pxd
right where the struct fields are defined.

Maybe... it possibly doesn't matter to users of the pxd file (not completely sure about this and how it interacts with derived classes) so it might make sense for it to be in the implementation instead.

But either way, what we're doing now doesn't sound ideal.

[Oscar Benjamin]

Earlier you observed that in principle Cython could automatically
detect that my example cdef classes can be no_gc because their fields
are no_gc. In general for that to work Cython would need to be able to
know if cdef classes in other modules are no_gc which I think means
that it would need to be able to see the no_gc decorator in the .pxd
files for those classes.

--
Oscar