__dealloc__ not being called at exit

[Henry Gomersall]

I've noticed that my __dealloc__ method is not being called when the
python code that imports the module exits.

I can explicitly delete the object (with del), at which point
__dealloc__ does get called, but I understood that this should happen at
exit.

How do I make sure that __dealloc__ is called at exit?

Cheers,

Henry

[Stefan Behnel]

Henry Gomersall, 06.07.2012 17:37:

> I've noticed that my __dealloc__ method is not being called when the
> python code that imports the module exits.
>
> I can explicitly delete the object (with del), at which point
> __dealloc__ does get called, but I understood that this should happen at
> exit.

It's always best to delete objects as soon as they are no longer needed.
That applies to both Python and Cython (and in general).

> How do I make sure that __dealloc__ is called at exit?

Where do you store the reference to the object? Is there a reference cycle?
Is it a module global variable? In the latter case, you have to enable the
cleanup code generation to clear the reference. See the
Cython.Compiler.Options module.

Stefan

[Robert Bradshaw]

Note that this is slightly unsafe, as the module can't be used after
it's been destructed, which is trickier than it sounds in case
references to this module (or instances of this module's classes) are
hanging around. But it's the best we can offer.

- Robert

[Henry Gomersall]

On Fri, 2012-07-06 at 22:32 -0700, Robert Bradshaw wrote:
> > Where do you store the reference to the object? Is there a reference
> cycle?
> > Is it a module global variable? In the latter case, you have to
> enable the
> > cleanup code generation to clear the reference. See the
> > Cython.Compiler.Options module.
>
> Note that this is slightly unsafe, as the module can't be used after
> it's been destructed, which is trickier than it sounds in case
> references to this module (or instances of this module's classes) are
> hanging around. But it's the best we can offer.

So my current workaround is to maintain state in a module global
dictionary, which keeps track of what needs to be tidied up at exit, and
then use an @atexit decorator to clean up when the module exits. Is
there a problem with this (apart from having state pulled out of my
classes)?

Cheers,

Henry

[Robert Bradshaw]

No, that should work. If you want to ensure state is cleaned up,
that's what you have to do.

- Robert

[Stefan Behnel]

Henry Gomersall, 07.07.2012 09:35:

You didn't answer my questions regarding the place where the reference to
this object is stored, i.e. who owns it. That leaves me unconvinced that
"at exit" is really the best time to clean them up. Why not do it earlier?

Stefan

[Henry Gomersall]

Apologies if I'm not being sufficiently explicit. In the original
problem example, the reference existed only in the .pyx file in the
which the object was defined and created (which was a single global
object), which, given your explanation about module global variables,
would explain why it's not being tidied up automatically. Is this what
you were asking?

Since this is a hardware interface, I really like the idea of having the
definitive state maintained by the module - in effect acting as a proxy
for the hardware's state.

Cheers,

Henry

[Stefan Behnel]

Henry Gomersall, 08.07.2012 10:06:

> On Sun, 2012-07-08 at 07:59 +0200, Stefan Behnel wrote:
>> Henry Gomersall, 07.07.2012 09:35:

>>> So my current workaround is to maintain state in a module global
>>> dictionary, which keeps track of what needs to be tidied up at
>>> exit, and then use an @atexit decorator to clean up when the module
>>> exits. Is there a problem with this (apart from having state pulled
>>> out of my classes)?
>>
>> You didn't answer my questions regarding the place where the
>> reference to this object is stored, i.e. who owns it. That leaves me
>> unconvinced that "at exit" is really the best time to clean them up.
>> Why not do it earlier?
>
> Apologies if I'm not being sufficiently explicit. In the original
> problem example, the reference existed only in the .pyx file in the
> which the object was defined and created (which was a single global
> object), which, given your explanation about module global variables,
> would explain why it's not being tidied up automatically. Is this what
> you were asking?

Yes, that explains it then. Module global variables are not cleaned up
automatically. In fact, before Python 3, there wasn't even a way to include
extension modules in garbage collection, and even with Py3, it's not
generally used. As Robert said, it's somewhat dangerous because it's not
defined in which order objects will be destroyed. Dependencies are not
taken into account and that means that an object that is still required for
cleaning up another one may already be dead at that point. Doesn't matter
that much in Python space (you'd get an exception if a reference is None
instead of what you expected), but a crash is a more likely result in C space.

> Since this is a hardware interface, I really like the idea of having the
> definitive state maintained by the module - in effect acting as a proxy
> for the hardware's state.

Ok, sure. That's a design decision.

Stefan

[Henry Gomersall]

Firstly, before I get the main body, I've resurrected an ancient thread
that I seems to have reared its head again.

The problem is that __dealloc__ is not being called, this is despite, as
far as I can tell, no global references or circular dependencies.

The advice from earlier seems to be no longer relevant to my code base
(the original issue, annoyingly, seems to have not been version
controlled). I've picked it up where it made sense...

The code is well contained. The object that I wish __dealloc__ to be
called on at delete exists once in one file, the main module. When the
main module exits, __dealloc__ is not called.

The cython specific bit of the code is found here:
https://github.com/hgomersall/PicoPy/blob/master/picopy/pico3k.pyx
(specifically, the Pico3k class) there is nothing that instantiates such
an object anywhere in the package.

gc.garbage outside of the test function is an empty list.

The class represents a piece of hardware, with each class holding a
handle. As I said, the workaround I have implemented is a global list of
handles that is populated when the hardware is opened, with an
@atexit.register decorated method that iterates through the list to
close them all at exit.

I _think_ the problem is Windows specific (though I can't test it under
Linux at the moment due to the current hardware I have that the class
represents being Windows only).

Any insight would be much appreciated.

Cheers,

Henry

[Henry Gomersall]

On 13/05/14 12:31, Henry Gomersall wrote:
> The problem is that __dealloc__ is not being called, this is despite,
> as far as I can tell, no global references or circular dependencies.

I've solved the problem. Contrary to my earlier assertions (and rather
embarrassingly) there was another reference. It works when this is
tidied up.

My apologies! (and thanks).

Henry