order that destructors get called?
Brendan Miller
like this:
{
Foo foo;
Bar bar;
// calls Bar::~Bar()
// calls Foo::~Foo()
}
I'm writing a ctypes wrapper for some native code, and I need to manage some
memory. I'm wrapping the memory in a python class that deletes the underlying
memory when the python class's reference count hits zero.
When doing this, I noticed some odd behaviour. I had code like this:
def delete_my_resource(res):
# deletes res
class MyClass(object):
def __del__(self):
delete_my_resource(self.res)
o = MyClass()
What happens is that as the program shuts down, delete_my_resource is released
*before* o is released. So when __del__ get called, delete_my_resource is now
None.
Obviously, MyClass needs to hang onto a reference to delete_my_resource.
What I'm wondering is if there's any documented order that reference counts
get decremented when a module is released or when a program terminates.
What I would expect is "reverse order of definition" but obviously that's not
the case.
Brendan
Stephen Hansen
> When doing this, I noticed some odd behaviour. I had code like this:
>
> def delete_my_resource(res):
> # deletes res
>
> class MyClass(object):
> def __del__(self):
> delete_my_resource(self.res)
>
> o = MyClass()
>
> What happens is that as the program shuts down, delete_my_resource is released
> *before* o is released. So when __del__ get called, delete_my_resource is now
> None.
The first thing Python does when shutting down is go and set the
module-level value of any names to None; this may or may not cause
those objects which were previously named such to be destroyed,
depending on if it drops their reference count to 0.
So if you need to call something in __del__, be sure to save its
reference for later, so that when __del__ gets called, you can be sure
the things you need are still alive. Perhaps on MyClass, in its
__init__, or some such.
> What I'm wondering is if there's any documented order that reference counts
> get decremented when a module is released or when a program terminates.
>
> What I would expect is "reverse order of definition" but obviously that's not
> the case.
AFAIR, every top level name gets set to None first; this causes many
things to get recycled. There's no order beyond that, though.
Namespaces are dictionaries, and dictionaries are unordered. So you
can't really infer any sort of order to the destruction: if you need
something to be alive when a certain __del__ is called, you have to
keep a reference to it.
--
--S
... p.s: change the ".invalid" to ".com" in email address to reply privately.
Gabriel Genellina
escribió:
> I'm used to C++ where destrcutors get called in reverse order of
> construction
> like this:
>
> {
> Foo foo;
> Bar bar;
>
> // calls Bar::~Bar()
> // calls Foo::~Foo()
> }
That behavior is explicitly guaranteed by the C++ language. Python does
not have such guarantees -- destructors may be delayed an arbitrary amount
of time, or even not called at all.
In contrast, Python does have a `try/finally` construct, and the `with`
statement. If Foo and Bar implement adequate __enter__ and __exit__
methods, the above code would become:
with Foo() as foo:
with Bar() as bar:
# do something
On older Python versions it is more verbose:
foo = Foo()
try:
bar = Bar()
try:
# do something
finally:
bar.release_resources()
finally:
foo.release_resources()
> I'm writing a ctypes wrapper for some native code, and I need to manage
> some
> memory. I'm wrapping the memory in a python class that deletes the
> underlying
> memory when the python class's reference count hits zero.
If the Python object lifetime is tied to certain lexical scope (like the
foo,bar local variables in your C++ example) you may use `with` or
`finally` as above.
If some other object with a longer lifetime keeps a reference, see below.
> When doing this, I noticed some odd behaviour. I had code like this:
>
> def delete_my_resource(res):
> # deletes res
>
> class MyClass(object):
> def __del__(self):
> delete_my_resource(self.res)
>
> o = MyClass()
>
> What happens is that as the program shuts down, delete_my_resource is
> released
> *before* o is released. So when __del__ get called, delete_my_resource
> is now
> None.
Implementing __del__ is not always a good idea; among other things, the
garbage collector cannot break a cycle if any involved object contains a
__del__ method. [1]
If you still want to implement __del__, keep a reference to
delete_my_resource in the method itself:
def __del__(self,
delete_my_resource=delete_my_resource):
delete_my_resource(self.res)
(and do the same with any global name that delete_my_resource itself may
reference).
The best approach is to store a weak reference to foo and bar somewhere;
weak references are notified right before the referent is destroyed. [4]
And last, if you want to release something when the program terminates,
you may use the atexit module.
> What I'm wondering is if there's any documented order that reference
> counts
> get decremented when a module is released or when a program terminates.
Not much, as Stephen Hansen already told you; but see the comments in
PyImport_Cleanup function in import.c [2] and in _PyModule_Clear in
moduleobject.c [3]
Standard disclaimer: these undocumented details only apply to the current
version of CPython, may change in future releases, and are not applicable
at all to other implementations. So it's not a good idea to rely on this
behavior.
[1] http://docs.python.org/reference/datamodel.html#object.__del__
[2] http://svn.python.org/view/python/trunk/Python/import.c?view=markup
[3]
http://svn.python.org/view/python/trunk/Objects/moduleobject.c?view=markup
[4] http://docs.python.org/library/weakref.html
--
Gabriel Genellina
Brendan Miller
In my case my resource isn't bound to a lexical scope, but the:
def __del__(self,
delete_my_resource=delete_my_resource):
pattern works quite well. I've made sure to prevent my class from
being part of a circular reference, so that the __del__ shouldn't be
an issue.
Brendan
Alf P. Steinbach
It may work but is unfortunately not guaranteed to work.
Use a 'with' statement where you need guaranteed cleanup.
Cheers & hth.,
- Alf