Segmentation faults - a theory

[Peter Williams]

As a newcomer (coming from PyGTK), I've been alarmed by the number of
segmentation faults that I've experienced using wxPython. These are the
only such problems that I have ever experienced using Python and based
on the workarounds I have used to dodge them (and the use of names such
as Destroy() for object methods) I have developed a theory as to what's
causing them.

I suspect that some of these Destroy() methods end up invoking memory
release within the C++ code. This can cause problems in Python in (at
least) two ways:

1. there are still (active) references to the released memory and if
these references are accessed a segmentation fault will result, and
2. Python's garbage collector doesn't notice that the memory has been
released and tries to release it itself when it notices that the object
is no longer accessible (i.e. normal procedure) resulting in a
segmentation fault.

I think that the only long term solution to this problem is to stop
invoking C++ functions that release memory (at least for memory that
Python knows about) and trust the Python garbage collector to do its job.

In short, using two competing memory management mechanisms at the same
time is asking for trouble. There are no explicit memory management
functions in Python for this reason.

Cheers
Peter
PS For the convenience of programmers, using a obj.Show(False) as a
substitute for obj.Destroy() would have the desired effect from a user
perspective.

[Werner]

Hi Peter,

On 16/09/2012 07:32, Peter Williams wrote:
> As a newcomer (coming from PyGTK), I've been alarmed by the number of
> segmentation faults that I've experienced using wxPython.

Do you have some sample code showing these seg faults?

It is pretty rare that I see seg faults and I am by no means an expert
programmer.

BTW, what platform and versions are you using.

Werner

[Andrea Gavana]

Hi,

On 16 September 2012 07:32, Peter Williams wrote:
> As a newcomer (coming from PyGTK), I've been alarmed by the number of
> segmentation faults that I've experienced using wxPython. These are the
> only such problems that I have ever experienced using Python and based on
> the workarounds I have used to dodge them (and the use of names such as
> Destroy() for object methods) I have developed a theory as to what's causing
> them.
>
> I suspect that some of these Destroy() methods end up invoking memory
> release within the C++ code. This can cause problems in Python in (at
> least) two ways:

The Destroy() method has its own niche applications for non-toplevel
windows, i.e. you do not use it all the time for all the widgets: if
you Destroy() a wx.Frame (for example when you close a window) all its
children are automatically destroyed for you.

>
> 1. there are still (active) references to the released memory and if these
> references are accessed a segmentation fault will result, and
> 2. Python's garbage collector doesn't notice that the memory has been
> released and tries to release it itself when it notices that the object is
> no longer accessible (i.e. normal procedure) resulting in a segmentation
> fault.

I have very little experience with Linux so I can't really comment on
these statements. You will probably have to show some minimal,
standalone sample application that demonstrates the problem.

> I think that the only long term solution to this problem is to stop invoking
> C++ functions that release memory (at least for memory that Python knows
> about) and trust the Python garbage collector to do its job.
>
> In short, using two competing memory management mechanisms at the same time
> is asking for trouble. There are no explicit memory management functions in
> Python for this reason.
>
> Cheers
> Peter
> PS For the convenience of programmers, using a obj.Show(False) as a
> substitute for obj.Destroy() would have the desired effect from a user
> perspective.

From a user perspective it may be doing what you think, but calling
obj.Show(False) simply hides the widget: it is still there, it's
occupying GDI objects (which on Windows can become a scarce resource)
and it can sometimes mess your your interface anyway.


Andrea.

"Imagination Is The Only Weapon In The War Against Reality."
http://xoomer.alice.it/infinity77/

# ------------------------------------------------------------- #
def ask_mailing_list_support(email):

if mention_platform_and_version() and include_sample_app():
send_message(email)
else:
install_malware()
erase_hard_drives()
# ------------------------------------------------------------- #

[Peter Williams]

On 09/17/2012 08:54 PM, Werner wrote:
> Hi Peter,
>
> On 16/09/2012 07:32, Peter Williams wrote:
>> As a newcomer (coming from PyGTK), I've been alarmed by the number of
>> segmentation faults that I've experienced using wxPython.
>
> Do you have some sample code showing these seg faults?

Try deleting an item with an embedded window from an UltimateListCtrl
for a start.

Also see changes that are being made to methods such as Sizers.Remove()
to avoid segmentation faults caused when it used to destroy the removed
object.

In other words, I'm not imagining it and what I suggest is happening in
an uncoordinated and ad hoc manner to fix segmentation faults. What I'm
suggesting is a coordinated approach to the problem to fix it for good.
The current approach is leading to lots of inconsistencies between
different parts of wxPython and also with the documentation e.g. the 2.8
documentation for Sizer.Remove() is now very inaccurate in that it
explicitly states that the removed object is destroyed but it no longer
is (as the previously mentioned bug fix for a segmentation fault).

>
> It is pretty rare that I see seg faults and I am by no means an expert
> programmer.

The kind of things that cause the problems would usually only occur in
dynamic GUIs where widgets are coming and going (i.e. adding/removing
controls rather than just enabling/disabling or showing/hiding them).
So unless your GUIs do that you're probably safe but that doesn't mean
that they shouldn't be fixed.

>
> BTW, what platform and versions are you using.

Linux but that shouldn't be relevant. If an operating system doesn't do
a segmentation fault in these circumstances then the outcome can be much
worse up to and including a complete system crash due to corrupted memory.

Peter

[Peter Williams]

On 09/17/2012 09:35 PM, Andrea Gavana wrote:
> Hi,
>
> On 16 September 2012 07:32, Peter Williams wrote:
>> As a newcomer (coming from PyGTK), I've been alarmed by the number of
>> segmentation faults that I've experienced using wxPython. These are the
>> only such problems that I have ever experienced using Python and based on
>> the workarounds I have used to dodge them (and the use of names such as
>> Destroy() for object methods) I have developed a theory as to what's causing
>> them.
>>
>> I suspect that some of these Destroy() methods end up invoking memory
>> release within the C++ code. This can cause problems in Python in (at
>> least) two ways:
>
> The Destroy() method has its own niche applications for non-toplevel
> windows, i.e. you do not use it all the time for all the widgets: if
> you Destroy() a wx.Frame (for example when you close a window) all its
> children are automatically destroyed for you.

That's where it's most dangerous.

>
>>
>> 1. there are still (active) references to the released memory and if these
>> references are accessed a segmentation fault will result, and
>> 2. Python's garbage collector doesn't notice that the memory has been
>> released and tries to release it itself when it notices that the object is
>> no longer accessible (i.e. normal procedure) resulting in a segmentation
>> fault.
>
> I have very little experience with Linux so I can't really comment on
> these statements. You will probably have to show some minimal,
> standalone sample application that demonstrates the problem.

See my other reply on this thread.

>
>> I think that the only long term solution to this problem is to stop invoking
>> C++ functions that release memory (at least for memory that Python knows
>> about) and trust the Python garbage collector to do its job.
>>
>> In short, using two competing memory management mechanisms at the same time
>> is asking for trouble. There are no explicit memory management functions in
>> Python for this reason.
>>
>> Cheers
>> Peter
>> PS For the convenience of programmers, using a obj.Show(False) as a
>> substitute for obj.Destroy() would have the desired effect from a user
>> perspective.
>
> From a user perspective it may be doing what you think, but calling
> obj.Show(False) simply hides the widget: it is still there, it's
> occupying GDI objects (which on Windows can become a scarce resource)
> and it can sometimes mess your your interface anyway.

Only while there is a variable or object field referencing it. As soon
as there are none then the Python garbage collector will dispose of it
and it should be stressed that this is the only time that it can be
safely disposed.

Peter

[Andrea Gavana]

Hi,

On 18 September 2012 01:34, Peter Williams wrote:
> On 09/17/2012 09:35 PM, Andrea Gavana wrote:
>>
>> Hi,
>>
>>
>> On 16 September 2012 07:32, Peter Williams wrote:
>>>
>>> As a newcomer (coming from PyGTK), I've been alarmed by the number of
>>> segmentation faults that I've experienced using wxPython. These are the
>>> only such problems that I have ever experienced using Python and based on
>>> the workarounds I have used to dodge them (and the use of names such as
>>> Destroy() for object methods) I have developed a theory as to what's
>>> causing
>>> them.
>>>
>>> I suspect that some of these Destroy() methods end up invoking memory
>>> release within the C++ code. This can cause problems in Python in (at
>>> least) two ways:
>>
>>
>> The Destroy() method has its own niche applications for non-toplevel
>> windows, i.e. you do not use it all the time for all the widgets: if
>> you Destroy() a wx.Frame (for example when you close a window) all its
>> children are automatically destroyed for you.
>
>
> That's where it's most dangerous.

I see it as "useful", and not particularly dangerous: I have never
seen a segmentation fault report because a wx.Frame destroyed its
children while closing.

>>>
>>> 1. there are still (active) references to the released memory and if
>>> these
>>> references are accessed a segmentation fault will result, and
>>> 2. Python's garbage collector doesn't notice that the memory has been
>>> released and tries to release it itself when it notices that the object
>>> is
>>> no longer accessible (i.e. normal procedure) resulting in a segmentation
>>> fault.
>>
>>
>> I have very little experience with Linux so I can't really comment on
>> these statements. You will probably have to show some minimal,
>> standalone sample application that demonstrates the problem.
>
>
> See my other reply on this thread.

I haven't used 2.8 since a veeeeery long time (2.9 is way much better
than 2.8 overall), but a couple of comments anyway:

- Allowing the users to rely on wx.Sizer.Remove to destroy its managed
windows was a big mistake, in my opinion: wx.Sizer is not a sub-class
of wx.Window and it cannot have any parent-child relationship with any
wxPython control. I am happy this behaviour has been removed in 2.9;

- The issues with UltimateListCtrl (or any other AGW widget) should
not be blamed on the library behaviour, but on the sloppy programmer
who created them (me, in this case). In general, however, when you
have an issue with a custom control (i.e., everything in wx.lib and
its sub-directories), you should do the following:

1) Get the latest SVN version of the problematic widget (as they can
evolve much faster than wxPython does over time);
2) Test again.

Most of them are wxPython-version-independent (to an extent), so they
can be used with 2.8 and 2.9 without modification. If the issue
persists, file a ticket to the wxTrac:

http://trac.wxwidgets.org/

Specifying the component (AGW in this case).

>>> I think that the only long term solution to this problem is to stop
>>> invoking
>>> C++ functions that release memory (at least for memory that Python knows
>>> about) and trust the Python garbage collector to do its job.
>>>
>>> In short, using two competing memory management mechanisms at the same
>>> time
>>> is asking for trouble. There are no explicit memory management functions
>>> in
>>> Python for this reason.
>>>
>>> Cheers
>>> Peter
>>> PS For the convenience of programmers, using a obj.Show(False) as a
>>> substitute for obj.Destroy() would have the desired effect from a user
>>> perspective.
>>
>>
>> From a user perspective it may be doing what you think, but calling
>> obj.Show(False) simply hides the widget: it is still there, it's
>> occupying GDI objects (which on Windows can become a scarce resource)
>> and it can sometimes mess your your interface anyway.
>
>
> Only while there is a variable or object field referencing it. As soon as
> there are none then the Python garbage collector will dispose of it and it
> should be stressed that this is the only time that it can be safely
> disposed.

There is nothing you or I can do about this because this approach will
probably require a big rewrite of the underlying logic of the SWIG
wrappers. As long as a widget has been created and it is sitting in a
frame, no Python garbage collector will be able to remove it from
there, whatever the variable referring to it in your code is doing: it
can go out of scope, it can be deleted, it doesn't matter. The widget
will still be there.

[Werner]

On 18/09/2012 01:29, Peter Williams wrote:
> On 09/17/2012 08:54 PM, Werner wrote:
>> Hi Peter,
>>
>> On 16/09/2012 07:32, Peter Williams wrote:
>>> As a newcomer (coming from PyGTK), I've been alarmed by the number of
>>> segmentation faults that I've experienced using wxPython.
>>
>> Do you have some sample code showing these seg faults?
>
> Try deleting an item with an embedded window from an UltimateListCtrl
> for a start.
>
> Also see changes that are being made to methods such as
> Sizers.Remove() to avoid segmentation faults caused when it used to
> destroy the removed object.

A picture is worse a 1,000 words, or a in this case a sample app;-)

http://wiki.wxpython.org/MakingSampleApps

I am not saying that you are wrong but I can't believe that it could be
as bad as you think it is. Otherwise we would see many more messages
about this kind of problem on here.

BTW, Linux/Windows/Mac does matter, just the other day there was an
issue where something worked on Linux but it didn't on WIndows and after
the code was corrected it worked correctly on both.

Werner

[Sam Partington]

Hello All,

> I am not saying that you are wrong but I can't believe that it could be as
> bad as you think it is. Otherwise we would see many more messages about
> this kind of problem on here.

Actually I think it is quite bad in it's current state. We get quite
a few seg faults here resulting from Destroy. Though simply
Destroy'ing a window which the python still has a reference to is not
enough, that will result in wx.PyDeadObjectError which is by no means
something to be ignored because we've found that we need to fix as
often enough it is a segfault waiting to happen when the code around
it changes, especially on GTK which does seem to be a lot more crash
happy than the other platforms.

The most common cross platform segfaults we've experienced have been
to do when an event handler is in the callstack for a window which is
then destroyed. As this sample demonstrates :

import wx

def on_kill_focus(evt):
evt.GetEventObject().Destroy()

def show_dialog(parent):
wx.Dialog(parent).ShowModal()

if __name__ == "__main__":
app = wx.App()
frame = wx.Frame(None)
ctrl = wx.TextCtrl(frame, style=wx.LC_REPORT)
ctrl.Bind(wx.EVT_KILL_FOCUS, on_kill_focus)
wx.CallLater(100, show_dialog, ctrl)
frame.Show()
app.MainLoop()

In this case the creation of a dialog causes the kill focus event on
the text ctrl to fire. This handler then destroys the text ctrl. We
think that when the kill focus event handler returns to C++ land the
this pointer for the code in wxTextCtrl::HandlEvent pointer is then
dead.

This is just conjecture at the moment though. My initial attempt at
fixing this is :

1. on each event handler HandleEvent add this to an
wxApp::do_not_destroy set container. Remove it on exit from the
HandleEvent fn.
2. Have Destroy check this set, and if the window is in do_not_destroy then:
a. add the window to a pending_destroys container,
b. call show(False) instead of actually destroying the window,
c. Post a CallAfter event to call flush_pending_destroys
3. flush_pending_destroys will then try to destroy any windows in the
pending_delete list, obviously checking they are not in the
do_not_destory container. If the pending_destroys is not empty then
post another CallAfter event to call flush_pending_destroys again.

I've not finished it yet though :-(

Another option I'm considering is to separate the Destroy from the
memory deletion. Have Destroy put the window object into the
unconstructed state (and raising PyDeadObjectError probably for any
calls from python). Let C++ track the Destroys through the
parent/child ownership and, let python track the memory delete through
gc. This seems like the better fix to me but it's a slightly more
radical change.

We will of course post anything we come up with.

Sam

> --
> To unsubscribe, send email to wxPython-dev...@googlegroups.com
> or visit http://groups.google.com/group/wxPython-dev?hl=en

[Werner]

I am sure Robin will chip in when he has a moment and give us all an
explanation on why it is done the way it is done (in 2.9 and 2.9 Phoenix).

Above still crashes for me in 2.9 classic, by changing the event handler to:

def on_kill_focus(evt):
obj = evt.GetEventObject()
if obj:
wx.CallAfter(obj.Destroy)

The crash no longer happens, obviously it would be nice if the framework
could do this and maybe it is already doing a better job at it in Phoenix.

Werner

[Sam Partington]

On 18 September 2012 11:03, Werner <werner...@sfr.fr> wrote:
> Above still crashes for me in 2.9 classic, by changing the event handler to:
>
> def on_kill_focus(evt):
> obj = evt.GetEventObject()
> if obj:
> wx.CallAfter(obj.Destroy)
>
> The crash no longer happens, obviously it would be nice if the framework
> could do this and maybe it is already doing a better job at it in Phoenix.

Yes quite often that is how we fix them when they occur. You
wouldn't believe how many CallAfter calls we make in our code base :-)

But as this comes up fairly often I would like to fix, or at least
guard for this in the C++ layer. My belief is that it should be
impossible to segfault from errors in the python code.

Sam

[Kevin Ollivier]

Hi Sam,

It's a nice idea, but that is pretty much impossible to get working reliably. TBH, the real issue here is that you don't realize at a low level what you're actually doing… You're inside wx.TextCtrl's own event handling code when you delete it. That won't work in Python any more than it will work in C++, it's just that the C++ gives you a segfault instead of an error. Sometimes when you program, you have to make assumptions, like that programmers won't delete the object itself while it's still running one of its methods or callbacks (except for Destroy, of course). :)

I do not think any library-level fix for this will be accepted. It's not a common case, and to make this not lead to a segfault you'd need to effectively make Destroy() use wx.CallAfter, which means that for all existing code out there that uses Destroy() to actually destroy the control at that time, the behavior will change, and worse, wx.CallAfter happens at an indeterminate time, so the control will be alive and around for an indefinite period of time after Destroy() is called, possibly seconds or more if the event queue is backed up.

In short, use wx.CallAfter or preferably re-design your approach so that you do not delete a control within its own event handlers. Maybe just do a RemoveChild(widget) on the parent then widget.Hide(), and put it in a deletion queue.

Regards,

Kevin

> Sam

[Robin Dunn]

On 9/15/12 10:32 PM, Peter Williams wrote:
> As a newcomer (coming from PyGTK), I've been alarmed by the number of
> segmentation faults that I've experienced using wxPython. These are the
> only such problems that I have ever experienced using Python and based
> on the workarounds I have used to dodge them (and the use of names such
> as Destroy() for object methods) I have developed a theory as to what's
> causing them.
>
> I suspect that some of these Destroy() methods end up invoking memory
> release within the C++ code. This can cause problems in Python in (at
> least) two ways:
>
> 1. there are still (active) references to the released memory and if
> these references are accessed a segmentation fault will result, and
> 2. Python's garbage collector doesn't notice that the memory has been
> released and tries to release it itself when it notices that the object
> is no longer accessible (i.e. normal procedure) resulting in a
> segmentation fault.

Python's GC doesn't (and can't) deal with C/C++ memory and object
allocations. Only with Python objects. In wxPython all the widgets and
other wx classes are only proxy objects on the Python side, the real
object is the C++ one and so we must use C++ to allocate and deallocate
them, and to follow the patterns implemented by C++ and also wx for that.

One of those patterns is that parent windows own their child objects,
meaning that the parents normally will manage the destruction of the
child objects, (usually when the parent is itself being destroyed).
This pattern is common in wx, but unfortunately there are a few
exceptions here and there to the pattern like in most software systems.
This also means that we can not allow the Python proxy objects to own
the C++ objects, because they are usually owned by something else.

There are also other cases of object ownership transfers in wx. Since
sizers have been brought up in this thread I'll comment on those too.
Sizers own their SizerItems, and if the item is pointing at another
sizer then it owns that too. However since windows are owned by their
parent then the sizer or sizeritem can not own the window. This is why
the Remove method was changed a few years back to not be able to destroy
the window, because it broke the pattern that wx is using. OTOH, a
window does own its sizer if it has one and will delete it when the
window is destroyed or if you assign a new sizer to the window.

In Classic wxPython there were some home-grown kludges that grew over
time to deal with things like this, but they really were kludges because
SWIG didn't have good support for this type of thing to begin with, and
then when they did it was different from what I was doing and so I
decided to keep my own code instead. However it was easier than it
should be for bugs to creep in, and sometimes I didn't know about a case
of ownership transfer until somebody reported a bug about it. For
Phoenix SIP has much better support for dealing with and transferring
object ownership, so it's easier to correctly handle the normal and
outlier cases and the runtime management is better too.

However the simple fact remains, with Classic or Phoenix, that child
window objects belong to their parents and that they can not be GC'd by
Python, only their proxies can be GC'd.

IMO there are a few normal, legitimate uses of Destroy, and one
legitimate but abnormal use case. The normal cases are things like:

* Destroying a modal dialog when you are finished with it

* Calling self.Destroy in the EVT_CLOSE handler for a frame if Skip is
not being called.

* A wx.TaskbarIcon should be destroyed when the application is exiting
because, like top-level windows a TBI will keep the main loop from
exiting so it can continue to feed events to the TBI.

* A wx.Menu used with PopupMenu could be Destroyed if you're not going
to use it again or if it's easier to just recreate it when needed.
Since the menu is not being assigned to a menu bar nobody owns it and so
the programmer should manage it instead. (This case is changing in
Phoenix.)

I call the remaining case 'abnormal' because it lets you side-step the
normal pattern used by wx. This is the case of destroying non top-level
windows before their parent is being destroyed. Because it is abnormal
and not following the established pattern then the programmer must take
care to not cause C++ to shoot them in the foot. As has been discussed
already, you must ensure that Destroy is not called while methods or
event handlers belonging to the widget to be destroyed are still active,
and you must ensure that there are no pending events for the widget.
There is no way around this that will not break or at least hamper the
normal cases or other related code. But since this is an abnormal use
case then it should be expected to have to be careful when using it.

> The current approach is leading to lots of inconsistencies between different parts of wxPython and also with the documentation e.g. the 2.8 documentation for Sizer.Remove() is now very inaccurate in that it explicitly states that the removed object is destroyed but it no longer is (as the previously mentioned bug fix for a segmentation fault).

Actually it doesn't. The 2.8 C++ docs explicitly says that the
Remove(window) version does not destroy the window and that Detach
should be used instead. IIRC it has had that text there for a very long
time.

On 9/18/12 3:03 AM, Werner wrote:
> obviously it would be nice if the framework could do this and maybe it is already doing a better job at it in Phoenix.

I suppose I could add something like this:

def SelfDestruct(self): # or maybe SafelyCommitSuicide ? ;-)
self.Hide()
wx.CallAfter(self.Destroy)

In 2.9 we now have wx.App.ScheduleForDestruction, which should also
protect against cases where SelfDesctuct (or wx.CallAfter(self.Destroy))
is called more than once. So the above would then become something like
this:

def SelfDestruct(self):
self.Hide()
wx.GetApp().ScheduleForDestruction(self)

That's pretty darn simple IMO and I would even debate the need to add a
new method for this since the building blocks are already there and the
Hide may not even be needed in some cases.

--
Robin Dunn
Software Craftsman
http://wxPython.org

[Peter Williams]

On 09/18/2012 03:23 PM, Andrea Gavana wrote:
> Hi,
>
> On 18 September 2012 01:34, Peter Williams wrote:
>> On 09/17/2012 09:35 PM, Andrea Gavana wrote:
>>>
>>> Hi,
>>>
>>>
>>> On 16 September 2012 07:32, Peter Williams wrote:
>>>>
>>>> As a newcomer (coming from PyGTK), I've been alarmed by the number of
>>>> segmentation faults that I've experienced using wxPython. These are the
>>>> only such problems that I have ever experienced using Python and based on
>>>> the workarounds I have used to dodge them (and the use of names such as
>>>> Destroy() for object methods) I have developed a theory as to what's
>>>> causing
>>>> them.
>>>>
>>>> I suspect that some of these Destroy() methods end up invoking memory
>>>> release within the C++ code. This can cause problems in Python in (at
>>>> least) two ways:
>>>
>>>
>>> The Destroy() method has its own niche applications for non-toplevel
>>> windows, i.e. you do not use it all the time for all the widgets: if
>>> you Destroy() a wx.Frame (for example when you close a window) all its
>>> children are automatically destroyed for you.
>>
>>
>> That's where it's most dangerous.
>
> I see it as "useful", and not particularly dangerous: I have never
> seen a segmentation fault report because a wx.Frame destroyed its
> children while closing.

You've been lucky. It is not a good idea to be releasing memory
manually in an automatic memory management. That's why Python doesn't
have ANY way of doing it normally.

Programmer managed memory allocation/release is error prone and
difficult and therefore bad. Automatic memory management with a good
garbage collector is good. Mixing the two is very bad.

Another way of looking at this is to say don't use C++/Java programming
paradigms when programming in Python - they're different languages and
require different methodologies. I realize the wxPython is wrapper for
a C++ library and that it's not possible to completely avoid C++ey
things but they should be kept to a minimum.

>
>>>>
>>>> 1. there are still (active) references to the released memory and if
>>>> these
>>>> references are accessed a segmentation fault will result, and
>>>> 2. Python's garbage collector doesn't notice that the memory has been
>>>> released and tries to release it itself when it notices that the object
>>>> is
>>>> no longer accessible (i.e. normal procedure) resulting in a segmentation
>>>> fault.
>>>
>>>
>>> I have very little experience with Linux so I can't really comment on
>>> these statements. You will probably have to show some minimal,
>>> standalone sample application that demonstrates the problem.
>>
>>
>> See my other reply on this thread.
>
> I haven't used 2.8 since a veeeeery long time (2.9 is way much better
> than 2.8 overall),

Unfortunately, the documentation on wxpython's site is still for 2.8 so
newcomers such as myself expect the behaviour to match that. Also, as
2.9 its labelled (probably unjustly) "unstable" most distributions ship
2.8. Maybe they should be encouraged to ship 2.9?

But, more to my point, you will find if you look into it that 2.9 and
also 2.8 have undergone piecemeal changes similar to what I propose in
order to fix segmentation fault bugs. So what I'm really proposing is a
cessation of piecemeal changes and a systematic shift to an "automatic
only" memory management model.

> but a couple of comments anyway:
>
> - Allowing the users to rely on wx.Sizer.Remove to destroy its managed
> windows was a big mistake, in my opinion: wx.Sizer is not a sub-class
> of wx.Window and it cannot have any parent-child relationship with any
> wxPython control. I am happy this behaviour has been removed in 2.9;

No argument from me BUT it was removed to avoid segmentation faults. So
it wasn't just a philosophical error it was a fundamental programming error.

BTW I think it would make programmers' lives easier if it did Hide() on
the removed object to save the programmer the need to do it himself (btw
Python won't automatically destroy/release the window while its still
visible (in the Show(True) sense)). However, because it is possible
that the programmer does want the window to be still visible after it is
removed we are left with the problem of how to allow that.

So I would suggest changing the interface to Remove() to be

def Remove(self, child, hide=True):

this would allow programmers who didn't the window to be hidden after
removal to achieve their aim.

>
> - The issues with UltimateListCtrl (or any other AGW widget) should
> not be blamed on the library behaviour, but on the sloppy programmer
> who created them (me, in this case). In general, however, when you
> have an issue with a custom control (i.e., everything in wx.lib and
> its sub-directories), you should do the following:

No, the fault is wxPython's as the Destroy() method should NOT release
the memory but just do any other housekeeping that's necessary such as
removing it from any internal widget management structures.

BTW I emailed the address in the AGW documentation a bug report about
this issue and followed up with a email about the workaround I used
which was to redefine my embedded windows Destroy() method to be
Show(False). Did you receive these.

>
> 1) Get the latest SVN version of the problematic widget (as they can
> evolve much faster than wxPython does over time);

I'm not so wrapped in wxPython that I want to use other than the latest
version in the Linux distribution that I'm using (at this stage).

But I do look in the bug system to see if my problem has been
reported/fixed and if not I report it.

I then develop a workaround which I document with a # WORKAROUND comment
that shows up in my editors "TO DO" list so that I'm prompted to revisit
them when Fedora update wxPython.

> 2) Test again.

Of course.

>
> Most of them are wxPython-version-independent (to an extent), so they
> can be used with 2.8 and 2.9 without modification. If the issue
> persists, file a ticket to the wxTrac:
>
> http://trac.wxwidgets.org/

I've done this (see #14642) and I started this thread in response to a
request made in the discussion of that bug report.

>
> Specifying the component (AGW in this case).

OK. I didn't realize AGW bugs went here as well which is why I emailed
my bug report to the email address in the documentation.

If they are well compartmentalised it should not be a big change. The
alternative is to get there by stealth as people trigger segmentation
faults and you have fix them.

> As long as a widget has been created and it is sitting in a
> frame, no Python garbage collector will be able to remove it from
> there, whatever the variable referring to it in your code is doing: it
> can go out of scope, it can be deleted, it doesn't matter. The widget
> will still be there.

I'm not sure what you mean. But the fact that the segmentation faults
tend to be triggered when Python's garbage collector tries to release
the memory after it has already been released by the Destroy() seems to
belie your argument.

Peter
PS I suspect that the fix may be as simple as modifying the way that the
Destroy() methods in wxWidgets are mapped to wxPython. Probably only
the one that is defined in the widget at the bottom of the hierarchy
(with any luck).

[Peter Williams]

On 09/18/2012 04:29 PM, Werner wrote:
> On 18/09/2012 01:29, Peter Williams wrote:
>> On 09/17/2012 08:54 PM, Werner wrote:
>>> Hi Peter,
>>>
>>> On 16/09/2012 07:32, Peter Williams wrote:
>>>> As a newcomer (coming from PyGTK), I've been alarmed by the number of
>>>> segmentation faults that I've experienced using wxPython.
>>>
>>> Do you have some sample code showing these seg faults?
>>
>> Try deleting an item with an embedded window from an UltimateListCtrl
>> for a start.
>>
>> Also see changes that are being made to methods such as
>> Sizers.Remove() to avoid segmentation faults caused when it used to
>> destroy the removed object.
> A picture is worse a 1,000 words, or a in this case a sample app;-)
>
> http://wiki.wxpython.org/MakingSampleApps
>
> I am not saying that you are wrong but I can't believe that it could be
> as bad as you think it is. Otherwise we would see many more messages
> about this kind of problem on here.

Read #14642 bug report. It's been an ongoing problem which has caused
changes to wxPython functionality.

[Peter Williams]

On 09/18/2012 07:20 PM, Sam Partington wrote:
> Hello All,
>
>> I am not saying that you are wrong but I can't believe that it could be as
>> bad as you think it is. Otherwise we would see many more messages about
>> this kind of problem on here.
>
> Actually I think it is quite bad in it's current state. We get quite
> a few seg faults here resulting from Destroy. Though simply
> Destroy'ing a window which the python still has a reference to is not
> enough, that will result in wx.PyDeadObjectError which is by no means
> something to be ignored because we've found that we need to fix as
> often enough it is a segfault waiting to happen when the code around
> it changes, especially on GTK which does seem to be a lot more crash
> happy than the other platforms.

I've not seen that error as I didn't keep separate references to my
embedded windows and used wxPython methods to retrieve them when I need
to do something to them. What I've seen is segmentation faults when
Python tries to release memory that's already been released.

I had been enjoying using wxPython (after PyGTK) because its higher
level interface (and availability of good books and tutorials - well
better than PyGTK's anyway) makes programming much easier. The
segmentation faults had me on the verge of switching back to PyGTK until
I figured out what was causing them (with the help of a very helpful
Linux report on one of the faults) and was able to develop a workaround.

This is basically the same as what I'm proposing.

> Have Destroy put the window object into the
> unconstructed state (and raising PyDeadObjectError probably for any
> calls from python). Let C++ track the Destroys through the
> parent/child ownership and, let python track the memory delete through
> gc. This seems like the better fix to me but it's a slightly more
> radical change.

I like this approach. It should fix the problem once and for all.

Peter

[Werner]

Hi Peter,

On 19/09/2012 02:03, Peter Williams wrote:

...

>>> Sizers.Remove() to avoid segmentation faults caused when it used to
>>> destroy the removed object.
>> A picture is worse a 1,000 words, or a in this case a sample app;-)
>>
>> http://wiki.wxpython.org/MakingSampleApps
>>
>> I am not saying that you are wrong but I can't believe that it could be
>> as bad as you think it is. Otherwise we would see many more messages
>> about this kind of problem on here.
> Also see changes that are being made to methods such as
>
> Read #14642 bug report. It's been an ongoing problem which has caused
> changes to wxPython functionality.

The one re sizer issues? That is closed and vadz requested to have the
discussion here.

In the sample you attached to the bug report maybe use sizer.Detach
instead of Remove.

Anyhow hopefully Robin's response in this thread has answered your
initial question/suggestion.

Werner

[Sam Partington]

On 18 September 2012 20:21, Robin Dunn <ro...@alldunn.com> wrote:
> Python's GC doesn't (and can't) deal with C/C++ memory and object
> allocations. Only with Python objects.

Yes and no. Pythons GC could be used to track all deletions if *all*
allocated objects were owned by PyObject's objects. This is not
trivial though, and would require a lot of changes to wxWidgets. This
could be done upstream though, if hooks were put into wxWidgets to
make all allocations go through a wxTrackObject fn, and all deletes go
through a similar wxUntrackObject.

The default wxWidgets implementation of these would be

template<typename T> T* wxTrackObject(T* t) { return t; }
template<typename T> void wxUntrackObject(T* t) { delete t; }

Every allocation would then do (the first new I found in the source):

wxAuiToolBarArt* wxAuiDefaultToolBarArt::Clone()
{
return static_cast<wxAuiToolBarArt*>(wxTrackObject(new
wxAuiDefaultToolBarArt));
}

And every delete would then do:

wxAuiToolBar::~wxAuiToolBar()
{
wxUntrackObject(m_art); // was delete m_art;
...
}

wxPython could then somehow replace that with something that created a
PyObject with appropriate tp_new and tp_free. It gets slightly more
complicated with container types because you have to help the GC out,
but it's not terribly hard.

I am not really suggesting we do this, and it would be an enormous
effort but it is achievable IMO.

> IMO there are a few normal, legitimate uses of Destroy, and one legitimate
> but abnormal use case. The normal cases are things like:

(skip several valid use cases)

If there is even one valid use case then it must be supported. If it
is supported we must strive to make it stable even in the face of user
error.

> I suppose I could add something like this:
>
> def SelfDestruct(self): # or maybe SafelyCommitSuicide ? ;-)
> self.Hide()
> wx.CallAfter(self.Destroy)

The problem is that you don't always know that you are self
destructing. In a complex app many code paths can mean that an event
handler for a fn is far away up the callstack from the code that is
actually doing the destroy. In our most recent crash on win32 a
double click in a dialog mysteriously got converted into a drag
message in the frame that was behind the dialog, which caused the edit
control which owned the dialog to be destroyed, in turn destroying the
dialog. In this case a simple CallAfter did not resolve the problem,
we had to capture all mouse events for the lifetime of that dialog.
Nor would hiding the dialog be appropriate in this case.

What makes me nervous is that these things keep cropping up in ways
that can not be spotted by introspection of the code, there was
nothing obviously wrong with what was written. I know that we will be
receiving more support calls from customers for ones that we have not
managed to find during test.

> def SelfDestruct(self):
> self.Hide()
> wx.GetApp().ScheduleForDestruction(self)


The problem with that is that (at least in my version of the source,
2.9.4) DeletePendingObjets only seems to be called at application
shutdown.

Doing this for every destroy would result in an ever expanding working
set. And only doing it in SelfDestruct doesn't solve all the problem
because you just don't know sometimes that you are doing a
SelfDestruct.


Sam

[Sam Partington]

Hi Kevin,

On 18 September 2012 18:12, Kevin Ollivier <kev...@theolliviers.com> wrote:
>> But as this comes up fairly often I would like to fix, or at least
>> guard for this in the C++ layer. My belief is that it should be
>> impossible to segfault from errors in the python code.
>
> It's a nice idea, but that is pretty much impossible to get working reliably.
> TBH, the real issue here is that you don't realize at a low level what you're
> actually doing… You're inside wx.TextCtrl's own event handling code when
> you delete it. That won't work in Python any more than it will work in C++,
> it's just that the C++ gives you a segfault instead of an error. Sometimes
> when you program, you have to make assumptions, like that programmers
> won't delete the object itself while it's still running one of its methods or
> callbacks (except for Destroy, of course). :)

> I do not think any library-level fix for this will be accepted. It's not a common
> case, and to make this not lead to a segfault you'd need to effectively make
> Destroy() use wx.CallAfter, which means that for all existing code out there
> that uses Destroy() to actually destroy the control at that time, the behavior
> will change, and worse, wx.CallAfter happens at an indeterminate time, so the
> control will be alive and around for an indefinite period of time after Destroy() is
> called, possibly seconds or more if the event queue is backed up.

Well, I am attempting a library level fix for this. Because actually
it is possible to know which objects have events being handled. And
if you know that then it is definitely better to either 1) have
controls hanging around for an indeterminate amount of time to crash.
Or 2) to have the call to Destroy raise an exception. (perhaps
PyAssertionError("You are attempting to destroy a window which is
currently handling an exception"))

I will post the patch when it is ready, I hope that wxPython will
consider it for acceptance because I don't think it is appropriate for
any python extension to crash under any circumstance.

> In short, use wx.CallAfter or preferably re-design your approach so that you do
> not delete a control within its own event handlers. Maybe just do a
> RemoveChild(widget) on the parent then widget.Hide(), and put it in a deletion queue.

As I said in my other reply to Robin, you just don't know that you are
deleting a control within it's own event handler half the time. We are
now (after at least 10 segfaults caused by this problem) reasonably
aware that we need to tread carefully when using Destroy. Often
enough a CallAfter is sufficient, but not always. We have even had to
do wx.CallAfter(wx.CallLater, .... ) here and there :-)

It's a hack, and I am 99% certain that we have not fixed all of our
potential crashes yet.

Sam

[Robin Dunn]

It is also called from wxAppConsoleBase::ProcessIdle, and is the same
mechanism used when destroying top-level windows.

[Peter Williams]

No. I still think that you've broken Python by adding mechanisms where
the programmer can cause memory to be released immediately. What you've
added is the ability to cause segmentation faults to a system which was
free of this problem. This should be fixed.

I think part of the problem is that it's hard to forget about memory
management if you're coming from a C/C++ environment (it certainly was
for me when I first started using Python) but after you accept the idea
that memory management is no longer your problem (as an application
programmer - it clearly still is at the SWIG level) it's very liberating
and allows you to be much more productive.

The semantics of Destroy() needs to be "I'm finished with this object
and you can get rid of it when you're ready" not "I'm finished with this
object RELEASE its memory NOW".

Now that I understand the problem (and how to work around it), I will
(as I said) continue to use wxPython instead of returning to PyGTK. I'm
finding that although it's almost as hard to find out how to do
something in wxPython as it is in PyGTK (due to obvious difficulties in
comprehensively documenting such large and complex systems) when you do
find out how the resulting code is much simpler. I like that and once
I've found out how to do something once that part of the problem goes
away too :-)

Peter
PS The most frustrated I get with wxPython documentation is trying to
find the names and meanings of the many constants e.g. EVT_XXX, ID_XXX
etc. That's the one area where PyGTK documentation is better.
PPS In ten years of using PyGTK I never experienced any segmentation
faults so it is possible to get this right.

[Peter Williams]

On 09/19/2012 07:40 PM, Sam Partington wrote:
> On 18 September 2012 20:21, Robin Dunn <ro...@alldunn.com> wrote:
>> Python's GC doesn't (and can't) deal with C/C++ memory and object
>> allocations. Only with Python objects.
>
> Yes and no. Pythons GC could be used to track all deletions if *all*
> allocated objects were owned by PyObject's objects. This is not
> trivial though, and would require a lot of changes to wxWidgets.

It doesn't haven't to be done in one big change. It can be done
gradually but with an overall plan in mind.

> This
> could be done upstream though, if hooks were put into wxWidgets to
> make all allocations go through a wxTrackObject fn, and all deletes go
> through a similar wxUntrackObject.

Maybe if you knew which bits were being managed by the GC and refrained
from releasing them it would solve half the problem. But it still
leaves the onus on the programmer to ensure that he doesn't release an
object that is being referenced elsewhere in the program and this
removes one of the main advantages of Python.

>
> The default wxWidgets implementation of these would be
>
> template<typename T> T* wxTrackObject(T* t) { return t; }
> template<typename T> void wxUntrackObject(T* t) { delete t; }
>
> Every allocation would then do (the first new I found in the source):
>
> wxAuiToolBarArt* wxAuiDefaultToolBarArt::Clone()
> {
> return static_cast<wxAuiToolBarArt*>(wxTrackObject(new
> wxAuiDefaultToolBarArt));
> }
>
> And every delete would then do:
>
> wxAuiToolBar::~wxAuiToolBar()
> {
> wxUntrackObject(m_art); // was delete m_art;
> ...
> }
>
> wxPython could then somehow replace that with something that created a
> PyObject with appropriate tp_new and tp_free. It gets slightly more
> complicated with container types because you have to help the GC out,
> but it's not terribly hard.
>
> I am not really suggesting we do this, and it would be an enormous
> effort but it is achievable IMO.
>
>> IMO there are a few normal, legitimate uses of Destroy, and one legitimate
>> but abnormal use case. The normal cases are things like:
>
> (skip several valid use cases)
>
> If there is even one valid use case then it must be supported. If it
> is supported we must strive to make it stable even in the face of user
> error.

There are no valid cases. Write Python code. Don't try to write C++ in
Python.

>
>> I suppose I could add something like this:
>>
>> def SelfDestruct(self): # or maybe SafelyCommitSuicide ? ;-)
>> self.Hide()
>> wx.CallAfter(self.Destroy)
>
> The problem is that you don't always know that you are self
> destructing. In a complex app many code paths can mean that an event
> handler for a fn is far away up the callstack from the code that is
> actually doing the destroy. In our most recent crash on win32 a
> double click in a dialog mysteriously got converted into a drag
> message in the frame that was behind the dialog, which caused the edit
> control which owned the dialog to be destroyed, in turn destroying the
> dialog. In this case a simple CallAfter did not resolve the problem,
> we had to capture all mouse events for the lifetime of that dialog.
> Nor would hiding the dialog be appropriate in this case.

In a conventional programming environment such as C/C++ where the
programmer is responsible for memory allocation/release the biggest
problem is knowing/ensuring that all pointers to the allocated memory
have been set to NULL or are no longer accessible. This one of the
reasons that those languages are hard to use. GCs are designed to get
rid of this problem and Python's does this job very well. But wxPython
reintroduces the problem.

It also introduces the problem that it interferes with the GC by
releasing memory that the GC thinks it's managing. A double whammy.

>
> What makes me nervous is that these things keep cropping up in ways
> that can not be spotted by introspection of the code, there was
> nothing obviously wrong with what was written. I know that we will be
> receiving more support calls from customers for ones that we have not
> managed to find during test.
>
>> def SelfDestruct(self):
>> self.Hide()
>> wx.GetApp().ScheduleForDestruction(self)
>
>
> The problem with that is that (at least in my version of the source,
> 2.9.4) DeletePendingObjets only seems to be called at application
> shutdown.
>
> Doing this for every destroy would result in an ever expanding working
> set. And only doing it in SelfDestruct doesn't solve all the problem
> because you just don't know sometimes that you are doing a
> SelfDestruct.

Think like a Python programmer not a C++ programmer :-).

Peter

[Peter Williams]

I've just done some tests and it appears that the Destroy() methods are
needed to prevent memory leaks. This means things are worse than I
thought. wxPython programs aren't Python programs they're C++ programs
written in Python syntax. One of Python's key advantages has been
destroyed and one of C++'s many problems has been imported. :-(

Means I'll cancel my plans to migrate gquilt, gwsmhg and darning to
wxPython and my current wxPython project will probably be my last as
responses to this thread don't give me any hope of things getting any
better.

Peter

[Kevin Ollivier]

On Sep 19, 2012, at 8:55 PM, Peter Williams wrote:

[snip]

>> Peter
>> PS The most frustrated I get with wxPython documentation is trying to
>> find the names and meanings of the many constants e.g. EVT_XXX, ID_XXX
>> etc. That's the one area where PyGTK documentation is better.
>> PPS In ten years of using PyGTK I never experienced any segmentation
>> faults so it is possible to get this right.
>
> I've just done some tests and it appears that the Destroy() methods are needed to prevent memory leaks.

Yes, precisely, there is a reason the method exists, and when called in a valid way and time, it should work exactly as designed. Perhaps it doesn't for you, but I've never figured out what precisely it does wrong in your case. As you describe the behavior you see, as designed you should just get a PyDeadObject error rather than a segfault. Why that is happening, well, I can't say because we have no code to try and test against.

> This means things are worse than I thought. wxPython programs aren't Python programs they're C++ programs written in Python syntax. One of Python's key advantages has been destroyed and one of C++'s many problems has been imported. :-(

This is true of any wrapper API, including PyGTK and PyQT. And, BTW, users have gotten both of those to segfault as well as wxPython. Google knows this. :) The truth is, quite many wxPython users don't hit segfaults, just as you did not hit segfaults with PyGTK, but it is incorrect to think that somehow any of these toolkits are immune to them just because you are using them from Python. They are running C or C++ code under the hood, and they have their own ways of dealing with memory management that are separate from Python's. If there is a bug within that code, or if you use an API improperly or send it malformed data, a segfault is always a possibility.

> Means I'll cancel my plans to migrate gquilt, gwsmhg and darning to wxPython and my current wxPython project will probably be my last as responses to this thread don't give me any hope of things getting any better.

Aside from someone writing a pure Python GUI toolkit, this will always be an issue.

Regards,

Kevin

> Peter

[Kevin Ollivier]

Hi Sam,

Any Python extension can crash, this is just reality. Any time you get "closer to the metal" that is the case. You also get speed benefits that come from C++'s simplified and faster methods of managing memory, and without those, Python would probably be too slow to even run GUI apps. So there are trade-offs, it is not a one way issue of getting the 'badness' from C++. You get its good points too, and thanks to Python, you spend a lot less time dealing with its bad parts. Less time is not no time, though.

>> In short, use wx.CallAfter or preferably re-design your approach so that you do
>> not delete a control within its own event handlers. Maybe just do a
>> RemoveChild(widget) on the parent then widget.Hide(), and put it in a deletion queue.
>
> As I said in my other reply to Robin, you just don't know that you are
> deleting a control within it's own event handler half the time. We are
> now (after at least 10 segfaults caused by this problem) reasonably
> aware that we need to tread carefully when using Destroy.

The solution is simple - do not do deletion inside event handlers. Schedule them for deletion later. C++ programmers can't do this without getting segfaults either, BTW. ;-)

> Often
> enough a CallAfter is sufficient, but not always. We have even had to
> do wx.CallAfter(wx.CallLater, .... ) here and there :-)
>
> It's a hack, and I am 99% certain that we have not fixed all of our
> potential crashes yet.

It's not really a hack, it's how you handle this use case. There are just certain things you should not do inside event handlers.

Bottom line is that modifying Destroy() on the Python side would probably not be welcomed because it can cause behavior changes in existing apps for whom Destroy() is working as expected. Under the hood behavior changes of this nature for APIs really only make sense when they are fundamentally broken. Destroy() not working in event handlers is not really brokenness from the perspective that it is not meant to be called from them in the first place, in Python or C++ or any language. (Perhaps a note to this effect in the docs would be welcomed, though.)

It would, though, be great to have some sort of design pattern for this case, in terms of sample code and maybe some documentation so that someone who comes along and needs to do this later has some guide to follow. And if Robin's suggestions aren't sufficient, you will probably find him or others willing to help work out the kinks.

[Sam Partington]

Hi Kevin,

On 20 September 2012 06:20, Kevin Ollivier <kev...@theolliviers.com> wrote:
> Any Python extension can crash, this is just reality. Any time you get
> "closer to the metal" that is the case. You also get speed benefits that
> come from C++'s simplified and faster methods of managing memory,
> and without those, Python would probably be too slow to even run GUI
> apps. So there are trade-offs, it is not a one way issue of getting the
> 'badness' from C++. You get its good points too, and thanks to Python,
> you spend a lot less time dealing with its bad parts. Less time is not no time, though.

Any extension can crash. No extension should crash. I have no idea
what you mean by 'closer to the metal', it sounds like a cop out to
me. "Our extension is too complex to make it safe".

You don't know if my proposed fix will slow down wxPython at all, let
alone noticeably. Until you have measured the slow down there is no
way you can decide whether the pay off is worth it or not. If it does
turn out to be significantly slower then we could certainly make it a
build time option, and so let the user of the library decide, assuming
they are happy to build their own version of wxPython, we already do
this because when we find and fix bugs in wxPython we can't wait for
the wxPython release cycle to incorporate our fixes (note that I'm
most definately not complaining here, just stating the facts). We do
submit all bug fixes upstream.

In any case, for the software that my company writes, I would accept a
slower stable version over a faster unstable version every single
time. If performance were that important we wouldn't be using python
at all.

>>> In short, use wx.CallAfter or preferably re-design your approach so that you do
>>> not delete a control within its own event handlers. Maybe just do a
>>> RemoveChild(widget) on the parent then widget.Hide(), and put it in a deletion queue.
>>
>> As I said in my other reply to Robin, you just don't know that you are
>> deleting a control within it's own event handler half the time. We are
>> now (after at least 10 segfaults caused by this problem) reasonably
>> aware that we need to tread carefully when using Destroy.
>
> The solution is simple - do not do deletion inside event handlers. Schedule them for deletion later. C++ programmers can't do this without getting segfaults either, BTW. ;-)

You've clearly not read what I wrote in that paragraph which is that
most of the recent segfaults that we have experienced have not
resulted from calling Destroy directly in an event handler but from a
unusual chain of events through several layers of abstraction and
indirection that would make it more or less impossible to tell from
inspection that Destroy was being called from an event handler.

As the application complexity increases the likelihood of these things
taking place increases.

It is, BTW, completely irrelevant that C++ can cause crashes in the
same way. It is precisely after 10 years of writing C++ applications
which would, from time to time, crash when a precondition was not
meant that we have decided to rewrite the whole thing in python.

>> Often
>> enough a CallAfter is sufficient, but not always. We have even had to
>> do wx.CallAfter(wx.CallLater, .... ) here and there :-)
>>
>> It's a hack, and I am 99% certain that we have not fixed all of our
>> potential crashes yet.
>
> It's not really a hack, it's how you handle this use case. There are just certain things you should not do inside event handlers.

See above, we didn't (directly).

> Bottom line is that modifying Destroy() on the Python side would
> probably not be welcomed because it can cause behavior changes
> in existing apps for whom Destroy() is working as expected. Under
> the hood behavior changes of this nature for APIs really only make
> sense when they are fundamentally broken. Destroy() not working in
> event handlers is not really brokenness from the perspective that it
> is not meant to be called from them in the first place, in Python or
> C++ or any language. (Perhaps a note to this effect in the docs
> would be welcomed, though.)

I am amazed that you think that the current API of wxPython crashing
is not fundamentally broken.

My proposed fix is to detect Destroy being called from within an event
handler and only then to behave differently. I have not yet decided
whether that should then raise an exception or defer the Destroy.
Perhaps in this case explicit is better than implicit.

Yes this would be a behaviour change, but one that I think most users
would welcome.

API : Call destroy (directly, or indirectly) from an event handler:

Current behaviour: crash your app.
Proposed behaviour: raise exception, or defer the destroy until is is
safe to do so.

I find it hard to imagine that there are many users who would be upset
about this change in behaviour.

Sam

[Werner]

Hi Sam,

Not that I will be of any help doing any of this, but

> My proposed fix is to detect Destroy being called from within an event
> handler and only then to behave differently. I have not yet decided
> whether that should then raise an exception or defer the Destroy.
> Perhaps in this case explicit is better than implicit.
>
> Yes this would be a behaviour change, but one that I think most users
> would welcome.
>
> API : Call destroy (directly, or indirectly) from an event handler:
>
> Current behaviour: crash your app.

-1
> Proposed behaviour: raise exception,
+1

> or defer the destroy until is is
> safe to do so.

I prefer the exception right there, instead of deferring and then maybe
still get a crash

Werner

[Kevin Ollivier]

Hi Sam,

On Sep 20, 2012, at 4:27 AM, Sam Partington wrote:

> Hi Kevin,
>
> On 20 September 2012 06:20, Kevin Ollivier <kev...@theolliviers.com> wrote:
>> Any Python extension can crash, this is just reality. Any time you get
>> "closer to the metal" that is the case. You also get speed benefits that
>> come from C++'s simplified and faster methods of managing memory,
>> and without those, Python would probably be too slow to even run GUI
>> apps. So there are trade-offs, it is not a one way issue of getting the
>> 'badness' from C++. You get its good points too, and thanks to Python,
>> you spend a lot less time dealing with its bad parts. Less time is not no time, though.
>
> Any extension can crash. No extension should crash. I have no idea
> what you mean by 'closer to the metal', it sounds like a cop out to
> me. "Our extension is too complex to make it safe".

There is a reason there are no pure Python GUIs. All the safety checks Python runs make it impossible to write code that needs to be as fast and tight as GUI code needs to be. You can call that a cop out if you like, but it's a plain fact. Slowing down Destroy() is probably not a huge deal, but if you really want wxPython never to be able to crash under any circumstance, you will indeed need to slow it to the point where it is no longer usable speed-wise.

If it's more or less impossible for you to know how and where your code is being called, I think that is a (perhaps the) problem.

> As the application complexity increases the likelihood of these things
> taking place increases.
>
> It is, BTW, completely irrelevant that C++ can cause crashes in the
> same way. It is precisely after 10 years of writing C++ applications
> which would, from time to time, crash when a precondition was not
> meant that we have decided to rewrite the whole thing in python.
>
>>> Often
>>> enough a CallAfter is sufficient, but not always. We have even had to
>>> do wx.CallAfter(wx.CallLater, .... ) here and there :-)
>>>
>>> It's a hack, and I am 99% certain that we have not fixed all of our
>>> potential crashes yet.
>>
>> It's not really a hack, it's how you handle this use case. There are just certain things you should not do inside event handlers.
>
> See above, we didn't (directly).

It is more correct to say "we didn't realize our code was doing that", but that still means it did do it.

>> Bottom line is that modifying Destroy() on the Python side would
>> probably not be welcomed because it can cause behavior changes
>> in existing apps for whom Destroy() is working as expected. Under
>> the hood behavior changes of this nature for APIs really only make
>> sense when they are fundamentally broken. Destroy() not working in
>> event handlers is not really brokenness from the perspective that it
>> is not meant to be called from them in the first place, in Python or
>> C++ or any language. (Perhaps a note to this effect in the docs
>> would be welcomed, though.)
>
> I am amazed that you think that the current API of wxPython crashing
> is not fundamentally broken.

The API does, and is designed to, delete the control. There is no better indication of "use with caution" than that. As Peter said, and was quite right about, you ARE using a C++ API, even if you are using it from Python. Python does not have methods like this because they use ref-counting for everything. C++ and wxWidgets sometimes use an object ownership model instead, and that is the case here.

> My proposed fix is to detect Destroy being called from within an event
> handler and only then to behave differently. I have not yet decided
> whether that should then raise an exception or defer the Destroy.
> Perhaps in this case explicit is better than implicit.
>
> Yes this would be a behaviour change, but one that I think most users
> would welcome.
>
> API : Call destroy (directly, or indirectly) from an event handler:
>
> Current behaviour: crash your app.
> Proposed behaviour: raise exception, or defer the destroy until is is
> safe to do so.
>
> I find it hard to imagine that there are many users who would be upset

> about this change in behavior.

Turning an expected Destroy() crash into an exception, provided that you can *always* perfectly determine when Destroy() is going to lead to a crash, is probably not a harmful change. However, that's the sort of change that sounds like it would use some sort of heuristics for determining where it is called from (perhaps parsing a stack trace?), which is probably very tough to get right, and may very well lead to exceptions being thrown when the programmer knows what they're doing and is calling the API in a safe and valid way. Or, also, in some limited cases calling from an event handler may be perfectly acceptable, such as, say, a menu event handler or a user-defined event, which we would now likely forbid. Prediction is very, very hard to get right unconditionally.

In addition, altering Destroy() in this way doesn't help anyone who may trigger a crash, just this specific case, so it's usefulness is limited as well. A more general fix for crashes is probably a much better option. For example, this may more or less solve the problem for you, and not only apply to the Destroy() case, but any possible crash scenario:

http://pypi.python.org/pypi/faulthandler/

In fact, this was included in Python as of 3.3, w00t! :) If this module is reliable (and inclusion suggests it has at least been vetted), I personally would be in favor of including something like this in wxPython, at least for versions < 3.3, as getting a stack trace when a crash does happen is pretty nice, IMHO, and makes tracking down crashes and fixing them much simpler.

Now, deferring the Destroy(), on the other hand, is not just an error catch but a behavior change, and the only way to defer it reasonably would be to make it asynchronous. That means that, for everyone calling it, the Destroy() will now happen at some unknown time. That, to me, is scary. If your wx event queue gets really backed up, it could delay the call for several seconds. In the meantime, code expecting the control to be deleted may run, which could cause any number of unexpected behaviors. This is especially possible if the code has many abstraction layers, as you well know. :)

[Chris Barker]

On Wed, Sep 19, 2012 at 10:05 PM, Kevin Ollivier
<kev...@theolliviers.com> wrote:
> As you describe the behavior you see, as designed you should just get a
> PyDeadObject error rather than a segfault. Why that is happening, well, I can't say
> because we have no code to try and test against.

This may have nothing to do with this particular issue, but I recall a
while back a thread on this list that surrounded the fact that users
on certain Linux distros were getting a LOT of hard crashes, whereas
most of us get None.

It turns out it was because of how wx was compiled -- I don't know the
details, but wxPython assumes that wx is compiled (in debug mode?) in
a way that some of this kind of dynamic checking goes on, rather than
a hard crash -- exactly to help with teh impedance mis-match between a
static and dynamic language.

Maybe that's the issue here?

It really strikes me that Peter is experiencing a really different
environemnt than the rest of -- so he's either doing something
different:
- A far more dynamic GUI than most?
- Trying to overide wx's natural destruction of Windows, rahter than
letting it do its thing:
- I don't hink I"ve ever called Destroy()

or there is something wrong in the build he's using.

It's nothing to me if one used decided wx is not for him/her, but it'd
be a shame if it really was a fixable issue.

>> This means things are worse than I thought. wxPython programs aren't Python programs they're C++ programs written in Python syntax. One of Python's key advantages has been destroyed and one of C++'s many problems has been imported. :-(

Sad, but maybe unavoidable -- I've used Pyton to wrap the occasional C
and C++ code, and unless your C or C++ has a reference counting scheme
in place, there is going to be a bit of an impedance mismatch -- maybe
GTK does use a reference counting scheme that PyGTK ties into?

-Chris


--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[Sam Partington]

Hello Kevin,

On 20 September 2012 17:17, Kevin Ollivier <kev...@theolliviers.com> wrote:
>> On 20 September 2012 06:20, Kevin Ollivier <kev...@theolliviers.com> wrote:
>> Any extension can crash. No extension should crash. I have no idea
>> what you mean by 'closer to the metal', it sounds like a cop out to
>> me. "Our extension is too complex to make it safe".
>
> There is a reason there are no pure Python GUIs. All the safety checks
> Python runs make it impossible to write code that needs to be as fast
> and tight as GUI code needs to be. You can call that a cop out if you like

> but it's a plain fact. Slowing down Destroy() is probably not a huge deal,
> but if you really want wxPython never to be able to crash under any
> circumstance, you will indeed need to slow it to the point where it is no
> longer usable speed-wise.

Well I don't buy that personally. I don't think that there are no
pure python apis because of speed, IMO it's because a gui api needs to
interface with the OS a lot. Most OS apis are C/C++. Objective C and
pyobjc is interesting because that is very nearly a pure python GUI
API as it stands. The performance is quite acceptable. Java has some
more-or-less pure java UI's, and I don't think the performance of
python is significantly worse than Java.

Actually I have started a wxPython clone in pure python, using ctypes
to call win32 and gtk. But it is still very immature at the moment,
and I've not even started on Mac OS yet. I've not measured speed yet
mind you. But I wouldn't even be that surprised if a pure python ui
running under PyPy it might easily outperform a CPython version with a
C extension and all the nasty thread locking that has to go on.

Nor do I believe it is impossible to make wxPython considerably more
stable than it is now.

> If it's more or less impossible for you to know how and where your code
> is being called, I think that is a (perhaps the) problem.

Yes in a very large multi threaded app, with many developers, many
OSs, many users, many distros, and many windows managers - I don't
know when, where, and how our code is called at all times. The many
OSs being the main problem - the order that events occur varies from
OS to OS. The last segfault we had was structured like this

hypertreelist -> edit ctrl -> modal dialog -> list ctrl.

A double click in the list ctrl closed the dialog (EndModal). But for
some as yet unexplained reason on win32 the double click event was
converted into a item drag event in the hypertreelist when the mouse
cursor was in just the right place. The simplified callstack was like
this :

app.MainLoop -> edit_ctrl.on_edit_keydown -> dialog.ShowModal() ->
hypertreelist.on_selection_changed -> edit_ctrl.Destroy()

So apart from the fact that the Destroy was in agw code, not ours, how
could you predict that a double click in a list ctrl in a modal dialog
would cause a drag event in a hypertreelist *behind* the dialog which
is still there? Easy to fix sure, hard to spot before it happens.

And no-one did anything inherently wrong here. Why shouldn't an
on_key_down bring up a dialog, and why shouldn't the hypertreelist
delete it's own edit when the selection was changed?

>> I am amazed that you think that the current API of wxPython crashing
>> is not fundamentally broken.
>
> The API does, and is designed to, delete the control. There is no better
> indication of "use with caution" than that. As Peter said, and was quite
> right about, you ARE using a C++ API, even if you are using it from
> Python. Python does not have methods like this because they use
> ref-counting for everything. C++ and wxWidgets sometimes use an
> object ownership model instead, and that is the case here.

Another excuse, not a solution.

>> API : Call destroy (directly, or indirectly) from an event handler:
>>
>> Current behaviour: crash your app.
>> Proposed behaviour: raise exception, or defer the destroy until is is
>> safe to do so.
>>
>> I find it hard to imagine that there are many users who would be upset
>> about this change in behavior.
>
> Turning an expected Destroy() crash into an exception, provided that you
> can *always* perfectly determine when Destroy() is going to lead to a crash,
> is probably not a harmful change. However, that's the sort of change that
> sounds like it would use some sort of heuristics for determining where it
> is called from (perhaps parsing a stack trace?), which is probably very
> tough to get right, and may very well lead to exceptions being thrown
> when the programmer knows what they're doing and is calling the API
> in a safe and valid way. Or, also, in some limited cases calling from an
> event handler may be perfectly acceptable, such as, say, a menu event
> handler or a user-defined event, which we would now likely forbid.
> Prediction is very, very hard to get right unconditionally.

Actually it's very straightforward in this case. enter an event
handler: increment a counter, leave event handler: decrement counter.
block destroy when counter > 0.

> In addition, altering Destroy() in this way doesn't help anyone who may
> trigger a crash, just this specific case, so it's usefulness is limited as
> well. A more general fix for crashes is probably a much better option.

But for now I can try to fix the segfaults I know about, we've had
about 10 caused by this problem now, so it's worth me investing some
time in it. In fact it's the only type of segfault we've had since I
fixed all the places that weren't taking the GIL where it should have
been taken.

Unfortunately, I don't have time to fix the much harder general fix
right now. Hopefully at some point I will.

> For example, this may more or less solve the problem for you, and not
> only apply to the Destroy() case, but any possible crash scenario:
>
> http://pypi.python.org/pypi/faulthandler/
>
> In fact, this was included in Python as of 3.3, w00t! :) If this module is
> reliable (and inclusion suggests it has at least been vetted), I personally
> would be in favor of including something like this in wxPython, at least
> for versions < 3.3, as getting a stack trace when a crash does happen
> is pretty nice, IMHO, and makes tracking down crashes and fixing them
> much simpler.

It would help diagnose, but the horse has already bolted. Incidentally
gdb does a pretty good job of getting a callstack too with the python
plugin.

> Now, deferring the Destroy(), on the other hand, is not just an error catch
> but a behavior change, and the only way to defer it reasonably would be
> to make it asynchronous. That means that, for everyone calling it, the
> Destroy() will now happen at some unknown time. That, to me, is scary.
> If your wx event queue gets really backed up, it could delay the call for
> several seconds. In the meantime, code expecting the control to be
> deleted may run, which could cause any number of unexpected
> behaviors. This is especially possible if the code has many abstraction
> layers, as you well know. :)

Why do you have to make it asynchronous for everyone? I have never
suggested that. You can just Destroy it when the counter goes to zero,
and Destroy has been called.

Amyway, we'll implement our fix and submit a patch. I hope it'll be
accepted, but either way we'll use it and our application will be more
stable as a result.

Sam

[Peter Williams]

On 09/20/2012 03:05 PM, Kevin Ollivier wrote:
>
> On Sep 19, 2012, at 8:55 PM, Peter Williams wrote:
>
> [snip]
>
>>> Peter
>>> PS The most frustrated I get with wxPython documentation is trying to
>>> find the names and meanings of the many constants e.g. EVT_XXX, ID_XXX
>>> etc. That's the one area where PyGTK documentation is better.
>>> PPS In ten years of using PyGTK I never experienced any segmentation
>>> faults so it is possible to get this right.
>>
>> I've just done some tests and it appears that the Destroy() methods are needed to prevent memory leaks.
>
> Yes, precisely, there is a reason the method exists,

This is why I think that the likelihood of this issue being fixed
properly is zero. You don't understand how this solution to memory
leaks was completely inappropriate in a Python package. It has broken
one of Python's best features -- automatic memory management.

And if you don't think it's a problem you won't fix it and that's your
privilege. But from my point of view it suddenly makes writing bug free
wxPython applications much harder than writing PyGTK applications due to
the need to guard against memory leaks and segmentation faults. So I
intend to go with the easier option.

> and when called in a valid way and time, it should
> work exactly as designed.

If I wanted to program like that I would use wxWidgets directly in C++
rather than use wxPython.

> Perhaps it doesn't for you, but I've never figured out what precisely it does
> wrong in your case.

At the moment, you have a situation where if you call Destroy() on some
widgets you will get a segmentation fault when the GC tries to release
the same memory and if you don't call it on some other widgets you will
get a memory leak because the GC doesn't know about them. The technical
term for this is "a right mess".

You should peruse your bug report system to see countless instances that
illustrate what I'm talking about. The piecemeal fixes being used to
fix these issues is making the problem worse rather than better.

> As you describe the behavior you see, as designed you should just get a PyDeadObject
> error rather than a segfault.

That's what you'll get if you release memory the GC knows about while
there are still references to it. The segmentation faults occur if the
reference you used for the Destroy() was the last reference and the GC
tries to release when that reference is broken.

> Why that is happening, well, I can't say because we have no code to try and test against.
>
>> This means things are worse than I thought. wxPython programs aren't Python programs they're C++ programs written in Python syntax. One of Python's key advantages has been destroyed and one of C++'s many problems has been imported. :-(
>
> This is true of any wrapper API, including PyGTK and PyQT. And, BTW, users have
> gotten both of those to segfault as well as wxPython. Google knows
this. :)

The difference is that they fixed the problem properly.

> The truth
> is, quite many wxPython users don't hit segfaults, just as you did
not hit segfaults with
> PyGTK, but it is incorrect to think that somehow any of these
toolkits are immune to them
> just because you are using them from Python. They are running C or
C++ code under the hood

> and they have their own ways of dealing with memory management that
are separate from Python's.

It is precisely this attitude to the problem that has made me realize
that the problem will never be fixed properly and PyGTK is the easier
path in spite of its more difficult low level interface.

> If there is a bug within that code, or if you use an API improperly or send it malformed data

> a segfault is always a possibility.

Not with a properly implemented Python extension.

>
>> Means I'll cancel my plans to migrate gquilt, gwsmhg and darning to wxPython and my current wxPython project will probably be my last as responses to this thread
don't give me any hope of things getting any better.
>
> Aside from someone writing a pure Python GUI toolkit, this will always be an issue.

Rubbish.

Cheers
Peter

[Peter Williams]

You have to get rid of Destroy() completely or, at least, stop it
releasing memory directly. The only acceptable solution to this problem
is to hand all memory management to Python's GC.

Having to worry about memory leaks is anathema to Python programmers.
Not having to do that is one of the major compensations for the fact
that Python runs slower than compiled languages.

Peter

[Peter Williams]

On 09/21/2012 03:37 AM, Chris Barker wrote:
> On Wed, Sep 19, 2012 at 10:05 PM, Kevin Ollivier
> <kev...@theolliviers.com> wrote:
>> As you describe the behavior you see, as designed you should just get a
>> PyDeadObject error rather than a segfault. Why that is happening, well, I can't say
>> because we have no code to try and test against.
>
> This may have nothing to do with this particular issue, but I recall a
> while back a thread on this list that surrounded the fact that users
> on certain Linux distros were getting a LOT of hard crashes, whereas
> most of us get None.
>
> It turns out it was because of how wx was compiled -- I don't know the
> details, but wxPython assumes that wx is compiled (in debug mode?) in
> a way that some of this kind of dynamic checking goes on, rather than
> a hard crash -- exactly to help with teh impedance mis-match between a
> static and dynamic language.
>
> Maybe that's the issue here?
>
> It really strikes me that Peter is experiencing a really different
> environemnt than the rest of -- so he's either doing something
> different:
> - A far more dynamic GUI than most?

Yes.

> - Trying to overide wx's natural destruction of Windows, rahter than
> letting it do its thing:
> - I don't hink I"ve ever called Destroy()

It gets called a lot internally.

The only calls to destroy have are a result of following examples in
books (e.g. calling Destroy() on a wx.Dialog after the user closes it)
or on line tutorials (e.g. keeping track of the wx.Menu object used for
a pop menu so that Destroy() can be called). It was the latter example
that I used as the basis for my tests to detect the memory leaks.

>
> or there is something wrong in the build he's using.

More denial. The problem is in the way that wxPython is implemented.

>
> It's nothing to me if one used decided wx is not for him/her, but it'd
> be a shame if it really was a fixable issue.
>
>>> This means things are worse than I thought. wxPython programs aren't Python programs they're C++ programs written in Python syntax. One of Python's key advantages has been destroyed and one of C++'s many problems has been imported. :-(
>
> Sad, but maybe unavoidable -- I've used Pyton to wrap the occasional C
> and C++ code, and unless your C or C++ has a reference counting scheme
> in place, there is going to be a bit of an impedance mismatch -- maybe
> GTK does use a reference counting scheme that PyGTK ties into?

I'm wondering if there isn't some documentation a python.org that
explains how to do this correctly as an awful lot of people seem to have
got it right. e.g. a lot of the standard Python library is wrappers
around C code for performance reasons.

Peter

[Kevin Ollivier]

Hi Sam,

Oh, I do not believe it is impossible at all. It is just very hard to get right, and requires lots of time and energy. I'd love for someone to invest more in beefing up wx unit testing, for example.

>> If it's more or less impossible for you to know how and where your code
>> is being called, I think that is a (perhaps the) problem.
>
> Yes in a very large multi threaded app, with many developers, many
> OSs, many users, many distros, and many windows managers - I don't
> know when, where, and how our code is called at all times. The many
> OSs being the main problem - the order that events occur varies from
> OS to OS. The last segfault we had was structured like this
>
> hypertreelist -> edit ctrl -> modal dialog -> list ctrl.
>
> A double click in the list ctrl closed the dialog (EndModal). But for
> some as yet unexplained reason on win32 the double click event was
> converted into a item drag event in the hypertreelist when the mouse
> cursor was in just the right place. The simplified callstack was like
> this :
>
> app.MainLoop -> edit_ctrl.on_edit_keydown -> dialog.ShowModal() ->
> hypertreelist.on_selection_changed -> edit_ctrl.Destroy()
>
> So apart from the fact that the Destroy was in agw code, not ours, how
> could you predict that a double click in a list ctrl in a modal dialog
> would cause a drag event in a hypertreelist *behind* the dialog which
> is still there? Easy to fix sure, hard to spot before it happens.

Does this stack trigger before or after the EndModal call? If it's after the EndModal call, it's probably just because you close on mouse down rather than mouse up, and the events right after that mouse down (which might include a small movement) probably get translated to the control below. Or, if it's before EndModal, this could even be a bug in the wx toolkit itself. The full stack trace would help figure that out, as would a small reproducible sample if it's reproducible.

> And no-one did anything inherently wrong here. Why shouldn't an
> on_key_down bring up a dialog, and why shouldn't the hypertreelist
> delete it's own edit when the selection was changed?

The answer to that question, as is always the case, lies in the details, and by that I mean the precise details rather than a simplified version. :)

Well, it really isn't that simple, but at this point I think it's probably just best to wait for the patch. :)

By asynchronous, I mean that Destroy takes effect at some uncertain time unknowable to the developer. That is how I understand asynchronous.

Regards,

Kevin

> Amyway, we'll implement our fix and submit a patch. I hope it'll be
> accepted, but either way we'll use it and our application will be more
> stable as a result.
>
> Sam
>

[Kevin Ollivier]

Hi Peter,

On Sep 20, 2012, at 5:05 PM, Peter Williams wrote:

[snip]

>> It's nothing to me if one used decided wx is not for him/her, but it'd
>> be a shame if it really was a fixable issue.
>>
>>>> This means things are worse than I thought. wxPython programs aren't Python programs they're C++ programs written in Python syntax. One of Python's key advantages has been destroyed and one of C++'s many problems has been imported. :-(
>>
>> Sad, but maybe unavoidable -- I've used Pyton to wrap the occasional C
>> and C++ code, and unless your C or C++ has a reference counting scheme
>> in place, there is going to be a bit of an impedance mismatch -- maybe
>> GTK does use a reference counting scheme that PyGTK ties into?
>
> I'm wondering if there isn't some documentation a python.org that explains how to do this correctly as an awful lot of people seem to have got it right. e.g. a lot of the standard Python library is wrappers around C code for performance reasons.

Doesn't segfault for you != cannot segfault, in the same way that "drove to work without crashing for 5 years straight" != "unable to crash when driving to work". See: http://wiki.python.org/moin/CrashingPython

However, if the Destroy thing is a deal-breaker for you, then there's not much point in continuing these discussions. We cannot use Python reference counting for wxWindows without a substantial rewrite of the toolkit, so unless you've got enough money to hire a couple full-time programmers to totally redo the memory management for the wxWidgets toolkit at the C++ level (and you'd probably need to fork it in the process and get a QA staff to debug all that reworking), it's not going to turn into what you want it to be.

[Peter Williams]

On 09/21/2012 08:18 AM, Sam Partington wrote:
> Hello Kevin,
>
> On 20 September 2012 17:17, Kevin Ollivier <kev...@theolliviers.com> wrote:
>>> On 20 September 2012 06:20, Kevin Ollivier <kev...@theolliviers.com> wrote:
>>> Any extension can crash. No extension should crash. I have no idea
>>> what you mean by 'closer to the metal', it sounds like a cop out to
>>> me. "Our extension is too complex to make it safe".
>>
>> There is a reason there are no pure Python GUIs. All the safety checks
>> Python runs make it impossible to write code that needs to be as fast
>> and tight as GUI code needs to be. You can call that a cop out if you like
>> but it's a plain fact. Slowing down Destroy() is probably not a huge deal,
>> but if you really want wxPython never to be able to crash under any
>> circumstance, you will indeed need to slow it to the point where it is no
>> longer usable speed-wise.
>
> Well I don't buy that personally. I don't think that there are no
> pure python apis because of speed, IMO it's because a gui api needs to
> interface with the OS a lot. Most OS apis are C/C++. Objective C and
> pyobjc is interesting because that is very nearly a pure python GUI
> API as it stands. The performance is quite acceptable.

Well said.

> Java has some
> more-or-less pure java UI's, and I don't think the performance of
> python is significantly worse than Java.

Java and Python use similar technologies, ie. virtual machine and byte
code, and should have similar performance. But I prefer Python's syntax.

>
> Actually I have started a wxPython clone in pure python, using ctypes
> to call win32 and gtk. But it is still very immature at the moment,
> and I've not even started on Mac OS yet. I've not measured speed yet
> mind you. But I wouldn't even be that surprised if a pure python ui
> running under PyPy it might easily outperform a CPython version with a
> C extension and all the nasty thread locking that has to go on.

Very interesting. I was thinking along similar lines myself last night.
My first thought was "wxPython is clearly not going to be fixed so
maybe I should fork a version and fix it myself" then I thought "there's
other parts of the wxPython that I don't like but was going to put up
with so why bother - why not just put a higher level wrapper on PyGTK".

Are you looking for volunteers?

>
> Nor do I believe it is impossible to make wxPython considerably more
> stable than it is now.

I agree but don't think it will happen because the power brokers don't
seem to see that there's even a problem.

Good luck
Peter

[Peter Williams]

On 09/21/2012 10:05 AM, Peter Williams wrote:
>
> I'm wondering if there isn't some documentation a python.org that
> explains how to do this correctly as an awful lot of people seem to have
> got it right. e.g. a lot of the standard Python library is wrappers
> around C code for performance reasons.
>
> Peter

<http://docs.python.org/c-api/>
<http://docs.python.org/extending/index.html#extending-index>

Even though it says 'c-api' it covers C++.

Cheers
Peter
PS I love the quality of Python's documentation.

[Peter Williams]

Good news. PyGTK's documentation has improved markedly since last time
I read it (and the interface is becoming higher level) so going back to
PyGTK won't be as onerous as I'd expected. :-)

Cheers
Peter

[Werner]

Hi Peter,

On 21/09/2012 06:04, Peter Williams wrote:
>
> Good news. PyGTK's documentation has improved markedly since last
> time I read it (and the interface is becoming higher level) so going
> back to PyGTK won't be as onerous as I'd expected. :-)

So what is/was the point of this thread? Just stir things up a bit or
are you really switching UI kits like you change your .....?

Werner

P.S.
I am just a low level user of wxPython

[Robin Dunn]

On 9/20/12 10:37 AM, Chris Barker wrote:
> On Wed, Sep 19, 2012 at 10:05 PM, Kevin Ollivier
> <kev...@theolliviers.com> wrote:
>> As you describe the behavior you see, as designed you should just get a
>> PyDeadObject error rather than a segfault. Why that is happening, well, I can't say
>> because we have no code to try and test against.
>
> This may have nothing to do with this particular issue, but I recall a
> while back a thread on this list that surrounded the fact that users
> on certain Linux distros were getting a LOT of hard crashes, whereas
> most of us get None.
>
> It turns out it was because of how wx was compiled -- I don't know the
> details, but wxPython assumes that wx is compiled (in debug mode?) in
> a way that some of this kind of dynamic checking goes on, rather than
> a hard crash -- exactly to help with teh impedance mis-match between a
> static and dynamic language.
>
> Maybe that's the issue here?

Probably. Or at least part of the problem.

For the record when wxWidgets 2.8 is configured with --enable-debug_flag
then the code includes a whole bunch of debugging macros for assertions,
precondition checks, etc. and I'm able to catch these and turn them into
Python exceptions. When compiled without that option then those become
no-ops in the compiled code and the things that they were intended to
protect against are not blocked and the code will likely crash
eventually. In Debian/Ubuntu's great wisdom their policy requires that
things like that be left turned off so it can be a few milliseconds
faster, so using their packages during development is a very bad idea
IMNSHO because simple errors that would be caught and warned about are
simply ignored until they cause a crash. There are some wx -dbg
packages available for Ubuntu (at least there used to be) so I would try
using that and see if you get any runtime assertions triggered. Or
build wxPython yourself with the option enabled.

In wxWidgets 2.9 the equivalent of the --enable-debug_info is turned on
by default, and you have to use a different flag to explicitly turn it
off. The wxWidgets team is recommending that it not be turned off
except for situations like embedded devices where memory constraints are
tight. Now we just have to convince Ubuntu/Debian to start providing
packages for 2.9...

[Robin Dunn]

On 9/20/12 4:43 PM, Peter Williams wrote:
> On 09/20/2012 03:05 PM, Kevin Ollivier wrote:

>> Perhaps it doesn't for you, but I've never figured out what precisely
>> it does
> > wrong in your case.
>
> At the moment, you have a situation where if you call Destroy() on some
> widgets you will get a segmentation fault when the GC tries to release
> the same memory and if you don't call it on some other widgets you will
> get a memory leak because the GC doesn't know about them.

Uh, no. The problems we've been discussing arise when you tell a widget
to destroy itself while C++ is still using that pointer, such as what it
might do after returning from an event handler or if there are other
posted events waiting for it in the pending event queue. Nothing on the
Python side cares about it because when the C++ object is destroyed the
python proxy object will have its __class__ changed to one that raises
the dead object exception. And in the cases when the proxy object is
GC'd before the C++ object is destroyed there is still no conflict
because unless there is a bug in the wrapper's constructor then the
proxy will be told that it doesn't own the C++ object so it will not try
to destroy it when it is GC'd.

>> As you describe the behavior you see, as designed you should just get
>> a PyDeadObject
> > error rather than a segfault.
>
> That's what you'll get if you release memory the GC knows about while
> there are still references to it. The segmentation faults occur if the
> reference you used for the Destroy() was the last reference and the GC
> tries to release when that reference is broken.

No. See above.

>
> It is precisely this attitude to the problem that has made me realize
> that the problem will never be fixed properly and PyGTK is the easier
> path in spite of its more difficult low level interface.

Never say never. Although it is using a similar model, as I mentioned
the other day Phoenix will have a much better support for dealing with
object ownership transfers and better runtime support.

[Jonathan Morgan]

As far as faulthandler goes, for the record I added it in a wxPython app I was maintaining which was principally distributed using py2exe.  It doesn't work with py2exe's stderr because py2exe is trying to trap stderr and log it elsewhere without using an OS-level file handle.  I believe this could be fixed, but as a result I just didn't enable faulthandler for the py2exe distribution (which made it not very useful).

Jon

[Kevin Ollivier]

Hi Jonathan,

On Sep 21, 2012, at 7:58 AM, Jonathan Morgan wrote:

As far as faulthandler goes, for the record I added it in a wxPython app I was maintaining which was principally distributed using py2exe.  It doesn't work with py2exe's stderr because py2exe is trying to trap stderr and log it elsewhere without using an OS-level file handle.  I believe this could be fixed, but as a result I just didn't enable faulthandler for the py2exe distribution (which made it not very useful).

Yeah, that's a long-standing issue with py2exe that I wish they would fix, because it wants to create the log file in the application's directory, which under newer versions of Windows causes problems due to the user not having permissions to modify folders inside C:\\Program Files unless you run the app as admin.

The redirect happens in py2exe bootstrapping code though, so you can (and probably should) redirect it again to wherever you would like the output to go. I'm going to play with this for some apps I work on shortly, actually, as it would be much nicer to throw up some quick prompt saying "Attach this log file and send email to what...@address.com to file a bug report" when a crash gets triggered somehow.

Regards,

Kevin

[Chris Barker]

Peter,

You've taken a very aggressive an critical stance in this thread --
it's a credit to everyone that no one has taken the bait and turned
this into a flame war (this really is a great group!), but please try
to remember that this is an open-source project, with, in fact, very
few folks that contribute) or even understand) the low-level issues at
hand, and try to be a bit nicer.

Regardless of folks being defensive of their work, but think we do all
take it as a given the the ideal is that a user should NEVER be able
to trigger a hard crash with pure python code. However, attaining that
ideal can be a challenge -- we all know that at some level one is
pushing bytes around -- wx is a C++ library, and hard crashes and
memory leaks are simply part of the deal with C++.

No matter how you slice it, there is an impedance mismatch between C==
and Python, and yes, there are recommended and common ways to deal
with memory management when calling C++ from Python, but there is no
universal cure -- it all depends on how the C++ lib is designed, and
wx is designed in a way that makes this particular issue a real
challenge.

One of the ways that this challenge as been addressed it to add some
run-time checking in -- this would be a very common and recommended
way to deal with it in C++. Rather than add a bunch of custom code to
wx (which might be a good idea, but a lot of work), Robin has taken
advantage of compiler features -- see his note for details.

>> On Wed, Sep 19, 2012 at 10:05 PM, Kevin Ollivier
>> <kev...@theolliviers.com> wrote:
>>>
>>> As you describe the behavior you see, as designed you should just get a
>>> PyDeadObject error rather than a segfault. Why that is happening, well, I
>>> can't say
>>> because we have no code to try and test against.

This is a really good point -- whereas the core devs have acknowledged
that the whole Destroy() thing is ugly and prone to problems, you
really do seem to be be getting more crashes than most folks, so maybe
you indeed are seeing a bug that could be addressed -- but you need to
post some simple code that shows your problem so that we can all test
ans see what the issue really is in this case.

>> It really strikes me that Peter is experiencing a really different
>> environemnt than the rest of -- so he's either doing something
>> different:
>> - A far more dynamic GUI than most?
>
> Yes.

So that explains why you've run into this issue so many times.

>> or there is something wrong in the build he's using.
>
> More denial. The problem is in the way that wxPython is implemented.

Please don't be so condescending -- if you read Robin's note -- it
looks like you may indeed be using an improper build -- wxPython is
implemented to be used against a wx lib built a certain way -- if
you're using a wx lib built differently, it really is the build's
fault. You may prefer that wx had hand-written all that run-time
checking, but the current solution is a good one, and works well, with
the only cost a little performance hit (and maybe slightly larger
binaries).

On most systems (Windows, OS-X, etc) most of us use binaries shipped
with wxPython, and therefor built the way it is intended to be used.
The fact that the Ubuntu folks didn't understand the issue is
unfortunate.

Give a different build a try -- you may be much happier.

[Peter Williams]

On 09/21/2012 05:06 PM, Werner wrote:
> Hi Peter,
>
> On 21/09/2012 06:04, Peter Williams wrote:
>>
>> Good news. PyGTK's documentation has improved markedly since last
>> time I read it (and the interface is becoming higher level) so going
>> back to PyGTK won't be as onerous as I'd expected. :-)
> So what is/was the point of this thread? Just stir things up a bit or
> are you really switching UI kits like you change your .....?

I was trying to get wxPython fixed so that it didn't break Python so
that I could use it instead of PyGTK. I've now given up on that dream
and am returning to PyGTK (from whence I came) and was just reporting
that one of the reasons I'd left PyGTK for wxPython (i.e. documentation
availability) seems to have improved while I've been away.

So it's more like dumping the new girlfriend and moving back in with the
wife :-)

[Peter Williams]

On 09/22/2012 02:44 AM, Chris Barker wrote:
> Peter,
>
> You've taken a very aggressive an critical stance in this thread --
> it's a credit to everyone that no one has taken the bait and turned
> this into a flame war (this really is a great group!), but please try
> to remember that this is an open-source project, with, in fact, very
> few folks that contribute) or even understand) the low-level issues at
> hand, and try to be a bit nicer.

I started out being nice and changed when the responses were so negative.

>
> Regardless of folks being defensive of their work, but think we do all
> take it as a given the the ideal is that a user should NEVER be able
> to trigger a hard crash with pure python code. However, attaining that
> ideal can be a challenge -- we all know that at some level one is
> pushing bytes around -- wx is a C++ library, and hard crashes and
> memory leaks are simply part of the deal with C++.

You still don't understand the issue which is why I've been more
aggressive. To put it bluntly:

"wx breaks Python - specifically the garbage collector".

>
> No matter how you slice it, there is an impedance mismatch between C==
> and Python, and yes, there are recommended and common ways to deal
> with memory management when calling C++ from Python, but there is no
> universal cure -- it all depends on how the C++ lib is designed, and
> wx is designed in a way that makes this particular issue a real
> challenge.

Python, itself, is a wrapper around C code.

>
> One of the ways that this challenge as been addressed it to add some
> run-time checking in -- this would be a very common and recommended
> way to deal with it in C++. Rather than add a bunch of custom code to
> wx (which might be a good idea, but a lot of work), Robin has taken
> advantage of compiler features -- see his note for details.

It just needs some low level (in the SWIG code) fixing of the way memory
is allocated and it will all be hunky dory.

>
>>> On Wed, Sep 19, 2012 at 10:05 PM, Kevin Ollivier
>>> <kev...@theolliviers.com> wrote:
>>>>
>>>> As you describe the behavior you see, as designed you should just get a
>>>> PyDeadObject error rather than a segfault. Why that is happening, well, I
>>>> can't say
>>>> because we have no code to try and test against.
>
> This is a really good point -- whereas the core devs have acknowledged
> that the whole Destroy() thing is ugly and prone to problems, you
> really do seem to be be getting more crashes than most folks, so maybe
> you indeed are seeing a bug that could be addressed -- but you need to
> post some simple code that shows your problem so that we can all test
> ans see what the issue really is in this case.

In the trade, this is known as "papering over the cracks".

>
>>> It really strikes me that Peter is experiencing a really different
>>> environemnt than the rest of -- so he's either doing something
>>> different:
>>> - A far more dynamic GUI than most?
>>
>> Yes.
>
> So that explains why you've run into this issue so many times.

So this means it shouldn't be fixed?

>
>>> or there is something wrong in the build he's using.
>>
>> More denial. The problem is in the way that wxPython is implemented.
>
> Please don't be so condescending -- if you read Robin's note -- it
> looks like you may indeed be using an improper build -- wxPython is
> implemented to be used against a wx lib built a certain way -- if
> you're using a wx lib built differently, it really is the build's
> fault. You may prefer that wx had hand-written all that run-time
> checking, but the current solution is a good one, and works well, with
> the only cost a little performance hit (and maybe slightly larger
> binaries).
>
> On most systems (Windows, OS-X, etc) most of us use binaries shipped
> with wxPython, and therefor built the way it is intended to be used.
> The fact that the Ubuntu folks didn't understand the issue is
> unfortunate.
>
> Give a different build a try -- you may be much happier.

More evidence that you do not understand the problem. This is why I'm
upset and also why I see no hope of a proper fix ever occurring.

Good luck with the ad hoc bug fixing
Peter

[Kevin Ollivier]

On Sep 21, 2012, at 4:11 PM, Peter Williams wrote:

[snip]

>>

>> One of the ways that this challenge as been addressed it to add some
>> run-time checking in -- this would be a very common and recommended
>> way to deal with it in C++. Rather than add a bunch of custom code to
>> wx (which might be a good idea, but a lot of work), Robin has taken
>> advantage of compiler features -- see his note for details.
>
> It just needs some low level (in the SWIG code) fixing of the way memory is allocated and it will all be hunky dory.

I don't really think there is much point in continuing these discussions. Bottom line: You and/or Sam submit a patch and prove how little we understand. What people here have been trying to do with our 'negativity' is to help you guys understand the issues involved with creating wrappers so that we can all come to the best solution here. However, the response to our attempts has consistently been that we don't know what the heck we're talking about and that wrappers like this are all a very simple business indeed. Every attempt to explain that mixing two memory management paradigms requires some care and occasionally some compromise has been met with replies of 'excuses', 'rubbish', etc.

So, if you're really right, you should easily be able to fix the issue once and for all with a patch that will make wxPython unable to segfault, and not introduce any bugs or regressions in the process. I'd love to see that. If one or perhaps both of you guys can come up with such a patch, it will certainly improve the wx project, and will almost certainly be accepted, again provided it does not introduce any bugs or regressions. I personally don't mind being wrong, if the project benefits. :)

Regards,

Kevin

[Chris Barker]

On Fri, Sep 21, 2012 at 4:11 PM, Peter Williams <pwil...@gmail.com> wrote:
> "wx breaks Python - specifically the garbage collector".

That may actually be true -- IIUC, the problem at hand is wx's model
for managing the destruction of Windows -- i.e. the C++ wx library
itself, NOT the Python wrappers. We could debate all day about whether
that was/is a good design, but in any case, that's not the issue, the
question is what we ca do about it:

A) dump wx and use something else -- perhaps GTK has an inherently
better memory management model -- I have no idea.

B) refactor the C++ wx libs -- maybe a great idea, but a LOT of work.

C) find a work-around by doing some run-time checking -- indeed that
is EXACTLY what Robin has done, but it only works if the libs are
built the right way for that -- I have no idea why you seem to
unwilling to give that a try. What I can tell you is that I haven't
had a wx hard crash in many years -- so something is up.

> Python, itself, is a wrapper around C code.

No, it's not the cPython interpreter is written in C, but it's C that
was designed for, indeed defines, the memory management model. wx was
not originally designed to be used with Python -- huge difference.

>> One of the ways that this challenge as been addressed it to add some
>> run-time checking in -- this would be a very common and recommended
>> way to deal with it in C++. Rather than add a bunch of custom code to
>> wx (which might be a good idea, but a lot of work), Robin has taken
>> advantage of compiler features -- see his note for details.
>
> It just needs some low level (in the SWIG code) fixing of the way memory is
> allocated and it will all be hunky dory.

It seems your an my understanding of the problem is different -- I
don't think the wrappers are the issue here, I don't think the
allocation and deletion of memory by the wrapper is the issue, the
issue is the allocation and deletion by the C++ code -- that's where
the fix would need to be. i.e if wx.Windows used a reference counting
scheme, rather than direct deletion by user code. But, again, that
would be major refactor.

>> This is a really good point -- whereas the core devs have acknowledged
>> that the whole Destroy() thing is ugly and prone to problems, you
>> really do seem to be be getting more crashes than most folks, so maybe
>> you indeed are seeing a bug that could be addressed -- but you need to
>> post some simple code that shows your problem so that we can all test
>> ans see what the issue really is in this case.
>
>
> In the trade, this is known as "papering over the cracks".

huh? You are reporting fatal bugs, but you have not posted test code
that crashes for you that anyone else can test and see if it crashes
for them -- either it would, in which case we could debug and make
sure we understand the problem, or it wouldn't in which case we could
try to figure out how your system is different. Instead you are making
educated guesses about the problem, and then blaming the devs for not
saying they'll fix it.

How the heck is that papering over the cracks?

>> So that explains why you've run into this issue so many times.
>
> So this means it shouldn't be fixed?

no -- it means that may b the only difference between your results and
the rest of us -- i.e. you are pushing the system and revealing bugs
(or design flaws), but that may also not be the case. If you didn't
have an unusually dynamic GUI I would say you definitely have
something wrong with your build. wx has its flaws, but it's really not
that unreliable. But maybe you really are just pushing the system
harder than everyone else.

>> Give a different build a try -- you may be much happier.
>
> More evidence that you do not understand the problem.

How the heck are we supposed to understnd the problem when you haven't
posted code us to try?

And I really think you are misunderstanding one possible problem --
again don't know if this is it, because you haven't tested, or given
us something to test, but, one more time:

IF YOU ARE RUNNING WITH A BUILD OF WX THAT WAS NOT BUILT WITH
--enable-debug_flag YOU WILL EXPERIENCE HARD CRASHES THAT COULD BE
AVOIDED.

And Ubuntu is known to build wx that way (at least in some versions).

You may think this solution is a hack, but it is, none the less, a
solution that you don't appear to be willing to try. Fine, go use
PyGTK, but don't post on here that wx has issues that no one wants to
fix when you haven't tried a solution offered.

Just as an analogy, if Ubuntu were to build python with PyFree()
replaced with a no-op -- would you use it and say that Python is crap
because it's full of memory leaks?

> This is why I'm upset
> and also why I see no hope of a proper fix ever occurring.

Of course, patches for proper fixes are always welcome.

[Peter Williams]

My point is that fixing it in the SWIG layer so that the GC knows about
all the allocations and then leaving it to the GC to manage memory is
the easiest solution. My understanding is that the Python to C/C++
interface can do this if all memory allocation is on the heap (not the
stack).

>
>>> This is a really good point -- whereas the core devs have acknowledged
>>> that the whole Destroy() thing is ugly and prone to problems, you
>>> really do seem to be be getting more crashes than most folks, so maybe
>>> you indeed are seeing a bug that could be addressed -- but you need to
>>> post some simple code that shows your problem so that we can all test
>>> ans see what the issue really is in this case.
>>
>>
>> In the trade, this is known as "papering over the cracks".
>

> huh? You are reporting fatal bugs but you have not posted test code

> that crashes for you that anyone else can test and see if it crashes
> for them -- either it would, in which case we could debug and make
> sure we understand the problem, or it wouldn't in which case we could
> try to figure out how your system is different. Instead you are making
> educated guesses about the problem, and then blaming the devs for not
> saying they'll fix it.
>
> How the heck is that papering over the cracks?

Reading the activity reported in your bug tracking system is where I
formed this opinion. My problem is just another one on the end of a
long list of similar problems e.g. the changes to Sizer.Remove() to
avoid segmentation faults.

>
>>> So that explains why you've run into this issue so many times.
>>
>> So this means it shouldn't be fixed?
>
> no -- it means that may b the only difference between your results and
> the rest of us -- i.e. you are pushing the system and revealing bugs
> (or design flaws), but that may also not be the case. If you didn't
> have an unusually dynamic GUI I would say you definitely have
> something wrong with your build. wx has its flaws, but it's really not
> that unreliable. But maybe you really are just pushing the system
> harder than everyone else.
>
>>> Give a different build a try -- you may be much happier.
>>
>> More evidence that you do not understand the problem.
>
> How the heck are we supposed to understnd the problem when you haven't
> posted code us to try?

I've told you what the problem is. Any first year computer science
student would understand what I'm talking about.

>
> And I really think you are misunderstanding one possible problem --
> again don't know if this is it, because you haven't tested, or given
> us something to test, but, one more time:
>
> IF YOU ARE RUNNING WITH A BUILD OF WX THAT WAS NOT BUILT WITH
> --enable-debug_flag YOU WILL EXPERIENCE HARD CRASHES THAT COULD BE
> AVOIDED.
>
> And Ubuntu is known to build wx that way (at least in some versions).
>
> You may think this solution is a hack, but it is, none the less, a
> solution that you don't appear to be willing to try. Fine, go use
> PyGTK, but don't post on here that wx has issues that no one wants to
> fix when you haven't tried a solution offered.
>
> Just as an analogy, if Ubuntu were to build python with PyFree()
> replaced with a no-op -- would you use it and say that Python is crap
> because it's full of memory leaks?
>
>> This is why I'm upset
>> and also why I see no hope of a proper fix ever occurring.
>
> Of course, patches for proper fixes are always welcome.
>

One last try. By forcing the programmer to keep track of various
objects and manually release their memory you have broken Python turning
it into C++ with Python syntax. As C++ is one of the worst programming
languages ever invented this is not a good thing.

Good bye
Peter
PS I have resisted the temptation to air this issue to a wider audience.

[David Holl]

Could you please just attach some sample code demonstrating the crash?  (not a partial code snippet, but a whole .py file where one might simply reproduce the undesirable behavior via "python blah.py")

On Fri, Sep 21, 2012 at 11:49 PM, Peter Williams <pwil...@gmail.com> wrote:

On 09/22/2012 01:04 PM, Chris Barker wrote:

On Fri, Sep 21, 2012 at 4:11 PM, Peter Williams <pwil...@gmail.com> wrote:

On ...

 

- David

[Chris Mellon]

Your understanding is incorrect, there are no such interfaces. In fact
Python doesn't even *have* a GC, and you *must* manually manage memory
when you inter-operate between C and Python. Nobody who has ever
written anything using the Python C api would make the mistake you're
making here.

I'm afraid this statement has no basis in reality, and no
correspondence whatsoever with good programming practice. Object
lifetime is the responsibility of the programmer and cannot be
otherwise. For a case study, you need look no farther than the file
object in Python. Incidentally, it also demonstrates a fundamental
misunderstanding of the problem you had, which is caused by attempting
to keep track of objects and manually destroy them (not just release
their memory, which is different. CS201 topic, though). If you didn't
do this, you would have no errors.

> Good bye
> Peter
> PS I have resisted the temptation to air this issue to a wider audience.
>

Oh my goodness.

[Kevin Ollivier]

Hi Chris,

On Sep 22, 2012, at 3:57 AM, Chris Mellon wrote:

[snip]

> Your understanding is incorrect, there are no such interfaces. In fact
> Python doesn't even *have* a GC, and you *must* manually manage memory
> when you inter-operate between C and Python. Nobody who has ever
> written anything using the Python C api would make the mistake you're
> making here.

BTW, Python did add GC functionality in version 2.0, which can actually be turned on or off via the gc module, but you're right that it's not what Python's memory management is based around. (Core memory management functionality clearly is not a feature you add in v2.0 of your product ;-)

Regards,

Kevin

[Sam Partington]

On 22 September 2012 01:01, Kevin Ollivier <kev...@theolliviers.com> wrote:
>
> On Sep 21, 2012, at 4:11 PM, Peter Williams wrote:
>
> [snip]

> I don't really think there is much point in continuing these discussions. Bottom
> line: You and/or Sam submit a patch and prove how little we understand. What
> people here have been trying to do with our 'negativity' is to help you guys
> understand the issues involved with creating wrappers so that we can all come
> to the best solution here. However, the response to our attempts has
> consistently been that we don't know what the heck we're talking about and that
> wrappers like this are all a very simple business indeed. Every attempt to explain
> that mixing two memory management paradigms requires some care and
> occasionally some compromise has been met with replies of 'excuses', 'rubbish',
> etc.

Kevin,

Peter and I are not on one side, and the whole wx community on the
other here as you suggest.

I am suggesting that I can provide a patch for a specific problem -
that is a crash caused when destroy takes place on a window which has
an active event handler. The negativity I experienced was "you can't
fix it, and even if you can we won't accept it". Which I found rather
frustrating, as I was not saying "this is broke you need to fix it"
but "this is broke and I will try to fix it". If my company wasn't so
reliant on wx then at that point I would have been tempted to walk
away at that point.

My 'excuse, not a solution' comment was in response to your assertion
that Destroy may crash, that's obvious because it's a C++ extension
and C++ crashes some times, and it seemed to me that you were
advocating that it was ok for it to crash. When I was actively trying
to find a solution to prevent that crash.

On the other hand, Peter is suggesting that the entire model of
handling memory management is wrong in wx. I actually think he is
probably right, but I think that he is wrong that it can be fixed
easily in the swig layer. To fix that would require some major
modifications to the whole wx model and I think it would be extremely
difficult to do, though I do hope to try one day if no-one else does.

> So, if you're really right, you should easily be able to fix the issue once and for all with
> a patch that will make wxPython unable to segfault, and not introduce any bugs or
> regressions in the process. I'd love to see that. If one or perhaps both of you guys
> can come up with such a patch, it will certainly improve the wx project, and will
> almost certainly be accepted, again provided it does not introduce any bugs or
> regressions. I personally don't mind being wrong, if the project benefits. :)

I am not so naive to think that I can make it unable to segfault. But
I must try to make it less likely to segfault if I can.

It is however good to hear that you have changed your mind and will
consider accepting a patch.

Sam

[Kevin Ollivier]

Hi Sam,

On Sep 24, 2012, at 2:07 AM, Sam Partington wrote:

> On 22 September 2012 01:01, Kevin Ollivier <kev...@theolliviers.com> wrote:
>>
>> On Sep 21, 2012, at 4:11 PM, Peter Williams wrote:
>>
>> [snip]
>> I don't really think there is much point in continuing these discussions. Bottom
>> line: You and/or Sam submit a patch and prove how little we understand. What
>> people here have been trying to do with our 'negativity' is to help you guys
>> understand the issues involved with creating wrappers so that we can all come
>> to the best solution here. However, the response to our attempts has
>> consistently been that we don't know what the heck we're talking about and that
>> wrappers like this are all a very simple business indeed. Every attempt to explain
>> that mixing two memory management paradigms requires some care and
>> occasionally some compromise has been met with replies of 'excuses', 'rubbish',
>> etc.
>
> Kevin,
>
> Peter and I are not on one side, and the whole wx community on the
> other here as you suggest.
>
> I am suggesting that I can provide a patch for a specific problem -
> that is a crash caused when destroy takes place on a window which has
> an active event handler. The negativity I experienced was "you can't
> fix it, and even if you can we won't accept it".

No, what we said was "the way you are trying to fix the crash won't work". The problem is that our explanations for why it won't work are either ignored, passed off as not valid reasons, or even labeled as excuses, and any alternatives we suggest get branded as hacks and workarounds. While your and Peter are clearly very different, in this one (important) way your reaction has been largely the same.

> Which I found rather
> frustrating, as I was not saying "this is broke you need to fix it"
> but "this is broke and I will try to fix it". If my company wasn't so
> reliant on wx then at that point I would have been tempted to walk
> away at that point.
>
> My 'excuse, not a solution' comment was in response to your assertion
> that Destroy may crash, that's obvious because it's a C++ extension
> and C++ crashes some times, and it seemed to me that you were
> advocating that it was ok for it to crash. When I was actively trying
> to find a solution to prevent that crash.

No one has yet advocated that it is "okay" for it to crash. To paraphrase what Python's BDFL said about crashes, "it's not that Python is uncrashable, but we consider it a bug unless the user is doing something they shouldn't". I think that's a good outlook and pretty accurate to both how Python and wxPython view and tackle crashes.

What people have been saying is that due to the nature of what the function does, if you call it when you shouldn't, it is impossible to keep it from crashing because it deletes memory. That is, if it's crashing, how it's being called is a bug of some form or another.

I think your assertion that Destroy is in the wrong comes from what you said earlier about "nobody doing anything wrong in the code", but the behavior you described in your case was clearly wrong. (The unresolved question is as to what precisely triggered the wrong behavior.) Even if Destroy didn't crash in that case, some action the user did not intentionally initiate would have fired, and some change the user did not intend to make would have happened. To the user, incorrect behavior is incorrect behavior, regardless of the form. fault handler would be the best approach here - give you a segfault to clearly see the code path that fired so you can diagnose what went wrong.

> On the other hand, Peter is suggesting that the entire model of
> handling memory management is wrong in wx. I actually think he is
> probably right, but I think that he is wrong that it can be fixed
> easily in the swig layer. To fix that would require some major
> modifications to the whole wx model and I think it would be extremely
> difficult to do, though I do hope to try one day if no-one else does.

What we've been saying is that your idea that memory management is 'wrong' as a paradigm in wx is based on the flawed assumption that because Python does memory management well, for other applications to do memory management well, they must use an approach much like (or the same as) Python's. The model wx uses for its widgets is actually a very commonly chosen one for GUI objects and is regarded by many toolkits (not just ours) to be a good memory management approach for this scenario.

The issue your outlook causes is that your solutions to the problems you're seeing all are viewed through this 'broken memory management' prism, and so you and Peter have both become absolutely convinced the problem is wx's memory management of GUI objects and the fix must address that. You and Peter differ in that you're more pragmatically suggesting a localized treatment (for now at least) involving the function that triggers the crash.

>> So, if you're really right, you should easily be able to fix the issue once and for all with
>> a patch that will make wxPython unable to segfault, and not introduce any bugs or
>> regressions in the process. I'd love to see that. If one or perhaps both of you guys
>> can come up with such a patch, it will certainly improve the wx project, and will
>> almost certainly be accepted, again provided it does not introduce any bugs or
>> regressions. I personally don't mind being wrong, if the project benefits. :)
>
> I am not so naive to think that I can make it unable to segfault. But
> I must try to make it less likely to segfault if I can.

And also not introduce any changes in the behavior of Destroy which may lead to breaking existing code. Unfortunately, a change to when Destroy actually takes effect stands to potentially break at lest some amount of code using it, and trigger at least some unexpected results. The worst part is that, similar to using threads, changing behavior from synchronous to asynchronous will not break everything, it will just break some things some of the time, and often we won't know precisely how until the bug / crash occurs. Asynchronous bugs are the hardest to find and to fix, so they should be avoided when possible.

> It is however good to hear that you have changed your mind and will
> consider accepting a patch.

No one ever said a patch would not be considered, it's just that based on your description of how you intend to tackle the problem, we already have an idea of what results the approach you are taking will have. For the toolkit as a whole it will probably be a net minus as it will cause existing Destroy calls to behave in a somewhat unpredictable manner, but as I said above it's anybody's guess as to exactly how the issues might manifest them.

The only real difference between your approach and the "workarounds" Robin suggested is that your approach tries to hide the workaround from the user by building it into the Destroy call. Your and Robin's approaches essentially do the same thing, it's just that Robin's suggestion means that everyone that reads the code realizes what it is doing and what to expect by doing it that way. If you've coded for a long time, you should realize that is a plus. :)

You really underestimate the amount of time and thought put into these issues by people who have been working on these wrappers for years and years. Particularly your suggestion that a ctypes-based wrapper would be "pure Python" and have safer memory management than wx strongly hints there are pieces of the memory management puzzle you are missing. In that "Crashing Python" wiki page I linked earlier, it is listed as a 'dangerous' module prone to triggering crashes, and for very good reason.

Kevin

> Sam

[Chris Barker]

On Fri, Sep 21, 2012 at 8:49 PM, Peter Williams <pwil...@gmail.com> wrote:

> My point is that fixing it in the SWIG layer so that the GC knows about all
> the allocations and then leaving it to the GC to manage memory is the
> easiest solution. My understanding is that the Python to C/C++ interface
> can do this if all memory allocation is on the heap (not the stack).

Then your understanding is incorrect -- if there is C/C++ code that
calls free() or delete() -- there is NOTHING that you can do in
wrapper code to prevent that.

>>>> -- but you need to
>>>> post some simple code that shows your problem so that we can all test
>>>> ans see what the issue really is in this case.

> Reading the activity reported in your bug tracking system is where I formed
> this opinion.

>> How the heck are we supposed to understnd the problem when you haven't
>> posted code us to try?
>
> I've told you what the problem is. Any first year computer science student
> would understand what I'm talking about.

That's fine -- we all acknowledge that Destroy() has problems -- but I
think they are in fact not as severe or as common as you think -- but
without knowing exactly what's crashing for you why, we can't
determine that. If every single call to .Destroy() resulted in a seg
fault, then we wouldn't need sample code, but they don't all result in
that. Indeed, from what we've seen of your description, we think you
should have gotten a PyDeadObjectError rather than a segfault, so
unless you post code that crashes for you that we can test, we have no
way of knowing.

Indeed, I just messed around a bit with Destroy(), and while I didn't
do much, I have no been able to get it to crash -- it's actually
better behaved than I expected -- I seem to be able to call methods on
objects ( via Callafter() ) after calling Destroy on them -- not sure
why that works. But in any case, when I try to Destroy() a Dialog
twice, I get a PyDeadObjectError, not a hard crash:

Traceback (most recent call last):
File "Destroy_test.py", line 28, in OnButton
self.Dialog.Destroy()
File "/usr/local/lib/wxPython-2.9.3.1/lib/python2.7/site-packages/wx-2.9.3-osx_cocoa/wx/_core.py",
line 16713, in __getattr__
raise PyDeadObjectError(self.attrStr % self._name)
wx._core.PyDeadObjectError: The C++ part of the Test_Dialog object has
been deleted, attribute access no longer allowed.

See enclosed, small, self-contained example. Does it cause a hard
crash for you when you press the button multiple times? If so then
there is either a bug in wxGTK that is not in wxMac, or your build is
incorrect.

> One last try. By forcing the programmer to keep track of various objects
> and manually release their memory you have broken Python turning it into C++
> with Python syntax. As C++ is one of the worst programming languages ever
> invented this is not a good thing.

nevertheless, wx is written in C++, so we have no choice but to deal with it.

[Sam Partington]

Kevin,

On 24 September 2012 17:37, Kevin Ollivier <kev...@theolliviers.com> wrote:
> No, what we said was "the way you are trying to fix the crash won't work".
> The problem is that our explanations for why it won't work are either
> ignored, passed off as not valid reasons, or even labeled as excuses, and
> any alternatives we suggest get branded as hacks and workarounds. While
> your and Peter are clearly very different, in this one (important) way your
> reaction has been largely the same.

I said that my own solution to my problem of CallAfter(CallLater...)
is a hack. I still think it is. The general case of
CallAfter(Destroy) I say is a work around. But an ok one. I think it
would be better if there to be able to do window.DestroyWhenSafe().
But I think it would be better still if there were just one Destroy
that was always safe if it is practical to have one.

The only time I mentioned excuse is when you said that Destroy may
crash because it's a C++ extension. You also said that python doesn't
have methods like Destroy, but it does: file.close is just such a
method. It doesn't segfault it you call close 'badly' even though it
is implemented in C.

>> My 'excuse, not a solution' comment was in response to your assertion
>> that Destroy may crash, that's obvious because it's a C++ extension
>> and C++ crashes some times, and it seemed to me that you were
>> advocating that it was ok for it to crash. When I was actively trying
>> to find a solution to prevent that crash.
>
> No one has yet advocated that it is "okay" for it to crash. To paraphrase
> what Python's BDFL said about crashes, "it's not that Python is
> uncrashable, but we consider it a bug unless the user is doing something
> they shouldn't". I think that's a good outlook and pretty accurate to both
> how Python and wxPython view and tackle crashes.

You're misquoting AND misrepresenting what the BDFL said here.
Misquoting: He said "hairbrained" not "something they shouldn't".

Misrepresenting: He was referring to things which were actually
trying to cause crashes, or things that noone would reasonably do in
normal code. Not something that I rather suspect some users do by
accident, because it looks perfectly reasonable until it crashes on
you. Also most of those reported cases are now fixed. If I can call
close on a file object in my write method then why shouldn't I call
Destroy in an event handler?

And you were saying it was ok to crash in this instance when you said
"Any Python extension can crash, this is just reality" and when you
said that a library fix wouldn't be accepted, and various other
similar statements about changing the behaviour of Destroy away from
segfaults.

> What people have been saying is that due to the nature of what the function does,
> if you call it when you shouldn't, it is impossible to keep it from crashing
> because it deletes memory. That is, if it's crashing, how it's being called is
> a bug of some form or another.

Destroy doesn't need to destroy memory, it could just release the OS
objects, and set it's internal state to the default constructed state,
and delete the memory when it safe to do so. Or wxWidgets could use
ref counted pointers throughout instead of passing around raw pointers
and hoping that the pointer is still valid when you used.

And just to be absolutely clear, I'm not saying that we should do
either of these things now, or even ever, but it is not impossible.

> I think your assertion that Destroy is in the wrong comes
> from what you said earlier about "nobody doing anything
> wrong in the code", but the behavior you described in your
> case was clearly wrong.
> (The unresolved question is as to what precisely triggered
> the wrong behavior.) Even if Destroy didn't crash in that
> case, some action the user did not intentionally initiate
> would have fired, and some change the user did not intend
> to make would have happened. To the user, incorrect
> behavior is incorrect behavior, regardless of the form.
> fault handler would be the best approach here - give you a
> segfault to clearly see the code path that fired so you can
> diagnose what went wrong.

Oh my, do you really think that? Instead of a crash the user would
have experienced an incorrect change of selection in the hypertreelist
ctrl, leaving them thinking 'oh well I must have done a triple click
instead of a double click'. Would that really be worse than the
application crashing and them losing potentially hours of work? Me,
I'd rather have a recoverable error condition any day.

But yes you're right, that behaviour was wrong and it does need to be
fixed, and it's in our bug tracker as urgent, to come back to when we
can. But the bug is somewhere deep in wx code as far as we can tell
and we've not had time to figure out how that message is getting
there. (In response to your question regarding that we were binding
the dbl click event, not the mouse down).

>> On the other hand, Peter is suggesting that the entire model of
>> handling memory management is wrong in wx. I actually think he is
>> probably right, but I think that he is wrong that it can be fixed
>> easily in the swig layer. To fix that would require some major
>> modifications to the whole wx model and I think it would be extremely
>> difficult to do, though I do hope to try one day if no-one else does.
>
> What we've been saying is that your idea that memory management is
> 'wrong' as a paradigm in wx is based on the flawed assumption that
> because Python does memory management well, for other applications
> to do memory management well, they must use an approach much
> like (or the same as) Python's. The model wx uses for its widgets is
> actually a very commonly chosen one for GUI objects and is regarded
> by many toolkits (not just ours) to be a good memory management
> approach for this scenario.

It is a good approach, but not necessarily the best for this task.
The java GUIs are all garbage collected because it matches the
language. wxWidgets works reasonably well in C++, although personally
I would much rather be in charge of the memory management more
explicitly. I don't know enough about PyQt or PyGTK to know what they
do.

Though it would be a pig to change, wxPython does not necessarily have
to use same model as wxWidgets, it could change.

> The issue your outlook causes is that your solutions to
> the problems you're seeing all are viewed through this
> 'broken memory management' prism, and so you and Peter
> have both become absolutely convinced the problem is wx's
> memory management of GUI objects and the fix must address
> that. You and Peter differ in that you're more
> pragmatically suggesting a localized treatment (for now at
> least) involving the function that triggers the crash.

Oh look, I am not absolutely convinced at all. i just said "I think
he[Peter] is probably right. I never said the fix must address memory
management. Peter did. Sam is not Peter.

>> It is however good to hear that you have changed your mind and will
>> consider accepting a patch.
>
> No one ever said a patch would not be considered,

You said "I do not think any library-level fix for this will be
accepted". Not a definite no, but pretty close, and very
discouraging.

> it's just that based
> on your description of how you intend to tackle the problem, we
> already have an idea of what results the approach you are taking will
> have. For the toolkit as a whole it will probably be a net minus as it
> will cause existing Destroy calls to behave in a somewhat unpredictable
> manner, but as I said above it's anybody's guess as to exactly how the
> issues might manifest them.
>
> The only real difference between your approach and the "workarounds" Robin
> suggested is that your approach tries to hide the workaround from the user by
> building it into the Destroy call. Your and Robin's approaches essentially do
> the same thing, it's just that Robin's suggestion means that everyone that
> reads the code realizes what it is doing and what to expect by doing it that
> way. If you've coded for a long time, you should realize that is a plus. :)

You clearly didn't even read what my approach is, again, which is the
singularly most frustrating thing about this thread for me. The
approach is to change *only* calls to Destroy when an active event
handler is in the callstack so only those calls which would definitely
have segfaulted would be affected. Detecting this is really
straightforward in event.cpp without any heuristics or guesswork and
practically no performance impact.

The question then is what to do when this is detected 1) defer the
destroy until the last event handler exits 2) raise an exception. I
lean towards 1 because it going to be harmless 99% of the time and
because in the last few issues we've had it would have been better
that way, but explicit is better than implicit so perhaps 2 is better.
It could easily be a runtime choice and allow the user to decide.

Has anyone read the Destroy docs BTW?

http://wxpython.org/docs/api/wx.Window-class.html#Destroy
"""
Destroy(self)

Destroys the window safely. Frames and dialogs are not destroyed
immediately when this function is called -- they are added to a list
of windows to be deleted on idle time, when all the window's events
have been processed. This prevents problems with events being sent to
non-existent windows.

Returns True if the window has either been successfully deleted, or it
has been added to the list of windows pending real deletion.
"""

You keep saying that it is ok for Destroy to be a bit dangerous, but
the docs most definitely do not agree with you "Destroys the window
SAFELY" is a big sign to me to "use when you like", not "use with
caution".

There is also enough ambiguity in the docs there for the window
destroy to be deferred a little bit.

> You really underestimate the amount of time and thought put into these
> issues by people who have been working on these wrappers for years and
> years.

Look now you're putting even more words in my mouth, I really really
don't underestimate it at all, I maintain several large wrappers
myself I know it's hard. I have a huge respect for the work that Robin
et al have done, and it is a really good piece of work but it is far
from perfect. Only a few months ago I posted a large patch because
the Python API was being called in literally hundreds of places
without the GIL being taken, and this is another case where it can be
improved, but it feels like you (and actually I mean only you, there
has been no negativity from anyone else) are fighting against it for
some reason I just don't understand.

> Particularly your suggestion that a ctypes-based wrapper would be
> "pure Python" and have safer memory management than wx strongly hints
> there are pieces of the memory management puzzle you are missing. In
> that "Crashing Python" wiki page I linked earlier, it is listed as a 'dangerous'
> module prone to triggering crashes, and for very good reason.

A pure python ctypes wrapper would solve the memory management
problem, of course memory management is only a small part of the
resource management problem, but it would be one way to marry up the
python/C methodologies in the same way, and of course it wouldn't be
really pure python, but we are talking about interfacing with the a C
OS, not PyOS. I have been writing C++ and python for 15 and 10 years
respectively, I think I have an idea about the issues involved.

Frankly I've had enough of being patronised like this when I am trying
to make a contribution, and I've had enough of you replying to me like
Peter and I are the same person, and I've had enough of you responding
to my messages without actually reading them.


Sam

[Chris Barker]

Folks,

Sorry to keep beating a seemingly dead horse here, but:

Peter: if you are still paying attention, while you seem incredably
reluctant to give an example, or try to see if you are running the
right wx build, I looked back at the threadand found this:


On Tue, Sep 18, 2012 at 2:20 AM, Sam Partington
<sam.par...@gmail.com> wrote:
> Actually I think it is quite bad in it's current state. We get quite
> a few seg faults here resulting from Destroy.

<i'm pretty sure SAM is running wxGTK, perhaps with the same
not-done-right-Ubuntu build>

> As this sample demonstrates :

Whoo hoo! an actual crashing sample we can try!

Well, on my machine, with wxPython 2.9.3.1, wxMac Cocoa -- I don't get
a crash, nor even a DeadObjectError. No problems with wxPython
2.8.12.1 wxMac (Carbon) either.

So Sam (and probably Peter, also) are experience more crashes due to
this issue than teh rest of because:

(1) There is a wxGTK bug/Issue that is not there with wxMac (nor wxWin32)

or

(2) They are running an improper build.

Peter, and Sam, I"d really like you to explore possibility (2) --
none of us want crashing code, it's worth knowing what he deal is
here. IF it's option (2), then we can all go on our merry way knowing
that wx is pretty darn stable. If it's (1) then maybe there is a bug
to be squashed.

Both Sam and Peter:

For the record, I do think you are basically right -- the wx memory
management approach is a mis-match with Python that is kind of kludged
over, and perhaps ad-hoc patched bit my bit. However, Robin and the
very few other folks that have dug into it deeply enough are really
smart people and have worked really hard to make wxPython robust, and
done a great job of it.

A lot of people use wxPython without crashing, so I really think there
is something else afoot here that simply poor design. That doesn't
mean its impossible to abuse Destroy() (and who knows what else) and
cause a crash, but it is rare, and it is very unlikely that there is
an easy solution that would fix the whole thing.

Not that you shouldn't try -- sometimes new eyes on a problem really
do turn up a brilliant previously-untried solutions.

-Chris

Code Sam Posted that I tried, and couldn't get to crash:

> import wx
>
> def on_kill_focus(evt):
> evt.GetEventObject().Destroy()
>
> def show_dialog(parent):
> wx.Dialog(parent).ShowModal()
>
> if __name__ == "__main__":
> app = wx.App()
> frame = wx.Frame(None)
> ctrl = wx.TextCtrl(frame, style=wx.LC_REPORT)
> ctrl.Bind(wx.EVT_KILL_FOCUS, on_kill_focus)
> wx.CallLater(100, show_dialog, ctrl)
> frame.Show()
> app.MainLoop()
>
> In this case the creation of a dialog causes the kill focus event on
> the text ctrl to fire. This handler then destroys the text ctrl. We
> think that when the kill focus event handler returns to C++ land the
> this pointer for the code in wxTextCtrl::HandlEvent pointer is then
> dead.

[Sam Partington]

On 24 September 2012 23:53, Chris Barker <chris....@noaa.gov> wrote:
> On Tue, Sep 18, 2012 at 2:20 AM, Sam Partington
> <sam.par...@gmail.com> wrote:
>> Actually I think it is quite bad in it's current state. We get quite
>> a few seg faults here resulting from Destroy.
>
> <i'm pretty sure SAM is running wxGTK, perhaps with the same
> not-done-right-Ubuntu build>

Hi Chris,

Unfortunately this will crash irrespective of the debug settings.
This crashes on windows and gtk, but not on mac. We build our own
version of wxPython and it definitely has the debug settings.

Peter's problem I suspect would be fixed by the debug flag.

> A lot of people use wxPython without crashing, so I really think there
> is something else afoot here that simply poor design. That doesn't
> mean its impossible to abuse Destroy() (and who knows what else) and
> cause a crash, but it is rare, and it is very unlikely that there is
> an easy solution that would fix the whole thing.

I think that our application is quite big, dynamic, lots of threads,
and it is run on many different platforms by many very vocal users. I
think that is why we experience more than most. It's still not /that/
many mind - we've only had maybe 3 in the last six months
attributable to wx. In a 100000 line app I wouldn't say it's that
bad, but more than there should be. A lot less than an equivalent C++
app for example.

For example the threading thing that I found and fixed a way back,
apparently no-one else had noticed it but we had a lot of problems
caused by that.

Sam

[Robin Dunn]

On 9/24/12 3:53 PM, Chris Barker wrote:
> On Tue, Sep 18, 2012 at 2:20 AM, Sam Partington
> <sam.par...@gmail.com> wrote:
>> Actually I think it is quite bad in it's current state. We get quite
>> a few seg faults here resulting from Destroy.
>
> <i'm pretty sure SAM is running wxGTK, perhaps with the same
> not-done-right-Ubuntu build>
>
>> As this sample demonstrates :
>
> Whoo hoo! an actual crashing sample we can try!
>
> Well, on my machine, with wxPython 2.9.3.1, wxMac Cocoa -- I don't get
> a crash, nor even a DeadObjectError. No problems with wxPython
> 2.8.12.1 wxMac (Carbon) either.

Most likely because kill-focus events are not working very well in 2.9
cocoa currently, and may have not been generated when showing the modal
dialog in 2.8 wxMac either.

[Robin Dunn]

On 9/24/12 3:48 PM, Sam Partington wrote:
> Kevin,
>
> On 24 September 2012 17:37, Kevin Ollivier <kev...@theolliviers.com> wrote:
>> No, what we said was "the way you are trying to fix the crash won't work".
>> The problem is that our explanations for why it won't work are either
>> ignored, passed off as not valid reasons, or even labeled as excuses, and
>> any alternatives we suggest get branded as hacks and workarounds. While
>> your and Peter are clearly very different, in this one (important) way your
>> reaction has been largely the same.
>
> I said that my own solution to my problem of CallAfter(CallLater...)
> is a hack. I still think it is. The general case of
> CallAfter(Destroy) I say is a work around. But an ok one. I think it
> would be better if there to be able to do window.DestroyWhenSafe().
> But I think it would be better still if there were just one Destroy
> that was always safe if it is practical to have one.
>
> The only time I mentioned excuse is when you said that Destroy may
> crash because it's a C++ extension. You also said that python doesn't
> have methods like Destroy, but it does: file.close is just such a
> method. It doesn't segfault it you call close 'badly' even though it
> is implemented in C.

Well, that's not quite the same thing. At least in my opinion.

The file.close() does deallocate some memory, but just a buffer that was
created for the PyFileObject and won't ever be used again since the file
is closed. The actual C FILE structure still exists and will continue
to do so until the PyFileObject is deallocated when its refcount goes to
zero. Also, there will not typically be any other C library code or 3rd
party code that are going to try to use that FILE pointer after you call
close, but even if there was it's okay because the FILE structure still
exists and the system can just set errno if they try to do something
with a closed file.

For Destroy, if the window is not a top-level window it essentially does
"delete this" and lets the destructor do all kinds of cleanup. Such as
remove the window from the parent's child list, detatching it from a
sizer if it's in one, destroying its children if it has any, cleaning up
its event binding table, and so on and then also
deleting/dereferencing/or whatever to the native widget and its resources.

So IMO there's a big difference in the pattern used and also in the
level of complexity. Could wxWidgets be changed to be more like the
file.close pattern? Sure. Will it? That's hard to answer since it
could have a huge impact on wxWidgets and cause big ripple effects in
existing software.

> And you were saying it was ok to crash in this instance when you said
> "Any Python extension can crash, this is just reality" and when you
> said that a library fix wouldn't be accepted, and various other
> similar statements about changing the behaviour of Destroy away from
> segfaults.

Can crash -> yes
Ok to crash -> no

Big difference.

But reaching the ideal 100% of the time can become exponentially more
difficult as the size of a project and the size of the user base grows,
that is the true reality here IMO.

[Sam Partington]

On 25 September 2012 00:46, Robin Dunn <ro...@alldunn.com> wrote:
> On 9/24/12 3:48 PM, Sam Partington wrote:
>>
>> The only time I mentioned excuse is when you said that Destroy may
>> crash because it's a C++ extension. You also said that python doesn't
>> have methods like Destroy, but it does: file.close is just such a
>> method. It doesn't segfault it you call close 'badly' even though it
>> is implemented in C.
>
> Well, that's not quite the same thing. At least in my opinion.
>
> The file.close() does deallocate some memory, but just a buffer that was
> created for the PyFileObject and won't ever be used again since the file is
> closed. The actual C FILE structure still exists and will continue to do so
> until the PyFileObject is deallocated when its refcount goes to zero. Also,
> there will not typically be any other C library code or 3rd party code that
> are going to try to use that FILE pointer after you call close, but even if
> there was it's okay because the FILE structure still exists and the system
> can just set errno if they try to do something with a closed file.

Yes like I said, Destroy could return the window to the default
constructed state.

> For Destroy, if the window is not a top-level window it essentially does
> "delete this" and lets the destructor do all kinds of cleanup. Such as
> remove the window from the parent's child list, detatching it from a sizer
> if it's in one, destroying its children if it has any, cleaning up its event
> binding table, and so on and then also deleting/dereferencing/or whatever to
> the native widget and its resources.
>
> So IMO there's a big difference in the pattern used and also in the level of
> complexity. Could wxWidgets be changed to be more like the file.close
> pattern? Sure. Will it? That's hard to answer since it could have a huge
> impact on wxWidgets and cause big ripple effects in existing software.

It would be impossible in the wxWidgets world without a major change
in the API because the user may be expecting their destructors (of
derived classes) to be called in the right order and at the right
time. That's not really an issue for wxPython though, you can never
rely on __del__ being called when you expect. Leaving the object in a
default constructed state wouldn't be that hard, though you'd have to
do it in such a way that it didn't affect the normal wxWidgets build
so it might be a bit messy.

>> And you were saying it was ok to crash in this instance when you said
>> "Any Python extension can crash, this is just reality" and when you
>> said that a library fix wouldn't be accepted, and various other
>> similar statements about changing the behaviour of Destroy away from
>> segfaults.
>
> Can crash -> yes
> Ok to crash -> no
>
> Big difference.

Fair point, in that instance Kevin didn't say it was ok so it was a
bad example. But Kevin did say :

>>> Destroy() not working in
>>> event handlers is not really brokenness from the perspective that it
>>> is not meant to be called from them in the first place"

"not really brokenness" to me means he considers it ok.

> But reaching the ideal 100% of the time can become exponentially more
> difficult as the size of a project and the size of the user base grows, that
> is the true reality here IMO.

Indeed, but I feel that this is one case where we can try to edge a bit closer.

Sam

[Kevin Ollivier]

Hi Sam,

On Sep 24, 2012, at 3:48 PM, Sam Partington wrote:

> Kevin,
>
> On 24 September 2012 17:37, Kevin Ollivier <kev...@theolliviers.com> wrote:
>> No, what we said was "the way you are trying to fix the crash won't work".
>> The problem is that our explanations for why it won't work are either
>> ignored, passed off as not valid reasons, or even labeled as excuses, and
>> any alternatives we suggest get branded as hacks and workarounds. While
>> your and Peter are clearly very different, in this one (important) way your
>> reaction has been largely the same.
>
> I said that my own solution to my problem of CallAfter(CallLater...)
> is a hack. I still think it is. The general case of
> CallAfter(Destroy) I say is a work around. But an ok one. I think it
> would be better if there to be able to do window.DestroyWhenSafe().
> But I think it would be better still if there were just one Destroy
> that was always safe if it is practical to have one.

Having a new method which basically delayed deletion until the Python refcount goes to 0 might be nice, but it still probably wouldn't have stopped your crash unless you had a live Python reference to the object outside of the event handler.

> The only time I mentioned excuse is when you said that Destroy may
> crash because it's a C++ extension. You also said that python doesn't
> have methods like Destroy, but it does: file.close is just such a
> method. It doesn't segfault it you call close 'badly' even though it
> is implemented in C.

The point I was trying to make with those comments was that methods like Destroy don't fit with Python's 'native' ref-counting memory management.

>>> My 'excuse, not a solution' comment was in response to your assertion
>>> that Destroy may crash, that's obvious because it's a C++ extension
>>> and C++ crashes some times, and it seemed to me that you were
>>> advocating that it was ok for it to crash. When I was actively trying
>>> to find a solution to prevent that crash.
>>
>> No one has yet advocated that it is "okay" for it to crash. To paraphrase
>> what Python's BDFL said about crashes, "it's not that Python is
>> uncrashable, but we consider it a bug unless the user is doing something
>> they shouldn't". I think that's a good outlook and pretty accurate to both
>> how Python and wxPython view and tackle crashes.
>
> You're misquoting AND misrepresenting what the BDFL said here.
> Misquoting: He said "hairbrained" not "something they shouldn't".
>
> Misrepresenting: He was referring to things which were actually
> trying to cause crashes, or things that noone would reasonably do in
> normal code. Not something that I rather suspect some users do by
> accident, because it looks perfectly reasonable until it crashes on
> you. Also most of those reported cases are now fixed. If I can call
> close on a file object in my write method then why shouldn't I call
> Destroy in an event handler?

So long as it's not from within any of the control's own event handlers, you often can. There are surely exceptions, though, and if you can't be sure it's safe, then you have to do workarounds. In that case, I think explicit is better than implicit.

> And you were saying it was ok to crash in this instance when you said
> "Any Python extension can crash, this is just reality" and when you
> said that a library fix wouldn't be accepted, and various other
> similar statements about changing the behaviour of Destroy away from
> segfaults.

Saying crashes happen is not saying it's okay for it to crash, as pointed out, and by "not really brokenness", I meant no one would expect control.Destroy to work when called from, say, control.SetFocus, or by any events or callbacks fired from within it. Destroying the control in that scenario will and should lead to some form of failure. You can argue whether or not a segfault or a traceback is a better way to fail, but it is a critical failure and the app needs to shut down at that point. (See below as to why.)

>> What people have been saying is that due to the nature of what the function does,
>> if you call it when you shouldn't, it is impossible to keep it from crashing
>> because it deletes memory. That is, if it's crashing, how it's being called is
>> a bug of some form or another.
>
> Destroy doesn't need to destroy memory, it could just release the OS
> objects,

^^^^^^ But THIS is quite possibly what is causing the crash in the first place. (And if by chance it isn't for your case, it surely will in others.) The event is coming *from* the OS object, so until you've completely left the OS's own code handling for that event, it is not safe to release the OS object(s) the control references.

It really depends on the app. In some cases, it wouldn't be a big problem; in other cases, it would be a very big problem and you could end up with corrupted or lost data. (Say that faulty mouse event fires over a delete button, etc.) Still, a user will take a crash that doesn't corrupt data over a non-crash that corrupts data any day. Stopping data corruption or loss I would argue is far higher priority than preventing crashes (without lost/corrupt data, you just restart and are good to go), and so the worst thing you can do IMHO is allow the app to continue in a state that could lead to that happening.

> But yes you're right, that behaviour was wrong and it does need to be
> fixed, and it's in our bug tracker as urgent, to come back to when we
> can. But the bug is somewhere deep in wx code as far as we can tell
> and we've not had time to figure out how that message is getting
> there. (In response to your question regarding that we were binding
> the dbl click event, not the mouse down).

I personally feel that figuring out why exactly Destroy was called when it should not have been is *vitally* important to getting us to a 100% reliable fix. Again, there's a good chance your Destroy() wrapper won't even stop the crash, as just because you're trying to release the memory later doesn't mean you're waiting until you know it's really okay.

So basically, the question, from what I understand, is whether or not dbl click is firing on mouse down or mouse up. It would be very helpful to see some code that would show us how this might happen.

[snip]

>> it's just that based
>> on your description of how you intend to tackle the problem, we
>> already have an idea of what results the approach you are taking will
>> have. For the toolkit as a whole it will probably be a net minus as it
>> will cause existing Destroy calls to behave in a somewhat unpredictable
>> manner, but as I said above it's anybody's guess as to exactly how the
>> issues might manifest them.
>>
>> The only real difference between your approach and the "workarounds" Robin
>> suggested is that your approach tries to hide the workaround from the user by
>> building it into the Destroy call. Your and Robin's approaches essentially do
>> the same thing, it's just that Robin's suggestion means that everyone that
>> reads the code realizes what it is doing and what to expect by doing it that
>> way. If you've coded for a long time, you should realize that is a plus. :)
>
> You clearly didn't even read what my approach is, again, which is the
> singularly most frustrating thing about this thread for me. The
> approach is to change *only* calls to Destroy when an active event
> handler is in the callstack so only those calls which would definitely
> have segfaulted would be affected. Detecting this is really
> straightforward in event.cpp without any heuristics or guesswork and
> practically no performance impact.

Except the controls are not only wx controls, they are *native* controls. OS controls. To make this actually work, you not only need to detect when wx is out of its event handling code, you also need to detect when the *native OS GUI library* has completely finished running all of the native control's event handling code as well. If the OS runs code after it fires off the native event, for example, you need to know about that, but there is no way that I am aware of to do so.

So yes, I have read what you are suggesting, and I have pointed out (not for the first time) what I see to be the problems with it.

[snip]

> There is also enough ambiguity in the docs there for the window
> destroy to be deferred a little bit.

Maybe, but the practical ramification for the toolkit is that there's lots of existing code that is written with the existing, non-deffered, behavior. I will agree with you on the "Destroys the window safely" part though; I think it was written directly to address the wxTopLevelWindow implementation of it, and the text belongs in that method's docs.

>> You really underestimate the amount of time and thought put into these
>> issues by people who have been working on these wrappers for years and
>> years.
>
> Look now you're putting even more words in my mouth, I really really
> don't underestimate it at all, I maintain several large wrappers
> myself I know it's hard. I have a huge respect for the work that Robin
> et al have done, and it is a really good piece of work but it is far
> from perfect. Only a few months ago I posted a large patch because
> the Python API was being called in literally hundreds of places
> without the GIL being taken, and this is another case where it can be
> improved, but it feels like you (and actually I mean only you, there
> has been no negativity from anyone else) are fighting against it for
> some reason I just don't understand.

Again, re-read why I'm suggesting it won't work and directly address what you see to be the flaw in my arguments about it, and we will get somewhere.

>> Particularly your suggestion that a ctypes-based wrapper would be
>> "pure Python" and have safer memory management than wx strongly hints
>> there are pieces of the memory management puzzle you are missing. In
>> that "Crashing Python" wiki page I linked earlier, it is listed as a 'dangerous'
>> module prone to triggering crashes, and for very good reason.
>
> A pure python ctypes wrapper would solve the memory management
> problem, of course memory management is only a small part of the
> resource management problem, but it would be one way to marry up the
> python/C methodologies in the same way, and of course it wouldn't be
> really pure python, but we are talking about interfacing with the a C
> OS, not PyOS. I have been writing C++ and python for 15 and 10 years
> respectively, I think I have an idea about the issues involved.

Roughly the same amount of time as I have, coincidentally.

> Frankly I've had enough of being patronised like this when I am trying
> to make a contribution, and I've had enough of you replying to me like
> Peter and I are the same person, and I've had enough of you responding
> to my messages without actually reading them.

Well, I'm certainly not trying to patronize anyone, but if we are to have a proper discussion, we can't be talking over each other. The points I've raised above are not new; I raised pretty much all of them in previous responses to your emails. What I don't remember is having any direct discussion on those points that I made, even though I feel those points are by far the most critical for us to get anywhere in this discussion. If I'm wrong about what I say, then I will apologize for it; it will not be the first time I've been wrong or made a mistake. And perhaps you've misunderstood the points I was trying to make in previous emails, in which case, I probably poorly explained myself. I can only go on your replies, though, and so far, nothing you've said indicates to me that you see technical flaws in my arguments about the OS level event handling issue, which I feel is *the* critical point to be addressed when attempting to modify Destroy as you proposed.

For the record, my email regarding both you and Peter was in exasperation over not having these concerns directly addressed or discussed, with both of you saying you would be able to address the issue satisfactorily (but not explaining how you work around the issues I mentioned). If we can't somehow find a way to productively discuss the technical issues, there really is not anything to do but wait for a patch so we can actually apply the patch and see if it would indeed alter Destroy behavior in existing code.

Regards,

[Werner]

Hi Sam,

On 25/09/2012 01:17, Sam Partington wrote:
> I think that our application is quite big, dynamic, lots of threads,
> and it is run on many different platforms by many very vocal users.

Could your "dialog double click" problem have something to do with your
threads handling. I don't use/need threads (lucky for me as it might be
over my head) but following the list I heard that any kind of GUI action
in a thread can cause strange crashes.

Werner

[Sam Partington]

All of our calls to wxPython except wx.CallAfter take place from the
wx main thread. We also have a special build of wxPython which
enforces this on each an every call to make absolutely sure of it, and
we run our test suite in this mode.

So I'm afraid that this has already been ruled out.

Sam

[Sam Partington]

On 25 September 2012 02:14, Kevin Ollivier <kev...@theolliviers.com> wrote:
> Having a new method which basically delayed deletion until the Python
> refcount goes to 0 might be nice, but it still probably wouldn't have
> stopped your crash unless you had a live Python reference to the object
> outside of the event handler.

That's wouldn't fix my bug at all, and anyway that is already
protected by PyDeadObjectError.

Anyway I wasn't suggesting that we wait until the python ref count
goes to zero. But until the active event handlers go to zero.

> Saying crashes happen is not saying it's okay for it to crash, as pointed
> out, and by "not really brokenness", I meant no one would expect
> control.Destroy to work when called from, say, control.SetFocus, or by any
> events or callbacks fired from within it. Destroying the control in that
> scenario will and should lead to some form of failure. You can argue whether
> or not a segfault or a traceback is a better way to fail, but it is a critical
> failure and the app needs to shut down at that point. (See below as to why.)

Why would I no-one expect Destroy to work from a event handler? Why
should it lead to failure? The docs don't say that it will fail in
that case, in fact the docs say that it is safe. it does not say
"safely, except in it's own event handler".

So if was not the docs that make you expect that, perhaps it's prior
art, let's look at some other UI frameworks
MFC(shudder) ... no crash
win32 ... no crash
AWT/Swing ... no crash
javascript/DOM ... no crash
tkinter ... no crash
pygtk.. no crash (I only tried on windows as on mac os doesn't seem to
have a simple install)
pyqt - no crash (using the interestingly named deleteLater, there is no delete)

So I don't think it is obvious at all that you have to be careful.

The only reason that you could possibly be aware of Destroy being
dangerous is that you have experienced or heard of these crashed
through another channel.

Well, I think it would have been better if new users don't have to
experience them in the first place.

>> Destroy doesn't need to destroy memory, it could just release the OS
>> objects,
>
> ^^^^^^ But THIS is quite possibly what is causing the crash in the first
> place. (And if by chance it isn't for your case, it surely will in others.)
> The event is coming *from* the OS object, so until you've completely
> left the OS's own code handling for that event, it is not safe to release
> the OS object(s) the control references.

That is a very good point. I had assumed that the OS wouldn't crash
when it's widget is destroyed, but I have not verified it... I'm
almost sure that win32 and cocoa don't crash, but I'm not at all sure
about gtk.

> It really depends on the app. In some cases, it wouldn't be a big
> problem; in other cases, it would be a very big problem and you
> could end up with corrupted or lost data. (Say that faulty mouse
> event fires over a delete button, etc.) Still, a user will take a crash
> that doesn't corrupt data over a non-crash that corrupts data any day.
> Stopping data corruption or loss I would argue is far higher priority
> than preventing crashes (without lost/corrupt data, you just restart
> and are good to go), and so the worst thing you can do IMHO is
> allow the app to continue in a state that could lead to that happening.

I give up on arguing this point. Clearly your customers and my
customers have different needs and expectations. If we make is a
runtime choice whether to raise an exception or defer the destroy then
the user can decide.

> I personally feel that figuring out why exactly Destroy was called when it should not
> have been is *vitally* important to getting us to a 100% reliable fix. Again, there's a
> good chance your Destroy() wrapper won't even stop the crash, as just because
> you're trying to release the memory later doesn't mean you're waiting until you know
> it's really okay.
>
> So basically, the question, from what I understand, is whether or not dbl click
> is firing on mouse down or mouse up. It would be very helpful to see some
> code that would show us how this might happen.

I know exactly why Destroy was called. I don't know why the double
click in a modal dialog leaked through to the parent window. We have
worked around that in that partiucular case for the time being,

These are orthogonal issues though. So I will discuss that in a
separate thread when we reopen that bug and I rule out the possibility
of a bug in our code. It is irrelevant to this discussion.

> Maybe, but the practical ramification for the toolkit is that there's lots of
> existing code that is written with the existing, non-deffered, behavior.
> I will agree with you on the "Destroys the window safely" part though;
> I think it was written directly to address the wxTopLevelWindow
> implementation of it, and the text belongs in that method's docs.

Perhaps, but the text is there as the very first statement of the
docs, whatever the original motivation for it.

> Again, re-read why I'm suggesting it won't work and directly address

> that you see to be the flaw in my arguments about it, and we will get
> somewhere.

I have just reread all of your posts and I feel that I have addressed
all your previous arguments. All of the previous arguments were to do
with you not understanding my fix because you thought that either

1) That I was going to defer all Destroys with CallAfter. I told you
that I had never intended that.
2) That I would heuristically guess whether an event handler was
active, and that I would get false positives. I told you that I wasn't
doing this.
3) That these runtime checks would slow down wxPython. I said at the
time it was impossible to know that until you measure it. We are
talking about an extra increment and decrement on every event handle,
and a fairly small amount of work on the rarely called Destroy method.
I don't think that is going to be measurable.

This new concern about OS objects is valid, and requires some testing
to determine whether it is a real issue or not. If is is so then it
will indeed be necessary to defer to later than the end of the event
handler. I personally think this would be preferable than crashing
however.

> Well, I'm certainly not trying to patronize anyone, but if we are to have a
> proper discussion, we can't be talking over each other. The points I've
> raised above are not new; I raised pretty much all of them in previous
> responses to your emails. What I don't remember is having any direct
> discussion on those points that I made, even though I feel those points
> are by far the most critical for us to get anywhere in this discussion. If
> I'm wrong about what I say, then I will apologize for it; it will not be the
> first time I've been wrong or made a mistake. And perhaps you've
> misunderstood the points I was trying to make in previous emails, in
> which case, I probably poorly explained myself. I can only go on your
> replies, though, and so far, nothing you've said indicates to me that you
> see technical flaws in my arguments about the OS level event handling
> issue, which I feel is *the* critical point to be addressed when attempting
> to modify Destroy as you proposed.

If you've mentioned this issue before then I did misundersood it, I've
checked gmane and I'm sure I didn't miss an email, and searching all
emails for OS doesn't find it. So I either the list didn't receive
it, you haven't mentioned it, or it was so poorly explained that I
still can't see it now with hindsight.

Anyway, now we have a technical issue to discuss at least. Does
anyone have any experience as to whether the various OS's will crash
if you destroy a widget in it's own event handler.

Sam

[Kevin Ollivier]

The simple fact is that events are fired by controls when a call to one of their methods or their own handlers induces a state change. Events are not all the same, and sometimes, you can do things like skip an event or veto / cancel an event, which will cause OS level handlers to apply some default behavior when control returns to them.

So to use the set focus example again, do you feel control.SetFocus, or all user code that uses it, should take into account you may destroy the control from within that method? Because code like that fires events, which then fires your handler, and it is all synchronous, meaning that when you get EVT_SET_FOCUS, you are still inside SetFocus, or more accurately, still inside the OS equivalent call for SetFocus.

When running your tests, there are a variety of factors like this that you need to consider when designing a proper test. And because wxWidgets (and wxPython) are largely different codebases on Win, GTK and Mac, there are certainly cases that will crash one but not the others. That is precisely why library developers are generally skittish about changing a long-standing core method's behavior on all platforms.

> The only reason that you could possibly be aware of Destroy being
> dangerous is that you have experienced or heard of these crashed
> through another channel.
>
> Well, I think it would have been better if new users don't have to
> experience them in the first place.
>
>>> Destroy doesn't need to destroy memory, it could just release the OS
>>> objects,
>>
>> ^^^^^^ But THIS is quite possibly what is causing the crash in the first
>> place. (And if by chance it isn't for your case, it surely will in others.)
>> The event is coming *from* the OS object, so until you've completely
>> left the OS's own code handling for that event, it is not safe to release
>> the OS object(s) the control references.
>
> That is a very good point. I had assumed that the OS wouldn't crash
> when it's widget is destroyed, but I have not verified it... I'm
> almost sure that win32 and cocoa don't crash, but I'm not at all sure
> about gtk.

If the OS is still accessing it, it most certainly will unless they check the OS object for validity prior to each call to it. (Obj-C/Cocoa has that feature built-in, but I've rarely seen other toolkits do that, though certain ref-counting systems do.) This is not something you can verify by running one test, just as calling Destroy does not lead to problems in all cases, or even most cases.

>> It really depends on the app. In some cases, it wouldn't be a big
>> problem; in other cases, it would be a very big problem and you
>> could end up with corrupted or lost data. (Say that faulty mouse
>> event fires over a delete button, etc.) Still, a user will take a crash
>> that doesn't corrupt data over a non-crash that corrupts data any day.
>> Stopping data corruption or loss I would argue is far higher priority
>> than preventing crashes (without lost/corrupt data, you just restart
>> and are good to go), and so the worst thing you can do IMHO is
>> allow the app to continue in a state that could lead to that happening.
>
> I give up on arguing this point. Clearly your customers and my
> customers have different needs and expectations. If we make is a
> runtime choice whether to raise an exception or defer the destroy then
> the user can decide.

You call GUI code after the GUI is already in an indeterminate state, and just expect it will work and not cause further problems?

>> I personally feel that figuring out why exactly Destroy was called when it should not
>> have been is *vitally* important to getting us to a 100% reliable fix. Again, there's a
>> good chance your Destroy() wrapper won't even stop the crash, as just because
>> you're trying to release the memory later doesn't mean you're waiting until you know
>> it's really okay.
>>
>> So basically, the question, from what I understand, is whether or not dbl click
>> is firing on mouse down or mouse up. It would be very helpful to see some
>> code that would show us how this might happen.
>
> I know exactly why Destroy was called. I don't know why the double
> click in a modal dialog leaked through to the parent window. We have
> worked around that in that partiucular case for the time being,
>
> These are orthogonal issues though. So I will discuss that in a
> separate thread when we reopen that bug and I rule out the possibility
> of a bug in our code. It is irrelevant to this discussion.

Again, Destroy will only crash in certain circumstances, and when it does there is a problem in its calling code somewhere. You can disagree with this if you like, but this much I can say is true.

I realize you're more concerned about Destroy not crashing than getting your problem fixed, but getting your problem fixed is the sure-fire accurate and reliable way to keep Destroy from crashing!

>> Maybe, but the practical ramification for the toolkit is that there's lots of
>> existing code that is written with the existing, non-deffered, behavior.
>> I will agree with you on the "Destroys the window safely" part though;
>> I think it was written directly to address the wxTopLevelWindow
>> implementation of it, and the text belongs in that method's docs.
>
> Perhaps, but the text is there as the very first statement of the
> docs, whatever the original motivation for it.
>
>> Again, re-read why I'm suggesting it won't work and directly address
>> that you see to be the flaw in my arguments about it, and we will get
>> somewhere.
>
> I have just reread all of your posts and I feel that I have addressed
> all your previous arguments. All of the previous arguments were to do
> with you not understanding my fix because you thought that either
>
> 1) That I was going to defer all Destroys with CallAfter. I told you
> that I had never intended that.

What you are doing is effectively the same thing, only slightly different because you don't give the OS queue time to empty out as well, which I think is an important part of why wx.CallAfter works and is recommended for this sort of scenario in the first place.

> 2) That I would heuristically guess whether an event handler was
> active, and that I would get false positives. I told you that I wasn't
> doing this.

Come back and re-write that once you've verified that you can never, under any circumstance, using any OS event, get Destroy to crash any longer. I'm pretty sure your tests have been pretty much for unique, specific cases but your fix is for the entire library and applies to all events and all calls of Destroy. Your fix affects far more use cases than your actual problem triggers in, and often times when the fix affects much more code than the problem itself does, you get unintended side-effects, sometimes ones worse than the original problem.

> 3) That these runtime checks would slow down wxPython. I said at the
> time it was impossible to know that until you measure it. We are
> talking about an extra increment and decrement on every event handle,
> and a fairly small amount of work on the rarely called Destroy method.
> I don't think that is going to be measurable.

I was referring instead to the general issue of changing wx memory management, which you later said you weren't considering.

> This new concern about OS objects is valid, and requires some testing
> to determine whether it is a real issue or not. If is is so then it
> will indeed be necessary to defer to later than the end of the event
> handler. I personally think this would be preferable than crashing
> however.

It's not a new concern, it's the core of my argument and the entire reason I say what you're doing is a heuristic, because, as I said before, your counter isn't going to the OS level, so it will sometimes be right, sometimes not, depending on the native OS handling of the event. Clearly I explained poorly (or perhaps assumed certain parts of my argument were just understood), but I don't think it helped that instead of asking me what I meant, you just said I was wrong without trying to better understand my arguments.

In any case, if it is indeed necessary to defer to later than the end of the event in some cases, and I do believe it is, we should recommend the same fix for all cases to be safe. And, that fix is simple, of course - wx.CallAfter. If you're going to alter Destroy and do it right, you might as well just change wxPython's Destroy to use wx.CallAfter. Though again, for the same reasons stated before, a change to when Destroy takes effect probably won't be accepted because it changes existing code from "destroy now" to "destroy at some indeterminate point in the future", which depending on when that "indeterminate point" is, could lead to unexpected, and not always reproducible, runtime behavior changes with existing code. That's the other major reason your fix is unlikely to be accepted, and this one I can ensure you I've pointed out repeatedly.

And perhaps that is the most important point to press. The fact that changing Destroy behavior silently at runtime for all wxPython apps (and perhaps you even intend the change for all wx apps, since your counting is happening in wx C++) is going to require an immense amount of testing to ensure the change is truly safe, and I doubt anyone would really feel that the effort is worth preventing the occasional crash when things go wonky. Particularly when such improvement can't even be precisely measured, since we don't even yet know the actual reason that Destroy is crashing for you in the first place and hence don't know whether or not your fix would actually improve or even change anything at all in that case.

>> Well, I'm certainly not trying to patronize anyone, but if we are to have a
>> proper discussion, we can't be talking over each other. The points I've
>> raised above are not new; I raised pretty much all of them in previous
>> responses to your emails. What I don't remember is having any direct
>> discussion on those points that I made, even though I feel those points
>> are by far the most critical for us to get anywhere in this discussion. If
>> I'm wrong about what I say, then I will apologize for it; it will not be the
>> first time I've been wrong or made a mistake. And perhaps you've
>> misunderstood the points I was trying to make in previous emails, in
>> which case, I probably poorly explained myself. I can only go on your
>> replies, though, and so far, nothing you've said indicates to me that you
>> see technical flaws in my arguments about the OS level event handling
>> issue, which I feel is *the* critical point to be addressed when attempting
>> to modify Destroy as you proposed.
>
> If you've mentioned this issue before then I did misundersood it, I've
> checked gmane and I'm sure I didn't miss an email, and searching all
> emails for OS doesn't find it. So I either the list didn't receive
> it, you haven't mentioned it, or it was so poorly explained that I
> still can't see it now with hindsight.
>
> Anyway, now we have a technical issue to discuss at least. Does
> anyone have any experience as to whether the various OS's will crash
> if you destroy a widget in it's own event handler.

I suspect you yourself may, with the edit_ctrl.on_edit_keydown leading to a crash when the edit_ctrl gets Destroy()'ed, but without knowing the precise details of the crash, we can't know either way for sure. In any case, I personally think that would be a very good thing to know before proceeding further.

[Chris Barker]

On Mon, Sep 24, 2012 at 4:17 PM, Sam Partington
<sam.par...@gmail.com> wrote:

> Unfortunately this will crash irrespective of the debug settings.
> This crashes on windows and gtk, but not on mac. We build our own
> version of wxPython and it definitely has the debug settings.

Good to get that cleared up -- I suppose I should have taken the time
to fire up teh VM and test on Windows, too. Interesting that it
doesn't crash on wxMac, though.

> Peter's problem I suspect would be fixed by the debug flag.

That's what I suspected as well -- not sure why he was so insistant on
not even checking that out.

> I think that our application is quite big, dynamic, lots of threads,
> and it is run on many different platforms by many very vocal users. I
> think that is why we experience more than most.

It does sound like you are pushing the limits.

> It's still not /that/
> many mind - we've only had maybe 3 in the last six months
> attributable to wx. In a 100000 line app I wouldn't say it's that
> bad, but more than there should be. A lot less than an equivalent C++
> app for example.

THanks for quantifying -- one crash his too many, but this thread
seemed to be implying "lots of crashing" -- which didn't make sense to
me. I think unfortunately, your reports and Peter's got confused.

> For example the threading thing that I found and fixed a way back,
> apparently no-one else had noticed

Well, threading is known to be a bit ugly, so we all usually keep it
to a minimum...But with all the multi=-cores these days, I know I
appreciate your efforts to improve wx's robustness around threading.

Robin Dunn wrote:
> Most likely because kill-focus events are not working very well in 2.9 cocoa
> currently, and may have not been generated when showing the modal dialog in 2.8
> wxMac either.

Hmm I didn't even check for that -- not too smart!

Checking now -- the KILL_FOCUS event doesn't fire until the man frame
is closed (with 2.8 Carbon), and, indeed, not at all with 2.9 Cocoa.


Anyway Sam, it sound like you've got a good idea for a patch, I'm
looking forward to seeing how it works out.

-Chris

[Robin Dunn]

It's not just native UI crashes that are a worry here (although I expect
that the chances of actual crashes are very small, there could be other
bugs though) but also that there could be other attempts from wx to use
the widgets or the now invalid wx object upon return from the event
handler and the widgets have now vanished unexpectedly. And of course
the same old pending event issue still exists.

As far as I can see, just deleting the native widget and returning the
wx object to an uninitialized state simply moves things from a bad
pointer problem to a missing widget problem. It's arguably a better
problem than now but it's still a problem, and to fix it most or maybe
all wx APIs will need to check if the widget still exists and fail
gracefully if it doesn't. So it's not a simple, targeted change but
something that will have to touch all parts of wx to be done right.

[Sam Partington]

On 25 September 2012 17:55, Robin Dunn <ro...@alldunn.com> wrote:
> On 9/25/12 7:33 AM, Sam Partington wrote:
>>
>> On 25 September 2012 02:14, Kevin Ollivier <kev...@theolliviers.com>
>> wrote:
>>
>> Anyway, now we have a technical issue to discuss at least. Does
>> anyone have any experience as to whether the various OS's will crash
>> if you destroy a widget in it's own event handler.
>
>
> It's not just native UI crashes that are a worry here (although I expect
> that the chances of actual crashes are very small, there could be other bugs
> though) but also that there could be other attempts from wx to use the
> widgets or the now invalid wx object upon return from the event handler and
> the widgets have now vanished unexpectedly. And of course the same old
> pending event issue still exists.

Ok, but IMO these are also bugs, whenever wx calls out to user code it
should not assume anything on return.

But you have convinced me that trying to destroy the window when the
last event handler has finished is wrong, or at least two difficult to
attempt now, and that we should simply raise an exception when Destroy
is called in an event handler for that control (or one of it's
children).

> As far as I can see, just deleting the native widget and returning the wx
> object to an uninitialized state simply moves things from a bad pointer
> problem to a missing widget problem. It's arguably a better problem than
> now but it's still a problem, and to fix it most or maybe all wx APIs will
> need to check if the widget still exists and fail gracefully if it doesn't.
> So it's not a simple, targeted change but something that will have to touch
> all parts of wx to be done right.

Ah, I would have thought that since it is possible to create widgets
in that state that the majority of the code must check for that
already, and if it doesn't then that would be a bug already. Of
course widgets going from a good state to a bad state would be new so
that would mean (possibly many) more checks,

I never thought that this was an easy solution though.

Sam

[Sam Partington]

On 25 September 2012 17:29, Kevin Ollivier <kev...@theolliviers.com> wrote:
> The simple fact is that events are fired by controls when a call to one of their
> methods or their own handlers induces a state change. Events are not all the
> same, and sometimes, you can do things like skip an event or veto / cancel
> an event, which will cause OS level handlers to apply some default behavior
> when control returns to them.

Yes you are right, it is more difficult than I had imagined. I am now
convinced that trying to automatically defer the destroy until 'safe'
is too difficult to attempt in one patch, or in a point release, or
perhaps ever. Thank you for taking the time to help me grok that.

It still leaves the option to raise an exception when Destroy is
called in an event handler, which is an option I have offered before,
and the only option to receive a +1 so far.

> So to use the set focus example again, do you feel control.SetFocus, or all
> user code that uses it, should take into account you may destroy the control
> from within that method? Because code like that fires events, which then
> fires your handler, and it is all synchronous, meaning that when you get
> EVT_SET_FOCUS, you are still inside SetFocus, or more accurately, still
> inside the OS equivalent call for SetFocus.

Well, yes I think it should. Whenever a library calls user callbacks
it must check it's preconditions again. I'm sure the OS apis do this.
Take win32 for example, it never assumes that a HWND is valid, it
always checks and returns failure if it is invalid.

> When running your tests, there are a variety of factors like this that you need
> to consider when designing a proper test. And because wxWidgets (and
> wxPython) are largely different codebases on Win, GTK and Mac, there are
> certainly cases that will crash one but not the others. That is precisely why
> library developers are generally skittish about changing a long-standing core
> method's behavior on all platforms.

Which is totally understandable when you are changing working
behaviour to a different working behaviour. But going from crashing
to non-crashing is not, IMO one that I would not be skittish about.

> If the OS is still accessing it, it most certainly will unless they check the
> OS object for validity prior to each call to it. (Obj-C/Cocoa has that
> feature built-in, but I've rarely seen other toolkits do that, though
> certain ref-counting systems do.) This is not something you can
> verify by running one test, just as calling Destroy does not lead to
> problems in all cases, or even most cases.

My tests on the other frameworks suggest that they all check this. As
any responsible API would do which has callbacks and a Destroy method.

> You call GUI code after the GUI is already in an indeterminate state, and just expect it will work and not cause further problems?

It was only in an indeterminate state because a bug in agw and/or wx
caused a spurious mouse event to occur. (Which we have worked
around).

On this occasion that bug caused Destroy to be called by agw code from
an event handler and crashed. But many other times it wouldn't have
caused Destroy and the indeterminate state may well have gone
undetected, as would be the case for many other classes of bugs in wx
code, agw code, or our code. But whatever the cause, I don't think
we should use Destroy crashing as a litmus test for the application
being in a determinate state as that would only catch a minority of
them anyway.

In my application it would always be better to have a spurious mouse
event to occur than a crash. It would be impossible (in our
application) for that spurious mouse event to do anything that the
user could not put right in seconds - i.e. a single mouse event never
does anything particularly powerful, and the undo buffer could have
resolved it quickly. But a crash could easily cost the user hours of
work. It's the lesser of two evils for our application, you'll just
have to take my word for that. YMMV of course.

Unfortunately we have experienced quite a lot of weird bugs like this
in wxPython and though it got a lot better in 2.9, it also introduced
many more with it. So we test, a lot. It was during testing we found
this one I'm glad to say, but it could have ended up on a customers
desktop before being discovered.

In any case, if Destroy had raised an exception and not crashed we
would have been able to handle it much better.

> Again, Destroy will only crash in certain circumstances, and when it
> does there is a problem in its calling code somewhere. You can
> disagree with this if you like, but this much I can say is true.

I never argued otherwise. Though I could - the docs actively suggest
that you could, and other frameworks allow it.

> I realize you're more concerned about Destroy not crashing than getting
> your problem fixed, but getting your problem fixed is the sure-fire
> accurate and reliable way to keep Destroy from crashing!

I have fixed/worked around that particular problem, both Destroy and
the click through are in my bug tracker to fix, I am discussing the
Destroy fix now because it is relevant to this thread topic.

>> 2) That I would heuristically guess whether an event handler was
>> active, and that I would get false positives. I told you that I wasn't
>> doing this.
>
> Come back and re-write that once you've verified that you can never,
> under any circumstance, using any OS event, get Destroy to crash any
> longer.

Come back and rewrite what? That I wasn't using a heuristic mechanism
to detect being in an event handler? I will write that many times as
you like. That's irrelevant to Destroy being never able to crash.

This argument seems to lead to "Let's not fix anything unless we can
fix everything". I never claimed to fix them all, but if we can fix
10% of them, then we can look at the next 10%.

>> This new concern about OS objects is valid, and requires some testing

> It's not a new concern, it's the core of my argument

Honestly, you hadn't mentioned this before, if you disagree then
please can you post a link to the precise message where you did? I am
at a loss as to what might possibly have been alluding to it.

> and the entire
> reason I say what you're doing is a heuristic, because, as I said before,
> your counter isn't going to the OS level, so it will sometimes be right,
> sometimes not, depending on the native OS handling of the event.

There are two issues to debate here, I'll separate them so that we can
discuss them more clearly.

1 Can we detect that Destroy has been called from an event handler?
2 What should we do if we do detect this?

1 Can we detect that Destroy has been called from an event handler?

Perhaps I misunderstand the way the events reach python code. Don't
all events that end up in python go through
wxAppConsoleBase::CallEventHandler? If they do all go through there,
and the counting is done there, then how could the counter be wrong?

As far as I can imagine, we're not concerned with events that don't go
to python because then the python code can't call Destroy, and here we
are concerned with cases where the python calls Destroy.

Are you concerned with false positives or negatives, just to be clear:

false positives : we think we're in an event handler but we're not
false negatives : we think we're not in an event handler but actually we are

false positives would be very bad as it would prevent legitimate calls
to Destroy, and that would be a show stopper.

false negatives would be a shame because it would mean that
illegitimate calls to Destroy would be allowed, leading to some
crashes that would have happened before, still crashing now, but not a
showstopper.

IMO there would be neither false positive nor false negatives, but
clearly there are people on this list who understand all this better
than I do.

> In any case, if it is indeed necessary to defer to later than the
> end of the event in some cases, and I do believe it is, we
> should recommend the same fix for all cases to be safe. And,
> that fix is simple, of course - wx.CallAfter. If you're going to
> alter Destroy and do it right, you might as well just change
> wxPython's Destroy to use wx.CallAfter. Though again, for the
> same reasons stated before, a change to when Destroy takes
> effect probably won't be accepted because it changes existing
> code from "destroy now" to "destroy at some indeterminate
> point in the future", which depending on when that "indeterminate
> point" is, could lead to unexpected, and not always reproducible,
> runtime behavior changes with existing code. That's the other
> major reason your fix is unlikely to be accepted, and this one I
> can ensure you I've pointed out repeatedly.

Yes you have repeatedly pointed out that the fix I have never
suggested would never be accepted.

2 What should we do if we (can and) do detect this?

I suggested two options for what could happen when a Destroy was

called in an event handler

1. defer the Destroy (either with CallAfter or sooner if it were
possible, it seems that this is not practical however)
2. raise an exception

I know you are against option 1. Are you also against option 2?

I also suggested this could be a runtime option, with either 1 or 2 as
the default. Are you against this as well?

> And perhaps that is the most important point to press. The fact that
> changing Destroy behavior silently at runtime for all wxPython apps
> (and perhaps you even intend the change for all wx apps, since your
> counting is happening in wx C++)

The detection code would be in wx I think as it is easier to achieve
there, but what to do about it when detected would only be in
wxPython. I would let the wxWidgets developers decide whether they
wanted to act on this, or whether they want to compile out the code
which allows the detection so they don't have to pay the negligible
performance penalty.

It would only affect those wxPython apps which we agree need to be fixed anyway.

> is going to require an immense amount of testing to ensure the change
> is truly safe,

I would think that any improvement in safety, would be, well, an improvement.

The side effects depend on the course of action chosen.

> and I doubt anyone would really feel that the effort is worth
> preventing the occasional crash when things go wonky. Particularly
> when such improvement can't even be precisely measured, since we
> don't even yet know the actual reason that Destroy is crashing for you
> in the first place and hence don't know whether or not your fix would
> actually improve or even change anything at all in that case.

I have posted example code that several people have reproduced as
crashing, is that not enough?

If the code doesn't fix anything of course there is no point at all is there.

Sam

[Kevin Ollivier]

Hi Sam,

Sorry to not reply to your individual points this time, but I think I should take a cue from Robin here and try to respond to the points / thread as a whole. Going back over my messages, I realize some of it was rushed and so I didn't express myself quite clearly enough, and that was part of the problem tangling the threads of communication. Anyway, let me summarize my main points on the Destroy issues:

My theory on the Destroy problem is that deleting a control from within one of its own event handlers is sometimes unexpected by the C++ code and this is evidenced by the crash. I don't know in general what the various toolkit's policies on this are, and I'm not saying I know it's not allowed, or that it isn't wx somehow behaving badly. ;-)

Throwing an exception is definitely an improvement over segfaulting in my book, but I also don't think we should throw an exception for cases that would not lead to a crash, as we don't want to suddenly have existing, working code throwing exceptions. Also, if it is wx behaving badly, this offers the possibility that we could fix wx so that instead of catching crashes, we remove them entirely, and I think that's a possibility worth exploring before going too much further.

So the next steps that I see for dealing with this issue are:

- Diagnose the exact C++ code that is crashing for the simple case you gave so we know exactly what is causing the crash and discuss the best fix for it. I'll try to find some time myself to build a debug wxPython over this weekend or next.

- If we can't reliably institute a crash fix at the wx level, see if we can at least reliably determine what code paths we should raise exceptions for, and if so, do so. We could also perhaps have the exception recommend wx.CallAfter usage. Improve documentation as well if it is needed.

- Worst case, we add some documentation to Destroy suggesting the use of wx.CallAfter and / or a new method that we recommend over Destroy, perhaps even deprecating or at least discouraging direct Destroy usage at the Python level.

So what do you think? Are we in agreement on this?

Thanks,

Kevin

[Sam Partington]

> Hi Sam,
>
> Sorry to not reply to your individual points this time, but I think I should take a cue from Robin here and try to respond to the points / thread as a whole. Going back over my messages, I realize some of it was rushed and so I didn't express myself quite clearly enough, and that was part of the problem tangling the threads of communication. Anyway, let me summarize my main points on the Destroy issues:
>
> My theory on the Destroy problem is that deleting a control from within one of its own event handlers is sometimes unexpected by the C++ code and this is evidenced by the crash. I don't know in general what the various toolkit's policies on this are, and I'm not saying I know it's not allowed, or that it isn't wx somehow behaving badly. ;-)
>
> Throwing an exception is definitely an improvement over segfaulting in my book, but I also don't think we should throw an exception for cases that would not lead to a crash, as we don't want to suddenly have existing, working code throwing exceptions. Also, if it is wx behaving badly, this offers the possibility that we could fix wx so that instead of catching crashes, we remove them entirely, and I think that's a possibility worth exploring before going too much further.
>
> So the next steps that I see for dealing with this issue are:
>
> - Diagnose the exact C++ code that is crashing for the simple case you gave so we know exactly what is causing the crash and discuss the best fix for it. I'll try to find some time myself to build a debug wxPython over this weekend or next.
>
> - If we can't reliably institute a crash fix at the wx level, see if we can at least reliably determine what code paths we should raise exceptions for, and if so, do so. We could also perhaps have the exception recommend wx.CallAfter usage. Improve documentation as well if it is needed.
>
> - Worst case, we add some documentation to Destroy suggesting the use of wx.CallAfter and / or a new method that we recommend over Destroy, perhaps even deprecating or at least discouraging direct Destroy usage at the Python level.
>
> So what do you think? Are we in agreement on this?

Hi Kevin,

Wonderful, I agree with all your points, especially that the inline
posting was worsening matters.

In the end you're right that there are cases where Destroy from an
event handler does not segfault on some platforms. So there may be
apps out there that are working just fine on a restricted set of
platforms, that would we would break if we made destroy raise as
discussed. And though those apps would probably have problems should
they port to other platforms not all projects are multi-platform and
it's not our place to force them without a deprecation phase of some
sort.

I agree about the doc change for destroy, and I do think there should
be a new method called DestroySoon/After/WhenSafe/InGoodTime or
something like that. Exactly like Robins SelfDestruct, but named
DestroyWhatever so that it is right next to the entry for Destroy.

If it turns out I would even consider adding a new method
DestroyImmediately either as an alias for Destroy, or as a variant
that does raise an exception if called from an event handler. This
allows you to deprecate the Destroy method, and in a later release
potentially remove it altogether. It also aids developers to search
for potentially incorrect uses of the current Destroy and actively
document that they mean destroy now or later.

I'm glad that at last we are in agreement. I will try to report back
about the exact cause of the crash soon, but ironically enough my VC9
installation seems to have self-immolated itself right now :-(

Sam

[Robin Dunn]

On 9/27/12 2:18 AM, Sam Partington wrote:

> I agree about the doc change for destroy, and I do think there should
> be a new method called DestroySoon/After/WhenSafe/InGoodTime or
> something like that. Exactly like Robins SelfDestruct, but named
> DestroyWhatever so that it is right next to the entry for Destroy.

I've been thinking about using DestroyLater for the name, and also a
couple more things that would be good for it to do besides just
self.Hide and ScheduleForDestruction

[Kevin Ollivier]

Hi Robin and all,

On Sep 27, 2012, at 8:54 AM, Robin Dunn wrote:

> On 9/27/12 2:18 AM, Sam Partington wrote:
>> I agree about the doc change for destroy, and I do think there should
>> be a new method called DestroySoon/After/WhenSafe/InGoodTime or
>> something like that. Exactly like Robins SelfDestruct, but named
>> DestroyWhatever so that it is right next to the entry for Destroy.
>
> I've been thinking about using DestroyLater for the name, and also a couple more things that would be good for it to do besides just self.Hide and ScheduleForDestruction

Just as an update, I've spent a bit of time trying Sam's sample code on Mac, since I've found that it's easier to setup a debug environment there due to not actually needing to build Python debug.

So the first problem I ran into was related to the issue where EVT_KILL_FOCUS on Mac actually fires *before* the focus is lost. So wx would crash because it stores a reference to the currently focused control, and doesn't expect it to be destroyed. So then I did wx.CallAfter on the Destroy call to see what would happen, and then I got a weird crash… It's crashing in wxOSX_mouseEvent, and I'm not quite sure how that's possible unless Cocoa is still sending events to the dead control. I didn't have much time to look into it, but hopefully I'll be able to spend some more time with it next weekend or so. One thing it shows though is that the different order of operations makes this something that we probably have to first examine for each platform and then determine what the solution should be.

Robin, FWIW, I was using the Phoenix / Cocoa build to test the sample. Also, I wasn't able to get proper debug symbols with my build, at least at the wxWidgets level, though I didn't spend much time tracking down why. Should that be working with a Phoenix build.py build when passing --debug?

Thanks,

Kevin

> --
> Robin Dunn
> Software Craftsman
> http://wxPython.org
>

[Robin Dunn]

On 10/7/12 10:25 PM, Kevin Ollivier wrote:
> Robin, FWIW, I was using the Phoenix / Cocoa build to test the sample. Also, I wasn't able to get proper debug symbols with my build, at least at the wxWidgets level, though I didn't spend much time tracking down why. Should that be working with a Phoenix build.py build when passing --debug?
>

Yes it should, and has in the past I think. I'll look into that soon.