My Python / wxPython app crashing - suggestions?

[Ross]

Hello All,

I have an app built on python 2.5.2 on mac osx 10.4.11. The GUI
elements are built on wxPython 2.8.10.1. Development has gone
pretty well, but I've got an intermittent rather silent crash that
happens without spewing any error messages to my console at all.

I'm developing in Eclipse with PyDev, and I build with py2app to make
an executable app.

Whether in Eclipse/PyDev or running standalone, the crash occurs
randomly without any messages almost every run, though it takes me 10
-15min of usage before I get the fateful crash. The only data I can
get is from the CrashReporter Log directory where a MyApp.crash.log
tells me that Thread 4 crashed with:

Thread: 4

Exception: EXC_BAD_ACCESS (0x0001)
Codes: KERN_PROTECTION_FAILURE (0x0002) at 0x00000004

and:

Thread 4 crashed with X86 Thread State (32-bit):
eax: 0x00000000 ebx: 0x9083d561 ecx: 0x00000074 edx: 0x92e341ac
edi: 0x14442b60 esi: 0x00000000 ebp: 0xb0370b58 esp: 0xb0370b58
ss: 0x0000001f efl: 0x00010286 eip: 0x92e30522 cs: 0x00000017
ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037

Given the lack of other info, this doesn't tell me much. Any
suggestions?

One other (perhaps related issue?) very rarely, much less common
than the crash is a spurious malloc error from deep in python
somewhere. Not sure that my code is responsible. I don't recognize
this format of complaint.

Python(9544,0xa000d000) malloc: *** error for object 0x1a05c8f0:
double free
Python(9544,0xa000d000) malloc: *** set a breakpoint in
szone_error to debug

Can anyone suggest somewhere to start? If I start putting random
breakpoints and prints in trying to catch the crash, I could be here
for years as I can't yet nail down the conditions that precipitate it.

Regards,
Ross.

[Mike Conley]

I have seen two causes of malloc errors crashing Python

1) when using a grid and setting an attribute to a cell or column, you need to remember to issue and IncRef() call for the attribute otherwise wx will lose track of how many times the attr object is being used

2) this one is tougher for me to explain because I don't fully understand what I do wrong; maybe I will try to together a sample and post it. When putting together widgets and sizers, if you do it wrong, it is possible to get into a situation where the same widget is a child twice in the same hierarchy. When the hierarchy is destroyed / freed that widget is freed twice and crashes Python on the second free. You can see the problem by using the Inspection tool, but it might be hard to find in a complicated application.

Hope these give some useful hints.

--
Mike Conley

 

[Ross]

Thanks for the ideas - I'll take a look at what I might be doing along those areas.

Ross.

On 27-Nov-09, at 9:22 PM, Mike Conley wrote:

 

 

I have seen two causes of malloc errors crashing Python

1) when using a grid and setting an attribute to a cell or column, you need to remember to issue and IncRef() call for the attribute otherwise wx will lose track of how many times the attr object is being used

2) this one is tougher for me to explain because I don't fully understand what I do wrong; maybe I will try to together a sample and post it. When putting together widgets and sizers, if you do it wrong, it is possible to get into a situation where the same widget is a child twice in the same hierarchy. When the hierarchy is destroyed / freed that widget is freed twice and crashes Python on the second free. You can see the problem by using the Inspection tool, but it might be hard to find in a complicated application.

Hope these give some useful hints.

--
Mike Conley

 

--
To unsubscribe, send email to wxPython-user...@googlegroups.com
or visit http://groups.google.com/group/wxPython-users?hl=en

[Sam23]

"Try ripping out everything that is not essential to make this crash
happen"

I had somewhat of a similar problem a few months ago. My wxPython
program crashed without any error details i.e. without the usual
python traceback that enables tracing the problem to a particular line
of python code.

I eventually solved the problem; aided by numerous suggestions from
this forum. Werner was especially patient and provided a lot of tips
that helped solve the problem (wx CallAfter!)

I provide some details below on how I eventually tracked down the
problem and solved it. You can also view the details in the
discussion in this forum - the title is 'unhandled exception'.

Some disclaimers:
a) I am no expert in wxPython; and the crashing app was my first
attempt at wx. Lots of folks here are light-years ahead of me. If my
suggestions are incorrect, please feel free to correct.
b) I was running wxPython in Windows XP and did not compile to exe.

--OK here it is:
My application crashed with "Unhandled exception" being displayed.
The error seems to come from Window rather than Python, and there is
no details at all apart from the unhelpful Windows dump. Eventually,
I found that the error was caused by this:

Mouse click on a panel -> Event handler for mouse click ->
Redraw entire screen consisting of many panels, including deleting
panel that was the source of the event.

Apparently, it is not good to delete a window inside the event
handler if the window is the source of the event. I didn't know this
at the time I wrote the code, and the error didn't appear until a
few months into development; even though I had been doing this
deletion for quite a long time prior to the crash. It is
intermittent, sometimes it doesn't happen at all, sometimes it
happens within 2 or 3 clicks. But somewhere along the development as
the number of application features increased, it became frequent.

For such types of errors, print is of no help in isolating the
problem. This is because the error does not occur in python. I think
it is not even in wxPython but in the C++ code of wx widgets. And
the error in not synchronous with the code that causes the error i.e.
the timing is different. For example, the deletion of objects in C++
may be deferred and say that the error is caused by the deletion. So
this may be long after the line of code in Python that actually
issues the delete command has passed.

I didn't know this at first. I started by putting print statements
everywhere. Finally to my surprise, I saw that the print statement
at the very last line of Python code was executed, and that the crash
happened after this; i.e. after my wxpython program had already
completed processing the event that caused the crash. In fact, all the
expected logic and screen updates had completed successfully when
the crash happened. The screen and data was already changed
correctly.

I eventually solved the problem based on this suggestion from Robin:
"Try ripping out everything that is not essential to make this crash
happen"

This sounds daunting and initially I thought I could not do it. There
were a lot of code, and I couldn't easily take code out without
stuff breaking. I also thought that it would take forever to arrive
at a state where there is little code and the problem still occurs;
since it seems to occur after some sequence of events involving most
of the code. Eventually, out of lack of choice, I went ahead to do
this. It took a long time. It was several days of work - but not
forever as I thought it might be.

What is the best way to go about this? Well, the crash is in wx, so
the crash must have something to do with screen drawing or updates
or changes to wx objects. So non-wx logic can be removed e.g. the
business logic can be removed; replaced with stubs; e.g. return dummy
values. Next, you can try ripping out large sections of code at a
time and see if the error still occurs. Finally, keep track of
versions between ripping out code; so that you will know what to put
back if the error disappears. You may have to make small changes to
the code to invoke the gui functions required after removing chunks of
code.

At the end of the process, you will hopefully have a small segment of
code that is responsible for the error. At this time you may already
have figured out what it is. If not, put that code into this forum
and someone here will probably be able to tell you why.

For my case, I was eventually able to produce a very small wxPython
program that replicated the error; just 1 frame screen with 2 or 3
panels. Of course this was after I already figured it out.

I also found out that adding evt.Skip() in the event handler caused
the crash. Although I was doing something wrong - deleting a panel in
the event handler triggered by the panel, this didn't cause the
crash until I added evt.Skip() much much later into development. And
the crash was intermittent, so I didn't see it initially i.e. right
after I added the Skip(). By the time the crash came out, I couldn't
easily revert to a version that didn't crash.

In any event, keeping versions of code is useful for comparing
between when it started crashing and when it did not. After I
isolated the code that caused the problem, a code comparison between
earlier and later versions showed me the difference was evt.Skip().

I think it sounds daunting; but it is probably do-able unless your
code is really huge. Painful but doable... It would be nice if
wxWidgets can throw up at least a bit of info on the source of the
error e.g. the names of wx objects linked to the error, but it seems
like it doesn't.

Therefore this take stuff out approach for these type of crashes may
be the only available approach without use of any advanced debugger
tools - if there is any available. I don't think so.

Hope this helps.

[Ross]

Thanks for your thoughtful reply! It's at very least helpful to
know that someone else managed to find some success dealing with
this. It is truly very elusive for me, but no doubt a good learning
experience.

I'm very daunted by the idea of ripping things out. My app is a
rather complex thing with some multithreading and complex UI
elements. I can't imagine ripping things out without everything
coming to a halt, as there are many UI operations that gather input
info that allows the user to then proceed with a more complex UI
task. But I will certainly hold that in reserve.

Like you I added copious printing to try and figure out where the
fault was happening, but I've similarly discovered that there is no
consistency with the position in the code for the crash. The
preceding operation can be almost anything.

But, I've learned a little bit more about seg faults, and realize
that there is a more useful part of the log file that might aid
someone with a better eye than I for the under-lying workings of
wx. It seems likely from these crashed-thread traces on the Mac OS
X platform that wx is intimately involved in the crash.

Excuse the slightly long content that follows, but note the
similarities in these couple of traces that represent two different
crashes in totally different parts of the user experience. If
someone is familiar with these elements and has a thought I'd LOVE to
hear it. Meanwhile, I will start refactoring my code in the hopes
that a general clean-up will twig some ideas.

Crash1:

Thread 4 Crashed:
0 libobjc.A.dylib 0x90a594c7 objc_msgSend + 23
1 com.apple.HIToolbox 0x92e28b18 HIView::SetFrame
(CGRect const&) + 708
2 com.apple.HIToolbox 0x92e42e57 HIViewSetFrame + 57
3 libwx_macud-2.8.0.dylib 0x026ad8fe wxMacControl::SetRect
(Rect*) + 86
4 libwx_macud-2.8.0.dylib 0x0273b440 wxWindow::DoMoveWindow
(int, int, int, int) + 546
5 libwx_macud-2.8.0.dylib 0x027374a9 wxWindow::DoSetSize
(int, int, int, int, int) + 191
6 libwx_macud-2.8.0.dylib 0x02720142 wxStaticText::SetLabel
(wxString const&) + 326
7 _core_.so 0x02051444 _wrap_Window_SetLabel
+ 208 (_core_wrap.cpp:34767)
8 org.python.python 0x0020eb00 PyObject_Call + 45
(abstract.c:1861)
9 org.python.python 0x00297b67 PyEval_EvalFrameEx +
13791 (ceval.c:3853)
10 org.python.python 0x0029ae0e PyEval_EvalCodeEx +
1819 (ceval.c:2836)
11 org.python.python 0x00298618 PyEval_EvalFrameEx +
16528 (ceval.c:3669)
12 org.python.python 0x002999ce PyEval_EvalFrameEx +
21574 (ceval.c:3659)
13 org.python.python 0x002999ce PyEval_EvalFrameEx +
21574 (ceval.c:3659)
14 org.python.python 0x0029ae0e PyEval_EvalCodeEx +
1819 (ceval.c:2836)
15 org.python.python 0x0022fe6b function_call + 320
(funcobject.c:517)
16 org.python.python 0x0020eb00 PyObject_Call + 45
(abstract.c:1861)
17 org.python.python 0x002167ae instancemethod_call +
401 (classobject.c:2519)
18 org.python.python 0x0020eb00 PyObject_Call + 45
(abstract.c:1861)
19 org.python.python 0x002936fe
PyEval_CallObjectWithKeywords + 112 (ceval.c:3442)
20 org.python.python 0x002cdfae t_bootstrap + 62
(threadmodule.c:424)
21 libSystem.B.dylib 0x90023d67 _pthread_body + 84

Crash2:

Thread 4 Crashed:
0 com.apple.HIToolbox 0x92e30522
TThemeCacheObject::GetRefCount() + 6
1 com.apple.HIToolbox 0x92e341f4
TThemeTextCache::TextDictRelease(TThemeText*) + 22
2 com.apple.CoreFoundation 0x9083d729
CFDictionaryRemoveValue + 470
3 com.apple.HIToolbox 0x92e2ed53 TThemeTextCache::Create
(__CFString const*, _HIThemeTextInfo const*) + 71
4 com.apple.HIToolbox 0x92e2ec6d ThemeTextCreate
(__CFString const*, _HIThemeTextInfo const*) + 33
5 com.apple.HIToolbox 0x92e2eab3
DataEngine::GetTextDimensions(__CFString const*, float,
_HIThemeTextInfo*, float*, float*, float*) + 239
6 com.apple.HIToolbox 0x92e2e969
HIThemeGetTextDimensions + 131
7 com.apple.HIToolbox 0x92f1b148
HIStaticTextView::GetOptimalSizeSelf(CGSize*, float*) + 238
8 com.apple.HIToolbox 0x92e4e359
HIView::SendGetOptimalBounds(CGRect*, float*) + 137
9 com.apple.HIToolbox 0x92e4e2ab HIView::GetOptimalSize
(CGSize*, float*) + 47
10 com.apple.HIToolbox 0x92e7515f GetBestControlRect + 107
11 libwx_macud-2.8.0.dylib 0x120bc962
wxMacControl::GetBestRect(Rect*) + 34
12 libwx_macud-2.8.0.dylib 0x1212f434
wxStaticText::DoGetBestSize() const + 336
13 libwx_macud-2.8.0.dylib 0x1212f191 wxStaticText::SetLabel
(wxString const&) + 405
14 _core_.so 0x005bb444 wxIntersectRect
(wxRect*, wxRect*) + 196444
15 org.python.python 0x0040eb00 PyObject_Call + 45
16 org.python.python 0x00497b67 PyEval_EvalFrameEx +
13791
17 org.python.python 0x0049ae0e PyEval_EvalCodeEx + 1819
18 org.python.python 0x00498618 PyEval_EvalFrameEx +
16528
19 org.python.python 0x004999ce PyEval_EvalFrameEx +
21574
20 org.python.python 0x004999ce PyEval_EvalFrameEx +
21574
21 org.python.python 0x0049ae0e PyEval_EvalCodeEx + 1819
22 org.python.python 0x0042fe6b PyFunction_SetClosure
+ 2019
23 org.python.python 0x0040eb00 PyObject_Call + 45
24 org.python.python 0x004167ae PyMethod_New + 2432
25 org.python.python 0x0040eb00 PyObject_Call + 45
26 org.python.python 0x004936fe
PyEval_CallObjectWithKeywords + 112
27 org.python.python 0x004cdfae _PyObject_GC_NewVar +
1965
28 libSystem.B.dylib 0x90023d67 _pthread_body + 84


I'm not 100% sure how to read those yet. Is the 0th entry the last
operation before the crash, or the higher number (guessing 0). Is
the presence of a block of PyEval's in there significant, or is that
just a normal part of python cmd parsing? How about the
StaticText::SetLabel - that is also always in the chain for my crashes.

Again, thanks for the comments so far!

-Ross

> --
> To unsubscribe, send email to wxPython-users
> +unsub...@googlegroups.com
> or visit http://groups.google.com/group/wxPython-users?hl=en

[werner]

Hi Ross,

Ross wrote:
> Thanks for your thoughtful reply! It's at very least helpful to
> know that someone else managed to find some success dealing with
> this. It is truly very elusive for me, but no doubt a good learning
> experience.
>
> I'm very daunted by the idea of ripping things out.

I don't ripp things out - I kind of go the other way.

I make a small application, most often by making a copy the module in
question/suspected and add whatever is needed to make this run in a test
folder, if the problem doesn't happen then I add additional stuff in, if
it happens I start to comment stuff.

Another approach is to add print statements all over the place (areas
you suspect) in your app and see which one is the last to show - a more
sophisticated variation to this is using the Python logging module, you
can leave all the log calls in your app and just change the log level to
de-activate them.

Werner

[Mike Driscoll]

I don't rip everything out either. I usually just comment out sections
or calls to methods until the problem stops, then I start uncommenting
until I find the issue. I've also done what Werner does where you take
some naughty code and put it in its own file so I can poke at it.

It's always a good idea to write a few lines, test, fix, repeat. Don't
go writing several functions and then test. You'll regret it later -
snake bites aren't exactly fun after all.

-------------------
Mike Driscoll

Blog: http://blog.pythonlibrary.org

[Ross]

Makes good sense of course, but in this case the cause is more
elusive. I have tested each feature to ensure it worked then moved
on. But the crashing has begun very late in development, and without
any repeatability it seems. I try to do the same things over and it
doesn't crash at all. Sometimes I can use the app for 20min without
the crash, other times, through the same path, and it crashes after a
minute.

With regards to the print statement comments - again, I've tried
that but the crashes happen at random times, and the prints have
proved inconclusive so far.

:(

R.

On 3-Dec-09, at 1:50 PM, Mike Driscoll wrote:
>
> I don't rip everything out either. I usually just comment out sections
> or calls to methods until the problem stops, then I start uncommenting
> until I find the issue. I've also done what Werner does where you take
> some naughty code and put it in its own file so I can poke at it.
>
> It's always a good idea to write a few lines, test, fix, repeat. Don't
> go writing several functions and then test. You'll regret it later -
> snake bites aren't exactly fun after all.
>
> -------------------
> Mike Driscoll
>
> Blog: http://blog.pythonlibrary.org
>

[Sam23]

(a) The logs look interesting. There is no equivalent in Windows. Maybe
someone here with can interpret the logs and provide some clues to the
crash. That could save a lot of time with the troubleshooting.

(b) If I interpret your log correctly, you are doing wx operations in
threads, and that itself might be the cause of the crash. This is
because wx is not threadsafe. All wx operations must be done in the main
thread.

You can do calculations or data processing in threads but when you need
to do any wx operation, you have to do it in the main thread. You can
use wx CallAfter from a thread to get the main thread to execute a
method. You may also need to use queues or locks to synchronise between
the threads. Or use queues or locks to communicate/coordinate with the
main thread if you don't want to use wxCallAfter.

(c) As Werner and Mike points out, you can also do the comment approach
or the build small model approach. I think all 3: delete, comment and
small model are based on the same idea, trying to get the smallest
subset of code possible to reproduce the problem.

Delete and comment are similar in that you start with a large block of
code that *definitely* has the problem and try to successively disable
chunks of code either by deletion or commenting to isolate the code that
has the problem. I think it is a matter preference whether to delete or
comment.

I find it easier to delete because it is easier for me to delete than to
comment (putting back is no problem either). It is easier to look at
when I do the comparison (between the changed and unchanged version)
using diff in vim. I also end up with successively less and less code to
look at with each iteration. But of course it is just 1 or 2 lines,
comment would be faster.

The build new small model is not bad either. I think to get results
faster with this approach, some intuition about which parts of the code
are suspect would help. The advantage with delete/comment, is that you
know for *sure* the problem is there in the code you start with.

(d) If the problem is not due to running wx actions in threads, and if
you do decide to try the approach of isolating the problem with either
of the 3 methods; some things that may be worth highlighting again.

Although your application is complex, the cause of the problem is
probably simple. The idea is that you need to get rid of all the
complexity that is not related to GUI. For example, you can get replace
all the threading with stubs in the main thread, since there shouldn't
be any wx in threads. UI interactions need to be part of the "remove and
test" because they are wx operations and can be the cause of the crash.
Anything else you should try to remove. To remove UI interactions while
still letting the app run may require replacing them with stubs that
simply return the expected result of the UI interaction.

I know it sounds daunting. I also felt that it wasn't possible with my
code, even though it was just a prototype - it already had enough code
and flow between different states to make it look impossible. I didn't
try it for a quite after Werner and Robin suggested it..... didn't have
a clue how to start. I only did it when I saw that I had no other choice.

Maybe your application is already too complex to do try this. But you
want to give it a shot if there is no other choice. The key idea (for
encouragement!) is that the application is complex, but the problem
usually isn't - could be as trivial as mine proved to be. You need to
figure out how to get rid of all the application-related complexity, and
leave behind only the GUI-related code (mouse events, key events, widget
painting, etc) for the divide-and-conquer approach.

If you want to go with the build small model approach, one possibility
is as follows:
Start the most suspicious part - in this case, it may be your complex UI
task after the many user input operations. Create only the complex ui
part and use stubs to substitute for the information gathered by the
preceding UI operations.

Good luck!

Ross wrote the following on 12/3/2009 11:16 PM:
> I'm very daunted by the idea of ripping things out. My app is a
> rather complex thing
>

[Mike Driscoll]

On Dec 4, 12:15 am, Sam23 <qm0...@gmail.com> wrote:
> (a) The logs look interesting. There is no equivalent in Windows. Maybe
> someone here with can interpret the logs and provide some clues to the
> crash. That could save a lot of time with the troubleshooting.
>
> (b) If I interpret your log correctly, you are doing wx operations in
> threads, and that itself might be the cause of the crash.  This is
> because wx is not threadsafe. All wx operations must be done in the main
> thread.
>
> You can do calculations or data processing in threads but when you need
> to do any wx operation, you have to do it in the main thread. You can
> use wx CallAfter from a thread to get the main thread to execute a
> method. You may also need to use queues or locks to synchronise between
> the threads. Or use queues or locks to communicate/coordinate with the
> main thread if you don't want to use wxCallAfter.

I noticed that he was using threads right after I sent my previous
message. You are correct about wx not being thread-safe. wxPostEvent
(which wx.CallAfter calls) and wx.CallLater are three thread-safe
methods. See also this wiki page on the topic: http://wiki.wxpython.org/LongRunningTasks

>
> (c) As Werner and Mike points out, you can also do the comment approach
> or the build small model approach. I think all 3: delete, comment and
> small model are based on the same idea, trying to get the smallest
> subset of code possible to reproduce the problem.
>
> Delete and comment are similar in that you start with a large block of
> code that *definitely* has the problem and try to successively disable
> chunks of code either by deletion or commenting to isolate the code that
> has the problem.  I think it is a matter preference whether to delete or
> comment.
>
> I find it easier to delete because it is easier for me to delete than to
> comment (putting back is no problem either). It is easier to look at
> when I do the comparison (between the changed and unchanged version)
> using diff in vim. I also end up with successively less and less code to
> look at with each iteration.  But of course it is just 1 or 2 lines,
> comment would be faster.

This is interesting. IDLE and Wing IDE both provide ways to comment
out any line (or lines) that are selected. I'm surprised vim doesn't
have a way to do this also.

[C M]

On Fri, Nov 27, 2009 at 7:56 PM, Ross <ros...@gmail.com> wrote:

> Hello All,
>
> I have an app built on python 2.5.2 on mac osx 10.4.11. The GUI
> elements are built on wxPython 2.8.10.1.   Development has gone
> pretty well, but I've got an intermittent rather silent crash that
> happens without spewing any error messages to my console at all.

Do you happen to have a previous version(s) of the app that didn't show
the crashing problem? If so, can you use that to try to figure out what
new change you might have made just before you noticed the it?

I would think that the problem must have come pretty late in
development, or else you would have addressed it before you built
the app up to the complexity it is now. Was there anything out of
the ordinary (beyond basic Python and stock GUI stuff) that you did
as a late-stage addition to the project?

Che

[cptnwillard]

.

[cptnwillard]

> I'm not 100% sure how to read those yet. Is the 0th entry the last
> operation before the crash, or the higher number (guessing 0).

The last operation before the crash is the 0th entry. Thus, everything
that happens before "PyObject_Call" is interpretation of your Python
code which you typically do not care about.

What you should care about is this :

> 0 libobjc.A.dylib 0x90a594c7 objc_msgSend + 23
> 1 com.apple.HIToolbox 0x92e28b18 HIView::SetFrame
> (CGRect const&) + 708
> 2 com.apple.HIToolbox 0x92e42e57 HIViewSetFrame + 57
> 3 libwx_macud-2.8.0.dylib 0x026ad8fe wxMacControl::SetRect
> (Rect*) + 86
> 4 libwx_macud-2.8.0.dylib 0x0273b440 wxWindow::DoMoveWindow
> (int, int, int, int) + 546
> 5 libwx_macud-2.8.0.dylib 0x027374a9 wxWindow::DoSetSize
> (int, int, int, int, int) + 191
> 6 libwx_macud-2.8.0.dylib 0x02720142 wxStaticText::SetLabel
> (wxString const&) + 326
> 7 _core_.so 0x02051444 _wrap_Window_SetLabel
> + 208 (_core_wrap.cpp:34767)
> 8 org.python.python 0x0020eb00 PyObject_Call + 45
> (abstract.c:1861)

What is most important here is the first call after "PyObject_Call",
which is "_wrap_Window_SetLabel" and probably means a call to
"wx.Window.SetLabel" is what triggered the crash (but the problem may
be elsewhere).

> ...
>
> read more »

[Ross]

Thanks again for the thoughtful input from Sam23 and Mike!

Some promising suggestions there. I wasn't aware of the wx in
threads issues (my greenitude showing). Amazing that it can work
for thousand of operations without complaint. Would be nice to see
some exceptions thrown to highlight that mistake. Hopefully that is
the source of my woes. I'll start digging into that. I have been
employing locks to avoid thread collisions in some file and list
management.

The process I was currently exploring was moving to Python 2.6.4 in
the hope that there was better error reporting to catch something bad
I was doing in 2.5.2. Interestingly there were indeed a few errors,
while 2.5.2 had run clean. But alas, after fixing those and running
my app for 20min in 2.6.4, I did get another crash.

Onto some reading on using wxPostEvent, wx.CallAfter and
wx.CallLater, and following up on the link to http://
wiki.wxpython.org/LongRunningTasks.

Thanks - I'll be sure to follow up with outcomes for other poor souls
that might follow this path.

Ross

[Cody Precord]

Hi,

Here is the problem you are making UI calls from a background thread.
All UI calls should be in thread 0. This stack trace shows that ui
calls are being evaluated in thread 4 (which unless you created your
app object in this background thread) is causing the crash.

Cody

[Cody Precord]

Hi,

On Fri, Dec 4, 2009 at 10:53 AM, Ross <ros...@gmail.com> wrote:
> Thanks again for the thoughtful input from Sam23 and Mike!
>
> Some promising suggestions there.  I wasn't aware of the wx in
> threads issues (my greenitude showing).   Amazing that it can work
> for thousand of operations without complaint. Would be nice to see
> some exceptions thrown to highlight that mistake.   Hopefully that is
> the source of my woes.  I'll start digging into that.   I have been
> employing locks to avoid thread collisions in some file and list
> management.

The gui is not thread safe, it doesn't mean that it can't be accessed
from background threads but it does not guarantee safe access to
memory owned by objects running on the main thread.

So it is a bit of a timing issue, if the you make a gui call from a
background thread that modifies memory in the ui it will be fine as
long as no other object on the ui thread is trying to make
simultaneous access to that same memory, when that happens you get the
crash. Because of this is why you don't see it everytime the routine
is invoked.


Cody

[Ross]

Thanks a lot Cody and cptnwillard. That is helpful to better
understand those crash traces. Sounds like good consensus that I've
messed up with wx clashing on threading.

That would fit with my sense that it was some unpredictable collision
that would be hard to track down with print statements. Tramping on
the wrong memory loc'n based on thread actions seemed independent of
what the user was doing at any moment.

Again - thanks for the great help!

Ross.






>
> Cody

[Mike Driscoll]

What Cody and cptnwillard didn't mention here is that the common "fix"
is to wrap wxPython calls in wx.CallAfter (or one of the other methods
already mentioned). So for the SetLabel example, it would be something
like this:

wx.CallAfter(mywxProgram.someWidget.SetLabel, "some text")

[Sam23]

Ross,
At this point, after so many wxPython experts have weighed in, I think the source of the crash has been clearly identified. I just add some additional comments below.

There isn't much doubt the source of the problem is due to the use of wx code in threads. I guess the fact that it can work for thousands of operations without crashing simply shows that the likelihood of a collision is very low. There must be a conflicting access by  2 or more threads at exactly the same time.

But as the number of threads and operations are increased, the likelihood becomes increasingly likely; especially over a prolonged period.  If you made a small app that has multiple threads running the same set of gui operations that you have in your crashing app; and make the threads run continuously in an endless loop, you can probably precipitate a crash in well under a minute.

I had a further look at the logs of the 2 crash that you provided, and see that the 2 crashes had entirely difference sequences. This is in line with the idea that the crashes are caused by randomly colliding threads - colliding at different parts of the code. If there was a bug in a fixed position of the code causing the crash (as in my case), the sequence should be similar for both crashes. So in your case, the code is really crashing all over the place as the print statements suggests.  I wish Windows had these types of logs. 

This is from the wxWidgets reference in the wxPython demo:

There are two fundamentally different ways to use threads in GUI programs and either way has to take care of the fact that the GUI library itself usually is not multi-threading safe, i.e. that it might crash if two threads try to access the GUI class simultaneously. One way to prevent that is have a normal GUI program in the main thread and some worker threads which work in the background.

Cheers and good luck with your fixing.  Hopefully, wx.CallAfter will do the trick for you. It has been invaluable for me.

[Sam23]

The standard install of VIM doesn't do very much for Python aside from
syntax highlighting and auto-indentation. I started learning VIM at the
same time as Python, and yup as a VB guy, had a lot of pain. I had to
comment very manually, ^i#ESC ^i # etc. I resorted to using ''' for
commenting out large swathes of code, and eventually just deleted them.

Much, much later (after my unhandled exception scare), I found a vim
script that made VIM do much more for Python editing including the
ability to comment highlighted rows. But I still think that for the
disable and test, delete may be faster.. Still, maybe it is from my
experience before having found that utility.

On another note, I still can't get very good python editing capabilities
in vim. Folding works most of the time but fouls up when there is line
continuation e.g. for if statement with multiple conditions; for blank
lines, etc. If I have the same program open in 2 tabs, folding fails
in one of them.

Anyone knows a good script for vim for python let me know! The one I
have is pretty good - can even folds comments, imports, etc, and can
build a list of classes and methods - but as I said, it messes up now
and then .

[Sam23]

These logs offer some glimpse into a crash that happens outside of
Python. Is there anyway to get a similar log in Windows for crashes
outside Python?

[Sam23]

Yes, wx.CallAfter is real neat for solving this type of problem. If it
works for your design, you probably won't even need to restructure your
existing threaded code much to get wx calls back into main thread.

[Ross]

Thanks everyone who chimed in. It was a very helpful to get a handle
on my issue.

CallAfter does seem to have managed those collisions for me. One of
my threads is a timer thread which synchronizes between both some UI
elements and some media playback elements, so the text labelling of
the timer display seems to be that culprit.

In another thread I do some wx stuff, but it is confined to that
thread I beleive, but I'll be taking a close look to ensure I don't
have some other, less frequent crashes waiting to happen.

Thanks again.

Ross.

[Ross]

Hello All,

I learned about the rudiments of wx.CallAfter for multithreaded GUI
projects from some conversations on this list, and it has been
serving me well.

I have two questions related to that, and appropriate usage.

1) When doing some graphicsDC work, I am using wx.CallAfter for
drawing commands with good success. I'm wondering it is appropriate
to use wx.CallAfter() isolation for ALL cmds that interact with the
DC or does it make sense only for one's which result in visible GUI
changes.

ie. Do I need to CallAfter for all these dc interactions:

wx.CallAfter(dc.BeginDrawing)
wx.CallAfter(dc.SetPen, wx.Pen('GREEN'))
wx.CallAfter(dc.DrawRectangle, bw, bh, RectWide, RectHi)

or only for

wx.CallAfter(dc.DrawRectangle, bw, bh, RectWide, RectHi)

as that last call is the only one that effects a display change?

2) Let's say I have two GUI interacting routines DrawSomething() and
MkDwgChanges() that are both in my app thread, and I occasionally
call MkDwgChanges() from another thread.

I use wx.CallAfter() wrappers in my MkDwgChanges() routine so that it
will not cause cross-thread collision crashes. But will those
wx.CallAfter()'s cause any concerns when they are called within in
the Thread0 execution? ie where they are not strictly needed because
they are in the non-cross-threaded context with the wx GUI widgets in
the same thread?

Thanks in advance for any help on understanding those Q's.

Regards,

Ross.

[Christopher Barker]

Ross wrote:
> 1) When doing some graphicsDC work, I am using wx.CallAfter for drawing
> commands with good success. I'm wondering it is appropriate to use
> wx.CallAfter() isolation for ALL cmds that interact with the DC or does
> it make sense only for one's which result in visible GUI changes.

yup - you never quote know what might cause problems -- AY wx cal should
be in the main thread.

> ie. Do I need to CallAfter for all these dc interactions:
>
> wx.CallAfter(dc.BeginDrawing)
> wx.CallAfter(dc.SetPen, wx.Pen('GREEN'))
> wx.CallAfter(dc.DrawRectangle, bw, bh, RectWide, RectHi)

usually those are done all together, in a single method, so it would be
only one CallAfter to call that method.

In fact, particularly for dc.BeginDrawing, you realy wouldn't want all
the other stuff called in separate events.

> But will those
> wx.CallAfter()'s cause any concerns when they are called within in the
> Thread0 execution?

no -- CallAfter is not a thread-specific =call -- it is just a thread
safe one. What it does is put an event on the event stack that will call
your function when it reaches the top (bottom?) of the stack. There are
often uses for it even within a single thread.

-Chris

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[Ross]

Thanks Chris, that is helpful.

Ross.