Importing wx in main thread before creating App and running MainLoop in another
Peter Hansen
main thread if you were not going to run MainLoop() there too.
Sometime around 2.5.x this was changed, and wx initialization was
moved to when App() was created, allowing you to run wx in non-main
threads. Life was wonderful. :) (See
http://groups.google.com/group/wxpython-users/browse_thread/thread/ef66076486b84a40/0320124d25d73dda
for some background. Entirely a coincidence that I was involved in
that thread too.)
Today I've just isolated a problem we've been having with OSX
(10.4/10.5) using wxPython version 2.8.10.1 Unicode (on Python 2.6),
and the above problem seems to be back.
As it turns out, of one main application and a dozen in-module test
fixtures and utilities, the only ones which did not freeze as soon as
the main Frame appeared were the single-threaded ones, which call
MainLoop() in the main thread, where wx was first imported.
The main application actually was freezing in the same way a couple of
weeks ago, but in the interim I refactored the startup code to ensure
wx was not imported for the first time until it was done in the thread
which would eventually run the GUI. (Even though we believed this was
no longer necessary, it's long been considered one of our internal
best practices, and when I saw we were violating it I changed the
code. A fortunate thing, since I might not have tracked this down any
time soon if I had noticed that the main app suddenly worked again on
Mac.)
The same problem does NOT happen on Windows with the same scripts. It
also looks like it was probably not happening with Python 2.5 and an
earlier version of wxPython (2.8.7.1) on Mac, since we've had no such
issue in spite of the main app being "broken" in this way for probably
well over a year now.
Is this a regression? Intentional change? Not reported by anyone
else yet? (I did check the bug list, but saw nothing I could match
up, nor anything in the change history.)
--
Peter Hansen
Robin Dunn
> Is this a regression? Intentional change? Not reported by anyone
> else yet? (I did check the bug list, but saw nothing I could match
> up, nor anything in the change history.)
A regression, although it's possible that there may be some issue with
Carbon or how we use it that is causing the problem. Can you provide a
small sample so I can be sure I'm looking at the same thing you are?
--
Robin Dunn
Software Craftsman
http://wxPython.org
Peter Hansen
> Can you provide a
> small sample so I can be sure I'm looking at the same thing you are?
Here's a start. Unfortunately this may expand the scope beyond just
wx, since it turns out there's a connection to matplotlib as well.
------------------
# osx_freeze.py
# demonstration of wx freeze on OSX
# run with "python osx_freeze.py freeze" to show the problem (non-
responsiveness after ~3 seconds)
# and without the argument "freeze" to show it working
# removing the following import avoids the freeze
# a simple "import matplotlib" also avoids the freeze
import matplotlib.figure
import wx
print 'matplotlib', matplotlib.__version__
print 'wx', wx.version()
if __name__ == '__main__':
import sys
def gui():
app = wx.App(False)
frame = wx.Frame(None)
wx.StaticText(frame, label='Goodbye, cruel world')
frame.Show()
app.MainLoop()
if len(sys.argv) > 1 and sys.argv[1] == 'freeze':
import threading
threading.Thread(target=gui).start()
else:
gui()
----------------
So the above running on Python 2.6, mpl 0.99.1.1 and wx 2.8.10.1 (mac-
unicode) will freeze. It won't freeze on Windows with the same
versions.
The same thing running on Python 2.5, mpl 0.91.4, wx 2.8.7.1 (mac-
unicode) will not freeze.
All Mac tests on OSX 10.5.
I can do further narrowing-down if required. If this suggests
anything to you or you can tell me more about what "same thing" you
might have noticed, it might help focus my efforts if I knew more
about it.
--
Peter Hansen
Peter Hansen
> Here's a start. Unfortunately this may expand the scope beyond just
> wx, since it turns out there's a connection to matplotlib as well.
>
> unicode) will not freeze.
Okay, I was able to get on another Mac here, running 10.4. Using the
Python 2.5 installation, I upgraded to wxPython 2.8.10.1 and confirmed
that it does NOT freeze, which somewhat points to changes from mpl
0.91.4 to 0.99.1 as the problem. I'll attempt to build matplotlib on
this 10.4 machine (as there are no pre-built binaries that I can find).
Christopher Barker
> Okay, I was able to get on another Mac here, running 10.4. Using the
> Python 2.5 installation, I upgraded to wxPython 2.8.10.1 and confirmed
> that it does NOT freeze, which somewhat points to changes from mpl
> 0.91.4 to 0.99.1 as the problem. I'll attempt to build matplotlib on
> this 10.4 machine (as there are no pre-built binaries that I can find).
yes there are -- the ones labeled "10.3" are for 10.3.9 and above --
i.e. the python.org python build, and should work fine on 10.4:
http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-0.99.1/
I think you want:
matplotlib-0.99.1.1-py2.5-macosx-10.3-i386.egg
it says "i386" which you may want anyway, but if you need PPC, it may be
there too -- setuptools puts the architecture name it was built on in
there, even if it was indeed built at a universal binary.
worth a try anyway,
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Robin Dunn
I can verify the apparent matplotlib connection, and also the fact that
it works if only matplotlib is imported and not matplotlib.figure. I
have matplotlib 0.98.5.1 installed currently.
Peter Hansen
works.. unfortunately I think two days of trying to build matplotlib
from source (problems with libpng) and then using macports (issues
relating to atlas, which everyone is having) I think I've buggered up
the Python 2.5 install enough that it's no longer a useful testbed.
I'll reinstall later maybe.)
On Jan 28, 2:51 pm, Robin Dunn <ro...@alldunn.com> wrote:
> I can verify the apparent matplotlib connection, and also the fact that
> it works if only matplotlib is imported and not matplotlib.figure. I
> have matplotlib 0.98.5.1 installed currently.
Wow... with that I went digging into matplotlib.figure, cutting out
stuff until I had only matplotlib.axes, then from there to
matplotlib.dates, then to "pytz"! Amazing... it's (apparently, at
this stage) nothing to do with matplotlib... other than maybe the
older one didn't import pytz there.
I'll continue digging, but a surprising result there so far. Changing
my test code to just "import pytz, wx" with no matplotlib in sight
still leads to the freeze.
--
Peter
Peter Hansen
> I'll continue digging, but a surprising result there so far. Changing
> my test code to just "import pytz, wx" with no matplotlib in sight
> still leads to the freeze.
Continuing on... pytz is not at fault either, but it appears
pkg_resources (part of setuptools) is, somewhere after 0.6c8.
Upgrading to 0.6c11 on the Python 2.5 installation changes a "no
freeze" situation to a "freeze" situation.
--
Peter
Peter Hansen
> Continuing on... pytz is not at fault either, but it appears
> pkg_resources (part of setuptools) is, somewhere after 0.6c8.
> Upgrading to 0.6c11 on the Python 2.5 installation changes a "no
> freeze" situation to a "freeze" situation.
And specifically it was 0.6c10 which first shows the failure. Next
stop: find culprit line(s).
For anyone checking this, the test code imports are now merely "import
pkg_resources, wx".
--
Peter
Peter Hansen
> And specifically it was 0.6c10 which first shows the failure. Next
> stop: find culprit line(s).
>
> For anyone checking this, the test code imports are now merely "import
> pkg_resources, wx".
Okay, can't blame that package either!
The change in pkg_resources that triggers this involves finding the
Mac OS version info, but it uses the standard library "platform"
module to do it. Narrowing it down further, I've got the test code
doing "from platform import mac_ver; mac_ver()" and it freezes with
just that (no pkg_resources involved).
(Robin, ringing any bells yet? I'm hoping at some point you'll
suddenly see a connection to something in recent wxPythons, or you or
someone else will realize why Mac version detection would have any
effect on this. I know next to nothing about Macs, so unfortunately
I'm likely very near the end of my utility here...)
--
Peter
Peter Hansen
without guidance.
Here's all it takes (replacing the original "import
matplotlib.figure"):
from gestalt import gestalt
try:
print gestalt('sysu')
except Exception, ex:
print 'gestalt raised', ex
import wx
That "gestalt" module is called by _mac_ver_lookup() in the standard
library platform.py. It's called with a tuple of three items ('sysv',
'sysu', 'sysa'). You can safely pass in the "sysv" or "sysa" values
without this triggering a freeze (they return actual results) but with
"sysu" you get a MacOS.Error exception and then you'll get the freeze
when wx is later started in a non-main thread.
In fact, if I just move the "import wx" into the non-main thread as
well, I still get the freeze, so this isn't quite the earlier case of
"import wx in main thread, then build App and run MainLoop in other
thread".
This gestalt thing is from http://www.rgaros.nl/gestalt/ according to
comments in platform.py.
Any input from anyone on what any of this means would be welcome. I'm
happy to go somewhere and submit a bug report, if that would help, but
I have no idea which package it would be in, other than wxPython since
it's the only thing I've noticed that's being negatively affected by
whatever side-effect there is of a call to gestalt() that raises an
exception.
--
Peter Hansen
Robin Dunn
The gestalt module is in the Python source tree in
Mac/Modules/gestaltmodule.c. It's a very small wrapper around the
Carbon function Gestalt which is documented here:
http://developer.apple.com/mac/library/DOCUMENTATION/Carbon/Reference/Gestalt_Manager/Reference/reference.html
I've no idea why calling it is causing this problem, but I can guess
that it must be initializing enough of Carbon (in the context of the
main thread) to cause problems when wx is trying to use it from an
alternate thread. I also have no idea about what to do for a fix,
although patching setuptools might be an adequate workaround.
Peter Hansen
> The gestalt module ...
> I've no idea why calling it is causing this problem, but I can guess
> that it must be initializing enough of Carbon (in the context of the
> main thread) to cause problems when wx is trying to use it from an
> alternate thread. I also have no idea about what to do for a fix,
> although patching setuptools might be an adequate workaround.
Since it only causes the problem when called with a "selector"
that leads to an exception (in this case "sysu", which is apparently
for getting system update version info
http://developer.apple.com/mac/library/DOCUMENTATION/Carbon/Reference/Gestalt_Manager/Reference/reference.html#//apple_ref/doc/constant_group/System_Update_Version_Selector),
perhaps there's no reason setuptools needs to try that selector.
Or perhaps the fact that it's failing is actually a sign of some
other
problem that can be fixed.
I'm guessing this is the sort of thing that neither the Python
developers
nor the setuptools developers will be interested in investigating,
let alone fixing (since it may not be a "bug" per se), unless someone
can offer a patch with no negative side effects and clear
documentation
about why it's a necessary improvement.
I wonder if calling gestalt.gestalt('sysu') *ever* returns a value
instead
of raising an exception. Maybe if we can correlate the exception with
a
particular situation we can figure out a way of deciding in advance
when
not to call it with "sysu". Or maybe someone can show that it causes
this
wx problem even when it successfully returns a value with "sysu", in
which
case there might be an alternative to get the same data safely.
--
Peter Hansen
Peter Hansen
> I wonder if calling gestalt.gestalt('sysu') *ever* returns a value
> instead of raising an exception.
It probably does not: see http://bugs.python.org/issue780461
It's unfortunate that calling this has this side effect.
The only short-term solution I can think of would be to hot-patch
platform *prior* to importing anything else that might import it
and call mac_ver(). You'd substitute a custom version of mac_ver()
which doesn't bother calling gestalt() with "sysu". It would have
no effect on anything since as noted in the comments for that
function:
# NOTE: this block is left as documentation of the
# intention of this function, the 'sysu' gestalt is no
# longer available and there are no alternatives.
I'll create a bug report for Python, asking that this block be
removed because of this previously unreported side effect.
--
Peter Hansen
Peter Hansen
> I'll create a bug report for Python, asking that this block be
> removed because of this previously unreported side effect.
Created as http://bugs.python.org/issue7812
--
Peter Hansen
Christopher Barker
Great detective work --thanks!
Peter Hansen wrote:
> I'll create a bug report for Python, asking that this block be
> removed because of this previously unreported side effect.
A post to the python-mac group is in order too.
In the long term, Carbon is deprecated anyway, so I have no idea what
the plans are.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
Robin Dunn
Yes, that approach sounds good. And a runtime monkey-patch of the
platform module would be simple to do in the meantime.
Peter Hansen
> A post to the python-mac group is in order too.
Christopher, if you think so, would you mind doing it? To
demonstrate
how little I actually know about Macs and this whole area, I'll point
out that I don't even know what Carbon *is*, other than what I can
infer from Robin's words (something to do with the Mac GUI). Given
that, I'd probably sound clueless (because I am) when trying to
explain what any of this means and why they should care.
--
Peter Hansen
Christopher Barker
> On Jan 30, 11:06 am, Christopher Barker <Chris.Bar...@noaa.gov> wrote:
>> A post to the python-mac group is in order too.
>
> Christopher, if you think so, would you mind doing it?
I could, though I have purged my email, and don't have a summary of the
final identification of the problem.
> To
> demonstrate
> how little I actually know about Macs and this whole area, I'll point
> out that I don't even know what Carbon *is*, other than what I can
> infer from Robin's words (something to do with the Mac GUI). Given
> that, I'd probably sound clueless (because I am) when trying to
> explain what any of this means and why they should care.
Well, you don't need to know anything, or worry about sounding clueless
(we could all use a lot more clues than we have!). Simply identifying
what happens is a great start -- just so it gets into the right hands.
FWIW, Carbon is Apple's library that is more-or-less compatible with the
old MAC OS System -- mostly GUI, but other things as well. Since wx was
originally built on the old Mac, it still relies a lot on Carbon. Carbon
is now deprecated (will never be 64 bit), so the latest wx is built on
the newer stuff: Cocoa, but ti's not quite production ready yet.
Anyway, if you send me a sample script that breaks something, I'll post
it on the Mac list.
-Chris
> --
> Peter Hansen
>
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Peter Hansen
> Anyway, if you send me a sample script that breaks something, I'll post
> it on the Mac list.
I will email this to Chris directly, but am posting here for
posterity, as this version summarizes everything and doesn't
muddle things with the "import wx" being in the main thread, etc.
---------------------
# Demonstration of application freeze on OSX.
# Running "python osx_freeze.py" shows the symptom:
# non-responsiveness after about 3 seconds. In theory
# this will occur in any wxPython app which runs
# the GUI event loop in a thread other than the one
# in which the failing gestalt('sysu') call (see below)
# is first executed.
# Enabling any of these imports can trigger the problem
# as each one imports the next one down in the list.
# This will happen only with pkg_resources v0.6c10 or later
# although it's not pkg_resources' fault.
# import matplotlib.figure
# import matplotlib.axes
# import matplotlib.dates
# import pytz
# import pkg_resources
# The next import alone doesn't lead to failure,
# but the function call does.
# This same code is executed by pkg_resources starting
# with setuptools v0.6c10.
# from platform import mac_ver; mac_ver()
# The platform.mac_ver() call does the following, which
# is what ultimately leads to the failure. Calls
# to gestalt() with 'sysa' (for example) do not
# trigger the problem, either because the problem is
# related to the 'sysu' itself, or to the fact that
# an exception is raised because the selector "sysu"
# is no longer recognized.
from gestalt import gestalt
try:
print gestalt('sysu')
except Exception, ex:
print 'gestalt call failed'
if __name__ == '__main__':
def gui():
import wx
app = wx.App(False)
frame = wx.Frame(None)
wx.StaticText(frame, label='Goodbye, cruel world')
frame.Show()
app.MainLoop()
import threading
threading.Thread(target=gui).start()
--
Peter Hansen