Interactive usage of wxPython

[Brian Granger]

Hi,

I am one of the core developers of IPython.  One of IPython's most popular features is that GUI toolkits (like wxPython) can be used interactively from it.  This is used by packages like matplotlib and Mayavi to provide (extremely popular) interactive plotting.  However, our current implementation uses threads in a manner that is very subtle, easy to break and difficult to maintain.  We are currently exploring alternatives that would be more robust.

The core issue that needs to be solved is that the GUI event loop needs to continue to run when IPython's input prompt is waiting for user input.

A number of the other GUI toolkits that IPython supports (pyqt, pygtk, tk, cocoa/pyobjc) have recently added features that make their interactive use much more straightforward.  All of these toolkits are now using Python's somewhat poorly documented PyOS_InputHook hook.  This makes it possible to use these toolkits interactive from both IPython and the regular Python prompt with no threading.

I am writing for two reasons:

1.  I have a Cython based prototype that implements the PyOS_InputHook for wxPython.  I don't know much about wxPython and I need some help getting it to work in a better manner.

2.  This code should really be in wxPython in the first place.  If we can get the code figured out and in wxPython, all wxPython users could use/dev wxPython apps interactively.

Here is a link to a post on the ipython-dev list describing more of the details of this:

http://mail.scipy.org/pipermail/ipython-dev/2009-February/004856.html

Most relevant, is Michiel de Hoon's post about the implementation of this in the other GUI toolkits:

I wrote the code in PyGTK that uses PyOS_InputHook for interactivity, as well as the Mac OS X native backend for matplotlib that uses PyOS_InputHook in exactly the same way. PyQT and Tkinter also use PyOS_InputHook, though the code is a bit kludgy on Windows. So I definitely agree that PyOS_InputHook is the right way to go.

Your current code should work, but there's a better way to do it. If I understand the code correctly, you rely on the fact that PyOS_InputHook is called repeatedly by readline, and you use PyOS_InputHook to process wx events that need to be processed at that time. A better way is to use PyOS_InputHook to start the wx event loop, but have this event loop check stdin. As soon as some input is available on stdin, you exit the event loop, which means that PyOS_InputHook returns, and Python can proceed to handle the command that was just entered by the user on stdin.

Essentially, think of wx's event loop as sitting in a call to select(), waiting for the next wx event to arrive. You want to add fileno(stdin) to the set of file descriptors watched by select().

There are two advantages to this approach. First, it does not rely on readline calling PyOS_InputHook repeatedly. This is important, since Python may not be using readline at all, and if it is, depending on the Python version and how readline was installed it may call PyOS_InputHook only once. Second, this approach is more efficient (not wasting processor cycles going back and forth between readline and PyOS_InputHook), and gives a better response time (essentially immediate).

The best place to put this code is in wxPython. Hopefully (I haven't checked this), wx exposes enough of the event loop to allow you to have it watch stdin. This may be an issue, since for example qt4 does not on Windows, which is why the event loop is kludgy with PyQT on Windows. You could have a look at the PyOS_InputHook code in PyGTK (you'll need to get the developer's version of PyGTK, since this code is not yet in an official release). It's actually quite straightforward and you may be able to modify it directly for wx.

My current cython based implementation (attached to this post) doesn't do what Michiel recommends, but it is a step in the right direction.

Is there interest amongst the wxPython devs in exploring this issue with me?

Cheers,

Brian

[Carl Karsten]

I don't know if this is relevant or not, if it is, you don't want to
miss it, and if it isn't, you will probably like watching it anyway:

http://blip.tv/file/2232410

David Beazley: mind-blowing presentation about how the Python GIL
actually works and why it's even worse than most people even imagine.

--
Carl K

[Robin Dunn]

Brian Granger wrote:

> 2. This code should really be in wxPython in the first place.

I agree.

> If we
> can get the code figured out and in wxPython, all wxPython users could
> use/dev wxPython apps interactively.

Thanks for brining this to my attention, I was not aware of
PyOS_InputHook before. Is there a minimum version of Python required or
has it been there all along?

I'll dig into the code and the mail messages about this later this
evening, and see if any good ideas pop out.

--
Robin Dunn
Software Craftsman
http://wxPython.org

[Robin Dunn]

Thanks. I had seen the slides already but hearing the Beaz give the
talk is much better.

[Brian Granger]

Robin,

> 2.  This code should really be in wxPython in the first place.

I agree.

Cool
 

> If we
> can get the code figured out and in wxPython, all wxPython users could
> use/dev wxPython apps interactively.

Thanks for brining this to my attention, I was not aware of
PyOS_InputHook before.  Is there a minimum version of Python required or
has it been there all along?

I am not sure, but I do know that this is how tk works, so I am guessing that it has been there all along.
 

I'll dig into the code and the mail messages about this later this
evening, and see if any good ideas pop out.

Great.  I did find it very useful to look at how this is implemented in PyQt, pygtk and tk.
 
Cheers,

Brian

[Robin Dunn]

Brian Granger wrote:

> I'll dig into the code and the mail messages about this later this
> evening, and see if any good ideas pop out.
>
>
> Great. I did find it very useful to look at how this is implemented in
> PyQt, pygtk and tk.
>

Hi Brian,

Attached is a patch implementing a first pass of this. It essentially
takes the same approach you did but integrates it into wx.App, with the
actual hook function being called as an overridable member of the wx.App
class. There is also a wx.IApp class that turns it on automatically.
Using wx.EventLoop means that wx.Yield is not needed. The patch was
made against the trunk (2.9) version of my SVN workspace, but I don't
think there is anything in there that is 2.9 specific so it should apply
to a 2.8.10.1 source tree too with a little help. I've only tested so
far on OS X, but don't anticipate any problems on the other platforms.

I haven't yet tried doing something like what you quoted from Michiel de
Hoon but the critical part is in Python so if you're able to build your
own copy with the patch then you can experiment by overriding
wx.IApp.OnInputHook.

Things that have their own nested event loop, like modal dialogs and
popup menus, will still end up blocking the interactive interpreter, but
everything else should be fine. (And in 2.9 we may be able to work
around that...)

I've also attached a screenshot of wx.IApp in action. Notice that there
is no call to MainLoop yet the events are being dispatched as expected.

Let me know what you think.

[Brian Granger]

Robin,

Fantastic.  Thanks for working on this.  See my comments inline.

 

Attached is a patch implementing a first pass of this.  It essentially
takes the same approach you did but integrates it into wx.App, with the
actual hook function being called as an overridable member of the wx.App
class.  There is also a wx.IApp class that turns it on automatically.
Using wx.EventLoop means that wx.Yield is not needed.  The patch was
made against the trunk (2.9) version of my SVN workspace, but I don't
think there is anything in there that is 2.9 specific so it should apply
to a 2.8.10.1 source tree too with a little help.  I've only tested so
far on OS X, but don't anticipate any problems on the other platforms.

One thing that I want to make sure of is that people don't have to modify their wx code for it to be used interactively.  I am hoping that we (in IPython) will simply be able to call some wx function that set's the proper hook for interactive work and that users can then just import and use their wx.App's as usual.  For this type of usage case, do you think it makes sense to put this code in wx.App/wx.IApp?
 
The other thing is that during the transition (i.e., before "everyone" is using a version of wx that has this feature) we will probably want to ship a standalone version with IPython.

I haven't yet tried doing something like what you quoted from Michiel de
Hoon but the critical part is in Python so if you're able to build your
own copy with the patch then you can experiment by overriding
wx.IApp.OnInputHook.

Yes, I will begin to play with it.  I am very curious to see how it works.  One thing that I found in my prototype is that simple things worked as expected, but that when I actually tried something more complex with matplotlib, it froze up and I had problems.  I will definitely test this out to see if those problems persist.
 

Things that have their own nested event loop, like modal dialogs and
popup menus, will still end up blocking the interactive interpreter, but
everything else should be fine.  (And in 2.9 we may be able to work
around that...)

Ok.
 

I've also attached a screenshot of wx.IApp in action.  Notice that there
is no call to MainLoop yet the events are being dispatched as expected.

Let me know what you think.

I will definitely start to play with this tomorrow.

Thanks so much!

Cheers,

Brian
 

--

Robin Dunn
Software Craftsman
http://wxPython.org

Index: include/wx/wxPython/wxPython_int.h
===================================================================
--- include/wx/wxPython/wxPython_int.h  (revision 61109)
+++ include/wx/wxPython/wxPython_int.h  (working copy)
@@ -629,6 +629,9 @@
 //---------------------------------------------------------------------------
 // The wxPythonApp class

+typedef int (*PyOS_InputHook_t)(void);
+
+
 enum {
    wxPYAPP_ASSERT_SUPPRESS  = 1,
    wxPYAPP_ASSERT_EXCEPTION = 2,
@@ -667,6 +670,11 @@
    virtual void ExitMainLoop();
    virtual int FilterEvent(wxEvent& event);

+    // allow interactions with the Python Input Hook
+    void SetInputHook();
+    void ResetInputHook();
+    virtual int OnInputHook();
+
    // For catching Apple Events
    virtual void MacOpenFile(const wxString& fileName);
    virtual void MacOpenURL(const wxString& url);
@@ -696,6 +704,7 @@
    int m_assertMode;
    bool m_startupComplete;
    bool m_callFilterEvent;
+    PyOS_InputHook_t m_oldInputHook;
 };

 extern wxPyApp *wxPythonApp;
Index: src/_app.i
===================================================================
--- src/_app.i  (revision 61109)
+++ src/_app.i  (working copy)
@@ -390,7 +390,25 @@
 //                              wxEvent& event) const;


+
+    DocDeclStr(
+        void , SetInputHook(),
+        "Set Python's input hook to call our OnInputHook method.", "");
+
+    DocDeclStr(
+        void , ResetInputHook(),
+        "Reset Python's input hook to what it was before `SetInputHook` was
+called.", "");
+
+    DocDeclStr(
+        virtual int , OnInputHook(),
+        "After `SetInputHook` has been called Python will periodically call
+this method while it is waiting for input at the interactive
+interpreter's prompt.  By default it will do nothing, but it can be
+overridden in a derived class.", "");
+

+

 // #ifdef __WXMAC__
 //     void MacRequestUserAttention(wxNotificationOptions);
Index: src/_app_ex.py
===================================================================
--- src/_app_ex.py      (revision 61109)
+++ src/_app_ex.py      (working copy)
@@ -286,6 +286,38 @@



+class IApp(wx.App):
+    """
+    This application class can be used from Python in interactive
+    interpreter mode (such as when running plain Python or IPython
+    from a console or terminal window without a GUI) and doesn't
+    require that MainLoop() be called in order for events to be
+    dispatched.
+    """
+    def __init__(self, *args, **kw):
+        wx.App.__init__(self, *args, **kw)
+        self.SetInputHook()
+
+    def __del__(self):
+        self.ResetInputHook()
+
+    def OnInputHook(self):
+        assert wx.Thread_IsMain()
+
+        # Make a temporary event loop and process system events until
+        # there are no more waiting, then allow idle events (which
+        # will also deal with pending or posted wx events.)
+        evtloop = wx.EventLoop()
+        ea = wx.EventLoopActivator(evtloop)
+        while evtloop.Pending():
+            evtloop.Dispatch()
+        self.ProcessIdle()
+        del ea
+
+
+
+
+
 # Is anybody using this one?
 class PyWidgetTester(wx.App):
    def __init__(self, size = (250, 100)):
Index: src/helpers.cpp
===================================================================
--- src/helpers.cpp     (revision 61291)
+++ src/helpers.cpp     (working copy)
@@ -318,6 +318,44 @@
 }


+// The Python Input Hook is a function pointer that is called periodically
+// when Python is in interactive interpreter mode.  These methods enable a method
+// of a wx.App derived class to be called from the hook, which hopfully will
+// enable us to experiment with various approaches to enabling using wx from
+// an terminal based interpreter like IPython without needing to block on the
+// wx MainLoop.
+extern "C"
+static int wxPyInputHookHelper(void)
+{
+    wxCHECK_MSG(wxPythonApp != NULL, 0, wxT("no wxPyApp yet!"));
+    return wxPythonApp->OnInputHook();
+}
+
+void wxPyApp::SetInputHook()
+{
+    m_oldInputHook = PyOS_InputHook;
+    PyOS_InputHook = wxPyInputHookHelper;
+}
+
+void wxPyApp::ResetInputHook()
+{
+    PyOS_InputHook = m_oldInputHook;
+    m_oldInputHook = NULL;
+}
+
+int wxPyApp::OnInputHook()
+{
+    int rval=0;
+    wxPyBlock_t blocked = wxPyBeginBlockThreads();
+    if (wxPyCBH_findCallback(m_myInst, "OnInputHook"))
+        rval = wxPyCBH_callCallback(m_myInst, Py_BuildValue("()"));
+    wxPyEndBlockThreads(blocked);
+    return rval;
+}
+
+
+
+// Turn wx assertions into Python exceptions
 void wxPyApp::OnAssertFailure(const wxChar *file,
                              int line,
                              const wxChar *func,

[Brian Granger]

Robin,

I am working on getting wx built with this inputhook patch applied.  I have tried both the 2.9 trunk and the 2.8 svn repo.  Problems with both:

2.9 Trunk
=======

Everything builds fine, but make install gives (after a while):

..........
cp: ../include/wx/osx/carbon/databrow.h: No such file or directory
 
 ------------------------------------------------------
 
 The installation of wxWidgets is finished.  On certain
 platforms (e.g. Linux) you'll now have to run ldconfig
 if you installed a shared library and also modify the
 LD_LIBRARY_PATH (or equivalent) environment variable.
 
 wxWidgets comes with no guarantees and doesn't claim
 to be suitable for any purpose.
 
 Read the wxWidgets Licence on licencing conditions.
 
 ------------------------------------------------------

This almost looks, OK, but there is the missing file error that stopped the install process.  Thoughts.

2.8 SVN
======

wxWidgets builds find, and I have built wxPython inplace with the inputhook patch applied and a patched SWIG 1.3.29.

I can do import wx, but app = wx.IApp() or app = wx.App() open up a stdout window and then freeze.  Not sure what is going on there.

Thanks,

Brian

[Robin Dunn]

Brian Granger wrote:
> Robin,
>
> Fantastic. Thanks for working on this. See my comments inline.
>
>
>
> Attached is a patch implementing a first pass of this. It essentially
> takes the same approach you did but integrates it into wx.App, with the
> actual hook function being called as an overridable member of the wx.App
> class. There is also a wx.IApp class that turns it on automatically.
> Using wx.EventLoop means that wx.Yield is not needed. The patch was
> made against the trunk (2.9) version of my SVN workspace, but I don't
> think there is anything in there that is 2.9 specific so it should apply
> to a 2.8.10.1 source tree too with a little help. I've only tested so
> far on OS X, but don't anticipate any problems on the other platforms.
>
>
> One thing that I want to make sure of is that people don't have to
> modify their wx code for it to be used interactively. I am hoping that
> we (in IPython) will simply be able to call some wx function that set's
> the proper hook for interactive work and that users can then just import
> and use their wx.App's as usual. For this type of usage case, do you
> think it makes sense to put this code in wx.App/wx.IApp?

I can understand the desire to have it function this way, but it just
won't work. The way things are currently architected there can only
ever be one wx.App in a process. So even if the input hook
functionality were pulled out of wx.App there would be problems the 2nd
time somebody tried to run a module that creates its own wx.App, even if
the first one has terminated and gc'd the first app object. (Not to
mention the fact that the code in question is probably also calling
MainLoop and would block there.)

Probably the best thing is to encourage the users to write their code
such that the app is created and MainLoop called only inside of a "if
__main__ ..." block, and then if they want to play with it interactively
then they can create a wx.IApp and then import the module and create an
instance of their main frame or whatever it is that they want to test.
(This is how I often do things in PyCrust, where I can essentially reuse
the already instantiated wx.App that PyCrust created for itself.)

[Robin Dunn]

Brian Granger wrote:
> Robin,
>
> I am working on getting wx built with this inputhook patch applied. I
> have tried both the 2.9 trunk and the 2.8 svn repo. Problems with both:

I probably should have mentioned that the current wxPython trunk does
not correspond with the current wxWidgets trunk, but rather with the
2.9.0 branch. (There are various unimportant reasons for this, but this
discrepancy should not last much longer so 'why' doesn't matter too
much...) So you will want to get the wxWidgets part of your source tree
from here:

http://svn.wxwidgets.org/svn/wx/wxWidgets/branches/WX_2_9_0_BRANCH/

> 2.8 SVN
> ======
>
> wxWidgets builds find, and I have built wxPython inplace with the
> inputhook patch applied and a patched SWIG 1.3.29.
>
> I can do import wx, but app = wx.IApp() or app = wx.App() open up a
> stdout window and then freeze. Not sure what is going on there.

Ok, I'll try to take a look at this on 2.8 in the next day or so.

[Brian Granger]

On Sat, Jul 4, 2009 at 2:08 PM, Robin Dunn <ro...@alldunn.com> wrote:

Brian Granger wrote:
> Robin,
>
> I am working on getting wx built with this inputhook patch applied.  I
> have tried both the 2.9 trunk and the 2.8 svn repo.  Problems with both:

I probably should have mentioned that the current wxPython trunk does
not correspond with the current wxWidgets trunk, but rather with the
2.9.0 branch.  (There are various unimportant reasons for this, but this
discrepancy should not last much longer so 'why' doesn't matter too
much...)  So you will want to get the wxWidgets part of your source tree
from here:

http://svn.wxwidgets.org/svn/wx/wxWidgets/branches/WX_2_9_0_BRANCH/

Ahhh, that would explain it.  I will give this a shot and see how it goes.  Thanks. 
 

> 2.8 SVN
> ======
>
> wxWidgets builds find, and I have built wxPython inplace with the
> inputhook patch applied and a patched SWIG 1.3.29.
>
> I can do import wx, but app = wx.IApp() or app = wx.App() open up a
> stdout window and then freeze.  Not sure what is going on there.

Ok, I'll try to take a look at this on 2.8 in the next day or so.

Thanks.

Brian
 

[Brian Granger]

Robin,

I was on vacation until yesterday.  I am working on testing out the PyOX_InputHook patch that you put together.  I have now successfully built the 2.9 trunk of wxPython/wxWidgets with your patch applied.

I have tested it with your basic example as well as some matplotlib examples.  Here is what I have found:

* Overall, it works very well.  Both your basic example as well as many of the matplotlib examples work just fine.

* With the example you gave, the setting of the background color doesn't work.  The background stays grey.  But, when text is written to the panel, it has a pink background (the text only).

* One of the most important things that we need to work it interrupt handling (ctrl-C).  Currently, when I am using an IApp instance, if I do:

import time; time.sleep(10)

And then hit ctrl-C, python/ipython crash.  Any thoughts as to why this would happen?

* Michiel de Hoon (who implemented this same logic for pyGTK and PyObjC) claims that there is a better (performane wise) way of structuring the actual inputhook (IApp.OnInputHook).  He claims that this function should 1) start the main event loop, 2) watch stdin until there is something, 3) quit the event loop and return.  I don't really understand this approach, but this is how PyGTK, PyQT and PyObjC all handle it.  I would like to give it a try to see if there is a performance difference.  Question:  how can I have wx watch stdin and trigger an action when something is available on it?

Have you tried any of this on 2.8 yet?

Cheers and thanks for you help.

Brian

[Brian Granger]

Robin,

* One of the most important things that we need to work it interrupt handling (ctrl-C).  Currently, when I am using an IApp instance, if I do:

import time; time.sleep(10)

And then hit ctrl-C, python/ipython crash.  Any thoughts as to why this would happen?

I figured this out:

app = wx.IApp(clearSigInt=False)

This seems to fix the problem, unless you see this solution isn't optimal in some way.

Cheers,

Brian

[Robin Dunn]

Brian Granger wrote:
> Robin,
>
> I was on vacation until yesterday. I am working on testing out the
> PyOX_InputHook patch that you put together. I have now successfully
> built the 2.9 trunk of wxPython/wxWidgets with your patch applied.
>
> I have tested it with your basic example as well as some matplotlib
> examples. Here is what I have found:
>
> * Overall, it works very well. Both your basic example as well as many
> of the matplotlib examples work just fine.
>
> * With the example you gave, the setting of the background color doesn't
> work. The background stays grey. But, when text is written to the
> panel, it has a pink background (the text only).

It probably just needs a Refresh() call so the panel will be repainted
after the color change.

>
> * Michiel de Hoon (who implemented this same logic for pyGTK and PyObjC)
> claims that there is a better (performane wise) way of structuring the
> actual inputhook (IApp.OnInputHook). He claims that this function
> should 1) start the main event loop, 2) watch stdin until there is
> something, 3) quit the event loop and return. I don't really understand
> this approach, but this is how PyGTK, PyQT and PyObjC all handle it. I
> would like to give it a try to see if there is a performance
> difference. Question: how can I have wx watch stdin and trigger an
> action when something is available on it?

wx doesn't have anything built-in yet for watching for file handle
activity, although there is a GSoC project this summer that is adding
it. Assuming it is completed and merged into the code it will likely be
in some 2.9.x release. However it will likely be such that it sends wx
events when the file handle is read-ready, so I'm not sure that will
make sense to integrate directly into an event loop. With the current
code you could probably do it by adding an outer loop, like the
following, although it will use more CPU because of the polling nature
or the loop:

while not self.CheckForInput():
while evtloop.Pending():
evtloop.Dispatch()
self.ProcessIdle()

In the CheckForInput method on unix-like systems it could use the select
module to check if there is any input available for stdin. On Windows
it would probably have to use a Win32 API as select only works for
sockets IIRC.

>
> Have you tried any of this on 2.8 yet?

Not yet.

[Robin Dunn]

I don't remember details, but this may only work on wxGTK. The various
toolkits mess with the system signals in different ways...

It will take some experimentation and maybe some black magic (platform
specific API calls) to figure out what to do for this on each of the
platforms so Ctrl-C always generates a KeyboardInterrupt exception. But
once that is figured out we can make IApp do it by default.

[Brian Granger]

> * With the example you gave, the setting of the background color doesn't
> work.  The background stays grey.  But, when text is written to the
> panel, it has a pink background (the text only).

It probably just needs a Refresh() call so the panel will be repainted
after the color change.

Yep, that did it.  In general, when is a Refresh needed?
 

>
> * Michiel de Hoon (who implemented this same logic for pyGTK and PyObjC)
> claims that there is a better (performane wise) way of structuring the
> actual inputhook (IApp.OnInputHook).  He claims that this function
> should 1) start the main event loop, 2) watch stdin until there is
> something, 3) quit the event loop and return.  I don't really understand
> this approach, but this is how PyGTK, PyQT and PyObjC all handle it.  I
> would like to give it a try to see if there is a performance
> difference.  Question:  how can I have wx watch stdin and trigger an
> action when something is available on it?

wx doesn't have anything built-in yet for watching for file handle
activity, although there is a GSoC project this summer that is adding
it.  Assuming it is completed and merged into the code it will likely be
in some 2.9.x release.  However it will likely be such that it sends wx
events when the file handle is read-ready, so I'm not sure that will
make sense to integrate directly into an event loop.  

I guess we will have to wait on this approach then.  I think it will be fine that it sends wx events. 
 

With the current
code you could probably do it by adding an outer loop, like the
following, although it will use more CPU because of the polling nature
or the loop:

    while not self.CheckForInput():

        while evtloop.Pending():
            evtloop.Dispatch()
        self.ProcessIdle()

I will try to look at something like this.  The fact that this is polling might not be that bad.
 

In the CheckForInput method on unix-like systems it could use the select
 module to check if there is any input available for stdin.  On Windows
it would probably have to use a Win32 API as select only works for
sockets IIRC.

Yes.

In the meantime, I am going to begin to discuss your patch with some of the projects that use these interactive capabilities in IPython (matplotlib and enthought mainly).  Are you open to committing your patch to the 2.9 trunk so that it is easier for people to try this out?

Cheers,

Brian

[Brian Granger]

Robin,

I am attaching a ctypes based version of your patch.  I am creating this to enable a wider range of people to test this out without having to rebuild a patched version of wx.  Also, we will ship this ctypes based version with IPython until everyone is using a version of wx that has this capability.  This will allow a smoother transition.

I did re-try this ctypes version on wx 2.8 and it has the same problem as before.  When you get a chance, it would be great if you could help figure out what is going on with 2.8.

Thanks again!

Cheers,

Brian

[Brian Granger]

Robin,

I did re-try this ctypes version on wx 2.8 and it has the same problem as before.  When you get a chance, it would be great if you could help figure out what is going on with 2.8.

After digging around in the wx.App code, I found that one of the default arguments was chagned in 2.9.  The argument is redirect.  It was True in 2.8, and in 2.9 it is False.  Setting it to False makes everything work fine on 2.8.  Just a warning, the wx.App doc strings in 2.9 have not been updated to reflect this change.

I will keep playing around with this and keep you posted.

Cheers,

Brian
 

[Brian Granger]

Robin,

People from various projects have been testing the PyOS_InputHook stuff in various settings.  Overall, the results have been good.  But, some users are reporting slow GUI responses, which this new stuff is used interactively.  Here is the current hypothesis about what the performance issue is related to:

The chunkiness probably comes from the fact that inputhook_wx is called repeatedly. This is different from how PyOS_InputHook is being used in Tkinter, PyGTK, and the Mac OS X backend.

Schematically, this is how the Tkinter/PyGTK/MacOSX event loops work:

1) PyOS_InputHook is called when Python is waiting for the user to type in the next Python command.

2) The hook function sets up the event loop such that stdin is being monitored while the event loop is running.

3) The hook function then starts the event loop.

4) When input is available on stdin, the hook function exits the event loop, and returns.

This is how the proposed Wx event loop currently works:

1) PyOS_InputHook is called when Python is waiting for the user to type in the next Python command.

2) The hook function processes whatever events are available at the time.

3) The hook function returns.

4) If still no input is available on stdin, Python calls the hook function again via PyOS_InputHook after a timeout.

I believe the timeout is 0.1 seconds by default. However, Python may not call PyOS_InputHook repeatedly at all; this depends on which Python version is being used, and the version of the readline library. In some configurations (particularly on Windows), PyOS_InputHook is called only once, so wx will freeze between Python commands.

Do you have any ideas on how these issues can be addressed.  Obviously, being able to monitor stdin in the event loop would help, but it sounds like that isn't possible yet.

Thanks for you help.

Cheers,

Brian

[Robin Dunn]

Brian Granger wrote:
>
>
> > * With the example you gave, the setting of the background color
> doesn't
> > work. The background stays grey. But, when text is written to the
> > panel, it has a pink background (the text only).
>
> It probably just needs a Refresh() call so the panel will be repainted
> after the color change.
>
>
> Yep, that did it. In general, when is a Refresh needed?

Whenever a visual change is made to a widget and there isn't a "natural'
refresh pending. In the interactive commands I showed in the screenshot
I wrote a comment "# Manually move and resize the window..." That would
have caused a natural refresh as it resized the panel to fit the new
size of the frame.

[Robin Dunn]

Brian Granger wrote:
> Robin,
>
>
> I did re-try this ctypes version on wx 2.8 and it has the same
> problem as before. When you get a chance, it would be great if you
> could help figure out what is going on with 2.8.
>
>
> After digging around in the wx.App code, I found that one of the default
> arguments was chagned in 2.9. The argument is redirect. It was True in
> 2.8, and in 2.9 it is False. Setting it to False makes everything work
> fine on 2.8.

Ah, I hadn't thought of the stdout redirect since I almost always turn
it off myself. I'm glad you found this before I wasted time chasing my
tail in some other direction. You can add a call to app.RestoreStdio to
your inputhook script to make sure that the stdout is where it needs to be.

> Just a warning, the wx.App doc strings in 2.9 have not
> been updated to reflect this change.

Thanks.

[Robin Dunn]

Brian Granger wrote:
> Robin,
>
> People from various projects have been testing the PyOS_InputHook stuff
> in various settings. Overall, the results have been good. But, some
> users are reporting slow GUI responses, which this new stuff is used
> interactively. Here is the current hypothesis about what the
> performance issue is related to:
>
> The chunkiness probably comes from the fact that inputhook_wx is called
> repeatedly.

And the frequency of those calls will play a big role in the
responsiveness of the GUI. IOW, events will be processed (in short
bursts) only as often as the input hook is called.

[...]

> Do you have any ideas on how these issues can be addressed. Obviously,
> being able to monitor stdin in the event loop would help, but it sounds
> like that isn't possible yet.

Something that can be done today with 2.8 is to add the outer loop I
mentioned in a previous message, controlled by a function that is
polling the stdin for input-readiness. Then it will stay in the event
loop until there is input available for the interactive interpreter and
won't depend on the hook being called more than once and/or often.