wxWidgets help needed to show console output

1,500 views
Skip to first unread message

kfj

unread,
Jun 21, 2011, 4:01:33 AM6/21/11
to hugin and other free panoramic software
I put this into a new thread, because the previous location is a bit
obscure. The discussion is about showing a python plugin's (or any
other) console output in a wxWidgets window, which I think I have
worked out but I need a bit of wxWidgets help.

On 18 Jun., 04:55, Yuval Levy <goo...@levy.ch> wrote:
> On June 17, 2011 06:06:20 AM kfj wrote:
>
> > On 15 Jun., 14:25, Yuval Levy <goo...@levy.ch> wrote:
> > > What I still would like to see (don't know if it is possible/feasible in
> > > short time as the release is more important) is a pop up window like the
> > > one of hugin-stitch_project that displays the console output in the GUI.
>
> > I looked into the matter a bit yesterday.
>
> Thank you!
>
> > The problem was just that python's print statements seem to go
> > directly to the underlying stdio handles (stdin, stdout and stderr)
> > instead of C++'s cin, cout, cerr - so redirecting the output worked
> > only for any operator<< calls within C++. Maybe python can be coerced
> > to do things differently and use cin etc instead.

I've found an elegant, simple solution the the problem, but I need a
bit of help on the wxWidgets side to make it work nicely. This is what
I do - with a bit of background to show how and why it works:

1) wxWidgets has a class wxStreamToTextRedirector. This class can
redirect C++ stream output to a wxText widget. This works just fine,
but needs more effort for python's output, as python does not output
to C++ cout or cerr, but to it's notion of standard output, a python
object called stdout which can be accessed via the sys module.

2) python's sys.stdout is not some god-given immutable thing, but it
can be assigned any object that has a write() method. All python
output to stdout - so, that's print statements etc. - goes to that
object's write method. Easy to redirect python stdout.

3) hsi has already wrapped std::ios, so cout and cerr are available to
python

4) to put it all together. In python, I create a class echo (I do this
in hsi.py, so it's done automatically for every plugin call):

import hsi
import sys

class echo :

def write ( self , s ) :
hsi.cout.write ( s , len ( s ) )
hsi.cout.flush()

Then I assign an instance of class echo to python's sys.stdout:

sys.stdout = echo()

now all python output (and, which is an added bonus, all output from
any subprocesses python launches) goes to hugin's C++ cout and can be
redirected with a wxStreamToTextRedirector.

5) integrate the wxStreamToTextRedirector in hugin. This is where I
need help. I have managed to code enough to demonstrate that the
rerouting actually works, but my knowledge of wxWidgets is
insufficient. To demonstrate the method, I'm creating the redirector
in wxPanoCommand.cpp, in processPanorama():

bool PythonScriptPanoCmd::processPanorama(Panorama& pano)
{
wxTextCtrl *text = new wxTextCtrl(MainFrame::Get(),
-1,
wxString(),
wxPoint(0,0),
wxSize(600,600),
wxTE_MULTILINE | wxTE_READONLY
);
wxStreamToTextRedirector redirect(text);

// the redirect is now in effect.
// ... stuff happens, all cout output is redirected to 'text'

return true;
}

Obviously tis is amateurishly done. What happens is that the text
window only shows after the plugin has terminated - it does have all
the text inside that it should have, though. I suspect that the
wxTextCtrl has to be put inside it's own Window, which has to be shown
explicitly before the plugin starts and closed once the plugin is
done. Apart from that I think the problem is solved. Help with the
last step would be appreciated.

Kay


kfj

unread,
Jun 21, 2011, 4:06:11 AM6/21/11
to hugin and other free panoramic software


On 21 Jun., 10:01, kfj <_...@yahoo.com> wrote:

> 4) to put it all together. In python, I create a class echo (I do this
> in hsi.py, so it's done automatically for every plugin call):

I correct - I put it into hpi.py.

Kay

kfj

unread,
Jun 21, 2011, 1:37:00 PM6/21/11
to hugin and other free panoramic software
I played with this new mechanism for a bit. My initial statement that
'... as an added bonus, all output from any subprocesses python
launches goes to hugin's C++ cout' was wrong, but it's easy to capture
the subprocess' output in python and submit it to the same mechanism
of 'forwarding' to the C++ streams which then feed the wxText control.
I've refined the technique a bit. In hpi.py it's now:

**********************************************

import sys
import hsi

# class tee captures write operations to it's 'victim' and sends them
# to 'copy', which must have a write method taking the length of the
# string as second argument (so we can use hsi.cout and hsi.cerr)
# and a flush method (maybe that's not needed)

class tee :

# we take the stream to be diverted as our 'victim'

def __init__ ( self , victim , copy ) :
self.victim = victim
self.copy = copy

# if write is called, we write to 'copy' first
# and to 'victim' next

def write ( self , s ) :
self.copy.write ( s , len ( s ) )
self.copy.flush()
self.victim.write ( s )

# all other activities are delegated to 'victim'

def __getattr__ ( self , name ) :

return getattr ( self.victim , name )

# finally, we use two tee objects, one for cout and one for cerr.

sys.stdout = tee ( sys.stdout , hsi.cout )
sys.stderr = tee ( sys.stderr , hsi.cerr )

************************************************

on the C++ side, my clumsy attempt reads like this:

***********************************************

bool PythonScriptPanoCmd::processPanorama(Panorama& pano)
{
wxDialog pd (NULL, -1, _("plugin console"),
wxDefaultPosition, wxSize(650,650),
wxDEFAULT_DIALOG_STYLE,
_("console") ) ;
wxTextCtrl *text = new wxTextCtrl(&pd,
-1,
wxString(),
wxPoint(0,0),
wxSize(600,600),
wxTE_MULTILINE | wxTE_READONLY
);
wxStreamToTextRedirector redirect(text) ;
pd.Show(1) ;

// now the plugin interface is called
// ...

***********************************************

What's missing currently is showing the output as it's generated - I
can only see it once the plugin is terminated, even though I've now
put the wxTextCtrl into a dialog and call Show() on it right away. It
just doesn't show when I call Show() and only shows at all if the
plugin doesn't return zero and the 'plugin returned X' dialog box is
shown. So I still need some wxWindows help. What am I missing?

I suspect that a usable GUI element for this purpose could be derived
from MyExecPanel. Maybe it's just not possible to see the data as they
are produced unless the plugin happens in a separate thread? If the
plugin is quick, it doesn't matter, but lengthy stuff like woa should
display, so the progress can be seen.

Kay

T. Modes

unread,
Jun 21, 2011, 2:11:43 PM6/21/11
to hugin and other free panoramic software

> Apart from that I think the problem is solved.

That's too early. It works on your machine. But there was no test on
other systems than yours.
To implement it in a correct way there are more changes necessary.
E.g. the calling of the python scripts needs to moved into a separate
thread. Otherwise you won't get progress messages while running a
script. This requires some more changes to make the code thread safe.

Thomas

PS: Please read your mails. (You are expecting that other read your
mails, but don't read yours). There is a bug in the Python interface
which needs to be fixed:
https://bugs.launchpad.net/hugin/+bug/799905

kfj

unread,
Jun 21, 2011, 5:02:21 PM6/21/11
to hugin and other free panoramic software


On 21 Jun., 20:11, "T. Modes" <Thomas.Mo...@gmx.de> wrote:
> > Apart from that I think the problem is solved.
>
> That's too early. It works on your machine. But there was no test on
> other systems than yours.

Okay, point taken. But what makes me hopeful is that the python side
is totally standard - all output is simply written to std::cout or
srd::cerr. If the diversion in wxWidgets with wxStreamToTextRedirector
isn't some new feature which isn't available everywhere it might be a
general solution. The documentation seems to indicate that there are
system dependencies:

NB: Some compilers and/or build configurations don't support multiply
inheriting wxTextCtrl from std::streambuf in which case this class is
not compiled in.

If there is a problem with this approach, I can still direct all
comunication through a common channel. It doesn't have to be cout/cerr
- we might just as easily pass in any other object that can accept a
write call, simplest would be another ostream. That's if the
wxStreamToTextRedirector isn't generally available or problematic.

> To implement it in a correct way there are more changes necessary.
> E.g. the calling of the python scripts needs to moved into a separate
> thread. Otherwise you won't get progress messages while running a
> script. This requires some more changes to make the code thread safe.

I wonder if it might not be easier the other way round: put the echo
window into a separate thread. It has only input and needs to see none
of hugin's state, might be easier to make it threadsafe. Just
guessing.

> PS: Please read your mails. (You are expecting that other read your
> mails, but don't read yours). There is a bug in the Python interface
> which needs to be fixed:https://bugs.launchpad.net/hugin/+bug/799905

Thank you for telling me. The mail landed in my spam folder, don't
know why. Normally they just come into the inbox. I'm innocent. I'll
deal with it as soon as I can.

Kay

T. Modes

unread,
Jun 22, 2011, 2:41:08 PM6/22/11
to hugin and other free panoramic software
> Okay, point taken. But what makes me hopeful is that the python side
> is totally standard - all output is simply written to std::cout or
> srd::cerr. If the diversion in wxWidgets with wxStreamToTextRedirector
> isn't some new feature which isn't available everywhere it might be a
> general solution. The documentation seems to indicate that there are
> system dependencies:

On Windows: First approach is working.
Second approach is failing at first print statement. It prints the
line, but then it stops with error -1.
I removed the line self.victim.write ( s ), now it is working on
Windows.

> I wonder if it might not be easier the other way round: put the echo
> window into a separate thread. It has only input and needs to see none
> of hugin's state, might be easier to make it threadsafe. Just
> guessing.

No, that's not working. If you calling Python in the main thread, you
are blocking the whole program (because the message queue for the
application is processed in the main thread). Only the main thread
should access the GUI directly. All other threads must use a different
(more complicated) approach to get access to the gui.

I implemented a dialog to show the output of Python scripts. It's
working at Windows. I taken your modification to hpi.py as base, but
removed the not working line.

Thomas


kfj

unread,
Jun 22, 2011, 4:31:05 PM6/22/11
to hugin and other free panoramic software
On 22 Jun., 20:41, "T. Modes" <Thomas.Mo...@gmx.de> wrote:

> On Windows: First approach is working.
> Second approach is failing at first print statement. It prints the
> line, but then it stops with error -1.
> I removed the line self.victim.write ( s ), now it is working on
> Windows.

The self.victim.write was only there to still to do the console
output. That's why I called it tee, like the UNIX command - a T
junction. If you only write to copy, you needn't store victim, and you
need no access to it's attributes either, so I suppose the class
definition could be simply

class OutputToCStream :

def __init__ ( self , copy ) :

self.copy = copy

def write ( self , s ) :

self.copy.write ( s , len ( s ) )
self.copy.flush()

if you don't want to write anything to the original 'victim' - if I
remeber correctly, that doesn't work on Windows anyway.

> > I wonder if it might not be easier the other way round: put the echo
> > window into a separate thread. It has only input and needs to see none
> > of hugin's state, might be easier to make it threadsafe. Just
> > guessing.
>
> No, that's not working. If you calling Python in the main thread, you
> are blocking the whole program (because the message queue for the
> application is processed in the main thread). Only the main thread
> should access the GUI directly. All other threads must use a different
> (more complicated) approach to get access to the gui.

It was a quick shot. You're right - I read up on the wx threads. I
managed to come up with a version which would at least show the output
nicely after the plugin has terminated, but nothing to show it while
it's produced. So I'm curious to see your solution.

> I implemented a dialog to show the output of Python scripts. It's
> working at Windows. I taken your modification to hpi.py as base, but
> removed the not working line.

Great. I hope I'll still get round to check it out - I'm off again on
Saturday for a trip to the Alps. Bit of real world for a change, was
getting square eyes...

Kay

kfj

unread,
Jun 23, 2011, 8:51:57 AM6/23/11
to hugin and other free panoramic software
On 22 Jun., 20:41, "T. Modes" <Thomas.Mo...@gmx.de> wrote:

> I implemented a dialog to show the output of Python scripts. It's
> working at Windows. I taken your modification to hpi.py as base, but
> removed the not working line.

I'm not sure if it's because of your code, but on my system, no plugin
runs anymore with bleeding edge (Pre-Release 2011.3.0.fa65e6eaba77). I
get a window flashing up briefly which starts outputting something,
then hugin crashes. If I run hugin from the comand line, I get plenty
of errors:

(hugin:9457): Gtk-WARNING **: Invalid text buffer iterator: either the
iterator is uninitialized, or the characters/pixbufs/widgets in the
buffer have been modified since the iterator was created.
You must use marks, character numbers, or line numbers to preserve a
position across buffer modifications.
You can apply tags and insert marks without invalidating your
iterators,
but any mutation that affects 'indexable' buffer contents (contents
that can be referred to by character offset)
will invalidate all outstanding iterators

(hugin:9457): Pango-CRITICAL **: pango_layout_get_cursor_pos:
assertion `index >= 0 && index <= layout->length' failed

(hugin:9457): Pango-CRITICAL **: pango_layout_get_cursor_pos:
assertion `index >= 0 && index <= layout->length' failed

(hugin:9457): Gtk-CRITICAL **: gtk_text_layout_real_invalidate:
assertion `layout->wrap_loop_count == 0' failed

(hugin:9457): Gtk-CRITICAL **: gtk_text_layout_real_invalidate:
assertion `layout->wrap_loop_count == 0' failed

etc. etc., ending in

(hugin:9742): Pango-CRITICAL **: pango_layout_get_iter: assertion
`PANGO_IS_LAYOUT (layout)' failed
Speicherzugriffsfehler

so the last one is a memory fault. This is on Kubuntu 11.4 on a 32bit
intel system. Maybe I'm missing something?

Kay

T. Modes

unread,
Jun 23, 2011, 12:38:36 PM6/23/11
to hugin and other free panoramic software


On 23 Jun., 14:51, kfj <_...@yahoo.com> wrote:
> On 22 Jun., 20:41, "T. Modes" <Thomas.Mo...@gmx.de> wrote:
>
> > I implemented a dialog to show the output of Python scripts. It's
> > working at Windows. I taken your modification to hpi.py as base, but
> > removed the not working line.
>
> I'm not sure if it's because of your code, but on my system, no plugin
> runs anymore with bleeding edge (Pre-Release 2011.3.0.fa65e6eaba77). I
> get a window flashing up briefly which starts outputting something,
> then hugin crashes. If I run hugin from the comand line, I get plenty
> of errors:
>

I pushed some changes to default branch. It works for me at Windows
and at Fedora in a virtual machine.
Please test again.

Thomas

PS: I committed also the changes you proposed for hpi.py -> removed
not necessary lines.

kfj

unread,
Jun 23, 2011, 1:53:36 PM6/23/11
to hugin and other free panoramic software
On 23 Jun., 18:38, "T. Modes" <Thomas.Mo...@gmx.de> wrote:

> I pushed some changes to default branch. It works for me at Windows
> and at Fedora in a virtual machine.
> Please test again.

still no joy. And it only crashes with the diversion of python's
stdxxx active and dies precisely with the first print() in hpi.py. The
window shows, and as soon as the first print happens the whole thing
goes ka-boof:

kfj@Anja:~/src/hugin/hugin_clean.b$ hugin
MainFrame::RestoreLayoutOnNextResize()

(hugin:14350): Gtk-WARNING **: Invalid text buffer iterator: either
the iterator is uninitialized, or the characters/pixbufs/widgets in
the buffer have been modified since the iterator was created.
You must use marks, character numbers, or line numbers to preserve a
position across buffer modifications.
You can apply tags and insert marks without invalidating your
iterators,
but any mutation that affects 'indexable' buffer contents (contents
that can be referred to by character offset)
will invalidate all outstanding iterators

(hugin:14350): Pango-CRITICAL **: pango_layout_get_iter: assertion
`PANGO_IS_LAYOUT (layout)' failed
Speicherzugriffsfehler
kfj@Anja:~/src/hugin/hugin_clean.b$

where's the code that opens and writes to the window? Maybe I can spot
something.

Kay

T. Modes

unread,
Jun 23, 2011, 2:04:04 PM6/23/11
to hugin and other free panoramic software


On 23 Jun., 19:53, kfj <_...@yahoo.com> wrote:
> On 23 Jun., 18:38, "T. Modes" <Thomas.Mo...@gmx.de> wrote:
>
> > I pushed some changes to default branch. It works for me at Windows
> > and at Fedora in a virtual machine.
> > Please test again.
>
> still no joy. And it only crashes with the diversion of python's
> stdxxx active and dies precisely with the first print() in hpi.py. The
> window shows, and as soon as the first print happens the whole thing
> goes ka-boof:

It works for me on Fedora. So I have no idea why it works for me and
crashes at your machine.

> where's the code that opens and writes to the window? Maybe I can spot
> something.

In src/hugin1/hugin/PythonProgress.cpp. It's using
wxStreamToTextRedirector for redirecting the output.

Thomas

kfj

unread,
Jun 23, 2011, 2:22:14 PM6/23/11
to hugin and other free panoramic software


On 23 Jun., 20:04, "T. Modes" <Thomas.Mo...@gmx.de> wrote:

> It works for me on Fedora. So I have no idea why it works for me and
> crashes at your machine.
>
> > where's the code that opens and writes to the window? Maybe I can spot
> > something.
>
> In src/hugin1/hugin/PythonProgress.cpp. It's using
> wxStreamToTextRedirector for redirecting the output.

But it does look as if you were using GUI calls in the thread. I tried
the same and crashed. The wx docu says your mileage will vary with
this and recommends not to. Ubuntu isn't Fedora - I think ist' not
just my machine.

On my afternoon walk today I had an idea: Why are we so keen on using
a thread? Let's just use a separate process. Send hugins cout to the
process' stdin, let that process have a single modal window with a
text control, and when no more input arrives sleep a minute and die.
If the user clicks on it or scolls, let it live until explicitly
killed. Might be simpler.

Could even launch the process from hpi.py and only divert python's
output, not the C++ streams (they are of little use anyway, just
telling which file is used and what it returns, and hpi.py does that
as well) maybe wxPython code - doesn't have to be compiled - and as a
subprocess it wouldn't collide with anything in hugin, run on all
systems. And as wxWidgets is already in memory, it should load in next
to no time.

Kay

T. Modes

unread,
Jun 23, 2011, 2:42:58 PM6/23/11
to hugin and other free panoramic software
> But it does look as if you were using GUI calls in the thread. I tried
> the same and crashed. The wx docu says your mileage will vary with
> this and recommends not to.

Only the function PythonProgress::Entry is running as a separate
thread. And in this function there is no access to the GUI directly.
So I don't understand your statement.

Thomas

kfj

unread,
Jun 23, 2011, 3:58:11 PM6/23/11
to hugin and other free panoramic software
This is what the docu says:

GUI calls, such as those to a wxWindow or wxBitmap are explicitly not
safe at all in secondary threads and could end your application
prematurely. This is due to several reasons, including the underlying
native API and the fact that wxThread does not run a GUI event loop
similar to other APIs as MFC.

It makes no difference if you call the GUI function 'yourself' or
whether it's executed a few calls down by the
wxStreamToTextRedirector. the wx call is still from within the
separate thread.

If I uderstand your code correctly, the thread executes the plugin in
the entry function. the plugin echos to cout. cout writes into a
window. So in effect, the thread does execute GUI code, never mind it
doesn't call any wxFuncions directly, it's doing it indirectly and
that's the problem.

I think you were quite right pointing out to me initially that there
is no way the output could be seen unless it's in a different thread.
I hoped one might get away with it, still, and tried to code it the
other way round by having the display in a separate thread, and it
didn't work.

The only way I see how one could avoid the problem and use threads is
shoving the data in a buffer. Have the plugin in a separate thread (as
you do it) which does no direct or indirect GUI stuff and let it write
to a buffer. The displaying of the data can then happen in the main
thread, which is allowed to do GUI stuff. Just that the displaying
thread must pick up the data from the buffer and display it by itself.
Maybe you could do it with a pipe.

The problem with this approach, though, is when we want to start using
wxPython from plugins. If the plugin is in a separate thread and wants
to use the same wxWindows instance, we have the same problem all over:
a separate thread wanting to do GUI calls.

Turn it as you like, it looks like a separate process is simpler.

Kay
Reply all
Reply to author
Forward
0 new messages