BEBUG_WARN, cerr and Python

[kfj]

Hi all!

I have a problem with hugin outputting data to cerr when I use it from
Python, but it may point to a general problem. Hugin uses a set of
macros to produce warning and error output when in debug mode.
Recently I've experienced two occasions when these macros weren't made
dependent on the setting of the DEBUG define but just activated
anyway; currently this is the case in src/hugin_base/hugin_utils/
utils.h., line 87-94.This wouldn't be a great problem, if the output
to cerr didn't crash Python when it uses hugin or other programs
linking to libhuginbase.so the very moment the first string is out.
Assume the C++ code does something like

cerr << "WARN:" << "blah blah" << endl ;

All I get to see in Python is

WARN: Speicherzugriffsfehler

(that is, memory error.) - so I get to see the first string 'WARN:'
but then the crash happens, which looks like an error in
cerr.operator<<(). The strange thing is that this only occurs with
output to cerr, cout is unaffected. Does anyone have any ideas if
maybe something is done to cerr which is incompatible - like some evil
trickery to make it do things it's not supposed to? I've managed to
reduce the problem to the linking in of libhugin.so - If I run an
ordinary C++ program from Python as a subprocess, using cerr is just
fine, even mixed with stdio routines, but as soon as I link in
libhugin.so, the call to cerr shows the erroneous behaviour. For now,
I'll modify utils.h back to only do output when DEBUG is set. I'd be
grateful if, whoever unleashes the DEBUG_ macros outside DEBUG mode
would only do these changes locally. Any ideas about the cerr problem
would be welcome, though, because there definitely is a problem, and
just avoiding it isn't a good strategy.

Kay

[Pablo d'Angelo]

Hi Kay,

Am 23.01.2011 09:32, schrieb kfj:

> cerr<< "WARN:"<< "blah blah"<< endl ;
>
> All I get to see in Python is
>
> WARN: Speicherzugriffsfehler
>
> (that is, memory error.) - so I get to see the first string 'WARN:'
> but then the crash happens, which looks like an error in
> cerr.operator<<(). The strange thing is that this only occurs with
> output to cerr, cout is unaffected. Does anyone have any ideas if
> maybe something is done to cerr which is incompatible - like some evil
> trickery to make it do things it's not supposed to?

Hmm, not sure about that. Maybe some problems with static initialization
and linking order? Does the problem also occur for a small dummy
extension (thus running in the same process as python) using c++, or
does it only occur from within hugin?

ciao
Pablo

[kfj]

On 23 Jan., 11:09, Pablo d'Angelo <pablo.dang...@web.de> wrote:

> Hmm, not sure about that. Maybe some problems with static initialization
> and linking order? Does the problem also occur for a small dummy
> extension (thus running in the same process as python) using c++, or
> does it only occur from within hugin?

It doesn't even have to be an extension. It's enough if the program
uses libhuginbase.so and if it's started by python as a subprocess. If
that helps, I can put a demo online.

Kay

[kfj]

On 23 Jan., 11:19, kfj <_...@yahoo.com> wrote:

> It doesn't even have to be an extension. It's enough if the program
> uses libhuginbase.so and if it's started by python as a subprocess. If
> that helps, I can put a demo online.

hmmm... tricky. Currently I can't reproduce the bug with a subproces,
but it occurs with a small dummy module. I'll post again if I can be
more precise.

Kay

[kfj]

On 23 Jan., 13:59, kfj <_...@yahoo.com> wrote:

> hmmm... tricky. I'll post again if I can be more precise.

Now this is really a curious matter. I took a good while to
investigate. Let me first explain the problem again: In SWIG python
Modules that are linked to huginbase and made with cmake statements
like this:

SET_SOURCE_FILES_PROPERTIES(bogous.i PROPERTIES CPLUSPLUS ON)
SWIG_ADD_MODULE(bogous python bogous.i)
SWIG_LINK_LIBRARIES(bogous huginbase)

calls to cerr result in a memory fault. In this case, bogous.i is a
perfectly harmless SWIG interface file:

%module bogous
%{
#include <iostream>
void use_cerr() { std::cerr << "hallo" << "hallo" << std::endl ; }
%}
#include <iostream>
void use_cerr() ;

quite plainly it does nothing but offer a function to trigger cerr
output. Where does the error occur? During the call to
cerr.operator<<(). The first 'hallo' is printed, the next isn't.
Looking at the stack trace with a debugger, it turns out that the
memory fault doesn't occur in cerr.operator<<() itself, but in a
funtion std::uncaught_exception() it calls in libstdc.so:

>>> bogous.uses_cerr()
hallo
Program received signal SIGSEGV, Segmentation fault.
0x02834e77 in std::uncaught_exception() () from /usr/lib/libstdc++.so.
6
(gdb) bt
#0 0x02834e77 in std::uncaught_exception() () from /usr/lib/libstdc+
+.so.6
#1 0x0280a4cb in std::basic_ostream<char, std::char_traits<char> >&
std::__ostream_insert<char, std::char_traits<char>
>(std::basic_ostream<char, std::char_traits<char> >&, char const*,
int) ()
from /usr/lib/libstdc++.so.6
#2 0x0280a7cc in std::basic_ostream<char, std::char_traits<char> >&
std::operator<< <std::char_traits<char> >(std::basic_ostream<char,
std::char_traits<char> >&, char const*) () from /usr/lib/libstdc++.so.
6
#3 0x004f8dea in uses_cerr () at /home/kfj/src/hugin/hugin2010.4.hg/
src/iosbug/uses_cerr.cpp:7
#4 0x004f805b in _wrap_uses_cerr (args=0xb7f9f02c)
at /home/kfj/src/hugin/hugin2010.4.hg-build/src/iosbug/
bogousPYTHON_wrap.cxx:2794
...

This is an obscure error, but not unheard-of. There is a small body of
problem reports of memory faults in std::uncaught_exception(), often
in conjunction, more or less directly, with boost or other foreign
libraries. The problem has never been tracked down, though - usually
the threads dealing with it dead-end or some (often no less obscure)
workaround is applied. It also seems to be a Linux- or Unix- specific
issue, since I've not found any problem reports on other platforms. It
might even be a library bug.

I managed to find a work-around myself, but it is equally obscure: If
I build above module 'bogous' without linking to huginbase, I can
create a module which, when loaded before the offending module, will
make the problem cease. I can even reduce it to the absolute minimum,
let me call it 'vaccine.i':

%module vaccine
%{
%}

quite plainly this module does nothing at all. If I have _vaccine.so
and vaccine.py built by cmake like this:

SET_SOURCE_FILES_PROPERTIES(vaccine.i PROPERTIES CPLUSPLUS ON)
SWIG_ADD_MODULE(vaccine python vaccine.i)

I can do the following (in the output directory, or with the modules
in the module path:)

first, produce the error:

kfj@Anja:~/src/hugin/hpi02.hg-build/src/hsi$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import bogous
>>> bogous.use_cerr()
halloSpeicherzugriffsfehler

now demonstrate the 'vaccine':

kfj@Anja:~/src/hugin/hpi02.hg-build/src/hsi$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import vaccine
>>> import bogous
>>> bogous.use_cerr()
hallohallo
>>>

as you can see, if I import vaccine before bogous, the error does not
occur.

Why does this affect us? Because there are some macros in utils.h
which do output to cerr even if DEBUG is not set. These macros are
invoked when errors occur and usually print a message to the console
to give an idea of a problem while nothing is seen in the GUI. I
noticed their output (and it's failing) when I tested
Panorama.readData() (from a Python session with hsi-enabled hugin) and
called it twice on the same ifstream; obviously the second call must
fail. All I got to see on the console was

WARNSpeicherzugriffsfehler

this reminded me of the recent occasion where someone had unleashed
DEBUG_WARN etc by just outcommenting the #ifdef DEBUG, see

http://groups.google.com/group/hugin-ptx/browse_thread/thread/5ee9374207a404ec#

so I was miffed because I thought the same thing had happened again.
But it looked like this time it was deliberate - and, anyway, the
program shouldn't die on a call to cerr.

You may sympathize with my aversion to a workaround like my 'vaccine'.
It does work for hsi/hpi-enabled hugin, though. Any ideas what the
cause of the problem might be, and a more satisfying way of dealing
with it would be welcome. If it helps I can report it as a bug in the
tracker, with the relevant files attached, but since hsi isn't
mainstream (yet) I'm not sure this is appropriate. Echoes welcome ;-)

Kay