Different behavior of Sire script when executed by python or ipython

[julien]

Hi Chris,

I have come across a strange problem with Sire. In the process of testing a free energy setup script, I have found one set of input files (the only case so far) that causes this behavior on my machine.

* If I execute the script with python

python /home/julien/local/bin/sirescripts/genmorph3.py -i /scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/cyclopropyl-benzene/vacuum.top -j /scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/cyclopropyl-benzene/vacuum.crd -k /scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/benzene/vacuum.top -l /scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/benzene/vacuum.crd

The script crashes with this error message

scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/cyclopropyl-benzene/vacuum.top
/scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/cyclopropyl-benzene/vacuum.crd
/scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/benzene/vacuum.top
/scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/benzene/vacuum.crd
2 molecules converted
36 audit log messages
/home/julien/local/lib/python2.6/site-packages/rdkit/Chem/__init__.py:18: RuntimeWarning: to-Python converter for std::vector<double, std::allocator<double> > already registered; second conversion method ignored.
  from rdkit import rdBase
Running fmcs with arguments ...['--compare', 'topology', '--output-all', '--output-format', 'complete-sdf', '--save-atom-indices-tag', 'mcs', '--output', 'mcs.sdf', 'pair.sdf']

Traceback (most recent call last):
  File "/home/julien/local/bin/sirescripts/genmorph3.py", line 295, in <module>
    lig_morph = lig_morph.add( CGName("0") )\
UserWarning: Exception 'SireError::program_bug' thrown by the thread 'master'.
An EditMolData can only identify 4294967295 unique objects!

Thrown from FILE: /home/julien/software/devel/sire/corelib/src/libs/SireMol/structureeditor.cpp, LINE: 1326, FUNCTION: quint32 SireMol::detail::EditMolData::getNewUID()
__Backtrace__
(  0) /home/julien/local/lib/libSireError.so.0 ([0x7f14a3c674ab] ++0x2b)
  -- SireError::getBackTrace()

(  1) /home/julien/local/lib/libSireError.so.0 ([0x7f14a3c647c7] ++0x97)
  -- SireError::exception::exception(QString, QString)

/home/julien/local/lib/libSireMol.so.0(+0x436748) [0x7f149d081748]
(  3) /home/julien/local/lib/libSireMol.so.0 ([0x7f149d089041] ++0x31)
  -- SireMol::StructureEditor::addCutGroup()

(  4) /home/julien/local/lib/libSireMol.so.0 ([0x7f149cfca709] ++0x19)
  -- SireMol::MolStructureEditor::add(SireMol::CGName const&)

/home/julien/local/lib/python2.6/site-packages/Sire/Mol/_Mol.so(+0x60e577) [0x7f149d9d7577]
(  6) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a479dc2e] ++0x28e)
  -- boost::python::objects::function::call(_object*, _object*) const

/usr/lib/libboost_python-py26.so.1.40.0(+0x30ed8) [0x7f14a479ded8]
(  8) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a5753] ++0x53)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

(  9) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdmolops.so ([0x7f14946c42d7] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<RDKit::MolSanitizeException, void (*)(RDKit::MolSanitizeException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(RDKit::MolSanitizeException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 10) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 11) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdmolops.so ([0x7f14946c4277] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<ValueErrorException, void (*)(ValueErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(ValueErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 12) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 13) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdmolops.so ([0x7f14946c4217] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<IndexErrorException, void (*)(IndexErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(IndexErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 14) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 15) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdmolfiles.so ([0x7f1494bf8977] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<RDKit::BadFileException, void (*)(RDKit::BadFileException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(RDKit::BadFileException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 16) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 17) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdmolfiles.so ([0x7f1494bf8917] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<RDKit::MolSanitizeException, void (*)(RDKit::MolSanitizeException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(RDKit::MolSanitizeException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 18) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 19) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdmolfiles.so ([0x7f1494bf88b7] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<ValueErrorException, void (*)(ValueErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(ValueErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 20) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 21) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdmolfiles.so ([0x7f1494bf8857] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<IndexErrorException, void (*)(IndexErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(IndexErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 22) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 23) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdchem.so ([0x7f1496dedc37] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<RDKit::ConformerException, void (*)(RDKit::ConformerException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(RDKit::ConformerException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 24) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 25) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdchem.so ([0x7f1496dc5b17] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<RDKit::MolSanitizeException, void (*)(RDKit::MolSanitizeException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(RDKit::MolSanitizeException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 26) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 27) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdchem.so ([0x7f1496dc5ab7] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<ValueErrorException, void (*)(ValueErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(ValueErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 28) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 29) /home/julien/local/lib/python2.6/site-packages/rdkit/Chem/rdchem.so ([0x7f1496dc5a57] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<IndexErrorException, void (*)(IndexErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(IndexErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 30) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 31) /home/julien/local/lib/python2.6/site-packages/rdkit/Geometry/rdGeometry.so ([0x7f14972b3a17] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<ValueErrorException, void (*)(ValueErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(ValueErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 32) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 33) /home/julien/local/lib/python2.6/site-packages/rdkit/Geometry/rdGeometry.so ([0x7f14972b39b7] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<IndexErrorException, void (*)(IndexErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(IndexErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 34) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 35) /home/julien/local/lib/python2.6/site-packages/rdkit/DataStructs/cDataStructs.so ([0x7f1497821d07] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<ValueErrorException, void (*)(ValueErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(ValueErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 36) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 37) /home/julien/local/lib/python2.6/site-packages/rdkit/DataStructs/cDataStructs.so ([0x7f1497821ca7] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<IndexErrorException, void (*)(IndexErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(IndexErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 38) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 39) /home/julien/local/lib/python2.6/site-packages/rdkit/rdBase.so ([0x7f1498fb0497] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<ValueErrorException, void (*)(ValueErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(ValueErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 40) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

( 41) /home/julien/local/lib/python2.6/site-packages/rdkit/rdBase.so ([0x7f1498fb0437] ++0x17)
  -- boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<IndexErrorException, void (*)(IndexErrorException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(IndexErrorException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)

( 42) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

/home/julien/local/lib/python2.6/site-packages/Sire/Error/_Error.so(+0x2866) [0x7f14a2a01866]
( 44) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a572d] ++0x2d)
  -- boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const

/home/julien/local/lib/python2.6/site-packages/Sire/Error/_Error.so(+0x281a) [0x7f14a2a0181a]
( 46) /usr/lib/libboost_python-py26.so.1.40.0 ([0x7f14a47a5523] ++0x33)
  -- boost::python::handle_exception_impl(boost::function0<void>)

/usr/lib/libboost_python-py26.so.1.40.0(+0x2d7d8) [0x7f14a479a7d8]
( 48) python ([0x41f0c7] ++0x47)
  -- PyObject_Call

( 49) python ([0x4a7378] ++0x4888)
  -- PyEval_EvalFrameEx

( 50) python ([0x4a9671] ++0x911)
  -- PyEval_EvalCodeEx

( 51) python ([0x4a9742] ++0x32)
  -- PyEval_EvalCode

( 52) python ([0x4c9a0e] ++0x13e)
  -- PyRun_FileExFlags

( 53) python ([0x4c9c24] ++0xd4)
  -- PyRun_SimpleFileExFlags

( 54) python ([0x41a7ff] ++0x9ef)
  -- Py_Main

( 55) /lib/libc.so.6 ([0x7f14a53f0c4d] ++0xfd)
  -- __libc_start_main

python() [0x4199f9]
__EndTrace__

Note how the backtrace mixes messages from Sire and another python library (rdkit) which the script genmorph.py also uses.

*  As I said above, this only happens for that particular set of input files ( ca. 30 other perturbations do not cause this error). Now IF i launch ipython and then execute this script, this error
does NOT happen and the script terminates correctly.

julien@michelws001:test$ ipython
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
Type "copyright", "credits" or "license" for more information.

IPython 0.10 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: run /home/julien/local/bin/sirescripts/genmorph3.py -i /scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/cyclopropyl-benzene/vacuum.top -j /scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/cyclopropyl-benzene/vacuum.crd -k /scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/benzene/vacuum.top -l /scratch/julien/projects/thrombin/sire-setup-hannes/tests/08/ligands/benzene/vacuum.crd

(lots of output deleted...)

Initial and Final potentials are the same
 IMPROPER  AtomName('C') AtomName('C4') AtomName('C5') AtomName('C6')
 morph potential [1.1000000000000001, 2.0, 3.141594]
 initial potential [1.1000000000000001, 2.0, 3.141594]
 final potential [1.1000000000000001, 2.0, 3.141594]
Initial and Final potentials are the same
 IMPROPER  AtomName('C3') AtomName('C5') AtomName('C4') AtomName('H4')
 morph potential [1.1000000000000001, 2.0, 3.141594]
 initial potential [1.1000000000000001, 2.0, 3.141594]
 final potential [1.1000000000000001, 2.0, 3.141594]
Initial and Final potentials are the same
GENMORPH TERMINATED SUCCESSFULLY

In [2]:


* Any ideas? I don't even know where to start :-)

[Christopher Woods]

Hi Julien,

That is very strange behaviour. The error is thrown because more than
2^32 objects have been added to a single molecule, although I don't
know how that was possible. It sounds like something is corrupting
memory somewhere. Are the test scripts part of the CCP-Biosim
repository yet? I'd like to download them and try to run everything
under a debugger.

Cheers,

Christopher

[julien]

Hi Chris,

The problem is hard to debug because it does not systematically happens.
 I think the errors happens because  genmorph imports another python module (fmcs) halfway through execution, and that -sometimes- causes
Sire to crash.  Since moving the statement import fmcs to the top of the python script, I cannot reproduce the error. 

A clue that there is a compatibility problem with Sire and fmcs is:

* If I import fmcs before Sire.Base I get this runtimewarning message

/home/julien/local/lib/python2.6/site-packages/Sire/Base/__init__.py:15: RuntimeWarning: to-Python converter for std::vector<double, std::allocator<double> > already registered; second conversion method ignored.
  from Sire.Base._Base import *

So far no crashes

* If I import fmcs after Sire.Base

/home/julien/local/lib/python2.6/site-packages/rdkit/Chem/__init__.py:18: RuntimeWarning: to-Python converter for std::vector<double, std::allocator<double> > already registered; second conversion method ignored.
  from rdkit import rdBase

With this order of module import I get -sometimes- crashes.

The scripts are not on the CCP-BioSim repository yet. You would need to install new dependencies to test the code. I can email you a tarfile if you want,
but it might be better to wait and see if I truly have no errors if fmcs is imported before Sire.Base.

Julien

[Christopher Woods]

Hi Julien,

That is very strange, as the RunTime warnings should not affect the
C++ code (as each converter should do the same job and go from a
python list to a C++ QVector). They certainly shouldn't affect the UID
in EditMolData. I'd really like to debug this, as it could be an
indication of something more serious underneath. Could you email me
the tar file?

Cheers,

Christopher