cmake help needed for python interface
kfj
I'm working on the python plugin interface and have run into a
problem. I am using SWIG to generate the wrapper code, and to access
the SWIG-generated types from C++, I have to include an automatically
generated header file. On the command line I can generate this file by
typing
swig -c++ -python -external-runtime
Which produces the file in the current directory with the default name
'swigpyrun.h'. I can also specify the filename, like in
swig -c++ -python -external-runtime use_another_name.h
If I'm not happy with the default name or want it somewhere else.
Cmake is set up as per Cmake wiki, this is the section I use to
generate the wrapper code from the two files hsi.i and hsi.cpp:
FIND_PACKAGE(SWIG REQUIRED)
INCLUDE(${SWIG_USE_FILE})
FIND_PACKAGE(PythonLibs)
INCLUDE_DIRECTORIES(${PYTHON_INCLUDE_PATH})
INCLUDE_DIRECTORIES(${CMAKE_CURRENT_SOURCE_DIR})
SET(CMAKE_SWIG_FLAGS "")
SET_SOURCE_FILES_PROPERTIES(hsi.i PROPERTIES CPLUSPLUS ON)
SWIG_ADD_MODULE(hsi python hsi.i hsi.cpp)
SWIG_LINK_LIBRARIES(hsi ${PYTHON_LIBRARIES} ${common_libs})
That works well and automatically generates the relevant wrapper files
hsi.py and _hsi.so, but so far I haven't found a way of telling Cmake
to also generate above-mentioned header file swigpyrun.h. It's not
totally crucial to generate it from scratch every time, but it must be
compatible with the wrapper code, so I'd rather have it generated than
later run into obscure incompatibilities for using one I make manually
now.
I also have another question: Cmake generates a C++ file which
contains the wrapping. This file is clean C++ and does not rely on any
SWIG libraries or headers to compile - it can be used to distribute
SWIG's output without SWIG. This file is generated from the interface
definition file; in my case, with the CMakeLists.txt as above, it is
generated from hsi.i, named automatically as hsiPYTHON_wrap.cxx and
written to the output directory. then it is compiled and linked with
hsi.cpp and the output, _hsi.so and hsi.py is created. My question is
how I could customize the behaviour to:
- create hsiPYTHON_wrap.cxx only if SWIG and hsi.i are available
- use a prefabricated version of it if SWIG is not available and not
bother with hsi.i at all in that case
Help with these problems would be greatly appreciated
Kay
T. Modes
On 18 Jan., 16:47, kfj <_...@yahoo.com> wrote:
> That works well and automatically generates the relevant wrapper files
> hsi.py and _hsi.so, but so far I haven't found a way of telling Cmake
> to also generate above-mentioned header file swigpyrun.h. It's not
> totally crucial to generate it from scratch every time, but it must be
> compatible with the wrapper code, so I'd rather have it generated than
> later run into obscure incompatibilities for using one I make manually
> now.
e.g. something like
ADD_CUSTOM_COMMAND(OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/swigpyrun.h"
COMMAND "${SWIG_EXECUTABLE}"
ARGS "-c++" "-python" "-external-runtime"
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
COMMENT "Build python header file..."
)
SET_SOURCE_FILES_PROPERTIES("${CMAKE_CURRENT_BINARY_DIR}/swigpyrun.h"
GENERATED)
ADD_CUSTOM_TARGET(hsi_header DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/
swigpyrun.h")
>
> - create hsiPYTHON_wrap.cxx only if SWIG and hsi.i are available
FIND_PACKAGE(SWIG)
IF(SWIG_FOUND)
..
ENDIF()
> - use a prefabricated version of it if SWIG is not available and not
> bother with hsi.i at all in that case
generated if the (hugin) source was changed.
So the scripting interface should only be build if swig is found.
Thomas
PS: I tried hsi on windows. It compiles. But the generated lib is
called _hsi.pyd (not hsi.pyd).
And when I try to run in python your example, I'm getting errors. I
can run the procedure, but I'm can access the object:
>>> from _hsi import *
>>> p=pano_open('974-1-975-1.pto')
>>> img0=p.getImage(0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'SwigPyObject' object has no attribute 'getImage'
>>> p.getCtrlPoints()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'SwigPyObject' object has no attribute 'getCtrlPoints'
T. Modes
> PS: I tried hsi on windows. It compiles. But the generated lib is
> called _hsi.pyd (not hsi.pyd).
> And when I try to run in python your example, I'm getting errors. I
> can run the procedure, but I'm can access the object:
>
But I found an issue: I can not access the image variables of
SrcPanoImage like getImage(0).getYaw() and so on
Thomas
Pablo d'Angelo
Am 18.01.2011 19:24, schrieb T. Modes:
>
> But I found an issue: I can not access the image variables of
> SrcPanoImage like getImage(0).getYaw() and so on
Thats probably because these functions are defined using some macros
inside Panorama.h. Maybe swig needs to call the C preprocessor or we
need to call swig on an already preprocessed Panorama.h.
ciao
Pablo
T. Modes
> > But I found an issue: I can not access the image variables of
> > SrcPanoImage like getImage(0).getYaw() and so on
>
> Thats probably because these functions are defined using some macros
> inside Panorama.h.
not in Panorama.h ;-)
In hsi.i there are some includes commented out.
I think at least ImageVariableGroup.h is also needed to access the
information about stacks and lenses. So this should also added.
But this is a task for someone with swig knowledge ;-)
Thomas
PS: When compiling hsi, the following warnings come up:
>src/hugin_base\panodata\PanoramaVariable.h(71) : Warning 314: 'print' is a python keyword, renaming to '_print'
>src/hugin_base\panodata\Panorama.h(58) : Warning 362: operator= ignored
>src/hugin_base\algorithms\assistant_makefile\AssistantMakefilelibExport.h(65) : Warning 325: Nested struct not currently supported (AssistantPrograms ignored)
>src/hugin_base\algorithms\panorama_makefile\PanoramaMakefilelibExport.h(84) : Warning 325: Nested struct not currently supported (PTPrograms ignored)
>src/hugin_base\algorithms\optimizer\PhotometricOptimizer.h(134) : Warning 401: Base class 'SmartOptimizerStub' undefined.
>src/hugin_base\algorithms\optimizer\PTOptimizer.h(124) : Warning 401: 'SmartOptimizerStub' must be defined before it is used as a base class.
>src/hugin_base\panodata\Lens.h(126) : Warning 462: Unable to set dimensionless array variable
>src/hugin_base\panodata\ControlPoint.h(111) : Warning 462: Unable to set dimensionless array variable
Maybe this helps to improve the interface.
kfj
On 18 Jan., 20:35, "T. Modes" <Thomas.Mo...@gmx.de> wrote:
Hi guys!
I've just uploaded a recent tar file with all the code needed to run
Python plugins from hugin. You may want this - the hsi part is pretty
unchanged, but the plugin interface for hugin is new. You may want to
wait until I have a modified hugin repo up, though, which should be
the case soon. Until then, pick
http://bazaar.launchpad.net/~kfj/+junk/script/view/head:/main/hsi.src.tgz
In the tar file, there's also up-to-date information. Just make a
backup copy of your hsi directory and unpack the tar file to get a new
hsi directory - changes outside are reflected in the files in the
'collateral' subdirectory. For now, don't worry about the warnings (I
get the same here); the scope of this wrap is so great that a measly
few warnings shouldn't worry us - the main thing is that the code
performs largely as intended - finetuning can follow later once the
groundwork is laid.
Kay
kfj
>
> > > But I found an issue: I can not access the image variables of
> > > SrcPanoImage like getImage(0).getYaw() and so on
img.getVar('y')
I'll have a lok and see if I can figure out what's amiss.
Kay
kfj
On 19 Jan., 09:21, kfj <_...@yahoo.com> wrote:
> On 18 Jan., 20:35, "T. Modes" <Thomas.Mo...@gmx.de> wrote:
>
> > Hi Pablo,
>
> > > > But I found an issue: I can not access the image variables of
> > > > SrcPanoImage like getImage(0).getYaw() and so on
* Used for lazy metaprogramming, we include image_variables.h several
times
* with different defintions of image_variable to get all the
repetitive bits
* out the way. This should reduce typos and cut and paste errors.
SWIG has two %include modes: one mode is to wrap the header at hand
and all it includes. This is usually unwise, because you don't want
every system header inside your module, which bloats it very much. You
can do that if your header is independent from system stuff. Also,
wrapping everything would usually throw up problems since the system
headers often use nonstandard extensions to C++.
The other SWIG mode is to just include the top level of the header at
hand. This is the mode I'm currently using. The 'lazy metaprogramming'
would work here only if the header was actually %included into SWIG
several times with the relevant #defines active. This may work, but is
it necessary? Until then, as posted previously, use the getVar()
method as a workaround.
Kay
T. Modes
On 19 Jan., 09:32, kfj <_...@yahoo.com> wrote:
> This may work, but is
> it necessary? Until then, as posted previously, use the getVar()
> method as a workaround.
like yaw, pitch and so on. But with this I don't get e.g. the
projection or the crop region of the image, just to name some of the
also. Also without the macros I can not access the information about
linked variables (isLinked/isLinkedWith) in python.
Thomas
kfj
the multiple inclusion of SrcPanoImage.h which instantiates the getXXX
macros etc. with different variables - is happening? Or am I maybe
misunderstanding the mechanism? - in which case please clarify how the
accessor functions come to be. Then I can try and emulate the process
in the interface definition file and make it generate the accessor
functions. Now that the first step is done and the branch is
established in the SF repo, I can start looking at details. Sory for
having ben sloppy - I should have noticed the omission earlier. Blame
it on my inexperience with the hugin body of code.
Kay
kfj
requested in the initial posting. I would really appreciate if someone
could actually at least touch the issue. I set up a thread
http://groups.google.com/group/hugin-ptx/browse_thread/thread/9a8e2872f77c41b6/0edb8167dc8c4c38#0edb8167dc8c4c38
for developers to discuss the scripting interface in general, so the
discussion about the accessor macros in SrcPanoImage.h would probably
fit in there better.
Kay
kfj
On 20 Jan., 09:53, kfj <_...@yahoo.com> wrote:
> On 18 Jan., 16:47, kfj <_...@yahoo.com> wrote:
>
> > Help with these problems would be greatly appreciated
>
> That was, help with cmake, as stated on the thread headline and
oops... I'm sorry, I'll have to swallow my words, since the problem
has been adressed already! I must have overlooked the relevant bit in
T. Modes' initial reply. I apologize.
Kay
T. Modes
> > > Help with these problems would be greatly appreciated
>
> > That was, help with cmake, as stated on the thread headline and
most of the problems (I commited it in the meanwhile to the
repository).
Thomas
kfj
meantime figured it out from your initial reply to my post, just that
I had to change it so that swigpyrun.h appears in the source
directory, not in the target. I'll see if I can get your patch from
the repo. Thanks again. Sometimes I really wish I was back in the old
times when it was just writing makefiles, I was quite good at that in
my time... but I realize that CMake is a great help - once one has
mastered it :(
Kay
kfj
I'm still struggling with cmake for hsi/hpi, and I just can't get my
head round it's strange ways, and what online documentation there is I
don't find helpful either. So maybe someone can help me with this
simple problem:
I have a C++ header file zz.h in my source directory
I want this preprocessed by the C preprocessor into a file zz.i in the
target directory
This is a simple operation, and cmake even generates i-files as
default targets in C++ projects. So if I have a simple CMakeLists.txt
going like
add_executable(xx xx.cpp)
(really only this one line!) and xx.cpp is also very simple:
#define mac "MACRO"
int main ( int argc , char * argv[] )
{
char * str = mac ;
}
if i now run
cmake .
make xx.i
I get this output:
Preprocessing CXX source to CMakeFiles/xx.dir/xx.i
lo and behold, CMakeFiles/xx.dir/xx.i is the C-preprocessed file. I
didn't need to specify anything in the CMakeLists.txt, cmake knows
perfectly well how to invoke the C/C++ preprocessor and make an i-file
from a .cpp file. My question is: how can I get it to uses this
capability on my source file zz.h to get zz.i in the target directory?
So far the answer was to run a custom command, with differentiation
between MSVC and gcc, probably use of a cmake command file - T. Modes
has managed to coerce cmake into doing what's needed for hsi/hpi with
a good dozen lines of code, but I have the feeling this is not at all
needed - and the current code is so obscure that I'm not at all happy
with it, because it's really such a standard task that in a classic
makefile it'd be there by default, like .i .h. But I can't figure it
out. Help!
I'd also appreciate a link to documentation that explains the concepts
behind cmake. I get the feeling that the online documentation for
cmake is deliberately sketchy to promote the sales of the book which
is featured on the main page. It's either making hello-world-type
projects with three source files and a bit of configuration or
extremely terse 'Documentation' which has no conceptual explanations
or examples. You know what cmake syntax reminds me of? Lisp. and that
has always been good for giving me a headache. There's a down side to
a simple syntax: it's all in semantics.
Kay
Kornel Benko
Sorry, there is no such rule as in makefile. You have to add_custom_commad() for each .h file.
If you have a list of files, then you could achieve the effect of ".h.i" rule like this:
file(GLOB hlist ${CMAKE_CURRENT_SOURCE_DIR}*.h)
foreach(hfile ${hlist})
get_filename_component(hbase ${hfile} NAME_WE)
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${hbase}.i"
COMMAND ${CMAKE_CXX_COMPILER}
ARGS -E ${hfile} > "${CMAKE_CURRENT_BINARY_DIR}/${hbase}.i"
DEPENDS ${hfile}
)
SET_SOURCE_FILES_PROPERTIES("${CMAKE_CURRENT_BINARY_DIR}/${hbase}.i" GENERATED)
endforeach()
But, why not rename xx.h into xx.cpp?
> I'd also appreciate a link to documentation that explains the concepts
> behind cmake. I get the feeling that the online documentation for
> cmake is deliberately sketchy to promote the sales of the book which
> is featured on the main page. It's either making hello-world-type
> projects with three source files and a bit of configuration or
> extremely terse 'Documentation' which has no conceptual explanations
> or examples. You know what cmake syntax reminds me of? Lisp. and that
> has always been good for giving me a headache. There's a down side to
> a simple syntax: it's all in semantics.
I don't have that either.
> Kay
>
Kornel
kfj
On 26 Jan., 12:37, Kornel Benko <Kornel.Be...@berlin.de> wrote:
> But, why not rename xx.h into xx.cpp?
zz.cpp in my source directory and I want it to be C-preprocessed into
a file z.i in my binary directory, from where it's included into
another file. What do I have to tell cmake to do this? My problem is
really very basic, it should be a one-liner. Please excuse my
ignorance. It would then be equivalent to the make rule of
.i .c
which I know cmake can do.
Kay
Kornel Benko
Am Mittwoch, 26. Januar 2011 schrieb kfj:
> On 26 Jan., 12:37, Kornel Benko <Kornel.Be...@berlin.de> wrote:
> > But, why not rename xx.h into xx.cpp?
>
> Hey, no problem, I can rename it into anything. So let's assume I have
> zz.cpp in my source directory and I want it to be C-preprocessed into
> a file z.i in my binary directory, from where it's included into
to "zz.i"
> another file. What do I have to tell cmake to do this?
You have to define a dependency
e.g.
add_custom_command(
...
DEPENDS zz.i ....
)
> My problem is
> really very basic, it should be a one-liner. Please excuse my
> ignorance. It would then be equivalent to the make rule of
>
> .i .c
You mean ".c.i" here, don't you?
> which I know cmake can do.
>
> Kay
Kornel
kfj
> You have to define a dependency
>
> e.g.
> add_custom_command(
> ...
> DEPENDS zz.i ....
> )
IF(MSVC)
SET(PREPROCESSOR_FLAGS "/EP")
ELSE()
SET(PREPROCESSOR_FLAGS "-E")
ENDIF()
SET(C_PREPROCESS ${CMAKE_CXX_COMPILER} ${PREPROCESSOR_FLAGS})
macro(preprocess in out)
add_custom_command(OUTPUT ${out}
COMMAND ${C_PREPROCESS} ${in} -o ${out}
DEPENDS ${in}
)
endmacro(preprocess)
preprocess(${CMAKE_CURRENT_SOURCE_DIR}/Panorama_accessors.h
Panorama_accessors.i)
preprocess(${CMAKE_CURRENT_SOURCE_DIR}/SrcPanoImage_accessors.h
SrcPanoImage_accessors.i)
ADD_CUSTOM_TARGET(hsi_accessors
DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/Panorama_accessors.i"
"${CMAKE_CURRENT_BINARY_DIR}/SrcPanoImage_accessors.i"
)
now I can make hsi_accessors. It works, but it seems like a rather
circuitious route to do something very basic. Now the next step I have
to do is get the dependency chain to work. I have the following
dependencies:
XXX_accessors.h depends on <panodata/image_variables.h>
XXX_accessors.i depends on XXX_accessors.h
hsi.i depends on XXX_accessors.i
additionally, hsi.i also depends on all the header files it %includes
to generat the wrapper code from. So I'd like to express that hsi.i
needs to be processed by SWIG whenever any of the files it depends on
changes. But hsi.i isn't made from the files it includes, it only has
to be considered out of date when it's older than any of the files it
includes. I don't know how to express that in cmake.
The 'lazy metaprogramming' is sure turning out expensive when my time
is concerned... :(
> > .i .c
> You mean ".c.i" here, don't you?
old-fashioned to use rules like this. I was confused because usually
the semantics in makefiles are target before source, like in %.i : %.c
So cmake has this rule built-in, but I can't figure out how to get it
to use it.
Kay
Kornel Benko
Am Donnerstag, 27. Januar 2011 schrieb kfj:
> On 26 Jan., 21:34, Kornel Benko <Kornel.Be...@berlin.de> wrote:
> > You have to define a dependency
> >
> > e.g.
> > add_custom_command(
> > ...
> > DEPENDS zz.i ....
> > )
>
> So I tried this:
>
> IF(MSVC)
> SET(PREPROCESSOR_FLAGS "/EP")
> ELSE()
> SET(PREPROCESSOR_FLAGS "-E")
> ENDIF()
This is btw not needed, even MSVC understands "-".
> SET(C_PREPROCESS ${CMAKE_CXX_COMPILER} ${PREPROCESSOR_FLAGS})
>
> macro(preprocess in out)
> add_custom_command(OUTPUT ${out}
> COMMAND ${C_PREPROCESS} ${in} -o ${out}
> DEPENDS ${in}
> )
> endmacro(preprocess)
>
> preprocess(${CMAKE_CURRENT_SOURCE_DIR}/Panorama_accessors.h
> Panorama_accessors.i)
> preprocess(${CMAKE_CURRENT_SOURCE_DIR}/SrcPanoImage_accessors.h
> SrcPanoImage_accessors.i)
>
> ADD_CUSTOM_TARGET(hsi_accessors
> DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/Panorama_accessors.i"
> "${CMAKE_CURRENT_BINARY_DIR}/SrcPanoImage_accessors.i"
> )
>
> now I can make hsi_accessors. It works, but it seems like a rather
> circuitious route to do something very basic.
Yes, I am also not happy with this.
> Now the next step I have
> to do is get the dependency chain to work. I have the following
> dependencies:
>
> XXX_accessors.h depends on <panodata/image_variables.h>
> XXX_accessors.i depends on XXX_accessors.h
> hsi.i depends on XXX_accessors.i
>
> additionally, hsi.i also depends on all the header files it %includes
> to generat the wrapper code from. So I'd like to express that hsi.i
> needs to be processed by SWIG whenever any of the files it depends on
> changes. But hsi.i isn't made from the files it includes, it only has
> to be considered out of date when it's older than any of the files it
> includes. I don't know how to express that in cmake.
There is a construct in cmake for this.
First define a project before creating hsi.i
project(hsi)
...
add_custom_command() # to create hsi.i
...
add_custom_target(hsi ALL DEPENDS ${from_whatever})
Then define a project for SWIG
project(swig)
...
add_custom_commad() # whatever
...
add_custom_target(swig ALL DEPENDS ${from_whatever})
You may also consider dependencies between targets
add_dependencies(swig hsi)
...
> The 'lazy metaprogramming' is sure turning out expensive when my time
> is concerned... :(
You have always to learn. 'lazy' is valid for simple tasks only.
> Kay
Kornel
kfj
On 27 Jan., 11:04, Kornel Benko <Kornel.Be...@berlin.de> wrote:
> You have always to learn. 'lazy' is valid for simple tasks only.
satisfaction. I hope you don't mind me moaning about cmake, in the end
it can be persuaded to do the right thing and once it's set up
properly, it does it's job really nicely. Once I'm through with
setting up cmake, I think it'll soon be time to upload a new revision
of hsi/hpi with even more functionality. I really want to USE the
thing, but first things first...
Kay
Kornel Benko
Am Donnerstag, 27. Januar 2011 schrieb kfj:
> On 27 Jan., 11:04, Kornel Benko <Kornel.Be...@berlin.de> wrote:
> > You have always to learn. 'lazy' is valid for simple tasks only.
>
> Kornel, thank you for your help. Now I think I can set it up to my
> satisfaction.
Nice to hear ...
> I hope you don't mind me moaning about cmake, in the end
> it can be persuaded to do the right thing and once it's set up
> properly, it does it's job really nicely. Once I'm through with
> setting up cmake, I think it'll soon be time to upload a new revision
> of hsi/hpi with even more functionality. I really want to USE the
> thing, but first things first...
Yes it's nice once mastered :)
> Kay
Kornel