"unresolved external symbol" On Windows trying to call Panda3D's C++ API
Craig Macomber
http://trac.cython.org/cython_trac/ticket/106
I'm trying to compile a trivial Cython script that makes a call into
Panda3D's C++ API (Panda has a great python API, but it's easy to mix
with the C++ API, which seems like a nice job for Cython). It works
great for me on my Macs. On Windows, I have to use MSVC compiler (Form
Visual Studio 2008 Express) to work with Panda3D, and I get this error
when compiling:
Creating library build\temp.win32-2.6\Release\testx.lib and object
build\temp
.win32-2.6\Release\testx.exp
testc.obj : error LNK2019: unresolved external symbol
"__declspec(dllimport) pub
lic: void __thiscall Geom::clear_cache(void)" (__imp_?
clear_cache@Geom@@QAEXXZ)
referenced in function "struct _object * __cdecl
__pyx_pf_5testx_test(struct _ob
ject *,struct _object *)" (?
__pyx_pf_5testx_test@@YAPAU_object@@PAU1@0@Z)
P:\CythonCXXTest\testc\testx.pyd : fatal error LNK1120: 1 unresolved
externals
error: command '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC
\BIN\link.
exe"' failed with exit status 1120
(Full output here: http://dpaste.com/238519/ )
My Cython source is:
cdef extern from "geom.h":
cdef cppclass Geom:
void clear_cache()
from panda3d.core import GeomVertexFormat,GeomVertexData
from panda3d.core import Geom as pGeom
print "Lets roll with C++"
format=GeomVertexFormat.getV3n3c4t2()
vdata=GeomVertexData('vdata', format, pGeom.UHStatic)
geom=pGeom(vdata)
v=geom.this
print "getting ptr"
cdef long dataPtr=v
cdef Geom* g=<Geom*>dataPtr
print "calling C++ method"
g.clear_cache()
print "done"
My setup.py:
import sys
import os
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
# make setup do what we want,
# build extension modules in place
sys.argv.append('build_ext')
sys.argv.append('--inplace')
if os.name=='nt':
include_dirs=[r'C:\Panda3D-1.7.0\include']
elif os.name=='posix':
include_dirs=['/Developer/Panda3D/include/']
else:
# Specify Panda3D include directory here for other systems
include_dirs=['']
setup(
name = 'Demos',
ext_modules=[
Extension("testx",
sources=["testc.pyx"],
language="c++",
include_dirs=include_dirs,
),
],
cmdclass = {'build_ext': build_ext},
)
print "Importing module"
from panda3d.core import Geom
import testx
I have tried the the Rectangles C++ sample with this setup on Windows,
and it did work fine, and if I remove the Call to panda's C++ API, my
code works fine, so I'm pretty sure the issue is linked to the C++
call.
If possible, I'd like to actually make this work on Windows. Any
suggestions for what to try would be greatly appreciated as I'm
completely out of ideas. I'd be happy to do further testing if it
would help resolve the issue, but I don't know what to try.
Thanks!
Lisandro Dalcin
> I believe I might be seeing this issue, but I'm not sure:
> http://trac.cython.org/cython_trac/ticket/106
>
No, your issue is unrelated.
Is geom.h your own code or it is a panda3d header?
Any chance you are missing to pass 'libraries' arg to Extension
constructor? panda3d should have a library, right?
>
> I have tried the the Rectangles C++ sample with this setup on Windows,
> and it did work fine, and if I remove the Call to panda's C++ API, my
> code works fine, so I'm pretty sure the issue is linked to the C++
> call.
>
Indeed, this smells to you not passing libraries and libray_dirs for
the linker to grab a panda3d library.
> If possible, I'd like to actually make this work on Windows.
...
> Any
> suggestions for what to try would be greatly appreciated as I'm
> completely out of ideas. I'd be happy to do further testing if it
> would help resolve the issue, but I don't know what to try.
>
Could you point a link to some panda3d docs were the way to build apps
calling panda3d API is explained?
--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
Craig Macomber
> Is geom.h your own code or it is a panda3d header?
It's from Panda3d. Info here along with a link to the actual header: http://www.panda3d.org/reference/cxx/class!_geom
>
> Any chance you are missing to pass 'libraries' arg to Extension
> constructor? panda3d should have a library, right?
It was unneeded on Mac, but this did fix that issue on Windows. Thanks!
Now I have a pyd file that throws a "SystemError: dynamic module not
initialized properly" when I import it.
Any suggestions for fixing this?
Some of the warnings are worrisome, but I don't understand what they
mean.
The full output from the build is here:
P:\CythonCXXTest\testc>ppython setup.py
running build_ext
cythoning testc.pyx to testc.cpp
building 'testc' extension
creating build
creating build\temp.win32-2.6
creating build\temp.win32-2.6\Release
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /
nologo /Ox
/MD /W3 /GS- /DNDEBUG -IC:\Panda3D-1.7.0\include -IC:
\Panda3D-1.7.0\python\inclu
de -IC:\Panda3D-1.7.0\python\PC /Tptestc.cpp /Fobuild
\temp.win32-2.6\Release\tes
tc.obj
testc.cpp
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE
\xlocale(342) : war
ning C4530: C++ exception handler used, but unwind semantics are not
enabled. Sp
ecify /EHsc
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE
\vector(439) : warn
ing C4275: non dll-interface class 'std::_Container_base_aux' used as
base for d
ll-interface class 'std::_Container_base_aux_alloc_real<_Alloc>'
with
[
_Alloc=pallocator_array<TypedWritable *>
]
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE
\xutility(3
77) : see declaration of 'std::_Container_base_aux'
c:\panda3d-1.7.0\include\vector_src.h(64) : see reference to
class templ
ate instantiation 'std::vector<_Ty,_Ax>' being compiled
with
[
_Ty=TypedWritable *,
_Ax=pallocator_array<TypedWritable *>
]
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE
\vector(439) : warn
ing C4275: non dll-interface class 'std::_Container_base_aux' used as
base for d
ll-interface class 'std::_Container_base_aux_alloc_real<_Alloc>'
with
[
_Alloc=pallocator_array<std::string>
]
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE
\xutility(3
77) : see declaration of 'std::_Container_base_aux'
c:\panda3d-1.7.0\include\vector_src.h(64) : see reference to
class templ
ate instantiation 'std::vector<_Ty,_Ax>' being compiled
with
[
_Ty=std::string,
_Ax=pallocator_array<std::string>
]
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE
\vector(439) : warn
ing C4275: non dll-interface class 'std::_Container_base_aux' used as
base for d
ll-interface class 'std::_Container_base_aux_alloc_real<_Alloc>'
with
[
_Alloc=pallocator_array<unsigned char>
]
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE
\xutility(3
77) : see declaration of 'std::_Container_base_aux'
c:\panda3d-1.7.0\include\vector_src.h(64) : see reference to
class templ
ate instantiation 'std::vector<_Ty,_Ax>' being compiled
with
[
_Ty=unsigned char,
_Ax=pallocator_array<unsigned char>
]
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE
\vector(439) : warn
ing C4275: non dll-interface class 'std::_Container_base_aux' used as
base for d
ll-interface class 'std::_Container_base_aux_alloc_real<_Alloc>'
with
[
_Alloc=pallocator_array<int>
]
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE
\xutility(3
77) : see declaration of 'std::_Container_base_aux'
c:\panda3d-1.7.0\include\vector_src.h(64) : see reference to
class templ
ate instantiation 'std::vector<_Ty,_Ax>' being compiled
with
[
_Ty=int,
_Ax=pallocator_array<int>
]
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /
DLL /nologo
/INCREMENTAL:NO /LIBPATH:C:\Panda3D-1.7.0\python\libs /LIBPATH:C:
\Panda3D-1.7.0\
python\PCbuild C:\Panda3D-1.7.0\lib\libp3framework.lib C:
\Panda3D-1.7.0\lib\libp
anda.lib C:\Panda3D-1.7.0\lib\libpandafx.lib C:\Panda3D-1.7.0\lib
\libpandaexpres
s.lib C:\Panda3D-1.7.0\lib\libp3dtool.lib C:\Panda3D-1.7.0\lib
\libp3dtoolconfig.
lib C:\Panda3D-1.7.0\lib\libp3pystub.lib C:\Panda3D-1.7.0\lib
\libp3direct.lib /E
XPORT:inittestc build\temp.win32-2.6\Release\testc.obj /OUT:P:
\CythonCXXTest\tes
tc\testc.pyd /IMPLIB:build\temp.win32-2.6\Release\testc.lib /
MANIFESTFILE:build\
temp.win32-2.6\Release\testc.pyd.manifest
Creating library build\temp.win32-2.6\Release\testc.lib and object
build\temp
.win32-2.6\Release\testc.exp
C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\mt.exe -nologo -
manifest build
\temp.win32-2.6\Release\testc.pyd.manifest -outputresource:P:
\CythonCXXTest\test
c\testc.pyd;2
Importing module
Traceback (most recent call last):
File "setup.py", line 44, in <module>
import testc
SystemError: dynamic module not initialized properly
P:\CythonCXXTest\testc>
>
> Could you point a link to some panda3d docs were the way to build apps
> calling panda3d API is explained?
>
This page lists the libraries I needed, as some other things:
http://www.panda3d.org/manual/index.php/How_to_build_a_CXX_Panda3D_game_using_Microsoft_Visual_Studio_2008
Lisandro Dalcin
In order to remove thad warning, you could try to pass
extra_compile_args=['/EHsc']
to the Extension constructor.
These warning seems a MSVC nonsense, and they are likely harmless...
> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DLL
> /nologo
> /INCREMENTAL:NO /LIBPATH:C:\Panda3D-1.7.0\python\libs
> /LIBPATH:C:\Panda3D-1.7.0\
> python\PCbuild C:\Panda3D-1.7.0\lib\libp3framework.lib
> C:\Panda3D-1.7.0\lib\libp
> anda.lib C:\Panda3D-1.7.0\lib\libpandafx.lib
> C:\Panda3D-1.7.0\lib\libpandaexpres
> s.lib C:\Panda3D-1.7.0\lib\libp3dtool.lib
> C:\Panda3D-1.7.0\lib\libp3dtoolconfig.
> lib C:\Panda3D-1.7.0\lib\libp3pystub.lib
> C:\Panda3D-1.7.0\lib\libp3direct.lib /E
> XPORT:inittestc build\temp.win32-2.6\Release\testc.obj
> /OUT:P:\CythonCXXTest\tes
> tc\testc.pyd /IMPLIB:build\temp.win32-2.6\Release\testc.lib
> /MANIFESTFILE:build\
> temp.win32-2.6\Release\testc.pyd.manifest
> Creating library build\temp.win32-2.6\Release\testc.lib and object
> build\temp
> .win32-2.6\Release\testc.exp
> C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\mt.exe -nologo -manifest
> build
> \temp.win32-2.6\Release\testc.pyd.manifest
> -outputresource:P:\CythonCXXTest\test
> c\testc.pyd;2
> Importing module
> Traceback (most recent call last):
> File "setup.py", line 44, in <module>
> import testc
> SystemError: dynamic module not initialized properly
>
Mmm... I think you did it wrong, I would try:
library_dirs=[r'C:\Panda3D-1.7.0\python\libs', r'C:\Panda3D-1.7.0\lib']
libraries=['libp3framework', 'libpanda', 'libpandafx', ...]
Craig Macomber
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\xlocale(342) :warning C4530: C++ exception handler used, but unwind semantics are notenabled. Specify /EHsc
In order to remove thad warning, you could try to pass
extra_compile_args=['/EHsc']
to the Extension constructor.
Traceback (most recent call last):File "setup.py", line 44, in <module>import testcSystemError: dynamic module not initialized properly
Mmm... I think you did it wrong, I would try:
library_dirs=[r'C:\Panda3D-1.7.0\python\libs', r'C:\Panda3D-1.7.0\lib']
libraries=['libp3framework', 'libpanda', 'libpandafx', ...]
Lisandro Dalcin
About this line in the pyx file
v=geom.this
What geom.this actually is? Any chance Panda3D Python API is
generated with SWIG? What do you get if you do 'print geom.this' in a
Python prompt? In such case, I doubt such code will ever work (at
least, it would not work with the SWIG versions I worked in the past,
let say up to 1.3.37), you will need something smarter
Still not sure what that error means... Perhaps it is a
Windows-specific error? I'll have to check Python sources to figure
out where that error could came from...
Craig Macomber
It is a pointer to the C++ object as a python integer, and it does
work on mac, so I don't see why I would be an issue on Windows.
Panda3D's python bindings are made with SWIG (Via a very large
collection of hand written .I files I believe). I'm not sure how the
'this' property is implemented on all the Panda3D objects, but it does
work. Anyway though, none of the code in my module ever runs (It fails
to import), and the 'this' call is on the python API which I know
works fine.
Lisandro Dalcin
Does Panda3D exports a C API for writing Python extension modules, for
example like NumPy? In such case, could you point me to such docs?
Perhaps you have to make a import_panda3d() call at the very beginning
of your Cython code?
Do you have any example of a hand-written Python ext module using the
Panda3D API?
Craig Macomber
I'm not very knowledgeable about such things, but I believe Panda3d
just has a C++ API, and a Python API. There is a language reference
for each, and all the manual pages have a toggle link at the top to
switch between Python and C++. Docs are at http://www.panda3d.org/documentation.php
My extension module I'm making is imported from a python module (in
this case my setup.py) after importing the python modules which
corresponds to the C++ modules the code needs to have loaded. This
worked on mac, but on mac, I also didn't need to list the libraries,
so it may be somewhat different as to what initialization is required
before loading the module.
I get the feeling that whatever is in the body of my Cython code is
not useful for the initializing at all as I get this error befor the
first line in my Cython code runs (I tried putting a print at the very
top). The only relevant things as far as I can tell are the externs,
and compile settngs, and the initilization done before importing the
Cython module (such as panda3d imports from python).
On Mac, if I don't import the proper modules from panda before
importing my Cython one, I get this:
ImportError: dlopen(/somePathHere/testc.so, 2): Symbol not found:
__ZTI11TypedObject
Referenced from: /somePathHere/testc.so
Expected in: dynamic lookup
That is significantly different from the error I get on Windows, so I
assumed it's a different issue.
For the sake of simplicity, I can reduce my Cython code to just the
one line 'cdef extern from "geom.h": pass' with no change in behavior
on Windows.
>
> Do you have any example of a hand-written Python ext module using the
> Panda3D API?
>
I have a large collection of Cython modules that use the Panda's
Python API, and one C extension module that I used with panda but does
not use its API. Most panda users just seem to either code in pure
python, or pure C++, so I can't find any actual python extension
modules that call Panda directly.
I know C (somewhat), but I don't know enough C++ to try and write one
that does anything, but if it would be useful, I can try and scrap
together a minimal C++ python extension module and stick the Panda3D C+
+ hello world sample in it ( http://www.panda3d.org/manual/index.php/The_Window_Framework
(see the C++ version of that page) ), then try and make one closer
to what I'm trying to do (assume Panda is already setup properly and
make some call). Also, I could try editing the generated C++ file, or
just try and run a panda3d C++ project on Windows to make sure the
compiler it working with Panda properly (I've never actually written a
C++ project of any kind, much less run one one on Windows). What do
you think would be most useful for me to try?
Thanks!
Lisandro Dalcin
I think there is the root of the issue.
>
> That is significantly different from the error I get on Windows, so I
> assumed it's a different issue.
>
You are in a different OS, so a different error should be expected
:-). The dynamic linking semantics in OS X (and POSIX systems in
general) are rather different...
Please, go to you Mac and figure out where the TypedObject symbol comes from.
Craig Macomber
I fixed that issue by importing the appropriate modules in python, or
so I thought. (In this case with 'from panda3d.core import Geom' )
I got it to work on Windows now! Panda3D's python package package
implements a lazy loading setup, so my attempts to force all the
libraries to load had failed. Also, some of the libraries I linked in
were not actually needed, and one of them would never have gotten
loaded. by removing some unneeded libraries, and digging into the lazy
loader and forcing it to load what I needed (which on Windows is quite
a bit more than on Mac for some reason I still don't understand), I
got it to work. I'll tidy up all my source and publish my work for
other Panda3D users, and post a link here when its ready. Thanks for
your help resolving my compile errors and warnings, as well as your
help resolving this final loading/initialization issue. May I credit
you in my article (probably destined for the Panda3D site's blog) on
using Cython with Panda3D?
Thanks again,
-Craig