Unresolved external symbol PyInit___init__

[Joshua Moore-Oliva]

When I attempt to install wheezy.template (https://pypi.python.org/pypi/wheezy.template/) I get the following error

c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL /nologo

 /INCREMENTAL:NO /LIBPATH:C:\Python34\libs /LIBPATH:C:\Python34\PCbuild /EXPORT:

PyInit___init__ build\temp.win32-3.4\Release\src\wheezy\template\__init__.obj /O

UT:build\lib.win32-3.4\wheezy\template\__init__.pyd /IMPLIB:build\temp.win32-3.4

\Release\src\wheezy\template\__init__.lib /MANIFESTFILE:build\temp.win32-3.4\Rel

ease\src\wheezy\template\__init__.pyd.manifest

LINK : error LNK2001: unresolved external symbol PyInit___init__

build\temp.win32-3.4\Release\src\wheezy\template\__init__.lib : fatal error LNK1

120: 1 unresolved externals

error: command 'c:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\BIN\\

link.exe' failed with exit status 1120

From what I can tell from the setup.py this is installing with cython, and the file it is failing to compile is fairly simple

"""

"""

__version__ = '0.1.151'

# flake8: noqa

from wheezy.template.engine import Engine

from wheezy.template.ext.code import CodeExtension

from wheezy.template.ext.core import CoreExtension

from wheezy.template.loader import DictLoader

from wheezy.template.loader import FileLoader

from wheezy.template.loader import PreprocessLoader

Based on my rudimentary windows linker knowledge, it appears that the command that is failing is attempting to export the symbol "PyInit___init__". However, the generated __init__.c does not have that symbol defined within in.

Based on that, I thought this may be an issue with cython, so I was hoping for some feedback, or possibly explanation.

Thanks, Josh

[Denis Akhiyarov]

I'm trying to cythonnize ptvsd debugger from Visual Studio and getting the same error:

unresolved external symbol PyInit___init__

Here is my modifications to ptvsd to be able to cythonize it:

https://www.dropbox.com/s/9ny16rwyy83q208/ptvsd.zip?dl=0

Full compilation:

C:\Python\Python35_64b\Lib\site-packages\ptvsd>python setup.py build_ext --inplace
Compiling src\visualstudio_py_repl.py because it changed.
Compiling src\visualstudio_py_util.py because it changed.
[1/2] Cythonizing src\visualstudio_py_repl.py
[2/2] Cythonizing src\visualstudio_py_util.py
running build_ext
building 'src.__init__' extension
creating build
creating build\temp.win-amd64-3.5
creating build\temp.win-amd64-3.5\Release
creating build\temp.win-amd64-3.5\Release\src
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Python\Python35_64b\include -IC:\Python\Python35_64b\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.15063.0\winrt" /Tcsrc\__init__.c /Fobuild\temp.win-amd64-3.5\Release\src\__init__.obj
__init__.c
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Python\Python35_64b\libs /LIBPATH:C:\Python\Python35_64b\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.15063.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.15063.0\um\x64" /EXPORT:PyInit___init__ build\temp.win-amd64-3.5\Release\src\__init__.obj /OUT:C:\Python\Python35_64b\Lib\site-packages\ptvsd\src\__init__.cp35-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.5\Release\src\__init__.cp35-win_amd64.lib
LINK : error LNK2001: unresolved external symbol PyInit___init__
build\temp.win-amd64-3.5\Release\src\__init__.cp35-win_amd64.lib : fatal error LNK1120: 1 unresolved externals
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed with exit status 1120

[Denis Akhiyarov]

The compilation works if I exclude `__init__.py`, but this doesn't look right anyway, since the imports fail:

https://www.dropbox.com/s/ss9bmftpyb7rsdu/ptvsd_no_init.zip?dl=0

[Stefan Behnel]

Hi,

this line looks like there shouldn't be an __init__.py file in that direktory:

"building 'src.__init__' extension"

Are you sure that "src" is supposed to be a Python package?

Stefan

[Denis Akhiyarov]

Good point about .\src sub-directory! I moved back all *.py files to root of ptvsd package, where setup.py is located. Then I had to "hack" to get __file__ working with Cython-compiled extensions using this trick:

https://stackoverflow.com/a/38975766/2230844

Here is the final compiled Cython package in subdirectory ptvsd which is now importable from Python:

https://www.dropbox.com/s/77yc2e4bornorb4/ptvsd_.zip?dl=0

Note that __init__.py and __main__.py were just manually copied over.

Next would be to see if I get any speed-up with PTVS debugger?!

Here is the original issue in PTVS issue tracker:

https://github.com/Microsoft/PTVS/issues/749

[Stefan Behnel]

Denis Akhiyarov schrieb am 25.07.2017 um 07:24:
> Good point about .\src sub-directory! I moved back all *.py files to root of ptvsd package, where setup.py is located.

... or delete the __init__.py file from the non-package subdirectory. It
does not belong there, according to Python semantics.

> Then I had to "hack" to get __file__ working with Cython-compiled extensions using this trick:
>
> https://stackoverflow.com/a/38975766/2230844

Yes, this is only supported by CPython 3.5+, together with this pull request:

https://github.com/scoder/cython/tree/pep489_multi_phase_init

> Here is the final compiled Cython package in subdirectory ptvsd which is now importable from Python:
>
> https://www.dropbox.com/s/77yc2e4bornorb4/ptvsd_.zip?dl=0
>
> Note that __init__.py and __main__.py were just manually copied over.

Both must live in a package, though. I also wonder if you really need to
compile them. Are they performance critical? Can't you just leave them as
Python modules?

> Next would be to see if I get any speed-up with PTVS debugger?!

Ah, you're working on a debugger. In that case, I'd really suggest not
compiling the entire package, but only writing a tracer function in Cython
and leave the rest in Python. I.e. only compile or re-write the truly
performance critical parts.

This might also be interesting for you:

https://www.youtube.com/watch?v=NdObDUbLjdg

Stefan

[Denis Akhiyarov]

Yes, I watched that talk. But there is no explanation how they cythonized the settrace function in the pydevd debugger. BTW, here is the code for that talk:

https://github.com/Elizaveta239/frame-eval

Without adding Cython static annotations I'm getting no more than 15% speedup in ptvsd.

Writing the cythonized version of tracing function by hand is well beyond my programming skills.

Is their C-API version of sys.settrace that I can call from Cython?

[Stefan Behnel]

Denis Akhiyarov schrieb am 26.07.2017 um 07:55:
> Yes, I watched that talk. But there is no explanation how they cythonized the settrace function in the pydevd debugger.

She's working on PyCharm, not PyDev.
And the whole point was to *not* use a trace function. ;)

> Without adding Cython static annotations I'm getting no more than 15% speedup in ptvsd.

That's not unexpected. The big gains always come from dropping Python
semantics and replacing them with something C-ish, such as the C-level
tracing API in your case.

> Writing the cythonized version of tracing function by hand is well beyond my programming skills.

You'd probably start with the Python code that you have, change the trace
function signature into a Py_tracefunc compatible cdef function, and
register it through the C tracing API. Then, profile and continue from there.

> Is their C-API version of sys.settrace that I can call from Cython?

https://docs.python.org/3/c-api/init.html#profiling-and-tracing

Stefan

[Denis Akhiyarov]

Thanks for the pointers! pycharm and pydev share the same debugger pydevd optimized with Cython. yes, I'm aware that their debugger now uses frame evaluation API, but it is only available for Python 3.6+, so settrace is still required for older versions.

Stefan

--

---
You received this message because you are subscribed to a topic in the Google Groups "cython-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cython-users/DSmOcM3B2jA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.