Importing modules with DLLs fails with Internal Server Error

[baloan]

Hello,

I have tried to setup apache with hg, trac and web2py running on
Windows XP SP3 but failed. Below is a detailed list of components I
have installed.

hg, trac and web2py work fine standalone (tracd 8000, web2py -
a'<recycle>') but when I try to run them behind Apache with WSGI I see
ImportErrors when importing psycopg2 or mercurial specific modules.
The installation seems to be fine because using PostgreSQL works from
the command line (see below).

I have then setup a 'Hello World' WSGI to prove importing psycopg2 is
the problem. See below for sys.path and Apache log details.

Who knows what is wrong and how to fix?

Regards, Andreas
bal...@gmail.com

-------------
Windows XP SP3
Python 2.6.5
psycopg 2.2.1
trac 0.12
mercurial 1.5.4
web2py 1.79.1

Apache 2.2.15
mod_wsgi 3.0 (win32 ap22py26)

PostgreSQL 8.4.3-1
-------------

-- console --
D:\Home\web\apache\conf>python
Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2 as pg
>>> db = pg.connect(database='invest', user='invest', password='invest')
>>> db.close()
>>> quit()
-------------

-- Apache configuration --
...
WSGIPythonHome "C:/Apps/Python26"
WSGIPythonPath "C:/Apps/Python26/Lib/site-packages"
WSGIScriptAlias /hw "D:/home/web/apache/cgi-bin/hw.wsgi"
WSGIScriptAlias /hg "D:/home/web/apache/cgi-bin/hgwebdir.wsgi"
WSGIApplicationGroup %{GLOBAL}
...
-------------


-- hgwebdir.wsgi --
from mercurial import demandimport; demandimport.enable()
from mercurial.hgweb.hgwebdir_mod import hgwebdir
application = hgwebdir('D:/home/web/hg/etc/hgweb.config')
-------------

-- Apache error log --
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] mod_wsgi
(pid=2332): Target WSGI script 'D:/Home/web/apache/cgi-bin/
hgwebdir.wsgi' cannot be loaded as Python module.
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] mod_wsgi
(pid=2332): Exception occurred processing WSGI script 'D:/Home/web/
apache/cgi-bin/hgwebdir.wsgi'.
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] Traceback
(most recent call last):
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "D:/
Home/web/apache/cgi-bin/hgwebdir.wsgi", line 51, in <module>
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24]
application = hgwebdir('D:/home/web/hg/etc/hgweb.config')
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\hgweb\
\hgwebdir_mod.py", line 52, in __init__
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24]
self.refresh()
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\hgweb\
\hgwebdir_mod.py", line 61, in refresh
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] u =
ui.ui()
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\ui.py", line 38, in
__init__
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] for f
in util.rcpath():
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\demandimport.py", line
75, in __getattribute__
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24]
self._load()
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\demandimport.py", line
47, in _load
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] mod =
_origimport(head, globals, locals)
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\util.py", line 552, in
<module>
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] from
windows import *
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\demandimport.py", line
85, in _demandimport
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] return
_origimport(name, globals, locals, fromlist)
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\windows.py", line 21,
in <module>
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24]
posixfile.__doc__ = osutil.posixfile.__doc__
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\demandimport.py", line
75, in __getattribute__
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24]
self._load()
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\demandimport.py", line
47, in _load
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24] mod =
_origimport(head, globals, locals)
[Sat Jun 19 14:32:35 2010] [error] [client 192.168.178.24]
ImportError: DLL load failed: The specified module could not be found.
-------------

-- hw.wsgi --
import socket
import sys

# Now for the hello world application.
def application(environ, start_response):
status = '200 OK'
output = '\n'.join(sys.path)
response_headers = [('Content-type', 'text/plain'), ('Content-
Length', str(len(output)))]
start_response(status, response_headers)
return [output]
-------------

-- Output --
C:\Apps\Python26\Lib\site-packages\trac-0.12-py2.6-win32.egg
C:\Apps\Python26\Lib\site-packages\genshi-0.6-py2.6.egg
C:\Apps\Python26\lib\site-packages\trac-0.12-py2.6-win32.egg
C:\Apps\Python26\lib\site-packages\genshi-0.6-py2.6.egg
C:\WINDOWS\system32\python26.zip
C:\Apps\Python26\DLLs
C:\Apps\Python26\lib
C:\Apps\Python26\lib\plat-win
C:\Apps\Python26\lib\lib-tk
C:\Program Files\Apache2.2\bin
C:\Apps\Python26
C:\Apps\Python26\lib\site-packages
C:\Apps\Python26\lib\site-packages\win32
C:\Apps\Python26\lib\site-packages\win32\lib
C:\Apps\Python26\lib\site-packages\Pythonwin
-------------

-- hw.wsgi --
import socket
import sys
import psycopg2

# Now for the hello world application.
def application(environ, start_response):
status = '200 OK'
output = '\n'.join(sys.path)
response_headers = [('Content-type', 'text/plain'), ('Content-
Length', str(len(output)))]
start_response(status, response_headers)
return [output]
-------------

-- Apache error log --
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] mod_wsgi
(pid=2332): Exception occurred processing WSGI script 'D:/Home/web/
apache/cgi-bin/hw.wsgi'.
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] Traceback
(most recent call last):
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] File "D:/
Home/web/apache/cgi-bin/hw.wsgi", line 7, in application
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] db =
pg.connect(database = 'invest', user='invest', password='invest')
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\demandimport.py", line
75, in __getattribute__
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24]
self._load()
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\demandimport.py", line
47, in _load
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] mod =
_origimport(head, globals, locals)
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\psycopg2\\__init__.py", line 69,
in <module>
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] from
_psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] File "C:\
\Apps\\Python26\\lib\\site-packages\\mercurial\\demandimport.py", line
106, in _demandimport
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24] mod =
_origimport(name, globals, locals)
[Sat Jun 19 14:37:06 2010] [error] [client 192.168.178.24]
ImportError: DLL load failed: The specified module could not be found.
-------------

[Graham Dumpleton]

Apache runs as special service user and not you. Thus that user needs
to have appropriate access permissions to everything that is required.
If the DLLs are readable to that user, it will not work.

Another issue that comes up is where packages are installed for a
specific user and not for all users. This can result in registry
settings only being set for user who installs it and will be missing
in machine context and thus Apache user will not see them.

Graham

> --
> You received this message because you are subscribed to the Google Groups "modwsgi" group.
> To post to this group, send email to mod...@googlegroups.com.
> To unsubscribe from this group, send email to modwsgi+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
>
>

[Graham Dumpleton]

One more thing. As I understand it, DLLs can also be searched on user
exec executable search Path in Windows. If you are relying on that for
user account, the Apache user account will not likely have it set up.

Graham

[baloan]

Hi Graham,

thanks for the swift repsonse. Apache is run as user "web" which I can
confirm in task manager. The python console check which works was run
as user "web". Running Apache from an interactive prompt and trying
hw.wsgi with "import psycopg2" fails like when run as a service.

Both Apache from an interactive prompt and the python interpreter
(interactive console test) use the same
a) user
b) PATH
c) permission
with different results.

What else might be wrong?

--
Andreas

-------------
[SC] GetServiceConfig SUCCESS

SERVICE_NAME: apache2.2
TYPE : 10 WIN32_OWN_PROCESS
START_TYPE : 3 DEMAND_START
ERROR_CONTROL : 1 NORMAL
BINARY_PATH_NAME : "C:\Program Files\Apache2.2\bin
\httpd.exe" -k runservice
LOAD_ORDER_GROUP :
TAG : 0
DISPLAY_NAME : Apache2.2
DEPENDENCIES : Tcpip
: Afd
SERVICE_START_NAME : .\web
-------------

On Jun 19, 3:40 pm, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:

> One more thing. As I understand it, DLLs can also be searched on user
> exec executable search Path in Windows. If you are relying on that for
> user account, the Apache user account will not likely have it set up.
>
> Graham
>

> On 19 June 2010 23:06, Graham Dumpleton <graham.dumple...@gmail.com> wrote:
>
> > Apache runs as special service user and not you. Thus that user needs
> > to have appropriate access permissions to everything that is required.
> > If the DLLs are readable to that user, it will not work.
>
> > Another issue that comes up is where packages are installed for a
> > specific user and not for all users. This can result in registry
> > settings only being set for user who installs it and will be missing
> > in machine context and thus Apache user will not see them.
>
> > Graham
>

> > On 19 June 2010 22:57, baloan <balo...@googlemail.com> wrote:
> >> Hello,
>
> >> I have tried to setup apache with hg, trac and web2py running on
> >> Windows XP SP3 but failed. Below is a detailed list of components I
> >> have installed.
>
> >> hg, trac and web2py work fine standalone (tracd 8000, web2py -
> >> a'<recycle>') but when I try to run them behind Apache with WSGI I see
> >> ImportErrors when importing psycopg2 or mercurial specific modules.
> >> The installation seems to be fine because using PostgreSQL works from
> >> the command line (see below).
>
> >> I have then setup a 'Hello World' WSGI to prove importing psycopg2 is
> >> the problem. See below for sys.path and Apache log details.
>
> >> Who knows what is wrong and how to fix?
>
> >> Regards, Andreas

> >> balo...@gmail.com

[baloan]

How to solve ImportError: DLL load failed: The specified module could
not be found.

To analyse the problem first download http://www.dependencywalker.com
and check the unresolved DLL dependencies of your modules.

In my _psycopg.pyd (which can be opened with Dependency Walker) I can
see LIBPQ.DLL, LIBEAY32.DLL and MSVCR90.DLL being unresolved. I have
built/compiled psycopg2-2.2.1 on Windows XP SP3 with Visual Studio
2008.

The LIBPQ.DLL and LIBEAY32.DLL were easily resolved by adding ...
\PostgrSQL\8.4\bin and ...\PostgrSQL\8.4\lib to the PATH.

The MS VC runtime library MSVCR90.DLL is more difficult and I have not
solved the problem yet. The simple approach of copying MSVCR90.DLL to
C:\Windows\system32 leads to the following dialog box when requesting
a wsgi page that import additional modules:

-------------------------------------------------------------------------------
c:\Program Files\Apache2.2\bin

R6034
An application has made an attempt to load the C runtime library
incorrectly.
Please contact the application's support team for more information.
-------------------------------------------------------------------------------

Python native modules like os, sys seem to work, though. Modules like
hg or psycopg2 which contain compiled DLLs (or pyd) seem to fail.

Looking how python.exe depend on the runtime library I find: c:\windows
\winsxs\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.4148_x-
ww_d495ac4e\MSVCR90.DLL. This long path seems to be related with
WinSXS (side-by-side) which allows to operate multiple versions of a
DLL in one Windows installation.

Potentially the incompatibility is created during module compilation
when the MSVCR90.DLL is not resolved properly. Importing psycopg2 or
starting hg from the interactive prompt works fine.

Who can contribute to solve the issue?

Regards, Andreas
bal...@gmail.com

> ...
>
> read more »

[baloan]

To whom it may concern: "ImportError: DLL load failed" solved.

I finally got the python modules involved working with Apache and
mod_wsgi.

I took the following steps:
1. uninstall Microsoft Visual C++ 2008
2. install MinGW
3. Reboot

Mercurial
3. recompile Mercurial with MSVCR90 using MinGW (setup.py build -c
mingw32)
4. then setup.py install --skip-build
(compiling and running linked against MSVCRT did not work; for that I
did setup.py build -c mingw32 >build.log and manually created a batch
with changed -lmsvcr* directives)

Psycopg2
5. On another box I had a compiled version linked against MSVCRT.DLL
which worked on my box as well.
6. then setup.py install --skip-build

In all cases I could run the modules standalone from the interactive
python prompt. They only failed when used with Apache mod_wsgi.

The components now are linked against runtime libs as follows:

python 2.6 -> c:\windows\winsxs
\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.21022.8_x-
ww_d08d0375\MSVCR90.DLL
apache (httpd.exe) -> c:\windows\system32\MSVCRT.DLL
mod_wsgi -> c:\windows\winsxs
\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.21022.8_x-
ww_d08d0375\MSVCR90.DLL
psycopg2 -> c:\windows\system32\MSVCRT.DLL
mercurial -> MSVCR90.DLL (base85.pyd, no path in the dll)

I suspect there is a problem with the binaries linked against
different runtime libraries.

Graham, should I log this as a bug with mod_wsgi?

Regads, Andreas
--
bal...@gmail.com

On Jun 21, 11:42 pm, baloan <balo...@googlemail.com> wrote:
> How to solve ImportError: DLL load failed: The specified module could
> not be found.
>

> To analyse the problem first downloadhttp://www.dependencywalker.com

> balo...@gmail.com

> ...
>
> read more »

[Graham Dumpleton]

Doesn't strictly sound like anything I can do anything about as
relates to environment, in particular how code is compiled or what
DLLs are installed on the system and not the code itself. Thus, not
strictly a bug, albeit is something to be aware of.

As I am not a Windows person, if anyone, if above doesn't cover it
already, can explain with reasonable certainty what the issue may have
been, and the solution, in a way that people coming across this thread
in the future can understand, would be appreciated. That way this can
act as a record of it I can point people at.

Thanks.

Graham

[baloan]

ImportError: DLL load failed - How to compile psycopg2 to make it
cooperate with mod_wsgi.

1. install mingw
2. download psycopg2
3. setup.py build -c mingw32
4. replace "-lmsvcr90" with "-lmsvcrt" in build link step (see below)
5. execute link step again manually
6. setup.py install --skip-build
7. works for me ;-)

I don't know how to tweak distutils to programmatically change the
msvc runtime lib. This is my poor man's fix.

Regards, Andreas
bal...@gmail.com

old (fails with mod_wsgi)
c:\Apps\MinGW\bin\gcc.exe -mno-cygwin -shared -s build
\temp.win32-2.6\Release\psycopg\psycopgmodule.o build
\temp.win32-2.6\Release\psycopg\pqpath.o build\temp.win32-2.6\Release
\psycopg\typecast.o build\temp.win32-2.6\Release\psycopg
\microprotocols.o build\temp.win32-2.6\Release\psycopg
\microprotocols_proto.o build\temp.win32-2.6\Release\psycopg
\connection_type.o build\temp.win32-2.6\Release\psycopg
\connection_int.o build\temp.win32-2.6\Release\psycopg\cursor_type.o
build\temp.win32-2.6\Release\psycopg\cursor_int.o build
\temp.win32-2.6\Release\psycopg\lobject_type.o build
\temp.win32-2.6\Release\psycopg\lobject_int.o build
\temp.win32-2.6\Release\psycopg\adapter_qstring.o build
\temp.win32-2.6\Release\psycopg\adapter_pboolean.o build
\temp.win32-2.6\Release\psycopg\adapter_binary.o build
\temp.win32-2.6\Release\psycopg\adapter_asis.o build
\temp.win32-2.6\Release\psycopg\adapter_list.o build
\temp.win32-2.6\Release\psycopg\adapter_datetime.o build
\temp.win32-2.6\Release\psycopg\adapter_pfloat.o build
\temp.win32-2.6\Release\psycopg\adapter_pdecimal.o build
\temp.win32-2.6\Release\psycopg\green.o build\temp.win32-2.6\Release
\psycopg\utils.o build\temp.win32-2.6\Release\psycopg\_psycopg.def -LC:
\Apps\Python26\libs -LC:\Apps\Python26\PCbuild -LC:/PROGRA~1/
POSTGR~1/8.4/lib -lpython26 -lmsvcr90 -lpq -lws2_32 -ladvapi32 -o build
\lib.win32-2.6\psycopg2\_psycopg.pyd


new (works with mod_wsgi)
c:\Apps\MinGW\bin\gcc.exe -mno-cygwin -shared -s build
\temp.win32-2.6\Release\psycopg\psycopgmodule.o build
\temp.win32-2.6\Release\psycopg\pqpath.o build\temp.win32-2.6\Release
\psycopg\typecast.o build\temp.win32-2.6\Release\psycopg
\microprotocols.o build\temp.win32-2.6\Release\psycopg
\microprotocols_proto.o build\temp.win32-2.6\Release\psycopg
\connection_type.o build\temp.win32-2.6\Release\psycopg
\connection_int.o build\temp.win32-2.6\Release\psycopg\cursor_type.o
build\temp.win32-2.6\Release\psycopg\cursor_int.o build
\temp.win32-2.6\Release\psycopg\lobject_type.o build
\temp.win32-2.6\Release\psycopg\lobject_int.o build
\temp.win32-2.6\Release\psycopg\adapter_qstring.o build
\temp.win32-2.6\Release\psycopg\adapter_pboolean.o build
\temp.win32-2.6\Release\psycopg\adapter_binary.o build
\temp.win32-2.6\Release\psycopg\adapter_asis.o build
\temp.win32-2.6\Release\psycopg\adapter_list.o build
\temp.win32-2.6\Release\psycopg\adapter_datetime.o build
\temp.win32-2.6\Release\psycopg\adapter_pfloat.o build
\temp.win32-2.6\Release\psycopg\adapter_pdecimal.o build
\temp.win32-2.6\Release\psycopg\green.o build\temp.win32-2.6\Release
\psycopg\utils.o build\temp.win32-2.6\Release\psycopg\_psycopg.def -LC:
\Apps\Python26\libs -LC:\Apps\Python26\PCbuild -LC:/PROGRA~1/
POSTGR~1/8.4/lib -lpython26 -lmsvcrt -lpq -lws2_32 -ladvapi32 -o build
\lib.win32-2.6\psycopg2\_psycopg.pyd