DLL load failed?

[Nicholas Riley]

I'm trying to configure IIS on Windows Server 2008 R2 to serve
Mercurial repositories out of the site root. For some reason, the
instructions I found did not cause Mercurial to be served; instead, it
just served the sample WSGI handler. When I updated hgwebdir_wsgi.py
with some of the sample code, I think it worked, but now I get this
error:

Internal Extension Error: Failed to import callback module
'hgwebdir_wsgi'
Last Windows error: The specified module could not be
found.
Traceback (most recent call
last):
File "C:\Python26\lib\site-packages\mercurial\demandimport.py", line
106, in _demandimport
return _origimport(name, globals, locals, fromlist, level)
File "C:\repositories\hgwebdir_wsgi.py", line 32, in
<module>
application =
hgwebdir(hgweb_config)
File "C:\Python26\lib\site-packages\mercurial\hgweb
\hgwebdir_mod.py", line 52, in
__init__

self.refresh()
File "C:\Python26\lib\site-packages\mercurial\hgweb
\hgwebdir_mod.py", line 61,
in
refresh
u =
ui.ui()
File "C:\Python26\lib\site-packages\mercurial\ui.py", line 38, in
__init__
for f in
util.rcpath():
File "C:\Python26\lib\site-packages\mercurial\demandimport.py", line
77, in
__getattribute__

self._load()
File "C:\Python26\lib\site-packages\mercurial\demandimport.py", line
49, in
_load
mod = _origimport(head, globals,
locals)
File "C:\Python26\lib\site-packages\mercurial\util.py", line 562, in
<module>
from windows import
*
File "C:\Python26\lib\site-packages\mercurial\demandimport.py", line
87, in
_demandimport
return _origimport(name, globals, locals,
fromlist)
File "C:\Python26\lib\site-packages\mercurial\windows.py", line 21,
in
<module>
posixfile.__doc__ =
osutil.posixfile.__doc__
File "C:\Python26\lib\site-packages\mercurial\demandimport.py", line
77, in
__getattribute__

self._load()
File "C:\Python26\lib\site-packages\mercurial\demandimport.py", line
49, in
_load
mod = _origimport(head, globals,
locals)
ImportError: DLL load failed: The specified module could not be
found.

The DLL it's trying to import is osutil.pyd, which is in C:
\Python26\Lib\site-packages\mercurial\osutil.pyd.

Any ideas why it can't find this directory?

It would also be useful if the need to install the IIS 6 compatibility
role (not just WMI but Metabase compatibility) for IIS 7 would be
mentioned in the docs, and perhaps the exception caught and a better
message displayed - I found it on this mailing list but nowhere else.

In case it helps, here's my complete hgwebdir_wsgi.py, which is
installed in c:\repositories next to _hgwebdir_wsgi.dll.

# Configuration file location
hgweb_config = r'c:\repositories\hgweb.config'

# Global settings for IIS path translation
path_strip = 0 # Strip this many path elements off (when using url
rewrite)
path_prefix = 0 # This many path elements are prefixes (depends on
the
# virtual path of the IIS application).

import sys

# Enable tracing. Run 'python -m win32traceutil' to debug
if hasattr(sys, 'isapidllhandle'):
import win32traceutil

# To serve pages in local charset instead of UTF-8, remove the two
lines below
import os
os.environ['HGENCODING'] = 'UTF-8'


import isapi_wsgi
from mercurial import demandimport; demandimport.enable()
from mercurial.hgweb.hgwebdir_mod import hgwebdir

# Example tweak: Replace isapi_wsgi's handler to provide better error
message
# Other stuff could also be done here, like logging errors etc.
class WsgiHandler(isapi_wsgi.IsapiWsgiHandler):
error_status = '500 Internal Server Error' # less silly error
message

isapi_wsgi.IsapiWsgiHandler = WsgiHandler

# Only create the hgwebdir instance once
application = hgwebdir(hgweb_config)

def handler(environ, start_response):

# Translate IIS's weird URLs
url = environ['SCRIPT_NAME'] + environ['PATH_INFO']
paths = url[1:].split('/')[path_strip:]
script_name = '/' + '/'.join(paths[:path_prefix])
path_info = '/'.join(paths[path_prefix:])
if path_info:
path_info = '/' + path_info
environ['SCRIPT_NAME'] = script_name
environ['PATH_INFO'] = path_info

return application(environ, start_response)

def __ExtensionFactory__():
return isapi_wsgi.ISAPISimpleHandler(handler)

if __name__=='__main__':
from isapi.install import *
params = ISAPIParameters()
sm = [ScriptMapParams(Extension='*', Flags=0)]
vd = VirtualDirParameters(Name='/', Description='Mercurial',
ScriptMaps=sm,
ScriptMapUpdate='replace')
params.VirtualDirs = [vd]
HandleCommandLine(params)

Thanks.

[Mark Rees]

Hi,

Can you confirm that the python c extension osutil.pyd is in the
mercurial directory? There have been instances when users have tried
to get mercurial running under Windows as an apache cgi script where
they get the same message import fail and it appears to be related to
how they installed mercurial. See:

http://stackoverflow.com/questions/644322/how-do-i-get-mercurials-hgwebdir-working-on-windows

Mark

[Mark Hammond]

On 10/06/2010 6:28 AM, Nicholas Riley wrote:
> ImportError: DLL load failed: The specified module could not be
> found.
>
> The DLL it's trying to import is osutil.pyd, which is in C:
> \Python26\Lib\site-packages\mercurial\osutil.pyd.
>
> Any ideas why it can't find this directory?

This implies Windows found the .pyd, but failed to locate a DLL this
.pyd depends on. It seems possible you are failing to load the MS CRT,
but you may like to run the 'depends' tool over that .pyd to check if
any other DLLs are referenced which don't exist, or to try and get some
clue why they can't be found when loaded by the user running IIS (eg, if
you rely on a PATH being setup for your current user, that will probably
fail when executed by IIS)

Cheers,

Mark

[Preston Landers]

Yeah... I had problems loading PyODBC under IIS with Python 2.6.5 due to a change in how Python compiles C extensions on Windows.  It wasn't declaring a dependency on MSVC9 runtime inside the DLL.  The extension (PyODBC) worked fine from the command line, but not under IIS / isapi_wsgi.

I think it was related to these issues:

http://bugs.python.org/issue4120
http://bugs.python.org/issue7833

I solved it by commenting out a section of distutils\msvc9compiler.py from lines 658 to 676.    I can copy my version to you if you need it.  I think ultimately it needs to be fixed upstream in Python.

Also make sure that if you are running a 64 bit IIS, that your Python and all its extensions are compiled 64 bit.

Good luck,
Preston

--
You received this message because you are subscribed to the Google Groups "isapi_wsgi-dev" group.
To post to this group, send email to isapi_w...@googlegroups.com.
To unsubscribe from this group, send email to isapi_wsgi-de...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/isapi_wsgi-dev?hl=en.

[Mark Hammond]

On 10/06/2010 9:55 AM, Preston Landers wrote:
> Yeah... I had problems loading PyODBC under IIS with Python 2.6.5 due to
> a change in how Python compiles C extensions on Windows. It wasn't
> declaring a dependency on MSVC9 runtime inside the DLL. The extension
> (PyODBC) worked fine from the command line, but not under IIS / isapi_wsgi.
>
> I think it was related to these issues:
>
> http://bugs.python.org/issue4120
> http://bugs.python.org/issue7833
>
> I solved it by commenting out a section of distutils\msvc9compiler.py
> from lines 658 to 676. I can copy my version to you if you need it.
> I think ultimately it needs to be fixed upstream in Python.

I can imagine this would be the case if the CRT DLLs were installed in
the Python directory; that relies on the fact python.exe is the 'host'
executable, which clearly is not true in an IIS environment. I'd
*expect* things to work better if you used the stand-alone "vs 2008
redistributables" installer, which should in theory install them
globally where they will be found regardless of configuration. Sadly, I
can't recall how Python 2.6 installs these DLLs...

Cheers,

Mark

[Preston Landers]

On Wed, Jun 9, 2010 at 6:59 PM, Mark Hammond <mham...@skippinet.com.au> wrote:

I can imagine this would be the case if the CRT DLLs were installed in the Python directory; that relies on the fact python.exe is the 'host' executable, which clearly is not true in an IIS environment.  I'd *expect* things to work better if you used the stand-alone "vs 2008 redistributables" installer, which should in theory install them globally where they will be found regardless of configuration.  Sadly, I can't recall how Python 2.6 installs these DLLs...

To the best of my knowledge the CRT DLLs used in my case were in the windows sxs directory and we are using the redistributable version of MSVCR9 included with Installshield 2009.

The problem seemed to be that the module dll (pyodbc.pyd) did not declare the dependency in its internal manifest.  The section of msvc9compiler.py that I commented out was intentionally removing that dependency, with the following comment as explanation:

# Remove references to the Visual C runtime, so they will fall through to the
# Visual C dependency of Python.exe.  This way, when installed for a
# restricted user (e.g.  runtimes are not in WinSxS folder, but in Python's
# own folder), the runtimes do not need to be in every folder with .pyd's.

Once I removed that bit of code from msvc9compiler, I could see the CRT dependency listed in the extension pyd when I opened it in a hex editor, and everything worked under IIS.  Nor did I have to copy the CRT DLLs anywhere.  However I'm not sure I understand the restricted user scenario the comment mentions.  My Python app runs as the IIS application pool user identity, but that's probably not the kind of thing the comment is referring to.

regards,
-Preston

[Mark Hammond]

On 10/06/2010 10:45 AM, Preston Landers wrote:
> On Wed, Jun 9, 2010 at 6:59 PM, Mark Hammond <mham...@skippinet.com.au
> <mailto:mham...@skippinet.com.au>> wrote:
>
>
> I can imagine this would be the case if the CRT DLLs were installed
> in the Python directory; that relies on the fact python.exe is the
> 'host' executable, which clearly is not true in an IIS environment.
> I'd *expect* things to work better if you used the stand-alone "vs
> 2008 redistributables" installer, which should in theory install
> them globally where they will be found regardless of configuration.
> Sadly, I can't recall how Python 2.6 installs these DLLs...
>
>
> To the best of my knowledge the CRT DLLs used in my case were in the
> windows sxs directory and we are using the redistributable version of
> MSVCR9 included with Installshield 2009.
>
> The problem seemed to be that the module dll (pyodbc.pyd) did not
> declare the dependency in its internal manifest. The section of
> msvc9compiler.py that I commented out was intentionally removing that
> dependency, with the following comment as explanation:

Right - this is probably another example of
http://bugs.python.org/issue7833.

*sob* :)

Mark

[Nicholas Riley]

On Jun 9, 2010, at 7:47 PM, Mark Hammond wrote:
> This implies Windows found the .pyd, but failed to locate a DLL this .pyd depends on. It seems possible you are failing to load the MS CRT, but you may like to run the 'depends' tool over that .pyd to check if any other DLLs are referenced which don't exist, or to try and get some clue why they can't be found when loaded by the user running IIS (eg, if you rely on a PATH being setup for your current user, that will probably fail when executed by IIS)

Thanks, I had no idea that was what the error message meant.

I ran depends.exe against osutil.pyd and found three unresolved dependencies:

msvcr90.dll
gpsvc.dll
ieshims.dll

The last two are set up by IIS, so you're right - it is the MS CRT.

I built Mercurial myself (because that's what the instructions in hgwebdir_wsgi.py instructed), using Visual Studio 2008 Express. I looked at osutil.pyd in a binary distribution available from the Mercurial site and it had no problems linking to msvcr90.dll; when I replaced my version with this version, IIS worked just fine.

So, it seems the problem was with the way I built Mercurial (python setup.py install). How can I cause distutils to build the .pyd such that I don't have this dependency problem when running under IIS?

--
Nicholas Riley <njr...@illinois.edu>

[Preston Landers]

On Thu, Jun 10, 2010 at 8:59 AM, Nicholas Riley <njr...@illinois.edu> wrote:

So, it seems the problem was with the way I built Mercurial (python setup.py install).  How can I cause distutils to build the .pyd such that I don't have this dependency problem when running under IIS?

 
Did you see my response about msvc9compiler.py?   If you make the change I mentioned to that and recompile, it should work.

Attached is my modified copy of msvc9compiler.py for Python 2.6.5.   It ought to be fixed upstream in Python at some point:  http://bugs.python.org/issue7833

regards,
Preston