Modwsgi compatibility issues with Django, MySQL 32 bit mode on OS X 10.6.6
peter-ctp
I'm trying to deploy mod_wsgi to run my Django application, using a
MySQL backend, on SnowLeopard (10.6.6).
As I have read up on my posts from elsewhere, I know that MySQL and
Python/Django all have to be compiled into 32 bit mode to work in
SnowLeopard, so I have done that, and all is working fine in my
development site, using the standard python/django manage.py runserver
command.
So, now, I want to deploy this into the Apache 2 server running on my
mac, to test out some more dependencies, before moving the whole lot
over to our production Linux machine (where, happily, these problems I
am now having will go away -- but I need to test things on this
machine first). So first thing to do was recompile Apache 2 in 32
bit, and get that started as the default webserver in my local
machine, including the mod_wsgi.so also compiled to 32 bits.
So now the fun starts. When I have apache call the mod_wsgi handler
(which I am using to run a django user authentication etc module) I
get errors which I do NOT get when runing the same application in the
development server.
First error: from the apache error log --
[Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] mod_wsgi
(pid=9128): Exception occurred processing WSGI script '/Peter stuff/
SDE/subscribers/django access control/anaaccess/apache/django.wsgi'.
[Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] Traceback
(most recent call last):
[Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/core/handlers/wsgi.py", line 230, in __call__
[Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100]
self.load_middleware()
[Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/core/handlers/base.py", line 42, in load_middleware
[Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] raise
exceptions.ImproperlyConfigured, 'Error importing middleware %s: "%s"'
% (mw_module, e)
[Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100]
ImproperlyConfigured: Error importing middleware request.middleware:
"No module named request.middleware"
OK, so I am using the django request middleware to log all user
accesses to a database so I can build nice traffic profiles etc. I
would like that, and don't understand why mod_wsgi can't find the
request stuff. The request package is right there in the same folder
along with everything else.
Second error. So I comment out the request stuff and try again. We
lose this error. But we get this one now:
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] mod_wsgi
(pid=9273): Exception occurred processing WSGI script '/Peter stuff/
SDE/subscribers/django access control/anaaccess/apache/django.wsgi'.
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] Traceback
(most recent call last):
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/core/handlers/wsgi.py", line 241, in __call__
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] response
= self.get_response(request)
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/core/handlers/base.py", line 73, in get_response
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] response
= middleware_method(request)
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/contrib/sessions/middleware.py", line 10, in
process_request
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] engine =
import_module(settings.SESSION_ENGINE)
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/utils/importlib.py", line 35, in import_module
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100]
__import__(name)
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/contrib/sessions/backends/db.py", line 2, in <module>
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] from
django.contrib.sessions.models import Session
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/contrib/sessions/models.py", line 4, in <module>
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] from
django.db import models
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/db/__init__.py", line 41, in <module>
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] backend
= load_backend(settings.DATABASE_ENGINE)
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/db/__init__.py", line 17, in load_backend
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] return
import_module('.base', 'django.db.backends.%s' % backend_name)
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/utils/importlib.py", line 35, in import_module
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100]
__import__(name)
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] File "/
Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
packages/django/db/backends/mysql/base.py", line 13, in <module>
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] raise
ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100]
ImproperlyConfigured: Error loading MySQLdb module: dlopen(/usr/local/
python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.3-fat.egg-tmp/
_mysql.so, 2): Library not loaded: libmysqlclient.16.dylib
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] Referenced
from: /usr/local/python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.3-
fat.egg-tmp/_mysql.so
[Fri Mar 18 08:58:29 2011] [error] [client 192.168.1.100] Reason:
image not found
Ok, the main one at the bottom seems to be that it cannot find the
libmysqlclient.16.dylib needed by the 32 bit mysql application. I had
this when compiling for the development version, and fixed it by
adding export DYLD_LIBRARY_PATH=/usr/local/mysql/lib/ to my bash
profile. But this seems to be ignored by mod_wsgi. Adding this to
the django.wsgi file
os.environ['DYLD_LIBRARY_PATH'] = '/usr/local/mysql/lib'
also does not work.
Help!
Peter Robinson
Graham Dumpleton
> Dear everyone
> I'm trying to deploy mod_wsgi to run my Django application, using a
> MySQL backend, on SnowLeopard (10.6.6).
> As I have read up on my posts from elsewhere, I know that MySQL and
> Python/Django all have to be compiled into 32 bit mode to work in
> SnowLeopard, so I have done that, and all is working fine in my
> development site, using the standard python/django manage.py runserver
> command.
Actually, you generally want everything to be 64 bit and not 32 bit if you can.
> So, now, I want to deploy this into the Apache 2 server running on my
> mac, to test out some more dependencies, before moving the whole lot
> over to our production Linux machine (where, happily, these problems I
> am now having will go away -- but I need to test things on this
> machine first). So first thing to do was recompile Apache 2 in 32
> bit, and get that started as the default webserver in my local
> machine, including the mod_wsgi.so also compiled to 32 bits.
How are you ensuring that Apache was compile ONLY as 32 bit and not a
full fat binary. The compiler by default will make it fat and if you
have 64 bit capable CPU and its is fat binary, MacOSX will run it as
64 bit and not 32 bit.
Please confirm whether you are using Apple supplied Apache or your own
compiled Apache. Provide the output of running the file command on it,
showing the full path to where Apache httpd binary is installed that
you are using. For example, if using Apache supplied by Apple you
would see.
$ file /usr/sbin/httpd
/usr/sbin/httpd: Mach-O universal binary with 3 architectures
/usr/sbin/httpd (for architecture x86_64): Mach-O 64-bit executable x86_64
/usr/sbin/httpd (for architecture i386): Mach-O executable i386
/usr/sbin/httpd (for architecture ppc7400): Mach-O executable ppc
Also go into the Activity Monitor application for MacOS X and look at
the Kind column for Apache and see whether it says 'Intel' or 'Intel
(64 bit)'.
> So now the fun starts. When I have apache call the mod_wsgi handler
> (which I am using to run a django user authentication etc module) I
> get errors which I do NOT get when runing the same application in the
> development server.
>
> First error: from the apache error log --
> [Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] mod_wsgi
> (pid=9128): Exception occurred processing WSGI script '/Peter stuff/
> SDE/subscribers/django access control/anaaccess/apache/django.wsgi'.
> [Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] Traceback
> (most recent call last):
> [Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] File "/
> Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
> packages/django/core/handlers/wsgi.py", line 230, in __call__
> [Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100]
> self.load_middleware()
> [Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] File "/
> Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-
> packages/django/core/handlers/base.py", line 42, in load_middleware
> [Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100] raise
> exceptions.ImproperlyConfigured, 'Error importing middleware %s: "%s"'
> % (mw_module, e)
> [Fri Mar 18 08:14:05 2011] [error] [client 192.168.1.100]
> ImproperlyConfigured: Error importing middleware request.middleware:
> "No module named request.middleware"
That is a problem with how you have set sys.path in WSGI script file
or a permissions problem where by user that Apache runs as can't read
the location where package is installed. What is the contents of your
WSGI script file? What are all the permissions on the package
directory and the parent directory of it.
What do you get for:
file /usr/local/python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.3-fat.egg-tmp/_mysql.so
What do you get for:
otool -L /usr/local/python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.3-fat.egg-tmp/_mysql.so
and:
DYLD_LIBRARY_PATH="" otool -L
/usr/local/python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.3-fat.egg-tmp/_mysql.so
For the libmysqlclient.16.dylib it finds, what do you get for:
file .....somepath/libmysqlclient.16.dylib
> Ok, the main one at the bottom seems to be that it cannot find the
> libmysqlclient.16.dylib needed by the 32 bit mysql application.
Not necessarily. Believe that can mean that the a .so or dynamic
library doesn't include required architecture.
> I had
> this when compiling for the development version, and fixed it by
> adding export DYLD_LIBRARY_PATH=/usr/local/mysql/lib/ to my bash
> profile. But this seems to be ignored by mod_wsgi. Adding this to
> the django.wsgi file
> os.environ['DYLD_LIBRARY_PATH'] = '/usr/local/mysql/lib'
> also does not work.
Normally Apache is started as root and so it will not inherit stuff in
your personal account setup. Setting it in the WSGI script file is too
late as has to be set before process starts.
The checks I told you to run above should show what is what and where
things are being found or not. Once can see that can suggest best way
of solving problem.
Graham