mod_wsgi cannot load MySQLdb

[Jim]

Hello folks,

This probably has been discussed many times, but I still can't find any solution yet. Basically, it turns out that python can load MySQLdb just fine, but Apache can't load MySQLdb from the wsgi script.

Here is the configuration of my machine.

Mac OS X Snow Leopard 10.6.8

Apache2 (shipped with Snow Leopard)

Python 2.7 (from python.org)

virtualenv

MySQL-python 1.2.3

BTW, I can import MySQLdb with no problem under python interpreter.

I am using python2.7 in a virtual environment created by virtualenv. Here is the error info extracted from the Apache error log. To make the lines shorter, I deleted all the time tags in the brackets.

Any ideas how to fix it?

[] mod_wsgi (pid=3136): Target WSGI script '/Users/jianbao/projects/tao.com/mysite/apache/django.wsgi' cannot be loaded as Python module.

[] mod_wsgi (pid=3136): Exception occurred processing WSGI script '/Users/jianbao/projects/tao.com/mysite/apache/django.wsgi'.

[] Traceback (most recent call last):

[]   File "/Users/jianbao/projects/tao.com/mysite/apache/django.wsgi", line 56, in <module>

[]     import MySQLdb

[]   File "/Users/jianbao/projects/tao.com/mysite/lib/python2.7/site-packages/MySQLdb/__init__.py", line 19, in <module>

[]     import _mysql

[] ImportError: dlopen(/Users/jianbao/projects/tao.com/mysite/lib/python2.7/site-packages/_mysql.so, 2): Library not loaded: libmysqlclient.18.dylib

[]   Referenced from: /Users/jianbao/projects/tao.com/mysite/lib/python2.7/site-packages/_mysql.so

[]   Reason: image not found

[Graham Dumpleton]

On 24 August 2011 08:58, Jim <jianb...@gmail.com> wrote:
> Hello folks,
> This probably has been discussed many times, but I still can't find any
> solution yet. Basically, it turns out that python can load MySQLdb just
> fine, but Apache can't load MySQLdb from the wsgi script.
> Here is the configuration of my machine.
> Mac OS X Snow Leopard 10.6.8
> Apache2 (shipped with Snow Leopard)

Which is fat and by default will run as 64 bit.

> Python 2.7 (from python.org)

Which I believe now comes as fat binary so should be okay.

> virtualenv
> MySQL-python 1.2.3

This is where problems start.

Because Apache is going to run as 64 bit, and because MySQL client
libraries are not fat but for a specific architecture, you need to
ensure you installed 64 bit MySQL and not 32 but version.

For example, on my system I have:

mysql-5.1.53-osx10.6-x86_64

and get:

$ file /usr/local/mysql/lib/libmysqlclient.16.dylib
/usr/local/mysql/lib/libmysqlclient.16.dylib: Mach-O 64-bit
dynamically linked shared library x86_64

I suspect you will find that yours is only 32 bit and not 64 bit. That
is why you would get error:

Reason: image not found

Can you run the 'file' command on the dylib and see what it says.

Graham

> BTW, I can import MySQLdb with no problem under python interpreter.
> I am using python2.7 in a virtual environment created by virtualenv. Here is
> the error info extracted from the Apache error log. To make the lines
> shorter, I deleted all the time tags in the brackets.
> Any ideas how to fix it?
> [] mod_wsgi (pid=3136): Target WSGI script
> '/Users/jianbao/projects/tao.com/mysite/apache/django.wsgi' cannot be loaded
> as Python module.
> [] mod_wsgi (pid=3136): Exception occurred processing WSGI script
> '/Users/jianbao/projects/tao.com/mysite/apache/django.wsgi'.
> [] Traceback (most recent call last):
> []   File "/Users/jianbao/projects/tao.com/mysite/apache/django.wsgi", line
> 56, in <module>
> []     import MySQLdb
> []   File
> "/Users/jianbao/projects/tao.com/mysite/lib/python2.7/site-packages/MySQLdb/__init__.py",
> line 19, in <module>
> []     import _mysql
> [] ImportError:
> dlopen(/Users/jianbao/projects/tao.com/mysite/lib/python2.7/site-packages/_mysql.so,
> 2): Library not loaded: libmysqlclient.18.dylib
> []   Referenced from:
> /Users/jianbao/projects/tao.com/mysite/lib/python2.7/site-packages/_mysql.so
> []   Reason: image not found
>

> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/modwsgi/-/_S2GuYUGqosJ.
> To post to this group, send email to mod...@googlegroups.com.
> To unsubscribe from this group, send email to
> modwsgi+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/modwsgi?hl=en.
>

[Jianbao Tao]

Thank you so much, Graham. You may have pointed me to the right direction.

So, the following is what I get, where I basically filed all libmysqlclient* files under /usr/local/mysql/lib/. It seems the mysql I installed is for i386 architecture, but all the libmysqlclient files seem for x86_64, which is really confusing. Anyway, do you have any ideas of what's happening here?

>$ mysql --version

mysql  Ver 14.14 Distrib 5.5.14, for osx10.6 (i386) using readline 5.1

>$ ls /usr/local/mysql/lib/libmysqlclient*

/usr/local/mysql/lib/libmysqlclient.18.dylib   /usr/local/mysql/lib/libmysqlclient_r.18.dylib

/usr/local/mysql/lib/libmysqlclient.a          /usr/local/mysql/lib/libmysqlclient_r.a

/usr/local/mysql/lib/libmysqlclient.dylib      /usr/local/mysql/lib/libmysqlclient_r.dylib

>$ file /usr/local/mysql/lib/libmysqlclient.18.dylib 

/usr/local/mysql/lib/libmysqlclient.18.dylib: Mach-O 64-bit dynamically linked shared library x86_64

>$ file /usr/local/mysql/lib/libmysqlclient.dylib 

/usr/local/mysql/lib/libmysqlclient.dylib: Mach-O 64-bit dynamically linked shared library x86_64

>$ file /usr/local/mysql/lib/libmysqlclient.a

/usr/local/mysql/lib/libmysqlclient.a: current ar archive random library

>$ file /usr/local/mysql/lib/libmysqlclient_r.18.dylib 

/usr/local/mysql/lib/libmysqlclient_r.18.dylib: Mach-O 64-bit dynamically linked shared library x86_64

>$ file /usr/local/mysql/lib/libmysqlclient_r.dylib 

/usr/local/mysql/lib/libmysqlclient_r.dylib: Mach-O 64-bit dynamically linked shared library x86_64

>$ file /usr/local/mysql/lib/libmysqlclient_r.a

/usr/local/mysql/lib/libmysqlclient_r.a: current ar archive random library

[Graham Dumpleton]

What do you get if you run 'file' on _mysql.so extension module file in Python installation? Also run 'otool -L _mysql.so' as well to see what library it is finding. You have 64 bit libraries so should be okay unless you have 32 bit somewhere else and it is finding that instead.

Graham

> To unsubscribe from this group, send email to modwsgi+u...@googlegroups.com <modwsgi%2Bunsu...@googlegroups.com>.

[Jianbao Tao]

Graham,

This is what I got.

>$ file ./lib/python2.7/site-packages/_mysql.so

./lib/python2.7/site-packages/_mysql.so: Mach-O 64-bit bundle x86_64

>$ otool -L ./lib/python2.7/site-packages/_mysql.so

./lib/python2.7/site-packages/_mysql.so:

libmysqlclient.18.dylib (compatibility version 18.0.0, current version 18.0.0)

/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)

So, it seems _mysql.so is already for x86_64 architecture. But there is a inconsistency about libSystem.B.dylib according the versions reported by otool. But I am not sure if that dylib is ever used by MySQLdb or not.

Any thoughts?

Jim

To unsubscribe from this group, send email to modwsgi+u...@googlegroups.com.

[Graham Dumpleton]

On 24 August 2011 12:31, Jianbao Tao <jianb...@gmail.com> wrote:
> Graham,
> This is what I got.
>>$ file ./lib/python2.7/site-packages/_mysql.so
> ./lib/python2.7/site-packages/_mysql.so: Mach-O 64-bit bundle x86_64
>>$ otool -L ./lib/python2.7/site-packages/_mysql.so
> ./lib/python2.7/site-packages/_mysql.so:
> libmysqlclient.18.dylib (compatibility version 18.0.0, current version
> 18.0.0)

Can you change working directory and run this again with full path
name to _mysql.so.

It says libmysqlclient.18.dylib but missing full path context to know
for sure where that is coming from.

> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
> 125.2.0)
> So, it seems _mysql.so is already for x86_64 architecture. But there is a
> inconsistency about libSystem.B.dylib according the versions reported by
> otool. But I am not sure if that dylib is ever used by MySQLdb or not.

No so worried about the system library versioning. Probably normal.

Graham

[Jianbao Tao]

Sure. Here is what I got.

>$ file /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so 

/Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so: Mach-O 64-bit bundle x86_64

>$ otool -L /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so 

/Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so:

libmysqlclient.18.dylib (compatibility version 18.0.0, current version 18.0.0)

/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)

I don't see why full path would make any difference, though. 

Any thoughts?

Jim

[Graham Dumpleton]

On 24 August 2011 13:55, Jianbao Tao <jianb...@gmail.com> wrote:
> Sure. Here is what I got.
>>$ file
>> /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so
> /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so:
> Mach-O 64-bit bundle x86_64
>>$ otool -L
>> /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so
> /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so:
> libmysqlclient.18.dylib (compatibility version 18.0.0, current version
> 18.0.0)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
> 125.2.0)
> I don't see why full path would make any difference, though.
> Any thoughts?

The otool output should show a full path name for
'libmysqlclient.18.dylib' else it is indicating that it didn't know
where to find it. This worries me a bit as if otool doesn't know where
it is coming from then under Apache it will not either. Ie., could not
resolve it and thus why image not found.

This may be the actual problem. Ie., Apache doesn't know where to find
the library at run time and not the 32 bit vs 64 bit issue.

Try running:

DYLD_LIBRARY_PATH=/usr/local/mysql/lib otool -L
/Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so

What do you get for that?

Graham

[Jianbao Tao]

Alright. Here is what I got.

>$ DYLD_LIBRARY_PATH=/usr/local/mysql/lib otool -L /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so

/Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so:

libmysqlclient.18.dylib (compatibility version 18.0.0, current version 18.0.0)

/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)

Not so much of difference, though. Any ideas?

[Graham Dumpleton]

When I do it I get:

$ otool -L _mysql.so
_mysql.so:
/usr/local/mysql/lib/libmysqlclient_r.16.dylib (compatibility version
17.0.0, current version 17.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.3)

/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current

version 125.2.10)

See what I mean about having full path. That way I know for sure that
is being able to be found.

Without the full path, says that doesn't actually know where to find it.

What happens if you do:

cd /usr/local/mysql/lib

otool -L /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so

I can't remember if the current directory is looked at.

The try:

DYLD_FALLBACK_LIBRARY_PATH=/usr/local/mysql/lib otool -L
/Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so

but run it from outside of the lib directory.

Finally you might also try:

otool -l /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so

That is lower case 'L' instead of upper case.

What do you get for load commands related to libraries in the output.

Load command 7
cmd LC_LOAD_DYLIB
cmdsize 72
name /usr/local/mysql/lib/libmysqlclient_r.16.dylib (offset 24)
time stamp 2 Thu Jan 1 10:00:02 1970
current version 17.0.0
compatibility version 17.0.0
Load command 8
cmd LC_LOAD_DYLIB
cmdsize 48
name /usr/lib/libz.1.dylib (offset 24)
time stamp 2 Thu Jan 1 10:00:02 1970
current version 1.2.3
compatibility version 1.0.0
Load command 9
cmd LC_LOAD_DYLIB
cmdsize 56
name /usr/lib/libSystem.B.dylib (offset 24)
time stamp 2 Thu Jan 1 10:00:02 1970
current version 125.2.10
compatibility version 1.0.0

Graham

[Jianbao Tao]

I tried all the 'otool -L' suggestions, but nothing stand out compared to previous results.

Here is what I got from 'otool -l', which I have no clue what these lines mean:

>$ otool -l  /Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so

/Users/jianbao/projects/tao.com/djangoSite/lib/python2.7/site-packages/_mysql.so:

Load command 0

      cmd LC_SEGMENT_64

  cmdsize 632

  segname __TEXT

   vmaddr 0x0000000000000000

   vmsize 0x0000000000006000

  fileoff 0

 filesize 24576

  maxprot 0x00000007

 initprot 0x00000005

   nsects 7

    flags 0x0

Section

  sectname __text

   segname __TEXT

      addr 0x0000000000000860

      size 0x00000000000037c8

    offset 2144

     align 2^0 (1)

    reloff 0

    nreloc 0

     flags 0x80000400

 reserved1 0

 reserved2 0

Section

  sectname __stubs

   segname __TEXT

      addr 0x0000000000004028

      size 0x0000000000000258

    offset 16424

     align 2^1 (2)

    reloff 0

    nreloc 0

     flags 0x80000408

 reserved1 0 (index into indirect symbol table)

 reserved2 6 (size of stubs)

Section

  sectname __stub_helper

   segname __TEXT

      addr 0x0000000000004280

      size 0x00000000000003f8

    offset 17024

     align 2^2 (4)

    reloff 0

    nreloc 0

     flags 0x80000400

 reserved1 0

 reserved2 0

Section

  sectname __cstring

   segname __TEXT

      addr 0x0000000000004678

      size 0x0000000000000854

    offset 18040

     align 2^3 (8)

    reloff 0

    nreloc 0

     flags 0x00000002

 reserved1 0

 reserved2 0

Section

  sectname __const

   segname __TEXT

      addr 0x0000000000004ecc

      size 0x0000000000000100

    offset 20172

     align 2^0 (1)

    reloff 0

    nreloc 0

     flags 0x00000000

 reserved1 0

 reserved2 0

Section

  sectname __unwind_info

   segname __TEXT

      addr 0x0000000000004fcc

      size 0x00000000000000f8

    offset 20428

     align 2^0 (1)

    reloff 0

    nreloc 0

     flags 0x00000000

 reserved1 0

 reserved2 0

Section

  sectname __eh_frame

   segname __TEXT

      addr 0x00000000000050c8

      size 0x0000000000000f38

    offset 20680

     align 2^3 (8)

    reloff 0

    nreloc 0

     flags 0x00000000

 reserved1 0

 reserved2 0

Load command 1

      cmd LC_SEGMENT_64

  cmdsize 552

  segname __DATA

   vmaddr 0x0000000000006000

   vmsize 0x0000000000004000

  fileoff 24576

 filesize 16384

  maxprot 0x00000007

 initprot 0x00000003

   nsects 6

    flags 0x0

Section

  sectname __got

   segname __DATA

      addr 0x0000000000006000

      size 0x0000000000000060

    offset 24576

     align 2^3 (8)

    reloff 0

    nreloc 0

     flags 0x00000006

 reserved1 100 (index into indirect symbol table)

 reserved2 0

Section

  sectname __nl_symbol_ptr

   segname __DATA

      addr 0x0000000000006060

      size 0x0000000000000010

    offset 24672

     align 2^3 (8)

    reloff 0

    nreloc 0

     flags 0x00000006

 reserved1 112 (index into indirect symbol table)

 reserved2 0

Section

  sectname __la_symbol_ptr

   segname __DATA

      addr 0x0000000000006070

      size 0x0000000000000320

    offset 24688

     align 2^3 (8)

    reloff 0

    nreloc 0

     flags 0x00000007

 reserved1 114 (index into indirect symbol table)

 reserved2 0

Section

  sectname __data

   segname __DATA

      addr 0x00000000000063a0

      size 0x00000000000037bf

    offset 25504

     align 2^5 (32)

    reloff 0

    nreloc 0

     flags 0x00000000

 reserved1 0

 reserved2 0

Section

  sectname __const

   segname __DATA

      addr 0x0000000000009b60

      size 0x0000000000000018

    offset 39776

     align 2^4 (16)

    reloff 0

    nreloc 0

     flags 0x00000000

 reserved1 0

 reserved2 0

Section

  sectname __bss

   segname __DATA

      addr 0x0000000000009b78

      size 0x0000000000000068

    offset 0

     align 2^3 (8)

    reloff 0

    nreloc 0

     flags 0x00000001

 reserved1 0

 reserved2 0

Load command 2

      cmd LC_SEGMENT_64

  cmdsize 72

  segname __LINKEDIT

   vmaddr 0x000000000000a000

   vmsize 0x0000000000007000

  fileoff 40960

 filesize 25800

  maxprot 0x00000007

 initprot 0x00000001

   nsects 0

    flags 0x0

Load command 3

            cmd LC_DYLD_INFO_ONLY

        cmdsize 48

     rebase_off 40960

    rebase_size 184

       bind_off 41144

      bind_size 264

  weak_bind_off 0

 weak_bind_size 0

  lazy_bind_off 41408

 lazy_bind_size 2496

     export_off 43904

    export_size 128

Load command 4

     cmd LC_SYMTAB

 cmdsize 24

  symoff 44032

   nsyms 638

  stroff 55096

 strsize 11664

Load command 5

            cmd LC_DYSYMTAB

        cmdsize 80

      ilocalsym 0

      nlocalsym 520

     iextdefsym 520

     nextdefsym 5

      iundefsym 525

      nundefsym 113

         tocoff 0

           ntoc 0

      modtaboff 0

        nmodtab 0

   extrefsymoff 0

    nextrefsyms 0

 indirectsymoff 54240

  nindirectsyms 214

      extreloff 0

        nextrel 0

      locreloff 0

        nlocrel 0

Load command 6

     cmd LC_UUID

 cmdsize 24

    uuid 2239CFAD-CD6E-35D1-90C4-6B95C7A944A0

Load command 7

          cmd LC_LOAD_DYLIB

      cmdsize 48

         name libmysqlclient.18.dylib (offset 24)

   time stamp 2 Wed Dec 31 17:00:02 1969

      current version 18.0.0

compatibility version 18.0.0

Load command 8

          cmd LC_LOAD_DYLIB

      cmdsize 56

         name /usr/lib/libSystem.B.dylib (offset 24)

   time stamp 2 Wed Dec 31 17:00:02 1969

      current version 125.2.0

compatibility version 1.0.0

[Graham Dumpleton]

Do you remember what you did when you build the MySQL-python package
in the first place?

Did you explicitly put MySQL bin directory in PATH or did you modify
the site.cfg file to say where it was?

Did you modify any of the default settings in site.cfg?

If you rebuilt it multiple times or have upgraded MySQL, did you start
over building Python package from a clean source package and not one
with existing build/dist directories present?

If you still have the source package and haven't cleaned it, what do
you get for:

$ otool -L build/lib.macosx-10.6-universal-2.6/_mysql.so
build/lib.macosx-10.6-universal-2.6/_mysql.so:

/usr/local/mysql/lib/libmysqlclient_r.16.dylib (compatibility version
17.0.0, current version 17.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.3)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 125.2.10)

in the source package.

Right now thinking something went wrong with the original build.

Not only aren't you using the reentrant MySQL library, but no full
path encoded for it.

Perhaps go back to fresh MySQL-python package and build again and show
us the steps you did and the output as shown below as an example. Most
interested to see the final link line, which for me is:

gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup
build/temp.macosx-10.6-universal-2.6/_mysql.o -L/usr/local/mysql/lib
-lmysqlclient_r -lz -lm -lmygcc -o
build/lib.macosx-10.6-universal-2.6/_mysql.so -arch x86_64

$ rm -rf build clean

$ PATH=/usr/local/mysql-5.1.53-osx10.6-x86_64/bin:$PATH python setup.py build
running build
running build_py
creating build
creating build/lib.macosx-10.6-universal-2.6
copying _mysql_exceptions.py -> build/lib.macosx-10.6-universal-2.6
creating build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/__init__.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/converters.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/connections.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/cursors.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/release.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/times.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
creating build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/__init__.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/CR.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/FIELD_TYPE.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/ER.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/FLAG.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/REFRESH.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/CLIENT.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
running build_ext
building '_mysql' extension
creating build/temp.macosx-10.6-universal-2.6
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv
-Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -pipe
-Dversion_info=(1,2,3,'final',0) -D__version__=1.2.3
-I/usr/local/mysql-5.1.53-osx10.6-x86_64/include
-I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6
-c _mysql.c -o build/temp.macosx-10.6-universal-2.6/_mysql.o -g -Os
-arch x86_64 -fno-common -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE
-DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT
-DDONT_DECLARE_CXA_PURE_VIRTUAL
In file included from _mysql.c:36:
/usr/local/mysql-5.1.53-osx10.6-x86_64/include/my_config.h:1069:1:
warning: "HAVE_WCSCOLL" redefined
In file included from
/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/Python.h:8,
from pymemcompat.h:10,
from _mysql.c:29:
/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/pyconfig.h:803:1:
warning: this is the location of the previous definition
gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup
build/temp.macosx-10.6-universal-2.6/_mysql.o
-L/usr/local/mysql-5.1.53-osx10.6-x86_64/lib -lmysqlclient_r -lz -lm
-lmygcc -o build/lib.macosx-10.6-universal-2.6/_mysql.so -arch x86_64
Grumpys-MacBook-Pro-15:MySQL-python-1.2.3 graham$ rm -rf build
cleanGrumpys-MacBook-Pro-15:MySQL-python-1.2.3 graham$
PATH=/usr/local/mysql/bin:$PATH python setup.py build
running build
running build_py
creating build
creating build/lib.macosx-10.6-universal-2.6
copying _mysql_exceptions.py -> build/lib.macosx-10.6-universal-2.6
creating build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/__init__.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/converters.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/connections.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/cursors.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/release.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
copying MySQLdb/times.py -> build/lib.macosx-10.6-universal-2.6/MySQLdb
creating build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/__init__.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/CR.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/FIELD_TYPE.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/ER.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/FLAG.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/REFRESH.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
copying MySQLdb/constants/CLIENT.py ->
build/lib.macosx-10.6-universal-2.6/MySQLdb/constants
running build_ext
building '_mysql' extension
creating build/temp.macosx-10.6-universal-2.6
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv
-Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -pipe
-Dversion_info=(1,2,3,'final',0) -D__version__=1.2.3
-I/usr/local/mysql/include
-I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6
-c _mysql.c -o build/temp.macosx-10.6-universal-2.6/_mysql.o -g -Os
-arch x86_64 -fno-common -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE
-DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT
-DDONT_DECLARE_CXA_PURE_VIRTUAL
In file included from _mysql.c:36:
/usr/local/mysql/include/my_config.h:1069:1: warning: "HAVE_WCSCOLL" redefined
In file included from
/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/Python.h:8,
from pymemcompat.h:10,
from _mysql.c:29:
/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/pyconfig.h:803:1:
warning: this is the location of the previous definition
gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup
build/temp.macosx-10.6-universal-2.6/_mysql.o -L/usr/local/mysql/lib
-lmysqlclient_r -lz -lm -lmygcc -o
build/lib.macosx-10.6-universal-2.6/_mysql.so -arch x86_64

Graham

[Jianbao Tao]

Thank you, Graham.

I installed MySQL-python inside the virtual environment created by virtualenv with 

<shell>

>$ pip install MySQL-python

</shell>

So, I don't really know what configuration pip used. I think I also built MySQL-python from the source code before, but that didn't work out either.

Jianbao

[Graham Dumpleton]

On 25 August 2011 13:19, Jianbao Tao <jianb...@gmail.com> wrote:
> Thank you, Graham.
> I installed MySQL-python inside the virtual environment created by
> virtualenv with
> <shell>
>>$ pip install MySQL-python

That could well be the problem. If mysql_config is not found when pip
is running to do install it will not be able to link library properly.
Looking in the code it does:

elif enabled(options, 'threadsafe'):
libs = mysql_config("libs_r")
client = "mysqlclient_r"
if not libs:
libs = mysql_config("libs")
client = "mysqlclient"

So, mysql_config must have failed and so couldn't be sure that could
use mysqlclient_r and used mysqlclient instead.

Further, because it didn't know where libs were, couldn't embed the
proper library path to find library. Then at run time it isn't finding
it or isn't liking it.

> </shell>
> So, I don't really know what configuration pip used. I think I also built
> MySQL-python from the source code before, but that didn't work out either.

Would still recommend you go back and try that and set PATH to include
bin directory were mysql_config is located like I showed I did. Check
with otool of _mysql.so in the build directory before installing it in
virtual environment.

Alternatively, try:

PATH=/usr/local/mysql/bin:$PATH pip install MySQL-python

to try to ensure pip can find mysql_config. This may no work depending
on whether pip cleans the user environment before doing stuff.

Graham

[Jianbao Tao]

I prefer a virtual environment without site-packages from the local system. So, building MySQL-python from the source code again doesn't give me much help. I also tried the alternative within the virtual environment, but as you suspected, it didn't work. Anyway, I want to move on from this problem for now. Maybe some day a solution will come around.

However, I want to show my gratitude to you, Graham. Thank you so much for helping me with this one. I did learn a lot how to troubleshoot this kind of issues. Like the 'file' and 'otool' commands, I have never used them before, and didn't even know they are around.

Best regards,

Jianbao

[Graham Dumpleton]

On 25 August 2011 14:47, Jianbao Tao <jianb...@gmail.com> wrote:
> I prefer a virtual environment without site-packages from the local system.
> So, building MySQL-python from the source code again doesn't give me much
> help.

What??????

There is nothing stopping you from installing MySQL-python from source
code into a virtual environment.

Care to explain why that doesn't work.

$ virtualenv --no-site-packages /tmp/myvirtualenv
New python executable in /tmp/myvirtualenv/bin/python
Installing setuptools............done.

$ source /tmp/myvirtualenv/bin/activate

(myvirtualenv)$

Output from both setup.py and pip below. Both work.

At this point since not even actually showing the output from your
builds though, one can only gaze into the crystal ball and wonder.

$ PATH=/usr/local/mysql/bin:$PATH python setup.py install
running install
running bdist_egg
running egg_info
writing MySQL_python.egg-info/PKG-INFO
writing top-level names to MySQL_python.egg-info/top_level.txt
writing dependency_links to MySQL_python.egg-info/dependency_links.txt
reading manifest file 'MySQL_python.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'MANIFEST'
warning: no files found matching 'ChangeLog'
warning: no files found matching 'GPL'
writing manifest file 'MySQL_python.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.6-universal/egg
running install_lib

creating build/bdist.macosx-10.6-universal
creating build/bdist.macosx-10.6-universal/egg
copying build/lib.macosx-10.6-universal-2.6/_mysql.so ->
build/bdist.macosx-10.6-universal/egg
copying build/lib.macosx-10.6-universal-2.6/_mysql_exceptions.py ->
build/bdist.macosx-10.6-universal/egg
creating build/bdist.macosx-10.6-universal/egg/MySQLdb
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/__init__.py ->
build/bdist.macosx-10.6-universal/egg/MySQLdb
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/connections.py ->
build/bdist.macosx-10.6-universal/egg/MySQLdb
creating build/bdist.macosx-10.6-universal/egg/MySQLdb/constants
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/constants/__init__.py
-> build/bdist.macosx-10.6-universal/egg/MySQLdb/constants
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/constants/CLIENT.py
-> build/bdist.macosx-10.6-universal/egg/MySQLdb/constants
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/constants/CR.py ->
build/bdist.macosx-10.6-universal/egg/MySQLdb/constants
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/constants/ER.py ->
build/bdist.macosx-10.6-universal/egg/MySQLdb/constants
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/constants/FIELD_TYPE.py
-> build/bdist.macosx-10.6-universal/egg/MySQLdb/constants
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/constants/FLAG.py
-> build/bdist.macosx-10.6-universal/egg/MySQLdb/constants
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/constants/REFRESH.py
-> build/bdist.macosx-10.6-universal/egg/MySQLdb/constants
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/converters.py ->
build/bdist.macosx-10.6-universal/egg/MySQLdb
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/cursors.py ->
build/bdist.macosx-10.6-universal/egg/MySQLdb
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/release.py ->
build/bdist.macosx-10.6-universal/egg/MySQLdb
copying build/lib.macosx-10.6-universal-2.6/MySQLdb/times.py ->
build/bdist.macosx-10.6-universal/egg/MySQLdb
byte-compiling build/bdist.macosx-10.6-universal/egg/_mysql_exceptions.py
to _mysql_exceptions.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/__init__.py
to __init__.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/connections.py
to connections.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/constants/__init__.py
to __init__.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/constants/CLIENT.py
to CLIENT.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/constants/CR.py
to CR.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/constants/ER.py
to ER.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/constants/FIELD_TYPE.py
to FIELD_TYPE.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/constants/FLAG.py
to FLAG.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/constants/REFRESH.py
to REFRESH.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/converters.py
to converters.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/cursors.py
to cursors.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/release.py
to release.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/MySQLdb/times.py
to times.pyc
creating stub loader for _mysql.so
byte-compiling build/bdist.macosx-10.6-universal/egg/_mysql.py to _mysql.pyc
creating build/bdist.macosx-10.6-universal/egg/EGG-INFO
copying MySQL_python.egg-info/PKG-INFO ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO
copying MySQL_python.egg-info/SOURCES.txt ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO
copying MySQL_python.egg-info/dependency_links.txt ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO
copying MySQL_python.egg-info/top_level.txt ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO
writing build/bdist.macosx-10.6-universal/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg' and
adding 'build/bdist.macosx-10.6-universal/egg' to it
removing 'build/bdist.macosx-10.6-universal/egg' (and everything under it)
Processing MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg
Removing /private/tmp/myvirtualenv/lib/python2.6/site-packages/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg
Copying MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg to
/private/tmp/myvirtualenv/lib/python2.6/site-packages
MySQL-python 1.2.3 is already the active version in easy-install.pth

Installed /private/tmp/myvirtualenv/lib/python2.6/site-packages/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg
Processing dependencies for MySQL-python==1.2.3
Finished processing dependencies for MySQL-python==1.2.3

Now pip.

$ PATH=/usr/local/mysql/bin:$PATH pip install MySQL-python
Downloading/unpacking MySQL-python
Running setup.py egg_info for package MySQL-python
warning: no files found matching 'MANIFEST'
warning: no files found matching 'ChangeLog'
warning: no files found matching 'GPL'
Installing collected packages: MySQL-python
Running setup.py install for MySQL-python
building '_mysql' extension

gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g
-fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -pipe
-Dversion_info=(1,2,3,'final',0) -D__version__=1.2.3
-I/usr/local/mysql/include
-I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6
-c _mysql.c -o build/temp.macosx-10.6-universal-2.6/_mysql.o -g -Os
-arch x86_64 -fno-common -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE
-DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT
-DDONT_DECLARE_CXA_PURE_VIRTUAL
In file included from _mysql.c:36:
/usr/local/mysql/include/my_config.h:1069:1: warning:
"HAVE_WCSCOLL" redefined
In file included from
/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/Python.h:8,
from pymemcompat.h:10,
from _mysql.c:29:
/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/pyconfig.h:803:1:
warning: this is the location of the previous definition
gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup
build/temp.macosx-10.6-universal-2.6/_mysql.o -L/usr/local/mysql/lib
-lmysqlclient_r -lz -lm -lmygcc -o
build/lib.macosx-10.6-universal-2.6/_mysql.so -arch x86_64

warning: no files found matching 'MANIFEST'
warning: no files found matching 'ChangeLog'
warning: no files found matching 'GPL'
Successfully installed MySQL-python
Cleaning up...

Let us check that.