py3.2 sysconfig.get_config_var('CC') wrong value returned

[Simon Yarde]

Throwing this out there.. woes with Python 3.2 sysconfig and more..

I recently started migrating a Pyramid app to py3.2 and hit a uWSGI build issue with the python installer;

sysconfig.get_config_var returns 'gcc-4.2', which doesn't exist on my system, but py2.7 reports 'llvm-gcc-4.2' correctly.

I got uwsgi to compile by setting env 'CC=gcc', causing the installer to avoid sysconfig and use the os.environ.get value instead.

Then hit another error; Pip error info didn't help me much, but easy_install reports 'ld: library not found for -lpython3.2m'. lpython3.2m doesn't exist but python3.2m does.

I can see there is a problem with the Py3.2 sysconfig and not the uwsgi installer, but I'm scratching my head to diagnose the second error and wonder if it's symptomatic of the first?

I suspect an Apple/Darwin issue with Python3 ..anyone seem similar problems?

Best, S

Transcript below:

### Py3.2 - Install via pip - mis-reports gcc as ``gcc-4.2``. Installer gets 'None' for dumpversion because it doesn't exist.

Traceback (most recent call last):

File "uwsgiconfig.py", line 304, in __init__

gcc_major = int(gcc_version.split('.')[0])

ValueError: invalid literal for int() with base 10: 'None'

(py3)$ python
Python 3.2.2 (v3.2.2:137e45f15c0b, Sep 3 2011, 17:28:59)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('CC')
'gcc-4.2'
>>> quit()
(py3)$ gcc-4.2 -dumpversion
-bash: gcc-4.2: command not found

(py3)$ llvm-gcc-4.2 -dumpversion
4.2.1
(py3)$ llvm-gcc -dumpversion
4.2.1
(py3)$ gcc -dumpversion
4.2.1

### Py2.7 reports CC correctly

(py3)$ deactivate; cd ../py27; . bin/activate;
(py27)$ python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('CC')
'llvm-gcc-4.2'
>>> quit()
(py27)$ llvm-gcc-4.2 -dumpversion
4.2.1

### Py3.2 - Install via pip with env ``export CC=gcc`` - now there is a linking error

*** error linking uWSGI ***·
·
----------------------------------------·
·
Command /Users/simonyarde/development/py3/bin/python -c "import setuptools;__file__='/Users/simonyarde/development/py3/build/uwsgi/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /var/folders/pb/6qk5pbtd6klfd020q1wnkhb40000gn/T/pip-1inrqx-record/install-record.txt --install-headers /Users/simonyarde/development/py3/include/site/python3.2 failed with error code 1 in /Users/simonyarde/development/py3/build/uwsgi·
·
Exception information:·
Traceback (most recent call last):·
File "/Users/simonyarde/development/py3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/basecommand.py", line 104, in main·
status = self.run(options, args)·
File "/Users/simonyarde/development/py3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/commands/install.py", line 250, in run·
requirement_set.install(install_options, global_options)·
File "/Users/simonyarde/development/py3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/req.py", line 1133, in install·
requirement.install(install_options, global_options)·
File "/Users/simonyarde/development/py3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/req.py", line 577, in install·
cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)·
File "/Users/simonyarde/development/py3/lib/python3.2/site-packages/pip-1.1-py3.2.egg/pip/__init__.py", line 256, in call_subprocess·
% (command_desc, proc.returncode, cwd))·
pip.exceptions.InstallationError: Command /Users/simonyarde/development/py3/bin/python -c "import setuptools;__file__='/Users/simonyarde/development/py3/build/uwsgi/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /var/folders/pb/6qk5pbtd6klfd020q1wnkhb40000gn/T/pip-1inrqx-record/install-record.txt --install-headers /Users/simonyarde/development/py3/include/site/python3.2 failed with error code 1 in /Users/simonyarde/development/py3/build/uwsgi·

### Py3.2 - Install via easy_install with env ``export CC=gcc``

*** uWSGI linking ***
gcc -o /Users/simonyarde/development/py3/bin/uwsgi utils.o protocol.o socket.o logging.o master.o master_utils.o emperor.o notify.o mule.o subscription.o plugins.o lock.o cache.o queue.o event.o signal.o cluster.o rpc.o gateway.o loop.o lib/rbtree.o lib/amqp.o rb_timers.o main.o proto/base.o proto/uwsgi.o proto/http.o proto/fastcgi.o async.o ini.o yaml.o ldap.o snmp.o sendfile.o xmlconf.o sqlite3.o spooler.o plugins/python/python_plugin.o plugins/python/pyutils.o plugins/python/pyloader.o plugins/python/wsgi_handlers.o plugins/python/wsgi_headers.o plugins/python/wsgi_subhandler.o plugins/python/web3_subhandler.o plugins/python/pump_subhandler.o plugins/python/gil.o plugins/python/uwsgi_pymodule.o plugins/python/profiler.o plugins/python/symimporter.o plugins/ping/ping_plugin.o plugins/cache/cache.o plugins/nagios/nagios.o plugins/rrdtool/rrdtool.o plugins/carbon/carbon.o plugins/rpc/rpc_plugin.o plugins/fastrouter/fastrouter.o plugins/http/http.o plugins/ugreen/ugreen.o plugins/signal/signal_plugin.o plugins/syslog/syslog_plugin.o plugins/rsyslog/rsyslog_plugin.o plugins/logsocket/logsocket_plugin.o -lpthread -lm -rdynamic -lldap -lxml2 -lz -lpthread -licucore -lm -lsqlite3 -ldl -framework CoreFoundation -L/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/config-3.2m -lpython3.2m
ld: library not found for -lpython3.2m
collect2: ld returned 1 exit status
*** error linking uWSGI ***
error: Setup script exited with 1

[Marius Gedminas]

On Wed, Feb 29, 2012 at 03:01:59PM +0000, Simon Yarde wrote:
> Then hit another error; Pip error info didn't help me much, but
> easy_install reports 'ld: library not found for -lpython3.2m'.
> lpython3.2m doesn't exist but python3.2m does.

It's looking for libpython3.2m.so in $LD_LIBRARY_PATH.

At least that's what it would be on Linux. I never worked on Mac OS X.

> I suspect an Apple/Darwin issue with Python3 ..anyone seem similar
> problems?

Marius Gedminas
--
Was a time, all we got for christmas were bitfields. And we were happy!
-- Mike Barton
You had bitfields? When I wanted more bits I had to pull out my wirewrap tool
and put them in myself!
-- Andrew Dalke

[Graham Higgins]

On Wednesday, February 29, 2012 5:01:38 PM UTC, Marius Gedminas wrote:

It's looking for libpython3.2m.so in $LD_LIBRARY_PATH.

At least that's what it would be on Linux.  I never worked on Mac OS X.

It's $DYLD_LIBRARY_PATH  on OS X

so something like:

DYLD_LIBRARY_PATH="/path/to/dircontaininglibpython3.2m.so:$DYLD_LIBRARY_PATH"

should get you a bit further.

You might want to check to see if "gcc_select" is an available command. If so, that's a supported means of switching between gcc versions.

(Apologies for being vague, the logic board on my PB is fried and I can't decide whether to repair it or junk it).

Cheers,

Graham

[Simon Yarde]

Thanks Graham and Marius, much appreciated. 

I'll get stuck into this tomorrow and report back.

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To view this discussion on the web visit https://groups.google.com/d/msg/pylons-discuss/-/KxUkE4GfcoAJ.
To post to this group, send email to pylons-...@googlegroups.com.
To unsubscribe from this group, send email to pylons-discus...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.

[Simon Yarde]

Thanks again chaps. gcc_select isn't available, and the installer continued to grumble, but you got me headed in the right direction.

From my notes:

The Python 3.2.2 package installer for OSX was compiled under a previous
version of Xtools that used ``gcc-4.2``. This can cause problems for
installers that rely on ``sysconfig.get_config_var('CC')`` to identify
the C-Compiler because ``gcc-4.2`` is not available from XCode 4.2
onward. Apple instead provides ``llvm-gcc-4.2`` and the improved
``clang`` 4.2.1, intended to replace ``gcc``.

In case anyone else hits this on their dev machines, I compiled Python with clang:

$ ./configure \
> --prefix=/usr/local/python-3.2.2 \
> --enable-shared \
> CC=clang \
> MACOSX_DEPLOYMENT_TARGET=10.7

Python 3.2.2 (default, Mar 2 2012, 14:25:01) [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.10.1)] on Darwin-11.3.0-x86_64-i386-64bit

Confirm Pyramid and uWSGI working.

I guess this isn't really a Python build issue as-such, but more of an issue with how uWSGI locates the compiler on Darwin, and that Apple have phased out gcc rather brutally.