[Python-Dev] build from source on mac uses clang, python.org binary uses gcc

[Chris Withers]

Hi All,

I've run into "some issues" installing lxml for python 3.3 on my mac:


One of the stumbling blocks I've hit is that I built python 3.3 from
source (./configure && make && make altinstall), and it used clang:

buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe
Python 3.3.0 (default, Jan 23 2013, 09:56:03)
[GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

...which makes lxml pretty unhappy.

However, when I install the binary Python 3.3 from python.org, it's
built with GCC:

buzzkill:virtualenvs chris$ python3.3
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Why the difference?

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
_______________________________________________
Python-Dev mailing list
Pytho...@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/dev-python%2Bgarchive-30976%40googlegroups.com

[Chris Withers]

On 14/02/2013 09:18, Chris Withers wrote:
> Hi All,
>
> I've run into "some issues" installing lxml for python 3.3 on my mac:

(forgot the links)

https://mailman-mail5.webfaction.com/pipermail/lxml/2013-February/006730.html
https://mailman-mail5.webfaction.com/pipermail/lxml/2013-February/006731.html

[Ronald Oussoren]

On 14 Feb, 2013, at 10:18, Chris Withers <ch...@simplistix.co.uk> wrote:

> Hi All,
>
> I've run into "some issues" installing lxml for python 3.3 on my mac:
>
>
> One of the stumbling blocks I've hit is that I built python 3.3 from source (./configure && make && make altinstall), and it used clang:
>
> buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe
> Python 3.3.0 (default, Jan 23 2013, 09:56:03)
> [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>>
>
> ...which makes lxml pretty unhappy.

Why is that? Clang is a working C and C++ compiler. I've noticed in the past that lxml's automatic compilation
of libxml2 and libxslt is dodgy on OSX, but haven't had time yet to work on a patch that calculates the right
compiler flags instead of hardcoding some guesses in setup.py. Also, libxml2 won't compile on OSX 10.8 at all
unless you use the version in the repository (an incompatibility with the pthread headers on 10.8).

You get clang instead of GCC when configure detects that clang is the only usable compiler on OSX,
where usable means that the compiler is present and is not llvm-gcc. Llvm-gcc contains bugs that
miscompile at least parts of the unicode implementation in CPython 3.3 and word on the street is that
these bugs won't get fixed unless they are also present in clang.

>
> However, when I install the binary Python 3.3 from python.org, it's built with GCC:
>
> buzzkill:virtualenvs chris$ python3.3
> Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11)
> [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>>
>
> Why the difference?

The binary was created on an older release of OSX where gcc is a valid compiler choice. Distutils
should still pick clang when it detects that gcc is actually llvm-gcc though, doesn't it do that on your system?

BTW. Which OSX release are you using? And which variant of the OSX installer?

Ronald

P.S. Isn't this a question for python-list?

>
> cheers,
>
> Chris
>
> --
> Simplistix - Content Management, Batch Processing & Python Consulting
> - http://www.simplistix.co.uk
> _______________________________________________
> Python-Dev mailing list
> Pytho...@python.org
> http://mail.python.org/mailman/listinfo/python-dev

> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

[Chris Withers]

On 14/02/2013 12:15, Ronald Oussoren wrote:
>
>> buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe
>> Python 3.3.0 (default, Jan 23 2013, 09:56:03)
>> [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>
>>
>> ...which makes lxml pretty unhappy.
>
> Why is that?

I'm not lxml's maintainer, I'd suggest asking Stefan...

> The binary was created on an older release of OSX where gcc is a valid compiler choice. Distutils
> should still pick clang when it detects that gcc is actually llvm-gcc though, doesn't it do that on your system?

No. The gcc-based Python 3.3.0 from python.org uses gcc to compile lxml.

> BTW. Which OSX release are you using? And which variant of the OSX installer?

10.7.5. If by "OSX Installer" you mean that for Python 3.3.0, then it
was the 64-bit one from python.org.

> P.S. Isn't this a question for python-list?

I think questions about compiling python and how python chooses to
compile its extensions, and weird things on particular operating systems
related to that, are fair game for python-dev.

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
_______________________________________________
Python-Dev mailing list
Pytho...@python.org
http://mail.python.org/mailman/listinfo/python-dev

Unsubscribe: http://mail.python.org/mailman/options/python-dev/dev-python%2Bgarchive-30976%40googlegroups.com

[Ned Deily]

In article <511D350B...@simplistix.co.uk>,

Chris Withers <ch...@simplistix.co.uk> wrote:

> On 14/02/2013 12:15, Ronald Oussoren wrote:
> >
> >> buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe
> >> Python 3.3.0 (default, Jan 23 2013, 09:56:03)
> >> [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin
> >> Type "help", "copyright", "credits" or "license" for more information.
> >>>>>
> >>
> >> ...which makes lxml pretty unhappy.
> >
> > Why is that?
>
> I'm not lxml's maintainer, I'd suggest asking Stefan...
>
> > The binary was created on an older release of OSX where gcc is a valid
> > compiler choice. Distutils
> > should still pick clang when it detects that gcc is actually llvm-gcc
> > though, doesn't it do that on your system?
>
> No. The gcc-based Python 3.3.0 from python.org uses gcc to compile lxml.

Hmm, that's odd. It should be using clang unless on 10.7 with a current
Xcode 4 installed unless something is overriding the compiler selection,
e.g. setting CC=clang. (Also, I note you appear to be using an
out-of-date clang; Xcode 4.6 and its Command Line Tools are current:
$ clang --version
Apple LLVM version 4.2 (clang-425.0.24) (based on LLVM 3.2svn
Because of the relative newness of clang in the OS X world, it's
important to keep up-to-date.)

But, installing lxml 3.1.0 (from the tarball you cite) with the Python
3.3.0 64-bit/32-b it install seems to work fine for me. At least, it
didn't get any import errors.

Looking at your traceback (from
https://mailman-mail5.webfaction.com/pipermail/lxml/2013-February/006730.
html) appears to show that the lxml is picking up libxml2 and libxslt
from /Library/Frameworks/Python.framework/Versions/7.2/lib
and or /Library/Frameworks/Python.framework/Versions/7.2/lib. That
almost certainly is the root of your problems. I'm not sure what those
are but possibly EPD Python distributions. Your 3.3 builds should not
be trying to used them to satisfy libraries. Perhaps there are links in
/usr/local/*. Removing those links or files in /usr/local should allow
you to build lxml with the Apple-supplied libxml2 and libxslt. If you
want newer versions or the libs, either build them for 64-/32-bit archs
or use a third-party package manager like Homebrew or MacPorts to supply
them (and/or supply a Python 3.3).

--
Ned Deily,
n...@acm.org

[Ronald Oussoren]

On 14 Feb, 2013, at 20:03, Chris Withers <ch...@simplistix.co.uk> wrote:

> On 14/02/2013 12:15, Ronald Oussoren wrote:
>>
>>> buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe
>>> Python 3.3.0 (default, Jan 23 2013, 09:56:03)
>>> [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>>
>>>
>>> ...which makes lxml pretty unhappy.
>>
>> Why is that?
>
> I'm not lxml's maintainer, I'd suggest asking Stefan...

I should have been more clear: what error does lxml give when compiling?

Ronald