--enable-darwin and Python

[gmn]

I'm sending this here rather than to vim_use because it's a Mac-specific
problem, even though it's about the official console vim from Bram's hg repo
(aka "BramVim").

A short while ago, I discovered I had been calling --disable-darwin for the
configuring of BramVim. I'd been doing this for years, disguised 'neath a
bash alias. I couldn't recall why I was doing something so apparently dumb.

Well, I discovered why, I think.

If I don't --disable-darwin, then building vim chokes on enabling the python
interpreter:

objects/if_python.o objects/os_macosx.o objects/os_mac_conv.o objects/main.o objects/memfile.o -lm -lncurses -liconv -lintl -framework Cocoa -framework Python
Undefined symbols:
"__PyObject_NextNotImplemented", referenced from:
__PyObject_NextNotImplemented$non_lazy_ptr in if_python.o
"_PyCapsule_GetPointer", referenced from:
_convert_dl in if_python.o
"_PyCapsule_New", referenced from:
_convert_dl in if_python.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make[1]: *** [vim] Error 1
make: *** [first] Error 2

If I do --disable-darwin, I get python support just fine.

It should be known that:
0. This is on both Tiger and Leopard. There's no problem on 10.7 or even 10.8
1. My $PYTHONPATH is set to /usr/local/gmnpyth, which is where python 2.7.3
lives.
2. Even on Tiger and Leopard, building *MacVim* with the same $PYTHONPATH and
--enable-pythoninterp works just fine.
3. The culprit is not any of the recent patches to vim dealing with python
compilation problems (surprisingly).
4. It took a while to show up after I stopped using --disable-darwin because I
hadn't obliterated the build caches inside my clone of the hg repo in a long
time. Once I did so, pow!

-gmn

[Ben Schmidt]

On 5/08/12 7:13 AM, gmn wrote:
> I'm sending this here rather than to vim_use because it's a Mac-specific
> problem, even though it's about the official console vim from Bram's hg repo
> (aka "BramVim").
>
> A short while ago, I discovered I had been calling --disable-darwin for the
> configuring of BramVim. I'd been doing this for years, disguised 'neath a
> bash alias. I couldn't recall why I was doing something so apparently dumb.
>
> Well, I discovered why, I think.
>
> If I don't --disable-darwin, then building vim chokes on enabling the python
> interpreter:
>
> objects/if_python.o objects/os_macosx.o objects/os_mac_conv.o objects/main.o objects/memfile.o -lm -lncurses -liconv -lintl -framework Cocoa -framework Python
> Undefined symbols:
> "__PyObject_NextNotImplemented", referenced from:
> __PyObject_NextNotImplemented$non_lazy_ptr in if_python.o
> "_PyCapsule_GetPointer", referenced from:
> _convert_dl in if_python.o
> "_PyCapsule_New", referenced from:
> _convert_dl in if_python.o
> ld: symbol(s) not found
> collect2: ld returned 1 exit status
> make[1]: *** [vim] Error 1
> make: *** [first] Error 2

Hmm. Strange. I've never had any problem building (Mac)Vim against
Python, and I certainly don't use --disable-darwin. I suspect I've
always been building against a Python from MacPorts, though.

> If I do --disable-darwin, I get python support just fine.

Interesting.

> It should be known that:
> 0. This is on both Tiger and Leopard. There's no problem on 10.7 or even 10.8

I guess I'll have to fire up an older machine to see if I can
investigate this further at all. I should be able to do that if I need
to.

> 1. My $PYTHONPATH is set to /usr/local/gmnpyth, which is where python 2.7.3
> lives.

How did it get there?

> 2. Even on Tiger and Leopard, building *MacVim* with the same $PYTHONPATH and
> --enable-pythoninterp works just fine.

Also interesting. I wonder if there's a patch in MacVim that should be
'pushed upstream' or moved outside a preprocessor conditional or
something.

When you attempt to build BramVim, are you doing it using the MacVim
sources, but disabling the GUI, or with sources directly from Bram. Can
you try it both ways and see if there is any difference?

> 3. The culprit is not any of the recent patches to vim dealing with python
> compilation problems (surprisingly).

Good to know.

> 4. It took a while to show up after I stopped using --disable-darwin because I
> hadn't obliterated the build caches inside my clone of the hg repo in a long
> time. Once I did so, pow!

Two of Murphy's Laws:

"Everything that can go wrong will go wrong, and at the worst possible
time."

"If everything seems to be going well, you have obviously overlooked
something."

Ben.

[gmn]

I forgot to mention that the problem also occurs on Snow Leopard. So
it's not a PPC/Intel issue (my Tiger and Leopard machines are PPC).

> >1. My $PYTHONPATH is set to /usr/local/gmnpyth, which is where python 2.7.3
> >lives.
>
> How did it get there?

I put it there---not just the variable, but python itself. Why?
Because I wanted python 2.7.x on a Tiger machine and a Leopard machine,
both of which come with outdated pythons. Why did I put a new python
*there*? Because it's easier for me to obliterate and troubleshoot,
rather than futzing around with frameworks and such.

(Because I don't understand any of this stuff, I try to plan escape
routes in advance and try not to move system stuff).

Note that when I build python 2.7.x from source, I *don't* do
--enable-framework. I *do* remember to set
MACOSX_DEPLOYMENT_TARGET=blah.

> >2. Even on Tiger and Leopard, building *MacVim* with the same $PYTHONPATH and
> >--enable-pythoninterp works just fine.
>
> Also interesting. I wonder if there's a patch in MacVim that should be
> 'pushed upstream' or moved outside a preprocessor conditional or
> something.
>
> When you attempt to build BramVim, are you doing it using the MacVim
> sources, but disabling the GUI, or with sources directly from Bram. Can
> you try it both ways and see if there is any difference?

BramVim, by definition, is the sources directly from Bram's hg repo. I
have never tried building Vim from Bj�rn's git repo with a --disable-gui
flag; I will do so now.

-gmn

[gmn]

Ben Schmidt, on 08/06/12 at 00:28:58 +1000, wrote:

> When you attempt to build BramVim, are you doing it using the MacVim
> sources, but disabling the GUI, or with sources directly from Bram. Can
> you try it both ways and see if there is any difference?

OK, I just tried --disable-gui in my clone of the MacVim repo, and I get the same python error as before.

-gmn
>

[björn]

On Sat, Aug 4, 2012 at 11:13 PM, gmn wrote:
> I'm sending this here rather than to vim_use because it's a Mac-specific
> problem, even though it's about the official console vim from Bram's hg repo
> (aka "BramVim").
>
> A short while ago, I discovered I had been calling --disable-darwin for the
> configuring of BramVim. I'd been doing this for years, disguised 'neath a
> bash alias. I couldn't recall why I was doing something so apparently dumb.
>
> Well, I discovered why, I think.
>
> If I don't --disable-darwin, then building vim chokes on enabling the python
> interpreter:
>
> objects/if_python.o objects/os_macosx.o objects/os_mac_conv.o objects/main.o objects/memfile.o -lm -lncurses -liconv -lintl -framework Cocoa -framework Python
> Undefined symbols:
> "__PyObject_NextNotImplemented", referenced from:
> __PyObject_NextNotImplemented$non_lazy_ptr in if_python.o
> "_PyCapsule_GetPointer", referenced from:
> _convert_dl in if_python.o
> "_PyCapsule_New", referenced from:
> _convert_dl in if_python.o
> ld: symbol(s) not found
> collect2: ld returned 1 exit status
> make[1]: *** [vim] Error 1
> make: *** [first] Error 2
>

> 1. My $PYTHONPATH is set to /usr/local/gmnpyth, which is where python 2.7.3
> lives.

I'm guessing that there is a version mismatch between the headers that
you build against and the library that is being linked with. In the
output you see "-framework Python" which means it will link against
the system Python (/System/Library/Frameworks/Python.framework). You
can look inside src/auto/config.mk to see which headers are being used
(probably your custom version).

To get it to work you have to make it build and link the same Python
version. You can do this either by patching src/configure.in (which I
find to be a PITA but would be the right thing to do) or as a
temporary hack by editing src/auto/config.mk (easier). The latter
gets overwritten each time you run configure however...

Björn

[gmn]

Though I started this thread as a question about BramVim, it now appears
to me that the mystery is really about MacVim. Allow me to explain.

Examining src/configure.in for BramVim reveals that the python libs get
set starting at line 898, where they get made into "-framework Python"
if $MACOSX is set, and some other gibberish otherwise. $MACOSX is
governed by --enable/--disable-darwin. So when I --disable-darwin, the
"gibberish" in fact sets the python libs, correctly, to my
/usr/local/gmnpyth. When I enable darwin, it gets what Bj�rn said above
and chokes.

End of mystery re BramVim. (I bet the reason my problem only appeared
on 10.6 and under is that 10.8 and probly 10.7 have their system
python's as 2.7.x anyway, avoiding the choke).

Now, mystery re MacVim: the src/configure.in for MacVim has the
corresponding line at 921. And $MACOSX is set in MacVim. So the python
libs get set to "-framework Python" just like in BramVim with darwin
enabled. So then why does MacVim not choke? (And why does it choke
when gui is explicitly disabled?). MacVim's src/auto/config.mk seems
identical to that of BramVim-with-darwin-enabled on python-related
aspects (though not on everything).

I notice that in fact MacVim, though it doesn't choke, is not picking up
my custom python and instead uses the sytem's old one. That now makes sense.
The further mystery is how I can get MacVim (and darwin-enabled BramVim)
to pick up my custom python location. (Setting --with-python-config-dir
doesn't help).

-gmn

[Lukas Ciszewski]

Hello,

a little bit late, but i had the same problem with the installation of the 7.4 branch from sources on my OSX 10.6.8 and investigated a little bit. My solution to the Problem was to install Python 2.7.3 as a framework merging the resulting folder into the /System/Library/.../Python.framework folder and then changing the .../Python.framework/Versions/Current pointer to the newly added 2.7 Version. Then compiling and running vim made no problems. As this is only a workaround, i personally think, as stated before, the configuration process needs to be adjusted.

Cheers
Lukas