Using --enable-framework on Mac OS X?
Prabhu Ramachandran
I had a mini-vacation and I spent a while trying to get sage working
with VTK on a Macbook. The machine is somewhat slow and experimenting
takes a long while. I've managed to get VTK built with Sage but
required a framework build. Here are the details.
I've been using sage-3.4 (since thats what I had with me). I have a
fresh sage (3.4) install where I installed cmake from Jaap's package here:
http://sage.math.washington.edu/home/jsp/SPKGS/MayaviOSMesa/
I then got
http://sage.math.washington.edu/home/jsp/SPKGS/MayaviOSMesa/vtk-cvs-20090316osmesa.spkg
and changed the spkg-install so it builds on the Mac.
I first tried building VTK with the existing sage python build but kept
running into build errors with VTK unable to find several Python
symbols. I configured VTK to build with the libpython2.5.a but that
failed to build with link errors that I forgot to note down (I must have
tried 10 different things trying to get this going). After various
failed attempts I finally decided to change the way sage builds its
Python and added:
--enable-framework="$SAGE_LOCAL/Frameworks"
to the configure command in the spkg-install of python-2.5.2.p9 that
ships with sage-3.4. This built successfully. I basically untarred the
standard spkg, changed the spkg-install suitably and ran it within a
sage -sh session. I then built VTK with the following configuration for
cmake:
-DPYTHON_EXECUTABLE:FILEPATH=$SAGE_LOCAL/Frameworks/Python.framework/Versions/2.5/bin/python\
-DPYTHON_INCLUDE_PATH:PATH=$SAGE_LOCAL/Frameworks/Python.framework/Headers\
-DPYTHON_LIBRARY:FILEPATH=$SAGE_LOCAL/Frameworks/Python.framework/Versions/2.5/Python\
-DVTK_USE_CARBON:BOOL=ON
This built VTK perfectly.
I initially ran into problems like so:
sage: import vtk
Fatal Python error: Interpreter not initialized (version mismatch?)
$SAGE_LOCAL/bin/sage-sage: line 205: 1407 Abort trap
sage-ipython "$@" -i
But figured that my existing builds were confusing matters and once I
moved that old build out of the way, everything worked great. I was
able to do this:
sage: import vtk
sage: c = vtk.vtkConeSource()
[...]
I have the following questions:
1. Why isn't --enable-framework turned on by default for the sage
python build on the Mac? If this isn't a hassle can you please consider
making this the default?
2. What else would it take to get the VTK package to optional quality?
Thanks!
cheers,
prabhu
mabshoff
wrote:
> Hi,
Hi Prabhu,
> I had a mini-vacation and I spent a while trying to get sage working
> with VTK on a Macbook. The machine is somewhat slow and experimenting
> takes a long while. I've managed to get VTK built with Sage but
> required a framework build. Here are the details.
> I have the following questions:
>
> 1. Why isn't --enable-framework turned on by default for the sage
> python build on the Mac? If this isn't a hassle can you please consider
> making this the default?
certain bits of Sage's build system to deal with OSX specific settings
with frameworks. None of the components in Sage require Python as a
framework and IIRC it caused relocation problems, but those might be
fixed now.
I don't understand why VTK needs Python to be a framework. Googling
around for 5 minutes did not reveal anything useful. Can you give me
some pointers and/or documentation on this?
Another problem is that if we changed the default for OSX we would
need to force a rebuild of every other python component in Sage, so
since we don't have any problems as it is seems plainly not worth it.
> 2. What else would it take to get the VTK package to optional quality?
> Thanks!
>
> cheers,
> prabhu
Cheers,
Michael
Prabhu Ramachandran
>> 1. Why isn't --enable-framework turned on by default for the sage
>> python build on the Mac? If this isn't a hassle can you please consider
>> making this the default?
>
> It caused some issues in the past and we would have to special case
> certain bits of Sage's build system to deal with OSX specific settings
> with frameworks. None of the components in Sage require Python as a
> framework and IIRC it caused relocation problems, but those might be
> fixed now.
>
> I don't understand why VTK needs Python to be a framework. Googling
> around for 5 minutes did not reveal anything useful. Can you give me
> some pointers and/or documentation on this?
I really don't know if this is needed or not. It could well be a VTK
issue. I need to build VTK with Carbon support and I'm not sure that
requires a framework build of Python. Anyway, VTK tries to detect the
Python library to link to when it builds the Python wrappers. I think
it finds the system Python by default which obviously will not work.
When I point it to the libpython25.a file in
SAGE_LOCAL/lib/python2.5/config, I get link errors since my guess is
that file is not a proper universal library? Upon some digging,
fortunately, I seem to have an install.log lying around. Here is the
exact error message I got when linking to libpython25.a:
Linking CXX shared library ../bin/libvtkCommonPythonD.dylib
ld: common symbols not allowed with MH_DYLIB output format with the
-multi_module option
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(object.o)
definition of common _Py_DivisionWarningFlag (size 16)
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(pythonrun.o)
definition of common _Py_VerboseFlag (size 16)
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(pythonrun.o)
definition of common _Py_DebugFlag (size 16)
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(pythonrun.o)
definition of common _Py_FrozenFlag (size 16)
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(pythonrun.o)
definition of common _Py_IgnoreEnvironmentFlag (size 16)
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(pythonrun.o)
definition of common _Py_InteractiveFlag (size 16)
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(pythonrun.o)
definition of common _Py_NoSiteFlag (size 16)
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(parsetok.o)
definition of common _Py_TabcheckFlag (size 16)
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(myreadline.o)
definition of common _PyOS_ReadlineFunctionPointer (size 16)
/Users/prabhu/src/build/sage-3.4/local/lib/python2.5/config/libpython2.5.a(myreadline.o)
definition of common __PyOS_ReadlineTState (size 16)
/usr/libexec/gcc/i686-apple-darwin8/4.0.1/libtool: internal link edit
command failed
If I leave the Python library unspecified it runs into linker errors
with symbols being undefined at link time. None of this happens with a
framework build.
I guess I could retry with a non-framework build but it really takes a
while to build on my machine and all previous attempts failed.
> Another problem is that if we changed the default for OSX we would
> need to force a rebuild of every other python component in Sage, so
> since we don't have any problems as it is seems plainly not worth it.
But isn't everything anyways built from source? So how would this
affect anything? In my case nothing else seems affected -- I rebuilt
Python and installed that over the existing build. I didn't test too
much apart from VTK related scripts though.
cheers,
prabhu
mabshoff
On May 10, 9:19 am, Prabhu Ramachandran <pra...@aero.iitb.ac.in>
wrote:
> > I don't understand why VTK needs Python to be a framework. Googling
> > around for 5 minutes did not reveal anything useful. Can you give me
> > some pointers and/or documentation on this?
>
> I really don't know if this is needed or not. It could well be a VTK
> issue. I need to build VTK with Carbon support and I'm not sure that
> requires a framework build of Python. Anyway, VTK tries to detect the
> Python library to link to when it builds the Python wrappers. I think
> it finds the system Python by default which obviously will not work.
> When I point it to the libpython25.a file in
> SAGE_LOCAL/lib/python2.5/config, I get link errors since my guess is
> that file is not a proper universal library? Upon some digging,
> fortunately, I seem to have an install.log lying around. Here is the
> exact error message I got when linking to libpython25.a:
>
> Linking CXX shared library ../bin/libvtkCommonPythonD.dylib
>
> ld: common symbols not allowed with MH_DYLIB output format with the
> -multi_module option
Ok, check out
http://www.mail-archive.com/g...@gcc.gnu.org/msg05799.html
for a probable solution. It might then expose the next issue, but I
guess this is step by step ;).
> If I leave the Python library unspecified it runs into linker errors
> with symbols being undefined at link time. None of this happens with a
> framework build.
>
> I guess I could retry with a non-framework build but it really takes a
> while to build on my machine and all previous attempts failed.
> > Another problem is that if we changed the default for OSX we would
> > need to force a rebuild of every other python component in Sage, so
> > since we don't have any problems as it is seems plainly not worth it.
>
> But isn't everything anyways built from source? So how would this
> affect anything? In my case nothing else seems affected -- I rebuilt
> Python and installed that over the existing build. I didn't test too
> much apart from VTK related scripts though.
problem is when a user runs -upgrade for Sage.
> cheers,
> prabhu
Cheers,
Michael
Prabhu Ramachandran
>> Linking CXX shared library ../bin/libvtkCommonPythonD.dylib
>>
>> ld: common symbols not allowed with MH_DYLIB output format with the
>> -multi_module option
>
> <SNIP>
>
> Ok, check out
>
> http://www.mail-archive.com/g...@gcc.gnu.org/msg05799.html
>
> for a probable solution. It might then expose the next issue, but I
> guess this is step by step ;).
Well, I have seen this when I got the error first time (google is our
friend) but didn't believe the answer would work for the simple reason
that VTK is really well tested. I didn't want to mess with compilation
arguments before testing other options. Regardless, I did try a manual
link with -Wl,-single_module and it didn't work -- I got different
errors. I could try -fno-common but that would have to be done for the
Python build and not the VTK build. It appears that the problem is not
with VTK but the Python build since the linking fails for the Python
library. The VTK libraries are linked to each other quite nicely.
>> I guess I could retry with a non-framework build but it really takes a
>> while to build on my machine and all previous attempts failed.
>
> Sure. Let me know if the above works.
I've tried the above and a few other things too. I'm still all ears
though, so keep the suggestions coming.
>>> Another problem is that if we changed the default for OSX we would
>>> need to force a rebuild of every other python component in Sage, so
>>> since we don't have any problems as it is seems plainly not worth it.
>> But isn't everything anyways built from source? So how would this
>> affect anything? In my case nothing else seems affected -- I rebuilt
>> Python and installed that over the existing build. I didn't test too
>> much apart from VTK related scripts though.
>
> Well, it doesn't matter for builds from source too much, the real
> problem is when a user runs -upgrade for Sage.
Sorry for being dense. You mean when the user runs -upgrade with a
non-framework install? Suppose a user has a fresh install, would the
framework build cause problems when this user does a -upgrade?
IOW, is this only a problem for those with older non-framework builds?
Thanks.
cheers,
prabhu
mabshoff
wrote:
<SNIP>
> Well, I have seen this when I got the error first time (google is our
> friend) but didn't believe the answer would work for the simple reason
> that VTK is really well tested. I didn't want to mess with compilation
> arguments before testing other options. Regardless, I did try a manual
> link with -Wl,-single_module and it didn't work -- I got different
> errors. I could try -fno-common but that would have to be done for the
> Python build and not the VTK build. It appears that the problem is not
> with VTK but the Python build since the linking fails for the Python
> library. The VTK libraries are linked to each other quite nicely.
on OSX are Framework builds. Adding some custom flags like -fno-common
and/or -fPIC wouldn't be a problem for Sage IMHO.
I am aware of some other problems with for example dyanmic python
libraries with modern toolchains on Linux since they do require -fPIC.
This problem seems to be unresolved in Python 2.5, but I think
something has been done about this in Python 2.6.
> >> I guess I could retry with a non-framework build but it really takes a
> >> while to build on my machine and all previous attempts failed.
>
> > Sure. Let me know if the above works.
>
> I've tried the above and a few other things too. I'm still all ears
> though, so keep the suggestions coming.
the time to do that now :(.
<SNIP>
> > Well, it doesn't matter for builds from source too much, the real
> > problem is when a user runs -upgrade for Sage.
>
> Sorry for being dense. You mean when the user runs -upgrade with a
> non-framework install? Suppose a user has a fresh install, would the
> framework build cause problems when this user does a -upgrade?
> IOW, is this only a problem for those with older non-framework builds?
else: When you build python as framework without ever having build it
as non-framework there are some problems. They are fixable, but so far
I have not been convinced that building the Python in Sage as a
framework solves more problems that it causes.
> Thanks.
>
> cheers,
> prabhu
Cheers,
Michael
Brian Granger
build is actually needed. A while back I created an spkg for qt/pyqt
and I remember that I needed to do a framework build to get it to
work. My recollection is that if you want Python to be able to do
anything with the native Mac GUI, it needs to be a framework build.
If this is indeed the case, I think it would be a great idea to have
the Sage python be a framework build and I have an older spkg around
somewhere that does this without any bugs (that I know of ;-)). I
know that in SPD, we would like to be able to have an spkgs for
matplotlib that includes real GUI support. Plus I know that some
folks are obviously linterested in VTK, Mayavi stuff as well.
I will dig up my framework python spkg and get back after I hear from
the pythonmac-sig folks.
Cheers,
Brian
mabshoff
On May 13, 9:46 am, Brian Granger <ellisonbg....@gmail.com> wrote:
Hi Brian,
> I just pinged the pythonmac-sig group about why and when a framework
> build is actually needed. A while back I created an spkg for qt/pyqt
> and I remember that I needed to do a framework build to get it to
> work. My recollection is that if you want Python to be able to do
> anything with the native Mac GUI, it needs to be a framework build.
> If this is indeed the case, I think it would be a great idea to have
> the Sage python be a framework build and I have an older spkg around
> somewhere that does this without any bugs (that I know of ;-)).
This can and will be fixed, but if I have learned one thing about
FrameWorks on OSX is to avoid them whenever possible, i.e that
absolute crap issue with the IOKit and libpng has scared me for
life ;)
Cheers,
Michael
Brian Granger
> This can and will be fixed, but if I have learned one thing about
> FrameWorks on OSX is to avoid them whenever possible, i.e that
> absolute crap issue with the IOKit and libpng has scared me for
> life ;)
What are the current issues with the framework build? A while back I
had a framework python build in sage that worked fine. But maybe
there was something I was missing or that I didn't take into account.
Also, what libpng and IOkit issues are you referring to?
Cheers,
Brian
mabshoff
On May 14, 12:42 am, Brian Granger <ellisonbg....@gmail.com> wrote:
> > As is Sage doesn't even build if you do a straight up framework build.
> > This can and will be fixed, but if I have learned one thing about
> > FrameWorks on OSX is to avoid them whenever possible, i.e that
> > absolute crap issue with the IOKit and libpng has scared me for
> > life ;)
>
> What are the current issues with the framework build?
> A while back I
> had a framework python build in sage that worked fine. But maybe
> there was something I was missing or that I didn't take into account.
> Also, what libpng and IOkit issues are you referring to?
IOKit somewhere. For various reasons Apple's linker decides to use the
libpng from the IOKit when frameworks are involved. The solution was
fixed by using libpng12.dylib in Sage and fixing all spkgs to link
against that dylib.
To this day it pisses me off to no end that Apple was so idiotic to
modify a library like libpng in an incompatible way and not send the
fixes upstream/rename the library. Another thing about libpng by
Apple: They provide libPng.h - notice the difference? That leads to
all kinds of fun when using filesystems with non-default properties
either by Apple or god forbid something like an NFS mount from a non-
Apple server. Sigh.
> Cheers,
>
> Brian
Cheers,
Michael
Prabhu Ramachandran
> I just pinged the pythonmac-sig group about why and when a framework
> build is actually needed. A while back I created an spkg for qt/pyqt
> and I remember that I needed to do a framework build to get it to
> work. My recollection is that if you want Python to be able to do
> anything with the native Mac GUI, it needs to be a framework build.
> If this is indeed the case, I think it would be a great idea to have
> the Sage python be a framework build and I have an older spkg around
> somewhere that does this without any bugs (that I know of ;-)). I
This explains why my build of wxPython worked at all! I had done this
after making the modifications to create a framework build. I haven't
tried without the framework build.
> know that in SPD, we would like to be able to have an spkgs for
> matplotlib that includes real GUI support. Plus I know that some
> folks are obviously linterested in VTK, Mayavi stuff as well.
Yes, indeed, my humble vote in addition for a framework build.
cheers,
prabhu
Prabhu Ramachandran
>>> Well, it doesn't matter for builds from source too much, the real
>>> problem is when a user runs -upgrade for Sage.
>> Sorry for being dense. You mean when the user runs -upgrade with a
>> non-framework install? Suppose a user has a fresh install, would the
>> framework build cause problems when this user does a -upgrade?
>> IOW, is this only a problem for those with older non-framework builds?
>
> Sorry, I wasn't quite correct - at least I wanted to write something
> else: When you build python as framework without ever having build it
> as non-framework there are some problems. They are fixable, but so far
> I have not been convinced that building the Python in Sage as a
> framework solves more problems that it causes.
OK, I will try this sometime and see if I run into difficulties.
cheers,
prabhu
Ondrej Certik
>
> I just pinged the pythonmac-sig group about why and when a framework
> build is actually needed. A while back I created an spkg for qt/pyqt
> and I remember that I needed to do a framework build to get it to
> work. My recollection is that if you want Python to be able to do
> anything with the native Mac GUI, it needs to be a framework build.
> If this is indeed the case, I think it would be a great idea to have
> the Sage python be a framework build and I have an older spkg around
> somewhere that does this without any bugs (that I know of ;-)). I
> know that in SPD, we would like to be able to have an spkgs for
> matplotlib that includes real GUI support. Plus I know that some
> folks are obviously linterested in VTK, Mayavi stuff as well.
>
> I will dig up my framework python spkg and get back after I hear from
> the pythonmac-sig folks.
Could you please share it? I have created sip (sip is something like
swig, needed to built pyqt) and pyqt packages here:
http://femhub.googlecode.com/files/sip.spkg
http://femhub.googlecode.com/files/pyqt.p1.spkg
the sip is 0.5MB, pyqt is 1MB, sip builds instantly, pyqt in about 10
min on my laptop and 7 min on my workstation (not in parallel).
So it nicely works on linux, however on mac, the sip package refuses
to build, because python is not built with framework support.
I will use pyqt using traits-qt backend for my gui applications.
VTK also doesn't build without the framework support, but we'll look
at it at scipy09 with Prabhu, so maybe we'll fix it. however, from
what I understood above, there is no way to fix pyqt to work with Sage
on Mac, unless we use framework.
Am I missing something? If that is the case, then it's an absolute
showstopper for me, if I can't use Sage as a framework for our gui
apps. I don't have time thought to invest it into porting things to
mac, so for the time being I chose the strategy to disable packages
that just don't build on Mac in our distribution (femhub.org) and
that's it.
Also, what is the state of the windows port of Sage --- will it be
possible to just maintain one spkg and it would build on windows, or
will it be necessary to maintain one spkg for linux/mac and another
one for windows?
Ondrej
William Stein
At present the Windows port spkg's are completely different than on Linux/mac.
Obviously that's not what we want in the long run. However, things
are *very* different in Windows land than UNIX land, and something
that works under UNIX isn't going to "just work" under native Windows.
-- William
bsdz
> On Mon, Aug 3, 2009 at 12:02 PM, Ondrej Certik<ond...@certik.cz> wrote:
>
spkg-windows.bat script in the root of your spkg and following the
simple set of rules set out in the Windows' port "spkg\examples\"
folder (renamed in next release to "local\build_lib\templates\") then
you're mostly there. However, you will still need to make sure your
package builds with Visual Studio and/or nmake.
Ondrej Certik
>> Also, what is the state of the windows port of Sage --- will it be
>> possible to just maintain one spkg and it would build on windows, or
>> will it be necessary to maintain one spkg for linux/mac and another
>> one for windows?
>
> At present the Windows port spkg's are completely different than on Linux/mac.
> Obviously that's not what we want in the long run. However, things
> are *very* different in Windows land than UNIX land, and something
> that works under UNIX isn't going to "just work" under native Windows.
Thanks for the clarification. Unfortunately I can't help with the
windows port in the next couple weeks.
My biggest problem is that if I want to try something on windows, I
need to bootup my virtualbox with windows 7, then the net connection
usually doesn't work, so I need to restart windows and it just takes
too much time. On mac, I just log to bsd.math and quickly
port/try/test something, it only takes a minute. Another problem is
that I am very slow doing any changes in the windows terminal, I can't
easily copy things in and out of the virtualbox etc.
Is there some way to install openssh into windows, so that I can leave
it running on my workstation somewhere and just use my terminal + ssh
to log into there and port to windows using this approach? That would
make me way more effective and I could help more with the porting and
testing my code in windows.
Ondrej
bsdz
run the sshd-config script (from memory - so please check).
Also http://www.rdesktop.org/ might allow you to connect to the
virtual server using RDP. A bit like VNC.
Prabhu Ramachandran
>> I will dig up my framework python spkg and get back after I hear from
>> the pythonmac-sig folks.
>
>
> Could you please share it? I have created sip (sip is something like
> swig, needed to built pyqt) and pyqt packages here:
Ondrej, I put up a simple spkg and William kindly put it up as a nice
ticket along with all the resulting problems:
http://trac.sagemath.org/sage_trac/ticket/3924
Unfortunately, I have not had the time to look into this at all for the
last several weeks.
cheers,
prabhu
Ondrej Certik
Thanks. That sounds like lots of stuff to fix. I think windows is a
higher priority for me now. Ironically, people who only use C++ in my
group already make nice windows packages, because they just ship one
binary and one or two libs. Me, as python user, need to ship lots of
stuff and libraries.
Ondrej
Ondrej Certik
Thanks for sharing it. I will investigate both options. I am positive
that there must be some solution so that I can connect to in in 5
seconds and start hacking in 10 seconds and do everything from my
(ubuntu) desktop.
I will report later.
Ondrej
William Stein
isn't in Cygwin, so I'm not sure if it helps.
I'm surprised by how bad you say VirtualBox support for Windows is.
Using Windows via Parallels or VMware (which I've done) is pretty nice
-- cut and paste, networking, etc., all typically works perfectly.
Incidentally, I was literally about to install Windows Vista on
Virtual Box, since I've been using Linux on Virtual Box (on my mac)
lately and really like it. I hope my experience isn't as bad as for
you.
> Also http://www.rdesktop.org/ might allow you to connect to the
> virtual server using RDP. A bit like VNC.
> >
>
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
Ondrej Certik
Yeah, that's what I thought. But maybe I can at least use it to test
the spkg packages. Maybe not, I don't know.
>
> I'm surprised by how bad you say VirtualBox support for Windows is.
> Using Windows via Parallels or VMware (which I've done) is pretty nice
> -- cut and paste, networking, etc., all typically works perfectly.
> Incidentally, I was literally about to install Windows Vista on
> Virtual Box, since I've been using Linux on Virtual Box (on my mac)
> lately and really like it. I hope my experience isn't as bad as for
> you.
Try it, it's not that bad. The network works for me in about 50% of
cases, etc. It could be a bug in Windows 7, that I use, I don't know.
But it's not as fast as just sshing into some box. So I want to figure
it out somehow.
Ondrej