[Numpy-discussion] help from OS X 10.5 users wanted
Ralf Gommers
There have been many reports from OS X 10.5(.8) users that the provided binaries on Sourceforge for numpy versions >= 1.4.0 do not work, see http://projects.scipy.org/numpy/ticket/1322. If you're an OS X 10.5 user I would like your help to test if the same issue occurs for the following binary:
http://sourceforge.net/projects/numpy/files/Temporary/numpy-2.0.0.dev-py2.6-osx10.5-python.org.dmg/download
Please comment on the above ticket.
Thanks,
Ralf
Vincent Davis
with some testing. I have a macbookpro that does not turn of (Hardware
issue) but it is good for testing. I could setup remote access on this
if of interest to you.
Is there anything like the Python devs build bot for automated install
and testing testing of numpy and scipy?
Vincent
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Di...@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
--
Thanks
Vincent Davis
720-301-3003
_______________________________________________
NumPy-Discussion mailing list
NumPy-Di...@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Ralf Gommers
Did you get any responses on this? I can install 10.5 and help out
with some testing. I have a macbookpro that does not turn of (Hardware
issue) but it is good for testing. I could setup remote access on this
if of interest to you.
Yes I got one response at http://projects.scipy.org/numpy/ticket/1322, a negative one unfortunately. My conclusion is that binaries built on 10.6 do not work on 10.5 even when using the correct SDK and deployment target. A lot of software seems to have this same problem. Backward compatibility on OS X is good, but forward compatibility is not.
Sage distributes separate binaries for each OS X version, presumably compiled on those same versions. If we want to have a 10.5 binary we should just do the same. However, gives the lack of responses I'm not sure how much of a need there is for 10.5 binaries (please speak up ...).
The binary naming scheme should also change to include the version.
Is there anything like the Python devs build bot for automated install
and testing testing of numpy and scipy?
There's a numpy buildbot: http://buildbot.scipy.org/, but it does not build/install binaries. I also just noticed it has not been updated for the switch to git - perhaps an indication of how much attention we're paying to it.
Ralf
Friedrich Romstedt
> Did you get any responses on this? I can install 10.5 and help out
> with some testing. I have a macbookpro that does not turn of (Hardware
> issue) but it is good for testing. I could setup remote access on this
> if of interest to you.
I can also help with the installer - I have some (some) experience
with building Mac OS X installers using the PackageMaker provided by
Apple. Just lacking a 10.5. But since I need some anyway (for
controlling a 10.5 server), Vincent, if you don't need your 10.5
anymore, can we transfer the license in some way from you to me? I'm
serious, one cannot buy 10.5 from Apple anymore, and I need a legal
license. I have 10.6 and a VMware Fusion v3.
When anyone can inform me how the installation scheme for numpy
binaries is I can then provide the installers, I believe. I strongly
support 10.5 support, I believe we should support at least the next to
last version.
For my own installer for upy, I followed the route: Unpacking the
package into some /private/var/tmp directory, and running setup.py
install there (since we are root when installing). upy is pure
Python, no compilation. I see so far three routes for numpy: a) just
installing the precompiled binaries using a setup.py file, b)
compiling in the background for the user (shouldn't be a problem on
Mac OS X, and would give us opportunity to include support for
complementary packages in a "binary installer". Tough it wouldn't be
really binary anymore.) c) Hardcoding the /Frameworks/ directory and
simply copying.
What do we like best?
Friedrich
Vincent Davis
<friedric...@gmail.com> wrote:
> 2010/10/9 Vincent Davis <vin...@vincentdavis.net>:
>> Did you get any responses on this? I can install 10.5 and help out
>> with some testing. I have a macbookpro that does not turn of (Hardware
>> issue) but it is good for testing. I could setup remote access on this
>> if of interest to you.
>
> I can also help with the installer - I have some (some) experience
> with building Mac OS X installers using the PackageMaker provided by
> Apple. Just lacking a 10.5. But since I need some anyway (for
> controlling a 10.5 server), Vincent, if you don't need your 10.5
> anymore, can we transfer the license in some way from you to me? I'm
> serious, one cannot buy 10.5 from Apple anymore, and I need a legal
> license. I have 10.6 and a VMware Fusion v3.
Let me see what I can do about getting you a license. Can you install
regular osx (not server) as a vm machine ? I have 10.5 server running
on vm.
Vincent
>
> When anyone can inform me how the installation scheme for numpy
> binaries is I can then provide the installers, I believe. I strongly
> support 10.5 support, I believe we should support at least the next to
> last version.
>
> For my own installer for upy, I followed the route: Unpacking the
> package into some /private/var/tmp directory, and running setup.py
> install there (since we are root when installing). upy is pure
> Python, no compilation. I see so far three routes for numpy: a) just
> installing the precompiled binaries using a setup.py file, b)
> compiling in the background for the user (shouldn't be a problem on
> Mac OS X, and would give us opportunity to include support for
> complementary packages in a "binary installer". Tough it wouldn't be
> really binary anymore.) c) Hardcoding the /Frameworks/ directory and
> simply copying.
>
> What do we like best?
>
> Friedrich
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Di...@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
--
Thanks
Vincent Davis
720-301-3003
Ralf Gommers
2010/10/9 Vincent Davis <vin...@vincentdavis.net>:
> Did you get any responses on this? I can install 10.5 and help outI can also help with the installer
> with some testing. I have a macbookpro that does not turn of (Hardware
> issue) but it is good for testing. I could setup remote access on this
> if of interest to you.
That would be very helpful, thanks. Please keep me up to date on your progress with the 10.5 license, and once you are set up we can coordinate building the binaries.
- I have some (some) experience
with building Mac OS X installers using the PackageMaker provided by
Apple. Just lacking a 10.5. But since I need some anyway (for
controlling a 10.5 server), Vincent, if you don't need your 10.5
anymore, can we transfer the license in some way from you to me? I'm
serious, one cannot buy 10.5 from Apple anymore, and I need a legal
license. I have 10.6 and a VMware Fusion v3.
When anyone can inform me how the installation scheme for numpy
binaries is I can then provide the installers, I believe.
The toolchain you need is documented at http://projects.scipy.org/numpy/wiki/MakingReleases. If you have all the dependencies it's simply a matter of
$ paver dmg -p 2.5 (or 2.6/7)
and a dmg installer is built.
I strongly
support 10.5 support, I believe we should support at least the next to
last version.
For my own installer for upy, I followed the route: Unpacking the
package into some /private/var/tmp directory, and running setup.py
install there (since we are root when installing). upy is pure
Python, no compilation. I see so far three routes for numpy: a) just
installing the precompiled binaries using a setup.py file, b)
compiling in the background for the user (shouldn't be a problem on
Mac OS X, and would give us opportunity to include support for
complementary packages in a "binary installer". Tough it wouldn't be
really binary anymore.) c) Hardcoding the /Frameworks/ directory and
simply copying.
Cheers,
Ralf
Vincent Davis
Is there any reason 10.5 server addition would not work for building release ?
Friedrich, you can't just install 10.5 (non server) on vmware. I just
tried, It responds This is not a server addition. There are hacks
but...
I'll try to do a few testes this week., Where does
"numpy-macosx-installer "
http://github.com/cournape/numpy-macosx-installer fit in to running
this command. "paver dmg -p 2.5" or does it? I have installed from
source many times but never built a dmg.
Is there any interest in a current Dev snapshot dmg?
And how about a numpy scipy combo dmg?
A little bit of a separate issue, does the build bot or so,ething/one
other than developers run numpy.test() to monitor test that may fail
on different systems, as numpy scipy, python get updated. It seems
like a bot could build everything from source weekly and commit a test
log via git/github to monitor and record the condition and changes in
tests. Maybe there is no point to this, just sound like a neat way to
track test results.
Vincent Davis
720-301-3003
>
>>
>> I strongly
>> support 10.5 support, I believe we should support at least the next to
>> last version.
>>
>> For my own installer for upy, I followed the route: Unpacking the
>> package into some /private/var/tmp directory, and running setup.py
>> install there (since we are root when installing). upy is pure
>> Python, no compilation. I see so far three routes for numpy: a) just
>> installing the precompiled binaries using a setup.py file, b)
>> compiling in the background for the user (shouldn't be a problem on
>> Mac OS X, and would give us opportunity to include support for
>> complementary packages in a "binary installer". Tough it wouldn't be
>> really binary anymore.) c) Hardcoding the /Frameworks/ directory and
>> simply copying.
>>
> The way it works is (c), so for the binaries the installers from python.org
> are the one we build against. The dmg contains an mpkg plus built docs. Of
> your other options, (a) would be similar only less user-friendly, (b) is a
> very bad idea.
>
> Cheers,
> Ralf
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Di...@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
--
Thanks
Vincent Davis
720-301-3003
Ralf Gommers
This now lives under tools/numpy-macosx-installer in the git repo, so that is not an external dependency anymore.
I have installed from
source many times but never built a dmg.
Is there any interest in a current Dev snapshot dmg?
Probably not. The master branch is not binary compatible with scipy. But 1.5.1rc1 is planned for this coming Sunday, so that would be useful to have a 10.5 binary of.
And how about a numpy scipy combo dmg?
Not sure if that's a very good way to spend your time, what's wrong with separate installers? The release cycles are not synchronized, so this would just be extra work.
A little bit of a separate issue, does the build bot or so,ething/one
other than developers run numpy.test() to monitor test that may fail
on different systems, as numpy scipy, python get updated.
It does, just go to Waterfall display, then click one of the "stdio" links to see test results.
Cheers,
Ralf
Vincent Davis
Ok, Friedrich and I are working on setting up a machine for building
on 10.5 and hope (I do) to have that going for 1.5.1rc1
I hope to do a 10.6 for 1.5rc1 also.
>
>> And how about a numpy scipy combo dmg?
>
> Not sure if that's a very good way to spend your time, what's wrong with
> separate installers? The release cycles are not synchronized, so this would
> just be extra work.
Or a learning experience :-) which is always work.
Thanks
Vincent
>>
>> A little bit of a separate issue, does the build bot or so,ething/one
>> other than developers run numpy.test() to monitor test that may fail
>> on different systems, as numpy scipy, python get updated.
>
> It does, just go to Waterfall display, then click one of the "stdio" links
> to see test results.
>
> Cheers,
> Ralf
>
>
>>
>> It seems
>> like a bot could build everything from source weekly and commit a test
>> log via git/github to monitor and record the condition and changes in
>> tests. Maybe there is no point to this, just sound like a neat way to
>> track test results.
>>
>
Christopher Barker
> Yes I got one response at http://projects.scipy.org/numpy/ticket/1322, a
> negative one unfortunately.
For what it's worth, the posted binary seems to work fine on my OS-X
10.5 PPC system (python2.6), so maybe the PPC part works, the Intel part
not.
> My conclusion is that binaries built on 10.6
> do not work on 10.5 even when using the correct SDK and deployment
> target.
Well, I still think if you do everything right, it works, but yes, it is
a pain to get right, and a pain to test. Building on the oldest system
you want to support is a lot easier.
> Sage distributes separate binaries for each OS X version, presumably
> compiled on those same versions. If we want to have a 10.5 binary we
> should just do the same.
I don't think so -- that's a lot to maintain, and the older versions
work on the newer OS -- so why build the newer?
Also a lot (OK -- some, I have no idea how many) of us re-distribute,
via py2app, and so need Universal binaries that work on all the systems
we need to support.
10.4 or greater would be great -- basically matching the python.org builds.
> The binary naming scheme should also change to include the version.
yup.
Thanks for working in this!
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Christopher Barker
> Ok, Friedrich and I are working on setting up a machine for building
> on 10.5 and hope (I do) to have that going for 1.5.1rc1
> I hope to do a 10.6 for 1.5rc1 also.
why have a separate 10.6 build?
And have we given up on 10.4?
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Vincent Davis
<Chris....@noaa.gov> wrote:
> Vincent Davis wrote:
>> Ok, Friedrich and I are working on setting up a machine for building
>> on 10.5 and hope (I do) to have that going for 1.5.1rc1
>> I hope to do a 10.6 for 1.5rc1 also.
>
> why have a separate 10.6 build?
>
> And have we given up on 10.4?
I could do 10.4 also. For me doing this is maybe best described as a
curiosity. But my understanding was that some where having trouble
install on 10.5 with binaries made on 10.6. So if I can learn
something and help others that is great.
Therefore I am interested in any input or advice you have.
So what exacty do you think should be done?
Thanks
Vincent
>
> -Chris
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris....@noaa.gov
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Di...@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
--
Thanks
Vincent Davis
720-301-3003
Christopher Barker
> I could do 10.4 also. For me doing this is maybe best described as a
> curiosity. But my understanding was that some where having trouble
> install on 10.5 with binaries made on 10.6. So if I can learn
> something and help others that is great.
Well, I appreciate the effort.
> Therefore I am interested in any input or advice you have.
>
> So what exacty do you think should be done?
Ideally, I think the Mac numpy binaries should match the python.org
python binaries:
Python 2.5: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
Python 2.6: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
Python 2.7 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
Python 2.7 PPC/i386/x86-64 for Mac OS X 10.5 or later
Python 3 ??? (similar I think)
Supporting 10.3.9 may be a bit much these days, so it's probably easiest
to build on a 10.4 machine, except for the 2.7 64bit build on 10.5.
In theory, it should all be buildable on 10.6, but I know I've struggled
with doing that sort of thing -- but maybe the macpython folks could help.
In the past, numpy builds have "just worked", so maybe it's as simple as
finding a 10.4 machine to build on (I don't have one anymore, we're not
allowed to have them on our network, since Apple stopped supplying
security patches)
Thanks,
Vincent Davis
<Chris....@noaa.gov> wrote:
> On 10/12/10 2:00 PM, Vincent Davis wrote:
>> I could do 10.4 also. For me doing this is maybe best described as a
>> curiosity. But my understanding was that some where having trouble
>> install on 10.5 with binaries made on 10.6. So if I can learn
>> something and help others that is great.
>
> Well, I appreciate the effort.
>
>> Therefore I am interested in any input or advice you have.
>>
>> So what exacty do you think should be done?
>
> Ideally, I think the Mac numpy binaries should match the python.org
> python binaries:
>
>
> Python 2.5: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>
> Python 2.6: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>
> Python 2.7 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>
> Python 2.7 PPC/i386/x86-64 for Mac OS X 10.5 or later
>
> Python 3 ??? (similar I think)
Sound good to me but I am not the one that makes the decision, or
maybe I do since it is my computer :) but I am not the one that
uploads it to sourceforge and makes it official.
It's been a long time since I installed from a binary. Are all the
current osx version 32bit only, I don't see anything about 32/64 bit.
Vincent
>
> Supporting 10.3.9 may be a bit much these days, so it's probably easiest
> to build on a 10.4 machine, except for the 2.7 64bit build on 10.5.
>
> In theory, it should all be buildable on 10.6, but I know I've struggled
> with doing that sort of thing -- but maybe the macpython folks could help.
>
> In the past, numpy builds have "just worked", so maybe it's as simple as
> finding a 10.4 machine to build on (I don't have one anymore, we're not
> allowed to have them on our network, since Apple stopped supplying
> security patches)
>
> Thanks,
>
> -Chris
>
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris....@noaa.gov
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Di...@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
--
Thanks
Vincent Davis
720-301-3003
Ralf Gommers
Ralf Gommers wrote:For what it's worth, the posted binary seems to work fine on my OS-X
> Yes I got one response at http://projects.scipy.org/numpy/ticket/1322, a
> negative one unfortunately.
10.5 PPC system (python2.6), so maybe the PPC part works, the Intel part
not.
> My conclusion is that binaries built on 10.6Well, I still think if you do everything right, it works, but yes, it is
> do not work on 10.5 even when using the correct SDK and deployment
> target.
a pain to get right, and a pain to test. Building on the oldest system
you want to support is a lot easier.
I don't think so - at least I have not been able to find a fix for the error below in the many many bug reports one can find:
ImportError?: dlopen(somefile.so, 2): no suitable image found. Did find:
somefile.so: unknown required load command 0x80000022
I don't think so -- that's a lot to maintain, and the older versions
> Sage distributes separate binaries for each OS X version, presumably
> compiled on those same versions. If we want to have a 10.5 binary we
> should just do the same.
work on the newer OS -- so why build the newer?
Good point. I was starting from the assumption I would be building on 10.6 anyway, since that's the only version I have available myself.
Ralf Gommers
Vincent Davis wrote:why have a separate 10.6 build?
> Ok, Friedrich and I are working on setting up a machine for building
> on 10.5 and hope (I do) to have that going for 1.5.1rc1
> I hope to do a 10.6 for 1.5rc1 also.
And have we given up on 10.4?
10.4 works with current binaries AFAIK.
Ralf
Christopher Barker
> For what it's worth, the posted binary seems to work fine on my OS-X
> 10.5 PPC system (python2.6), so maybe the PPC part works, the Intel part
> not.
>
> Thanks, good to know.
>
> > My conclusion is that binaries built on 10.6
> > do not work on 10.5 even when using the correct SDK and deployment
> > target.
It sounds like they work on everything except Intel 10.5 -- is that right?
> Well, I still think if you do everything right, it works, but yes, it is
> a pain to get right, and a pain to test. Building on the oldest system
> you want to support is a lot easier.
>
> I don't think so - at least I have not been able to find a fix for the
> error below in the many many bug reports one can find:
>
> ImportError?: dlopen(somefile.so, 2): no suitable image found. Did find:
>
> somefile.so: unknown required load command 0x80000022
Well, there are two options now:
1) keep trying to fix that error -- a post to the pythonmac list might
help there.
2) Punt, and just build on an older system. It sounds like Vincent may
be able to help out with that.
(I've got a Intel 10.6 system and a PPC 10.5 system now, but I'll
probably lose the 10.5 one soon)
On 10/12/10 6:12 PM, Vincent Davis wrote:
>>> install on 10.5 with binaries made on 10.6. So if I can learn
>>> something and help others that is great.
Vincent -- if you are game, I suggest:
Set up a 10.4 system, install the python.org 2.5, 2.6 and 32bit 2.7
binaries.
On that, build:
>> Python 2.5: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>>
>> Python 2.6: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>>
>> Python 2.7 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
On a 10.5 system, build:
>> Python 2.7 PPC/i386/x86-64 for Mac OS X 10.5 or later
As for Python 3k -- I have no idea!
I might be able to do the 10.5 builds -- but I don't know for how long.
Do you have the machines (or VMs) to do that?
Thanks so much!
NOTE: are there buildbots somewhere that could be used for this?
Vincent Davis
No it's an injured MacBook Pro. Installing on different partitions
10.5 and 10.6 need to look for 10.4. Its an intel and I think only
some 10.4 DVDs work, I think some only have the ppc version.
Friedrich and I are working on it. We just got a vpn setup for him to
connect to the computer. I am in Colorado and he is in Germany. I hope
to do some builds. What is the best way to test them? Try to install
and run numpy.test() ?
Vincent
Vincent
>
> Thanks so much!
>
> NOTE: are there buildbots somewhere that could be used for this?
>
> -Chris
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris....@noaa.gov
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Di...@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
--
Thanks
Vincent Davis
720-301-3003
Friedrich Romstedt
> On 10/12/10 2:00 PM, Vincent Davis wrote:
>> I could do 10.4 also. For me doing this is maybe best described as a
>> curiosity. But my understanding was that some where having trouble
>> install on 10.5 with binaries made on 10.6. So if I can learn
>> something and help others that is great.
>
> Well, I appreciate the effort.
Thanks a lot.
> Ideally, I think the Mac numpy binaries should match the python.org
> python binaries:
>
> Python 2.5: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
> Python 2.6: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
> Python 2.7 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
> Python 2.7 PPC/i386/x86-64 for Mac OS X 10.5 or later
> Python 3 ??? (similar I think)
>
> Supporting 10.3.9 may be a bit much these days, so it's probably easiest
> to build on a 10.4 machine, except for the 2.7 64bit build on 10.5.
We have to decide about how to install Python on the machines. It
aren't VMs, as one cannot virtualise workstation Mac OS X using
acceptable tools.
The most straightforward way would be to just install the python.org
binaries available. Note that Python 2.5 *must* be compiled on Mac,
there is no binary on python.org.
I would like to discuss the feasibility of compiling numpy against a
host-built Python built from the sources from python.org. In this
case I would compile including the x86_64 arch, if it is available on
10.5 at all? Could both users of a self-built Python (e.g. 2.6 on
10.6, this would be my case), use this precompiled numpy then, as well
as users using the python.org binaries? It would enable users to use
x86_64 arch without self-compiling.
Is there a check if the python.org installer was run and is registered
in the computer's package database?
Ralf, what is the advantage of hard-coding the destination directory
over running a setup.py script as postflight in the package installer?
(I understood that background-compilation is really not such a bright
idea.) The hidden setup.py would be compatible with all installation
locations of the user's Python.
> In theory, it should all be buildable on 10.6, but I know I've struggled
> with doing that sort of thing -- but maybe the macpython folks could help.
I would like to concentrate first on the 10.5 machine of Vincent.
I had the attitude to set up an openvpn first, because I personally
don't like to use 3rd party servers when transferring passwords. The
VPN is up and running, and the authentication is PKI based.
This are the open questions:
1) Installation scheme of the different Pythons on the packager
machine (binary or host-compiled)
2) Layout of the installer (hardcoded or hidden setup.py)
Friedrich
Christopher Barker
> 2010/10/13 Christopher Barker<Chris....@noaa.gov>:
>> Ideally, I think the Mac numpy binaries should match the python.org
>> python binaries:
>>
>> Python 2.5: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>> Python 2.6: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>> Python 2.7 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>> Python 2.7 PPC/i386/x86-64 for Mac OS X 10.5 or later
>> Python 3 ??? (similar I think)
> We have to decide about how to install Python on the machines. It
> aren't VMs, as one cannot virtualise workstation Mac OS X using
> acceptable tools.
there is no problem with multiple python versions being installed at
once on one machine -- only one is "Current", but all you need to do to
build for others is specify:
python2.5 setup.py build
python2.6 setup.py build
etc...
> The most straightforward way would be to just install the python.org
> binaries available. Note that Python 2.5 *must* be compiled on Mac,
> there is no binary on python.org.
there is one for 2.5.4, but not for 2.5.5 -- in fact, it looks like
there are NO binaries for 2.5.5 on python.org. I don't see any reason to
worry about it thought, 2.5.4 should be fine -- that's what people are
likely to be running anyway.
> I would like to discuss the feasibility of compiling numpy against a
> host-built Python built from the sources from python.org.
What's the point? Anyone that builds their own python can build their
own numpy, the point of the binaries is to target the less experienced user.
> In this
> case I would compile including the x86_64 arch, if it is available on
> 10.5 at all? Could both users of a self-built Python (e.g. 2.6 on
> 10.6, this would be my case), use this precompiled numpy then, as well
> as users using the python.org binaries? It would enable users to use
> x86_64 arch without self-compiling.
I suppose so, but only if you distribute the python binary too -- I"d
say if you want 64bit, use 2.7. I think Ronald made a pretty good choice
with the way he set up the binaries on python.org.
> Is there a check if the python.org installer was run and is registered
> in the computer's package database?
I have not idea how to do that -- but OS_X's "package manager" is really
pretty lame -- it's not really a package manager, really just an installer.
> Ralf, what is the advantage of hard-coding the destination directory
> over running a setup.py script as postflight in the package installer?
> (I understood that background-compilation is really not such a bright
> idea.) The hidden setup.py would be compatible with all installation
> locations of the user's Python.
if you are going to do that, you might as well have them download a
tarball and "setup.py install", or use a egg. The goal of the binaries
is to be, well, binary, and be an easy point&click install. If you use
macport or fink for your python, use them for numpy, if you build your
own python, build your own numpy.
In the MacPython community, we are trying (though largely failing!) to
be able to recommend one simple way to do python on the Mac: the
python.org installers, and binary packages built for them.
There are two fairly distinct user groups for OS-X: The unix-y folks
that use the command line compile stuff for themselves, etc, and the Mac
folks that excepct point+click installers -- the point of the binaries
is to target the second group.
> I would like to concentrate first on the 10.5 machine of Vincent.
but can we build on 10.5 and have it work on 10.4? In theory yes, but
then in theory we can build on 10.6, too...
> 1) Installation scheme of the different Pythons on the packager
> machine (binary or host-compiled)
see above -- host compiled really isn't the point -- there are already
ways to do that (by the way, not every python user even has the compiler
installed)
> 2) Layout of the installer (hardcoded or hidden setup.py)
I don't know if you can embed a setup.py in a mpkg, but I wouldn't
bother, with all the binary compatibility issues, this will be an
installer designed for a particular python install, anyway.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Christopher Barker
>> Do you have the machines (or VMs) to do that?
>
> No it's an injured MacBook Pro. Installing on different partitions
> 10.5 and 10.6 need to look for 10.4. Its an intel and I think only
> some 10.4 DVDs work, I think some only have the ppc version.
what a pain -- how I long for open source....But I think I have a 104
Intel DVD at home, let me know if I should look for it. (I'm running
10.5 on that machine, now)
> What is the best way to test them? Try to install
> and run numpy.test() ?
That's about all I know how to do.
-CHB
Friedrich Romstedt
normal route and make a working solution. So we're going to use the
standard (in the best sense) python.org installers and we'll compile
against them. I'll test the installers on my own 10.6 notebook and on
the rather virgin 10.5 notebook of a colleague, with all the nose
bells'n'whistles.
2010/10/14 Christopher Barker <Chris....@noaa.gov>:
> there is no problem with multiple python versions being installed at
> once on one machine -- only one is "Current", but all you need to do to
> build for others is specify:
>
> python2.5 setup.py build
> python2.6 setup.py build
>
> etc...
good. I guess the paver mentioned before does the rest of the work.
We'll look into it.
>> The most straightforward way would be to just install the python.org
>> binaries available. Note that Python 2.5 *must* be compiled on Mac,
>> there is no binary on python.org.
>
> there is one for 2.5.4, but not for 2.5.5 -- in fact, it looks like
> there are NO binaries for 2.5.5 on python.org. I don't see any reason to
> worry about it thought, 2.5.4 should be fine -- that's what people are
> likely to be running anyway.
Okay thanks.
>> I would like to discuss the feasibility of compiling numpy against a
>> host-built Python built from the sources from python.org.
>
> What's the point? Anyone that builds their own python can build their
> own numpy, the point of the binaries is to target the less experienced user.
I agree.
>> In this
>> case I would compile including the x86_64 arch, if it is available on
>> 10.5 at all? Could both users of a self-built Python (e.g. 2.6 on
>> 10.6, this would be my case), use this precompiled numpy then, as well
>> as users using the python.org binaries? It would enable users to use
>> x86_64 arch without self-compiling.
>
> I suppose so, but only if you distribute the python binary too -- I"d
> say if you want 64bit, use 2.7. I think Ronald made a pretty good choice
> with the way he set up the binaries on python.org.
Just btw, I remember lots of issues with 2.7, but it may have been matplotlib.
>> Is there a check if the python.org installer was run and is registered
>> in the computer's package database?
>
> I have not idea how to do that -- but OS_X's "package manager" is really
> pretty lame -- it's not really a package manager, really just an installer.
Hm, not really, you can specify URLs where to download from, have you
ever looked into the (10.6) PackageMaker? From my impression, it's a
click-and-point package manager. (I.e., one downloads an executable
for each download ...)
>> Ralf, what is the advantage of hard-coding the destination directory
>> over running a setup.py script as postflight in the package installer?
>> (I understood that background-compilation is really not such a bright
>> idea.) The hidden setup.py would be compatible with all installation
>> locations of the user's Python.
>
> if you are going to do that, you might as well have them download a
> tarball and "setup.py install", or use a egg. The goal of the binaries
> is to be, well, binary, and be an easy point&click install. If you use
> macport or fink for your python, use them for numpy, if you build your
> own python, build your own numpy.
Ok, I see, it's just a matter of taste (not mine obviously).
> In the MacPython community, we are trying (though largely failing!) to
> be able to recommend one simple way to do python on the Mac: the
> python.org installers, and binary packages built for them.
Everyone on the numpy list knows this never-ending story ... :-) At
least in principle.
> There are two fairly distinct user groups for OS-X: The unix-y folks
> that use the command line compile stuff for themselves, etc, and the Mac
> folks that excepct point+click installers -- the point of the binaries
> is to target the second group.
Very clear. Thanks.
>> I would like to concentrate first on the 10.5 machine of Vincent.
>
> but can we build on 10.5 and have it work on 10.4? In theory yes, but
> then in theory we can build on 10.6, too...
Nothing against using a 10.4 on Vincent's computer, the "concentrate
on" was meant somehow else, but I don't remember.
We're ready to start now, and will come back when we're stuck somehow.
Friedrich
>> 1) Installation scheme of the different Pythons on the packager
>> machine (binary or host-compiled)
>
> see above -- host compiled really isn't the point -- there are already
> ways to do that (by the way, not every python user even has the compiler
> installed)
>
>> 2) Layout of the installer (hardcoded or hidden setup.py)
>
> I don't know if you can embed a setup.py in a mpkg, but I wouldn't
> bother, with all the binary compatibility issues, this will be an
> installer designed for a particular python install, anyway.
>
> -Chris
Ralf Gommers
On 10/13/10 8:30 AM, Ralf Gommers wrote:It sounds like they work on everything except Intel 10.5 -- is that right?
> For what it's worth, the posted binary seems to work fine on my OS-X
> 10.5 PPC system (python2.6), so maybe the PPC part works, the Intel part
> not.
>
> Thanks, good to know.
>
> > My conclusion is that binaries built on 10.6
> > do not work on 10.5 even when using the correct SDK and deployment
> > target.
> Well, I still think if you do everything right, it works, but yes, it isWell, there are two options now:
> a pain to get right, and a pain to test. Building on the oldest system
> you want to support is a lot easier.
>
> I don't think so - at least I have not been able to find a fix for the
> error below in the many many bug reports one can find:
>
> ImportError?: dlopen(somefile.so, 2): no suitable image found. Did find:
>
> somefile.so: unknown required load command 0x80000022
1) keep trying to fix that error -- a post to the pythonmac list might
help there.
Will try that.
2) Punt, and just build on an older system. It sounds like Vincent may
be able to help out with that.
(I've got a Intel 10.6 system and a PPC 10.5 system now, but I'll
probably lose the 10.5 one soon)
Vincent -- if you are game, I suggest:
On 10/12/10 6:12 PM, Vincent Davis wrote:
>>> install on 10.5 with binaries made on 10.6. So if I can learn
>>> something and help others that is great.
Set up a 10.4 system, install the python.org 2.5, 2.6 and 32bit 2.7
binaries.
On that, build:
On a 10.5 system, build:
>> Python 2.5: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>>
>> Python 2.6: 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
>>
>> Python 2.7 32-bit PPC/i386 for Mac OS X 10.3.9 through 10.6
As for Python 3k -- I have no idea!
>> Python 2.7 PPC/i386/x86-64 for Mac OS X 10.5 or later
I might be able to do the 10.5 builds -- but I don't know for how long.
Do you have the machines (or VMs) to do that?
Thanks so much!
NOTE: are there buildbots somewhere that could be used for this?
Ralf
Christopher Barker
> NOTE: are there buildbots somewhere that could be used for this?
>
> There's only one OS X buildbot
> (http://buildbot.scipy.org/builders/MacOSX_x86_64), but last time it
> connected was in July.
maybe a post to the pythonmac list would help this, too -- someone
recently offered up buildbots for python itself, maybe they'd be willing
to help out scipy, too.
Come to think of it, I have a Intel 10.6 machine I could offer up -- it
may not always have the same ip address, but I keep it up all the time.
Can you point me to docs on how to set it up?
Friedrich Romstedt
15.10.10 05:34:35] Vincent Davis: so you have a dmg now?
[15.10.10 05:34:48] Friedrich Romstedt: I cant believe it, I want to
see the file first
[15.10.10 05:35:06] Friedrich Romstedt: we have an mpkg in dist
[15.10.10 05:35:37] Friedrich Romstedt: yes, we have one
[15.10.10 05:36:01] Friedrich Romstedt: In
Users/Shared/GitHub/project-numpy/owner-numpy/numpy-deployment/tools/numpy-macosx-installer
[15.10.10 05:36:18] Friedrich Romstedt: -rw-r--r--@ 1 Friedrich wheel
8190646 Oct 14 21:33 numpy-2.0.0.dev-py2.5-python.org.dmg
[15.10.10 05:36:29] Friedrich Romstedt: it's unbelievable
[15.10.10 05:37:39] Friedrich Romstedt: Seems I'm too tired to be
happy, but it's enjoyable anyway
:-)
Friedrich
P.S.: Building v1.5.0rc1 failed because of ticket 1596 which seems to
be closed anyway?
This is py2.5 on Mac OS X 10.5.
Christopher Barker
> Vincent and me had some success:
good news.
> [15.10.10 05:36:01] Friedrich Romstedt: In
> Users/Shared/GitHub/project-numpy/owner-numpy/numpy-deployment/tools/numpy-macosx-installer
> [15.10.10 05:36:18] Friedrich Romstedt: -rw-r--r--@ 1 Friedrich wheel
> 8190646 Oct 14 21:33 numpy-2.0.0.dev-py2.5-python.org.dmg
Can I find that somewhere to download and test?
> This is py2.5 on Mac OS X 10.5.
good start.
Thanks again for all your work on this,
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Russell E. Owen
control, but for the record I have a 10.4 Intel machine I keep around
for the sole purpose of building backward-compatible binary installers
for PIL and matplotlib. If help is still wanted I'm happy to build numpy
as well.
I did this because I was not able to figure out how to build matplotlib
or PIL on 10.5 or 10.6 in such a way that it would run on older Macs. (I
made static universal libraries for their dependencies, using the 10.4
SDK, but somehow that was not enough).
So I'm not surprised you are having problems.
Note that the Mac Python 2.5 from python.org has some known issues with
Tkinter so I never use it, but I could install it for this purpose (just
not test it very easily) since numpy doesn't care about Tkinter.
-- Russell
In article <4CB5E208...@noaa.gov>,
Christopher Barker <Chris....@noaa.gov> wrote:
_______________________________________________
Christopher Barker
> I have a 10.4 Intel machine I keep around
> for the sole purpose of building backward-compatible binary installers
> for PIL and matplotlib. If help is still wanted I'm happy to build numpy
> as well.
I'll let Ralf and Friedrich and Vincent respond, but that sounds like a
great option.
And thanks for PIL and MPL, too!
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Vincent Davis
<Chris....@noaa.gov> wrote:
> On 10/15/10 10:54 AM, Russell E. Owen wrote:
>> I have a 10.4 Intel machine I keep around
>> for the sole purpose of building backward-compatible binary installers
>> for PIL and matplotlib. If help is still wanted I'm happy to build numpy
>> as well.
>
> I'll let Ralf and Friedrich and Vincent respond, but that sounds like a
> great option.
Friedrich and myself we are interested in 10.5 and 10.6 first
Christopher, you mentioned that you might have a 10.4 dvd intel that
you could share, I/you can contact me off list. I could not find one.
Christopher we will see about getting you a link to download it. am
working on getting 10.6 installed and repartitioning the drive so no
easy access to the files right now.
Vincent
>
> And thanks for PIL and MPL, too!
>
> -Chris
>
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris....@noaa.gov
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Di...@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
--
Thanks
Vincent Davis
720-301-3003
Friedrich Romstedt
> On 10/15/10 10:54 AM, Russell E. Owen wrote:
>> I have a 10.4 Intel machine I keep around
>> for the sole purpose of building backward-compatible binary installers
>> for PIL and matplotlib. If help is still wanted I'm happy to build numpy
>> as well.
>
> I'll let Ralf and Friedrich and Vincent respond, but that sounds like a
> great option.
Thanks for waiting for us! Here is our status:
Vincent is currently setting up the machine with a crucial
carbon-copy-clone, reformat, and ccc back. This is because we had
only around 50 MB left in the end on the partition, and according to
Vincent enlarging the partition wasn't possible for some reason be
both don't know. Well, technicalities.
We're having to set up py2.6 & py2.7, what shouldn't be an issue, and
will have to compile mpl for both of them since the build process
should not change Python version while its running, i.e., when
building the py2.6 dmg, the docs will be built using py2.6, and when
py2.5 dmg, then py2.5, that's what I mean.
Also LaTeX is needed, Russell, as you might know. MacTeX-2010 is
about 1.6 GB download.
Here come now the interesting facts:
1) Some tests of numpy failed in 2.0.0.dev. When the machine is
running again I can send the logs. All some strange-looking typecode
string tests with dtype('...') iirc.
2) I noticed that the paver at some late point tried to switch from
py2.5 to py2.6, what is rather strange to me. I must have a look
where precisely the build failed for this reason. py2.6 is the
DEFAULT_PYTHON_VERSION (iirc) in pavement.py, and changing it to 2.5
fixed it. Strange.
3) I found no v1.5.1 tag yet (yesterday). Will the HEAD become 1.5.1?
Here a comparison of our two systems (Russell's and our's):
* We will have 10.4 (?), 10.5, 10.6 available on the same machine
with vpn access for everyone who wants a cert.
* But we need time to set it up properly. We're unwilling to do
half-baken things, so I agree that Vincent it installing 10.6 right
now (I just got the message), but time is rare this weekend.
* So my suggestion would be, Russell, if you could do the build more
easily then we can, just feel free, I was hoping Vincent and me would
get the credits though ;-), but first it must succeed on time.
Vincent, what do you think?
* For future, I would prefer Vincent's machine. We have dyndns and
the machine can be dedicated for building this stuff. If we get a
10.4, we have 10.4 to 10.6 all together on a single place, and we
could do with it whatever we feel like.
Friedrich
> And thanks for PIL and MPL, too!
Nice to meet you. Btw, I ran into some trouble when compiling mpl on
10.5, different trouble than on 10.6, I'm planning to write it up,
it's solved, because it would be nice to have a *working* instruction
online (different from make.osx, which I dislike for its magic
approach).
Friedrich
Friedrich Romstedt
>> [15.10.10 05:36:01] Friedrich Romstedt: In
>> Users/Shared/GitHub/project-numpy/owner-numpy/numpy-deployment/tools/numpy-macosx-installer
>> [15.10.10 05:36:18] Friedrich Romstedt: -rw-r--r--@ 1 Friedrich wheel
>> 8190646 Oct 14 21:33 numpy-2.0.0.dev-py2.5-python.org.dmg
>
> Can I find that somewhere to download and test?
I can upload to my web server friedrichromstedt.org, and I would first
test it on a 10.5 laptop from my colleague. "Double holds better", as
we say in Germany :)
I will also test on 10.6 on my own laptop.
Maybe we should put the test results in a new thread, to not mix them
with the build stuff in this thread here? Pro: No mixing, Con: No
context.
Friedrich
Friedrich Romstedt
binaries? I've heard the names MKL and BLAS, LAPACK, ... all stuff I
don't know really tbh, but do we have to install them?
On the http://projects.scipy.org/numpy/wiki/MakingReleases page,
Cython is mentioned, I've seen no call to cython while compiling, it
that correct? (Cython is installed anyway)
Russell E. Owen
<AANLkTikgOu-ics2cgNnpR...@mail.gmail.com>,
Friedrich Romstedt <friedric...@gmail.com> wrote:
> 2010/10/15 Christopher Barker <Chris....@noaa.gov>:
> > On 10/15/10 10:54 AM, Russell E. Owen wrote:
> >> I have a 10.4 Intel machine I keep around
> >> for the sole purpose of building backward-compatible binary installers
> >> for PIL and matplotlib. If help is still wanted I'm happy to build numpy
> >> as well.
> >
> > I'll let Ralf and Friedrich and Vincent respond, but that sounds like a
> > great option.
>
> Thanks for waiting for us! Here is our status:
>
> Vincent is currently setting up the machine with a crucial
> carbon-copy-clone, reformat, and ccc back. This is because we had
> only around 50 MB left in the end on the partition, and according to
> Vincent enlarging the partition wasn't possible for some reason be
> both don't know. Well, technicalities.
>
> We're having to set up py2.6 & py2.7, what shouldn't be an issue, and
> will have to compile mpl for both of them since the build process
> should not change Python version while its running, i.e., when
> building the py2.6 dmg, the docs will be built using py2.6, and when
> py2.5 dmg, then py2.5, that's what I mean.
>
> Also LaTeX is needed, Russell, as you might know. MacTeX-2010 is
> about 1.6 GB download.
Interesting. I had no idea but I can install that.
> Here come now the interesting facts:
>
> 1) Some tests of numpy failed in 2.0.0.dev. When the machine is
> running again I can send the logs. All some strange-looking typecode
> string tests with dtype('...') iirc.
I have no idea what to make of this.
> 2) I noticed that the paver at some late point tried to switch from
> py2.5 to py2.6, what is rather strange to me. I must have a look
> where precisely the build failed for this reason. py2.6 is the
> DEFAULT_PYTHON_VERSION (iirc) in pavement.py, and changing it to 2.5
> fixed it. Strange.
>
> 3) I found no v1.5.1 tag yet (yesterday). Will the HEAD become 1.5.1?
>
> Here a comparison of our two systems (Russell's and our's):
>
> * We will have 10.4 (?), 10.5, 10.6 available on the same machine
> with vpn access for everyone who wants a cert.
Mine has 10.4 and 10.5 available (via an old two-partition external hard
drive dedicated to building python packages). It has no public access
and I'd prefer to keep it that way. It's also off most of the time and
there are times when I will not have it at all (since I have it on
long-term loan).
> * But we need time to set it up properly. We're unwilling to do
> half-baken things, so I agree that Vincent it installing 10.6 right
> now (I just got the message), but time is rare this weekend.
>
> * So my suggestion would be, Russell, if you could do the build more
> easily then we can, just feel free, I was hoping Vincent and me would
> get the credits though ;-), but first it must succeed on time.
> Vincent, what do you think?
>
> * For future, I would prefer Vincent's machine. We have dyndns and
> the machine can be dedicated for building this stuff. If we get a
> 10.4, we have 10.4 to 10.6 all together on a single place, and we
> could do with it whatever we feel like.
>...
It sounds like you have things under control. I propose to leave it to
you. It sounds like you are doing a great and very thorough job. If you
need a confirming build for some reason I'm happy to do that.
Are you also building matplotlib then? If you are then please install
ActiveState Tcl/Tk so the matplotlib will be compatible with 3rd party
Tcl/Tk (as well as Apple's built-in Tcl/Tk). The best version for
python.org's Python 2.6 is 8.4.19. I'm betting the same is true of the
32-bit Python 2.7. I'm not sure what version of Tcl/Tk the 64-bit
version of Python 2.7 was built against, but that's the one to match.
I'm hoping to build PIL and matplotlib for Python 2.7 in the next month
or so, depending if I can figure out how to do it. (The 32-bit version
should be easy; it's the 64-bit version I'm worried about).
-- Russell
Chris Barker
> One more question: Do we support any external library in the official
> binaries? I've heard the names MKL and BLAS, LAPACK, ... all stuff I
> don't know really tbh, but do we have to install them?
>
be set up to link to that -- It probably still is, so no need for any
external libs.
-Chris
Ralf Gommers
One more question: Do we support any external library in the official
binaries? I've heard the names MKL and BLAS, LAPACK, ... all stuff I
don't know really tbh, but do we have to install them?
On the http://projects.scipy.org/numpy/wiki/MakingReleases page,
Cython is mentioned, I've seen no call to cython while compiling, it
that correct? (Cython is installed anyway)
Cython is not needed. The C sources generated from Cython code are checked into the repo, so you only need Cython if you want to work on actual Cython code. I'll clarify that in the doc.
Ralf
Ralf Gommers
Not too surprising, that's the master branch which is not tested much on py2.5 / OS X.
2) I noticed that the paver at some late point tried to switch from
py2.5 to py2.6, what is rather strange to me. I must have a look
where precisely the build failed for this reason. py2.6 is the
DEFAULT_PYTHON_VERSION (iirc) in pavement.py, and changing it to 2.5
fixed it. Strange.
Did you have the the bootstrap virtualenv set up and active? The following should work from a clean checkout:
paver bootstrap
source bootstrap/bin/activate
python setupsconsegg.py install
paver dmg -p 2.5
If that fails it's a bug. The first build is with python 2.6 if that's your default python, it's needed to make sure the docs are built from the exact same commit as the binary itself.
3) I found no v1.5.1 tag yet (yesterday). Will the HEAD become 1.5.1?
You mean the master branch? HEAD just points at the most recent commit on your currently active branch. In that case no, 1.5.1 will be tagged from the maintenance/1.5.x branch. The tag v1.5.1rc1 or v1.5.1 do not exist yet, tags are only created once the actual release takes place. So tag v1.5.1rc1 should appear tomorrow.
Cheers,
Ralf
Friedrich Romstedt
> On Sat, Oct 16, 2010 at 3:53 AM, Friedrich Romstedt
> <friedric...@gmail.com> wrote:
>> Here come now the interesting facts:
>>
>> 1) Some tests of numpy failed in 2.0.0.dev. When the machine is
>> running again I can send the logs. All some strange-looking typecode
>> string tests with dtype('...') iirc.
>
> Not too surprising, that's the master branch which is not tested much on
> py2.5 / OS X.
Okay.
>> 2) I noticed that the paver at some late point tried to switch from
>> py2.5 to py2.6, what is rather strange to me. I must have a look
>> where precisely the build failed for this reason. py2.6 is the
>> DEFAULT_PYTHON_VERSION (iirc) in pavement.py, and changing it to 2.5
>> fixed it. Strange.
>
> Did you have the the bootstrap virtualenv set up and active? The following
> should work from a clean checkout:
> paver bootstrap
> source bootstrap/bin/activate
> python setupsconsegg.py install
> paver dmg -p 2.5
At that time my system Python was py2.5. What I did is in
http://friedrichromstedt.org/numpy/Releases/SystemSetup/2010.rst, and
the offending call is logged in
http://friedrichromstedt.org/numpy/Releases/Logs/10-10-15/paver-dmg.5.log.
I did not issue the commands you gave before calling $paver dmg.
Also I installed paver using py2.5, and renamed the executable to
paver2.5, to have different pavers for all the Pythons. Is the -p 2.5
switch fully equivalent?
In pavement.py:
@task
@needs("pdf")
@cmdopts([("python-version=", "p", "python version")])
def dmg(options):
...
_build_mpkg(pyver)
def _build_mpkg(pyver):
...
<calls setupegg.py bdist_mpkg>
in setupegg.py:
<calls just setup.py bdist_mpkg>
So I thought it would create it locally, without using the /Framework
installed numpy libs. True?
I would like to avoid $pave nuke, since I want to use the same repo
directory for all Python version builds, and $nuke would remove the
other installers. They shouldn't interfere with each other since the
Python version is encoded in the build directory. The tasks clean and
clean_bootstrap are okay with me.
Seems that the docs show up with 1.4.1rc1 since I omitted the
$setupsconsegg.py install, the autodoc found the previous install from
the numpy-host/, where I didn't use 2.0.0.dev.
Seems that it surpisingly albeit my irgnorance of the instructions in
pavement.py did work quite nicely.
Can you explain in more detail what the virtualenv is for? I guess
it's to not install the numpy binaries built into the system's
directory? Shouldn't there be a $deactivate call before cleaning the
virtualenv? Why cleaning the virtualenv at all? I guess I might need
to clean it for the different Pythons to not interfere, but doesn't
virtualenv do the job too?
> If that fails it's a bug. The first build is with python 2.6 if that's your
> default python, it's needed to make sure the docs are built from the exact
> same commit as the binary itself.
"default Python" = DEFAULT_PYTHON in pavement.py? I guess you mean
the paver Python, the Python used to install Paver. From the log
http://friedrichromstedt.org/numpy/Releases/Logs/10-10-15/paver-dmg.5.log
(the same as above), the docs were built, and there was no 2.6
installed at that time. It tried to call py2.6 *after* the successful
call to pdflatex. I would like to know more about the internals of
the build process, currently it's kind of "gray box" to me.
What is DEFAULT_PYTHON for? It is used in the options() call in
pavement.py, why? Else I see no substantial place of use in
pavement.py.
>> 3) I found no v1.5.1 tag yet (yesterday). Will the HEAD become 1.5.1?
>
> You mean the master branch? HEAD just points at the most recent commit on
> your currently active branch. In that case no, 1.5.1 will be tagged from the
> maintenance/1.5.x branch. The tag v1.5.1rc1 or v1.5.1 do not exist yet, tags
> are only created once the actual release takes place. So tag v1.5.1rc1
> should appear tomorrow.
Okay, I didn't mention master since:
105osxpython:numpy-deployment Friedrich$ git branch
* master
This was like this from the beginning after $git clone
http://github.com/numpy/numpy.git. I thought owner-numpy
project-numpy would be the official numpy repo, am I wrong?
Btw: how do I find out what commit a tag is pointing to, and how do I
display those commit's details (using shell git commands)?
Ralf Gommers
<friedric...@gmail.com> wrote:
> 2010/10/16 Ralf Gommers <ralf.g...@googlemail.com>:
>> On Sat, Oct 16, 2010 at 3:53 AM, Friedrich Romstedt
>> <friedric...@gmail.com> wrote:
>>> 2) I noticed that the paver at some late point tried to switch from
>>> py2.5 to py2.6, what is rather strange to me. I must have a look
>>> where precisely the build failed for this reason. py2.6 is the
>>> DEFAULT_PYTHON_VERSION (iirc) in pavement.py, and changing it to 2.5
>>> fixed it. Strange.
>>
>> Did you have the the bootstrap virtualenv set up and active? The following
>> should work from a clean checkout:
>> paver bootstrap
>> source bootstrap/bin/activate
>> python setupsconsegg.py install
>> paver dmg -p 2.5
>
> At that time my system Python was py2.5. What I did is in
> http://friedrichromstedt.org/numpy/Releases/SystemSetup/2010.rst, and
> the offending call is logged in
> http://friedrichromstedt.org/numpy/Releases/Logs/10-10-15/paver-dmg.5.log.
> I did not issue the commands you gave before calling $paver dmg.
> Also I installed paver using py2.5, and renamed the executable to
> paver2.5, to have different pavers for all the Pythons. Is the -p 2.5
> switch fully equivalent?
2.5 is your default, so that's what was used to build the docs. Then
it tried to build an installer for 2.6, since that's the default given
in pavement.py. Nothing strange so far. The log ends with:
/bin/sh: /Library/Frameworks/Python.framework/Versions/2.6/bin/python:
No such file or directory
So you just didn't have python 2.6 installed, that's all.
The -p 2.5 is not for the paver version, but for the Python version to
compile numpy with.
>
> In pavement.py:
>
> @task
> @needs("pdf")
> @cmdopts([("python-version=", "p", "python version")])
> def dmg(options):
> ...
> _build_mpkg(pyver)
>
> def _build_mpkg(pyver):
> ...
> <calls setupegg.py bdist_mpkg>
>
> in setupegg.py:
> <calls just setup.py bdist_mpkg>
>
> So I thought it would create it locally, without using the /Framework
> installed numpy libs. True?
True. The installed numpy was used only to build the docs. Basically,
during doc build numpy is imported and the docstrings are obtained
through introspection. Doc build finishes, you have a pdf. Then it
goes to part 2: build numpy against the /Framework installed Python.
>
> I would like to avoid $pave nuke, since I want to use the same repo
> directory for all Python version builds, and $nuke would remove the
> other installers. They shouldn't interfere with each other since the
> Python version is encoded in the build directory. The tasks clean and
> clean_bootstrap are okay with me.
It doesn't remove the installers. See the release.sh script in the
repo. It builds all installers at once.
>
> Seems that the docs show up with 1.4.1rc1 since I omitted the
> $setupsconsegg.py install, the autodoc found the previous install from
> the numpy-host/, where I didn't use 2.0.0.dev.
correct
>
> Seems that it surpisingly albeit my irgnorance of the instructions in
> pavement.py did work quite nicely.
>
> Can you explain in more detail what the virtualenv is for? I guess
> it's to not install the numpy binaries built into the system's
> directory?
Yep. Numpy needs to be installed for the doc build to work, as I
explained above. You don't want to do that in global site-packages.
> Shouldn't there be a $deactivate call before cleaning the
> virtualenv? Why cleaning the virtualenv at all? I guess I might need
> to clean it for the different Pythons to not interfere, but doesn't
> virtualenv do the job too?
It's not cleaned. What is cleaned are the build/ and dist/ dirs. The
virtualenv is under bootstrap/, and the installer is under release/.
>
>> If that fails it's a bug. The first build is with python 2.6 if that's your
>> default python, it's needed to make sure the docs are built from the exact
>> same commit as the binary itself.
>
> "default Python" = DEFAULT_PYTHON in pavement.py?
I meant system default. And with first build I meant the one in release.sh.
> I guess you mean
> the paver Python, the Python used to install Paver. From the log
> http://friedrichromstedt.org/numpy/Releases/Logs/10-10-15/paver-dmg.5.log
> (the same as above), the docs were built, and there was no 2.6
> installed at that time. It tried to call py2.6 *after* the successful
> call to pdflatex.
This should be clear now I think
> I would like to know more about the internals of
> the build process, currently it's kind of "gray box" to me.
>
That's how it was for me too half a year ago. Maybe it still is a bit...
> What is DEFAULT_PYTHON for? It is used in the options() call in
> pavement.py, why? Else I see no substantial place of use in
> pavement.py.
It just defines what version you are building a dmg for when doing "$
paver dmg". Using the -p switch overrides that default.
>
>>> 3) I found no v1.5.1 tag yet (yesterday). Will the HEAD become 1.5.1?
>>
>> You mean the master branch? HEAD just points at the most recent commit on
>> your currently active branch. In that case no, 1.5.1 will be tagged from the
>> maintenance/1.5.x branch. The tag v1.5.1rc1 or v1.5.1 do not exist yet, tags
>> are only created once the actual release takes place. So tag v1.5.1rc1
>> should appear tomorrow.
>
> Okay, I didn't mention master since:
>
> 105osxpython:numpy-deployment Friedrich$ git branch
> * master
>
> This was like this from the beginning after $git clone
> http://github.com/numpy/numpy.git. I thought owner-numpy
> project-numpy would be the official numpy repo, am I wrong?
You're not wrong. It's just git's behavior to only clone the master
branch by default. You can obtain other local branches as needed, for
example:
$ git checkout -b 1.5.x origin/maintenance/1.5.x
>
> Btw: how do I find out what commit a tag is pointing to, and how do I
> display those commit's details (using shell git commands)?
>
$ git log v1.5.0
commit c069eee07f2a976b3c2660670dc7bca6438ee94f
<...details...>
Cheers,
Ralf