pypy! was: Adventures in optimising the CCC code.
Tom Roche
David Jones 11:47 am 13 Jul 2010
> take home message[:] "PyPy makes it 5 times faster"
Or a bit more: on my dual-core laptop
me@it:~$ lsb_release -ds
Ubuntu 10.04.3 LTS
me@it:~$ uname -rv
2.6.32-33-generic #70-Ubuntu SMP Thu Jul 7 21:13:52 UTC 2011
me@it:~$ python --version
Python 2.6.5
me@it:~$ pypy --version
Python 2.7.1 (b590cf6de419, Apr 30 2011, 02:00:34)
[PyPy 1.5.0-alpha0 with GCC 4.4.3]
a full run of ccc-gistemp-read-only (including preflight) requires
- 130 min using python
+ 21 min using pypy
Setup of pypy took more fiddling than I expected, due to
- absence of ubuntu or debian packages
- my ubuntu being too downlevel to `alien` the Fedora RPMs (gotta upgrade!)
- (IMHO) lack of clarity in the pypy.org docs
but I was able to install the latest tarball using the following script:
URI="https://bitbucket.org/pypy/pypy/downloads/pypy-1.5-linux64.tar.bz2"
TMP_DIR_ROOT="/tmp/pypy/pypy-c-jit-43780-b590cf6de419-linux64"
TARGET_DIR_ROOT="/opt/pypy-c-jit-1.5.0-alpha0"
TARGET_PYPY_EXEC="${TARGET_DIR_ROOT}/bin/pypy"
USR_PYPY_EXEC="/usr/local/bin/pypy"
for CMD in \
"rm -fr $(dirname ${TMP_DIR_ROOT})" \
"mkdir -p ${TMP_DIR_ROOT}" \
"pushd $(dirname ${TMP_DIR_ROOT})" \
"wget -O - ${URI} | tar xvjf -" \
"ls -alh" \
"popd" \
"sudo rm -fr ${TARGET_DIR_ROOT}" \
"sudo mkdir -p ${TARGET_DIR_ROOT}" \
"sudo cp -r ${TMP_DIR_ROOT}/* ${TARGET_DIR_ROOT}/" \
"sudo chmod -R go+rX ${TARGET_DIR_ROOT}/" \
"sudo ln -s ${TARGET_PYPY_EXEC} ${USR_PYPY_EXEC}" \
"which pypy" \
"sudo ls -al ${USR_PYPY_EXEC}" \
"${USR_PYPY_EXEC} --version" \
"sudo ls -al ${TARGET_DIR_ROOT}" \
"pypy --version" \
"pypy" \
"popd" \
; do
echo -e "$ ${CMD}"
eval "${CMD}"
done
HTH, Tom Roche <Tom_...@pobox.com>
Filipe Pires Alvarenga Fernandes
====> Timing Summary ====
Run took 367.2 seconds
Wonderful result, this is the first time I run pypy it is really impressive.
Just a caveat for others that might try your installation script.
1) Be aware that the script is downloading the 64 bit pypy, if you are
on a 32 bit machine change:
URI="https://bitbucket.org/pypy/pypy/downloads/pypy-1.5-linux.tar.bz2"
and
TMP_DIR_ROOT="/tmp/pypy/pypy-c-jit-43780-b590cf6de419-linux/"
2) Some libraries might not be present or present in a different
version. I had to symlink libbz2:
sudo ln -s /lib/libbz2.so.1 /lib/libbz2.so.1.0
and install another version of libssl (libopenssl0_9_8)
-Filipe
> --
> You received this message because you are subscribed to the Google Groups "CCC GISTEMP discussion" group.
> To post to this group, send email to ccc-giste...@googlegroups.com.
> To unsubscribe from this group, send email to ccc-gistemp-dis...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/ccc-gistemp-discuss?hl=en.
>
>
Tom Roche
A few more notes on pypy install: first its benefits, then its quirks (in the hope that the latter will find its way into the pypy docs, hence my CC: to Maciej Fijalkowski of pypy):
David Jones 11:47 am 13 Jul 2010
>>> take home message[:] "PyPy makes it 5 times faster"
Tom Roche Sun, 31 Jul 2011 03:12:33 -0400
>> Or a bit more: on my dual-core laptop
>> me@it:~$ lsb_release -ds
>> Ubuntu 10.04.3 LTS
>> me@it:~$ uname -rv
>> 2.6.32-33-generic #70-Ubuntu SMP Thu Jul 7 21:13:52 UTC 2011
>> me@it:~$ python --version
>> Python 2.6.5
>> me@it:~$ pypy --version
>> Python 2.7.1 (b590cf6de419, Apr 30 2011, 02:00:34)
>> [PyPy 1.5.0-alpha0 with GCC 4.4.3]
>> a full run of ccc-gistemp-read-only (including preflight) requires
>> - 130 min using python
>> + 21 min using pypy
Filipe Pires Alvarenga Fernandes Sun, 31 Jul 2011 08:35:18 -0400
> I could not resist and just tried it on my machine
> ====> Timing Summary ====
> Run took 367.2 seconds
Sweet! On what sort of box (e.g., what hardware, OS) did that run?
>> Setup of pypy took more fiddling than I expected, due to
>> - absence of ubuntu or debian packages
>> - my ubuntu being too downlevel to `alien` the Fedora RPMs (gotta upgrade!)
(more on that below)
>> - (IMHO) lack of clarity in the pypy.org docs
(which I'm hoping the pypy folks can fix, since there doesn't seem to be a wiki or other means for users to contribute documentation)
>> but I was able to install the latest tarball using the following script:
>> URI="https://bitbucket.org/pypy/pypy/downloads/pypy-1.5-linux64.tar.bz2"
>> TMP_DIR_ROOT="/tmp/pypy/pypy-c-jit-43780-b590cf6de419-linux64"
>> TARGET_DIR_ROOT="/opt/pypy-c-jit-1.5.0-alpha0"
(note on the above lines, below)
> Just a caveat for others that might try your installation script.
> 1) Be aware that the script is downloading the 64 bit pypy,
Correct, 64-bit linux to be precise. For more options, ...
> if you are on a 32 bit machine change:
> URI="https://bitbucket.org/pypy/pypy/downloads/pypy-1.5-linux.tar.bz2"
> and
> TMP_DIR_ROOT="/tmp/pypy/pypy-c-jit-43780-b590cf6de419-linux/"
... see the pypy download page
> 2) Some libraries might not be present or present in a different
> version. I had to symlink libbz2:
> sudo ln -s /lib/libbz2.so.1 /lib/libbz2.so.1.0
> and install another version of libssl (libopenssl0_9_8)
If you, like Filipe, run a more uplevel linux than I am currently (e.g., if your linux has libssl.so.10), try an RPM from
http://rpm.pbone.net/index.php3?stat=3&search=pypy&srodzaj=3
esp if you run an RHL-derived distro. If you run a debian-derived distro, you can probably still consume the .rpm with `alien`: unfortunately there is not currently either a .deb or a PPA for pypy. (Not since hardy, anyway, and that's *way* downlevel.)
HTH, Tom Roche <Tom_...@pobox.com>
Filipe Pires Alvarenga Fernandes
$cat /proc/cpuinfo | grep "model name"
model name : Intel(R) Core(TM)2 Duo CPU T6400 @ 2.00GHz
$ lsb_release -ds
"openSUSE 11.4 (i586)
$ uname -rv
3.0.0-38-desktop #1 SMP PREEMPT Sun Jul 24 06:15:52 UTC 2011 (5a1af7d)
$ python --version
Python 2.7
$ pypy --version
Python 2.7.1 (b590cf6de419, Apr 30 2011, 02:00:38)
[PyPy 1.5.0-alpha0 with GCC 4.4.3]
I forgot to mention that "367.2 seconds" was without the download.
Also, the input data I used was:
http://ccc-gistemp.googlecode.com/files/ccc-gistemp-input-20101130.zip
-Filipe