Psyco testing feed-back wanted
Armin Rigo
I should have mentioned here that the second Psyco release (0.4.1) is out.
Reminder: it's a C extension module that lets you 'mark' the most
performance-sensitive functions of your programs and seriously accelerates
them. It runs on the unmodified Python interpreters (2.1 or 2.2); only for
Pentium CPUs right now. See details at the end of this message.
I'm looking for feed-back. Psyco works reasonably well; I'd even have a
hard time designing code specifically to break it. (There are a few
documented limitations at http://psyco.sourceforge.net/bugs.html.)
Nevertheless it is a complex bunch of low-level C code. I occasionally run
into subtle special-case bugs (which I have always been able to fully
understand and fix up to now). These bugs could just crash the interpreter
or more subtly lead your programs to give wrong results. However, these
problems should always remain located inside of the functions that you
specifically 'marked', and should be easy to discover by testing, given
that Psyco should never behave differently on small or large data inputs.
So it is safe to say that you should definitely try it :-) I would
appreciate any feed-back. Any Python code that causes incorrect behavior
with Psyco would be most welcome. So far, Psyco has been tested with
numerous small utilities and with Python's regression tests; it needs some
larger real-world long-running tests, if only to make sure that its memory
use never grows unboundedly.
Home page: http://psyco.sourceforge.net. New in this version are the
support for almost all the "internal system" functions, e.g.
sys.exc_info(), sys._getframe()... It means that you can use Psyco for
just-in-time specialization and compilation of almost any code.
* acceleration factors are again typically between 2x and 10x or more;
* the memory overhead for compiled code has been considerably reduced;
* also reduced is the 'apparent pause' of programs at start-up due to
compilation.
* don't use 'psyco.jit()'; use 'psyco.bind()' or 'psyco.proxy()'.
* don't overlook the documentation in file 'psyco/__init__.py'.
Armin.
Jimmy Retzlaff
I've seen raves about Psyco, but I hadn't tried it myself until today.
My test case was a function that tests if a point is in a polygon. I
timed several runs with different approaches. All tests are whether 1
point is in a polygon made up of 5,000,000 points. The timing was done
on just the function call itself. Here are some results:
Python (no types declared, floats used) - 14.2 seconds (1x)
Python (no types declared, ints used) - 12.0 seconds (1.2x)
Pyrex (coordinates declared as object, floats used) - 4.8 seconds (3x)
Pyrex (coordinates declared as object, ints used) - 4.7 seconds (3x)
Pyrex (coordinates declared as float) - 2.3 seconds (6x)
Pyrex (coordinates declared as int) - 1.5 seconds (9.5x)
Psyco (no types declared, floats used) - 1.47 (9.5x)
C (coordinates declared as double) - 0.99 seconds (14x)
Psyco (no types declared, ints used) - 0.41 seconds (35x)
C (coordinates declared as unsigned int) - 0.20 seconds (71x)
It's amazing to get this type of code performing this close to C (in
this case VC6 optimized for speed) with no modifications and no type
declarations. For those of you who haven't tried Psycho, here is the
code that had to appear after my function definition:
import psyco
psyco.bind(pointInPolygon)
No other modifications were made to my Python code and Psyco requires no
modification to the Python installation, just drop the Psyco folder into
site-packages. The Python version of the test function is below.
Jimmy
def pointInPolygon((x, y), polygon):
"""Determine if the point (x, y) is in the polygon.
>>> polygon = [(0, 0), (3, 3), (0, 6), (-3, 3)]
>>> pointInPolygon((1, 2), polygon)
1
>>> pointInPolygon((2, 1), polygon)
0
"""
# Algorithm is from http://home.earthlink.net/~bobstein/inpoly
# also appears at http://www.linuxjournal.com/article.php?sid=2029
if len(polygon) < 3:
return False
result = False
xOld, yOld = polygon[-1]
for xNew, yNew in polygon:
if xNew > xOld:
if xOld < x <= xNew and (y-yOld)*(xNew-xOld) <
(yNew-yOld)*(x-xOld):
result = not result
elif xNew < x <= xOld and (y-yNew)*(xOld-xNew) <
(yOld-yNew)*(x-xNew):
result = not result
xOld = xNew
yOld = yNew
return result
RPM1
"Armin Rigo" <ar...@ulb.ac.be> wrote in message
news:Pine.OSF.3.96.1020823...@aster3.ulb.ac.be...
> Hello everybody,
>
> So it is safe to say that you should definitely try it :-) I would
> appreciate any feed-back. Any Python code that causes incorrect behavior
> with Psyco would be most welcome. So far, Psyco has been tested with
> numerous small utilities and with Python's regression tests; it needs some
> larger real-world long-running tests, if only to make sure that its memory
> use never grows unboundedly.
>
Sorry, no bugs yet. :-)
I'm using psyco in a chess program I'm writing. I was going down the
path of extending, but pysco is so much easier. Eventually it should
probably be part of the standard library.
I easily get 4x speedup with psyco in my move generation code.
Thanks for a great product!
Patrick
David Rushby
and psyco doubled the speed of the overall program with no more work
on my part than a couple of bind() calls. The increase in the
performance of the array-handling code itself is more than 2x. If I
had designed the program with Psyco or a manually written C
accelerator in mind (that is, with less frequent transitions between
accelerated and vanilla code), the overall performance increase would
have been even better.
psyco is very impressive, particularly with regard to programmer
convenience. Thanks!
Syver Enstad
> Hello everybody,
>
> I should have mentioned here that the second Psyco release (0.4.1) is
> out.
Very nice Armin. I have checked the Psyco site a couple of times, good
that the project is not dead.
>
> I'm looking for feed-back. Psyco works reasonably well; I'd even have
> a hard time designing code specifically to break it. (There are a few
> documented limitations at http://psyco.sourceforge.net/bugs.html.)
I've tested it on a couple of pygame project's and it seems to work
well. Experienced about 2x the frame rate by using psyco on a program
that has some tight python rendering loops in it.
> * don't use 'psyco.jit()'; use 'psyco.bind()' or 'psyco.proxy()'.
Seconded. psyco.jit() doesn't seem to do much on the code I have
tried, whereas psyco.bind seems to matter a lot.
--
Vennlig hilsen
Syver Enstad
Ian Holmes
> Wow!!!
>
> I've seen raves about Psyco, but I hadn't tried it myself until today.
> My test case was a function that tests if a point is in a polygon. I
> timed several runs with different approaches. All tests are whether 1
> point is in a polygon made up of 5,000,000 points. The timing was done
> on just the function call itself. Here are some results:
>
I had never heard of it before reading the "Python to exceed C in
2030" thread but as soon as it was mentioned I had to test it :)
I saw 3-5x speed increases on a big list / for loop (test data only)
that ran for 8s without psyco.
top product - code used as follows (fairly trivial I admit)
----
import string,time,random,psyco
from strings import * #imports a list of 4 strings
LOOPS = 100000
def method_one(body, x):
for a in range(0, LOOPS):
body.append(strings[x])
x = random.randint(0,3)
return body
print "Starting test one"
body = []
x = 0
START_TIME = time.time()
text = string.join(method_one(body,x))
print "append method (list) =>", time.time()-START_TIME,"s"
strlen = len(text)
string_size = strlen / 1024
print "Size is",string_size,"KB"
print 'starting psyco'
meth1 = psyco.proxy(method_one)
print "Starting test Two"
body = []
x = 0
START_TIME = time.time()
text = string.join(meth1(body,x))
print "append method (list) =>", time.time()-START_TIME,"s"
strlen = len(text)
string_size = strlen / 1024
print "Size is",string_size,"KB"