(one more comment: shedskin -b is not the end of it. shedskin -bw is
often faster (though one has be sure that doesn't change the program).
On Fri, Jan 20, 2012 at 10:43 AM, Mark Dufour <mark.duf...
> ha raphael,
> thanks for testing. it looks like it may be the case something is
> slowing shedskin down on your pc. perhaps the old cpu, an old version
> of gcc or libgc or windows. could you perhaps provide some more
> details about your system..? ;-)
> I ran the matmul and sudoku tests here (linux 3.0, gcc 4.6.1, libgc
> 7.1, shedskin 0.9.1, Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz,
> default FLAGS), and obtained the following results (each time taking
> the best time out of three runs).
> matmul v1:
> cpython 253
> pypy1.7 16.4
> shedskin 2.9
> shedskin -b 1.7
> (so a factor of 5 faster than pypy, compared to the 2 times you see..
> without -b that is ;-) -b is of course very safe since usually you are
> only interested in index-out-of-bounds exceptions during testing.).
> matmul v2
> cpython 232
> shedskin 14.8
> shedskin -b 8.8
> pypy1.7 7.0
> (okay, something seems wrong here, and I'm guessing array element
> access is sub-optimal. I will try to improve this and report back. but
> usually one will use method v1 and not the array module of course.
> this example may only exist to make cpython run faster.)
> matmul v3
> cpython 296
> shedskin 22
> shedskin -b 12.4
> pypy1.7 4.6
> (fastest pypy time here, still about 3 times slower than shedskin -b.
> will investigate why shedskin takes 22 seconds here, probably
> something silly.)
> so I would say we can conclude that given the task of multiplying a
> 1000 by 1000 matrix, based on these tests, shedskin can do it 3 times
> faster than pypy1.7 on a modern system.
> as for the sudoku test:
> cpython 115
> pypy1.7 14.8
> shedskin 5.4
> shedskin -b 3.5
> it's great pypy is improving, but (again) I don't think it will be
> easy to surpass a static compiler on a restricted subset of python.
> jit compilers aren't magic. compiling away such things as arbitrary
> size arithmetic, bounds- and wrap-around checking sound nice, but in
> practice often end up fairly limited in their scope.
> the dict benchmark mostly tests memory allocation within the dict
> implementation, so there won't be much difference there. cpython is
> obviously highly optimized here. the pattern matching benchmark mostly
> tests the used re engine, libpcre in case of shedskin. not much I can
> do about that.
> thanks again!
> On Wed, Jan 18, 2012 at 9:56 PM, Raphael Clifford <drr...@gmail.com> wrote:
>>> On Mon, Jan 16, 2012 at 5:20 PM, Raphael Clifford <drr...@gmail.com> wrote:
>>>> Well.. that is a very old version of pypy from November 2010 :) It
>>>> would be fair to look at pypy 1.7 at least.
>>> well, it should only take a few minutes do re-run these tests with
>>> more recent versions of pypy (and shedskin).. please share the results
>>> if you decide to do so.. :-)
>> It's not quite as simple as it sounds to run the tests as they don't
>> provide exactly the code they ran it seems. Also, some of the code
>> comes in different versions for reasons I am not clear on. Here are
>> some example timings on my rather slow amd computer.
>> For pypy I used a recent nightly build. For shedskin I used 0.91. The
>> timings are rounded to the nearest half second to eliminate random
>> noise. Timings and memory were given by /usr/bin/time
>> For the matrix multiplication tests you have to set a size and I chose n = 800
>> pypy : timing 6s , memory 188688maxresident
>> shedskin : timing 3s , memory 162640maxresident
>> pypy: timing 4s, memory 154208maxresident
>> shedskin : 26s, memory 177328maxresident
>> pypy : timing 5s, memory 154368maxresident
>> shedskin: timing 46s, memory 111776maxresident
>> The dictionary test seems very bad for both shedskin and pypy.
>> cat test.dict |/usr/bin/time ./dict_v1
>> pypy : 9s, memory 404592maxresident
>> shedskin : 10s, memory 787344maxresident
>> shedskin also outputs warning from the garbage collector GC.
>> cpython 2.6 : 6s, memory 308560maxresident
>> The Sudoku test doesn't come with the code they actually used that I
>> could see (it says the test is run 50 times). So instead I used the
>> sudoku17 set from http://people.csse.uwa.edu.au/gordon/sudoku17 and
>> piped the output to /dev/null as it's rather large.
>> pypy : 48s, 190384maxresident
>> shedskin : 31s, 8400maxresident
>> and just for fun
>> C : 7s, 1664maxresident
>> The pattern matching test seems to finish almost instantly with no
>> output so I omitted it. If anyone knows it should be run I would be
>> happy to run it.
>> Hope this helps.
>> You received this message because you are subscribed to the Google Groups "shedskin-discuss" group.
>> To post to this group, send email to email@example.com.
>> To unsubscribe from this group, send email to firstname.lastname@example.org.
>> For more options, visit this group at http://groups.google.com/group/shedskin-discuss?hl=en.