What are the best results on micro-benchmarks "slices"?

[Valery]

Hi

I've got a sintetic test where Unladen-Swallow Q3 is 2 times faster
than a plain Python 2.6.4 (r264:75706, Nov 2 2009, 14:44:17) under
ia64, Ubuntu:

from random import randrange
from itertools import groupby

#let's create a list of tuples of string pairs
lst = map(lambda x: (str(randrange(x)),'bla-bla-bla'), xrange
(1,2000000))

# now let's count number of appearances for each pair:
r = [ (x,len(list(y))) for x,y in groupby(sorted(lst))]

the last operation takes:

21s for Unladen-Swallow Q3 and
45s for a plain Python2.6

OK, one have to check probably against 2.5 and ensure in particular
that "groupby" and "sorted" have the same implementations...

well, anyway, just wanted to share this and ask others for similar
promissing micro-bmarks slices.

BTW, the great results in the real tests is rather a consequence of
many great results in micro-bmarks, not vice-versa. The real tests are
just the "driving criteria" :)

regards,
Valery

[Collin Winter]

Hi Valery,

On Wed, Dec 2, 2009 at 8:46 AM, Valery <kham...@gmail.com> wrote:
> Hi
>
> I've got a sintetic test where Unladen-Swallow Q3 is 2 times faster
> than a plain Python 2.6.4 (r264:75706, Nov  2 2009, 14:44:17) under
> ia64, Ubuntu:
>
> from random import randrange
> from itertools import groupby
>
> #let's create a list of tuples of string pairs
> lst = map(lambda x: (str(randrange(x)),'bla-bla-bla'), xrange
> (1,2000000))
>
> # now let's count number of appearances for each pair:
> r = [ (x,len(list(y))) for x,y in groupby(sorted(lst))]
>
> the last operation takes:
>
> 21s for Unladen-Swallow Q3 and
> 45s for a plain Python2.6
>
> OK, one have to check probably against 2.5 and ensure in particular
> that "groupby" and "sorted" have the same implementations...

If you're comparing 2.6 vs US Q3, why would you check Python 2.5? Our
baseline is 2.6.1.

> well, anyway, just wanted to share this and ask others for similar
> promissing micro-bmarks slices.

Was this operation a hotspot in one of your applications? What led you
to benchmark this particular snippet of code?

Thanks,
Collin Winter

[Valery Khamenya]

If you're comparing 2.6 vs US Q3, why would you check Python 2.5? Our
baseline is 2.6.1.

oh, right.

 

> well, anyway, just wanted to share this and ask others for similar
> promissing micro-bmarks slices.

Was this operation a hotspot in one of your applications?

yes.

 

What led you to benchmark this particular snippet of code?

it is a bottleneck in some text-analysis tasks. In DNA analysis it is not really a bottleneck but just because the real-time queries there is not yet the case :)

Of course, I never use this 1-line snippet in real apps, albeit it is quite normal for functional languages and is very Pythonish too. It is 10+ times more slow than, say:

def inc(d,e):

    d[e] = d.get(e,0)+1

    return d

reduce(inc, [1,2,1,3,2,2], {})

And, as always, the straight-forward loop is the fastest one:

def count_iteratively(lst):

  d = {}

  for e in lst:

     d[e] = d.get(e,0)+1

return d

count_iteratively([1,2,1,3,2,2])

Clearly, I'd be more happy to use short Python statements and leave more optimizations to the language implementation. And I believe LLVM here is a right and already a quite mature choice.

regards

Valery

[Collin Winter]

Hey Valery,

On Wed, Dec 2, 2009 at 4:07 PM, Valery Khamenya <kham...@gmail.com> wrote:
>> If you're comparing 2.6 vs US Q3, why would you check Python 2.5? Our
>> baseline is 2.6.1.
>
> oh, right.
>
>>
>> > well, anyway, just wanted to share this and ask others for similar
>> > promissing micro-bmarks slices.
>>
>> Was this operation a hotspot in one of your applications?
>
> yes.

Is this code open source or open source-able? I'd like to include
something like this in our benchmark suite.

Thanks,
Collin Winter

[Valery Khamenya]

Hi Collin, 

 

Is this code open source or open source-able? I'd like to include
something like this in our benchmark suite.

my app is not open-sourced, sorry. 

However... 

The life on earth is estimated approx as 4.5 billion years old. One of our oldest grannies are perhaps not the apes, but prokaryotes. Let's take a Exiguobacterium arabatum pEspA plasmid (http://www.ncbi.nlm.nih.gov/nuccore/AM980831)

This tiny has a 4K-chars long DNA only. Not the shortest, not the oldest, but sort of. Think of it as a snapshot of 1+ billion years ago :)

Now let's make a typical student homework and count frequence of appearances of DNA residues.

The Python code is attached and it is, of course, shorter than the DNA :)

Is it OK as for mini-benchmark test?

if DNA is too short and 20ms is too fast, let me know. We could take more modern species ;)

Kind regards,

Valery