Report: Benchmark a new memory allocation scheme, TCMalloc

[Pavel Vinogradov]

  Hello.

  This is my fist version of memory allocation scheme benchmarking ( http://code.google.com/p/google-highly-open-participation-psf/issues/detail?id=105&colspec=ID%20Status%20Summary ).
 
  In my work i compared python 2.6, python 3.0 with malloc, pymalloc and tcmalloc scheme. I was use my Laptop with Intel Celeron M 1.7 Ghz, 512 Mb RAM to run benchmark and try to minimize load from other programs.

  Detailed results of benchmark attached to mail. There i write some summary and comments.

Python 2.6

  python2.6 --configure
Totals:                           9055ms   9232ms

  python2.6 ./configure --without-pymalloc
Totals:                           9261ms   9433ms

  python2.6 ./configure --with-libs="-ltcmalloc"
Totals:                           9071ms   9206ms

  python2.6 ./configure --without-pymalloc --with-libs="-ltcmalloc"
Totals:                           9240ms   9350ms

  Python 3.0

  python3.0 ./configure
Totals:                           9673ms   9828ms

  python3.0 ./configure --without-pymalloc
Totals:                           9853ms   9974ms

  python3.0 ./configure --with-libs="-ltcmalloc"
Totals:                           9614ms   9726ms

  python3.0 ./configure --without-pymalloc --with-libs="-ltcmalloc"
Totals:                           9907ms   9981ms

  My comments:

Memory allocation scheme:

  malloc     - standart libc memory allocation scheme. Show slowest results in all test.
  pymalloc - python optimized memory allocation scheme. Use by default in python.
  tcmalloc  - google optimized memory allocation scheme. Show some speed improvements in compare to malloc and pymalloc, but this improvement in fault range. But on project page declared up to 6 times.

Python version:

  Python 3.0 show some lower result than 2.6. I think that reason of this issue is current alpha status of py3k. And this will be improved during py3k development cycle.

  Tcmalloc:

  I was confused with result of this library. I recheck result twice with linking and LD_PRELOAD method. But result not changes. Maybe i make mistake there, but issue have very limited information about tcmalloc integration.

[Pavel Vinogradov]

  I'm continue my work and now make new test. I write simple walker program - this script travel from given directory and read each file into memory line be line. After each file i call gc.collect. This script attached.
  I run it on my ~/tmp dir which contain 835Mb of various files. I receive very interest results:
  python 2.6 compiled with tcmalloc - failed this test. On files more then 30 Mb - script use more than 90% CPU and process this files more then 10 minutes (and not finished it).
 
  More detailed results:
python2.6:
--default
    full (800Mb dir):
        02:97
    (260Mb dir)
        0:08
--without-pymalloc
    full:
        2.27
    (260Mb dir)
        0.48
--with-tcmalloc
    full:
          failed, 99% on files more that 45Mb
    (260Mb dir)
          05.42
--without-pymalloc --with-tcmalloc
    full:
           failed, 99% on files more that 45Mb
    (260Mb dir)
           05.83

  I can't test python 3 because my script not run on it. Any comments about tcmalloc?

2007/11/28, Pavel Vinogradov < fas...@gmail.com>:

[Titus Brown]

On Sat, Dec 01, 2007 at 11:01:55PM +0400, Pavel Vinogradov wrote:
-> I'm continue my work and now make new test. I write simple walker program
-> - this script travel from given directory and read each file into memory
-> line be line. After each file i call gc.collect. This script attached.
-> I run it on my ~/tmp dir which contain 835Mb of various files. I receive
-> very interest results:
-> python 2.6 compiled with tcmalloc - failed this test. On files more then
-> 30 Mb - script use more than 90% CPU and process this files more then 10
-> minutes (and not finished it).
->
-> More detailed results:
-> python2.6:
-> --default
-> full (800Mb dir):
-> 02:97
-> (260Mb dir)
-> 0:08
-> --without-pymalloc
-> full:
-> 2.27
-> (260Mb dir)
-> 0.48
-> --with-tcmalloc
-> full:
-> failed, 99% on files more that 45Mb
-> (260Mb dir)
-> 05.42
-> --without-pymalloc --with-tcmalloc
-> full:
-> failed, 99% on files more that 45Mb
-> (260Mb dir)
-> 05.83
->
-> I can't test python 3 because my script not run on it. Any comments about
-> tcmalloc?

Hi, Pavel, I'm shocked and amused by this result! Have you contacted
the tcmalloc author directly about this? I can forward your e-mail on
to them. It might be good to provide the Python script, too.

It seems like pymalloc causes a bit of a slowdown, too. I don't know
enough about memory allocation to comment intelligently, so I won't.

--titus

[Pavel Vinogradov]

On Dec 3, 2007 3:39 PM, Titus Brown <ti...@caltech.edu> wrote:

Hi, Pavel, I'm shocked and amused by this result!  Have you contacted
the tcmalloc author directly about this?  I can forward your e-mail on
to them.  It might be good to provide the Python script, too.

It seems like pymalloc causes a bit of a slowdown, too.  I don't know
enough about memory allocation to comment intelligently, so I won't.

  I don't contacted tcmalloc author directly because rely on comments from anybody on the list about this issue. I'm write letter now and provide link on this thread, maybe he can give some comments about this.

[Pavel Vinogradov]

  I'm stated thread about this issue on google-perftools googlegroup.

http://groups.google.com/group/google-perftools/browse_thread/thread/4cc0f545c25caecc

[Titus Brown]

On Mon, Dec 03, 2007 at 04:15:27PM +0400, Pavel Vinogradov wrote:
-> I'm stated thread about this issue on google-perftools googlegroup.
->
-> http://groups.google.com/group/google-perftools/browse_thread/thread/4cc0f545c25caecc
->

the cowards! they have not yet responded to you!

--titus

[Pavel Vinogradov]

  Yes, i don't know what i can do for it. But i plan to rewrite my walker example in plain C and try to reproduce this behavior.
But in any case, case anybody confirm python+tcmalloc problem on large files?

[Titus Brown]

On Thu, Dec 06, 2007 at 12:12:56PM +0400, Pavel Vinogradov wrote:
-> On Dec 5, 2007 1:49 PM, Titus Brown <ti...@caltech.edu> wrote:
->
-> >
-> > On Mon, Dec 03, 2007 at 04:15:27PM +0400, Pavel Vinogradov wrote:
-> > -> I'm stated thread about this issue on google-perftools googlegroup.
-> > ->
-> > ->
-> > http://groups.google.com/group/google-perftools/browse_thread/thread/4cc0f545c25caecc
-> > ->

-> > the cowards! they have not yet responded to you!

->
-> Yes, i don't know what i can do for it. But i plan to rewrite my walker
-> example in plain C and try to reproduce this behavior.
-> But in any case, case anybody confirm python+tcmalloc problem on large
-> files?

I'll look at it tomorrow; sorry for the delay! Could you send me the
walker script? (I know, it may be simple, but anything I don't have tow
rite... ;)

[Pavel Vinogradov]

On Dec 6, 2007 1:08 PM, Titus Brown <ti...@caltech.edu> wrote:

I'll look at it tomorrow; sorry for the delay!  Could you send me the
walker script?  (I know, it may be simple, but anything I don't have tow
rite... ;)

    My script attached to second letter in this thread. But i'm but it there again.

[Pavel Vinogradov]

Hi Titus, any progress there?
I have some response from tmalloc maillist and want know that this
behavior is real bug or undocumented feature:)