Mem usage in pprof do not match with top

2,546 views
Skip to first unread message

Davies Liu

unread,
Sep 26, 2010, 10:43:55 AM9/26/10
to golang-nuts
Hi, all:

I have a proxy application wroted in go, and  try to use pprof to find 
out memory usage in Go, but it seems strange that memory usage 
showed in top, pprof tools and http://localhost:6060/debug/pprof/heap
do not match each other. 

Top:  
     RES: 266M
     20M
     85M

What does it mean ?

-- 
 - Davies


heap profile: 159: 20388648 [152494: 28693394696] @ heap/1048576
1: 32 [1: 32] @ 0x40544b 0x405a67 0x410121 0x405ed2 0x4381c6 0x466985 0x45fe98 0x45b3fa 0x4613f4 0x45460b 0x41bbc0 0x402eae
1: 14080 [1: 14080] @ 0x40544b 0x405a67 0x4074a7 0x407de7 0x40845f 0x40878d 0x40936e 0x409528 0x465c9c 0x4662ab 0x402eae
1: 159744 [8: 532480] @ 0x40544b 0x4102fd 0x4b6403 0x4b6596 0x4669ac 0x467a76 0x4617c1 0x4618ac 0x416ee9 0x401122 0x409df7 0x402eae
1: 112 [97: 10864] @ 0x40544b 0x405a67 0x4074a7 0x40768a 0x408d81 0x408fd3 0x413c2f 0x417c37 0x4185cf 0x4186a4 0x41a8db 0x400971 0x415232 0x41666d 0x41b9e9 0x402eae
1: 160 [1: 160] @ 0x40544b 0x4055e6 0x403eec 0x4040b2 0x403f82 0x41721b 0x401122 0x409df7 0x402eae
1: 32 [181: 5792] @ 0x40544b 0x405a67 0x4015d1 0x40182e 0x4018f0 0x43b27b 0x43b724 0x43b94a 0x416959 0x41b9e9 0x402eae
1: 16 [92: 1472] @ 0x40544b 0x405a67 0x406314 0x4064ac 0x416887 0x41b9e9 0x402eae
1: 32 [174: 5568] @ 0x40544b 0x405a67 0x405a90 0x43ae42 0x43b1ce 0x43b724 0x43b94a 0x416959 0x41b9e9 0x402eae
1: 32 [221: 7048] @ 0x40544b 0x40c83a 0x448936 0x448def 0x412889 0x412a10 0x4165e5 0x41b9e9 0x402eae
1: 8 [22: 176] @ 0x40544b 0x405a67 0x405a90 0x460a54 0x461811 0x4618ac 0x416ee9 0x401122 0x409df7 0x402eae
2: 224 [617: 69104] @ 0x40544b 0x405a67 0x408cca 0x408fd3 0x4154aa 0x41666d 0x41b9e9 0x402eae
1: 32 [186: 5952] @ 0x40544b 0x405a67 0x405a90 0x43ae42 0x43b194 0x43b724 0x43b94a 0x416959 0x41b9e9 0x402eae
1: 64 [352: 22544] @ 0x40544b 0x405a67 0x4015d1 0x40182e 0x4018f0 0x43b772 0x43b94a 0x416959 0x41b9e9 0x402eae
3: 12288 [3460: 14172160] @ 0x40544b 0x40c83a 0x4477da 0x447866 0x417c17 0x4185cf 0x4186a4 0x41a8db 0x400971 0x415232 0x41666d 0x41b9e9 0x402eae
3: 280832 [6239: 619910400] @ 0x40544b 0x40c83a 0x413014 0x4165e5 0x41b9e9 0x402eae
5: 20480 [9205: 37703680] @ 0x40544b 0x40c83a 0x448f67 0x448fe9 0x4165b2 0x41b9e9 0x402eae
3: 12288 [9269: 37965824] @ 0x40544b 0x40c83a 0x4477da 0x447866 0x416574 0x41b9e9 0x402eae
75: 19859840 [101278: 27896783744] @ 0x40544b 0x40c83a 0x414528 0x417c37 0x418157 0x41a555 0x400876 0x4155cc 0x41666d 0x41b9e9 0x402eae
5: 20480 [21039: 86175744] @ 0x40544b 0x40c83a 0x4477da 0x447866 0x417c17 0x418157 0x41a555 0x400876 0x4155cc 0x41666d 0x41b9e9 0x402eae
1: 16 [1: 16] @ 0x40544b 0x405a67 0x4b6471 0x4b6596 0x41987f 0x402eae
1: 16 [1: 16] @ 0x40544b 0x405a67 0x4b6471 0x4b6596 0x4175c6 0x41927b 0x400f1d 0x409df7 0x402eae
49: 7840 [49: 7840] @ 0x40544b 0x4055e6 0x403eec 0x40b883 0x4035e6 0x40bc1c

# runtime.MemStats
# Alloc = 85718360
# TotalAlloc = 114860211888
# Sys = 1410111736
# Lookups = 1171539771
# Mallocs = 425243698
# HeapAlloc = 85545416
# HeapSys = 275906560
# HeapIdle = 0
# HeapInuse = 92737536
# Stack = 0 / 131072
# MSpan = 295632 / 524288
# MCache = 55200 / 131072
# MHeapMapSys = 3670016
# BuckHashSys = 1439992
# NextGC = 87729760
# PauseNs = 249778014000
# NumGC = 6652
# EnableGC = true
# DebugGC = false
# BySize = Size * (Active = Mallocs - Frees)
# (Excluding large blocks.)
#   8 * (16439 = 54612820 - 54596381)
#   16 * (83345 = 138386226 - 138302881)
#   32 * (29454 = 71094527 - 71065073)
#   48 * (14199 = 103507011 - 103492812)
#   64 * (9126 = 21900003 - 21890877)
#   80 * (2958 = 8619926 - 8616968)
#   96 * (2187 = 2423045 - 2420858)
#   112 * (7531 = 14653746 - 14646215)
#   128 * (38 = 1344 - 1306)
#   144 * (36 = 4674 - 4638)
#   160 * (1641 = 1212673 - 1211032)
#   176 * (57 = 771 - 714)
#   192 * (50 = 372 - 322)
#   208 * (23 = 59 - 36)
#   224 * (19 = 161 - 142)
#   240 * (7 = 83 - 76)
#   256 * (10 = 26 - 16)
#   288 * (3 = 4637 - 4634)
#   320 * (49 = 126 - 77)
#   352 * (0 = 1 - 1)
#   384 * (1 = 2 - 1)
#   448 * (1 = 9 - 8)
#   576 * (15 = 22 - 7)
#   640 * (3 = 157 - 154)
#   704 * (1 = 1043 - 1042)
#   768 * (1 = 321 - 320)
#   832 * (1 = 664 - 663)
#   960 * (0 = 1486 - 1486)
#   1152 * (14 = 25655 - 25641)
#   1280 * (6 = 9606 - 9600)
#   1408 * (12 = 19428 - 19416)
#   1536 * (33 = 33248 - 33215)
#   1664 * (21 = 49749 - 49728)
#   1920 * (28 = 89940 - 89912)
#   2048 * (29 = 135263 - 135234)
#   2304 * (65 = 139047 - 138982)
#   2560 * (62 = 133293 - 133231)
#   2816 * (53 = 75890 - 75837)
#   3072 * (44 = 83360 - 83316)
#   3328 * (31 = 66771 - 66740)
#   3840 * (93 = 210544 - 210451)
#   4096 * (2514 = 5902062 - 5899548)
#   4352 * (478 = 82532 - 82054)
#   4608 * (30 = 55198 - 55168)
#   5120 * (49 = 109384 - 109335)
#   5888 * (92 = 143357 - 143265)
#   6656 * (105 = 151442 - 151337)
#   6912 * (34 = 51493 - 51459)
#   7936 * (134 = 194000 - 193866)
#   8192 * (25 = 39699 - 39674)
#   8704 * (86 = 71616 - 71530)
#   9984 * (93 = 133209 - 133116)
#   10496 * (16 = 36471 - 36455)
#   12032 * (49 = 72622 - 72573)
#   12288 * (2 = 8255 - 8253)
#   14080 * (30 = 45722 - 45692)
#   16128 * (27 = 37533 - 37506)
#   17664 * (32 = 22087 - 22055)
#   20224 * (22 = 32753 - 32731)
#   21248 * (9 = 8550 - 8541)
#   24320 * (22 = 52462 - 52440)
#   24832 * (1 = 2151 - 2150)
#   28416 * (13 = 19811 - 19798)
#   32512 * (10 = 20408 - 20398)
#   32768 * (0 = 866 - 866)

Russ Cox

unread,
Sep 27, 2010, 10:56:44 AM9/27/10
to Davies Liu, golang-nuts
> Top:
>      RES: 266M

This is the amount of physical RAM dedicated to your
program right now. It includes parts of the program text
and the allocated heap, but it may not include all of it,
if some of the text has not yet been paged in from disk
or if some of the heap has been paged out to disk.

> http://localhost:6070/debug/pprof/heap:
>      20M


> heap profile: 159: 20388648 [152494: 28693394696] @ heap/1048576

This means that the heap profiler is tracking
159 currently allocated objects taking up 20 MB;
overall it has tracked 152,494 objects taking up 28.7 GB.
The @heap/1048576 means that the heap profiler
is trying to sample about one byte per half megabyte
(there's an implicit divide by two due to a mistake
made long before Go was an idea).

If you've got 159 objects totaling 20 MB, that's
125 kB each, so sampling one byte per half megabyte
means you're probably hitting only 25% of the allocations,
so an estimate of the total amount in use would be
20 MB / 0.25 = 80 MB.

> pprof --text http://boromir:6070/pprof/heap
>      85M

I was close. In fact the calculation I just did is done
for each of the lines following the heap profile line,
so it ends up a bit more accurate. But it's still an
estimate, and only an estimate of what's in use right
now. The process might have allocated more
memory and freed it already, but that memory is
sitting in free lists, not given back to the operating
system.

> # runtime.MemStats
> # Alloc = 85718360
> # TotalAlloc = 114860211888

Over the lifetime of the program the memory allocator
has served requests for 114 GB.

> # Sys = 1410111736

It asked the operating system for 1.4 GB.
This number is buggy: the garbage collector
recently started allocating and returning
temporary memory during garbage collections,
and it doesn't adjust this properly when it returns
the memory. Normally this number should be
the sum of the XxxSys numbers below.

> # HeapAlloc = 85545416

This is the exact number of bytes allocated right now.
pprof was pretty accurate.

> # HeapSys = 275906560

275 MB is being used by the heap.
That matches what top said, so that's good.
This includes memory reclaimed by the
garbage collector and not yet given out again.
It's likely that even though 85 MB is in use right now,
earlier in the program there was more memory in use,
probably around 140 MB, which is why the allocator
let the heap grow to 275 MB.

Russ

Davies Liu

unread,
Sep 28, 2010, 1:18:57 PM9/28/10
to r...@golang.org, golang-nuts


On Wed, Sep 29, 2010 at 1:13 AM, Davies Liu <davie...@gmail.com> wrote:
Hi, Russ:

Thank you very much, for the details descriped above.

According to the report of pprof, there was some memory leak in my database
application, called beasdb,  something like memcached with a log-structure store in disk. 

After running almost one day, the memory used by  beansdb increased quickly,
about 100M per hour, and it never stop.

In the heap profile (was attached below), there are many objects not freed by GC:

1161: 348853248 [60311: 17089668736] @ 0x407407 0x403430 0x445d11 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1

They were buffers used to store record data, should been freed after sending to client.

The call stacks are followed (was cutted):

0x43e9a8 memcache.*ServerConn·Serve
--------------------------------------------------------------------------------------------------
func (c *ServerConn) Serve(store Storage, stats *Stats) (e os.Error) {
   for {
       req := new(Request)
       e = req.Read(rbuf)
       resp := req.Process(store, stats)
       if !resp.noreply {
           resp.Write(wbuf)
       }
   }
}
--------------------------------------------------------------------------------------------------
0x43d907 memcache.*Request·Process
--------------------------------------------------------------------------------------------------
func (req *Request) Process(store Storage, stat *Stats) (resp *Response) {
   resp = new(Response)
   switch req.Cmd {
   case "get", "gets":
       resp.items = make(map[string]*Item, len(req.Keys))
       for _, key := range req.Keys {
           item := store.Get(key)
           if item != nil {
               resp.items[key] = item
           }
       }
   }
   return resp
}
--------------------------------------------------------------------------------------------------
0x401952 main.*BitcaskStore·Get
--------------------------------------------------------------------------------------------------
func (p *BitcaskStore) Get(key string) *memcache.Item {
   bc := p.getByKey(key)
   r := bc.GetRecord(key)
   if r != nil {
       return &memcache.Item{Body: r.Value, Flag: int(r.Flag)}
   }
   return nil
}
--------------------------------------------------------------------------------------------------
0x444925 bitcask.*Bitcask·GetRecord
--------------------------------------------------------------------------------------------------
func (b *Bitcask) GetRecord(key string) *Record {
   item := b.htree.Get(key)
   file := b.buckets[item.Bucket]
   file.Seek(int64(item.Pos), 0)
   r := new(Record)
   n, err := r.Read(file)
   return r
}
--------------------------------------------------------------------------------------------------
0x445d11 bitcask.*Record·Read
--------------------------------------------------------------------------------------------------
func (r *Record) Read(ro io.Reader) (n int, err os.Error) {
   var crc, ksz, vsz uint32
   buf := make([]byte, RECORD_HEADER)
   n, err = ro.Read(buf)
   o := bytes.NewBuffer(buf)
   binary.Read(o, order, &crc)

   crc_sum := crc32.ChecksumIEEE(o.Bytes())
   binary.Read(o, order, &r.Tstamp)
   binary.Read(o, order, &r.Flag)
   binary.Read(o, order, &r.Version)
   binary.Read(o, order, &ksz)
   binary.Read(o, order, &vsz)

   size := ksz + vsz
   buf = make([]byte, size)
   m, err := ro.Read(buf)
   r.Key = string(buf[:ksz])
   r.Value = buf[ksz : ksz+vsz]

   return n + m, err
}
--------------------------------------------------------------------------------------------------
0x403430 runtime.makeslice
--------------------------------------------------------------------------------------------------
0x407407 mallocgc
--------------------------------------------------------------------------------------------------

Something strange was that, I have two beansdb deployment, one for text (2k per record), 
another one for images (100k per record),  only the later one had memory leak problem.

How are the larger memory blocks (>32k) managed by GC? Are they managed different from small memory blocks?

How can I find why the data buffers are not freed by GC ?

Thank you very much. I will publish the whole code a bit later, if it can help.

 - Davies

The complete heap profile result:
=============================================================

heap profile: 1401: 468380240 [95361: 28283696880] @ heap/1048576
1: 240 [1: 240] @ 0x407407 0x407a23 0x4b5317 0x4b5c57 0x4b62cf 0x4b5b75 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
2: 480 [2: 480] @ 0x407407 0x407a23 0x4b5317 0x4b5c57 0x4b62cf 0x4b5b75 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
2: 224 [2: 224] @ 0x407407 0x407a23 0x4b5317 0x4b59d2 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
10: 1120 [11: 1232] @ 0x407407 0x407a23 0x4b5317 0x4b59d2 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 384 [7: 1616] @ 0x407407 0x4031ee 0x401519 0x43d907 0x43e9a8 0x440783 0x404dc1
16: 2162688 [16: 2162688] @ 0x407407 0x407a23 0x4b5317 0x4b5c57 0x4b62cf 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 32 [13: 416] @ 0x407407 0x407a23 0x407a4c 0x455521 0x454b46 0x49192f 0x4454de 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
4: 256 [4: 256] @ 0x407407 0x407a23 0x407a4c 0x444ea6 0x44427f 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
1: 32 [118: 3776] @ 0x407407 0x407a23 0x407a4c 0x41d3a8 0x41d6fa 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
2: 64 [97: 3104] @ 0x407407 0x407a23 0x407a4c 0x455521 0x454383 0x49155c 0x445c98 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
2: 54960128 [69: 547435264] @ 0x407407 0x403430 0x445d11 0x444925 0x4442d3 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
20: 1280 [22: 1408] @ 0x407407 0x407a23 0x407a4c 0x444ea6 0x44427f 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 64 [235: 15040] @ 0x407407 0x407a23 0x407a4c 0x4382e3 0x438684 0x41d4f1 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
1: 16 [68: 1088] @ 0x407407 0x407a23 0x4082d0 0x408468 0x43ebc2 0x440783 0x404dc1
1: 32 [89: 2848] @ 0x407407 0x407a23 0x407a4c 0x4556bf 0x454b46 0x4914a2 0x445b55 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [79: 2528] @ 0x407407 0x407a23 0x407a4c 0x401b8b 0x43d907 0x43e9a8 0x440783 0x404dc1
2: 32 [254: 4064] @ 0x407407 0x407a23 0x407a4c 0x473797 0x480b7d 0x404dc1
1: 64 [219: 14016] @ 0x407407 0x407a23 0x407a4c 0x4448df 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
9: 1008 [326: 36512] @ 0x407407 0x407a23 0x4b6b3a 0x4b6e43 0x43d7e5 0x43e9a8 0x440783 0x404dc1
1: 96 [285: 27360] @ 0x407407 0x407a23 0x407a4c 0x4311f2 0x445884 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
24: 1088 [117: 4176] @ 0x407407 0x407a23 0x402b48 0x403179 0x45ef24 0x43abc4 0x43ad4b 0x43e920 0x440783 0x404dc1
1: 32 [105: 3360] @ 0x407407 0x407a23 0x407a4c 0x4556bf 0x454b46 0x4914a2 0x445c98 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1161: 348853248 [60311: 17089668736] @ 0x407407 0x403430 0x445d11 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
18: 1090816 [10094: 2680426752] @ 0x407407 0x403430 0x430490 0x430596 0x4455cc 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
2: 224 [283: 31696] @ 0x407407 0x407a23 0x4b5317 0x4b54fa 0x4b6bf1 0x4b6e43 0x43d7e5 0x43e9a8 0x440783 0x404dc1
24: 3227648 [12513: 4744894208] @ 0x407407 0x403430 0x430490 0x430596 0x445669 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
22: 56948736 [9945: 3217405824] @ 0x407407 0x403430 0x43b34f 0x43e920 0x440783 0x404dc1
16: 1114112 [22: 1531904] @ 0x407407 0x407a23 0x4b5317 0x4b54fa 0x4b6bf1 0x4b6e43 0x443d79 0x44441a 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 2048 [1: 2048] @ 0x407407 0x407a23 0x4034af 0x440875 0x400b90 0x4024a8 0x409871 0x404dc1
1: 32 [1: 32] @ 0x407407 0x407a23 0x402b48 0x403179 0x42fa25 0x42ff85 0x440b6e 0x400b90 0x4024a8 0x409871 0x404dc1
3: 6144 [3: 6144] @ 0x407407 0x403430 0x440be7 0x400b90 0x4024a8 0x409871 0x404dc1
49: 7840 [49: 7840] @ 0x407407 0x4075a2 0x405dff 0x40b2fd 0x4054f9

# runtime.MemStats
# Alloc = 1181107696
# TotalAlloc = 76062000960
# Sys = 2201004280
# Lookups = 976782026
# Mallocs = 200162154
# HeapAlloc = 1183374752
# HeapSys = 1727463424
# HeapIdle = 0
# HeapInuse = 1252605952
# Stack = 0 / 131072
# MSpan = 3482064 / 4063232
# MCache = 55200 / 131072
# MHeapMapSys = 7864320
# BuckHashSys = 1439992
# NextGC = 1972644160
# PauseNs = 276075832000
# NumGC = 2515
# EnableGC = true
# DebugGC = false
# BySize = Size * (Active = Mallocs - Frees)
# (Excluding large blocks.)
#   8 * (69018 = 48162125 - 48093107)
#   16 * (70869 = 37384345 - 37313476)
#   32 * (169663 = 61675569 - 61505906)
#   48 * (254342 = 31962845 - 31708503)
#   64 * (275293 = 9625084 - 9349791)
#   80 * (3519 = 2248668 - 2245149)
#   96 * (5132 = 3353599 - 3348467)
#   112 * (88342 = 3288414 - 3200072)
#   128 * (71 = 18191 - 18120)
#   144 * (127 = 26929 - 26802)
#   160 * (425 = 29538 - 29113)
#   176 * (82 = 15038 - 14956)
#   192 * (53 = 16816 - 16763)
#   208 * (67 = 35197 - 35130)
#   224 * (80 = 45345 - 45265)
#   240 * (6265 = 21132 - 14867)
#   256 * (21 = 14789 - 14768)
#   288 * (12 = 1112 - 1100)
#   320 * (326 = 12401 - 12075)
#   352 * (83 = 48973 - 48890)
#   384 * (15 = 977 - 962)
#   448 * (15 = 1057 - 1042)
#   480 * (14 = 743 - 729)
#   576 * (42 = 1024 - 982)
#   640 * (17 = 174 - 157)
#   704 * (0 = 68 - 68)
#   768 * (15 = 1183 - 1168)
#   832 * (0 = 60 - 60)
#   960 * (2 = 152 - 150)
#   1152 * (285 = 3363 - 3078)
#   1280 * (90 = 10757 - 10667)
#   1408 * (7 = 4876 - 4869)
#   1536 * (379 = 39713 - 39334)
#   1664 * (7 = 4810 - 4803)
#   1920 * (680 = 66207 - 65527)
#   2048 * (1168 = 44511 - 43343)
#   2304 * (848 = 64377 - 63529)
#   2560 * (1325 = 96511 - 95186)
#   2816 * (616 = 45836 - 45220)
#   3072 * (580 = 42901 - 42321)
#   3328 * (561 = 45786 - 45225)
#   3840 * (1190 = 100938 - 99748)
#   4096 * (725 = 51497 - 50772)
#   4352 * (613 = 43110 - 42497)
#   4608 * (472 = 38796 - 38324)
#   5120 * (881 = 96168 - 95287)
#   5888 * (1265 = 99238 - 97973)
#   6656 * (1278 = 105566 - 104288)
#   6912 * (439 = 35730 - 35291)
#   7936 * (1637 = 138063 - 136426)
#   8192 * (377 = 30232 - 29855)
#   8704 * (703 = 54816 - 54113)
#   9984 * (1331 = 100387 - 99056)
#   10496 * (332 = 28544 - 28212)
#   12032 * (710 = 61013 - 60303)
#   12288 * (83 = 7851 - 7768)
#   14080 * (472 = 47307 - 46835)
#   16128 * (376 = 44876 - 44500)
#   17664 * (236 = 27220 - 26984)
#   20224 * (280 = 32653 - 32373)
#   21248 * (88 = 9229 - 9141)
#   24320 * (4291 = 162028 - 157737)
#   24832 * (23 = 2350 - 2327)
#   28416 * (154 = 15563 - 15409)
#   32512 * (173 = 15247 - 15074)
#   32768 * (10 = 931 - 921)

After I complete the email, it became:


heap profile: 1899: 586871992 [100012: 28801272576] @ heap/1048576
1: 240 [1: 240] @ 0x407407 0x407a23 0x4b5317 0x4b5c57 0x4b62cf 0x4b5b75 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
2: 480 [2: 480] @ 0x407407 0x407a23 0x4b5317 0x4b5c57 0x4b62cf 0x4b5b75 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
2: 224 [2: 224] @ 0x407407 0x407a23 0x4b5317 0x4b59d2 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
11: 1232 [12: 1344] @ 0x407407 0x407a23 0x4b5317 0x4b59d2 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 384 [7: 1616] @ 0x407407 0x4031ee 0x401519 0x43d907 0x43e9a8 0x440783 0x404dc1
16: 2162688 [16: 2162688] @ 0x407407 0x407a23 0x4b5317 0x4b5c57 0x4b62cf 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 8 [5: 40] @ 0x407407 0x407a23 0x4091ef 0x454afe 0x49192f 0x44539a 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 8 [3: 24] @ 0x407407 0x403430 0x491a46 0x44543c 0x4444c4 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
1: 8 [27: 216] @ 0x407407 0x407a23 0x407a4c 0x445779 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [2: 64] @ 0x407407 0x407a23 0x407a4c 0x401819 0x43d907 0x43e9a8 0x440783 0x404dc1
4: 256 [4: 256] @ 0x407407 0x407a23 0x407a4c 0x444ea6 0x44427f 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
1: 8 [32: 256] @ 0x407407 0x407a23 0x407a4c 0x445792 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [79: 2528] @ 0x407407 0x4031ee 0x400fad 0x40192a 0x43d907 0x43e9a8 0x440783 0x404dc1
2: 64 [121: 3872] @ 0x407407 0x407a23 0x407a4c 0x41d3a8 0x41d6fa 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
1: 16 [52: 832] @ 0x407407 0x407a23 0x4082d0 0x408468 0x43cff1 0x43ea19 0x440783 0x404dc1
2: 192 [55: 5280] @ 0x407407 0x407a23 0x407a4c 0x4311f2 0x445251 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 8 [26: 208] @ 0x407407 0x407a23 0x407a4c 0x4457ab 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
4: 39510016 [73: 586945280] @ 0x407407 0x403430 0x445d11 0x444925 0x4442d3 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 48 [168: 7728] @ 0x407407 0x407a23 0x402b48 0x403179 0x430202 0x413eeb 0x41de8c 0x43ec94 0x440783 0x404dc1
1: 32 [88: 2816] @ 0x407407 0x407a23 0x402b48 0x403179 0x445fde 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
21: 1344 [23: 1472] @ 0x407407 0x407a23 0x407a4c 0x444ea6 0x44427f 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 8 [23: 184] @ 0x407407 0x4031ee 0x44d58a 0x43d0f9 0x43ea19 0x440783 0x404dc1
1: 64 [239: 15296] @ 0x407407 0x407a23 0x407a4c 0x4382e3 0x438684 0x41d4f1 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
2: 96 [169: 8112] @ 0x407407 0x407a23 0x407a4c 0x43d390 0x43e9a8 0x440783 0x404dc1
1: 32 [119: 3808] @ 0x407407 0x407a23 0x402b48 0x402da5 0x402e67 0x41d8f7 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
1: 32 [80: 2560] @ 0x407407 0x407a23 0x407a4c 0x401b8b 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [96: 3072] @ 0x407407 0x407a23 0x407a4c 0x455521 0x454383 0x49155c 0x445bf8 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 16 [67: 1072] @ 0x407407 0x407a23 0x4082d0 0x408468 0x43eb89 0x440783 0x404dc1
4: 64 [261: 4176] @ 0x407407 0x407a23 0x407a4c 0x473797 0x480b7d 0x404dc1
4: 256 [240: 15360] @ 0x407407 0x407a23 0x407a4c 0x43eb41 0x440783 0x404dc1
2: 32 [57: 912] @ 0x407407 0x407a23 0x402b48 0x402da5 0x402e67 0x41d7aa 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
2: 128 [221: 14144] @ 0x407407 0x407a23 0x407a4c 0x4448df 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 16 [76: 1216] @ 0x407407 0x407a23 0x402b48 0x402da5 0x402e67 0x41d63b 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
1: 8 [20: 160] @ 0x407407 0x407a23 0x4091ef 0x454afe 0x4914a2 0x445c98 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [129: 4128] @ 0x407407 0x407a23 0x407a4c 0x41d3a8 0x41d5b4 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
11: 1232 [332: 37184] @ 0x407407 0x407a23 0x4b6b3a 0x4b6e43 0x43d7e5 0x43e9a8 0x440783 0x404dc1
2: 192 [290: 27840] @ 0x407407 0x407a23 0x407a4c 0x4311f2 0x445884 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 48 [371: 17808] @ 0x407407 0x407a23 0x407a4c 0x41ee08 0x480d1f 0x404dc1
24: 1088 [117: 4176] @ 0x407407 0x407a23 0x402b48 0x403179 0x45ef24 0x43abc4 0x43ad4b 0x43e920 0x440783 0x404dc1
1: 32 [101: 3232] @ 0x407407 0x407a23 0x407a4c 0x454e17 0x454383 0x49155c 0x445b55 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [108: 3456] @ 0x407407 0x407a23 0x407a4c 0x4556bf 0x454b46 0x4914a2 0x445c98 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [130: 4160] @ 0x407407 0x407a23 0x407a4c 0x41d3a8 0x41d734 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
1: 32 [94: 3008] @ 0x407407 0x407a23 0x407a4c 0x454e17 0x454383 0x49155c 0x445aac 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [107: 3424] @ 0x407407 0x407a23 0x407a4c 0x4556bf 0x454b46 0x4914a2 0x445aac 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [81: 2592] @ 0x407407 0x407a23 0x407a4c 0x4556bf 0x454b46 0x4914a2 0x445bf8 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [154: 4928] @ 0x407407 0x407a23 0x407a4c 0x401110 0x4016bb 0x43d907 0x43e9a8 0x440783 0x404dc1
1475: 398708736 [61248: 17301198848] @ 0x407407 0x403430 0x445d11 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
2: 160 [313: 25040] @ 0x407407 0x407a23 0x407a4c 0x43e903 0x440783 0x404dc1
56: 25199104 [10255: 2735204992] @ 0x407407 0x403430 0x430490 0x430596 0x4455cc 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
4: 448 [286: 32032] @ 0x407407 0x407a23 0x4b5317 0x4b54fa 0x4b6bf1 0x4b6e43 0x43d7e5 0x43e9a8 0x440783 0x404dc1
1: 32 [113: 3616] @ 0x407407 0x407a23 0x407a4c 0x41d3a8 0x41d57a 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
1: 64 [243: 15392] @ 0x407407 0x4031ee 0x44d58a 0x41dd19 0x41deb0 0x43ec94 0x440783 0x404dc1
1: 32 [118: 3776] @ 0x407407 0x407a23 0x407a4c 0x41d3a8 0x41d6c0 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
1: 4096 [19: 77824] @ 0x407407 0x403430 0x45f065 0x45f0e7 0x43e8ed 0x440783 0x404dc1
83: 55060224 [12744: 4861977856] @ 0x407407 0x403430 0x430490 0x430596 0x445669 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
61: 65088000 [10117: 3311861760] @ 0x407407 0x403430 0x43b34f 0x43e920 0x440783 0x404dc1
16: 1114112 [22: 1531904] @ 0x407407 0x407a23 0x4b5317 0x4b54fa 0x4b6bf1 0x4b6e43 0x443d79 0x44441a 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 2048 [1: 2048] @ 0x407407 0x407a23 0x4034af 0x440875 0x400b90 0x4024a8 0x409871 0x404dc1
1: 32 [1: 32] @ 0x407407 0x407a23 0x402b48 0x403179 0x42fa25 0x42ff85 0x440b6e 0x400b90 0x4024a8 0x409871 0x404dc1
3: 6144 [3: 6144] @ 0x407407 0x403430 0x440be7 0x400b90 0x4024a8 0x409871 0x404dc1
49: 7840 [49: 7840] @ 0x407407 0x4075a2 0x405dff 0x40b2fd 0x4054f9

# runtime.MemStats
# Alloc = 1545143272
# TotalAlloc = 77360115520
# Sys = 2545236216
# Lookups = 982373279
# Mallocs = 203303881
# HeapAlloc = 1547709512
# HeapSys = 2002845696
# HeapIdle = 0
# HeapInuse = 1593593856
# Stack = 0 / 131072
# MSpan = 3887928 / 4587520
# MCache = 55200 / 131072
# MHeapMapSys = 8388608
# BuckHashSys = 1439992
# NextGC = 2123655552
# PauseNs = 277198331000
# NumGC = 2517
# EnableGC = true
# DebugGC = false
# BySize = Size * (Active = Mallocs - Frees)
# (Excluding large blocks.)
#   8 * (259535 = 48924448 - 48664913)
#   16 * (226649 = 37986927 - 37760278)
#   32 * (402841 = 62616882 - 62214041)
#   48 * (299951 = 32474785 - 32174834)
#   64 * (313071 = 9770769 - 9457698)
#   80 * (11864 = 2283008 - 2271144)
#   96 * (16690 = 3403467 - 3386777)
#   112 * (111769 = 3342398 - 3230629)
#   128 * (125 = 18434 - 18309)
#   144 * (229 = 27265 - 27036)
#   160 * (595 = 30081 - 29486)
#   176 * (147 = 15299 - 15152)
#   192 * (124 = 17098 - 16974)
#   208 * (166 = 35689 - 35523)
#   224 * (310 = 46092 - 45782)
#   240 * (7380 = 22340 - 14960)
#   256 * (96 = 15035 - 14939)
#   288 * (24 = 1133 - 1109)
#   320 * (487 = 12703 - 12216)
#   352 * (325 = 49754 - 49429)
#   384 * (42 = 1041 - 999)
#   448 * (40 = 1126 - 1086)
#   480 * (41 = 806 - 765)
#   576 * (87 = 1127 - 1040)
#   640 * (33 = 206 - 173)
#   704 * (8 = 80 - 72)
#   768 * (19 = 1194 - 1175)
#   832 * (0 = 60 - 60)
#   960 * (2 = 156 - 154)
#   1152 * (299 = 3402 - 3103)
#   1280 * (122 = 10842 - 10720)
#   1408 * (22 = 4925 - 4903)
#   1536 * (500 = 40036 - 39536)
#   1664 * (23 = 4865 - 4842)
#   1920 * (886 = 66792 - 65906)
#   2048 * (1378 = 45079 - 43701)
#   2304 * (1124 = 65203 - 64079)
#   2560 * (1683 = 97588 - 95905)
#   2816 * (831 = 46441 - 45610)
#   3072 * (746 = 43488 - 42742)
#   3328 * (758 = 46405 - 45647)
#   3840 * (1702 = 102342 - 100640)
#   4096 * (997 = 52315 - 51318)
#   4352 * (851 = 43741 - 42890)
#   4608 * (643 = 39331 - 38688)
#   5120 * (1182 = 98331 - 97149)
#   5888 * (1866 = 100799 - 98933)
#   6656 * (1744 = 107139 - 105395)
#   6912 * (616 = 36242 - 35626)
#   7936 * (2321 = 140034 - 137713)
#   8192 * (504 = 30683 - 30179)
#   8704 * (911 = 55591 - 54680)
#   9984 * (1782 = 101844 - 100062)
#   10496 * (448 = 28917 - 28469)
#   12032 * (996 = 61921 - 60925)
#   12288 * (125 = 7982 - 7857)
#   14080 * (685 = 48024 - 47339)
#   16128 * (599 = 45604 - 45005)
#   17664 * (338 = 27651 - 27313)
#   20224 * (429 = 33229 - 32800)
#   21248 * (129 = 9371 - 9242)
#   24320 * (6003 = 166560 - 160557)
#   24832 * (33 = 2389 - 2356)
#   28416 * (254 = 15810 - 15556)
#   32512 * (275 = 15511 - 15236)
#   32768 * (17 = 951 - 934)


--
 - Davies

Davies Liu

unread,
Sep 28, 2010, 1:13:08 PM9/28/10
to r...@golang.org, golang-nuts

heap profile: 1401: 468380240 [95361: 28283696880] @ heap/1048576
1: 240 [1: 240] @ 0x407407 0x407a23 0x4b5317 0x4b5c57 0x4b62cf 0x4b5b75 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
2: 480 [2: 480] @ 0x407407 0x407a23 0x4b5317 0x4b5c57 0x4b62cf 0x4b5b75 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
2: 224 [2: 224] @ 0x407407 0x407a23 0x4b5317 0x4b59d2 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
10: 1120 [11: 1232] @ 0x407407 0x407a23 0x4b5317 0x4b59d2 0x4b6314 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 384 [7: 1616] @ 0x407407 0x4031ee 0x401519 0x43d907 0x43e9a8 0x440783 0x404dc1
16: 2162688 [16: 2162688] @ 0x407407 0x407a23 0x4b5317 0x4b5c57 0x4b62cf 0x4b65fd 0x4b71de 0x4b7398 0x444560 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 32 [13: 416] @ 0x407407 0x407a23 0x407a4c 0x455521 0x454b46 0x49192f 0x4454de 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
4: 256 [4: 256] @ 0x407407 0x407a23 0x407a4c 0x444ea6 0x44427f 0x444da1 0x401d47 0x43d69d 0x43e9a8 0x440783 0x404dc1
1: 32 [118: 3776] @ 0x407407 0x407a23 0x407a4c 0x41d3a8 0x41d6fa 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
2: 64 [97: 3104] @ 0x407407 0x407a23 0x407a4c 0x455521 0x454383 0x49155c 0x445c98 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
2: 54960128 [69: 547435264] @ 0x407407 0x403430 0x445d11 0x444925 0x4442d3 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
20: 1280 [22: 1408] @ 0x407407 0x407a23 0x407a4c 0x444ea6 0x44427f 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 64 [235: 15040] @ 0x407407 0x407a23 0x407a4c 0x4382e3 0x438684 0x41d4f1 0x41dc8a 0x41deb0 0x43ec94 0x440783 0x404dc1
1: 16 [68: 1088] @ 0x407407 0x407a23 0x4082d0 0x408468 0x43ebc2 0x440783 0x404dc1
1: 32 [89: 2848] @ 0x407407 0x407a23 0x407a4c 0x4556bf 0x454b46 0x4914a2 0x445b55 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1: 32 [79: 2528] @ 0x407407 0x407a23 0x407a4c 0x401b8b 0x43d907 0x43e9a8 0x440783 0x404dc1
2: 32 [254: 4064] @ 0x407407 0x407a23 0x407a4c 0x473797 0x480b7d 0x404dc1
1: 64 [219: 14016] @ 0x407407 0x407a23 0x407a4c 0x4448df 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
9: 1008 [326: 36512] @ 0x407407 0x407a23 0x4b6b3a 0x4b6e43 0x43d7e5 0x43e9a8 0x440783 0x404dc1
1: 96 [285: 27360] @ 0x407407 0x407a23 0x407a4c 0x4311f2 0x445884 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
24: 1088 [117: 4176] @ 0x407407 0x407a23 0x402b48 0x403179 0x45ef24 0x43abc4 0x43ad4b 0x43e920 0x440783 0x404dc1
1: 32 [105: 3360] @ 0x407407 0x407a23 0x407a4c 0x4556bf 0x454b46 0x4914a2 0x445c98 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
1161: 348853248 [60311: 17089668736] @ 0x407407 0x403430 0x445d11 0x444925 0x401952 0x43d907 0x43e9a8 0x440783 0x404dc1
18: 1090816 [10094: 2680426752] @ 0x407407 0x403430 0x430490 0x430596 0x4455cc 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
2: 224 [283: 31696] @ 0x407407 0x407a23 0x4b5317 0x4b54fa 0x4b6bf1 0x4b6e43 0x43d7e5 0x43e9a8 0x440783 0x404dc1
24: 3227648 [12513: 4744894208] @ 0x407407 0x403430 0x430490 0x430596 0x445669 0x4444c4 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
22: 56948736 [9945: 3217405824] @ 0x407407 0x403430 0x43b34f 0x43e920 0x440783 0x404dc1
16: 1114112 [22: 1531904] @ 0x407407 0x407a23 0x4b5317 0x4b54fa 0x4b6bf1 0x4b6e43 0x443d79 0x44441a 0x401c9d 0x43d56d 0x43e9a8 0x440783 0x404dc1
1: 2048 [1: 2048] @ 0x407407 0x407a23 0x4034af 0x440875 0x400b90 0x4024a8 0x409871 0x404dc1
1: 32 [1: 32] @ 0x407407 0x407a23 0x402b48 0x403179 0x42fa25 0x42ff85 0x440b6e 0x400b90 0x4024a8 0x409871 0x404dc1
3: 6144 [3: 6144] @ 0x407407 0x403430 0x440be7 0x400b90 0x4024a8 0x409871 0x404dc1
49: 7840 [49: 7840] @ 0x407407 0x4075a2 0x405dff 0x40b2fd 0x4054f9

# runtime.MemStats
# Alloc = 1181107696
# TotalAlloc = 76062000960
# Sys = 2201004280
# Lookups = 976782026
# Mallocs = 200162154
# HeapAlloc = 1183374752
# HeapSys = 1727463424
# HeapIdle = 0
# HeapInuse = 1252605952
# Stack = 0 / 131072
# MSpan = 3482064 / 4063232
# MCache = 55200 / 131072
# MHeapMapSys = 7864320
# BuckHashSys = 1439992
# NextGC = 1972644160
# PauseNs = 276075832000
# NumGC = 2515
# EnableGC = true
# DebugGC = false
# BySize = Size * (Active = Mallocs - Frees)
# (Excluding large blocks.)
#   8 * (69018 = 48162125 - 48093107)
#   16 * (70869 = 37384345 - 37313476)
#   32 * (169663 = 61675569 - 61505906)
#   48 * (254342 = 31962845 - 31708503)
#   64 * (275293 = 9625084 - 9349791)
#   80 * (3519 = 2248668 - 2245149)
#   96 * (5132 = 3353599 - 3348467)
#   112 * (88342 = 3288414 - 3200072)
#   128 * (71 = 18191 - 18120)
#   144 * (127 = 26929 - 26802)
#   160 * (425 = 29538 - 29113)
#   176 * (82 = 15038 - 14956)
#   192 * (53 = 16816 - 16763)
#   208 * (67 = 35197 - 35130)
#   224 * (80 = 45345 - 45265)
#   240 * (6265 = 21132 - 14867)
#   256 * (21 = 14789 - 14768)
#   288 * (12 = 1112 - 1100)
#   320 * (326 = 12401 - 12075)
#   352 * (83 = 48973 - 48890)
#   384 * (15 = 977 - 962)
#   448 * (15 = 1057 - 1042)
#   480 * (14 = 743 - 729)
#   576 * (42 = 1024 - 982)
#   640 * (17 = 174 - 157)
#   704 * (0 = 68 - 68)
#   768 * (15 = 1183 - 1168)
#   832 * (0 = 60 - 60)
#   960 * (2 = 152 - 150)
#   1152 * (285 = 3363 - 3078)
#   1280 * (90 = 10757 - 10667)
#   1408 * (7 = 4876 - 4869)
#   1536 * (379 = 39713 - 39334)
#   1664 * (7 = 4810 - 4803)
#   1920 * (680 = 66207 - 65527)
#   2048 * (1168 = 44511 - 43343)
#   2304 * (848 = 64377 - 63529)
#   2560 * (1325 = 96511 - 95186)
#   2816 * (616 = 45836 - 45220)
#   3072 * (580 = 42901 - 42321)
#   3328 * (561 = 45786 - 45225)
#   3840 * (1190 = 100938 - 99748)
#   4096 * (725 = 51497 - 50772)
#   4352 * (613 = 43110 - 42497)
#   4608 * (472 = 38796 - 38324)
#   5120 * (881 = 96168 - 95287)
#   5888 * (1265 = 99238 - 97973)
#   6656 * (1278 = 105566 - 104288)
#   6912 * (439 = 35730 - 35291)
#   7936 * (1637 = 138063 - 136426)
#   8192 * (377 = 30232 - 29855)
#   8704 * (703 = 54816 - 54113)
#   9984 * (1331 = 100387 - 99056)
#   10496 * (332 = 28544 - 28212)
#   12032 * (710 = 61013 - 60303)
#   12288 * (83 = 7851 - 7768)
#   14080 * (472 = 47307 - 46835)
#   16128 * (376 = 44876 - 44500)
#   17664 * (236 = 27220 - 26984)
#   20224 * (280 = 32653 - 32373)
#   21248 * (88 = 9229 - 9141)
#   24320 * (4291 = 162028 - 157737)
#   24832 * (23 = 2350 - 2327)
#   28416 * (154 = 15563 - 15409)
#   32512 * (173 = 15247 - 15074)
#   32768 * (10 = 931 - 921)

Russ Cox

unread,
Sep 28, 2010, 1:37:57 PM9/28/10
to Davies Liu, golang-nuts
Are you using the 386 or amd64 port?
It's possible that the big images stick around
because of false positives in the conservative
garbage collector.

Russ

Davies Liu

unread,
Sep 28, 2010, 1:57:15 PM9/28/10
to r...@golang.org, golang-nuts
6g, amd64.

Maybe, I should manage the read buffer manually, 
re-use the buffer in the same goroutine,  to stop it 
from allocing new one. 

If the memory leak problem can not been solved in Go, 
I HAVE to re-write it in C instead, very sadly.

--
 - Davies

Russ Cox

unread,
Sep 28, 2010, 2:23:24 PM9/28/10
to Davies Liu, golang-nuts
If it's amd64 it's not the conservative garbage collector issue.
Are you sure that you are releasing all the references
to the Records being created? The stack you showed is
entirely reasonable if the Records are being kept around
somewhere.

Russ

Reply all
Reply to author
Forward
0 new messages