memory consuption

63 views
Skip to first unread message

gms...@gmail.com

unread,
Oct 16, 2014, 4:58:34 AM10/16/14
to nod...@googlegroups.com
Hi all,

I'm trying to investigate strange memory consumption, but
 I have no explanation
.

Tested data is 100kb json text.

Tested script:
var fs = require('fs');
var data = fs.readFileSync(__dirname + '/' +'chartype.letter.json', 'utf8');

var p0 = JSON.parse(data);
var p1 = JSON.parse(data);
var p2 = JSON.parse(data);
var p3 = JSON.parse(data);
var p4 = JSON.parse(data);
var p5 = JSON.parse(data);
var p6 = JSON.parse(data);
var p7 = JSON.parse(data);
var p8 = JSON.parse(data);
var p9 = JSON.parse(data);

Empty case:
/usr/bin/time -f "%MKb" ./node empty.js
35360Kb

So expected result for main case will be empty case + data * count_times and it should be something like 35360Kb + 100Kb * 10 ~~ 37Mb

Main case:
/usr/bin/time -f "%MKb" ./node bench.js
276672Kb

Actual result is 270Mb.

Running mtrace :
Addr            Size       Count  Traffic  Module                            Symbol                          Offs 
--------------  ---------  -----  -------  --------------------------------  ------------------------------  ------
0x6b3bb7        0          0      2        ./build/bin/node                  ares_init_options               +0x857
0x6b3df7        0          0      2        ./build/bin/node                  ares_init_options               +0xa97
0x6b93c6        0          0      2        ./build/bin/node                  ares__read_line                 +0xd6
0x7ff99d96aebb  0          0      6        /lib/x86_64-linux-gnu/libc.so.6                                        
0x7ff99d984ab2  3          1      3        /lib/x86_64-linux-gnu/libc.so.6   __strdup                        +0x22
0x6b41e0        360        1      5        ./build/bin/node                  ares_init_options               +0xe80
0x90a519        928        4      738      ./build/bin/node                  _ZN2v88internal8Malloced3NewEm  +0x9 
0x7ff99e44936d  5270       8      870      /usr/lib/x86_64-linux-gnu/lib...  _Znwm                           +0x1d
0x6b33da        74216      1      1        ./build/bin/node                  ares_init_options               +0x7a
--------------  ---------  -----  -------  --------------------------------  ------------------------------  ------
Totals          80777      15     1629

Traising GC.
./build/bin/node --trace_gc --trace_gc_verbose ../bench.js

[26109]       19 ms: Scavenge 2.1 (34.0) -> 2.0 (36.0) MB, 0 ms [allocation failure].
[26109] Memory allocator,   used:  36880 KB, available: 1462256 KB
[26109] New space,          used:    251 KB, available:    772 KB, committed:   2048 KB
[26109] Old pointers,       used:    708 KB, available:      1 KB, committed:   1519 KB
[26109] Old data space,     used:    570 KB, available:      0 KB, committed:   1199 KB
[26109] Code space,         used:    395 KB, available:      0 KB, committed:    996 KB
[26109] Map space,          used:     83 KB, available:      0 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:      0 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1461215 KB, committed:      0 KB
[26109] All spaces,         used:   2024 KB, available:    773 KB, committed:   6019 KB
[26109] Total time spent in GC  : 0 ms
[26109]       20 ms: Scavenge 2.7 (36.0) -> 2.7 (37.0) MB, 0 ms [allocation failure].
[26109] Memory allocator,   used:  37904 KB, available: 1461232 KB
[26109] New space,          used:    251 KB, available:    772 KB, committed:   2048 KB
[26109] Old pointers,       used:    716 KB, available:      1 KB, committed:   1519 KB
[26109] Old data space,     used:   1316 KB, available:      0 KB, committed:   2207 KB
[26109] Code space,         used:    395 KB, available:      0 KB, committed:    996 KB
[26109] Map space,          used:     83 KB, available:      0 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:      0 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1460191 KB, committed:      0 KB
[26109] All spaces,         used:   2778 KB, available:    773 KB, committed:   7027 KB
[26109] Total time spent in GC  : 0 ms
[26109]       20 ms: Scavenge 3.6 (37.0) -> 3.6 (37.0) MB, 0 ms [allocation failure].
[26109] Memory allocator,   used:  37904 KB, available: 1461232 KB
[26109] New space,          used:    503 KB, available:   1544 KB, committed:   4096 KB
[26109] Old pointers,       used:    908 KB, available:      1 KB, committed:   1519 KB
[26109] Old data space,     used:   1818 KB, available:      0 KB, committed:   2207 KB
[26109] Code space,         used:    395 KB, available:      0 KB, committed:    996 KB
[26109] Map space,          used:     83 KB, available:      0 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:      0 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1460191 KB, committed:      0 KB
[26109] All spaces,         used:   3724 KB, available:   1545 KB, committed:   9075 KB
[26109] Total time spent in GC  : 0 ms
[26109]       22 ms: Scavenge 5.1 (37.0) -> 5.1 (39.0) MB, 0 ms [allocation failure].
[26109] Memory allocator,   used:  39952 KB, available: 1459184 KB
[26109] New space,          used:    503 KB, available:   1544 KB, committed:   4096 KB
[26109] Old pointers,       used:    908 KB, available:      1 KB, committed:   1519 KB
[26109] Old data space,     used:   3326 KB, available:      2 KB, committed:   4223 KB
[26109] Code space,         used:    395 KB, available:      0 KB, committed:    996 KB
[26109] Map space,          used:     83 KB, available:      0 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:      0 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1458143 KB, committed:      0 KB
[26109] All spaces,         used:   5233 KB, available:   1547 KB, committed:  11090 KB
[26109] Total time spent in GC  : 0 ms
[26109]       24 ms: Scavenge 6.6 (39.0) -> 6.6 (40.0) MB, 1 ms [allocation failure].
[26109] Memory allocator,   used:  40976 KB, available: 1458160 KB
[26109] New space,          used:   1007 KB, available:   3088 KB, committed:   8192 KB
[26109] Old pointers,       used:    908 KB, available:      1 KB, committed:   1519 KB
[26109] Old data space,     used:   4331 KB, available:      2 KB, committed:   5230 KB
[26109] Code space,         used:    395 KB, available:      0 KB, committed:    996 KB
[26109] Map space,          used:     83 KB, available:      0 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:      0 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1457119 KB, committed:      0 KB
[26109] All spaces,         used:   6741 KB, available:   3091 KB, committed:  16194 KB
[26109] Total time spent in GC  : 1 ms
[26109]       28 ms: Mark-sweep 10.3 (41.0) -> 2.9 (41.0) MB, 2 ms [allocation failure] [promotion limit reached].
[26109] Memory allocator,   used:  42000 KB, available: 1457136 KB
[26109] New space,          used:      0 KB, available:   4096 KB, committed:   8192 KB
[26109] Old pointers,       used:   1441 KB, available:    894 KB, committed:   2527 KB
[26109] Old data space,     used:   1092 KB, available:   3564 KB, committed:   5230 KB
[26109] Code space,         used:    384 KB, available:    612 KB, committed:    996 KB
[26109] Map space,          used:     65 KB, available:     62 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:    113 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1456095 KB, committed:      0 KB
[26109] All spaces,         used:   2997 KB, available:   9343 KB, committed:  17202 KB
[26109] Total time spent in GC  : 3 ms
[26109]       30 ms: Scavenge 7.0 (41.0) -> 7.0 (44.0) MB, 1 ms [allocation failure].
[26109] Memory allocator,   used:  45072 KB, available: 1454064 KB
[26109] New space,          used:   1007 KB, available:   3088 KB, committed:   8192 KB
[26109] Old pointers,       used:   1633 KB, available:    284 KB, committed:   2527 KB
[26109] Old data space,     used:   4109 KB, available:   3566 KB, committed:   8254 KB
[26109] Code space,         used:    384 KB, available:    612 KB, committed:    996 KB
[26109] Map space,          used:     65 KB, available:     62 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:    113 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1453023 KB, committed:      0 KB
[26109] All spaces,         used:   7213 KB, available:   7726 KB, committed:  20225 KB
[26109] Total time spent in GC  : 4 ms
[26109]       32 ms: Scavenge 10.7 (45.0) -> 7.0 (45.0) MB, 0 ms [allocation failure].
[26109] Memory allocator,   used:  46096 KB, available: 1453040 KB
[26109] New space,          used:    118 KB, available:   8073 KB, committed:  16384 KB
[26109] Old pointers,       used:   2401 KB, available:    894 KB, committed:   3535 KB
[26109] Old data space,     used:   4139 KB, available:   3566 KB, committed:   8254 KB
[26109] Code space,         used:    384 KB, available:    612 KB, committed:    996 KB
[26109] Map space,          used:     65 KB, available:     62 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:    113 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1451999 KB, committed:      0 KB
[26109] All spaces,         used:   7123 KB, available:  13321 KB, committed:  29425 KB
[26109] Total time spent in GC  : 4 ms
[26109]       37 ms: Scavenge 14.7 (45.0) -> 14.5 (51.0) MB, 3 ms [allocation failure].
[26109] Memory allocator,   used:  52240 KB, available: 1446896 KB
[26109] New space,          used:   2015 KB, available:   6176 KB, committed:  16384 KB
[26109] Old pointers,       used:   3169 KB, available:   1134 KB, committed:   4543 KB
[26109] Old data space,     used:   9150 KB, available:   3568 KB, committed:  13292 KB
[26109] Code space,         used:    384 KB, available:    612 KB, committed:    996 KB
[26109] Map space,          used:     65 KB, available:     62 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:    113 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1445855 KB, committed:      0 KB
[26109] All spaces,         used:  14799 KB, available:  11667 KB, committed:  35471 KB
[26109] Total time spent in GC  : 7 ms
[26109] Increasing marking speed to 3 due to high promotion rate
[26109]       45 ms: Mark-sweep 20.4 (51.0) -> 16.3 (54.0) MB, 4 ms (+ 2 ms in 35 steps since start of marking, biggest step 0.533936 ms) [allocation failure] [promotion limit reached].
[26109] Memory allocator,   used:  55312 KB, available: 1443824 KB
[26109] New space,          used:      0 KB, available:   8192 KB, committed:  16384 KB
[26109] Old pointers,       used:   3117 KB, available:   1378 KB, committed:   4543 KB
[26109] Old data space,     used:  13075 KB, available:   3239 KB, committed:  16316 KB
[26109] Code space,         used:    384 KB, available:    612 KB, committed:    996 KB
[26109] Map space,          used:     65 KB, available:     62 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:    113 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1442783 KB, committed:      0 KB
[26109] All spaces,         used:  16656 KB, available:  13597 KB, committed:  38495 KB
[26109] Total time spent in GC  : 11 ms
[26109]       50 ms: Scavenge 24.9 (54.0) -> 22.7 (56.0) MB, 2 ms [allocation failure].
[26109] Memory allocator,   used:  57360 KB, available: 1441776 KB
[26109] New space,          used:   4030 KB, available:  12353 KB, committed:  32768 KB
[26109] Old pointers,       used:   4077 KB, available:    418 KB, committed:   4543 KB
[26109] Old data space,     used:  14701 KB, available:   3240 KB, committed:  18331 KB
[26109] Code space,         used:    384 KB, available:    612 KB, committed:    996 KB
[26109] Map space,          used:     65 KB, available:     61 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:    113 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1440735 KB, committed:      0 KB
[26109] All spaces,         used:  23274 KB, available:  16798 KB, committed:  56894 KB
[26109] Total time spent in GC  : 13 ms
[26109]       65 ms: Mark-sweep 33.8 (57.0) -> 12.8 (58.0) MB, 5 ms (+ 3 ms in 40 steps since start of marking, biggest step 0.824951 ms) [StackGuard GC request] [GC in old space requested].
[26109] Memory allocator,   used:  59408 KB, available: 1439728 KB
[26109] New space,          used:      0 KB, available:  16384 KB, committed:  32768 KB
[26109] Old pointers,       used:   5407 KB, available:    911 KB, committed:   6558 KB
[26109] Old data space,     used:   7214 KB, available:  11116 KB, committed:  18331 KB
[26109] Code space,         used:    373 KB, available:    622 KB, committed:    996 KB
[26109] Map space,          used:     65 KB, available:     62 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:    113 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1438687 KB, committed:      0 KB
[26109] All spaces,         used:  13075 KB, available:  29210 KB, committed:  58910 KB
[26109] Total time spent in GC  : 18 ms
[26109]       72 ms: Scavenge 28.5 (58.0) -> 20.3 (62.0) MB, 3 ms [allocation failure].
[26109] Memory allocator,   used:  63504 KB, available: 1435632 KB
[26109] New space,          used:   4030 KB, available:  12353 KB, committed:  32768 KB
[26109] Old pointers,       used:   6943 KB, available:   1391 KB, committed:   8574 KB
[26109] Old data space,     used:   9356 KB, available:  10989 KB, committed:  20347 KB
[26109] Code space,         used:    373 KB, available:    622 KB, committed:    996 KB
[26109] Map space,          used:     65 KB, available:     61 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:    113 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1434591 KB, committed:      0 KB
[26109] All spaces,         used:  20784 KB, available:  25531 KB, committed:  62941 KB
[26109] Total time spent in GC  : 21 ms
[26109]       76 ms: Scavenge 32.0 (62.0) -> 19.5 (62.0) MB, 1 ms [allocation failure].
[26109] Memory allocator,   used:  63504 KB, available: 1435632 KB
[26109] New space,          used:   3053 KB, available:  13330 KB, committed:  32768 KB
[26109] Old pointers,       used:   6943 KB, available:   1391 KB, committed:   8574 KB
[26109] Old data space,     used:   9479 KB, available:  10866 KB, committed:  20347 KB
[26109] Code space,         used:    373 KB, available:    622 KB, committed:    996 KB
[26109] Map space,          used:     65 KB, available:     61 KB, committed:    128 KB
[26109] Cell space,         used:     14 KB, available:    113 KB, committed:    128 KB
[26109] Large object space, used:      0 KB, available: 1434591 KB, committed:      0 KB
[26109] All spaces,         used:  19930 KB, available:  26385 KB, committed:  62941 KB
[26109] Total time spent in GC  : 22 ms



Why memory consumption is so HUGE? Why is so different?
Reply all
Reply to author
Forward
0 new messages