Can't rebalance

46 views
Skip to first unread message

Lasse Schou

unread,
Mar 23, 2014, 5:07:39 PM3/23/14
to couc...@googlegroups.com
Hi,

I have two Couchbase clusters that run pretty smoothly, however whenever I try to rebalance one of the cluster if always fails with a "wait_backfill_complete, infinity" message. I know that I have had issues with memory fragmentation due to the fact that I write small documents that are later changed to bigger ones, so I've tried installing the latest tcmalloc as suggested by Aliaksey. I've tried restarting all the nodes, while stopping all incoming data, but the result is always the same. When reaching the bucket "html" it constantly fails after a while.

I hope someone can help shed some light on the cause of these issues and let me know how to complete the rebalance. You can find the cbcollect_info zip file links below. 

Thanks,
Lasse Schou

Aliaksey Kandratsenka

unread,
Mar 24, 2014, 6:13:02 PM3/24/14
to couc...@googlegroups.com
On Sun, Mar 23, 2014 at 2:07 PM, Lasse Schou <lasse...@gmail.com> wrote:
Hi,

I have two Couchbase clusters that run pretty smoothly, however whenever I try to rebalance one of the cluster if always fails with a "wait_backfill_complete, infinity" message. I know that I have had issues with memory fragmentation due to the fact that I write small documents that are later changed to bigger ones, so I've tried installing the latest tcmalloc as suggested by Aliaksey. I've tried restarting all the nodes, while stopping all incoming data, but the result is always the same. When reaching the bucket "html" it constantly fails after a while.

I hope someone can help shed some light on the cause of these issues and let me know how to complete the rebalance. You can find the cbcollect_info zip file links below. 

Here's what I see so far:

* you successfully built and installed gperftools 2.1. I'm seeing it is used (lsof + memcached memory map confirms that)

*  I do see significant swap usage

* I do see some very long IO ops in stats. And erlang timeouts. Both point to swap storm

* I still see very bad tcmalloc stats (see below) on node .47. And that's part I struggle to understand. tcmalloc 2.1 is supposed to prevent this from happening. Particularly, the fix for https://code.google.com/p/gperftools/issues/detail?id=368 is supposed to at least return all those unused pages back to OS when such severe memory fragmentation occurs. But this is not what happens in your case.

* Interestingly I only see bad malloc stats from node .47 and not from other nodes. But other nodes apparently don't deal with very large items so extensively.


Can you send me malloc stats from node .47 during rebalance ? That will be output of # /opt/couchbase/bin/cbstats 127.0.0.1:11210 raw allocator -b mouseflow_html

You may want to restart service on node .47 before doing it.
 

Here's malloc stats from box .47

 detailed: ------------------------------------------------
MALLOC:     5751436528 ( 5485.0 MiB) Bytes in use by application
MALLOC: +  26852524032 (25608.6 MiB) Bytes in page heap freelist
MALLOC: +    396512360 (  378.1 MiB) Bytes in central cache freelist
MALLOC: +     13468944 (   12.8 MiB) Bytes in transfer cache freelist
MALLOC: +     39827864 (   38.0 MiB) Bytes in thread cache freelists
MALLOC: +     86208664 (   82.2 MiB) Bytes in malloc metadata
MALLOC:   ------------
MALLOC: =  33139978392 (31604.7 MiB) Actual memory used (physical + swap)
MALLOC: +    816209920 (  778.4 MiB) Bytes released to OS (aka unmapped)
MALLOC:   ------------
MALLOC: =  33956188312 (32383.1 MiB) Virtual address space used
MALLOC:
MALLOC:         443221              Spans in use
MALLOC:             24              Thread heaps in use
MALLOC:           8192              Tcmalloc page size
------------------------------------------------
Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).
Bytes released to the OS take up virtual address space but no physical memory.
------------------------------------------------
Total size of freelists for per-thread caches,
transfer cache, and central cache, by size class
------------------------------------------------
class   1 [        8 bytes ] :     5762 objs;   0.0 MiB;   0.0 cum MiB
class   2 [       16 bytes ] :    42137 objs;   0.6 MiB;   0.7 cum MiB
class   3 [       32 bytes ] :   140238 objs;   4.3 MiB;   5.0 cum MiB
class   4 [       48 bytes ] :    62521 objs;   2.9 MiB;   7.8 cum MiB
class   5 [       64 bytes ] :   166392 objs;  10.2 MiB;  18.0 cum MiB
class   6 [       80 bytes ] :    36827 objs;   2.8 MiB;  20.8 cum MiB
class   7 [       96 bytes ] :    17402 objs;   1.6 MiB;  22.4 cum MiB
class   8 [      112 bytes ] :  1290887 objs; 137.9 MiB; 160.3 cum MiB
class   9 [      128 bytes ] :    41672 objs;   5.1 MiB; 165.4 cum MiB
class  10 [      144 bytes ] :     6863 objs;   0.9 MiB; 166.3 cum MiB
class  11 [      160 bytes ] :    10272 objs;   1.6 MiB; 167.9 cum MiB
class  12 [      176 bytes ] :     3359 objs;   0.6 MiB; 168.4 cum MiB
class  13 [      192 bytes ] :     7361 objs;   1.3 MiB; 169.8 cum MiB
class  14 [      208 bytes ] :      838 objs;   0.2 MiB; 169.9 cum MiB
class  15 [      224 bytes ] :      915 objs;   0.2 MiB; 170.1 cum MiB
class  16 [      240 bytes ] :      971 objs;   0.2 MiB; 170.4 cum MiB
class  17 [      256 bytes ] :     1473 objs;   0.4 MiB; 170.7 cum MiB
class  18 [      288 bytes ] :     4063 objs;   1.1 MiB; 171.8 cum MiB
class  19 [      320 bytes ] :     3665 objs;   1.1 MiB; 173.0 cum MiB
class  20 [      352 bytes ] :    13692 objs;   4.6 MiB; 177.6 cum MiB
class  21 [      384 bytes ] :    27321 objs;  10.0 MiB; 187.6 cum MiB
class  22 [      416 bytes ] :    13316 objs;   5.3 MiB; 192.8 cum MiB
class  23 [      448 bytes ] :     8304 objs;   3.5 MiB; 196.4 cum MiB
class  24 [      480 bytes ] :     1215 objs;   0.6 MiB; 196.9 cum MiB
class  25 [      512 bytes ] :    12652 objs;   6.2 MiB; 203.1 cum MiB
class  26 [      576 bytes ] :     2334 objs;   1.3 MiB; 204.4 cum MiB
class  27 [      640 bytes ] :     7102 objs;   4.3 MiB; 208.7 cum MiB
class  28 [      704 bytes ] :     9813 objs;   6.6 MiB; 215.3 cum MiB
class  29 [      768 bytes ] :    11172 objs;   8.2 MiB; 223.5 cum MiB
class  30 [      832 bytes ] :     9456 objs;   7.5 MiB; 231.0 cum MiB
class  31 [      896 bytes ] :     9758 objs;   8.3 MiB; 239.4 cum MiB
class  32 [      960 bytes ] :     7663 objs;   7.0 MiB; 246.4 cum MiB
class  33 [     1024 bytes ] :     6315 objs;   6.2 MiB; 252.5 cum MiB
class  34 [     1152 bytes ] :     7117 objs;   7.8 MiB; 260.4 cum MiB
class  35 [     1280 bytes ] :     5746 objs;   7.0 MiB; 267.4 cum MiB
class  36 [     1408 bytes ] :     5466 objs;   7.3 MiB; 274.7 cum MiB
class  37 [     1536 bytes ] :     3782 objs;   5.5 MiB; 280.2 cum MiB
class  38 [     1792 bytes ] :     3397 objs;   5.8 MiB; 286.1 cum MiB
class  39 [     2048 bytes ] :     4526 objs;   8.8 MiB; 294.9 cum MiB
class  40 [     2304 bytes ] :     2065 objs;   4.5 MiB; 299.4 cum MiB
class  41 [     2560 bytes ] :     1880 objs;   4.6 MiB; 304.0 cum MiB
class  42 [     2816 bytes ] :     2059 objs;   5.5 MiB; 309.5 cum MiB
class  43 [     3072 bytes ] :     1400 objs;   4.1 MiB; 313.6 cum MiB
class  44 [     3328 bytes ] :     1735 objs;   5.5 MiB; 319.2 cum MiB
class  45 [     4096 bytes ] :     1510 objs;   5.9 MiB; 325.1 cum MiB
class  46 [     4608 bytes ] :     1366 objs;   6.0 MiB; 331.1 cum MiB
class  47 [     5120 bytes ] :      674 objs;   3.3 MiB; 334.3 cum MiB
class  48 [     6144 bytes ] :     1355 objs;   7.9 MiB; 342.3 cum MiB
class  49 [     6656 bytes ] :     1320 objs;   8.4 MiB; 350.7 cum MiB
class  50 [     8192 bytes ] :      434 objs;   3.4 MiB; 354.1 cum MiB
class  51 [     9216 bytes ] :     1081 objs;   9.5 MiB; 363.6 cum MiB
class  52 [    10240 bytes ] :      456 objs;   4.5 MiB; 368.0 cum MiB
class  53 [    12288 bytes ] :      293 objs;   3.4 MiB; 371.4 cum MiB
class  54 [    13312 bytes ] :      276 objs;   3.5 MiB; 374.9 cum MiB
class  55 [    16384 bytes ] :       24 objs;   0.4 MiB; 375.3 cum MiB
class  56 [    20480 bytes ] :      195 objs;   3.8 MiB; 379.1 cum MiB
class  57 [    24576 bytes ] :       14 objs;   0.3 MiB; 379.5 cum MiB
class  58 [    26624 bytes ] :      113 objs;   2.9 MiB; 382.3 cum MiB
class  59 [    32768 bytes ] :       20 objs;   0.6 MiB; 383.0 cum MiB
class  60 [    40960 bytes ] :       13 objs;   0.5 MiB; 383.5 cum MiB
class  61 [    49152 bytes ] :       11 objs;   0.5 MiB; 384.0 cum MiB
class  62 [    57344 bytes ] :       14 objs;   0.8 MiB; 384.7 cum MiB
class  63 [    65536 bytes ] :       14 objs;   0.9 MiB; 385.6 cum MiB
class  64 [    73728 bytes ] :       13 objs;   0.9 MiB; 386.5 cum MiB
class  65 [    81920 bytes ] :       12 objs;   0.9 MiB; 387.5 cum MiB
class  66 [    90112 bytes ] :        9 objs;   0.8 MiB; 388.2 cum MiB
class  67 [    98304 bytes ] :       11 objs;   1.0 MiB; 389.3 cum MiB
class  68 [   106496 bytes ] :        9 objs;   0.9 MiB; 390.2 cum MiB
class  69 [   114688 bytes ] :       10 objs;   1.1 MiB; 391.3 cum MiB
class  70 [   122880 bytes ] :        8 objs;   0.9 MiB; 392.2 cum MiB
class  71 [   131072 bytes ] :        9 objs;   1.1 MiB; 393.3 cum MiB
class  72 [   139264 bytes ] :       13 objs;   1.7 MiB; 395.1 cum MiB
class  73 [   147456 bytes ] :        8 objs;   1.1 MiB; 396.2 cum MiB
class  74 [   155648 bytes ] :        6 objs;   0.9 MiB; 397.1 cum MiB
class  75 [   163840 bytes ] :        7 objs;   1.1 MiB; 398.2 cum MiB
class  76 [   172032 bytes ] :        6 objs;   1.0 MiB; 399.2 cum MiB
class  77 [   180224 bytes ] :        6 objs;   1.0 MiB; 400.2 cum MiB
class  78 [   188416 bytes ] :        4 objs;   0.7 MiB; 400.9 cum MiB
class  79 [   196608 bytes ] :        5 objs;   0.9 MiB; 401.9 cum MiB
class  80 [   204800 bytes ] :        3 objs;   0.6 MiB; 402.4 cum MiB
class  81 [   212992 bytes ] :        4 objs;   0.8 MiB; 403.3 cum MiB
class  82 [   221184 bytes ] :        4 objs;   0.8 MiB; 404.1 cum MiB
class  83 [   229376 bytes ] :        1 objs;   0.2 MiB; 404.3 cum MiB
class  84 [   237568 bytes ] :        3 objs;   0.7 MiB; 405.0 cum MiB
class  85 [   245760 bytes ] :        3 objs;   0.7 MiB; 405.7 cum MiB
class  86 [   253952 bytes ] :        1 objs;   0.2 MiB; 405.9 cum MiB
class  87 [   262144 bytes ] :        8 objs;   2.0 MiB; 407.9 cum MiB
------------------------------------------------
PageHeap: 127 sizes; 25608.6 MiB free;  778.4 MiB unmapped
------------------------------------------------
     1 pages *   3476 spans ~   27.2 MiB;   27.2 MiB cum; unmapped:   10.2 MiB;   10.2 MiB cum
     2 pages *   6949 spans ~  108.6 MiB;  135.7 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
     3 pages *   5068 spans ~  118.8 MiB;  254.5 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
     4 pages *   4637 spans ~  144.9 MiB;  399.4 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
     5 pages *   6005 spans ~  234.6 MiB;  634.0 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
     6 pages *   3997 spans ~  187.4 MiB;  821.4 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
     7 pages *   4008 spans ~  219.2 MiB; 1040.5 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
     8 pages *   3855 spans ~  240.9 MiB; 1281.5 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
     9 pages *   3162 spans ~  222.3 MiB; 1503.8 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    10 pages *   3404 spans ~  265.9 MiB; 1769.7 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    11 pages *   2614 spans ~  224.6 MiB; 1994.4 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    12 pages *   2713 spans ~  254.3 MiB; 2248.7 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    13 pages *   2244 spans ~  227.9 MiB; 2476.6 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    14 pages *   2193 spans ~  239.9 MiB; 2716.5 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    15 pages *   2042 spans ~  239.3 MiB; 2955.8 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    16 pages *   1858 spans ~  232.2 MiB; 3188.0 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    17 pages *   1683 spans ~  223.5 MiB; 3411.6 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    18 pages *   1740 spans ~  244.7 MiB; 3656.2 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    19 pages *   1546 spans ~  229.5 MiB; 3885.7 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    20 pages *   1488 spans ~  232.5 MiB; 4118.2 MiB cum; unmapped:    0.0 MiB;   10.2 MiB cum
    21 pages *   1290 spans ~  211.6 MiB; 4329.9 MiB cum; unmapped:    0.2 MiB;   10.3 MiB cum
    22 pages *   1300 spans ~  223.4 MiB; 4553.3 MiB cum; unmapped:    0.2 MiB;   10.5 MiB cum
    23 pages *   1216 spans ~  218.5 MiB; 4771.8 MiB cum; unmapped:    0.2 MiB;   10.7 MiB cum
    24 pages *   1112 spans ~  208.5 MiB; 4980.3 MiB cum; unmapped:    0.2 MiB;   10.9 MiB cum
    25 pages *    954 spans ~  186.3 MiB; 5166.6 MiB cum; unmapped:    0.2 MiB;   11.1 MiB cum
    26 pages *   1076 spans ~  218.6 MiB; 5385.2 MiB cum; unmapped:    0.2 MiB;   11.3 MiB cum
    27 pages *   1027 spans ~  216.6 MiB; 5601.8 MiB cum; unmapped:    0.2 MiB;   11.5 MiB cum
    28 pages *    898 spans ~  196.4 MiB; 5798.3 MiB cum; unmapped:    0.2 MiB;   11.7 MiB cum
    29 pages *    800 spans ~  181.2 MiB; 5979.5 MiB cum; unmapped:    0.2 MiB;   11.9 MiB cum
    30 pages *    869 spans ~  203.7 MiB; 6183.2 MiB cum; unmapped:    0.2 MiB;   12.1 MiB cum
    31 pages *    737 spans ~  178.5 MiB; 6361.7 MiB cum; unmapped:    0.2 MiB;   12.4 MiB cum
    32 pages *    690 spans ~  172.5 MiB; 6534.2 MiB cum; unmapped:    0.2 MiB;   12.6 MiB cum
    33 pages *    614 spans ~  158.3 MiB; 6692.5 MiB cum; unmapped:    0.3 MiB;   12.9 MiB cum
    34 pages *    628 spans ~  166.8 MiB; 6859.3 MiB cum; unmapped:    0.3 MiB;   13.2 MiB cum
    35 pages *    566 spans ~  154.8 MiB; 7014.1 MiB cum; unmapped:    0.3 MiB;   13.4 MiB cum
    36 pages *    551 spans ~  155.0 MiB; 7169.0 MiB cum; unmapped:    0.3 MiB;   13.7 MiB cum
    37 pages *    526 spans ~  152.0 MiB; 7321.1 MiB cum; unmapped:    0.3 MiB;   14.0 MiB cum
    38 pages *    499 spans ~  148.1 MiB; 7469.2 MiB cum; unmapped:    0.3 MiB;   14.3 MiB cum
    39 pages *    474 spans ~  144.4 MiB; 7613.6 MiB cum; unmapped:    0.3 MiB;   14.6 MiB cum
    40 pages *    428 spans ~  133.8 MiB; 7747.4 MiB cum; unmapped:    0.3 MiB;   14.9 MiB cum
    41 pages *    410 spans ~  131.3 MiB; 7878.7 MiB cum; unmapped:    0.3 MiB;   15.2 MiB cum
    42 pages *    409 spans ~  134.2 MiB; 8012.9 MiB cum; unmapped:    0.3 MiB;   15.6 MiB cum
    43 pages *    407 spans ~  136.7 MiB; 8149.6 MiB cum; unmapped:    0.3 MiB;   15.9 MiB cum
    44 pages *    382 spans ~  131.3 MiB; 8281.0 MiB cum; unmapped:    0.3 MiB;   16.2 MiB cum
    45 pages *    371 spans ~  130.4 MiB; 8411.4 MiB cum; unmapped:    0.4 MiB;   16.6 MiB cum
    46 pages *    332 spans ~  119.3 MiB; 8530.7 MiB cum; unmapped:    0.4 MiB;   17.0 MiB cum
    47 pages *    315 spans ~  115.7 MiB; 8646.4 MiB cum; unmapped:    0.4 MiB;   17.3 MiB cum
    48 pages *    322 spans ~  120.8 MiB; 8767.1 MiB cum; unmapped:    0.4 MiB;   17.7 MiB cum
    49 pages *    297 spans ~  113.7 MiB; 8880.8 MiB cum; unmapped:    0.4 MiB;   18.1 MiB cum
    50 pages *    268 spans ~  104.7 MiB; 8985.5 MiB cum; unmapped:    0.4 MiB;   18.5 MiB cum
    51 pages *    261 spans ~  104.0 MiB; 9089.5 MiB cum; unmapped:    0.4 MiB;   18.9 MiB cum
    52 pages *    292 spans ~  118.6 MiB; 9208.1 MiB cum; unmapped:    0.4 MiB;   19.3 MiB cum
    53 pages *    239 spans ~   99.0 MiB; 9307.1 MiB cum; unmapped:    0.4 MiB;   19.7 MiB cum
    54 pages *    249 spans ~  105.0 MiB; 9412.1 MiB cum; unmapped:    0.4 MiB;   20.1 MiB cum
    55 pages *    229 spans ~   98.4 MiB; 9510.5 MiB cum; unmapped:    0.4 MiB;   20.5 MiB cum
    56 pages *    208 spans ~   91.0 MiB; 9601.5 MiB cum; unmapped:    0.4 MiB;   21.0 MiB cum
    57 pages *    246 spans ~  109.5 MiB; 9711.1 MiB cum; unmapped:    0.4 MiB;   21.4 MiB cum
    58 pages *    214 spans ~   97.0 MiB; 9808.0 MiB cum; unmapped:    0.5 MiB;   21.9 MiB cum
    59 pages *    215 spans ~   99.1 MiB; 9907.1 MiB cum; unmapped:    0.5 MiB;   22.3 MiB cum
    60 pages *    183 spans ~   85.8 MiB; 9992.9 MiB cum; unmapped:    0.5 MiB;   22.8 MiB cum
    61 pages *    186 spans ~   88.6 MiB; 10081.6 MiB cum; unmapped:    0.5 MiB;   23.3 MiB cum
    62 pages *    184 spans ~   89.1 MiB; 10170.7 MiB cum; unmapped:    0.5 MiB;   23.8 MiB cum
    63 pages *    185 spans ~   91.1 MiB; 10261.7 MiB cum; unmapped:    0.5 MiB;   24.3 MiB cum
    64 pages *    181 spans ~   90.5 MiB; 10352.2 MiB cum; unmapped:    0.5 MiB;   24.8 MiB cum
    65 pages *    159 spans ~   80.7 MiB; 10433.0 MiB cum; unmapped:    0.5 MiB;   25.3 MiB cum
    66 pages *    141 spans ~   72.7 MiB; 10505.7 MiB cum; unmapped:    0.5 MiB;   25.8 MiB cum
    67 pages *    127 spans ~   66.5 MiB; 10572.2 MiB cum; unmapped:    0.5 MiB;   26.3 MiB cum
    68 pages *    131 spans ~   69.6 MiB; 10641.8 MiB cum; unmapped:    0.5 MiB;   26.8 MiB cum
    69 pages *    116 spans ~   62.5 MiB; 10704.3 MiB cum; unmapped:    0.5 MiB;   27.4 MiB cum
    70 pages *    125 spans ~   68.4 MiB; 10772.6 MiB cum; unmapped:    0.5 MiB;   27.9 MiB cum
    71 pages *    128 spans ~   71.0 MiB; 10843.6 MiB cum; unmapped:    0.6 MiB;   28.5 MiB cum
    72 pages *    109 spans ~   61.3 MiB; 10905.0 MiB cum; unmapped:    0.6 MiB;   29.0 MiB cum
    73 pages *     83 spans ~   47.3 MiB; 10952.3 MiB cum; unmapped:    0.6 MiB;   29.6 MiB cum
    74 pages *    107 spans ~   61.9 MiB; 11014.2 MiB cum; unmapped:    0.6 MiB;   30.2 MiB cum
    75 pages *     86 spans ~   50.4 MiB; 11064.5 MiB cum; unmapped:    0.6 MiB;   30.8 MiB cum
    76 pages *     88 spans ~   52.2 MiB; 11116.8 MiB cum; unmapped:    0.6 MiB;   31.4 MiB cum
    77 pages *     87 spans ~   52.3 MiB; 11169.1 MiB cum; unmapped:    0.6 MiB;   32.0 MiB cum
    78 pages *     76 spans ~   46.3 MiB; 11215.4 MiB cum; unmapped:    0.6 MiB;   32.6 MiB cum
    79 pages *     90 spans ~   55.5 MiB; 11271.0 MiB cum; unmapped:    0.0 MiB;   32.6 MiB cum
    80 pages *     83 spans ~   51.9 MiB; 11322.9 MiB cum; unmapped:    0.6 MiB;   33.2 MiB cum
    81 pages *     81 spans ~   51.3 MiB; 11374.1 MiB cum; unmapped:    1.3 MiB;   34.5 MiB cum
    82 pages *     77 spans ~   49.3 MiB; 11423.5 MiB cum; unmapped:    0.6 MiB;   35.1 MiB cum
    83 pages *     68 spans ~   44.1 MiB; 11467.5 MiB cum; unmapped:    0.6 MiB;   35.8 MiB cum
    84 pages *     69 spans ~   45.3 MiB; 11512.8 MiB cum; unmapped:    0.0 MiB;   35.8 MiB cum
    85 pages *     59 spans ~   39.2 MiB; 11552.0 MiB cum; unmapped:    0.7 MiB;   36.4 MiB cum
    86 pages *     56 spans ~   37.6 MiB; 11589.6 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    87 pages *     54 spans ~   36.7 MiB; 11626.3 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    88 pages *     65 spans ~   44.7 MiB; 11671.0 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    89 pages *     53 spans ~   36.9 MiB; 11707.9 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    90 pages *     57 spans ~   40.1 MiB; 11748.0 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    91 pages *     56 spans ~   39.8 MiB; 11787.8 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    92 pages *     48 spans ~   34.5 MiB; 11822.3 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    93 pages *     34 spans ~   24.7 MiB; 11847.0 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    94 pages *     35 spans ~   25.7 MiB; 11872.7 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    95 pages *     52 spans ~   38.6 MiB; 11911.3 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    96 pages *     45 spans ~   33.8 MiB; 11945.0 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    97 pages *     40 spans ~   30.3 MiB; 11975.3 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    98 pages *     49 spans ~   37.5 MiB; 12012.8 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
    99 pages *     42 spans ~   32.5 MiB; 12045.3 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   100 pages *     41 spans ~   32.0 MiB; 12077.4 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   101 pages *     38 spans ~   30.0 MiB; 12107.3 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   102 pages *     27 spans ~   21.5 MiB; 12128.9 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   103 pages *     39 spans ~   31.4 MiB; 12160.2 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   104 pages *     42 spans ~   34.1 MiB; 12194.4 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   105 pages *     38 spans ~   31.2 MiB; 12225.5 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   106 pages *     31 spans ~   25.7 MiB; 12251.2 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   107 pages *     26 spans ~   21.7 MiB; 12272.9 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   108 pages *     24 spans ~   20.2 MiB; 12293.2 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   109 pages *     28 spans ~   23.8 MiB; 12317.0 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   110 pages *     27 spans ~   23.2 MiB; 12340.2 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   111 pages *     26 spans ~   22.5 MiB; 12362.8 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   112 pages *     32 spans ~   28.0 MiB; 12390.8 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   113 pages *     19 spans ~   16.8 MiB; 12407.6 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   114 pages *     24 spans ~   21.4 MiB; 12428.9 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   115 pages *     26 spans ~   23.4 MiB; 12452.3 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   116 pages *     28 spans ~   25.4 MiB; 12477.7 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   117 pages *     27 spans ~   24.7 MiB; 12502.4 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   118 pages *     22 spans ~   20.3 MiB; 12522.6 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   119 pages *     25 spans ~   23.2 MiB; 12545.9 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   120 pages *     26 spans ~   24.4 MiB; 12570.2 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   121 pages *     29 spans ~   27.4 MiB; 12597.7 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   122 pages *     21 spans ~   20.0 MiB; 12617.7 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   123 pages *     20 spans ~   19.2 MiB; 12636.9 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   124 pages *     14 spans ~   13.6 MiB; 12650.5 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   125 pages *     20 spans ~   19.5 MiB; 12670.0 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   126 pages *     25 spans ~   24.6 MiB; 12694.6 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
   127 pages *     16 spans ~   15.9 MiB; 12710.5 MiB cum; unmapped:    0.0 MiB;   36.4 MiB cum
>255   large *   1563 spans ~ 13676.5 MiB; 26387.0 MiB cum; unmapped:  742.0 MiB;  778.4 MiB cum


--
You received this message because you are subscribed to the Google Groups "Couchbase" group.
To unsubscribe from this group and stop receiving emails from it, send an email to couchbase+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages