About the ScyllaDB's memory use

[qleelulu@gmail.com]

We are doing some test with scylladb, insert some data into scylladb, and found it use so much memory than the data size.

The data size is 9GB, but scylladb use 110G memory.

Installation details
Scylla version (or git commit hash): 2.0.1-0.20171115.7b19167
Cluster size: 1
OS (RHEL/CentOS/Ubuntu/AWS AMI): CentOS 7.2

Hardware details (for performance issues)
Platform (physical/VM/cloud instance type/docker): physical
Hardware: sockets= cores=24 hyperthreading= memory=256G
Disks: (SSD/HDD, count) SSD

cqlsh:test> DESC bd_dmp2 ;

CREATE TABLE test.bd_dmp2 (

    uid text PRIMARY KEY,

    tags map<text, tinyint>

) WITH bloom_filter_fp_chance = 0.01

    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}

    AND comment = ''

    AND compaction = {'class': 'SizeTieredCompactionStrategy'}

    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}

    AND crc_check_chance = 1.0

    AND dclocal_read_repair_chance = 0.1

    AND default_time_to_live = 0

    AND gc_grace_seconds = 864000

    AND max_index_interval = 2048

    AND memtable_flush_period_in_ms = 0

    AND min_index_interval = 128

    AND read_repair_chance = 0.0

    AND speculative_retry = '99.0PERCENTILE';

cqlsh:test> SELECT * from test.bd_dmp2 limit 5;

 uid                          | tags

------------------------------+------------------------------------------

 10024:did:{12cb5487dcad8220} | {'375': 1, '376': 1, '377': 1, '378': 1}

 10024:did:{268cfa88d06fcbc7} | {'191': 1, '202': 1, '377': 1, '378': 1}

 10024:did:{cbde51375883ce35} | {'375': 1, '376': 1, '377': 1, '378': 1}

 10024:did:{cb331845382366ba} | {'375': 1, '376': 1, '377': 1, '378': 1}

 10024:did:{ddec05decd7153ac} | {'375': 1, '376': 1, '377': 1, '378': 1}

(5 rows)

cqlsh:test>

[sunteng@ac73:~]$ nodetool info

ID                     : a59fc18e-d19d-42e1-a375-e6bfc8a4b6e5

Gossip active          : true

Thrift active          : true

Native Transport active: true

Load                   : 8.84 GB

Generation No          : 1511164939

Uptime (seconds)       : 326979

Heap Memory (MB)       : 89.60 / 247.50

Off Heap Memory (MB)   : 23817.88

Data Center            : datacenter1

Rack                   : rack1

Exceptions             : 0

Key Cache              : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests, 0.000 recent hit rate, 0 save period in seconds

Row Cache              : entries 70989426, size 67.7 MB, capacity 1.95 TB, 210357097 hits, 210634684 requests, 0.999 recent hit rate, 0 save period in seconds

Counter Cache          : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests, 0.000 recent hit rate, 0 save period in seconds

Token                  : (invoke with -T/--tokens to see all 256 tokens)

----------------

Keyspace: test

Read Count: 5

Read Latency: 0.0 ms.

Write Count: 96620062

Write Latency: 1.3193626392001281E-5 ms.

Pending Flushes: 0

Table: bd_dmp2

SSTable count: 53

SSTables in each level: [44/4]

Space used (live): 9487024432

Space used (total): 9487024432

Space used by snapshots (total): 0

Off heap memory used (total): 24964362965

SSTable Compression Ratio: 0.362089

Number of keys (estimate): 70988730

Memtable cell count: 25631303

Memtable data size: 19487720297

Memtable off heap memory used: 24891359232

Memtable switch count: 53

Local read count: 0

Local read latency: NaN ms

Local write count: 96620046

Local write latency: 0.013 ms

Pending flushes: 0

Bloom filter false positives: 0

Bloom filter false ratio: 0.00000

Bloom filter space used: 44317024

Bloom filter off heap memory used: 46137608

Index summary off heap memory used: 26866125

Compression metadata off heap memory used: 0

Compacted partition minimum bytes: 104

Compacted partition maximum bytes: 17084

Compacted partition mean bytes: 259

Average live cells per slice (last five minutes): 0.0

Maximum live cells per slice (last five minutes): 0

Average tombstones per slice (last five minutes): 0.0

Maximum tombstones per slice (last five minutes): 0

i want to know why data size is 9GB, Off heap memory used (total) 23G, but scylladb use 110G memory.

[bdenes@scylladb.com]

On Tue, 2017-11-28 at 20:29 -0800, qlee...@gmail.com wrote:

We are doing some test with scylladb, insert some data into scylladb, and found it use so much memory than the data size.

The data size is 9GB, but scylladb use 110G memory.

ScyllaDB uses (almost) all of the hosts' available memory for caching purposes. While it's recommended that you let it do this if you wish to limit it's memory usage start scylla with e.g. --memory=20G. Alternatively you can specify the amount of memory ScyllaDB should leave to the OS with --reserve-memory.

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
To post to this group, send email to scyllad...@googlegroups.com.
Visit this group at https://groups.google.com/group/scylladb-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/e4e9d53d-8236-4afb-ab03-606ad9a332ea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.