Redis Key memory utilization

[basdeve...@gmail.com]

Running a Redis v 3.0.7 , allocated memory is 10 GB to redis. Want to know the maximum key redis will support in 10 GB of memory.

[CharSyam]

it depends on value size also

2017년 3월 31일 금요일, <basdeve...@gmail.com>님이 작성한 메시지:

Running a Redis v 3.0.7 , allocated memory is 10 GB to redis. Want to know the maximum key redis will support in 10 GB of memory.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[hva...@gmail.com]

Your question cannot be answered by us.  The memory consumption of a key depends on the size of the key's name (string) and the size of the key's value.  Names and values with fewer bytes consume less memory, so more of them can be fit into 10GB.  Names and values with more bytes consume more memory, so fewer of them can be fit into 10GB.

There is a famous description of very efficient key/value use from several years ago by Instagram, which shows one way of making keys/values very efficient.  The blog entry is here.  Note that approach is useful only for the specific type of data Instagram was using, not all types of data.  I mention it only as an example of what's possible with some kinds of data.

The number of keys that you can fit into 10GB of memory depends on the key names you choose, the sizes of the values, and any optimizations you can apply (like Instagram did).  Since we don't know your keys and values, we can't give any useful estimate of how much of your data will fit into 10GB

[charlie kwon]

http://redisgate.com/redis/configuration/server_memory.php

--

[basdeve...@gmail.com]

Ok...Very useful information provided Thanks for the same.

Well my key is like this

ATRACETESTTEST_150112_orgid_000109_deviceid_062071_devicetype_D_attendancetype_O_             biodeviceid_ZXPOPE1420I03...@1.7.0.2_authresponse_0.58_authstatus_Y_errcode_O_attdate_2017-03-17_atttime_18:02:47_creationdate_2017-03-17 18:02:47_ipaddr_11.111.1.11_deviceip_E:111.111.11.1:_gpslat_11.1111_gpslong_11.1111_networktype_E_145

Please advice on the size it would take.

[hva...@gmail.com]

I count an 81 character key and 261 character value in your example.  As long as these are encoded in ASCII, they consume one byte per character.  I'll leave performing the arithmetic about the combined consumption and how many will fit into 10G to you.  Redis has some overhead per key, but it's pretty small.

One thing I'll note about your keys and values - they're pretty obviously a set of full-word labels and their values concatenated together with "_".  E.g., devicetype=D attendancetype=O become "devicetype_D_attendancetype_O" (29 characters).  If you reduce the labels down to a smaller string, you can reduce the size a good amount while still keeping them human readable:  "devtype_D_atttype_O" (19 characters).  If you don't care so much about human readability in Redis, you can likely convert each label to a 1-character or 2-character code and get even more dramatic reductions in size.

On Monday, April 3, 2017 at 4:02:23 AM UTC-7, basdeve...@gmail.com wrote:

Ok...Very useful information provided Thanks for the same.

Well my key is like this

ATRACETESTTEST_150112_orgid_000109_deviceid_062071_devicetype_D_attendancetype_O_             biodeviceid_ZXPOPE1420I035197_biodevicem...@1.7.0.2_authresponse_0.58_authstatus_Y_errcode_O_attdate_2017-03-17_atttime_18:02:47_creationdate_2017-03-17 18:02:47_ipaddr_11.111.1.11_deviceip_E:111.111.11.1:_gpslat_11.1111_gpslong_11.1111_networktype_E_145

[Itamar Haber]

The per-key overhead is about 68 bytes in 64-bit architectures IIRC - that's not huge, but also definitely not "pretty small" when considering datasets with millions of keys :) 

--