Re: high per key overhead in redis

[Salvatore Sanfilippo]

On Sun, Oct 7, 2012 at 10:44 AM, mefa <mefa...@gmail.com> wrote:
> My application data combined average kv size is only 70 Bytes.
> This leaves 90 Bytes overhead.
> Seams extremely high.
> Am I missing something?

Try to query your .txt file to see how fast it is ;)

Jokes apart, it is the price to pay for the features and lookup speed
Redis offers, we did a lot of optimizations in the past to reduce the
overhead but there going more along this like will give use more
complex code, and more space-time tradeoff so we are going to use more
CPU and provide less queries per second. I don't want to say that we
are at the perfect overhead value but it is pretty reasonable already
and there are no chances of enhancing this *dramatically*.

However what you do try is to organize your keys in 256 elements
hashes or alike, this will go near to 70 bytes per key, but because
you are paying the tradeoff of having less info per key, not aggregate
types but just strings as hash values, and a lot. Check the memory
optimizations section in the redis.io site for more info.

Cheers,
Salvatore

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org

Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
— David Gelernter

[mefa]

Fair enough :)