Re: help me understand this memory usage

[Josiah Carlson]

On Mon, Oct 1, 2012 at 7:56 AM, Xavier Noria <f...@hashref.com> wrote:
> I am running some tests to estimate memory consumption of different data
> sets.
>
> In each run we do a flushall and insert 10K keys with a list of the same
> size, with the same repeated element. Sizes are 250, 500, 750, and 1000 in
> turn.
>
> The repeated element is "68719476735" in some runs, and its representation
> in base 36 "vkhsvlr" in some others.
>
> The surprising thing is that the hypothetical optimization of working in
> base 36 becomes only apparent for sizes 750 and 1000:
>
> http://pastie.org/4891544
>
> I wondered whether that had to do with list-max-ziplist-entries, which has a
> default of 512, but I run the script with list-max-ziplist-entries set to
> 1024 and got the same numbers.
>
> Can anybody explain those figures?

It has everything to do with your max ziplist entries. Below that
number, based on your configuration, Redis will store an encoded
version of your data, whose encoding will depend on the data you are
storing. In your case, because your values can be seen as a decimal
number smaller than a signed long long integer, it will get encoded in
a particular way, which is almost as short as your base-36
representation.

In order for ziplist entry count to take effect, you may need to restart Redis.

Regards,
- Josiah

[Xavier Noria]

El dilluns 1 d’octubre de 2012 18:04:03 UTC+2, Josiah Carlson va escriure:

It has everything to do with your max ziplist entries. Below that
number, based on your configuration, Redis will store an encoded
version of your data, whose encoding will depend on the data you are
storing. In your case, because your values can be seen as a decimal
number smaller than a signed long long integer, it will get encoded in
a particular way, which is almost as short as your base-36
representation.

In order for ziplist entry count to take effect, you may need to restart Redis.

Ahhh, I see what happened.

I am working with Redis 2.4.0 compiled by hand under ~/redis. I thought redis-server would pick redis/redis.conf automatically, but I see I need to pass the file to redis-server explicitly.

With the 1024 configured the figures do match.

Thanks!