vm.overcommit_memory=1 vs. LRU. Fight!

[Stuart Reynolds]

I'd like to configure redis as an LRU cache. 

Looking at:

   https://redis.io/topics/lru-cache

recommends setting a specific amount max-memory. However, really I'd prefer to to use all the available memory and apply the eviction policy when this fails (since the total host memory is a volatile thing in VM worlds and I don't want to configure redis each time I reconfigure host hardware).

However, this seems to be in conflict with vm.overcommit_memory=1.

which is recommended here: redis https://redis.io/topics/admin 

Why is vm.overcommit_memory=1 recommended? It doesn't seem safe and would seem to prevent applying the eviction policy when out of system memory.

- Stuart

[Stuart Reynolds]

I guess -- more to the point, is there a way to configure maxmemory
to, say 90% of the available system memory?

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.

[Chris Stanaway]

vm.overcommit_memory=1 is recommended per https://redis.io/topics/faq#background-saving-fails-with-a-fork-error-under-linux-even-if-i-have-a-lot-of-free-ram.

Also, I don't think you can use Redis the way you described.  Specifically, you said you want to use it as a LRU cache (presumably by setting maxmemory-policy to either volatile-lru or allkeys-lru), but not set a maxmemory.  You stated:

Looking at:
   https://redis.io/topics/lru-cache
recommends setting a specific amount max-memory.

However, there is no such "recommendation".  It is a requirement.

Setting maxmemory to zero results into no memory limits. This is the default behavior for 64 bit systems, while 32 bit systems use an implicit memory limit of 3GB.

 

When the specified amount of memory is reached, it is possible to select among different behaviors, called policies. Redis can just return errors for commands that could result in more memory being used, or it can evict some old data in order to return back to the specified limit every time new data is added.

The exact behavior Redis follows when the maxmemory limit is reached is configured using the maxmemory-policyconfiguration directive.

If there is no specified amount of memory (maxmemory), then it is not reached.  If it is not reached, then none of the different behaviors (policies) are applied.

How the eviction process works
It is important to understand that the eviction process works like this:

• A client runs a new command, resulting in more data added.
  
• Redis checks the memory usage, and if it is greater than the maxmemory limit , it evicts keys according to the policy.
  

Also, per the extensive documentation in redis.conf:

# Set a memory usage limit to the specified amount of bytes.

# When the memory limit is reached Redis will try to remove keys

# according to the eviction policy selected (see maxmemory-policy).

[...snip...]

# This option is usually useful when using Redis as an LRU or LFU cache

[...snip...]

# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory

# is reached.

If you don't set a maxmemory, then the maxmemory-policy never kicks in and Redis will continue to consume all available memory and you risk either Redis will crash for out of memory or the Linux kernel OOM killer will kill the Redis process.

On Wed, May 30, 2018 at 1:36 PM, Stuart Reynolds <stuart....@gmail.com> wrote:

I guess -- more to the point, is there a way to configure maxmemory
to, say 90% of the available system memory?

On Wed, May 30, 2018 at 11:26 AM, Stuart Reynolds
<stuart....@gmail.com> wrote:
> I'd like to configure redis as an LRU cache.
>
> Looking at:

>    https://urldefense.proofpoint.com/v2/url?u=https-3A__redis.io_topics_lru-2Dcache&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=iFscOJjSPU6NRC7pQfnidhe5ud8gAKKPOvrNSkeXhXA&e=

> recommends setting a specific amount max-memory. However, really I'd prefer
> to to use all the available memory and apply the eviction policy when this
> fails (since the total host memory is a volatile thing in VM worlds and I
> don't want to configure redis each time I reconfigure host hardware).
>
> However, this seems to be in conflict with vm.overcommit_memory=1.

> which is recommended here: redis https://urldefense.proofpoint.com/v2/url?u=https-3A__redis.io_topics_admin&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=IUvtw0C11c2hGaj4wqhAgrST8OhF6TbhdcRrXZgu7mk&e=

>
> Why is vm.overcommit_memory=1 recommended? It doesn't seem safe and would
> seem to prevent applying the eviction policy when out of system memory.
>
> - Stuart
>
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to redis-db+unsubscribe@googlegroups.com.

> To post to this group, send email to redi...@googlegroups.com.

> Visit this group at https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_group_redis-2Ddb&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=1gsZ83pwngVZBa0FE5oVPWb97BphUf84joRSRXY94To&e=.
> For more options, visit https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_optout&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=rVDMKpJsmOkMusBcM9kGww-58GQx26v0wtJfPBtLPIE&e=.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.

Visit this group at https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_group_redis-2Ddb&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=1gsZ83pwngVZBa0FE5oVPWb97BphUf84joRSRXY94To&e=.
For more options, visit https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_optout&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=rVDMKpJsmOkMusBcM9kGww-58GQx26v0wtJfPBtLPIE&e=.

[Stuart Reynolds]

Thanks.

Hmm... hard coding an amount of memory isn't very devops friendly --
it needs to be reset when the amount of physical RAM for the VM goes
up (or worse) or goes down.

It seems like setting:
maxmemory = 90percent
might be a more friendly default whenever maxmemory-policy is defined.

The config's defaults are to simply run out of system memory. For me
(I think) this manifested with redis rejecting new sockets, and there
was no helpful messages in the log (at least, applying a maxmemory
limit fixed this)
:-/

>> > email to redis-db+u...@googlegroups.com.

>> > To post to this group, send email to redi...@googlegroups.com.
>> > Visit this group at
>> > https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_group_redis-2Ddb&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=1gsZ83pwngVZBa0FE5oVPWb97BphUf84joRSRXY94To&e=.
>> > For more options, visit
>> > https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_optout&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=rVDMKpJsmOkMusBcM9kGww-58GQx26v0wtJfPBtLPIE&e=.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Redis DB" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to redis-db+u...@googlegroups.com.

>> To post to this group, send email to redi...@googlegroups.com.
>> Visit this group at
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_group_redis-2Ddb&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=1gsZ83pwngVZBa0FE5oVPWb97BphUf84joRSRXY94To&e=.
>> For more options, visit
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_optout&d=DwIBaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=N1Msa0ZexCqrdtWLgqlbMZlNrFKF4YmyP5otZCNJ7mmERCt9HPx5t646YC5_jvy8&m=xmLYpLHIJYHtVQ-zOKEH5dZpAHcnHzFj86ArJLcsVD0&s=rVDMKpJsmOkMusBcM9kGww-58GQx26v0wtJfPBtLPIE&e=.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to redis-db+u...@googlegroups.com.

> To post to this group, send email to redi...@googlegroups.com.

> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.

[hva...@gmail.com]

I see two options:

1. You have a configuration management system that installs Redis and creates the redis.conf config file.  Have your config management system measure the machine memory and set maxmemory according to your desires.  When you change machine types, re-deploy Redis.  (good config mgmt systems won't actually re-install an app that's already present, only update the config file and restart the app)
2. Change the start-up script(s) to measure the memory, set maxmemory in the config file, and start Redis.  As above, the start-up script(s) can be whatever you want, since they are created by your config management system as it installs Redis.  Every time Redis is (re-)started, its maxmemory is adjusted to the desired amount for the machine it's on.
   

IMO 90% is not a good threshold.  The OS uses memory for the disk read/write cache, and starving the kernel of this cache can slow the machine down.  If your Redis config saves data to disk, it also makes use of the disk cache, and a too-small cache will slow down Redis's writes.  My advice is leave 80% free for the kernel to use as cache.  If your Redis is replicating to a slave as well as saving snapshots to disk, then you will want to leave more free memory.  This post from a few years ago does a good job explaining why.

[hva...@gmail.com]

I meant to say I recommend configuring Redis to use 80% of the memory rather than 90%, leaving 20% free for the kernel to use for the disk cache (and various small OS processes).