Recommendation on locking redis pages into RAM

[Birdy]

Hi Redis community,

In our Redis setup(1 Master- 1 slave/4GB max memory), Redis memory pages were swapped out by OS and when it happens Redis stops responding.

The other application which is eating up memory(RAM) on same system can work with page faults.

So I am planning to lock redis pages by using mlockall(MCL_FUTURE). (changing redis code)

1. What is the recommendation of Redis community on using mlockall?

2. Is it possible to use mlockall with redis because its documentation suggest that reserve the memory that you want to lock. So i guess we need to allocate all redis memory initially before calling mlockall.

3. Will it somehow impact the replication functionality of redis? (When main process forks child for SYNC)

4. What will happen when system's RAM is utilized fully and then redis forks for replication. Will mlockall fail because it can not reserve required memory?

5. Is there any other way to prevent redis pages from swapping out?

Thanks,

Anurag Berdia

[Josiah Carlson]

1. I don't speak for the community.

2. I'm going to answer anyway.

On Wed, Jul 31, 2013 at 10:50 PM, Birdy <anurag...@gmail.com> wrote:

Hi Redis community,

In our Redis setup(1 Master- 1 slave/4GB max memory), Redis memory pages were swapped out by OS and when it happens Redis stops responding.

The other application which is eating up memory(RAM) on same system can work with page faults.

So I am planning to lock redis pages by using mlockall(MCL_FUTURE). (changing redis code)

1. What is the recommendation of Redis community on using mlockall?

A purist would say that your problem is that you don't have enough memory, and your solution to the problem is temporary.

The pragmatist in me says, "do it, but be aware that you're just delaying your inevitable Redis/app partitioning to different servers - which you should have done in the first place".

 

2. Is it possible to use mlockall with redis because its documentation suggest that reserve the memory that you want to lock. So i guess we need to allocate all redis memory initially before calling mlockall.

No. At any point in time you can call mlockall(MCL_CURRENT|MCL_FUTURE) and it will lock all memory currently allocated, along with all memory to be allocated in the future.

3. Will it somehow impact the replication functionality of redis? (When main process forks child for SYNC)

Maybe. The main process will be fine. Hard to say whether the memory lock will apply to any non-shared memory in the child. You should test this.

4. What will happen when system's RAM is utilized fully and then redis forks for replication. Will mlockall fail because it can not reserve required memory?

I suspect that the shared memory will stay shared. If you've got overcommit set to 1 (as listed in various docs), you may be okay. Again, you should test this.

5. Is there any other way to prevent redis pages from swapping out?

Not that I personally know, but then again, I haven't needed this sort of behavior, so I haven't spent any time researching this.

 - Josiah

Thanks,

Anurag Berdia

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Birdy]

Hi Josiah,

Thanks for the quick reply.

Regarding your response:

A purist would say that your problem is that you don't have enough memory, and your solution to the problem is temporary.

Anurag> My second application that is running on the same server works like it takes all the memory on system. So even if I increase the RAM on server from 16 GB to 64 GB there will be a time when the second application will take all 64 GB of memory and then again I will face same issue if I do not lock redis pages.

FYI I am anyways updating my servers RAM to 64 GB but want to make sure that redis does not face the same issue again.

No. At any point in time you can call mlockall(MCL_CURRENT|MCL_FUTURE) and it will lock all memory currently allocated, along with all memory to be allocated in the future.

Anurag> AFAIK redis allocates memory when new key-value pairs arrive. So when redis calls malloc for new memory, will it fail (because currently the system does not have any memory) OR OS will swap out second application's pages from memory to accommodate the malloc of redis?

Also redis uses allocators, so once redis reaches to its max memory it would have reserved and locked the 4 GB. Is it correct?

Maybe. The main process will be fine. Hard to say whether the memory lock will apply to any non-shared memory in the child. You should test this.

Anurag> I am planning to place mallocall when main process forks and secondly when new process is forked for replication. So I guess the other redis process for replication will also reserve required memory.

Thanks again for prompt response.

Regards,

Anurag Berdia

[Javier Guerra Giraldez]

On Thu, Aug 1, 2013 at 1:48 AM, Birdy <anurag...@gmail.com> wrote:
> My second application that is running on the same server works like it takes
> all the memory on system. So even if I increase the RAM on server from 16 GB
> to 64 GB there will be a time when the second application will take all 64
> GB of memory and then again I will face same issue if I do not lock redis
> pages.

that's a very good reason to isolate such "bad neighbors" applications
on its own machine. or at least, use an LXC container to limit the
amount of RAM it can swallow.

--
Javier

[Josiah Carlson]

On Wed, Jul 31, 2013 at 11:48 PM, Birdy <anurag...@gmail.com> wrote:

Hi Josiah,

Thanks for the quick reply.

Regarding your response:

A purist would say that your problem is that you don't have enough memory, and your solution to the problem is temporary.

Anurag> My second application that is running on the same server works like it takes all the memory on system. So even if I increase the RAM on server from 16 GB to 64 GB there will be a time when the second application will take all 64 GB of memory and then again I will face same issue if I do not lock redis pages.

FYI I am anyways updating my servers RAM to 64 GB but want to make sure that redis does not face the same issue again.

The only reason you are facing this issue is because you didn't put Redis on its own machine - which is always my advice.

 

No. At any point in time you can call mlockall(MCL_CURRENT|MCL_FUTURE) and it will lock all memory currently allocated, along with all memory to be allocated in the future.

Anurag> AFAIK redis allocates memory when new key-value pairs arrive. So when redis calls malloc for new memory, will it fail (because currently the system does not have any memory) OR OS will swap out second application's pages from memory to accommodate the malloc of redis?

Also redis uses allocators, so once redis reaches to its max memory it would have reserved and locked the 4 GB. Is it correct?

I would wager that the OS should swap the other process out. If you're locking Redis into RAM, that's a signal to the OS that Redis is more important than the other process. But as an example, if your platform only has 4 gigs of RAM, and Redis is using 3.5 gigs of that, a fork + malloc in the child process would likely return "out of memory".

 - Josiah