stop-writes-on-bgsave-error behavior

635 views
Skip to first unread message

Daniel MacDougall

unread,
Jul 29, 2013, 10:47:42 PM7/29/13
to redi...@googlegroups.com
Hi, I'm hoping someone can help me understand the behavior of "stop-writes-on-bgsave-error yes" when the background saving child is unable to fork. From the documentation for that setting I would expect that if the background save fails, redis will stop accepting writes to in-memory data. What I'm seeing in practice, and the problem I'm having, is this:

1. The redis db grows to a point where it is unable to save in the background. The following error is repeated in the logs:

[18313] 29 Jul 06:28:27.907 * 1 changes in 900 seconds. Saving...
[18313] 29 Jul 06:28:27.907 # Can't save in background: fork: Cannot allocate memory

2. The in-memory db of redis *continues to grow*, not writing checkpoints, and eventually completely filling the RAM on the box.

3. At this point redis crashes or we restart it, and it reloads its db from the latest version of dump.rdb that it was able to save, which may be hours or days old at this point.

I'm familiar with enabling vm.overcommit_memory to avoid the forking error and using AOF-mode for more durable persistence, but I'm curious how this situation can arise, where redis continues to grow even while unable to write backups, given that we have stop-writes-on-bgsave-error enabled. Am I misunderstanding this configuration?

We're using redis version 2.6.14.

Thanks,
Daniel
Reply all
Reply to author
Forward
0 new messages