We have used redis for about 6+ month in production. There are a couple of issues that I've observed with redis-2.4.4
a) is there a possible memory leak? we've seen memory gone from 200M to 10G in 6 month, but the data size hasn't changed. We are using mostly hash data set, using jedis java library. Restarting the server seems to drop the memory back down.
b) keys command and expire command doesn't seem to work with specific keys? say i have a key a set a 10 expire a 10
however, key a is expired, a get still returns 10, though documentation says that after a key expired, it will be deleted automatically
are these bugs in 2.4.4?
thx, and btw, is redis cluster going to be released anytime soon?
On Mon, Sep 17, 2012 at 10:40 PM, zs <zsl...@gmail.com> wrote:
> We have used redis for about 6+ month in production. There are a couple of
> issues that I've observed with redis-2.4.4
There are a few known issues with Redis 2.4.4, which is why 2.4.17 exists ;)
> a) is there a possible memory leak? we've seen memory gone from 200M to 10G
> in 6 month, but the data size hasn't changed. We are using mostly hash data
> set, using jedis java library. Restarting the server seems to drop the
> memory back down.
There are a few potential causes of this, all of which are fixed in
releases up to and including Redis 2.4.17 . I would suggest you
upgrade.
> b) keys command and expire command doesn't seem to work with specific keys?
> say i have a key a
> set a 10
> expire a 10
> however, key a is expired, a get still returns 10, though documentation says
> that after a key expired, it will be deleted automatically
> are these bugs in 2.4.4?
I do not remember such a bug, but my memory may be flawed. Regardless,
you should upgrade to 2.4.17, it should fix your memory bug.
> thx, and btw, is redis cluster going to be released anytime soon?
Sadly, no. Redis 2.6 and sentinel are first, and while cluster has a
lot of work done for it, there is still enough work to be done that
Salvatore isn't even recommending people run it for testing. On the
other hand, Redis 2.6 has been in RC mode for a few months, and
sentinel has been available for several weeks.
>> thx, and btw, is redis cluster going to be released anytime soon?
> Sadly, no. Redis 2.6 and sentinel are first, and while cluster has a
> lot of work done for it, there is still enough work to be done that
> Salvatore isn't even recommending people run it for testing. On the
> other hand, Redis 2.6 has been in RC mode for a few months, and
> sentinel has been available for several weeks.
> Regards,
> - Josiah
Is Sentinel in the 2.6.0 RC7 tarball, or do I need to go to Github for it?
-- Twitter: http://twitter.com/znmeb; Computational Journalism Publishers
Workbench: http://j.mp/QCsXOr
How the Hell can the lion sleep with all those people singing "A weem
oh way!" at the top of their lungs?
On Wednesday, September 19, 2012 1:41:22 AM UTC+9, M Edward Borasky wrote:
> On Tue, Sep 18, 2012 at 6:04 AM, Josiah Carlson > <josiah....@gmail.com <javascript:>> wrote:
> [snip]
> >> thx, and btw, is redis cluster going to be released anytime soon?
> > Sadly, no. Redis 2.6 and sentinel are first, and while cluster has a > > lot of work done for it, there is still enough work to be done that > > Salvatore isn't even recommending people run it for testing. On the > > other hand, Redis 2.6 has been in RC mode for a few months, and > > sentinel has been available for several weeks.
> > Regards, > > - Josiah
> Is Sentinel in the 2.6.0 RC7 tarball, or do I need to go to Github for it? > -- > Twitter: http://twitter.com/znmeb; Computational Journalism Publishers > Workbench: http://j.mp/QCsXOr
> How the Hell can the lion sleep with all those people singing "A weem > oh way!" at the top of their lungs?
Using 2.4.17 at least solved the expiration problem.
However, I've observed another issue with 2.4.17.
I have maxmemory set to 256M. However, I've seen the log showed that redis used the memory proportional to the number of clients (each might be writing updating the same key). There are only 3 keys used for this cache, total memory for these should be around 75M. And occationally, i get ERR memory exceeding maxmemory, but not always.
So my question is: what's maxmemory? how is it observed? and how can total memory used be bigger than maxmemory?
[14749] 21 Sep 11:34:31 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:34:31 - 17 clients connected (0 slaves), 554518168 bytes in use [14749] 21 Sep 11:34:36 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:34:36 - 17 clients connected (0 slaves), 638404152 bytes in use [14749] 21 Sep 11:34:41 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:34:41 - 17 clients connected (0 slaves), 667764216 bytes in use [14749] 21 Sep 11:34:46 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:34:46 - 17 clients connected (0 slaves), 667764176 bytes in use [14749] 21 Sep 11:34:51 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:34:51 - 17 clients connected (0 slaves), 667764144 bytes in use [14749] 21 Sep 11:34:56 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:34:56 - 17 clients connected (0 slaves), 667764144 bytes in use [14749] 21 Sep 11:35:01 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:01 - 17 clients connected (0 slaves), 667764144 bytes in use [14749] 21 Sep 11:35:06 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:06 - 16 clients connected (0 slaves), 630006888 bytes in use [14749] 21 Sep 11:35:11 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:11 - 13 clients connected (0 slaves), 516735128 bytes in use [14749] 21 Sep 11:35:16 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:16 - 12 clients connected (0 slaves), 478977856 bytes in use [14749] 21 Sep 11:35:21 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:21 - 10 clients connected (0 slaves), 403463328 bytes in use [14749] 21 Sep 11:35:26 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:26 - 9 clients connected (0 slaves), 365706072 bytes in use [14749] 21 Sep 11:35:31 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:31 - 9 clients connected (0 slaves), 365706064 bytes in use [14749] 21 Sep 11:35:36 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:36 - 8 clients connected (0 slaves), 327948848 bytes in use [14749] 21 Sep 11:35:41 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:41 - 8 clients connected (0 slaves), 327948848 bytes in use [14749] 21 Sep 11:35:46 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:46 - 8 clients connected (0 slaves), 327948840 bytes in use [14749] 21 Sep 11:35:51 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:51 - 7 clients connected (0 slaves), 290191560 bytes in use [14749] 21 Sep 11:35:56 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:35:56 - 6 clients connected (0 slaves), 252434280 bytes in use [14749] 21 Sep 11:36:01 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:36:01 - 6 clients connected (0 slaves), 252434304 bytes in use [14749] 21 Sep 11:36:06 - DB 0: 2 keys (0 volatile) in 4 slots HT. [14749] 21 Sep 11:36:06 - 6 clients connected (0 slaves), 252434304 bytes in use [14749] 21 Sep 11:36:11 - DB 0: 2 keys (0 volatile) in 4 slots HT.
On Tuesday, September 18, 2012 6:04:47 AM UTC-7, Josiah Carlson wrote:
> On Mon, Sep 17, 2012 at 10:40 PM, zs <zsl...@gmail.com <javascript:>> > wrote: > > We have used redis for about 6+ month in production. There are a couple > of > > issues that I've observed with redis-2.4.4
> There are a few known issues with Redis 2.4.4, which is why 2.4.17 exists > ;)
> > a) is there a possible memory leak? we've seen memory gone from 200M to > 10G > > in 6 month, but the data size hasn't changed. We are using mostly hash > data > > set, using jedis java library. Restarting the server seems to drop the > > memory back down.
> There are a few potential causes of this, all of which are fixed in > releases up to and including Redis 2.4.17 . I would suggest you > upgrade.
> > b) keys command and expire command doesn't seem to work with specific > keys? > > say i have a key a > > set a 10 > > expire a 10
> > however, key a is expired, a get still returns 10, though documentation > says > > that after a key expired, it will be deleted automatically
> > are these bugs in 2.4.4?
> I do not remember such a bug, but my memory may be flawed. Regardless, > you should upgrade to 2.4.17, it should fix your memory bug.
> > thx, and btw, is redis cluster going to be released anytime soon?
> Sadly, no. Redis 2.6 and sentinel are first, and while cluster has a > lot of work done for it, there is still enough work to be done that > Salvatore isn't even recommending people run it for testing. On the > other hand, Redis 2.6 has been in RC mode for a few months, and > sentinel has been available for several weeks.
If your clients are using that much memory, then they likely either
have very large outgoing buffers associated with them, or had very
large outgoing buffers associated with them in the past that were not
released.
Can you check INFO and CLIENT LIST output? That may tell us if it is
related to outgoing buffers, or previous outgoing buffers.
On Fri, Sep 21, 2012 at 8:42 AM, zs <zsl...@gmail.com> wrote:
> Using 2.4.17 at least solved the expiration problem.
> However, I've observed another issue with 2.4.17.
> I have maxmemory set to 256M. However, I've seen the log showed that redis
> used the memory
> proportional to the number of clients (each might be writing updating the
> same key). There are only
> 3 keys used for this cache, total memory for these should be around 75M. And
> occationally, i get ERR memory
> exceeding maxmemory, but not always.
> So my question is:
> what's maxmemory? how is it observed? and how can total memory used be
> bigger than maxmemory?
> [14749] 21 Sep 11:34:31 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:34:31 - 17 clients connected (0 slaves), 554518168 bytes
> in use
> [14749] 21 Sep 11:34:36 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:34:36 - 17 clients connected (0 slaves), 638404152 bytes
> in use
> [14749] 21 Sep 11:34:41 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:34:41 - 17 clients connected (0 slaves), 667764216 bytes
> in use
> [14749] 21 Sep 11:34:46 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:34:46 - 17 clients connected (0 slaves), 667764176 bytes
> in use
> [14749] 21 Sep 11:34:51 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:34:51 - 17 clients connected (0 slaves), 667764144 bytes
> in use
> [14749] 21 Sep 11:34:56 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:34:56 - 17 clients connected (0 slaves), 667764144 bytes
> in use
> [14749] 21 Sep 11:35:01 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:01 - 17 clients connected (0 slaves), 667764144 bytes
> in use
> [14749] 21 Sep 11:35:06 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:06 - 16 clients connected (0 slaves), 630006888 bytes
> in use
> [14749] 21 Sep 11:35:11 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:11 - 13 clients connected (0 slaves), 516735128 bytes
> in use
> [14749] 21 Sep 11:35:16 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:16 - 12 clients connected (0 slaves), 478977856 bytes
> in use
> [14749] 21 Sep 11:35:21 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:21 - 10 clients connected (0 slaves), 403463328 bytes
> in use
> [14749] 21 Sep 11:35:26 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:26 - 9 clients connected (0 slaves), 365706072 bytes in
> use
> [14749] 21 Sep 11:35:31 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:31 - 9 clients connected (0 slaves), 365706064 bytes in
> use
> [14749] 21 Sep 11:35:36 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:36 - 8 clients connected (0 slaves), 327948848 bytes in
> use
> [14749] 21 Sep 11:35:41 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:41 - 8 clients connected (0 slaves), 327948848 bytes in
> use
> [14749] 21 Sep 11:35:46 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:46 - 8 clients connected (0 slaves), 327948840 bytes in
> use
> [14749] 21 Sep 11:35:51 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:51 - 7 clients connected (0 slaves), 290191560 bytes in
> use
> [14749] 21 Sep 11:35:56 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:35:56 - 6 clients connected (0 slaves), 252434280 bytes in
> use
> [14749] 21 Sep 11:36:01 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:36:01 - 6 clients connected (0 slaves), 252434304 bytes in
> use
> [14749] 21 Sep 11:36:06 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> [14749] 21 Sep 11:36:06 - 6 clients connected (0 slaves), 252434304 bytes in
> use
> [14749] 21 Sep 11:36:11 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> On Tuesday, September 18, 2012 6:04:47 AM UTC-7, Josiah Carlson wrote:
>> On Mon, Sep 17, 2012 at 10:40 PM, zs <zsl...@gmail.com> wrote:
>> > We have used redis for about 6+ month in production. There are a couple
>> > of
>> > issues that I've observed with redis-2.4.4
>> There are a few known issues with Redis 2.4.4, which is why 2.4.17 exists
>> ;)
>> > a) is there a possible memory leak? we've seen memory gone from 200M to
>> > 10G
>> > in 6 month, but the data size hasn't changed. We are using mostly hash
>> > data
>> > set, using jedis java library. Restarting the server seems to drop the
>> > memory back down.
>> There are a few potential causes of this, all of which are fixed in
>> releases up to and including Redis 2.4.17 . I would suggest you
>> upgrade.
>> > b) keys command and expire command doesn't seem to work with specific
>> > keys?
>> > say i have a key a
>> > set a 10
>> > expire a 10
>> > however, key a is expired, a get still returns 10, though documentation
>> > says
>> > that after a key expired, it will be deleted automatically
>> > are these bugs in 2.4.4?
>> I do not remember such a bug, but my memory may be flawed. Regardless,
>> you should upgrade to 2.4.17, it should fix your memory bug.
>> > thx, and btw, is redis cluster going to be released anytime soon?
>> Sadly, no. Redis 2.6 and sentinel are first, and while cluster has a
>> lot of work done for it, there is still enough work to be done that
>> Salvatore isn't even recommending people run it for testing. On the
>> other hand, Redis 2.6 has been in RC mode for a few months, and
>> sentinel has been available for several weeks.
> To post to this group, send email to redis-db@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
On Friday, September 21, 2012 9:05:42 AM UTC-7, Josiah Carlson wrote:
> If your clients are using that much memory, then they likely either > have very large outgoing buffers associated with them, or had very > large outgoing buffers associated with them in the past that were not > released.
> Can you check INFO and CLIENT LIST output? That may tell us if it is > related to outgoing buffers, or previous outgoing buffers.
> - Josiah
> On Fri, Sep 21, 2012 at 8:42 AM, zs <zsl...@gmail.com <javascript:>> > wrote: > > Using 2.4.17 at least solved the expiration problem.
> > However, I've observed another issue with 2.4.17.
> > I have maxmemory set to 256M. However, I've seen the log showed that > redis > > used the memory > > proportional to the number of clients (each might be writing updating > the > > same key). There are only > > 3 keys used for this cache, total memory for these should be around 75M. > And > > occationally, i get ERR memory > > exceeding maxmemory, but not always.
> > So my question is: > > what's maxmemory? how is it observed? and how can total memory used be > > bigger than maxmemory?
> > [14749] 21 Sep 11:34:31 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:34:31 - 17 clients connected (0 slaves), 554518168 > bytes > > in use > > [14749] 21 Sep 11:34:36 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:34:36 - 17 clients connected (0 slaves), 638404152 > bytes > > in use > > [14749] 21 Sep 11:34:41 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:34:41 - 17 clients connected (0 slaves), 667764216 > bytes > > in use > > [14749] 21 Sep 11:34:46 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:34:46 - 17 clients connected (0 slaves), 667764176 > bytes > > in use > > [14749] 21 Sep 11:34:51 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:34:51 - 17 clients connected (0 slaves), 667764144 > bytes > > in use > > [14749] 21 Sep 11:34:56 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:34:56 - 17 clients connected (0 slaves), 667764144 > bytes > > in use > > [14749] 21 Sep 11:35:01 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:01 - 17 clients connected (0 slaves), 667764144 > bytes > > in use > > [14749] 21 Sep 11:35:06 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:06 - 16 clients connected (0 slaves), 630006888 > bytes > > in use > > [14749] 21 Sep 11:35:11 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:11 - 13 clients connected (0 slaves), 516735128 > bytes > > in use > > [14749] 21 Sep 11:35:16 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:16 - 12 clients connected (0 slaves), 478977856 > bytes > > in use > > [14749] 21 Sep 11:35:21 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:21 - 10 clients connected (0 slaves), 403463328 > bytes > > in use > > [14749] 21 Sep 11:35:26 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:26 - 9 clients connected (0 slaves), 365706072 > bytes in > > use > > [14749] 21 Sep 11:35:31 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:31 - 9 clients connected (0 slaves), 365706064 > bytes in > > use > > [14749] 21 Sep 11:35:36 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:36 - 8 clients connected (0 slaves), 327948848 > bytes in > > use > > [14749] 21 Sep 11:35:41 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:41 - 8 clients connected (0 slaves), 327948848 > bytes in > > use > > [14749] 21 Sep 11:35:46 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:46 - 8 clients connected (0 slaves), 327948840 > bytes in > > use > > [14749] 21 Sep 11:35:51 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:51 - 7 clients connected (0 slaves), 290191560 > bytes in > > use > > [14749] 21 Sep 11:35:56 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:35:56 - 6 clients connected (0 slaves), 252434280 > bytes in > > use > > [14749] 21 Sep 11:36:01 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:36:01 - 6 clients connected (0 slaves), 252434304 > bytes in > > use > > [14749] 21 Sep 11:36:06 - DB 0: 2 keys (0 volatile) in 4 slots HT. > > [14749] 21 Sep 11:36:06 - 6 clients connected (0 slaves), 252434304 > bytes in > > use > > [14749] 21 Sep 11:36:11 - DB 0: 2 keys (0 volatile) in 4 slots HT.
> > On Tuesday, September 18, 2012 6:04:47 AM UTC-7, Josiah Carlson wrote:
> >> On Mon, Sep 17, 2012 at 10:40 PM, zs <zsl...@gmail.com> wrote: > >> > We have used redis for about 6+ month in production. There are a > couple > >> > of > >> > issues that I've observed with redis-2.4.4
> >> There are a few known issues with Redis 2.4.4, which is why 2.4.17 > exists > >> ;)
> >> > a) is there a possible memory leak? we've seen memory gone from 200M > to > >> > 10G > >> > in 6 month, but the data size hasn't changed. We are using mostly > hash > >> > data > >> > set, using jedis java library. Restarting the server seems to drop > the > >> > memory back down.
> >> There are a few potential causes of this, all of which are fixed in > >> releases up to and including Redis 2.4.17 . I would suggest you > >> upgrade.
> >> > b) keys command and expire command doesn't seem to work with specific > >> > keys? > >> > say i have a key a > >> > set a 10 > >> > expire a 10
> >> > however, key a is expired, a get still returns 10, though > documentation > >> > says > >> > that after a key expired, it will be deleted automatically
> >> > are these bugs in 2.4.4?
> >> I do not remember such a bug, but my memory may be flawed. Regardless, > >> you should upgrade to 2.4.17, it should fix your memory bug.
> >> > thx, and btw, is redis cluster going to be released anytime soon?
> >> Sadly, no. Redis 2.6 and sentinel are first, and while cluster has a > >> lot of work done for it, there is still enough work to be done that > >> Salvatore isn't even recommending people run it for testing. On the > >> other hand, Redis 2.6 has been in RC mode for a few months, and > >> sentinel has been available for several weeks.
> > To post to this group, send email to redi...@googlegroups.com<javascript:>.
> > To unsubscribe from this group, send email to > > redis-db+u...@googlegroups.com <javascript:>. > > For more options, visit this group at > > http://groups.google.com/group/redis-db?hl=en.
The information is really only useful when your server is having the
issue. From the INFO output, you only seem to have 1 client... yet you
have many lines from CLIENT LIST? That seems strange...
But specifically, CLIENT LIST shows lines with "oll=" information,
that is the length of the output buffer list in items. Redis 2.6 gives
you the actual bytes used, but knowing the number of items in the
output buffer is a good start for 2.4 .
> CLIENT LIST show a bunch of
> fd=8 idle=9 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=get
> not sure how to read them
> On Friday, September 21, 2012 9:05:42 AM UTC-7, Josiah Carlson wrote:
>> If your clients are using that much memory, then they likely either
>> have very large outgoing buffers associated with them, or had very
>> large outgoing buffers associated with them in the past that were not
>> released.
>> Can you check INFO and CLIENT LIST output? That may tell us if it is
>> related to outgoing buffers, or previous outgoing buffers.
>> - Josiah
>> On Fri, Sep 21, 2012 at 8:42 AM, zs <zsl...@gmail.com> wrote:
>> > Using 2.4.17 at least solved the expiration problem.
>> > However, I've observed another issue with 2.4.17.
>> > I have maxmemory set to 256M. However, I've seen the log showed that
>> > redis
>> > used the memory
>> > proportional to the number of clients (each might be writing updating
>> > the
>> > same key). There are only
>> > 3 keys used for this cache, total memory for these should be around 75M.
>> > And
>> > occationally, i get ERR memory
>> > exceeding maxmemory, but not always.
>> > So my question is:
>> > what's maxmemory? how is it observed? and how can total memory used be
>> > bigger than maxmemory?
>> > [14749] 21 Sep 11:34:31 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:34:31 - 17 clients connected (0 slaves), 554518168
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:34:36 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:34:36 - 17 clients connected (0 slaves), 638404152
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:34:41 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:34:41 - 17 clients connected (0 slaves), 667764216
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:34:46 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:34:46 - 17 clients connected (0 slaves), 667764176
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:34:51 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:34:51 - 17 clients connected (0 slaves), 667764144
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:34:56 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:34:56 - 17 clients connected (0 slaves), 667764144
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:35:01 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:01 - 17 clients connected (0 slaves), 667764144
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:35:06 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:06 - 16 clients connected (0 slaves), 630006888
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:35:11 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:11 - 13 clients connected (0 slaves), 516735128
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:35:16 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:16 - 12 clients connected (0 slaves), 478977856
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:35:21 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:21 - 10 clients connected (0 slaves), 403463328
>> > bytes
>> > in use
>> > [14749] 21 Sep 11:35:26 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:26 - 9 clients connected (0 slaves), 365706072
>> > bytes in
>> > use
>> > [14749] 21 Sep 11:35:31 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:31 - 9 clients connected (0 slaves), 365706064
>> > bytes in
>> > use
>> > [14749] 21 Sep 11:35:36 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:36 - 8 clients connected (0 slaves), 327948848
>> > bytes in
>> > use
>> > [14749] 21 Sep 11:35:41 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:41 - 8 clients connected (0 slaves), 327948848
>> > bytes in
>> > use
>> > [14749] 21 Sep 11:35:46 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:46 - 8 clients connected (0 slaves), 327948840
>> > bytes in
>> > use
>> > [14749] 21 Sep 11:35:51 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:51 - 7 clients connected (0 slaves), 290191560
>> > bytes in
>> > use
>> > [14749] 21 Sep 11:35:56 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:35:56 - 6 clients connected (0 slaves), 252434280
>> > bytes in
>> > use
>> > [14749] 21 Sep 11:36:01 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:36:01 - 6 clients connected (0 slaves), 252434304
>> > bytes in
>> > use
>> > [14749] 21 Sep 11:36:06 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > [14749] 21 Sep 11:36:06 - 6 clients connected (0 slaves), 252434304
>> > bytes in
>> > use
>> > [14749] 21 Sep 11:36:11 - DB 0: 2 keys (0 volatile) in 4 slots HT.
>> > On Tuesday, September 18, 2012 6:04:47 AM UTC-7, Josiah Carlson wrote:
>> >> On Mon, Sep 17, 2012 at 10:40 PM, zs <zsl...@gmail.com> wrote:
>> >> > We have used redis for about 6+ month in production. There are a
>> >> > couple
>> >> > of
>> >> > issues that I've observed with redis-2.4.4
>> >> There are a few known issues with Redis 2.4.4, which is why 2.4.17
>> >> exists
>> >> ;)
>> >> > a) is there a possible memory leak? we've seen memory gone from 200M
>> >> > to
>> >> > 10G
>> >> > in 6 month, but the data size hasn't changed. We are using mostly
>> >> > hash
>> >> > data
>> >> > set, using jedis java library. Restarting the server seems to drop
>> >> > the
>> >> > memory back down.
>> >> There are a few potential causes of this, all of which are fixed in
>> >> releases up to and including Redis 2.4.17 . I would suggest you
>> >> upgrade.
>> >> > b) keys command and expire command doesn't seem to work with specific
>> >> > keys?
>> >> > say i have a key a
>> >> > set a 10
>> >> > expire a 10
>> >> > however, key a is expired, a get still returns 10, though
>> >> > documentation
>> >> > says
>> >> > that after a key expired, it will be deleted automatically
>> >> > are these bugs in 2.4.4?
>> >> I do not remember such a bug, but my memory may be flawed. Regardless,
>> >> you should upgrade to 2.4.17, it should fix your memory bug.
>> >> > thx, and btw, is redis cluster going to be released anytime soon?
>> >> Sadly, no. Redis 2.6 and sentinel are first, and while cluster has a
>> >> lot of work done for it, there is still enough work to be done that
>> >> Salvatore isn't even recommending people run it for testing. On the
>> >> other hand, Redis 2.6 has been in RC mode for a few months, and
>> >> sentinel has been available for several weeks.
>> >> Regards,
>> >> - Josiah
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "Redis DB" group.
>> > To view this discussion on the web visit
>> > https://groups.google.com/d/msg/redis-db/-/RoyKzS4vFkoJ.
>> > To post to this group, send email to redi...@googlegroups.com.
>> > To unsubscribe from this group, send email to
>> > redis-db+u...@googlegroups.com.
>> > For more options, visit this group at
>> > http://groups.google.com/group/redis-db?hl=en.
> To post to this group, send email to redis-db@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.
On Friday, September 21, 2012 10:20:02 AM UTC-7, Josiah Carlson wrote:
> I'm sorry I wasn't more explicit.
> The information is really only useful when your server is having the > issue. From the INFO output, you only seem to have 1 client... yet you > have many lines from CLIENT LIST? That seems strange...
> But specifically, CLIENT LIST shows lines with "oll=" information, > that is the length of the output buffer list in items. Redis 2.6 gives > you the actual bytes used, but knowing the number of items in the > output buffer is a good start for 2.4 .
> Regards, > - Josiah
> On Fri, Sep 21, 2012 at 9:12 AM, zs <zsl...@gmail.com <javascript:>> > wrote: > > here is the output of info:
> > CLIENT LIST show a bunch of > > fd=8 idle=9 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r > cmd=get
> > not sure how to read them
> > On Friday, September 21, 2012 9:05:42 AM UTC-7, Josiah Carlson wrote:
> >> If your clients are using that much memory, then they likely either > >> have very large outgoing buffers associated with them, or had very > >> large outgoing buffers associated with them in the past that were not > >> released.
> >> Can you check INFO and CLIENT LIST output? That may tell us if it is > >> related to outgoing buffers, or previous outgoing buffers.
> >> - Josiah
> >> On Fri, Sep 21, 2012 at 8:42 AM, zs <zsl...@gmail.com> wrote: > >> > Using 2.4.17 at least solved the expiration problem.
> >> > However, I've observed another issue with 2.4.17.
> >> > I have maxmemory set to 256M. However, I've seen the log showed that > >> > redis > >> > used the memory > >> > proportional to the number of clients (each might be writing updating > >> > the > >> > same key). There are only > >> > 3 keys used for this cache, total memory for these should be around > 75M. > >> > And > >> > occationally, i get ERR memory > >> > exceeding maxmemory, but not always.
> >> > So my question is: > >> > what's maxmemory? how is it observed? and how can total memory used > be > >> > bigger than maxmemory?
> >> > On Tuesday, September 18, 2012 6:04:47 AM UTC-7, Josiah Carlson > wrote:
> >> >> On Mon, Sep 17, 2012 at 10:40 PM, zs <zsl...@gmail.com> wrote: > >> >> > We have used redis for about 6+ month in production. There are a > >> >> > couple > >> >> > of > >> >> > issues that I've observed with redis-2.4.4
> >> >> There are a few known issues with Redis 2.4.4, which is why 2.4.17 > >> >> exists > >> >> ;)
> >> >> > a) is there a possible memory leak? we've seen memory gone from > 200M > >> >> > to > >> >> > 10G > >> >> > in 6 month, but the data size hasn't changed. We are using mostly > >> >> > hash > >> >> > data > >> >> > set, using jedis java library. Restarting the server seems to drop > >> >> > the > >> >> > memory back down.
> >> >> There are a few potential causes of this, all of which are fixed in > >> >> releases up to and including Redis 2.4.17 . I would suggest you > >> >> upgrade.
> >> >> > b) keys command and expire command doesn't seem to work with > specific > >> >> > keys? > >> >> > say i have a key a > >> >> > set a 10 > >> >> > expire a 10
> >> >> > however, key a is expired, a get still returns 10, though > >> >> > documentation > >> >> > says > >> >> > that after a key expired, it will be deleted automatically
> >> >> > are these bugs in 2.4.4?
> >> >> I do not remember such a bug, but my memory may be flawed. > Regardless, > >> >> you should upgrade to 2.4.17, it
You have 2 keys in your database. One is 25 megabytes, the other is 35
megabytes. And all you are doing is reading and writing those two
values?
Your memory use is caused by one of three things:
1. incoming buffers
2. outgoing buffers
3. memory fragmentation
Aside from changing your access patterns (breaking your data up into
smaller pieces that *won't* lead to fragmentation), forcing Redis to
pre-allocate fixed-size blocks for big allocations like this to
prevent fragmentation when you SET/GET values, or a few other things,
I am out of "simple" solutions.
More specifically, you have access patterns that the current memory
allocator (and Redis) was not designed to handle. I'm not surprised
that you're having a bad time.
If you can explain the problem you are attempting to solve, we might
be able to offer a better solution that doesn't involve these
operations, that would give you better memory behavior.
> On Friday, September 21, 2012 10:20:02 AM UTC-7, Josiah Carlson wrote:
>> I'm sorry I wasn't more explicit.
>> The information is really only useful when your server is having the
>> issue. From the INFO output, you only seem to have 1 client... yet you
>> have many lines from CLIENT LIST? That seems strange...
>> But specifically, CLIENT LIST shows lines with "oll=" information,
>> that is the length of the output buffer list in items. Redis 2.6 gives
>> you the actual bytes used, but knowing the number of items in the
>> output buffer is a good start for 2.4 .
>> Regards,
>> - Josiah
>> On Fri, Sep 21, 2012 at 9:12 AM, zs <zsl...@gmail.com> wrote:
>> > here is the output of info:
>> > CLIENT LIST show a bunch of
>> > fd=8 idle=9 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r
>> > cmd=get
>> > not sure how to read them
>> > On Friday, September 21, 2012 9:05:42 AM UTC-7, Josiah Carlson wrote:
>> >> If your clients are using that much memory, then they likely either
>> >> have very large outgoing buffers associated with them, or had very
>> >> large outgoing buffers associated with them in the past that were not
>> >> released.
>> >> Can you check INFO and CLIENT LIST output? That may tell us if it is
>> >> related to outgoing buffers, or previous outgoing buffers.
>> >> - Josiah
>> >> On Fri, Sep 21, 2012 at 8:42 AM, zs <zsl...@gmail.com> wrote:
>> >> > Using 2.4.17 at least solved the expiration problem.
>> >> > However, I've observed another issue with 2.4.17.
>> >> > I have maxmemory set to 256M. However, I've seen the log showed that
>> >> > redis
>> >> > used the memory
>> >> > proportional to the number of clients (each might be writing updating
>> >> > the
>> >> > same key). There are only
>> >> > 3 keys used for this cache, total memory for these should be around
>> >> > 75M.
>> >> > And
>> >> > occationally, i get ERR memory
>> >> > exceeding maxmemory, but not always.
>> >> > So my question is:
>> >> > what's maxmemory? how is it observed? and how can total memory used
>> >> > be
>> >> > bigger than maxmemory?
The use case is to cache a couple of big data structure.
I can use compression to reduce the size of the data, but is that the real problem? redis documents says that value can be 512M.
The other thing is that one of the key (with 35M-length value) has a expiry. I know expiry didn't quite work in 2.4.4, could there be a problem with 2.4.17 as well? (i.e., key expired, but memory didn't really get de-allocated?).
How do you avoid memory fragmentation? (mem_fragmentation_ratio:0.98, does it mean that it's 98percent fragmented?)
On Sunday, September 23, 2012 7:56:45 AM UTC-7, Josiah Carlson wrote:
> Wait...
> You have 2 keys in your database. One is 25 megabytes, the other is 35 > megabytes. And all you are doing is reading and writing those two > values?
> Your memory use is caused by one of three things: > 1. incoming buffers > 2. outgoing buffers > 3. memory fragmentation
> Aside from changing your access patterns (breaking your data up into > smaller pieces that *won't* lead to fragmentation), forcing Redis to > pre-allocate fixed-size blocks for big allocations like this to > prevent fragmentation when you SET/GET values, or a few other things, > I am out of "simple" solutions.
> More specifically, you have access patterns that the current memory > allocator (and Redis) was not designed to handle. I'm not surprised > that you're having a bad time.
> If you can explain the problem you are attempting to solve, we might > be able to offer a better solution that doesn't involve these > operations, that would give you better memory behavior.
> Regards, > - Josiah
> On Fri, Sep 21, 2012 at 12:03 PM, c.z <cz...@rocketfuelinc.com<javascript:>> > wrote: > > I did a bunch of CLIENT LIST calls.
> > I see up to 30 clients, but all most all of them have 0 (the same output > i > > pasted).
> > I did on two instances where I see qbuff = 6047336 and qbuf = 1779140.
> > My two keys are of size 25M and 35M, and those are the only two keys I > use. > > Clients > > either get or set on these keys.
> > The info command showed that we have used up to 500M, how could that > happen?
> > On Friday, September 21, 2012 10:20:02 AM UTC-7, Josiah Carlson wrote:
> >> I'm sorry I wasn't more explicit.
> >> The information is really only useful when your server is having the > >> issue. From the INFO output, you only seem to have 1 client... yet you > >> have many lines from CLIENT LIST? That seems strange...
> >> But specifically, CLIENT LIST shows lines with "oll=" information, > >> that is the length of the output buffer list in items. Redis 2.6 gives > >> you the actual bytes used, but knowing the number of items in the > >> output buffer is a good start for 2.4 .
> >> Regards, > >> - Josiah
> >> On Fri, Sep 21, 2012 at 9:12 AM, zs <zsl...@gmail.com> wrote: > >> > here is the output of info:
> >> > CLIENT LIST show a bunch of > >> > fd=8 idle=9 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r > >> > cmd=get
> >> > not sure how to read them
> >> > On Friday, September 21, 2012 9:05:42 AM UTC-7, Josiah Carlson wrote:
> >> >> If your clients are using that much memory, then they likely either > >> >> have very large outgoing buffers associated with them, or had very > >> >> large outgoing buffers associated with them in the past that were > not > >> >> released.
> >> >> Can you check INFO and CLIENT LIST output? That may tell us if it is > >> >> related to outgoing buffers, or previous outgoing buffers.
> >> >> - Josiah
> >> >> On Fri, Sep 21, 2012 at 8:42 AM, zs <zsl...@gmail.com> wrote: > >> >> > Using 2.4.17 at least solved the expiration problem.
> >> >> > However, I've observed another issue with 2.4.17.
> >> >> > I have maxmemory set to 256M. However, I've seen the log showed > that > >> >> > redis > >> >> > used the memory > >> >> > proportional to the number of clients (each might be writing > updating > >> >> > the > >> >> > same key). There are only > >> >> > 3 keys used for this cache, total memory for these should be > around > >> >> > 75M. > >> >> > And > >> >> > occationally, i get ERR memory > >> >> > exceeding maxmemory, but not always.
> >> >> > So my question is: > >> >> > what's maxmemory? how is it observed? and how can total memory > used > >> >> > be > >> >> > bigger than maxmemory?
On Sun, Sep 23, 2012 at 6:23 PM, c.z <c...@rocketfuelinc.com> wrote:
> The use case is to cache a couple of big data structure.
Does the entire structure change every time it is written? Only a
small part? Can you encode the changes as a series of deltas, which
are rolled up occasionally?
> I can use compression to reduce the size of the data, but is that the real
> problem? redis documents says that value can be 512M.
*Can be* is not the same as *should be*.
Redis is limited by what the underlying memory allocator is able to
do. Redis currently uses Jemalloc, which does a few things to
reduce/minimize memory fragmentation, and which has solved the issue
for the majority of use-cases. But your particular use-case seems like
it is stressing the memory allocator in ways that it wasn't intended
(a few large allocations that are repeated and freed).
> The other thing is that one of the key (with 35M-length value) has a expiry.
> I know expiry didn't quite work in 2.4.4, could there be a problem with
> 2.4.17 as well? (i.e., key expired, but memory didn't really get
> de-allocated?).
When data "expires" in Redis, it is not an active expiration. That is,
if a key passes expiration, that doesn't mean it is immediately
deallocated. Typically, Redis will make a handful of random probes to
clean out a few keys if it finds any that should be expired. If too
many are found, it performs a scan to clean out as many expired keys
as possible.
Looking at your most recent INFO output:
db0:keys=2,expires=0
You don't have any keys that should be expired.
That said, if you only have a few keys, they should be expired almost
as though they were actively expired (unless I'm missing something).
> How do you avoid memory fragmentation? (mem_fragmentation_ratio:0.98, does
> it mean that it's 98percent fragmented?)
No. 1.0 means that Redis is using exactly as much memory as is
expected. < 1.0 means that it is using less memory than is expected. >
1.0 means it is using more. If you are seeing < 1.0, that is usually
a sign of a bug somewhere. Typical fragmentation ratios with the
current jemalloc allocator are usually in the 1.0 to 1.15 range, which
means up to 15% memory is used that is not accounted for in other
parts.
Also, "memory fragmentation ratio" is an attempt to measure an issue
that occurs in all processes that run on computers, specifically a
combination of "internal fragmentation" and "external fragmentation"
as described: http://en.wikipedia.org/wiki/Fragmentation_(computing) .
Though the ratio that is calculated by Redis is different than the
description on the wikipedia page.
> On Sunday, September 23, 2012 7:56:45 AM UTC-7, Josiah Carlson wrote:
>> Wait...
>> You have 2 keys in your database. One is 25 megabytes, the other is 35
>> megabytes. And all you are doing is reading and writing those two
>> values?
>> Your memory use is caused by one of three things:
>> 1. incoming buffers
>> 2. outgoing buffers
>> 3. memory fragmentation
>> Aside from changing your access patterns (breaking your data up into
>> smaller pieces that *won't* lead to fragmentation), forcing Redis to
>> pre-allocate fixed-size blocks for big allocations like this to
>> prevent fragmentation when you SET/GET values, or a few other things,
>> I am out of "simple" solutions.
>> More specifically, you have access patterns that the current memory
>> allocator (and Redis) was not designed to handle. I'm not surprised
>> that you're having a bad time.
>> If you can explain the problem you are attempting to solve, we might
>> be able to offer a better solution that doesn't involve these
>> operations, that would give you better memory behavior.
>> Regards,
>> - Josiah
>> On Fri, Sep 21, 2012 at 12:03 PM, c.z <cz...@rocketfuelinc.com> wrote:
>> > I did a bunch of CLIENT LIST calls.
>> > I see up to 30 clients, but all most all of them have 0 (the same output
>> > i
>> > pasted).
>> > I did on two instances where I see qbuff = 6047336 and qbuf = 1779140.
>> > My two keys are of size 25M and 35M, and those are the only two keys I
>> > use.
>> > Clients
>> > either get or set on these keys.
>> > The info command showed that we have used up to 500M, how could that
>> > happen?
>> > On Friday, September 21, 2012 10:20:02 AM UTC-7, Josiah Carlson wrote:
>> >> I'm sorry I wasn't more explicit.
>> >> The information is really only useful when your server is having the
>> >> issue. From the INFO output, you only seem to have 1 client... yet you
>> >> have many lines from CLIENT LIST? That seems strange...
>> >> But specifically, CLIENT LIST shows lines with "oll=" information,
>> >> that is the length of the output buffer list in items. Redis 2.6 gives
>> >> you the actual bytes used, but knowing the number of items in the
>> >> output buffer is a good start for 2.4 .
>> >> Regards,
>> >> - Josiah
>> >> On Fri, Sep 21, 2012 at 9:12 AM, zs <zsl...@gmail.com> wrote:
>> >> > here is the output of info:
>> >> > CLIENT LIST show a bunch of
>> >> > fd=8 idle=9 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r
>> >> > cmd=get
>> >> > not sure how to read them
>> >> > On Friday, September 21, 2012 9:05:42 AM UTC-7, Josiah Carlson wrote:
>> >> >> If your clients are using that much memory, then they likely either
>> >> >> have very large outgoing buffers associated with them, or had very
>> >> >> large outgoing buffers associated with them in the past that were
>> >> >> not
>> >> >> released.
>> >> >> Can you check INFO and CLIENT LIST output? That may tell us if it is
>> >> >> related to outgoing buffers, or previous outgoing buffers.
>> >> >> - Josiah
>> >> >> On Fri, Sep 21, 2012 at 8:42 AM, zs <zsl...@gmail.com> wrote:
>> >> >> > Using 2.4.17 at least solved the expiration problem.
>> >> >> > However, I've observed another issue with 2.4.17.
>> >> >> > I have maxmemory set to 256M. However, I've seen the log showed
>> >> >> > that
>> >> >> > redis
>> >> >> > used the memory
>> >> >> > proportional to the number of clients (each might be writing
>> >> >> > updating
>> >> >> > the
>> >> >> > same key). There are only
>> >> >> > 3 keys used for this cache, total memory for these should be
>> >> >> > around
>> >> >> > 75M.
>> >> >> > And
>> >> >> > occationally, i get ERR memory
>> >> >> > exceeding maxmemory, but not always.
>> >> >> > So my question is:
>> >> >> > what's maxmemory? how is it observed? and how can total memory
>> >> >> > used
>> >> >> > be
>> >> >> > bigger than maxmemory?
showed that keys did get expired. But based on your explanation, it might be possible that key-value stays in memory for a much longer period, as they don't get removed right away...
On Mon, Sep 24, 2012 at 10:11 AM, c.z <c...@rocketfuelinc.com> wrote:
>>That said, if you only have a few keys, they should be expired almost
>> as though they were actively expired (unless I'm missing something).
> by 'actively expired', do you mean that the client should delete the keys
> itself?
Some systems with expiration keep a priority queue of items to be
expired. Redis doesn't. Redis will randomly check keys to be expired,
and expire them.
> showed that keys did get expired. But based on your explanation, it might be
> possible that key-value
> stays in memory for a much longer period, as they don't get removed right
> away...
That is not the case. It is either fragmentation, or it might be kept
around to finish the transfer to clients that requested it before it
had expired (especially for slow clients).
> On Friday, September 21, 2012 10:20:02 AM UTC-7, Josiah Carlson wrote:
> I'm sorry I wasn't more explicit.
> The information is really only useful when your server is having the > issue. From the INFO output, you only seem to have 1 client... yet you > have many lines from CLIENT LIST? That seems strange...
> But specifically, CLIENT LIST shows lines with "oll=" information, > that is the length of the output buffer list in items. Redis 2.6 gives > you the actual bytes used, but knowing the number of items in the > output buffer is a good start for 2.4 .
> Regards, > - Josiah
> On Fri, Sep 21, 2012 at 9:12 AM, zs <zsl...@gmail.com> wrote: > > here is the output of info:
> > CLIENT LIST show a bunch of > > fd=8 idle=9 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=get
> > not sure how to read them
> > On Friday, September 21, 2012 9:05:42 AM UTC-7, Josiah Carlson wrote:
> >> If your clients are using that much memory, then they likely either > >> have very large outgoing buffers associated with them, or had very > >> large outgoing buffers associated with them in the past that were not > >> released.
> >> Can you check INFO and CLIENT LIST output? That may tell us if it is > >> related to outgoing buffers, or previous outgoing buffers.
> >> - Josiah
> >> On Fri, Sep 21, 2012 at 8:42 AM, zs <zsl...@gmail.com> wrote: > >> > Using 2.4.17 at least solved the expiration problem.
> >> > However, I've observed another issue with 2.4.17.
> >> > I have maxmemory set to 256M. However, I've seen the log showed that > >> > redis > >> > used the memory > >> > proportional to the number of clients (each might be writing updating > >> > the > >> > same key). There are only > >> > 3 keys used for this cache, total memory for these should be around 75M. > >> > And > >> > occationally, i get ERR memory > >> > exceeding maxmemory, but not always.
> >> > So my question is: > >> > what's maxmemory? how is it observed? and how can total memory used be > >> > bigger than maxmemory?
> >> > On Tuesday, September 18, 2012 6:04:47 AM UTC-7, Josiah Carlson wrote:
> >> >> On Mon, Sep 17, 2012 at 10:40 PM, zs <zsl...@gmail.com> wrote: > >> >> > We have used redis for about 6+ month in production. There are a > >> >> > couple > >> >> > of > >> >> > issues that I've observed with redis-2.4.4
> >> >> There are a few known issues with Redis 2.4.4, which is why 2.4.17 > >> >> exists > >> >> ;)
> >> >> > a) is there a possible memory leak? we've seen memory gone from 200M > >> >> > to > >> >> > 10G > >> >> > in 6 month, but the data size hasn't changed. We are using mostly > >> >> > hash > >> >> > data > >> >> > set, using jedis java library. Restarting the server seems to drop > >> >> > the > >> >> > memory back down.
> >> >> There are a few potential causes of this, all of which are fixed in > >> >> releases up to and including Redis 2.4.17 . I would suggest you > >> >> upgrade.
> >> >> > b) keys command and expire command doesn't seem to work with specific > >> >> > keys? > >> >> > say i have a key a > >> >> > set a 10 > >> >> > expire a 10
