Memory consumption of sentinel

[Dbz Fan]

How much memory can the sentinel consume atmost . What are the paramters that can vary the memory consumption of sentinel and is there any way to monitor it?

[Jan-Erik Rediger]

Sentinel itself doesn't store much data except some state about the
known instances it monitors.
Therefore the memory consumption should be in the low megabytes (if so
much at all, though there currently seems to be a memory leak in case
you have password-protected instances and misconfigured Sentinel:
https://github.com/antirez/redis/issues/2535)

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at http://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.

[Dbz Fan]

Ok that issues seems to be resolved by antirez as stated in the link it was hiredis specific issue.
So I am using jedis and currently observing 12 mb rss on my sentinels . I have 3 sentinels monitoring one master with 2 slaves .There has been no incident of slave or master going down 

[Dbz Fan]

Just to add to my previous comment initially i was seeing around 7.5 mb rss memory usage for sentinel(using top command) and gradually the rss memory kept on increasing while i was continously inserting keys in redis master .
I am not sure why is there an increase in memory consumption from sentinel side , is it possible that even jedis has memory leak issue like hiredis did.? 

[The Baldguy]

On Wednesday, April 29, 2015 at 8:20:45 AM UTC-5, Dbz Fan wrote:

Just to add to my previous comment initially i was seeing around 7.5 mb rss memory usage for sentinel(using top command) and gradually the rss memory kept on increasing while i was continously inserting keys in redis master .

Are you sure you weren't seeing the memory use of the master or a slave? Adding keys to Redis should not be affecting memory usage in Sentinel at all.

 

I am not sure why is there an increase in memory consumption from sentinel side , is it possible that even jedis has memory leak issue like hiredis did.? 

A leak in Jedis would increase client memory usage, not Redis or Sentinel memory usage. 

Cheers,

Bill

[Dbz Fan]

Yup rechecked it, all the 3 sentinels are showing increase in memory . So my configuration is this : 3 pcs A,B,C . A has master and sentinel ; B has slave and sentinel ; C has slave and sentinel. Is the increase possible due to configuring of master/slave on same PC as the sentinel ?

Also in the issue reported here https://github.com/antirez/redis/issues/2535 suggests to me that the increase was seen in sentinel memory even though memory leak was in hiredis. In that context I was thinking maybe the issue could be in jedis .Please correct me here if  I am missing anything here.

[Jan-Erik Rediger]

Redis Sentinel uses hiredis internally.

[Bill E. Anderson]

> On Apr 29, 2015, at 10:34 AM, Dbz Fan <manass...@gmail.com> wrote:
>
> Yup rechecked it, all the 3 sentinels are showing increase in memory . So my configuration is this : 3 pcs A,B,C . A has master and sentinel ; B has slave and sentinel ; C has slave and sentinel. Is the increase possible due to configuring of master/slave on same PC as the sentinel ?

Not likely but it is possible you’re looking at the wrong process as depending on the command line used to launch sentinel and redis they’ll both show as “redis-server’ in the process listing. ;) It is also possible your sentinels aren’t getting a valid connection to your Redis master instances - which would trigger the bug referenced.

When you connect to the sentinel(s) and execute a “sentinel slaves <name>” do the slaves show up? If not, you aren’t establishing an effective connection to the master.

On a side note, it isn’t a good idea to run sentinel on the same nodes as your Redis instances.

> Also in the issue reported here https://github.com/antirez/redis/issues/2535 suggests to me that the increase was seen in sentinel memory even though memory leak was in hiredis.
> In that context I was thinking maybe the issue could be in jedis .Please correct me here if I am missing anything here.

Jedis is a client library, not a server-side one. If Jedis leaks your client application will grow in memory, not sentinel. The leak described was when Sentinel, using Hiredis, connected to a Redis *server* and could not obtain a valid connection such as bad password, timeout, nod not listening, etc.. In this case Sentinel is acting as a client, so a leak in a client library caused the memory increase to be visible in Sentinel-as-a-client. Similarly if Jedis leaks your application which uses Jedis would see the memory usage, not the server.


It is hypothetically possible that sentinel could have a leak when clients connect and disconnect a lot. Are your clients connecting to Sentinel to learn of the master location at all or merely connecting directly to the master?


Cheers,
Bill

[Dbz Fan]

Oh I thought hiredis was just a redis client like jedis moslty due to the folllowing link : http://redis.io/clients.
Anyways thanks for that info.
So , one incident i would like to add here that one of my slave is shown to be +SDOWN in all three sentinels even though that slave is up and running with rreplication happening ,also checked the master with "info " command its showing the slave online.
Can that coupled with the hiredis bug be the cause?

[Dbz Fan]

So I am seeing "redis-server" and "redis-sentinel" as seperate processes .So I am pretty sure that I am checking the correct process

[Bill E. Anderson]

> On Apr 29, 2015, at 10:50 AM, Dbz Fan <manass...@gmail.com> wrote:
>
> Oh I thought hiredis was just a redis client like jedis moslty due to the folllowing link : http://redis.io/clients.
> Anyways thanks for that info.
> So , one incident i would like to add here that one of my slave is shown to be +SDOWN in all three sentinels even though that slave is up and running with rreplication happening ,also checked the master with "info " command its showing the slave online.

That’s rather important information to have here. ;)

> Can that coupled with the hiredis bug be the cause?

Absolutely - if the sentinels are unable to obtain a connection that would, I believe, be triggering the bug.


Cheers,
Bill

[Dbz Fan]

Ok so I will try to remove the slave which is "seemingly" disconected from my configuration and check if the memory growth is still occuring in sentinels and i"ll update accordingly

[Salvatore Sanfilippo]

Hello, big warning flag: without "make distclean; make" the hiredis
version will not get updated even after you used the latest commit
with the fix, so you'll still experiment the bug. Please use:

make distclean
make

Then use the new binary for Redis Sentinel.

Cheers,
Salvatore

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at http://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.

--
Salvatore 'antirez' Sanfilippo
open source developer - Pivotal http://pivotal.io

"If a system is to have conceptual integrity, someone must control the
concepts."
— Fred Brooks, "The Mythical Man-Month", 1975.

[Dbz Fan]

Ok thanks for that update .I have currently not updated my redis version as I didnt know this bug would affect me , now that it seems it is I think its time to upgrade.
Just to confirm I am using jedis ,would I still need to upgrade as the issue was reported for hiredis only.

[Bill E. Anderson]

On Apr 29, 2015, at 11:41 AM, Dbz Fan <manass...@gmail.com> wrote:

Ok thanks for that update .I have currently not updated my redis version as I didnt know this bug would affect me , now that it seems it is I think its time to upgrade.
Just to confirm I am using jedis ,would I still need to upgrade as the issue was reported for hiredis only.

Yes, you will need to update your sentinel version as *sentinel uses Hiredis*.