Usage monitor command cause use-memory decrease to 400M from 150G in redis-server 4.0.1.

[liu tired]

This I use info sampled information. I don't understand why. who can explain this thing?

[hva...@gmail.com]

Does your Redis logfile or your operating system logfiles indicate any sort of trouble during that time?  For example the Redis server crashed and took time to re-fill its database? 

[liu tired]

I examined redis's logfile. This time log show:

This log is normal. I think have no action cause use-memory decreasing. 

在 2018年2月3日星期六 UTC+8下午5:00:37，hva...@gmail.com写道：

[hva...@gmail.com]

Yes, I see in your logfile snippet the same process (133201) is still running through the time in question, so there was no crash.  There's some confirming evidence the database grew smaller in the log -  the child process that saved the snapshot reported 509 MB used at 00:25, and only 6 MB used at 06:00.

There is one entry in between that looks unexpected to me:  The report of a CONFIG REWRITE at 00:52.  What rewrites your configuration in your running Redis processes?  What was the reason for rewriting the configuration?  Did this machine change master/slave roles or attach itself as slave to a new master?  The change in memory consumption may have been related to the config change.

Are you using Sentinel to control your Redis processes?  Or perhaps you're using Redis Cluster for sharding?

[liu tired]

I confirmed with our redis dba.He only modified client-output-buffer-limit to 'normal 0 1024mb 10' from 'normal 0 0 0'. I think the action not lead to the problem I describe.

在 2018年2月4日星期日 UTC+8上午5:08:49，hva...@gmail.com写道：

[hva...@gmail.com]

I don't understand.  Your Redis dba performed a command that would change the size of all the client output buffers from unlimited to 1 GB.  Why do you feel that this could not cause the clients with buffers larger than 1GB to be disconnected, and their buffered data to be discarded, which would reduce the memory being used by your Redis server?

[liu tired]

I dump redis to rdb file at 06:00 every day. I analyzed the January 24, 2018 rdb. All the data is 90G. So I think wharever you have too much cache, the use-memory should be not less than 90G.But it is 400M from my picture.

在 2018年2月5日星期一 UTC+8下午7:11:36，hva...@gmail.com写道：

[hva...@gmail.com]

I can't tell you what happened on your Redis server.  You have an unusual configuration change that was performed about the same time as an unusual change in memory usage began started on your graphs.  With the little information I have, it seems to me most likely that the configuration change caused the change in memory usage.

There isn't a way for someone outside your organization, who doesn't have access to your other monitoring graphs, doesn't have access to your Redis configuration files and logfiles, and doesn't know what kind of data you keep in this Redis or how much is transient cache vs. permanent data, can tell you what happened in this incident.  My best suggestion is to add graphs for metrics like the number of keys in the database, the number of connected clients, and so on, to your monitoring.  This will give you more information to analyze after an incident like this, so you can find the cause.  Raising the Redis logfile level to "notice" or "verbose" may also provide useful information to analyze an incident afterward.

[liu tired]

Thank you. I got it. I will add some detail monitor info.

在 2018年2月6日星期二 UTC+8下午11:52:43，hva...@gmail.com写道：