Redis master node crashing again and again

[Rohit Tanwar]

Hi team,

I'm seeking your help on the below issue.

below are the stats for data load and suddenly after around 40 minutes, the Redis-master restarted again due to which we are experiencing timeout errors in our application.

We are using Redis 5.0.7 with 1 master and 2 slaves (EC2 instance)

Getting the below errors in the logs

26976:M 31 Jan 2022 10:47:00.947 - Accepted 127.0.0.1:38718
26976:signal-handler (1643626020) Received SIGTERM scheduling shutdown...
12059:signal-handler (1643626020) Received SIGTERM scheduling shutdown...
26976:M 31 Jan 2022 10:47:00.976 - Client closed connection
26976:M 31 Jan 2022 10:47:00.993 # User requested shutdown...
26976:M 31 Jan 2022 10:47:00.993 # There is a child rewriting the AOF. Killing it!
26976:M 31 Jan 2022 10:47:00.993 * Calling fsync() on the AOF file.
26976:M 31 Jan 2022 10:47:01.047 * Saving the final RDB snapshot before exiting.
26976:M 31 Jan 2022 10:48:19.882 * DB saved on disk
26976:M 31 Jan 2022 10:48:19.882 * Removing the pid file.
26976:M 31 Jan 2022 10:48:19.883 # Redis is now ready to exit, bye bye...

During this time the connection from slave also breaks

# Replication
role:master
connected_slaves:0

redis-cli --stat 

------- data ------ --------------------- load -------------------- - child -
keys       mem      clients blocked requests            connections
16607963   167.87G  236     0       291994353 (+164508) 835         SAVE
16607911   167.96G  236     0       292157428 (+163075) 836         SAVE
16607920   168.06G  236     0       292323037 (+165609) 836         SAVE
16607962   168.16G  236     0       292485553 (+162516) 836         SAVE
16607873   168.25G  236     0       292650983 (+165430) 836         SAVE
16607788   168.35G  236     0       292815203 (+164220) 836         SAVE
16607764   168.44G  236     0       292978052 (+162849) 836         SAVE
16607706   168.54G  236     0       293141599 (+163547) 836         SAVE
16607664   168.63G  236     0       293305391 (+163792) 836         SAVE
16607566   168.73G  236     0       293470147 (+164756) 836         SAVE
16607560   168.83G  236     0       293632666 (+162519) 836         SAVE
16607539   168.92G  236     0       293796461 (+163795) 837         SAVE
16607411   169.02G  236     0       293960603 (+164142) 837         SAVE
16607345   169.11G  236     0       294124735 (+164132) 837         SAVE
16607242   169.21G  236     0       294287719 (+162984) 837         SAVE
16607210   169.30G  236     0       294450991 (+163272) 837         SAVE
16607229   169.40G  236     0       294614606 (+163615) 837         SAVE
16607203   169.49G  236     0       294779166 (+164560) 837         SAVE
16607133   6.71G    236     0       294897735 (+118569) 837         SAVE
16607140   6.72G    238     0       294922676 (+24941)  869         AOF
------- data ------ --------------------- load -------------------- - child -
keys       mem      clients blocked requests            connections
16607093   6.72G    238     0       294943261 (+20585)  869         AOF
16607088   6.73G    237     0       294963676 (+20415)  876         AOF
16607043   6.74G    236     0       295004609 (+40933)  876         AOF
16607060   6.79G    236     0       295105367 (+100758) 877         AOF
16607007   6.82G    236     0       295246972 (+141605) 877         AOF
16607040   6.88G    236     0       295406666 (+159694) 877         AOF
16607030   6.94G    236     0       295571765 (+165099) 877         AOF
16606957   7.00G    236     0       295752310 (+180545) 877         AOF
16606942   7.06G    236     0       295938520 (+186210) 877         AOF
16606877   7.12G    236     0       296128675 (+190155) 877         AOF
16606887   7.19G    236     0       296325585 (+196910) 877         AOF
16606820   7.26G    236     0       296525700 (+200115) 877         AOF
16606831   7.30G    236     0       296721063 (+195363) 878         AOF
16606881   7.38G    236     0       296925246 (+204183) 878         AOF
16606693   7.46G    236     0       297133971 (+208725) 878         AOF
16606712   7.53G    236     0       297341622 (+207651) 878         AOF
16606746   7.60G    236     0       297545301 (+203679) 878         AOF
16606620   7.66G    236     0       297747312 (+202011) 878         AOF
16606526   7.73G    236     0       297956015 (+208703) 878         AOF
16606561   7.79G    236     0       298164094 (+208079) 878         AOF