redis-server blocks for more that 120 seconds

[Amar Nv]

Has anyone came across below logs in syslog that indicates redis-server is blocked for more than 120 seconds?
When this issue started to happen - I logged into the system and went to the directory where redisdb is stored. Executing 'ls' command in this path is hanging forever - This made me to suspect file-system instead of redis-server.
BUT wanted opinions from other when we can these logs and what are all the options available to recover from this state.

Aug 22 17:55:49 7311-6 kernel: [4173757.550224] INFO: task redis-server:6728 blocked for more than 120 seconds.
Aug 22 17:55:49 7311-6 kernel: [4173757.557344] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 22 17:55:49 7311-6 kernel: [4173757.565355] redis-server    D ffffffff81806200     0  6728   2944 0x00000000
Aug 22 17:55:49 7311-6 kernel: [4173757.565363]  ffff88014ce55c78 0000000000000082 0000000000000293 ffff8801ba85c100
Aug 22 17:55:49 7311-6 kernel: [4173757.565370]  ffff88014ce55fd8 ffff88014ce55fd8 ffff88014ce55fd8 0000000000013780
Aug 22 17:55:49 7311-6 kernel: [4173757.565375]  ffff880403e01700 ffff8803f2aeae00 ffff8803f3d9bf30 ffff8803f3d9bf30
Aug 22 17:55:49 7311-6 kernel: [4173757.565381] Call Trace:
Aug 22 17:55:49 7311-6 kernel: [4173757.565391]  [<ffffffff8165817f>] schedule+0x3f/0x60
Aug 22 17:55:49 7311-6 kernel: [4173757.565397]  [<ffffffff81658f87>] __mutex_lock_slowpath+0xd7/0x150
Aug 22 17:55:49 7311-6 kernel: [4173757.565402]  [<ffffffff81658b9a>] mutex_lock+0x2a/0x50
Aug 22 17:55:49 7311-6 kernel: [4173757.565408]  [<ffffffff81185bc4>] do_last+0x2b4/0x730
Aug 22 17:55:49 7311-6 kernel: [4173757.565412]  [<ffffffff811873e1>] path_openat+0xd1/0x3f0
Aug 22 17:55:49 7311-6 kernel: [4173757.565417]  [<ffffffff81187822>] do_filp_open+0x42/0xa0
Aug 22 17:55:49 7311-6 kernel: [4173757.565423]  [<ffffffff81318e21>] ? strncpy_from_user+0x31/0x40
Aug 22 17:55:49 7311-6 kernel: [4173757.565427]  [<ffffffff81182b6a>] ? do_getname+0x10a/0x180
Aug 22 17:55:49 7311-6 kernel: [4173757.565432]  [<ffffffff8165a08e>] ? _raw_spin_lock+0xe/0x20
Aug 22 17:55:49 7311-6 kernel: [4173757.565437]  [<ffffffff81194ae7>] ? alloc_fd+0xf7/0x150
Aug 22 17:55:49 7311-6 kernel: [4173757.565442]  [<ffffffff81176e5d>] do_sys_open+0xed/0x220
Aug 22 17:55:49 7311-6 kernel: [4173757.565446]  [<ffffffff81176fb0>] sys_open+0x20/0x30
Aug 22 17:55:49 7311-6 kernel: [4173757.565450]  [<ffffffff81662682>] system_call_fastpath+0x16/0x1b

[Josiah Carlson]

Are you on AWS? Is your filesystem stored remotely? Have you checked whether your filesystem has errors? What other debugging have you performed?

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.