Hikari CP pool giving "Thread starvation" error

[abhi...@dailyninja.in]

I got the below error while using HikariCP . After this message the connection pool stopped working and the application stopped taking requests.

I am using HikariCP with Spring boot and the application is deployed in AWS server.

2016-08-09 20:04:14 WARN  c.z.h.p.HikariPool:620 - 20m10s174ms687μs975ns - Thread starvation or clock leap detected (housekeeper delta=HikariCP).

My environment setting:

Hikari version : 2.4.7

Spring boot version : 1.4.0.RELEASE

Java version : 1.8.0_74

Database : MySQL Server 5.7

Mysql connector version : 5.1.39

HikariCP config : 

thread pool size = 60

Can someone help me solve this issue.

[Brett Wooldridge]

This message indicates that the pool housekeeping thread, which runs every 30 seconds on a ScheduledExecutorService, did not run for 20 minutes and 10 seconds.

There are not too many things that could cause that.  Either the AWS instance was suspended or possibly the MySQL database instance was suspended.  Or something strange, like the IP address of the machine or the database machine changed dynamically -- causing a TCP-level block until the OS network stack recovered.

Are there other logs on the machine that indicate that system activity was continuing during this time?  Can you post your full configuration?

-Brett