Abandonned ConnectionProvider connections on redeploy/undeploy

[Fernando Bárcenas Martínez]

Good day.

A couple weeks ago I implemented HikariCP Pooling in our applications (Spring, Tomcat, Mysql). Everything was ok until one coworker told me he couldn't access the system even if it was live. When i checked on it, it threw the well-known error 'too many connections'. The fix for that, as I was busy in other stuff was to restart the server.

Later on I investigate the issue and fount that the connections of this webapp were building up at a fixed pase of n per redeployment, where n is the minimum idle connections I set up. Later check with our other webapps and found the same behaviour in one. They share in common the feature of multitenancy and the both use Hibernate's HikariCPConnectionProvider. Also worth mentioning that these apps have one DataSource for a particular database, and this one didn't stack up connections. 

Anyway, after checking a lot of resources, read many threads, post and docs, I came to three conclusions: 1) It is not Hikari's fault (By this I mean my config of it), 2) The server keeps the connections hsotage (If I remove the .war file from tomcat, the connections persisted. If I restarted the DB service, once it was up the connectoins were filled again, even if the .war file was not depoloyed) and 3) this only happens when we hot-redeploy (Which is our preffered way to do so).

I've been trying almost anything I can in the side of the code, even changin from HikariCPConnectionProvider to HikariDataSource (This was half a fix half more problems), register both of them as beans, extending from them to add a custom stop() method, manually looping through the collection of connections at contextDestroyed(), but so far no luck (I have a ServerFault post touching a bit more in detail all of this).

The only fix thus far is to stop hot-redeploying but I honestly think this shouldn't be an issue, so I look here hoping someone can guide me on this. I ended up going back to the ConnectionProvider method, so is there a way to properly close these connectoins? 

I'm on Hibernate 5.0.12 and HikariCP 4.0.3 (updating anything brakes too much of the apps to be considered, unfortunetly), using the right package for them (The one provided from HIbernate).

Also, if we should just stop using hot-redeloyments is perfectly valid too. I'm already thinking on how to change our server so it is not an issue (We have multiple apps per tomcat, not redundant atm).

Any help will be appreciated.

Thank you!