In our application, we have Hystrix commands that call external services using an HTTP connection pool (Apache HttpClient 4.2.5). Sometimes these calls fail due to socket read timeouts, connection timeouts, etc. In this case, our getFallback() method in our command object attempts to abort the connection to return it to the pool.
What we're seeing in our logs is the connection pool reaching capacity over time:
From our logs (we dump the connection pool stats in our fallback):
Connection.Pool.Total.Stats=[leased: 200 pending: 0; available: 0; max: 200]
Are there scenarios in Hystrix where a command failure, either via a FAILURE, SHORT_CIRCUITED, or TIMEOUT event could fail to return the http connection back to the pool or are we simply using the wrong semantics when cleaning up a failed command?
-Mike