Recover from redis connection timeout or redis server reset

[Zoz]

I wanted my client to be able to recover from redis server closed
connections, network problems, and redis server resets easily. To
have my client auto recover I used "tedpennings" JedisProxy idea to
wrap the calls to Jedis in a proxy that handles the exception stuff.
This is working pretty well.
But one problem I am having is that to use a proxy you need an
interface and the JedisCommands interface is missing some commands
like BLPOP, MULTI, PIPELINED, INFO etc.

On a more general note, we might want to think how this issue should
be handled in general. What I am doing is a hack, but I don't see a
better way. I think most people using Jedis from a web server
environment will want to do something like this or perform exception
handling and retry logic on every call to jedis (yuk).
The basic idea of the proxy is if a jedis command fails on a pooled
jedis connection then call returnBrokenResource, get a new jedis from
pool, and try the command again.

Proxy code snipit:

@Override
public Object invoke(Object proxy, Method m, Object[] args) throws
Throwable {
Object result = null;
int MAX_BROKEN_POOL_RESOURCES = 100;
int i = 0;
boolean success = false;
while (i < MAX_BROKEN_POOL_RESOURCES && !success) {
i++;
try {
result = m.invoke(jedis, args);
success = true;
} catch (InvocationTargetException e) {
Logger.info("Jedis Pool Problem: attempt: " + i + " "
+ e.getTargetException().getMessage());
jedisPool.returnBrokenResource(jedis);
jedis = obtainJedis(); // calls
jedisPool.getResource();
}
}
if (!success) {
Logger.error("RedisSession, unable to get resource from
pool.");
throw new Exception("RedisSession, unable to get resource
from pool.");
}
return result;
}

[Jonathan Leibiusky]

Hi! As of jedis 1.5.0 RC2 this is not necessary since we are using the
apache pool, where you can tell the pull to test connection on borrow.

Jonathan

[Zoz]

Thanks Jonathan for the clarification.

I am using 1.5.0 RC2, I just tested test on borrow and that works
great. I had missed that. I guess I will keep my proxy to avoid the
PING round trip, but from what I can tell the PING takes very little
time.

So when using "test on borrow" do you ever have to call
returnBrokenResource or is that done by the test or borrow code?

Zoz

[Zoz]

I just looked at the apache code and if "test on borrow" is on it
loops through broken connections invalidating them until it finds a
good one. So, I don't think I need to call returnBrokenResource if
"test on borrow" is on. sweet. I think I can live with the extra
PING's.

Thanks Jonathan, I am enjoying the library,
Zoz

[Jonathan Leibiusky]

Yes, you are right. I'm glad it works for you.

On 12/8/10, Zoz <zoz...@gmail.com> wrote: