MemcachedClient keeps reconnecting to dead server instance

[Simon]

Hi,

I'm just testing the behaviour when a server node suddenly dies at run
time. What I'm getting is that the MemcachedClient keeps trying to
reconnect. All the subsequent calls seem to be blocked. The message
I'm getting is:

2009-08-10 15:43:35.916 WARN net.spy.memcached.MemcachedConnection:
Closing, and reopening {QA sa=localhost/127.0.0.1:11211, #Rops=0,
#Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0},
attempt 38.
2009-08-10 15:43:39.819 INFO net.spy.memcached.MemcachedConnection:
Reconnecting {QA sa=localhost/127.0.0.1:11211, #Rops=0, #Wops=0,
#iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
2009-08-10 15:43:39.820 INFO net.spy.memcached.MemcachedConnection:
Connection state changed for sun.nio.ch.SelectionKeyImpl@49a51bb9
2009-08-10 15:43:39.821 INFO net.spy.memcached.MemcachedConnection:
Reconnecting due to failure to connect to {QA sa=localhost/
127.0.0.1:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null,
toWrite=0, interested=0}
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:
574)
at net.spy.memcached.MemcachedConnection.handleIO
(MemcachedConnection.java:294)
at net.spy.memcached.MemcachedConnection.handleIO
(MemcachedConnection.java:192)
at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1444)

BTW, all I'm doing are "get" operations. There is no set/write.

[Jason Buberel]

I consider the reconnect-attempt-behavior a desirable feature, assuming that it does NOT block other connections from being used.

Or am I missing something?

-jason

--
Jason L. Buberel
ja...@buberel.org

[Simon]

Thanks for the reply, Jason.

Yeah, I guess that the reconnect behaviour is desirable. Here is what
I expect:

* I configure hibernate-memcached with spymemcached client and two
nodes, host1 and host2.
* I start the application which reads 50000 records with both host1
and host2 running.
* While the application is still running, I kill host1.
* I expect to see the error messages regarding reconnecting. But I
also expect that the application will keep reading.

But the reality is that the application just keeps trying to reconnect
to host1. I don't see any further read.

If I configure with two nodes but make only one of them available
BEFORE I start the application, it DOES behave as I expected: it'll
keep reading while outputs reconnecting messages.

[Ray Krueger]

I don't think it's blocking it will end up causing a cache miss before
it fails over. If you're sure it's blocking I'd post on the
spymemcached list.

Sent from my iPhone

[Simon]

OK, this is very interesting. I got the blocking retry problem when I
include the distributed jar file in the classpath. But if I use the
source code directly, it'll retry but not block.

On Aug 10, 5:49 pm, Ray Krueger <raykrue...@gmail.com> wrote:
> I don't think it's blocking it will end up causing a cache miss before  
> it fails over. If you're sure it's blocking I'd post on the  
> spymemcached list.
>
> Sent from my iPhone
>