NoHostAvailableException happen and cannot recover

244 views
Skip to first unread message

강인

unread,
Jun 25, 2015, 3:54:32 AM6/25/15
to java-dri...@lists.datastax.com
Hello, I'm using java driver from maven

<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>2.1.6</version>

I got unwanted exception

my application server run correctly at first and meet this situation

2015/06/25 15:27:37.221 DEBUG (Connection.java:618) - Connection[/192.168.0.202:9042-13, inFlight=1, closed=true] closing connection
2015/06/25 15:27:37.221 DEBUG (Connection.java:655) - Not terminating Connection[/192.168.0.202:9042-13, inFlight=1, closed=true]: there are still pending requests
2015/06/25 15:27:37.222 DEBUG (Connection.java:618) - Connection[/192.168.0.202:9042-14, inFlight=0, closed=true] closing connection
2015/06/25 15:27:37.222 DEBUG (Cql3Table.java:412) - CQL CUD ok   : 
2015/06/25 15:27:37.222 DEBUG (Cql3Table.java:404) - CQL CUD try  : 
2015/06/25 15:27:37.223 DEBUG (RequestHandler.java:154) - Error querying /192.168.0.202:9042, trying next host (error is: com.datastax.driver.core.ConnectionException: [/192.168.0.202:9042] Pool is CLOSING)
2015/06/25 15:27:37.223 ERROR (Cql3Table.java:418) - CQL CUD fail : 
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /192.168.0.202:9042 (com.datastax.driver.core.ConnectionException: [/192.168.0.202:9042] Pool
 is CLOSING))
        at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
        at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:291)
        at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:205)
        at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
        at com.nzincorp.zinny.db.cassandra2.Cql3Table.executeCqlCUD(Cql3Table.java:409)
        at com.nzincorp.zinny.db.cassandra2.Cql3Table.insert(Cql3Table.java:261)
        at com.nzincorp.zinny.db.cassandra2.Cql3Table.insertColumns(Cql3Table.java:632)
        at com.nzincorp.zinny.db.cassandra2.Cql3Table.insertColumns(Cql3Table.java:619)
        at com.nzincorp.zinny.profile.dao.impl.ProfileDaoImpl.createPlayerV2(ProfileDaoImpl.java:539)
        at com.nzincorp.zinny.profile.dao.impl.ProfileDaoImpl.createPlayer(ProfileDaoImpl.java:466)
        at com.nzincorp.zinny.profile.api.v2.player.AdapterPlayerProfileFastApi.create(AdapterPlayerProfileFastApi.java:280)
        ... 16 more
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /192.168.0.202:9042 (com.datastax.driver.core.ConnectionException: [/192.168.0.202:9042] Pool
 is CLOSING))
        at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:216)
        at com.datastax.driver.core.RequestHandler.access$900(RequestHandler.java:45)
        at com.datastax.driver.core.RequestHandler$SpeculativeExecution.sendRequest(RequestHandler.java:276)
        at com.datastax.driver.core.RequestHandler.startNewExecution(RequestHandler.java:118)
        at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:94)
        at com.datastax.driver.core.SessionManager.execute(SessionManager.java:559)
        at com.datastax.driver.core.SessionManager.executeQuery(SessionManager.java:598)
        at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:97)
        ... 24 more


and after below repeatedly this error happen

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)
        at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
        at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:291)
        at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:205)
        at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
        at com.nzincorp.zinny.profile.dao.impl.ProfileDaoImpl.getAppInfoAll(ProfileDaoImpl.java:1556)
        at com.nzincorp.zinny.profile.api.v1.AdapterProfileFetchApiV1.getAppInfoAll(AdapterProfileFetchApiV1.java:61)
        at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.nzincorp.zinny.command.CommonZinnyCommand.invokeMethod(CommonZinnyCommand.java:153)
        at com.nzincorp.zinny.command.CommonZinnyCommand.doJob(CommonZinnyCommand.java:106)
        at com.nzincorp.zinny.command.ZinnyCommand.run(ZinnyCommand.java:145)
        at com.nzincorp.zinny.command.ZinnyCommand.run(ZinnyCommand.java:25)
        at com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:1263)
        at com.netflix.hystrix.HystrixCommand.access$2200(HystrixCommand.java:97)
        at com.netflix.hystrix.HystrixCommand$5.call(HystrixCommand.java:1169)
        at com.netflix.hystrix.strategy.concurrency.HystrixContextCallable.call(HystrixContextCallable.java:45)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)
        at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:216)
        at com.datastax.driver.core.RequestHandler.access$900(RequestHandler.java:45)
        at com.datastax.driver.core.RequestHandler$SpeculativeExecution.sendRequest(RequestHandler.java:276)
        at com.datastax.driver.core.RequestHandler.startNewExecution(RequestHandler.java:118)
        at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:94)
        at com.datastax.driver.core.SessionManager.execute(SessionManager.java:559)
        at com.datastax.driver.core.SessionManager.executeQuery(SessionManager.java:598)
        at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:97)
        ... 18 more

and at that time, I can find this log and cassandra works fine.

2015/06/25 15:30:20.504 DEBUG (Connection.java:1011) - Connection[/192.168.0.202:9042-15, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
2015/06/25 15:30:20.505 DEBUG (Connection.java:1082) - Connection[/192.168.0.202:9042-15, inFlight=0, closed=false] heartbeat query succeeded

What was wrong?

Olivier Michallat

unread,
Jun 25, 2015, 10:19:37 AM6/25/15
to java-dri...@lists.datastax.com
Looks like the driver temporarily lost connection with the host, and was able to reconnect.

Do you see other messages in the logs before 15:27:37 that would explained why the connection got closed in the first place?

--

Olivier Michallat

Driver & tools engineer, DataStax


To unsubscribe from this group and stop receiving emails from it, send an email to java-driver-us...@lists.datastax.com.

이강인

unread,
Jun 25, 2015, 11:49:13 AM6/25/15
to java-dri...@lists.datastax.com

There are no other strange logs

Reply all
Reply to author
Forward
0 new messages