me.prettyprint.hector.api.exceptions.HTimedOutException

328 views
Skip to first unread message

SunDance

unread,
Nov 19, 2012, 1:05:25 PM11/19/12
to hector...@googlegroups.com
Hello,
We are using hector-core1.1-1 with Cassandra 1.1.5 (3 nodes).
We are inserting batches in Cassandra using me.prettyprint.hector.api.mutation.Mutator.execute()
At times we get HTimeoutExceptions (see below).
Question:
1. Does this means we lost data? (i.e no insertion was made in Cassandra and we just discarded the data?)
2. How should our client code handle this?  (they are RunTime exceptions so technically they shouldn't be catched)
3. In the logs they appear with a WARN. Does this mean that the underlying Hector code recovered from it? or Does it mean that the Hector code tried a number of times and failed? (in which case shouldn't it be reported as an ERROR in the logs?).
4. As I understand the mechanism, Hector operates in failover which for us means: it tries to issue a request to one node a configurable number of times (sleeping in the middle for a configurable amount of time) and if it doesn't manage with that node it tries the next node, and so on until it exhausted all nodes it knows of and then it throws the HTimeoutExceptions to our client code. Is this an accurate description? If yes how can we configure the number of times it tries per node and how much it sleeps in between?

Cheers,
Alex

WARN  HConnectionManager - Could not fullfill request on this host CassandraClient<pc-tdq-bst-05:9160-25>
WARN  HConnectionManager - Exception:
me.prettyprint.hector.api.exceptions.HTimedOutException: TimedOutException()
        at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:35)
        at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:264)
        at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:97)
        at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:243)
        at ch.cern.pbeast.insertion.CassandraDBClient.executeBatchInsert(CassandraDBClient.java:3293)
        at ch.cern.pbeast.insertion.FilteringTaskNew.run(FilteringTaskNew.java:408)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: TimedOutException()
        at org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:19495)
        at org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:1035)
        at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:1009)
        at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:246)
        at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:243)
        at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
        at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258)
        ... 7 more

Bibin t john

unread,
Jan 13, 2014, 11:13:55 PM1/13/14
to hector...@googlegroups.com
Do you know what is the rootcause of this issue. i am also getting same issue and my loader process keep on failing with same error.
below are my configuration.
hector-core-1.1-1.jar
cassandra 1.2.5
8 node cluster
32GB JVM
replication Factor=3
read consistency=QUORUM
write consistency=QUORUM

Any help will be greatly appreciated.
Reply all
Reply to author
Forward
0 new messages