Question: Cassandra failure during read query at consistency LOCAL_ONE exception is thrown when querying Cassandra

8,799 views
Skip to first unread message

Abdulfattah Safa

unread,
Jan 19, 2017, 2:39:31 PM1/19/17
to DataStax Spark Connector for Apache Cassandra
For some queries, the following error appears:

Cassandra failure during read query at consistency LOCAL_ONE (1 responses were required but only 0 replica responded, 1 failed)

When check Cassandra log, the following error is thrown:
StorageProxy.java:1822 - Scanned over 100001 tombstones during query 'SELECT * FROM wiki.treated_article_posting WHERE language = en AND article_wikibase_item = Q4022382 LIMIT 5000' (last scanned row partion key was ((en), btw1965)); query aborted

I'm using Cassandra with Spark 2.1 with Java.
The data trying to read is from one Partition and is not that huge (<10K rows)

I know this question is a duplicate of others, but non of the solutions for the other questions worked with me.

Russell Spitzer

unread,
Jan 19, 2017, 2:45:03 PM1/19/17
to DataStax Spark Connector for Apache Cassandra
You have a Cassandra Server problem. Your Cassandra partition is full of tombstones. Lots and lots of tombstones. This is usually a sign that something has gone terribly wrong in the data modeling. To stop this kind of issue from killing the C* server, there was a limit set to stop any queries that read too many tombstones.

http://stackoverflow.com/questions/21755286/what-exactly-happens-when-tombstone-limit-is-reached

Basically the C* server is opting to stop rather than returning a value because it is afraid of OOMing or stalling out. You can manually adjust the limit in your C* yaml but having this many tombstones is usually a sign you need to rethink the write pattern.

--
You received this message because you are subscribed to the Google Groups "DataStax Spark Connector for Apache Cassandra" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spark-connector-...@lists.datastax.com.
--

Russell Spitzer
Software Engineer




DS_Sig2.png

Reply all
Reply to author
Forward
0 new messages