Failed reads with Regular Secondary Index on Collection Column + SASI index on static column

[Matthew Tamayo]

Table: ( partition_key, clustering_column ) enum_column, collection_column 

Three different queries:

1. select * partition_key from table where clustering_column = <value> AND enum_column = <enum value> 
2. select * partition_key from table where clustering_column = <value>  AND collection_column contains <elem>
3. select * partition_key from table where clustering_column = <value> AND enum_column = <enum value> AND collection_column contains <elem>

Queries 1 and 2 work just fine. Query 3 seems to blow up on the server, even though it is technically allowed. This is probably more of a server question, but what's the best way to report a bug? It seems to be that sasi and regular indices don't like being used in the same query.

java.lang.RuntimeException: java.lang.ClassCastException: org.apache.cassandra.index.internal.composites.CollectionKeyIndex cannot be cast to org.apache.cassandra.index.sasi.SASIIndex

at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2490) ~[apache-cassandra-3.9.jar:3.9]

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_45]

at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) [apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) [apache-cassandra-3.9.jar:3.9]

at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]

Caused by: java.lang.ClassCastException: org.apache.cassandra.index.internal.composites.CollectionKeyIndex cannot be cast to org.apache.cassandra.index.sasi.SASIIndex

at org.apache.cassandra.index.sasi.plan.QueryController.getIndex(QueryController.java:94) ~[apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.index.sasi.plan.Operation.analyzeGroup(Operation.java:281) ~[apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:432) ~[apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.index.sasi.plan.QueryPlan.analyze(QueryPlan.java:57) ~[apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.index.sasi.plan.QueryPlan.execute(QueryPlan.java:68) ~[apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.index.sasi.SASIIndex.lambda$searcherFor$2(SASIIndex.java:286) ~[apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.index.sasi.SASIIndex$$Lambda$330/1572566461.search(Unknown Source) ~[na:na]

at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:398) ~[apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1801) ~[apache-cassandra-3.9.jar:3.9]

at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2486) ~[apache-cassandra-3.9.jar:3.9]

... 5 common frames omitted

[DuyHai Doan]

Right now, it is not possible to MIX regular 2nd index AND SASI index

The query optimizer from SASI will not use regular secondary index

[Matthew Tamayo]

Thanks!

Figured as much. We just made the SASI index a regular index for now.

[Ho Chung]

We are still having trouble with query 3. Following DuyHai's advice we used a regular secondary index on the collection column, and it's still failing.

Setup:

Table: ( partition_key, clustering_column ) (secondary index) enum_column, (secondary index) collection_column 

After some effort we traced down the problem to this code and did some testing. We are using the cassandra java driver.

elements.stream()
	.map( element ->
	[bind a prepared statement that restricts all the clustering columns, restrict the enum column, and restrict the collection column to contain "element" ) )
	.map( session::executeAsync )
	.map( ResultSetFuture::getUninterruptibly )
	.peek( rs -> {
	// double check the result set we are getting back
	int bad = 0;
	for(Row row : rs ){
	if( [clustering columns of row satisfy the specified restriction] ){
	System.out.println( "LIFE'S GOOD" );
	} else {
	bad++;
	System.out.println( "LIFE SUCKS" );
	}
	}
	System.out.println( "This query fails " + bad + " times." );
	})

It fails randomly. Whenever it fails, (based on the count in the snippet) it seems like the query would return all the rows in our table.

Any idea on why this is happening? We suspect that secondary index on both static column and collection column are causing weird issues.