How to perform range query on composite index, plus filter by secondary index.

261 views
Skip to first unread message

Andy Miller

unread,
Dec 12, 2012, 5:13:07 PM12/12/12
to hector...@googlegroups.com
I am new to Cassandra, so please bear with me. I am hoping someone can steer me in the right direction. I have the following CF, although I’m happy to model differently if that helps to do what I want:

CREATE TABLE objects4 (
  user_id uuid,
  created_at timestamp,
  object_type varchar,
  ins_order int,
  some_other_text varchar,
  object_val varchar,
  PRIMARY KEY (user_id, created_at)
);
CREATE INDEX ON objects4 (object_type);
CREATE INDEX ON objects4 (some_other_text);

I am using hector-core 1.1-2 API. I moved to Cassandra 1.2.0-beta3, since it allows adding secondary indexes on tables with composite keys. In my final CF, I will have a few more non-indexed fields, so I’d prefer not to duplicate the data in multiple CFs, unless that’s the suggested way to do it.
The values for object_type and some_other_text have relatively low cardinality (100's or 1000s of distinct values), so as I understand it these are reasonable candidates for secondary indexes.
I want to perform the following 3 queries:

select * from objects4 where user_id = 'dd430330-c5d5-4f1e-a0b6-2f2e71de0ace' and object_type = 'loan_form';
Works in cqlsh, but I cannot figure out how to make this work with RangeSliceQuery.

select * from objects4 where user_id = 'dd430330-c5d5-4f1e-a0b6-2f2e71de0ace' and some_other_text = 'some_value’;
Works in cqlsh, but I cannot figure out how to make this work with RangeSliceQuery.

select * from objects4 where created_at < 1350171576000;
Works in cqlsh, but have to add “…allow filtering”, else get error.
I am able to do this using RangeSliceQuery (but perhaps it won’t perform well because “….it might involve data filtering…”?).

I’m trying to perform the first query as follows:
    Composite fld_name_object_type = new Composite();
    fld_name_object_type.add(0, 1350171576000L);   // Really I want all created_at values, but just to get something working, try this…
    fld_name_object_type.add(1, "object_type");

    RangeSlicesQuery<UUID, Composite, byte[]> rangeSlicesQuery = HFactory
      .createRangeSlicesQuery(contextKeyspace, UUIDSerializer.get(), CompositeSerializer.get(), BytesArraySerializer.get())
      .setColumnFamily("objects4")
      .setKeys(UUID.fromString("d2d174c1-80aa-4ef1-aaa2-7b447b0bcf78"), UUID.fromString("d2d174c1-80aa-4ef1-aaa2-7b447b0bcf78"))
      .addEqualsExpression(fld_name_object_type, "loan_form".getBytes())
      .setRange(null, null, false, Integer.MAX_VALUE)  // Get all
      .setRowCount(row_count);

If I remove the “addEqualsExpression”, the query works, but I cannot figure out how to add appropriate equals expression for “object_type”, as the column name includes the value of the created_at portion of the PK. The above isn’t even what I want, since I want all created_at values, but even the above returned 0 rows, although the query returned rows through cqlsh.

I also tried all 3 queries using cassandra.model.CqlQuery, but I can only get that to work for “select * from objects4”…any “where” clause I add causes exceptions on “execute” call (even though they work fine in cqlsh). For example:

CqlQuery cqlQuery = new CqlQuery(contextKeyspace, UUIDSerializer.get(), CompositeSerializer.get(), BytesArraySerializer.get());
cqlQuery.setQuery("select * from objects4 where user_id = 'dd430330-c5d5-4f1e-a0b6-2f2e71de0ace' and object_type = 'loan_form'");
QueryResult<CqlRows<UUID, Composite, byte[]>> result = cqlQuery.execute();
Exception in thread "main" me.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:unable to coerce 'object_type' to a  formatted date (long))
  at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:45)
  at me.prettyprint.cassandra.model.CqlQuery$1.execute(CqlQuery.java:130)
  at me.prettyprint.cassandra.model.CqlQuery$1.execute(CqlQuery.java:100)
  at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
  at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258)
  at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:97)
  at me.prettyprint.cassandra.model.CqlQuery.execute(CqlQuery.java:99)
  at com.west.testjson.TestHectorObjects4.executeGetAllRange(TestHectorObjects4.java:86)
  at com.west.testjson.TestHectorObjects4.execute(TestHectorObjects4.java:51)
  at com.west.testjson.TestJson.main(TestJson.java:181)
Caused by: InvalidRequestException(why:unable to coerce 'object_type' to a  formatted date (long))
  at org.apache.cassandra.thrift.Cassandra$execute_cql_query_result.read(Cassandra.java:33477)
  at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
  at org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql_query(Cassandra.java:1402)
  at org.apache.cassandra.thrift.Cassandra$Client.execute_cql_query(Cassandra.java:1388)
  at me.prettyprint.cassandra.model.CqlQuery$1.execute(CqlQuery.java:106)
  ... 8 more

Or:
cqlQuery.setQuery("select * from objects4 where user_id = 'dd430330-c5d5-4f1e-a0b6-2f2e71de0ace'");
Exception in thread "main" me.prettyprint.hector.api.exceptions.HectorException: java.lang.IllegalArgumentException
  at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:67)
  at me.prettyprint.cassandra.model.CqlQuery$1.execute(CqlQuery.java:130)
  at me.prettyprint.cassandra.model.CqlQuery$1.execute(CqlQuery.java:100)
  at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
  at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258)
  at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:97)
  at me.prettyprint.cassandra.model.CqlQuery.execute(CqlQuery.java:99)
  at com.west.testjson.TestHectorObjects4.executeGetAllRange(TestHectorObjects4.java:82)
  at com.west.testjson.TestHectorObjects4.execute(TestHectorObjects4.java:51)
  at com.west.testjson.TestJson.main(TestJson.java:180)
Caused by: java.lang.IllegalArgumentException
  at java.nio.Buffer.limit(Unknown Source)
  at me.prettyprint.hector.api.beans.AbstractComposite.getBytes(AbstractComposite.java:729)
  at me.prettyprint.hector.api.beans.AbstractComposite.getWithShortLength(AbstractComposite.java:736)
  at me.prettyprint.hector.api.beans.AbstractComposite.deserialize(AbstractComposite.java:708)
  at me.prettyprint.cassandra.serializers.CompositeSerializer.fromByteBuffer(CompositeSerializer.java:35)
  at me.prettyprint.cassandra.serializers.CompositeSerializer.fromByteBuffer(CompositeSerializer.java:17)
  at me.prettyprint.cassandra.model.HColumnImpl.getName(HColumnImpl.java:111)
  at me.prettyprint.cassandra.model.ColumnSliceImpl.getColumnByName(ColumnSliceImpl.java:45)
  at me.prettyprint.cassandra.model.CqlRows.<init>(CqlRows.java:40)
  at me.prettyprint.cassandra.model.CqlQuery$1.execute(CqlQuery.java:125)
  ... 8 more

Thanks.

Reply all
Reply to author
Forward
0 new messages