HList Macro for enforcing CQL typesafety?

[Andy Polack]

Greetings,

I'm an intermediate Scala developer with just enough knowledge to get himself intro trouble thinking big thoughts and trying (perhaps) silly things.

Recently I found myself frustrated by the lack of typesafety in Datastax Cassandra driver.  There are Scala alternatives that seek to alleviate this (Phantom comes to mind: https://github.com/websudos/phantom) but generally they lag behind the "latest and greatest" features added by Datastax into the CQL standard. 

So the basic structure for making a (repeatable) Cassandra query in the Datastax driver is designed to look very much like JDBC - generally something like this (ignoring for now the reactive/async model which is preferable and that preparing a statement multiple times is an anti-pattern):

val statement: PreparedStatement = session.prepare("INSERT INTO TestKeyspace.TestTable (a, b, c) VALUES (?, ?, ?)")
val result: ResultSet = session.execute(statement.bind(1, List(1, 2, 3).toJava, "hello world"))

The 'bind(...)' varargs call on the PreparedStatement must disaggregate up to the highest type (since the Datastax driver is Java) so it's just a varargs of Java "Object" types.

It is possible to inspect a constructed statement and get a list of Class types back representing the class types for each bound argument.  I assume then, that for a PreparedStatement such as the one above, target type information may be obtained at compile time.

So this brings us to Shapeless and HList.  I wondered to myself - "would it be possible to define a compile-time macro to in-line code to force typesafety when binding a statement?"

This lead me to a basic structure for my target macro that looks something like this:

import com.datastax.driver.core.{BoundStatement, PreparedStatement}

import shapeless.HList

import scala.reflect.macros._

import scala.language.experimental.macros

/**
 * The idea is to try to build a Scala macro that will leverage Shapeless HList to provide a typesafe compile-time
 * alternative to the very un-typesafe Datastax cassandra driver PreparedStatement implementation.
 *
 * I'm going to try to achieve this by passing the macro a PreparedStatement and then trying to generate code
 * that returns a first-class function mapping an instance of the provided HList subtype to a BoundStatement
 * (by iterating over the HList and using it as the varargs to the .bind(...) method on PreparedStatement).
 *
 * An example may serve better... so:
 *
 * Target Usage:
 *
 * cqlsh> CREATE TABLE TestKeyspace.TestTable (a int, b list<text>, c double)
 * 
 * import scala.collection.JavaConverters._
 * import scala.collection.JavaConversions._
 * val typesafe  = CQL_TYPESAFE_FUNC(session.prepare("""INSERT INTO TestKeyspace.TestTable (a, b, c) VALUES (?, ?, ?)"""))
 * val result: ResultSet = session.execute(typesafe(new java.lang.Integer(1) :: List("a", "b", "c").asJava :: new java.lang.Double(3.0) :: HNil))
 *
 * Ideally typesafety is checked at compile time, an appropriate HList type is generated based on inspecting the
 * provided PreparedStatement instance, and the function returned by the macro then requires an HList of that type
 * to produce the corresponding BoundStatement.  This would eliminate the constant runtime errors caused by
 * forgetting to call ".toJava" everywhere on the various types - those errors would be moved to compile-time issues.
 */

object CQL_TYPESAFE_FUNC {

  def apply(statement: PreparedStatement): _ <: HList => BoundStatement = macro impl

  def impl(c: whitebox.Context)(statement: c.Expr[PreparedStatement]): c.Expr[_ <: HList => BoundStatement] = {
    import c.universe._
    //TODO: ??? (here be dragons!)
  }
}

And that's where I get stuck.  My lack of further knowledge about how HList and Macros work gets the better of me.

So - my question is two-fold:

1. Given the above information, do you believe it may be possible to introduce a macro to provide compile-time type safety on 'bind(...)' through a technique like the one sketched out above - i.e. by producing a function which requires a subtype of HList to produce a BoundStatement? (or have I grossly misunderstood what's possible with Scala macros?)
2. What resources would you recommend for me to get smarter on how I might accomplish what I've set out to attempt? (provided, of course, that the answer to no. 1 above is in the affirmative)

I've done some google-ing around but the best I've come up with thus-far is the scala documentation itself and a whitepaper on Quasiquotes. 

Thanks ahead of time for any insight you may be able to provide.

Kind Regards,
Andy

[Andy Polack]

I've also considered an alternative version since, as it currently stands I believe I would need a running Cassandra instance at compile time to allow the PreparedStatement to resolve its type information properly (not sure how it would do so otherwise, but need to dig more into their code to find out).

The alternative would be to add a type parameter to the 'apply' def - i.e. def apply[T <: HList] and corresponding parameter to the 'impl' def.  This would make it so that the developer had to state the proper type information once (it would no longer be inferred by the PreparedStatement's database introspection but stated instead as a HList type on 'apply'), and thereafter echo that type when calling the returned first-class function. 

Still - I'm not sure if that makes any more sense or if I'm attempting something that someone with more Shapeless and Macro experience would clearly know is impossible.  Thoughts and criticisms certainly welcome!

Thanks,
Andy

[Dave Gurnell]

Hi Andy,

I have put some vague thoughts below . My shapeless-fu is weak so there are parts that I don't know how to fill in yet. Also I'm writing this on my phone so apologies for typos.

You could write a string interpolation macro to turn the query string into some sort of QueryBuilder object with the right type parameter.

So the effect would be:

cql"""select..."""

compiling to:

QueryBuilder[???]("""select ...""")

Here's an example of a string interpolation macro for producing a musical score from guitar tablature. The macro literally scans through the characters in the tab and accumulates Trees representing Score objects:

https://github.com/underscoreio/compose/blob/master/src/main/scala/compose/tab/TablatureSyntax.scala

Your macro could be loads simpler than this. Just count the ? symbols and create a QueryBuilder that expects the corresponding number of parameters.

There's more info on macros here. I don't cover string interpolation macros but the other content of the talk is applicable:

http://underscore.io/training/courses/essential-macros/

The next step would be to provide the type information you need. I'm a shapeless newbie but perhaps others can chime in here. You'd need to give QueryBuilder some type parameter representing the number of placeholders in the query. I don't know if this would be a Nat or some sort of HList. You could generate either in your macro.

I believe HLists are covariant so you could perhaps build a type like 'Any :: Any ... HNil' and require the user to restrict it later.

You then give QueryBuilder a method called that allows the user to superimpose their own type information. The approximate code would be:

queryBuilder.as[A :: B :: C :: HNil]

creating a value of type:

Query[A :: B :: C :: HNil]

You'd need some implicit arguments that prove that the type parameters on the builder and the method match up. You could also use a Generic to convert to/from an arbitrary user-supplied type to an HList of the right type.

Finally, you can add a 'bind' method to Query that accepts a value of the right type and turns it into a List[Any] or whatever the underlying query API requires:

query.bind(a :: b :: c :: HNil).run

Hope that helps. Hopefully other list members can chime in if with details or corrections.

Dave

--
You received this message because you are subscribed to the Google Groups "Shapeless Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shapeless-de...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Miles Sabin]

Hi Andy,

I think the first thing you should probably do is take a look at Rob
Norris's Doobie,

https://github.com/tpolecat/doobie

Cheers,


Miles

> --
> You received this message because you are subscribed to the Google Groups
> "Shapeless Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to shapeless-de...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Miles Sabin
tel: +44 7813 944 528
skype: milessabin
gtalk: mi...@milessabin.com
http://milessabin.com/blog
http://twitter.com/milessabin

[Andy Polack]

Dave & Miles - thank you both for your detailed and thoughtful responses! 

I will most certainly dig into the recommended materials :-)

If I make any progress I'll be sure to share it here.

Kind Regards,
Andy