Spark Cassandra Connector with Scylla driver

[wang Wu]

Any one aware of a variant of spark-cassandra-connector built against Scylla Java driver?

Context: We write from Spark streaming to Scylla DB. We use sparl-cassandra-connector v 2.5.1

We assume that if we rebuild spak-casandra-connector to use Scylla driver, we will have performance gain. Currently spark-cassabdra-connector is using Java driver

val driverCore = "com.datastax.oss" % "java-driver-core-shaded" % DataStaxJavaDriver driverCoreExclude()
val driverMapper = "com.datastax.oss" % "java-driver-mapper-runtime" % DataStaxJavaDriver driverCoreExclude()

Is it possible and advised to change to Scylla driver? Something like

val driverCore = "com.scylladb" % "jscylla-driver-core" % XXX

[Piotr Jastrzębski]

Hi Wang Wu,

It is possible. We've done that in the past multiple times. All you have to do is to make sure that you use the right version of Scylla driver. For example if your connector was running with Datastax driver 4.8.0 then you should use Scylla driver 4.8.0-scylla-0.

I'm adding to the CC Moreno who has some experience running Spark connector with Scylla driver.

Please let me know if you have any questions.

Best regards,

Piotr

--
You received this message because you are subscribed to the Google Groups "DataStax Java Driver for Apache Cassandra User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-driver-us...@lists.datastax.com.
To view this discussion on the web visit https://groups.google.com/a/lists.datastax.com/d/msgid/java-driver-user/239a16f7-1e91-41aa-ac90-7402f0c78758n%40lists.datastax.com.