ClassCastException: Cannot be cast to BinaryWritable

[Han JU]

Hi,

I'm trying to use Spark with Thrift. I use EB's Thrift outputFormat like this:

LzoThriftB64LineOutputFormat.setClassConf(classOf[JUrlLink], job.getConfiguration)

test.saveAsNewAPIHadoopFile(output,

      keyClass = classOf[Void],

      valueClass = classOf[JUrlLink],

      outputFormatClass = classOf[LzoThriftB64LineOutputFormat[JUrlLink]],

      conf = job.getConfiguration)

Basically it just registers the Thrift class and uses the LzoThriftB64LineOutputFormat.

But I keep getting this exception, for both Block and Line output format:

JUrlLink cannot be cast to com.twitter.elephantbird.mapreduce.io.BinaryWritable

I'm using EB 3.0.8, it's a little bit old but it's required by Parquet.

Any suggestion? Thanks!