Hi,
I'm trying to use Spark with Thrift. I use EB's Thrift outputFormat like this:
LzoThriftB64LineOutputFormat.setClassConf(classOf[JUrlLink], job.getConfiguration)
test.saveAsNewAPIHadoopFile(output,
keyClass = classOf[Void],
valueClass = classOf[JUrlLink],
outputFormatClass = classOf[LzoThriftB64LineOutputFormat[JUrlLink]],
conf = job.getConfiguration)
Basically it just registers the Thrift class and uses the LzoThriftB64LineOutputFormat.
But I keep getting this exception, for both Block and Line output format:
JUrlLink cannot be cast to com.twitter.elephantbird.mapreduce.io.BinaryWritable
I'm using EB 3.0.8, it's a little bit old but it's required by Parquet.
Any suggestion? Thanks!