No companion ThriftStructCodec found for interface

15 views

Skip to first unread message

bs...@twitter.com

unread,

Mar 16, 2018, 6:22:30 PM3/16/18

to cascading-user

I am writing Scalding which includes a Scala class which is Scrooge generated (from Thrift). However once a while I am failing the job from the very beginning because of "cascading.flow.planner.PlannerException: could not build flow from assembly" exception.

Have you ever been this kind of behavior happening with Cascading not able to read companion ThriftStructCodec? What could be a possible solution for this?

I have definitely included the scala package for that thrift class.

Any help is appreciated, thank you!!!!
Caused by: cascading.flow.planner.PlannerException: could not build flow from assembly: [No companion ThriftStructCodec found for interface com.twitter.ads.spendserver.thriftscala.SpendServerEvent or its parents]
at cascading.flow.planner.FlowPlanner.handleExceptionDuringPlanning(FlowPlanner.java:578)
at cascading.flow.hadoop.planner.HadoopPlanner.buildFlow(HadoopPlanner.java:286)
at cascading.flow.hadoop.planner.HadoopPlanner.buildFlow(HadoopPlanner.java:80)
at cascading.flow.FlowConnector.connect(FlowConnector.java:459)
at com.twitter.scalding.ExecutionContext$class.buildFlow(ExecutionContext.scala:74)
at com.twitter.scalding.ExecutionContext$$anon$1.buildFlow(ExecutionContext.scala:165)
at com.twitter.scalding.Execution$EvalCache$$anon$2.go$1(Execution.scala:390)
at com.twitter.scalding.Execution$EvalCache$$anon$2.run(Execution.scala:413)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: No companion ThriftStructCodec found for interface com.twitter.ads.spendserver.thriftscala.SpendServerEvent or its parents
at com.twitter.scalding_internal.scrooge.ScroogeThriftStructSerializer$$anonfun$5.apply(ScroogeThriftStructSerializer.scala:48)
at com.twitter.scalding_internal.scrooge.ScroogeThriftStructSerializer$$anonfun$5.apply(ScroogeThriftStructSerializer.scala:48)
at scala.Option.getOrElse(Option.scala:121)
at com.twitter.scalding_internal.scrooge.ScroogeThriftStructSerializer$.findThriftStructCodecCompanion(ScroogeThriftStructSerializer.scala:47)
at com.twitter.scalding_internal.scrooge.ScroogeThriftStructSerializer$.constructCodec(ScroogeThriftStructSerializer.scala:57)
at com.twitter.scalding_internal.scrooge.ScroogeThriftStructSerializer$.binaryConverter(ScroogeThriftStructSerializer.scala:131)
at com.twitter.scalding_internal.multiformat.format.SchemeGen$$anon$3$$anonfun$apply$1.apply(SchemeGen.scala:78)
at com.twitter.scalding_internal.multiformat.format.SchemeGen$$anon$3$$anonfun$apply$1.apply(SchemeGen.scala:77)
at com.twitter.scalding_internal.source.LzoGenericScheme.sourceConfInit(LzoGenericScheme.scala:117)
at com.twitter.scalding_internal.source.LzoGenericScheme.sourceConfInit(LzoGenericScheme.scala:103)
at cascading.tap.Tap.sourceConfInit(Tap.java:183)
at cascading.tap.hadoop.Hfs.sourceConfInitComplete(Hfs.java:438)
at cascading.tap.hadoop.Hfs.applySourceConfInitIdentifiers(Hfs.java:426)
at cascading.tap.hadoop.Hfs.sourceConfInit(Hfs.java:404)
at com.twitter.scalding.ConfPropertiesHfsTap.sourceConfInit(HfsConfPropertySetter.scala:37)
at com.twitter.scalding.ConfPropertiesHfsTap.sourceConfInit(HfsConfPropertySetter.scala:26)
at com.twitter.scalding_internal.multiformat.tap.MultiFormatSourceTap$$anonfun$2.apply(MultiFormatSourceTap.scala:114)
at com.twitter.scalding_internal.multiformat.tap.MultiFormatSourceTap$$anonfun$2.apply(MultiFormatSourceTap.scala:102)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at com.twitter.scalding_internal.multiformat.tap.MultiFormatSourceTap.sourceConfInit(MultiFormatSourceTap.scala:102)
at com.twitter.scalding_internal.multiformat.tap.MultiFormatSourceTap.sourceConfInit(MultiFormatSourceTap.scala:36)
at cascading.flow.hadoop.HadoopFlowStep.initFromSources(HadoopFlowStep.java:349)
at cascading.flow.hadoop.HadoopFlowStep.getInitializedConfig(HadoopFlowStep.java:103)
at cascading.flow.hadoop.HadoopFlowStep.createFlowStepJob(HadoopFlowStep.java:208)
at cascading.flow.hadoop.HadoopFlowStep.createFlowStepJob(HadoopFlowStep.java:73)
at cascading.flow.planner.BaseFlowStep.getFlowStepJob(BaseFlowStep.java:768)
at cascading.flow.BaseFlow.initializeNewJobsMap(BaseFlow.java:1253)
at cascading.flow.BaseFlow.initialize(BaseFlow.java:214)
at cascading.flow.hadoop.planner.HadoopPlanner.buildFlow(HadoopPlanner.java:280)

Reply all

Reply to author

Forward

0 new messages