Is google-guice API version 4.0 is serialized ???

[Monika Chaudhary]

Hi,

I am a newbie for google-guice API. I have an assignment in which I have to make use of google-guice API in Apache spark. When I tried I got following Exception:

15/11/26 16:57:39 ERROR main: Task not serializable

org.apache.spark.SparkException: Task not serializable

at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:166)

at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:158)

at org.apache.spark.SparkContext.clean(SparkContext.scala:1622)

at org.apache.spark.rdd.RDD.map(RDD.scala:286)

at org.apache.spark.api.java.JavaRDDLike$class.map(JavaRDDLike.scala:81)

at org.apache.spark.api.java.JavaRDD.map(JavaRDD.scala:32)

at com.data.core.ExtractorEtc.run(ExtractorEtc.java:58)

at com.data.core.App.run(App.java:128)

at com.data.encycle.EApp.main(EApp.java:46)

Caused by: java.io.NotSerializableException: com.google.inject.internal.InjectorImpl

Serialization stack:

- object not serializable (class: com.google.inject.internal.InjectorImpl, value: Injector[bindings=[ProviderInstanceBinding[key=Key[type=com.google.inject.Injector, annotation=[none]], source=[unknown source], scope=Scopes.NO_SCOPE, provider=Provider<Injector>], ProviderInstanceBinding[key=Key[type=java.util.logging.Logger, annotation=[none]], source=[unknown source], s.... 

does guice API supports parallel processing?? 

Please suggest...

[Puneet Lakhina]

It appears like you are trying to use an object in spark that holds onto a reference of guice injector either directly or transitively. My limited knowledge of spark tells me that any class types (either explicit or anonymous inner class types) you use in spark might need to be serialized and sent across to nodes in your cluster. You should look at the types you are using with spark operations and make sure you are not referencing the injector from any of those types.

--
You received this message because you are subscribed to the Google Groups "google-guice" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-guice...@googlegroups.com.
To post to this group, send email to google...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-guice.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-guice/d226b473-6690-450b-adad-d5d1eeffdfdc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.