java.io.OptionalDataException trying to send a message to a remote actor

[debasish]

I am trying to send a message to a remote actor from a Play application (running Play 2.1.5, Akka 2.2.4 and Scala 2.10.0). The remote actor is on a system that runs Scala 2.10.3 and Akka 2.2.4. I get the following exception .. However if I switch back to Scala 2.10.0 in both the systems, things work ok.

12:22:33.807 [JobManagerService-akka.actor.default-dispatcher-4] ERROR akka.remote.EndpointWriter - AssociationError [akka.tcp://JobManagerService@localhost:9303] <- [akka.tcp://WebLayer@localhost:9302]: Error [null] [

java.io.OptionalDataException

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1349)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)

at scala.collection.immutable.$colon$colon.readObject(List.scala:366)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:979)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1873)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)

at scala.collection.immutable.$colon$colon.readObject(List.scala:366)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:979)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1873)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)

at scala.collection.immutable.$colon$colon.readObject(List.scala:366)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:979)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1873)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)

at scala.collection.immutable.$colon$colon.readObject(List.scala:366)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:979)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1873)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)

at scala.collection.immutable.$colon$colon.readObject(List.scala:366)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:979)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1873)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)

at akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

at scala.util.Try$.apply(Try.scala:161)

at akka.serialization.Serialization.deserialize(Serialization.scala:98)

at akka.remote.serialization.MessageContainerSerializer.fromBinary(MessageContainerSerializer.scala:58)

at akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

at scala.util.Try$.apply(Try.scala:161)

at akka.serialization.Serialization.deserialize(Serialization.scala:98)

at akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:771)

at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

at akka.actor.ActorCell.invoke(ActorCell.scala:456)

at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)

at akka.dispatch.Mailbox.run(Mailbox.scala:219)

at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:385)

at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

]

Suggestions ? Is this expected ?

Thanks.

- Debasish

[Björn Antonsson]

Hi Debasish,

It looks like there are some differences between the object getting serialized in 2.10.0 and 2.10.3. Do you know what object the remote system is trying to deserialize? Are you sending case classes? There might be some incompatibilities there.

There is no way to log it at the receiver, before it gets deserialized, but maybe you can log it at the sender by enabling:

akka.remote.log-sent-messages = on

B/

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 

Björn Antonsson

Typesafe – Reactive Apps on the JVM

twitter: @bantonsson

JOIN US. REGISTER TODAY!

Scala

Days

June 16th-18th,

Berlin

[Patrik Nordwall]

On Wed, Apr 2, 2014 at 4:23 PM, Björn Antonsson <bjorn.a...@typesafe.com> wrote:

Hi Debasish,

It looks like there are some differences between the object getting serialized in 2.10.0 and 2.10.3. Do you know what object the remote system is trying to deserialize? Are you sending case classes? There might be some incompatibilities there.

There is no way to log it at the receiver, before it gets deserialized, but maybe you can log it at the sender by enabling:

akka.remote.log-sent-messages = on

and

akka.loglevel=DEBUG

/P

[debasish]

I will try out these diagnostics. But this is a running application and runs fine with 2.10.0 in *all* components. Just changing the version of the target component to 2.10.3 causes this exception. Also all messages exchanged are case classes.

Thanks.

[Björn Antonsson]

Hi Debasish,

On 2 April 2014 at 19:20:47, debasish (ghosh.d...@gmail.com) wrote:

I will try out these diagnostics. But this is a running application and runs fine with 2.10.0 in *all* components. Just changing the version of the target component to 2.10.3 causes this exception. Also all messages exchanged are case classes.

That unfortunately sounds like an incompatibility in the way that some scala classes are serialized.

B/

[debasish]

Here are some more details .. The message that the sender side (the Play side) sends is the following case class named Job ..

Job(None,None,some dbimport name, DbImportJob,None,Some(testuser),None,None,Status(Waiting,Some(0.0),None),List(),None,List(DbImport(List(DbSource(emps,jdbc:mysql://127.0.0.1:3306/emps,Credentials(xxx,Some(yyy)),Some(Includes(List(employee, department))),List(),None,None,None,List())))),None,false)

In this case class there is a field named "operations" which is the highlighted text. I think what's happening is that Scala 2.10.0 serializes this in some way which Scala 2.10.3 cannot de-serialize. If I make this field List.empty, the message goes through.

If this is true, then I don't think we can use Scala 2.10.3 and Scala 2.10.0 even across different applications in a portable way.

Thanks.

[Björn Antonsson]

Hi Debasish,

That might be related to this list serialization issue that was "fixed" between 2.10.0 and 2.10.1

Would it be possible to change the list to some other collection?

B/

[debasish]

Let me see if List can be changed. It's difficult since it's a huge application and Job is one of the core abstractions. Moreover things should be ok when we migrate fully to 2.10.3, for which the major roadblock is Play. Play 2.2.2 is a significantly different version that 2.1.5 and we need to budget for that appropriately. 

Anyway thanks for the help .. at least we have diagnosed the problem now ..

[debasish]

Tried with 2.10.1 .. no luck :-(

-- 

Björn Antonsson

<a href="http://typesafe.com/" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75htt

...