Exception when remote deploying in code

[Fredrik Ludvigsson]

Hi

I am trying to do a simple master slave scenario where the master
remotely deploys its workers to a remote node. I have based my code on
what is written in the documentation. I am using java and Akka 2.0.
The slave node just starts an akka system with remoting enabled and
the code that is run on the master is the following:

"
Address addr = new Address("akka", AkkaConstants.AKKA_SYSTEM_NAME,
"127.0.0.1", 2553);
ActorRef ref = context().system().actorOf(new
Props(CommandWorker.class).withDeploy(new Deploy(new
RemoteScope(addr))));
"
This code is executed within an actor as an action in response to an
incoming message. I am getting an java.io.NotSerializableException
referring to the akka.remote.netty.NettyRemoteTransport class. To me
it seems odd that this class should be serialized at all or what am I
missing / doing wrong here?

Full log from master with some debugging enabled:

[DEBUG] [03/06/2012 16:46:24.826] [main] [EventStream(akka://
Itemizer)] logger log1-Logging$DefaultLogger started
[DEBUG] [03/06/2012 16:46:24.832] [main] [EventStream(akka://
Itemizer)] Default Loggers started
[DEBUG] [03/06/2012 16:46:25.8] [main]
[akka.serialization.Serialization(akka://Itemizer)] Using
serializer[akka.serialization.JavaSerializer] for message
[akka.event.Logging$Info]
[INFO] [03/06/2012 16:46:25.9] [main] [ActorSystem(Itemizer)] REMOTE:
RemoteServerStarted@akka://Item...@127.0.0.1:2552
[DEBUG] [03/06/2012 16:46:25.14] [Itemizer-akka.actor.default-
dispatcher-2] [akka.serialization.Serialization(akka://Itemizer)]
Using serializer[akka.serialization.JavaSerializer] for message
[akka.remote.RemoteActorRefProvider$$anonfun$1]
[INFO] [03/06/2012 16:46:25.20] [main] [ActorSystem(Itemizer)]
Booting..
[DEBUG] [03/06/2012 16:46:25.26] [Itemizer-akka.actor.default-
dispatcher-3] [akka.serialization.Serialization(akka://Itemizer)]
Using serializer[akka.serialization.JavaSerializer] for message
[akka.actor.Props$$anonfun$$init$$2]
[INFO] [03/06/2012 16:46:25.31] [main] [ActorSystem(Itemizer)]
Finished booting
[DEBUG] [03/06/2012 16:46:25.37] [main]
[akka.serialization.Serialization(akka://Itemizer)] Using
serializer[akka.serialization.JavaSerializer] for message
[com.itemizer.framework.actor.message.ConfigurationManagerDiscovered]
[INFO] [03/06/2012 16:46:25.67] [Itemizer-akka.actor.default-
dispatcher-2] [akka://Itemizer/user/systemManager] Received message:
com.itemizer.framework.actor.message.ConfigurationManagerDiscovered@1615f43d
[DEBUG] [03/06/2012 16:46:25.68] [Itemizer-akka.actor.default-
dispatcher-2] [akka.serialization.Serialization(akka://Itemizer)]
Using serializer[akka.serialization.JavaSerializer] for message
[com.itemizer.framework.actor.message.RegisterSystem]
[INFO] [03/06/2012 16:46:25.70] [Itemizer-akka.actor.default-
dispatcher-3] [akka://Itemizer/user/configurationManager] Received
message: com.itemizer.framework.actor.message.RegisterSystem@591913b3
[INFO] [03/06/2012 16:46:25.76] [Itemizer-akka.actor.default-
dispatcher-3] [akka://Itemizer/user/configurationManager] System
registered: akka://Itemizer/user/systemManager
[DEBUG] [03/06/2012 16:46:25.77] [Itemizer-akka.actor.default-
dispatcher-3] [akka.serialization.Serialization(akka://Itemizer)]
Using serializer[akka.serialization.JavaSerializer] for message
[com.itemizer.framework.actor.message.SystemRegistrationStatus]
[DEBUG] [03/06/2012 16:46:25.83] [Itemizer-akka.actor.default-
dispatcher-3] [akka.serialization.Serialization(akka://Itemizer)]
Using serializer[akka.serialization.JavaSerializer] for message
[com.itemizer.framework.actor.message.StartSystem]
[INFO] [03/06/2012 16:46:25.84] [Itemizer-akka.actor.default-
dispatcher-3] [akka://Itemizer/user/systemManager] Received message:
com.itemizer.framework.actor.message.StartSystem@3bddf948
[DEBUG] [03/06/2012 16:46:25.85] [Itemizer-akka.actor.default-
dispatcher-3] [akka.serialization.Serialization(akka://Itemizer)]
Using serializer[akka.serialization.JavaSerializer] for message
[akka.actor.CreateRandomNameChild]
[DEBUG] [03/06/2012 16:46:25.97] [Itemizer-akka.actor.default-
dispatcher-2] [RemoteActorRefProvider(null)] [akka://Itemizer/]
Instantiating Remote Actor [akka://Item...@127.0.0.1:2553/remote/
Item...@127.0.0.1:2552/user/$a]
[DEBUG] [03/06/2012 16:46:25.105] [Itemizer-akka.actor.default-
dispatcher-2] [RemoteClient(akka://Itemizer)] Starting remote client
connection to [akka://Item...@127.0.0.1:2553|/127.0.0.1]
[DEBUG] [03/06/2012 16:46:25.229] [Itemizer-akka.actor.default-
dispatcher-2] [akka.serialization.Serialization(akka://Itemizer)]
Using serializer[akka.serialization.JavaSerializer] for message
[akka.remote.RemoteClientStarted]
mar 06, 2012 4:46:25 EM org.jboss.netty.channel.StaticChannelPipeline
WARNING: An exception was thrown by a user handler while handling an
exception event ([id: 0x4d1650df, /127.0.0.1:50934 => /127.0.0.1:2553]
EXCEPTION: java.io.NotSerializableException:
akka.remote.netty.NettyRemoteTransport)
java.io.NotSerializableException:
akka.remote.netty.NettyRemoteTransport
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.writeObject(Unknown Source)
at akka.serialization.JavaSerializer$$anonfun$toBinary$1.apply$mcV
$sp(Serializer.scala:114)
at akka.serialization.JavaSerializer$$anonfun$toBinary
$1.apply(Serializer.scala:114)
at akka.serialization.JavaSerializer$$anonfun$toBinary
$1.apply(Serializer.scala:114)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
at akka.serialization.JavaSerializer.toBinary(Serializer.scala:114)
at akka.serialization.Serialization.serialize(Serialization.scala:62)
at akka.dispatch.Envelope.<init>(AbstractDispatcher.scala:27)
at akka.actor.ActorCell.tell(ActorCell.scala:475)
at akka.actor.LocalActorRef.$bang(ActorRef.scala:322)
at akka.event.EventStream.publish(EventStream.scala:37)
at akka.event.EventStream.publish(EventStream.scala:23)
at akka.event.SubchannelClassification$$anonfun$publish
$1.apply(EventBus.scala:172)
at akka.event.SubchannelClassification$$anonfun$publish
$1.apply(EventBus.scala:172)
at scala.collection.immutable.Set$Set1.foreach(Set.scala:86)
at akka.event.SubchannelClassification$class.publish(EventBus.scala:
172)
at akka.event.EventStream.publish(EventStream.scala:23)
at akka.remote.RemoteTransport.notifyListeners(RemoteTransport.scala:
193)
at akka.remote.netty.ActiveRemoteClient.notifyListeners(Client.scala:
121)
at
akka.remote.netty.ActiveRemoteClientHandler.exceptionCaught(Client.scala:
317)
at
org.jboss.netty.handler.timeout.IdleStateAwareChannelHandler.handleUpstream(IdleStateAwareChannelHandler.java:
43)
at
org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.run(ChannelUpstreamEventRunnable.java:
44)
at
org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor
$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:312)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

[DEBUG] [03/06/2012 16:46:25.239] [Itemizer-6]
[akka.serialization.Serialization(akka://Itemizer)] Using
serializer[akka.serialization.JavaSerializer] for message
[akka.remote.RemoteClientConnected]
[DEBUG] [03/06/2012 16:46:25.243] [Itemizer-6]
[akka.serialization.Serialization(akka://Itemizer)] Using
serializer[akka.serialization.JavaSerializer] for message
[akka.remote.RemoteClientError]

[√iktor Ҡlang]

You're somehow trying to send a "akka.remote.RemoteClientStarted" message over the wire.

Cheers,

√

 

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
Viktor Klang

Akka Tech Lead

Typesafe - The software stack for applications that scale

Twitter: @viktorklang

[Fredrik Ludvigsson]

Ok, this is not something I am doing explicity. I changed the code so
that the only thing I am doing in master after instantiating the akka
system is try to remote deploy an actor and I still get the exception.

My configurationfile looks like this:
akka {
loglevel = DEBUG
debug {
lifecycle = on
}

actor {
provider = "akka.remote.RemoteActorRefProvider"
serialize-messages = on
serialize-creators = on
}

remote {
log-received-messages = on

netty {
hostname = "127.0.0.1"
port = 2552
}
}

}

The only difference in the slave configuration is the port.

> > RemoteServerStarted@akka://Itemi...@127.0.0.1:2552

> > Instantiating Remote Actor [akka://Itemi...@127.0.0.1:2553/remote/
> > Itemi...@127.0.0.1:2552/user/$a]

> > [DEBUG] [03/06/2012 16:46:25.105] [Itemizer-akka.actor.default-
> > dispatcher-2] [RemoteClient(akka://Itemizer)] Starting remote client

> > connection to [akka://Itemi...@127.0.0.1:2553|/127.0.0.1]

> Typesafe <http://www.typesafe.com/> - The software stack for applications
> that scale
>
> Twitter: @viktorklang

[√iktor Ҡlang]

Do you get this error message without the debug logging option turned on?

Cheers,

√

Typesafe - The software stack for applications that scale

Twitter: @viktorklang

[Roland Kuhn]

I fear that this is an incompatibility between remoting, serialize-messages and DEBUG loglevel: the RemoteLifeCycleEvents are published on the event bus, which sends them as messages to the logging listener, during which it is serialized. I guess you found the first “bug” (assuming that others agree with my assessment that the RemoteTransport field in these messages should be marked @transient to fix this). The easiest fix of course is to disable serialize-messages.

Regards,

Roland

Roland Kuhn
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn

[√iktor Ҡlang]

I don't think it should be transient. Then you'll end up with nulls

[rkuhn]

OTOH there is no other plausible value than null to be reconstructed on the other end of that network tunnel, is there?

>>> akka-user+unsubscribe@googlegroups.com.

>>> For more options, visit this group at
>>> http://groups.google.com/group/akka-user?hl=en.
>>
>> --
>> Viktor Klang
>>
>> Akka Tech Lead
>> Typesafe <http://www.typesafe.com/> - The software stack for applications
>> that scale
>>
>> Twitter: @viktorklang
>
> --
> You received this message because you are subscribed to the Google Groups "Akka User List" group.
> To post to this group, send email to akka...@googlegroups.com.

> To unsubscribe from this group, send email to akka-user+unsubscribe@googlegroups.com.

> For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.
>

Roland Kuhn
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn


--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.

To unsubscribe from this group, send email to akka-user+unsubscribe@googlegroups.com.

[Fredrik Ludvigsson]

As a response to Viktor Klang: No, I do not get the error when debug
logging is turned off.
I do get a similiar error if I include a router in my remote
deployment such as in:

ActorRef ref = system.actorOf(new Props(CommandWorker.class)
.withRouter(new SmallestMailboxRouter(10))
.withDeploy(new Deploy(new RemoteScope(addr))));

But now it is the SmallestMailboxRouter class which gets the
java.io.NotSerializableException.

/Fredrik

> ...
>
> läs mer »

[√iktor Ҡlang]

On Tue, Mar 6, 2012 at 8:34 PM, Fredrik Ludvigsson <fredrik.m....@gmail.com> wrote:

As a response to Viktor Klang: No, I do not get the error when debug
logging is turned off.
I do get a similiar error if I include a router in my remote
deployment such as in:

ActorRef ref = system.actorOf(new Props(CommandWorker.class)
                               .withRouter(new SmallestMailboxRouter(10))
                               .withDeploy(new Deploy(new RemoteScope(addr))));

But now it is the SmallestMailboxRouter class which gets the
java.io.NotSerializableException.

Hmmm, so you're trying to create a router on another machine?

That's a feature we have yet to support, but it sounds very cool, so open a ticket.

Cheers,

√

 

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.

To unsubscribe from this group, send email to akka-user+...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--

Viktor Klang

Akka Tech Lead

Typesafe - The software stack for applications that scale

Twitter: @viktorklang

[Roland Kuhn]

On Mar 6, 2012, at 20:52 , √iktor Ҡlang wrote:

On Tue, Mar 6, 2012 at 8:34 PM, Fredrik Ludvigsson <fredrik.m....@gmail.com> wrote:

As a response to Viktor Klang: No, I do not get the error when debug
logging is turned off.
I do get a similiar error if I include a router in my remote
deployment such as in:

ActorRef ref = system.actorOf(new Props(CommandWorker.class)
                               .withRouter(new SmallestMailboxRouter(10))
                               .withDeploy(new Deploy(new RemoteScope(addr))));

But now it is the SmallestMailboxRouter class which gets the
java.io.NotSerializableException.

Hmmm, so you're trying to create a router on another machine?

That's a feature we have yet to support, but it sounds very cool, so open a ticket.

Well, actually I … ahem … implemented that already … ;-) 

So it seems that SmallestMailboxRouter contains a field with a ThreadLocal, which is not serializable. I have the feeling that there are several classes which are not serializable enough …

Regards,

Roland

Roland Kuhn
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn

[Fredrik Ludvigsson]

Oh, ok my bad. I made assumptions from reading first post in thread
http://groups.google.com/group/akka-user/browse_thread/thread/d7c15086c73dbd24/53a6fbe19be064c1?show_docid=53a6fbe19be064c1

Thank you for clearing things out.

/Fredrik

On 6 mar, 20:52, √iktor Ҡlang <viktor.kl...@gmail.com> wrote:
> On Tue, Mar 6, 2012 at 8:34 PM, Fredrik Ludvigsson <
>

> > > >> >>>        at akka.serialization.JavaSerializer$$anonfun$toBinary...
>
> läs mer »

[√iktor Ҡlang]

On Tue, Mar 6, 2012 at 8:58 PM, Roland Kuhn <goo...@rkuhn.info> wrote:

On Mar 6, 2012, at 20:52 , √iktor Ҡlang wrote:

On Tue, Mar 6, 2012 at 8:34 PM, Fredrik Ludvigsson <fredrik.m....@gmail.com> wrote:

As a response to Viktor Klang: No, I do not get the error when debug
logging is turned off.
I do get a similiar error if I include a router in my remote
deployment such as in:

ActorRef ref = system.actorOf(new Props(CommandWorker.class)
                               .withRouter(new SmallestMailboxRouter(10))
                               .withDeploy(new Deploy(new RemoteScope(addr))));

But now it is the SmallestMailboxRouter class which gets the
java.io.NotSerializableException.

Hmmm, so you're trying to create a router on another machine?

That's a feature we have yet to support, but it sounds very cool, so open a ticket.

Well, actually I … ahem … implemented that already … ;-) 

Doesn't look like it works to me, that feature test might have a small loop hole? ;-)

 

So it seems that SmallestMailboxRouter contains a field with a ThreadLocal, which is not serializable. I have the feeling that there are several classes which are not serializable enough …

It's either a case of lacking docs on the router itself, or a missing test that in this case would fail.

Cheers,

√