Explicit remote/cluster spawning in Akka 2.0

[Anders Engström]

Hi guys!

We have custom infrastructure for Akka 1.x that allows us to spawn actors across a cluster. In short it uses akka-camel to dispatch messages to actors and the explicit remote API available in Akka 1.x to spawn actors on remote nodes. We use a distributed Map (Hazelcast) to keep track of available cluster nodes and their Akka remote attributes.

The akka 2.0 remote API is implicit - there is no (?) way to spawn an actor on a remote ActorSystem - only through configuration which is static:

akka {

  actor {

    provider: "akka.remote.RemoteActorRefProvider"

    deployment { /sampleActor {

      remote: "akka://systemName@address:port"

    }}

  }}

This basically pins the actor to the remote node - which does not fit well in a cluster which is not statically defined. 

I know that this will be made more flexible and dynamic with Akka 2.1 - and we're aiming at leveraging the cluster support in Akka 2.1 (possibly writing a custom Hazelcast plugin), but until then would still like to use the Akka 2.0 goodies *grin*.

So - would it be possible/reasonable to extend the remote spawning to take "strategy" that at runtime decides where to spawn an actor? In my mind it would look something like:

akka {

  actor {

    provider: "akka.remote.RemoteActorRefProvider"

    deployment { /sampleActor {

      remoteStrategy: "CustomStrategyClass"

    }}

  }}

where the CustomStrategyClass would implement an interface that returns an URI that points to the location where the actor should be spawned (which would be evaluated on every 'actorOf' call)?

A similar strategy could be used to create the sequence of URIs when remoting is used with routers.

Best Regards //Anders

[√iktor Ҡlang]

Hi Anders,

have you've tried just having an actor statically consufigured, that receives messages containing Props, spawns the props and replies with the newly spawned actor?

Does that suffice?

Cheers,
√

2012/1/10 Anders Engström <epir...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/6US5oDeGPugJ.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
Viktor Klang

Akka Tech Lead

Typesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang

[rkuhn]

Well, you can with some evil, unsupported downcasting get hold of the Deployer and inject config sets dynamically just before creating actors. But don't tell anyone that I said that ;-)

(There's even a test which uses this technique, I just don't remember the name right now)

Regards,

Roland

[Anders Engström]

Hi! Thanks for the reply!

On Tuesday, January 10, 2012 4:14:17 PM UTC+1, Viktor Klang wrote:

have you've tried just having an actor statically consufigured, that receives messages containing Props, spawns the props and replies with the newly spawned actor?

Does that suffice?

 I've thought about that approach as well, it's basically the same way it works in our current solution. The downside is that it breaks the actor hierarchy - the remotely spawned actor would be a child of the "spawner" actor (the statically configured actor) - not a child of its logical parent. Unless there's another way is probably how we will solve it until Akka 2.1 :)

I guess it would be possible to supply the actor-system with a custom Config instance that resolves the "remote" property dynamically at runtime.. I haven't checked if current usage of the Config object in Akka is compatible with this though... :)

//Anders

[Anders Engström]

Sounds a bit scary *grin* - on the other hand this would kinda mean that my proposed solution isn't entirely "un-implementable" :)

I will give the "custom config hack" a try first (the one mentioned in my reply to Victor) - if that doesn't work I'll look into the evil downcast :)

//Anders

[Anders Engström]

Alright - I've investigated injecting a custom com.typesafe.config.Config instance when defining the ActorSystem but doesn't really fly :/ 

The idea was to inject a custom Config instance in ActorSystem("app", customConfig) that would somehow make sure that when Config#getString("akka.actor.deployment./sampleActor.remote") the correct URL would be calculated based on the cluster state.

I didn't find any obvious base-class to extend in order to safely implement the Config interface so I tried wrapping the default Config instance with a JDK dynamic proxy instead. After a while I ran into an issue where my Config (proxy) instance was downcast to an internal class com.typesafe.config.impl.MergeableValue. 

But - using the Config system to implement this is probably not the right way. The configuration should be considered static, and the application (akka) shouldn't have to re-evaluate a property more than once. 

So - if I wan't to maintain the actor supervision hierarchy in my Akka 2.0 dynamic cluster (as described in the initial post), are there any other options available?

[rkuhn]

As I said, inject your own deployments. Currently this involves some down-casting to ActorSystemImpl to get .provider.deployer, but I just created http://www.assembla.com/spaces/akka/tickets/1644

Regards,

Roland

[Anders Engström]

Thanks - that great! 

I will give 'manual' deployment a try and let you know how it goes!

//Anders

[Anders Engström]

Hi again.

I finally had some time to try out the work-around suggested. It fails to work the way I would like it to though *grin*.

Please have a look at some sample code here:

https://gist.github.com/1621339#file_code

I launch two JVMs with remote providers on port 2552 and 2553, and tries to spawn actors as remote actors dynamically. The actual spawning seems to work fine - but messages failes to be delivered to the remote actors (this is the output on the "RingServer" instance):

https://gist.github.com/1621339#file_error+message

Am I using the Deployer in an incorrect way?

Best Regards //Anders

[Anders Engström]

I got it working - the trick was to specify the remote address both as a Config and as a RemoteScope instance:

def registerRemoteDeployment(system: ActorSystem, path: ActorPath, remoteHost: String, remotePort: Int) {

  val deployer = system.asInstanceOf[ActorSystemImpl].provider.deployer

  val remoteUri = "akka://%s@%s:%s/".format(system.name, remoteHost, remotePort)

  val systemAddress = RemoteAddressExtractor.unapply(remoteUri).orElse(throw new Exception("Failed to parse URI"))

  val remoteCfg = ConfigFactory.parseString("remote: \"%s\"".format(remoteUri))

  val deployment = Deploy(path.elements.tail.mkString("/", "/", ""), remoteCfg, None, NoRouter, RemoteScope(systemAddress.get))

  deployer.deploy(deployment)

}

This would then be used from an actor like: 

class SomeActor extends Actor {

  // ...

  def spawnChild(name: String) = {

    registerRemoteDeployment(context.system, self.path / name, 192.168.1.2, 2552)

    context.actorOf(Props[SomeChildActor], name)  

  }

}

A bit clunky (and not very DRY) - but it works (for now). Any suggestions for improvements are welcome.

Thanks for adding the issue Roland - It would be really great if there was a native API for specifying the deployment details on 'actorOf' on ActorContext/ActorSystem.

As a side-note: I tried creating a custom ActorRefProvider extending the default RemoteActorRefProvider to override the "actorOf" -- this would have made the custom deployment code a bit more clean. Unfortunately this was not possible since the InternalActorRef class is not accessible outside of its package. Is this by design?

//Anders

[Roland Kuhn]

On 17 jan 2012, at 15:49, Anders Engström <epir...@gmail.com> wrote:

I got it working - the trick was to specify the remote address both as a Config and as a RemoteScope instance:

def registerRemoteDeployment(system: ActorSystem, path: ActorPath, remoteHost: String, remotePort: Int) {

  val deployer = system.asInstanceOf[ActorSystemImpl].provider.deployer

  val remoteUri = "akka://%s@%s:%s/".format(system.name, remoteHost, remotePort)

  val systemAddress = RemoteAddressExtractor.unapply(remoteUri).orElse(throw new Exception("Failed to parse URI"))

  val remoteCfg = ConfigFactory.parseString("remote: \"%s\"".format(remoteUri))

  val deployment = Deploy(path.elements.tail.mkString("/", "/", ""), remoteCfg, None, NoRouter, RemoteScope(systemAddress.get))

  deployer.deploy(deployment)

}

ewww …

This would then be used from an actor like: 

class SomeActor extends Actor {

  // ...

  def spawnChild(name: String) = {

    registerRemoteDeployment(context.system, self.path / name, 192.168.1.2, 2552)

    context.actorOf(Props[SomeChildActor], name)  

  }

}

A bit clunky (and not very DRY) - but it works (for now). Any suggestions for improvements are welcome.

Thanks for adding the issue Roland - It would be really great if there was a native API for specifying the deployment details on 'actorOf' on ActorContext/ActorSystem.

Yes, I would definitely prefer that also from our perspective. 

As a side-note: I tried creating a custom ActorRefProvider extending the default RemoteActorRefProvider to override the "actorOf" -- this would have made the custom deployment code a bit more clean. Unfortunately this was not possible since the InternalActorRef class is not accessible outside of its package. Is this by design?

The new code base lends itself much better to extensions, which we included in several places officially, but the full scope is not yet decided on this matter. What you observe is simply us being conserative, because it is much easier to open up than to close down an API. And before we can really support custom ActorRefProviders we will have to think a few issues through and improve documentation a lot (mostly because of the slightly tricky boot-strap process of an ActorSystem). 

Regards,

Roland