Sharing a connection pool between actors

[Nathan Stults]

I have a collection of DB workers behind a router, and I would like them to share a scala-redis connection pool. My inclination is to have the supervisor own the pool and provide it to the constructor of the workers, but this violates the actor-sharing-state principal. I am wondering what the recommended approach would be?

//What I would like to do in essence:

class RedisActor extends Actor {

   @volatile

   val pool = new RedisClientPool(...)

   override def onStart() {

        val props = Props(

           new RedisWorker(pool) //  <----------- Not ok?

        ).withRouter(...)

        actorOf(props)

   }

}

[√iktor Ҡlang]

Hi Nathan!

On Thu, May 3, 2012 at 8:47 PM, Nathan Stults <plasti...@gmail.com> wrote:

I have a collection of DB workers behind a router, and I would like them to share a scala-redis connection pool. My inclination is to have the supervisor own the pool and provide it to the constructor of the workers, but this violates the actor-sharing-state principal. I am wondering what the recommended approach would be?

Using a connection pool to a database is a solution, not a problem.

What's the problem you're trying to solve?

Cheers,

√

 

//What I would like to do in essence:

class RedisActor extends Actor {

   @volatile

   val pool = new RedisClientPool(...)

   override def onStart() {

        val props = Props(

           new RedisWorker(pool) //  <----------- Not ok?

        ).withRouter(...)

        actorOf(props)

   }

}

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/agxSpdtg2IsJ.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
Viktor Klang

Akka Tech Lead

Typesafe - The software stack for applications that scale

Twitter: @viktorklang

[Nathan Stults]

Good question - I don't have a particular problem, I was just trying to keep a bound on the number of client connections created by my app. I will have a number of actors, and each actor will have a number of workers, and I was hoping to not have an ungodly number of individual connections open to the DB. 

On Thursday, May 3, 2012 11:59:22 AM UTC-7, √ wrote:

Hi Nathan!

On Thu, May 3, 2012 at 8:47 PM, Nathan Stults <plasti...@gmail.com> wrote:

I have a collection of DB workers behind a router, and I would like them to share a scala-redis connection pool. My inclination is to have the supervisor own the pool and provide it to the constructor of the workers, but this violates the actor-sharing-state principal. I am wondering what the recommended approach would be?

Using a connection pool to a database is a solution, not a problem.

What's the problem you're trying to solve?

Cheers,

√

 

//What I would like to do in essence:

class RedisActor extends Actor {

   @volatile

   val pool = new RedisClientPool(...)

   override def onStart() {

        val props = Props(

           new RedisWorker(pool) //  <----------- Not ok?

        ).withRouter(...)

        actorOf(props)

   }

}

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/agxSpdtg2IsJ.
To post to this group, send email to akka...@googlegroups.com.

To unsubscribe from this group, send email to akka-user+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

[√iktor Ҡlang]

On Thu, May 3, 2012 at 9:08 PM, Nathan Stults <plasti...@gmail.com> wrote:

Good question - I don't have a particular problem, I was just trying to keep a bound on the number of client connections created by my app. I will have a number of actors, and each actor will have a number of workers, and I was hoping to not have an ungodly number of individual connections open to the DB. 

Represent the DB as an Actor and send messages to it?

Cheers,

√

 

To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/P7G48g63OSAJ.

To post to this group, send email to akka...@googlegroups.com.

To unsubscribe from this group, send email to akka-user+...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

[Nathan Stults]

Great, will do thanks.

[Giovanni]

On Fri, May 4, 2012 at 3:11 AM, √iktor Ҡlang <viktor...@gmail.com> wrote:

 

Represent the DB as an Actor and send messages to it?

I used a similar approach. However, in order to scale up my application in the future, instead of using a single actor for representing the DB, I used a master DB actor, which creates a router with a configurable number of DB actors.

Each DB actor will open a connection to the DB and will process the messages coming from the master DB actor through the router.

Is this a good solution? Or are there any problems with this approach?

Best regards,
giovanni

[√iktor Ҡlang]

That will work just fine. You could also consider creating a Router.

Cheers,

√

 

Best regards,
giovanni

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

[Giovanni]

Thanks, Victor, for your reply.

What do you mean for "creating a Router"?

Best regards,
giovanni

[√iktor Ҡlang]

On Fri, May 4, 2012 at 10:50 AM, Giovanni <pino...@gmail.com> wrote:

Thanks, Victor, for your reply.

What do you mean for "creating a Router"?

¨

http://doc.akka.io/docs/akka/2.0.1/scala/routing.html

Cheers,

√

[Giovanni]

Thanks

giovanni

[Derek Wyatt]

You should consider the potential race condition.

If you have a single DB Actor then sequencing is guaranteed.  Actor A says "Update record 12, set b = 10" and then says "Update record 12, set b = 14" then eventually b = 14.  With a router, those two requests will go to arbitrary back-end routers.  There is no longer a guaranteed ordering.

Regs,

Derek

[√iktor Ҡlang]

On Fri, May 4, 2012 at 11:10 AM, Derek Wyatt <de...@derekwyatt.org> wrote:

You should consider the potential race condition.

If you have a single DB Actor then sequencing is guaranteed.  Actor A says "Update record 12, set b = 10" and then says "Update record 12, set b = 14" then eventually b = 14.  With a router, those two requests will go to arbitrary back-end routers.  There is no longer a guaranteed ordering.

Since he had multiple connections anyway he'd have that problem anyway. Also, if some other application or admin modifies the contents of the database, all bets are off if anyway? (If you're doing destructive updates that is)

Cheers,

√

[Derek Wyatt]

All true.  However, the first time I used a router for this purpose, my use case hit this race condition and bad things happened.  All was not lost, of course, as I simply created a stable load balancer that pinned senders to the same back-end router.  Throughput was still very high as the distribution was pretty even, so there was no real back-end bottleneck.

I just figured it needed pointing out.

[Nathan Stults]

Would it make sense to use Futures w/some appropriate executor to allow use of the driver provided connection pools?

class DB extends Actor {

    @volatile

    private val pool = new DBPool(...)

    def receive = {

          case cmd : SomeCommand => exec(...) pipeTo sender

          ...

    }

    def exec[T](body : DBClient => T) : T = {

        implicit val execCtx = getDispatcherFromConfig("my-configured-dispatcher") <--- is this possible?

        Future {  pool.withClient { client => body(client) } }

    }

    

}

It should allow any kind of appropriate threading model. Or is this just another form of leaking actor state?

[Akka Team]

What would happen if the Actor was moved to another node at runtime?

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/Ifwzyg4pXZAJ.

To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--

Akka Team

Typesafe - The software stack for applications that scale

Blog: letitcrash.com
Twitter: @akkateam

[Nathan Stults]

Not sure what the mechanics of a move are, but if the “pool” variable was marked as transient it seems like it should work OK. A new DBPool would just get instantiated after the move.

 

Or is the issue that the move could invalidate the original actor with futures still pending?

[Jason Mason]

what are the semantics on "moving an actor to another node at runtime". what happens to messages that are in flight ? 

recreating the pool on the other side is ok i guess if the persistence store is transactional but how would the "clients" be notified?

one potential option i see is:

* mark the actor as prepping for moving

* finish the current message being processed

* stop processing the actor mailbox 

* serialize the actor, kill it and deserialize on the new node

[Viktor Klang]

On Fri, May 4, 2012 at 6:06 PM, Nathan Stults <plasti...@gmail.com> wrote:

Would it make sense to use Futures w/some appropriate executor to allow use of the driver provided connection pools?

class DB extends Actor {

    @volatile

    private val pool = new DBPool(...)

    def receive = {

          case cmd : SomeCommand => exec(...) pipeTo sender

          ...

    }

    def exec[T](body : DBClient => T) : T = {

        implicit val execCtx = getDispatcherFromConfig("my-configured-dispatcher") <--- is this possible?

system.dispatchers.lookup: https://github.com/akka/akka/blob/master/akka-actor/src/main/scala/akka/dispatch/Dispatchers.scala#L71

 

        Future {  pool.withClient { client => body(client) } }

    }

    

}

It should allow any kind of appropriate threading model. Or is this just another form of leaking actor state?

On Friday, May 4, 2012 1:34:47 AM UTC-7, Giovanni wrote:

On Fri, May 4, 2012 at 3:11 AM, √iktor Ҡlang <viktor...@gmail.com> wrote:

 

Represent the DB as an Actor and send messages to it?

I used a similar approach. However, in order to scale up my application in the future, instead of using a single actor for representing the DB, I used a master DB actor, which creates a router with a configurable number of DB actors.

Each DB actor will open a connection to the DB and will process the messages coming from the master DB actor through the router.

Is this a good solution? Or are there any problems with this approach?

Best regards,
giovanni

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/Ifwzyg4pXZAJ.

[Jason Mason]

so were there any conclusions here? i think the original question is a good one.

it seems mostly pointless to have a single actor "representing a database" that maintains a pool of connections when message processed sequentially? especially if each message is processed synchronously due to not using futures ("What would happen if the Actor was moved to another node at runtime?"). 

so one solution discussed was: create a (round robin) router "representing a database" which spawns a number of routees, each maintaining a connection pool of 1 to the database. (using a connection pool library just to get the idle connection/test connection stuff for free). 

any other/better solutions? 

[Jason Mason]

good call out derek. sounds like a sticky session router :)

did you create a custom router maintaining a map of getSender() -> routee?

[√iktor Ҡlang]

On Sun, May 6, 2012 at 6:44 PM, Jason Mason <jason....@gmail.com> wrote:

so were there any conclusions here? i think the original question is a good one.

it seems mostly pointless to have a single actor "representing a database" that maintains a pool of connections when message processed sequentially?

I didn't say that he couldn't spawn a child for every connection.

 

especially if each message is processed synchronously due to not using futures ("What would happen if the Actor was moved to another node at runtime?"). 

This would be fine as long as the DB-actor always was moved if a child is moved, so children always stay at the same node as the DB-actor.

 

so one solution discussed was: create a (round robin) router "representing a database" which spawns a number of routees, each maintaining a connection pool of 1 to the database. (using a connection pool library just to get the idle connection/test connection stuff for free). 

That's worse than my solution since in my solution the DB actor can supervise the connection-actors, but in the router approach someone else has to be the parent(supervisor).

Cheers,

√

 

any other/better solutions? 

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/NpHpzCPKGrYJ.

To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

[Giovanni]

On Sun, May 6, 2012 at 6:44 PM, Jason Mason <jason....@gmail.com> wrote:

so one solution discussed was: create a (round robin) router "representing a database" which spawns a number of routees, each maintaining a connection pool of 1 to the database. (using a connection pool library just to get the idle connection/test connection stuff for free). 

In the akka system, which I am currently developing, I am using this solution.

Best regards,
giovanni

[√iktor Ҡlang]

On Sun, May 6, 2012 at 6:56 PM, √iktor Ҡlang <viktor...@gmail.com> wrote:

On Sun, May 6, 2012 at 6:44 PM, Jason Mason <jason....@gmail.com> wrote:

so were there any conclusions here? i think the original question is a good one.

it seems mostly pointless to have a single actor "representing a database" that maintains a pool of connections when message processed sequentially?

I didn't say that he couldn't spawn a child for every connection.

 

especially if each message is processed synchronously due to not using futures ("What would happen if the Actor was moved to another node at runtime?"). 

This would be fine as long as the DB-actor always was moved if a child is moved, so children always stay at the same node as the DB-actor.

 

so one solution discussed was: create a (round robin) router "representing a database" which spawns a number of routees, each maintaining a connection pool of 1 to the database. (using a connection pool library just to get the idle connection/test connection stuff for free). 

That's worse than my solution since in my solution the DB actor can supervise the connection-actors, but in the router approach someone else has to be the parent(supervisor).

Simplification of course, since you can supply a SupervisorStrategy in your router config.

 

Cheers,

√

 

any other/better solutions? 

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/NpHpzCPKGrYJ.

To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
Viktor Klang

Akka Tech Lead

Typesafe - The software stack for applications that scale

Twitter: @viktorklang

[Derek Wyatt]

It wasn't a router, actually. It was a LoadBalancer, which doesn't exist in 2.0 as far as I know. A router would do the job though. The solution is quite simple. All you have to do is hash the sender to the endpoint actor. I found the hash to be a pretty decent way of doing things, as it distributed pretty evenly. And on top of that it increased throughput by a factor of 10. A router of this kind should work out very well. But, if you really don't need it, then a round robin router should work just fine.

Sent from my BlackBerry

---

From: Jason Mason <jason....@gmail.com>

Sender: akka...@googlegroups.com

Date: Sun, 6 May 2012 09:49:28 -0700 (PDT)

To: <akka...@googlegroups.com>

ReplyTo: akka...@googlegroups.com

Subject: Re: [akka-user] Sharing a connection pool between actors

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/0YMjoxAvNasJ.

[Giovanni]

Actually my solution has a MasterDBActor, which creates a round-robin router of DBActors. The DBActor open the connection to the DB.

The other actors, which need a database service, send messages to the MasterDBActor, which forward them to router, which consequently forwards them to the DBActors.

Best regards,
giovanni

[Jason Mason]

On Sunday, May 6, 2012 9:56:33 AM UTC-7, √ wrote:

On Sun, May 6, 2012 at 6:44 PM, Jason Mason wrote:

so were there any conclusions here? i think the original question is a good one.

it seems mostly pointless to have a single actor "representing a database" that maintains a pool of connections when message processed sequentially?

I didn't say that he couldn't spawn a child for every connection.

 

especially if each message is processed synchronously due to not using futures ("What would happen if the Actor was moved to another node at runtime?"). 

This would be fine as long as the DB-actor always was moved if a child is moved, so children always stay at the same node as the DB-actor.

 

so one solution discussed was: create a (round robin) router "representing a database" which spawns a number of routees, each maintaining a connection pool of 1 to the database. (using a connection pool library just to get the idle connection/test connection stuff for free). 

That's worse than my solution since in my solution the DB actor can supervise the connection-actors, but in the router approach someone else has to be the parent(supervisor).

oh interesting, can't one achieve the same thing with new Router(x).withSupervisorStrategy()? 

thanks for the input!

 

[Roland Kuhn]

Hi Jason,

Yes, indeed, the head actor of the router will use that as its supervisor strategy. Apart from that, it is not very much configurable, though. This means that the router approach would work fine if there is nothing more to coordinate; otherwise I’d recommend creating a full-fledged custom actor. It should be noted that database requests will not be sent in any particular order, no matter which approach you chose, but that is probably fine since you are using a connection pool anyway.

Regards,

Roland Kuhn
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn