Pub/Sub communication mechanism for LiftActors

[Piotr Dyraga]

Hey folks,

I'd like to share with you something we worked on with Antonio Salazar Cardozo last week. We find it really helpful in our project so it might be worth considering making it a part of Lift.

https://github.com/pdyraga/lift-message-bus

The idea is to allow any two or more Comets communicate with each other even if they all live in separate user sessions. Lift does not allow to obtain reference to Comet actors from other user sessions and this is reasonable limitation. However, sometimes, we need to send an instant notification to all Comets rendering the given page so that they can execute partialUpdate as soon as some event occurs. Polling database each second for changes from all Comets displaying the given page is less than optimal in most cases. Here the MessageBus comes into play.

Comets (actually any descendants of LifActor) subscribe to Topics which are abstractions allowing to specify in which type of messages the given listener is interested. In our implementation, Topic is a trait with one method def name: String.
Here are two example Topic implementations:

case class WombatPhotosTopic(wombatId: String) extends Topic {
  val name = "wombat-topic-" + wombatId
}

case class BeaverPhotosTopic(beaverId: String) extends Topic {
  val name = "beaver-topic-" + beaverId
}

Actors can subscribe to MessageBus with Subscribe message (for Comets they will be usually sent in localStartup method):

MessageBus ! Subscribe(listener1, WombatPhotosTopic("1"))
MessageBus ! Subscribe(listener2, WombatPhotosTopic("2"))
MessageBus ! Subscribe(listener3, BeaverPhotosTopic("3"))

There are two ways to send a message to other actors: For and ForAll. The payload of For message will be delivered by MessageBus to all LiftActors that subscribed to the given Topic (we look for their equality). The payload of ForAll message will be delivered by MessageBus to all LiftActors that subscribed to the given Topic type.

If we take the topics and listeners that we've set up above and do:

MessageBus ! For(WombatPhotosTopic("1"), WombatPhoto('w1.jpg')) // 1
MessageBus ! ForAll[WombatPhotosTopic](WombatPhoto('w2.jpg')) // 2

The first message will be delivered only to listener1 while second message will be delivered both to listener1 and listener2.

Actors can unsubscribe from the given Topic sending Unsubscribe message:

MessageBus ! Unsubscribe(this, BeaverPhotosTopic("3"))

Actors should listen for messages from MessageBus in the same way as they listen for any other type of message. That is, for example, messageHandler method for LiftActors  and high/low/mediumPriority methods for CometActors:

case object BeaverPhotosTopic extends Topic {
  val name = "beaver-photos"
}

class BeaverPhotosWall extends NamedCometActorTrait {
  override def localSetup = {
    super.localSetup MessageBus ! Subscribe(this, BeaverPhotosTopic)
  }

  override def localShutdown = {
    super.localShutdown MessageBus ! Unsubscribe(this, BeaverPhotosTopic)
  }

  override def lowPriority = {
    case NewBeaverPhotoPosted(photo) =>
      partialUpdate(renderNewPhoto(photo))
  }
}

You can find some more examples in the specification for MessageBus: https://github.com/pdyraga/lift-message-bus/blob/master/src/test/scala/net/liftmodules/bus/MessageBusSpec.scala

Please let us know what you think about it. If you find this mechanism helpful and interesting enough to make it a part of Lift, we'll be happy to open a PR.

Thanks,
Piotr

[Diego Medina]

Thanks for sharing this!

Did you guys see this

https://groups.google.com/d/topic/liftweb/-gwez1cOo7A/discussion

which was added to Lift 2.6 (can't remember if I got it in time for 2.5 or not)

What you describe here and what is already in Lift sound similar, but maybe you had another use case in mind.

Thanks

Diego

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code

---
You received this message because you are subscribed to the Google Groups "Lift" group.
To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Diego Medina
Lift/Scala Consultant
di...@fmpwizard.com
http://blog.fmpwizard.com/

[Antonio Salazar Cardozo]

Interestingly, I thought named comets were mostly for intra-session

communication rather than inter-session. I think the key with Piotr's

implementation is that it's actually generic over all LiftActors, rather

than being built with just CometActors in mind, and makes “topics”

a first-class datatype vs just using simple strings (although it boils

down to a simple string at the core, right now). This allows a bit more

type-level discernment.

Fundamentally, though, I think the mechanisms are relatively similar

to each other. The MessageBus implementation is arguably just an

evolution of the NamedComet* stuff. If we were to add it to Lift, I think

it would go into the actor package, rather than the web package, given

its more generic nature. Indeed, one could imagine NamedCometListener

and friends implemented on top of MessageBus while maintaining the

same API.

Thanks,

Antonio

To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Diego Medina]

> ...If we were to add it to Lift, I think

it would go into the actor package, rather than the web package, given

its more generic nature. Indeed, one could imagine NamedCometListener

and friends implemented on top of MessageBus while maintaining the

same API.

If we add it, it would be great to update namedCometActor and friends to be based on this work, vs having two similar but independent code paths.

Thanks

Diego

 

Thanks,

Antonio

To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Diego Medina
Lift/Scala Consultant
di...@fmpwizard.com
http://blog.fmpwizard.com/

--

--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code

---
You received this message because you are subscribed to the Google Groups "Lift" group.

To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Piotr Dyraga]

Agree with everything you said here.

One question, though.

NamedCometListener looks for me as just a registry for named comets. Just like Antonio said, it looks for me as intra-session communication thing - you create one comet to rule multiple pages, so we have "communication" here straight from the fact that this is actually one comet under the hood. Isn't that?

Regarding communication between comets we should rather look at CometListener and ListenerManager and refactor these two components to use MessageBus internally. Is my understanding correct?

Thanks,
Piotr

[Diego Medina]

On Thu, Apr 28, 2016 at 4:13 PM, Piotr Dyraga <piotr....@softwaremill.com> wrote:

Agree with everything you said here.

One question, though.

NamedCometListener looks for me as just a registry for named comets. Just like Antonio said, it looks for me as intra-session communication thing - you create one comet to rule multiple pages, so we have "communication" here straight from the fact that this is actually one comet under the hood. Isn't that?

Regarding communication between comets we should rather look at CometListener and ListenerManager and refactor these two components to use MessageBus internally. Is my understanding correct?

CometListener, ListenerManager and NamedCometListener were all added at the same time to work together and as a whole they do the communication across sessions

You can see the PR that added all the different traits, objects, etc here

https://github.com/lift/framework/pull/1251

Thanks

[Piotr Dyraga]

OK, so if you all agree for adding MessageBus to Lift I can open an issue and try to port the existing Diego's mechanism for comets to use MessageBus internally. Is that fine?

[Antonio Salazar Cardozo]

Well, we don't generally accept code from non-committers. I think

to start with if the project you linked could be published to sonatype

so others could use it, that would be a good start. Then as we see

what use it gets, we can look at pulling it into the Lift core.

Thanks,

Antonio

[Piotr Dyraga]

Fair enough. Would it be preferred to publish it to net.liftmodules or should I chose my own repository on sonatype?

[Antonio Salazar Cardozo]

That's actually a good question… I'm not sure what the setup

is for Lift modules, in particular what publish rights there are or

are not…

Maybe Richard or someone can chime in?

Thanks,

Antonio

[Matt Farmer]

I think doing this involves making a comment on a JIRA thread that Sonatype monitors or something, but I’m not sure.

—
Matt Farmer | Blog | Twitter
GPG: CD57 2E26 F60C 0A61 E6D8  FC72 4493 8917 D667 4D07

[Peter Petersson]

This seems like a perfect fitt for a lift module (at least to begin with) so using net.liftmodules seems to me the right thing to do. I remember sending a mail to someone at sonatype mentioning net.liftmodules I think I got the info in a mail from Richard, I am on the phone right now but I can try digg up the information later. Probably someone needs to clear the net.liftmodule usage
Best regards Peter Petersson

[Richard Dallaway]

Permission had already been granted for Piotr at sonatype. The system works \o/

[Antonio Salazar Cardozo]

Nice! Thanks y'all!

Antonio

To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code

---
You received this message because you are subscribed to the Google Groups "Lift" group.

To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code

---
You received this message because you are subscribed to the Google Groups "Lift" group.

To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+unsubscribe@googlegroups.com.

[Piotr Dyraga]

I've released the module over this weekend:
https://oss.sonatype.org/content/repositories/releases/net/liftmodules/messagebus_3.0_2.11/

More info:
http://www.ontheserverside.com/blog/2016/05/09/communication-between-lift-comets/
https://github.com/pdyraga/lift-message-bus

Is there any way to make it automatically build / released for all cross Liftweb and Scala versions? Should I add it to buildhub of liftmodules (https://github.com/liftmodules/buildhub) ?

Cheers,
Piotr

[Peter Petersson]

There are a lot of other ways to do this but you can take a look at the FoBo module. It's a multi project / module build that has a cross java/scala/Lift version bash/sbt build script for easy public or local artifact release, see project/autobuild folder. It's not the cleanest build system but it works for me ;) Maybe it will give you some ideas
Best regards Peter Petersson

[Richard Dallaway]

The automated module builds all are on Cloudbees. But I believe we're looking to move away from the Cloudbees. I guess we need to figure out alternatives. Travis, I suppose, but suggestions welcome.

Richard

[Henrik Härkönen]

Hi Piotr!

While I was searching for solution where I could easily message comets from outside the session, I found this MessageBus, it's brilliant, thanks! :) It's so... clean and versatile solution.

Btw, there was Scala 2.11 version available in the repository, would it be big effort to have 2.12 as well?

-Henrik

[Piotr Dyraga]

Hi Henrik,

I am really happy you like it! 

Here you go: http://repo1.maven.org/maven2/net/liftmodules/messagebus_3.0_2.12/1.0/

Btw, some time ago I wrote a blogpost touching this topic, so you may find it interesting: http://www.ontheserverside.com/blog/2016/05/09/communication-between-lift-comets/

Cheers,

Piotr

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code

---

You received this message because you are subscribed to a topic in the Google Groups "Lift" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/liftweb/HYyKtUOi_og/unsubscribe.
To unsubscribe from this group and all its topics, send an email to liftweb+u...@googlegroups.com.

[Henrik Härkönen]

Whoa, that was quick! :)

Thanks, I'll soon upgrade my project to 2.12, now all the components are available, brilliant. :)

Your blog post was very good, I enjoyed reading it, and actually it was the page where I learned about the MessageBus module in the first place. After that I searched this list for it too. :)

-Henrik