REST for fan-in pub-sub

[Dominic V]

Hi,

I'm currently looking at alternatives to message brokering for a large distributed system where the focus is on robustness over speed and latency, which seems to make REST a good choice.  So the three main patterns we have are pub-sub, which works well with Atom as described in the book, competing-consumer, which works well enough with the If-match optimistic locking, and then a kind of fan-in pub-sub which may be more problematic.  For an example situation using brokering, you want to send an email from a bunch of micro-services so each one publishes a SendEmailCommand to the message broker and emails eventually get sent.  If we want to move to an ATOM approach then there is trouble because we don't want to persist the SendEmailCommand like we would with the standard pub-sub, maybe the service doesn't have any persistence available.  Keeping in mind there may still be more than one subscriber and we don't really want to increase the overall synchronous latency by making extra outgoing calls.  Is there a way that we can solve this problem without some highly available middleware style queueing?

Thanks.

[Christian Blunden]

​Hi Dominic

We implemented our own simple hypermedia based message queue which I have just made public here. Check the readme to get the gist of what goes on.  It was not built for public consumption yet, so caveat emptor!

https://github.com/uswitch/hypermq

​We have been using it in production for well over a year, and we never need to check it.. "it just works" (tm).  The deployment story is not great at the moment, the plan was to get into a deb that can just be apt-get installed. So you may want to fork it and make changes if you want to use it.

We have clients for both ruby & clojure

https://github.com/uswitch/clj-hypermq

https://github.com/uswitch/hypermq-ruby-client

Also another alternative we have moved to is Kafka which is a distributed log. Not as simple to setup, but the components are free (zookeeper & kafka broker).  Also the clients pull data from the brokers just like in a REST system, so you can achieve the same "restart where you left off" if the system fails.  You can implement all types of message patterns (pub-sub, competing consumer etc).. all the complexity is pushed to the clients like in good RESTFUL system design.

--
You received this message because you are subscribed to the Google Groups "restinpractice" group.
To unsubscribe from this group and stop receiving emails from it, send an email to restinpractic...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.