Existing models for PUB/SUB: Analysis
Martin Sustrik
I have read through lot of pub/sub documentation for various
products/protocols. This email tries to summarise the functionality
available in the wild at the moment.
1. Broadcasting data. This is the essence of pub/sub and is shared by
all the pub/sub systems.
2. Subscriptions. Subscriptions allow for asking for only a subset of
data based on criteria specified by the consumer. There's are different
kind of filters used for the task:
a. No filters. Always get all the messages.
b. Filter messages based on fixed topic.
c. Topics form a tree. Filter messages from a specific sub-tree.
d. Filter messages based on regular expressions.
e. Complex filtering based on the message content (SQL-like selectors
and alike)
2. Direct 1-hop pub/sub:
a. No support. Data have to pass through a broker.
b. Direct pub/sub is available.
3. Multi-hop pub/sub:
a. No explicit support. User can achieve multi-hop by implementing a
bridging application that receives data from one topology and
republishes them to another topology.
b. Support of multi-hop via federated brokers.
c. Support of multi-hop without a need for broker.
4. Avoiding cycles in the distribution tree:
a. No support. Administrators are responsible not to introduce cycles
into the topology.
b. Hop-counting. Each message expires after N hops and thus cannot
enter infinite cycle in the topology.
c. Creating a spanning tree, ie. disabling the flows between nodes
automatically in such a way that the resulting topology is a tree.
5. Forwarding subscriptions through a multi-hop topology:
a. No support. Subscriptions work only on hop-by-hop basis.
b. Subscriptions are forwarded to the last-but-one hop. This model is
used in various broker-based solutions. Subscription is forwarded to the
most upstream broker, however, it is not forwarded to the publisher.
c. Full support. Subscriptions are forwarded all the way from the
consumer to the publisher.
6. Unsubscriptions.
a. No support. Subscriptions can be removed only by closing the
connection.
b. Weak unsubscriptions. Consumer can unsubscribe, however, it can
still get the messages from the canceled subscription for some amount of
time.
c. Strong unsubscriptions. Once the unsubscription is done consumer
won't get any further messages from that subscription.
7. Late joiners.
a. No special support. Late joiners get data from the point they've
joined in.
b. Late joiners can request certain amount of historical data after
joining in.
---
The list is not likely to be complete. We can add more functionality as
we get to understand the existing pub/sub models better.
In any case, the goal with SP is to define such semantics that would
allow to bridge any two of these systems (system1 <-> SP <-> system2).
It is not mandatory to support *all* the functionality provided by
individual systems though.
There are several techniques to achieve the goal:
- Defining the highest common denominator for the functionality.
- Optionally allowing for the functionality that can be safely used in
part of the topology without breaking the rest of the topology.
- Explicitly forbidding the functionality that would break or negatively
impact the whole topology if used in part of the topology.
Thoughts?
Martin
Alexis Richardson
What about propagation of unsubscribe, and topic trees?
--
Note Well: This discussion group is meant to become an IETF working group in the future. Thus, the posts to this discussion should comply with IETF contribution policy as explained here: http://www.ietf.org/about/note-well.html
Martin Sustrik
> What about propagation of unsubscribe, and topic trees?
Right. When I wrote "propagation of subscriptions" what I meant was
"propagation of subscriptions & unsubscriptions". I would be great to
have a term that would refer to both subs and unsubs. I am not aware of
any such term being used today though.
Ideas?
Martin
Alexis Richardson
Links?
On Jun 27, 2011 9:41 PM, "Martin Sustrik" <sus...@250bpm.com> wrote:On 06/27/2011 10:38 PM, Alexis Richardson wrote: > > What about propagation of unsubscribe, and topi...
Alexis Richardson
Relationships? Bindings?
On Jun 27, 2011 9:44 PM, "Alexis Richardson" <ale...@rabbitmq.com> wrote:
Links?
On Jun 27, 2011 9:41 PM, "Martin Sustrik" <sus...@250bpm.com> wrote:On 06/27/2011 10:38 PM, Alexis Richardson wrote: > > What about propagation of unsubscribe, and topi...
> > Right. When I wrote "propagation of subscriptions" what I meant was "propagation of subscriptio...
Martin Sustrik
subscriptions" or even "un- subscriptions"? That way the term
subscription would refer to both.
On 06/27/2011 10:47 PM, Alexis Richardson wrote:
> Relationships? Bindings?
>
> On Jun 27, 2011 9:44 PM, "Alexis Richardson" <ale...@rabbitmq.com
> <mailto:ale...@rabbitmq.com>> wrote:
>
> Links?
>
>> On Jun 27, 2011 9:41 PM, "Martin Sustrik" <sus...@250bpm.com
>> <mailto:sus...@250bpm.com>> wrote:
>>
>> On 06/27/2011 10:38 PM, Alexis Richardson wrote: > > What about
>> propagation of unsubscribe, and topi...
>>
>> > > Right. When I wrote "propagation of subscriptions" what I meant
>> was "propagation of subscriptio...
>>
Alexis Richardson
database queries were explicit.
More to come.
Martin Sustrik
> I don't think that's a good way to go. Imagine if all negative
> database queries were explicit.
Ok. Fair enough.
So, basically, what we have here are two different but related kind of
entities:
1. Filtering criteria stored at a specific node. I believe, the term
"subscriptions" is appropriate in this case.
2. Commands sent on the wire to modify the filtering criteria at the
peer node. Basically a delta to be applied to the above -- it can either
add new criteria or remove old criteria. It's not clear what term use
for these.
What about calling the latter something self-explanatory, such as
"subscription commands"?
Martin
Alexis Richardson
>
> 2. Commands sent on the wire to modify the filtering criteria at the peer
> node. Basically a delta to be applied to the above -- it can either add new
> criteria or remove old criteria. It's not clear what term use for these.
>
> What about calling the latter something self-explanatory, such as
> "subscription commands"?
I am ok with that.
By the way, I thought of something missing from your list. It's a
form of unicast that only makes sense in broadcast networks ;-)
The dual operation to "post to all subscribers" is "pick one
subscriber and post to it".
Note that this is a CHOICE of exactly one subscriber, and therefore it
is logically a quantifier. Filters are predicates. To some extent it
is covered by the SQL type queries, but it would be much cleaner to
say this:
1. Broadcast
1a. broadcast to all
1b. broadcast to one
2. Predicates
2a. ... etc ...
Michael Bridgen
> On Tue, Jun 28, 2011 at 9:51 AM, Martin Sustrik<sus...@250bpm.com> wrote:
>>
>> 2. Commands sent on the wire to modify the filtering criteria at the peer
>> node. Basically a delta to be applied to the above -- it can either add new
>> criteria or remove old criteria. It's not clear what term use for these.
>>
>> What about calling the latter something self-explanatory, such as
>> "subscription commands"?
>
> I am ok with that.
>
> By the way, I thought of something missing from your list. It's a
> form of unicast that only makes sense in broadcast networks ;-)
>
> The dual operation to "post to all subscribers" is "pick one
> subscriber and post to it".
I think this is trying to wodge too much into pub/sub. Isn't "all
subscribers in the predicate get the message" the defining feature of
publish/subscribe? Otherwise we admit essentially any model of
distribution -- i.e., the domain becomes "networks".
Martin Sustrik
>> The dual operation to "post to all subscribers" is "pick one
>> subscriber and post to it".
>
> I think this is trying to wodge too much into pub/sub. Isn't "all
> subscribers in the predicate get the message" the defining feature of
> publish/subscribe? Otherwise we admit essentially any model of
> distribution -- i.e., the domain becomes "networks".
I think Michael is right. PUB/SUB is characterised by consumers
determining via subscriptions whether to get the message or not.
There are other patterns where the destination for a message is defined
in a different way (eg. load-balancing). These should be treated
separately from pub/sub though.
Martin
Tony Garnock-Jones
The dual operation to "post to all subscribers" is "pick one
subscriber and post to it".
"Anycast".
The names for the various operations have traditionally been "broadcast", "multicast", "anycast", and "unicast".
Tony Garnock-Jones
1. Filtering criteria stored at a specific node. I believe, the term "subscriptions" is appropriate in this case.
I think of these as queries over the set of messages flowing through a given network, so "queries" makes more sense to me.
2. Commands sent on the wire to modify the filtering criteria at the peer node. Basically a delta to be applied to the above -- it can either add new criteria or remove old criteria. It's not clear what term use for these.
What about calling the latter something self-explanatory, such as "subscription commands"?
Yep, "subscription management" perhaps. "A subscription is a query combined with a routing destination."