multiple dealers routing to common rpc?

[Greg Fausak]

I am working on architecture (still) to move to a newer version of AB. I am still trying to understand some of the basic concepts.  Like the dealer which connects an rpc caller to the rpc call itself.  I think I get that.  So, I could have a web page->dealer->dbquery (for example) rpc call from the web page through the dealer to the dbquery rpc. I like that architecture, except that the dealer is always involved.  I wonder if there is a way to bypass that dealer, (or all dealers).  For example, once my web page->dealer->dbquery is done the first time, I could have an rpc call that returns the 'address' of the dbquery, so that subsequent queries could effectively do web page->dbquery.  Or for that matter, could I have two paths to a dbquery rpc, like a second route from web page-> other dealer->dbquery.   The only thing different in this chain is a 'other dealer'.  In these examples 3 routes are considered from a web page ----- X ----> dbquery.  Is it possible to have multiple routes to the same endpoint?

-g

[Tobias Oberstein]

Hi Greg,

Am 08.08.2014 06:08, schrieb Greg Fausak:
> I am working on architecture (still) to move to a newer version of AB. I
> am still trying to understand some of the basic concepts. Like the

Since you've been using this stuff for a longer time and still don't
feel confident understanding the concepts, I'd call that our fault;(

I really want to improve on this. My problem is: how can we make it
easier to understand? What is missing?

When you look at these two resources

http://tavendo.com/blog/post/is-crossbar-the-future-of-python-web-apps/
http://wamp.ws/why/

do you have a hint which one is more approachable for you? Then we can
work from there and improve. Or any other hints?

> dealer which connects an rpc caller to the rpc call itself. I think I
> get that. So, I could have a web page->dealer->dbquery (for example)

Yep. The complete picture looks like this

Web Page <-- WAMP --> Dealer <-- WAMP --> Backend Component <-- SQL -->
Database

> rpc call from the web page through the dealer to the dbquery rpc. I like
> that architecture, except that the dealer is always involved. I wonder

"Dealer is always involved": That is by design. It is _necessary_ to
achieve the goal of decoupling. If you kick out the Dealer, then the
Caller would need knowledge of where and how to reach the Callee. Which
means, both would be coupled.

Why are you concerned about "Dealer is always involved"?

> if there is a way to bypass that dealer, (or all dealers). For example,
> once my web page->dealer->dbquery is done the first time, I could have
> an rpc call that returns the 'address' of the dbquery, so that

The "address" of a Callee is exactly what we do _not_ want to have the
Caller know about.

> subsequent queries could effectively do web page->dbquery. Or for that

No. Not possible.

> matter, could I have two paths to a dbquery rpc, like a second route
> from web page-> other dealer->dbquery. The only thing different in
> this chain is a 'other dealer'. In these examples 3 routes are
> considered from a web page ----- X ----> dbquery. Is it possible to
> have multiple routes to the same endpoint?

Not sure I get what you mean. Multiple routes to the same endpoint will
be a common situation with _clustered_ routers:


Caller1 ---> Dealer1 ---> Callee1
^
|
Caller2 ---> Dealer2


Here, Dealer1 and Dealer2 together form a "clustered WAMP router".

Caller1 and Caller2 in above are connected to different Dealers, whereas
Callee1 is only connected to Dealer1.

That means, to have Caller2 call into Callee1 requires routing of the
call from Dealer2 to Dealer1 and then to Callee1.

Hopefully above does not confuse you futher;)

Cheers,
/Tobias

>
> -g
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com
> <mailto:autobahnws+...@googlegroups.com>.
> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/autobahnws/7f83fc4d-27af-4659-8a1c-0f44237ea51d%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/7f83fc4d-27af-4659-8a1c-0f44237ea51d%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Greg Fausak]

On Friday, August 8, 2014 3:50:20 AM UTC-5, Tobias Oberstein wrote:

Hi Greg,

Am 08.08.2014 06:08, schrieb Greg Fausak:
> I am working on architecture (still) to move to a newer version of AB. I
> am still trying to understand some of the basic concepts.  Like the

Since you've been using this stuff for a longer time and still don't
feel confident understanding the concepts, I'd call that our fault;(

I really want to improve on this. My problem is: how can we make it
easier to understand? What is missing?

When you look at these two resources

http://tavendo.com/blog/post/is-crossbar-the-future-of-python-web-apps/
http://wamp.ws/why/

do you have a hint which one is more approachable for you? Then we can
work from there and improve. Or any other hints?

I have been using v1 for quite some time (perhaps a year?).  Many of the concepts were new to me.  Twisted, defereds, just to name a couple.  Programming this way is contrary to what I've been doing for 20 years.  I guess I'm an old dog, and this is a new trick :-)

 

> dealer which connects an rpc caller to the rpc call itself.  I think I
> get that.  So, I could have a web page->dealer->dbquery (for example)

Yep. The complete picture looks like this

Web Page <-- WAMP --> Dealer <-- WAMP --> Backend Component <-- SQL -->
Database

> rpc call from the web page through the dealer to the dbquery rpc. I like
> that architecture, except that the dealer is always involved.  I wonder

"Dealer is always involved": That is by design. It is _necessary_ to
achieve the goal of decoupling. If you kick out the Dealer, then the
Caller would need knowledge of where and how to reach the Callee. Which
means, both would be coupled.

Why are you concerned about "Dealer is always involved"?

I'm not, that is a good design.  However, it is a single point of failure, and it is a single point of congestion.  Just like routing.  When I set up a high availability site I connect it to as many providers as I can.  I would like some route diversity.

 

> if there is a way to bypass that dealer, (or all dealers).  For example,
> once my web page->dealer->dbquery is done the first time, I could have
> an rpc call that returns the 'address' of the dbquery, so that

The "address" of a Callee is exactly what we do _not_ want to have the
Caller know about.

I don't necessarily want to bypass a dealer.  By design, the dealer and rpc can be bound in the same process, so logically, there could still be a dealer to connect to.

 

> subsequent queries could effectively do web page->dbquery.  Or for that

No. Not possible.

> matter, could I have two paths to a dbquery rpc, like a second route
> from web page-> other dealer->dbquery.   The only thing different in
> this chain is a 'other dealer'.  In these examples 3 routes are
> considered from a web page ----- X ----> dbquery.  Is it possible to
> have multiple routes to the same endpoint?

Not sure I get what you mean. Multiple routes to the same endpoint will
be a common situation with _clustered_ routers:


Caller1 ---> Dealer1 ---> Callee1
                 ^
                 |
Caller2 ---> Dealer2


Here, Dealer1 and Dealer2 together form a "clustered WAMP router".

Caller1 and Caller2 in above are connected to different Dealers, whereas
Callee1 is only connected to Dealer1.

That means, to have Caller2 call into Callee1 requires routing of the
call from Dealer2 to Dealer1 and then to Callee1. 

Hopefully above does not confuse you futher;)

I appreciate your discussion.  I think this is a very important design point.

Crossbar promises a buss like connectivity.  Anything can talk directly to anything else.  Logically, that is perfect.  Physically I see connectivity as a hierarchy.  Any dealer piece that is missing will isolate two networks.  This is the basis of my concern. 

Best,

-g

Cheers,
/Tobias

>
> -g
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com

> <mailto:autobahnws+unsub...@googlegroups.com>.

[Tobias Oberstein]

Hi Greg,

snip

> I have been using v1 for quite some time (perhaps a year?). Many of the
> concepts were new to me. Twisted, defereds, just to name a couple.
> Programming this way is contrary to what I've been doing for 20 years.
> I guess I'm an old dog, and this is a new trick :-)

Ah;)

Yep, asynch. programming is quite different from the classic sequential
style. FWIW, we've now have a page

http://autobahn.ws/python/asynchronous-programming.html

which contains some pointers and discusses some aspects.

> "Dealer is always involved": That is by design. It is _necessary_ to
> achieve the goal of decoupling. If you kick out the Dealer, then the
> Caller would need knowledge of where and how to reach the Callee. Which
> means, both would be coupled.
>
> Why are you concerned about "Dealer is always involved"?
>
>
> I'm not, that is a good design. However, it is a single point of
> failure, and it is a single point of congestion. Just like routing.
> When I set up a high availability site I connect it to as many
> providers as I can. I would like some route diversity.

When Router-to-Router communication and clustering arrives, this will no
longer be the case.

When the specific router a client is connected to is lost, the client
might first try to reconnect to the (lost) router, and then fallback to
connecting to another router from a list of routers.

It could be made transparent by using a DNS-based round-robin scheme or
a load-balancer sitting in front of the cluster.

>
> > if there is a way to bypass that dealer, (or all dealers). For
> example,
> > once my web page->dealer->dbquery is done the first time, I could
> have
> > an rpc call that returns the 'address' of the dbquery, so that
>
> The "address" of a Callee is exactly what we do _not_ want to have the
> Caller know about.
>
>
> I don't necessarily want to bypass a dealer. By design, the dealer and
> rpc can be bound in the same process, so logically, there could still be
> a dealer to connect to.

Yes, the important point is that Routers and App Components are separate
_logically_.

Whether they reside on different hosts, same host/different processes or
within same process doesn't matter. As long as they are "separate" and
the Routers do not _directly_ embed app related code, that's fine.

> Hopefully above does not confuse you futher;)
>
>
> I appreciate your discussion. I think this is a very important design
> point.
>
> Crossbar promises a buss like connectivity. Anything can talk directly
> to anything else. Logically, that is perfect. Physically I see
> connectivity as a hierarchy. Any dealer piece that is missing will
> isolate two networks. This is the basis of my concern.

The issue of network splits ("split-brain") in clustered architectures
is a tricky one.

This is even more true since with WAMP we really want to be able to form
clusters connected over WAN connections.

I cannot give you a final answer right now .. we'll see. But you are
right: this ("split brain") needs to be addressed.

/Tobias

>
> Best,
> -g
>
>
> Cheers,
> /Tobias
>
> >
> > -g
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Autobahn" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send

> > an email to autobahnws+...@googlegroups.com <javascript:>
> > <mailto:autobahnws+...@googlegroups.com <javascript:>>.

> > To post to this group, send email to autob...@googlegroups.com

> <javascript:>
> > <mailto:autob...@googlegroups.com <javascript:>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/autobahnws/7f83fc4d-27af-4659-8a1c-0f44237ea51d%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/7f83fc4d-27af-4659-8a1c-0f44237ea51d%40googlegroups.com>
>
> >

> <https://groups.google.com/d/msgid/autobahnws/7f83fc4d-27af-4659-8a1c-0f44237ea51d%40googlegroups.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/autobahnws/7f83fc4d-27af-4659-8a1c-0f44237ea51d%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com

> <mailto:autobahnws+...@googlegroups.com>.

> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autobahnws/382cc42d-d56d-47d5-bf89-7e03564f9984%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/382cc42d-d56d-47d5-bf89-7e03564f9984%40googlegroups.com?utm_medium=email&utm_source=footer>.