If our RPCs are implemented on top of TCP, when should we create new TCP connections?

[Max Marrone]

Broadly, I see two possibilities:

1. We create one TCP connection for each RPC we send.
2. We create one TCP connection per server and multiplex all RPC calls to that server on that single connection.

Option 1 makes it trivial to match RPC responses to their corresponding requests.  It also makes sense to me because algorithmically, each RPC round-trip is independent of all others.  However, it incurs the handshake overhead for each RPC.  In practice, I don't know how significant that overhead is.  I also don't know how well flow control works under this strategy.

Option 2 avoids the per-RPC handshake overhead.  But I'm worried that it would cause TCP's in-order delivery guarantee to work against us.  If one message to a server is dropped or delayed, subsequent messages will also be delayed, unnecessarily.  It also complicates the RPC layer, since we have to manually incorporate a way to match responses to requests.

What are your experiences?  Is one method clearly better than the other in practice?

[Oren Eini (Ayende Rahien)]

TCP connection times can be _long_. It is much better to have a single connection.

Note that you don't need multiplexing here. Instead, you have one TCP connection from the leader to each follower, and you read/write in order.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "raft-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raft-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.