gRPC java multiple bi-directional streams share same channel is faster than 1 channel per streaming

[eleano...@gmail.com]

Hi, 

I am doing some experiment with gRPC java to determine the right gRPC call type to use. 

here is my finding:

creating 4 sets of StreamObservers (1 for Client to send request, 1 for Server to send response), sending on the same channel is slightly after than sending on 1 channel per stream.

I have already elimiated the time of creating initial tcp connection by making a initial call to let the connection to be established, then start the timer. 

I just wonder why this is the case?

Thanks!

[Carl Mastrangelo]

gRPC does connection management for you.  If you don't have any active RPCs, it will not actively create connections for you.  

You can force gRPC to create a connection eaglerly by calling ManagedChannel.getState(true), which requests the channel enter the ready state. 

Do note that in Java, class loading is done lazily, so you may be measuring connection time plus classload time if you only measure on the first connection.

[eleano...@gmail.com]

Hi Carl, 

Thanks for the explanation, however, that still does not explain why using single tcp for multiple streamObserver is faster than using 1 tcp per stream. 

[Carl Mastrangelo]

I may have misinterpretted your question; are you asking why gRPC prefers to use a single connection, or why you observed using a separate TCP connection per stream was faster?

If the first, the reason is that the number of TCP connections may be limitted.   For example, making gRPC requests from the browser may limit how many connections can exist.   Also, a Proxy between the client and server may limit the number of connections.   Connection setup and teardown is slower due to the TCP 3-way handshake, so gRPC (really HTTP/2) prefers to reuse a connection.

If the second, then I am not sure.   If you are benchmarking with Java, I strongly recommend using the JMH benchmarking framework.  It's difficult to setup, but it provides the most accurate, believe benchmark results.    

[eleano...@gmail.com]

Hi Carl, 

Thanks for the very detailed explanation! my question is why I observed using a separate TCP connection per stream was SLOWER!

If the single TCP connection for multiple streams are faster (Regardless the reason), will the connection get saturated? e.g. too many streams sending on the same TCP connection.

[Srini Polavarapu]

Could you provide some stats on your observation and how you are measuring this? Two streams sharing a connection vs. separate connections could be faster due to these reasons:

- One less socket to service: less system calls, context switching, cache misses etc.

- Better batching of data from different streams on a single connection resulting in better connection utilization and larger av. pkt size on the wire.

[eleano...@gmail.com]

Hi Srini, 

The way how I do it:

for single connection:

1. send 1 request via  request StreamObserver, to let initial connection established 

2. start the timer, send 10000 requests

3. end the timer when see all results from the response StreamObserver.onNext() that the client passed to the server. the logic is just System.out.println

for multiple connections:

1. send 1 request for each channel created, to let initial connection established

2 start the timer, send 1000 per connection, total 10 connections, so total 10000 requests

3. end the timer when see all the results from the response StreamObserver.onNext() that the client passed to the server, for all connections, the logic is just System.out.println

Thanks!

for multiple connection:

[eleano...@gmail.com]

The single connection time for the whole process is about 2.75 sec, and the multiple connection time is about 3.5 sec. I run this test multiple times, and the single connection is always a bit faster than the multiple connections

On Saturday, August 18, 2018 at 8:37:22 PM UTC-7, Srini Polavarapu wrote:

[Carl Mastrangelo]

Can you show your code?   This may just be a threading problem.  

[eleano...@gmail.com]

Hi Carl, 

It is hard to show my code as I have a wrapper API on top of grpc. 

However, are you suggesting that using 1 tcp connection per stream should be faster than using 1 tcp connection for all streams?

[Carl Mastrangelo]

1 TCP connection for all streams (really RPCs) should be faster.