Help with Compression on grpc

45 views
Skip to first unread message

grpc-noob

unread,
Mar 7, 2019, 4:57:44 PM3/7/19
to grpc.io
Hello folks,

I recently started using grpc and would like to clarify few doubts related to Compression.

I have looked at the following doc: gRPC Compression and wanted to check if my understanding is correct.

In our current use case, we send requests via REST Endpoints and we gzip our requests. Reduced Compression is close to 10-15 times than original. (eg. 70MB payload is compressed to 5MB payload)

In case of gRpc, eg. if grpc client sends entire 70MB payload using either channel compression (eg. blockingStub.withCompression("gzip")) or using high/medium level compressions, then does that mean

1. On the wire, it'll be compressed to 5MB payload and on the serve side when I open up the gRpc request, Payload will be of 70MB size ?

2. On the Server side, I do not have to explicitly mention that Server is going to handle Compressed and/or Uncompressed messages. But gRpc protocol will take care of Compressed and   
    Uncompressed format, by keeping server transparent of any of these formats. It'll only complain if it's not able to handle certain Compression format which Sever can NOT handle. Is this 
    assumption correct ?

3. Apart from gzip, what other compression formats are supported eg. Snappy ?

4. Is there an easy way to hook up Monitoring System like Prometheus on Individual Channel basis so that I can test or monitor above compression functionality by checking Throughput numbers on 
    the channel ?

Any help will be much appreciated

Thanks,

Eric Anderson

unread,
Mar 7, 2019, 6:32:01 PM3/7/19
to grpc-noob, grpc.io
1. On the wire, it'll be compressed to 5MB payload and on the serve side when I open up the gRpc request, Payload will be of 70MB size ?

Yes. 5 MB on-the-wire. The gRPC server will decompress the payload automatically for the server.

2. On the Server side, I do not have to explicitly mention that Server is going to handle Compressed and/or Uncompressed messages. But gRpc protocol will take care of Compressed and   
    Uncompressed format, by keeping server transparent of any of these formats. It'll only complain if it's not able to handle certain Compression format which Sever can NOT handle. Is this 
    assumption correct ?

Yes.

I'll note that the server's response will be independently compressed (or not). Whether a client sends a compressed request does not impact whether the response is compressed.

3. Apart from gzip, what other compression formats are supported eg. Snappy ?

It's basically just gzip today.

4. Is there an easy way to hook up Monitoring System like Prometheus on Individual Channel basis so that I can test or monitor above compression functionality by checking Throughput numbers on 
    the channel ?

It sounds like you are using Java. The tracing system in Java gets to see the wire vs uncompressed sizes for the messages. I can't say whether Prometheus has plugged into this functionality. But see ClientStreamTracer/ServerStreamTracer. If you want to do a quick-and-dirty test, decide whether you want to test on the client or server, implement a tracer (I'd just have it call System.out.println), and then enable it via ClientInterceptor+CallOptions.withStreamTracerFactory() or ServerBuilder.addStreamTracerFactory().

grpc-noob

unread,
Mar 9, 2019, 1:38:08 AM3/9/19
to grpc.io
Thanks Eric, I'll try what you have suggested.
Reply all
Reply to author
Forward
0 new messages