[Question] Using grpc compression weirdly leads to low throughput

[Hao Jiang]

Hi,

I have a service using grpc to stream data to client-side.
For the original implementation, the average throughput could be ~110MBps with a 1Gbps NIC, so I think bandwidth itself is a bottleneck. The usual way is to enable compression/decompression.

I found grpc supports compression natively. The only line of code I added at server side is

builder.SetDefaultCompressionLevel(GRPC_COMPRESS_LEVEL_LOW);

with no change at client-side. But the throughput drops stably.

With dstat -n, the throughput looks like

33M 121k 2948B 1756B 306B 166B 3026B 688B

which looks like the average throughput is 7MBps.
And with response.ByteSize(), I calculated the throughput is ~11MBps, still way below the one before compression.

The observation doesn't change as the compression level, whether it's low, medium or high.

I want to know, if I'm using the compression in grpc wrongly?

Thanks!

Best,

Hao

[Mark D. Roth]

It's hard to say what's going on here without knowing more about your environment.  A few things to consider:

• Compression is negotiated between client and server based on what algorithms are supported by each one, as described in https://github.com/grpc/grpc/blob/master/doc/compression.md#compression-levels-and-algorithms.  You might want to check to see what compression is actually being negotiated on the wire.  In C-core, you can try setting the env vars GRPC_VERBOSITY=DEBUG and GRPC_TRACE=compression to enable additional trace logging for the compression code.
• If your client and server are having CPU bottlenecks, then adding compression can slow things down, because it basically trades off network usage for CPU usage.
• If your application is sending many small messages instead of fewer larger messages, then the CPU overhead of compression may be higher.

I hope this information is helpful.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/52e90489-0347-4614-824c-ceeac41c8086n%40googlegroups.com.

--

Mark D. Roth <ro...@google.com>
Software Engineer
Google, Inc.