Configuring gRPC keep-alive

[Lajos Rancz]

Hi all,

I'm developing an IoT application, both the server (runs on a Digital Ocean droplet) and the client device application (runs on an embedded Linux) are written in Go. The internet connection between the devices and the server is provided by LTE Cat M1.
The application requires a live connection between the device and the server (because of real-time events), so I use server-side streaming:
message DeviceWatchRequest {
int32 token = 1;
}

message DeviceWatchResponse {
int64 id = 1;
uint32 type = 2;
bytes data = 3;
}

service DeviceService {
...
rpc Watch(DeviceWatchRequest) returns (stream DeviceWatchResponse);
}

The whole system works well, BUT the data usage is unacceptably huge, I send only a few bytes (DeviceWatchResponse.data) hourly. So, I've dumped the TCP traffic with the tcpdump utility.
What I see is that the vast majority of data usage is due to TCP keepalive:

So, a keep-alive operation consists of 4 packets, all four are 68 bytes long; eg. one keep-alive operation is 272 bytes long. There are 86400 seconds in a day, so 86400/15 = 5760 keep-alive operations are done in a day; it means 5760*272 = 1 566 720 bytes ~ 1.5 MB/day!
My whole monthly data usage could be that much, not my daily. As https://www.emnify.com/pricing: 1 MB/month is around 1 EUR/month/device, 50 MB is 2.65 EUR/month/device.
So, I've tried to set the keep-alive options on client side:
conn, err := grpc.Dial(*addr, grpc.WithInsecure(), grpc.WithKeepaliveParams(keepalive.ClientParameters{Time: 60 * time.Second, Timeout: 5 * time.Second, PermitWithoutStream: true}))
But after 3 minutes the connection is dropped. The keep-alive documentation (https://github.com/grpc/grpc/blob/master/doc/keepalive.md and https://pkg.go.dev/google.golang.org/grpc/keepalive) is contradictory.

What can I do?

Regards,
Lajos

[Lajos Rancz]

Hi!

I've run strace on the deivce, and I got:
setsockopt(3, SOL_TCP, TCP_KEEPINTVL, [15], 4) = 0

So the device process sets the TCP keep alive to 15 sec.

[Yuri Golobokov]

Hi Lajos,

The gRPC keep-alive configuration is only for the HTTP/2 keep-alive, not for the TCP keep-alive.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/27f8b79d-3f62-4916-881c-0c7f50d4e0cfn%40googlegroups.com.

[Lajos Rancz]

Hi Yuri,

Ok! In this case, the documentation should be more clear :/ So on both the device and server-side, I have to reach somehow the underlying TCP connection and set the timeouts correspondingly. I assume that on the device I have to use a custom dialer (grpc.WithContextDialer); and on the service, I have to use a custom listener (net.ListenConfig{KeepAlive: 5 * 60 * time.Second}). Don't I?

Regards,

Lajos