transport.newBufWrite go grpc not releasing memory

[Bhaskar Rana]

HI,
I have  sever app  (golang) which provides streaming connection to client.
I'm running few clients which make grpc stream connection with server.  I continuously

restart client by doing kill.After 30 minutes, on server I see memory is leaked by  transport.newBufWrite Has the pporf output.

Has anybody seen such issue ?

(pprof) top

Showing nodes accounting for 505.10MB, 97.08% of 520.31MB total
Dropped 56 nodes (cum <= 2.60MB)
Showing top 10 nodes out of 27
      flat  flat%   sum%        cum   cum%
  331.91MB 63.79% 63.79%   331.91MB 63.79%  google.golang.org/grpc/internal/transport.newBufWriter (inline)
  153.19MB 29.44% 93.23%   153.19MB 29.44%  bufio.NewReaderSize (inline)
       8MB  1.54% 94.77%        8MB  1.54%  golang.org/x/net/http2/hpack.(*headerFieldTable).addEntry (inline)
    5.50MB  1.06% 95.83%     7.50MB  1.44%  google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders
    4.50MB  0.87% 96.69%   491.60MB 94.48%  google.golang.org/grpc/internal/transport.NewServerTransport
       2MB  0.38% 97.08%     3.16MB  0.61%  google.golang.org/grpc.(*Server).processStreamingRPC
         0     0% 97.08%        5MB  0.96%  golang.org/x/net/http2.(*Framer).ReadFrame
         0     0% 97.08%        5MB  0.96%  golang.org/x/net/http2.(*Framer).readMetaFrame
         0     0% 97.08%        5MB  0.96%  golang.org/x/net/http2/hpack.(*Decoder).Write

         0     0% 97.08%        5MB  0.96%  golang.org/x/net/http2/hpack.(*Decoder).parseFieldLiteral

Thanks

B

[dfa...@google.com]

Please try enabling keepalive on your server so it will notice the connection was lost:

https://pkg.go.dev/google.golang.org/grpc/keepalive#ServerParameters (set Time to something like 30 seconds - 5 minutes depending on your needs)

https://pkg.go.dev/google.golang.org/grpc#KeepaliveParams