Danil,
As far as I know, the grpc-web library doesn't support gRPC's usual message compression - the decompression would need to occur in Javascript, where it would be too slow to be useful. If you have your proxy server (usually Envoy) set up correctly, this should work automatically because browsers always set the "Accept-Encoding" request header. If you look at the response headers in your browser's network inspector, you should see "Content-Encoding" set to gzip or br.
In your first attempt, you're correctly setting the grpc-accept-encoding header. I don't know the Java gRPC APIs, but it sounds like you're compressing the response correctly. Unfortunately, when the browser receives the response, it doesn't know how to decompress it and you're getting an error. The response can't decompress with standard gzip tools because it isn't valid gzip: following the gRPC protocol, each gzipped protobuf message is wrapped in five bytes of enveloping metadata. Because of the enveloping and the blob of gRPC-Web trailers at the end of the body, the response as a whole isn't valid gzip.
Your second attempt has the same problem. The correct response header is "Content-Encoding" (rather than "Content-Encryption"), but you end up with the same problem - gRPC's per-message compression doesn't produce the sort of compressed response that browsers expect.
Hope that helps,
Akshay