Issue with Google cloud logging in Django application running on a container

[Abhijith R]

Hi All,

I have an application running on a docker container hosted internally, which uses cloud logging to log information to GCP. 

The application and logging was working fine for few minutes the I saw a SSL error similar to the one below.

"E1004 21:08:52.855508502 45 ssl_transport_security.cc:523] Corruption detected. "

"E1004 21:08:52.855608271 45 ssl_transport_security.cc:499] error:100003fc:SSLroutines:OPENSSL_internal:SSLV3_ALERT_BAD_RECORD_MAC "

"E1004 21:08:52.855621453 45 secure_endpoint.cc:205] Decryption error: TSI_DATA_CORRUPTED "

to fix this issue, I set the below env variables

ENV GRPC_POLL_STRATEGY "epoll1"

ENV GRPC_ENABLE_FORK_SUPPORT "1"

This fixed the above error but now I see a new error as below:

File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 498, in grpc._cython.cygrpc.Channel.segregated_call

File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 353, in grpc._cython.cygrpc._segregated_call

File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 357, in grpc._cython.cygrpc._segregated_call

ValueError: Cannot invoke RPC on closed channel!

Can someone guide me on how to approach this problem.

Thanks,

Abhijith

[Lidi Zheng]

I saw a similar ask from another channel. I will copy the answer here, so hopefully it can be useful for more people.

Forking support document can be found at: https://github.com/grpc/grpc/blob/master/doc/fork_support.md. gRPC channels won't be able to be used across processes (well, no sockets can). I'm not certain what the Django-q forking mechanism is. You probably seeing the closed channel in the child process, that is expected. I would recommend to only create gRPC objects after forking. For example:

```py
### Before ###
channel = grpc.insecure_channel(...)
stub = foo_pb2.FooStub(channel)

def handler():
  response = stub.Bar(...)


### After ###
def handler():
  channel = grpc.insecure_channel(...)
  stub = foo_pb2.FooStub(channel)
  response = stub.Bar(...)
```