grpc C++ client fail to reconnect (leaking file descriptors)

[haiya...@dejero.com]

Hello,

I am working a grpc client application (C++) to talk to grpc server on remote:

1. First I use grpc::CreateChannel(...) to create a new channel instance and it is successful:

    auto  spChannel = grpc::CreateChannel("unix:/tmp/test_sock", grpc::InsecureChannelCredentials());

2. Then I check channel state:

    auto chanState = spChannel->GetState(false);

    if (chanState == GRPC_CHANNEL_IDLE)

    {

        spChannel->GetState(true);  // this will trigger connect to server

    }

3. It could be server side is not ready yet, as such the connect would fail. Then I remove old channel and

    create a new one in step 1).

     spChannel = grpc::CreateChannel("unix:/tmp/test_sock", grpc::InsecureChannelCredentials());

4. Repeat step 2) - 3) until server is up (backoff timing is omitted).

The logic above works without problem when server comes up later. However, there is an issue when channel instance

is destroyed: the file descriptor id associated with channel is not freed. Let's say if server comes up in a long time, the following complaint

from grpc lib tells us that no file description id available eventually:

Jun 08 04:00:28 conn[25285]: E0608 04:00:28.076832140 25285 ev_epoll_linux.c:1825] add_poll_object: {" 
created":"@1496894428.076821026","description":"polling_island_create","file":"src/core/lib/iomgr/ev_epoll_linux.c","file_line":284,
"referenced_errors":[{"created":"@1496894428.076819933","description":"OS Error","errno":24,"file":"src/core/lib/iomgr/ev_epoll_linu 
x.c","file_line":561,"os_error":"Too many open files","syscall":"epoll_create1"}]}{quote} 

Tracing code to src/core/lib/iomgr/ev_epoll_linux.c, the function polling_island_create() is the one to create file descriptor id (epoll_id).

However, the polling_island_delete(...) should be called when channel is destroyed as well. Otherwise, the file descriptor ids will be

leaked and led to the above error.

Has anyone seen this ? Or I didn't use the grpc channel properly ?

Thanks,

Haiyang Yin

[kaustubh.k...@gmail.com]

Even I'm facing a similar issue. Can someone please guide ?

[David Garcia Quintas]

Which version of grpc are you using?

[Sree Kuchibhotla]

Based on the file name (ev_epoll_linux.c), this seems like an older version (earlier than 1.4). Could you please upgrade to the latest version 1.7 ?

thanks,

Sree