CreateInsecureChannelFromFd() - ownership of the given file descriptor.

[Krzysztof Rybak]

Hi,

I'm creating a grpc channel for the grpc client using function CreateInsecureChannelFromFd() and giving the file descriptor to it. 

Should I handle the connection for the given file descriptor afterwards in application or is it handled in grpc framework?

For example on application side when messages are not delivered I reconnect using shutdown() or close() etc. 

Is it necessary? From strace log I found multiple calls to close() on the same fd like some are called by application and some by grpc framework.

Sorry, if something is written badly - my first post here.

[Mark D. Roth]

gRPC takes ownership of the fd when you pass it to CreateInsecureChannelFromFd(), so you don't need to shut it down or close it.

[Krzysztof Rybak]

So what is the proper way to run grpc based on fd-s ? I use grpc tagged v1.19.1. 

The problem is that when I release the fd on the client side in CreateInsecureChannelFromFd(), the client doesn't reconnect when server is restarted. From strace log: the client does a shutdown() on the given fd and doesn't recover from that state. 

In my application I do close() on the fd passed to CreateInsecureChannelFromFd(), create a new socket and pass it to CreateInsecureChannelFromFd(), but from the previous response this is not a solution.

I also tried (on client side) not to close the passed fd, but create a new socket and pass fd to CreateInsecureChannelFromFd() - this caused assertion errors on grpc.

Also tried just leave it without any handling but looks like grpc client cannot recover in case of for example server restart.

The simple example with one client (based on helloworld example) is below, is that way correct? (I omitted intentionally checking return codes and other part of the program).

server side:

  int server_fd;
  struct sockaddr_in address;

  server_fd = socket(AF_INET, SOCK_STREAM, 0);

  address.sin_family = AF_INET;
  address.sin_addr.s_addr = INADDR_ANY;
  address.sin_port = htons(PORT);

  bind(server_fd, (struct sockaddr *)&address, sizeof(address);
  listen(server_fd, 0);

  int flags = fcntl(server_fd, F_GETFL);
  fcntl(server_fd, F_SETFL, flags | O_NONBLOCK);

  GreeterServiceImpl service;
  ServerBuilder builder;
  builder.RegisterService(&service);
  std::unique_ptr<Server> server(builder.BuildAndStart());

  int client_fd = -1;
  while(1){
    client_fd = accept(server_fd, nullptr, nullptr);
    if (client_fd == -1 ) continue;

    flags = fcntl(client_fd, F_GETFL);
    fcntl(client_fd, F_SETFL, flags | O_NONBLOCK);
	
    ::grpc::AddInsecureChannelFromFd(server.get(), client_fd);

  }

client side:
  int client_fd;
  struct sockaddr_in serv_addr;

  client_fd = socket(AF_INET, SOCK_STREAM, 0);

  serv_addr.sin_family = AF_INET;
  serv_addr.sin_port = htons(PORT);

  inet_pton(AF_INET, "127.0.0.1", &serv_addr.sin_addr);

  connect(client_fd, (struct sockaddr *)&serv_addr, sizeof(serv_addr));

  int flags = fcntl(client_fd, F_GETFL);
  fcntl(client_fd, F_SETFL, flags | O_NONBLOCK);

  auto channel = grpc::CreateInsecureChannelFromFd(Greeter::service_full_name(), client_fd); 

    GreeterClient greeter(channel);

  while (1){
    // calling SayHello -> stub->SayHello and when status is not ok, should I handle it? 
  }

[Mark D. Roth]

What I said earlier is correct: once you pass an fd to CreateInsecureChannelFromFd(), you should not close it, because the channel has already taken ownership of it.  But it sounds like the question you're actually asking here is about why the channel doesn't automatically reconnect.

When you create a normal channel via CreateChannel(), you get a full client channel that contains all the logic for name resolution, load balancing, and connection management.  Since it contains the code to create the fd for each connection, it can automatically create new connections as needed; if the connection to a given address fails, it can try to establish a new connection.

In contrast, when you use CreateInsecureChannelFromFd(), you are creating what is called a "direct" channel.  Direct channels do not do name resolution, load balancing, or any connection management; they have just the one fd that you created, and when that fd no longer works, the channel is no longer usable.  A direct channel cannot automatically create a new connection when the fd you gave it fails, because it has no idea how you created the connection in the first place -- all it was given is the already-created fd.  So if the connection fails, you have to create a whole new channel by creating a new fd and passing it to CreateInsecureChannelFromFd() again.

You say that you tried calling CreateInsecureChannelFromFd() again and that you ran into assertion errors, but I don't know why that would happen.  If you can share your code and the assertion error you're seeing, maybe we can help tell you what's wrong.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/4db72044-dbd3-4171-8aa9-fe7d67cb0929%40googlegroups.com.

--

Mark D. Roth <ro...@google.com>
Software Engineer
Google, Inc.

[Krzysztof Rybak]

You're right, it works now. 

There was another bug with race condition in my code and I thought it's in grpc. 

Topic to be closed, thank You very much for support.  

To unsubscribe from this group and stop receiving emails from it, send an email to grp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/4db72044-dbd3-4171-8aa9-fe7d67cb0929%40googlegroups.com.