Custom channels

[pavol.o...@gmail.com]

Hello. I looked at current release of grpc++ and I still do not understand how to create channel not based on HTTP2.

My examples are:

1. I want to build custom channel that instead of HTTP2 uses Inter-Process Communication (LPC on Windows).
2. I want to build custom channel that instead of HTTP2 uses DNS tunelling
   

Inside generated code, every reference to Channel is to grpc::Channel.

• Can I create custom channel and inject it into grpc?
  
• Can I have more implementations of Channel at once?
• Can I switch these implementations at runtime?

Thank you.

[Nicolas Noble]

The way it works is lower level than that. What you want is a transport implementation, not a channel implementation. Transports aren't exposed to the C++ interface, so you'll have to add them to the C core. This is located in https://github.com/grpc/grpc/tree/master/src/core/transport - and if you want a protocol that isn't HTTP2, this is where you need to start poking at. More specifically, look at https://github.com/grpc/grpc/blob/master/src/core/transport/transport_impl.h and this is where you'll see the interface used by the rest of the code to "talk protocol".

There's probably some plumbing needed to properly bubble up the transport switch to the C++ API however.

But given what you are talking about, it seems you rather want an endpoint instead. This might prove easier. It'll still use HTTP2 as an encapsulating protocol, but not on top of a TCP connection. What I am saying is that I am arguing that HTTP2 and "LPC" are two complementary things, not interchangeable. But TCP and LPC are.

This is the code that is located there instead: https://github.com/grpc/grpc/tree/master/src/core/iomgr - there is a Windows and a Linux / posix implementation of TCP endpoints. If you add another endpoint, the core will use your code instead, and the plumbing to change it at runtime should already be there and easier to use.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/e93585f6-30d2-462c-867b-6b6ff3b063de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pavol Ostertag]

Thank you for comprehensive answer. Comprehensive answers produce more questions, so here are mine :)

 

1. Is it planned to make that plumbing in C++ official in the future? (so that the plumbing does not need to be rewritten with every GRPC update)

2. If I am getting you right, you suggest tunneling HTTP2 through LPC or DNS. Interesting idea

3. Would it be similarly easy to replace serializer (or marshaller) with another one serializing into XML? (in our company the transfer to GRPC would be far easier if the change can be done in two-steps - clients first and backend later, when "GRPC client" deployments reach some level)

[Michael Larson]

just as a reference point, I'm using GRPC TCP for windows inter process comm and it's working fine for what I need.  The only annoying bit is there is a 10 second delay on disconnect sometimes, but I think this is a known bug that may have already been addressed in new versions.  The bonus from using regular TCP is I can split it to multiple servers or other operating systems at will in the future.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/028101d10079%2413dabc40%243b9034c0%24%40gmail.com.

[Nicolas Noble]

Well, so there's really 3 layers in gRPC.

1. There's what we call the endpoint, which in our case is currently TCP. The role of an endpoint is to send and receive bytes, and signal the upper layers of events, such as connection lost, or there's bytes ready to read. The API for that one is really simple, and I think we've reached the point where it's mature enough it's not going to change. So writing another endpoint should prove fairly stable over time. It's not a guarantee however, and we kind of reserve the right to break that API in the future. We don't have any plan to expose it above the C core. Meaning that we've made it so that one could provide a plugin to gRPC to add another endpoint. Now that plugin on your side could be written in any language you want, as long as it provides a C interface for the core to use. We might move this to the official, immutable API some time.

2. Then there's the transport, which right now is HTTP2. It's used to transport the RPC payloads, and also to transport metadata. These metadata are encoded using HTTP headers. Here's the actual details of that: https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md - if you wanted to change the transport, you'd also need to make sure the metadata are properly encoded and transmitted. Which is why I was suggesting NOT changing that layer, but rather the endpoint one, so you only have to implement the first layer that is pumping bytes from one side to another. You wouldn't have to worry about the metadata.

3. Finally there's the actual RPC payload. gRPC technically doesn't care about what's in that payload. Our examples and some upper layers of the code are using protobuf, but the C++ layer should let you use any kind of payload you want instead. The code should already be there to allow for that. More specifically, from my memory, there should be a generic template, and a protobuf-specific one, in the C++ headers. So, JSON, flatbuffers, or XML, what you want, just specialize the template.

On Tue, Oct 6, 2015 at 1:53 PM, Pavol Ostertag <pavol.o...@gmail.com> wrote:

[Saigut]

Hi, how to custom the endpoint, is there any document or example?  I want to manage the sending and receiving of bytes myself.
Thank you.

[Zach Reyes]

At least for gRPC GoLang, Custom Transports are not currently supported.

[Zach Reyes]

We use the standard library net.Conn, which you can think of as a TCP connection.