Can more than one grpc service run in a single process?

[John Langley]

Hope this isn't too naive a question, I've only been playing with grpc for an hour or so. 

Although grpc seems great for "microservices" where each service is running in it's own process. Would you expect any challenges if the architecture required multiple services to be running in the same process? I'm particularly interested in the C++ implementations at this point, but Java is interesting too. 

TIA

-- Langley 

[Eric Anderson]

On Tue, Mar 10, 2015 at 1:57 PM, John Langley <dig...@gmail.com> wrote:

Although grpc seems great for "microservices" where each service is running in it's own process. Would you expect any challenges if the architecture required multiple services to be running in the same process?

gRPC absolutely supports that. All the gRPC implementations support multiple services in the same process and port. All implementations also support using different ports for each the service, if you have a reason to prefer that.

[Eric Anderson]

More info for Java:

You basically just need an addService line for each service. It is possible to even change the supported services on-the-fly by specifying your own HandlerRegistry. Other implementations may not support changing the services without bringing up a new server instance (but could still be in the same process).

[keu...@gmail.com]

But you can't currently run multiple instances of the same service on the same port, right? At least not with the standard gRPC+protobuf idiom? Or am I missing something?

To elaborate a bit, in a traditional REST-style API, you can instantiate multiple instances of a handler and map them to multiple HTTP request paths off the same port. For example, your service might route requests to

myserver:80/myapi/foo

myserver:80/myapi/bar

to dynamically created instances of the same handler code (which might, for example, be configured differently at runtime).

Judging by the gRPC+protobuf code samples I've seen, to accomplish something similar, you'd have to define multiple service names in the protobuf IDL code, because the standard message -> handler dispatching logic just looks at ServiceName.MethodName. In statically typed languages, at least, this means you have to write O(N) shims which are declared to implement each service interface (or do the equivalent using runtime code generation).

Or, somewhat better, you could define a "union" service which contains, for every method M in your underlying service with argument message type T, a method M' which takes an argument of type

message T_prime {

  string path = 1;

  T payload = 2;

}

and implement your own dispatching. Obviously, this comes with the downside that it must be repeated for every RPC type you wish to serve in this fashion.

Or you could put the service instances on different ports, but if you map a lot of instances of the service you end up consuming a lot of ports. This has other disadvantages: port numbers are less mnemonic than paths, and you may have to reconfigure firewalls to open up ranges of ports rather than individual ports.

It seems to me that this limitation is not fundamental to the protocol, but an artifact of the fact that the request path for gRPC+proto happens to have the fixed form "/ServiceName.MethodName". On the wire, the path is just a string, right? Are there any plans to provide support for "mounting" protobuf service implementations at non-root paths?

~k

[Eric Anderson]

On Tue, Mar 31, 2015 at 8:08 PM, <keu...@gmail.com> wrote:

But you can't currently run multiple instances of the same service on the same port, right?

Ah. Yes, approximately so.

We do have support on the client to prepend a prefix to the path used for the RPC, primarily to deal with reverse proxies. However, it isn't inconceivable allowing the same thing happening on server-side.

You could actually do it today by creating your own ServerServiceDefinition, maybe based on the one returned from the bindService() codegen'd method, but it would be a bit annoying.

At least not with the standard gRPC+protobuf idiom? Or am I missing something?

No, you aren't missing anything. That's not really how the RPC system is intended to be used.

 

To elaborate a bit, in a traditional REST-style API, you can instantiate multiple instances of a handler and map them to multiple HTTP request paths off the same port. For example, your service might route requests to

myserver:80/myapi/foo

myserver:80/myapi/bar

to dynamically created instances of the same handler code (which might, for example, be configured differently at runtime).

gRPC ≠ REST. gRPC is also not object-oriented conceptually; the RPCs are more of functions than methods. In gRPC, all input parameters are via the messages.

Judging by the gRPC+protobuf code samples I've seen, to accomplish something similar, you'd have to define multiple service names in the protobuf IDL code, because the standard message -> handler dispatching logic just looks at ServiceName.MethodName. In statically typed languages, at least, this means you have to write O(N) shims which are declared to implement each service interface (or do the equivalent using runtime code generation).

No, you wouldn't do this. Way to costly. If you really wanted this, then you would muck with ServerServiceDefinition.

Or, somewhat better, you could define a "union" service which contains, for every method M in your underlying service with argument message type T, a method M' which takes an argument of type

message T_prime {

  string path = 1;

  T payload = 2;

}

and implement your own dispatching. Obviously, this comes with the downside that it must be repeated for every RPC type you wish to serve in this fashion.

This is at the opposite end of the spectrum and also would be weak.

I believe there is the middle ground, which is how we would expect things to be organized.

Consider the GitHub Repo API. You issue a GET to /users/:username/repos list the repos for that user. To quickly model this as an RPC:

message ListRequest {

  string username = 1;

}

service RepoService {

  rpc List(ListRequest) returns (ListResponse);

}

Or you could put the service instances on different ports, but if you map a lot of instances of the service you end up consuming a lot of ports. This has other disadvantages: port numbers are less mnemonic than paths, and you may have to reconfigure firewalls to open up ranges of ports rather than individual ports.

The core problem is why do you need to expose multiple instances of the same service? Normally with gRPC, there aren't implicit arguments. Arguments in the path instead get placed in the request message.

Is there an example of when you need the same service multiple times?

[keu...@gmail.com]

On Wednesday, April 1, 2015 at 9:50:17 AM UTC-7, Eric Anderson wrote:

On Tue, Mar 31, 2015 at 8:08 PM, <keu...@gmail.com> wrote:

To elaborate a bit, in a traditional REST-style API, you can instantiate multiple instances of a handler and map them to multiple HTTP request paths off the same port. For example, your service might route requests to

myserver:80/myapi/foo

myserver:80/myapi/bar

to dynamically created instances of the same handler code (which might, for example, be configured differently at runtime).

gRPC ≠ REST. gRPC is also not object-oriented conceptually; the RPCs are more of functions than methods.

gRPC, like every RPC system I know of, is inescapably object-oriented in the sense that an RPC has a receiver. Currently, in gRPC+proto, the receiver is named by the host:port/servicetype of the target. I propose that it should be easy to locate multiple receivers of the same type on the same host:port by distinguishing them with a path prefix.

The core problem is why do you need to expose multiple instances of the same service?

It is natural to define an abstract service interface based on the methods it provides, and only later decide on the cardinality of service instances and the mapping of those service instances onto network resources. You might want to locate multiple instances of the same service type on a host:port for reasons similar to the reasons that you might want to run on multiple ports on the same host, or indeed to have multiple hosts in the first place.

Is there an example of when you need the same service multiple times?

As I noted in my previous post, and as your Github example reiterates, every such example can be rephrased by defining for the underlying message type T an alternative message type T' that explicitly encodes the path (and/or any variable parameters that would be passed in the path) as a message field.

However, I suspect that there will be cases where this is more cumbersome than simply allowing users to mount service instances at non-root paths. You may find it suggestive that essentially every HTTP server request routing framework allows you to map multiple routes on a host to differently configured instances of the same handler code.

~k

[Josh Humphries]

I think that's because of GET methods and REST semantics over HTTP. For GET messages, the standard way to parameterize is via path or query string, so mounting on different paths is a reasonable need. Similar for REST, where arguments that describe the resource being accessed are in the path (with query string often used for query criteria in retrieval requests).

Your use case description is very general. Perhaps you have something more concrete that explains why using request message fields is unsuitable?

For just enumerating resources that expose the same interface, it would work more-or-less like you do it in non-OO paradigms -- pass the receiver as an explicit parameter (just like Eric already mentioned). To enumerate such resources, add and implement a "list" method in the service.

If this is an extremely common use case, you might be able to reduce the boiler-plate with custom libraries or maybe even AOP (for example, declaring custom options on the methods and using interceptors on the server to implement something more general or cross-cutting).

 

~k

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/47b96fd6-0455-49d6-808d-3fb19c3a095e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Eric Anderson]

On Wed, Apr 1, 2015 at 9:49 PM, <keu...@gmail.com> wrote:

gRPC, like every RPC system I know of, is inescapably object-oriented in the sense that an RPC has a receiver. Currently, in gRPC+proto, the receiver is named by the host:port/servicetype of the target.

D-Bus, COM, CORBA, and RMI are object-oriented RPCs. You can get objects and call remote methods on those objects. Objects implement interfaces, etc.

HTTP/REST is also roughly object-oriented as you have methods (GET/POST/PUT) applied to resources (URLs). You also return references to other objects via URLs in the response.

But gRPC passes structures (data only). Servicetype/method is the method name; the servicetype acts as namespace. There is a receiver of host:port, but treating that like an object leads to the issues you were describing.

Hopefully that and Josh's response helps your mental model of how you can most easily use gRPC. What you are hoping for may be possible, but I don't think you would be happy with the results given gRPC's design.

[falco...@gmail.com]

Sorry for digging this topic up, but I have a similar use case. I have a framework, containing a modular set of components. Each component implements a couple of the service definitions (sharing only some configuration). Each service defines a set of methods, that only depend on the input and the configuration of the component. If only looking at one component, it's pretty easy to use protocolbuffer to implement this. If I want to use GRPC, I would have to use a grpc-server with a unique host-port-combination for each component and use the framework only as a coordinator instead of the main-server. The only thing I'm missing, is the possibility to add a prefix on the server-side to separate them logicaly instead of having to use multiple ports (which is a real pain if you have to open them all in the firewall at customers site). So I would prefer to have a single port per framework-installation, being able to let them comunicate with each other.

 

The background of the architecture is, that it is a modular system, where you can chain the components to build a processing chain. There can be methods in a service that process the data and reply with a structure (as GRPC intends it to be) or to relay the reply to the next component in the chain (by calling a method of thats service) and only reply to the caller a status. The chaining is usally done by configuration. It is not really object-oriented and more a set of microservices, since the reply (in most cases) only depends on the configuration and the input. That's why I like the design of GRPC and using it over using RMI/REST. Especially since most of the features offered (sync/async, lameducking, streaming, ...) and the close integration with protocol buffer solve a lot of problems that I would have to solve myself otherwise.

 

Right now, this architecture is already in use for around 8 years and works pretty good. In most cases, the modules only have to be plugged together and configured to satisfy the customers needs. At the moment, the complete communication logic is done by a properitary system, but since the requirements grow, I would like to switch to a framework like GRPC, so that we don't have to maintain this by ourselfes.

[Eric Anderson]

It sounds like you could have your framework implement the various services, and include some identifier for the module to select in the requests. Then, for each request delegate to the appropriate module based on the identifier in the request.

Yes, that is a different organization, but it should function about the same.

--

You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/a374af44-d06f-4fe3-bfce-a30550e596f0%40googlegroups.com.

[falco...@gmail.com]

This would work, but then I would need to have the identifier added to every request and also need to find a way for accessing this identifier, since the framework is generated once and not with every build. Therefor it doesn't know the generated classes and can only work on the serialized data. Would be more work to achieve this.

Right know, I added a layer between the ServiceDevinitions list and the server. So the server contains a list of components instead of ServiceDefinitions, and the definitions are know in my component class. The component class only contains the prefix added to all service handles and can be activated and deactivated. Since modules can be added at runtime in my framework, I need to change the list of services supported at runtime. The server implementation does not allow this, since the normal use case would be to stop the server. So if I would want to use multiple ports, I still could do everything within the regular standard. But this isn't possible, even if I prefer it since it makes everything easier.

[Louis Ryan]

On Tue, Dec 1, 2015 at 10:19 PM, <falco...@gmail.com> wrote:

This would work, but then I would need to have the identifier added to every request and also need to find a way for accessing this identifier, since the framework is generated once and not with every build. Therefor it doesn't know the generated classes and can only work on the serialized data. Would be more work to achieve this.

Right know, I added a layer between the ServiceDevinitions list and the server. So the server contains a list of components instead of ServiceDefinitions, and the definitions are know in my component class. The component class only contains the prefix added to all service handles and can be activated and deactivated. Since modules can be added at runtime in my framework, I need to change the list of services supported at runtime. The server implementation does not allow this, since the normal use case would be to stop the server. So if I would want to use multiple ports, I still could do everything within the regular standard. But this isn't possible, even if I prefer it since it makes everything easier.

Why not create your own implementation of HandlerRegistry which resolves components based on the path prefix passed via methodName and the delegates the lookup to a component (which itself can implement HandlerRegistry) ?

 

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/2fe55400-6db5-4ac2-8820-024e7076d4d8%40googlegroups.com.

[falco...@gmail.com]

Sounds like a good option. But as far as I can see, the HandlerRegistry and the InProcess things are (right now) only found in the Java implementation (or at least not in C# which I am using). So I hope they get carried over :)

Thanks for the Tip!

[Louis Ryan]

Opened tracking issue https://github.com/grpc/grpc/issues/4287

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/9689ab85-7401-424b-b4d5-a07b9a38fdec%40googlegroups.com.

[Niklas Schnelle]

Hey gRPC community, so I think I've got a similiar problem when trying to think up a good gRPC API for my IoT server which would replace its current REST interface.
The scenario I'm working on involves a server/home hub thing that controls any number of devices, currently mostly different kind of lamps.

So on the one hand there is a service that just provides listings of devices which is easily mapped to gRPC. But then every device provides functionality too,
with both the number of devices only known at runtime and several different device types with a hierarchy. In Go the device types are modeled like this:

https://github.com/niklas88/feel-at-home-server/blob/master/devices/lampbase.go

So now I'm wondering how to best map this to a gRPC API. The only thing I have come up with until now is having one controlling service for each device type

which takes some kind of deviceId as one of each of its methods arguments and then uses that to lookup the device. I'm not sure whether I have missed some kind of

mixin concept but I guess each service would also need to copy all methods the device supports? I'm sure I could wrap this in a nice API but it still feels like somewhat of

a clutch so I'd be thankful for suggestions.

[Louis Ryan]

On Fri, Dec 18, 2015 at 12:22 PM, Niklas Schnelle <niklas....@gmail.com> wrote:

Hey gRPC community, so I think I've got a similiar problem when trying to think up a good gRPC API for my IoT server which would replace its current REST interface.
The scenario I'm working on involves a server/home hub thing that controls any number of devices, currently mostly different kind of lamps.

So on the one hand there is a service that just provides listings of devices which is easily mapped to gRPC. But then every device provides functionality too,
with both the number of devices only known at runtime and several different device types with a hierarchy. In Go the device types are modeled like this:

https://github.com/niklas88/feel-at-home-server/blob/master/devices/lampbase.go

So now I'm wondering how to best map this to a gRPC API. The only thing I have come up with until now is having one controlling service for each device type

That would be a very reasonable approach. i.e have a LampController and then ToasterController etc.

 

which takes some kind of deviceId as one of each of its methods arguments and then uses that to lookup the device. I'm not sure whether I have missed some kind of

mixin concept but I guess each service would also need to copy all methods the device supports? I'm sure I could wrap this in a nice API but it still feels like somewhat of

a clutch so I'd be thankful for suggestions.

--

You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/b6653e4b-4954-4921-9d82-0b0471673d62%40googlegroups.com.

[Eric Anderson]

On Fri, Dec 18, 2015 at 12:22 PM, Niklas Schnelle <niklas....@gmail.com> wrote:

So on the one hand there is a service that just provides listings of devices which is easily mapped to gRPC. But then every device provides functionality too,
with both the number of devices only known at runtime and several different device types with a hierarchy. In Go the device types are modeled like this:

https://github.com/niklas88/feel-at-home-server/blob/master/devices/lampbase.go

So now I'm wondering how to best map this to a gRPC API. The only thing I have come up with until now is having one controlling service for each device type

which takes some kind of deviceId as one of each of its methods arguments and then uses that to lookup the device.

That could look like:

Device.enumerate

Device.power

DimLamp.Brightness

DimLamp.Fade

...

ColorLamp.Color

ColorLamp.ColorFade

...

Effectively, each interface becomes a service, and each method other than enumerate would have the deviceId like you mentioned.

That would be a very similar design to using net/rpc.

I think you are taking an OO-approach. An similar but alternative approach would be feature-based, where there is no "is-a" relationship but instead a "supports" relationship. That would produce a highly-similar set of methods, but may be more natural conceptually. For example, enumerate can provide the list of interfaces each device supports, instead of just providing the singular "type" of a device and the client needing to know about the type hierarchy. This is actually more similar to how classes/interfaces work in Go.

[Eric Anderson]

On Tue, Mar 21, 2017 at 3:23 AM, <ale...@gmail.com> wrote:

For C++ gRPC I do not currently see any option for this, I cannot add more than one service in builder.RegisterService.  

Is there some other way to do this

It's not polite to resurrect really old threads. In the future, creating a new thread and just linking to the old one would work better.

I'm not an expert of the C# API, but why can't you just call RegisterService multiple times, one for each service?

[falco...@gmail.com]

Hey Eric. He asked about the C++ API and not C# :) I didn't answer since I don't know the C++ API.

[ale...@gmail.com]

On Thursday, March 23, 2017 at 11:27:49 AM UTC+5:30, falco...@gmail.com wrote:

Hey Eric. He asked about the C++ API and not C# :) I didn't answer since I don't know the C++ API.

I thought I had deleted my post; Realize now that mail would have been triggered. Initially because of another error from Cient side ,I thought calling RegisterService twice was not working. But it is working thanks; 

builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());
builder.RegisterService(&obj_detect_svc);
builder.RegisterService(&serverCheck);
std::unique_ptr<Server> server(builder.BuildAndStart());

(I opened this thread as Google search was pointing this as the most relevant and thought updating it may help others (sorry was using this like StackOverflow) )

[Eric Anderson]

On Wed, Mar 22, 2017 at 11:09 PM, <ale...@gmail.com> wrote:

On Thursday, March 23, 2017 at 11:27:49 AM UTC+5:30, falco...@gmail.com wrote:

Hey Eric. He asked about the C++ API and not C# :) I didn't answer since I don't know the C++ API.

I don't know where I saw C#, but I did realize that the RegisterService API was C++. So I just sort of answered in a way that didn't matter which was being used :)

I thought I had deleted my post; Realize now that mail would have been triggered. Initially because of another error from Cient side ,I thought calling RegisterService twice was not working. But it is working thanks; 

builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());
builder.RegisterService(&obj_detect_svc);
builder.RegisterService(&serverCheck);
std::unique_ptr<Server> server(builder.BuildAndStart());

(I opened this thread as Google search was pointing this as the most relevant and thought updating it may help others (sorry was using this like StackOverflow) )

It's fine to post to an old thread if it is fixed later, but generally that should be done by someone in the original thread since they are in a position to know how it was resolved. A lot happens in a year and so questions that appear similar frequently have different causes. And if you see, this thread had already devolved from the original question. Starting a new thread reduces the confusion in figuring out what is being talked about.

StackOverflow has less issue with threads "morphing" and discussing many different things.