(gRPC-java) Why are all services singletons?

[Ryan Michela]

I'd like to know the design rationale for why gRPC services implementations are all concurrently executing singletons. There are many possible instancing and threading modes that could have been used.

• Singleton instancing
• Per-call instancing
• Per-session instancing

• Concurrent execution
• Sequential execution

Concurrent singletons make sense from an absolute throughput angle - no object instantiation or blocking. But concurrent singletons are hardest for developers to work with - service implementors must be keenly aware of shared state and mult-threading concerns. 

1. Why was concurrent singleton chosen as the only out-of-the-box way to implement gRPC (java) services? 
2. Would API for supporting other threading and instancing modes be accepted in a PR?

[Carl Mastrangelo]

What do you mean by Service?   There are hardly any places in our code where something is a singleton.  

[Ryan Michela]

I mean the instance of the class that implements my service operations. The instance you pass to ServerBuilder.addService(). 

Isn't that instance a singleton from the perspective of gRPC?

[Carl Mastrangelo]

No?  I don't know where you could have got that impression but you can make as many as you like, and share them between Servers as you please.

[Ryan Michela]

Each server can only reference one instance of a service implementation for the lifetime of the service, and all requests to that service are routed concurrently to that single, shared instance, correct? 

[Carl Mastrangelo]

Have you actually tried this?  Can you include an error showing that this is not possible?

[Josh Humphries]

I think this is referring to the fact that you bind a single server object for the life of the GRPC server.

Go: https://github.com/grpc/grpc-go/blob/master/server.go#L276

Java: https://github.com/grpc/grpc-java/blob/master/compiler/src/testLite/golden/TestService.java.txt#L167

So it's not singleton in a traditional pattern sense -- e.g. global/static singleton. But it is a singleton within the scope of a GRPC server.

This question has come up before. I think, in the past, it has been asked that URL prefixes could be used to route requests for the same service to different instances. For example, POST to "/service1/my.package.MyService/MyMethod" invokes myMethod on some server instance A, and "/service2/my.package.MyService/MyMethod" invokes it for a different instance.

I think the justification in the past has been that this would complicate the protocol as targeting specific implementations of the same service suddenly requires new behavior in both clients and servers. Instead, the recommended pattern is to use metadata (e.g. a header) and have an aggregate implementation re-dispatch to another implementation based on incoming metadata.

----
Josh Humphries
jh...@bluegosling.com

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/50eb23e0-4092-40f6-9f87-e5fb1a6251e2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Ryan Michela]

Josh, this is exactly what I am talking about.

Each service implementation is a singleton from the perspective of gRPC. You cannot have more than one service implementation instance handle requests from callers. You cannot get a fresh instance for every request.

Is there a specific gRPC design reason that limits the number of service implementation instances per server to one?

----
Josh Humphries
jh...@bluegosling.com

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.

[Carl Mastrangelo]

Ok, so what you mean to ask, is why can only one Method path be handled by a single implementation?  That is still not correct.  The existing code makes it easy to map a method to a handler in an immutable and thread safe way.  If you are willing to give up the speed benefits of this approach in favor of more flexibility, you can use ServerBuilder.fallbackHandlerRegistry and dispatch as you see fit.