Deploying GRPC in production and monitoring

[Mahesh Lal]

Hi,

We are using GRPC for our internal services. Currently, we package the service as a Jar and running it directly using the JVM (java -jar <jar name>). We aren't using any application server like Jetty or Netty. In a production environment, is that okay?

We use New Relic through out our infrastructure for monitoring. Is anyone aware of how to integrate New Relic with Grpc based services which arent deployed on any App server. Do we need to deploy Grpc services to an App server at all?

-- Thanks and Regards

   Mahesh Lal

[Eric Anderson]

I can't answer at all about New Relic.

On Fri, Jun 10, 2016 at 4:13 AM, Mahesh Lal <mahes...@gmail.com> wrote:

We are using GRPC for our internal services. Currently, we package the service as a Jar and running it directly using the JVM (java -jar <jar name>). We aren't using any application server like Jetty or Netty. In a production environment, is that okay?

Yep, that sounds fine.

Do we need to deploy Grpc services to an App server at all?

No, it's not necessary. gRPC doesn't really care. If you're already using an App server, that's fine. If you just run binaries directly, that's fine. 

[Mahesh Lal]

Thanks for the reply Eric. 

I got my New Relic question answered by a bit of experimentation.

I can definitely use New Relic to monitor the gRPC service simply by setting the -javaagent:/fully/qualified/path/to/newRelic.jar even if I am running it using the JVM directly.

-- Thanks and Regards

   Mahesh Lal

[Alex Borysov]

Hi Manesh,

For New Relic, you might need to use experimental java agent to use Netty 4: NewRelic netty-4.0

Thanks,

Alex Borysov

[Selcuk Bozdag]

Hi,

We just started experimenting with gRPC services. As Manesh, we also created a number of services(in form of executable jar files) which they serve individually. 

We are coming from a world of monoliths where each stack is something deployable on an app server and now looking for the best practices around to figure out how to run our services.

Any ideas or recommendations?

Thanks 

[Mahesh Lal]

This could be a long discussion. I am listing down some of the things we follow:

1) We use gRPC only for internal API calls. Anything that is exposed to the outside world is done via a REST wrapper. 

2) Use Consul for service discovery and use channel pooling for load balancing. 

3) If you are using Newrelic, you can put the @Trace annotation on top of each service method so that it gets captured. 

For all practical purposes, our gRPC services run okay on 4 core 8GB machines. Be extra careful around error handling though, and also take care when configuring channel pool sizes.

Let me know if you want to discuss things in further details. 

-- Thanks and Regards

   Mahesh Lal

[Selcuk Bozdag]

We are now starting our server by:

ServerBuilder.forPort(port)
 .addService(new Service_A_Impl())
 .addService(new Service_B_Impl())
 .addService(new Service_C_Impl())
 .addService(new Service_D_Impl())
 .addService(new Service_E_Impl())
 .addService(new Service_F_Impl())
 .build()
 .start();

Is it okay? What are the pros and cons of starting the server like that? Sorry if it sounds a lot newbie.

Thanks

[Evan Jones]

This doesn't seem unreasonable, if it makes sense to have these different services running in a single process, for whatever reason. This is probably a bit more on the "monolith" end of the architecture spectrum instead of the "independent services" end. This is a potentially enormous discussion, but some of the trade-offs I can think of off the top of my head are:

* A single process means there are fewer things to build, deploy and monitor, which reduces configuration overhead.

* You have to deploy changes to the services together, which can require some careful coordination to ensure you don't break ServiceB while you are deploying a change to ServiceD.

This is probably a fine way to get started though. It lets you get the infrastructure for building a client and a service working, before adding the complexity of managing independent processes/services. As this monolith grows, you can split the services apart, or add new independent services.