gRPC vs RST a naive performance test

[Dharshana Ratnayake]

Hi guys..

Ive very recently discovered the world of gRPC and im super enthusiastic about how i can use it.. Just so i can understand the potential benefits. i set up the following benchmark test..

track 1: JSON over http1 using a spring boot app running on a google compute engine.

public static void main(String[] args){
    HttpClient httpClient = HttpClientBuilder.create().build();

    ObjectMapper mapper = new ObjectMapper();

    try {
        long startTime = System.currentTimeMillis();
        for(int i=0;i<1000;i++) {
            MessageRequest value = new MessageRequest(new CharacterMessage(UUID.randomUUID(), new Vector(10, 20, 30)));

            HttpPost request = new HttpPost("http://[google compute engine ip]:8080/character");
            StringEntity params = new StringEntity(mapper.writeValueAsString(value));
            request.addHeader("content-type", "application/json");
            request.addHeader("Accept", "application/json");

            request.setEntity(params);
            HttpResponse response = httpClient.execute(request);
            BufferedReader br = new BufferedReader(new InputStreamReader((response.getEntity().getContent())));

            String output;
            StringBuilder sb = new StringBuilder();
            while ((output = br.readLine()) != null) {
                sb.append(output);
            }
            MessageResponse mr = mapper.readValue(sb.toString(), MessageResponse.class);
            System.out.println("Output from Server .... \n");
            System.out.println(mr.getCharacterMessage().getPossition());
        }

        System.out.println("all done!!:"+(System.currentTimeMillis()-startTime));
    }catch (Exception ex) {
        ex.printStackTrace();
    }

  track 2: proto buffer over http2 using grpc server running on google compute engine

public static void main(String[] args){
    ManagedChannel channel = ManagedChannelBuilder
            .forAddress("[googgle compute engine ip]", 8080)
            .usePlaintext(true)
            .build();

    CharacterServiceGrpc.CharacterServiceBlockingStub stub = CharacterServiceGrpc.newBlockingStub(channel);
    long startTime = System.currentTimeMillis();

    for(int i=0;i<1000;i++) {
        MessageResponse response = stub.updateCharacter(
                MessageRequest.newBuilder()
                        .setCharacterMessage(
                                CharacterMessage.newBuilder()
                                        .setId(ByteString.copyFrom(asBytes(UUID.randomUUID())))
                                        .setPosstion(Vector.newBuilder()
                                                .setX(10).setY(20).setZ(30).build())
                                        .build())
                        .build());

        System.out.println("response:" + response.getCharacterMessage().getPosstion());
    }

    System.out.println("finished message transfer:"+(System.currentTimeMillis()-startTime));
}

public static byte[] asBytes(UUID uuid) {
    ByteBuffer bb = ByteBuffer.wrap(new byte[16]);
    bb.putLong(uuid.getMostSignificantBits());
    bb.putLong(uuid.getLeastSignificantBits());
    return bb.array();
}

So i ran both cases (each does a 1000) request from my place in New Zealand to us-west. I was obviously expecting the gRPC to do better.. But to my surprise (and disappointment) it didn't.. The JSON/HTTP1 completed in 349270 milliseconds but the gRPC scenario took 364541 milliseconds to finish..

Can anyone explain what might be going on here?

I would have thought because http being binary more easy to parse and being more compact that it would do better than REST.. Maybe spring-boot is doing something clever? I do know that on a case like this you would use a stream and that finishes in 1974 milliseconds.. But i would have through that just call to call gRPC would do faster as well.

[Carl Mastrangelo]

Hey, 

It looks like this needs to be rewritten.  There are a lot of factors that the code doesn't account for (like memory allocation).  You should consider rewriting your benchmark with a benchmarking harness like JMH or Caliper.   

[Dharshana Ratnayake]

hmm sure memory allocation could be different.. but i suspect it could be something todo with

PoolingHttpClientConnectionManager

that im using in the REST scenario.. 

[Ray Tsang]

I suspect it's the gRPC startup time. Would you be able to test in the following way, such as making the warm up requests first, and then benchmark the subsequent requests post-warmup.

There is also a benchmark that you can run / replicate here: https://github.com/david-cao/gRPCBenchmarks

Cheers,

[Dharshana Ratnayake]

Thanks Ray..

That was it.. Added the warm up and now gRPC is consistently faster...

Thanks for your help!! 

[Makarand Dharmapurikar]

Hi Darshana - Would you be open to publish your results and methodology, perhaps as a simple README, so others can benefit from your investigation ? 

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/48dfb1d2-4845-4d74-afd2-83a30d9e233e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Dharshana Ratnayake]

sure.. no problem at all

the code is

https://github.com/darthShana/grpc-demo

ive added a README

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.