gRPC Java: How are the QPS benchmarks calculated?

[Anjaneya Srujan Narkedamalli]

Hi,

I was trying to understand how are QPS stats calculated displayed on https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5636470266134528 .

In the benchmarks module in gprc-java, is it the LoadDriver (LoadClient/LoadServer)  with Closed Loop used to calculate the QPS numbers provided above?

Also is below indicated scenario used for unary QPS? And is the QPS number of RPCs per client or both the clients combined?

 > QPS - the messages/second rate when there are 2 clients and a total of 64 channels, each of which has 100 outstanding message

Thanks

[Carl Mastrangelo]

It's been a while since I looked at this code, but I'm probably the most familiar with it.  The close loop benchmark you are looking at records each latency in a histogram called `recorder`.  At the end of the benchmark, the latencies are added up and the 50th / 99th percentile is reported back.   Also, the total number of RPCs made during the interval are sent back too.  (based on the the number of items recorded to the histogram).

I believe this code is used for all the dashboard results.   The client and server are driven by their input configuration, which is actually triggered by benchmarking code in https://github.com/grpc/grpc/blob/836f4c64d006581b219b69de4b46bb212d5ddda0/tools/run_tests/performance/run_worker_java.sh

The setup for this is more complicated than we would ideally like it to be.   The code you are looking was intended for you to see how we setup our code under test; the rest of the scaffolding is not as easy to follow sadly.