How to run Python performance tests locally?

[Gustavo Covas]

Hello,
I'm interested in running the performance test suite for the Python language locally. However, when running tools/run_tests/run_performance_tests.py -l python, the script usually fail with timeouts in some scenarios. I have tried at Ubuntu and macOS.

- Does it even make sense to run these benchmarks at localhost? Given that I am mostly concerned with the QPS capacity measure, do I need to take care in simulating network latency?
- Will these benchmarks run fine at the usual development laptop (MacBook Air), or do I need more powerful hardware?
- Where could I find some description about the test scenarios provided? From where should I start to read the source in order to get a better understanding of the results that these benchmarks provide?

Thanks!

[Ken Payson]

On Tue, Jun 13, 2017 at 5:43 PM, Gustavo Covas <gtc...@gmail.com> wrote:

Hello,
I'm interested in running the performance test suite for the Python language locally. However, when running tools/run_tests/run_performance_tests.py -l python, the script usually fail with timeouts in some scenarios. I have tried at Ubuntu and macOS.

- Does it even make sense to run these benchmarks at localhost? Given that I am mostly concerned with the QPS capacity measure, do I need to take care in simulating network latency?

Running the benchmarks on localhost gives very similar results to running on two machines in a low latency network. 

- Will these benchmarks run fine at the usual development laptop (MacBook Air), or do I need more powerful hardware?

You should be able to run the tests on most hardware.  

 

- Where could I find some description about the test scenarios provided? From where should I start to read the source in order to get a better understanding of the results that these benchmarks provide?

I don't think we have much documentation on the test naming, but as an example

python_protobuf_sync_streaming_ping_pong

This tests the synchronous python client streaming API using protobuf messages.  Ping pong describes how the test operates; send a message, wait for a response, then send another message.

tools/run_tests/performance/scenario_config.py is a good place to look at some more of the parameters for the tests. 

Thanks!

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/9dcae7b0-fa6b-4ba6-9f10-e09ff39587d3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.