Few updates to blink_perf testing with Telemetry

5 views
Skip to first unread message

Ned Nguyen

unread,
May 22, 2017, 9:46:33 AM5/22/17
to blink-dev, Chrome benchmarking, Charlie Andrews, Xianzhu Wang
Here is the list of bug fixes & feature improvements we recently made to blink_perf testing through Telemetry:
1) We recently fixed the bug with about blink_perf test crashed when "--browser" flag isn't specified. If you frequently build content browser, you no longer need to add "--browser" flag, as Telemetry will pick up the most recent local build in your Chromium tree as the default.
2) We also added the general blink_perf benchmark. You can now run any perf tests in third_party/WebKit/PerformanceTests by:

./tools/perf/run_benchmark blink_perf --test-path=<path to test(s) directory>

This means no need to add temporary benchmark code to tools/perf/blink_perf.py when you just want to run your new tests using telemetry test script.

For example:
./tools/perf/run_benchmark blink_perf --test-path=TestData runs all the tests in third_party/WebKit/PerformanceTests/TestData/ directory

./tools/perf/run_benchmark blink_perf --test-path=Layout/layers_overlap_3d.html runs only third_party/WebKit/PerformanceTests/Layout/layers_overlap_3d.html test

Of course this also works with trybot. You can kick off a trybot run of your CL on all Layout tests using:
./tools/perf/run_benchmark try android-nexus5X blink_perf --test-path=Layout

*Note if your CL include new perf tests, "Before patch" run won't include those tests.

3) We added new APIs to PerfTestRunner that allows you to collect tracing metrics when your perf tests is run through Telemetry. This is expect to help lowering test noises & and make perf regression from the lab more actionable as the trace used by test run will be uploaded to the perf dashboard.

For detailed information about how to use the new tracing metrics APIs, please refer to https://chromium.googlesource.com/chromium/src/+/master/third_party/WebKit/PerformanceTests/README.md 


Finally, feel free to send your proposal about how we can improve blink_perf tests further to benchmar...@chromium.org or filing bugs with Component="Speed>Benchmarks".


Cheers,
Ned

Reply all
Reply to author
Forward
0 new messages