Chromium Blink performance with Telemetry

[Michael C]

Hi Telemetry team,

I am trying to compare the performance between two Chromium browser executables of various Blink benchmarks (blink_perf.dom, blink_perf.events, blink_perf.xml_http_request) to determine the performance impact of various changes. I am running Telemetry from within chrome/src/tools/perf/run_benchmark.

I was wondering how Chrome compares two Telemetry runs (columns). I want to determine which browser executable is statistically slower/faster or not statistically different, however, it appears the p-Values and U-statistic only say whether or not the executables (samples) are different.

I see on the Github page for telemetry, they seem to run a different version than the one in the src/perf directory that shows a green/red/gray emoji if a column is statistically greater/lower or not statistically different respectively to a reference column. How do they determine this given the values provided?

Kind regards,
Michael

[Ned]

+Ben Hayden +Ethan Kuefner 

--
You received this message because you are subscribed to the Google Groups "telemetry" group.
To unsubscribe from this group and stop receiving emails from it, send an email to telemetry+...@chromium.org.

[Ben Hayden]

Apologies for the delay!
I am working on adding improvement direction data to the legacy unit information used to convert blink_perf results to the new Histogram format used by results.html. Once it lands, that should enable new results.html files to colorize cells red or green depending on whether they improved or regressed, and display smileys or frownies if they are significantly different.
If you want to upgrade existing results.html files with that change, you can checkout catapult and run
tracing/bin/valueset2html results.html

Does that help?

[Michael C]

This is great! Thank you