PerfCompare Statistics Project Update
Kala Severe
Hello everyone,
Perftools has successfully completed the first part of the PerfCompare Statistics Project to significantly improve the accuracy and interpretability of performance data analysis!
You can test it now on PerfCompare by selecting Mann-Whitney-U from the test version dropdown (default is set to Student-T) in the table controls of the results page.
The new test uses Mann-Whitney U Test, Cliff's Delta, and the Common Language Effect Size (CLES) to determine if the changes you made are statistically significant. Additionally, more statistical information is provided to help with interpreting the results when the row is expanded.
The goal is to provide developers with clearer, more reliable, and easily interpretable results on performance changes.
The Mann-Whitney comparison technique is still in the experimental stage. If you encounter any issues, have suggestions for improvements, or wish to provide feedback, please report it on the Testing :: Perfcompare Bugzilla component, or reach out to us in the #perfcompare Matrix channel.
--
