about the results stability of smoothness benchmark

13 views
Skip to first unread message

Yanbo Xing

unread,
Aug 13, 2015, 9:49:01 PM8/13/15
to tele...@chromium.org

Hello

 

   For the smoothness benchmark, I found some test results are very unstable(--use-live-sites), such as:

  

test indicators

comments

mean_frame_time

for the most of the sites, the error is within 20%, individual website reached 50%

avg_surface_fps

max_frame_delay

Error is greater than 50%

jank_count

mean_pixels_approximated

mean_pixels_checkerboarded

first_gesture_scroll_update_latency

 

The attachment is the detail results.

 

For such the unstable results, I’m not sure what factors caused, the network ? the case itself ?

Do you have any suggestions about how to get more stable test results ?

 

 

Thanks

Yanbo

Marvell(Shanghai) LTD

 

smothness_results.xlsx

Yanbo Xing

unread,
Aug 17, 2015, 3:17:21 AM8/17/15
to tele...@chromium.org

Looking forward to your reply!thanks!


Best Regards,
Yanbo
<smothness_results.xlsx>

Ned Nguyen

unread,
Aug 17, 2015, 9:26:41 PM8/17/15
to telemetry, xin...@marvell.com
Hi Yanbo,
We use web-page-replay to record the network requests & responses.
You can follow the guide to record the page set:

After you record the pages, just run the benchmark against without passing --use-live-sites
Reply all
Reply to author
Forward
0 new messages