When testing v43 to v58 chrome sunspider performance, found that the performance gradually back

[sinoory sin]

When using sunspider-0.9.1 to test the performance of each chrome version v8, the performance is found to gradually roll back.

Test using ubuntu 14.04 virtual machine, the test results are as follows:

Each version of the test 5 times, take the average value.

average(ms)

v58 743

v57 753

v55 1007

v53 509

v51 601

v49 533

v47 502

v45 497

v43 416

There are two questions:

1. Is there any v8 flag to improve v8 sunspider run points? Measured some v8 flag, did not work.

2. Is it a bad idea to change v8 version manually？eg，Use v8 in v43 to replace v8 in v47, so as to improve v47 run points。

[PhistucK]

It might have been an intentional speed drop in favor of speedier real world performance. See https://v8project.blogspot.co.il/2016/12/how-v8-measures-real-world-performance.html.

1. Probably not. v8-users may know more, though I would not count on a flag being available.

2. V8 versions are usually incompatible with their unintended Chrome versions. If it works, it is a coincidence and not an intended or supported configuration.

☆PhistucK

--
--
Chromium Discussion mailing list: chromium...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-discuss

---
You received this message because you are subscribed to the Google Groups "Chromium-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-discuss+unsubscribe@chromium.org.

[Chen Zhixiang]

Interesting. :-)

But i guess it's the ```default flag``` problem. and sunspider as a perf test, mainly designed for JIT performance, is not fit for '''real-world''' case.

[Primiano Tucci]

Sunspider has been retired by the browser community in 2015: https://trac.webkit.org/changeset/187526/webkit

See the recent https://v8project.blogspot.co.uk/2017/04/retiring-octane.html, which in turn describes why Sunspider was retired long before.

TL;DR those benchmarks are not representative for real-word user experience and browser vendors shifted to tracking more realistic metrics instead.

> 1. Is there any v8 flag to improve v8 sunspider run points? 

No. The all point of retiring it was to focus on changes that matter for the user at the cost of regressing a benchmark which most browser ended up micro-optimizing for.

Essentially we reached the point where a higher score in sunspider wasn't reflecting into a faster browser. 

> eg，Use v8 in v43 to replace v8 in v47, so as to improve v47 run points.

AFAIK V8 doesn't have a stable API surface. what you describe might be just technically extremely hard.

--

[sinoory sin]

在 2017年4月25日星期二 UTC+8下午8:25:07，sinoory sin写道：

Update :

Thanks for all your suggestions.

Although it's said that sunspider is not representative for real-word user experience, 

test js execution time of the real page in v43 to v47, the results are consistent with sunspider , are getting slower and slower.

Test the js execution time on http://www.bbc.co.uk/iplayer/?config=precert&lloc=playerpage ， the average time as bellow： 

v47	1.72s
v45	1.67s
v44	1.5s
v43	1.35s

I will test the performance in https://v8project.blogspot.co.il/2016/12/how-v8-measures-real-world-performance.html if time enough.