Issue 456507 in chromium: Performance regress in three.js WebGL stress tests

[chro...@googlecode.com]

Status: Unconfirmed
Owner: ----
Labels: Pri-2 Via-Wizard Type-Bug-Regression OS-Windows

New issue 456507 by postfil...@gmail.com: Performance regress in three.js
WebGL stress tests
https://code.google.com/p/chromium/issues/detail?id=456507

UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/40.0.2214.94 Safari/537.36 OPR/27.0.1689.66

Steps to reproduce the problem:
1. Visit these WebGL stress tests:

http://threejs.org/examples/webgl_performance.html
http://threejs.org/examples/webgl_performance_static.html
http://alteredqualia.com/three/examples/webgl_performance_static.html
(older three.js lib version that was able to get 60 fps long ago)
http://threejs.org/examples/webgl_sandbox.html

What is the expected behavior?
These stress tests were set up to run at 60 fps (on Thinkpad W520 with
Quadro 2000M).

What went wrong?
Performance gradually declined over multiple Chromium builds.

Additionally Chrome stable and canary channels seem to have extra
performance loss in addition to regresses of Chromium continuous build.

Here is what I observed:

http://threejs.org/examples/webgl_performance.html

Canary 42.0.2298.0 (315192) - 40-43 fps
Canary 42.0.2297.0 (314983) - 40 fps
Chrome 40.0.2214.94 stable - 40 fps

Chromium 42.0.2296.0 (314621) - 50 fps
Chromium 42.0.2292.0 (313778) - 47-54 fps
Chromium 42.0.2286.0 (313010) - 45-48 fps
Chromium 42.0.2281.0 (312054) - 45 fps
Chromium 42.0.2273.0 (311008) - 48-55 fps
Chromium 41.0.2268.0 (310034) - 47-54 fps
Chromium 41.0.2245.0 (307601) - 55-57 fps
Chromium 41.0.2230.0 (305509) - 55-60 fps
Chromium 41.0.2222.0 (304405) - 57-60 fps
Chromium 41.0.2217.0 (303546) - 50-55 fps
Chromium 40.0.2214.0 (303303) - 50-58 fps
Chromium 40.0.2213.0 (303095) - 56-60 fps
Chromium 40.0.2212.0 (302911) - 58-60 fps
Chromium 40.0.2209.0 (302418) - 57-60 fps
Chromium 39.0.2131.0 (291212) - 60 fps
Chromium 38.0.2087.0 (282480) - 60 fps
Chromium 37.0.1994.0 (270654) - 60 fps
Chromium 36.0.1933.0 (262830) - 60 fps

------------------------

http://threejs.org/examples/webgl_performance_static.html

Canary 42.0.2298.0 (315192) - 36 fps
Canary 42.0.2297.0 (314983) - 36 fps
Chrome 40.0.2214.94 stable - 36 fps

Chromium 42.0.2296.0 (314621) - 40 fps
Chromium 42.0.2292.0 (313778) - 40 fps
Chromium 42.0.2286.0 (313010) - 40 fps
Chromium 42.0.2281.0 (312054) - 40 fps
Chromium 42.0.2273.0 (311008) - 40 fps
Chromium 41.0.2268.0 (310034) - 40 fps
Chromium 41.0.2245.0 (307601) - 46 fps
Chromium 41.0.2230.0 (305509) - 47 fps
Chromium 41.0.2222.0 (304405) - 48-50 fps
Chromium 41.0.2217.0 (303546) - 40 fps
Chromium 40.0.2214.0 (303303) - 40 fps
Chromium 40.0.2213.0 (303095) - 40 fps
Chromium 40.0.2212.0 (302911) - 48-50 fps
Chromium 40.0.2209.0 (302418) - 47-50 fps
Chromium 39.0.2131.0 (291212) - 50-52 fps
Chromium 38.0.2087.0 (282480) - 49-52 fps
Chromium 37.0.1994.0 (270654) - 42 fps
Chromium 36.0.1933.0 (262830) - 50 fps

------------------------

http://alteredqualia.com/three/examples/webgl_performance_static.html

Canary 42.0.2298.0 (315192) - 38-40 fps
Canary 42.0.2297.0 (314983) - 38-39 fps
Chrome 40.0.2214.94 stable - 40 fps

Chromium 42.0.2296.0 (314621) - 41-43 fps
Chromium 42.0.2292.0 (313778) - 40 fps
Chromium 42.0.2286.0 (313010) - 40 fps
Chromium 42.0.2281.0 (312054) - 40 fps
Chromium 42.0.2273.0 (311008) - 40 fps
Chromium 41.0.2268.0 (310034) - 41-43 fps
Chromium 41.0.2245.0 (307601) - 42-45 fps
Chromium 41.0.2230.0 (305509) - 46 fps
Chromium 41.0.2222.0 (304405) - 47-50 fps
Chromium 41.0.2217.0 (303546) - 45 fps
Chromium 40.0.2214.0 (303303) - 45 fps
Chromium 40.0.2213.0 (303095) - 45 fps
Chromium 40.0.2212.0 (302911) - 48-50 fps
Chromium 40.0.2209.0 (302418) - 47-48 fps
Chromium 39.0.2131.0 (291212) - 57-58 fps
Chromium 38.0.2087.0 (282480) - 56-58 fps
Chromium 37.0.1994.0 (270654) - 45-60 fps
Chromium 36.0.1933.0 (262830) - 60 fps

------------------------

http://threejs.org/examples/webgl_sandbox.html

Canary 42.0.2298.0 (315192) - 41-45 fps
Canary 42.0.2297.0 (314983) - 45 fps
Chrome 40.0.2214.94 stable - 52 fps

Chromium 42.0.2296.0 (314621) - 54-58 fps
Chromium 42.0.2292.0 (313778) - 52-54 fps
Chromium 42.0.2286.0 (313010) - 54-59 fps
Chromium 42.0.2281.0 (312054) - 50-52 fps
Chromium 42.0.2273.0 (311008) - 54-58 fps
Chromium 41.0.2268.0 (310034) - 58-59 fps
Chromium 41.0.2245.0 (307601) - 60 fps
Chromium 41.0.2230.0 (305509) - 60 fps
Chromium 41.0.2222.0 (304405) - 60 fps
Chromium 41.0.2217.0 (303546) - 51-55 fps
Chromium 40.0.2214.0 (303303) - 51-58 fps
Chromium 40.0.2213.0 (303095) - 57-60 fps
Chromium 40.0.2212.0 (302911) - 60 fps
Chromium 40.0.2209.0 (302418) - 60 fps
Chromium 39.0.2131.0 (291212) - 60 fps
Chromium 38.0.2087.0 (282480) - 60 fps
Chromium 37.0.1994.0 (270654) - 60 fps
Chromium 36.0.1933.0 (262830) - 60 fps

Did this work before? Yes

Chrome version: 40.0.2214.94 Channel: stable
OS Version: 6.1 (Windows 7, Windows Server 2008 R2)
Flash Version:

This issue is a split from issue 451670 (where multiple WebGL demos with
performance regresses were reported):

https://code.google.com/p/chromium/issues/detail?id=451670

At least one performance regress cause was already identified there (slower
trigonometric functions).

This issue collects the rest of examples mentioned there with so far
unexplained performance regress causes.

These tests are slow just with ANGLE, not with OpenGL. Numbers above are
from D3D11 ANGLE (if available as default at particular build). I did a
quick check with D3D9 ANGLE in the latest Canary and there performance is
even worse.

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[chro...@googlecode.com]

Updates:
Cc: baj...@chromium.org z...@chromium.org k...@chromium.org
shannonw...@chromium.org jmad...@chromium.org geof...@chromium.org
Labels: Cr-Blink-WebGL

Comment #1 on issue 456507 by k...@chromium.org: Performance regress in

three.js WebGL stress tests
https://code.google.com/p/chromium/issues/detail?id=456507

@postfilter: thanks for gathering all of this data.

I realized after looking at it that the continuous build archive was out of
date. The "last known good revision" wasn't updated for a long period -- 3
days, it looks like. I'm not sure why this happened and have filed Issue
456562 (sorry, internal view only) about it.

https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win/
has
more builds -- those which compiled at the particular revision, as
opposed to those which both compiled and passed tests.

Could you use that archive to confirm that Chromium r314621 (or nearby) is
~50 FPS with http://threejs.org/examples/webgl_performance.html and
Chromium r314983 (or nearby) is ~40 FPS? It isn't necessary to measure more
than just those two builds. That seems like a recent and significant
performance drop that should be investigated ASAP. Thanks.

[chro...@googlecode.com]

Comment #2 on issue 456507 by postfil...@gmail.com: Performance regress in

three.js WebGL stress tests
https://code.google.com/p/chromium/issues/detail?id=456507

@kbr I'm afraid still nothing apparent with Chromium builds:

Chromium 42.0.2299.0 (315242) - 46-52 fps
Canary 42.0.2298.0 (315192) - 40 fps
Chromium 42.0.2297.0 (314985) - 49-51 fps
Chromium 42.0.2297.0 (314981) - 47-51 fps
Chromium 42.0.2296.0 (314622) - 46-52 fps
Chromium 42.0.2296.0 (314621) - 48-52 fps

I thought: maybe I'm doing something silly with my Canary that makes it
slow vs those Chromium builds, so I reset all flags to default, disabled
all extensions, removed all command line flags. Still the same 40 fps there
vs those "nearing 50 fps from below and sometimes humps a bit over" in
Chromiums.

If it's of any help, I attached traces from the latest Canary (315192) and
the latest Chromium (315242) running
http://threejs.org/examples/webgl_performance.html (I selected "javascript
and rendering" preset).

One thing that came to my mind: both stable Chrome and Canary have much
less variance in framerates than Chromiums. It just jumps a bit at start
and then pretty much stays at 40 fps.

Framerates in Chromiums, on the other hand can be quite all over the place
(46-52 fps), both between different runs, and inside a single run. So maybe
it could have something to do with this tricky v-sync issue?

https://code.google.com/p/chromium/issues/detail?id=422000

I remember sometime in ancient WebGL history, this very stress test used to
get a big framerate slowdown in Chrome when frames happened to land on the
wrong side of v-sync. V-sync acted as amplifier of tiny performance
differences anywhere else in Chrome. Though at that time these were "round"
v-sync multiples framerates like 30 fps or 45 fps. 40 fps is unusual.



Attachments:
trace-canary-315192.zip 1.3 MB
trace-chromium-315242.zip 1.4 MB

[chro...@googlecode.com]

Comment #3 on issue 456507 by postfil...@gmail.com: Performance regress in

three.js WebGL stress tests
https://code.google.com/p/chromium/issues/detail?id=456507

For what it's worth, I tried also Brandon's special Chromium build from
that v-sync issue:

https://code.google.com/p/chromium/issues/detail?id=422000#c97

Chromium 42.0.2293.0 (314160) (no flags) - 40-43 fps
Chromium 42.0.2293.0 (314160) (--hardware-vsync-only) - 40-42 fps
Chromium 42.0.2293.0 (314160) (--software-vsync-only) - 46-50 fps

[chro...@googlecode.com]

Comment #4 on issue 456507 by postfil...@gmail.com: Performance regress in

three.js WebGL stress tests
https://code.google.com/p/chromium/issues/detail?id=456507

And I found another tricky issue that seems to be sensitive to different
flags from that Brandon's v-sync build:

https://code.google.com/p/angleproject/issues/detail?id=893

Those weird tearings show only without any flags or
with "hardware-vsync-only" flag, but there are no tearings
with "software-vsync-only" flag.

Not sure if "software-vsync-only" is just masking real causes or if it's
the real fix but right now it seems to help with 3 separate issues (at
least on my two notebooks):

- less framerate variance
- higher framerate
- eliminates diagonal tearings

[chro...@googlecode.com]

Updates:
Status: Available

Comment #5 on issue 456507 by k...@chromium.org: Performance regress in

three.js WebGL stress tests
https://code.google.com/p/chromium/issues/detail?id=456507

Marking this as Available since it seems clear there has been a performance
regression on multiple apps. The ANGLE team seems to have already
identified a couple of potentially problematic CLs.

[chro...@googlecode.com]

Comment #6 on issue 456507 by k...@chromium.org: Performance regress in

three.js WebGL stress tests
https://code.google.com/p/chromium/issues/detail?id=456507

Establishing the relationship between this and the previously filed parent
bug.

[chro...@googlecode.com]

Updates:
Blockedon: chromium:373812

Comment #7 on issue 456507 by k...@chromium.org: Performance regress in

three.js WebGL stress tests
https://code.google.com/p/chromium/issues/detail?id=456507

Making this depend on Issue 373812 so we don't forget to re-enable those
performance tests.