blink_web_tests on linux will now use threaded compositing by default

[Yotam Hacohen]

tl;dr - web_tests will switch to running with threaded compositing by default.

A long-standing deficiency in the blink web_test framework is that content_shell runs with the blink compositor in single-threaded mode, which is not a configuration that we ship. We are working to remedy this, and as a first step we turned on threaded compositing by default for non-wpt web_tests on linux. We plan to expand this to all (web|wpt)_tests and to Mac/Windows trybots soon.

This change shouldn't increase failures, flakiness, or test runtime, and should not increase the gardener's burden.

There are ~100 tests that fail deterministically when threaded compositing is enabled, and a longer list (but less than 5% of all web_tests) that become flaky (or flakier). These tests have been added to a “quarantine list”, and will continue to run using the single-threaded compositor.

In a later phase of this project, we will investigate specific failures and drive down the size of the quarantine list. By default, all new (web|wpt)_tests will run with threaded compositing. If you suspect a new test fails due to threaded compositing, please reach out to us. 

Yotam

[Morten Stenshorne]

Yotam Hacohen <yo...@chromium.org> writes:

> tl;dr - web_tests will switch to running with threaded compositing by default.

> [...]

> This change shouldn't increase failures, flakiness, or test runtime,
> and should not increase the gardener's burden.

This does affect test runtime. It seems especially bad when combining
WPT tests and non-WPT tests - about twice as slow as before.

Without crrev.com/c/4599846 it takes 33 seconds. With it, 59 seconds.

The command I used:
tools/run_web_tests.py -t Release -f external/wpt/css/css-multicol fast/multicol

--
Morten Stenshorne, Software developer,
Blink/Layout, Google, Oslo, Norway

[Stefan Zager]

On Mon, Jun 12, 2023 at 2:59 AM Morten Stenshorne <mste...@chromium.org> wrote:

Yotam Hacohen <yo...@chromium.org> writes:

> tl;dr - web_tests will switch to running with threaded compositing by default.
> [...]
> This change shouldn't increase failures, flakiness, or test runtime,
> and should not increase the gardener's burden.

This does affect test runtime. It seems especially bad when combining
WPT tests and non-WPT tests - about twice as slow as before.

Without crrev.com/c/4599846 it takes 33 seconds. With it, 59 seconds.

The command I used:
tools/run_web_tests.py -t Release -f external/wpt/css/css-multicol fast/multicol

Morten, can you file a bug about this?

We did analyze try bot runtimes before launching, and found little to no effect.

We haven't yet had a chance to analyze runtimes across the CI fleet after launching, but we intend to.

The test runner spins up a new content_shell for each unique permutation of command-line

flags it encounters. It's likely that the test runner is creating more content_shells with this feature

enabled, and I wonder if it's leading to thrashing on your system.

Thanks,

Stefan

 

--
Morten Stenshorne, Software developer,
Blink/Layout, Google, Oslo, Norway

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/ozzi4jndc86x.fsf%40aeneas.osl.corp.google.com.

[Morten Stenshorne]

Sure - bug filed: https://bugs.chromium.org/p/chromium/issues/detail?id=1454242

[Yotam Hacohen]

I created a doc with information about diagnosing the cause of flakiness as threaded compositing, and debugging tests that are failing or becoming flaky due to threaded compositing. 

This is a living document and we will add more information about debugging these kind of tests as we learn more.

I hope you find it useful.

(Link to the doc - https://docs.google.com/document/d/1IwpQ_Od3EiWmzgEXPzPNf_V9ZM9Yq-Y9sy3anfAxnMI/edit?usp=sharing&resourcekey=0-2B2NrXBGxpcRKJ_X9-kjtg)

Yotam 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.