web-platform-tests quarterly update - Q2 2020

[Stephen Mcgruer]

[This update covers both work done by the Chromium Ecosystem Infra team and by members of the wider WPT community.]

Do you use - or want to use - WPT for your tests? Check out this brief taste of what has been happening in WPT over the last 3 months!

What's New?

web-platform-tests now has a Code of Conduct! We appreciate everyone who helped contribute to its creation, and a particular thanks to the people that stepped up to join our new moderator team.

New testharness.js APIs: assert_implements/assert_implements_optional. These are useful to document intent, as well as avoid unnecessary work when a feature just isn't supported. Check out the documentation.

A second HTTPS port has been added. This enables testing features like Origin Isolation that enforce constraints on secure cross-origin requests. For an easy way to use it, just include 'common/get-host-info.sub.js' in your test and call get_host_info().

Testing print layout is now possible in upstream WPT. The new 'print-reftest' test type allows comparison of layout and rendering of a paginated document. Note that print-reftests are not yet supported in run_web_tests.py - see 'Upcoming Work' below.

(Even more) new testharness.js APIs: step_wait*. In a perfect world, asynchronous tests would always be able to wait on some event being fired from the browser. Unfortunately, we don't live in a perfect world and sometimes you have to fallback to a setTimeout loop. The new step_wait* functions provide a convenient way to wait for some condition to become true, whilst also respecting the speed of the machine they run on. Give them a try!

Tips & Tricks

An example OWNERS file with a WPT-NOTIFY flag

Did you know you can be automatically notified when newly failing WPT tests are imported? Just add an OWNERS file to the external/wpt directory you are interested in, containing the WPT-NOTIFY flag. Check out the documentation, an example OWNERs file, and an example bug.

Annoyed with unnecessary boilerplate? Two WPT features that can help are *.js tests, which generate their own HTML shell, and single page tests, which remove the need for wrapper functions when your file contains only one test.

Upcoming Work

Real wptrunner in the Chromium CI. In Q3, we are renewing our focus on using the same infrastructure in both upstream WPT and in the Chromium CI. This long-running project will reduce code duplication, eliminate the latency between features being supported in WPT and in Chromium, and will enable the use of WebDriver-based tests in Chromium's CI. Expect more information in the next quarterly update!

MojoJS support in upstream WPT. Although we continue to endorse WebDriver as the best way to remotely control the browser under test, the Ecosystem Infra team understands that it doesn't work for all features today. A popular alternative in Chromium is MojoJS, and we're happy to say that we are close to making this work in upstream WPT - which means wpt.fyi will finally start showing the correct result for test suites such as webxr!

(Potentially) print-reftest support in Chromium. As noted above, print-reftests were recently added to upstream WPT. Unfortunately supporting them in run_web_tests.py is non-trivial, so we are still evaluating the work needed here. If print-reftests sound useful for your testing needs, please get in touch and let us know so we can prioritize accordingly!

As always, we'd love to hear your testing needs and woes - you can reach us at ecosyst...@chromium.org

Thanks,

Stephen