web-platform-tests quarterly update - Q3 2020

[Stephen Mcgruer]

[This update covers both work done by the Chromium Ecosystem Infra team and by members of the wider WPT community.]

Do you use - or want to use - WPT for your tests? Check out this brief taste of what has been happening in WPT over the last 3 months!

What's New?

WebDriver and WPT now have support for mouse-wheel inputs. Thanks to spec and implementation work by Lan Wei, WPT tests can now dispatch wheel events by importing testdriver.js and dispatching a scroll action (example).

MojoJS-based tests now work in upstream WPT. Thanks to months of hard work by Robert Ma and others, tests that rely on MojoJS to back their test-only APIs now work in upstream WPT. This means wpt.fyi now has accurate results in Chrome for webxr, webusb, and others.

(Hopefully) clearer warnings about stability and lint failures. Failing lint and stability checks for WPT pull requests now dump their errors directly into the GitHub Checks UI (example), so you don't have to dig through TaskCluster logs. Also, Blink WPT Bot will give you a proactive heads-up if your exported Chromium CL has failed one of these checks in upstream WPT, even before your CL has landed Chromium-side.

Tips & Tricks

Example log output from Chrome when running 'wpt run' with '--log-mach=-'

Want more output when testing Chrome? Thanks to recent work by Stephen McGruer, Chrome browser logs are now enabled for WPT when using 'wpt run'. This allows developers to see console logs, LOG messages, and stack traces from crashes - all useful when trying to debug a test! To use this, turn on WPT logging by passing '--log-mach=-' as a flag to your 'wpt run' command.

(Using run_web_tests.py instead? The relevant flag there is '--driver-logging'!)

Need to handle a callback-based API in a promise_test? Try this neat trick:

  promise_test(async t => {

    let result = await new Promise(resolve => {

      myCallbackBasedAPI('foo', 'bar', resolve);

    });

    assert_equals(result.foo, 'bar');
  }, 'My promise test');

Note the importance of minimizing the work inside the Promise executor function; assert calls must happen after the promise resolves (or rejects), or testharness.js will not be able to capture them. For more information see this talk (Google-internal, sorry!) or the promise_test documentation.

Upcoming Work

Testdriver support in child and cross-origin frames. A long standing problem with testdriver.js is that tests can only submit input to the main test frame - you can't click, scroll, or do any sort of automation to a child iframe or different window. Thankfully, James Graham (Mozilla) has stepped up to deal with this problem with an RFC filed to allow testdriver input for arbitrary* frames.

* Well, apart from rel=noopener frames. That case is rather tricky.

WPT @ TPAC

TPAC 2020 is coming up, and as usual WPT will be there (albeit virtually this year). We have meetings scheduled for October 19th and 21st, 14:00-17:30 UTC, and all are welcome to join. Feel free to check out our preliminary agenda and contribute suggested topics for discussion.

As always, we'd love to hear your testing needs and woes - you can reach us at ecosyst...@chromium.org

Thanks,

Stephen