Devtools protocol and input handling

[Alexander Semashko]

It is possible for devtools clients to simulate various input events. But there is no way to know if a particular event has been processed, other than waiting for some specific outcome of that event. This is a big inconvenience for the clients, and I'd like to discuss whether we should address it.

In detail, the problem is:

 - Devtools input commands are handled in the browser process, where they  generate usual input event IPCs; devtools response is sent immediately.

 - In the renderer input event IPCs are first examined on the compositor thread, and then posted to the main thread.

This means that the next devtools command might get to DevToolsAgent before the input event is handled on the main thread.

A lot of chromedriver end-to-end tests in chrome/test/chromedriver/test/run_py_tests.py are flaky because of this. To reproduce failures you can add a bit of delay in InputEventFilter::ForwardToHandler.

I suppose that this problem also affects many chromedriver users.

One way to fix the tests is to change common pattern "someElement.Click(); assertTrue(someCondition())" to "someElement.Click(); assertTrue(WaitForCondition(someCondition))".

The other way is to send the response only after handling the input event. This will require adding a possibility to request ACKs for all input events, and a way for devtools' InputHandler to be notified by InputRouterImpl when ACK for a specific event is received.

It seems to me that the latter is the right way to go. What do you think?

[Timothy Dresser]

+Sam Uong, +Lan Wei who have been looking at the future of ChromeDriver.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Sam Uong]

The recommended way to write WebDriver tests is using the second pattern you mention: "assertTrue(WaitForCondition(someCondtiion))". This is especially true for web apps where a click can trigger a async AJAX requests, animations, and other things which take time. WebDriver libraries have implicit and explicit wait functions to make it easier for tests to do this. However the tests in run_py_tests.py use a very basic test library that we only use for internal testing of ChromeDriver.

But still I agree this would be a good thing to have for testers, since it would make it easier to write non-flaky tests.

I should mention that we're currently in the process of modifying the ChromeDriver input commands so that they all go through the synthetic input system. So we'll stop dispatching input events using functions like InputHandler::DispatchMouseEvent() and move towards something more like InputHandler::SynthesizeTapGesture().

InputHandler::SynthesizeTapGesture() calls QueueSyntheticGesture() with a gesture to enqueue, and a callback. The SyntheticGestureController will then process the gesture and call InputHandler::SendSynthesizeTapGestureResponse() when it is finished. This sends a success response back to the client, or an error message if the gesture failed for some reason.

I don't think the SyntheticGestureController provides any strict guarantees about when the callback will be called relative to when the click event happens on the page. It would be really nice if the call to the callback could be delayed until after all the expected ACKs are received, and even better if it we could ensure that any pending navigations are started as well (e.g. if a link was clicked, ideally the callback would only get called after the first Page.frameStartedLoading event).

I don't really know the synthetic gesture code well enough to say how much work this would require, maybe Lan can say more?

Sam.

[Timothy Dresser]

We run into this exact problem in our browsertests which test input. They're notoriously flaky, because we don't have a good way of determining when input has been "fully" processed.

One of the tricky bits here is deciding what we mean by "fully" processed.

• Is it fully processed when the renderer has dispatched it's ack?
  
• Is it fully processed when the next paint has occurred after the ack was dispatched? What if there was no resulting paint?
  
• If this is a touch event, is it fully processed when the touch event has been processed, or do we need to wait for the associated gesture to finish processing?
  

Here's an example of a browsertest that ran into exactly this issue. dtapuska@ did some good work trying to deflake it, but we're still having problems.