Intent to Prototype: CapturedMouseEvent

[Elad Alon]

Contact emails

elad...@chromium.org

Explainer

https://github.com/screen-share/mouse-events/blob/main/README.md

Specification

https://screen-share.github.io/mouse-events

Design document

https://docs.google.com/document/d/1FOvWG9WZlulMI9kEc5X0_sspqgdE28q3DFjeaHTmhGk/edit?usp=sharing&resourcekey=0-sQkZvVQUGk9uOkX37QDPYg

Summary

Web applications can use getDisplayMedia() to capture any display-surface - tabs, windows or screens. When they do, they can also specify the cursor constraint to control whether the cursor's pixels are captured or not. But what if the application wishes to programmatically observe the location of the cursor? That can be done by scanning each frame and employing heuristics to detect the cursor. But that's neither simple, nor efficient, nor robust. To that end, we define a mechanism for exposing mouse coordinates over a captured surface to a capturing application.

Blink component

Blink>GetDisplayMedia

Motivation

Web applications can use getDisplayMedia() to capture any display-surface - tabs, windows or screens. When they do, they can also specify the cursor constraint to control whether the cursor's pixels are captured or not. But what if the application wishes to programmatically observe the location of the cursor? That can be done by scanning each frame and employing heuristics to detect the cursor. But that's neither simple, nor efficient, nor robust. To that end, we define a mechanism for exposing mouse coordinates over a captured surface to a capturing application.

Initial public proposal

https://github.com/screen-share/meetings/blob/main/minutes/2023-02-16.md#exposing-mouse-coordinates

TAG review

Too early; not yet sent.

TAG review status

Pending

Risks

Interoperability and Compatibility

Gecko: No signal

WebKit: No signal

Web developers: Strongly positive (https://github.com/screen-share/meetings/blob/main/minutes/2023-02-16.md#exposing-mouse-coordinates)

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

Debuggability

Is this feature fully tested by web-platform-tests?

No

Flag name

CapturedMouseEvent

Requires code in //chrome?

False

Estimated milestones

No milestones specified

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5928661338816512

This intent message was generated by Chrome Platform Status.