Intent to Prototype: Multi-Screen Window Placement

[Mike Wasserman]

Contact emails

m...@chromium.org

Explainer (up-to-date)

https://github.com/webscreens/window-placement

Specification (out-of-date)

https://webscreens.github.io/window-placement/

Design docs

https://web.dev/multi-screen-window-placement/

Summary

Adds new screen information APIs and makes incremental improvements to existing window placement APIs, allowing web applications to offer compelling multi-screen experiences. The existing singular window.screen offers a limited view of available screen space, and window placement functions generally clamp bounds to the current screen. This feature unlocks modern multi-screen workspaces for web applications. Note: We are addressing feedback from our Chrome M86-M88 Origin Trial

Blink component

UI>Browser>WebAppInstalls

Motivation

As multi-screen devices and applications become a more common and critical part of user experiences, it becomes more important to give developers information and tools to leverage that expanded visual environment. This feature extends the web platform's single-screen paradigm to support multi-screen devices. Providing information about the set of screens connected to a device, and extending support for multi-screen window placement coordinates, will enable many powerful multi-screen windowing behaviors. Use cases enabled by these APIs include: - Slideshow app presents on a projector, shows speaker notes on a laptop screen - Financial app opens a dashboard of windows across multiple monitors - Medical app opens images (e.g. x-rays) on a high-resolution grayscale display - Creativity app shows secondary windows (e.g. pallete) on a separate screen - Conference room app shows controls on a touch screen device and video on a TV - Multi-screen layouts in gaming, signage, artistic, and other types of apps - Site optimizes content and layout when a window spans multiple screens

Initial public proposal

https://discourse.wicg.io/t/proposal-supporting-window-placement-on-multi-screen-devices/3948

Search tags

window placement, screen enumeration, window, open, moveTo, moveBy, requestFullscreen, screen, display, monitor, multi-screen, multi-display, multi-monitor, cross-screen, cross-display, cross-monitor

TAG review

https://github.com/w3ctag/design-reviews/issues/413,

https://github.com/w3ctag/design-reviews/issues/522,

https://github.com/w3ctag/design-reviews/issues/602

TAG review status

Pending

Risks

Interoperability and Compatibility

Addition of a RuntimeEnabled 'change' EventHandler requires the existing Screen IDL interface to inherit EventTarget. Inheritance cannot currently be gated as a RuntimeEnabled feature; so we expect that inheritance change to have a minor effect on the stable JS API exposed to the web. Feature detection of new screen information APIs and Permission API integration allows sites to handle different levels of feature support. Existing window placement APIs generally use compatible multi-screen coordinates, but implementations often restrict bounds to the current screen. We expect low levels of risk in supporting permitted cross-screen placement requests, and falling back on legacy same-screen behavior otherwise. This work is included in the W3C Second Screen CG charter to seek consensus and support for broad interoperability: https://webscreens.github.io/cg-charter/

Gecko: No signal We'll request a position shortly, to give time for feedback before I2S. Firefox supports cross-screen window.open/move*() requests. This work partly pursues compatibility with that behavior.

Edge: No signal We'll request a position shortly, to give time for feedback before I2S. Edge has a related proposal for foldable devices: https://github.com/webscreens/window-segments

WebKit: No signal We'll request a position shortly, to give time for feedback before I2S.

Web developers: Positive (https://github.com/webscreens/window-placement/issues) This work is of interest to GSuite / Google Slides.

Ergonomics

The minor improvements to window.open|move*() API behaviors have no effect on their poor ergonomics (synchronous, features string shape, etc.). This API does not preclude future work from improving the ergonomics of those existing APIs. Extending the requestFullscreen dictionary with an optional screen should pose no ergonomic risks. New dual-screen and foldable screen devices are entering the consumer market, increasing the complexity of exposing reliable and actionable screen information for all device form factors. Additionally, operating systems and their window managers support different levels of screen placement APIs, and new paradigms may emerge that complicate the availability of window placement features. See the Second Screen's "Form Factors Explained" report for more explorations on these topics: https://webscreens.github.io/form-factors/

Activation

This feature incrementally extends existing screen information and window placement interfaces with basic multi-screen support. This immediately enables new compelling experiences through progressive enhancement of existing applications.

Security

Security and Privacy risks are explored in repo documents: <https://github.com/webscreens/window-placement>. That analysis and review uncovered minimal risks. Privacy concerns center around fingerprinting screen information, while security risks center around deceptive, surreptitious, or annoying window placements. Gating new functionality with a permission helps mitigate these concerns, and may aid or inspire similar protections for legacy API behavior with similar abuse vectors. The overall added risks are low and further mitigated by limiting to secure contexts, providing origin-scoped generated screen ids that reset when cookies are cleared, being selective about display information to expose, and continuing to prevent off-screen window placement.

Debuggability

New and existing screen and window APIs are readily debuggable in existing developer tools.

Is this feature fully tested by web-platform-tests?

No

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=897300

Launch bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1169426

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5252960583942144

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/LDymWZeV7jo, https://groups.google.com/a/chromium.org/g/blink-dev/c/SFyBx9MrGRo
Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/C6xw8i1ZIdE

This intent message was generated by Chrome Platform Status.

[Chris Harrelson]

On Tue, Jan 26, 2021 at 3:34 PM 'Mike Wasserman' via blink-dev <blin...@chromium.org> wrote:

Addition of a RuntimeEnabled 'change' EventHandler requires the existing Screen IDL interface to inherit EventTarget. Inheritance cannot currently be gated as a RuntimeEnabled feature; so we expect that inheritance change to have a minor effect on the stable JS API exposed to the web.

Just calling attention to this paragraph (bold added by me): it seems our current IDL infrastructure can't hide this inheritance behind a runtime enabled flag. If so, the only way to move forward to the prototype would be to ship this particular change to web-exposed behavior. I think this is ok given the constraints, as long as we're pretty sure Screen will end up being an EventTarget. Anyone else with feedback about risks? +Philip Jägenstedt any hacks you can think of to avoid this?

[Jeremy Roman]

One hack would be to copy all of the members of EventTarget to the Screen IDL interface, make those runtime-enabled, and then not inherit until ready to ship. The prototype chain will be wrong, but you can inherit from EventTarget in the C++ implementation and everything should just work except for code that writes "screen instanceof EventTarget" in JavaScript.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw8cO9A05tmT1%3DKhzQ%3DC_ws_g_kikWAooAyJY%3D%2BK1k4Tow%40mail.gmail.com.

[Thomas Steiner]

Note: We are addressing feedback from our Chrome M86-M88 Origin Trial

The changes compared to the previous origin trial mentioned above are documented in CHANGES.md. 

 

[Mike Wasserman]

Thanks for the idea, but would you recommend actually using such a hack? Is there a strong incentive to avoid an inheritance change?

See our active discussion at https://crrev.com/c/2628705; I would definitely appreciate guidance to unblock that work.

FWIW, I found one instance where we made a similar inheritance change for experimentation at https://crrev.com/c/2242846.

[Mike Wasserman]

Thanks again, https://crrev.com/c/2628705 now follows your suggested pattern; we can avoid changing stable for now!