Intent to Prototype: Screen Fold API

[Alexis Menard]

Contact emails

alexis...@intel.com, kenneth.r.c...@intel.com, baul...@samsung.com

Explainers

https://github.com/SamsungInternet/Explainers/blob/master/Foldables/FoldState.md

https://webscreens.github.io/form-factors/ (for context between various efforts around the problem space)

Specification

https://w3c.github.io/screen-fold/

Summary

New types of mobile devices (phone and laptop segments) are coming to market with the ability to fold the screen in some capacity, either by using foldable displays or multiple internal displays (MIDs). Examples include the Samsung Galaxy Fold, Motorola Razr, Lenovo ThinkPad X1 Fold. 

While the technology enabling these devices is rather new, there is a resurfacing of a trend started by ultrabooks and 2 in 1’s which have had different operating “modes” or postures depending on the hinge angle.

To note a few, “book”, “tent” and “tablet” modes were available on some devices. All these postures are triggered depending on the hinge/fold angle. The above specification would expose the fold angle as well as the postures.

From enhancing the usability of a website by avoiding the area of a fold, to enabling innovative use cases for the web, knowing the fold angle can help developers tailor their content to different devices. It can also enable to detect different postures the device might be in.

Blink component

Blink>FoldableAPIs

Motivation

Knowing the fold angle enables interesting new opportunities in the area of responsive design. With these new devices, the user can choose to consume content and browse the web even when the device is not flat, in which case the developer might want to provide a different layout for the content depending on the posture.

We propose a way to expose information about the screen fold angle to the developer. Additionally, developers would like to adopt content depending on the various modes and potentially also animate some of these transitions.

Initial public proposal

https://w3c.github.io/screen-fold/

TAG review

Requested : https://github.com/w3ctag/design-reviews/issues/575

Risks

None

Interoperability and Compatibility

No compat risk as this would be a new feature.

Gecko: No signal

WebKit: No signal

Web developers: No signal, devices are slowly getting in the market. Doing the prototyping would allow us to gather interest and feedback.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Windows and Android seem to be applicable today. If more foldable devices ship on different OSes we would update the implementation to support it.

Is this feature fully tested by web-platform-tests?

No but we intend to add tests as we implement the proposal.

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1066842

Link to entry on the Chrome Platform Status

None yet.

--

Alexis Menard
Software Engineer @ Intel

[David Benjamin]

Hi Alexis!

Some comments from security and privacy review:

First, we’re concerned about the fold.angle property, as that seems a high entropy fingerprinting surface. The specification mentions this, but leaves it to the implementer and does not touch on the level of precision and anonymization the API is trying to achieve. (The suggestions of lowering resolution and fuzzing are really two strategies for the same thing.)

Ultimately, any strategy like that will need to address this precision question. The precision of the enum, fold.posture, looks a more reasonable way to cover the core responsive design use case. (Note even this contributes fingerprinting entropy, so it should be integrated into mitigations like Privacy Budget.) We’d recommend starting with fold.posture and dropping fold.angle initially.

Do you have specific use cases that need the full angle information? The spec has an example of angle-driven animations, but that already seems incompatible with the resolution/fuzzing mitigation it suggests. It seems this could use more exploration into use cases and how to balance them with privacy requirements.

Second, screen fold changes (either via onchange or polling) are an ephemeral fingerprinting vector. Given this API is meant for responsive design, we recommend limiting it to visible browsing contexts. I see section 7.2 does constrain onchange for UX reasons. It should also be listed under “Security and Privacy considerations”. The mitigation should also be applied to other ways to query the property, such as polling. (Perhaps defer all updates to the page’s copy of the state until visible, not just the onchange event, or leave the APIs and CSS queries unavailable to hidden pages altogether.)

David

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOaK9AmoV58GAfaWyo35uiN7dvii3JhkfLJp7_yWtShUN%2BmaqA%40mail.gmail.com.

[Alexis Menard]

Hi,

Thanks for the feedback.

I opened two issues to track your concerns and comments so all the parties are involved in the discussions can see it. We can also reference these issues if we're going to update the spec.

https://github.com/w3c/screen-fold/issues/46 and https://github.com/w3c/screen-fold/issues/45

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAF8qwaDE26YhCqQy-JWJ9vez%3Dq4KnLDwb3KEdOsidHWHsJAm0w%40mail.gmail.com.

[Alexis Menard]

Also I'd like to add that there is PR I put out a few weeks ago beefing up the Security and Privacy section but it hasn't been merged yet. Feedback welcomed so I can address more of it before we merge it.

https://github.com/w3c/screen-fold/pull/42