Intent to Ship: User Activation API

[Dave Tapuska]

Explainer

https://github.com/dtapuska/useractivation

Spec

https://github.com/whatwg/html/pull/4009

https://whatpr.org/html/4009/index.html

Tag review has been open for a few weeks with little activity.

Summary

Expose the ability to query the user activation state via a javascript API. This is useful for pages that embed iframes to vet postMessages coming from them to determine if they occurred within the context of a user activiation.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/d/msg/blink-dev/peRlfsYaDIw/bw2CJHrvAgAJ

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Risks

Interoperability and Compatibility

Edge: No signals

Firefox: No signals

Safari: No signals

Web developers: No signals

Ergonomics

There have been work arounds as explained in the explainer how to get this information about the browser through performing actions and checking whether they were sucessful or not. So ergonomics is improved.

Activation

Users can check for the existence of userActivation on the navigator object.

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Yes, check out:

https://github.com/web-platform-tests/wpt/tree/master/html/user-activation

Entry on the feature dashboard

https://www.chromestatus.com/feature/5131233151418368

[Philip Jägenstedt]

Hi Dave,

Have you had any conversations with other vendors about this API? The WHATWG policy on spec changes is to have interest from two implementers, which would be blocking getting https://github.com/whatwg/html/pull/4009 merged. Hopefully that's just a matter of getting more eyeballs on it.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHgVhZXi0DkZa_ZZiWsFEeoXB7tCU4rN20JufeAwu2jp_6WQog%40mail.gmail.com.

[Daniel Bratell]

Looks like no progress on the whatwg side here. Another question, will this API survive the changes discussed in "User Activation v2"? Specifically bzbarsky mentioned in that thread that just two kinds of activations might not be enough.

/Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYdRxS3g0P76oEXNkWy6Vvp0zu_URwAsP%3Dm_JCmuiK734A%40mail.gmail.com.

--

/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

[Dave Tapuska]

+Boris in cc..

Ya there has been little response from other vendors. It should survive any changes coming out of User Activation V2. The main debate in that discussion is around the timeout and how some APIs may require a shorter timeout than others. If that did occur we could certainly mirror another attribute in the UserActivation object. Even if User Activation V2 had no timeout this API would still survive.

As indicated in the explainer I've provided an example how this information can be retrieved today but is descructive to the user context.

dave.

[Yoav Weiss]

+Mounir Lamouri 

Can you expand on the use-cases that this API will tackle?

The explainer mostly discusses autoplay, so there seems to be some overlap with https://github.com/whatwg/html/issues/3617

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHgVhZUpEZAeysY%3Dz3QRCRDSbuKx3YePArNUmYZ0ndxHPLVt9Q%40mail.gmail.com.

[Mounir Lamouri]

Hi,

The use case is a bit different from the issue you pointed to. The intent here is to be aware that at a given time T, the code is running in a context where it is "user activated".

However, the autoplay use case is to know if autoplay is allowed before attempting it, which is only slightly related to this. One could use this API to find out that playback will work but autoplay could be allowed even if this API returns that the context is not currently user activated. There are also more details around when autoplay is allowed or not.

-- Mounir

[Philip Jägenstedt]

"Will video.play() work?" (#3617) is indeed a pretty involved question. But it's my understanding from the explainer that asking "will this work" isn't the main purpose of introducing this API. Rather, the example given is an iframe asking its parent frame to resize it, and the parent making sure that the user is interacting with the iframe. (The parent can resize the iframe without a user gesture too.)

Additionally, I presume the API also allows a parent to delegate a user gesture to a child iframe, for example to start playing a video in an iframe based on a click outside the iframe? Dave, if that's a use case enabled by this, can you add an example to the explainer?

Note that there's been some reaction on https://github.com/w3ctag/design-reviews/issues/300#issuecomment-423694149 now, with questions.

[Dave Tapuska]

No this API doesn't allow a parent to delegate a user gesture to a child frame. Currently in Chrome this does happen with same origin iframes but not with cross origin iframes.  But Mustaq is fixing all that in User Activation V2.

That is an interesting use case and I presume something like that could be fixed with another PostMessageOption attribute in the dictionary. But I haven't seen an ask for it thus far.

dave.

[Mustaq Ahmed]

With User Activation v2, we had an early discussion to possibly allow delegating activation to subframes through a feature-policy (UAv2 by default makes a user gesture available to ancestor frames only).  We are not aware of any specific use case as Dave mentioned.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHgVhZWtaZKxUZ9-625na-%2B0q%3DYPZQuhfy8ytJY-WRoDHLqEBQ%40mail.gmail.com.

[Philip Jägenstedt]

Just looking at the shape of the API, I assumed it would just work? If we wanted to make it work, wouldn't this API be a good way of doing it?

[Dave Tapuska]

If we wanted to have an API that passed the user gesture from one frame to another we could do it via a postMessage but probably with another "propagateUserActivation" dictionary member. 

The includeUserActivation captures the state of the frame sending the message, be it a parent or child frame. If you are sending it from a parent to a child there are security implications of this because you are revealing to the iframe that the user is interacting with the parent document (which cannot be figured out about today). But passing off or propagating the gesture is new functionality that isn't really related to this API.

dave.

[Philip Jägenstedt]

Oh, then this API is narrower in scope than I thought. If I now
understand correctly, whether an action succeeds never changes because
of using this API, it's merely reflecting read-only state that was
previously hidden?

[Dave Tapuska]

Yes that is all that it does it captures the state of the world at the time of the postMessage (snapshot). And gives you access to read your state off the navigator object (live object). Please check out my example demonstrating the use case.

dave.

[Yoav Weiss]

OK, so I'll try to sum up my understanding of the use-cases. Please let me know if I'm in the general ballpark.

IIUC, this API addresses use-cases where the page or its parent need to perform different actions based on the page's activation state.

Examples:

* Resize iframe (by its parent) based on the iframe's activation state, e.g. because it's about to play a video

    - Are there other uses for that?

    - This use case seems like it can be addressed by a "will video play?" mechanism + piping that info through the message. Having that info on any message would indeed be more convenient, but may add some overhead to unrelated messages, right?

* Present different UI to users to enable clipboard access (e.g. "click to copy something to clipboard" messages only when actually needed)

* Other?

Otherwise, Have you tried to gather signals from other vendors? Are there open issues on their bug trackers?

[Dave Tapuska]

Yes the main example is resizing the iframe but it isn't tied to whether video will play at all. Video can be prevented in an iframe for a number of reasons, enterprise policy, feature policy, user settings. 

Another example is to think of a series of thumbnails like a picture editor. If you touch the thumbnail then it requests a larger iframe size so you can then edit the picture.

Other vendors have been silent on my requests to provide feedback on this API solicited here.

dave.

[Yoav Weiss]

Given the discussion on the other thread, and the concrete use cases presented there, I now have a better understanding of the use cases this API tackles.

It's a shame that other vendors haven't chimed in, but at the same time, this seems like a small API change, and if this will turn up to be a mistake, we can probably revert course (as developers will need to feature detect for this feature in any case).

LGTM1

[Rick Byers]

Thanks Yoav, I agree!

LGTM2

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEgN3SPxW3Wkz2MfioWRwBBCRFDa6vW_%2BHGemMGJDP7KJw%40mail.gmail.com.

[Chris Harrelson]

Sorry to be a contrary voice, but I'm concerned about shipping the API without any feedback on the API shape or someone trying out the API to see if it actually solves the use cases in practice. I would like to see a real customer try this API before shipping. It doesn't necessarily have to be an origin trial, just testing a real application with the feature turned on may be enough.

Alternatively, it might also suffice to have some more feedback on the API from developers with real-world implementation experience.

Chris

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY9pcRJ4rrMN-%2BWOVjMDjPU3T1V-kYTEsdWt%3DDo%3DAmvpug%40mail.gmail.com.

[Mounir Lamouri]

FWIW, I was at FOMS yesterday and JWPlayer mentioned that they call `load()` on their media during a user gesture in order to "bless" them for the purpose of autoplay (because some broswers/platforms still require user gesture on the element itself). They expressed concerns that when their player is embedded, they could not actually know for sure that the blessing would work as they didn't know if the user gesture token was still valid. I believe that this API would resolve their concern. This said, in practice, it would mostly help them if Safari iOS were to implement it but I think it speaks to the general use case of an embeddee or embedder trying to do an action based on the other party's signalling.

-- Mounir

[dvoy...@google.com]

Hi All! I'm chiming in from AMP side, but both use cases very relevant to Web at large.

Two use cases I wanted to bring up: "iframe resize" and "open a popup". Both are in the context of 3p iframes.

1. Iframe resize and lightbox.

The iframe wants to be bigger. A special case - the iframe wants to take over the whole viewport (position:fixed, but not using full screen API). We call this lightbox mode. In many contexts, this is a major UX operation and we'd like to confirm if this is what user actually wants to do, i.e. if user has actually activated the iframe that asks for the change. A good model to think: interstitial ads. We don't allow them by default, unless user actually clicks on it.

In this case, operation certainly *can* succeed, the question is whether it *should*.

2. Open a popup.

A classical case - social sign-in. For instance, Google Sign-in button can be rendered in the iframe. Clicking on it should open a Google Sign-in popup. Right now it works like this: button click -> parent.postMessage("please open popup") -> window.open("popup"). The iframe wants the parent page to open the popup b/c this provides the right referrer and messaging context.

But how to do this with scale? What if we do not entirely trust 3p context? It'd be again great to know where the user has actually activated iframe before we allow opening a popup. If there was no user gesture, the popup will be blocked, however there are few problems with relying on the popup blocker:

a. It's still a bad UX.

b. Many libraries actually do redirect when popup blocker blocks the popup to workaround overly aggressive popup blockers and to support single-window browsers (think iGSA).

-d

[Chris Harrelson]

LGTM3

After digging deeper into the research leading to this API and hearing positive feedback from a user who has tried it (AMP), I'm satisfied that this is ok to ship.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/ebfa17ab-f437-45fd-87d6-94f5327e42be%40chromium.org.