Intent to Ship: Document picture-in-picture

已查看 1,329 次
跳至第一个未读帖子

Tommy Steimel

未读,
2023年6月12日 12:51:152023/6/12
收件人 blink-dev

Contact emails

ste...@chromium.orglibe...@chromium.orgy...@chromium.org

Explainer

https://github.com/WICG/document-picture-in-picture/blob/main/README.md

Specification

https://wicg.github.io/document-picture-in-picture

Summary

Document PiP adds a new API to open an always-on-top window that can be populated with arbitrary HTMLElements. This is an expansion upon the existing HTMLVideoElement API that only allows for an HTMLVideoElement to be put into a PiP window. This allows web developers to provide a better PiP experience to users.



Blink component

Blink>Media>PictureInPicture

TAG review

https://github.com/w3ctag/design-reviews/issues/798

TAG review status

Issues open

Risks



Interoperability and Compatibility



Gecko: No signal (https://github.com/mozilla/standards-positions/issues/670)

WebKit: No signal (https://github.com/WebKit/standards-positions/issues/41) no official position, but have some concerns

Web developers: Positive (https://discourse.wicg.io/t/proposal-document-picture-in-picture/5736/2?u=steimel) In addition to the linked comment, we have some signals from partners that this would be valuable, and are checking to see if they are OK making that public

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

N/A since we are not enabling this API on Android



Debuggability

The new methods and properties proposed in this spec will show up in autocomplete functionality (e.g. window.documentPictureInPicture). The enter event will support event listener breakpoints in the "Picture-in-Picture" category.



Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

At least initially, we're not supporting Android since the Android APIs don't easily support always-on-top windows with arbitrary interaction



Is this feature fully tested by web-platform-tests?

Yes

Flag name

blink::features::kDocumentPictureInPictureAPI

Requires code in //chrome?

True

Tracking bug

https://crbug.com/1315351

Launch bug

https://crbug.com/1269059

Measurement

We have UseCounters for the documentPictureInPicture APIs

Availability expectation

Feature is available only in Chromium browsers for the foreseeable future

Adoption expectation

Feature is used by specific partner(s) to provide functionality within 12 months of launch in Chrome

Adoption plan

Feature is already being used in the OT by partners who plan to continue using the feature in stable

Non-OSS dependencies

Does the feature depend on any code or APIs outside the Chromium open source repository and its open-source dependencies to function?

N/A

Estimated milestones

Shipping on desktop116
OriginTrial desktop last115
OriginTrial desktop first111


Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

No future backwards-incompatible spec changes anticipated

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5755179560337408

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGyVZ8L-SmBFbBMvvbm0x3TwZ66JQ1Fm7_zb3nSiBvYhXavAuA%40mail.gmail.com Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/Tz1gUh92dXs


This intent message was generated by Chrome Platform Status.

Domenic Denicola

未读,
2023年6月12日 22:16:182023/6/12
收件人 Tommy Steimel、blink-dev
I was the spec mentor for Tommy's work on this feature, and am chiming in to provide the requested spec maturity summary.

I'm very happy with the spec work for this feature. Tommy worked hard to incorporate all my feedback about how to ensure everything was rigorously defined, and integrated well with existing concepts like opening new windows (navigables). The monkey patches are small, focused, and easy to understand.

There is some TAG feedback that seems to wish this was part of window.open() or some other more-general API, but I think that advice is not correct, and the current API design is good, due to the singleton-per-top-level-traversable nature of a document PiP window. In my opinion this makes the window.documentPictureInPicture entrypoint, with its requestWindow(), window, and onenter properties, a good API for the use case.


--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAE-AwAr2EHEEbqD0X-7bt_4NGAxFhyRz_0wv2c6Bvi65iYw30Q%40mail.gmail.com.

Philip Jägenstedt

未读,
2023年6月13日 10:06:402023/6/13
收件人 Domenic Denicola、Tommy Steimel、blink-dev
Thanks Domenic for mentoring and vouching :) I took a very cursory look at the API shape and sent a small fix, thanks for reviewing that.

The concerns on https://github.com/WebKit/standards-positions/issues/41 are device independence and portability. For the Android support, can you say anything about future expectations? Is this feature simply not important on mobile and it might be a desktop-only API for the foreseeable future, or would Android support make sense for some use cases? If so, does it seem tractable given the requisite changes to Android APIs?

Tommy Steimel

未读,
2023年6月13日 13:25:112023/6/13
收件人 Philip Jägenstedt、Domenic Denicola、blink-dev
In the future, we may be able to do something on Android that doesn't support input without any change to Android APIs. If we wanted input in the PiP window, it would require changes to Android itself. That said, we haven't seen any interest from web developers for an inputless document PiP on Android, and there isn't really anything you can do with an inputless document PiP that you can't do with a canvas-back video PiP, so we haven't pursued anything on Android.

Yoav Weiss

未读,
2023年6月14日 02:17:262023/6/14
收件人 Tommy Steimel、Philip Jägenstedt、Domenic Denicola、blink-dev
On Tue, Jun 13, 2023 at 7:25 PM 'Tommy Steimel' via blink-dev <blin...@chromium.org> wrote:
In the future, we may be able to do something on Android that doesn't support input without any change to Android APIs. If we wanted input in the PiP window, it would require changes to Android itself. That said, we haven't seen any interest from web developers for an inputless document PiP on Android, and there isn't really anything you can do with an inputless document PiP that you can't do with a canvas-back video PiP, so we haven't pursued anything on Android.

On Tue, Jun 13, 2023 at 7:06 AM Philip Jägenstedt <foo...@chromium.org> wrote:
Thanks Domenic for mentoring and vouching :) I took a very cursory look at the API shape and sent a small fix, thanks for reviewing that.

The concerns on https://github.com/WebKit/standards-positions/issues/41 are device independence and portability. For the Android support, can you say anything about future expectations? Is this feature simply not important on mobile and it might be a desktop-only API for the foreseeable future, or would Android support make sense for some use cases? If so, does it seem tractable given the requisite changes to Android APIs?

On Tue, Jun 13, 2023 at 4:16 AM Domenic Denicola <dom...@chromium.org> wrote:
I was the spec mentor for Tommy's work on this feature, and am chiming in to provide the requested spec maturity summary.

I'm very happy with the spec work for this feature. Tommy worked hard to incorporate all my feedback about how to ensure everything was rigorously defined, and integrated well with existing concepts like opening new windows (navigables). The monkey patches are small, focused, and easy to understand.

There is some TAG feedback that seems to wish this was part of window.open() or some other more-general API, but I think that advice is not correct, and the current API design is good, due to the singleton-per-top-level-traversable nature of a document PiP window. In my opinion this makes the window.documentPictureInPicture entrypoint, with its requestWindow(), window, and onenter properties, a good API for the use case.


On Tue, Jun 13, 2023 at 1:51 AM 'Tommy Steimel' via blink-dev <blin...@chromium.org> wrote:

The examples use the unload event for the PiP document window where, where elsewhere we're working on getting rid of it..
Is there an alternative API we can recommend instead?
 


Specification

https://wicg.github.io/document-picture-in-picture

Summary

Document PiP adds a new API to open an always-on-top window that can be populated with arbitrary HTMLElements. This is an expansion upon the existing HTMLVideoElement API that only allows for an HTMLVideoElement to be put into a PiP window. This allows web developers to provide a better PiP experience to users.



Blink component

Blink>Media>PictureInPicture

TAG review

https://github.com/w3ctag/design-reviews/issues/798

TAG review status

Issues open

Risks



Interoperability and Compatibility



Gecko: No signal (https://github.com/mozilla/standards-positions/issues/670)

WebKit: No signal (https://github.com/WebKit/standards-positions/issues/41) no official position, but have some concerns

Web developers: Positive (https://discourse.wicg.io/t/proposal-document-picture-in-picture/5736/2?u=steimel) In addition to the linked comment, we have some signals from partners that this would be valuable, and are checking to see if they are OK making that public

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

N/A since we are not enabling this API on Android



Debuggability

The new methods and properties proposed in this spec will show up in autocomplete functionality (e.g. window.documentPictureInPicture). The enter event will support event listener breakpoints in the "Picture-in-Picture" category.



Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

At least initially, we're not supporting Android since the Android APIs don't easily support always-on-top windows with arbitrary interaction


Do I understand correctly that the Android apps will not have a PiP experience advantage over the web, since the other PiP APIs enable similar (input-less) experiences on the mobile web compared to mobile apps?
  

François Beaufort

未读,
2023年6月14日 02:56:152023/6/14
收件人 Yoav Weiss、Tommy Steimel、Philip Jägenstedt、Domenic Denicola、blink-dev
On Wed, Jun 14, 2023 at 8:17 AM Yoav Weiss <yoav...@chromium.org> wrote:


On Tue, Jun 13, 2023 at 7:25 PM 'Tommy Steimel' via blink-dev <blin...@chromium.org> wrote:
In the future, we may be able to do something on Android that doesn't support input without any change to Android APIs. If we wanted input in the PiP window, it would require changes to Android itself. That said, we haven't seen any interest from web developers for an inputless document PiP on Android, and there isn't really anything you can do with an inputless document PiP that you can't do with a canvas-back video PiP, so we haven't pursued anything on Android.

On Tue, Jun 13, 2023 at 7:06 AM Philip Jägenstedt <foo...@chromium.org> wrote:
Thanks Domenic for mentoring and vouching :) I took a very cursory look at the API shape and sent a small fix, thanks for reviewing that.

The concerns on https://github.com/WebKit/standards-positions/issues/41 are device independence and portability. For the Android support, can you say anything about future expectations? Is this feature simply not important on mobile and it might be a desktop-only API for the foreseeable future, or would Android support make sense for some use cases? If so, does it seem tractable given the requisite changes to Android APIs?

On Tue, Jun 13, 2023 at 4:16 AM Domenic Denicola <dom...@chromium.org> wrote:
I was the spec mentor for Tommy's work on this feature, and am chiming in to provide the requested spec maturity summary.

I'm very happy with the spec work for this feature. Tommy worked hard to incorporate all my feedback about how to ensure everything was rigorously defined, and integrated well with existing concepts like opening new windows (navigables). The monkey patches are small, focused, and easy to understand.

There is some TAG feedback that seems to wish this was part of window.open() or some other more-general API, but I think that advice is not correct, and the current API design is good, due to the singleton-per-top-level-traversable nature of a document PiP window. In my opinion this makes the window.documentPictureInPicture entrypoint, with its requestWindow(), window, and onenter properties, a good API for the use case.


On Tue, Jun 13, 2023 at 1:51 AM 'Tommy Steimel' via blink-dev <blin...@chromium.org> wrote:

The examples use the unload event for the PiP document window where, where elsewhere we're working on getting rid of it..
Is there an alternative API we can recommend instead?

Good catch! We should indeed recommend using the pagehide event instead.
I'll be updating spec/explainer and documentation.
 

François Beaufort

未读,
2023年6月14日 05:47:302023/6/14
收件人 Yoav Weiss、Tommy Steimel、Philip Jägenstedt、Domenic Denicola、blink-dev
Android apps have the ability today to show different icons in the PiP window than what is currently in https://w3c.github.io/mediasession/#enumdef-mediasessionaction
However, regarding user input, Android apps and Web apps behave the same. Users can only interact with the PiP window icons, provided by the app.


Yoav Weiss

未读,
2023年6月14日 09:33:442023/6/14
收件人 François Beaufort、Tommy Steimel、Philip Jägenstedt、Domenic Denicola、blink-dev
LGTM1

Thanks for addressing the `unload` documentation and for your answers! :)

slightlyoff via Chromestatus

未读,
2023年6月14日 11:47:242023/6/14
收件人 blin...@chromium.org
LGTM2

Mike Taylor

未读,
2023年6月14日 11:47:582023/6/14
收件人 slightlyoff via Chromestatus、blin...@chromium.org

LGTM3

On 6/14/23 11:47 AM, slightlyoff via Chromestatus wrote:
LGTM2
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Mariano M

未读,
2023年10月4日 19:26:232023/10/4
收件人 blink-dev、Mike Taylor、slightlyoff via Chromestatus
Hey, is there any official release date for the new PiP window without having to have the chrome flag enabled? Thanks!

François Beaufort

未读,
2023年10月5日 03:18:092023/10/5
收件人 Mariano M、blink-dev、Mike Taylor、slightlyoff via Chromestatus

Mariano M

未读,
2023年10月5日 11:50:352023/10/5
收件人 blink-dev、François Beaufort

Oh!! You're right, i no longer need the Chrome flag. So awesome! Good job on this feature, love it!

Kevin Groat

未读,
2024年4月26日 13:45:494月26日
收件人 blink-dev
Love this feature, but it appears to only be available on desktop.  Is there any word on when it might be made available for Android?

ari picker

未读,
2024年4月29日 08:43:264月29日
收件人 blink-dev、Tommy Steimel、blink-dev
Thank you for all your work on PiP, it's awesome!
I am very interested in PiP on Android. Is any progress being made, and is there an ETA for PiP landing on Android?

Sahil Talwar

未读,
2024年5月1日 15:59:375月1日
收件人 blink-dev、ari picker、Tommy Steimel、blink-dev
+1 - would love PiP on Android as well.

Tommy Steimel

未读,
2024年5月2日 14:06:115月2日
收件人 Maratona app、blink-dev、Sahil Talwar、ari picker
Thanks for the interest in Document PiP on Android!

Currently, we're not sure that it's feasible to make the pip window interactive on Android (e.g. allowing HTML buttons to be clicked). Would an API that only allows non-interactive pip (so you'd create an HTML document that we'd show but would not receive user inputs) be useful in your opinion? Or would it only be a useful API if the user could actually interact with it?

Thanks,
Tommy

On Wed, May 1, 2024 at 6:11 PM Maratona app <man...@maratona.app> wrote:
+1 for PiP on Android, it works really well for features like Pomodoro Timer and To-do tasks and since PWA is growing... it seems a great feature to support. (Currently using PWA -> TWA)

I know there is an workaround that is to build the feature using a <canvas>, capture stream and pass to video for original PiP support.... but it makes really hard to map all interactions.

Anyway, awesome work !

Maratona app

未读,
2024年5月2日 14:13:255月2日
收件人 blink-dev、Sahil Talwar、ari picker、Tommy Steimel、blink-dev
+1 for PiP on Android, it works really well for features like Pomodoro Timer and To-do tasks and since PWA is growing... it seems a great feature to support. (Currently using PWA -> TWA)

I know there is an workaround that is to build the feature using a <canvas>, capture stream and pass to video for original PiP support.... but it makes really hard to map all interactions.

Anyway, awesome work !

On Wednesday, May 1, 2024 at 4:59:37 PM UTC-3 Sahil Talwar wrote:

Maratona app

未读,
2024年5月2日 14:32:165月2日
收件人 Tommy Steimel、blink-dev、Sahil Talwar、ari picker
Hi Tommy, thank you for replying.

My first instinct is not really about what to do on Android, but feature parity with Desktop, for example: https://dlitsman.github.io/document-pip/ since it's already possible to make interactive PiP, it seems odd to not have the same experience for mobile, I know the implementations for Android may be a restriction for this, so only sharing my thoughts as an user. Other APIs like Push Notifcation, Web Share and Badging all seem to have feature parity, even if the UI is different for each platform, they are supported somehow. 

It feels unmotivated to adopt a new API that will only work for a specific platform, if there is a security reason behind this, it would be nice to prompt the user to allow the interaction or not, similar to allowing/denying Push Notifcations. At least I really enjoy this new API as a temporary widget for mobile devices, since it's not possible to build native widgets through PWA. I don't wanna be mean, only sharing initial thoughts and open to understand the real blockers.


Thomas Steiner

未读,
2024年5月3日 04:37:155月3日
收件人 Maratona app、Tommy Steimel、blink-dev、Sahil Talwar、ari picker
FWIW, the video PiP already allows for interactive controls (see the attached screenshot from a German news web app https://tagesschau.de). It's a small surface, but it seems fine to leave it to app authors to make sure the controls for their app work.

Screenshot_20240503-103230.png

ari picker

未读,
2024年5月6日 17:05:285月6日
收件人 blink-dev、Maratona app、blink-dev、Sahil Talwar、ari picker、Tommy Steimel
+1 for feature parity, but honestly, what I need right now is Android no-video PiP content.

Sahil Talwar

未读,
2024年5月6日 18:29:415月6日
收件人 blink-dev、ari picker、Maratona app、blink-dev、Sahil Talwar、Tommy Steimel
My 2 cents - rendering a document PiP would be great, but it wouldn't help our use case unless we also have interactivity - the reason we want a document is precisely because we want the user to be able to interact with the PiP. Rendering a document while keeping the video controls would be an acceptable fallback, but wouldn't really add too much value for our use case.
回复全部
回复作者
转发
0 个新帖子