Intent to Ship: Remote Playback API

[Anton Vayvod]

Contact emails

ava...@chromium.org, mlam...@chromium.org

Spec

https://w3c.github.io/remote-playback/.

Tag review: https://github.com/w3ctag/spec-reviews/issues/145

Summary

The goal of the Remote Playback API is to let websites use HTMLMediaElement to load, play and control media on a remote playback device (e.g. an HDMI stick, smart TV, wireless speaker).

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/d/topic/blink-dev/Sf1C6blXmUo/discussion

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

All except for WebView. WebView is not feasible for the same reasons as for the Presentation API. See this bug for details.

Note that while the API will be present on desktop, the backend implementation will be shipped later. The API will behave as if there were no available remote playback devices. Part of the API will be implemented by Media Remoting (see the discussion here) and the rest will be added using the Chromium’s MediaRouter framework when the desktop team gets to it.

Demo link

https://avayvod.github.io/remote-playback/test.html

Interoperability and Compatibility Risk

* Backward compatibility risk doesn’t exist since this is a new API.

* There is a webkit proprietary API on Safari iOS similar to this API to control Airplay devices.

* Mozilla expressed support to solve this use case but it is a low priority for them.

* Apple expressed interest and is implementing the API (https://bugs.webkit.org/show_bug.cgi?id=162971).

* No feedback from Microsoft although they have a similar feature in Edge to play videos remotely.

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=578833

[Philip Jägenstedt]

There are some open issues, most filed by Chromium developers. Can you go through and resolve as many as possible? Issues which can't possibly change observable behavior are fine to leave, of course.

Browsing the IDL, I see that the implementation doesn't have the "connecting" RemotePlaybackState enum value, is that the tip of an iceberg or would it be trivial to implement?

Is there a shared test suite for this, so that other implementers can have some confidence that they're passing the same tests as our implementation? I realize that automated tests for the actual behavior would require something that doesn't exist yet, but the surface-level APIs should be possible to test.

[Anton Vayvod]

Comments inline:

On Wed, Nov 16, 2016 at 1:40 AM Philip Jägenstedt <foo...@chromium.org> wrote:

There are some open issues, most filed by Chromium developers. Can you go through and resolve as many as possible? Issues which can't possibly change observable behavior are fine to leave, of course.

Mounir and I looked at the issues today. There's one minor issue that might result in an observable behavior change depending on what we agree at: https://github.com/w3c/remote-playback/issues/63.

We're going to resolve it asap (I'll update this thread).

 

Browsing the IDL, I see that the implementation doesn't have the "connecting" RemotePlaybackState enum value, is that the tip of an iceberg or would it be trivial to implement?

This was actually just a missing enum value in the idl file, the rest of the iceberg has been implemented before the Intent to Ship :)

I've fixed the IDL, thanks for noticing (I wonder if Blink should verify enum values used?).

 

Is there a shared test suite for this, so that other implementers can have some confidence that they're passing the same tests as our implementation? I realize that automated tests for the actual behavior would require something that doesn't exist yet, but the surface-level APIs should be possible to test.

We have some test-harness based tests for the various edge cases that don't require actual remote playback devices. The Second Screen WG is working on a test suite for the Presentation API atm, I expect their setup to work well for the Remote Playback API too.

Do you have an example in mind of an additional tests we could write today though?

[Philip Jägenstedt]

On Thu, Nov 17, 2016 at 12:59 AM Anton Vayvod <ava...@chromium.org> wrote:

Comments inline:

On Wed, Nov 16, 2016 at 1:40 AM Philip Jägenstedt <foo...@chromium.org> wrote:

There are some open issues, most filed by Chromium developers. Can you go through and resolve as many as possible? Issues which can't possibly change observable behavior are fine to leave, of course.

Mounir and I looked at the issues today. There's one minor issue that might result in an observable behavior change depending on what we agree at: https://github.com/w3c/remote-playback/issues/63.

We're going to resolve it asap (I'll update this thread).

Great! As long as you're confident that it'll be resolved in a specific way, that other implementers will support it, and that you'll match that when shipping, this intent need not block, but if there's no hurry we can just wait for it to be resolved.

 

Browsing the IDL, I see that the implementation doesn't have the "connecting" RemotePlaybackState enum value, is that the tip of an iceberg or would it be trivial to implement?

This was actually just a missing enum value in the idl file, the rest of the iceberg has been implemented before the Intent to Ship :)

I've fixed the IDL, thanks for noticing (I wonder if Blink should verify enum values used?).

Yes, we should! The predictability program is working this quarter on Blink IDL spec sync, and the tooling that +Mark Dittmer is building will eventually allow us to see what differences there are for IDL files across Blink, and possible also have it as a presubmit check in the long term.

Is there a shared test suite for this, so that other implementers can have some confidence that they're passing the same tests as our implementation? I realize that automated tests for the actual behavior would require something that doesn't exist yet, but the surface-level APIs should be possible to test.

We have some test-harness based tests for the various edge cases that don't require actual remote playback devices. The Second Screen WG is working on a test suite for the Presentation API atm, I expect their setup to work well for the Remote Playback API too.

Do you have an example in mind of an additional tests we could write today though?

It looks like only prompt-twice-throws.html uses internals, so contributing the others to web-platform-tests would be great. To be clear, this is not yet a required part of any process, because exporting tests isn't super easy, yet.

In addition to those tests, idlharness.js tests might be useful just to cover the things that can be automatically tests based on the spec's IDL alone, which cuts down on some very boring tests like checking that RemotePlayback does in fact inherit directly from EventTarget and so on.

[Anton Vayvod]

On Thu, Nov 17, 2016 at 2:43 AM Philip Jägenstedt <foo...@chromium.org> wrote:

On Thu, Nov 17, 2016 at 12:59 AM Anton Vayvod <ava...@chromium.org> wrote:

Comments inline:

On Wed, Nov 16, 2016 at 1:40 AM Philip Jägenstedt <foo...@chromium.org> wrote:

There are some open issues, most filed by Chromium developers. Can you go through and resolve as many as possible? Issues which can't possibly change observable behavior are fine to leave, of course.

Mounir and I looked at the issues today. There's one minor issue that might result in an observable behavior change depending on what we agree at: https://github.com/w3c/remote-playback/issues/63.

We're going to resolve it asap (I'll update this thread).

Great! As long as you're confident that it'll be resolved in a specific way, that other implementers will support it, and that you'll match that when shipping, this intent need not block, but if there's no hurry we can just wait for it to be resolved.

We checked how Safari is implementing and resolved the issue with no change to the spec.

 

 

Browsing the IDL, I see that the implementation doesn't have the "connecting" RemotePlaybackState enum value, is that the tip of an iceberg or would it be trivial to implement?

This was actually just a missing enum value in the idl file, the rest of the iceberg has been implemented before the Intent to Ship :)

I've fixed the IDL, thanks for noticing (I wonder if Blink should verify enum values used?).

Yes, we should! The predictability program is working this quarter on Blink IDL spec sync, and the tooling that +Mark Dittmer is building will eventually allow us to see what differences there are for IDL files across Blink, and possible also have it as a presubmit check in the long term.

Great to hear!

 

Is there a shared test suite for this, so that other implementers can have some confidence that they're passing the same tests as our implementation? I realize that automated tests for the actual behavior would require something that doesn't exist yet, but the surface-level APIs should be possible to test.

We have some test-harness based tests for the various edge cases that don't require actual remote playback devices. The Second Screen WG is working on a test suite for the Presentation API atm, I expect their setup to work well for the Remote Playback API too.

Do you have an example in mind of an additional tests we could write today though?

It looks like only prompt-twice-throws.html uses internals, so contributing the others to web-platform-tests would be great. To be clear, this is not yet a required part of any process, because exporting tests isn't super easy, yet.

In addition to those tests, idlharness.js tests might be useful just to cover the things that can be automatically tests based on the spec's IDL alone, which cuts down on some very boring tests like checking that RemotePlayback does in fact inherit directly from EventTarget and so on.

Ok, we've filed a tracking issue for this work and will target M57.

 

[Chris Harrelson]

LGTM1

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Philip Jägenstedt]

LGTM2

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Anton Vayvod]

A gentle ping. We'd need one more LGTM in order to ship. Thanks!

[Anton Vayvod]

...and an after-holiday ping :)

[Jochen Eisinger]

I was wondering why there's no security & privacy consideration section (open issue https://github.com/w3c/remote-playback/issues/67)?

[Dimitri Glazkov]

LGTM3

[Anton Vayvod]

On Wed, Nov 30, 2016 at 11:33 AM Jochen Eisinger <joc...@chromium.org> wrote:

I was wondering why there's no security & privacy consideration section (open issue https://github.com/w3c/remote-playback/issues/67)?

We don't expect to have more there than the Presentation API (https://w3c.github.io/presentation-api/#security-and-privacy-considerations) since Remote Playback API doesn't have the receiver browser context or identifiers for reconnection to a device and exposes only one bit of information in a similar way about device availability.

That said, the issue definitely slipped through and I'll look into updating the issue with the self-questionnaire responses and adding a draft section to the spec now. Thanks for noticing!

[Muyao Xu]

Hi all,

The Remote Playback API is now targeting to ship to M121 on Win/Mac/Linux. There's no change to the spec or the API. The Chrome Status entry has been updated.

[Anton Bershanskyi]

Hi all,

the specification contains numerous (9) references to "user permission", yet there is no section about Permissions API and Permissions Policy integration, as well as Permissions Policy feature/declaration and default allow list. Is this intentional? Of course, this could be accidental since the API dates to circa 2016 when Permissions Policy was not a thing yet and Permissions API was very fresh.

Thanks