Intent to Implement and Ship: Media Remoting
Yuri Wiitala
Contact emails
m...@chromium.org, sko...@chromium.org
Spec
N/A, since this is a browser feature (with possible web-exposure). Design doc here.
Note: This is similar in functionality to the Remote Playback API, but as an optimization within an existing UA, user-controlled feature; rather than site-controlled. Future Remote Playback API intents will describe any new points of integration (if any) with this feature.
Summary
For Chrome Tab Mirroring, one of the most common use cases is to stream video content to a remote device, such as a Chromecast or Android TV. However, by design, mirroring introduces some loss of quality and also consumes a rather large portion of the sending system's CPU, GPU, memory, and networking resources. Video playback occurs locally in the HTMLMediaElement on the sender device (download → decryption → decoding → rendering → compositing), and then the result is re-captured → scaled → re-encoded → transmitted to the receiver device.
Media Remoting is an optimization that activates during Tab Mirroring. It redirects the original content bitstream of a video directly to the receiver device. In doing this, all of the steps listed above are eliminated. This both saves huge amounts of system resources while also eliminating the main causes of quality degradation.
Intervention/Web-Exposure concern: While a video is being remoted, the HTMLMediaElement will not be rendering the video locally. Instead, a still-frame interstitial, based on the poster image and browser-specific graphics, will be shown to indicate the video content is being remoted. There are concerns that this could break the user experience for certain web sites and, as such, that there needs to be a way for users and/or authors to workaround problems. We propose resolving issues with a combination of manual user intervention and/or the Remote Playback API's disableRemotePlayback attribute (on HTMLMediaElement).
Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Desktop platforms only, at this point.
Demo link
Implementation in-progress. For now, we have UX mocks.
Interoperability and Compatibility Risk
Low. Such issues should be rare; and resolved by a combination of manual user intervention or a site author's use of the disableRemotePlayback attribute.
This feature intends to provide the most compelling representation of a page's content for a user that has turned on a second screen for viewing its content (i.e., by mirroring a Chrome browser tab). It's also explicitly intended for use only with pages that do not use other "remoting" web platform APIs, such as the Remote Playback API. As such, when a video occupies the vast majority of the viewport, local rendering of the video content is automatically shut-off and diverted to a receiver device (the second screen; e.g., a TV).
One can easily come up with examples of sites where this behavior breaks the user experience. For example, a page author might choose to layer video and other DOM elements together in complex ways to provide a composed, interactive video game. If the video content were to be remoted, this might result in a very broken display on both the local and remote screens. Nevertheless, we believe such breakage will be rare; especially when compared to the majority of use cases (e.g., streaming video sites) where Media Remoting will vastly improve the user experience.
This feature can only activate after the user takes the manual step of starting Chrome Tab Mirroring. Therefore, in terms of interoperability/compatibility issues, we believe it's reasonable to have the user resolve these by either turning off tab mirroring or using a per-domain "remoting disabled" override setting. The latter means that only Tab Mirroring, which is transparent to web content by design, would be used. Where possible (e.g., an external failure), we will provide UI to aid the user in restoring normal operation.
Also, site authors that have identified problems can use the disableRemotePlayback attribute to preempt the need for manual user intervention. Note that using this attribute is consistent with how other browsers disable their video remoting interventions.
Launch tracking bug
Entry on the feature dashboard
Chris Harrelson
Contact emails
m...@chromium.org, sko...@chromium.org
Spec
N/A, since this is a browser feature (with possible web-exposure). Design doc here.
Note: This is similar in functionality to the Remote Playback API, but as an optimization within an existing UA, user-controlled feature; rather than site-controlled. Future Remote Playback API intents will describe any new points of integration (if any) with this feature.
Summary
For Chrome Tab Mirroring, one of the most common use cases is to stream video content to a remote device, such as a Chromecast or Android TV. However, by design, mirroring introduces some loss of quality and also consumes a rather large portion of the sending system's CPU, GPU, memory, and networking resources. Video playback occurs locally in the HTMLMediaElement on the sender device (download → decryption → decoding → rendering → compositing), and then the result is re-captured → scaled → re-encoded → transmitted to the receiver device.
Media Remoting is an optimization that activates during Tab Mirroring. It redirects the original content bitstream of a video directly to the receiver device. In doing this, all of the steps listed above are eliminated. This both saves huge amounts of system resources while also eliminating the main causes of quality degradation.Intervention/Web-Exposure concern: While a video is being remoted, the HTMLMediaElement will not be rendering the video locally. Instead, a still-frame interstitial, based on the poster image and browser-specific graphics, will be shown to indicate the video content is being remoted. There are concerns that this could break the user experience for certain web sites and, as such, that there needs to be a way for users and/or authors to workaround problems. We propose resolving issues with a combination of manual user intervention and/or the Remote Playback API's disableRemotePlayback attribute (on HTMLMediaElement).
Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Desktop platforms only, at this point.
Demo link
Implementation in-progress. For now, we have UX mocks.
Interoperability and Compatibility Risk
Low. Such issues should be rare; and resolved by a combination of manual user intervention or a site author's use of the disableRemotePlayback attribute.
This feature intends to provide the most compelling representation of a page's content for a user that has turned on a second screen for viewing its content (i.e., by mirroring a Chrome browser tab). It's also explicitly intended for use only with pages that do not use other "remoting" web platform APIs, such as the Remote Playback API. As such, when a video occupies the vast majority of the viewport, local rendering of the video content is automatically shut-off and diverted to a receiver device (the second screen; e.g., a TV).
One can easily come up with examples of sites where this behavior breaks the user experience. For example, a page author might choose to layer video and other DOM elements together in complex ways to provide a composed, interactive video game. If the video content were to be remoted, this might result in a very broken display on both the local and remote screens. Nevertheless, we believe such breakage will be rare; especially when compared to the majority of use cases (e.g., streaming video sites) where Media Remoting will vastly improve the user experience.
This feature can only activate after the user takes the manual step of starting Chrome Tab Mirroring. Therefore, in terms of interoperability/compatibility issues, we believe it's reasonable to have the user resolve these by either turning off tab mirroring or using a per-domain "remoting disabled" override setting. The latter means that only Tab Mirroring, which is transparent to web content by design, would be used. Where possible (e.g., an external failure), we will provide UI to aid the user in restoring normal operation.
Also, site authors that have identified problems can use the disableRemotePlayback attribute to preempt the need for manual user intervention. Note that using this attribute is consistent with how other browsers disable their video remoting interventions.
Launch tracking bug
Entry on the feature dashboard
None, since this is a browser feature.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
Anton Vayvod
Sounds like a really nice feature. A question below.On Fri, Nov 4, 2016 at 5:49 PM, Yuri Wiitala <m...@chromium.org> wrote:Contact emails
m...@chromium.org, sko...@chromium.org
Spec
N/A, since this is a browser feature (with possible web-exposure). Design doc here.
Note: This is similar in functionality to the Remote Playback API, but as an optimization within an existing UA, user-controlled feature; rather than site-controlled. Future Remote Playback API intents will describe any new points of integration (if any) with this feature.
Summary
For Chrome Tab Mirroring, one of the most common use cases is to stream video content to a remote device, such as a Chromecast or Android TV. However, by design, mirroring introduces some loss of quality and also consumes a rather large portion of the sending system's CPU, GPU, memory, and networking resources. Video playback occurs locally in the HTMLMediaElement on the sender device (download → decryption → decoding → rendering → compositing), and then the result is re-captured → scaled → re-encoded → transmitted to the receiver device.
Media Remoting is an optimization that activates during Tab Mirroring. It redirects the original content bitstream of a video directly to the receiver device. In doing this, all of the steps listed above are eliminated. This both saves huge amounts of system resources while also eliminating the main causes of quality degradation.Intervention/Web-Exposure concern: While a video is being remoted, the HTMLMediaElement will not be rendering the video locally. Instead, a still-frame interstitial, based on the poster image and browser-specific graphics, will be shown to indicate the video content is being remoted. There are concerns that this could break the user experience for certain web sites and, as such, that there needs to be a way for users and/or authors to workaround problems. We propose resolving issues with a combination of manual user intervention and/or the Remote Playback API's disableRemotePlayback attribute (on HTMLMediaElement).
Will the site author be able to observe that mirroring is happening?
Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Desktop platforms only, at this point.
Demo link
Implementation in-progress. For now, we have UX mocks.
Interoperability and Compatibility Risk
Low. Such issues should be rare; and resolved by a combination of manual user intervention or a site author's use of the disableRemotePlayback attribute.
This feature intends to provide the most compelling representation of a page's content for a user that has turned on a second screen for viewing its content (i.e., by mirroring a Chrome browser tab). It's also explicitly intended for use only with pages that do not use other "remoting" web platform APIs, such as the Remote Playback API. As such, when a video occupies the vast majority of the viewport, local rendering of the video content is automatically shut-off and diverted to a receiver device (the second screen; e.g., a TV).
One can easily come up with examples of sites where this behavior breaks the user experience. For example, a page author might choose to layer video and other DOM elements together in complex ways to provide a composed, interactive video game. If the video content were to be remoted, this might result in a very broken display on both the local and remote screens. Nevertheless, we believe such breakage will be rare; especially when compared to the majority of use cases (e.g., streaming video sites) where Media Remoting will vastly improve the user experience.
This feature can only activate after the user takes the manual step of starting Chrome Tab Mirroring. Therefore, in terms of interoperability/compatibility issues, we believe it's reasonable to have the user resolve these by either turning off tab mirroring or using a per-domain "remoting disabled" override setting. The latter means that only Tab Mirroring, which is transparent to web content by design, would be used. Where possible (e.g., an external failure), we will provide UI to aid the user in restoring normal operation.
Also, site authors that have identified problems can use the disableRemotePlayback attribute to preempt the need for manual user intervention. Note that using this attribute is consistent with how other browsers disable their video remoting interventions.
Launch tracking bug
Entry on the feature dashboard
None, since this is a browser feature.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
Chris Harrelson
On Fri, Nov 4, 2016 at 9:22 PM Chris Harrelson <chri...@chromium.org> wrote:Sounds like a really nice feature. A question below.On Fri, Nov 4, 2016 at 5:49 PM, Yuri Wiitala <m...@chromium.org> wrote:Contact emails
m...@chromium.org, sko...@chromium.org
Spec
N/A, since this is a browser feature (with possible web-exposure). Design doc here.
Note: This is similar in functionality to the Remote Playback API, but as an optimization within an existing UA, user-controlled feature; rather than site-controlled. Future Remote Playback API intents will describe any new points of integration (if any) with this feature.
Summary
For Chrome Tab Mirroring, one of the most common use cases is to stream video content to a remote device, such as a Chromecast or Android TV. However, by design, mirroring introduces some loss of quality and also consumes a rather large portion of the sending system's CPU, GPU, memory, and networking resources. Video playback occurs locally in the HTMLMediaElement on the sender device (download → decryption → decoding → rendering → compositing), and then the result is re-captured → scaled → re-encoded → transmitted to the receiver device.
Media Remoting is an optimization that activates during Tab Mirroring. It redirects the original content bitstream of a video directly to the receiver device. In doing this, all of the steps listed above are eliminated. This both saves huge amounts of system resources while also eliminating the main causes of quality degradation.Intervention/Web-Exposure concern: While a video is being remoted, the HTMLMediaElement will not be rendering the video locally. Instead, a still-frame interstitial, based on the poster image and browser-specific graphics, will be shown to indicate the video content is being remoted. There are concerns that this could break the user experience for certain web sites and, as such, that there needs to be a way for users and/or authors to workaround problems. We propose resolving issues with a combination of manual user intervention and/or the Remote Playback API's disableRemotePlayback attribute (on HTMLMediaElement).
Will the site author be able to observe that mirroring is happening?The rest of the Remote Playback API has not shipped yet (stay tuned!) but we plan to expose media remoting (not tab mirroring) to the web pages via the RemotePlayback.state and the corresponding state change event handlers when it does.So the short answer is yes.
Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Desktop platforms only, at this point.
Demo link
Implementation in-progress. For now, we have UX mocks.
Interoperability and Compatibility Risk
Low. Such issues should be rare; and resolved by a combination of manual user intervention or a site author's use of the disableRemotePlayback attribute.
This feature intends to provide the most compelling representation of a page's content for a user that has turned on a second screen for viewing its content (i.e., by mirroring a Chrome browser tab). It's also explicitly intended for use only with pages that do not use other "remoting" web platform APIs, such as the Remote Playback API. As such, when a video occupies the vast majority of the viewport, local rendering of the video content is automatically shut-off and diverted to a receiver device (the second screen; e.g., a TV).
One can easily come up with examples of sites where this behavior breaks the user experience. For example, a page author might choose to layer video and other DOM elements together in complex ways to provide a composed, interactive video game. If the video content were to be remoted, this might result in a very broken display on both the local and remote screens. Nevertheless, we believe such breakage will be rare; especially when compared to the majority of use cases (e.g., streaming video sites) where Media Remoting will vastly improve the user experience.
This feature can only activate after the user takes the manual step of starting Chrome Tab Mirroring. Therefore, in terms of interoperability/compatibility issues, we believe it's reasonable to have the user resolve these by either turning off tab mirroring or using a per-domain "remoting disabled" override setting. The latter means that only Tab Mirroring, which is transparent to web content by design, would be used. Where possible (e.g., an external failure), we will provide UI to aid the user in restoring normal operation.
Also, site authors that have identified problems can use the disableRemotePlayback attribute to preempt the need for manual user intervention. Note that using this attribute is consistent with how other browsers disable their video remoting interventions.
Launch tracking bug
Entry on the feature dashboard
None, since this is a browser feature.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
Mounir Lamouri
> On Fri, Nov 4, 2016 at 6:44 PM, Anton Vayvod <ava...@chromium.org>
> wrote:
>
> > On Fri, Nov 4, 2016 at 9:22 PM Chris Harrelson <chri...@chromium.org>
> > wrote:
> >
> > Sounds like a really nice feature. A question below.
> >
> > On Fri, Nov 4, 2016 at 5:49 PM, Yuri Wiitala <m...@chromium.org> wrote:
> >
> > Contact emails
> >
> > m...@chromium.org, sko...@chromium.org
> >
> > Spec
> >
> > N/A, since this is a browser feature (with possible web-exposure). Design
> > doc here
> > .
> > Note: This is similar in functionality to the Remote Playback API
> > Future Remote Playback API intents will describe any new points of
> > integration (if any) with this feature.
> >
> > Summary
> >
> > For Chrome Tab Mirroring
> > as a Chromecast or Android TV. However, by design, mirroring introduces
> > some loss of quality and also consumes a rather large portion of the
> > sending system's CPU, GPU, memory, and networking resources. Video playback
> > occurs locally in the HTMLMediaElement on the sender device (download →
> > decryption → decoding → rendering → compositing), and then the result is
> > re-captured → scaled → re-encoded → transmitted to the receiver device.
> >
> >
> > Media Remoting is an optimization that activates during Tab Mirroring. It
> > redirects the original content bitstream of a video directly to the
> > receiver device. In doing this, all of the steps listed above are
> > eliminated. This both saves huge amounts of system resources while also
> > eliminating the main causes of quality degradation.
> >
> > Intervention/Web-Exposure concern: While a video is being remoted, the
> > HTMLMediaElement will not be rendering the video locally. Instead, a
> > still-frame interstitial, based on the poster image and browser-specific
> > graphics, will be shown to indicate the video content is being remoted.
> > There are concerns that this could break the user experience for certain
> > web sites and, as such, that there needs to be a way for users and/or
> > authors to workaround problems. We propose resolving issues with a
> > combination of manual user intervention and/or the Remote Playback API's
> > disableRemotePlayback
> >
> >
> > Will the site author be able to observe that mirroring is happening?
> >
> >
> > The rest of the Remote Playback API has not shipped yet (stay tuned!) but
> > we plan to expose media remoting (not tab mirroring) to the web pages via
> > the RemotePlayback.state
> > So the short answer is yes.
> >
>
> Yes, but only after that API ships, right? When do you expect that would
> be? Is it feasible to ship it or some of it together with this feature?
most likely it will ship before the Media Remoting project.
-- Mounir
Chris Harrelson
Philip Jägenstedt
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
Rick Byers
LGTM2
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.