Intent to Implement and Ship: MediaStreamTrack resizeMode constrainable property
Guido Urdaneta
Contact emails
Explainer
https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia
https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints#Properties_of_video_tracks
https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackSettings#Properties_of_video_tracks
Design doc/Spec
Spec: This is part of Media Capture and Streams
Tag review: https://github.com/w3ctag/design-reviews/issues/57
Summary
Add support for the resizeMode constrainable property to getUserMedia() and related MediaStreamTrack methods: getSettings(), getConstraints(), getCapabilities() and applyConstraints().
Motivation
The Media Capture and Streams spec allows users to specify constraints to control the resolution of a video track. The user agent is allowed to apply video-processing techniques to satisfy such constraints. For example, if a user specifies a maximum height of 1000, the user agent may downscale a native resolution of 1080p in order to satisfy the constraint.
Chrome has always used cropping and downscaling to satisfy resolution constraints, while other browsers use native resolutions only (Firefox is adding support for downscaling as well, see https://bugzilla.mozilla.org/show_bug.cgi?id=1286945). While both approaches are spec compliant, the difference in behavior is an issue in some cases (see https://crbug.com/863722).
The resizeMode constrainable property helps address this issue by allowing the application to constrain the method the browser uses to derive the resolution for a track. Two values will be supported:
"none": Only the native resolution produced by the camera will be used. If constraints cannot be satisfied with native resolutions, the request will fail. This is the only mode required by the spec and is the mode supported by all major browsers other than Chrome.
"crop-and-scale": The browser may use cropping or downscaling to adjust the resolution produced by the camera so that constraints can be satisfied. This mode is part of the spec but it is not mandatory. This is the only mode currently supported by Chrome.
Risks
Interoperability and Compatibility
This property is not yet implemented by other browsers. However, implementing it in Chrome will improve interoperability with them. The main reason is that this property allows applications to tell Chrome to disable rescaling, so that it can derive track resolutions using the same method other browsers use. For example, if an application uses the "none" value for resizeMode (the only one mandated by the spec), Chrome will work like other browsers without rescaling, while other browsers will ignore the constraint and will behave just like Chrome will. In addition to this, this property is part of a spec that all browsers are committed to support and at least Firefox has plans to support it once they add support for rescaling.
Existing applications will be unaffected by this change since they do not currently use the resizeMode constrainable property. We do not plan to change Chrome's default behavior at this time, although we are open to that possibility in the future. The spec does not mandate any specific default behavior.
Edge: No signals
Firefox: Public support
Safari: No signals
Web developers: Anecdotally, some Web developers have expressed interest in this feature. See, for example, https://crbug.com/863722.
Ergonomics
Are there any other platform APIs this feature will frequently be used in tandem with?
This is part of getUserMedia() and MediaStreamTrack, so it is used frequently together with RTCPeerConnections and media elements.
Could the default usage of this API make it hard for Chrome to maintain good performance (i.e. synchronous return, must run on a certain thread, guaranteed return timing)?
No.
Activation
Will it be challenging for developers to take advantage of this feature immediately, as-is?
No.
Would this feature benefit from having polyfills, significant documentation and outreach, and/or libraries built on top of it to make it easier to use?
No polyfills are required. Extra documentation in places such as MDN would be beneficial, although not strictly required, since there is already some documentation available.
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Yes.
Is this feature fully tested by web-platform-tests?
Web Platform Tests will be written together with the feature.
There are currently no tests for this property. The WPT suite for the Media Capture and Streams spec is available at https://wpt.fyi/results/mediacapture-streams.
Link to entry on the feature dashboard
https://www.chromestatus.com/feature/6206218032906240
Requesting approval to ship?
Yes
Daniel Bratell
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f1bbd638-d163-470f-a1f6-8b6c84d6b220%40chromium.org.
Guido Urdaneta
LGTM1, but...1. The tag review is for the initial version of the spec. This part was added 2-3 years later so I wouldn't think it's applicable. On the other hand, this is a small addition so I don't see a tag review as a requirement.
2. The part about public support in Firefox seems to not be directly supported by the link. Now that bug was written by a mozilla developer that also wrote this part of the spec so I believe that they can be seen as supporting this feature but it's not obvious.
Chris Harrelson
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BBuZxZgcAemeQmyEx3076PFoUpZTU%3DtYTbCH1yvrscyx0uXCw%40mail.gmail.com.
Yoav Weiss
Contact emails
Explainer
https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia
https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints#Properties_of_video_tracks
https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackSettings#Properties_of_video_tracks
Design doc/Spec
Spec: This is part of Media Capture and Streams
Tag review: https://github.com/w3ctag/design-reviews/issues/57
Summary
Add support for the resizeMode constrainable property to getUserMedia() and related MediaStreamTrack methods: getSettings(), getConstraints(), getCapabilities() and applyConstraints().
Motivation
The Media Capture and Streams spec allows users to specify constraints to control the resolution of a video track. The user agent is allowed to apply video-processing techniques to satisfy such constraints. For example, if a user specifies a maximum height of 1000, the user agent may downscale a native resolution of 1080p in order to satisfy the constraint.
Chrome has always used cropping and downscaling to satisfy resolution constraints, while other browsers use native resolutions only (Firefox is adding support for downscaling as well, see https://bugzilla.mozilla.org/show_bug.cgi?id=1286945). While both approaches are spec compliant, the difference in behavior is an issue in some cases (see https://crbug.com/863722).
The resizeMode constrainable property helps address this issue by allowing the application to constrain the method the browser uses to derive the resolution for a track. Two values will be supported:
"none": Only the native resolution produced by the camera will be used. If constraints cannot be satisfied with native resolutions, the request will fail. This is the only mode required by the spec and is the mode supported by all major browsers other than Chrome.
"crop-and-scale": The browser may use cropping or downscaling to adjust the resolution produced by the camera so that constraints can be satisfied. This mode is part of the spec but it is not mandatory. This is the only mode currently supported by Chrome.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f1bbd638-d163-470f-a1f6-8b6c84d6b220%40chromium.org.
Harald Alvestrand
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEgJ85AdCGbkM--MD5mSNkK6r_VaKtUoV7cfzZS8UA9PnQ%40mail.gmail.com.