Intent to Implement: Fix Chrome MSE PTS/DTS Compliance
Matthew Wolenetz
Contact email
Spec
https://www.w3.org/TR/media-source/
(No TAG review deemed necessary since this spec is already a W3C REC)
Summary
This change is intended to fix Chrome Media Source Extensions (MSE) implementation's management and reporting of buffered media timeline ranges to be done compliantly by presentation timestamp (PTS) ranges, rather than by decode timestamp (DTS) ranges. Longstanding assumptions of equivalence of PTS and DTS throughout the Chrome MSE implementation dating back to initial work based on formats like WebM, where there was no such distinction between PTS and DTS, must be fixed to achieve compliance.
Motivation
Though many MSE media streams do not differentiate DTS and PTS (WebM and MP3 in particular), ISO-BMFF (MP4) streams frequently contain out-of-order presentation versus decode time streams (eg, for codecs like h.264) causing DTS and PTS to differ. MP4 MSE streams can also contain a rudimentary ELST (edit list box) allowing further shifting of PTS from DTS.
The result is that Chrome MSE reports (usually just slightly) different buffered ranges and duration values than expected of a compliant implementation for these streams, surprising the API’s users (and failing relevant portions of the W3C media-source web-platform-tests).
Interoperability and Compatibility Risk
Firefox: Shipped (Comment on related MSE spec issue)
Edge: Public support (Multiple comments and author of related MSE spec clarification by MSFT co-editor on related issue)
Safari: Public support (Comment on related MSE spec issue)
Web developers: Positive (Shaka Player tracking of PTS/DTS issues, Multiple stars on related crbugs)
Ongoing technical constraints
None other than maintaining both old and new buffered range management algorithms while experimenting and before fully shipping.
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Yes
Philip Jägenstedt
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAADho6M8TGJB%3DoG%2BABMfdNTynUdo2-PqCJ2m8XvZrh6wwrvP3g%40mail.gmail.com.
Matthew Wolenetz
No LGTMs necessary for an Intent to Implement, but I take it you intend to just try shipping this with no flag? That makes sense, and I'd consider it a bug fix more than anything else.
Are these format-specific facets of MSE tested by web-platform-tests? Some of this is undoubtedly best tested using unit tests, but for the web-facing APIs (buffered/duration) will it be possible to write some shared tests?
Philip Jägenstedt
No LGTMs necessary for an Intent to Implement, but I take it you intend to just try shipping this with no flag? That makes sense, and I'd consider it a bug fix more than anything else.
Before fully launching, since this API compliance change can impact web app behavior (and since the implementation will require significant changes), I plan to experiment and such experiments will likely be overrideable by a flag -- probably MediaSourceExperimental. Googlers can refer to the internal launch bug: https://crbug.com/760264. If that is *not* the recommended way to proceed, please let me know ASAP. Thanks!
Are these format-specific facets of MSE tested by web-platform-tests? Some of this is undoubtedly best tested using unit tests, but for the web-facing APIs (buffered/duration) will it be possible to write some shared tests?This format-specific piece is covered by web-platform-tests, but those tests will need some additional work to run all supported media types (or at least swap MP4 with WebM in preference order of which format to test if both are supported, since PTS==DTS always in WebM). This is why Chrome passed the tests (WebM-based expectations match) but wouldn't pass MP4 versions of some of those same tests until this ships.
