Honoring Content-Type for <video> (or not)

[Philip Jägenstedt]

Hi blink-dev!

Hixie is now trying to figure out what to do with the Content-Type
HTTP header for <video> and wanted to know if Chromium/Blink could
change to honor Content-Type. I'm not a fan [1] and said so:

https://www.w3.org/Bugs/Public/show_bug.cgi?id=11984#c59

If anyone in the Chromium/Blink community has something to add, please
do so on that bug.

(Currently this is all controlled by Chromium code, but since this is
a Web standards issue I thought blink-dev was a more appropriate
forum.)

[1] http://www.w3.org/html/wg/wiki/ChangeProposals/NoVideoContentType#Rationale

Philip

[Adam Barth]

Do we have any use counters for how often we would fail to play a video if we started honoring the Content-Type header?  I would be surprised if we would change our behavior without data showing the compatibility impact.

Adam

[Philip Jägenstedt]

No, I can't see anything like that in UseCounter.h. Here's how I think
such a counter would have to work:

For any media resource that successfully reaches readyState
HAVE_METADATA, compare the supplied type [1] to the sniffed type.[2]
(If mimesniff is already implemented, this becomes easier.) If they
are represent different container types, assume decoding would have
failed using the supplied type and tell UseCounter about it. Under the
(unverified) assumption that the current sniffing mostly matches
mimesniff, that should tell us how much would break by changing the
behavior. (Given that the actual network bits for <video> are in
Chromium, one may have to add a callback on WebMediaPlayerClient to do
the counting in Blink.)

At this point, since both IE and Firefox do strict checking it's
rather likely to be Web compatible. However, sniffing surely must be
as well! No browser has any real incentive to change here...

Assuming that Chromium/Blink *could* adopt strict Content-Type
checking, do people think that we actually *should*? Do we want new
features on the Web platform to rely on sniffing or on Content-Type?
If there is a consensus on that question, then this old feature can
probably be made to fall in line.

[1] http://mimesniff.spec.whatwg.org/#supplied-mime-type-detection-algorithm
[2] http://mimesniff.spec.whatwg.org/#sniffing-in-an-audio-or-video-context

Philip

[Adam Barth]

On Mon, Jan 20, 2014 at 11:28 PM, Philip Jägenstedt <phi...@opera.com> wrote:

No, I can't see anything like that in UseCounter.h. Here's how I think
such a counter would have to work:

For any media resource that successfully reaches readyState
HAVE_METADATA, compare the supplied type [1] to the sniffed type.[2]
(If mimesniff is already implemented, this becomes easier.) If they
are represent different container types, assume decoding would have
failed using the supplied type and tell UseCounter about it. Under the
(unverified) assumption that the current sniffing mostly matches
mimesniff, that should tell us how much would break by changing the
behavior. (Given that the actual network bits for <video> are in
Chromium, one may have to add a callback on WebMediaPlayerClient to do
the counting in Blink.)

At this point, since both IE and Firefox do strict checking it's
rather likely to be Web compatible. However, sniffing surely must be
as well! No browser has any real incentive to change here...

<video> is more popular on mobile-optimized web sites because such web sites cannot use Flash.  IE and Firefox are less often used to render mobile-optimized content, which makes it harder to draw these sorts of conclusions in this case.

Assuming that Chromium/Blink *could* adopt strict Content-Type
checking, do people think that we actually *should*? Do we want new
features on the Web platform to rely on sniffing or on Content-Type?
If there is a consensus on that question, then this old feature can
probably be made to fall in line.

Historically, the strongest opponent of strict Content-Type checking for <video> has been Apple.  My understanding is that they have some technical limitations in this area, but I don't fully understand them.

As a general rule, I'd prefer less sniffing to more sniffing, but I doubt we're going to get all browsers to converge on not sniffing.  So, the question really boils down to whether you prefer theoretical purity (not sniffing) over eventually interoperability (sniffing).  In my book, interoperability trumps theoretical purity, but other people might have a different view on that topic or might believe that if we joined IE and Firefox we could eventually convince Safari to join the non-sniffing camp.

I suspect that the data will show that not sniffing will impact compatibility, which would make the above moot anyway.

Adam

[jlu...@chromium.org]

On Tuesday, January 21, 2014 12:14:34 PM UTC-5, Adam Barth wrote:

On Mon, Jan 20, 2014 at 11:28 PM, Philip Jägenstedt <phi...@opera.com> wrote:

No, I can't see anything like that in UseCounter.h. Here's how I think
such a counter would have to work:

For any media resource that successfully reaches readyState
HAVE_METADATA, compare the supplied type [1] to the sniffed type.[2]
(If mimesniff is already implemented, this becomes easier.) If they
are represent different container types, assume decoding would have
failed using the supplied type and tell UseCounter about it. Under the
(unverified) assumption that the current sniffing mostly matches
mimesniff, that should tell us how much would break by changing the
behavior. (Given that the actual network bits for <video> are in
Chromium, one may have to add a callback on WebMediaPlayerClient to do
the counting in Blink.)

At this point, since both IE and Firefox do strict checking it's
rather likely to be Web compatible. However, sniffing surely must be
as well! No browser has any real incentive to change here...

<video> is more popular on mobile-optimized web sites because such web sites cannot use Flash.  IE and Firefox are less often used to render mobile-optimized content, which makes it harder to draw these sorts of conclusions in this case.

This will shift in 2014, IMO. Recently Vimeo started serving <video> by default to desktop browsers that support it, also Funny or Die, Kickstarter, etc., and YouTube is moving in that direction.

 

Assuming that Chromium/Blink *could* adopt strict Content-Type
checking, do people think that we actually *should*? Do we want new
features on the Web platform to rely on sniffing or on Content-Type?
If there is a consensus on that question, then this old feature can
probably be made to fall in line.

Historically, the strongest opponent of strict Content-Type checking for <video> has been Apple.  My understanding is that they have some technical limitations in this area, but I don't fully understand them.

As a general rule, I'd prefer less sniffing to more sniffing, but I doubt we're going to get all browsers to converge on not sniffing.  So, the question really boils down to whether you prefer theoretical purity (not sniffing) over eventually interoperability (sniffing).  In my book, interoperability trumps theoretical purity, but other people might have a different view on that topic or might believe that if we joined IE and Firefox we could eventually convince Safari to join the non-sniffing camp.

+1 on interop

[Andrew Scherkus]

On Tue, Jan 21, 2014 at 9:14 AM, Adam Barth <aba...@chromium.org> wrote:

On Mon, Jan 20, 2014 at 11:28 PM, Philip Jägenstedt <phi...@opera.com> wrote:

No, I can't see anything like that in UseCounter.h. Here's how I think
such a counter would have to work:

For any media resource that successfully reaches readyState
HAVE_METADATA, compare the supplied type [1] to the sniffed type.[2]
(If mimesniff is already implemented, this becomes easier.) If they
are represent different container types, assume decoding would have
failed using the supplied type and tell UseCounter about it. Under the
(unverified) assumption that the current sniffing mostly matches
mimesniff, that should tell us how much would break by changing the
behavior. (Given that the actual network bits for <video> are in
Chromium, one may have to add a callback on WebMediaPlayerClient to do
the counting in Blink.)

At this point, since both IE and Firefox do strict checking it's
rather likely to be Web compatible. However, sniffing surely must be
as well! No browser has any real incentive to change here...

<video> is more popular on mobile-optimized web sites because such web sites cannot use Flash.  IE and Firefox are less often used to render mobile-optimized content, which makes it harder to draw these sorts of conclusions in this case.

Assuming that Chromium/Blink *could* adopt strict Content-Type
checking, do people think that we actually *should*? Do we want new
features on the Web platform to rely on sniffing or on Content-Type?
If there is a consensus on that question, then this old feature can
probably be made to fall in line.

Historically, the strongest opponent of strict Content-Type checking for <video> has been Apple.  My understanding is that they have some technical limitations in this area, but I don't fully understand them.

I believe the argument was along the lines of what Dave Singer posted on the bug (requires web server configuration, people just want to play the darn file, etc...)

https://www.w3.org/Bugs/Public/show_bug.cgi?id=11984#c10

I'm still in the let's-just-sniff camp.

As a general rule, I'd prefer less sniffing to more sniffing, but I doubt we're going to get all browsers to converge on not sniffing.  So, the question really boils down to whether you prefer theoretical purity (not sniffing) over eventually interoperability (sniffing).  In my book, interoperability trumps theoretical purity, but other people might have a different view on that topic or might believe that if we joined IE and Firefox we could eventually convince Safari to join the non-sniffing camp.

I suspect that the data will show that not sniffing will impact compatibility, which would make the above moot anyway.

It'd take some carefully crafted histogramming code to cross-reference the Content-Type header with what we found in the file, but it's doable.

Of course ... the real snag with trying to standardize sniffing is facing the reality that not all browsers have access to the network byte stream when loading media, but maybe that's an overly noble goal :\

Andrew