Intent to Implement - TTML

[Glenn Adams]

Contact emails

gl...@chromium.org

Specifications

[1] http://www.w3.org/TR/2013/REC-ttml1-20130924/ (W3C REC - 2nd Edition)

[2] http://www.w3.org/TR/2010/REC-ttaf1-dfxp-20101118/ (W3C REC - 1st Edition)

[3] https://dvcs.w3.org/hg/ttml/raw-file/default/ttml1-api/Overview.html

[4] https://dvcs.w3.org/hg/ttml/raw-file/default/ttml2-api/Overview.html

Summary

Add support for <track> element to load and render caption/subtitle data using TTML [1][2][3][4].

Motivation

(1) In response to the "Twenty-First Century Communications and Video Accessibility Act of 2010", published as US 111th Congress House Resolution 301 [3], the FCC published [4] a Final Rule in the Federal Register, Volume 77, Number 62, Page 19508, March 30, 2012, regarding 47 CFR Parts 15 and 79 as follows:

"V. Technical Standards for IP–Delivered Video Programming 124. For the reasons set forth below, we adopt the Society of Motion Picture and Television Engineers (‘‘SMPTE’’) Timed Text format (SMPTE ST 2052–1:2010: ‘‘Timed Text Format (SMPTE–TT)’’ 2010) (‘‘SMPTE–TT’’) as a safe harbor interchange and delivery format."

[5] http://www.gpo.gov/fdsys/pkg/BILLS-111hr3101pcs/pdf/BILLS-111hr3101pcs.pdf

[6] http://www.gpo.gov/fdsys/pkg/FR-2012-03-30/pdf/2012-7247.pdf

[7] https://www.smpte.org/sites/default/files/st2052-1-2010.pdf (SMPTE ST2052-1:2010)

(2) The European Broadcasting Union (EBU) has adopted [8] TTML, in the form of EBU-TT [9], as the follow-on format to the popular STL format [10]. In turn, the HbbTV Association [11] has adopted EBU-TT in its new HbbTV 2.0 specification (under development).

[8] http://tech.ebu.ch/ebu-tt

[9] http://tech.ebu.ch/docs/tech/tech3350.pdf?vers=1.0

[10] http://tech.ebu.ch/docs/tech/tech3264.pdf

[11] http://www.hbbtv.org/

[12] http://en.wikipedia.org/wiki/Hybrid_Broadcast_Broadband_TV

(3) According to an active member in the DASH Industry Forum [13]:

"TTML is specified everywhere in DASH, has a complete encoding, delivery, and decoding (HRM) spec and conformance tests in DECE (UltraViolet), qualifies as the “safe harbor” format in FCC regs, and the storage format for DASH delivery is now specified as MPEG-4 Part 30 (ISO/IEC 14496-30) [14], and Subtitles added as a new track type in the ISO Base Media File Format (MPEG-4 Part 12) [15] to make TTML tracks an equal citizen to audio and video."

[13] http://dashif.org

[14] http://mpeg.chiariglione.org/standards/mpeg-4/timed-text-and-other-visual-overlays-iso-base-media-file-format/text-isoiec-cd

[15] http://standards.iso.org/ittf/PubliclyAvailableStandards/c061988_ISO_IEC_14496-12_2012.zip

(4) I have contacted a number of broadcasters and commercial television providers who deliver A/V content over the Web, including BBC, Comcast, Cox, Netflix, and Time Warner, and was told that TTML support is a strong requirement for browsers, in order to satisfy accessibility regulations and requirements and interoperate with their caption/subtitle content catalogs.

(5) Open Issues in Blink/Chrome include:

https://code.google.com/p/chromium/issues/detail?id=255572

https://code.google.com/p/chromium/issues/detail?id=270413

A comment on the latter [16] from Google TV folks states:

"This is also requested by one of our premium Android TV/Google TV device maker partners, could you consider raising the priority of this?"

[16] https://code.google.com/p/chromium/issues/detail?id=270413#c1

Compatibility Risk

As this is a new feature for Blink, there is no compatibility risk with existing Blink functionality. At present, one other browser, IE10-11, supports TTML [6][7]. It is anticipated that other browsers will add support for TTML as well, at least in large part based on customer and regulatory requirements.

[6] http://msdn.microsoft.com/en-us/library/hh834514(v=vs.85).aspx

[7] http://msdn.microsoft.com/en-us/library/ie/dn265042(v=vs.85).aspx

Ongoing technical constraints

None

Will this feature be supported on all five Blink platforms (Windows, Mac, Linux, Chrome OS and Android)?

Yes

OWP launch tracking bug?

To be created.

Link to entry on the feature dashboard

To be created.

Requesting approval to ship?

No. This functionality will first employ a compile time flag until adequate experience and testing permits introduction of a runtime flag. After an acceptable period of use and verification, a subsequent Intent to Ship will be proposed.

[James Robinson]

On Fri, Nov 22, 2013 at 5:55 PM, Glenn Adams <gl...@chromium.org> wrote:

Requesting approval to ship?

No. This functionality will first employ a compile time flag until adequate experience and testing permits introduction of a runtime flag. After an acceptable period of use and verification, a subsequent Intent to Ship will be proposed.

We don't use compile-time flags for any features in Blink currently.  Why would this feature require such a heavyweight mechanism?  Is it expected to be highly intrusive while in development?

- James

[Adam Barth]

This feature appears to have a dependency on XML.  Generally speaking, we're not eager to take more dependencies on XML and are looking to remove our existing XML dependencies.  Obviously, removing XML support from Blink is a long term project and one we might never finish completely, but a feature that adds more XML dependencies is going to meet with some resistance.

Adam

On Fri, Nov 22, 2013 at 5:55 PM, Glenn Adams <gl...@chromium.org> wrote:

[Glenn Adams]

On Sat, Nov 23, 2013 at 10:09 AM, Adam Barth <aba...@chromium.org> wrote:

This feature appears to have a dependency on XML.  Generally speaking, we're not eager to take more dependencies on XML and are looking to remove our existing XML dependencies.  Obviously, removing XML support from Blink is a long term project and one we might never finish completely, but a feature that adds more XML dependencies is going to meet with some resistance.

I expect to make use of the same mechanisms as the current SVG implementation.

My personal opnion is that that Priority of Constituencies [1] applies, which you appear to support [2]. In the case of TTML, failure to support reduces accessibility for the end user, and increases costs for the content author (translation with potential loss of information, less interoperability, etc).

[1] http://www.w3.org/TR/html-design-principles/#priority-of-constituencies

[2] http://www.schemehostport.com/2011/10/priority-of-constituencies.html

[Adam Barth]

On Fri, Nov 22, 2013 at 6:17 PM, Glenn Adams <gl...@chromium.org> wrote:

On Sat, Nov 23, 2013 at 10:09 AM, Adam Barth <aba...@chromium.org> wrote:

This feature appears to have a dependency on XML.  Generally speaking, we're not eager to take more dependencies on XML and are looking to remove our existing XML dependencies.  Obviously, removing XML support from Blink is a long term project and one we might never finish completely, but a feature that adds more XML dependencies is going to meet with some resistance.

I expect to make use of the same mechanisms as the current SVG implementation.

SVG is an example of where we've been experimenting with removing the XML dependency:

https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/TcnL06ma3X4

It's true that we currently use XML for a number of features, but I'm not sure we want to add more features that depend on XML.

My personal opnion is that that Priority of Constituencies [1] applies, which you appear to support [2]. In the case of TTML, failure to support reduces accessibility for the end user, and increases costs for the content author (translation with potential loss of information, less interoperability, etc).

That's fine, but we have the freedom to choose which features we wish to support in trunk.  If you'd like to implement this feature outside of trunk, you have the freedom to do that as well.

Adam

[Glenn Adams]

On Sat, Nov 23, 2013 at 10:29 AM, Adam Barth <aba...@chromium.org> wrote:

On Fri, Nov 22, 2013 at 6:17 PM, Glenn Adams <gl...@chromium.org> wrote:

On Sat, Nov 23, 2013 at 10:09 AM, Adam Barth <aba...@chromium.org> wrote:

This feature appears to have a dependency on XML.  Generally speaking, we're not eager to take more dependencies on XML and are looking to remove our existing XML dependencies.  Obviously, removing XML support from Blink is a long term project and one we might never finish completely, but a feature that adds more XML dependencies is going to meet with some resistance.

I expect to make use of the same mechanisms as the current SVG implementation.

SVG is an example of where we've been experimenting with removing the XML dependency:

https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/TcnL06ma3X4

It's true that we currently use XML for a number of features, but I'm not sure we want to add more features that depend on XML.

Further on this point. I wouldn't be adverse to transitioning to the HTML parser if the experiment with SVG is successful. However, I would prefer to start the work with the existing XML parser and treat such a transition as a follow-on optimization.

[Glenn Adams]

https://code.google.com/p/chromium/issues/detail?id=270413

[Adam Barth]

Looking at the specification in more detail, TTML also appears to also have a dependency on XSL, which is another technology that we'd like to remove from Blink eventually:

https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/zIg2KC7PyH0

In summary, I don't think we're interested in implementing this feature in Blink.

Kind regards,

Adam

[Eric Seidel]

I guess I agree with Adam. XML is bad news bears. It complicates our
engine dramatically, the integration into our engine is riddled with
bugs, and has extremely low usage (as a rendered format -- I expect
the XHR usage is non-zero). XSL is even worse. Both technologies
have lost on the web (at least as client-side display formats) and we
shouldn't be adding new usage of such.

[Glenn Adams]

Reply to all this time.

If my response below is true, and I assure you it is because I wrote the spec, then wouldn't it be reasonable to have the implementation go forward and then evaluate the results when I get to the point of proposing an Intent to Ship? If my implementation introduces the dependencies you worry about, then it would be fair to point that out as a reason for not shipping ... but it doesn't seem particularly reasonable to not implement when you are speculating about eventual dependencies, especially dependencies that are not logically implied by the specification.

There are other reasons to allow an implementation to go forward:

(1) it establishes prior art in an open source forum, from which Google and others can draw upon even if it doesn't eventually pass muster for shipping enabled;

(2) it removes speculation about what is or isn't required to support TTML, about what complexity it introduces (or not), etc.;

(3) it could be moved from core into a chrome module;

(4) it encourages ongoing experimentation, although in this case TTML is an established REC, and not a draft that entails experimenting with an unstable specification;

G.

On Sat, Nov 23, 2013 at 3:13 PM, Glenn Adams <gl...@chromium.org> wrote:

On Sat, Nov 23, 2013 at 1:27 PM, Eric Seidel <ese...@chromium.org> wrote:

I guess I agree with Adam.  XML is bad news bears.  It complicates our
engine dramatically, the integration into our engine is riddled with
bugs, and has extremely low usage (as a rendered format -- I expect
the XHR usage is non-zero).  XSL is even worse.  Both technologies
have lost on the web (at least as client-side display formats) and we
shouldn't be adding new usage of such.

I'm afraid this is an incorrect interpretation of the spec. There is no concrete dependency on XML, XSLT, or XSL-FO as far as either parsing or formatting is concerned, e.g., see [1][2]:

[1] http://www.w3.org/TR/2013/REC-ttml1-20130924/#conformance-generic-processor

"A TTML Content Processor conforms to this specification if the following generic processor criteria are satisfied:

1. The processor provides at least one mechanism for notionally instantiating a Reduced XML Infoset representation of a conformant Document Instance."

[2] http://www.w3.org/TR/2013/REC-ttml1-20130924/#semantics-region-layout-step-2

"In this section, the use of XSL FO is intended to be conceptual only, employed solely for the purpose of defining the normative presentation semantics of TTML. An actual implementation of this algorithm is not required to create or process XSL-FO representations. In particular, it is possible to implement these semantics using alternative presentation models, such as Cascading Style Sheets (CSS)."

The syntax and validity of TTML is defined in terms of a Reduced Infoset [3], which can be constructed from either an XML based parser or a parser like the current Blink HTML parser. In TTML1, the formatting semantics are described using XML-FO terminology but will in fact be expressed using HTML/CSS in this work. In TTML2 (work in progress), a formal definition of the HTML/CSS mapping will be specified. An early draft of one possible mapping can be found here [4].

[3] http://www.w3.org/TR/2013/REC-ttml1-20130924/#reduced-infoset

[4] http://www.w3.org/wiki/TTML/changeProposal005

The bottom line is that there is no impediment to using the HTML parser and no impediment to using the current HTML/CSS renderers. My implementation plan was to use the XML parser code used by SVG as a first step, but as I pointed out, this can be readily converted to using the HTML parser as an optimization.

So there is no "new usage of such" [XML or XSL or XSL-FO] implied (if one allows use of the existing XML parser as an interim step in implementation, with a follow-on to convert to the HTML Parser if the experiment with SVG proves successful).

[Adam Barth]

On Fri, Nov 22, 2013 at 11:53 PM, Glenn Adams <gl...@chromium.org> wrote:

Reply to all this time.

If my response below is true, and I assure you it is because I wrote the spec, then wouldn't it be reasonable to have the implementation go forward and then evaluate the results when I get to the point of proposing an Intent to Ship?

That would be a reasonable approach if we had a high degree of interest in the feature, but I don't think we do.  I'm sorry, but we're not interested in accepting an implementation of TTML into trunk at this time.

If my implementation introduces the dependencies you worry about, then it would be fair to point that out as a reason for not shipping ... but it doesn't seem particularly reasonable to not implement when you are speculating about eventual dependencies, especially dependencies that are not logically implied by the specification.

Everyone who contributes to this project shares responsibility for maintaining the code that lives in trunk.  For this reason, engineering concerns such as unwanted dependencies or constraints are appropriate to address at the implement stage rather than at the shipping stage.

 

There are other reasons to allow an implementation to go forward:

(1) it establishes prior art in an open source forum, from which Google and others can draw upon even if it doesn't eventually pass muster for shipping enabled;

(2) it removes speculation about what is or isn't required to support TTML, about what complexity it introduces (or not), etc.;

(3) it could be moved from core into a chrome module;

(4) it encourages ongoing experimentation, although in this case TTML is an established REC, and not a draft that entails experimenting with an unstable specification;

You're welcome to implement TTML in your own branch of Blink, which would have benefits (1-4) without imposing costs on everyone else contributing to trunk.

 

On Sat, Nov 23, 2013 at 3:13 PM, Glenn Adams <gl...@chromium.org> wrote:

On Sat, Nov 23, 2013 at 1:27 PM, Eric Seidel <ese...@chromium.org> wrote:

I guess I agree with Adam.  XML is bad news bears.  It complicates our
engine dramatically, the integration into our engine is riddled with
bugs, and has extremely low usage (as a rendered format -- I expect
the XHR usage is non-zero).  XSL is even worse.  Both technologies
have lost on the web (at least as client-side display formats) and we
shouldn't be adding new usage of such.

I'm afraid this is an incorrect interpretation of the spec. There is no concrete dependency on XML, XSLT, or XSL-FO as far as either parsing or formatting is concerned, e.g., see [1][2]:

[1] http://www.w3.org/TR/2013/REC-ttml1-20130924/#conformance-generic-processor

"A TTML Content Processor conforms to this specification if the following generic processor criteria are satisfied:

1. The processor provides at least one mechanism for notionally instantiating a Reduced XML Infoset representation of a conformant Document Instance."

[2] http://www.w3.org/TR/2013/REC-ttml1-20130924/#semantics-region-layout-step-2

"In this section, the use of XSL FO is intended to be conceptual only, employed solely for the purpose of defining the normative presentation semantics of TTML. An actual implementation of this algorithm is not required to create or process XSL-FO representations. In particular, it is possible to implement these semantics using alternative presentation models, such as Cascading Style Sheets (CSS)."

The syntax and validity of TTML is defined in terms of a Reduced Infoset [3], which can be constructed from either an XML based parser or a parser like the current Blink HTML parser. In TTML1, the formatting semantics are described using XML-FO terminology but will in fact be expressed using HTML/CSS in this work. In TTML2 (work in progress), a formal definition of the HTML/CSS mapping will be specified. An early draft of one possible mapping can be found here [4].

[3] http://www.w3.org/TR/2013/REC-ttml1-20130924/#reduced-infoset

[4] http://www.w3.org/wiki/TTML/changeProposal005

The bottom line is that there is no impediment to using the HTML parser and no impediment to using the current HTML/CSS renderers. My implementation plan was to use the XML parser code used by SVG as a first step, but as I pointed out, this can be readily converted to using the HTML parser as an optimization.

I don't really understand how that's possible given that TTML appears to use XML namespaces, which aren't supported by the HTML parser.  Perhaps your branch implementation will shed light on these questions.

 

So there is no "new usage of such" [XML or XSL or XSL-FO] implied (if one allows use of the existing XML parser as an interim step in implementation, with a follow-on to convert to the HTML Parser if the experiment with SVG proves successful).

You're welcome to take that approach in your branch, but we are not going to accept an implementation of TTML into trunk at this time.

Adam

[Glenn Adams]

XML namespaces are just attributes, and can be processed at the point of application usage. I have to wonder how your SVG experiment will handle them.

 

 

So there is no "new usage of such" [XML or XSL or XSL-FO] implied (if one allows use of the existing XML parser as an interim step in implementation, with a follow-on to convert to the HTML Parser if the experiment with SVG proves successful).

You're welcome to take that approach in your branch, but we are not going to accept an implementation of TTML into trunk at this time.

I am left with the impression that there is no distinction between Intent to Implement and Intent to Ship. You appear to be pre-judging the proposed work using criteria that applies to the latter.

I guess the priority of constituencies also has little relevance in Blink after all, since it appears my citations of user and content author needs are being dismissed as irrelevant.

[Adam Barth]

XML namespaces are just attributes, and can be processed at the point of application usage. I have to wonder how your SVG experiment will handle them.

 

 

So there is no "new usage of such" [XML or XSL or XSL-FO] implied (if one allows use of the existing XML parser as an interim step in implementation, with a follow-on to convert to the HTML Parser if the experiment with SVG proves successful).

You're welcome to take that approach in your branch, but we are not going to accept an implementation of TTML into trunk at this time.

I am left with the impression that there is no distinction between Intent to Implement and Intent to Ship. You appear to be pre-judging the proposed work using criteria that applies to the latter.

The short answer is that intent-to-implement is more about impact on the codebase whereas intent-to-ship is more about impact on the web ecosystem.  That's why engineering considerations weigh heavily in the former and standards/compatibility weigh heavily in the latter.

I guess the priority of constituencies also has little relevance in Blink after all, since it appears my citations of user and content author needs are being dismissed as irrelevant.

I'm not sure I agree with your characterization, but I'm not sure it would be that productive to pursue this point.

[Glenn Adams]

In that case, can we look at the impact on the codebase alone? The proposed work would almost entirely occur in a new html/track/ttml subdirectory, with a very small integration work occurring outside this directory in:

• loader/TextTrackLoader
• html/track/InbandTextTrack
• html/track/LoadableTextTrack
• rendering/RenderTTMLCue (new)

Could you elaborate what potential negative impact you see from this?

[Adam Barth]

As I wrote before, the negative impact is the additional dependency on XML.

Adam

[Glenn Adams]

If I do the initial implementation based solely on the existing HTML Parser, would you be willing to let it go forward in trunk?

 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Adam Barth]

If you're interested in experimenting with that sort of non-standards based approach, I'd recommend performing that experiment on a branch.

Glenn, this thread has gone on longer than necessary.  We're not interested in having an implementation of this feature in trunk.  Please take no for an answer.

[Glenn Adams]

I'm sorry for belaboring this, but I'm trying to determine what your response is based upon. You say that the Intent to Implement criteria is based on impact to codebase. So I'm attempting to understand what that impact is. However, you appear to be saying "no for an answer" based upon other criteria that you aren't explaining. Are you suggesting it isn't reasonable for me to ask about that?

Regarding "non-standards based approach", could you explain what you mean by that? What I am proposing is a standards based approach, which I have cited the specification above. What part of that is an experiment?

[Brett Wilson]

On Sat, Nov 23, 2013 at 6:11 PM, Glenn Adams <gl...@skynav.com> wrote:
> I'm sorry for belaboring this, but I'm trying to determine what your
> response is based upon. You say that the Intent to Implement criteria is
> based on impact to codebase. So I'm attempting to understand what that
> impact is. However, you appear to be saying "no for an answer" based upon
> other criteria that you aren't explaining. Are you suggesting it isn't
> reasonable for me to ask about that?
>
> Regarding "non-standards based approach", could you explain what you mean by
> that? What I am proposing is a standards based approach, which I have cited
> the specification above. What part of that is an experiment?

I don't really understand this feature, but wouldn't implementing
something that specifies XSL/XML using the HTML parser violate the
spec?

Brett

[Adam Barth]

On Sat, Nov 23, 2013 at 6:11 PM, Glenn Adams <gl...@skynav.com> wrote:

I'm sorry for belaboring this, but I'm trying to determine what your response is based upon. You say that the Intent to Implement criteria is based on impact to codebase. So I'm attempting to understand what that impact is. However, you appear to be saying "no for an answer" based upon other criteria that you aren't explaining. Are you suggesting it isn't reasonable for me to ask about that?

TTML has a number of serious problems:

1) TTML is based on XML, specifically upon XML namespaces.  Developers consistently have trouble authoring content that uses XML namespaces correctly.  Obviously not everyone agrees that XML namespaces make poor APIs, but developer have clearly voted with their feet.  The only browser feature I'm aware of that uses XML namespaces that has significant usage is SVG, and even in that case 88.5% of SVG content that Chrome renders uses SVG-in-HTML, which avoids making authors deal with namespaces.

As I've written upthread, we'd like to reduce our dependencies on XML.  For example, rather than parse 11.5% of SVG content with the XML parser and 88.5% with the HTML parser, I would like to see us parse 100% of SVG content with the HTML parser.  Similarly, I'd like to see us move XSLT support into an optional component that can be installed at runtime and that runs on top of the platform rather than requiring the deep integration it requires today.  These are both long term projects, but introducing a new XML dependency is the wrong direction for the codebase.

2) TTML overlaps with WebVTT, which is already widely implemented [1].  I'm sure that TTML supports more use cases and/or does a better job at supporting some use cases than WebVTT, but I would much rather see us evolve our WebVTT support towards supporting those use cases than see us add a largely redundant feature.  Unlike TTML, WebVTT takes a more modern approach to designing a web platform API and better integrates with HTML.  I understand there are some standards politics involved, but frankly I don't care about that stuff.  I care about ending up with the best technology in the long run, regardless of whether the spec authors aren't popular personalities in the W3C.

3) Although not a problem unto itself, it's red flag that WebVTT has reached REC without any significant browser implementations.  Maybe there are implementations I'm not aware of?  I looked at caniuse.com [2] and Wikipedia [3], neither of which mentioned any implementations.  Compare those references with [4] and [5], which both list a number of significant implementations.

In summary, we're not interested in implementing TTML.  To the extent that WebVTT is insufficient for important use cases, I would much prefer to see us evolve WebVTT in concert with other browser vendors.  If you're still excited about TTML, you're welcome to implement the feature in your own private branch or you're welcome to convince other browser vendors to implement and ship TTML.

I realize that you're likely to be unhappy with this decision and that we might well lose you as a contributor to this project as a result.  I genuinely hope that doesn't happen and that you'll continue to contribute to Blink despite our lack of enthusiasm for TTML.

Kind regards,

Adam

[1] https://developer.mozilla.org/en-US/docs/HTML/WebVTT#Compatibility

[2] http://caniuse.com/ttml

[3] http://en.wikipedia.org/wiki/Timed_Text

[4] http://caniuse.com/webvtt

[5] http://en.wikipedia.org/wiki/WebVTT#WebVTT

[Glenn Adams]

Thanks for asking. 

Firstly, TTML doesn't specify use of XSL/XML, as I pointed out in [1]. In particular, it doesn't use XSLT and it employs XSL-FO only to describe one possible formatting model, a model that happens to be concretely expressible in CSS. It also doesn't require the use of XML and it defines content constraints and semantics in terms of an abstract infoset [2], which is a subset of what is represented by the current DOM representation in Blink. Any parser that can deserialize any concrete representation of TTML into such an infoset satisfies the first step of TTML Generic Processor Conformance [3].

[1] https://groups.google.com/a/chromium.org/d/msg/blink-dev/vXuOTK5M0hM/wRZavcVLmvkJ

[2] http://www.w3.org/TR/2013/REC-ttml1-20130924/#reduced-infoset

[3] http://www.w3.org/TR/2013/REC-ttml1-20130924/#conformance-generic-processor

Secondly, if SVG can be parsed by the Blink HTML parser, then an XML representation of TTML can be parsed by the Blink HTML Parser. TTML doesn't mandate a particular concrete representation of a TTML Document, about which see [4], nor does it mandate a well-formed-ness or validity constraint on a concrete representation.

[4] http://www.w3.org/TR/2013/REC-ttml1-20130924/#concrete-encoding

It is certainly true that TTML documents are typically serialized as XML, but the specification only requires a conforming processor provide "at least one mechanism for notionally instantiating a Reduced XML Infoset representation of a conformant Document Instance".

This conformance requirement is satisfied by any type of parser that produces the resulting infoset. As a consequence, it is entirely standards compliant to use the existing Blink HTML parser rather than an full XML parser.

Note that this is in contrast with SVG which requires:

(1) that a Conforming SVG Document Fragment "is XML well-formed" [5]; and

(2) that Conforming SVG Viewer provide "complete support for the XML 1.0 specification" and "complete support for inclusion of non-SVG namespaces within SVG content" [6].

[5] http://www.w3.org/TR/SVG/conform.html#ConformingSVGDocuments

[6] http://www.w3.org/TR/SVG/conform.html#ConformingSVGViewers

 

Brett

[Glenn Adams]

On Sun, Nov 24, 2013 at 3:28 PM, Adam Barth <aba...@chromium.org> wrote:

On Sat, Nov 23, 2013 at 6:11 PM, Glenn Adams <gl...@skynav.com> wrote:

I'm sorry for belaboring this, but I'm trying to determine what your response is based upon. You say that the Intent to Implement criteria is based on impact to codebase. So I'm attempting to understand what that impact is. However, you appear to be saying "no for an answer" based upon other criteria that you aren't explaining. Are you suggesting it isn't reasonable for me to ask about that?

TTML has a number of serious problems:

I'm afraid I have to disagree with both your premises and conclusions in all of the following points.

 

1) TTML is based on XML, specifically upon XML namespaces.  Developers consistently have trouble authoring content that uses XML namespaces correctly.  Obviously not everyone agrees that XML namespaces make poor APIs, but developer have clearly voted with their feet.  The only browser feature I'm aware of that uses XML namespaces that has significant usage is SVG, and even in that case 88.5% of SVG content that Chrome renders uses SVG-in-HTML, which avoids making authors deal with namespaces.

This conclusion appears to be based on the perception of HTML authors, and not authors of XML documents. TTML authors and users have not expressed any concern about the uses of namespaces. Indeed they consistently support the use of namespaces and their ability to manage extensibility.

 

As I've written upthread, we'd like to reduce our dependencies on XML.  For example, rather than parse 11.5% of SVG content with the XML parser and 88.5% with the HTML parser, I would like to see us parse 100% of SVG content with the HTML parser.  Similarly, I'd like to see us move XSLT support into an optional component that can be installed at runtime and that runs on top of the platform rather than requiring the deep integration it requires today.  These are both long term projects, but introducing a new XML dependency is the wrong direction for the codebase.

I have on numerous occasions in this thread claimed that support for TTML does not increase the reliance upon XML, or at least does not do so in a material way (in the sense of depending on existing Blink XML parser and XML specific functionality). Perhaps you doubt my claims. I guess my only option is to proceed with implementing TTML and post a CL that demonstrates what I'm saying.

 

2) TTML overlaps with WebVTT, which is already widely implemented [1].

WebVTT may be more widely implemented as an experimental feature in common browsers,  but it is not more widely used or authored. In fact, the opposite is the case. Commercial television providers and studios primarily use TTML. Have you consulted with them about their requirements? I would be happy to arrange a conference in person or by phone to bring stakeholders together.

 

I'm sure that TTML supports more use cases and/or does a better job at supporting some use cases than WebVTT, but I would much rather see us evolve our WebVTT support towards supporting those use cases than see us add a largely redundant feature.

Adding support for TTML does not interfere with or detract from evolving WebVTT support. Indeed, I have recently been actively contributing to Blink's WebVTT implementation, in part because I wished to demonstrate that I support WebVTT (as well as TTML) and am not playing favorites. As you may know, WebVTT is a W3C Community Group draft, which still has some distance to go before it moves into and emerges from the W3C REC track process. At this stage, there is none of the usual W3C IPR commitments regarding WebVTT, while TTML is covered under W3C RF IPR policy and is a stable spec (REC), widely adopted by external standards organizations.

 

Unlike TTML, WebVTT takes a more modern approach to designing a web platform API and better integrates with HTML.  I understand there are some standards politics involved, but frankly I don't care about that stuff.  I care about ending up with the best technology in the long run, regardless of whether the spec authors aren't popular personalities in the W3C.

I'm not sure about your comment about personalities, but I have no preferences in this regard, and view both TTML and WebVTT as independent formats with different features and capabilities, different author and user communities, and so on. However, I do know that content studios, video applications, and video service providers use TTML as their primary caption/subtitle format. Lack of support for TTML hampers the ability of content authors to deliver captions/subtitles, negatively impacts the user experience, and, in some context, may be contrary to government regulations.

 

3) Although not a problem unto itself, it's red flag that WebVTT has reached REC without any significant browser implementations.

I assume you mean TTML here since WebVTT hasn't entered the REC track. On that assumption, are you suggesting the IE10/11 implementation of TTML is insignificant?

 

Maybe there are implementations I'm not aware of?

I provided the following pointers in my initial proposal. Perhaps you didn't look at them.

[6] http://msdn.microsoft.com/en-us/library/hh834514(v=vs.85).aspx

[7] http://msdn.microsoft.com/en-us/library/ie/dn265042(v=vs.85).aspx

 

I looked at caniuse.com [2] and Wikipedia [3], neither of which mentioned any implementations.  Compare those references with [4] and [5], which both list a number of significant implementations.

I do notice that from [4] that Firefox is not yet shipping WebVTT (as of 27.0), so support for it is certainly not yet universal, which is no surprise given the early stage it is in within the W3C process. Indeed, it is undergoing significant design change as we speak (related to WebVTT regions).

 

In summary, we're not interested in implementing TTML.

Again, I wonder a bit who "we" is here, since I've been told by Google TV folks that not only are they interested, but that they would like to assign it a high priority. I also wonder what the Android and ChromeOS folks desire given that they include customers in the C.E. industry and Television and Set Top Box manufacturers who have regulatory requirements to support TTML.

Perhaps you are just saying that you believe TTML is not yet ready for Blink but perhaps could be added to Chromium outside the Blink trunk, perhaps as a module or standard plug-in or add-on?

I was assuming that the Intent to Implement process actually provided an opportunity to incrementally implement a feature over which time it could be further evaluated prior to making a decision when an Intent to Ship was proposed. However, in the present case, you appear to be preempting the former process by pre-judging the results based on your estimation of whether the latter process will succeed. And further, it is not clear to me that you have sufficient input from industry or even internal Google customers to reach a conclusion this early. I say this because it contradicts what I'm hearing from industry, from Google TV folks, etc.

 

To the extent that WebVTT is insufficient for important use cases, I would much prefer to see us evolve WebVTT in concert with other browser vendors.

Again, adding support for TTML doesn't detract from WebVTT, and, indeed, is likely to have the opposite effect: improve WebVTT for the following reasons:

(1) integration between and harmonization WebVTT and TTML semantics will improve since their will be efforts along both paths and recognition of common needs and uses;

(2) we won't need to keep having this rather pointless WebVTT versus TTML argument, which serves nobody, particularly end users and content authors;

 

If you're still excited about TTML, you're welcome to implement the feature in your own private branch or you're welcome to convince other browser vendors to implement and ship TTML.

I intend to implement TTML and upload a CL with that implementation regardless of whether I am pre-empted from the normal process which I assumed would come with a community of users: mutual respect for valuable work, whether or not it actually ships in the long run.

 

I realize that you're likely to be unhappy with this decision and that we might well lose you as a contributor to this project as a result.  I genuinely hope that doesn't happen and that you'll continue to contribute to Blink despite our lack of enthusiasm for TTML.

I don't feel unhappy as much as I feel rather slighted as a good willed member to this community. The motivations for TTML that I cited in my proposal appear to have been largely ignored without making an effort to contact stakeholders: has anyone on the Blink Team discussed TTML with the companies I cite? BBC, Comcast, Cox, Netflix, Time Warner, and others, such as Movielabs, EBU, DECE? Is the Blink Team saying their needs or the needs of their end users (who use the Open Web to access their services) do not matter?

Overall, I get the sense that the process you described "intent-to-implement is more about impact on the codebase whereas intent-to-ship is more about impact on the web ecosystem" is not being followed here. I say this because I don't see any negative "impact on the codebase".

I guess my only option, if the team's decision remains negative, is to proceed with implementing and post a CL containing that implementation that demonstrates there is no negative impact on the codebase. To that end, it would be useful for the Blink Team to arrange for a public mirror on github that facilitates feature development in a public forum. [At present, there doesn't seem to be an official mirror, only a few unofficial ones.]

[Adam Barth]

On Sun, Nov 24, 2013 at 1:48 AM, Glenn Adams <gl...@chromium.org> wrote:

I guess my only option, if the team's decision remains negative, is to proceed with implementing and post a CL containing that implementation that demonstrates there is no negative impact on the codebase.

You're welcome to upload a CL, but please be aware that such a CL will not be accepted into trunk.

Adam

[Jochen Eisinger]

When Adam says "we", I guess he refers to the set of Blink API owners: http://www.chromium.org/blink#TOC-API-Owners

It's possible to create branches directly on chromium.googlesource.com: https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/ytcz72nXJ5s Of course, using github is also fine.

hth

-jochen

[Peter Beverloo]

Using "we" in discussions like this is very ambiguous.  It would be a lot clearer if we would only do that when clarifying the context under which such a consensus was created: meeting notes, discussion among API owners, and so on.  Reading this discussion it very much feels like two individuals disagreeing, especially now that Glenn proposed to implement TTML based on the HTML parser.

It has to be noted that the Internet Explorer implementation of TTML[1] is very incomplete at best, and should thus not be considered following Blink's compatibility guidelines unless you plan to implement the exact same subset of TTML.  Furthermore, Mozilla's stance on TTML support is negative[2], and they raised concerns in developing WebVTT and TTML in a single working group[3].  David also states in the dev.platform list that he received a bunch of (off-list?) pushback on not supporting TTML.  I would expect the Google TV and Chromecast interests in TTML to exist for similar reasons.

I'm very much in favor of not supporting several technologies for reaching the same result, and think it would be a lot more productive to pursue development of WebVTT for any missing features.  Perhaps you could invest time in a high quality TTML to WebVTT converter instead?  (Possibly even as a JavaScript library so sites can do this automatically.)  At the same time, I also think it would be good for someone working on Google TV or Cast to join this discussion to clarify their point of view.

Thanks,

Peter

[1] See the PDF on http://msdn.microsoft.com/en-us/library/hh834514(v=vs.85).aspx

[2] http://markmail.org/message/5f6udrvlyi6t7pxo#query:+page:1+mid:5f6udrvlyi6t7pxo+state:results

[3] https://lists.w3.org/Archives/Member/w3c-ac-forum/2013AprJun/0136.html

[PhistucK]

As an outsider, I completely agree with the first paragraph.

☆PhistucK

[Glenn Adams]

On Sun, Nov 24, 2013 at 7:30 PM, Peter Beverloo <pe...@chromium.org> wrote:

Using "we" in discussions like this is very ambiguous.  It would be a lot clearer if we would only do that when clarifying the context under which such a consensus was created: meeting notes, discussion among API owners, and so on.  Reading this discussion it very much feels like two individuals disagreeing, especially now that Glenn proposed to implement TTML based on the HTML parser.

It has to be noted that the Internet Explorer implementation of TTML[1] is very incomplete at best, and should thus not be considered following Blink's compatibility guidelines unless you plan to implement the exact same subset of TTML.  Furthermore, Mozilla's stance on TTML support is negative[2], and they raised concerns in developing WebVTT and TTML in a single working group[3].

The W3C Timed Text WG met last week in Shenzhen at TPAC13, with joint participation from the Timed Text (WebVTT) Community Group, including the WebVTT editor (Silvia Pfieffer) and chair of that CG (Dave Singer), and others. There is no disagreement or concerns on proceeding with joint work among the key players, and nobody in the meeting expressed a negative opinion about conducting the work jointly. The W3C Management expects to recharter the TTWG with deliverables of both TTML2 and WebVTT (for REC track), and it is my understanding that the WG will be co-chaired by Nigel Meggit (BBC) and Dave Singer (Apple).

I am a member of both TTWG and TTCG, as well as a W3C Advisory Committee representative, so I'm following these developments closely. By and large, Mozilla has been inactive in these areas.

 

David also states in the dev.platform list that he received a bunch of (off-list?) pushback on not supporting TTML.  I would expect the Google TV and Chromecast interests in TTML to exist for similar reasons.

I don't know about that, but I do know that there are a significant number of full W3C Members (not anonymous individuals) in full support of both TTML and WebVTT, neither to the exclusion of the other, and that expect browser vendors to rectify the discrepancy in lack of support of TTML. I'm sure that Google and other vendors not yet supporting TTML will eventually add support for TTML as a matter of course or because their policy wonks require them to do so to satisfy US and European regulatory requirements.

Most of the rationale given in the past for not implementing TTML was based on (1) lack of understanding of the technical requirements of TTML, and (2) lack of interface with and understanding of the television and video services industries and the well established use of TTML in the commercial caption/subtitle production chain.

That WebVTT can or will supplant this usage seems a far reach, and expecting that authors must author in WebVTT or convert to WebVTT in order to deliver captions to a browser is not only technically questionable, but stands the priority of constituencies on its head.

 

I'm very much in favor of not supporting several technologies for reaching the same result, and think it would be a lot more productive to pursue development of WebVTT for any missing features.

Would you also suggest every image author or video author or audio author to use one and only one format? TTML and WebVTT address different requirements, have different capabilities, and different user and authoring communities. While I am completely certain that TTML cannot be translated to WebVTT without loss of information, it may also hold in the opposite direction (though I can't say I've attempted a study of that question).

 

Perhaps you could invest time in a high quality TTML to WebVTT converter instead?

Perhaps after I post a CL implementing TTML in Blink. The exercise of creating a converter will be useful, if only to determine the facts about what can translate and what can't translate. It is clearly evident that they are not isomorphic formats, or even close for that matter in some key areas.

 

(Possibly even as a JavaScript library so sites can do this automatically.)  At the same time, I also think it would be good for someone working on Google TV or Cast to join this discussion to clarify their point of view.

I expect some input from Andrew Jeon on this point, though he is OOO until Monday. Andrew is the one who filed the comment at https://code.google.com/p/chromium/issues/detail?id=270413#c1.

[PhistucK]

Just a note regarding "one and only one format" - JavaScript is such an example, for better or for worse.

☆PhistucK

[Glenn Adams]

On Sun, Nov 24, 2013 at 10:44 PM, PhistucK <phis...@gmail.com> wrote:

Just a note regarding "one and only one format" - JavaScript is such an example, for better or for worse.

It's the only content category in the OWP where that's true. In the case of captions/subtitles, it is definitely not true, as there are dozens of formats, both in production and delivery. Numerically, the most common format for broadcast (in US) is CEA-608, and for stored media is DVD Subtitles (a strictly image based format); in early Web uses, there were QText (Apple), RealText (RealNetworks), and SAMI (MSFT). In Europe, there are STL (EBU 3264) and DVB-Subtitling (ETSI EN 300 743), as well as the older WST (CCIR 653), updated as Enhanced Teletext (ETSI ETS 300 706).

TTML work started in 2003, with initial goal of providing common interchange format among (most of) the above cited formats, then published as W3C REC in 2010, and in the interim widely adopted by the television production communities, as well as adopted by regional and international television standards bodies, including SMTPE, EBU, ISO, ITU; WebVTT is a very recent development and, though carriage of it was recently adopted in ISO/IEC 14496-30, the format itself has not been endorsed or used in commercial television content production.

Lack of cross-browser support has forced content providers to employ JS as a polyfill, downloading TTML to client with XHR then performing translation to HTML/CSS; this reduces performance and increases network bandwidth usage and leads to inconsistent presentation among different polyfills; the work proposed here seeks to eliminate those disadvantages by providing browser support for performing that TTML download and translation to HTML/CSS.

With the adoption of TTML as safe harbor format for interchange and delivery by the FCC (US) and developments around EBU-TT (EU) [1], the call for support will only increase.

WebVTT is based on the SubRip Text (SRT) format, a home grown format developed in 2006 [2]. Although based on SRT, WebVTT syntax has diverged in some areas. WebVTT is not used in commercial caption/subtitle production as far as I'm aware. WebVTT is beginning to be supported by some caption service providers [3]. The predominant use of WebVTT appears to be captioning of YouTube content.

[1] http://www.youtube.com/watch?v=I9FDQyxBXEo

[2] http://en.wikipedia.org/wiki/SubRip

[3] http://www.cpcweb.com/solutions/web.htm

[Anne van Kesteren]

On Sun, Nov 24, 2013 at 1:12 PM, Glenn Adams <gl...@skynav.com> wrote:
> I am a member of both TTWG and TTCG, as well as a W3C Advisory Committee
> representative, so I'm following these developments closely. By and large,
> Mozilla has been inactive in these areas.

Actually, on behalf of a number of people in Mozilla interested in
this space I participated in a panel during an AC meeting and pushed
back on having two formats. The feedback we gave on the W3C activities
surrounding this is available publicly here:
http://lists.w3.org/Archives/Public/www-archive/2013May/0034.html


--
http://annevankesteren.nl/

[Anne van Kesteren]

On Mon, Nov 25, 2013 at 11:48 AM, Anne van Kesteren <ann...@annevk.nl> wrote:
> http://lists.w3.org/Archives/Public/www-archive/2013May/0034.html

More statements about Mozilla's preference for WebVTT only can be
found here: https://groups.google.com/d/topic/mozilla.dev.platform/m9K94yLq9-U/discussion


--
http://annevankesteren.nl/

[Andrew Scherkus]

I do have input from industry and internal Google customers. I understand there's industry support for TTML, but that's what the TextTrack API is for: to provide a clear and supported path for rendering captioning formats that aren't a good candidate for browser engines.

Given Adam's objections, I don't see a compelling reason to move forward with a TTML implementation in Blink.

Andrew

[Elliott Sprehn]

On Mon, Nov 25, 2013 at 5:12 PM, Andrew Scherkus <sche...@chromium.org> wrote:

...

I do have input from industry and internal Google customers. I understand there's industry support for TTML, but that's what the TextTrack API is for: to provide a clear and supported path for rendering captioning formats that aren't a good candidate for browser engines.

Given Adam's objections, I don't see a compelling reason to move forward with a TTML implementation in Blink.

+1. If WebVTT and TextTrack APIs are insufficient we should expand them such that you can support TTML and other formats with js libraries.

- E

[Glenn Adams]

It is worth pointing out that according to this logic, support for WebVTT could be removed from Blink. Present support merely translates WebVTT to HTML/CSS, so client JS could similarly be used to perform this translation.

In any case, in order to effectively support TTML using the TextTrackCue API with client JS translation, some small amount of additional work is needed, particularly to support in-band TTML embedded in media streams. This could be enabled by implementing [1], which does not require the browser to translate TTML to a renderable form (HTML/CSS), but exposes it to client JS to perform this translation.

[1] https://dvcs.w3.org/hg/ttml/raw-file/default/ttml1-api/Overview.html

 

Given Adam's objections, I don't see a compelling reason to move forward with a TTML implementation in Blink.

How would you formulate Google's response to customers that require TTML support to satisfy the FCC safe harbor format requirements? Clearly MSFT felt compelled to address this by providing TTML rendering support (albeit a subset [2]). Are you saying that Google will ignore these requirements or that Google's solution is to expose TTML content to client JS and require application to do rendering? It seems you are suggesting the latter, but I haven't heard anyone express this explicitly with respect to the FCC order. Does the loss of efficiency and possible interoperability problems entailed by this approach not negatively impact end users?

I keep hearing folks say that TTML is not a good candidate for browser engines. But I don't hear people saying how they will satisfy FCC requirements and maintain good performance and end user experiences.

[2] http://www.w3.org/TR/2013/NOTE-ttml10-sdp-us-20130205/

 

Andrew

[Andrew Scherkus]

There are many different solutions that are beyond the scope of this Intent To Implement. Server-side translation. Contributing to the development of JS libraries [1]. Using third-party services to translate captioning formats and/or host content that already satisfy FCC requirements. I haven't heard any compelling reasons why a server and/or client side solution is unacceptable, but I've heard plenty of reasons why implementing TTML isn't great for browser engines.

We're now venturing off topic. As far the Intent To Implement is concerned I believe you have an answer and I apologize that it's not the one you were looking for.

Andrew 

[1] http://captionatorjs.com/

[Silvia Pfeiffer]

Have they tried to convert from TTML to WebVTT for presentation in
browsers? Since all major browsers now support WebVTT, it would the
path of least pain. It would also help to find out which TTML features
cannot be presented in WebVTT. You might find that to be a very small
set.


Regards,
Silvia.

[Silvia Pfeiffer]

On Mon, Nov 25, 2013 at 12:12 AM, Glenn Adams <gl...@skynav.com> wrote:
>
> On Sun, Nov 24, 2013 at 7:30 PM, Peter Beverloo <pe...@chromium.org> wrote:
>>
>> Using "we" in discussions like this is very ambiguous. It would be a lot
>> clearer if we would only do that when clarifying the context under which
>> such a consensus was created: meeting notes, discussion among API owners,
>> and so on. Reading this discussion it very much feels like two individuals
>> disagreeing, especially now that Glenn proposed to implement TTML based on
>> the HTML parser.
>>
>> It has to be noted that the Internet Explorer implementation of TTML[1] is
>> very incomplete at best, and should thus not be considered following Blink's
>> compatibility guidelines unless you plan to implement the exact same subset
>> of TTML. Furthermore, Mozilla's stance on TTML support is negative[2], and
>> they raised concerns in developing WebVTT and TTML in a single working
>> group[3].
>
>
> The W3C Timed Text WG met last week in Shenzhen at TPAC13, with joint
> participation from the Timed Text (WebVTT) Community Group, including the
> WebVTT editor (Silvia Pfieffer) and chair of that CG (Dave Singer), and
> others. There is no disagreement or concerns on proceeding with joint work
> among the key players, and nobody in the meeting expressed a negative
> opinion about conducting the work jointly.

There have been objections raised by people not present in the
meeting. Since only David and myself were present from the WebVTT
community, you can't take our participation to imply the rest of the
WebVTT community and developers. In particular, Mozilla's objections
still stand and we will have to address them.

> The W3C Management expects to
> recharter the TTWG with deliverables of both TTML2 and WebVTT (for REC
> track), and it is my understanding that the WG will be co-chaired by Nigel
> Meggit (BBC) and Dave Singer (Apple).

I do not want to stir up the discussion between TTML and WebVTT, but
I'd like to point out that browsers don't care about the REC track -
it is only other standards organisations and the FCC that care about
the REC track. I personally only agreed to take WebVTT on the REC
track in the W3C because it will cause less trouble with such
organisations. REC track has nothing to do with the maturity of the
technology or its implementation status in browsers. If it did, TTML
could not be a Recommendation.

Regards,
Silvia.

[Silvia Pfeiffer]

On Tue, Nov 26, 2013 at 11:16 AM, Glenn Adams <gl...@skynav.com> wrote:
>
> I keep hearing folks say that TTML is not a good candidate for browser
> engines. But I don't hear people saying how they will satisfy FCC
> requirements and maintain good performance and end user experiences.

The FCC requirements are a list of features that have to be supported
- the FCC does not state that you have to support TTML. TTML is only
specified as a "safe harbour" format, but any other format that
satisfies the list of features is acceptable, too.

(Sorry I came to this thread late and am only sending some after-thoughts.)

Regards,
Silvia.