epub-type (EPUB Zero straw man proposal)

[Baldur Bjarnason]

# epub-type (EPUB Zero straw man proposal)

## Problem

E0 needs a mechanism for attaching ebook specific metadata to nodes in HTML files.

Ideally, the mechanism should be able to reuse the [EPUB 3 Structural Semantics Vocabulary][1].

Just using epub:type from the EPUB format is problematic as it is almost certain to cause problems with the HTML serialisation.

## Proposal

The easiest way to solve this problem is to define a new attribute that is functionally identical to epub:type but won't cause problems in HTML.

So, I've suggested using a vendor prefix mechanism and that we use epub-type as an attribute that is otherwise identical to epub:type.

## Why not data-epub-type?

Using a custom attribute clearly signals that this is an extension of the default HTML semantics and not a carrier for internal application state.

As the HTML5 spec says:

> Custom data attributes are intended to store custom data private to the page or application, for which there are no more appropriate attributes or elements.
>
> These attributes are not intended for use by software that is independent of the site that uses the attributes.

Which would seem to indicate that data-* attributes should be reserved for use by scripts embedded in the ebook or for storing private data for the reading system itself and not for storing transferable and interoperable metadata

Now there is no practical reason that prevents us from using data-epub-type but I personally think that this would be a clear case of abusing the feature just to avoid the validation issue with epub-type.

Which brings me to…

## Issues

The major issue with using epub-type is that regular XHTML5 and HTML5 validators would flag files using it as invalid. This should not affect the functionality of the files.

## Alternatives

[RDFa lite][2] is one alternative that I find markedly nicer to look at than microdata.

This would look something like this (of course we'd have to settle on a URL for e0 specific types):

<nav vocab="http://epubzero.blogspot.com/" typeof="Spine">

A link to a footnote using epub's vocab would look something like this:

<a href="#footnote1" vocab="http://idpf.org/epub/vocab/structure/" typeof="noteref">link to footnote</a>

Microdata is considerably less readable with its overuse of item* properties and has the additional downside of being only a working draft while RDFa lite is a recommendation.

Using RDFa lite also avoids the validation issue since [newer validators][3] already support it (as well as microdata, but I don't care about microdata because it looks ugly)

***

My personal preference is in this order, from most preferred to the least:

1. epub-type
2. RDFa lite
3. data-epub-type or data-e0-type
4. Microdata (because it looks ugly)

I can live with any one of them, except for microdata. Because it looks ugly.

- best
- baldur

[1]: http://idpf.org/epub/vocab/structure/
[2]: http://www.w3.org/TR/rdfa-lite/
[3]: http://validator.w3.org/nu/

[Hadrien Gardeur]

## Issues

The major issue with using epub-type is that regular XHTML5 and HTML5 validators would flag files using it as invalid. This should not affect the functionality of the files.

Do we have any feedback from the HTML5 ARIA WG on that ?

They had to deal with the same problems, what motivated their choice ?

Hadrien

[Baldur Bjarnason]

Well, from what I can discover by browsing things like bugzilla discussions[1][2], W3C list archives[3], and ARIA WG minutes[4] it looks like they felt they had no other choice.

Using namespaces introduced interoperability problems between the XHTML and HTML serialisations and problems with legacy browsers. Using some other mechanism for attaching that metadata would have involved teaching authors and developers two complicated things (i.e. RDF and ARIA) instead of one and would have absolutely murdered adoption.

Using the aria-* prefix was, it looks like, their only option, and it involved convincing the HTML5 people to add it wholesale to the spec. The same, AFAICT, was done with RDFa Lite later on. It was also, judging by the bugzilla discussions, the solution that browser implementors favoured.

Also as Henri Sivonen said in [4] "You can still write RELAX-NG for aria-*. Can also write SAX code to deal with this. Validation technology shouldn't affect authoring."

As Al Gillman summarised here in an email to the org.w3.www-tag mailing list in 2008[5]

"Agreed. Since aria:* cannot work consistently (in terms of syntax and DOM representation) everywhere but aria-* can, aria-* was chosen."

Now, we aren't quite in the same situation as ARIA was since we have two valid alternatives we can use. One involves abusing the data-* attribute space. The other involves RDFa Lite (microdata being still just a working draft and, y'know, ugly).

Another issue with using the data-* space is that it's becoming quite common for javascript libraries and other systems to use it for their data instead of loading everything onto classes. The data-epub-type or data-e0-type attributes could very easily and quickly become lost in the data-* noise as its use proliferates.

Now, my feeling is that epub-type is a much better solution for authors. It's much easier to use, understand, and implement than the alternatives. But, long term, it would probably mean either trying to get the HTML5 WG to accept it or, at the very least, making sure that authors have access to validation tools modified to understand epub-type.

- best
- baldur

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=391713
[2]: https://bugzilla.mozilla.org/show_bug.cgi?id=398910
[3]: http://markmail.org/message/tizkmyxtkx5rkzkk#query:+page:1+mid:2pmfsojmmowfwxqh+state:results
[4]: http://www.w3.org/2007/11/06-aria-minutes.html
[5]: http://markmail.org/message/tizkmyxtkx5rkzkk#query:+page:1+mid:nsldaqtyy4dpaayf+state:results

[Daniel Weck]

FYI:

RDFA-lite / microdata is under consideration for EPUB 3.0.1

http://code.google.com/p/epub-revision/issues/detail?id=267

"We punted on adding either rdfa or microdata for semantic enrichment of content in the last revision as it was not clear which direction the W3C would take with the two.

As it's now clear that both will co-exist, we should consider adding support for them."

/daniel

[Eric Daspet]

Hi Baldur

> E0 needs a mechanism for attaching ebook specific metadata to nodes in HTML files.

May I ask the basic naive question ?

Do we *really* need such metadata ? That is : I am sure it would be
really really great to have many metadata, but to achieve our
purposes, is this necessary ?

HTML don't have a way to express even a quarter of these attributes
(but some of them may have an html 5 dedicated tag or attribute). This
didn't blocked anyone from reading HTML so far, and HTML is the most
widespread format so far for reading (far before epub). This may be an
hint that they are not really necessary for our purpose (even if
great).

In facts even epub readers don't use most of them and very few are
widely used with a real final-user added value. Should them be
necessary for reading, they would probably already be in HTML 5. I
suspect we probably could leave most of them if not all to epub 3 and
have an E0 with no dedicated attribute.

Further, if one or some of these attributes are really usefull and
somewhat necessary for reading, we should probably add them to HTML
first instead of using multiple vocabularies. The only one I see for
that may be footnote.


--
Éric Daspet
http://eric.daspet.name/

[Baldur Bjarnason]

I don't think that's a naive question. It's something I've asked as well. Apparently, there is a consensus that we need the ability to have separate spine and toc. For that to work we need a metadata mechanism.

Now, I personally think that it's all unnecessary and that epub-type/epub:type/whatever is complicated enough to never attain serious adoption anyway, but I don't disagree with it strongly enough to play Gandalf on the bridge standing against the Balrog. And if the consensus is that we need a metadata mechanism I'd rather limit the damage by choosing the simplest, easiest to use alternative.

OTOH, RDFa Lite *is* in HTML5 and is a W3C Rec so going with that would fulfil your 'use what comes in the HTML5 box' suggestion. That would have the side benefit of enabling the use of EPUB3 vocab on the wider, non-ebook, web since RDFa Lite should work everywhere.

- best
- baldur

[Dave Cramer]

I see the need for a separate spine vs nav, and the other things we mentioned. But I don't think all, or even most, books will need those features. index.html with a single nav wouldn't need epub-type or RDFa at all, I would hope. If we need to be complicated, at least let us be optionally complicated.

I'm leaning towards an RDFa approach, as I am quite concerned about validation, but that could just be my hang-up from too many years of DocBook and epubcheck :)

I do think the issue of these  attributes will also depend on how we deal with metadata in general. If we decide we need RDFa to express container and document metadata, it would make sense to also use it for the epub-type function.

Dave

[Bill McCoy]

> HTML is the most
>  widespread format so far for reading (far before epub)...RDFa is in HTML5...would have the side benefit of enabling the use of EPUB3 vocab on the wider, non-ebook, web since RDFa Lite should work everywhere.
>
RDFa Lite may be right direction as epub:type was the stand-in 2+ years ago, but Im just curious, in what practical way does it presently "work" and epub:type is presently not "enabled"... tag soup and xhtml parsers in browsers surely ignore unknown attributes, do they not? And epub is prob the biggest use of xhtml. Buy maybe I'm missing something...

[Baldur Bjarnason]

On 8 Mar 2013, at 21:19, Bill McCoy <whm...@gmail.com> wrote:

> > HTML is the most
> > widespread format so far for reading (far before epub)...RDFa is in HTML5...would have the side benefit of enabling the use of EPUB3 vocab on the wider, non-ebook, web since RDFa Lite should work everywhere.
> >

> RDFa Lite may be right direction as epub:type was the stand-in 2+ years ago, but Im just curious, in what practical way does it presently "work" and epub:type is presently not "enabled"... tag soup and xhtml parsers in browsers surely ignore unknown attributes, do they not? And epub is prob the biggest use of xhtml. Buy maybe I'm missing something…
>

Yes. You are missing something.

There are three different ways that browsers handle colons in HTML attributes both in terms of the DOM and in terms of CSS selectors:

1. HTML in any of the pre-10 IEs treat it in a buggy way which has yet to be reverse-engineered and documented properly.
2. XHTML in non-IE browsers use XML parsing and are namespaces-aware.
3. HTML in non-IE browsers and IE 10 treat it in the same way they treat any other unknown attribute (i.e. identically to the way they treat, say, epub-type).

No current browser ignores unknown attributes. Nor do they ignore unknown tags for that matter. They haven't for a long time, that's why it was relatively easy for HTML5 to add new semantic tags such as section and article (except for, as always, the eternal issues with getting the older IEs to accept unknown tags, but that was enabled with a fairly simple hack).

You've been able to use arbitrary attributes in HTML and XHTML in web browsers for more than a decade because none of them attempt to validate the document. You can make up an attribute and it'll be accessible in both the DOM and in CSS, just like any other.

Here, for example is an article from 2003 on their use: http://www.digital-web.com/articles/forms_usability_and_the_w3c_dom/ (scroll down to 'Custom attributes').

Here's a discussion of the pros and cons of custom attribute use use by dojo in 2007. http://danwebb.net/2007/10/7/custom-attributes-and-class-names

The problem is that if you include a colon, they have three different behaviours when it comes to the DOM and CSS (IE's, namespace-aware DOM, and regular HTML DOM).

As soon as you lose the colon these three groups converge and behave the same.

Having three different behaviours for the same attribute is the opposite of interoperability. You'd end up, for example, having to include three different selectors in your stylesheet if you wanted to style a link marked with epub:type="noteref". As your use of epub:type increases and a soon as you start to include scripting into the picture, the more this triplication will inconvenience authors.

What would happen is that people will avoid using epub:type in their e0 files to avoid this mess. It would destroy adoption.

If we ignore the IEs for the moment (which isn't something we could do in actual practice) then we still have a problematic interop split: colons have a special meaning in XHTML but don't in HTML. There is always going to be an interoperability issue between the two when it comes to namespaced attributes and elements. It would result in a permanent duplication of code in JS and CSS that tried to target both the XHTML and HTML serialisations.

RDFa Lite, being composed of a set of regular attributes without a colon work in every major browser today using the same DOM methods and the same CSS selectors. It is also being included in new versions of most, if not all, validators. So, it 'works', according to most definitions of the word, with or without the scare quotes. epub-type also works, but with validation issues. data-epub-type works too but is overloading an already overused prefix. epub:type doesn't work in this context because it has interop problems between HTML and XHTML.

And, the point I made in my email was this "enabling the use of EPUB3 vocab on the wider, non-ebook, web". It was not about whether epub:type was enabled or not but about enabling the author to use that vocabulary in an accepted and valid metadata system.

- best
- baldur

[Bill McCoy]

Thanks Baldur,

Between #2 and #3, how would the CSS selector have to differ between say Chrome and IE10 to style a link marked with epub:type="noteref"? It does seem inconvenient if they have to be different. Do they have to differ regardless of with what Content-Type the content is being served up?

I guess then the issues with epub:type are maybe a subset of the issues one faces in creating what is called "polyglot" markup (markup that is validly both HTML and XHTML), is that right?

--Bill

[Hadrien Gardeur]

May I ask the basic naive question ?

Do we *really* need such metadata ? That is : I am sure it would be
really really great to have many metadata, but to achieve our
purposes, is this necessary ?

I can think of several use cases without thinking too hard about it:

• the general consensus is that having a separate reading order and ToC can be useful in many cases, we'll need to identify different <nav> elements
• footnotes definitely deserve some special attention
• when generating a sample file from a full publication, detecting where the publication really starts (excluding the preface, copyright pages, cover etc.)  can be incredibly useful and avoid the distribution of samples where only the ToC is available
  
• dictionaries are useless without this kind of information
• aside from reading order and ToC, being able to identify a <nav> element for panel by panel navigation for comics could be extremely useful

I'm certainly not saying that a RS will understand all of these semantic elements, but I expect some of them to be widely adopted for some very common use cases.

Since these elements are all optional, they don't really add any kind of complexity to EPUB NG, and I don't see any reason to exclude them from the scope of our project.

Hadrien

[Baldur Bjarnason]

On 8 Mar 2013, at 23:23, Bill McCoy <whm...@gmail.com> wrote:

> Thanks Baldur,
>
> Between #2 and #3, how would the CSS selector have to differ between say Chrome and IE10 to style a link marked with epub:type="noteref"? It does seem inconvenient if they have to be different. Do they have to differ regardless of with what Content-Type the content is being served up?

It would depend on whether the document is XHTML or not. The problem does not really lie in implementation differences between any of the current browsers but in the differences between XHTML and HTML. IE10, thankfully, is the first IE to do things properly and so would behave exactly like Chrome. I should have included it in #2. Earlier IEs and Chrome however are different no matter the content type, as in attaching the style won't work in HTML (it might work in XHTML in IE9, which was the first IE to support XHTML).

To style a link marked with epub:type="noteref" you would use a[epub\:type="noteref"] as a selector when targeting HTML (escaping the colon) but in XHTML it would be:

@namespace epub "http://www.idpf.org/2007/ops";
a[epub|type="noteref"] { font-weight: bold; }

So, to target both you would have to do:

@namespace "http://www.w3.org/1999/xhtml"; /* This is the default namespace */
@namespace epub "http://www.idpf.org/2007/ops";
a[epub\:type="noteref"] , a[epub|type="noteref"] { font-weight: bold; }

This is a lot messier and a lot more complicated than is usable. Most people and authoring tools would simply skip this and just use a class or a data-* attribute and select on that.

Styling using an attribute selector with a colon in it is simply not possible in IE9 or lower, as far as I know. Not reliably. Except maybe in XHTML in IE9.

The same kind of XHTML/HTML schism appears in the DOM where in the former you would have to use getAttributeNS while in the latter you could use the regular getAttribute method leading to code duplication for scripts and libraries trying to target both serialisation types.

The XHTML/HTML schism is more important than the buggy IEs since IE10 fixes almost all of the bugs and is generally a first rate browser. Time will now take care of the IE problem.

>
> I guess then the issues with epub:type are maybe a subset of the issues one faces in creating what is called "polyglot" markup (markup that is validly both HTML and XHTML), is that right?

In a way, yeah. epub:type or anything like it is exactly the sort of thing you can't do with polyglot markup at the moment. Polyglot markup does not allow for arbitrary namespaces and only includes markup recognised by both HTML5 and XML (xlink, svg, mathml, and aria). There are people who want to add extensibility to polyglot markup but it would involve finding new ways of embedding arbitrary markup (XML islands in script tags that then get parsed by javascript, for example) which doesn't help solve our problem.

The goal with polyglot is to create a document that is both valid HTML5 and well-formed XML and where the DOM and CSS work in the same way in both (again this is possible mostly due to the inclusion of svg, mathml, and xlink in HTML5).

The only way to accomplish that is by discarding the extensibility of XML completely. AFAICT, polyglot markup is mainly a way to be able to serve the same document up both as application/xhtml+xml and as text/html. It also lets a developer tap into the XML toolchain, but regular XHTML5 would allow for that as well (but may not reliably be served as text/html).

Personally, I don't see the point in it.

XML serialisation of HTML isn't higher quality markup in any way as it doesn't catch any of the issues likely to cause problems in the long run and the HTML tools ecosystem is rich enough on its own.

XML's strictness can't tell you which alt attributes are legitimately empty and which aren't. It doesn't tell you which ARIA roles are misapplied and which aren't. It doesn't tell you if your metadata is correct, or if the position of the script tag in the markup tree is going to cause it to block rendering. It won't tell you whether the outline of H1-5 tags will make sense in a screen reader or whether the structure of your forms will have issues—whether the data submitted will make sense to the server. It won't tell you whether you've overloaded your ids, causing problems when various people in your team separately work on the links, ARIA support, javascript, CSS, and microdata in your site (they are all bound tightly to ids, too tightly). It won't tell you whether you're using spans and divs as buttons instead of anchors and real buttons. These are all markup quality issues that cause problems either now or later. XML's strictness doesn't help us at all in any significant way.

Whether any given piece of markup is good and of high quality or bad and of low quality is completely orthogonal to whether it is HTML or XML.

XML's strictness in web development only helps you fix trivial issues like typos and unclosed tags. In HTML, those sort of errors are either non-issues (i.e. the markup is still works predictably and reliably across all major browsers) or they break the page in an obvious way and get fixed quickly (just like in XML). Moreover, XML's strictness is only really beneficial when you're hand-coding and if you're doing that you've already lost. You hand-code the templates and test the hell out of them but your actual pages should always be generated programatically. HTML and XML is just too complicated and messy to write by hand on a regular basis. Doing so is slow and invites serious breakage. And, of course, end users, writers, authors should never ever have to see or worry about HTML or XML in any way.

XML's strictness does not address any of the real issues web developers are facing (especially for those working in teams, a context that brings to light a host of problems with our current crop of web tech).

The only real advantage XML has over HTML is extensibility and polyglot markup jettisons that.

Of course, XML's extensibility brings with it a level of complexity that compounds that of HTML, which is already too complex. This complexity (namespaces and strictness) cause problems for web developers that more than outweigh the benefits.

But, I digress… None of this has anything to do with epub-type/epub:type and Epub Zero. :-D

- best
- baldur

[Bill McCoy]

Thanks Baldur for the further details, it is a bit messy but in today's EPUB 3.0 we only have XHTML to worry about so it seems that with your correction that all current browsers including IE10 treat XHTML the same way have a unified way to do this right now for epub:type as it is, it's not quite as bad a situation as it sounded.

It seems an interesting question whether the EPUB Zero thought experiment is intending to define both a new XHTML serialization or only an HTML serialization. epub:type exists and is already supported in shipping EPUB 3 reading systems (e.g. popup footnotes in iBooks) so it seems that real-world style sheets targeting XHTML would in any case likely end up aiming to handle that.

> Most people and authoring tools would simply skip this and 

> just use a class or a data-* attribute and select on that.

This seems true whether or not the necessary selector is simple. Of course we see lots of authoring tools and hand-coded stylesheets today that give every e.g. <a> element the same class attribute and style based on that even though it's redundant.

> The only real advantage XML has over HTML is extensibility and polyglot markup jettisons that... 

You missed what I see as the key advantage of XML, especially for publishing workflows operating at scale: simplified parsing everywhere, including stream-oriented processing. Finally in HTML5 we have for the first time a well-defined parsing algorithm for "tag soup" but, besides being very complicated, it's only suited to a DOM-building parser. On any modern server or client environment whether it be PHP, Python, Ruby, Java, C#, C++, node.js... (you name it)... XML processing is available out of the box, and a few lines of code can process XML in either DOM or streaming manner. And you also have query/script processing with  XSLT and XQuery. This is not true of HTML. And AFAIK translators for HTML to XHTML are only available on a couple of platforms. This advantage doesn't IMO necessarily outweigh the compelling argument that the portable document format for the Open Web Platform should be able to handle the prevalent markup language for that platform, which is for sure HTML not XHTML, but it does seem like a concrete advantage that should be acknowledged and the implications of publications with content that doesn't have these advantages be fully thought through. 

For example, given the problem, "find all occurrences of the word 'sympathy' in a document and return a reliable (in the face of non-significant manipulations of e.g. white space) data structure that denotes their locations in the content", and one assigned the job of doing this on a randomly chosen programming environment. I don't know about you but I would far rather have XHTML than XML for the first part, and I'm not even sure how in HTML one could define the equivalent of EPUB 3.0's Canonical Fragment Identifiers for the second part.

--Bill

[Hadrien Gardeur]

I just saw the following post yesterday (it's in French): http://www.la-grange.net/2013/04/30/notes-epub

I must say that this makes a lot of sense.

Since the EPUB 3 spec says:

The epub:type attribute is intended to be functionally equivalent to the W3C Role Attribute [Role], but with restrictions as specified in Vocabulary Association.

Why don't we just use role and avoid RDFa, custom attributes etc.  for EPUB NG ?

[Baldur Bjarnason]

Yes. Now that you say it, this is the obviously correct solution.

You can see the spec for the role attribute here: http://www.w3.org/TR/role-attribute/

- best
- baldur

[Hadrien Gardeur]

Cool, that's one more problem solved.

What else are we missing aside from metadata ? I think that's the only item left on our list.

[Dave Cramer]

Agreed, although @role will forever remind me of bad docbook tagging--para role="h1"

I'm assuming some HTML validators will complain about @role.

For metadata, can we live with meta in html head, as in my very first blog post? In index.html, this would be document metadata, in other files it would be section metadata.

Dave

[Baldur Bjarnason]

On 2 May 2013, at 16:51, Dave Cramer <dau...@gmail.com> wrote:

> Agreed, although @role will forever remind me of bad docbook tagging--para role="h1"
>
> I'm assuming some HTML validators will complain about @role.

Fewer than you think. We should get validator support for free as they role out proper ARIA support.

>
> For metadata, can we live with meta in html head, as in my very first blog post? In index.html, this would be document metadata, in other files it would be section metadata.
>
> Dave

I can live with that.

- best
- baldur

[Hadrien Gardeur]

For metadata, can we live with meta in html head, as in my very first blog post? In index.html, this would be document metadata, in other files it would be section metadata. 

Are we allowed to nest these elements together ? I've never seen nested meta elements.

We need parent/child relationships for metadata. 

[Alberto Pettarin]

I agree; I find very inefficient (and I strongly dislike) the "refines"
approach; nesting would be way better.

AlPe

[Markus Gylling]

Hi all,

I agree too re the use of @role. FYI, the original intent in EPUB3 was to use this too, but due to unclarities re the extensibility of the attribute value space, we were recommended by the HTML WG chairs to use our own attribute instead, at least until the extensibility questions have been settled. Robin Berjon has recently opened https://www.w3.org/Bugs/Public/show_bug.cgi?id=21412 for this issue, so it should be clarified soon. In other words I expect EPUB3 to migrate to @role in a not too distant future, as well. 

/markus

[Baldur Bjarnason]

On 2 May 2013, at 17:21, Markus Gylling <markus....@gmail.com> wrote:

> Hi all,
> I agree too re the use of @role. FYI, the original intent in EPUB3 was to use this too, but due to unclarities re the extensibility of the attribute value space, we were recommended by the HTML WG chairs to use our own attribute instead, at least until the extensibility questions have been settled. Robin Berjon has recently opened https://www.w3.org/Bugs/Public/show_bug.cgi?id=21412 for this issue, so it should be clarified soon. In other words I expect EPUB3 to migrate to @role in a not too distant future, as well.

This makes me happy :-)

- best
- baldur

[Daniel Glazman]

On 02/05/13 18:17, Alberto Pettarin wrote:

>> We need parent/child relationships for metadata.
>
> I agree; I find very inefficient (and I strongly dislike) the "refines"
> approach; nesting would be way better.

Inefficiency is not the problem. The refining mechanism is an ID/IDREF
mechanism and that requires to either prompt the user for an ID or
let the software determine one for the author and that's risky for
maintenance... The former should only happen when it's absolutely
necessary, the latter is suboptimal.

</Daniel>

[DBOOKMARKS]

Hi Bill,

E0 is only idea. But XML structure is very useful. I agree with your thoughts about XML. In my opinion, With a new regulatory element of many innovations, such as /> <epub:meta may become useful (For example, to alert the reader from the author. Like) in the XHTML file. Because, author is not javascript. This element should be developed. 

<head>

<epub:meta ev:event="2013-07-15" action="alert" text="Hello my reader" />

</head>

N. Erhan Uzumcu

@dbookmarks

from Turkey

Our project: http://code.google.com/p/teyid/