Links to external resources in EPUB 3

[Bill Kasdorf]

I’m working with two pretty key interest groups who are bumping up against our current limitation of linking externally (online, out of the container) only to audio and video. We set those limitations because (1) we were conscious of the need for streaming audio and video (it being impractical to embed many audio and video files in the EPUB itself) and (2) we were (and are) concerned about exposing what should be a safe and self-contained EPUB to potentially unsecure external resources. (We also have limits on scripts for the latter reason especially.)

 

But here’s what I’m encountering:

 

MAGAZINE PUBLISHERS have “media resources” that they need or want to provide externally to the EPUB, and which include but are not limited to audio and video. There are two fundamental examples: (1) slideshows or image galleries that are “too big” for inclusion but which are image files, not audio and video; and (2) media resources that need to be dynamic, that change from time to time.

 

TEXTBOOK PUBLISHERS have the many of the same concerns as magazine publishers in this regard. But in addition, it is VERY common for a textbook to link to other ancillary resources, often through an LMS (Learning Management System) or CMS (Course Management System, not Content Management System). These can be many types of files, even things like PDFs, spreadsheets, etc. Of course in a print book effort is expected to get to those other resources, but in an electronic book a user expects to be able to link to them (understanding that you’d need to be online, not offline, to do so, just as with the streaming media).

 

And of course there are many other kinds of publishers who share these same issues as well (trade, scholarly, scientific—I could go on and on). They’re just coming to the fore for me with these two.

 

The fact that we currently don’t enable linking to these kinds of external resources is going to be a serious problem for these folks. I know it’s late in the game, but I really think we should reconsider that limitation to streaming audio and video only.

 

--Bill Kasdorf

 

[Keith Fahlgren]

Hi Bill,

On Wed, Jul 27, 2011 at 2:40 PM, Bill Kasdorf
<bkas...@apexcovantage.com> wrote:
> But in addition, it is VERY common for a textbook
> to link to other ancillary resources, often through an LMS (Learning
> Management System) or CMS (Course Management System, not Content Management
> System).

Would you clarify which requirements these publications have when
referencing these resources beyond what's
provided by a (lowly) link in the content itself?

<p>For more on ant colonies, see <a
href="http://learning-management-system/ant-colonies.m4v">this video
(external link)</a>.</p>

<p>To get a better sense of how boring these can be, see <a
href="http://course-management-system/q4results.ppt">this spreadsheet
(external link)</a>.</p>

Thanks,
Keith

[Brady Duga]

+1 to Keith's question. You are always allowed to link to external resources, they just aren't considered part of the publication and may not be available on all Reading Systems.

--

Brady Duga | Software Engineer | du...@google.com | (760) 462-5972

[Matt Garrish]

The “third-party resources” aspect of the publication resource definition may be to blame here (I say, being the one who added that para). The idea was that the content in the container is first party as it’s required for direct rendering/referencing and you know it belongs to the current EPUB, and everything linked to externally can be lumped under the label of a third party resource since there’s no reliable way to make statements about ownership once you leave the container (and no need to as those resources aren’t required for rendering).

 

In hindsight, it’s probably needlessly confusing and I can see how it could get read as all my things I link to have to be in the container.

 

But an <a> tag that references a resource in the container using a relative IRI requires that resource to be listed in the manifest and included in the container, which also makes “outbound hyperlinks” a bit vague. Maybe it would be better to rewrite the paragraph in question along the lines of:

 

Examples of resources that are not Publication Resources include those identified by the Package Document link element and those identified in outbound hyperlinks that resolve outside the container (e.g., identified in [HTML5] a element href attributes).

 

Thoughts welcome, though.

 

Matt

 

---

[Levantovsky, Vladimir]

I also heard similar concerns, and the specific subject of using remote fonts as another use case for external links has been brought up by Makoto a while ago. The issue here (at least, one of them, as I see it) is that access to some of the resources is protected and requires a mechanism to identify where the request is coming from. Sometimes it is accomplished by referrer checking on a server side, other times it is done by a browser (e.g. same-origin restriction).

 

In general, for the external resources to be available and reliable, a request has to be provided with a proper HTTP Referrer header. I have no idea what this header would be if externally linked content is requested while viewing EPUB content (is it “file”, “localhost” or something else?), but if we want to allow this to happen, and I assume we do and should allow it – we also need to specify what the referrer header should be. I’ve spent some time thinking about it and the “EPUB Unique ID” seem to be a good candidate for it – it will uniquely identify the origin of a request, and would allow server-side access control to a protected external content. There may be other things we need to do to address all possible use cases, and I think the question raised by Bill is both timely (better late than never) and very appropriate.

 

Thank you,

Vladimir

[Leonard Rosenthol]

"Remote" fonts are a problem for numerous reasons and I would strongly recommend that you not attempt to address it.  If you do, you will really need to involve folks such as the CSS-fonts committee as well as various font vendors.

Adobe and the ISO PDF committees have looked into supporting explicitly referenced external fonts (as opposed to simple "reference by name") at various times over the last decade and each time there has been one or more problems (usually the same ones) that keep cropping up.  These problems are not just technical but also legal/licensing related.

Historically, the folks pushing on the issue have been the Japanese and then later joined by China and Korea due to the large size of fonts that represent those glyphs.  While PDF offers "subsetting" (and you can do a limited form of this with WOFF), it was still felt that for a complex document the size, in some cases was a problem.  And that assumes that the font foundry even allows you to embed the font.  There are still many non-roman fonts that are tagged as non-embeddable by their foundry.   This is, I suspect, the problem that Makoto presented.

The biggest problem that has kept this particular problem from being solved is the inability to "uniquely identify a font".   As font vendors update their fonts based on glyph additions and/or changes (new monetary symbols, for example) or metric improvements, it is important that the EXACT version of the font be used that the person authored with – since otherwise the viewer won't see the same thing.  Various solutions for unique identification have been proposed (CRC, cryptographic hash, naming conventions, etc.) but they've all had one problem or another.   So this problem would need to be addressed – but in a non-EPUB-centric way – hence the need to bring others in on the problem.  (obviously this is more of a problem for a static publication such as an EPUB or PDF than for a dynamic medium such as the web itself).

More recently, there has been a desire to not embed "known fonts" - but known being determined by the use case.  For example, consider a phone company that issues monthly bills to their customers and needs to store them away.  They all use the same font(s) and as long as the font itself is archived in a known location and all material is under the company's control.  This is a very special case and while it could potentially be solved, there are various reasons that it hasn't been.  This one is less problematic since the company has complete control of the document, BUT it has that underlying nature.  If these documents needed to be distributed and/or archived outside the controlled environment, then things break down.    I also don't see this being relevant to EPUB (at this time, at least) - but I raise it for completeness.

Hope this bit of history and context is helpful in the discussion of remote fonts and why it's a bad idea.

Leonard

[Levantovsky, Vladimir]

Leonard,

 

While many of the reasons you mentioned have historically hampered the adoption of remote (or even embedded) fonts in various document formats, the most recent advancements in font technologies, publication of the OpenType font format as the ISO standard, the adoption of WOFF as a single interoperable webfont format, and wide-spread support for CSS3 @font-face in all major browsers have dramatically changed the overall landscape.

 

I believe that addressing the remote fonts today is both doable and appropriate – this is a problem that effectively has already been solved on the web, and will be IMO much easier to address in EPUB. The same font file that can be encapsulated in local EPUB content can be delivered online from a remote location. It’s 100% under publisher’s control, and the fonts can be uniquely identified by file name and location so there is no risk of getting a “wrong” font (although there is a risk of not getting the font when you need it). Virtually all existing web font services offer benefits that include CDN and subsetting, so that the fonts can be delivered fast, and from the source that is highly reliable. And, if a font is available as a webfont via a web font service, content authors need not be concerned with legal/licensing issues since those have been addressed by a web service provider.

 

Regards,

[Garth Conboy]

I'm clearly in the "EPUB 3 (by definition packaged in an OCF) wants to be a complete publication" camp, so don't get me started!  :-)

But, I think we should table (in the American use of the term) this issue for now -- it was explicitly discussed and decided by the working group to disallow remote fonts in EPUB 3.

This decision could be reconsidered for future versions, but we're set for our initial release.

Best,

  Garth

[Leonard Rosenthol]

I know Garth wants to table the discussion, but at least for academic purposes it can't hurt to continue this part of it…

I don't see how the standardization of OTF or WOFF has any bearing on the issue.  Those are just formats – they change nothing.

@font-face (or really the specification of a standard way to refer to URI-targetted font files) has certainly provided a common way to refer to the data.  HOWEVER, it doesn't change the issue.  What is on the other side of the URL is NOT versioned, controlled, etc.  If the person hosting the font data replaces v1 with v2, then the result on the documents that refer to it is UNKNOWN.  In a live content like the web, that's not a problem since the referring page can easily be updated/fixed.  But with a document format (eg. EPUB), you have NO SUCH CONTROL.  You need to be able to determine that the font is the same font that you originally chose.   So the problem of unique identification remains.

Web font services (such as TypeKit, Extensis, etc.) are a completely different (and more complex) problem for both identification as well as licensing.  Yes, again, they work well with the web but documents/publications are a completely different story.  You can't have a document rely on a service for rendering – too many possibilities for failure…

[Bill McCoy]

I'll second Garth's point but want to give some additional background and say a bit more about potential next steps.

One reason we tabled explicitly allowing external font resources to be considered "part of the publication" (which is really all we are talking about here, as Brady pointed out) was the realization that we need to solve a bigger issue: networked publications, i.e. publications that are packaged (partially or completely) virtually, not physically into a single ZIP file. This is important for a lot of workflows and in an increasingly online browser-based world a monolithic single-file representation will arguably be less and less useful. But we couldn't on EPUB 3 schedule solve this, given a number of issues including security in browser-based implementations, so also deferred it.

Multimedia resources clearly needed to be able to be referenced externally, mostly due to size but also in part due to our inability to agree on a standard video format. And in theory a publication should still work w/out these resources (and for accessibility reasons, it had better). But w/out the right fonts some publications might be totally unreadable, so there was hesitancy at going down that path and as Leonard pointed out there are some subtle issues about versioning/identification of external resources. But I don't believe these versioning/identification issues are unique to fonts so I look forward to seeing us solve the more generally to support the networked publication use case. EPUB's overall value as a representation of Web content that's structured, semantically enhanced, and able to be reliably used for interchange & delivery transcends whether or not it's zipped into a single file.

And, as we are going to develop EPUB in a more modular manner work on networked publications can potentially get started rather quickly if there's interest in pursuing it. Garth mentioned "future versions" but of course really we should be thinking in terms of "future extensions" wherever possible.  

--Bill

--

Bill McCoy

Executive Director

International Digital Publishing Forum (IDPF)
bmc...@idpf.org

+1 (206) 353-0233

[Leonard Rosenthol]

Maybe I am missing something here, but to me the MAIN USE of EPUB is that it is "a single file containing web content….."

If it's not a single file, then what makes it different form a web page?  Why would someone use it in that case?

Thanks in advance for the explanations.

Leonard

[Bill Kasdorf]

I just want to point out that my original issue had nothing to do with remote fonts. I agree with Garth that that was thoroughly discussed in the WG and we should leave it alone for now.

 

As for my original issue—that publishers such as magazine and textbook publishers need to be able to provide external links to things other than streaming audio and video—based on the following from the spec at http://idpf.org/epub/30/spec/epub30-publications.html#sec-resource-locations:

 

Publication Resource Locations

All Publication Resources must be located in the EPUB Container, with the following exceptions:

·         › Audio resources may be located in the Container or remotely.

·         › Video resources may be located in the Container or remotely.

Authors should prefer locating audio and video resources in the Container to allow the user access to the entire presentation regardless of connectivity status.

 

Perhaps we are just dealing with a semantic issue that some editing can clarify, prompted by the first two replies to my original e-mail, Keith’s and Matt’s. The problem may come down to clarifying what is meant by a “publication resource.” (We may be clear on that elsewhere and simply need to be clearer in the spot I just cited.) But I can say pretty confidently that the magazine folks I’m referring to will consider the slideshow or image gallery a “publication resource.” It is not just a related thing of interest; it is often created expressly and only for THAT publication, and the digital publication is not complete without it. That’s less clear for the textbook cases (per Keith’s examples)—those things he’s linking to can probably reasonably be considered related and supplementary but not really “part” of the publication.

 

FYI this is a huge debate in scholarly and scientific publishing right now. Those folks are dealing with what has in the past been considered “supplementary content”—spreadsheets, powerpoints, datasets, all kinds of stuff—that is increasingly (some of it, not all of it) becoming considered “part of the publication” (in those cases mostly journal articles).

 

--Bill Kasdorf

[Brady Duga]

I largely agree with these comments. I would like to approach networked publications as a whole, instead of glomming external links on top of OCF. It may well be that OCF is *not* a reasonable packaging format for publications with remote resources. Before we start tossing in exceptions willy-nilly, lets figure out if there is a better design.

[Bill McCoy]

This is of course debatable and right now the single-file nature still is certainly the most prominent "feature" of EPUB. But to me the fact that it's structured, semantically-enhanced, and navigable web content is what's more fundamentally distinct and important about EPUB.  After all there's lots of ways to put HTML into single-file containers but a random collection of Web pages packaged thusly would not necessarily have any defined reading order, would not be something that could be ingested in an interchange scenario, would not necessarily be able to be reliably manipulated in downstream processing workflows, etc.

One might even go so far as to say (exaggerating a bit) that W3C has thrown Semantic Web and documents under the bus to concentrate on Web applications and that IDPF has picked it up and is running with it to meet the needs of the publishing industry. That's perhaps an overstatement but if you think of the IDPF's charter wrt EPUB as to do whatever it takes to make Web content useful in a publication context that has relatively little to do, at root, with single-file representation.

--Bill

[Peter Sorotokin]

Bill,

I think that EPUB3 solution is that a “landing page” that presents external resources should be created and a hyperlink to that location included in the document.

A see a case for treating other supplemental materials in a more integrated fashion, but that would require some experiments. I do not think that we can converge on an interoperable solution without going though them.

Peter

On 7/28/11 9:38 AM, "Bill Kasdorf" <bkas...@apexcovantage.com> wrote:

I just want to point out that my original issue had nothing to do with remote fonts. I agree with Garth that that was thoroughly discussed in the WG and we should leave it alone for now.
 
As for my original issue—that publishers such as magazine and textbook publishers need to be able to provide external links to things other than streaming audio and video—based on the following from the spec at http://idpf.org/epub/30/spec/epub30-publications.html#sec-resource-locations:
 
Publication Resource Locations

All Publication Resource <http://idpf.org/epub/30/spec/epub30-publications.html#gloss-publication-resource-cmt-or-foreign> s must be located in the EPUB Container <http://idpf.org/epub/30/spec/epub30-publications.html#gloss-container> , with the following exceptions:
·         › <http://idpf.org/epub/30/spec/epub30-publications.html#sec-resource-locations-audio>  Audio resources <http://idpf.org/epub/30/spec/epub30-publications.html#cmt-grp-audio>  may be located in the Container or remotely.
·         › <http://idpf.org/epub/30/spec/epub30-publications.html#sec-resource-locations-video>  Video resources may be located in the Container or remotely.

[Bill Kasdorf]

As chair of the nextPub/PRISM-to-EPUB mapping committee and the liaison between our EPUB 3 WG and that IDEAlliance WG, I can say that this could drive magazine publishers away from EPUB as a standard deliverable. . . . (Probably not textbook publishers; what we have can probably work for them.) Photo galleries and slideshows are probably every bit as common and important to magazines as video and audio (often moreso). I will of course try to make the case with them that they _can_ put those resources into the EPUB, but that doesn’t address the fact that those kinds of things are often dynamic, changing over time. So maybe what this comes down to is that EPUB will work for them only if they can make those things static and of a size they consider practical to incorporate in an EPUB. The danger is that they will just say “well EPUB turns out not to work so well for us after all.”

 

I will defer to Dianne Kennedy, who is really central to those efforts, to provide examples of what the magazine folks really need. (She may have already corresponded with Markus on that; he’s on vacation right now.) It could be that a single specific addition to the list in the spec that I cited in my previous e-mail would do the trick for those folks. It’s really those non-video image collections that are mainly at issue here.

 

--Bill Kasdorf

[Bill Kasdorf]

Excellent suggestion. A good case can be made for this, at least as an interim solution until EPUB addresses the issue through a modular supplemental spec. It won’t be entirely appealing to magazine publishers—they want their users to just “see” these kinds of things with a minimum of indirection—but this is certainly a reasonable alternative to the even simpler “just put it in the container” answer.

 

--Bill

[Levantovsky, Vladimir]

I agree that we already talked about remote fonts before and decided to postpone it. My intention was _not_ to bring it up again now - if you re-read my first post on this subject (http://groups.google.com/group/epub-working-group/msg/ae4e445e4d59bccc) you will see that I mentioned remote fonts only as an example of yet another use case for external links.

 

Disregard the very first sentence of my original email – the rest of the message was purely about external resources in general, and the need to have a well-formed request to make various types of content, including protected resources, available if needed. I believe this is something we need to resolve if you we want to see EPUB used by magazine publishers with external content that e.g. is only available on subscription basis.

[Peter Sorotokin]

Bill,

I am not sure what we can do about this in EPUB 3.0 short of delaying the standard. I agree that there may be business requirements there, but I do not see any obvious solution. I do not see “just” allowing image resources to be external as solving the problem, for example - but do I see it as a big burden on the implementers and end-users.

Count me as a skeptic on an urgent need for integrated updatable resources for magazines. Looking at the currently-employed solutions in this space, it seems that magazine publications lives are typically much shorter than books and the size does not seem to be of a particular concern. While I agree that integrated updatable photo galleries and such is an issue, but I do not see it as a “make it or break it” thing.

We need to gather the requirements, come up with ideas, test them and agree on an interoperable solution. This is a non-trivial amount of work. I would also argue that we need direct participation from interested publishers for magazine-friendly EPUB to be successful.

Peter

[Levantovsky, Vladimir]

I agree that more discussions on remote fonts are needed assuming that we will talk (and not shout) at each other.

 

As to why things different now comparing to what they used to be …

 

“I don't see how the standardization of OTF or WOFF has any bearing on the issue.  Those are just formats – they change nothing.”

-          OTF and WOFF standardization created the basis for interoperable solutions, clarified the IP positions and allowed all devices and applications to implement support for downloadable fonts;

 

“What is on the other side of the URL is NOT versioned, controlled, etc.  If the person hosting the font data replaces v1 with v2, then the result on the documents that refer to it is UNKNOWN”

-          CSS @font-face-src points to a particular file at a particular location. If I am a content owner who authored and hosted the content – I have *full* control of my resources, what was in the font file I hosted, and what goes into it if I decide to update it.

 

“You can't have a document rely on a service for rendering – too many possibilities for failure…”

-          If this is true than Google Docs and other cloud-based solutions are just a myth(?)

 

Don’t get me wrong – I am not arguing against having fonts embedded in EPUB, but I do believe that using remote fonts should be _allowed_.

 

Thank you,

Vlad

[Leonard Rosenthol]

I hope I am not shouting.  If it came across that way, sorry.

I would be nice if everyone properly implemented both OTF and WOFF, but "in the real world" there are still issues with some implementations only supporting TTF-based OTF files and not CFF-based ones.  Since both are equally part of the OTF standard, it doesn't provide for "interoperable solutions" - though I agree that it's a start!

Yes, I agree.  If you authored the epub itself AND also put the font file online – then you are right, it's no different than embedding it (assuming a connection). It's when the author and the distributor/hoster are not one and the same and thus you can't rely on that level of control of the resources.  Thus you need to have some solution that enables the UA/viewer to detect the inconsistency and "fallback" (whatever that means).

I don't think that Google Docs is a myth, just that what is hosted there are not "documents" in the traditional sense – and certainly not "documents of record" that can be used in a legal or mandated setting of any fashion.  That's why most/all of those services offer the ability to produce a traditional document version (be it PDF, Word, EPUB, etc.).   Although I believe that for some types of content (publications, for one) such distributed documents serve a useful purpose, there remain many (many) other types that can't work in that type of world.  And for those, font services are problematic.

[rka...@sympatico.ca]

Peter,

Why do you see allowing image resources to be external to be a big burden on implementers and end users? Why is it any bigger burden than it is on the Internet now?

BK

Sent via BlackBerry from T-Mobile

---

From: Peter Sorotokin <psor...@adobe.com>

Sender: epub-work...@googlegroups.com

Date: Thu, 28 Jul 2011 11:44:43 -0700

To: epub-work...@googlegroups.com<epub-work...@googlegroups.com>

ReplyTo: epub-work...@googlegroups.com

Cc: Dianne Kennedy<dken...@idealliance.org>

Subject: Re: Links to external resources in EPUB 3

Bill,

I am not sure what we can do about this in EPUB 3.0 short of delaying the standard. I agree that there may be business requirements there, but I do not see any obvious solution. I do not see “just” allowing image resources to be external as solving the problem, for example - but do I see it as a big burden on the implementers and end-users.

Count me as a skeptic on an urgent need for integrated updatable resources for magazines. Looking at the currently-employed solutions in this space, it seems that magazine publications lives are typically much shorter than books and the size does not seem to be of a particular concern. While I agree that integrated updatable photo galleries and such is an issue, but I do not see it as a “make it or break it” thing.

We need to gather the requirements, come up with ideas, test them and agree on an interoperable solution. This is a non-trivial amount of work. I would also argue that we need direct participation from interested publishers for magazine-friendly EPUB to be successful.

Peter


On 7/28/11 10:07 AM, "Bill Kasdorf" <bkas...@apexcovantage.com> wrote:

As chair of the nextPub/PRISM-to-EPUB mapping committee and the liaison between our EPUB 3 WG and that IDEAlliance WG, I can say that this could drive magazine publishers away from EPUB as a standard deliverable. . . . (Probably not textbook publishers; what we have can probably work for them.) Photo galleries and slideshows are probably every bit as common and important to magazines as video and audio (often moreso). I will of course try to make the case with them that they_can_ put those resources into the EPUB, but that doesn’t address the fact that those kinds of things are often dynamic, changing over time. So maybe what this comes down to is that EPUB will work for them only if they can make those things static and of a size they consider practical to incorporate in an EPUB. The danger is that they will just say “well EPUB turns out not to work so well for us after all.”

[Peter Sorotokin]

Implementers: One reason is that image size is an input to the layout engine which implies that layout engine should either run on a separate thread or suspend itself while image is being downloaded asynchronously. The same applies for rendering, but it is a bit easier to handle. Browser engines do that, of course, but that is a fairly complex thing to do. And even if you work on top of the browser engine, you have to worry about layout if you do pagination – progressive layout in scrollable mode is simpler and more intuitive to the user.

End users: people do have an expectation of repeatable experience with the document. If images that are part of the book layout start to change on you, it’s not really reliable anymore – and how you, as a user, would know which documents are “stable” and which are not? Video and audio were not there in the paper world, so people intuitively put them in the different category of things. Slideshows were not on paper either, of course, but allowing any image to be remote is too broad for expressing those.

Peter


On 7/28/11 4:36 PM, "rka...@sympatico.ca" <rka...@sympatico.ca> wrote:

Peter,Why do you see allowing image resources to be external to be a big burden on implementers and end users? Why is it any bigger burden than it is on the Internet now?BK

Sent via BlackBerry from T-Mobile

[Bill McCoy]

Re: end users, I don't disagree w/ Peter about consumer expectations but it may be worth thinking about "brand promise" of a format vs. inherent capabilities.

It is not widely known but PDF documents can perfectly well reference external resources including images (via filespec so generally resolved in context of whatever file system the PDF lives on, I'm not sure about URLs or what happens for web-deployed PDFs).  Last time I saw this demoed, Adobe Reader didn't provide any user notification that there was an external reference being resolved (perhaps Leonard could dig up a demo of this from his colleague Jim King).

So, PDFs despite the reputation are not necessarily self-contained (who knew?). But, because we all think PDFs are there is a disincentive for content creators to utilize such features, unless necessary. The point being that the presence of such a feature doesn't in practice confuse end users, because it's not used except where necessary so the general expectation of reliable display is met.

The flip side is everyone understands and expects Web page display to depend on retrieving resources from potentially a wide variety of places.

Where the expectation for EPUB will end up is not totally clear to me given the increasing prevalence of browser-based implementations but it could be argued from the PDF example that user expectations need not always constrain capabilities but rather be organically determined by the uses the community commonly makes of those capabilities.

The flip side though (and this goes more to the question of implementation effort) is that this capability in PDF has largely gone unused. So that's a waste of development and maintenance time and potentially (depending on how well such fringe features get tested)

a threat vector. Features aren't free.

--Bill

[Leonard Rosenthol]

The feature you refer to is a called "external streams" and, while a feature of PDF since 1.2, has been always been implemented as a preference (OFF BY DEFAULT) in Adobe's products.  It's disabled by default for three reasons: 1) it goes against the philosophy that PDFs are "self-contained, 2) it would enable the creation of a PDF that looked different each time you opened it and 3) it's a security hole to enable arbitrary data to be used.

I will also point out that it's features of PDF like these (ones that have security considerations and/or aren't used in the real world) that caused the original need for "PDF subsets" such as PDF/X, PDF/A, etc.  In addition, such features have been deprecated in the upcoming PDF 2.0 (ISO 32000-2).

Leonard  

[Bill Kasdorf]

Thanks, Dianne, this is just what I was hoping for from you.

 

Thanks for the “we’re not going to abandon EPUB” message! Obviously, we’re going to forge ahead with our mapping, use it to the extent possible, and supplement it as necessary. That’s always been the plan.

I think what I should have been clearer about is that _a lot of magazine publishers_ may be less likely to use EPUB (certainly not EPUB 3 that exactly conforms to the 3.0 spec) if they can’t reference image collections remotely. And we all know how it undermines use of a standard if it is perceived as working only some of the time, for only some content.

 

For that reason, I still want to pursue the discussion of a possible solution in the context of EPUB 3, and I am getting the sense that there are others who would like to see this happen as well.

 

One important perspective was Peter Sorotokin’s, who clarified the problems he envisioned this might create for reading systems, layout, etc. That leads me to a really fundamental question for you:

 

--If we could require, as part of the spec, that any such image gallery or slideshow would be implemented to occupy a certain well defined area (just as a video would play in a certain area)—whether that definition were in absolute or relative terms—would that work for most magazine publishers? My hope, obviously, is that we can define this in such a way that it is more workable from the reading system point of view, per Peter’s issues. And this might even be viewed as a plus by magazine publishers, since controlling the layout is actually an important concern of theirs. They wouldn’t _want_ to do this in a way that would create unpredictable results. So maybe we can figure out how to make sure it won’t.

 

Thanks again for your thoughtful response!

--Bill Kasdorf

 

From: Dianne Kennedy [mailto:dken...@idealliance.org]
Sent: Friday, July 29, 2011 10:38 AM
To: Bill Kasdorf; epub-work...@googlegroups.com
Cc: 'peter...@timeinc.com'; David Steinhardt
Subject: RE: Links to external resources in EPUB 3

 

Sorry for the lag in my response time. I’m in the midst of releasing 3 specs in other IDEAlliance domains and just could not get back to you sooner.

 

Ever since the release of the first magazines on the iPad, magazine publishers have made extensive use of externally referenced content.  Two issues seem to be driving this

1)     Including all images, audio and video in the package means an unacceptable increase in download time and in the footprint of the publication on a device.  This means not just audio and video but images as well.  As Bill indicated, photo galleries and slide shows are as common, perhaps more common, than audio and video and so are often linked to externally in today’s digital magazine implementations.

2)     Currency of the content is also an issue for us.  Publishers want to be able to dynamically change/update content externally to the publication.  And I believe this requirement will only increase over time.

 

I appreciate Peter’s point that “people do have an expectation of repeatable experience with the document [book],” but this really does not apply in the magazine world.

If you cannot include the ability to add images to the objects that can be linked to externally without holding up the spec, then that is what you have to do.  But this will fall short of what magazine publishers really want and need. We need images, in addition to audio and video as external links. 

 

I don’t agree with Bill that identifying areas where EPUB3 will not meet the needs of magazine publishers (such as not allowing for external images) will “drive magazine publishers away from EPUB.”  .  Our goal is, and will remain, recommending EPUB3 if possible.  Where we find differences based on the differences between magazine publishing and book publishing, nextPub will develop a set of restrictions and extensions in much the same way IDPF has done with their recommendation for the use of HTML5 in EPBUB3.

 

Currently the IDEAlliance nextPub Working Groups are working to both create a mapping from nextPub to EPUB3 and to model the media functionality our business requires.  When this work is complete in September we will be able to identify how EPUB-compliant magazines will work/function.  My guess is that for some magazines, EPUB will work well.  But, likely we will discover a number of areas where EPUB3 does not exactly meet the business requirements of other magazine publishers.  In these cases, we will define a number of EPUB3 extensions and restrictions that will enable us to deliver magazine content to the broader, tablet publishing channel where browser-based display technologies are employed.  Our goal is, and will remain, to stay as close to EPUB3 as possible.  It is our hope that our work to define magazine specific extensions can be used by IDPF as input for future versions of EPUB so that the two specs stay as synchronized as is possible.

 

We will be having a working meeting in NYC at Time, Inc. on September 14 to review the work of each of our working groups and formulate Version 1.0 of our nextPub specifications.  You are certainly invited to participate and contribute.  Please contact me directly if you wish to attend.

Dianne Kennedy                                                                                             
VP of Emerging Technologies 
IDEAlliance
Powering the TOTAL Media Chain
office: 630-941-8197
cell: 630-908-0770
fax: 630-941-8196
dken...@idealliance.org
www.idealliance.org

---

[Bill Kasdorf]

Bill and Leonard—

 

Thanks to both of you for these good clarifications.

 

I would point out that although this feature may not have been used in PDFs, that doesn’t mean it wouldn’t be _needed_ and used in EPUBs. And the fact that it may be deprecated in PDF 2.0 just underscores the difference between PDF and EPUB. PDF is ultimately all about page fidelity above all else; that is its raison d’être. That is not ultimately what EPUB is about; and in the environment we find ourselves now, what was unused in PDF may become essential in EPUB.

 

I can’t resist mentioning how much this discussion reminds me of a white paper I wrote many years ago, called “PDF and XML: Why We Need Both.” It led to a string of very enjoyable Seybold presentations over the years with none other than the aforementioned Jim King, one of my heroes! (I still use what I think of as one of his mantras all the time, and always credit him: it’s meaningless to talk in the abstract about XML, it is always necessary to say “XML _for what_?”) Please say hi to Jim for me.

 

--Bill Kasdorf

 

From: epub-work...@googlegroups.com [mailto:epub-work...@googlegroups.com] On Behalf Of Leonard Rosenthol

Sent: Thursday, July 28, 2011 8:41 PM
To: epub-work...@googlegroups.com
Cc: Dianne Kennedy

[Bill Kasdorf]

You just did—thanks! And again, just what I hoped you’d say.

--Bill

 

From: Dianne Kennedy [mailto:dken...@idealliance.org]
Sent: Friday, July 29, 2011 11:05 AM
To: Bill Kasdorf; epub-work...@googlegroups.com
Cc: 'peter...@timeinc.com'; David Steinhardt
Subject: RE: Links to external resources in EPUB 3

 

I think it is ok to say something to the effect of

External image references used in media objects such as gallery or slideshow are expected to display within the area defined by accompanying style sheet. 

 

If this would make a difference to the ePUB3 folks, please communicate this to them.

[Leonard Rosenthol]

> PDF is ultimately all about page fidelity above all else; that is its raison d’être. That is not ultimately what EPUB is about; 

>

I agree that page fidelity is certainly not what EPUB is about.  However, I would put forth that fidelity of content is a key part of EPUB – at least in terms of its use for books and documents.  These types of "publications" should not change over time, with respect to their content.  Sure, they may layout differently on various screens/devices – but (I believe) that users have an expectation that the text, images, etc. will be the same each time they open it up.

Magazines (or other types of periodicals), however, clearly have different goes in mind – mostly due to advertising that is featured in them.  In fact, I would speculate that the publishers themselves do not want the editorial content to change (and again, that their users expect unchanging editorials) - BUT the ability to enable dynamic advertisements (in all the various ways that could be portrayed) has merit (especially in terms of $$).

Perhaps we should be investigating a specific "profile" of EPUB for magazines – much as there also seems to be a desire for a "lighter-weight" EPUB that would suffice for manga/comics, etc.

Leonard

[Hadrien Gardeur]

Perhaps we should be investigating a specific "profile" of EPUB for magazines – much as there also seems to be a desire for a "lighter-weight" EPUB that would suffice for manga/comics, etc.

Yes, we need to be able to identify such publications, and ideally, we need to identify them using a different media type or media type parameter. 

[Peter Sorotokin]

One way to approach this would be to start with a hyperlink. If we create a new epub:type value for a hyperlink to the supplemental content than Reading Systems that want to process external content could instantiate it in-place, and non-supplemental content-aware Reading Systems (including EPUB2 Reading Systems) will display a clickable poster which will open supplemental content in the browser if clicked.

Here is an example:

<a href=”http://magazine.example.com/slideshow1” epub:type=”supplemental”>
<section>
   <h1>Slideshow1</h1>
   <p>Click here to play interactive slidehow</p>
   <img src=”poster.jpg”/>
  </section>
</section>
</a>

Reading Systems that cannot or don’t want to download external content would process it as if there is no epub:type=”supplemental”.

Reading Systems that process external content would act as an iframe with src=”http://magazine.example.com/slideshow1” is instantiated in the place that <a> element occupies.

Would that work?

Peter

On 7/29/11 8:13 AM, "Bill Kasdorf" <bkas...@apexcovantage.com> wrote:

You just did—thanks! And again, just what I hoped you’d say.
--Bill
 

From: Dianne Kennedy [mailto:dken...@idealliance.org]
Sent: Friday, July 29, 2011 11:05 AM
To: Bill Kasdorf; epub-work...@googlegroups.com
Cc: 'peter...@timeinc.com'; David Steinhardt
Subject: RE: Links to external resources in EPUB 3

I think it is ok to say something to the effect of
External image references used in media objects such as gallery or slideshow are expected to display within the area defined by accompanying style sheet.  
 
If this would make a difference to the ePUB3 folks, please communicate this to them.


Dianne Kennedy
VP of Emerging Technologies
IDEAlliance
Powering the TOTAL Media Chain
office: 630-941-8197
cell: 630-908-0770
fax: 630-941-8196

dken...@idealliance.org <mailto:dken...@idealliance.org>  
www.idealliance.org <http://www.idealliance.org>

---

dken...@idealliance.org <mailto:dken...@idealliance.org>  
www.idealliance.org <http://www.idealliance.org>

[Bill Kasdorf]

This looks promising, Peter—thanks. I will defer to Dianne Kennedy to answer more definitively.

 

--Bill Kasdorf

[Lee Passey]

On Thu, July 28, 2011 6:03 pm, Peter Sorotokin wrote:

> End users: people do have an expectation of repeatable experience with the
> document. If images that are part of the book layout start to change on you,
> it�s not really reliable anymore � and how you, as a user, would know which
> documents are �stable� and which are not? Video and audio were not there in
> the paper world, so people intuitively put them in the different category of
> things. Slideshows were not on paper either, of course, but allowing any image
> to be remote is too broad for expressing those.

Discussion of end user expectations seems a bit rare in these discussions, and
I am very glad to see Mr. Sorotokin bring it up here. So, for what it's worth,
here are some of my expectations for external resources.

I want complete access to all significant portions of a publication when I'm
not "in the cloud" (which I try to stay out of as much as possible). When I
buy an e-book in Newark, I want to be able to enjoy it all the way to London.
When I'm on the beach in Costa Rica I want to be able to select from the
library of beach reads I loaded up at home.

Links to resources in "the cloud", which augment the publication I have
purchased are welcomed and desired. A "for more information see:" link in a
footnote is a great idea. When I connect to "the cloud" I frequently like to
explore this additional information.

Fancy ads with graphics and video are mostly just annoying, but I can tolerate
them so long as downloading them doesn't count against my cell phone
provider's data limits. The only thing worse than annoying ads is having to
pay for the "privilege" of being subjected to them.

So, the principles I derive from these expectations are that external links
should /always/ be permitted, and /never/ be required. Any resource that is
required to make full use of the publication /must/ be included in the ePub
package. Downloading supplemental resources, whatever their nature (whether
ads or videos of scientific experiments) /must/ be under the control of the
end user; the failure to access these external resources should /never/
substantially affect the reading experience.

In that extremely rare situation where internet connectivity and access to
external resources are absolutely required an ePublication /must/ be clearly
identified as requiring active internet connectivity for use.

These constraints are obviously not enforceable by an automated process; the
best the IDPF could do would be to clearly document them as best practices,
and rely on industry to comply. However, if the industry cannot be relied upon
to allow me to control the content on my own hardware I would advocate for
removing support for external links altogether.