Question on video targeting in HTML5

[James Smith]

I'm making much progress on our streaming video annotation client, but I'm running into a problem that I'm not sure how to resolve.

I'm working with HTML5 <video/> right now because that seems to be the easiest to work with. However, I'm not sure which URL is the target of the annotation. For example, I have the following in my test page:

<video preload="metadata" controls>
<source src="http://html5demos.com/assets/dizzy.mp4" type="video/mp4" />
<source src="http://html5demos.com/assets/dizzy.webm" type="video/webm" />
<source src="http://html5demos.com/assets/dizzy.ogv" type="video/ogv" />
</video>

The browser selects the file format it's most comfortable with. The implication/expectation/assumption is that all three are equivalent. Which one am I targeting? Which URL is the value of the oa:hasSource property?

In this example, the only difference between the sources is the file extension, but there's nothing in the HTML5 spec that requires that. They aren't relying on HTTP content negotiation. The sources could be from three different services. The assumption is that they represent different formats with the same semantic content, even if the bit streams differ. Annotations on one should work against the others.

If we select one as the source and the others as alternatives, how do we choose which one to make the canonical source? How does this affect aggregating annotations?

I'll keep thinking about this, but I have to spend most of my time finishing up the code. For now, I'm making the target URL/URI a configuration option of the annotation application, so I've pushed any code-related decision off to the future.

-- Jim

[Bruce D'Arcus]

In general, in RDF you'd typically target the extension-less URI
("http://html5demos.com/assets/dizzy") as the more generic/conceptual
document you're targeting, and the URIs with extensions as formats of
that more generic document.

Does that help in this case?

[James Smith]

On Apr 29, 2012, at 2:56 PM, "Bruce D'Arcus" <bda...@gmail.com> wrote:

> In general, in RDF you'd typically target the extension-less URI
> ("http://html5demos.com/assets/dizzy") as the more generic/conceptual
> document you're targeting, and the URIs with extensions as formats of
> that more generic document.
>
> Does that help in this case?
>
> On Sat, Apr 28, 2012 at 10:24 AM, James Smith <jgs...@gmail.com> wrote:
>> I'm making much progress on our streaming video annotation client, but I'm running into a problem that I'm not sure how to resolve.
>>
>> I'm working with HTML5 <video/> right now because that seems to be the easiest to work with. However, I'm not sure which URL is the target of the annotation. For example, I have the following in my test page:
>>
>> <video preload="metadata" controls>
>> <source src="http://html5demos.com/assets/dizzy.mp4" type="video/mp4" />
>> <source src="http://html5demos.com/assets/dizzy.webm" type="video/webm" />
>> <source src="http://html5demos.com/assets/dizzy.ogv" type="video/ogv" />
>> </video>
>>

It only helps when the <source/> elements all differ only in their extension and type. HTML 5 doesn't require that to be the case. The more general situation would be the following:

<video>
<source src="http://www.example.com/stream1.mp4" type="video/mp4" />
<source src="http://www.example.com/b/stream2.webm" type="video/webm" />
<source src="http://foo.example.com/c/stream3.ogv" type="video/ogv" />
</video>

If we attach an annotation client to this, which stream are we annotating? If we are only annotating one of them (likely the one the client selects to show), then what do we do for another client that selects a different stream? The grouping of the <source/> elements within the <video/> can be seen as an assertion on the author's part that all three are equivalent but in different formats.

There's also a problem with the video being shown in different resolutions without changing the conceptual target. Annotations targeting the medium resolution version of a video should just as well target the high resolution video when the only difference is the resolution selected by the user watching the video.

-- Jim

[Robert Sanderson]

Can you use multiple targets, one for each stream?

For example, if the streams were out of sync you'd need to have
different Selectors to select the correct time range.

Rob

[James Smith]

On Apr 29, 2012, at 3:36 PM, Robert Sanderson <azar...@gmail.com> wrote:

> Can you use multiple targets, one for each stream?
>
> For example, if the streams were out of sync you'd need to have
> different Selectors to select the correct time range.
>

I think I can do that. Multiple targets seem reasonable. How do the semantics of multiple targets work when the client should only select one target? My impression has been that multiple targets are considered multiple parts that aren't alternatives. For example, if I target two areas in an image, it's not an either/or, but a both/and. Will tools expect all three videos to be available (assuming I'm targeting three different video formats of the same content)?

I suspect that if the code I'm writing is being used with HTML5 <video/> tags, then the <source/> list will be relatively stable for a given video shown by a given website. I think for now I'll assume that the <source/> list is stable and worry about the alternative later.

The only problem that comes to mind if the videos aren't in sync is that I can't know if they are or not. I have to assume that they are. The video player only loads one of them. That's only for creating annotations. For showing annotations that are already created, it's not an issue (we just match up the right selector for the time range with the video being played, assuming we can discover that through the <video/> element).

I'll run with this during the week and see how far I get.

-- Jim

[Leyla Jael García Castro]

Hi James, 

As Robert suggests, I would also use multiple targets.

I am not quite familiar with differences in video formats but... When you say "The implication/expectation/assumption is that all three are equivalent", does it mean that one annotation on one format also applies (without any changes) to a different format? If the answer is yes, why do not add multiple sources? or, in case you want to know what format was used when the annotation was added, cannot you add owl:sameAs among the different formats?

Cheers, 

Leyla

[Robert Sanderson]

If you only want a single target, then the other option seems to be
multiple annotations with the same Body?

You could annotate an ORE aggregation of the resources, but that seems
even less likely to be supported.

If you were in a Shared Canvas environment, you could define a Canvas
and then annotate each of the videos on to it, and then annotate the
Canvas with the "real" annotation. (But that seems overkill, and also
unlikely to be broadly supported)

Rob

On Sun, Apr 29, 2012 at 1:50 PM, James Smith <jgs...@gmail.com> wrote:
> On Apr 29, 2012, at 3:36 PM, Robert Sanderson <azar...@gmail.com> wrote:
>> Can you use multiple targets, one for each stream?