[RFC] WebM temporal metadata
James Zern
I want to start a preliminary discussion on temporal metadata to get
some sign off before we begin talking about specific types that we'd
like to formalize.
I propose that we use webvtt to contain the metadata [1]. This has
some benefits:
- we've added support for embedding other kinds of webvtt files in
webm [2]; so this should map similarly
- this allows the same formatting to be reused out of band via the
text track with a metadata kind
WebVTT metadata text isn't given many formatting constraints, but I
think for our purposes name=value pairs should work, e.g.:
1
00:00:15.000 --> 00:00:17.950
METADATA1=METADATA1_PAYLOAD
METADATA2=METADATA2_PAYLOAD
Open questions for this method:
Should all the metadata for a file be in one track?
Or should separate metadata be in separate tracks?
The latter may make be easier for editing purposes.
Thoughts?
[1] http://dev.w3.org/html5/webvtt/#webvtt-metadata-text
[2] http://www.webmproject.org/code/specs/container/#webvtt-guidelines
Silvia Pfeiffer
Is there a reason that we need to constrain the metadata format? What
is the use case?
BTW: I agree that we should use WebVTT to transport timed metadata and
the kind=metadata. I just don't understand why we need to restrict the
format in which metadata is provided.
Thanks,
Silvia.
> --
> You received this message because you are subscribed to the Google Groups "WebM Discussion" group.
> To post to this group, send email to webm-d...@webmproject.org.
> To unsubscribe from this group, send email to webm-discuss...@webmproject.org.
> For more options, visit this group at http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
>
James Zern
> Hi James,
>
> Is there a reason that we need to constrain the metadata format? What
> is the use case?
>
format of the (mixed) metadata payload? Unless there's an indicator at
the file level, which would make separate webvtt files a necessity.
The use case we've been working with is GPS data, how to format that
will be an open question after this is settled.
Silvia Pfeiffer
> On Thu, Apr 5, 2012 at 00:51, Silvia Pfeiffer <silviap...@gmail.com> wrote:
>> Hi James,
>>
>> Is there a reason that we need to constrain the metadata format? What
>> is the use case?
>>
> Without something like name=value pairs how are we to determine the
> format of the (mixed) metadata payload? Unless there's an indicator at
> the file level, which would make separate webvtt files a necessity.
> The use case we've been working with is GPS data, how to format that
> will be an open question after this is settled.
I suppose if somebody wants to make use of the metadata track
possibilities, we will require two things:
1. an indication what format a parser will need to expect to be able
to parse out information from the cues
2. an indication of what data is actually included in the track and
therefore what applications will be able to make any sense of the
included data.
These are also issues that WebVTT should solve, so it's not just a
matter of WebM. I would expect to solve this through adding fields to
the file-wide metadata that indication what 1 and 2 are. That means
we really have to get file-wide metadata into the WebVTT spec, see
this bug [1].
Let's hypothetically assume we are using file-wide metadata as
described in [2] and encapsulate it as described in [2] in the
CodecPrivate sub-element of the Track element. I think what we could
for example do to include 1 and 2:
DataFormat = { NameValue, JSON, XML, ...}
DataHint = GPS data
The first field is from a set list of values, the second one is free
text string that a parser could read to decide whether it is able to
interpret the data.
In this way it would be possible to specify as many metadata tracks as
one needs and use one of a fixed set of formats that parsers typically
are able to cope with.
Would something like this work for you?
Cheers,
Silvia.
[1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=15851
[2] http://wiki.webmproject.org/webm-metadata/temporal-metadata/webvtt-in-webm
James Zern
> On Fri, Apr 6, 2012 at 3:56 AM, James Zern <jz...@google.com> wrote:
>> On Thu, Apr 5, 2012 at 00:51, Silvia Pfeiffer <silviap...@gmail.com> wrote:
>>> Hi James,
>>>
>>> Is there a reason that we need to constrain the metadata format? What
>>> is the use case?
>>>
>> Without something like name=value pairs how are we to determine the
>> format of the (mixed) metadata payload? Unless there's an indicator at
>> the file level, which would make separate webvtt files a necessity.
>> The use case we've been working with is GPS data, how to format that
>> will be an open question after this is settled.
>
> I suppose if somebody wants to make use of the metadata track
> possibilities, we will require two things:
>
> 1. an indication what format a parser will need to expect to be able
> to parse out information from the cues
> 2. an indication of what data is actually included in the track and
> therefore what applications will be able to make any sense of the
> included data.
>
> These are also issues that WebVTT should solve, so it's not just a
> matter of WebM.
Right, which is why I thought it would be beneficial to start with
WebVTT and then translate that into an in-band webm track.
> I would expect to solve this through adding fields to
> the file-wide metadata that indication what 1 and 2 are. That means
> we really have to get file-wide metadata into the WebVTT spec, see
> this bug [1].
>
> Let's hypothetically assume we are using file-wide metadata as
> described in [2] and encapsulate it as described in [2] in the
> CodecPrivate sub-element of the Track element. I think what we could
> for example do to include 1 and 2:
>
> DataFormat = { NameValue, JSON, XML, ...}
> DataHint = GPS data
>
> The first field is from a set list of values, the second one is free
> text string that a parser could read to decide whether it is able to
> interpret the data.
>
> In this way it would be possible to specify as many metadata tracks as
> one needs and use one of a fixed set of formats that parsers typically
> are able to cope with.
>
> Would something like this work for you?
>
Yes I think so. It's conceptually simpler to have separate
tracks/files for each metadata subtype, so the translation should be
simple enough.
Silvia Pfeiffer
Cool. Let's work on that basis then.
How do you think an example WebVTT file for your requirements would look like?
Cheers,
Silvia.
James Zern
WEBVTT
DataFormat: GeoJSON
DataHint: GPS data
00:00.000 --> 00:01.000
{
"type":"Feature",
"id":"OpenLayers.Feature.Vector_314",
"properties":{},
"geometry":{ "type":"Point", "coordinates":[97.03125, 39.7265625] },
"crs":{
"type":"OGC",
"properties":{ "urn":"urn:ogc:def:crs:OGC:1.3:CRS84" }
}
}
Other formats could be used as the payload: NMEA, GPX, KML, though it
might be useful to translate any comments (if the format supports
them) to a cue identifier.