A metadata specification for WebM has been a long requested feature. We plan on addressing this in the near term, but wanted to have a general discussion before we propose a solution.
One requirement we have is the support for temporal metadata to store things like geolocation data, e.g., GPS coordinates. The other is the traditional global metadata, e.g., title, author, etc. Initially we were thinking to define a separate standard for both global and temporal, which would be stored in its own track. After a bit of discussion it came to feel as though global was merely a special case of the temporal, which we knew we wanted.
Consider some examples: i) A video clip with an audio soundtrack that changes sources through the clip. With one audio and one video track it would be simple enough to have an artist label that covered each or both. With multiple audio sources in one track you could similarly have an artists tag. This though begins to decouple the attribution for each segment. ii) Appending 2 or more unrelated videos. This is a similar problem. You get into an issue with having to merge metadata from each segment into one global blob. At that point which takes precedence as the 'artist' for the clip? iii) Live presentations. Once more the performer, etc. can change throughout the clip, like a shoutcast stream.
It is true that a global mapping could be constructed to handle the above cases, but it seems that it might be simpler to have a timed metadata track that could indicate the duration that the value applied to. Global or live metadata could simply have no duration to indicate it was for the entire clip or until a new value was encountered (for the live case).
Does anyone have any thoughts on this? Is there any reason we shouldn’t explore a single metadata solution using a timed track?
> A metadata specification for WebM has been a long requested feature. > We plan on addressing this in the near term, but wanted to have a > general discussion before we propose a solution.
> One requirement we have is the support for temporal metadata to store > things like geolocation data, e.g., GPS coordinates. The other is the > traditional global metadata, e.g., title, author, etc. > Initially we were thinking to define a separate standard for both > global and temporal, which would be stored in its own track. After a > bit of discussion it came to feel as though global was merely a > special case of the temporal, which we knew we wanted.
> Consider some examples: > i) A video clip with an audio soundtrack that changes sources through the clip. > With one audio and one video track it would be simple enough to have > an artist label that covered each or both. With multiple audio sources > in one track you could similarly have an artists tag. This though > begins to decouple the attribution for each segment. > ii) Appending 2 or more unrelated videos. > This is a similar problem. You get into an issue with having to merge > metadata from each segment into one global blob. At that point which > takes precedence as the 'artist' for the clip? > iii) Live presentations. > Once more the performer, etc. can change throughout the clip, like a > shoutcast stream.
> It is true that a global mapping could be constructed to handle the > above cases, but it seems that it might be simpler to have a timed > metadata track that could indicate the duration that the value applied > to. Global or live metadata could simply have no duration to indicate > it was for the entire clip or until a new value was encountered (for > the live case).
> Does anyone have any thoughts on this? Is there any reason we > shouldn’t explore a single metadata solution using a timed track?
(Note, I'm neither a codec nor a container format developer, just a geeky user, so take all of this with that knowledge first.)
I see the argument for the global metadata being a special case of the temporal, but in the global case, there is usually a high likelihood that the metadata is always in the same place (e.g., either at the beginning or the end of the file) which yields a much, much simpler method of extraction for client software.
In the temporal case, I suppose it could be constructed in such a way that, in the case that it doesn't real change, then it would effectively behave as the global case - that is, it would always be located in the same place, and remain easily extracted.
I would suggest something along the lines of temporal data being considered updates to existing metadata, and if a previous value is not marked as having been replaced by subsequent metadata, then it should persist as long as its associated tracks do.
Finally, and I'm sure this has already been considered, but what already exists within the Matroska world with regards to metadata? I would strongly urge that whatever is developed be made as compatible as possible with what is already out there.
<abu_huray...@hidayahonline.org> wrote: > On 01/18/2012 03:20 PM, James Zern wrote: >> A metadata specification for WebM has been a long requested feature. >> We plan on addressing this in the near term, but wanted to have a >> general discussion before we propose a solution.
>> One requirement we have is the support for temporal metadata to store >> things like geolocation data, e.g., GPS coordinates. The other is the >> traditional global metadata, e.g., title, author, etc. >> Initially we were thinking to define a separate standard for both >> global and temporal, which would be stored in its own track. After a >> bit of discussion it came to feel as though global was merely a >> special case of the temporal, which we knew we wanted.
>> Consider some examples: >> i) A video clip with an audio soundtrack that changes sources through the clip. >> With one audio and one video track it would be simple enough to have >> an artist label that covered each or both. With multiple audio sources >> in one track you could similarly have an artists tag. This though >> begins to decouple the attribution for each segment. >> ii) Appending 2 or more unrelated videos. >> This is a similar problem. You get into an issue with having to merge >> metadata from each segment into one global blob. At that point which >> takes precedence as the 'artist' for the clip? >> iii) Live presentations. >> Once more the performer, etc. can change throughout the clip, like a >> shoutcast stream.
>> It is true that a global mapping could be constructed to handle the >> above cases, but it seems that it might be simpler to have a timed >> metadata track that could indicate the duration that the value applied >> to. Global or live metadata could simply have no duration to indicate >> it was for the entire clip or until a new value was encountered (for >> the live case).
>> Does anyone have any thoughts on this? Is there any reason we >> shouldn’t explore a single metadata solution using a timed track?
> (Note, I'm neither a codec nor a container format developer, just a > geeky user, so take all of this with that knowledge first.)
> I see the argument for the global metadata being a special case of the > temporal, but in the global case, there is usually a high likelihood > that the metadata is always in the same place (e.g., either at the > beginning or the end of the file) which yields a much, much simpler > method of extraction for client software.
> In the temporal case, I suppose it could be constructed in such a way > that, in the case that it doesn't real change, then it would effectively > behave as the global case - that is, it would always be located in the > same place, and remain easily extracted.
> I would suggest something along the lines of temporal data being > considered updates to existing metadata, and if a previous value is not > marked as having been replaced by subsequent metadata, then it should > persist as long as its associated tracks do.
Semantically this sounds similar to special casing the global metadata. It sounds as though you're more concerned about where the base data lives in the file and how to extract that.
> Finally, and I'm sure this has already been considered, but what already > exists within the Matroska world with regards to metadata? I would > strongly urge that whatever is developed be made as compatible as > possible with what is already out there.
Matroska defines a hierarchical tag system [1], which allows tags to be attached to tracks or chapters. Note chapters are not part of the webm specification currently. I agree that using an existing system for which there are tools to manipulate the data is usually best, which is part of the reason for this thread. With tags, for instance though, I think we'd have to extend them to reference clusters or blocks. That or an excessive amount of chapters might be needed. Beyond this, will all tag systems, Matroska tags, XMP, etc. we will need to have a list of well-known entities as well as define new entities where we see a need. In this case some of the tools for extraction will need to be updated as well. Given we're talking about a web format, should we use these as a reference and provide a solution that fits better in this framework rather than worry about compatibility?
Talking about metadata and timed metadata: have you considered how this hooks into what is being done in HTML5?
There, we have defined WebVTT as a file format for external time-synchronized data (including captions, subtitles, descriptions, and - yes - general metadata).
In the past it has been stated that the idea for captions in WebM would be to use the solution that is being used in HTML5 and encapsulate it into WebM, i.e. encapsulate WebVTT into WebM. Since WebVTT is similar to SRT and there are existing specs for how to encapsulate SRT into Matroska, putting WebVTT into WebM should be possible in a similar manner.
I would be totally supportive of solving the timed metadata challenge with WebVTT and by creating a WebVTT encapsulation spec for WebM.
As for the non-timed metadata: Matroska also has a solution for header-style metadata, i.e. metadata such as id3 tags that need to be available to seach in apps such as iTunes. I would suggest building on that existing solution.
On Thu, Jan 19, 2012 at 11:39 AM, James Zern <jz...@google.com> wrote: > On Wed, Jan 18, 2012 at 12:31, Basil Mohamed Gohar > <abu_huray...@hidayahonline.org> wrote: >> On 01/18/2012 03:20 PM, James Zern wrote: >>> A metadata specification for WebM has been a long requested feature. >>> We plan on addressing this in the near term, but wanted to have a >>> general discussion before we propose a solution.
>>> One requirement we have is the support for temporal metadata to store >>> things like geolocation data, e.g., GPS coordinates. The other is the >>> traditional global metadata, e.g., title, author, etc. >>> Initially we were thinking to define a separate standard for both >>> global and temporal, which would be stored in its own track. After a >>> bit of discussion it came to feel as though global was merely a >>> special case of the temporal, which we knew we wanted.
>>> Consider some examples: >>> i) A video clip with an audio soundtrack that changes sources through the clip. >>> With one audio and one video track it would be simple enough to have >>> an artist label that covered each or both. With multiple audio sources >>> in one track you could similarly have an artists tag. This though >>> begins to decouple the attribution for each segment. >>> ii) Appending 2 or more unrelated videos. >>> This is a similar problem. You get into an issue with having to merge >>> metadata from each segment into one global blob. At that point which >>> takes precedence as the 'artist' for the clip? >>> iii) Live presentations. >>> Once more the performer, etc. can change throughout the clip, like a >>> shoutcast stream.
>>> It is true that a global mapping could be constructed to handle the >>> above cases, but it seems that it might be simpler to have a timed >>> metadata track that could indicate the duration that the value applied >>> to. Global or live metadata could simply have no duration to indicate >>> it was for the entire clip or until a new value was encountered (for >>> the live case).
>>> Does anyone have any thoughts on this? Is there any reason we >>> shouldn’t explore a single metadata solution using a timed track?
>> (Note, I'm neither a codec nor a container format developer, just a >> geeky user, so take all of this with that knowledge first.)
>> I see the argument for the global metadata being a special case of the >> temporal, but in the global case, there is usually a high likelihood >> that the metadata is always in the same place (e.g., either at the >> beginning or the end of the file) which yields a much, much simpler >> method of extraction for client software.
>> In the temporal case, I suppose it could be constructed in such a way >> that, in the case that it doesn't real change, then it would effectively >> behave as the global case - that is, it would always be located in the >> same place, and remain easily extracted.
>> I would suggest something along the lines of temporal data being >> considered updates to existing metadata, and if a previous value is not >> marked as having been replaced by subsequent metadata, then it should >> persist as long as its associated tracks do.
> Semantically this sounds similar to special casing the global > metadata. It sounds as though you're more concerned about where the > base data lives in the file and how to extract that.
>> Finally, and I'm sure this has already been considered, but what already >> exists within the Matroska world with regards to metadata? I would >> strongly urge that whatever is developed be made as compatible as >> possible with what is already out there.
> Matroska defines a hierarchical tag system [1], which allows tags to > be attached to tracks or chapters. Note chapters are not part of the > webm specification currently. > I agree that using an existing system for which there are tools to > manipulate the data is usually best, which is part of the reason for > this thread. With tags, for instance though, I think we'd have to > extend them to reference clusters or blocks. That or an excessive > amount of chapters might be needed. > Beyond this, will all tag systems, Matroska tags, XMP, etc. we will > need to have a list of well-known entities as well as define new > entities where we see a need. In this case some of the tools for > extraction will need to be updated as well. Given we're talking about > a web format, should we use these as a reference and provide a > solution that fits better in this framework rather than worry about > compatibility?
> -- > You received this message because you are subscribed to the Google Groups "WebM Discussion" group. > To post to this group, send email to webm-disc...@webmproject.org. > To unsubscribe from this group, send email to webm-discuss+unsubscr...@webmproject.org. > For more options, visit this group at http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
> I see the argument for the global metadata being a special case of the > temporal, but in the global case, there is usually a high likelihood > that the metadata is always in the same place (e.g., either at the > beginning or the end of the file) which yields a much, much simpler > method of extraction for client software.
I agree, if you are only looking at supporting global metadata vs temporal metadata. If you want to support both then having global metadata be a special case of temporal is most likely simpler because you only need to support one method of extraction/insertion vs two if they are use separate storing methods.
> In the temporal case, I suppose it could be constructed in such a way > that, in the case that it doesn't real change, then it would effectively > behave as the global case - that is, it would always be located in the > same place, and remain easily extracted.
Yeah, that is the idea. If you wanted some temporal metadata to behave like global metadata you would set the time from the start of the file to the duration of the file.
> I would suggest something along the lines of temporal data being > considered updates to existing metadata, and if a previous value is not > marked as having been replaced by subsequent metadata, then it should > persist as long as its associated tracks do.
One issue with this is that there will be temporal metadata that should only be valid during playback at certain times and not at setup time. The other issue is implementation wise we will still be using two different formats to store the data.
> Finally, and I'm sure this has already been considered, but what already > exists within the Matroska world with regards to metadata? I would > strongly urge that whatever is developed be made as compatible as > possible with what is already out there.
As James mentioned Tags would work for global metadata but not for temporal metadata. Silvia mentioned WebVTT, which I think would be a nice fit for temporal metadata. It is very close to SRT format which is already in use by Matroska for some subtitles today.
> -- > You received this message because you are subscribed to the Google Groups > "WebM Discussion" group. > To post to this group, send email to webm-disc...@webmproject.org. > To unsubscribe from this group, send email to > webm-discuss+unsubscr...@webmproject.org. > For more options, visit this group at > http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
> Talking about metadata and timed metadata: have you considered how > this hooks into what is being done in HTML5?
> There, we have defined WebVTT as a file format for external > time-synchronized data (including captions, subtitles, descriptions, > and - yes - general metadata).
> In the past it has been stated that the idea for captions in WebM > would be to use the solution that is being used in HTML5 and > encapsulate it into WebM, i.e. encapsulate WebVTT into WebM. Since > WebVTT is similar to SRT and there are existing specs for how to > encapsulate SRT into Matroska, putting WebVTT into WebM should be > possible in a similar manner.
> I would be totally supportive of solving the timed metadata challenge > with WebVTT and by creating a WebVTT encapsulation spec for WebM.
> As for the non-timed metadata: Matroska also has a solution for > header-style metadata, i.e. metadata such as id3 tags that need to be > available to seach in apps such as iTunes. I would suggest building on > that existing solution.
> HTH.
> Cheers, > Silvia.
I don't know the specifics of WebVTT nor SRT, but if what Silvia has described really is feasible, then it sounds great. I would break it down as follows:
For "global" metadata, we use Matroska's existing features for "tag"-style information. This would require basically zero modification to existing implementations that already support.
For temporal metadata, as this is a new feature, this WebVTT encapsulation seems to be appropriate. It would not conflict with anything, and support for temporal metadata already requires new code to be written, and it would be one less bit of technology to have to deal with (namely, HTML5 is already pushing for this format for metadata of a similar nature).
Now, does the HTML5 spec itself mandate that the metadata actually be external to the stream, or just that it be stored separately in WebVTT format (allowing it to be encapsulated and/or extracted)? This is more a question for Silvia.
> On Thu, Jan 19, 2012 at 11:39 AM, James Zern<jz...@google.com> wrote: >> On Wed, Jan 18, 2012 at 12:31, Basil Mohamed Gohar >> <abu_huray...@hidayahonline.org> wrote: >>> On 01/18/2012 03:20 PM, James Zern wrote: >>>> A metadata specification for WebM has been a long requested feature. >>>> We plan on addressing this in the near term, but wanted to have a >>>> general discussion before we propose a solution.
>>>> One requirement we have is the support for temporal metadata to store >>>> things like geolocation data, e.g., GPS coordinates. The other is the >>>> traditional global metadata, e.g., title, author, etc. >>>> Initially we were thinking to define a separate standard for both >>>> global and temporal, which would be stored in its own track. After a >>>> bit of discussion it came to feel as though global was merely a >>>> special case of the temporal, which we knew we wanted.
>>>> Consider some examples: >>>> i) A video clip with an audio soundtrack that changes sources through the clip. >>>> With one audio and one video track it would be simple enough to have >>>> an artist label that covered each or both. With multiple audio sources >>>> in one track you could similarly have an artists tag. This though >>>> begins to decouple the attribution for each segment. >>>> ii) Appending 2 or more unrelated videos. >>>> This is a similar problem. You get into an issue with having to merge >>>> metadata from each segment into one global blob. At that point which >>>> takes precedence as the 'artist' for the clip? >>>> iii) Live presentations. >>>> Once more the performer, etc. can change throughout the clip, like a >>>> shoutcast stream.
>>>> It is true that a global mapping could be constructed to handle the >>>> above cases, but it seems that it might be simpler to have a timed >>>> metadata track that could indicate the duration that the value applied >>>> to. Global or live metadata could simply have no duration to indicate >>>> it was for the entire clip or until a new value was encountered (for >>>> the live case).
>>>> Does anyone have any thoughts on this? Is there any reason we >>>> shouldn’t explore a single metadata solution using a timed track?
>>> (Note, I'm neither a codec nor a container format developer, just a >>> geeky user, so take all of this with that knowledge first.)
>>> I see the argument for the global metadata being a special case of the >>> temporal, but in the global case, there is usually a high likelihood >>> that the metadata is always in the same place (e.g., either at the >>> beginning or the end of the file) which yields a much, much simpler >>> method of extraction for client software.
>>> In the temporal case, I suppose it could be constructed in such a way >>> that, in the case that it doesn't real change, then it would effectively >>> behave as the global case - that is, it would always be located in the >>> same place, and remain easily extracted.
>>> I would suggest something along the lines of temporal data being >>> considered updates to existing metadata, and if a previous value is not >>> marked as having been replaced by subsequent metadata, then it should >>> persist as long as its associated tracks do. >> Semantically this sounds similar to special casing the global >> metadata. It sounds as though you're more concerned about where the >> base data lives in the file and how to extract that.
>>> Finally, and I'm sure this has already been considered, but what already >>> exists within the Matroska world with regards to metadata? I would >>> strongly urge that whatever is developed be made as compatible as >>> possible with what is already out there. >> Matroska defines a hierarchical tag system [1], which allows tags to >> be attached to tracks or chapters. Note chapters are not part of the >> webm specification currently. >> I agree that using an existing system for which there are tools to >> manipulate the data is usually best, which is part of the reason for >> this thread. With tags, for instance though, I think we'd have to >> extend them to reference clusters or blocks. That or an excessive >> amount of chapters might be needed. >> Beyond this, will all tag systems, Matroska tags, XMP, etc. we will >> need to have a list of well-known entities as well as define new >> entities where we see a need. In this case some of the tools for >> extraction will need to be updated as well. Given we're talking about >> a web format, should we use these as a reference and provide a >> solution that fits better in this framework rather than worry about >> compatibility?
>> -- >> You received this message because you are subscribed to the Google Groups "WebM Discussion" group. >> To post to this group, send email to webm-disc...@webmproject.org. >> To unsubscribe from this group, send email to webm-discuss+unsubscr...@webmproject.org. >> For more options, visit this group at http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
<silviapfeiff...@gmail.com> wrote: > Talking about metadata and timed metadata: have you considered how > this hooks into what is being done in HTML5?
> There, we have defined WebVTT as a file format for external > time-synchronized data (including captions, subtitles, descriptions, > and - yes - general metadata).
Yes this is what initially got our discussion going. Using something along these lines we'd then only need one implementation for metadata rather than two.
> In the past it has been stated that the idea for captions in WebM > would be to use the solution that is being used in HTML5 and > encapsulate it into WebM, i.e. encapsulate WebVTT into WebM. Since > WebVTT is similar to SRT and there are existing specs for how to > encapsulate SRT into Matroska, putting WebVTT into WebM should be > possible in a similar manner.
> I would be totally supportive of solving the timed metadata challenge > with WebVTT and by creating a WebVTT encapsulation spec for WebM.
> As for the non-timed metadata: Matroska also has a solution for > header-style metadata, i.e. metadata such as id3 tags that need to be > available to seach in apps such as iTunes. I would suggest building on > that existing solution.
> HTH.
> Cheers, > Silvia.
> On Thu, Jan 19, 2012 at 11:39 AM, James Zern <jz...@google.com> wrote: >> On Wed, Jan 18, 2012 at 12:31, Basil Mohamed Gohar >> <abu_huray...@hidayahonline.org> wrote: >>> On 01/18/2012 03:20 PM, James Zern wrote: >>>> A metadata specification for WebM has been a long requested feature. >>>> We plan on addressing this in the near term, but wanted to have a >>>> general discussion before we propose a solution.
>>>> One requirement we have is the support for temporal metadata to store >>>> things like geolocation data, e.g., GPS coordinates. The other is the >>>> traditional global metadata, e.g., title, author, etc. >>>> Initially we were thinking to define a separate standard for both >>>> global and temporal, which would be stored in its own track. After a >>>> bit of discussion it came to feel as though global was merely a >>>> special case of the temporal, which we knew we wanted.
>>>> Consider some examples: >>>> i) A video clip with an audio soundtrack that changes sources through the clip. >>>> With one audio and one video track it would be simple enough to have >>>> an artist label that covered each or both. With multiple audio sources >>>> in one track you could similarly have an artists tag. This though >>>> begins to decouple the attribution for each segment. >>>> ii) Appending 2 or more unrelated videos. >>>> This is a similar problem. You get into an issue with having to merge >>>> metadata from each segment into one global blob. At that point which >>>> takes precedence as the 'artist' for the clip? >>>> iii) Live presentations. >>>> Once more the performer, etc. can change throughout the clip, like a >>>> shoutcast stream.
>>>> It is true that a global mapping could be constructed to handle the >>>> above cases, but it seems that it might be simpler to have a timed >>>> metadata track that could indicate the duration that the value applied >>>> to. Global or live metadata could simply have no duration to indicate >>>> it was for the entire clip or until a new value was encountered (for >>>> the live case).
>>>> Does anyone have any thoughts on this? Is there any reason we >>>> shouldn’t explore a single metadata solution using a timed track?
>>> (Note, I'm neither a codec nor a container format developer, just a >>> geeky user, so take all of this with that knowledge first.)
>>> I see the argument for the global metadata being a special case of the >>> temporal, but in the global case, there is usually a high likelihood >>> that the metadata is always in the same place (e.g., either at the >>> beginning or the end of the file) which yields a much, much simpler >>> method of extraction for client software.
>>> In the temporal case, I suppose it could be constructed in such a way >>> that, in the case that it doesn't real change, then it would effectively >>> behave as the global case - that is, it would always be located in the >>> same place, and remain easily extracted.
>>> I would suggest something along the lines of temporal data being >>> considered updates to existing metadata, and if a previous value is not >>> marked as having been replaced by subsequent metadata, then it should >>> persist as long as its associated tracks do.
>> Semantically this sounds similar to special casing the global >> metadata. It sounds as though you're more concerned about where the >> base data lives in the file and how to extract that.
>>> Finally, and I'm sure this has already been considered, but what already >>> exists within the Matroska world with regards to metadata? I would >>> strongly urge that whatever is developed be made as compatible as >>> possible with what is already out there.
>> Matroska defines a hierarchical tag system [1], which allows tags to >> be attached to tracks or chapters. Note chapters are not part of the >> webm specification currently. >> I agree that using an existing system for which there are tools to >> manipulate the data is usually best, which is part of the reason for >> this thread. With tags, for instance though, I think we'd have to >> extend them to reference clusters or blocks. That or an excessive >> amount of chapters might be needed. >> Beyond this, will all tag systems, Matroska tags, XMP, etc. we will >> need to have a list of well-known entities as well as define new >> entities where we see a need. In this case some of the tools for >> extraction will need to be updated as well. Given we're talking about >> a web format, should we use these as a reference and provide a >> solution that fits better in this framework rather than worry about >> compatibility?
>> -- >> You received this message because you are subscribed to the Google Groups "WebM Discussion" group. >> To post to this group, send email to webm-disc...@webmproject.org. >> To unsubscribe from this group, send email to webm-discuss+unsubscr...@webmproject.org. >> For more options, visit this group at http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
> -- > You received this message because you are subscribed to the Google Groups "WebM Discussion" group. > To post to this group, send email to webm-disc...@webmproject.org. > To unsubscribe from this group, send email to webm-discuss+unsubscr...@webmproject.org. > For more options, visit this group at http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
James Zern wrote: > Does anyone have any thoughts on this? Is there any reason we > shouldn t explore a single metadata solution using a timed track?
In 99% of all use cases, you will have only global metadata, so whatever you do, please make sure that global metadata, aka artist, album, song title can be extracted extremely fast and easily.
This means making it part of the matroska header section and not having to parse a data track to access that. Even for a video that has N different performers, it's annoying to have to seek through a potentially huge file to access the metadata only, the file could be on some slow HTTP connection after all..
> I don't know the specifics of WebVTT nor SRT, but if what Silvia has > described really is feasible, then it sounds great. I would break it down > as follows:
> For "global" metadata, we use Matroska's existing features for "tag"-style > information. This would require basically zero modification to existing > implementations that already support.
> For temporal metadata, as this is a new feature, this WebVTT encapsulation > seems to be appropriate. It would not conflict with anything, and support > for temporal metadata already requires new code to be written, and it would > be one less bit of technology to have to deal with (namely, HTML5 is already > pushing for this format for metadata of a similar nature).
> Now, does the HTML5 spec itself mandate that the metadata actually be > external to the stream, or just that it be stored separately in WebVTT > format (allowing it to be encapsulated and/or extracted)?
The TextTrack API for HTML5 supports both in-band and external text tracks. WebVTT is the format that all browsers have basically agreed to implement support for as the baseline external text track file format. On top of that, the TextTrack API also interfaces with text tracks provided in-band. They are basically mapped to the same data structure as is being parsed from a WebVTT file. What format browsers use there is still up in the air, but since we have the choice for WebM it makes sense to pick WebVTT itself.
For some side information: 1. I've heard MPEG people talk about defining a encapsulation means for WebVTT in MPEG-4, too. 2. I also think that WebVTT in Ogg would be simple and should be supported. We could even re-use KATE tracks for WebVTT encapsulation.
On Thu, Jan 19, 2012 at 1:46 PM, Frank Galligan <fgalli...@google.com> wrote:
>> I see the argument for the global metadata being a special case of the >> temporal, but in the global case, there is usually a high likelihood >> that the metadata is always in the same place (e.g., either at the >> beginning or the end of the file) which yields a much, much simpler >> method of extraction for client software.
> I agree, if you are only looking at supporting global metadata vs temporal > metadata. If you want to support both then having global metadata be a > special case of temporal is most likely simpler because you only need to > support one method of extraction/insertion vs two if they are use separate > storing methods.
I think it would be a mistake to deal with them in the same manner.
Applications such as iTunes and other media players as well as asset management systems typically find metadata in a defined location in a media file without having to seek, without having to parse more than the first few KB of data, and without having to parse all text tracks to determine where to find metadata.
Thus, the disadvantages of having a single method of dealing with metadata in my mind far outweigh the advantages. Header-style metadata (i.e. "tags") and timed metadata are two fundamentally different types of data and should be handled in two separate ways.
In Ogg we solved the problem of having changing file-wide metadata by allowing chaining of media files, i.e. essentially concatenating media resources. I am not sure that would be possible with WebM nor whether that is a good solution. I would actually much prefer introducing the concept of playlists for this situation and managing it in this way. It is a problem that I don't think we should solve on the file level.
On 20 January 2012 17:13, Silvia Pfeiffer <silviapfeiff...@gmail.com> wrote:
> Header-style metadata > (i.e. "tags") and timed metadata are two fundamentally different types > of data and should be handled in two separate ways.
I generally agree with this. File level metadata like title, creator, language, license, etc. have a different use case. Moreover, Matroska already has an established tag mechanism which many media players already know. I am in favour of using that encapsulation for WebM over a custom format, or the XMP blob idea mentioned on IRC.
That doesn't give a fixed offset for the metadata, but one can at least require it to be at the start (or end) of the file so the buffering requirements are minimal. The ebml parser subset needed is quite small.
For Ogg we did define a baseline semantic vocabulary for tags. I think that was helpful for interoperability, so I would support publishing such a set for WebM. Afaict Matroska itself doesn't itself do so?
Metadata of this type is generally for the convenience of indexers (online or in a media player). For example, our media code in Firefox completely ignores the tags. I'm not sure it's especially necessary to provide an interface for this either, since one can parse the resource directly in javascript.
I'm also excited to hear you're interested in timed metadata. For that I would point out the kind=metadata variant of webvtt.
Timed metadata is always going to be a niche application, and needs specific application support to be useful. We're already implementing webvtt as a format for HTML5 <track> elements, so it's easy to attach to files; the specification also allows embedding the same data in the media resource itself. The most webby thing to do would be to use the same format for subtitles and for timed metadata so we content can use the same interface to support whatever it wants to. Right now the interpretation of kind=metadata webvtt is completely up to the application, but for interoperability we could suggest a general framework, like json cue text with a defined semantic vocabulary for common use cases like geolocation, camera parameters and so on.
FWIW, -r
-- Ralph Giles Xiph.org Foundation for open multimedia
On Tue, Jan 24, 2012 at 8:56 PM, Ralph Giles <gi...@xiph.org> wrote: > On 20 January 2012 17:13, Silvia Pfeiffer <silviapfeiff...@gmail.com> > wrote:
> > Header-style metadata > > (i.e. "tags") and timed metadata are two fundamentally different types > > of data and should be handled in two separate ways.
> I generally agree with this. File level metadata like title, creator, > language, license, etc. have a different use case. Moreover, Matroska > already has an established tag mechanism which many media players > already know. I am in favour of using that encapsulation for WebM over > a custom format, or the XMP blob idea mentioned on IRC.
> That doesn't give a fixed offset for the metadata, but one can at > least require it to be at the start (or end) of the file so the > buffering requirements are minimal. The ebml parser subset needed is > quite small.
> For Ogg we did define a baseline semantic vocabulary for tags. I think > that was helpful for interoperability, so I would support publishing > such a set for WebM. Afaict Matroska itself doesn't itself do so?
The plan is to define a set of standard metadata for WebM, both global and temporal.
> Metadata of this type is generally for the convenience of indexers > (online or in a media player). For example, our media code in Firefox > completely ignores the tags. I'm not sure it's especially necessary to > provide an interface for this either, since one can parse the resource > directly in javascript.
I'm not sure about this either. That will be up to the clients whether they want to expose this information.
> I'm also excited to hear you're interested in timed metadata. For that > I would point out the kind=metadata variant of webvtt.
Good to know (my vote too by the way). We first wanted to get everyone's opinion on the idea of just changing all metadata to be stored temporally. The next step I think will be to decide how to store the metadata within the WebM files. Then define a standard set of metadata.
> Timed metadata is always going to be a niche application, and needs > specific application support to be useful. We're already implementing > webvtt as a format for HTML5 <track> elements, so it's easy to attach > to files; the specification also allows embedding the same data in the > media resource itself.
This was one of the pros for storing all data temporally (and if we store it as WebVTT) we will pretty much get all metadata supported very quickly by the browsers.
The most webby thing to do would be to use the
> same format for subtitles and for timed metadata so we content can use > the same interface to support whatever it wants to. Right now the > interpretation of kind=metadata webvtt is completely up to the > application, but for interoperability we could suggest a general > framework, like json cue text with a defined semantic vocabulary for > common use cases like geolocation, camera parameters and so on.
The thinking was we would standardize a set of metadata for temporal as well.
> -- > Ralph Giles > Xiph.org Foundation for open multimedia
> -- > You received this message because you are subscribed to the Google Groups > "WebM Discussion" group. > To post to this group, send email to webm-disc...@webmproject.org. > To unsubscribe from this group, send email to > webm-discuss+unsubscr...@webmproject.org. > For more options, visit this group at > http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
> On Tue, Jan 24, 2012 at 8:56 PM, Ralph Giles <gi...@xiph.org > <mailto:gi...@xiph.org>> wrote:
> On 20 January 2012 17:13, Silvia Pfeiffer > <silviapfeiff...@gmail.com <mailto:silviapfeiff...@gmail.com>> wrote:
> > Header-style metadata > > (i.e. "tags") and timed metadata are two fundamentally different > types > > of data and should be handled in two separate ways.
> I generally agree with this. File level metadata like title, creator, > language, license, etc. have a different use case. Moreover, Matroska > already has an established tag mechanism which many media players > already know. I am in favour of using that encapsulation for WebM over > a custom format, or the XMP blob idea mentioned on IRC.
> That doesn't give a fixed offset for the metadata, but one can at > least require it to be at the start (or end) of the file so the > buffering requirements are minimal. The ebml parser subset needed is > quite small.
> For Ogg we did define a baseline semantic vocabulary for tags. I think > that was helpful for interoperability, so I would support publishing > such a set for WebM. Afaict Matroska itself doesn't itself do so?
> The plan is to define a set of standard metadata for WebM, both global > and temporal.
> Metadata of this type is generally for the convenience of indexers > (online or in a media player). For example, our media code in Firefox > completely ignores the tags. I'm not sure it's especially necessary to > provide an interface for this either, since one can parse the resource > directly in javascript.
> I'm not sure about this either. That will be up to the clients whether > they want to expose this information.
> I'm also excited to hear you're interested in timed metadata. For that > I would point out the kind=metadata variant of webvtt.
> Good to know (my vote too by the way). We first wanted to get > everyone's opinion on the idea of just changing all metadata to be > stored temporally. The next step I think will be to decide how to > store the metadata within the WebM files. Then define a standard set > of metadata.
> Timed metadata is always going to be a niche application, and needs > specific application support to be useful. We're already implementing > webvtt as a format for HTML5 <track> elements, so it's easy to attach > to files; the specification also allows embedding the same data in the > media resource itself.
> This was one of the pros for storing all data temporally (and if we > store it as WebVTT) we will pretty much get all metadata supported > very quickly by the browsers.
> The most webby thing to do would be to use the > same format for subtitles and for timed metadata so we content can use > the same interface to support whatever it wants to. Right now the > interpretation of kind=metadata webvtt is completely up to the > application, but for interoperability we could suggest a general > framework, like json cue text with a defined semantic vocabulary for > common use cases like geolocation, camera parameters and so on.
> The thinking was we would standardize a set of metadata for temporal > as well.
> FWIW, > -r
I know that WebM is frequently touted as a Web format, and sometimes I worry that this loses sight of the fact that it will be used, regardless of the intentions of the original implementers, as a normal file format as well. Why I bring this is up is that a lot of the decisions related to WebM seem to be focused on its usage as planned in the web browser, and sometimes that is to the detriment of usages outside of that framework.
For example, as Matroska already has a lot of metadata support, and implementations of Matroska already have good support for that, it seems like it should be paramount that this also be utilized as extensively as possible within WebM so as to make support for WebM within the wider Matroska infrastructure as easy as possible. Yes, WebM has the distinction of being a Web-prioritized format, but I don't think this should come at the expense of compatibility with other implementations of the parent format.
What I mean is, if something that this metadata proposal hopes to achieve can already be done by existing implementations, and doing so would basically require little to no work to support, then it should be the goal to make it be so. If it's not possible (as opposed to being just not favorable), then fine, we should go ahead and propose new ideas, but still in a way that wouldn't be "annoying" to existing implementations.
I hope I don't sound overly hostile, but I want WebM to reach the widest audience possible, and there is an installed base of Matroska users and developers already existing, and just dumping a new set of requirements to implement that incompatible will just cause an annoyance and, in my humble opinion, will stifle adoption of an otherwise outstanding free format.
<key points> So, to summarize my concerns, it seems that the temporal metadata appears to be a novel idea, and there is nothing significantly standardized or pre-existing within the Matroska world to support it, and HTML5 already is standardizing on WebVTT, and there exists an encapsulation of WebVTT for Matroska already (I hope I got that right), so the way forward for this seems clear - use and/or build on the existing support for WebVTT in Matroska, in a way that is compliant with the spirit and the letter of the HTML5 standard for temporal metadata.
As for so-called global and non-temporal metadata, I find (again, as a non-developer and a user only) no reason to not use Matroska's existing infrastructure for metadata/tag support. It already works, and doing something else seems foolish. On top of that, Matroska *does* seem to have published some guidelines as to a standard set of metadata, and we should take as much as possible from that, so as to take advantage of existing infrastructure. </key points>
Could someone better in the know point out to me what, if anything, seems faulty in the above two paragraphs that needs to be addressed, or is a case of "devil in the details", and we all agree on the basic idea of not reinventing the wheel, just making sure it's round enough? :)
<abu_huray...@hidayahonline.org> wrote: > On 01/25/2012 10:23 AM, Frank Galligan wrote: >> CIL
>> On Tue, Jan 24, 2012 at 8:56 PM, Ralph Giles <gi...@xiph.org >> <mailto:gi...@xiph.org>> wrote:
>> On 20 January 2012 17:13, Silvia Pfeiffer >> <silviapfeiff...@gmail.com <mailto:silviapfeiff...@gmail.com>> wrote:
>> > Header-style metadata >> > (i.e. "tags") and timed metadata are two fundamentally different >> types >> > of data and should be handled in two separate ways.
>> I generally agree with this. File level metadata like title, creator, >> language, license, etc. have a different use case. Moreover, Matroska >> already has an established tag mechanism which many media players >> already know. I am in favour of using that encapsulation for WebM over >> a custom format, or the XMP blob idea mentioned on IRC.
>> That doesn't give a fixed offset for the metadata, but one can at >> least require it to be at the start (or end) of the file so the >> buffering requirements are minimal. The ebml parser subset needed is >> quite small.
>> For Ogg we did define a baseline semantic vocabulary for tags. I think >> that was helpful for interoperability, so I would support publishing >> such a set for WebM. Afaict Matroska itself doesn't itself do so?
>> The plan is to define a set of standard metadata for WebM, both global >> and temporal.
>> Metadata of this type is generally for the convenience of indexers >> (online or in a media player). For example, our media code in Firefox >> completely ignores the tags. I'm not sure it's especially necessary to >> provide an interface for this either, since one can parse the resource >> directly in javascript.
>> I'm not sure about this either. That will be up to the clients whether >> they want to expose this information.
>> I'm also excited to hear you're interested in timed metadata. For that >> I would point out the kind=metadata variant of webvtt.
>> Good to know (my vote too by the way). We first wanted to get >> everyone's opinion on the idea of just changing all metadata to be >> stored temporally. The next step I think will be to decide how to >> store the metadata within the WebM files. Then define a standard set >> of metadata.
Yes I think so. We can now handle these in two separate tracks and eventually start a new discussion around webvtt metadata and how we might extend or update the global entries using it.
>> Timed metadata is always going to be a niche application, and needs >> specific application support to be useful. We're already implementing >> webvtt as a format for HTML5 <track> elements, so it's easy to attach >> to files; the specification also allows embedding the same data in the >> media resource itself.
>> This was one of the pros for storing all data temporally (and if we >> store it as WebVTT) we will pretty much get all metadata supported >> very quickly by the browsers.
>> The most webby thing to do would be to use the >> same format for subtitles and for timed metadata so we content can use >> the same interface to support whatever it wants to. Right now the >> interpretation of kind=metadata webvtt is completely up to the >> application, but for interoperability we could suggest a general >> framework, like json cue text with a defined semantic vocabulary for >> common use cases like geolocation, camera parameters and so on.
>> The thinking was we would standardize a set of metadata for temporal >> as well.
>> FWIW, >> -r
> I know that WebM is frequently touted as a Web format, and sometimes I > worry that this loses sight of the fact that it will be used, regardless > of the intentions of the original implementers, as a normal file format > as well. Why I bring this is up is that a lot of the decisions related > to WebM seem to be focused on its usage as planned in the web browser, > and sometimes that is to the detriment of usages outside of that framework.
> For example, as Matroska already has a lot of metadata support, and > implementations of Matroska already have good support for that, it seems > like it should be paramount that this also be utilized as extensively as > possible within WebM so as to make support for WebM within the wider > Matroska infrastructure as easy as possible. Yes, WebM has the > distinction of being a Web-prioritized format, but I don't think this > should come at the expense of compatibility with other implementations > of the parent format.
> What I mean is, if something that this metadata proposal hopes to > achieve can already be done by existing implementations, and doing so > would basically require little to no work to support, then it should be > the goal to make it be so. If it's not possible (as opposed to being > just not favorable), then fine, we should go ahead and propose new > ideas, but still in a way that wouldn't be "annoying" to existing > implementations.
That's fair. Some of the initial debate was focused on making life easier for the browser in extracting/parsing, that's why we brought the discussion here. I think the cons presented currently out weigh any pros (presumed or otherwise), however.
> I hope I don't sound overly hostile, but I want WebM to reach the widest > audience possible, and there is an installed base of Matroska users and > developers already existing, and just dumping a new set of requirements > to implement that incompatible will just cause an annoyance and, in my > humble opinion, will stifle adoption of an otherwise outstanding free > format.
> <key points> > So, to summarize my concerns, it seems that the temporal metadata > appears to be a novel idea, and there is nothing significantly > standardized or pre-existing within the Matroska world to support it, > and HTML5 already is standardizing on WebVTT, and there exists an > encapsulation of WebVTT for Matroska already (I hope I got that right),
Not quite yet, that will be coming in parallel with some of this.
> so the way forward for this seems clear - use and/or build on the > existing support for WebVTT in Matroska, in a way that is compliant with > the spirit and the letter of the HTML5 standard for temporal metadata.
> As for so-called global and non-temporal metadata, I find (again, as a > non-developer and a user only) no reason to not use Matroska's existing > infrastructure for metadata/tag support. It already works, and doing > something else seems foolish. On top of that, Matroska *does* seem to > have published some guidelines as to a standard set of metadata, and we > should take as much as possible from that, so as to take advantage of > existing infrastructure. > </key points>
> Could someone better in the know point out to me what, if anything, > seems faulty in the above two paragraphs that needs to be addressed, or > is a case of "devil in the details", and we all agree on the basic idea > of not reinventing the wheel, just making sure it's round enough? :)
> On Wed, Jan 25, 2012 at 07:38, Basil Mohamed Gohar > <abu_huray...@hidayahonline.org> wrote:
>> <key points> >> So, to summarize my concerns, it seems that the temporal metadata >> appears to be a novel idea, and there is nothing significantly >> standardized or pre-existing within the Matroska world to support it, >> and HTML5 already is standardizing on WebVTT, and there exists an >> encapsulation of WebVTT for Matroska already (I hope I got that right), > Not quite yet, that will be coming in parallel with some of this.
Right. I misremembered something Silvia mentioned earlier in the thread. *SRT* has an encapsulation, WebVTT is *like* SRT, so it should be *possible*. I hope I got *that* right now...
>> so the way forward for this seems clear - use and/or build on the >> existing support for WebVTT in Matroska, in a way that is compliant with >> the spirit and the letter of the HTML5 standard for temporal metadata.
<abu_huray...@hidayahonline.org> wrote: > Right. I misremembered something Silvia mentioned earlier in the > thread. *SRT* has an encapsulation, WebVTT is *like* SRT, so it should > be *possible*. I hope I got *that* right now...
That's my understanding as well. There's no proposal for how exactly to encapsulate webvtt in webm, but we should should be able to reuse the current matroska srt mapping: http://matroska.org/technical/specs/subtitles/srt.html
An open question is how to handle any of the positioning directives.
-r
-- Ralph Giles Xiph.org Foundation for open multimedia
> <key points> > So, to summarize my concerns, it seems that the temporal metadata > appears to be a novel idea, and there is nothing significantly > standardized or pre-existing within the Matroska world to support it, > and HTML5 already is standardizing on WebVTT, and there exists an > encapsulation of WebVTT for Matroska already (I hope I got that right), > so the way forward for this seems clear - use and/or build on the > existing support for WebVTT in Matroska, in a way that is compliant with > the spirit and the letter of the HTML5 standard for temporal metadata.
> As for so-called global and non-temporal metadata, I find (again, as a > non-developer and a user only) no reason to not use Matroska's existing > infrastructure for metadata/tag support. It already works, and doing > something else seems foolish. On top of that, Matroska *does* seem to > have published some guidelines as to a standard set of metadata, and we > should take as much as possible from that, so as to take advantage of > existing infrastructure. > </key points>
> Could someone better in the know point out to me what, if anything, > seems faulty in the above two paragraphs that needs to be addressed, or > is a case of "devil in the details", and we all agree on the basic idea > of not reinventing the wheel, just making sure it's round enough? :)
So, there are several things at work here:
Firstly we want to make sure WebM has the best possible solution both for global and timed metadata. Secondly we want to make sure that it works both with the Web and with Desktop applications. Thirdly it would be nice to be able to re-use existing tools that had been built for Matroska also for WebM.
I must admit that I am not completely opposed to reinventing the wheel if that means we get a better wheel. I.e. I would take the first goal over the third goal. However, it has to be very clear that it is indeed a better solution. For example, if WebM had a much better way of storing global metadata than the existing Matroska way, I would go for it - since we want to look forward to the future with WebM and not back.
Now, what criteria would we use for deciding whether one thing is better than another? I have some that I care about - do add your own, cause I'm sure I've overlooked a few.
I believe a good solution must satisfy the following: * it works well with what HTML5 has specified * it does not conflict with how existing tools work for Matroska * the way to use it is obvious: e.g. global metadata is not used for the same purpose as timed metadata * it does not confuse people as to what to do: e.g. there is only one way of doing global metadata
I can't think of anything else right now, so do add your own.
So, back to the technical discussion about global/mutiplexed metadata:
* I agree that WebVTT's timed metadata mechanism is the way to go for timed metadata in WebM, simply because that's how we do it in HTML5 and there is no mechanism for that in Matroska yet. Thus, as we determine how to encapsulate captions, subtitles and descriptions in WebM, we should encapsulate WebVTT metadata in exactly the same way and use that as the solution for timed metadata in WebM.
* Global metadata is a different issue: it really depends on what we want to be able to handle.
Is the global metadata that we want to support per track? If so, then it relates to more than just the text tracks and needs to also be able to be carried for audio and video tracks. In Ogg we do this with skeleton by having message header fields, and also by having VorbisComment headers on each track. This is useful because as you rip out certain tracks from a multitrack resource, you retain the metadata that relates to the individual track.
Or is it really global metadata that we want to support here? Then it should logically be carried independently of a track (independently of WebVTT tracks, too, which can have their own separate metadata). In Ogg we don't actually have a means for such global metadata.
Matroska's existing metadata (tagging) mechanism is very general: it allows to provide tags that either belong to a track, to the full resource, to a chapter, to a edition, or an attachment (IIUC). This may be too generic for us to support in WebM - in particular it seems to allow mixing track-based metadata with truely global metadata, which may be too confusing.
So, overall, I don't think we've properly specified the problem yet that we want to solve.
Also note that in HTML5 we actually don't have a way to expose global metadata to JavaScript (not truely global, not track-based global metadata): there is no API. The only thing that HTML5 knows right now is timed metadata through WebVTT. I would like to see such an API, which would also be used to e.g. expose ID3 tags for MPEG-based content to JavaScript, but I don't have high hopes for it because there also is no API to expose image metadata (e.g. EXIM) to JavaScript. The point that has been made in the past is that if JavaScript needs access to such metadata, the server should extract it from the file and hand it to the Web app rather than JS doing it itself.
On Wed, Jan 25, 2012 at 14:18, Ralph Giles <gi...@xiph.org> wrote: > On 26 January 2012 08:23, Basil Mohamed Gohar > <abu_huray...@hidayahonline.org> wrote:
>> Right. I misremembered something Silvia mentioned earlier in the >> thread. *SRT* has an encapsulation, WebVTT is *like* SRT, so it should >> be *possible*. I hope I got *that* right now...
> That's my understanding as well. There's no proposal for how exactly > to encapsulate webvtt in webm, but we should should be able to reuse > the current matroska srt mapping: > http://matroska.org/technical/specs/subtitles/srt.html
> An open question is how to handle any of the positioning directives.
A thread regarding WebVTT encapsulation was started here [1].
I don't think there is anything wrong with what you are saying. I think most of us go out of our way to make sure any new features are compatible with what is out there. I know when I'm looking at new features I always think how it is going to effect current tools and regularly shoot down ideas that would force a change to existing muxers/demuxers if there is a viable alternative. With that being said sometimes you need to look at ideas that might changes some things if you think in the long run the benefit will outweigh the pain. This thread was started as a RFC to see what people thought of the idea of treating all metadata as temporal. This wasn't to benefit the web, but to benefit all implementations as they didn't have to treat metadata separately as all data is temporal in media files (global being from start to duration). So far pretty much everyone has said they would not like to make this change.
I pretty much agree with your point that if a feature that people want implemented in WebM can already be achieved in Matroska we should look there first. But that doesn't mean we should use that feature without looking to make sure that is the best route long term for all groups. (Personally I look at current Matroska tools and clients, HTML5 browsers, and other video tools and clients.) So yes I don't want to reinvent the wheel, unless it is not round enough.
It looks like pretty much most people want separate metadata, global and temporal, and most people have said they would like to use Matroska Tags for the global metadata. How do people use MKV tags today? Are there any deficiencies with the tags spec?
Frank
On Wed, Jan 25, 2012 at 10:38 AM, Basil Mohamed Gohar <
abu_huray...@hidayahonline.org> wrote: > On 01/25/2012 10:23 AM, Frank Galligan wrote: > > CIL
> > On Tue, Jan 24, 2012 at 8:56 PM, Ralph Giles <gi...@xiph.org > > <mailto:gi...@xiph.org>> wrote:
> > On 20 January 2012 17:13, Silvia Pfeiffer > > <silviapfeiff...@gmail.com <mailto:silviapfeiff...@gmail.com>> > wrote:
> > > Header-style metadata > > > (i.e. "tags") and timed metadata are two fundamentally different > > types > > > of data and should be handled in two separate ways.
> > I generally agree with this. File level metadata like title, creator, > > language, license, etc. have a different use case. Moreover, Matroska > > already has an established tag mechanism which many media players > > already know. I am in favour of using that encapsulation for WebM > over > > a custom format, or the XMP blob idea mentioned on IRC.
> > That doesn't give a fixed offset for the metadata, but one can at > > least require it to be at the start (or end) of the file so the > > buffering requirements are minimal. The ebml parser subset needed is > > quite small.
> > For Ogg we did define a baseline semantic vocabulary for tags. I > think > > that was helpful for interoperability, so I would support publishing > > such a set for WebM. Afaict Matroska itself doesn't itself do so?
> > The plan is to define a set of standard metadata for WebM, both global > > and temporal.
> > Metadata of this type is generally for the convenience of indexers > > (online or in a media player). For example, our media code in Firefox > > completely ignores the tags. I'm not sure it's especially necessary > to > > provide an interface for this either, since one can parse the > resource > > directly in javascript.
> > I'm not sure about this either. That will be up to the clients whether > > they want to expose this information.
> > I'm also excited to hear you're interested in timed metadata. For > that > > I would point out the kind=metadata variant of webvtt.
> > Good to know (my vote too by the way). We first wanted to get > > everyone's opinion on the idea of just changing all metadata to be > > stored temporally. The next step I think will be to decide how to > > store the metadata within the WebM files. Then define a standard set > > of metadata.
> > Timed metadata is always going to be a niche application, and needs > > specific application support to be useful. We're already implementing > > webvtt as a format for HTML5 <track> elements, so it's easy to attach > > to files; the specification also allows embedding the same data in > the > > media resource itself.
> > This was one of the pros for storing all data temporally (and if we > > store it as WebVTT) we will pretty much get all metadata supported > > very quickly by the browsers.
> > The most webby thing to do would be to use the > > same format for subtitles and for timed metadata so we content can > use > > the same interface to support whatever it wants to. Right now the > > interpretation of kind=metadata webvtt is completely up to the > > application, but for interoperability we could suggest a general > > framework, like json cue text with a defined semantic vocabulary for > > common use cases like geolocation, camera parameters and so on.
> > The thinking was we would standardize a set of metadata for temporal > > as well.
> > FWIW, > > -r
> I know that WebM is frequently touted as a Web format, and sometimes I > worry that this loses sight of the fact that it will be used, regardless > of the intentions of the original implementers, as a normal file format > as well. Why I bring this is up is that a lot of the decisions related > to WebM seem to be focused on its usage as planned in the web browser, > and sometimes that is to the detriment of usages outside of that framework.
> For example, as Matroska already has a lot of metadata support, and > implementations of Matroska already have good support for that, it seems > like it should be paramount that this also be utilized as extensively as > possible within WebM so as to make support for WebM within the wider > Matroska infrastructure as easy as possible. Yes, WebM has the > distinction of being a Web-prioritized format, but I don't think this > should come at the expense of compatibility with other implementations > of the parent format.
> What I mean is, if something that this metadata proposal hopes to > achieve can already be done by existing implementations, and doing so > would basically require little to no work to support, then it should be > the goal to make it be so. If it's not possible (as opposed to being > just not favorable), then fine, we should go ahead and propose new > ideas, but still in a way that wouldn't be "annoying" to existing > implementations.
> I hope I don't sound overly hostile, but I want WebM to reach the widest > audience possible, and there is an installed base of Matroska users and > developers already existing, and just dumping a new set of requirements > to implement that incompatible will just cause an annoyance and, in my > humble opinion, will stifle adoption of an otherwise outstanding free > format.
> <key points> > So, to summarize my concerns, it seems that the temporal metadata > appears to be a novel idea, and there is nothing significantly > standardized or pre-existing within the Matroska world to support it, > and HTML5 already is standardizing on WebVTT, and there exists an > encapsulation of WebVTT for Matroska already (I hope I got that right), > so the way forward for this seems clear - use and/or build on the > existing support for WebVTT in Matroska, in a way that is compliant with > the spirit and the letter of the HTML5 standard for temporal metadata.
> As for so-called global and non-temporal metadata, I find (again, as a > non-developer and a user only) no reason to not use Matroska's existing > infrastructure for metadata/tag support. It already works, and doing > something else seems foolish. On top of that, Matroska *does* seem to > have published some guidelines as to a standard set of metadata, and we > should take as much as possible from that, so as to take advantage of > existing infrastructure. > </key points>
> Could someone better in the know point out to me what, if anything, > seems faulty in the above two paragraphs that needs to be addressed, or > is a case of "devil in the details", and we all agree on the basic idea > of not reinventing the wheel, just making sure it's round enough? :)
> -- > You received this message because you are subscribed to the Google Groups > "WebM Discussion" group. > To post to this group, send email to webm-disc...@webmproject.org. > To unsubscribe from this group, send email to > webm-discuss+unsubscr...@webmproject.org. > For more options, visit this group at > http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
When defining global metadata I think we should go with most people think of. Most people think of global metadata as "title", "creator", "license", .. These are tied to the file. I do understand that if you tied metadata to the track you satisfy some smaller use cases requirements (e.g. adding a music track to a video sequence), but for most uses cases you are going to be adding complexity and rules to define how global metadata should be resolved.
I think we should define global metadata as: * Pertaining to the duration of the file (or live stream). I.e. not to individual streams. * Not temporal.
Anything else anyone want to add?
Frank
On Wed, Jan 25, 2012 at 9:25 PM, Silvia Pfeiffer <silviapfeiff...@gmail.com>wrote:
> On Thu, Jan 26, 2012 at 2:38 AM, Basil Mohamed Gohar > <abu_huray...@hidayahonline.org> wrote: > <...>
> > <key points> > > So, to summarize my concerns, it seems that the temporal metadata > > appears to be a novel idea, and there is nothing significantly > > standardized or pre-existing within the Matroska world to support it, > > and HTML5 already is standardizing on WebVTT, and there exists an > > encapsulation of WebVTT for Matroska already (I hope I got that right), > > so the way forward for this seems clear - use and/or build on the > > existing support for WebVTT in Matroska, in a way that is compliant with > > the spirit and the letter of the HTML5 standard for temporal metadata.
> > As for so-called global and non-temporal metadata, I find (again, as a > > non-developer and a user only) no reason to not use Matroska's existing > > infrastructure for metadata/tag support. It already works, and doing > > something else seems foolish. On top of that, Matroska *does* seem to > > have published some guidelines as to a standard set of metadata, and we > > should take as much as possible from that, so as to take advantage of > > existing infrastructure. > > </key points>
> > Could someone better in the know point out to me what, if anything, > > seems faulty in the above two paragraphs that needs to be addressed, or > > is a case of "devil in the details", and we all agree on the basic idea > > of not reinventing the wheel, just making sure it's round enough? :)
> So, there are several things at work here:
> Firstly we want to make sure WebM has the best possible solution both > for global and timed metadata. > Secondly we want to make sure that it works both with the Web and with > Desktop applications. > Thirdly it would be nice to be able to re-use existing tools that had > been built for Matroska also for WebM.
> I must admit that I am not completely opposed to reinventing the wheel > if that means we get a better wheel. I.e. I would take the first goal > over the third goal. However, it has to be very clear that it is > indeed a better solution. For example, if WebM had a much better way > of storing global metadata than the existing Matroska way, I would go > for it - since we want to look forward to the future with WebM and not > back.
> Now, what criteria would we use for deciding whether one thing is > better than another? I have some that I care about - do add your own, > cause I'm sure I've overlooked a few.
> I believe a good solution must satisfy the following: > * it works well with what HTML5 has specified > * it does not conflict with how existing tools work for Matroska > * the way to use it is obvious: e.g. global metadata is not used for > the same purpose as timed metadata > * it does not confuse people as to what to do: e.g. there is only one > way of doing global metadata
> I can't think of anything else right now, so do add your own.
> So, back to the technical discussion about global/mutiplexed metadata:
> * I agree that WebVTT's timed metadata mechanism is the way to go for > timed metadata in WebM, simply because that's how we do it in HTML5 > and there is no mechanism for that in Matroska yet. Thus, as we > determine how to encapsulate captions, subtitles and descriptions in > WebM, we should encapsulate WebVTT metadata in exactly the same way > and use that as the solution for timed metadata in WebM.
> * Global metadata is a different issue: it really depends on what we > want to be able to handle.
> Is the global metadata that we want to support per track? If so, then > it relates to more than just the text tracks and needs to also be able > to be carried for audio and video tracks. In Ogg we do this with > skeleton by having message header fields, and also by having > VorbisComment headers on each track. This is useful because as you rip > out certain tracks from a multitrack resource, you retain the metadata > that relates to the individual track.
> Or is it really global metadata that we want to support here? Then it > should logically be carried independently of a track (independently of > WebVTT tracks, too, which can have their own separate metadata). In > Ogg we don't actually have a means for such global metadata.
> Matroska's existing metadata (tagging) mechanism is very general: it > allows to provide tags that either belong to a track, to the full > resource, to a chapter, to a edition, or an attachment (IIUC). This > may be too generic for us to support in WebM - in particular it seems > to allow mixing track-based metadata with truely global metadata, > which may be too confusing.
> So, overall, I don't think we've properly specified the problem yet > that we want to solve.
> Also note that in HTML5 we actually don't have a way to expose global > metadata to JavaScript (not truely global, not track-based global > metadata): there is no API. The only thing that HTML5 knows right now > is timed metadata through WebVTT. I would like to see such an API, > which would also be used to e.g. expose ID3 tags for MPEG-based > content to JavaScript, but I don't have high hopes for it because > there also is no API to expose image metadata (e.g. EXIM) to > JavaScript. The point that has been made in the past is that if > JavaScript needs access to such metadata, the server should extract it > from the file and hand it to the Web app rather than JS doing it > itself.
> Cheers, > Silvia.
> -- > You received this message because you are subscribed to the Google Groups > "WebM Discussion" group. > To post to this group, send email to webm-disc...@webmproject.org. > To unsubscribe from this group, send email to > webm-discuss+unsubscr...@webmproject.org. > For more options, visit this group at > http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
> When defining global metadata I think we should go with most people > think of. Most people think of global metadata as "title", "creator", > "license", .. These are tied to the file. I do understand that if you > tied metadata to the track you satisfy some smaller use cases > requirements (e.g. adding a music track to a video sequence), but for > most uses cases you are going to be adding complexity and rules to > define how global metadata should be resolved.
I believe this would cover the vast majority of cases and is also what people expect, because, at least in my experience, most metadata/tags in multimedia files are treated globally, and applied to all content of the file as a whole, as opposed to individual tracks. I believe the same is true for any multimedia used in a streaming or Web context (e.g., YouTube videos, streaming media, etc.) which could be considered some of possible new uses of WebM that were not so widespread before in a standard format.
> I think we should define global metadata as: > * Pertaining to the duration of the file (or live stream). I.e. not to > individual streams. > * Not temporal.
> Anything else anyone want to add?
I think this sounds found. Now, to be "that guy", I want to ask the question if this means there will basically be no way left to add useful and/or equivalent metadata on a track-by-track basis (e.g., maybe you do want to label a secondary audio track as a translation and attribute it to the translator as "creator"). Is that something that would be completely ruled-out? I am assuming that such a feature is already supported by Matroska.
I think this is a minor feature (i.e., being able to add metadata to tracks individually), but it would be nice to have it be *possible*, if not required. This way, pretty-much all options are there, and if an implementation does not want to bother with track-by-track metadata, then there'll be no blame on them, as long as they're still properly supporting the global metadata (which would apply to the file as a whole).
> On Wed, Jan 25, 2012 at 9:25 PM, Silvia Pfeiffer > <silviapfeiff...@gmail.com <mailto:silviapfeiff...@gmail.com>> wrote:
> On Thu, Jan 26, 2012 at 2:38 AM, Basil Mohamed Gohar > <abu_huray...@hidayahonline.org > <mailto:abu_huray...@hidayahonline.org>> wrote: > <...>
> > <key points> > > So, to summarize my concerns, it seems that the temporal metadata > > appears to be a novel idea, and there is nothing significantly > > standardized or pre-existing within the Matroska world to > support it, > > and HTML5 already is standardizing on WebVTT, and there exists an > > encapsulation of WebVTT for Matroska already (I hope I got that > right), > > so the way forward for this seems clear - use and/or build on the > > existing support for WebVTT in Matroska, in a way that is > compliant with > > the spirit and the letter of the HTML5 standard for temporal > metadata.
> > As for so-called global and non-temporal metadata, I find > (again, as a > > non-developer and a user only) no reason to not use Matroska's > existing > > infrastructure for metadata/tag support. It already works, and > doing > > something else seems foolish. On top of that, Matroska *does* > seem to > > have published some guidelines as to a standard set of metadata, > and we > > should take as much as possible from that, so as to take > advantage of > > existing infrastructure. > > </key points>
> > Could someone better in the know point out to me what, if anything, > > seems faulty in the above two paragraphs that needs to be > addressed, or > > is a case of "devil in the details", and we all agree on the > basic idea > > of not reinventing the wheel, just making sure it's round enough? :)
> So, there are several things at work here:
> Firstly we want to make sure WebM has the best possible solution both > for global and timed metadata. > Secondly we want to make sure that it works both with the Web and with > Desktop applications. > Thirdly it would be nice to be able to re-use existing tools that had > been built for Matroska also for WebM.
> I must admit that I am not completely opposed to reinventing the wheel > if that means we get a better wheel. I.e. I would take the first goal > over the third goal. However, it has to be very clear that it is > indeed a better solution. For example, if WebM had a much better way > of storing global metadata than the existing Matroska way, I would go > for it - since we want to look forward to the future with WebM and not > back.
> Now, what criteria would we use for deciding whether one thing is > better than another? I have some that I care about - do add your own, > cause I'm sure I've overlooked a few.
> I believe a good solution must satisfy the following: > * it works well with what HTML5 has specified > * it does not conflict with how existing tools work for Matroska > * the way to use it is obvious: e.g. global metadata is not used for > the same purpose as timed metadata > * it does not confuse people as to what to do: e.g. there is only one > way of doing global metadata
> I can't think of anything else right now, so do add your own.
> So, back to the technical discussion about global/mutiplexed metadata:
> * I agree that WebVTT's timed metadata mechanism is the way to go for > timed metadata in WebM, simply because that's how we do it in HTML5 > and there is no mechanism for that in Matroska yet. Thus, as we > determine how to encapsulate captions, subtitles and descriptions in > WebM, we should encapsulate WebVTT metadata in exactly the same way > and use that as the solution for timed metadata in WebM.
> * Global metadata is a different issue: it really depends on what we > want to be able to handle.
> Is the global metadata that we want to support per track? If so, then > it relates to more than just the text tracks and needs to also be able > to be carried for audio and video tracks. In Ogg we do this with > skeleton by having message header fields, and also by having > VorbisComment headers on each track. This is useful because as you rip > out certain tracks from a multitrack resource, you retain the metadata > that relates to the individual track.
> Or is it really global metadata that we want to support here? Then it > should logically be carried independently of a track (independently of > WebVTT tracks, too, which can have their own separate metadata). In > Ogg we don't actually have a means for such global metadata.
> Matroska's existing metadata (tagging) mechanism is very general: it > allows to provide tags that either belong to a track, to the full > resource, to a chapter, to a edition, or an attachment (IIUC). This > may be too generic for us to support in WebM - in particular it seems > to allow mixing track-based metadata with truely global metadata, > which may be too confusing.
> So, overall, I don't think we've properly specified the problem yet > that we want to solve.
> Also note that in HTML5 we actually don't have a way to expose global > metadata to JavaScript (not truely global, not track-based global > metadata): there is no API. The only thing that HTML5 knows right now > is timed metadata through WebVTT. I would like to see such an API, > which would also be used to e.g. expose ID3 tags for MPEG-based > content to JavaScript, but I don't have high hopes for it because > there also is no API to expose image metadata (e.g. EXIM) to > JavaScript. The point that has been made in the past is that if > JavaScript needs access to such metadata, the server should extract it > from the file and hand it to the Web app rather than JS doing it > itself.
> Cheers, > Silvia.
> -- > You received this message because you are subscribed to the Google > Groups "WebM Discussion" group. > To post to this group, send email to webm-disc...@webmproject.org > <mailto:webm-disc...@webmproject.org>. > To unsubscribe from this group, send email to > webm-discuss+unsubscr...@webmproject.org > <mailto:webm-discuss%2Bunsubscr...@webmproject.org>. > For more options, visit this group at > http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
> -- > You received this message because you are subscribed to the Google > Groups "WebM Discussion" group. > To post to this group, send email to webm-disc...@webmproject.org. > To unsubscribe from this group, send email to > webm-discuss+unsubscr...@webmproject.org. > For more options, visit this group at > http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
I think what you are asking for basically boils down to a specific subset of non-standard global metadata.
Like I said before I think the high-level process to define metadata in WebM should be: 1. Define what metadata is. I.E. global vs temporal, global definition, ... 2. Specify how metadata is stored. I.E. MKV tags, xmp, ... 3. Standardize a set of the metadata vocabulary.
Currently for 2. I have only heard of Tags and some people mention xmp. Both of which have support for track specific metadata. MKV tags has TrackUID element. XMP has pantries.
So I think what you want to do, you could do two different ways (unless someone comes up with another solution for #2 that everyone decides they want to use).
The first way would be to use the unsupported metadata implementation (assuming most people want to keep global metadata to the file only) that supports track specific metadata. e.g. for MKV tags you would set the TrackUID for your metadata. I think this will work for most muxers/demuxers but there could be some muxers/demuxers out there that would treat this file as invalid or strip the data.
The second way would be to use the supported metadata implementation with non-standard metadata. So basically you could define your names of the metadata like "audio track 1 creator", "audio 2 creator", "audio 3 creator", ...
I would vote to use the second method as it will be guaranteed to work with muxers/demuxers that implement the Webm metadata standard, even if the base standard has support for the first method.
Frank
On Thu, Jan 26, 2012 at 9:18 AM, Basil Mohamed Gohar <
abu_huray...@hidayahonline.org> wrote: > ** > On 01/26/2012 09:10 AM, Frank Galligan wrote:
> When defining global metadata I think we should go with most people think > of. Most people think of global metadata as "title", "creator", "license", > .. These are tied to the file. I do understand that if you tied metadata to > the track you satisfy some smaller use cases requirements (e.g. adding a > music track to a video sequence), but for most uses cases you are going to > be adding complexity and rules to define how global metadata should be > resolved.
> I believe this would cover the vast majority of cases and is also what > people expect, because, at least in my experience, most metadata/tags in > multimedia files are treated globally, and applied to all content of the > file as a whole, as opposed to individual tracks. I believe the same is > true for any multimedia used in a streaming or Web context (e.g., YouTube > videos, streaming media, etc.) which could be considered some of possible > new uses of WebM that were not so widespread before in a standard format.
> I think we should define global metadata as: > * Pertaining to the duration of the file (or live stream). I.e. not to > individual streams. > * Not temporal.
> Anything else anyone want to add?
> I think this sounds found. Now, to be "that guy", I want to ask the > question if this means there will basically be no way left to add useful > and/or equivalent metadata on a track-by-track basis (e.g., maybe you do > want to label a secondary audio track as a translation and attribute it to > the translator as "creator"). Is that something that would be completely > ruled-out? I am assuming that such a feature is already supported by > Matroska.
> I think this is a minor feature (i.e., being able to add metadata to > tracks individually), but it would be nice to have it be *possible*, if not > required. This way, pretty-much all options are there, and if an > implementation does not want to bother with track-by-track metadata, then > there'll be no blame on them, as long as they're still properly supporting > the global metadata (which would apply to the file as a whole).
> Frank
> On Wed, Jan 25, 2012 at 9:25 PM, Silvia Pfeiffer < > silviapfeiff...@gmail.com> wrote:
>> On Thu, Jan 26, 2012 at 2:38 AM, Basil Mohamed Gohar >> <abu_huray...@hidayahonline.org> wrote: >> <...>
>> > <key points> >> > So, to summarize my concerns, it seems that the temporal metadata >> > appears to be a novel idea, and there is nothing significantly >> > standardized or pre-existing within the Matroska world to support it, >> > and HTML5 already is standardizing on WebVTT, and there exists an >> > encapsulation of WebVTT for Matroska already (I hope I got that right), >> > so the way forward for this seems clear - use and/or build on the >> > existing support for WebVTT in Matroska, in a way that is compliant with >> > the spirit and the letter of the HTML5 standard for temporal metadata.
>> > As for so-called global and non-temporal metadata, I find (again, as a >> > non-developer and a user only) no reason to not use Matroska's existing >> > infrastructure for metadata/tag support. It already works, and doing >> > something else seems foolish. On top of that, Matroska *does* seem to >> > have published some guidelines as to a standard set of metadata, and we >> > should take as much as possible from that, so as to take advantage of >> > existing infrastructure. >> > </key points>
>> > Could someone better in the know point out to me what, if anything, >> > seems faulty in the above two paragraphs that needs to be addressed, or >> > is a case of "devil in the details", and we all agree on the basic idea >> > of not reinventing the wheel, just making sure it's round enough? :)
>> So, there are several things at work here:
>> Firstly we want to make sure WebM has the best possible solution both >> for global and timed metadata. >> Secondly we want to make sure that it works both with the Web and with >> Desktop applications. >> Thirdly it would be nice to be able to re-use existing tools that had >> been built for Matroska also for WebM.
>> I must admit that I am not completely opposed to reinventing the wheel >> if that means we get a better wheel. I.e. I would take the first goal >> over the third goal. However, it has to be very clear that it is >> indeed a better solution. For example, if WebM had a much better way >> of storing global metadata than the existing Matroska way, I would go >> for it - since we want to look forward to the future with WebM and not >> back.
>> Now, what criteria would we use for deciding whether one thing is >> better than another? I have some that I care about - do add your own, >> cause I'm sure I've overlooked a few.
>> I believe a good solution must satisfy the following: >> * it works well with what HTML5 has specified >> * it does not conflict with how existing tools work for Matroska >> * the way to use it is obvious: e.g. global metadata is not used for >> the same purpose as timed metadata >> * it does not confuse people as to what to do: e.g. there is only one >> way of doing global metadata
>> I can't think of anything else right now, so do add your own.
>> So, back to the technical discussion about global/mutiplexed metadata:
>> * I agree that WebVTT's timed metadata mechanism is the way to go for >> timed metadata in WebM, simply because that's how we do it in HTML5 >> and there is no mechanism for that in Matroska yet. Thus, as we >> determine how to encapsulate captions, subtitles and descriptions in >> WebM, we should encapsulate WebVTT metadata in exactly the same way >> and use that as the solution for timed metadata in WebM.
>> * Global metadata is a different issue: it really depends on what we >> want to be able to handle.
>> Is the global metadata that we want to support per track? If so, then >> it relates to more than just the text tracks and needs to also be able >> to be carried for audio and video tracks. In Ogg we do this with >> skeleton by having message header fields, and also by having >> VorbisComment headers on each track. This is useful because as you rip >> out certain tracks from a multitrack resource, you retain the metadata >> that relates to the individual track.
>> Or is it really global metadata that we want to support here? Then it >> should logically be carried independently of a track (independently of >> WebVTT tracks, too, which can have their own separate metadata). In >> Ogg we don't actually have a means for such global metadata.
>> Matroska's existing metadata (tagging) mechanism is very general: it >> allows to provide tags that either belong to a track, to the full >> resource, to a chapter, to a edition, or an attachment (IIUC). This >> may be too generic for us to support in WebM - in particular it seems >> to allow mixing track-based metadata with truely global metadata, >> which may be too confusing.
>> So, overall, I don't think we've properly specified the problem yet >> that we want to solve.
>> Also note that in HTML5 we actually don't have a way to expose global >> metadata to JavaScript (not truely global, not track-based global >> metadata): there is no API. The only thing that HTML5 knows right now >> is timed metadata through WebVTT. I would like to see such an API, >> which would also be used to e.g. expose ID3 tags for MPEG-based >> content to JavaScript, but I don't have high hopes for it because >> there also is no API to expose image metadata (e.g. EXIM) to >> JavaScript. The point that has been made in the past is that if >> JavaScript needs access to such metadata, the server should extract it >> from the file and hand it to the Web app rather than JS doing it >> itself.
>> Cheers, >> Silvia.
>> -- >> You received this message because you are subscribed to the Google >> Groups "WebM Discussion" group. >> To post to this group, send email to webm-disc...@webmproject.org. >> To unsubscribe from this group, send email to >> webm-discuss+unsubscr...@webmproject.org. >> For more options, visit this group at >> http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
> -- > You received this message because you are subscribed to the Google Groups > "WebM Discussion" group.
This is a lot harder to parse with tools than the structured approach where the track is called out explicitly in a field. Why not use the field when it already exists? Have you come across tools that ignore that field for MKV files?
Silvia.
On 27/01/2012, at 2:14 AM, Frank Galligan <fgalli...@google.com> wrote:
> I think what you are asking for basically boils down to a specific subset of non-standard global metadata.
> Like I said before I think the high-level process to define metadata in WebM should be: > 1. Define what metadata is. I.E. global vs temporal, global definition, ... > 2. Specify how metadata is stored. I.E. MKV tags, xmp, ... > 3. Standardize a set of the metadata vocabulary.
> Currently for 2. I have only heard of Tags and some people mention xmp. Both of which have support for track specific metadata. MKV tags has TrackUID element. XMP has pantries.
> So I think what you want to do, you could do two different ways (unless someone comes up with another solution for #2 that everyone decides they want to use).
> The first way would be to use the unsupported metadata implementation (assuming most people want to keep global metadata to the file only) that supports track specific metadata. e.g. for MKV tags you would set the TrackUID for your metadata. I think this will work for most muxers/demuxers but there could be some muxers/demuxers out there that would treat this file as invalid or strip the data.
> The second way would be to use the supported metadata implementation with non-standard metadata. So basically you could define your names of the metadata like "audio track 1 creator", "audio 2 creator", "audio 3 creator", ...
> I would vote to use the second method as it will be guaranteed to work with muxers/demuxers that implement the Webm metadata standard, even if the base standard has support for the first method.
> Frank
> On Thu, Jan 26, 2012 at 9:18 AM, Basil Mohamed Gohar <abu_huray...@hidayahonline.org> wrote: > On 01/26/2012 09:10 AM, Frank Galligan wrote:
>> When defining global metadata I think we should go with most people think of. Most people think of global metadata as "title", "creator", "license", .. These are tied to the file. I do understand that if you tied metadata to the track you satisfy some smaller use cases requirements (e.g. adding a music track to a video sequence), but for most uses cases you are going to be adding complexity and rules to define how global metadata should be resolved. > I believe this would cover the vast majority of cases and is also what people expect, because, at least in my experience, most metadata/tags in multimedia files are treated globally, and applied to all content of the file as a whole, as opposed to individual tracks. I believe the same is true for any multimedia used in a streaming or Web context (e.g., YouTube videos, streaming media, etc.) which could be considered some of possible new uses of WebM that were not so widespread before in a standard format.
>> I think we should define global metadata as: >> * Pertaining to the duration of the file (or live stream). I.e. not to individual streams. >> * Not temporal.
>> Anything else anyone want to add? > I think this sounds found. Now, to be "that guy", I want to ask the question if this means there will basically be no way left to add useful and/or equivalent metadata on a track-by-track basis (e.g., maybe you do want to label a secondary audio track as a translation and attribute it to the translator as "creator"). Is that something that would be completely ruled-out? I am assuming that such a feature is already supported by Matroska.
> I think this is a minor feature (i.e., being able to add metadata to tracks individually), but it would be nice to have it be *possible*, if not required. This way, pretty-much all options are there, and if an implementation does not want to bother with track-by-track metadata, then there'll be no blame on them, as long as they're still properly supporting the global metadata (which would apply to the file as a whole).
>> Frank
>> On Wed, Jan 25, 2012 at 9:25 PM, Silvia Pfeiffer <silviapfeiff...@gmail.com> wrote: >> On Thu, Jan 26, 2012 at 2:38 AM, Basil Mohamed Gohar >> <abu_huray...@hidayahonline.org> wrote: >> <...>
>> > <key points> >> > So, to summarize my concerns, it seems that the temporal metadata >> > appears to be a novel idea, and there is nothing significantly >> > standardized or pre-existing within the Matroska world to support it, >> > and HTML5 already is standardizing on WebVTT, and there exists an >> > encapsulation of WebVTT for Matroska already (I hope I got that right), >> > so the way forward for this seems clear - use and/or build on the >> > existing support for WebVTT in Matroska, in a way that is compliant with >> > the spirit and the letter of the HTML5 standard for temporal metadata.
>> > As for so-called global and non-temporal metadata, I find (again, as a >> > non-developer and a user only) no reason to not use Matroska's existing >> > infrastructure for metadata/tag support. It already works, and doing >> > something else seems foolish. On top of that, Matroska *does* seem to >> > have published some guidelines as to a standard set of metadata, and we >> > should take as much as possible from that, so as to take advantage of >> > existing infrastructure. >> > </key points>
>> > Could someone better in the know point out to me what, if anything, >> > seems faulty in the above two paragraphs that needs to be addressed, or >> > is a case of "devil in the details", and we all agree on the basic idea >> > of not reinventing the wheel, just making sure it's round enough? :)
>> So, there are several things at work here:
>> Firstly we want to make sure WebM has the best possible solution both >> for global and timed metadata. >> Secondly we want to make sure that it works both with the Web and with >> Desktop applications. >> Thirdly it would be nice to be able to re-use existing tools that had >> been built for Matroska also for WebM.
>> I must admit that I am not completely opposed to reinventing the wheel >> if that means we get a better wheel. I.e. I would take the first goal >> over the third goal. However, it has to be very clear that it is >> indeed a better solution. For example, if WebM had a much better way >> of storing global metadata than the existing Matroska way, I would go >> for it - since we want to look forward to the future with WebM and not >> back.
>> Now, what criteria would we use for deciding whether one thing is >> better than another? I have some that I care about - do add your own, >> cause I'm sure I've overlooked a few.
>> I believe a good solution must satisfy the following: >> * it works well with what HTML5 has specified >> * it does not conflict with how existing tools work for Matroska >> * the way to use it is obvious: e.g. global metadata is not used for >> the same purpose as timed metadata >> * it does not confuse people as to what to do: e.g. there is only one >> way of doing global metadata
>> I can't think of anything else right now, so do add your own.
>> So, back to the technical discussion about global/mutiplexed metadata:
>> * I agree that WebVTT's timed metadata mechanism is the way to go for >> timed metadata in WebM, simply because that's how we do it in HTML5 >> and there is no mechanism for that in Matroska yet. Thus, as we >> determine how to encapsulate captions, subtitles and descriptions in >> WebM, we should encapsulate WebVTT metadata in exactly the same way >> and use that as the solution for timed metadata in WebM.
>> * Global metadata is a different issue: it really depends on what we >> want to be able to handle.
>> Is the global metadata that we want to support per track? If so, then >> it relates to more than just the text tracks and needs to also be able >> to be carried for audio and video tracks. In Ogg we do this with >> skeleton by having message header fields, and also by having >> VorbisComment headers on each track. This is useful because as you rip >> out certain tracks from a multitrack resource, you retain the metadata >> that relates to the individual track.
>> Or is it really global metadata that we want to support here? Then it >> should logically be carried independently of a track (independently of >> WebVTT tracks, too, which can have their own separate metadata). In >> Ogg we don't actually have a means for such global metadata.
>> Matroska's existing metadata (tagging) mechanism is very general: it >> allows to provide tags that either belong to a track, to the full >> resource, to a chapter, to a edition, or an attachment (IIUC). This >> may be too generic for us to support in WebM - in particular it seems >> to allow mixing track-based metadata with truely global metadata, >> which may be too confusing.
>> So, overall, I don't think we've properly specified the problem yet >> that we want to solve.
>> Also note that in HTML5 we actually don't have a way to expose global >> metadata to JavaScript (not truely global, not track-based global >> metadata): there is no API. The only thing that HTML5 knows right now >> is timed metadata through WebVTT. I would like to see such an API, >> which would also be used to e.g. expose ID3 tags for MPEG-based >> content to JavaScript, but I don't have high hopes for it because >> there also is no API to expose image metadata (e.g. EXIM) to >> JavaScript. The point that has been made in the past is that if >> JavaScript needs access to such metadata, the server should extract it >> from the file and hand it to the Web app rather than JS doing it >> itself.
>> Cheers, >> Silvia.
>> -- >> You received this message because you are subscribed to the Google Groups "WebM Discussion" group. >> To post to this group, send email to
On Thu, Jan 26, 2012 at 4:25 PM, Silvia Pfeiffer <silviapfeiff...@gmail.com>wrote:
> This is a lot harder to parse with tools than the structured approach > where the track is called out explicitly in a field.
I don't understand. What is a lot harder to parse?
> Why not use the field when it already exists?
Just because something exists doesn't necessarily means it is a good idea to use it.
If we think it is a good idea to allow Tracks to have separate metadata from themselves and the file metadata that adds complexity to the whole system. E.g. if you take your example of adding an API to HTML5 to get the metadata. With only file metadata the api might just return a list of name value pairs. With file and tracks metadata the api must have a way of distinguishing the file metadata vs all of the tracks that may have metadata. The documentation becomes more complex for developers/users as now they need to know what a media file is and what a stream/track is. We may also have to define precedence rules. E.g. what happens if the file and tracks define different licenses?
Also it is much easier to add a feature to a spec or API later then it is to deprecate one.
> Have you come across tools that ignore that field for MKV files?
No, but most of my experience has been with WebM tools (tags aren't in spec) and FFmpeg which has support for it. I haven't come across any MKV files that had track specific data either, or maybe I have but the client app didn't have a track specific metadata UI. If you know of any I would like to check them out.
> On 27/01/2012, at 2:14 AM, Frank Galligan <fgalli...@google.com> wrote:
> I think what you are asking for basically boils down to a specific subset > of non-standard global metadata.
> Like I said before I think the high-level process to define metadata in > WebM should be: > 1. Define what metadata is. I.E. global vs temporal, global definition, ... > 2. Specify how metadata is stored. I.E. MKV tags, xmp, ... > 3. Standardize a set of the metadata vocabulary.
> Currently for 2. I have only heard of Tags and some people mention xmp. > Both of which have support for track specific metadata. MKV tags > has TrackUID element. XMP has pantries.
> So I think what you want to do, you could do two different ways (unless > someone comes up with another solution for #2 that everyone decides they > want to use).
> The first way would be to use the unsupported metadata implementation > (assuming most people want to keep global metadata to the file only) that > supports track specific metadata. e.g. for MKV tags you would set > the TrackUID for your metadata. I think this will work for most > muxers/demuxers but there could be some muxers/demuxers out there that > would treat this file as invalid or strip the data.
> The second way would be to use the supported metadata implementation with > non-standard metadata. So basically you could define your names of the > metadata like "audio track 1 creator", "audio 2 creator", "audio 3 > creator", ...
> I would vote to use the second method as it will be guaranteed to work > with muxers/demuxers that implement the Webm metadata standard, even if the > base standard has support for the first method.
> Frank
> On Thu, Jan 26, 2012 at 9:18 AM, Basil Mohamed Gohar < > abu_huray...@hidayahonline.org> wrote:
>> ** >> On 01/26/2012 09:10 AM, Frank Galligan wrote:
>> When defining global metadata I think we should go with most people think >> of. Most people think of global metadata as "title", "creator", "license", >> .. These are tied to the file. I do understand that if you tied metadata to >> the track you satisfy some smaller use cases requirements (e.g. adding a >> music track to a video sequence), but for most uses cases you are going to >> be adding complexity and rules to define how global metadata should be >> resolved.
>> I believe this would cover the vast majority of cases and is also what >> people expect, because, at least in my experience, most metadata/tags in >> multimedia files are treated globally, and applied to all content of the >> file as a whole, as opposed to individual tracks. I believe the same is >> true for any multimedia used in a streaming or Web context (e.g., YouTube >> videos, streaming media, etc.) which could be considered some of possible >> new uses of WebM that were not so widespread before in a standard format.
>> I think we should define global metadata as: >> * Pertaining to the duration of the file (or live stream). I.e. not to >> individual streams. >> * Not temporal.
>> Anything else anyone want to add?
>> I think this sounds found. Now, to be "that guy", I want to ask the >> question if this means there will basically be no way left to add useful >> and/or equivalent metadata on a track-by-track basis (e.g., maybe you do >> want to label a secondary audio track as a translation and attribute it to >> the translator as "creator"). Is that something that would be completely >> ruled-out? I am assuming that such a feature is already supported by >> Matroska.
>> I think this is a minor feature (i.e., being able to add metadata to >> tracks individually), but it would be nice to have it be *possible*, if not >> required. This way, pretty-much all options are there, and if an >> implementation does not want to bother with track-by-track metadata, then >> there'll be no blame on them, as long as they're still properly supporting >> the global metadata (which would apply to the file as a whole).
>> Frank
>> On Wed, Jan 25, 2012 at 9:25 PM, Silvia Pfeiffer < >> silviapfeiff...@gmail.com> wrote:
>>> On Thu, Jan 26, 2012 at 2:38 AM, Basil Mohamed Gohar >>> <abu_huray...@hidayahonline.org> wrote: >>> <...>
>>> > <key points> >>> > So, to summarize my concerns, it seems that the temporal metadata >>> > appears to be a novel idea, and there is nothing significantly >>> > standardized or pre-existing within the Matroska world to support it, >>> > and HTML5 already is standardizing on WebVTT, and there exists an >>> > encapsulation of WebVTT for Matroska already (I hope I got that right), >>> > so the way forward for this seems clear - use and/or build on the >>> > existing support for WebVTT in Matroska, in a way that is compliant >>> with >>> > the spirit and the letter of the HTML5 standard for temporal metadata.
>>> > As for so-called global and non-temporal metadata, I find (again, as a >>> > non-developer and a user only) no reason to not use Matroska's existing >>> > infrastructure for metadata/tag support. It already works, and doing >>> > something else seems foolish. On top of that, Matroska *does* seem to >>> > have published some guidelines as to a standard set of metadata, and we >>> > should take as much as possible from that, so as to take advantage of >>> > existing infrastructure. >>> > </key points>
>>> > Could someone better in the know point out to me what, if anything, >>> > seems faulty in the above two paragraphs that needs to be addressed, or >>> > is a case of "devil in the details", and we all agree on the basic idea >>> > of not reinventing the wheel, just making sure it's round enough? :)
>>> So, there are several things at work here:
>>> Firstly we want to make sure WebM has the best possible solution both >>> for global and timed metadata. >>> Secondly we want to make sure that it works both with the Web and with >>> Desktop applications. >>> Thirdly it would be nice to be able to re-use existing tools that had >>> been built for Matroska also for WebM.
>>> I must admit that I am not completely opposed to reinventing the wheel >>> if that means we get a better wheel. I.e. I would take the first goal >>> over the third goal. However, it has to be very clear that it is >>> indeed a better solution. For example, if WebM had a much better way >>> of storing global metadata than the existing Matroska way, I would go >>> for it - since we want to look forward to the future with WebM and not >>> back.
>>> Now, what criteria would we use for deciding whether one thing is >>> better than another? I have some that I care about - do add your own, >>> cause I'm sure I've overlooked a few.
>>> I believe a good solution must satisfy the following: >>> * it works well with what HTML5 has specified >>> * it does not conflict with how existing tools work for Matroska >>> * the way to use it is obvious: e.g. global metadata is not used for >>> the same purpose as timed metadata >>> * it does not confuse people as to what to do: e.g. there is only one >>> way of doing global metadata
>>> I can't think of anything else right now, so do add your own.
>>> So, back to the technical discussion about global/mutiplexed metadata:
>>> * I agree that WebVTT's timed metadata mechanism is the way to go for >>> timed metadata in WebM, simply because that's how we do it in HTML5 >>> and there is no mechanism for that in Matroska yet. Thus, as we >>> determine how to encapsulate captions, subtitles and descriptions in >>> WebM, we should encapsulate WebVTT metadata in exactly the same way >>> and use that as the solution for timed metadata in WebM.
>>> * Global metadata is a different issue: it really depends on what we >>> want to be able to handle.
>>> Is the global metadata that we want to support per track? If so, then >>> it relates to more than just the text tracks and needs to also be able >>> to be carried for audio and video tracks. In Ogg we do this with >>> skeleton by having message header fields, and also by having >>> VorbisComment headers on each track. This is useful because as you rip >>> out certain tracks from a multitrack resource, you retain the metadata >>> that relates to the individual track.
>>> Or is it really global metadata that we want to support here? Then it >>> should logically be carried independently of a track
On Fri, Jan 27, 2012 at 3:18 PM, Frank Galligan <fgalli...@google.com> wrote: > On Thu, Jan 26, 2012 at 4:25 PM, Silvia Pfeiffer <silviapfeiff...@gmail.com> > wrote:
>> This is a lot harder to parse with tools than the structured approach >> where the track is called out explicitly in a field.
> I don't understand. What is a lot harder to parse?
If be bury the fact that a certain name-value metadata pair applies to a track rather than the full file in the name of the name-value pair, then it becomes difficult to determine what that name-value pair applies to. For example:
I was under the impression that you were suggesting the first option.
>> Why not use the field when it already exists?
> Just because something exists doesn't necessarily means it is a good idea to > use it.
Sure.
> If we think it is a good idea to allow Tracks to have separate metadata from > themselves and the file metadata that adds complexity to the whole system.
Sure, I was replying under the assumption that we wanted to support it and your suggestion was to include it in the "name" field to which I object. If we don't want to support it, then sure we don't need this. But in this case we also should not try to support a hack for track-related metadata.
> E.g. if you take your example of adding an API to HTML5 to get the metadata. > With only file metadata the api might just return a list of name value > pairs. With file and tracks metadata the api must have a way of > distinguishing the file metadata vs all of the tracks that may have > metadata.
Right. There is one more field to parse. And that field already exists in the Matroska Tags.
We could certainly instead decide not to support that extra field and instead expect the tracks themselves to take care of their metadata. For example, text tracks in WebVTT may have metadata header fields that will be encoded in the CodePrivateData as per the spec proposal. That's another option for supporting track-related metadata. IIUC VorbisComment headers are already included in Vorbis tracks. So, track-related metadata would be another parsing step away, but it would be possible already.
> The documentation becomes more complex for developers/users as now > they need to know what a media file is and what a stream/track is. We may > also have to define precedence rules. E.g. what happens if the file and > tracks define different licenses?
It'a actually already a legal problem that a combination of content can have a different license to the individual components. This would be a way to represent the actual legal situation.
> Also it is much easier to add a feature to a spec or API later then it is to > deprecate one.
Yes, sure. I didn't want to say that we have to do it. I was just objecting to the proposed solution.
We should probably make a collection of Matroska files that have metadata and inspect them to see what is currently done. I'm pretty sure VLC supports it, but not sure to what extent.
