|VP9 Bitstream finalization update||Paul Wilkins||5/8/13 7:46 AM|
Here's the dated task list:
We wanted to update you all on the progress we are making with the vp9 bitstream and how we see things working out over the next couple of weeks as we moved towards bitstream finalization.
May 4: Intra filtering method at different scales completed.
May 7: The following unused experiments were removed: NEWBINTRAMODES, COMP_INTERINTRA_PRED
May 7: Merged sb8x8 code (extends the same divisions we did at superblock down to 8x8 blocks). +6% on cif, +2% on HD).May 19: Extend the same divisions above to 4x4 (4x4 8x4, 4x8 structure deprecating the existing splitmv and i4x4 modes.)
May 19: HW Requested reduction in Entropy context size and counters (current plan includes merge of SCATTERSCAN, and MODELCOEFFPROB experiments).
May 22: Color space / version / feature-set header bits added and defined:
May 25: June 17: Syntax test vectors / unit tests / further optimizations / any necessary last minute fixes.
May 24: Bitstream Final Candidate ( only minor bugs fixed)
June 17th : No more bitstream changes - Code checked into chrome / YouTube
|Re: VP9 Bitstream finalization update||Lou Quillio||5/10/13 4:16 PM|
On Fri, May 10, 2013 at 2:38 PM, Antony Williams <ant...@gmail.com> wrote:
> How does this relate to WebM ?
> WebM is currently VP8 w/ Vorbis and WebRTC is VP8 w/ Opus.
> Will WebM support VP8/9, or will there be a WebM2?
No WebM2: the existing WebM container will be extended to allow VP9
and Opus streams, and to accommodate any of their special features.
Content-type remains video/webm, and the codec strings are being
worked out with Matroska. Some threads on matroska-devel:
|Re: VP9 Bitstream finalization update||Basil Mohamed Gohar||5/12/13 6:16 PM|
I forget if this question has already been asked/answered, but are all
combinations of supported codecs also going to be supported? I realize
the question of "why" may be on everyone's mind, but there are always
cases where flexibility tends to be beneficial for certain usage cases
(e.g., hardware support for VP9 decoding, but only being able to decode
Vorbis for whatever reason). Likewise, maybe someone wants very high
quality and efficient audio with low latency, but only has the CPU to
encode/decode VP8 and not VP9.
|Re: VP9 Bitstream finalization update||John Koleszar||5/13/13 8:43 AM|
This is already supported, if I'm not mistaken. I think you can do
VP9+Vorbis today, for example. I'm sure someone will correct me if I'm
|Re: VP9 Bitstream finalization update||Ralph Giles||5/13/13 1:56 PM|
On 13-05-12 6:16 PM, Basil Mohamed Gohar wrote:The audio and video streams in Matroska are quite independent, so I
don't see any reason for implementations to limit themselves to only two
combinations of codecs.
|Re: VP9 Bitstream finalization update||Paul Wilkins||5/15/13 3:48 AM|
Here is the latest VP9 latest status update.
May 19: A hardware partner has raised a dependency issue, where reconstruction of neighbouring motion vectors is required in order to decode the current motion vector. We will review the implications of this dependency and report back by next week on solutions / impact.
|Re: VP9 Bitstream finalization update||John Koleszar||5/17/13 8:40 AM|
That document is still in progress, but should be available soon after
the bitstream is finalized. The best one to look at right now is
On Wed, May 15, 2013 at 8:59 PM, <shen...@gmail.com> wrote:
> Is there any document about the bitstream syntax and encoder algorithm
> 在 2013年5月8日星期三UTC+8下午10时46分50秒，Paul Wilkins写道：
>>>> May 19: Extend the same divisions above to 4x4 (4x4 8x4, 4x8 structure
>> deprecating the existing splitmv and i4x4 modes.)
>> (current plan includes merge of SCATTERSCAN, and MODELCOEFFPROB
>> May 19: 444, alpha and depth channel complete (In response to HW
>> May 22: Color space / version / feature-set header bits added and> --
> You received this message because you are subscribed to the Google Groups
> "WebM Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to webm-discuss...@webmproject.org.
> To post to this group, send email to webm-d...@webmproject.org.
> Visit this group at
> For more options, visit
|Re: VP9 Bitstream finalization update||Paul Wilkins||5/22/13 4:21 AM|
Here is the VP9 latest status update.
May 24: Extend the prediction sub divisions (4x4 8x4, 4x8 structure and deprecate the existing splitmv and i4x4 modes.) This work is essentially bit stream complete and should hopefully be merged out from under its configure flag later today. There are a few loose ends including correct behaviour of the loop filter and selection of the reference motion vector for small sub-blocks that should be resolved by the end of the week, along with some encoder side work and re-factoring relating to rd.
May 24: Dependency issue raised by a HW partner, where reconstruction of neighbouring motion vectors was required in order to decode the current motion vector. A patch has been submitted that should resolve this by only using the modes of neighbours to provide the context. This may need minor tweaking in the light of the work on the 8x4,4x8 etc. and does currently hurts compression performance by about 0.25%.
Intra prediction consistently tied to transform size. Work to insure that intra prediction is tied consistently to the transform size (as discussed at the VP9 summit) is bit stream complete and should be checked in today. Note that some further work is required (encoder side only) to insure the correct behaviour in the rate distortion loop.
A hardware partner has requested that we review the order of operations in the VP9 loop filter: Filter the vertical edges first, then the horizontal. This request is being reviewed and we will try and respond before the end of the week.
In response to a request, we have changed the frame sync code so that it is distinct from that used in VP8.
May 24: Color space / version / feature-set header bits added
10 bit: A number of strong representations have been made to us about the importance of 10-bit support for certain use cases and in the future for 4k video. In the light of these representations, and given that the 4:4:4 / alpha etc. work will now be subject to a version / profile flag (and not be a requirement for version 1 hardware), we are actively considering whether at some point in the future we should add support for 10 bit (also under a profile flag). We do not have any time-scale but continue to value your feedback on this and other issues.
|Re: VP9 Bitstream finalization update||Paul Wilkins||5/30/13 1:18 AM|
Extended the prediction sub divisions (4x4 8x4, 4x8 structure and deprecate the existing splitmv and i4x4 modes.) Removing the i4x4 and splitmv modes (from a bitstream perspective) is mainly checked in. Still some clean up and debug work to do and the splitmv and i4x4 modes still exist as "non bitstream normative" place holders for now, pending further re-factoring of the code.
444, alpha and depth channel: Mainly complete. Some ongoing work relating to requested changes to the loop filtering order (see below). NOTE: This work will now be placed under a version/profile flag in the bit stream header and support will not be required in version 1 hardware.
Hw vendor request for a change to the scan order for 32x32 blocks is under review.
Outside submission of an alternate WHT implementation that uses fewer shift operations is under review.
Color space / version / feature-set header bits added and defined: Patch for new color space bits submitted (waiting on some refactoring work for the uncompressed frame header). Definition of feature / profile bits etc. and their meanings on hold pending further input from partners.
|Re: VP9 Bitstream finalization update||Paul Wilkins||6/5/13 11:02 AM|
444, alpha and depth channel: Mainly complete.
Issues raised by / suggestions from HW partners.
· Loop filter order. Variant submitted and being tested in SW and re-checked by HW team.
· Optimised 32x32 scan order. A new more HW friendly scan order for 32x32 has been tested and merged.
· Last frame use for mv reference. Small change suggested by HW team tested and merged.
· Prior token context. Small change to calculation of prior token energy (suggested by HW team) tested and merged
· Fixed tile count off by one bug identified by HW partner.
Some header bits moved to the uncompressed partition.
Alternate WHT implementation tested and merged.
Removal of experiments that did not make the cut.
Version / feature-set header Definition of feature / profile bits etc. and their meanings on hold pending further input from partners.
Bitstream Final Candidate (only minor bugs fixed or issues raised by HW vendors).
|Re: VP9 Bitstream finalization update||Paul Wilkins||6/12/13 9:29 AM|
Here is the latest VP9 status update.
As of last night the VP9 (profile 0) bitstream is FINAL and the process of validating it on multiple platforms and in different build environments and testing for submission into Chrome is under way.
The last few days have been pretty hectic as we had to make several last minute changes. As always seems to be the case, these were more complex than we had hoped and I want to thank the whole team for working long hours to bring about closure. I also want to thank partners and hardware vendors who have submitted comments and suggestions over the past few weeks.
Details of the final issues resolved over the last 7 days are as follows:-
Changes to the loop filter. Changes to make the loop filter more HW friendly had to go through several iterations and gave rise to a couple of late bugs. As is often the case, initial attempts to make it more hardware friendly turned out to make things more difficult in SIMD software implementations. Hopefully the final solution is a comfortable compromise for both.
Changes to the way compound prediction modes are coded in the bitstream. Though not conceptually complex and necessary to correctly constrain the allowable compound prediction modes, this change touched a lot of files and getting it done on time and debugged was a Herculean effort by those involved.
Definition and implementation of behaviour at the edge of images. This relates to what prediction modes/sizes and transforms are allowed near the edge of an image that does not break down nicely into 64x64 blocks. I mentioned this at the summit and while it sounds simple, like all the simplest changes, it was actually quite hard to get right. Again a big thank you to all involved in working through the issues.
Some minor changes to the frame header (E.g. coding of frame size not needed for all frames)
Some minor changes to the entropy coding of some symbols and some updates to the default tables.
|Re: VP9 Bitstream finalization update||Pieter Kapsenberg||6/12/13 9:57 AM|
Paul, thanks for the update!
The code corresponding to the final spec is the experimental branch still or is it merged into the master branch?
|Re: VP9 Bitstream finalization update||Yy Chen||9/18/13 6:24 PM|
I still have question regarding to the following matter:
Since you removed the the dependency of reconstruction of neighbouring motion vectors to provide the context, can you also remove the dependency of reconstruction of neighbouring motion vectors to determine the
vp9_use_mv_hp? So far CABAC engine still needs to store the colocated motion vectors of last frame to find the best_mv(used to determed use_mv_hp) which adds a lot more complexity.
|Re: VP9 Bitstream finalization update||Yu Liu||9/24/13 4:38 PM|
In order to decode one bit "vp9_use_mv_hp", the whole collocated motion vector of last frame need to reconstructed and buffered, which is too unreasonable!
|Re: VP9 Bitstream finalization update||Yaowu Xu||9/24/13 5:35 PM|
We recognize that there is a dependency resulting from use_mv_hp(). Had the issue been brought up a few months earlier, we would have addressed it completely. Unfortunately, we have now passed the window of opportunity to address issue like this.
|Re: VP9 Bitstream finalization update||Yu Liu||9/24/13 11:22 PM|
I have another question related to superblock with partition shape PARTITION_HORZ and PARTITION_VERT cases:
For superblock size is > 8x8 and shape with PARTITION_HORZ/PARTITION_VERT case, each partition has its own mode_info, where mode_info contains the following information: segment_id, mb_skip_coeff, ref_frame, txfm_size. In this case, the superblock with PARTITION_HORZ or PARTITION_VERT may have different mode_info for these two partitions. The problem is whether VP9 allow two partitions’ ref_frame belong to different prediction type (one is INTRA, another is INTER).
For example, suppose that a superblock is 16x16 MB and there are two horizontal partitions (16x8s) inside this MB. Does VP9 specification allow these two partitions' ref_frame belong to different prediction types (One is INTRA_FRAME, another is LAST/GOLDEN/ALT_FRAME) ?
The reason I am asking this question is because doing both INTRA prediction and INTER prediction inside one MB may introduce some complexity or other issue.
From VP9 decoder reference software, I didn't see any constrain about this. But I want to know whether VP9 encoder or specification allows to do this?
|Re: VP9 Bitstream finalization update||Jingning Han||9/25/13 12:56 PM|
Yes, it is allowed. With respect to the bit-stream, the coding parameters of the two partitions (of superblock >= 16x16) can be determined independently.
|Re: VP9 Bitstream finalization update||zhijun lei||9/26/13 9:16 AM|
what is the minimum allow partition granauality for mixing INTRA and INTER type? for example, at 8x4 or 4x4 level, can we have INTRA type for one 8x4 block and INTER type for the other 8x4 block.
|Re: VP9 Bitstream finalization update||Jingning Han||9/26/13 11:23 AM|
Blocks of 8x16/16x8/8x8 inside a 16x16 partition can be coded as a mixture of intra and inter. When it goes down to 4x8/8x4/4x4 level, all blocks inside an 8x8 partition will share the same intra- or inter-frame prediction.
|Re: VP9 Bitstream finalization update||Ralph Giles||10/16/13 12:09 PM|
On 2013-05-10 4:16 PM, Lou Quillio wrote:We would like to push back on this. Extending 'video/webm' to include
new codecs introduces uncertainty as to whether it will play, and
changes the behaviour of canPlayType(). Let's not make webm even more
unpopular by having files randomly fail to play.
Instead, I propose:
video/webm remains vp8 + (optional) vorbis only, with the 'webm' ebml
doctype and filename extension.
video/webm is any combination of vp8, vp9, opus and vorbis, with 'webm2'
as the ebml doctype and filename extension.
This allows encoders to use vp9 when the superiour compression is
warranted, but also record vp8+opus streams from WebRTC without
transcoding, without confusing older players about what the 3-year-old
webm format means, and whether they can play it.
A webm file is thus a webm2 file, but not vice versa. Encoders using
only the vp8 and vorbis codecs SHOULD use the 'webm' doctype for
The HTML5 media element's canPlayType call SHOULD return 'probably' for
'video/webm' and 'audio/webm' in user-agents which support the v1
format, since there is no codec variation.
|Re: VP9 Bitstream finalization update||Ralph Giles||10/16/13 12:12 PM|
On 2013-10-16 12:09 PM, Ralph Giles wrote:Sorry, that should be 'video/webm2' for any combination of vp8, vp9,
vorbis, and opus.
|Re: VP9 Bitstream finalization update||Silvia Pfeiffer||10/16/13 1:38 PM|
FWIW I agree - mangling new codecs into an old mime type only leads to problems.
|Re: VP9 Bitstream finalization update||Silvia Pfeiffer||10/16/13 1:42 PM|
Also, FWIW, you should consider requiring a codecs partisan on the video/webm2 mime type so you can in future as other codecs to webm2 and also to distinguish between vp8 and vp9.
|Re: VP9 Bitstream finalization update||Philip Jägenstedt||10/17/13 12:17 AM|
Do you mean just a validity requirement to include the codecs
parameter, or to also require user agents to always reject
'video/webm2' with no codecs parameter? Doing both seems sensible, but
just a validity requirement doesn't seem useful.
|Re: VP9 Bitstream finalization update||Ralph Giles||10/17/13 10:47 AM|
On 2013-10-17 12:17 AM, Philip J�genstedt wrote:Wouldn't rejecting 'video/webm2' without a codecs parameter would
contradict what the html spec says about canPlayType?
'... must return "probably" if the user agent is confident that the type
represents a media resource that it can render if used in with this
audio or video element; and it must return "maybe" otherwise.'
I guess "maybe" is also consistent with rejecting the source, but seems
disingenuous when we know from the mime type alone we're going to block
playback. Can you suggest a rewording of the spec which would allow this?
|Re: VP9 Bitstream finalization update||Philip Jägenstedt||10/18/13 11:22 AM|
On Thu, Oct 17, 2013 at 7:47 PM, Ralph Giles <gi...@mozilla.com> wrote:
> On 2013-10-17 12:17 AM, Philip Jägenstedt wrote:
>> Do you mean just a validity requirement to include the codecs
>> parameter, or to also require user agents to always reject
>> 'video/webm2' with no codecs parameter? Doing both seems sensible, but
>> just a validity requirement doesn't seem useful.
> Wouldn't rejecting 'video/webm2' without a codecs parameter would
> contradict what the html spec says about canPlayType?
> '... must return "probably" if the user agent is confident that the type
> represents a media resource that it can render if used in with this
> audio or video element; and it must return "maybe" otherwise.'
> I guess "maybe" is also consistent with rejecting the source, but seems
> disingenuous when we know from the mime type alone we're going to block
> playback. Can you suggest a rewording of the spec which would allow this?
I read "the type" as the whole type including parameters, so I don't
know that there's a problem here. However, if it's a good idea to say
"no" to "video/webm2" and the spec needs some change, I'm sure Hixie
will change it for us.
The hard question is whether or not that idea is good... On reflection
I'm not so sure that it would help. webm2 would have open-ended codec
support (ish) like MPEG-4, but no one says "no" to "video/mp4". Also,
if authors got annoyed at having to specify the codecs, they'll just
always use "video/webm2; codecs=vorbis", since there's no
cross-checking (in spec or implementations I know of) that the codecs
|Re: VP9 Bitstream finalization update||Frank Galligan||10/18/13 3:46 PM|
What are the issues you are trying to fix? It looks like only developers that are using canPlayType("video/webm2").
There are a few of issues with webm2:
1. Some creation tools will get this wrong and write out .webm files that contain opus and/or vp9. (Or they just haven't been updated yet.)
2. Some developers will get it wrong and write canPlayType("video/webm") trying to play a file with vp9 or opus.
3. Probably the biggest issue, UA's that support opus or vp9 but not both (assuming vorbis and vp8 support) will return the empty string for canPlayType("video/webm2").
4. There are already files with opus and/or vp9 out in the wild, so in reality UA's will need to still handle opus and vp9 in webm files.
5. Branding... maybe?
MSE handles this by forcing the codec parameter. I don't think we can change this without breaking sites that only specify canPlayType("video/webm"). But this will only return "maybe". Developers that want to have more confidence that their videos will play should be specifying the codec parameters to return "probably", or the empty string for UAs that don't support opus or vp9.
Also I'm not sure how big of a deal this is as developers who are serving opus and vp9 webm files will hear from their users that the video doesn't play in UA X. Then the developers will investigate and learn that they should specify the codec parameter... I think.
|Re: VP9 Bitstream finalization update||Basil Mohamed Gohar||10/18/13 1:02 PM|
On 10/18/2013 02:22 PM, Philip J�genstedt wrote:
> I read "the type" as the whole type including parameters, so I don'tI'll be honest, I think this is important, but I'm not getting all the
cases. Can someone provide something either as a matrix of
possibilities or uses cases of how this could work and what problem this
discussion is aiming to solve?
For example, how current browsers would behave with existing WebM, WebM
with new codecs, and how introducing the new content type spec would
solve a problem, and what that problem is? Even a bulleted list of
these points with explanation would be useful. Maybe it's just because
it's late in the day at the end of the week, but I'm having a hard time
keeping track of what problem we're trying to solve.
(The "everyone already handles video/mp4 very loosely" case above made
me give consideration to the idea of just not using another content
type, but I didn't have a reason, hence my request for clarification.)
|Re: VP9 Bitstream finalization update||Silvia Pfeiffer||10/22/13 4:31 PM|
On Thu, Oct 17, 2013 at 6:17 PM, Philip Jägenstedt <phi...@opera.com> wrote:Yes, I think without codecs, it should be rejected. That should make
it future-proof, IMHO.
> On Wed, Oct 16, 2013 at 10:42 PM, Silvia Pfeiffer
> <silviap...@gmail.com> wrote:
>> Also, FWIW, you should consider requiring a codecs partisan on the
>> video/webm2 mime type so you can in future as other codecs to webm2 and also
>> to distinguish between vp8 and vp9.
>> On 17 Oct 2013 07:38, "Silvia Pfeiffer" <silviap...@gmail.com> wrote:
>>> FWIW I agree - mangling new codecs into an old mime type only leads to
>>> On 17 Oct 2013 06:12, "Ralph Giles" <gi...@mozilla.com> wrote:
>>>> On 2013-10-16 12:09 PM, Ralph Giles wrote:
>>>> > video/webm is any combination of vp8, vp9, opus and vorbis, with
>>>> > 'webm2'
>>>> > as the ebml doctype and filename extension.
>>>> Sorry, that should be 'video/webm2' for any combination of vp8, vp9,
>>>> vorbis, and opus.
|Re: VP9 Bitstream finalization update||Silvia Pfeiffer||10/22/13 4:39 PM|
On Sat, Oct 19, 2013 at 5:22 AM, Philip Jägenstedt <phi...@opera.com> wrote:I read it like you do. I don't think the spec needs a change.
Which is a problem: your browser generally supports the mp4 container,
so it returns "maybe", but then it doesn't play. A clear "no" would be
If you want a "probably", you have to give it a codecs parameter, too, e.g.
I think we only have "maybe" because of this inherent problem of
understanding the container but not knowing the codec, but "maybe" is
the worst answer a JS dev can get.
Yes, there is a way for devs to trick the browser. But if the
specified codec is not actually the one used in the file, that's the
|Re: VP9 Bitstream finalization update||Silvia Pfeiffer||10/22/13 5:05 PM|
Here's the problem:
-> returns "probably" in Firefox, which is perfectly fine, because
"video/webm" was originally defined to only contain vorbis and vp8
-> returns "maybe" in Chrome, which follows more
http://www.webmproject.org/docs/container/ (which I'm pretty sure was
changed from originally recommending to use "probably")
v.canPlayType('video/webm; codecs="vp8, vorbis"');
-> returns "probably" in both.
So, if you use v.canPlayType("video/webm") on a file containing opus
or VP9 in future, you will get "probably" even if your browser
(current version of Firefox) doesn't support these codecs.
Counter-argument could be to ignore legacy because there will be very
little of it. This may be particularly interesting because there isn't
actually a registration of the video/webm mime type at IANA yet, FAICT
On Sat, Oct 19, 2013 at 7:02 AM, Basil Mohamed Gohar
|Re: VP9 Bitstream finalization update||Frank Galligan||10/22/13 5:48 PM|
On Tue, Oct 22, 2013 at 5:05 PM, Silvia Pfeiffer <silviap...@gmail.com> wrote:
I don't think it is perfectly fine. We always talked about adding other codecs to webm in the future.
No, this was always defined to return "maybe" without a codec parameter (or wrong codec parameter like "vp8.tree")
Here is a link to the initial check-in to chrome. https://codereview.chromium.org/2093007
Does anyone know of any other browsers that behave like this?
|Re: VP9 Bitstream finalization update||Silvia Pfeiffer||10/23/13 2:39 AM|
On Wed, Oct 23, 2013 at 11:48 AM, Frank Galligan <fgal...@google.com> wrote:Right, I take it back - my memory must be clouded. I just found that
it was always supposed to return "maybe":
So, this is likely a bug in Firefox.
I just checked Opera 12 and it says "maybe".
I don't think there are other browsers that support webm.
I guess if Firefox was to move to "maybe" for "video/webm", you could
support vp9 through the codecs parameter in the same mime type.
BTW: I'd take this as an opportunity to specify the mime type and
register with IANA (you only need to write a small RFC for it).