Support of 16:9 DV Video in DShow?

[cpix]

Hi everybody,

DV video can come in 4:3 or as 16:9 video format; actually, the 16:9
format is just an anamorphic format, having the same physical
resolution as the 4:3 format (for example: 720x576 in PAL).

Displaying 16:9 DV video in DShow is easy, as long as you ue the VMR9 -
it honors the VIDEOINFOHEADER2, which hosts an aspect ratio setting.
The DV Decoder is able to set this correctly, and the VMR9 can use this
in displaying.

As far as I know, it's not possible to use the Sample Grabber filter
(for example for grabbing frames) with a 16:9-DV, since this filter
only honors the normal VIDEOINFOHEADER, which has no aspect ratio info.

My questions:

1. Where in the DV-AVI-File is the "16:9" information stored?
2. Is it possible to get this information with DShow for use with the
Sample Grabber (or does one have to write his own?)
3. Is it possible, and if, how, to write a 16:9-DV-stream to disc? I
have not found any information regarding a 16:9-flag in the doc or the
web. (I think the DV Muxer is responsable for setting the aspect ratio
- at least the output pin of this Muxer has this information. But there
is no interface for querying/setting this. Same seems to be true for
all other filters. The DV Encoder has some interfaces for setting the
format (IDVEnc::put_IFormatResolution), but nowhere a 16:9 flag or a
corresponding aspect ratio can be found...)

Well, would be great if someone could put me on the right track...

- cpix

[The March Hare [MVP]]

On 4 Jan 2006 23:43:32 -0800, cpix wrote:

Since nobody has posted, I'll take a stab at this one:

> 1. Where in the DV-AVI-File is the "16:9" information stored?

AFAIK, it's in each media sample of the DV stream (you want to look at it
before it gets through the DV splitter). You can probably find a technical
document but first try searching for VAUX on Google Groups for the
structure. For example, this post:

http://groups.google.com/group/microsoft.public.win32.programmer.directx.video/browse_thread/thread/de432e49867e285c/93db8c98d7475f49?lnk=st&q=vaux+group%3Amicrosoft.public.win32.programmer.directx.video&rnum=2#93db8c98d7475f49

> 2. Is it possible to get this information with DShow for use with the
> Sample Grabber (or does one have to write his own?)

You should be able to grab the samples from before the DV splitter using
the standard sample grabber. However, I don't use the s.g. myself.

> 3. Is it possible, and if, how, to write a 16:9-DV-stream to disc?

This is not something I have done myself, but I expect it is a question of
setting the values in the samples correctly.

--
Please read this before replying:
1. Dshow & posting help: http://tmhare.mvps.org/help.htm
2. Trim & respond inline (please don't top post or snip everything)
3. Benefit others: follow up if you are helped or you found a solution

[cpix]

- thank you very much for these hints. I try to follow them and if I
get some thing working, I'll post the solution here!

- cpix :)

[The March Hare [MVP]]

On 9 Jan 2006 09:30:22 -0800, cpix wrote:

> - thank you very much for these hints. I try to follow them and if I
> get some thing working, I'll post the solution here!

Great. I don't remember the 16:9 topic coming up before so it would be
nice to hear if you find some resolution to the issue :)

[cpix]

Ok, here's an update of what I found out:

In a DV file or stream, there is a so-called DVINFO structure; in this
structure, there is an additional structure called VAUX (Video
Auxcilary Structure), and here the 16:9 flag is hidden. Every frame
transmitted in a DV stream is accompanied by a DVINFO structure, and
3bits of this structure indicate whether it's a 4:3 or a 16:9 DV
(others bits of DVINFO seem to give information about the audio
streams, the timecode and the like).

Since (as The March Hare already pointed out) every frame has these
flags attached, it's possible to switch (among other things) aspect
ratios in the middle of a DV-stream!

This can happen for example when during a capture a camera is switched
from 4:3 to 16:9. But most programs I know of just analyze the
beginning of a DV stream and keep the aspect ratio found there for the
rest of the data.....

Now, coming to "directly" to Directshow, I found out the following
things:

1) Generally speaking, there seems to be no good 16:9 support for DV
avis in Directshow. People who are analyzing the DVINFO in AVI files do
this generally by direct analysis of the AVI structure, not by using
Directshow.

2) Specifically, the only thing which worked for me in Directshow was
the rendering of a 16:9 DV AVI through a filter graph composed of this:
AVI File -> AVI Splitter -> DV Splitter -> DV Decoder -> VMR9.

This works because up to DV Decoder, the stream transmits Video of the
DVSD type (which still has the 16:9 flag). The DV Decoder sees the 16:9
flag and can output this information into the VIDEOINFOHEADER2, which
is transmitted to the VMR.

The VMR finally uses the 16:9 aspect ratio setting to display a 16:9 DV
AVI correctly.

Note that this fails if only the standard video renderer is present,
since that renderer can make use only of the standard VIDEOINFOHEADER,
which has not aspect ratio setting.

The limited DV 16:9 support for displaying video is only possible since
the Directshow 2003 version; before that, the DV Decoder could only
output VIDEOINFOHEADER information, which has no aspect ratio setting
information.

3) This brings me to the point of grabbing a 16:9 DV stream. Since the
standard sample grabber supplied by Microsoft uses only the standard
VIDEOINFOHEADER for connections, it's not possible to grab 16:9 frames
with this sample grabber either.

(Of course, you might grab the anamorphic frame anyway, and correct for
the aspect ratio. But for this to work, the user has to select the
correct aspect ratio manually, by himself.).

For automatic detection of 16:9 DV content, the grabber sample in the
SDK docs might be of more use, since it uses VIDEOINFOHEADER2 and might
be able to understand the information the DV Decoder is giving out. But
I have not tried this.

4) Output of 16:9 DV material to AVIs seems to be impossible at first
sight, since DV Encoder as well as DV Muxer only support the standard
VIDEOINFOHEADER and, as already mentioned, there is no place to put an
aspect ration anywhere....

But: the DV Encoder has an IDVEnc::put_IFormatResolution interface
function, and this interface function seems to be able to set the
appropriate DVINFO structures!

It looks like this:

HRESULT put_IFormatResolution(
int VideoFormat,
int DVFormat,
int Resolution,
BYTE fDVInfo,
DVINFO *sDVInfo
);

Here "fDVInfo" is a "Boolean value specifying whether the sDVInfo
parameter contains a valid DVINFO structure. To set the stream format,
set this parameter to TRUE and specify the format chunk with the
sDVInfo parameter."

There is also a corresponding "get"-function.

However, at least my efforts on setting the correct parameters where
doomed. Setting and getting the Videoformat, DVFormat and Resolution
worked as expected. But reading out the DVINFO structure just after
insertion of the DV Encoder into the filtergraph gave me 0xFFFFFFFF for
all field of the DVINFO structure, which is clearly not valid data.

Setting these fields to sensible values seemed to succeed (HRESULT ==
S_OK on function call), but a subsequent read gave again the 0xFFFFFFFF
nonsense values...

This behaviour did not change when I shifted the getting/setting of the
DVINFO toward the end of the filter graph construction - even so
HRESULT indicated a setting of values without errors, the DVINFO values
were not set....

So my disappointing summary: it seems it's not possible to output a
16:9 DV avi with Directshow!?!!!

I found absolutely nothing about this in MSDN/Google Groups or Google
Search, as mentioned before..., but is this statement the final answer?
If anyone knows more I will be more than happy to follow any additional
hints/suggestions/comments!!

- cpix

[The March Hare [MVP]]

On 10 Jan 2006 05:18:27 -0800, cpix wrote:

...
> So my disappointing summary: it seems it's not possible to output a
> 16:9 DV avi with Directshow!?!!!
>
> I found absolutely nothing about this in MSDN/Google Groups or Google
> Search, as mentioned before..., but is this statement the final answer?
> If anyone knows more I will be more than happy to follow any additional
> hints/suggestions/comments!!

Thanks for the detailed summary.

Have you considered writing a transform filter that sets the DVINFO after
the DV Encoder but before the AVI MUX?

[cpix]

> Have you considered writing a transform filter that sets the
> DVINFO after the DV Encoder but before the AVI MUX?

no, not really. This would boil down to bitfield operations in an
environment that is not that well documented (namely streaming
Directshow dvsd data) - I have the impression that a simple AVI lib
for DV use should be easier to write from scratch....

I discovered in the mean time the following which I want to share: with
DV2-types of AVI files, all the video information seems to be stored in
a BITMAPINFOHEADER, with the *option* of extending this with a DVINFO
chunk.

Wiht other words: the DVINFO information might be there, but it must
not be.

Since there's nothing in the normal BITMAPINFOHEADER indicating 16:9
scaling, one might probably assume that DV2 AVI files are not suitable
for 16:9 work at all. However, I do have DV2 type AVIs from my cam
which are in 16:9....

Another funny thing adding to the mystery is that commercial DV
Directshow codecs can easily be switched to 16:9 output; analyzing the
resulting file with a hex-editor shows that indeed only a sinlge byte
(!)is changed to indicate a 16:9 movie (0xC8 goes to 0xCF for a 16:9
movie). If I only could figure out a way to tell this to the Directshow
filter graph...

Well, I will give the topic some more thoughts over the next days. Of
course, any additional information/ideas from the net community would
be very helpful!

thanks, cpix

[cpix]

Ok, here's a final summary of my findings:

1) 16:9 support of DV avi is not well documented, but it should be
possible in the Microsoft world: every frame in a DV avi stream has a
16:9 flag. Typically, this flag is coming straight from the camera. In
addition, there is normally a DVINFO structure in the avi header which
can carry 16:9 information. Presumably, this is a setting for the whole
avi file.

2) Since every frame has this flag, it's possible to switch aspect
ratios in the middle of a stream. But support of this feature varies:

2a) There are progams, which do not honor the 16:9 flag at all.
These prgs display 16:9 avis always as 4:3, with the wrong aspect
ratio. At least my version of Realplayer does it like this.

2b) Second, there are programs which obviously only read the DVINFO
structure hidden in the avi-header. These prgs load all 16:9-avis
correctly which have the correct DVINFO in the avi header, but fail to
load DV avis which switch aspect ratios in the middle of the stream.
Some video editors show this behaviour.

2c) Finally, there are prgs which read out the 16:9 information for
every frame. Actually, Graphedit in the combination with the DV
Decoder and VMR9 is such a prg. It nicely switches aspect ratios in the
middle of the stream. The older video renderers will not work. Most
newer video software which is using Directshow and the VMR9 falls into
this category.

3) Well, there seems to be no documentation available about Directshow
and writing 16:9 information into DV avis. It even seems that the
interface for setting this information in the DV Encoder is broken. At
least my tests failed to set bits correctly, with no error message
coming from the interface. I was not able to set the DV Encoder into
the 16:9 mode. In any case, the interface of the DV Encoder seems only
to be able to set these options globally, not for every frame as the DV
standard would allow.

4) This leaves 16:9 support in Directshow at an unknown, but rather
rudimentary level. If someone wants to use this feature, he has
probably really go along the path "The March Hare" has already
suggested: inserting a transform filter into the DV stream to set the
data manually. All the public code I could find addressing this issue
did not use Directshow, but turned to the option of writing the avi
file directl and inserting some magic numbers usually "stolen" from
some "valid" avi file....

This summarizes my knowledge so far. Well, if anybody has further
hints, please contribute to this thread...

Thanks all, cpix

[The March Hare [MVP]]

On 12 Jan 2006 02:34:50 -0800, cpix wrote:

> Ok, here's a final summary of my findings:
>
> 1) 16:9 support of DV avi is not well documented, but it should be
> possible in the Microsoft world: every frame in a DV avi stream has a
> 16:9 flag. Typically, this flag is coming straight from the camera. In
> addition, there is normally a DVINFO structure in the avi header which
> can carry 16:9 information. Presumably, this is a setting for the whole
> avi file.

...

Thanks again for your detailed report. This will help anyone looking into
this issue here in the future.