webm muxer directshow filter requirements

[Roman]

Hi All!

I am trying to implement input data filter for webm muxered data. I am using live muxer mode. Is there filter/pin requirements for webm muxer?

I built simple graph:

[video capture]->[inftee pin1]->[vp8 encoder]->[webm muxer (live mode)]->[my filter]

                  [inftee pin2]->[Video render]

When I did render without [my filter] everything is fine. When I connect [my filter] to [webm muxer (live mode)] I have video on render ~1sec and after that static picture on render.

I implemeted IStream, IMemAllocator and IMemInputPin ifaces on my input pin and my methods IStream::write and IStream::seek are successfully called by webm muxer outpin. My IMemAllocator::GetBuffer/ReleaseBuffer  - is not called. After ~1 sec of playing I have no data (no calls IStream::seek or IStream::write). Maybe is there filter/pin requirements for webm muxer? Which methods should I overload in my pin/filter? 

Thanks.

[Matthew Heaney]

It might be the case that the graph has stalled, but there are no
samples available.

Try this: get rid of the inftee filter, and attach the video renderer
to the preview pin of the vp8 encoder instead.

> --
> You received this message because you are subscribed to the Google Groups
> "WebM Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to webm-discuss...@webmproject.org.
> To post to this group, send email to webm-d...@webmproject.org.
> Visit this group at
> http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
> For more options, visit
> https://groups.google.com/a/webmproject.org/groups/opt_out.
>
>

[Roman]

Thank you so much for your reply!

I tried this preview pin from vp8 encoder filter. It not get positive results. I forgot to write about vp8 encoder settings for real video, I set next settings:

hr = m_vp8encoder->SetDeadline(kDeadlineRealtime);

hr = m_vp8encoder->SetPassMode(kPassModeOnePass);

hr = m_vp8encoder->SetThreadCount(2);

hr = m_vp8encoder->SetCPUUsed(4);

hr = m_vp8encoder->SetEndUsage(kEndUsageCBR);

hr = m_vp8encoder->SetTargetBitrate(1024);

hr = m_vp8encoder->SetUndershootPct(95);

hr = m_vp8encoder->SetDecoderBufferSize(6000);

hr = m_vp8encoder->SetDecoderBufferInitialSize(4000);

hr = m_vp8encoder->SetDecoderBufferOptimalSize(5000);

hr = m_vp8encoder->SetKeyframeMaxInterval(999999);

hr = m_vp8encoder->SetMinQuantizer(4);

hr = m_vp8encoder->SetMaxQuantizer(56);

I found them on - http://www.webmproject.org/docs/encoder-parameters/ . All hr values are S_OK. I set it when graph in stop state. 

I also removed my settings of live muxer mode before all pin connection (graph part after capture) and as a result I have live video on render (regadless on selected pin for render vp8encoder/inftee). But after ~1-10 min playback I have the same issue - static picture on render and my methods IStrem::Write/Seek do not call. I observe this issue only when I did connect my sink filter to webm muxer. I drop any frame in my sink filter and I return correct values for IStrem::Write/Seek. Also I did simple dump muxed data in my filter, in my dump file video is correct. Can anybody help me to solve this issue?

[Matthew Heaney]

I think the problem is this: the muxer holds DirectShow media samples
until has enough frames to make a Matroska cluster, and then the muxer
emits the Matroska cluster, and releases all of the DirectShow media
samples.

Make sure that when you connect the upstream filter to the video inpin
of the muxer, that you specify a sufficiently large number of samples
in the allocator (say, 3 sec or more -- how many depends on the
framerate).

You can also set the VP8 encoder settings to emit a keyframe more
frequently, say every 1 sec.

This is a known problem for the muxer when in live mode. We need to
interleave samples across pins (audio and video frames must be placed
into the stream in time order), but we don't need to enqueue all
frames until we have the entire cluster.

-Matt

[Andy Shaules]

Hello,

Although I did not do a ton of experimenting with the filters, I had
awesome results and very real-time encoding.

I made a simple coder based on the easy demo coder.

I took my source filters and ran them through a sample grabber.

I took another thread and let it run at a scheduled framerate/hz, which
copied the current samplegrabber-callback's buffer, and encoded it. Same
thing with the audio, using C based coders that took byte pointer. I
responded to the audio sample grabber callback by packing audio packets
into the out-container and interlacing the video packets where video
stream time= audio stream time. The advantage I have with this type of
application is also being able to pass in 'keyframe_requested' on the
hard cuts between sources initiated by user-clicks in the gui.

The issue compounding the muxer that does not release data when it is
written out of the coders, is the hardware has no contract for which
samples are available first, or what the full delta between the first
audio time and video time is. Ive seen two or thee second deltas between
audio input and video input that force the muxer to drop and wait for
video time to be ready for first presentation so there wasn't a ton of
audio in the output befor the video snapped in. Even worse is that I
received the video packet first with typically a 5000 millisecond
stream-time mark, and then I receive audio chunks starting at stream
time 2000. The video comes first with time-stamps far behind the audio,
which comes maybe a second after the first video... its really screwy
and also system dependent. This causes lag to be partially based on
hardware driver implementation.

Fun!

[Roman]

Thank you for reply!

There are 2 'live' modes:

1. real time mode on vp8 encoder, I set IVP8Encoder::SetDeadline(kDeadlineRealtime)

2. live mode on muxer, I set IWebmMux::SetMuxMode(kWebmMuxModeLive)

If I set point 1 I have the issue - static picture after ~1-10 min of start. point 2 not give this effect, everything is fine with point 2 (it does not matter on/off this feature). I can not say "upstream filter to the video inpin of the muxer" because it is vp8 encoder (only if rewrite this vp8 part), I set allocator properties after webm muxer on my input pin filter, I set 128 buffers by 128 kbytes at 720x480@30 fps.

I also set :

m_vp8encoder->SetKeyframeMode(kKeyframeModeDisabled);

m_vp8encoder->SetKeyframeMinInterval(30);

m_vp8encoder->SetKeyframeMaxInterval(60);

but issue is remained.

It seems I have to debug vp8encoder state machine/synchro part, because without point 1 everything is fine. May be there is some bug in this mode.

Thanks.

[Roman]

Thanks for reply!

Hi. Yes, may be there is bug in driver, but I think encoder/muxer should workaround these situations. By the way, everything is fine without my filter in graph.

Thanks.