Message Pump for a Graph with a NullRenderer?
cyrus
When constructing a graph with a video renderer, you need a window
dispatching mechanism to keep the graph going. (like mentioned in this
thread:
http://groups.google.com/group/microsoft.public.win32.programmer.directx.video/browse_thread/thread/d956fa8c31695b23/af4a501b99cc4f07)
What about a graph with a NullRenderer? There is no video window
involved, so how the graph messages are going to be dispatched and
handled?
In my case, I am using WebcamCaptureSource-->SampleGrabber--
>NullRenderer. Do I need a message pump to keep the graph functioning,
or I just do IMediaControl::Run to start the graph and
IMediaControl::Stop to stop the graph whenever I want?
Is it true that IMediaControl::Run will spawn a new thread to run
the graph while the main thread will return to take care of the rest
of the code?
Thanks for any suggestions/information.
Cyrus
Computing Science Department
University of Alberta
Chris P.
> When constructing a graph with a video renderer, you need a window
> dispatching mechanism to keep the graph going. (like mentioned in this
> thread:
> http://groups.google.com/group/microsoft.public.win32.programmer.directx.video/browse_thread/thread/d956fa8c31695b23/af4a501b99cc4f07)
>
> What about a graph with a NullRenderer? There is no video window
> involved, so how the graph messages are going to be dispatched and
> handled?
>
> In my case, I am using WebcamCaptureSource-->SampleGrabber--
>>NullRenderer. Do I need a message pump to keep the graph functioning,
> or I just do IMediaControl::Run to start the graph and
> IMediaControl::Stop to stop the graph whenever I want?
>
> Is it true that IMediaControl::Run will spawn a new thread to run
> the graph while the main thread will return to take care of the rest
> of the code?
The thread that creates the graph must dispatch messages. If the graph is
created from the main GUI thread then you already have this provided you
don't block this thread. If you create the graph on an alternate thread
than insert something similar to the following after you create the graph.
do {
MSG msg;
if (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
DispatchMessage(&msg);
} while (WaitForSingleObject(m_heventObjectClose, 10)!=WAIT_OBJECT_0);
The thread will stay alive until you signal the event m_heventObjectClose,
after which you can destroy the graph. You should also make sure you
destroy and create the graph on the same thread, I had some really weird
bugs until I did that.
--
http://www.chrisnet.net/code.htm
[MS MVP for DirectShow / MediaFoundation]
Alessandro Angeli
> do {
> MSG msg;
> if (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
> DispatchMessage(&msg);
> } while (WaitForSingleObject(m_heventObjectClose,
> 10)!=WAIT_OBJECT_0);
MsgWaitForMultipleObjects[Ex]() would be a better choice
than WaitForSingleObject(), since 10ms can be a long delay
for messages.
wxutil.cpp in the BaseClasses contains a
WaitDispatchingMessages() function that wraps
MsgWaitForMultipleObjects().
--
// Alessandro Angeli
// MVP :: DirectShow / MediaFoundation
// mvpnews at riseoftheants dot com
// http://www.riseoftheants.com/mmx/faq.htm
cyrus
> Thethreadthat creates the graph must dispatch messages. If the graph is
> created from the main GUI threadthen you already have this provided you
> don't block this thread. If you create the graph on an alternate thread
> than insert something similar to the following after you create the graph.
>
> do {
> MSG msg;
> if (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
> DispatchMessage(&msg);
> } while (WaitForSingleObject(m_heventObjectClose, 10)!=WAIT_OBJECT_0);
>
Thanks for clarifying on this!
In my case, this graph is an internal graph of a capture source
filter.
(some previous discussion here:
http://groups.google.com/group/microsoft.public.win32.programmer.directx.video/browse_thread/thread/ce5535e717850bab/f4095a6485c53ccd)
So the main thread is the wrapping capture source filter; that means I
have to manually spawn a new thread to take care of creating, pumping,
and destroying the internal graph?
I thought IMediaControl::Run actually spawns a new thread for running
the graph...
> Thethreadwill stay alive until you signal the event m_heventObjectClose,
> after which you can destroy the graph. You should also make sure you
> destroy and create the graph on the samethread, I had some really weird
> bugs until I did that.
>
> --http://www.chrisnet.net/code.htm
cyrus
> created from the main GUI thread then you already have this provided you
> don't block this thread. If you create the graph on an alternate thread
> than insert something similar to the following after you create the graph.
>
> do {
> MSG msg;
> if (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
> DispatchMessage(&msg);
> } while (WaitForSingleObject(m_heventObjectClose, 10)!=WAIT_OBJECT_0);
>
Thanks for clarifying!
In my case, the graph with the null renderer is an internal graph to a
capture source filter (previous discussion here:
http://groups.google.com/group/microsoft.public.win32.programmer.directx.video/browse_thread/thread/ce5535e717850bab/f4095a6485c53ccd)
so the main thread is working with the wrapper source filter; does
this mean I have to manually spawn a new thread to take care of
internal graph stuff (creating, message dispatching, and destroying)?
where is the proper place to spawn the internal graph thread?
(CSourceStream::OnThreadStar is what I do IMediaConbtrol::Run on the
internal graph now). and I once thought that IMediaControl::Run
actually spawns a new thread automatically, which I guess is not quite
right...
sorry for those questions as I'm now still learning baby steps with
DShow and COM.
Cyrus
Computing Science Department
University of Alberta
> The thread will stay alive until you signal the event m_heventObjectClose,
> after which you can destroy the graph. You should also make sure you
> destroy and create the graph on the same thread, I had some really weird
> bugs until I did that.
>
> --http://www.chrisnet.net/code.htm
Chris P.
> In my case, the graph with the null renderer is an internal graph to a
> capture source filter (previous discussion here:
> http://groups.google.com/group/microsoft.public.win32.programmer.directx.video/browse_thread/thread/ce5535e717850bab/f4095a6485c53ccd)
>
> so the main thread is working with the wrapper source filter; does
> this mean I have to manually spawn a new thread to take care of
> internal graph stuff (creating, message dispatching, and destroying)?
> where is the proper place to spawn the internal graph thread?
> (CSourceStream::OnThreadStar is what I do IMediaConbtrol::Run on the
> internal graph now). and I once thought that IMediaControl::Run
> actually spawns a new thread automatically, which I guess is not quite
> right...
>
> sorry for those questions as I'm now still learning baby steps with
> DShow and COM.
No, you could theoretically create both graphs from the same thread
provided that your main thread continues to pump messages. Just whatever
thread it is must pump messages and should stay alive for the life of the
graph.
--
cyrus
late reply...
> No, you could theoretically create both graphs from the same thread
> provided that your main thread continues to pump messages. Just whatever
> thread it is must pump messages and should stay alive for the life of the
> graph.
I've made it work...
A detailed description of my problem is stated here:
http://groups.google.com/group/microsoft.public.win32.programmer.directx.video/browse_thread/thread/60717658ae5d440d#
In short, I'm having trouble with the skype integration for my virtual
webcam source filter, as every time it freezes at some point; but with
GraphEdit or MSN it has no problem.
I start to think that it might be because I didn't do the message
dispatching properly, as someone else has ran into graph frozen
condition before (http://groups.google.com/group/
microsoft.public.win32.programmer.directx.video/browse_thread/thread/
d956fa8c31695b23/af4a501b99cc4f07)
But later I found out that, the time stamp of the sample is not sent
properly in my FillBuffer function. I didn't do IMediaSample::SetTime.
Seems Skype is not happy dealing with media sample without time
stamps.... Some one might have some better and thorough explanation
on this.... After properly setting it, the freezing problem just went
away~!~
Back to the Message Pump issue; so it seems the internal graph somehow
is still kept running. I guess CSourceStream::DoBufferProcessingLoop
and CSourceStream::ThreadProc already acts the message pump
responsibility, for both the internal graph and the source filter
itself?
Some more explanation is more than welcome....
Thanks!
Chris P.
> But later I found out that, the time stamp of the sample is not sent
> properly in my FillBuffer function. I didn't do IMediaSample::SetTime.
> Seems Skype is not happy dealing with media sample without time
> stamps.... Some one might have some better and thorough explanation
> on this.... After properly setting it, the freezing problem just went
> away~!~
If you encode or render without a reference clock then the timestamps are
irrelevant. Reference clocks aren't always used on video capture as it is
assumed that the frames are being delivered at a fixed interval. Time
stamps would then be set by the encoder.
> Back to the Message Pump issue; so it seems the internal graph somehow
> is still kept running. I guess CSourceStream::DoBufferProcessingLoop
> and CSourceStream::ThreadProc already acts the message pump
> responsibility, for both the internal graph and the source filter
> itself?
DoBufferProcessingLoop does not process Window messages, it only looks for
custom commands. ThreadProc does not process Window messages either, it is
merely the host thread for running DoBufferProessingLoop. Besides, these
have nothing to do with the graph, these are merely custom threads created
inside the filter.
cyrus
> On Tue, 17 Feb 2009 17:26:36 -0800 (PST), cyrus wrote:
> > But later I found out that, the time stamp of the sample is not sent
> > properly in my FillBuffer function. I didn't do IMediaSample::SetTime.
> > Seems Skype is not happy dealing with media sample without time
> > stamps.... Some one might have some better and thorough explanation
> > on this.... After properly setting it, the freezing problem just went
> > away~!~
>
> If you encode or render without a reference clock then the timestamps are
> irrelevant. Reference clocks aren't always used on video capture as it is
> assumed that the frames are being delivered at a fixed interval. Time
> stamps would then be set by the encoder.
>
Thanks for the clarification! I don't know how Skype/MSN handles the
reference clock thing internally; so I did a small experiment.
AvgTimePerFrame is set to 0.05s for CSourceStream::m_mt;
SetTime is called in CSourceStream::FillBuffer like below:
rtNow = m_rtLastTime;
m_rtLastTime += avgFrameTime;
pms->SetTime(&rtNow, &m_rtLastTime);
pms->SetSyncPoint(TRUE);
SampleCB is running at about 15 fps, since it captures from a real
webcam.
Then, in GraphEdit:
FillBuffer/SampleCB ratio
w/o SetTime 20
w/ SetTime 1
The graph is: VCam->ColorSpaceConverter->VMR
In Skype, local video testing:
FillBuffer/SampleCB ratio
w/o SetTime 100
w/ SetTime 20
I don't how to get Skype register the graph in the ROT, so no idea
whether what encoder is thrown in here or not.
In Skype, video over internet:
FillBuffer/SampleCB ratio
w/o SetTime 60
w/ SetTime 3
In MSN, local video testing:
FillBuffer/SampleCB ratio
w/o SetTime 30
w/ SetTime 30
IMediaSample::SetTime brings down the speed of calling FillBuffer
quite a bit. And I don't think the overhead of SetTime is the reason
of the FillBuffer loop slowing down.
But, from what I've learned, the time stamps are merely indications
for the downstream renderer/encoders; they don't force the frequency
of FillBuffer being called; DoBufferProcessingLoop will keep
FillBuffer and Deliver the data.
So I am not sure what these little experiments shows...
> > Back to the Message Pump issue; so it seems the internal graph somehow
> > is still kept running. I guess CSourceStream::DoBufferProcessingLoop
> > and CSourceStream::ThreadProc already acts the message pump
> > responsibility, for both the internal graph and the source filter
> > itself?
>
> DoBufferProcessingLoop does not process Window messages, it only looks for
> custom commands. ThreadProc does not process Window messages either, it is
> merely the host thread for running DoBufferProessingLoop. Besides, these
> have nothing to do with the graph, these are merely custom threads created
> inside the filter.
>
> --http://www.chrisnet.net/code.htm
Chris P.
> IMediaSample::SetTime brings down the speed of calling FillBuffer
> quite a bit. And I don't think the overhead of SetTime is the reason
> of the FillBuffer loop slowing down.
> But, from what I've learned, the time stamps are merely indications
> for the downstream renderer/encoders; they don't force the frequency
> of FillBuffer being called; DoBufferProcessingLoop will keep
> FillBuffer and Deliver the data.
>
> So I am not sure what these little experiments shows...
The time stamps are an indication of what time span the video frame
represents. The encoder may use these times to encode the frames such as
marking one frame longer or inserting empty frame output data in the case
of a dropped sample.
Rather than creating arbitrary times on your source based on average frame
duration you should probably be passing down the sample time from the sub
graph, reindexed to start from 0 in your case as you are derived from
CSource. This is important so that the encoding graph will correctly drop
frames where needed. Your sub-graph will also need to drop frames rather
than buffering infinitely, a 2 frame buffer should be all that is required.
--
cyrus
> [MS MVP for DirectShow / MediaFoundation]
Great suggestion... for now I'm just using one frame buffer, and
FillBuffer will update the media sample just based on whatever is in
that frame buffer... lame design.
I think I will introduce a frame queue, let SampleCB push frames into
it, and FillBuffer get frames out of it; when the queue is empty, just
waiting (while looping) in the FillBuffer until new sample comes in.
Back to the Message Pump problem; I remember several weeks ago when I
first tried to write a DShow app (playback), I used something like
this:
... // build the graph, init all interfaces
IMediaControl::Run();
Sleep(100000);
IMediaControl::Stop();
return;
I didn't use any message pump; but the graph runs for 100 seconds,
without any problem. I believe this is OK until some special graph
messages comes in but no dispatching and processing module is existing
to deal with it. That, is exactly what I'm doing now...
With the souce filter + sub graph design, I still cannot see how u can
have a message dispatching for the sub graph without creating a new
thread within the source filter. I embed the sub graph actually in the
output pin class which derives from CSourceStream. I build the sub
graph in the ctor of the output pin, then IMediaControl::Run in the
OnThreadCreate; and it is not possible to throw in the message
dispatching code also in OnThreadCreate, since that will block the
ThreadProc of the output pin from running...
I do believe that my design of integrating the source filter and the
sub graph is flawed... and that might be the reason I cannot fully
appreciate the suggestions you gave in previous posts:
> No, you could theoretically create both graphs from the same thread
> provided that your main thread continues to pump messages. Just
> whatever thread it is must pump messages and should stay alive for
> the life of the graph.
Thanks for your patience and kindness!