Using Direct Show To Capture Camera Video, Stream Across Network, and Preview?
Thales
I want to take data from a camera, capture it, send it across a
network (tcp/ip), capture it on a remote computer, and then preview or
display it on the remote computer. In other words, I want real time
(or close to real time) camera feed from one computer to another.
This will include both video and audio data. The format of the data
doesn't matter, so long as I can get the data across the network
smoothly. Resolution and frame rate need to be reasonable. I'm
thinking 800x600 res, and 30 fps would be good, but that's just
ballpark.
I was going to use DirectShow, but I'm not sure exactly how to do
this.
My outline so far is the following:
1> Create capture graph builder using ICaptureGraphBuilder2
(g_Cpature), and IGraphBuilder (g_pGraph), and add filter with
g_pCapture->SetFiltergraph(g_pGraph);
2> Enumerate video input devices ( find camera), using ICreateDevEnum,
and IMoniker, return IBaseFilter from device, pSrcFilter
3> Add filter to IGraphBuilder, g_pGraph->AddFilter(pSrcFilter,
L"Video Capture");
Then somehow I'd go to a buffer, an IMediaSample?
Some how get the data out to tcp/ip across the network to the remote
computer.
At remote computer, somehow get data into a buffer at other end.
With ICaptureGraphBuilder? And use RenderStream() preview to render
to a window?
That's just a fuzzy outline, not sure how much on track I am here.
Can anyone offer details, or ideas on how to best do this?
Any feedback appreciated.
Thanks,
...John
Alessandro Angeli
> 1> Create capture graph builder using
You do not need a CaptureGraphBuilder.
> Then somehow I'd go to a buffer, an IMediaSample?
This is a FAQ here: the best way to do it is to write a
custom sink (a type of renderer that doesn't actually render
the samples) otherwise you can use a trans-in-place filter
followed by a NullRenderer. You can use the Dump sample as a
base to write your sink. You can use the SampleGrabber
sample (from the old version of the SDK bundled with the
DirectX SDK 9.0b) as trans-in-place filter or you can use
the pre-built stock SampleGrabber filter configured in
sample mode or you can write it from scratch (see the Gargle
sample).
> Some how get the data out to tcp/ip across the network to
> the remote computer.
That's up to you. You get the data in your
IMemInputPin::Receive() method (or the SampleGrabber's
callback) and you can do with it as you please, but what you
do is completely outside DirectShow.
> At remote computer, somehow get data into a buffer at
> other end.
You need to write a push source that receives the data from
the network and pushes it downstream. See the Push sample
and the CSource/CSourceStream BaseClasses.
Since you have more than 1 stream, synchronization will be
easier if you have a single push source with multiple output
pins.
You can use the MultiMediaStreaming objects (see the
appendix to the DirectShow docs), which are built on top of
DirectShow, to at least extract the samples from the capture
graph, but I am not sure it will be easier than writing a
sink or using a grabber.
--
// Alessandro Angeli
// MVP :: DirectShow / MediaFoundation
// mvpnews at riseoftheants dot com
// http://www.riseoftheants.com/mmx/faq.htm
Tim Roberts
>From: "Thales"
>
>> At remote computer, somehow get data into a buffer at
>> other end.
>
>You need to write a push source that receives the data from
>the network and pushes it downstream. See the Push sample
>and the CSource/CSourceStream BaseClasses.
I've been surprised that I haven't found a simple sample "network source"
filter on the web. I would have thought this was a rather common need.
--
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.
Michel Roujansky - DirectShow Consultant and Trainer
source"
> filter on the web. I would have thought this was a rather common need.
In one of the older SDKs (2002), there was an example called
DSNetwork, with sink and source filters for multicast MPEG2 transport
stream on tcp/ip. These can be easily modified for program stream or
other protocols.
Alessandro Angeli
> I've been surprised that I haven't found a simple sample
> "network source" filter on the web. I would have thought
> this was a rather common need.
I bet it is but all the people who asked never thought about
sharing their code (or maybe they gave up because it was too
complex a task, who knows). The old sample Michel is
referring to was not all that useful and most likely no
sample can be, since there is no standard way to transmit
stream across a network (too many different formats,
protocols and requirements).
However, I forgot to mention that Windows contains a working
set of network filters, that is the WindowsMedia runtime's
implementation of an MMS-over-HTTP client and server. So, if
the OP can live with a TCP-based protocol and his stream
formats are AVI-compatible (most are), he doesn't need to
write a push source for the receiver at all, since he can
just use the WMASFReader filter to open the sender's URL. On
the sending (server) side, he can use a network sink with
the WMASFWriter (as explained later) or extract the samples
from the graph as already explained and then send them over
the network using the WMWriter object directly (still
requires a network sink).
To use a network sink with the WMASFWriter filter:
1. create the WMASFWriter
2. add it to the graph
3. configure it, including setting the output file with
IFileSinkFilter::SetFileName()
4. get IWMWriterAdvanced2 through IServiceProvider
5. use IWMWriterAdvanced2::GetSinkCount() and
IWMWriterAdvanced2::RemoveSink() to remove all sinks
6. create a network sink with
wmvcore.dll!WMCreateWriterNetworkSink()
7. configure the sink
8. add it to the writer using IWMWriterAdvanced2::AddSink()
9. run the graph
You will end up with an empty file which you can delete
after the graph stopped/destroyed.
Thales
wrote:
> From: "Thales"
> --
> // Alessandro Angeli
> // MVP :: DirectShow / MediaFoundation
> // mvpnews at riseoftheants dot com
> //http://www.riseoftheants.com/mmx/faq.htm
Somehow my other post did not fully post, just the header? Anyway, I
wanted to thank you for the thorough answer!
Regards,
...John
Daniel S
I have done a similiar filter setup. I started with the Null renderer
and the push source samples which worked out to be a good solution for
me. I have also used JRTPLib (http://research.edm.uhasselt.be/~jori/
page/index.php?n=CS.Jrtplib) for the network communication. I use one
render filter for each stream (audio and video) and one receiver
filter with output pins for the different streams.
Unfortunately I can not share the code since it was written for the
company I work for.
Thales
wrote:
> From: "Tim Roberts"
>
> > I've been surprised that I haven't found a simple sample
> > "network source" filter on the web. I would have thought
> > this was a rather common need.
>
> I bet it is but all the people who asked never thought about
> sharing their code (or maybe they gave up because it was too
> complex a task, who knows). The old sample Michel is
> referring to was not all that useful and most likely no
> sample can be, since there is no standard way to transmit
> stream across a network (too many different formats,
> protocols and requirements).
>
> However, I forgot to mention that Windows contains a working
> set of network filters, that is the WindowsMedia runtime's
> implementation of an MMS-over-HTTP client and server. So, if
> the OP can live with a TCP-based protocol and his stream
> formats are AVI-compatible (most are), he doesn't need to
> write a push source for the receiver at all, since he can
> just use the WMASFReader filter to open the sender's URL. On
> the sending (server) side, he can use a network sink with
> the WMASFWriter (as explained later) or extract the samples
> from the graph as already explained and then send them over
> the network using the WMWriter object directly (still
> requires a network sink).
Do you require the WMFSDK for this? I looked for it on MS's web
page and could not find it. The only version I found on MS's website
was for 64 bit Windows, nothing for 32 bit windows.
Thanks for any feedback,
...John
Alessandro Angeli
> Do you require the WMFSDK for this? I looked for it on
> MS's web page and could not find it. The only version I
> found on MS's website was for 64 bit Windows, nothing for
> 32 bit windows.
http://msdn2.microsoft.com/en-us/windowsmedia/bb190309.aspx
Thales
wrote:
> From: "Thales"
>
> > Do you require the WMFSDK for this? I looked for it on
> > MS's web page and could not find it. The only version I
> > found on MS's website was for 64 bit Windows, nothing for
> > 32 bit windows.
>
> http://msdn2.microsoft.com/en-us/windowsmedia/bb190309.aspx
Thanks much, Alessandro. That worked. : )
I have another question.
Are there any ASF compression/decompression filters, so that I can
use a sink, ala the Dump sample, to output the data the way I want?
Or, can I get enough control over the WM ASF Writer Filter to be able
to output the data the way I want, e.g. to a circular buffer? I'd
want to write to a circular buffer, read from it to tcp/ip the data
across a network, write to another circular buffer, and then read from
it using WM ASF Reader Filter.
I may be wrong, but it seems as if packaging the compressor with
the sink (ASFWriter) and the decompressor with the source (ASFReader)
make it more difficult to manipulate the data.
Alessandro Angeli
> Are there any ASF compression/decompression filters, so
ASF is not a compression format but a container file format.
> that I can use a sink, ala the Dump sample, to output the
> data the way I want? Or, can I get enough control over
> the WM ASF Writer Filter to be able to output the data
> the way I want, e.g. to a circular buffer? I'd want to
> write to a circular buffer, read from it to tcp/ip the
> data across a network, write to another circular buffer,
> and then read from it using WM ASF Reader Filter.
>
> I may be wrong, but it seems as if packaging the
> compressor with the sink (ASFWriter) and the decompressor
> with the source (ASFReader) make it more difficult to
> manipulate the data.
ASF is a container file format just like AVI and it can
contain almost any kind of elementary stream (e.g. WMV or
MPEG-4 video and WMA or MP3 audio).
The WindowsMedia codecs are packaged as DMOs and can be used
by themselves:
http://msdn2.microsoft.com/en-us/library/aa391564.aspx
The WMASFReader filter in DirectShow is a light wrapper for
the WMReader object in the WMF runtime. The WMReader object
can read an ASF stream from a file, an MMS URL or from any
IStream object and parse it thus extracting the elementary
streams. It can output the compressed data or optionally
decompress it using a suitable DMO or WMM decompressor (by
default, it decompresses the data). The WMASFReader allows
you to open a file or MMS URL in its internal WMReader
object via IFileSourceFilter::Load() and always configures
the WMReader to output compressed samples (which means that
also DirectShow filter can be used in the graph to
decompress the data as well as the DMOs or WMM decoders).
The WMASFWriter filter is a wrapper for the WMWriter object
which can mux elementary streams into an ASF file and write
the ASF stream to any IWMWriterSink object. The WMF runtime
provides a file, a MMS-over-HTTP pull network and an
MMS-over-HTTP push network sink and the file sink is the
default one. Optionally, the WMWriter can compress the data
using the WMx DMO encoders (which is the default behavior).
The WMASFWriter allows you to output to a file sink via
IFileSinkFilter::SetFileName(), to configure its internal
WMWriter with a WMProfile object and to choose whether to
compress or not the input samples.
If a TCP-based network protocol is OK for your needs, you
can use the WMF built-in MMS-over-HTTP implementation. The
WMASFReader can open an MMS-over-HTTP URL by itself and you
can force the WMASFWriter to use an MMS-over-HTTP pull
network sink instead of a file one:
http://groups.google.com/group/microsoft.public.win32.programmer.directx.video/msg/6775456e768652e9
If MMS-over-HTTP is not adeguate, the WMASFReader supports
all of the MMS family of protocols, like MMS-over-TCP
(MMST), MMS-over-UDP (MMSU), MMS-over-UDP-multicast and
MMS-over-RTP/RTSP (and I think the legacy MSBD protocol).
You would need to implement the one you like yourself in a
custom IWMWriterSink. Unfortunately they are undocumented,
but some of them have been reverse-engineered (see
http://sdp.ppona.com/ and VLC's source code
http://www.videolan.org/vlc/features.html) or you can
license the implementation from MS
(http://www.microsoft.com/windows/windowsmedia/licensing/default.mspx).
If you want to implement your own protocol, you can
implement it in a custom sink on the writer part. Getting
the WMASFReader to use it is going to be harder. You can
implement your custom receiver in an IStream and try to get
the WMASFReader to read from it:
http://groups.google.com/group/microsoft.public.windowsmedia.sdk/msg/2aa9d7efb0c290a8
If that fails, you have 2 options:
1. hook wmvcore.dll!WMCreateReader() to intercept the
creation of the internal WMReader and, when it is created,
hook its IWMReader::Open() to intercept the URL so that,
when it is called, you can call
IWMReaderAdvanced::OpenStream() instead
2. use Geraint's ASF parser (www.gdcl.co.uk) which uses the
WMReader internally just like the WMASFReader but it reads
from an upstream async reader filter, so you can implement
your custom receiver as an async reader. You can write your
aync reader from scratch or you can very quickly base it on
the base class in the Async sample:
http://groups.google.com/group/microsoft.public.win32.programmer.directx.video/msg/01dde303fa3088c5
You can also use the stock URLReader filter as async reader
and write a custom protocol implementation for it. The
URLReader uses network monikers, so you can write a network
moniker implementation or let it use the generic network
moniker which wraps installed Internet Explorer's pluggable
protocol handlers, so that you can write a simpler PPH.
Notice that the WMReader works in pull mode (async mode in
DirectShow terms) so it will not work well with a push-mode
protocol (the IStream and IAyncReader interfaces are both
pull-mode). In you need a push-mode ASF parser, then you
will need to write your own based on the ASF syntax
specification:
http://www.microsoft.com/windows/windowsmedia/forpros/format/asfspec.aspx