handling dynamic format change in copy transform filter
Satire
Hi,
My graph looks like the following:
Source --> Decoder --> Video Transform --> VMR9
I want to handle the dynamic format change that the VMR9 requests and
pass this request to the decoder upstream. For example, decoder and
transform agree to 320x240 YUY2. Similarly transform and VMR agree to
the same. When I run the graph, VMR9 asks for 512x(-240) YUY2. So I
have do the format conversion. I followed the article described in the
DShow documentation and came up with my allocator in the Video
Transform filter. I am unable to get it working. My Transform function
never gets called. The allocator GetBuffer() function gets called, but
the transform never gets called. Also, in my GetBuffer() function, if
I do not change the mediatype to the new type, the filter works (I
have to do the format conversion in the transform filter). This is all
with the copy transform filter.The inplace transform filter works
seamlessly with format changes. However the issue with inplace
transform is that the transform is really slow. Can I speed it up in
some way?
I have been struggling with this for past two days and hence the
detailed code below. Any help is very much appreciated.
Code with explanation is as follows:
// Create my own allocator - just a wrapper over CMemAllocator
class CMyAllocator : public CMemAllocator
{
friend class CVideoPostProcess;
public:
CMyAllocator(TCHAR *pName, LPUNKNOWN pUnk, HRESULT *phr,
CVideoPostProcess *pTransformFilter)
: CMemAllocator(pName, pUnk, phr)
{
m_pVPP = pTransformFilter;
}
STDMETHODIMP GetBuffer(IMediaSample **ppBuffer,
REFERENCE_TIME *pStartTime,
REFERENCE_TIME *pEndTime,
DWORD dwFlags)
{
HRESULT hr = CMemAllocator::GetBuffer(ppBuffer, pStartTime,
pEndTime, dwFlags);
if(m_pVPP->m_bTypeChange)
{
m_pVPP->m_bTypeChange = false;
if(*ppBuffer)
(*ppBuffer)->SetMediaType(&m_pVPP->m_NewType);
}
return hr;
}
private:
CVideoPostProcess* m_pVPP;
};
// derived Outputpin to trap the queryaccept call
class CMyTransformOutputPin : public CTransformOutputPin
{
friend class CVideoPostProcess;
public:
CMyTransformOutputPin::CMyTransformOutputPin(
TCHAR *pObjectName,
CVideoPostProcess *pTransformFilter,
HRESULT * phr,
LPCWSTR pName):
CTransformOutputPin(pObjectName, pTransformFilter, phr, pName)
{
m_pVPP = pTransformFilter;
}
STDMETHODIMP QueryAccept(
const AM_MEDIA_TYPE *pmt)
{
HRESULT hr;
// Pass the query accept upstream
hr = m_pVPP->m_pInput->GetConnected()->QueryAccept(pmt);
if(S_OK == hr)
{
CMediaType newType(*pmt);
m_pVPP->ChangeMediaType(&newType);
}
if(S_OK != CTransformOutputPin::QueryAccept(pmt))
return S_FALSE;
return hr;
}
private:
CVideoPostProcess* m_pVPP;
};
// derived Inputpin to give the custom allocator to the decoder
class CMyTransformInputPin : public CTransformInputPin
{
friend class CVideoPostProcess;
public:
CMyTransformInputPin::CMyTransformInputPin(
TCHAR *pObjectName,
CVideoPostProcess *pTransformFilter,
HRESULT * phr,
LPCWSTR pName):
CTransformInputPin(pObjectName, pTransformFilter, phr, pName)
{
m_pVPP = pTransformFilter;
m_pAlloc = NULL;
}
STDMETHODIMP GetAllocator(
IMemAllocator **ppAllocator)
{
CAutoLock cObjectLock(m_pLock);
HRESULT hr = S_OK;
if (m_pAlloc == NULL)
{
m_pAlloc = new CMyAllocator(NAME("My Allocator"), NULL, &hr,
m_pVPP);
*ppAllocator = m_pAlloc;
m_pAlloc->AddRef();
}
return hr;
}
STDMETHODIMP NotifyAllocator(
IMemAllocator * pAllocator,
BOOL bReadOnly)
{
CTransformInputPin::NotifyAllocator(pAllocator, bReadOnly);
if(pAllocator == m_pAlloc)
return S_OK;
return E_FAIL;
}
private:
CVideoPostProcess* m_pVPP;
CMyAllocator* m_pAlloc;
};
//override getPin(0 to create my own pins
CBasePin *
CVideoPostProcess::GetPin(int n)
{
HRESULT hr = S_OK;
// Create an input pin if necessary
if (m_pInput == NULL) {
m_pInput = new CMyTransformInputPin(
NAME("Transform input pin"),
this, // Owner filter
&hr, // Result code
L"XForm In"); // Pin name
// Can't fail
ASSERT(SUCCEEDED(hr));
if (m_pInput == NULL) {
return NULL;
}
m_pOutput = (CTransformOutputPin *)
new CMyTransformOutputPin(
NAME("Transform output pin"),
this, // Owner filter
&hr, // Result code
L"XForm Out"); // Pin name
// Can't fail
ASSERT(SUCCEEDED(hr));
if (m_pOutput == NULL) {
delete m_pInput;
m_pInput = NULL;
}
}
// Return the appropriate pin
if (n == 0) {
return m_pInput;
} else
if (n == 1) {
return m_pOutput;
} else {
return NULL;
}
}
HRESULT CVideoPostProcess::CheckTransform(
const CMediaType * mtIn,
const CMediaType * mtOut)
{
CheckPointer(mtIn,E_POINTER);
CheckPointer(mtOut,E_POINTER);
HRESULT hr;
if(FAILED(hr = CheckInputType(mtIn)))
return hr;
if(m_bTypeChange)
{
if(*mtOut == m_NewType)
return S_OK;
else
return E_FAIL;
}
return S_OK;
}
HRESULT CVideoPostProcess::GetMediaType(
int iPosition,
CMediaType * pMediaType)
{
// Is the input pin connected
if(m_pInput->IsConnected() == FALSE)
{
return E_UNEXPECTED;
}
// This should never happen
if(iPosition < 0)
{
return E_INVALIDARG;
}
// Do we have more items to offer
if(iPosition > 0)
{
return VFW_S_NO_MORE_ITEMS;
}
CheckPointer(pMediaType,E_POINTER);
CMediaType InputMediaType = m_pInput->CurrentMediaType();
*pMediaType = InputMediaType;
return NOERROR;
}
HRESULT CVideoPostProcess::CheckInputType(
const CMediaType * mtIn)
{
CheckPointer(mtIn,E_POINTER);
// Check this is a VIDEOINFO type
if(*mtIn->FormatType() != FORMAT_VideoInfo)
return E_INVALIDARG;
return S_OK;
}
HRESULT CVideoPostProcess::SetMediaType(
PIN_DIRECTION direction,
const CMediaType * pmt)
{
// Call the base class to do its thing
CTransformFilter::SetMediaType(direction, pmt);
// Reconnect where necessary.
if( m_pInput->IsConnected() && m_pOutput->IsConnected() )
{
FILTER_INFO fInfo;
QueryFilterInfo( &fInfo );
if ((direction == PINDIR_OUTPUT && *pmt !=
m_pInput->CurrentMediaType()) ||
(m_bTypeChange && m_NewType != *pmt))
{
LOG_TRACE("Output type does not match input. Reconnecting the
input \n");
// This never happens for the copy transform filter, but it does
happen for inplace
fInfo.pGraph->Reconnect( m_pInput );
}
QueryFilterInfoReleaseGraph( fInfo );
ASSERT(!(direction == PINDIR_INPUT && *pmt !=
m_pOutput->CurrentMediaType()));
}
m_bYUY2 = (pmt->subtype == MEDIASUBTYPE_YUY2);
return NOERROR;
}
My DecideBufferSize() function is fine.. It allocates a big enough
buffer to handle all the cases.
Thanks,
Satish
Iain
> Hi,
>
> My graph looks like the following:
>
> Source --> Decoder --> Video Transform --> VMR9
>
> I want to handle the dynamic format change that the VMR9 requests and
> pass this request to the decoder upstream. For example, decoder and
> transform agree to 320x240 YUY2. Similarly transform and VMR agree to
> the same. When I run the graph, VMR9 asks for 512x(-240) YUY2. So I
> have do the format conversion. I followed the article described in the
> DShow documentation and came up with my allocator in the Video
> Transform filter. I am unable to get it working. My Transform function
> never gets called. The allocator GetBuffer() function gets called, but
> the transform never gets called. Also, in my GetBuffer() function, if
> I do not change the mediatype to the new type, the filter works (I
> have to do the format conversion in the transform filter). This is all
> with the copy transform filter.The inplace transform filter works
> seamlessly with format changes. However the issue with inplace
> transform is that the transform is really slow. Can I speed it up in
> some way?
>
> I have been struggling with this for past two days and hence the
> detailed code below. Any help is very much appreciated.
I wasn't able to read your code - NNTP doesn't help the formatting.
However all you should have to do is to check to see if the OUTPUT sample
in the Transform function has a media type associated with it and if so to
make that your new media type and adjust your transform (e.g. to invert it)
automatically.
Also, in general you do not want to do read operations in the buffer you
get from downstream. This is likely a DirectX surface in the Video card
and reading is slow.
I don't know if this helped.
Iain
--
Iain Downs (DirectShow MVP)
Commercial Software Therapist
www.idcl.co.uk
Satire
Thanks Iain.. I got the point about not using the renderer buffer for doing
read operations. Will modify my in place transform filter to make a temporary
copy of the buffer, do the transform using that buffer and copy back the
transformed buffer to the renderer buffer. That may help my case for inplace
transform.
But the real question is about passing the format change upstream from the
transform filter (not in place transform) to the decoder filter. Note that
this is for a copy transform filter. I do not want to handle the format
conversion inside my transform filter. Let the decoder give me the format
desired by vmr. I'll perform the transform operation if my transform
algorithm can work on the new format. If not, I'll just copy the input sample
to the output. Does that make my question clear?
Satish
Jeremy Noring
> My graph looks like the following:
>
> Source --> Decoder --> Video Transform --> VMR9
>
> I want to handle the dynamic format change that the VMR9 requests and
> pass this request to the decoder upstream. For example, decoder and
> transform agree to 320x240 YUY2. Similarly transform and VMR agree to
> the same. When I run the graph, VMR9 asks for 512x(-240) YUY2.
All YUV dibs are top-down, so the sign of the height can be ignored. Read
this article for more information:
When comparing the media types, you can simply take the absolute value of
the height; the negative value doesn't matter.
> So I
> have do the format conversion. I followed the article described in the
> DShow documentation and came up with my allocator in the Video
> Transform filter.
I don't think writing an allocator is going to fix the problem; you probably
need to account for the stride requested by the video renderer. Video
hardware request images formatted in varying width for efficiency reasons.
Then again, I don't know much about allocators or under what circumstances
they'd be useful, but I know the video hardware is requesting that width for
a specific reason.
Copying YUY2 is really, really easy--the following formula should work:
for( int rowline = 0; rowline < InputHeight; rowline++)
{
memcpy( pVideoRendererBuffer, pInputImageBuffer, ( InputWidth * 2 ) );
pVideoRendererBuffer += (OutputWidth * 2 ); // point to next scan line
pInputImageBuffer += (InputWidth * 2 ); // point to next scan line
}
> The inplace transform filter works
> seamlessly with format changes. However the issue with inplace
> transform is that the transform is really slow. Can I speed it up in
> some way?
The inplace transform is probably slow because you're reading from video
memory, which is really slow. You'll have to make your filter using the
regular transform filter or only write to video memory, which is fast.
Jeremy Noring
>
> >
> > It sounds like you know fully well how to handle the format change in your
> > own filter, but you're looking to have the decoder filter simply output the
> > correct format YUY2, right?
>
> Absolutely right.. As of now I'm handling format changes inside the
> transform filter, but I want the decoder to give me the correct format. There
> is an article on this in DShow help, which I followed, but did not succeed in
> passing the format change to the decoder. Note that if I have an inplace
> transform, then this format change is handled gracefully by the baseclasses
> and passed upstream to the decoder. I debugged through it and found that the
> input pin of the inplace transform filter receives a BreakConnect call after
> the queryaccept() call, but my copy transform filter does not.
I was wondering that exact thing looking through the documentation and
searching through the baseclasses--it seemed to me that the in-place filters
might already do this, and it made sense that they did. I've never had to
account for stride issues when doing an in-place transform, which led me to
believe they passed on the modified media type to the decoder.
I'm going to go through the baseclasses and see if I can figure out how this
mechanism works, but that's some nasty code if I've ever seen it.
I wonder if it'd be easier to modify the in-place filter to have access to
the buffer before it's been copied into video memory (the in-place is derived
from a regular transform filter, so I'd assume that the option is available
at some point in time).
Geraint Davies
> Sorry, after looking at some of the other posts, I think I understand what
> you're trying to accomplish.
>
> It sounds like you know fully well how to handle the format change in your
> own filter, but you're looking to have the decoder filter simply output the
> correct format YUY2, right?
>
> I'm not sure how to do it, but I have a transform filter where I handle the
> change internal, and I'd love to have it happen the way you're proposing.
What you need is for the input of the transform filter to implement a
custom allocator, derived from CMemAllocator with a couple of methods
overridden. You then need to provide an overridden input pin that offers
this allocator from GetAllocator (and perhaps overrides NotifyAllocator to
reject the connection if the allocator is not the one chosen).
Override the allocator's GetBuffer call, and at that point call GetBuffer
downstream and cache the downstream buffer in your filter -- i.e. at this
point you are acquiring two buffers -- the transform's input and output
buffers. The input one from your allocator you will be returning to the
caller, and the output one from downstream you will cache in the input pin
for later. Check for a media type change on the output buffer, and if
present, attach the same media type change to the input buffer that you are
returning.
If the new format requires a larger buffer, you will need to fix that in
your allocator -- you could allocate each buffer separately instead of
using a pool of fixed-size pre-allocated buffers, or you could ensure that
the buffer count is 1 so that in GetBuffer you can decommit and recommit at
the new dimensions. Finally, in the input pin's Receive method, you need to
remember that you have the output sample already cached.
G
Satire
Thanks for the response.. I tried your approach and it works fine, but I do
not want to handle the downstream buffer and override receive().
I found a better way is to override the transform output pin and handle
queryaccept().. Pass the query accept call to the upstream output pin... On
success, save the new media type and attach this media type when GetBuffer()
is called on the allocator.
My problem was with the GetMediaType function.. I should have passed the
upstream output pin IEnumMediaTypes to the downstream filter.. I was instead
giving only the currentmediatype of the transform input pin.. Now everything
works fine.. thanks again for all your help..