Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

shared DLL VS static Link, Are they different?

126 views
Skip to first unread message

asm23

unread,
Aug 3, 2008, 4:15:17 AM8/3/08
to
hi, everyone, I meet a strange problem:
I'm using VC6 and create a dialog based applications. And I want to add
a third part library( Open computer library--OpenCV AS DLL). I linked
with the cxcored.lib. My APP will call functions in cxcore001d.dll.


I only add two functions in the OnInitDialog function:
/////////////////////////////////////////////////////////////////
BOOL COpenCVdialogDlg::OnInitDialog()
{
......
m_iplImage = cvCreateImage(ImgSize,IPL_DEPTH_8U,IMAGE_CHANNELS);
cvReleaseImage(&m_iplImage);
......
}
////////////////////////////////////////////////////////////////
In the code above, I only create an Image and release it quickly, which
seems I do Nothing. ^_^ ,The two functions are exported from
cxcore100d.dll( associated with cxcored.lib)

Here comes my problem. When I build the APP with static MFC library, the
program works fine. But When I build the APP with "Using MFC as shared
DLL" option, there are many memory leak report when I exit the APP.

It is very strange that "using MFC as shared DLL" and "using MFC as a
static library" are much different? Otherwise, There are something wrong
with the source code generating cxcore100d.dll?

By the way, OpenCV is an open source library,so, I have the full source
code of "cxcore100d.dll" , I examine the code and can't find anything wrong.
Here is its DLL Main entry:
////////////////////////////////////////////////////////////////
#if defined WIN32 || defined WIN64
BOOL WINAPI DllMain( HINSTANCE, DWORD fdwReason, LPVOID )
{
CvContext *pContext;

switch (fdwReason)
{
case DLL_PROCESS_ATTACH:
g_TlsIndex = TlsAlloc();
if( g_TlsIndex == TLS_OUT_OF_INDEXES ) return FALSE;
//break;

case DLL_THREAD_ATTACH:
pContext = icvCreateContext();
if( pContext == NULL)
return FALSE;
TlsSetValue( g_TlsIndex, (LPVOID)pContext );
break;

case DLL_THREAD_DETACH:
if( g_TlsIndex != TLS_OUT_OF_INDEXES )
{
pContext = (CvContext*)TlsGetValue( g_TlsIndex );
if( pContext != NULL )
icvDestroyContext( pContext );
}
break;

case DLL_PROCESS_DETACH:
if( g_TlsIndex != TLS_OUT_OF_INDEXES )
{
pContext = (CvContext*)TlsGetValue( g_TlsIndex );
if( pContext != NULL )
icvDestroyContext( pContext );
}
TlsFree( g_TlsIndex );
break;
default:
;
}
return TRUE;
}
////////////////////////////////////////////////////////////////
I was confused that why the "break" statement is commented after the
"case DLL_PROCESS_ATTACH:"?

Further more, the cvCreateImage and cvReleaseImage function in my APP is
regularly simple, they only allocate a piece of memory and delete it.

Thanks for reading my Post!

David Lowndes

unread,
Aug 3, 2008, 4:33:39 AM8/3/08
to
>program works fine. But When I build the APP with "Using MFC as shared
>DLL" option, there are many memory leak report when I exit the APP.

And if you remove those 2 lines of code, do you still see leaks
reported when you use MFC DLL?

>I was confused that why the "break" statement is commented after the
>"case DLL_PROCESS_ATTACH:"?

Presumably the writer wanted the thread calling the process attach to
do the same as new threads would. Note that the documentation mentions
"A thread calling the DLL entry-point function with DLL_PROCESS_ATTACH
does not call the DLL entry-point function with DLL_THREAD_ATTACH. "

>Further more, the cvCreateImage and cvReleaseImage function in my APP is
> regularly simple, they only allocate a piece of memory and delete it.

And is it that memory that's reported as a leak, or something else
(like MFC global variables)?

Dave

asm23

unread,
Aug 3, 2008, 4:50:12 AM8/3/08
to
Thank you for your quick reply.

If I remove those 2 lines of code, the APP runs very well and NO memory
leaks reported. ( in Both DLL or static MFC mode )

Further more, I create the Dialog based APP with MFC APP wizard, and it
is so simple that I only add a pointer member IplImage * m_iplImage in
class COpenCVdialogDlg .

The source code of " cxcore001.dll" is build with the option :
"NOT using MFC "
"debug multithread DLL"

Is there something wrong?

asm23

unread,
Aug 3, 2008, 4:59:36 AM8/3/08
to
I also find a web page talk about the problem a little.

http://www.lund.gl/ImageProcess/OpenCV.html

the author says: "In step 3 of 4 of the MFC App wizard set a mark in the
radio button "As a static linked library" in "How would you use the MFC
Library?". Somehow the memory leaks is avoided by this setting!!."

so, I don't know why we should use this setting...

David Lowndes

unread,
Aug 3, 2008, 9:03:25 AM8/3/08
to
>Is there something wrong?

There's nothing obvious that I can see from what you've mentioned.

All I can suggest is that you investigate further since you have the
source for both components you should be able to identify the source
of the alleged leaks.

Dave

David Wilkinson

unread,
Aug 3, 2008, 9:54:41 AM8/3/08
to

asm:

Normally, a DLL is designed to work with the DLL version of the CRT, and the
client application should do this also. This causes the app and the DLL to use a
single heap, so that memory can be allocated in one module and freed in the
other. Normally, also, you should have a debug and release version of the DLL,
which you use in your debug and release build respectively.

Some thoughts:

1. You say the dll is built using *debug* multi-threaded DLL (this is /MDd). Do
you not have a release version also? What happens if you compile your app with
/MDd. Or is this what you are doing?

2. I'm not sure about this, but maybe the DLL is leaking memory (not your
fault), but you only see this memory leak if the DLL is using the same heap as
your application.

3. There is one thing in your code that seems odd to me:

m_iplImage = cvCreateImage(ImgSize,IPL_DEPTH_8U,IMAGE_CHANNELS);
cvReleaseImage(&m_iplImage);

Are you sure the second line shouldn't be

cvReleaseImage(m_iplImage);

?

4. Now it may be that the interface of this DLL is such that it does not require
the client to be using the same heap (the cvReleaseImage() suggests this). In
this case, IMHO, it makes sense to compile both the DLL and the application with
static linking, so as to avoid deployment issues on target machines.
[Personally, I will do almost anything to avoid dynamic linking...]

Apart from the memory leak, what happens if you actually try to use this
library? Does in work with both static and dynamic linking?

Third party DLL's can be a real pain...

--
David Wilkinson
Visual C++ MVP

asm23

unread,
Aug 3, 2008, 10:01:24 AM8/3/08
to
Hi, David, thanks for your another reply.
I know it is not easy to solve the problem.
But There is another thing I would mention: when I build the
cvcore100.dll and cvcore.lib in release mode. then there is NO memory
leak when I run My APP.

How can I explain this situation? Does this means that a Debug mode DLL
is different with a Release mode DLL? or, a Debug mode DLL is prone to
memory leak?

Giovanni Dicanio

unread,
Aug 3, 2008, 10:46:28 AM8/3/08
to

"asm23" <asmwa...@gmail.com> ha scritto nel messaggio
news:g74dro$e2f$1...@aioe.org...

> But There is another thing I would mention: when I build the cvcore100.dll
> and cvcore.lib in release mode. then there is NO memory leak when I run My
> APP.
>
> How can I explain this situation? Does this means that a Debug mode DLL is
> different with a Release mode DLL? or, a Debug mode DLL is prone to memory
> leak?

I think that the Release build - for performance reasons - just does not
have debugging facilities to identify memory leaks. I believe that if debug
build signals you memory leaks, you should stop and kill them before passing
to a release build.
You should do a release build only after debug build is clear.

Giovanni

Giovanni Dicanio

unread,
Aug 3, 2008, 10:47:58 AM8/3/08
to

"David Wilkinson" <no-r...@effisols.com> ha scritto nel messaggio
news:uUnW7BX9...@TK2MSFTNGP03.phx.gbl...

> 3. There is one thing in your code that seems odd to me:
>
> m_iplImage = cvCreateImage(ImgSize,IPL_DEPTH_8U,IMAGE_CHANNELS);
> cvReleaseImage(&m_iplImage);
>
> Are you sure the second line shouldn't be
>
> cvReleaseImage(m_iplImage);

The OP's code is correct.

cvReleaseImage use a double indirection pointer (I think it is good to avoid
dangling references - when the object is destroyed the pointer is set to
NULL by the function).

Giovanni

Joseph M. Newcomer

unread,
Aug 3, 2008, 11:23:10 AM8/3/08
to
See below...

****
See comment below: NEVER return FALSE from DLL_PROCESS_ATTACH. It looks easy, it is a
complete and utter tech support disaster of the first order, and must never be done if you
ever expect to support this product in the field

Also, never, ever put the body of an if-statement on the same line; you are creating a
debugging nightmare. Use two lines!
*****
****


> //break;
>
> case DLL_THREAD_ATTACH:
> pContext = icvCreateContext();

****
What does this do? IF it calls malloc, new, or uses the C library, a COM interface, etc.,
then it must not appear in DLLMain during the DLL_PROCESS_ATTACH. Read about DllMain.

Generally, you should not under any circumstances use DLL_THREAD_[AT|DE]TACH unless you
are prepared for the consequences, one of which is memory leaks. Note that if you get a
DLL_PROCESS_DETACH, *none* of the code in DLL_THREAD_DETACH will be executed for that
thread or any other thread, thus resulting in memory leakage of whatever was allocated. So
what you are seeing is not at all surprising.

When I teach my course, I have a slide that says "Are DLLs hard to write?". The next
slide, using PowerPoint animation, answers this question: "No" appears, then "Yes", then
"Maybe". Key to all of this is to design a DLL that falls into the "No" category, not
hard to write. As soon as you add DLL_THREAD_[AT|DE]TACH, you not only move into the
"Yes" category, you go screaming over to the "about as difficult as possible to get right"
category.


> if( pContext == NULL)
> return FALSE;

****
It is never appropriate to return FALSE here. This is particularly true for
DLL_PROCESS_ATTACH. In my course I go into great detail as to why, but essentially if you
ever return FALSE from DLL_PROCESS_ATTACH you have created a tech support nightmare that
will haunt your days, because when you load the app, it fails with a message that the DLL
"failed to load" and therefore you have no clue if the DLL failed to load because it
wasn't found, or was in error, or was the wrong DLL, or had some internal condition that
returned FALSE. My advice is "always return TRUE, and make your own internal indication
that the DLL failed to initialize, and why. Preferrably, have a STRINGTABLE explanation
in your DLL, so you store the IDS_ code during the failure, and then have a
QueryStartupValid call that the user must issue, e.g.,
BOOL QueryStartupValid(CString & s);
which returns FALSE if there was an error and puts the message in the CString. Note that
if the user fails to call this, you would still have to check for successful startup in
all calls and return some error indicating if initialization failed. This is tech-support
managable, but returning FALSE is unsupportable.
****


> TlsSetValue( g_TlsIndex, (LPVOID)pContext );
> break;
>
> case DLL_THREAD_DETACH:
> if( g_TlsIndex != TLS_OUT_OF_INDEXES )
> {
> pContext = (CvContext*)TlsGetValue( g_TlsIndex );
> if( pContext != NULL )
> icvDestroyContext( pContext );

****
Unless the thread terminates before DLL_PROCESS_DETACH, this code will never be executed.
Do not trust that this will happen. Try to avoid under any circumstances using DLL_THREAD
events. Your life will be so much easier.
****


> }
> break;
>
> case DLL_PROCESS_DETACH:
> if( g_TlsIndex != TLS_OUT_OF_INDEXES )
> {
> pContext = (CvContext*)TlsGetValue( g_TlsIndex );
> if( pContext != NULL )
> icvDestroyContext( pContext );
> }

****
This deletes the content ONLY for the main thread, and will not do it for any other thread
****


> TlsFree( g_TlsIndex );
> break;
> default:
> ;
> }
> return TRUE;
>}
>////////////////////////////////////////////////////////////////
>I was confused that why the "break" statement is commented after the
>"case DLL_PROCESS_ATTACH:"?
>
>Further more, the cvCreateImage and cvReleaseImage function in my APP is
> regularly simple, they only allocate a piece of memory and delete it.
>
>Thanks for reading my Post!

Joseph M. Newcomer [MVP]
email: newc...@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

Joseph M. Newcomer

unread,
Aug 3, 2008, 11:28:24 AM8/3/08
to
See below..

On Sun, 03 Aug 2008 22:01:24 +0800, asm23 <asmwa...@gmail.com> wrote:

>David Lowndes wrote:
>>> Is there something wrong?
>>
>> There's nothing obvious that I can see from what you've mentioned.
>>
>> All I can suggest is that you investigate further since you have the
>> source for both components you should be able to identify the source
>> of the alleged leaks.
>>
>> Dave
>Hi, David, thanks for your another reply.
>I know it is not easy to solve the problem.
>But There is another thing I would mention: when I build the
>cvcore100.dll and cvcore.lib in release mode. then there is NO memory
>leak when I run My APP.

****
You have obviously confused the concepts "No memory leak reported" and "no memory leak
occurred". They are only vaguely related. If a memory leak is reported, it occurred. If
it is not reported, you can't know if it occured. And guess what? "Release" mode means
"Do not track memory to look for memory leaks". You still have them, they just aren't
reported because release mode doesn't DO that!
****


>
>How can I explain this situation? Does this means that a Debug mode DLL
>is different with a Release mode DLL? or, a Debug mode DLL is prone to
>memory leak?

*****
No, Debug mode, is, not suprisingly, prone to DETECTING memory leaks, whereas release
mode, by its very design, does not (this is like the people who say "I never use Debug
mode because the MFC runtime is full of bugs: it keeps failing on ASSERT statements, but
these never happen in release mode, so by working only in release mode I don't hit all the
MFC bugs that are present in the Debug version of the libraries", where the correct
statement is "Debug mode finds hundreds of bugs in my code, which Release mode ignores, so
my code must be correct if no bugs are reported")
joe
****

Doug Harrison [MVP]

unread,
Aug 3, 2008, 12:06:44 PM8/3/08
to
On Sun, 03 Aug 2008 16:15:17 +0800, asm23 <asmwa...@gmail.com> wrote:

>hi, everyone, I meet a strange problem:
>I'm using VC6 and create a dialog based applications. And I want to add
>a third part library( Open computer library--OpenCV AS DLL). I linked
>with the cxcored.lib. My APP will call functions in cxcore001d.dll.
>
>
>I only add two functions in the OnInitDialog function:
>/////////////////////////////////////////////////////////////////
>BOOL COpenCVdialogDlg::OnInitDialog()
>{
> ......
> m_iplImage = cvCreateImage(ImgSize,IPL_DEPTH_8U,IMAGE_CHANNELS);
> cvReleaseImage(&m_iplImage);
> ......
>}
>////////////////////////////////////////////////////////////////
>In the code above, I only create an Image and release it quickly, which
>seems I do Nothing. ^_^ ,The two functions are exported from
>cxcore100d.dll( associated with cxcored.lib)
>
>Here comes my problem. When I build the APP with static MFC library, the
>program works fine. But When I build the APP with "Using MFC as shared
>DLL" option, there are many memory leak report when I exit the APP.
>
>It is very strange that "using MFC as shared DLL" and "using MFC as a
>static library" are much different? Otherwise, There are something wrong
> with the source code generating cxcore100d.dll?

The order of initialization of library static duration data (e.g. globals)
is not as well defined for static linking vs. dynamic linking.

Perhaps because the author wanted the DLL_PROCESS_ATTACH case to fall
through to the DLL_THREAD_ATTACH case. If this is true, it would be much
clearer to replace:

//break;

with:

// FALLTHROUGH

>Further more, the cvCreateImage and cvReleaseImage function in my APP is
> regularly simple, they only allocate a piece of memory and delete it.
>
>Thanks for reading my Post!

I would bet the problem is due to the non-MFC DLL loading before the MFC
DLL. This means the non-MFC DLL's static duration data is destroyed after
MFC exits, and if there is any memory that is freed at that time, MFC will
report it as leaked, since it is still allocated when MFC dumps leaks in
its DLL_PROCESS_DETACH. This is a long-standing defect in MFC, which has no
business dumping leaks; it should wait until the CRT DLL dumps leaks after
all static duration objects have been destroyed. For a workaround, see if
these messages help:

http://groups.google.com/group/microsoft.public.vc.debugger/msg/6b90e68f21529e56

http://groups.google.com/group/microsoft.public.vc.mfc/browse_frm/thread/73493ddec165a5cb/e651b944da9c619d?#e651b944da9c619d

Though I never tried it, the second one should help with those DLLs you
can't modify.

--
Doug Harrison
Visual C++ MVP

Giovanni Dicanio

unread,
Aug 3, 2008, 12:39:08 PM8/3/08
to
As Joe and Doug pointed out, that library has problems with DllMain and
thread local storage, etc.

I did some stepping in the code.

The problem is that that library is a mainly-C-interface library (even if
the sources are .cpp), and there is a structure called "CvContext" in that
library.

The authors provided internal functions like icvCreateContext(),
icvDestroyContext(), and icvGetContext() - see file cxcore\src\cxerror.cpp
(where DllMain is also defined).

The problem is that this "context" management is a real mess, and if you try
to log the calls to icvCreateContext() and the calls to icvDestroyContext(),
you will see that they do not match!

I saw that inserting a custom code in icvCreateContext() and
icvDestroyContext() bodies, this custom code basically uses
OutputDebugString to trace message like "icvCreateContext - CvContext
allocated at %08x <memory address here...>").

The problem is that icvGetContext is badly designed IMHO (with some
confusing #ifdef's, too), and there is a mix of TlsGetValue(), and also
calls to icvCreateContext from icvGetContext...

There are global variables defined in that DLL, and their constructor calls
icvGetContext - you can see that using call stack:
CvType constructor calls
cvGetErrStatus calls
icvGetContext

It may happen that icvGetContext is called like shown above *before* DllMain
(because the global variables are constructed before DllMain is called).

If you put breakpoints you can see how bad and non-linear the code flow is
to get these "CvContext"s...

And if you use proper tracing, you will see that *not all* the instances of
CvContext created on the heap using malloc() are properly destroyed!
(BTW: this library is a C++ library, but uses malloc instead of new...)

In general, I think that DllMain is a poor place to do initialization and
cleanup - better providing external public functions - and if there is some
"context" state to be shared by DLL functions, better passing it explicitly
(or use a C++ library, so the "state" is inside the class body).

A couple articles about DllMain...

http://blogs.msdn.com/oleglv/archive/2003/10/24/56141.aspx

http://blogs.msdn.com/larryosterman/archive/2004/04/23/118979.aspx

HTH,
Giovanni

"asm23" <asmwa...@gmail.com> ha scritto nel messaggio
news:g73pir$kcu$1...@aioe.org...

asm23

unread,
Aug 3, 2008, 1:21:15 PM8/3/08
to
Doug Harrison [MVP] wrote:
>
>> Further more, the cvCreateImage and cvReleaseImage function in my APP is
>> regularly simple, they only allocate a piece of memory and delete it.
>>
>> Thanks for reading my Post!
>
> I would bet the problem is due to the non-MFC DLL loading before the MFC
> DLL. This means the non-MFC DLL's static duration data is destroyed after
> MFC exits, and if there is any memory that is freed at that time, MFC will
> report it as leaked, since it is still allocated when MFC dumps leaks in
> its DLL_PROCESS_DETACH. This is a long-standing defect in MFC, which has no
> business dumping leaks; it should wait until the CRT DLL dumps leaks after
> all static duration objects have been destroyed. For a workaround, see if
> these messages help:
>
> http://groups.google.com/group/microsoft.public.vc.debugger/msg/6b90e68f21529e56
>
> http://groups.google.com/group/microsoft.public.vc.mfc/browse_frm/thread/73493ddec165a5cb/e651b944da9c619d?#e651b944da9c619d
>
> Though I never tried it, the second one should help with those DLLs you
> can't modify.
>
Thank you Doug for your help. You have point out the truth of these
"spurious leak reports".

I add the code you supplied in the post
http://groups.google.com/group/microsoft.public.vc.debugger/msg/6b90e68f21529e56
and rebuild the cvcore100d.dll. It seems that the number of memory leak
reports has reduced to ONLY ONE! (There are more than 7 reports before.)

Maybe, the Last ONE report is a REAL memory leak. I will try to find it.

Thanks.

Geoff

unread,
Aug 3, 2008, 2:17:13 PM8/3/08
to
On Sun, 03 Aug 2008 11:23:10 -0400, Joseph M. Newcomer
<newc...@flounder.com> wrote:

>Also, never, ever put the body of an if-statement on the same line; you are creating a
>debugging nightmare. Use two lines!

What about

if (!something) return;
or
if (something) continue;

I would think setting a BP on the line and single stepping through it would
be simple enough. How is it a debugging nightmare?

Joseph M. Newcomer

unread,
Aug 3, 2008, 3:21:23 PM8/3/08
to
see below...

****
For example, suppose you had
{
SomeType variable;
various stuff
if(condition) return;
do stuff
}

If you have the line as shown, and the condition is false, then you end up leaving scope
on the next single-step and you can't recover. If condition depended upon evaluating
values of 'variable' you can no longer look inside it because it is gone. But if you
write
{
SomeTime variable;
various stuff
if(condition)
return;
do stuff
}
and you are on the if-line, and single-step, you fall into the return line. At this
point, you have not actually executed the return statement, you can examine the values in
variable (which might a complex class), and you can even "set next statement" to retry a
compputation, or move to the do stuff, which can save a rebuild cycle (at least).

In a line-oriented debugger world, putting this code on one line is counterproductive.
joe

Norbert Unterberg

unread,
Aug 3, 2008, 3:28:42 PM8/3/08
to

Geoff schrieb:

But what if that if is within a loop that runs 1000 times and you want to set a
breakpoint on the single instance where something is true, and this only happens
after the 500th iteration?

However, this is a limitation of the MSVC debugger; I have seen debuggers where
you could actually set breakpoints on statements and not on lines.

Norbert

Joseph M. Newcomer

unread,
Aug 3, 2008, 10:09:44 PM8/3/08
to
See below...
On Sun, 03 Aug 2008 21:28:42 +0200, Norbert Unterberg <nunte...@newsgroups.nospam>
wrote:

>
>Geoff schrieb:
>> On Sun, 03 Aug 2008 11:23:10 -0400, Joseph M. Newcomer
>> <newc...@flounder.com> wrote:
>>
>>> Also, never, ever put the body of an if-statement on the same line; you are creating a
>>> debugging nightmare. Use two lines!
>>
>> What about
>>
>> if (!something) return;
>> or
>> if (something) continue;
>>
>> I would think setting a BP on the line and single stepping through it would
>> be simple enough. How is it a debugging nightmare?
>
>But what if that if is within a loop that runs 1000 times and you want to set a
>breakpoint on the single instance where something is true, and this only happens
>after the 500th iteration?

****
Well, you could use the existing conditional breakpoint mechanism to set the iteration
count to 500, or set a conditional value to cause the breakpoint. Note that whether you
put the if statement on one line or two, you have to deal with complex conditions; in
fact, I've sometimes done
if(complex condinational expression)
ASSERT(FALSE);
as a way of getting a breakpoint, especially because with C++/MFC collections you can't
actually write a conditional expression in the debugger.
*****


>
>However, this is a limitation of the MSVC debugger; I have seen debuggers where
>you could actually set breakpoints on statements and not on lines.

****
Yes, but we don't have that kind of debugger, we have a line-oriented debugger.
joe
****
>
>Norbert

Giovanni Dicanio

unread,
Aug 4, 2008, 4:51:46 AM8/4/08
to

"Joseph M. Newcomer" <newc...@flounder.com> ha scritto nel messaggio
news:0lib94pjsj0ue33u1...@4ax.com...

> See comment below: NEVER return FALSE from DLL_PROCESS_ATTACH. It looks
> easy, it is a
> complete and utter tech support disaster of the first order, and must
> never be done if you
> ever expect to support this product in the field
>

> [...]


>
> Generally, you should not under any circumstances use
> DLL_THREAD_[AT|DE]TACH unless you
> are prepared for the consequences, one of which is memory leaks. Note
> that if you get a
> DLL_PROCESS_DETACH, *none* of the code in DLL_THREAD_DETACH will be
> executed for that
> thread or any other thread, thus resulting in memory leakage of whatever
> was allocated. So
> what you are seeing is not at all surprising.
>
> When I teach my course, I have a slide that says "Are DLLs hard to
> write?". The next
> slide, using PowerPoint animation, answers this question: "No" appears,
> then "Yes", then
> "Maybe". Key to all of this is to design a DLL that falls into the "No"
> category, not
> hard to write. As soon as you add DLL_THREAD_[AT|DE]TACH, you not only
> move into the
> "Yes" category, you go screaming over to the "about as difficult as
> possible to get right"
> category.

I think a good approach is to do like GDI+, i.e. exporting a couple of
functions for DLL initialization and cleanup (like GdiplusStartup and
GdiplusShutdown).

http://msdn.microsoft.com/en-us/library/ms534077(VS.85).aspx

Giovanni

Giovanni Dicanio

unread,
Aug 4, 2008, 4:58:27 AM8/4/08
to

"Doug Harrison [MVP]" <d...@mvps.org> ha scritto nel messaggio
news:l1lb94pv2o1616rve...@4ax.com...

> I would bet the problem is due to the non-MFC DLL loading before the MFC
> DLL. This means the non-MFC DLL's static duration data is destroyed after
> MFC exits, and if there is any memory that is freed at that time, MFC will
> report it as leaked, since it is still allocated when MFC dumps leaks in
> its DLL_PROCESS_DETACH. This is a long-standing defect in MFC, which has
> no
> business dumping leaks; it should wait until the CRT DLL dumps leaks after
> all static duration objects have been destroyed. For a workaround, see if
> these messages help:
>
> http://groups.google.com/group/microsoft.public.vc.debugger/msg/6b90e68f21529e56

I may be wrong, but I suspect that the aforementioned DLL does some leaks
(OK, these leaks may not be harmful, because they are memory leaks, they are
not non-memory resource leaks e.g. like files; so, being memory leaks, I
think that the OS will free the heap memory after the process is closed).

That DLL is open-source, and if memory allocation tracking (using e.g.
OutputDebugString) is added in internal helper functions icvCreateContext()
and icvDestroyContext(), it seems to me that there are "contexts" create
with icvCreateContext() but not destroyed by icvDestroyContext().

Of course, this is not OP's fault - it's a library problem.

Giovanni

Joseph M. Newcomer

unread,
Aug 4, 2008, 8:57:09 AM8/4/08
to
See below...
On Mon, 4 Aug 2008 10:58:27 +0200, "Giovanni Dicanio" <gdicanio@_NOSPAM_email_DOT_it>
wrote:

>
>"Doug Harrison [MVP]" <d...@mvps.org> ha scritto nel messaggio
>news:l1lb94pv2o1616rve...@4ax.com...
>
>> I would bet the problem is due to the non-MFC DLL loading before the MFC
>> DLL. This means the non-MFC DLL's static duration data is destroyed after
>> MFC exits, and if there is any memory that is freed at that time, MFC will
>> report it as leaked, since it is still allocated when MFC dumps leaks in
>> its DLL_PROCESS_DETACH. This is a long-standing defect in MFC, which has
>> no
>> business dumping leaks; it should wait until the CRT DLL dumps leaks after
>> all static duration objects have been destroyed. For a workaround, see if
>> these messages help:
>>
>> http://groups.google.com/group/microsoft.public.vc.debugger/msg/6b90e68f21529e56
>
>I may be wrong, but I suspect that the aforementioned DLL does some leaks
>(OK, these leaks may not be harmful, because they are memory leaks, they are
>not non-memory resource leaks e.g. like files; so, being memory leaks, I
>think that the OS will free the heap memory after the process is closed).

****
There is a persistent myth that failure to free memory results in memory leakage that
eventually will bring the system down, even though the program that was running has long
since stopped, and even sillier, that running a program that leaks time after time will
cause the system to lose more and more memory.

When a process terminals, ALL RESOURCES OWNED BY THAT PROCESS ARE RELEASED. This means
all memory is freed, all handles used by that process are closed, all GDI resources used
by that process are released, etc.

Note that closing a handle does not release the resources associated with the handle if
another process still has a handle to the object (mutex, semaphore, event, etc.). A file
is not officially closed until *all* handles used by *all* processes are closed, but the
handles used by a process are all closed when the process terminates.

Clipboard data is not owned by the process (in general) but is owned by the system, so it
is not released. The exception is (and from time to time you will get this message)
"thus-and-such a program has data on the clipboard, do you want it left for other
programs?" This is because some programs used deferred clipboard resolution; the data is
not *actually* in the clipboard, but is held by the program itself. If you exit the
program, it will not be able to respond with the data; if you answer "yes" to the
question, it will at that point create the clipboard information and place it in the
clipboard (typically this applies to large images, and that mostly because you are doing
copy-and-paste within the program so it doesn't need to make a copy of the data until you
paste it).

OTOH, a DLL that fails to release memory or other resources it owns can degrade the
performance of the process, if, for example, a sequence of LoadLibrary/FreeLibrary calls
bring it in multiple times and then release it. If the only instance of a handle or
pointer to memory is kept in a variable known only to the DLL and not to the application,
the application cannot free the resource; when the DLL goes away, the only known resource
reference (handle or pointer) disappears. But handles and memory are owned by the
process, and removing the DLL cannot free these resources because the OS does not know or
care that they were allocated in a DLL (this is what DLL_PROCESS_DETACH is all about: if
the DLL allocates resources that only it knows about, it must free them in
DLL_PROCESS_DETACH).

DLL_THREAD_ events are very, very dangerous to depend on. For example, if a FreeLibrary
call is done, the DLL is unloaded if its reference count goes to 0, but not a single
DLL_THREAD_DETACH event happens. So if pointers or handles are kept in TLS, they simply
evaporate in a puff of greasy blue smoke, leaving the resources they reference still
allocated, and generally there is no possible way to free them. Assuming that every
thread has successfully stopped before the DLL is unloaded is a dangerous assumption. I
therefore strongly discourage the belief that DLL_THREAD_DETACH is a reliable mechanism.
IT is fundamentally untrustworthy. I prefer your suggestion of explicit
initialization/cleanup, preferrably NOT done by the caller. For example, the calling
thread calls a Startup routine that returns it a pointer to a data structure (which can be
an LPVOID), and it is the responsibility of the thread to call a Finish routine passing
that same pointer back in to free it. If you don't want the internal structure known, you
can treat it as a "handle" and before returning the pointer, cast it to a handle type, and
when it is passed in, cast it from a handle type back to the pointer type you want; this
gives you a completely opaque data type that can only be used by methods of the DLL, and
cannot be seen or touched internally by the caller.

For example, in my .h file I will add
DECLARE_HANDLE (MYHANDLE)

and write my DLL's interface in terms of MYHANDLE.

MYHANDLE Startup(Some parameters here)
{
LPMYTHING thing = new MYTHING;
...
return (HANDLE)thing;
}

BOOL DoSomething(MYHANDLE h, other parameters here)
{
LPMYTHING thing = (LPMYTHING)h;
...
return some boolean result;
}

void Finish(MYHANDLE h)
{
LPMYTHING thing = (LPMYTHING) h;
delete thing;
}
Now it is really explicit: if your thread doesn't clean up after itself, the thread is
itself erroneous. Nothing magical happens in a DLL_THREAD_DETACH event, and nobody cares
if the DLL gets unloaded; it is a programming error to unload it while there are active
threads, because the threads have to make sure they have properly terminated.

One way to handle this is to have every thread load the DLL:

/* static */ UINT CMyClass::threadfunc(LPVOID p)
{
HANDLE lib = ::LoadLibrary(_T("MyDLL"));
MyFunctions f;
f.Whatever = (WHATEVERFUNC)::GetProcAddress(lib, "Whatever");
f.Thing = (THINGFUNC)::GetProcAddress(lib, "Thing");
... etc.
CMyClass * self = (CMyClass *)p;

self->runthread(f);
::FreeLibrary(lib);
return 0;
}

That way, the DLL cannot be unloaded until every thread terminates (note that it is a
programming error to call ::ExitThread, _endthread, _endthreadex, or similar functions
that can terminate the thread without returning off the top-level thread function. This
works amazingly well.
joe

*****


>
>That DLL is open-source, and if memory allocation tracking (using e.g.
>OutputDebugString) is added in internal helper functions icvCreateContext()
>and icvDestroyContext(), it seems to me that there are "contexts" create
>with icvCreateContext() but not destroyed by icvDestroyContext().
>
>Of course, this is not OP's fault - it's a library problem.
>
>Giovanni
>
>

Giovanni Dicanio

unread,
Aug 4, 2008, 11:52:38 AM8/4/08
to

"Joseph M. Newcomer" <newc...@flounder.com> ha scritto nel messaggio
news:ogtd949k0jnhii212...@4ax.com...

> When a process terminals, ALL RESOURCES OWNED BY THAT PROCESS ARE
> RELEASED. This means
> all memory is freed, all handles used by that process are closed, all GDI
> resources used
> by that process are released, etc.

I used to think that the OS releases only memory resources, not file
handles, etc.
Thanks for correcting me.

However, I think we should always pay attention to avoid leaks... I think
quality code is non-leaking code.


> For example, the calling
> thread calls a Startup routine that returns it a pointer to a data
> structure (which can be
> an LPVOID), and it is the responsibility of the thread to call a Finish
> routine passing
> that same pointer back in to free it. If you don't want the internal
> structure known, you
> can treat it as a "handle" and before returning the pointer, cast it to a
> handle type, and
> when it is passed in, cast it from a handle type back to the pointer type
> you want; this
> gives you a completely opaque data type that can only be used by methods
> of the DLL, and
> cannot be seen or touched internally by the caller.

Exactly!

Giovanni

asm23

unread,
Aug 4, 2008, 11:29:44 PM8/4/08
to

Thank you Giovanni.
Follow your advice, I add these code in icvCreateContext() body
//************************************************************
char msgbuf[100];
sprintf(msgbuf, "memory create here %08x \n", (void*)context);
OutputDebugString(msgbuf);
//************************************************************

add these code in icvDestroyContext() body
//************************************************************
char msgbuf[100];
sprintf(msgbuf, "memory destroyed here %08x \n", (void*)context);
OutputDebugString(msgbuf);
//************************************************************

When debug the cvcore100d.dll I found icvCreateContext() is called twice.
**First: Before DLLMain


CvType constructor calls
cvGetErrStatus calls

icvGetContext calls
icvCreateContext. ********Here!
This is before the DLLMain is entered. It seems there are some global
variable in Cvcore100d.dll, and will be initialized before DLLMain.
**Second: in DLLMain
case DLL_THREAD_ATTACH:
pContext = icvCreateContext(); ********Here!
...

But the icvDestroyContext() will be called ONCE when Exit the APP.
It's still in DLLMain


case DLL_PROCESS_DETACH:
if( g_TlsIndex != TLS_OUT_OF_INDEXES )
{
pContext = (CvContext*)TlsGetValue( g_TlsIndex );
if( pContext != NULL )

icvDestroyContext( pContext ); ********Here!
}
TlsFree( g_TlsIndex );
break;

------------------------------------------------------------------
So, I think the Memory created in global variable in DLL files will not
released, That the *CAUSE* of Memory leak. Even I download the latest
code from openCV's CVS, the problems still exit.
If I add preprocessor definition CV_DLL, there is another memory leak.
------------------------------------------------------------------


case DLL_PROCESS_DETACH:
if( g_TlsIndex != TLS_OUT_OF_INDEXES )
{
pContext = (CvContext*)TlsGetValue( g_TlsIndex );
if( pContext != NULL )
icvDestroyContext( pContext );
}
TlsFree( g_TlsIndex );
break;

-------------------------------------------------------------------
here is the code in icvGetContext()
//****************************************************************
static CvContext*
icvGetContext(void)
{
#ifdef CV_DLL


#if defined WIN32 || defined WIN64

CvContext* context;

//assert(g_TlsIndex != TLS_OUT_OF_INDEXES);


if( g_TlsIndex == TLS_OUT_OF_INDEXES )

{


g_TlsIndex = TlsAlloc();
if( g_TlsIndex == TLS_OUT_OF_INDEXES )

FatalAppExit( 0, "Only set CV_DLL for DLL usage" );
}

context = (CvContext*)TlsGetValue( g_TlsIndex );
if( !context ) ******NOTE**
{
context = icvCreateContext();
if( !context )
FatalAppExit( 0, "OpenCV. Problem to allocate memory for
TLS OpenCV context." );

TlsSetValue( g_TlsIndex, context );
}
return context;
......
//****************************************************************
NOTE: context will always be checked in icvGetContext(), if it is NULL,
then icvCreateContext will be called, and a memory will be allocated.
Before DLL_PROCESS_ATTACH or After DLL_PROCESS_DETACH
, the TlsGetValue() will always return NULL, so, the memory will created
and never be released.

Further More, I'm not familiar with OpenCV, so, I don't know What
"CvContext" used for?


Giovanni Dicanio

unread,
Aug 5, 2008, 5:39:18 AM8/5/08
to

"asm23" <asmwa...@gmail.com> ha scritto nel messaggio
news:g78hjf$36e$1...@aioe.org...

> Thank you Giovanni.

You're welcome.


> Follow your advice, I add these code in icvCreateContext() body

[...]

Exactly, your analysis is similar to what I got: icvCreateContext and
icvDestroyContext calls do not match: the DLL leaks memory.
Note how good is OutputDebugString and tracing in general!

> Further More, I'm not familiar with OpenCV, so, I don't know What
> "CvContext" used for?

This is the first time I saw that library; I just think that CvContext is a
private C structure used by the library to store "context" i.e. "status"
information (e.g. last operation error, or similar things). I think that in
library authors' mind this should be a thread-specific thing, but I think
they failed in properly managing that (causing leaks).

I think that the better way to create and destroy this "context" is not in
DllMain and DLL_THREAD... messages, but exporting explicit functions to init
and cleanup this context.
See above post by Joe on a technique I agree with (using opaque pointers).

The library is open source, so if you can spend some time on that, you may
adjust and fix this context thing; I've no time now, but I think it would be
a good service to the library community.

Giovanni

Giovanni Dicanio

unread,
Aug 5, 2008, 6:05:14 AM8/5/08
to

"asm23" <asmwa...@gmail.com> ha scritto nel messaggio
news:g78hjf$36e$1...@aioe.org...

> Further More, I'm not familiar with OpenCV, so, I don't know What
> "CvContext" used for?

To add to my previous post, I think that lots of open source projects have
community and web forums or mailing-lists to exchange information, questions
and answer.

You may consider posting the problem we diagnosed in this thread also to the
OpenCV community using some web forum or other community tool.

Giovanni

Joseph M. Newcomer

unread,
Aug 5, 2008, 8:10:15 AM8/5/08
to
See below...
On Mon, 4 Aug 2008 17:52:38 +0200, "Giovanni Dicanio" <gdicanio@_NOSPAM_email_DOT_it>
wrote:

>


>"Joseph M. Newcomer" <newc...@flounder.com> ha scritto nel messaggio
>news:ogtd949k0jnhii212...@4ax.com...
>
>> When a process terminals, ALL RESOURCES OWNED BY THAT PROCESS ARE
>> RELEASED. This means
>> all memory is freed, all handles used by that process are closed, all GDI
>> resources used
>> by that process are released, etc.
>
>I used to think that the OS releases only memory resources, not file
>handles, etc.
>Thanks for correcting me.
>
>However, I think we should always pay attention to avoid leaks... I think
>quality code is non-leaking code.

****
Yes, because if there is a leak, it doesn't necessarily leak just once, but often "each
time" some event occurs. So it really does degrade the application while it is running.

OTOH, certain kinds of static initializers will allocate memory but the memory leak report
happens before the static destructors, so you get "false positives". MFC is noted for
this.

The automatic closing of a handle has interesting implications for driver writers, and I
have a lab devoted to the issues of handle closing in the driver course. The real truth
is much more subtle, and relies critically on the device driver being correctly written.
joe
****


>
>
>> For example, the calling
>> thread calls a Startup routine that returns it a pointer to a data
>> structure (which can be
>> an LPVOID), and it is the responsibility of the thread to call a Finish
>> routine passing
>> that same pointer back in to free it. If you don't want the internal
>> structure known, you
>> can treat it as a "handle" and before returning the pointer, cast it to a
>> handle type, and
>> when it is passed in, cast it from a handle type back to the pointer type
>> you want; this
>> gives you a completely opaque data type that can only be used by methods
>> of the DLL, and
>> cannot be seen or touched internally by the caller.
>
>Exactly!
>
>Giovanni
>
>

asm23

unread,
Aug 6, 2008, 1:59:58 AM8/6/08
to
Thanks.

I debugged the code whole day, I finally located the piece of code which
generate memory leak.
It's mainly because OpenCV will CHECK every cv*** call and keep a memory
thread local storage for error handling. Normally,the TLS will be freed
after PROCESS_DETACHED. But, unluckily, the TLS memory will not be
released in my situation mentioned in my first Post.

Actually, I have reported this problem in the Yahoo openCV community,
But, this problems is so related to MFC, so, nobody replied there...

I'm happy there are many enthusiastic help in this group. and Thank you
very much.

Giovanni Dicanio

unread,
Aug 6, 2008, 1:34:38 PM8/6/08
to
I did simple adjustments to that library, modifiying a couple of files
(cxerror.cpp and cxcore.h), and adding explicit library init/cleanup
routines:

http://www.geocities.com/giovanni.dicanio/vc/giovanni_cv.zip

I included also the VC6 MFC dialog-based test app.

I have no memory leaks now (you must also apply Doug's suggestions to avoid
spurious dumps).

Basically, if during DLL global variables constructor code execution there
is a call to cvGetErrStatus, instead of creating a CvContext on the heap
that will be leaked on DLL unload, I check if there is a CvContext already
available, and I use that context if it is present. Instead, if there is no
context available, it means that the DLL initialization routine was not
called, so I just return the default error status in cvGetErrStatus.
The context will be created after DLL initialization routine is called
(better than DllMain stuff, IMHO).
DLL shutdown routine must be called too, before app exits.

HTH,
Giovanni


"asm23" <asmwa...@gmail.com> ha scritto nel messaggio

news:g7bep6$ia0$1...@aioe.org...

0 new messages