Tracking Object Creation and Destruction
Ross Boylan
of a test suite:
initializeCounts();
exerciseTest();
checkCounts();
The counts would be things like "class A had 10 instances allocated and
10 freed." That is, they are on a class basis. Mostly, I want to be
sure that all A's created in the course of the test are freed afterward.
Any advice? In particular, is there some code that would be a good
basis for doing such things, and how could it be adapted? The rest of
this message summarizes my review so far.
In a thread started Jan 29 I asked about using a template to wrap the
class(es) of interest, but as I push on I notice this approach begins to
require significant code changes for some tests (e.g., all the spots in
my code that say Path would now need to use Path<Node> so that I could
use my special counted Node). Since existing tools (mostly memory
checkers) already have generic hooks into memory use, maybe one does
what I need or could be adapted.
None of the tools I've investigated look quite right, though perhaps
some could be adapted. Most are oriented to individual memory
allocations rather than the classes doing the allocating. Many work by
wrapping an entire run and generating reports, which is awkward for
automated testing of a particular chunk of code.
It would probably be sufficient for me to track objects created on the
heap, though it might be nice to track other forms of object creation as
well.
Here are some tools that seem in the ballpark (of course, working with C
++ rather than just C is a requirement; I'm not sure they all meet that
requirement):
LeakTracer http://www.andreasen.org/LeakTracer/
captures the call stack (using a GNU specific feature) and provides
printed interpretation of it by controlling a gdb session thru perl.
Not a very natural base for tracing a part of a program. Moving
programatically from the call stack to the class seems challenging.
Does C++ only.
mpatrol http://www.cbmamiga.demon.co.uk/mpatrol/
might have sufficient hooks to make something work, though the way in
which types that I know get mapped to the type info that it uses are not
clear.
libcwd http://libcwd.sourceforge.net/
Also might have sufficient hooks, though again the type mapping is
unclear and its natural orientation is to individual memory allocations,
not the association of those allocations with particular classes. Very
strongly oriented to producing printed output. It says it dropped
support for BFD, which surprised me; BFD sounds like a good basis for
portable debugging.
ccmalloc http://www.inf.ethz.ch/personal/biere/projects/ccmalloc/
At first blush looks too stripped down; it's not clear it tracks type
info, or that it offers any hooks for programmatic testing.
valgrind http://valgrind.org
Not clear it captures C++ type info.
The biggest challenge seems to be connecting a particular memory
allocation to a particular class. The connection may require both
translation from a call stack to a class, and translation between
different representations of class types (type as entered in the source
code, type as stored by the particular tool, C++ standard typeinfo,
printed output).
My main development is on Linux/i386, though the target is OS-X/PowerPC
(and we also have some Suns around--some discussion I've seen suggests
they have particularly strong tools for this kind of work). I'd prefer
a free software solution, though I've used Purify (and been very
impressed with it). Because Purify is closed, I doubt it could be
adapted beyond what it already does; I'm also not sure of the quality of
their Linux/Unix implementations (I used it on MS Windows).
Also, I'd like to be able to redistribute the program including the test
suite.
--
Ross Boylan wk: (415) 514-8146
185 Berry St #5700 ro...@biostat.ucsf.edu
Dept of Epidemiology and Biostatistics fax: (415) 514-8150
University of California, San Francisco
San Francisco, CA 94107-1739 hm: (415) 550-1062
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]
Thomas Richter
> I would like to be able to do the following test automatically, as part
> of a test suite:
> initializeCounts();
> exerciseTest();
> checkCounts();
> The counts would be things like "class A had 10 instances allocated and
> 10 freed." That is, they are on a class basis. Mostly, I want to be
> sure that all A's created in the course of the test are freed afterward.
>
> Any advice? In particular, is there some code that would be a good
> basis for doing such things, and how could it be adapted? The rest of
> this message summarizes my review so far.
Thus, your requirement is
a) no major code changes for existing code and
b) keeping track of the class info?
The problem you're facing is of course that the type information is
lost as soon as you call an operator new() (it only has the size of the
class), and changing that requires either to pass in an additional
parameter, identifying the class, or providing class-specific operator
news. But both of these require changes in the existing code, or at
least some macro-magic. Thus, the question goes back to you whether this
is an option.
Personally, I would rather follow a different route: I would use the
valgrind tool for testing whether there are any memory leaks (and make
valgrind runs part of the test suite, of course). It includes the
construction/allocation point for any leaked memory, and from there it's
easy to see which class was responsible for the leak. Of course, that's
some manual work, but that's debugging in first place.
> In a thread started Jan 29 I asked about using a template to wrap the
> class(es) of interest, but as I push on I notice this approach begins to
> require significant code changes for some tests (e.g., all the spots in
> my code that say Path would now need to use Path<Node> so that I could
> use my special counted Node). Since existing tools (mostly memory
> checkers) already have generic hooks into memory use, maybe one does
> what I need or could be adapted.
That would require that the checker has an idea for which class this
memory is used. This is typically not the case, and up to my knowledge
not achievable with C++ methods alone as the corresponding allocators,
operator new namely, has no information on the purpose of the class.
> None of the tools I've investigated look quite right, though perhaps
> some could be adapted. Most are oriented to individual memory
> allocations rather than the classes doing the allocating. Many work by
> wrapping an entire run and generating reports, which is awkward for
> automated testing of a particular chunk of code.
Sorry, I wonder in which sense, or what your test suite is supposed to
do. Doesn't that imply that you run the program once, entirely?
So long,
Thomas
--
Otis Bricker
news:1170813856.28...@iron.psg.net:
> I would like to be able to do the following test automatically, as
> part of a test suite:
> initializeCounts();
> exerciseTest();
> checkCounts();
> The counts would be things like "class A had 10 instances allocated
> and 10 freed." That is, they are on a class basis. Mostly, I want to
> be sure that all A's created in the course of the test are freed
> afterward.
>
>
I'm no expert so there may be something missing from this that I would
be happy to see corrected but the following template classes might do
what you want. I think that any class that derives privately from
Counter<> as shown below will have its counts output.
// Counter.hpp
template<typename T> class CountImpl{
int createCount;
int destructCount;
public:
CountImpl(){
createCount=0;
destructCount=0;
};
~CountImpl(){
std::cout<<"created "
<<typeid(T).name()<<" "<<createCount<<std::endl;
std::cout<<"destroyed "
<<typeid(T).name()<<" "<<destructCount<<std::endl;
}
void create(){createCount++;};
void destroy(){destructCount++;};
};
template<typename T>class Counter
{
static CountImpl<T> myCount;
public:
protected:
Counter(){myCount.create();
};
Counter(const Counter&){
myCount.create();
};
virtual ~Counter(){myCount.destroy();
}
};
template<typename T> CountImpl<T> Counter<T>::myCount;
// End Counter.hpp
//Test.cpp
#include <algorithm>
#include <string>
#include <vector>
#include <iostream>
#include "Counter.hpp"
struct Toto:private Counter<Toto>
{
int id ;
std::string value ;
Toto( int id, std::string const& value )
: id( id )
, value( value )
{
}
} ;
struct Cmp:private Counter<Cmp>
{
bool operator()( Toto const& lhs, int rhs ) const
{
return lhs.id < rhs ;
};
bool operator()(int lhs, Toto const& rhs ) const
{
return lhs< rhs.id ;
}
} ;
struct B:public Toto,private Counter<B>{
B():Toto(1,""){};
};
int main()
{
B test;
std::vector< Toto > v ;
static char const* const
init[] =
{
"one", "two", "three", "four", "five"
} ;
for ( int i = 0 ; i < 5 ; ++ i ) {
v.push_back( Toto( i + 1, init[ i ] ) ) ;
}
std::vector< Toto >::iterator
i = std::upper_bound( v.begin(),
v.end(),
3,
Cmp() ) ;
std::cout << i->value << std::endl ;
return 0 ;
}
Please tell me what I missed.
Otis Bricker
--
Mathias Gaunard
> I would like to be able to do the following test automatically, as part
> of a test suite:
> initializeCounts();
> exerciseTest();
> checkCounts();
> The counts would be things like "class A had 10 instances allocated and
> 10 freed." That is, they are on a class basis. Mostly, I want to be
> sure that all A's created in the course of the test are freed afterward.
The title of your thread was about object creation and destruction.
What you want to track seems to be allocation and deallocation on the
freestore, rather.
For that the best is probably debuggers and the like.
Unfortunetely, since most of those are malloc/free based, they won't
detect if you use delete with a different pointer type than the type
used with new. (which is undefined behaviour in C++, unless the
pointer type is a base class with a virtual destructor)
Note also that if you follow RAII, memory leaks are very unlikely to
happen.
--
Andrew Marlow
> I would like to be able to do the following test automatically, as part
> of a test suite:
> initializeCounts();
> exerciseTest();
> checkCounts();
> The counts would be things like "class A had 10 instances allocated and
> 10 freed." That is, they are on a class basis. Mostly, I want to be
> sure that all A's created in the course of the test are freed afterward.
>
> Any advice?
It sounds like you want to make sure there are no memory leaks. Assuming
your code is portable you could run it on linux using valgrind. That would
tell you.
-Andrew Marlow
--
Ross Boylan
> Ross Boylan wrote:
>
>> I would like to be able to do the following test automatically, as part
>> of a test suite:
>> initializeCounts();
>> exerciseTest();
>> checkCounts();
>> The counts would be things like "class A had 10 instances allocated and
>> 10 freed." That is, they are on a class basis. Mostly, I want to be
>> sure that all A's created in the course of the test are freed afterward.
>>
>> Any advice? In particular, is there some code that would be a good
>> basis for doing such things, and how could it be adapted? The rest of
>> this message summarizes my review so far.
>
> Thus, your requirement is
> a) no major code changes for existing code and
Yes. Modifications, for example using templates, whether with the method
Otis outlined earlier in this thread or those in the "templates,
constructors, and references" thread starting 2007-01-30, get awkward. And
I wouldn't want the code modifications for testing to be part of the
regular version of the program.
> b) keeping track of the class info?
Yes. That's the hard part.
There is also
c) Run automatically from a test suite. I'm using Boost for the test
suite.
This makes firing up a debugger or valgrind cumbersome, though I guess I
could run valgrind outside of boost but as part of my test target (and
check the return value?)
d) Being able to examine particular sections of code. valgrind or most of
the other tools want to operate on a complete program run, and so don't fit
this requirement well out of the box. I also may want check the counts of
allocations in mid-process, to see if I'm reusing objects effectively. A
simple yes/no to memory leaks can't accomplish that.
>
> The problem you're facing is of course that the type information is
> lost as soon as you call an operator new() (it only has the size of the
> class), and changing that requires either to pass in an additional
> parameter, identifying the class, or providing class-specific operator
> news.
There seem to be other ways, because a number of the tools capture some type
info without any modification to the source code. The only one for which
I have identified the mechanism (LeakTracer) does this by capturing the
call stack above the call to new (comments within LeakTracer seem to say
this trick was borrowed from other tools). Purify rewrites the object
code, I believe.
Both of these mechanisms go outside the C++ language spec.
> But both of these require changes in the existing code, or at
> least some macro-magic. Thus, the question goes back to you whether this
> is an option.
>
> Personally, I would rather follow a different route: I would use the
> valgrind tool for testing whether there are any memory leaks (and make
> valgrind runs part of the test suite, of course). It includes the
> construction/allocation point for any leaked memory, and from there it's
> easy to see which class was responsible for the leak. Of course, that's
> some manual work, but that's debugging in first place.
It may be that approach will be good enough, given the apparent hurdles with
a better solution.
>
>> In a thread started Jan 29 I asked about using a template to wrap the
>> class(es) of interest, but as I push on I notice this approach begins to
>> require significant code changes for some tests (e.g., all the spots in
>> my code that say Path would now need to use Path<Node> so that I could
>> use my special counted Node). Since existing tools (mostly memory
>> checkers) already have generic hooks into memory use, maybe one does
>> what I need or could be adapted.
>
> That would require that the checker has an idea for which class this
> memory is used. This is typically not the case, and up to my knowledge
> not achievable with C++ methods alone as the corresponding allocators,
> operator new namely, has no information on the purpose of the class.
See my earlier discussion. The original post noted various tools that
appear to capture some type info.
>> None of the tools I've investigated look quite right, though perhaps
>> some could be adapted. Most are oriented to individual memory
>> allocations rather than the classes doing the allocating. Many work by
>> wrapping an entire run and generating reports, which is awkward for
>> automated testing of a particular chunk of code.
>
> Sorry, I wonder in which sense, or what your test suite is supposed to
> do. Doesn't that imply that you run the program once, entirely?
The test suite does not run the program I'm building; it exercises
particular components of the program and runs explicit tests that the
expected behavior/results occur. The boost test framework produces output
like "Ran n tests, passed m of them" (and lots more details too, if you
want them).
As an example of what I'm currently doing with a template solution, I have a
Factory class that is supposed to be smart about recycling objects. The
rest of the program only gets and releases Nodes through the Factory. So
one test asks the Factory for 20 Nodes, releases them, and then asks for 10
more. The I have a Boost test check that the total number of Nodes
allocated is 20, not 30 (i.e., that I'm recycling the old Nodes).
Ross
P.S. Mathias Gaunard wrote
> The title of your thread was about object creation and destruction.
> What you want to track seems to be allocation and deallocation on the
> freestore, rather.
Buried in the original post was the statement that I'd like to track
creation and destruction, but could live with tracking only allocation and
deallocation. I believe all the tools I cited do only the latter.
Ross
Greg Herlihy
Boylan" <ro...@biostat.ucsf.edu> wrote:
> Thomas Richter wrote:
>
>> Ross Boylan wrote:
>>
>>> I would like to be able to do the following test automatically, as part
>>> of a test suite:
>>> initializeCounts();
>>> exerciseTest();
>>> checkCounts();
>>> The counts would be things like "class A had 10 instances allocated and
>>> 10 freed." That is, they are on a class basis. Mostly, I want to be
>>> sure that all A's created in the course of the test are freed afterward.
>>>
>>> Any advice? In particular, is there some code that would be a good
>>> basis for doing such things, and how could it be adapted? The rest of
>>> this message summarizes my review so far.
>>
>> Thus, your requirement is
>> a) no major code changes for existing code and
> Yes. Modifications, for example using templates, whether with the method
> Otis outlined earlier in this thread or those in the "templates,
> constructors, and references" thread starting 2007-01-30, get awkward. And
> I wouldn't want the code modifications for testing to be part of the
> regular version of the program.
The principal shortcoming of each of two solutions cited was not the use of
a class template - but rather the use of inheritance to implement the
solution. After all, program instrumentation should have as little effect as
possible on the program it is observing - both to ensure the accuracy of the
data collected and to ensure that the instrumentation code itself can be
safely removed in the shipping version of the program. Therefore any
solution that requires temporarily replacing an existing class (or
temporarily changing the class hierarchy of the program) is probably too
significant a change to justify solely on the grounds of program
instrumentation. While it is true that the aim of every program is to be
correct, it is also true that no program has as its primary goal - the
ability to self-test for correctness. So program instrumentation is only
justified as long as it does not interfere with fulfilling the primary
purpose of the program (whatever that purpose may be).
> As an example of what I'm currently doing with a template solution, I have a
> Factory class that is supposed to be smart about recycling objects. The
> rest of the program only gets and releases Nodes through the Factory. So
> one test asks the Factory for 20 Nodes, releases them, and then asks for 10
> more. The I have a Boost test check that the total number of Nodes
> allocated is 20, not 30 (i.e., that I'm recycling the old Nodes).
The solution I would suggest relies on containment (adding a data member)
then on inheritance. Containment is usually a better strategy than
inheritance for adding a new capability to a class (see this Herb Sutter
article for some good supporting arguments:
http://www.gotw.ca/publications/mill06.htm ) In particular, adding a
"ObjectCounter" data member to the class (I believe) would be reasonably
limited in its effects so that it could be justified.
This new data member would simply record its own construction and
destruction as a proxy for its enclosing object's construction and
destruction. So the constructing, copying and assignment of this data member
would be incidental to the construction, copying or assignment of the object
itself. In other words, the data member merely observes those operations
that would take place even if the data member did not exist. In this way the
role of the data member is that of an observer - and not as a participant in
the program's execution.
As noted above, it is important that this instrumentation have a limited
effect upon the program itself. In this regard, the two principal effects of
adding this data member would be: first, a nominal increase in the size of
the class object (because even though the data member class is empty, the
data member must still occupy storage within its object) and second: a
nominal amount of additional overhead when constructing, copying or
assigning an object of the class being instrumented. Now there is no reason
to expect that either of the changes would affect the correctness of the
program itself - or if they did, it would have to be due to an error in the
program itself (such hard-coding the the size of the class object instead of
using the sizeof() operator) and not the fault of the instrumentation code
itself. And as long as the effects of the instrumentation are limited in
this way: removing this data member (in the shipping build of the program)
should be a safe change to make.
One slightly tricky aspect to the implementation of the "ObjectCounter" data
member would be to distinguish properly copy-construction from assignment.
Clearly only the former operation should be counted as construction and the
latter, not. The code below illustrates how a ObjectCounter data member
might be implemented:
#include <iostream>
template <class T>
struct ObjectCounter
{
ObjectCounter()
{
++sObjectCount;
}
ObjectCounter(const ObjectCounter& )
{
++sObjectCount;
}
~ObjectCounter() { --sObjectCount; }
ObjectCounter& operator=(const ObjectCounter&)
{
// do not increment sObjectCount
return *this;
}
static int CountObjects()
{
return sObjectCount;
}
private:
static int sObjectCount;
};
template <class T>
int ObjectCounter<T>::sObjectCount;
// Class of objects to be counted
class A
{
public:
A() : i(0) {}
A(const A&) : i(0) {}
private:
int i;
ObjectCounter<A> counter;
};
using std::cout;
int main(int argc, char *argv[])
{
{
A a1;
A a2;
A a3 = a1; // count as construction
a1 = a2; // do not count as construction
cout << ObjectCounter<A>::CountObjects();
cout << " A objects\n";
}
cout << ObjectCounter<A>::CountObjects();
cout << " A objects\n";
}
Program Output:
3 A objects
0 A objects
Greg