Corrected and Supplemented -> Re: STL and Usual C : Average time-cost of execution
Alex Vinokur
David Wragg <d...@doc.ic.ac.uk> writes:
> Alex Vinokur <al...@tibam.elex.co.il> writes:
> > Several functions (methods, operators) were measured.
> >
> > --------------------------------------
> > C-functions | STL-methods (operators)
> > --------------------------------------
> > strcpy | operator= (string)
> > strcat | operator+= (string)
> > strlen | size () (string)
> > malloc&free | new&delete (string)
> > --------------------------------------
> >
> > Average time-cost of execution was obtained.
> >
>
> [snip code]
>
[snip]
>
> > Test#1 [malloc&free ] : Average time = 4680 nsec
> > Test#1 [STL : new&delete] : Average time = 2671 nsec
>
> Your test didn't actually do a new/delete. It just did a
> constructor/destructor call. You probably want to time something like
>
> delete (new String("ABCD"));
>
> instead. (Though this will include the constructor/destructor cost as
> well as for new/delete.)
>
[snip]
=============================================================
Michael J. Tobler <mto...@ibm.net> writes :
>
> In article <36822EF0...@tibam.elex.co.il>,
al...@tibam.elex.co.il
> says...
> [snip]
>
> You cant be conclusive about your results:
>
[snip]
>
> >
> > Test#1 [malloc&free ] : Average time = 4680 nsec
> > Test#1 [STL : new&delete] : Average time = 2671 nsec
>
> new and delete utilize malloc() and free() on some implementations, so
> you are already recording the result of those functions with calls to
new
> and delete
>
[snip]
=============================================================
My colleagues are right.
Thank you very much.
Here are the corrected and supplemented results of the experiment.
The following functions (methods, operators) were measured.
----------------------------------------------------
Standard C Library | STL
----------------------------------------------------
char* | constructor&destructor (string)
char-array | constructor&destructor (string)
malloc&free | new&delete (string)
(malloc+init)&free | new&delete (string)
----------------------------------------------------
Average time-cost of execution was obtained.
Alex
//##########################################################################
//################# 1. Corrected and supplemented program
##################
//##########################################################################
//=================================================
#include <iostream>
#include <string>
#include <sys/time.h>
//##########################################################################
int main()
{
#define PRINT(x, y, z) cout << "Test#" << x << " [" << y << "]" << z <<
"\t: "
<< "\tAverage time = " << (time_end - time_start) / TOTAL_Iterations <<
" nsec"
<< endl
#define PRINTS(x, y, z) PRINT(x, y, z) << endl
//=======================
const int TOTAL_Tests = 5;
const int TOTAL_Iterations = 1000;
//===========
hrtime_t time_start;
hrtime_t time_end;
//===========
//===============
//======================
for (int theTest = 1; theTest <= TOTAL_Tests; theTest++)
{
//=======================
cout << endl << "\t" << "##### Test#" << theTest << "
#####" <<
endl;
//===============================
//======= char* ================
time_start = gethrtime();
for (int theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
{
char* dummy = "ABCD";
} // for (theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
time_end = gethrtime();
PRINT (theTest, "char*", "\t\t");
//===============================
//======= char-array ================
time_start = gethrtime();
for (int theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
{
char dummy[] = "ABCD";
} // for (theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
time_end = gethrtime();
PRINT (theTest, "char-array", "\t");
//===============================
//======= constructor&destructor ================
time_start = gethrtime();
for (int theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
{
string ("ABCD");
} // for (theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
time_end = gethrtime();
PRINTS (theTest, "STL : constructor&destructor", "");
//===============================
//======= malloc ================
time_start = gethrtime();
for (int theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
{
free ((char *) malloc (5));
} // for (theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
time_end = gethrtime();
PRINT (theTest, "malloc&free", "\t");
//===============================
//======= malloc+init ================
time_start = gethrtime();
for (int theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
{
char* dummy = (char*) malloc (5);
strcpy (dummy, "ABCD");
free (dummy);
} // for (theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
time_end = gethrtime();
PRINT (theTest, "(malloc+init)&free", "");
//===============================
//======= new&delete ================
time_start = gethrtime();
for (int theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
{
delete (new string ("ABCD"));
} // for (theIteration = 1; theIteration <=
TOTAL_Iterations;
theIteration++)
time_end = gethrtime();
PRINT (theTest, "STL : new&delete", "");
//===============================
} // for (theTest = 1; theTest <= TOTAL_Tests; theTest++)
//======================
return 0;
}
//##########################################################################
//################# 2. Platform etc
########################################
//##########################################################################
SunOS 5.6 Generic_105181-09 sun4m sparc SUNW,SPARCstation-5
gcc version 2.8.1
//##########################################################################
//################# 3. Corrected and supplemented results of experiment
####
//##########################################################################
##### Test#1 #####
Test#1 [char*] : Average time = 133 nsec
Test#1 [char-array] : Average time = 751 nsec
Test#1 [STL : constructor&destructor] : Average time = 3226 nsec
Test#1 [malloc&free] : Average time = 4735 nsec
Test#1 [(malloc+init)&free] : Average time = 4573 nsec
Test#1 [STL : new&delete] : Average time = 6947 nsec
##### Test#2 #####
Test#2 [char*] : Average time = 131 nsec
Test#2 [char-array] : Average time = 750 nsec
Test#2 [STL : constructor&destructor] : Average time = 3604 nsec
Test#2 [malloc&free] : Average time = 4070 nsec
Test#2 [(malloc+init)&free] : Average time = 5863 nsec
Test#2 [STL : new&delete] : Average time = 6867 nsec
##### Test#3 #####
Test#3 [char*] : Average time = 131 nsec
Test#3 [char-array] : Average time = 826 nsec
Test#3 [STL : constructor&destructor] : Average time = 3059 nsec
Test#3 [malloc&free] : Average time = 4066 nsec
Test#3 [(malloc+init)&free] : Average time = 4809 nsec
Test#3 [STL : new&delete] : Average time = 7244 nsec
##### Test#4 #####
Test#4 [char*] : Average time = 132 nsec
Test#4 [char-array] : Average time = 751 nsec
Test#4 [STL : constructor&destructor] : Average time = 3060 nsec
Test#4 [malloc&free] : Average time = 4142 nsec
Test#4 [(malloc+init)&free] : Average time = 4490 nsec
Test#4 [STL : new&delete] : Average time = 7092 nsec
##### Test#5 #####
Test#5 [char*] : Average time = 130 nsec
Test#5 [char-array] : Average time = 751 nsec
Test#5 [STL : constructor&destructor] : Average time = 3058 nsec
Test#5 [malloc&free] : Average time = 4138 nsec
Test#5 [(malloc+init)&free] : Average time = 4489 nsec
Test#5 [STL : new&delete] : Average time = 7026 nsec
//##########################################################################
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
[ about comp.lang.c++.moderated. First time posters: do this! ]
Jerry Coffin
alexande...@telrad.co.il says...
>
> David Wragg <d...@doc.ic.ac.uk> writes:
> > Alex Vinokur <al...@tibam.elex.co.il> writes:
> > > Several functions (methods, operators) were measured.
> > >
> > > --------------------------------------
> > > C-functions | STL-methods (operators)
> > > --------------------------------------
> > > strcpy | operator= (string)
> > > strcat | operator+= (string)
> > > strlen | size () (string)
> > > malloc&free | new&delete (string)
> > > --------------------------------------
> > >
> > > Average time-cost of execution was obtained.
[ results and code elided ]
I decided to rewrite your code to add support for Win32. While I was
doing that, I did a bit of rewriting on the rest of the code to put it
in an style I liked better. I made no attempt at preserving the same
format in the output, though that should be easy enough to do if
anybody really cares. This code should make it a bit easier to add
more tests, if desired. The code presently assumes that the standard
library is in the std namespace, where it belongs. I've added a macro
to define `std' to nothing if the code isn't compiled for Win32 -- if
anybody cares enough about the code, they'll want to fix this as time
goes on and more compilers put the standard library in the correct
namespace.
Other than assuming that the standard library is where it belongs and
that headers have their correct names, I believe nearly every compiler
around implements all the features this code uses. With most
implementations, the latter is pretty trivial to handle: for example,
simply create a file named `iostream' that just includes `iostream.h'.
//=================================================
#include <iostream>
#include <string>
#include <vector>
#include <algorithm>
#ifdef WIN32
#include <windows.h>
#else
#include <sys/time.h>
#define std
#endif
//####################################################################
######
#ifdef WIN32
typedef LARGE_INTEGER hrtime_t;
inline hrtime_t gethrtime() {
hrtime_t now;
QueryPerformanceCounter(&now);
return now;
}
inline double gethrres() {
hrtime_t res;
QueryPerformanceFrequency(&res);
return double(res.QuadPart);
}
inline double operator-(LARGE_INTEGER &a, LARGE_INTEGER &b) {
return double(a.QuadPart - b.QuadPart);
}
#endif
class test {
virtual void run(unsigned iters) const = 0;
virtual char *name() const = 0;
public:
virtual std::ostream &print(std::ostream &str) const {
const int iterations = 100000;
hrtime_t start = gethrtime();
run(iterations);
hrtime_t end = gethrtime();
str << name() << " took: "
<< ((end-start)/gethrres())/iterations
<< " seconds." << std::endl;
return str;
}
static void do_run(test const *&t) {
t->print(std::cout);
}
static void killer(test const *&t) {
delete t;
}
virtual ~test() {}
};
// Code for the individual tests. The individual test only has
// to implement the test code itself and provide a printable name
// for the test. In theory, even this could be reduced somewhat, by
// putting the loop in the base class, and having the derived class
// implement only a single iteration. However, given the speed of
// some operations in question, the overhead of a large number of
// virtual function calls could make differences in the operations
// themselves less apparent. With sufficient work, the base class
// could attempt to factor that overhead out, but this seemed an
// easier approach -- the effort in implementing a derived class
// remains extremely minimal beyond the effort of designing the few
// lines of code being tested.
class char_star : public test {
virtual char *name() const { return "Pointer to char"; }
virtual void run(unsigned iters) const {
for (int i = 1; i <= iters; i++)
char* dummy = "ABCD";
}
};
class char_array : public test {
virtual char *name() const { return "Array of char"; }
virtual void run(unsigned iters) const {
for (int i = 1; i <= iters; i++)
char dummy[] = "ABCD";
}
};
class ctor : public test {
virtual char *name() const { return "string
Constructor/Destructor"; }
virtual void run(unsigned iters) const {
for (int i= 1; i <= iters; i++)
std::string ("ABCD");
}
};
class use_malloc : public test {
virtual char *name() const { return "malloc/free"; }
virtual void run(unsigned iters) const {
for (int i=0; i<iters; i++)
free ((char *) malloc (5));
}
};
class mallocinit : public test {
virtual char *name() const { return "malloc and init"; }
virtual void run(unsigned iters) const {
for (int i=0; i<iters; i++) {
char* dummy = (char*) malloc (5);
strcpy (dummy, "ABCD");
free (dummy);
}
}
};
class newdelete : public test {
virtual char *name() const { return "new and delete"; }
virtual void run(unsigned iters) const {
for (int i=0; i<iters; i++)
delete new std::string("ABCD");
}
};
int main() {
// This will hold the objects that represent the individual
// tests run in the set.
std::vector<test *> tests;
// The following block of code is the only thing that knows about
// _all_ the blocks of code to be run. To add a new test, about
// all you should need to do is add a new class (derived from test)
// and add a line here to put a pointer to an instance of your new
// class into `tests'.
tests.push_back(new char_star);
tests.push_back(new char_array);
tests.push_back(new ctor);
tests.push_back(new use_malloc);
tests.push_back(new mallocinit);
tests.push_back(new newdelete);
// Run all the tests.
std::for_each(tests.begin(), tests.end(), test::do_run);
// and destroy all the test objects.
std::for_each(tests.begin(), tests.end(), test::killer);
return 0;
}
As-is, I haven't attempted to include anything to show variability.
If I were going to add it, I'd do so in `test' itself, showing the
standard deviation of all the tests run, rather than running several
complete sets of tests and showing the raw results for each set --
this should provide results that are both more comprehensive and
comprehensible.