How to test a program
arnuld
>> How do clc folks test a C program ?
>> I have this another thread titled "Array based Binary Heap in C". My
>> test was simple one, just add some elements and remove some and if
>> it works fine, that's great. (That is my usual way of testing)
>> My seniors want me to do automatic test which can add/remove around
>> a million elements without any manuals intervention, the input just
>> have to be configurable. Someone advised to write a Client-Server
>> program using threads to test it where client will send input but I
>> did not like that very complex idea.
>> So I want to know, how do people here at clc test a C program ?
> Volume testing is certainly one important step.
> I'm not sure why you are being advised to involve threads in the test
> harness - that's more likely to introduce new bugs than to uncover
> old ones.
> Test every possible code path. That isn't always easy, and it's
> probably fair to say that it isn't always /possible/, but it should
> be attempted, at least.
> It is at testing time that you discover how useful program scaffolding
> can be. For example, one big problem with testing is "for
> error-handling tests, how do you make a resource fail in exactly the
> place you want?" It can be tricky to get a particular, exact "right
> the hell this one here" malloc call to fail, for example, and yet
> have all the prior and subsequent calls succeed. But a relatively
> trivial wrapper around the allocation routines can give you this
> flexibility (and can be shed for production code).
My boss said, I need to do removal/addition simultaneously, so one thread
will keep on adding while other at the same time will be removing the
elements. I did not get this idea because no matter how much simultaneous
work goes on, program executes only thing at a time, e.g. like http
server gets 1000 requests and it may accept/reply to first requestin 1
nanosecond while 2nd at 10th nanosecond and 3rd at 2nd nanosecond. It
will be so fast that it will look like simultaneous work but server will
be handling everything individually. Hence I don't see any point in
making 1 threads, I think a a for loop with 1 million iterations will do
the same job with wayyyyy... less complexity.
I can use 2 for loops, while outer for loop iterates 1000 times, inner
iteration 1 million times for each outer iteration. What say ?
--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
"Usenet is a strange place" - dmr 29 July 1999
Sig line vacant - apply within
--
www.lispmachine.wordpress.com
my email is @ the above blog.
Pascal J. Bourguignon
> My boss said, I need to do removal/addition simultaneously, so one thread
> will keep on adding while other at the same time will be removing the
> elements. I did not get this idea because no matter how much simultaneous
> work goes on, program executes only thing at a time, e.g. like http
> server gets 1000 requests and it may accept/reply to first requestin 1
> nanosecond while 2nd at 10th nanosecond and 3rd at 2nd nanosecond. It
> will be so fast that it will look like simultaneous work but server will
> be handling everything individually. Hence I don't see any point in
> making 1 threads, I think a a for loop with 1 million iterations will do
> the same job with wayyyyy... less complexity.
Haven't you heard about multi-core processors?
http://en.wikipedia.org/wiki/Multi-core
When you have several processors in a computer, or a multi-core
processor, several threads run at the same time, and can indeed do
things in real parallelism.
When you write a multi-threaded program, even on a single processor,
you can still consider that the threads run in parallel, exactly like
with multi-processor systems, because interruptions may occur at any
time, and therefore you can have the same kind of bad thread
interactions in both cases.
You should read about multithreading programming.
https://computing.llnl.gov/tutorials/pthreads/
--
__Pascal Bourguignon__
arnuld
> Haven't you heard about multi-core processors?
> http://en.wikipedia.org/wiki/Multi-core
> .. SNIP...
I don't get If I ask one thread to add an int x = 2; and at the same time
ask another thread to remove it, what will happen, it add/remove half of
it.
I don't think so, I think it will be added and then removed, as i said,
may be the difference between 2 activities is 10000000th fraction of
second.
Pascal J. Bourguignon
>> On Tue, 03 Nov 2009 14:04:29 +0100, Pascal J. Bourguignon wrote:
>
>> Haven't you heard about multi-core processors?
>> http://en.wikipedia.org/wiki/Multi-core
>> .. SNIP...
>
> I don't get If I ask one thread to add an int x = 2; and at the same time
> ask another thread to remove it, what will happen, it add/remove half of
> it.
>
> I don't think so, I think it will be added and then removed, as i said,
> may be the difference between 2 activities is 10000000th fraction of
> second.
You're wrong. It may happen that it is only added. Or only removed.
Or worse, on some computers, you may get mixed bits in the result.
Compiling:
x = x + 1;
we may get instructions such as:
load x into register A
add 1 to register A
store register A into x
Compiling:
x = x - 1;
we may get instructions such as:
load x into register A'
subtract 1 from register A'
store register A' into x
(I noted A and A' to differenciate them because while the register may
be the same, when the different threads are running each gets its own
instances of the registers).
Now, when the threads run in parallel, you may get both threads
ordered in a lot of different ways, including these:
load x into register A ; t1
add 1 to register A ; t1
store register A into x ; t1
load x into register A' ; t2
subtract 1 from register A' ; t2
store register A' into x ; t2
<=> assert(x==alpha); x=x+1; x=x-1; assert(x==alpha);
load x into register A' ; t2
load x into register A ; t1
add 1 to register A ; t1
store register A into x ; t1
subtract 1 from register A' ; t2
store register A' into x ; t2
<=> assert(x==alpha); a2=x; x=x+1; x=a2-1; assert(x==alpha-1);
load x into register A ; t1
load x into register A' ; t2
subtract 1 from register A' ; t2
store register A' into x ; t2
add 1 to register A ; t1
store register A into x ; t1
<=> assert(x==alpha); a1=x; x=x-1; x=a1+1; assert(x==alpha+1);
There is also a problem with the store register. If you have a
multi-core or multi-processor system, then it may happen that storing
a multi-byte value such as x occurs in several bus cycles, and
therefore when two processors access the memory bus, they may store
each a part of the value. If x = 0x0000ffff then x+1 =0x00010000 and
x-1 = 0x0000fffe, and you may get as final result either 0x0000ffff,
0x00010000, 0x0000ffffe, 0x00000000, or 0x0001fffe!
In particular even on architectures where there are 32-bit, or even
64-bit or 128-bit atomic bus accesses, you can still get this kind of
problems when you store bigger structures, or when you store data
across alignment boundaries.
--
__Pascal Bourguignon__
Patricia Shanahan
....
> My boss said, I need to do removal/addition simultaneously, so one thread
> will keep on adding while other at the same time will be removing the
> elements. I did not get this idea because no matter how much simultaneous
> work goes on, program executes only thing at a time, e.g. like http
> server gets 1000 requests and it may accept/reply to first requestin 1
> nanosecond while 2nd at 10th nanosecond and 3rd at 2nd nanosecond. It
> will be so fast that it will look like simultaneous work but server will
> be handling everything individually. Hence I don't see any point in
> making 1 threads, I think a a for loop with 1 million iterations will do
> the same job with wayyyyy... less complexity.
>
> I can use 2 for loops, while outer for loop iterates 1000 times, inner
> iteration 1 million times for each outer iteration. What say ?
I think it depends on the answer to two questions:
1. Is the program under test multi-threaded?
2. Is the program under test required to keep track of multiple sources
of requests, and return results to them?
If the answer to either of those questions is "yes" then I think
multi-threaded driving is a good idea. If the answer to both is "no", so
the program under test is a simple single threaded, single request
source program, then I would use a single driver.
Note that "multi-threaded" in question 1 would include some non-obvious
cases, such as a transaction-orientated web server that runs under a
framework that uses multiple threads to run the transactions.
Patricia
Alan Morgan
arnuld <sun...@invalid.address> wrote:
>> On Tue, 03 Nov 2009 14:04:29 +0100, Pascal J. Bourguignon wrote:
>
>> Haven't you heard about multi-core processors?
>> http://en.wikipedia.org/wiki/Multi-core
>> .. SNIP...
>
>I don't get If I ask one thread to add an int x = 2; and at the same time
>ask another thread to remove it, what will happen, it add/remove half of
>it.
If you are very, very lucky the code will crash. If you are unlucky
it will continue on, not giving any indication that there is a problem.
The problem, you see, is that "add an element" is not an atomic operation.
It requires several steps. So does "remove an element". What happens
if these steps are interleaved? Again, if you are really, really lucky
it will throw some sort of exception or otherwise die horribly.
>I don't think so, I think it will be added and then removed, as i said,
>may be the difference between 2 activities is 10000000th fraction of
>second.
This is true until you install the software on a client's machine. At
the point it will die immediately, with extreme prejudice. The machine
probably won't burst into flames, but I wouldn't rule it out.
Alan
--
Defendit numerus
Pascal J. Bourguignon
> My boss said, I need to do removal/addition simultaneously, so one thread
Also, while I don't see the relationship between this question and the
subject (perhaps you should have changed the subject?), testing
multithreaded programs can be done with: valgrind --tool=helgrind
which will give you indications where unprotected concurrent access to
memory are done.
When you have several thread accessing the same memory, you need to
protect it with a semaphore, a mutex, or some other synchronization
primitive.
--
__Pascal Bourguignon__