Thread control (initially posted to dev by mistake)

[ilya sterin]

Cedric, just starting a thread here per your TheServerside suggestion.

Here is what was discussed...

------------------------------------------------------------------------------------------------------

Yeah, it would be nice for more fine grained thread control.

Right now, the threads and invocation don't really have a 1 to 1
mapping, so although there are three threads, there is no guarantee
that say 2 invocations will run in the same thread, etc... It would be
nice to add such a feature.

@Test(threadPoolSize = 3, invocationCount = 6, threadMap =
{2, 2, 2})

Basically stating that each thread has 2 invocations each. I'm sure
there is a cleaner, friendlier way of doing this, with maybe more
flexibility, but in general, we need to control thread/invocation
combos.

Ilya

------------------------------------------------------------------------------------------------------
I added a few functionalities recently to enable some of this. For
example, you can specify that all the test methods inside a class
should be run in the same thread, even if the current suite is being
run in parallel (@Test(sequential = true)).

This was necessary for users who want to run their tests multithreaded
for the speed gain, but who sometimes have to test methods that are not
multithread-safe. Now, they can simply put these methods in a class
that has the above attribute and TestNG will run them in the same
thread.

As for your request and the threadMap, let me think about that (and
let's take the discussion to the testng-users mailing-list).

--
Cedric

[Cédric Beust ♔]

Thanks, Ilya.

I'd like to take this opportunity to see what kinds of improvements people would like to see in the thread support.  In particular, have you ever needed to limit the number of invocations for certain threads, as Ilya is suggesting?

Ilya, can you explain in what situation you need this?

--
Cedric

[Alexandru Popescu]

(initially answer posted on dev):

lya the idea sounds interesting, but I would definitely like to hear
more from you about the scenario you are trying to achieve. Lately, I
have done lots of tests based on TestNG parallel mode (both for
concurrency issues and also to simulate load testing), but never felt
the need to know what threads are involved as long as I was sure they
are used :-).

./alex
--
.w( the_mindstorm )p.
TestNG co-founder
EclipseTestNG Creator

[Ilya Sterin]

Cedric, Alexandru, sorry for the late response, I've been traveling.  I have a few use cases for what I described below and I'll post them here shortly today.

Thanks.

Ilya Sterin

[Ilya Sterin]

Ok, I'm back in town, and have some time to dedicate to maybe helping you guys with some use cases for this...

So, basically the initial use case for me to allow more fine grained thread control was the fact that I noticed that say, when I specified 3 threads and 6 invocations, there was no guarantee of 2x2, though it prevented me from writing sort of a deterministic test, which relied on a particular invocation for thread.

Of course, I could have structured my test in multiple ways, but I was basically testing an ObjectManager for my framework, which was a ThreadLocal implementation.  The idea was that I would run 2 invocations per thread and contain them in some sort of a class based data structure, which would then get evaluated in the following test method.  So each entry in a collection would basically equal to a thread and contain some other structure that would contain some reference(s) to the ObjectManager and/or it's hash code.  I would then ensure that there were no more than one ObjectManager per thread and that each thread had a reference to a different ObjectManager object.

I'm not sure if this was the best way to structure this particular test, but it basically demonstrates a test method that relies upon a fine grained control of the threaded invocations.  If say, it executed three threads, but the third thread only received one invocation, it would make the test fail, since there was nothing to compare ( i.e. t3om[0] == t3om[1]).  I at least know that at least one thread would get more than one invocation, but that IMO wouldn't be enough if I wanted to run multiple scenarios within the same test method, rather than executing multiple test methods that test the same object manager.

Ok, am I speaking foreign here:-)  Sorry, I'm sometimes not good at explaining what seems easy to comprehend for me to others.  Just let me know with a "What in the world are you talking about?", and I'll try to clear things up).

Ilya

[Alexandru Popescu]

Ilya, I must confess that I had read your message a couple of times
before posting the following comments, so please bare with me.

- in most of the cases I have used this feature, I wasn't interested
in the allocation of the threads and their dependencies; I was rather
testing the behavior under concurrent access
- in case I would need to be aware of the way threads are allocated I
would most probably use more complex test scenarios using parallelism
on multiple methods, groups and before/after mechanisms

Also, after reading your message I have got the impression that you
would like to have this advanced feature inside TestNG to cover a very
specific test you need to write (and for which there may be other
approaches available).

Hope I haven't completely misunderstood your scenario,