JUnit has a nice @RunWith annotation that enables to specify a test
runner.
I've been doing lots of testing with Guice and TestNG and I have to
extend a base class all the time - where I have a @BeforeClass,
@AfterClass, @BeforeMethod and @AfterMethod.
I'm not certain that specifying a test runner is the most elegant
solution, probably in most cases you may simply want to have
@BeforeClass, @AfterClass, @BeforeMethod and @AfterMethod in a different
class.
What I would really like is something like this

@Test
@RunWith(GuiceInitializer.class) // also be able to specify this
globally for all my tests
class A {
   @Inject
   private Database db;
...

class GuiceInitializer {

  @BeforeClass
  public void initTestClass(Object testInstance) {
        ...
        injector.injectInstance(testInstance);
  }

Any thoughts?

Erik.

I've often thought this would be good, one way you could implement it
already without upstream changes would be having a base test class implement
IHookable, and have the associated
run(org.testng.IHookCallBack,org.testng.ITestResult iTestResult) method look
for a @RunWith annotation on the method, and delegate as necessary.

I've not tried this but it should work...

On Wed, Jan 21, 2009 at 9:17 AM, Putrycz, Erik
<Erik.Putr...@nrc-cnrc.gc.ca>wrote:

The only benefit I can see is that it lets you extend a different class if
you so desire, but I'm not really convinced it's really that useful.
Let me ask you this:  can you think of scenarios where the annotations that
you need cannot be put on the class you extend?

Besides, @RunWith is also a headache for integration since most IDE's will
ignore it and they will therefore not make these symbols available to the
plug-in...

--
***Cédric
*

On Tue, Jan 20, 2009 at 12:17 PM, Putrycz, Erik <Erik.Putr...@nrc-cnrc.gc.ca

I don’t think there is any case where you cannot put the annotations in a parent class, or use several levels of inheritance. It’s more a matter of organization, I’d like to have a generic Guice testing framework and use the inheritance to group functions and my own annotations for each of my modules.

Why do you think @RunWith would be a headache for integration? Aren’t they all only detecting the @Test and then hand the execution to the test runner? In my example, you won’t pass a test runner to @RunWith, rather somekind of interceptor – so the IDE shouldn’t need to handle it.

Erik.

________________________________

From: testng-dev@googlegroups.com [mailto:testng-dev@googlegroups.com] On Behalf Of Cédric Beust ?
Sent: January-20-09 3:43 PM
To: testng-dev@googlegroups.com
Subject: Re: @RunWith annotation

The only benefit I can see is that it lets you extend a different class if you so desire, but I'm not really convinced it's really that useful.

Let me ask you this:  can you think of scenarios where the annotations that you need cannot be put on the class you extend?

Besides, @RunWith is also a headache for integration since most IDE's will ignore it and they will therefore not make these symbols available to the plug-in...

--

C餲ic

JUnit has a nice @RunWith annotation that enables to specify a test
runner.
I've been doing lots of testing with Guice and TestNG and I have to
extend a base class all the time - where I have a @BeforeClass,
@AfterClass, @BeforeMethod and @AfterMethod.
I'm not certain that specifying a test runner is the most elegant
solution, probably in most cases you may simply want to have
@BeforeClass, @AfterClass, @BeforeMethod and @AfterMethod in a different
class.
What I would really like is something like this

@Test
@RunWith(GuiceInitializer.class) // also be able to specify this
globally for all my tests
class A {
  @Inject
  private Database db;
...

class GuiceInitializer {

 @BeforeClass
 public void initTestClass(Object testInstance) {
       ...
       injector.injectInstance(testInstance);
 }

Any thoughts?

Erik.

One place I've seen this pattern somewhat useful is when mixing some other
test frameworks, such as concordion.  In this scenario, theres
common/standard before/after boiler plate code that just initializes an
environment for the test.  A more 'testng' like approach would be say an
@AroundMethod annotation that sat on a method conforming to IHookable's
run() method parameters.  It would just be a more succinct way of expressing
BeforeMethod/AfterMethod(runalways=true) patterns.

I could see this working better when using groovy/scala traits/mixins.  But
could also complicate things more.

On Wed, Jan 21, 2009 at 9:43 AM, Cédric Beust ♔ <cbe...@google.com> wrote:

2009/1/20 Putrycz, Erik <Erik.Putr...@nrc-cnrc.gc.ca>

Ok.

The problem is that @RunWith is basically a way to substitute your own
runner to whatever other is currently in place.  IDE's typically use their
own runner to keep track of progress, so you usually lose a lot of IDE
functionality as soon as you use @RunWith in your tests.

The way I see it, JUnit wasn't designed with enough listening and hooking
infrastructure in place, so they had to resort to @RunWith to give at least
some flexibility to developers.  "Our engine is a black box, but you can
replace it completely".

Fixing this was one of my early goals with TestNG, which I achieved by
providing not just more configuration annotations but also more listeners
and hooks into the engine so that its behavior can be modified and
overridden without compromising tooling.

--
***Cédric
*

So instead of a @RunWith maybe specifying the listeners and hooks with annotations would do the trick (I never use the suite.xml files)?

I haven’t found in the listeners something that would correspond to a @BeforeClass but if there is something equivalent, this might be a better way.

Something such as @Listeners(GuiceInitializer.class)

Erik.

________________________________

From: testng-dev@googlegroups.com [mailto:testng-dev@googlegroups.com] On Behalf Of Cédric Beust ?
Sent: January-20-09 4:11 PM
To: testng-dev@googlegroups.com
Subject: Re: @RunWith annotation

2009/1/20 Putrycz, Erik <Erik.Putr...@nrc-cnrc.gc.ca>

        Why do you think @RunWith would be a headache for integration? Aren't they all only detecting the @Test and then hand the execution to the test runner? In my example, you won't pass a test runner to @RunWith, rather somekind of interceptor so the IDE shouldn't need to handle it.

The problem is that @RunWith is basically a way to substitute your own runner to whatever other is currently in place.  IDE's typically use their own runner to keep track of progress, so you usually lose a lot of IDE functionality as soon as you use @RunWith in your tests.

The way I see it, JUnit wasn't designed with enough listening and hooking infrastructure in place, so they had to resort to @RunWith to give at least some flexibility to developers.  "Our engine is a black box, but you can replace it completely".

Fixing this was one of my early goals with TestNG, which I achieved by providing not just more configuration annotations but also more listeners and hooks into the engine so that its behavior can be modified and overridden without compromising tooling.

--

C餲ic

2009/1/20 Putrycz, Erik <Erik.Putr...@nrc-cnrc.gc.ca>

This would be easy to add.  The pro is that it would benefit from static
validation, the con is that you need to recompile whenever you want to add
or remove a listener.

I'm still not convinced it's a big gain over just specifying these listeners
in the base class, though...

--
***Cédric
*

IDE runners don't have to be aware of @RunWith except that they need
to
recognize classes with this annotation as JUnit tests. JUnit will do
the
rest. Sorting and filtering tests should still work, unless the
user-provider runner doesn't support it.

Is it possible to write a custom runner for TestNG that gives me full
control over test execution like JUnit does? Last time I checked, I
couldn't
find a way to do this.

Cheers,
Peter

On Jan 20, 10:11 pm, Cédric Beust ♔ <cbe...@google.com> wrote:

I think what Cedric meant is that if something such as @RunWith was
added, all the IDE plugins would need to "recognize" it whereas changes
in the test runner only are much easier for all the IDE plugin
maintainers.

Could you detail what you need custom runner for?
I looked at the JUnit Runner API and it doesn't look too friendly. There
should be better ways to customize the test execution.

Erik.

On Wed, Jan 21, 2009 at 7:55 AM, Putrycz, Erik
<Erik.Putr...@nrc-cnrc.gc.ca>wrote:

Exactly.  With @RunWith, developers writing runner classes now compete with
IDE's, and by taking away the IDE's ability to run the tests themselves,
they will lose functionalities such as runtime reporting, stack trace
displaying, etc...

I would rather explore the possibility where IDE's and runner developers
alike can develop their own piece of code and they can all work together
instead of one replacing the other.

I was going to ask just that to Peter:  please explain what you are trying
to do and let's see what we can do.

--
***Cédric
*

Just to make sure that we are on the same page: For IDEs that use JUnit in
the way it is intended to be used, @RunWith is totally transparent: IDE
calls JUnit calls @RunWith runner. As of JUnit 4.5, this should take you
pretty far. I'm wondering if it's still necessary to "abuse" JUnit to offer
good IDE integration, but I'm not an expert here.
From the standpoint of a framework developer, the good thing about @RunWith
is that it gives you a lot of flexibility. The bad thing is that it isn't
obvious how to write a fully conforming runner, and that you have to
reinvent the wheel even if you only need a slight variation of the standard
runner. That's why JUnit 4.5 has introduced an API that facilitates the
writing of runners that, to some extent, follow the JUnit model.

I'm writing my own test framework and would like to reuse TestNG's IDE and
build tool integration. Not sure if you consider this a valid use case. :-)
Anyway, @RunWith works pretty well for me, but I haven't found a way to
integrate with TestNG.

Cheers,
Peter
--
View this message in context: http://www.nabble.com/%40RunWith-annotation-tp21570801p21590053.html
Sent from the testng-dev mailing list archive at Nabble.com.

Just to make sure that we are on the same page: For IDEs that use
JUnit in the way it is intended to be used, @RunWith is totally
transparent: IDE calls JUnit calls @RunWith runner. As of JUnit 4.5,
this should take you pretty far. I'm wondering if it's still necessary
to "abuse" JUnit to offer good IDE integration, but I'm not an expert
here.
From the standpoint of a framework developer, the good thing about
@RunWith is that it gives you a lot of flexibility. The bad thing is
that it isn't obvious how to write a fully conforming runner, and that
you have to reinvent the wheel even if you only need a slight
variation of the standard runner. That's why JUnit 4.5 has introduced
an API that facilitates the writing of runners that, to some extent,
follow the JUnit model.

I'm writing my own test framework and would like to reuse TestNG's IDE
and build tool integration. Not sure if you consider this a valid use
case. :-) Anyway, @RunWith works pretty well for me, but I haven't
found a way to integrate with TestNG.

Cheers,
Peter

On Wed, Jan 21, 2009 at 11:08 AM, Peter Niederwieser <pnied...@gmail.com>wrote:

Yeah, that's probably a bit out there for a valid use case :-)

Still, I'm curious:  what kind of functionalities do your @RunWith runners
accomplish?

--
***Cédric
*

So what is specific about your test framework?

Erik.

okay, i have to warm up this discussion because this is exactly what
we (Support TestNG in Pax Exam).

What we currently can do with Junit is this:
@RunWith(OurInvoker.class)
class Foo
@Test
public void bla()
{

@OurSpecialConfigutationAnnotationStuff
public Class configure {
.. bla

What we do when in OurInvoker:
- recognize all tests, annotations (junit specifics + our specifics)
- wrap the test classes up, make a jar (osgi bundle to be specific)
for each test do that:
- invoke a totally fresh container in a new VM (container= osgi
runtime, but you can think of any container you might think of, say, a
tomcat instance)
- invoke the execution of test via rmi

There's a lot stuff you don't have to think about now, but just the
use case for having a RunWith interceptor.
And Eclipse,Maven and IntelliJ at least recognize this RunWith in case
of Junit to let us intercept the test invokation.

Sure, it is up to the implementor of OurInvoker to delegate as much
reporting / statistics to the real runner (test ng).
So, @Cédric, is there something you can point to in TestNG to support
this in an extensible, futuresafe way without "hacking testng" ?

Hi Toni,

I can think of several ways you can do this.

The easiest one is to create your own TestNG object, do all your
configuration and annotation finding there, add whatever listeners you need
and then run it.

Another way would be to use an ITestListener and do this before the suite
begins.

Let me know if you need help with either of these.

--
Cedric

--
***Cédric
*