@Fixture annotation?

[Barry Kaplan]

I have a few fixtures which contains methods that need to be annotated @Configurable(before...). Currently, I must do something like:

public class MyTest {

private MyFixture myFixture = new ...

@Configurable(beforeTestClass=true)
public void setup() {
myFixture.setup();
}

@Configurable(afterTestClass=true)
public void teardown() {
myFixture.teardown();
}
...
}

But What I would really like to do is:

public class MyTest {

@Fixture
private MyFixture myFixture = new ...
...
}

Where the fixture's methods are annotated with the appropriate @Configurable.

I have yet to look at any of the testng source. Would such a thing be possible?

-barry

PS, Can I tag a code block in jive?
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=33349#33349

[Alexandru Popescu]

#: Barry Kaplan changed the world a bit at a time by saying (astral date: 2/4/2006 7:43 PM) :#

Hi!

(pls check my previous answer about asking what is really a fixture). Than I cannot understand what
is the above scenario trying to do. Can you please detail?

As regards tagging code, I think you can use [ code ] [ / code ] (without spaces).

cheers,

./alex
--
.w( the_mindstorm )p.

[Barry Kaplan]

The way I am using it, a "fixture" is a class that contains only test setup code. ie, methods that are annoated only with @Configuration(before/after*=). Or maybe its better to say no methods that are annoated with @Test.

So what I am trying to get is to not have to create @Configuration(befofe/after) methods in the test itself that do nothing other than delegate to the "fixture" methods. I'm looking for a more declarative style to say "please add this aspect of fixture to my test". I don't want to put this in a base test class, because then I'm back the old junit way of having a chain of base classes to add in all the test fixture aspects.

Ok, I think could support this myself using a single base class with before/after methods that scan the current test for variables annotated with @Fixture, and then scan that class for methods with @Configuration... But then I would be duplicating the code that already exists to detect these methods and add them to the ordered collection of configuration methods.

If I could find a way to get access to the test-result, then I could define a parameter-provider to poke in the test-result to the configuration method.

Well, since I'm just right now looking at the impl of testng, there may be lots of better ways that I will discover.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=33365#33365

[Alexandru Popescu]

#: Barry Kaplan changed the world a bit at a time by saying (astral date: 2/4/2006 9:32 PM) :#

Hmmm... I should probably re-read, but this smells pretty much of aop. Have you considered using
AspectJ for doing this? If you can provide some detailed example maybe we can look over it and see
what would be a solution for it. Also please let me know if AspectJ usage is not an option for you
or if I am seeing it used wrongly.

[Barry Kaplan]

We are using aspectj, but I don't (yet) see a direct way to do what I desire. I have just finished porting my log4j fixture from junit to testng. I don't think I need to show the show the whole thing to get my point across:

In Log4jTestFixture:
[code]
@Configuration(beforeSuite=true)
public void addAppender() {
LogManager.getRootLogger().removeAllAppenders();
LogManager.getRootLogger().addAppender(collectingAppender);
TestNG.getDefault().addListener(testListener);
}

@Configuration(beforeTestMethod=true)
public void redirectConsoleOutput() {
systemErr = System.err;
systemOut = System.out;
System.setErr(new PrintStream(new LoggingOutputStream(LogManager.getRootLogger(), Level.ERROR), true));
System.setOut(new PrintStream(new LoggingOutputStream(LogManager.getRootLogger(), Level.INFO), true));
}

@Configuration(afterTestMethod=true)
public void logFailedTests() {
System.setErr(systemErr);
System.setOut(systemOut);
if (testListener.isFailures() || alwaysPrintEvents) {
collectingAppender.printEvents();
}
collectingAppender.reset();
testListener.reset();
}
[/code]

And now in the concrete test:
[code]
@Test(groups="opentrader.samples")
public class SpringConfigurableTest {
...
private Log4jTestFixture log4jTestFixture = new Log4jTestFixture();

@Configuration(beforeSuite=true)
public void addAppender() {
log4jTestFixture.addAppender();
}

@Configuration(beforeTestMethod=true)
public void redirectConsoleOutput() {
log4jTestFixture.redirectConsoleOutput();
}

@Configuration(afterTestMethod=true)
public void logFailedTests() {
log4jTestFixture.logFailedTests();
}
...
}
[/code]

My test should not need to know about the details of how the log4j fixture works. And now as the fixture changes its implementation, each test will need to be updated.

I can create the base test class for all tests to implement, but really the annotations are there to remove that need.

Maybe I can inject the fixture as a parent of the test class, but I really don't use aspectj for unit tests -- I have 10k unit tests that currently run in about 30 seconds. I don't want to increase that to many minutes while aspectj weaves in the classes.

It would be much simpler and way faster for testng to recoginize something like @Fixture, and scan the objects methods for @Configurable and add them to the to the methods-to-run list.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=33376#33376

[Barry Kaplan]

We are using aspectj, but I don't (yet) see a direct way to do what I desire. I have just finished porting my log4j fixture from junit to testng. I don't think I need to show the show the whole thing to get my point across:

In Log4jTestFixture:

@Configuration(beforeSuite=true)

public void addAppender() {
LogManager.getRootLogger().removeAllAppenders();
LogManager.getRootLogger().addAppender(collectingAppender);
TestNG.getDefault().addListener(testListener);
}

@Configuration(beforeTestMethod=true)
public void redirectConsoleOutput() {
systemErr = System.err;
systemOut = System.out;
System.setErr(new PrintStream(new LoggingOutputStream(LogManager.getRootLogger(), Level.ERROR), true));
System.setOut(new PrintStream(new LoggingOutputStream(LogManager.getRootLogger(), Level.INFO), true));
}

@Configuration(afterTestMethod=true)
public void logFailedTests() {
System.setErr(systemErr);
System.setOut(systemOut);
if (testListener.isFailures() || alwaysPrintEvents) {
collectingAppender.printEvents();
}
collectingAppender.reset();
testListener.reset();
}

And now in the concrete test:

@Test(groups="opentrader.samples")

public class SpringConfigurableTest {
...
private Log4jTestFixture log4jTestFixture = new Log4jTestFixture();

@Configuration(beforeSuite=true)
public void addAppender() {
log4jTestFixture.addAppender();
}

@Configuration(beforeTestMethod=true)
public void redirectConsoleOutput() {
log4jTestFixture.redirectConsoleOutput();
}

@Configuration(afterTestMethod=true)
public void logFailedTests() {
log4jTestFixture.logFailedTests();
}
...
}

My test should not need to know about the details of how the log4j fixture works. And now as the fixture changes its implementation, each test will need to be updated.

I can create the base test class for all tests to implement, but really the annotations are there to remove that need.

Maybe I can inject the fixture as a parent of the test class, but I really don't use aspectj for unit tests -- I have 10k unit tests that currently run in about 30 seconds. I don't want to increase that to many minutes while aspectj weaves in the classes.

It would be much simpler and way faster for testng to recoginize something like @Fixture, and scan the objects methods for @Configurable and add them to the to the methods-to-run list.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=33377#33377

[Barry Kaplan]

(The [ code ] did not seem to work. It put everything on one line)

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=33378#33378

[Alexandru Popescu]

#: Barry Kaplan changed the world a bit at a time by saying (astral date: 2/4/2006 10:57 PM) :#

If I read it correctly, are you saying that you think runtime analysis/logic will be more rapid than
a compile time or even ltw aspectj step? This is making me really curious :-).

On the other hand your example would be extremely simple in AspectJ ;-).

1/ you declare the test as implementing a Log4jTestFixture interface
2/ you provide the default implementation of Log4jTestFixture

But the problem would be that the annotations are not inheritable, so this would probably not work :-).

Right now it's quite late, so I will think about this interesting problem tomorrow.

cheers,

[Cédric Beust ♔]

Hi Barry,

Interesting.  Here are a few thoughts.

The traditional way to configure TestNG tests is to either mix @Test and @Configuration methods in your test class or to put your @Configuration methods in a base class (cleaner in my opinion).

If I understood correctly, you'd like to go one step further and be able to put your @Configuration methods in a class that is totally unrelated to the test class and tell TestNG to "go run the @Configuration methods in that particular class, called the "fixture" class".

I can see some potential in this approach but I also think that you can already do it with the current "base class" approach, where that base class will invoke the fixture classes that are outside the hierarchy:

public class BaseClass {
  @Configuration(beforeSuite = true)
  public void init() {
    m_log4J = FixtureClass.initLog4J();
    m_jdbc = FixtureClass.initJdbc();
    ...
  }
}

I agree that this creates a little bit of extra effort on your part (you are basically manually implementing delegation), but I'd like to make sure that there is a clear need before deciding to add an annotation to the TestNG core.

As you mentioned, there might be a middleground where the @Configuration method does this look up of @Fixture annotations itself...

Let me know how your investigation goes, I'm curious to see where this will take you...

--
Cedric

--
Cédric

[Alexandru Popescu]

#: Cédric Beust ♔ changed the world a bit at a time by saying (astral date: 2/5/2006 9:37 PM) :#

I think I might have an idea of how to solve this, but I might as well be wrong :-)

What if you test class declares a @Configuration method that depends on a group. And this group
would be defined in your external fixture.

Would this solve your scenario?

[Barry Kaplan]

> What if you test class declares a @Configuration
> method that depends on a group. And this group
> would be defined in your external fixture.

Hmm, this sounds clean. If the @Test on the class can define a depends-on-group that would be even cooler. I'll give this a try and see how it works -- should be great.

thanks!

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=33698#33698

[Barry Kaplan]

One problem with the approach is that I cannot set @Configuration at the class level, requiring me to annotate the fixture with @Test. But that class has gobs of public helper methods which testng then wants to run.

I added enabled=false to each method, but the fixture methods are not being invoked. (Don't know why yet. I'm still experimenting.)

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=33838#33838

[Cédric Beust ♔]

On 2/6/06, Barry Kaplan <testng...@opensymphony.com> wrote:

One problem with the approach is that I cannot set @Configuration at the class level

Correct, that wouldn't make a lot of sense since you still need to specify before/after attributes on each individual methods anyway.

requiring me to annotate the fixture with @Test. But that class has gobs of public helper methods which testng then wants to run.

I added enabled=false to each method, but the fixture methods are not being invoked. (Don't know why yet. I'm still experimenting.)

Keep us posted.

--
Cédric

[Barry Kaplan]

>> > One problem with the approach is that I cannot set
> @Configuration at the
> > class level
>
> Correct, that wouldn't make a lot of sense since you
> still need to specify
> before/after attributes on each individual methods
> anyway.

Yes, but if I could set @Configuration instead of @Test then at least I wouldn't have to specify @Test(enabled=false) for all non-configuration methods.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=33882#33882

[Alexandru Popescu]

#: Barry Kaplan changed the world a bit at a time by saying (astral date: 2/6/2006 11:22 PM) :#

I am not sure I am following you. The @Configuration must be used at method level. And I don't see
the reason why you need to put @Test at class level. Can you detail why you need this?

[Cédric Beust ♔]

On 2/6/06, Barry Kaplan <testng...@opensymphony.com> wrote:

>> > One problem with the approach is that I cannot set
> @Configuration at the
> > class level
>
> Correct, that wouldn't make a lot of sense since you
> still need to specify
> before/after attributes on each individual methods
> anyway.

Yes, but if I could set @Configuration instead of @Test then at least I wouldn't have to specify @Test(enabled=false) for all non-configuration methods.

You can use partial annotations for that.  If you set @Test(enabled=false) at the class level, then each @Test method on the class will be disabled unless you set enabled=true for it...

--
Cédric

[Barry Kaplan]

The @Configuration is only allowed at the method level:

@Target(java.lang.annotation.ElementType.METHOD)
public @interface Configuration {

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=34038#34038

[Alexandru Popescu]

#: Barry Kaplan changed the world a bit at a time by saying (astral date: 2/7/2006 4:38 PM) :#

That was exactly what I've said. Still I don't understand the need for @Test in your fixture. Or
maybe I just missed the meaning of the latest mails in the thread.

[Barry Kaplan]

Ah, I think I see now. I thought that testng would scan all files in the classpath for files marked with @Configuration/Test. But that is not the case, so as long as I add the class to the testng.xml then the methods will be scanned for annotations (no?).

But, since I am trying to build a fixture that does some arbitrary configuration (for log4j in the current case) I would like to be able to specify the group once-and-only-once. And that would mean in the @Configuration at the class level. But if I need to repeat the group for method, no big deal.

-barry

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=34163#34163

[Cédric Beust ♔]

On 2/7/06, Barry Kaplan <testng...@opensymphony.com> wrote:

Ah, I think I see now. I thought that testng would scan all files in the classpath for files marked with @Configuration/Test. But that is not the case, so as long as I add the class to the testng.xml then the methods will be scanned for annotations (no?).

Correct.  The only time where TestNG scans your classpath is when you use <package>.  If you use <classes>, only these classes are examined for TestNG annotations.

But, since I am trying to build a fixture that does some arbitrary configuration (for log4j in the current case) I would like to be able to specify the group once-and-only-once. And that would mean in the @Configuration at the class level. But if I need to repeat the group for method, no big deal.

I have been thinking of ways to be able to use variables defined in testng.xml to allow this:

@Test(groupsVar = "groups")

<suite>
  <parameters var="groups" value="unit,regression,database" />

Or, taking it further, allow a BeanShell expression as a value.

Would that help?

--
Cédric

[Barry Kaplan]

Does <package> exists now? I don't see it in the docs or with any example.

On getting the fixture the run by expliciting listing the test-class and fixture-class in using <classes> in the testng.xml:

The log4-jfixture has:

@Configuration(beforeSuite=true, groups="fixture.log4j")
public void addAppender() { ... }

@Configuration(beforeTestMethod=true, groups="fixture.log4j")
public void redirectConsoleOutput() { ... }

@Configuration(afterTestMethod=true, groups="fixture.log4j")
public void logFailedTests() { ... }

All of these methods are noops right now.

In the test-class I have:

@Test(groups="opentrader.samples", dependsOnGroups="fixture.log4j")
public class LingoSample { ...}

My test methods always get skipped, but even with logging set to 10 I can't seem to figure out why. Below is the output. Any clues would be appreciated.

----
[RunInfo] Adding method selector: org.testng.internal.XmlMethodSelector@cf2c80 priority: 10
[TestClass] Creating TestClass for [ClassImpl org.opentrader.platform.samples.lingo.clientserver.LingoSample]
[TestClass] Adding method org.opentrader.platform.samples.lingo.clientserver.LingoSample.runClient() on TestClass class org.opentrader.platform.samples.lingo.clientserver.LingoSample
[TestClass] Creating TestClass for [ClassImpl org.opentrader.itest.Log4jTestFixture]
[RunInfo] Adding method selector: org.testng.internal.XmlMethodSelector@b166b5 priority: 10
[TestClass] Creating TestClass for [ClassImpl org.opentrader.platform.samples.lingo.clientserver.LingoSample]
[TestClass] Adding method org.opentrader.platform.samples.lingo.clientserver.LingoSample.runClient() on TestClass class org.opentrader.platform.samples.lingo.clientserver.LingoSample
[Invoker 16416372] Invoking org.opentrader.itest.Log4jTestFixture.addAppender()
[Invoker 16416372] Invoking org.opentrader.platform.samples.lingo.clientserver.LingoSample.startServer(java.lang.String)
[TestRunner] Running test Single Service on 2 classes, included groups:[] excluded groups:[]
[TestClass]
======
TESTCLASS: org.opentrader.platform.samples.lingo.clientserver.LingoSample
[TestClass] Test : org.opentrader.platform.samples.lingo.clientserver.LingoSample.runClient()
[TestClass]
======

[TestClass]
======
TESTCLASS: org.opentrader.itest.Log4jTestFixture
[TestClass] BeforeMethod: org.opentrader.itest.Log4jTestFixture.redirectConsoleOutput()
[TestClass] AfterMethod : org.opentrader.itest.Log4jTestFixture.logFailedTests()
[TestClass]
======

[TestRunner] PARALLEL LIST:
[TestRunner] SEQUENTIAL LIST:
[TestRunner] org.opentrader.platform.samples.lingo.clientserver.LingoSample.runClient()
[TestRunner] ===
[TestRunner] Found 1 applicable methods

*********** INVOKED METHODS

***********

SKIPPED: runClient

===============================================
Single Service
Tests run: 1, Failures: 0, Skips: 1
===============================================

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=34315#34315

[Barry Kaplan]

(I found <package>)

I don't fully understand the Runner control flow just yet, but...

By the time the Runner gets to Invoker.checkDependencies, which does

if (testMethod.getMissingGroup() != null) {
return false;
}

The group "fixture.log4j" is considered missing. Is this because the Log4jFixture class is not @Test, but only has @Configuration methods?

Message was edited by:
Barry Kaplan

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=34338#34338

[Alexandru Popescu]

#: Barry Kaplan changed the world a bit at a time by saying (astral date: 2/8/2006 4:14 PM) :#

As a quick test: can you include in the Fixture class a fake/empty @Test method and try it again?

[Cédric Beust ♔]

On 2/8/06, Barry Kaplan <testng...@opensymphony.com> wrote:

Does <package> exists now? I don't see it in the docs or with any example.

Yes, sorry, I am currently rewriting the testng.xml documentation to include the latest additions.  Right now, the DTD is your only friend (and remember, you can use a the nicer HTML version).

On getting the fixture the run by expliciting listing the test-class and fixture-class in using <classes> in the testng.xml:

The log4-jfixture has:

    @Configuration(beforeSuite=true, groups="fixture.log4j")
    public void addAppender() { ... }

    @Configuration(beforeTestMethod=true, groups=" fixture.log4j")
    public void redirectConsoleOutput() { ... }

    @Configuration(afterTestMethod=true, groups="fixture.log4j")
    public void logFailedTests() { ... }

All of these methods are noops right now.

In the test-class I have:

@Test(groups="opentrader.samples", dependsOnGroups="fixture.log4j")
public class LingoSample { ...}

 

My test methods always get skipped, but even with logging set to 10 I can't seem to figure out why. Below is the output. Any clues would be appreciated.

@Test and @Configuration groups are separated.  You can't have a @Test method "depend on" a @Configuration:  it doesn't make sense because it's implicit (@Test methods will always be run after @Configuration(before) methods).

I suspect that you don't have any @Test method that belong to the group "fixture.log4j", so that group is non-existent.

One way to fix your problem is to make @Test belong to "fixture.log4j" and then include that group in your run.  Then you should see all your @Test and @Configuration methods run as expected.

Please let me know if you see something different...

--
Cédric

[Barry Kaplan]

Yup, that did it. The fixture now works exactly as I desire without having to do any delegation! All I need to incorporate a fixture in a test is add that fixture's group to the test's group:

@Test(groups={"opentrader.samples", "fixture.log4j"})
public class LingoSample { ... }

Thank you very much Alex and Cedric for all the help.

-barry

PS. If anybody else reading this thread does not understand what I've been trying to do (and would like understand) let me know and I will either create a standalone example and/or add this topic my pending blog topics.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=16977&messageID=34592#34592

[Cédric Beust ♔]

On 2/9/06, Barry Kaplan <testng...@opensymphony.com> wrote:

PS. If anybody else reading this thread does not understand what I've been trying to do (and would like understand) let me know and I will either create a standalone example and/or add this topic my pending blog topics.

Indeed, it would be nice to post a summary of what you did and how you solved it somewhere (on your blog if you have one?).  I'll be happy to link to it from the TestNG documentation.

--
Cédric