JUnit extension on TestNG 5.1 wont work

[aalmiray]

Hi,

I recently gathered the courage to try out TestNG and liked what I saw.
Decided to test how a JUnit extension works on TestNG. I followed the
directions on Andrew Glover's blog and maven's surefire plugin and
everything works fine with JDK 1.5.0_06, maven 2.0.4 and TestNG 4.7
(I'm using javadoc tags, not annotations). It even lets me run Junit
tests and TestNG test at the same time.

But when I change to TestNG 5.1 everything crashed. It won't allow
JUnit tests and TestNG at the same time. If I remove the JUnit tests
and leav only the TestNG ones, it wont detect my tests.

Has the support for JUnit has been changed between 4.7 and 5.1 to cause
this problem ?

[Jesse Kuhnert]

Sorry for this. It is the result of this unfortunately :( http://jira.codehaus.org/browse/MSUREFIRE-134

Theoretically the patches have been applied/fixed. I didn't notice a 2.3-SNAPSHOT version of surefire in the apache snapshot repos so.....If you'd like a working version use:

http://howardlewisship.com/repository and "2.8-SNAPSHOT" version.

I can't apologize enough for this. It pisses me off to no end when things like this break in my own builds.
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=41955&messageID=83915#83915

[aalmiray]

Oh, I wasn't aware that the surefire plugin had a new release. I'll try
it on the evening and post results.

Thanks

[aalmiray]

Jesse,
I added the repo to my settings.xml under "surefire" profile. When I
try to run the test I get the following error

$ mvn -Psurefire clean test
[INFO] Scanning for projects...
[INFO]
----------------------------------------------------------------------------
[INFO] Building jstester
[INFO] task-segment: [clean, test]
[INFO]
----------------------------------------------------------------------------
[INFO] snapshot
org.apache.maven.plugins:maven-surefire-plugin:2.8-SNAPSHOT: checking
for updates from surefire.snapshots
Downloading:
http://howardlewisship.com/repository//org/apache/maven/plugins/maven-surefire-plugin/2
.8-SNAPSHOT/maven-surefire-plugin-2.8-20060825.194242-4.pom
3K downloaded
Downloading:
http://howardlewisship.com/repository//org/apache/maven/plugins/maven-plugins/2-SNAPSHO
T/maven-plugins-2-SNAPSHOT.pom
[WARNING] Unable to get resource from repository surefire.snapshots
(http://howardlewisship.com/repo
sitory/)
[INFO]
------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO]
------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

GroupId: org.apache.maven.plugins
ArtifactId: maven-plugins
Version: 2-SNAPSHOT

Reason: Unable to download the artifact from any repository

org.apache.maven.plugins:maven-plugins:pom:2-SNAPSHOT

from the specified remote repositories:
central (http://repo1.maven.org/maven2),
surefire.snapshots (http://howardlewisship.com/repository/)

[INFO]
------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 2 seconds
[INFO] Finished at: Fri Sep 01 20:35:47 CDT 2006
[INFO] Final Memory: 2M/4M
[INFO]
------------------------------------------------------------------------

The problem is that it tries to download
"maven-plugins-2-SNAPSHOT.pom", and for what I can see, that file
doesn't exist in either howardlewisship.com nor ibiblio.org

=(

[aalmiray]

Decided to change manually the plugin pom so it would download the
version 2 of the parent instead of version 2-SNAPSHOT. It finally
downloaded the plugins and all the artifacts but if didn't find any
tests yet. =(

$ mvn -Psurefire clean test
[INFO] Scanning for projects...
[INFO]
----------------------------------------------------------------------------
[INFO] Building jstester
[INFO] task-segment: [clean, test]
[INFO]
----------------------------------------------------------------------------

Downloading:
http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-plugins/2/maven-plugins-2.
pom
6K downloaded
Downloading:
http://repo1.maven.org/maven2/org/apache/maven/maven-parent/4/maven-parent-4.pom
9K downloaded
Downloading:
http://repo1.maven.org/maven2/org/apache/apache/3/apache-3.pom
3K downloaded
Downloading:
http://howardlewisship.com/repository//org/apache/maven/plugins/maven-surefire-plugin/2
.8-SNAPSHOT/maven-surefire-plugin-2.8-20060825.194242-4.jar
14K downloaded
[INFO] [clean:clean]
[INFO] Deleting directory
C:\cygwin\home\ositos\devel\sf.net\jstester\target
[INFO] Deleting directory
C:\cygwin\home\ositos\devel\sf.net\jstester\target\classes
[INFO] Deleting directory
C:\cygwin\home\ositos\devel\sf.net\jstester\target\test-classes
[INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
[WARNING]
Artifact log4j:log4j:jar:1.2.13:runtime retains local scope
'runtime' overriding broader sco
pe 'compile'
given by a dependency. If this is not intended, modify or
remove the local scope.

[INFO] [compiler:compile]
Compiling 2 source files to
C:\cygwin\home\ositos\devel\sf.net\jstester\target\classes
[INFO] [resources:testResources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:testCompile]
Compiling 1 source file to
C:\cygwin\home\ositos\devel\sf.net\jstester\target\test-classes
[INFO] snapshot org.apache.maven.surefire:surefire-booter:2.8-SNAPSHOT:

checking for updates from surefire.snapshots

[INFO] snapshot org.apache.maven.surefire:surefire-booter:2.8-SNAPSHOT:
checking for updates from apache.snapshots
Downloading:
http://howardlewisship.com/repository//org/apache/maven/surefire/surefire-booter/2.8-SN
APSHOT/surefire-booter-2.8-20060811.195125-7.pom
966b downloaded
[INFO] snapshot org.apache.maven.surefire:surefire:2.8-SNAPSHOT:

checking for updates from surefire.snapshots

[INFO] snapshot org.apache.maven.surefire:surefire:2.8-SNAPSHOT:
checking for updates from apache.snapshots
Downloading:
http://howardlewisship.com/repository//org/apache/maven/surefire/surefire/2.8-SNAPSHOT/
surefire-2.8-20060811.195125-12.pom
5K downloaded
[INFO] snapshot org.apache.maven.surefire:surefire-api:2.8-SNAPSHOT:

checking for updates from surefire.snapshots

[INFO] snapshot org.apache.maven.surefire:surefire-api:2.8-SNAPSHOT:
checking for updates from apache.snapshots
Downloading:
http://howardlewisship.com/repository//org/apache/maven/surefire/surefire-api/2.8-SNAPS
HOT/surefire-api-2.8-20060811.195125-8.pom
914b downloaded
Downloading:
http://howardlewisship.com/repository//org/apache/maven/surefire/surefire-api/2.8-SNAPS
HOT/surefire-api-2.8-20060811.195125-8.jar
50K downloaded
Downloading:
http://howardlewisship.com/repository//org/apache/maven/surefire/surefire-booter/2.8-SN
APSHOT/surefire-booter-2.8-20060811.195125-7.jar
29K downloaded
[INFO] [surefire:test]
[INFO] snapshot org.apache.maven.surefire:surefire-testng:2.8-SNAPSHOT:

checking for updates from surefire.snapshots
Downloading:

http://howardlewisship.com/repository//org/apache/maven/surefire/surefire-testng/2.8-SN
APSHOT/surefire-testng-2.8-20060825.193523-3.pom
2K downloaded
[INFO] snapshot
org.apache.maven.surefire:surefire-providers:2.8-SNAPSHOT: checking for
updates from surefire.snapshots
Downloading:
http://howardlewisship.com/repository//org/apache/maven/surefire/surefire-providers/2.8
-SNAPSHOT/surefire-providers-2.8-20060825.193523-4.pom
1K downloaded
Downloading:
http://howardlewisship.com/repository//org/apache/maven/surefire/surefire-testng/2.8-SN
APSHOT/surefire-testng-2.8-20060825.193523-3.jar
14K downloaded
[INFO] Surefire report directory:
C:\cygwin\home\ositos\devel\sf.net\jstester\target\surefire-report
s

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running net.sf.jstester.TestJ
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.062
sec

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 25 seconds
[INFO] Finished at: Fri Sep 01 20:47:48 CDT 2006
[INFO] Final Memory: 5M/11M
[INFO]
------------------------------------------------------------------------

[aalmiray]

Lastly (sorry for so many msgs), I added the JUnit tests alongside the
TestNG, and those are executed correctly. I forgot to mention I'm not
using a suite.xml, as it wasn't needed when TestNG 4.7 was declared as
a dependency.

Tried with a simple testng.xml using
<suiteXmlFiles>src/test/testng.xml</suiteXmlFiles> inside
<configuration> and got the following

[INFO]
------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO]
------------------------------------------------------------------------

[INFO] Failed to configure plugin parameters for:
org.apache.maven.plugins:maven-surefire-plugin:2.8-SNAPSHOT

(found static expression: 'src/test/testng.xml' which may act as a
default value).

Cause: Cannot assign configuration entry 'suiteXmlFiles' to 'class
[Ljava.io.File;' from 'src/test/testng.xml', which is of type class
java.lang.String
[INFO]
------------------------------------------------------------------------

[Jesse Kuhnert]

Hmm....I would interested in knowing what your pom looked like. I've tested this version of the plugin under all scnerios that I know of and have it working fine. (I don't use testng.xml files either).

You can use this file as well to help...Please do respond back if you get it working so I know what might need to be made clearer for everyone else.

http://svn.apache.org/viewvc/tapestry/tapestry4/trunk/pom.xml?view=markup

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=41955&messageID=84092#84092

[aalmiray]

Jesse,

I reviewed the sample pom and made changes accordingly, but it still
wont work. I'm starting to think that perhaps it is my environment, but
its weird that with TestNG 4.7 everything works fine.

Here is the url for the pom I'm testing
http://jstester.sourceforge.net/jstester-pom.xml

[Jesse Kuhnert]

Ok you got me :) I wasn't expecting the jdk14 dependency and can confirm that this doesn't work in my plugin tests either. I will poke around some more and get back to you when I have a solution. (I predict it will be finished in the next hour or two.)

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=41955&messageID=84190#84190

[Jesse Kuhnert]

Ok...Try it now.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=41955&messageID=84200#84200

[aalmiray]

=( It doesn't work yet.And I had to hack again the 2-SNAPSHOT bit
regarding the parent maven-plugins (??). I'm going to switch to jdk15
to see if this works, if not then there is surely something wrong with
my installation.

BTW, I deleted all references of 2.8-SNAPSHOT before trying.

[Jesse Kuhnert]

Oh...This is only for 2.8-SNAPSHOT..

I don't have access to the real surefire plugins hosted by apache.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=41955&messageID=84204#84204

[aalmiray]

But the 2.8-SNAPSHOT is served from howardlewisship.com right ?

[Jesse Kuhnert]

I should add in that you are wasting your time upgrading to jdk15 unless you also use the 2.8-SNAPSHOT version of the plugin..

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=41955&messageID=84207#84207

[Jesse Kuhnert]

Yep.

>
> But the 2.8-SNAPSHOT is served from
> howardlewisship.com right ?
>
>
> >
>

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=41955&messageID=84210#84210

[aalmiray]

Ok, tried with jdk15 and it works. Reverted back to jdk14 and javadoc
tags and it breaks.

Have tags been changed between 4.7 and 5.x ?

[Jesse Kuhnert]

I'm not entirely sure, this is a fragment of the javadoc based test class I used to test testngjdk14 functionality in the 2.8-SNAPSHOT version of surefire:

public class TestNGJavadocTest {

/**
* Sets up testObject
* @testng.before-class alwaysRun = "true" groups = "functional"
*/
public void configureTest()
{
System.out.println("configureTest running");
testObject = new Object();
}

Object testObject;

/**
* Tests reporting an error
* @testng.test groups = "functional, notincluded"
*/
public void isTestObjectNull()
{
Assert.assertNotNull(testObject, "testObject is null");
}

/**
* Sample method that shouldn't be run by test suite.
* @testng.test groups = "notincluded"
*/
public void shouldNotRun()
{
Assert.assertTrue(false, "Group specified by test shouldnt be run.");

}
}
---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=41955&messageID=84212#84212

[aalmiray]

Grabbed a copy of your testcase and got the following:

TestNG 4.7 with @testng.before-class alwaysRun = "true" groups =
"functional"
causes error because configureTest() is not called

TestNG 4.7 with @testng.configuration beforeTest = "true"
works! =)

TestNG 5.1 with @testng.before-class alwaysRun = "true" groups =
"functional"
TestNG 5.1 with @testng.configuration beforeTest = "true"
fail! =(

[Cédric Beust ♔]

before-class was introduced in TestNG5.

--
Cedric

[aalmiray]

Thanks for the heads up Cedric, I don't know why but I'm having a hard
time hunting the jdk14 tags documentation.

Here are the results of my latest attempt:

1. deleted ALL references of surefire and maven-surefire-plugin from m2
repository
2. TestNGJavadocTest testcase is the only test I'm going to use
3. added annotations to the testcase, configured classifier as jdk15
4. configured 2.8-SNAPSHOT of plugin and 5.1 of TestNG
5. "mvn clean test" works!

6. commented out annotations on testcase
7. changed classifier from jdk15 to jdk14
8. "mvn clean test" fails =(

[Jesse Kuhnert]

When you say that you commented out the annotations and changed classifier to "jdk14" does that mean that you also changed your jre to 1.4? Unless you are running a 1.4 jre it's not going to pick up your unit tests. (I think)

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=41955&messageID=84223#84223

[Alexandru Popescu]

On 9/3/06, Jesse Kuhnert <testng...@opensymphony.com> wrote:
>
> When you say that you commented out the annotations and changed classifier to "jdk14" does that mean that you also changed your jre to 1.4? Unless you are running a 1.4 jre it's not going to pick up your unit tests. (I think)

Even if I am not using maven, from the core perspective running
javadoc test with JDK5 requires an additional parameter set in TestNG:
-target=1.4. Also, I think this functionality may have been broken in
the last release (I have fixed yesterday something related to this.

./alex
--
:Architect of InfoQ.com:
.w( the_mindstorm )p.

ps: have I ever told you that I love maven? :-]

[aalmiray]

Changed the jvm to jdk1.4.2_07 and created a junit test that is a
mirror of the TestNGJavadocTest, I have the following results:

+ TestNG 4.7 with surefire 2.8-SNAPSHOT, classifier jdk14
both test compile and run perfectly but I get a warning on pom
validation

[INFO] [compiler:testCompile]

Compiling 2 source files to

C:\cygwin\home\ositos\devel\sf.net\jstester\target\test-classes
[INFO] [surefire:test]
[WARNING] POM for 'org.testng:testng:pom:5.1:compile' is invalid. It
will be ignored for artifact resolution. Reason: Failed to validate POM

+ TestNG 5.1 with surefire 2.8-SNAPSHOT, classifier jdk14
only the TestNG test is run and got more warnings

[INFO] [compiler:compile]
Compiling 2 source files to
C:\cygwin\home\ositos\devel\sf.net\jstester\target\classes
[INFO] [resources:testResources]
[INFO] Using default encoding to copy filtered resources.

[WARNING] POM for 'org.testng:testng:pom:5.1:test' is invalid. It will
be ignored for artifact resolution. Reason: Failed to validate POM
[INFO] [compiler:testCompile]

Compiling 2 source files to

C:\cygwin\home\ositos\devel\sf.net\jstester\target\test-classes
[WARNING] POM for 'org.testng:testng:pom:5.1:test' is invalid. It will
be ignored for artifact resolution. Reason: Failed to validate POM
[INFO] [surefire:test]
[WARNING] POM for 'org.testng:testng:pom:5.1:test' is invalid. It will
be ignored for artifact resolution. Reason: Failed to validate POM
[WARNING] POM for 'org.testng:testng:pom:5.1:compile' is invalid. It
will be ignored for artifact resolution. Reason: Failed to validate POM

[INFO] Surefire report directory:
C:\cygwin\home\ositos\devel\sf.net\jstester\target\surefire-reports

-------------------------------------------------------
T E S T S
-------------------------------------------------------

Running net.sf.jstester.TestJunitTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
sec
configureTest running
Running net.sf.jstester.TestNGJavadocTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015
sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

----------------------------
Remember that I had to change maven-surefire-plugin-2.8-SNAPSHOT.pom
manually to
<parent>
<artifactId>maven-plugins</artifactId>
<groupId>org.apache.maven.plugins</groupId>
<version>2</version>
</parent>

it was originally
<parent>
<artifactId>maven-plugins</artifactId>
<groupId>org.apache.maven.plugins</groupId>
<version>2-SNAPSHOT</version>
</parent>

[aalmiray]

Ok, I tested on another computer with JDK1.4.2_08 and JDK1.5.0_08 and
finally got something working

++ TestNG 5.1 -- classifier jdk14 -- JVM1.4.2_08
Running testing.TestNGJavadocTest
configureTest running
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
sec
Running testing.TestJUnitTest
configureTest running

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015
sec

++ TestNG 5.1 -- classifier jdk14 -- JVM1.5.0_08
Running testing.TestNGJavadocTest

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
sec

Running testing.TestJUnitTest
configureTest running
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
sec

++ TestNG 5.1 -- classifier jdk15 -- JVM1.5.0_08
configureTest running
Running testing.TestNGJavadocTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
sec
Running testing.TestJUnitTest
configureTest running
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016
sec