sbt, ScalaTest and TestNG

[Miles Sabin]

Hi folks,

I'm trying to get sbt to run TestNG tests via ScalaTest on Scala 2.8.0.Beta1.

I can successfully run vanilla ScalaTest tests via sbt (so I think I
have the basic setup and combination of releases correct[1]), and I
can run the TestNG test via ScalaTest using the command line. However
when I run the tests via sbt, the TestNG test isn't found.

Any ideas? What should I be looking for?

[1] Scala 2.8.0.Beta, sbt 0.7.1, ScalaTest
1.0.1-for-scala-2.8.0.Beta1-with-test-interfaces-0.3-SNAPSHOT, TestNG
testng-5.11-jdk15.jar.

Cheers,

Miles

--
Miles Sabin
tel: +44 (0)7813 944 528
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Josh Cough]

Miles, 

Do you have TestNG as a managed dependency via sbt?

-Josh

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalate...@googlegroups.com
To unsubscribe from this group, send email to
scalatest-use...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest

To unsubscribe from this group, send email to scalatest-users+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.

[Miles Sabin]

No, I don't just as a vanilla library for the project. Is that my problem?

Cheers,

Miles

On 22 Mar 2010 09:09, "Josh Cough" <josh...@gmail.com> wrote:

Miles, 

Do you have TestNG as a managed dependency via sbt?

-Josh

On Mon, Mar 22, 2010 at 9:49 AM, Miles Sabin <mi...@milessabin.com> wrote:

>
> Hi folks,
>
> I'm trying to get sbt to run TestNG tests via ScalaTest on Scala 2.8.0.Beta1.
>

> ...

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalate...@googlegroups.com
To unsubscribe from this group, send email to
scalatest-use...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest

To unsubscribe from this group, send email to scalatest-users+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.

[Josh Cough]

I think its the right place to start. 

[Miles Sabin]

On Mon, Mar 22, 2010 at 9:22 AM, Josh Cough <josh...@gmail.com> wrote:
> I think its the right place to start.

OK, I now have a project configuration that looks as below, but no
change: the vanilla ScalaTest test run, but the TestNG tests don't.

import sbt._

class TestNGProject(info: ProjectInfo) extends DefaultProject(info) {

val newReleaseToolsRepository = "Scala Tools Repository" at
"http://nexus.scala-tools.org/content/repositories/snapshots/"

val scalatest = "org.scalatest" % "scalatest" %
"1.0.1-for-scala-2.8.0.Beta1-with-test-interfaces-0.3-SNAPSHOT" % "test"

val testng = "org.testng" % "testng" % "5.11" classifier "jdk15"
}

Any hints on where to look next would be very much appreciated.

[Miles Sabin]

On Mon, Mar 22, 2010 at 9:34 PM, Miles Sabin <mi...@milessabin.com> wrote:
> On Mon, Mar 22, 2010 at 9:22 AM, Josh Cough <josh...@gmail.com> wrote:
>> I think its the right place to start.
>
> OK, I now have a project configuration that looks as below, but no
> change: the vanilla ScalaTest test run, but the TestNG tests don't.
>
> import sbt._
>
> class TestNGProject(info: ProjectInfo) extends DefaultProject(info) {
>
>  val newReleaseToolsRepository = "Scala Tools Repository" at
>    "http://nexus.scala-tools.org/content/repositories/snapshots/"
>
>  val scalatest = "org.scalatest" % "scalatest" %
>    "1.0.1-for-scala-2.8.0.Beta1-with-test-interfaces-0.3-SNAPSHOT" % "test"
>
>  val testng = "org.testng" % "testng" % "5.11" classifier "jdk15"
> }

OK, it seems that the problem was with the definition of the test.
Taking the ScalaTest/TestNG examples from here,

http://www.scalatest.org/getting_started_with_testng

tests of the first form (ie. ExampleSuite extends Assertions) are not
detected when run via sbt (although they are when run from the command
line); whereas tests of the second form (ie. ExampleSuite extends
TestNGSuite) are detected when run via sbt. Is this the expected
behaviour?

[Josh Cough]

I thought so. Especially given: "If you wish to give yourself the option of running with either TestNG or ScalaTest, mix in org.scalatest.testng.TestNGSuite. Trait TestNGSuite already mixes in Assertions:"

 

The first style is just for running via the testng runner. Though you had said, "and I

can run the TestNG test via ScalaTest using the command line." I'm a little curious which style you used to get that to work. Which command line do you mean?

For SBT, I only run subclasses of org.scalatest.Suite, and Assertions is not one of those. 

Let me know if theres anything else you'd like me to look at. 

Cheers,


Miles

--
Miles Sabin
tel: +44 (0)7813 944 528
skype:  milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Miles Sabin]

On Tue, Mar 23, 2010 at 1:18 AM, Josh Cough <josh...@gmail.com> wrote:
> The first style is just for running via the testng runner. Though you had
> said, "and I can run the TestNG test via ScalaTest using the command line."
> I'm a little curious which style you used to get that to work. Which command
> line do you mean?

The one given just below the example, which I now see is invoking the
TestNG runner as you say. I understand what's going on now, thanks.

> For SBT, I only run subclasses of org.scalatest.Suite, and Assertions is not
> one of those.

Gotcha. It might be worth documenting that somewhere on the sbt site:
The ordering of the examples on the ScalaTest + TestNG page suggests
that the "extends Assertions" style ought to work as well (or at least
doesn't give any indication that it doesn't work with sbt).

[Mark Harrah]

On Tuesday 23 March 2010, David Bernard wrote:
> From my understanding test-interface+sbt don't detect test by
> Annotation but only throught a marker interface.

Correct. I'd like to add the annotation detection, though. I might have asked this before, but exactly what is the fingerprint for annotation-based test frameworks? Is there always an annotation on a class or could there be method annotations only and no class annotations?

Thanks,
Mark

>
> /davidB

[Mark Harrah]

On Tuesday 23 March 2010, David Bernard wrote:

> On Tue, Mar 23, 2010 at 14:28, Mark Harrah <dmha...@gmail.com> wrote:
> > On Tuesday 23 March 2010, David Bernard wrote:
> >> From my understanding test-interface+sbt don't detect test by
> >> Annotation but only throught a marker interface.
> >
> > Correct. I'd like to add the annotation detection, though. I might have asked this before, but exactly what is the fingerprint for annotation-based test frameworks? Is there always an annotation on a class or could there be method annotations only and no class annotations?
>

> xxx.yy.zz.Test Annotations could on class and/or method

Thanks, David. sbt head (will be 0.7.3) supports discovering tests using annotations and will support tests written in Java. It is up to the test frameworks to use the annotation-based discovery, however.

-Mark