Testing a `scalacheck` Property in `scalatest`

[Kevin Meredith]

Using sbt, scalacheck and scalatest, I'm trying to make sbt's test command fail upon a failed property test.

object PersonSpecification extends Properties("Person") {

  val validPersonAges: Prop = forAll(zeroOrGreater) { 

  x => makePerson(x).isRight == true

  }

  val invalidPersonAges: Prop = forAll(negative) { 

  x => makePerson(x).isLeft == false // NOTE - this should fail after a single scalacheck test

  }

} 

Here's my scalatest test:

class PersonTest extends FlatSpec {

"A Person with an age between 0 and Int.MaxValue" should "be valid" in {

PersonSpecification.validPersonAges.check

}

"A Person with an age between Int.MinValue and -1" should "be invalid" in {

PersonSpecification.invalidPersonAges.check

}

}

But, when I run the above with sbt's test, I get:

> test

+ OK, passed 100 tests.

! Falsified after 0 passed tests.

> ARG_0: -1

> ARG_0_ORIGINAL: -111982084

[info] PersonTest:

[info] A Person with an age between 0 and Int.MaxValue

[info] - should be valid

[info] A Person with an age between Int.MinValue and -1

[info] - should be invalid

[info] ScalaCheck

[info] Passed: Total 0, Failed 0, Errors 0, Passed 0

[info] ScalaTest

[info] Run completed in 993 milliseconds.

[info] Total number of tests run: 2

[info] Suites: completed 1, aborted 0

[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0

[info] All tests passed.

[info] Passed: Total 2, Failed 0, Errors 0, Passed 2

[success] Total time: 1 s, completed Jun 22, 2015 6:17:10 PM

How can I make the scalatest tests fail in sbt's test output for the above test?

Thanks,

Kevin

[Bill Venners]

Hi Kevin,

You lack an assertion. What is the output of the check method? If it
is Boolean, just say:

assert(PersonSpecification.invalidPersonAges.check)

However, to be more ScalaTest-like I'd suggest you use one of the two
integration points: Checkers or GeneratorDrivenPropertyChecks, both in
org.scalatest.prop. Since you're using ScalaCheck Prop's, I'd
recommend you try Checkers first. Documentation is here:

http://doc.scalatest.org/2.2.4/index.html#org.scalatest.prop.Checkers

Bill

> --
> You received this message because you are subscribed to the Google Groups
> "scalacheck" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scalacheck+...@googlegroups.com.
> To post to this group, send email to scala...@googlegroups.com.
> Visit this group at http://groups.google.com/group/scalacheck.
> For more options, visit https://groups.google.com/d/optout.



--
Bill Venners
Artima, Inc.
http://www.artima.com

[Kevin Meredith]

Per your help, I modified my test code to use the `check` method:

import org.scalatest._

import org.scalatest.prop.Checkers._

class PersonTest extends FlatSpec {

"Person#create with an age between 0 and Int.MaxValue" should "be valid" in {

check(PersonSpecification.validPersonAges)

}

"Person#create with an age between Int.MinValue and -1" should "be invalid" in {

check(PersonSpecification.invalidPersonAges)

}

Now running sbt test produces the expected result - a test failure.

I had been using org.scalacheck.Prop#check's method, which returns Unit. I had assumed incorrectly that it would throw an exception upon failure.

Thanks!

[Bill Venners]

Hi Kevin,

No worries. I'm curious what ScalaCheck's check does if it returns
Unit but doesn't throw an exception.

By the way you might be interested in org.scalactic.anyvals coming in
3.0.0, which has positive numeric types. It is currently available
only as a milestone release, 3.0.0-M3, but I'm hoping to get RC1 out
this week. Here's an example:

scala> import org.scalactic.anyvals._
import org.scalactic.anyvals._

scala> PosZInt(0)
res0: org.scalactic.anyvals.PosZInt = PosZInt(0)

scala> PosZInt(1)
res1: org.scalactic.anyvals.PosZInt = PosZInt(1)

scala> PosZInt(Int.MaxValue)
res2: org.scalactic.anyvals.PosZInt = PosZInt(2147483647)

scala> PosZInt(-1)
<console>:11: error: PosZInt.apply can only be invoked on a
non-negative (i >= 0) integer literal, like PosZInt(42).
PosZInt(-1)
^

scala> PosZInt(Int.MinValue)
<console>:11: error: PosZInt.apply can only be invoked on a
non-negative (i >= 0) integer literal, like PosZInt(42).
PosZInt(Int.MinValue)
^

For literals, you get a compiler error. For non-literals, you'll get
an Option (using the PosZInt.from factory method). I think you might
want an error message though from the looks of your code. Anyway,
since you're dealing with validating positive Ints (and zero) I
thought I'd mention this. I also haven't published Scaladoc yet for
3.0.0, but you can get the idea here by reading the Scaladoc source
here:

https://github.com/scalatest/scalatest/blob/2.3.x/scalactic-macro/src/main/scala/org/scalactic/anyvals/PosZInt.scala

Bill

On Mon, Jun 22, 2015 at 11:45 AM, Kevin Meredith

[Kevin Meredith]

Thanks for the pointer to the PosZInt code in the new release.

I posted the following "issue" to discuss whether org.scalacheck.Prop#check's behavior is useful - https://github.com/rickynils/scalacheck/issues/175.