Prop.exists fails with ScalaTest matchers

[Juan Rodríguez Hortalá]

Hi, 

I have a problem about using ScalaTest matchers inside a Prop.exists property. When the property runs and a value is generated such that the matcher fails, it throws a TestFailedException and the property fails, instead of trying with the next generated value until reaching the limit in the number of tests. I guess this is because ScalaTest is designed around TestFailedExceptions, while ScalaCheck is designed around Prop objects. I have devised the following workaround, and I wanted to you what you think about it: 

import org.scalacheck.{Properties, Gen}

import org.scalacheck.Prop.{forAll, exists, AnyOperators}

import org.scalacheck.Prop

import org.scalatest._

import org.scalatest.Matchers._

import scala.util.{Try, Success, Failure}

import org.scalacheck.util.Pretty

object ExistsBrokeTest extends Properties("Breaking Prop.exists with ScalaTest matchers") {

  property("forAll works ok with ScalaTest matchers, for successful tests") = 

    forAll { x : Int => 

      x + x should be (2 * x)

      true

    }

  

  property("forAll works ok with ScalaTest matchers, for failing tests") = 

    forAll { x : Int => 

      x + x should be (2 * x + 1)

      true

    }

  

  property("exists works ok without ScalaTest matchers") =

    exists ("x" |: Gen.choose(1, 20)) { x : Int =>

      x ?= 1

    }

  

  property("exists fails with ScalaTest matchers, for otherwise successful tests") =

    exists ("x" |: Gen.choose(0, 10)) { x : Int =>

      x should be (1)

      true

    }

  

  property("exists works ok with ScalaTest matchers wrapped by Try") = 

    exists ("x" |: Gen.choose(0, 10)) { x : Int =>

      Try(x should be (1)).isSuccess 

  }

  

  def safeProp[P <% Prop](p : => P) : Prop = {

    Try(p) match {

      case Success(_) => Prop.proved

      case Failure(t) => t match {

        case _: TestFailedException => Prop.falsified

        case _ => Prop.exception(t) 

      }

    }

  }

  

  def safeExists[A, P](g: Gen[A])(f: (A) => P)

                      (implicit pv: (P) => Prop, pp: (A) => Pretty): Prop = {    

    exists(g)((x : A) => safeProp(pv(f(x))))(identity[Prop], pp)

    // This doesn't work for reasons unknown

    // exists(g)((x : A) => Prop.secure(pv(f(x))))(identity[Prop], pp)

  }  

  property("safeExists works ok with ScalaTest matchers, for successful tests") =

   safeExists("x" |: Gen.choose(0, 10)) { x : Int =>

     x should be (1)

     true

   }

  

  property("safeExists works ok with ScalaTest matchers, for failing tests") =

   safeExists("x" |: Gen.choose(0, 10)) { x : Int =>

     x should be (1000)

     true

   }

}

+ Breaking Prop.exists with ScalaTest matchers.forAll works ok with ScalaTe

  st matchers, for successful tests: OK, passed 100 tests.

! Breaking Prop.exists with ScalaTest matchers.forAll works ok with ScalaTe

  st matchers, for failing tests: Exception raised on property evaluation.

> ARG_0: -2147483648

> Exception: org.scalatest.exceptions.TestFailedException: 0 was not equal 

  to 1

org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:1

  60)

org.scalatest.Matchers$ShouldMethodHelper$.shouldMatcher(Matchers.scala:623

  1)

org.scalatest.Matchers$AnyShouldWrapper.should(Matchers.scala:6265)

es.ucm.fdi.sscheck.ExistsBrokeTest$$anonfun$2.apply$mcZI$sp(ExistsBrokeTest

  .scala:20)

es.ucm.fdi.sscheck.ExistsBrokeTest$$anonfun$2.apply(ExistsBrokeTest.scala:1

  9)

+ Breaking Prop.exists with ScalaTest matchers.exists works ok without Scal

  aTest matchers: OK, proved property.

> x: 1

! Breaking Prop.exists with ScalaTest matchers.exists fails with ScalaTest 

  matchers, for otherwise successful tests: Exception raised on property ev

  aluation.

> x: 3

> Exception: org.scalatest.exceptions.TestFailedException: 3 was not equal 

  to 1

org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:1

  60)

org.scalatest.Matchers$ShouldMethodHelper$.shouldMatcher(Matchers.scala:623

  1)

org.scalatest.Matchers$AnyShouldWrapper.should(Matchers.scala:6265)

es.ucm.fdi.sscheck.ExistsBrokeTest$$anonfun$3.apply$mcZI$sp(ExistsBrokeTest

  .scala:31)

es.ucm.fdi.sscheck.ExistsBrokeTest$$anonfun$3.apply(ExistsBrokeTest.scala:3

  0)

+ Breaking Prop.exists with ScalaTest matchers.exists works ok with ScalaTe

  st matchers wrapped by Try: OK, proved property.

> x: 1

+ Breaking Prop.exists with ScalaTest matchers.safeExists works ok with Sca

  laTest matchers, for successful tests: OK, proved property.

> x: 1

! Breaking Prop.exists with ScalaTest matchers.safeExists works ok with Sca

  laTest matchers, for failing tests: Gave up after only 0 passed tests. 10

  1 tests were discarded.

Also, I have noticed that org.scalatest.prop.GeneratorDrivenProperyChecks doesn't define exists, is there any technical reason for that?  I would be glad to work on it if you think it is possible, 

Thanks a lot in advance.

Greetings, 

Juan Rodriguez Hortala