Changing the default minimum successful tests for 1 specification

[kkli...@gmail.com]

I have several scala check specifications (building in sbt 0.13.0) and the default minimum successful tests of 100 is appropriate for them.  I have 1 specification that I would like to always run with a much higher minimum successful tests setting, but I don't want the other specifications to be impacted by that.  is this possible?

[Rickard Nilsson]

This is actually something I was thinking about earlier today.

It could be that it is possible in sbt to specify different parameters
for different test, I haven't found an obvious way though.

However, in ScalaCheck it is not possible to attach test parameters to
individual properties, but it could be a sensible feature to add. Maybe
it could be implemented by adding a "Test" or "Check" class that pairs
test parameters with one or several properties. sbt could then pick up
such an instance and test with the specified parameters. However, I
don't know how that would work together "global" test parameters, what
should override what.

Maybe this feature is best placed in the test runner itself, in your
case sbt, after all. I mean things like dev vs prod testing (with
"quick" parameters for dev, and more thourough parameters for prod),
test filtering etc is in my opinion best put in the build definition,
not in the specification itself. I'm open for discussion, though.


/ Rickard

[Dave Stevens]

I use ScalaCheck through Specs2 runner. This has it's problems, output for nested properties for instance, but it does allow you to set test parameters per property. I have tens of tests which use scalacheck and I set minTestsOk for a few of these.

See http://etorreborre.github.io/specs2/guide/org.specs2.guide.Matchers.html#ScalaCheck 

[josh...@gmail.com]

Was there ever any progress on this? I was just needing the same thing. 

[Rüdiger Klaehn]

I also need this. It would also be nice to be able to set the RNG for each test.

[Rickard Nilsson]

Hi,

I have opened the following issue to keep track of this:
https://github.com/rickynils/scalacheck/issues/120

/ Rickard

> --
> You received this message because you are subscribed to the Google
> Groups "scalacheck" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to scalacheck+...@googlegroups.com
> <mailto:scalacheck+...@googlegroups.com>.
> To post to this group, send email to scala...@googlegroups.com
> <mailto:scala...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/scalacheck.
> For more options, visit https://groups.google.com/d/optout.

[etorreborre]

Hi Dave,

Can you please give me a small example of the problematic output for nested properties?

(And any other kind of problem you ran into, thanks!)

Eric.

[Dave Stevens]

Hi Eric,

When a "Properties" is evaluated and a child "Property" is not satisfied, it is not possible to determine which property failed from the output. I am not necessarily faulting Specs2 here, as the same thing happens when a "Properties.property" returns "Properties" as well. However, it is possible to work around this with "Properties.include". See the differences in output from the three tests.

https://github.com/drstevens/specs2-scalacheck/blob/master/src/test/scala/com/daverstevens/specs2scalacheck/NestedPropertiesSpec.scala#L3-L15

Disclaimer: I have copy/pasted scalaz.ScalazProperties into this demonstration project in order to use the latest version of Specs2. It was lifted from the scalaz repo

https://github.com/scalaz/scalaz/blob/series/7.2.x/scalacheck-binding/src/main/scala/scalaz/scalacheck/ScalazProperties.scala

--
You received this message because you are subscribed to the Google Groups "scalacheck" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scalacheck+...@googlegroups.com.
To post to this group, send email to scala...@googlegroups.com.

Visit this group at http://groups.google.com/group/scalacheck.

[etorreborre]

Hi Dave,

Thanks for the example.

It looks like the difference is in the execution model for Properties.

As a work-around we can do the following, if ps is a Properties object:

   val newProperties  = new Properties(ps.name) {

          override def apply(params: Gen.Parameters): Prop.Result =

            Prop.all(ps.properties.map { case (n, p) => p :| n }:_*)(params) 

    }

That is simply label each individual property with its name instead of discarding it as it is done now. 

I can add this workaround in the next specs2 (major) version but this should probably be fixed inside ScalaCheck itself.

Eric.

[Dave Stevens]

Thanks for looking into it. When I originally made the statement about specs2 output, I incorrectly believed that scalacheck would output results from "Properties" within "Properties" better then it actually does. As soon as I looked deeper while coming up with that example, I realized it was a scalacheck thing.

It might be more straightforward if "org.scalacheck.Properties" didn't extend "org.scalacheck.Prop". At least you would be prevented from putting Properties within Properties. This would be a pretty huge breaking change though.

[Howard Wang]

I'm tried to override the mainRunner of Properties trait, but the parameters are not changed as expected. What's the correct way to achieve this?

import org.scalacheck.Prop.forAll
import org.scalacheck.Test
import org.scalacheck.{Gen, Properties}
import org.scalatest.Matchers

class CoordinateTruncProp extends Properties("CoordinateTruncProperties") with Matchers  {

  override def mainRunner(args: Array[String]): Int = {
      val res = Test.checkProperties(Test.Parameters.default.withMinSize(200).withMaxSize(1000).withMinSuccessfulTests(200), this)
      val failed = res.filter(!_._2.passed).size
      failed
  }

  property("CoordinateTrunc") =
    forAll(Gen.choose(-400.0, 400.0)) { (coord) =>
      coord - truncateDoubles.truncate(coord) < 0.1
    }
...
}

Some relative refs: http://booksites.artima.com/scalacheck/examples/html/ch07.html