ScalaMeter with SBT not working

[Marcel Ackermann]

Hi,

I'm new to Scala so I might miss something obvious, but I am not able to get the example from the website with sbt running and I'd appreciate your help.

I created an empty folder and placed there the following two files:

RangeBenchmark.scala with this content (copied from your example):

import org.scalameter.api._

object RangeBenchmark extends PerformanceTest.Quickbenchmark {

  val sizes = Gen.range("size")(300000, 1500000, 300000)

  val ranges = for {

    size <- sizes

  } yield 0 until size

  performance of "Range" in {

    measure method "map" in {

      using(ranges) in {

        r => r.map(_ + 1)

      }

    }

  }

}

and build.sbt with this content (again copied, current version replaced):

resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"

libraryDependencies += "com.storm-enroute" %% "scalameter" % "0.6"

testFrameworks += new TestFramework("org.scalameter.ScalaMeterFramework")

  

logBuffered := false

Then I open the sbt console in the newly created folder, executing compile and then test getting  Total time: 0 s, completed 15.04.2015 11:49:12. When executing "test-only RangeBenchmark.scala" I get 

[info] Passed: Total 0, Failed 0, Errors 0, Passed 0

[info] No tests to run for test:testOnly

What am I doing wrong? (I also tried by creating a fresh IntelliJ Scala project and placing the files there, but still not working). Scala version is 2.11.6, java version "1.8.0_31".

Thank you,

Marcel

[Marcel Ackermann]

Solution was to include the following two lines in build.sbt:

scalaSource in Compile <<= baseDirectory { _ / "src" }

scalaSource in Test <<= scalaSource in Compile

I'd like to give you the following two points as feedback:

- it would be great to have a fully working sbt example as an intro

- latest version is 0.7-SNAPSHOT which should be documented in the example, too

Best,

Marcel

[Aleksandar Prokopec]

Hi!

Sorry for the delay in getting back to you!

On Wednesday, 15 April 2015 13:45:04 UTC+2, Marcel Ackermann wrote:

Solution was to include the following two lines in build.sbt:

scalaSource in Compile <<= baseDirectory { _ / "src" }

scalaSource in Test <<= scalaSource in Compile

Generally, the benchmarks should be put into the test directory, if you want to run them with the `test` command. This is the case with other testing frameworks like ScalaTest and ScalaCheck, too.

We generally recommend using a "bench" configuration in sbt to separate benchmarks from normal tests.
See example here:

https://github.com/storm-enroute/reactive-collections/blob/master/project/Build.scala#L185
 

I'd like to give you the following two points as feedback:

- it would be great to have a fully working sbt example as an intro

We're actually working on this.
Our GSoC student Chris has this as one of the first tasks on his TODO list, and the repo for examples is already created in the ScalaMeter org (currently it is empty, but example projects will be added soon).
 

- latest version is 0.7-SNAPSHOT which should be documented in the example, too

We normally don't put references to snapshots to the examples.
There are two reasons for this:
1) SNAPSHOT may be unstable, or have regressions
2) Once we bump up the major version and release the SNAPSHOT for the next version, the old SNAPSHOT is automatically dropped, so this could invalidate the examples. Stable versions are always available, so the examples are not invalidated if they are stale.

Thanks!
Alex
 

[Aleksandar Prokopec]

Hello, just an update about this - here are a couple of examples:

https://github.com/scalameter/scalameter-examples/blob/master/basic-with-separate-config/build.sbt

We recommend that you use a separate SBT configuration for benchmarks.
When you do this, you can use the `bench:test` task to run the benchmarks separate from the unit tests.

> bench:test
> bench:testOnly