Info on inline benchmarking

[Julien Vion]

Hi,

I am very interested in the inline benchmarking features introduced in the latest version of ScalaMeter. However, I have two questions and/or feature requests (if not currently possible):

* is it possible to measure both execution time and memory footprint in one run?

* is it possible to obtain the return value(s) of the benchmarked functions?

Thank you in advance,

-- 

Julien

[Aleksandar Prokopec]

Hi Julien,

Yes - this is indeed possible.
During the last GSoC, we had discussions about whether or not to provide some idiomatic support,
so that you could combine any two measurers.

https://docs.google.com/document/d/1CccM92GcuWvKA8rr95v4DLNXzRaDxbftFxGM11apaBQ/edit?usp=sharing

TL;DR In the end, this turned out too difficult, and such a measurer would have obscure semantics, so we decided not to implement this (yet).
However, it should be possible to implement a custom measurer that does this - both combine execution time and memory,
and also obtain the return value of the benchmarked function.

class MyCustomMeasurer extends Measurer[(Double, Int)] {
  def name = "time-and-memory"
  def measure[T](ctx: Context, measurements: Int, setup: T => Any, tear: T => Any, regen: () => T, snippet: T => Any): Seq[Quantity[(Double, Int)]] = {
    // rough implementation (see Measurer.scala in scalameter repository for better, complete implementations)
    val x = regen()
    val runtime = Runtime.getRuntime
    val atStart = System.nanoTime
    val memStart = runtime.totalMemory - runtime.freeMemory
    snippet(x)
    val atEnd = System.nanoTime
    val memEnd = runtime.totalMemory - runtime.freeMemory
    (atEnd - atStart, memEnd - memStart)
  }
}

class CustomBench extends Bench[(Double, Int)] {
  // implement executor, persistor and reporter here
  // make sure that the executor uses your custom measurer
}

Note that the HtmlReporter can currently only represent Numeric results.
Since tuples are not numeric (unclear how they would be represented on the graph), that leaves logging reporter for displaying results, which dumps them to command line.

https://github.com/scalameter/scalameter/blob/master/src/main/scala/org/scalameter/reporting/LoggingReporter.scala

--
You received this message because you are subscribed to the Google Groups "ScalaMeter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalameter+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.