setupBeforeWarmUp and teardownAfterMeasure

[Changgeng Li]

Hello,

Thanks for developing such a great tool.

I'm trying to build a test case, but come to a problem. Basically there's an object that creating and destroying it are very expensive(something like a connection pool)

At first I'm trying to use beforeTests to create it and afterTests to destroy it, but I found that would somewhat make the warmup doesn't make much sense, because beforeTests/afterTests would be called between the warmup and measure.
That would make the first several iteration of measure quite slow.

If I don't clean up(call the "allocation.shutdown()" in this case), the separated VM will hang there forever because some threads created are not daemon threads(netty workers).

So my final approach is to introduce a flag, that if the afterTests block is called for the second time, do this cleanup. The code is like this:


  var teardownCount = 0
  val allocation = very expensive initialization
  performance of "allocation" in {
    measure method "get" in {
      using(subjects) afterTests {
                println("AFTER ALL TESTS")
                if(teardownCount == 0){
                        teardownCount +=1
                }else {
                        println("Real shutdown")
                        allocation.shutdown();
                }
      } in {
        subjectId => {
              //.....
        }
      }
    }
  }


I don't think this is a clean way to do that. How about we introduce another two callbacks that can be called before warm up and after measure, and each of these call back would be invoked only once per JVM. Or is there any other suggestions?

Thanks,
Changgeng

[Aleksandar Prokopec]

As far as I can see, the current behaviour is:
- call the beforeTests before the warmup
- call the afterTests after the warmup
- call the beforeTests before the benchmarks
- call the afterTests after the benchmarks

I think that we can change this, and the proper behaviour should be:
- call the beforeTests before the warmup
- call the afterTests after the benchmarks

I'm not sure I see a valid use of the first behaviour, so I will make the change.
You should be able to see it in the 0.7-SNAPSHOT version soon.

[Aleksandar Prokopec]

See commit here: https://github.com/scalameter/scalameter/commit/e8193e8a3461dd698d577c2f70d040bf381e4bba