Akka test fails with Scalatest 3.0

[Gavin Baumanis]

Hi Bill / et al.

Firstly - thanks very much for the work you put into the library.

I just updated the library and ran my tests without change.

Now one of my tests is failing.

I return to using 2.2.4 - and the test passes.

UNfortunately I haven;t had time yet to look into / or even understand it and am going away for the weekend.

but nonetheless wanted to alert you to it - just in case it was something that you needed to pay atattnetion to - as opposed to a change I needed to make in my test.

Anyway;

Here is my test (stolen wholly from a typesafe akka template) that fails and the error I receive;

package silentactor03 {

  class SilentActorTest extends TestKit(ActorSystem("testsystem"))

    with WordSpecLike

    with MustMatchers

    with StopSystemAfterAll {

    "A Silent Actor" must {

      "change internal state when it receives a message, multi" in {

        import SilentActor._ //import protocol

        val silentActor = system.actorOf(Props[SilentActor], "s3") //create actor"/>

        silentActor ! SilentMessage("whisper1")

        silentActor ! SilentMessage("whisper2")

        silentActor ! GetState(testActor) //get state

        expectMsg(Vector("whisper1", "whisper2")) //expectMsg

      }

    }

  }

  object SilentActor {

    case class SilentMessage(data: String)

    case class GetState(receiver: ActorRef) //getstate msg"/>

  }

  class SilentActor extends Actor {

    import SilentActor._

    var internalState = Vector[String]()

    def receive = {

      case SilentMessage(data) =>

        internalState = internalState :+ data

      case GetState(receiver) => receiver ! internalState //process getstate msg

    }

  }

}

[info] A Silent Actor

[info] - must change internal state when it receives a message, multi *** FAILED ***

[info]   java.lang.AssertionError: assertion failed: expected Succeeded, found Vector(whisper1, whisper2)

[info]   at scala.Predef$.assert(Predef.scala:165)

[info]   at akka.testkit.TestKitBase$class.expectMsg_internal(TestKit.scala:351)

[info]   at akka.testkit.TestKitBase$class.expectMsg(TestKit.scala:327)

[info]   at akka.testkit.TestKit.expectMsg(TestKit.scala:735)

[info]   at com.thespidernet.scalablog.models.silentactor03.SilentActorTest$$anonfun$2$$anonfun$apply$mcV$sp$2.apply(akkaSuite.scala:117)

[info]   at com.thespidernet.scalablog.models.silentactor03.SilentActorTest$$anonfun$2$$anonfun$apply$mcV$sp$2.apply(akkaSuite.scala:110)

[info]   at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)

[info]   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)

[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)

[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)

[info]   ...

The werid part to me is - that the test reads that it expects the "Vector(whisper1, whisper2)" result - which the error message says it is actually getting - but somehow - states that the test is expecting "Success".

So at a very brief look - I don't understand the error.

-Gavin.

[Bill Venners]

Hi Gavin,

I just got off a long flight. On my phone only right now,
But it looks like our Compatibility implicit conversion from Any => Assertion was applied inside the parameter list of expectMsg call. This conversion "discards" the vector value by placing ; Succeeded after it. Once I get to the hotel I'll try and understand why that conversion was triggered in that spot.

Bill

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalate...@googlegroups.com
To unsubscribe from this group, send email to
scalatest-use...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bill Venners]

Hi Gavin,

expectMsg is declared like this:

def expectMsg[T](o: T): T

It returns the msg that is expected, if it arrives before the timeout, so that you can do further assertions on it. By converting the object inside the parens to Succeeded, that causes it to return Succeeded which does fix the type error. But I would have rather it just plopped a ; Succeeded at the end. That's what you can do to get it to work, just change the last line of your test:

expectMsg(Vector("whisper1", "whisper2")) //expectMsg

To:

expectMsg(Vector("whisper1", "whisper2")) //expectMsg

succeed

But this is a big problem. I will see if there is some other way to do that implicit conversion that would not trigger it this way. But first, 2nd breakfast.

Bill

--

Bill Venners
Artima, Inc.
http://www.artima.com

[Bill Venners]

Hi Gavin (and anyone else intersted in the direction of 3.0),

I'm teaching this week so was busy with prep and didn't have time to try alternatives. I will try that, but I don't expect them to work, and even if they do work, I have come to feel this is to clever for ScalaTest. I think we'll need to go back to Unit as a registration type for all the existing styles. That means we'll likely need to add another set of styles where the registration type is Assertion. I'm thinking of calling those SafeX, e.g., SafeFunSuite.

That way upgrading to 3.0 will be a snap. All existing code will work the same way and compile just as fast (or slow) as before. AsyncFunSuite and its siblings will just require Future[Assertion] full stop. I'll just drop the AsyncSafety and AsyncCompatibility traits. That won't break any code because none exists (except perhaps against 3.0 milestones). It might feel a bit like a pain when you're using 3rd party libraries like Akka TestKit, ScalaMock, or any of the many Java testing libraries like the mocking frameworks, DBUnit, etc. But I think it will be worth adding a "succeed" at the end of your mapping functions, because otherwise I think people will make a lot of mistakes in future space that they'll have to catch at test time (if they catch it), and that's a pain too.

In SafeFunSuite, the result type of tests will be Assertion, full stop. No Safety or Compatibility implicits there either. I may leae these out of 3.0 entirely, because the driving use for these is the ability to use Expectations, which I'm also leaving out of 3.0. Expectations (and WillMatchers) will be a variant of assertions (and matchers) that always return a result: instead of throwing an exception when the assertion fails, they return a value. I "expect" these to come out in 3.1 along with a syntax for laws testing, and once we have those, I think it is important to have a way to use them in a non-async style test. Unit registration type would just discard the value, and so it won't work in classic styles. We're going to see if we can add a free feature to SuperSafe compiler plugin to catch that kind of mistake at compile time. And I'll add a warning to the documentation of the classic styles and expectations traits. But Expectations will work just fine in SafeFunSuite and its siblings, because there's no value discarding, and we'll provide an implicit conversion from Expectation to Assertion that throws the relevant exception.

Expectations does exist and work, but it will be left out of the 3.0 release because I haven't done the laws testing feature yet, and I want to make sure laws testing is done before I release Expectations, in case they need to change to server their driving use case.

So in conclusion, what I'm thinking we'll need to do is abandon the Safety and Compatibility implicit approach and just accept a bit more bloat by having distinct style classes for safety (SafeFunSuite) and compatibility (FunSiite) for synchronous style tests, and the async styles (AsyncFunSuite) will just always be safe.

As always, thoughts are welcome. Please post here or in the gitter channel for scalatest.

Thanks.

Bill

[Gavin Baumanis]

Hi Bill,

Thanks for your time spent thus far invrstigating.

As long as the Documentation is appropriate / there are working examples or the "new" way and also an appropriate migration guide - to get you're old tests working with the new version -I think that would probably cover it.

Any of the old typesafe templates will of course use 2.2.4 or earlier as specificed in build.sbt - and continue to test successfully.

And since you're raising the "base" version number too - there is no need to promise backwards compatibility either.

I found the akka error - just by copying the new libaryt into my build file and running the tests without changing anything - as a test in and of itself.

Like everything in the software world - if some migration work is required between major releases - so be it.

I just thought I would this one back - as it weas the only test out 50 that failed - and the error message read like it "should" have worked - but just didn't.

So initially it looked more like a bug - than a change.

Thanks again,

Gavin

[Bill Venners]

Hi Gavin,

You can get that test working under 3.0.0-M10 by sticking a succeed after your expectMsg call.

Thanks for your input. There will be a few breaking changes in 3.0, but very minor ones. But I try to minimize that as much as possible. And I want to avoid any kind of breaking change of the "old tests compile but don't do the same thing anymore" variety. And I definitely don't want to break Akka TestKit tests, and expectMsg calls are going to be very common in Akka TestKit tests.

Also, on the Unit registration type, frankly no one has ever complained about that. I think it isn't broken and therefore doesn't need fixing. It works fine, and is what people are used to. So we're going to try the approach I described next: existing style will continue to have registration type Unit, async styles will require registration type Future[Assertion], when we release Expectations we'll likely release "safe" styles that require registration type Assertion. And hopefully the word bloat won't come up too often when people talk about ScalaTest! At least each choice will be simple and focused, so that once you decide what it is you want to use in your project, I think you'll have a very focused, simple test framework for your project.

Bill

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalate...@googlegroups.com
To unsubscribe from this group, send email to
scalatest-use...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.