Escaping from Eventually early

[Ryan O'Rourke]

Hi,

Is it possible to get a test to fail out of an eventually {...} block before the patience expires?  It doesn't seem like it's doable.

The use case I have for this is an integration test that can be rather long lived.  I'm using ScalaTest to drive a workflow api, monitoring a series of steps being processed by external services.  It could take several minutes for the workflow task to pass through all the services and complete successfully.  So an eventually with a timeout of around 5 minutes makes sense.  However, it's possible for any of the steps along the way to fail.  At that point, processing stops.  I can detect that a step has failed through the workflow api, and it would make sense to fail the test immediately at that point.  But there seems to be no way to "escape" from the eventually.

I realize my use of scalatest here is pretty far afield from its primary purpose... but is there a way to escape that I'm missing? If not, maybe a subclass of TestFailedException could be added that Eventually knows to let through?

Thanks,

Ryan

[Bill Venners]

Hi Ryan,

I think you can just say fail("reason...") wherever you want to escape.

Bill

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalate...@googlegroups.com
To unsubscribe from this group, send email to
scalatest-use...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ryan O'Rourke]

Hi Bill,

That doesn't seem to work.  :(

  describe("timeout escape") {

    it("can we escape?") {

      eventually(timeout(Span(50, Seconds))) {

        fail("go!")

      }

    }

  }

The code passed to eventually never returned normally. Attempted 3288 times over 50.002816833 seconds. Last failure message: go!.

org.scalatest.exceptions.TestFailedDueToTimeoutException: The code passed to eventually never returned normally. Attempted 3288 times over 50.002816833 seconds. Last failure message: go!.

It seems like we'd need a separate case block in the try/catch inside eventually's "makeAValiantAttempt", something like:

    def makeAValiantAttempt(): Either[Throwable, T] = {

      try {

        Right(fun)

      }

      catch {

        case tpe: TestPendingException => throw tpe

        case tfnta: TestFailedNoTryAgainException => throw tfi

        case e: Throwable if !anExceptionThatShouldCauseAnAbort(e) => Left(e)

      }

    }

Assuming a new "TestFailedNoTryAgainException" class... but maybe with a better name than that. :)

[Bill Venners]

Hi Ryan,

Sorry, I didn't think of that. It will just keep retrying. You are trapped indeed!

Maybe we should add an escape hatch. Perhaps the way to enable escape is for us to define a new exception subtype of TestFailedException that is thrown by a new method "escape", and that one gets through eventually at least. I'd like to think about that for a while.

For the immediate need, I think perhaps you can take advantage of the fact that ScalaTest does allow certain exceptions to pass through. That's described here:

http://doc.scalatest.org/2.2.0/index.html#org.scalatest.Suite@errorHandling

I'll see if I can whip up an escape method using that approach this morning and will post it here.

Bill

--
Bill Venners
Artima, Inc.
http://www.artima.com

[Bill Venners]

Hi Ryan,

Sorry I didn't notice your suggestion of a TestFailedNoTryAgainException until now. Same thought I had.

Bill

[Bill Venners]

Hi Ryan,

This does escape:

scala> def escape(msg:String): Nothing = throw new UnknownError(msg)
escape: (msg: String)Nothing

scala> eventually(timeout(Span(10, Seconds))) {
     |   escape("I escaped!")
     | }
java.lang.UnknownError: I escaped!
    at .escape(<console>:23)

What I'd probably do for now is make a subclass of VirtualMachineError that has a better name than UnknownError (like TestEscapedException or something), then make your own escape method or methods and use those. I'll think about it some more, but I think that might make a good addition to ScalaTest itself.

Bill

[Ryan O'Rourke]

Thanks for the ideas Bill.

I will use an error for now... the nice thing about a dedicated TestEscapedException is that it could extend TestFailedException and still be reported as a regular test failure by the test runner. 

[Bill Venners]

Hi Ryan,

On Thu, Jul 10, 2014 at 10:31 AM, Ryan O'Rourke <rya...@gmail.com> wrote:

Thanks for the ideas Bill.

I will use an error for now... the nice thing about a dedicated TestEscapedException is that it could extend TestFailedException and still be reported as a regular test failure by the test runner. 

Yes. I will look to see if there are any other places where escaping might be needed. If not, I'd be included to add the escape methods to trait Eventually (i.e., not to trait Assertions), in which case I'd probably name the exception EventuallyEscapedException, which is kind of funny. Yes, it would just extend TestFailedException and have a good stack depth and all the other services of TFE. Unfortunately it would be a binary incompatible change, so it can't go into a 2.2.x release, but will need to wait until 2.3.0.

Bill

[Alex Man]

Sorry for reviving an old thread. I'm having the same problem in master. What's the plan to move forward?

[Bill Venners]

Hi Alex,

In master? Do you mean ScalaTest master? The default branch is 3.1.x, so that's where new stuff is going. I don't believe we did this in 3.0. Now that that is done we can look at doing an escape exception.

Bill

To post to this group, send email to scalatest-users@googlegroups.com

To unsubscribe from this group, send email to

scalatest-users+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[andrea....@thron.com]

Hi,

there is some new to make a eventually to programmatically quit before timeout?

my use case is

"wait until a process ends, but if the process say "ERROR", you can end to wait and go forward".

i tried the "throw new Error" solution, but this ends the entire scalatest suite.

Thanks.

[Bill Venners]

Hi,

You can catch the specific exception inside the eventually block and
throw it away. That will cause the eventually block to succeed.

Bill

[andrea....@thron.com]

how?

i tried "throw new Error" but this make the ENTIRE suite to stop. i'm making something wrong? should the throw new Error makes only the eventually block to pass?

On Thursday, March 28, 2019 at 12:57:39 PM UTC+1, Bill Venners wrote:

Hi,

You can catch the specific exception inside the eventually block and
throw it away. That will cause the eventually block to succeed.

Bill

On Thu, Mar 28, 2019 at 7:56 AM andrea.bisello via scalatest-users
<scalate...@googlegroups.com> wrote:
>
> Hi,
> there is some new to make a eventually to programmatically quit before timeout?
> my use case is
>
> "wait until a process ends, but if the process say "ERROR", you can end to wait and go forward".
>
> i tried the "throw new Error" solution, but this ends the entire scalatest suite.
>
> Thanks.
>
> --
> You received this message because you are subscribed to the Google
> Groups "scalatest-users" group.
> To post to this group, send email to scalate...@googlegroups.com
> To unsubscribe from this group, send email to

> scalate...@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/scalatest-users?hl=en
> ScalaTest itself, and documentation, is available here:
> http://www.artima.com/scalatest
> ---
> You received this message because you are subscribed to the Google Groups "scalatest-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to scalate...@googlegroups.com.

[Bill Venners]

Hi,

You need to catch, not throw. The eventually clause will succeed only
if an exception is *not* thrown by the block. If there's some
exception you want to allow, catch it, set a flag (perhaps set an
Option[Throwable] with the exception, then allow the block to complete
without an exception. Then assert that the flag was not set next. If
the flag was set, that would mean you exited the block early, and
therefore the test should fail.

Or, another way you could do it is catch the Error you're throwing now
and replace it with an assert(false) that would replace it with a
TestFailedException.

Bill

On Fri, Mar 29, 2019 at 3:22 AM andrea.bisello via scalatest-users

> scalatest-use...@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/scalatest-users?hl=en
> ScalaTest itself, and documentation, is available here:
> http://www.artima.com/scalatest
> ---
> You received this message because you are subscribed to the Google Groups "scalatest-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-use...@googlegroups.com.