ScalaTest's new "path" traits are released in a new 1.8-SNAPSHOT

[Bill Venners]

Hi All,

I have deployed a new ScalaTest 1.8-SNAPSHOT that contains
feature-complete implementations of path.FunSpec and path.FreeSpec.
Please give them a try if you have time. The documentation is here:

http://www.artima.com/docs-scalatest-1.8-26.Feb.2012/#org.scalatest.path.FunSpec

And:

http://www.artima.com/docs-scalatest-1.8-26.Feb.2012/#org.scalatest.path.FreeSpec

These are deployed at oss.sonatype.org, for all Scala 2.8 and 2.9
artifact IDs except 2.8.0:

https://oss.sonatype.org/content/groups/public/org/scalatest/

The code for 1.8 is in this branch, in case you want to look at the
implementation:

https://scalatest.googlecode.com/svn/branches/to-release-as-1.8

I decided to only implement this style for FunSpec and FreeSpec,
because those are ones for which they make the most sense. In these
style traits you can specify a bit of text, then implement it in code,
as described and demonstrated at the ScalaDoc links above.

These traits were certainly the trickiest thing I've implemented in
ScalaTest so far---the first time I really needed a debugger to help
me figure out what was going on as I went. Although I saw the value, I
was always very worried this kind of trait (first suggested by Esko
Luontola) would be too magical for ScalaTest's goal of obvious client
code, but I spent a long time trying to find a way to design and
implement them that would yield test code that is easy to reason
about. The details of how it is executed is in the "How it executes"
section of the Scaladoc pages. Let me know if you have any suggestions
for improvements.

This is the last major piece of the 1.8 puzzle, but I have a few more
odds and ends to do for 1.8, and even a little bit more to do on the
path traits. I'll probably release 1.8.RC1 early in the week of March
5, so if you have time this coming week please try 1.8-SNAPSHOT out.
(I CC'd scala-user this time to try and snag a few more tryer-outers
prior to RC1.) Please post comments and suggestions to
scalatest-users.

Thanks.

Bill
----
Bill Venners
Artima, Inc.
http://www.artima.com

On Sat, Dec 3, 2011 at 7:47 AM, Richard Apodaca
<rapo...@metamolecular.com> wrote:
> Esko,
>
>> In the documentation you can see the aforementioned stack
>> example: http://specsy.orgdocumentation.html#isolated_execution_model
>
> Isolated execution mode is exactly the behavior I'm looking for. Thanks for
> the link. Bill, is there any chance of implementing this in ScalaTest?
>
> Best,
> Rich
>
>
> On Sat, Dec 3, 2011 at 6:40 AM, Esko Luontola <esko.l...@gmail.com>
> wrote:
>>
>>
>>
>> On Thursday, December 1, 2011 8:58:51 PM UTC+2, Bill Venners wrote:
>>>
>>> Hi Richard,
>>>
>>> On Thu, Dec 1, 2011 at 10:37 AM, Richard Apodaca
>>> <rapo...@metamolecular.com> wrote:
>>> > Hello Bill,
>>> >
>>> > I hadn't thought of eliminating 'before' blocks... now that I have, I
>>> > kind
>>> > of like it.
>>> >
>>> As far as I know, I think this approach was first suggested for Scala
>>> testing by Esko Luontola. He created the specsy framework to do
>>> something like this. You may want to take a look at that as well:
>>>
>>> https://github.com/orfjackal/specsy
>>
>> Yes. :)
>>
>> In the documentation you can see the aforementioned stack
>> example: http://specsy.org/documentation.html#isolated_execution_model
>>
>> In Specsy the "before" blocks are implicit, in addition to which there are
>> multiple "after" blocks which are executed in LIFO order as described
>> in http://specsy.org/documentation.html#before_and_after_blocks
>>
>> Nowadays there is also a non-isolated execution mode, which can be useful
>> as a performance optimization or for writing tests for multi-step processes
>> (if you wish to avoid deep nesting).
>> http://specsy.org/documentation.html#nonisolated_execution_model
>>
>> BTW, adding that non-isolated execution mode was surprisingly simple, just
>> modifying two lines and adding a couple
>> more: https://github.com/orfjackal/specsy/commit/c31f8508969098b05f7bedac0a9fe9a1b6fe833a#diff-3
>>
>>
>>>
>>> The p is used only to pass the Path up to the superclass Spec:
>>>
>>> class StackSpec(p: Path = All) extends Spec(p) { ...
>>>
>>> It is relatively ugly boilerplate.
>>
>> In Specy I pass it using a thread local [1] where the value is stored
>> right before the test class instance is created [2] and then the value is
>> taken from there in the test class' constructor [3].
>>
>>
>> [1] https://github.com/orfjackal/specsy/blob/master/src/main/scala/net/orfjackal/specsy/core/ContextDealer.scala
>>
>> [2] https://github.com/orfjackal/specsy/blob/master/src/main/scala/net/orfjackal/specsy/core/SpecClassRunner.scala
>>
>> [3] https://github.com/orfjackal/specsy/blob/master/src/main/scala/net/orfjackal/specsy/core/ContextDealer.scala
>>
>> In the Go language there are no thread locals, so with GoSpec the
>> path/context is passed as a parameter to the test function [4].
>>
>>
>> [4] https://github.com/orfjackal/gospec/blob/release/examples/stack_test.go
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "scalatest-users" group.
>> To post to this group, send email to scalate...@googlegroups.com
>> To unsubscribe from this group, send email to
>> scalatest-use...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/scalatest-users?hl=en
>> ScalaTest itself, and documentation, is available here:
>> http://www.artima.com/scalatest
>
>
>
>
> --
> Richard L. Apodaca
> Founder and CEO
> Metamolecular, LLC
> 8070 La Jolla Shores Drive #464
> La Jolla, CA 92037
> 858.658.0860
>
> http://metamolecular.com
>
> --
> You received this message because you are subscribed to the Google
> Groups "scalatest-users" group.
> To post to this group, send email to scalate...@googlegroups.com
> To unsubscribe from this group, send email to
> scalatest-use...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/scalatest-users?hl=en
> ScalaTest itself, and documentation, is available here:
> http://www.artima.com/scalatest

--
Bill Venners
Artima, Inc.
http://www.artima.com

[Naftoli Gugenheim]

Very exciting!

One thing I could use sometimes in a test framework is a way to fail without terminating. Sometimes it's useful to get a table of successes and failures --- sometimes you can see a pattern that way, that you can't see when the failure terminates the test.

Also, one thing I miss in ScalaTest that Specs has is showing a "diff" of, e.g., two strings. For instance suppose you're generating a JSON string and comparing it against the intended result. The test reports that they're not equal but the difference is hard to spot. Even better would be arbitrary objects (perhaps a diff of the toString, or configurable, e.g., class Decompose[A] extends (A=>List[Symbol,Any])).

[Bill Venners]

Hi Naftoli,

On Sun, Feb 26, 2012 at 4:18 PM, Naftoli Gugenheim <nafto...@gmail.com> wrote:
> Very exciting!
>
> One thing I could use sometimes in a test framework is a way to fail without
> terminating. Sometimes it's useful to get a table of successes and failures
> --- sometimes you can see a pattern that way, that you can't see when the
> failure terminates the test.
>

Can you post a specific example of such a situation to help me
understand the use case?

> Also, one thing I miss in ScalaTest that Specs has is showing a "diff" of,
> e.g., two strings. For instance suppose you're generating a JSON string and
> comparing it against the intended result. The test reports that they're not
> equal but the difference is hard to spot. Even better would be arbitrary
> objects (perhaps a diff of the toString, or configurable, e.g., class
> Decompose[A] extends (A=>List[Symbol,Any])).
>

Actually ScalaTest has provided string diffs for years, so long you
mean things like "h[o] did not equal h[i]", but only when using
assertions. For about two years I didn't realize it wasn't working in
matchers (because I didn't notice and no one complained about it), but
it does as of 1.7.1. So just use that release instead and you'll have
your diffs in matchers too.

Bill

[Naftoli Gugenheim]

On Sunday, February 26, 2012, Bill Venners wrote:

Hi Naftoli,

On Sun, Feb 26, 2012 at 4:18 PM, Naftoli Gugenheim <nafto...@gmail.com> wrote:
> Very exciting!
>
> One thing I could use sometimes in a test framework is a way to fail without
> terminating. Sometimes it's useful to get a table of successes and failures
> --- sometimes you can see a pattern that way, that you can't see when the
> failure terminates the test.
>
Can you post a specific example of such a situation to help me
understand the use case?

Usually it would be in conjunction with ScalaCheck.

Reactive-core's DeltaSeq propagates sequence diffs across operations like map, flatMap, filter, etc. I tested it using ScalaTest and ScalaCheck. Some situations were not working, some were. If I could see for instance that all the problems involved cases of Includes, while Removes all worked, it would be helpful.

A POS webapp presented an ajax-editable table of items and of payments, displaying in real time totals, credit surcharges, etc. Sometimes things would get out of sync. I wrote selenium-style tests (using reactive's testing system) that generated random interactions. It would be helpful to see if there was a pattern when things stopped working, and when they didn't. (Was it certain numbers that triggered rounding errors? Did something weakly referenced get garbage collected somehow so that once it broke it stayed broken?)

I suppose the ideal output format might be an html or csv table.

[Bill Venners]

Hi Naftoli,

OK. I have an html table planned for 2.0 that might address that need.
But I hadn't thought about having that work with generators though,
just with an explicitly specified set of values (a table). I figured
there would be too much information with generators. It is also aimed
more at acceptance testing. I'll look at whether there's a nice way to
offer such a thing in unit tests.

Bill

--