Test Isolation with ScalaTest? - scalatest-users

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion Test Isolation with ScalaTest?
MIME-Version: 1.0
Date: Thu, 22 Jul 2010 18:44:54 -0700 (PDT)
In-Reply-To: <AANLkTik9tgH7NCdyBFLHvCAxX4fP7KWYvSLyP0okKxoX@mail.gmail.com>
References: <39c628e1-11b2-4d4c-ab4e-47dfc19b152f@s9g2000yqd.googlegroups.com> 
	<AANLkTik9tgH7NCdyBFLHvCAxX4fP7KWYvSLyP0okKxoX@mail.gmail.com>
User-Agent: G2/1.0
Message-ID: <5055380e-dd6d-42e7-ba89-cf4fcc30a21f@j8g2000yqd.googlegroups.com>
Subject: Re: Test Isolation with ScalaTest?
From: Sukant Hajra <a9h5mpz...@snkmail.com>
To: scalatest-users <scalatest-users@googlegroups.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Jul 17, 7:01=A0pm, Bill Venners wrote:
>
> I've finally got a bit of time to explain the idea behind the design and =
show
> you some alternative ways to write your tests.

Thanks for taking the time to follow up.  I actually muted my
subscription
after your original reply.  Fortunately, I Googled the thread to
revisit the
discussion.

> The main idea is to minimize what people need to know about the framework=
, to
> make it easy for people reading the code to guess what's going on. When y=
ou
> see a closure being updated, then later being updated again farther down =
the
> code, what you'd normally expect in Scala code is that the second update
> would see the changes to the first.

I'm definitely respectful of your concern.  Especially with a language
like
Scala internal DSLs can rapidly become dangerously expressive (rather
than
verbose like in Java).  Here's some of the arguments I have for the
other side:

    - Allowing for a little weirdness/non-intuitiveness in tests might
be okay
      if the behavior is broadly cross-cutting.

    - Test isolation is generally good, so it doesn't strike me as
inherently
      bad default.

    - Many times, we really can't shield end users from understanding
what a
      framework does for long.  A lot of frameworks try, but many
don't
      succeed.  And at the end of the day, it's not always so bad if a
little
      bit of a framework's internals are exposed, just as long as it's
not
      gushing out.

    - When writing tests, we really need to know the lifecycle of
      instance/closures anyway, because we're ultimately trying to
control with
      some specificity the setup of our tests; nobody wants to write
      broken/fragile/slow tests.

    - Perhaps what's intuitive might be a bit relative.  Whenever I
pass a
      closure to a third-party, I make no assumption on when or under
what
      conditions that closure will be called.  I ultimately have to
look for
      documentation or source code.

Please don't get the idea that I'm trying to be combative.  Right now,
there's
things I like about both Specs and ScalaTest.  ScalaTest's DSL seems
more
readable and straight-forward.  With Specs, I find the default test
isolation
just suits my personal preference, but overall, the framework has way
too many
bells and whistles for my tastes.

> If you want to isolate tests, I prefer that it be explicit in your code, =
such
> as by mixing in OneInstancePerTest. That way when someone else comes alon=
g to
> read it, they don't need to dig into the details of a framework's executi=
on
> model to understand what's going on.

I think this is a wonderful idea.  In fact, I tried to mix in
OneInstancePerTest to get the behavior I wanted, but I misunderstood
the exact
semantic (I understand now).

> That said, ScalaTest is designed to support different styles of testing, =
and
> the style I felt you wanted is not really the style intended by Spec. Wha=
t I
> felt you wanted was a "nested given/when/then" syntax

So here's my take on "given/when/then."  I don't find the
differentiation
between "given" and "when" to be all that interesting for most testing
situations.  I'd be happy enough with just "when/then" where "given"
is just
a special case of "when" as in:

    when("given an empty list") {
     =A0 =A0val list =3D ListBuffer[Int]()
        when("1 is inserted") {
  =A0 =A0 =A0     list +=3D 1
            then("it should have only 1 in it") {
  =A0 =A0 =A0 =A0       list should be (ListBuffer(1))
            }
        }
    }

Especially when working with side-effecting systems, I'm just getting
a system
to a certain state.  The nesting just lets me pop in and out of state
a little
for some finer-grained testing; essentially giving me some light code
reuse in
a natural way without needing to invest in separate helper methods.

However, I make no claims that this is necessarily a style that should
be
broadly applied.  The nesting introduces a complexity that might
impede
comprehension in some situations.

> This [WordSpec] has the disadvantage that the first line of each test is
> duplicated, although it does have the advantage that I only need to look =
at
> those three lines of code to understand the test. But ScalaTest has sever=
al
> ways to reduce that code duplication. The more functional way is to use t=
he
> org.scalatest.fixture package. I"ll show that last, but if what you were
> really after is Given/When/Then style documentation, you could mix in
> GivenWhenThen

Okay, so I looked at your examples for WordSpec and GivenWhenThen, and
the code
duplication was a little annoying, which is part of what I'm trying to
resolve
with nesting.  So let's take a look at the FixtureWordSpec approach.

> class MutableListScalaTestWordSpec3 extends FixtureWordSpec with
> ShouldMatchers {
>
> =A0 type FixtureParam =3D ListBuffer[Int]
>
> =A0 def withFixture(test: OneArgTest) {
> =A0 =A0 test(ListBuffer[Int]())
> =A0 }
>
> =A0 "A List" when {
>
> =A0 =A0 "1 is inserted" should {
>
> =A0 =A0 =A0 "have only 1 in it" in { list =3D>
>
> =A0 =A0 =A0 =A0 =A0list +=3D 1
> =A0 =A0 =A0 =A0 =A0list should be (ListBuffer(1))
> =A0 =A0 =A0 }
> =A0 =A0 }
>
> =A0 =A0 "2 is inserted" should {
>
> =A0 =A0 =A0 "have only 2 in it" in { list =3D>
>
> =A0 =A0 =A0 =A0 =A0list +=3D 2
> =A0 =A0 =A0 =A0 =A0list should be (ListBuffer(2))
> =A0 =A0 =A0 }
> =A0 =A0 }
> =A0 }
>
> }

My only problem with this is that I might have multiple assertions for
a given
action (like adding an element above).  I apologize my original list
example
didn't highlight this.  Even with the reuse of a fixture (system under
test),
I'm not getting reuse of contexts.  Of course, I could have multiple
"should"
asserts in a context, but then the text output of the tests loose
specificity.

So here's something Specs does that meets my needs (no duplication,
even with
multiple assertions for a context).

    class ListSpec extends Specification {

        val list =3D ListBuffer[Int]()

        val oneAdded =3D beforeContext {list +=3D 1}
        val twoAdded =3D beforeContext {list +=3D 2}

        "a list" when oneAdded should {

            "have only 1 in it" in {
                list must_=3D=3D ListBuffer(1)
            }

            "still have only 1 in it" in {
                list must_=3D=3D ListBuffer(1)
            }
        }

        "a list" when twoAdded should {

            "have only 2 in it" in {
                list must_=3D=3D ListBuffer(2)
            }

            "still have only 2 in it" in {
                list must_=3D=3D ListBuffer(2)
            }
        }
    }

I wonder how this meets with your sensibilities.  One thing I like
about this
is that it uses closures in what I think is a natural way, without
introducing
the complexity of something like OneArgTest.

By the way, I definitely am not making an argument to turn ScalaTest
into Specs
piece by piece.   I am using Specs currently because I figured out how
to make
it meet my needs and expectations.  But I am extremely interested in
testing
patterns, so even if I continue to use Specs, I'm interested in the
discussion.

> So lots of options, but the one that isn't in there now is the nested
> given/when/then one I showed first. I'm curious how people feel about the
> tradeoffs there.

Since I'm currently a happy (for now) Specs user, I don't feel I have
a horse
in this race.  But thanks for taking the time to write back.  It
really helped
me better understand your opinions on testing strategies and styles.

Best,
Sukant
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
