Inconsistencies.

1 view
Skip to first unread message

ijuma

unread,
Oct 2, 2008, 7:22:44 PM10/2/08
to specs-users
Hi,

I ran across a blog entry the other day where the author claimed to be
disappointed at some aspects of specs. I asked him to clarify and he
seems to raise some valid points. Eric, I thought you might be
interested so here's a link:

http://johlrogge.wordpress.com/2008/10/01/phasing-over-from-java-to-scala-retrospect-1/#comments

Regards,
Ismael

etorreborre

unread,
Oct 2, 2008, 11:33:59 PM10/2/08
to specs-users
Hi all,

I copy here the long comment that Joakim left on his blog about specs.
Then I will add my comment to each of Joakim's points.

Before that, I want to say that I appreciate Joakim thorough analysis
on specs and I encourage everyone on the list to share his/her own
opinion on the points below. I developed specs with limited feedback
and I'm sure this project will benefit from your own real-world
experience (well, it already has, as I had some great suggestions
already).

============================================
from: http://johlrogge.wordpress.com/2008/10/01/phasing-over-from-java-to-scala-retrospect-1/#comment-76

see my comments enclosed in ********
============================================

Of course I will give an example since I made a statement it’s only
fair. My main problem with specs is that it doesn’t resonate with me.
I find it unintuitive and I have not really been blown away by it. My
gut feeling is that I’m not too happy with it.

But you and Eric (I’m sure he is a great guy) deserve some more
concrete examples and I will try to elaborate. I realize that “syntax”
was sloppily put of me, some of my remarks are related to other areas.

Here are some of the things that I don’t care much for:

the ‘in’ keyword, I don’t see the logic. From the specs 5 minute
tutorial:

“‘hello world’ has 11 characters” in {
“hello world”.size must_== 11
}

To me ‘in’ does not make sense there. You could of course
replace ‘in’ with ‘>>’ which is what I have been doing but… i don’t
get in. I rspec it is easier since do is dictated by ruby, so no
choice there but in scala specs ‘in’ is chosen for some reason I don’t
get.

*********************
My original idea with "in" was that you would describe "something" you
expect from the system ( when declaring "my system" should ...)

Those "something" are examples:

"do this"
"do that"
"and all that stuff"

But a string is not executable and can't verify expectations so you
add an "implementation", a "concretization" of that example by being
more specific about the situation you're "in":

"do something" in { // this situation, this block of code }

When I started specs, I was a bit bothered by the mandatory use of
"describe", "it" and "do" in rspec. I wanted the user to be able to
stay as textual as possible with very few operators linking
descriptions and code together. Hence "should" and "in".

I was also aware that "in" may not make sense for some people, even if
it does for me, so I added a ">>" meaningless operator. I'm very open
to suggestions in that field, which other keyword could be
preferable?

Btw, one other reason for "in" was that it is only 2 letters long,
which is good for something you write often.

*********************

I think the specs don’t read well in the equal case which is a
really common one:

“hi” must beEqual(”hupp”)

a ‘to’ would help readabiliy:

“hi” must beEqualTo(”hupp”)

or

“hi” must equal(”hupp”)

To me readability is a big deal and overlooking such an obvious
thing tells me that the framework maker does not share my opinion here
which is fine but makes it less appealing to me.


This leads me to inconsistency:

Instead of:
“hi” must beEqual(”hupp”)

you can write:
“hi” must_== (”hupp”)

all of a sudden you have a different version of must while the
form in general is value must matcher. These small things makes it
slightly harder to grasp since there are exceptions to the general
rules.

*********************
Two things here.

The first one is "beEqual" which should be "beEqualTo", I agree and
will fix that.

On the "must_==" front, the motivation here is again a "frequency of
use" argument. "beEqualTo" is indeed the most literate form of the
matcher and follow the convention "value must matcher".

Yet, the equality matcher is used very frequently, so I think that
having a short cut is very convenient. I've pushed this idea for other
matchers but so far "must_==" is the one I use most.

*********************

Another example of inconsistency:

“hi” must equalIgnoreCase(”hupp”)
“hi” must beEqual(”hupp”)

why “be” in the second case but not in the first?

*********************

You're right about this and I will fix it too.

*********************

About redundancy:

For every matcher there is a “not” version

“hi” must notEqualIgnoreCase(”hupp”)

(inconsistently there is not a notBeEqual)

But I much prefer the

“hi” must not(beEqual(”hupp”))

form since it renders all the notXxxx methods redundant. Why
specs supports both is beyond me. Specs is not very opinionated but
seems to provide as many options as possible in the hope that there is
something for everyone. To me it’s just confusing. My remove
duplication bells ring all over the place :S)

*********************

Now, this is maybe a matter of personal preferences here.

I like notXXX matchers because, as a user, I have less parenthesis to
write and, to my eyes, they hurt readability. There is indeed API
redundancy since you can negate a matcher, but I would say that it is
a facility offered by the library maintainer to its users (at its own
expense of maintaining it, and maybe forgetting some notXXX matchers
sometimes).

In that respect, specs doesn't follow a "Python-like" attitude where
there is only one right way to do things. I personally appreciate this
flexibility but I understand that this can be confusing. I'm sometimes
considering the creation of "Profile" traits where you would compose,
personalize specs fine-grained behavior to your own liking.

*********************

Often naming is something I have a problem with.

What does:

“myCar must be also null”

mean?

myCar must beAlsoNull

Well: “a must beAlsoNull(b) is ok if a is null when b is null or
a is not null when b is null”

Que? That one is so weird that it must make perfect sense in
some context that I’m not aware of. An example would help :S

*********************

I'm open to every suggestion on naming. On a normal project I would
have spent hours discussing names with my teammates,...

beAlsoNull is really an edge case and I agree that the name isn't
really appropriate. It can be written differently as 2 expectations
(I've added the following to the wiki site):

thisObject must beNull.when(thatObject == null)
thatObject must beNull.when(thisObject == null)

or maybe

((thisObject == null) && (thatObject == null) || (thisObject != null)
&& (thatObject != null)) must beTrue

But you would get a less explicit error message with the second form.

Does anyone have a suggestion for this?

*********************

One feature I really liked at first glance was contexts but
what’s up with the ->- operator? And I can never remember if it goes
before or after “should”

*********************
Me too!

This is something I thought would be useful to create and pass
fixtures/contexts to leave only expectations in the examples
implementation. The "->-" operator was chosen to be read as a
"translation" for your system under specification and I wanted to keep
the context as close as possible to the system description:

val full = beforeContext(createStack(stack.capacity))
"A full stack" ->-(full)

In the code above, the "full" context is an "executable translation"
of what "A full stack" is.

Maybe one way to see and understand specs is to see it a "translation"
between descriptive, textual specifications and executable code:

"My system under specification" ->- anExecutableContext
"My system under specification" should { // a block containing
examples showing what the system should do }
"for example do this" in { // the example
implementation with expectations }

*********************

Also contexts can be a bit scary since in general you want to
set up some fixture with the context like:

var list = List[String]()

“lists with one item ” ->-(beforeContext({list =
List(”oneItem”)})) should {
“have a size of one” >> {
list.size must beEqual(1)
}
}

*********************
Actually, concurrency issues aside, I would reserve the ->- operator
to situations where you pass an already constructed object. If you
don't want to go this way, I think the more "classical" doBefore is
the way to go:

“lists with one item ” should { doBefore(list =
List(”oneItem”))
“have a size of one” >> {
list.size must beEqual(1)
}
}


*********************

Now when I find this part in the documentation:

“By default, the execution mode of examples of a Specification
is “on demand”. The examples are only executed when a Runner requests
the number of failures or errors. This enables a runner to do a
concurrent execution of examples.

However, if you need to execute samples sequentially (as in a
literate specification), you can use the method setSequential:

object mySpec extends Specification {
“The first functionality” should { setSequential

“have example 1 execute first” in {…}
// example 2 uses the results of example 1
“have example 2 execute in second” in {…}
}
}”

You start wondering what happens when you run a few tests like
that concurrently. The list is a variable so it mutates and a spec is
an object. Unlike JUnit you don’t get a fresh instance for each test
so you have to make sure that you don’t have unpleasant side-effects
due to concurrency but contexts seem to work against that, since you
don’t store things on the context instance but on the singleton spec.

I don’t want to have to think about those issues when I write specs, I
want to focus on the actual problem I’m trying to solve.

In a BDD framework I would expect the concept of contexts more
similar to:

var list =
given(”a list with one item”, List(”one item”}) {
when “an item is added” {list = list ::: List(”another item”) }
then list.size must be(2)
}

pardon my poor scala but I hope my point comes through…

*********************
This is a very valid point.

I haven't yet given hard thought to concurrent execution of specs (and
by concurrent I mean one thread by example, not several threads by
example).

I don't like very much the idea of sharing variables anyway between
examples. With JUnit, many of us made the mistake to use static
variables to initialize some persistent context between tests, as a
workaround for the "fresh instance for each test". And the result was
usually painful,... I don't think that any framework can prevent you
from doing this.

Yet, there is no current solution in specs to avoid variables sharing.
I've recently came across "Dynamic variables" in Scala (http://
www.scala-lang.org/docu/files/api/scala/util/DynamicVariable.html).
That might provide an elegant solution to this issue.

*********************

Those are some reasons why I’m not happy with specs, of course
YMMV, these are my highly subjective opinions.

Did I manage to make any sense?
/Joakim
============================================

Again thanks for the feedback. I'm sure specs has still a long way to
go and your insights, even when subjective (aren't we all?), are very
valuable. I hope I brought enough background to why specs looks the
way it is now.

Cheers,

Eric.

PS: and thanks to Ismael for relaying this blog post to the list!
Reply all
Reply to author
Forward
0 new messages