[ mocha-Bugs-17118 ] Expectations should take precedence over stubs

[James Mead]

I thought this comment on an existing bug report [1] was better suited to discussion on the mailing list.

I've added my responses in-line...

    Comment By: Rick Bradley (rickbradley)
    Date: 2008-08-05 19:15

    Message:
    Argh.  I've been getting bitten by this too.  At least I understand the source of the problem now.

I'm sorry to hear that you were bitten by this.

    Just my opinions, but:

    (1) From what I'm reading here, jMock seems broken.

Can you be more specific? Do you mean specifically jMock's "mock method dispatch" [2] mechanism?

    (2) It seems quite odd to emulate a Java mocking tool in a language as dynamic as Ruby, I've never met a Java testing or mocking tool that I've liked, though, to be fair, that's primarily because of Java as a language.

Mocha is not a port of jMock into Ruby. However, given the constraints of the Java language, I think jMock is an excellent mocking library. jMock has been around a lot longer than Mocha and was written by people much smarter than me, so I've tried (where appropriate) to benefit from their experience and not to repeat their mistakes. That being said, Mocha does take advantage of many of the dynamic features of Ruby.

Although I admit that my response in the original bug report did not make this clear, the change was made to deal with some real problems & inconsistencies I was seeing. The change was not just made to bring Mocha "into line" with jMock. It was more that using a "mock method dispatch" mechanism like jMock's solved the problems I was seeing.

    (3) I've never bought Jay's argument that no setup is not only a good thing but The Way It Should Be; until now I've not had any reason to care that I was on the other side of that issue.

This is not just Jay's argument. I know that (I and others) have independently come to this same conclusion. My link to his page was a shortcut to explaining why I prefer not to use setup methods. I hoped it might help people. I never intended to imply that this should be the One True Way. I'm quite happy to accept that other people might prefer to use setup methods and would prefer "mock method dispatch" to work a different way. See my comments about open source below.

    (4) The state-based mocking seems like it would promote more bad development practices than prolific use of setup methods -- most time I've seen the need for mocking states, "scenarios", etc., it's masking the fact that the code under test is poorly designed.  It SHOULD be painful to write tests for bad code, state-machining relieves the pain of testing poor code.

I assume you're referring to the Mocha::Standalone#states [3] functionality. I think it's confusing to talk about "state-based mocking", because it sounds too similar to the "state-based testing" referred to by Martin Fowler [4]. To be honest, I don't really understand your argument. Can you give an example of how mocking with states masks poorly designed code? As I see it, if you want a mock to have a default behaviour defined in a setup method and a different behaviour in some (but not all) of the tests, the mock object is by definition in different states, because it exhibits different behaviour in different tests. Using the Mocha::Standalone#states simply makes this change of state more explicit.

    (5) Regardless, this was a surprising (and quiet) change to mocha -- existing tests aren't going to break, and TDDing new code exposes this (in a confusing, "wtf, mocha's busted for some reason" way), but characterizing existing code gets a false sense of certainty.

I agree that this was a regrettably quiet change (although I think there was discussion about it on the mailing list at the time). In hindsight I might have done things differently and I can only apologise for the problems it has caused you.

    As far as I can tell, basically this change says "unless you write tests without setup methods, prepare to be surprised or be lulled into a false sense of confidence", without explicitly saying so... except for here, which took me months to find. :-/

The beauty of open source is that if you disagree with the way some software works, you can suggest a patch or even fork the project.

Here are some ideas for a constructive approach to solving your problem (in no particular order)...

- Suggest a patch which gives a warning when you might be "being lulled into a false sense of confidence".
- Write or modify existing acceptance tests to fully describe the behaviour you want.
- Change the "mock method dispatch" mechanism to work the way you want and see whether any acceptance tests break.
- Suggest a patch to provide an configurable alternative "mock method dispatch" mechanism.

I hope that helps.
--
James.
http://blog.floehopper.org

[1] http://rubyforge.org/tracker/?func=detail&atid=7477&aid=17118&group_id=1917
[2] http://www.jmock.org/dispatch.html
[3] http://mocha.rubyforge.org/classes/Mocha/Standalone.html#M000009
[4] http://martinfowler.com/articles/mocksArentStubs.html

[James Mead]

Here are my responses to a subsequent comment from the same bug report [1]...

Comment By: Yossef Mendelssohn (ymendel)
Date: 2008-08-06 00:17

Message:
I'd like to add my vote to this issue. I'm glad that the state machine feature makes what I want to test possible with a setup method (or more precisely, a before block, since I like RSpec -- and I think Mocha is better than the built-in RSpec mocking), but it's kind of a pain and not entirely nice to work with or look at.

I'm glad that you think Mocha is better than the built-in RSpec mocking. Is there a way we could make using the state machine less of a pain? e.g. a helper method...?

I understand that you prefer it to work a specific way, that you follow Jay Fields's examples of duplicating code in your tests. What I'm not understanding is

 1. Why it's not possible to write Mocha so that it can be used in a way others expect as well as your preferred style.

As you've already pointed out, it's not that it can't be used the way you want, it's just that it is a bit more awkward. You could suggest a patch to provide the alternative behaviour as a configurable option.

 2. Why Mocha needs to be brought into line with jMock v1, gotchas and all.

As I explained to Rick Bradley: Although I admit that my response in the original bug report did not make this clear, the change was made to deal with some real problems & inconsistencies I was seeing. The change was not just made to bring Mocha "into line" with jMock. It was more that using a "mock method dispatch" mechanism like jMock's solved the problems I was seeing.

 3. To take the words out of a colleague's mouth, why you would build a mocking library in a dynamic language modeled after *anything* written in Java?

Not everything in Java is bad. Not everything in Ruby is good. I like to cherry pick good ideas on their merits rather than based on some arbitrary prejudice. Both Java & Ruby are object-oriented langauges. Mock objects are all about object-orientation. It seems obvious that there are likely to be good ideas in jMock that would be worth using in Mocha.

Note that I have answers that may catch your initial responses, or at least point out that I've done some research.

 1. I admit there's probably more to this than just this one example. Constrained to just this, it seems like letting expectations take precedence over stubs would work with the way I (and the reporter, and some colleagues, and unknown others) want to write the tests as well as the way you (and Jay Fields, and unknown others) want to write them. It would also change other behavior that some people (you, Jay Fields, unknown others?) might depend on.

You are right - there is more to this. See my answer to your first point 2 (above). One thing you could try is change the "mock method dispatch" mechamism and see which Mocha acceptance tests break.

 2. Yes, I know that it's using "a syntax like that of jMock", but that doesn't mean it has to be the Ruby version of jMock. If it does mean that, at least you didn't call it rMock.

Actually in this case it's not about using a "syntax like that of jMock". In this case, I have chosen to use similar *behaviour* to jMock because it seemed like the best way to solve some specific problems.

 3. Dude, I don't know what to say. I'm not a fan of Java. At all. Especially Java files that have .rb extensions.

Thanks for such an insightful final comment.

What I don't understand is that you seem to have assumed that I would not want to help you. You've not made any constructive suggestions or offered anything in the way of a patch. Can you explain why exactly I (or anyone else on this list) should bother giving up any of their time to help you....?

[Yossef Mendelssohn]

Thank you for the reply, James. Some responses will be inline,
followed by a conclusion after the main body.

On Aug 6, 4:03 pm, "James Mead" <jamesmea...@gmail.com> wrote:
> Here are my responses to a subsequent comment from the same bug report
> [1]...
>
> Comment By: Yossef Mendelssohn (ymendel)
>
> > Date: 2008-08-06 00:17
>
> > Message:
> > I'd like to add my vote to this issue. I'm glad that the state machine
> > feature makes what I want to test possible with a setup method (or more
> > precisely, a before block, since I like RSpec -- and I think Mocha is better
> > than the built-in RSpec mocking), but it's kind of a pain and not entirely
> > nice to work with or look at.
>
> I'm glad that you think Mocha is better than the built-in RSpec mocking. Is
> there a way we could make using the state machine less of a pain? e.g. a
> helper method...?
>
> I understand that you prefer it to work a specific way, that you follow Jay
>
> > Fields's examples of duplicating code in your tests. What I'm not
> > understanding is
>
> >  1. Why it's not possible to write Mocha so that it can be used in a way
> > others expect as well as your preferred style.
>
> As you've already pointed out, it's not that it can't be used the way you
> want, it's just that it is a bit more awkward. You could suggest a patch to
> provide the alternative behaviour as a configurable option.

My main point was not that my workflow was not supported, since it
obviously is. The state-machine approach allows my needs to be met,
but in an awkward and involved way. You mention possibly using a
helper method to make this less of a pain, but I don't see what that
would do to alleviate the problem. It'll still mean I need to set up
an object to hold state and attach the stubs and expectations to the
different states. Hiding that behind a helper method still means I
have relatively the same amount of code in my tests that's not
actually directly related to testing (in my view).

>  2. Why Mocha needs to be brought into line with jMock v1, gotchas and all.
>
> As I explained to Rick Bradley: Although I admit that my response in the
> original bug report did not make this clear, the change was made to deal
> with some real problems & inconsistencies I was seeing. The change was not
> just made to bring Mocha "into line" with jMock. It was more that using a
> "mock method dispatch" mechanism like jMock's solved the problems I was
> seeing.

This was obviously a misunderstanding, then. I would be interested in
knowing what problems you were seeing that necessitated the change,
and I can do some digging to get a better understanding of the history
involved.

<snip snip>

>  3. Dude, I don't know what to say. I'm not a fan of Java. At all.
>
> > Especially Java files that have .rb extensions.
>
> Thanks for such an insightful final comment.

Okay, I deserved that. I have personal issues, I'll admit. Being in
the Ruby world for as long as I have and enjoying it as much as I do,
and especially being in the position of a consultant/independent
contractor, I see far too much of what I deem is completely
unnecessary Ruby code. Or more to the point, completely unnecessary
code in files that purport to be Ruby, but are really born from
concepts that don't fit in with "the Ruby way".

> What I don't understand is that you seem to have assumed that I would not
> want to help you. You've not made any constructive suggestions or offered
> anything in the way of a patch. Can you explain why exactly I (or anyone
> else on this list) should bother giving up any of their time to help
> you....?

This in part was due to misunderstanding your original response. At
the time I read it, it seemed very final about how Mocha would and
should work. Further interaction has proven that not to be the case.

> --
> James.http://blog.floehopper.org
>
> [1]http://rubyforge.org/tracker/?func=detail&atid=7477&aid=17118&group_i...

Overall, I'm a big fan of Mocha. I continue to use it because it feels
the most natural and sensible of all the Ruby choices out there.
Having looked into the internals before, I realize there's a lot of
complexity that goes into a mocking framework, especially one as rich
and powerful as Mocha, and like any project with a large userbase,
there are conflicting desires and expectations.

Allowing expectations to override stubs would let both your* and my*
ways of testing work without any awkwardness. Any conversation about
this (whether or not either of us happens to be involved) seems to
come down to a disagreement about how tests should be written, with
neither party willing to back down. I can respect that others work or
think in different ways, and my general goal is to allow for different
workflows to coexist at the same relative comfort level. (Where
possible and where sensible, of course. This doesn't mean that they
should both be made difficult.)

(* For lack of better labels.)

I'll make some time to look into the test suite, hopefully sometime
soon, and see what would be involved in having my desired behavior be
optional (or dare I dream? default).

--
-yossef