How to clear replay of a mocked obj?

[Vitaly]

I'd want to write the following code:

{
SomeObj someObj = mock(someObj);
... dosmth1..
verify(someObj, atLeastOnce()).someFunc()

... dosmth2...
verify(someObj, times(0)).someFunc()
}

But verify(someObj, times(0)).someFunc() always fails even if dosmth2
to doesn't call someFunc.
How to fix this?

[szczepiq]

Hi Vitaly,

There is no reset() function. You can just assign the mock again like this:

{

 SomeObj someObj = mock(someObj);
... dosmth1..
verify(someObj, atLeastOnce()).someFunc()

 someObj = mock(someObj);
... dosmth2...
verify(someObj, times(0)).someFunc()
}

but....

You're clearly testing completely different behaviour, so there should be 2 test methods and potentially setup method to reduce duplication:

setup {

  SomeObj someObj = mock(someObj);
}

{
... dosmth1..
verify(someObj, atLeastOnce()).someFunc()

{
... dosmth2...
verify(someObj, times(0)).someFunc()
}

I am not a fan of reset(). I think it doesn't promote simplicity and allow writing big test methods that test multiple behaviours.

However, I'd be interested if someone could provide a scenario where reset() is really useful :)

One quick thought about your code:

If you desire no interactions on someObj, you can use this method to make test code more descriptive:

verifyZeroInteractions(someObj);

Cheers
Szczepan

[Vitaly Berov]

Hi, Szczepan

Yes, I thought about recreating a mock, but I use MockitoAnnotations.Mock annotation (my example isn't totally correct here:) ) so I though there was a more elegant way.

I understand you reasons about reset functions, it can be misused. The problems is that dosmth2 heavily depends on dosmth1 (dosmth2 can't be done without dosmth1).

Thanks for verifyZeroInteractions advice :)

Vitaly

[Vitaly Berov]

Moreover, I can't recreate a mock because someObj is referenced by other objects. In other words, I need to check exactly someObj and not a new instance.

Vitaly

[szczepiq]

Vitaly,

I tried to design Mockito interface thin & simple so that it pushes back if you're trying to do something too fancy :) All that to keep test code simple and clean.

If I cannot test-drive the code using mockito it means that class under test is not easily testable. I refactor the class to make it testable. Also, I can always write a stub from scratch if it really requires extra stuff (but it never happened yet :)

If you can provide more code for your case then perhaps I can help.

Cheers,
Szczepan

2008/2/7 Vitaly Berov <vitaly...@nivalonline.com>:

[Vitaly Berov]

Ok, I see you point :)

Vitaly

[melody]

Hi,
I really like Mockito but I too would like a way to reset. Perhaps you
can help me with a better way. I basically want to be able to test the
same behavior for many account types. With easymock I wrote a test for
each category of account type (with about 10 per category). I reset
the mocks after each test. If I had to write a test case for each
account type it would be a large increase in lines of code. What I
want to do looks something like this:

@Test
public void syncGroupAccountsInsert() {
verifyInsertSync(AccountType.UNINCORPORATED_ORGANIZATION);
verifyInsertSync(AccountType.BUSINESS_TRUST);
verifyInsertSync(AccountType.CORPORATION);
verifyInsertSync(AccountType.INVESTMENT_CLUB);
verifyInsertSync(AccountType.GENERAL_PARTNERSHIP);
verifyInsertSync(AccountType.LIMITED_PARTNERSHIP);
verifyInsertSync(AccountType.LLC);
verifyInsertSync(AccountType.SOLE_PROPRIETORSHIP);
verifyInsertSync(AccountType.QUALIFIED_RETIREMENT_PLAN);
}

--Melody

On 7 Feb, 11:04, Vitaly Berov <vitaly.be...@nivalonline.com> wrote:
> Ok, I see you point :)
> Vitaly
> szczepiq wrote:Vitaly,
> I tried to design Mockito interface thin & simple so that it pushes back if you're trying to do something too fancy :) All that to keep test code simple and clean.
> If I cannot test-drive the code using mockito it means that class under test is not easily testable. I refactor the class to make it testable. Also, I can always write a stub from scratch if it really requires extra stuff (but it never happened yet :)
> If you can provide more code for your case then perhaps I can help.
> Cheers,

> Szczepan2008/2/7 Vitaly Berov <vitaly...@nivalonline.com>:Moreover, I can't recreate a mock because someObj is referenced by other objects. In other words, I need to check exactly someObj and not a new instance.
> Vitaly

[szczepiq]

Hi Melody,

I see your point. I'll think more about adding reset().

However, the verifyInsertSync() looks a bit suspicious. I'm not saying it's bad, though :) But If test method is getting too smart it may hint that class under test requires refactoring.

I'm guessing that you have a lot of different AccountTypes and some of them require insertSync() and some of then not. So the code under test is something like that:

    if (accountType == AccountTypes.X || accountType == AccountTypes.Y || accountType == AccountTypes.Z  .....  ) {
       insertSync();
    } else {
       doSomethingElseWithTheRestOfAccountTypes();
    }

Why not to move knowledge about syncable to enum?

   if (accountType.isSyncable()) {
       insertSync();
   } else {
       doSomethingElseWithTheRestOfAccountTypes();
   }

Then the unit test becomes very easy because you can verify insertSync() using single, syncable accountType.

Cheers,
Szczepan

[melody]

Hi Szczepan,
You are right that we are syncing certain account types. In this case,
the logic for which accounts are synced is already in another class.
The method call to the other class is static and would have to be made
an instance method and would need to be mocked. Then we would only
test the variations of account types from the class with the
isSyncable method. Since this class would not require any mocks, I can
easily test all of my account types. In this case that probably is the
best thing to do. I know that we have this type of scenario - where we
rely on reset - in quite a few of our tests. I can't say if it always
points to a smell but it is possible. Not having reset certainly
forces you to really evaluate where logic should be and where it
should be tested.

The only time I can think where this would be a bigger problem is in
dealing with legacy code which cannot easily be refactored. I'm not
sure how strong of an argument this is but if legacy code is
untestable and more "black box" I would want to leave my tests right
where they are since I might only be able to know what the legacy code
should do in the context of the code I am writing/testing. Granted, it
would have to be some really ugly legacy code, but we certainly have
code that is very bad.

--Melody

On 14 Feb, 18:03, szczepiq <szcze...@gmail.com> wrote:
> Hi Melody,
>
> I see your point. I'll think more about adding reset().
>
> However, the verifyInsertSync() looks a bit suspicious. I'm not saying it's
> bad, though :) But If test method is getting too smart it may hint that
> class under test requires refactoring.
>
> I'm guessing that you have a lot of different AccountTypes and some of them
> require insertSync() and some of then not. So the code under test is
> something like that:
>
> if (accountType == AccountTypes.X || accountType == AccountTypes.Y ||
> accountType == AccountTypes.Z ..... ) {
> insertSync();
> } else {
> doSomethingElseWithTheRestOfAccountTypes();
> }
>
> Why not to move knowledge about syncable to enum?
>
> if (accountType.isSyncable()) {
> insertSync();
> } else {
> doSomethingElseWithTheRestOfAccountTypes();
> }
>
> Then the unit test becomes very easy because you can verify insertSync()
> using single, syncable accountType.
>
> Cheers,
> Szczepan
>

> > > Szczepan2008/2/7 Vitaly Berov <vitaly.be...@nivalonline.com>:Moreover, I

[Igor Czechowski]

Hi Melody,

I can only assume the implementation of verifyInsertSync(...) method.
It would be much easier for me to help, if you reveal a little bit
internals of verifyInsertSync(...).

Thanks,
Igor

[szczepiq]

Melody,

>You are right that we are syncing certain account types. In this case,
>the logic for which accounts are synced is already in another class.
>The method call to the other class is static and would have to be made
>an instance method and would need to be mocked. Then we would only
>test the variations of account types from the class with the
>isSyncable method. Since this class would not require any mocks, I can
>easily test all of my account types. In this case that probably is the
>best thing to do.

Now you're talking :) Mocking is always heavyweight compared to dead simple asserts. If you can test isSyncable outside of the mocks' playground - that's the way to go.

>The method call to the other class is static and would have to be made
>an instance method and would need to be mocked.

Maybe. You can also keep it static and verify insertSync() using only relevant variants of accountType. Obviously in that case nicely isolated implementation of isSyncable creeps into your insertSync() test. Sometimes it's just the way to go: keeping it simple.

Don't get me wrong: I'm not in love with static methods. In your case I would kill this static entity that decides who is syncable. Instead, I would let account types know if they are syncable: boolean field for example?

That's just guessing really - I don't know your code (see, I'm a refactoring geek and it was you who started! ;).

> The only time I can think where this would be a bigger problem is in
> dealing with legacy code which cannot easily be refactored.

Given a horrible legacy class, you've got to refactor it anyway to apply some tests using mocks, right? E.g. push some logic to separate classes so you can mock out those classes. Maybe you're on something here, but I think there are so many other problems in dealing with legacy code that lack of reset() would be minor one.

So what's your opinion about reset():  +1 or -1 ?

I think I'm still -1 but I'm not opinionated (hell no:). It's just I've never had to use it (even in EasyMock) and I'm so fond of tiny API that pushes to refactor the application code instead hacking test code :D

Thanks,
Szczepan Faber

[SkullCruncha]

No reset()!

You have the API correct - there should be no reset.

I used to code reset in my hand coded mocks when I switched from NUnit
to JUnit, because I couldn't run setup functions to create new mock
objects for each test. With JUnit 4.0 that became unnecissary, and
there's no need to add it to Mockito!

Keep up the good work.

[Vitaly Berov]

It's not an argument.
I can say the same: "Reset must be preset!"

[Phil]

+1 for reset()

There's just too many times where you want to assert intermediate
results. If you force

myMock.op();
verify(myMock).op();
verifyNoMoreInteractions(myMock);

// XXX

myMock.op();
verify(myMock, times(2)).op();

you can't insert further test code at point XXX without having to
"refactor" all the times(2) ops that follow.

FWIW... I've just discovered mockito today, having just ranted to a
colleague about the "set expectations first" approach of EasyMock.
Loving it, and it has every chance of deprecating my crufty homespun
mock library I use for The Grinder (http://grinder.svn.sourceforge.net/
viewvc/grinder/trunk/source/tests-src/net/grinder/testutility/
RandomStubFactory.java?view=markup if you are interested).

[szczepiq]

Hi Phil,

>you can't insert further test code at point XXX without having to
>"refactor" all the times(2) ops that follow.

You mean something like that:

@Test public void test...() {
   
    setup1();
    
    someCodeThatInteractsWithMocks();
   
    verify1(....)
    verify1(....)

    reset();

    setup2();

    someCodeThatInteractsWithMocks();
  
    verify2(....)
    verify2(....)
}

Doesn't second verification block deserve its own test method?

Can you show an example from your code where you find reset() helpful?

>has every chance of deprecating my crufty homespun
>mock library I use for The Grinder

Thanks for feedback! Don't hesitate to post any suggestions.

Cheers,
Szczepan Faber

[Philip Aston]

(sorry for the delay).

Your example is nearly what I mean, but without the second setup().

@Test public void test...() {

setup1();

someCodeThatInteractsWithMocks();

verify1(....)
verify1(....)

reset(mock1, mock2);

someMoreCodeThatInteractsWithMocks();

verify2(....)
verify2(....)
}

Maybe its just me, but I often find it more obvious to write longer
procedural tests. Particularly if you are doing something that requires
non-trivial setup, and you want to check things are OK after step 1
before going on to step2. Without reset, I may be unnaturally forced to
write:

@Test public void test1...() {
setup1();

someCodeThatInteractsWithMocks();

verify1(....)
verify1(....)
}

@Test public void test2...() {
setup1();

someCodeThatInteractsWithMocks();

someMoreCodeThatInteractsWithMocks();

verify2(....)
verify2(....)
}

which just feels wrong.

If you are looking for more concrete examples, check out some of The
Grinder tests, e.g.
http://grinder.svn.sourceforge.net/viewvc/grinder/trunk/source/tests-src/net/grinder/communication/TestMessageDispatchSender.java?view=markup
(This uses my home grown mocking framework rather than Mockito).

Finally, I also think it's similarly annoying that I can't do this:

stub(mymock.op()).toThrow(someException);
// ...
stub(mymock.op()).toReturn(blah); // Can't change behaviour - throws
// someException.

Regards,

- Phil

> <http://grinder.svn.sourceforge.net/viewvc/grinder/trunk/source/tests-src/net/grinder/testutility/RandomStubFactory.java?view=markup>
> if you are interested).
>
>
>

[Igor Czechowski]

Hi Philip,

Actually, you are naturally forced to split the test into two separate
ones and you naturally end up with more fine grained tests and get rid
of duplication - which is as much important as in production code.
Tests are much easier to read, understand and maintain.

Maybe the example we debate about is not the best one, however after
the split it became obvious that test1 is a subset of test2. So do you
really need both of them ?

I rather think your intention was to make assumption about
interactions, that have already taken place, before you continue to
introduce some more. Besides, you want to be sure that second method
invocation has introduced some new interactions. Apparently, verifying
all at the end wouldn't prove where interactions have happened. Is it
the intention you would like to have reset for ?

Personally form me, these are candidates for separate tests and
excessive setup is a hint that maybe design is not the best one
possible (hint not a syndrom).

Second, you may simplify it a little bit :)
@Before
public void setup() {
}

@Test public void test1...() {
someCodeThatInteractsWithMocks();

verify1(....)
verify1(....)
}

@Test public void test2...() {

someCodeThatInteractsWithMocks();

someMoreCodeThatInteractsWithMocks();

verify2(....)
verify2(....)
}

What's wrong with writing times(2) ??

What's between stub and stub ? Verification ? Execution logic ?

stub(mymock.op()).toThrow(someException);
// ... ????????????????????

stub(mymock.op()).toReturn(blah); // Can't change behaviour - throws

Cheers
Igor

[szczepiq]

Hi Phil,

>stub(mymock.op()).toThrow(someException);
>// ...
>stub(mymock.op()).toReturn(blah); // Can't change behaviour

This is true, unfortunately there is no way of mitigating it due to the nature of mockito syntax (actually there is one but very hacky :). However, it's a very rare case, because good tests (e.g: small and focused) would look like this:

@test {

    stub(mymock.op()).toThrow(someException);
    ...

@test {

    stub(mymock.op()).toReturn(blah);
    ...

I'm stubbing same method (and same arguments) but with different return values. Clearly they hint separate variants of behavior and they deserve separate test methods :)

But I can see your point - it might be annoying in some very rare cases. I'd be interested if you could give us a good example where it bit you. Perhaps we can do something about it.

Cheers,
Szczepan Faber

[Philip Aston]

Igor, Szczepan,

Thanks for the feedback.

I'm quite willing to believe that any example I can come up with can be
reduced in this manner; and in fact I've had little trouble in working
around these issues. However, I find it annoying that the tool is taking
away my choice of how I want to structure tests. I'd rather be focussing
my refactoring time on the code under test.

(I find it particularly annoying, because otherwise Mockito is very
pretty and I've begun to care about it).

- Phil

[szczepiq]

Thanks for your input Phil - it's always good to hear interesting opinions.

>I'm quite willing to believe that any example I can come up with can be
>reduced in this manner;

We don't want to reduce your examples - not at all! A good example is
a main motivator to enhance the library and we are eager to do it.
Trust me, I looked at
TestMessageDispatchSender.

>and in fact I've had little trouble in working around these issues.

Missing reset() is not an issue. It's a deliberate action to promote
good testing habits and make the API simpler. I'm strongly biased in
favour of frameworks/tools that enforce good practices.

>However, I find it annoying that the tool is taking away my choice of
how I want to structure tests.

Frankly, that's the point. I think the goal of any tools (and even
practices or methodologies) is to reduce choices, leaving only good
choices. Small, focused test method is a good choice.

>I find it particularly annoying, because otherwise Mockito is very
>pretty and I've begun to care about it).

I'm sorry to disappoint you :( For me, mockito is not a mere test
tool. It's also a vision of how I see decent test code: simple, clean,
small. Long, procedural-like tests don't fit there. IMHO, Long,
scenario-like tests make sense for acceptance/functional tests where
mocks should not be used anyway.

I'm not saying NO to reset(). I've just never needed it. More over, I
used easymock for years and never used it. Whenever I saw reset() in
existing code, the test was dodgy and reset() was an attempt to
deodorize different smells in code.

Obviously, I can be wrong. That's why I'm still waiting for decent examples.

Cheers,
Szczepan Faber

[Philip Aston]

I'm willing to believe.

Unfortunately, I won't have a chance to use Mockito in anger until I up
The Grinder to Java 5. In the meantime I'll keep my eye out for
interesting, multi-step unit tests.

I strongly suggest that a distillation of this should be in the FAQ.
Here's a quick draft:

Q. Each mock keeps a full account of all its calls. Why can't I "reset"
a mock in the middle of a test?

A. The lack of a reset method is deliberate to promote good testing
habits and to make the API simpler. Re-factor your test into a number of
smaller tests.

Q. Once I've stubbed a mock to throw an exception, how can I reset this
behaviour?

A. You can't. Once a method has been stubbed to throw an exception, it
will always throw an exception. Unfortunately there is no clean way of
implementing this due to the nature of Mockito syntax.

However, the ability to alter a stubs behaviour is rarely required.
Consider splitting your test into two smaller tests, or using two
separate mocks. If you have an example of a unit test that requires this
ability, please send details to the Mockito Google group.

- Phil

[szczepiq]

> I'm willing to believe.

Thank you :)

> I strongly suggest that a distillation of this should be in the FAQ.

Thanks for a suggestion. I'm going to put distilled version of some
interesting threads in FAQ. Questions about restet() and alike tend to
come up very often.

Thanks,
Szczepan Faber

[ste...@gotruemotion.com]

Hi,

I ran into what I believe is a compelling use case for reset.  I have an object O that has internal states and I want to test transitions between its states.  It interacts with (at least) one mock object. Suppose O initializes to state A, and I want to test the transition B -> C.  I have to call O's method to transition it from A -> B, which causes some interactions with the mocks.  Then I want to reset the mocks, call O's method to transition B -> C, then verify that the mocks got the expected calls.

Without the ability to reset, I have two choices, neither of which is good:

1. I could make the test for B -> C look for the union of calls from A -> B and B -> C.  That makes the test for B -> C fragilely dependent on the A -> B transition.
2. I could get into the implementation of O and add the ability to initialize it into whatever state I might want.  That would add a bunch of extra code only used in testing and that has to result in the same thing as other code.  In other words this extra code has to be kept in sync with the real transition code.
   

The philosophy of designing a tool to steer its users toward good practices and away from bad is fine, but it can be misapplied.  I've had the misfortune of wasting inordinate amounts of time fighting with tools that don't work for any use case that the authors didn't have in mind when they wrote it.

Thanks for creating a great tool!

Steve Clark