Native Scala mocking library - very early first draft
Paul Butcher
Here's an example of it in action (a reworked version of the example in Martin Fowler's "Mocks Aren't Stubs"[1]):
> class OrderTest extends WordSpec with MockFactory {
>
> "An order" when {
> "in stock" should {
> "remove inventory" in {
> val warehouse = mock[Warehouse]
> warehouse.expects('hasInventory).withArgs("Talisker", 50).returns(true).once
> warehouse.expects('remove).withArgs("Talisker", 50).once
>
> val order = new Order("Talisker", 50)
> order.fill(warehouse)
>
> assert(order.isFilled)
> }
> }
>
> "out of stock" should {
> "remove nothing" in {
> val warehouse = mock[Warehouse]
> warehouse.expects('hasInventory).returns(false).once
>
> val order = new Order("Talisker", 50)
> order.fill(warehouse)
>
> assert(!order.isFilled)
> }
> }
> }
> }
The source is on GitHub:
https://github.com/paulbutcher/smock
I'd be very interested in feedback (on the interface or the implementation). I'm still very much finding my feet in Scala, so I'm very open to any and all suggestions for how I could improve things. Thanks in advance.
[1] http://martinfowler.com/articles/mocksArentStubs.html
--
paul.butcher->msgCount++
Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?
http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: pa...@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher
Andreas Flierl
I think this looks very nice already. Just my 2c: I personally find verification of mock calls after the calls (Mockito-style [1]) much clearer. Just like one usually asserts things near the end.
Kind regards
Andreas
Paul Butcher
> Just my 2c: I personally find verification of mock calls after the calls (Mockito-style [1]) much clearer. Just like one usually asserts things near the end.
As it happens, Smock already allows you to switch off automatic verification. So instead of:
> class SomeTest extends SomeSuite with MockFactory {
> ... a test ... {
> val mock = mock[...]
> ... setup expectations ...
> ... exercise ...
> }
> }
You can do this:
> class SomeTest extends SomeSuite with MockFactory {
> disableAutoVerify
>
> ... a test ... {
> val mock = mock[...]
> ... setup expectations ...
> ... exercise ...
> verifyExpectations
> }
> }
Personally, given that I can't think of any situations in which I would want to setup mocks and then not verify them, I prefer the automatic verification approach. To my eye it cuts down on clutter and removes an opportunity to make the mistake of failing to verify your mocks. But I'm aware that it's a question of taste.
Note that verifyExpecations verifies all expectations on all mocks. I'm aware that some mocking frameworks take an "each mock verifies itself" approach like this:
> val mock1 = mock[...]
> val mock2 = mock[...]
>
> ... setup expectations...
> ... exercise ...
>
> mock1.verify
> mock2.verify
I'm not currently planning on doing this because it makes it difficult to specify interactions between mock objects. So, for example, I'm planning to allow the ordering of calls to different mocks to be specified with something like this (don't take the syntax too literally):
> val mock1 = mock[...]
> val mock2 = mock[...}
>
> inSequence {
> mock1.expects(...)
> mock2.expects(...)
> }
>
> ... exercise ...
Which will be very difficult to enforce if each mock only knows about itself.
I'd be very interested to hear your thoughts on whether this is likely to cause any issues. I will give some thought to whether I can find a way to construct things so that it supports both ways of working.
Daniel Sobral
Daniel C. Sobral
I travel to the future all the time.
Paul Butcher
> I suppose the comment was that instead of "setup expectations", you'd have "verify what happened" after the fact. I suspect that might be impossible, depending on how you are doing stuff.
Ah - having now looked at the Mockito documentation, I see what you mean :-)
I can't think of anything in what I have already written that would be antagonistic to that way of working - I might have a play to see how easy it would be to allow Smock to support both styles.
Having said that, my immediate reaction is that I'm unsure whether this style is appropriate for a language with a functional flavour like Scala. Perhaps this is just my entrenched habits speaking though?
Depending on how the code under test works, some things *have* to be setup beforehand. In particular, if the mocks' return values are going to be important to the code under test, you need to setup return values. Mockito does support this (it calls it "stubbing"), but the "normal" way of things seems to be that mocks return null or zero values with non-trivial return values being the exception.
If you do use Mockito's stubbing, then you're effectively back to the "specify expectations before exercising the code under test" style. As the Mockito documentation says:
> Although it is possible to verify a stubbed invocation, usually it's just redundant
I would guess that most Scala code (certainly more Scala code than Java code) will end up having to work this way?
But, that's little more than a gut reaction based on skimming the Mockito documentation without any experience of using it in anger.
> So, how does this framework fares with regards to reflection? And can you mock classes, or just traits? Previous discussions seemed to indicate there were limitations about what would be possible.
I'm not sure what you mean by "fares with regards to reflection"? Certainly mocks won't appear identical under reflection to the things that they're standing in for, but that's not likely to be important for anything other than a tiny subset of code. Unless I'm missing something?
It can only mock traits, not classes. And for that matter it can't mock other "difficult" things like constructors, static methods, etc.
For my purposes at least, this isn't a huge problem. In my experience, mocking classes (and the other "difficult" things) is only really of value when trying to retrofit tests to a significant body of existing code that hasn't been designed with testing in mind. I've never needed this kind of thing when constructing the code and tests in tandem.
There are a lot of interesting things that it doesn't support yet, but will soon (I hope). Foremost among these being higher order functions.
Andreas Flierl
Paul Butcher <pa...@paulbutcher.com> wrote:
> Depending on how the code under test works, some things *have* to be
> setup beforehand. In particular, if the mocks' return values are
> going
> to be important to the code under test, you need to setup return
> values. Mockito does support this (it calls it "stubbing"), but the
> "normal" way of things seems to be that mocks return null or zero
> values with non-trivial return values being the exception.
The normal way is what they show in the documentation:
when(mockedList.get(0)).thenReturn("first")
Is that what you mean with "non-trivial return value" already or did
you mean a calculated value of sorts?
So Mockito's way would be:
- mock something
- "stub" method invocations [optional]
- invoke some stuff
- verify what happened
> If you do use Mockito's stubbing, then you're effectively back to the
> "specify expectations before exercising the code under test" style.
> As
> the Mockito documentation says:
>
>> Although it is possible to verify a stubbed invocation, usually it's
>> just redundant
I use Mockito that way with Specs, though:
http://code.google.com/p/specs/wiki/UsingMockito
> For my purposes at least, this isn't a huge problem. In my
> experience, mocking classes (and the other "difficult" things) is
> only
> really of value when trying to retrofit tests to a significant body
> of
> existing code that hasn't been designed with testing in mind. I've
> never needed this kind of thing when constructing the code and tests
> in tandem.
Sadly (very much so), I almost always have to work with "a significant
body of existing code that hasn't been designed with testing in mind" :(
Being able to mock classes has helped me out many times in that
situation, as sad as I find that.
Paul Butcher
> The normal way is what they show in the documentation:
> when(mockedList.get(0)).thenReturn("first")
>
> Is that what you mean with "non-trivial return value" already or did you mean a calculated value of sorts?
Yes, this is exactly what I meant by a "non-trivial return value" (i.e. anything other than what Mockito describes as "null, an empty collection or appropriate primitive/primitive wrapper value (e.g: 0, false, ... for int/Integer, boolean/Boolean, ...)").
The example you give is exactly what Mockito means by stubbing, isn't it? (I recognise it from the section entitled "How about some stubbing?"). In which case, it goes on to say "Although it is possible to verify a stubbed invocation, usually it's just redundant". So, as you say, Mockito's way would be:
> - mock something
> - "stub" method invocations [optional]
> - invoke some stuff
> - verify what happened
But for most (functional style) Scala code doesn't that effectively become:
> - mock something
> - "stub" method invocations [theoretically optional, but in practice necessary for almost all Scala code]
> - invoke some stuff
> - verify what happened [but there's no point because it's redundant given that we're stubbing everything]
It seems that "the Mockito way" works well if and only if the majority of the methods you're mocking either have no return value, or the code that's calling them ignores the return value. Which will clearly be the case for some code, but probably only a minority of idiomatic Scala code?
What am I missing?
> Sadly (very much so), I almost always have to work with "a significant body of existing code that hasn't been designed with testing in mind" :(
> Being able to mock classes has helped me out many times in that situation, as sad as I find that.
You have my sympathy.
The challenges that I face are almost the direct opposite - I'm trying to write code on Android, which none of the existing mocking frameworks (that I've been able to find, at least) support because they all rely on code generation, which Android's Dalvik VM doesn't support. So I'm stuck (for the time being, at least - it's possible that a compiler plugin could help eventually) with techniques that will only work for interfaces/traits.
Having said that, the code that generates a proxy instance is pretty self-contained. It would be easy enough to replace it with something that used cglib for those platforms that support it.
If you fancy having a go at a cglib backend, perhaps we could collaborate on something that gives us the best of both worlds?
Kevin Wright
Kevin Wright
gtalk / msn : kev.lee...@gmail.com
Alec Zorab
used lambda's to great effect. I'm not sure if the same is possible in
scala because I believe Moq relied heavily on the expression trees
stuff that appeared along with Linq.
Worth looking at the getting started page
(http://code.google.com/p/moq/wiki/QuickStart) just to see the api in
action though
Andreas Flierl
Paul Butcher <pa...@paulbutcher.com> wrote:
> The example you give is exactly what Mockito means by stubbing, isn't
> it?
Yes, it is.
> (I recognise it from the section entitled "How about some
> stubbing?"). In which case, it goes on to say "Although it is
> possible
> to verify a stubbed invocation, usually it's just redundant".
Ah, yes. You totally have a point there but I object to this statement
from the Mockito site, which is why I linked to the specs example [1]
(the one under "usage"). The main benefit of this style is separating
the stubbing from the verification. But that's really just a matter of
taste, not brevity, I admit.
> But for most (functional style) Scala code doesn't that effectively
> become:
(snip)
> It seems that "the Mockito way" works well if and only if the
> majority of the methods you're mocking either have no return value,
> or
> the code that's calling them ignores the return value. Which will
> clearly be the case for some code, but probably only a minority of
> idiomatic Scala code?
Apart from the separation of stubbing and verification, you are
completely right.
> What am I missing?
Nothing, I guess. I hadn't looked at it that way before.
Although, keep in mind that albeit it would be totally cool if most
Scala code would be in that style (i.e. referentially transparent
methods/functions or at least meaningful return values), I see many
methods returning Unit and lots of mutable state in use in many projects
I looked at (at github, googlecode etc.). I am probably not qualified
enough to judge what proportion of the total Scala code is that way,
though. Just a thought.
> If you fancy having a go at a cglib backend, perhaps we could
> collaborate on something that gives us the best of both worlds?
Sorry, I fear I won't find the time for that in the foreseeable future.
:-(
Kind regards
Andreas
Paul Butcher
> The nicest mock framework I've used was Moq, a .net framework that
> used lambda's to great effect. I'm not sure if the same is possible in
> scala because I believe Moq relied heavily on the expression trees
> stuff that appeared along with Linq.
One of the really nice things in Moq is that it's statically typed. By contrast, what I've done is pretty much untyped (it just passes around AnyRefs all over the place).
I suspect, as you say, that this will be difficult without either code generation or something like expression trees.
Paul Butcher
> You could make this a lot nicer if `withArgs` were to accept a single argument of the rather awesome `Product` type (supertype of tuples and case classes).
Thanks for the pointer - I've not previously come across Product.
One of the things that I want to do in any case is allow wildcards and more general matchers. This might be exactly the right route to do so.