A pattern/gem for keeping test doubles in sync with production objects

[Steve Jorgensen]

Hi all,

One downside that I'm always fighting with in Ruby and other dynamic languages is that tests are expensive if they involve too many classes or classes with too many responsibilities, but when testing classes in isolation, it's hard to be sure that the test doubles are in sync with the actual objects that the test subject will interact with.

Recently, I created the no-moss gem (https://github.com/livingsocial/no_moss) with input from Sam Livingston-Gray, and it seemed to be working out well at first, but has not quite panned out over time.

The next strategy I tried was employing more dumb data objects that could be used instead of ad hoc test doubles, but there were often good reasons for having smarter objects be used for the interaction such as when an attribute value was expensive to compute and not always needed.

My newest idea is to have a data-like object that can be given either simple values or callable objects for its attributes. When an attribute is read, if the underlying value is a callable (something that responds to #call) then the result of invoking #call is returned. Otherwise, the object is returned. Also, the result of a #call invocations is memoized, so whatever execution cost it has will only be paid once.

Since these objects can be treated as simple data objects, they should be great as stubs, and since attributes can be filled using lambdas, they can delegate to attributes of ad hoc mock objects as well.

The gem: https://github.com/stevecj/consensus

I just finished writing this and pushing it up, so I don't know yet whether it will work out well, but I am hopeful.

As of this writing, there is also some experimental functionality on the "fill-attributes-from-other-object" branch that I think will be useful. I'm holding off merging that into master because it might be premature (given that I haven't tried to use the basic functionality in a real project yet).

Thoughts, opinions, reactions?

-- Steve J.

[Rolf Bjaanes]

Hi Steve,

Have you looked at rspec-fire or bogus to solve the problem of keeping doubles / mocks in sync?

https://github.com/xaviershay/rspec-fire

https://github.com/psyho/bogus

Practical Object-Oriented Design in Ruby by Sandi Metz also solves this problem in a nice way.

http://www.poodr.com/

Cheers,

Rolf

--
You received this message because you are subscribed to the Google Groups "Objects on Rails" group.
To unsubscribe from this group and stop receiving emails from it, send an email to objects-on-rai...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Piotr Szotkowski]

Steve Jorgensen:

> when testing classes in isolation, it's hard to be
> sure that the test doubles are in sync with the actual
> objects that the test subject will interact with.

You might be interested in Bogus¹, which fails stubs and mocks
if the test double does not respond to the message you stub/mock
or the message takes a different number of arguments.

Bogus also attempts² to implement contract tests: if tests
for Foo stub/mock an interaction with a collaborator Bar,
you can enforce that tests for Bar actually do exercise the
exact scenario that’s being stubbed/mocked in Foo’s tests.

¹ https://github.com/psyho/bogus
² It’s Still an Experimental Feature™

For stub/mock synchro there’s also rspec-fire³, and
I think this functionality might be rolled into RSpec 3.

³ https://github.com/xaviershay/rspec-fire

> My newest idea is to have a data-like object that can be given
> either simple values or callable objects for its attributes.
> When an attribute is read, if the underlying value is a callable
> (something that responds to #call) then the result of invoking
> #call is returned. Otherwise, the object is returned.

Bogus’s implementation is similar; this seems to work quite well. The
clumsy situation is when you want the stub to return a callable – in
these cases you need to wrap it in an additional callable layer.

> The gem: https://github.com/stevecj/consensus

I like the idea, but I fear it applies best to greenfield projects
(where you can indeed use it both in tests and code under test)
and might need some more work to be applicable to many real-world
edge cases – but I’m looking forward to how it how it develops!

— Piotr Szotkowski
--
How many seconds are there in a year? If I tell you there are 3.155×10⁷,
you won’t even try to remember it. On the other hand, who could forget
that – to within half a percent – π seconds is a nanocentury.
[Tom Duff]

[Steve Jorgensen]

I am aware of rspec-fire, and it has never felt like the right approach to me for several reasons. It still requires a test of one thing to know the exact constant name of something else that will fill its dependencies. It assumes that there is one canonical other thing that defines the API for a test-double's role, when there could actually be several. It requires that the actual class be loaded (which can be expensive). It can't handle the case of methods that are created dynamically on demand.

I am a big fan of POODR and definitely use testing techniques presented in that book. That material is not comprehensive though, and does not cover every kind of situation. It would be nice to come up with a pattern and tools that tend to lead the programmer (e.g. me) who is sometimes lazy and not always firing on all thrusters to naturally do things in a way that produces a fairly good outcome. I am hopeful that "consensus" will turn out to be a helpful tool in that regard.

[Kerry Buckley]

On Wed, Feb 12, 2014 at 7:06 AM, Steve Jorgensen <ste...@stevej.name> wrote:

Hi all,

One downside that I'm always fighting with in Ruby and other dynamic languages is that tests are expensive if they involve too many classes or classes with too many responsibilities, but when testing classes in isolation, it's hard to be sure that the test doubles are in sync with the actual objects that the test subject will interact with.

Out-of-sync doubles is one of those potential issues with dynamic languages that I find doesn't seem to bite me as often as you'd think, but in any case you might find this talk by Gary Bernhardt interesting if you haven't seen it already:

https://www.destroyallsoftware.com/talks/boundaries

He talks about an approach which minimises the need for test doubles, but still allows objects to be tested in isolation.

Kerry

[Piotr Szotkowski]

Steve Jorgensen:

> [rspec-fire] still requires a test of one thing to know

> the exact constant name of something else that will fill
> its dependencies. It assumes that there is one canonical
> other thing that defines the API for a test-double's
> role, when there could actually be several.

FWIW, Bogus can be configured to work against interface-like situations;
for example, you can specify that a double stands in for a ‘logger’ and
say a logger is the lowest common denominator of LocalFilesystemLogger,
NetworkLogger and AvianCarrierLogger and then the stubs/mocks will fail
if a given call is not supported by all of them.

> It requires that the actual class be loaded (which can be expensive).

Right, this is a problem that’s hard to side-step.

> It can't handle the case of methods
> that are created dynamically on demand.

This is a big problem with doubles for Active Record
models; Bogus special-cases them (but this is ugly). :(

(Apologies if I sound like advertising Bogus here – I just
wanted to show how it tries to address these situations.
I’m really looking forward to how Consensus unfolds!)

— Piotr Szotkowski
--
My favorite productivity tip: sit down and do a bunch of work.
[Johnnie Manzari]

[Steve Jorgensen]

Hi Piotr,

Bogus seems to have similar goals to my no_moss gem. The difference of approach is that in no_moss, you define a role, verify that objects under test fulfill the role, and create wrappers around mocks/stubs to enforce the role. I like that because it makes everything conform to the roles instead of using some production classes as the definitions of the roles. Initially, I was getting really nice results from using no_moss, but applying it consistently is a discipline, and everyone who works on a project must learn the discipline and apply it in order for it to do much good. Other people I collaborate with have not caught the bug, and now I've managed to all but forget about its existence.

Anyway, regardless of whether something like bogus or no_moss is used, I think there is value in having production artifacts that are more structural than behavioral and act as boundaries that can be included in separate tests of inter-operating units. Technically, no library is required in order to follow such a pattern, but I think that Consensus (or something like it) might significantly facilitate that. I see Consensus as probably being most useful for implementing things like ultra-lightweight presenters or view models.

[Matt Wynne]

I once presented this problem to Steve Freeman a few years ago (one of the authors of the original mock objects paper) and he said to me "oh, that's just crap testing".

If you read Steve's book, GOOS, you'll learn that the assumption made by the people who first started using mocks was that you'd have multiple levels of tests. If your mocks in a lower-level test drift out of sync with the collaborator's API, you'll find out when a higher-level test that integrates the two units fails. I use this style, and have never felt the need for things like bogus and rspec-fire.

cheers,

Matt

--

http://mattwynne.net

email: ma...@mattwynne.net

twitter: @mattwynne

skype: mattwynne
google hangouts: matt....@gmail.com

tel: +447974430184

[Steve Jorgensen]

Yes. I have not tried to put that into practice as of yet, but I'm very interested in the approach. Have you tried using that, and if so, what have your experiences been?

-- Steve J.

[Kerry Buckley]

On 12 Feb 2014, at 09:39, Steve Jorgensen <ste...@stevej.name> wrote:

On Wednesday, February 12, 2014 1:12:02 AM UTC-8, Kerry Buckley wrote:

Out-of-sync doubles is one of those potential issues with dynamic languages that I find doesn't seem to bite me as often as you'd think, but in any case you might find this talk by Gary Bernhardt interesting if you haven't seen it already:

https://www.destroyallsoftware.com/talks/boundaries

He talks about an approach which minimises the need for test doubles, but still allows objects to be tested in isolation.

Yes. I have not tried to put that into practice as of yet, but I'm very interested in the approach. Have you tried using that, and if so, what have your experiences been?

Sorry, no – as I said mocks going out of step with classes that implement roles isn't something I’ve ever found to be a problem (yet!)

Kerry

[Steve Jorgensen]

OK, but what constitutes a "level" for purposes of this discussion? If we're talking in Hexagonal terms, would it be something like classes vs components (Hexagons)?

-- Steve J.

[Sascha Faun Winter]

Have y'all looked at the double verification in rspec 3? It aims to solve this problem.

https://github.com/rspec/rspec-mocks/tree/master/features/verifying_doubles

Faun

[Matt Wynne]

Well, a level is anything that clumps together smaller things from a lower level to give some meaningful behaviour. Steve and Nat call these "composite" objects, with the axiom being that a composite is "greater than the sum of its parts". In other words it's a new abstraction.

Getting more concrete, here's an example from the open-source code I know best at the moment:

Deep in the engine-room of Cucumber 2.0 we have this spec: https://github.com/cucumber/cucumber-ruby-core/blob/master/spec/cucumber/core/test/mapper_spec.rb

The mapper's responsibility is to take steps from your Gherkin specification like "Given I am logged in" and map them to a bit of Ruby code that can execute them.

The specs here assume that the "receiver" collaborator supports a protocol where you can send it a "test_case" message. The mapper receives "test_case" messages, creates a new test case with mapped steps, then passes that on to the receiver.

If I renamed the message to something else in that protocol, these tests would never let me know. They'd continue to pass.

Fortunately though, we have higher-level integration tests for the "execute" method on the entry point to the whole of the core:

https://github.com/cucumber/cucumber-ruby-core/blob/master/spec/cucumber/core_spec.rb#L208

They test that I can pass in a feature and some mappings, and have those scenarios execute as expected. They don't test all the edge-cases of the mapper's behaviour under different circumstances - we leave that up to the fine-grained tests for the mapper - but I'm confident that if we changed the protocol that the mapper must respond to, they would let me know.

Does that make any sense?

cheers,

[C. Farmer]

On Feb 12, 2014, at 8:04 AM, Matt Wynne wrote:

> Steve and Nat call these "composite" objects

I think this parallels what Gary Bernhardt was on about in that "Boundaries" talk that Kerry linked to earlier. It's an approach that at least intuitively does seem like it would mitigate against a lot of these out-of-sync-mocks troubles -- though I think the idea can be reduced to a simple principle I would state as "cluster your dependencies together." You have some objects that know the core logic of your program, and other objects that know how to assemble other objects together -- these latter are where the dependencies are clustered. The others are what more of your tests target.

[Piotr Szotkowski]

Sascha Faun Winter:

> Have y'all looked at the double verification
> in rspec 3? It aims to solve this problem.

> https://github.com/rspec/rspec-mocks/tree/master/features/verifying_doubles

Verified doubles provide one part of the solution (they make sure
that the doubled calls can be made at all, as far as method name and
argument counts go), but by themselves they don’t provide the other
part: that the call – when made against an actual collaborator
– wouldn’t blow up and would return what the double returns.

If you want to be sure that tests for class Foo actually exercise
all the calls that doubles of class Foo stub/mock elsewhere
in your test suite, then you need contract tests for class
Foo (or exercise all such scenarios in integration tests).

<shameless_plug>I gave a talk about this at wroc_love.rb and
the slides are at http://talks.chastell.net/wrocloverb-2014
– I tried to make them readable by themselves.
Please bear with all the awful puns.</shameless_plug>

— Piotr Szotkowski
--
Tech talk template: I don’t know what I’m doing, and here’s how.
[Pinboard]