Mocking multimethods

[Alyssa Kwan]

Hi everyone,

Does anyone have any experience in mocking multimethods? I'm working
on a version control framework modeled after Git:

(def
^{:private true}
patches- (ref [])

(defn patches []
(seq @patches-))

(defn do-patch! [fn & args]
(dosync
(apply fn args)
(let [patch {:fn fn
:args (vec args)}]
(alter patches- conj patch)
patch)))

I need to be able to undo recorded patches, which means at undo-time,
translating the recorded do-patch! fn and args into an undo-fn which
can be called. If an undo-fn cannot be found, it throws an
exception. This has to be extensible so that as I add operations, I
can add the counterpart undo-fn. Multimethods to the rescue!

(defmulti undo-fn :fn)

(defn undo-patch [patch]
(let [fn (undo-fn patch)]
(dosync
(fn)
(alter patches- #(remove #(= patch %) %)))))

So in another package, I add the operation create-table! to the
system, which has a counterpart drop-table!:

(defn create-table!- [name columns]
(... create table stuff ...))

(defn drop-table!- [name]
(... drop table stuff ...))

(defn create-table! [name columns]
(make-patch! create-table!- name columns))

(defmethod undo-fn create-table!- [patch]
#(drop-table!- (:name patch)))

So calling undo-patch! with a patch that was the result of create-
table! will call undo-fn, which will dispatch on the :fn member of the
patch, which is create-table!-, which will end up returning an
anonymous function that calls drop-table!- with the :name from
the :args member of the patch.

OK, this all works.

But from a unit testing perspective, how do I cleanly test undo-patch!
without any undo-fn methods declared? When testing, I just want to
know that undo-patch! attempts to resolve undo-fn, will throw an error
if it can't, and will call the anonymous function that undo-fn returns
if it can. In other words, how do I mock undo-fn?

Thanks,
Alyssa

[Alex Baranosky]

Hi Alyssa,

Using the midje library I was able to do your first test.  I'm pretty tired so I this might be it for the night.

(fact "throws an error if can't resolve undo function"

  (undo-patch [2]) => (throws IllegalArgumentException "No method in multimethod 'undo-fn' for dispatch value: null"))

Is this what you were thinking?

Alex

[Alex Baranosky]

So I lied, I couldn't resist doing just one more:

(defn some-fn [] nil)

(fact "calls the anonymous function that undo-fn returns"

  (undo-patch ...patch...) => @patches-

  (provided

    (undo-fn ...patch...) => some-fn

    (some-fn) => nil))

The two provided statements are mboth mocking and stubbing.  So if either of those functions isn't called with the expected arguments they will fail.  The line 

(undo-patch ...patch...) => @patches- is saying when I call undo-patch with any patch, it should return the value inside of patches-  This is probably ok, since the different undo-fns are thoroughly unit tested.

There might be some better way to not have to create that some-fn that we just use for testing.  Maybe put it in a let or something?

Best,

Alex

P.S. Midje can be found on Github and Clojars. 

https://github.com/marick/Midje

[Alyssa Kwan]

Hi Alex,

Unfortunately the some-fn has to be created at the top level; it has
to be a var to be dynamically bindable, and Midje (and AFAIK, all
other mocking frameworks) use dynamic binding.

Here's what I came up with, using clojure.contrib.mock.test-adapter:

(deftest test-undo-patch!-calls-undo-fn
(def mock-fn +)
(def mock-undo-fn -)
(expect [mock-fn (times once (has-args [1 2]))
mock-undo-fn (times once (has-args []))]
(defmethod undo-fn mock-fn [patch]
mock-undo-fn)
(let [patch (do-patch! mock-fn 1 2)]
(undo-patch! patch))
(remove-method undo-fn mock-fn))
(ns-unmap *ns* 'mock-fn)
(ns-unmap *ns* 'mock-undo-fn))

Note that the multimethod cannot be mocked, so that we have to use
defmethod/remove-method. Extending the mock frameworks to support
this shouldn't be difficult.

The var for dynamic binding issue is a bigger deal. Does anyone know
if all statements in the deftest are guaranteed to be executed, in
case a statement in the middle throws an exception? Otherwise, the
namespace may not be cleaned properly.

Thanks,
Alyssa

On Dec 22, 1:26 am, Alex Baranosky <alexander.barano...@gmail.com>
wrote:

[Meikel Brandmeyer]

Hi,

maybe a different approach could be to use a richer datatype than a function, which carries both: the command and the undo command.

(deftype Command [action undo])

Then you could do something like:

(defn do-patch!
[command args]
(dosync
(let [patch {:command command :args (vec args)}]
(apply (.action command) args)

(alter patches- conj patch)
patch)))

(defn undo-patch!
[]
(dosync
((.undo (:command patch)))
(alter patches- pop)))

Then you could provide specially crafted Commands and there would not be the need to stub anything. Also Commands wouldn't have to be global objects.

Sincerely
Meikel

[Brian Marick]

I think I misunderstand the issue, because this works for me:

(ns midje.util.git
(:use [midje.sweet]))

;; Code except for undo-patch is the same as before.
;; ...

;; I pulled remove-patch out of undo-patch because I was
;; getting a screwy read error I didn't want to figure out
;; at 5 in the morning, but I'd probably do that anyway:
;; http://codebetter.com/jeremymiller/2006/12/03/composed-method-pattern/

(defn remove-patch [patch]
(alter patches- #(remove (fn [p] (= patch p)) %)))

(defn undo-patch [patch]
(let [fn (undo-fn patch)]
(dosync
(fn)

(remove-patch patch))))

(fact "The patch's undo-fn is called for its side effect and the patch is forgotten"
(let [visible-evidence-of-a-side-effect (atom nil)]
(undo-patch ...patch...) => anything
(provided
(undo-fn ...patch...) => (fn [] (reset! visible-evidence-of-a-side-effect :happened!))
(remove-patch ...patch...) => :nothing-of-interest)
@visible-evidence-of-a-side-effect => :happened!))

1855 $ lein midje midje.util.git
All claimed facts (2) have been confirmed.
1856 $

-----
Brian Marick, Artisanal Labrador
Contract programming in Ruby and Clojure
Author of /Ring/ (forthcoming; sample: http://bit.ly/hfdf9T)
www.exampler.com, www.exampler.com/blog, www.twitter.com/marick

[Alyssa Kwan]

I'd like to discuss this design approach. (It's unrelated to the
testing issue.) I avoided this design because the undo-fn is
determined at do-patch! time. The use case is for a persistent system
like Git where patches may be undone long after being done - e.g. long
after the patch is written to the database. In the meantime, the
system may be upgraded and the undo-fn for a patch may change. If the
undo-fn and args is written to the database at do-patch! time, for
backwards compatibility, at undo-patch! time, the undo-fn would have
to be updated to whatever the new call is.

Contrast that with my current approach where only the fn is saved and
the undo-fn is determined at undo-patch! time. For backwards
compatibility, the undo-fn would have to support legacy fn formats,
which I conjecture are slightly less likely to change than the
corresponding undo-fn. This is a pure guess and is not based on real
requirements yet; I don't know that fns themselves are necessarily
more stable than their counterpart undo-fns. It's a vague sense that
undo-fns are more subject to bug fixes: fns themselves are processed
right away at do-patch! time, while undo-fns are like time bombs that
lay dormant until a patch has to be undone, long after the initial
patch.

In addition, in the current implementation I outlined, the undone
patch disappears. The real implementation will most likely be a patch
with undo-patch! as the :fn and the patch being undone as the :arg.
This is necessary to support merging with remote branches like in
Git. undo-patch!'s may themselves be undone, which would require 2
degrees of backwards compatibility: converting the undo-fn in the
first undo, and converting the fn in the second undo. Both approaches
have the same 2 degrees of backwards compatibility, but the command
approach still somehow seems more fragile.

I'd love to get your thoughts on this. Anyone with experience working
on a similar system?

Thanks!
Alyssa

[Alyssa Kwan]

The issue is where do I specify that:

(undo-fn ...patch...) => (fn [] (reset! visible-evidence-of-a-side-
effect :happened!))

undo-fn is a multimethod in my design, which requires a corresponding
defmethod for each patch type. I need to create one for the scope of
the test, but defmethod by definition alters the top-level state of
the system, and doesn't automatically unroll like let or binding does.

[Brian Marick]

On Dec 22, 2010, at 6:52 AM, Alyssa Kwan wrote:

> The issue is where do I specify that:
> (undo-fn ...patch...) => (fn [] (reset! visible-evidence-of-a-side-
> effect :happened!))

The code you quoted is that specification. It doesn't matter that undo-fn is a multimethod.

Here's what the notation of the test says:

When called with an arbitrary patch, undo-patch will produce a particular side effect. It does that because it uses undo-patch, which--when given that arbitrary patch--returns a function that produces that side effect.
It also calls remove-patch with the given patch.
undo-patch can return anything it wants. We don't care.

>
> (fact "The patch's undo-fn is called for its side effect and the patch is forgotten"
> (let [visible-evidence-of-a-side-effect (atom nil)]
> (undo-patch ...patch...) => anything
> (provided
> (undo-fn ...patch...) => (fn [] (reset! visible-evidence-of-a-side-effect :happened!))
> (remove-patch ...patch...) => :nothing-of-interest)
> @visible-evidence-of-a-side-effect => :happened!))

[Alyssa Kwan]

Thanks, Brian! I obviously didn't understand the nature of the
provided form. That's really cool notation! This is exactly what I
want.

What are the cons of using midje? Any reason I shouldn't migrate all
my unit testing to it?

Thanks!
Alyssa

[Alex Baranosky]

I love Midje and think migrating all the tests to it is a great idea. 

[Brian Marick]

> What are the cons of using midje? Any reason I shouldn't migrate all
> my unit testing to it?

I'm biased, but I think Midje is ready for production use. There's a smooth migration path because Midje uses the same reporting as clojure.test, so you can mix styles in the same file and still use (for example) lein test to see if anything failed. (I'll go write up a Migrating page in the wiki after I send this mail.)

There are things clojure.test does that Midje doesn't:

* Named tests (deftest). So you can't write runner functions like this:

(deftest arithmetic-tests
(subtraction-tests)
(addition-tests))

unless you also :use clojure.test and wrap the Midje facts in deftests (which works fine).

* There's no equivalent to with-test.

* There's no #'are, though I suspect using checker functions would work as well.

Midje isn't a superset of the features of other clojure.test alternatives. For example, it doesn't have the auto-runner that LazyTest does, and it doesn't have the trimmed stack traces of Expectations. I plan to keep stealing ideas, though.