Printing the differences between expected an actual output

[Brian Marick]

At work, I use Midje for Clojure and Rspec for Rails. The single most obvious superiority of Rspec is how it shows differences between actual and expected values when the values are complex structures.

I'd like to make Midje better about that. To that end, I've today started to use `difform` https://github.com/GeorgeJahad/difform

If you want to see what that looks like, see Midje 1.6.4-SNAPSHOT. (It also contains various pull requests and updates to more recent versions of libraries.) Here are two examples:

user=> (fact [1 2 3 4] => [2 3])
FAIL at (form-init7087011490254113911.clj:2)
Expected: +
Actual: -
-----------
[
- 1
2 3
- 4
]

user=> (fact {:a 1, :b 2} => {:z 100, :a 1, :b 3, :c 88})

FAIL at (form-init7087011490254113911.clj:2)
Expected: +
Actual: -
-----------
{:a 1, :b
- 2
+ 3, :c 88, :z 100
}

For shorter collections, the old style will be used:

user=> (fact {:a 1, :b 2} => {:z 100, :a 1, :b 3})

FAIL at (form-init7087011490254113911.clj:2)
Expected: {:a 1, :b 3, :z 100}
Actual: {:a 1, :b 2}

I personally find the Rspec style of describing differences more understandable because it doesn't try to be minimalist. But `difform` will be easier to implement.

Opinions?


--------
Latest book: /Functional Programming for the Object-Oriented Programmer/
https://leanpub.com/fp-oo

[Giorgio Valoti]

I like the new style. It seems easier to spot the differences, especially in large collections.

--
Giorgio Valoti

>
>
> --------
> Latest book: /Functional Programming for the Object-Oriented Programmer/
> https://leanpub.com/fp-oo
>

> --
> You received this message because you are subscribed to the Google Groups "Midje" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to midje+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Ben Mabey]

The new diff output is a great improvement over the old way, I like it!

I'm curious, did you consider building something off of
clojure.data/diff[1]? Using that I think you could implement similar
(or better) to what rspec emits.

Either way, this is a great addition.

Thanks!

1. http://clojuredocs.org/clojure_core/clojure.data/diff

[Ben Mabey]

Now that I've been using the new way for a bit I have ran into cases
where the new output isn't very helpful. For example, look at this
output for a large vector containing maps:

https://gist.github.com/bmabey/7f389421386e45bf687c

The old way is preferable since I could at least copy the actual and
expected values and do a manual comparison in my editor. What do you
think about providing a flag that allows you to turn off the diffing?

-Ben

[Sebastian Hennebrueder]

Hi,

I find the diff hard to read. You probably get used to it, but if there was a easier to understand solution, I would prefer it. From the discussion there seems to be the need to see actual and expected output. From my own experience, I want to very often only what is wrong but not the matches.

I came up with 3 ideas:

Positions to indicate difference:

---------------------------------

I could imagine something like

   Expected: [1 3]

   Actual:   [1   2]

FAIL at (core_test.clj:21)

    Expected: (0 1 2 3 4 5 6 7 8 9 10 21    12 13 14 15 16 17 18)

      Actual: (0 1 2 3 4 5 6 7 8 9 10    11 12 13 14 15 16 17 18 19)

(fact {:foo 1 :bar 2} => {:foo 1 :bar 3})

Expected: {:bar 3,         :foo 1}

Actual:   {        :bar 2, :foo 1}

 

(fact (zipmap (map #(keyword (str "foo" %)) (range 20)) (repeat 1)) => (assoc (zipmap (map #(keyword (str "foo" %)) (range 20)) (repeat 1)) :foo8 2))

 Expected:    {:foo0 1, :foo1 1, :foo10 1, :foo11 1, :foo12 1, :foo13 1, :foo14 1, :foo15 1, :foo16 1, :foo17 1, :foo18 1, :foo19 1, :foo2 1, :foo3 1, :foo4 1, :foo5 1, :foo6 1, :foo7 1, :foo8 2,          :foo9 1}

      Actual: {:foo0 1, :foo1 1, :foo10 1, :foo11 1, :foo12 1, :foo13 1, :foo14 1, :foo15 1, :foo16 1, :foo17 1, :foo18 1, :foo19 1, :foo2 1, :foo3 1, :foo4 1, :foo5 1, :foo6 1, :foo7 1,          :foo8 1, :foo9 1}

(fact {:foo {:house {:room {:cupboards [{:name "foo" :age 22} {:name "bar" :age 22} {:name "bazz" :age 11} {:name "foo2" :age 22}]}}}}

=> {:foo {:house {:room {:cupboards [{:name "foo" :age 22} {:name "bar" :age 22} {:name "bazz" :age 12} {:name "foo2" :age 22}]}}}})

Expected: {:foo {:house {:room {:cupboards [{:age 22, :name "foo"} {:age 22, :name "bar"} {:age 12,          :name "bazz"} {:age 22, :name "foo2"}]}}}}

Actual:   {:foo {:house {:room {:cupboards [{:age 22, :name "foo"} {:age 22, :name "bar"} {         :age 11, :name "bazz"} {:age 22, :name "foo2"}]}}}}

Underlines

----------

I could imagine something like

   Expected: [1 3]

   Actual:   [1 2]

   Delta        -

FAIL at (core_test.clj:21)

    Expected: (0 1 2 3 4 5 6 7 8 9 10 21 12 13 14 15 16 17 18)

    Actual:   (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19)

    Delta                             --                      --

(fact {:foo 1 :bar 2} => {:foo 1 :bar 3})

    Expected: {:bar 3, :foo 1}

    Actual:   {:bar 2, :foo 1}

    Delta      ------  

 

(fact (zipmap (map #(keyword (str "foo" %)) (range 20)) (repeat 1)) => (assoc (zipmap (map #(keyword (str "foo" %)) (range 20)) (repeat 1)) :foo8 2))

    Expected:    {:foo0 1, :foo1 1, :foo10 1, :foo11 1, :foo12 1, :foo13 1, :foo14 1, :foo15 1, :foo16 1, :foo17 1, :foo18 1, :foo19 1, :foo2 1, :foo3 1, :foo4 1, :foo5 1, :foo6 1, :foo7 1, :foo8 2, :foo9 1}

    Actual:      {:foo0 1, :foo1 1, :foo10 1, :foo11 1, :foo12 1, :foo13 1, :foo14 1, :foo15 1, :foo16 1, :foo17 1, :foo18 1, :foo19 1, :foo2 1, :foo3 1, :foo4 1, :foo5 1, :foo6 1, :foo7 1, :foo8 1, :foo9 1}

    Delta:                                                                                                      

                      -------

      

(fact {:foo {:house {:room {:cupboards [{:name "foo" :age 22} {:name "bar" :age 22} {:name "bazz" :age 11} {:name "foo2" :age 22}]}}}}

=> {:foo {:house {:room {:cupboards [{:name "foo" :age 22} {:name "bar" :age 22} {:name "bazz" :age 12} {:name "foo2" :age 22}]}}}})

Expected: {:foo {:house {:room {:cupboards [{:age 22, :name "foo"} {:age 22, :name "bar"} {:age 12, :name "bazz"} {:age 22, :name "foo2"}]}}}}

Actual:   {:foo {:house {:room {:cupboards [{:age 22, :name "foo"} {:age 22, :name "bar"} {:age 11, :name "bazz"} {:age 22, :name "foo2"}]}}}}

    Delta:                                                                                     --------                 

Vertical presentation

------------------------

(fact {:foo {:house {:room {:cupboards [{:name "foo" :age 22} {:name "bar" :age 22} {:name "bazz" :age 11} {:name "foo2" :age 22}]}}}}

=> {:foo {:house {:room {:cupboards [{:name "foo" :age 22} {:name "bar" :age 22} {:name "bazz" :age 12} {:name "foo2" :age 22}]}}}})

Expected: Actual:

{:foo {:foo

{:house {:house

 {:room {:room

  {:cupboards [  {:cupboards [

   {:age 22, :name "foo"}   {:age 22, :name "foo"}

{:age 22, :name "bar"}   {:age 22, :name "bar"}

-> {:age 11, :name "bazz"}   {:age 12, :name "bazz"}

{:age 22, :name "foo2"}]}}}}   {:age 22, :name "foo2"}]}}}}

Best Regards Sebastian

[Roelof Wobben]

What do you think about the way groovy is doing things. http://beta.groovy-lang.org/testing.html

Roelof

Op maandag 3 november 2014 12:36:22 UTC+1 schreef Sebastian Hennebrueder:

[Alf Kristian Støyle]

Hi!

Would recommend checking out flare (https://github.com/andersfurseth/flare). I've been using it for quite some time, and it is very useful. Added it to my profiles.clj.

Works with Midje and clojure.test. It tries to give better/additional diffs for all data structures. I don't think I am doing it justice, but here are couple of examples.

Ex 1:

{:k1 1 :k2 2} => {:k1 2 :k2 3}

FAIL "..."
    Expected: {:k1 2, :k2 3}
      Actual: {:k1 1, :k2 2}

in [:k1] expected: 2, was 1
in [:k2] expected: 3, was 2


Ex 2:
      {:k1 1 :k2 [1 2 3]} => {:k1 1 :k2 [1 2 {:k 1}]}

FAIL "..."
    Expected: {:k1 1, :k2 [1 2 {:k 1}]}
      Actual: {:k1 1, :k2 [1 2 3]}

in [:k2 2] expected: {:k 1}, was 3


Ex 3 (string diffs adds color coding):
      "abcdefg" => "abc123defg"

FAIL "..."
    Expected: "abc123defg"
      Actual: "abcdefg"

string differ: abc123defg

Ex 4
[1 2 3 4 5 6 7 8 10] => [1 2 3 4 5 6 7 10]

FAIL "..."
    Expected: [1 2 3 4 5 6 7 10]
      Actual: [1 2 3 4 5 6 7 8 10]

in [7] expected: 10, was 8

Cheers,
Alf

[Alf Kristian Støyle]

Would recommend checking out flare (https://github.com/andersfurseth/flare). I've been using it for quite some time, and it is very useful. Added it to my profiles.clj.

Works with midje and clojure.test. It tries to give better/additional diffs for all data structures. I don't think I am doing it justice, but here are couple of examples.