Midje wants *you*
Brian Marick
When I decided I couldn't stand Motorola any more, my business plan was:
1. trade consulting to Motorola in exchange for rights to my GCC-based C coverage tool.
2. Open-source it.
3. Charge users for enhancements.
4. Profit!
In my first year, I made USD 500.
My newly-acquired mother-in-law was concerned.
Stumbling over a training opportunity led to a new plan, and I've survived (often thrived) as an independent for 20 years. However, let's just say my marketing chops are still not great.
I'm proud of the work I've done on Midje. I think it has some potential to push the boundaries of TDD. To do so -- to justify pouring effort into it -- I think it needs to become something close to the default Clojure coverage tool, akin to Rspec in Ruby. Because of my poor marketing chops, I doubt I can do that alone. At this moment, the 33 of you hold Midje's future in your hands.
1. Please download the 1.0 release and let me know that it's OK for you.
2. If you use Midje, share that.
-----
Brian Marick, Artisanal Labrador
Contract programming in Ruby and Clojure
Author of /Ring/ (forthcoming; sample: http://bit.ly/hfdf9T)
www.exampler.com, www.exampler.com/blog, www.twitter.com/marick
Ben Mabey
> 1. Please download the 1.0 release and let me know that it's OK for you.
edition. (Oh, which BTW I'd like to get my midje stickers. ;) ) This is
the failure:
FAIL at (core.clj:36)
Expected: #:com.leadtune.predict.models.weka.WekaNominalModel{:kind
:bayes, :algorithm :naive, :model-options {:class-attribute :sale,
:nominal-label-to-predict "1"}, :filter-options {}}
Actual: {:kind :bayes, :algorithm :naive, :model-options
{:class-attribute :sale, :nominal-label-to-predict "1"}, :filter-options {}}
The problem is that the record's type (WekaNominalModel) is being
stripped from the "actual" (left hand side) form. I believe the bug
crept in on this line:
https://github.com/marick/Midje/blob/master/src/midje/util/laziness.clj#L21
(You are giving #'into a hashmap where it needs to be the record's type.)
I've been working on a patch. I'll finish it tomorrow and send you a
pull request.
> 2. If you use Midje, share that.
Midje is really a breath of fresh air in the clojure testing world and
have enjoyed using it on my current project. One of my new year's
resolutions is to revive my dead blog. I'll try to work midje into one
of my clojure blog posts to help spread the word.
-Ben
Brian Marick
On Jan 17, 2011, at 11:39 PM, Ben Mabey wrote:
> FAIL at (core.clj:36)
> Expected: #:com.leadtune.predict.models.weka.WekaNominalModel{:kind :bayes, :algorithm :naive, :model-options {:class-attribute :sale, :nominal-label-to-predict "1"}, :filter-options {}}
> Actual: {:kind :bayes, :algorithm :naive, :model-options {:class-attribute :sale, :nominal-label-to-predict "1"}, :filter-options {}}
Sorry about that. There may be some subtleties about records. What'll be the results of these:
(defrecord MyFoo [foo bar])
(def m (MyFoo. 1 3))
(fact
m => (MyFoo. 1 3)
m => {:foo 1 :bar 2}
m => (contains {:foo 1})
m => (just {:foo 1, :bar 2})
m => (just (MyFoo. 1 3))
I could see a rule that you can use maps on the right-hand-side, which means "ignore record-ness". So:
m => {:foo 1 :bar 2} ; pass
m => (contains {:foo 1}) ; pass
m => (just {:foo 1, :bar 2}) ; pass
Then I could see a rule that mentioning the concrete type means that not only the key/value pairs must be the same but the type must also be the same.
{:foo 1, :bar 3} => (MyFoo. 1 3) ; false
That's easy, because it's what equality does now.
I think the rule could be implemented for this:
m => (just (MyFoo. 1 3))
... without too much trouble. The same thing works for contains, which sort of makes sense:
(assoc m :quux 33) => (contains (MyFoo. 1 3))
Better rules? Simpler rules? We could just say that maps and records are indistinguishable to Midje, so that
{:foo 1, :bar 3} => (MyFoo. 1 3) ; is true
Only advantage, I guess, is less work.
-----
Brian Marick, Artisanal Labrador
Contract programming in Ruby and Clojure
Author of /Ring/ (forthcoming; sample: http://exampler.com/tmp/ring.pdf)
www.exampler.com, www.exampler.com/blog, www.twitter.com/marick
Brian Marick
On Jan 17, 2011, at 11:39 PM, Ben Mabey wrote:
> Expected: #:com.leadtune.predict.models.weka.WekaNominalModel{:kind :bayes, :algorithm :naive, :model-options {:class-attribute :sale, :nominal-label-to-predict "1"}, :filter-options {}}
> Actual: {:kind :bayes, :algorithm :naive, :model-options {:class-attribute :sale, :nominal-label-to-predict "1"}, :filter-options {}}
[More discussion in a previous note, but by the way, this is a hacky workaround if you need it:
(defrecord MyFoo [foo bar])
(def m (MyFoo. 1 3))
(fact
m => (just (MyFoo. 1 3)))
That is, the #'just turns the MyFoo into a plain map, so it matches the plain map that forcing out laziness produces.]
-----
Brian Marick, Artisanal Labrador
Contract programming in Ruby and Clojure
Author of /Ring/ (forthcoming; sample: http://exampler.com/tmp/ring.pdf)
www.exampler.com, www.exampler.com/blog, www.twitter.com/marick
Ben Mabey
Sorry about that. There may be some subtleties about records. What'll be the results of these: (defrecord MyFoo [foo bar]) (def m (MyFoo. 1 3)) (fact m => (MyFoo. 1 3) m => {:foo 1 :bar 2} m => (contains {:foo 1}) m => (just {:foo 1, :bar 2}) m => (just (MyFoo. 1 3)) I could see a rule that you can use maps on the right-hand-side, which means "ignore record-ness". So: m => {:foo 1 :bar 2} ; pass m => (contains {:foo 1}) ; pass m => (just {:foo 1, :bar 2}) ; pass
Then I could see a rule that mentioning the concrete type means that not only the key/value pairs must be the same but the type must also be the same.
{:foo 1, :bar 3} => (MyFoo. 1 3) ; false
That's easy, because it's what equality does now.
I think the rule could be implemented for this:
m => (just (MyFoo. 1 3))
... without too much trouble. The same thing works for contains, which sort of makes sense:
(assoc m :quux 33) => (contains (MyFoo. 1 3))
Better rules? Simpler rules? We could just say that maps and records are indistinguishable to Midje, so that
{:foo 1, :bar 3} => (MyFoo. 1 3) ; is true
Only advantage, I guess, is less work.
I started doing a patch, but the issue is kinda messy. Again, the problem line is:
(m (into form (map eagerly form))
We want to preserve the record type and since #'into will force the realization I didn't think this should be an issue. However, when I made this change it somehow broke the eagerness of it as this test broke:
(= (type eagered) clojure.lang.LazySeq) => falsey
https://github.com/marick/Midje/blob/master/test/midje/util/t_laziness.clj#L36
I am confused why this broke, especially since we aren't even
using a map in this test (just a vector of ints).. So I didn't
know how the above line was even being executed by that test.
However, even with an empty-record function this seems problematic. How do we know that the form is a record and not some other class that implements IPersistentMap? The eagerly function is already doing it's fair share of conditional logic based on the type. It seems like a polymorphic solution would be better. (i.e. implement empty-record on the known IPersistentMap types and have an Object fallback implementation that uses the methods proposed in the SO question for actual records). This seems very complex too for such a simple problem though. Does anyone have any simpler ideas on how to fix the problem?
-Ben
Brian Marick
On Jan 18, 2011, at 5:37 PM, Ben Mabey wrote:
> (map? form)
> (m (into (if (sorted? form) (sorted-map) {}) (map eagerly form)))
>
> Now, my first thought was to change it to:
>
> (m (into form (map eagerly form))
>
It's the #'into.
Here's the object we're working on:
user> (def mmm { (map identity [1 2 3]) 'foo })
Here are some print statements:
(map? form)
(let [sub-result (map eagerly form)]
(prn "RESULT OF MAPPING OF EAGERLY" sub-result)
(prn "KEY:" (type (ffirst sub-result)) (ffirst sub-result))
(let [result (m (into form sub-result))]
(prn "RESULT of INTO:" result)
(prn "KEY:" (type (ffirst result)) (ffirst result))
result))
Here's what they print:
user> (def result (midje.util.laziness/eagerly mmm))
"RESULT OF MAPPING OF EAGERLY" ([(1 2 3) foo])
"KEY:" clojure.lang.PersistentList (1 2 3)
"RESULT of INTO:" {(1 2 3) foo}
"KEY:" clojure.lang.LazySeq (1 2 3)
#'user/result
HOWEVER, this doesn't matter. The purpose of all this is to make sure that every LazySeq is evaluated given within the context of any mocks. If those results get put back into a LazySeq, that doesn't matter.
SO: if this solution lets equality work for your defrecord, I think we're OK. The tests of eagerly will need to be more indirect, though.
Ben Mabey
https://github.com/bmabey/Midje/commit/9feb25281725affb96544eb8980cb9c9bbf2cf4a
The eagerly tests will have to be rewritten, as you say. I don't know
how one would verify that an entire lazy seq has been realized, but it
seems like that is what the assertion should be doing.
Lyle Johnson
> I'm proud of the work I've done on Midje. I think it has some potential to push the boundaries of TDD.
> To do so -- to justify pouring effort into it -- I think it needs to become something close to the default
> Clojure coverage tool, akin to Rspec in Ruby. Because of my poor marketing chops, I doubt I can do that alone.
> At this moment, the 33 of you hold Midje's future in your hands.
>
> 1. Please download the 1.0 release and let me know that it's OK for you.
> 2. If you use Midje, share that.
do like it so far. By an astonishing coincidence, however, I ran into
the same problem that Ben's reported here, on my very first Midje
test. ;) But I'm OK with using one of the workarounds discussed above
until you decide how you'd like to handle comparing records.
Ilmari Vacklin
1. Please download the 1.0 release and let me know that it's OK for you.
2. If you use Midje, share that.
Brian Marick
On Feb 2, 2011, at 2:25 PM, Ilmari Vacklin wrote:
> - On failing facts, the "lein midje" output does not tell you which fact failed, only line numbers. It could print the description of the fact if one was given, or even the thunk being tested.
Do other people find this a problem? I have a (probably) unusual workflow in that I have Emacs code to insert error messages into the code text and other code that jumps quickly from a line of output to the source. So I don't find more output particularly helpful.
> - (fact '() => nil) is a failure. This confused us more than the students.
nil and the empty list are different objects in Clojure, even though it's hard to make them behave differently. But see (= nil '()) or (if '() "emptiness is true" "emptiness is false"). "Is nil the empty list?" was still a big topic during my Lisp days: heated debate over whether Common Lisp or Scheme got it right. That given, making a different choice than the language designer did seems likely to get me in trouble.
> - Sets do not match other collections, e.g. (fact [1 2 3] => #{1 2 3}) fails. We expected this to mean the same as (fact [1 2 3] => (just [1 2 3] :in-any-order)).
I don't know. I originally wanted to have => mean either clojure equality or application of a checker function. But then I added special treatment for regular expressions when the actual value is a string, and then special treatment for maps when the actual value is a record, so I've gone too far down the slippery slope to have any principled argument against this.
It somehow seems more wrong to say an unordered thing "counts as the same as" an ordered thing. To which you might ask: why did I make (fact [1 2 3] => (just #{1 2 3})), then? And I have to say that seems pretty dubious to me now, too.
Again: how do other people feel?
Ben Mabey
>> - Sets do not match other collections, e.g. (fact [1 2 3] => #{1 2 3}) fails. We expected this to mean the same as (fact [1 2 3] => (just [1 2 3] :in-any-order)).
> I don't know. I originally wanted to have => mean either clojure equality or application of a checker function. But then I added special treatment for regular expressions when the actual value is a string, and then special treatment for maps when the actual value is a record, so I've gone too far down the slippery slope to have any principled argument against this.
>
> It somehow seems more wrong to say an unordered thing "counts as the same as" an ordered thing. To which you might ask: why did I make (fact [1 2 3] => (just #{1 2 3})), then? And I have to say that seems pretty dubious to me now, too.
>
> Again: how do other people feel?
As we all know, Clojure has a very strict (and opinionated) view of
equality. I don't think we should vary much from that. I would expect
that [1 2 3] => #{1 2 3} to fail since (= [1 2 3] #{1 2 3}) returns
false. What is interesting is that (= [1 2 3] (list 1 2 3)) returns
true even though the data structure is different and you might care
about that. In my experience, when I return a set I am doing so very
intentionally and with good reason. So I would be in favor of keeping
the current strict behavior.
I'm conflicted as well though. I like how regexps work currently, and I
see how this could be seen as a similar issue.
Ilmari Vacklin
On Feb 2, 2011, at 2:25 PM, Ilmari Vacklin wrote:
> - On failing facts, the "lein midje" output does not tell you which fact failed, only line numbers. It could print the description of the fact if one was given, or even the thunk being tested.Do other people find this a problem? I have a (probably) unusual workflow in that I have Emacs code to insert error messages into the code text and other code that jumps quickly from a line of output to the source. So I don't find more output particularly helpful.
> - Sets do not match other collections, e.g. (fact [1 2 3] => #{1 2 3}) fails. We expected this to mean the same as (fact [1 2 3] => (just [1 2 3] :in-any-order)).
I don't know. I originally wanted to have => mean either clojure equality or application of a checker function. But then I added special treatment for regular expressions when the actual value is a string, and then special treatment for maps when the actual value is a record, so I've gone too far down the slippery slope to have any principled argument against this.
It somehow seems more wrong to say an unordered thing "counts as the same as" an ordered thing. To which you might ask: why did I make (fact [1 2 3] => (just #{1 2 3})), then? And I have to say that seems pretty dubious to me now, too.
(facts "powerset"
(powerset []) => '(())
(powerset [1 2 4]) => (just
[empty?
[4] [2] [1]
(just [2 4] :in-any-order)
(just [1 4] :in-any-order)
(just [1 2] :in-any-order)
(just [1 2 4] :in-any-order)]
:in-any-order))
A shorter way to write this would be nice.
Brian Marick
On Feb 5, 2011, at 12:59 PM, Ilmari Vacklin wrote:
> The original use case was a fact for the powerset function the students were asked to implement. Now the fact looks like this:
>
> (facts "powerset"
> (powerset []) => '(())
> (powerset [1 2 4]) => (just
> [empty?
> [4] [2] [1]
> (just [2 4] :in-any-order)
> (just [1 4] :in-any-order)
> (just [1 2] :in-any-order)
> (just [1 2 4] :in-any-order)]
> :in-any-order))
>
> A shorter way to write this would be nice.
You could do this:
(defn as-sets [& expected]
(let [set-of-sets #(set (map set %))]
(fn [actual] (= (set-of-sets expected) (set-of-sets actual)))))
user> (fact (powerset [1 2 4]) => (as-sets [ ] [4] [2] [1] [2 4] [1 2 4]))
FAIL at (NO_SOURCE_FILE:1)
Actual result did not agree with the checking function.
Actual result: [[4] [2] [1] [2 4] [1 4] [1 2] [1 2 4] []]
Checking function: (as-sets [] [4] [2] [1] [2 4] [1 2 4])
false
I thought I'd get a more detailed failure from this:
(defn as-sets [& expected]
(let [set-of-sets #(set (map set %))]
(fn [actual]
( (just (set-of-sets expected)) (set-of-sets actual)))))
... but I don't. I'm guessing that's a bug.