Midje tests run, but failed assertions are ignored

[Stefan Holzmüller]

Hey guys,

I started using clojure-maven-plugin yesterday and happily ran some clojure.test tests. Now I switched to the Midje test framework and realized that those tests are executed, but failures and successes are not handled. I'm attaching a simple test project - just run "mvn test" to reproduce.

Maybe this is a known problem? Maybe a custom "test launcher script" is needed?

Your insight is very welcome.

Cheers,

Stefan

[Mark Derricutt]

Hi Stefan,

Sorry I've not replied sooner - I just spent a few minutes looking at
your demo code and now remember what the problem is. It's been reported
before but I'm not sure the best way of handling it.

The "problem" is that you're not _actually_ running midge tests as part
of "mvn test" but rather they're being "evaled" during the "mvn
testCompile" phase.

Because Clojure is inherently a REPL/Reader based language there isn't
really the notion of a strictly "compile" phase - the only different
between "compile" and "run" is that compile outputs the JVM .class files
to disk.

When you write "(fact....)" in midge, you're not declaring a fact to run
at a later time as one does with "(deftest...)" but rather you're
evaluating the fact immediately during the "compilation", my guess is
that because midge has no separate run step, its not triggering any form
of VM termination/failure.

One solution I've seen people do in the past is wrap their (fact...)
expressions INSIDE a (deftest...) - but that's rather unpleasant.

If anyone can offer a good solution to this I'm open-ears cause I'm
kinda stumped for a decent solution...

Mark

[Stefan Holzmüller]

Thanks Mark,

for looking into this issue. Wrapping the (fact) in a (deftest) seems to work - at least there is a workaround. But as you said, that solution is far from perfect.

Please correct me if I'm wrong: the compiled .class files only contain junk, because they are just the result of "executing" the Midje tests? Provided that I don't mix Midje and clojure.test tests, could I configure clojure-maven-plugin to skip the compile phase of test namespaces, and compile them in the test phase instead? It's just a thought, sort of brainstorming. But could test failures easily be picked up then?

I'll probably try something on the weekend, but honestly, I'm not too optimistic. If there is no clever solution, I'll have to live with crude workarounds. Too bad.

Stefan

PS: I really like clojure-maven-plugin. Java/Clojure interoperability was working "out of the box" and I guess I just feel more comfortable using Maven (instead of Leiningen).

[Stefan Holzmüller]

Just a small update: I couldn't think of anything better than the "wrapping in (deftest)" workaround. I even wrote a macro in order to do the wrapping for me. That'll do for now.

Thanks again for your explanation, Mark.