Clojure testing standards: What is the desired approach?

[Scott Lowe]

Hello,

I have a question regarding what is typically expected of unit tests in the Clojure codebase, with specific regard to test/is function calls and failures.

Clearly, when making changes or additions to a codebase, it's a good thing to make new code mesh with the style and idioms of the existing code, and I've tried to do this; however, having examined the Clojure codebase, I have found it hard to discern if failing tests should cause build failure (or not).

For example:

In "test/clojure/test_clojure/reader.clj" many of the test assertions are wrapped in a (testing …) function which will cause the build to fail upon test failure.

The pattern in test/clojure/test_clojure/data_structures.clj is very different – long lists of test/is calls, but nothing is wrapped in (testing …) functions. Consequently, failing calls to test/is simply outputs an error (no proper failures), the Clojure build completes and it spits out a jar.

Personally, I don't think it should be possible to build a package when tests are failing, but there may be reasons for taking the opposite approach. Please comment on the desired approach, so that the next time I write a test, I can "do the right thing".

If there is a preferred testing standard, then perhaps it should be added to the Library Coding Standards section at http://dev.clojure.org/display/design/Library+Coding+Standards.

Thanks,

Scott

[Tassilo Horn]

Scott Lowe <scottwit...@gmail.com> writes:

Hi Scott,

> In "test/clojure/test_clojure/reader.clj" many of the test assertions
> are wrapped in a (testing …) function which will cause the build to
> fail upon test failure.
>
> The pattern in test/clojure/test_clojure/data_structures.clj is very
> different – long lists of test/is calls, but nothing is wrapped in
> (testing …) functions. Consequently, failing calls to test/is simply
> outputs an error (no proper failures), the Clojure build completes and
> it spits out a jar.

Huh? The `testing` macro (not a function) is only meant for structuring
larger `deftests` to give nicer failure messages. It should have no
influence on the build process at all.

> Personally, I don't think it should be possible to build a package
> when tests are failing, but there may be reasons for taking the
> opposite approach.

I changed some assertions in a `testing` form one time in
"test/clojure/test_clojure/reader.clj" and another time in
"test/clojure/test_clojure/data_structures.clj" without being wrapped in
`testing`, and both give a failure and make the build fail.

Bye,
Tassilo

[Scott Lowe]

Hi Tassilo,

Well, this is embarrassing: You're absolutely right.

I'm not sure what just happened, but whatever it was, I must have been doing something wrong, because now, everything fails as expected. I can't explain this!

I hope readers of this list will accept my humble apologies for the unnecessary distraction.

Regards,

Scott

On Thursday, May 31, 2012 1:55:26 PM UTC+1, Tassilo Horn wrote:

Scott Lowe  writes:

[Stuart Sierra]

I wrote clojure.test, and I put `testing` in for the RSpec fans. It's just an aid to documentation; it has no effect on the tests themselves. The top-level form of a test must always be `deftest`, not `testing`.

There is no "preferred" style, just the usual testing recommendations: keep tests fairly small, independent, etc.
-S