What's the best way to test private functions?

[Benjamin Esham]

Hi all,

I am writing a library [1] which has only one function that should be
exposed to users. I'd like to be able to test all of the other functions,
which are marked private with "defn-". Of course, these functions are
inaccessible from the testing namespace (I'm using the testing boilerplate
that Leiningen generates).

[1]: https://github.com/bdesham/clj-schulze

I searched the mailing list archives for an answer to this problem, and I
found one thread [2] which defined a macro "defn*" which would expand to
defn or defn-, depending on whether testing was going on. I also found a
thread [3] in which Stuart Sierra proposed

(defn refer-private [ns]
(doseq [[symbol var] (ns-interns ns)]
(when (:private (meta var))
(intern *ns* symbol var))))

As he says, "this is slightly evil, and I would never recommend it for any
purpose except unit testing, but there it is." This works, and the macro
approach also makes sense to me. But these threads are respectively three
and two years old. Have there been any changes in Clojure since then which
solve this problem or provide a recommended workaround?

[2]: http://groups.google.com/group/clojure/browse_thread/thread/2b00d48f009d4837/631c3edf625bc88b
[3]: http://groups.google.com/group/clojure/browse_thread/thread/3835e5405ab930f6/66f15396411e49e9

(You might tell me that I should only test the user-facing parts of the
library, since everything else is implementation details and these shouldn't
be reflected in tests. I agree in principle, but this library implements an
election method: it takes a bunch of ballots and returns only the name of
the winning candidate. Given how little information comes out of this
function, and the possibility that something could be going horribly wrong
behind the scenes without this being reflected in the final result, I'd
rather be safe than sorry and test all of the library's intermediate steps.)

Thanks for the information!
--
Benjamin D. Esham | bde...@gmail.com | www.bdesham.info
L’APOSTROPHE: Let our battle’s begin!
BOB THE ANGRY FLOWER: God, you even _say_ it wrong!
— http://www.angryflower.com/plural.gif

[Tassilo Horn]

Benjamin Esham <bde...@gmail.com> writes:

Hi Benjamin,

> (defn refer-private [ns]
> (doseq [[symbol var] (ns-interns ns)]
> (when (:private (meta var))
> (intern *ns* symbol var))))
>
> As he says, "this is slightly evil, and I would never recommend it for
> any purpose except unit testing, but there it is." This works, and the
> macro approach also makes sense to me.

I use it, too.

> But these threads are respectively three and two years old. Have there
> been any changes in Clojure since then which solve this problem or
> provide a recommended workaround?

Not that I know of.

> (You might tell me that I should only test the user-facing parts of
> the library, since everything else is implementation details and these
> shouldn't be reflected in tests.

No. Testing also the private functions helps to spot errors and
regressions as soon as possible. You can still split the tests of the
public functions into a separate file and point users to that as usage
example.

Bye,
Tassilo

[Brian Marick]

On Jun 17, 2011, at 1:21 PM, Benjamin Esham wrote:
> I am writing a library [1] which has only one function that should be
> exposed to users. I'd like to be able to test all of the other functions,
> which are marked private with "defn-". Of course, these functions are
> inaccessible from the testing namespace (I'm using the testing boilerplate
> that Leiningen generates).

I put the private functions in their own namespace that's :used by the API namespace.

(ns midje.sweet
(:use midje.midje-forms.recognizing ...

-----------
(ns midje.t-sweet ;;; Corresponding file of tests.
(:use [midje.sweet])
(:use [midje.test-util]))

;;; Since Midje is a testing tool, I'm using it to test itself.
;;; Here I'm testing `tabular`, which is part of the public API
(tabular
(fact (+ ?a ?b) => ?result )
?a ?b ?result
1 2 3)
------------

(ns midje.midje-forms.recognizing .... ;;; Helper functions.

(defn loc-is-at-full-expect-form? [loc]
(and (zip/branch? loc)
(namespacey-match '(expect) (zip/down loc))))

-----------

(ns midje.midje-forms.t-recognizing ... ;;; and their tests

(tabular
(fact "an embedded expect form can be recognized"
(loc-is-at-full-expect-form? (zip/seq-zip ?form)) => ?expected)

?form ?expected
'(expect x => y) truthy
'(midje.semi-sweet/expect x => y) truthy
'(+ x y) falsey
'expect falsey)

-----
Brian Marick, Artisanal Labrador
Contract programming in Ruby and Clojure
Occasional consulting on Agile
www.exampler.com, www.twitter.com/marick

[Jonathan Fischer Friberg]

If you're in a repl*, why not simply use (in-ns <namespace>) ?
Of course, this wont work in all cases (e.g. testing private functions from two different namespaces).
In that case, I find it simpler to remove the '-' temporary. When you're finished, it really doesn't take that much effort to add them again.

Jonathan

* Or anywhere, really

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

[Alex Baranosky]

The best way to test private methods is to have very few of them.  Test the ones you do have via the public API, and if you have too many then IMHO they should be public methods in a separate namespace.  Separate the two namespaces by responsibility.  I do this all the time on OOP languages, and the principle should still apply in Clojure.

[Stuart Halloway]

   (defn refer-private [ns]

     (doseq [[symbol var] (ns-interns ns)]

       (when (:private (meta var))

         (intern *ns* symbol var))))

As he says, "this is slightly evil, and I would never recommend it for

any purpose except unit testing, but there it is." This works, and the

macro approach also makes sense to me.

I use it, too.

But these threads are respectively three and two years old. Have there

been any changes in Clojure since then which solve this problem or

provide a recommended workaround?

Not that I know of.

To access a private var, simply deref through the var:

@#'some-ns/some-private-var

This is in the coding standards doc (http://dev.clojure.org/display/design/Library+Coding+Standards). The doc is pretty short and worth reading if you haven't.

Stu

Stuart Halloway
Clojure/core
http://clojure.com

[Ken Wesson]

On Sat, Jun 18, 2011 at 7:16 AM, Stuart Halloway
<stuart....@gmail.com> wrote:
> To access a private var, simply deref through the var:
> @#'some-ns/some-private-var
> This is in the coding standards doc
> (http://dev.clojure.org/display/design/Library+Coding+Standards). The doc is
> pretty short and worth reading if you haven't.

That's a bit awkward to manually type in e.g. tests.

On the other hand it's very useful if you want a macro to emit a call
to a function but don't feel that function should be public. In that
case it only has to be typed the once, in the macro definition.

--
Protege: What is this seething mass of parentheses?!
Master: Your father's Lisp REPL. This is the language of a true
hacker. Not as clumsy or random as C++; a language for a more
civilized age.

[Stephen C. Gilardi]

On Jun 18, 2011, at 7:16 AM, Stuart Halloway wrote:

To access a private var, simply deref through the var:

@#'some-ns/some-private-var

As Rich noted here: http://groups.google.com/group/clojure/msg/513367afb934d41b , when the var names a function and it's used in an expression emitted from a macro, prefer invoking the var:

(#'some-ns/some-private-var some args)

over invoking the function to which it is currently bound:

(@#'some-ns/some-private-var some args)

Perhaps the coding standards could be updated along these lines:

•To call a private function in another namespace (e.g., for testing), use a form like: (#'some.ns/some-fn some args)

•To access the value bound to a private var in another namespace (e.g., for testing), use an expression like: @#'some.ns/some-var

--Steve

[Ken Wesson]

On Mon, Jun 27, 2011 at 2:13 AM, Stephen C. Gilardi <sque...@mac.com> wrote:
> As Rich noted
> here: http://groups.google.com/group/clojure/msg/513367afb934d41b , when the
> var names a function and it's used in an expression emitted from a
> macro, prefer invoking the var:
> (#'some-ns/some-private-var some args)
> over invoking the function to which it is currently bound:
> (@#'some-ns/some-private-var some args)

I'm not sure of Rich's rationale for this. With

`(do-some-stuff
(~@#'some-ns/some-private-fn some ~@args))

you'd have the problem of embedding the function object in the code,
but not with

`(do-some-stuff
(@#'some-ns/some-private-fn some ~@args))

which will just emit

(do-some-stuff
((deref (var (some-ns/some-private-fn))) some arg1 arg2 arg3))

or the like. As for moving the cost of the deref to macroexpansion
time, I don't think that can be done without embedding the function
object, for obvious reasons.

[Benjamin Esham]

Brian Marick wrote:

> Benjamin Esham wrote:
>
> > I am writing a library [1] which has only one function that should be
> > exposed to users. I'd like to be able to test all of the other
> > functions, which are marked private with "defn-". Of course, these
> > functions are inaccessible from the testing namespace (I'm using the
> > testing boilerplate that Leiningen generates).
>
> I put the private functions in their own namespace that's :used by the API
> namespace.

Good idea. I'll have to think about whether this would be a good choice for
me.

--
Benjamin D. Esham | bde...@gmail.com | www.bdesham.info

“Come to think of it, there are already a million monkeys on a
million typewriters, and Usenet is NOTHING like Shakespeare!”
— Blair Houghton

[Matthew Boston]

A good pattern to follow would be to have your exposed functions in
one clj file and any dependent functions (your private ones in this
case) in a separate ns (i.e. separate clj file). This hides the
implementation to the caller as well as gives them a good idea of the
functions they're able to use.