Testing Private Functions

[Johnny Kwan]

Hi All,

I'm trying to test some private functions using the Clojure convention of placing tests in a separate test- namespace.  Of course, you can't reference private vars from other namespaces (please correct me if I'm wrong).  Not knowing anything at all about how stuff is interned in Clojure, I found this blog post:  http://formpluslogic.blogspot.com/2009/08/clojure-unit-testing-part-1.html.  Using ns-resolve works.

I'm trying to generalize it into a macro named use-private

(defmacro

  #^{:private true}

  use-private

  [ns sym]

  `(def

     #^{:private true}

     ~sym (ns-resolve ~ns ~sym)))

This, however, doesn't work.  The first ~sym macroexpands to (quote sym), which cannot be passed as a name to def.  An exception is thrown saying that the first argument must be a symbol, and this is because, I assume, that the quote form is a form.  So...

1. Is there a better way the community has found to test private functions?
2. Is there a way to reference private functions from a test- namespace?  Is there some proposal floating around to allow this?
3. Is there a better approach than my ghetto macro, outside of extending the core language and libraries?
4. In my ghetto macro, how do I get the sym arg to macroexpand to just the sym and not a quote form?

Thanks,

Johnny

[Meikel Brandmeyer]

Hi,

On Feb 22, 5:40 am, Johnny Kwan <johnny.c.k...@gmail.com> wrote:

> I'm trying to generalize it into a macro named use-private
>
> (defmacro
>   #^{:private true}
>   use-private
>   [ns sym]
>   `(def
>      #^{:private true}
>      ~sym (ns-resolve ~ns ~sym)))

First some fixes:

(defmacro #^{:private true} use-private
[ns sym]

`(def ~(with-meta sym {:private true}) @(ns-resolve '~ns '~sym)))

user=> (use-private clojure.core spread)
#'user/spread
user=> (type spread)
clojure.core$spread__3429
user=> (macroexpand-1 '(use-private clojure.core spread))
(def spread (clojure.core/deref (clojure.core/ns-resolve (quote
clojure.core) (q
uote spread))))

> This, however, doesn't work.  The first ~sym macroexpands to (quote sym), which cannot be passed as a name to def.  An exception is thrown saying that the first argument must be a symbol, and this is because, I assume, that the quote form is a form.  So...

This only expands to (quote sym) because you put in (quote sym). You
haven't shown the invocation of your macro, but that's most likely the
cause.

> Is there a better way the community has found to test private functions?
> Is there a way to reference private functions from a test- namespace?  Is there some proposal floating around to allow this?
> Is there a better approach than my ghetto macro, outside of extending the core language and libraries?
> In my ghetto macro, how do I get the sym arg to macroexpand to just the sym and not a quote form?

Use @#'foo.bar/private-fn. Or...

My approach to this is to get rid of private function entirely.
Everything is public. Put "private" functions of the namespace foo.bar
in the namespace foo.bar.internal. Then simply :use the internal
namespace in foo.bar. Make it clear that users using foo.bar.internal
have to face eternal doom in the hottest corners of hell.

This makes some things really easy: testing "private" functions is
suddenly not a problem anymore. You don't have shout (@#') at private
functions in macros.

Sincerely
Meikel

[Johnny Kwan]

Hi,

On Feb 22, 2010, at 2:20 AM, Meikel Brandmeyer wrote:

> First some fixes:
>
> (defmacro #^{:private true} use-private
> [ns sym]
> `(def ~(with-meta sym {:private true}) @(ns-resolve '~ns '~sym)))

Awesome!

>> This, however, doesn't work. The first ~sym macroexpands to (quote sym), which cannot be passed as a name to def. An exception is thrown saying that the first argument must be a symbol, and this is because, I assume, that the quote form is a form. So...
>
> This only expands to (quote sym) because you put in (quote sym). You
> haven't shown the invocation of your macro, but that's most likely the
> cause.

You're absolutely right. I use 'ns and 'sym because that's what all of the ns- functions use. I realize now that this is a macro, not a fn... :) I would like to make the fact that this is a macro transparent, so that it only accepts symbol arguments. The caller shouldn't have to know that this is a macro.

>> Is there a better way the community has found to test private functions?
>> Is there a way to reference private functions from a test- namespace? Is there some proposal floating around to allow this?
>> Is there a better approach than my ghetto macro, outside of extending the core language and libraries?
>> In my ghetto macro, how do I get the sym arg to macroexpand to just the sym and not a quote form?
>
> Use @#'foo.bar/private-fn. Or...

Thanks!

> My approach to this is to get rid of private function entirely.
> Everything is public. Put "private" functions of the namespace foo.bar
> in the namespace foo.bar.internal. Then simply :use the internal
> namespace in foo.bar. Make it clear that users using foo.bar.internal
> have to face eternal doom in the hottest corners of hell.

That's what I figured, since the few library implementations I've read seem to do that. I assume there's a way to remove these namespaces from generated docs, so there's at least some obfuscation?

Thanks,
Johnny

[Meikel Brandmeyer]

Hi,

On Feb 22, 11:55 am, Johnny Kwan <johnny.c.k...@gmail.com> wrote:

> You're absolutely right.  I use 'ns and 'sym because that's what all of the ns- functions use.  I realize now that this is a macro, not a fn...  :)  I would like to make the fact that this is a macro transparent, so that it only accepts symbol arguments.  The caller shouldn't have to know that this is a macro.

Here we go:

(defmacro #^{:private true} use-private
[ns sym]

`(def ~(with-meta (second sym) {:private true}) @(ns-resolve ~ns
~sym)))

Not tested, though, but should come close.

> > Use @#'foo.bar/private-fn. Or...
>
> Thanks!

This feels a little hacky, however.

> That's what I figured, since the few library implementations I've read seem to do that.  I assume there's a way to remove these namespaces from generated docs, so there's at least some obfuscation?

I'd say, that depends on what you use (and how you use it) to generate
the documentation.

Sincerely
Meikel

[Alan Dipert]

Hi, This is a problem I was working on this morning (with the help of
chouser, thank you!), and ended up with a 'with-private-fns' macro that
works like this:

(with-private-fns [org.foo.bar [fn1 fn2]]
(deftest test-fn1..)
(deftest test-fn2..))

I'm a Clojure greenhorn, but I like this approach because my private fn
tests are consolidated. I think in the end thought it boils down to a
matter of personal style :)

The macro itself looks like this:

(defmacro with-private-fns [[ns fns] & tests]
"Refers private fns from ns and runs tests in context."
`(let ~(reduce #(conj %1 %2 `(ns-resolve '~ns '~%2)) [] fns)
~@tests))

A test file making use of it can be found here:
http://github.com/alandipert/clj-utils/blob/master/test/org/dipert/utils/test/math.clj

Cheers,
Alan

Excerpts from Meikel Brandmeyer's message of 2010-02-22 07:08:59 -0500:

[Meikel Brandmeyer]

Hi,

On Mon, Feb 22, 2010 at 02:04:37PM -0500, Alan Dipert wrote:

> (defmacro with-private-fns [[ns fns] & tests]
> "Refers private fns from ns and runs tests in context."
> `(let ~(reduce #(conj %1 %2 `(ns-resolve '~ns '~%2)) [] fns)
> ~@tests))

Without a deref around the ns-resolve, this stores Vars in the locals.
For functions this is not a problem, because Vars relay invoke to their
contents. However if you have different objects (eg. a map to be used
with get) you will run into trouble.

(defmacro with-private-vars [[ns fns] & tests]

"Refers private fns from ns and runs tests in context."

`(let ~(reduce #(conj %1 %2 `@(ns-resolve '~ns '~%2)) [] fns)
~@tests))

Note the @ in front of ns-resolve. Here some illustration of the
problem:

user=> (ns foo.bar)
nil
foo.bar=> (def #^{:private true} x {:a :b :c :d})
#'foo.bar/x
foo.bar=> (in-ns 'user)
#<Namespace user>
user=> (with-private-fns [foo.bar [x]] (x :a))
:b
user=> (with-private-fns [foo.bar [x]] (get x :a))
nil
user=> (with-private-vars [foo.bar [x]] (get x :a))
:b

Sincerely
Meikel

[Alan Dipert]

Hi Meikel, thank you as always for your help! This makes a lot of
sense. Am I correct in understanding that at some level there is a
separation of var and fn namespace? Or is the need for deref more just
a consequence of the way objects respond with their contents? Please forgive
my ignorance, and thanks in advance for clarification.

Best,
Alan

Excerpts from Meikel Brandmeyer's message of 2010-02-22 15:23:04 -0500:

[Meikel Brandmeyer]

Hi,

On Mon, Feb 22, 2010 at 05:07:33PM -0500, Alan Dipert wrote:
> Hi Meikel, thank you as always for your help!

You are welcome. :)

> This makes a lot of sense. Am I correct in understanding that at some
> level there is a separation of var and fn namespace? Or is the need
> for deref more just a consequence of the way objects respond with
> their contents?

No. There is no separation of var and fn namespace. Clojure is a Lisp-1
(for some suitable definition of Lisp-1). Clojure stores all global
bindings in Vars. Be it a function, a number, a map, whatever. Now the
need for deref comes from the fact, that ns-resolve returns the Var
itself and not its contents. These are retrieved with the deref.

Sincerely
Meikel

[Alan Dipert]

I get it, thank you!
Alan

Excerpts from Meikel Brandmeyer's message of 2010-02-22 17:22:19 -0500: