Confusion about binding *ns* before defining a var

[ian.tegebo]

I went to write a context macro so that I could define functions in another namespace.  After the obligatory googling, I found "with-ns":

http://richhickey.github.io/clojure-contrib/with-ns-api.html

Clicking through to look at the source, I was surprised to see that "eval" was being wrapped around each form in the body of "binding".  I thought, "surely forms within the body of a binding expression are evaluated in a context of the bindings?!":

user> (ns blah)

nil

blah> (in-ns 'user)

#<Namespace user>

user> (binding [*ns* (the-ns 'blah)] *ns*)

#<Namespace blah>

Okay, no surprise there.  Yet, to my surprise:

user> (binding [*ns* (the-ns 'blah)] (defn foo []))

#'user/foo

user> (binding [*ns* (the-ns 'blah)] (eval '(defn foo [])))

#'blah/foo

The extra eval is necessary after all.  But wait, what does the one without eval expand into?

user> (clojure.walk/macroexpand-all '(binding [*ns* (the-ns 'blah)] (defn foo [])))

(let*

 []

 (clojure.core/push-thread-bindings

  (clojure.core/hash-map #'*ns* (the-ns 'blah)))

 (try

  (def foo (fn* ([])))

  (finally (clojure.core/pop-thread-bindings))))

Huh.  Alright, let's remind ourselves of what "def" means:

user> (doc def)

-------------------------

def

  (def symbol doc-string? init?)

Special Form

  Creates and interns a global var with the name

  of symbol in the current namespace (*ns*) or locates such a var if

  it already exists.  If init is supplied, it is evaluated, and the

  root binding of the var is set to the resulting value.  If init is

  not supplied, the root binding of the var is unaffected.

  Please see http://clojure.org/special_forms#def

Now my confusion: isn't binding *ns* exactly how one sets the "current namespace"?  It seems like "def" is not behaving as advertised.  I looked briefly at jvm.clojure.lang.Compiler.DefExpr, but I wasn't able to figure out why the symbol and namespace resolution going on in that class didn't consult thread-bindings (by design?).

Assuming this isn't a bug, what's the rationale for the current behavior?

[Stephen Gilardi]

On May 29, 2014, at 7:11 PM, ian.tegebo <ian.t...@gmail.com> wrote:

user> (binding [*ns* (the-ns 'blah)] (defn foo []))

#'user/foo

user> (binding [*ns* (the-ns 'blah)] (eval '(defn foo [])))

#'blah/foo

clojure.core/eval evaluates a form by compiling it and then executing the compiled code. For a def form, it’s the ns that is current when the form is compiled that determines in which namespace the resulting var is created.

In the first case, the defn form is compiled before the binding to (the-ns ‘blah) is in effect.

In the second case, the defn form is quoted and remains unevaluated while the binding form is compiled. While executing the compiled code for the binding form, eval compiles the defn form (and then executes its compiled code). In this case, the defn is compiled after the binding to (the-ns ‘blah) is in effect.

—Steve

[ian.tegebo]

On Thursday, May 29, 2014 8:43:58 PM UTC-7, squeegee wrote:

On May 29, 2014, at 7:11 PM, ian.tegebo <ian.t...@gmail.com> wrote:

user> (binding [*ns* (the-ns 'blah)] (defn foo []))

#'user/foo

user> (binding [*ns* (the-ns 'blah)] (eval '(defn foo [])))

#'blah/foo

In the first case, the defn form is compiled before the binding to (the-ns ‘blah) is in effect.

It's exactly this point that's confusing.

The defn form expands into a def, whose documentation says it uses the current namespace.  The current namespace, as I understand it, is what's bound to *ns* which IIUC should have been accomplished with the "binding" macro.  The first code snippet was meant to demonstrate that.  Here's another variant in case there's confusing about *ns* being lexical:

user> (defn println-ns [] *ns*)

#'user/println-ns

user> (binding [*ns* (the-ns 'blah)] (println-ns))

#<Namespace blah>

I don't see the reason why def should behave as it currently does; it seems like it should lookup the current thread-binding for *ns*, making the second case's use of eval unnecessary.  Since it doesn't, I'd like to know why it couldn't (or shouldn't) do the thing that seems more intuitive.

That said, if the special form let* does not evaluate its arguments in left to right order such that let* actually evaluates def's before anything else, then I would understand why the current behavior is the way it is.  However, then I'd just be curious about let*...

 

[Luc Prefontaine]

*ns* is bound at execution time,
not at compile time.

Eval here postpones the definition at
runtime after the bindings have been
evaluated and *ns* gets rebinded.

Otherwise the definition would be
created at compile time in the
namespace of the caller.

The full form has to be compiled to
get the new bindings evaled.

Leaving def unwrapped would get it
done at compilation time,
not at runtime.

Is this a bit clearer ?

Luc P.

>
> On Thursday, May 29, 2014 8:43:58 PM UTC-7, squeegee wrote:
> >
> >

> > On May 29, 2014, at 7:11 PM, ian.tegebo <ian.t...@gmail.com <javascript:>>

> > wrote:
> >
> > user> (binding [*ns* (the-ns 'blah)] (defn foo []))
> > #'user/foo
> > user> (binding [*ns* (the-ns 'blah)] (eval '(defn foo [])))
> > #'blah/foo
> >
> > In the first case, the defn form is compiled before the binding to (the-ns
> > ‘blah) is in effect.
> >
>
> It's exactly this point that's confusing.
>
> The defn form expands into a def, whose documentation says it uses the
> current namespace. The current namespace, as I understand it, is what's
> bound to *ns* which IIUC should have been accomplished with the "binding"
> macro. The first code snippet was meant to demonstrate that. Here's
> another variant in case there's confusing about *ns* being lexical:
>
> user> (defn println-ns [] *ns*)
> #'user/println-ns
> user> (binding [*ns* (the-ns 'blah)] (println-ns))
> #<Namespace blah>
>
> I don't see the reason why def should behave as it currently does; it seems
> like it should lookup the current thread-binding for *ns*, making the
> second case's use of eval unnecessary. Since it doesn't, I'd like to know
> why it couldn't (or shouldn't) do the thing that seems more intuitive.
>
> That said, if the special form let* does not evaluate its arguments in left
> to right order such that let* actually evaluates def's before anything
> else, then I would understand why the current behavior is the way it is

> However, then I'd just be curious about let*...
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
Luc Prefontaine<lprefo...@softaddicts.ca> sent by ibisMail!

[Stephen Gilardi]

On May 30, 2014, at 12:57 AM, ian.tegebo <ian.t...@gmail.com> wrote:

I don't see the reason why def should behave as it currently does; it seems like it should lookup the current thread-binding for *ns*, making the second case's use of eval unnecessary.  Since it doesn't, I'd like to know why it couldn't (or shouldn't) do the thing that seems more intuitive.

One reason is performance.

The compile-time resolution of:

  - symbols into fully qualified symbols, and then

  - fully qualified symbols into direct references to the vars they represent

is important in allowing Clojure code to execute as fast as it does. These are relatively time consuming operations. Deferring them to execution time would make Clojure code execution slower to an unacceptable degree. Instead, they are done once when the code is compiled and in the general case executed many times without further lookups.

In your particular example, the code is compiled once, executed, and then not used again so the performance distinction doesn’t matter. However, the repl and eval don’t get any special treatment from the compiler, so you see the same behavior using them as you do when you load a library full of code using :require in an ns form.

Another design choice for the Clojure compiler that impacts your example is that each top-level form is completely compiled before any part of it is executed [1]. There’s a nice writeup of another similar implication of that here: http://technomancy.us/143 .

The distinction between compile time and execution time for Clojure code is something that rarely has an impact on understanding the behavior of the code. You’ve found a case where the distinction does matter. The doc for def talks about the “current” namespace without giving a detailed description of what instant of time “current” refers to. It turns out to be “current at the time the code is compiled”, not “current at the time the code is executed”.

In the rare case that the binding of *ns* changes between those times, the behavior can be confusing. One way to avoid this confusion is to keep all defs at the top level and treat *ns* as something that can be set! or manipulated with the associated tools like in-ns, but not bound using bind.

—Steve

[1] Except a top-level “do” in Clojure 1.1+ as described in the blog post

[ian.tegebo]

Thanks Steve!

It's clear to me now that I neither understand Clojure's compilation model, nor its special forms.  I also see that I probably don't want def to respect dynamic scope, but instead would want a compile-time (non top-level) version of in-ns (or an explanation about why that's a bad idea).