Better documentation and error messages are needed for the ns macro

[John Harrop]

So I have

(ns foo.bar.baz)

and I want to grab clojure.contrib.core/seqable?

What do I do?

(ns foo.bar.baz

  (use clojure.contrib.core :only seqable?))

#<CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: Boolean (NO_SOURCE_FILE:0)>

(ns foo.bar.baz

  (use [clojure.contrib.core :only seqable?]))

#<CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: Boolean (NO_SOURCE_FILE:0)>

(ns foo.bar.baz

  (:use [clojure.contrib.core :only seqable?]))

#<CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: Symbol (NO_SOURCE_FILE:0)>

etc.

Anyone else noticing a pattern here? a) a need to play a variation of the old command-line-prompt guess-the-synonym game to get it to work, because the documentation for "ns" doesn't give actual examples nor does that for "use" and b) completely uninformative error messages that don't even identify the ns form as the problem, though it is.

Really, to refer to the exact syntax for a use clause in an ns form I shouldn't have to do more than a) go to clojure.org/api, b) go to the entry on "ns", and c) skim. But it takes careful reading and pointer-following (from "ns" to "use" and then "require", then backtrack, then "refer") instead of a quick and easy skim, largely owing to the lack of actual examples of usage in the docs. Which are unnecessary in most places, but the "ns" macro is an exception since it has its own mini-DSL inside it. (The for/doseq/doall triad is the other major culprit in that regard.)

In case anyone was wondering, apparently it wants

(ns foo.bar.baz

  (:use [clojure.contrib.core :only (seqable?)]))

(and thus violates the usual clojure rule of using vectors rather than lists for groupings that are not invocations -- that is, function calls, macro calls, or special form calls).

[Richard Newman]

> (ns foo.bar.baz
> (:use [clojure.contrib.core :only (seqable?)]))
>
> (and thus violates the usual clojure rule of using vectors rather
> than lists for groupings that are not invocations -- that is,
> function calls, macro calls, or special form calls).

It works fine with a vector.

user=> (ns foo.bar.baz

(:use [clojure.contrib.core :only [seqable?]]))

nil
foo.bar.baz=> seqable?
#<core$seqable_QMARK___1454 clojure.contrib.core
$seqable_QMARK___1454@4f2b6c89>

[John Harrop]

On Tue, Nov 10, 2009 at 9:11 PM, Richard Newman <holy...@gmail.com> wrote:

> (ns foo.bar.baz
>   (:use [clojure.contrib.core :only (seqable?)]))
>
> (and thus violates the usual clojure rule of using vectors rather
> than lists for groupings that are not invocations -- that is,
> function calls, macro calls, or special form calls).

It works fine with a vector.

Yeah, but it doesn't require one, versus:

foo.bar.baz=> (let (a 10) (println a))

#<CompilerException java.lang.IllegalArgumentException: let requires a vector for its binding (NO_SOURCE_FILE:11)>

[David Brown]

On Tue, Nov 10, 2009 at 09:08:31PM -0500, John Harrop wrote:

>In case anyone was wondering, apparently it wants
>
>(ns foo.bar.baz
> (:use [clojure.contrib.core :only (seqable?)]))
>
>(and thus violates the usual clojure rule of using vectors rather than lists
>for groupings that are not invocations -- that is, function calls, macro
>calls, or special form calls).

You can use vectors for everything other than the outside parens, if
you'd like. They just need to be sequences. I'm guessing people
usually use parens on the :use, because it at least looks like an
invocation (and is similar to the (use ...) call). I've seen the
:only followed by a vector.

David

[Albert Cardona]

YES please. If I could upvote this message I would.

A half-a-dozen of examples on ns/in-ns and require/use/refer and the
differences in using them at the prompt or inside a ns would be
fantastic.

The ns macro is one of the obscure corners of clojure. It relates to
the java class path problem, and its setup is completely non-obvious.
The ns macro has the potential of putting off an otherwise committed
newcomer. Clear and abundant short examples would go a long way, and
be very appreciated.

As a practical note, what I do now to find out the misterious usage of
use/refer/require is to grep the clojure contrib git repository for
examples.

Albert

[Miron Brezuleanu]

Hello,

On Wed, Nov 11, 2009 at 2:57 PM, Albert Cardona <sapr...@gmail.com> wrote:
> YES please. If I could upvote this message I would.
>
> A half-a-dozen of examples on ns/in-ns and require/use/refer and the
> differences in using them at the prompt or inside a ns would be
> fantastic.

Some more ns documentation is on this page:
http://en.wikibooks.org/wiki/Clojure_Programming/Tutorials_and_Tips

Agreed, it's only a third of a dozen examples, and they cover little
of the syntax supported by ns.

> The ns macro is one of the obscure corners of clojure. It relates to
> the java class path problem, and its setup is completely non-obvious.
> The ns macro has the potential of putting off an otherwise committed
> newcomer. Clear and abundant short examples would go a long way, and
> be very appreciated.
>

--
Miron Brezuleanu

[Stephen C. Gilardi]

On Nov 10, 2009, at 9:08 PM, John Harrop wrote:

> (ns foo.bar.baz
> (:use [clojure.contrib.core :only (seqable?)]))
>
> (and thus violates the usual clojure rule of using vectors rather
> than lists for groupings that are not invocations -- that is,
> function calls, macro calls, or special form calls).

I agree it's too complicated and that examples would help.

Rich has also encouraged a look at simplifying the whole thing in the
past and I would like to work on that.

Here are some of the ideas I've liked best for how to do it.

- replace :use, :require, and :load with one common thing with good
conventions and easy ways to configure away from convention

- maybe call it :uses

- require that each "libspec" (reference to a lib) be a vector,
disallowing naked symbols and prefix extraction via prefix lists. This
would mean that everything after :uses will be a vector which is known
to be a complete specification of the dependency's name and how this
lib uses it. (This regularity would help humans understand and outside-
of-Clojure tools have an easier time parsing.)

- don't "refer" any names from the target namespace into the current
namespace by default

- support ":only []", ":rename {}", ":exclude []", and ":refer-all
true" options

(:refer-all true is not strictly necessary as it's a synonym
for :exclude [], but the latter is much less clear to the human reader)

- automatically alias the leaf name of the lib as a reference to the
lib:

- (:uses [clojure.contrib.jmx]) would allow referring to the names
in clojure.contrib.jmx by prefixing them with jmx/

- disallow conflicts in leaf names, requiring one or the other lib
to have an ":as" option giving another alias

- open question as to whether using :only, :rename, :exclude,
or :refer-all would suppress the automatic alias and require an
explicit :as if an alias is also desired.

- remove the special ":refer-clojure" clause in favor of an optional
"[clojure.core]" within :uses that changes the default for
clojure.core away from ":refer-all true"

- (based on this thread) require that all seqs within :uses be vectors
rather than lists.

Here is a portion of an ns form before and after these changes:

Before:

(:refer-clojure :exclude [read])
(:require (clojure.contrib [graph :as graph] [fcase :as fcase])
[clojure.contrib.stream-utils :as su])
(:use [clojure.contrib def except server-socket]
clojure.contrib.lazy-xml
[clojure.contrib.java-utils :only [as-str]]
[clojure.stacktrace :only (root-cause)]
[clojure.walk :only [postwalk]])

After:

(:uses [clojure.core :exclude [read])
[clojure.contrib.graph]
[clojure.contrib.fcase]
[clojure.contrib.stream-utils :as su]
[clojure.contrib.def :refer-all true]
[clojure.contrib.except :refer-all true]
[clojure.contrib.server-socket :refer-all true]
[clojure.contrib.lazy-xml :refer-all true]
[clojure.contrib.java-utils :only [as-str]]
[clojure.stacktrace :only [root-cause]]
[clojure.walk :only [postwalk]])

(Over time I would expect many ":refer-all true" options to become
more selective references once this change makes referencing in
everything slightly more difficult.)

I'd appreciate hearing ideas and critiques.

--Steve

[John Harrop]

On Wed, Nov 11, 2009 at 1:12 PM, Stephen C. Gilardi <sque...@mac.com> wrote:

Before:

 (:refer-clojure :exclude [read])
 (:require (clojure.contrib [graph :as graph] [fcase :as fcase])
           [clojure.contrib.stream-utils :as su])
 (:use [clojure.contrib def except server-socket]
       clojure.contrib.lazy-xml
       [clojure.contrib.java-utils :only [as-str]]
       [clojure.stacktrace :only (root-cause)]
       [clojure.walk :only [postwalk]])


After:

 (:uses [clojure.core :exclude [read])
        [clojure.contrib.graph]
        [clojure.contrib.fcase]
        [clojure.contrib.stream-utils :as su]
        [clojure.contrib.def :refer-all true]
        [clojure.contrib.except :refer-all true]
        [clojure.contrib.server-socket :refer-all true]
        [clojure.contrib.lazy-xml :refer-all true]
        [clojure.contrib.java-utils :only [as-str]]
        [clojure.stacktrace :only [root-cause]]
        [clojure.walk :only [postwalk]])

(Over time I would expect many ":refer-all true" options to become more selective references once this change makes referencing in everything slightly more difficult.)

I'd appreciate hearing ideas and critiques.

+1 

[Andrew Boekhoff]

> > (:uses [clojure.core :exclude [read])
> > [clojure.contrib.graph]
> > [clojure.contrib.fcase]
> > [clojure.contrib.stream-utils :as su]
> > [clojure.contrib.def :refer-all true]
> > [clojure.contrib.except :refer-all true]
> > [clojure.contrib.server-socket :refer-all true]
> > [clojure.contrib.lazy-xml :refer-all true]
> > [clojure.contrib.java-utils :only [as-str]]
> > [clojure.stacktrace :only [root-cause]]
> > [clojure.walk :only [postwalk]])

+1

As a thought (which may offend some fans of explicit delimiters),
dropping the brackets and delimiting on symbols could be an option.

(ns foo
(uses clojure.contrib.graph
clojure.contrib.stream-utils :as su
clojure.contrib.java-utils :only [as-str]))

[Chouser]

On Wed, Nov 11, 2009 at 1:12 PM, Stephen C. Gilardi <sque...@mac.com> wrote:
>

> Here are some of the ideas I've liked best for how to do it.

Thanks for pulling this together. I like the whole direction
you're going here.

> - require that each "libspec" (reference to a lib) be a vector, disallowing
> naked symbols and prefix extraction via prefix lists. This would mean that
> everything after :uses will be a vector which is known to be a complete
> specification of the dependency's name and how this lib uses it. (This

> regularity would help humans understand and outside-of-Clojure tools have an
> easier time parsing.)

The only time this would bother me in the least is at the REPL.
I find the suggestion of not grouping them at all someone
interesting as well -- a bit like :gen-class.

> - don't "refer" any names from the target namespace into the current
> namespace by default

Hooray!

> - automatically alias the leaf name of the lib as a reference to the lib:

I think this may be ok, maybe even really great, though I think
Rich pointed out that it will encourage the creation of
namespaces with short leaf names.

>  - open question as to whether using :only, :rename, :exclude, or :refer-all
> would suppress the automatic alias and require an explicit :as if an alias
> is also desired.

I vote for minimal magic -- auto alias happens unless you say :as
explicitly. That does raise the question of how to prevent any
alias. I don't particularly like either ":as dummy-alias-2" or
":alias false" but I'm not sure what else it could be. Any other
ideas?

> - remove the special ":refer-clojure" clause in favor of an optional
> "[clojure.core]" within :uses that changes the default for clojure.core away
> from ":refer-all true"

Maybe require [clojure.core :refer-all false] if that's what you
mean? Seems to me (:uses [clojure.core]) looks a lot like
a no-op.

I think all the other suggestions are good, and maybe more of
them deserve "Hoorays" as well. :-)

--Chouser

[Christophe Grand]

Hi!

On Wed, Nov 11, 2009 at 7:12 PM, Stephen C. Gilardi <sque...@mac.com> wrote:
> Here are some of the ideas I've liked best for how to do it.

I like where this is heading.

> - don't "refer" any names from the target namespace into the current
> namespace by default

YES!

>  - support ":only []", ":rename {}", ":exclude []", and ":refer-all true"
> options

and of course ":as alias" that you mention below

>    (:refer-all true is not strictly necessary as it's a synonym for :exclude
> [], but the latter is much less clear to the human reader)

In general :exclude suffers from the same problems as refer.
The new system should drive users away from :exclude to :only.

> - automatically alias the leaf name of the lib as a reference to the lib:

This is the only proposition I don't agree with. What about allowing
lib authors to specify a default alias?

Christophe

[Laurent PETIT]

2009/11/11 Andrew Boekhoff <boek...@gmail.com>:

+1 on giving up on the extra vectors, since they give nothing more if
the dsl becomes that simple (and I also like it becomes that simple
!).

One question, to be sure to correctly understand: only one optional
option is allowed after the symbol of a namespace ? (that is possible
options are considered mutually exclusive ?)

Also, I thing it is not too complicated for users to understand that :
* either they don't provide an option and the behaviour defaults to [
last segment of the ns, or what chouser / Christophe suggest ]
* either they provide an option, so the optional behaviour does not
apply, and it's the provided option that applies

My 0.02 €,

--
Laurent

[John Harrop]

Not necessarily; if the parsing rule is that a keyword makes a following symbol part of an option, rather than a namespace name,  and all options apply to the most recently seen namespace name. So

(ns foo

  (uses clojure.contrib.duck-streams :as ds :only [reader-on spit]

        clojure.contrib.seq-utils :as su))

would alias everything in seq-utils under su, e.g. su/includes?, and would also produce ds/reader-on and ds/spit.

Other combinations might be allowed too. For example, perhaps

(ns foo

  (uses clojure.contrib.duck-streams :only [reader-on spit] :as ds ))

should produce bindings for reader-on and spit in foo and for everything as ds/whatever, and even

(ns foo

  (uses clojure.contrib.duck-streams

    :only [reader-on spit]

    :as ds :only [writer-on]))

would get you reader-on, spit, and ds/writer-on.

One question: how would Java class imports be dealt with? I think it should be unified:

(ns foo

  (uses java.io :only [File FileInputStream] :as io))

gets you File, FileInputStream, io/FileOutputStream, io/FileNotFoundException, etc.

The one difference being that :refer-all can't work easily for Java packages, since there's no easy way that I'm aware of to programmatically get a global list of everything in a Java package, although I suppose any bare symbol not otherwise resolved could be searched for in each Java package in turn that was refer-all'd. Detecting and forcing resolution of name conflicts wouldn't work in this case, though, which bothers me; either conflicts involving two Java classes brought in via refer-all would not be detected at all, or they'd be detected only at the point where the name resolution happens, thus possibly not until run-time, and then they would depend on what jars might be on that particular install's classpath. Ouch. So, I vote for not allowing :refer-all for Java packages.

As for how it can distinguish Java classes from functions:

First, in the case above the java.io package has no loadable Clojure namespace in it, so obviously any name being used from it should be assumed to be a Java class. Now if we use foo.bar and there's both a foo/bar.clj with a (defn my-fn ...) and a foo/bar/Baz.class on the classpath, and we have (uses foo.bar :only [my-fn Baz]), it should find my-fn in the Clojure namespace, fail to find any Baz var in the Clojure namespace, and then see if there's a Baz class in a Java package foo.bar, and so end up bringing in the my-fn Clojure function and the Baz class as one would expect.

[Jason Wolfe]

I like almost all of this a lot. My only disagreement is on prefix
lists ... I wouldn't want to lose them, and in fact would prefer to
see them extended to recursive prefix lists (trees).

-Jason

>  smime.p7s
> 3KViewDownload

[Andrew Boekhoff]

With clojure-in-clojure on the horizon (if not around the corner) I wonder if
an imports clause would be appropriate, and solve some of the complexities
of discerning between clojure and java/clr/javascript/objectivec/(go?)

(ns foo
(uses clojure.contrib.repl-utils)
(imports java.util [List Map Set]))

For the case of host-libraries import * is not even available,
so a single symbol for a class or a prefix symbol followed by
a vector is syntax simple enough for even myself to implement.

It might add some consistency if the above 'uses also worked at the repl as-is
(import already works as a beginner would anticipate). So,

user> (uses clojure.contrib.Math) just worked without quoting.

As for how many options are allowed with keywords, as long the essential cases
are provided for it will always be possible to layer on a my-better-ns* macro
that adds more sugar (like recursive prefix lists etc.)

[Chouser]

On Wed, Nov 11, 2009 at 4:24 PM, John Harrop <jharr...@gmail.com> wrote:
> One question: how would Java class imports be dealt with? I think it should
> be unified:
> (ns foo
>   (uses java.io :only [File FileInputStream] :as io))

I think this is a bad idea. Java (or other host) classes are not
Clojure functions, nor are host packages the same as Clojure
namespaces. Trying to treat them the same (or worse yet
not quite the same because of options that work for one but not
the other) actually *increases* confusion, not decreases it.

Unifying require and use makes sense because they do almost the
same thing on exactly the same kinds of objects. import does
something lese and should be left as its own form.

--Chouser

[John Harrop]

On Fri, Nov 13, 2009 at 1:23 PM, Chouser <cho...@gmail.com> wrote:

On Wed, Nov 11, 2009 at 4:24 PM, John Harrop <jharr...@gmail.com> wrote:
> One question: how would Java class imports be dealt with? I think it should
> be unified:
> (ns foo
>   (uses java.io :only [File FileInputStream] :as io))

I think this is a bad idea.  Java (or other host) classes are not
Clojure functions

Not everything in a Clojure namespace is necessarily a Clojure function either. Anything a var can hold (so, in fact, any Object) can be in a Clojure namespace.

The point was to simplify syntax, rather than to simplify the underlying mental model; the latter I agree can't simplify further since, as you say, the host interop is not the same as Clojure.