Declaration Order, Namespaces & Modules
Toralf Wittner
It seems that top-level definitions cannot be declared in arbitrary
order, e.g.
(defn f []
(g))
(defn g []
nil)
will cause a clojure.lang.Compiler$CompilerException with message
"Unable to resolve symbol: g in this context". Is this just a limitation
of the current implementation?
Also, are there plans to allow function definitions to be hidden in
namespaces or even to provide a module system similar to the one in
Scheme R6RS (cf.
http://www.r6rs.org/final/html/r6rs/r6rs-Z-H-10.html#node_chap_7)?
Thanks,
Toralf
Rich Hickey
On Dec 17, 1:22 pm, Toralf Wittner <toralf.witt...@gmail.com> wrote:
> Hello group,
>
> It seems that top-level definitions cannot be declared in arbitrary
> order, e.g.
>
> (defn f []
> (g))
>
> (defn g []
> nil)
>
> will cause a clojure.lang.Compiler$CompilerException with message
> "Unable to resolve symbol: g in this context". Is this just a limitation
> of the current implementation?
(defn f []
(typo))
This is a runtime error in many Lisps, but caught at compile time in
Clojure.
You can easily and safely refer to not-yet-defined names by
'declaring' them with empty defs:
(def g) ;creates a var g that is unbound
(defn f []
(g)) ;ok
(defn g [] ;f will call this
nil)
Should you fail to eventually def a value for g you'll get a runtime
error for access to an unbound var.
>
> Also, are there plans to allow function definitions to be hidden in
> namespaces
an *exports* list which exposes the names it intends for public use
(as does boot.clj for the clojure namespace). Consuming code can use
these names by doing:
(refer lib/*exports*)
or some filtered version of the export list. If explicitly qualified
lib/some-name references appear in some code chances are that is
breaking encapsulation, and visibly so, but it is not prohibited - it
can be tremendously convenient when diagnosing a problem vs having to
change access specifiers or export lists in more restrictive
languages.
> or even to provide a module system similar to the one in
> Scheme R6RS (cf.http://www.r6rs.org/final/html/r6rs/r6rs-Z-H-10.html#node_chap_7)?
>
adequate, if imperfect, way to manage very large libraries from
multiple sources. Modules, with their lexical, closed nature are much
too static for my taste, and not what I want for Clojure.
Rich
John Cowan
> Yikes! No. Java and .Net have demonstrated that namespaces are an
> adequate, if imperfect, way to manage very large libraries from
> multiple sources. Modules, with their lexical, closed nature are much
> too static for my taste, and not what I want for Clojure.
I think you must have the wrong definition of "module" in your mental
hash tables. R6RS libraries are actually very like JVM packages or
CLR namespaces: each one has a hierarchical name, and they are
just a way of wrapping up a bunch of names and making them available
to other libraries or main programs. Like packages, libraries allow you
to hide some names from the outside world and to import names either
selectively or wholesale; unlike packages, libraries also allow you to
give both an internal and an external name to the same referent, and
allow a library to refer to names in another library by various kinds of
convenient local aliases as well as mere shortened forms.
The only other features of libraries are (a) some small support for
versioning that Java could usefully have also, and (b) a mechanism
for specifying exactly when during the compilation process a library
is to be loaded, if at all, so that library routines can be invoked at
compile time from macro expanders.
--
GMail doesn't have rotating .sigs, but you can see mine at
http://www.ccil.org/~cowan/signatures
Rich Hickey
On Dec 17, 6:04 pm, "John Cowan" <johnwco...@gmail.com> wrote:
>
> > Yikes! No. Java and .Net have demonstrated that namespaces are an
> > adequate, if imperfect, way to manage very large libraries from
> > multiple sources. Modules, with their lexical, closed nature are much
> > too static for my taste, and not what I want for Clojure.
>
> I think you must have the wrong definition of "module" in your mental
> hash tables.
http://www.r6rs.org/final/html/r6rs/r6rs-Z-H-10.html#node_chap_7
the first thing I see is this:
-----------------
A library definition must have the following form:
(library <library name>
(export <export spec> ...)
(import <import spec> ...)
<library body>)
-----------------
Which implies (to me) that everything in a library must be in a single
file between those parens. Is it not the case that the entirety of a
library definition must appear in one place? That is what I mean by
lexical and closed.
> R6RS libraries are actually very like JVM packages or
> CLR namespaces: each one has a hierarchical name, and they are
> just a way of wrapping up a bunch of names and making them available
> to other libraries or main programs.
that hides an essential difference. JVM packages and .Net namespaces
are merely prefixes - they aren't even reified or enumerable. They are
open sets - e.g. the set of all names beginning with java.util. It is
possible at runtime to create new members of the set. From what I've
read about R6RS libraries (please correct me if I'm wrong), that's not
the case there. That's what I mean by static.
> Like packages, libraries allow you
> to hide some names from the outside world and to import names either
> selectively or wholesale; unlike packages, libraries also allow you to
> give both an internal and an external name to the same referent, and
> allow a library to refer to names in another library by various kinds of
> convenient local aliases as well as mere shortened forms.
>
be provided in either.
> The only other features of libraries are (a) some small support for
> versioning that Java could usefully have also, and (b) a mechanism
> for specifying exactly when during the compilation process a library
> is to be loaded, if at all, so that library routines can be invoked at
> compile time from macro expanders.
>
adoption and success. I wish them luck, but it's not something I'm
going to emulate right now.
Rich
Henk Boom
> On Dec 17, 1:22 pm, Toralf Wittner <toralf.witt...@gmail.com> wrote:
> >
> > Also, are there plans to allow function definitions to be hidden in
> > namespaces
>
> Not prohibitively. By convention, a library namespace should include
> an *exports* list which exposes the names it intends for public use
> (as does boot.clj for the clojure namespace). Consuming code can use
> these names by doing:
>
> (refer lib/*exports*)
>
> or some filtered version of the export list. If explicitly qualified
> lib/some-name references appear in some code chances are that is
> breaking encapsulation, and visibly so, but it is not prohibited - it
> can be tremendously convenient when diagnosing a problem vs having to
> change access specifiers or export lists in more restrictive
> languages.
Visibly so? How can you tell whether someone just wants to take
advantage of the namespace (i.e. using gl/color because color is too
general) or whether they are abusing it? IMO a big advantage of
namespaces is that you can use long names for disambiguation while
using short names in the namespace itself for clear/concise code.
I would think it better to restrict the /-notation to exported
variables, and if desired add sufficiently "ugly" notation when
equally ugly encapsulation breakage is needed.
Henk
John Cowan
> Which implies (to me) that everything in a library must be in a single
> file between those parens. Is it not the case that the entirety of a
> library definition must appear in one place? That is what I mean by
> lexical and closed.
Well, the Standard says nothing about files: there could be an include
mechanism that applies at a lower level than the level of libraries.
But in the typical case you are correct.
However, I argue that this doesn't matter very much. Although Java
requires each public class in a package to be in a single file (C# has
no such restrictions), it is extremely unusual for people to add
classes to one another's packages. In practice, the multiple files
that make up a package constitute a single protection boundary, and in
compiled form one or more packages are usually represented in their
entirety in a JAR file. So packages are effectively closed from the
viewpoint of their users.
> I understand the intent is similar, but this is an oversimplification
> that hides an essential difference. JVM packages and .Net namespaces
> are merely prefixes - they aren't even reified or enumerable.
Libraries are definitely not enumerable, and I don't believe they are
reified at run time either. Presumably a sensible compiler would
reify them.
> They are
> open sets - e.g. the set of all names beginning with java.util. It is
> possible at runtime to create new members of the set.
Possible but not usual; in fact, pretty much confined to systems that
generate classes at run time either by compiling or by bytecode
generation, neither of which is found in typical Java code, only in
specialized libraries.
> From what I've
> read about R6RS libraries (please correct me if I'm wrong), that's not
> the case there. That's what I mean by static.
Hmm. I think you are right about that.
> R6RS libraries represent an experiment that has yet to see widespread
> adoption and success. I wish them luck, but it's not something I'm
> going to emulate right now.
Fair enough.
Rich Hickey
On Dec 17, 7:35 pm, "John Cowan" <johnwco...@gmail.com> wrote:
but I think they are more due to the static nature of Java than
anything fundamental about namespaces. The needs and usage patterns
will be different with a dynamic language. My basic point is,
namespaces/prefixes are open, modules are (gererally) closed, and the
former is IMO a better fit for Clojure, which skews towards open/
extensible.
That said, I think there is a lot of value in reifying namespaces and
supporting reflective access to them, while leaving them open and
dynamic (more like CL packages, although the analogy only extends so
far). I am planning to add that to Clojure.
Rich
Rich Hickey
On Dec 17, 7:14 pm, "Henk Boom" <lunarcri...@gmail.com> wrote:
distinguished from the valid use case you present. Doing better
requires some more 'official' enumerability of at least the export
list of a namespace, such that only exported names can get the simple
access qualifier while non-exported must get the distinguished
qualifier (in CL they are : and :: respectively, could be / and // in
Clojure, it need not be ugly only grep-able :).
As I said in another post in this thread, I am planning to add more
tangibility and reflective access to namespaces in Clojure.
Rich
Toralf Wittner
> I consider it an important feature. Consider:
>
> (defn f []
> (typo))
>
> This is a runtime error in many Lisps, but caught at compile time in
> Clojure.
>
> You can easily and safely refer to not-yet-defined names by
> 'declaring' them with empty defs:
>
> (def g) ;creates a var g that is unbound
>
> (defn f []
> (g)) ;ok
>
> (defn g [] ;f will call this
> nil)
>
> Should you fail to eventually def a value for g you'll get a runtime
> error for access to an unbound var.
>
It seems then that declaring names with empty defs defeats the very same
reason why you require definition before use in the first place. Anyway
I don't think I like this restriction. It reminds me of doing C
programming and I haven't experienced the problem it tries to address
often enough to appreciate the restriction. Just my two Cents.
> >
> > Also, are there plans to allow function definitions to be hidden in
> > namespaces
>
> Not prohibitively. By convention, a library namespace should include
> an *exports* list which exposes the names it intends for public use
> (as does boot.clj for the clojure namespace). Consuming code can use
> these names by doing:
>
> (refer lib/*exports*)
>
> or some filtered version of the export list. If explicitly qualified
> lib/some-name references appear in some code chances are that is
> breaking encapsulation, and visibly so, but it is not prohibited - it
> can be tremendously convenient when diagnosing a problem vs having to
> change access specifiers or export lists in more restrictive
> languages.
>
But how do you deal with name clashes? Given
(in-namespace 'a)
(defn x [] (print 42))
(def *exports* '(a x))
(in-namespace 'b)
(defn x [] (print 43))
(def *exports* '(b x))
(refer a/*exports*)
(refer b/*exports*)
java.lang.Exception: Name conflict: x already exists in this refer map
as: a/x
I think it is legitimate to use a/x and b/x in client code to avoid such
issues.
> > or even to provide a module system similar to the one in
> > Scheme R6RS (cf.http://www.r6rs.org/final/html/r6rs/r6rs-Z-H-10.html#node_chap_7)?
> >
>
> Yikes! No. Java and .Net have demonstrated that namespaces are an
> adequate, if imperfect, way to manage very large libraries from
> multiple sources. Modules, with their lexical, closed nature are much
> too static for my taste, and not what I want for Clojure.
>
Well, I didn't expect you to copy Scheme's module system as it is. For
me the essential characteristics of a module system are the ability to
declare the public interface of the module and to hide certain
definitions. Namespaces are good enough for me but something like
anonymous namespaces à la C++ would come in handy then. Also I think an
abstraction from the filesystem is needed when loading modules (i.e.
load-file is too basic).
> Rich
>
Thanks,
Toralf
Henk Boom
> Also I think an
> abstraction from the filesystem is needed when loading modules (i.e.
> load-file is too basic).
I agree that load-file is too basic, I think to modularize code there
needs to be a variant of load-file which:
1) loads from a filename relative to the code which is calling it
(i.e. (load-file "bar.clj") inside "tmp/foo.clj" loads "tmp/bar.clj")
2) loads a file if and only if it has not already been loaded
Henk
Rich Hickey
On Dec 17, 8:43 pm, Toralf Wittner <toralf.witt...@gmail.com> wrote:
> On Mon, 2007-12-17 at 13:00 -0800, Rich Hickey wrote:
> > I consider it an important feature. Consider:
>
> > (defn f []
> > (typo))
>
> > This is a runtime error in many Lisps, but caught at compile time in
> > Clojure.
>
> > You can easily and safely refer to not-yet-defined names by
> > 'declaring' them with empty defs:
>
> > (def g) ;creates a var g that is unbound
>
> > (defn f []
> > (g)) ;ok
>
> > (defn g [] ;f will call this
> > nil)
>
> > Should you fail to eventually def a value for g you'll get a runtime
> > error for access to an unbound var.
>
> It seems then that declaring names with empty defs defeats the very same
> reason why you require definition before use in the first place.
unit' (and define what that is) about undefined free references, then
no, it is not the same at all. That's why the example was 'typo',
which is far easier to make than accidental matching declaration and
use with missing definition.
> Anyway
> I don't think I like this restriction. It reminds me of doing C
> programming and I haven't experienced the problem it tries to address
> often enough to appreciate the restriction. Just my two Cents.
>
use declaring defs and don't find it a restriction at all, while
getting the typos flagged right away has been a productivity boost. In
any case, it's a small thing, I hope you can live with it.
> But how do you deal with name clashes? Given
>
> (in-namespace 'a)
> (defn x [] (print 42))
> (def *exports* '(a x))
>
> (in-namespace 'b)
> (defn x [] (print 43))
> (def *exports* '(b x))
>
> (refer a/*exports*)
> (refer b/*exports*)
> java.lang.Exception: Name conflict: x already exists in this refer map
> as: a/x
>
> I think it is legitimate to use a/x and b/x in client code to avoid such
> issues.
>
encapsulation-violating qualifiers will be distinguished.
> Also I think an
> abstraction from the filesystem is needed when loading modules (i.e.
> load-file is too basic).
>
Rich
Toralf Wittner
On Mon, 2007-12-17 at 18:08 -0800, Rich Hickey wrote:
> On Dec 17, 8:43 pm, Toralf Wittner <toralf.witt...@gmail.com> wrote:
> > It seems then that declaring names with empty defs defeats the very same
> > reason why you require definition before use in the first place.
>
> Unless the compiler is going to balk at the end of a 'compilation
> unit' (and define what that is) about undefined free references, then
> no, it is not the same at all. That's why the example was 'typo',
> which is far easier to make than accidental matching declaration and
> use with missing definition.
>
What I meant is that (defn f [] (g)) and (def g) (defn f [] (g)) would
both fail at runtime given that no checks are made during compilation.
> Ok. For myself, other than mutual recursion I don't find much need to
> use declaring defs and don't find it a restriction at all, while
> getting the typos flagged right away has been a productivity boost. In
> any case, it's a small thing, I hope you can live with it.
Yes, I certainly can. It just feels a little inconvenient to me.
> > Also I think an
> > abstraction from the filesystem is needed when loading modules (i.e.
> > load-file is too basic).
> >
>
> I'd certainly welcome suggestions as to what's needed.
>
Many language implementations utilize a path to search for entities to
load, e.g. Java has a CLASSPATH, Python has a PYTHONPATH, Lua has a
LUA_PATH and so on and so forth. I think something like CLOJURE_PATH
should do it for us as well. Maybe like Python we also consider the
current working directory. Also as Henk suggested, entities should be
loaded only once.
Cheers,
Toralf
Henk Boom
> What I meant is that (defn f [] (g)) and (def g) (defn f [] (g)) would
> both fail at runtime given that no checks are made during compilation.
Aren't checks made during compilation? The first of those definitely
doesn't work for me.
> ...
> Many language implementations utilize a path to search for entities to
> load, e.g. Java has a CLASSPATH, Python has a PYTHONPATH, Lua has a
> LUA_PATH and so on and so forth. I think something like CLOJURE_PATH
> should do it for us as well.
I like this idea better than my previous idea wrt the path of the current file.
Henk
Henk Boom
> load, e.g. Java has a CLASSPATH, Python has a PYTHONPATH, Lua has a
> LUA_PATH and so on and so forth. I think something like CLOJURE_PATH
> should do it for us as well.
Given that is may be common for java and clojure code to be mixed,
would it make sense to simply look for these files in the java
classpath?
Henk
Toralf Wittner
I attached an experimental patch which introduces a new special function
"require" which utilizes load-file to load files from either the current
working directory or the first directory in the union of CLASSPATH and
CLOJURE_PATH directories where it finds the filename. JAR files listed
in CLASSPATH are skipped. It will also only load files once. Please note
that the implementation is rather naive and only meant to foster the
discussion.
Thanks,
Toralf
> Henk
>
> >
Toralf Wittner
On Tue, 2007-12-18 at 11:44 -0500, Henk Boom wrote:
> On 18/12/2007, Toralf Wittner <toralf....@gmail.com> wrote:
> > What I meant is that (defn f [] (g)) and (def g) (defn f [] (g)) would
> > both fail at runtime given that no checks are made during compilation.
>
> Aren't checks made during compilation? The first of those definitely
> doesn't work for me.
>
Yes there are checks. I meant both would fail *if* no checks are made.
The second case fails during runtime if you forget to add a definition
for g whereas the first one would fail during runtime if you either
misspell g in the body of f or forget to add a definition of g (and the
compiler *would* not require you to at least declare g before defining
f). So Rich is right when he says that the requirement to declare before
use captures the typo problem but this hasn't been so much of a problem
for me and it still leaves open the possibility that you declare g but
forget to define it later unless you don't declare g but reorder the
code such that you have g properly defined before defining f.
> > ...
> > Many language implementations utilize a path to search for entities to
> > load, e.g. Java has a CLASSPATH, Python has a PYTHONPATH, Lua has a
> > LUA_PATH and so on and so forth. I think something like CLOJURE_PATH
> > should do it for us as well.
>
> I like this idea better than my previous idea wrt the path of the current file.
>
See my reply to your second email.
> Henk
>
> >