ns/load/require/use overhaul
Stuart Halloway
thing for Clojure beginners. I have created a ticket [1] that
includes some "musts" as well as some "maybes". (Also see khinsen's
post at [2]). Would love to see some discussion on this. Once Rich is
happy with the design goals I will make sure that a patch gets tested
and accepted quickly.
Steve, I know you had been thinking about an overhaul -- anything we
should know? :-)
Stu
[1] http://www.assembla.com/spaces/clojure/tickets/272-load-ns-require-use-overhaul
[2] http://onclojure.com/2010/02/17/managing-namespaces
Sean Devlin
http://fulldisclojure.blogspot.com/2010/02/thoughts-on-namespace-management.html
Sean
On Feb 18, 3:52 am, Stuart Halloway <stuart.hallo...@gmail.com> wrote:
> In my experience namespace management is the single most irritating
> thing for Clojure beginners. I have created a ticket [1] that
> includes some "musts" as well as some "maybes". (Also see khinsen's
> post at [2]). Would love to see some discussion on this. Once Rich is
> happy with the design goals I will make sure that a patch gets tested
> and accepted quickly.
>
> Steve, I know you had been thinking about an overhaul -- anything we
> should know? :-)
>
> Stu
>
> [1]http://www.assembla.com/spaces/clojure/tickets/272-load-ns-require-us...
> [2]http://onclojure.com/2010/02/17/managing-namespaces
Konrad Hinsen
> In my experience namespace management is the single most irritating thing for Clojure beginners. I have created a ticket [1] that includes some "musts" as well as some "maybes". (Also see khinsen's post at [2]). Would love to see some discussion on this. Once Rich is happy with the design goals I will make sure that a patch gets tested and accepted quickly.
Thanks!
After your musts, I think the most important point is to reorganize the ns functions such that ":require" and ":use ... :only" are encouraged and the global :use is discouraged. Which leaves the big question of how to handle clojure.core.
It might be helpful to look at how these issues are handled in Python (the language I know best). The equivalent of plain :use is "from XXX import *" and is discouraged except for interactive use. The equivalent of clojure.core is the __builtin__ module, whose names are available in all modules (the Python equivalent of namespaces). The big difference to Clojure is that all names can be redefined at any time, so a local definition that shadows something from __builtin__ is simply not a problem.
Trying to transpose the Python import syntax into Clojure, I would get something like the following:
(:use package.namespace)
current :require
(:use package.namespace a b c)
current (:use [package.namespace :only (a b c)])
(:use package.namespace :all)
current (:use package.namespace)
I think this is a lot simpler to understand than the current syntax. No parentheses or brackets - if you use several external namespaces, you just write several :use clauses. What disappears is the :exclude option. It could be replaced by
(:exclude x y z)
remove symbols x y and z from the namespace, no matter where they came from
I just don't have a great idea for solving the clojure.core problem.
Konrad.
Mike Hinchey
Instead, could support support an :as clause something like this:
(:import java.io :as jio)
To be used like:
(jio.InputStream.)
This would be unambiguous, allow static lookup, and eliminate the need to type each classname into the ns clause. A dot would still be the separator because the classname is part of the "namespace", but this would allow a shortcut.
-Mike
Meikel Brandmeyer
On Sun, Feb 21, 2010 at 11:55:11PM -0800, Mike Hinchey wrote:
> Instead, could support support an :as clause something like this:
> (:import java.io :as jio)
>
> To be used like:
> (jio.InputStream.)
+1 This is really nice!
Sincerely
Meikel
Timothy Pratley
>
>> Instead, could support support an :as clause something like this:
>> (:import java.io :as jio)
>>
>> To be used like:
>> (jio.InputStream.)
I like this very much also. To be consistent it might need to be
grouped like (:import [java.io :as jio])?
Konrad Hinsen
> Trying to transpose the Python import syntax into Clojure, I would
> get something like the following:
>
> (:use package.namespace)
> current :require
>
> (:use package.namespace a b c)
> current (:use [package.namespace :only (a b c)])
>
> (:use package.namespace :all)
> current (:use package.namespace)
I have added this (under the name :from) to my nstools package, so if
you want to play with it, go ahead:
http://code.google.com/p/clj-nstools/
http://clojars.org/nstools
Konrad.
Chouser
So these are package aliases? I'm not at all opposed.
But I'm much more interested in class name aliases, the most
frequent use case being deeply-nested or long-named nested
classes. For example, here's a full class name from Google's
protobuf library:
com.google.protobuf.Descriptors$FieldDescriptor$Type
If I want to type-hint this, for example as the return type of
a defn, the shortest I can possibly make it is:
Descriptors$FieldDescriptor$Type
Bleh.
--Chouser
http://joyofclojure.com/
Mike Hinchey
(:import [com.google.protobuf.Descriptors$FieldDescriptor :as gfd])
(gfd$Type.)
-Mike
--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To post to this group, send email to cloju...@googlegroups.com.
To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.
John R. Williams
established language that gets of lot of things right. Here's how I
would suggest adding Python's "as" keyword to your proposal:
Python: import a as x
Current: Not possible?
Proposed: (:use a :as x)
Python: import a.b as x
Current: (:require '(a (b :as x)))
Proposed: (:use a.b :as x)
Python: from a import b as x, c as y, d, e
Current: (:use a :rename {b x, c y}) (:use a :only [d e])
Proposed: (:use a b :as x, c :as y, d, e)
I also have an idea for how to deal with clojure.core. Basically, any
name implicitly imported from clojure.core would need to receive a
tentative binding that could be replaced by a def, with one caveat:
the first time a tentatively-bound name is resolved, it becomes
permanent. The caveat is to prevent the same name from resolving to
two different vars in the same namespace, like this:
(ns foo)
(assert (= #'inc #'clojure.core/inc))
(def inc nil)
(assert (= #'inc #'foo/inc))
I think it makes sense for any symbol imported with :all to have a
tentative binding. Having tentative bindings would also reduce or
eliminate the need for an :exclude clause.
--jw
Brian Hurt
In my experience namespace management is the single most irritating thing for Clojure beginners. I have created a ticket [1] that includes some "musts" as well as some "maybes". (Also see khinsen's post at [2]). Would love to see some discussion on this. Once Rich is happy with the design goals I will make sure that a patch gets tested and accepted quickly.
Steve, I know you had been thinking about an overhaul -- anything we should know? :-)
Stu
I'm using clojure in a 30+ KLOC project (and growing rapidly) that heavily uses namespaces. As such, I thought I'd offer some observations.
First off, I'm not sure I like the idea of using beginners as the standard to measure namespace usefulness against. First of all, that ways lies basic. Second of all, you can go a hell of a long way in clojure with little more than the occasional use or requires statement.
Second, using name spaces requires a slightly different approach in a lot of things. Some rules of thumb we've come up with:
1) Prefer requires over uses. What *inevitably* happens, when you just dump symbols into name spaces willy-nilly, is that people start implement pseudo-name-spaces- they start prefixing every symbol in the foobar library with foobar-, so they stop name colliding when you include foobar with other libraries. And/or you get boatloads of name collisions. Then, when you do try to switch back, you end up typing foobar/foobar-whatever, which is just annoying. No. Use the name-space capability built into the language from the get go.
2) Private is your friend. Private should be the default for symbols, don't export what you don't need. As a side note, I wish there was a def- and defmacro- to go with defn-.
3) Don't fear the name spaces. There is very little cost to introducing a new namespace, so do it. If you have a foobar namespace, and some code only needs the foo part, and some code only needs the bar part, split it into two different name spaces, a foo and a bar. Code that needs both should import both. If the have a common substrate, make that it's own name space.
4) One name space per file, and the file should have the same name. Also, every file should be it's own name space.
5) Indent and sort you imported/used/required modules.
6) Only require/use/import in the ns declaration.
Given these rules of thumb, I don't have the problems that Stephen seems to have. I don't have copy&paste problems with ns declarations, or name collisions, etc.
Lastly, don't break all of my code. Please.
Brian
Brian Hurt
I like the idea of stealing ideas from Python, because it's an
established language that gets of lot of things right. Here's how I
would suggest adding Python's "as" keyword to your proposal:
Python: import a as x
Current: Not possible?
Proposed: (:use a :as x)
nil
user=> (doc my-set/union)
-------------------------
clojure.set/union
([] [s1] [s1 s2] [s1 s2 & sets])
Return a set that is the union of the input sets
nil
user=>
Just FYI.
Chas Emerick
Richard Newman
> heavily uses namespaces. As such, I thought I'd offer some
> observations.
My 2¢: I've converged on exactly the same approach on my 10KLOC of
libraries and apps. One namespace per file, hierarchical, lots of
namespaces, using 'require' for almost everything, keeping things
private, and keeping things neat. It works really well, it scales,
it's self-documenting, and it makes finding things easier.
It also gives you more indication about structural problems: the
pattern of requires, the things you use, and their interdependencies
can give you clues about which things should be smushed together and
which should be split apart. It's an indicator of coupling.
I also agree that optimizing for simple, beginner-oriented use — e.g.,
trying to replicate Python import statements — seems the wrong way to
go. Much better would be to find the biggest, most dependency rich app
you could find, and study that instead.
Constantine Vetoshev
> There is very little cost to introducing a new namespace
Since namespaces cannot mutually depend on each other, this is not
entirely true. I've found that introducing a new namespace can lead to
lock-in during exploratory programming. A sensible refactoring of code
can easily introduce circularity, and all the resulting headaches of
resolving it.
Mark Engelberg
On Tue, Mar 23, 2010 at 5:08 PM, Brian Hurt <bhu...@gmail.com> wrote:
> There is very little cost to introducing a new namespace
Another "hidden cost" I've found comes from the need to carefully match namespaces to their filenames and directory structure. When I want to copy or rename directories or files to create a fork of my code for exploration, there is a lot of name-changing that has to occur.
PLT Scheme used to have a similar namespace-declaration-at-top-of-file-must-match-filename rule (called modules though, not namespaces), and it was similarly annoying until eventually they abandoned this and made the namespace-declaration implicit and automatic, based on the filename. Python files also don't require any sort of explicit namespace declaration at the top of the file, and are implicit by filename. This way of doing things seems much more convenient to me, and I don't understand what is gained by Clojure's way.
Richard Newman
> declaration at the top of the file, and are implicit by filename.
> This way of doing things seems much more convenient to me, and I
> don't understand what is gained by Clojure's way.
Firstly, if you need to step outside of convention, you have a way to
do so. (E.g., if you wish to put multiple namespaces in a single file,
perhaps generating one via a macro.)
Secondly, Python's implicit namespaces and search paths cause
atrocious bugs: as an example, name a file in the current directory
"calendar.py", or "codecs.py", or something else which collides with
the standard library. (Do you know every Python library on your
system?) Tada! You've silently shadowed a core module. This has bitten
me so many times it's just not funny: one of my own files will shadow
a *transitive* dependency of another file, and suddenly importing that
file will stop working.
Thirdly, Python files tend to start with a ton of imports. One might
as well collect all of the use/require/import statements in a single
form, associating them explicitly with a namespace. It's not very
Lispy to have a sequence of imperatives in a file.
Konrad Hinsen
> First off, I'm not sure I like the idea of using beginners as the
> standard to measure namespace usefulness against.
Definitely not. It's one criterion among many, and I would actually
rephrase it as "keep things as simple as possible", which is good not
only for beginners.
> 1) Prefer requires over uses.
An important point, but not one encouraged by the current ns syntax.
The shortest way to access another namespace, and the one I see most
often, is (:use namespace), and that's exactly what it worst in the
long run.
> 2) Private is your friend. Private should be the default for
> symbols, don't export what you don't need. As a side note, I wish
> there was a def- and defmacro- to go with defn-.
They are in clojure.contrib, but I agree they should be in core.
> 3) Don't fear the name spaces. There is very little cost to
> introducing a new namespace, so do it. If you have a foobar
> namespace, and some code only needs the foo part, and some code only
> needs the bar part, split it into
I don't quite agree there, as this quickly leads to cyclic
dependencies, which are forbidden.
> 4) One name space per file, and the file should have the same name.
> Also, every file should be it's own name space.
Nice as a principle, but not always practical. When you have a lot of
code and you can't factor it into reasonable namespaces because of
cyclic dependencies, breaking up a namespace into multiple files is
the lesser evil.
> Given these rules of thumb, I don't have the problems that Stephen
> seems to have. I don't have copy&paste problems with ns
> declarations, or name collisions, etc.
I do. Here is a combination that I need very frequently:
(ns ...
(:refer-clojure :exclude (+ - * / zero? pos? neg? > >= < <= min max))
(:use [clojure.contrib.generic.arithmetic
:only (+ - * /)]
[clojure.contrib.generic.comparison
:only (zero? pos? neg? > >= < <= min max)]
[clojure.contrib.generic.math-functions]))
That's what it takes to replace number-only arithmetic with generic
arithmetic. It's the use case that made me write nstools (http://code.google.com/p/clj-nstools/
)
Konrad.
Brian Hurt
I disagree here. A circular dependency means something is wrong with the structure of your code base. Split modules apart, combine them together, but most importantly, stop and think about what is going on, and what is depending upon what.
In extremis, there are some tricks you can do to break circular dependencies, all variations of passing the higher level function in for the lower level function to call. For example, let's say there are two functions, function f in name space foo, and function g in name space bar, that want to be circularly dependent. Well, you just pick one, let's say g, and change it so that it takes an extra argument- the f to call. Then when f calls g, it passes itself in as the f to call back.
But this is a code smell. 99% or more of the time, the correct answer is to refactor things so the circular dependency goes away. And, in my experience, the code ends up being better for it.
Brian
Brian Hurt
On Tue, Mar 23, 2010 at 3:00 PM, Constantine Vetoshev <gepa...@gmail.com> wrote:On Tue, Mar 23, 2010 at 5:08 PM, Brian Hurt <bhu...@gmail.com> wrote:
> There is very little cost to introducing a new namespace
Another "hidden cost" I've found comes from the need to carefully match namespaces to their filenames and directory structure. When I want to copy or rename directories or files to create a fork of my code for exploration, there is a lot of name-changing that has to occur.
This is what global search and replace is for. Or maybe :as.
(:requires [ some-module-ver2 :as some-module ])
Brian
Brian Hurt
I don't quite agree there, as this quickly leads to cyclic dependencies, which are forbidden.
3) Don't fear the name spaces. There is very little cost to introducing a new namespace, so do it. If you have a foobar namespace, and some code only needs the foo part, and some code only needs the bar part, split it into
The worst structure code can have is everything depends upon everything else- and thus to change anything, you have to change everything. This is the big ball of mud antipattern. Cyclical dependencies are how you get to the big ball of mud. A cyclic dependency means you have a "little ball of mud".
If you have two modules, foobar and bazquux, that are circularly dependent, the questions you should be asking yourself are:
1. Can I split foobar into two modules, foo and bar, where foo does not depend upon bazquux and bazquux does not depend upon bar?
2. Can I split bazquux similarly?
3. Should I maybe have four modules, instead of two?
4. Should I maybe make the modules foobaz and barquux?
etc.
The code base I'm working on has a fairly heavy amount of module churn, as I'm doing exploratory programming. New modules appear, old modules go away or get merged or split, modules radically change their responsibilities and dependencies, large hunks of code migrate around, etc. It is in this exact situation that keeping a clear concept of the structure of the code is most important.
Nice as a principle, but not always practical. When you have a lot of code and you can't factor it into reasonable namespaces because of cyclic dependencies, breaking up a namespace into multiple files is the lesser evil.
4) One name space per file, and the file should have the same name. Also, every file should be it's own name space.
The namespace/file mapping is completely orthogonal to the cyclical dependency problem. If you take all of your code and cat it into one big file, you haven't changed the program one whit.
I do. Here is a combination that I need very frequently:
Given these rules of thumb, I don't have the problems that Stephen seems to have. I don't have copy&paste problems with ns declarations, or name collisions, etc.
(ns ...
(:refer-clojure :exclude (+ - * / zero? pos? neg? > >= < <= min max))
(:use [clojure.contrib.generic.arithmetic
:only (+ - * /)]
[clojure.contrib.generic.comparison
:only (zero? pos? neg? > >= < <= min max)]
[clojure.contrib.generic.math-functions]))
That's what it takes to replace number-only arithmetic with generic arithmetic. It's the use case that made me write nstools (http://code.google.com/p/clj-nstools/)
I think I'd be lobbying to have generic arithmetic become the standard, rather than lobbying to change the name space handling.
Brian
Constantine Vetoshev
> The worst structure code can have is everything depends upon everything
> else- and thus to change anything, you have to change everything.
No one on this thread actually suggested allowing circular
dependencies. You originally said introducing namespaces is cheap.
Then, when Konrad and I brought up circular dependencies, you said
that refactoring may be necessary to resolve them. Therefore,
introducing namespaces is not necessarily cheap. That's all. Whether
or not this improves the code for long-term maintainability is beside
the point.
Brian Hurt
Then is fixing circular dependencies a cost of name spaces, or a cost of refactoring? More to the point, would introducing a change to allow name spaces to be circularly dependent actually help anything? Why add a feature to a language that you should never, under any circumstances, use?
Brian
Konrad Hinsen
> The worst structure code can have is everything depends upon everything else- and thus to change anything, you have to change everything. This is the big ball of mud antipattern. Cyclical dependencies are how you get to the big ball of mud. A cyclic dependency means you have a "little ball of mud".
I agree completely. I don't want cyclic dependencies either. I just brought them up to illustrate that breaking up a namespace into several smaller ones is not always a simple task.
>> Nice as a principle, but not always practical. When you have a lot of code and you can't factor it into reasonable namespaces because of cyclic dependencies, breaking up a namespace into multiple files is the lesser evil.
>>
>
> The namespace/file mapping is completely orthogonal to the cyclical dependency problem.
Unless you want to limit the maximum file size.
> I think I'd be lobbying to have generic arithmetic become the standard, rather than lobbying to change the name space handling.
I wouldn't want generic arithmetic to become the standard, as it has a serious performance cost. I would like to have the choice, and namespaces, with proper management, are a very good way to provide that choice. Moreover, I'd be surprised if the example I quoted were the only one where more flexible name space management is useful.
BTW, I don't necessarily want to change name space handling. Everything I would like to see can be added to the current functionality without breaking backwards compatibility.
Konrad.