simplifying the require/use situation

813 views
Skip to first unread message

Phil Hagelberg

unread,
Nov 16, 2011, 3:02:20 PM11/16/11
to cloju...@googlegroups.com
There's been discussion previously about the complexity of the ns
macro. In particular the fact that :use causes all vars to be referred
by default is widely seen as unfortunate, and the distinction between
use and require is a source of conceptual overhead for people new to
the language.

We can't change the fact that :use refers everything by default
without breaking lots of existing code. But it would be possible to
enhance :require to support referring specified vars, leaving us free
to deprecate or otherwise discourage the use of :use.

Rather than this form:

(ns mork.test
(:use [mork.stats :only [summarize-group]]
[mork.utils :only [strip resolve-fn]]
[clojure.test])
(:require [mork.view :as view]
[clojure.java.io :as io]
[cheshire.core :as json])
(:import (java.io PushbackReader))

We could use this:

(ns mork.test
(:require [mork.stats :refer [summarize-group]]
[mork.utils :refer [strip resolve-fn]]
[clojure.test :refer :all]
[mork.view :as view]
[clojure.java.io :as io]
[cheshire.core :as json])
(:import (java.io PushbackReader))

It has been agreed upon in previous threads that keeping :import as a
distinct concept from :require is desirable, and I agree that we
shouldn't conflate between Clojure vars and Java classes.

There are rare cases when referring all the vars in a namespace is
acceptable, (in particular when writing test namespaces it seems
reasonable to bring in all of clojure.test and the namespace under
test) so we should probably support :refer :all for such a case, as
long as it's not the default.

Previous discussion of the ns macro:
http://groups.google.com/group/clojure/msg/e8165da1ceebba9d

Thoughts? I've tried to make this as low-impact as possible, but I
think it could result in some desirable simplification.

-Phil

Kevin Downey

unread,
Nov 16, 2011, 3:08:22 PM11/16/11
to cloju...@googlegroups.com
as long as we are in there:

(ns foo.bar
(:require [x.y :as y]))

should do the same as

(ns foo.bar
(require [x.y :as y]))

and I think the second should be the preferred form.

> --
> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.
>
>

--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

Ben Smith-Mannschott

unread,
Nov 16, 2011, 3:46:15 PM11/16/11
to cloju...@googlegroups.com

I don't have anything concrete to add to this discussion yet (I'm
still pondering) except to say: YES, PLEASE.

I've been using Clojure for two years at least, and I still find
myself bouncing from the API docs for use via a detour at refer before
landing at require and puzzling through that wall of text without
examples. Maybe it's just my poor memory.

(Java's import statements, by contrast, are stupifyingly inexpressive
and verbose, but even a chimp could remember how they are meant to be
used.)

Sorry, guess I'm feeling a bit grumpy this evening. I'll get some
sleep. Perhaps I'll have some more constructive suggestions by
morning.

;; Ben

Rich Hickey

unread,
Nov 16, 2011, 3:50:41 PM11/16/11
to cloju...@googlegroups.com

On Nov 16, 2011, at 3:08 PM, Kevin Downey wrote:

> as long as we are in there:
>
> (ns foo.bar
> (:require [x.y :as y]))
>
> should do the same as
>
> (ns foo.bar
> (require [x.y :as y]))
>
> and I think the second should be the preferred form.
>
>

Should because you say so, or is there reasoning for and against?

Please, please people, stop the polling/voting. Make reasoned arguments and make sure you identify the tradeoffs associated with your favored positions.

Thanks,

Rich

Alan Malloy

unread,
Nov 16, 2011, 4:05:09 PM11/16/11
to Clojure Dev
On Nov 16, 12:08 pm, Kevin Downey <redc...@gmail.com> wrote:
> as long as we are in there:
>
> (ns foo.bar
>   (:require [x.y :as y]))
>
> should do the same as
>
> (ns foo.bar
>   (require [x.y :as y]))

It already does. Whether it should be preferred is another matter.

Kevin Downey

unread,
Nov 16, 2011, 4:07:36 PM11/16/11
to cloju...@googlegroups.com

I've seen code that makes that mistake fairly often: require in the ns
form instead of :require.

It is also a point of confusion for people just learning the language.
Another point of confusion is quoting with (require …) outside of the
ns form vs. (:require …) inside, so I've intact seen various
combinations of quoting and keywords vs. quoting and symbols vs. no
quoting at all, etc.

The keyword vs. symbol issue is simple enough, no reason to quickly
remove support for the keyword, just add support for the symbol.

The quoting issue is insidious. It is possible that making require in
the ns form like more like require outside that more confusion will
arise, but it will eliminate a dimension of confusion. Instead of
confusion of magnitude (symbol vs. keyword) * (quoting vs. no quoting)
it is only of magnitude (quoting vs. no quoting)

Alan Malloy

unread,
Nov 16, 2011, 4:08:51 PM11/16/11
to Clojure Dev
One other thing you agreed was a good idea when we discussed it at the
conj but haven't included here; I'm adding in case you forgot. It
would be nice to have import/:as, for example

(ns foo.bar (:import [java.util :as u]))
...
(u.LinkedList.)

It creates a nice symmetry with require, without conflating the two,
and allows whole packages to be "mentioned" in the same way that
require brings in whole namespaces.

Phil Hagelberg

unread,
Nov 16, 2011, 4:19:33 PM11/16/11
to cloju...@googlegroups.com
On Wed, Nov 16, 2011 at 1:08 PM, Alan Malloy <al...@malloys.org> wrote:
> Thoughts? I've tried to make this as low-impact as possible, but I
> think it could result in some desirable simplification.

While I appreciate the fact that there are various other ways in which
the ns macro could be simplified and/or improved, I am proposing a
very small, very specific change and would like to hear feedback on
that.

Perhaps "general ns brainstorming" could be spun off into its own
thread and/or wiki page?

-Phil

Chouser

unread,
Nov 16, 2011, 4:46:19 PM11/16/11
to cloju...@googlegroups.com
On Wed, Nov 16, 2011 at 3:02 PM, Phil Hagelberg <ph...@hagelb.org> wrote:
> There's been discussion previously about the complexity of the ns
> macro. In particular the fact that :use causes all vars to be referred
> by default is widely seen as unfortunate, and the distinction between
> use and require is a source of conceptual overhead for people new to
> the language.

Thanks for picking this up again.

> We could use this:
>
>    (ns mork.test
>      (:require [mork.stats :refer [summarize-group]]
>                [mork.utils :refer [strip resolve-fn]]
>                [clojure.test :refer :all]
>                [mork.view :as view]
>                [clojure.java.io :as io]
>                [cheshire.core :as json])
>      (:import (java.io PushbackReader))

This being purely additive means it is not a breaking change, which
should ease adoption.

I'm in favor of adding the ability to refer vars from the require
function because it would allow the use of one top-level feature
instead of two (require and use). In this case it is not a matter of
complecting two simple things because use already has access to all
the features of require, just with poorer defaults. Giving require
these features may allow use to fall into dis-use (haha) and perhaps
be deprecated. This would result in a simpler system.

The refer function has :exclude, :only, and :rename. It seems that
:refer in your proposal maps to :only. I've never needed :exclude, so
don't mind if that's missing. But I would hope :rename would be
available via require, perhaps as another keyword arg?

(ns mork.test
(:require [mork.stats :refer [summarize-group] :rename
{summarize-group sum-grp}]))

It may be helpful to avoid adding other suggestions to this proposal.
There are plenty of things that can be improved (adding :as to import,
etc.), and I suggest we take each of these on its own unless the
design of one depends on the design of another, which does not seem to
be the case yet. If :rename falls into this category, please say so
and I'll re-introduce that proposal when this one is complete (one way
or the other).

--Chouser

Rich Hickey

unread,
Nov 16, 2011, 4:55:10 PM11/16/11
to cloju...@googlegroups.com

Sounds good. A piled-on proposal is less likely to get in.


Phil Hagelberg

unread,
Nov 16, 2011, 5:11:57 PM11/16/11
to cloju...@googlegroups.com
On Wed, Nov 16, 2011 at 1:46 PM, Chouser <cho...@gmail.com> wrote:
> The refer function has :exclude, :only, and :rename.  It seems that
> :refer in your proposal maps to :only.  I've never needed :exclude, so
> don't mind if that's missing.  But I would hope :rename would be
> available via require, perhaps as another keyword arg?
>
>    (ns mork.test
>      (:require [mork.stats :refer [summarize-group] :rename
> {summarize-group sum-grp}]))

I omitted mentioning :rename because I have never seen it used in
practice (I believe :as is preferable when there are conflicts since
it's clearer) but if people are attached to :rename that may make it
difficult to deprecate :use if :require doesn't support it.

> It may be helpful to avoid adding other suggestions to this proposal.
> There are plenty of things that can be improved (adding :as to import,
> etc.), and I suggest we take each of these on its own unless the
> design of one depends on the design of another, which does not seem to
> be the case yet.  If :rename falls into this category, please say so
> and I'll re-introduce that proposal when this one is complete (one way
> or the other).

The essence of this proposal is "turn :require into a full replacement
for :use", so I don't think it's out of scope. It's just a question of
whether :rename is a prudent way of handling conflicts; personally I
am not keen on it but could be convinced otherwise.

On the other hand :exclude is just useful for the :use without :only
case, so it's probably safe to drop.

-Phil

Lee Hinman

unread,
Nov 16, 2011, 5:14:25 PM11/16/11
to cloju...@googlegroups.com
On Wednesday, November 16, 2011 at 3:11 PM, Phil Hagelberg wrote:
> The essence of this proposal is "turn :require into a full replacement
> for :use", so I don't think it's out of scope. It's just a question of
> whether :rename is a prudent way of handling conflicts; personally I
> am not keen on it but could be convinced otherwise.
>
> On the other hand :exclude is just useful for the :use without :only
> case, so it's probably safe to drop.
>
> -Phil

+1, I agree with Phil's original proposal of turning :require into a full replacement of :use.

- Lee Hinman

Chouser

unread,
Nov 16, 2011, 5:19:22 PM11/16/11
to cloju...@googlegroups.com
On Wed, Nov 16, 2011 at 5:11 PM, Phil Hagelberg <ph...@hagelb.org> wrote:
>
> I omitted mentioning :rename because I have never seen it used in
> practice (I believe :as is preferable when there are conflicts since
> it's clearer) but if people are attached to :rename that may make it
> difficult to deprecate :use if :require doesn't support it.

I admit to using :rename rarely in the past, but that may change.
I've started working on a proposal for a slight change (fully
qualified local names) which I'll be bringing up later. With that in
place, :rename could become very useful for managing a small number of
dependencies on large multi-namespace libraries.

> On the other hand :exclude is just useful for the :use without :only
> case, so it's probably safe to drop.

The only place I've seen :exclude used with any amount of regularity
is in :refer-clojure, but I think this still leads to source code that
is more confusing than necessary, since it means things that are
usually from clojure core are instead from somewhere else. Since this
proposal doesn't touch :refer-clojure at all, maybe we can ignore this
point.

--Chouser

Justin Balthrop

unread,
Nov 16, 2011, 7:40:54 PM11/16/11
to cloju...@googlegroups.com

It seems to me that :rename and :exclude are both advanced functionality and having to use (refer ...) in this case is not unreasonable.

Instead of:

(ns mork.test
  (:require [mork.stats :refer [summarize-group] :rename {summarize-group sum-grp}]
            [mork.utils :exclude [strip]]))

You would just do:

(ns mork.test
  (:require mork.stats mork.utils)
  (:refer [mork.stats :only [summarize-group] :rename {summarize-group sum-grp}]
          [mork.stats :exclude [strip]]))

This also leaves open the option to deprecate them in the future.


Cosmin Stejerean

unread,
Nov 16, 2011, 9:38:41 PM11/16/11
to cloju...@googlegroups.com
On Thu, Nov 17, 2011 at 9:11 AM, Phil Hagelberg <ph...@hagelb.org> wrote:
> On Wed, Nov 16, 2011 at 1:46 PM, Chouser <cho...@gmail.com> wrote:
>> The refer function has :exclude, :only, and :rename.  It seems that
>> :refer in your proposal maps to :only.  I've never needed :exclude, so
>> don't mind if that's missing.  But I would hope :rename would be
>> available via require, perhaps as another keyword arg?
>>
>>    (ns mork.test
>>      (:require [mork.stats :refer [summarize-group] :rename
>> {summarize-group sum-grp}]))

In order to have :require be a replacement for :use we probably need
to support rename, but this particular way of renaming requires
repeating each name twice, which feels unnecessary. How about
something like this

(ns mork.test
(:require [mork.stats :refer-as {summarize-group sum-grp}]))


This would avoid having to repeat summarize-group twice, and feels
much cleaner and easier to remember to me.


--
Cosmin Stejerean
http://offbytwo.com

Phil Hagelberg

unread,
Nov 16, 2011, 10:10:47 PM11/16/11
to cloju...@googlegroups.com
On Wed, Nov 16, 2011 at 6:38 PM, Cosmin Stejerean <cos...@offbytwo.com> wrote:
> In order to have :require be a replacement for :use we probably need
> to support rename, but this particular way of renaming requires
> repeating each name twice, which feels unnecessary.

Can you explain why you prefer to refer and rename individual
conflicting vars rather than simply aliasing the namespace with
:require :as? The latter just seems so much simpler since you can use
the var's actual name rather than inventing a new thing to call it.

-Phil

Cosmin Stejerean

unread,
Nov 16, 2011, 10:44:45 PM11/16/11
to cloju...@googlegroups.com
On Thu, Nov 17, 2011 at 2:10 PM, Phil Hagelberg <ph...@hagelb.org> wrote:
> Can you explain why you prefer to refer and rename individual
> conflicting vars rather than simply aliasing the namespace with
> :require :as? The latter just seems so much simpler since you can use
> the var's actual name rather than inventing a new thing to call it.

Well, in my case it's partially because I'm used to doing this in
Python with "from foo import bar as baz". Now "it works this way in
Python" is not a valid reason. This behavior however comes in handy
when I don't like the actual name of the var and I would prefer to
call it something different. For example the name might be too long,
or not descriptive enough for my context, etc (preventing conflicts is
not the only reason to rename).


Most of the time :as would probably be preferred to renaming, but even
then in order to make it easier to deprecate :use we would need some
form of support for it. If we're going to add support for it, I
believe something like ":refer-as {original new}" is shorter, cleaner
and easier to parse than ":refer [original] :rename {original new}".

It does prevent something like ":refer [first second] :rename [second
new-second]" but I would prefer that we don't support this particular
use case. It seems much cleaner to separate the things being refered
by their original names from the things being renamed into two
separate :require lines.

Chouser

unread,
Nov 16, 2011, 11:40:37 PM11/16/11
to cloju...@googlegroups.com
On Wed, Nov 16, 2011 at 7:40 PM, Justin Balthrop
<jus...@justinbalthrop.com> wrote:
> It seems to me that :rename and :exclude are both advanced functionality and
> having to use (refer ...) in this case is not unreasonable.
>
> Instead of:
>
> (ns mork.test
> (:require [mork.stats :refer [summarize-group] :rename {summarize-group
> sum-grp}]
> [mork.utils :exclude [strip]]))
>
> You would just do:
>
> (ns mork.test
> (:require mork.stats mork.utils)
> (:refer [mork.stats :only [summarize-group] :rename {summarize-group
> sum-grp}]
> [mork.stats :exclude [strip]]))

Nice! This is undocumented, but it works. I think it's a great idea.
There's no harm in requiring something first and then refering and/or
renaming the vars later, other than having to repeat the namespace
name. This repetition may or may not be worth while for the :only
case, but it's probably fine for :rename.

Also note how this makes both :require and :refer simpler -- the first
doesn't have to be able to :rename, and the second doesn't have to be
able to require (or :as, etc.)

I'm withdrawing my request for :require to support :rename -- we can
deal with that (including possibly more convenient syntax) at some
other time.

--Chouser

Justin Balthrop

unread,
Nov 17, 2011, 12:52:49 AM11/17/11
to cloju...@googlegroups.com
Cosmin,

I agree that your syntax for :rename is way better. If renaming is going to stay around long term, a simple way to transition would be to make the existing :rename refer the from symbol if it isn't already referred. So you could just do:

(ns mork.test
(:require [mork.stats :rename {summarize-group sum-grp}]))

This would be completely backward compatible. But, this change should probably be a separate proposal.

-Justin

Justin Balthrop

unread,
Nov 17, 2011, 12:58:45 AM11/17/11
to cloju...@googlegroups.com
In the context of my other suggestion, the example should be:

(ns mork.test
(:require mork.stats)
(:refer [mork.stats :rename {summarize-group sum-grp}]))

Stuart Sierra

unread,
Nov 17, 2011, 5:01:21 PM11/17/11
to cloju...@googlegroups.com
I have made fairly exhaustive notes on this topic:
http://dev.clojure.org/display/design/Loading%2C+Compiling%2C+and+Namespaces

-S

Phil Hagelberg

unread,
Nov 17, 2011, 5:09:29 PM11/17/11
to cloju...@googlegroups.com
On Wed, Nov 16, 2011 at 9:52 PM, Justin Balthrop
<jus...@justinbalthrop.com> wrote:
> (ns mork.test
>   (:require [mork.stats :rename {summarize-group sum-grp}]))
>
> This would be completely backward compatible. But, this change should probably be a separate proposal.

Considering that a separate rename-supporting :refer exists and that
there is not consensus that :rename is worth canonizing in this way, I
created a ticket for simply adding :refer to :require. I'll try to put
together a patch soon.

http://dev.clojure.org/jira/browse/CLJ-879

-Phil

Rich Hickey

unread,
Nov 18, 2011, 7:49:57 AM11/18/11
to cloju...@googlegroups.com

Great - thanks!

Rich

Phil Hagelberg

unread,
Nov 23, 2011, 6:45:54 PM11/23/11
to cloju...@googlegroups.com
On Thu, Nov 17, 2011 at 2:09 PM, Phil Hagelberg <ph...@hagelb.org> wrote:
> http://dev.clojure.org/jira/browse/CLJ-879

I'm putting together a patch for this. I have something that allows
:refer to be added to require calls, but it also causes :refer to work
from use calls as well since use is implemented in terms of require.

I could add in some logic to cause it to not work in use calls, but I
think it would introduce unnecessary complexity, especially
considering use may end up deprecated. This wouldn't be the first case
of functionality intended for require being accessible via use as the
:as clause is already this way. So I'm inclined to let it be.

Does that seem sound?

-Phil

Stuart Halloway

unread,
Nov 24, 2011, 5:36:04 PM11/24/11
to cloju...@googlegroups.com

Sounds good to me. We should not document the 'use capability, or perhaps even specifically document it as not supported.

Stu

Phil Hagelberg

unread,
Dec 15, 2011, 8:23:39 PM12/15/11
to cloju...@googlegroups.com
On Wed, Nov 23, 2011 at 3:45 PM, Phil Hagelberg <ph...@hagelb.org> wrote:
> On Thu, Nov 17, 2011 at 2:09 PM, Phil Hagelberg <ph...@hagelb.org> wrote:
>> http://dev.clojure.org/jira/browse/CLJ-879
>
> I'm putting together a patch for this. I have something that allows
> :refer to be added to require calls, but it also causes :refer to work
> from use calls as well since use is implemented in terms of require.

Uploaded the patch a while ago, but I thought I'd mention it here so
it doesn't get overlooked.

-Phil

Reply all
Reply to author
Forward
0 new messages