Is there a reason that def- isn't part of the core lib?

509 views
Skip to first unread message

Rob Lally

unread,
Sep 17, 2011, 10:54:08 AM9/17/11
to clo...@googlegroups.com
Hi all,

Whilst trying to minimise the visible surface areas of namespaces, I've often felt the need for a def- function/macro that marks a def'ed var with :private metadata. An analog of defn-, if you will.

Is there a reason that I shouldn't do this or a reason that it doesn't seem to be a member of the core lib?

In the spirit of Hunt & Thomas' "Select isn't broken", when I encounter something this trivial, useful and absent I tend to conclude that I'm the one who's got things wrong.


Thanks in advance,


Rob.

Mark Rathwell

unread,
Sep 17, 2011, 11:37:42 AM9/17/11
to clo...@googlegroups.com
A previous discussion on the topic can be found here [1]. You can
easily add the private metadata yourself:

Clojure 1.2: (def ^{:private true} size 25)
Clojure 1.3: (def ^:private size 25)

I think probably the reason against it is that generally there is not
as much reason to use a constant, for example, outside of its
namespace as there is with functions, and therefore more need for
marking functions meant only for internal use as such (but that is
just guessing at other peoples' thought processes).

[1] http://groups.google.com/group/clojure/browse_thread/thread/80d873ef625221ac/243a0ad490150d3e?lnk=gst&q=def-#243a0ad490150d3e

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

Armando Blancas

unread,
Sep 17, 2011, 5:58:44 PM9/17/11
to Clojure
There are other nine defs in core that don't have a dash version,
either. I guess if they were to take def- they'd have to add the
others and then add any new defs in pairs. But there's no reason to
write ugly code; just write your own or use a contrib, if available.

Mark Engelberg

unread,
Sep 18, 2011, 2:59:48 AM9/18/11
to clo...@googlegroups.com
Count me among those who would like to see at least def- added to core, and possibly private versions of the other types of def as well.

Alan Malloy

unread,
Sep 18, 2011, 3:10:29 AM9/18/11
to Clojure
I would rather see defn- removed, than add any more foo- functions. If
there are any more function "attributes" we want to tag things with
(say, automatically memoized?) that would be 2^N new "convenient"
definition forms, in an already-packed clojure.core namespace. And
it's not hard at all to (let [foo-const 1] (defn foofn [x] ...) (defn
more-foo [y z] ...)), or use ^:private if you really want a private
global.

Meikel Brandmeyer

unread,
Sep 18, 2011, 4:12:01 AM9/18/11
to clo...@googlegroups.com
> I would rather see defn- removed, than add any more foo- functions.

+1


Rob Lally

unread,
Sep 18, 2011, 5:14:39 AM9/18/11
to clo...@googlegroups.com
Thanks for the reference Mark, much appreciated.

I add the private meta-data by hand, but (defn- foo 2) is so much more succinct than (def ^:private foo 2) - where ^:private is almost half the declaration. I also find the placement of the meta-data tag to be intrusive and disruptive when reading... but that's just a personal peccadillo.

I find your comments that you don't feel as much need to mark defs as private as often as defns really interesting, because it is quite different to my own practice. I tend to look at ^:private primarily as a way to inform users of a namespace that this def is something they can safely ignore because it is an implementation detail; you only need to pay attention to this if you're going to modify the code.

I've spent 5 mins looking at random source files from a few clojure projects and I seem to use more defs than most people. I tend to pull out a fairly high percentage of literals into expanatory defs, whereas the small sample of files that I trolled through seemed to have a different feel for the tipping point when a def is appropriate.

I am surprised at other comments in this chain where people would rather see the removal of def- than the addition of additional variants of def derivatives. I'm not arguing that I'm right, or that things should change, but personally I feel that controlling the exposed surface area of an API is crucial to comprehensibility and maintainability, and that making minimisation of surface area as trivial as possible will benefit us all.

Thanks to everyone for taking the time to write down their thoughts,


R.

Colin Yates

unread,
Sep 18, 2011, 5:36:53 AM9/18/11
to clo...@googlegroups.com
+1 for easily controlling the surface area of an API and explanatory
defs over inline/expanded literals.

Sent from my iPad

Chas Emerick

unread,
Sep 18, 2011, 5:57:06 AM9/18/11
to clo...@googlegroups.com
Just to pile on with Alan here: defn- is the oddball here, a vestige of a time when metadata on vars was more cumbersome and less common. Adding a def- would cover one very particular case; to make the point rhetorically, what def* form would be required to produce (for example):

(def ^:private ^:dynamic foo …)

I'm guessing we haven't seen the last of the ^:bar sort of var metadata.

- Chas

Brian Marick

unread,
Sep 18, 2011, 6:16:02 AM9/18/11
to clo...@googlegroups.com

>> I would rather see defn- removed, than add any more foo- functions.


We will see this subject line in this mailing list every few months until the end of time, or until def- is added to core.

The burden, no matter how distributed, of answering that question forevermore greatly outweighs the burden of moving a def- macro from a contrib file to a core file. It even outweighs the aesthetic concerns (which I confess I don't understand).

-----
Brian Marick, Artisanal Labrador
Contract programming in Ruby and Clojure
Occasional consulting on Agile
www.exampler.com, www.twitter.com/marick

Stuart Halloway

unread,
Sep 18, 2011, 9:38:23 AM9/18/11
to clo...@googlegroups.com
> We will see this subject line in this mailing list every few months until the end of time, or until def- is added to core.

So be it.

> The burden, no matter how distributed, of answering that question forevermore greatly outweighs the burden of moving a def- macro from a contrib file to a core file. It even outweighs the aesthetic concerns (which I confess I don't understand).

Alan covered the biggest problem succinctly: it pollutes the language core to provide convenience APIs that are combinatorial in nature.

I don't really think the people that want def- need it in the language core. They just want to know where to get it, and maybe automatically include it in their own projects. So make a community-centralized, maven-ready, CI-friendly place to put such things. In fact: such a place already exists: https://github.com/clojure/core.incubator. Add a slew of custom def forms to incubator.clj and party on.

Stu

Brian Marick

unread,
Sep 18, 2011, 10:34:15 AM9/18/11
to clo...@googlegroups.com

On Sep 18, 2011, at 3:38 PM, Stuart Halloway wrote:
>> The burden, no matter how distributed, of answering that question forevermore greatly outweighs the burden of moving a def- macro from a contrib file to a core file. It even outweighs the aesthetic concerns (which I confess I don't understand).
>
> Alan covered the biggest problem succinctly: it pollutes the language core to provide convenience APIs that are combinatorial in nature.

I mean this in the friendliest and most supportive possible way, but since Justin Gehtland recently called me an iconoclast, I am obliged:

The API already contains `find` and `contains?`, which are woefully misleading names. That's not a big deal: I'm *terrible* at naming. I tell people that all the time. I've learned to accept it. It doesn't detract from my wonderfulness. It doesn't detract from the underlying wonderfulness of the software I produce. It just means I have to rethink and revise names, especially after other people use them and say "WTF??".

It's a shame it's too late to do that for `find` and `contains?`. Fortunately, additions are easier than changes.

I hope `def-` hasn't become an issue of pride, a matter of digging in. I sense it has. I hope `def` is not a matter of saying "But if we make this one exception, we have no principled reason to reject any other exception." That's not how human negotiation and communication works.

I really think `def-` would be a good gesture, a minor but emblematic step on the way to widespread acceptance of Clojure, which (for my sake) I really, really, really hope for.

Stuart Halloway

unread,
Sep 18, 2011, 11:44:28 AM9/18/11
to clo...@googlegroups.com
> The API already contains `find` and `contains?`, which are woefully misleading names. That's not a big deal: I'm *terrible* at naming. I tell people that all the time. I've learned to accept it. It doesn't detract from my wonderfulness. It doesn't detract from the underlying wonderfulness of the software I produce. It just means I have to rethink and revise names, especially after other people use them and say "WTF??".
>
> It's a shame it's too late to do that for `find` and `contains?`. Fortunately, additions are easier than changes.

This argument supports my position, not yours. It is much easier to revise names in an incubator library, rather than experimenting in core.

> I really think `def-` would be a good gesture, a minor but emblematic step on the way to widespread acceptance of Clojure, which (for my sake) I really, really, really hope for.

I don't understand how a casual approach to adding things to core would be a good gesture to anybody. Can you send me a list of languages that add features to core after this level of discussion (and without clear agreement) because some people find them useful? I am not aware of any.

To restate my earlier point: "Why isn't this in core?" and "Why isn't this conveniently available to me?" are very different questions. If you have a problem that can be trivially solved outside of core, why are you in such a hurry to change core? Making unnecessary changes at the bottom strikes me as bad design and bad stewardship.

Here's a thought experiment: There is a ton of useful stuff in my .emacs file. I can think of a few dozen things in my .emacs file that are in the "def-" category for me, e.g. I like them, and I want them around always, and immediately when I launch. So those things are core to me. What do you think would happen if I joined the emacs dev list, picked my very favorite convenience, and said "please add this to emacs core?" Emacs doesn't work that way. It is modular, and people solve problems in *libraries*. Sometimes large groups of people agree about certain things. Even then you often get awesome tools like the emacs starter kit *without* changes to emacs core.

The clojure.core namespace evolves at the speed of design, not at the speed of itch-scratching. I think this is a fundamental value.

Stu


Mark Engelberg

unread,
Sep 18, 2011, 12:47:10 PM9/18/11
to clo...@googlegroups.com
On Sun, Sep 18, 2011 at 6:38 AM, Stuart Halloway <stuart....@gmail.com> wrote:
Alan covered the biggest problem succinctly: it pollutes the language core to provide convenience APIs that are combinatorial in nature.


I don't understand your comment about "polluting" the language core.  Do you really think people are going to use def- for some other purpose?  If you don't, then it is not pollution.

I think the big issue here is that certain functions in Clojure core *imply* the existence of other certain functions in the core.  When they don't exist, it comes as a surprise.  Surprise is bad.

defn- implies the existence of def-

The other example that immediately leaps to mind is that the family of get-in, get, and update-in implies the existence of update.  It is rather startling to me that update does not exist in the core.
 

Stuart Halloway

unread,
Sep 18, 2011, 1:10:20 PM9/18/11
to clo...@googlegroups.com
> I don't understand your comment about "polluting" the language core. Do you really think people are going to use def- for some other purpose? If you don't, then it is not pollution.

Fair enough. Maybe pollution wasn't the best word. Introducing a combinatorial set of names is a [some other word for bad thing] for core, even if we agree what the names mean.

> I think the big issue here is

I think that the big issue here is that we do not agree on how careful the dev team should be about adding things to core. I think we should be quite careful. The name is "core" not "kitchen-sink". If anything, core is already too big.

> that certain functions in Clojure core *imply* the existence of other certain functions in the core. When they don't exist, it comes as a surprise. Surprise is bad.

Agreed, but this is how you argue for a complete set, not for a convenient subset. No one seems to be asking for defmacro-, even though core itself defines a private macro.

> defn- implies the existence of def-

Then let us deprecate defn-.

> The other example that immediately leaps to mind is that the family of get-in, get, and update-in implies the existence of update. It is rather startling to me that update does not exist in the core.

This is a good question. I don't know why I never noticed its absence. Have other people missed this?

Stu

Chas Emerick

unread,
Sep 18, 2011, 1:44:59 PM9/18/11
to clo...@googlegroups.com

On Sep 18, 2011, at 1:10 PM, Stuart Halloway wrote:

>> The other example that immediately leaps to mind is that the family of get-in, get, and update-in implies the existence of update. It is rather startling to me that update does not exist in the core.
>
> This is a good question. I don't know why I never noticed its absence. Have other people missed this?

What would an `update` function do? update-in is for swapping out a value accessible via a particular "chain" of keys. Dropping the -in suffix implies to me that `update` would operate on the top-level data structure, but that's what assoc/dissoc/conj do already.

- Chas

Mark Engelberg

unread,
Sep 18, 2011, 2:45:47 PM9/18/11
to clo...@googlegroups.com
@Chas, update uses a function to perform the update.  Think of it as being like swap! and alter, but for persistent associative structures.  So you can write something like: (update {:foo 0} :foo inc) to increment the value of :foo. 

update is to update-in as assoc is to assoc-in.

If you write a lot of code that uses swap! or alter, the absence of something similar for persistent associative structures becomes more conspicuous.  And the fact that such a function (update-in) exists for nested associative structures makes its absence even odder.

@Stuart, yes I'd estimate this has come up on the lists approx. 3 times per year over the past couple of years.

Unfortunately, the word "update" is sufficiently common that it makes it difficult to search the groups for this discussion topic.

But I was able to find a discussion of this as far back as April 2009:
http://groups.google.com/group/clojure/browse_thread/thread/4eb34e04a8d4fe6b/3017a1502a3a04a8

I have seen people define update in their own code, but more often than not I see people do something like
(update-in map [k] f)
because they don't want to bother defining update themselves.  I personally think it would be more elegant to put the implied "update" function directly into the core.

Alan Malloy

unread,
Sep 18, 2011, 4:32:55 PM9/18/11
to Clojure
Agreed that this is why it doesn't exist. Personally I wouldn't mind
having update (and have asked for it before), but you have to decide
what to use the varargs for: extra arguments to the update function,
or a set of top-level keys? That is, given m={:a 1, :b 2}:

(update m :a :b inc) => {:a 2, :b 3}
--OR--
(update m :a + 2) => {:a 3, :b 2}

Alan Malloy

unread,
Sep 18, 2011, 5:08:34 PM9/18/11
to Clojure
What really puzzles me is that it doesn't seem to be generally
regarded as idiomatic Clojure style to just use top-level (let)s for
your "private" globals. This has lots of benefits:

- If you do this you can make them actually, genuinely private, rather
than just marked as "please don't use this"
- You get more self-documenting source code: (let [x 1] (defn foo []
(...use x...))) is clear about how and where x will be used, while
(def- x 1) (defn foo [] (...use x...)) leaves open the possibility
that x is important all over the namespace.
- You avoid runtime var-deref costs for constants that will never
change.

I find this style so useful and readable that I'm curious why it isn't
more popular in the community at large.

On Sep 18, 2:14 am, Rob Lally <rob.la...@gmail.com> wrote:
> Thanks for the reference Mark, much appreciated.
>
> I add the private meta-data by hand, but (defn- foo 2) is so much more succinct than (def ^:private foo 2) - where ^:private is almost half  the declaration. I also find the placement of the meta-data tag to be intrusive and disruptive when reading... but that's just a personal peccadillo.
>
> I find your comments that you don't feel as much need to mark defs as private as often as defns really interesting, because it is quite different to my own practice. I tend to look at ^:private primarily as a way to inform users of a namespace that this def is something they can safely ignore because it is an implementation detail; you only need to pay attention to this if you're going to modify the code.
>
> I've spent 5 mins looking at random source files from a few clojure projects and I seem to use more defs than most people. I tend to pull out a fairly high percentage of literals into expanatory defs, whereas the small sample of files that I trolled through seemed to have a different feel for the tipping point when a def is appropriate.
>
> I am surprised at other comments in this chain where people would rather see the removal of def- than the addition of additional variants of def derivatives. I'm not arguing that I'm right, or that things should change, but personally I feel that controlling the exposed surface area of an API is crucial to comprehensibility and maintainability, and that making minimisation of surface area as trivial as possible will benefit us all.
>
> Thanks to everyone for taking the time to write down their thoughts,
>
> R.
>
> On 17 Sep 2011, at 16:37, Mark Rathwell wrote:
>
> > A previous discussion on the topic can be found here [1].  You can
> > easily add the private metadata yourself:
>
> > Clojure 1.2:  (def ^{:private true} size 25)
> > Clojure 1.3:  (def ^:private size 25)
>
> > I think probably the reason against it is that generally there is not
> > as much reason to use a constant, for example, outside of its
> > namespace as there is with functions, and therefore more need for
> > marking functions meant only for internal use as such (but that is
> > just guessing at other peoples' thought processes).
>
> > [1]http://groups.google.com/group/clojure/browse_thread/thread/80d873ef6...

Rob Lally

unread,
Sep 18, 2011, 6:11:53 PM9/18/11
to clo...@googlegroups.com
Personally, I don't use this particular style for.. probably pretty poor reasons. I find that I end up declaring every constant that might be needed in a file in a single let at the top of the file because:

a) Clojure's declare before use semantics make ordering technically important rather than simply a matter of aesthetic or conceptual grouping.
b) If I have several constants, trying to scope them only to the functions that use them isn't easy/always possible.
c) I have assumed ( based on no evidence ) that Clojure would be lazy enough to defer initialisation of def'd items until they were used, but a top level let would always have to be executed at file load time.
d) I find that indenting almost the whole file because it is inside the let.. displeasing. But not indenting it is worse. I'm a bit neurotic about layout.

if you think it is a good practice, I might well have another try at it and see if I can make peace with it.


R.

Luc Prefontaine

unread,
Sep 18, 2011, 6:12:02 PM9/18/11
to clo...@googlegroups.com
ha ! ha !

I really like the comparison core/kitchen sink :)))
I find it appropriate.

It never really bothered me to require an external lib to get these
fancy definitions. And I use them every where ....

Core should restricted to ... core things. What makes Clojure run by itself.
Syntactic sugar needs to hosted elsewhere. def- and similar are just that,
syntactic sugar. They're not needed to get Clojure up and running.

And yes, defn- should be located elsewhere than in core. +1 for moving it
out of core, at least, defn- should not be make publicly available by core.

By the time we get Clojure written in itself, the core has to be cleaned up of
non-essential things and moved to layers were implementation differences have less/no impacts
while helping shrink it.

Luc P.

--
Luc P.

================
The rabid Muppet

Stuart Halloway

unread,
Sep 18, 2011, 8:29:34 PM9/18/11
to clo...@googlegroups.com
What really puzzles me is that it doesn't seem to be generally
regarded as idiomatic Clojure style to just use top-level (let)s for
your "private" globals. This has lots of benefits:

- If you do this you can make them actually, genuinely private, rather
than just marked as "please don't use this"
- You get more self-documenting source code: (let [x 1] (defn foo []
(...use x...))) is clear about how and where x will be used, while
(def- x 1) (defn foo [] (...use x...)) leaves open the possibility
that x is important all over the namespace.
- You avoid runtime var-deref costs for constants that will never
change.

I find this style so useful and readable that I'm curious why it isn't
more popular in the community at large.

Top-level lets make it more difficult for tools to statically analyze code.

Of course said tools are largely hypothetical at present. :-)

Stu

Stuart Halloway
Clojure/core
http://clojure.com

Ken Wesson

unread,
Sep 19, 2011, 8:51:04 AM9/19/11
to clo...@googlegroups.com
On Sun, Sep 18, 2011 at 6:12 PM, Luc Prefontaine
<lprefo...@softaddicts.ca> wrote:
> And yes, defn- should be located elsewhere than in core. +1 for moving it
> out of core, at least, defn- should not be make publicly available by core.

I'm of more or less the opposite appearance: anything that core itself
uses, and that therefore already exists in core, and that is likely to
be useful to a significant number of users, ought to be public.
Otherwise it will end up wastefully duplicated in core and at least
one third-party library, and there might even be a lot of
wheel-reinvention -- with some of them probably turning out square.

--
Protege: What is this seething mass of parentheses?!
Master: Your father's Lisp REPL. This is the language of a true
hacker. Not as clumsy or random as C++; a language for a more
civilized age.

Colin Yates

unread,
Sep 19, 2011, 9:04:13 AM9/19/11
to clo...@googlegroups.com
I also think it is worth remembering the difficulty people have with getting started with Clojure (wrt to IDEs).  There are *so* many third party libraries out there at the moment that a beginner could easily be overwhelmed.  The more self contained and feature complete the core is the better.

Fine line though...and one I am glad I am not in charge of.

Col

Stuart Halloway

unread,
Sep 19, 2011, 10:33:27 AM9/19/11
to clo...@googlegroups.com
> I also think it is worth remembering the difficulty people have with getting started with Clojure (wrt to IDEs). There are *so* many third party libraries out there at the moment that a beginner could easily be overwhelmed. The more self contained and feature complete the core is the better.

A self contained complete starter env (e.g. including all the modular contribs) is a good idea. It should not be core.

Stu

Geoff Wilson

unread,
Sep 19, 2011, 10:39:47 AM9/19/11
to clo...@googlegroups.com
Wouldn't a set of standard entries in a lein project file be sufficient for a beginner?

I’m not sure that Clojure needs something as packaged as the Haskell Platform (http://hackage.haskell.org/platform/), and it certainly should not be part of core.

Stephen Compall

unread,
Sep 19, 2011, 10:00:23 PM9/19/11
to clo...@googlegroups.com
On Sun, 2011-09-18 at 13:10 -0400, Stuart Halloway wrote:
> This is a good question. I don't know why I never noticed [update's]

> absence. Have other people missed this?

I would say about half the calls to update-in I've written took literal
vectors with one expression in them. Including the two earlier today.
While I noticed `update' was missing, it didn't bother me.

--
Stephen Compall
^aCollection allSatisfy: [:each|aCondition]: less is better

Alex Baranosky

unread,
Feb 4, 2012, 10:55:18 PM2/4/12
to clo...@googlegroups.com
I'd be for deprecating defn- 
  • it creates symmetry (no defmacro-, no def-, defmulti- etc) 
  • The '-' is too small a visual cue to distinguish it from defn. In short, it is hard to read.  I personally prefer the obviousness of (defn ^:private [x] :foo)
Reply all
Reply to author
Forward
0 new messages