Consistency of the API

26 views
Skip to first unread message

Tiago Antão

unread,
Nov 9, 2009, 2:57:22 PM11/9/09
to clo...@googlegroups.com
Hi all,

Just a question about the consistency of the API:
When one passes a "strange" (ie, wrong type) object to contains?, say
(contains? 'blab 'a)
the result is a false.
But if one passes the wrong type to, e.g., even?, like
(even? 'a)
The result is:
java.lang.ClassCastException: clojure.lang.Symbol cannot be cast to
java.lang.Number (NO_SOURCE_FILE:0)

This seems a bit inconsistent, in the sense that one would expect the
same behavior for the same kind of error.

So the question is: is there any thing less obvious here (at least for
me), or is this a real issue with the API?

Thanks,
Tiago

--
"The hottest places in hell are reserved for those who, in times of
moral crisis, maintain a neutrality." - Dante

Kevin Downey

unread,
Nov 9, 2009, 3:08:05 PM11/9/09
to clo...@googlegroups.com
I don't understand, the error message you get is the error that occurred.

the docstring from even? says it throws an exception if the argument
is not and integer.

I would hope that anyone that has read the docstring for contains?
would not use (contains? 'foo 'bar), because, uh, that just makes no
sense. I mean that usage does not match what is indicated in the
docstring

2009/11/9 Tiago Antão <tiago...@gmail.com>:
--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

Tiago Antão

unread,
Nov 9, 2009, 3:17:36 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 8:08 PM, Kevin Downey <red...@gmail.com> wrote:
>
> I don't understand, the error message you get is the error that occurred.

Both of them honor their documentation - no doubt. My point is not
that, my point is that the behavior is different between the 2
functions for the same kind of issue:

What is the rationale for even? and contains? having different
behaviors for the exact same error (ie, one throws the other works
fine and just returns false on a type error)? From a design
perspective this seems to increase the cognitive load to programmers
without any (apparent) reason.

One would imagine that both functions should have the same behavior
for the same kind of error...

Mark Engelberg

unread,
Nov 9, 2009, 3:18:37 PM11/9/09
to clo...@googlegroups.com
The general philosophy in Clojure seems to be that if you use a
function in a way that is not intended, there's no guarantee about
what might happen. You might get an error, or you might just get a
strange result.

Mark Engelberg

unread,
Nov 9, 2009, 3:31:48 PM11/9/09
to clo...@googlegroups.com
2009/11/9 Tiago Antão <tiago...@gmail.com>:

> What is the rationale for even? and contains? having different
> behaviors for the exact same error (ie, one throws the other works
> fine and just returns false on a type error)? From a design
> perspective this seems to increase the cognitive load to programmers
> without any (apparent) reason.
>

I imagine the rationale is efficiency. Every core function could
conceivably do a number of runtime checks to make sure that each input
is the right kind of type, and then Clojure might feel more sluggish.
So instead, the core functions just worry about what to do for the
appropriate inputs. If you pass a bogus input, the consequence
depends entirely on how that particular function was coded. It might
return a spurious result, or it might error. There are numerous
examples of this in the Clojure API.

It wouldn't surprise me if there are a number of programmers who would
be turned off by the ease with which one can shoot yourself in the
foot without getting any kind of error message, but in practice I
haven't gotten bit by this yet.

Kevin Downey

unread,
Nov 9, 2009, 3:32:44 PM11/9/09
to clo...@googlegroups.com
the behavior of functions outside of their domain is undefined. I
guess I still don't get it. why would you use a function on something
outside of its domain? do people just pick functions at random to
compose their programs?


2009/11/9 Tiago Antão <tiago...@gmail.com>:

Mark Engelberg

unread,
Nov 9, 2009, 3:35:49 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 12:32 PM, Kevin Downey <red...@gmail.com> wrote:
>
> the behavior of functions outside of their domain is undefined. I
> guess I still don't get it. why would you use a function on something
> outside of its domain? do people just pick functions at random to
> compose their programs?

Of course they don't pick functions at random, but people do make
mistakes. Although I don't mind Clojure's approach, it is reasonable
for people to want clear error messages when they use a function
improperly.

Mark Engelberg

unread,
Nov 9, 2009, 3:49:34 PM11/9/09
to clo...@googlegroups.com
Here's another way to think about it.

Why is functional programming better than imperative programming?

One common answer to this question is that functional programs are
easier to debug. Why? Because in an imperative program, if one part
has an error, the error doesn't necessarily manifest in the portion of
code that has the error. Instead, a little piece of memory or state
can be corrupted in one spot, and then much, much later, an error
happens as a result of this inconsistency. Thus the need for
sophisticated debuggers and steppers to identify where the corruption
happened (since the manifestation of the error doesn't really help you
know the source of the problem).

However, in a functional program, you have these well-defined pieces
that reliably return the same output for a given set of inputs, making
it easier to test individual components, and when something produces
an error, it's pretty easy to pinpoint the function where things went
wrong.

Unfortunately, when core functions don't produce errors for invalid
inputs, then you have a similar problem as with imperative languages.
It becomes rather easy to write a program where the consequence of an
error is far removed from the source.

I have been told that Erlang programmers have developed a culture
where errors are generally not caught or disguised in any way.
There's sort of a "crash early and crash hard" philosophy, to increase
the likelihood that a crash will happen in the block of code that is
causing the problem and not later.

Kevin Downey

unread,
Nov 9, 2009, 4:18:28 PM11/9/09
to clo...@googlegroups.com
what makes functional programming better is the reduction of state. so
for example, if I decide that the function call out to contains? is
too much overhead in a tight loop, I can just copy and paste the
relevant code without being concerned that I might miss some crucial
piece of state it twiddles somewhere.

The true beauty of functional programming has little directly to do
with modes of failure. The beauty of functional programming is
transparency of code. I would argue that the kinds of errors you see
are an exact result of this transparency, each one is a little X-ray
window into the inner workings of the function.

On Mon, Nov 9, 2009 at 12:49 PM, Mark Engelberg

Tiago Antão

unread,
Nov 9, 2009, 4:23:41 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 8:31 PM, Mark Engelberg <mark.en...@gmail.com> wrote:
> I imagine the rationale is efficiency.  Every core function could
> conceivably do a number of runtime checks to make sure that each input
> is the right kind of type, and then Clojure might feel more sluggish.
> So instead, the core functions just worry about what to do for the
> appropriate inputs.  If you pass a bogus input, the consequence
> depends entirely on how that particular function was coded.  It might
> return a spurious result, or it might error.  There are numerous
> examples of this in the Clojure API.


I understand this, but it creates another kind of problem: When
programmers make mistakes (and I, being not a perfect human being,
make lots of mistakes) the system behaves inconsistently in a place
where one would expect consistency (type checking): sometimes it blows
in your face (like even?), sometimes it does that silently (like
contains?).

From a software engineering perspective is seems a bit dangerous.
Unless of course, programmers are perfect and never make mistakes, for
those kind, this discussion might seem ridiculous.

I, for one, prefer the blow in your face (ie, like even?) design
approach than the silent one. But above all, consistency would be nice
to have.

But I understand the rationale for efficiency. Tough I question if
there is really a rationale of efficiency or this issue was really
never really thought about (considering that Clojure is such a fast
moving target, that would be normal).

Peace,
Tiago

Konrad Hinsen

unread,
Nov 9, 2009, 4:53:42 PM11/9/09
to clo...@googlegroups.com
Tiago Antão a écrit :

> Just a question about the consistency of the API:
> When one passes a "strange" (ie, wrong type) object to contains?, say
> (contains? 'blab 'a)
> the result is a false.
> But if one passes the wrong type to, e.g., even?, like
> (even? 'a)
> The result is:
> java.lang.ClassCastException: clojure.lang.Symbol cannot be cast to
> java.lang.Number (NO_SOURCE_FILE:0)
>
> This seems a bit inconsistent, in the sense that one would expect the
> same behavior for the same kind of error.

Is it really the same kind of error? I am not sure. In fact, I'd say
this depends on the precise definitions of the two predicates.

The behaviour of contains? makes sense if you define "a contains b" as
"a is a collection AND b is an element of a". This definition implies a
return value of false whenever a is not a collection.

In the same spirit, one could define "x is even" as "x is an integer AND
x mod 2 = 0", concluding that (even? 'a) should return false because 'a
is not an integer. But that definition has its weak points as well, e.g.
that even? and odd? are not complementary.

I don't think it is possible to define a perfectly coherent set of
definitions of everything in the Clojure API that satisfies
simultaneously all conditions that one might expect to hold. In the end
it's a matter of priorities, and Rich's choices for Clojure are mostly
in the "pragmatic" category: a compromise between principles,
simplicity, and efficiency.

Konrad.


__________ Information provenant d'ESET NOD32 Antivirus, version de la base des signatures de virus 4589 (20091109) __________

Le message a été vérifié par ESET NOD32 Antivirus.

http://www.eset.com


John Harrop

unread,
Nov 9, 2009, 5:20:07 PM11/9/09
to clo...@googlegroups.com
Even more interesting is the behavior of contains? when passed strings:

user=> (contains? "foo" \o)
false
user=> (contains? "foo" 2)
true
user=> (contains? "foo" 3)
false
user=> (contains? 'foo 2)
false

It seems to treat strings as it does vectors, seeing if an index is in bounds or not. It doesn't treat symbols as anything though.

The clojure.contrib.seq-utils/includes? function gives true for "foo" and \o, so it seems to recognize strings as collections (probably because they're seqable).

On the matter of efficiency vs. error-catching, perhaps one can have one's cake and eat it too? How about adding a throw-if! macro to the language that checks, at macroexpansion time, if it's in debug mode and if not emits no code, but if so emits (if ~cond (throw (new ~exception-class ~message))). (It looks like a macro that expands to nil emits no code, and also doesn't screw things up in any way, anyway.)

How to decide "if it's in debug mode" though? Some global binding check? Of course, there's also the issue that to turn on or off error-checking in library functions would require recompiling the library.

Perhaps what's really needed is a new special form, one which emits the same bytecodes as Java asserts when compiled, and thus can be turned on or off at runtime in the same way, without recompiling anything (only reloading affected classes).

Alex Osborne

unread,
Nov 9, 2009, 5:50:55 PM11/9/09
to clo...@googlegroups.com
Mark Engelberg wrote:
> 2009/11/9 Tiago Antão <tiago...@gmail.com>:
>> What is the rationale for even? and contains? having different
>> behaviors for the exact same error (ie, one throws the other works
>> fine and just returns false on a type error)?

> I imagine the rationale is efficiency.

Here's the function from clojure/lang/RT.java:

static public Object contains(Object coll, Object key){
if(coll == null)
return F;
else if(coll instanceof Associative)
return ((Associative) coll).containsKey(key) ? T : F;
else if(coll instanceof IPersistentSet)
return ((IPersistentSet) coll).contains(key) ? T : F;
else if(coll instanceof Map) {
Map m = (Map) coll;
return m.containsKey(key) ? T : F;
}
else if(key instanceof Number && (coll instanceof String ||
coll.getClass().isArray())) {
int n = ((Number) key).intValue();
return n >= 0 && n < count(coll);
}
return F;
}


That last return could be changed to a throw and it wouldn't make things
any slower (in the non-error case). Note that 'get' always behaves the
same way, so I guess it's probably intentional for associative lookups,
I can't see why though.

(get 3 3)
=> nil

Tiago Antão

unread,
Nov 9, 2009, 6:39:57 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 10:20 PM, John Harrop <jharr...@gmail.com> wrote:
> It seems to treat strings as it does vectors, seeing if an index is in
> bounds or not. It doesn't treat symbols as anything though.
> The clojure.contrib.seq-utils/includes? function gives true for "foo" and


I did not want to make this a discussion about contains? per se, but
about the general design philosophy about dealing with type erros
(which I think is a bigger issue, IMHO).

But it is a bit difficult to avoid noticing that contains? is a bit
unintuitive, so to say.

Even with vectors
(contains? [1 5] 5) being false sounds somewhat strange.

I understand the initial intent, like this:
(contains? {'a 5} 5) being false

But the end result with strings and vectors is a tad unintuitive...

Tiago

Mark Engelberg

unread,
Nov 9, 2009, 7:00:17 PM11/9/09
to clo...@googlegroups.com
2009/11/9 Tiago Antão <tiago...@gmail.com>:

> But the end result with strings and vectors is a tad unintuitive...

Right, strings and vectors can be thought of as either collections, or
as associative mappings from integers to characters/objects.
contains? treats them as associative mappings. Yes, it's unintuitive,
but it has a certain degree of internal consistency.

Richard Newman

unread,
Nov 9, 2009, 7:48:23 PM11/9/09
to clo...@googlegroups.com
> Right, strings and vectors can be thought of as either collections, or
> as associative mappings from integers to characters/objects.
> contains? treats them as associative mappings. Yes, it's unintuitive,
> but it has a certain degree of internal consistency.

This certainly encourages you to use sets whenever you want, um, set-
like behavior...

John Harrop

unread,
Nov 9, 2009, 8:28:39 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 5:50 PM, Alex Osborne <a...@meshy.org> wrote:
Mark Engelberg wrote:
> 2009/11/9 Tiago Antão <tiago...@gmail.com>:
>> What is the rationale for even? and contains? having different
>> behaviors for the exact same error (ie, one throws the other works
>> fine and just returns false on a type error)?

> I imagine the rationale is efficiency.

Here's the function from clojure/lang/RT.java:

static public Object contains(Object coll, Object key){
        if(coll == null)
                return F;
        else if(coll instanceof Associative)
                return ((Associative) coll).containsKey(key) ? T : F;
        else if(coll instanceof IPersistentSet)
                return ((IPersistentSet) coll).contains(key) ? T : F;
        else if(coll instanceof Map) {
                Map m = (Map) coll;
                return m.containsKey(key) ? T : F;
        }
        else if(key instanceof Number && (coll instanceof String ||
                        coll.getClass().isArray())) {
                int n = ((Number) key).intValue();
                return n >= 0 && n < count(coll);
        }
        return F;
}

Why not:

static public Object contains(Object coll, Object key){
        if(coll == null)
                return F;
        else if(coll instanceof Map)
                return ((Map) coll).containsKey(key) ? T : F;
        else if(coll instanceof IPersistentSet)
                return ((IPersistentSet) coll).contains(key) ? T : F;
        else if(key instanceof Number && (coll instanceof String ||
                        coll.getClass().isArray())) {
                int n = ((Number) key).intValue();
                return n >= 0 && n < count(coll);
        }
        return F;
}

instead?

Oh, and who else finds it odd that the above implies that

user=> (contains? [1 2 3] 0.71)
true

when in actuality

user=> (contains? [1 2 3] 0.71)
false

despite

user=> (.intValue 0.71)
0
user=> (contains? [1 2 3] 0)
true

?

(I suppose T and F are static fields holding java.lang.Booleans, and this code was written pre-autoboxing?)

John Harrop

unread,
Nov 9, 2009, 8:41:25 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 8:28 PM, John Harrop <jharr...@gmail.com> wrote:
Why not:

static public Object contains(Object coll, Object key){
        if(coll == null)
                return F;
        else if(coll instanceof Map)
                return ((Map) coll).containsKey(key) ? T : F;
        else if(coll instanceof IPersistentSet)
                return ((IPersistentSet) coll).contains(key) ? T : F;
        else if(key instanceof Number && (coll instanceof String ||
                        coll.getClass().isArray())) {
                int n = ((Number) key).intValue();
                return n >= 0 && n < count(coll);
        }
        return F;
}

instead?

Sort of answered my own question:

user=> (ancestors (class [1 2 3]))
#{clojure.lang.Associative java.lang.Comparable java.lang.Iterable
  clojure.lang.IPersistentVector java.util.RandomAccess clojure.lang.IMeta
  clojure.lang.IObj clojure.lang.IPersistentCollection
  clojure.lang.Sequential clojure.lang.IFn clojure.lang.Seqable
  java.util.concurrent.Callable java.io.Serializable clojure.lang.Streamable
  clojure.lang.AFn clojure.lang.Obj clojure.lang.IPersistentStack
  clojure.lang.Counted java.util.Collection java.lang.Object java.util.List
  clojure.lang.Reversible clojure.lang.APersistentVector java.lang.Runnable}

Looks like Associative doesn't inherit Map for some reason. I wonder why not? There's no reason for it not to, with IPersistentVector for example functioning as a Map<Integer,Object>. Since none of the Map modifying operations would be supported it's not like someone could add a new mapping to [1 2 3] like {5000000, "foo"} or {-12, 'bar} or something like that that would screw things up.

On the other hand,

static public Object contains(Object coll, Object key){
        if(coll == null)
                return F;
        else if(coll instanceof Map)
                return ((Map) coll).containsKey(key) ? T : F;
        else if(coll instanceof IPersistentSet)
                return ((IPersistentSet) coll).contains(key) ? T : F;
        else if(key instanceof Number && (coll instanceof String ||
                        coll.getClass().isArray() ||
                        coll instanceof Collection)) {
                int n = ((Number) key).intValue();
                return n >= 0 && n < count(coll);
        }
        return F;
}

catches IPersistentVector and friends with strings and arrays in clause 3, and neatly handles maps, sets, and list-like collections in three cleanly-separated clauses, instead of mixing maps and some list-like collections.

In the meantime, the main thing still missing from Clojure is a convenient queue. Lists and vectors both add and remove efficiently only at one end, and at the same end for add and remove in both cases. Doubly-linked lists can't be made persistent without massive problems, but laziness has its own issues:

(defn queue-peek [q] (first q))
(defn queue-pop [q] (rest q))
(defn queue-push [q obj] (concat q [obj]))

(let [q (reduce queue-push nil (range 1000000))]
  (reduce (fn [_ q] (queue-pop q)) nil q))
#<CompilerException java.lang.StackOverflowError (NO_SOURCE_FILE:0)>


Mark Engelberg

unread,
Nov 9, 2009, 8:53:28 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 5:41 PM, John Harrop <jharr...@gmail.com> wrote:
> In the meantime, the main thing still missing from Clojure is a convenient
> queue.

What's wrong with clojure.lang.PersistentQueue?

David Brown

unread,
Nov 9, 2009, 8:54:36 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 09, 2009 at 08:41:25PM -0500, John Harrop wrote:

>In the meantime, the main thing still missing from Clojure is a convenient
>queue. Lists and vectors both add and remove efficiently only at one end,
>and at the same end for add and remove in both cases. Doubly-linked lists
>can't be made persistent without massive problems, but laziness has its own
>issues:

Perhaps the GHC Data.Sequence library could be ported. It's based on
2-3 finger trees, and allows efficient adding and removal from either
end of the sequence.

Depending on use behavior, you can also make a decent lazy queue just
out a two lists, where you reverse and append whenever the source side
fills up.

David

David Brown

unread,
Nov 9, 2009, 9:00:32 PM11/9/09
to clo...@googlegroups.com

The only clojure constructor I could find for this is in
clojure.contrib.accumulators. But, once you have the empty one
'clojure.lang.PersistentQueue/EMPTY' you can use it pretty well with
the rest of the language.

Perhaps 'empty-queue' should be moved into core?

David

David Brown

unread,
Nov 9, 2009, 9:01:15 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 09, 2009 at 05:54:36PM -0800, David Brown wrote:

>Depending on use behavior, you can also make a decent lazy queue just
>out a two lists, where you reverse and append whenever the source side
>fills up.

Ok, this is what PersistentQueue is, except without the reverse and
append, so it actually performs well.

David

Mark Engelberg

unread,
Nov 9, 2009, 9:07:45 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 5:54 PM, David Brown <clo...@davidb.org> wrote:
> Perhaps the GHC Data.Sequence library could be ported.  It's based on
> 2-3 finger trees, and allows efficient adding and removal from either
> end of the sequence.

I've tried porting finger trees to Scheme before, and although it is
efficient in an algorithmic sense, I found the constants involved to
be so high, performance was very poor. Perhaps it was a fault with my
port, but I've become quite skeptical of the value of finger trees.

Anyway, I've been satisfied with clojure.lang.PersistentQueue for queues.

I often want a priority queue, but John Harrop addressed that need not
long ago. Before that, I would sometimes just use Clojure's sorted
set.

Sometimes I still want a persistent deque, but I haven't needed it
quite badly enough to warrant coding one myself.

John Harrop

unread,
Nov 9, 2009, 10:20:26 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 8:41 PM, John Harrop <jharr...@gmail.com> wrote:
In the meantime, the main thing still missing from Clojure is a convenient queue. Lists and vectors both add and remove efficiently only at one end, and at the same end for add and remove in both cases. Doubly-linked lists can't be made persistent without massive problems, but laziness has its own issues:

(defn queue-peek [q] (first q))
(defn queue-pop [q] (rest q))
(defn queue-push [q obj] (concat q [obj]))

(let [q (reduce queue-push nil (range 1000000))]
  (reduce (fn [_ q] (queue-pop q)) nil q))
#<CompilerException java.lang.StackOverflowError (NO_SOURCE_FILE:0)>

Solved.

(defn queue-peek [[v i]] (nth v i))
(defn queue-pop [[v i]]
  (if (> (* 2 i) (count v))
    [(vec (drop (inc i) v)) 0]
    [(assoc v i nil) (inc i)]))
 (defn queue-push [[v i] obj] [(conj v obj) i])

user=> (reduce queue-push [[] 0] (range 20))
[[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19] 0]
user=> (queue-peek (nth (iterate queue-pop (reduce queue-push [[] 0] (range 20))) 14))
14
user=> (take 20 (iterate queue-pop (reduce queue-push [[] 0] (range 20))))
([[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19] 0]
 [[nil 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19] 1]
 [[nil nil 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19] 2]
 [[nil nil nil 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19] 3]
 [[nil nil nil nil 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19] 4]
 [[nil nil nil nil nil 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19] 5]
 [[nil nil nil nil nil nil 6 7 8 9 10 11 12 13 14 15 16 17 18 19] 6]
 [[nil nil nil nil nil nil nil 7 8 9 10 11 12 13 14 15 16 17 18 19] 7]
 [[nil nil nil nil nil nil nil nil 8 9 10 11 12 13 14 15 16 17 18 19] 8]
 [[nil nil nil nil nil nil nil nil nil 9 10 11 12 13 14 15 16 17 18 19] 9]
 [[nil nil nil nil nil nil nil nil nil nil 10 11 12 13 14 15 16 17 18 19] 10]
 [[nil nil nil nil nil nil nil nil nil nil nil 11 12 13 14 15 16 17 18 19]
  11]
 [[12 13 14 15 16 17 18 19] 0]
 [[nil 13 14 15 16 17 18 19] 1]
 [[nil nil 14 15 16 17 18 19] 2]
 [[nil nil nil 15 16 17 18 19] 3]
 [[nil nil nil nil 16 17 18 19] 4]
 [[nil nil nil nil nil 17 18 19] 5]
 [[18 19] 0]
 [[nil 19] 1])

As you can see, it uses a vector internally, plus an internal pointer to where the current "first" element is. Consumed elements are nulled out so if the queue is used to hold very large objects, stale references to these are not retained. If more than half the present length of the vector is consumed entries, the vector is copied to a smaller vector with no consumed entries, mirroring how things like ArrayList grow by doublings and ensuring the copying cost is amortized O(1) if there's a long run of queue-consumption.

The queue is persistent and immutable though, much as the underlying vector is. (Question: can Clojure's vector hold onto stale references in parts that are unused? Or does it null out array cells that are not part of any currently existing vector? If the latter, how does it know -- finalizers? ReferenceQueues?)

As for why I didn't use

(defn queue-peek2 [q] (first q))
(defn queue-pop2 [q] (subvec q 1))
(defn queue-push2 [q obj] (conj q obj))

instead:

user=> (nth (iterate #(subvec (into % [1 2 3]) 3) [0 0 0]) 1000000000)
#<10 minutes of nothing happening>
#<CompilerException java.lang.OutOfMemoryError: Java heap space (NO_SOURCE_FILE:0)>

Subvec appears to blow up the heap if you keep growing at one end and chopping at the other for long enough, unlike copying the vector entirely. Meanwhile:

user=> (nth (iterate #(queue-pop (queue-push % 'foo)) [['foo 'foo] 0]) 1000000000)
#<30 minutes of nothing happening>
[[foo foo] 0]

John Harrop

unread,
Nov 9, 2009, 10:22:52 PM11/9/09
to clo...@googlegroups.com
There is one? There's no corresponding API in clojure.core, at least not in Clojure 1.0. 

Mark Engelberg

unread,
Nov 9, 2009, 10:37:02 PM11/9/09
to clo...@googlegroups.com
Yes, it's in Clojure 1.0, it just doesn't have a convenient name.

So give it a convenient name like this:
(def empty-queue clojure.lang.PersistentQueue/EMPTY)

and then you're ready to go.

conj, peek, pop, into and all the other sequence-based functions work
the way you'd expect.

The implementation cleverly uses a list for the first part of the
queue, and a vector for the second part of the queue, and is efficient
and persistent.

John Harrop

unread,
Nov 9, 2009, 10:49:52 PM11/9/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 10:37 PM, Mark Engelberg <mark.en...@gmail.com> wrote:
Yes, it's in Clojure 1.0, it just doesn't have a convenient name.

So give it a convenient name like this:
(def empty-queue clojure.lang.PersistentQueue/EMPTY)

and then you're ready to go.

conj, peek, pop, into and all the other sequence-based functions work
the way you'd expect.

The implementation cleverly uses a list for the first part of the
queue, and a vector for the second part of the queue, and is efficient
and persistent.

How does it efficiently deal with when the list-part has been completely consumed? 

Mark Engelberg

unread,
Nov 9, 2009, 10:58:34 PM11/9/09
to clo...@googlegroups.com
> How does it efficiently deal with when the list-part has been completely
> consumed?

Well, the latest code is here:
http://github.com/richhickey/clojure/blob/master/src/jvm/clojure/lang/PersistentQueue.java
I don't know whether it has changed since 1.0.
Just look in the src/jvm/clojure/lang directory of your clojure distribution.

But basically, when the front list is exhausted, the front part
becomes the seq of the vector, and the rear part becomes an empty
vector ready to accept more elements.

Rich Hickey

unread,
Nov 10, 2009, 8:05:05 AM11/10/09
to clo...@googlegroups.com
On Mon, Nov 9, 2009 at 3:31 PM, Mark Engelberg <mark.en...@gmail.com> wrote:
>
> 2009/11/9 Tiago Antão <tiago...@gmail.com>:
>> What is the rationale for even? and contains? having different
>> behaviors for the exact same error (ie, one throws the other works
>> fine and just returns false on a type error)? From a design
>> perspective this seems to increase the cognitive load to programmers
>> without any (apparent) reason.
>>
>
> I imagine the rationale is efficiency. Every core function could
> conceivably do a number of runtime checks to make sure that each input
> is the right kind of type, and then Clojure might feel more sluggish.
> So instead, the core functions just worry about what to do for the
> appropriate inputs. If you pass a bogus input, the consequence
> depends entirely on how that particular function was coded. It might
> return a spurious result, or it might error. There are numerous
> examples of this in the Clojure API.
>

Efficiency is certainly part of it. However, it is not spurious.
Errors will become exceptions, not random nonsense. Some things one
might consider errors are not considered as such, and that is quite a
different thing.

One presumption here is that this is a simple matter of types. It is
not (simple). There are some functions whose domain is constrained by
a known/fixed (family of) type(s), but many that are not. even? is
(Numbers), contains? is not (open set of associative-like things,
false otherwise). This type-openness of many Clojure core functions is
one key to its power. I'm sure making contains?/get tolerant of
non-associative things gave us benefits elsewhere (i.e. made
associative destructuring tolerant of, and thus useful upon,
heterogeneous sequences, only some of whose elements were
associative). If that were not the case, then any code needed to
handle that would need a predicate for an open set in order to filter
first. Such predicates are extremely icky to construct and maintain,
and, brittle to rely upon.

The work I am doing on protocols may help out quite a bit here, as
they will provide a unifying point for open abstractions, encompassing
explicit type relationships (inheritance form interfaces),
superimposed capabilities (e.g. the way Strings become seq-able), and
open functional extension (a la multimethods), in a way such that
asking (implements? protocol x) will become an open type predicate.

> It wouldn't surprise me if there are a number of programmers who would
> be turned off by the ease with which one can shoot yourself in the
> foot without getting any kind of error message, but in practice I
> haven't gotten bit by this yet.
>

Everyone would like more help from a system when they have made a
mistake. But everyone needs to have a realistic view about what can be
detected, and at what cost, (especially to produce a revelatory
message), as well as a general understanding of open functions. I
imagine at some point we may have a diagnostic mode, with many
(expensive) runtime checks, and correspondingly more detailed
error/mistake reports. But you are right, the emphasis right now is on
making correct programs work well.

Rich

Kresten Krab Thorup

unread,
Nov 11, 2009, 3:13:33 AM11/11/09
to Clojure


On Nov 10, 2:28 am, John Harrop <jharrop...@gmail.com> wrote:
> (I suppose T and F are static fields holding java.lang.Booleans, and this
> code was written pre-autoboxing?)

It's still much faster to use a "pre-boxed Boolean" than to create a
new boxed value every time around.

Kresten
Reply all
Reply to author
Forward
0 new messages