Utilities for clojure.contrib?
Jason Wolfe
I've got lots of utilities that I think people might find useful, and
might fit well in clojure.contrib (just sent in my contrib agreement
last week). I don't know how the process works to get code in there,
but I figured I'd go ahead and post them all here so people can chime
in about what they like / don't like first.
Some of these depend on each-other, but I think the collection as a
whole is self-sufficient.
;;; Sequences.
(defn combinations "Take a seq of seqs and return a lazy seq of
ordered combinations (pick 1 from each seq)"
[seqs]
(if (empty? seqs) '(())
(forcat [item (first seqs)]
(map #(cons item %) (combinations (rest seqs))))))
(defn power-set
"Returns a lazy seq of possible subvectors of seq s."
[s]
(loop [s (seq s), sets [[]]]
(if s
(recur (rest s) (lazy-cat sets (map #(conj % (first s))
sets)))
sets)))
(defn random-permutation [s]
"Return a random permutation of this seq."
(let [arr (to-array s) len (alength arr)]
(dotimes [i (dec len)]
(let [r (+ i (rand-int (- len i))),
prev (aget arr i)]
(aset arr i (aget arr r))
(aset arr r prev)))
(seq arr)))
(defn random-element [s]
"Return a random element of this seq"
(nth s (rand-int (count s))))
(defn maximal-elements [f s]
"Return a seq of elements of s maximizing (f elt)."
(when (seq s)
(loop [max-elts (first s),
max-val (f (first s)),
rest-elts (rest s)]
(if (empty? rest-elts)
max-elts
(let [next-val (f (first rest-elts))]
(cond (< next-val max-val) (recur max-elts max-val (rest rest-
elts))
(= next-val max-val) (recur (cons (first rest-elts) max-elts) max-
val (rest rest-elts))
(> next-val max-val) (recur [(first rest-elts)] next-val (rest rest-
elts))))))))
(import '(java.util HashSet))
(defn distinct-elts? "Are all of the elements of this sequence
distinct? Works on infinite sequences with repititions, making it
useful for, e.g., detecting cycles in graphs."
[s]
(let [hs (HashSet.)]
(loop [s (seq s)]
(cond (empty? s) true
(.contains hs (first s)) false
:else (do (.add hs (first s)) (recur (rest s)))))))
(defn concat-elts "Lazily concatenate elements of a seq of seqs." [s]
(when (seq s) (lazy-cat (first s) (concat-elts (rest s)))))
(defn lazy-mapcat "Like mapcat but is lazy like map" [f s]
(concat-elts (map f s)))
(defn map-when "Like map but discards logically false entries"
[fn & seqs]
(filter identity (apply map fn seqs)))
(defn iterate-while [f x]
(take-while identity (iterate f x)))
(defn chunk "Lazily break s into chunks of length n (or less, for the
final chunk)."
[n s]
(when (seq s)
(lazy-cons (take n s)
(chunk n (drop n s)))))
(defn mevery? "Like every but takes multiple seq args like map."
([f & seqs]
(or (some empty? seqs)
(and (apply f (map first seqs))
(recur f (map rest seqs))))))
;;; Maps
(defn map-map "Like map, but expects f to return pairs/map entries
that are combined to make a map return value."
[f & maps] (reduce #(conj %1 %2) {} (apply map f maps)))
(defmacro lazy-get "Like get but lazy about evaluating the default
value"
[m k d]
`(if-let [pair# (find ~m ~k)]
(second pair#)
~d))
(defn safe-get "Like get but throws an exception if not found"
[m k]
(lazy-get m k (throw (IllegalArgumentException. (format "Key %s not
found" k)))))
(defn merge-agree "Like merge but returns nil if there are
inconsistent key/value pairs."
([] {})
([map] map)
([m1 m2 & maps]
(when (every? (fn [[k v]] (= v (get m1 k v))) m2)
(apply merge-agree (merge m1 m2) maps))))
(defn merge-reduce "Combines maps, reducing sets of values with same
key. Assumes nil value = not present. The first map entry must be a
real map, but the remaining arguments can be seqs of map entries/key-
value pairs."
([f ] {})
([f m1 & maps]
(reduce (fn [m [k v]]
(if-let [v2 (get m k)]
(assoc m k (f v2 v))
(assoc m k v)))
m1
(concat-elts maps))))
;;; Miscellaneous
(defn prln "Print all arguments and return the first argument"
[& args] (do (println (apply print-str args)) (first args)))
(defn xor [& args]
(odd? (count (filter identity args))))
(defmacro forcat
"Like for, but concatenates the results."
[& args] `(concat-elts (for ~@args)))
(defmacro assert-is
"Like assert, but prints some more info (like is) about the
offending form (may multiple eval on error)"
([form] `(assert-is ~form ""))
([form format-str & args]
`(when-not ~form
(throw (Exception. (str (format ~format-str ~@args)
": Got " '~form " as " (cons '~(first form) (list ~@(rest
form)))))))))
(I have more too, but these are the ones I use most frequently.)
Comments/criticisms welcome. Also, many of these are placeholders for
more efficient versions, which I planned on doing if needed (but would
be happy to do if they're going into clojure.contrib).
-Jason
Chouser
>
> (defn map-when "Like map but discards logically false entries"
> [fn & seqs]
> (filter identity (apply map fn seqs)))
I'd use map-when.
> (defn iterate-while [f x]
> (take-while identity (iterate f x)))
This one too.
It raises a question, though -- how much functionality should a
function provide to be worth making everyone who reads the code learn
the new vocabulary? I've written each of these inline when I've
needed them. Are they better as idioms or functions?
--Chouser
GS
>
> It raises a question, though -- how much functionality should a
> function provide to be worth making everyone who reads the code learn
> the new vocabulary? I've written each of these inline when I've
> needed them. Are they better as idioms or functions?
that. But I think a rich utility library is good, so long as it's
clear from a function's name and parameters what it does. There are
only a few in the collection above whose purpose I don't understand;
for most I see it easily.
One question about the collection:
> (defn chunk "Lazily break s into chunks of length n (or less, for the
> final chunk)."
> [n s]
> (when (seq s)
> (lazy-cons (take n s)
> (chunk n (drop n s)))))
The general question it raises for _me_ is this: why is such a basic,
useful and generally applicable function like 'chunk not included in
the core? Or 'random-element?
Final comment: I think collection parameters should be named "coll",
not "s", so that the docs are consistent with the core functions.
Gavin
Nick Vogel
Chouser
Nick nailed that one.
> The general question it raises for _me_ is this: why is such a basic,
> useful and generally applicable function like 'chunk not included in
> the core? Or 'random-element?
'chunk' is very similar to the built-in 'partition', the difference
being how the last group is handled.
user=> (partition 2 '[a b c d e])
((a b) (c d))
--Chouser
Jason Wolfe
> > [fn & seqs]
> > (filter identity (apply map fn seqs)))
>
> I'd use map-when.
>
> > (defn iterate-while [f x]
> > (take-while identity (iterate f x)))
>
> This one too.
>
> It raises a question, though -- how much functionality should a
> function provide to be worth making everyone who reads the code learn
> the new vocabulary? I've written each of these inline when I've
> needed them. Are they better as idioms or functions?
First, many of these aren't really "new vocabulary", since you can
clearly tell what they do from their name and knowledge of the
existing core functions. If you know map and you know take-when and
take-while, you already know what map-when and map-while do.
Moreover, I think some of these actually fill in mysterious gaps in
the language. For and map are essentially two different ways of
saying the same thing; which to use in a given situation seems to be
largely a matter of aesthetics (except when there are multiple
iterates...). Yet, for supports :when and :while, whereas there are
no map-when and map-while forms. Conversely, we get mapcat but no
forcat.
Even if you don't buy this, the other advantage of having, e.g., (map-
when ...) rather than treating (filter identity (map ...)) as
idiomatic is that map-when can be implemented more efficiently, so
that it doesn't cons up an extra list; however, the more efficient
form is much uglier and almost certainly won't become an idiom.
So, to summarize, IMO adding utilities like map-when give you
- More uniformity in the language
- More concise programs (without sacrificing readibility)
- Greater efficiency
Finally, I want to mention that some of the idioms these functions
replace are not as easy to read as that for map-when. For instance, I
find myself needing to use map-map or merge-reduce fairly frequently,
and the (reduce (fn [m [k v]] (assoc m k (f v))) {} m) type idioms
they replace are much more difficult to read than calls to the
utilities IMO.
Cheers,
Jason
map-map
Chouser
>>
>> It raises a question, though -- how much functionality should a
>> function provide to be worth making everyone who reads the code learn
>> the new vocabulary? I've written each of these inline when I've
>> needed them. Are they better as idioms or functions?
>
> I would say they're better as functions, for several reasons.
>
> First, many of these aren't really "new vocabulary", since you can
> clearly tell what they do from their name and knowledge of the
> existing core functions. If you know map and you know take-when and
> take-while, you already know what map-when and map-while do.
>
> Moreover, I think some of these actually fill in mysterious gaps in
> the language. For and map are essentially two different ways of
> saying the same thing; which to use in a given situation seems to be
> largely a matter of aesthetics (except when there are multiple
> iterates...). Yet, for supports :when and :while, whereas there are
> no map-when and map-while forms. Conversely, we get mapcat but no
> forcat.
Those are good arguments. I guess what I fear is some of the reaction
I've had to reading Common Lisp code. The difference in feeling
between CL and Clojure is important to me because I tried repeatedly
and failed to adopt CL into my toolbox, but Clojure went quite
smoothly.
One of the things I found difficult with CL was the extremely large
number of builtin functions -- a large vocabulary with subtle
differences between nearly synonymous-sounding words. It meant that
at first glance, a particular block of could would look simple -- 5
lines, how hard could it be? But in reading it I would discover I
only new 2 of the 15 functions used. So I'd fight through it, feeling
like I was learning something useful, but the next function would use
15 completely different functions.
Now perhaps I'm just whining, perhaps I needed better reference tools,
perhaps I'm mis-identifying the problem. But it is a real source of
my hesitancy to ask for more built-in functions.
> For instance, I
> find myself needing to use map-map or merge-reduce fairly frequently,
> and the (reduce (fn [m [k v]] (assoc m k (f v))) {} m) type idioms
> they replace are much more difficult to read than calls to the
> utilities IMO.
Yes, that's a mouthful. I'll need to study merge-reduce and map-map a
bit more to see when to use them instead of merge, merge-with, and
into.
--Chouser
Jason Wolfe
> One of the things I found difficult with CL was the extremely large
> number of builtin functions -- a large vocabulary with subtle
> differences between nearly synonymous-sounding words. It meant that
> at first glance, a particular block of could would look simple -- 5
> lines, how hard could it be? But in reading it I would discover I
> only new 2 of the 15 functions used. So I'd fight through it, feeling
> like I was learning something useful, but the next function would use
> 15 completely different functions.
much reading of other peoples' code. The size of the reference
section of ANSI CL is definitely daunting though.
I guess there's a delicate balance here between functionality and
complexity. I think my personal preference is towards more
functionality at the expense of added complexity, but I can see the
other side too.
One thing worth pointing out is that, if a utility is really useful
many people will independently recreate it with slightly different
names and functionality, and use it in their code. It seems that this
results in code that's much more difficult to read than if the utility
was standardized in the first place.
> Yes, that's a mouthful. I'll need to study merge-reduce and map-map a
> bit more to see when to use them instead of merge, merge-with, and
> into.
almost identical to merge-with (except that it allows seqs of map-
entries for all maps but the first), and map-map is equivalent to
(into {} (map ...)).
I'm not sure if this example supports or detracts from my argument
above. On one hand, these utilities are clearly useful enough that
I've independently recreated them (this also happened verbatim for
several utilities I later discovered in clojure.contrib). On the
other hand, given that they already existed (in clojure.core, no
less), this would seem to support your point that having more
standardized utilities just means that nobody will ever know them all
well.
-Jason
GS
>
> One thing worth pointing out is that, if a utility is really useful
> many people will independently recreate it with slightly different
> names and functionality, and use it in their code. It seems that this
> results in code that's much more difficult to read than if the utility
> was standardized in the first place.
your functions in clojure.contrib if the objective is to standardize
them? Inclusion in the core is the most secure path to
standardization :)
I realise the core shouldn't be expanded forever, but in the case of
map-when, map-while, etc:
* these are _fundamental_ functions, as opposed to domain-specific
functions (DB access, web programming, ...) which suggest "library"
rather than "core"
* they are _named_ like other fundamental functions, as you pointed
out,
so they add to the utility of the core without adding to its
complexity
* provided they're useful enough, they're crying out to be
standardized
in this way: it's odd to use take-when freely but have to load a
library
to use map-when
I would say this argument does not hold for functions such as
'combinations.
>
> > [...]
> I'm not sure if this example supports or detracts from my argument
> above. On one hand, these utilities are clearly useful enough that
> I've independently recreated them (this also happened verbatim for
> several utilities I later discovered in clojure.contrib). On the
> other hand, given that they already existed (in clojure.core, no
> less), this would seem to support your point that having more
> standardized utilities just means that nobody will ever know them all
> well.
clojure's core functions, coming from an OO background where classes
prevent global namespace pollution (and make it easier, for example,
to read about all functions on strings, or lists, or ...). Therefore
I would not be keen on a huge core. But I think it's important to
plug the gaps (any?, map-when) with consistent naming schemes so that
new users feel Clojure is well-organised, and a sensible, rich
vocabulary develops amongs all users.
I've spent a lot of time documenting Ruby's standard library, and
based on that experience, think it's a damn good idea to get a lot of
these "basics" sorted out as early as possible. On the other hand, I
lack extensive experience in Clojure, so can't necessarily offer good
judgement here. But please be assured I have read a _lot_ of messages
in this group before posting any :)
Gavin
Stuart Sierra
> One of the things I found difficult with CL was the extremely large
> number of builtin functions -- a large vocabulary with subtle
> differences between nearly synonymous-sounding words.
hand, CL also has the problem that everyone defines their own batch of
little utility functions, with a lot of overlap but all slightly
different.
clojure.contrib is slowly evolving into a standard library, but so far
it has been more valuable as a testing ground for features that later
went into core -- the lib system, several predicates, and defonce,
just to name a few. I think contrib is a good place to post any code
that might be generally useful. If there are things in contrib that
you feel are critical to Clojure itself, you can lobby for their
inclusion in the core.
-Stuart Sierra
GS
> > > final chunk)."
> > > [n s]
> > > (when (seq s)
> > > (lazy-cons (take n s)
> > > (chunk n (drop n s)))))
>
> > Should that "seq" be "seq?". If not, why not?
> seq returns nil when a collection has no items. According to the
> documentation for empty?, empty? is the same as (not (seq coll)) so you
> should use seq for expressing the opposite of empty?
'seq or 'seq? is used. Intuitively (i.e. without actually knowing :)
seq? is more efficient because it returns a boolean, not a newly-
allocated sequence.
So, either:
1. My experiment was wrong, and seq? is not a valid stand-in
for seq in the above code.
2. My intuition is wrong, and 'seq? is not more efficient than 'seq.
3. 'seq and 'seq? are equally valid, but 'seq is preferred
because of the definition of 'empty?
Or something else. Which is it?
Thanks,
Gavin
Chouser
builtin functions than needed, because it makes that code harder to
read as well. So I will consider each of these more carefully.
My comments below are of course highly influence by my personal
experiences using Clojure. I'm quite willing to admit that even if
I've never needed a function, it's certainly possible that everyone
else has.
On Mon, Jan 12, 2009 at 6:48 PM, Jason Wolfe <jaw...@berkeley.edu> wrote:
>
> ;;; Sequences.
>
> (defn combinations "Take a seq of seqs and return a lazy seq of
> ordered combinations (pick 1 from each seq)"
> [seqs]
> (if (empty? seqs) '(())
> (forcat [item (first seqs)]
> (map #(cons item %) (combinations (rest seqs))))))
I don't think I've ever needed this. When I've needed similar, I've
used 'for', but being a macro it can be harder to compose and requires
knowing the depth of the nesting ahead of time. I'm not aware of
anything other builtin that provides this kind of nesting behavior, so
I'd tend to say this pays for itself.
Almost identical to clojure.contrib.lazy-seqs/combinations
(require '[clojure.contrib.lazy-seqs :as ls])
user=> (combinations [[1 2] [4 5] [7 8]])
((1 4 7) (1 4 8) (1 5 7) (1 5 8) (2 4 7) (2 4 8) (2 5 7) (2 5 8))
user=> (ls/combinations [1 2] [4 5] [7 8])
([1 4 7] [1 4 8] [1 5 7] [1 5 8] [2 4 7] [2 4 8] [2 5 7] [2 5 8])
> (defn power-set
> "Returns a lazy seq of possible subvectors of seq s."
> [s]
> (loop [s (seq s), sets [[]]]
> (if s
> (recur (rest s) (lazy-cat sets (map #(conj % (first s))
> sets)))
> sets)))
Never needed it, but looks like a good candidate for clojure-contrib
I certainly wouldn't want to create it from scratch if I did need it.
> (defn random-permutation [s]
> "Return a random permutation of this seq."
> (let [arr (to-array s) len (alength arr)]
> (dotimes [i (dec len)]
> (let [r (+ i (rand-int (- len i))),
> prev (aget arr i)]
> (aset arr i (aget arr r))
> (aset arr r prev)))
> (seq arr)))
>
> (defn random-element [s]
> "Return a random element of this seq"
> (nth s (rand-int (count s))))
I wouldn't argue against those.
> (defn maximal-elements [f s]
> "Return a seq of elements of s maximizing (f elt)."
> (when (seq s)
> (loop [max-elts (first s),
> max-val (f (first s)),
> rest-elts (rest s)]
> (if (empty? rest-elts)
> max-elts
> (let [next-val (f (first rest-elts))]
> (cond (< next-val max-val) (recur max-elts max-val (rest rest-
> elts))
> (= next-val max-val) (recur (cons (first rest-elts) max-elts) max-
> val (rest rest-elts))
> (> next-val max-val) (recur [(first rest-elts)] next-val (rest rest-
> elts))))))))
I'm having a hard time imagining when maximal-elements would be
useful. What have you used it for? Looks like a good candidate for
application code. :-)
> (import '(java.util HashSet))
> (defn distinct-elts? "Are all of the elements of this sequence
> distinct? Works on infinite sequences with repititions, making it
> useful for, e.g., detecting cycles in graphs."
> [s]
> (let [hs (HashSet.)]
> (loop [s (seq s)]
> (cond (empty? s) true
> (.contains hs (first s)) false
> :else (do (.add hs (first s)) (recur (rest s)))))))
Is there any reason the builtin 'distinct?' couldn't handle these
cases as well? What does "elts" stand for?
> (defn concat-elts "Lazily concatenate elements of a seq of seqs." [s]
> (when (seq s) (lazy-cat (first s) (concat-elts (rest s)))))
>
> (defn lazy-mapcat "Like mapcat but is lazy like map" [f s]
> (concat-elts (map f s)))
>
> (defn map-when "Like map but discards logically false entries"
> [fn & seqs]
> (filter identity (apply map fn seqs)))
Sufficient laziness is another conversation entirely. :-)
> (defn iterate-while [f x]
> (take-while identity (iterate f x)))
As I said, I've written this out myself. If it was included I would
certainly use it. And as you pointed out, the words in its name mean
exactly what they mean in the body. I'd vote for this.
> (defn chunk "Lazily break s into chunks of length n (or less, for the
> final chunk)."
> [n s]
> (when (seq s)
> (lazy-cons (take n s)
> (chunk n (drop n s)))))
This is so close to 'partition', that I think I'd prefer there be only
one, or at least have related names.
> (defn mevery? "Like every but takes multiple seq args like map."
> ([f & seqs]
> (or (some empty? seqs)
> (and (apply f (map first seqs))
> (recur f (map rest seqs))))))
Is that the same as this?
(defn my-mevery? [f & seqs]
(every? identity (apply map f seqs)))
Someone suggested all the functions that currently take a predicate
and one seq ought to take multiple seqs and work like 'map'. I
suppose this would include: every? any? some etc. Seems pretty
convenient, and I've not heard arguments against it. But without
that, I think I'd rather use 'identity' and 'apply map' than have an
m-version of all the seq functions.
> ;;; Maps
>
> (defn map-map "Like map, but expects f to return pairs/map entries
> that are combined to make a map return value."
> [f & maps] (reduce #(conj %1 %2) {} (apply map f maps)))
As discussed, perhaps (into {} (map f ...)) is good enough.
> (defmacro lazy-get "Like get but lazy about evaluating the default
> value"
> [m k d]
> `(if-let [pair# (find ~m ~k)]
> (second pair#)
> ~d))
I've never needed this. How expensive is your default value?
> (defn safe-get "Like get but throws an exception if not found"
> [m k]
> (lazy-get m k (throw (IllegalArgumentException. (format "Key %s not
> found" k)))))
Ah, I see now. Unless you're storing nils in your map, this is the
same as (or (m k) (throw ...))
I think I'd rather this be a feature of the map itself, rather than
depend on each 'get' to be a 'safe-get'. Perhaps if 'get' finds
nothing and no default value is given, it could check the map's
metadata for a :default-value or :throw-unless-found item.
> (defn merge-agree "Like merge but returns nil if there are
> inconsistent key/value pairs."
> ([] {})
> ([map] map)
> ([m1 m2 & maps]
> (when (every? (fn [[k v]] (= v (get m1 k v))) m2)
> (apply merge-agree (merge m1 m2) maps))))
Maybe if it accepted the = function as a arg (like merge-with) it
would be a little more general?
> (defn merge-reduce "Combines maps, reducing sets of values with same
> key. Assumes nil value = not present. The first map entry must be a
> real map, but the remaining arguments can be seqs of map entries/key-
> value pairs."
> ([f ] {})
> ([f m1 & maps]
> (reduce (fn [m [k v]]
> (if-let [v2 (get m k)]
> (assoc m k (f v2 v))
> (assoc m k v)))
> m1
> (concat-elts maps))))
As discussed, this is like the builtin merge-with.
> (defn prln "Print all arguments and return the first argument"
> [& args] (do (println (apply print-str args)) (first args)))
You use this for debugging?
> (defn xor [& args]
> (odd? (count (filter identity args))))
Never needed it.
> (defmacro forcat
> "Like for, but concatenates the results."
> [& args] `(concat-elts (for ~@args)))
Maybe instead of adding 'forcat' for symmetry with 'mapcat', we should
ask for 'mapcat' to be removed.
--Chouser
Chouser
>
> On Jan 13, 7:17 pm, "Nick Vogel" <voge...@gmail.com> wrote:
>> seq returns nil when a collection has no items. According to the
>> documentation for empty?, empty? is the same as (not (seq coll)) so you
>> should use seq for expressing the opposite of empty?
>
> According to my experiment, the code above works equally well whether
> 'seq or 'seq? is used. Intuitively (i.e. without actually knowing :)
> seq? is more efficient because it returns a boolean, not a newly-
> allocated sequence.
>
> So, either:
>
> 1. My experiment was wrong, and seq? is not a valid stand-in
> for seq in the above code.
Right on the first try! :-)
user=> (seq-chunk 2 [1 2 3 4 5])
((1 2) (3 4) (5))
user=> (seq?-chunk 2 [1 2 3 4 5])
nil
This is because a vector is not itself a seq, though it is seq-able.
Thus 'seq?' returns false, which 'seq' returns a sequence as long as
the vector is not empty.
> 2. My intuition is wrong, and 'seq? is not more efficient than 'seq.
Both are usually very fast -- fast enough to not worry about them, and
certainly fast enough that you should use the correct one rather than
the fast one. :-)
However, for the record, depending on the type of object being
examined, either may be faster. 100,000 times each, fastest to
slowest:
(seq '(1 2 3)))) ==> "Elapsed time: 8.086614 msecs"
(seq? '(1 2 3)))) ==> "Elapsed time: 11.290486 msecs"
(seq? [1 2 3]))) ==> "Elapsed time: 19.127055 msecs"
(seq [1 2 3]))) ==> "Elapsed time: 20.471575 msecs"
--Chouser
Michael Harrison (goodmike)
too have been turned off by CL in part because of the enormous number
of subtly distinct built-in functions. I'm partial to Scheme, though,
so maybe I'm best viewed as a fanatic on this point. :-)
That said, it does seem pleasantly Clojure-ish to have functions named
x-when, like "map-when". This is pretty self-explanatory, so I'd be
happy having these sorts of functions, especially when they do replace
complex chaining of other functions.
But I'm not afraid of using reduce, map, and apply to prepare values
to use as arguments to built-ins. And I'm not afraid to encapsulate
this into my own functions when I need to. I'm OK with building up a
bit of a language for whatever problem I'm tackling. (There's the
schemaniac in me again.) There's a golden mean between complexity and
clarity. At first glance, Jason's suggestions seem close to that mean
to me, and I really like what he's done.
Thanks for the good work, Jason!
Michael
On Jan 13, 5:42 pm, Chouser <chou...@gmail.com> wrote:
Jason Wolfe
> I also think it's unhelpful for codebases to stray further from the
> builtin functions than needed, because it makes that code harder to
> read as well. So I will consider each of these more carefully.
out the parts where I don't think comments are needed.
> > (defn combinations "Take a seq of seqs and return a lazy seq of
> > ordered combinations (pick 1 from each seq)"
> > [seqs]
> > (if (empty? seqs) '(())
> > (forcat [item (first seqs)]
> > (map #(cons item %) (combinations (rest seqs))))))
> (require '[clojure.contrib.lazy-seqs :as ls])
this.
> > (defn maximal-elements [f s]
> > "Return a seq of elements of s maximizing (f elt)."
> > (when (seq s)
> > (loop [max-elts (first s),
> > max-val (f (first s)),
> > rest-elts (rest s)]
> > (if (empty? rest-elts)
> > max-elts
> > (let [next-val (f (first rest-elts))]
> > (cond (< next-val max-val) (recur max-elts max-val (rest rest-
> > elts))
> > (= next-val max-val) (recur (cons (first rest-elts) max-elts) max-
> > val (rest rest-elts))
> > (> next-val max-val) (recur [(first rest-elts)] next-val (rest rest-
> > elts))))))))
>
> I'm having a hard time imagining when maximal-elements would be
> useful. What have you used it for? Looks like a good candidate for
> application code. :-)
particularly as (first (maximal-elements ...)) or (random-element
(maximal-elements ...)).
But, I can imagine it would be useful in many situations, i.e. things
like.
user> (maximal-elements second {:bob 10 :lisa 20 :peter 20})
([:peter 20] [:lisa 20])
There don't seem to be any other utils that pull out the "best" element
(s) of a seq, so there seems to be a role for this. But, I've only
needed it once so far, so maybe it's not so important.
> > (import '(java.util HashSet))
> > (defn distinct-elts? "Are all of the elements of this sequence
> > distinct? Works on infinite sequences with repititions, making it
> > useful for, e.g., detecting cycles in graphs."
> > [s]
> > (let [hs (HashSet.)]
> > (loop [s (seq s)]
> > (cond (empty? s) true
> > (.contains hs (first s)) false
> > :else (do (.add hs (first s)) (recur (rest s)))))))
>
> Is there any reason the builtin 'distinct?' couldn't handle these
> cases as well? What does "elts" stand for?
impression that apply always evaluated the entire arg-seq, but I just
realized that this isn't the case -- I guess it evaluates everything
but the "& rest" params, which can remain a lazy seq? So, for
instance, this works
user> (apply distinct? (interleave (iterate inc 0) (iterate inc 10)))
false
which was the whole point of distinct-elts? for me. So, ignore this
one too.
> > (defn concat-elts "Lazily concatenate elements of a seq of seqs." [s]
> > (when (seq s) (lazy-cat (first s) (concat-elts (rest s)))))
>
> > (defn lazy-mapcat "Like mapcat but is lazy like map" [f s]
> > (concat-elts (map f s)))
>
> > (defn map-when "Like map but discards logically false entries"
> > [fn & seqs]
> > (filter identity (apply map fn seqs)))
>
> Sufficient laziness is another conversation entirely. :-)
built-in concat (which seems to be used by mapcat) was changed to be
less eager (in the apply sense just mentioned above).
user> (take 0 (apply concat (report-seq "" [[1] [2] [3] [4] [5]])))
(first [1] )
(rest after [1] )
(first [2] )
(rest after [2] )
(first [3] )
(rest after [3] )
nil
This can end up being a real drag in situations like (apply concat
(map #(recurive-call ...) ...)), since it may result in an exponential
amount of extra work being done. Any chance of this being changed?
(It should be possible to get rid of the last 4 evaluations here,
without getting into the territory of our last conversation about
laziness :))
> > (defn chunk "Lazily break s into chunks of length n (or less, for the
> > final chunk)."
> > [n s]
> > (when (seq s)
> > (lazy-cons (take n s)
> > (chunk n (drop n s)))))
>
> This is so close to 'partition', that I think I'd prefer there be only
> one, or at least have related names.
forgotten about), but I think this one is at least as useful as
partition (You can recover partition from this one by (take-when #(=
(count %) n) (chunk n s))) but not vice-versa.) Any suggestions on a
name?
> > (defn mevery? "Like every but takes multiple seq args like map."
> > ([f & seqs]
> > (or (some empty? seqs)
> > (and (apply f (map first seqs))
> > (recur f (map rest seqs))))))
>
> Is that the same as this?
>
> (defn my-mevery? [f & seqs]
> (every? identity (apply map f seqs)))
> Someone suggested all the functions that currently take a predicate
> and one seq ought to take multiple seqs and work like 'map'. I
> suppose this would include: every? any? some etc. Seems pretty
> convenient, and I've not heard arguments against it. But without
> that, I think I'd rather use 'identity' and 'apply map' than have an
> m-version of all the seq functions.
seqs and work like map.
If not, I guess I agree with you that it's better not to create m-
versions of everything.
> > (defn map-map "Like map, but expects f to return pairs/map entries
> > that are combined to make a map return value."
> > [f & maps] (reduce #(conj %1 %2) {} (apply map f maps)))
>
> As discussed, perhaps (into {} (map f ...)) is good enough.
> > (defmacro lazy-get "Like get but lazy about evaluating the default
> > value"
> > [m k d]
> > `(if-let [pair# (find ~m ~k)]
> > (second pair#)
> > ~d))
>
> I've never needed this. How expensive is your default value?
typical use-case is where I am working with maps where values are big
structs tailored to the key, and nil values are allowed. Anyway, the
main purpose of this was for safe-get, which I definitely want to work
with nil values.
> > (defn safe-get "Like get but throws an exception if not found"
> > [m k]
> > (lazy-get m k (throw (IllegalArgumentException. (format "Key %s not
> > found" k)))))
>
> Ah, I see now. Unless you're storing nils in your map, this is the
> same as (or (m k) (throw ...))
>
> I think I'd rather this be a feature of the map itself, rather than
> depend on each 'get' to be a 'safe-get'. Perhaps if 'get' finds
> nothing and no default value is given, it could check the map's
> metadata for a :default-value or :throw-unless-found item.
safe-get. But, one or the other is definitely useful ... I've been
bitten several times by misspelled keys, which can create difficult-to-
find bugs.
> > (defn merge-agree "Like merge but returns nil if there are
> > inconsistent key/value pairs."
> > ([] {})
> > ([map] map)
> > ([m1 m2 & maps]
> > (when (every? (fn [[k v]] (= v (get m1 k v))) m2)
> > (apply merge-agree (merge m1 m2) maps))))
>
> Maybe if it accepted the = function as a arg (like merge-with) it
> would be a little more general?
not =)?
I suppose this should also take a default value like get, so one can
distinguish empty-map from inconsistent-maps. I've only found one use
for this one so far, so maybe others can chime in if they think they'd
find it useful.
> > (defn merge-reduce "Combines maps, reducing sets of values with same
> > key. Assumes nil value = not present. The first map entry must be a
> > real map, but the remaining arguments can be seqs of map entries/key-
> > value pairs."
> > ([f ] {})
> > ([f m1 & maps]
> > (reduce (fn [m [k v]]
> > (if-let [v2 (get m k)]
> > (assoc m k (f v2 v))
> > (assoc m k v)))
> > m1
> > (concat-elts maps))))
>
> As discussed, this is like the builtin merge-with.
actually really useful to be able to pass seqs of pairs or map entries
rather than maps.
This lets me do things like
user> (let [accounts {:bob 2, :annie 3}
deposits [[:bob 10] [:annie 3] [:chouser 100] [:bob 5]]
withdrawls [[:bob -2]]]
(merge-reduce + accounts deposits withdrawls))
{:chouser 100, :bob 15, :annie 6}
With merge-with, I think I would have to say
(apply merge-with + accounts (map (fn [[k v]] (hash-map k v)) (concat
deposits withdrawls)))
which is considerably uglier and less clear IMO. Any chance of
changing merge-with in this way?
(Note this is actually similar to clojure.set union/intersection/
difference, which only require that the first element be a set and the
other can be a seq. And, while I'm on this tangent, any reason those
functions don't accept arbitrary numbers of arguments?)
> > (defn prln "Print all arguments and return the first argument"
> > [& args] (do (println (apply print-str args)) (first args)))
>
> You use this for debugging?
> > (defn xor [& args]
> > (odd? (count (filter identity args))))
>
> Never needed it.
predicates (xor (p1 ...) (p2 ...)) is much nicer than
(let [p1? (p1 ...)
p2? (p2 ...)]
(or (and p1? (not p2?)) (and (not p1?) p2?)))
> > (defmacro forcat
> > "Like for, but concatenates the results."
> > [& args] `(concat-elts (for ~@args)))
>
> Maybe instead of adding 'forcat' for symmetry with 'mapcat', we should
> ask for 'mapcat' to be removed.
wouldn't be devastated to say (apply concat (map/for ..)) instead.
But, I do kinda think it should be both or neither.
-Jason
GS
>
> > 1. My experiment was wrong, and seq? is not a valid stand-in
> > for seq in the above code.
>
> Right on the first try! :-)
> user=> (seq-chunk 2 [1 2 3 4 5])
> ((1 2) (3 4) (5))
> user=> (seq?-chunk 2 [1 2 3 4 5])
> nil
>
> This is because a vector is not itself a seq, though it is seq-able.
> Thus 'seq?' returns false, which 'seq' returns a sequence as long as
> the vector is not empty.
clojure.core/seq?
([x])
Return true if x implements ISeq
It's asking a lot from me to know whether vectors implement ISeq.
> > 2. My intuition is wrong, and 'seq? is not more efficient than 'seq.
>
> Both are usually very fast -- fast enough to not worry about them, and
> certainly fast enough that you should use the correct one rather than
> the fast one. :-)
Thanks for your help, as always.
Gavin
Jason Wolfe
>
> clojure.core/seq?
> ([x])
> Return true if x implements ISeq
>
> It's asking a lot from me to know whether vectors implement ISeq.
If you don't know, you can always just ask the language :)
user> (ancestors (class [1 2 3]))
#{clojure.lang.IPersistentStack clojure.lang.Streamable
java.io.Serializable java.lang.Runnable clojure.lang.IFn
clojure.lang.IObj clojure.lang.Associative clojure.lang.Reversible
clojure.lang.Obj clojure.lang.Sequential java.util.RandomAccess
java.util.List clojure.lang.IPersistentVector
clojure.lang.APersistentVector clojure.lang.AFn java.lang.Object
clojure.lang.IPersistentCollection java.lang.Iterable
java.lang.Comparable java.util.Collection java.util.concurrent.Callable}
user> (contains? (ancestors (class [1 2 3])) clojure.lang.ISeq)
false
But, I agree; the seq/coll distinction can be very confusing at first,
especially when faced with seemingly contradictory outputs like
user> (= '(1) [1])
true
user> (= '() [])
false
user> {'(1) true}
{(1) true}
user> (get *1 [1])
true
user> (hash-map '(1) true)
{(1) true}
user> (get *1 [1])
nil
-Jason
Chouser
>
> user> (contains? (ancestors (class [1 2 3])) clojure.lang.ISeq)
> false
Also there's 'isa?':
user=> (isa? (class [1 2 3]) clojure.lang.ISeq)
false
user=> (isa? (class (seq [1 2 3])) clojure.lang.ISeq)
true
And 'instance?':
user=> (instance? clojure.lang.ISeq [1 2 3])
false
user=> (instance? clojure.lang.ISeq (seq [1 2 3]))
true
> But, I agree; the seq/coll distinction can be very confusing at first,
> especially when faced with seemingly contradictory outputs like
>
> user> (= '(1) [1])
> true
>
> user> (= '() [])
> false
Hm. That does seem rather odd. I wonder if that paper defining
'egal' address this kind of issue. I haven't read it yet -- perhaps
the time has come.
> user> {'(1) true}
> {(1) true}
>
> user> (get *1 [1])
> true
>
> user> (hash-map '(1) true)
> {(1) true}
>
> user> (get *1 [1])
> nil
The different map types use different kinds of equality, and not all
of these are even defined for all object types: Hash-maps use hash
functions, sorted-maps use a comparator, and array-maps use =
--Chouser
Rich Hickey
On Jan 14, 5:02 pm, Chouser <chou...@gmail.com> wrote:
>
> > user> (contains? (ancestors (class [1 2 3])) clojure.lang.ISeq)
> > false
>
> Also there's 'isa?':
>
> user=> (isa? (class [1 2 3]) clojure.lang.ISeq)
> false
> user=> (isa? (class (seq [1 2 3])) clojure.lang.ISeq)
> true
>
> And 'instance?':
>
> user=> (instance? clojure.lang.ISeq [1 2 3])
> false
> user=> (instance? clojure.lang.ISeq (seq [1 2 3]))
> true
>
> > But, I agree; the seq/coll distinction can be very confusing at first,
> > especially when faced with seemingly contradictory outputs like
>
> > user> (= '(1) [1])
> > true
>
> > user> (= '() [])
> > false
>
> Hm. That does seem rather odd.
Rich
Jason Wolfe
>>> true
>>
>>> user> (= '() [])
>>> false
>>
>> Hm. That does seem rather odd.
>
> Fixed - svn 1208.
Oh, I always assumed this was intentional ... I guess I never tried
switching the order of arguments. Well, that makes a bit more sense
then :).
Chouser
>
> To keep this manageable, I'll cut out the parts where I don't think
> comments are needed.
Good idea!
>> > (defn concat-elts "Lazily concatenate elements of a seq of seqs." [s]
>> > (when (seq s) (lazy-cat (first s) (concat-elts (rest s)))))
>>
>> > (defn lazy-mapcat "Like mapcat but is lazy like map" [f s]
>> > (concat-elts (map f s)))
>
> Yes. I would be happy to drop concat-elts and lazy-mapcat, if the
> built-in concat (which seems to be used by mapcat) was changed to be
> less eager (in the apply sense just mentioned above).
>
> user> (take 0 (apply concat (report-seq "" [[1] [2] [3] [4] [5]])))
> (first [1] )
> (rest after [1] )
> (first [2] )
> (rest after [2] )
> (first [3] )
> (rest after [3] )
> nil
>
> This can end up being a real drag in situations like (apply concat
> (map #(recurive-call ...) ...)), since it may result in an exponential
> amount of extra work being done. Any chance of this being changed?
> (It should be possible to get rid of the last 4 evaluations here,
> without getting into the territory of our last conversation about
> laziness :))
I don't see why the built-in concat couldn't be defined like yours.
>> > (defn chunk "Lazily break s into chunks of length n (or less, for the
>> > final chunk)."
>> > [n s]
>> > (when (seq s)
>> > (lazy-cons (take n s)
>> > (chunk n (drop n s)))))
>>
>> This is so close to 'partition', that I think I'd prefer there be only
>> one, or at least have related names.
>
> OK, I agree the name should be closer to partition (which I'd
> forgotten about), but I think this one is at least as useful as
> partition (You can recover partition from this one by (take-when #(=
> (count %) n) (chunk n s))) but not vice-versa.) Any suggestions on a
> name?
Perhaps 'partition-all' ?
I'd vote to have this in contrib.
>> (defn my-mevery? [f & seqs]
>> (every? identity (apply map f seqs)))
>
> Yes, I believe so ... that's a much nicer way of writing it.
>
>> Someone suggested all the functions that currently take a predicate
>> and one seq ought to take multiple seqs and work like 'map'. I
>> suppose this would include: every? any? some etc. Seems pretty
>> convenient, and I've not heard arguments against it. But without
>> that, I think I'd rather use 'identity' and 'apply map' than have an
>> m-version of all the seq functions.
>
> Yes, I would of course prefer that the built-in versions take multiple
> seqs and work like map.
> If not, I guess I agree with you that it's better not to create m-
> versions of everything.
These could be changed to act like 'my-mevery?' above. Is this a
complete list? --> every? not-every? some not-any?
I believe functions that return the values from the input seq are NOT
candidates for this? That includes:
filter remove take-while drop-while split-with
Instead the first three might suggest corollaries:
map-when map-when-not map-while
>> I think I'd rather this be a feature of the map itself, rather than
>> depend on each 'get' to be a 'safe-get'. Perhaps if 'get' finds
>> nothing and no default value is given, it could check the map's
>> metadata for a :default-value or :throw-unless-found item.
>
> I think I'd be equally happy with "safe maps" as you suggest as using
> safe-get. But, one or the other is definitely useful ... I've been
> bitten several times by misspelled keys, which can create difficult-to-
> find bugs.
I know people have asked previously for maps that know their own
default value. I suppose the metadata could be supplied directly:
#^{:default "not found"} (hash-map :a 1, :b 2)
That's a bit ugly, but you could use it on {} literals. Otherwise I
guess you'd need extra constructors.
(hash-map-default "not found" :a 1, :b 2)
(sorted-map-throw :a 1, :b 2)
(safe-array-map :a 1, :b 2)
Hm. Is it worth it? Any naming ideas?
>> > (defn merge-agree "Like merge but returns nil if there are
>> > inconsistent key/value pairs."
>> > ([] {})
>> > ([map] map)
>> > ([m1 m2 & maps]
>> > (when (every? (fn [[k v]] (= v (get m1 k v))) m2)
>> > (apply merge-agree (merge m1 m2) maps))))
>>
>> Maybe if it accepted the = function as a arg (like merge-with) it
>> would be a little more general?
>
> Well, then which value would you use (if two values are my-equal but
> not =)?
Oh. Good point. I guess I was thinking it would be like
'merge-with', but if the conflict resolution function returned nil or
false, you'd get back a nil instead of a map.
> With merge-with, I think I would have to say
> (apply merge-with + accounts (map (fn [[k v]] (hash-map k v)) (concat
> deposits withdrawls)))
>
> which is considerably uglier and less clear IMO. Any chance of
> changing merge-with in this way?
Yes, it would just require replacing a use of 'key' and 'value' with
'first' and 'second' in the definition of 'merge-with'. I can't think
of a good reason not to do this.
--Chouser
Jason Wolfe
OK, great. I then ask that the core concat be changed to
(defn my-concat [& args]
(when (seq args)
(lazy-cat (first args) (my-concat (rest args)))))
which has more or less the desired effect:
user> (take 0 (apply my-concat (report-seq "" [[1] [2] [3] [4]])))
(first [1] )
(rest after [1] )
nil
Actually, this is still slightly more eager than concat-elts:
user> (take 0 (concat-elts (report-seq "" [[1] [2] [3] [4]])))
(first [1] )
nil
which seems to be due to the definition of apply. Why does the rest
get evaluated above?
>>>> (defn chunk "Lazily break s into chunks of length n (or less, for
>>>> the
>>>> final chunk)."
>>>> [n s]
>>>> (when (seq s)
>>>> (lazy-cons (take n s)
>>>> (chunk n (drop n s)))))
>>>
>>> This is so close to 'partition', that I think I'd prefer there be
>>> only
>>> one, or at least have related names.
>>
>> OK, I agree the name should be closer to partition (which I'd
>> forgotten about), but I think this one is at least as useful as
>> partition (You can recover partition from this one by (take-when #(=
>> (count %) n) (chunk n s))) but not vice-versa.) Any suggestions
>> on a
>> name?
>
> Perhaps 'partition-all' ?
> I'd vote to have this in contrib.
Works for me, thanks.
>>> (defn my-mevery? [f & seqs]
>>> (every? identity (apply map f seqs)))
>>
> These could be changed to act like 'my-mevery?' above. Is this a
> complete list? --> every? not-every? some not-any?
Sounds perfect ... I think that's all of them.
> I believe functions that return the values from the input seq are NOT
> candidates for this? That includes:
> filter remove take-while drop-while split-with
>
> Instead the first three might suggest corollaries:
> map-when map-when-not map-while
Perfect.
>>> I think I'd rather this be a feature of the map itself, rather than
>>> depend on each 'get' to be a 'safe-get'. Perhaps if 'get' finds
>>> nothing and no default value is given, it could check the map's
>>> metadata for a :default-value or :throw-unless-found item.
>>
>> I think I'd be equally happy with "safe maps" as you suggest as using
>> safe-get. But, one or the other is definitely useful ... I've been
>> bitten several times by misspelled keys, which can create difficult-
>> to-
>> find bugs.
>
> I know people have asked previously for maps that know their own
> default value. I suppose the metadata could be supplied directly:
>
> #^{:default "not found"} (hash-map :a 1, :b 2)
>
> That's a bit ugly, but you could use it on {} literals. Otherwise I
> guess you'd need extra constructors.
>
> (hash-map-default "not found" :a 1, :b 2)
> (sorted-map-throw :a 1, :b 2)
> (safe-array-map :a 1, :b 2)
>
> Hm. Is it worth it? Any naming ideas?
Well, I've thought about this a bit more.
I guess I'm a bit wary about your proposal, since as far as I know
none of the existing core functions (except metay-things like ns,
doc, ..) have their behavior changed by (especially, explicitly user-
set) metadata. (Well, to be more precise, I don't know much of
anything about how metadata is used by the core). Right now, I use
metadata in a couple places in my code to store information about
objects that I don't want to affect their equality semantics. If I
could accidentally clobber metadata used internally by Clojure in
doing so, that could be a bad thing. Is there a list somewhere of
"reserved metadata keywords" that I shouldn't be using in this way?
If not, after adding this it seems one would definitely be needed.
On the other hand, part of me still likes safe-get, since the meaning
is clear in the code: I will get the object associated with this key,
or die trying. I may want to write a function that takes an arbitrary
map and safe-gets some specific key, regardless of where the map came
from. Or, I might like to sometimes "safe-get" and sometimes just
"get" from the same map (i.e., if there are some required and some
optional keys). Basically, it seems to me that most of the time,
whether you want get or safe-get is a function of the *use* of the map
and not the map itself.
That being said, I guess there may be times when you really want the
map itself to be safe (e.g., structs-maps?) or have a particular
default value (come to think of it, I think I wanted this just
yesterday). So maybe, both couldn't hurt? The names you give seem
fine, or maybe (hash-map-no-default ...) or (hash-map-safe ...) and
(hash-map-with-default def ...).
>>>> (defn merge-agree "Like merge but returns nil if there are
>>>> inconsistent key/value pairs."
>>>> ([] {})
>>>> ([map] map)
>>>> ([m1 m2 & maps]
>>>> (when (every? (fn [[k v]] (= v (get m1 k v))) m2)
>>>> (apply merge-agree (merge m1 m2) maps))))
>>>
>>> Maybe if it accepted the = function as a arg (like merge-with) it
>>> would be a little more general?
>>
>> Well, then which value would you use (if two values are my-equal but
>> not =)?
>
> Oh. Good point. I guess I was thinking it would be like
> 'merge-with', but if the conflict resolution function returned nil or
> false, you'd get back a nil instead of a map.
Hmmm, but sometimes I like to use nil/false values ... anyway, I'll
agree that this utility has fairly narrow use cases and perhaps isn't
worth putting in contrib.
>> With merge-with, I think I would have to say
>> (apply merge-with + accounts (map (fn [[k v]] (hash-map k v)) (concat
>> deposits withdrawls)))
>>
>> which is considerably uglier and less clear IMO. Any chance of
>> changing merge-with in this way?
>
> Yes, it would just require replacing a use of 'key' and 'value' with
> 'first' and 'second' in the definition of 'merge-with'. I can't think
> of a good reason not to do this.
>
That would be perfect.
-Jason
Jason Wolfe
> But I'm not afraid of using reduce, map, and apply to prepare values
> to use as arguments to built-ins. And I'm not afraid to encapsulate
> this into my own functions when I need to. I'm OK with building up a
> bit of a language for whatever problem I'm tackling. (There's the
> schemaniac in me again.) There's a golden mean between complexity and
> clarity. At first glance, Jason's suggestions seem close to that mean
> to me, and I really like what he's done.
>
> Thanks for the good work, Jason!
Rich and all the other contributors who have made Clojure so great,
and have been extremely understanding and helpful throughout my
learning process.
-Jason
Jason Wolfe
> > map-when map-when-not map-while
>
> Perfect.
>
useful? Unless you're interested in the difference between nil and
false, I think it's equivalent to (replicate (count (remove pred s))
nil).
Also, can I do anything to help implement/integrate the changes we've
agreed upon thus far?
Thanks, Jason
Jason Wolfe
> > > (import '(java.util HashSet))
> > > (defn distinct-elts? "Are all of the elements of this sequence
> > > distinct? Works on infinite sequences with repititions, making it
> > > useful for, e.g., detecting cycles in graphs."
> > > [s]
> > > (let [hs (HashSet.)]
> > > (loop [s (seq s)]
> > > (cond (empty? s) true
> > > (.contains hs (first s)) false
> > > :else (do (.add hs (first s)) (recur (rest s)))))))
>
> > Is there any reason the builtin 'distinct?' couldn't handle these
> > cases as well? What does "elts" stand for?
>
> I suppose this is the same as (apply distinct? s).
fails rather than returning true. Does adding a 0-arg case to
distinct? sound reasonable?
-Jason
Jason Wolfe
if the first element ends up being maximal. Here's a fixed version.
(defn maximal-elements [f s]
"Return a seq of elements of s maximizing (f elt)."
(when (seq s)
Jason Wolfe
> (defn random-permutation [s]
> "Return a random permutation of this seq."
> (let [arr (to-array s) len (alength arr)]
> (dotimes [i (dec len)]
> (let [r (+ i (rand-int (- len i))),
> prev (aget arr i)]
> (aset arr i (aget arr r))
> (aset arr r prev)))
> (seq arr)))
(defn shuffle-java
"Shuffles coll using a Java ArrayList. Blows shuffle out of the
water on
speed and space."
[coll]
(let [l (ArrayList. coll)]
(Collections/shuffle l)
(seq l)))
is both much simpler and much much much faster than my implementation.
[1] http://groups.google.com/group/clojure/browse_thread/thread/180842eb58c58370
> (defmacro lazy-get "Like get but lazy about evaluating the default
> value"
> [m k d]
> `(if-let [pair# (find ~m ~k)]
> (second pair#)
> ~d))
(defmacro lazy-get "Like get but lazy about evaluating the default
value"
[m k d]
`(if-let [pair# (find ~m ~k)]
~d))
Changing the "second" to a "val" speeds both this and safe-get up by
more than a factor of 4! [2]
[2] http://w01fe.com/blog/2009/01/more-on-clojure-map-accessor-speeds/
Also, once again (Chouser?), is there anything I can do to help
implement the changes we've talked about? I can try to make patches
for the changes in core, and/or improve and document other utilities
for clojure.contrib, if that would be helpful.
Cheers,
Jason
Chouser
>
> Also, once again (Chouser?), is there anything I can do to help
> implement the changes we've talked about?
I'm sorry, Jason, I really thought I answered this the first time you
asked. But I can't find any such answer in the archives, so I must
have been mistaken.
> I can try to make patches for the changes in core, and/or improve
> and document other utilities for clojure.contrib, if that would be
> helpful.
I think it might be most useful to split up the changes into smaller
chunks, perhaps as small as a single function each, or maybe a couple
of related functions together. Each of these chunks can be posted as
a feature-request on the issues page. This would allow any further
discussion of implementation details and any patches to be collected,
and would allow Rich to accept or reject each chunk independently.
--Chouser
Jason Wolfe
> asked. But I can't find any such answer in the archives, so I must
> have been mistaken.
No worries, thanks for all your help with this.
>> I can try to make patches for the changes in core, and/or improve
>> and document other utilities for clojure.contrib, if that would be
>> helpful.
>
> I think it might be most useful to split up the changes into smaller
> chunks, perhaps as small as a single function each, or maybe a couple
> of related functions together. Each of these chunks can be posted as
> a feature-request on the issues page. This would allow any further
> discussion of implementation details and any patches to be collected,
> and would allow Rich to accept or reject each chunk independently.
OK, I'll get on this then. It this just for changes to core, or
should I post proposed functions for contrib there/ on the contrib
issues page too? If not, what should I do with them?
Thanks!
Jason
Stephen C. Gilardi
OK, I'll get on this then. It this just for changes to core, or
should I post proposed functions for contrib there/ on the contrib
issues page too? If not, what should I do with them?
Chouser
>
> I recommend that proposed changes for clojure-contrib be tracked as
> clojure-contrib issues.
I agree.
> My understanding of the issue policy for Clojure is that Rich would still
> like to approve them either here or on #clojure (irc) before they're
> entered. (ref: his recent posting on the topic.) I'm not aware of whether or
> not he has approved entering issue(s) in this case.
That's a very interesting point. My impression has been that lack of
objection from him here (or on IRC) is sufficient approval to post
something on the issues page. If he was completely opposed to these
proposals, he's had several days to make his opinion known.
Of course that's no guarantee that the issue or any particular patch
will be approved, but it makes sure that issues and proposed patches
aren't lost in archives.
Hopefully Rich will clarify his wishes on this.
--Chouser
Rich Hickey
On Jan 20, 4:15 pm, Chouser <chou...@gmail.com> wrote:
first. If a member of clojure-contrib approves, they should become
clojure-contrib issues, and any related patches attached to such
issues. clojure-contrib members can incorporate (or not!) those
patches (when coming from registered contributors), check them in, and
resolve the issues.
I.e., I am trusting the members of clojure-contrib to be its stewards.
If I think something is off track I'll chime in.
Thanks all for your help and effort!
Rich
Jason Wolfe
Thanks Rich. But, I think this answers only one of the questions at
hand (about clojure.contrib issues). The other question (to which I
think Chouser was referring above) was about issues for Clojure core,
and whether or not an explicit sign-off from you was desired before
they are posted there. This came up in this thread since after
discussing with Chouser, several of my utilities seemed better-suited
as changes/additions to Clojure core rather than contrib.
-Jason
Rich Hickey
I didn't find any of them compelling enough for core just yet. Putting
them in contrib first lets people try them out and refine them, point
out redundancies and conflicts etc.
As a general rule I haven't pulled many simple combining functions
from contrib, as they just pollute the namespaces. Plus, I don't think
things are as cut and dried as you make out, for instance I'd expect
map-when and iterate-while to take predicates.
Rich
Jason Wolfe
> them in contrib first lets people try them out and refine them, point
> out redundancies and conflicts etc.
>
> As a general rule I haven't pulled many simple combining functions
> from contrib, as they just pollute the namespaces. Plus, I don't think
> things are as cut and dried as you make out, for instance I'd expect
> map-when and iterate-while to take predicates.
I'd like to see in core are:
- 0-arg distinct? returns true, not an exception (so (apply distinct?
nil) = true)
- rewrite concat so that (apply concat seq) doesn't evaluate the
first three elements of seq
- make every?, not-every?, some, not-any? take multiple seq args like
map, i.e., (every? not= [1 2 3] [2 3 4])
- allow arguments to merge-with after the first to be lists of pairs.
More details on each of these are above. None of these affect the
existing functionality of these core functions, and it doesn't seem to
make sense to create new versions in clojure.contrib just to add the
new functionality. Will you accept issues for any of these?
Thanks,
Jason