Clojure 1.3 Alpha 7
Christopher Redinger
1 Changes from 1.3 Alpha 5 to 1.3 Alpha 6
2 Changes from 1.3 Alpha 4 to 1.3 Alpha 5
3 Changes from 1.3 Alpha 3 to 1.3 Alpha 4
4 Changes from 1.3 Alpha 2 to 1.3 Alpha 3
5 Changes from 1.3 Alpha 1 to 1.3 Alpha 2
6 Changes from 1.2 to 1.3 Alpha 1
7 About Alpha Releases
Issue Tracking: http://dev.clojure.org/jira
= 0 Changes from 1.3 Alpha 6 to 1.3 Alpha 7 (05/13/2011)
* print/read syntax for defrecords (CLJ-374)
* several primitive math improvements:
(CLJ-184, CLJ-784, CLJ-690, CLJ-782)
* case now handles hash collisions (CLJ-426)
= 1 Changes from 1.3 Alpha 5 to 1.3 Alpha 6 (03/11/2011)
* improved startup time
* several "holding onto head" fixes (CLJ-708)
* internal keyword map uses weak refs
* fix perf on some numeric overloads (CLJ-380)
* detect and report cyclic load dependencies (CLJ-8)
= 2 Changes from 1.3 Alpha 4 to 1.3 Alpha 5 (01/14/2011)
* pprint respects *print-length*
* into-array now coerces numeric types
* Java's line.separator property for newline
* compilation and deployment via Maven
= 3 Changes from 1.3 Alpha 3 to 1.3 Alpha 4 (12/12/2010)
* normalized unchecked-* fn names
* added *unchecked-math* support
* fixes to binding conveyance (and *agent*)
= 4 Changes from 1.3 Alpha 2 to 1.3 Alpha 3 (11/05/2010)
* fixed filter performance issue introduced in 1.3A2
* with-redefs macro (useful for stubbing)
* print-table
= 5 Changes from 1.3 Alpha 1 to 1.3 Alpha 2 (10/10/2010)
* code path for using vars is now *much* faster for the common case,
and you must explicitly ask for :dynamic bindability
* new: clojure.reflect/reflect
http://dev.clojure.org/display/design/Reflection+API
* new: clojure.data/diff
= 6 Changes from 1.2 to 1.3 Alpha 1 (09/23/2010)
* enhanced primitive support
(http://dev.clojure.org/display/doc/Enhanced+Primitive+Support)
* better exception reporting
* ancillary namespaces no longer auto-load on startup:
clojure.set, clojure.xml, clojure.zip
= 7 About Alpha Releases
1.3 is the first release of Clojure that will include a series of
alpha builds. We are adding these builds to support maven and
leiningen users, who want a specific artifact that they can target (as
opposed to building from master or "moving-target" snapshots).
If you are the kind of person who used to track master by building
from source, but no longer do so because you are using maven or
leiningen, alpha releases are for you.
Christopher Redinger
Clojure/core
http://clojure.com
Sean Corfield
To save me digging into git commits, do you (or anyone else) happen to
know for sure when "realized?" was added? It's not in the release
notes below... I *think* it's new in Alpha 7...?
Sean
Christopher Redinger
To save me digging into git commits, do you (or anyone else) happen toknow for sure when "realized?" was added? It's not in the release
notes below... I *think* it's new in Alpha 7...?
You are correct, it was added in Alpha 7.
Sean Corfield
> You are correct, it was added in Alpha 7.
Thanx Christopher.
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/
Railo Technologies, Inc. -- http://www.getrailo.com/
"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)
Tassilo Horn
> = 0 Changes from 1.3 Alpha 6 to 1.3 Alpha 7 (05/13/2011)
>
> [...]
> * case now handles hash collisions (CLJ-426)
I've just updated my git master checkout, built it, and tried to use it
in my project. But still I get a hash collision error:
java.lang.IllegalArgumentException: No distinct mapping found
at clojure.core$min_hash.invoke (core.clj:5805)
So it seems to me that CLJ-426 is not really fixed...
Bye,
Tassilo
Christopher Redinger
java.lang.IllegalArgumentException: No distinct mapping found
at clojure.core$min_hash.invoke (core.clj:5805)
Can you supply a small example where this is happening?
For reference, this code that failed in alpha6:
user=> *clojure-version*
{:major 1, :minor 3, :incremental 0, :qualifier "alpha6"}
user=> (defn buggy-case [n]
(case (int n)
0 :null
1 :load
0x70000000 :loproc))
CompilerException java.lang.IllegalArgumentException: No distinct mapping found, compiling:(NO_SOURCE_PATH:17)
Now works in alpha7:
user=> *clojure-version*
{:major 1, :minor 3, :incremental 0, :qualifier "alpha7"}
user=> (defn buggy-case [n]
(case (int n)
0 :null
1 :load
0x70000000 :loproc))
#'user/buggy-case
Thanks!
Alan
"1" hashes to 49, so there's a hash collision.
Christopher Redinger
I'm not on alpha7 yet, but what about (case 49 "1" 'string 49 'int)?
"1" hashes to 49, so there's a hash collision.
user=> (case 49 "1" 'string 49 'int)
int
Tassilo Horn
Hi Christopher,
> Can you supply a small example where this is happening?
No, I don't have a small, standalone example. :-(
The problem appeared when I converted my project from multimethods to
protocols. You can clone my mercurial project and run the tests, which
will trigger the error for the `coupling-by-objects' test.
$ hg clone https://anonymous:sec...@hg.uni-koblenz.de/horn/funql
The project needs some java library which you can download from
http://www.uni-koblenz.de/~horn/jgralab.jar
and put into a local maven repository using:
$ mvn install:install-file \
-Dfile=/path/to/jgralab.jar \
-DgroupId=ist \
-DartifactId=jgralab \
-Dversion=1.0.0 \
-Dpackaging=jar
Then fetch the deps and run the tests with either
$ cake deps
$ cake test
or
$ lein deps
$ lein test
> For reference, this code that failed in alpha6:
> [...]
Well, that works fine now:
user> *clojure-version*
{:interim true, :major 1, :minor 3, :incremental 0, :qualifier "master"}
user> (defn buggy-case [n]
(case (int n)
0 :null
1 :load
0x70000000 :loproc))
#'user/buggy-case
Bye,
Tassilo
ataggart
case, though it is still used by the protocols internals, so that's
what's running into the collision.
Tassilo Horn
Hi,
> The min-hash function throwing that exception is no longer used by
> case, though it is still used by the protocols internals, so that's
> what's running into the collision.
Ah, ok. I've thought protocols dispatch using `case', so I expected my
problem being fixed.
Is there already a bug report for the hash collisions on protocol
dispatch?
Bye,
Tassilo
Jules
[jules@megalodon dada]$ java -jar ~/.m2/repository/org/clojure/clojure/1.3.0-alpha6/clojure-1.3.0-alpha6.jar
Clojure 1.3.0-alpha6
user=> (defrecord Foo [a b c d e f g h i j k l m n o p q r s t])
user.Foo
user=>
[jules@megalodon dada]$ java -jar ~/.m2/repository/org/clojure/clojure/1.3.0-alpha7/clojure-1.3.0-alpha7.jar
Clojure 1.3.0-alpha7
user=> (defrecord Foo [a b c d e f g h i j k l m n o p q r s t])
CompilerException java.lang.RuntimeException: Can't specify more than 20 params, compiling:(NO_SOURCE_PATH:1)
user=>
sorry to the bearer of bad tidings :-(
Jules
Ken Wesson
It's actually worse than that! There are exactly twenty slots there,
so even though it's saying "can't specify more than 20" it's actually
choking on any amount more than NINETEEN.
--
Protege: What is this seething mass of parentheses?!
Master: Your father's Lisp REPL. This is the language of a true
hacker. Not as clumsy or random as C++; a language for a more
civilized age.
Christopher Redinger
Is there already a bug report for the hash collisions on protocol
dispatch?
Fogus
couple of auxiliary functions (for a record R) named map->R and ->R.
The first takes a map and returns a function of its contents. The
second takes the same number of arguments as the record constructor.
It's the definition of this second function that is reporting an
error. It's easy enough to fix, but I guess the question is should it
be fixed? That is, do people define records with 19+ fields? Does
this pose a problem for existing code?
:F
László Török
Hi,
What's the incremental runtime cost of increasing the max number of fields to 40? :-)
Las
sent from my mobile device
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To p...
David Nolen
pmbauer
Armando Blancas
composition, which you can do after you bring in a tuple into a
record. Here's probably where this limitation in Clojure hurts the
most, even when tables are fully normalized.
Fogus
However, there are some existing caveats:
1. The 20+ args path is much slower than the <20 args path. This is a
limitation that may not go away unless the <20 function args
limitation does.
The speed difference comes from the ability to perform (R. ~@args) in
the <20 path and the need for (into record-of-<20-args built-map-of-
remaining-args) in the 20+ path. (note paths determined at compile
time)
2. Any extra values passed to ->R in the 20+ case are lost.
e.g.
(defrecord R [a ... u])
(->R 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
;=> #user.R{:a 1, :b 2, :c 3, :d 4, :e 5, :f 6, :g 7, :h 8, :i 9, :j
10, :k 11, :l 12, :m 13, :n 14, :o 15, :p 16, :q 17, :r 18, :s 19, :u
21, :t 20}
22 is missing.
Thoughts?
Ken Wesson
> On May 20, 2011 7:55 PM, "Fogus" <mef...@gmail.com> wrote:
>> In the alpha7 release the defrecord macro was changed to generate a
>> couple of auxiliary functions (for a record R) named map->R and ->R.
>> The first takes a map and returns a function of its contents. The
>> second takes the same number of arguments as the record constructor.
>> It's the definition of this second function that is reporting an
>> error. It's easy enough to fix, but I guess the question is should it
>> be fixed? That is, do people define records with 19+ fields? Does
>> this pose a problem for existing code?
>
> What's the incremental runtime cost of increasing the max number of fields
> to 40? :-)
Oh, there's no need for that. Just add a little smarts to the
constructor generating macro to detect 19+ fields and then define it
as
(defn ->Foo [& fields]
(if (not= (count fields) whatever)
(throw-arity-exception)
(Foo. (nth fields 0) (nth fields 1) (nth fields 2) ...)))
or maybe for efficiency use a let cascade:
(let [p1 (first fields)
fields (rest fields)
p2 (first fields)
fields (rest fields)
...]
...)
Of course there's probably some JVM limit on the number of constructor
args, too.
I'd suggest if you have records that large to consider restructuring
them to have a smaller number of fields that are more composite --
seqs, vectors, sets, maps, even nested records. Surely there's *some*
structure beyond a heterogeneous assortment of twenty-plus objects?
Jules
<rant>
I think this issue reinforces my belief that arbitrary limits are bad.
Re. defrecord - Should we put the onus on everyone using defrecord to manipulate wide datasets to remember that there is an arbitrary limit of 19 fields, or put some smarts into the defrecord macro ?
I think we are in agreement that we should put some smarts in the defrecord macro.
But, hold on, why do we need smarts in the defrecord macro ? Why ? because of an arbitrary limit in the number of params that a function can handle.
So, why don't we put the smarts around function compilation/application to handle any number of params (within the JVMs ability). This way, we can avoid having to have smarts everywhere that a macro might expand to a function call with num params > 19. I have direct experience of being bitten by this.
If handling this many params means a performance hit, then, by all means lets log a warning about it, but we should not trip up perfectly valid code.
</rant>
Ah! that feels better :-)
Jules
Ken Wesson
+1. I think this can be done. It just needs to be that 20+ params
routes through .applyTo/applyToHelper instead of the .invoke methods.
There will no doubt be a performance hit though, and perhaps a
substantial one.
daly
One of the useful features of lisp programming is that limits are mostly
constrained by the amount of available memory. This isn't always true
but the exceptional cases are very rare.
This becomes important because people generally write automatic code
generator macros to support their domain specific languages. While you
might never code a 20 parameter defrecord by hand it is almost certain
that some macro will generate code well past that limit.
I am working with code that auto-generates variables and collects them
into the "let" form so there can be hundreds of them, mostly temporary
variables that were gensym'ed. If the code were moved to keep the
variables in a defrecord I would expect it to "just work".
I'm surprised that defrecord isn't a hashed trie data structure of
some sort, allowing the usual 2^32 entries.
Tim Daly
Literate Software
da...@literatesoftware.com
Ken Wesson
> I'm surprised that defrecord isn't a hashed trie data structure of
> some sort, allowing the usual 2^32 entries.
For speed, it's a Java class with instance variables for the
predefined keys (and a hash trie for any non-standard-for-that-record
keys that are added to a particular instance).
IMO, records should support anything maps support -- in particular,
they should support near-arbitrary numbers of entries. If that means
that bigger ones (in terms of numbers of predefined keys) switch to a
less performant hash trie for their predefined keys, so be it. It's
not unlike promoting integers to BigIntegers in arithmetic that way.
(Similarly, fns should be overloadable for specific arities over 20,
with any call with over 20 parameters going through .applyTo instead
of a .invoke, and then being checked for arity. For instance, if the
fn can take 1, 15, 25, or 30 & more arguments, and you pass it 27,
applyTo gets called and then blows up since it wants either 25 or over
29 arguments, but not 27, in that case. There'll be a performance
penalty in calling it with over 20 arguments without varargs, the same
as calling it with varargs or with apply, caused by having to box the
args into a seq and unbox them again, but that beats it not working at
all.)
David Nolen
IMO, records should support anything maps support -- in particular,
they should support near-arbitrary numbers of entries.
Ken Wesson
The suggestions would make records (and functions) no slower for the
situations where they currently work at all, and would make them work
in situations where they currently fail.
To make them stay fast, without hitting ivar and other limits, you'd
have to do something fundamentally new (and a bit icky): use an array
of Object under the hood to hold the contents/params. There'd still be
a slight penalty for "huge" arglists and records, due to the second
indirection to get from record pointer to array pointer to element in
the case of records, and the indirection to get the arg out of the
array in the case of functions. This also couldn't be done with
deftype (as it would break volatile mutable volatility).
Getting rid of the indirection would require either fundamental JVM
changes (unlikely, at least in the near term) or making records
directly be object arrays "under the hood" rather than deftypes. Then
records aren't their own classes and a lot of how type and .foo
dispatch works for records would have to be changed. Also, records
become thread-unsafe if there's any way to break "hygiene" and
directly get at the array cells to mutate them. I do not recommend
going that far.
Frankly, I'd prefer "large" records just use hash tries. They're
tried, tested, and true (so to speak) and support structure sharing,
unlike anything directly backed by Object arrays. Furthermore, they
won't be any slower to access in the case of <33 fields than Object
arrays and get slower only logarithmically, and it's a very
slow-growing logarithm at that -- in particular, there's only one more
indirection for records with 33-1024 fields. Locality also stays good
(for the field value pointers, rather than the value objects
themselves) for up to 32 fields.
So, one indirection for defined fields in records of up to 20 defined
fields, and for the first 20 defined fields of any record; two for the
next 32 fields of any record; three for the next 992 fields of any
record; past that, seq becomes more efficient for traversal than a
series of direct field accesses that eventually hits them all (and
especially has superior memory locality), and just as efficient as
traversing, say, a vector or a hash-map.