Scouring the Clojure-verse for collections that need Clojure 1.6 hash updates

Skip to first unread message

Andy Fingerhut

Feb 14, 2014, 2:36:53 PM2/14/14
It would be great if all Clojure contrib libraries were updated (if
needed) for Clojure 1.6's new hash function, before Clojure 1.6 is

It would also be nice to update to 3rd party Clojure libraries that
implement their own custom collections, that would otherwise still
calculate old hash values.  Having a custom set or map that implements
= correctly when comparing against other set and map implementations,
but still calculates the old hash function, can lead to bugs.  For
example, not being able to find a set element/map key if those
collections are themselves used as elements in PersistentHashSet or
keys in PersistentHashMap collections.

Summary of current hash update status:

+ 1 test in Clojure has not been updated for new hash function

+ at least 4 Clojure contrib libraries still need updates

+ 1 3rd party library definitely needs updates, and I'm looking for

More details below, if you are interested.  Please do reply if you can
think of any other Clojure contrib or 3rd party libraries not already
mentioned below that implement collections that might need updating.


+ 1 test in Clojure has not been updated for new hash function

Rich disabled a few failing test in Clojure itself when making the
hash changes in this commit:

I believe all but one of those disabled tests have since been updated
to be independent of Clojure's hash function with this commit:

The only exception I know of is a still-commented-out test in file
test/clojure/test_clojure/java_interop.clj at or near line 311:

    ;;(test-to-passed-array-for hash-set)

+ at least 4 Clojure contrib libraries still need updates

Several Clojure contrib projects had test failures when Clojure
changed its hash functions, but have since been updated to correct

    tools.namespace -
    tools.nrepl -

There were probably one or two more such projects in the past, but I
have forgotten the details.

Projects that have test failures, and patches written that correct the
test failures, but not committed yet:

    core.cache -
    core.match -

Projects that have no test failures, but only because their Maven-run
test suites currently contain no tests that depend upon Clojure's hash
function -- if they did, they would have test failures.  There are
tests to catch this in Zach Tellman's collection-check library, but
they are currently only run via Leiningen's 'lein test', not Maven.
The tickets mentioned have patches adding such tests and updating the
hash functions for Clojure 1.6, but the patches could use updating to
take advantage of the Clojure APIs Alex Miller added to calculate
hashes of ordered and unordered collections:

    core.rrb-vector -
    data.avl -

+ 1 3rd party library definitely needs updates, and I'm looking for

There are other 3rd party libraries that could use updating for
Clojure 1.6's new hash functions.  Off the top of my head, this one
definitely needs updating:

    immutable-bitset -

There are others that I don't know whether updates are needed:

     core.matrix -
     flatland/useful -
     flatland/ordered -

Does anyone know of any other collection implementations to check?


Feb 14, 2014, 3:07:24 PM2/14/14
data.finger-tree attempts to participate in equality and hashing with
the core collections. Not sure how many combinations are covered in
the tests.

Andy Fingerhut

Feb 15, 2014, 5:19:10 PM2/15/14
Proposed patch to update data.finger-tree here:

It appears like there were no tests for the hash functions yet.  I added a few.


You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To post to this group, send email to
Visit this group at
For more options, visit


Feb 16, 2014, 3:13:57 AM2/16/14
I can confirm that core.matrix is fine with the latest 1.6 beta.

(It doesn't make any assumptions about hash behaviour, so no big surprise there)

core.matrix doesn't make any assumptions about hashing, so I think it is fine. 

Mark Engelberg

Feb 16, 2014, 5:15:40 PM2/16/14
to clojure-dev
Instaparse will need some rethinking, since it relies on incremental hashing.  I believe that it will be possible to make it work with the 1.6 hashing scheme, since Rich took my request for supporting incremental hashing into account, but it will not be as trivial as some of the other collection updates.  Not sure yet whether it will be possible to accommodate the new hashing scheme and keep it backwards compatible with Clojure 1.5.


Feb 17, 2014, 8:50:01 AM2/17/14
Hi all,
may these projects: and be useful to anyone testing their own collections?


Alex Miller

Feb 17, 2014, 9:49:13 AM2/17/14

Zach Tellman

Feb 17, 2014, 5:35:23 PM2/17/14
Potemkin has a map implementation that is currently breaking for 1.6.  I will fix it this week.

Andy Fingerhut

Feb 18, 2014, 2:55:41 PM2/18/14
I have created a table at the bottom of the "Better Hashing" page to help track libraries that need updating for Clojure 1.6 hashing, and their current status regarding those updates.

Thanks for all those who have replied so far -- I believe I have included their feedback in that table.  I will attempt to follow up with those libraries periodically to see how things are going.   


Mark Engelberg

Feb 19, 2014, 2:13:19 AM2/19/14
to clojure-dev
I have finished the work needed to migrate instaparse's incremental hashing to Clojure 1.6.  I have also successfully made it backwards-compatible, using the compile-if macro you suggested.

I'm curious, what kind of speed difference are the rest of you seeing between 1.5.1 and 1.6?  On my code, 1.6 seems around 10-20% slower.  (Even using criterium to benchmark I get quite a bit of variation on 1.6 running times, so it's hard to pinpoint the speed difference better than that).
Reply all
Reply to author
0 new messages