better collections through code generation

[Zach Tellman]

At Clojure/West I spoke with Rich over lunch, and he mentioned that he was perfectly willing to consider putting something like clj-tuple [1] in the core implementation, as long as it was written in Java.  Towards this end, I made a thing [2].  It uses code that looks like this:

  https://github.com/ztellman/cambrian-collections/blob/master/generate/cambrian_collections/vector.clj

to generate code like this:

  https://github.com/ztellman/cambrian-collections/blob/master/collections/clojure/lang/PersistentVector2.java

There are benchmarks that demonstrate sizable performance gains relative to both lists and generic vectors, except in the case of conj'ing onto a 5-arity fixed size vector, where it has to spill over into a generic vector.  This cost is quite small, though, and I feel is more than made up for everywhere else.

I intend to make something similar for sets and maps, but first I'd like to make absolutely certain that this can make it into the core implementation.  Feedback from the JIRA gatekeepers would be greatly appreciated.

Zach

[1] https://github.com/ztellman/clj-tuple

[2] https://github.com/ztellman/cambrian-collections

[David Pollak]

I'm having flashbacks... https://github.com/scala/scala/blob/2.11.x/src/library/scala/Product1.scala

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.
For more options, visit https://groups.google.com/d/optout.

--

Brick Alloy http://brickalloy.com

Lift, the simply functional web framework http://liftweb.net

Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im

[Rich Hickey]

Yes, still interested, awesome - thanks!

It would be best if the PV1-4 end up as nested classes in a single class/file, and if that class had various factory methods that encapsulated the types.

We'll wire it up into vector creation separately, so please keep the patch to the data structures alone.

Thanks again,

Rich

[Brandon Bloom]

I'm curious if there is any interest/benefit for similar small maps or sets...

On Monday, August 25, 2014 8:07:57 PM UTC-4, Zach Tellman wrote:

[Mikera]

Possibly a crazy idea - but how about making the 2-Tuple work as a map entry as well? 

This could replace clojure.lang.MapEntry.....  as well as killing some redundant code I expect it would be a minor performance win (polymorphic inline caching on the JVM really likes it when you re-use the same classes....)

[Jozef Wagner]

clj-tuple already does that, https://github.com/ztellman/clj-tuple/blob/d5a11aa3ac576f472ce056d92ddf7ac1e402dc28/src/clj_tuple.clj#L129

[Zach Tellman]

Definitely planning to do this for maps and sets as well, just wanted to make sure I wasn't string-concatenating all this Java for nothing.  Work is under way.

[Zach Tellman]

Issue opened at http://dev.clojure.org/jira/browse/CLJ-1517