reduce and IReduce vs IReduceInit (and CollReduce)

[Sean Corfield]

Over the last few days there has been an interesting discussion, in #clojure-dev on Slack, about the pros and cons of CollReduce, IReduce, and IReduceInit. This came up because I was implementing “reducible queries” in clojure.java.jdbc.

 

The bottom line is that I’ve implemented IReduce and provided both ‘reduce’ signatures (both with and without the ‘init’ value).

 

Looking at the source for ‘reduce’, if you call (reduce f coll) – no init value – on something that implements (only) IReduceInit, then ‘reduce’ will delegate to ‘coll-reduce’ and that in turn does the following:

 

     (.reduce ^IReduce coll f)

 

https://github.com/clojure/clojure/blob/master/src/clj/clojure/core/protocols.clj#L88

 

and that will blow up with a ClassCastException.

 

The docstring for ‘reduce’ is very specific about when and how ‘f’ is called:

 

If an ‘init’ value is specified:

If the ‘coll’ has no elements, ‘init’ is returned and ‘f’ is not called.

            If the ‘coll’ has one element, ‘f’ is called on ‘init’ and that element.

            If the ‘coll’ has multiple elements, ‘f’ is called as often as needed…

 

If an ‘init’ value is NOT specified:

            If the ‘coll’ has no elements, ‘f’ is called with no arguments and that value is returned.

            If the ‘coll’ has one element, that element is returned and ‘f’ is not called.

            If the ‘coll’ has multiple elements, ‘f’ is called (with two arguments) as often as needed…

 

If you implement IReduce, you can satisfy that contract – and the caller may need to ensure ‘f’ can be called with no arguments.

 

If you implement IReduceInit, you cannot satisfy half of the contract. It just blows up. But ‘f’ only ever needs to support two arguments.

 

First question: is that deliberate behavior? (that (reduce f ireduce-thing) blows up and does not satisfy ‘reduce’s contract)

 

Second question: what are the pros and cons of implementing IReduce vs IReduceInit? Does Clojure/core have any guidance on which to prefer and when?

 

Third question: is there any reason to use the CollReduce protocol these days? (i.e., in Clojure 1.7 onward)

 

Sean Corfield -- (904) 302-SEAN -- (970) FOR-SEAN

An Architect's View -- http://corfield.org/

 

"Perfection is the enemy of the good."

-- Gustave Flaubert, French realist novelist (1821-1880)

 

 

[Alex Miller]

1) yes, Rich and I discussed this - when you get a ClassCastException saying you don't implement IReduce, that's your clue that you should implement IReduce or use an init value.

2) generally, I'd say you should implement IReduceInit unless you expect it to be useful without an unit value, in which case you should implement IReduce. I think implementing it for jdbc is fine.

3) The IReduce interfaces should be preferred when you control the type as it is an explicit check in reduce and transduce and will be the fastest path. CollReduce is a protocol so you can use it for open extension even on types you don't control.

[Sean Corfield]

I’ll answer in reverse order…

On 7/1/17, 7:44 AM, "Alex Miller" <cloju...@googlegroups.com on behalf of al...@puredanger.com> wrote:

> 3) The IReduce interfaces should be preferred when you control the type as it is an explicit check in reduce and transduce and will be the fastest path. CollReduce is a protocol so you can use it for open extension even on types you don't control.

Thanks. Makes sense.

> 2) generally, I'd say you should implement IReduceInit unless you expect it to be useful without an unit value, in which case you should implement IReduce. I think implementing it for jdbc is fine.

Thanks. It seems as a library author I probably should not second-guess my users who might well want to write a reducing function that has a 0-arity version to return the result when the JDBC ResultSet is empty (almost the only case where IReduce and IReduceInit differ). Although, I must admit that reading that sentence over makes it feel unlikely! (

> 1) yes, Rich and I discussed this - when you get a ClassCastException saying you don't implement IReduce, that's your clue that you should implement IReduce or use an init value.

Thanks. Just wanted to be sure this was known, intended behavior. It isn’t exactly _friendly_ behavior – and it seems to be one of those cases where a try/catch could translate this case into a much nicer error message:

clojure.lang.IReduceInit
(coll-reduce
([coll f]
(try
(.reduce ^clojure.lang.IReduce coll f)
(catch ClassCastException e
(if (instance? clojure.lang.IReduce coll)
(throw e) ; error was from something else
(throw (ex-info “IReduce not implemented, or init value not provided”
{:type (type coll)}))))))
([coll f val] (.reduce coll f val)))

(would wrapping code with a try/catch here be too much overhead for a better error message?)

Sean

[Alex Miller]

>   1) yes, Rich and I discussed this - when you get a ClassCastException saying you don't implement IReduce, that's your clue that you should implement IReduce or use an init value.

Thanks. Just wanted to be sure this was known, intended behavior. It isn’t exactly _friendly_ behavior – and it seems to be one of those cases where a try/catch could translate this case into a much nicer error message:

  clojure.lang.IReduceInit
  (coll-reduce
    ([coll f]
      (try
        (.reduce ^clojure.lang.IReduce coll f)
        (catch ClassCastException e
          (if (instance? clojure.lang.IReduce coll)
            (throw e) ; error was from something else
            (throw (ex-info “IReduce not implemented, or init value not provided”
                            {:type (type coll)}))))))
    ([coll f val] (.reduce coll f val)))

(would wrapping code with a try/catch here be too much overhead for a better error message?)

Maybe. Certainly moving the error message generation out of the mainline would help reduce the chances of hitting the inlining budget.

If you want to dig into it, go for it!