Why transducers are not (yet) fundamental?

[rebo...@gmail.com]

I'm doing a little research for a talk and asking clojurists around. The thesis I'm supporting is that transducers should completely replace "normal" (non-reducing based) sequential processing. People have different reactions to this, usually going from "what's wrong with threading macros" to "I only use them for performances" to "they are less readable". I think it's mostly habit.

Personally, I can't find any good reason not to ditch Clojure <1.7 sequential processing and use transducers exclusively. It took me a couple of years to kill my habit but hey, they came late. If 1.0 shipped with transducers and 1.7 introduced thread macros to create data pipelines, people would probably go now: "why should I use a macro instead of comp", "they are slow", "they don't compose easily" etc.

What do you think? Am I missing something?

Renzo

[Alex Miller]

FAQ entry:

https://clojure.org/guides/faq#transducers_vs_seqs

Both have their place, in my opinion. I often use transducers with `into` where I would have used ->> in the past, but not always - it depends on context.

While transducers will generally perform better if you have many transformations or large (reducible) collections, lazy sequences (which automatically leverage chunking and transients) are surprisingly fast. Additionally, in cases where won't actually use all of the result, lazy sequences are faster by avoiding work that doesn't need to be completed. Transducers also don't cope well with potentially large or infinite intermediate results (even in the context of `sequence` with a transducer as transducers are a pull-based model).

[Renzo Borgatti]

> On 17 Jun 2017, at 17:43, Alex Miller <al...@puredanger.com> wrote:
>
> FAQ entry:
>
> https://clojure.org/guides/faq#transducers_vs_seqs
>
> Both have their place, in my opinion. I often use transducers with `into` where I would have used ->> in the past, but not always - it depends on context.
>
> While transducers will generally perform better if you have many transformations or large (reducible) collections, lazy sequences (which automatically leverage chunking and transients) are surprisingly fast. Additionally, in cases where won't actually use all of the result, lazy sequences are faster by avoiding work that doesn't need to be completed. Transducers also don't cope well with potentially large or infinite intermediate results (even in the context of `sequence` with a transducer as transducers are a pull-based model).

I wouldn't go as far to deprecate the current (or anything like that). Still, I would reach for transduce/sequence as a default. The advantages, in terms of how a program with data pipelines could be designed differently, are evident (for me). Even if you don't do data pipelines constantly, there is still an advantage in coding them with transducers by default. Presented as a fair alternative, they end up not being used as they deserve. They are presented like yet another option so having to pick between something they know or something they have to learn, people stay where they are.

The "Consumer control" part of the FAQ sounds contradictory to me. Combining input and transformation restrict consumer control (and increase producer control as "you're going to get my API this way or not").

> On Saturday, June 17, 2017 at 9:07:26 AM UTC-5, rebo...@gmail.com wrote:
> I'm doing a little research for a talk and asking clojurists around. The thesis I'm supporting is that transducers should completely replace "normal" (non-reducing based) sequential processing. People have different reactions to this, usually going from "what's wrong with threading macros" to "I only use them for performances" to "they are less readable". I think it's mostly habit.
>
> Personally, I can't find any good reason not to ditch Clojure <1.7 sequential processing and use transducers exclusively. It took me a couple of years to kill my habit but hey, they came late. If 1.0 shipped with transducers and 1.7 introduced thread macros to create data pipelines, people would probably go now: "why should I use a macro instead of comp", "they are slow", "they don't compose easily" etc.
>
> What do you think? Am I missing something?
> Renzo
>

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Phillip Lord]

Well, inertia is probably the biggest reason, combined with the
documentation. When you have a FAQ which is "what are good use cases for
transducers", surely this is an issue.

I've never used one yet, except by accident when I forget an argument to
a function that previously would have crashed.

Phil