What to read after 3 dozen "introduction to transducers" blog posts

817 views
Skip to first unread message

Matching Socks

unread,
May 6, 2017, 1:56:13 PM5/6/17
to Clojure

"To be honest, this is a somewhat advanced usage of the transducers machinery," says the Grammarly Engineering Blog, right after shoehorning a BufferedReader into the mold with "reify IReduceInit".  I already felt I'd got my money's worth from reading up to this half-way point.  But I was astonished at what came next.  

Michael Willis

unread,
May 9, 2017, 11:27:16 AM5/9/17
to Clojure
After he uses this one weird trick, you'll never guess what happens next!

Luke Burton

unread,
May 10, 2017, 3:16:42 AM5/10/17
to clo...@googlegroups.com

> On May 6, 2017, at 10:56 AM, Matching Socks <phill...@gmail.com> wrote:
>
> This one. https://tech.grammarly.com/blog/building-etl-pipelines-with-clojure
>
> "To be honest, this is a somewhat advanced usage of the transducers machinery," says the Grammarly Engineering Blog, right after shoehorning a BufferedReader into the mold with "reify IReduceInit". I already felt I'd got my money's worth from reading up to this half-way point. But I was astonished at what came next.

Though it pains me to speak of it, I tell you now I have seen this "blog post", and seen too what came next. Astonishing indeed! I admire your simple description of these events, as it suggests either a man of great fortitude in the face of horror, or a man who was able to flee and forget, and I know that I am neither.

I remember the reified specimen – still alive, god have mercy – placed upon an altar to be offered up to `eduction`. Yes! Placed squarely on its unsightly variadic first argument. Why? For what purpose? We are told: "it's a recipe for the values to come." The blood drained from my face.

It was soon apparent what eldritch function would feast upon the values begotten by this unholy recipe. I saw the invocation of `transduce` and it brought back sweeter memories, of burritos and airport conveyor belts. I could see the `eduction`, presumably still wrapping the reified IReduceInit, perched atop a ziggurat of lesser transducers. I looked, and at the very bottom lay a blasphemous side-effectful invocation of map. I did not belong in this place.

Beyond astonished, I tell you, but still at this point I clung to sanity. I cried out: Enough! The values have suffered enough! They were good values, persisted to disk, why are they now flung into dimensions far beyond our own? Why have you turned your back on open(2)? Why have you forsaken the file descriptor?

Alas, my sanity could not stand this final blow. The Grammarly necromancer spoke the words: "Here Comes The Parallel Implementation." And lo, he did throw `core.async` into the mix. I saw `pipeline` swallow `transduce` whole, like a multi-armed multi-beaked creature from a Boschian hellscape. The whine of the CPU fans drowned out my last cry: aren't you supposed to use `pipeline-blocking`?

You may think me sane now in having told you all this, but I assure you, I am no longer. I should have run when I had the chance, but now I dream of educing the pipeline, reifying the transducer, and shouting the misbegotten outcomes into every channel that will >!! my message. It is too late for me, but perhaps not too late for you.

John Gabriele

unread,
May 10, 2017, 11:18:58 AM5/10/17
to Clojure

Hehehe! I don't understand much of the technical details here, but that was brilliant. :)

Alexander Yakushev

unread,
May 16, 2017, 6:43:13 AM5/16/17
to Clojure
This was beautiful. I was held in suspense through the whole story, and I cried in the end. But I must tell you that such sacrifices to data gods are justified, so I will keep reifying and transducing until the very last drop of bytes leaks from the oblatory value.
Reply all
Reply to author
Forward
0 new messages