Possible low hanging fruit for Beancount performance improvement?

[Simon Guest]

Having been designing a plugins system for limabean using Clojure transducers, I happened to notice a very different implementation for plugins in OG Beancount.

Clojure transducers compose, so if you have a number of plugins, the limabean plugin framework loops once over all the directives, passing each directive through the whole chain of plugins.

I noticed that with the way plugins have been implemented in OG Beancount, each plugin iterates over the entire list of directives. And then the next plugin does the same, and so on.  I wonder if there is the potential here for iterating just once, and passing each directive through the chain of plugins?

Just to be clear, I'm only throwing this idea out.  I won't be working on this! 😅

cheers,

Simon

PS. Upon reflection, it may not be very low hanging!  But worth considering for vNext, surely.

[redst...@gmail.com]

Not a possiblity: each plugin considers the entire ledger and transforms it as a whole. Many plugins depend on doing so. Also: plugins aren’t the primary performance bottleneck anyway.

[Martin Blais]

It's a bit difficult because some plugins process the entire thing and insert directives after seeing the entirety of the stream before making modifications. In other words it's not "online".  I think it's possible for some subset of plugins though.

I don't think you'll get a lot of performance benefit from that. Do some profiling. Maybe finding a way to reuse balance accumulation across plugin runs might help, but honestly I would let profiling guide us.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/beancount/CAFhGSbuY8fz7ETo2%3DCJguqYsnhZC0_Um9xoF-8XPh9hAzeARpg%40mail.gmail.com.

[Simon Guest]

OK, fair cop, I haven't profiled anything! 

As to what's possible though, it is perfectly possible for a stateful transducer that gives the appearance of operating on each directive at a time to in fact slurp in them all before spitting anything out, so this scenario is covered.  That's part of the genius of the transducer design.  Even if you're not primarily interested in Clojure, Rich Hickey's original presentation on transducers is both compelling and entertaining.  Worth a watch.  (And worth thinking about whether there's anything possible like this in Python land, or whatever is your language of choice.)

I do understand that this isn't something that's feasible to change in OG Beancount, but I do think next generation developers should pay attention to such things!

 

To view this discussion visit https://groups.google.com/d/msgid/beancount/CAK21%2BhPQvip3AJ6RMemWGH%3D%3D2ZbfT5NcxQOXz_31PJfp50xj5Q%40mail.gmail.com.

[Martin Blais]

On Thu, Mar 5, 2026 at 2:53 PM 'Simon Guest' via Beancount <bean...@googlegroups.com> wrote:

OK, fair cop, I haven't profiled anything! 

As to what's possible though, it is perfectly possible for a stateful transducer that gives the appearance of operating on each directive at a time to in fact slurp in them all before spitting anything out, so this scenario is covered.

Yeah but then you won't have any of the benefit

 

  That's part of the genius of the transducer design.  Even if you're not primarily interested in Clojure, Rich Hickey's original presentation on transducers is both compelling and entertaining.  Worth a watch.  (And worth thinking about whether there's anything possible like this in Python land, or whatever is your language of choice.)

I used to attend NYC Clojure at Google NYC and I think Rich presented it one time. He's a brilliant guy.

I do understand that this isn't something that's feasible to change in OG Beancount, but I do think next generation developers should pay attention to such things!

 

On Fri, 6 Mar 2026 at 03:09, Martin Blais <bl...@furius.ca> wrote:

It's a bit difficult because some plugins process the entire thing and insert directives after seeing the entirety of the stream before making modifications. In other words it's not "online".  I think it's possible for some subset of plugins though.

I don't think you'll get a lot of performance benefit from that. Do some profiling. Maybe finding a way to reuse balance accumulation across plugin runs might help, but honestly I would let profiling guide us.

On Wed, Mar 4, 2026 at 11:09 PM 'Simon Guest' via Beancount <bean...@googlegroups.com> wrote:

Having been designing a plugins system for limabean using Clojure transducers, I happened to notice a very different implementation for plugins in OG Beancount.

Clojure transducers compose, so if you have a number of plugins, the limabean plugin framework loops once over all the directives, passing each directive through the whole chain of plugins.

I noticed that with the way plugins have been implemented in OG Beancount, each plugin iterates over the entire list of directives. And then the next plugin does the same, and so on.  I wonder if there is the potential here for iterating just once, and passing each directive through the chain of plugins?

Just to be clear, I'm only throwing this idea out.  I won't be working on this! 😅

cheers,

Simon

PS. Upon reflection, it may not be very low hanging!  But worth considering for vNext, surely.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/beancount/CAFhGSbuY8fz7ETo2%3DCJguqYsnhZC0_Um9xoF-8XPh9hAzeARpg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/beancount/CAK21%2BhPQvip3AJ6RMemWGH%3D%3D2ZbfT5NcxQOXz_31PJfp50xj5Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/beancount/CAFhGSbvS_8NS2QVRer7P4NP3bsEm%3DEFmzVJJpH9Qp8CTH3QwPA%40mail.gmail.com.

[Simon Guest]

Well, you avoid making every plugin pay the price for the ones that need access to all of the directives up front. I enjoy the elegance in that. 😎

To view this discussion visit https://groups.google.com/d/msgid/beancount/CAK21%2BhNDMiS5XQvcAr%3D0hHvjGpengrAQC33L%2BDaFavc%2BugAhrA%40mail.gmail.com.

[redst...@gmail.com]

As to what's possible though, it is perfectly possible for a stateful transducer that gives the appearance of operating on each directive at a time to in fact

I do understand that this isn't something that's feasible to change in OG Beancount, but I do think next generation developers should pay attention to such things!

That’s a great video, thanks for sharing! Sure, “not possible” is a strong statement. And I’d absolutely encourage the next generation to explore all kinds of ideas like this.

That said, it’s worth thinking about the root motivation and goals behind any change. With plugins, performance is arguably not the main concern IMHO. More important are things like:

• Ease of writing plugins. The current model is extremely simple: take the ledger, transform it, return it.
• A simple mental model. Plugin authors can always assume they take the whole ledger and transform it before the ledger is handed off to the next plugin
• Approachability. A lot of plugins are home-cooked, and writing one is often a user’s first experience hacking on Beancount. Given that, the model should stay friendly and easy to reason about.
• Ecosystem importing. Many users have years of data and a collection of existing plugins they rely on.
• Performance. Important, but plugins haven’t so far been the main bottleneck.

So while transducer-style ideas or streaming (eg: an indexing pass, then a transform pass) are interesting, I’d be cautious about optimizing the plugin model for performance if it complicates the authoring model or breaks existing assumptions. And in practice with a bit of profiling, you’ll find that making plugins faster wouldn’t move the needle much overall.

There are serious performance issues elsewhere though, and other account modeling issues that could both benefit a huge part of the userbase as written about in the Vnext doc. And perhaps your RFC idea (great idea, BTW) could be leveraged to explore what issues to prioritize and focus on. My two cents.

​