Open source opinionated Beancount code formatter and parser

[Fang-Pen Lin]

Hi there,

First time making a post here, hopefully it's okay to share open source tools I built here. 

I was working on my web base beancount SaaS product and realized that adding new entries without formatting over time will soon end up with a mess. I really like Python's black formatter and how it works. With the needs in mind, I built two projects:

• beancount-parser - Standalone Lark based Beancount syntax parser

• beancount-black - Opinionated beancount code formatter based on beancount-parser

Both were open sourced under MIT license (there's no dependency to Beancount lib). The reason I don't use Beancount lib is that I want to handle things like comments and emac org syntax. It would be much easier and cleaner to build a Lark based parser and work on top of that. Since the product I build is SaaS and there's zero dependency to the Beancount core, there was actually no requirement for open sourcing them. But I really like the plantext accounting community and would like to give back as much as I can, so I open sourced them here.

You can try the formatter online here if you don't want to install them:

https://beanhub.io/tools/beancount-formatter

Some of the corner case syntax may not be supported, and since the tool is pretty new, you may also encounter bugs, please feel free report them or provide any feedbacks. There are more features to be added to the formatter in the near future, such as

• Account renaming

• Currency renaming

Anyway, hope these tools I build could be useful to some of you 😄🙌

[Daniele Nicolodi]

Hello Fang-Peng,

thank you for sharing. I think having these are interesting tools!

On 09/04/2022 08:26, Fang-Pen Lin wrote:
>
> I was working on my web base beancount SaaS product [...]

I am curious about the busyness model for such an enterprise, but maybe
the Beancount mailing list is not the best place to discuss it...

> I really like Python's black formatter and how it works. With the
> needs in mind, I built two projects:
>
> beancount-parser

> <https://github.com/LaunchPlatform/beancount-parser> - Standalone

> Lark based Beancount syntax parser

I don't know Lark much and I had just a quick look, but something seems
odd: how do you handle indentation? I don't see anything about it in the
grammar files. I realize that ignoring this aspect may be fine if the
goal is to generate a syntax three to use in a formatting tool operating
on known valid input files. Otherwise, this is not a validating parser.
Maybe something to mention in the documentation.

> The reason I don't use Beancount lib is that I want to
> handle things like comments and emac org syntax

Please note that preserving comments (org-mode headers are just a
special kind of comments as far as the Beancount parser is concerned) is
a goal for the overhaul of the parser coming in Beancount v3. If you
feel to contribute, I would be very happy to review patches adding this
feature to Beancount.

> You can try the formatter online here if you don't want to install them:
>
> https://beanhub.io/tools/beancount-formatter

I tries this, but whatever I enter, I always get an error:

500
Sorry! Something Went Wrong


Cheers,
Dan

[Fang-Pen Lin]

Sorry, somehow my reply doesn't show up, I guess I press the wrong button to reply to author instead of the mailing list, so do it here again.

On Saturday, April 9, 2022 at 12:53:06 PM UTC-7 dan...@grinta.net wrote:

Hello Fang-Peng,

thank you for sharing. I think having these are interesting tools!

On 09/04/2022 08:26, Fang-Pen Lin wrote:
>
> I was working on my web base beancount SaaS product [...]

I am curious about the busyness model for such an enterprise, but maybe
the Beancount mailing list is not the best place to discuss it...

Sure, won't discuss it here then.

 

> I really like Python's black formatter and how it works. With the
> needs in mind, I built two projects:
>
> beancount-parser
> <https://github.com/LaunchPlatform/beancount-parser> - Standalone
> Lark based Beancount syntax parser

I don't know Lark much and I had just a quick look, but something seems
odd: how do you handle indentation? I don't see anything about it in the
grammar files. I realize that ignoring this aspect may be fine if the
goal is to generate a syntax three to use in a formatting tool operating
on known valid input files. Otherwise, this is not a validating parser.
Maybe something to mention in the documentation.

Yeah, you are right, it does not validate the syntax. The parser was not meant to be a replacement for beancount core lib but only to facilitate the formatter, that's why it doesn't care about the syntax is valid or not. I have already updated the README to make it clear that it does not validate

 

> The reason I don't use Beancount lib is that I want to
> handle things like comments and emac org syntax
Please note that preserving comments (org-mode headers are just a
special kind of comments as far as the Beancount parser is concerned) is
a goal for the overhaul of the parser coming in Beancount v3. If you
feel to contribute, I would be very happy to review patches adding this
feature to Beancount.

> You can try the formatter online here if you don't want to install them:
>
> https://beanhub.io/tools/beancount-formatter

I tries this, but whatever I enter, I always get an error:

500
Sorry! Something Went Wrong

I just fix a couple of bugs and redeployed, please try again now.

 

Cheers,
Dan

[Fang-Pen Lin]

On Saturday, April 9, 2022 at 12:53:06 PM UTC-7 dan...@grinta.net wrote:

Hello Fang-Peng,

thank you for sharing. I think having these are interesting tools!

On 09/04/2022 08:26, Fang-Pen Lin wrote:
>
> I was working on my web base beancount SaaS product [...]

I am curious about the busyness model for such an enterprise, but maybe
the Beancount mailing list is not the best place to discuss it...

Sure, won't discuss it here then.

 

> I really like Python's black formatter and how it works. With the

> needs in mind, I built two projects:
>
> beancount-parser
> <https://github.com/LaunchPlatform/beancount-parser> - Standalone
> Lark based Beancount syntax parser

I don't know Lark much and I had just a quick look, but something seems
odd: how do you handle indentation? I don't see anything about it in the
grammar files. I realize that ignoring this aspect may be fine if the
goal is to generate a syntax three to use in a formatting tool operating
on known valid input files. Otherwise, this is not a validating parser.
Maybe something to mention in the documentation.

You are right, my parser tool won't validate the syntax. It's not meant to be a replacement of beancount core library, only to facilitate the formatter. And yeah, I will update the document to make it clear that there won't be validation for syntax.

 

> The reason I don't use Beancount lib is that I want to
> handle things like comments and emac org syntax
Please note that preserving comments (org-mode headers are just a
special kind of comments as far as the Beancount parser is concerned) is
a goal for the overhaul of the parser coming in Beancount v3. If you
feel to contribute, I would be very happy to review patches adding this
feature to Beancount.

> You can try the formatter online here if you don't want to install them:
>
> https://beanhub.io/tools/beancount-formatter

I tries this, but whatever I enter, I always get an error:

500
Sorry! Something Went Wrong

There were couple of bugs I just fixed and redeployed. Could you please try again now.

 

Cheers,
Dan

[Daniele Nicolodi]

On 10/04/2022 00:35, Fang-Pen Lin wrote:
> On Saturday, April 9, 2022 at 12:53:06 PM UTC-7 dan...@grinta.net wrote:
> > I really like Python's black formatter and how it works. With the
> > needs in mind, I built two projects:
> >
> > beancount-parser
> > <https://github.com/LaunchPlatform/beancount-parser
> <https://github.com/LaunchPlatform/beancount-parser>> - Standalone
> > Lark based Beancount syntax parser
>
> I don't know Lark much and I had just a quick look, but something seems
> odd: how do you handle indentation? I don't see anything about it in
> the
> grammar files. I realize that ignoring this aspect may be fine if the
> goal is to generate a syntax three to use in a formatting tool
> operating
> on known valid input files. Otherwise, this is not a validating parser.
> Maybe something to mention in the documentation.
>
>
> You are right, my parser tool won't validate the syntax. It's not meant
> to be a replacement of beancount core library, only to facilitate the
> formatter. And yeah, I will update the document to make it clear that
> there won't be validation for syntax.

Without handling indentation, how do you distinguish between this:

* Assets:Section:With:Funny:Name

and this:

2022-04-10 * "Test"
* Assets:Posting:With:Flag

or between this two metadata entries:

2022-04-10 * "Test"
transaction-metadata: 42
Assets:Tests 1 TEST
posting:metadata: 25

?

I gave it a quick try via the online formatter and the posting flag is
interpreted as an org-mode heading and posting metadata entries are not
supported.

It also seems that flag characters are limited to '*' and '!'.

Cheers,
Dan

[Fang-Pen Lin]

To be honest, I have little idea how indentation works for beancount, there seems to be little document talking about how the syntax works in details. But yeah, my code probably won't be able to handle a corner case like that. Thanks for letting me know, I will find a time to look into it and fix it.