autobean-format: deep cleanup, recursive formatting, entry sorting, and more

188 views
Skip to first unread message

Archimedes Smith

unread,
Mar 31, 2023, 6:25:08 PM3/31/23
to Beancount
Hi beancounters,

I've written yet another formatter for beancount, autobean-format, powered by my earlier project autobean-refactor, a library for parsing and programmatically manipulating beancount files.

Highlights:
  • Deep cleanup
    • Unlike bean-format which is regex-based, autobean-format is based on a proper parser, allowing it to format every corner of your ledger, including arithmetic expressions.
  • Recursive formatting
    • Optionally formats all included files.
    • Honors glob include.
  • Sort entries
    • Optionally sorts dated entries.
    • Honors "time" meta (HH:MM, HH:MM:SS, epoch seconds, epoch ms, epoch us).
    • Honors pushtag / pushmeta.
    • Preserves existing comments, structure, and ordering whenever possible.
  • Preview-able
    • Optionally prints diff.
  • Thousands separators.
    • Optionally add, remove, or keep (default) thousands separators.
  • Typed and extensible
    • For example, the code for supporting thousands separator is as simple as this.
Please check out autobean-format if you are interested. Bug reports / suggestions / contributions are appreciated.

Regards,
SEIAROTg

Ben Blount

unread,
Mar 31, 2023, 6:46:47 PM3/31/23
to Beancount
This is fantastic, thanks for your hard work on it!!

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/962be409-86aa-4fed-859e-2ff983fe0d95n%40googlegroups.com.

Martin Blais

unread,
Apr 1, 2023, 11:10:50 AM4/1/23
to bean...@googlegroups.com
Added to the contrib doc, thank you for this,

FYI I gave it a spin and hit this error, probably just needs an adjustment on your parser:

lumia [git|main]:~/src/.../SEIAROTg/autobean-format$ autobean-format $L
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/lark/lexer.py", line 590, in lex
    yield lexer.next_token(lexer_state, parser_state)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/lark/lexer.py", line 528, in next_token
    raise UnexpectedCharacters(lex_state.text, line_ctr.char_pos, line_ctr.line, line_ctr.column,
lark.exceptions.UnexpectedCharacters: No terminal matches '^' in the current parser context, at line 68996 col 73

rade:Main:Cash "Intra-Account Transfer" ^td-25441737451
                                        ^
Expected one of:
        * WHITESPACE
        * _NEWLINE_INDENT_COMMENT
        * INLINE_COMMENT
        * _NEWLINE
        * BLOCK_COMMENT
        * INDENT

Previous tokens: Token('WHITESPACE', ' ')


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/autobean-format", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/autobean_format/main.py", line 79, in main
    for file in formatter.load_files(filename):
  File "/usr/local/lib/python3.11/site-packages/autobean_format/main.py", line 45, in load_files
    model = self._parser.parse(text, models.File)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/autobean_refactor/parser.py", line 135, in parse
    return self._parse(text, target, self._lark)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/autobean_refactor/parser.py", line 156, in _parse
    for token in parser.lexer_thread.lex(parser.parser_state):
  File "/usr/local/lib/python3.11/site-packages/autobean_refactor/parser.py", line 54, in process
    for token in stream:
  File "/usr/local/lib/python3.11/site-packages/lark/lexer.py", line 599, in lex
    raise UnexpectedToken(token, e.allowed, state=parser_state, token_history=[last_token], terminals_by_name=self.root_lexer.terminals_by_name)
lark.exceptions.UnexpectedToken: Unexpected token Token('LINK', '^td-25441737451') at line 68996, column 73.
Expected one of:
        * WHITESPACE
        * _NEWLINE_INDENT_COMMENT
        * INLINE_COMMENT
        * _NEWLINE
        * BLOCK_COMMENT
        * INDENT
Previous tokens: [Token('WHITESPACE', ' ')]




--
Message has been deleted
Message has been deleted

Red S

unread,
Apr 1, 2023, 11:01:19 PM4/1/23
to Beancount
This is...therapeutic :).

I'd long given up on having a neat, consistent ledger sources, given that I automate importing and mostly consume my ledger via Beancount (scripts, bql, Fava, etc.). But I do occasionally have to open up my ledger, and am never satisfied with the ugliness scattered over. autobean-format is a magic wand to banish all of that!

For anyone who wants to cross check the basic autobean-format formatting, use "git diff --ignore-space-change --ignore-blank-lines". If you don't store your ledger under version control, now's a good time to do it.

Thanks a bunch! And I appreciated the --recursive.

Archimedes Smith

unread,
Apr 4, 2023, 6:45:45 PM4/4/23
to bean...@googlegroups.com, bl...@furius.ca
Reposting as the previous two replies somehow got (automatically?) deleted.

Thanks Martin for flagging this.

I thought I replicated the full v2 syntax except out-of-line tags / links in transaction but apparently I missed that note also supports tags / links, which is now added. Can you try upgrading autobean-refactor to 0.2.3 and see if it works?

Also, behind `--sort` is a non-standard O(N^2) algorithm which probably won't work with this huge file with 68996+ lines. It could be made O(N logN) but I just didn't think about files of that size…


You received this message because you are subscribed to a topic in the Google Groups "Beancount" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beancount/JSSc3yRnGyk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhM38ZStM7VL_BCu1riWghviJjqOMf94C_Gy%2BXp9ym-rzw%40mail.gmail.com.

Martin Blais

unread,
Apr 4, 2023, 6:47:11 PM4/4/23
to bean...@googlegroups.com


On Tue, Apr 4, 2023 at 6:45 PM Archimedes Smith <seia...@gmail.com> wrote:
(reposting as the previous reply somehow got deleted)
Somehow you got caught by the spam filter. I whitelisted you.

 

Thanks Martin for flagging this.

I thought I replicated the full v2 syntax except out-of-line tags / links in transaction but apparently I missed that note also supports tags / links, which is now added. Can you try upgrading autobean-refactor to 0.2.3 and see if it works?

Also, behind `--sort` is a non-standard O(N^2) algorithm which probably won't work with this huge file with 68996+ lines. It could be made O(N logN) but I just didn't think about files of that size...

Reply all
Reply to author
Forward
0 new messages