bean-query intermediate AST and compiled representation

[Chary Chary]

Dear all,

I have a question regarding practical usage of intermediate AST and compiled representation of bean-query, obtained by EXPLAIN prefix

• What are these intermediate AST and compiled languages?
• Are they defined somewhere?
• Can they be used programmatically from Python script directly (something like a low-level language)?
• Are they more powerful, than bean-query language?

[Martin Blais]

On Tue, Dec 1, 2020 at 6:20 AM Chary Chary <char...@gmail.com> wrote:

Dear all,

I have a question regarding practical usage of intermediate AST and compiled representation of bean-query, obtained by EXPLAIN prefix

• What are these intermediate AST and compiled languages?

It's just the output of the parser (more below).

 

• Are they defined somewhere?

In v2 / Python, it's reusing the same namedtuple containers.

https://github.com/beancount/beancount/blob/v2/beancount/core/data.py

The differences are not really visible there. You'd have to read the parser code to figure out how they're partially filled, here:

https://github.com/beancount/beancount/blob/v2/beancount/parser/grammar.py

https://github.com/beancount/beancount/blob/v2/beancount/parser/grammar.y

Off the top of my head, the differences are:

- missing numbers and currencies are replaced by a special "MISSING" object instance

- on postings, instead of Cost, there is CostSpec

- in Balance directives, the calculated diff (error) amount is not stored

- tags and links aren't set to empty set objects consistently

and a few more things probably.

In v3 there will be some dedicated distinct objects for that.

in "beancount.inter" namespace:

https://github.com/beancount/beancount/blob/master/beancount/cparser/parser.proto

vs. the finalized schema:

https://github.com/beancount/beancount/blob/master/beancount/ccore/data.proto

• Can they be used programmatically from Python script directly (something like a low-level language)?

Not really. Plugins are accepting finalized transactions and expected to produce finalized ones too.

The booking and interpolation does not run on the output of your plugins.

• Are they more powerful, than bean-query language?

It's a good question that comes up from time to time whether a plugin interface that would run /before/ the booking and interpolation would be useful.

In a nutshell, here's how Beancount works:

- A top-level file is parsed to a set of partially filled directives, more or less mapping to the input, and includes files discovered during parsing are also parsed (in any order).

- The directives are merged together to a single list and sorted. 

- We iterate over the sorted transactions, accumulating inventory states, and in the process interpolating missing numbers, currencies and cost, and matching reductions against inventories to produce finalized postings.

- Then the plugins are run.

- Then some balance checks and other validation checks are run.

The product is a list of finalized directives, errors produced along the way, and options parsed. 

bean-query merely translates a SQL statement to a map/fiter/group loop, with a special "sum()" operation.

 

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/6162d225-3372-49c2-ad2f-2b9b425617b3n%40googlegroups.com.