bean-doctor context upgrades (now works with invalid transaction!)

[Martin Blais]

Beancount users!

I've just fixed a long-standing bug that's been driving me crazy and that has a high impact on usability, I need to bring this to your attention. In fact, this fix is so important I've decided to make an exception on new code in v2 and put it in that branch so you can all enjoy it immediately.

doctor: Use parsed transaction to improve context on invalid transactions.

The "bean-doctor context" command didn't use to work on transactions with
errors. This is because the booking code is pretty opinionated about outputting
only correct results and transactions that had postings that couldn't be
resolved unambiguously were produced without those postings.

The context command was attempting to use those incomplete booked
transactions and failing to find the accounts. So on an input like this, which
is common:

  2021-01-28 * "(TRD) SELL TRADE" #liquidation-2021
    Assets:US:Adelitrade:Main:FB        -15 FB {} @ 282.6100 USD
    Expenses:Financial:Fees            0.09 USD
    Assets:US:Adelitrade:Main:Cash  4239.06 USD

Let's say the stock account is configured in 'STRICT' booking mode and the input
is ambiguous. You would get an error like this:

    ...filename...:57895:   Ambiguous matches for "-15 FB {}": 15 FB {179.3700 USD, 2018-08-30}, 100 FB {179.4100 USD, 2018-08-30}

       2021-01-28 * "(TRD) SELL TRADE" #liquidation-2021
         Assets:US:Adelitrade:Main:FB        -15 FB {} @ 282.6100 USD
         Expenses:Financial:Fees            0.09 USD
         Assets:US:Adelitrade:Main:Cash  4239.06 USD

and "bean-doctor context" was producing this, which was thoroughly unsatisfying:

    bean-doctor context blais.beancount 57897
    Hash:dfe2f18a91b4896d7474bd4518fcadd6
    Location: /home/blais/r/q/office/accounting/blais.beancount:57895

    ------------ Balances before transaction


    ------------ Transaction

    2021-01-28 * "(TRD) SELL TRADE" #liquidation-2021


    Tolerances: CAD=0.005, USD=0.005

    ------------ Balances after transaction


This left you with no information whatsoever about the prior state of accounts,
which in the case of having a 'STRICT' booking strategy, is PRECISELY what you
would want. Super annoying.

So for years, I've gone fishing for my lot history by hand and I've grumbled
quietly to myself about how annoying this is. It even led to scripts like this
one to go fish for the lots automatically:
https://github.com/beancount/beanlabs/blob/master/beanlabs/scripts/print_lots.py

I've just found a simple way to fix it: I'm fetching the incomplete transaction
from the parser, and I'm now including it in the context output, and using the
accounts it provides to produce the before balances. You now get the balances
even if the transaction has an error, and you can use that to insert the
specific lots you want. The new output looks like this:


    ** Transaction Id --------------------------------

    Hash:dfe2f18a91b4896d7474bd4518fcadd6
    Location: /home/blais/r/q/office/accounting/blais.beancount:57895


    ** Balances before transaction --------------------------------

      Assets:US:Adelitrade:Main:FB                     15 FB {179.3700 USD, 2018-08-30}
      Assets:US:Adelitrade:Main:FB                    100 FB {179.4100 USD, 2018-08-30}

      Expenses:Financial:Fees                                              1563.650 USD

      Assets:US:Adelitrade:Main:Cash                                      116141.83 USD


    ** Unbooked Transaction --------------------------------

    2021-01-28 * "(TRD) SELL TRADE" #liquidation-2021
      Assets:US:Adelitrade:Main:FB        -15 FB {} @ 282.610  ;
      Expenses:Financial:Fees            0.09 USD              ;    0.09 USD
      Assets:US:Adelitrade:Main:Cash  4239.06 USD              ; 4239.06 USD


    ** Transaction --------------------------------

    2021-01-28 * "(TRD) SELL TRADE" #liquidation-2021


    ** Residual and Tolerances --------------------------------

    Tolerances: CAD=0.005, USD=0.005


    ** Balances after transaction --------------------------------

      Assets:US:Adelitrade:Main:FB                     15 FB {179.3700 USD, 2018-08-30}
      Assets:US:Adelitrade:Main:FB                    100 FB {179.4100 USD, 2018-08-30}

      Expenses:Financial:Fees                                              1563.650 USD

      Assets:US:Adelitrade:Main:Cash                                      116141.83 USD


You get the unbooked transaction with its missing invalid postings (in this
case, no postings at all), and the one from the parser.

This is a HUGE improvement in usability, because this provides you with a
convenient method to obtain the lots you need to cut-and-paste in your input if
you're using the specific tax lots ID method, right there from Emacs.


I hope some of you enjoy this.

The corresponding commit is here:

https://github.com/beancount/beancount/commit/06127f16e05c6bfa286eef6b5f7fc9ba653fad53

[redst...@gmail.com]

Nice going, and simple solution! This definitely was an annoyance during lot selection.

[sm...@davethehappysinger.com]

Hi Martin,

Just popping in to let you know what a fantastic fix this has been for me. I use a FIFO methodology but with STRICT booking, entering the lots explicitly. Having the ability to reliably use vim-beancount's GetContext command on an incomplete WIP transaction to find the appropriate lots to reduce is a MASSIVE boon to me. Thank you so much for all your constant work on beancount and this fix in particular. It really helps me.

Dave

[Martin Blais]

No worries!

Glad it had made impact.

Thanks for the note,

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/1b7c83f0-0bdb-4dbf-9ed2-8100ad9de6e5n%40googlegroups.com.