How to debug shares being incorrectly reported as a decimal value

[Cameron Wood]

Hello all,

I'm a relatively new beancount user.

I've been exporting my online banking data, converting that to beancount format, and then using the include directive to pull that into my main beancount ledger. Everything has been going smoothly, until recently when I created a couple of new accounts to hold some new investments, and noticed they were being rendered as decimals, even though there wasn't any decimal and they should have been represented as integers.

To try and track this down, I've played around with the posting, changing both sides of the posting, changing the name, and all sorts of things, but without any luck. While doing this I randomly tried using the name of a stock I had previously held, and noticed it then rendered correctly. I tried renaming this to a couple of other stocks that I had previously held as well, and noticed there was one other stock/account which also rendered incorrectly as a decimal.

The transaction in question is about as simple as it can get, so I'm wondering if I'm encountering a bug or there's another problem with my ledger somewhere that I'm not yet aware of. When I've run bean-check and bean-doctor against it, it's passed without any errors, and I usually use Fava locally while editing to help check for any errors. Here is what the transaction looks like for reference:

2020-07-13 * "Buying company shares"

  Assets:ING-DiBa:Shares:ACME        100 ACME {40.27 EUR}

  Assets:ING-DiBa:Account              -4027 EUR

And if I change ACME to be MSFT or TEAM it renders correct, but if I leave it as ACME or change it to DIS, it renders incorrectly as a decimal.

At this point I'm at a bit of a loss about how to proceed with working out why this happen. So I was hoping someone here might be able to give me some pointers to get me on the right path.

Any help would be greatly appreciated, thanks!

Regards,

Cameron.

[c79m...@gmail.com]

I don't have any immediate help, but was wondering if you could give an example of where it's being represented as a decimal?  What do the opening / initialization lines look like for your new account? (vs the old one?)

- Cameron Murphy

[Martin Blais]

Read this: http://furius.ca/beancount/doc/tolerances

TL;DR

Beancount infers good defaults for the tolerance and precision automatically from the history of numbers appearing in your ledger file.

You can override the defaults using options.

Eventually I'd like to add an option to just set / force the values (but still have good defaults if not set).

Search on the mailing-list for "tolerance" and/or "precision" for more commentary.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/e8fa0fc1-ff5e-4b67-9bf8-b13feae32134n%40googlegroups.com.

[Cameron Wood]

Hello, and thank you for your reply.

On Saturday, 14 November 2020 at 22:19:53 UTC+1 c79m...@gmail.com wrote:

I don't have any immediate help, but was wondering if you could give an example of where it's being represented as a decimal?  What do the opening / initialization lines look like for your new account? (vs the old one?)

So I see this when looking at the holding or balance sheet in Fava, or when using bean-report to run the holdings and balances reports.

The opening/initialisation lines are pretty much identical, which is what lead to my comment about thinking there is an error somewhere else in my ledger or if this could be bug even. Below are the openings in question...

1970-01-01 open Assets:ING-DiBa:Shares:DIS                         DIS

1970-01-01 open Assets:ING-DiBa:Shares:MSFT                        MSFT

1970-01-01 open Assets:ING-DiBa:Shares:NFLX                        NFLX

1970-01-01 open Assets:ING-DiBa:Shares:TEAM                        TEAM

1970-01-01 open Assets:ING-DiBa:Shares:ACME                        ACME

so if I change the posting to be MSFT, NFLX, or TEAM, it renders as `100 XXXX` in the reports/views, but if I set it to ACME or DIS it displays as `100.00 XXXX` which has me at a bit of a loss currently.

Thanks again, and if you happen to think of any ideas/suggestions, it'd be greatly appreciated.

Kind regards,

Cameron.

[Cameron Wood]

Hi, and thanks for your reply.

On Saturday, 14 November 2020 at 22:54:52 UTC+1 bl...@furius.ca wrote:

Read this: http://furius.ca/beancount/doc/tolerances

TL;DR

Beancount infers good defaults for the tolerance and precision automatically from the history of numbers appearing in your ledger file.

You can override the defaults using options.

Eventually I'd like to add an option to just set / force the values (but still have good defaults if not set).

Search on the mailing-list for "tolerance" and/or "precision" for more commentary.

I did give the tolerances doc a few reads already, but I'll be sure to give it another once over, as I guess I've missed something there.

And then I'll follow your suggestion and see what other commentary I can find on the mailing list around this that might help explain what I'm seeing currently.

Kind regards,

Cameron.

[Martin Blais]

Use "bean-doctor display_context <filename>" to view the tolerances inferred.

They're inferred from the other numbers in your file.

This is an issue that's annoying me a lot; I'd gladly fix it, but it's easily a couple of days of work to do this properly. It's needed because some numbers are intended to be rounded (e.g. resulting from arithmetic operations). Fixing it is not as straightforward as providing one default per-currency, because the precision to be used for rendering depends on whether the price is being used to render units in the currency itself or used to render a price, and even then, the number of digits to be used depends on what other commodity it's pricing. I need to change the design so that a dictionary of precision per currency PAIRS is inferred and can be overridden (not just the default) by the user, for each pair. Ideally one should be able to just set the precision for

- currency

- currency when used as a price (default for all commodities)

- currency when used to price commodity X

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/5cb82a34-8c1b-46f0-8a38-18220087f9ffn%40googlegroups.com.

[Martin Blais]

For a bit more context on the internals for this, here's where the original design for this had led me:

https://github.com/beancount/beancount/blob/master/beancount/core/display_context.py#L71

- MOST_COMMON is used for currencies themselves

- MAXIMUM is used when currency rendered as price for something else.

I should 

(a) extend this to arbitrary price pairs

(b) allow users to override them and always use explicitly set values

Another detail is that in v3 that'll get used in the SQL query tool, so I'll need to a way to transmit that precision information to it.

[Daniele Nicolodi]

I'll be happy to work on this if we can come up with a design for how
this should work and for how the concept will be exposed in the
beancount syntax.

One extra complication, I think, is that tolerance and precision should
be two independent concepts.

By the way, does anyone how other accounting software (not only of the
text accounting family) deal with this?

Cheers,
Dan

> <mailto:beancount+...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/beancount/5cb82a34-8c1b-46f0-8a38-18220087f9ffn%40googlegroups.com

> <https://groups.google.com/d/msgid/beancount/5cb82a34-8c1b-46f0-8a38-18220087f9ffn%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to beancount+...@googlegroups.com

> <mailto:beancount+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/beancount/CAK21%2BhM4M7_RnbPwnEzE5epjRv3ebJk7cn2qB4Evs1FPXpDX3Q%40mail.gmail.com
> <https://groups.google.com/d/msgid/beancount/CAK21%2BhM4M7_RnbPwnEzE5epjRv3ebJk7cn2qB4Evs1FPXpDX3Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Cameron Wood]

On Sunday, 15 November 2020 at 15:38:42 UTC+1 bl...@furius.ca wrote:

Use "bean-doctor display_context <filename>" to view the tolerances inferred.

They're inferred from the other numbers in your file.

Thanks for suggesting this.

I ran bean-doctor display_context over the file in question, and got the following...

 

ACME            : sign=0   integer_max=3   fractional_common=0   fractional_max=0   "000" "000"

EUR             : sign=1   integer_max=4   fractional_common=2   fractional_max=2   "-0000.00" "-0000.00"

__default__     : sign=0   integer_max=1   fractional_common=_   fractional_max=_   "0.*" "0.*"

which I think confirms what I thought, that there postings which should have caused this to render as a decimal.

That said I recognise this is just a cosmetic thing, and it's only my obsessive tendency that's really bothered by it, so it's not the end of the world if I can't change this.

I came across this post on the mailing list, https://groups.google.com/g/beancount/c/tsYAf02DSq4/m/8E86elGQCAAJ and thought what I'm seeing could be related, given that my postings are split across many files and then included into a main ledger.

This is an issue that's annoying me a lot; I'd gladly fix it, but it's easily a couple of days of work to do this properly. It's needed because some numbers are intended to be rounded (e.g. resulting from arithmetic operations). Fixing it is not as straightforward as providing one default per-currency, because the precision to be used for rendering depends on whether the price is being used to render units in the currency itself or used to render a price, and even then, the number of digits to be used depends on what other commodity it's pricing. I need to change the design so that a dictionary of precision per currency PAIRS is inferred and can be overridden (not just the default) by the user, for each pair. Ideally one should be able to just set the precision for

- currency

- currency when used as a price (default for all commodities)

- currency when used to price commodity X

Thanks for sharing this extra context, it's always nice to have a bit more understanding of the constraints in play and what things have to be taken into account for any potential solution.

Just wanted to say thank you for all your hard work developing and supporting beancount, it's a fantastic tool and is greatly appreciated.

Kind regards,

Cameron.