Implicit rounding of inferred numbers

[Zhuoyun Wei]

Hi Martin,

Beancount seems to round inferred numbers by default:

1970-01-01 open Assets:Cash
1970-01-01 open Assets:Positions
1970-01-01 open Expenses:Financial:Commissions

2018-03-29 * "Buy 10 BILI"
Assets:Cash
Assets:Positions 10 BILI {11.4899 USD, 2018-03-29} @ 11.4899 USD
Expenses:Financial:Commissions 0.1 USD

turns into:

2018-03-29 * "Buy 10 BILI"
Assets:Cash -115.0 USD
Assets:Positions 10 BILI {11.4899 USD, 2018-03-29} @ 11.4899 USD
Expenses:Financial:Commissions 0.1 USD

The full-precision number for Assets:Cash should be -114.9990 USD, but
it was incorrectly rounded to -115.0 USD.

In the documentation
https://docs.google.com/document/d/1lgHxUUEY-UVEgoF6cupz2f_7v7vEF7fiJyiSlYYlhOo/
section "Precision of Inferred Numbers":

> If the default tolerance is not overridden in the input file—and
> therefore is zero—the full precision will be used; no rounding occurs.

In the minimal example above, the precision seems to be inferred from
0.1 USD commission. If I change the input file, changing commission from
0.1 USD to 0.1000 USD, the output is correct:

2018-03-29 * "Buy 10 BILI"
Assets:Cash -114.9990 USD
Assets:Positions 10 BILI {11.4899 USD, 2018-03-29} @ 11.4899 USD
Expenses:Financial:Commissions 0.1000 USD

My broker-dealer returns the commission as 0.1 USD because it is a
floating point number in JSON (you cannot end numbers with zeros). But
behind the scenes it calculates the cash balance with the full-precision
numbers (e.g. 0.1000 USD). After a few transactions, the balance of
Assets:Cash has a difference of more than 0.005 USD with the
broker-dealer and thus fails the balance assertion.

Is there a way to turn off the implicit rounding without padding 0.1 USD
to 0.1000 USD in the input file?

I am using tagged release 2.0.0.


--
Zhuoyun Wei

[Martin Blais]

This is correct.

First, the PRINT command of the shell (which is what I believe you're using) will render the numbers at their stored precision:

https://bitbucket.org/blais/beancount/src/4a3995617e7df5ffa3377711382c7ad2f9d8935b/beancount/query/query_execute.py?at=default&fileviewer=file-view-default#query_execute.py-92

I kept it this way originally because PRINT was intended for debugging, so rounding using the most common precision seen in the file seemed it would create problems with interpreting the data.  I'm not 100% this was the right choice.

Second, about rounding: by default the tolerance used is that which is inferred on the transaction:

https://bitbucket.org/blais/beancount/src/4a3995617e7df5ffa3377711382c7ad2f9d8935b/beancount/core/interpolate.py?at=default&fileviewer=file-view-default#interpolate.py-324

The comment you're referring to applies to the case where a tolerance cannot be inferred at all (like in the first example in that section). In that case, no rounding occurs. I'll clarify the docs.

(This hurts my head too.)

--
Zhuoyun Wei

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180401091341.GA26139%40tarball.wzyboy.org.
For more options, visit https://groups.google.com/d/optout.

[Zhuoyun Wei]

Thanks for explaination.

It seems that the implicit rounding occurs not only in "PRINT", but also
has effect on balance calculation:

1970-01-01 open Assets:Cash
1970-01-01 open Assets:Positions
1970-01-01 open Expenses:Financial:Commissions

2018-03-29 * "Buy 10 BILI"
Assets:Cash
Assets:Positions 10 BILI {11.4899 USD, 2018-03-29} @ 11.4899 USD
Expenses:Financial:Commissions 0.1 USD

2018-04-01 balance Assets:Cash -114.9990 USD


The last balance assertion fails:


Balance failed for 'Assets:Cash': expected -114.9990 USD != accumulated -115.0 USD (0.0010 too little)

Is there a way to turn off the implicit rounding without padding 0.1 USD

to 0.1000 USD?




2018-04-04 01:53:19 Martin Blais <bl...@furius.ca>:

> beancount+...@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/

> CAK21%2BhPeexbubq8dEjvYOYuKd1tMOXh36CuByWPAG_yJa%3Df7zw%40mail.gmail.com.

> For more options, visit https://groups.google.com/d/optout.

--
Zhuoyun Wei

[Martin Blais]

On Wed, Apr 4, 2018 at 2:45 AM, Zhuoyun Wei <wzy...@wzyboy.org> wrote:

Thanks for explaination.

It seems that the implicit rounding occurs not only in "PRINT", but also
has effect on balance calculation:


1970-01-01 open Assets:Cash
1970-01-01 open Assets:Positions
1970-01-01 open Expenses:Financial:Commissions

2018-03-29 * "Buy 10 BILI"
  Assets:Cash
  Assets:Positions                 10 BILI {11.4899 USD, 2018-03-29} @ 11.4899 USD
  Expenses:Financial:Commissions  0.1 USD

2018-04-01 balance Assets:Cash -114.9990 USD


The last balance assertion fails:


Balance failed for 'Assets:Cash': expected -114.9990 USD != accumulated -115.0 USD (0.0010 too little)

I believe you may have misunderstood my explanation.

The interpolated number for cash /does/ use the inferred tolerance from this transaction to round it to a reasonable value.

Is there a way to turn off the implicit rounding without padding 0.1 USD
to 0.1000 USD?

Not at the moment, no.

What you could do is ensure your importer always formats those floating-point numbers with two digits of precision.

(I'm wondering if, in addition to 

- the tolerance and 

- the rendering precision, 

we should be able to set 

- the rounding resolution

in those new options I'll add to override all the inferred values.)

> beancount+unsubscribe@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/

> CAK21%2BhPeexbubq8dEjvYOYuKd1tMOXh36CuByWPAG_yJa%3Df7zw%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

--
Zhuoyun Wei

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180404064534.GA19470%40herus.wzyboy.org.

[Zhuoyun Wei]

2018-04-04 02:51:03 Martin Blais <bl...@furius.ca>:

>
> I believe you may have misunderstood my explanation.
> The interpolated number for cash /does/ use the inferred tolerance from this transaction to round it to a reasonable
> value.
>
>

Oh, okay. This is surprising indeed. I thought Beancount is very strict
on internal calculation so the rounding only occurs when the result is
for human eyes (e.g. PRINT). I never though Beancount does implicit
rounding for non-human-readable things...

Is there a reason for such a design? The product of two
two-decimal-digit numbers is only a four-decimal-digit number. It is
even not bad for human eyes IMHO.

>
> Is there a way to turn off the implicit rounding without padding 0.1 USD
> to 0.1000 USD?
>
>
> Not at the moment, no.
>
> What you could do is ensure your importer always formats those floating-point numbers with two digits of precision.
>

Thanks. I'll take care of the decimal digits in the importer in the
future.

> (I'm wondering if, in addition to 
> - the tolerance and 
> - the rendering precision, 
> we should be able to set 
> - the rounding resolution
> in those new options I'll add to override all the inferred values.)
>
>

Sounds like a good idea! For the pedant and the OCD, turning of rounding
all entirely is a good choice :-)

> > beancount+...@googlegroups.com.

> > To post to this group, send email to bean...@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/
> > CAK21%2BhPeexbubq8dEjvYOYuKd1tMOXh36CuByWPAG_yJa%3Df7zw%40mail.gmail.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> Zhuoyun Wei
>
> --
> You received this message because you are subscribed to the Google Groups "Beancount" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@
> googlegroups.com.
> To post to this group, send email to bean...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180404064534.GA19470%
> 40herus.wzyboy.org.

> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> beancount+...@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/

> CAK21%2BhNtSDWCxkFS8BbcBgp5wbXvLSSind089th%2Btf_Y9QAiWw%40mail.gmail.com.

[Martin Michlmayr]

Can you confirm that this is a bug:
https://bitbucket.org/blais/beancount/issues/235

In short, price statements affect the precision in the display context.

I have 15.99 GB00BPN5P782 {113.6367 GBP}

15.99 * 113.6367 = 1817.050833

But due to
option "inferred_tolerance_default" "GBP:0.01"

I get 1817.05 which is what I want.

However, after a `price GB00BPN5P782 113.6382 GBP` statement, CONVERT()
gives me 1817.0748 GBP. I would expect 1817.07 GBP here.

--
Martin Michlmayr
http://www.cyrius.com/

[Martin Blais]

On Wed, Apr 4, 2018 at 8:14 AM, Martin Michlmayr <t...@cyrius.com> wrote:

Can you confirm that this is a bug:
https://bitbucket.org/blais/beancount/issues/235

In short, price statements affect the precision in the display context.

It's more general than that: any number seen in the input affects the precision in the display context.

I have 15.99 GB00BPN5P782 {113.6367 GBP}

15.99 * 113.6367 = 1817.050833

But due to
   option "inferred_tolerance_default" "GBP:0.01"

I get 1817.05 which is what I want.

No.

inferred_tolerance_default affects only rounding.

Rounding only occurs on interpolation, which is when you omit a number and let Beancount calculate it for you.

The number that's calculated there is 1817.050833.

bean-doctor context rounding.beancount 3

Hash:2b8a5cee474bed43d08b9de72fcd2202

Location: /home/blais/r/q/beancount-data/user/michlmayr/rounding.beancount:3

------------ Balances before transaction

  Assets:Something

  Assets:Cash

------------ Transaction

2018-04-05 *

  Assets:Something       15.99 GB00BPN5P782 {113.6367 GBP, 2018-04-05}  ;  1817.050833 GBP

  Assets:Cash       -1817.0508 GBP                                      ; -1817.050833 GBP

Tolerances: GB00BPN5P782=0.005

Basis: (1817.050833 GBP)

------------ Balances after transaction

* Assets:Something      15.99 GB00BPN5P782 {113.6367 GBP, 2018-04-05}

* Assets:Cash                                        -1817.050833 GBP

It's too complicated.

I have succeeded in creating a system which infers reasonable values automatically.

But I have failed to create a simple system.

I should probably add all explicit options to let users override tolerance, display, and rounding values for both units and cost/prices and call it a day.

However, after a `price GB00BPN5P782 113.6382 GBP` statement, CONVERT()
gives me 1817.0748 GBP.  I would expect 1817.07 GBP here.

--
Martin Michlmayr
http://www.cyrius.com/

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180404121416.nlvakxlvrps2vvfd%40jirafa.cyrius.com.