Decimal(): understanding, syntax

[nugget]

Hi all,
i get these very long decimal numbers when using the Decimal() method. See my minimal example. How can I fix this? rounding to two digits seems not to be the solution. Thanks!

2020-01-01 * "format" "1"

  myAccount  1 CHF

2020-01-01 * "format" "1/9"

  myAccount  0.111111111111111104943205418749130330979824066162109375 CHF

2020-01-01 * "format" "round(1/9,2)"

  myAccount  0.11000000000000000055511151231257827021181583404541015625 CHF

2020-01-01 * "format" "10000"

  myAccount  10000 CHF

minimal example code:

from beancount.core import data, amount

from beancount.core.amount import D

from beancount.parser import printer

number = [1,1/9,round(1/9,2),10000]

narration=['1','1/9','round(1/9,2)','10000']

for i, num in enumerate(number):

    num_ = D(num)

    amount_ = amount.Amount(D(num),'CHF')

    P1 = data.Posting(account = 'myAccount',

                    units = amount_,

                    cost = None, price = None, flag = None, meta = None)

    P2 = data.Posting(account = 'myAccount',

                    units = None,

                    cost = None, price = None, flag = None, meta = None)

    T=data.Transaction(meta=None,

                    date='2020-01-01',

                    flag="*",

                    payee='format',

                    narration=narration[i],

                    tags=data.EMPTY_SET,

                    links=data.EMPTY_SET,

                    postings=[P1])

    printer.print_entry(T)

[Martin Michlmayr]

* nugget <c.bo...@gmail.com> [2020-08-07 14:49]:

> i get these very long decimal numbers when using the Decimal()
> method. See my minimal example. How can I fix this? rounding to two
> digits seems not to be the solution. Thanks!

This is what I use to round decimal numbers:

def d_round(d):
return d.quantize(D('.01'), rounding=decimal.ROUND_HALF_UP)

--
Martin Michlmayr
https://www.cyrius.com/

[Daniele Nicolodi]

On 07/08/2020 15:49, nugget wrote:
> Hi all,
> i get these very long decimal numbers when using the Decimal() method.
> See my minimal example. How can I fix this?

What is exactly the problem you want to "fix"?

1/9 is a periodic number that cannot be represented with a finite number
of decimal digits, thus the result you obtain is the expected one.

Similarly, when you round 1/9 to two decimal places you obtain the 0.11
decimal number, which cannot be represented in floating point notation
without rounding error. This becomes evident when you try to obtain a
decimal representation from this floating point representation. Passing
a floating point number to Decimal() (which D() wraps) is almost always
wrong.

If you want exact decimal rounding of a Decimal number, you can call
round() on the Decimal instance, or use quantize() as Martin suggested.

Cheers,
Dan

[Martin Blais]

On Sat, Aug 8, 2020 at 12:31 AM Daniele Nicolodi <dan...@grinta.net> wrote:

On 07/08/2020 15:49, nugget wrote:
> Hi all,
> i get these very long decimal numbers when using the Decimal() method.
> See my minimal example. How can I fix this?

What is exactly the problem you want to "fix"?

1/9 is a periodic number that cannot be represented with a finite number
of decimal digits, thus the result you obtain is the expected one.

Similarly, when you round 1/9 to two decimal places you obtain the 0.11
decimal number, which cannot be represented in floating point notation
without rounding error. This becomes evident when you try to obtain a
decimal representation from this floating point representation. Passing
a floating point number to Decimal() (which D() wraps) is almost always
wrong.

One idea would be to change D() to disallow floating point numbers.

Beancount seems to have a number of users which aren't coming from the open source community or who might not understand these details. Raising an exception might be a good way to signal "you're doing something wrong"

 

If you want exact decimal rounding of a Decimal number, you can call
round() on the Decimal instance, or use quantize() as Martin suggested.

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/c6150170-cf92-4604-1adf-cab1310ed00c%40grinta.net.

[nugget]

thanks for your replies. 

I actually thought Decimal() is meant to turn a float into something nice for bookkeeping (that was my interpretation of the docs)

My underlying question is what is the intended way to handle rounding errors? I.e. when splitting up a transaction into multiple transactions. say distribute a yearly transaction over 12 months or a weekly one over 7 days. 

Naively, I would take  1 CHF and divide it by 12, and make 12 transactions. I dislike having my posting look like "0.08333333333333332870740406406184774823486804962158203125".But simply rounding will introduce an error of 4%:

round(1/12,2) = 0.8, and 

12*0.8 = 0.96

My next best idea is to make sure that of the transactions covers the error:

round(1 - 11*round(1/12)) or something. but also this looks too ugly to be a "correct" way. What would you recommend?

[Martin Michlmayr]

* nugget <c.bo...@gmail.com> [2020-08-08 00:11]:

> I.e. when splitting up a transaction into multiple transactions.
> say distribute a yearly transaction over 12 months or a weekly one
> over 7 days.
>
> Naively, I would take 1 CHF and divide it by 12, and make 12
> transactions. I dislike having my posting look like "
> 0.08333333333333332870740406406184774823486804962158203125".But
> simply rounding will introduce an error of 4%: round(1/12,2) = 0.8,
> and 12*0.8 = 0.96

I'm not sure what your transaction is about, but I'd ask a completely
different question: is it material? If we're only talking about 1
CHF, does it really matter whether you book it on one day vs splitting
it up every month. If we're talking about 1000 CHF, it might make a
material difference, but for 1 CHF I'd say that splitting it across
multiple months is just not worth it.

If it does matter, I would book 0.08 for 11 months and 0.12 in the
last month (or the other way around).

[Daniele Nicolodi]

I think you are looking at this from the wrong perspective. Thus you see
a problem that is not there. beancount ise designed to record real
financial transactions. These involve the exchange of exactly defined
quantities of something, represented by exact decimal numbers (I think
the introduction of the Python documentation covers this nicely for
someone not into the technicalities
https://docs.python.org/3.8/library/decimal.html). For example, if you
split a 1 CHF expense with two friends, two of you will have to pay 0.33
CHF but one will have to pay 0.34 CHF: there is not way you can split a
CHF with smaller granularity.

Of course, you see that the fact that you cannot split CHF with smaller
granularity leads to some "funny" situations: what if I need to split a
0.02 CHF expense in three tranches? There is no way of doing it.

Who designed the Swiss monetary system (and all other monetary systems)
decided that rounding errors (and accumulation of rounding errors) at
the 0.01 CHF level is acceptable.

What this means is that the tool you use to amortize these amounts over
some time units (or that does other divisions) needs to be aware of this
characteristic of real financial transactions. In the case of dividing 1
CHF over 12 months, the best thing you can do is to book 0.08 CHF for 8
months and 0.09 CHF for 4 months. I don't know if there is a library
function somewhere that can do this computation for you.

Please note that this has noting to do with beancount or with Decimal
numbers, but only with the physical basis of the monetary systems (the
granularity of financial transactions is determine by the smaller coin
issued).

You can decide to write your transactions is some form of virtual Swiss
Franc that has higher resolution, but this only moves the problem to a
different decimal place, because you still need to decide what this
resolution is going to be and proper deal with rounding that that level
to have the transactions balance.

Cheers,
Dan

[Daniele Nicolodi]

On 07/08/2020 23:48, Martin Blais wrote:
> On Sat, Aug 8, 2020 at 12:31 AM Daniele Nicolodi <dan...@grinta.net
> <mailto:dan...@grinta.net>> wrote:
>
> On 07/08/2020 15:49, nugget wrote:
> > Hi all,
> > i get these very long decimal numbers when using the Decimal() method.
> > See my minimal example. How can I fix this?
>
> What is exactly the problem you want to "fix"?
>
> 1/9 is a periodic number that cannot be represented with a finite number
> of decimal digits, thus the result you obtain is the expected one.
>
> Similarly, when you round 1/9 to two decimal places you obtain the 0.11
> decimal number, which cannot be represented in floating point notation
> without rounding error. This becomes evident when you try to obtain a
> decimal representation from this floating point representation. Passing
> a floating point number to Decimal() (which D() wraps) is almost always
> wrong.
>
>
> One idea would be to change D() to disallow floating point numbers.
> Beancount seems to have a number of users which aren't coming from the
> open source community or who might not understand these details. Raising
> an exception might be a good way to signal "you're doing something wrong"

It seems the confusion in this case comes from do not realizing that
there is no way to split $1.00 into three equal parts. I don't know if
changing D() to do not accept floats would solve this.

Another thing to potentially consider is to reduce the number of decimal
digits used when serializing Decimal numbers. Maybe 12 (or so) digits
are enough instead that the current 28.

Cheers,
Dan

[Daniele Nicolodi]

On 08/08/2020 10:41, Daniele Nicolodi wrote:
> Another thing to potentially consider is to reduce the number of decimal
> digits used when serializing Decimal numbers. Maybe 12 (or so) digits
> are enough instead that the current 28.

I am sure someone has a model for when 1e-12 BTC will be worth the
equivalent of $0.01 :-) Right now we should be fine with 12 digits.

Cheers,
Dan

[nugget]

As a summary, what i think I leared:

• The design of Beancounts D() technically allows allows higher precision (float) numbers as input.
• But the design philosophy does not. It requires string typed numbers with maximum precision to the second digit. Higher precision is not supported and might cause problems.
  
• It is up to the user to to deal with higher precisions. For example, treating reminders of divisions. There is no general solution on how to treat this.

This works very well for me, now that I know it.

Thanks all for your replies.

best,
nugget

[Martin Michlmayr]

* nugget <c.bo...@gmail.com> [2020-08-10 02:52]:
> - The design of Beancounts D() technically allows allows higher

> precision (float) numbers as input.

> - But the design philosophy does not. It requires string typed numbers

> with maximum precision to the second digit. Higher precision is not
> supported and might cause problems.

> - It is up to the user to to deal with higher precisions. For example,

> treating reminders of divisions. There is no general solution on how to
> treat this.
>
> This works very well for me, now that I know it.
> Thanks all for your replies.
>
> best,
> nugget
>
> On Sunday, August 9, 2020 at 11:15:33 PM UTC+2, Daniele Nicolodi wrote:
> >
> > On 08/08/2020 10:41, Daniele Nicolodi wrote:
> > > Another thing to potentially consider is to reduce the number of decimal
> > > digits used when serializing Decimal numbers. Maybe 12 (or so) digits
> > > are enough instead that the current 28.
> >
> > I am sure someone has a model for when 1e-12 BTC will be worth the
> > equivalent of $0.01 :-) Right now we should be fine with 12 digits.
> >
> > Cheers,
> > Dan
> >
>

> --
> You received this message because you are subscribed to the Google Groups "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/95c3b5fb-41cf-4409-a989-4dbf3ae2b5ffo%40googlegroups.com.

[Martin Michlmayr]

* nugget <c.bo...@gmail.com> [2020-08-10 02:52]:

> - The design of Beancounts D() technically allows allows higher

> precision (float) numbers as input.

Not float as in the data type float. You pass a string to Decimal()
because floats already have rounding that you can avoid with Decimal()

> - But the design philosophy does not. It requires string typed

> numbers with maximum precision to the second digit. Higher
> precision is not supported and might cause problems.

That's incorrect. Beancount handles more precision just fine and
people use that all the time, e.g. for Bitcoin.

The limitation is in the currency you're using (CHF). Beancount can
easily have more fractions for CHF (0.12345 CHF), but unfortunately
the real world doesn't work that way. But you can use 0.12345 CHF
in beancount without a problem.

[Daniele Nicolodi]

On 10/08/2020 03:52, nugget wrote:
> As a summary, what i think I leared:
>

> * The design of Beancounts D() technically allows allows higher

> precision (float) numbers as input.

Floating points numbers have finite precision. Decimal numbers (as in
Python Decimal objects) have arbitrary precision. Thus this statement is
technically false.

Furthermore, floating point numbers do not represent numerical values in
a decimal (as in "base 10") notation, thus they cannot represent exactly
the decimal numbers used to count currency. For example, a floating
point number cannot represent 1.1 or 2.2 exactly. You can convince
yourself of this trying to sum those two numbers:

>>> 1.1 + 2.2
3.3000000000000003

Note that this particular representation of the result depends on how,
by default, Python displays floating point numbers:

>>> '{:.32f}'.format(1.1 + 2.2)
'3.30000000000000026645352591003757'

That's why financial transactions (among other things) are better
recorded using a numerical representation rooted on a base 10 notation,
as the one offered by Python Decimal numbers.

Of course, if you try to convert a floating point number to a Decimal
number you cannot recover from the rounding error introduced by the
floating point representation. Thus:

>>> Decimal(1.1) + Decimal(2.2)
Decimal('3.300000000000000266453525910')

However,

>>> Decimal('1.1') + Decimal('2.2')
Decimal('3.3')

I think this is nicely explained in the documentation I linked in a
previous mail: https://docs.python.org/3.8/library/decimal.html. Is
there anything not clear in the documentation?

> * But the design philosophy does not. It requires string typed numbers

> with maximum precision to the second digit. Higher precision is not
> supported and might cause problems.

No! You can use as many decimal digits as you want. However, this this
choice is determined by the currency you use as it defines the resiltuon
at which financial transactions can be recorded. For USD an EUR (the
currencies I use the most) the resolution is cents, thus my examples use
two decimal digits. If there would be no Euro and Italy would still be
using the Italian Lira (ITL) I would be using a resolution of 50 ITL...

> * It is up to the user to to deal with higher precisions. For example,

> treating reminders of divisions. There is no general solution on how
> to treat this.

I don't know what you mean with "dealing with higher precision" as
beancount and Python Decimal use arbitrary precision, thus provide you
tools to work with any precision you desire.

Treating reminder of division has nothing to do with beancount, it is a
property of quantized physical systems: there is no way to split a $0.01
coin (and have it still be valid currency) as there is no way to split a
photon. By extension, this also apply to transactions that do not
physically exchange coins or bills (you cannot write a check for amounts
with resolution higher than $0.01, etc...).

Cheers,
Dan