Balance calculation order

[tripun goel]

I am implementing rounding option and to do so at the correct level, I am trying to determine the importance of the order in which balance is calculated.  I noted that ledger calculates and add to balance in order of parsing the postings. Even if I change the order of the postings without changing dates and any other data , the result is going to be the same, since ledger does not round at all for precision. Here is an example in which I have two commodities to be evaluated in terms of GBP.

D 1000.000 GBP
  
2010-01-10 * Buy XXX
    Assets:P                      1 XXX @@ 8.123 GBP
    Assets:C

2010-01-11 * Buy XXX
    Assets:P                      1 XXX @@ 8.123 GBP
    Assets:C

2010-02-10 * Buy XXX
    Assets:P                      1 XXX @@ 8.123 GBP
    Assets:C

2010-03-10 * Buy XXX
    Assets:P                      1 XXX @@ 8.123 GBP
    Assets:C

2010-01-10 * Buy AAA
    Assets:P                      1 AAA @@ 9.123 GBP
    Assets:C
   
2010-02-10 * Buy AAA
    Assets:P                      1 AAA @@ 9.123 GBP
    Assets:C
   
2010-03-10 * Buy AAA
    Assets:P                      1 AAA @@ 9.123 GBP
    Assets:C
   
2010-01-12 * Buy AAA
    Assets:P                      1 AAA @@ 9.123 GBP
    Assets:C
   

P 2011/01/01 XXX  10.123 GBP
P 2011/01/01 AAA 10.123 GBP

or this example

D 1000.000 GBP
  
2010-01-10 * Buy XXX
    Assets:P                      1 XXX @@ 8.123 GBP
    Assets:C1

2010-01-11 * Buy XXX
    Assets:P                      1 XXX @@ 8.123 GBP
    Assets:C1

2010-02-10 * Buy AAA
    Assets:P                      1 AAA @@ 9.123 GBP
    Assets:C
   
2010-03-10 * Buy AAA
    Assets:P                      1 AAA @@ 9.123 GBP
    Assets:C
   
2010-02-10 * Buy XXX
    Assets:P                      1 XXX @@ 8.123 GBP
    Assets:C1

2010-03-10 * Buy XXX
    Assets:P                      1 XXX @@ 8.123 GBP
    Assets:C1

2010-01-10 * Buy AAA
    Assets:P                      1 AAA @@ 9.123 GBP
    Assets:C
       
2010-01-12 * Buy AAA
    Assets:P                      1 AAA @@ 9.123 GBP
    Assets:C
   

P 2011/01/01 XXX 10.123 GBP
P 2011/01/01 AAA 10.123 GBP
 
gives the same result

          12.000 GBP  Assets
         -36.492 GBP    C
         -32.492 GBP    C1
          80.984 GBP    P
--------------------
          12.000 GBP

, now if I round to two places after decimal  per posting that does not make sense since it does not add up to the total

          12.000 GBP  Assets
         -36.480 GBP    C
         -32.480 GBP    C1
          80.960 GBP    P
--------------------
          12.000 GBP

So in order to round correctly , I must add the total such that it adds up postings of one commodity and round it and then add to the balance. So the result rounded to two places should be
          12.000 GBP  Assets
         -36.490 GBP    C
         -32.490 GBP    C1
          80.980 GBP    P
--------------------
          12.000 GBP


My purpose is to sort the postings based on order of parsing and commodity. So that I can round the total amount calculated from postings per commodity,  to get something like 40.49 GBP (AAA) + 40.49 GBP (XXX) = 80.98 GBP

- Will changing the order of postings affect other aspects or calculations? Any counter example?

Regards
Tripun Goel

[Martin Michlmayr]

* tripun goel <tri...@gmail.com> [2014-07-17 07:15]:

> I am implementing rounding option and to do so at the correct level,
> I am trying to determine the importance of the order in which
> balance is calculated.

I guess I don't fully understand why it's required to sort. Can you
tell me how ledger calculates the sum when you have one commodity
(e.g. AAA) and that's valued in another (e.g. USD).

You said it does not use number of AAA * value of AAA in USA. So how
is each individual step valued?

let's say we have 2 AAA and then we have 2 P entries showing that the
value of AAA in USD changes. How does ledger calculate this?

What if we have 1 AAA and later get another 2 AAA, how are the 3
valued in USD?

What I'm trying to get at: at the moment, ledger keeps all precision,
but it shows you the info in whatever you specify with D. Sometimes,
ledger generates an "Adjustment" entry when rounding happens.

Why cannot we generate such Adjustment entries too when necessary?
e.g. let's take an example

D $1000.00

2014-05-01 Test 1
Assets:Investment 1 AAA at $10.454
Assets:Cash

2014-06-01 Test 2
Assets:Investment 1 AAA at $10.454
Assets:Cash

ledger reg:
2014-05-01 Test 1 $10.450 $10.450
2014-06-01 Test 2 $10.450 $10.450
2014-06-01 <Adjustment> $0.010 $20.910

Why is this not possible?

Hmm, I remember in one of our previous conversations arguing against
such a thing; maybe this is something that needs to be configurable;
for example, when you have expenses of $10.454 on 2014-05-1 and again
$10.454 on 2014-06-01 you probably want your total to be $20.90 (with
rounding); but when I track how much my shares are worth, I definitely
want the result as number of shares * current value, so we'd need to
do the Adjustment above.

--
Martin Michlmayr
http://www.cyrius.com/

[tripun goel]

As far as I understand, ledger keeps a list of amounts from each post with balance object , before displaying a value function is called and then it iterates again through the list. The amounts are then valued and added to the balance. The total is accurate because there is no rounding in any calculation here. Why this is needed? I think it has more to do with object oriented design the ledger has. A balance value is value equivalent amounts valued determined by market valuation at specific time.  Any better explanation will be helpful.

if we round per transaction and keep an adjustment account with each commodity doesn't seems to be a good idea. Although , I have not investigated this path but I feel it will be more complex.

Why I need to sort? iterating through the sorted lists it will be easier to round subtotals and hence matching the multiplication of total of quantity *  current price .

--
Martin Michlmayr
http://www.cyrius.com/

--

---
You received this message because you are subscribed to a topic in the Google Groups "Ledger" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ledger-cli/yX1PFXXJekM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ledger-cli+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Wiegley]

>>>>> tripun goel <tri...@gmail.com> writes:

> As far as I understand, ledger keeps a list of amounts from each post with
> balance object , before displaying a value function is called and then it
> iterates again through the list. The amounts are then valued and added to
> the balance. The total is accurate because there is no rounding in any
> calculation here. Why this is needed? I think it has more to do with object
> oriented design the ledger has. A balance value is value equivalent amounts
> valued determined by market valuation at specific time.  Any better
> explanation will be helpful. if we round per transaction and keep an
> adjustment account with each commodity doesn't seems to be a good
> idea. Although , I have not investigated this path but I feel it will be
> more complex.
>
> Why I need to sort? iterating through the sorted lists it will be easier to
> round subtotals and hence matching the multiplication of total of quantity
> *  current price .

Tripun,

I really did not understand anything from these two paragraphs. Can you give
some concrete examples of (1) what you think the problem is, and (2) what your
proposed solution would look like? I still cannot fathom how sorting could
have any relevance to the question of rounding in this case.

Thanks, John

[tripun goel]

John,

Problem is that I have to implement precision directive as discussed with examples in this post
 https://groups.google.com/forum/#!topic/ledger-cli/m-TgILbfrwA.
In short, if you mention precision 2 means it should round the amounts to 2 places after decimal point. The situation is if I round the amounts individually, I get rounding error in the total. Therefore, if you multiply total quantity of a commodity by its final exchange price, it does not match the calculated balance which should logically happen.

Now in a journal there will be multiple commodities which may have exchange price. The amount to be rounded must be round of  (total quantity of a commodity * exchange price ) and then added to the balance.

In order to get this I believe I can sort the list of amounts by commodity. so that while iterating through the list it calculates value of each amount and I round at each point where all amounts of a particular commodity have been valued. The balance will then be round(total of commodity A) + round (total of commodity B) ... and so on.

Right now, the ledger keeps the amounts in the order of the posts are parsed.

Consider the following example,  the posts are not sorted by date but by commodity .

 After valuation  AAA ( 4 * 10,123=40.492 GBP)  should be rounded to 40.49 GBP and XXX ( 4 *  10.123 =40.492 GBP) should be rounded to 40.49  GBP

so the total  balance in account "P" should be 80.980 GBP after rounding.

As of now ,  If I enter posts in any manner it does not matter for ledger and the balance will always be the same.  But keeping it sorted by commodity will be helpful for rounding the subtotal of each commodity.

Again, if I would have rounded each amount after valuation I would get 80.960 GBP ( 10.12 + 10.12 ... 8 times) which is not correct.

In the program, I think that I can stable sort the list of posts (std::list<post_t *>) posts with account object and the amounts map (std::map<commodity_t *, amount_t>) of balance object to get the desired order.

[John Wiegley]

>>>>> tripun goel <tri...@gmail.com> writes:

> Right now, the ledger keeps the amounts in the order of the posts are
> parsed.

> Consider the following example,  the posts are not sorted by date but by
> commodity .

>  After valuation  AAA ( 4 * 10,123=40.492 GBP)  should be rounded to 40.49
> GBP and XXX ( 4 *  10.123 =40.492 GBP) should be rounded to 40.49  GBP so
> the total  balance in account "P" should be 80.980 GBP after rounding. As
> of now ,  If I enter posts in any manner it does not matter for ledger and
> the balance will always be the same.  But keeping it sorted by commodity
> will be helpful for rounding the subtotal of each commodity. Again, if I
> would have rounded each amount after valuation I would get 80.960 GBP (
> 10.12 + 10.12 ... 8 times) which is not correct.

I still don't see what sorting is doing to fix anything in this example. Why
are you rounding the value of individual postings? What does sorting have to
do with that? The balance_t type's job is to accumulate value segregated by
commodity. Are you meaning to use that?

I would never trust a rounding methodology based on sorting postings. We must
find a solution that is not so ad-hoc. I'd like to see some math involved
here as well, rather than just descriptions of how the code could be changed
to produce the expected value.

Right now the model is very simple: Every posting has a quantity, a commodity,
and optional cost. That cost is always normalized to a "per unit" cost
internally, so it doesn't matter if the user uses @ or @@, it is the same to
Ledger. The historical value of a posting is the cost times the quantity; the
current "market" value of a posting is the valuation of its commodity times
its quantity.

And account's balance is the sum of its postings. An account's value either
historically or present-day is the sum of the values of its postings, as
described previously.

Now, where can rounding come into play here? There are several places:

- Round postings with extra precision right away.
- Round after multiplying the cost by the quantity
- Round after multiplying the present value by the quantity
- Whenever any two postings are summed, round the sum, thereby producing a
rather lossy rounding of the account balance (in all cases).
- Round the account total after summing the rationals internally.

No one of these methods is universally correct. Some system may require one
or more.

None of these methods involve sorting.

So let us start this issue over, from the beginning, and get to the bottom of
what is potentially correct, and how best to introduce options to enable those
features. The default behavior will remain as it is now, but I am open to
several possibilities for adding these extensions:

command-line options
global directives
account/payee/commodity specific directives
metadata directives

John

[tripun goel]

On Thu, Jul 24, 2014 at 3:05 PM, John Wiegley <jo...@newartisans.com> wrote:

>>>>> tripun goel <tri...@gmail.com> writes:

> Right now, the ledger keeps the amounts in the order of the posts are
> parsed.

> Consider the following example,  the posts are not sorted by date but by
> commodity .

>  After valuation  AAA ( 4 * 10,123=40.492 GBP)  should be rounded to 40.49
> GBP and XXX ( 4 *  10.123 =40.492 GBP) should be rounded to 40.49  GBP so
> the total  balance in account "P" should be 80.980 GBP after rounding.  As
> of now ,  If I enter posts in any manner it does not matter for ledger and
> the balance will always be the same.  But keeping it sorted by commodity
> will be helpful for rounding the subtotal of each commodity.  Again, if I
> would have rounded each amount after valuation I would get 80.960 GBP (
> 10.12 + 10.12 ... 8 times) which is not correct.

I still don't see what sorting is doing to fix anything in this example.  Why
are you rounding the value of individual postings?  What does sorting have to
do with that?  The balance_t type's job is to accumulate value segregated by
commodity.  Are you meaning to use that?

This example is already sorted , I mentioned it above. Rounding individual postings is fine at the time of multiplying by cost. This is one of the place you mentioned at  end of the mail.  It is not correct to round at the time of multiplying by current value, i was just trying to indicate and compare the error. I have discussed this in the end. Yes, i want to use the balance_t object.

I would never trust a rounding methodology based on sorting postings.  We must
find a solution that is not so ad-hoc.  I'd like to see some math involved
here as well, rather than just descriptions of how the code could be changed
to produce the expected value.

I am also skeptical about this method. Probably we need to find another method but I am less aware of any accounting trick on it   The problem is how do you find total worth of your shares. You will simply multiply the total quantity with market value? The rounding feature is expected to match this total irrespective of how these shares were purchased. Sorting will help round only subtotal of each commodity if required.  If not sorting then some kind of metadata may be helpful ( I haven't thought about this fully).

 

Right now the model is very simple: Every posting has a quantity, a commodity,
and optional cost.  That cost is always normalized to a "per unit" cost
internally, so it doesn't matter if the user uses @ or @@, it is the same to
Ledger.  The historical value of a posting is the cost times the quantity; the
current "market" value of a posting is the valuation of its commodity times
its quantity.
 

Thanks for making the model clear.

 

And account's balance is the sum of its postings.  An account's value either
historically or present-day is the sum of the values of its postings, as
described previously.

 

 Now this sum is what we need to check . An account may have multiple commodities and the postings may not be sorted by commodity but you will expect your total to match worth of your commodities.As in this example.


D 1000.00 EUR

2012-01-01 *
   A                        1 AAA @@ 10.00 EUR
   C
2012-02-01 *
   A                        1 BBB @@ 20.00 EUR

   C

P 2012-07-01 AAA 10.123 EUR
P 2012-07-01 BBB 20.123 EUR

 

10.12 + 20.12 = 30.24 EUR

Well this example has been discussed many times. If you round the total sum you will get 30.25 EUR but in actual you will have 30.24 EUR. This is because you will get 10.12 EUR from valuation of total quantity of AAA which is one in this case and 20.12 EUR from valuation of total quantity of BBB.

Now, where can rounding come into play here?  There are several places:

  - Round postings with extra precision right away.

not required, purpose is to maintain as much precision we can.

 

  - Round after multiplying the cost by the quantity

This is required. Practically, transactions rounded to two places after decimal are common. For example ,
2012-01-01 *
   A                        1 AAA @@ 10.123 EUR
   C

 
C should reflect 10.12 EUR  in balance. Please correct if you meant differently.

  - Round after multiplying the present value by the quantity

This is to be avoided, for simple reason that the rounding error will reflect in the total.

 

  - Whenever any two postings are summed, round the sum, thereby producing a
    rather lossy rounding of the account balance (in all cases).

This is to be avoided, not required.

 

  - Round the account total after summing the rationals internally.

Not exactly the total but whenever postings related to one commodity are summed, its subtotal should be rounded. Please correct me if I am wrong.

No one of these methods is universally correct.  Some system may require one
or more.

None of these methods involve sorting.

None of these require sorting because it does not tackle the  condition when account has multiple commodities , rounding the account total after summing the rationals internally is required but then it should be done for subtotal of each commodity and not just the composite of all.

So let us start this issue over, from the beginning, and get to the bottom of
what is potentially correct, and how best to introduce options to enable those
features.  The default behavior will remain as it is now, but I am open to
several possibilities for adding these extensions:

  command-line options
  global directives
  account/payee/commodity specific directives
  metadata directives

I don't want to change the default behavior but only if the feature is used. This is a commodity specific directive.

 

John

[John Wiegley]

>>>>> tripun goel <tri...@gmail.com> writes:

> Now, where can rounding come into play here?  There are several places:

>   - Round postings with extra precision right away.

> not required, purpose is to maintain as much precision we can.

Actually, this has value. People who do time accounting for ledger, for
example a lawyer who bills his clients, will want to round up to the nearest
hour at the posting level.

>   - Round after multiplying the cost by the quantity

> This is required. Practically, transactions rounded to two places after
> decimal are common. For example , 2012-01-01 *    A                        1
> AAA @@ 10.123 EUR    C   C should reflect 10.12 EUR  in balance. Please
> correct if you meant differently.

OK, I can certainly see this as an option.

>   - Round after multiplying the present value by the quantity

> This is to be avoided, for simple reason that the rounding error will
> reflect in the total.

Again, there may be non-monetary scenarios where this is desired.

>   - Round the account total after summing the rationals internally.

> Not exactly the total but whenever postings related to one commodity are
> summed, its subtotal should be rounded. Please correct me if I am wrong.

Since account balances use balance_t, postings are always summed according to
their commodity. This is why I remain mystified that sorting has entered the
discussion.

John

[Martin Michlmayr]

* John Wiegley <jo...@newartisans.com> [2014-07-24 04:35]:

> Now, where can rounding come into play here? There are several places:
>
> - Round postings with extra precision right away.
> - Round after multiplying the cost by the quantity
> - Round after multiplying the present value by the quantity
> - Whenever any two postings are summed, round the sum, thereby producing a
> rather lossy rounding of the account balance (in all cases).
> - Round the account total after summing the rationals internally.
>
> No one of these methods is universally correct. Some system may require one
> or more.

John, what's the best way to move forward from where we are? Would
more use scenarios and test cases help, so we can get a better idea of
where/when we need rounding.

Since we started discussing rounding, I've definitely came across
several situations that I hadn't thought of before.

[tripun goel]

On Fri, Jul 25, 2014 at 12:50 AM, John Wiegley <jo...@newartisans.com> wrote:

>>>>> tripun goel <tri...@gmail.com> writes:

>     Now, where can rounding come into play here?  There are several places:

>       - Round postings with extra precision right away.

> not required, purpose is to maintain as much precision we can.

Actually, this has value.  People who do time accounting for ledger, for
example a lawyer who bills his clients, will want to round up to the nearest
hour at the posting level.

Well this is a new perspective. I think at all places where rounding is not required for monetary purpose, a flag can be added which when set will round at that place too. for example , at the posting level as you said.

 

>       - Round after multiplying the cost by the quantity

> This is required. Practically, transactions rounded to two places after
> decimal are common. For example , 2012-01-01 *    A                        1
> AAA @@ 10.123 EUR    C   C should reflect 10.12 EUR  in balance. Please
> correct if you meant differently.

OK, I can certainly see this as an option.

>       - Round after multiplying the present value by the quantity

> This is to be avoided, for simple reason that the rounding error will
> reflect in the total.

Again, there may be non-monetary scenarios where this is desired.

Again, a flag can be added and set if this is required.

 

>       - Round the account total after summing the rationals internally.

> Not exactly the total but whenever postings related to one commodity are
> summed, its subtotal should be rounded. Please correct me if I am wrong.

Since account balances use balance_t, postings are always summed according to
their commodity.  This is why I remain mystified that sorting has entered the
discussion.

ah, I see !! this has been the point of contention, when we use valuation , all those commodities are exchanged with one single primary commodity before being actually added. So, it does not maintain the balance per commodity if you valuate. The order in which they are added is the order of parsing. If you do not valuate then it will add the quantities according to their respective commodities.

Tripun

 

[John Wiegley]

>>>>> tripun goel <tri...@gmail.com> writes:

> Well this is a new perspective. I think at all places where rounding is not
> required for monetary purpose, a flag can be added which when set will round
> at that place too. for example , at the posting level as you said.

But it will need to selectable per-commodity and/or per-account, so my feeling
is that options are not the way to go here, but flags on commodities and
accounts.

commodity billhour
round 2 up

I think this has been proposed before.

John

[John Wiegley]

>>>>> Martin Michlmayr <t...@cyrius.com> writes:

> John, what's the best way to move forward from where we are? Would more use
> scenarios and test cases help, so we can get a better idea of where/when we
> need rounding.

I think we need to stop thinking about the implementation, and just talk about
the model we all share in common. What forms of rounding are there? When
might it be wanted? When should we round values, and when should we round
only displayed values?

John

[tripun goel]

I agree, it will be a commodity sub directive.