An idea about commodity reporting

[John Wiegley]

I was thinking about commodity valuations last night, and I realized there is
fundamental detail which Ledger has no concept of right now: the finality of
postings.

There are postings which are "open", waiting to be "closed" on another day.
For example, if I get paid at work, I now have money that is waiting to be
spent. This money is "open", in the sense that it's future has not been
determined. It therefore has a fluctuating value based on the cost of the
dollar. It's a movement of funds *in transition* from one place to another.

Other postings are final, like when I pay $20 to see a movie. That $20 is
now finally and absolutely *spent*. It has no future, with regard to my
finances. Whatever valuation the amount had on the day I saw the movie is
the valuation it must always have.

Reflecting on this, it seems that Income/Expenses are by nature final
categories, while Assets/Liabilities are open. Then again, sometimes there
are individual postings which should be considered final, even though their
parent account is not by rule.

To accomodate this detail (which some have tried to express through fixated
commodities), I suggest the following additional syntax:

final REGEXP

Any accounting matching REGEXP is considered a final category.
Postings which move amounts into or from these accounts will
always show amounts in terms of the posting's value at that time.

In addition, if a period (.) is found at the end of a posting's amount or
cost, the amount of that posting is considered final:

Expenses:Food $20.00.

Internally, this usage would be modelled using fixated commodities. The $
would get annotated with the date of the posting, and then any use of -V or
-X would constrain the valuation to that date. If multiple commodities occur
in the transaction, then a fixed price is automatically set as well.

I would like to know what those who use multiple commodities think about
this, and if it would make it possible to easily model your dealings with
multiple currencies.

John

[Roel Vanhout]

> Postings which move amounts into or from these accounts will
> always show amounts in terms of the posting's value at that time.

I'm a bit confused about this part. How would this work for cash
accounts? How is the value at the time calculated? More specifically,
does this mean that it will show me the Net Present Value of my
accounts? That would be strange, considering there would be no actual
cash changes. I think this feature is moving into the realm of finance
more than accounting, I'm not sure of the added value.

I guess this is related to the example in a previous post about using
HOME as a commodity, right? Although I see the appeal of tracking
floating values over time like this, it's non-standard in the
accounting world. The 'regular' way to track this is to revaluate the
asset every now and then, and make an explicit appreciation journal
entry, debiting the Equity post. Or am I misunderstanding your
proposal?

cheers,

roel

[John Wiegley]

On Jul 30, 2010, at 4:26 AM, Roel Vanhout wrote:

> I'm a bit confused about this part. How would this work for cash
> accounts? How is the value at the time calculated? More specifically,
> does this mean that it will show me the Net Present Value of my
> accounts? That would be strange, considering there would be no actual
> cash changes. I think this feature is moving into the realm of finance
> more than accounting, I'm not sure of the added value.

What I mean about a "final value" is this: Say you spend $100 from your
credit card to pay someone 100 EUR (I can dream!). The value of those
$, in any future report, should be fixed, since this value cannot change:
it's already spent.

But I buy $100 using 100 EUR and put into my bank account, _those_ $ will
fluctuate based on market values. These values are not "final".

The default will be as it now, that all values are open. I simply propose
adding syntax will let you state that some transactions are now historical
fact, and should vary in their reported value on any future report.

John

[John Wiegley]

On Jul 30, 2010, at 4:58 AM, John Wiegley wrote:

> The default will be as it now, that all values are open. I simply propose
> adding syntax will let you state that some transactions are now historical
> fact, and should vary in their reported value on any future report.

I meant:

The default will be as it now, that all values are open. I simply propose

adding syntax that will let you state that some transactions are now
historical fact, and should NOT vary in their reported value on any future
report.

John

[Roel Vanhout]

So this is only for currency fluctuations? Because the 20$ movie
example made it seem like it was about accounting for time value
fluctuations.

Still, in a 'traditional' accounting system you'd convert the amount
to your 'native' currency, using the fixed exchange rate, at the time
you book the transaction. This would also give you an entry in which
to incorporate conversion costs. I'm not sure what extra you'd get
from keeping the original entry and tagging it later to mean final.

(because that would the usage, right? You'd make an entry and later go
back and tag it as 'final'? Personally I like to make my entry file
purely additive: I add entries at the bottom and never go back and
change old entries. I may cheap every now and then when I see that I
made a typo a few days ago, but generally (theoretically) even those
should be corrected with another journal post.)

cheers,

roel

[John Wiegley]

On Jul 30, 2010, at 5:07 AM, Roel Vanhout wrote:

> So this is only for currency fluctuations? Because the 20$ movie
> example made it seem like it was about accounting for time value
> fluctuations.
>
> Still, in a 'traditional' accounting system you'd convert the amount
> to your 'native' currency, using the fixed exchange rate, at the time
> you book the transaction. This would also give you an entry in which
> to incorporate conversion costs. I'm not sure what extra you'd get
> from keeping the original entry and tagging it later to mean final.

Let's give a quick example that is the actual reason for this idea:

2009-04-17 * Got KRW from the ATM
Assets:Cash 170000 KRW
Assets:Current -102.71 EUR

2009-04-18 * Business dinner
Expenses:Dining 165000 KRW
Assets:Cash

So, here we exchange some EUR for KRW on 4/17. Today, if I valuate these
KRW, the value should in terms of today's price for EUR.

But in the second transaction, I've spent some of those KRW. If today I
ask for a reporting of expenses in terms of EUR, I should use the price
of KRW from 4/18, not today.

This makes the second transaction "final", but the first one not.

John

[Roel Vanhout]

On Fri, Jul 30, 2010 at 11:25 AM, John Wiegley <jo...@newartisans.com> wrote:
> Let's give a quick example that is the actual reason for this idea:
>
> 2009-04-17 * Got KRW from the ATM
>    Assets:Cash               170000 KRW
>    Assets:Current           -102.71 EUR
>
> 2009-04-18 * Business dinner
>    Expenses:Dining           165000 KRW
>    Assets:Cash
>
> So, here we exchange some EUR for KRW on 4/17.  Today, if I valuate these
> KRW, the value should in terms of today's price for EUR.
>
> But in the second transaction, I've spent some of those KRW.  If today I
> ask for a reporting of expenses in terms of EUR, I should use the price
> of KRW from 4/18, not today.
>
> This makes the second transaction "final", but the first one not.

But then we come back to the line in your first mail, "Reflecting on

this, it seems that Income/Expenses are by nature final categories,

while Assets/Liabilities are open." Which is deeply embedded in the
whole double bookkeeping system already, and also how ledger work
already. You can track assets and liabilities in various currencies
(or commodities) and convert to one final valuation at reporting time,
while income and expenses are valuated at the time they are incurred
('fixed').

I've (briefly) looked for transactions where assets would be 'fixed'
or income would not be, but I can only come up with contrived examples
like future incoming cash flows for which the price is fixed through
an option. But again that's finance not accounting.

cheers,

roel

[John Wiegley]

On Jul 30, 2010, at 5:41 AM, Roel Vanhout wrote:

> But then we come back to the line in your first mail, "Reflecting on
> this, it seems that Income/Expenses are by nature final categories,
> while Assets/Liabilities are open." Which is deeply embedded in the
> whole double bookkeeping system already, and also how ledger work
> already. You can track assets and liabilities in various currencies
> (or commodities) and convert to one final valuation at reporting time,
> while income and expenses are valuated at the time they are incurred
> ('fixed').
>
> I've (briefly) looked for transactions where assets would be 'fixed'
> or income would not be, but I can only come up with contrived examples
> like future incoming cash flows for which the price is fixed through
> an option. But again that's finance not accounting.

Another possibility is adding a -H <COMM> flag, which is exactly like
-X <COMM>, except that it converts values using the day of the
transaction, rather than today. This just means you can't have an
"all-in-one" balance report. Rather, you'd need to reflect the
difference between final and open accounts thusly:

ledger bal assets liab -X EUR
ledger bal income expense -H EUR

John

[John Wiegley]

I guess the complexity I'm trying to resolve here is this:

For any given posting there are three possible amounts:

-O The quantity, or the "amount" of the posting
-B The optional cost (per-unit or combined)
-I The optional annotated price (fixed or not fixed)

When you create a report, you want one of these three. Then you want a
reported commodity. Sometimes this is determined by context (-V),
sometimes you want a specific commodity that isn't present in any of
these three (-X COMM).

Then, on top of all that, is the date from which pricing translations
are to be made. Sometimes you want the price current at the time of the
posting (fixed price commodity), sometimes you want the current price,
sometimes you want a specific date (--now=DATE).

And even THEN, sometimes within a single report you want multiple
combinations of these various details, with specific combinations
determined based on the context of usage (which account was involved,
which commodities were involved, when did they happen, etc).

I can either try to invent syntax so that all of the amounts in the final
report mean what you expect them to mean; or I can require you to make
multiple reports, each with an explicit set of options so that Ledger
doesn't try to automate any of it.

Already it's been noticed that Ledger has no way of saying, "Make -V use
historical values".

John

[Roel Vanhout]

On Fri, Jul 30, 2010 at 11:44 AM, John Wiegley <jo...@newartisans.com> wrote:
> Another possibility is adding a -H <COMM> flag, which is exactly like
> -X <COMM>, except that it converts values using the day of the
> transaction, rather than today.  This just means you can't have an
> "all-in-one" balance report.  Rather, you'd need to reflect the
> difference between final and open accounts thusly:
>
>  ledger bal assets liab -X EUR
>  ledger bal income expense -H EUR

I don't fully understand the implications of this yet (and I need to
get some work done too today :) ) but this still seems superfluous to
me. When would you track income and expenses against their history
value? How would you determine historic value of an expanse - let's
say I get amount X from an ATM at an exchange rate of X' on day T1,
and amount Y at an exchange rate Y' on T2. Now I have a pocket full of
money, and I spend (X + Y / 2). How would you value those
historically? There are several valuation models in accounting
practice - historic cost, replacement cost etc. (I don't know if these
are the actual terms in English, they're literal translations from the
Dutch forms). You'd have to provide for these valuation models *in the
code*, whereas normally these are done externally, so explicitly from
the point of view of the ledger. I'm not convinced the added
functionality would add so much value as to offset the added
complexity.

[Russell Adams]

On Fri, Jul 30, 2010 at 05:58:50AM -0400, John Wiegley wrote:
> Already it's been noticed that Ledger has no way of saying, "Make -V use
> historical values".
>
> John
>

This did sound like a rehash of our "fixed commodities" argument (ie:
mileage rates, customer rates per hour, where the value is set at TXN
time and cannot change afterward).

I'm curious to see where this goes.

Thanks.

------------------------------------------------------------------
Russell Adams RLA...@AdamsInfoServ.com

PGP Key ID: 0x1160DCB3 http://www.adamsinfoserv.com/

Fingerprint: 1723 D8CA 4280 1EC9 557F 66E8 1154 E018 1160 DCB3

[Simon Michael]

On 7/30/10 2:41 AM, Roel Vanhout wrote:
> But then we come back to the line in your first mail, "Reflecting on
> this, it seems that Income/Expenses are by nature final categories,
> while Assets/Liabilities are open." Which is deeply embedded in the
> whole double bookkeeping system already,

I think that's right, and that Equity also falls in the second group.

> and also how ledger work
> already. You can track assets and liabilities in various currencies
> (or commodities) and convert to one final valuation at reporting time,
> while income and expenses are valuated at the time they are incurred
> ('fixed').

Is there something in current ledger that treats assets/liabilities/equity different from income/expenses ? I don't
think so. I think John is proposing to add that distinction (with the "final" directive, because obviously you can't
just check accountname =~ "^(income|expenses)(:|$)"...). I agree that some way of specifying account types is necessary.
This is independent of the whole commodity/pricing topic; you need it simply to print a standard-format balance sheet.

It's worth spending time to find the most intuitive mechanism, too, because it's getting hard to call the ledger file
format simple.

I wouldn't call it "final". There must be some standard accounting terminology.
http://en.wikipedia.org/wiki/Chart_of_accounts calls them Balance Sheet Accounts and Profit & Loss accounts, eg.

Also we might find it useful to specify more detailed account types some day, as Quicken/Gnucash do, I guess to adjust
sign and terminology and perhaps function. It suggests something like "account-type TYPE ACCOUNTREGEX" ? Or, if it turns
out detailed type is orthogonal to the major distinction above - I'm not sure - there might be "account-kind ..." as well.

[Simon Michael]

PS if it were me, I would not add any any syntax for this at the individual amount level, at least not until trying it
strictly at the account level first. When folks have this unusual need let them define a special account for it.

[John Wiegley]

On Jul 30, 2010, at 1:31 PM, Simon Michael wrote:

> Also we might find it useful to specify more detailed account types some day, as Quicken/Gnucash do, I guess to adjust sign and terminology and perhaps function. It suggests something like "account-type TYPE ACCOUNTREGEX" ? Or, if it turns out detailed type is orthogonal to the major distinction above - I'm not sure - there might be "account-kind ..." as well.

For a long time I've had a feature request for "account definitions":

account NAME
kind <...>
commodity $
strict

etc. The type of mechanism you're suggesting would fit into that scheme as well.

John

[Simon Michael]

On 8/1/10 3:23 PM, John Wiegley wrote:
> For a long time I've had a feature request for "account definitions":
>
> account NAME
> kind<...>
> commodity $
> strict

Interesting. It would be nice if this could define valid account names as well, when desired, to catch mispelings.
Keeping it optional, lightweight and simple enough. I need this too.

Would the account attributes be a small known set, or would you allow anything, like metadata on transactions ?

[John Wiegley]

On Aug 2, 2010, at 11:38 AM, Simon Michael wrote:

> Interesting. It would be nice if this could define valid account names as well, when desired, to catch mispelings. Keeping it optional, lightweight and simple enough. I need this too.

Yes, they could be used for defining the allowable set of account names.

> Would the account attributes be a small known set, or would you allow anything, like metadata on transactions ?

I imagine it would be both. If an account had metadata, then there could be
an option that every posting affecting that account would receive the same
metadata. There could be a flag for "propagating" metadata...

John

[Łukasz Stelmach]

John Wiegley <jo...@newartisans.com> writes:

Then the getquote "interface" should support specifing the date to get
the exchange rate at the time (day) of the transaction[*], save it in
the pricedb in the correct place, and query the pricedb when building
reports. In general the idea looks very promising.

[*] note: credit cards payments introduce delay between the payment,
exchange, and booking in a bank hence there is some uncertainty.
--
Miłego dnia,
Łukasz Stelmach

[Łukasz Stelmach]

John Wiegley <jo...@newartisans.com> writes:

> On Jul 30, 2010, at 5:07 AM, Roel Vanhout wrote:
>
>> So this is only for currency fluctuations? Because the 20$ movie
>> example made it seem like it was about accounting for time value
>> fluctuations.

[...]

> Let's give a quick example that is the actual reason for this idea:
>
> 2009-04-17 * Got KRW from the ATM
> Assets:Cash 170000 KRW
> Assets:Current -102.71 EUR
>
> 2009-04-18 * Business dinner
> Expenses:Dining 165000 KRW
> Assets:Cash
>
> So, here we exchange some EUR for KRW on 4/17. Today, if I valuate these
> KRW, the value should in terms of today's price for EUR.
>
> But in the second transaction, I've spent some of those KRW. If today I
> ask for a reporting of expenses in terms of EUR, I should use the price
> of KRW from 4/18, not today.

In this particular example it should be price of KRW from 4/17 because
it was when you bought the cash. The cash (but not only) becomes "final"
from the moment you spend it back to the time you purchased it.

A thought about tracking all this scares me.

[John Wiegley]

On Aug 16, 2010, at 12:58 PM, Łukasz Stelmach wrote:

> Then the getquote "interface" should support specifing the date to get
> the exchange rate at the time (day) of the transaction[*], save it in
> the pricedb in the correct place, and query the pricedb when building
> reports. In general the idea looks very promising.

>> But in the second transaction, I've spent some of those KRW. If today I

>> ask for a reporting of expenses in terms of EUR, I should use the price
>> of KRW from 4/18, not today.
>
> In this particular example it should be price of KRW from 4/17 because
> it was when you bought the cash. The cash (but not only) becomes "final"
> from the moment you spend it back to the time you purchased it.

For historical transactions, Ledger knows the exact price of the exchange, there is no need to query a server for it, or use a price from the day before. The price is what you paid at that point, not what the market says it is.

John

[Ben Alexander]

Oh! Please, don't use a period at the end of a number to mean anything.  Some people and places already get easily confused by 2,000.01.  Is that equivalent to two thousand and one hundredth or two and one hundred-thousandth?

Even if there is no parsing ambiguity for ledger, I think some people would become cross-eyed trying to read it.

Also, you would be changing the meaning for anyone who has casually edited 2.00 into 2. some places and into a plain 2 in other places.

Cheers,

Ben