Change in definition of cost

[Martin Michlmayr]

John and I had a long discussion on IRC a few days ago which resulted
in a patch that changes the definition of cost in ledger (-B).

The background of the discussion was bugs #712 and #713.

Bug #712 is about Equity:Capital Gains and Equity:Capital Loss
entries ledger automatically generates that are in my opinion
incorrect. (<http://bugs.ledger-cli.org/show_bug.cgi?id=712>)

Bug #713 is about strange behaviour with -B after a capital gain or
loss is made (<http://bugs.ledger-cli.org/show_bug.cgi?id=713>).

As it turns out, both of these issues can be addressed by changing the
definition of cost in ledger. In the past, if you bought 1 AAA for
$10 and then sold it for $12, ledger would take $12 as the cost. With
the patch, the cost is $10. In my opinion, this brings ledger in line
with accounting expectations.

I've done extensive tests on my own ledger files, but John and I would
like to ask for more people to test the patch before it's applied
(especialy people who made capital gains or losses). You can find the
patch on the 713-costs branch. Please test that branch and let me
know if you see any strange behaviour.

I've attached two files from John that show how ledger sees some
transactions (for the old and new behaviour).

--
Martin Michlmayr
http://www.cyrius.com/

[Martin Blais]

On Wed, May 14, 2014 at 6:15 PM, Martin Michlmayr <t...@cyrius.com> wrote:

John and I had a long discussion on IRC a few days ago which resulted
in a patch that changes the definition of cost in ledger (-B).

The background of the discussion was bugs #712 and #713.

Bug #712 is about Equity:Capital Gains and Equity:Capital Loss
entries ledger automatically generates that are in my opinion
incorrect. (<http://bugs.ledger-cli.org/show_bug.cgi?id=712>)

Bug #713 is about strange behaviour with -B after a capital gain or
loss is made (<http://bugs.ledger-cli.org/show_bug.cgi?id=713>).

As it turns out, both of these issues can be addressed by changing the
definition of cost in ledger.  In the past, if you bought 1 AAA for
$10 and then sold it for $12, ledger would take $12 as the cost.  With
the patch, the cost is $10.  In my opinion, this brings ledger in line
with accounting expectations.

That's how I do it too and it works for me.

I believe I nailed that model right in the v2 rewrite:

Beancount uses "Inventory" objects which contain lots, modified by postings, by account.

An inventory is a MAPPING of "Lot" to a number of units.

A Lot is a triplet of (currency, cost, lot-date).  

(The cost is itself a tuple of (amount, cost-currency)).

Both the cost and lot-date are optional and may be null (None in Python).

If both are null, that's the degenerate case of "just a currency."

That is elegant: operations on currencies and units held at cost work the same way!

For example, a possible inventory for an account could be this map:

    (USD, None, None) -> 12000

    (GOOG, (512.30, USD), None) -> 50

This would be an inventory with $12,000 in USD and 50 units of GOOG bought at $512.30 USD each.

(Typically I keep those in separate accounts, but to make this example demonstrative I put them in the same.)

Selling with a transaction like this:

2014-05-01 * "Selling some Googs"

  Assets:Investments   -10 GOOG {512.30 USD} @ 523.20 USD

  Assets:Investments          5222.00 USD

  Expenses:Commissions   10.00 USD

  Income:CapitalGains

does the following:

- Balancing computes the cost of each posting. For the first leg, this is -10 x 512.30 = -5123.00 USD. The fourth posting is automatically computed from the residual of the sum of costs, that is, -5123.00 + 5232.00 

- The 523.20 USD price on the first posting is NOT USED in the calculation; however, it is used in that it creates a data point in the price database for the rate of "GOOG in USD".

Inventory operations are only allowed to modify IDENTICAL LOTS.

That's the key constraint.

The resulting inventory for the Investments account would be:

    (USD, None, None) -> 6778

    (GOOG, (512.30, USD), None) -> 40

One drawback of this is that the capital gains automatically calculated that way will include the commissions amount. One way to get around this is that you can subtract the commissions account when you do use the amounts to e.g. declare your taxes.

Splits are handled by a transaction that empties all the lots and recreates them:

2014-05-01 * "Example split"

  Assets:Investments         -40 GOOG {512.30 USD}

  Assets:Investments         80 GOOG {256.15 USD}

Would result in this:

    (USD, None, None) -> 6778

    (GOOG, (256.15, USD), None) -> 80

Another constraint is that if a lot has a cost, booking against a lot may not result in a negative number (that raises an error). Relaxing that constraint would allow one to trade in instruments with negative prices, e.g., spread futures.

Dealing with average cost trading or cost basis readjustments (not implemented yet) involves joining together multiple lots and recreating new ones in a way that preserves the total cost in the inventory; with this data structure / model it's quite obvious how to implement them as basic operations on an inventory.

I really like the simplicity of this and am wondering if we could make it even simpler.

Automatic booking against an inventory, e.g. adding automatic FIFO or LIFO, would require having the date of each lot always inserted in the key of the inventory items (from the transaction, not from the lot-date field), along with special rules for selecting which lots a posting is allowed to modify, essentially ignoring the lot-date from the inventory if the posting does not specify it. This is partly why I'm considering making the "lot-date" compulsory and adding a "lot identifier" used to disambiguate booking against an inventory with multiple matching lots of the same cost with just differing dates, which would only be required if the posting constrains it to. One can imagine relaxing the matching rules between a posting and inventory further to allow one to just specify "-40 GOOG" above, without a cost, and if unambiguous, to allow it to just select the only lot that is available.

[thierry]

Martin, John,

I have read again 712 and 713 bugs descriptions. I then tested branch 713-costs. I had to "git cherry-pick 8b3f8d1", but this is anecdote.

There is no difference for me between HEAD and branch 713-costs on *my* data. I have some scripts to do graphics on my ledger data, those scripts are using --basis (= -B) option, and I do not have any regression, nor difference in results.

I just wonder what is now the difference between --price (=-I) and --basis (=-B, --cost) ?

I personally do not use --price option, so I would argue that this patch is correcting bug #712 and #713, and that I vote for merging it in master branch.

Thanks
Thierry

[Martin Michlmayr]

* Martin Michlmayr <t...@cyrius.com> [2014-05-14 18:15]:

> The background of the discussion was bugs #712 and #713.
>
> Bug #712 is about Equity:Capital Gains and Equity:Capital Loss
> entries ledger automatically generates that are in my opinion
> incorrect. (<http://bugs.ledger-cli.org/show_bug.cgi?id=712>)
>
> Bug #713 is about strange behaviour with -B after a capital gain or
> loss is made (<http://bugs.ledger-cli.org/show_bug.cgi?id=713>).
>
> As it turns out, both of these issues can be addressed by changing the
> definition of cost in ledger.

I've merged these patches now. Thanks everyone.