beancount.plugins.sellgains

[Martin Blais]

Hi,

I've added a powerful new feature to Beancount this weekend, as a plugin.

Take this transaction:

1999-07-31 * "Sell"

  Assets:US:BRS:Company:ESPP          -81 ADSK {26.3125 USD} @ 26.4375 USD

  Assets:US:BRS:Company:Cash      2141.36 USD

  Expenses:Financial:Fees            0.08 USD

  Income:US:Company:ESPP:PnL      -10.125 USD

By default, the balance check that will be performed is against the "weight" of each posting, for each currency. Here we have just USD's:

  -81 x 26.3125 + 2141.36 + 0.08 + -10.125 = 0 USD

However, because a price annotation is attached to the first posting, there is another check we can do:

  -81 x 26.4375 + 2141.36 + 0.08 = 0 USD

In generic terms: if a price annotation is present on a posting, we can compute an alternative balance using the price (instead of the cost) if present on a posting, or the simple value, for all postings EXCEPT those posted against Income accounts. This is an idea I had talked about a long while ago on one of the mailing-lists, that had been sitting in the back of my mind. 

Recently I was on a long train ride and was integrating some really old data I had written down more than 10 years ago, translating it in Beancount format, some of which wasn't well noted down, so it had occasional mistakes (this was an old text file I started in the 90's where I used to copy-paste transaction details but it wasn't parsed nor verified... I suppose it was the very beginnings of what would become the current system, at the time I wasn't sure how I could represent and use that data).

So I built a plugin to perform this extra check and help me avoid errors with the translation. It can be enabled with

  plugin "beancount.plugins.sellgains"

It only kicks in on transactions that have price annotations attached to them, and ignores the other ones.

Interestingly enough, when I turned it on in my personal file (I have a single large text file with basically _all_ my transactions), ... it discovered some invalid transactions!  This is because most of the time the Income leg amount is left unspecified and Beancount automatically calculates the gains from the transaction. However, this freedom leaves space for making mistakes in the transcription of amounts. (This yet again reinforced my belief that without a lot of automated assertions it's practically impossible to manually enter a meaningful number of stock trading transactions without making a mistake.)

Anyhow, within an hour I had fixed all the errors that were now showing due to the tighter verification. My personal file is now error free and I've got this enabled permanently... I'll leave this extra check as an optional plugin, but I would recommend Beancount users turn it on and tighten their input file with this extra check; you might find some errors in your data entry.

Enjoy,

[Aaron Bull Schaefer]

On Tuesday, April 21, 2015 at 8:46:59 PM UTC-7, Martin Blais wrote:

I've added a powerful new feature to Beancount this weekend, as a plugin.

Anyhow, within an hour I had fixed all the errors that were now showing due to the tighter verification. My personal file is now error free and I've got this enabled permanently... I'll leave this extra check as an optional plugin, but I would recommend Beancount users turn it on and tighten their input file with this extra check; you might find some errors in your data entry.

Very nice! I turned it on and caught an error as well. Always nice to have more safety checks to ensure things are entered correctly. Thank you.

--
Aaron Bull Schaefer
http://elasticdog.com/

[Matthew Harris]

Thanks for adding this. As expected, this reveals a few kinds of errors I don't know how to resolve.

1. Basic rounding errors:

Invalid price vs. proceeds/gains: (0.03576 USD) vs. ()

  2011-01-25 * "Transfer of Assets, 3467.90 USD"

    * Assets:Investments:RothIRA:Vanguard:VTIVX  250.752 VTIVX {18.35 USD} @ 13.83000000000000 USD  

    * Assets:Investments:RothIRA:DodgeCox:DODGX  -30.892 DODGX {148.93 USD} @ 112.26000000000000 USD

One such transaction already has a 2-cent Expenses:RoundingError posting to make everything balance. I suppose I could add a couple cents to an Income:RoundingError account too, but I'd much rather that Beancount recognize that 0.001 DODGX is worth 0.14893 USD, so a rounding error of 0.03576 USD can be ignored.

Do you not have this problem regularly?

2. ESPP (Employee Stock Purchase Plan) purchases:

Invalid price vs. proceeds/gains: (-3570.24557 USD) vs. (-3213.35 USD)

  2005-01-05 * "ESPP Purchase"

    * Assets:Investments:MutualFunds:Fidelity:MSFT   133.667 MSFT {24.04 USD} @ 26.71000000000000 USD

    * Assets:Investments:MutualFunds:Fidelity:Cash  -3213.35 USD                                     

It appears you have some experience with ESPP transactions, so how do you represent this (and could you please add it to the Cookbook)? Each share is worth $26.71, but we can buy it for $24.04. It's been many years since I've had to deal with this, but it appears that the discount is taxed as regular income at the time of sale, and the cost basis ends up being the market price. I don't see how to record at purchase time what won't be income until a future sale.

3. Unknown accounts represented as Equity:UnknownSourceOrSink:

Invalid price vs. proceeds/gains: (15516.59644 USD) vs. ()

  2011-01-20 * "Shares Redeemed"

    * Assets:Investments:MutualFunds:ClipperFund:CFIMX  -245.788 CFIMX {84.13 USD} @ 63.13000000000000 USD

    Equity:UnknownSourceOrSink                             15516.60 USD                                      

    Income:Investments:MutualFunds:ClipperFund:PnL       5162.40 USD                                      

I have some transactions for which I'm missing the corresponding bank statements. I sometimes have a good idea of which account is involved, but I'm not sure. I've been using Equity:UnknownSourceOrSink to represent these so I don't distort my balance sheet. The sellgains plugin specifically looks for Assets accounts. What do you suggest here?

Matthew

On Tuesday, April 21, 2015 at 8:46:59 PM UTC-7, Martin Blais wrote:

[Daniel Clemente]

>
> 1. Basic rounding errors:
>
> Invalid price vs. proceeds/gains: (0.03576 USD) vs. ()
> 2011-01-25 * "Transfer of Assets, 3467.90 USD"
> * Assets:Investments:RothIRA:Vanguard:VTIVX 250.752 VTIVX {18.35 USD} @ 13.83000000000000 USD
> * Assets:Investments:RothIRA:DodgeCox:DODGX -30.892 DODGX {148.93 USD} @ 112.26000000000000 USD
>

I fixed a similar transaction by slightly adjusting the @ or @@ until the warning went away. I don't know about your example, but maybe 13.83 or 112.26 USD aren't exact numbers, maybe they had some decimals like 13.831 USD. I often get reports with 2 decimals while transactions use 3.

> One such transaction already has a 2-cent Expenses:RoundingError posting to make everything balance. I suppose I could add a couple
> cents to an Income:RoundingError account too, but I'd much rather that Beancount recognize that 0.001 DODGX is worth 0.14893 USD, so
> a rounding error of 0.03576 USD can be ignored.

Maybe you know it, but…: the way to ignore small amounts is: option "tolerance" "0.001"

But I would say that 0.03576 USD is already too big to go unnoticed. I would use an Expenses:RoundingError account for these cases. Instead of having Expenses:Rounding and Income:Rounding I post everything (+ or -) to the same one.

> that Beancount recognize that 0.001 DODGX is worth 0.14893 USD

Or you could write it explicitly: 0.001 DODGX @ 0.14893 USD

But anyway 0.001 DODGX = 0.11226 USD from the @ you wrote


These are some vague ideas which may or may not help with your transaction. I don't know about the other 2, sorry.


--
Daniel

[Martin Blais]

On Sat, May 2, 2015 at 6:22 PM, Matthew Harris <mharr...@gmail.com> wrote:

Thanks for adding this. As expected, this reveals a few kinds of errors I don't know how to resolve.

1. Basic rounding errors:

Invalid price vs. proceeds/gains: (0.03576 USD) vs. ()

  2011-01-25 * "Transfer of Assets, 3467.90 USD"

    * Assets:Investments:RothIRA:Vanguard:VTIVX  250.752 VTIVX {18.35 USD} @ 13.83000000000000 USD  

    * Assets:Investments:RothIRA:DodgeCox:DODGX  -30.892 DODGX {148.93 USD} @ 112.26000000000000 USD

One such transaction already has a 2-cent Expenses:RoundingError posting to make everything balance. I suppose I could add a couple cents to an Income:RoundingError account too, but I'd much rather that Beancount recognize that 0.001 DODGX is worth 0.14893 USD, so a rounding error of 0.03576 USD can be ignored.

Matthew, 

Your bug report is _very_ timely!

I've actually been working hard on revamping the mechanism to infer precision over the past weekend and implement the rounding/precision proposal and this example still fails to get handled properly with my new mechanism (though you can set a default of 0.05 USD for all transactions but I assume we don't want that).

I like your suggestion of using the smallest digit of the converted cost in order to locally infer the precision for the cost currency... I hadn't thought of that at all, it's a good idea. The way I carried out inference in the new method, this transaction would not have an inferred value for USD's (precision on cost and price numbers are ignored) and would revert to the default tolerance (which you can override).

I might be able to just add in inference using the method you propose, that is, infer USD precision from max(0.001 x 18.35, 0.001 x 148.93)/2.

Let me see what I can do, I'll be working on this today.

Do you not have this problem regularly?

No, so far it I haven't encountered it in this form.

Your transaction is a not an uncommon use case, however, and I think Beancount needs to provide a solution somehow.

 

2. ESPP (Employee Stock Purchase Plan) purchases:

Invalid price vs. proceeds/gains: (-3570.24557 USD) vs. (-3213.35 USD)

  2005-01-05 * "ESPP Purchase"

    * Assets:Investments:MutualFunds:Fidelity:MSFT   133.667 MSFT {24.04 USD} @ 26.71000000000000 USD

    * Assets:Investments:MutualFunds:Fidelity:Cash  -3213.35 USD                                     

It appears you have some experience with ESPP transactions,

Yes, I have some of those, but in Canada, so taxation might be different.

 

so how do you represent this (and could you please add it to the Cookbook)?

Good idea, I'll put it in the Cookbook TODO list.

 

Each share is worth $26.71, but we can buy it for $24.04. It's been many years since I've had to deal with this, but it appears that the discount is taxed as regular income at the time of sale, and the cost basis ends up being the market price. I don't see how to record at purchase time what won't be income until a future sale.

Are you sure?

What makes you think that about the taxation of the discount?

The way I accounted for these is that at the time of vesting/purchase, there is a gain recognized for the discount and the cost basis was the FMV on the day of vesting. Then everything proceeds further as expected. This was in Canada, so perhaps there is a difference in taxation, but somehow I doubt the gain on the discount would be recognized at the time of sale only, but I might be wrong. If that is the case, carrying the shares at the discounted price should fix your problem.  In my recollection - this is more than 15 years ago - the taxable discount showed up as taxable income on the Canadian equivalent of the W-2.

In my example below, I booked the discount obtained on the shares at vesting time to a dedicated account (Income:CA:Hooli:ESPP:Discount).  The money I put it myself did not come from cash but from a paycheck, so it is booked that through an income account as well (Income:CA:Hooli:ESPP:Contrib).  The discounted price was $18.5980 and the FMV on that day was $21.8800.

1999-09-30 * "Vest HOOL ESPP - Bought at discount: 18.5980 USD"

  Assets:US:SmithBarney:Hooli:ESPP       54 HOOL {21.8800 USD}

  Income:CA:Hooli:ESPP:Contrib     -1467.84 CAD @ 0.6842 USD

  Income:CA:Hooli:ESPP:Discount     -259.03 CAD @ 0.6842 USD

1999-09-30 * "Sell"

  Assets:US:SmithBarney:Hooli:ESPP      -54 HOOL {21.8800 USD} @ 21.1243 USD

  Assets:US:SmithBarney:Hooli:Cash  1140.71 USD

  Income:CA:Hooli:ESPP:PnL

3. Unknown accounts represented as Equity:UnknownSourceOrSink:

Invalid price vs. proceeds/gains: (15516.59644 USD) vs. ()

  2011-01-20 * "Shares Redeemed"

    * Assets:Investments:MutualFunds:ClipperFund:CFIMX  -245.788 CFIMX {84.13 USD} @ 63.13000000000000 USD

    Equity:UnknownSourceOrSink                             15516.60 USD                                      

    Income:Investments:MutualFunds:ClipperFund:PnL       5162.40 USD                                      

I have some transactions for which I'm missing the corresponding bank statements. I sometimes have a good idea of which account is involved, but I'm not sure. I've been using Equity:UnknownSourceOrSink to represent these so I don't distort my balance sheet. The sellgains plugin specifically looks for Assets accounts. What do you suggest here?

Use an Assets account.

I suppose I can add Equity type to the list of "proceed" account types, I don't think it should hurt anyone.

Would have solve the problem?

Try it, here, add equity to this list:

https://bitbucket.org/blais/beancount/src/4ff00ffbb130e03be08e7194d58e094c71e1f7b7/src/python/beancount/plugins/sellgains.py?at=default#cl-77

and let me know if it works; if so, I can probably add it in.

As always, thanks for your valuable and detailed feedback,

[Martin Blais]

On Sun, May 3, 2015 at 9:50 AM, Daniel Clemente <n14...@gmail.com> wrote:

>
> 1. Basic rounding errors:
>
> Invalid price vs. proceeds/gains: (0.03576 USD) vs. ()
>   2011-01-25 * "Transfer of Assets, 3467.90 USD"
>     * Assets:Investments:RothIRA:Vanguard:VTIVX  250.752 VTIVX {18.35 USD} @ 13.83000000000000 USD
>     * Assets:Investments:RothIRA:DodgeCox:DODGX  -30.892 DODGX {148.93 USD} @ 112.26000000000000 USD
>

I fixed a similar transaction by slightly adjusting the @ or @@ until the warning went away. I don't know about your example, but maybe 13.83 or 112.26 USD aren't exact numbers, maybe they had some decimals like 13.831 USD. I often get reports with 2 decimals while transactions use 3.

I think this is incorrect; this should make no difference.

The effect of using total price (@@ ...) or total cost ({{...}}) is that Beancount will calculate the value for you, possibly resulting in a higher precision for the price or cost number.

But for the purpose of balancing the transaction, the prices (13.83 and 112.26) are ignored.

The "weight" of each posting is used, that is, the cost (250.752 x 18.35 and -30.892 x 148.93).

Also, the imbalance in this case is 0.55 USD, and not 0.03576 USD.

bean-doctor context /home/blais/p/beancount-data/bug-reports/mharris/2015-05-03.precision.beancount 4

Hash:7af5054b554502f3178e99632cc11aaf

Location: /home/blais/p/beancount-data/bug-reports/mharris/2015-05-03.precision.beancount:3

2011-01-25 * "Transfer of Assets, 3467.90 USD"

  * Assets:Investments:RothIRA:Vanguard:VTIVX  250.752 VTIVX {18.35 USD} @ 13.83 USD    ;  4601.30 USD

  * Assets:Investments:RothIRA:DodgeCox:DODGX  -30.892 DODGX {148.93 USD} @ 112.26 USD  ; -4600.75 USD

;;; Residual: (0.55 USD)

 

> One such transaction already has a 2-cent Expenses:RoundingError posting to make everything balance. I suppose I could add a couple
> cents to an Income:RoundingError account too, but I'd much rather that Beancount recognize that 0.001 DODGX is worth 0.14893 USD, so
> a rounding error of 0.03576 USD can be ignored.

Maybe you know it, but…: the way to ignore small amounts is:     option "tolerance"  "0.001"

Actually, this is also not quite correct: this "tolerance" value is only used for Balance assertions and Pad directives.

It does not affect transactions.

The precision for transactions is currently 0.005 <anything> (yes, this is not great, which is why I'm fixing it).

It will actually get deprecated with the new precision mechanism, for which Balance and Pad will infer the precision from the numbers.

(Coming soon.)

But I would say that 0.03576 USD is already too big to go unnoticed. I would use an Expenses:RoundingError account for these cases. Instead of having Expenses:Rounding and Income:Rounding I post everything (+ or -) to the same one.

> that Beancount recognize that 0.001 DODGX is worth 0.14893 USD

Or you could write it explicitly:    0.001 DODGX @ 0.14893 USD

But anyway 0.001 DODGX = 0.11226 USD  from the @ you wrote


These are some vague ideas which may or may not help with your transaction. I don't know about the other 2, sorry.


--
Daniel

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/871tix3h38.wl-n142857%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Martin Blais]

On Sun, May 3, 2015 at 10:59 AM, Martin Blais <bl...@furius.ca> wrote:

On Sat, May 2, 2015 at 6:22 PM, Matthew Harris <mharr...@gmail.com> wrote:

Thanks for adding this. As expected, this reveals a few kinds of errors I don't know how to resolve.

1. Basic rounding errors:

Invalid price vs. proceeds/gains: (0.03576 USD) vs. ()

  2011-01-25 * "Transfer of Assets, 3467.90 USD"

    * Assets:Investments:RothIRA:Vanguard:VTIVX  250.752 VTIVX {18.35 USD} @ 13.83000000000000 USD  

    * Assets:Investments:RothIRA:DodgeCox:DODGX  -30.892 DODGX {148.93 USD} @ 112.26000000000000 USD

One such transaction already has a 2-cent Expenses:RoundingError posting to make everything balance. I suppose I could add a couple cents to an Income:RoundingError account too, but I'd much rather that Beancount recognize that 0.001 DODGX is worth 0.14893 USD, so a rounding error of 0.03576 USD can be ignored.

Matthew, 

Your bug report is _very_ timely!

I've actually been working hard on revamping the mechanism to infer precision over the past weekend and implement the rounding/precision proposal and this example still fails to get handled properly with my new mechanism (though you can set a default of 0.05 USD for all transactions but I assume we don't want that).

I like your suggestion of using the smallest digit of the converted cost in order to locally infer the precision for the cost currency... I hadn't thought of that at all, it's a good idea. The way I carried out inference in the new method, this transaction would not have an inferred value for USD's (precision on cost and price numbers are ignored) and would revert to the default tolerance (which you can override).

I might be able to just add in inference using the method you propose, that is, infer USD precision from max(0.001 x 18.35, 0.001 x 148.93)/2.

Let me see what I can do, I'll be working on this today.

Actually, I need to correct myself on this one: the residual is 0.55364 USD, not 0.03576 USD (because the weight is what is used to compute the residual balance of the transaction, see other email in this thread). So neither of 0.14893 USD nor 0.01835 USD are large enough to account for this discrepancy. This transaction does not balance like this.

I think you must be missing some digits, or they must have made a payment to compensate your for the difference. IMHO it would be appropriate to account for it in a separate posting in this case, due to the magnitude of the difference. What do you think?

It also looks to me like they made a mistake in inferring the cost basis for the conversion:

mandarine:~$ dc -e "6k 250.752 13.83 * 112.26 /   250.752 18.35 * r/ p"

148.949459

vs. the 148.93 USD

[Matthew Harris]

On Sunday, May 3, 2015 at 7:59:16 AM UTC-7, Martin Blais wrote:

On Sat, May 2, 2015 at 6:22 PM, Matthew Harris <mharr...@gmail.com> wrote:

2. ESPP (Employee Stock Purchase Plan) purchases:

 

Each share is worth $26.71, but we can buy it for $24.04. It's been many years since I've had to deal with this, but it appears that the discount is taxed as regular income at the time of sale, and the cost basis ends up being the market price. I don't see how to record at purchase time what won't be income until a future sale.

Are you sure?

What makes you think that about the taxation of the discount?

The first Google hit for [espp taxation] is this TurboTax help page, which says, "When the company buys the shares for you, you do not owe any taxes. … When you sell the stock, the discount that you received when you bought the stock is generally considered additional compensation to you, so you have to pay taxes on it as regular income." It's been many years since I had an ESPP, so I have no memory of how it was taxed.

But really I have no expectation of being able to use Beancount to get my exact W-2 numbers, especially for multiple jobs ago, where I have missing records, so maybe I should just record the discount as income in the ESPP purchase transaction and move on.

 

3. Unknown accounts represented as Equity:UnknownSourceOrSink:

Invalid price vs. proceeds/gains: (15516.59644 USD) vs. ()

  2011-01-20 * "Shares Redeemed"

    * Assets:Investments:MutualFunds:ClipperFund:CFIMX  -245.788 CFIMX {84.13 USD} @ 63.13000000000000 USD

    Equity:UnknownSourceOrSink                             15516.60 USD                                      

    Income:Investments:MutualFunds:ClipperFund:PnL       5162.40 USD                                      

I have some transactions for which I'm missing the corresponding bank statements. I sometimes have a good idea of which account is involved, but I'm not sure. I've been using Equity:UnknownSourceOrSink to represent these so I don't distort my balance sheet. The sellgains plugin specifically looks for Assets accounts. What do you suggest here?

Use an Assets account.

Is that the right answer here? Then I'd have an Assets:Unknown account with random garbage in it, and Assets:* - Liabilities:* would no longer be representative of net worth.

I suppose I can add Equity type to the list of "proceed" account types, I don't think it should hurt anyone.

Would have solve the problem?

Try it, here, add equity to this list:

https://bitbucket.org/blais/beancount/src/4ff00ffbb130e03be08e7194d58e094c71e1f7b7/src/python/beancount/plugins/sellgains.py?at=default#cl-77

and let me know if it works; if so, I can probably add it in.

Adding it did work.

[Matthew Harris]

Heh. This is one of those transactions for which I manually computed the cost basis. It's actually expressed in my Beancount file like this:

2011-01-25 * "Transfer of Assets, 3467.90 USD"

  * Assets:Investments:RothIRA:Vanguard:VTIVX  250.752 VTIVX {{4600.665240000001 USD}} @ 13.83 USD ; [2011-01-26]

  * Assets:Investments:RothIRA:DodgeCox:DODGX  -30.892 DODGX {{4600.665240000001 USD}} @ 112.26 USD

I don't see how I could have done this any better manually.

This gives me an opportunity to mention my most desired feature in Beancount: the {*} feature for automatically computing average cost. (And a less important but still useful feature: posting dates, so I can still use balance assertions in two accounts even when half of a transaction is delayed.)

[Martin Blais]

On Sun, May 3, 2015 at 2:59 PM, Matthew Harris <mharr...@gmail.com> wrote:

On Sunday, May 3, 2015 at 7:59:16 AM UTC-7, Martin Blais wrote:

On Sat, May 2, 2015 at 6:22 PM, Matthew Harris <mharr...@gmail.com> wrote:

2. ESPP (Employee Stock Purchase Plan) purchases:

 

Each share is worth $26.71, but we can buy it for $24.04. It's been many years since I've had to deal with this, but it appears that the discount is taxed as regular income at the time of sale, and the cost basis ends up being the market price. I don't see how to record at purchase time what won't be income until a future sale.

Are you sure?

What makes you think that about the taxation of the discount?

The first Google hit for [espp taxation] is this TurboTax help page, which says, "When the company buys the shares for you, you do not owe any taxes. … When you sell the stock, the discount that you received when you bought the stock is generally considered additional compensation to you, so you have to pay taxes on it as regular income." It's been many years since I had an ESPP, so I have no memory of how it was taxed.

I read this page in detail just now; this looks crazy to me. This would imply that in order to correctly account for this, you have to carry two separate buckets of cost basis because the discount portion is taxed at a different rate (as ordinary income) and the rest as capital gains. So if the stock goes down, what are you supposed to do: Declare the full discount at a gain + a capital loss, or declare a smaller gain, taxed as income? 

More research confirms your finding indeed:

http://personal.fidelity.com/products/stockoptions/faqpurchase.shtml

Q. How am I taxed in my ESPP?

A. With qualified Section 423 employee stock purchase plans, you are not taxed at the time the shares are purchased, only when you sell. Depending on whether the shares were held for the required holding period, a portion of your gain may be taxed as capital gains or as ordinary income.

I'm floored. I don't understand why the government does this. I suppose it's to avoid accruing a tax liability that you may not be able to cover (I guess they assume people will spend everything they earn, which is possibly true for a lot of people). I haven't booked anything like this so far.

I can imagine that you could use the mirror accounting method to track the discount portion of the capital gain with separate units. Then, you could write a script or a plugin that makes some adjustments to your transactions automatically, e.g. that distributes the capital gains into two buckets when you sell the stock and automates this allocation as much as possible.

But it's not a straightforward thing supported in Beancount to have separate buckets for cost basis.

But really I have no expectation of being able to use Beancount to get my exact W-2 numbers, especially for multiple jobs ago, where I have missing records, so maybe I should just record the discount as income in the ESPP purchase transaction and move on.

That's what I would do (though it's incorrect).

 

3. Unknown accounts represented as Equity:UnknownSourceOrSink:

Invalid price vs. proceeds/gains: (15516.59644 USD) vs. ()

  2011-01-20 * "Shares Redeemed"

    * Assets:Investments:MutualFunds:ClipperFund:CFIMX  -245.788 CFIMX {84.13 USD} @ 63.13000000000000 USD

    Equity:UnknownSourceOrSink                             15516.60 USD                                      

    Income:Investments:MutualFunds:ClipperFund:PnL       5162.40 USD                                      

I have some transactions for which I'm missing the corresponding bank statements. I sometimes have a good idea of which account is involved, but I'm not sure. I've been using Equity:UnknownSourceOrSink to represent these so I don't distort my balance sheet. The sellgains plugin specifically looks for Assets accounts. What do you suggest here?

Use an Assets account.

Is that the right answer here? Then I'd have an Assets:Unknown account with random garbage in it, and Assets:* - Liabilities:* would no longer be representative of net worth.

No I would use the account you think you used at the time, and pad it with an appropriate contribution from equity; it's just one more level of indirection for that bit of money. That's what I've done to record my very distant past (I have stock transactions but not checking account records). It should result in the same balance sheet totals (you've got assertions anyhow, I presume, so you shouldn't have to worry).

I suppose I can add Equity type to the list of "proceed" account types, I don't think it should hurt anyone.

Would have solve the problem?

Try it, here, add equity to this list:

https://bitbucket.org/blais/beancount/src/4ff00ffbb130e03be08e7194d58e094c71e1f7b7/src/python/beancount/plugins/sellgains.py?at=default#cl-77

and let me know if it works; if so, I can probably add it in.

Adding it did work.

Awesome.

I'll add it then.

 

[Martin Blais]

Just more digits, e.g. this balances:

2011-01-25 * "Transfer of Assets, 3467.90 USD"

  * Assets:Investments:RothIRA:Vanguard:VTIVX  250.752 VTIVX {18.34747 USD} @ 13.83 USD ; [2011-01-26]

  * Assets:Investments:RothIRA:DodgeCox:DODGX  -30.892 DODGX {148.92740 USD} @ 112.26 USD

 

This gives me an opportunity to mention my most desired feature in Beancount: the {*} feature for automatically computing average cost. (And a less important but still useful feature: posting dates, so I can still use balance assertions in two accounts even when half of a transaction is delayed.)

Yes! I know, it's always in the back of my mind actually.

It's a lot of work, sorry it's taking so long.

[Martin Blais]

I suppose I can add Equity type to the list of "proceed" account types, I don't think it should hurt anyone.

Would have solve the problem?

Try it, here, add equity to this list:

https://bitbucket.org/blais/beancount/src/4ff00ffbb130e03be08e7194d58e094c71e1f7b7/src/python/beancount/plugins/sellgains.py?at=default#cl-77

and let me know if it works; if so, I can probably add it in.

Adding it did work.

Awesome.

I'll add it then.

 

 https://bitbucket.org/blais/beancount/commits/2af67236eb0002ec831d283142294eb4ba060ca2

[Martin Blais]

On Sun, May 3, 2015 at 11:15 AM, Martin Blais <bl...@furius.ca> wrote:

It will actually get deprecated with the new precision mechanism, for which Balance and Pad will infer the precision from the numbers.

(Coming soon.)

I'm not going to be finished today.

Quantizing the auto-posting numbers based on inference is hard, it turns out.

I'll need another week or two, really thought I'd be done by today (docs written and all).

 

[Martin Blais]

On Sun, May 3, 2015 at 10:59 AM, Martin Blais <bl...@furius.ca> wrote:

On Sat, May 2, 2015 at 6:22 PM, Matthew Harris <mharr...@gmail.com> wrote:

Thanks for adding this. As expected, this reveals a few kinds of errors I don't know how to resolve.

1. Basic rounding errors:

Invalid price vs. proceeds/gains: (0.03576 USD) vs. ()

  2011-01-25 * "Transfer of Assets, 3467.90 USD"

    * Assets:Investments:RothIRA:Vanguard:VTIVX  250.752 VTIVX {18.35 USD} @ 13.83000000000000 USD  

    * Assets:Investments:RothIRA:DodgeCox:DODGX  -30.892 DODGX {148.93 USD} @ 112.26000000000000 USD

One such transaction already has a 2-cent Expenses:RoundingError posting to make everything balance. I suppose I could add a couple cents to an Income:RoundingError account too, but I'd much rather that Beancount recognize that 0.001 DODGX is worth 0.14893 USD, so a rounding error of 0.03576 USD can be ignored.

Matthew, 

Your bug report is _very_ timely!

I've actually been working hard on revamping the mechanism to infer precision over the past weekend and implement the rounding/precision proposal and this example still fails to get handled properly with my new mechanism (though you can set a default of 0.05 USD for all transactions but I assume we don't want that).

I like your suggestion of using the smallest digit of the converted cost in order to locally infer the precision for the cost currency... I hadn't thought of that at all, it's a good idea. The way I carried out inference in the new method, this transaction would not have an inferred value for USD's (precision on cost and price numbers are ignored) and would revert to the default tolerance (which you can override). 

I might be able to just add in inference using the method you propose, that is, infer USD precision from max(0.001 x 18.35, 0.001 x 148.93)/2.

Let me see what I can do, I'll be working on this today.

So, after half a day of test this out I'm concluding that using the cost and the last digit of the stock is not really workable, it's problematic in more than one way. It results in many odd cases whereby the inferred tolerance will actually be smaller than the default precision, and then it's not clear which precision should get used (smaller than default tolerances should be able to exist when the user provides numbers that have this number of digits... so then the meaning would become "use the numbers in priority, or the numbers inferred from the cost, but only if they're larger than the default tolerance"). It makes the precision used non-trivial to understand. I want to keep things simple.

I think just using a higher precision on the cost numbers will resolve the problem you're describing.

I've kept the code around and might play with this some more in the future, but for now I'm aborting this foray, as I want to complete the precision work.