Per posting effective dates plugin

[Red S]

I find being able to specify different dates for different legs (aka postings) of a transaction to be valuable. I was using a plugin I'd hacked up for a long while, and I finally got around to rewriting it to allow for per-posting dates with configurable holding accounts. Sharing it. Feedback welcome:

https://github.com/redstreet/beancount_plugins_redstreet/tree/master/effective_date

Cases where I use it:

- booking a tax refund to the correct year

- booking the last paycheck of the year to the correct year (when it gets deposited a day later in the next year)

- spread a lumpsum expense across time (eg: booking insurance premiums paid upfront for one year across several months)

- booking expenses to the correct month (eg: rent paid a couple days before the start of a month)

etc.

The plugin also inserts a beancount link to link the original transaction and all its associated newly inserted transactions to help you find them easily in beancount or fava.

Here's the README:

Effective dates plugin for Beancount

Double entry bookkeeping requires each transaction to occur instantaneously in time. In Beancount, that translates to each transaction occuring on a single date. However, it is occasionally useful to view different legs of a transaction occuring across difference periods of time. For example, consider:

2014-12-15 * "Annual Insurance payment for 2015"
    Liabilities:Credit-Card   100 USD
    Expenses:Insurance

Here, the payment was made in Dec 2014 for an expense to be booked to 2015. To reflect this, one could book it thus:

2014-12-15 * "Annual Insurance payment for 2015"
    Liabilities:Credit-Card   100 USD
    Assets:Hold:Insurance

2015-01-01 * "Annual Insurance payment for 2015"
    Assets:Hold:Insurance  -100 USD
    Expenses:Insurance

The expense is booked in 2015, while the credit card transaction, in 2014. The "Assets:Hold:Insurance" holds the money for the period in between.

This plugin automates the process above. One can simply enter a single transaction with an 'effective_date' metadata field for the posting (not the transaction) that needs to occur later (or earlier):

2014-12-15 * "Annual Insurance payment for 2015"
    Liabilities:Credit-Card   100 USD
    Expenses:Insurance
      effective_date: 2015-01-01

The plugin also allows for legs to occur on multiple different dates. For example:

2015-02-01 * "Car insurance: 3 months"
 Liabilities:Mastercard    -600 USD
 Expenses:Car:Insurance     200 USD
   effective_date: 2015-03-01
 Expenses:Car:Insurance     200 USD
   effective_date: 2015-04-01
 Expenses:Car:Insurance     200 USD
   effective_date: 2015-05-01

The examples.bc shows you how the plugin can be configured for your choice of holding accounts.

[Justus Pendleton]

On Sunday, March 1, 2020 at 4:09:52 AM UTC+7, Red S wrote:

I find being able to specify different dates for different legs (aka postings) of a transaction to be valuable.

Very cool. I have (relatively) frequent need for this as well. There are, unfortunately, all kinds of financial transfers that don't occur instantly. In some cases I can live with the date on one leg being wrong but in others I prefer more accuracy, especially when there are exchange rates or other external factors involved. I've been manually doing this for years and will be glad to switch over to a plugin to make it slightly more obvious what is happening.

A few quick comments based on scanning the plugin:

For the LINK_FORMAT you might consider adding the date of the "real" posting as part of it. e.g. something like edate-20191225-xkjm. I do that in mine and find that having a tiny bit of human understandable context often helps and prevents me needing to do more complicated querying/digging. "Oh, it is probably that wire transfer, I think I did that in late December".

One thing I've struggled with is a nice way to do balance statements. You want some assurance that no money got left behind in limbo in some fake account I never look at. Of course, that's a bigger problem when it is all done manually. But it is hard to autogenerate balance statements when there could be multiple transactions in-flight using the holding account. e.g. you can't just make a balance statement after the final edate to assert $0. How do you handle balance statements for this Hold account in your own usage?

[Red S]

Thanks for the feedback! Responses inline:

On Sunday, March 1, 2020 at 7:13:58 AM UTC-8, Justus Pendleton wrote:

On Sunday, March 1, 2020 at 4:09:52 AM UTC+7, Red S wrote:

I find being able to specify different dates for different legs (aka postings) of a transaction to be valuable.

Very cool. I have (relatively) frequent need for this as well. There are, unfortunately, all kinds of financial transfers that don't occur instantly. In some cases I can live with the date on one leg being wrong but in others I prefer more accuracy, especially when there are exchange rates or other external factors involved. I've been manually doing this for years and will be glad to switch over to a plugin to make it slightly more obvious what is happening.

Good use cases I hadn't thought of. I use my zerosum plugin for some of those use cases,  but depending upon one's preference to dedup their ingest, the effective_date plugin solves the same problem in a different way.

A few quick comments based on scanning the plugin:

For the LINK_FORMAT you might consider adding the date of the "real" posting as part of it. e.g. something like edate-20191225-xkjm. I do that in mine and find that having a tiny bit of human understandable context often helps and prevents me needing to do more complicated querying/digging. "Oh, it is probably that wire transfer, I think I did that in late December".

I'd started out with exactly that format, but didn't like the length of the links. Instead, the plugin add the "original_date" metadata to the newly created transaction. In conjunction with the fact that the payee/narration remains the same, it provides enough human readable context for me. However, your feedback is now making me reconsider pushing the date into the LINK_FORMAT and removing original_date completely. That's simpler.

 

One thing I've struggled with is a nice way to do balance statements. You want some assurance that no money got left behind in limbo in some fake account I never look at. Of course, that's a bigger problem when it is all done manually. But it is hard to autogenerate balance statements when there could be multiple transactions in-flight using the holding account. e.g. you can't just make a balance statement after the final edate to assert $0. How do you handle balance statements for this Hold account in your own usage?

I personally don't find that to be a problem with effective_date, because the holding account is used solely by the plugin and is always eventually zero barring bugs. So the only use I'd see is to uncover bugs in the plugin-code. Are you seeing other use cases?

However, I do run exactly into the problem you mention when using the zerosum plugin. There, a non-zero value may indicate incorrect ingest or worse, a banking error. To catch these, I have my zerosum accounts tree auto-expanded visually (see this patch in fava) whenever it is non-zero.

[Red S]

Updated. I kept the original_date metadata which is useful in queries. I contracted the link (6dig date + 3char rand_string). Thanks for the suggestion!

[Red S]

To clarify: the links now look like: ^edate-141215-xlu, to refer to an original transaction date of 2014-12-15.

[Justus Pendleton]

On Monday, March 2, 2020 at 6:57:23 AM UTC+7, Red S wrote:

Good use cases I hadn't thought of. I use my zerosum plugin for some of those use cases,  but depending upon one's preference to dedup their ingest, the effective_date plugin solves the same problem in a different way.

All of my transactions are manually entered since cash payments make up >50% my transactions and my primary bank account & credit card don't have any kind of export, so zerosum doesn't fit my use case, unfortunately. But I can how it would work with importers. effective_date looks like exactly what I want for my workflow, so I'll be using it from now on.

I personally don't find that to be a problem with effective_date, because the holding account is used solely by the plugin and is always eventually zero barring bugs. So the only use I'd see is to uncover bugs in the plugin-code. Are you seeing other use cases?

Probably I'm just worrying/OCD about nothing. My hold account previously had manual transactions in it but going forward I suppose it won't anymore, so the need for balance statements to catch errors is reduced.

[Red S]

All of my transactions are manually entered since cash payments make up >50% my transactions and my primary bank account & credit card don't have any kind of export, so zerosum doesn't fit my use case, unfortunately. But I can how it would work with importers. effective_date looks like exactly what I want for my workflow, so I'll be using it from now on.

Makes sense, and happy to hear this is useful. I'm open to suggestions and PRs, feel free to open issues on github or post here.

[Chad Michael Eyer]

Whenever I try to use this plugin, I receive this console error:

  File "/usr/local/lib/python3.7/site-packages/beancount_reds_plugins/effective_date/effective_date.py", line 118, in effective_date

    holding_account = holding_accts[found_acct]['earlier']

KeyError: ''

Anything I should be looking for?

[Booster Hidrogen]

How do you run this plugin ?

[Red S]

That's what I'd ask too. Could you please post a minimal example that fails, including the config with which you're invoking the plugin? Also, have you taken a look at the example?

https://github.com/redstreet/beancount_reds_plugins/blob/master/beancount_reds_plugins/effective_date/examples.beancount

It'll fail that way if you're calling it for an account for which you haven't supplied a corresponding config.

[Chad Michael Eyer]

Thank you. That was the issue. I needed to declare an earlier/later pair 
for the Asset account.

I appreciate your help.

[Booster Hidrogen]

Sir, could share step by step to use this plugin ? thanks.

[Red S]

See installation and usage:
https://github.com/redstreet/beancount_reds_plugins

https://github.com/redstreet/beancount_reds_plugins/blob/master/beancount_reds_plugins/effective_date/examples.beancount

Please post in this thread if that doesn't work for you.

[Bman Q]

Red, hi

Useful plugin!

Did you thought about simplifying/automating syntax for " spread a lumpsum expense across time" function of plugin?

Now it's

# realizing expenses later, split across multiple dates

2015-02-01 * "Car insurance: 3 months"
 Liabilities:Mastercard    -600 USD
 Expenses:Car:Insurance     200 USD
   effective_date: 2015-03-01
 Expenses:Car:Insurance     200 USD
   effective_date: 2015-04-01
 Expenses:Car:Insurance     200 USD
   effective_date: 2015-05-01

Which works for 3-4 month splits, but i believe most frequent usage is 12 months equal split, then it becomes a bit troublesome.

Do you think it will be possible to do something like this:

2015-02-01 * "Car insurance: 12 months"
 Liabilities:Mastercard    -2400 USD

   split_parts: 12

   split_interval: m                        #(m-month, y-year, or exact number, like 30, cause some providers count m = 30 days)

   effective_date: 2015-03-01    #(or start_date)

And it will be equivalent to:

# realizing expenses later, split across multiple dates
2015-02-01 * "Car insurance: 12 months"
 Liabilities:Mastercard    -2400 USD

 Expenses:Car:Insurance     200 USD
   effective_date: 2015-03-01
 Expenses:Car:Insurance     200 USD
   effective_date: 2015-04-01

...............

...............

 Expenses:Car:Insurance     200 USD

   effective_date: 2016-02-01

If amount is not divided by split_parts exactly then add 1 cent weight to last one(s), example

# realizing expenses later, split across multiple dates
2015-02-01 * "Car insurance: 12 months"
 Liabilities:Mastercard    -2401 USD
 Expenses:Car:Insurance     200.08 USD
   effective_date: 2015-03-01
 Expenses:Car:Insurance     200.08 USD
   effective_date: 2015-04-01
...............
...............

  Expenses:Car:Insurance     200.09 USD
   effective_date: 2015-11-01

...............

...............

Expenses:Car:Insurance     200.09 USD
   effective_date: 2016-02-01

[Martin Blais]

In v3 you'll be able to insert a date in front of a posting and have the difference automatically inserted & removed to/from a transfer account.

For the equal split case, it should be pretty straightforward to build a plugin to do just what you describe below.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/2e4d1c27-a31d-4a7e-a923-a71bdedad256n%40googlegroups.com.

[redst...@gmail.com]

> Did you thought about simplifying/automating syntax for " spread a lumpsum expense across time" function of plugin?

Yes, I've definitely wanted this for a long while, but just haven't had the time yet. Happy to accept a well designed patch :). The part of the plugin that does this (spread across a time period) would still be useful in v3, even though this plugin's current basic functionality would not be necessary in v3.