Why I don’t want to use auxiliary dates for bank reconciliation … or anything else

121 views
Skip to first unread message

Peter McArthur

unread,
May 30, 2017, 10:39:29 PM5/30/17
to Ledger

First I should describe my philosophy.

We’re all computer nerds here, I think, so we all know the value of simple, flexible abstractions with strong mathematical underpinnings. Relational databases, the lambda calculus, finite automata, context free grammars and so on. I believe double-entry book keeping deserves a place in that last.


I shan’t go into this in any depth, but I don’t think many people – even professional accountants – appreciate the mathematical brilliance that underlies the double-entry system. But it was this brilliant idea that turned all our ad hoc book keeping systems into a coherent, flexible whole, with lots of side benefits I won’t go into here.


So, just as we have SQL for talking about relational databases, or regular expressions for talking about finite automata, I see Ledger as a language for talking about double-entry book keeping, and in my view that is a very good thing.

This topic arose in the context of bank reconciliation, so, let’s talk about bank reconciliation. The traditional, pen-and-paper way of doing it is:

  1. Go through your accounts and your bank statement, matching them item by item, and checking off matched items.
  2. At the end of this process, there will probably be a few unmatched items in your accounts, because they haven’t cleared yet. Make a note of these in a Reconciliation Report, and check that they are all reconciled next month.

The advantage of this method is that it is quick and efficient. That mattered a lot in the era of pen-and-paper accounting, but now it doesn’t, so I think we should consider its disadvantages too:

  1. It’s hard to detect and correct errors.
  2. There is no record of which item in your accounts matched which item in your bank statement, so if you ever need to check again your work is partially wasted.
  3. The Reconciliation Report is a new kind of artefact that exists outside your core accounting system.

All of the above can be solved if we split the bank account into two sub accounts: one for uncleared transactions and one for cleared transactions. It’s the double-entry way of doing it. They didn’t do this in the era of pen-and-paper accounting because it wasn’t worth the extra effort, but now it just seems the obvious and natural thing to do.


“So why don’t professional accountants do it?” I hear you ask. That’s a good question. I went and asked that very question on an accountant’s forum (which I was already a member of), and I got five answers. They were:

1, 2 & 3. But that’s more complicated the way we do it now. [I disagree!]
3. That would make life harder if the computers stopped working and we had to go back to pen-and-paper. [Fair point.]
5. Our accounting software already does things that way, and it works great!


I believe that accountants could start using the new system now, if they wanted, but most are reluctant to move on from a system they’ve invested so much in learning. Pretty much how my wife felt when I persuaded her to switch to a Mac. ;-)

So, now, on to Ledger’s way of doing things. I said earlier I disliked the * tag. That’s not really true. I have no problem with metadata tags without special semantics. But I REALLY dislike auxiliary dates. They are not a core concept of double-entry; they have unclear semantics; and they violate some important design heuristics without good reason, namely:

  • Zero, one, infinity: An entity should be forbidden, one should be allowed, or any number should be allowed. [Definitely not two!]
  • Minimalism: A designer knows he has achieved perfection when there is nothing left to take away. [I do not think auxiliary dates are ever necessary.]
The manual shows us how unclear the semantics are. Most people seem to use auxiliary dates as a date when a cheque is cleared. But the manual also shows them being used when a cheque is paid:


  2008/01/01=2008/01/15 Client invoice

    Assets:Accounts Receivable            $100.00

    Income: Client name


or when it is not paid:


2008/10/16 * (2090) Bountiful Blessings Farm

    Expenses:Food:Groceries                  $ 37.50  ; [=2008/10/01]

    Expenses:Food:Groceries                  $ 37.50  ; [=2008/11/01]

    Expenses:Food:Groceries                  $ 37.50  ; [=2008/12/01]

    Expenses:Food:Groceries                  $ 37.50  ; [=2009/01/01]

    Expenses:Food:Groceries                  $ 37.50  ; [=2009/02/01]

    Expenses:Food:Groceries                  $ 37.50  ; [=2009/03/01]

    Assets:Checking


If you included all three of these in your accounts, you’d end up in a terrible mess. And in any case, a trained accountant would look at the two examples above and say, “No, you’re doing it wrong! You should …” and then describe a method that doesn’t use auxiliary dates at all. (I’ll leave the details as an exercise for the reader.)

To be clear, I’ve nothing against adding extra dates to accounts per se. But I dislike the privileged status given to auxiliary dates. I would much rather see Ledger improve its support for typed tag values (e.g. cleared:: [05/30]) so that auxiliary dates could be downgraded to a special syntax. E.g. [=05/30] might become a synonym for AuxiliaryDate:: [05/30].

o1bigtenor

unread,
May 31, 2017, 7:20:16 AM5/31/17
to ledge...@googlegroups.com

--

Greetings

Didn't trim the foregoing as I didn't know what to cut to still keep all the important parts alive and nothing else.

Something like this 'typed tag value' would be very valuable for invoicing in a small business.
There possibly also could be a use for this in inventory control (which makes me wonder how I could use Ledger for just that!).

Maybe Ledger 4.0 (or 3.5 (or whatever)) could do this for us small business owners who want that plain text archive yet need tools to work on the informaiton in that text file!!. (Ledger does a good job of what its doing so far!!

Dee

John Wiegley

unread,
May 31, 2017, 1:58:10 PM5/31/17
to Peter McArthur, Ledger
>>>>> "PM" == Peter McArthur <peter.m...@gmail.com> writes:

PM> The manual shows us how unclear the semantics are. Most people seem to use
PM> auxiliary dates as a date when a cheque is *cleared*. But the manual also
PM> shows them being used when a cheque is *paid*:

It should be noted here that one of the "features" of Ledger is that it leaves
as much of the semantics of your accounting as possible up to you. This is a
reason why beancount was created: To choose the best notion of what things
mean in a financial setting, and then to impose those semantics to help you
avoid errors.

C++Ledger, however, is just a glorified calculator. It doesn't tell you what
your inputs or your outputs should mean.

--
John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2

o1bigtenor

unread,
May 31, 2017, 6:09:37 PM5/31/17
to ledge...@googlegroups.com, Peter McArthur
On Wed, May 31, 2017 at 12:59 PM, John Wiegley <jo...@newartisans.com> wrote:
>>>>> "PM" == Peter McArthur <peter.m...@gmail.com> writes:

PM> The manual shows us how unclear the semantics are. Most people seem to use
PM> auxiliary dates as a date when a cheque is *cleared*. But the manual also
PM> shows them being used when a cheque is *paid*:

It should be noted here that one of the "features" of Ledger is that it leaves
as much of the semantics of your accounting as possible up to you.  This is a
reason why beancount was created: To choose the best notion of what things
mean in a financial setting, and then to impose those semantics to help you
avoid errors.

C++Ledger, however, is just a glorified calculator. It doesn't tell you what
your inputs or your outputs should mean.

So - -- from your perspective you don't see a value in having something defined 
that would help with the date paid, date cleared, date issued (more for invoicing) 
as a specific 'data type' available for use?

As a small business owner that really doesn't want to buy the rights to the use 
(sort of) of some Win program that isn't really that responsive (to my needs at 
least) I would like to see something that would enable me to run some kind of a 
report. This report would then be able to tell me which invoices to be paid or those 
due me that are outstanding (possibly with a way to make aging easy). 

I understand that you see C++Ledger as a 'simple' tool - - - I think it is a tool 
that appears simple but has the framework to do even more. (To use the analogy 
of a calculator it looks like perhaps just a 4 function machine and then when you 
dig a little deeper you see the scientific functions and then deeper yet you find 
the statistical functions and the vector/polar calculations and and and. This is 
how I see ledger - - - its a tool with little showing on the surface but there is a 
vast world below!)

Dee

Peter McArthur

unread,
May 31, 2017, 6:10:58 PM5/31/17
to Ledger, peter.m...@gmail.com, jo...@newartisans.com


On Wednesday, 31 May 2017 18:58:10 UTC+1, John Wiegley wrote:
>>>>> "PM" == Peter McArthur <peter.m...@gmail.com> writes:

PM> The manual shows us how unclear the semantics are. Most people seem to use
PM> auxiliary dates as a date when a cheque is *cleared*. But the manual also
PM> shows them being used when a cheque is *paid*:

It should be noted here that one of the "features" of Ledger is that it leaves
as much of the semantics of your accounting as possible up to you.

Yes, the semantics are up to you, but you can only have one kind of auxiliary date. That’s quite a restriction. If you’ve used auxiliary dates to spread your payment to Bountiful Blessing Farm over time, as the manual suggest, you can’t also use them to mark when a cheque cleared.

Compare that to tags. While there are two special tags called * and !, there's nothing they can do that other tags can't, as far as I'm aware. For instance, this:

  2017-05-20 McDonald’s
    food and drink     £ -1.99
    * current account  £  1.99

  $ ledger -f journal balance --cleared
  17-May-20 McDonald’s            current account              £ 1.99       £ 1.99
  $ ledger -f journal balance --uncleared
  17-May-20 McDonald’s            food and drink              £ -1.99      £ -1.99

Is pretty much the same as this:

  2017-05-20 McDonald’s
    food and drink   £ -1.99
    current account  £  1.99  ; :cleared:

  $ ledger -f journal balance %/cleared/
  17-May-20 McDonald’s            current account              £ 1.99       £ 1.99
  $ ledger -f journal balance 'not(%/cleared/)'
  17-May-20 McDonald’s            food and drink              £ -1.99      £ -1.99

I would like to similar freedom with dates.

This is a  reason why beancount was created: To choose the best notion of what things mean in a financial setting, and then to impose those semantics to help you  avoid errors.

I'm not experienced with beancount but, so I may have misunderstood you, but I don't really see what you're getting at. Power and flexibility are good. The ability to add constraints is also good. They work best when used together.

John Wiegley

unread,
May 31, 2017, 6:32:44 PM5/31/17
to o1bigtenor, ledge...@googlegroups.com, Peter McArthur
>>>>> "o" == o1bigtenor <o1big...@gmail.com> writes:

o> So - -- from your perspective you don't see a value in having something
o> defined that would help with the date paid, date cleared, date issued (more
o> for invoicing) as a specific 'data type' available for use?

Not as part of the internals of Ledger, no. As something in the form of
external scripting, such as by using the Python support, sure.

o> I understand that you see C++Ledger as a 'simple' tool - - - I think it is
o> a tool that appears simple but has the framework to do even more. (To use
o> the analogy of a calculator it looks like perhaps just a 4 function machine
o> and then when you dig a little deeper you see the scientific functions and
o> then deeper yet you find the statistical functions and the vector/polar
o> calculations and and and. This is how I see ledger - - - its a tool with
o> little showing on the surface but there is a vast world below!)

You might want to give beancount a look, since it has much more of an
accounting focus.

John Wiegley

unread,
May 31, 2017, 6:33:50 PM5/31/17
to Peter McArthur, Ledger
>>>>> "PM" == Peter McArthur <peter.m...@gmail.com> writes:

PM> I'm not experienced with beancount but, so I may have misunderstood you,
PM> but I don't really see what you're getting at. Power and flexibility are
PM> good. The ability to add constraints is also good. They work best when
PM> used together.

I want you to add the constraint, not me. If there's a constraint you want to
encode but can't, I'd want to find a way to generalize what you're trying to
accomplish.

Brian Exelbierd

unread,
May 31, 2017, 9:17:19 PM5/31/17
to Peter McArthur, Ledger



On Wed, May 31, 2017, at 04:39 AM, Peter McArthur wrote:

This topic arose in the context of bank reconciliation, so, let’s talk about bank reconciliation. The traditional, pen-and-paper way of doing it is:

  1. Go through your accounts and your bank statement, matching them item by item, and checking off matched items.
  2. At the end of this process, there will probably be a few unmatched items in your accounts, because they haven’t cleared yet. Make a note of these in a Reconciliation Report, and check that they are all reconciled next month.

The advantage of this method is that it is quick and efficient. That mattered a lot in the era of pen-and-paper accounting, but now it doesn’t, so I think we should consider its disadvantages too:

  1. It’s hard to detect and correct errors.
  2. There is no record of which item in your accounts matched which item in your bank statement, so if you ever need to check again your work is partially wasted.
  3. The Reconciliation Report is a new kind of artefact that exists outside your core accounting system.

All of the above can be solved if we split the bank account into two sub accounts: one for uncleared transactions and one for cleared transactions. It’s the double-entry way of doing it. They didn’t do this in the era of pen-and-paper accounting because it wasn’t worth the extra effort, but now it just seems the obvious and natural thing to do.


I was very as very interested in this idea, however based on your example from the other thread I don't understand how you've changed the situation.  

Your example, aiui:

05/25 McDonald’s
  expenses:food and drink:fast food  £ -0.99
  assets:current account:uncleared   £  0.99
  assets:current account:uncleared  £ -0.99  ; [05/27]
  assets:current account:cleared    £  0.99  ; [05/27]

With regard to the problems raised above:

1. This seems to make it no easier to detect and correct errors than a tick mark on your bank statement. To add the final lines you still have to go through the statement line by line. 

2. There still seems to be no record tieing your entry to the line item on your bank statement. Had you added a comment containing the exact details from your statement then you could match. This could also be accomplished by just numbering the entries on your bank statement and then adding a cleared line number comment.  

3. Agreed as solved if a suitable parser is written.

It seems your syntax could be expanded to solve number 2 by encoding it as something like:

05/25 McDonald’s
  expenses:food and drink:fast food  £ -0.99
  assets:current account:XYZ   £  0.99

05/27 McD Flughafen Wien
  assets:current account:XYZ  £ -0.99
  assets:current account:    £  0.99

This presumes you can load the transactions from your statement. You would have to tie the two entries together via the XYZ subaccount where XYZ is a unique transaction identifier. 

A suitable parser could then generate a reconciliation report.

However, this seems to have no significant value for error detection and correction (item #1) over the existing process you've described or the process of making tick marks on statements. 

I suspect I've missed something in your explanation and look forward to your response. 

Regards,

bex 

Peter McArthur

unread,
Jun 1, 2017, 12:17:17 AM6/1/17
to Ledger, peter.m...@gmail.com
1. This seems to make it no easier to detect and correct errors than a tick mark on your bank statement. To add the final lines you still have to go through the statement line by line.

2. There still seems to be no record tieing your entry to the line item on your bank statement. Had you added a comment containing the exact details from your statement then you could match. This could also be accomplished by just numbering the entries on your bank statement and then adding a cleared line number comment.  

3. Agreed as solved if a suitable parser is written.

I haven’t tested this, but I believe Ledger offers a simpler way to do it with the Payee tag. Payee is a special metadata tag which allows you to modify the payee field for individual posts within a transaction. So if this is my bank statement:

Date            Transaction Type      Description          Money out  Money in  Balance
...             ...                   ...                        ...       ...      ...
27th May 2017   Contactless Payment   MCDONALDS WIEN DE        £0.99            £173.48

Here’s what I would put in my ledger file:

05/25 McDonald’s
  expenses:food and drink:fast food  £ -0.99
  assets:current account:uncleared   £  0.99
  ;
  assets:current account:uncleared  £ -0.99  ; [05/27]  Payee: MCDONALDS WIEN DE
  assets:current account:cleared    £  0.99  ; [05/27]  Payee: MCDONALDS WIEN DE

All four posts are now linked by being part of the same transaction, without any need for transaction codes.

By the way, I’m uncomfortable with the way you used an :XYZ subaccount. Is it not possible to do the same thing with tags?

Brian Exelbierd

unread,
Jun 1, 2017, 1:57:44 AM6/1/17
to ledge...@googlegroups.com
Certainly, but early this morning this isn't what first popped into my mind :).

My concern remains with #1.  I feel like we've added a lot of work (assuming no automation) without solving the problem.

regards,

bex

Peter McArthur

unread,
Jun 1, 2017, 2:43:04 AM6/1/17
to Ledger
I feel like we've added a lot of work (assuming no automation)

Agreed. Without automation, it’s probably not worth the effort. In fact, this whole discussion started with me asking for help with the automation, and others asking why I would want to do such a thing.
 
without solving the problem.

If it doesn’t solve any problem you have, and if it makes life harder for you in some way, then of course you shouldn’t do it.

If it doesn’t solve any problem you have, but it doesn’t create any work, then I would suggest that my way is better, because it hews closer to the double-entry model. The closer you stay to that model, the more use you can get from Ledger’s general purpose tools for handling double-entry accounts.
Reply all
Reply to author
Forward
0 new messages