Gnucash "close books" functionality, remove transactions when import to beancount?

[Oon-Ee Ng]

With Gnucash, one of the things I did every calendar year end (private
accounts) was to 'close books'. In summary, what it does is create two
transactions, the objective of which is to balance all
incomes/expenses to zero. The period of time covered is normally a
year, but configurable.

Quick example, initial stage:-

Expenses:Food 200 USD
Expenses:Drinks 100 USD
Income:Salary -400 USD

The 'close book' function would then create two transactions which
would be (in beancount syntax):-

2014-12-31 *
Expenses:Food -200 USD
Expenses:Drink -100 USD
Equity:CummulativeExpenses 300 USD
2014-12-31 *
Income:Salary 400 USD
Equity:RetainedEarnings 400 USD

The destination accounts would be configurable, I can't remember where
I got their names from really.

I found it necessary to do this every year so that within the year I
could have a clean view of how much I had spent just in this calendar
year in the account summary (without having to run reports). Of course
I now realize I should have used the internal view filter in gnucash,
but...

Anyway the primary question I guess is whether such 'close book'
transactions make sense, or whether I should get rid of them entirely.

[Martin Blais]

In Beancount this procedure is something that is done automatically at reporting time by the software. We call the operation "clear" and there's no particular reason to stick with the antiquated method of you having to do this explicitly yourself. We have computers; the software can do that for you.

Well, perhaps performance could be one reason, but Beancount is easily fast enough for 10-15 years of a normal person's data, and the way forward is to do some performance optimizations instead of forcing the user through this antiquated practice of having to close on some arbitrary time boundary. I know H/Ledger users are fond of following old accounting practices for some sort of romantic attachment, but I question the status quo, and so far I see no rationale to force you to have to go through this rigamarole just because people did it this way 20 years ago and commercial software cathers to people used to that. Just keep your entire dataset in one place and process it every time. It gives you the freedom to go back to any point in time and draw a balance sheet or income statement for any arbitrary period. bean-web by default provides year-boundary subsets, just like you need it. This is an artifact of reporting.

If that doesn't work out well for some reason, please let us know (and please be very specific if you do so, I've thought about this a lot).

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAGQ70esYPA%3Dm-KiSq2DoHNSy%3DMDHx-FELBpZqKEJDQWq%3Dsy8TQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Oon-Ee Ng]

Thanks Martin, that sort of confirms my thoughts on the matter. Most
likely because my own background is engineering and computer science
rather than accounting, though.

Regarding performance, perhaps there can be a convention regarding
splitting input files by year. So I'd have one per year, and two
'main' files, one of which includes every year, and one of which
includes only this year (maybe plus the previous 1 or 2 years if
necessary).

>> email to beancount+...@googlegroups.com.

>> To post to this group, send email to bean...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/beancount/CAGQ70esYPA%3Dm-KiSq2DoHNSy%3DMDHx-FELBpZqKEJDQWq%3Dsy8TQ%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to beancount+...@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/beancount/CAK21%2BhOc3S_Y6dTGvbxvr8LEGYhN%3DDVrGcZjg8rAfQ8o_ZSWOg%40mail.gmail.com.

[Martin Blais]

On Sat, Dec 24, 2016 at 4:24 PM, Oon-Ee Ng <ngoone...@gmail.com> wrote:

Thanks Martin, that sort of confirms my thoughts on the matter. Most
likely because my own background is engineering and computer science
rather than accounting, though.

Regarding performance, perhaps there can be a convention regarding
splitting input files by year. So I'd have one per year, and two
'main' files, one of which includes every year, and one of which
includes only this year (maybe plus the previous 1 or 2 years if
necessary).

A couple of notes:

- I think it should be possible to automatically generate the offsetting closing and opening transactions to insert in each of your files to bring the opening balances to what they should be. Note that if you change anything in the past files, those transactions may require updates (that sound like a PIA to me). Anyhow, my point is that the act of "closing" should be automatable with a script. (I'm pretty sure I must have hacked this in the past.)

- About performance: I'm revisiting the caching code now so that the parsing stage will have a per-file cache instead of a single large cache for the entire result. That by itself might make a multi-file configuration faster, without an explicit closing of each year, though parsing is only about %30 of processing time.

>> email to beancount+unsubscribe@googlegroups.com.

>> To post to this group, send email to bean...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/beancount/CAGQ70esYPA%3Dm-KiSq2DoHNSy%3DMDHx-FELBpZqKEJDQWq%3Dsy8TQ%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to beancount+unsubscribe@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/beancount/CAK21%2BhOc3S_Y6dTGvbxvr8LEGYhN%3DDVrGcZjg8rAfQ8o_ZSWOg%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAGQ70eu_FRoUJpPzTPfQiHRN0-Oxjwq-PXvdFHiecnOi03W%3Dwg%40mail.gmail.com.

[Oon-Ee Ng]

Sounds very much like using your 'pad' command on 31st December every
year for each account?

>> >> email to beancount+...@googlegroups.com.

>> >> To post to this group, send email to bean...@googlegroups.com.
>> >> To view this discussion on the web visit
>> >>
>> >> https://groups.google.com/d/msgid/beancount/CAGQ70esYPA%3Dm-KiSq2DoHNSy%3DMDHx-FELBpZqKEJDQWq%3Dsy8TQ%40mail.gmail.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "Beancount" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to beancount+...@googlegroups.com.

>> > To post to this group, send email to bean...@googlegroups.com.
>> > To view this discussion on the web visit
>> >
>> > https://groups.google.com/d/msgid/beancount/CAK21%2BhOc3S_Y6dTGvbxvr8LEGYhN%3DDVrGcZjg8rAfQ8o_ZSWOg%40mail.gmail.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Beancount" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to beancount+...@googlegroups.com.

>> To post to this group, send email to bean...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/beancount/CAGQ70eu_FRoUJpPzTPfQiHRN0-Oxjwq-PXvdFHiecnOi03W%3Dwg%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to beancount+...@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/beancount/CAK21%2BhO3z_%3Dq8x%3DEqmQ4y0JvY_gHAf7YgHTxecVqeWtQHnys_A%40mail.gmail.com.

[Martin Blais]

Sort of; it would be generating one big transaction to remove everything from all accounts, and one corresponding big transaction to add everything to all accounts. Ideally, if you used both files, they cancel each other out, and really, they should be removed automatically somehow. 

>> >> email to beancount+unsubscribe@googlegroups.com.

>> >> To post to this group, send email to bean...@googlegroups.com.
>> >> To view this discussion on the web visit
>> >>
>> >> https://groups.google.com/d/msgid/beancount/CAGQ70esYPA%3Dm-KiSq2DoHNSy%3DMDHx-FELBpZqKEJDQWq%3Dsy8TQ%40mail.gmail.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "Beancount" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to beancount+unsubscribe@googlegroups.com.

>> > To post to this group, send email to bean...@googlegroups.com.
>> > To view this discussion on the web visit
>> >
>> > https://groups.google.com/d/msgid/beancount/CAK21%2BhOc3S_Y6dTGvbxvr8LEGYhN%3DDVrGcZjg8rAfQ8o_ZSWOg%40mail.gmail.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Beancount" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to beancount+unsubscribe@googlegroups.com.

>> To post to this group, send email to bean...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/beancount/CAGQ70eu_FRoUJpPzTPfQiHRN0-Oxjwq-PXvdFHiecnOi03W%3Dwg%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to beancount+unsubscribe@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/beancount/CAK21%2BhO3z_%3Dq8x%3DEqmQ4y0JvY_gHAf7YgHTxecVqeWtQHnys_A%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAGQ70evV6T%3D0VmALHze7A_6g05P0cE%2B7pr5w52PEYwgVGEifQQ%40mail.gmail.com.

[Stefano Zacchiroli]

On Sun, Dec 25, 2016 at 05:24:14AM +0800, Oon-Ee Ng wrote:
> Regarding performance, perhaps there can be a convention regarding
> splitting input files by year.

A note of warning: last time I checked, file inclusion in Beancount was
not (yet) completely transparent. There is stuff you can move to a
separate file that you then include (e.g., transactions, account
declarations); and stuff that you cannot include and must be in the main
file you process if you want it to be effective (e.g., plugins).

So if you want your per-year files to behave, semantically, as your
entire multi-year ledger, you'll probably have to have *two* files per
year, one with the real content + one with the plugin declarations and
the include line. The multi-year ledger will have the plugin
declarations + one include per year.

I remember reading in the doc that this is something that Martin planned
to fix, but I don't know if it has happened yet (I'm running a beancount
that dates to a few months back).

Cheers.
--
Stefano Zacchiroli . za...@upsilon.cc . upsilon.cc/zack . . o . . . o . o
Computer Science Professor . CTO Software Heritage . . . . . o . . . o o
Former Debian Project Leader . OSI Board Director . . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

[Martin Blais]

Just put all the plugins and options in the top-level file.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20161225082740.skfbejxueummpqfa%40upsilon.cc.

[Stefano Zacchiroli]

On Sun, Dec 25, 2016 at 12:45:27PM -0500, Martin Blais wrote:
> Just put all the plugins and options in the top-level file.

My point is precisely that that doesn't work. More precisely: it will
have the effect of making the plugins effective when you process the
top-level file, but ineffective when you process per-year files.

Depending on how invasive the plugins in use are, this asymmetry might
result in very surprising side-effects.

[Oon-Ee Ng]

As of now that's not (yet) a huge concern for me starting from ground
zero, but I can see how that would be (maybe?) an issue going forward.
Another option would be to declare the plugins at the top of each file
(there won't be a ton of files this way) in that situation?

My other concern with multi-file is account declaration/closing. If an
account is declared in 2014 and closed in 2016 then both files would
have to be included for beancount to make sense of it. Conversely if
2015 is not included then balances would be off (unless user practices
padding at the end of every month).

> --
> You received this message because you are subscribed to the Google Groups "Beancount" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20161225182511.edre3t3rrtavnwpi%40upsilon.cc.

[Martin Blais]

If the close directive is absent, it just means that the account isn't closed. Presumably if you don't include the 2016 file your intent is not to consider history from those times.

Frankly... just put everything in one file. Let the reporting do the work. That's how it's designed to be used. Somebody wanted multiple files at some point and I gave it to them quickly with a caveat and a warning, but the options and plugins processing haven't really been designed with that in mind.

> To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20161225182511.edre3t3rrtavnwpi%40upsilon.cc.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAGQ70etvsxyC_s28V14T7w-ePYMg1gd%2BRvWuE%2B7gxqxqs9fPMw%40mail.gmail.com.

[Martin Blais]

Tell me, how would you expect / like that the options and plugins directives be interpreted over multiple files?

What seems to be a good behavior in your view?

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20161225182511.edre3t3rrtavnwpi%40upsilon.cc.

[Stefano Zacchiroli]

On Sun, Dec 25, 2016 at 02:35:08PM -0500, Martin Blais wrote:
> Tell me, how would you expect / like that the options and plugins
> directives be interpreted over multiple files?
> What seems to be a good behavior in your view?

JFTR: I use a single "big ledger" file, so I don't personally suffer
from the problem of repeating plugins in multiple files. Still, from a
"ledger engineering" point of view, I'd love to have a single "include
plugins.beancount" directive somewhere.

Answering your question, as a user I'd love "include" to be completely
transparent. That is: the behavior of beancount on a file that contains
one or more "include" lines should be exactly the same of beancount
called on a different file where all includes have been (transitively)
resolved inlining the content(s) of the referenced file(s).

I'd be totally fine if, to achieve this, you'll end up imposing
draconian restrictions such as: "all 'plugin' and/or 'option' directives
must appear at the beginning of the file obtained after resolving all
'include' directives".

I do realize that this might entail implementing a two-stage parsing of
sort, which might be annoying. But you asked about my favorite behavior,
and that would be it :)

[Martin Blais]

https://bitbucket.org/blais/beancount/issues/150/review-behaviour-of-options-and-plugins

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20161226075138.slac6l52pkvrnrij%40upsilon.cc.

[Stefano Zacchiroli]

Thanks!

>> email to beancount+...@googlegroups.com.

>> To post to this group, send email to bean...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/beancount/20161226075138.slac6l52pkvrnrij%40upsilon.cc.
>> For more options, visit https://groups.google.com/d/optout.
>>

--
Sent from my mobile phone. Please excuse my brevity and top-posting.