a single, cumulative journal, or annual journal files?

[Chris Ryan]

Hi all. As my little solo business is coming to the end of its first calendar year of operation (started in June), I guess I face a decision. Should I keep just one continuous, cumulative journal file, or should I start using a new one each year? What is the conventional recommendation, if any?

Thanks.

--
Chris Ryan
rya...@fastmail.co.uk

[Francesco Ariis]

Hello Chris,

Il 28 dicembre 2023 alle 03:09 Chris Ryan ha scritto:
> Hi all. As my little solo business is coming to the end of its first
> calendar year of operation (started in June), I guess I face a decision.
> Should I keep just one continuous, cumulative journal file, or should I
> start using a new one each year? What is the conventional recommendation, if
> any?

One for each year.

Archive old journals, accountants do not use erasers or they end
up in jail
—F

[Henning Thielemann]

On Thu, 28 Dec 2023, Chris Ryan wrote:

> Hi all. As my little solo business is coming to the end of its first
> calendar year of operation (started in June), I guess I face a decision.
> Should I keep just one continuous, cumulative journal file, or should I
> start using a new one each year? What is the conventional
> recommendation, if any?

I guess this depends on the volume of data you acquire. I think you do not
need to decide this immediately. As soon as data becomes unhandy, you can
finish one journal and start the next one. For my accounting I have not
split journals into years but into sources of data. That is, I have a
journal for data from my bank account. I extend it using CSV tables from
my bank. Then I have a journal for payments without involvement of my bank
account and another journal for my working time. I manage all those
journals in a darcs repository.

[Simon Michael]

Re changing old data. Heh, I sometimes do cleanups in old journals and they have not caught me yet ! Usually guess the main consumer of your journals and reports is you, and the first purpose of the tool is to give you clarity. So I don't hesitate to clean up or fix the historical record in ways that seem useful to me. Tracking all changes in version control of course, so that I can see how the data arrived in its present state and could revert if needed.

I would say: especially when starting out, less is more. These little complexities add up and multiply. All else being equal, the fewer files and moving parts, the simpler your life will be. People with not overmuch data and/or fast machines may find performance quite acceptable for a number of years.

But over the long term most of us find it convenient to break things up by year, for

1. performance - so that there's no significant delay in your everyday hledger commands.
2. compartmentalisation - so that there's no worry about accidentally disturbing old data
3. focus - so that you see recent transactions by default and don't have to always exclude old data from reports

There are slight complications around ending and starting balances (see the close command; I think we can improve things here).

Some people break up files by criteria other than time period, eg data source. This tends to bring more complexity than dividing only by time, eg because of inter-file transfers. Though if you have some accounts which are naturally very separate, it may work fine.

It usually isn't too hard to switch between these data organisations, if you're comfortable with text editing / version control. So you may want to experiment.

[Chris Ryan]

I'm beginning to see the pros and cons of each approach.

In the US, federal income taxes revolve around a calendar year. Notionally, all the business activity occuring in one year goes into that year's tax return. So annual journal files have something going for them, conceptually.

On the other hand, I expect my .journal file will grow very slowly, and my machine is plenty capable. It would not be difficult to get into the habit of including a reporting period in my report commands. Would maintaining a single, cumulative .journal allow me to generate a report by year, so I can see how my e.g. travel expenses differed this year from last year? I can see that being very useful.

Thanks.
--
Chris Ryan
rya...@fastmail.co.uk

> --
> You received this message because you are subscribed to the Google
> Groups "hledger" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hledger+u...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/hledger/5B0A2382-8D04-479C-A580-B0927B30AD92%40joyful.com.

[Simon Michael]

> On Dec 28, 2023, at 07:35, Chris Ryan <rya...@fastmail.co.uk> wrote:
>
> I'm beginning to see the pros and cons of each approach.
>
> In the US, federal income taxes revolve around a calendar year. Notionally, all the business activity occuring in one year goes into that year's tax return. So annual journal files have something going for them, conceptually.

Indeed this is another choice - if dividing files by time, do you divide by calendar year or tax year, which is usually offset by a few months ? I think most people do the first as it's more natural; then tax reporting involves at least two year files. I've heard at least one person doing the second; I imagine it makes tax reporting a bit simpler, though even in this case I bet you have to get some data from past years.

> On the other hand, I expect my .journal file will grow very slowly, and my machine is plenty capable. It would not be difficult to get into the habit of including a reporting period in my report commands. Would maintaining a single, cumulative .journal allow me to generate a report by year, so I can see how my e.g. travel expenses differed this year from last year? I can see that being very useful.

Certainly. Reports are independent of how the files are structured.

[Henning Thielemann]

On Thu, 28 Dec 2023, Simon Michael wrote:

> Indeed this is another choice - if dividing files by time, do you divide
> by calendar year or tax year, which is usually offset by a few months ?

As a freelancer in Germany I have even two other options, to make things
more complicated: I can choose tax declaration with respect to bill dates
or with respect to transaction dates. Oh and for acquisition of certain
products, like an expensive computer, I have to spread the purchase
virtually across multiple years. Currently I just write all part
transactions at one place but they address several future years. If I
divided my files by years, I would have to touch more than one annual
journal file for preparing a tax declaration.

[Diego Depaoli]

I started using one master file, then I switched to one per year,
Easier to maintain, edit, review... with some minor downside

My regards

Il giorno gio 28 dic 2023 alle ore 04:13 Francesco Ariis
<fa...@ariis.it> ha scritto:

> --
> You received this message because you are subscribed to the Google Groups "hledger" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hledger+u...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/hledger/ZYznzo1ttl390xHl%40x270.casa.



--
Diego Depaoli

[Peter Sagerson]

I have about ten years of data (personal finances), which I divide up by year for performance and general tidiness. I've struggled a bit with how to manage the opening/closing balances: closing balances at the end of a year messes with year-end balance reports and opening balances at the beginning of a year messes with cash flow reports (which in fairness I care less about).

I recently settled on an arrangement that seems to be working well for me. It uses a number of files, so it may or may not be over your complexity threshold.

1. My root hledger data directory contains a journal file for each year (e.g. 2023.journal). These contain only transactions, plus perhaps minimal directives like decimal-mark or Y.
2. Global configuration (accounts, commodities, prices, etc.) live in files inside a "global" subdirectory.
3. The root also contains a couple of top-level journal files, which provide a bit of indirection and tie everything together:

all.journal:

# Alias directives at the top, if needed.

include global/*.journal

# This matches numerical filenames:

include <->.journal

current.journal:

# Alias directives at the top, if needed

include global/*.journal

# Output of hledger close --open -p 2022 (date adjusted):

2022-12-31 Opening balances

    ...

include 2023.journal

LEDGER_FILE normally points to current.journal for speed, but I can switch it to all.journal to look at past years or year-over-year reports. And I can always add additional top-level journals (previous.journal, etc.) if needed.

In other words, the rule is that my LEDGER_FILE always points to something that does four things:

1. Define any necessary aliases (since they can't be included).
2. Include global configuration.
   
3. Establish opening balances on the day prior to the target period.
4. Include transactions in some target range.
   

[Simon Michael]

# This matches numerical filenames:

include <->.journal

Really! How does that work ?

In other words, the rule is that my LEDGER_FILE always points to something that does four things:

1. Define any necessary aliases (since they can't be included).
2. Include global configuration.
   
3. Establish opening balances on the day prior to the target period.
4. Include transactions in some target range.
   

Thanks for sharing this setup. 

My accounts evolve over the years (especially in the early years) so I tend to split accounts and aliases by year also.

I wonder what you'd think of using balance assignments as in my 12/28 "Simplifying multiple year files, closing/opening balances" post.

[Peter Sagerson]

On Tue, Jan 2, 2024, at 12:02 PM, Simon Michael wrote:

# This matches numerical filenames:

include <->.journal

Really! How does that work ?

I just followed the link in the manual to the System.FilePath.Glob documentation:

<m-n>

Matches any integer in the range m to n, inclusive. The range may be open-ended by leaving out either number: "<->", for instance, matches any integer.

I wonder what you'd think of using balance assignments as in my 12/28 "Simplifying multiple year files, closing/opening balances" post.

For some reason, I never considered that; I think I just got focused on the close command and never stepped back. The error-checking strategy of visually comparing end-of-year assertions to beginning-of-year assignments sounds plausible, although I suspect that comparing balance reports from theoretically equivalent transaction streams might be easier (and easier to automate). I'd have to play with that a bit.

The proposed hybrid approach certainly sounds like it could solve the problem. Any file could then serve as either the beginning or middle of a transaction stream with the semantics adapting to be most appropriate to the situation. One thing that makes me a little queasy at first glance is the conflation of a zero balance with an absent balance. Ideally, I would probably want the semantics to be "if this account has 0 previous transactions, then assign, else assert."

Of course, the narrowest approach would be "if this is the first transaction, then assign all, else assert all." That might be useful, but it's getting a bit idiosyncratic and would seem to imply transaction-level syntax rather than posting-level syntax. Probably a bridge too far.

Thanks,

Peter

[Simon Michael]

Interesting, thanks. 

I guess you'd agree with my feeling that the workflow requiring matching end-of-file closing and start-of-file opening transactions is a bit of a pain as files and accounts and historical edits increase.  Your system of keeping opening balances in a parent file is an alternative I should try more. But it still feels a bit of a complicated setup to present to newcomers or casual users.

[Simon Michael]

> On Jan 2, 2024, at 14:35, Simon Michael <si...@joyful.com> wrote:
> Interesting, thanks.
>
> I guess you'd agree with my feeling that the workflow requiring matching end-of-file closing and start-of-file opening transactions is a bit of a pain as files and accounts and historical edits increase. Your system of keeping opening balances in a parent file is an alternative I should try more. But it still feels a bit of a complicated setup to present to newcomers or casual users.

Another thought I've had for a while is to specially mark opening transactions, and have reports automatically ignore those when it sees earlier-dated postings for an account. Ie, automating our current manual methods of excluding redundant/unwanted opening/closing transactions.

[Simon Michael]

On Jan 1, 2024, at 11:01, Peter Sagerson <peter.s...@gmail.com> wrote:

I recently settled on an arrangement that seems to be working well for me. It uses a number of files, so it may or may not be over your complexity threshold.

1. My root hledger data directory contains a journal file for each year (e.g. 2023.journal). These contain only transactions, plus perhaps minimal directives like decimal-mark or Y.
2. Global configuration (accounts, commodities, prices, etc.) live in files inside a "global" subdirectory.
3. The root also contains a couple of top-level journal files, which provide a bit of indirection and tie everything together:

all.journal:

# Alias directives at the top, if needed.

include global/*.journal

# This matches numerical filenames:

include <->.journal

current.journal:

# Alias directives at the top, if needed

include global/*.journal

# Output of hledger close --open -p 2022 (date adjusted):

2022-12-31 Opening balances

    ...

include 2023.journal

LEDGER_FILE normally points to current.journal for speed, but I can switch it to all.journal to look at past years or year-over-year reports. And I can always add additional top-level journals (previous.journal, etc.) if needed.

Hi Peter,

I'm following up on this as I gather notes at #2151. Am I right in thinking your setup is a variant of https://github.com/simonmichael/hledger/issues/2151#issuecomment-1885306843 > method E ? Would you agree generally with my Simpler methods (E,F) > busywork remarks there ?

Thanks

[Peter Sagerson]

I'm following up on this as I gather notes at #2151. Am I right in thinking your setup is a variant of https://github.com/simonmichael/hledger/issues/2151#issuecomment-1885306843 > method E ? Would you agree generally with my Simpler methods (E,F) > busywork remarks there?

More or less. Although referring again to System.FilePath.Glob, <2020->.journal (e.g.) should match all years starting from 2020, so one could easily arrange things so the parent files don't need to be updated every year.

I've always imagined my sequence of annual journals as a kind of linked list with extra heads. If we had one of those magical starting-balances mechanisms, I wonder if you could also arrange the years in a daisy-chain configuration? So, for example, 2020.journal would contain:

1. Global declarations (accounts, commodities, prices, etc.), directly or via include
   
2. Starting balances, in case this is the first year
   
3. 2020 transactions
   
4. include 2021.journal
   

The global config would end up being re-asserted each year, which is a tad untidy, but presumably a negligible performance hit. It precludes some narrowly scoped and discontinuous transaction sets, but I suspect those are pretty fringe cases for a lot of people. I don't hesitate to work with my full transaction set as needed; it's noticeably slower, but entirely usable (a little under 10k transactions).

I'm not sure if I would actually prefer a daisy chain to parent files, but it could be interesting.

Thanks,

Peter

[Will Robinson]

Just a general note to give some thought to folks who want to retain earnings (and/or do other things like consolidate conversion accounts into owners' equity) at annual boundaries as part of their periodic rollover process as well. I'm not sure this would interact badly (or at all) with any of Simon's proposals, but I didn't see an explicit mention of it...

Thanks,

Will

--

You received this message because you are subscribed to the Google Groups "hledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hledger+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hledger/3f8deb2e-bf69-4b5c-8861-a2d1580d86a8%40app.fastmail.com.

--

Will Robinson (willro...@gmail.com)

[Simon Michael]

Thanks Will. No plans to reduce the close command's flexibility or --retain mode at the moment.

It would be interesting to hear any more details, such as why you do these, whether it's personal or for a business, any year-end command samples, is it a separate step from migrating AL balances ?

[Will Robinson]

I retain earnings for my (very) small business. I do it because I read some reference to doing so in an accounting textbook. :-) Basically when I save off my "final" income statement for 2023, I get to see a retained-earnings line heading into 2024 that matches my income statement's net earnings for 2023. In theory this gives me (or a future auditor) some additional sanity-check in looking back at years of history that the balance sheet and income statements relate to each other as you'd expect them to. At least that's the theory as I fuzzily understand it.

It's also somewhat nice heading into the year to have everything in an asset, liability, or equity account. The "separateness" of revenue and expense accounts feels good for mid-year churn, but feels incomplete when I'm closing the books on the year.

I consolidate the conversion equity accounts because I just thought it made for a cleaner balance sheet to read at the end/beginning of the year, but I suppose I'm not really wedded to that one.

Will

--
You received this message because you are subscribed to the Google Groups "hledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hledger+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hledger/2D2540DC-BC6B-4FC4-BB42-190DF6FE572C%40joyful.com.

--

Will Robinson (willro...@gmail.com)

[Simon Michael]

Thanks. I don't think we have nailed down "best practices" in these things at all, so it's interesting to hear what folks are doing.

Do you migrate all equity to next year ? Except for the opening/closing balances account ?

To view this discussion on the web visit https://groups.google.com/d/msgid/hledger/CAAOg-BfGdAfMHhrkGJ0MYSQu-H5CfKsacEDwmGD6TK1mCTLwrQ%40mail.gmail.com.

[Will Robinson]

Yes. My close-out balances for 2023 contain two equity accounts:

  retained earnings

  owner's equity

[and no revenue or expense balances; I've rolled all those into retained earnings].

...and then my first transaction in 2024 then rolls retained earnings into owner's equity. So I go into 2024 proper with only assets, liabilities, and owner's equity.

I do still have the close/open transaction issues between the years that you are thinking about iterating on in this thread.

Last note on this setup: if I want to see an income statement that spans the year boundary correctly, I also need to ignore the "close --retain" transaction where I rolled revenue/expense into retained earnings. I add a "retain:" tag to those transactions (and only those), and use "hledger is not:tag:retain" if I want to look at income statements touching earlier years. It's clunky but it does seem to work.

Will

To view this discussion on the web visit https://groups.google.com/d/msgid/hledger/3A41EFF8-7B12-42C5-A9CA-885755505233%40joyful.com.

--

Will Robinson (willro...@gmail.com)