Transactions in one ledger or split in multiple ledgers (using the include statement)

1,078 views
Skip to first unread message

Mattijs Hoitink

unread,
Aug 21, 2016, 6:45:44 PM8/21/16
to Beancount

I'm going down a path where I have my accounts set up in a main ledger file and include a bunch of sub-ledgers with long lists of transactions. I’m running into some issues (all solvable), for example when ingesting bank statements that contain transactions that should go into separate sub-ledgers.


I was wondering if I’m shooting myself in the foot by doing this and making my life unnecessarily hard. I was also wondering how other people set this up, single big file or separate smaller files and how they handle ingestion.

Martin Blais

unread,
Aug 21, 2016, 10:21:51 PM8/21/16
to Beancount
On Sun, Aug 21, 2016 at 6:45 PM, Mattijs Hoitink <mattijs...@gmail.com> wrote:

I'm going down a path where I have my accounts set up in a main ledger file and include a bunch of sub-ledgers with long lists of transactions. I’m running into some issues (all solvable), for example when ingesting bank statements that contain transactions that should go into separate sub-ledgers.

What issues are you running into, specifically?

I was wondering if I’m shooting myself in the foot by doing this and making my life unnecessarily hard. I was also wondering how other people set this up, single big file or separate smaller files and how they handle ingestion.

Personally, I just use a single large file. I love to be able to quickly i-search all over with Emacs, and I use org-mode to make it manageable.

I know others work like you suggest you do, one top-level file with includes and subfiles.

(One concern is on the treatment of options. I think the top-level file is the one whose option directive take effect. This hasn't been thought out super well TBH, because the ability to add include files has been bolted on late, as per someone's request. I think what ought to be done is to review all the options and categorized them between file- or global- scopes, or something like that.)



Daniël Bos

unread,
Aug 21, 2016, 10:33:44 PM8/21/16
to Beancount

Here's what I do:

Have all options and settings in a master file and include a bunch of files containing my transactions split per year, prices, queries and budgets (Fava feature)

I used to have files split per institution, but this got difficult to navigate pretty quickly. (Not to mention where to put cross-institution transactions)

Apart from prices, I don't ingest much, given the abysmal state of online banking in China, but I hear others ingest into a temp file, give it a look-over to correct any inconsistencies and then append it to their master file. This sounds reasonable to me.


--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhOUVK_%2B51ho5L7UaXxn8ZUPGGMeu6j8rzjVc2Jo6KQPaw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--

Best regards,
Daniël Bos

Your government is reading your email. Slow them down with encryption.

My public key: http://goo.gl/gms497 (4096 bit RSA, id EF2D5D91)
Fingerprint  : D8D0 9FBE F075 F709 7B52  2F73 326C 2123 EF2D 5D91

Martin Blais

unread,
Aug 21, 2016, 10:41:38 PM8/21/16
to Beancount
On Sun, Aug 21, 2016 at 10:33 PM, Daniël Bos <cor...@gmail.com> wrote:

Here's what I do:

Have all options and settings in a master file and include a bunch of files containing my transactions split per year, prices, queries and budgets (Fava feature)

I used to have files split per institution, but this got difficult to navigate pretty quickly. (Not to mention where to put cross-institution transactions)

That's also why I don't split, I like to have everything there, e.g. search for an amount within Emacs, often that's unique enough to find a particular matching transaction. 

I've been thinking lately about moving to a 3 file split: 
1. Open & active accounts, 
2. Closed, old inactive accounts, 
3. Prices.

(Another note: In the background I've started revamping the caching code a few weeks ago, and eventually there will be a cache for the parsing stage for each file. I'm not sure that will make much of a speed difference though, because processing the parsed directives is a substantial portion of the running time.)

Apart from prices, I don't ingest much, given the abysmal state of online banking in China, but I hear others ingest into a temp file, give it a look-over to correct any inconsistencies and then append it to their master file. This sounds reasonable to me.

That's how I do it myself, I run bean-extract to a temp file, and I manually copy over the new bits into their corresponding per institution sections in my gigantic file.


 


On Mon, Aug 22, 2016, 10:21 Martin Blais <bl...@furius.ca> wrote:
On Sun, Aug 21, 2016 at 6:45 PM, Mattijs Hoitink <mattijs...@gmail.com> wrote:

I'm going down a path where I have my accounts set up in a main ledger file and include a bunch of sub-ledgers with long lists of transactions. I’m running into some issues (all solvable), for example when ingesting bank statements that contain transactions that should go into separate sub-ledgers.

What issues are you running into, specifically?

I was wondering if I’m shooting myself in the foot by doing this and making my life unnecessarily hard. I was also wondering how other people set this up, single big file or separate smaller files and how they handle ingestion.

Personally, I just use a single large file. I love to be able to quickly i-search all over with Emacs, and I use org-mode to make it manageable.

I know others work like you suggest you do, one top-level file with includes and subfiles.

(One concern is on the treatment of options. I think the top-level file is the one whose option directive take effect. This hasn't been thought out super well TBH, because the ability to add include files has been bolted on late, as per someone's request. I think what ought to be done is to review all the options and categorized them between file- or global- scopes, or something like that.)



--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.
--

Best regards,
Daniël Bos

Your government is reading your email. Slow them down with encryption.

My public key: http://goo.gl/gms497 (4096 bit RSA, id EF2D5D91)
Fingerprint  : D8D0 9FBE F075 F709 7B52  2F73 326C 2123 EF2D 5D91

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

Mattijs Hoitink

unread,
Aug 21, 2016, 11:20:13 PM8/21/16
to Beancount
What issues are you running into, specifically?
Issues might be overstating it, they are more practical hurdles.

A couple were already mentioned like searchability and navigation. Cross-institution transactions is another good one. Also my autocomplete (Vim) does not work when the accounts are not in the same file.

The biggest thing with splitting the ledgers was that I can't pipe the output from bean-extract into my sub-ledgers because the output might need to be split over multiple files. This could be done manually off course but I wanted some degree of automation. 

I whipped up a simple structure for defining configuration for doing sub-ledger imports, very very very simple to what beancount already had with the config file. Doing this felt a bit weird hence my question.

I've been thinking lately about moving to a 3 file split: 
1. Open & active accounts, 
2. Closed, old inactive accounts, 
3. Prices.

I've migrated my transactions back to a single file but splitting off the prices is a good idea!

Apart from prices, I don't ingest much, given the abysmal state of online banking in China, but I hear others ingest into a temp file, give it a look-over to correct any inconsistencies and then append it to their master file. This sounds reasonable to me.
That's how I do it myself, I run bean-extract to a temp file, and I manually copy over the new bits into their corresponding per institution sections in my gigantic file.
This sounds reasonable to me too and I should incorporate that into my ingestion workflow.

cheers,
mattijs

On Sunday, August 21, 2016 at 7:41:38 PM UTC-7, Martin Blais wrote:
On Sun, Aug 21, 2016 at 10:33 PM, Daniël Bos <cor...@gmail.com> wrote:

Here's what I do:

Have all options and settings in a master file and include a bunch of files containing my transactions split per year, prices, queries and budgets (Fava feature)

I used to have files split per institution, but this got difficult to navigate pretty quickly. (Not to mention where to put cross-institution transactions)

That's also why I don't split, I like to have everything there, e.g. search for an amount within Emacs, often that's unique enough to find a particular matching transaction. 

I've been thinking lately about moving to a 3 file split: 
1. Open & active accounts, 
2. Closed, old inactive accounts, 
3. Prices.

(Another note: In the background I've started revamping the caching code a few weeks ago, and eventually there will be a cache for the parsing stage for each file. I'm not sure that will make much of a speed difference though, because processing the parsed directives is a substantial portion of the running time.)

Apart from prices, I don't ingest much, given the abysmal state of online banking in China, but I hear others ingest into a temp file, give it a look-over to correct any inconsistencies and then append it to their master file. This sounds reasonable to me.

That's how I do it myself, I run bean-extract to a temp file, and I manually copy over the new bits into their corresponding per institution sections in my gigantic file.


 
On Mon, Aug 22, 2016, 10:21 Martin Blais <bl...@furius.ca> wrote:
On Sun, Aug 21, 2016 at 6:45 PM, Mattijs Hoitink <mattijs...@gmail.com> wrote:

I'm going down a path where I have my accounts set up in a main ledger file and include a bunch of sub-ledgers with long lists of transactions. I’m running into some issues (all solvable), for example when ingesting bank statements that contain transactions that should go into separate sub-ledgers.

What issues are you running into, specifically?

I was wondering if I’m shooting myself in the foot by doing this and making my life unnecessarily hard. I was also wondering how other people set this up, single big file or separate smaller files and how they handle ingestion.

Personally, I just use a single large file. I love to be able to quickly i-search all over with Emacs, and I use org-mode to make it manageable.

I know others work like you suggest you do, one top-level file with includes and subfiles.

(One concern is on the treatment of options. I think the top-level file is the one whose option directive take effect. This hasn't been thought out super well TBH, because the ability to add include files has been bolted on late, as per someone's request. I think what ought to be done is to review all the options and categorized them between file- or global- scopes, or something like that.)



--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.
--

Best regards,
Daniël Bos

Your government is reading your email. Slow them down with encryption.

My public key: http://goo.gl/gms497 (4096 bit RSA, id EF2D5D91)
Fingerprint  : D8D0 9FBE F075 F709 7B52  2F73 326C 2123 EF2D 5D91

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

Jason Chu

unread,
Aug 29, 2016, 12:58:51 PM8/29/16
to bean...@googlegroups.com

A couple were already mentioned like searchability and navigation. Cross-institution transactions is another good one. Also my autocomplete (Vim) does not work when the accounts are not in the same file.

Assuming you use vim-beancount, this is not actually true:


You need to set b:beancount_root to the master file and vim-beancount will support proper account completion.

I also have a PR that will support many more forms of completion.  Hopefully it will be accepted soon...
Reply all
Reply to author
Forward
0 new messages