Budgeting in hledger

[Edward Amsden]

Hello,

I've spent the last several days going over my financial history and getting it all into hledger. The detailed look into my personal finances is exciting (and slightly terrifying!)

Prior to encountering hledger, I've been using a system I cooked up in Haskell to answer the question "How much can I spend on date X without running out of money for future expenses?"

That is, I have what is essentially a combined predicted and actual register for my checking account. I can mark transactions that have already posted in order to verify balances, day-by-day balance changes, &c with the csvs that I download from my bank. But this log also includes projected future expenses and income, and lets me run a report of available balances by day, i.e. how much money could I spend on a day such that the minimum future balance (as far as the log goes forward) is 0?

This system has worked well for answering this question, but having now spent a lot of time getting things put into hledger in more detail, I'd like to figure out a way to answer this question using hledger.

Ideally, an hledger budgeting system would let me

 - Write future transactions as ledger --budget style periodic transactions, rather than writing them out week by week, month by month, quarter by quarter, year by year, and usually forgetting at least one of the quarterly or yearly ones

  + Instantiations of periodic transactions would be either "fulfilled" or "unfulfilled"

    = unfulfilled -> included in balance history when forward predictions are run

    = fulfilled -> not included in balance history when forward predictions are run, as they were placeholders for an amount that has now posted

       fulfilled transactions can still be used to measure performance vs budget (i.e. this account/category is over or under budget)

 - Distinguish between point and bucket budget transactions

    + certain incomes/expenses are best budgeted in buckets, others as one transaction per period

    + Periodic bucket transactions could be considered fulfilled after the period for that transaction ends, whether there are 0, 1, or many fulfilling transactions

       = Ex groceries, eating out, entertainment, shopping

    + Periodic point transactions could be marked fulfilled when matched with a corresponding real transaction

       = Ex utility bills, loan payments, any expense for which you expect to be billed later

 - Allow one-time alterations to periodic transactions

  + Often know a bill amount well ahead of actual payment

  + Sometimes a bucket should be expanded i.e. groceries for holidays or a planned visit, entertainment for a planned family getaway

I've taken a look at the hledger-budget addon and it doesn't really seem suited to answering the question: "how much can I spend without overdrafting my bank account."

If someone could be easily available to help me navigate the hledger codebase I'd be willing to try to contribute a tool matching the spec above. I'm very fluent in Haskell (I'm lucky enough to develop in Haskell all day for work!) but fairly new to plain text accounting and very unfamiliar with the codebase.

I'm also open to suggestions and criticisms of these ideas. Have at it!

- Edward

[Simon Michael]

Welcome Edward!

> On Sep 27, 2017, at 3:26 PM, Edward Amsden <edward...@gmail.com> wrote:
> Hello,
>
> I've spent the last several days going over my financial history and getting it all into hledger. The detailed look into my personal finances is exciting (and slightly terrifying!)

Excellent. I relate. :)

> Prior to encountering hledger, I've been using a system I cooked up in Haskell to answer the question "How much can I spend on date X without running out of money for future expenses?"
>
> That is, I have what is essentially a combined predicted and actual register for my checking account. I can mark transactions that have already posted in order to verify balances, day-by-day balance changes, &c with the csvs that I download from my bank. But this log also includes projected future expenses and income, and lets me run a report of available balances by day, i.e. how much money could I spend on a day such that the minimum future balance (as far as the log goes forward) is 0?
>
> This system has worked well for answering this question, but having now spent a lot of time getting things put into hledger in more detail, I'd like to figure out a way to answer this question using hledger.

Before we discuss the alternative, I want to speak well of the manual method. For quite some time I maintained a future.journal, which I could include whenever I wanted (alias future="hledger -f $LEDGER_FILE -f future.journal") to add forecasts to my reports. This wasn't as much work as it sounds:

- it contained only the main predictable transactions for the coming year

- it contained org nodes for easy folding and navigation by month/subsection

- I could generate and update future transactions quite quickly using copy/paste/interactive macros in emacs

- it handled irregular or one-time transactions easily, unlike recurring transaction rules

- this allowed more separation of transaction planning, entry writing, and timely journal updating. Ie I could plan my future transactions, and figure out the right journal entries for them, ahead of time, without pressure. Then during the week, "in the thick of battle", I had only to move entries from future.journal to main journal.

> Ideally, an hledger budgeting system would let me
>
> - Write future transactions as ledger --budget style periodic transactions, rather than writing them out week by week, month by month, quarter by quarter, year by year, and usually forgetting at least one of the quarterly or yearly ones
> + Instantiations of periodic transactions would be either "fulfilled" or "unfulfilled"
> = unfulfilled -> included in balance history when forward predictions are run
> = fulfilled -> not included in balance history when forward predictions are run, as they were placeholders for an amount that has now posted
> fulfilled transactions can still be used to measure performance vs budget (i.e. this account/category is over or under budget)
> - Distinguish between point and bucket budget transactions
> + certain incomes/expenses are best budgeted in buckets, others as one transaction per period
> + Periodic bucket transactions could be considered fulfilled after the period for that transaction ends, whether there are 0, 1, or many fulfilling transactions
> = Ex groceries, eating out, entertainment, shopping
> + Periodic point transactions could be marked fulfilled when matched with a corresponding real transaction
> = Ex utility bills, loan payments, any expense for which you expect to be billed later
> - Allow one-time alterations to periodic transactions
> + Often know a bill amount well ahead of actual payment
> + Sometimes a bucket should be expanded i.e. groceries for holidays or a planned visit, entertainment for a planned family getaway
>
> I've taken a look at the hledger-budget addon and it doesn't really seem suited to answering the question: "how much can I spend without overdrafting my bank account."
> If someone could be easily available to help me navigate the hledger codebase I'd be willing to try to contribute a tool matching the spec above. I'm very fluent in Haskell (I'm lucky enough to develop in Haskell all day for work!) but fairly new to plain text accounting and very unfamiliar with the codebase.
>
> I'm also open to suggestions and criticisms of these ideas. Have at it!

I like your thinking! And.. budgeting gets complicated fast, doesn't it ?!

I haven't given hledger-budget a real try, but AFAIK it's not a real usable solution yet (Correct me if wrong.. and it has bitrotted recently and needs an update.)

The periodic transaction functionality added for hledger-budget is available in hledger-lib, though not yet always-on.

In the past I wanted periodic transaction rules that automatically deactivate themselves when a sufficiently similar actual journal entry exists in the time period. Eg a rule for weekly groceries purchase will be ignored in weeks where a groceries purchase is already recorded. I don't remember exactly why, now. Perhaps to allow mixing rule-based and manual (past and future) transactions more effectively.

I found some related mockups:

https://github.com/simonmichael/hledger/blob/master/doc/mockups/budget-reporting.txt

https://gist.github.com/simonmichael/d303bc30d2deb96519600afefdccc2e5

[Simon Michael]

PS re the codebase: http://hledger.org/contributing.html has a lot of maybe-still-current info. "make" list some useful make rules (like "make ghcid"). http://hledger.org/contributing.html#code gives a quick overview. I'm happy to answer questions here or in #hledger.

[Edward Amsden]

Before we discuss the alternative, I want to speak well of the manual method. For quite some time I maintained a future.journal, which I could include whenever I wanted (alias future="hledger -f $LEDGER_FILE -f future.journal") to add forecasts to my reports. This wasn't as much work as it sounds:

- it contained only the main predictable transactions for the coming year

- it contained org nodes for easy folding and navigation by month/subsection

- I could generate and update future transactions quite quickly using copy/paste/interactive macros in emacs

- it handled irregular or one-time transactions easily, unlike recurring transaction rules

- this allowed more separation of transaction planning, entry writing, and timely journal updating. Ie I could plan my future transactions, and figure out the right journal entries for them, ahead of time, without pressure. Then during the week, "in the thick of battle", I had only to move entries from future.journal to main journal.

This is what I've done too: created a budget.journal that contains an include directive for the main journal and future or yet-to-post transactions, which can be moved to the main journal (or deleted if hledger-autosync grabs them) It definitely has the merits you mention. It's pretty much equivalent to what I was doing before hledger, except my system was able to do a report of "on a given date, what's the minimum future balance [of the checking account]" i.e. "how much can we spend without overdrafting the checking account at some future point?" For the sort of on-the-fly budget decisions necessary for a family, this was extremely helpful. Now I scan the output of `hledger register -H assets:checking -p "from today"` and look for the lowest future balance.

 

> Ideally, an hledger budgeting system would let me
>
>  - Write future transactions as ledger --budget style periodic transactions, rather than writing them out week by week, month by month, quarter by quarter, year by year, and usually forgetting at least one of the quarterly or yearly ones
>   + Instantiations of periodic transactions would be either "fulfilled" or "unfulfilled"
>     = unfulfilled -> included in balance history when forward predictions are run
>     = fulfilled -> not included in balance history when forward predictions are run, as they were placeholders for an amount that has now posted
>        fulfilled transactions can still be used to measure performance vs budget (i.e. this account/category is over or under budget)
>  - Distinguish between point and bucket budget transactions
>     + certain incomes/expenses are best budgeted in buckets, others as one transaction per period
>     + Periodic bucket transactions could be considered fulfilled after the period for that transaction ends, whether there are 0, 1, or many fulfilling transactions
>        = Ex groceries, eating out, entertainment, shopping
>     + Periodic point transactions could be marked fulfilled when matched with a corresponding real transaction
>        = Ex utility bills, loan payments, any expense for which you expect to be billed later
>  - Allow one-time alterations to periodic transactions
>   + Often know a bill amount well ahead of actual payment
>   + Sometimes a bucket should be expanded i.e. groceries for holidays or a planned visit, entertainment for a planned family getaway
>
> I've taken a look at the hledger-budget addon and it doesn't really seem suited to answering the question: "how much can I spend without overdrafting my bank account."
> If someone could be easily available to help me navigate the hledger codebase I'd be willing to try to contribute a tool matching the spec above. I'm very fluent in Haskell (I'm lucky enough to develop in Haskell all day for work!) but fairly new to plain text accounting and very unfamiliar with the codebase.
>
> I'm also open to suggestions and criticisms of these ideas. Have at it!

I like your thinking! And.. budgeting gets complicated fast, doesn't it ?!

Quite complicated. This is what keeps me away from commercial tools like YouNeedABudget and Mint. They seem to only handle bucket-type budgets, and not with the precision of expense prediction that I need. I know when my electric bill comes, and I have a pretty good idea of how much it will be well ahead of time. A monthly "bucket" for electricity doesn't handle that well.

 

I haven't given hledger-budget a real try, but AFAIK it's not a real usable solution yet (Correct me if wrong.. and it has bitrotted recently and needs an update.)

It's not, and afaict it doesn't yet even have the functionality of ledger --budget (which is also not sufficient for my needs.

 

The periodic transaction functionality added for hledger-budget is available in hledger-lib, though not yet always-on.

I was reading through those documents. Something more than just periodic transactions will be necessary I think. At least, the distinction between once-per-period ("point") transactions and periodic buckets. 

In the past I wanted periodic transaction rules that automatically deactivate themselves when a sufficiently similar actual journal entry exists in the time period. Eg a rule for weekly groceries purchase will be ignored in weeks where a groceries purchase is already recorded. I don't remember exactly why, now. Perhaps to allow mixing rule-based and manual (past and future) transactions more effectively.

This sounds like what I want for once-per-period budget transactions. For bucket-style transactions (we budget our groceries bucket-style) deactivation after the time period for the bucket has passed seems like a good way to go. I.e. if the bucket is for the month of September 2017 and it's now October, any grocery transactions should come out of October's budget and the remaining September bucket should be considered coming in under-budget i.e. winning :)

It would be good to use budgets not just for prediction but also for ledger --budget style reports that show how well you are doing with your budget. This is another downside of the future journal file: unless you keep a copy and figure out the diffs yourself, you have only mental approximations or transaction-by-transaction indications (as you match actual to predicted transactions) of how well you are staying in budget.

 

I found some related mockups:

https://github.com/simonmichael/hledger/blob/master/doc/mockups/budget-reporting.txt

https://gist.github.com/simonmichael/d303bc30d2deb96519600afefdccc2e5

I'll take a look at these!

 

--
You received this message because you are subscribed to a topic in the Google Groups "hledger" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hledger/jI8FlZBvMkA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to hledger+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Simon Michael]

More ui possibilities:

is -M --budget-diff  # instead of inflows/outflows, show difference from budgeted amount

is -M --budget-percent  # instead of inflows/outflows, show percentage of budgeted amount

Edward, would you have a beginner-friendly explanation of point-in-time vs bucket budgeting ? I'm unclear on this.

You received this message because you are subscribed to the Google Groups "hledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hledger+u...@googlegroups.com.

[Edward Amsden]

Well I am somewhat unclear on it too. I am trying to invent concepts that I haven't seen elsewhere in the budgeting world (though maybe they are out there!)

Bucket based budgeting says "I plan to spend $X/month on thing Y and, if you have actually spent $Z so far, I expect that you'll still spend $X-Z this month and then $X each future month."

So if I've budget $400/mo for groceries and I've spend $100, it expects that I'll still spend $300 on groceries this month, and if you're running future totals for your budget, totals the current month accordingly.

Point based budgeting says "I think I will have to spend $X/month on thing $Y, but if I have actually spent $Z this month, then that's the actual amount". For expenses like utility bills, this is more appropriate. If I budget $200/mo for my electric bill, and my bill due June 15 comes in the mail for $150 and I enter it into the budget, there's no reason for the budgeting program to say "Well he has planned to spend $200 in june so we'll plan on him spending another $50 past that bill just in case". It should run the totals going forward into the future with that $150 figure for June.

I don't have the concept down precisely in my head, but I'm trying to describe my issue with ledger --budget style budgeting for budget prediction

[André Fincato]

Just chiming in to say that what I am reading about a possible hledger budget development looks very exciting to me as well, as I was indeed figuring out if hledger would handle that and eventually going to test ledger. 

But it seems than neither can do it, and what Edward is sharing is very good so far.

Keep going :D

[Edward Amsden]

Ledger has some budgeting facilities but (from what I've read) it's pretty simple bucketing a la YNAB.

I'm using hledger for our budget now, using the approach Simon outlined, and it works well. Don't get turned away just because there's no budget functionality yet!

[Jeff Richards]

I'm very interested in this conversation as well. 

I'm currently running a bucket(envelope) style budget, but struggling to be able to reconcile the buckets to the accounts that hold money at the end of each month (I'm on my third month).  Additionally, as a contractor, there's a certain amount of variability to my income that resists just putting the income straight into the buckets at regular intervals with regular amounts.  So I've employed a bucket called "unallocated" all the income goes in there and then comes out of there on a monthly basis to fill up the buckets.  Doing a monthly review allows me to make adjustments like in Edward's utility bill case, to decide if I need to leave the unspent money from the bucket in the bucket for a future bill (in my case, we heat with electricity so that varies by month of the year) or put that money back into the unallocated bucket.  

I've been using ledger with the virtual transactions but it's been somewhat challenging to get it set up and keep it all straight.

In my perfect world, I would be able to see the balances of my bank account and credit card and have that equal the same number as my buckets. My monthly review would involve me making transfer to buckets from my unallocated amounts, when buckets are overspent and that troubles me, and from buckets to unallocated when an unnecessary build up has occurred , THEN when asked the decision, can we do this unexpected thing, I'd be able to look at the unallocated amount and make a judgement if we have enough unallocated money to do said thing.  

I feel like I'm close with my current ledger/virtual transactions based system, but I can't quite get it to add up yet, (which may be just human error).  I'm happy to share the details of that if anyone is interested.  But I'm also very happy to switch to a different system if someone has come up something that meets my needs!

To unsubscribe from this group and all its topics, send an email to hledger+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

-- 

You received this message because you are subscribed to the Google Groups "hledger" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hledger+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to a topic in the Google Groups "hledger" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hledger/jI8FlZBvMkA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to hledger+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "hledger" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hledger+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "hledger" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hledger+unsubscribe@googlegroups.com.

[Simon Michael]

I need to focus FOSS time on the 1.4 release for a bit, so excuse brief input from me.

envelope or bucket ? "envelope budgeting" more familiar to the world at large, or less ?

Inflow/outflow budgeting vs envelope (bucket) budgeting.

With the former, you set inflow/outflow targets (per account, per period..), and then you report actual performance compared to those goals. Ledger users set the goals with periodic transaction rules, and see performance with the --budget report. hledger-budget does the same, I believe. The goals exist outside of your journal, and are informational only.

With the latter, instead of setting informational goals you allocate funds to specific envelopes/accounts/virtual subaccounts, each with its own purpose. As YNAB says: "give every dollar a job". This involves actual transactions and movement of money, at least in your journal if not in the real world. You keep track of the evolving balance of each envelope, and can derive performance from this. There are choices to make such as:

- how often do I "budget" (set goals, update envelope balances..) ?
- when can funds be added to an envelope ?
- can funds be transferred between envelopes ?
- what happens to money remaining in envelopes at the end of a budget period
- what happens if I overspend an envelope ? (This can't happen with real envelopes, which is the point, but it can happen with our virtual equivalents)

I saw two kinds of envelopes: the kind you empty and refill each period, used to control spending; and the kind you keep filling over time, and withdraw from seldom. I called them "cashflow envelopes" and "savings envelopes" or some such.

http://plaintextaccounting.org/#budgeting has examples of some of the different approaches.

"Envelope budgeting with ordinary accounts" is some notes on fully manual envelope system I used for a year. It doesn't require any special support from hledger/ledger. It is somewhat labour-intensive, but when you need a rock-solid command of your planning and cashflow, and total flexibility, I can recommend it.

I dropped it this year, because in combination with other things I was doing (household shared expenses, other things I forget) it was too much hassle - the complexity multiplied and figuring out the journal entries became too much of a chore. (With more support for automated postings in hledger, this might be less of a problem).

Anyway, I'd like my next budgeting system to not complicate ordinary transactions or reports; it should be an optional overlay. I'd like to try a more lightweight target-based system.

We should support both approaches. Perhaps there are others ?

Ok, so brief....

[Simon Michael]

> On Sep 29, 2017, at 12:30 PM, Simon Michael <si...@joyful.com> wrote:
> see performance with the --budget report. hledger-budget does the same, I believe. The goals exist outside of your journal, and are informational only.

Outside of your transaction history, I meant. (In the Ledger case they are configured in the journal file, using pseudo transactions which aren't used except for setting goals for --budget).

> Ok, so brief....

*not* so brief...

[Stephen Christie]

I guess I have good timing. I just came over to see what the status was for budgets in hledger. I currently use ledger-cli, mostly because hledger itself told me to use ledger if I want to use it for budgeting, but I've never quite gotten it how I've wanted. After some experimenting, this is the simple setup I've got:

•  I use the high-level expense accounts directly as my "buckets": a negative transaction on them cancels out the (positive) expenses. The expense accounts balance against the Assets, and the difference is the un-budgeted safe-to-spend money (or over-budget spending if negative). Of course it's rather odd to look at negative (and often red) numbers in the expense accounts as a good thing (budget available to spend in this category). Another way to look at is this category is over-budgeted -- but I don't use it this way (more on that in a bit)
  
• I keep only true transactions in my main file, and therefore use unbalanced transactions to fill the buckets. This is important as I still need to keep track of and categorize my income.
• To hide non-budgeted expenses, (especially sub-categories, since I budget on a higher-level than I track), I place a zero-value transaction into a recurring transaction. Ledger hides any non-budget accounts when viewing --budget.
• Right now, I have three things that go into my budget:
• Housing: I budget monthly, as a virtual rent payment higher than my actual rent. Since it goes into the housing expense account, this amount also covers other housing expenses such as repairs. More importantly, this is also where payment for a house would expense to, so really this is my savings rate for buying a house. So a very full (or over-budget, red envolope) is one that means I have a lot saved up, and gives reasons to try to cut back on expenses in related areas (like finding lower rent). This is my only recurring transaction for the moment, but similar ones could follow.
  
• Tithe: 10% is budgeted directly with every transaction from Income:Taxable. This is probably the easiest place to budget. For instance, a %30 retirement savings rate, or a $10 celebratory schnitzel on payday.

So I guess I use a lot of Ledger's functionality not quite in hledger yet. It's still awkward though, as I have to run several reports. I'd love for it to be a bit more smooth.

Sincerely,

Stephen

[Stephen Christie]

That's weird, somehow when I sent this, it lost one of my more interesting points.

I keep track of mileage in ledger. I actually have made a simplified syntax for it, since everything uses the same accounts: An account for my car's total mileage (separate from my finances) budgets to my car expenses via virtual transaction. This expense account covers gas, repairs and upgrades, and eventually a new car. This way, the wear I put on my car is saved up at a rate that it will pay for the car's replacement, at the end of it's life. I call this "usage based budgeting", and I could see it being used for other consumables, for instance groceries.

Sorry for the double post,
Stephen Christie

[Simon Michael]

Hi Jeff,

On 9/29/17 2:18 PM, Jeff Richards wrote:
> amounts. So I've employed a bucket called "unallocated" all the income
> goes in there and then comes out of there on a monthly basis to fill up the
> buckets.

I did the same (I called it "available") and found this to be a key
element of envelope budgeting. Especially when starting out, leaving
some funds in there makes a very useful buffer.

> In my perfect world, I would be able to see the balances of my bank account
> and credit card and have that equal the same number as my buckets.

Did you see the "Envelope budgeting with ordinary accounts" example
referenced in my 9/29 reply ? Contrary to some other tutorials out
there, the envelopes are virtual subaccounts of your real-world bank
account(s). That keeps everything adding up in an intuitive way.

[Simon Michael]

The hledger-budget script in master (bin/hledger-budget.hs) runs/builds
again and now includes all of its docs in the --help output. You may
want to give it another try.

I felt like I got more clarity around budgeting, but I didn't get around
to writing/posting.. maybe that's good for brevity!


1. I'm starting to think of the "goal-based" and "envelope" budgeting
styles as "expense(/income) budgeting" and "cash budgeting".


2. When I hear "budgeting" I have tended to think of a number of things:

- planning: setting goals for spending (and maybe earning) by category
and period. Usually repeated periodically.

- reviewing performance: showing actual activity (past and present)
compared to goals

- managing cashflow: ensuring I will not overspend available cash

- forecasting: estimating cash balance and performance with respect to
budget goals in the future

- controlling spending: making choices about spending in order to comply
with my budget goals

- adjusting: changing budget goals during a budget period, eg when doing
envelope budgeting and you unavoidably overspend in a category

- anything else ?

We can consider these separately.


3. In essence budgeting involves planning, measurement, and control.

Planning: you set goals before a budget period starts.

Measurement: you track performance, and how it compares to goals, as the
period proceeds.

Control: you try to maximize compliance with the goals in various ways:
- by controlling spending(/earning), to the extent that is possible
- by adjusting the budget (reallocating the goals or resizing the
overall budget), between periods or perhaps during a period

Measurement is often the first step, eg you might review past
performance to decide the first goals.

These are often repeated cyclically. This is not unlike the
observe-orient-decide-act loop: https://en.wikipedia.org/wiki/OODA_loop

[Simon Michael]

4. How budget goals are set

Some ways:
- keep 'em in your head
- write them down
- a column in a spreadsheet
- put limited amounts of cash in labelled envelopes (cash budgeting)
- virtual cash/envelope budgeting (model the above in any accounting
tool, one way or another)
- write "periodic transactions" in your journal file (expense budgeting
- Ledger, hledger-budget.hs)

I always found "periodic transactions" a bit troubling. They were
invisible to most commands (print, register), so I couldn't easily see
and troubleshoot the generated transactions. And I couldn't use them for
forecasting. And actually, they aren't transactions; they don't
represent a movement of funds. They're just a notation for specifying
recurring budget goals, reusing the transaction parser for expedience.

We might be able to invent a new notation for budget goals that more
clear, convenient, and expressive.

Separately, we could make the periodic transaction notation generate
actual transactions. This might be useful for forecasting or for
generating future journal entries.

[Simon Michael]

On 10/17/17 12:22 PM, Simon Michael wrote:
> 4. How budget goals are set

> ...

> We might be able to invent a new notation for budget goals that more
> clear, convenient, and expressive.

Oh yes. What I was dreaming about was something like

budget PERIODEXPR
ACCOUNT AMOUNT
...

reusing the command line period expressions,
http://hledger.org/manual.html#period-expressions . One or more of these
directives could appear anywhere in the journal. Eg:

budget monthly in 2017
expenses:food $300
expenses:rent $1200

# tweak budget for this month
budget 2017/10
expenses:food $400

It's actually a rather small change to Ledger's syntax. It reuses
existing syntax from the command line and should be quite flexible, eg
controlling the start/end date and budget interval is easy.
(Limited-time budgets used to be a pain point with Ledger's periodic
transactions, though I've just learned they have acquired new syntax for
that). And it makes clearer these are budget goals not transactions.

Allowing multiple overlapping directives as in the example above is
probably more complicated to implement/maintain/use than it looks.
Perhaps there are ways to simplify.

Just an idea, any interest ?

[Simon Michael]

On 10/17/17 11:49 AM, Simon Michael wrote:
> 2. When I hear "budgeting" I have tended to think of a number of things:
>
> - planning: setting goals for spending (and maybe earning) by category
> and period. Usually repeated periodically.
>
> - reviewing performance: showing actual activity (past and present)
> compared to goals
>
> - managing cashflow: ensuring I will not overspend available cash
>
> - forecasting: estimating cash balance and performance with respect to
> budget goals in the future
>
> - controlling spending: making choices about spending in order to comply
> with my budget goals
>
> - adjusting: changing budget goals during a budget period, eg when doing
> envelope budgeting and you unavoidably overspend in a category
>
> - anything else ?

- reviewing goals: showing the budget goals you have configured (past,
present, future)

[Stephen Christie]

That's actually what Ledger already has, only they use '~' instead of "budget". My ledge budget file is the following:

~ Monthly from 2016-08-28
    (Expenses:Housing)                         $800.00
    (Expenses:Transportation:Car)                   $0
    (Expenses:Giving:Tithe)                         $0
    (Assets)                                        $0
    (Liabilities)                                   $0

~ Every 6 months from 2017-01
    (Expenses:Education:Tuition)              $9,000.00

= /^Income:Taxable/
    (Expenses:Giving:Tithe)                     0.10

[Simon Michael]

Nice! As usual, corrections and updates to my Ledger knowledge are appreciated.

In hledger we might be simply renaming these to something like "budget goals", then. Perhaps adding a "budget" keyword as canonical form of "~".

You also have the = directive in your budget file. Another old hobby-horse of mine. Documented as "automated transactions", but more precisely it adds postings to existing transactions, so I try to call these "automated postings". I believe hledger's rewrite command as well as the hledger-budget addon make use of these. Easily mixed up with ~, so I wouldn't mind if we had a keyword form of this too - I think I suggested some (for adding/modifying/removing postings) on some issue page.
(Though, keywords increase language-centric-ness.)

You received this message because you are subscribed to the Google Groups "hledger" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hledger+u...@googlegroups.com.

[Simon Michael]

Brain dump continues. Slightly correcting myself:

Ledger's periodic transactions are dual-purpose. They specify budget goals which affect --budget reports. And they generate recurring transactions which appear in --forecast reports. This is explained in http://ledger-cli.org/3.0/doc/ledger3.html#Budgeting-and-Forecasting .

I don't know how how many of Ledger's commands support --budget and --forecast.

[Carel Fellinger]

We use periodic transactions a lot. Coming from GnuCash that might
not be a surprise. And the first thing I did when switching over was
getting a decent periodic transaction scheme working again. After
some fiddling we decided on a two fased approach: folded and unfolded.

Clearing a periodic transaction is troublesome, which "real"
transaction got cleared and so, but an unfolded one is actual a plain
transaction, so no problemo.

Instead of a plain iso date as transaction date we allow specifying an
iso repeat date, the rest of the transaction syntax remains the same,
except no status flags are allowed nor a transaction code.


e.g.
folded transaction file:
-----------------------
R5/2012-10-25/P1M Electra NEXT-MONTH
some postings

this periodic transaction gets treated as if the next input was found

folded transaction file:
-----------------------
2012-10-25/P1M Electra Nov
some postings

2012-11-25/P1M Electra Dec
some postings

2012-12-25/P1M Electra Jan
some postings

2013-01-25/P1M Electra Feb
some postings

2013-02-25/P1M Electra March
some postings


and unfolding it upto dec would yield:

unfolded transaction file:
-------------------------
2012-10-25 Electra Nov
some postings

2012-11-25 Electra Dec
some postings

2012-12-25 Electra Jan
some postings

folded transaction file:
-----------------------
R2/2013-01-25/P1M Electra NEXT-MONTH
some postings


The program deals with invalid months/periods by barking loudly unless
the required rounding method is specified:

R3/2012-01-31/P3M => 2012-01-31 2012-02-31 BARK BARK BARK

would bark at februari not having 31 days
but requesting rounding down gives us:

R3/2012-01-31/P3Mądown => 2012-01-31 2012-02-28 2012-03-31

and requesting rounding up gives:

R3/2012-01-31/P3Mąup => 2012-01-31 2012-03-01 2012-03-31


A repeat count of zero (R0/) is treated as commenting that whole
transaction, handy as starting point for the next bunch for that
particular periodec one.


Periodic transactions are often tentative, the amount might slightly
be off, or the date is not as predictable as it seemed and so on. But
in an automaticly unfolded an now real transaction such problems are
easily delt with.

This way we have all the power of normal transactions when we write a
periodic transaction, and we can easily adjust the individual
transactions thanks to the unfolding program/command.


Unfortunately, besides being written in Python, the program works on
our meta hledger variant. But it's a simple program so porting it to
hledger should be easy for someone versed in both.

--
groetjes, carel

[Simon Michael]

Thanks for mentioning ISO repeat dates. More to be discovered at
https://en.wikipedia.org/wiki/ISO_8601 .

What's your folded/unfolded files workflow ? Folded/concise format is
the temporary high-level one you write by hand, on phones, and that gets
expanded to unfolded/expanded format in the permanent main journal ?

[Carel Fellinger]

On Wed, Oct 18, 2017 at 04:39:39PM -0700, Simon Michael wrote:
> What's your folded/unfolded files workflow ? Folded/concise format is the
> temporary high-level one you write by hand, on phones, and that gets
> expanded to unfolded/expanded format in the permanent main journal ?

On the phones normally just the day to day expences are noted in a
plain Notes file. If required photos can be added. Each notes file
should have a header stating the period the file is ment to cover.

This file is often and repeatedly send to our local mail host,
filtered into a special imap folder, and on request entered into our
subversion finance archive keeping track of "upstream" (i.e. on the
phone) vs "local" (i.e. in our sandbox) changes.

This works like a charm, finally everybody enters their day to day
expences, at least a few times per week on the train, in the bus or
comfortably on the couch. No need to fire-up a computer, no special
program to understand. The biggest problem of accounting solved :)

The only time they tent to forget to enter a transaction is when they
buy online, but that's improving as wel.

Then we each have a "buro" folder in our privat finance archive to
keep our "folded" journals with repeat transactions for rent, electra
bills, insurance and such. Those journals are always included and
automatically unfolded in our hledger runs.

Whenever we reconcile, we explicitly unfold those transactions to
allow us to mark postings as reconciled and to adjust to the bank's
posting dates. Unfolding is an extra, but automated step, and it
keeps the books straight.

The whole process is configurable via an INI file with sections like:

[general]
includes = # journals to be fed to hledger
we.mlg # previously in gnucash
audit/**/*.mlg # needed to reconciliate
buro/**/*.mlg # local/generic/folded
mobile/**/*.mlg # remote/sent journals

[path patterns]
mailed = Mail/{me}/{year}-{month:02}/{name}-{month.long}
merged = mobile/{year}-{month:02}/{me}/{name}-{month.long}
folded = buro/{name}-folded.mlg
unfolded = buro/{year}/{name}-unfolded.mlg


There is nothing prohibiting mailed journals to contain repeat
transactions, they get unfolded automatically, but there is no
provision to help you to explicitly unfold them, unless included
in the folded path pattern.

Bytheway: having repeat transactions automatically unfolde before
handing them over to hledger simplifies things but for the error
reporting. To help there I add a comment line to each automatically
unfolded transaction indicating which repeat transaction is to blame.

--
groetjes, carel

[Simon Michael]

Very interesting, thanks!

[Edward Amsden]

Just wanted to pop back in and say I've added a PR for the one bit of functionality I was missing from my now-obsoleted bespoke budgeting tool. 

https://github.com/simonmichael/hledger/pull/638

It lets you see, within the given period, what the minimum future balance is for each posting, though for readability I added a `groupBy` call to eliminate long strings of the same minimum future balance. This is very handy for seeing if you can afford to make a purchase or otherwise add things into your budget. Example usage in the PR

On Thursday, October 19, 2017 at 2:25:53 PM UTC-4, Simon Michael (sm) wrote:

Very interesting, thanks!

[Simon Michael]

On 10/31/17 3:25 PM, Edward Amsden wrote:
> Just wanted to pop back in and say I've added a PR for the one bit of
> functionality I was missing from my now-obsoleted bespoke budgeting tool.
>
> https://github.com/simonmichael/hledger/pull/638
>
> It lets you see, within the given period, what the minimum future balance
> is for each posting, though for readability I added a `groupBy` call to
> eliminate long strings of the same minimum future balance. This is very
> handy for seeing if you can afford to make a purchase or otherwise add
> things into your budget. Example usage in the PR

Cool. I have yet to process the PR, but I see the example command:

|$ hledger register -p "from yesterday to 2018/12/31" -f "budget.journal"
--available assets:checking|

I'll share a general tip here, which perhaps should go in docs. I don't
really know what the above does, without spending time downloading and
testing and thinking. This happens very often and slows things down. In
such cases if you can show a complete minimal reproducible example, ie
the input file(s) and a transcript of the command(s) run and output
seen, we will understand much faster, there'll be more discussion, and
quick code review/merging is much more likely.

[Edward Amsden]

I'm planning to add that to the PR next time I get a couple of hours free for open source work :)

--
You received this message because you are subscribed to a topic in the Google Groups "hledger" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hledger/jI8FlZBvMkA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to hledger+u...@googlegroups.com.

[Simon Michael]

Yay. :) Or I may get to it sooner.

I will add to that tip. "This applies double for not-yet-implemented things. Mockups of imaginary syntax, usage and output are extremely effective."

You received this message because you are subscribed to the Google Groups "hledger" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hledger+u...@googlegroups.com.

[Simon Michael]

Also highly effective: drafts of help and docs..

On 11/4/17 8:22 AM, Simon Michael wrote:
> I will add to that tip. "This applies double for not-yet-implemented things. Mockups of imaginary syntax, usage and output are extremely effective."

[Edward Amsden]

I've added a blurb about --available including a full example to the manpage, but I'm having trouble getting Shake.hs up and running to make sure I did it right. I'm of a mind to just commit it and let you (SM) try it (if you have a working Shake/pandoc), and squash in any commits needed to fix it.

[Simon Michael]

On 11/4/17 6:17 PM, Edward Amsden wrote:
> I've added a blurb about --available including a full example to the
> manpage, but I'm having trouble getting Shake.hs up and running to make
> sure I did it right. I'm of a mind to just commit it and let you (SM) try
> it (if you have a working Shake/pandoc), and squash in any commits needed
> to fix it.

Feel free to share blurbs here or on issues too! It doesn't have to be
all rendered and finalised.

What's going wrong with the shakefile ? In theory, "./Shake.hs" is
supposed to compile it.