New feature: pre-declaring accounts, payees, commodities and tags
John Wiegley
This feature comes with some breaking changes, even damaging backwards
compatibility with 2.x. I believe it is worth it for the gain in consistency.
------------------------------------------------------------------------------
# BREAKING CHANGES
## 'account' directive
In 2.x, you could enclose a group of transactions within a parent account:
account My Master
...
end account
This is now done with "apply account" instead of "account":
apply account My Master
...
end apply
You can also use "end apply account", and Ledger will verify that it matches
an enclosing "apply account".
## 'tag' directive
In 3.x, you could apply a tag to a group of transactions:
tag Foo: Bar
...
end tag
This is now done with "apply tag" instead of "tag":
apply tag Foo: Bar
...
end apply tag
------------------------------------------------------------------------------
# NEW DIRECTIVES
There are four all new directives:
account
payee
commodity
tag
## New 'account' directive
You can now pre-declare account names. This only has effect if --strict or
--pedantic is used (see below).
account Expenses:Food
account Expenses:Gas
### Account sub-directives
The 'account' directive supports several optional sub-directives, if they
immediately follow the account directive and if they begin with whitespace:
account Expenses:Food
note This account is all about the chicken!
alias food
payee ^(KFC|Popeyes)$
check commodity == "$"
assert commodity == "$"
eval print("Hello!")
default
The 'note' sub-directive associates a textual note with the account. This can
be accessed later using the 'note' valexpr function in any account context.
The 'alias' sub-directive, which can occur multiple times, allows the alias to
be used in place of the full account name anywhere that account names are
allowed.
The 'payee' sub-directive, which can occur multiple times, provides regexps
that identify the account if that payee is encountered and an account within
its transaction ends in the name "Unknown". Example:
2012-02-27 KFC
Expenses:Unknown $10.00 ; Read now as "Expenses:Food"
Assets:Cash
The 'check' and 'assert' directives warn or error (respectively) if the given
value expression evaluates to false within the context of any posting.
The 'eval' directive evaluates the value expression in the context of the
account at the time of definition. At the moment this has little value.
The 'default' directive specifies that this account should be used as the
"balancing account" for any future transactions that contain only a single
posting.
## New 'payee' directive
You can now pre-declare payee names. This only has effect if --check-payees
is used in addition to --strict or --pedantic (see below).
payee KFC
payee Payless
### Payee sub-directives
The 'payee' directive supports one optional sub-directive, if it immediately
follows the payee directive and if it begins with whitespace:
payee KFC
alias KENTUCKY FRIED CHICKEN
The 'alias' directive provides a regexp which, if it matches a parsed payee,
the declared payee name is substituted:
2012-02-27 KENTUCKY FRIED CHICKEN ; will be read as being 'KFC'
...
## New 'commodity' directive
You can now pre-declare commodity names. This only has effect if --strict or
--pedantic is used (see below).
commodity $
commodity CAD
### Commodity sub-directives
The 'commodity' directive supports several optional sub-directives, if they
immediately follow the commodity directive and if they begin with whitespace:
commodity $
note American Dollars
format $1,000.00
nomarket
default
The 'note' sub-directive associates a textual note with the commodity. At
present this has no value other than documentation.
The 'format' directive gives you a way to tell Ledger how to format this
commodity. In future using this directive will disable Ledger's observation
of other ways that commodity is used, and will provide the "canonical"
representation.
The 'nomarket' directive states that the commodity's price should never be
auto-downloaded.
The 'default' directive marks this as the "default" commodity, in contexts
where that applies (the same as the current 'D' directive).
## New 'tag' directive
You can now pre-declare tag names. This only has effect if --strict or
--pedantic is used (see below).
tag Receipt
tag CSV
### Tag sub-directives
The 'tag' directive supports two optional sub-directives, if they immediately
follow the tag directive and if they begin with whitespace:
tag Receipt
check value =~ /pattern/
assert value != "foobar"
The 'check' and 'assert' directives warn or error (respectively) if the given
value expression evaluates to false within the context of any use of the
related tag. In such a context, "value" is bound to the value of the tag
(which may not be a string if typed-metadata is used!). Such checks or
assertions are not called if no value is given.
------------------------------------------------------------------------------
# "KNOWN" ENTITIES
Normally, an account/payee/commodity/tag is considered "known" to Ledger if
it:
1. Occurs within a predeclaration, using the directives above.
2. Occurs within a cleared or pending posting or transaction.
Otherwise, if the account/payee/commodity/tag is first encountered in an
uncleared posting or transaction, it is considered "unknown".
By default, this distinction has no meaning whatsoever, and is not maintained
for the sake of speed. It only has meaning if --strict or --pedantic is used
(see next section).
------------------------------------------------------------------------------
# NEW OPTIONS
## --explicit
When --explicit is given, *only* predeclarations establish the "known-ness" of
parsed entities. I.e., if you didn't predeclare it, you don't expect to ever
see it.
## --strict
With --strict, referring to unknown entities causes a warning.
## --pedantic
With --pedantic, referring to unknown entities causes an error.
## --check-payees
By default, even with --strict or --pedantic, payees are not checked for
known-ness because it is quite typical that new payees are used in uncleared
transactions, or without declaring them. The --check-payees option enables
--strict and --pedantic checking for payees as well.
------------------------------------------------------------------------------
# EXAMPLE
Here's a real example from the baseline tests. Remember that options can be
specified directly within your Ledger file!
--explicit
--pedantic
commodity $
format $1,000.00
account Assets:Cash
assert abs(amount) <= 20
check commodity == '$'
account Expenses:Food
alias food
payee KFC
2012-02-27 KFC
Expenses:Unknown $20.00
Assets:Cash
2012-02-28 KFC
food $20.00
Assets:Cash
test reg --strict
12-Feb-27 KFC Expenses:Food $20.00 $20.00
Assets:Cash $-20.00 0
12-Feb-28 KFC Expenses:Food $20.00 $20.00
Assets:Cash $-20.00 0
end test
Craig Earls
Craig, Corona De Tucson, AZ
enderw88.wordpress.com
Gabriel Kerneis
thanks for this new feature!
On Mon, Feb 27, 2012 at 12:39:26PM -0600, John Wiegley wrote:
> The 'payee' sub-directive, which can occur multiple times, provides regexps
> that identify the account if that payee is encountered and an account within
> its transaction ends in the name "Unknown".
I don't like when ledger uses hard-coded (English) strings. IIRC, there are a
few other places in ledger's code that do the same (something about equity?).
I don't have better ideas (except maybe provide a regexp explicitly as a
second parameter to payee). And I won't fight against it, since I don't plan
to use this sub-feature. But I feel uneasy about such behaviour. Especially
since I don't use English in my ledger files.
Best regards,
--
Gabriel
Johann Klähn
> I don't like when ledger uses hard-coded (English) strings. IIRC, there are a
> few other places in ledger's code that do the same (something about equity?).
>
> I don't have better ideas (except maybe provide a regexp explicitly as a
> second parameter to payee). And I won't fight against it, since I don't plan
> to use this sub-feature. But I feel uneasy about such behaviour. Especially
> since I don't use English in my ledger files.
When 3.0 is released ledger can be translated using gettext. (You
could even do that for those and other strings now. I don't think that
much will change until 3.0. One could end up doing that work twice.)
But given how DIY and modular ledger is in other regards it would be
nice to have further configurability here too. I don't think that
there is _one_ naming scheme everyone agrees on and it's too much work
to change the localization files just for account names.
Gabriel Kerneis
> When 3.0 is released ledger can be translated using gettext.
I am aware of gettext but I certainly don't want my ledger to rely on
my locale to work. It would be even worse than to stick some "Unknown"
accounts in my non-english file, IMO.
--
Gabriel
Johann Klähn
with portability. With the current setup a ledger using english
account names would not work correctly with the locale set to some
other language.
John Wiegley
> I don't like when ledger uses hard-coded (English) strings. IIRC, there are
> a few other places in ledger's code that do the same (something about
> equity?).
Ledger is 100% localized, using the gettext library. You can provide a
translation file during the build process in order to switch languages at
runtime based on the current locale. That's why I haven't done more to make
these strings configurable.
That said, perhaps there should be an easier way to allow users to define
important strings like this. Maybe a series of "--string-XXX" options?
John
John Wiegley
Ok, that does it, I will create configuration options for significant strings.
John
Martin Michlmayr
> Normally, an account/payee/commodity/tag is considered "known" to Ledger if
> it:
>
> 1. Occurs within a predeclaration, using the directives above.
> 2. Occurs within a cleared or pending posting or transaction.
> ## --explicit
>
> When --explicit is given, *only* predeclarations establish the "known-ness" of
> parsed entities.
I interpreted your "*only*" to mean that this would also apply to
cleared transactions. This is not the case currently but I'd find
this behaviour useful.
What do you think?
--
Martin Michlmayr
http://www.cyrius.com/
John Wiegley
> I interpreted your "*only*" to mean that this would also apply to cleared
> transactions.
--explicit means that knownness is *only* from pre-declarations, not from
cleared transactions.
John
Martin Michlmayr
> > I interpreted your "*only*" to mean that this would also apply to cleared
> > transactions.
>
> --explicit means that knownness is *only* from pre-declarations, not from
> cleared transactions.
Right, this is what I expected based on your email, but this is not
what ledger actually does afaict.
John Wiegley
>> --explicit means that knownness is *only* from pre-declarations, not from
>> cleared transactions.
> Right, this is what I expected based on your email, but this is not what
> ledger actually does afaict.
That would be a bug then!
John
Russell Adams
issue with error reporting.
When I had a metadata tag that wasn't defined, I got a clear error
that recursed into my include directives:
Error: Unknown metadata tag 'OWNER'
In file included from ".ledger", line 106:
While parsing file "./ER/AISER0044.dat", line 72:
However when a value failed its check, instead I got just an error on
the line of the include directive.
Warning: ".ledger", line 113: Metadata check failed for (RECEIPT: Y):
(value =~ /^Receipts/.*.jpg/)
Can this be expanded to include additional detail?
Thanks.
On Mon, Feb 27, 2012 at 12:39:26PM -0600, John Wiegley wrote:
Russell Adams RLA...@AdamsInfoServ.com
PGP Key ID: 0x1160DCB3 http://www.adamsinfoserv.com/
Fingerprint: 1723 D8CA 4280 1EC9 557F 66E8 1154 E018 1160 DCB3
Russell Adams
> >
> > The 'tag' directive supports two optional sub-directives, if they immediately
> > follow the tag directive and if they begin with whitespace:
> >
> > tag Receipt
> > check value =~ /pattern/
> > assert value != "foobar"
What would be the best way to compare against a long but finite list
of acceptable values? Obviously /ONE|TWO|THREE.../ overflows.
Thanks.
Russell Adams
> So I'm trying out the new explicit features, and ran into a minor
> issue with error reporting.
>
> When I had a metadata tag that wasn't defined, I got a clear error
> that recursed into my include directives:
>
> Error: Unknown metadata tag 'OWNER'
> In file included from ".ledger", line 106:
> While parsing file "./ER/AISER0044.dat", line 72:
>
>
> However when a value failed its check, instead I got just an error on
> the line of the include directive.
>
> Warning: ".ledger", line 113: Metadata check failed for (RECEIPT: Y):
> (value =~ /^Receipts/.*.jpg/)
>
> Can this be expanded to include additional detail?
errors are great!
Thanks.