Non standard flags on transactions/postings

[shreedha...@gmail.com]

Hi,

I wanted to flag transactions (and postings) with a flag that is not one of the standard *, !, txn. The documentation in "Beancount - Language Syntax" suggested that I should be able to use any character for transactions & postings (and perhaps even a string for the transaction).

However, for the example below, bean-check complains with "Invalid token: 'D'".

2017-01-01 open Income:Salary USD

2017-01-01 open Assets:Bank USD

2017-01-02 D "Paycheck"

  Income:Salary -10 USD

  Assets:Bank  10 USD

It is, however, happy with the characters M or T. 

I was trying to reason why that is the case. Looking at the parsing grammar in grammar.y, I thought any character should be ok. TXN, ASTERISK and HASH are handled specially, but the FLAG option might reference any character : 

%token <character> FLAG    /* Valid characters for flags */

But then there is code in lexer.l that restricts the characters to [!&#?%PSTCURM]; and some python macros for them in flags.py.

Is there some underlying reason for this restriction? How can I go about flagging transactions in a custom way? Using metadata seems like one way (although, I did hope that there was a way to use custom flags) 

Thanks,

Shree

PS: I should also mention that I'm not very familiar with YACC - so I might also be missing something.

[Martin Blais]

On Thu, Aug 9, 2018 at 1:51 PM <shreedha...@gmail.com> wrote:

Hi,

I wanted to flag transactions (and postings) with a flag that is not one of the standard *, !, txn. The documentation in "Beancount - Language Syntax" suggested that I should be able to use any character for transactions & postings (and perhaps even a string for the transaction).

However, for the example below, bean-check complains with "Invalid token: 'D'".

2017-01-01 open Income:Salary USD

2017-01-01 open Assets:Bank USD

2017-01-02 D "Paycheck"

  Income:Salary -10 USD

  Assets:Bank  10 USD

It is, however, happy with the characters M or T. 

I was trying to reason why that is the case. Looking at the parsing grammar in grammar.y, I thought any character should be ok. TXN, ASTERISK and HASH are handled specially, but the FLAG option might reference any character : 

%token <character> FLAG    /* Valid characters for flags */

But then there is code in lexer.l that restricts the characters to [!&#?%PSTCURM]; and some python macros for them in flags.py.

Is there some underlying reason for this restriction?

Limitations on the lexer.

At some point I intend to rewrite it as a custom lexer instead of using flex.

Hasn't been a huge priority.

Agreed it should be documented better (feel free to suggest in the doc, I'll accept).

 

How can I go about flagging transactions in a custom way? Using metadata seems like one way (although, I did hope that there was a way to use custom flags) 

Yes, that, or tags.

 

Thanks,

Shree

PS: I should also mention that I'm not very familiar with YACC - so I might also be missing something.

No you're right, it's defined that way.

 

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/14ac7f0c-b4eb-481d-8152-9db28ad04f79%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[shreedha...@gmail.com]

Limitations on the lexer.

At some point I intend to rewrite it as a custom lexer instead of using flex.

Hmm.. so you're seeing other pains with it? I believe YACC (bison) can do some basic lexical analysis. I haven't gone through all of lexer.l to know the complexity myself. 

Hasn't been a huge priority.

Agreed it should be documented better (feel free to suggest in the doc, I'll accept).

Sure, I can do that.

 

 

How can I go about flagging transactions in a custom way? Using metadata seems like one way (although, I did hope that there was a way to use custom flags) 

Yes, that, or tags.

Would it work if the relevant line in lexer.l is changed to [!&#?%A-Za-z]? The letters allowed would then be a superset of those defined as macros in core.flags.py. Also, do you have a sense of how much testing that would require? If it's not much, I can probably create a PR with the changes and tests (in lexer_test.py and/or parser_test.py), if you're ok going that route...

[Martin Blais]

On Thu, Aug 9, 2018 at 7:28 PM <shreedha...@gmail.com> wrote:

Limitations on the lexer.

At some point I intend to rewrite it as a custom lexer instead of using flex.

Hmm.. so you're seeing other pains with it?

Irregular use of whitespace based on context.

Or maybe I wasn't able to wrangle it properly (not an expert).

 

I believe YACC (bison) can do some basic lexical analysis. I haven't gone through all of lexer.l to know the complexity myself. 

It's the lexer that's at issue, the parser generator works fine.

If I wrote my own lexer I could handle the special case of whitespace and do some lookahead.

I will, eventually (it's not difficult).

Hasn't been a huge priority.

Agreed it should be documented better (feel free to suggest in the doc, I'll accept).

Sure, I can do that.

 

 

How can I go about flagging transactions in a custom way? Using metadata seems like one way (although, I did hope that there was a way to use custom flags) 

Yes, that, or tags.

Would it work if the relevant line in lexer.l is changed to [!&#?%A-Za-z]?

I'm pretty sure that's the first thing I wrote, so already tried.

 

The letters allowed would then be a superset of those defined as macros in core.flags.py. Also, do you have a sense of how much testing that would require? If it's not much, I can probably create a PR with the changes and tests (in lexer_test.py and/or parser_test.py), if you're ok going that route...

 

Thanks,

Shree

PS: I should also mention that I'm not very familiar with YACC - so I might also be missing something.

No you're right, it's defined that way.

 

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/14ac7f0c-b4eb-481d-8152-9db28ad04f79%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/9d833fe5-546e-4076-9b7d-1006da24f0af%40googlegroups.com.