simple templating system

[Alessandro]

Hi all,
I'm programming a simple templating system. The simplest expression
I have to manage is:
" bla bla bla .. $FUNC ( bla bla bla.. ) bla bla.."

Here my code:

tokens = ( 'OTHER', 'FUNC' )

def t_FUNC(t):
r'\$FUNC'
return t

def t_OTHER(t):
r'[^ $]+'
return t

t_ignore = " \t"

def t_error(t):
print("Illegal character '%s'" % t.value[0])
t.lexer.skip(1)

import ply.lex as lex
lex.lex()

def p_expression(p):
'''expression : OTHER expression
| func expression
| empty'''
pass

def p_func(p):
"func : FUNC '(' expression ')'"

def p_empty(p):
'empty :'
pass


def p_error(p):
if p:
print("Syntax error at '%s'" % p.value)
else:
print("Syntax error at EOF")

import ply.yacc as yacc
yacc.yacc(debug=1)

yacc.parse(' bla bla $FUNC( bli bli ) ')


But I get the "Syntax error at '(' " error. Probably the "(" is
interpreted as "OTHER" and not as "func".
How can I solve?
I have the same problem with "FUNC"; it must start with "$"; if I
remove it (or if I put it into "OTHER" definition) the "FUNC" word is
interpreted as a "OTHER" token.

It's a precedence issue?

Thanks
Alessandro

[D.Hendriks (Dennis)]

Hello Alessandro,

This rule:

> def t_OTHER(t):
> r'[^ $]+'
> return t

is a negative rule. It matches way to much. You should probably specify
it in a positive way. I recommend introducing a t_ID to scan identifiers.

Also, take a look at http://www.dabeaz.com/ply/ply.html (sections 4.3
and 4.4). This explains how to do lexing of identifiers vs keywords.

Best,
Dennis

> --
>
> You received this message because you are subscribed to the Google Groups "ply-hack" group.
> To post to this group, send email to ply-...@googlegroups.com.
> To unsubscribe from this group, send email to ply-hack+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/ply-hack?hl=en.
>
>

[Alessandro Tufi]

Subject: Re: simple templating system
From: D.Hendriks (Dennis) <D.Hen...@tue.nl>
To: ply-...@googlegroups.com <ply-...@googlegroups.com>
Date: Tue Nov 24 2009 08:09:23 GMT+0100 (CET)

> Hello Alessandro,
>
> This rule:
>
> > def t_OTHER(t):
> > r'[^ $]+'
> > return t
>
> is a negative rule. It matches way to much. You should probably specify
> it in a positive way. I recommend introducing a t_ID to scan identifiers.
>
> Also, take a look at http://www.dabeaz.com/ply/ply.html (sections 4.3
> and 4.4). This explains how to do lexing of identifiers vs keywords.
>
> Best,
> Dennis
>

Thanks Dennis.

I solved with this code:

....
tokens = (
'OTHER', 'FUNCTION'
)

literals = [ '(',')' ]

def t_FUNCTION(t):
r'\$FUNCTION'
return t

def t_OTHER(t):
r'[^ $\(\)]+'

return t

t_ignore = " \t"

...

def p_expression(p):
'''expression : function expression
| OTHER expression
| parexpression expression
| empty'''
p[0] = ' '.join([str(x) for x in p[1:] ])

def p_parexpression(p):
'''parexpression : '(' expression ')' '''
p[0] = "( %s )" % p[2]

def p_function(p):
"function : FUNCTION parexpression "
p[0]= '<%s>' % p[1] + p[2] # some modifs.

def p_empty(p):
'empty :'
p[0] = ''
.....


I think I must use the negative rule because I want a template system.
Now I understand why most templating language are using particular
tokens to go into a "second state" where you can use the templating
language. For example:

bla bla bla
${
if ..
for...
..
}$
bla bla

The "${" and the "$}" simplify the parsing very much!

Alessandro