Cloning lexer to include statements.

[lb_40]

Hi,

i'm trying to develop a parser for the Kconfig language used to define
the config option of the linux kernel.
In this language, you've got a source statement which is like an
include statement.

In the doc, it seems that clone() is the right choice for me.
So i've got this piece of work... Basically, what i do is that i've an
exclusive state for 'source'
When i'm entering this state the next path is going to be considered
as a path to a file i must include.

def t_begin_source(t):
r'source '
t.lexer.begin('source')

def t_source_path(t):
r'[^\n]+\n+'
t.lexer.begin('INITIAL')
global path
source_lexer = lexer.clone()
source_file_name = (path + t.value.strip(' \"\n'))
sourced_file = file(path + t.value.strip(' \"\n')).read()

source_lexer.input(sourced_file)
while True:
tok = source_lexer.token()
if not tok:
break
print "VALUE ", tok

The problem that I have is that i don't really understand how the
parser and the lexer interact and so even if the lexer tokenizes
everything I don't return anything from this t_source_path function,
so the parser doesn't see any of these sourced tokens.


thanks for the help.