No handlers could be found

[Magus]

Hello Andrew,

I am getting the following run-time parsing error with both 2.4 and
3.1 versions.

No handlers could be found for logger "lepl.memo.PerCallCache"

What does that mean? Can give me a direction where to look please? :)

Thank you,
Nick

[andrew cooke]

Hi,

That's just Python's logging library saying that it's not configured.
You can safely ignore it - you just won't see any log messages.

A simple way to configure this from within Python, so that you only
see error messages, is to include the code:

from logging import basicConfig, ERROR
basicConfig(level=ERROR)

For more info on the Python logging library, see
http://docs.python.org/library/logging.html

If your program is crashing you can enable more logging with, for example:

from logging import basicConfig, DEBUG
basicConfig(level=DEBUG)

Thanks for raising this - I should add some details in the
documentation as I know that, despite it being a standard library, few
people using logging.

Andrew


2009/9/1 Magus <nbm...@gmail.com>:

[andrew cooke]

Also, you meant 2.6, right? There's no way LEPL is going to work
under 2.4 (BUT having said that, a 3to2 tool has just been released -
I hope to make a beta release of LEPL with offside rule handling this
month and will then try that tool. If it works, then I'll make a
release of LEPL for Python 2).

[Magus]

Thanks, Andrew.

No, I mean Lepl 2.4 and 3.1. I am using Python 2.6 for both.

Enabled level=DEBUG. Now I am getting

DEBUG:lepl.context.Global:Getting operators/<class
'lepl.operators.DefaultNamespace'>
INFO:lepl.lexer.rewriters.lexer_rewriter:Lexer rewriter used, but no
tokens found.

WARNING:lepl.memo.PerCallCache:A view completed before the cache was
complete: Or(Line(u'sys.func()')[0:]). This typically means that the
grammar contains a matcher that does not consume input within a loop
and is usually an error.

It looks like an infinite loop. I guess I screwed up my grammar with
cross references, right?

Thanks,
Nick

[andrew cooke]

2009/9/1 Magus <nbm...@gmail.com>:

>
> Thanks, Andrew.
>
> No, I mean Lepl 2.4 and 3.1. I am using Python 2.6 for both.

Ah OK :o)

> Enabled level=DEBUG. Now I am getting
>
> DEBUG:lepl.context.Global:Getting operators/<class
> 'lepl.operators.DefaultNamespace'>
> INFO:lepl.lexer.rewriters.lexer_rewriter:Lexer rewriter used, but no
> tokens found.

These can be ignored.

> WARNING:lepl.memo.PerCallCache:A view completed before the cache was
> complete: Or(Line(u'sys.func()')[0:]). This typically means that the
> grammar contains a matcher that does not consume input within a loop
> and is usually an error.
>
> It looks like an infinite loop. I guess I screwed up my grammar with
> cross references, right?

Oh, that's a nasty one. Yes, the most likely explanation is that your
grammar is wrong. For example, something like this might trigger that
error:

a = Delayed()
b = a | Word()
a += b | Integer()

Because trying to match a will try to match b, which will try to match
a, all without consuming any input.

Looking at the error in more detail I would be suspicious of the [0:]
you have there. If that's inside a loop then you will get the kind of
problem I describe because each time it is called it will first try to
match zero instances (ie match nothing). You might need to change it
to [1:] and then make the whole loop optional (for example).

So, expanding on that, this will fail:

a = Delayed()
a += Foo()[0:] & a
b = a & ....

Because it will match zero Foo's and then call a, which will match
zero Foo's and call a which... You could avoid that with, for exaple:

a = Delayed()
a += Foo()[1:] & a
b = Optional(a)

where the Optional() handles the case of matching nothing, making the
loop always consume something.

It's also possible that there's a bug in the code, but to be convinced
of that I'd need to see a grammar that's compact, easily
understandable, and still fails.

So I'd suggest trying to isolate the few rules in your grammar that
are causing a problem, and simplifying them. You will hopefully find
the loop as above or end up with code that obviously should work, in
which case I'll (try to) fix the bug...

Good luck,
Andrew

PS Similar topics are covered in the tutorial - see
http://www.acooke.org/lepl/intro-4.html#ambiguity-and-left-recursion

PPS It seems to me I should be able to handle this automatically and
simply "break the loop". I will look at that sometime in the future,
but it won;t be for some time.

[Magus]

Thanks, again. I will isolate the code trying to figure out what is
wrong with the grammar.

Cheers,
Nick

On Sep 1, 8:37 pm, andrew cooke <acooke....@gmail.com> wrote:
> 2009/9/1 Magus <nbma...@gmail.com>:

> PS Similar topics are covered in the tutorial - seehttp://www.acooke.org/lepl/intro-4.html#ambiguity-and-left-recursion

[Magus]

Hi Andrew,

I tried to isolate the problem. Now if I run the following code

from lepl import *

w = Word(Letter())
d = Literal('.')

b = Delayed()
c = Delayed()

a = b > 'a'
b += c | a > 'b'
c += w | b & d & w > 'c'

line = a & Eos()

parser = line.string_parser()
result = parser("sys.func ")
print result

1. I am getting infinite loop here with

WARNING:lepl.memo.PerCallCache:A view completed before the cache was
complete: O

r(Line(u'sys.func ')[0:]). This typically means that the grammar

contains a matc
her that does not consume input within a loop and is usually an error.

Please note the last space. The same result with any other symbol
(like slash for example) instead of the space.

2. If I remove aliases ('a', 'b', 'c')?! - the loop stopped quickly
with the same WARNING.

3. If I remove the last space it is parsed OK.

I hope you can say if the extract of the grammar is OK.

Thanks a lot,
Nick

[Magus]

Hi Andrew,

If I am running the following piece of code it crashed with a
different error. The grammar however is more similar with what I am
really using. I am saying this because a = b may look stupid to you.
It is however a bit difficult to extract a part of the grammar
retaining the error. :)

from lepl import *

optional_spaces = Drop(Whitespace()[0:])

w = Word(Letter())
d = Literal('.')

x = Literal('()')

b = Delayed()
c = Delayed()

a = b & optional_spaces & x > 'a'

b += c | a > 'b'

c += w | b & optional_spaces & d & optional_spaces & w > 'c'

line = a & Eos()

parser = line.string_parser()
result = parser("sys.func")
print result

No handlers could be found for logger "lepl.parser.trampoline"
Traceback (most recent call last):
File "C:\Code\LEPL\test5.py", line 19, in <module>

result = parser("sys.func")

File "C:\Python26\lib\site-packages\lepl\parser.py", line 249, in
single
return next(matcher(arg))[0]
File "C:\Python26\lib\site-packages\lepl\parser.py", line 182, in
trampoline
raise value
AttributeError: 'NoneType' object has no attribute '_match'

I really appreciate your help. Thanks.

[andrew cooke]

Note that the error has changed! This is a different problem -
somehow you've got a None value being treated as a matcher (the _match
method is what matchers must implement).

Is debug logging still enabled? That often gives a better error
message than the one you get from trampoline.

The first thing I would do here is put parentheses around various
expressions to be sure that the binding is what you expect. So try:

from lepl import *

w = Word(Letter())

b = Delayed()
c = Delayed()

a = (b / '()') > 'a'
b += (c | a) > 'b'
c += (w | b / '.' / w) > 'c'

(i've also replaced your optional_spaces with / and put the literals in-line)

Also, apart from this new error, you have a problem with the following:
what happens if "c" is matched against something that is not a "w"?

"c" will try to match "w", fail, and try "b", which means it will try
"c", which means it will try "w", fail, and try "b", which means...

At the very least, you want:

b += (a|c) > 'b'

but I doubt that will be enough, because you have the same problem if "a" fails.

When you get in a muddle (no offense) like this, the best thing to do
is sit down and start again from scratch. You have something
recursive, that looks a bit like a dotted path. You need to rethink
how that is going to work so that the recursion is "neat". I don't
know how to describe it better and am in a bit of a rush right now
(bought a new CPU and am getting memory errors; the local computer
shop closes soon....)

I will look at this again later if I have time, but if you can provide
a description of what a, b and c are - preferably a single word or
phrase like "function" or "path element" or "dotted path" - you'll
help me and probably yourself, because I suspect once those are clear,
the correct way to do this will be easy.

In summary I suggest:
1 - enable debug logging and check back to see if a better error
scrolled off the screen
2 - simplify more and add parentheses
3 - provide names for the a, b, c

Andrew


2009/9/1 Magus <nbm...@gmail.com>:

[Magus]

Hi Andrew,

Thanks again. Indeed the grammar is for dotted names.

With this code

from lepl import *

from logging import basicConfig, DEBUG
basicConfig(level=DEBUG)

name = Word(Letter()) #> 'name'

expression = Delayed()
variable = Delayed()

function = (expression / '()') > 'function'
expression += (variable | function) #> 'expression'
variable += (name | expression / '.' / name) #> 'variable'

dotted_name = function & Eos()

parser = dotted_name.string_parser()
result = parser("func")
print result

I am getting

WARNING:lepl.parser.trampoline:Exception at epoch 6
WARNING:lepl.parser.trampoline:Top of stack: Transform(Apply)(Line
(u'func')[0:])

WARNING:lepl.parser.trampoline:Traceback (most recent call last):
File "C:\Python26\lib\site-packages\lepl\parser.py", line 147, in
trampoline
value = next(value)
File "C:\Python26\lib\site-packages\lepl\parser.py", line 69, in
next
return self.__next__()
File "C:\Python26\lib\site-packages\lepl\parser.py", line 63, in
__next__
return next(self.__generator)
File "C:\Python26\lib\site-packages\lepl\matchers.py", line 767, in
_match
generator = self.matcher._match(stream_in)

AttributeError: 'NoneType' object has no attribute '_match'

WARNING:lepl.parser.trampoline:Stack: RMemo(And)
WARNING:lepl.parser.trampoline:Stack: RTable(And)
WARNING:lepl.parser.trampoline:Stack: And
WARNING:lepl.parser.trampoline:Stack: RMemo(Transform(Apply))
WARNING:lepl.parser.trampoline:Stack: RTable(Transform(Apply))
WARNING:lepl.parser.trampoline:Stack: Transform(Apply)

Traceback (most recent call last):

File "C:\Code\LEPL\test7.py", line 18, in <module>
result = parser("func")

File "C:\Python26\lib\site-packages\lepl\parser.py", line 249, in
single
return next(matcher(arg))[0]
File "C:\Python26\lib\site-packages\lepl\parser.py", line 182, in
trampoline
raise value
AttributeError: 'NoneType' object has no attribute '_match'

Thanks,
Nick

> 2009/9/1 Magus <nbma...@gmail.com>:

[andrew cooke]

OK, so there's ceratinly a bug in my code - during the rewrite to add
the memoizers it's losing almost all your matchers. Sorry. :o(

I think the problem must be related to the two nested Delayed() instances.

In this particular case, you can force that not to happen by using a
different configuration. For example, the empty configuration won't
do any rewriting, and so won't replace most of your parse with None:

class MagusTest(TestCase):

def test_magus(self):
basicConfig(level=DEBUG)

name = Word(Letter())

expression = Delayed()
variable = Delayed()

function = (expression / '()') > 'function'
expression += (variable | function)

variable += (name | expression / '.' / name)

dotted_name = function & Eos()

parser = dotted_name.string_parser(Configuration())
print(parser.matcher)

result = parser("func")

However, that *will* go into an infinite loop.

I need time to fix the bug, but I can perhaps write a parser for this
fragment that works.

As far as I can tell, you want to parse the following:

foo
foo.bar
foo.bar.baz

where any of foo, bar and baz can be function invocations.

If so, I would use:

atom = Word(Letter())
path = atom[1:,'.'] # a sequence of atoms with dots between
function = path & '()'
expression = function[0:,'.'] & Optional('.' & path) & Eos()

If you need spaces everywhere look at putting everything in
with Separator(...):
or (better) use Tokens. See the tutorial.

Maybe that is not what you want, because there is some abiguity about
what you whether the path leading to a function is part of a function
or something else. But perhaps it will at least help - it's the
simplest non-ambiguous interpretation I can put on what you gave.

I will look at the bug, but cannot promise a rapid reply (unless it's
a complete disaster I would hope to have it fixed before the end of
the weekend).

Andrew

PS Note I haven't tested the above - there may be a typo or simple
stupid error...


2009/9/1 Magus <nbm...@gmail.com>:

[andrew cooke]

For your information, in case it helps with a workaround, the root
cause of the "None" error is that you had a series of Delayed
matchers, and then used one defined "in the middle" to create your
parser. It turns out that all the testing I have done uses the last
Delayed() that's "tied up" (with +=) and that the clone algorithm
relies on that.

So if you have

a = Delayed()

b = Delayed()
c = Delayed()

...
blah
c += ....
blah
a += ...
blah
b += ...

You cannot (currently) use c or a as the source of your parser,
because "b" was "tied up" after them.

I can see how to fix this (I need to sort the graph rather than make
assumptions about it), but the fix is non-trivial, so it really won't
be until this weekend earliest.

Sorry,
Andrew

2009/9/1 andrew cooke <acook...@gmail.com>:

[Magus]

Hi Andrew,

> As far as I can tell, you want to parse the following:
>
>   foo
>   foo.bar
>   foo.bar.baz
>
> where any of foo, bar and baz can be function invocations.
>

Exactly.

> I will look at the bug, but cannot promise a rapid reply (unless it's
> a complete disaster I would hope to have it fixed before the end of
> the weekend).
>
> Andrew
>

No problem and no worries. Take your time. Meanwhile I will review
your suggestions. Thanks again for your support.

Cheers,
Nick

[andrew cooke]

The latest code at http://code.google.com/p/lepl/source/list has a fix
for this issue (incorrect cloning of graphs when the parser is not
based on the "top" Delayed instance). This is a temporary solution
(each clone introduces more Delayed instances), but solves the
immediate problem.

So Nick if you want to grab this from mercurial, it should fix the
issue with "None".

I am continuing to work on this (I will add a separate rewriter that
removes Delayed instances from a graph) and then release a new version
of LEPL - hopefully this weekend, but it could be later in the month.

I wouldn't suggest people use the subversion code unless they need to
fix this same issue as it's not well-tested and also contains other
changes related to (incomplete) offside rule parsing.

Cheers,
Andrew

[Magus]

Thanks, Andrew. I will give it a try. I'm looking forward for a new
release of your nice library.

Cheers,
Nick

On Sep 3, 2:02 pm, andrew cooke <acooke....@gmail.com> wrote:
> The latest code athttp://code.google.com/p/lepl/source/listhas a fix

[Magus]

Hi Andrew,

I tried the new version 3.2. It seems to be working OK with my test
grammar, thanks. I didn't test it thoroughly yet. Unfortunately it
crashed on an invalid input. I've just put an invalid symbol in front
of the input string. Can you please check it?

Thanks,
Nick

from lepl import *

#from logging import basicConfig, DEBUG
#basicConfig(level=DEBUG)

name = Word(Letter()) > 'name'

expression = Delayed()
variable = Delayed()

function = (expression / '()') > 'function'

expression += (variable | function) > 'expression'
variable += (name | expression / '.' / name) > 'variable'

dotted_name = function & Eos()

parser = dotted_name.string_parser()

result = parser("1func()")
print result

LEPL 3.1

No handlers could be found for logger "lepl.parser.trampoline"
Traceback (most recent call last):

File "C:\Code\LEPL>test11.py", line 18, in <module>
result = parser("1func()")

File "C:\Python26\lib\site-packages\lepl\parser.py", line 249, in
single
return next(matcher(arg))[0]
File "C:\Python26\lib\site-packages\lepl\parser.py", line 182, in
trampoline
raise value
AttributeError: 'NoneType' object has no attribute '_match'

LEPL 3.2

No handlers could be found for logger "lepl.parser.trampoline"
Traceback (most recent call last):

File "C:\Code\LEPL>test11.py", line 18, in <module>
result = parser("1func()")
File "C:\Python26\lib\site-packages\lepl\parser.py", line 252, in
single
return next(matcher(arg))[0]
File "C:\Python26\lib\site-packages\lepl\parser.py", line 183, in
trampoline
raise value
TypeError: 'NoneType' object is not iterable

[andrew cooke]

Oh dear. Sorry about this. I don't yet understand what the problem
is - although it looks like the same problem as before, it seems to be
different somehow.

Because I don't understand the problem yet I don't understand how
reliable this workaround is, but it seems that you can avoid the error
by replacing
parser = dotted_name.string_parser()
with
parser = dotted_name.string_parser(
Configuration(rewriters=[auto_memoize()],
monitors=[TraceResults(False)]))

Which might help you continue while I try work out what is happening.

Thanks for the bug report,
Andrew


2009/9/6 Magus <nbm...@gmail.com>:

[andrew cooke]

I've tracked this down to incorrect detection of loops in matcher
graphs, which broke when the clone code changed - see
http://code.google.com/p/lepl/issues/detail?id=12 (which also contains
a workaround)

A fix should be possible, but I can't give a good estimate as it's
taken me "all weekend" (ie the time I had free) just to work out what
the problem was... this was one really obscure bug :o(

Andrew

[Magus]

Thank you, Andrew. I am sorry about your weekend but I thought you
should know that problem. I hate obscure bugs too. :/

Cheers,
Nick

On Sep 7, 12:46 am, andrew cooke <acooke....@gmail.com> wrote:
> I've tracked this down to incorrect detection of loops in matcher

> graphs, which broke when the clone code changed - seehttp://code.google.com/p/lepl/issues/detail?id=12(which also contains

[andrew cooke]

It's not your fault - my stupid code (or lack of tests) Andrew

2009/9/7 Magus <nbm...@gmail.com>:

[andrew cooke]

There's now a fix in hg that appears to make the loop detection work
correctly (and so fixes this issue). I need to add another test (or
extend an existing one), update the docs, and do various release-
related things before a new release, but you can grab the latest code
from this url if you (anyone!) want(s) - http://code.google.com/p/lepl/source/checkout
(if you do, and there are problems, please shout)

Cheers,
Andrew