Error in parsing white space

17 views
Skip to first unread message

philos99

unread,
Mar 9, 2009, 8:24:25 PM3/9/09
to Treetop Development
I can't understand why the following test fails.
Could you tell me why? Thanks, in advance.

The version of the operating system is Ubuntu 8.10
the ruby version is ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-
linux]

The following is the result of gem list --local

$ gem list --local

*** LOCAL GEMS ***

actionmailer (2.2.2)
actionpack (2.2.2)
activerecord (2.2.2)
activeresource (2.2.2)
activesupport (2.2.2)
cgi_multipart_eof_fix (2.5.0)
daemons (1.0.10)
fastthread (1.0.1)
gem_plugin (0.2.3)
hoe (1.9.0)
mongrel (1.1.5)
polyglot (0.2.5) <----
rails (2.2.2)
rake (0.8.3)
rubyforge (1.0.3)
sqlite3-ruby (1.2.4)
treetop (1.2.4) <----


# in arithmetic.treetop

grammar Arithmetic
rule num
[1-9] [0-9]* {
def value
text_value
end
}
end

rule space
[ \t\n\r]+ {
def value
text_value
end
}
end
end



# in test_parser.rb

require "rubygems"
require "treetop"
require "arithmetic"

parser = ArithmeticParser.new

p parser.parse("123")
p parser.parse(" \n\t\r")
puts parser.failure_reason



# the execution result

$ ruby test_parser.rb
SyntaxNode+Num1+Num0 offset=0, "123" (value):
SyntaxNode offset=0, "1"
SyntaxNode offset=1, "23":
SyntaxNode offset=1, "2"
SyntaxNode offset=2, "3"
nil
nil


Clifford Heath

unread,
Mar 9, 2009, 8:44:38 PM3/9/09
to treet...@googlegroups.com
On 10/03/2009, at 11:24 AM, philos99 wrote:
> I can't understand why the following test fails.

You've asked Treetop to parse "num", which is a
string of digits starting with 1-9. The "space" rule
can never get called, so it's irrelevant. When you
call the parser like this:

> p parser.parse("123")

You get this:

> SyntaxNode+Num1+Num0 offset=0, "123" (value):
> SyntaxNode offset=0, "1"
> SyntaxNode offset=1, "23":
> SyntaxNode offset=1, "2"
> SyntaxNode offset=2, "3"

which is exactly what's expected. The returned top
node from the parse has your value method, but you
haven't called that. Check it:

t = parser.parse("123")
puts t.value

When you call the parser like this:

> p parser.parse(" \n\t\r")
> puts parser.failure_reason

I'm surprised that you get no failure reason. I suspect
that's a bug - but in any case, the parse will fail, because
Treetop is looking for "num" and the input doesn't match
that.

Clifford Heath.

Markus

unread,
Mar 9, 2009, 8:51:22 PM3/9/09
to treet...@googlegroups.com

> # in arithmetic.treetop
>
> grammar Arithmetic
> rule num
> [1-9] [0-9]* {
> def value
> text_value
> end
> }
> end
>
> rule space
> [ \t\n\r]+ {
> def value
> text_value
> end
> }
> end
> end

The parser only parses things that match the top (first) rule. You
never call the space rule, so it is never used. Perhaps you wanted
something structured more like this (semantic defs omitted for clarity)?

grammar Arithmetic
rule space_or_num
space / num
end


rule num
[1-9] [0-9]*

end
rule space
[ \t\n\r]+

end
end

-- MarkusQ

philos99

unread,
Mar 9, 2009, 9:17:16 PM3/9/09
to Treetop Development
Thanks a lot for your insightful explanation!
I didn't major in the computer science but enjoy programming just for
fun as an amateur programmer.
Now I come to know that a top rule must include all the sub-rules
below.
Thanks again!

spir

unread,
Mar 10, 2009, 5:18:28 AM3/10/09
to *treetop
Le Mon, 09 Mar 2009 17:51:22 -0700,
Markus <mar...@reality.com> s'exprima ainsi:

> grammar Arithmetic
> rule space_or_num
> space / num
> end
> rule num
> [1-9] [0-9]*
> end
> rule space
> [ \t\n\r]+
> end
> end
>
> -- MarkusQ

Or:
rule num_list
(space / num)+
end
?

Denis
------
la vita e estrany

spir

unread,
Mar 10, 2009, 5:23:34 AM3/10/09
to treet...@googlegroups.com
Le Mon, 9 Mar 2009 18:17:16 -0700 (PDT),
philos99 <phil...@gmail.com> s'exprima ainsi:

> Now I come to know that a top rule must include all the sub-rules
> below.

Rather:
It must describe (the pattern of) the whole of what you intend to parse. Further rules define sub-patterns of subsets of this whole.

denis

Reply all
Reply to author
Forward
0 new messages