Packrat Parser bug?

[Harshad]

Hi all,

I have this toy grammar with some left-recursion in it. I have used Packrat parsers to get around the left-recursion problem. Parsing with a string works fine but parsing from a file throws a stackoverflow error.

object test extends App {

  object ToyParser extends StandardTokenParsers with PackratParsers {
    lexical.delimiters ++= List("+", "/", "=", "(", ")", ":")

    private lazy val number = accept("number", { case lexical.NumericLit(n) => n.toInt })
    private lazy val expression: PackratParser[Int] = numericExpr
    private lazy val numericExpr: PackratParser[Int] = (addExpr | numericLiteralExpr)
    private lazy val numericLiteralExpr: PackratParser[Int] = number
    private lazy val addExpr: PackratParser[Int] = ((numericExpr <~ "+") ~ numericExpr) ^^ { case a ~ b => a + b }

    def parseAll = phrase(expression)
  }

  val scanner = new ToyParser.lexical.Scanner("10 + 20 + 30")
  println(ToyParser.parseAll(scanner)) // Success

  val scanner2 = new ToyParser.lexical.Scanner(StreamReader(io.Source.fromFile("test").reader))
  println(ToyParser.parseAll(scanner2)) // StackOverflowError

}

Is this a bug, or am I doing something wrong?

[Harshad]

Forgot to mention I am on Scala 2.10.1

thanks,
Harshad

[Sébastien Doeraene]

Hi,

Packrat parsers require you to give them a PagedSeqReader. If you
don't, all kinds of weird things happen. It seems you do not do that.
I use the following helper function to create a PagedSeqReader from a
file name:

private def readerForFile(fileName: String) = {
new PagedSeqReader(PagedSeq.fromReader(
new BufferedReader(new FileReader(fileName))))
}

Cheers,
Sébastien

> --
> You received this message because you are subscribed to the Google Groups
> "scala-language" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scala-languag...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Harshad]

Thanks Sébastien; your suggestion works perfectly. StreamReader inherits from PagedSeqReader so I thought it would work too.

Btw, it would be great if this suggestion is documented in the API docs. (I thought of creating a pull request myself and then saw the "Contributor agreement requirement" and thought that would be too much overhead)

thanks again,
Harshad