What's the plan for scala-actors and scala-parser-combinators in Scala 2.12?

[Simon Ochsenreither]

Are we still planning to

• ship them in the standard distro
• publish them in the repository

?

[Adriaan Moors]

I think we should spin off scala-actors as a deprecated module, but no longer distribute or maintain it. (The main reason to keep is that we want trivial cross-building between 2.11 and 2.12)

Why change the status of scala-parser-combinators?

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Simon Ochsenreither]

I think we should spin off scala-actors as a deprecated module, but no longer distribute or maintain it. (The main reason to keep is that we want trivial cross-building between 2.11 and 2.12)

Agree.
 

Why change the status of scala-parser-combinators?

What _is_ the status of parser combinators from your point of view?

I was assuming that the status of it was similar to scala-actors: Largely unmaintained, better replacement available plus technical debts which makes it unlikely to ever be competitive with alternatives.

(That's kind of the point why I wanted to split it from the standard lib in the first place: To get rid of it as fast as possible.)

Recent painful experiences with scala-parser-combinators just reminded me to bring this up again. ("Let's use this innocent looking class from util.parsing, it seems like it does exactly what we need!" ... leading to a performance regressions of roughly two magnitudes.)

Having something in the scala namespace, even if it's just a "module", is just asking people to assume a level of implementation quality which is not given here.

[Haoyi Li]

scala-parser-combinators is pretty bad. Basic things that you'd expect work (e.g. using log  to print out the text a rule is trying to parse) don't, and haven't worked properly for years:

trying my-rule at scala.util.parsing.input.CharSequenceReader@1c274bca

On the other hand, what are the better replacements available? Parboiled2 is still in the "blow up in your face compiler-crash for every mistake" phase, which is pretty rough, even though in many cases it is much better.

--

[Adriaan Moors]

I would love to replace the parser combinators -- I agree they're pretty bad (I would know, I wrote a lot of that code ;-)), but we'd need a viable replacement first.

[Simon Ochsenreither]

On the other hand, what are the better replacements available? Parboiled2 is still in the "blow up in your face compiler-crash for every mistake" phase, which is pretty rough, even though in many cases it is much better.

I think that even having no replacement would be better than keeping parser-combinators un-deprecated. (Not sure what the motivation behind un-deprecating the worst, already deprecated parts was.)

I think parboiled2, with parboiled as a fallback for those "blow up" situations would be a good mid- to long-term solution. (Although I have never experienced such a situation myself, .... do you have a link?)

[Haoyi Li]

Don't have any links right now; you'll just have to take my word for it writing a 200+ line parboiled parser for a whilespace-delimited variant of Scala/Twirl.

--

[Simon Ochsenreither]

Don't have any links right now; you'll just have to take my word for it writing a 200+ line parboiled parser for a whilespace-delimited variant of Scala/Twirl.

Sorry, I didn't want to make it look like as if I was questioning your experience, I was just interested in having more details to file issues against parboiled2 as necessary.

[Haoyi Li]

Don't have anything right now =P The reason I haven't filed any tickets is because parboiled2 still isn't even at the state where there's a specification/expected-behavior I can file tickets against, at least w.r.t. treating the parboiled code as Scala (i.e. HOFs, nesting, inheritance, etc.) and not as some glorified EBNF. I'd give it a bit more time to stabilize...

On Sun, Nov 9, 2014 at 10:40 AM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

Don't have any links right now; you'll just have to take my word for it writing a 200+ line parboiled parser for a whilespace-delimited variant of Scala/Twirl.

Sorry, I didn't want to make it look like as if I was questioning your experience, I was just interested in having more details to file issues against parboiled2 as necessary.

--

[Mathias Doenitz]

Haoyi,

thanks for sharing your experience!

> parboiled2 still isn't even at the state where there's a specification/expected-

> behavior I can file tickets against, at least w.r.t. treating the parboiled code as Scala (i.e. HOFs, nesting, inheritance, etc.) and not as some glorified EBNF.

This appears to be important.
How can we improve things here from our side, i.e. what is it exactly that you are missing with regard to "specification/expected-behavior" so that filing tickets would make sense to you?

So far we are not really aware of the issues you are experiencing, so I'd really like to learn more in order to have the chance to improve things where possible.

Thanks and cheers,
Mathias

[Haoyi Li]

Well, since you asked nicely... here's a list from least serious to most serious =P

The Value Stack is mind-melting

I know what it is. I think I've figured out why it's better than just returning tuples. Regardless, it was a huge leap for someone who has used scala-parser-combinators extensively and written my own PEG combinator library in python.

Generally, I feel it is much more flexible than it needs to be. For most practical purposes, it provides a way of automatically turning a Parser0 into a Parser1 into a Parser2/3/4 without needing to manually skip all the unwanted results like you do in scala-parser-combinators. For the vast majority of use-cases, they're basically HList tuples 

However, this point isn't made anywhere at all. In fact, the readme actively distracts you from having this realization by saying that your parsers are operating via side-effects on some mutable stack somewhere, then telling us that these side-effects are special and we shouldn't side-effect anywhere else, only here. 

This may be true technically, but obscures the underlying elegance of the value stack as a easier way of writing pure-functional parsers, which is probably what 99% of what people want.

Undefined error behavior

I have no idea what the error reporting is meant to be like in parboiled2. As I mentioned here https://github.com/sirthias/parboiled2/issues/102, I *think* it tries to do some longest-successful-partial-parse thing, but I have no idea really. I'm just guessing. It's not talked about anywhere. scala-parser-combinators does a very-straightforward "backtrack out of everything" thing which is exactly what you'd expect from a PEG parser:

scala> object Cow extends RegexParsers{
     |   def thing = "i am cow" | "i am cow hear me moo"
     | }
defined object Cow

scala> Cow.parseAll(Cow.thing, "i am")
res1: Cow.ParseResult[String] =
[1.1] failure: `i am cow hear me moo' expected but `i' found

i am
^

Whereas I can only guess at where the parboiled error message is coming from

scala> import org.parboiled2._
import org.parboiled2._

scala> class C(val input: ParserInput) extends Parser{
     |  def thing = rule {"i am cow" | "i am cow hear me moo"}
     | }

scala> new C("i am ").thing.run().recover{case x: ParseError =>x}.get.asInstanceOf[ParseError].formatTraces
res11: String =
"2 rules mismatched at error location:
  thing / "i am cow" / 'c'
  thing / "i am cow hear me moo" / 'c'
"
       new C("i am ").thing.run().recover{case x: ParseError => x}.get.asInstanceOf[ParseError].position
res12: org.parboiled2.Position = Position(5,1,6)

Maybe it's better to have it further in the string, but I also have no idea how it's generated, and thus it is useless to me. Furthermore, the parboiled parsers also have...

No Cut

scala-parser-combinators has had it forever as

"hello" ~! " world"

Which, together with the predictable error reporting, makes constructing parsers with known error semantics easy. Maybe the parboiled errors are better/further-in-string, but I have no idea what their semantics are or where they're coming from, and they're not documented anywhere, so I can't rely on them to do anything I want. The dumb backtrack-out-of-everything + cut that semantic that scala-parser-combinators has is extremely simple and predictable in terms of where the final error position turns up.

Un-useful toString

Note also how the scala-parser-combinator's failure gives you a nice textual tostring which tells you exactly where the failure is. Here's my adventure trying to get that info out of parboiled

scala> new C("i am ").thing.run()
res2: scala.util.Try[Unit] = Failure(org.parboiled2.ParseError)

scala> new C("i am ").thing.run().toFailure
<console>:16: error: value toFailure is not a member of scala.util.Try[Unit]
              new C("i am ").thing.run().toFailure
                                         ^

scala> new C("i am ").thing.run().asFailure
<console>:16: error: value asFailure is not a member of scala.util.Try[Unit]
              new C("i am ").thing.run().asFailure
                                         ^

scala> new C("i am ").thing.run().error
<console>:16: error: value error is not a member of scala.util.Try[Unit]
              new C("i am ").thing.run().error
                                         ^

scala> new C("i am ").thing.run().recover{case x => x}
res6: scala.util.Try[Any] = Success(org.parboiled2.ParseError)

scala> new C("i am ").thing.run().recover{case x => x}.get
res7: Any = org.parboiled2.ParseError

scala> new C("i am ").thing.run().recover{case x: ParseError => x}.get
res8: Any = org.parboiled2.ParseError

scala> new C("i am ").thing.run().recover{case x: ParseError => x}.get.asInstanceOf[ParseError]
res9: org.parboiled2.ParseError = org.parboiled2.ParseError

scala> new C("i am ").thing.run().recover{case x: ParseError => x}.get.asInstanceOf[ParseError].formatExpectedAsString
res10: String = 'c'

scala> new C("i am ").thing.run().recover{case x: ParseError => x}.get.asInstanceOf[ParseError].formatTraces
res11: String =
"2 rules mismatched at error location:
  thing / "i am cow" / 'c'
  thing / "i am cow hear me moo" / 'c'
"
       new C("i am ").thing.run().recover{case x: ParseError => x}.get.asInstanceOf[ParseError].position
res12: org.parboiled2.Position = Position(5,1,6)

Maybe I just suck at Scala, but that was really hard! Why couldn't we just print it out in the first place!

No log() operator

As defined here. It is very useful and makes debugging actually possible, even though it's half borked, because it tells you when a rule succeeds *and* when it fails. run(println("Hello")) is what i've been doing and is not an good alternative, and maybe there's some cleverer way, but I've been unable to figure it out.

No Referential Transparency

None of the following work, for no clear reason. All these work great in scala-parser-combinators

Inlining a parser in another:

scala> class C(val input: ParserInput) extends Parser{
     |  def thing = rule {"i am cow" | rule{"hear me moo"}}
     | }
<console>:15: error: Invalid rule definition: {
  def wrapped: Boolean = C.this.__matchStringWrapped("hear me moo", "thing", C.this.__matchStringWrapped$default$3);
  val matched: Boolean = if (C.this.__collectingErrors)
    wrapped
  else
    C.this.__matchString("hear me moo", C.this.__matchString$default$2);
  if (matched)
    org.parboiled2.Rule
  else
    null
}.asInstanceOf[org.parboiled2.Rule[shapeless.HNil,shapeless.HNil]]
        def thing = rule {"i am cow" | rule{"hear me moo"}}

Higher-order parsers

scala> class C(val input: ParserInput) extends Parser{
     | def bracketed(inner: Rule0) = rule { '[' ~ inner ~ ']' }
     | def thing = bracketed(rule{"i am cow" ~ "hear me moo"})
     | }
defined class C

scala> new C("[i am cowhear me moo]").thing.run()
res5: scala.util.Try[Unit] = Failure(org.parboiled2.ParseError)

scala> new C("i am cowhear me moo").thing.run()
res6: scala.util.Try[Unit] = Failure(org.parboiled2.ParseError)

Putting parsers in objects 

scala> class C(val input: ParserInput) extends Parser{
     |  object KeepingThingsNeat{
     |    def thing = rule {"i am cow" | "i am cow hear me moo"}
     |  }
     | }
defined class C

scala> new C("i am cow").KeepingThingsNeat.thing.run()
<console>:12: error: Illegal `.run()` call base: new C(parboiled2.this.ParserInput.apply("i am cow")).KeepingThingsNeat.thing
              new C("i am cow").KeepingThingsNeat.thing.run()
                                                  ^

Even if the above fails, this works

scala> class C(val input: ParserInput) extends Parser{
     | object A{
     | def thing = rule{"omg"}
     | }
     | def thing = rule{A.thing ~ "wtf"}
     | }
defined class C

scala> new C("omgwtf").thing.run()
res3: scala.util.Try[Unit] = Success(())

For unclear reasons. 

Some things aren't meant to compile according to this. But they compile and do the wrong thing anyway

scala> class C(val input: ParserInput) extends Parser{
     | def expression = rule { bracketed(ab) ~ bracketed(cd) }
     | def ab = rule { "ab" }
     | def cd = rule { "cd" }
     | def bracketed(inner: Rule0) = rule { '[' ~ inner ~ ']' }
     | }
defined class C

scala> new C("[ab][cd]").expression.run()
res1: scala.util.Try[Unit] = Failure(org.parboiled2.ParseError)

An alternative is offered on the github-readme which makes no sense at all to someone using Scala for 3 years and has written my own parser combinator library in Python. It's also not clear if needing to do this weird dance a bug that's going away or a feature I'm stuck with.

I haven't managed to minimize any of the compiler crashes I was seeing, so I can't provide any examples here sorry!

Not Extendable

As described above, I am unable to write my own meta-rules, which is a huge blow to the usefulness of the framework. Even though scala-parser-combinators is clunky and slow, being able to write my own combinators meant that I could program at whatever level of abstraction I wanted. I made very heavy use of them e.g. here

Where I use it to abstract away the creation of a "block" which takes an arbitrary head and an indented body of things. In parboiled2, I have to do the weird meta-rule-dance, or start C&Ping my parsers

Furthermore, scala-parser-combinators had all parsers being a simple instance of Parser. I could define my own parsers manually simply by providing a 

apply(in: Input): ParseResult[T]

To do whatever I wanted. This includes new combinators, which are by their very nature meta-rules. In parboiled, all the parsers are defined as

  def ~[I2 <: HList, O2 <: HList](that: Rule[I2, O2])(implicit i: TailSwitch[I2, O @uncheckedVariance, I @uncheckedVariance],
                                                      o: TailSwitch[O @uncheckedVariance, I2, O2]): Rule[i.Out, o.Out] = `n/a`
  def |[I2 <: I, O2 >: O <: HList](that: Rule[I2, O2]): Rule[I2, O2] = `n/a`
  def unary_!(): Rule0 = `n/a`
  def named(name: String): this.type = `n/a`

Which means that whatever magic coolness you guys get being able to define your own rules, users like me can't do it and are stuck with the combinators provided.

The compilation errors are frightening

For example I make one small change/error in my code

   def BodyEx(exclusions: String = "") = rule{
-    push(offsetCursor) ~ oneOrMore(BodyItem(exclusions)) ~> {(i, x) =>
-      Ast.Block(x.flatten, i)
+    push(offsetCursor) ~ oneOrMore(BodyItem(exclusions)) ~> {(x) =>
+      Ast.Block(x.flatten)
     }
   }

And the error turns up as 

[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:16: type mismatch;

[error]  found   : shapeless.::[Int,shapeless.::[scalatex.stages.Ast.Block,shapeless.HNil]]

[error]  required: scalatex.stages.Ast.Block

[error]     new Parser(input, offset).Body.run().get

[error]                                          ^

^[[A[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:60: overloaded method value apply with alternatives:

[error]   [I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[I,shapeless.::[J,shapeless.::[K,shapeless.::[L,shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[J,shapeless.::[K,shapeless.::[L,shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[K,shapeless.::[L,shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[L,shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [Q, R, S, T, U, V, W, X, Y, Z, RR](f: (Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [R, S, T, U, V, W, X, Y, Z, RR](f: (R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [S, T, U, V, W, X, Y, Z, RR](f: (S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [T, U, V, W, X, Y, Z, RR](f: (T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [U, V, W, X, Y, Z, RR](f: (U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [V, W, X, Y, Z, RR](f: (V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [W, X, Y, Z, RR](f: (W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [X, Y, Z, RR](f: (X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [Y, Z, RR](f: (Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[Y,shapeless.::[Z,shapeless.HNil]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [Z, RR](f: (Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[Z,shapeless.HNil],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [RR](f: (scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [RR](f: (scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block.Text,shapeless.HNil],RR], implicit c: org.parboiled2.support.FCapture[(scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [RR](f: (Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block.Text,shapeless.::[scalatex.stages.Ast.Chain,shapeless.HNil]],RR], implicit c: org.parboiled2.support.FCapture[(Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [RR](f: scalatex.stages.Ast.Block => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block.Text,shapeless.::[scalatex.stages.Ast.Chain,shapeless.::[Int,shapeless.HNil]]],RR], implicit c: org.parboiled2.support.FCapture[scalatex.stages.Ast.Block => RR])org.parboiled2.Rule[j.In,j.Out] <and>

[error]   [RR](f: () => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block.Text,shapeless.::[scalatex.stages.Ast.Chain,shapeless.::[Int,shapeless.::[scalatex.stages.Ast.Block,shapeless.HNil]]]],RR], implicit c: org.parboiled2.support.FCapture[() => RR])org.parboiled2.Rule[j.In,j.Out]

[error]  cannot be applied to ((scalatex.stages.Ast.Chain, scalatex.stages.Ast.Block) => scalatex.stages.Ast.Chain)

[error]     IndentBlock ~> {

[error]                 ^

[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:71: The `optional`, `zeroOrMore`, `oneOrMore` and `times` modifiers can only be used on rules of type `Rule0`, `Rule1[T]` and `Rule[I, O <: I]`!

[error]     push(offsetCursor) ~ IfHead ~ BraceBlock ~ optional("else" ~ (BraceBlock | IndentBlock))

[error]                                                        ^

[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:74: The `optional`, `zeroOrMore`, `oneOrMore` and `times` modifiers can only be used on rules of type `Rule0`, `Rule1[T]` and `Rule[I, O <: I]`!

[error]     Indent ~ push(offsetCursor) ~ IfHead ~ IndentBlock ~ optional(Indent ~ "@else" ~ (BraceBlock | IndentBlock))

[error]                                                                  ^

[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:91: type mismatch;

[error]  found   : Int

[error]  required: String

[error]     ((a, b, c) => Ast.Block.For(b, c, a))

[error]                                 ^

[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:112: type mismatch;

[error]  found   : org.parboiled2.Rule[shapeless.HNil,shapeless.::[Int,shapeless.::[scalatex.stages.Ast.Block,shapeless.HNil]]]

[error]  required: org.parboiled2.Rule[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block,shapeless.HNil]]

[error]   def BraceBlock: Rule1[Ast.Block] = rule{ '{' ~ BodyNoBrace ~ '}' }

[error]                                                              ^

[error] 6 errors found

[error] (scalatexApi/compile:compile) Compilation failed

[error] Total time: 9 s, completed Nov 10, 2014 7:57:23 AM

That's a 15 kilobyte error message! Here it is formatted nicely

[info] Compiling 1 Scala source to /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/target/scala-2.11/classes...
[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:16: type mismatch;
[error]  found   : shapeless.::[Int,shapeless.::[scalatex.stages.Ast.Block,shapeless.HNil]]
[error]  required: scalatex.stages.Ast.Block
[error]     new Parser(input, offset).Body.run().get
[error]                                          ^
^[[A[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:60: overloaded method value apply with alternatives:
[error]   [I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[I,shapeless.::[J,shapeless.::[K,shapeless.::[L,shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[J,shapeless.::[K,shapeless.::[L,shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[K,shapeless.::[L,shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[L,shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[M,shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [N, O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[N,shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(N, O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [O, P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[O,shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(O, P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [P, Q, R, S, T, U, V, W, X, Y, Z, RR](f: (P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[P,shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(P, Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [Q, R, S, T, U, V, W, X, Y, Z, RR](f: (Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[Q,shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(Q, R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [R, S, T, U, V, W, X, Y, Z, RR](f: (R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[R,shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(R, S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [S, T, U, V, W, X, Y, Z, RR](f: (S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[S,shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(S, T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [T, U, V, W, X, Y, Z, RR](f: (T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[T,shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(T, U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [U, V, W, X, Y, Z, RR](f: (U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[U,shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(U, V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [V, W, X, Y, Z, RR](f: (V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[V,shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(V, W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [W, X, Y, Z, RR](f: (W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[W,shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(W, X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [X, Y, Z, RR](f: (X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[X,shapeless.::[Y,shapeless.::[Z,shapeless.HNil]]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(X, Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [Y, Z, RR](f: (Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[Y,shapeless.::[Z,shapeless.HNil]],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(Y, Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [Z, RR](f: (Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.::[Z,shapeless.HNil],shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(Z, scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [RR](f: (scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.HNil,RR], implicit c: org.parboiled2.support.FCapture[(scalatex.stages.Ast.Block.Text, scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [RR](f: (scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block.Text,shapeless.HNil],RR], implicit c: org.parboiled2.support.FCapture[(scalatex.stages.Ast.Chain, Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [RR](f: (Int, scalatex.stages.Ast.Block) => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block.Text,shapeless.::[scalatex.stages.Ast.Chain,shapeless.HNil]],RR], implicit c: org.parboiled2.support.FCapture[(Int, scalatex.stages.Ast.Block) => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [RR](f: scalatex.stages.Ast.Block => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block.Text,shapeless.::[scalatex.stages.Ast.Chain,shapeless.::[Int,shapeless.HNil]]],RR], implicit c: org.parboiled2.support.FCapture[scalatex.stages.Ast.Block => RR])org.parboiled2.Rule[j.In,j.Out] <and>
[error]   [RR](f: () => RR)(implicit j: org.parboiled2.support.ActionOps.SJoin[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block.Text,shapeless.::[scalatex.stages.Ast.Chain,shapeless.::[Int,shapeless.::[scalatex.stages.Ast.Block,shapeless.HNil]]]],RR], implicit c: org.parboiled2.support.FCapture[() => RR])org.parboiled2.Rule[j.In,j.Out]
[error]  cannot be applied to ((scalatex.stages.Ast.Chain, scalatex.stages.Ast.Block) => scalatex.stages.Ast.Chain)
[error]     IndentBlock ~> {
[error]                 ^
[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:71: The `optional`, `zeroOrMore`, `oneOrMore` and `times` modifiers can only be used on rules of type `Rule0`, `Rule1[T]` and `Rule[I, O <: I]`!
[error]     push(offsetCursor) ~ IfHead ~ BraceBlock ~ optional("else" ~ (BraceBlock | IndentBlock))
[error]                                                        ^
[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:74: The `optional`, `zeroOrMore`, `oneOrMore` and `times` modifiers can only be used on rules of type `Rule0`, `Rule1[T]` and `Rule[I, O <: I]`!
[error]     Indent ~ push(offsetCursor) ~ IfHead ~ IndentBlock ~ optional(Indent ~ "@else" ~ (BraceBlock | IndentBlock))
[error]                                                                  ^
[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:91: type mismatch;
[error]  found   : Int
[error]  required: String
[error]     ((a, b, c) => Ast.Block.For(b, c, a))
[error]                                 ^
[error] /Users/haoyi/Dropbox (Personal)/Workspace/scala-js-book/scalatexApi/src/main/scala/scalatex/stages/Parser.scala:112: type mismatch;
[error]  found   : org.parboiled2.Rule[shapeless.HNil,shapeless.::[Int,shapeless.::[scalatex.stages.Ast.Block,shapeless.HNil]]]
[error]  required: org.parboiled2.Rule[shapeless.HNil,shapeless.::[scalatex.stages.Ast.Block,shapeless.HNil]]
[error]   def BraceBlock: Rule1[Ast.Block] = rule{ '{' ~ BodyNoBrace ~ '}' }
[error]                                                              ^
[error] 6 errors found
[error] (scalatexApi/compile:compile) Compilation failed

Of the errors that turned up, none of them actually point to the location in the code I made the mistake. Maybe that's an artifact of type-inference, but it's not a problem in the traditional parser combinators where the compile errors were much less massive.

------------------------------------------------------------------------------------------------------

Hopefully this brings across my point: things that work in Scala, e.g. putting a variable in a local object, or inlining the contents of a definition in another, don't work for reasons not explained anywhere. For a parser which uses so many high-level type-system features, it feels extremely imperative and fragile, because even basic referential transparency doesn't apply. Even in the most mutable/imperative Python/Javascript code I've ever written, I can move definitions into namespaces without things blowing up, or inline them at the callsite. Not here!

It also feels like nobody's ever debugged a parser before, because there are no instructions, tools, or pointers as to how to debug your parser. I ended up discovering myself after maybe 10 hours of frustration you could put run(println())s to see what was happening, and even so It's still a significantly worse experience than the (broken!) log combinator that scala-parser-combinators provides. The compilation errors are massive and useless, and the runtime error-behavior is undefined (although I take your word for it that it's really very good ^_^).

Lastly, I can't define such abstractions/tools myself, or at least I don't know how to. Everything is undefined and left to macros to handle, meaning they exist on a higher plane than anything I can write. Meta-rules explicitly shouldn't compile, though they don't work at not-compiling even though they should. 

Hopefully this is useful ^_^ and substantiates my point that parboiled, for all its niceties, is far rougher to use than scala-parser-combinators. For all its oddities and rough-edges, at least scala-parser-combinators feels like Scala, while Parboiled2 currently does not.

[Matthew Pocock]

I use parser combinators compiled to js. They are for very simple languages but too complex to lex and hope. I would love to have much more  (errors i can show users, auto complete, and i would walk over burning coals if i could drop a parser onto a text area and get a language-aware editor) but for simple cases it works and i don't know an alternative.

Matthew

--

[Simon Ochsenreither]

Hey Haoyi,

thanks a lot for your write-up, I think this is extremely helpful!

Bye,

Simon

[Haoyi Li]

Here's something else

runSubParser is crazy

scala> class C(val input: ParserInput) extends Parser{

     |  def foo = rule{ "foo" ~ runSubParser(x => new C("bar").bar)}
     |  def bar = rule{ "bar" }
     | }
defined class C

scala> new C("foo").foo.run()
res0: scala.util.Try[Unit] = Failure(org.parboiled2.ParseError)

scala> new C("foobar").foo.run()
res1: scala.util.Try[Unit] = Success(())

What's the point of making me pass my ParserInput to runSubParser if you're just going to ignore it and parse what I was parsing anyway? 

Forgot to include it in the previous email, but this took ~3hrs of my time to figure out so it deserves its own special mention

--

[Mathias Doenitz]

Haoyi,

thank you very much for this comprehensive writeup of your thoughts and experiences with parboiled2!
We rarely get such a deep and detailed insight into what a library feels like from the a user’s perspective, so it really is very much appreciated!

I don’t want to spam this list with too much detail concerning a tool that is somewhat unrelated to the "Scala Internal” theme, so we should probably move further discussion over to the parboiled2 group [1]. Nevertheless, for the record here, I’d like to somewhat address your main points.

Overall, I could image that your significant prior experience with Scala’s Parser Combinators as well as your own PEG parser might actually be working against you in the case of pb2. It might be that it leads you to making assumptions that are not true and solution approaches that simply don’t fit the nature of the tool.

pb2 is a macro-based parser generator, not a library for writing pure-functional parsers in the way of Scala’s Parser Combinators. Even though both are used to solve similar problems and may look closely-related on the surface they differ significantly with regard to their inner workings and mode of operation.
In the latter aspect pb2 is actually closer to other parser generators like ANTLR, JavaCC, Aurochs or Rats! than to parser combinators.

Parser combinators, by their functional nature, give you a lot of flexibility to modify a parser rules with your own code, even at parser runtime. You can easily write higher-order logic that produce rules from rules from rules based on the actual parser input. If you keep your rule functions pure you get all the benefits of referential transparency, composition, etc.
While all this is great it does come with a cost: runtime overhead. Parser construction and parser execution are not separate phases, they are conflated into one. Effectively the parser parts are rebuilt over and over again while the input is digested resulting in parsing performance that is unacceptable for a significant share of all real-world applications.

It is these applications that pb2 is targeting. Being a parser generator, rule construction and rule execution are two separate phases. This allows for significantly higher parser performance at the cost of flexibility. Larger changes in parser behavior based on the actual input are hard, if not impossible. Higher-order rule-building logic becomes less convenient (Many external parser generators do not allow for *any* higher order rule logic at all!). Since rules are now not functions anymore they cannot be composed as easily.
All this is what you are paying for the performance benefits over purely-functional parsers.

Certain solutions that work great for functional parsers are unsuited or simply unavailable for parser generators. They are two different kinds of tools.

> The Value Stack is mind-melting

The Value Stack abstraction has worked great for parboiled 1.x, which is why we have kept it in pb2 and actually put even more weight on it.
However, it is operated on as a side effect and thus might appear “mind-melting” when you expect a purely functional mode of operation.
The key benefit over the functional approach: much much fewer allocation during parser runtime.

> Undefined error behavior

While you are right in that we can certainly do a better job at documenting it the error reporting behavior in pb2 is far from undefined.
In fact it is a lot more powerful than what is easily achievable with parser combinators or even other parser generators.
Without any additional work on your side it does produce error messages that are more informative and actionable to the user.
Still, we are probably still somewhat lacking in features with regard to allowing for customisation, but this is nothing that cannot be fixed.
Once again, the approaches that you might have found suitable for functional parsers don’t necessarily carry over to a parser generator!

> No Cut

Yes, that ticket is still open.
However, while it is a core and crucial feature for parser combinators it is a mere optimisation tool for pb2.
Error reporting works great without cut markers in pb2, while it wouldn't really work at all in parser combinators.
Once again, the delta in modes of operation between parser combinators and pb2 makes all the difference here.

> Un-useful toString
> ...

> Maybe I just suck at Scala, but that was really hard! Why couldn't we just print it out in the first place!

It’s just not how the API currently works. `ParseError` is a case class, which we haven’t given a custom `toString` (yet).
The README does mention “the `formatError` method available on the `Parser` class”, but you are right, documentation of error reporting is indeed lacking.
I’ve just created a ticket for that: https://github.com/sirthias/parboiled2/issues/103

> No log() operator

Yes, there is no reason not to have one built in.
And we need a docs section on parser debugging.
-> https://github.com/sirthias/parboiled2/issues/104, https://github.com/sirthias/parboiled2/issues/105

> No Referential Transparency
> None of the following work, for no clear reason. All these work great in scala-parser-combinators

> Inlining a parser in another, Higher-order parsers, Putting parsers in objects

The reason is simply: pb2 is a parser generator, not a purely-functional parser lib, as discussed above.

> Not Extendable
> As described above, I am unable to write my own meta-rules, which is a huge blow to the usefulness of the framework.

Same here. It’s not a framework, it’s a parser generator.
And it is indeed useful. For example, akka-http uses it quite successfully for its HTTP header parsing logic:
https://github.com/akka/akka/tree/release-2.3-dev/akka-http-core/src/main/scala/akka/http/model/parser

> The compilation errors are frightening

Yes, they can be.
The type-level logic combined with the quirks of the current macro implementation can indeed lead to scary compiler errors / crashes.
We might be able to improve things with more and smarter @implicitNotFound and friends, but this is ongoing work and likely an uphill battle.
A genuine disadvantage of the fundamental approach of combining macro-based parser generation and heavy type-level logic for maximum conciseness and type-safety.

> Hopefully this brings across my point: things that work in Scala, e.g. putting a variable in a local object, or inlining the contents of a definition in another, don't work for reasons not explained anywhere. For a parser which uses so many high-level type-system features, it feels extremely imperative and fragile, because even basic referential transparency doesn't apply. Even in the most mutable/imperative Python/Javascript code I've ever written, I can move definitions into namespaces without things blowing up, or inline them at the callsite. Not here!

Once again, your basic assumption that pb2 works similarly to parser combinators simply doesn’t hold.
It’s a parser generator that you configure with an internal Scala DSL.
Its closest cousins are probably ANTLR and Rats!, which you configure using external proprietary languages in separate files and run in dedicated build steps!

Still, your experience report makes it very clear that we definitely need to get hat "parboiled2 vs. Scala Parser Combinators” section written (which is currently only stubbed).

Thank you very much again for this valuable insight, Haoyi!

Cheers,
Mathias

[1]: https://groups.google.com/forum/#!forum/parboiled-user

---
mat...@parboiled.org
http://www.parboiled.org

> You received this message because you are subscribed to a topic in the Google Groups "scala-internals" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/scala-internals/4N-uK5YOtKI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to scala-interna...@googlegroups.com.

[Haoyi Li]

Looking forward to fleshing out that section =)

As I said at the start, I don't even know what i'm meant to expect, so when things don't work I don't know who's fault it is =P 

In most other libraries (e.g. scala-parser-combinators) this isn't a problem, because it's just Scala, but Parboiled2 is not "just Scala", so having a spec detailing what should works and what doesn't is important. You've documented all the "happy paths" on the readme, but that leaves unclear where the boundaries of that section are, and how I can avoid stepping outside it. e.g. for ParseError error reporting behavior, I'm sure it's excellent, I just don't know what it is =P

A strong, detailed specification of what you are allowed to do and not allowed to do, and what happens when things go wrong at compile- or run-time, would go a long way to making parboiled2 approachable ^_^

[Alexander Gryzlov]

Regarding parser combinator libraries, here's quite raw but still interesting one: 

https://github.com/tpolecat/atto

As the name says, it's a port of Haskell's Attoparsec.

[Naftoli Gugenheim]

On Mon, Nov 10, 2014 at 5:55 PM, Mathias Doenitz <mat...@parboiled.org> wrote:

They are two different kinds of tools.

I guess that takes us right back to the beginning of the discussion:

Simon:

I was assuming that the status of it was similar to scala-actors: Largely unmaintained, better replacement available plus technical debts which makes it unlikely to ever be competitive with alternatives.

Haoyi:

what are the better replacements available?

So it sounds like it's not a replacement, they're apples and oranges...

[Haoyi Li]

On the other hand, do we need a real replacement before we kick it out of the standard library? Didn't we kick out Scala-XML even without any real replacement?

[Jason Zaugg]

Both have been spun out to modules, not kicked out. 

Jason

—
Sent from Mailbox

[Som Snytt]

They've been transitioned to other roles in the ecosystem.

[Daniel Spiewak]

Most people implementing parser combinator libraries seem to steer clear of reimplementing Packrat with the same syntax as the built-in parser combinators specifically because it is competing with the built in parser combinators.  Kiama is the only one I can think of off-hand, and there it was mostly because (as I recall) Kiama implemented Packrat parsing before Scala did.

Thus, most of the alternatives either go with different algorithms (with very different tradeoffs and thus very much not compatible), or choose different syntax (like the parsec ports), or some combination of the two.  For example, my own project gll-combinators chooses a syntax very close to the built-in parser combinators with some improvements in the implicit machinery, but uses a radically different and incompatible algorithm.

So while there are numerous alternatives out there, I don't think anything exists right now that can be a drop-in replacement for the built-in parser combinators.  We could build one, certainly, and it honestly wouldn't be that hard, but no such project (to my knowledge) exists today simply because replacing built-in stuff in a corner as esoteric as parsing is unlikely to gain much traction.

Daniel

[Rüdiger Klaehn]

My two cents: I don't expect high performance from parser-combinators.
People that want to write a parser for a large, complex grammar will
look for something else like parboiled2.

But it is very convenient to have a small, non-macro-based library to
write a quick parser when performance is not an issue. And this
library is covered in all the first and second edition of the most
popular scala book, so a lot of people are going to use it for better
or worse.

So issues such as not having thread safety should be fixed or at least
thoroughly documented. And in the long term a drop-in replacement with
the same syntax would be very welcome.

[Adriaan Moors]

I agree. Should've been more precise last time: the parsers are here to stay, but the implementation could stand to be improved :) 

Would be great if the other parser libraries could be drop in replacements, or have a compatibility layer to do so. 

Cheers

Adriaan 

[Daniel Spiewak]

The problem with drop-in replacements is that algorithmic differences make this impossible.  We're not just talking about an API facade.  If you're swapping in a different parser algorithm and you were originally using PEG (i.e. traditional parser combinators) or even Packrat, you're going to need to change your grammar.  It's a little bit better going between implementations of parser algorithms which use actual grammars (i.e. an LALR framework to GLL or GLR), but you're still going to have to worry about the differences.

In a sense, even pretending to be a drop-in replacement would do more harm than good, since the code would compile and would produce identical results on trivial input, lulling the users into a false sense of security.  With a lot of these algorithmic differences, only an incredibly comprehensive test suite would catch the cases that change between algorithms.  Very few people test their parsers to that kind of level (I certainly don't, and my parser test suites are huge).

Daniel