Exception during macro expansion using the ~> action operator

[Michael]

Hi,

I want to create JSON from a semi-structured input. For better readability it would be great to write the JSON field name right after the corresponding captured string value, e.g. capture(…) ~> stringField("jsonAttributeName"). The action ~> operator expects a function, but there seems to be some limitation due to macro expansion, because the rule FruitShortInlineAttributeNames in the code below does not work, but some modified versions do. How to fix this, so that it can be written as short as in FruitShortInlineAttributeNames?

Side note: The resulting JSON object is constructed by maintaining a List of (String, JValue) tuples on parboileds value stack, which is used to create a JObject as last step.

import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.parboiled2._
import scala.util.{Failure, Success}

object Main {

  class TestParser(val input: ParserInput) extends Parser {

    def stringField(name: String) = { (fields: List[JField], value: String) => (name, JString(value)) :: fields }
    def intField(name: String)    = { (fields: List[JField], value: Int)    => (name, JInt(value))    :: fields }

    def PushNonSpaceString = rule { capture(oneOrMore(!' ' ~ ANY)) }
    def PushInt = rule { capture(oneOrMore(CharPredicate.Digit)) ~> { (_: String).toInt } }

    // does not work: exception during macro expansion
    def FruitShortInlineAttributeNames: Rule1[JObject] = rule {
      push(List[JField]()) ~
        "fruitName=" ~ PushNonSpaceString ~> stringField("name") ~ ' ' ~
        "fruitWeight=" ~ PushInt ~> intField("weight") ~> { (fields: List[JField]) => JObject(fields) }
    }


    // works, but too much to write
    def FruitInlineAttributeNames: Rule1[JObject] = rule {
      push(List[JField]()) ~
        "fruitName=" ~ PushNonSpaceString ~> { stringField("name")(_, _) } ~ ' ' ~
        "fruitWeight=" ~ PushInt ~> { intField("weight")(_, _) } ~> { (fields: List[JField]) => JObject(fields) }
    }


    // this works, but JSON attribute name not inline
    val nameField = stringField("name")
    val weightField = intField("weight")

    def FruitExternalFieldFunctions: Rule1[JObject] = rule {
      push(List[JField]()) ~
        "fruitName=" ~ PushNonSpaceString ~> nameField ~ ' ' ~
        "fruitWeight=" ~ PushInt ~> weightField ~> { (fields: List[JField]) => JObject(fields) }
    }
  }


  def main(args: Array[String]) {
    val input = "fruitName=Apple fruitWeight=123"
    val parser = new TestParser(input)
    parser.FruitInlineAttributeNames.run() match {
      case Success(obj: JObject) => println(compact(render(obj)))
      case Failure(ex: Throwable) => println("error: "+ ex.toString)
    }
  }
}

build.sbt:

name := "parboiled-test"

scalaVersion := "2.11.7"

scalacOptions ++= Seq("-unchecked", "-deprecation", "-feature", "-Xfatal-warnings")

libraryDependencies ++= Seq(

  "org.parboiled" %% "parboiled"      % "2.1.0",

  "org.json4s"    %% "json4s-jackson" % "3.2.10"

)

Thank you!

Michael

[Mathias Doenitz]

Michael,

I believe the problem you are seeing is a limitation of pb2’s current way of macro implementation.
I don’t know yet whether it’ll be possible to improve.

So, I’m sorry, but I currently don’t know of any way to make the very compact syntax that you are looking for actually work.

Cheers,
Mathias

---
mat...@parboiled.org
http://www.parboiled.org

> --
> You received this message because you are subscribed to the Google Groups "parboiled2.org User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to parboiled-use...@googlegroups.com.
> Visit this group at http://groups.google.com/group/parboiled-user.
> To view this discussion on the web visit https://groups.google.com/d/msgid/parboiled-user/057cbe61-0216-4b8a-8c0a-d720b8d71ee3%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Michael]

Hi Mathias,

thank you for your response!

Today I tried realizing this as a macro (as everything else in parboiled2 is very macro-based) and this way seems to work…

import org.json4s._
import scala.language.experimental.macros
import scala.reflect.macros.whitebox

object JsonFieldMacros {
  def stringField(name: String): (List[JField], String) => List[JField] = macro JsonFieldMacros.stringFieldImpl
  def intField(name: String): (List[JField], Int) => List[JField] = macro JsonFieldMacros.intFieldImpl

  def stringFieldImpl(c: whitebox.Context)(name: c.Expr[String]): c.Expr[(List[JField], String) => List[JField]] = {
    import c.universe._
    c.Expr(q"""{ (fields: List[JField], value: String) => ($name, JString(value)) :: fields }""")
  }

  def intFieldImpl(c: whitebox.Context)(name: c.Expr[String]): c.Expr[(List[JField], Int) => List[JField]] = {
    import c.universe._
    c.Expr(q"""{ (fields: List[JField], value: Int) => ($name, JInt(value)) :: fields }""")
  }
}

But as I didn't implement any Scala macros so far, I will need to read some docs first about how to do it right (especially how to setup the build configuration to circumvent the "macro implementation not found: … (the most common reason for that is that you cannot use macro implementations in the same compilation run that defines them)" error).

Regards,

Michael