[2.1 Scala] How to process and transform a Json list

[Guan]

From tutorial, all examples are transform a single js object.

I have following json

{data: [

   {"name": "a"}, {"name":"b"}

]}

and I want to add a field for each of item in the list to

{data: [

   {"name": "a", "age":20}, {"name":"b", "age":20}

]}

How can I do that?

[Guan]

Resolved. Kind of tricky. Hope there will be helpers for manipulating json array objects later.

[Tom Bocklisch]

I am also trying to solve the same problem, therefore it would be great if you could post an example how you solved it.

Best Regards 

[David P.]

So, this is what I did to update a JsArray lately:

  def fromArrayIds(field: String) = (__ \ field).json.update (

    of[JsArray].map { case JsArray(list) => JsArray(list.map(_ \ "$oid")) }

  )

This is used as a part of a broader transform, but I think it should be able to work by itself. As opposed to adding a field to an object though, it just simplifies the ObjectId for output to a Web Service. Just replace the innards of list.map(_ \ "$oid") to change each item in the array however you want. The list variable above is just a Scala List.

Hope that helps.

[Ahmad El-Melegy]

I am facing the same problem.
Please David If you can post a full example, as it says:

Cannot resolve symbol `of`
Cannot resolve reference `JsArray` with such signature 

Thanks.

[David P.]

Sounds like you don't have all of the imports you need. Here are the imports I generally use when writing transformers (up to Play 2.4):

import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
import play.modules.reactivemongo.json.BSONFormats._

My latest code for updating an Array (though to be honest, it's been a while since I've worked on it, hopefully there's a better way to do this now):

  /**
   * Transforms a JsArray using the provided Reads; cumulating errors.
   *
   * @param reads the transforming Reads
   * @tparam A    the Type deserialized by the given Reads
   * @return a JsArray transforming Reads
   */
  def tfList[A <: JsValue](reads: Reads[A])
  : Reads[JsArray] = Reads {
    case arr: JsArray =>
      val init: JsResult[Seq[JsValue]] = JsSuccess(Seq[JsValue]())
      arr.value.foldLeft(init) { (acc, e) =>
        acc.flatMap(seq => e.transform(reads).map(seq :+ _))
      } map JsArray
    case _ => JsError("expected JsArray")
  }

With this function, you should be able to transform the contents of an array using any `Reads` you need. For instance:

(__ \ "str_list").json.pickBranch(tfList(medStr))

That will check each String in the array "str_list" against the medStr validator, which is defined as:

val medStr: Reads[JsString] = of[JsString] keepAnd minLength[String](1) keepAnd maxLength[String](512)

You can also use it for much more complicated instances of Reads that can transform a whole document from one "format" to another. As an example, MongoDB JSON format is usually a little different than the format I use for REST API requests and responses. IDs, dates, etc... all need to be converted.

Hope that helps.

[Ahmad El-Melegy]

Thank you very much David, Really appreciate that you have answered on that old topic.

But unfortunately as I am doing my first steps with Scala and Play, I still can't make it.

I have this json:

{
  "activities": [
    {
      "id": "1",
      "project": {
        "p_id": "p_3"
      }
    },
    {
      "id": "2",
      "project": {
        "p_id": "p_4"
      }
    }
  ]
}


And I want to add an inner field into each field_name by concatenating a 
string and filed_id, so the output would be something like this:

{
  "activities": [
    {
      "id": "1",
      "project": {
        "p_id": "p_3",
        "field": "new_p_3"
      }
    },
    {
      "id": "2",
      "project": {
        "p_id": "p_4",
        "field": "new_p_4"
      }
    }
  ]
}

Please can you tell me How can I use your function to do this ?

[David P.]

Ooh... Not one but two relatively complicated things to do when you're new to this. Quite ambitious for you ;)

I'm going to assume that you get the original JSON from a database and you're looking to transform it to the output of a REST Service. I'm also going to assume that "default" serialization works for getting data in and out of the database. I don't have time to program it all out and test it for you today; but it's going to go something like this:

(Please note, much of what I'm basing this on was written in Play 2.3? Not sure if it still works in Play 2.5)

Model:

case class Activity(id: String, project: Project)
object Activity {

  import play.api.libs.json._
  import play.api.libs.json.Reads._
  import play.api.libs.functional.syntax.

_
  import MY.common.Validators._

  // For your "original" format, which I assumed was for an API to talk to a database.
  implicit val mongoFmt = Json.format[Activity]

  // Common functionality for all transformers
  //idStr is just the validator/Reads for your ID Strings.
  private val coreReads = (__ \ "id").json.pickBranch(idStr)

  val req2mongo = (
    (__ \ "project").json.pickBranch(Project.req2mongo) and
    coreReads
  ) reduce

  val mongo2resp = (
    (__ \ "project").json.pickBranch(Project.mongo2resp) and
    coreReads
  ) reduce
}

case class Project(p_id: String)
object Project {

  import play.api.libs.json._
  import play.api.libs.json.Reads._
  import play.api.libs.functional.syntax.

_
  import MY.common.Validators._ //where "idStr" is defined

  // For your "original" format, which I assumed was for an API to talk to a database.
  implicit val mongoFmt = Json.format[Activity]

  // Common functionality for all transformers
  private val coreReads = (__ \ "p_id").json.pickBranch(idStr)

  // Generates the value in field based on the data that was placed there in mongo2resp.
  private val genField = (__ \ "field").json.update (
    of[JsString].map { jsStr =>
      JsString("new_"+jsStr.value.trim)
    }
  )

  val req2mongo = coreReads

  val mongo2resp = (
    (__ \ "field").json.copyFrom((__ \ "id").json.pick) and
    coreReads reduce
  ) andThen genField
}

Then, presumably in a Controller somewhere:

// Here's where we actually use tfList.
private val activityListTransform = (__ \ "activities").json.pickBranch(tfList(Activity.mongo2resp))

protected def outputJson(input: JsObject): JsResult[JsObject] = {
  input.transform(activityListTransform)
}

[Ahmad El-Melegy]

Wow !

You are great man ! Thank you very very much.

Only one question, The `project` doesn't only contain one field `id`, There were other fields too.

The generated json contain only `id` and the new `field`.

How can I restore them all?

[David P.]

If I understand what you're asking, it should be like any other model.... I'll mod the code I had from before.

case class Project(
  p_id: String,
  name: String,
  description: String,
  users: Seq[ProjectUser]

)

object Project {
  import play.api.libs.json._
  import play.api.libs.json.Reads._
  import play.api.libs.functional.syntax.

_
  import MY.common.Validators._ //where "idStr", et al. are defined

  // For your "original" format, which I assumed was for an API to talk to a database.
  implicit val mongoFmt = Json.format[Activity]

  // Common functionality for all transformers
  private val coreReads = (

    (__ \ "p_id"       ).json.pickBranch(idStr)    and
    (__ \ "name"       ).json.pickBranch(smallStr) and
    (__ \ "description").json.pickBranch(longStr)
  ) reduce

  // Generates the value in field based on the data that was placed there in mongo2resp.

  private val genField = (__ \ "field").json.update (
    of[JsString].map { jsStr =>
      JsString("new_"+jsStr.value.trim)
    }
  )

  // The generated "field" isn't here, because I assume that you
  // don't want to save something like that to the database.
  val req2mongo = (
    (__ \ "users").json.pickBranch(tfList(ProjectUser.req2mongo)) and

    coreReads
  ) reduce

  val mongo2resp = (

    (__ \ "users").json.pickBranch(tfList(ProjectUser.mongo2resp)) and
    (__ \ "field").json.copyFrom((__ \ "id").json.pick) and

    coreReads reduce
  ) andThen genField
}

If anything else needs to be transformed between Database and REST API, put them in req2mongo and mongo2resp. If no transformation is required, put it in coreReads.

I think that ought to work.... Like before, I haven't been compiling anything myself.

[Ahmad El-Melegy]

Thank you very very much.

...