Recognizing child entities with natural IDs on parent insert/update

[Matt Duncan]

Hello again,

I feel this is an easy problem (and have made it work before) but I can't seem to find the solution..

Here's the scenario: I have two entities with natural IDs. They share a many-to-many relationship.

case class Guild(
            name: String,
            games: List[Game],
            members: List[GuildUser] = List(),
            id: Int = 0)


case class Game(name: String,
                publisher: String,
                website: String
                tournamentTypes: Set[TournamentType] = Set(),
                id: Int = 0)


object GuildEntity extends Entity[Int, SurrogateIntId, Guild]("guilds") {

  val id = key("id") autogenerated(_.id)
  val name = column("name") to (_.name)
  val users = onetomany(GuildUserEntity) to (_.members)
  val games = manytomany(GameEntity) join ("guilds_games","guilds_id","games_id") to (_.games)


  def constructor(implicit m: ValuesMap) = {
    new Guild(name, games, users, id) with Stored
  }
}


object GameEntity extends Entity[Int, NaturalIntId, Game]("games"){

  val id = key("id") autogenerated (_.id)
  val name = column("name") to (_.name)
  val publisher = column("publisher") to (_.publisher)
  val website = column("website") to (_.website)
  val tt = manytomany(TournamentTypeEntity) join ("games_tournamenttypes","games_id", "tournamenttypes_id") to (_.tournamentTypes)


  def constructor(implicit m: ValuesMap) = {
    new Game(name, publisher, website, tt, id) with Stored {
    }
  }
}

You may remember from my previous posts that I am using json4s to convert my DOs to and from JSON.

In my app when a Guild is created the JSON is extracted from a POST and turned into a clean case object. Here's an example of the JSON to be extracted

{
  "name": "A Guild",
  "games": [
    {
      "name": "Halo 3",
      "publisher": "Bungie",
      "website": "http://bungie.com",
      "tournamentTypes": [
        {
          "name": "Deathmatch",
          "userPlay": true,
          "teamPlay": false,
          "id": 1
        },
        {
          "name": "Team Deathmatch",
          "userPlay": false,
          "teamPlay": true,
          "id": 2
        },
        {
          "name": "Capture The Flag",
          "userPlay": false,
          "teamPlay": true,
          "id": 3
        }
      ],
      "id": 1,
    },
    {
      "name": "League of Legends",
      "publisher": "Riot Games",
      "website": "http://lol.com",
      "tournamentTypes": [],
      "id": 2,
    }
  ]
}

As you can see I return the full Game objects with the Guild object. json4s does the job of extracting the array of Games into a List[Game] as well as creating the clean Guild case object.

The problem I'm running in to is that when I do an insert of the extracted Guild mapperdao also inserts new Game objects even though they have IDs. It then also duplicates TournamentType objects. Essentially it creates a whole new set of Games/TournamentTypes even though they are configured as NaturalID entities.

Are they seen as new objects because I didn't first retrieve them from the DB with mapperdao's entity object wrapped around them? Do I need to change the autogenerated configuration for the id column and manage it myself? Or switch to surrogateIds?

My main question is: How can I get mapperdao to recognize the Games are persisted objects because they have IDs and not insert them as new objects?

Thanks!

[Kostas Kougios]

indeed that's due to that these object instances were not loaded via mapperdao.

mapperdao, when loading instances, it uses the contructor method and notice that it does: new Guild(...) with Stored . The Stored alias is adding to Guild all the information mapperdao needs to track changes & track the updates to the object.

when you deserialize json, that state is not present and mapperdao thinks it is a new entity (it doesn't use the id to figure that out because mapperdao can work with classes that don't have an id till they are loaded from the database)

I'll try to have a look if there is a solution to this and let you know.

--
You received this message because you are subscribed to the Google Groups "mapperdao" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapperdao+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matt Duncan]

Thanks for the fast reply! That makes sense. For now I'm getting around it by recreating the List after extraction

    val extractedGuild = parsedBody.extract[Guild]
    val persistedGames = extractedGuild.games.map { x => gameRepo.get(x.id).get}
    val newGuild = extractedGuild.copy(games = persistedGames)

but it's not ideal as it defeats some of the convenience of extraction.

While mapperdao is a fantastic ORM (seriously, love it) I think I may be pushing its limits. If my data wasn't as highly relational I'd probably migrate to a NoSQL solution like MongoDB where everything is json to begin with. 

          "teamPlay": <span style="color: #0

...

[Kostas Kougios]

what exactly are you trying to insert from the json data? Can your json data contain new items (ones that don't exist into your database)? Can your json data also contain existing items? Do you need to update existing items?

Cheers

--

[Matt Duncan]

In an ideal world I would be able to do all three. I'd love to be able to represent my domain models as json that I can then manipulate from the client, return to the server, extract into DOs, and then recognize persisted objects based on whether they have an ID or not. I'm afraid that's a bit too simplistic though and I'm dealing with too many disparate systems to make it work automagically. 

Practically what I what like is to standardize my REST api so that manipulation of child objects/relationships takes place on a different api call. And then make parent object creation/manipulation ignore relationship sets and child objects all together.

EX - Creating a new Guild -> POST site/guild

      - Add a game to a Guild -> PATCH site/guild/3/game

      - Remove a game from a guild -> DELETE site/guild/3/game/2

This way I don't have to deal with detecting child collections/object as new or existing(or at all, really). This would also solve the problem I'm having in this thread. The problem with that approach though is that json4s extracts the entire object and there's no easy way to tell it to ignore property X when serializing/extracting. (There is a way but it's boilerplate heavy, promotes tight coupling, and time-consuming to write.) 

Without using the hard method my other alternative is to write default values for all my DOs parameters so json4s doesn't get mad when extracting new objects from JSON as well as manually extract all object properties and then manually copy them into a retrieved persisted entity based on the ID from the rest call when manipulating them. (which is just as time-consuming and coupling as the first method)

[Matt Duncan]

P.S. I'm currently using a combination of those two approaches and trying to write as little boilerplate and coupling as possible. My business layer is going to need a thorough refactoring after the first release..

[Konstantinos Kougios]

Ok, had a better look.

At the moment such an update is not straight on supported by mapperdao. Since your json-deserialized entities are detached from mapperdao, it can't potentially know what needs to be inserted and what needs to be updated. Imagine if you were writing jdbc code. In order to insert or update your json-deserialized entities, you would first have to do a query for each entity and all child entities to figure out if those are in the database. If some of them are in the database, those would be updated, the rest should be inserted. But you can't avoid the query (even if you know the id, in fact knowing the id is a must in order to merge the detached entities into the database).

Now the MapperDao trait has a merge() method, but it doesn't do a very good job at the moment. It just checks the top-level entity to see if it needs updating or inserting. I will have a look  to see if it can be improved so that it correctly inserts/updates the whole entity tree. (it works as it does right now because mapperdao doesn't force entities to have an id and without the id it can't really merge detached entities).

So for the time being, I believe you're doing the right thing and your future json plan seems right. I'll let you know if I manage to improve mapperdao.

P.S. btw, do you know that i.e. spring rest api doesn't recommend to use the domain classes for json messaging? They recommend separate DTO classes for the json messages, http://spring.io/guides/tutorials/rest/

[Matt Duncan]

Thanks!

      "tournamentTypes":<spa

...