How to resolve a foreign key reference when persisting an entity via JAX-RS?

[Richard Midwinter]

Hi

Say I have a student entity:

@Entity
public class Student {
  @Getter
  @JsonProperty
  @Id
  @GeneratedValue(strategy = GenerationType.IDENTITY)
  private long id;

  @Getter
  @JsonProperty
  @ManyToOne
  private Clazz clazz;

  @Getter
  @JsonProperty
  private String
        firstName,
        lastName;
}

And I want to persist it by POSTing to a JAX-RS class:

@Path("/student")
@Produces(MediaType.APPLICATION_JSON)
  public class StudentResource {
  private StudentDAO studentDao;

  public StudentResource(StudentDAO studentDao) {
    this.studentDao = studentDao;
  }

  @POST
  @UnitOfWork
  @Consumes(MediaType.APPLICATION_JSON)
  public void persist(Student student) {
    this.studentDao.save(student);
  }
}

And my request body looks like:

{
  "firstName": "Joe",
  "lastName": "Bloggs",
  "clazz": 1
}

Then Jackson's going to blow up because:

DEBUG [2015-05-16 11:08:59,144] io.dropwizard.jersey.jackson.JsonProcessingExceptionMapper: Unable to process JSON
! com.fasterxml.jackson.databind.JsonMappingException: Can not instantiate value of type [simple type, class Clazz] from Integral number (1); no single-int-arg constructor/factory method

I understand why I'm getting this error; it can't deserialize the Clazz object from the key. I can't think of a few hacky ways to fix this (e.g. add a Clazz(int id) constructor and make the clazzDao statically available to look up an instance from the database and populate the fields from within that constructor). But can someone offer me a cleaner solution?

Thanks in advance.

Rich

[Ryan Kennedy]

I'd start by not having the Jackson object and the JPA entity use the same class. You're directly tying a dependency from your REST API to your database schema, which means you're in trouble down the road when you want to change one without changing the other.

Then StudentResource can take the class ID from the request and look up the Clazz entity, construct the Student entity with the Clazz instance, and persist it to the database. It's a little bit of extra code, but decoupling the API from the schema will probably be worth it.

Also, even if you could get Jackson to look the Clazz up in the database, would you want it to? For really simple applications it might work. But for more complicated applications you'll likely need to perform checks before blindly assigning students to classes (are they paid up? is there space in the class? have they met the qualifications?). Putting that kind of logic into the deserialization will become a bit cumbersome.

--
You received this message because you are subscribed to the Google Groups "dropwizard-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dropwizard-us...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Richard Midwinter]

Ah, of course. Makes perfect sense - many thanks.

[Marc Hebben]

I would like to ask another question regarding your example. How are you injecting the studentDAO class in the studentRessource?

[Rich Midwinter]

Sure, something like this:

environment.jersey().register(new StudentResource(studentDAO));

within the run method of the dropwizard Application.

On Monday, 18 May 2015, Marc Hebben <mheb...@googlemail.com> wrote:
> I would like to ask another question regarding your example. How are you injecting the studentDAO class in the studentRessource?
>

> --
> You received this message because you are subscribed to a topic in the Google Groups "dropwizard-user" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dropwizard-user/wB5nfvHbUgY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to dropwizard-us...@googlegroups.com.

[Gene Trog]

An option, to add to Ryan's suggestion, is to use an object mapper framework, like Orika or something similar and then have a separate 'DTO' class with all your Jackson annotations and a separate 'Entity' class with all your JPA annotations.  While it does as a bit of duplication, this will give you the ability to export just your DTO classes for use by clients separately, without having to pull in JPA dependencies.  Another useful reason for doing mapping like this is so your DTO the world sees can be different from the entity your database uses to persist.  I've tried out a few mappers and personally liked Orika because it will automatically map most things through simple conventions, allows custom mapping behavior, and, generates byte-code at runtime to do the mapping, so it's fast.

[Rupert Smith]

On Saturday, May 16, 2015 at 3:29:43 PM UTC+1, Ryan Kennedy wrote:

I'd start by not having the Jackson object and the JPA entity use the same class. You're directly tying a dependency from your REST API to your database schema, which means you're in trouble down the road when you want to change one without changing the other.

If you do that, then you need a mapping layer. Basically you end up writing code that does:

entityStudent.setName(representationStudent.getName());

and the opposite on the way out. You end up with 2 mappers, each of which has long lists of get/set, get/set. get/set, which looks like a bad code smell to me. An alternative might be to use a mapping tool like Moxy, but it will always need customization so often doesn't quite do it.

Personally I think mapping logic is unnecessary. Consider this:

1. Your data model is presented at the API 1:1 with the database model. An example might be a users profile, which they own. They can edit it in whatever way they like, so let them. In this case mapping logic is not needed because the API and database model are the same and always 1:1.

2. You API presents a transactional model to the user. A bank account is an example. The user cannot simply post entries to their account (or I would be very rich), instead they ask to make transactions between accounts, following the principle of double account book-keeping, which says that money can neither be created or destroyed (*ahem* unless of course you run the banks and make billions crashing the economy losing everyone's money and printing more). In this case mapping logic is not needed because you have to write some business logic to handle this anyway.

Read a book on Hibernate. You will see that a big part of its value is to eliminate the need to have a separate layer of value objects. In case 1 you don't need them, in case 2 you do, but they look completely different to your database objects, so you are not duplicating the same class in different layers.

 

Then StudentResource can take the class ID from the request and look up the Clazz entity, construct the Student entity with the Clazz instance, and persist it to the database. It's a little bit of extra code, but decoupling the API from the schema will probably be worth it.

Yes, and no. The Clazz must be looked up by its ID in the SudentResource. Yuk, yuk, yuk to decoupling such a simple API from the schema though. You may need it eventually, but don't do that until you do. Its much better to avoid doing that, since it helps to straighten out your thinking around a lot of other issues if you keep it simple. Mostly you won't need to separate them out, and will just end up having to maintain lots of boiler plate code if you do.

And my request body looks like:

{
  "firstName": "Joe",
  "lastName": "Bloggs",
  "clazz": 1
}

Pass this instead:

{
  "firstName": "Joe",
  "lastName": "Bloggs",
  "clazz": { id: 1 }

}

Then in your Resource class

saveStudent(Student student) {

   // Look up the Clazz reference.

   Clazz clazz = clazzDAOfindById(student.getClazz().getId());

   student.set(clazz);

   studentDao.update(student);

}

====

Now consider that your model is part of your API. Put your API and model in a separate module in your build, then clients can re-use them. You could also put them in one module each for even greater flexibility. That is you could have 2 APIs that work with the same model; one for a content store, one for search for example.

I think the reason people go for the mapping logic approach, is that they think, oh its wrong to put my database model in the API, since it is a lower down layer of the stack. In case 1, no it is not wrong, that user profile data model is perfectly ok for the user to interact with directly. In case 2, yes it is true, the user has no business directly modifying their own bank account, and must be given a transactional model instead.

What you are effectively doing in an API is slicing an object model up into document style fragments for the purpose of serializing it over the network. Draw a picture of your data model, and slice it up into suitably sized fragments. 

A reasonable rule of thumb for doing this is: If one object holds a uni-directional reference to another, it owns that other as a dependant, and the relationship is by composition. If two objects hold bi-directional reference to each other, neither owns the other, and the relationship is by aggregation. This is a rule of thumb only, and not always true, but good enough to get you most of the way there.

Draw the lines that slice up your data model through the aggregation relationships and around the composition ones. Build the API with one CRUD service per document fragment root, and you have a pretty good starting point for a nice API.

[Rupert Smith]

On Monday, May 18, 2015 at 3:38:34 PM UTC+1, Rupert Smith wrote:

saveStudent(Student student) {

   // Look up the Clazz reference.

   Clazz clazz = clazzDAOfindById(student.getClazz().getId());

   student.set(clazz);

   studentDao.update(student);

}

Let me just tidy that up a bit:

 

public void saveStudent(Student student) {

   // Look up the Clazz reference.

   Clazz clazz = clazzDAO.findById(student.getClazz().getId());

   student.setClazz(clazz);

   studentDao.update(student);

}

[Ryan Kennedy]

Rupert:

 in this approach, how would you handle the following two situations? These are honest questions. I'm not trying to be argumentative. I actually think these pattern discussions are really useful for everyone to hear and participate in.

1) I want to share the API representation classes with other projects (imagine a servicename-api module), so those projects don't need to write and maintain their own Jackson bindings. If I pass around JPA entities, now I'm forcing another service to take JPA as a runtime dependency. Obviously if I pass around Jackson annotated objects I'm forcing Jackson as a dependency, but I find developers are happier taking on a particular [de]serialization implementations than infrastructure (database classes). I'm sure there's several camps in conversations around whether system components ought to be sharing this code or not, so we can have that conversation as well if folks are interested.

2) I want to mix-in details from another service/source into the API response. For instance, students come from the database but maybe their presence information (online, offline, away, etc) comes from another service (another dropwizard service, for instance). You want the presence information in the API response but not in the database. Now you have to create and populate a bunch of @Transient properties on the entity objects before sending them back to the client.

Ryan

--

[Tatu Saloranta]

For what it is worth, Jackson can also do simple conversions between types using

ResultType result = mapper.convertValue(inputTypeInstance, ResultType.class);

which is roughly equivalent to serializing 'inputTypeInstance' as external format (like JSON), and deserializing into ResultType.

This works in cases where actual property names match, or can be configured via annotations (possibly mix-ins) and/or naming conventions; and where structures are compatible as-is or changeable with minor transformations that Jackson supports (simple wrapping, single-element arrays<->elements).

Jackson extension called Mr Bean can also materialize implementations of "bean-like" interfaces, so another possibility is to use pure interface-based DTO hierarchy.

It is also perfectly possible to keep most (and sometimes all) Jackson dependencies out of value classes using, even if annotation-based configuration is needed, by:

- Mix-in annotations (associating Jackson annotations with value types without adding them in value types)

- Annotation bundles: define your own Annotation types as sort of aliases for Jackson annotations, so the only dependency is from these annotation types but not from value types themselves.

I realize there are frameworks that are focused on mapping with more extensive structural transformations, but vast majority of Jackson functionality has nothing to do with JSON: instead it is for mapping of structured data to and from POJOs.
Given that DropWizard has first-class integration with Jackson, there are some benefits to makning full use of what is already integrated and available.

-+ Tatu +-

[Rupert Smith]

On Monday, May 18, 2015 at 4:21:11 PM UTC+1, Ryan Kennedy wrote:

Rupert:

 in this approach, how would you handle the following two situations? These are honest questions. I'm not trying to be argumentative. I actually think these pattern discussions are really useful for everyone to hear and participate in.

1) I want to share the API representation classes with other projects (imagine a servicename-api module), so those projects don't need to write and maintain their own Jackson bindings. If I pass around JPA entities, now I'm forcing another service to take JPA as a runtime dependency. Obviously if I pass around Jackson annotated objects I'm forcing Jackson as a dependency, but I find developers are happier taking on a particular [de]serialization implementations than infrastructure (database classes). I'm sure there's several camps in conversations around whether system components ought to be sharing this code or not, so we can have that conversation as well if folks are interested.

Yes, I have the same issue. I am using Hibernate only, but I believe the same can be done in a JPA portable way too. What I do is to use the hibernate .hbm.xml file and put the mapping in there. I think with JPA you can define an orm.xml file for the same purpose?

Everyone hates the .xml file and prefers annotations, but I prefer to use the .xml file than having that mapping logic around, or forcing the JPA/Hibernate dependency on the caller.

2) I want to mix-in details from another service/source into the API response. For instance, students come from the database but maybe their presence information (online, offline, away, etc) comes from another service (another dropwizard service, for instance). You want the presence information in the API response but not in the database. Now you have to create and populate a bunch of @Transient properties on the entity objects before sending them back to the client.

Transients is one way of doing it.

Another is simply to combine the entity model together with some other non-entity classes to form the representation model, in the ordinary way that you build up a data model out of object. Like:

class StudentWithStatus { // This is not an entity, representation only.

   private Student student; // The entity.

   private boolean online;

}

You can also have the opposite situation, here the fields of address will get mapped onto columns within the STUDENT table:

class Address { } // This is not an entity, its a representation of a re-usable address format.

class Student {

    private Address address; // This is not an entity-relationship, its an address embedded into an entity.

    ...

}

But yes, there does come a point where the representation and entities are sufficiently diverged. I suppose the choice comes down to which is messier, two layers or one. My personal preference is that you should almost always start out with one layer for entities and re-use them in the representation.

This is a more bottom-up approach to API design, but it can lead to simpler and more complete APIs from the outset. For example, if I wanted to update a students address using this approach, or to update a students grades, or to update a students fees account, this is all handled by:

class StudentService {

   void update(Student student);

Whereas a more top-down approach might give you:

class StudentService {

   void setAddress(Long studentId, Address address);

   void setGrades(Long studentId, Grades grades);

   void payFees(Long studentId, BigDecimal amount);

of course, my bottom up approach may not be the right one for this student admin service. Perhaps a student can set their own address, but only a professor can update their grades. By exposing an update service for both, I have to put business logic in it to determine whether the calling user has permission to make the update by checking what fields they have set on the Student object vs what the current values are. 

In this case I would recognize that the API is a transactional one, not a 1:1 mapping. I would remove the update() method from the service API (but leave it on the DAO) and create the transactional methods with appropriate roles/permissions.

What I see all the time is people assuming they need a transactional style API as the default position, diving in and creating one, along with reams of mapping logic, and end up with an inconvenient and less maintainable API, that in many cases they don't even need because their application doesn't call for it.

Top-down and bottom-up API design can happily co-exist. I almost always start by sketching out a bottom up API, and then refining it to a more top-down style as and when it is required.

Rupert

[Rupert Smith]

On Thursday, May 21, 2015 at 12:22:28 PM UTC+1, Rupert Smith wrote:

What I see all the time is people assuming they need a transactional style API as the default position, diving in and creating one, along with reams of mapping logic, and end up with an inconvenient and less maintainable API, that in many cases they don't even need because their application doesn't call for it.

To give an example. I worked on a government project where users had accounts with a fairly sizeable data model. The account could be modelled as a single document, even though it was fully mapped down onto relational tables.

They added specific operations for things like updating an address, updating bank details, adding a new employee to the users account and so on. All of these operations could be performed by any system user with admin privileges. 

One argument that was put to me, is that it is inefficient to pass around the entire account model, when simply changing an address. As if passing around a few Ks of data was going to make any difference, on an application with only tens of administrators using it.

Another argument, was that more fine grained concurrency may be required, so that one admin could update an address whilst another updates a bank account. Except of course this never happens (and could be handled automatically anyway by a variety of techniques; optimistic locking, last update wins, or merging).

In the old days, the users account would be on paper and held in a file. Want to change an account? just pull the whole file and take it to your desk, make the updates, then put it back. This is a very simple approach, and much of the time it is going to be all that is required. It is quick to develop and requires a lot less code (I turned a 10,000 line application into a 1,000 line one that did the same thing). 

The complexity can easily be added in as and when it is needed to handle specific business logic, or performance requirements.

Rupert