Several custom XML and JSON writers for several domain classes

[Miguel Expósito]

Hi!

I'm trying to write many custom XML and JSON providers, one for each domain class. However, only one custom class is working properly; the others are not being taken into account. For example, I have two classes in providers directory:

CustomDomainClass1Writer.groovy : tests for class DomainClass1 and collections of this class
CustomDomainClass2Writer.groovy : tests for class DomainClass2 and collections of this class

Only DomainClass1Writer is working, the XML and JSON output for the other class is the default one.

Am I missing something?

Thanks in advance and congrats for your plugin, it makes my life with Rest WS really easier!

[Martin Krasser]

Hi Miguel,

can you please share the code of your providers and Config.groovy?

Am 08.08.12 23:01, schrieb Miguel Expósito:

--
Martin Krasser

blog: http://krasserm.blogspot.com
code: http://github.com/krasserm
twitter: http://twitter.com/mrt1nz

[Miguel Expósito]

Hi again, Martin:

My Config.groovy related to the plugin:

org.grails.jaxrs.doreader.disable=true

org.grails.jaxrs.dowriter.disable=true

grails.converters.default.circular.reference.behaviour="INSERT_NULL"

grails.converters.default.pretty.print=true

grails.converters.json.default.deep=true

grails.converters.xml.default.deep=true

My custom providers are a little big, so I'm cutting what I think is irrelevant code:

CustomDomainClass1.groovy:

package application

import java.util.prefs.BackingStoreException;

import javax.ws.rs.Produces

import javax.ws.rs.ext.Provider

import groovy.xml.StreamingMarkupBuilder

import groovy.xml.MarkupBuilder

import groovy.xml.XmlUtil

import groovy.json.JsonBuilder

import grails.converters.JSON

import org.grails.jaxrs.support.DomainObjectWriterSupport

@Provider

@Produces(['text/xml', 'application/xml', 'text/x-json', 'application/json'])

class CustomDomainClass1Writer extends DomainObjectWriterSupport {

protected Object writeToXml(Object obj, OutputStream entityStream, String charset) {

if (obj instanceof List) { 

if (obj.head() instanceof CustomDomainClass1) {

def writer = new OutputStreamWriter(entityStream, charset)

def builder = new StreamingMarkupBuilder()

builder.encoding = charset

def xml = builder.bind(){

mkp.xmlDeclaration()

... 

}

}

XmlUtil.serialize(xml, writer)

writer.close()

}

}

else if (obj instanceofCustomDomainClass1){

def writer = new OutputStreamWriter(entityStream, charset)

def builder = new StreamingMarkupBuilder()

builder.encoding = charset

def xml = builder.bind(){

mkp.xmlDeclaration()

  ...

}

XmlUtil.serialize(xml, writer)

writer.close()

}

else {

super.writeToXml(obj, entityStream, charset)

}

}

protected Object writeToJson(Object obj, OutputStream entityStream, String charset) {

if (obj instanceof List) {

if (obj.head() instanceof CustomDomainClass1) {

def writer = new OutputStreamWriter(entityStream, charset)

def builder = new JsonBuilder()

...

}

writer << builder.toPrettyString()

writer.close()

}

}

else if (obj instanceof CustomDomainClass1){

def writer = new OutputStreamWriter(entityStream, charset);

def builder = new JsonBuilder()

...

writer << builder.toPrettyString()

writer.close()

}

else {

super.writeToJson(obj, entityStream, charset)

}

}

}

All my custom classes have the same structure, so one can imagine the others...

Thanks for your time

[Martin Krasser]

You need to override

- https://github.com/krasserm/grails-jaxrs/blob/jaxrs-0.6/src/groovy/org/grails/jaxrs/support/DomainObjectWriterSupport.groovy#L99 or
- https://github.com/krasserm/grails-jaxrs/blob/jaxrs-0.6/src/groovy/org/grails/jaxrs/support/DomainObjectReaderSupport.groovy#L77

to control which provider is selected.

Cheers,
Martin

Am 10.08.12 14:51, schrieb Miguel Expósito:

[Miguel Expósito]

Hi again, Martin:

I did as you said and managed to implement a custom isWriteable method for each class, controlling instances from both class and collection of class. Now I have one separated provider class for every domain class. Thanks a lot for your indications!

I've realised the providers are being loaded in alphabetical order; is it possible to specify a different order? I mean, I have URI methods which are invoked more frequently than others, so I'd rather that their provider classes were loaded in first place, in order to avoid unnecessary if-else structures. I suppose I can change the name of the classes too, but I want to do it in the best possible way... what do you think?

Thanks again for your help, I'm learning a lot with your plugin

[Miguel Expósito]

Hi again:

I've been working on this, and come to the conclusion that grails built in JSON and XML parsers are too slow for me (I have several compositions and many to many relationships).

Now I'm trying to change to Jackson JSON and XML parser. In order to get as much performance as possible, an ObjectMapper instance from Jackson library must be reused.  It seems FastXML people have written a custom provider for jax-rs using Jackson as serializer:

https://github.com/FasterXML/jackson-jaxrs-xml-provider
https://github.com/FasterXML/jackson-jaxrs-json-provider

Any experience with this? Is this as simple as throwing Jackson providers classes into grails providers directory? Is there any problem with the splitting of the provider in json and xml

[Miguel Expósito]

OK, I've just figured myself. To use jackson -jaxrs-json provider, the following dependencies mus be added to BuildConfig.groovy:

runtime 'com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.0.5'

And then, the following bean registered in resources.groovy:

jacksonJsonProvider(com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider){

}

This is basic configuration, but it'll just do.

May I suggest to include jackson and jackson-jaxrs-provider in the plugin in order to take advantage of jackson library great performance?

[Martin Krasser]

Hi Miguel,

thanks fo sharing your experiences. Comments inline ...

Am 04.09.12 09:21, schrieb Miguel Expósito:

OK, I've just figured myself. To use jackson -jaxrs-json provider, the following dependencies mus be added to BuildConfig.groovy:

runtime 'com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.0.5'

And then, the following bean registered in resources.groovy:

jacksonJsonProvider(com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider){

}

This is basic configuration, but it'll just do.

May I suggest to include jackson and jackson-jaxrs-provider in the plugin in order to take advantage of jackson library great performance?

Jersey (http://jersey.java.net) already uses jackson for its JSON provider (which is also a plugin dependency, see https://github.com/krasserm/grails-jaxrs/blob/master/grails-app/conf/BuildConfig.groovy#L41). How does the custom JSON provider you propose differ from Jersey's default JSON provider? Have you tried using that provider?

[Miguel Expósito]

Hi Martin:

Perhaps I misunderstood something here. I thought jax-rs Grails plugin did the JSON marshalling using Grails built-in Json serializer. I was having problems with deep cloning and circular references, and needed more flexible control over my objects to serialize, so did some research and ended up using Jackson 2.0 with its providers. For what I've seen, jersey-json 1.12 still uses Jackson 1.X; Jackson 2.X is said to offer better performance and allows XML marshalling too. 

Jackson 2.0 explicitly allows me using JSON serializing annotations in my domain classes, as well as custom serializers for any property. The truth is that I didn't know Jersey uses jackson; I guess Jersey using Jackson 2.0 version is a matter of time. What I don't completely get is the way serialization is made: if jax-rs plugin is using jersey, and jersey is using jackson, and serialization is accomplished via jackson providers and not Grails built-in providers, is it possible to annotate domain classes with Jackson annotations too?

Thanks for your time and my apologies for my inexperience and possible mix of concepts here.

[Martin Krasser]

Am 05.09.12 09:15, schrieb Miguel Expósito:

Hi Martin:

Perhaps I misunderstood something here. I thought jax-rs Grails plugin did the JSON marshalling using Grails built-in Json serializer.

Only if they are Grails domain objects. If you disable the custom grails-jaxrs providers and annotate your domain objects properly (so that they're recognized by the JSON provider shipped with jersey) then you'll have jackson-based JSON (un)marshalling.

I was having problems with deep cloning and circular references, and needed more flexible control over my objects to serialize, so did some research and ended up using Jackson 2.0 with its providers. For what I've seen, jersey-json 1.12 still uses Jackson 1.X; Jackson 2.X is said to offer better performance and allows XML marshalling too.

Ah ok, didn't know about these performance differences. In this case it makes absolutely sense to use them.

Jackson 2.0 explicitly allows me using JSON serializing annotations in my domain classes, as well as custom serializers for any property. The truth is that I didn't know Jersey uses jackson; I guess Jersey using Jackson 2.0 version is a matter of time. What I don't completely get is the way serialization is made: if jax-rs plugin is using jersey, and jersey is using jackson, and serialization is accomplished via jackson providers and not Grails built-in providers, is it possible to annotate domain classes with Jackson annotations too?

See my first comment.

[Miguel Expósito]

Hi again, Martin:

Thanks a lot for your explanations, it's much clearer for me. Some references:

http://wiki.fasterxml.com/JacksonRelease20Features
https://github.com/FasterXML/jackson-jaxrs-json-provider/wiki

I'm trying now jackson-jaxrs-xml-provider, with no success atm. I'll post if I make some advance.

Thanks again and congratulations for your work; the plugin is really awesome!

[Mike Wood]

Hi,

Been reading this thread and was also looking to switch to using jackson annotations in my domain classes. From what I understood from the above is that jackson v1 is employed by this plugin as long as you disable the default providers (with the two lines below). Also that jackson v1 will work with annotations but v2 is faster/better etc, and that I do not need to register the provider manually.

I disabled the org.grails providers with these:

org.grails.jaxrs.doreader.disable=true

org.grails.jaxrs.dowriter.disable=true

I added the following annotations to my grails domain class:

class User implements DateableIF

{

    static transients = [ 'groupAdmin' ]

    static belongsTo = Group

    static hasMany = [groups: Group, folders: Folder, projects: Project]

    @JsonProperty('uname')

    String username

    String password

    @JsonProperty('emailaddress')

    String email

    String firstName

    String lastName

.....

I set the Accept: application/json in the request header but I am getting the following exception:-

ERROR container.ContainerResponse  - A message body writer for Java class com.blah.User, and Java type class com.blah.User, and MIME media type application/json was not found

ERROR container.ContainerResponse  - The registered message body writers compatible with the MIME media type are:

application/json ->

  org.grails.jaxrs.provider.JSONWriter

  org.grails.jaxrs.provider.DomainObjectWriter

*/* ->

  com.sun.jersey.core.impl.provider.entity.FormProvider

  com.sun.jersey.core.impl.provider.entity.MimeMultipartProvider

  com.sun.jersey.server.impl.template.ViewableMessageBodyWriter

  com.sun.jersey.core.impl.provider.entity.StringProvider

  com.sun.jersey.core.impl.provider.entity.ByteArrayProvider

  com.sun.jersey.core.impl.provider.entity.FileProvider

  com.sun.jersey.core.impl.provider.entity.InputStreamProvider

  com.sun.jersey.core.impl.provider.entity.DataSourceProvider

  com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General

  com.sun.jersey.core.impl.provider.entity.ReaderProvider

  com.sun.jersey.core.impl.provider.entity.DocumentProvider

  com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider

  com.sun.jersey.core.impl.provider.entity.SourceProvider$SourceWriter

  com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General

  com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General

Not sure if I should see the jackson providers in there or not. They are in the classpath - at least my IDE was happy to locate the annotation class for @JsonProperty so assumed all was OK there (v1.9.2 to be precise).

Thanks,

Mike.

[Mike Wood]

I've had a tough time trying to get this plugin to be happy with jackson 2.1.0 (endless chasing of spring bean serialization issues and alikes). I tried co-existence as I needed 2.1.0 since Swagger needs v2 but I had to basically remove swagger from the picture and get back to jackson 1.9.2. I then ran into the problems of all the booleans in groovy classes having both the isXXX and getXXX getters. Trying to get the @JsonIgnoreProperties to ignore these - does not work. I am resorting to using custom JSON writers just as the original poster did. Seems like all this clever stuff gets too tangled and some good ol' hand-coding is necessary just to ease frustrations from the finicky declarative programing</vent>. It is actually less lines of code too.... Plus I anticipate having different versions of the API available so that rules out binding in the single domain class.

A further note on Swagger - since it supposedly can read the annotations too it would have been a good self-documenting solution but again I'll hand code a JSON API spec for it in less time that it's complicated annotations. If I can figure out a way to make swagger happy with jackson 1.9.2 I'll post back here. Would love to hear any similar findings of others.

[Mike Wood]

I have an update.... I had installed Swagger API support framework before grails-jaxrs plugin. Swagger required me to add jersey as a servlet in web.xml (which I did). When I installed the plugin I assumed it would see this servlet. What was happening is that since I was only looking at the template web.xml (where I added the jersey servlet) I did not see that the plugin was actually adding in a filter for jaxrs (org.grails.jaxrs.web.JaxrsFilter) so there were actually two contexts for jersey.

Anyways seems like since the filter operates on /* it is working for both my APIs and swagger just fine. Both seem to be working with Jackson 2.1.0 as well. I also dug deeper into the Json annotations and found the following class level annotation that means there's no auto-detection from jackson and everything to be serialized must be explicitly declared so no more isXX/getXX issues for boolean fields.

@JsonAutoDetect(fieldVisibility=JsonAutoDetect.Visibility.NONE,

                getterVisibility=JsonAutoDetect.Visibility.NONE,

                isGetterVisibility=JsonAutoDetect.Visibility.NONE)

Next challenge is to get OAuth 2.0 working for these APIs...

[Miguel Expósito]

Hi, Mike:

I remember seeing that error too. In my Config.groovy I have the following related config lines:

org.grails.jaxrs.doreader.disable=true

org.grails.jaxrs.dowriter.disable=true

org.grails.jaxrs.provider.init.parameters=[

'com.sun.jersey.api.json.POJOMappingFeature' : 'true',

'com.sun.jersey.config.property.packages': 'nameofyourpackage'

]

And in BuildConfig.groovy:

 dependencies {

runtime ('com.sun.jersey:jersey-json:1.8') {

excludes 'stax-api'

}

    }

This way I have no errors with providers. Is your config like this?

[Miguel Expósito]

Hi again:

I didn't knew about swagger, but now that I've read about it, I think it could be really useful and cool in order to document my restful API. Did you manage to get it working using Jackson version required by grails jax-rs plugin? I finally decided to use default version, since jackson 2.1 didn't solve some of my problems (I opened a request for enhacement in one of the components).

Could you share your BuildConfig.groovy dependencies in order to get it to work? If I didn't misunderstood from your posts, you don't have to modifiy web.xml since it's enough with the filter added by grails jax-rs plugin, do you?

Thanks for your sharing!

[Mike Wood]

I am using swagger and jaxrs with Jackson 2.1.0 as follows:-

BuildConfig.groovy 

    dependencies {

        compile 'org.codehaus.jackson:jackson-core-asl:1.9.2'

        compile 'org.codehaus.jackson:jackson-mapper-asl:1.9.2'

        compile 'com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.1.0'

    }

Config.groovy

org.grails.jaxrs.dowriter.disable = true

since I actually have both old and new Jackson I have to select the new (fasterxml) when adding annotations to my domain classes.

I took my <servlet> out of my web.xml (template) and just let grails add in the <filter> instead. I am not able to pass in values via the ServletConfig though so I need to find a way to pass parameters to the swagger stuff since it won't read the filter init-params. Maybe I can use Config.groovy.

[Miguel Expósito]

Hi again, Mike:

Are you using swagger-core + swagger-ui or swagger-jaxrs?

I've tried with the latter : 

 runtime 'com.wordnik:swagger-jaxrs_2.9.1:1.1.0'

But it's downloading scala libraries I'm afraid I dont need...

I think I would use the same version of Jackson in all its relative packages...

Thanks

[Mike Wood]

Here's what I have in my grails lib (and swagger-core does use scala lib make sure the version matches too).

scala-library-2.9.1

swagger-annotations-2.9.1

swagger-core-2.9.1

swagger-jaxrs-2.9.1

swagger-jaxrs-utils-2.9.1   <- don't know for sure I need this

Then I have swagger-ui installed as javascript under my web app assets. I could not find a built version so I got cake et.al to build it. I think there are prebuilt packages out there. They just updated it to version 1.1 too.

[Miguel Expósito]

Hi again, Mike:

I've managed to get it working almost completely. This is my Buildconfig.groovy now:

runtime ('com.sun.jersey:jersey-json:1.8') {

excludes 'stax-api'

}

runtime 'com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.1.0'

runtime ('com.wordnik:swagger-core_2.9.1:1.1.0') {

excludes 'jackson-annotations', 'slf4j-log4j12'

}

runtime ('com.wordnik:swagger-jaxrs_2.9.1:1.1.0') {

excludes 'jersey-client', 'jersey-core', 'jersey-server', 'jersey-servlet', 'servlet-api', 'jsr311-api', 'junit'

}

I've managed to get the resource listing after annotating resources classes with @Api annotation. I've been following doc here: https://github.com/wordnik/swagger-core/wiki/java-jax-rs

However, I'm not being able to change the listing path because the listingClass parameter in @Api annotation, since it always throws a "Class not found" error. The resource class I'm trying to look for can be found by jersey and is in the correct package, but SwaggerContext.scala is unable to find it.

Have you been into this issue? Any ideas?

Thanks again

[Mike Wood]

Oh yes I have that problem! Seems like grails adds in a couple more levels of class loaders. Basically it comes down to they [Grails root class loaders] won't let classes inside jar files see those classes outside of jars. I've tried some workarounds like exploding the swagger jars - but never found a place I could put the class files to be included. Or putting my apps classes into a jar. I was also thinking about re-writing SwaggerContext.scala to also use the grailsApplication classloader which will find all classes - but I am not a scala person.

On the bright side when I build a war file and deploy to tomcat 6 it does work. So once outside of the the grails framework it functions. Hope that helps.

I'll switch to your config - much cleaner that what I am doing with jars in the lib dir. Appreciate you sending them!

Mike.

[Gregory Krasnow]

I added this to my BuildConfig.groovy and I created the API listing class.  However, is there something I still need to do to get the servlet registered?  Do I need to edit the web.xml template?  Thanks.

[Mike Wood]

You should not have to. For me it is working via the <filter> that the grails-jaxrs plugin inserts automatically. You do need to pass the init-params via the lines in Config.groovy though - Swagger does need them I believe. Here's what I have in mine:-

org.grails.jaxrs.doreader.disable = true

org.grails.jaxrs.dowriter.disable = true

org.grails.jaxrs.provider.init.parameters=[

        'com.sun.jersey.api.json.POJOMappingFeature' : 'true',

        'swagger.api.basepath': 'http://myhost:8080/myappbase',

        'api.version': '0.2'

]

[Miguel Expósito]

Oh, dear, I should have figured it out. Thanks for sharing your experience, I think I'll test this part deploying directly on tomcat.

Probably you can adjust even more the excluded libraries of that configuration, but it seems it works showing no errors, so it just does fine for me.

So, Martin, wouldn't be nice including Jackson 2.X and swagger in grails jax-rs plugin? ;-)

[Miguel Expósito]

I agree with Mike, you should,'t need anything else... same configuration here.

[Martin Krasser]

Am 05.11.12 08:21, schrieb Miguel Expósito:

Oh, dear, I should have figured it out. Thanks for sharing your experience, I think I'll test this part deploying directly on tomcat.

Probably you can adjust even more the excluded libraries of that configuration, but it seems it works showing no errors, so it just does fine for me.

So, Martin, wouldn't be nice including Jackson 2.X and swagger in grails jax-rs plugin? ;-)

Sounds good to me. Do you want to open a pull request?

El viernes, 2 de noviembre de 2012 15:09:19 UTC+1, Mike Wood escribió:

Oh yes I have that problem! Seems like grails adds in a couple more levels of class loaders. Basically it comes down to they [Grails root class loaders] won't let classes inside jar files see those classes outside of jars. I've tried some workarounds like exploding the swagger jars - but never found a place I could put the class files to be included. Or putting my apps classes into a jar. I was also thinking about re-writing SwaggerContext.scala to also use the grailsApplication classloader which will find all classes - but I am not a scala person.

On the bright side when I build a war file and deploy to tomcat 6 it does work. So once outside of the the grails framework it functions. Hope that helps.

I'll switch to your config - much cleaner that what I am doing with jars in the lib dir. Appreciate you sending them!

Mike.

[Gregory Krasnow]

Even after doing this,  I still get a 404 when trying to access the /resources.json to get an API listing.  I do not see any exceptions in the log from swagger, so I still am not convinced that the servlet is actually being used.  I feel that something is still missing.

[Gregory Krasnow]

I added a line like this:

'com.sun.jersey.config.property.packages': 'com.myapplication;com.wordnik.swagger.jaxrs' 

When I go to http://localhost:8080/myapp/application.wadl I see the /resources.json in the list, but when I try to browse to /resources.json I get a 404.  I tried to set debug 'com.wordnik.swagger' in my log4j configuration but I am not seeing any information come from there.  

- Greg

[Gregory Krasnow]

Alright... I finally fiddled with enough to get it working. ;-)  One last question (hopefully)... when you list a responseClass in the ApiOperation -- is there a way to limit it to only the @JsonProperty properties in the domain class?  It seems to want to show everything including all the injected grails/gorm methods and fields which makes it quite messy.  Thanks.

- Greg

[Miguel Expósito]

I think Mike posted it on another thread.

I use this in order to serialize all properties:

@JsonAutoDetect(fieldVisibility=JsonAutoDetect.Visibility.NONE,getterVisibility=JsonAutoDetect.Visibility.ANY, isGetterVisibility=JsonAutoDetect.Visibility.NONE)

@JsonIgnoreProperties(['handler', 'errors', 'properties', 'hibernateLazyInitializer'])

If you want to serialize in an explicit way, try this:

@JsonAutoDetect(fieldVisibility=JsonAutoDetect.Visibility.NONE,

                getterVisibility=JsonAutoDetect.Visibility.NONE,

                isGetterVisibility=JsonAutoDetect.Visibility.NONE)

And then annotate the desired properties:

 @JsonProperty('id')

    Integer id

    .....

    @JsonProperty('user')

    User user;

I hope it helps...

[Gregory Krasnow]

Have either of you seen an issue with infinite recursion when running within the grails environment?  It seems that @JsonAutoDetect and @JsonIgnoreProperties are ignored and it tried to serialize through 'errors' and eventually gets a StackOverflow.  It runs just fine as a war file though.  

- Greg

[Gregory Krasnow]

Yes... this is true EXCEPT for the responseClass in the @ApiOperation().  It seems that swagger is using another mechanism to create a JSON representation of the responseClass.  This seems to try to serialize the injected GORM/Hibernate properties and can eventually blow up with a StackOverflow.

On Monday, November 5, 2012 11:40:41 PM UTC-8, Miguel Expósito wrote:

[Gregory Krasnow]

Ever figure out a good workaround so that the application could be run inside of Grails instead of requiring a WAR file?  Thanks.