JSON Response Has Escaped Quotations Using Jackson and JAX-RS Exception Mapper
53 views
Skip to first unread message
Uzh Valin
May 21, 2019, 2:20:57 PM5/21/19
to jackson-user
I have a simple requirement where, if application encounters an exception, my JAX-RS Rest endpoint should return a custom JSON response with 500 HTTP header status.
Data needed to construct the response comes from an object with several properties (see below). The problem is, I am only interested in one or two values from each property (out of several dozens). And I cannot modify any of these models/classes (some have a Jackson annotation for JSON processing, e.g. null properties should be discarded during serialization).
public class MainObject {
private FirstProperty firstProperty;
private SecondProperty secondProperty;
private ThirdProperty thirdProperty;
// other codes
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
public class FirstProperty {
private boolean bol = true;
private double dob = 5.0;
private List<String> subProperty;
// other properties
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
@JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)
public class SecondProperty {
private String str;
private List<String> subProperty;
// other properties
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
@JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)
public class ThirdProperty {
private int intProp = true;
private List<String> subProperty;
// other properties
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
The expected JSON that I should be seeing coming back is on the client side (say, a browser -- testing in Edge):
{
"firstProperty" : { "subProperty" : [ "val1" ] },
"secondProperty" : { "str" : "val2", "subproperty" : [ "val3", "val6" ] },
"thirdProperty" : { "subProperty" : [ "val4" ] }
}
Instead, all my field names and their values have their quotations escaped, and extra double quotes around the entire value, e.g.:
{
"firstProperty" : "{ \"subProperty\" : [ \"val1\" ] }",
"secondProperty" : "{ \"str\" : \"val2\", \"subproperty\" : [ \"val3\", \"val6\" ] }",
"thirdProperty" : "{ \"subProperty\" : [ \"val4\" ] }"
}
Please note the extra `"` before and after the curly brackets. My environment is:
Java 1.8.45
FasterXML Jackson 2.9.8
Spring Boot 2.0.1
RestEasy (JBoss) JAX-RS
JBoss 6.4
I eliminated the majority of "noise" in the code to see at what point this happens. This is the controller:
@Path("/"
public class MainController {
@GET
@Produces(MediaType.APPLICATION_JSON)
@Path("/rest/path")
public MainObject getMainObject throws MyCustomException {
// A service call that throws MyCustomException
}
}
And JAX-RS `ExceptionMapper` where I send the response back:
@Provider
public class MyCustomExceptionMapper extends ExceptionMapper<MyCustomException> {
@Override
public Response toResponse(MyCustomException ex) {
HashMap<String, Object> responseBody = new HashMap<>();
String strEx = ex.getStrEx(); // Comes from SecondProperty.str stored in MyCustomException, not that it matters
// Instantiate an empty object that contains
MainObject obj = new MainObject();
obj.getFirstProperty().setSubProperty(ex.getStrs());
obj.getSecondProperty().setStr(strEx);
obj.getSecondProperty().setSubProperty(ex.getStrs());
obj.getThirdProperty().setSubProperty(ex.getStrs());
responseBody.put("firstProperty", serializeFirstProperty(obj.getFirstProperty()));
responseBody.put("secondProperty", serializeSecondProperty(obj.getSecondProperty()));
responseBody.put("thirdProperty", serializeThirdProperty(obj.getThirdProperty()));
Response response = Response.status(/* 500 status */).entity(responseBody).build();
return response;
}
}
Since I only need to send back a very small subset of overall properties from each of my types, I created a custom `StdSerializer` that would only populate a needed property. For brevity, I only do `serializeFirstProperty()` but they are all more or less identical:
public StdSerializer<FirstProperty> getFPSerializer(FirstProperty firstProperty) {
return new StdSerializer<FirstProperty>(FirstProperty.class) {
@Override
public void serialize(FirstProperty value, JsonGenerator gen, SerializerProvider provider) throws IOException {
gen.writeStartObject();
if (/* there are items in FirstProperty.subProperty */) {
gen.writeArrayFieldStart("subProperty");
for (String str : value.getSubProperty())
gen.writeString(str);
gen.writeEndArray();
}
gen.writeEndObject();
}
}
public <T> ObjectMapper getCustomOM(StdSerializer<?> serializer) {
ObjectMapper mapper = new ObjectMapper();
SimpleModule sm = new SimpleModule();
sm.addSerializer(serializer);
mapper.registerModule(module);
return mapper;
}
Then use these helper methods like:
public String serializeFirstProperty(FirstProperty firstProperty) {
ObjectMapper mapper = getCustomOM(getFPSerializer(firstProperty));
String ser = null;
try { ser = mapper.writeValueAsString(firstProperty); }
catch (JsonProcessingException e) { return null; }
return ser;
}
I have tried countless of configurations with `ObjectMapper`, e.g. `disable(JsonParser.Feature.ALLOW_BACKLASH_ESCAPING_ANY_CHARACTER)` (couldn't find any relevant flag for `JsonGenerator` which I really want to disable in a similar fashion).
Or explicitly returning `Object` from `serializeFirstProperty()`.
Or set custom `StdSerializer`'s `JsonGenerator.setCharacterEscapes(new CharacterEscapes() { //... }` or play around with JAX-RS `Response` at no avail. I always seem to get a "string" value with quotations, e.g.:
"firstProperty" : "{ \"subProperty\" : [ \"val1\" ] }"
If I simply just do
responseBody.put("firstProperty", mapper.writeValueAsString(obj.getFirstProperty()));
somehow this produces the right JSON output, however, it includes a lot of unnecessary properties which I don't want in this exception handling case.
Funny thing is, when I peer into `response` (or `responseBody` map), everything looks right (I don't see values having double quotations).
Please also note that not only I can't modify the models, but some of their properties are instantiated during creation with default values, so not-null inclusion doesn't work, and they will appear in the final JSON if I don't use a custom serialization.
Does anyone know what's causing this escaped and extra quotations?
Data needed to construct the response comes from an object with several properties (see below). The problem is, I am only interested in one or two values from each property (out of several dozens). And I cannot modify any of these models/classes (some have a Jackson annotation for JSON processing, e.g. null properties should be discarded during serialization).
public class MainObject {
private FirstProperty firstProperty;
private SecondProperty secondProperty;
private ThirdProperty thirdProperty;
// other codes
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
public class FirstProperty {
private boolean bol = true;
private double dob = 5.0;
private List<String> subProperty;
// other properties
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
@JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)
public class SecondProperty {
private String str;
private List<String> subProperty;
// other properties
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
@JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)
public class ThirdProperty {
private int intProp = true;
private List<String> subProperty;
// other properties
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
The expected JSON that I should be seeing coming back is on the client side (say, a browser -- testing in Edge):
{
"firstProperty" : { "subProperty" : [ "val1" ] },
"secondProperty" : { "str" : "val2", "subproperty" : [ "val3", "val6" ] },
"thirdProperty" : { "subProperty" : [ "val4" ] }
}
Instead, all my field names and their values have their quotations escaped, and extra double quotes around the entire value, e.g.:
{
"firstProperty" : "{ \"subProperty\" : [ \"val1\" ] }",
"secondProperty" : "{ \"str\" : \"val2\", \"subproperty\" : [ \"val3\", \"val6\" ] }",
"thirdProperty" : "{ \"subProperty\" : [ \"val4\" ] }"
}
Please note the extra `"` before and after the curly brackets. My environment is:
Java 1.8.45
FasterXML Jackson 2.9.8
Spring Boot 2.0.1
RestEasy (JBoss) JAX-RS
JBoss 6.4
I eliminated the majority of "noise" in the code to see at what point this happens. This is the controller:
@Path("/"
public class MainController {
@GET
@Produces(MediaType.APPLICATION_JSON)
@Path("/rest/path")
public MainObject getMainObject throws MyCustomException {
// A service call that throws MyCustomException
}
}
And JAX-RS `ExceptionMapper` where I send the response back:
@Provider
public class MyCustomExceptionMapper extends ExceptionMapper<MyCustomException> {
@Override
public Response toResponse(MyCustomException ex) {
HashMap<String, Object> responseBody = new HashMap<>();
String strEx = ex.getStrEx(); // Comes from SecondProperty.str stored in MyCustomException, not that it matters
// Instantiate an empty object that contains
MainObject obj = new MainObject();
obj.getFirstProperty().setSubProperty(ex.getStrs());
obj.getSecondProperty().setStr(strEx);
obj.getSecondProperty().setSubProperty(ex.getStrs());
obj.getThirdProperty().setSubProperty(ex.getStrs());
responseBody.put("firstProperty", serializeFirstProperty(obj.getFirstProperty()));
responseBody.put("secondProperty", serializeSecondProperty(obj.getSecondProperty()));
responseBody.put("thirdProperty", serializeThirdProperty(obj.getThirdProperty()));
Response response = Response.status(/* 500 status */).entity(responseBody).build();
return response;
}
}
Since I only need to send back a very small subset of overall properties from each of my types, I created a custom `StdSerializer` that would only populate a needed property. For brevity, I only do `serializeFirstProperty()` but they are all more or less identical:
public StdSerializer<FirstProperty> getFPSerializer(FirstProperty firstProperty) {
return new StdSerializer<FirstProperty>(FirstProperty.class) {
@Override
public void serialize(FirstProperty value, JsonGenerator gen, SerializerProvider provider) throws IOException {
gen.writeStartObject();
if (/* there are items in FirstProperty.subProperty */) {
gen.writeArrayFieldStart("subProperty");
for (String str : value.getSubProperty())
gen.writeString(str);
gen.writeEndArray();
}
gen.writeEndObject();
}
}
public <T> ObjectMapper getCustomOM(StdSerializer<?> serializer) {
ObjectMapper mapper = new ObjectMapper();
SimpleModule sm = new SimpleModule();
sm.addSerializer(serializer);
mapper.registerModule(module);
return mapper;
}
Then use these helper methods like:
public String serializeFirstProperty(FirstProperty firstProperty) {
ObjectMapper mapper = getCustomOM(getFPSerializer(firstProperty));
String ser = null;
try { ser = mapper.writeValueAsString(firstProperty); }
catch (JsonProcessingException e) { return null; }
return ser;
}
I have tried countless of configurations with `ObjectMapper`, e.g. `disable(JsonParser.Feature.ALLOW_BACKLASH_ESCAPING_ANY_CHARACTER)` (couldn't find any relevant flag for `JsonGenerator` which I really want to disable in a similar fashion).
Or explicitly returning `Object` from `serializeFirstProperty()`.
Or set custom `StdSerializer`'s `JsonGenerator.setCharacterEscapes(new CharacterEscapes() { //... }` or play around with JAX-RS `Response` at no avail. I always seem to get a "string" value with quotations, e.g.:
"firstProperty" : "{ \"subProperty\" : [ \"val1\" ] }"
If I simply just do
responseBody.put("firstProperty", mapper.writeValueAsString(obj.getFirstProperty()));
somehow this produces the right JSON output, however, it includes a lot of unnecessary properties which I don't want in this exception handling case.
Funny thing is, when I peer into `response` (or `responseBody` map), everything looks right (I don't see values having double quotations).
Please also note that not only I can't modify the models, but some of their properties are instantiated during creation with default values, so not-null inclusion doesn't work, and they will appear in the final JSON if I don't use a custom serialization.
Does anyone know what's causing this escaped and extra quotations?
Tatu Saloranta
May 21, 2019, 2:40:27 PM5/21/19
to jackson-user
Just one question:
responseBody.put("firstProperty", serializeFirstProperty(obj.getFirstProperty()));
responseBody.put("secondProperty", serializeSecondProperty(obj.getSecondProperty()));
responseBody.put("thirdProperty", serializeThirdProperty(obj.getThirdProperty()));
Why do you serialize values? That is where "double-escaping" comes: you are adding JSON String within content to be JSON serialized. Just add values as is
responseBody.put("firstProperty", obj.getFirstProperty());
and contents would get serialized just once.
-+ Tatu +-
Uzh Valin
May 21, 2019, 6:57:53 PM5/21/19
to jackson-user
Ok, but how does Jackson know which custom mapper to call if we are simply putting obj.getFirstProperty() in the response map? I know need one value and would want the serializer to ignore all other properties regardless whether they have been initialized with default values.
Tatu Saloranta
May 21, 2019, 7:08:22 PM5/21/19
to jackson-user
On Tue, May 21, 2019 at 3:57 PM Uzh Valin <blahb...@gmail.com> wrote:
>
> Ok, but how does Jackson know which custom mapper to call if we are simply putting obj.getFirstProperty() in the response map? I know need one value and would want the serializer to ignore all other properties regardless whether they have been initialized with default values.
It doesn't, but what I am saying is that your approach will not work as shown.
>
> Ok, but how does Jackson know which custom mapper to call if we are simply putting obj.getFirstProperty() in the response map? I know need one value and would want the serializer to ignore all other properties regardless whether they have been initialized with default values.
If you add String values, they will be JSON Strings and must be
escaped. That's how Strings are handled -- otherwise Jackson would
have to somehow re-parse String into JSON, and then go back to
serializing contents, adding significant overhead that is not usually
needed. Assuming that String was valid JSON; if not it would either
have throw exception, or quietly determine it has to be used as-is...
or something.
If you want to apply different rules there are a few ways you could
achieve that -- custom serializers are one way -- or you could
serialize-as-String-then-deserialize if you want to apply filtering.
Perhaps latter is the way to go.
In fact, you could probably use something like:
JsonNode node = mapper.valueToTree(inputValue);
results.put(key, node);
which would convert from POJO into JsonNode -- and this does use
serialize() methods, filtering, but with less overhead -- and then add
JsonNode as value to be serialized by "parent" mapper.
I think this might achieve what you are attempting here?
-+ Tatu +-
>
>
> On Tuesday, May 21, 2019 at 2:40:27 PM UTC-4, Tatu Saloranta wrote:
>>
>> Just one question:
>>
>>
>>>
>>> responseBody.put("firstProperty", serializeFirstProperty(obj.getFirstProperty()));
>>> responseBody.put("secondProperty", serializeSecondProperty(obj.getSecondProperty()));
>>> responseBody.put("thirdProperty", serializeThirdProperty(obj.getThirdProperty()));
>>
>>
>> Why do you serialize values? That is where "double-escaping" comes: you are adding JSON String within content to be JSON serialized. Just add values as is
>>
>> responseBody.put("firstProperty", obj.getFirstProperty());
>>
>> and contents would get serialized just once.
>>
>> -+ Tatu +-
>>
>>
> You received this message because you are subscribed to the Google Groups "jackson-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
> To post to this group, send email to jackso...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/e9420cec-0ddb-4af4-bfc7-1383bede4920%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
Uzh Valin
May 21, 2019, 10:23:22 PM5/21/19
to jackson-user
Thanks Tatu, but how do I read it back when I send this as part exception JSON body? It seems like when if read the body with HttpStatusCodeException.getResponseBodyAsString(), it contains all sort of JsonNode data rather than the actual data that I send back from the Rest producer.
> To unsubscribe from this group and stop receiving emails from it, send an email to jackso...@googlegroups.com.
Tatu Saloranta
May 22, 2019, 1:10:21 AM5/22/19
to jackson-user
On Tue, May 21, 2019 at 7:23 PM Uzh Valin <blahb...@gmail.com> wrote:
>
> Thanks Tatu, but how do I read it back when I send this as part exception JSON body? It seems like when if read the body with HttpStatusCodeException.getResponseBodyAsString(), it contains all sort of JsonNode data rather than the actual data that I send back from the Rest producer.
I am not sure I understand -- `JsonNode` is simply node-based
>
> Thanks Tatu, but how do I read it back when I send this as part exception JSON body? It seems like when if read the body with HttpStatusCodeException.getResponseBodyAsString(), it contains all sort of JsonNode data rather than the actual data that I send back from the Rest producer.
representation of JSON, which serializes to textual content like
POJOs. There is nothing special about it, it's just one way to
manipulate JSON content that may or may not have specific structure.
The other side has no idea what kind of in-memory representation
server uses: it's just json.
> To post to this group, send email to jackso...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/ddd137c4-0f74-4976-9a86-64a9f7f1daba%40googlegroups.com.
Uzh Valin
May 22, 2019, 7:24:58 AM5/22/19
to jackson-user
Correct, when I construct this Map<String, Object> and pass it back as part of exception handling Rest response body (500 HTTP status), I see each item has a proper value.
However, when the exception is thrown on the caller side, and I try to examine the response body (HttpStatusCodeException.getResponseBodyAsString()), what I see is a JSON in the following format:
{
"firstProperty": {
"NodeType":"OBJECT",
"POJO":false,
... bunch of other properties
}
... repeat for other properties
}
Which I am not able to read into an object type (MainObject), and frankly, I don't see any relevant POJO/bean properties when the Map is constructed.
If I create a custom model and simply add that as part of object to Response, the serialization would yield a proper result on the caller side, and I am able to readValue() and map it to a proper type.
Uzh Valin
May 22, 2019, 7:28:49 AM5/22/19
to jackson-user
Correction:
"[..] I don't see any relevant POJO/bean properties when the Map is constructed."
Meant to say, I don't see any relevant properties of these beans when I examine the response body as string and see only the JSON sample I posted in the previous reply.
Tatu Saloranta
May 22, 2019, 3:46:01 PM5/22/19
to jackson-user
On Wed, May 22, 2019 at 4:25 AM Uzh Valin <blahb...@gmail.com> wrote:
>
> Correct, when I construct this Map<String, Object> and pass it back as part of exception handling Rest response body (500 HTTP status), I see each item has a proper value.
>
> However, when the exception is thrown on the caller side, and I try to examine the response body (HttpStatusCodeException.getResponseBodyAsString()), what I see is a JSON in the following format:
>
> {
> "firstProperty": {
> "NodeType":"OBJECT",
> "POJO":false,
> ... bunch of other properties
> }
> ... repeat for other properties
> }
>
> Which I am not able to read into an object type (MainObject), and frankly, I don't see any relevant POJO/bean properties when the Map is constructed.
>
> If I create a custom model and simply add that as part of object to Response, the serialization would yield a proper result on the caller side, and I am able to readValue() and map it to a proper type.
Jackson would never serialize `JsonNode` as POJO, which is what output
>
> Correct, when I construct this Map<String, Object> and pass it back as part of exception handling Rest response body (500 HTTP status), I see each item has a proper value.
>
> However, when the exception is thrown on the caller side, and I try to examine the response body (HttpStatusCodeException.getResponseBodyAsString()), what I see is a JSON in the following format:
>
> {
> "firstProperty": {
> "NodeType":"OBJECT",
> "POJO":false,
> ... bunch of other properties
> }
> ... repeat for other properties
> }
>
> Which I am not able to read into an object type (MainObject), and frankly, I don't see any relevant POJO/bean properties when the Map is constructed.
>
> If I create a custom model and simply add that as part of object to Response, the serialization would yield a proper result on the caller side, and I am able to readValue() and map it to a proper type.
above looks like.
Is it possible you might have something else configured to handle JSON
serialization?
-+ Tatu +-
> To post to this group, send email to jackso...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/cac2aa77-01ce-478f-bc9b-7ba6d9553bd3%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages