ObjectMapper with YAMLFactory to Parse a YAML File deleting special characters

[Omar Alkhateeb]

I have issue while using ObjectMapper with YAMLFactory to Parse a YAML File

 

1.       The YAML file I’m trying to parse : https://drive.google.com/open?id=1Q85OmjH-IAIkordikLTsC1oQVTg8ggc8

2.       Parsing the File using readValue as shown here :

 

ObjectMapper mapper = new ObjectMapper(new YAMLFactory().enable(Feature.MINIMIZE_QUOTES)//

                .disable(Feature.WRITE_DOC_START_MARKER)//

                .disable(YAMLGenerator.Feature.SPLIT_LINES));

// mapper.getFactory().setCharacterEscapes(new YAMLCharacterEscapes());

TypeReference<HashMap<String, Object>> typeRef = new TypeReference<HashMap<String, Object>>() {};

HashMap<String, Object> obj = mapper.readValue(responseBuffer.toString(), typeRef);

 

3.       Converting the Obj to json then to YAML again by :

 

JsonElement jsonElem = wrapJacksonObject(obj);

        String cloudTemplateJsonString = new GsonBuilder().disableHtmlEscaping().setPrettyPrinting()//

                .create()//

                .toJson(jsonElem);

 

        JsonNode jsonNode = mapper.readTree(cloudTemplateJsonString);

        String yaml = new YAMLMapper().enable(Feature.MINIMIZE_QUOTES)//

                .disable(Feature.WRITE_DOC_START_MARKER)//

                .writeValueAsString(jsonNode);

 

4.       After checking the last String, I see that these Special Characters are Changed/Deleted (they are Changed exactly after Point 2) :

 

a.       ‘ transferred to “ or Deleted

b.       ! : Regarding the exclamation mark : the whole string after it until first space is deleted totally :

 

Diff Examples on what I talking about :

 

5.       I try to use Escape Characters Customization By customizing Implementation for CharacterEscapes class but it didn’t help

 

Do you have any suggestions ? Please help me I really need your help to success my project

[Tatu Saloranta]

I am sorry but that is a very complex sequence of things, for some reason even including GSON for json parsing instead of Jackson.

If some characters get changed unexpectedly you will probably need to figure out the specific part of processing that does it.

Use of CharacterEscapes should not be necessary: text values should retain their exact contents.

-+ Tatu +- 

 

[Omar Alkhateeb]

Hi Tatu,

Thank you very much for your response :

We can neglect the Gson Parsing

the issue after debugging it exactly after calling readValue as shown here :

HashMap<String, Object> obj = mapper.readValue(responseBuffer.toString(), typeRef);

for example, let's take this Key,Value from the original File :

Conditions:
  ProvidedAdminEmail: !Not [!Equals [!Ref AdminEmail, '']]
  ProvidedTargetGroups: !Not [!Equals [!Ref GatewaysTargetGroups, '']]
  EnableCloudWatch: !Equals [!Ref CloudWatch, true]
  CreateELB: !Not [!Equals [!Ref ELBType, none]]

After Debugging, i found that ObjectMapper using YamlParser change the text that starting with ! as :

Conditions={ProvidedAdminEmail=[[AdminEmail, ]], ProvidedTargetGroups=[[GatewaysTargetGroups, ]], EnableCloudWatch=[CloudWatch, true], CreateELB=[[ELBType, none]]}

And i stopped the debugger before continue with Gson parsing, so the issue 100% with the YamlParser used by Deserializer 

Yesterday I debugged the YamlParser and found it is simply skipping the String that start with ! (maybe i'm wrong, you are expert than me)

i wrote here :

https://stackoverflow.com/questions/61784803/jackson-yaml-parser-deleted-special-characters 

After Debugging, i found that the issue with : Object value = deserialize(p, ctxt); Inside Class : UntypedObjectDeserializer when deserialize try to find the value, i see that the Parser has current token : !Ref but the YAMLParser advanced to the next token and neglect it in a strange way I'm talking about this line Number 648 JsonToken t = p.nextToken(); Inside deserialize Function I need here some one expert with YamlParser to find where is the issue 

i believe you are one of the most active and intelligent contributor to this Project and i sure you will find where is exactly the issue  

--
You received this message because you are subscribed to the Google Groups "jackson-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAL4a10iP%2BBUwsVaeZCB%2BUrTZBGJ8Gg-%3D-hfuu%2BaycEGUD%3DhyAw%40mail.gmail.com.

[Tatu Saloranta]

On Thu, May 14, 2020 at 6:34 PM Omar Alkhateeb <omar...@gmail.com> wrote:

Hi Tatu,

Thank you very much for your response : 

We can neglect the Gson Parsing

the issue after debugging it exactly after calling readValue as shown here :

HashMap<String, Object> obj = mapper.readValue(responseBuffer.toString(), typeRef);

for example, let's take this Key,Value from the original File :

Conditions:
  ProvidedAdminEmail: !Not [!Equals [!Ref AdminEmail, '']]
  ProvidedTargetGroups: !Not [!Equals [!Ref GatewaysTargetGroups, '']]
  EnableCloudWatch: !Equals [!Ref CloudWatch, true]
  CreateELB: !Not [!Equals [!Ref ELBType, none]]

After Debugging, i found that ObjectMapper using YamlParser change the text that starting with ! as :

Conditions={ProvidedAdminEmail=[[AdminEmail, ]], ProvidedTargetGroups=[[GatewaysTargetGroups, ]], EnableCloudWatch=[CloudWatch, true], CreateELB=[[ELBType, none]]}

And i stopped the debugger before continue with Gson parsing, so the issue 100% with the YamlParser used by Deserializer 

Yesterday I debugged the YamlParser and found it is simply skipping the String that start with ! (maybe i'm wrong, you are expert than me)

i wrote here :

https://stackoverflow.com/questions/61784803/jackson-yaml-parser-deleted-special-characters 

After Debugging, i found that the issue with : Object value = deserialize(p, ctxt); Inside Class : UntypedObjectDeserializer when deserialize try to find the value, i see that the Parser has current token : !Ref but the YAMLParser advanced to the next token and neglect it in a strange way I'm talking about this line Number 648 JsonToken t = p.nextToken(); Inside deserialize Function I need here some one expert with YamlParser to find where is the issue 

i believe you are one of the most active and intelligent contributor to this Project and i sure you will find where is exactly the issue  

:)

Ok, I think the problem is with String value that is not quoted, meaning that YAML parser (snakeYAML) will process possible notation.

And "!" happens to be special character, see for example:

https://stackoverflow.com/questions/9664113/what-does-a-single-exclamation-mark-do-in-yaml

which I think here gets removed (probably just adds "empty" tag, which parser discards).

Solution here would be to quote such String values.

I hope this helps,

-+ Tatu +-

 

To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAPAG0c3mtr4U093GjM2YLRMxHdFEw4dhh3Xg8FMw7J6eVCWrog%40mail.gmail.com.