Preserve tags

[rawb...@gmail.com]

Hi!

Given the following YAML structure:

---

the_dev_secret: !vault |
                $ANSIBLE_VAULT;1.1;AES256
                62613163386132636435396436326334366331356239343739656630346330346562316435316639
                3835326137353666393832613939626630313432393166360a373966333035393037663833363564
                34373161383766353963393866396264653166346330663264626261646631306235393531653039
                6565306230373338330a623232336533633861343431633763663530383238616137383364333862
                3134

I managed to read the YAML file using the following code:

YamlConfig config = new YamlConfig();
config.setClassTag("vault", String.class);
YamlReader reader = new YamlReader(fileContent, config);
Object yaml = reader.read();

However, when writing "yaml" back to a file the "!vault" tag obviously gets removed.

Is there any chance I can "preserve" this "!vault" tag when serializing the Java object again?

Thx

[Nate]

Hi,

YamlBeans has so far assumed tags are being used for class names. I think it's reasonable to be able to read the data into a map and access the tags. I've added 2 read config properties, so you can do this:

    static public void main (String[] args) throws Exception {

        YamlConfig config = new YamlConfig();

        config.readConfig.setClassTags(false);
        config.readConfig.setTagSuffix("-tag");
        YamlReader reader = new YamlReader(new StringReader( //
            "the_dev_secret: !vault |\n" //
                + "   $ANSIBLE_VAULT;1.1;AES256\n" //
                + "   62613163386132636435396436326334366331356239343739656630346330346562316435316639\n" //
                + "   3835326137353666393832613939626630313432393166360a373966333035393037663833363564\n" //
                + "   34373161383766353963393866396264653166346330663264626261646631306235393531653039\n" //
                + "   6565306230373338330a623232336533633861343431633763663530383238616137383364333862\n" //
                + "   3134"),
            config);
        Map map = (Map)reader.read();
        System.out.println(map.get("the_dev_secret"));
        System.out.println(map.get("the_dev_secret-tag"));
    }

Cheers,

-Nate

--
--
You received this message because you are subscribed to the "yamlbeans-users" group:
http://groups.google.com/group/yamlbeans-users

---
You received this message because you are subscribed to the Google Groups "yamlbeans-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to yamlbeans-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[rawb...@gmail.com]

Hi,

Thanks for that. However, I'm still wondering how to get the exact same YAML content when serializing the Java object again.

Following your instructions it will now produce:

the_dev_secret-tag: vault

the_dev_secret: |-

   $ANSIBLE_VAULT;1.1;AES256

   <snip>

whereas it ideally should be:

the_dev_secret: !vault |
                $ANSIBLE_VAULT;1.1;AES256

                <snip>

Cheers,

R

To unsubscribe from this group and stop receiving emails from it, send an email to yamlbeans-use...@googlegroups.com.

[Nate]

Makes sense to want to round trip the data. I've adjusted things so that is possible, see below. Note that only value tags are put in the map, not key tags. It's a bit of a hacky way to do it, but it's pretty simple and seems to be a common use case. A better way might be to support reading the data into an object model rather than a map, and to be able to write such objects back to YAML.

Personally I've found YAML to be too complex almost every time I've used it. I prefer the "minimal" JSON-like format in JsonBeans.

https://github.com/EsotericSoftware/jsonbeans

Cheers,

-Nate

    static public void main (String[] args) throws Exception {
        YamlConfig config = new YamlConfig();
        config.readConfig.setClassTags(false);

        config.setTagSuffix("-tag");
        config.writeConfig.setWriteRootTags(false);

        String input = "the_dev_secret: !vault |\n" //

            + "   $ANSIBLE_VAULT;1.1;AES256\n" //
            + "   62613163386132636435396436326334366331356239343739656630346330346562316435316639\n" //
            + "   3835326137353666393832613939626630313432393166360a373966333035393037663833363564\n" //
            + "   34373161383766353963393866396264653166346330663264626261646631306235393531653039\n" //
            + "   6565306230373338330a623232336533633861343431633763663530383238616137383364333862\n" //

            + "   3134";
        System.out.println(input + "\n");

        YamlReader reader = new YamlReader(new StringReader(input), config);
        Map map = (Map)reader.read();
        reader.close();

        System.out.println(map.get("the_dev_secret-tag"));
        System.out.println(map.get("the_dev_secret") + "\n");

        StringWriter buffer = new StringWriter();
        YamlWriter writer = new YamlWriter(buffer, config);
        writer.write(map);
        writer.close();

        System.out.println(buffer);
    }

To unsubscribe from this group and stop receiving emails from it, send an email to yamlbeans-users+unsubscribe@googlegroups.com.

[rawb...@gmail.com]

Hi Nate!

First of all sorry for the late response and thanks for your efforts!

Unfortunately, I still do have two issues with the current implementation:

(1)

The following piece of YAML code:

foo:

  foo1: bar1

... gets serialized as follows:

foo: !java.util.LinkedHashMap

   foo1: bar1

It would be great if there was an option not to output any tags but the "custom" ones.

(2)

As for the "custom tag" it does work now but there is an extra "-" added:

Original:

the_dev_secret: !vault |

                $ANSIBLE_VAULT;1.1;AES256

Transforms to:

the_dev_secret: !vault |-

   $ANSIBLE_VAULT;1.1;AES256

Thanks again for all your help!

Cheers,

Robert

[Nate]

Hi Robert,

1) Set config.writeConfig.setWriteRootTags(false);

2) Is the - a problem when reading the YAML elsewhere??

Cheers,

-Nate

To unsubscribe from this group and stop receiving emails from it, send an email to yamlbeans-users+unsubscribe@googlegroups.com.

[rawb...@gmail.com]

Hi Nate!

1) This is the piece of code I'm using in my test case:

Enter coYamlConfig config = new YamlConfig();

config.readConfig.setClassTags(false);
config.setTagSuffix("-tag");
config.writeConfig.setWriteRootTags(false);

YamlWriter writer = new YamlWriter(stringWriter, config);
writer.write(yaml);
writer.close();

String s = stringWriter.toString();de here...

2) Yep... unfortunately Ansible can't deal with that structure:

ERROR! Syntax Error while loading YAML.

The error appears to have been in '/home/robert/Documents/workspaces/Ansible/ansible_sandbox/ansible_encrypted_vars/group_vars/all.yml': line 1, column 6, but may

be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

foo: !java.util.LinkedHashMap

     ^ here

[Nate]

1) Your code is not executable. http://sscce.org/ If it were, running it would show there are no class tags.

2) The error is about the tag, not the block style indicator. |- is valid. http://yaml-multiline.info/

To unsubscribe from this group and stop receiving emails from it, send an email to yamlbeans-users+unsubscribe@googlegroups.com.

[rawb...@gmail.com]

Ok, sorry. So this would be a complete E2E unit test:

@Test
public void testYamlFileWithEncryptedVarsE2E() throws YamlException {
    String yamlFileContent = "---\n" +
            "foo:\n" +
            "  bar1: 1\n" +
            "  bar2: 2\n" +
            "the_dev_secret: !vault |\n" +
            "                $ANSIBLE_VAULT;1.1;AES256\n" +
            "                62613163386132636435396436326334366331356239343739656630346330346562316435316639\n" +
            "                3835326137353666393832613939626630313432393166360a373966333035393037663833363564\n" +
            "                34373161383766353963393866396264653166346330663264626261646631306235393531653039\n" +
            "                6565306230373338330a623232336533633861343431633763663530383238616137383364333862\n" +
            "                3134\n";
    
    // Config

    YamlConfig config = new YamlConfig();
    config.readConfig.setClassTags(false);
    config.setTagSuffix("-tag");
    config.writeConfig.setWriteRootTags(false);

    config.writeConfig.setIndentSize(2);

    // read YML file
    YamlReader reader = new YamlReader(yamlFileContent, config);
    Map yaml = (Map) reader.read();

    // write back to file
    StringWriter stringWriter = new StringWriter();

    YamlWriter writer = new YamlWriter(stringWriter, config);
    writer.write(yaml);
    writer.close();
    String s = stringWriter.toString();

    assertEquals(yamlFileContent, s);
}

Expected result:

---

foo:

  bar1: 1

  bar2: 2

the_dev_secret: !vault |

                $ANSIBLE_VAULT;1.1;AES256

                62613163386132636435396436326334366331356239343739656630346330346562316435316639

                3835326137353666393832613939626630313432393166360a373966333035393037663833363564

                34373161383766353963393866396264653166346330663264626261646631306235393531653039

                6565306230373338330a623232336533633861343431633763663530383238616137383364333862

                3134

Actual result:

foo: !java.util.LinkedHashMap

  bar1: '1'

  bar2: '2'

the_dev_secret: !vault |-

  $ANSIBLE_VAULT;1.1;AES256

  62613163386132636435396436326334366331356239343739656630346330346562316435316639

  3835326137353666393832613939626630313432393166360a373966333035393037663833363564

  34373161383766353963393866396264653166346330663264626261646631306235393531653039

  6565306230373338330a623232336533633861343431633763663530383238616137383364333862

  3134

Differences:

• "!java.util.LinkedHashMap" tag is added to "foo"
• The starting "---" are missing
• The numeric values in bar1 and bar2 are converted to Strings (surrounded by single quotes)

The last two points are not much of a problem for me but the first one is.

Thanks

[Nate]

1) The class tag is not on the root. Use: config.writeConfig.setWriteClassname(WriteClassName.NEVER);

2) The document separate is optional, but you can use: config.writeConfig.setExplicitFirstDocument(true);

3) YamlBeans does not support this yet. See this issue:

https://github.com/EsotericSoftware/yamlbeans/issues/65

I added a comment showing where this would need to be implemented, if you are interested in submitted a PR.

To unsubscribe from this group and stop receiving emails from it, send an email to yamlbeans-users+unsubscribe@googlegroups.com.

[rawb...@gmail.com]

Hi!

Thanks for your help again, it works now as expected.

I was able to sort out (3) with the following change: https://github.com/rawbertp/yamlbeans/commit/17cfa849bb67f4e7d95ddab4cae599de751cc148

Not worth a PR though but it satisfies my requirements at least.

Cheers

Robert

[Nate]

Glad you got it working. I committed a similar fix.

To unsubscribe from this group and stop receiving emails from it, send an email to yamlbeans-users+unsubscribe@googlegroups.com.