Why can't we clear UnknownFields from a Java instance?

[Ashwin Jayaprakash]

Hi, I have 2 simple messages defined - Person and PersonLite. PersonLite (5 fields) is actually a smaller "view" of Person (20 fields).

I was thinking of using PersonLite to read in the contents of a message that was written by Person. It works alright by reading 5 fields, but PersonLite also reads the remaining 15 fields into the UnknownFields map which is an unmodifiable collection. I was hoping to read the subset of the fields and then discard the unknown fields but alas..!!

Is there a reason why UnknownFields cannot be cleared by the user? I wanted to serialize PersonLite from Person's data and send only those 5 fields over the network. But now I realize that the entire 20 fields get serialized.

Proto below:
package com.javaforu.demo.protobuf;

option java_package = "com.javaforu.demo.protobuf";
option java_outer_classname = "DemoProtos";

message Person {
    required string field_1 = 1;
    optional string field_2 = 2;
    optional string field_3 = 3;
    optional string field_4 = 4;
    optional string field_5 = 5;
    optional string field_6 = 6;
    optional string field_7 = 7;
    optional string field_8 = 8;
    optional string field_9 = 9;
    optional string field_10 = 10;
    optional string field_11 = 11;
    optional string field_12 = 12;
    optional string field_13 = 13;
    optional string field_14 = 14;
    optional string field_15 = 15;
    optional string field_16 = 16;
    optional string field_17 = 17;
    optional string field_18 = 18;
    optional string field_19 = 19;
    optional string field_20 = 20;
}

message PersonLite {
    required string field_1 = 1;
    optional string field_5 = 5;
    optional string field_10 = 10;
    optional string field_15 = 15;
    optional string field_20 = 20;
}


Java code:
public class TestPerson {
    public static void main(String[] args) throws IOException {
        Person person = Person.parseFrom(new FileInputStream(args[0]));
        System.out.println("Read into " + person.getClass().getName() + " from " + args[0]);
        System.out.println(person);
        System.out.println("Bytes length " + person.getSerializedSize());

        System.out.println();
        PersonLite personLite = PersonLite.parseFrom(new FileInputStream(args[0]));
        System.out.println("Read into " + personLite.getClass().getName() + " from " + args[0]);
        System.out.println(personLite);
        System.out.println("Bytes length " + personLite.getSerializedSize());
    }
}


Ashwin.

[Pherl Liu]

You cannot clear the unknown fields in a message because messages are immutable. However, you can parse the bytes into a builder, and then clear the unknowns fields there.

Ashwin.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To view this discussion on the web visit https://groups.google.com/d/msg/protobuf/-/vJwtr6NZxJAJ.
To post to this group, send email to prot...@googlegroups.com.
To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

[Ashwin Jayaprakash]

Thanks! A simple example to do that in the docs would be very useful.

This is what I did and it works exactly as I wanted it to. Code in red below, shows the changes:

public class TestPerson {
    public static void main(String[] args) throws IOException {

        Person p1 = Person.newBuilder()
                .setField1("52114768")
                .setField2("54741573")
                .setField3("36702831")
                .setField4("1477556")
                .setField5("44421078")
                .setField6("48688590")
                .setField7("21657641")
                .setField8("51566604")
                .setField9("19799113")
                .setField10("31045088")
                .setField11("51944937")
                .setField12("18035997")
                .setField13("47693781")
                .setField14("44247784")
                .setField15("61035138")
                .setField16("62112467")
                .setField17("44173609")
                .setField18("45795251")
                .setField19("38467797")
                .setField20("18354530")
                .build();

        FileOutputStream fos = new FileOutputStream(args[0]);
        fos.write(p1.toByteArray());
        fos.close();

        Person person = Person.parseFrom(new FileInputStream(args[0]));
        System.out.println("Read into " + person.getClass().getName() + " from " + args[0]);
        System.out.println(person);
        System.out.println("Bytes length " + person.getSerializedSize());

        System.out.println();
        PersonLite personLite = PersonLite.parseFrom(new FileInputStream(args[0]));
        System.out.println("Read into " + personLite.getClass().getName() + " from " + args[0]);
        System.out.println(personLite);
        System.out.println("Bytes length " + personLite.getSerializedSize());

        System.out.println();
        Builder personLiteBuilder2 = PersonLite.newBuilder().mergeFrom(new FileInputStream(args[0]));
        personLiteBuilder2.setUnknownFields(UnknownFieldSet.getDefaultInstance());
        PersonLite personLite2 = personLiteBuilder2.build();
        System.out.println("Read into " + personLite2.getClass().getName() + " from " + args[0]);
        System.out.println(personLite2);
        System.out.println("Bytes length " + personLite2.getSerializedSize());
    }
}


Output:

Read into com.javaforu.demo.protobuf.DemoProtos$Person from D:\Dump\protobuf\in_out\person1.dat
field_1: "52114768"
field_2: "54741573"
field_3: "36702831"
field_4: "1477556"
field_5: "44421078"
field_6: "48688590"
field_7: "21657641"
field_8: "51566604"
field_9: "19799113"
field_10: "31045088"
field_11: "51944937"
field_12: "18035997"
field_13: "47693781"
field_14: "44247784"
field_15: "61035138"
field_16: "62112467"
field_17: "44173609"
field_18: "45795251"
field_19: "38467797"
field_20: "18354530"

Bytes length 204

Read into com.javaforu.demo.protobuf.DemoProtos$PersonLite from D:\Dump\protobuf\in_out\person1.dat
field_1: "52114768"
field_5: "44421078"
field_10: "31045088"
field_15: "61035138"
field_20: "18354530"
2: "54741573"
3: "36702831"
4: "1477556"
6: "48688590"
7: "21657641"
8: "51566604"
9: "19799113"
11: "51944937"
12: "18035997"
13: "47693781"
14: "44247784"
16: "62112467"
17: "44173609"
18: "45795251"
19: "38467797"

Bytes length 204

Read into com.javaforu.demo.protobuf.DemoProtos$PersonLite from D:\Dump\protobuf\in_out\person1.dat
field_1: "52114768"
field_5: "44421078"
field_10: "31045088"
field_15: "61035138"
field_20: "18354530"

Bytes length 51



Ashwin (http://www.ashwinjayaprakash.com)