proto3 backward compatibility problem

[sam sun]

I use proto3 to log user event and then run a mapreduce to stat same metrics. But it reports an error:

Exception in thread "Thread-2" java.lang.NoSuchMethodError: org.apache.hadoop.yarn.proto.YarnProtos$LocalResourceProto.hashLong(J)I

at org.apache.hadoop.yarn.proto.YarnProtos$LocalResourceProto.hashCode(YarnProtos.java:11555)

at org.apache.hadoop.yarn.api.records.impl.pb.LocalResourcePBImpl.hashCode(LocalResourcePBImpl.java:62)

at java.util.HashMap.hash(HashMap.java:338)

at java.util.HashMap.put(HashMap.java:611)

The problem is proto3 doesn't keep backward compatibility with proto2. hashLong is no longer exists in generated code with proto3. Now I faced a deed situation that neither proto2 nor proto3 can be loaded when i run a  mapreduce job.

[Feng Xiao]

It seems the problem is that your binary depends on both proto2 Java runtime and also proto3 Java runtime. One of them will be shadowed by the other and you will see such runtime errors. The suggestion is to stick with one version through out your project (including all your project's dependencies).

 

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[sam sun]

I knew it, because yarn depends on proto2 but my lib depends on proto3. The reason is that proto3 runtime doesn't keep backward compatibility. The problem can be reproduced if set option java_generate_equals_and_hash = true with proto2.

I hope msg with proto2 still work with proto3 jar since proto2 has been widely used in opensource system like yarn, hbase, etc.

在 2015年9月7日星期一 UTC+8下午2:31:04，Feng Xiao写道：

[sam sun]

Since the map type is important to me, i decided to use proto3, so i moved hashLong method from  com.google.protobuf.Internal.hashLong/hashEnum/xx to GeneratedMessage in proto3 source code and solved the problem,  however, to maintain a special version by myself is really a bad idea. 

I do think it's  very import to keep full backward compatibility with code generated by proto2 since so many widely used opensource libs depends on protobuf as i have mentioned before,   especially protobuf2.5.

Thanks

在 2015年9月7日星期一 UTC+8下午3:05:50，sam sun写道：

[Feng Xiao]

On Mon, Sep 14, 2015 at 5:23 AM, sam sun <sun...@gmail.com> wrote:

Since the map type is important to me, i decided to use proto3, so i moved hashLong method from  com.google.protobuf.Internal.hashLong/hashEnum/xx to GeneratedMessage in proto3 source code and solved the problem,  however, to maintain a special version by myself is really a bad idea. 

I do think it's  very import to keep full backward compatibility with code generated by proto2 since so many widely used opensource libs depends on protobuf as i have mentioned before,   especially protobuf2.5.

We will take this into consideration and see what we can do. Maintaining full compatibility is hard though (especially when we already have incompatible changes in v3.0.0 alpha/beta releases).

[sam sun]

Thanks.

在 2015年9月15日星期二 UTC+8上午4:17:40，Feng Xiao写道：