protobuf-java 2.4.X and 2.5.0 are incompatible

[Patrick Wendell]

Hi All,

I work on Apache Spark which is an open source project. We have recently been dealing with a lot of pain due to the fact that the Java Protobuf libraries for 2.4.X and 2.5.0 are not binary compatible. This makes it really difficult for users to include two dependencies A and B that depend on different versions of protobuf-java.

Are these incompatibilities an omission, or is this an intentional policy that protobuf is okay making API-breaking changes in minor versions? This violates typical semantic-versioning conventions and makes it pretty tough for downstream users.

I don't see any references to library compatibility in the Java protobuf page or the FAQ - apologies if this is covered somewhere...

https://developers.google.com/protocol-buffers/docs/javatutorial

https://developers.google.com/protocol-buffers/docs/faq

- Patrick

[Ilia Mirkin]

While I don't speak for Google, I believe it's fairly well-established
that 2.4 and 2.5 are considered to be "major" releases. Switching
between them requires regenerating the java files with protoc, as the
internal APIs used by the generated code tend to change. I believe
that in general the public API's remain the same, however that doesn't
let you have multiple protobuf versions without something like jarjar.
The minor releases (2.4.0 vs 2.4.1, etc) should be binary-compatible
AFAIK.

-ilia

> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+u...@googlegroups.com.
> To post to this group, send email to prot...@googlegroups.com.
> Visit this group at http://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.

[Patrick Wendell]

Established based on what conventions? I'm going based on the semantic versioning guidelines here:

http://semver.org/

Basically what I'd like to understand is whether Google cares about this or not, because changing public API's is a big problem for downstream projects. It means that if you want to write a library that uses proto-bufs you can't inter-operate with other libraries that also use protobufs.

- Patrick

[Ilia Mirkin]

On Wed, Apr 16, 2014 at 10:11 PM, Patrick Wendell <pwen...@gmail.com> wrote:
> Established based on what conventions?

On the convention of how protobuf releases have been done... Would you
be happier if it was protobuf2 4.0 and protobuf2 5.0?

> It means that if you want to write a library that uses proto-bufs you can't
> inter-operate with other libraries that also use protobufs.

Not ones that use protobufs compiled against a different major version
of the library. These come out every year or two. AFAIK this has
always been the case. (You can avoid this problem by just building the
proto as part of the build process rather than shipping the generated
files. I'm fairly sure that the generated API's should be
backwards-compatible.)

Hopefully someone from Google can address your other concerns.

-ilia

[Sangjin Lee]

One case where generated APIs are not backward compatible: https://code.google.com/p/protobuf/issues/detail?id=493

[Adrian Petrescu]

Is there any reason you couldn't shade your own protobuf dependency (using, say, the Maven shade plugin) to allow it to interoperate with a dependency's older version?

--