Guava 16

[cal...@snapchat.com]

The fact that we are very stuck on Guava 16 is a pain point any time druid needs to interact with some other framework. This thread is a collection of other major frameworks and what version of Guava they are on.

This has the not-so-subtle intention of driving a solution so that we are not stuck on Guava 16 forever

Spark 2.2.1 : Guava 14.0.1 : https://github.com/apache/spark/blob/v2.2.1/pom.xml#L382 

Beam 2.2.0 : Guava 20.0 : https://github.com/apache/beam/blob/v2.2.0/pom.xml#L128

Flink 1.4.0 : Guava 18.0 : https://github.com/apache/flink/blob/release-1.4.0/pom.xml#L102

Hadoop 3.0.0: Guava 11.0.1 : https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-project/pom.xml#L93

Google Java API Client 1.23.0 : Guava 17.0 : https://github.com/google/google-api-java-client/blob/1.23.0/pom.xml#L639 (increased to 19.0 in https://github.com/google/google-api-java-client/pull/1070 )

AWS Java SDK : Guava 18.0 AND Guava 16.0.1 : Transitive dependencies

Hadoop is one of the main reasons not to move forward, but how long do we want to be stuck on Hadoop's gauva dependency given so many other Apache (and non apache) frameworks are slowly moving beyond it?

[Gian Merlino]

Sometimes I feel like the Guava maintainers are engaged in an elaborate troll on the Java world. They made a utility library that got embedded in a ton of projects and they constantly add and remove APIs. I wonder if we should deal with the situation by removing the Guava dependency and instead copying any code we need into the Druid codebase. The license would allow it. Some of it is arguably obsoleted by Java 8 so we could also migrate some of that. Here's what we use (along with frequency): https://gist.github.com/gianm/efbfbfebfa0601984a7206755e2d3f09

We could also build a relocated Guava jar (replace com.google.common. with io.druid.guava.) and depend on that in Druid, rather than literally copying the code. It would make it easier to upgrade to newer Guavas.

Gian

--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-development+unsubscribe@googlegroups.com.
To post to this group, send email to druid-development@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/307e5025-893d-4f3a-950b-e4dede4ba953%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Himanshu Gupta]

Gian's suggestion sounds good to me.

It will probably still show up in Druid distro due to the transitive dependencies... but good for Druid to get rid of it as direct dependency.

-- Himanshu

[Roman Leventov]

Gian FYI they stopped removing APIs since Guava 21: https://plus.google.com/+googleguava/posts/E2axYogRWwR

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/190acddd-a9cb-4d1b-86d1-ce11988f8f9d%40googlegroups.com.

[Julian Hyde]

“Adding a relocated guava jar” is what quite a lot of other projects do. maven-shade-plugin is the way to do it. Shading usually works well, but it becomes problematic if you use Guava classes (e.g. ImmutableList) in Druid’s public APIs.

Speaking in Guava’s defense. They adhere to the letter of the law of semantic versioning. I.e. give a couple of major versions notice before removing APIs. The problem is that they release a couple of major versions per year.

In Calcite we were not able to shade, because we use Guava APIs in our public interface. Our strategy has been to stop using classes that are removed in later versions of Guava, and not use classes that are not present in Guava 14. As a result we can use any Guava between 14 and 22. I don’t know whether that’s a possibility for Druid.

Julian

To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CAB5L%3DwdJW2B7W_JhOAuJO4DQ8DxuvksPWU63njC48FDjW7Z56g%40mail.gmail.com.

[cal...@snapchat.com]

Took a crack at this looking at things that break in upcoming Guava releases. Came up with two PRs:

https://github.com/druid-io/druid/pull/5414

https://github.com/druid-io/druid/pull/5413

Gian

To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/307e5025-893d-4f3a-950b-e4dede4ba953%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Druid Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/190acddd-a9cb-4d1b-86d1-ce11988f8f9d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[cal...@snapchat.com]

So this way, even if we don't shade Guava, at least we have a wide swath of version compatibility 

[Julian Hyde]

Also, Guava deprecated Throwables.propagate in Guava 20. So, wise to stop using that now.

https://github.com/google/guava/wiki/Why-we-deprecated-Throwables.propagate 

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/05994484-550d-4854-a861-d36f53185e59%40googlegroups.com.