Plans for a separate artifact containing only ListenableFuture

[Chris Povirk]

Guava users,

Some Android library developers have told us that they would like to use ListenableFuture in their APIs, but they know that a full Guava dependency would be bad for apps that don't use Proguard. To address this, we are planning to release a separate artifact that contains only ListenableFuture.

To be clear, we don't expect to carve off similar artifacts for other parts of Guava; we see ListenableFuture as a unique case.

Part of the reason that it's unique is that it is an interface that we will likely never change -- not even to add methods. This ensures that, even if an app depends on both the new artifact and the main Guava artifact, it will always see the one and only version of ListenableFuture.

We still anticipate that some users -- including users of Android Studio -- will see "duplicate class" or "duplicate entry" errors if they depend on both artifacts. To fix the errors, such users can exclude their dependency on the ListenableFuture artifact.

However, we recognize that this is confusing and inconvenient. To shield future users from duplicate-class errors, we plan to make Guava depend on an alternative "version" of the ListenableFuture artifact, one that contains no classes because ListenableFuture is already available in Guava. This is similar to the old Version 99 Does Not Exist approach, except that our artifact will live in Maven Central alongside the "real" version. It lets new versions of Guava effectively exclude the dependency on the ListenableFuture artifact, without any action by developers.

We do still anticipate some problems in the meantime, and we're sorry for that. We hope that the value of a lightweight ListenableFuture API outweighs the costs over time. If you are aware of other problems that this change might cause, please let us know. Thanks.

--Chris Povirk, Guava Team

[Thomas Broyer]

Any specific reason you don't "simply" move ListenableFuture out of Guava and make Guava depend on the new artifact? No duplicate class without the need for any "trickery".

And if you want to also make a JAR with everything for those who don't use Maven/Gradle-like dependency management, do it as an "attached artifact" (similar to those "with-dependencies" JARs). I really don't think it's worth it though…

[Michiel Haisma]

Dear Chris, Guava users,

The proposal is quite confusing, as you indicate already.

To clarify this and the whole empty jar concept, perhaps, you could specify a 'current' and 'proposed' layout of the maven artifacts and their dependencies. Additionally, it would be helpful if you could provide an example for the most common use cases.

I fail to see (Like Thomas) why a 'normal' transitive dependency of Guava on a ListenableFuture package would not solve this problem.

Second question:

Will the guava-jre artifact be impacted by this plan?

Thanks.

Michiel Haisma

Oracle Labs

Op maandag 10 september 2018 23:20:08 UTC+2 schreef Chris Povirk:

[Chris Povirk]

On Tue, Sep 11, 2018 at 3:13 AM, Thomas Broyer <t.br...@gmail.com> wrote:

Any specific reason you don't "simply" move ListenableFuture out of Guava and make Guava depend on the new artifact? No duplicate class without the need for any "trickery".

Yeah, that was our initial thought. But JPMS (and OSGi) don't like it when multiple artifacts have classes in the same package, so we'd be creating problems for users of those systems.

[Thomas Broyer]

Ah, yes, indeed, JPMS strikes again… sigh 

[Chris Povirk]

The proposal is quite confusing, as you indicate already.

Sorry about that. Thanks for the feedback.

To clarify this and the whole empty jar concept, perhaps, you could specify a 'current' and 'proposed' layout of the maven artifacts and their dependencies.

Currently:

• guava
• jar contents
• c.g.c.util.concurrent.ListenableFuture
• c.g.c.util.concurrent.*
• other packages
• dependencies
• the existing packages of annotations

Proposed:

• guava
• jar contents
• c.g.c.util.concurrent.ListenableFuture
• c.g.c.util.concurrent.*
• other packages
• dependencies
• the existing packages of annotations
• listenablefuture-9999.0-empty-to-avoid-conflict-with-guava
• listenablefuture-1.0
  
• jar contents
• c.g.c.util.concurrent.ListenableFuture
  
• no dependencies
• listenablefuture-9999.0-empty-to-avoid-conflict-with-guava
• empty jar
• no dependencies

Additionally, it would be helpful if you could provide an example for the most common use cases.

Most users should just be able to depend on guava-jre or guava-android as they do now.

Any Android libraries that want to use ListenableFuture without the rest of Guava can depend on listenablefuture-1.0. We expect this to be fairly rare for now, but we encourage anyone who is interested!

Users who end up with a dependency on both guava and listenablefuture might have to exclude the listenablefuture dependency (or, to achieve the same thing, force their build systems to use version listenablefuture-9999.0-empty-to-avoid-conflict-with-guava). Alternatively, users can update to Guava 27.0 when it comes out next month, and some build systems (notably, Gradle, used by Android Studio) will use listenablefuture-9999.0-empty-to-avoid-conflict-with-guava automatically, avoiding the conflict.

Second question:

Will the guava-jre artifact be impacted by this plan?

Another good question, thanks. New versions of both guava-android and guava-jre will depend on the new listenablefuture-9999.0-empty-to-avoid-conflict-with-guava artifact.

[Chris Povirk]

Most users should just be able to depend on guava-jre or guava-android as they do now.

(OK, to be scrupulously correct: The next Guava release will likely also introduce another dependency on a `failureaccess` artifact, which has 2 tiny classes to support faster interoperability between Guava ListenableFutures and other ListenableFuture implementations. Most users will never have to care about that artifact or its classes at all. But if you're used to manually downloading the Guava jar and having everything you need in it, you'll need to download the `failureaccess` jar (which we expect to only ever have one version) to put on your classpath, too. This new artifact isn't really coupled to the `listenablefuture` artifact, and it's a much more "normal" dependency (nothing in the same package, no magic 9999.0 version), but it's happening at the same time and in support of the same goal.)

[Michiel Haisma]

Hi Chris, Users,

Thanks for clarifying. It's a shame it had to be this way due to technical limitations. But now the purpose and execution is clear (I think). Excluding the empty jar (or will it be marked optional?) should work as normal correct?

As for the `failureaccess` artifact, I suppose that's a more normal case. I suppose the good old days where guava was a straight forward dependency to include are over.

Thanks for the updates again.

Michiel

Op dinsdag 11 september 2018 17:51:45 UTC+2 schreef Chris Povirk:

[Chris Povirk]

Excluding the empty jar (or will it be marked optional?) should work as normal correct?

It should be fine to exclude the empty jar -- or "not include it," from the perspective of those who are just downloading jars rather than relying on Maven, etc. to pull in their transitive deps.

RE: optional: I hadn't thought about marking it optional, so I just investigated. This part of the doc sounds good: "The version of the dependency will be taken into account for dependency calculation if the library is used elsewhere" -- as we'd want, since we want to override listenablefuture-1.0. However: "It will not be passed on transitively" -- so I'm not sure if users of Guava, the ones that we actually want to see the dep, would see it at all.

I did some quick testing with a project that depends on a project that in turn has an optional dependency on another. The transitive optional dep appears to be invisible to `mvn dependency:tree`, `to mvn dependency:resolve`, to dependencyConvergence, and to requireUpperBoundDeps. Optional deps might matter only when building the project that itself declares the optional dep, as suggested by the doc. But there could be more subtleties that I'm missing.

In any case, do you think that optional would solve specific problems, or is it just a better conceptual fit? As you've noted, sadly, the best conceptual fit is not always the best technical fit here :)

As for the `failureaccess` artifact, I suppose that's a more normal case. I suppose the good old days where guava was a straight forward dependency to include are over.

Sadly yes. We will try to make this clear in the release notes and in our general docs.

[Michiel Haisma]

The behavior (and semantics as well) of optional maven dependencies have never been fully clear to me, and I feel it's not going to solve this issue. Usually I think of optional dependencies as hints: "Hey, if you happen to have this optional dependency on the classpath, we'll load it and you can do more stuff, but if it's not there, no problem."

Op woensdag 12 september 2018 17:16:46 UTC+2 schreef Chris Povirk:

[Thomas Broyer]

On Thursday, September 13, 2018 at 10:22:28 AM UTC+2, Michiel Haisma wrote:

The behavior (and semantics as well) of optional maven dependencies have never been fully clear to me, and I feel it's not going to solve this issue. Usually I think of optional dependencies as hints: "Hey, if you happen to have this optional dependency on the classpath, we'll load it and you can do more stuff, but if it's not there, no problem."

Either that or "no feature in that lib but X needs this dependency, so it's marked as optional; if you happen to use feature X, then you'll need to add the dependency on the classpath"

[Mike Minicki]

Why not change the java package of the ListenableFuture in the new artifact and leave the old deprecated copy in Guava as-is for some foreseeable future? That way you could keep those projects separate. You could also depend Guava on the new package version down the line, dropping the old one. Or you could keep the old one forever in Guava jars. That would avoid needing any "hacky" solutions. 

Having a copy wouldn't be a precedent as this is what happened between Guava and GoogleCollections, if I remember correctly. 

Mike

[Joachim Durchholz]

Am 12.09.2018 um 17:16 schrieb 'Chris Povirk' via guava-discuss:
> RE: optional: I hadn't thought about marking it optional, so I just
> investigated. This part of the doc

> <http://maven.apache.org/ref/3.5.4/maven-model/maven.html#class_dependency>

> sounds good: "The version of the dependency will be taken into account
> for dependency calculation if the library is used elsewhere" -- as we'd
> want, since we want to override listenablefuture-1.0. However: "It will

> not be passed on transitively" -- so I'm not sure if /users/ of Guava,

> the ones that we actually want to see the dep, would see it at all.

They won't. "optional" is just a nontransitive dependency.
The bit about the version number just defines version number resolution
if a dependency is pulled in normally and through a direct dependency:
the optional dependency does count (i.e. you can get a version conflict
from it even though the optional dependency does not pull the artifact in).

It does not help Guava since Guava is never the main module.

Regards,
Jo

[Chris Povirk]

Why not change the java package of the ListenableFuture in the new artifact and leave the old deprecated copy in Guava as-is for some foreseeable future? That way you could keep those projects separate. You could also depend Guava on the new package version down the line, dropping the old one. Or you could keep the old one forever in Guava jars. That would avoid needing any "hacky" solutions.

Migrating a large codebase from one copy to the other is challenging, especially for a type like ListenableFuture that's often exchanged across library boundaries. So in practice, we'd end up with two separate ListenableFuture ecosystems. People could of course use both, converting when needed; the question is whether the friction of an extra ecosystem is larger or smaller than the friction of our hacky solution. We're hopeful that the hack will cause little friction in the long run, but it's hard to prove that -- and even looking back 5 years from now, it will be hard to conclusively measure the effects (well, unless it turns out that we really, really messed up :))

Having a copy wouldn't be a precedent as this is what happened between Guava and GoogleCollections, if I remember correctly. 

(guava and google-collections did the evil thing of both publishing the classes under the same names -- com.google.common.collect.Lists, etc. We largely got away with it because few people had heard of the project yet, but even then, it caused some trouble, as you'd expect.)

[Mike Minicki]

On Friday, September 14, 2018 at 12:16:00 PM UTC-7, Chris Povirk wrote:

Why not change the java package of the ListenableFuture in the new artifact and leave the old deprecated copy in Guava as-is for some foreseeable future? That way you could keep those projects separate. You could also depend Guava on the new package version down the line, dropping the old one. Or you could keep the old one forever in Guava jars. That would avoid needing any "hacky" solutions.

Migrating a large codebase from one copy to the other is challenging, especially for a type like ListenableFuture that's often exchanged across library boundaries. So in practice, we'd end up with two separate ListenableFuture ecosystems. People could of course use both, converting when needed; the question is whether the friction of an extra ecosystem is larger or smaller than the friction of our hacky solution. We're hopeful that the hack will cause little friction in the long run, but it's hard to prove that -- and even looking back 5 years from now, it will be hard to conclusively measure the effects (well, unless it turns out that we really, really messed up :))

Migrating is challenging, I agree. But that's exactly why I have suggested keeping the deprecated class forever. Yes, that would split the ecosystem but it's the same way Optional (Guava's and Java's) split it. You basically said, use Java from now on but it's fine if you prefer to continue using Guava's. I would expect the experience to be about the same with ListenableFuture. 

As a side note, how about extending the class from the new package? Similarly to what happened to Guava's and java.util Function interface? I haven't used or looked into ListenableFuture myself, so I'm not even sure if that would be possible (probably not, after glancing at those static methods). However, that could potentially help with the split a bit. And, with JPMS you could secure the unwanted access. 

Anyway, just trying to help with my 2c. I'm sure whatever you decide will work just fine. 

Mike

[Chris Povirk]

Thanks for the comments. I do think that an ecosystem split is the other viable option -- though, at this point, we're going to try the separate artifact for c.g.c.util.concurrent.ListenableFuture itself. (We do feel a little bad about Guava's Function and especially Optional.) Having the old type extend the new does help with conversions somewhat (and our inability to do that with Optional is one of the difficulties there).

We'll see how things go....