In favor of Immutable interfaces

[Olivier Cailloux]

Dear Guava Team,

I’d like to argue for letting the guava ImmutableCollection types (ImmutableList, ImmutableGraph, and so on) implement new interfaces, say, ImmList, ImmGraph, and so on, that would reflect their contracts but allow subclassing. I realize that this has been evoked already but have not found detailed discussions about this specific proposal and its strengths (and weaknesses), so I have some hope that this proposal deserves to be argued for.

The proposal

In more details, there would be one Java interface for each Immutable* type. The interface would exhibit the contract of its corresponding Immutable type. For example, there would be an ImmCollection interface (or any other name considered adequate) that would extend Collection and whose instance methods signature and javadoc would be identical to ImmutableCollection, an ImmList interface that would extend ImmCollection and List with instance methods and javadoc identical to the ImmutableList type, and so on. The existing Immutable* types would be declared to implement those interfaces.

Some counter-arguments

Let me start with the usual counter-arguments that have probably sprung to the mind of the reader. It is usually said, correctly of course, that the fact that the Immutable* types are classes and not interfaces make them non implementable by anyone else than the Guava team. This is sometimes considered a feature, not a problem, as it may help the user trust that the implementations are indeed immutable. Also, it is observed that the Immutable* types actually correspond to several implementations, not just one, and that it is hard to see a reason for someone to want one’s own implementation for at least some of these types. Finally, interfaces also sometimes (when sealed) do not permit own subtyping, so the interdiction of “own subtyping”  is not specific to classes. As a result, the Guava team argues that the Immutable* types should effectively be treated as interfaces, not as classes. (Refs: Colin D, Louis Wasserman, Wiki, Chris Povirk.)

Argument in favor

I do not think that these arguments stand up to scrutiny when considering the specific current proposal. It is true that forbidding anyone else than the Guava team to implement Immutable* types may be trust-building, but isn’t it a bit lacking modesty (no offense intended) about the Guava team? If one agrees with the general advice to API builders to return interfaces rather than types in order to be free from swapping implementation in the future, it also seems reasonable to admit that for Immutable* types, the Guava team could not have come up with the best possible implementations for every possible use cases. Someone could face a very specific situation where a very specific implementation will feature better performances (hints of such situations here, here). Also, having interfaces may enable use cases that are extremely difficult to achieve without first convincing the Guava team to change their implementation (see here). In short: decentralization of decisions, by letting developers provide their own implementations, is usually considered in the object oriented world as outweighting the benefit of having just one organization controlling the implementation of a type, even though it is true that the one-organization-implementing approach gives more guarantees of correctness to the users who trust that organization.

Most importantly, the current proposal still provides the possibility for anyone to use Immutable* types if they want the guarantee of Guava-implementation of the types; it just _adds_ the possibility for an API developer to be not tied to the Guava-implementations for those who value the freedom of swapping implementation.

Let me make it clear that I do _not_ think that there is any reason to mistrust the quality of the implementation of the Guava team: it is clear that this library is of extremely high quality. But still, as a matter of principle and as a practical matter in specific cases, it seems to me to be valuable to let the users of Guava (at least the ones who provide API that they intend to maintain for some time) apply the conventional wisdom of giving their future selves an ability to come up with their own (or any other organisation than Guava) implementation of the types that they commit their API to return.

I hope that this proposal will be given some thoughts and I apologize if it is redundant.

[Chris Povirk]

I can't say I recall any past discussion of additionally providing interfaces along with the existing types.

There are for sure cases in which an alternative implementation would work better. For example, someone just today asked us about EnumSet/EnumMap-backed ImmutableMultimap implementations :)

My initial reaction, right or wrong, is that one effect we'd see inside Google in practice is that people would start mocking the types. And then any user of an ImmList who is debugging a problem may have to look with suspicion on basic operations like iteration.

That is, to be fair, already what can happen if you use plain List in your APIs. And people don't usually mock List, thankfully. And when they do, it's still not the end of the world. Plus, inside Google3, one of the first things we'd do is ban mocking of ImmList, eliminating this problem... for us :)

Still, the same kinds of problems can happen with other implementations. Yes, maybe we're being immodest, but ImmutableList and friends have undergone years of production use, and they're tested beyond what would be reasonable for most custom implementations. I don't think the difference is between "perfect" and "broken," but if ImmList were widely adopted, then I'd expect the difference between "extremely solid" and "very solid" to add up over time, and it may outweigh the gains that the interfaces could sometimes unlock.

If I were to engage in some wishcasting, I'd hope that the use cases for special immutable-collection types would be relatively localized. In such a case, a project could define its own ImmList class, with separate implementations for BasicImmList (backed by Guava?) and ConcatenatedImmList, etc.

Alternatively, if immutable-collection instances need to be passed around throughout a system, then maybe the situation is one in which Eclipse Collections or another alternative would be more suitable, since other libraries have been used in production and tested for difference use cases than Guava.

[j...@durchholz.org]

Just a remark from the sideline:

On 09.07.24 22:41, 'Chris Povirk' via guava-discuss wrote:
> That is, to be fair, already what can happen if you use plain List in

> your APIs. And people don't /usually/ mock List, thankfully. And when

> they do, it's still not the end of the world. Plus, inside Google3, one
> of the first things we'd do is ban mocking of ImmList, eliminating this
> problem... for us :)

If somebody starts mocking basic data structures, they own any problems
they create, and I believe most also know this and won't come asking for
help.
I also suspect that if people don't mock List, they will likely not mock
ImmList either, for the same reasons, whatever these are.

On adding interfaces in general:
They can be a *huge* improvement if done with a clear strategy in mind.
I have seen that done once, with the EiffelBase library, and it was a blast.
Done in an ad-hoc fashion, it's a very easy way to paint yourself into a
corner.

Regards,
Jo

[Chris Povirk]

If somebody starts mocking basic data structures, they own any problems
they create, and I believe most also know this and won't come asking for
help.
I also suspect that if people don't mock List, they will likely not mock
ImmList either, for the same reasons, whatever these are.

I am probably projecting too much of our own experiences onto the wider Java community: We are used to having our changes blocked because some team decided to mock a List to expect some part of Guava to call isEmpty() but not for it to call size(), leading to breakages when we change that part of Guava to call size() instead of isEmpty(). We also hear from other teams in similar situations. So we end up in a situation in which Project A mocks a type owned by Project B, leading to failures after a change by Project C. We have largely improved this by cracking down on bad mocking Google-wide. (And I don't mean to suggest that a lot of people had such mocks, but if even a very small fraction do, then changes like size()->isEmpty() break someone, and we need to fix the failing tests or at least verify that those tests don't indicate real problems, just bad tests.)

All that said, I've been over-focusing on mocking because of the problems in earlier years, which weren't always that bad but which always felt so unnecessary :)

 

On adding interfaces in general:
They can be a *huge* improvement if done with a clear strategy in mind.
I have seen that done once, with the EiffelBase library, and it was a blast.
Done in an ad-hoc fashion, it's a very easy way to paint yourself into a
corner.

Yes, this is probably closer to the main answer: We'd want to do new interfaces right, so we'd want a full understanding of the design space. That would mean a lot of investigation of Guava (e.g., should the interfaces live in common.base so that Splitter can use them?) and our users, hopefully both inside and outside Google. But we've increasingly shifted our efforts toward supporting platforms (e.g., Kotlin, Android with Java 8 support) rather than toward development of large, new libraries. That may or may not be the right decision, but it means that we have a backlog of cool ideas that would need time to investigate (example). And my guess is that ImmList and friends, while not requiring as complex a design as some other candidates, aren't going to make the top of the list anytime soon :(

[Olivier Cailloux]

Thanks for your thoughts.

Yes, this is probably closer to the main answer: We'd want to do new interfaces right, so we'd want a full understanding of the design space. That would mean a lot of investigation of Guava (e.g., should the interfaces live in common.base so that Splitter can use them?) and our users, hopefully both inside and outside Google. But we've increasingly shifted our efforts toward supporting platforms (e.g., Kotlin, Android with Java 8 support) rather than toward development of large, new libraries. That may or may not be the right decision, but it means that we have a backlog of cool ideas that would need time to investigate (example). And my guess is that ImmList and friends, while not requiring as complex a design as some other candidates, aren't going to make the top of the list anytime soon :(

I realize that the Guava team is the one to decide on priorities. Still, adding interfaces seems like very good value for reasonably low investment. I still think that the (virtually unanimously agreed upon among API designers) advice of returning interfaces, not sealed classes, when the implementation is not trivial, so as to allow for future implementation changes without breaking binary compatibility, should be followed by Guava, which generally makes it a point to follow and exemplify best practices. (Relatedly, note that observing that developers may switch to other libraries with Immutable interfaces when they see the need is not a reply to that point). So the value seems very high to me. On the other hand, I grant that this addition is not trivial, but apart from the package to put the interfaces into, as you mention, and probably a few other decisions to take, it seems not such a big work to me. After all, the design of these types is already conceived as if they were interfaces, so it seems to me that adding interfaces would mostly consist in copying their already existing (conceptual) interfaces. (I do realize that this needs careful attention, I am not suggesting to do it overnight by just blindly coping everything to interfaces. My point is simply that it does not seem such a huge investment of time.)

I hope that the Guava team will give this idea a second thought.

I can open a GitHub issue if you feel that it could be useful.

[Ben Manes]

In my opinion, the behavior being asked for is already provided by the root collection interfaces. As non-sealed interfaces, the receiver cannot make any strong assumptions about the implementation's motability, capabilities, performance, and does not provide any assurances for the application's integrity and security. The unmodifiable and immutable implementations provided by Java offer restricted versions to be produced, but receivers who have integrity requirements must act defensively. Guava introducing marker interfaces for its immutable types would provide no new behavior, could satisfy no expectations, and their usage would deceptively imply trust where none is upheld. The usage of the concrete types in the interface contracts is what provides these guarantees to the receiver. For those who want to make those promises softly, they can use the core interfaces and JavaDoc appropriately and leave it up to the receiver to decide how confident they are in relying on those claims.

--
guava-...@googlegroups.com
Project site: https://github.com/google/guava
This group: http://groups.google.com/group/guava-discuss
 
This list is for general discussion.
To report an issue: https://github.com/google/guava/issues/new
To get help: http://stackoverflow.com/questions/ask?tags=guava
---
You received this message because you are subscribed to the Google Groups "guava-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to guava-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/guava-discuss/21a6ad46-9359-4b6a-8849-2cb5354681f9n%40googlegroups.com.

[Joshua O'Madadhain]

My 2^n cents

* as a Guava contributor and originator/pseudo-owner of the common.graph piece of Guava

* but not as a full-time member of the Google Guava team (because I'm not one :) ):

(1) common.graph had the luxury of being able to take a fresh start at this (since we were creating our own base-level interfaces and not inheriting from existing ones), which led to https://github.com/google/guava/wiki/GraphsExplained#mutable-and-immutable-graphs.  In this case we left the mutation methods off the base interfaces, gave them Mutable* subinterfaces with mutation methods, and then had Immutable* implementations of the base interfaces.  I think that this works out pretty well; we still have Immutable implementations rather than interfaces, but Graph and its siblings don't have mutation methods, so even if you know that it's a MutableGraph under the hood, you have to cast it to MutableGraph to get access to those methods.

(2) However, Guava's common.collect obviously does not have the same freedom: partially because they've been around for rather longer, and partially because Guava is not in a position to create versions of those interfaces that _don't_ have the mutation methods in them.

Given that, I don't think that an ImmList extension of List would add a lot of value: interfaces-as-contracts work  better if the existence/absence of methods matches the intended semantics/capabilities.

(3) Yes, Guava could do this to make it easier for other implementors to provide alternate implementations that would be compatible with existing Guava types, and I think that this is an important consideration.  (We don't expect people to roll their own implementations for common.graph, but we have done a fair amount of work to make it possible.)  However, in this case I think that the benefits of alternate implementations are likely to be fairly minor, especially when balanced against the risks of people mis-implementing the interface.  I expect that anyone with really specific requirements will probably build their own immutable implementation and use that, and that seems like a reasonable outcome.

To view this discussion on the web visit https://groups.google.com/d/msgid/guava-discuss/CAGu0%3DMM4imh8%3DyV%2BxPWe40-ORu-epwxDbRxPoZcuDZdoTB6ssA%40mail.gmail.com.

--

   Joshua O'Madadhain: Information Scientist, Musician, Philosopher-At-Tall
  "It's that moment of dawning comprehension that I live for" -- Bill Watterson

[Olivier Cailloux]

I think that the discussion about presence of mutability methods in the collection interfaces leads us astray. I agree that it introduces inelegancies but this holds whether Guava decides to opt for introducing Imm* interfaces or not: the current contracts also suffer from this (inevitably, as point out).

In this case we left the mutation methods off the base interfaces, gave them Mutable* subinterfaces with mutation methods, and then had Immutable* implementations of the base interfaces.  I think that this works out pretty well; we still have Immutable implementations rather than interfaces, but Graph and its siblings don't have mutation methods, so even if you know that it's a MutableGraph under the hood, you have to cast it to MutableGraph to get access to those methods.

This creates the problem that I am talking about, however. If an API designer wants to leave room to her future self to swap implementation should the need arises, she may not expose a method that returns an ImmutableGraph. Because this currently ties her to the specific implementations designed by Guava. This is unfortunate because in many circumstances, when her method effectively returns an immutable graph, saying so publicly would be useful to her users. (This point seems consensual.)

I realize, considering multiple answers here, that in the following circumstance faced by an API designer, the best choice between two strategies is non consensual.

Strategy “Strong guarantees”. The API will return concrete types that allow the users to know that the types are implemented by a given team, known to be reputable, which helps build trust that the types do satisfy their contract.
Strategy “Flexible future”. The API will return interfaces that allow the API designer to swap implementation if this reveals useful in the future.

Note also that the strategy may depend on the types we are talking about: the API designer may “set the bar” as she likes that determines the quality level and the reputation of the team so as to decide whether she follows Strong guarantee or not. Nobody here I guess defends the strategy Strong guarantees for every case, people just argue that in the case of the Guava team, the types are so well implemented and the team so brilliant that the strategy Strong guarantee should be followed, but in other cases, such as using the JDK collection types, the strategy Flexible future could be adopted. So I guess that it is consensual that the choice should be either “Flexible future” always; or one or the other strategy depending on where one sets the bar.

My point is that the decision of which strategy to adopt, or where to set the bar that decides on the strategy, should be left to the API designer and their users. I mean that the real choice that we are talking about (whether Guava adds Imm* interfaces) makes the following difference.

Decision to add interfaces. The API designers that use Guava are free to follow “Strong guarantees” or “Flexible future”.
Decision to not add interfaces. The API designers that use Guava must follow “Strong guarantees”.

I realize that Guava thinks of itself as a highly qualified team. Also, probably, most other type providers think of themselves as highly qualified. Still, I believe that type providers should be liberal enough to allow their users (here in the sense of API designers using their types) to choose their strategy. In other words, if Guava feels authorized to mandate from its users that they follow the “Strong guarantees” strategy, this sets an example that might be followed by other teams. I prefer a more liberal world, and I hope that Guava also does. Note that if an API designer opts for “Flexible future”, her users may decide, as a reaction, to not use her library, if this indeed is such a bad choice. It seems better to leave the market decide than to forbid anyone from trying. Recall that even granting that the implementations are of very high quality (which I agree with), the “Flexible future” strategy still makes sense as no single team can cover all possible use cases (as observed previously in this thread).

Let me make it clear that this argumentation does not contradict Chris’ point that, even granting that it would be a good thing to do, the team might lack the time to do it. I just wish to argue that it would indeed be a good thing to do.

The argument that adding Imm* interfaces would risk the Guava team to lose time because of people misusing the liberty that this offers also is not addressed by my current point, but it seems more or less agreed upon that this will likely be pretty rare.

[Chris Povirk]

Selfishly, I would be happy to see an issue filed about this: I expect it to come up again eventually, and I will go looking for it in the issues before I go looking for it here :)

Beyond that, I still don't claim to have made a slam-dunk argument here. I have enough of a personal feel on the proposal that I don't expect to give it deep thought anytime soon, given other priorities, but things can always change over the years.

If it helps: At the moment, the main question in my mind is: If a user asked us when to use ImmutableList and when to use ImmList, would our answer round off to "Always use ImmutableList?" Users who do need ImmList would know that they need it, but the mere existence of ImmList would raise the question for a large number of users, some of whom will already be weighing ImmutableList vs. List vs. Collection vs. Iterable (vs. other designs like a custom class or an array). That's among the reasons that I'd want to have a good understanding of what we designs might offer and how frequently they might be useful.

[Jens]

IMHO the statement that ImmutableList and friends should be treated as interfaces and thus used in method signatures was a bad one. I mean we have ImmutableList.add() which kind of defeats the purpose of using a concrete type. 

If we embrace defensive copying then there is no need to communicate immutability via types. Instead a method that wants to use a collection needs to ask itself wether or not its defensive copy should be immutable or not and if that method is fine with an immutable copy then copying becomes a no-op if the source collection is immutable as well.

Java SDK already has List.of() and List.copyOf() with List.copyOf(List.of()) not copying anything. If Java SDK would provide Iterable.isImmutable() then List.copyOf(<some immutable xyz list>) would work for any immutable list implementation by just returning the input list. List.of() does not allow NULL elements so Iterable.allowsNull() might be needed and handled as well. Improving Java SDK is the only way go to.

IMHO a library API should always use the pure Java SDK interfaces and more complex Guava collection types like multimap, bimap, etc should be hidden using meaningful, domain specific names. As a library I would treat guava always as an implementation detail and would also repackage it in a different package because there are so many versions of guava out there that it sometimes results in headache when using a library that depends on guava.

-- J.