Why is Collections.unmodifiableList not defined to return List<? extends T> instead of List<T> ?

[srdjan mitrovic]

Changing Collections.unmodifiableList to return List<? extends T> instead of List<T> would then prevent adding and removing an element in compile time instead of throwing runtime exception.

Why was that not done in this way?

[Zhong Yu]

On Tue, Aug 9, 2016 at 10:42 AM, srdjan mitrovic <srki...@gmail.com> wrote:

Changing Collections.unmodifiableList to return List<? extends T> instead of List<T> would then prevent adding and removing an element in compile time instead of throwing runtime exception.

Why was that not done in this way?

Hi Srdjan,

If the method returns List<? extends T>, you can still call mutate methods on it at compile time, for example, clear(), add(null); and you can pass it to other mutate APIs like sort().

It's not the purpose of unmodifiableList() to return a read-only type, otherwise it should return something like ReadOnlyList. The method returns the same(*) static type as the input type, while preventing some List methods from being called at runtime.

In practice, wildcards are often correlated to the concept of read/write-only; but sometimes this kind of understanding leads to more confusions. I think it is best to understand wildcards in its original design intent - to establish subtyping relationsships among generic types.

I wrote an article on wildcards which may be helpful -
http://bayou.io/draft/Capturing_Wildcards.html

Back to unmodifiableList(). The basic purpose is to return a List<T> for a given List<T>. But it's made to be more general -- return a List<T> for a given List<S> where T is a supertype of S. This is more useful, for example, given a List<Cat>, we can create an unmodifiable view of it as List<Pet>.

To do that, the method could've been designed this way --

    public static <T, S extends T> List<T> unmodifiableList(List<S> input)

Personally, I don't mind this signature. It's more explicit, and it may be better in some corner cases.

However, the general practice of API design is to use wildcards instead of type variables, if possible. So we can replace <S> here with <? extends T>

    public static <T> List<T> unmodifiableList(List<? extends T> input)

    // this is the official signature

Interestingly, we could also keep S and remove T

    public static <S> List<? super S> unmodifiableList(List<S> input)

but obviously, this signature isn't very useful. A user of the method most likely wants to decide the return type; this signature will not allow him to do that.

Now, let's consider your proposed signature -

    public static <T> List<? extends T> unmodifiableList(List<? extends T> input)

The return type is more vague than necessary to express what the method does. A vague type will be more difficult to use than a specific type. Most often, we use this method to present a view to an outsider on an internal data structure, for example,

    private ArrayList<Integer> data = ...

    public List<Integer> viewData(){ return unmodifiableList(data); }

It won't compile using your proposed signature.

A purist may argue that, we ought to use wildcard on viewData() too

    public List<? extends Integer> viewData(){ return unmodifiableList(data); }

well, maybe he's right. But at some point, I'm just so sick of all the wildcards:) See the last section "Wildcard Hell" in my article.

Zhong Yu

bayou.io

[srdjan mitrovic]

Hi Zhong, thanks for your reply. I've read your article, it is a good read :)

I see why you are sick with wildcard usage, the syntax is ugly but I like them because I am a big fan of self-documenting code. By using PECS principle you can see from the code if the collections in parameters (or return type) are suppose to be producers or consumers. 

And yes, I'm a purist and viewData should return List<? extends Integer> :)

I agree that PECS principle cannot cover all modifications for producers (for example it cannot prevent you to call removeAll() or add(null)) but it makes things more safe and more self-documenting. 

[srdjan mitrovic]

And in your example if you have a List<Cat> then if you use my proposed signature your code would be List<? extends Pet> pets = Collections.unmodifiableList(cats) and by the type of your list you would know that it is a producer and it will be more compile-safe.

[Zhong Yu]

Srdjan, If you have <? extends Integer> in your APIs, you are a highly disciplined purist:)

After more thinking, I agree with you that, idealistically, wildcard should be used in that return type; even though it's contrary to the common practice of avoiding wildcards in return types.

Since List is a mixed read-write interface which may complicate the discussion here, let me start with a much simpler interface - Supplier<T> - which is read-only and intuitively covariant.

Suppose a method returns a supplier of pets; what should be the return type, Supplier<Pet> or Supplier<? extends Pet>?

To the caller of the method, the two types usually make no difference -- both have get() that returns a Pet. The common practice is to choose the non-wildcard version. You can check Java8's new APIs of functions and Stream and see that methods rarely return wildcard-ed types. It can be argued that since there are no practical difference to callers, the wildcard is just clutter that should be avoided; not only it's ugly in method declaration, it will also force callers to use wildcards too.

This school of thought appears to be dominant; it goes into static code checkers [1]; it's included in Joshua Bloch's book "Effective Java" [2]; it's probably originated from Angelika Langer's "Generics FAQs" [3] which probably reflects the thoughts of original designers of generics.

[1] http://stackoverflow.com/questions/30543400
[2] http://stackoverflow.com/questions/22815023
[3] http://www.angelikalanger.com/GenericsFAQ/FAQSections/ProgrammingIdioms.html#FAQ303

I am sympathetic to the sentiment that wildcard is ugly and distracting, so it should be avoided if that's not causing other problems. However I think it is too absolute to make it a general rule; rather, each method return type should be evaluated individually to see whether it should contain wildcards.

Angelika's FAQ gives a detailed analysis of why wildcard in return type may restrict legit use cases; I remember I read it thoroughly and I formed a sound rebuttal. However I can't find if I have written it down somewhere, and I don't have time to revisit it right now. I'll just assert here that her argument is flawed, if my past opinions can be trusted.

This school has another secret weapon - type erasure. If the method return a Supplier<Cat> object as Supplier<Pet>, there is no way to tell the difference at runtime; therefore it's OK not to use Supplier<? extends Pet> to cover Supplier<Cat>.

==

The purists, on the other hand, also have good arguments for returning Supplier<? extends Pet>. They do not dignify erasure -- all type checking should be sound at compile time. And they are not sympathetic to those stupid programmers who cannot handle wildcards.

Since Supplier is an intuitively covariant type, it should almost always be used with wildcards; it makes little sense to require a Supplier that returns exactly Pet, but not any subtype of. The concept of "supplier of pets" is most accurately expressed in Java as Supplier<? extends Pet>. The method implementation may choose to return a Supplier<Cat> object and that is perfectly fine.

( I've personally encountered a similar use case -- I have a subclass of Iterable<Pet> which is actually full of Cats only; when implementing the Iterable.iterator() method, I have an Iterator<Cat> ready to use; unfortunately I cannot simply return it, because iterator()'s return type is Iterator<Pet>, instead of Iterator<? extends Pet>. )

A use case can be raised against such wildcard-ed return types. Suppose Supplier<T> has the following method

    // return a new supplier that yields `head` first, followed by elements of `this` supplier.

    Supplier<T> prepend(T head);

This is a read-only method, but syntactically it consumes a T argument, therefore it cannot be invoked on Supplier<? extends Something>. If we have a Supplier<? extends Pet>, we cannot prepend a Pet to form a new supplier!

The problem is rooted in a deficiency in Java generics. Ideally, the method should be declared like this

    <S super T> Supplier<S> prepend(S head)

but Java does not support <S super T> (yet). So we have a dilemma here; choose your poison.

A workaround to that problem is a static util method

    static <T> Supplier<T> prepend(T head, Supplier<? extends T> tail)

it's a little uglier to use than an instance method (Java does not have something like C#'s extension method, yet).

==

Having examined wildcards on Supplier, let's go back to unmodifiableList(). Again, imagine a simpler scenario. Suppose there is a ReadOnlyList interface, then unmodifiableList() ought to return a ReadOnlyList type; the question is, should it include a wildcard?

If, in the previous discussion of Supplier, you were on the wildcard side, then you'll definitely return a ReadOnlyList<? extends T> here. Otherwise you'll return ReadOnlyList<T>.

==

Now, there is no ReadOnlyList in core API (damn, Joshua), we have to use a List interface here. If you were on the side of wildcard in the discussion of Supplier and ReadOnlyList, you obviously will vote for returning List<? extends T> here too, for the same arguments.

But, even if you were not on the wildcard side, in this case of List, you still face a choice - whether to use wildcard to effectively eliminate some methods of List; or in another word, whether to return a more abstract supertype that mimics ReadOnlyList to certain degree. This is Srdjan's argument, and I agree that it is a legitimate design choice.

After all, the intention of the method is to return a list that can be read for T elements; this is more accurately (but not exactly) expressed in Java as List<? extends T>.

So, hurray, purists! But remember, your ideal is against the popular opinion and common practice:)

Zhong