Problem with <? super A> bounds and @Nullable

[Piotr Zielinski]

Hi,

I'm not sure whether this is a bug, so I'd like to ask here first
before filing. I'd like to make the following code pass through the
nullness checker. It defines a simple Consumer interface, and a
type-safe casting operation, and -- without the nullness checker --
compiles without problems.

interface Consumer<A extends /*@Nullable*/ Object> {
public void consume(A object);
}

class Utils {
public static <A extends /*@Nullable*/ Object>
Consumer<A> cast(final Consumer<? super A> consumer) {
return new Consumer<A>() {
@Override public void consume(A object) {
consumer.consume(object);
}
};
}
}

With the nullness checker I get the following error

incompatible types.
consumer.consume(object);
^
found : @Nullable A extends @Nullable Object
required: ? extends @NonNull Object super @Nullable A extends @Nullable Object

Is there a way of telling the checker that '?' can be @Nullable?

Thanks,
Piotr

[Michael Ernst]

Piotr-

> Is there a way of telling the checker that '?' can be @Nullable?

This is an interesting problem. It's not a bug, but it is a limitation of
the syntax. I will present a workaround, but for the most part this email
expands on and explains the problem, and asks for feedback in finding the
best solution.


For type of the parameter to cast(), you want to write:

Consumer<? super A extends /*@Nullable*/ Object>

The type argument has both an upper bound ("extends") and a lower bound
("super"). However, Java permits you to specify at most one bound. From a
type-theoretic point of view, it makes sense to specify both: every type
parameter has an upper and a lower bound. (Java fills in any missing
bounds, using the trivial bounds "extends Object" or "super void".)

The Checker Framework's error message printed out both the upper and lower
bound (I have slightly edited for clarity):

> incompatible types.
> consumer.consume(object);
> ^
> found : @Nullable A extends @Nullable Object
> required: ? extends @NonNull Object super @Nullable A

even though you wouldn't be able to paste that in a Java program.

Since Java only lets you write one of the bounds, the other one gets filled
in for you automatically. You wrote, in your original code:

Consumer<? super A>

That is equivalent to:

Consumer<? super A extends Object>

You are using a default qualifier of @NonNull, so it is equivalent to:

Consumer<? super A extends @NonNull Object>

That's not what you want. You need a way to override the @NonNull that was
automatically inserted for you by the Checker Framework's defaulting
mechanism.

Here is code that changes the default locally, and that passes the nullness
type-checker:

---------------- start code ----------------
import checkers.nullness.quals.*;
import checkers.quals.*;

interface Consumer<A extends /*@Nullable*/ Object> {
public void consume(A object);
}

class Utils {

public static <A extends Object>
Consumer<A> cast(@DefaultQualifier("checkers.nullness.quals.Nullable") final /*@NonNull*/ Consumer<? super A> consumer) {

return new Consumer<A>() {
@Override public void consume(A object) {
consumer.consume(object);
}
};
}
}

---------------- end code ----------------

The change of default makes

Consumer<? super A>

equivalent to

Consumer<? super A extends @Nullable Object>

This syntax, with the @Nullable default applied to just one parameter, is
ugly and verbose.

In your example, using a default of @Nullable for the entire Utils class
also works. It is less ugly and, in a larger codebase, might even lead to
fewer annotations overall.


Here are some ways to cope with the problem in the future:

* Continue to use the current defaulting mechanism, which lets users
specify what default will be inserted. Probably, there is rarely a need
to use different defaults for different implicit type bounds in a single
method or class. In other words, the problem may be minor or
non-existent in practice. More experience would be needed to know for
sure.

Incidentally, some people like the idea of per-method, per-class, and
per-package defaults. Other people think that localized defaults will
lead to confusion, because the meaning of the types in a code snippet
cannot be understood without knowledge of the defaults. This question,
too, requires more experimentation and experience.

* Special-case the defaulting rules for implicit "extends" type bounds.
Permit the user to set these defaults independently of the overall
defaults. This may not help much if you have code that intermixes
classes that need different defaults.

This is somewhat similar to the fact that local variables can be
defaulted differently than other locations.

(I would prefer to avoid use of too many fine-grained defaulting
mechanisms. I fear that giving users fine-grained control over
defaulting will lead to confusion.)

* Introduce a way to write annotations on implicitly added type bounds.
The difficulty is in finding a syntactic location, since Java provides
no such location. One possibility is before the wildcard. Under this
syntax proposal,
Consumer<@Nullable ? super A>
would mean
Consumer<? super A extends @Nullable Object>
and
List<@Nullable ? super @NonNull Number>
would mean
List<? super @NonNull Number extends @Nullable Object>

For code that intermixes classes that need different defaults, this
syntax is more compact. It's related to the recent proposal on the
mailing list that
List<@Nullable T>
should mean the same as
List<T extends @Nullable Object>

I'm open to better suggestions, and to comments on these.

-Mike

[Adam Warski]

Hello,

well I guess the best solution would be to allow both upper and lower bounds for a type to be specified (if, as you write, there are no type-theoretic issues). Java 7 just got pushed back to the end of 2010, so maybe that wouldn't be impossible ;). But probably there's quite a lot of bureaucracy involved in pushing it through the JCP.

From the other solutions, as I just wrote on the other thread, I like the Consumer<@Nullable ? super A>/List<@Nullable ? super @NonNull Number> syntax. It's intuitive and quite clear on what it means. So I would vote for that :)

Fixing it using defaults seems like a bit of hacking, so if there are better ways to solve the problem, I would choose the other option.

Adam

[Piotr Zielinski]

Just to say, that my order of preference is
1. Consumer<@Nullable ? super A>
2. Consumer<? super A extends @Nullable Object>
3. Defaults.

I prefer 1 over 2 because it's shorter and adding it does not require
changes to the java language (how would we handle java6
compatibility?).

I don't like defaults much because they add complexity to the language
that can be avoided in options 1 and 2. They may reduce the amount of
code necessary for a few special cases, but would increase the
conceptual complexity of the language as a whole, which I think it's
not worth it.

Piotr

[Michael Ernst]

Piotr-

> Just to say, that my order of preference is
> 1. Consumer<@Nullable ? super A>
> 2. Consumer<? super A extends @Nullable Object>
> 3. Defaults.
>
> I prefer 1 over 2 because it's shorter and adding it does not require
> changes to the java language (how would we handle java6
> compatibility?).
>
> I don't like defaults much because they add complexity to the language
> that can be avoided in options 1 and 2. They may reduce the amount of
> code necessary for a few special cases, but would increase the
> conceptual complexity of the language as a whole, which I think it's
> not worth it.

Thanks for the feedback.

Just to be clear, defaults are part of the Checker Framework and are not
going away. They are useful in many circumstances and dramatically reduce
the number of annotations required in many cases. Although they can
improve readability, overusing them obviously has the opposite effect.

But, this doesn't mean that defaults are the best choice for the problem at
hand -- the need to annotate implicitly-added code such as "extends
Object". For that, one of the other proposals, such as #1 above, may be
better.

I'm still curious whether there is a need in practice to use different

defaults for different implicit type bounds in a single method or class. I

would be interested to see an example of such code.

-Mike

[Piotr Zielinski]

Not sure about generic defaults, but a default that would let me say,
for a given class, "consider all generic type parameters as 'extends
@Nullable Object'" could save a lot of typing in some places. This is
because many parametrized classes provide a lot of static methods with
the same generic parameters as the class, and one has to repeat the
bounds for each method.

Piotr

>
>                    -Mike
>

[Piotr Zielinski]

I've now come across another problem which I don't understand.
Consider the Consumer stuff defined as before (pasted below for
convenience) and the following method:

public static <A extends /*@Nullable*/ Object> Consumer<A> getConsumer() {
Consumer</*@Nullable*/ Object> nullConsumer = null; // null for
simplicity, but could be anything
return cast(nullConsumer);
}

What I get

incompatible types.
return cast(nullConsumer);
^
found : @NonRaw Consumer<@Raw A extends @Raw Object>
required: @NonRaw Consumer<@NonRaw A extends @NonRaw Object>

I'm not sure why ...

Thanks,
Piotr

interface Consumer<A extends /*@Nullable*/ Object> {
public void consume(A object);
}

class Utils {

public static <A extends /*@Nullable*/ Object>

Consumer<A> cast(final Consumer<? super A> consumer) {

return new Consumer<A>() {
@Override public void consume(A object) {
consumer.consume(object);
}
};
}
}

[Piotr Zielinski]

On Fri, Nov 27, 2009 at 4:00 PM, Piotr Zielinski
<piotr.z...@gmail.com> wrote:
> I've now come across another problem which I don't understand.
> Consider the Consumer stuff defined as before (pasted below for
> convenience) and the following method:
>
> public static <A extends /*@Nullable*/ Object> Consumer<A> getConsumer() {
>   Consumer</*@Nullable*/ Object> nullConsumer = null;  // null for
> simplicity, but could be anything
>   return cast(nullConsumer);
> }
>
> What I get
>
>  incompatible types.
>    return cast(nullConsumer);
>               ^
>  found   : @NonRaw Consumer<@Raw A extends @Raw Object>
>  required: @NonRaw Consumer<@NonRaw A extends @NonRaw Object>
>
> I'm not sure why ...
>
> Thanks,
> Piotr

Hi, could somebody shed some light on this? I'm not sure what
@Raw-ness has to do with anything here. Should I file a bug?

[Mahmood Ali]

Greetings Piotr,

Thank you again for submitting your bug fixes. I'm sorry that I
haven't addressed some of your emails yet, and I'm planning to go get
back to you about each report/bug report you (and others) have
submitted.

At first impression, this case you report here and the case about
inserting a `null` in Collection<E extends @Nullable Object> are
indeed bugs. I'll try to reproduce your work and correct it.

Also, feel free to open new tickets/issues at
http://code.google.com/p/jsr308-langtools/issues .

Regards,
Mahmood

[Michael Ernst]

Piotr-

> a default that would let me say,
> for a given class, "consider all generic type parameters as 'extends
> @Nullable Object'" could save a lot of typing in some places. This is
> because many parametrized classes provide a lot of static methods with
> the same generic parameters as the class, and one has to repeat the
> bounds for each method.

I wanted to explicitly respond to this request, mostly for the email
archives. As I think you already know, version 1.0.3 of the Checker
Framework (released 5 Dec 2009) supports this functionality. See
checkers.quals.DefaultLocation.UPPER_BOUNDS.

-Mike

[Piotr Zielinski]

Thanks. I've used this and it works.

>
>                    -Mike
>