Scala's subsumption can not be controlled

[Shelby]

I filed this as an improvement request at the Scala issue tracker.

https://issues.scala-lang.org/browse/SI-7857

After two years of deliberations, I have come to the conclusion that the inability to tell Scala's subsumption not to subsume the input type parameter on methods of covariant types can not be worked around in any way.

And it prevents correct implementation of certain features which also can not be worked around in any other way. I really wanted to mark this as a bug, instead of an improvement, because it blocks essential functionality that can not be achieved any other way. Feel free to change it to a bug if you agree with me. It is said that if a type system can't correctly type the "contains problem" then it is not worth going through the effort to enforce types at all at compile-time. Refer to the following links for the proof and example use cases which support this. 

http://stackoverflow.com/questions/8360413/selectively-disable-subsumption-in-scala-correctly-type-list-contains 

Note it is not just contains which is affected yet it is the ability to correct type anything that is an input to a covariant type: 

https://groups.google.com/d/msg/scala-debate/LWBz3-Q0pNI/rWG4hImvCQgJ
https://groups.google.com/d/msg/scala-language/lChLcLES_Dk/b78ywjsHcbMJ

I thought about whether this should go in scala-debate, yet it seems to me that unless my technical analysis is incorrect, that this is an unarguable issue. I gather that one of the very important principles for Scala is to not unnecessary hinder expression. This seems to be very clear case of making it impossible to properly implement covariant types in some cases. Having said that, I am open to the possibility that I have some myopia in my technical analysis, so I am cross-posting it here to see what others have to say?

Btw, if you've visited that SO page before, consider looking again and see my latest comments under the top-voted answers. My latest technical analysis is that Kim Stebel's and Peter Schmitz's answer are both unarguably entirely incorrect (they have incorrect functionality). I hadn't realized that before. Also soc's contention ignores that subsuming to Any causes general problems (see my links above) regardless whether he thinks something going on with an orthogonal concept of equals makes typing irrelevant.

No animosity intended. It is difficult to say I think something is incorrect without potentially flaming someone (and I might be wrong). Nothing personal and I have appreciated all the help I got on that issue from all the named parties.

This issue concerns very much. So much so that I joined SO back in 2011 just to post on this issue. And so much so that this issue is still haunting me today, when I consider how I want a library to be correctly implemented.

[Paul Phillips]

You're probably not aware of what I added to -Xlint over a year ago (but that still means it's only in 2.11.) I forgot about it myself. In my own code, if Any or AnyVal is inferred, it's invariably a mistake. For the very few occasions when I use one of those types, I'm willing to suck it up and type the few extra characters. This doesn't solve your problem, unless maybe you are very brave and combine it with -Xfatal-warnings, but may expand your options.

% scala3 -Xlint

Welcome to Scala version 2.11.0-20130917-220248-d45a3c8cc8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_40).

Type in expressions to have them evaluated.

Type :help for more information.

scala> List(1, "a")

<console>:8: warning: a type was inferred to be `Any`; this may indicate a programming error.

              List(1, "a")

                   ^

res0: List[Any] = List(1, a)

scala> List(1) contains "a"

<console>:8: warning: a type was inferred to be `Any`; this may indicate a programming error.

              List(1) contains "a"

                               ^

res1: Boolean = false

scala> List(1) contains 'a'

<console>:8: warning: a type was inferred to be `AnyVal`; this may indicate a programming error.

              List(1) contains 'a'

                               ^

res2: Boolean = false

scala> List(1) contains 1

res3: Boolean = true

scala> 

[Francois]

On 18/09/2013 17:22, Paul Phillips wrote:

You're probably not aware of what I added to -Xlint over a year ago (but that still means it's only in 2.11.) I forgot about it myself. In my own code, if Any or AnyVal is inferred, it's invariably a mistake. For the very few occasions when I use one of those types, I'm willing to suck it up and type the few extra characters. This doesn't solve your problem, unless maybe you are very brave and combine it with -Xfatal-warnings, but may expand your options.

Oh mother of Gods! I WANT that, why isn't it the default ?

(ok, I do know why it's not the default, I followed some threads on the subject... But seeing it in action is such a delight....)

% scala3 -Xlint

Welcome to Scala version 2.11.0-20130917-220248-d45a3c8cc8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_40).

Type in expressions to have them evaluated.

Type :help for more information.

scala> List(1, "a")

<console>:8: warning: a type was inferred to be `Any`; this may indicate a programming error.

              List(1, "a")

                   ^

res0: List[Any] = List(1, a)

scala> List(1) contains "a"

<console>:8: warning: a type was inferred to be `Any`; this may indicate a programming error.

              List(1) contains "a"

                               ^

res1: Boolean = false

scala> List(1) contains 'a'

<console>:8: warning: a type was inferred to be `AnyVal`; this may indicate a programming error.

              List(1) contains 'a'

                               ^

res2: Boolean = false

scala> List(1) contains 1

res3: Boolean = true

scala> 

--
You received this message because you are subscribed to the Google Groups "scala-language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-languag...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
Francois ARMAND
http://rudder-project.org
http://www.normation.com

[Shelby]

On Wednesday, September 18, 2013 11:22:18 PM UTC+8, Paul Phillips wrote:

You're probably not aware of what I added to -Xlint over a year ago (but that still means it's only in 2.11.) I forgot about it myself. In my own code, if Any or AnyVal is inferred, it's invariably a mistake. For the very few occasions when I use one of those types, I'm willing to suck it up and type the few extra characters. This doesn't solve your problem, unless maybe you are very brave and combine it with -Xfatal-warnings, but may expand your options.

At the scala-debate thread I linked in the OP, Justin Decour was making the same point that collections of Any are invariably a design error and I agreed with him of course. Does any one disagree?

As you admit and as far as I can see w.r.t. to the issue I raise, your warning can only tell me where to insert boilerplate at the call site to work around subsumption. See my answer at SO for an example of the boilerplate that needs to be written at the call site to get desired functionality. I don't consider such boilerplate a viable work around at all.

The problem I am raising now is that there is no way to write a method for a container (not just collections) which refuse to input values that is are neither a supertype nor subtype of the covariant type parameter of the container. I think everyone assumes that this is because the Liskov Substitution Principle requires that the input value have a type that is *equal* or a supertype of the covariant type parameter of the container.

The *equal* is I think the salient factor to recognize. If no subsumption is performed on that input value type, then only supertypes and subtypes can fulfill that equal or supertype criteria.

In my SO example, Int as the input value type is not a super nor subtype of String as the element type, unless the Int is subsumed to Any and then it is a supertype of String.

So my proposed fix is some new syntax to declare at the method definition site that the input value type should not be subsumed to any supertype, only to *equal* to the covariant element type. With my proposed fix, if the input value type is a supertype without subsumption, then that is fine and the method is subsumed to that type. So with this proposed fix implemented at a definition site, the call site can still override it with a cast on the input value argument.

The discussion at the scala-debate thread linked in the OP, explains how this problem makes it impossible to overload the + operator for both append and concatenation. This is a clue that this problem may have far reaching implications and probably infects the libraries in more ways than the two major cases enumerated thus far.

[Shelby]

On Thursday, September 19, 2013 12:14:13 AM UTC+8, Shelby wrote:

On Wednesday, September 18, 2013 11:22:18 PM UTC+8, Paul Phillips wrote:

You're probably not aware of what I added to -Xlint over a year ago (but that still means it's only in 2.11.) I forgot about it myself. In my own code, if Any or AnyVal is inferred, it's invariably a mistake. For the very few occasions when I use one of those types, I'm willing to suck it up and type the few extra characters. This doesn't solve your problem, unless maybe you are very brave and combine it with -Xfatal-warnings, but may expand your options.

At the scala-debate thread I linked in the OP, Justin Decour was making the same point that collections of Any are invariably a design error and I agreed with him of course. Does any one disagree?

As you admit and as far as I can see w.r.t. to the issue I raise, your warning can only tell me where to insert boilerplate at the call site to work around subsumption. See my answer at SO for an example of the boilerplate that needs to be written at the call site to get desired functionality. I don't consider such boilerplate a viable work around at all.

The problem I am raising now is that there is no way to write a method for a container (not just collections) which refuse to input values that is are neither a supertype nor subtype of the covariant type parameter of the container. I think everyone assumes that this is because the Liskov Substitution Principle requires that the input value have a type that is *equal* or a supertype of the covariant type parameter of the container.

The *equal* is I think the salient factor to recognize. If no subsumption is performed on that input value type, then only supertypes and subtypes can fulfill that equal or supertype criteria.

I want to emphasize that limiting subsumption does not violate the requirement of the Liskov Substitution Principle, rather it just allows the method to do something sane.

If a method can't exclude types which are neither supertypes nor substypes, then don't even bother writing an obfuscating lie:

trait List[+A] {

   def contains[B >: A](x: B): Boolean

}

Just write instead and be honest with yourself:

trait List[+A] {

   def contains(x: Any): Boolean

}

In my SO example, Int as the input value type is not a super nor subtype of String as the element type, unless the Int is subsumed to Any and then it is a supertype of String.

So my proposed fix is some new syntax to declare at the method definition site that the input value type should not be subsumed to any supertype, only to *equal* to the covariant element type. With my proposed fix, if the input value type is a supertype without subsumption, then that is fine and the method is subsumed to that type.

You see the Liskov Substitution Principle won't be violated, because supertypes will still be allowed.

 

So with this proposed fix implemented at a definition site, the call site can still override it with a cast on the input value argument.

I mean that if you want to force the method to subsume to Any, then input an Any. As Paul wrote, in those rare cases you want this misbehavior then you can write a tiny bit of boilerplate to get it.

 

The discussion at the scala-debate thread linked in the OP, explains how this problem makes it impossible to overload the + operator for both append and concatenation. This is a clue that this problem may have far reaching implications and probably infects the libraries in more ways than the two major cases enumerated thus far.

As far as I can see, there is no other way to work around this problem, except to turn on some warning mechanism and do boilerplate at every call site. 

I want to emphasize that the solution Martin had proposed (which is linked from the SO page) is not a solution to this problem. Requiring a context bound of Eq[B] is about as effective as renaming Any to Eq[B], because most every type will have an Eq[B] thus it will still be the same problem.

[Paul Phillips]

On Wed, Sep 18, 2013 at 9:39 AM, Shelby <she...@coolpage.com> wrote:

Just write instead and be honest with yourself:

trait List[+A] {

   def contains(x: Any): Boolean

}

It's not written that way precisely so I can issue that warning about inferring Any, because if written like this, Any is not inferred - it is explicit. You are right that outside of implementation issues this is an equivalent formulation.

[Paul Phillips]

On Wed, Sep 18, 2013 at 9:39 AM, Shelby <she...@coolpage.com> wrote:

I mean that if you want to force the method to subsume to Any, then input an Any.

I have argued many times that Any and AnyVal should never be inferred. A legitimate counterargument is that this only serves to push the problem upward to the next point of type intersection - it doesn't address it in any fundamental way. Next up on the block: "Product with Serializable".

scala> List(None, Nil)

res0: List[Product with Serializable] = List(None, List())

scala> 

[Francois]

Yeah, I remind these threads....
(but nonetheless, warning on Any and AnyVal today is better than a general solution in 3 years...)

[Shelby]

On Thursday, September 19, 2013 12:39:24 AM UTC+8, Shelby wrote: 

So with this proposed fix implemented at a definition site, the call site can still override it with a cast on the input value argument.

I mean that if you want to force the method to subsume to Any, then input an Any. As Paul wrote, in those rare cases you want this misbehavior then you can write a tiny bit of boilerplate to get it.

Also for example if you have a collection of Some[Int] and you call contains (that is fixed with my proposal) inputting a None, then you would need to cast that to Option, otherwise you would receive a compile error. Yet at least you have an error to work from and that is a probably a rare boilerplate price to pay, because most likely the type of your collection is Option[Int].

It is true that subsumption reduces boilerplate, yet at the cost of inability to exclude any type. That is why I propose to make it a definition site feature, so library designers can weigh the tradeoff.

[Vlad Patryshev]

I believe a natural transformation from covariant functor to contravariant is a solution to this.

Thanks,
-Vlad

--

[Shelby]

My proposed fix solves that in the context of getting the correct behavior on methods as I described. Since that List.apply in your example is calling the concatenation method, then my fix solves the above. You will get the type List[Nothing] in the above example with my fix. 

[Paul Phillips]

On Wed, Sep 18, 2013 at 9:51 AM, Shelby <she...@coolpage.com> wrote:

It is true that subsumption reduces boilerplate, yet at the cost of inability to exclude any type. That is why I propose to make it a definition site feature, so library designers can weigh the tradeoff.

Another thing I implemented the last time I was playing with this was pluggable advice - as it was becoming increasingly clear to me that the language was never going to do what I wanted, so my best-case scenario was to provide myself with an avenue to fix it at least for myself.

Skeletally, since the only point at the time was to be able to instruct the compiler not to infer Any - the interface could of course be much richer - it was something like

trait Guidance {

  def inferrable(tpe: Universe#Type): Boolean

}

object NoGuidance extends Guidance {

  def inferrable(tpe: Universe#Type) = true

}

object NoAnyGuidance extends Guidance {

  def inferrable(tpe: Universe#Type): Boolean = !(tpe exists (tp => tp.typeSymbol.fullName.toString == "scala.Any"))

}

Then once you supply command line option -Dscala.guidance=some.pkg.NoAnyGuidance, key type inference decisions are routed through the guidance object for validation. From my POV the only level of control I really need is to inhibit some of scala's less rigorous ambitions, like coercing Longs into Floats and crushing everything into Any - so it would be enough to have a few Boolean returning methods which all default to constant true, but which I can selectively make false based on the inputs.

[Shelby]

Excuse me I was wrong, you would receive a compiler error because List[Nothing] is not a supertype nor subtype of Option[Nothing]. So you would be forced to cast somewhere, e.g. if we had first-class unions List[Option[Nothing] | List[Nothing]](None, Nil).

If we had first-class unions, perhaps we would want another per method attribute to tell compiler to subsume to the union.

[Paul Phillips]

On Wed, Sep 18, 2013 at 10:12 AM, Shelby <she...@coolpage.com> wrote:

If we had first-class unions, perhaps we would want another per method attribute to tell compiler to subsume to the union.

How to communicate that you want a union type to be inferred vs. wanting subsumption to take place - and more importantly, how NOT to have to communicate this, because we'd very much like it to magically do what we want on an inference by inference basis - and how the tax might take shape is one of the great open questions about how union types would work out in actual practice.

[Shelby]

I am thinking the guidance needs to be per method to be useful, otherwise as you said, you can't pick any target, the problem just shifts up the hierarchy.

If had first-class unions, we only need one choice which is don't subsume or subsume to the union. Both of those are not typing holes unlike what we have now with subsumption to some arbitrary upper bound.

[Paul Phillips]

On Wed, Sep 18, 2013 at 10:23 AM, Shelby <she...@coolpage.com> wrote:

I am thinking the guidance needs to be per method to be useful, otherwise as you said, you can't pick any target, the problem just shifts up the hierarchy.

The richer interface I alluded to would include all the contextual information, such as the method definition and the call site being evaluated.

[Shelby]

 Why not annotate the method as follows?

trait List[+A] {

   def concat[B >: A](x: B): List[B] // accepts subsumption (to Any or if we have unions then to the union)

   def contains[B >= A](x: B): Boolean // denies subsumption

}

[Shelby]

s/concat/append/ 

[Shelby]

On Thursday, September 19, 2013 1:36:15 AM UTC+8, Shelby wrote:

On Thursday, September 19, 2013 1:15:00 AM UTC+8, Paul Phillips wrote:

On Wed, Sep 18, 2013 at 10:12 AM, Shelby <she...@coolpage.com> wrote:

If we had first-class unions, perhaps we would want another per method attribute to tell compiler to subsume to the union.

How to communicate that you want a union type to be inferred vs. wanting subsumption to take place - and more importantly, how NOT to have to communicate this, because we'd very much like it to magically do what we want on an inference by inference basis - and how the tax might take shape is one of the great open questions about how union types would work out in actual practice.

 Why not annotate the method as follows?

trait List[+A] {

   def append[B >: A](x: B): List[B] // accepts subsumption (to Any or if we have unions then to the union)

   def contains[B >= A](x: B): Boolean // denies subsumption

}

Then your example will type correctly without boilerplate assuming unions are implemented.

scala> List(None, Nil)

res0: List[Option[Nothing] | List[Nothing]] = List(None, List())

Unions are the only solution to your example. In the meantime, perhaps we get that subsumption to Any warning code you implemented for 2.11.

And then my contains example is solved.

Seems to me the complete solution is annotation of methods as above and the implementation of first-class unions.

Am I missing some case?

[Shelby]

I have chosen your answer that you added to my question at SO. It provides the correct functionality. That is great that there is work around.

Yet it seems your code has a higher overhead than what I am proposing to annotate a method to tell the Scala compiler to selectively not do subsumption.

Would it be possible to instead employ an implicit object somehow, so we don't have to implicitly construct the anonymous class on each call to contains?

[Jan Vanek]

IMO, one solution is to allow to explicitly "type" the implicit this parameter in method definitions, like e.g.:

trait Seq[+T] {
  def contains[S](this: Seq[S])(v: S): Boolean
                   ^
                   |
                   +-- implicit this parameter, must be 1st when declared
}

It allows to "capture" the the type of this as seen at the call-site by the typer. When we have:

val l = List(1, 2, 3)
l.contains("a")

the typer infers S = Int and reports an error. One could still do (l: List[Any]).contains("a"). AFAICS there is no substitution, and thus no violation of LSP. I believe @uncheckedVariance could be removed from the std-lib. Some other use cases are listed here:

http://permalink.gmane.org/gmane.comp.lang.scala.internals/14123

Granted, it would have to be explored further, e.g. to find out what possible types of this are eligible, and which are not. Last time I was not able to convince Paul about it, unfortunately...

Regards,
Jan

[Shelby]

I addressed Paul's argument against Vlad's solution at the other discussion thread:

https://groups.google.com/d/msg/scala-language/eU0yPJq589Y/TWfdC9K2zNEJ

Please read that post, as I make what I think is an unarguable logic about why all our problems with Any derive from the subsumption of the covariant type parameter of methods. And thus I am able to claim that my proposal upthread is the complete fix to all our problems with Any. I am awaiting any counter-logic and/or counter-examples.

I am thinking my proposal is the unified, simple, and clean solution. So we can be finished with this problem once and for all.

This is such a fundamental issue that annoys all of us. I hope we can reach consensus and get this fixed finally. Jason Zaugg has closed my tracker issue on this as "Incomplete" until we can reach some consensus, so can we try to reach consensus? (politics aside let us state all the technical issues and see if there is any solution that solves all of them)

Note that as far as I know Ceylon solves this problem with their subsumption to the inferred union. So we have competition on the horizon and I am hoping we can get this fixed in Scala before Ceylon gains momentum. The solution I have proposed is better than what Ceylon can do, because it gives us the per method annotation to subsume or not subsume. Also my proposal is orthogonal to the implementation of first-class unions, i.e. it can subsume to Any in the meantime when the method annotation is to accept subsumption.

On Thursday, September 19, 2013 5:21:06 AM UTC+8, JV wrote:

IMO, one solution is to allow to explicitly "type" the implicit this parameter in method definitions, like e.g.:

trait Seq[+T] {
  def contains[S](this: Seq[S])(v: S): Boolean

Unlike my proposed fix it doesn't allow to override the definition site choice by using a cast at the call site to a mutual upper bound. And it doesn't allow explicit (not subsumed) supertypes to subsume the type. Thus as far as I can see, it is a less general and less unified solution.

 

                   ^
                   |
                   +-- implicit this parameter, must be 1st when declared
}

It allows to "capture" the the type of this as seen at the call-site by the typer. When we have:

val l = List(1, 2, 3)
l.contains("a")

the typer infers S = Int and reports an error. One could still do (l: List[Any]).contains("a"). AFAICS there is no substitution, and thus no violation of LSP. I believe @uncheckedVariance could be removed from the std-lib. Some other use cases are listed here:

http://permalink.gmane.org/gmane.comp.lang.scala.internals/14123

All your use case examples will work with my proposed fix.

[Shelby]

I not good a politics. I put my foot in my mouth often.

I want to push for consensus, because this problem is major and fundamental (concerning correctness). And it is ridiculous that this problem has been around so long in a language that claims to be one of the most sophisticated for static typing.

If we don't reach consensus on fundamentals such as subsumption and unions, then what can soon happen is our ecosystem becomes fractured.

Some may choose Ceylon.

I will be forced to fork the compiler to get the library I want.

Paul may be forced to fork the compiler in his preferred way.

Etc.

I think we are rational and we can lay out all the use cases and corner cases that come to mind and then analyze which solution addresses them all.

The alternative is soon coming chaos, because we can't leave this is as and expect people to be satisfied. The reasons are numerous that this is major and fundamental. Do we need to enumerate them in order to motivate people to see that consensus needs to be pushed for now?

[Shelby]

On Thursday, September 19, 2013 11:09:14 AM UTC+8, Shelby wrote:

I not good a politics. I put my foot in my mouth often.

Haha, I am not good at politics. Shows that my mind is moving faster than I want to pay attention to all this English language noise I have to write just to get something done.

 

I want to push for consensus, because this problem is major and fundamental (concerning correctness). And it is ridiculous that this problem has been around so long in a language that claims to be one of the most sophisticated for static typing.

The reason I suck at politics, is that when I see there is only one solution, I implement. And that takes a lot less time than trying to get everyone focused on an issue and pulling their understanding along.

OTOH, I am not omniscient, and thus bouncing things off the group can be fruitful.

So I am saying that if there is no action on this, I will proceed to use Vlad's solution hidden behind the syntactical sugar I have proposed (unless someone can convince me there is a more unified syntax), which will be horridly slow. So then after proof-of-concept, then I would have to endeavor to fork the compiler.

I would prefer we can reach a consensus and get it into the official Scala compiler.

Otherwise, I can't wait. I will not wait. And I won't spend all my time on politics.  Signing off for now and see where you all go with this.

[Shelby]

Unfortunately on more careful thought, Vlad's solution doesn't give the correct functionality and it has the same but opposing problem as all other proposed solutions there and including JV's post here. The only solution I see is the one I have proposed.

I changed my comment on your SO answer which explains the issue:

Unfortunately, your answer is somewhat incorrect, and only achieves the opposing functionality as Kim Stebel's and my answer. (new Super) contains (new Sub) correctly does not generate an error. However, (new Sub) contains (new Super) won't compile! Same but opposing problem that I discovered in my answer, c.f. the types Super and Sub in my answer. And your solution has high overhead, and Stebel's is not as well integrated, thus my answer remains the best so far. The only solution is as I have described the my UPDATE on my question. 

[Shelby]

I elaborated as follows (and I will add a new point below) at the other discussion thread

https://groups.google.com/d/msg/scala-language/eU0yPJq589Y/in9hek1rU1cJ

The fix is not to stop any supertypes from qualifying for the input value. Rather that it is desirable, c.f. my reply to JV upthread. The goal is to stop the subsumption of the *destination* type parameter when the source input value type is neither a supertype nor a subtype-- thus it is not about picking an arbitrary choice for a mutual upper bound, rather it is about not subsuming the *destination* type parameter. Once we get a correct conceptualization of the issue then the overall perspective of the problem with Any changes.

And this is where I encourage Paul to have a paradigm shift on his perspective. Paul it appears to me (please correct me if I am wrong) that you want to view this (based on your posts in the above linked thread) as local information that can be decided with a global algorithm. Whereas, I think that is entirely the wrong way to conceptualize what is really going on in the type system.

We need both options locally, to turn off or leave on the subsumption of the *destination* covariant type parameter.

The key is to recognize that with static typing there is a known type for each reference (and it may reference a subtype but that is irrelevant to my point) and when assigning from one type to another, only the source type can be subsumed to the destination type. The destination type is never subsumed unless it is a covariant type parameter on a function or method, e.g. include the constructor function.

Thus realize that the notion that all types have a supertype of Any thus they are really Any is the wrong way to conceptualize a static typing system. They can subsume to Any, but they are not Any until they do. And the rules of subsumption are never subsume the destination type unless the destination type is a covariant parameter.

Thus we can see the only time that subsumption to a mutual upper bound of Any can occur is when the destination type is a covariant parameter. Thus the solution is to focus on where that is the case, and selectively turn off the mutual subsumption of the destination covariant type parameter as I have done in my proposed solution.

Never we will want to turn off the subsumption of the source type as that would be the antithesis of subtyping.

The problem again with Vlad's solution is he is turning off subtyping, yet  in the contravariant direction (the Container[+T]). Ditto JV's post upthread. The other proposed solutions in the answers at SE/SO were turning off subtyping in the covariant direction. None of these are solutions. Only my proposed solution at the upthread is.

Hope that makes it clear, now I am really out-the-door (and not going to walk back up the mountain again as I just did to type this).

[Vlad Patryshev]

Well, I just found a theoretical solution to an old nagging problem; how it looks in the code does not matter much. You can always import something.

Thanks,
-Vlad

[Simon Ochsenreither]

Note that as far as I know Ceylon solves this problem with their subsumption to the inferred union. So we have competition on the horizon and I am hoping we can get this fixed in Scala before Ceylon gains momentum. The solution I have proposed is better than what Ceylon can do, because it gives us the per method annotation to subsume or not subsume. Also my proposal is orthogonal to the implementation of first-class unions, i.e. it can subsume to Any in the meantime when the method annotation is to accept subsumption.

Welcome to the party: https://groups.google.com/d/topic/scala-language/J8LpYDmrOCg/discussion :-)

You can fill out this questionnaire regarding union types if you like and post your solutions to that thread to compare with others: https://gist.github.com/soc/5776653

[Johannes Rudolph]

On Wed, Sep 18, 2013 at 11:21 PM, Jan Vanek <j3v...@gmail.com> wrote:
> trait Seq[+T] {
> def contains[S](this: Seq[S])(v: S): Boolean
> ^
> |
> +-- implicit this parameter, must be 1st when declared
> }

You can have that already now:

implicit class SeqContains[T](seq: Seq[T]) { def contains(t: T): Boolean = ... }

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

[Shelby]

On Thursday, September 19, 2013 1:15:09 PM UTC+8, Shelby wrote: 

Thus realize that the notion that all types have a supertype of Any thus they are really Any is the wrong way to conceptualize a static typing system. They can subsume to Any, but they are not Any until they do. And the rules of subsumption are never subsume the destination type unless the destination type is a covariant parameter.

Thus we can see the only time that subsumption to a mutual upper bound of Any can occur is when the destination type is a covariant parameter. Thus the solution is to focus on where that is the case, and selectively turn off the mutual subsumption of the destination covariant type parameter as I have done in my proposed solution.

Never we will want to turn off the subsumption of the source type as that would be the antithesis of subtyping.

There is another case where the source types subsume to a mutual upper bound (possibly Any) when the type parameter of a function is (in) the type of two or more input values, e.g.

def add[A] = _ : A + _ : A

However, there is nothing for us to fix here (other than hopefully someday soon offering subsumption to the union instead of Any/AnyVal), because we can already control (turn off) the subsumption, e.g.

def add[A,B](op1: A, op2: B)(implicit evidence: Either[A<:<B, B<:<A]) = op1 + op2

def add[A,B] (op1: A, op2: B)(implicit tc:Plus[A,B]) = tc.plus(op1, op2)

All operators, e.g. even if-else, can be modeled as type parametrized functions. Thus these are the only ways I see that expressions subsume.

So I maintain the only fix we need where we can't currently control subsumption is on typed parameters of functions that have a lower bound. I am now removing the requirement for covariant and method, because it applies to any function with a type parameter with a lower bound. The lower bound happens to be required for methods that interact with a covariant type parameter of the class or trait.

[Shelby]

On Thursday, September 19, 2013 5:29:55 PM UTC+8, Johannes Rudolph wrote:

On Wed, Sep 18, 2013 at 11:21 PM, Jan Vanek <j3v...@gmail.com> wrote:
> trait Seq[+T] {
>   def contains[S](this: Seq[S])(v: S): Boolean
>                    ^
>                    |
>                    +-- implicit this parameter, must be 1st when declared
> }

You can have that already now:

implicit class SeqContains[T](seq: Seq[T]) { def contains(t: T): Boolean = ... }

Well yes you just copied Vlad's solution, as I said JV and Vlad's solution both turn off subtyping in the covariant direction, i.e. you can't pass a subtype of T, as you should be able to in my proposed fix. JV and Vlad's strategies causes problems (but less worse than the other strategies in the answers to my SO/SE question linked at the OP):

https://groups.google.com/d/msg/scala-debate/LWBz3-Q0pNI/TiPDzojXmqIJ

[Shelby]

On Thursday, September 19, 2013 8:15:41 PM UTC+8, Shelby wrote:

On Thursday, September 19, 2013 5:29:55 PM UTC+8, Johannes Rudolph wrote: 

Well yes you just copied Vlad's solution, as I said JV and Vlad's solution both turn off subtyping in the covariant direction, i.e. you can't pass a subtype of T

s/covariant/contravariant/ 

Hi Vlad,

As far as I can see there is no import or solution that can remove the problem with the theoretical solution. I am bit rusty on category theory, but I think we would be able to see from the theory why it loses the supertypes and only retains the subtypes.

The type parameter of a function with a lowerbound is accepting both supertypes and subtypes of the lowerbound as input. Your formulation can only accept the subtypes. When you turned off subsumption, you also turned off the supertypes, because you've reversed the direction of the covariance on the type parameter, i.e. it is not using an upper bound T instead of a lowerbound.

Whereas in the other answers to my question on SO/SE turn off subsumption by not allowing the input type to subsume at all, thus they allow the supertypes of the lowerbound but do not allow subtypes of the lowerbound.

Thus none of these strategies are correct-- they both kill subtyping in some form.

[Shelby]

On Thursday, September 19, 2013 8:15:41 PM UTC+8, Shelby wrote:

Well yes you just copied Vlad's solution, as I said JV and Vlad's solution both turn off subtyping in the covariant direction, i.e. you can't pass a subtype of T

s/subtype of T/supertype of T/ 

On Thursday, September 19, 2013 8:29:15 PM UTC+8, Shelby wrote: 

not using an upper bound T instead of a lowerbound.

s/not/now/

I am worn out. Hit my physical limit, need to get off computer for while.

[Rich Oliver]

On Wednesday, September 18, 2013 5:46:18 PM UTC+1, Paul Phillips wrote:

On Wed, Sep 18, 2013 at 9:39 AM, Shelby <she...@coolpage.com> wrote:

I mean that if you want to force the method to subsume to Any, then input an Any.

I have argued many times that Any and AnyVal should never be inferred. A legitimate counterargument is that this only serves to push the problem upward to the next point of type intersection - it doesn't address it in any fundamental way. Next up on the block: "Product with Serializable".

scala> List(None, Nil)

res0: List[Product with Serializable] = List(None, List())

Maybe I'm misunderstanding, but why does the compiler have to infer anything in such a situation?  Surely Any, AnyVal and AnyRef should require explicit type annotation. I trust that if we ever have union types they won't be infered. Maybe I'm over optimistic.

[Oliver Ruebenacker]

     Hello,

On Wed, Sep 18, 2013 at 12:46 PM, Paul Phillips <pa...@improving.org> wrote:

On Wed, Sep 18, 2013 at 9:39 AM, Shelby <she...@coolpage.com> wrote:

I mean that if you want to force the method to subsume to Any, then input an Any.

I have argued many times that Any and AnyVal should never be inferred. A legitimate counterargument is that this only serves to push the problem upward to the next point of type intersection - it doesn't address it in any fundamental way. Next up on the block: "Product with Serializable".

scala> List(None, Nil)

res0: List[Product with Serializable] = List(None, List())

scala> 

  Unintentional widening of types is only a problem if the code still compiles but behaves differently. The only case I can think of widened types of arguments of overloaded methods or non-explicit return types. How about a warning in such cases?

  The warning would be issues if the following three are true:

  (1) The type is not declared

  (2) The type is a widened: it is the result of two more branches merging, resulting in a type wider than the type of either branch.

  (3) The type is in a position where it could affect control flow:

    (a) It is a method return type

    (b) It is used as an argument in an overloaded method such that it satisfies the signature of more than one instance (e.g. Any satisfies f(Any) and f(String))

  The warning can always be resolved by adding a type declaration.

     Best,

     Oliver

--

Oliver Ruebenacker
IT Project Lead at PanGenX (http://www.pangenx.com)

Be always grateful, but never satisfied.

[Shelby]

Done:

https://groups.google.com/d/msg/scala-language/J8LpYDmrOCg/jBYraMN5P6kJ 

[Lex Spoon]

On Fri, Sep 20, 2013 at 10:49 AM, Oliver Ruebenacker <cur...@gmail.com> wrote:
> The warning can always be resolved by adding a type declaration.

As tempting as they are, compiler warnings tend to not work well for
the user experience of using a compiler. One of two things ends up
happening:

- The warning is left in place, in which case the warnings gradually
mount up and turn the compiler output into spam.

- The warning is addressed, in which case it's indistinguishable from
a compiler error.

Do or do not do. There's a real art in deciding where to draw the line....

Lex

[Shelby]

Agreed. I designed unions so we don't need the warning:

https://gist.github.com/shelby3/6639849 

[Grzegorz Kossakowski]

Would turning warnings into errors + providing an escape hatch (in form of some annotation) would be a good idea?

I'd like to promote certain warnings into errors if we are reasonably sure that they are not catching false positives.

--
Grzegorz Kossakowski

Scalac hacker at Typesafe

twitter: @gkossakowski

github: @gkossakowski

[Paul Phillips]

Many things which are warnings instead of errors are only that way because I lacked the power to make them errors. I agree with lex that warnings suck, but often total silence from the compiler is even worse. 

[Lex Spoon]

On Sat, Sep 21, 2013 at 2:22 PM, Grzegorz Kossakowski
<grzegorz.k...@gmail.com> wrote:
> Would turning warnings into errors + providing an escape hatch (in form of
> some annotation) would be a good idea?

I figure, yes. In the case of a false alert, the person looking at the
error message can use the escape hatch to say so. It's a simple way
for the computer and the programmer to have a simple back-and-forth
dialog.

Lex

[Shelby]

And then hopefully more likely to report any false positive as a bug, and then put a link to the bug report next to their annotation.