SI-7475 private members, pattern typing, and redundancy elimination

[Jason Zaugg]

I've been helping a few projects adapt to the results of SI-7545: private members are now no longer inherited.

Before the fix, you this was incorrectly allowed:

class C { private def foo = 0; def test(d: D) = d.foo}; class D extends C

I found a breakage in Akka and another in Scalaz.

The former was just a straight bug; on review of the code, they had no expectation that is ought to have worked in the first place.

But the Scalaz case was actually more interesting, and you could argue that its a regression.

Consider:

class C {

  private def foo: Any = 0

  this match {

    case d: D =>

      d.foo // foo is not a member

      // Would be okay:

      //

      // d.asInstanceOf[C with D].foo

      // (d: C).foo

  }

}

class D extends C

Here, we'd usually infer the type of `d` to have an intersection of the scrutinee type and the typed pattern type. But `infer.intersect(typeOf[C], typeOf[D])` eliminates the `C`. But while it is redundant on account of subtyping, it isn't redundant when it comes to private methods.

So `intersect` is actually too eager to eliminate redundancies. If the inferred type is going to be used in a place that has private access, to a class, it should not be eliminated. I prototyped this, and it doesn't seem too hard to implement. 

An alternative would by to just never eliminate redundancies. But there would be an efficiency argument against that, as well as likely leaks into error messages.

I have a feeling that this might pop up in the wild a bit.

Should we pursue this smart intersect? If so, when? If we want to leave if out of 2.11.0, could it hit 2.11.1? (Maybe: I think we can reason that it doesn't change erasure)

I'll raise a ticket for this after discussion.

-jason

[Rex Kerr]

I don't think the pattern match changes anything.  I thought the argument against private member inheritance was:

  class C { private def foo = 0; def test(d: D) = d.foo }

  - class D extends C

  + class D extends C { def foo = "salmon" }

where recompiling D in this way would leave C accessing its own foo instead of D's foo and wouldn't necessarily be caught at runtime.

This argument is identical whether or not you perform a match.  There's never any doubt that a D is also a C, match or no.

So I think both behaviors (match and not) should succeed or fail together.

  --Rex

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Som Snytt]

Are companions a special case of private access?

It's kind of obvious that x.foo means foo in the module, but I'm not sure why, if x will have type C with C.type.

Some evil corner of my heart hopes you say static overload resolution.

In fact, if the signatures don't match, it does complain about an overload. OK, never mind.  There's nothing special about the overloading of foo; the alternatives just happen to have the same signature.

class C {
  private def foo: Any = 0
}

object C extends C {
  private def foo: Any = 1
  def c = (this: C) match {
    case x: C.type => x.foo
    case y         => y.foo
  }
}

I thought maybe it had to do with the traditional nursery rhyme, "infer a singleton type, make your mamma gripe."

On Fri, Feb 14, 2014 at 1:57 PM, Jason Zaugg <jza...@gmail.com> wrote:

[Adriaan Moors]

In line with another topical platitude, I'll "say it with code":

class C {

  private def foo: Any = 0

}

object C extends C {

  private def foo: Any = 1

  def d = (new C).foo // yep, companions are in the private scope (returns 0)

  def c = (this: C) match { // returns 1

    case x: (C with C.type) => x.foo // same as x: C in this context

    case y         => y.foo

  }

}

[Adriaan Moors]

Right, but the spec says the type of this pattern is C with D.

Since (x: C with D).foo and (x: D).foo are observably different, the implemented "simplification" in intersect deviates from the spec, which mandates "C with D".

The principled fix, IMO, would be https://github.com/scala/scala/pull/3535,

but I'm not sure the severity of the issue warrants changes this late in the RC game.

[Rex Kerr]

On Fri, Feb 14, 2014 at 6:09 PM, Adriaan Moors <adr...@typesafe.com> wrote:

Right, but the spec says the type of this pattern is C with D.

Then this is a bug in the spec, isn't it?  As it will lead to unsound code, at least according to the argument for why C can't access its own foo in D.  The logic is no different.  If that is incorrect behavior, then the spec is wrong to specify C with D instead of just D (unless "C with D" is understood to be strictly a subclass of C, and therefore not have access to private members of C).

  --Rex

[Jason Zaugg]

I don't think that separate compilation scenario is supportable. If there wasn't a private method in C there might have been an extension method. Adding a public method to D requires a recompile of its clients 

[Rex Kerr]

Then how does one support the original restriction that (d: D) = d.foo shouldn't work?  It's C's foo; C knows that D <: C, so there's nothing else it could be.

I grant that because of linearization, "with" is not identically intersection (if it were, D <: C would imply D === C with D).  But if D <: C and you infer from a match that this C is actually a D, you don't know you're any more C-ish than you do if you just know you are a D <: C.

Altering visibility on the basis of different ways to express the exact same knowledge about the type seems like a bad idea to me: bad for reasoning about your code as a language user, and bad for bugs in corner cases as a language implementer.

  --Rex

[Jason Zaugg]

On Sat, Feb 15, 2014 at 10:04 AM, Rex Kerr <ich...@gmail.com> wrote:

Then how does one support the original restriction that (d: D) = d.foo shouldn't work?  It's C's foo; C knows that D <: C, so there's nothing else it could be.

The rule originates from Java. I actually don't know the philosophy behind it, but I suspect it is to be able to modularize the specification of members and access. One can determine the members of `D` in a location-independent manner. Then, at the call site, accessibility is checked.

 

I grant that because of linearization, "with" is not identically intersection (if it were, D <: C would imply D === C with D).  But if D <: C and you infer from a match that this C is actually a D, you don't know you're any more C-ish than you do if you just know you are a D <: C.

Altering visibility on the basis of different ways to express the exact same knowledge about the type seems like a bad idea to me: bad for reasoning about your code as a language user, and bad for bugs in corner cases as a language implementer.

We already have to implement the distinction in members of `C with D` and `D`; the spec says that "private members aren't inherited". By which implies that they *are* intersected.

Furthermore, pattern typing is specced to give the intersection of the scrutinee type and the pattern type. The current implementation doesn't faithfully follow this, which never mattered until we fixed SI-7475.

-jason

[Adriaan Moors]

On Sat, Feb 15, 2014 at 2:26 AM, Jason Zaugg <jza...@gmail.com> wrote:

We already have to implement the distinction in members of `C with D` and `D`; the spec says that "private members aren't inherited". By which implies that they *are* intersected.

I think Rex is right. Sounds to me we should clarify that this implication does not hold. Technically, a refinement has its own class, so that the type `C with D` is much like the type `E`, where `class E extends C with D {}` (you're of course not allowed to define concrete value members here). Would you expect to be able to call C's private member on an instance of E, even if you were in a scope that allowed private access to C's members? 

(True, the argument used to forbid this for real classes doesn't hold for refinement classes because they can't define concrete value members -- private or not -- that would shadow those of their parents.)

[Jason Zaugg]

The problem is that you will outlaw:

trait T { self: U => private def foo = 0; this.foo }

-jason

[Adriaan Moors]

You'd have to say (this: T).foo, similarly to the fix in the pattern matching example.

You could say that the compiler should subsume the type of the target of the selection so that the private member can be found (what if your type is C with D and both define a private member? or the dominant one in the linearisation at first doesn't, and then on an incremental change introduces one?). 

Maybe that's why we used to include private members in findMembers results, because together with refchecks, it implemented this subsumption (poorly). I'd prefer a simpler rule. If you want private access in ambiguous situations, it'll cost you.

--

[Rex Kerr]

Why?  (I mean in principle, not according to current implementation details.)  Doesn't
  T { self: U => ... }
imply at ... that the object is "U with T"-typed?  And hence that the private def is _only_ accessible from another U-with-T?

If the "private def" part thinks that the type of the enclosing class is a different type than the "this." thinks, isn't that just another cause for confusion and error?

  --Rex

 

[Adriaan Moors]

We were operating under the hypothesis that `T with U` does not contain the private members of its parents,

as private members are not inherited, and you could interpret `T with U` as an anonymous class that `extends T with U`,

and thus its members are defined by what it inherits from T and U.

Thus, for `U <: T` and `foo` a private member of `T`, `(x: T).foo ok` would not (in general) imply `(x: U).foo ok`,

even though that of course holds for non-private members.

[Rex Kerr]

Not sure who this was to, but if it was to me, then

trait T { self: U =>

  private def foo = 0

  this.foo

}

can only fail if you are using different notions of what the type is for "private" and "this".

If everyone agrees that the real type is _$1 extends U with T, then it's all cool: this is type _$1, and foo is private to type _$1.  If everyone agrees that the real type is just T (albeit T <: U), then this is of type T and it's private to type T, so again it's cool.  If "private" thinks one thing and "this" thinks the other, then we have a problem.

Maybe, just maybe,

  self.foo

would be a problem since self is explicitly typed as U.  (Assuming you interpret it that way, rather than that :U is a restriction on T, and self is just an alias for this.)

  --Rex

--

[Adriaan Moors]

I think we're talking about different things.

I was describing a way of computing the members of a type (as recently re-implemented by Jason in FindMembers).

The current interpretation of the spec is that a refinement type `T with U` has a different set of members from a class that `extends T with U`.

The refinement type sees the private members of its parents, whereas the class type does not.

Thus, (x: T with U).foo is allowed if foo is a member in T or U (private or not).

Now, define class C extends T with U and suddendly (x: C) does not have that member.

In code:

scala> trait  T { private def foo = 1 }

defined trait T

scala> trait U

defined trait U

scala> (??? : (T with U)).foo

<console>:10: error: method foo in class T$class cannot be accessed in T with U // if we can navigate to the right scope, it's there!

              (??? : (T with U)).foo

                                 ^

scala> trait  T { private def foo = 1 ; (??? : (T with U)).foo } // aha!

defined trait T

scala> class C extends T with U

defined class C

scala> (??? : C).foo

<console>:11: error: value foo is not a member of C // no way we can ever access foo on C -- from any "privacy scope" 

              (??? : C).foo

                        ^

scala> trait  T { private def foo = 1 ; (??? : C).foo } // let's try that trick again....

<console>:10: error: value foo is not a member of C

       trait  T { private def foo = 1 ; (??? : C).foo }

                                                  ^

[Adriaan Moors]

Sorry, that last try should've been:

scala> trait  T { private def foo = 1 ; class C extends T with U; (??? : C).foo }

<console>:12: error: value foo is not a member of T.this.C

       trait  T { private def foo = 1 ; class C extends T with U; (??? : C).foo }

                                                                            ^

[Naftoli Gugenheim]

Perhaps, trait T { this: U => } does not imply that this is just U, nor T with U --- but rather T & U, the intersection. So even though your argument that C with D is a subtype of and distinct from C stands, you can distinguish this case.

On the other hand once we say that scala 2 actually has (inexpressible) intersection types distinct from "with" refinements, and that they include the private members, then why shouldn't pattern matches involve them. 

 

--