Webkit not compiling?

[Matt Farmer]

I'm getting the following when I try to compile webkit on 2.10.1 or 2.10.3...

[error] /Users/matt/Sites/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:362: type mismatch;
[error]  found   : List[?T5 => ?T5 forSome { type ?T5 <: T }]
[error]  required: List[T => T]
[error]       stuff.toList.collect {
[error]                            ^
[error] /Users/matt/Sites/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:488: type mismatch;
[error]  found   : List[?T19 => ?T19 forSome { type ?T19 <: T }]
[error]  required: List[this.ValueType => this.ValueType]
[error]       }.toList
[error]         ^
[error] /Users/matt/Sites/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:590: type mismatch;
[error]  found   : List[?T33 => ?T33 forSome { type ?T33 <: T }]
[error]  required: List[this.ValueType => this.ValueType]
[error]       }.toList
[error]         ^
[error] /Users/matt/Sites/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:619: no type parameters for method flatMap: (f: AbstractScreen.this.FilterOrValidate[T] => scala.collection.GenTraversableOnce[B])(implicit bf: scala.collection.generic.CanBuildFrom[List[AbstractScreen.this.FilterOrValidate[T]],B,That])That exist so that it can be applied to arguments (AbstractScreen.this.FilterOrValidate[T] => List[?T36 => ?T36] forSome { type ?T36 <: T })
[error]  --- because ---
[error] argument expression's type is not compatible with formal parameter type;
[error]  found   : AbstractScreen.this.FilterOrValidate[T] => List[?T36 => ?T36] forSome { type ?T36 <: T }
[error]  required: AbstractScreen.this.FilterOrValidate[T] => scala.collection.GenTraversableOnce[?B]
[error]     }, stuff.toList.flatMap {
[error]                     ^
[error] /Users/matt/Sites/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:619: type mismatch;
[error]  found   : AbstractScreen.this.FilterOrValidate[T] => List[?T36 => ?T36] forSome { type ?T36 <: T }
[error]  required: AbstractScreen.this.FilterOrValidate[T] => scala.collection.GenTraversableOnce[B]
[error]     }, stuff.toList.flatMap {
[error]                             ^
[error] /Users/matt/Sites/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:757: no type parameters for method flatMap: (f: AbstractScreen.this.FilterOrValidate[T] => scala.collection.GenTraversableOnce[B])(implicit bf: scala.collection.generic.CanBuildFrom[Seq[AbstractScreen.this.FilterOrValidate[T]],B,That])That exist so that it can be applied to arguments (AbstractScreen.this.FilterOrValidate[T] => List[?T46 => ?T46] forSome { type ?T46 <: T })
[error]  --- because ---
[error] argument expression's type is not compatible with formal parameter type;
[error]  found   : AbstractScreen.this.FilterOrValidate[T] => List[?T46 => ?T46] forSome { type ?T46 <: T }
[error]  required: AbstractScreen.this.FilterOrValidate[T] => scala.collection.GenTraversableOnce[?B]
[error]           override val setFilter = stuff.flatMap {
[error]                                          ^
[error] /Users/matt/Sites/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:757: type mismatch;
[error]  found   : AbstractScreen.this.FilterOrValidate[T] => List[?T46 => ?T46] forSome { type ?T46 <: T }
[error]  required: AbstractScreen.this.FilterOrValidate[T] => scala.collection.GenTraversableOnce[B]
[error]           override val setFilter = stuff.flatMap {
[error]                                                  ^
[error] /Users/matt/Sites/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:794: no type parameters for method flatMap: (f: AbstractScreen.this.FilterOrValidate[T] => scala.collection.GenTraversableOnce[B])(implicit bf: scala.collection.generic.CanBuildFrom[Seq[AbstractScreen.this.FilterOrValidate[T]],B,That])That exist so that it can be applied to arguments (AbstractScreen.this.FilterOrValidate[T] => List[?T48 => ?T48] forSome { type ?T48 <: T })
[error]  --- because ---
[error] argument expression's type is not compatible with formal parameter type;
[error]  found   : AbstractScreen.this.FilterOrValidate[T] => List[?T48 => ?T48] forSome { type ?T48 <: T }
[error]  required: AbstractScreen.this.FilterOrValidate[T] => scala.collection.GenTraversableOnce[?B]
[error]           override val setFilter = stuff.flatMap {
[error]                                          ^
[error] /Users/matt/Sites/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:794: type mismatch;
[error]  found   : AbstractScreen.this.FilterOrValidate[T] => List[?T48 => ?T48] forSome { type ?T48 <: T }
[error]  required: AbstractScreen.this.FilterOrValidate[T] => scala.collection.GenTraversableOnce[B]
[error]           override val setFilter = stuff.flatMap {
[error]                                                  ^

Has anyone else seen this?

[Diego Medina]

Haven't tried it but you don't really need to compile lift for those versions, compiling for 2.10.0 should be enough

Sent from my tablet
Diego

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
 
---
You received this message because you are subscribed to the Google Groups "Lift" group.
To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[David Pollak]

Great... so the type system changed between 2.10.0 and 2.10.x

Please put some explicit types on variables like "stuff" and see what happens.

--

--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
 
---
You received this message because you are subscribed to the Google Groups "Lift" group.
To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Brick Alloy http://brickalloy.com

Lift, the simply functional web framework http://liftweb.net

Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im

[Matt Farmer]

I'm going to try and clean compile just to be sure it's not sbt fudging the pumpkin, but yeah that wouldn't surprise me. -____-

[Matt Farmer]

Yeah, looks like the type system changed. Clean compile yields same errors. I'll try to get a fix up and rolling.

[Matt Farmer]

Yep, so I see what's screwing it up. For some reason lines like:

stuff.collect {

  case AFilter(f) => f

}.toList

Don't properly tell the compilers newer than 2.10.0 that f is of type T=>T. Instead, it seems, you have to do something like...

stuff.collect {

  case filter: AFilter[T] => filter.f

}

... and it's happy. Though, this smells like the sort of thing that wouldn't be legit due to erasure. Am I wrong about that?

[Matt Farmer]

Can't keep playing with it this afternoon, but I went ahead and opened a PR with what I've done so far. This gets the webkit compiling, but someone familiar with LiftScreen should hammer it because I haven't had the opportunity to.

https://github.com/lift/framework/pull/1508

[Diego Medina]

Thanks, hopefully this will also get us closer to build for Scala 2.11.x

Diego
Sent from my cell

--

[Peter Brant]

Does the compiler complain (warn) about type erasure there?  or can the 2.10 pattern matcher do more here?  The fix looks fine to me.  Whatever works.  

The compiler might actually in the right here.  I don't remember the exact scenario offhand, but we have noticed that it's sometimes possible to add validation functions with an invalid type to a field definition (which then fail with a runtime exception when it should have been a compile exception).

Pete

--

[Peter Brant]

Looks like this was a type inference bug that was fixed in 2.10.1: https://issues.scala-lang.org/browse/SI-5189 .  The initial report more or less describes the situation we ran into, but the distilled version from paulp is easier to understand.  

Does anybody have any ideas how to fix this properly?  FilterOrValidate has to be covariant in T and AFilter has to be invariant in T (and that's the problem in a nutshell).

If not, just doing

case AFilter(v) => v.asInstanceOf[T => T]

is probably the appropriate band-aid for now.  It's what 2.9.x was effectively doing before (and what Matt's version does with the warning in 2.10.x).  In other words, we're no worse off than before.

I played around a bit with Naftoli's suggestion too, but I don't think there is a solution there.

Pete

[Matt Farmer]

My concern with asInstanceOf is that it can throw errors if the type doesn't match. I'd be fine with...

case AFilter(v) if v.isInstanceOf[T=>T] => v

which just throws away anything that doesn't match. Though you may still need the asInstanceOf. I haven't had the opportunity to play with any of Naftoli's solutions yet.

[Matt Farmer]

So someone check me on this, but it doesn't look like my code actually generates a warning... o_O

So.... should we just merge it in? haha.

[Peter Brant]

Thanks to type erasure that's actually no better (i.e. you're effectively writing

case AFilter(v) if v.isInstanceOf[Function1[_, _]] => v

).

I'll play around with Naftoli's second suggestion a bit.  It's an interesting idea.  Since FilterOrValidate is sealed we're dealing with a finite, known number of subclasses and there are only two subclasses that have the problematic invariant T.

That said, I don't think it will help with the root issue of the unsafe implicit conversion from T => T to FilterOrValidate[T].  I don't see a way out of that besides making T invariant on FilterOrValidate (which would likely be a pain in the neck for consumers of LiftScreen).

Pete

[Peter Brant]

Oh, I've got great confidence it will generate a warning.  :-)  I'll check though.

Pete

[Matt Farmer]

Any luck with this Peter?

[Peter Brant]

Nothing really satisfactory, I'm afraid.  Naftoli and I chatted about various ways of approaching this and I think we could restructure the code to make the compiler happy without resorting to a type cast.  I don't see a way to fix the fundamental type soundness issue without changing the API (or making the existing API much less usable) though so I'm not sure it's worth it.

In short, I think we're back to the .asInstanceOf band-aid as the best thing to do for now.  I think it would be worth exploring splitting the params arg into more type specific parts.  That definitely wouldn't be backwards compatible though so it would have to be a 3.0 thing.

I'll open a pull request for the band-aid.

Pete

P.S.  Re the warning, I had to add

scalacOptions in ThisBuild += "-unchecked"

to build.sbt to turn on unchecked warnings.  I then get e.g.

[warn] /home/pbrant/git/lift/framework/web/webkit/src/main/scala/net/liftweb/http/LiftScreen.scala:399: The outer reference in this type test cannot be checked at run time.

[warn]   protected final case class AFilter[T](f: T => T) extends FilterOrValidate[T]

Having emitted that warning, the compiler apparently doesn't emit additional ones.  However, if you look at the bytecode generated by the pattern matcher (below), it's pretty clear a warning would have been justified.

public final scala.collection.immutable.List apply(net.liftweb.http.AbstractScreen$FilterOrValidate);

  Code:

   0: aload_1

   1: astore_2

   2: aload_2

   3: instanceof #21; //class net/liftweb/http/AbstractScreen$AFilter

   6: ifeq 45

   9: aload_2

   10: checkcast #21; //class net/liftweb/http/AbstractScreen$AFilter

   13: astore_3

   14: getstatic #27; //Field scala/collection/immutable/List$.MODULE$:Lscala/collection/immutable/List$;

   17: getstatic #32; //Field scala/Predef$.MODULE$:Lscala/Predef$;

   20: iconst_1

   21: anewarray #34; //class scala/Function1

   24: dup

   25: iconst_0

   26: aload_3

   27: invokevirtual #38; //Method net/liftweb/http/AbstractScreen$AFilter.f:()Lscala/Function1;

   30: aastore

   31: checkcast #40; //class "[Ljava/lang/Object;"

   34: invokevirtual #44; //Method scala/Predef$.wrapRefArray:([Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray;

   37: invokevirtual #47; //Method scala/collection/immutable/List$.apply:(Lscala/collection/Seq;)Lscala/collection/immutable/List;

   40: astore 4

   42: goto 50

   45: getstatic #52; //Field scala/collection/immutable/Nil$.MODULE$:Lscala/collection/immutable/Nil$;

   48: astore 4

   50: aload 4

   52: areturn