Curious about the plan for SI-6548

[Bill Venners]

Hello PTB (powers that be),

I was curious what the plan for this issue is:

https://issues.scala-lang.org/browse/SI-6548

It prevents jackson-module-scala from being compiled under 2.10.0, which is being used at a client of ours that is eager to upgrade to 2.10. I figure it is too late for 2.10.0, but if so, when would a fix be likely?

Thanks.

Bill

[Grzegorz Kossakowski]

The issue talkes about reflection and exception thrown at runtime. Here you say it prevents some code from compiling. Can you clarify?

--
Grzegorz Kossakowski

[Christopher Currie]

Specifically, it prevents our tests from passing. Annotations are a big part of what makes Jackson work, and not being able to reflect them correctly breaks a lot of our use cases.

[Grzegorz Kossakowski]

On 7 December 2012 00:47, Christopher Currie <chris...@currie.com> wrote:

Specifically, it prevents our tests from passing. Annotations are a big part of what makes Jackson work, and not being able to reflect them correctly breaks a lot of our use cases.

Do you use Scala reflection in your tests?

--
Grzegorz Kossakowski

[Christopher Currie]

We use it in the core library code. Without going into to too much detail, the library is heavily dependent upon reflection. The 2.9 code base uses scalabeans to get the job done, but scalabeans doesn't have a 2.10 version, and is also currently missing some key features.

A very kind contributor did a lot of work to port jackson-module-scala to 2.10 and drop the dependency on scalabeans, but I can't release it until SI-6548 is fixed, because the annotation reflection is broken.

Because I know there's 2.10 demand out there, I'm currently (slowly) working at trying to get either the scalabeans maintainers to merge some patches and look at a 2.10 release, or making an internal build myself that I can shade into the library, but the infrastructure is tricky and I have limited time.

[Eugene Burmako]

The bug is fixed in https://github.com/scala/scala/pull/1727. I'll do
my best to push it into 2.10.1.

In the future if something reflection- or macro-related is a blocker,
please, ping me personally, and I'll evaluate and try to fix the issue
asap.

[Christopher Currie]

Eugene, 

Thanks very much for your work on this issue, I look forward to testing it on snapshots when the pull request has been merged. Is there any chance that this might be back ported to 2.10.0 prior to release?

Christopher

[Jason Zaugg]

On Sun, Dec 9, 2012 at 5:26 AM, Christopher Currie <chris...@currie.com> wrote:

Thanks very much for your work on this issue, I look forward to testing it on snapshots when the pull request has been merged. Is there any chance that this might be back ported to 2.10.0 prior to release?

We wouldn't block the 2.10.0 release for this as reflection is marked experimental, and as such is likely to need a few bug fixes like this in 2.10.1.

If there is another blocking bug in core functionality discovered, we could consider this one. But we're pretty hopeful that RC5 might be promoted to the release.

The only good news I can offer you is that 2.10.1 is scheduled for early 2013.

-jason

[Paolo G. Giarrusso]

How hard would it be for a user to use 2.10 + that pull request?

In a perfect world, it shouldn't take more than compiling scalac from source. But this world isn't perfect.

Is there some setting in SBT/Maven/... to use a different maven artifact for the compiler/library/etc., or just libs from some folder? Wasn't Paul complaining about this?

Otherwise, I'm gonna guess that if you just use scalac, you just need to compile from source, and if you use SBT it's going to be hard, lest you hack SBT to use a different Maven artifact for the compiler and you hack SBT's launcher to use a different Maven artifact for SBT itself. Alternatively, maybe one can release a custom version of the compiler in a custom Maven repository.

[Paul Phillips]

On Fri, Dec 14, 2012 at 2:56 PM, Paolo G. Giarrusso <p.gia...@gmail.com> wrote:

How hard would it be for a user to use 2.10 + that pull request?

In a perfect world, it shouldn't take more than compiling scalac from source. But this world isn't perfect.

In a perfect world it would take decidedly less than that!

% sbt -scala-version 2.10.0+54a84a36d5

is more like it. Although this still sets the "perfect world" bar pretttttty low.

Is there some setting in SBT/Maven/... to use a different maven artifact for the compiler/library/etc., or just libs from some folder? Wasn't Paul complaining about this?

Is this a trick question? Isn't it "yes" for most values of this?

I imagine what I was complaining about is the amazing difficulty I had getting sbt to use the scala-reflect.jar which went with the library and compiler jars, which I never did figure out how to make work.

// demonstration build.sbt

name := "proj"

scalaVersion := "2.10.0-RC5"

scalaHome := Some(file("/scala/inst/scala-2.10.1-20121210-141937-7fe7d25379"))

logLevel in Global := Level.Debug

libraryDependencies <++= (scalaVersion)(sv =>

  Seq(

    "org.scala-lang" % "scala-reflect" % sv,

    "org.scala-lang" % "scala-compiler" % sv

  )

)

If you use this to compile something, the debug output shows the classpath:

// [debug] Calling Scala compiler with arguments  (CompilerInterface):

// [debug]   -d

// [debug]   /project/Dec14-06PM/target/scala-2.10/classes

// [debug]   -bootclasspath

// [debug]   <snip jdk classes>:/scala/inst/scala-2.10.1-20121210-141937-7fe7d25379/lib/scala-library.jar

// [debug]   -classpath <see below>

// Placed on individual lines for readability

/project/Dec14-06PM/target/scala-2.10/classes

/Users/paulp/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.10.0-RC5.jar

/scala/inst/scala-2.10.1-20121210-141937-7fe7d25379/lib/scala-compiler.jar

So it's custom library, custom compiler, and stock reflect jar, despite my handling reflect and compiler identically and also despite the fact that scala-compiler depends on scala-reflect (in real life: I don't know if that's true in the mind of sbt.) Setting scalaVersion to that exact version only leads to it failing to find any scala-reflect.jar.

[Paolo G. Giarrusso]

Hm. Since the current thread is about a bug restricted to scala-reflect.jar, that's somewhat bad news. But this should be easy to fix.

Looking at SBT sources (without being an expert), the problem seems that scala-compiler and scala-library are hardcoded as special throughout SBT, for instance:

final class ScalaInstance(val version: String, val loader: ClassLoader, val libraryJar: File, val compilerJar: File, val extraJars: Seq[File], val explicitActual: Option[String]) extends xsbti.compile.ScalaInstance

(I'm not sure if this code is about the host or guest Scala instance, but the pattern seems always the same). I guess the handling of extra JARs has to be strengthened:

https://github.com/harrah/xsbt/issues/626

[Josh Suereth]

It's definitely possible to use your own by providing your own ScalaInstance.  If that patch is binary compatible, you'd be ok.  If not, you're in for a larger world of pain.

[Paolo G. Giarrusso]

Il giorno domenica 16 dicembre 2012 21:14:52 UTC+1, Josh Suereth ha scritto:

It's definitely possible to use your own by providing your own ScalaInstance.  If that patch is binary compatible, you'd be ok.  If not, you're in for a larger world of pain.

Do you mean that's doable via SBT configuration? I'm curious to see how. Also, that assumption is hardcoded also elsewhere.

But if the bug report I mentioned is invalid, feel free to act on it:

https://github.com/harrah/xsbt/issues/626

The particular pull request originally discussed has some renames in scala.reflect.runtime, but types don't change so it seems that it could be made binary compatible.

[Paolo G. Giarrusso]

Il giorno sabato 15 dicembre 2012 03:39:26 UTC+1, Paul Phillips ha scritto:

Is there some setting in SBT/Maven/... to use a different maven artifact for the compiler/library/etc., or just libs from some folder? Wasn't Paul complaining about this?

Is this a trick question? Isn't it "yes" for most values of this?

Believe it or not, I hadn't fully noticed :-D
Also, for values of "this" outside scala, I haven't had a chance to observe.