Scala success!

Showing 1-235 of 235 messages
Scala success! Alex Cruise 12/3/12 1:10 PM
Hi folks,

I found out about Avian today, compiled an openjdk build, and the Scala REPL works!  (Mostly... The last declaration failed the first time with an NPE, then succeeded the second time)

Thanks for the great work so far!  I'll be sticking around!

-0xe1a


[alex@alex-desktop linux-x86_64-openjdk]$ ./avian -cp /home/alex/.m2/repository/org/scala-lang/scala-library/2.9.2/scala-library-2.9.2.jar:/home/alex/.m2/repository/org/scala-lang/scala-compiler/2.9.2/scala-compiler-2.9.2.jar 'scala.tools.nsc.interpreter.Runner' -usejavacp
Failed to created JLineReader: java.lang.NoClassDefFoundError: scala/tools/jline/console/ConsoleReader
Falling back to SimpleReader.
Welcome to Scala version 2.9.2 (Avian, Java 1.7.0_09-icedtea).
Type in expressions to have them evaluated.
Type :help for more information. 
scala> val foo = (1 to 100).toList
foo: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100) 
scala> foo.map(x => x * x) 
res0: List[Int] = List(1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144, 169, 196, 225, 256, 289, 324, 361, 400, 441, 484, 529, 576, 625, 676, 729, 784, 841, 900, 961, 1024, 1089, 1156, 1225, 1296, 1369, 1444, 1521, 1600, 1681, 1764, 1849, 1936, 2025, 2116, 2209, 2304, 2401, 2500, 2601, 2704, 2809, 2916, 3025, 3136, 3249, 3364, 3481, 3600, 3721, 3844, 3969, 4096, 4225, 4356, 4489, 4624, 4761, 4900, 5041, 5184, 5329, 5476, 5625, 5776, 5929, 6084, 6241, 6400, 6561, 6724, 6889, 7056, 7225, 7396, 7569, 7744, 7921, 8100, 8281, 8464, 8649, 8836, 9025, 9216, 9409, 9604, 9801, 10000) 
scala> foo.map(x => x*x*x)
java.lang.NullPointerException
        at scala.tools.nsc.symtab.SymbolTable$perRunCaches$$anonfun$clearAll$1.reflMethod$Method1(SymbolTable.scala:140)
        at scala.tools.nsc.symtab.SymbolTable$perRunCaches$$anonfun$clearAll$1.apply(SymbolTable.scala:140)
        at scala.tools.nsc.symtab.SymbolTable$perRunCaches$$anonfun$clearAll$1.apply(SymbolTable.scala:135)
        at scala.collection.mutable.HashSet.foreach(HashSet.scala:76)
        at scala.tools.nsc.symtab.SymbolTable$perRunCaches$.clearAll(SymbolTable.scala:135)
        at scala.tools.nsc.Global$Run.compileSources(Global.scala:1025)
        at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compileAndSaveRun(IMain.scala:756)
        at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compile(IMain.scala:731)
        at scala.tools.nsc.interpreter.IMain$Request.compile(IMain.scala:873)
        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576)
        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:543)
        at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:694)
        at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:745)
        at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:651)
        at scala.tools.nsc.interpreter.ILoop.processLine$1(ILoop.scala:542)
        at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:550)
        at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:822)
        at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:838)
        at scala.tools.nsc.interpreter.Runner$.main(Runner.scala:10)
        at scala.tools.nsc.interpreter.Runner.main(Runner.scala)

Abandoning crashed session.
scala> foo.map(x => x*x*x)
res3: List[Int] = List(1, 8, 27, 64, 125, 216, 343, 512, 729, 1000, 1331, 1728, 2197, 2744, 3375, 4096, 4913, 5832, 6859, 8000, 9261, 10648, 12167, 13824, 15625, 17576, 19683, 21952, 24389, 27000, 29791, 32768, 35937, 39304, 42875, 46656, 50653, 54872, 59319, 64000, 68921, 74088, 79507, 85184, 91125, 97336, 103823, 110592, 117649, 125000, 132651, 140608, 148877, 157464, 166375, 175616, 185193, 195112, 205379, 216000, 226981, 238328, 250047, 262144, 274625, 287496, 300763, 314432, 328509, 343000, 357911, 373248, 389017, 405224, 421875, 438976, 456533, 474552, 493039, 512000, 531441, 551368, 571787, 592704, 614125, 636056, 658503, 681472, 704969, 729000, 753571, 778688, 804357, 830584, 857375, 884736, 912673, 941192, 970299, 1000000)

Re: Scala success! Joel Dice 12/3/12 1:29 PM
Hi Alex,

On Mon, 3 Dec 2012, Alex Cruise wrote:

> Hi folks,
> I found out about Avian today, compiled an openjdk build, and the Scala REPL
> works! �(Mostly... The last declaration failed the first time with an NPE,
> then succeeded the second time)

It looks a lot like this bug:

https://groups.google.com/group/avian/msg/9de455e1d200b202

Especially since it's intermittent and has not been resolved upstream
AFAIK.  The whole discussion is available here:

https://groups.google.com/group/avian/browse_thread/thread/6ff9f841b20ff636/9de455e1d200b202

Pablo Guerrero had some ideas about how to address it, but I don't know if
they found their way into a Scala release.
>       � � � � atscala.tools.nsc.symtab.SymbolTable$perRunCaches$$anonfun$clearAll$1.reflMet
>       hod$Method1(SymbolTable.scala:140)
>       � � � � atscala.tools.nsc.symtab.SymbolTable$perRunCaches$$anonfun$clearAll$1.apply(S
>       ymbolTable.scala:140)
>       � � � � atscala.tools.nsc.symtab.SymbolTable$perRunCaches$$anonfun$clearAll$1.apply(S
>       ymbolTable.scala:135)
>       � � � � at
>       scala.collection.mutable.HashSet.foreach(HashSet.scala:76)
>       � � � � atscala.tools.nsc.symtab.SymbolTable$perRunCaches$.clearAll(SymbolTable.scala
>       :135)
>       � � � � at
> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/avian/-/_B-HckKn0CMJ.
> To post to this group, send email to av...@googlegroups.com.
> To unsubscribe from this group, send email to
> avian+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/avian?hl=en.
>
>
Re: Scala success! Pablo Guerrero 12/12/12 8:10 AM
Hi Alex, 

As Joel said I was trying to solve this problem, but in the end I didn't  find a convincing solution.

The problem is with the weak reference, it's described in detail in the threads that Joel is pointing.  But after after checking more in detail, I'm not sure the solution to this problem doesn't introduce new ones, because they introduce the weak reference to solve other problem, if it's just removed this problem wil come back.

In any case, I didn't try with scala 2.10-RC3 that it's the last available. Checking quickly the code, it seems to me that they are doing code generation differently as I cannot find some of the things that were there before. If you have you environment ready it would be nice to give it a try.

I also copy here some more info I got while checking this issue (not present on the other threads) that can direct you to the right places in case you want to debug it.
Use the code from v2.9.1 tag as lines doesn't correspond with master links now.

General Info in scala compiler
http://www.sts.tu-harburg.de/people/mi.garcia/ScalaCompilerCorner/
http://lampwww.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/

The compiled code is wrong for structural types, the problem is generated form the file

https://github.com/scala/scala/blob/v2.9.1/src/library/scala/reflect/generic/UnPickler.scala

At line 401 you will see { def setFlag(mask: Long): this.type } that it's an structural type. The code generated for this includes the weak reference that fails in avian.

The code generation is made in this file:

https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/transform/CleanUp.scala

Look at lines:

Line 168: Look at the newMethod implementation
Line 249: The real implementation !!!! This code maps to the problem shown in the first answer here: http://www.scala-lang.org/node/11807

The bytecode generation for the JVM is on this file:
https://github.com/scala/scala/blob/v2.9.1/src/compiler/scala/tools/nsc/backend/jvm/GenJVM.scala

Then JClass seems to be important too, it's here.
https://github.com/scala/scala/tree/v2.9.1/src/fjbg/ch/epfl/lamp/fjbg

The information it's a little bit disperse, but it's all there. If you have any question, let me know.

I hope it's useful. 

Pablo

On Monday, December 3, 2012 10:29:27 PM UTC+1, Joel Dice wrote:
Hi Alex,

On Mon, 3 Dec 2012, Alex Cruise wrote:

> Hi folks,
> I found out about Avian today, compiled an openjdk build, and the Scala REPL
> works! �(Mostly... The last declaration failed the first time with an NPE,
> then succeeded the second time)

It looks a lot like this bug:

https://groups.google.com/group/avian/msg/9de455e1d200b202

Especially since it's intermittent and has not been resolved upstream
AFAIK.  The whole discussion is available here:

https://groups.google.com/group/avian/browse_thread/thread/6ff9f841b20ff636/9de455e1d200b202

Pablo Guerrero had some ideas about how to address it, but I don't know if
they found their way into a Scala release.

>
> Thanks for the great work so far! �I'll be sticking around!
>
> -0xe1a
>
>
>       [alex@alex-desktop linux-x86_64-openjdk]$ ./avian -cp/home/alex/.m2/repository/org/scala-lang/scala-library/2.9.2/scala-library-
> 2.9.2.jar:/home/alex/.m2/repository/org/scala-lang/scala-compiler/2.9.2/sca
>       la-compiler-2.9.2.jar 'scala.tools.nsc.interpreter.Runner'
>       -usejavacp
>       Failed to created JLineReader: java.lang.NoClassDefFoundError:
>       scala/tools/jline/console/ConsoleReader
>       Falling back to SimpleReader.
>       Welcome to Scala version 2.9.2 (Avian, Java 1.7.0_09-icedtea).
>       Type in expressions to have them evaluated.
>       Type :help for more information.�
>
>       scala> val foo = (1 to 100).toList
>       foo: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,
>       14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
>       30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45,
>       46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61,
>       62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77,
>       78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93,
>       94, 95, 96, 97, 98, 99, 100)�
>
>       scala> foo.map(x => x * x)�
>       res0: List[Int] = List(1, 4, 9, 16, 25, 36, 49, 64, 81, 100,
>       121, 144, 169, 196, 225, 256, 289, 324, 361, 400, 441, 484, 529,
>       576, 625, 676, 729, 784, 841, 900, 961, 1024, 1089, 1156, 1225,
>       1296, 1369, 1444, 1521, 1600, 1681, 1764, 1849, 1936, 2025,
>       2116, 2209, 2304, 2401, 2500, 2601, 2704, 2809, 2916, 3025,
>       3136, 3249, 3364, 3481, 3600, 3721, 3844, 3969, 4096, 4225,
>       4356, 4489, 4624, 4761, 4900, 5041, 5184, 5329, 5476, 5625,
>       5776, 5929, 6084, 6241, 6400, 6561, 6724, 6889, 7056, 7225,
>       7396, 7569, 7744, 7921, 8100, 8281, 8464, 8649, 8836, 9025,
>       9216, 9409, 9604, 9801, 10000)�
>
>       scala> foo.map(x => x*x*x)
>
>       java.lang.NullPointerException
>       � � � � atscala.tools.nsc.symtab.SymbolTable$perRunCaches$$anonfun$clearAll$1.reflMet
>       hod$Method1(SymbolTable.scala:140)
>       � � � � atscala.tools.nsc.symtab.SymbolTable$perRunCaches$$anonfun$clearAll$1.apply(S
>       ymbolTable.scala:140)
>       � � � � atscala.tools.nsc.symtab.SymbolTable$perRunCaches$$anonfun$clearAll$1.apply(S
>       ymbolTable.scala:135)
>       � � � � at
>       scala.collection.mutable.HashSet.foreach(HashSet.scala:76)
>       � � � � atscala.tools.nsc.symtab.SymbolTable$perRunCaches$.clearAll(SymbolTable.scala
>       :135)
>       � � � � at
>       scala.tools.nsc.Global$Run.compileSources(Global.scala:1025)
>       � � � � atscala.tools.nsc.interpreter.IMain$ReadEvalPrint.compileAndSaveRun(IMain.sca
>       la:756)
>       � � � � at
>       scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compile(IMain.scala:731)
>       � � � � at
>       scala.tools.nsc.interpreter.IMain$Request.compile(IMain.scala:873)
>       � � � � at
>       scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576)
>       � � � � at
>       scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:543)
>       � � � � at
>       scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:694)
>       � � � � at
>       scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:745)
>       � � � � at
>       scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:651)
>       � � � � at
>       scala.tools.nsc.interpreter.ILoop.processLine$1(ILoop.scala:542)
>       � � � � at
>       scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:550)
>       � � � � at
>       scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:822)
>       � � � � at
>       scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:838)
>       � � � � at
>       scala.tools.nsc.interpreter.Runner$.main(Runner.scala:10)
>       � � � � at scala.tools.nsc.interpreter.Runner.main(Runner.scala)
>
>       Abandoning crashed session.
>       scala> foo.map(x => x*x*x)
>       res3: List[Int] = List(1, 8, 27, 64, 125, 216, 343, 512, 729,
>       1000, 1331, 1728, 2197, 2744, 3375, 4096, 4913, 5832, 6859,
>       8000, 9261, 10648, 12167, 13824, 15625, 17576, 19683, 21952,
>       24389, 27000, 29791, 32768, 35937, 39304, 42875, 46656, 50653,
>       54872, 59319, 64000, 68921, 74088, 79507, 85184, 91125, 97336,
>       103823, 110592, 117649, 125000, 132651, 140608, 148877, 157464,
>       166375, 175616, 185193, 195112, 205379, 216000, 226981, 238328,
>       250047, 262144, 274625, 287496, 300763, 314432, 328509, 343000,
>       357911, 373248, 389017, 405224, 421875, 438976, 456533, 474552,
>       493039, 512000, 531441, 551368, 571787, 592704, 614125, 636056,
>       658503, 681472, 704969, 729000, 753571, 778688, 804357, 830584,
>       857375, 884736, 912673, 941192, 970299, 1000000)
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/avian/-/_B-HckKn0CMJ.
> To post to this group, send email to av...@googlegroups.com.
> To unsubscribe from this group, send email to
> avian+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/avian?hl=en.
>
>
Re: Scala success! Pablo Guerrero 12/12/12 8:28 AM
The problem is still present in scala 2.10 master:

If you check on https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/transform/CleanUp.scala line 225 there is a method addStaticMethodToClass

This method is generating the wrong bytecode structure that we can see in the coment here: http://www.scala-lang.org/node/11807#comment-53067

To make it clear, it's creating the soft reference at the beginning (in the code using mkNewPolyCache) without having any hard reference to it. And then is using it.

If you check the correct code, it's creating a hard reference in cache, and using it. That's fine. And only at the end creating a soft reference when it's not needed anymore.

The solution is to modify the code in addStaticMethodToClass to behave like to correct solution without breaking something. I tried it, in order to provide some kind of patch, but in the end I had to move to something else and I couldn't continue.

Cheers,
Pablo
Re: Scala success! Simon Ochsenreither 1/29/13 1:49 PM
Hi Joel, Hi Pablo!

Compiling Avian with make tails=true continuations=true openjdk=$JAVA_HOME and using the latest code from Scala's trunk:

LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/:. \
./avian-dynamic -cp /home/soc/Entwicklung/scala/build/pack/lib/scala-library.jar:/home/soc/Entwicklung/scala/build/pack/lib/scala-reflect.jar:/home/soc/Entwicklung/scala/build/pack/lib/scala-compiler.jar:/home/soc/Entwicklung/scala/build/pack/lib/jline.jar 'scala.tools.nsc.MainGenericRunner' -usejavacp
Welcome to Scala version 2.11.0-20130127-161738-f389f6a18f (Avian, Java 1.7.0_09).

Type in expressions to have them evaluated.
Type :help for more information.

scala> val foo = (1 to 100).toList
foo: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100)

scala> foo.map(x => x * x)
res0: List[Int] = List(1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144, 169, 196, 225, 256, 289, 324, 361, 400, 441, 484, 529, 576, 625, 676, 729, 784, 841, 900, 961, 1024, 1089, 1156, 1225, 1296, 1369, 1444, 1521, 1600, 1681, 1764, 1849, 1936, 2025, 2116, 2209, 2304, 2401, 2500, 2601, 2704, 2809, 2916, 3025, 3136, 3249, 3364, 3481, 3600, 3721, 3844, 3969, 4096, 4225, 4356, 4489, 4624, 4761, 4900, 5041, 5184, 5329, 5476, 5625, 5776, 5929, 6084, 6241, 6400, 6561, 6724, 6889, 7056, 7225, 7396, 7569, 7744, 7921, 8100, 8281, 8464, 8649, 8836, 9025, 9216, 9409, 9604, 9801, 10000)

scala> foo.map(x => x*x*x)
res1: List[Int] = List(1, 8, 27, 64, 125, 216, 343, 512, 729, 1000, 1331, 1728, 2197, 2744, 3375, 4096, 4913, 5832, 6859, 8000, 9261, 10648, 12167, 13824, 15625, 17576, 19683, 21952, 24389, 27000, 29791, 32768, 35937, 39304, 42875, 46656, 50653, 54872, 59319, 64000, 68921, 74088, 79507, 85184, 91125, 97336, 103823, 110592, 117649, 125000, 132651, 140608, 148877, 157464, 166375, 175616, 185193, 195112, 205379, 216000, 226981, 238328, 250047, 262144, 274625, 287496, 300763, 314432, 328509, 343000, 357911, 373248, 389017, 405224, 421875, 438976, 456533, 474552, 493039, 512000, 531441, 551368, 571787, 592704, 614125, 636056, 658503, 681472, 704969, 729000, 753571, 778688, 804357, 830584, 857375, 884736, 912673, 941192, 970299, 1000000)


This seems to work now!

Executing the test-case ...

LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/:. \
./avian-dynamic -cp /home/soc/Entwicklung/scala/build/pack/lib/scala-library.jar:/home/soc/Entwicklung/scala/build/pack/lib/scala-reflect.jar:/home/soc/Entwicklung/scala/build/pack/lib/scala-compiler.jar:/home/soc/Entwicklung/scala/build/pack/lib/jline.jar 'scala.tools.nsc.MainGenericRunner' -usejavacp
Welcome to Scala version 2.11.0-20130127-161738-f389f6a18f (Avian, Java 1.7.0_09).

Type in expressions to have them evaluated.
Type :help for more information.

scala> object Test {
     |   private type Clearable = { def clear(): Unit }
     |   private def choke() = {
     |     try new Array[Object]((Runtime.getRuntime().maxMemory min Int.MaxValue).toInt)
     |     catch {
     |       case _: OutOfMemoryError => // what do you mean, out of memory?
     |       case t: Throwable => println(t)
     |     }
     |   }
     |   private def f(x: Clearable) = x.clear()
     |   class Choker(id: Int) extends Thread {
     |     private def g(iteration: Int) = {
     |       val map = scala.collection.mutable.Map[Int, Int](1 -> 2)
     |       try f(map) catch { case t: NullPointerException => println(s"Failed at $id/$iteration") ; throw t }
     |       choke()
     |     }
     |     override def run() {
     |       1 to 50 foreach g
     |     }
     |   }
     |
     |   def main(args: Array[String]): Unit = {
     |     val threads = 1 to 3 map (id => new Choker(id))
     |     threads foreach (_.start())
     |     threads foreach (_.join())
     |     println("All threads completed.")
     |   }
     | }
warning: there were 1 feature warnings; re-run with -feature for details
defined object Test

scala> Test main null
java/lang/OutOfMemoryError
java/lang/OutOfMemoryError
java/lang/OutOfMemoryError
[1]    17650 abort (core dumped)  LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/:. ./avian-dynamic -cp   -usejavacp
LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/:. ./avian-dynamic -cp   -usejavacp  23,18s user 2,34s system 28% cpu 1:29,30 total


... doesn't seem to return the expected thing:

https://github.com/scala/scala/commit/417304514b664e5f66bfc06fb2b0e86b99d23a63#L1L-1

Not sure if this is the fault of the test-case or of Avian.

But extracting the meat from that ...

scala> type Clearable = { def clear(): Unit }
defined type alias Clearable

scala> def f(x: Clearable) = x.clear()
warning: there were 1 feature warnings; re-run with -feature for details
f: (x: Clearable)Unit

scala> val map = scala.collection.mutable.Map[Int, Int](1 -> 2)
map: scala.collection.mutable.Map[Int,Int] = Map(1 -> 2)

scala> 1 to 10000 foreach (_ => f(map))


... seems to work fine (or at least doesn't trigger the issue).

Looks like an improvement to me. :-)

Thanks a lot and bye,

Simon
Re: Scala success! Joel Dice 1/29/13 2:14 PM
On Tue, 29 Jan 2013, simon.och...@gmail.com wrote:

> scala> Test main null
> java/lang/OutOfMemoryError
> java/lang/OutOfMemoryError
> java/lang/OutOfMemoryError
> [1]��� 17650 abort (core dumped)�
> LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/:. ./avian-dynamic -cp�� -usejavacp
> LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/:. ./avian-dynamic -cp��
> -usejavacp� 23,18s user 2,34s system 28% cpu 1:29,30 total

Yeah, that's not good.  Even if the test is flawed somehow, it shouldn't
cause the process to abort.  I'll try to debug it when I have time.

By the way, I tried to build Scala from the Git master branch the other
day, and I couldn't do an "ant dist" because apparently scala-swing.jar
cannot be built with Java 7.  Is there a way I can build and test Scala
without scala-swing.jar, or do I need to install Java 6?
Re: Scala success! Simon Ochsenreither 1/29/13 2:37 PM
Hi Joel!


Yeah, that's not good.  Even if the test is flawed somehow, it shouldn't
cause the process to abort.  I'll try to debug it when I have time.

Ok, thanks, good to know!
 
By the way, I tried to build Scala from the Git master branch the other
day, and I couldn't do an "ant dist" because apparently scala-swing.jar
cannot be built with Java 7.  Is there a way I can build and test Scala
without scala-swing.jar, or do I need to install Java 6?

Yes, sure!

If you are only interested in building and testing Scala ant build or ant test.suite is perfectly fine. It figures out automatically that it shouldn't build Swing if it runs on JDK 7.
There is also ant test, which is a superset of ant test.suite, but that does only add tests which are pretty uninteresting for you, like tests for binary compatibility.

ant dist is pretty much meant to create a shippable distribution of everything involved, that's why it reacts pretty strongly even when non-critical stuff goes wrong.

Bye,

Simon
Re: Scala success! Simon Ochsenreither 1/30/13 3:07 PM
Hi Joel,

while trying do build scala with Avian, I sym-linked the avian executable to avian/bin/java, set $JAVA_HOME to that dir and tried to run ant (version 1.8.2) this way ...

LD_LIBRARY_PATH=/usr/lib/jvm/java-7-openjdk-amd64/:/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations/ ant

... but I got this almost immediately:

java.lang.NullPointerException
  at org.apache.tools.ant.launch.Launcher.run (line 176)
  at org.apache.tools.ant.launch.Launcher.main (line 109)


The code in question is this one (source from Ant 1.8.3):

    private int run(String[] args)
            throws LaunchException, MalformedURLException {
        String antHomeProperty = System.getProperty(ANTHOME_PROPERTY);
        File antHome = null;

        File sourceJar = Locator.getClassSource(getClass());
        File jarDir = sourceJar.getParentFile();              // <=== Line 176
        String mainClassname = MAIN_CLASS;


Any ideas?

Thanks and bye,

Simon
Re: Scala success! Joel Dice 1/31/13 8:41 AM
Hi Simon,
Thanks for reporting this, but I'm not able to reproduce it.  Here's what
I did:

cd avian
make openjdk=/home/dicej/p/deb/openjdk-7-7~u3-2.1.1/build/openjdk.build/j2sdk-image
cd ../my-ant-project
LD_LIBRARY_PATH=/home/dicej/p/avian/build/linux-x86_64-openjdk:/home/dicej/p/deb/openjdk-7-7~u3-2.1.1/build/openjdk.build/j2sdk-image/jre/lib/amd64/ JAVACMD=/home/dicej/p/avian/build/linux-x86_64-openjdk/avian-dynamic ant

I put an <exec executable="sleep"><arg line="10000"/></exec> in my
build.xml so I could run ps and verify that it was using avian-dynamic
instead of java, which it was.  I'm running Ant 1.8.2, though, so maybe
it's behaving differently.

What options did you use when you built Avian?
Re: Scala success! Simon Ochsenreither 1/31/13 9:55 AM
Hi Joel,


What options did you use when you built Avian?

I'm using tailcalls, continuations and openjdk.

Bye,

Simon
Re: Scala success! Joel Dice 1/31/13 10:02 AM
In that case, this (from your earlier email) looks wrong:

   LD_LIBRARY_PATH=/usr/lib/jvm/java-7-openjdk-amd64/:/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations/ ant

That should be "linux-x86_64-tails-continuations-openjdk", not
"linux-x86_64-tails-continuations".  Not sure if that's the cause of the
problem you're facing, but it's something to try.
Re: Scala success! Joel Dice 1/31/13 3:23 PM
On Thu, 31 Jan 2013, Joel Dice wrote:

> On Thu, 31 Jan 2013, simon.och...@gmail.com wrote:
>
>> I'm using tailcalls, continuations and openjdk.
>
> In that case, this (from your earlier email) looks wrong:
>
>  LD_LIBRARY_PATH=/usr/lib/jvm/java-7-openjdk-amd64/:/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations/
> ant
>
> That should be "linux-x86_64-tails-continuations-openjdk", not
> "linux-x86_64-tails-continuations".  Not sure if that's the cause of the
> problem you're facing, but it's something to try.

FWIW, I just tried again using Ant 1.8.3 with a
"linux-x86_64-tails-continuations-openjdk" and I didn't have any problems.
I wonder if your symlink trick is causing the wrong libjvm.so to be used
or something like that.  Here's what I did:

(cd ~/p/avian-dicej/ && make openjdk=/home/dicej/p/deb/openjdk-7-7~u3-2.1.1/build/openjdk.build/j2sdk-image tails=true continuations=true) && LD_LIBRARY_PATH=/home/dicej/p/avian-dicej/build/linux-x86_64-tails-continuations-openjdk:/home/dicej/p/deb/openjdk-7-7~u3-2.1.1/build/openjdk.build/j2sdk-image/jre/lib/amd64/ JAVACMD="/home/dicej/p/avian-dicej/build/linux-x86_64-tails-continuations-openjdk/avian-dynamic" /home/dicej/trash/p/apache-ant-1.8.3/bin/ant
Re: Scala success! Simon Ochsenreither 1/31/13 4:10 PM
Yes, that was the problem, thanks!

Although the Ant script reports java version: Avian 1.7.0_09 (1.7), the build times are pretty much identical to those when compiling with the OpenJDK runtime. It looks like Ant manages to use the OpenJDK runtime despite setting PATH, JAVA_HOME and JAVACMD to Avian:

PATH=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-openjdk/bin/:$PATH LD_LIBRARY_PATH=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-openjdk/:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/ JAVACMD="/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-openjdk/avian-dynamic" ant build

top also reports java and not avian or avian-dynamic in its listing. :-/

No idea why or even how Ant does this ...
Re: Scala success! Joel Dice 2/1/13 7:36 AM
On Thu, 31 Jan 2013, simon.och...@gmail.com wrote:

> Yes, that was the problem, thanks!
>
> Although the Ant script reports java version: Avian 1.7.0_09 (1.7), the
> build times are pretty much identical to those when compiling with the
> OpenJDK runtime. It looks like Ant manages to use the OpenJDK runtime
> despite setting PATH, JAVA_HOME and JAVACMD to Avian:
>
> PATH=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-ope
> njdk/bin/:$PATHLD_LIBRARY_PATH=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-contin
> uations-openjdk/:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/JAVACMD="/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations
> -openjdk/avian-dynamic" ant build
>
> top also reports java and not avian or avian-dynamic in its listing. :-/

I'm guessing something's different about how you're running it, because I
only see avian-dynamic in top.  Just to be sure, I unset JAVA_HOME and
replaced my java binary with an empty file.  I also ran lsof to verify
that avian-dynamic was
using linux-x86_64-tails-continuations-openjdk/libjvm.so and not the
OpenJDK version.  Finally, I added some printf statements to the VM at
key points.  All of which makes me sure that Ant is only using Avian on my
system.
Re: Scala success! Simon Ochsenreither 2/1/13 12:32 PM
Hi Joel,

I gave up and renamed moved the java executables out of the way, I think this looks pretty good:

https://groups.google.com/d/msg/scala-language/HW4FvHVLo8k/5OV4IU8le-wJ

:-D

(Test cases seem to cause an OOME/core dump, but the whole Scala distribution was boot-strapped successfully!)

Thanks and bye,

Simon
Re: Scala success! to...@telegraphics.com.au 2/1/13 7:35 PM
No such luck here on PowerPC. Any hints?

thanks
--Toby

OS X 10.5.8

g5:darwin-powerpc toby$ java -version
java version "1.5.0_30"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_30-b03-389-9M3425)
Java HotSpot(TM) Client VM (build 1.5.0_30-161, mixed mode, sharing)



g5:darwin-powerpc toby$ gdb ./avian
GNU gdb 6.3.50-20050815 (Apple version gdb-967) (Tue Jul 14 02:15:14 UTC 2009)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin"...Reading symbols for shared libraries ........ done

vacprs/toby/Downloads/scala-2.9.2//lib/scala-library.jar scala.tools.nsc.interpreter.Runner -useja
Starting program: /Volumes/data/Users/toby/Documents/git/avian/build/darwin-powerpc/avian -cp /Volumes/data/Users/toby/Downloads/scala-2.9.2//lib/scala-compiler.jar:/Volumes/data/Users/toby/Downloads/scala-2.9.2//lib/scala-library.jar scala.tools.nsc.interpreter.Runner -usejavacp
Reading symbols for shared libraries +++++++.................... done
java/lang/NoClassDefFoundError: java/util/concurrent/locks/ReentrantLock
  at scala/tools/nsc/interpreter/ILoopInit$class.$init$ (line 63)
  at scala/tools/nsc/interpreter/ILoop.<init> (line 34)
  at scala/tools/nsc/interpreter/ILoop.<init> (line 40)
  at scala/tools/nsc/interpreter/Runner$.main (line 10)
  at scala/tools/nsc/interpreter/Runner.main (unknown line)

Program exited with code 0377.
Re: Scala success! Joel Dice 2/1/13 7:41 PM
Hi Toby,

You'll need to build against the OpenJDK class library as described in the
readme.  The default Avian class library does not yet have all the classes
and methods needed to run Scala.
> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to avian+un...@googlegroups.com.
> To post to this group, send email to av...@googlegroups.com.
> Visit this group at http://groups.google.com/group/avian?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
> �
> �
>
>
Re: Scala success! to...@telegraphics.com.au 2/1/13 7:47 PM


On Friday, 1 February 2013 22:41:39 UTC-5, Joel Dice wrote:
Hi Toby,

You'll need to build against the OpenJDK class library as described in the
readme.  The default Avian class library does not yet have all the classes
and methods needed to run Scala.

Thanks for the superfast response. Sorry for not reading that first!

OpenJDK does not build on OS X PowerPC, as far as I know. Is that a showstopper or does Avian only need platform independent class files?

--Toby


 


On Fri, 1 Feb 2013, to...@telegraphics.com.au wrote:

> No such luck here on PowerPC. Any hints?
>
> thanks
> --Toby
>
> OS X 10.5.8
>
> g5:darwin-powerpc toby$ java -version
> java version "1.5.0_30"
> Java(TM) 2 Runtime Environment, Standard Edition (build
> 1.5.0_30-b03-389-9M3425)
> Java HotSpot(TM) Client VM (build 1.5.0_30-161, mixed mode, sharing)
>
>
>
> g5:darwin-powerpc toby$ gdb ./avian
> GNU gdb 6.3.50-20050815 (Apple version gdb-967) (Tue Jul 14 02:15:14 UTC
> 2009)
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.� Type "show warranty" for details.
> This GDB was configured as "powerpc-apple-darwin"...Reading symbols for
> shared libraries ........ done
>
> vacprs/toby/Downloads/scala-2.9.2//lib/scala-library.jar
> scala.tools.nsc.interpreter.Runner -useja
> Starting program:
> /Volumes/data/Users/toby/Documents/git/avian/build/darwin-powerpc/avian -cp/Volumes/data/Users/toby/Downloads/scala-2.9.2//lib/scala-compiler.jar:/Vol
> umes/data/Users/toby/Downloads/scala-2.9.2//lib/scala-library.jar
> scala.tools.nsc.interpreter.Runner -usejavacp
> Reading symbols for shared libraries +++++++.................... done
> java/lang/NoClassDefFoundError: java/util/concurrent/locks/ReentrantLock
> � at scala/tools/nsc/interpreter/ILoopInit$class.$init$ (line 63)
> � at scala/tools/nsc/interpreter/ILoop.<init> (line 34)
> � at scala/tools/nsc/interpreter/ILoop.<init> (line 40)
> � at scala/tools/nsc/interpreter/Runner$.main (line 10)
> � at scala/tools/nsc/interpreter/Runner.main (unknown line)
>
> Program exited with code 0377.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to avian+un...@googlegroups.com.
> To post to this group, send email to av...@googlegroups.com.
> Visit this group at http://groups.google.com/group/avian?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
> �
> �
>
>
Re: Scala success! Joel Dice 2/1/13 7:59 PM
On Fri, 1 Feb 2013, to...@telegraphics.com.au wrote:

>
>
> On Friday, 1 February 2013 22:41:39 UTC-5, Joel Dice wrote:
>       Hi Toby,
>
>       You'll need to build against the OpenJDK class library as
>       described in the
>       readme. �The default Avian class library does not yet have all
>       the classes
>       and methods needed to run Scala.
>
>
> Thanks for the superfast response. Sorry for not reading that first!
>
> OpenJDK does not build on OS X PowerPC, as far as I know. Is that a
> showstopper or does Avian only need platform independent class files?

It needs both the class files and the corresponding JNI code.  Your best
bet is to do an openjdk-src build as described in the readme, which will
attempt to build the minimum set of JNI code from source.  It's going to
have to match the class files pretty much exactly, though, so you'll need
to point Avian at the classes (e.g. rt.jar) from the same OpenJDK version
(built on another platform, perhaps).  I think it's all doable, but it may
take some effort.

There's also been some discussion about supplementing the Avian library
with the Android one, but nothing's been done yet, AFAIK:

https://groups.google.com/group/avian/msg/9fb4b179cc194e46
Re: Scala success! Joel Dice 2/3/13 1:39 PM
Hi Simon,

On Tue, 29 Jan 2013, simon.och...@gmail.com wrote:

> LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/:. \
> ./avian-dynamic -cp/home/soc/Entwicklung/scala/build/pack/lib/scala-library.jar:/home/soc/Entw
> icklung/scala/build/pack/lib/scala-reflect.jar:/home/soc/Entwicklung/scala/
> build/pack/lib/scala-compiler.jar:/home/soc/Entwicklung/scala/build/pack/li
I made some progress on this.  I never hit the abort, but I did run into
an issue with non-ASCII characters in a field name which led to an
assertion failure in the debug build.  It's possible that the same bug
caused a crash in your case with a non-debug build.  Anyway, that part's
fixed:

https://github.com/ReadyTalk/avian/commit/1890e348fb7d596c75051329f5f0059c20419a8d

Now what I'm seeing is that all the threads in your test case are dying
with uncaught OutOfMemoryErrors.  Obviously, OOMEs are not totally
unexpected given the nature of the test, but in this case they're being
thrown outside the try/catch block:

Breakpoint 1, vm::allocate3 (t=t@entry=0x7fffe01e5560, allocator=0x602460,
type=vm::Machine::MovableAllocation, sizeInBytes=40,
objectMask=objectMask@entry=false) at src/machine.cpp:3531
3531              throw_(t, root(t, Machine::OutOfMemoryError));
(gdb) call vmPrintTrace(t)
debug trace for thread 0x7fffe01e5560
   at java/lang/Class.getDeclaredMethods0 (native)
   at java/lang/Class.privateGetDeclaredMethods (line 2442)
   at java/lang/Class.getMethod0 (line 2685)
   at java/lang/Class.getMethod (line 1620)
   at $line3/$read$$iw$$iw$Test$.reflMethod$Method1 (line 17)
   at $line3/$read$$iw$$iw$Test$.$line3$$read$$iw$$iw$Test$$f (line 17)
   at $line3/$read$$iw$$iw$Test$Choker.$line3$$read$$iw$$iw$Test$Choker$$g
(line 21)
   at $line3/$read$$iw$$iw$Test$Choker$$anonfun$run$1.apply (line 26)
   at $line3/$read$$iw$$iw$Test$Choker$$anonfun$run$1.apply (line 26)
   at scala/collection/immutable/Range.foreach (line 148)
   at $line3/$read$$iw$$iw$Test$Choker.run (line 26)

On Avian, at least, Class.getDeclaredMethods0 requires some amount of
memory allocation.  So when there are other tests putting deliberate
pressure on the heap, it may throw an OOME.  In this case, though, the
giant array allocation in the choke method should just fail immediately
without any side effects on concurrent or subsequent execution (other than
some GC activity).  Somehow, the VM is getting fooled into actually
allocating that "impossible" array, and that's what causes OOMEs to be
thrown elsewhere.  I'll try to find a way to address that.

The other problem is that the test never finishes because Thread.join is
not returning when the threads die.  That's definitely a bug.
Re: Scala success! Joel Dice 2/3/13 3:31 PM
On Sun, 3 Feb 2013, Joel Dice wrote:

> On Avian, at least, Class.getDeclaredMethods0 requires some amount of memory
> allocation.  So when there are other tests putting deliberate pressure on the
> heap, it may throw an OOME.  In this case, though, the giant array allocation
> in the choke method should just fail immediately without any side effects on
> concurrent or subsequent execution (other than some GC activity).  Somehow,
> the VM is getting fooled into actually allocating that "impossible" array,
> and that's what causes OOMEs to be thrown elsewhere.  I'll try to find a way
> to address that.
>
> The other problem is that the test never finishes because Thread.join is not
> returning when the threads die.  That's definitely a bug.

OK, that's all fixed and the test passes.  I'm running a full Scala "ant
test" on Avian now.  It takes forever using HotSpot on my machine, so I
expect it to take forever * 4 using Avian.
Re: Scala success! Joel Dice 2/3/13 6:45 PM
On Sun, 3 Feb 2013, Joel Dice wrote:

> OK, that's all fixed and the test passes.  I'm running a full Scala "ant
> test" on Avian now.  It takes forever using HotSpot on my machine, so I
> expect it to take forever * 4 using Avian.

Well, not forever, since it crashed.  Yay, core dumps.
Re: Scala success! Simon Ochsenreither 2/4/13 11:19 AM
I guess you're ahead of me. :-)

When I try to run the ant test, I get the following output:

ant test
Buildfile: /home/soc/Entwicklung/scala/build.xml

init:
     [echo]        build time: 4 February 2013, 19:54:17
     [echo]      java version: Avian 1.7.0_09 (1.7)
     [echo]         java args: -Xms1536M -Xmx1536M -Xss1M -XX:MaxPermSize=192M -XX:+UseParallelGC
     [echo]        javac args:
     [echo]       scalac args: -Yreify-copypaste 
     [echo] scalac quick args: -Yreify-copypaste  
     [echo]          git date: 20130129-171234
     [echo]          git hash: d499db3800
     [echo]     maven version: 2.11.0-SNAPSHOT
     [echo]      OSGi version: 2.11.0.v20130129-171234-d499db3800
     [echo] canonical version: 2.11.0-20130129-171234-d499db3800

  <... snip ...>

test.suite:
  [partest] Compiling files that are expected to build
  [partest] testing: [...]/files/pos/t1937                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1642                                        [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t1836-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t1836
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t1409-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t1409
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t3521-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t3521
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t6485a-pos.log':
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t1176-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t6485a
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t1176
  [partest] java.lang.OutOfMemoryError
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/t1186                                        [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/java-access-pos-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/java-access-pos
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t3429-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t3429
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t5644-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t5644
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t1231-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t1231
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t5639-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t5639
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t1230-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t1230
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t1107b-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t1107b
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t5031-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t5031
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/scala/test/files/pos/t3249-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/scala/test/files/pos/t3249
  [partest] java.lang.OutOfMemoryError
[1]    5385 abort (core dumped)  ant test
ant test  95,89s user 5,34s system 108% cpu 1:33,12 total


I think I see a pattern here:
The tests with a single file work (t1937.scala, t1642.scala), but those where multiple files are in a directory fail (one exception: t1186/ is a directory, but has only a single java file and works).
So either the cause is related to the directories or to multiple files.

Running test folders individually results in a StackOverflowError:
test/partest test/files/pos/*
java/lang/reflect/InvocationTargetException
  at sun/reflect/NativeMethodAccessorImpl.invoke0 (native)
  at scala/tools/partest/nest/ReflectiveRunner.main (line 98)
caused by: java/lang/StackOverflowError
  at scala/collection/generic/TraversableForwarder$class.isEmpty (line 33)
  at scala/collection/mutable/ListBuffer.$plus$eq (line 167)
  at scala/collection/IndexedSeqOptimized$class.foreach (line 33)
  at scala/collection/generic/Growable$class.$plus$plus$eq (line 58)
  at scala/collection/mutable/ListBuffer.$plus$plus$eq (line 181)
  at scala/collection/TraversableLike$class.to (line 632)
  at scala/collection/TraversableOnce$class.toList (line 255)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  <a few hundred more of these>
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.loop$1 (line 97)
  at scala/tools/nsc/util/CommandLine.x$5$lzycompute (line 101)
  at scala/tools/nsc/util/CommandLine.x$5 (line 52)
  at scala/tools/nsc/util/CommandLine.residualArgs$lzycompute (line 52)
  at scala/tools/partest/nest/ConsoleRunner.main (line 77)
  at sun/reflect/NativeMethodAccessorImpl.invoke0 (native)
  at scala/tools/partest/nest/ReflectiveRunner.main (line 98)


Not sure if this isn't a Scala issue although it works with the OpenJDK runtime.

Any ideas?

Simon
Re: Scala success! Joel Dice 2/4/13 12:31 PM
On Mon, 4 Feb 2013, simon.och...@gmail.com wrote:

> ᅵ [partest] Compiling files that are expected to build
> ᅵ [partest] testing:
> [...]/files/pos/t1937ᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵ [ᅵ OKᅵ ]
> ᅵ [partest] testing:
> [...]/files/pos/t1642ᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵᅵ [ᅵ OKᅵ ]
> ᅵ [partest] Log file
> '/home/soc/Entwicklung/scala/test/files/pos/t1836-pos.log':
> ᅵ [partest] Possible compiler crash during test of:
> /home/soc/Entwicklung/scala/test/files/pos/t1836
> ᅵ [partest] java.lang.OutOfMemoryError
> ᅵ [partest] Log file
> '/home/soc/Entwicklung/scala/test/files/pos/t1409-pos.log':
> ᅵ [partest] Possible compiler crash during test of:
> /home/soc/Entwicklung/scala/test/files/pos/t1409
> ᅵ [partest] java.lang.OutOfMemoryError

That's really weird.  Are you using the latest code from the Git repo?
The tests run slowly for me, and I've seen the process abort a few times
(which I'm debugging as I have the time), but I haven't seen any OOMEs or
StackOverflowErrors.

The only thing I can think of is that the -Xmx1536M -Xss1M arguments
aren't being used somehow.  You could check that by adding some logging to
jnienv.cpp to print the heap and stack sizes used.  I don't understand why
it would be different on my machine, though.
Re: Scala success! Joel Dice 2/4/13 12:35 PM
Also, how exactly are you running Ant?  Are you using the shell script and
setting JAVACMD, or are you running avian-dynamic directly?  If the
latter, you'll need to override the default heap and stack sizes.
Re: Scala success! Simon Ochsenreither 2/4/13 1:19 PM
Hi Joel,

Also, how exactly are you running Ant?  Are you using the shell script and
setting JAVACMD, or are you running avian-dynamic directly?  If the
latter, you'll need to override the default heap and stack sizes.

Yes, I'm setting  JAVACMD (as well as PATH, LD_LIBRARY_PATH and JAVA_HOME and renaming OpenJDK's java binary).

First, I run this script, which makes the look like a valid JDK dir:

AVIAN_BUILD_DIR="/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-openjdk/"
JDK_DIR="/usr/lib/jvm/java-7-openjdk-amd64/"

if [ -d "$AVIAN_BUILD_DIR" ]; then

  if [ -d "$AVIAN_BUILD_DIR/bin" ]; then
    rm -r "$AVIAN_BUILD_DIR/bin"
  fi

  cd "$AVIAN_BUILD_DIR"
  mkdir bin
  cd bin
  ln -s ../avian-dynamic java
  ln -s $JDK_DIR/bin/javac javac

else
  echo "Dir does not exist!"
fi

Then I export the following settings:

export LD_LIBRARY_PATH=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-openjdk/:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/
export JAVA_HOME=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-openjdk
export JDK_HOME=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-openjdk
export PATH=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-openjdk/bin/:$PATH
export JAVACMD=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-openjdk/avian-dynamic

After that, I run ant test.suite and get the errors mentioned above.
Re: Scala success! Joel Dice 2/6/13 7:35 AM
> s-openjdk
> exportJDK_HOME=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations
> -openjdk
> exportPATH=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-ope
> njdk/bin/:$PATH
> exportJAVACMD=/home/soc/Entwicklung/avian/build/linux-x86_64-tails-continuations-
> openjdk/avian-dynamic
>
> After that, I run ant test.suite and get the errors mentioned above.

I'm going to try to reproduce this on a different machine.  Meanwhile, I
recommend adding some logging to MyHeap::limitExceeded in heap.cpp.  If
c.count > c.limit, print both values to find out how large the heap is
really being allowed to grow.  If it's not the same as what -Xmx
specifies, something's gone wrong setting the option.  Otherwise, we'll
have to look elsewhere.
Re: Scala success! Joel Dice 2/8/13 3:37 PM
On Wed, 6 Feb 2013, Joel Dice wrote:

> I'm going to try to reproduce this on a different machine.  Meanwhile, I
> recommend adding some logging to MyHeap::limitExceeded in heap.cpp.  If
> c.count > c.limit, print both values to find out how large the heap is really
> being allowed to grow.  If it's not the same as what -Xmx specifies,
> something's gone wrong setting the option.  Otherwise, we'll have to look
> elsewhere.

Okay, starting from scratch on a different machine, here's what I did:

cd /build/dicej/
git clone git@github.com:Readytalk/avian
git clone git@github.com:scala/scala
hg clone http://hg.openjdk.java.net/jdk7u/jdk7u-dev
cd jdk7u-dev/
bash get_source.sh
unset JAVA_HOME
export LANG=C
make ALLOW_DOWNLOADS=true ALT_BOOTDIR=/usr/lib/jvm/java-6-openjdk/
cd ..
# (download Ant 1.8.3)
tar xjf /tmp/apache-ant-1.8.3-bin.tar.bz2
cd scala
cp -r /build/dicej/jdk7u-dev/build/linux-amd64/j2sdk-image /build/dicej/jdk-avian
(cd /build/dicej/avian && make strip=: openjdk=/build/dicej/jdk-avian mode=debug-fast && cp build/linux-x86_64-debug-fast-openjdk/libjvm.so ../jdk-avian/jre/lib/amd64/server/libjvm.so) && LD_LIBRARY_PATH=/build/dicej/jdk-avian/jre/lib/amd64 ionice -c 3 nice /build/dicej/jdk-avian/jre/bin/java -Xmx1536m -Xss1m -classpath /build/dicej/apache-ant-1.8.3/lib/ant-launcher.jar -Dant.home=/build/dicej/apache-ant-1.8.3 -Dant.library.dir=/build/dicej/apache-ant-1.8.3/lib org.apache.tools.ant.launch.Launcher test

It's true that Ant insists on running the "java" command when running a
fork=true task.  I don't know how I convinced it to run avian-dynamic on
my other machine; maybe I was fooling myself when I thought it was.
Anyway, the above approach seems to work.  Note the mode=debug-fast part,
which means it will run even more slowly than normal, but at least we'll
get an understandable core dump if it crashes.

The last command above has been running for over 48 hours on the machine
in question.  It bootstrapped Scala fine and is gradually churning through
the test suite; no crashes, OutOfMemoryErrors, or StackOverflowErrors so
far.

If you're still having trouble, would you please try something like the
above sequence of commands?

Re: Scala success! Simon Ochsenreither 2/8/13 5:11 PM
Hi Joel,


The last command above has been running for over 48 hours on the machine
in question.  It bootstrapped Scala fine and is gradually churning through
the test suite; no crashes, OutOfMemoryErrors, or StackOverflowErrors so
far.

wow, great to hear!
 
If you're still having trouble, would you please try something like the
above sequence of commands?

Yes, I'll try that as soon as I'm in front of my workstation again.

Thanks a lot!

Bye,

Simon
Re: Scala success! Pablo Guerrero 2/11/13 12:04 AM
Hi Joel,

That sounds pretty good. Did it finish without errors? Or it's still running, xD.

Cheers,
Pablo



--
You received this message because you are subscribed to the Google Groups "Avian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to avian+un...@googlegroups.com.
To post to this group, send email to av...@googlegroups.com.
Visit this group at http://groups.google.com/group/avian?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.



Re: Scala success! Simon Ochsenreither 2/12/13 8:56 AM
Hi,

I tried that and I get the following errors:

% LD_LIBRARY_PATH=../jdk-avian/jre/lib/amd64 ionice -c 3 nice ../jdk-avian/jre/bin/java -Xmx2048m -Xss1m -classpath ../../lib/apache-ant-1.8.4/lib/ant-launcher.jar -Dant.home=../../lib/apache-ant-1.8.4 -Dant.library.dir=../../lib/apache-ant-1.8.3/lib org.apache.tools.ant.launch.Launcher test
Buildfile: /home/soc/Entwicklung/avian-build/scala/build.xml

init.jars.check:

init.jars:
     [echo] Updating bootstrap libs.  (To do this by hand, run ./pull-binary-libs.sh)
     [exec] Resolving [f93a2525b5616d3a4bee7848fabbb2856b56f653/lib/forkjoin.jar]
     [exec] Resolving [d7c6e69eba3dba2f75a0f44e56480cd3dbab8931/lib/scala-library.jar]
     [exec] Resolving [a5261e70728c1847639e2b47d953441d0b217bcb/lib/jline.jar]
     [exec] Resolving [2cba5a13ef44bf93133be26cc89ba3a640a5c28f/lib/scala-library-src.jar]
     [exec] Resolving [c0eed5dee0a3204239c9b35134cad8b3ad140121/lib/scala-reflect.jar]
     [exec] Resolving [7e50e3e227d834695f1e0bf018a7326e06ee4c86/lib/ant/maven-ant-tasks-2.1.1.jar]
     [exec] Resolving [943cd5c8802b2a3a64a010efb86ec19bac142e40/lib/ant/ant-contrib.jar]
     [exec] Resolving [3fc1e35ca8c991fc3488548f7a276bd9053c179d/lib/ant/ant-dotnet-1.0.jar]
     [exec] Resolving [7b456ca6b93900f96e58cc8371f03d90a9c1c8d1/lib/ant/ant.jar]
     [exec] Resolving [2c61d6e9a912b3253194d5d6d3e1db7e2545ac4b/lib/ant/vizant.jar]
     [exec] Resolving [f9f41fb909df6a0178906c9fd02e5d0efa15c9ed/lib/scala-compiler-src.jar]
     [exec] Resolving [dbc00bd4b09012aa52e802926dee3f8a02a767ff/lib/scala-reflect-src.jar]
     [exec] Resolving [c4cd524dc29d298a5034637f6b31122dccb300d6/lib/scala-compiler.jar]
     [exec] Resolving [cd33e0a0ea249eb42363a2f8ba531186345ff68c/test/files/lib/nest.jar]
     [exec] Resolving [346d3dff4088839d6b4d163efa2892124039d216/test/files/lib/jsoup-1.3.1.jar]
     [exec] Resolving [02fe2ed93766323a13f22c7a7e2ecdcd84259b6c/test/files/lib/annotations.jar]
     [exec] Resolving [be8454d5e7751b063ade201c225dcedefd252775/test/files/lib/methvsfield.jar]
     [exec] Resolving [b1ec8a095cec4902b3609d74d274c04365c59c04/test/files/lib/genericNest.jar]
     [exec] Resolving [b6f4dbb29f0c2ec1eba682414f60d52fea84f703/test/files/lib/scalacheck.jar]
     [exec] Resolving [981392dbd1f727b152cd1c908c5fce60ad9d07f7/test/files/lib/enums.jar]
     [exec] Resolving [e737b123d31eede5594ceda07caafed1673ec472/test/files/codelib/code.jar]
     [exec] Resolving [1b11ac773055c1e942c6b5eb4aabdf02292a7194/test/files/speclib/instrumented.jar]
     [exec] Resolving [a1883f4304d5aa65e1f6ee6aad5900c62dd81079/tools/push.jar]

init.maven.tasks:

init.maven.jars:
[artifact:dependencies] There were no dependencies specified

init.hasbuildnum:

init.build.snapshot:

init.build.release:

init.build.nopatch.release:

init.build.patch.release:

init.hasmavensuffix:

init.osgi.suffix:

init.osgi.suffix.final:

init.osgi.suffix.snapshot:

init.build.suffix.done:

init.version.git:

init.version.release:

init.version.snapshot:

init.version.done:

init.testjava6:

init.fail.bad.jdk:

init.warn.jdk7:
     [echo]  You are using JDK7 for this build.  While this will be able to build most of Scala, it will
     [echo]        not build the Swing project.   You will be unable to create a distribution.
     [echo]     

init.extra.tasks:
[artifact:dependencies] [WARNING] POM for 'biz.aQute:bndlib:pom:1.43.0:compile' is invalid.
[artifact:dependencies]
[artifact:dependencies] Its dependencies (if any) will NOT be available to the current build.

init:
    [mkdir] Created dir: /home/soc/Entwicklung/avian-build/scala/lib/extra
     [echo]        build time: 12 February 2013, 17:46:51
     [echo]      java version: Avian 1.7.0-internal (1.7)
     [echo]         java args: -Xms1536M -Xmx1536M -Xss1M -XX:MaxPermSize=192M -XX:+UseParallelGC
     [echo]        javac args:
     [echo]       scalac args: -Yreify-copypaste 
     [echo] scalac quick args: -Yreify-copypaste  
     [echo]          git date: 20130211-173604
     [echo]          git hash: 0c59fc9a14
     [echo]     maven version: 2.11.0-SNAPSHOT
     [echo]      OSGi version: 2.11.0.v20130211-173604-0c59fc9a14
     [echo] canonical version: 2.11.0-20130211-173604-0c59fc9a14

asm.start:

asm.lib:
    [mkdir] Created dir: /home/soc/Entwicklung/avian-build/scala/build/asm/classes
    [javac] Compiling 77 source files to /home/soc/Entwicklung/avian-build/scala/build/asm/classes
    [javac] warning: [options] bootstrap class path not set in conjunction with -source 1.5
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 1 warning
[stopwatch] [asm.lib.timer: 7.253 sec]

asm.done:

forkjoin.start:

forkjoin.lib:
    [mkdir] Created dir: /home/soc/Entwicklung/avian-build/scala/build/libs/classes/forkjoin
    [javac] Compiling 10 source files to /home/soc/Entwicklung/avian-build/scala/build/libs/classes/forkjoin

BUILD FAILED
/home/soc/Entwicklung/avian-build/scala/build.xml:567: null


I than compiled forkjoin manually, but that didn't get me much further:

% LD_LIBRARY_PATH=../jdk-avian/jre/lib/amd64 ionice -c 3 nice ../jdk-avian/jre/bin/java -Xmx2048m -Xss1m -classpath ../../lib/apache-ant-1.8.4/lib/ant-launcher.jar -Dant.home=../../lib/apache-ant-1.8.4 -Dant.library.dir=../../lib/apache-ant-1.8.3/lib org.apache.tools.ant.launch.Launcher test
Buildfile: /home/soc/Entwicklung/avian-build/scala/build.xml

init.jars.check:

init.jars:

init.maven.tasks:

init.maven.jars:
[artifact:dependencies] There were no dependencies specified

init.hasbuildnum:

init.build.snapshot:

init.build.release:

init.build.nopatch.release:

init.build.patch.release:

init.hasmavensuffix:

init.osgi.suffix:

init.osgi.suffix.final:

init.osgi.suffix.snapshot:

init.build.suffix.done:

init.version.git:

init.version.release:

init.version.snapshot:

init.version.done:

init.testjava6:

init.fail.bad.jdk:

init.warn.jdk7:
     [echo]  You are using JDK7 for this build.  While this will be able to build most of Scala, it will
     [echo]        not build the Swing project.   You will be unable to create a distribution.
     [echo]     

init.extra.tasks:
[artifact:dependencies] [WARNING] POM for 'biz.aQute:bndlib:pom:1.43.0:compile' is invalid.
[artifact:dependencies]
[artifact:dependencies] Its dependencies (if any) will NOT be available to the current build.

init:
     [echo]        build time: 12 February 2013, 17:51:09
     [echo]      java version: Avian 1.7.0-internal (1.7)
     [echo]         java args: -Xms1536M -Xmx1536M -Xss1M -XX:MaxPermSize=192M -XX:+UseParallelGC
     [echo]        javac args:
     [echo]       scalac args: -Yreify-copypaste 
     [echo] scalac quick args: -Yreify-copypaste  
     [echo]          git date: 20130211-173604
     [echo]          git hash: 0c59fc9a14
     [echo]     maven version: 2.11.0-SNAPSHOT
     [echo]      OSGi version: 2.11.0.v20130211-173604-0c59fc9a14
     [echo] canonical version: 2.11.0-20130211-173604-0c59fc9a14

asm.start:

asm.lib:

asm.done:

forkjoin.start:

forkjoin.lib:

forkjoin.pack:
      [jar] Building jar: /home/soc/Entwicklung/avian-build/scala/build/libs/forkjoin.jar

forkjoin.done:

locker.start:

locker.pre-lib:

locker.lib:
    [mkdir] Created dir: /home/soc/Entwicklung/avian-build/scala/build/locker/classes/library
    [javac] Compiling 32 source files to /home/soc/Entwicklung/avian-build/scala/build/locker/classes/library
    [javac] warning: [options] bootstrap class path not set in conjunction with -source 1.5
    [javac] Note: /home/soc/Entwicklung/avian-build/scala/src/library/scala/collection/concurrent/MainNode.java uses unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 1 warning
[scalacfork] Compiling 702 files to /home/soc/Entwicklung/avian-build/scala/build/locker/classes/library

BUILD FAILED
/home/soc/Entwicklung/avian-build/scala/build.xml:628: null


I tried various alternatives, but it seems like I'm stuck here. Did you experience these issues, too?

Thanks any bye,

Simon
Re: Scala success! Joel Dice 2/12/13 9:56 AM
On Mon, 11 Feb 2013, Pablo Guerrero wrote:

> Hi Joel,
> That sounds pretty good. Did it finish without errors? Or it's still
> running, xD.

It eventually crashed, but I've fixed the problem and am now rerunning it.

https://github.com/ReadyTalk/avian/commit/dc943c250a249f3e75614f95fca769d9ea9ba934
Re: Scala success! Joel Dice 2/12/13 9:58 AM
On Tue, 12 Feb 2013, simon.och...@gmail.com wrote:

> BUILD FAILED
> /home/soc/Entwicklung/avian-build/scala/build.xml:567: null

I do remember hitting something like that, but I don't remember exactly
what the problem was.  Would you please uncomment the call to printTrace
in the throw_ function in machine.h, rebuild Avian, and try again?  That
will allow us to see the exception that Ant is unwilling to show us.

By the way, I recommend doing a git pull, since I fixed a bug this morning
that eventually caused the VM to crash on one of the tests:

https://github.com/ReadyTalk/avian/commit/dc943c250a249f3e75614f95fca769d9ea9ba934
Re: Scala success! Simon Ochsenreither 2/12/13 10:12 AM
Hi Joel,


I do remember hitting something like that, but I don't remember exactly
what the problem was.  Would you please uncomment the call to printTrace
in the throw_ function in machine.h, rebuild Avian, and try again?  That
will allow us to see the exception that Ant is unwilling to show us.
 
I ran ant with -verbose and got this stack trace:

/home/soc/Entwicklung/avian-build/scala/build.xml:567: null
    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:116)
    at org.apache.tools.ant.Task.perform(Task.java:348)
    at org.apache.tools.ant.Target.execute(Target.java:392)
    at org.apache.tools.ant.Target.performTasks(Target.java:413)
    at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
    at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
    at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
    at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
    at org.apache.tools.ant.Main.runBuild(Main.java:811)
    at org.apache.tools.ant.Main.startAnt(Main.java:217)
    at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
    at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)


Could it be possible that it doesn't find javac anymore, because we unset Java_HOME earlier?

I'll compile it with printTrace, maybe that will be helpful too.

By the way, I recommend doing a git pull, since I fixed a bug this morning
that eventually caused the VM to crash on one of the tests:

https://github.com/ReadyTalk/avian/commit/dc943c250a249f3e75614f95fca769d9ea9ba934

Yes, just saw that and updated it. :-)

Thanks and bye,

Simon
Re: Scala success! Joel Dice 2/12/13 10:25 AM
On Tue, 12 Feb 2013, simon.och...@gmail.com wrote:

> Could it be possible that it doesn't find javac anymore, because we unset
> Java_HOME earlier?

Yes, that's it.  I forgot that I had re-exported JAVA_HOME after I built
the JDK.  Sorry for the omission; that's what happens when I try to edit
all the mistakes and dead-ends out of my shell history: I end up omitting
something crucial.

Try setting JAVA_HOME to the jdk-avian directory mentioned in my previous
instructions.
Re: Scala success! Simon Ochsenreither 2/12/13 10:44 AM

Try setting JAVA_HOME to the jdk-avian directory mentioned in my previous
instructions.

No, doesn't help.
Re: Scala success! Joel Dice 2/12/13 11:50 AM
On Tue, 12 Feb 2013, simon.och...@gmail.com wrote:

Okay, then please let me know what you see with that printTrace call
enabled.
Re: Scala success! Simon Ochsenreither 2/12/13 12:23 PM
Hi,


Okay, then please let me know what you see with that printTrace call
enabled.

I redirected it into a text file, resulting file size: 192MB

The last 200 lines:

tail -200 throw_log
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
org/apache/tools/ant/BuildException
  at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
  at org/apache/tools/ant/Task.perform (line 348)
  at org/apache/tools/ant/Target.execute (line 392)
  at org/apache/tools/ant/Target.performTasks (line 413)
  at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
  at org/apache/tools/ant/Project.executeTarget (line 1368)
  at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
  at org/apache/tools/ant/Project.executeTargets (line 1251)
  at org/apache/tools/ant/Main.runBuild (line 811)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
org/apache/tools/ant/BuildException
  at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
  at org/apache/tools/ant/Task.perform (line 348)
  at org/apache/tools/ant/Target.execute (line 392)
  at org/apache/tools/ant/Target.performTasks (line 413)
  at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
  at org/apache/tools/ant/Project.executeTarget (line 1368)
  at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
  at org/apache/tools/ant/Project.executeTargets (line 1251)
  at org/apache/tools/ant/Main.runBuild (line 811)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
org/apache/tools/ant/BuildException
  at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
  at org/apache/tools/ant/Task.perform (line 348)
  at org/apache/tools/ant/Target.execute (line 392)
  at org/apache/tools/ant/Target.performTasks (line 413)
  at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
  at org/apache/tools/ant/Project.executeTarget (line 1368)
  at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
  at org/apache/tools/ant/Project.executeTargets (line 1251)
  at org/apache/tools/ant/Main.runBuild (line 811)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
org/apache/tools/ant/BuildException
  at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
  at org/apache/tools/ant/Task.perform (line 348)
  at org/apache/tools/ant/Target.execute (line 392)
  at org/apache/tools/ant/Target.performTasks (line 413)
  at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
  at org/apache/tools/ant/Project.executeTarget (line 1368)
  at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
  at org/apache/tools/ant/Project.executeTargets (line 1251)
  at org/apache/tools/ant/Main.runBuild (line 811)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
org/apache/tools/ant/BuildException
  at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
  at org/apache/tools/ant/Task.perform (line 348)
  at org/apache/tools/ant/Target.execute (line 392)
  at org/apache/tools/ant/Target.performTasks (line 413)
  at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
  at org/apache/tools/ant/Project.executeTarget (line 1368)
  at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
  at org/apache/tools/ant/Project.executeTargets (line 1251)
  at org/apache/tools/ant/Main.runBuild (line 811)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
org/apache/tools/ant/BuildException
  at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
  at org/apache/tools/ant/Task.perform (line 348)
  at org/apache/tools/ant/Target.execute (line 392)
  at org/apache/tools/ant/Target.performTasks (line 413)
  at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
  at org/apache/tools/ant/Project.executeTarget (line 1368)
  at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
  at org/apache/tools/ant/Project.executeTargets (line 1251)
  at org/apache/tools/ant/Main.runBuild (line 811)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
org/apache/tools/ant/BuildException
  at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
  at org/apache/tools/ant/Task.perform (line 348)
  at org/apache/tools/ant/Target.execute (line 392)
  at org/apache/tools/ant/Target.performTasks (line 413)
  at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
  at org/apache/tools/ant/Project.executeTarget (line 1368)
  at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
  at org/apache/tools/ant/Project.executeTargets (line 1251)
  at org/apache/tools/ant/Main.runBuild (line 811)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
org/apache/tools/ant/BuildException
  at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
  at org/apache/tools/ant/Task.perform (line 348)
  at org/apache/tools/ant/Target.execute (line 392)
  at org/apache/tools/ant/Target.performTasks (line 413)
  at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
  at org/apache/tools/ant/Project.executeTarget (line 1368)
  at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
  at org/apache/tools/ant/Project.executeTargets (line 1251)
  at org/apache/tools/ant/Main.runBuild (line 811)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
java/lang/ClassNotFoundException: org/apache/tools/ant/util/DateUtils
  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at org/apache/tools/ant/DefaultLogger.buildFinished (line 177)
  at org/apache/tools/ant/Project.fireBuildFinished (line 2093)
  at org/apache/tools/ant/Main.runBuild (line 832)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
java/lang/ClassNotFoundException: org/apache/tools/ant/util/DateUtils
  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at org/apache/tools/ant/DefaultLogger.buildFinished (line 177)
  at org/apache/tools/ant/Project.fireBuildFinished (line 2093)
  at org/apache/tools/ant/Main.runBuild (line 832)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
java/lang/ClassNotFoundException: org/apache/tools/ant/util/DateUtils
  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at org/apache/tools/ant/DefaultLogger.buildFinished (line 177)
  at org/apache/tools/ant/Project.fireBuildFinished (line 2093)
  at org/apache/tools/ant/Main.runBuild (line 832)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
java/lang/ClassNotFoundException: org/apache/tools/ant/util/DateUtils
  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at org/apache/tools/ant/DefaultLogger.buildFinished (line 177)
  at org/apache/tools/ant/Project.fireBuildFinished (line 2093)
  at org/apache/tools/ant/Main.runBuild (line 832)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
java/lang/ClassNotFoundException: org/apache/tools/ant/util/DateUtils
  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at org/apache/tools/ant/DefaultLogger.buildFinished (line 177)
  at org/apache/tools/ant/Project.fireBuildFinished (line 2093)
  at org/apache/tools/ant/Main.runBuild (line 832)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)
java/lang/ClassNotFoundException: org/apache/tools/ant/util/DateUtils
  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at org/apache/tools/ant/DefaultLogger.buildFinished (line 177)
  at org/apache/tools/ant/Project.fireBuildFinished (line 2093)
  at org/apache/tools/ant/Main.runBuild (line 832)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)


BUILD FAILED
/home/soc/Entwicklung/avian-build/scala/build.xml:567: null

Total time: 35 seconds
org/apache/tools/ant/BuildException
  at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
  at org/apache/tools/ant/Task.perform (line 348)
  at org/apache/tools/ant/Target.execute (line 392)
  at org/apache/tools/ant/Target.performTasks (line 413)
  at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
  at org/apache/tools/ant/Project.executeTarget (line 1368)
  at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
  at org/apache/tools/ant/Project.executeTargets (line 1251)
  at org/apache/tools/ant/Main.runBuild (line 811)
  at org/apache/tools/ant/Main.startAnt (line 217)
  at org/apache/tools/ant/launch/Launcher.run (line 280)
  at org/apache/tools/ant/launch/Launcher.main (line 109)


Just tell me if you need more or if I should look for something specific!

Thanks and bye,

Simon
Re: Scala success! Simon Ochsenreither 2/12/13 12:26 PM
head -200 throw_log
java/lang/IllegalArgumentException: Unknown signal: HUP
  at sun/misc/Signal.<init> (line 143)
  at java/lang/Terminator.setup (line 60)
  at java/lang/System.initializeSystemClass (line 1148)
java/lang/IllegalArgumentException: Unknown signal: INT
  at sun/misc/Signal.<init> (line 143)
  at java/lang/Terminator.setup (line 64)
  at java/lang/System.initializeSystemClass (line 1148)
java/lang/IllegalArgumentException: Unknown signal: TERM
  at sun/misc/Signal.<init> (line 143)
  at java/lang/Terminator.setup (line 68)
  at java/lang/System.initializeSystemClass (line 1148)
java/lang/ClassNotFoundException: avian/file/Handler
  at java/lang/Class.forName0 (native)
  at java/lang/Class.forName (line 188)
  at java/net/URL.getURLStreamHandler (line 1191)
  at java/net/URL.<init> (line 411)
  at java/net/URL.<init> (line 302)
  at java/net/URL.<init> (line 325)
  at sun/net/www/ParseUtil.fileToEncodedURL (line 272)
  at sun/misc/Launcher.getFileURL (line 460)
  at sun/misc/Launcher.pathToURLs (line 399)
  at sun/misc/Launcher.access$300 (line 55)
  at sun/misc/Launcher$AppClassLoader$1.run (line 281)
  at sun/misc/Launcher$AppClassLoader$1.run (line 279)
  at java/security/AccessController.doPrivileged (native)
  at sun/misc/Launcher$AppClassLoader.getAppClassLoader (line 278)
  at sun/misc/Launcher.<init> (line 79)
  at sun/misc/Launcher.<clinit> (line 57)
java/lang/ClassNotFoundException: avian/file/Handler
  at java/lang/Class.forName0 (native)
  at java/lang/Class.forName (line 188)
  at java/net/URL.getURLStreamHandler (line 1191)
  at java/net/URL.<init> (line 411)
  at java/net/URL.<init> (line 302)
  at java/net/URL.<init> (line 325)
  at sun/net/www/ParseUtil.fileToEncodedURL (line 272)
  at sun/misc/Launcher.getFileURL (line 460)
  at sun/misc/Launcher.pathToURLs (line 399)
  at sun/misc/Launcher.access$300 (line 55)
  at sun/misc/Launcher$AppClassLoader$1.run (line 281)
  at sun/misc/Launcher$AppClassLoader$1.run (line 279)
  at java/security/AccessController.doPrivileged (native)
  at sun/misc/Launcher$AppClassLoader.getAppClassLoader (line 278)
  at sun/misc/Launcher.<init> (line 79)
  at sun/misc/Launcher.<clinit> (line 57)
java/lang/ClassNotFoundException: avian/file/Handler

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/net/URL.getURLStreamHandler (line 1195)
  at java/net/URL.<init> (line 411)
  at java/net/URL.<init> (line 302)
  at java/net/URL.<init> (line 325)
  at sun/net/www/ParseUtil.fileToEncodedURL (line 272)
  at sun/misc/Launcher.getFileURL (line 460)
  at sun/misc/Launcher.pathToURLs (line 399)
  at sun/misc/Launcher.access$300 (line 55)
  at sun/misc/Launcher$AppClassLoader$1.run (line 281)
  at sun/misc/Launcher$AppClassLoader$1.run (line 279)
  at java/security/AccessController.doPrivileged (native)
  at sun/misc/Launcher$AppClassLoader.getAppClassLoader (line 278)
  at sun/misc/Launcher.<init> (line 79)
  at sun/misc/Launcher.<clinit> (line 57)
java/lang/ClassNotFoundException: avian/file/Handler

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/net/URL.getURLStreamHandler (line 1195)
  at java/net/URL.<init> (line 411)
  at java/net/URL.<init> (line 302)
  at java/net/URL.<init> (line 325)
  at sun/net/www/ParseUtil.fileToEncodedURL (line 272)
  at sun/misc/Launcher.getFileURL (line 460)
  at sun/misc/Launcher.pathToURLs (line 399)
  at sun/misc/Launcher.access$300 (line 55)
  at sun/misc/Launcher$AppClassLoader$1.run (line 281)
  at sun/misc/Launcher$AppClassLoader$1.run (line 279)
  at java/security/AccessController.doPrivileged (native)
  at sun/misc/Launcher$AppClassLoader.getAppClassLoader (line 278)
  at sun/misc/Launcher.<init> (line 79)
  at sun/misc/Launcher.<clinit> (line 57)
java/lang/ClassNotFoundException: avian/file/Handler

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/net/URL.getURLStreamHandler (line 1195)
  at java/net/URL.<init> (line 411)
  at java/net/URL.<init> (line 302)
  at java/net/URL.<init> (line 325)
  at sun/net/www/ParseUtil.fileToEncodedURL (line 272)
  at sun/misc/Launcher.getFileURL (line 460)
  at sun/misc/Launcher.pathToURLs (line 399)
  at sun/misc/Launcher.access$300 (line 55)
  at sun/misc/Launcher$AppClassLoader$1.run (line 281)
  at sun/misc/Launcher$AppClassLoader$1.run (line 279)
  at java/security/AccessController.doPrivileged (native)
  at sun/misc/Launcher$AppClassLoader.getAppClassLoader (line 278)
  at sun/misc/Launcher.<init> (line 79)
  at sun/misc/Launcher.<clinit> (line 57)
java/lang/ClassNotFoundException: avian/file/Handler

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/net/URL.getURLStreamHandler (line 1195)
  at java/net/URL.<init> (line 411)
  at java/net/URL.<init> (line 302)
  at java/net/URL.<init> (line 325)
  at sun/net/www/ParseUtil.fileToEncodedURL (line 272)
  at sun/misc/Launcher.getFileURL (line 460)
  at sun/misc/Launcher.pathToURLs (line 399)
  at sun/misc/Launcher.access$300 (line 55)
  at sun/misc/Launcher$AppClassLoader$1.run (line 281)
  at sun/misc/Launcher$AppClassLoader$1.run (line 279)
  at java/security/AccessController.doPrivileged (native)
  at sun/misc/Launcher$AppClassLoader.getAppClassLoader (line 278)
  at sun/misc/Launcher.<init> (line 79)
  at sun/misc/Launcher.<clinit> (line 57)
java/lang/ClassNotFoundException: avian/file/Handler

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/net/URL.getURLStreamHandler (line 1195)
  at java/net/URL.<init> (line 411)
  at java/net/URL.<init> (line 302)
  at java/net/URL.<init> (line 325)
  at sun/net/www/ParseUtil.fileToEncodedURL (line 272)
  at sun/misc/Launcher.getFileURL (line 460)
  at sun/misc/Launcher.pathToURLs (line 399)
  at sun/misc/Launcher.access$300 (line 55)
  at sun/misc/Launcher$AppClassLoader$1.run (line 281)
  at sun/misc/Launcher$AppClassLoader$1.run (line 279)
  at java/security/AccessController.doPrivileged (native)
  at sun/misc/Launcher$AppClassLoader.getAppClassLoader (line 278)
  at sun/misc/Launcher.<init> (line 79)
  at sun/misc/Launcher.<clinit> (line 57)
java/lang/ClassNotFoundException: avian/file/Handler

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/net/URL.getURLStreamHandler (line 1195)
  at java/net/URL.<init> (line 411)
  at java/net/URL.<init> (line 302)
  at java/net/URL.<init> (line 325)
  at sun/net/www/ParseUtil.fileToEncodedURL (line 272)
  at sun/misc/Launcher.getFileURL (line 460)
  at sun/misc/Launcher.pathToURLs (line 399)
  at sun/misc/Launcher.access$300 (line 55)
  at sun/misc/Launcher$AppClassLoader$1.run (line 281)
  at sun/misc/Launcher$AppClassLoader$1.run (line 279)
  at java/security/AccessController.doPrivileged (native)
  at sun/misc/Launcher$AppClassLoader.getAppClassLoader (line 278)
  at sun/misc/Launcher.<init> (line 79)
  at sun/misc/Launcher.<clinit> (line 57)
java/lang/ClassNotFoundException: org/apache/tools/ant/launch/Launcher

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at sun/launcher/LauncherHelper.checkAndLoadMain (line 482)
java/lang/ClassNotFoundException: org/apache/tools/ant/launch/Launcher

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at sun/launcher/LauncherHelper.checkAndLoadMain (line 482)
java/lang/ClassNotFoundException: org/apache/tools/ant/launch/Launcher

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at sun/launcher/LauncherHelper.checkAndLoadMain (line 482)
java/lang/ClassNotFoundException: org/apache/tools/ant/launch/Launcher

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at sun/launcher/LauncherHelper.checkAndLoadMain (line 482)
java/lang/ClassNotFoundException: avian/jar/Handler
  at java/lang/Class.forName0 (native)
  at java/lang/Class.forName (line 188)
  at java/net/URL.getURLStreamHandler (line 1191)
  at java/net/URL.<init> (line 590)
  at java/net/URL.<init> (line 482)
  at java/net/URL.<init> (line 431)
  at avian/SystemClassLoader.findResource (line 44)
Re: Scala success! Joel Dice 2/12/13 1:45 PM
On Tue, 12 Feb 2013, simon.och...@gmail.com wrote:

> org/apache/tools/ant/BuildException
> ᅵ at org/apache/tools/ant/dispatch/DispatchUtils.execute (line 116)
> ᅵ at org/apache/tools/ant/Task.perform (line 348)
> ᅵ at org/apache/tools/ant/Target.execute (line 392)
> ᅵ at org/apache/tools/ant/Target.performTasks (line 413)
> ᅵ at org/apache/tools/ant/Project.executeSortedTargets (line 1399)
> ᅵ at org/apache/tools/ant/Project.executeTarget (line 1368)
> ᅵ at org/apache/tools/ant/helper/DefaultExecutor.executeTargets (line 41)
> ᅵ at org/apache/tools/ant/Project.executeTargets (line 1251)
> ᅵ at org/apache/tools/ant/Main.runBuild (line 811)
> ᅵ at org/apache/tools/ant/Main.startAnt (line 217)
> ᅵ at org/apache/tools/ant/launch/Launcher.run (line 280)
> ᅵ at org/apache/tools/ant/launch/Launcher.main (line 109)

We need to know what caused that BuildException (i.e. the exception it is
wrapping).  If you can compress the log and upload it somewhere, I can
take a look.
Re: Scala success! Simon Ochsenreither 2/12/13 1:59 PM
Hello Joel,



We need to know what caused that BuildException (i.e. the exception it is
wrapping).  If you can compress the log and upload it somewhere, I can
take a look.

wow, xz managed to compress the 192 MB down to less than 100 kB. I'm attaching the archive.

Thanks and bye,

Simon

Edit: “
An error (#340) occurred while communicating with the server.
” I uploaded it here: http://oxnrtr.de/upload/throw_log.xz
Re: Scala success! Joel Dice 2/12/13 3:08 PM
On Tue, 12 Feb 2013, simon.och...@gmail.com wrote:

> Hello Joel,
>
>
>       We need to know what caused that BuildException (i.e. the
>       exception it is
>       wrapping). �If you can compress the log and upload it somewhere,
>       I can
>       take a look.
>
>
> wow, xz managed to compress the 192 MB down to less than 100 kB. I'm
> attaching the archive.

Thanks.

This looks like at least part of the problem:

"org/apache/tools/ant/BuildException: ../jdk-avian/jre/../bin is not an
absolute path"

May I assume you set JAVA_HOME to a relative path?  If so, could you
change it to an absolute path and try again?

Re: Scala success! Simon Ochsenreither 2/12/13 5:12 PM
Hi Joel,


"org/apache/tools/ant/BuildException: ../jdk-avian/jre/../bin is not an
absolute path"

May I assume you set JAVA_HOME to a relative path?  If so, could you
change it to an absolute path and try again?

For god's sake, if that's the cause ... sorry for wasting your time then.

I'll test it as soon as I'm home again.

Bye,

Simon
Re: Scala success! Simon Ochsenreither 2/13/13 9:52 AM
Hi Joel,


"org/apache/tools/ant/BuildException: ../jdk-avian/jre/../bin is not an
absolute path"

May I assume you set JAVA_HOME to a relative path?  If so, could you
change it to an absolute path and try again?

I tried that and I get the same error message, although the log is shorter now (18.2 MB): http://oxnrtr.de/upload/throw_log2.xz

I think there is something else wrong; this happens when I run a simple java:

JAVA_HOME=/home/soc/Entwicklung/avian-build/jdk-avian LD_LIBRARY_PATH=/home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64 /home/soc/Entwicklung/avian-build/jdk-avian/jre/bin/javajava/lang/ClassNotFoundException: sun/launcher/resources/launcher_de_DE

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 724)
  at sun/launcher/LauncherHelper$ResourceBundleHolder.<clinit> (line 87)
  at sun/launcher/LauncherHelper.getLocalizedMessage (line 316)
  at sun/launcher/LauncherHelper.initHelpMessage (line 328)
java/lang/ClassNotFoundException: sun/launcher/resources/launcher_de_DE

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 724)
  at sun/launcher/LauncherHelper$ResourceBundleHolder.<clinit> (line 87)
  at sun/launcher/LauncherHelper.getLocalizedMessage (line 316)
  at sun/launcher/LauncherHelper.initHelpMessage (line 328)
java/lang/ClassNotFoundException: sun/launcher/resources/launcher_de_DE

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 724)
  at sun/launcher/LauncherHelper$ResourceBundleHolder.<clinit> (line 87)
  at sun/launcher/LauncherHelper.getLocalizedMessage (line 316)
  at sun/launcher/LauncherHelper.initHelpMessage (line 328)
java/lang/ClassNotFoundException: sun/launcher/resources/launcher_de_DE

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 724)
  at sun/launcher/LauncherHelper$ResourceBundleHolder.<clinit> (line 87)
  at sun/launcher/LauncherHelper.getLocalizedMessage (line 316)
  at sun/launcher/LauncherHelper.initHelpMessage (line 328)
Verwendung: java [-options] class [args...]
           (zur Ausf?hrung einer Klasse)
   oder  java [-options] -jar jarfile [args...]
           (zur Ausf?hrung einer JAR-Datei)
wobei options Folgendes umfasst:
    -d32      Verwendet ein 32-Bit-Datenmodell, sofern verf?gbar
    -d64      Verwendet ein 64-Bit-Datenmodell, sofern verf?gbar
    -server      zur Auswahl der "server" VM
                  Die Standard-VM ist server,
                  weil die Ausf?hrung auf einem Server-Class-Rechner erfolgt.


    -cp <Class-Suchpfad von Verzeichnissen und .zip-/.jar-Dateien>
    -classpath <Class-Suchpfad von Verzeichnissen und .zip-/.jar-Dateien>
                  Eine durch : getrennte Liste mit Verzeichnissen, .jar-Archiven
                  und .zip-Archiven zur Suche nach Klassendateien.
    -D<name>=<value>
                  Setzt eine Systemeigenschaft
    -verbose[:class|gc|jni]
                  Aktiviert die Verbose-Ausgabe
    -version      Druckt Produktversion und beendet das Programm
    -version:<value>
                  Erfordert die angegebene Version zur Ausf?hrung
    -showversion  Druckt Produktversion und f?hrt fort
    -jre-restrict-search | -no-jre-restrict-search
                  Bezieht private JREs des Benutzers in Versionssuche ein bzw. schlie?t sie aus
    -? -help      Druckt diese Hilfemeldung
    -X            Druckt Hilfe zu Nicht-Standardoptionen
    -ea[:<packagename>...|:<classname>]
    -enableassertions[:<packagename>...|:<classname>]
                  Aktiviert Assertionen mit angegebener Granularit?t
    -da[:<packagename>...|:<classname>]
    -disableassertions[:<packagename>...|:<classname>]
                  Deaktiviert Assertionen mit angegebener Granularit?t
    -esa | -enablesystemassertions
                  Aktiviert Systemassertionen
    -dsa | -disablesystemassertions
                  Deaktiviert Systemassertionen
    -agentlib:<libname>[=<options>]
                  L?dt native Agent Library <libname>, z.B. -agentlib:hprof
                  siehe auch, -agentlib:jdwp=help und -agentlib:hprof=help
    -agentpath:<pathname>[=<options>]
                  L?dt native Agent Library nach vollem Pfadnamen
    -javaagent:<jarpath>[=<options>]
                  L?dt Java-Programmiersprachen-Agent, siehe java.lang.instrument
    -splash:<imagepath>
                  Zeigt Startbildschirm mit angegebenem Bild
Weitere Einzelheiten finden Sie unter http://java.sun.com/javase/reference.
java/lang/Throwable
java/lang/Throwable
java/lang/Throwable
java/lang/Throwable


... and javac:

JAVA_HOME=/home/soc/Entwicklung/avian-build/jdk-avian LD_LIBRARY_PATH=/home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64 /home/soc/Entwicklung/avian-build/jdk-avian/bin/javacjava/lang/ClassNotFoundException: com/sun/tools/javac/Main

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at sun/launcher/LauncherHelper.checkAndLoadMain (line 482)
java/lang/ClassNotFoundException: com/sun/tools/javac/Main

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at sun/launcher/LauncherHelper.checkAndLoadMain (line 482)
java/lang/ClassNotFoundException: com/sun/tools/javac/Main

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at sun/launcher/LauncherHelper.checkAndLoadMain (line 482)
java/lang/ClassNotFoundException: com/sun/tools/javac/Main

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at sun/launcher/LauncherHelper.checkAndLoadMain (line 482)
java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 796)
  at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
  at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
  at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
  at com/sun/tools/javac/main/Main.help (line 151)
  at com/sun/tools/javac/main/Main.compile (line 377)
  at com/sun/tools/javac/main/Main.compile (line 353)
  at com/sun/tools/javac/main/Main.compile (line 342)
  at com/sun/tools/javac/main/Main.compile (line 333)
  at com/sun/tools/javac/Main.compile (line 76)
  at com/sun/tools/javac/Main.main (line 61)
java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 796)
  at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
  at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
  at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
  at com/sun/tools/javac/main/Main.help (line 151)
  at com/sun/tools/javac/main/Main.compile (line 377)
  at com/sun/tools/javac/main/Main.compile (line 353)
  at com/sun/tools/javac/main/Main.compile (line 342)
  at com/sun/tools/javac/main/Main.compile (line 333)
  at com/sun/tools/javac/Main.compile (line 76)
  at com/sun/tools/javac/Main.main (line 61)
java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 796)
  at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
  at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
  at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
  at com/sun/tools/javac/main/Main.help (line 151)
  at com/sun/tools/javac/main/Main.compile (line 377)
  at com/sun/tools/javac/main/Main.compile (line 353)
  at com/sun/tools/javac/main/Main.compile (line 342)
  at com/sun/tools/javac/main/Main.compile (line 333)
  at com/sun/tools/javac/Main.compile (line 76)
  at com/sun/tools/javac/Main.main (line 61)
java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 796)
  at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
  at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
  at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
  at com/sun/tools/javac/main/Main.help (line 151)
  at com/sun/tools/javac/main/Main.compile (line 377)
  at com/sun/tools/javac/main/Main.compile (line 353)
  at com/sun/tools/javac/main/Main.compile (line 342)
  at com/sun/tools/javac/main/Main.compile (line 333)
  at com/sun/tools/javac/Main.compile (line 76)
  at com/sun/tools/javac/Main.main (line 61)
java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac_de

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 796)
  at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
  at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
  at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
  at com/sun/tools/javac/main/Main.help (line 151)
  at com/sun/tools/javac/main/Main.compile (line 377)
  at com/sun/tools/javac/main/Main.compile (line 353)
  at com/sun/tools/javac/main/Main.compile (line 342)
  at com/sun/tools/javac/main/Main.compile (line 333)
  at com/sun/tools/javac/Main.compile (line 76)
  at com/sun/tools/javac/Main.main (line 61)
java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac_de

  at java/lang/ClassLoader.findBootstrapClass (native)
  at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
  at java/lang/ClassLoader.loadClass (line 412)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBundle.findBundle (line 1400)
  at java/util/ResourceBundle.findBundle (line 1354)
  at java/util/ResourceBundle.getBundleImpl (line 1296)
  at java/util/ResourceBundle.getBundle (line 796)
  at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
  at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
  at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
  at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
  at com/sun/tools/javac/main/Main.help (line 151)
  at com/sun/tools/javac/main/Main.compile (line 377)
  at com/sun/tools/javac/main/Main.compile (line 353)
  at com/sun/tools/javac/main/Main.compile (line 342)
  at com/sun/tools/javac/main/Main.compile (line 333)
  at com/sun/tools/javac/Main.compile (line 76)
  at com/sun/tools/javac/Main.main (line 61)
java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac_de

  at avian/SystemClassLoader.findVMClass (native)
  at avian/SystemClassLoader.findClass (line 26)
  at java/lang/ClassLoader.loadClass (line 423)
  at java/lang/ClassLoader.loadClass (line 410)
  at java/lang/ClassLoader.loadClass (line 356)
  at java/util/ResourceBundle$Control.newBundle (line 2565)
  at java/util/ResourceBundle.loadBundle (line 1436)
  at java/util/ResourceBu
...
Re: Scala success! Joel Dice 2/13/13 10:01 AM
On Wed, 13 Feb 2013, simon.och...@gmail.com wrote:

> Hi Joel,
>
>       "org/apache/tools/ant/BuildException: ../jdk-avian/jre/../bin is
>       not an
>       absolute path"
>
>       May I assume you set JAVA_HOME to a relative path? �If so, could
>       you
>       change it to an absolute path and try again?
>
>
> I tried that and I get the same error message, although the log is shorter
> now (18.2 MB): http://oxnrtr.de/upload/throw_log2.xz

I'm still seeing "org/apache/tools/ant/BuildException:
../jdk-avian/jre/../bin is not an absolute path" in the latest log.  Are
you absolutely sure JAVA_HOME is set to an absolute path?
> � at java/util/ResourceBundle.findBundle (line 1400)
> � at java/util/ResourceBundle.findBundle (line 1354)
> � at java/util/ResourceBundle.getBundleImpl (line 1296)
> � at java/util/ResourceBundle.getBundle (line 796)
> � at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
> � at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
> � at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
> � at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
> � at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
> � at com/sun/tools/javac/main/Main.help (line 151)
> � at com/sun/tools/javac/main/Main.compile (line 377)
> � at com/sun/tools/javac/main/Main.compile (line 353)
> � at com/sun/tools/javac/main/Main.compile (line 342)
> � at com/sun/tools/javac/main/Main.compile (line 333)
> � at com/sun/tools/javac/Main.compile (line 76)
> � at com/sun/tools/javac/Main.main (line 61)
> java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac_de
> � at avian/SystemClassLoader.findVMClass (native)
> � at avian/SystemClassLoader.findClass (line 26)
> � at java/lang/ClassLoader.loadClass (line 423)
> � at java/lang/ClassLoader.loadClass (line 410)
> � at java/lang/ClassLoader.loadClass (line 356)
> � at java/util/ResourceBundle$Control.newBundle (line 2565)
> � at java/util/ResourceBundle.loadBundle (line 1436)
> � at java/util/ResourceBundle.findBundle (line 1400)
> � at java/util/ResourceBundle.findBundle (line 1354)
> � at java/util/ResourceBundle.getBundleImpl (line 1296)
> � at java/util/ResourceBundle.getBundle (line 796)
> � at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
> � at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
> � at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
> � at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
> � at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
> � at com/sun/tools/javac/main/Main.help (line 151)
> � at com/sun/tools/javac/main/Main.compile (line 377)
> � at com/sun/tools/javac/main/Main.compile (line 353)
> � at com/sun/tools/javac/main/Main.compile (line 342)
> � at com/sun/tools/javac/main/Main.compile (line 333)
> � at com/sun/tools/javac/Main.compile (line 76)
> � at com/sun/tools/javac/Main.main (line 61)
> java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac_de
> � at avian/SystemClassLoader.findVMClass (native)
> � at avian/SystemClassLoader.findClass (line 26)
> � at java/lang/ClassLoader.loadClass (line 423)
> � at java/lang/ClassLoader.loadClass (line 356)
> � at java/util/ResourceBundle$Control.newBundle (line 2565)
> � at java/util/ResourceBundle.loadBundle (line 1436)
> � at java/util/ResourceBundle.findBundle (line 1400)
> � at java/util/ResourceBundle.findBundle (line 1354)
> � at java/util/ResourceBundle.getBundleImpl (line 1296)
> � at java/util/ResourceBundle.getBundle (line 796)
> � at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
> � at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
> � at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
> � at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
> � at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
> � at com/sun/tools/javac/main/Main.help (line 151)
> � at com/sun/tools/javac/main/Main.compile (line 377)
> � at com/sun/tools/javac/main/Main.compile (line 353)
> � at com/sun/tools/javac/main/Main.compile (line 342)
> � at com/sun/tools/javac/main/Main.compile (line 333)
> � at com/sun/tools/javac/Main.compile (line 76)
> � at com/sun/tools/javac/Main.main (line 61)
> java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac_de
> � at avian/SystemClassLoader.findVMClass (native)
> � at avian/SystemClassLoader.findClass (line 26)
> � at java/lang/ClassLoader.loadClass (line 423)
> � at java/lang/ClassLoader.loadClass (line 356)
> � at java/util/ResourceBundle$Control.newBundle (line 2565)
> � at java/util/ResourceBundle.loadBundle (line 1436)
> � at java/util/ResourceBundle.findBundle (line 1400)
> � at java/util/ResourceBundle.findBundle (line 1354)
> � at java/util/ResourceBundle.getBundleImpl (line 1296)
> � at java/util/ResourceBundle.getBundle (line 796)
> � at com/sun/tools/javac/util/JavacMessages.getBundles (line 121)
> � at com/sun/tools/javac/util/JavacMessages.setCurrentLocale (line 73)
> � at com/sun/tools/javac/util/JavacMessages.<init> (line 98)
> � at com/sun/tools/javac/util/JavacMessages.<init> (line 88)
> � at com/sun/tools/javac/main/Main.getLocalizedString (line 584)
> � at com/sun/tools/javac/main/Main.help (line 151)
> � at com/sun/tools/javac/main/Main.compile (line 377)
> � at com/sun/tools/javac/main/Main.compile (line 353)
> � at com/sun/tools/javac/main/Main.compile (line 342)
> � at com/sun/tools/javac/main/Main.compile (line 333)
> � at com/sun/tools/javac/Main.compile (line 76)
> � at com/sun/tools/javac/Main.main (line 61)
> java/lang/ClassNotFoundException: com/sun/tools/javac/resources/javac_de_DE
> � at java/lang/ClassLoader.findBootstrapClass (native)
> � at java/lang/ClassLoader.findBootstrapClassOrNull (line 1061)
> � at java/lang/ClassLoader.loadClass (line 412)
> � at java/lang/ClassLoader.loadClass (line 410)
> � at java/lang/ClassLoader.loadClass (line 356)
> � at java/util/ResourceBundle$Control.newBundle (line 2565)
> � at java/util/ResourceBundle.loadBundle (line 1436)
> � at ja...
Re: Scala success! Simon Ochsenreither 2/13/13 10:15 AM
Hi Joel,


I'm still seeing "org/apache/tools/ant/BuildException:
../jdk-avian/jre/../bin is not an absolute path" in the latest log.  Are
you absolutely sure JAVA_HOME is set to an absolute path?

Yes. I have absolutely no idea how or where it gets that path from (it is the correct path). I even checked whether there are any symlinks confusing Ant.
 
None of that looks abnormal to me.  When you print every exception that is
ever thrown, you get a lot of noise, but that doesn't mean anything is
wrong.

Ok, thanks.

Bye,

Simon
Re: Scala success! Joel Dice 2/13/13 10:27 AM
On Wed, 13 Feb 2013, simon.och...@gmail.com wrote:

> Hi Joel,
>
>       I'm still seeing "org/apache/tools/ant/BuildException:
>       ../jdk-avian/jre/../bin is not an absolute path" in the latest
>       log. �Are
>       you absolutely sure JAVA_HOME is set to an absolute path?
>
>
> Yes. I have absolutely no idea how or where it gets that path from (it is
> the correct path). I even checked whether there are any symlinks confusing
> Ant.

Are you still using this command?

% LD_LIBRARY_PATH=../jdk-avian/jre/lib/amd64 ionice -c 3 nice
../jdk-avian/jre/bin/java -Xmx2048m -Xss1m -classpath
../../lib/apache-ant-1.8.4/lib/ant-launcher.jar
-Dant.home=../../lib/apache-ant-1.8.4
-Dant.library.dir=../../lib/apache-ant-1.8.3/lib
org.apache.tools.ant.launch.Launcher test

If so, would you please try changing "../jdk-avian/jre/bin/java" to an
absolute path?  Maybe Ant is trying to find javac based on the java
command and getting confused by the relative path.
Re: Scala success! Simon Ochsenreither 2/13/13 10:31 AM

If so, would you please try changing "../jdk-avian/jre/bin/java" to an
absolute path?  Maybe Ant is trying to find javac based on the java
command and getting confused by the relative path.

I'm already using this:

JAVA_HOME=/home/soc/Entwicklung/avian-build/jdk-avian LD_LIBRARY_PATH=/home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64 /home/soc/Entwicklung/avian-build/jdk-avian/jre/bin/java -Xmx2048m -Xss1m -classpath /home/soc/Entwicklung/lib/apache-ant-1.8.4/lib/ant-launcher.jar -Dant.home=/home/soc/Entwicklung/lib/apache-ant-1.8.4 -Dant.library.dir=/home/soc/Entwicklung/lib/apache-ant-1.8.4/lib org.apache.tools.ant.launch.Launcher test

Re: Scala success! Joel Dice 2/13/13 10:40 AM
Looking at JavaEnvUtils in the Ant source tree, I see that it is creating
the path using the java.home system property, which is hard-coded into
Avian's libjvm.so.  Did you specify a relative path when building Avian?
E.g. openjdk=../jdk-avian>  If so, would you do a clean build with an
absolute path?
Re: Scala success! Simon Ochsenreither 2/13/13 10:42 AM

Thanks, I'll check that!
Re: Scala success! Simon Ochsenreither 2/13/13 10:56 AM
Wow, it seems as this was the error all the time. :-/

Would it make sense to fix/mention this somewhere?

Thanks,

Simon
Re: Scala success! Joel Dice 2/13/13 11:29 AM
On Wed, 13 Feb 2013, simon.och...@gmail.com wrote:

> Wow, it seems as this was the error all the time. :-/
>
> Would it make sense to fix/mention this somewhere?

Yeah; the question is what to do about it.  We could try to automatically
convert a relative path to an absolute one in the makefile, or just
document that only absolute paths are allowed.
Re: Scala success! Simon Ochsenreither 2/13/13 12:36 PM
Update: I successfully compiled locker.lib and locker.reflect, it took ages with locker.comp, so I aborted that, compiled Scala with OpenJDK/HotSpot instead.

I'm currently running the tests with Avian and until now all tests pass!
Re: Scala success! Simon Ochsenreither 2/13/13 4:05 PM
Still running the tests and no failures so far!

Do you have an idea why the tests run very slow? top tells me that only a single CPU is used, although partest should run tests in parallel.

Bye,

Simon
Re: Scala success! Joel Dice 2/13/13 4:14 PM
On Wed, 13 Feb 2013, simon.och...@gmail.com wrote:

If you followed my instructions, you built with mode=debug-fast, which
means optimization is disabled for all the native code.  I did that
because I want to get a sensible core dump if it crashes, but it also
means it will be much slower than an optimized build.  Also, my
instructions specified using ionice and nice to make the whole process the
lowest priority relative to everything else running on the system, which
also slows things down.  I always do that with long running processes so
it doesn't bog down my interactive apps.

I'm not sure if either of those things explain why it's only using one
CPU.  My current priority is to make sure the tests pass without crashing;
once that's verified (which may literally take many days at this rate),
I'll start profiling to see where time is being spent.
Re: Scala success! Joel Dice 2/13/13 4:17 PM
On Wed, 13 Feb 2013, Joel Dice wrote:

> I'm not sure if either of those things explain why it's only using one CPU.
> My current priority is to make sure the tests pass without crashing; once
> that's verified (which may literally take many days at this rate), I'll start
> profiling to see where time is being spent.

If I had to guess, though, I'd say it's spending almost all its time
garbage collecting.  That's not parallelized, so it would explain using
only one CPU, and it's especially sensitive to the C++
compiler optimization level.
Re: Scala success! Simon Ochsenreither 2/13/13 4:37 PM
Hi Joel,

I'm running the tests without mode=debug-fast (but with tails=true) and without ionice or nice already.
(If a test kills Avian, one can always run that individual test by calling partest directly like this: % test/partest test/files/run/t4080.scala)

Thanks and bye,

Simon
Re: Scala success! Simon Ochsenreither 2/13/13 4:39 PM

If I had to guess, though, I'd say it's spending almost all its time
garbage collecting.  That's not parallelized, so it would explain using
only one CPU, and it's especially sensitive to the C++
compiler optimization level.

Yes, sounds likely considering that the tests basically consist of using scalac to check that code X compiles and throwing it away afterwards.
Re: Scala success! Simon Ochsenreither 2/14/13 5:42 PM
Did your tests finish yet? Mine is still running (run/reify... currently).
Re: Scala success! Joel Dice 2/14/13 5:48 PM
On Thu, 14 Feb 2013, simon.och...@gmail.com wrote:

> Did your tests finish yet? Mine is still running (run/reify... currently).

No, still running.  When I have time, I'll profile an optimized build to
find out why they're so slow.
Re: Scala success! Simon Ochsenreither 2/16/13 3:26 AM
Hi Joel,

my results are in: http://oxnrtr.de/upload/test-log

Did it finish for you yet?

Thanks any bye,

Simon
Re: Scala success! Joel Dice 2/16/13 7:42 AM
On Sat, 16 Feb 2013, simon.och...@gmail.com wrote:

> my results are in: http://oxnrtr.de/upload/test-log

Thanks for posting them.

> Did it finish for you yet?

Unfortunately, I lost my SSH connection to the machine that it was running
on.  I really should have been running it in Screen, but I wasn't.  I'll
try profiling one of the tests to see if there are any easy ways to speed
it up before I try running the whole suite again.
Re: Scala success! Pablo Guerrero 2/16/13 10:29 AM
Hi Simon,

That looks great, appart from a couple of tests it's working fine. Do you know how many tests didn't run because of the crash?
Is it possible to run the remaining tests without retesting all the tests that were ok?

Cheers,
Pablo




--
You received this message because you are subscribed to the Google Groups "Avian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to avian+un...@googlegroups.com.
To post to this group, send email to av...@googlegroups.com.
Visit this group at http://groups.google.com/group/avian?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.



Re: Scala success! Simon Ochsenreither 2/16/13 10:43 AM

That looks great, appart from a couple of tests it's working fine. Do you know how many tests didn't run because of the crash?

I think a few dozen, at max.
 
Is it possible to run the remaining tests without retesting all the tests that were ok?

Yes, I'll do that with the failed test to see whether it was a spurious failure as well as the remaining ones.
Re: Scala success! Simon Ochsenreither 2/16/13 11:30 AM
Update:

testing: [...]/files/run/parserJavaIdent.scala                        [FAILED]

This seems to be a valid error (maybe an issue with changed Unicode character categories?)

testing: [...]/files/scalacheck/array-old.scala                       [  OK  ]

1 of the 2 remaining tests ran successfully, it still tests/hangs on the other one, test/files/scalacheck/Ctrie.scala.

Apart from that, we're finished. :-)
Re: Scala success! Pablo Guerrero 2/16/13 12:14 PM
Wow, thanks for testing it.



--
You received this message because you are subscribed to the Google Groups "Avian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to avian+un...@googlegroups.com.
To post to this group, send email to av...@googlegroups.com.
Visit this group at http://groups.google.com/group/avian?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Re: Scala success! Simon Ochsenreither 2/16/13 2:55 PM
Ok, Ctrie seems to cause tho OOME:

% test/partest test/files/scalacheck/Ctrie.scala

Testing individual files
java/lang/OutOfMemoryError
java/lang/OutOfMemoryError
Aborted (core dumped)
test/partest test/files/scalacheck/Ctrie.scala  2476,37s user 175,45s system 113% cpu 39:01,77 total

Re: Scala success! Joel Dice 2/18/13 7:53 AM
On Sat, 16 Feb 2013, simon.och...@gmail.com wrote:

I set Verbose = true in heap.cpp to see what was going on GC-wise.
Unsurprisingly given the (lack of) performance we've been seeing, the VM
was spending about half its time garbage collecting, but stayed steady at
about 130MB of heap used at first.  After a while, though, the heap
started growing again, and when the VM started running low on memory it
spent a lot more time GC'ing, to the point that it spent almost all its
time doing that.  I never got a core dump, but it did eventually run out
of memory.

I see two problems here:

  1. The Ctrie test is using an apparently unbounded amount of memory on
Avian, which I'll need to compare with HotSpot to see if there's an
Avian-specific leak.

  2. Avian slows way, way down when it runs low on memory, which in this
case just delays the inevitable OOME.  While it should certainly make its
best effort to GC prior to an OOME, the current strategy is too aggressive
too early.

More generally, it seems that the GC code is not well tuned for the Scala
compiler, which allocates and retains a lot more memory than the small
apps Avian was designed for.  I strongly suspect this is a big part
of the reason all these tests are slow.  To make these tests run faster,
we'll need to improve the allocation and collection code to scale up
automatically without hurting efficiency for smaller apps.  I doubt we'll
be able to get very close to HotSpot's performance without matching its
complexity, but we should be able to get a lot closer than we are now.
Re: Scala success! Simon Ochsenreither 2/28/13 1:13 AM
FYI: Scala 2.9.3 was released today including the necessary compiler fix for the SoftReference issue. The 2.10.x branch will follow with the 2.10.1 release in approximately two weeks.

Thanks again for your help!
Re: Scala success! Pablo Guerrero 2/28/13 1:46 AM
Hi Simon, that's great news !

Thanks for moving this issue.

Cheers,
Pablo


On Thu, Feb 28, 2013 at 10:13 AM, <simon.och...@gmail.com> wrote:
FYI: Scala 2.9.3 was released today including the necessary compiler fix for the SoftReference issue. The 2.10.x branch will follow with the 2.10.1 release in approximately two weeks.

Thanks again for your help!

--
You received this message because you are subscribed to the Google Groups "Avian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to avian+un...@googlegroups.com.
To post to this group, send email to av...@googlegroups.com.
Visit this group at http://groups.google.com/group/avian?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Re: Scala success! Simon Ochsenreither 2/28/13 2:04 AM

Thanks for moving this issue.

To be honest, apart from doing the backport, it was others who did the hard work. :-)

Nonetheless, I'm happy that Scala stuff will work on Avian with a lot less issues from now on.
Re: Scala success! Joel Dice 3/5/13 9:37 AM
On Sat, 16 Feb 2013, simon.och...@gmail.com wrote:

This is fixed:

https://github.com/ReadyTalk/avian/commit/84f99f0dcad3a274f8118479409eccbc0857ad0a

It's still depressingly slow, but it's a lot faster than it used to be,
and it actually succeeds now.
Re: Scala success! Simon Ochsenreither 3/5/13 12:07 PM
Hi Joel,


This is fixed:

https://github.com/ReadyTalk/avian/commit/84f99f0dcad3a274f8118479409eccbc0857ad0a

It's still depressingly slow, but it's a lot faster than it used to be,
and it actually succeeds now.

Wow, great! I'll test it when I arrive at home and report back.

Thanks a lot,

Simon
Re: Scala success! Simon Ochsenreither 3/6/13 4:04 AM
Hi Joel,

I built Scala successfully on Avian and it seems to be faster than before! The tests are still running (against Scala trunk), but I experience a lot of failures:

% JAVA_HOME=/home/soc/Entwicklung/avian-build/jdk-avian/jre/ LD_LIBRARY_PATH=/home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64 /home/soc/Entwicklung/avian-build/jdk-avian/jre/bin/java -Xmx2048m -Xss1m -classpath /home/soc/Entwicklung/lib/apache-ant-1.8.4/lib/ant-launcher.jar -Dant.home=/home/soc/Entwicklung/lib/apache-ant-1.8.4 -Dant.library.dir=/home/soc/Entwicklung/lib/apache-ant-1.8.4/lib org.apache.tools.ant.launch.Launcher test.suite
Buildfile: /home/soc/Entwicklung/avian-build/scala/build.xml

init.jars.check:

init.jars:

init.maven.tasks:

init.maven.jars:
[artifact:dependencies] There were no dependencies specified

init.hasbuildnum:

init.build.snapshot:

init.build.release:

init.build.nopatch.release:

init.build.patch.release:

init.hasmavensuffix:

init.osgi.suffix:

init.osgi.suffix.final:

init.osgi.suffix.snapshot:

init.build.suffix.done:

init.version.git:

init.version.release:

init.version.snapshot:

init.version.done:

init.testjava6:

init.fail.bad.jdk:

init.warn.jdk7:
     [echo]  You are using JDK7 for this build.  While this will be able to build most of Scala, it will
     [echo]        not build the Swing project.   You will be unable to create a distribution.
     [echo]     

init.extra.tasks:
[artifact:dependencies] [WARNING] POM for 'biz.aQute:bndlib:pom:1.43.0:compile' is invalid.
[artifact:dependencies]
[artifact:dependencies] Its dependencies (if any) will NOT be available to the current build.

init:
     [echo]        build time: 6 March 2013, 12:19:36
     [echo]      java version: Avian 1.7.0-internal (1.7)
     [echo]         java args: -Xms1536M -Xmx1536M -Xss1M -XX:MaxPermSize=192M -XX:+UseParallelGC
     [echo]        javac args:
     [echo]       scalac args: -Yreify-copypaste 
     [echo] scalac quick args: -Yreify-copypaste  
     [echo]          git date: 20130212-222356
     [echo]          git hash: bafebe1c16
     [echo]     maven version: 2.11.0-SNAPSHOT
     [echo]      OSGi version: 2.11.0.v20130212-222356-bafebe1c16
     [echo] canonical version: 2.11.0-20130212-222356-bafebe1c16

asm.start:

asm.lib:

asm.done:

forkjoin.start:

forkjoin.lib:

forkjoin.pack:

forkjoin.done:

locker.start:

locker.pre-lib:

locker.lib:

locker.pre-reflect:

locker.reflect:

locker.pre-comp:

locker.comp:

locker.done:

quick.start:

quick.pre-lib:

quick.lib:

quick.swing:

quick.lib.done:

quick.pre-reflect:

quick.reflect:

quick.pre-comp:

quick.comp:

quick.pre-plugins:

quick.plugins:

quick.pre-scalacheck:

quick.scalacheck:

quick.pre-scalap:

quick.scalap:

quick.pre-partest:

quick.partest:

quick.pre-bin:

quick.bin:

quick.done:

pack.start:

pack.pre-lib:

pack.lib:

pack.swing:

pack.pre-reflect:

pack.reflect:

pack.pre-comp:

pack.comp:

pack.pre-plugins:

pack.plugins:

pack.scalacheck:

pack.pre-partest:

pack.partest:

pack.pre-scalap:

pack.scalap:

pack.pre-bin:

pack.bin:

pack.done:

test.suite:
  [partest] Compiling files that are expected to build
  [partest] testing: [...]/files/pos/t1937                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1987b                                       [  OK  ]
  [partest] testing: [...]/files/pos/t4205                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1642                                        [  OK  ]
  [partest] testing: [...]/files/pos/t5692a                                       [  OK  ]
  [partest] testing: [...]/files/pos/t3429                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1836                                        [  OK  ]
  [partest] testing: [...]/files/pos/t6601                                        [  OK  ]
  [partest] testing: [...]/files/pos/java-access-pos                              [  OK  ]
  [partest] testing: [...]/files/pos/t1186                                        [  OK  ]
  [partest] testing: [...]/files/pos/attachments-typed-another-ident              [  OK  ]
  [partest] testing: [...]/files/pos/t1409                                        [  OK  ]
  [partest] testing: [...]/files/pos/t6485a                                       [  OK  ]
  [partest] testing: [...]/files/pos/t3521                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1176                                        [  OK  ]
  [partest] testing: [...]/files/pos/t5639                                        [  OK  ]
  [partest] testing: [...]/files/pos/t4737                                        [  OK  ]
  [partest] testing: [...]/files/pos/t2741                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1107b                                       [  OK  ]
  [partest] testing: [...]/files/pos/t1230                                        [  OK  ]
  [partest] testing: [...]/files/pos/t5031                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1235                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1231                                        [  OK  ]
  [partest] testing: [...]/files/pos/t5644                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1722                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3951                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3249                                        [  OK  ]
  [partest] testing: [...]/files/pos/t2377                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1896                                        [  OK  ]
  [partest] testing: [...]/files/pos/t294                                         [  OK  ]
  [partest] testing: [...]/files/pos/t1459                                        [  OK  ]
  [partest] testing: [...]/files/pos/t4744                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1614                                        [  OK  ]
  [partest] testing: [...]/files/pos/t6245                                        [  OK  ]
  [partest] testing: [...]/files/pos/t6976                                        [  OK  ]
  [partest] testing: [...]/files/pos/t0695                                        [  OK  ]
  [partest] testing: [...]/files/pos/package-implicit                             [  OK  ]
  [partest] testing: [...]/files/pos/t3486                                        [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/cycle-pos.log':
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/ilya2-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/cycle
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/ilya2
  [partest] java.lang.OutOfMemoryError
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/ilya-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/ilya
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t2433-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t2433
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t5013-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t5013
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t2409-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t2409
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/t2464                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3349                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3567                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3938                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1840                                        [  OK  ]
  [partest] testing: [...]/files/pos/t5692b                                       [  OK  ]
  [partest] testing: [...]/files/pos/attachments-typed-ident                      [  OK  ]
  [partest] testing: [...]/files/pos/t2569                                        [  OK  ]
  [partest] testing: [...]/files/pos/t5604                                        [  OK  ]
  [partest] testing: [...]/files/pos/t2956                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3946                                        [  OK  ]
  [partest] testing: [...]/files/pos/t6028                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3404                                        [  OK  ]
  [partest] testing: [...]/files/pos/chang                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1152                                        [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t1203b-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t1203b
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/t4757                                        [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t3076-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t3076
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/t1263                                        [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t2413-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t2413
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/t1196                                        [  OK  ]
  [partest] testing: [...]/files/pos/hk-match                                     [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t6942-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t6942
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/super                                        [  OK  ]
  [partest] testing: [...]/files/pos/annotated-treecopy                           [  OK  ]
  [partest] testing: [...]/files/pos/raw-map                                      [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t3642-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t3642
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/t1102                                        [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t1232-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t1232
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/t5031_3                                      [  OK  ]
  [partest] testing: [...]/files/pos/t1150                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1101                                        [  OK  ]
  [partest] testing: [...]/files/pos/t2764                                        [  OK  ]
  [partest] testing: [...]/files/pos/t2726                                        [  OK  ]
  [partest] testing: [...]/files/pos/t5957                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3419                                        [  OK  ]
  [partest] testing: [...]/files/pos/t2940                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1751                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1942                                        [  OK  ]
  [partest] Note: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t3622/AsyncTask.java uses unchecked or unsafe operations.
  [partest] Note: Recompile with -Xlint:unchecked for details.
  [partest] testing: [...]/files/pos/t6485b                                       [  OK  ]
  [partest] testing: [...]/files/pos/t4402                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1782                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1711                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3622                                        [  OK  ]
  [partest] testing: [...]/files/pos/javaReadsSigs                                [  OK  ]
  [partest] testing: [...]/files/pos/t0612                                        [  OK  ]
  [partest] testing: [...]/files/pos/t0288                                        [  OK  ]
  [partest] testing: [...]/files/pos/t3999                                        [  OK  ]
  [partest] testing: [...]/files/pos/t715                                         [  OK  ]
  [partest] testing: [...]/files/pos/t1197                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1254                                        [  OK  ]
  [partest] testing: [...]/files/pos/t1029                                        [  OK  ]
  [partest] testing: [...]/files/pos/t5604b                                       [  OK  ]
  [partest] testing: [...]/files/pos/t0774                                        [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t5703-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t5703
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/protected-static-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/protected-static
  [partest] java.lang.OutOfMemoryError
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/t5504-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/t5504
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/t5545                                        [  OK  ]
  [partest] Log file '/home/soc/Entwicklung/avian-build/scala/test/files/pos/CustomGlobal-pos.log':
  [partest] Possible compiler crash during test of: /home/soc/Entwicklung/avian-build/scala/test/files/pos/CustomGlobal.scala
  [partest] java.lang.OutOfMemoryError
  [partest] testing: [...]/files/pos/t4603                                        [  OK  ]
  [partest] testing: [...]/files/pos/A.scala                                      [  OK  ]
  [partest] testing: [...]/files/pos/FPTest.scala                                 [  OK  ]
  [partest] testing: [...]/files/pos/t1745                                        [  OK  ]
  [partest] testing: [...]/files/pos/signatures                                   [  OK  ]
  [partest] testing: [...]/files/pos/List1.scala                                  [  OK  ]
  [partest] testing: [...]/files/pos/t5165                                        [  OK  ]
  [partest] testing: [...]/files/pos/inline-access-levels                         [  OK  ]
  [partest] testing: [...]/files/pos/NoCyclicReference.scala                      [  OK  ]
  [partest] testing: [...]/files/pos/S3.scala                                     [  OK  ]
  [partest] testing: [...]/files/pos/S1.scala                                     [  OK  ]
  [partest] testing: [...]/files/pos/MailBox.scala                                [  OK  ]
  [partest] testing: [...]/files/pos/S5.scala                                     [  OK  ]
  [partest] testing: [...]/files/pos/S8.scala                                     [  OK  ]
  [partest] testing: [...]/files/pos/SI-5788.scala                                [  OK  ]
  [partest] testing: [...]/files/pos/SI-7060.scala                                [  OK  ]
  [partest] testing: [...]/files/pos/X.scala                                      [  OK  ]
  [partest] testing: [...]/files/pos/SI-7100.scala                                [  OK  ]
  [partest] testing: [...]/files/pos/Z.scala    &nb
...
Re: Scala success! Joel Dice 3/6/13 7:57 AM
On Wed, 6 Mar 2013, simon.och...@gmail.com wrote:

> Hi Joel,
>
> I built Scala successfully on Avian and it seems to be faster than before!
> The tests are still running (against Scala trunk), but I experience a lot of
> failures:

Ha!  Breaking half the tests is a small price to pay for the speed
improvement, right?

Just kidding, I'll look into it.
Re: Scala success! Simon Ochsenreither 3/6/13 8:26 AM


Ha!  Breaking half the tests is a small price to pay for the speed
improvement, right?

Well, the new timings look certainly impressive: https://groups.google.com/d/msg/scala-language/HW4FvHVLo8k/GardlMrZWgoJ :-)
Re: Scala success! Joel Dice 3/6/13 8:37 AM
On Wed, 6 Mar 2013, simon.och...@gmail.com wrote:

>
>
It just confirms that garbage collection is the bottleneck, since that's
the only thing that's really changed from a performance perspective.  The
good news is there's still more we can do to make it faster, including
thread-local heaps (better parallelism), value types (less churn on the
heap), and just tuning the heuristics and constants in heap.cpp.
Re: Scala success! Joel Dice 3/7/13 10:32 AM
On Wed, 6 Mar 2013, Joel Dice wrote:

> On Wed, 6 Mar 2013, simon.och...@gmail.com wrote:
>
>> I built Scala successfully on Avian and it seems to be faster than before!
>> The tests are still running (against Scala trunk), but I experience a lot
>> of
>> failures:
>
> Ha!  Breaking half the tests is a small price to pay for the speed
> improvement, right?
>
> Just kidding, I'll look into it.

Okay, this is a tricky one.  The recent GC performance improvements have
exposed an issue that I'm not sure how to address.

What's happening is that some of the tests are generating finalizable
objects (i.e. instances of classes which override Object.finalize) faster
than the finalizer thread can finalize them.  This wasn't an issue
previously because the VM was much more aggressive about doing major
collections, especially as memory got tight.  That meant the number of
finalizable objects to accumulate between GC cycles was small enough that
the finalizer thread had no trouble keeping up.  Now, however, the VM
tries to minimize the number of major collections until it's truly almost
out of memory, which produces a large list of finalizable objects at a
point where it's too late to do anything about them, and we're forced to
throw an OOME.

Specifically, I'm seeing a case in the Scala test suite where the VM needs
to do a major collection to make room for an allocation request, and the
result is that only about 350MB of the 1.5GB heap is reachable by "normal"
objects, while the rest is reachable only via finalizable objects.  If we
could block the collection and wait for the finalizer thread to do its
job, we could free up 1.15GB (assuming the finalize methods are well
behaved and don't make stuff reachable again).  However, we can't just
block and wait for that to happen because the finalizer might end up
executing arbitrary code and therefore might try to acquire a lock that
the collecting thread already holds, leading to deadlock.  Nor can we just
finalize the objects directly from the collecting thread, since we might
end up trying to acquire a lock that some other thread holds while its
waiting for the collecting thread to finish (besides which, it would
violate the Java spec to do so).

Apparently the same thing is possible in HotSpot (see 3.1.1 of
http://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/memleaks.html)
and probably any JVM.  OTOH, the whole Scala test suite runs on HotSpot
without errors, so there must be some way to minimize the chance of this
happening even if we can't avoid it in all cases.

Any ideas?

P.S. Object.finalize is an abomination.  I hate, hate, hate, hate, hate
it.  Hate it.
Re: Scala success! Joshua Warner 3/7/13 11:23 AM
My suggestion would be to detect cases where the finalizer thread is running behind and then either do something to bump it's priority, or do something to slow down the rest of the VM.  That slow down could be something like forcing the GC back into the super-aggressive mode, or make every object allocation yield (using sleep(0) or somesuch).

To be clear, I wouldn't suggest these measures to slow down the VM if bumping the thread priority works.

After the finalizer thread has caught up, or at perhaps as it is getting close to being caught up, we remove the slow-down measures, and/or reduce the finalizer thread priority back to normal.

-Joshua




--
You received this message because you are subscribed to the Google Groups "Avian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to avian+un...@googlegroups.com.
To post to this group, send email to av...@googlegroups.com.
Visit this group at http://groups.google.com/group/avian?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.



Re: Scala success! Simon Ochsenreither 3/7/13 11:26 AM

Any ideas?

I think the weird thing is the amount of objects who need to be finalized in the first place.
Even on Hotspot it takes 30 times more time to collect an instance with a finalizer than an instance without, as far as I know.

Do you have a hint which classes are the worst offenders?
I searched in the code base, but didn't find anything which could explain these results.

 
P.S. Object.finalize is an abomination.  I hate, hate, hate, hate, hate
it.  Hate it.

Let's say we are absolutely agree here. :-)
Re: Scala success! Joel Dice 3/7/13 11:38 AM
On Thu, 7 Mar 2013, simon.och...@gmail.com wrote:

>       Any ideas?
>
> I think the weird thing is the amount of objects who need to be finalized in
> the first place.
> Even on Hotspot it takes 30 times more time to collect an instance with a
> finalizer than an instance without, as far as I know.
>
> Do you have a hint which classes are the worst offenders?
> I searched in the code base, but didn't find anything which could explain
> these results.

I've looked more closely now, and it's starting to look like it's
something internal to the VM.  Avian has two kinds of finalizable objects:
the regular, Object.finalize kind and its own internal kind which are
implemented in C++ inside the VM and which don't require a separate thread
to run.  We use the latter to remove interned strings from the intern map,
for example.  It's the internal kind that seem to be the big problem, so
it may have nothing to do with finalizer thread after all.  I'll post when
I have an update.
Re: Scala success! Simon Ochsenreither 3/7/13 11:39 AM

I've looked more closely now, and it's starting to look like it's
something internal to the VM.  Avian has two kinds of finalizable objects:
the regular, Object.finalize kind and its own internal kind which are
implemented in C++ inside the VM and which don't require a separate thread
to run.  We use the latter to remove interned strings from the intern map,
for example.  It's the internal kind that seem to be the big problem, so
it may have nothing to do with finalizer thread after all.  I'll post when
I have an update.

Ah ok, thanks!

By the way, did you see the changes to String made by Oracle in 7u6 (no structural sharing anymore)? Will this have a good or bad impact on Avian, if Avian's implementation does the same?
Re: Scala success! Joel Dice 3/7/13 7:51 PM
On Thu, 7 Mar 2013, simon.och...@gmail.com wrote:

>       I've looked more closely now, and it's starting to look like
>       it's
>       something internal to the VM. �Avian has two kinds of
>       finalizable objects:
>       the regular, Object.finalize kind and its own internal kind
>       which are
>       implemented in C++ inside the VM and which don't require a
>       separate thread
>       to run. �We use the latter to remove interned strings from the
>       intern map,
>       for example. �It's the internal kind that seem to be the big
>       problem, so
>       it may have nothing to do with finalizer thread after all. �I'll
>       post when
>       I have an update.

Yeah, it was a relatively old bug in the VM which was being hidden by the
large number of GC cycles it used to do in low memory situations.  Now
that it does fewer such cycles (which is a good thing), the bug has been
exposed.  It's fixed now:

https://github.com/ReadyTalk/avian/commit/3c44cdc50b6be41440529c8dbd648dd972621593

The good news, besides the fact that the bug was found and fixed, is that
it forced me to think hard about how the VM should act when very low on
memory, and I think it's a lot more robust and performant now than it used
to be.

I've got the Scala test suite running now; so far, so good.

> By the way, did you see the changes to String made by Oracle in 7u6 (no
> structural sharing anymore)? Will this have a good or bad impact on Avian,
> if Avian's implementation does the same?

Yeah, the VM automatically detects whether the class library uses
shared-structure strings or not and matches that behavior for
internally-generated strings.  It's neither good nor bad for Avian; each
strategy has its advantages and disadvantages, and the VM has no problem
accomodating either one.
Re: Scala success! Simon Ochsenreither 3/8/13 2:29 AM


Yeah, it was a relatively old bug in the VM which was being hidden by the
large number of GC cycles it used to do in low memory situations.  Now
that it does fewer such cycles (which is a good thing), the bug has been
exposed.

Looks like Scala and its horrible test-cases have finally been good for something. :-)


 
The good news, besides the fact that the bug was found and fixed, is that
it forced me to think hard about how the VM should act when very low on
memory, and I think it's a lot more robust and performant now than it used
to be.

The compilation seems to be a bit slower than before, but the test run very fast now!
https://groups.google.com/d/msg/scala-language/HW4FvHVLo8k/EaOFceMg88AJ

This looks really great. :-)

Thanks a lot!

Simon
Re: Scala success! Simon Ochsenreither 3/8/13 6:48 AM
What's your test status? It looks pretty good at my side, running both fast and mostly successful, but it seems to hang on scalacheck/Ctrie.scala: https://gist.github.com/soc/5116915

The weird things is that there isn't any CPU usage visible at all (my previous tries with older versions of Avian always caused 100% CPU usage).
Re: Scala success! Joel Dice 3/8/13 8:21 AM
On Fri, 8 Mar 2013, simon.och...@gmail.com wrote:

> What's your test status? It looks pretty good at my side, running both fast
> and mostly successful, but it seems to hang on scalacheck/Ctrie.scala:
> https://gist.github.com/soc/5116915

Yeah, mine did the same thing.  But then I ran the Ctrie test separately
and it finished after 10 minutes.  Now I'm thinking it's the test that
comes after Ctrie that's hanging, since partest gave Ctrie an OK.  I'm
rerunning the suite now with more verbose logging.

> The weird things is that there isn't any CPU usage visible at all (my
> previous tries with older versions of Avian always caused 100% CPU usage).

What I'm seeing is that the suite uses 200-500% CPU on my multicore box
except when GC'ing, but I didn't check the CPU when it hung.
Re: Scala success! Simon Ochsenreither 3/8/13 9:26 AM

Now I'm thinking it's the test that comes after Ctrie that's hanging,
since partest gave Ctrie an OK.
 
Oh right, you're absolutely correct. It's the test after Ctrie.

Re: Scala success! Joel Dice 3/8/13 1:55 PM
On Fri, 8 Mar 2013, simon.och...@gmail.com wrote:

>
>       Now I'm thinking it's the test that comes after Ctrie that's
>       hanging,
>       since partest gave Ctrie an OK.
>
> ᅵ
> Oh right, you're absolutely correct. It's the test after Ctrie.

Looks like it's test/files/scalacheck/parallel-collections, and it's
reproducable when running that test directly.  Can you confirm that that
test ran to completion on an older version of Avian?

I've attached a full thread dump captured during the hang in case it
offers any clues.  I'm still not familiar enough with Scala to know the
best way to debug this.  Any tips?
Re: Scala success! Joel Dice 3/8/13 5:54 PM
Nevermind, I figured it out.  There's a lovely bit of code in
scala/collection/parallel/package.scala:

   private[parallel] def getTaskSupport: TaskSupport =
     if (scala.util.Properties.isJavaAtLeast("1.6")) {
       val vendor = scala.util.Properties.javaVmVendor
       if ((vendor contains "Oracle") || (vendor contains "Sun") || (vendor contains "Apple")) new ForkJoinTaskSupport
       else new ThreadPoolTaskSupport
     } else new ThreadPoolTaskSupport

If I change that to just

   private[parallel] def getTaskSupport: TaskSupport = new ForkJoinTaskSupport

the parallel-collections test passes on Avian and HotSpot.  However, if I
change it to

   private[parallel] def getTaskSupport: TaskSupport = new ThreadPoolTaskSupport

it hangs on both Avian and HotSpot.  In other words, the test simply won't
complete when using ThreadPoolTaskSupport, regardless of the VM, which
means the test can only pass if the VM vendor is Oracle, Sun, or Apple.
Ouch.
Re: Scala success! Simon Ochsenreither 3/9/13 4:36 AM
Hi Joel,


Nevermind, I figured it out.  There's a lovely bit of code in
scala/collection/parallel/package.scala:

   private[parallel] def getTaskSupport: TaskSupport =
     if (scala.util.Properties.isJavaAtLeast("1.6")) {
       val vendor = scala.util.Properties.javaVmVendor
       if ((vendor contains "Oracle") || (vendor contains "Sun") || (vendor contains "Apple")) new ForkJoinTaskSupport
       else new ThreadPoolTaskSupport
     } else new ThreadPoolTaskSupport

I'm wondering what would be he best way to fix it ... changing Avian's vendor name to something like "Avian (Sun-like)" like browser vendors do with their user-agents or just adding Avian to the list of checks above.

(Apart from that, fork join and all the other concurrency stuff would be a nice addition to Avian, because as far as I know all the code is in the public domain.)


If I change that to just

   private[parallel] def getTaskSupport: TaskSupport = new ForkJoinTaskSupport

the parallel-collections test passes on Avian and HotSpot.  However, if I
change it to

   private[parallel] def getTaskSupport: TaskSupport = new ThreadPoolTaskSupport

it hangs on both Avian and HotSpot.  In other words, the test simply won't
complete when using ThreadPoolTaskSupport, regardless of the VM, which
means the test can only pass if the VM vendor is Oracle, Sun, or Apple.
Ouch.

Wow, I'd say that's a horrible bug. I'll create a ticket for that.

Thanks and bye,

Simon
Re: Scala success! Simon Ochsenreither 3/9/13 7:07 AM
Here we go: https://groups.google.com/d/msg/scala-internals/7N-7HzoH8tM/9mx4bZd4JfoJ
Re: Scala success! Joel Dice 3/9/13 7:44 AM
On Sat, 9 Mar 2013, simon.och...@gmail.com wrote:

> Here we go:
> https://groups.google.com/d/msg/scala-internals/7N-7HzoH8tM/9mx4bZd4JfoJ

Thanks!

> (Apart from that, fork join and all the other concurrency stuff would be
> a nice addition to Avian, because as far as I know all the code is in
> the public domain.)

Yes, it's certainly compatible from a licensing perspective, but as I've
said before I don't like forking upstream code when it's already being
maintained by someone else.  The world doesn't need ten different versions
of e.g. ForkJoinPool, each maintained by different people.
Re: Scala success! Maarten Koopmans 3/19/13 6:55 AM
I stand in awe thinking of what Avian and Scala can do.... Quick question: if I understand this thread correctly, I should be able to compile scala packaged jars with Avian, it's the scala tests that fail when using parallel collections.

Am I correct?

--Maarten

Op zaterdag 9 maart 2013 16:44:27 UTC+1 schreef Joel Dice het volgende:
Re: Scala success! Joel Dice 3/19/13 7:36 AM
On Tue, 19 Mar 2013, Maarten Koopmans wrote:

> I stand in awe thinking of what Avian and Scala can do.... Quick question:
> if I understand this thread correctly, I should be able to compile scala
> packaged jars with Avian, it's the scala tests that fail when using parallel
> collections.
> Am I correct?

The bug that causes the parallel collections test(s) to fail
(https://groups.google.com/forum/#!msg/scala-internals/7N-7HzoH8tM/9mx4bZd4JfoJ)
could possibly cause your app to fail too, depending on if and how it
uses parallel collections.  You can fix that easily, though, by editing
scala/collection/parallel/package.scala so that TaskSupport is
unconditionally initialized as a new ForkJoinTaskSupport instance.

Also, I'm not sure what you mean by "compile scala packaged jars".  If you
mean embed your Scala app in an executable along with the contents of
scala-library.jar and any other dependencies using Avian, yes that should
work fine.  There's an example here: https://github.com/dicej/scala-avian.
Let me know if you have any trouble.
Re: Scala success! Maarten Koopmans 3/19/13 8:28 AM


Op dinsdag 19 maart 2013 15:36:56 UTC+1 schreef Joel Dice het volgende:
On Tue, 19 Mar 2013, Maarten Koopmans wrote:

> I stand in awe thinking of what Avian and Scala can do.... Quick question:
> if I understand this thread correctly, I should be able to compile scala
> packaged jars with Avian, it's the scala tests that fail when using parallel
> collections.
> Am I correct?

The bug that causes the parallel collections test(s) to fail
(https://groups.google.com/forum/#!msg/scala-internals/7N-7HzoH8tM/9mx4bZd4JfoJ)
could possibly cause your app to fail too, depending on if and how it
uses parallel collections.  You can fix that easily, though, by editing
scala/collection/parallel/package.scala so that TaskSupport is
unconditionally initialized as a new ForkJoinTaskSupport instance.

Good, I don't think I use that, but you never know what some dependencies use. 

Also, I'm not sure what you mean by "compile scala packaged jars".  If you
mean embed your Scala app in an executable along with the contents of
scala-library.jar and any other dependencies using Avian, yes that should
work fine.  There's an example here: https://github.com/dicej/scala-avian.
Let me know if you have any trouble.

That's precisely what I meant. I'll give it a go. Thanks for the great work.
 
Re: Scala success! Simon Ochsenreither 3/19/13 9:05 AM
Hi Maarten,


Good, I don't think I use that, but you never know what some dependencies use. 

A fix for this is already in Scala's pull request queue: https://github.com/scala/scala/pull/2252

It should be included in 2.9.4, 2.10.2 and 2.11.0.

Bye,

Simon
Re: Scala success! Simon Ochsenreither 4/3/13 3:08 AM
The fixes for the thread pool selection has been merged into 2.10.x and 2.11 trunk. The next releases, 2.10.2 and 2.11 will ship with this fix.

Thanks a lot to everyone involved, I'm rebuilding and testing Scala on Avian now.
Re: Scala success! Simon Ochsenreither 4/3/13 6:00 AM
Hi Joel,

looks like I found a potential Avian bug!

The test https://github.com/scala/scala/blob/master/test/files/run/blame_eye_triple_eee.scala fails with this error on Avian:

if (NaN == NaN) is broken
if (x == x) is good
if (x == NaN) is good
if (NaN != NaN) broken
if (x != x) is good
if (NaN != x) is good
x matching was good
NaN matched 0!
loop with NaN was goood


Expected results: https://github.com/scala/scala/blob/master/test/files/run/blame_eye_triple_eee.check

The Scala compiler runs with -optimize for this test. I haven't checked whether it can be reproduced without -optimize yet, but the test works on HotSpot.
The test was added very recently, so this is not something which started failing suddenly (e. g. due to a regression).

Bye,

Simon
Re: Scala success! Simon Ochsenreither 4/3/13 6:40 AM
It finished running, but the way it finished doesn't look too good:

...
[partest] Running instrumented files
[partest] testing: [...]/files/instrumented/InstrumentationTest.scala [ OK ]
[partest] testing: [...]/files/instrumented/t6611.scala [ OK ]
[partest] testing: [...]/files/instrumented/inline-in-constructors [ OK ]
[1] 8022 illegal hardware instruction (core dumped)

Apart from that almost all tests, succeeded: https://gist.github.com/soc/5301272
Re: Scala success! Joel Dice 4/3/13 3:21 PM
Yes, it's a bug; thanks for reporting it.  The only computer I have
access to right now is very slow, so I haven't tried building Scala
and running the test suite on it, but I was able to replicate the
problem in Java and fix it:

https://github.com/ReadyTalk/avian/commit/b436bd460a10fefedbfc7ec8faa2634d1308e788
Re: Scala success! Joel Dice 4/3/13 3:25 PM
On 4/3/13, Simon Ochsenreither <simon.och...@gmail.com> wrote:
Thanks for the update.  My only computer right now is an old Intel
Atom with 1GB of RAM, and I don't want to even try running the test
suite on it.  Next week I'll have a much faster machine to work with,
so I'll investigate this then.
Re: Scala success! Simon Ochsenreither 4/4/13 4:55 AM
Hi Joel,


Yes, it's a bug; thanks for reporting it.  The only computer I have
access to right now is very slow, so I haven't tried building Scala
and running the test suite on it, but I was able to replicate the
problem in Java and fix it:

I think it is almost, but not completely, fixed! I ran the tests with the following results:

  [partest] testing: [...]/files/run/blame_eye_triple_eee.scala                   [FAILED]
  [partest] if (NaN == NaN) is broken
  [partest] if (x == x) is good
  [partest] if (x == NaN) is good
  [partest] if (NaN != NaN) is good
  [partest] if (x != x) is good
  [partest] if (NaN != x) is good
  [partest] x matching was good
  [partest] NaN matching was good
  [partest] loop with NaN was goood


Bye,

Simon
Re: Scala success! Joel Dice 4/4/13 7:06 PM
Try this: https://github.com/ReadyTalk/avian/commit/8c1419fb8939cb5e0c260208c07b31b3c156b6c1
Re: Scala success! Simon Ochsenreither 4/5/13 1:50 AM
I'm on it!
Re: Scala success! Simon Ochsenreither 4/5/13 3:22 AM
Yes, this seems to fix it!

  [partest] testing: [...]/files/run/blame_eye_triple_eee-double.scala            [  OK  ]
  [partest] testing: [...]/files/run/blame_eye_triple_eee-float.scala             [  OK  ]


I have also added a float version to the Scala test suite to make sure stuff keeps working on both the Scala and the Avian side for all NaNs available.

Thanks a lot!

Simon
Re: Scala success! Joel Dice 4/9/13 6:02 PM
I just ran the full test suite using the v2.10.1 tag in the Git repo
(ant test in master didn't get very far using HotSpot or Avian).  It ran
to completion without aborting, although several tests failed.  I ran it
again with HotSpot (i.e. a stock OpenJDK 7) and it looks like the same
tests failed that way, too.  How can I tell which tests "should" fail
besides running them on OpenJDK?  Should any of them fail?
Re: Scala success! Simon Ochsenreither 4/10/13 1:24 AM
This is weird. All tests should pass, because a commit won't be merged if the test suite fails.

Can you list the tests which caused issues?
Re: Scala success! Joel Dice 4/10/13 9:05 AM
It seems to be a bit non-deterministic.  The first time I ran it, ten
tests failed (see test.txt, attached), but I just did a clean build and
test and only one failed (test2.txt).
Re: Scala success! Simon Ochsenreither 4/10/13 10:46 AM
Mhhh, interesting. The first thing I see from the logs is that the build system has updated its bootstrap libraries and recompiled the distribution. But the Git hash is the same, so this shouldn't make any difference.
I have experienced an issue with one jvm test in the past which both failed on Hotspot and Avian on the first try.

I'm currently not at my workstation where I was compiling the latest Avian against the latest Scala distribution. I'll drive home and check the results.

Btw, if you build against trunk (instead of 2.10.1) try the new ant test.suite.color ... much nicer. :-)
Re: Scala success! Simon Ochsenreither 4/10/13 11:52 AM
Ok, these are my results: https://gist.github.com/soc/5357355

As you see, no test failures, but
illegal hardware instruction (core dumped)
while testing test/files/run.


Re: Scala success! Simon Ochsenreither 4/10/13 1:41 PM
Looking at the test failure in question it seems that the issue is (depending on the exception handling semantics of the parallel collection framework) that the test expects a CompositeThrowable (which basically says "multiple exceptions occurred") but only a single exception (the one originally thrown) occurs on Avian.

So either the catching of the RuntimeException from the various worker threads doesn't work properly or the operation is aborted before another exception comes in.
Re: Scala success! Joel Dice 4/10/13 1:49 PM
On Wed, 10 Apr 2013, Simon Ochsenreither wrote:

Which failure are you refering to?  If it's one of the ones in the logs I
posted, note that both of them were run using a stock OpenJDK 7 (IcedTea),
not on Avian.  I just checked out master and ran the suite (again, on
HotSpot, not Avian) with the attached results.  I'm running it on Avian
now.
Re: Scala success! Simon Ochsenreither 4/10/13 2:13 PM

Which failure are you refering to?  If it's one of the ones in the logs I
posted, note that both of them were run using a stock OpenJDK 7 (IcedTea),
not on Avian.

I was referring to files/run/t5375.scala. Ooops, didn't realize that at all ... then it is even more weird. On OpenJDK all tests are expected to pass of course!
 
I just checked out master and ran the suite (again, on HotSpot, not Avian)
with the attached results.  I'm running it on Avian now.

Ok, this is weird ... could you post more information about the machine and the operating system?
I'm wondering whether it's a timing issue or a glitch in that specific Java version. Did you try a different Java 7 version?

Mine is

java version "1.7.0_17"
OpenJDK Runtime Environment (IcedTea7 2.3.8) (7u17-2.3.8-1ubuntu1)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)


and I'm not seeing the issues.

If the issue is not related to that, I can bring it up on the Scala mailing list. The design of the exception handling is questionable anyway.
Re: Scala success! Joel Dice 4/10/13 2:19 PM
On Wed, 10 Apr 2013, Simon Ochsenreither wrote:

>
>       Which failure are you refering to? �If it's one of the ones in
>       the logs I
>       posted, note that both of them were run using a stock OpenJDK 7
>       (IcedTea),
>       not on Avian.
>
>
> I was referring to files/run/t5375.scala. Ooops, didn't realize that at all
> ... then it is even more weird. On OpenJDK all tests are expected to pass of
> course!
> �
>       I just checked out master and ran the suite (again, on HotSpot,
>       not Avian)
>       with the attached results. �I'm running it on Avian now.
>
>
> Ok, this is weird ... could you post more information about the machine and
> the operating system?

x86_64 Debian Wheezy

  $ java -version
java version "1.7.0_03"
OpenJDK Runtime Environment (IcedTea7 2.1.7) (7u3-2.1.7-1)
OpenJDK 64-Bit Server VM (build 22.0-b10, mixed mode)
  $ cat /proc/cpuinfo
processor        : 0
vendor_id        : GenuineIntel
cpu family        : 6
model                : 58
model name        : Intel(R) Core(TM) i7-3667U CPU @ 2.00GHz
stepping        : 9
cpu MHz                : 2505.847
cache size        : 6144 KB
fpu                : yes
fpu_exception        : yes
cpuid level        : 5
wp                : yes
flags                : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc
up rep_good nopl pni monitor ssse3 lahf_lm
bogomips        : 5011.69
clflush size        : 64
cache_alignment        : 64
address sizes        : 36 bits physical, 48 bits virtual
power management:

(it's actually a dual core CPU, but the guest VM I'm running only has one
core allocated to it.  It's got 4GB of memory.)

> I'm wondering whether it's a timing issue or a glitch in that specific Java
> version. Did you try a different Java 7 version?

Not yet; I'll try that next.
Re: Scala success! Joel Dice 4/10/13 3:58 PM
On Wed, 10 Apr 2013, Joel Dice wrote:

> On Wed, 10 Apr 2013, Simon Ochsenreither wrote:
>
>> I'm wondering whether it's a timing issue or a glitch in that specific Java
>> version. Did you try a different Java 7 version?
>
> Not yet; I'll try that next.
>

Just tried with the latest OpenJDK 7 from
http://hg.openjdk.java.net/jdk7u/jdk7u-dev, and eventually ran out of
PermGen space; see attached.
Re: Scala success! Simon Ochsenreither 4/10/13 4:46 PM
Weird, can't reproduce. Can you try a larger -XX:MaxPermSize value?
But the other issue wasn't fixed anyway, so I'll bring it up on the Scala mailing list tomorrow because it doesn't seem to be dependant on the JVM version/implementation.

Thanks for testing!

Re: Scala success! Simon Ochsenreither 4/11/13 7:00 AM
I updated Avian, Scala and OpenJDK to the latest version, but I'm not getting that far to check whether I can reproduce the PermGen issue with Avian + OpenJDK class lib because of the “illegal hardware instruction” crasher.
I'm testing it now with HotSpot + OpenJDK.

Re: Scala success! Simon Ochsenreither 4/12/13 5:11 AM
Regarding files/run/t5375.scala:

After investigation, I'm pretty sure Scala is to blame here.

The exception handling behaviour is what I consider broken: If a single exception occurs, that exception is reported, but if multiple exceptions occur, they are put in a Set and wrapped in a CompositeThrowable.
The issue is that it is completely nondeterministic how many exceptions are catched before the operation is aborted. It depends on the machine, the available cores and hyper-threading, the operating system, the threadpool implementation and configuration, the size of the collection and the runtime.

Because this is so incredibly unintuitive, the creators of the test didn't expect it and forgot to add an additional clause catching the RuntimeException.
Re: Scala success! Simon Ochsenreither 4/13/13 3:18 PM
I recompiled Avian without tail calls and now it doesn't crash anymore, but there seems to be a null value at the “wrong” place:

...
[partest] ok 1342 - run/t3714.scala
[partest] ok 1343 - run/t7214.scala
[partest] ok 1344 - run/t5545.scala
[partest] ok 1345 - run/t5824.scala
 
BUILD FAILED
/home/soc/Entwicklung/avian-build/scala/build.xml:1443: null

Full log: https://gist.github.com/soc/5357355


Thanks and bye!

Simon

Re: Scala success! Simon Ochsenreither 4/13/13 5:18 PM
I just diff'ed the test output against the directory contents and the winner is, once again: run/parserJavaIdent.scala.
Something in that file really manages to kill Avian.
Re: Scala success! Simon Ochsenreither 4/14/13 3:14 PM
I executed the code of that test stand-alone and while it didn't crash, the output seems to be incorrect:

./build/linux-x86_64-tails-openjdk/avian-dynamic -cp /home/soc/Entwicklung/lib/scala-current/lib/scala-library.jar:/home/soc/Entwicklung/lib/scala-current/lib/scala-reflect.jar:/home/soc/Entwicklung/lib/scala-current/lib/scala-compiler.jar:. JavaIdent
[1.7] parsed: simple
[1.8] parsed: with123
[1.6] parsed: with$
[1.10] parsed: with?????
[1.6] parsed: with_
[1.6] parsed: _with
[1.1] failure: java identifier expected

3start
^
[1.1] failure: java identifier expected

-start
^
[1.5] failure: java identifier expected

with-s
    ^
[1.3] failure: java identifier expected

we?scala
  ^
[1.6] failure: java identifier expected

with space
     ^
soc@laptop ~/Entwicklung/avian (git)-[master] % scala JavaIdent
[1.7] parsed: simple
[1.8] parsed: with123
[1.6] parsed: with$
[1.10] parsed: withøßöèæ
[1.6] parsed: with_
[1.6] parsed: _with
[1.1] failure: java identifier expected

3start
^
[1.1] failure: java identifier expected

-start
^
[1.5] failure: java identifier expected

with-s
    ^
[1.3] failure: java identifier expected

we♥scala
  ^
[1.6] failure: java identifier expected

with space
     ^

Re: Scala success! Joel Dice 4/16/13 6:39 PM
This should fix it:

https://github.com/ReadyTalk/avian/commit/aa513c2c1dcde7fb9e329f6a7a491da544d73e42

However, the Scala test might be broken anyway, since it seems to rely on
e.g. the "♥"character being representable in the system's default
character set.  If that set is ASCII, for example, the character won't be
representable.  Perhaps the test should be changed to use UTF-8
explicitly.
Re: Scala success! Simon Ochsenreither 4/17/13 2:34 AM
Hi Joel,


This should fix it:

https://github.com/ReadyTalk/avian/commit/aa513c2c1dcde7fb9e329f6a7a491da544d73e42

thanks, I'll rebuild and test it as soon as I'm home!
 
However, the Scala test might be broken anyway, since it seems to rely on
e.g. the "♥"character being representable in the system's default
character set.  If that set is ASCII, for example, the character won't be
representable.  Perhaps the test should be changed to use UTF-8
explicitly.

Ok, that's certainly something I have to investigate. I assumed that the test-suite had set the encoding explicitly somewhere in the build.xml for all tests.

Thanks a lot!

Simon
Re: Scala success! Joel Dice 4/17/13 9:45 AM
On Wed, 17 Apr 2013, Simon Ochsenreither wrote:

> Ok, that's certainly something I have to investigate. I assumed that the
> test-suite had set the encoding explicitly somewhere in the build.xml for
> all tests.

It probably does; I think it was just broken because we were running the
test directly instead of via Ant.

BTW, I have noticed that the forked processes responsible for running each
test are not given e.g. -Xmx1536m -Xss1m arguments, which causes Avian to
run out of memory pretty quickly (it defaults to a heap limit of 128MB).
I worked around this by changing the default in the VM, but that's a hack.
Re: Scala success! Simon Ochsenreither 4/17/13 1:06 PM
It looks pretty good now:
https://gist.github.com/soc/5407293

(The issue with jvm/t7253 can be ignored, it fails on standard OpenJDK7u at my machine, too)
Re: Scala success! Joel Dice 4/17/13 2:46 PM
On Wed, 17 Apr 2013, Simon Ochsenreither wrote:

I don't think the tests are actually running on Avian based on the results
you posted.  I'm running the suite now and getting numerous failures.  The
ones I've looked into so far have been legitimate (see
https://github.com/ReadyTalk/avian/commit/81d77867160c69040b7f3b01fae1cc93459416ca).
I've attached an in-progress log (the suite is still running).

On my system, Ant forks processes using the java command in the PATH,
which may not be the same java Ant itself was launched with.  I discovered
this by using "lsof | grep libjvm", which showed me that Ant was using
Avian's libjvm.so, but the subprocesses which actually ran the tests were
using HotSpot's libjvm.so.  I suspect the same thing is happening on your
system.  You can verify that by running java -version before you run ant;
if it doesn't say Avian, it's not using Avian.  You can fix it by putting
the directory containing your Avian-based java command first in your PATH.
Re: Scala success! Joel Dice 4/17/13 7:57 PM
On Wed, 17 Apr 2013, Joel Dice wrote:

> I don't think the tests are actually running on Avian based on the results
> you posted.  I'm running the suite now and getting numerous failures.  The
> ones I've looked into so far have been legitimate (see
> https://github.com/ReadyTalk/avian/commit/81d77867160c69040b7f3b01fae1cc93459416ca).
> I've attached an in-progress log (the suite is still running).

I've attached to complete log.  It timed out before it could run all the
tests, apparently.

I'll investigate the failures as I have time.
Re: Scala success! Simon Ochsenreither 4/18/13 5:14 AM
Hi Joel,


> I don't think the tests are actually running on Avian based on the results
> you posted.  I'm running the suite now and getting numerous failures.

*sigh* ... I somehow don't manage to prevent Ant from using the wrong stuff. :-(
I guess I'll set up a VM, so I can solve this issue once and for all.
 
I've attached to complete log.  It timed out before it could run all the
tests, apparently.

I'll (temporarily?) increase the time limit. I'll report back as soon as the change is merged.
Until this happens, you can also change it yourself in src/partest/scala/tools/partest/nest/DirectRunner.scala on line 43, before you build Scala:
    try if (!pool.awaitTermination(4, TimeUnit.HOURS))
 
I'll investigate the failures as I have time.

Thanks, I will also look into it. Let me know if there are any issues at the Scala side!

Bye,

Simon

PS: The Scala developers brought in a new person recently who is tasked with working on Scala's build infrastructure, testing more platforms, moving the Jenkins config to GitHub etc. This will hopefully make it easier to integrate Avian into the testing infrastructure and to prevent regressions in Avian and Scala in the future.
Re: Scala success! Joel Dice 4/18/13 10:39 AM
On Thu, 18 Apr 2013, Simon Ochsenreither wrote:

>       I've attached to complete log.  It timed out before it could run
>       all the
>       tests, apparently.
>
>
> I'll (temporarily?) increase the time limit. I'll report back as soon as the
> change is merged.
> Until this happens, you can also change it yourself in
> src/partest/scala/tools/partest/nest/DirectRunner.scala on line 43, before
> you build Scala:
>     try if (!pool.awaitTermination(4, TimeUnit.HOURS))

Perhaps it would be better to specify a per-test time limit rather than a
total limit for the whole suite.  That way slower hardware and VMs have a
chance to complete it.  Anyway, I can just change it by hand as you
suggest; no need to merge a change upstream just for this purpose.

>       I'll investigate the failures as I have time.
>
>
> Thanks, I will also look into it. Let me know if there are any issues at the
> Scala side!

scala.actors.scheduler.ThreadPoolConfig.useForkJoin only tolerates Oracle,
Sun, and Apple, reminiscent of
https://issues.scala-lang.org/browse/SI-7236,
which causes concurrent-stream.scala to hang randomly.  The workaround, of
course, is to just set it to true.

I've fixed a few more bugs on the Avian side, and I'm re-running the suite
now.
Re: Scala success! Simon Ochsenreither 4/18/13 12:54 PM
Hi Joel,


Perhaps it would be better to specify a per-test time limit rather than a
total limit for the whole suite.  That way slower hardware and VMs have a
chance to complete it.

Will happen soon. Someone on the mailing list already offered to do it.

scala.actors.scheduler.ThreadPoolConfig.useForkJoin only tolerates Oracle,
Sun, and Apple, reminiscent of
https://issues.scala-lang.org/browse/SI-7236,
which causes concurrent-stream.scala to hang randomly.  The workaround, of
course, is to just set it to true.

I'll have a look at it. Scala actors are deprecated anyway. If Scala actors will still ship in 2.11, I'll fix it, if not, don't bother about it.

I've fixed a few more bugs on the Avian side, and I'm re-running the suite
now.

Cool! Let me know how it works out, I haven't set up the VM yet.

Thanks a lot any bye,

Simon
Re: Scala success! Simon Ochsenreither 4/18/13 1:01 PM

I've fixed a few more bugs on the Avian side, and I'm re-running the suite
now.

Just looking at the recent commits, especially https://github.com/ReadyTalk/avian/commit/9918ea6cddd01648ce09cf9db7537174955fe21b

It seems Scala is a nice test-suite for Avian. :-)
Re: Scala success! Simon Ochsenreither 4/22/13 2:45 PM
Hi Joel,

I installed the mentioned VM-with-absolutely-no-Java-except-Avian to run the tests and I'm finally seeing the issues you encountered. I can't believe that I wasted so much time not testing stuff I thought I was testing!

The pull request to add a --timeout option is already in the PR queue: https://github.com/scala/scala/pull/2418

I'll report back as soon as the option is included.

Did you find any issues on the Scala side which I can debug/triage/fix?

Bye,

Simon
Re: Scala success! Joel Dice 4/22/13 9:13 PM
On Mon, 22 Apr 2013, Simon Ochsenreither wrote:

> The pull request to add a --timeout option is already in the PR queue:
> https://github.com/scala/scala/pull/2418
>
> I'll report back as soon as the option is included.

Cool, thanks.

> Did you find any issues on the Scala side which I can debug/triage/fix?

These are the failures I'm currently seeing:

!!  99 - run/t5375.scala                           [non-zero exit code]
!! 123 - run/t6488.scala                           [non-zero exit code]
!! 702 - run/stream_length.scala                   [non-zero exit code]
!! 897 - run/t4023.scala                           [output differs]
!! 1033 - run/t5353.scala                           [output differs]
!! 1277 - run/reflection-magicsymbols-invoke.scala  [output differs]

There are probably more, but the timeout is still causing the suite to
exit prematurely, and I haven't bothered to change it yet.

   t5375: our friend CompositeThrowable, which fails consistently on both
Avian and HotSpot on my machine

   t6488: fails because scala.sys.process.ProcessImpl.SimpleProcess.destroy
uses Thread.stop, which is not implemented in Avian and probably never
will be.  Any code that uses Thread.stop is broken and needs to be fixed.

   stream_length: fails with an OOME; haven't studied it closely enough yet
to know why

   t4023: expected output depends on Class.getDeclaredClasses returning an
array sorted in a certain order.  Avian provides the same data as HotSpot
but in a different order, which appears to be valid since the JavaDoc says
nothing about the order.  I think the test ought to explicitly sort the
array if it's going to rely on a specific order.

   t5353 and reflection-magicsymbols-invoke: expected output depends on
ClassCastExceptions and CloneNotSupportedExceptions having certain message
formats, respectively.  Avian uses different formats for the messages than
HotSpot, hence the failures.  Again, this seems valid, since the messages
attached to exceptions thrown by the VM are not part of any specification
and may even change from one release to the next.
Re: Scala success! Simon Ochsenreither 4/23/13 4:49 PM
Here are my failures:

  [partest] !! 269 - run/t5353.scala                           [output differs]
  [partest] !! 270 - run/reflection-magicsymbols-invoke.scala  [output differs]
  [partest] java.lang.OutOfMemoryError
  [partest]
  [partest] !! 345 - run/inline-ex-handlers.scala              [non-zero exit code]
  [partest] !! 486 - run/repl-javap-outdir-funs                [output differs]
  [partest] java.lang.RuntimeException: calibration failure
  [partest]     at scala.sys.package$.error(package.scala:27)
  [partest]     at Test$.calibrate(tailcalls.scala:298)
  [partest]     at Test$.main(tailcalls.scala:310)
  [partest]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  [partest]     at scala.tools.nsc.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  [partest]     at scala.tools.nsc.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:68)
  [partest]     at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
  [partest]     at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:71)
  [partest]     at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:93)
  [partest]     at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:102)
  [partest] !! 525 - run/tailcalls.scala                       [non-zero exit code]
  [partest] !! 538 - run/repl-power.scala                      [non-zero exit code]
  [partest] java.lang.OutOfMemoryError
  [partest]
  [partest] !! 599 - run/t3502.scala                           [non-zero exit code]
  [partest] java.lang.OutOfMemoryError
  [partest]
  [partest] !! 653 - run/t6584.scala                           [non-zero exit code]
  [partest] java.lang.OutOfMemoryError
  [partest]
  [partest] !! 688 - run/t4294.scala                           [non-zero exit code]
  [partest] !! 711 - run/t6488.scala                           [non-zero exit code]
  [partest] !! 721 - run/shutdownhooks.scala                   [output differs]
  [partest] java.lang.RuntimeException: i div 37
  [partest]     at scala.sys.package$.error(package.scala:27)
  [partest]     at Test$$anonfun$main$1.apply$mcII$sp(t5375.scala:13)
  [partest]     at Test$$anonfun$main$1.apply(t5375.scala:13)
  [partest]     at scala.collection.parallel.immutable.ParRange$ParRangeIterator.map2combiner(ParRange.scala:104)
  [partest]     at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1058)
  [partest]     at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:54)
  [partest]     at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:53)
  [partest]     at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:62)
  [partest]     at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:56)
  [partest]     at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:155)
  [partest]     at scala.concurrent.forkjoin.RecursiveAction.exec(Unknown Source)
  [partest]     at scala.concurrent.forkjoin.ForkJoinTask.doExec(Unknown Source)
  [partest]     at scala.concurrent.forkjoin.ForkJoinTask.doJoin(Unknown Source)
  [partest]     at scala.concurrent.forkjoin.ForkJoinTask.join(Unknown Source)
  [partest]     at scala.collection.parallel.ForkJoinTasks$WrappedTask$class.sync(Tasks.scala:381)
  [partest]     at scala.collection.parallel.ForkJoinTasks$class.executeAndWaitResult(Tasks.scala:429)
  [partest]     at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:962)
  [partest]     at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:54)
  [partest]     at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:53)
  [partest]     at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:62)
  [partest]     at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:56)
  [partest]     at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:155)
  [partest]     at scala.concurrent.forkjoin.RecursiveAction.exec(Unknown Source)
  [partest]     at scala.concurrent.forkjoin.ForkJoinTask.doExec(Unknown Source)
  [partest]     at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(Unknown Source)
  [partest]     at scala.concurrent.forkjoin.ForkJoinPool.runWorker(Unknown Source)
  [partest]     at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(Unknown Source)
  [partest] !! 751 - run/t5375.scala                           [non-zero exit code]                      
  [partest] !! 757 - run/t6439.scala                           [non-zero exit code]
  [partest] java.lang.ExceptionInInitializerError
  [partest]     at Test$delayedInit$body.apply(bitsets.scala:206)
  [partest]     at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
  [partest]     at scala.App$$anonfun$main$1.apply(App.scala:71)
  [partest]     at scala.collection.immutable.List.foreach(List.scala:300)
  [partest]     at scala.App$class.main(App.scala:71)
  [partest]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  [partest]     at scala.tools.nsc.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  [partest]     at scala.tools.nsc.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:68)
  [partest]     at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
  [partest]     at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:71)
  [partest]     at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:93)
  [partest]     at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:102)
  [partest] !! 771 - run/bitsets.scala                         [non-zero exit code]


Complete log: https://gist.github.com/soc/5448382

It seems to hang after test 889, no progress for multiple hours ... it's running almost 6 hours now.
Maybe the next partest update will help. (I suggested printing the test number/file name combo *before* running the actual test, so that one can easily see on which test the hang occurs.)
Re: Scala success! Joel Dice 4/23/13 4:57 PM
On Tue, 23 Apr 2013, Simon Ochsenreither wrote:

> Complete log: https://gist.github.com/soc/5448382
>
> It seems to hang after test 889, no progress for multiple hours ... it's
> running almost 6 hours now.
> Maybe the next partest update will help. (I suggested printing the test
> number/file name combo *before* running the actual test, so that one can
> easily see on which test the hang occurs.)

I mentioned this earlier, but you may not have noticed it: PartestTask
does not pass -Xmx or -Xss flags to the java process which executes the
test, so you get Avian's default 128MB heap, which is not enough for many
of the tests, as you found.  To fix it, you can either change the default
stack and heap sizes in Avian (look for 128 in jnienv.cpp) or modify
PartestTask (or build.xml, or whatever) to specify the appropriate flags.

On my machine, it's gone through the full suite with only one new error
besides the ones I last posted: repl-javap-outdir-funs, which is odd
because it was working okay when I ran it directly.  The suite seems to be
running a second time now.  Is this expected?  Is it run twice with
different settings in each case?
Re: Scala success! Simon Ochsenreither 4/24/13 1:22 AM
Hi Joel,


I mentioned this earlier, but you may not have noticed it: PartestTask
does not pass -Xmx or -Xss flags to the java process which executes the
test, so you get Avian's default 128MB heap, which is not enough for many
of the tests, as you found.  To fix it, you can either change the default
stack and heap sizes in Avian (look for 128 in jnienv.cpp) or modify
PartestTask (or build.xml, or whatever) to specify the appropriate flags.

Alright, I read it, but didn't realize that this issue had this impact on the tests.
I'll figure out what's the suggested (and supported) way to run ant/partest with other parameters.
 
On my machine, it's gone through the full suite with only one new error
besides the ones I last posted: repl-javap-outdir-funs, which is odd
because it was working okay when I ran it directly.

It's possible that running partest directly figures out a different classpath/tools.jar or something similar (I assume that the failure is something along the lines of "couldn't find javap").
 
The suite seems to be running a second time now.  Is this expected?
Is it run twice with different settings in each case?

No, I think that's a bug in partest: https://groups.google.com/d/topic/scala-internals/j8ml3WR41Vk/discussion

Bye,

Simon
 
Re: Scala success! Simon Ochsenreither 5/2/13 7:51 AM
Hi Joel,

here is a short update:

A timeout option has been added recently. Syntax is --timeout "3 hours", translating that back to the Ant task, something like -Dpartest.timeout="3 hours" should work (no idea about the quoting, though).
Additionally, I fixed the issue with t4023.
A change for t6488 is in the pipeline, too.

I'll start a new test run and report back!

Thanks and bye,

Simon
Re: Scala success! Joel Dice 5/2/13 8:30 AM
Hi Simon,

I did a complete run of the suite on Tuesday.  Here are the failures:

!!   29 - run/t4294.scala                           [non-zero exit code]
!!   99 - run/t5375.scala                           [non-zero exit code]
!!  123 - run/t6488.scala                           [non-zero exit code]
!!  148 - run/repl-javap-outdir-funs                [output differs]
!!  705 - run/stream_length.scala                   [non-zero exit code]
!!  900 - run/t4023.scala                           [non-zero exit code]
!! 1037 - run/t5353.scala                           [output differs]
!! 1 - instrumented/InstrumentationTest.scala    [output differs]
!! 3 - instrumented/inline-in-constructors       [output differs]

  * t4294 and repl-javap-outdir-funs: I can't get these to fail when I run
them directly; they only fail in the suite, so I don't know of a practical
way to debug them

  * instrumented/*: these are failing because Avian doesn't implement
java.lang.instrument or the -javaagent option.  I'm not sure how much work
that would be, but probably not trivial, and it's not a high priority for
me.

I discussed the other failures ina a previous email.

Please let me know if you see different results from your run.
Re: Scala success! Simon Ochsenreither 5/10/13 2:44 PM
Hi Joel,

sorry for my late reply, I've been pretty busy with other Scala-related stuff.

I have recompiled Avian with the heap set to 1GB and stack set to 1 MB in jnienv.cpp, here are my results of test/files/run:

  [partest] !!   43 - run/repl-javap-def.scala                  [output differs]
  [partest] !!  167 - run/repl-javap-outdir                     [output differs]
  [partest] !!  284 - run/t5353.scala                           [output differs]
  [partest] !!  285 - run/reflection-magicsymbols-invoke.scala  [output differs]
  [partest] !!  491 - run/repl-javap-outdir-funs                [output differs]
  [partest] !!  530 - run/tailcalls.scala                       [non-zero exit code]
  [partest] !!  629 - run/repl-javap-memfun.scala               [output differs]
  [partest] !!  645 - run/t6488.scala                           [non-zero exit code]
  [partest] !!  718 - run/t4294.scala                           [non-zero exit code]
  [partest] !!  753 - run/t5375.scala                           [non-zero exit code]
  [partest] !!  795 - run/repl-javap-mem.scala                  [output differs]
  [partest] !!  802 - run/shutdownhooks.scala                   [output differs]
  [partest] !!  814 - run/repl-javap.scala                      [output differs]
  [partest] !!  875 - run/repl-javap-fun.scala                  [output differs]
  [partest] !! 1108 - run/repl-javap-more-fun.scala             [output differs]

Full version: http://pastebin.com/xg7m3mNV

  * run/tailcalls.scala


Having only glanced at the code, I think the issue is that it tries to determine the stack size by running a recursive method which cannot be optimized by scalac, but happens to be optimized by Avian.
I'll either fix the test or make just it pass on Avian.

  * run/shutdownhooks.scala


The order in which the shutdown hooks are run seems to differ ... I'm not sure if this is specified by the JVM spec at all. On the other hand, they Thread.sleep values in the various threads should be large enough to guarantee the output order. Ideas?


  * t5375: our friend CompositeThrowable, which fails consistently on both
Avian and HotSpot on my machine

I'll fix this.

 
  * t6488: fails because scala.sys.process.ProcessImpl.
SimpleProcess.destroy
uses Thread.stop, which is not implemented in Avian and probably never
will be.  Any code that uses Thread.stop is broken and needs to be fixed.

Will be fixed, too.


  * stream_length: fails with an OOME; haven't studied it closely enough yet
to know why

I'm not seeing this one.
 

  * t4023: expected output depends on Class.getDeclaredClasses returning an
array sorted in a certain order.  Avian provides the same data as HotSpot
but in a different order, which appears to be valid since the JavaDoc says
nothing about the order.  I think the test ought to explicitly sort the
array if it's going to rely on a specific order.

This is fixed.


  * t5353 and reflection-magicsymbols-invoke: expected output depends on
ClassCastExceptions and CloneNotSupportedExceptions having certain message
formats, respectively.  Avian uses different formats for the messages than
HotSpot, hence the failures.  Again, this seems valid, since the messages
attached to exceptions thrown by the VM are not part of any specification
and may even change from one release to the next.

I agree that the actual message format is not specified and I can look into changing the tests so that the differences don't trigger a test failure, but I think modifying the error messages in Avian to line up with other implementations would simplify things, especially for people new to Avian. I can create the pull requests if you agree.
 

  * t4294 and repl-javap-outdir-funs: I can't get these to fail when I run
them directly; they only fail in the suite, so I don't know of a practical
way to debug them
 
I will look into this. The fact that it tries to select the Java 6 version of javap despite running on Java 7 could be a cause.


  * instrumented/*: these are failing because Avian doesn't implement
java.lang.instrument or the -javaagent option.  I'm not sure how much work
that would be, but probably not trivial, and it's not a high priority for
me.
 
Agree, I don't care much about it too. I'd just not run these tests on Avian.
 
Thanks and bye,

Simon
Re: Scala success! Simon Ochsenreither 5/14/13 7:28 AM
Hi Joel,

status update:

  * run/tailcalls.scala
  * run/shutdownhooks.scala

Seems to work now, couldn't reproduce anymore.


* t5375: our friend CompositeThrowable, which fails consistently on both Avian and HotSpot on my machine

I'll fix this. Discussions about the best approach is still under way.


  * t6488: fails because scala.sys.process.ProcessImpl.SimpleProcess.destroy uses Thread.stop

I'll have a look what happened with the pull request.

 
  * t5353 and reflection-magicsymbols-invoke
 
  * t4294 and repl-javap-outdir-funs

I still need to investigate that.

files/run is pretty much done, files/jvm has +2 failing tests, including one which causes a core dump. I'll look into that.

Thanks and bye,

Simon
Re: Scala success! Joel Dice 5/14/13 1:39 PM
On Tue, 14 May 2013, Simon Ochsenreither wrote:

> Fix pending: https://github.com/scala/scala/pull/2528
>
>       � * run/shutdownhooks.scala
>
>
> Seems to work now, couldn't reproduce anymore.

Might have been fixed by
https://github.com/ReadyTalk/avian/commit/6507150246a936fb8f68f6a25a2ff5722a7e9451

>       � * t5353 and reflection-magicsymbols-invoke
>
>
> PR: https://github.com/ReadyTalk/avian/pull/56

Merged.  Thanks.
Re: Scala success! Simon Ochsenreither 5/18/13 5:52 PM
Hi Joel,

taking all pending pull requests into account, I'm now down to 2 test failures for pos/neg/run/jvm: t4294 and repl-javap-outdir-fun.

The issue with repl-javap-outdir-fun seems to be that the :javap command can't find the class file to inspect (I created the class directly from the REPL and then it worked). This is weird and I think it is an issue on the Scala side.

t4294 fails with an OutOfMemoryError. I already tried to run it with a larger heap/stack, but this didn't change it.
The type in question, Stream, is a lazily evaluated, memoizising singly linked list. Therefore, it can be prone to eat up all memory if improperly used (== if one holds onto the head of the list although it is not used anymore), but the usage here seems to be fine. Method collect takes a partial function which serves as a predicate and causes the Stream to evaluate until the first element which satisfies the predicate is found. But this shouldn't cause excessive memory consumption, because each evaluated element can be thrown away if it doesn't satisfy the predicate, returning a new Stream which starts with the next number.
The associated issue is interesting, too: https://issues.scala-lang.org/browse/SI-4294

What do you think? Could it be an issue with the garbage collector?

Thanks and bye,

Simon
Re: Scala success! Simon Ochsenreither 5/19/13 4:41 AM
As far as I have investigated, the implementation seems to be carefully designed to prevent both StackOverflows and OutOfMemoryErrors: https://github.com/scala/scala/blob/master/src/library/scala/collection/immutable/Stream.scala#L384

In fact, I'm running (Stream.from(1).collect{case x if x > Int.MaxValue-2 => x}: Stream[Int]) right now on HotSpot and memory usage is constant.

Weird: If I run the same expression in the Scala REPL on Avian I'm seeing an ExceptionInInitializerError instead of an OutOfMemoryError ... not sure if this is just a result of the latter:

scala> (Stream.from(1).collect{case x if x > Int.MaxValue-2 => x}: Stream[Int])
java.lang.ExceptionInInitializerError
    at .$print$lzycompute(<console>:7)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:976)
    at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:623)
    at scala.tools.nsc.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
    at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:622)
    at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:541)
    at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:574)
    at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:619)
    at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:531)
    at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:387)
    at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:404)
    at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:687)
    at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:673)
    at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:95)
    at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:673)
    at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:81)
    at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:94)
    at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)


Thanks and bye,

Simon

Re: Scala success! Joel Dice 5/20/13 10:35 AM
On Sun, 19 May 2013, Simon Ochsenreither wrote:

> As far as I have investigated, the implementation seems to be carefully
> designed to prevent both StackOverflows and OutOfMemoryErrors:https://github.com/scala/scala/blob/master/src/library/scala/collection/imm
> utable/Stream.scala#L384

Unfortunately, Avian is not currently smart enough to determine the exact
lifetime of each local/stack reference, and therefore considers the "this"
refererence to be reachable throughout the body of Stream.collect.
That's something I'd like to fix, but I don't know of an efficient way to
do it.
Re: Scala success! Simon Ochsenreither 5/20/13 12:51 PM
Hi Joel,

Unfortunately, Avian is not currently smart enough to determine the exact
lifetime of each local/stack reference, and therefore considers the "this"
reference to be reachable throughout the body of Stream.collect.
That's something I'd like to fix, but I don't know of an efficient way to
do it.

Ok, thanks for letting me know!

I'm not sure whether it is possible/viable to rewrite the collect implementation any further considering that people worked a lot on trying to avoid situations like that.

I think it is a minor issue anyway, so I probably just make this test skip on Avian for now.

Thanks and bye!

Simon
Re: Scala success! Joel Dice 5/20/13 2:42 PM
On Mon, 20 May 2013, Simon Ochsenreither wrote:

> I'm not sure whether it is possible/viable to rewrite the collect
> implementation any further considering that people worked a lot on trying to
> avoid situations like that.

Yeah, I can't think of a way to do it without changing the API and making
collect a non-member function which takes a Stream reference as an
explicit parameter.

> I think it is a minor issue anyway, so I probably just make this test skip
> on Avian for now.

Sounds good.  Meanwhile, I'll think some more about how to address this in
Avian.  The JIT compiler technically already has the information we need;
it's just not available at the point where the GC root maps for stack
frames are generated.  I've been wanting to rework that code anyway, and
this is another good reason to do it.
Re: Scala success! Simon Ochsenreither 6/5/13 3:21 PM
Hi Joel,

small status update:

I'm currently experiencing the following 5 issues:
  • run/t4023.scala: This one is failing with
    java.lang.ExceptionInInitializerError
        at Test$.main(t4023.scala:24)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
        at scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:68)
        at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
        at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:72)

        at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:94)
        at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)

    I'm pretty sure this one worked earlier .. any ideas how it could have regressed?
  • run/tailcalls.scala: Fix pending.
  • run/repl-javap-outdir-funs: This is the issue with javap, which I still need to investigate.
  • run/t3425b: This one fails because it checks for reflective calls by searching the stack for java.lang.reflect, although Avian is using a different package name. I think I'll just update the test to take Avian into account.
  • run/ctries-old-run: This one hangs (or runs very very slowly). Not sure what's the cause, will try with larger heap/stack sizes.

Any ideas regarding t4023 or ctries-old-run?

Thanks!

Simon

Re: Scala success! Simon Ochsenreither 6/5/13 3:53 PM
Good news, run/ctries-old-run just ran successfully!
Another test in jvm, jvm/t2359.scala, which caused issues earlier was tested correctly, too.
Re: Scala success! Joel Dice 6/5/13 4:06 PM
On Wed, 5 Jun 2013, Simon Ochsenreither wrote:

> Good news, run/ctries-old-run just ran successfully!

Does that mean it sometimes works and sometimes doesn't?  If so, can you
try passing e.g. -Davian.trace.port=2222 to the VM when running the test
and then getting a thread dump if/when it hangs using netcat (e.g. using
"nc localhost 2222")?

I'll take a look at t4023 when I get a chance; maybe tomorrow.
Re: Scala success! Simon Ochsenreither 6/5/13 4:14 PM

Does that mean it sometimes works and sometimes doesn't?  If so, can you
try passing e.g. -Davian.trace.port=2222 to the VM when running the test
and then getting a thread dump if/when it hangs using netcat (e.g. using
"nc localhost 2222")?

No, if I remember correctly, I was just not patient enough in the past, aborted it manually and assumed that it hung.

But why it takes so long is still a good question ...
Re: Scala success! Simon Ochsenreither 6/5/13 4:31 PM
By the way, all tests in jvm run successfully!

I have one remaining issue: I'm suffering from a lot of spurious(?) Inconsistency detected by ld.so: dl-close.c: 765: _dl_close: Assertion `map->l_init_called' failed! messages which seem to occur randomly and cause tests to fail, because the resulting test output doesn't match the expected output anymore.

Failure rate seems to be somewhere around 2.5 to 3 percent.

Ideas?
Re: Scala success! Joel Dice 6/5/13 4:46 PM
On Wed, 5 Jun 2013, Simon Ochsenreither wrote:

> I have one remaining issue: I'm suffering from a lot of spurious(?)
> Inconsistency detected by ld.so: dl-close.c: 765: _dl_close: Assertion
> `map->l_init_called' failed! messages which seem to occur randomly and cause
> tests to fail, because the resulting test output doesn't match the expected
> output anymore.
>
> Failure rate seems to be somewhere around 2.5 to 3 percent.
>
> Ideas?

I haven't seen that before.  A bit of searching suggests that it might be
due to calling dlclose too many times, although I can't see where Avian
might do that.  There's also this:
http://sourceware.org/bugzilla/show_bug.cgi?id=11941, but I don't quite
understand from the description whether it's a bug in pstoedit or glibc,
and nobody seems to done anything to fix it.  If we could reproduce it
under GDB, maybe we could get a stack trace that would tell us something.
Re: Scala success! Joel Dice 6/5/13 4:54 PM
One other thing: try removing the call to dlclose in posix.cpp and see if
the messages go away.  I wonder if the VM is trying to close its own
library (libjvm.so).  That might explain this, although one would expect
it to fail more consistently.
Re: Scala success! Simon Ochsenreither 6/5/13 5:26 PM
I just did

      //if (not mainExecutable) dlclose(p);

and the amount of errors decreased from around 4 of 121 to 0 of 121.
I only ran it once with this change, so I'm not sure if the improvement is statistically relevant, but it seems like the error message went away. :-)
Re: Scala success! Joel Dice 6/5/13 6:22 PM
On Wed, 5 Jun 2013, Simon Ochsenreither wrote:

Thanks.  Would you please try again using the attached patch?  If it looks
good, I'll commit it.

As I mentioned before, I think the problem is that when the VM is run from
libjvm.so, we call dlopen on it even though it's already loaded (or else
how would the VM be running?) because we need a handle we can call dlsym
with to look for JNI methods.  In theory, calling dlclose on that handle
when the VM exits should really be a no-op since the dlopen and dlclose
calls are balanced and it will still have been opened elsewhere, but maybe
glibc still doesn't like it for some reason.
Re: Scala success! Joshua Warner 6/5/13 6:46 PM
My understanding of dlopen and dlclose is that they keep a reference count and unload the library when that drops to zero.  Couldn't this patch cause libjvm.so to never be unloaded, in an application that dynamically loads avian and later unloads it?

-Joshua




--
You received this message because you are subscribed to the Google Groups "Avian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to avian+un...@googlegroups.com.
To post to this group, send email to av...@googlegroups.com.
Visit this group at http://groups.google.com/group/avian?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.



Re: Scala success! Joel Dice 6/5/13 6:56 PM
On Wed, 5 Jun 2013, Joshua Warner wrote:

> My understanding of dlopen and dlclose is that they keep a reference count
> and unload the library when that drops to zero. �Couldn't this patch cause
> libjvm.so to never be unloaded, in an application that dynamically loads
> avian and later unloads it?

That was my understanding, too, but I wonder what's supposed to happen if
you dlopen and dlclose a library that was already loaded automatically by
ld.so.  If ld.so doesn't like that (and clearly there's something we're
doing which it doesn't like), then what can we do to make it happy
without breaking the case you mention?

Anyway, I'll hold off committing the patch, but I'll still be interested
to know if it resolves the assertion failure.
Re: Scala success! Simon Ochsenreither 6/6/13 5:18 AM
Hi Joel, hi Joshua,


Thanks.  Would you please try again using the attached patch?  If it looks
good, I'll commit it.

I have applied the patch and started the test suite against run, jvm, res and presentation.
I'll report back!

Thanks,

Simon
Re: Scala success! Simon Ochsenreither 6/6/13 5:25 AM
I have only run 90 tests, but the error message Inconsistency detected by ld.so: dl-close.c: 765: _dl_close: Assertion `map->l_init_called' failed! started to show up again.
Re: Scala success! Joel Dice 6/6/13 6:31 AM
Okay, thanks for trying it.  I'll see if I can reproduce and debug it on
my machine.
Re: Scala success! Joel Dice 6/6/13 5:39 PM
On Wed, 5 Jun 2013, Simon Ochsenreither wrote:

>  *  run/t4023.scala: This one is failing with
>     java.lang.ExceptionInInitializerError
>     ��� at Test$.main(t4023.scala:24)
>     ��� at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     ��� atscala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoad
>     er.scala:31)
>     ��� atscala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.sca
>     la:68)
>     ��� at
>     scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
>     ��� at
>     scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:72)
>     ��� at
>     scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:94)
>     ��� at
>     scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
>     I'm pretty sure this one worked earlier .. any ideas how it could have
>     regressed?

This is fixed now:

https://github.com/ReadyTalk/avian/commit/b3cb20d6ebd750deba64eee548d4ee2fce6dadce

Thanks for reporting it.
Re: Scala success! Simon Ochsenreither 6/7/13 2:10 PM
I can confirm that it works as expected now:

# starting 7 tests in run
ok 1 - run/t4023.scala                        
!! 2 - run/repl-javap-outdir-funs                [output differs]
!! 3 - run/t5313.scala                           [output differs]
!! 4 - run/tailcalls.scala                       [non-zero exit code]
!! 5 - run/t3425b                                [output differs]
!! 6 - run/stream_length.scala                   [non-zero exit code]
!! 7 - run/t4294.scala                           [non-zero exit code]
# 1/7 passed, 6 failed in run


Thanks and bye,

Simon
Re: Scala success! Simon Ochsenreither 6/7/13 3:22 PM
Fix for t3425b is pending: https://github.com/scala/scala/pull/2642
Re: Scala success! Simon Ochsenreither 6/10/13 3:02 PM
Hi!

Fix for t3425b has been merged!

The fix for tailcalls, t5313 and some bit of infrastructure to handle differences between various runtimes (which can be used to ignore known issues on Avian) is pending here: https://github.com/scala/scala/pull/2561

Regarding the Inconsistency detected by ld.so issue, is there anything I can do to help debugging this issue? Not sure if it is important, but I'm running Avian on a Lubuntu 13.04 64bit system installed in a Virtualbox VM, the only interesting Avian compile time change is the increase in heap and stack size and activated support for tail calls.

Bye,

Simon

Re: Scala success! Joel Dice 6/10/13 3:10 PM
On Mon, 10 Jun 2013, Simon Ochsenreither wrote:

> Fix for t3425b has been merged!

Cool, thanks.

> Regarding the Inconsistency detected by ld.so issue, is there anything I can
> do to help debugging this issue? Not sure if it is important, but I'm
> running Avian on a Lubuntu 13.04 64bit system installed in a Virtualbox VM,
> the only interesting Avian compile time change is the increase in heap and
> stack size and activated support for tail calls.

I haven't looked at it yet, but my plan of attack was to add logging
before and after the dlopen and dlclose calls in posix.cpp (including the
name of the library being loaded or closed) and then look at the output
for one of the cases where the ld.so logging occurs.  If you have time to
try that, please do, and post the output.
Re: Scala success! Simon Ochsenreither 6/12/13 3:45 AM
I'll be in the hospital for a few days, so I'll probably able to try it, but I'll likely not able to report back due to no Internet. :-/
Re: Scala success! Simon Ochsenreither 6/21/13 9:29 AM
Hi Joel!

We are now officially down to 4 failures now!

# starting 4 tests in run
!! 1 - run/t6308.scala                           [output differs]

!! 2 - run/repl-javap-outdir-funs                [output differs]
!! 3 - run/stream_length.scala                   [non-zero exit code]
!! 4 - run/t4294.scala                           [non-zero exit code]


This means I'll have to debug the javap issues (t4294, repl-javap-outdir-funs), disable the failing tests involving Stream and figure out the problem with dlopen/dlclose.

After that, I think the whole test suite is green and I'll try to get Avian integrated into Scala's continouos integration/build system.

Bye,

Simon
Re: Scala success! Simon Ochsenreither 6/21/13 3:30 PM
t6308 looks interesting. It seems as if some stack frames are missing on Avian between main and the point at which the stacktrace is generated.
Any ideas?
Re: Scala success! Joel Dice 6/21/13 4:20 PM

Are you building with tails=true, and are the missing frames for methods which do tail calls?

On Jun 21, 2013 3:30 PM, "Simon Ochsenreither" <simon.och...@gmail.com> wrote:
t6308 looks interesting. It seems as if some stack frames are missing on Avian between main and the point at which the stacktrace is generated.
Any ideas?

--
You received this message because you are subscribed to the Google Groups "Avian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to avian+un...@googlegroups.com.
To post to this group, send email to av...@googlegroups.com.
Visit this group at http://groups.google.com/group/avian.

For more options, visit https://groups.google.com/groups/opt_out.
 
 
Re: Scala success! Simon Ochsenreither 6/23/13 4:04 AM

Are you building with tails=true, and are the missing frames for methods which do tail calls?

Ouch, yes, that's the case. I'm too accustomed to only look for tail-recursive code it seems ... :-)
 
Re: Scala success! Simon Ochsenreither 6/28/13 11:12 AM
Status update: One failure remaining (+ the dl_close issue)!

!! 1 - run/repl-javap-outdir-funs                [output differs]

As far as I remember, this test has also caused issues in the standard build ... I'll ping the scala-internals mailing list again about it.
Re: Scala success! Pablo Guerrero 7/5/13 3:12 AM
Hi Simon,

I'm trying to run the scala test suite on avian + android and I'm getting a null pointer exception.

Here: https://github.com/scala/scala/blob/v2.10.2/src/partest/scala/tools/partest/nest/ReflectiveRunner.scala
there is a call to new URLClassLoader(sepUrls, null)

And that intentionally throws an exception on Android.

Did you have the same problem when running on OpenJDK? Is there something I can do to avoid it?

I'm just running it directly from scala.tools.partest.nest.NestRunner.

Thanks !
Pablo
Re: Scala success! Simon Ochsenreither 7/5/13 3:25 AM
Hi Pablo,


I'm trying to run the scala test suite on avian + android and I'm getting a null pointer exception.

I haven't run it with the Android class lib yet, so I have no idea.
 
Here: https://github.com/scala/scala/blob/v2.10.2/src/partest/scala/tools/partest/nest/ReflectiveRunner.scala

I also didn't run 2.10, all tests I'm doing are on 2.11 trunk.
 
there is a call to new URLClassLoader(sepUrls, null)

And that intentionally throws an exception on Android.

What do you mean with that? Does it have a different behavior than OpenJDK? Maybe that class isn't really supported on Android...
 
Did you have the same problem when running on OpenJDK? Is there something I can do to avoid it?

No, it works fine on OpenJDK.

Sorry, that I'm not very helpful ... looks like there are plenty of possible combinations which I never tried.
After I get Avian + OpenJDK production-ready, I'll look into it. I think it is important to be able to run Avian without an Oracle-related class library.

Things are just incredibly busy right now, because I'm also trying to get as much of Paul's old branches merged.

Bye,

Simon
Re: Scala success! Pablo Guerrero 7/5/13 3:46 AM
Hi Simon,

 
there is a call to new URLClassLoader(sepUrls, null)

And that intentionally throws an exception on Android.

What do you mean with that? Does it have a different behavior than OpenJDK? Maybe that class isn't really supported on Android...

The class is supported on Android, but the behavior when you call it with null is not defined on the OpenJDK api, and they seem to do different things.   

Sorry, that I'm not very helpful ... looks like there are plenty of possible combinations which I never tried.
After I get Avian + OpenJDK production-ready, I'll look into it. I think it is important to be able to run Avian without an Oracle-related class library.

Things are just incredibly busy right now, because I'm also trying to get as much of Paul's old branches merged.

Ok, I'll try to run other kind of tests before to check if the android classpath works properly.

Right now it runs really simple scala programs, but if we want it to be useful we need to be able to run as many tests as possible, but that can wait after OpenJDK.

Cheers,
Pablo 
Re: Scala success! Joel Dice 7/5/13 1:54 PM
Hi Simon,

I just ran through a large chunk of the Scala test suite (it quit due to
the thread pool timeout, which I forgot how to change), but I never got
any dlclose errors, nor were there any "output differs" results.  Since I
can't seem to reproduce it, I think we'll need to wait until after 0.7 to
address it unless you have any new information about it.
Re: Scala success! Simon Ochsenreither 7/5/13 3:15 PM
Yes, I think it doesn't make sense to delay Avian for this issue. Sorry that I didn't manage to debug it, I got tons of additional stuff to work on. :-/
Re: Scala success! Simon Ochsenreither 7/22/13 7:28 AM
Found another regression in a recent change on Scala's side thanks to Avian. (Stringifying values by going into a endless recursion, catching a StackOverflowError and then falling back to a different approach is not the best way to go :-D)
Btw, Avian seems to throw a lot of ExceptionInIntitializerErrors where things like StackOverflowErrors occur, is that known/intended?
Re: Scala success! Joel Dice 7/29/13 5:02 PM
On Mon, 22 Jul 2013, Simon Ochsenreither wrote:

> Btw, Avian seems to throw a lot of ExceptionInIntitializerErrors where
> things like StackOverflowErrors occur, is that known/intended?

Can you provide an example?  I'm not aware of any cases where Avian
behaves differently than HotSpot in this reguard.
Re: Scala success! Simon Ochsenreither 7/29/13 7:46 PM
I'm re-running the test suite right now. I'll report back with the right test case.
Re: Scala success! Simon Ochsenreither 7/29/13 11:35 PM
Hi Joel,

I couldn't reproduce the issues right now (partest is a bit broken right now), but here is the original thread with all the pieces of information I have found: https://groups.google.com/forum/#!topic/scala-internals/gVTe8DIwC74.

Bye,

Simon
Re: Scala success! Simon Ochsenreither 8/16/13 12:20 PM
Hi everyone!

The last two test suite failures are now waiting in Scala's PR queue and I finally managed to look into the dl-close issue I'm experiencing!

In posix.cpp, I set Verbose to true and when starting up the test suite I'm now seeing output like

soc@soc-vm:~/Entwicklung/avian-build/scala$ test/partest --jvm
open (null) as 0x7fee45055268
open libjvm.so as 0x1ba7530
open /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libverify.so as 0x7fee3c012190
open /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libjava.so as 0x7fee3c012670
dlerror opening /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so: /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so: cannot open shared object file: No such file or directory
dlerror opening /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/xawt//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so: /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/xawt//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so: cannot open shared object file: No such file or directory
open /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so as 0x7fee3c0424c0
Selected 121 tests drawn from 1 named test categories

# starting 121 tests in jvm
dlerror opening /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnet.so: /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnet.so: cannot open shared object file: No such file or directory
dlerror opening /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/xawt//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnet.so: /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/xawt//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnet.so: cannot open shared object file: No such file or directory
open /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnet.so as 0x7fee2424d180
dlerror opening /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnio.so: /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnio.so: cannot open shared object file: No such file or directory
dlerror opening /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/xawt//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnio.so: /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/xawt//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnio.so: cannot open shared object file: No such file or directory
open /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libnio.so as 0x7fee240e6a20
!!   1 - jvm/actor-exceptions.scala                [output differs]
!!   2 - jvm/actor-executor.scala                  [output differs]
!!   3 - jvm/actor-executor3.scala                 [output differs]
!!   4 - jvm/actor-executor2.scala                 [output differs]
!!   5 - jvm/actor-getstate.scala                  [output differs]
...


and the test suite outputs are also filled with the same messages:

soc@soc-vm:~/Entwicklung/avian-build/scala$ cat test/files/jvm/actor-exceptions-jvm.log
OK
open (null) as 0x7ffe0f9db268
open libjvm.so as 0xf47560
open /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libverify.so as 0x7ffe080ace10
open /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libjava.so as 0x7ffe080ad2f0
dlerror opening /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so: /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so: cannot open shared object file: No such file or directory
dlerror opening /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/xawt//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so: /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/xawt//home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so: cannot open shared object file: No such file or directory
open /home/soc/Entwicklung/avian-build/jdk-avian/jre/lib/amd64/libzip.so as 0x7ffe08313800


I'm pretty sure this is closely related to the dl-close issues. I already checked where the faulty paths could come from but didn't find anything suspicious yet.

Any hints?

Thanks,

Simon
Re: Scala success! Simon Ochsenreither 8/19/13 6:49 AM
I started the test suite with LD_DEBUG=libs and it looked like this: https://gist.github.com/soc/6269270
Re: Scala success! Joshua Warner 8/21/13 8:54 PM
Simon,

Sorry I took a while in looking at this.  The output in the gist seems a little strange to me.  The last bit is another short process running (git; my guess would be as part of your prompt or something).  That shows lines like 'calling fini: <library>', but there aren't any such lines for the avian process (20940, if I don't miss my guess).  Are we not unloading any libraries (not calling dlclose), or are those not showing up for some other reason (i.e. I could be misinterpreting the name, and they're actually not related)?

As to the errors opening libraries (in your previous output post), my guess would be that that's a red herring, but I can't completely rule it out.

-Joshua


On Monday, August 19, 2013 7:49:07 AM UTC-6, Simon Ochsenreither wrote:
I started the test suite with LD_DEBUG=libs and it looked like this: https://gist.github.com/soc/6269270
Re: Scala success! Simon Ochsenreither 8/30/13 5:41 PM
It looks like the dl-close issue is some Ubuntu-specific problem/bug. I bootstrapped and tested Scala on Avian in a Fedora 19 VM today, and the errors disappeared.
I'm now trying to get Scala-on-Avian testing included in Scala's Jenkins installation: https://groups.google.com/d/topic/scala-internals/gA3syROZPLA/discussion
Re: Scala success! Simon Ochsenreither 9/8/13 4:47 PM
Hi Joel,


Can you provide an example?  I'm not aware of any cases where Avian
behaves differently than HotSpot in this reguard.

one example I found is test/files/run/repl-trim-stack-trace.scala.

On Hotspot, I get

java.lang.Exception: Uh-oh
  at .f(<console>:7)
  ... 69 elided


while on Avian I get

java.lang.ExceptionInInitializerError
  at .$print$lzycompute(<console>:7)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:969)
  at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:607)
  at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:606)
  at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:525)
  at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:748)
  at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:793)
  at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:660)
  at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:427)
  at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:444)
  at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:862)
  at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:848)
  at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:95)
  at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:848)
  at scala.tools.nsc.interpreter.ILoop$$anonfun$runForTranscript$1$$anonfun$apply$1.apply(ILoop.scala:883)
  at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
  at scala.tools.nsc.interpreter.ILoop$$anonfun$runForTranscript$1.apply(ILoop.scala:883)
  at scala.tools.nsc.interpreter.ILoop$$anonfun$runForTranscript$1.apply(ILoop.scala:882)
  at scala.tools.nsc.util.package$.stringFromStream(package.scala:63)
  at scala.tools.partest.ReplTest.eval(ReplTest.scala:39)
  at scala.tools.partest.SessionTest.show(ReplTest.scala:72)
  at scala.tools.partest.DirectTest.delayedEndpoint$scala$tools$partest$DirectTest$1(DirectTest.scala:92)
  at scala.tools.partest.DirectTest$delayedInit$body.apply(DirectTest.scala:18)
  at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
  at scala.App$$anonfun$main$1.apply(App.scala:71)
  at scala.collection.immutable.List.foreach(List.scala:304)
  at scala.App$class.main(App.scala:71)

  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  at scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:68)
  at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
  at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:72)
  at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:94)
  at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)


Any ideas?

Bye,

Simon
Re: Scala success! Simon Ochsenreither 11/12/13 9:13 AM
Any suggestions how a fix could look like? I updated both OpenJDK, Scala and Avian to trunk and the issue remains.
Re: Scala success! Joel Dice 12/5/13 8:31 AM
On Tue, 12 Nov 2013, Simon Ochsenreither wrote:

> Any suggestions how a fix could look like? I updated both OpenJDK, Scala and
> Avian to trunk and the issue remains.

Hi Simon.  I'm finally getting around to looking at this.  Can you remind
me which test in the Scala suite I need to run, and what command I should
use to run that one specifically (rather than the whole suite)?

Thanks.
Re: Scala success! Simon Ochsenreither 12/5/13 10:53 AM
Hi Joel,


Hi Simon.  I'm finally getting around to looking at this.  Can you remind
me which test in the Scala suite I need to run, and what command I should
use to run that one specifically (rather than the whole suite)?

thanks a lot. I'm always hoping that I'm not annoying you and Joshua too much with my Scala stuff. Please tell me if it gets annoying.

This is the command you need (executed from within the root folder of the Scala git repo): test/partest test/files/run/repl-trim-stack-trace.scala
This is the test in question: https://github.com/scala/scala/blob/master/test/files/run/repl-trim-stack-trace.scala

It basically tests some code which is executed in Scala's REPL.

Let me know if I can provide anything else to help!

Thanks and bye,

Simon
Re: Scala success! Joel Dice 12/5/13 10:58 AM
Thanks.  I got that far, although I had to use the v2.11.0-M7 tag rather
than master, since I had errors building master.  Now I'm trying to
convince test/partest to use the Avian libjvm.so.  Despite setting my
JAVA_HOME and PATH to point to my custom JDK, it still finds the system
JDK (i.e. /usr/lib/jvm/java-7-openjdk-amd64) and uses that instead.  Any
ideas?
Re: Scala success! Simon Ochsenreither 12/5/13 11:31 AM

Thanks.  I got that far, although I had to use the v2.11.0-M7 tag rather
than master, since I had errors building master.  Now I'm trying to
convince test/partest to use the Avian libjvm.so.  Despite setting my
JAVA_HOME and PATH to point to my custom JDK, it still finds the system
JDK (i.e. /usr/lib/jvm/java-7-openjdk-amd64) and uses that instead.  Any
ideas?

I have honestly given up trying to make Ant do what I want. I test Avian inside a VM where I have the guarantee that no other JDK even exists.
But as far as I remember, there are a few additional variables one can set for Ant. I'll see what I can find.
I'll try to reproduce you problems. How are you seeing which library it is using? (Sorry, I'm not a great Unix/Linux/Posix person :-/)
Re: Scala success! Simon Ochsenreither 12/5/13 11:35 AM
Does this help?

The wrapper scripts use the following environment variables (if set):

  • JAVACMD - full path of the Java executable. Use this to invoke a different JVM than JAVA_HOME/bin/java(.exe).
  • ...
http://ant.apache.org/manual/running.html#envvars
Re: Scala success! Joel Dice 12/5/13 12:12 PM
On Thu, 5 Dec 2013, Simon Ochsenreither wrote:

> Does this help?

No :)

But I got it working anyway, and the problem may have been be something
specific to my system.  For some reason, the test/partest script doesn't
work for me (i.e. it uses the system JDK instead of my custom one), but
when I run the ConsoleRunner class directly using the same options and
arguments, it does use my custom JDK.

Anyway, I've reproduced the problem, and I'll try to debug it now.
Re: Scala success! Simon Ochsenreither 12/5/13 1:52 PM

But I got it working anyway, and the problem may have been be something
specific to my system.  For some reason, the test/partest script doesn't
work for me (i.e. it uses the system JDK instead of my custom one), but
when I run the ConsoleRunner class directly using the same options and
arguments, it does use my custom JDK.

Ok, looks like partest doesn't share all the logic/settings with a test suite executed via ant. *sigh*


Anyway, I've reproduced the problem, and I'll try to debug it now.

Cool, thanks!
Re: Scala success! Joel Dice 12/5/13 4:02 PM
On Thu, 5 Dec 2013, Simon Ochsenreither wrote:

>       Anyway, I've reproduced the problem, and I'll try to debug it
>       now.
>
>
> Cool, thanks!

This fixes it for me:

https://github.com/dicej/avian/commit/e23037e56ee102a91ef3bc876533248afba3a44d

Can you give it a try on your side and let me know what happens?
Re: Scala success! Simon Ochsenreither 12/5/13 5:43 PM

This fixes it for me

For me too!

# starting 1 test in run
ok 1 - run/repl-trim-stack-trace.scala        

1/1 passed (elapsed time: 00:00:24)
Test Run PASSED


Thanks a lot!
Re: Scala success! Simon Ochsenreither 12/6/13 11:49 AM
I just ran the complete test.run and everything passes except:

# starting 3 tests in run
!! 1 - run/t5313.scala                           [non-zero exit code]
!! 2 - run/t6591_5.scala                         [non-zero exit code]
!! 3 - run/t6591_6.scala                         [non-zero exit code]
# 0/3 passed, 3 failed in run

I'll investigate and report back!
Re: Scala success! Simon Ochsenreither 12/6/13 12:03 PM
Hi,

both run/t6591_5.scala and run/t6591_6.scala have this line in the test:

case ex: InvocationTargetException if ex.getCause.isInstanceOf[NotImplementedError] =>

I think this could be related to the recent fix for t5313, I haven't seen t6591 in testing before.
Re: Scala success! Joel Dice 12/6/13 12:05 PM
The commit I sent you was broken anyway.  Try this one:
https://github.com/dicej/avian/commit/abe8bc6fda143101a6e5d63fbedd43503cbf61c9

Sorry for the confusion.
Re: Scala success! Simon Ochsenreither 12/6/13 8:08 PM
Hi Joel,

that change makes all three tests pass!

ok 1 - run/repl-trim-stack-trace.scala
ok 2 - run/t6591_5.scala                      
ok 3 - run/t6591_6.scala

Looks like this is the fix!

Thanks,

Simon
Re: Scala success! Pablo Guerrero 12/7/13 1:54 AM
Hi Simon,

That's great !

That this mean that all tests are passing for Scala 2.11... ?? Did you try with Scala 2.10.3?

If that's the case, I think the next step would be to try with the android classpath. I hope there are not many errors, because scala currently works on android.

I've been trying to run a simple scala/akka app using 2.10.3 (see other threads) and I'm also trying to cross compile it for windows to start trying with that too, but I didn't try to run the scala tests with the android classpath.

Cheers,
Pablo
Re: Scala success! Simon Ochsenreither 12/7/13 12:23 PM
Hi!


That this mean that all tests are passing for Scala 2.11... ?? Did you try with Scala 2.10.3?

No, t5353 is still missing. I recommend using 2.11. Most recent fixes to Scala are not backported to 2.10.x.

If that's the case, I think the next step would be to try with the android classpath. I hope there are not many errors, because scala currently works on android.

Agree, testing with the Android classpath would be nice, but I currently hope that I can get a nightly build of Scala-on-Android running on Jenkins/Travis with the help of the people of Typesafe/EPFL first. Because I'd like to get to a state where I don't have re-check after any change to Scala that it hasn't broken Scala-on-Avian again.

If you're interested, why not give it a try and report back? I'll be happy to help debugging issues you come across!

I've been trying to run a simple scala/akka app using 2.10.3 (see other threads) and I'm also trying to cross compile it for windows to start trying with that too, but I didn't try to run the scala tests with the android classpath.

Really, use 2.11. There are things which are known to be broken when running Scala 2.10.x on Avian.

Bye,

Simon
Re: Scala success! Pablo Guerrero 12/8/13 12:56 AM
Yes, Scala-on-Avian + Scala-on-Android on the nightly builds should be
a good starting point.

I'll start testing with 2.11 on avian-android, but I don't I'll run
the whole test suite yet.

Cheers,
Pablo
> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to avian+un...@googlegroups.com.
> To post to this group, send email to av...@googlegroups.com.
> Visit this group at http://groups.google.com/group/avian.
> For more options, visit https://groups.google.com/groups/opt_out.
Re: Scala success! Simon Ochsenreither 12/8/13 9:38 AM
I just added a PR against Scala to skip the unsupported instrumentation tests: https://github.com/scala/scala/pull/3238

I guess this really leaves us with only t5353 failing. :-)
Re: Scala success! Simon Ochsenreither 2/8/14 4:45 PM
Hi,

recently I'm starting to see the following issue when trying to bootstrap Scala:

...
test.bc:

test.junit.init:

test.junit.comp:

test.junit:
     [echo] Note: details of failed tests will be output to /home/soc/Entwicklung/avian-build/scala/build/junit
    [junit] Running NumericTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.432 sec
    [junit] Running scala.collection.SetMapConsistencyTest
    [junit] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 78.654 sec
    [junit] Running scala.collection.TraversableOnceTest
    [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.556 sec
    [junit] Running scala.collection.convert.MapWrapperTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.9 sec
    [junit] Running scala.collection.immutable.PagedSeqTest
    [junit] Running scala.collection.immutable.PagedSeqTest
    [junit] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec
Aborted (core dumped)


I tried to figure out the details using the core file and gdb, but it looks like I'm not really getting any useful information:

$ gdb ../jdk-avian/bin/java core
GNU gdb (GDB) 7.6.1-ubuntu
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/soc/Entwicklung/avian-build/jdk-avian/bin/java...(no debugging symbols found)...done.
[New LWP 11255]
[New LWP 11257]
[New LWP 11279]
[New LWP 11281]
[New LWP 11277]
[New LWP 11289]
[New LWP 11278]
[New LWP 11258]
[New LWP 11282]
[New LWP 11250]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/home/soc/Entwicklung/avian-build/jdk-avian/jre/bin/java -classpath /home/soc/E'.
Program terminated with signal 6, Aborted.
#0  0x00007fbe1d6c7f77 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56    ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) where
#0  0x00007fbe1d6c7f77 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007fbe1d6cb5e8 in __GI_abort () at abort.c:90
#2  0x00007fbe1cf94fec in (anonymous namespace)::handleSignal (signal=11, context=0x7fbe1ca67000) at src/vm/system/posix.cpp:1046
#3  <signal handler called>
#4  0x000000004070f622 in ?? ()
#5  0x00007fbe18eaa8b8 in ?? ()
#6  0x0000000000000003 in ?? ()
#7  0x0000000000000000 in ?? ()

Any hints? (I compiled it with make tails=true openjdk=/home/soc/Entwicklung/avian-build/jdk-avian mode=debug-fast)

Thanks a lot!

Simon
Re: Scala success! Simon Ochsenreither 3/7/14 6:38 PM
Output of (gdb) thread apply all bt full is

Thread 8 (Thread 0x7f318bfff700 (LWP 7139)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f31b1905556 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#2  0x00007f31b1954bb4 in Avian_sun_misc_Unsafe_park ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#3  0x00007f31b1922ef0 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#4  0x00000000407b407b in ?? ()
No symbol table info available.
#5  0x0000000000000080 in ?? ()
No symbol table info available.
#6  0x00000000409faa7b in ?? ()
No symbol table info available.
#7  0x00007f31756ebd70 in ?? ()
No symbol table info available.
#8  0x0000000000000000 in ?? ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.

Thread 7 (Thread 0x7f319ffff700 (LWP 7138)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f31b1905556 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#2  0x00007f31b1954bb4 in Avian_sun_misc_Unsafe_park ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#3  0x00007f31b1922ef0 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#4  0x00000000407b407b in ?? ()
No symbol table info available.
#5  0x0000000000000080 in ?? ()
No symbol table info available.
#6  0x00000000409faa7b in ?? ()
No symbol table info available.
#7  0x00007f31756ebd70 in ?? ()
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#8  0x0000000000000000 in ?? ()
No symbol table info available.

Thread 6 (Thread 0x7f319c98f700 (LWP 6957)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f31b1905556 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#2  0x00007f31b1954bb4 in Avian_sun_misc_Unsafe_park ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#3  0x00007f31b1922ef0 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#4  0x00000000407b407b in ?? ()
No symbol table info available.
#5  0x0000000000000080 in ?? ()
No symbol table info available.
#6  0x00000000409faa7b in ?? ()
No symbol table info available.
#7  0x00007f31756ebd70 in ?? ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#8  0x0000000000000000 in ?? ()
No symbol table info available.

Thread 5 (Thread 0x7f319d190700 (LWP 6956)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f31b1905556 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#2  0x00007f31b1954bb4 in Avian_sun_misc_Unsafe_park ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#3  0x00007f31b1922ef0 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#4  0x00000000407b407b in ?? ()
No symbol table info available.
#5  0x0000000000000080 in ?? ()
No symbol table info available.
#6  0x00000000409faa7b in ?? ()
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#7  0x00007f31756ebd70 in ?? ()
No symbol table info available.
#8  0x0000000000000000 in ?? ()
No symbol table info available.

Thread 4 (Thread 0x7f319d991700 (LWP 6955)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f31b1905556 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#2  0x00007f31b1954bb4 in Avian_sun_misc_Unsafe_park ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#3  0x00007f31b1922ef0 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#4  0x00000000407b407b in ?? ()
No symbol table info available.
#5  0x0000000000000080 in ?? ()
No symbol table info available.
#6  0x00000000409faa7b in ?? ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#7  0x00007f31756ebd70 in ?? ()
No symbol table info available.
#8  0x0000000000000000 in ?? ()
No symbol table info available.

Thread 3 (Thread 0x7f31a4f50700 (LWP 6934)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f31b194ace1 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#2  0x00007f31b18ffd79 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#3  0x00007f31b1960d82 in detectFeature ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#4  0x00007f31b18eeb2a in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#5  0x00007f31b1924d44 in ?? ()
---Type <return> to continue, or q <return> to quit---
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#6  0x00007f31b19487da in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#7  0x00007f31b0de5f33 in start_thread (arg=0x7f31a4f50700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f31a4f50700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139851197646592, 6398891759236634736,
                0, 0, 139851197647296, 139851197646592, -6436163047332947856,
                -6436189662217021328}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0,
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
#8  0x00007f31b160cded in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 2 (Thread 0x7f31acc0b700 (LWP 6933)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
---Type <return> to continue, or q <return> to quit---
No locals.
#1  0x00007f31b1905556 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#2  0x00007f31b1940acf in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#3  0x00007f31b1960d82 in detectFeature ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#4  0x00007f31b18eeb2a in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#5  0x00007f31b193fd5c in JVM_MonitorWait ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#6  0x00007f31b1960d20 in detectFeature ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#7  0x00007f31b191df6a in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#8  0x00007f31b1922df5 in ?? ()
---Type <return> to continue, or q <return> to quit---
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#9  0x00000000407b407b in ?? ()
No symbol table info available.
#10 0x0000000000000000 in ?? ()
No symbol table info available.

Thread 1 (Thread 0x7f31b1d90740 (LWP 6932)):
#0  0x0000000040947428 in ?? ()
No symbol table info available.
#1  0x00007f3179f6ae00 in ?? ()
No symbol table info available.
#2  0x0000000000ae0b00 in ?? ()
No symbol table info available.
#3  0x000000000100fde8 in ?? ()
No symbol table info available.
#4  0x000000000100fde8 in ?? ()
No symbol table info available.
#5  0x00007f3179fd4388 in ?? ()
No symbol table info available.
#6  0x00007f3179ffe380 in ?? ()
No symbol table info available.
#7  0x00007f3175f7e058 in ?? ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#8  0x00000000408f35fd in ?? ()
No symbol table info available.
#9  0x00007f3179ffe1c0 in ?? ()
No symbol table info available.
#10 0x00007ffff5817f48 in ?? ()
No symbol table info available.
#11 0x00007f31b1b885d0 in ?? ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.
#12 0x0000000000ae0890 in ?? ()
No symbol table info available.
#13 0x00000000f5817f70 in ?? ()
No symbol table info available.
#14 0x00007f31b1960dea in detectFeature ()
   from /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-tails-openjdk/libjvm.so
No symbol table info available.

Re: Scala success! Simon Ochsenreither 3/7/14 6:50 PM
Running it with valgrind crashes almost immediately (otherwise it takes quite a while and seems to work until it reaches test.run):

$ valgrind /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-debug-fast-tails-openjdk/avian-dynamic -Xmx2048m -Xss1m -classpath /home/soc/Entwicklung/avian-build/apache-ant-1.9.2/lib/ant-launcher.jar -Dant.home=/home/soc/Entwicklung/avian-build/apache-ant-1.9.2 -Dant.library.dir=/home/soc/Entwicklung/avian-build/apache-ant-1.9.2/lib org.apache.tools.ant.launch.Launcher test.run
==7408== Memcheck, a memory error detector
==7408== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==7408== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==7408== Command: /home/soc/Entwicklung/avian-build/avian/build/linux-x86_64-debug-fast-tails-openjdk/avian-dynamic -Xmx2048m -Xss1m -classpath /home/soc/Entwicklung/avian-build/apache-ant-1.9.2/lib/ant-launcher.jar -Dant.home=/home/soc/Entwicklung/avian-build/apache-ant-1.9.2 -Dant.library.dir=/home/soc/Entwicklung/avian-build/apache-ant-1.9.2/lib org.apache.tools.ant.launch.Launcher test.run
==7408==
==7408== Invalid write of size 8
==7408==    at 0x4C2B473: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7408==    by 0x4ED9388: avian::codegen::x86::MyAssembler::write() (assembler.cpp:1102)
==7408==    by 0x4E8D803: (anonymous namespace)::local::finish((anonymous namespace)::local::MyThread*, avian::util::FixedAllocator*, avian::codegen::Assembler*, char const*, unsigned int) (compile.cpp:6663)
==7408==    by 0x4E98CC3: (anonymous namespace)::local::compileThunks((anonymous namespace)::local::MyThread*, avian::util::FixedAllocator*) (compile.cpp:9887)
==7408==    by 0x4E962C2: (anonymous namespace)::local::MyProcessor::boot(vm::Thread*, vm::BootImage*, unsigned char*) (compile.cpp:9125)
==7408==    by 0x4E5178F: (anonymous namespace)::boot(vm::Thread*) (machine.cpp:2823)
==7408==    by 0x4E54350: vm::Thread::init() (machine.cpp:3265)
==7408==    by 0x4E9443F: (anonymous namespace)::local::MyProcessor::makeThread(vm::Machine*, vm::Object*, vm::Thread*) (compile.cpp:8629)
==7408==    by 0x4EC2380: JNI_CreateJavaVM (jnienv.cpp:3939)
==7408==    by 0x40144F: main (main.cpp:259)
==7408==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==7408==
==7408==
==7408== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==7408==  Access not within mapped region at address 0x0
==7408==    at 0x4C2B473: memcpy@@GLIBC_2.14 (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7408==    by 0x4ED9388: avian::codegen::x86::MyAssembler::write() (assembler.cpp:1102)
==7408==    by 0x4E8D803: (anonymous namespace)::local::finish((anonymous namespace)::local::MyThread*, avian::util::FixedAllocator*, avian::codegen::Assembler*, char const*, unsigned int) (compile.cpp:6663)
==7408==    by 0x4E98CC3: (anonymous namespace)::local::compileThunks((anonymous namespace)::local::MyThread*, avian::util::FixedAllocator*) (compile.cpp:9887)
==7408==    by 0x4E962C2: (anonymous namespace)::local::MyProcessor::boot(vm::Thread*, vm::BootImage*, unsigned char*) (compile.cpp:9125)
==7408==    by 0x4E5178F: (anonymous namespace)::boot(vm::Thread*) (machine.cpp:2823)
==7408==    by 0x4E54350: vm::Thread::init() (machine.cpp:3265)
==7408==    by 0x4E9443F: (anonymous namespace)::local::MyProcessor::makeThread(vm::Machine*, vm::Object*, vm::Thread*) (compile.cpp:8629)
==7408==    by 0x4EC2380: JNI_CreateJavaVM (jnienv.cpp:3939)
==7408==    by 0x40144F: main (main.cpp:259)
==7408==  If you believe this happened as a result of a stack
==7408==  overflow in your program's main thread (unlikely but
==7408==  possible), you can try to increase the size of the
==7408==  main thread stack using the --main-stacksize= flag.
==7408==  The main thread stack size used in this run was 8388608.
==7408==
==7408== HEAP SUMMARY:
==7408==     in use at exit: 1,233,980 bytes in 84 blocks
==7408==   total heap usage: 138 allocs, 54 frees, 2,333,596 bytes allocated
==7408==
==7408== LEAK SUMMARY:
==7408==    definitely lost: 0 bytes in 0 blocks
==7408==    indirectly lost: 0 bytes in 0 blocks
==7408==      possibly lost: 0 bytes in 0 blocks
==7408==    still reachable: 1,233,980 bytes in 84 blocks
==7408==         suppressed: 0 bytes in 0 blocks
==7408== Rerun with --leak-check=full to see details of leaked memory
==7408==
==7408== For counts of detected and suppressed errors, rerun with: -v
==7408== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)
Segmentation fault

Re: Scala success! Simon Ochsenreither 3/11/14 4:56 PM
Hi Joel, hi Joshua,

is there anything I can do to debug/fix this problem?

Thanks and bye,

Simon
Re: Scala success! Joel Dice 3/11/14 5:28 PM
Hi Simon,

Sorry I've been so unresponsive.  Every time I see an email about this
I kick myself for forgetting to follow up on one of the previous ones,
and then I forget again.  Anyway, I'll try to look at it this weekend,
if not sooner.
> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to avian+un...@googlegroups.com.
> To post to this group, send email to av...@googlegroups.com.
> Visit this group at http://groups.google.com/group/avian.
> For more options, visit https://groups.google.com/d/optout.
>
Re: Scala success! Joshua Warner 3/11/14 6:02 PM
Simon,

I have to apologize as well - I looked at this, thought of suggesting the following, and then thought, "Joel will probably beat me to it..." :)

The first thing I would do is hook up a debugger and figure out exactly what is null there.  You can get valgrind to drop into a debugger with --db-attach=yes.  Glancing at the stack trace and the code, I don't see anything that would be obviously null there.

-Joshua
Re: Scala success! Simon Ochsenreither 3/11/14 7:14 PM
Weird ... without tails=true, I get this far:

test.junit.init:

test.junit.comp:
    [mkdir] Created dir: /home/soc/Entwicklung/avian-build/scala/build/junit/classes
[scalacfork] Compiling 34 files to /home/soc/Entwicklung/avian-build/scala/build/junit/classes
[scalacfork] warning: there were 7 deprecation warning(s); re-run with -deprecation for details
[scalacfork] one warning found
[stopwatch] [test.junit.compiler.timer: 1:00.120 sec]


test.junit:
     [echo] Note: details of failed tests will be output to /home/soc/Entwicklung/avian-build/scala/build/junit
    [junit] Running NumericTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.807 sec
    [junit] Running scala.collection.SetMapConsistencyTest
    [junit] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.36 sec
    [junit] Running scala.collection.TraversableOnceTest
    [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.888 sec
    [junit] Running scala.collection.convert.MapWrapperTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.581 sec
    [junit] Running scala.collection.immutable.PagedSeqTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.914 sec
    [junit] Running scala.collection.immutable.QueueTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.804 sec
    [junit] Running scala.collection.immutable.RangeConsistencyTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.52 sec
    [junit] Running scala.collection.mutable.ArraySortingTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.957 sec
    [junit] Running scala.collection.mutable.PriorityQueueTest
    [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.152 sec
    [junit] Running scala.collection.mutable.VectorTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.188 sec
    [junit] Running scala.concurrent.impl.DefaultPromiseTest
    [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.71 sec
    [junit] Running scala.math.BigDecimalTest
    [junit] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.759 sec
    [junit] Running scala.reflect.internal.util.SourceFileTest
    [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.017 sec
    [junit] Running scala.reflect.internal.util.StringOpsTest
    [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.916 sec
    [junit] Running scala.reflect.internal.util.WeakHashSetTest
    [junit] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.134 sec
    [junit] Running scala.reflect.io.ZipArchiveTest
    [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.493 sec
    [junit] Running scala.runtime.ScalaRunTimeTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.538 sec
    [junit] Running scala.tools.nsc.interpreter.TabulatorTest
    [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.381 sec
    [junit] Running scala.tools.nsc.settings.SettingsTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.376 sec
    [junit] Running scala.tools.nsc.symtab.CannotHaveAttrsTest
    [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.972 sec
    [junit] Running scala.tools.nsc.symtab.FreshNameExtractorTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.56 sec
    [junit] Running scala.tools.nsc.symtab.StdNamesTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.583 sec
    [junit] Running scala.tools.nsc.symtab.SymbolTableTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.16 sec
    [junit] Running scala.tools.nsc.test.SampleTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.813 sec
    [junit] Running scala.tools.nsc.util.StackTraceTest
    [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.361 sec
    [junit] Running scala.tools.testing.AssertThrowsTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.875 sec
    [junit] Running scala.util.TryTest
    [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.89 sec
    [junit] Running scala.util.matching.CharRegexTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.905 sec
    [junit] Running scala.util.matching.RegexTest
    [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.934 sec
    [junit] Running scala.util.test.SpecVersionTest
    [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.873 sec
[stopwatch] [test.junit.timer: 2:23.614 sec]

partest.task:

test.suite.init:
[propertyfile] Creating new property file: /home/soc/Entwicklung/avian-build/scala/build/pack/partest.properties

test.interactive:

BUILD FAILED
/home/soc/Entwicklung/avian-build/scala/build.xml:1492: The following error occurred while executing this line:
/home/soc/Entwicklung/avian-build/scala/build-ant-macros.xml:723: java.lang.NullPointerException
    at scala.tools.partest.nest.FileManager.findArtifact(FileManager.scala:125)
    at scala.tools.partest.nest.FileManager.compilerUnderTest$lzycompute(FileManager.scala:106)
    at scala.tools.partest.nest.FileManager.compilerUnderTest(FileManager.scala:106)
    at scala.tools.partest.nest.SuiteRunner.banner(Runner.scala:762)
    at scala.tools.partest.nest.AntRunner.execute(AntRunner.scala:48)
    at scala.tools.partest.PartestTask.execute(PartestTask.scala:151)
    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
    at org.apache.tools.ant.Task.perform(Task.java:348)
    at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:68)
    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
    at org.apache.tools.ant.Task.perform(Task.java:348)
    at org.apache.tools.ant.taskdefs.MacroInstance.execute(MacroInstance.java:396)
    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
    at org.apache.tools.ant.Task.perform(Task.java:348)
    at org.apache.tools.ant.Target.execute(Target.java:435)
    at org.apache.tools.ant.Target.performTasks(Target.java:456)
    at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
    at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
    at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
    at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
    at org.apache.tools.ant.Main.runBuild(Main.java:851)
    at org.apache.tools.ant.Main.startAnt(Main.java:235)
    at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
    at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)

Total time: 9 minutes 44 seconds

Re: Scala success! Simon Ochsenreither 3/12/14 5:11 PM
Hey Joel, hey Joshua,

absolutely no need to apologize!

I tried running with valgrind, but I think the almost immediate error is something where valgrind gets confused probably due to this: http://valgrind.org/docs/manual/manual-core.html#manual-core.signals I tried both options, but without access.
So I'm not even getting to the "real" error, as far as I understand.

Bye,

Simon
Re: Scala success! Joshua Warner 3/12/14 5:37 PM
The link you pointed to doesn't make me think that valgrind is erroneously generating that signal - it just says that it may not handle some signal throwing patterns well.  And I can tell you, we're not supposed to be getting a segfault there.  We do use SIGUSR1 and SIGUSR2 for various threading operations, and I suppose valgrind could be misinterpreting those, but I'm not so sure.  It still could be some random bug in the valgrind translation or instrumentation process.

Are you still at a point where you can trap the eventual ("real") failure in a debugger?  If so, you can get a more informative stack trace using vmPrintTrace($rbx) when you're in a jitted frame.  Also, a bootimage build includes native symbols for all aot-compiled methods (just classpath classes, unless you expand that by hacking the makefile).

-Joshua


--
You received this message because you are subscribed to the Google Groups "Avian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to avian+un...@googlegroups.com.
To post to this group, send email to av...@googlegroups.com.
Visit this group at http://groups.google.com/group/avian.
For more options, visit https://groups.google.com/d/optout.

Re: Scala success! Simon Ochsenreither 3/12/14 5:52 PM


The link you pointed to doesn't make me think that valgrind is erroneously generating that signal - it just says that it may not handle some signal throwing patterns well.  And I can tell you, we're not supposed to be getting a segfault there.  We do use SIGUSR1 and SIGUSR2 for various threading operations, and I suppose valgrind could be misinterpreting those, but I'm not so sure.  It still could be some random bug in the valgrind translation or instrumentation process.

Ah, ok. I assumed it was related to some VM implementation technique where additional stores are emitted in methods/loops so that the runtime can stop the execution of mutator threads if necessary.
 
Are you still at a point where you can trap the eventual ("real") failure in a debugger?  If so, you can get a more informative stack trace using vmPrintTrace($rbx) when you're in a jitted frame.  Also, a bootimage build includes native symbols for all aot-compiled methods (just classpath classes, unless you expand that by hacking the makefile).

Ok, I'll try to both debug this segfault and figure out whether the later NPE crash is an issue caused by Scala or Avian.
It's kind of confusing for me that things fail in three different ways depending on whether tail calls are enabled and valgrind is attached ...
Re: Scala success! Joshua Warner 3/12/14 7:44 PM

Ah, ok. I assumed it was related to some VM implementation technique where additional stores are emitted in methods/loops so that the runtime can stop the execution of mutator threads if necessary.

Something I'd like to do at some point :).  There's even a github issue for it (https://github.com/ReadyTalk/avian/issues/136).

Re: Scala success! Joel Dice 3/14/14 9:20 AM
On Tue, 11 Mar 2014, Simon Ochsenreither wrote:

> is there anything I can do to debug/fix this problem?

There were four things going wrong:

  * Stack trace creation and stack unwinding was not working properly for
StackOverflowErrors in tails=true builds:
https://github.com/dicej/avian/commit/918b7828f1fba1af977325cf2a558d1b5f1ba3be

  * The signal handling code was going into an infinite loop if no Java
frame was found for the signal:
https://github.com/dicej/avian/commit/7cfbdc8fdba9561bb0145fa8f685733811f46f97

  * Stack unwinding failed for certain try/finally code in tails=true
builds:
https://github.com/dicej/avian/commit/73e60adeab8e61d7027dedd460723ff44b8d42bd

  * StackOverflow errors were being thrown in Scala's PagedSeqTest because
the default Avian maximum stack size of 128KiB was not enough due to deep
recursion in PagedSeq.  Changing the Avian default to 1MiB "fixes" it, but
I'm not going to check that in.  I suggest passing an -Xss1m flag to the
JUnit subprocess if you need this to work.
Re: Scala success! Simon Ochsenreither 3/16/14 10:55 PM
Wow, this is incredibly impressive! Thanks Joel!

I wasn't in reach of a workstation to test Scala/Avian for the last few days, but I'll test this immediately now and report back!

Thanks a lot, I'm sure I wouldn't have figured out these issues on my own.

Bye,

Simon
Re: Scala success! Simon Ochsenreither 3/17/14 1:00 AM
Update: Things seem to work, although very slowly, but I haven't seen any issue yet! I'll keep you updated.
Re: Scala success! Simon Ochsenreither 3/17/14 2:14 AM
One core dump:

  [partest] Running 4 tests in 'scalacheck' at Mon Mar 17 09:15:42 CET 2014
Aborted (core dumped)


Apart from that things haven't finished, but look fine.

I'll try to look into the core dump when all tests have run and report back.
Re: Scala success! Simon Ochsenreither 3/17/14 5:38 AM
There seems to be a new test failure in run/t4929.scala:

success
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)
java/lang/InterruptedException
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.doAcquireSharedNanos (line 1038)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.tryAcquireSharedNanos (line 1326)
  at Test$$anonfun$1$$anon$1.run (line 20)


I'll look at it later, need to go now, my part of the city is currently being evacuated because a WWII bomb was discovered on a construction site nearby which needs to be defused/detonated. I hope I'll be back later again.

Bye,

Simon

Re: Scala success! Simon Ochsenreither 3/17/14 12:45 PM
Ok, after running partest, this is the status:
  • The categories pos, neg, jvm and scalacheck are all green.
  • Two (expected) failures in instrumentation.
  • A new issue in run: t4929.scala
  • A known issue in run: run/repl-javap-outdir-funs

Re: Scala success! Joel Dice 3/17/14 12:46 PM
On Mon, 17 Mar 2014, Simon Ochsenreither wrote:

> Ok, after running partest, this is the status:
>
>   - The categories pos, neg, jvm and scalacheck are all green.
>   - Two (expected) failures in instrumentation.
>   - A new issue in run: t4929.scala
>   - A known issue in run: run/repl-javap-outdir-funs

Thanks for the update.  Can you remind me how to run t4929.scala
specifically without running the whole suite?
Re: Scala success! Simon Ochsenreither 3/17/14 12:58 PM

Thanks for the update.  Can you remind me how to run t4929.scala
specifically without running the whole suite?

You can do

test/partest test/files/run/t4929.scala

(If the test runner complains that it can't find some file, you probably need to compile Scala first (with ant build)).
Re: Scala success! Joel Dice 3/17/14 3:47 PM
On Mon, 17 Mar 2014, Simon Ochsenreither wrote:

>
>
I don't have a t4929.scala in my clone of git@github.com:scala/scala.  Is
it in some specific branch?
Re: Scala success! Simon Ochsenreither 3/17/14 11:24 PM
Weird, looks like a stale file. The file was supposed to be moved when parser-combinators were split from the main library: https://github.com/scala/scala-parser-combinators/blob/master/src/test/scala/scala/util/parsing/combinator/t4929.scala
Re: Scala success! Joel Dice 3/18/14 7:13 AM
On Mon, 17 Mar 2014, Simon Ochsenreither wrote:

Thanks.  How would one run that test, starting from freshly-cloned copies
of the scala and scala-parser-combinators repositories?
Re: Scala success! Simon Ochsenreither 3/18/14 8:16 AM

Thanks.  How would one run that test, starting from freshly-cloned copies
of the scala and scala-parser-combinators repositories?

Mhhh, I would have probably just copied the files in question from an older revision to test/files/run again: https://github.com/scala/scala/blob/2.10.x/test/files/run/t4929.scala and https://github.com/scala/scala/blob/2.10.x/test/files/run/t4929.check.
Re: Scala success! Joel Dice 3/18/14 8:46 AM
Thanks.  For me, the test fails with HotSpot/OpenJDK as well as with
Avian/OpenJDK, and in the exact same way:

  $ cat /home/dicej/trash/p/scala/test/files/run/t4929-run.log
warning: there were 1 deprecation warning(s); re-run with -deprecation for
details
success
Re: Scala success! Simon Ochsenreither 3/18/14 12:48 PM

$ cat /home/dicej/trash/p/scala/test/files/run/t4929-run.log
warning: there were 1 deprecation warning(s); re-run with -deprecation for
details
success

Weird. The difference due to the deprecation warning is not important.
Re: Scala success! Joel Dice 3/18/14 12:51 PM
On Tue, 18 Mar 2014, Simon Ochsenreither wrote:

Are you sure?  Won't the test fail if there is any difference between
t4929-run.log and t4929.check?

Is the test passing for you when using a stock OpenJDK (i.e. no Avian)?
Re: Scala success! Simon Ochsenreither 3/18/14 6:31 PM

Are you sure?  Won't the test fail if there is any difference between
t4929-run.log and t4929.check?

Yes, sorry, I was imprecise. I wanted to say that it's not a meaningful difference (and especially not the one I saw which made it fail) ...

Is the test passing for you when using a stock OpenJDK (i.e. no Avian)?

Yes.

I'll investigate later again.
Re: Scala success! Simon Ochsenreither 4/25/14 8:16 AM
I'm running the test suite right now again, and it seems we have a new issue.

The test fails for test/files/neg/macro-invalidret because on Avian the stacktrace emits one line less.

https://github.com/scala/scala/blob/master/test/files/neg/macro-invalidret.check#L22 expects:
scala.NotImplementedError: an implementation is missing
at scala.Predef$.$qmark$qmark$qmark(Predef.scala:225)
at Impls$.foo3(Impls_1.scala:7)

But on Avian, we only get:

scala.NotImplementedError: an implementation is missing
at scala.Predef$.$qmark$qmark$qmark(Predef.scala:225)

Seems to be a recent change ...
Re: Scala success! Simon Ochsenreither 4/25/14 8:18 AM
There is also a new issue in the junit tests:

test.junit:
     [echo] Note: details of failed tests will be output to /home/soc/Entwicklung/avian-build/scala/build/junit
    [junit] Running NumericTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.025 sec
    [junit] Running scala.collection.SetMapConsistencyTest
    [junit] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.254 sec
    [junit] Running scala.collection.TraversableOnceTest
    [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.111 sec
    [junit] Running scala.collection.convert.MapWrapperTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.716 sec
    [junit] Running scala.collection.immutable.PagedSeqTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.19 sec
    [junit] Running scala.collection.immutable.QueueTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.977 sec
    [junit] Running scala.collection.immutable.RangeConsistencyTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.522 sec
    [junit] Running scala.collection.mutable.ArraySortingTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.212 sec
    [junit] Running scala.collection.mutable.PriorityQueueTest
    [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.429 sec
    [junit] Running scala.collection.mutable.VectorTest
    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.463 sec
    [junit] Running scala.concurrent.impl.DefaultPromiseTest
    [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.741 sec
    [junit] Running scala.math.BigDecimalTest
    [junit] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.921 sec
    [junit] Running scala.reflect.internal.MirrorsTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.107 sec
    [junit] Running scala.reflect.internal.PrintersTest
    [junit] Tests run: 178, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 38.91 sec

BUILD FAILED
/home/soc/Entwicklung/avian-build/scala/build.xml:1438: Test scala.reflect.internal.PrintersTest failed


I haven't looked into that yet, I'll keep you posted.
Re: Scala success! Joel Dice 4/29/14 12:27 PM
Hi Simon,

Can you give me steps to reproduce the two problems you recently reported?
I often procrastinate when it comes to looking at these because it's hard
to remember the build process and I'm not sure which version of Scala to
use.

That's why I love reports like this:
https://github.com/ReadyTalk/avian/issues/197.  Everything I need to know
is right there, and I can reproduce it in just a few minutes.

Thanks!
Re: Scala success! Joshua Warner 4/29/14 12:43 PM
If I might suggest, Simon:

A good way to do this might be to build a Dockerfile (and corresponding uploaded image) based on joshuawarner32/avian-build (see the Dockerfile in the root of the repository) - for building scala + avian.  That way, it'll be really easy to get up and running with scala and avian.

-Joshua



--
You received this message because you are subscribed to the Google Groups "Avian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to avian+un...@googlegroups.com.
To post to this group, send email to av...@googlegroups.com.
Visit this group at http://groups.google.com/group/avian.
For more options, visit https://groups.google.com/d/optout.

Re: Scala success! Simon Ochsenreither 4/29/14 2:58 PM
Thanks, I'll look into it!
More topics »