Error with Akka library using openjdk

[sir...@gmail.com]

Hi all,

I've discovered Avian a few days ago, and I think it's a great project, so first of all, I want to congratulate you for your work.

I'm trying to use it in a  medium-big Scala application using Netty, Akka (http://akka.io) and OrientDB with the embedded openjdk. So far, Scala and Netty are working fine, and the really basic stuff of Akka too.

But I've starting to have some problems with Akka that make the application crash without any error output. I've done some debugging, and I suspect that it's related with the thread pools. Akka is an actor library, and the actors are executed using a pool of threads, and when and I think that when an actor is unallocated from the thread, it crashes. I also suspect of the native method sun.misc.Unsafe.park() from some debug I've done with VisualVM, but I'm not sure.

So, I've created a simple test to reproduce the error (in Java for simplicity, but Akka uses scala internally). It's not the only way of reproduce it, but the easiest one I've found.

import akka.actor.ActorRef;

import akka.actor.ActorSystem;

import akka.actor.Props;

import akka.actor.UntypedActor;

import akka.routing.RoundRobinRouter;

public class Test {

    public static class HelloActor extends UntypedActor {

        public void onReceive(Object message) throws Exception {

            System.out.println("Hello World");

        }

    }

    public static void main(String[] args) {

        ActorSystem system = ActorSystem.create("Test");

        ActorRef hello = system.actorOf(new Props(HelloActor.class).withRouter(new RoundRobinRouter(2)));      

        hello.tell("something");

    }

}

That crashes without error, but if you remove .withRouter(new RoundRobinRouter(2)), it works. The router forces the creation of 2 actors, and one is automatically unallocated, so the problem arrises.

To easily compile it, this two libraries are needed.

http://repo.typesafe.com/typesafe/releases/com/typesafe/akka/akka-actor/2.0/akka-actor-2.0.jar

http://repo.typesafe.com/typesafe/releases/org/scala-lang/scala-library/2.9.1-1/scala-library-2.9.1-1.jar 

Then

$ javac -cp akka-actor-2.0.jar:scala-library-2.9.1-1.jar Test.java

$ java -cp .:akka-actor-2.0.jar:scala-library-2.9.1-1.jar Test

Hello World

(and the actors keep running, kill with ctrl-c)

$ avian -cp .:akka-actor-2.0.jar:scala-library-2.9.1-1.jar Test

(crashes without output)

Thanks in advance,

Pablo

[Joel Dice]

Hi Pablo,

Thanks for the report and the concise test case. I had to fix several
bugs and missing features in Avian to get the test working. Please pull
the latest from the Git repo and let me know how it works for you.

By the way, Akka likes to silently call System.exit if it catches an error
by default, which makes debugging a challenge. If you add a file named
application.conf with the following content to the root of your classpath,
you can override that behavior:

akka.loglevel = DEBUG
akka.stdout-loglevel = DEBUG
akka.jvm-exit-on-fatal-error = off

I hope that helps.

> .1-1/scala-library-2.9.1-1.jar�

>
> Then
>
> $ javac -cp akka-actor-2.0.jar:scala-library-2.9.1-1.jar�Test.java
>
> $ java -cp .:akka-actor-2.0.jar:scala-library-2.9.1-1.jar�Test
> Hello World
> (and the actors keep running, kill with ctrl-c)
>
> $ avian -cp .:akka-actor-2.0.jar:scala-library-2.9.1-1.jar�Test
> (crashes without output)
>
> Thanks in advance,
> Pablo
>

> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/avian/-/ruxYtP_7zMsJ.
> To post to this group, send email to av...@googlegroups.com.
> To unsubscribe from this group, send email to
> avian+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/avian?hl=en.
>
>

[Pablo Guerrero]

Hi Joel,

Thanks a lot for you quick answer. I'll get the latest version and I'll try again.

And thanks too for the akka tip, I was not aware of this, and sure it will help me debugging. 

I'll let you know how it goes.

Cheers,

Pablo

On Sunday, March 11, 2012 7:05:24 PM UTC+1, Joel Dice wrote:

Hi Pablo,

Thanks for the report and the concise test case.  I had to fix several
bugs and missing features in Avian to get the test working.  Please pull
the latest from the Git repo and let me know how it works for you.

By the way, Akka likes to silently call System.exit if it catches an error
by default, which makes debugging a challenge.  If you add a file named
application.conf with the following content to the root of your classpath,
you can override that behavior:

akka.loglevel = DEBUG
akka.stdout-loglevel = DEBUG
akka.jvm-exit-on-fatal-error = off

I hope that helps.

On Sun, 11 Mar 2012, sir...@gmail.com wrote:

> Hi all,
> I've discovered Avian a few days ago, and I think it's a great project, so

> first of all,�I want to�congratulate�you for your work.
>
> I'm trying to use it in a �medium-big Scala application using Netty, Akka

> (http://akka.io) and OrientDB with the embedded openjdk. So far, Scala and
> Netty are working fine, and the really basic stuff of Akka too.
>
> But I've starting to have some problems with Akka that make the application
> crash without any error output. I've done some debugging, and I suspect that
> it's related with the thread pools. Akka is an actor library, and the actors
> are executed using a pool of threads, and when and I think that when an

> actor is�unallocated�from the thread, it crashes. I also suspect of the
> native method�sun.misc.Unsafe.park() from some debug I've done with

> VisualVM, but I'm not sure.
>
> So, I've created a simple test to reproduce the error (in Java for
> simplicity, but Akka uses scala internally). It's not the only way of
> reproduce it, but the easiest one I've found.
>
> import akka.actor.ActorRef;
> import akka.actor.ActorSystem;
> import akka.actor.Props;
> import akka.actor.UntypedActor;
> import akka.routing.RoundRobinRouter;
>
> public class Test {
>

> � � public static class HelloActor extends UntypedActor {
> � � � � public void onReceive(Object message) throws Exception {
> � � � � � � System.out.println("Hello World");
> � � � � }
> � � }
>
> � � public static void main(String[] args) {
> � � � � ActorSystem system = ActorSystem.create("Test");
> � � � � ActorRef hello = system.actorOf(new
> Props(HelloActor.class).withRouter(new RoundRobinRouter(2))); � � �
> � � � � hello.tell("something");
> � � }
> }
>
> That crashes without error, but if you remove�.withRouter(new

> RoundRobinRouter(2)), it works. The router forces the creation of 2 actors,
> and one is automatically unallocated, so the problem arrises.
>
> To easily compile it, this two libraries are needed.
>
> http://repo.typesafe.com/typesafe/releases/com/typesafe/akka/akka-actor/2.0
> /akka-actor-2.0.jar
> http://repo.typesafe.com/typesafe/releases/org/scala-lang/scala-library/2.9

> .1-1/scala-library-2.9.1-1.jar�
>
> Then
>
> $ javac -cp akka-actor-2.0.jar:scala-library-2.9.1-1.jar�Test.java
>
> $ java -cp .:akka-actor-2.0.jar:scala-library-2.9.1-1.jar�Test

> Hello World
> (and the actors keep running, kill with ctrl-c)
>

> $ avian -cp .:akka-actor-2.0.jar:scala-library-2.9.1-1.jar�Test

> (crashes without output)
>
> Thanks in advance,
> Pablo
>
> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/avian/-/ruxYtP_7zMsJ.
> To post to this group, send email to av...@googlegroups.com.
> To unsubscribe from this group, send email to

> avian+unsubscribe@googlegroups.com.

[Pablo Guerrero]

Hi Joel,

Your fixes work perfectly. 

Now I'm having some problems with OrientDB, but I'll need some more debugging to produce a simple test case.

Thanks,

Pablo

[Pablo Guerrero]

Hi Joel,

I've been trying to run sbt (build tool for scala) with avian, and I've found this little problem.

According to the java api here:  http://docs.oracle.com/javase/6/docs/api/java/io/File.html#createNewFile()

If the file already exist, it should return false, but in avian with openjdk it throws an exception IOException instead.

Here it's a really simple test that prints false using oracle jvm, but throws an exception in avian.

import java.io.File;

class Test {

    public static void main(String args[]) throws java.io.IOException {

        File f = new File("/tmp/foo");

        Boolean r = f.createNewFile();

        System.out.println(r);

    }

}

I think that the solution would be to modify in src/classpath-openjdk.cpp this function:

extern "C" JNIEXPORT jint JNICALL

EXPORT(JVM_Open)(const char* path, jint flags, jint mode)

{

  return OPEN(path, flags, mode);

}

To check errno == EEXIST and return JVM_EEXIST instead as shown here:

http://hg.openjdk.java.net/jdk7/jdk7/hotspot/file/tip/src/share/vm/prims/jvm.cpp

     2481 JVM_LEAF(jint, JVM_Open(const char *fname, jint flags, jint mode))

     2482   JVMWrapper2("JVM_Open (%s)", fname);

     2483 

     2484   //%note jvm_r6

     2485   int result = os::open(fname, flags, mode);

     2486   if (result >= 0) {

     2487     return result;

     2488   } else {

     2489     switch(errno) {

     2490       case EEXIST:

     2491         return JVM_EEXIST;

     2492       default:

     2493         return -1;

     2494     }

     2495   }

     2496 JVM_END

I've done a quick check and it pass the simple test case (sbt still not working for other reasons) but I'm not sure about the implications of this on windows, therefore, I don't provide a patch, but I think with this info it should be easy to fix it.

Cheers,

Pablo

[Joel Dice]

Hi Pablo,

Thanks for the report. This should fix it:

http://oss.readytalk.com/gitweb?p=avian.git;a=commitdiff;h=4aefa211a3e0f7a578e3664f97fcd109068b2608

I've tested on OS X, Linux, and Windows, and it seems to work on all
three.

On Mon, 12 Mar 2012, Pablo Guerrero wrote:

> Hi Joel,
>
> I've been trying to run sbt (build tool for scala) with avian, and I've
> found this little problem.
>
> According to the java api here:�http://docs.oracle.com/javase/6/docs/api/java/io/File.html#createNewFile()
>
> If the file already exist, it should return false, but in avian with openjdk
> it throws an exception IOException instead.
> Here it's a really simple test that prints false using oracle jvm, but
> throws an exception in avian.
>
> import java.io.File;
>
> class Test {
>
> � � public static void main(String args[]) throws java.io.IOException {
> � � � � File f = new File("/tmp/foo");
> � � � � Boolean r = f.createNewFile();
> � � � � System.out.println(r);
> � � }
>
> }
>
> I think that the solution would be to modify in src/classpath-openjdk.cpp
> this function:
>
> extern "C" JNIEXPORT jint JNICALL
> EXPORT(JVM_Open)(const char* path, jint flags, jint mode)
> {
> � return OPEN(path, flags, mode);
> }
>
> To check errno == EEXIST and return JVM_EEXIST instead as shown here:
> http://hg.openjdk.java.net/jdk7/jdk7/hotspot/file/tip/src/share/vm/prims/jv
> m.cpp
>

> � � �2481 JVM_LEAF(jint, JVM_Open(const char *fname, jint flags, jint mode))

> � � �2482 � JVMWrapper2("JVM_Open (%s)", fname);
> � � �2483�
> � � �2484 � //%note jvm_r6
> � � �2485 � int result = os::open(fname, flags, mode);
> � � �2486 � if (result >= 0) {
> � � �2487 � � return result;
> � � �2488 � } else {
> � � �2489 � � switch(errno) {
> � � �2490 � � � case EEXIST:
> � � �2491 � � � � return JVM_EEXIST;
> � � �2492 � � � default:
> � � �2493 � � � � return -1;
> � � �2494 � � }
> � � �2495 � }
> � � �2496 JVM_END
>
> I've done a quick check and it pass the simple test case (sbt still not
> working for other reasons) but I'm not sure about the implications of this
> on windows, therefore, I don't provide a patch, but I think with this info
> it should be easy to fix it.
>
> Cheers,
> Pablo
>

> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/avian/-/xGeL5HoHl9UJ.

> To post to this group, send email to av...@googlegroups.com.
> To unsubscribe from this group, send email to

> avian+un...@googlegroups.com.

[Pablo Guerrero]

Thanks, I've tested, and it works.

What I've been trying to do is to run the akka unit test suite on avian to evaluate how well supported it is, right now. As Akka is written in Scala, it's also a good way to see the Scala support in avian.

As sbt is failing, I've tried to run the suite directly but it's not even starting. I've used gdb to run it, and the missing piece is JVM_GetThreadStateValues (at least, because I couldn't go further), that it's not currently implemented. So it's not a bug, just a missing feature, xD.

About the error on sbt, I don't know if it's important to get it working, as it's just a tool, but it might point out an interesting bug. Running it with the lastest version from git (under gdb) I get this:

gdb --args ./avian -jar sbt-launch.jar 

Program received signal EXC_BAD_ACCESS, Could not access memory.

Reason: KERN_PROTECTION_FAILURE at address: 0x0000000103af59eb

0x0000000103af59eb in ?? ()

(gdb) info stack

#0  0x0000000103af59eb in ?? ()

#1  0x00000001000ba84e in apply (c=0x1071a9ac8, op=vm::MoveZ, s1Size=2, s1Low=0x1071adfe8, s1High=0x1071adfe8, s2Size=8, s2Low=0x1071b4940, s2High=0x1071b4940) at compiler.cpp:3001

#2  0x00000001000c5ae1 in maybeMove (c=0x1071a9ac8, type=vm::MoveZ, srcSize=8, srcSelectSize=2, src=0x1071ae000, dstSize=8, dst=0x1071ae150, dstMask=@0x7fff5fbfde60) at compiler.cpp:3618

#3  0x00000001000c1d7d in compile (this=0x1071ae198, c=0x1071a9ac8) at compiler.cpp:3834

#4  0x00000001000c6480 in compile (c=0x1071a9ac8, stackOverflowHandler=4370464944, stackLimitOffset=2360) at compiler.cpp:5834

#5  0x00000001000b867c in compile (this=0x1071a9ac0, stackOverflowHandler=4370464944, stackLimitOffset=2360) at compiler.cpp:6920

#6  0x000000010007d1fb in finish (t=0x10401d208, allocator=0x1039147a8, context=0x7fff5fbfe350) at compile.cpp:7015

#7  0x000000010007e09c in compile (t=0x10401d208, allocator=0x1039147a8, bootContext=0x0, method=0x10402c9e8) at compile.cpp:10018

#8  0x000000010007e655 in compileMethod2 (t=0x10401d208, ip=0x104911078) at compile.cpp:9073

#9  0x0000000100067d60 in compileMethod (t=0x10401d208) at compile.cpp:7322

#10 0x000000010480001b in ?? ()

#11 0x000000010007f14c in invoke (thread=0x10401d208, method=0x10402bca0, arguments=0x7fff5fbfe878) at compile.cpp:8332

#12 0x00000001000676fd in invokeList (this=0x103914728, t=0x10401d208, method=0x10402bca0, this_=0x0, indirectObjects=false, arguments=0x7fff5fbfead8) at compile.cpp:8757

#13 0x00000001000334a5 in vm::Processor::invoke (this=0x103914728, t=0x10401d208, method=0x10402bca0, this_=0x0) at processor.h:163

#14 0x00000001000284e0 in vm::initClass (t=0x10401d208, c=0x10402cbf0) at machine.cpp:3825

#15 0x0000000100060803 in tryInitClass (t=0x10401d208, class_=0x10402cbf0) at compile.cpp:2286

#16 0x0000000104910fce in ?? ()

Previous frame inner to this frame (gdb could not unwind past this frame)

I had a quick look, and I couldn't make any sense out of it, but maybe you can see an obvious bug somewhere.

Thanks a lot for your great works.

Cheers,

Pablo

[Joel Dice]

On Tue, 13 Mar 2012, Pablo Guerrero wrote:

> Thanks, I've tested, and it works.
> What I've been trying to do is to run the akka unit test suite�on avian to
> evaluate how well supported it is, right now.�As Akka is written in Scala,
> it's also a good way to see the Scala support in avian.
>
> As sbt is failing, I've tried to run the suite directly but it's not even
> starting.�I've used gdb to run it, and the missing piece is
> JVM_GetThreadStateValues (at least, because I�couldn't�go further), that
> it's not currently implemented. So it's not a bug, just a missing feature,
> xD.

I'll see if I can get that working when I have some time. Will I need
anything besides the two jars you mentioned in your earlier email? What
command should I run?

Yeah, that looks like a serious bug. I'll investigate it when I get a
chance. Is this the right place to get sbt-launch.jar?

http://typesafe.artifactoryonline.com/typesafe/ivy-releases/org.scala-tools.sbt/sbt-launch/0.11.2/sbt-launch.jar

>
> I had a quick look, and I couldn't make any sense out of it, but maybe you
> can see an obvious bug somewhere.
>
> Thanks a lot for your great works.
>
> Cheers,
> Pablo
>
> On Tuesday, March 13, 2012 5:56:50 PM UTC+1, Joel Dice wrote:
> Hi Pablo,
>
> Thanks for the report. �This should fix it:
>
> http://oss.readytalk.com/gitweb?p=avian.git;a=commitdiff;h=4aefa211a3e0f7a5
> 78e3664f97fcd109068b2608
>
> I've tested on OS X, Linux, and Windows, and it seems to work on
> all
> three.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/avian/-/DCnmdco0Z54J.

[Pablo Guerrero]

Hi Joel,

Actually it's kind of tricky to run the Akka tests without sbt. It's tightly integrated with the Akka's build system, and you have to build part of it to execute the tests.

As the first problem is that the tests are not even starting, and it uses scalatest, I can try to produce a simple test with scalatest to see if it generates the same error. Also, if you find some time to implement JVM_GetThreadStateValues, I can test it too.

The link for sbt should be fine, I've used 0.11.2 too.

Cheers,

Pablo

[Joel Dice]

Hi Pablo,

I've fixed the sbt failure:

http://oss.readytalk.com/gitweb?p=avian.git;a=commitdiff;h=f2e26791a4ade61345849a98111eaf27be1d3c4e

Will this make it easier to reproduce the JVM_GetThreadStateValues code
path? If so, please let me know what command I need to run.

Thanks.

On Tue, 13 Mar 2012, Pablo Guerrero wrote:

> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/avian/-/WLUHqyDz-skJ.

[Pablo Guerrero]

Hi Joel,

Using sbt it's really simple. What I did yesterday was:

$ git clone git://github.com/akka/akka.git

$ cd akka

$ avian -jar sbt-launch.jar

> project akka-actor-tests

> test

That will have to build the akka-actor and akka-testkit projects (automatically) and then run the tests.

If you want to run it in the standard way to check it:

$ java -Xms512M -Xmx1536M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=384M -jar sbt-launch.jar

> project 

> test

I'll try to check if it works with your latest changes later.

Cheers,

Pablo

On Wednesday, March 14, 2012 12:05:44 AM UTC+1, Joel Dice wrote:

Hi Pablo,

I've fixed the sbt failure:

http://oss.readytalk.com/gitweb?p=avian.git;a=commitdiff;h=f2e26791a4ade61345849a98111eaf27be1d3c4e

Will this make it easier to reproduce the JVM_GetThreadStateValues code
path?  If so, please let me know what command I need to run.

Thanks.

On Tue, 13 Mar 2012, Pablo Guerrero wrote:

> Hi Joel,
> Actually it's kind of tricky to run the Akka tests without sbt. It's tightly
> integrated with the Akka's build system, and you have to build part of it to
> execute the tests.
>
> As the first problem is that the tests are not even starting, and it uses
> scalatest, I can try to produce a simple test with scalatest to see if it
> generates the same error. Also, if you find some time to

> implement�JVM_GetThreadStateValues, I can test it too.

>
> The link for sbt should be fine, I've used 0.11.2 too.
>
> Cheers,
> Pablo
>
> --
> You received this message because you are subscribed to the Google Groups
> "Avian" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/avian/-/WLUHqyDz-skJ.
> To post to this group, send email to av...@googlegroups.com.
> To unsubscribe from this group, send email to

> avian+unsubscribe@googlegroups.com.

[Pablo Guerrero]

Hi,

I've checked, and there is an error when sbt tries to compile some classes.

[info] Compiling 6 Scala sources to /home/siriux/scala/akka/project/target/scala-2.9.1/sbt-0.11.2/classes...

[error] error while loading package, Missing dependency 'class java.lang.Object', required by /home/siriux/.sbt/boot/scala-2.9.1/lib/scala-library.jar(scala/package.class)

[error] {file:/home/siriux/scala/akka/project/}default-c5553a/compile:compile: scala.tools.nsc.MissingRequirementError: class java.lang.String not found.

Cheers,

Pablo

[Joel Dice]

Yes, I ran into this a few weeks ago when I tried running Scala on Avian
for the first time. Scala likes to create its own classloader based on
the classpath of the VM's boot classloader, which it accesses via the
non-standard sun.boot.class.path system property. Since Avian didn't set
that property, you got a MissingRequirementError.

I've addressed that in the jdk7 branch of the Git repo:

http://oss.readytalk.com/gitweb?p=avian.git;a=commitdiff;h=c3b72a3dd544b9362bbed58de0d0207b52ddaeca

I recommend switching to that branch, since it's where new development is
being done on the OpenJDK port. It's not backwards-compatible with
OpenJDK 6, though, so you'll need to switch to OpenJDK 7.

Also note that the openjdk-src build will not currently work with Scala,
since it seems to be confused by the fact that the boot classpath includes
an embedded jar file which doesn't actually exist on the filesystem. I'm
not sure yet if that's fixable and I haven't looked into it. Meanwhile,
a non-embedded openjdk build should get you past the above error, although
you'll probably hit a StackOverflowError next, since the Scala compiler
uses a lot of stack and Avian currently ignores -Xss options. That's what
I'll work on next.

[Pablo Guerrero]

Good, I'll move to openjdk7 and try to test there.

Thanks,

Pablo

[Joel Dice]

On Wed, 14 Mar 2012, Joel Dice wrote:

> Also note that the openjdk-src build will not currently work with Scala,
> since it seems to be confused by the fact that the boot classpath includes an
> embedded jar file which doesn't actually exist on the filesystem. I'm not
> sure yet if that's fixable and I haven't looked into it. Meanwhile, a
> non-embedded openjdk build should get you past the above error, although
> you'll probably hit a StackOverflowError next, since the Scala compiler uses
> a lot of stack and Avian currently ignores -Xss options. That's what I'll
> work on next.

I've implemented support for -Xss options, and now the Akka test is almost
working, except there are intermittent NullPointerExceptions which cause
it to fail. They seem to be happening in
scala.reflect.generic.UnPickler$Scan$$anonfun$readType$3.reflMethod$Method1(java.lang.Class),
which, when disassembled, looks like this:

public static java.lang.reflect.Method
reflMethod$Method1(java.lang.Class);
Code:
Stack=5, Locals=2, Args_size=1
0: getstatic #34; //Field reflPoly$Cache1:Ljava/lang/ref/SoftReference;
3: invokevirtual #44; //Method java/lang/ref/SoftReference.get:()Ljava/lang/Object;
6: checkcast #46; //class scala/runtime/MethodCache
9: ifnonnull 29
12: new #22; //class java/lang/ref/SoftReference
15: dup
16: new #24; //class scala/runtime/EmptyMethodCache
19: dup
20: invokespecial #27; //Method scala/runtime/EmptyMethodCache."<init>":()V
23: invokespecial #30; //Method java/lang/ref/SoftReference."<init>":(Ljava/lang/Object;)V
26: putstatic #34; //Field reflPoly$Cache1:Ljava/lang/ref/SoftReference;
29: getstatic #34; //Field reflPoly$Cache1:Ljava/lang/ref/SoftReference;
32: invokevirtual #44; //Method java/lang/ref/SoftReference.get:()Ljava/lang/Object;
35: checkcast #46; //class scala/runtime/MethodCache
38: aload_0
39: invokevirtual #49; //Method scala/runtime/MethodCache.find:(Ljava/lang/Class;)Ljava/lang/reflect/Method;
42: astore_1
43: aload_1
44: ifnonnull 89
47: getstatic #55; //Field scala/runtime/ScalaRunTime$.MODULE$:Lscala/runtime/ScalaRunTime$;
50: aload_0
51: ldc #57; //String setFlag
53: getstatic #20; //Field reflParams$Cache1:[Ljava/lang/Class;
56: invokevirtual #61; //Method java/lang/Class.getMethod:(Ljava/lang/String;[Ljava/lang/Class;)Ljava/lang/reflect/Method;
59: invokevirtual #65; //Method scala/runtime/ScalaRunTime$.ensureAccessible:(Ljava/lang/reflect/Method;)Ljava/lang/reflect/Method;
62: astore_1
63: new #22; //class java/lang/ref/SoftReference
66: dup
67: getstatic #34; //Field reflPoly$Cache1:Ljava/lang/ref/SoftReference;
70: invokevirtual #44; //Method java/lang/ref/SoftReference.get:()Ljava/lang/Object;
73: checkcast #46; //class scala/runtime/MethodCache
76: aload_0
77: aload_1
78: invokevirtual #69; //Method scala/runtime/MethodCache.add:(Ljava/lang/Class;Ljava/lang/reflect/Method;)Lscala/runtime/MethodCache;
81: invokespecial #30; //Method java/lang/ref/SoftReference."<init>":(Ljava/lang/Object;)V
84: putstatic #34; //Field reflPoly$Cache1:Ljava/lang/ref/SoftReference;
87: aload_1
88: areturn
89: aload_1
90: areturn

The NPE is thrown at bytecode 78, which is not surprising, since the
allocation at bytecode 63 may trigger a GC, causing the reflPoly$Cache1
soft reference to be cleared. There's no null pointer check before the
invokevirtual, so you get an NPE if you're not lucky. This is more likely
on Avian's VM than e.g. HotSpot because Avian treats soft references the
same as weak references to minimize memory footprint, but it could happen
on any VM.

This was reported a few months ago on a Scala forum:

http://www.scala-lang.org/node/11807

However, that discussion digressed into other issues, and it's not clear
that the bug was ever fixed. I'd like to try fixing it myself, but I
can't seem to find what part of the Scala compiler is generating the above
bytecode. scala/tools/nsc/transform/CleanUp.scala has the closest match I
can find, but I'm not experienced enough with Scala to see how it maps to
the bytecode or how to fix it.

[Pablo Guerrero]

Hi Joel,

I have no much idea about how the scala compiler works, but I've been checking, and I think I understand what's going on, but I still don't know how to solve it.

I summarize here, because, even if all the info is on the forum, it took me quite some time to really understand everything:

The first thing I've found is that the compiled code comes exactly form https://github.com/scala/scala/blob/v2.9.1/src/library/scala/reflect/generic/UnPickler.scala line 401.

Is this structural typing what creates the problem: { def setFlag(mask: Long): this.type }

I've been back and forth through all the files, and finally I've found that in the CleanUp.scala file there is more than I saw at the first sight. The first time I just saw the addStaticMethodToClass definition, and I didn't see that the real work is done in the anonymous function that comes in the parameter forBody. So, the "real" bytecode generation is done in the call at line 284 in https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/transform/CleanUp.scala

It's using functions to define an AST from here:

https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/ast/TreeDSL.scala

(the actual byte code generation it's here: https://github.com/scala/scala/blob/v2.9.1/src/compiler/scala/tools/nsc/backend/jvm/GenJVM.scala and https://github.com/scala/scala/tree/v2.9.1/src/fjbg/ch/epfl/lamp/fjbg)

So, it seems that just converting the code as explained in the first answer here: http://www.scala-lang.org/node/11807 would work.

But reading the forum more carefully I've found that this issue https://issues.scala-lang.org/browse/SI-2365 was opened to use soft references and allow classes to be garbage collected (in sbt, for example).

Right now, that's what I've found so far. Maybe that are no news for you, but I wanted to write it down to remember it, just in case.

What I think it's clear to me is that any solution have to take into account the issue 2365, or otherwise they will not accept it. 

Any ideas?

Cheers,

Pablo 

[Pablo Guerrero]

Hi again,

I just thought that it might be really simple. Instead of converting the soft reference into a hard reference, as proposed in the forum, it would be enough just keeping a hard reference (in addition to the soft) in the scope of the function.

This way, we are sure that the object is not gc during during the function, but it can still be gc any other time. So, we don´t get any NPE, and the issue 2365 is not broken either.

I will have to give it a second thought and find the implementation details, but I have to go now.

Cheers,

Pablo

[Pablo Guerrero]

Now that I re-read it carefully, the proposed solution it's actually the same I was proposing. I'll try to see how to implement that on the compiler, it should be easy.

[Pablo Guerrero]

Thanks a lot for reporting it!

I hope it gets solved at some point.

Cheers,

Pablo

On Mon, Jan 14, 2013 at 6:41 PM, <simon.och...@gmail.com> wrote:

Hi Joel,

the bug is now being tracked under issue numbers SI-6969¹ and SI-6970².

Thanks a lot!

Simon

PS: A lot of people are pretty impressed by what you are doing: https://groups.google.com/d/msg/scala-language/HW4FvHVLo8k/pC02zfuBcQQJ :-)

¹ https://issues.scala-lang.org/browse/SI-6969
² https://issues.scala-lang.org/browse/SI-6970

--
You received this message because you are subscribed to the Google Groups "Avian" group.

To view this discussion on the web visit https://groups.google.com/d/msg/avian/-/dRD3EyWVTsUJ.

To post to this group, send email to av...@googlegroups.com.

To unsubscribe from this group, send email to avian+un...@googlegroups.com.