Global.Run NullPointerException

[Guillaume Yziquel]

Hi.

I'm trying to use the scala.tools.nsc.interpreter.IMain class within an
OSGi framework, and am stumbling on the following stack trace (using
scala 2.9.1):

Caused by: java.lang.NullPointerException
at scala.tools.nsc.Global$Run.<init>(Global.scala:663)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compileAndSaveRun(IMain.scala:755)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compile(IMain.scala:731)
at scala.tools.nsc.interpreter.IMain.bind(IMain.scala:591)
at scala.tools.nsc.interpreter.IMain.bind(IMain.scala:626)

which is triggered by code like that:

val mapping : Array[String] => (F, L) = {

val code : String = {
val path = "myCode.scala"
var code : String = null
def resource = this.getClass.getClassLoader.getResourceAsStream(_)
val source = scala.io.Source.fromInputStream (resource (path))
source.withClose(() => code = source.mkString); code
}

import scala.tools.nsc._
val settings = new Settings; settings.usejavacp.value = true
val interp = new interpreter.IMain (settings) {
interp =>

def execute (code : String) : Array[String] => (F, L) = {

val result = new Array[Array[String] => (F, L)](1)
/* Blows up on the following line: */
interp.bind(interpreter.NamedParam("res", "Array[Array[String] => (F, L)]", result))

interp.interpret(code)

result(0)
}

}

While this kind of code works almost fine in the REPL, it blows up when
deployed in an OSGi bundle. As it is a constructor of Global.Run that
fails, it looks like I have failed to initialise it properly. Would very
much like to know where the 2.9.1 version of scala.tools.nsc is
documented in detail.

--
Guillaume Yziquel
Crossing-Tech

[Guillaume Yziquel]

Hi.

After digging a bit into the source code of the IMain class, I noticed
that the NullPointerException is triggered because to access the Run
constructor, you need to already have the englobing Global class
instantiated in the IMain.global lazy val.

In my case, IMain.global is evaluated to null, which means that the
following code fails to initialise properly the IMain.global val.

/** the public, go through the future compiler */
lazy val global: Global = {
if (isInitializeComplete) _compiler
else {
// If init hasn't been called yet you're on your own.
if (_isInitialized == null) {
repldbg("Warning: compiler accessed before init set up. Assuming no postInit code.")
initialize(())
}
// blocks until it is ; false means catastrophic failure
if (_isInitialized()) _compiler
else null /* Execution ends up getting there */
}
}

How do I make sure that the IMain.global value is properly initialised?

Le Thursday 16 Feb 2012 � 09:30:05 (+0100), Guillaume Yziquel a �crit :

[Szemere]

Guillaume,

I have the same issue, with a slightly different setup. I run the Scala IMain from Java. It works fine in a JUnit test, but fails with an NPE at the same place as yours, if run in a servlet environment (mvn jetty:run).

The interpret(..) method works fine, but reset() or bind(..) cause this failure.

Do we need to setup the class loader, classpath or other compiler settings in a particular way to work in this environment?

Szemere

[Guillaume Yziquel]

Le Tuesday 20 Mar 2012 � 07:49:36 (-0700), Szemere a �crit :

I do not know exactly. I mean, I knew, and I do not know any more.

Using an embedded compiler, it seems to me that the most mature projects
are Sling's Guggla embedding of the Scala compiler (there's a version
working in Scalate's source code), and Slang for deployment in an OSGi
container.

The embedded interpreter and compiler are indeed quite sensitive to many
small details. It usually works nicely in the REPL, but you have
surprises when trying to run in richer environments. That's why I'd now
avoid trying to reinvent the wheel, and reuse stuff from Guggla or
Slang, depending on what you want (which is the real issue).

--
Guillaume Yziquel.

[Szemere]

Thanks. I have now tried Guggla, despite it looking lightly maintained (e.g. Scala compiler set at 1.8.1). Running in a normal classloading environment (in this case, a JUnit test) works fine, but running the same code in a servlet context (under Jetty) fails, with "object scala not found":

scala.tools.nsc.MissingRequirementError: object scala not found.
    at scala.tools.nsc.symtab.Definitions$definitions$.getModuleOrClass(Definitions.scala:517)
    at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackage(Definitions.scala:37)
    at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackageClass(Definitions.scala:38)
    at scala.tools.nsc.symtab.Definitions$definitions$.UnitClass(Definitions.scala:83)
    at scala.tools.nsc.symtab.Definitions$definitions$.init(Definitions.scala:789)
    at scala.tools.nsc.Global$Run.<init>(Global.scala:604)
    at guggla.interpreter.ScalaInterpreter.compile(ScalaInterpreter.scala:124)
    at guggla.interpreter.ScalaInterpreter.compile(ScalaInterpreter.scala:140)
    at guggla.interpreter.ScalaInterpreter.compile(ScalaInterpreter.scala:150)
    at guggla.ScalaScriptEngine$$anonfun$1.apply(ScalaScriptEngine.scala:112)
    at guggla.ScalaScriptEngine$$anonfun$1.apply(ScalaScriptEngine.scala:112)
    at guggla.ScalaScriptEngine.writeLocked(ScalaScriptEngine.scala:168)
    at guggla.ScalaScriptEngine.eval(ScalaScriptEngine.scala:111)
    at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:216)

I do not know if this is simple to resolve. I've run out of time on my Scala scripting experiment and am falling back to something else.

I had high hopes for Scala being useable as a scripting language to define snippets of code, in a similar way to Beanshell or Groovy, but with rigorous functional programming. It seems the few folks who have tried this, have found it hard but possible in a normal classloading environment. I've managed to translate calling the interpreter from Scala to calling it from Java. However I have found no way or documented way to make it work in a situation with complex classloading (e.g. servlet context).

Incidentally my Java code (Apache 2 licensed) for the Guggla piece is of the form:
    private  ScriptEngine getScriptEngine() {
        Iterator<ScriptEngineFactory> factories = ServiceRegistry.lookupProviders(ScriptEngineFactory.class);
        ScriptEngineFactory scalaEngineFactory = null;
         
        while (factories.hasNext()) {
            ScriptEngineFactory factory = factories.next();
            if (factory.getEngineName() == "Scala Scripting Engine") {
                scalaEngineFactory = factory;
            }
        }
         
        if (scalaEngineFactory != null) {
            return scalaEngineFactory.getScriptEngine();
        } else {
            throw new RuntimeException("Scala Scripting Engine not found");
        }
    }

        Bindings b = scriptEngine.getBindings(ScriptContext.ENGINE_SCOPE);
        String preamble =
            "package guggla {\n" +
            "class Script(args: ScriptArgs) {\n" +
            "import args._\n";

        String postamble = "\n}}";
        String code = preamble + script + postamble;

        StringWriter stringWriter = new StringWriter();
        scriptEngine.getContext().setWriter(stringWriter);
        try {
            scriptEngine.eval(code, b);
        } catch (ScriptException e) {
            throw new RuntimeException(e);
        }

[Guillaume Yziquel]

Le Wednesday 21 Mar 2012 � 02:54:03 (-0700), Szemere a �crit :

> Thanks. I have now tried Guggla, despite it looking lightly maintained
> (e.g. Scala compiler set at 1.8.1). Running in a normal classloading
> environment (in this case, a JUnit test) works fine, but running the
> same code in a servlet context (under Jetty) fails, with "object scala
> not found":
> scala.tools.nsc.MissingRequirementError: object scala not found.

This is typical of a classloading issue of the default scala jar. I've
experienced it in an OSGi environment.

The main issue is that the classloading inside the scala compiler does
not use classloader, but is essentially filesystem based.

However, the Scala compiler API provides the notion of AbstractFiles,
which allows to "fake" a filesystem on top of OSGi bundles (in my case).

So there are two steps:

[1] Implement such a fake filesystem to help the scala compiler dive
into the OSGi bundles.

[2] Provide the classpath to the compiler. As the compiler expects the
classpath to be fed in a textual format, there is no way to
"off-the-shelf", use the AbstractFile trait.

You have to override the rootClassloader of the Global class.

See here how this is done for on-the-fly compilation and deployment of
OSGi bundles.

https://github.com/Crossing-Tech/slang/blob/master/scala/deployer/src/main/scala/org/fusesource/slang/scala/deployer/compiler/ScalaCompiler.scala#L95

(Arguably not the best way to do it.)

Not so sure about compiling classes directly in OSGi environments.
Deployment of compiled bundles seems to me an important step to ensure
safety of the code (class cast exceptions).

Hope this helps if you have more time.

--
Guillaume Yziquel
Crossing-Tech
Parc Scientifique EPFL