How to set class loader for scala.tools.nsc.Global compiler

[Adam Retter]

Hi there,

I am attempting to compile some Scala code programatically by making use of the compiler scala.tools.nsc.Global.

The problem I have is that the class loader of my current thread, has all of the classes available to it that I am interested in, however when I compile some code that imports one of these classes (presumably within the same thread), the compiler cannot load the class. I guess the Global compiler is using a different class loader?

My question is how can I set the classloader for the Global compiler? I do not know the classpath directly, as it was set when the host application was executed by the system and will vary as the application is upgraded over-time.

For example:

//this works

val xstreamCl = classOf[com.thoughtworks.xstream.XStream]

println("OK Able to access: " + xstreamCl)

//this does not work

val settings = new GenericRunnerSettings(msg => listener.fatalError(message))

settings.embeddedDefaults(Thread.currentThread().getContextClassLoader)        //this does not seem to set the classloader for the compiler... why???

val reporter = new ConsoleReporter(settings, Console.in, new PrintWriter(listener.getLogger))

val compiler = new Global(settings, reporter)

val run = new compiler.Run

run.compile(List(script))

Where the content of the file indicated by the script variable looks like:

import com.thoughtworks.xstream.XStream

println("import OK)

Thanks Adam.

[Roman Janusz]

AFAIK, the compiler is not using class loaders to look up classes needed for compilation. Instead, it scans the classpath by itself and manually unpickles classfiles.

I don't know if there is a way to force standard compiler (scala.tools.nsc.Global) to see some additional classes from classloaders. However, there is a separate incarnation of the compiler that uses reflection instead of manually scanning classpath. It is scala.tools.reflect.ReflectGlobal. However, be aware that this version of the compiler shares some serious bugs with Scala runtime reflection. See https://groups.google.com/forum/#!topic/scala-user/tLz-B7d0I80

Cheers,
Roman

[Chris Twiner]

Have you looked at https://code.google.com/p/scalascriptengine/ ?

It takes care of compiler interactions for you, it may do the right thing class loader wise.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Adam Retter]

Yes thanks I have looked at scalascriptengine but as far as I can tell it does not do anything special with class loaders.

[Adam Retter]

Ah ha! Yes that seems to do exactly what I wanted. Thanks.

I still have a bit to do around caching compilation, and hopefully cleaning up the compilers to avoid the issues you mentioned. However I have the basics working now: https://github.com/adamretter/jenkins-scala-plugin

[Grzegorz Kossakowski]

On 2 January 2014 19:17, Roman Janusz <romeqj...@gmail.com> wrote:

AFAIK, the compiler is not using class loaders to look up classes needed for compilation. Instead, it scans the classpath by itself and manually unpickles classfiles.

I don't know if there is a way to force standard compiler (scala.tools.nsc.Global) to see some additional classes from classloaders. However, there is a separate incarnation of the compiler that uses reflection instead of manually scanning classpath. It is scala.tools.reflect.ReflectGlobal. However, be aware that this version of the compiler shares some serious bugs with Scala runtime reflection. See https://groups.google.com/forum/#!topic/scala-user/tLz-B7d0I80

That's right. Global doesn't use classloaders to load classes it's compiling against. The reason is classloaders do not give an access to raw bytes corresponding to given class. That's what Global needs to load all necessary information from class file. Therefore, Global has its own classpath abstraction and you have to specify it in order to make Global find the classes you are compiling your code against.

--
Grzegorz Kossakowski

Scalac hacker at Typesafe

twitter: @gkossakowski

github: @gkossakowski

[Eugene Burmako]

What information does Global need apart from what's available via Java reflection?

[Grzegorz Kossakowski]

On 4 January 2014 15:48, Eugene Burmako <eugene....@epfl.ch> wrote:

What information does Global need apart from what's available via Java reflection?

AFAIK, Java reflection does not allow you to list known packages and classes in them, right?

[Eugene Burmako]

That's correct, it does not. Is there anything else that comes in mind? Given that we fix this problem with ReflectGlobal, would that give us feature parity with Global or there would be something else to fix?

[Grzegorz Kossakowski]

On 4 January 2014 16:08, Eugene Burmako <eugene....@epfl.ch> wrote:

That's correct, it does not. Is there anything else that comes in mind? Given that we fix this problem with ReflectGlobal, would that give us feature parity with Global or there would be something else to fix?

How do you "fix" this problem with ReflectGlobal?

It used to be that pickled information was stored in class file attribute which is not accessible through Java reflection but it's stored in a runtime visible annotation since Scala 2.8.

Other than that, the inliner requires reading method bodies in order to be able to inline the code. Therefore, ReflectGlobal wouldn't be able to run with -optimize turned on.

[Eugene Burmako]

We could special-case ReflectGlobal to treat enumerable classloaders specially. That seems feasible and useful.

Good point about the inliner!

[Grzegorz Kossakowski]

On 4 January 2014 16:28, Eugene Burmako <eugene....@epfl.ch> wrote:

We could special-case ReflectGlobal to treat enumerable classloaders specially. That seems feasible and useful.

In environments you can do that it's fine. However, there many environments that do implement their own classloaders and do not implement standard method of enumerating packages and classes.

I'm coming to conclusion that enumerating packages and classes is not fundamentally required by Scala compiler. I believe you could support all scenarios by simply asking on-demand. However, I don't know whether it would be efficient (I think so) and I'm sure it would be a lot of work to get rid of an assumption that you can enumerate the package instead of asking for each entry individually.

PS. We are quickly moving to scala-internals territory. :)

[Oliver Ruebenacker]

     Hello,

  Couldn't you do something along the lines of

  myClass.getClassLoader.getSystemResourceAsStream(myClass.getSimpleName + ".class")

  ?

     Best,

     Oliver

 

--
Grzegorz Kossakowski

Scalac hacker at Typesafe

twitter: @gkossakowski

github: @gkossakowski

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Oliver Ruebenacker

Founder at Relomics Consulting

Be always grateful, but never satisfied.