Maven, JUnit and the Scala Interpreter

[etorreborre]

Hi,

I posted this directly to David B. but I thought that this could be of
interest for the rest of you too.

I just created some JUnit tests (well, some specs which can be
executed as JUnit tests). Those tests are using the Scala Interpreter
class and the code runs ok if I run it from the command line (as
regular specs Specifications).

But when I run "mvn test" I get exceptions like:

object scala not found.
at scala.tools.nsc.symtab.Definitions$definitions$.getModuleOrClass
(Definitions.scala:355)
at scala.tools.nsc.symtab.Definitions$definitions$.getModule
(Definitions.scala:326)
at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackage
(Definitions.scala:34)
at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackageClass
(Definitions.scala:35)
at scala.tools.nsc.symtab.Definitions$definitions$.init
(Definitions.scala:663)
at scala.tools.nsc.Global$Run.<init>(Global.scala:476)
at scala.tools.nsc.Interpreter.interpret(Interpreter.scala:477)
at org.specs.util.ScalaInterpreter$$anonfun$interpret$1.apply
(ScalaInterpreter.scala:28)

I tried to add this to my pom.xml in the surefire plugin
configuration:

<additionalClasspathElements>
<additionalClasspathElement>C:\local_repository\org\scala-lang
\scala-library\2.7.3\scala-library-2.7.3.jar</
additionalClasspathElement>
<additionalClasspathElement>C:\local_repository\org\scala-lang
\scala-compiler\2.7.3\scala-compiler-2.7.3.jar</
additionalClasspathElement>
</additionalClasspathElements>

But this didn't work. Any clue?

Thanks a lot.

Eric.

[David Bernard]

Eric,

Could you provide ?
* the full stack trace
* the pom.xml and the unit test that break (if it's not under the SVN of specs)

/davidB

[etorreborre]

Here is a full stack trace:

-------------------------------------------------------------------------------
Test set: org.specs.util.interpreterSpec
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.016
sec <<< FAILURE!
initializationError0(org.specs.util.interpreterSpec) Time elapsed: 0
sec <<< ERROR!
scala.tools.nsc.FatalError: object scala not found.

at scala.tools.nsc.symtab.Definitions$definitions$.getModuleOrClass
(Definitions.scala:355)
at scala.tools.nsc.symtab.Definitions$definitions$.getModule
(Definitions.scala:326)
at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackage
(Definitions.scala:34)
at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackageClass
(Definitions.scala:35)
at scala.tools.nsc.symtab.Definitions$definitions$.init
(Definitions.scala:663)
at scala.tools.nsc.Global$Run.<init>(Global.scala:476)
at scala.tools.nsc.Interpreter.interpret(Interpreter.scala:477)

at org.specs.util.interpreterSpec.<init>(interpreterSpec.scala:11)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance
(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance
(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
at java.lang.Class.newInstance0(Class.java:350)
at java.lang.Class.newInstance(Class.java:303)
at org.specs.runner.JUnitSuiteRunner.<init>(JUnitSuiteRunner.scala:
20)
at sun.reflect.GeneratedConstructorAccessor3.newInstance(Unknown
Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance
(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
at org.junit.internal.requests.ClassRequest.buildRunner
(ClassRequest.java:33)
at org.junit.internal.requests.ClassRequest.getRunner
(ClassRequest.java:28)
at org.apache.maven.surefire.junit4.JUnit4TestSet.<init>
(JUnit4TestSet.java:45)
at
org.apache.maven.surefire.junit4.JUnit4DirectoryTestSuite.createTestSet
(JUnit4DirectoryTestSuite.java:56)
at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.locateTestSets
(AbstractDirectoryTestSuite.java:96)
at org.apache.maven.surefire.Surefire.createSuiteFromDefinition
(Surefire.java:209)
at org.apache.maven.surefire.Surefire.run(Surefire.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
(SurefireBooter.java:338)
at org.apache.maven.surefire.booter.SurefireBooter.main
(SurefireBooter.java:997)

And the pom file:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">

<modelVersion>4.0.0</modelVersion>
<groupId>org.scala-tools.testing</groupId>
<artifactId>specs</artifactId>
<version>1.4.4-SNAPSHOT</version>
<packaging>jar</packaging>
<name>${project.artifactId}</name>
<description>specs is a Behaviour-Driven-Design
framework</description>
<url>http://code.google.com/p/specs/</url>
<inceptionYear>2007</inceptionYear>
<licenses>
<license>
<name>MIT License</name>
<url>http://www.opensource.org/licenses/mit-license.php</url>
<distribution>repo</distribution>
</license>
</licenses>
<scm>
<connection>
scm:svn:http://specs.googlecode.com/svn/trunk</connection>
<developerConnection>
scm:svn:https://specs.googlecode.com/svn/trunk</
developerConnection>
<url>http://specs.googlecode.com/svn/trunk</url>
</scm>
<issueManagement>
<system>code.google</system>
<url>http://code.google.com/p/specs/issues/</url>
</issueManagement>
<developers>
<developer>
<id>etorreborre</id>
<name>Eric Torreborre</name>
<timezone>+8</timezone>
</developer>
</developers>
<properties>
<scala.version>2.7.3</scala.version>
<java.src.version>1.5</java.src.version>
</properties>
<repositories>
<repository>
<id>scala-tools.org</id>
<name>Scala-Tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</repository>
<repository>
<id>scala-tools.org - snapshots</id>
<name>Scala-Tools Maven2 Repository - snapshots</name>
<url>http://scala-tools.org/repo-snapshots</url>
</repository>
<repository>
<id>maven2-repository.dev.java.net</id>
<name>Java.net Repository for Maven</name>
<url>http://download.java.net/maven/2/</url>
<layout>default</layout>
</repository>
<repository>
<id>powermock-repo</id>
<url>http://powermock.googlecode.com/svn/repo/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-Tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</pluginRepository>
<pluginRepository>
<id>scala-tools.org.snapshots</id>
<name>Scala-Tools Maven2 Repository - snapshots</name>
<url>http://scala-tools.org/repo-snapshots</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-compiler</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.4</version>
</dependency>
<dependency>
<groupId>net.java</groupId>
<artifactId>textile-j</artifactId>
<version>2.2</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.scala-tools.testing</groupId>
<artifactId>scalatest</artifactId>
<version>0.9.5</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.scalacheck</groupId>
<artifactId>scalacheck</artifactId>
<version>1.5</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.jmock</groupId>
<artifactId>jmock</artifactId>
<version>2.4.0</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.7</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.powermock.api</groupId>
<artifactId>powermock-api-mockito</artifactId>
<version>1.2</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>cglib</groupId>
<artifactId>cglib</artifactId>
<version>2.1_3</version>
<scope>test</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.objenesis</groupId>
<artifactId>objenesis</artifactId>
<version>1.0</version>
<scope>test</scope>
<optional>true</optional>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<extensions>
<extension>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-webdav</artifactId>
<version>1.0-beta-2</version>
</extension>
</extensions>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.8</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
<jvmArgs>
<jvmArg>-Xmx512m</jvmArg>
<jvmArg>-Xms64m</jvmArg>
</jvmArgs>
</configuration>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>${java.src.version}</source>
<target>${java.src.version}</target>
<optimise>true</optimise>
<debug>true</debug>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/*Unit.java</include>
<include>**/*Spec.java</include>
</includes>
<excludes>
<exclude>**/*Test.java</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>jar</goal>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
<configuration></configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<executions>
<execution>
<id>attach-sources</id>
<phase>verify</phase>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<reporting>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.8</version>
<configuration>
<jvmArgs>
<jvmArg>-Xmx512m</jvmArg>
</jvmArgs>
<vscaladocVersion>1.2-SNAPSHOT</vscaladocVersion>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
</plugins>
</reporting>
<distributionManagement>
<repository>
<id>scala-tools.org</id>
<url>
http://scala-tools.org:8081/nexus/content/repositories/releases</url>
</repository>
<snapshotRepository>
<id>scala-tools.org</id>
<url>
http://scala-tools.org:8081/nexus/content/repositories/snapshots</url>
<uniqueVersion>false</uniqueVersion>
</snapshotRepository>
</distributionManagement>
<profiles>
<profile>
<id>release</id>
<distributionManagement>
<repository>
<id>scala-tools.org</id>
<url>dav:http://dav.scala-tools.org/repo-releases</url>
</repository>
<site>
<id>scala-tools.org</id>
<url>
dav:http://dav.scala-tools.org/mvnsites/specs/$
{project.version}</url>
</site>
</distributionManagement>
</profile>
</profiles>
</project>

And finally, the unit test:

package org.specs.util
import org.specs._
import scala.tools.nsc.{Interpreter, Settings, InterpreterResults}
import java.io.{PrintWriter, StringWriter}
import org.specs.runner._

class interpreterSpec extends Specification with JUnit {

val writer = new StringWriter
val interpreter = new Interpreter(new Settings, new PrintWriter
(writer))
interpreter.interpret("1 + 1")
println(writer.toString)
}

You need specs-1.4.4 to run it but I can that you will have the same
issue with a standard JUnit test.

Thanks for your help,

Eric.

[David Bernard]

Hi,

When I wrote the scala macro interpreter for Jedit I need to provide a
custom classpath to the interpreter to allow it to find scala-library
and scala-compiler :
import _root_.scala.tools.nsc.{Interpreter, Settings}
...

val settings = new Settings(null)
settings.classpath.value = List(
settings.classpath.value,
new File(dir, "scala-library.jar").getAbsolutePath,
new File(dir, "scala-compiler.jar").getAbsolutePath
).mkString(File.pathSeparator)

val out = new StringWriter()
val interp = new Interpreter(settings, new PrintWriter(out)){
override def parentClassLoader =
classOf[MacroHandler4Scala].getClassLoader
}
try {
interp.beQuietDuring({
interp.bind("view", classOf[View].getName, view)
interp.bind("buffer", classOf[Buffer].getName, view.getBuffer)
interp.bind("editPane", classOf[EditPane].getName, view.getEditPane)
interp.bind("textArea", classOf[JEditTextArea].getName,
view.getTextArea)
})
Source.fromFile(macro.getPath).getLines.foreach(interp.interpret(_))
val output = out.toString
if (output.length > 0) {
new TextAreaDialog(view, "scala-eval", "output",
UIManager.getIcon("OptionPane.errorIcon"), output)
}

I think you have the same issue. And I suppose it work as standalone
because in this case scala is part of the system classloader.

[etorreborre]

Hi David,

> When I wrote the scala macro interpreter for Jedit I need to provide a
> custom classpath to the interpreter to allow it to find scala-library
> and scala-compiler :
> import _root_.scala.tools.nsc.{Interpreter, Settings}
> ...
>
> val settings = new Settings(null)
> settings.classpath.value = List(
> settings.classpath.value,
> new File(dir, "scala-library.jar").getAbsolutePath,
> new File(dir, "scala-compiler.jar").getAbsolutePath
> ).mkString(File.pathSeparator)

My question here is: how do I get those values when I'm running this
code inside JUnit?

In the code above where do you get the "dir" variable from and how
would that be possible to write code which could work using both JUnit
and scala (on the command line)?

Thanks,

Eric.

[David Bernard]

First try to keep the setting.classpath empty and defile the
classloader of the interpreter to the current one something like

val cl = Thread.currentThread.getContextClassLoader

val interp = new Interpreter(settings, new PrintWriter(out)){
override def parentClassLoader = cl
}

If it not work I'll search in my notes (older how you could retreive
scala-library.jar location from classpath)

[etorreborre]

Hmmm, this doesn't work. I think I need some classpath surgery to
retrieve the scala-library from the classpath!

On Mar 19, 6:13 pm, David Bernard <david.bernard...@gmail.com> wrote:
> First try to keep the setting.classpath empty and defile the
> classloader of the interpreter to the current one something like
>
>    val cl = Thread.currentThread.getContextClassLoader
>    val interp = new Interpreter(settings, new PrintWriter(out)){
>      override def parentClassLoader = cl
>    }
>
> If it not work I'll search in my notes (older how you could retreive
> scala-library.jar location from classpath)
>

[David Bernard]

a code fragment that you could use to find a jar in classpath :

var cl = classLoader.getOrElse(Thread.currentThread.getContextClassLoader())
while (cl != null) {
if (cl.isInstanceOf[URLClassLoader]) {
val ucl = cl.asInstanceOf[URLClassLoader]
ucl.getURLs().foreach {url =>
if (url.getProtocol().equals("file")) {
val file = new File(URLDecoder.decode(url.getPath(),
"UTF-8")).getCanonicalFile()
...

[etorreborre]

Cool, I'll try that. Thanks!

On Mar 19, 6:59 pm, David Bernard <david.bernard...@gmail.com> wrote:
> a code fragment that you could use to find a jar in classpath :
>
>     var cl = classLoader.getOrElse(Thread.currentThread.getContextClassLoader())
>     while (cl != null) {
>       if (cl.isInstanceOf[URLClassLoader]) {
>         val ucl = cl.asInstanceOf[URLClassLoader]
>         ucl.getURLs().foreach {url =>
>           if (url.getProtocol().equals("file")) {
>             val file = new File(URLDecoder.decode(url.getPath(),
> "UTF-8")).getCanonicalFile()
> ...
>

[Josh Suereth]

Yes, when embedding the interpreter you either need to pass in the
classpath, or override one of the classloader methods(although I don't
think the later is supported)

Also, feel free to upgrade to maven-scala-plugin 2.10.1

Sent from my iPhone

On Mar 19, 2009, at 2:56 AM, David Bernard <david.bernard.

[Mark Harrah]

If I understand the problem correctly, the following should also work:

new File(classOf[ScalaObject].getProtectionDomain.getCodeSource.getLocation.toURI)

It gets the jar file containing ScalaObject.

-Mark

[Josh Suereth]

Neato!

Sent from my iPhone

[Josh Suereth]

Mark, does that trick work within an OSGi container?

[Mark Harrah]

Hey Josh,

I've not used OSGi, so I'll have to give a general answer. It depends
on how OSGi loads classes. If OSGi uses a SecureClassLoader and
provides a CodeSource instance for the original jar to
SecureClassLoader.defineClass, then I think it should work. If OSGi
masks the true source and provides a null CodeSource to defineClass,
it won't work.

I'd just try it and see if it works.

-Mark

[michid]

> Mark, does that trick work within an OSGi container?

Not with Apache Felix (http://felix.apache.org/).

I embedded Scala with Apache Sling (http://incubator.apache.org/)
which uses Felix. To get it working I had to extend the Scala Compiler
such that it can load classed from the class loader provided by the
OSGi container. Not sure how portable this is though. See
http://svn.apache.org/repos/asf/incubator/sling/trunk/contrib/scripting/scala/

Michael

[Mark Harrah]

This is a problem with the custom class loader,
AbstractFileClassLoader, used in the interpeter. It does not specify
the CodeSource/ProtectionDomain of the classes loaded, although this
could probably be addressed. Therefore, I would not expect the
technique to work in the interpreter even outside an OSGi container.
The scala runner uses a URLClassLoader and doesn't have a problem.

For what it's worth, note that because of this all libraries loaded by
the interpreter will go into the default protection domain. A
security policy using codeBase or signedBy will not work properly.

-Mark

>
> Michael

[Mark Harrah]

I don't know if the following is a good test, but I added:

System.out.println(getClass().getProtectionDomain().getCodeSource());

to the start method of the Apache Felix Tutorial Example 1[*] and it
prints the correct URL of the jar when run in Felix.

-Mark

[*] http://felix.apache.org/site/apache-felix-tutorial-example-1.html

On 3/19/09, Josh Suereth <joshua....@gmail.com> wrote:

[michid]

Yes it work sometimes. But it doesn't do always. Felix adds various
specific class loaders and it seems that some of them return null for

getClass().getProtectionDomain().getCodeSource());

at least when I last checked ;-)

Michael

On Mar 19, 7:01 pm, Mark Harrah <dmhar...@gmail.com> wrote:
> I don't know if the following is a good test, but I added:
>
> System.out.println(getClass().getProtectionDomain().getCodeSource());
>
> to the start method of the Apache Felix Tutorial Example 1[*] and it
> prints the correct URL of the jar when run in Felix.
>
> -Mark
>
> [*]http://felix.apache.org/site/apache-felix-tutorial-example-1.html
>

> On 3/19/09, Josh Suereth <joshua.suer...@gmail.com> wrote:
>
> > Mark, does that trick work within an OSGi container?
>

> > On Thu, Mar 19, 2009 at 7:12 AM, Mark Harrah <dmhar...@gmail.com> wrote:
>
> >> If I understand the problem correctly, the following should also work:
>
> >> new
> >> File(classOf[ScalaObject].getProtectionDomain.getCodeSource.getLocation.toURI)
>
> >> It gets the jar file containing ScalaObject.
>
> >> -Mark
>

> >> On 3/19/09, David Bernard <david.bernard...@gmail.com> wrote:
>
> >> > a code fragment that you could use to find a jar in classpath :
>
> >> >     var cl =
> >> > classLoader.getOrElse(Thread.currentThread.getContextClassLoader())
> >> >     while (cl != null) {
> >> >       if (cl.isInstanceOf[URLClassLoader]) {
> >> >         val ucl = cl.asInstanceOf[URLClassLoader]
> >> >         ucl.getURLs().foreach {url =>
> >> >           if (url.getProtocol().equals("file")) {
> >> >             val file = new File(URLDecoder.decode(url.getPath(),
> >> > "UTF-8")).getCanonicalFile()
> >> > ...
>

> >> > On Thu, Mar 19, 2009 at 10:30, etorreborre <etorrebo...@gmail.com>

[etorreborre]

Hi all,

I tried David B. suggestion of get the full classpath with the code
snippet he sent me and it worked ok when I also added to my Maven
surefire configuration:

<useSystemClassLoader>false</useSystemClassLoader>

I also tried this neat trick: new File(classOf
[ScalaObject].getProtectionDomain.getCodeSource.getLocation.toURI)

but it gave me only the location of one class while I needed all jars,
not only scala-library.

Thanks a lot for your support!

Eric.