Using Scala Interpreter in Unit Tests with sbt

[Ben Jackman]

Hi,

I am having a pretty hard time getting the Scala Interpreter class to
run with in my unit tests. I am attempting to instantiate a
scala.tools.nsc.Interpreter, because we use an interpreter within our
program to do dynamic code compilation. Unfortunately it seems that
sbt is not setting the "java.class.path" environmental variable before
running the tests. When I run with maven i see a rather lengthy
classpath returned, when I run ~test in sbt i only see:

/opt/sbt/sbt-launcher-current.jar

The scala interpreter uses the "java.class.path" setting to pass onto
the compiler it creates so that the compiler can find needed files
when compiling. Since it is missing the scala compiler that is made by
the interpreter we are running in our program is unable to see any of
the other source code in our project.

Is there anyworkaround to this issue that anyone can suggest?

~Ben

[Mark Harrah]

Hi Ben,

The usual way of configuring the interpreter is to work with a Settings
instance. Assuming you have created your Settings object 'settings',

settings.bootclasspath.value = bootClasspathString
settings.classpath.value = classpathString

bootClasspathString should contain scala-library.jar and Java boot classpath.
classpathString should be the project classpath.

-Mark

[Ben Jackman]

I guess I am having a hard time getting the classpathString into the
tests being run by sbt.

should I specify this in my Project.scala file somewhere?

I am unsure of how to have sbt set the classpath variable that can
then be read within the unit tests.

I understand how to modify the scala.tools.nsc.Settings variable that
the I use when I instantiate the interpreter, I just don't know where
I can read the classpath from inside the program when running in sbt.
In maven, maven sets the "java.class.path" variable before running a
unit test, I can then get the classpath with a call to
System.getProperties, sbt doesn't seem to set it before running a unit
test, there the "java.class.path" variable is just /opt/sbt/sbt-
launcher-current.jar which is the original classpath variable that sbt
had when it stared.

~Ben

[Ben Jackman]

I'll try to be little more clear, supposing I make a unit test and run
it with sbt
by saying test
in that unit test I
println(System.getProperty("java.class.path"))
all that is returned is
/opt/sbt/sbt-launcher-current.jar
rather than all of the dependencies of that unit test.
this is also an issue for the (console / run) sbt commands

when running a console in sbt:
[info] Starting scala interpreter...
[info]
Welcome to Scala version 2.7.7.final (Java HotSpot(TM) 64-Bit Server
VM, Java 1.6.0_15).
Type in expressions to have them evaluated.
Type :help for more information.

scala> System.getProperty("java.class.path")
res0: java.lang.String = /opt/sbt/sbt-launcher-current.jar

and when running a console just by typing scala at the bash command
line:
$ scala
Welcome to Scala version 2.7.5.final (Java HotSpot(TM) 64-Bit Server
VM, Java 1.6.0_15).
Type in expressions to have them evaluated.
Type :help for more information.

scala> System.getProperty("java.class.path")
res0: java.lang.String =/opt/scala/default/lib/sbaz.jar:/opt/scala/
default/lib/sbaz-tests.jar:/opt/scala/default/lib/scala-compiler.jar:/
opt/scala/default/lib/scala-dbc.jar:/opt/scala/default/lib/scala-
library.jar:/opt/scala/default/lib/scala-swing.jar

without having the system property set it makes it hard within a
running program to get an idea of what the valid classpath is for that
program. I think that before running code sbt needs to set that
"java.class.path" property with the proper paths. I understand that it
makes it's own classloader and that works great for most things,
however when you need to explore the classpath for your application
you can't really go through classloaders to enumerate the
dependencies.

[Mark Harrah]

Hi Ben,

Maven probably runs tests in a separate jvm, which is probably why
'java.class.path' is set. sbt doesn't fork tests, so 'java.class.path' is as
you see it. I'm a bit surprised that 'java.class.path' is writable, but this
means you can always set it yourself. You can't run multiple projects in
parallel of course, which is why sbt can't do it.

I think the underlying issue is easily passing information from the build to
tests.

[Ben Jackman]

Ok I understand now, what is the best way to pass information from sbt
to the tests?

[Mark Harrah]

I guess I meant it as an open question. Is this something that test
frameworks should be responsible for? Do test frameworks have an
existing way to pass options to tests? Or, do people have to write
resource files that tests read (which is what I do)?

-Mark

> --
> You received this message because you are subscribed to the Google Groups
> "simple-build-tool" group.
> To post to this group, send email to simple-b...@googlegroups.com.
> To unsubscribe from this group, send email to
> simple-build-t...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/simple-build-tool?hl=en.
>
>

[Stefan Langer]

This would be a nice feature for testing plugin projects programmatically. This would keep the burden of having to write tests scripts low if you could simply access the projects settings from your test cases.

-Stefan

2010/2/8 Mark Harrah <dmha...@gmail.com>

[Mark Harrah]

Hi Stefan,

Do you have an idea of what this would look like?

-Mark

>> > simple-build-t...@googlegroups.com<simple-build-tool%2Bunsu...@googlegroups.com>

>> .
>> > For more options, visit this group at
>> > http://groups.google.com/group/simple-build-tool?hl=en.
>> >
>> >
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "simple-build-tool" group.
>> To post to this group, send email to simple-b...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> simple-build-t...@googlegroups.com<simple-build-tool%2Bunsu...@googlegroups.com>

[Josh Cough]

I haven't really been following at all, I've been off in space mostly, but is this still a question that people need an answer too/documentation for:

> Ok I understand now, what is the best way to pass information from sbt to the tests?

Josh

[Stefan Langer]

Not really, but what I would love to have is that you can access the variables you are used to accessing in your task from within your test either through special classes that you would need to implement or through some object which you can obtain. Stuff like dependencies specified,  sbtVersions the path to the resources, build, etc....

[Ben Jackman]

It thinks it's going to be tricky because of the parallel execution
within the same jvm issues, perhaps a threadlocal variable that
contains information about the environment from sbt? This will keep
things safe for when many things are being run simultaneously. Perhaps
before running a task sbt could set the variable with all of it's
data? Then in the internal code calls sbt.getCurrentConfiguration :
Option[SbtRuntimeConfiguration] and proceeds accordingly.
Of course it would be nice if the size of the jar that needs to be
embedded in the program is as minimal as possible. I realize that all
this isn't trivial ...

~Ben

On Feb 8, 9:51 am, Stefan Langer <mailtolan...@googlemail.com> wrote:
> Not really, but what I would love to have is that you can access the
> variables you are used to accessing in your task from within your test
> either through special classes that you would need to implement or through
> some object which you can obtain. Stuff like dependencies specified,
> sbtVersions the path to the resources, build, etc....
>
> -Stefan
>

> 2010/2/8 Mark Harrah <dmhar...@gmail.com>

>
>
>
> > Hi Stefan,
>
> > Do you have an idea of what this would look like?
>
> > -Mark
>

> > On 2/8/10, Stefan Langer <mailtolan...@googlemail.com> wrote:
> > > This would be a nice feature for testing plugin projects
> > programmatically.
> > > This would keep the burden of having to write tests scripts low if you
> > could
> > > simply access the projects settings from your test cases.
>
> > > -Stefan
>

> > > 2010/2/8 Mark Harrah <dmhar...@gmail.com>

>
> > >> I guess I meant it as an open question.  Is this something that test
> > >> frameworks should be responsible for?  Do test frameworks have an
> > >> existing way to pass options to tests?  Or, do people have to write
> > >> resource files that tests read (which is what I do)?
>
> > >> -Mark
>

> > >> > simple-build-t...@googlegroups.com<simple-build-tool%2Bunsubscr i...@googlegroups.com>
> > <simple-build-tool%2Bunsu...@googlegroups.com<simple-build-tool%252Buns ubsc...@googlegroups.com>

>
> > >> .
> > >> > For more options, visit this group at
> > >> >http://groups.google.com/group/simple-build-tool?hl=en.
>
> > >> --
> > >> You received this message because you are subscribed to the Google
> > Groups
> > >> "simple-build-tool" group.
> > >> To post to this group, send email to simple-b...@googlegroups.com
> > .
> > >> To unsubscribe from this group, send email to

> > >> simple-build-t...@googlegroups.com<simple-build-tool%2Bunsubscr i...@googlegroups.com>
> > <simple-build-tool%2Bunsu...@googlegroups.com<simple-build-tool%252Buns ubsc...@googlegroups.com>

>
> > >> .
> > >> For more options, visit this group at
> > >>http://groups.google.com/group/simple-build-tool?hl=en.
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "simple-build-tool" group.
> > > To post to this group, send email to simple-b...@googlegroups.com.
> > > To unsubscribe from this group, send email to

> > > simple-build-t...@googlegroups.com<simple-build-tool%2Bunsubscr i...@googlegroups.com>

> > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/simple-build-tool?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "simple-build-tool" group.
> > To post to this group, send email to simple-b...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > simple-build-t...@googlegroups.com<simple-build-tool%2Bunsubscr i...@googlegroups.com>

[Mark Harrah]

On Monday 08 February 2010, Josh Cough wrote:
> I haven't really been following at all, I've been off in space mostly, but
> is this still a question that people need an answer too/documentation for:

Yes, how can we associate key/value pairs with a test from the build and then access those in the test?

-Mark

[Mark Harrah]

The biggest problem here is that even if your tests had access to the project object, your build definition and sbt use one version of Scala and the tests use another.

-Mark

[Stefan Langer]

I haven't really looked into the Project class itself but how hard would it be to set up a mock like Project which only contains the scala version of the project and the base path of the project? This way at least small tests that do not need the full blown build environment but need access to the test resources could obtain them in a easy to use way.

I currently have a use case where I'm unit testing a class for my eclipsify project for which I would like to grab a few resources from the test/resources folder. But obtaining that folder isn't quite as easy as I would have hoped for. I currently help my self by setting up a test script and then doing the complete build but this isn't quite as elegant as having a unit test in my opinion.

-Stefan

2010/2/9 Mark Harrah <dmha...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
To post to this group, send email to simple-b...@googlegroups.com.

[Mark Harrah]

Test resources should be available on the test classpath, so you should be able to access them with Class.getResource.

-Mark

[Stefan Langer]

I actually generate files from those resources in the root of the project which I need to access. But since I can get a reference to the resource folder I should be able to determine the root of the project from a url generated for those resources. Maybe the requirement isn't as important as I first thought since I should be able to calculate the project root and once you have that you can easily access any of the other folders.

Thanks Mark for the reminder

-Stefan

2010/2/9 Mark Harrah <dmha...@gmail.com>

[Mark Harrah]

If you declare those resources, they will go on the classpath as well:

def yourResources = ("a" / "b") +++ "c" +++ ("d" * "*.txt")
override def testResources = super.testResources +++ yourResources

[Stefan Langer]

Even though they do not exist before the test starts?

2010/2/9 Mark Harrah <dmha...@gmail.com>

[Josh Cough]

All, I've written up almost everything I currently know about passing arguments to test frameworks in sbt. I'll be happy to add it to the wiki as well, but for now its here: http://jackcoughonsoftware.blogspot.com/2010/02/sbt-and-test-arguments.html

Please feel free to provide feedback, it will help.

[Doug Tangren]

awesome. I'll definitely read this tonight. I got some tests that can use something like that.

-Doug Tangren
http://lessis.me

[Ben Jackman]

Thanks Josh!

I also will be looking into this asap. Hopefully now I can resume
running my unit tests in parallel.

On Feb 24, 12:00 pm, Josh Cough <joshco...@gmail.com> wrote:
> All, I've written up almost everything I currently know about passing
> arguments to test frameworks in sbt. I'll be happy to add it to the wiki as

> well, but for now its here:http://jackcoughonsoftware.blogspot.com/2010/02/sbt-and-test-argument...

>
> Please feel free to provide feedback, it will help.
>
> On Tue, Feb 9, 2010 at 7:10 AM, Stefan Langer

> <mailtolan...@googlemail.com>wrote:

>
>
>
> > Even though they do not exist before the test starts?
>

> > 2010/2/9 Mark Harrah <dmhar...@gmail.com>

>
> >> If you declare those resources, they will go on the classpath as well:
>
> >>  def yourResources = ("a" / "b") +++ "c" +++ ("d" * "*.txt")
> >>  override def testResources = super.testResources +++ yourResources
>
> >> -Mark
>
> >> On Tuesday 09 February 2010, Stefan Langer wrote:
> >> > I actually generate files from those resources in the root of the
> >> project
> >> > which I need to access. But since I can get a reference to the resource
> >> > folder I should be able to determine the root of the project from a url
> >> > generated for those resources. Maybe the requirement isn't as important
> >> as I
> >> > first thought since I should be able to calculate the project root and
> >> once
> >> > you have that you can easily access any of the other folders.
>
> >> > Thanks Mark for the reminder
>
> >> > -Stefan
>

> >> > 2010/2/9 Mark Harrah <dmhar...@gmail.com>

> >> > > > 2010/2/9 Mark Harrah <dmhar...@gmail.com>

>
> >> > > > > The biggest problem here is that even if your tests had access to
> >> the
> >> > > > > project object, your build definition and sbt use one version of
> >> Scala
> >> > > and
> >> > > > > the tests use another.
>
> >> > > > > -Mark
>
> >> > > > > On Monday 08 February 2010, Stefan Langer wrote:
> >> > > > > > Not really, but what I would love to have is that you can access
> >> the
> >> > > > > > variables you are used to accessing in your task from within
> >> your
> >> > > test
> >> > > > > > either through special classes that you would need to implement
> >> or
> >> > > > > through
> >> > > > > > some object which you can obtain. Stuff like dependencies
> >> specified,
> >> > > > > > sbtVersions the path to the resources, build, etc....
>
> >> > > > > > -Stefan
>

> >> > > > > > 2010/2/8 Mark Harrah <dmhar...@gmail.com>

>
> >> > > > > > > Hi Stefan,
>
> >> > > > > > > Do you have an idea of what this would look like?
>
> >> > > > > > > -Mark
>

> >> > > > > > > On 2/8/10, Stefan Langer <mailtolan...@googlemail.com> wrote:
> >> > > > > > > > This would be a nice feature for testing plugin projects
> >> > > > > > > programmatically.
> >> > > > > > > > This would keep the burden of having to write tests scripts
> >> low
> >> > > if
> >> > > > > you
> >> > > > > > > could
> >> > > > > > > > simply access the projects settings from your test cases.
>
> >> > > > > > > > -Stefan
>

> >> > > > > > > > 2010/2/8 Mark Harrah <dmhar...@gmail.com>

>
> >> > > > > > > >> I guess I meant it as an open question.  Is this something
> >> that
> >> > > test
> >> > > > > > > >> frameworks should be responsible for?  Do test frameworks
> >> have
> >> > > an
> >> > > > > > > >> existing way to pass options to tests?  Or, do people have
> >> to
> >> > > write
> >> > > > > > > >> resource files that tests read (which is what I do)?
>
> >> > > > > > > >> -Mark
>

> ...
>
> read more »

[Josh Cough]

Thanks Guys. I've sort of been out of the loop so please let me know if things are missing that you need.

One thing is running tests in parallel. Ben, how were you doing that before, just via sbt? I'm wondering if its a feature I'm missing in ScalaTest argument parsing that I'll need to enable.

Also, if there's any other way people see to make this stuff better, please let me know.

--