ScalaTest & Detecting Suites via the classpath vs -R

[Eric Peters]

Right now I'm having to do something like: ./runApp.sh -Dfm.domain_data_provider=test org.scalatest.tools.Runner -R "`cat $SCALATEST_RUNPATH_FILE`" -o $ARGS

(runApp.sh simply sets the class path/executes java with the parameters, and SCALATEST_RUNPATH_FILE has the paths for my compiled test classes)

I tried adding the test classes from the -R/SCALATEST_RUNPATH_FILE directly into the CLASSPATH in runApp.sh, (so my ./test.sh becomes something more like: ./runApp.sh -Dfm.domain_data_provider=test org.scalatest.tools.Runner -o)  and the discovery doesn't find any test suites

eric@fmoffice:~/frugalmechanic-scala/fm-orig$ ./test.sh 

DEBUG: Discovery Starting

DEBUG: Discovery Completed: 4 milliseconds

Run starting. Expected test count is: 0

DiscoverySuite:

Run completed in 208 milliseconds.

Total number of tests run: 0

Suites: completed 1, aborted 0

Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0

All tests passed.

The very first lines of my CLASSPATH are the test directories:

/home/eric/scala/fm-orig/target/scala-2.10/test-classes:/home/eric/scala/fm-util/target/scala-2.10/test-classes:/home/eric/scala/fm-model/target/scala-2.10/test-classes:....

If I remove these lines from the CLASSPATH and add them into the -R parameters, all of the tests are discovered and run fine.

Thoughts, am I doing this completely wrong?

Regards,

Eric

[Eric Peters]

Oh I should have mentioned in the scaladocs it says:

Specifying a runpath

A runpath is the list of filenames, directory paths, and/or URLs that Runner uses to load classes for the running test. If runpath is specified, Runner creates a custom class loader to load classes available on the runpath. The graphical user interface reloads the test classes anew for each run by creating and using a new instance of the custom class loader for each run. The classes that comprise the test may also be made available on the classpath, in which case no runpath need be specified.

The runpath is specified with the -R option. The -R must be followed by a space, a double quote ("), a white-space-separated list of paths and URLs, and a double quote. If specifying only one element in the runpath, you can leave off the double quotes, which only serve to combine a white-space separated list of strings into one command line argument. If you have path elements that themselves have a space in them, you must place a backslash (\) in front of the space. Here's an example:

I'm trying to shoot for the "The classes that comprise the test may also be made available on the classpath, in which case no runpath need be specified."

-Eric

[Bill Venners]

Hi Eric,

Let me peak at the code to refresh my memory on where we look during
discovery...

Bill

> --
> --
> You received this message because you are subscribed to the Google
> Groups "scalatest-users" group.
> To post to this group, send email to scalate...@googlegroups.com
> To unsubscribe from this group, send email to
> scalatest-use...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/scalatest-users?hl=en
> ScalaTest itself, and documentation, is available here:
> http://www.artima.com/scalatest
> ---
> You received this message because you are subscribed to the Google Groups
> "scalatest-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scalatest-use...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
Bill Venners
Artima, Inc.
http://www.artima.com

[Bill Venners]

Hi Eric,

OK, the missing piece of the story is that *discovery* only is
performed on the runpath. You can indeed run test classes that are on
the class path, but just not via discovery. To run test classes off
the class path, you'll have to name them explicitly with -s <suite
name> when you invoke Runner.

Is there some reason you don't want to use a runpath? By the way, the
main reason we don't perform discovery on the class path is to help
minimize the time required for discovery by focusing it just on
potential test classes. A secondary reason is to enable discovery to
be performed multiple times in the same JVM instance, and get
different results each time if things have been recompiled.

While we're on the subject, one way everyone out there can speed
discovery right now up is by using a -Q or -q option to enable Runner
to immediately ignore classes that don't follow your naming convention
for test classes (such as *Spec, *Suite, etc.). Before long I believe
we'll be doing an enhancement such that execution doesn't wait until
discovery is done. Currently we wait until discovery is done so that a
nice progress bar can be rendered. I think that should continue to be
available as an option in the future, but the default should be that
test execution starts in parallel with discovery.

Bill

[Eric Peters]

Mostly just making sure I won't shoot myself in the foot...

So I have some reflections that are typically in my main classpath files, where I create a bunch of classes that extend a class/trait and are all in the same package, I then have another  object create an array of all of the classes that were defined (basically a bunch of configuration settings I'm defining in classes)

For some tests, rather than using "production" configuration settings I'm trying to define test instances of those class in the appropriate package name/etc.

In production environment those test classes would never get included, so would never get discovered via reflection, but obviously necessary for test cases to run properly.

Based upon how the discovery works, will I run into any name collision problems, or is that all handled by URLCLassLoader?

I'm not really sure what the "proper" ScalaTest setup is, should classpath include both production classes and test class paths, and then specify via the commandline the runPath for just what to scan?  will that fubar anything?

or Should we JUST include the normal production classes on runpath and specify the test classes via runpath?

Cheers,

Eric

  private def configClasses = Vector() ++ ClassUtil.findImplementingClasses(classPackage, classTag[T].runtimeClass, classTag[T].runtimeClass.getClassLoader)

  Example: I have trait "Sites" where the Object " I'm adding some test-specific 

[Bill Venners]

Hi Eric,

Each "run" gets a new "runpath class loader," which delegates to the
class path class loader. So if a class isn't found on the runpath, the
JVM will look on the class path. So if I understand you're goal
correctly, you should be fine doing it the recommended way, which is
what I think you were trying to say last:

> or Should we JUST include the normal production classes on runpath and
> specify the test classes via runpath?

I think you meant: production classes on the classpath and test
classes on the runpath. That's how it is intended to be used. You can
put production classes on the runpath too. It would work, but it would
slow down discovery. Let me know if you have any more troubles trying
that approach.

Bill

[Eric Peters]

That's exactly the problem I'm running into, I'm crossing streams :)

Basically I have some code in my runPath that my CLASSPATH needs when it runs the suites in my runPath

It works if I put my runPath into my CLASSPATH, but that may not be the most desirable behavior...

Hmmm...

[Bill Venners]

Hi Eric,

On Fri, Feb 1, 2013 at 5:04 PM, Eric Peters <ericp...@gmail.com> wrote:
> That's exactly the problem I'm running into, I'm crossing streams :)
>
> Basically I have some code in my runPath that my CLASSPATH needs when it
> runs the suites in my runPath
>

I see. Hmm. So your production code is reflectively loading stuff from
your test classes? If so I think it should work fine to include the
test classes both on the class path and on the runpath.

> It works if I put my runPath into my CLASSPATH, but that may not be the most
> desirable behavior...
>

Ah, good, that works. Yes, that should be fine. The test classes will
be discovered off filenames in the runpath, but then the ones that
actually get loaded will be off the class path. The reason is that the
URLClassLoader always delegates first to its parent, the class path
class loader, and that one will always find them. So you will only
ever actually load classes off the class path. The runpath will only
be used to allow ScalaTest to find out the names of classes to load.

Bill

[Eric Peters]

OK, We'll throw in the test classes into the class path and hope we don't run into any future problems :)

It helps we're getting a pseudo blessing from the man himself, you promise we won't get fucked later? :)

-Eric

[Bill Venners]

Hi Eric,

On Fri, Feb 1, 2013 at 5:23 PM, Eric Peters <ericp...@gmail.com> wrote:
> OK, We'll throw in the test classes into the class path and hope we don't
> run into any future problems :)
>
> It helps we're getting a pseudo blessing from the man himself, you promise
> we won't get fucked later? :)
>

I promise! (but I'm not showing whether my fingers are crossed behind
my back.) The other way you could do is, by the way, is plop
production and test code on the runpath. Now that I think about it,
that's probably a nicer way to do it if it doesn't slow down discovery
too much. I might try it both ways and measure if discovery is
significantly slower with production code on the runpath. Especially
if you use -Q or -q, it may not be significant, and then you
definitely should be fine in the future.

Bill