Running tests in Glean

[Evgeny]

Greetings All,

I have a build file for my project that has a "test" target, in that target I execute junit with an appropriate
classpath, several sysproperties (for the logger), and some additional customized information.

There are at least two tools in Glean that also run tests as part of the Glean suit of reports, which is
emma and cobertura. It's great that it works and all, but is there no way to "subant" my own target
that runs my tests -- that way I won't need to change glean test.xml ....



Regards,

Evgeny
Israel

[John Brugge]

Evgeny,

The unit testing macro for Glean does have its limitations, but if there are ways we could improve it, I'm all ears.

One thing to consider is that the testing process run by Glean need not be identical to the one you run during your normal build. The goal of the unit tests for Glean is for reporting and providing data to some other tools (such as Emma or Cobertura). So, for instance, if the system properties you normal pass in to support the logger don't affect the outcome of the tests, then as far as the Glean tools are concerned, it doesn't matter.

For the classpath you provide to the tests, how particular is that? The Glean test macro allows for a property called "extra.test.path" that you could override. It is a "path" element, (<pathelement path="${extra.test.path}"/>) so you could define it with a full set of locations if you needed. It is currently placed at the end of the larger test classpath, but I don't remember offhand is that's significant or not; I could look at what would happen if it were at the beginning of the path, of that would make a difference to you.

For the other customized information you mentioned, how important are they to the outcome of the tests? And what type of settings on the junit task are these?

[Evgeny]

From what I understand, emma and cobertuna do some instrumentation to the classes. But then after it's done, they don't really care how you execute your junit tests. As long as the instrumentation is in place -- the statistics cobertura needs are dumped somewhere for it to analyze.

Right now glean has it's own gleat-test macro that runs a junit, so I was wondering how hard would it be to replace that with "call my own build with target xtest" or something.

That's the general idea I was thinking of. But since I am not an expert and didn't have time to dig really deep into this, I don't know if it's possible at all. If it IS possible, then having a "default" glean testing macro is nice - but having an option to somehow override it with whatever it is that the project is using can be extremely nice.

[Evgeny]

Currently the project I am trying to use Glean with has this as the junit target :
http://pastie.caboo.se/158249

One of the most important things in there is actually the "timeout=300" ... because I fail to educate our programmers on the importance of separation. Separation of unit-tests that run fast and are suitable for continuous integration, and integration tests that require jboss and mysql up and running and the application deployed to work. And some of the test just "do something" indefinetely, especially when there is no timeout set. They just go-on forever.

[John Brugge]

Evgeny,

Let me look into this. The main thing that the coverage tools need is the
instrumented classes placed ahead of regular classes on the classpath, and
some custom JVM args.

I'll see what I can do.


John