"Optional" dependency hack, and its drawbacks

[Dan Fabulich]

I wanted to write this down somewhere, so rather than blogging it I
decided to write it here. :-)

Executive summary: You can very nearly get "optional dependencies" using
TestNG 5.7, by a funky use of dependsOnGroups. It has a weird test
ordering drawback though: it will run all @BeforeClass methods (from all
classes) and then start running tests.

1) What are optional dependencies?
2) How does the hack work?
3) What's the catch?

WHAT ARE OPTIONAL DEPENDENCIES?

As I've ranted before in this group, I've been trying to arrange to have
what I'm (now) calling "optional dependencies" in TestNG. [Previously
I've also called these "weak" dependencies, but "weak" sounds too much
like "soft."]

Optional dependencies are different from "soft" dependencies
(alwaysRun=true) because if an optional dependency fails, the optionally
dependent method will be skipped, just like a regular "hard" dependency.
The difference is if the optional dependency is missing, the dependent
method will still get run.

(Optional dependency is a useful performance feature for slow tests, where
you want to skip as much as possible. Skipping dependents is great, but
hard dependencies force you to rerun them [they are "implicitly included"]
even when you're just trying to rerun failures in dependent methods, which
can be a waste of time if the dependency is slow. You'd use optional
dependencies when feature X depends on feature Y, but test X doesn't need
test Y to run; if Y fails, skip X, but you still want to run X in
isolation sometimes.)

HOW DOES THE HACK WORK?

Make an abstract parent class with an empty method belonging to the test
group, like this:

--
public abstract class AbstractParent {
@Test(groups={"basic"}) public void empty() {} // empty!
}
--
public class TestBasic {
@Test(groups={"basic"}) public void basic() {
MyClass.basicStuff();
}
}
--
@Test(dependsOnGroups={"basic"}) public class TestAdvanced extends AbstractParent {
public void advanced() {
MyClass.advancedStuff();
}
}
--

When you run TestAdvanced by itself, it will run "empty()" (doing nothing)
and then run the "advanced()" method. When you run TestAdvanced together
with TestBasic, it will run "empty()" as well as "basic()"; if the basic
test fails, the advanced test will be skipped.

WHAT'S THE CATCH?

Wonderful, no? Only one problem. Suppose you've got multiple advanced
tests, with @BeforeClass methods:

--
@Test(dependsOnGroups={"basic"}) public class TestAdvanced1 extends AbstractParent {
@BeforeClass public void setUp1() {}
public void advanced1() {
MyClass.advancedStuff();
}
}
--
@Test(dependsOnGroups={"basic"}) public class TestAdvanced2 extends AbstractParent {
@BeforeClass public void setUp2() {}
public void advanced2() {
MyClass.advancedStuff();
}
}
--

Well, you're in trouble if you try running both of these together, because
the order will look like this:

TestAdvanced1.setUp1
TestAdvanced1.empty
TestAdvanced2.setUp2
TestAdvanced2.empty
TestAdvanced1.advanced1
TestAdvanced2.advanced2

That is, all of the setUp methods will run, then all of the tests will
run. That's because the _entire_ "basic" group (including every "empty"
method) has to get run before *any* of the advanced tests can run; to do
that, every @BeforeClass method has to run first.

This is a big problem if you're running a UI test and you're expecting
@BeforeClass to setup your UI state (e.g. setUp1 navigates to http://url1
and setUp2 navigates to http://url2). Normally you'd expect your methods
to run immediately after @BeforeClass; in this case, not so much.

I was hoping to use this to avoid having to code up "real" optional
dependencies in TestNG, but since I mostly use TestNG for UI testing
(specifically Selenium testing,) it's clear that I'm out of luck.
Hopefully this will help *someone*, even if it won't work for me.

-Dan