Faster non-UI Robolectric tests

[Nikita Borodikhin]

Hi,

First of all, thank you guys for the project, it really helps with testing while developing for Android.

Robolectric is great, it provides reasonably full Android implementation, and it makes testing much easier than just relying on instrumentation tests.

However, Android is two-fold - it is both a library (TextUtils and so on) and a runtime environment (providing resource management and so on).

With proper design approach, applications are structured in the way that the layer which depends on runtime behavior (let's vaguely call it a "View" layer) is separated from the runtime-agnostic layer (let's say "Controller" layer).

View layer uses Android as a runtime environment, and Robolectric really shines when you are testing View layer.  It sets up Context, resources, Application, Activity, everything you need.  It is great, and is blazing-fast comparing to instrumentation tests.

Controller layer does not use runtime environment, but it quite often uses Android as a library.  There are a lot of platform-optimized utilities like SparseArrays and TextUtils and non-UI classes like Bundles and so on.  There is no reason to not use them while writing applications for Android.

Robolectric is a natural way to go to for Controller layer tests (it has implementation of Android library, after all), but it no longer looks that impressive.  It takes too much startup time setting up application and resources (and all that is not needed for Controllers!), making Controller test time almost too high for comfortable fast testing and TDD-like development.

It is so much of a pain that people a looking for ways to live without Robolectric.  I can refer to a great talk:  https://www.slideshare.net/dpreussler/unit-testing-without-robolectric-droidcon-berlin-2016

There are generally a few ways to have fast tests without Robolectric:

* stick to plain JRE code or use java-only libraries (for example, Guava) instead of Android library

* abstract Android object creation into factories and mock them (e.g. IntentFactory from the talk)

* use PowerMock-style mocking for Android library

* use Robolectric SDK as a runtime library through great https://github.com/bjoernQ/unmock-plugin

They require much more work to isolate Android code from Java code, except the last one.  The last approach is a funny one.  It uses Robolectric-provided Android library without Robolectric.  It is the easiest one to use, as it just provides what developers expect from the platform, but it is a pity that they have to use a side plugin to be able to use Robolectric SDK for fast testing.  It would be great if Robolectric had a fast runner for plain Android SDK testing and a regular one for full runtime environment simulation.

Well, what's the difference between hypothetical fast runner and the regular one?  Where the time is spent and what corner can we cut to get there?  Robolectric spends much of startup time setting up parallel universe, which is not needed for fast tests anyway.  Can we strip it?  Sure - all the work is in beforeTest() and afterTest().

So, I came up with the following runner which strips parallel universe setup from Robolectric.  It cuts the time of first test method execution (which includes test class setup time) from 5.5 seconds to 2.1 seconds, which is a great improvement for a suite that took 5.6 seconds in the first place.

public class PlainAndroidRunner extends RobolectricTestRunner {
  public PlainAndroidRunner(Class<?> testClass) throws InitializationError {
    super(testClass);
  }

  @Override
  protected void afterTest(FrameworkMethod method, Method bootstrappedMethod) {
  }

  @Override
  protected void beforeTest(Sandbox sandbox, FrameworkMethod method, Method bootstrappedMethod) throws Throwable {
  }

  @Override
  protected org.robolectric.internal.SandboxTestRunner.HelperTestRunner getHelperTestRunner(Class bootstrappedTestClass) {
    try {
      return new SandboxTestRunner.HelperTestRunner(bootstrappedTestClass);
    } catch (InitializationError initializationError) {
      throw new RuntimeException(initializationError);
    }
  }

  @Override
  protected Config buildGlobalConfig() {
    return new Config.Builder().setManifest(Config.NONE).build();
  

public class PlainAndroidRunner extends RobolectricTestRunner {

  public PlainAndroidRunner(Class<?> testClass) throws InitializationError {

    super(testClass);

  }

  @Override

  protected void afterTest(FrameworkMethod method, Method bootstrappedMethod) {

  }

  @Override

  protected void beforeTest(Sandbox sandbox, FrameworkMethod method, Method bootstrappedMethod) throws Throwable {

  }

  @Override

  protected org.robolectric.internal.SandboxTestRunner.HelperTestRunner getHelperTestRunner(Class bootstrappedTestClass) {

    try {

      return new SandboxTestRunner.HelperTestRunner(bootstrappedTestClass);

    } catch (InitializationError initializationError) {

      throw new RuntimeException(initializationError);

    }

  }

  @Override

  protected Config buildGlobalConfig() {

    return new Config.Builder().setManifest(Config.NONE).build();

  }

}

I know it is not a perfect solution (I hate the usage of internals here), but it works and is surprisingly clean and easy to use.

So what do you think about this?  Does this kind of fast runner fit Robolectric vision?  Can I shave more time off test setup without losing Android SDK as a library?  Does it make sense to spend time trying to fit it into Robolectric itself?

Thank you for feedback,

- Nikita

public class PlainAndroidRunner extends RobolectricTestRunner {  public PlainAndroidRunner(Class<?> testClass) throws InitializationError {
    super(testClass);
  }

  @Override
  protected void afterTest(FrameworkMethod method, Method bootstrappedMethod) {
  }

  @Override
  protected void beforeTest(Sandbox sandbox, FrameworkMethod method, Method bootstrappedMethod) throws Throwable {
  }

  @Override
  protected org.robolectric.internal.SandboxTestRunner.HelperTestRunner getHelperTestRunner(Class bootstrappedTestClass) {
    try {
      return new SandboxTestRunner.HelperTestRunner(bootstrappedTestClass);
    } catch (InitializationError initializationError) {
      throw new RuntimeException(initializationError);
    }
  }

  @Override
  protected Config buildGlobalConfig() {
    return new Config.Builder().setManifest(Config.NONE).build();
  }
}

public class PlainAndroidRunner extends RobolectricTestRunner {
  public PlainAndroidRunner(Class<?> testClass) throws InitializationError {
    super(testClass);
  }

  @Override
  protected void afterTest(FrameworkMethod method, Method bootstrappedMethod) {
  }

  @Override
  protected void beforeTest(Sandbox sandbox, FrameworkMethod method, Method bootstrappedMethod) throws Throwable {
  }

  @Override
  protected org.robolectric.internal.SandboxTestRunner.HelperTestRunner getHelperTestRunner(Class bootstrappedTestClass) {
    try {
      return new SandboxTestRunner.HelperTestRunner(bootstrappedTestClass);
    } catch (InitializationError initializationError) {
      throw new RuntimeException(initializationError);
    }
  }

  @Override
  protected Config buildGlobalConfig() {
    return new Config.Builder().setManifest(Config.NONE).build();
  }
}

[igor.ga...@mjdinteractive.com]

This is an interesting idea.  Did you end up using in your production tests?