Unable to run Android instrumentation tests

[shi...@google.com]

Hello!

I am experimenting with the instrumentation test support (neat!).

I followed the "Building Android with Bazel" tutorial and extended the sample project with a basic instrumentation test. I hit the following when I tried running the test. I verified that I was able to launch the device manually with bazel run. Any pointers will be much appreciated.

Thanks,
Billy

WARNING: AndroidDeviceTestSuite Native suite creation failed!
.E
Time: 0.006
There was 1 failure:
1) initializationError(com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite)
java.lang.IllegalStateException: Adb call failed. Exit Code: 1. Args: [/usr/local/google/home/shiufai/.cache/bazel/_bazel_shiufai/b109b1a583e0a6bd274fc08d20edfd44/sandbox/linux-sandbox/2/execroot/__main__/bazel-out/k8-fastbuild/bin/mediarecorder/javatests/com/example/android/mediarecorder/perf_test.runfiles/__main__/../androidsdk/platform-tools/adb, -s, localhost:53029, shell, chmod, 666, /dev/graphics/fb0, /dev/fb0]
at com.google.common.base.Preconditions.checkState(Preconditions.java:698)
at com.google.android.apps.common.testing.broker.AdbController.makeCheckedCall(AdbController.java:844)
at com.google.android.apps.common.testing.broker.AdbController.makeCheckedCall(AdbController.java:835)
at com.google.android.apps.common.testing.broker.AdbController.makeAdbCall(AdbController.java:831)
at com.google.android.apps.common.testing.broker.AdbController.makeAdbCall(AdbController.java:826)
at com.google.android.apps.common.testing.broker.WrappedEmulatedDeviceBroker.leaseDevice(WrappedEmulatedDeviceBroker.java:294)
at com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite$Builder.makeNativeTestSuite(AndroidDeviceTestSuite.java:140)
at com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite$Builder.build(AndroidDeviceTestSuite.java:103)
at com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite.suite(AndroidDeviceTestSuite.java:64)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.internal.runners.SuiteMethod.testFromSuiteMethod(SuiteMethod.java:35)
at org.junit.internal.runners.SuiteMethod.<init>(SuiteMethod.java:24)
at org.junit.internal.builders.SuiteMethodBuilder.runnerForClass(SuiteMethodBuilder.java:11)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
at com.google.testing.junit.runner.internal.junit4.MemoizingRequest.createRunner(MemoizingRequest.java:48)
at com.google.testing.junit.runner.internal.junit4.MemoizingRequest.getRunner(MemoizingRequest.java:34)
at com.google.testing.junit.runner.junit4.JUnit4TestModelBuilder.get(JUnit4TestModelBuilder.java:51)
at com.google.testing.junit.runner.junit4.JUnit4TestModelBuilder.get(JUnit4TestModelBuilder.java:30)
at com.google.testing.junit.runner.util.MemoizingSupplier.get(MemoizingSupplier.java:42)
at com.google.testing.junit.runner.junit4.JUnit4Runner.run(JUnit4Runner.java:92)
at com.google.testing.junit.runner.BazelTestRunner.runTestsInSuite(BazelTestRunner.java:144)
at com.google.testing.junit.runner.BazelTestRunner.main(BazelTestRunner.java:82)

[Jingwen Chen]

Hey Billy, do you have a reproducible example / repository I can debug this with? 

Also: could you please try the samples in https://github.com/googlesamples/android-testing (e.g. `bazel test //ui/espresso/BasicSample:BasicSampleInstrumentationTest_23`) and see if that works? 

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/e89a3654-0163-4b7c-bec9-472cf9305904%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Billy Lam]

Hi Jingwen,

Thanks for the prompt reply.

The sample you linked worked fine. I did a quick comparison and found that the issue had to do with the device I was using (generic_phone:google_26_x86_qemu2). Is it a known issue with device API > 23?

So I tried switching to android_23_x86_qemu2 and was able to get a bit further, but now I am hitting the error below.

Sorry I am quite new to bazel and android expresso test still, so it might be something simple, but I couldn't immediately see the difference between my test vs the sample. I have attached my test project, simply run "bazel test ..." should repro.

Thanks for your help!

Billy

1) ExampleInstrumentedTest-activityTest(com.google.android.apps.common.testing.suite.AndroidGoogleTest)

java.lang.RuntimeException: Something went wrong during test instrumentation execution (check "logcat-ExampleInstrumentedTest-activityTest.txt" file below to see if your test threw any uncaught exceptions).

Debug Info: Instrumentation results are expected to have only 1 test, with two parts: Test started, and Test completed (passed, failed, errored).

Test method: com.example.android.mediarecorder.ExampleInstrumentedTest#activityTest

Test output: []

at com.google.android.apps.common.testing.suite.AndroidGoogleTest.runGoogleAndroidTestCase(AndroidGoogleTest.java:320)

at com.google.android.apps.common.testing.suite.AndroidGoogleTest.runTest(AndroidGoogleTest.java:393)

at junit.framework.TestCase.runBare(TestCase.java:141)

at junit.framework.TestResult$1.protect(TestResult.java:122)

at junit.framework.TestResult.runProtected(TestResult.java:142)

at junit.framework.TestResult.run(TestResult.java:125)

at junit.framework.TestCase.run(TestCase.java:129)

at junit.framework.TestSuite.runTest(TestSuite.java:252)

at junit.framework.TestSuite.run(TestSuite.java:247)

at junit.framework.TestSuite.runTest(TestSuite.java:252)

at junit.framework.TestSuite.run(TestSuite.java:247)

at junit.framework.TestSuite.runTest(TestSuite.java:252)

at junit.framework.TestSuite.run(TestSuite.java:247)

at junit.framework.TestSuite.runTest(TestSuite.java:252)

at junit.framework.TestSuite.run(TestSuite.java:247)

at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86)

at com.google.testing.junit.runner.internal.junit4.CancellableRequestFactory$CancellableRunner.run(CancellableRequestFactory.java:89)

at org.junit.runner.JUnitCore.run(JUnitCore.java:137)

at org.junit.runner.JUnitCore.run(JUnitCore.java:115)

at com.google.testing.junit.runner.junit4.JUnit4Runner.run(JUnit4Runner.java:112)

at com.google.testing.junit.runner.BazelTestRunner.runTestsInSuite(BazelTestRunner.java:144)

at com.google.testing.junit.runner.BazelTestRunner.main(BazelTestRunner.java:82)

[Billy Lam]

Quick update. So I got the android_23_x86_qemu2 config to work, turns out (I think) I was using the wrong AndroidJunitRunner, so I am semi-unblocked for now.

I am still running into the error related to the generic_phone:google_26_x86_qemu2 config too. Please let me know if you were able to debug and find out what's wrong..

Thanks,

Billy

[Jingwen Chen]

Good to hear! 

Do you mind filing a bug on https://github.com/bazelbuild/bazel/issues so we can keep track of the issues here? I'll take a look later this week (I don't have access to a Linux machine at the moment).

[Billy Lam]

Hi Jingwen,

Sorry for the delay. Filed bug here.

A follow up question: do android instrumentation tests support outputting to the bazel test output directories (e.g. TEST_TMPDIR or TEST_UNDECLARED_OUTPUTS_DIR).

I am wondering if there is a recommended way to write files back to the host for further processing/analysis.

Thanks!

Billy